2024-04-25T08:50:36.021 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T08:50:36.021 DEBUG:teuthology.run:Teuthology command: teuthology --name neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} --archive /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006 --owner scheduled_neesingh@teuthology -- /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/orig.config.yaml 2024-04-25T08:50:36.066 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:50:36.160 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673006' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: 5787d42d10024ba410e822a8aba474331cb9af02 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5869 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 5787d42d10024ba410e822a8aba474331cb9af02 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHMnWXFNDmuTYiUV92lJ3TMit5J5Xs+Q/H2oIivYKoPZqJO/wt8rhPFUHuF/0d3myBhfP8KnLij1ATvxswCAaLc= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMx7oyGubUIs9wXizG7gH/lHca55JUIEaB7eeEqpf3/AmudOgYHcnkAhKDWZKruOKEwWLL68EiRoDcftHhzZcDE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_08:15:18 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T08:50:36.160 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa; will attempt to use it 2024-04-25T08:50:36.161 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks 2024-04-25T08:50:36.161 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T08:50:36.163 INFO:teuthology.task.internal:Checking packages... 2024-04-25T08:50:36.188 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T08:50:36.189 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T08:50:36.189 INFO:teuthology.packaging:ref: None 2024-04-25T08:50:36.189 INFO:teuthology.packaging:tag: None 2024-04-25T08:50:36.189 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T08:50:36.189 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:50:36.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T08:50:36.338 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T08:50:36.340 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T08:50:36.351 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T08:50:36.351 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T08:50:36.394 INFO:teuthology.task.internal:Saving configuration 2024-04-25T08:50:36.412 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T08:50:36.423 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:50:36.449 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:44:03.148787', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHMnWXFNDmuTYiUV92lJ3TMit5J5Xs+Q/H2oIivYKoPZqJO/wt8rhPFUHuF/0d3myBhfP8KnLij1ATvxswCAaLc='} 2024-04-25T08:50:36.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:44:03.149985', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMx7oyGubUIs9wXizG7gH/lHca55JUIEaB7eeEqpf3/AmudOgYHcnkAhKDWZKruOKEwWLL68EiRoDcftHhzZcDE='} 2024-04-25T08:50:36.472 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T08:50:36.483 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T08:50:36.483 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T08:50:36.483 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T08:50:36.562 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcfb04d1160>, signals=[15]) 2024-04-25T08:50:36.562 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T08:50:36.573 INFO:teuthology.task.internal:Opening connections... 2024-04-25T08:50:36.573 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T08:50:36.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:50:36.695 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T08:50:36.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:50:36.771 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T08:50:36.782 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-04-25T08:50:36.801 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-04-25T08:50:36.802 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-04-25T08:50:36.859 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-04-25T08:50:36.859 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-04-25T08:50:36.859 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-04-25T08:50:36.859 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T08:50:36.860 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T08:50:36.861 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-04-25T08:50:36.882 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-04-25T08:50:36.900 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-04-25T08:50:36.900 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-04-25T08:50:36.957 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-04-25T08:50:36.958 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T08:50:36.959 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T08:50:36.959 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T08:50:36.959 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-04-25T08:50:36.974 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T08:50:36.991 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T08:50:37.001 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T08:50:37.002 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:50:37.004 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:50:37.021 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T08:50:37.033 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T08:50:37.033 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:50:37.063 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:50:37.093 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T08:50:37.149 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T08:50:37.177 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T08:50:37.178 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T08:50:37.178 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T08:50:37.178 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:50:37.179 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:50:37.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:50:37.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T08:50:37.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:50:37.199 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T08:50:37.368 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:50:37.368 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T08:50:37.368 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-04-25T08:50:37.381 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:50:37.381 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T08:50:37.381 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-04-25T08:50:37.386 INFO:teuthology.orchestra.run.smithi121.stdout:5.14.0-437.el9.x86_64 2024-04-25T08:50:37.386 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:50:37.386 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T08:50:37.387 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum upgrade -y linux-firmware 2024-04-25T08:50:37.399 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-437.el9.x86_64 2024-04-25T08:50:37.400 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:50:37.400 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T08:50:37.400 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum upgrade -y linux-firmware 2024-04-25T08:50:39.861 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-04-25T08:50:40.295 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.0 MB 00:01 2024-04-25T08:50:43.137 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T08:50:43.345 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-04-25T08:50:49.346 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-04-25T08:50:49.412 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-04-25T08:50:50.938 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-25T08:50:51.372 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 16 kB 00:00 2024-04-25T08:50:53.913 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 21 MB 00:02 2024-04-25T08:50:54.256 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-04-25T08:50:58.980 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-25T08:50:59.398 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-04-25T08:51:00.938 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-25T08:51:00.939 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-25T08:51:00.940 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-25T08:51:01.039 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T08:51:01.039 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:51:01.205 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T08:51:01.206 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T08:51:01.208 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi121.front.sepia.ceph.com to complete... 2024-04-25T08:51:01.457 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-25T08:51:01.458 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-04-25T08:51:01.458 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-25T08:51:01.571 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T08:51:01.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:51:01.721 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T08:51:01.721 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T08:51:01.724 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi062.front.sepia.ceph.com to complete... 2024-04-25T08:51:22.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25 08:51:22 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T08:51:22.024 INFO:teuthology.orchestra.run.smithi121.stderr:FINISHED --2024-04-25 08:51:22-- 2024-04-25T08:51:22.024 INFO:teuthology.orchestra.run.smithi121.stderr:Total wall clock time: 21s 2024-04-25T08:51:22.024 INFO:teuthology.orchestra.run.smithi121.stderr:Downloaded: 1 files, 341M in 20s (16.7 MB/s) 2024-04-25T08:51:22.027 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T08:51:22.028 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T08:51:24.121 INFO:teuthology.orchestra.run.smithi121.stdout:Verifying... ######################################## 2024-04-25T08:51:24.270 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing... ######################################## 2024-04-25T08:51:24.382 INFO:teuthology.orchestra.run.smithi121.stdout:Updating / installing... 2024-04-25T08:51:32.330 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T08:51:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25 08:51:45 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T08:51:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:FINISHED --2024-04-25 08:51:45-- 2024-04-25T08:51:45.451 INFO:teuthology.orchestra.run.smithi062.stderr:Total wall clock time: 44s 2024-04-25T08:51:45.452 INFO:teuthology.orchestra.run.smithi062.stderr:Downloaded: 1 files, 341M in 43s (7.87 MB/s) 2024-04-25T08:51:45.457 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T08:51:45.457 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T08:51:47.637 INFO:teuthology.orchestra.run.smithi062.stdout:Verifying... ######################################## 2024-04-25T08:51:47.769 INFO:teuthology.orchestra.run.smithi062.stdout:Preparing... ######################################## 2024-04-25T08:51:47.887 INFO:teuthology.orchestra.run.smithi062.stdout:Updating / installing... 2024-04-25T08:51:56.167 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T08:52:11.464 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi121.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T08:52:11.465 DEBUG:teuthology.orchestra.run.smithi121:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T08:52:11.536 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:11.536 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:11.536 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:11.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T08:52:11.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T08:52:11.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T08:52:11.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T08:52:11.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T08:52:11.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T08:52:11.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T08:52:11.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T08:52:11.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T08:52:11.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T08:52:11.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T08:52:11.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T08:52:11.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T08:52:11.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T08:52:11.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T08:52:11.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T08:52:11.548 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T08:52:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T08:52:11.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T08:52:11.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T08:52:11.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T08:52:11.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T08:52:11.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T08:52:11.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T08:52:11.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T08:52:11.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T08:52:11.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T08:52:11.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T08:52:11.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T08:52:11.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T08:52:11.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T08:52:11.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T08:52:11.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T08:52:11.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T08:52:11.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T08:52:11.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T08:52:11.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T08:52:11.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T08:52:11.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T08:52:11.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T08:52:11.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T08:52:11.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T08:52:11.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T08:52:11.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T08:52:11.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T08:52:11.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T08:52:11.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T08:52:11.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T08:52:11.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T08:52:11.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T08:52:11.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T08:52:11.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T08:52:11.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T08:52:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T08:52:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T08:52:11.597 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T08:52:11.598 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T08:52:11.599 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T08:52:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T08:52:11.601 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T08:52:11.602 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T08:52:11.603 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T08:52:11.604 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T08:52:11.605 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T08:52:11.606 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T08:52:11.607 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T08:52:11.608 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T08:52:11.609 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T08:52:11.610 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T08:52:11.611 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T08:52:11.612 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T08:52:11.613 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T08:52:11.614 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T08:52:11.615 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T08:52:11.616 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T08:52:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T08:52:11.618 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T08:52:11.619 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T08:52:11.620 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T08:52:11.621 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T08:52:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T08:52:11.623 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T08:52:11.624 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T08:52:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T08:52:11.626 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T08:52:11.627 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T08:52:11.628 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T08:52:11.629 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T08:52:11.630 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T08:52:11.631 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T08:52:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T08:52:11.633 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T08:52:11.634 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T08:52:11.635 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T08:52:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T08:52:11.637 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T08:52:11.638 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T08:52:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T08:52:11.640 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T08:52:11.641 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T08:52:11.642 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T08:52:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T08:52:11.644 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T08:52:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T08:52:11.646 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T08:52:11.647 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T08:52:11.648 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T08:52:11.649 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T08:52:11.650 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T08:52:11.651 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T08:52:11.652 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T08:52:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T08:52:11.654 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T08:52:11.655 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T08:52:11.656 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T08:52:11.657 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T08:52:11.658 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T08:52:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T08:52:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T08:52:11.661 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T08:52:11.662 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T08:52:11.663 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T08:52:11.664 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T08:52:11.665 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T08:52:11.666 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T08:52:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T08:52:11.668 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T08:52:11.669 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T08:52:11.670 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T08:52:11.671 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T08:52:11.672 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T08:52:11.673 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T08:52:11.674 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T08:52:11.675 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T08:52:11.676 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T08:52:11.677 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T08:52:11.678 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T08:52:11.678 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T08:52:11.678 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T08:52:11.678 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T08:52:11.678 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T08:52:11.682 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:11.682 DEBUG:teuthology.orchestra.run.smithi121:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T08:52:11.704 INFO:teuthology.orchestra.run.smithi121.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T08:52:11.704 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout:for file in vmlinuz System.map config; do 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout: fi 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout:done 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-04-25T08:52:11.705 INFO:teuthology.orchestra.run.smithi121.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T08:52:11.706 INFO:teuthology.orchestra.run.smithi121.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:11.706 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-04-25T08:52:11.706 INFO:teuthology.orchestra.run.smithi121.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T08:52:11.706 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T08:52:11.706 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:11.706 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-04-25T08:52:11.707 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -qi grub2-tools 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Name : grub2-tools 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Epoch : 1 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Version : 2.06 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Release : 68.el9 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Architecture: x86_64 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Group : Unspecified 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Size : 8264254 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:License : GPLv3+ 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T08:52:11.789 INFO:teuthology.orchestra.run.smithi121.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:Packager : builder@centos.org 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:Vendor : CentOS 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:Summary : Support tools for GRUB. 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:Description : 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:hardware devices. 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T08:52:11.790 INFO:teuthology.orchestra.run.smithi121.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T08:52:11.791 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T08:52:11.791 INFO:teuthology.task.kernel:Updating grub on smithi121 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:11.791 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T08:52:12.362 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2024-04-25T08:52:13.714 INFO:teuthology.orchestra.run.smithi121.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T08:52:13.750 INFO:teuthology.orchestra.run.smithi121.stderr:done 2024-04-25T08:52:13.753 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-04-25T08:52:13.794 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.lPce35yTP2 2024-04-25T08:52:13.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lPce35yTP2 2024-04-25T08:52:13.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.lPce35yTP2 2024-04-25T08:52:14.004 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.lPce35yTP2 is 6KB 2024-04-25T08:52:14.055 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.lPce35yTP2 2024-04-25T08:52:14.070 DEBUG:teuthology.orchestra.run.smithi121:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T08:52:14.131 INFO:teuthology.orchestra.run.smithi121.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T08:52:14.131 INFO:teuthology.orchestra.run.smithi121.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T08:52:14.131 INFO:teuthology.orchestra.run.smithi121.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T08:52:14.131 INFO:teuthology.orchestra.run.smithi121.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T08:52:14.132 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:14.281 DEBUG:teuthology.orchestra.run.smithi121:> sudo shutdown -r now 2024-04-25T08:52:34.212 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T08:52:34.213 DEBUG:teuthology.orchestra.run.smithi062:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T08:52:34.335 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T08:52:34.336 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T08:52:34.337 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T08:52:34.338 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T08:52:34.339 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T08:52:34.340 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T08:52:34.341 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T08:52:34.342 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T08:52:34.343 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T08:52:34.344 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T08:52:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T08:52:34.346 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T08:52:34.347 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T08:52:34.348 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T08:52:34.349 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T08:52:34.350 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T08:52:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T08:52:34.352 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T08:52:34.353 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T08:52:34.354 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T08:52:34.355 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T08:52:34.356 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T08:52:34.357 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T08:52:34.358 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T08:52:34.359 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T08:52:34.360 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T08:52:34.361 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T08:52:34.362 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T08:52:34.363 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T08:52:34.364 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T08:52:34.365 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T08:52:34.366 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T08:52:34.367 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T08:52:34.368 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T08:52:34.369 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T08:52:34.370 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T08:52:34.371 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T08:52:34.372 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T08:52:34.373 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T08:52:34.374 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T08:52:34.375 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T08:52:34.376 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T08:52:34.377 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T08:52:34.378 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T08:52:34.379 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T08:52:34.380 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T08:52:34.381 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T08:52:34.382 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T08:52:34.383 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T08:52:34.384 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T08:52:34.385 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T08:52:34.386 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T08:52:34.387 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T08:52:34.388 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T08:52:34.389 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T08:52:34.390 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T08:52:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T08:52:34.392 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T08:52:34.393 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T08:52:34.394 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T08:52:34.395 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T08:52:34.396 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T08:52:34.397 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T08:52:34.398 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T08:52:34.399 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T08:52:34.400 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T08:52:34.401 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T08:52:34.402 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T08:52:34.403 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T08:52:34.404 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T08:52:34.405 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T08:52:34.406 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T08:52:34.407 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T08:52:34.408 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T08:52:34.409 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T08:52:34.417 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T08:52:34.421 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T08:52:34.422 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T08:52:34.424 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T08:52:34.425 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T08:52:34.426 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T08:52:34.427 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T08:52:34.428 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T08:52:34.429 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T08:52:34.430 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T08:52:34.431 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T08:52:34.432 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T08:52:34.433 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T08:52:34.434 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T08:52:34.435 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T08:52:34.436 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T08:52:34.437 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T08:52:34.438 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T08:52:34.439 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T08:52:34.440 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T08:52:34.441 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T08:52:34.442 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T08:52:34.443 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T08:52:34.444 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T08:52:34.445 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T08:52:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T08:52:34.447 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T08:52:34.448 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T08:52:34.449 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T08:52:34.450 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T08:52:34.451 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T08:52:34.452 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T08:52:34.453 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T08:52:34.454 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T08:52:34.455 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T08:52:34.456 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T08:52:34.457 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T08:52:34.458 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T08:52:34.459 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T08:52:34.460 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T08:52:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T08:52:34.462 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T08:52:34.463 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T08:52:34.464 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T08:52:34.465 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T08:52:34.466 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T08:52:34.467 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T08:52:34.468 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T08:52:34.469 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T08:52:34.470 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T08:52:34.471 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T08:52:34.472 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T08:52:34.472 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T08:52:34.472 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T08:52:34.472 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T08:52:34.472 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T08:52:34.472 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T08:52:34.472 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T08:52:34.472 INFO:teuthology.orchestra.run.smithi062.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T08:52:34.476 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:34.476 DEBUG:teuthology.orchestra.run.smithi062:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout:for file in vmlinuz System.map config; do 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout: fi 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout:done 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T08:52:34.499 INFO:teuthology.orchestra.run.smithi062.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:34.500 INFO:teuthology.orchestra.run.smithi062.stdout:fi 2024-04-25T08:52:34.501 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2024-04-25T08:52:34.574 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2024-04-25T08:52:34.574 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2024-04-25T08:52:34.574 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.06 2024-04-25T08:52:34.574 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 68.el9 2024-04-25T08:52:34.574 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2024-04-25T08:52:34.574 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T08:52:34.574 INFO:teuthology.orchestra.run.smithi062.stdout:Group : Unspecified 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 8264254 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : builder@centos.org 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T08:52:34.575 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T08:52:34.576 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2024-04-25T08:52:34.576 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T08:52:34.576 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T08:52:34.576 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T08:52:34.577 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:34.577 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T08:52:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2024-04-25T08:52:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T08:52:36.479 INFO:teuthology.orchestra.run.smithi062.stderr:done 2024-04-25T08:52:36.482 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-04-25T08:52:36.499 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.cnj49H1DSD 2024-04-25T08:52:36.499 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cnj49H1DSD 2024-04-25T08:52:36.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.cnj49H1DSD 2024-04-25T08:52:36.713 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.cnj49H1DSD is 6KB 2024-04-25T08:52:36.761 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.cnj49H1DSD 2024-04-25T08:52:36.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T08:52:36.838 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T08:52:36.838 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T08:52:36.838 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T08:52:36.839 INFO:teuthology.orchestra.run.smithi062.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T08:52:36.839 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T08:52:36.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2024-04-25T08:52:44.291 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T08:52:44.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-25T08:52:44.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:53:02.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-04-25T08:53:06.995 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T08:53:06.996 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T08:53:06.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:53:11.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-25T08:53:11.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:53:14.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-04-25T08:53:25.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-04-25T08:53:26.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-25T08:53:26.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:53:34.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T08:53:34.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:53:37.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-04-25T08:53:49.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T08:53:49.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:54:07.907 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2024-04-25T08:54:22.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T08:54:22.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:54:23.338 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-25T08:54:24.050 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T08:54:24.051 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T08:54:24.051 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-04-25T08:54:24.096 INFO:teuthology.orchestra.run.smithi062.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T08:54:24.096 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T08:54:24.096 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T08:54:24.096 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T08:54:25.098 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:26.692 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T08:54:41.693 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-25T08:54:41.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:54:42.053 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-04-25T08:54:42.700 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-25T08:54:42.700 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T08:54:42.700 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-04-25T08:54:42.752 INFO:teuthology.orchestra.run.smithi121.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T08:54:42.752 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T08:54:42.753 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T08:54:42.753 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T08:54:43.754 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:43.754 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T08:54:43.766 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T08:54:43.766 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:54:43.776 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:54:43.831 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T08:54:43.842 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T08:54:43.861 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T08:54:43.862 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:54:43.868 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:54:44.037 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T08:54:44.046 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T08:54:44.046 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:54:44.053 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:54:44.187 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:54:44.244 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:54:44.421 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:54:44.534 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:54:44.539 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T08:54:44.547 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T08:54:44.548 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:54:44.553 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:54:44.642 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T08:54:44.652 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T08:54:44.652 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:54:44.660 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:54:44.716 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:54:44.802 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:54:44.904 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:54:44.990 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:54:45.083 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T08:54:45.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:54:45.169 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:54:45.258 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:54:45.347 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:54:45.431 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:54:45.511 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T08:54:45.511 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:54:45.590 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-04-25T08:54:45.598 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-04-25T08:54:45.713 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:54:45.750 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:54:46.132 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T08:54:46.140 INFO:teuthology.task.internal:Starting timer... 2024-04-25T08:54:46.141 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T08:54:46.158 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T08:54:46.184 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-04-25T08:54:46.300 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-04-25T08:54:46.304 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-04-25T08:54:46.423 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-04-25T08:54:46.428 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T08:54:46.428 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-04-25T08:54:46.482 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-04-25T08:54:46.482 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-04-25T08:54:46.541 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-04-25T08:54:46.541 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2024-04-25T08:54:46.542 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:54:46.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:46.617 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:54:46.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:46.695 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T08:54:46.695 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-04-25T08:54:46.759 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-04-25T08:54:46.847 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T08:54:46.858 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T08:54:46.876 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T08:54:46.878 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T09:00:01.768 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2024-04-25T09:00:01.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T09:00:01.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:00:01.893 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-25T09:00:02.061 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T09:00:02.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-25T09:00:02.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:00:02.174 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-04-25T09:00:02.329 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-25T09:00:02.330 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T09:00:02.341 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T09:00:02.342 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:00:02.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:00:02.353 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:00:02.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:00:02.440 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:00:02.446 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:00:02.481 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:00:02.491 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:00:02.553 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-04-25T09:00:02.567 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-04-25T09:00:02.583 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-04-25T09:00:02.601 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-04-25T09:00:02.623 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:00:02.645 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:00:02.665 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:00:02.691 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:00:02.810 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:00:02.819 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:00:02.820 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-04-25T09:00:02.820 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:00:02.820 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:00:02.820 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:00:02.820 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:00:02.835 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-25T09:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:00:02.845 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:00:02.847 INFO:teuthology.run_tasks:Running task install... 2024-04-25T09:00:02.856 DEBUG:teuthology.task.install:project ceph 2024-04-25T09:00:02.856 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T09:00:02.856 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T09:00:02.856 INFO:teuthology.task.install:Using flavor: default 2024-04-25T09:00:02.862 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T09:00:02.862 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T09:00:02.863 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:00:02.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:00:02.865 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:00:02.866 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:00:03.043 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:00:03.043 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:00:03.049 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:00:03.050 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:00:03.185 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:00:03.185 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:00:03.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:00:03.196 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:00:03.196 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:00:03.196 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:00:03.281 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:00:03.281 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:00:03.285 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:00:03.285 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:00:03.411 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:00:03.412 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:00:03.520 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:00:03.526 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:00:03.597 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-04-25T09:00:03.601 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-04-25T09:00:03.612 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-04-25T09:00:03.616 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-04-25T09:00:03.917 INFO:teuthology.orchestra.run.smithi121.stdout:48 files removed 2024-04-25T09:00:03.946 INFO:teuthology.orchestra.run.smithi062.stdout:48 files removed 2024-04-25T09:00:03.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:00:03.986 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:00:04.848 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 177 kB/s | 82 kB 00:00 2024-04-25T09:00:04.885 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 183 kB/s | 82 kB 00:00 2024-04-25T09:00:05.352 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-04-25T09:00:05.356 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-04-25T09:00:05.857 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-25T09:00:05.860 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-25T09:00:08.242 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-04-25T09:00:09.763 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.0 MB 00:03 2024-04-25T09:00:12.130 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T09:00:12.909 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 19 MB 00:03 2024-04-25T09:00:17.073 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-25T09:00:18.252 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-04-25T09:00:18.990 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-04-25T09:00:20.472 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-04-25T09:00:20.651 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-25T09:00:22.377 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T09:00:26.616 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T09:00:28.379 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T09:00:29.075 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:00:29.077 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:00:29.097 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi121.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:00:29.099 INFO:teuthology.orchestra.run.smithi121.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:00:29.101 INFO:teuthology.orchestra.run.smithi121.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:00:29.103 INFO:teuthology.orchestra.run.smithi121.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:00:29.107 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:00:29.108 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:00:29.178 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:00:29.207 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:00:29.208 INFO:teuthology.orchestra.run.smithi121.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:00:29.209 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:00:29.210 INFO:teuthology.orchestra.run.smithi121.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:00:29.211 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:00:29.212 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:00:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:00:29.214 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:00:29.215 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:00:29.216 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:00:29.217 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout:Install 148 Packages 2024-04-25T09:00:29.218 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 4 Packages 2024-04-25T09:00:29.219 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:00:29.219 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 215 M 2024-04-25T09:00:29.219 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi121.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-25T09:00:30.842 INFO:teuthology.orchestra.run.smithi121.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 3.4 MB/s | 887 kB 00:00 2024-04-25T09:00:30.899 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:00:30.901 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:00:30.923 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:00:30.925 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:00:30.926 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:00:30.928 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:00:30.930 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:00:30.934 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:00:30.935 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:00:30.942 INFO:teuthology.orchestra.run.smithi121.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.4 MB/s | 145 kB 00:00 2024-04-25T09:00:31.014 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-25T09:00:31.030 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T09:00:31.030 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-04-25T09:00:31.030 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T09:00:31.030 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-04-25T09:00:31.030 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:00:31.030 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:00:31.030 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:00:31.031 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:00:31.032 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:00:31.033 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:00:31.034 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:00:31.035 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:00:31.036 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:00:31.037 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:00:31.038 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:00:31.039 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:00:31.040 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout:Install 148 Packages 2024-04-25T09:00:31.041 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 4 Packages 2024-04-25T09:00:31.042 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:00:31.042 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 215 M 2024-04-25T09:00:31.042 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-25T09:00:31.201 INFO:teuthology.orchestra.run.smithi121.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 6.0 MB/s | 5.3 MB 00:00 2024-04-25T09:00:31.374 INFO:teuthology.orchestra.run.smithi121.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 4.9 MB/s | 2.1 MB 00:00 2024-04-25T09:00:31.477 INFO:teuthology.orchestra.run.smithi121.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-04-25T09:00:32.010 INFO:teuthology.orchestra.run.smithi121.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 7.4 MB/s | 4.7 MB 00:00 2024-04-25T09:00:32.222 INFO:teuthology.orchestra.run.smithi062.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-25T09:00:32.869 INFO:teuthology.orchestra.run.smithi121.stdout:(8/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x86 12 MB/s | 17 MB 00:01 2024-04-25T09:00:32.919 INFO:teuthology.orchestra.run.smithi121.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 515 kB/s | 25 kB 00:00 2024-04-25T09:00:32.989 INFO:teuthology.orchestra.run.smithi062.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.1 MB/s | 887 kB 00:00 2024-04-25T09:00:33.156 INFO:teuthology.orchestra.run.smithi062.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 879 kB/s | 145 kB 00:00 2024-04-25T09:00:33.161 INFO:teuthology.orchestra.run.smithi121.stdout:(10/152): ceph-common-19.0.0-1628.g97ae6cc9.el9 7.0 MB/s | 20 MB 00:02 2024-04-25T09:00:33.295 INFO:teuthology.orchestra.run.smithi121.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 1.6 MB/s | 220 kB 00:00 2024-04-25T09:00:33.403 INFO:teuthology.orchestra.run.smithi121.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 252 kB/s | 27 kB 00:00 2024-04-25T09:00:33.503 INFO:teuthology.orchestra.run.smithi121.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 314 kB/s | 31 kB 00:00 2024-04-25T09:00:33.712 INFO:teuthology.orchestra.run.smithi121.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 3.5 MB/s | 744 kB 00:00 2024-04-25T09:00:33.762 INFO:teuthology.orchestra.run.smithi121.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 3.2 MB/s | 163 kB 00:00 2024-04-25T09:00:33.846 INFO:teuthology.orchestra.run.smithi121.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.5 MB/s | 127 kB 00:00 2024-04-25T09:00:34.013 INFO:teuthology.orchestra.run.smithi121.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 3.0 MB/s | 501 kB 00:00 2024-04-25T09:00:34.163 INFO:teuthology.orchestra.run.smithi121.stdout:(18/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 4.5 MB/s | 9.8 MB 00:02 2024-04-25T09:00:34.230 INFO:teuthology.orchestra.run.smithi121.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 691 kB/s | 45 kB 00:00 2024-04-25T09:00:34.313 INFO:teuthology.orchestra.run.smithi121.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.6 MB/s | 131 kB 00:00 2024-04-25T09:00:34.397 INFO:teuthology.orchestra.run.smithi121.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.9 MB/s | 162 kB 00:00 2024-04-25T09:00:34.523 INFO:teuthology.orchestra.run.smithi121.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 2.5 MB/s | 321 kB 00:00 2024-04-25T09:00:34.665 INFO:teuthology.orchestra.run.smithi121.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 2.1 MB/s | 298 kB 00:00 2024-04-25T09:00:34.740 INFO:teuthology.orchestra.run.smithi121.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 99 kB 00:00 2024-04-25T09:00:34.807 INFO:teuthology.orchestra.run.smithi121.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-25T09:00:34.973 INFO:teuthology.orchestra.run.smithi062.stdout:(4/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 1.2 MB/s | 2.1 MB 00:01 2024-04-25T09:00:35.074 INFO:teuthology.orchestra.run.smithi121.stdout:(26/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 4.8 MB/s | 5.1 MB 00:01 2024-04-25T09:00:35.182 INFO:teuthology.orchestra.run.smithi121.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 1.6 MB/s | 170 kB 00:00 2024-04-25T09:00:35.282 INFO:teuthology.orchestra.run.smithi121.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 317 kB/s | 31 kB 00:00 2024-04-25T09:00:35.391 INFO:teuthology.orchestra.run.smithi121.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.3 MB/s | 138 kB 00:00 2024-04-25T09:00:35.982 INFO:teuthology.orchestra.run.smithi062.stdout:(5/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 1.5 MB/s | 1.5 MB 00:01 2024-04-25T09:00:36.183 INFO:teuthology.orchestra.run.smithi121.stdout:(30/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 15 MB/s | 47 MB 00:03 2024-04-25T09:00:36.266 INFO:teuthology.orchestra.run.smithi121.stdout:(31/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 2.1 MB/s | 3.1 MB 00:01 2024-04-25T09:00:36.282 INFO:teuthology.orchestra.run.smithi062.stdout:(6/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 1.2 MB/s | 5.3 MB 00:04 2024-04-25T09:00:36.325 INFO:teuthology.orchestra.run.smithi121.stdout:(32/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 1.9 MB/s | 1.8 MB 00:00 2024-04-25T09:00:36.434 INFO:teuthology.orchestra.run.smithi121.stdout:(33/152): ceph-mgr-modules-core-19.0.0-1628.g97 1.5 MB/s | 248 kB 00:00 2024-04-25T09:00:36.459 INFO:teuthology.orchestra.run.smithi121.stdout:(34/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 372 kB/s | 49 kB 00:00 2024-04-25T09:00:36.493 INFO:teuthology.orchestra.run.smithi121.stdout:(35/152): ceph-prometheus-alerts-19.0.0-1628.g9 267 kB/s | 15 kB 00:00 2024-04-25T09:00:36.610 INFO:teuthology.orchestra.run.smithi121.stdout:(36/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 1.7 MB/s | 264 kB 00:00 2024-04-25T09:00:36.785 INFO:teuthology.orchestra.run.smithi121.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-25T09:00:36.810 INFO:teuthology.orchestra.run.smithi121.stdout:(38/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 91 kB/s | 18 kB 00:00 2024-04-25T09:00:36.852 INFO:teuthology.orchestra.run.smithi121.stdout:(39/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 2.0 MB/s | 748 kB 00:00 2024-04-25T09:00:37.128 INFO:teuthology.orchestra.run.smithi121.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 236 kB/s | 80 kB 00:00 2024-04-25T09:00:37.229 INFO:teuthology.orchestra.run.smithi121.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 521 kB/s | 195 kB 00:00 2024-04-25T09:00:37.478 INFO:teuthology.orchestra.run.smithi121.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-04-25T09:00:37.511 INFO:teuthology.orchestra.run.smithi121.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 661 kB/s | 253 kB 00:00 2024-04-25T09:00:37.515 INFO:teuthology.orchestra.run.smithi062.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 3.1 MB/s | 4.7 MB 00:01 2024-04-25T09:00:37.753 INFO:teuthology.orchestra.run.smithi121.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 388 kB/s | 106 kB 00:00 2024-04-25T09:00:37.770 INFO:teuthology.orchestra.run.smithi121.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 524 kB/s | 135 kB 00:00 2024-04-25T09:00:38.029 INFO:teuthology.orchestra.run.smithi121.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 1.5 MB/s | 1.2 MB 00:00 2024-04-25T09:00:38.054 INFO:teuthology.orchestra.run.smithi121.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 417 kB/s | 125 kB 00:00 2024-04-25T09:00:38.146 INFO:teuthology.orchestra.run.smithi121.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 572 kB/s | 215 kB 00:00 2024-04-25T09:00:38.288 INFO:teuthology.orchestra.run.smithi121.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-04-25T09:00:38.313 INFO:teuthology.orchestra.run.smithi121.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 643 kB/s | 107 kB 00:00 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi121.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 435 kB/s | 21 kB 00:00 2024-04-25T09:00:38.388 INFO:teuthology.orchestra.run.smithi121.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-04-25T09:00:38.414 INFO:teuthology.orchestra.run.smithi121.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 439 kB/s | 33 kB 00:00 2024-04-25T09:00:38.447 INFO:teuthology.orchestra.run.smithi121.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-25T09:00:38.473 INFO:teuthology.orchestra.run.smithi121.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-04-25T09:00:38.515 INFO:teuthology.orchestra.run.smithi121.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 169 kB/s | 11 kB 00:00 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi121.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-25T09:00:38.574 INFO:teuthology.orchestra.run.smithi121.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 785 kB/s | 45 kB 00:00 2024-04-25T09:00:38.641 INFO:teuthology.orchestra.run.smithi121.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-25T09:00:38.732 INFO:teuthology.orchestra.run.smithi121.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-04-25T09:00:38.758 INFO:teuthology.orchestra.run.smithi121.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 220 kB/s | 25 kB 00:00 2024-04-25T09:00:38.791 INFO:teuthology.orchestra.run.smithi121.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-04-25T09:00:38.816 INFO:teuthology.orchestra.run.smithi121.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 573 kB/s | 47 kB 00:00 2024-04-25T09:00:38.850 INFO:teuthology.orchestra.run.smithi121.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T09:00:38.876 INFO:teuthology.orchestra.run.smithi121.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 607 kB/s | 35 kB 00:00 2024-04-25T09:00:38.901 INFO:teuthology.orchestra.run.smithi121.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-25T09:00:39.060 INFO:teuthology.orchestra.run.smithi121.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-25T09:00:39.168 INFO:teuthology.orchestra.run.smithi121.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.3 MB/s | 911 kB 00:00 2024-04-25T09:00:39.227 INFO:teuthology.orchestra.run.smithi121.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T09:00:39.257 INFO:teuthology.orchestra.run.smithi062.stdout:(8/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el9 5.6 MB/s | 9.8 MB 00:01 2024-04-25T09:00:39.307 INFO:teuthology.orchestra.run.smithi062.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 517 kB/s | 25 kB 00:00 2024-04-25T09:00:39.319 INFO:teuthology.orchestra.run.smithi121.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-25T09:00:39.436 INFO:teuthology.orchestra.run.smithi121.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-25T09:00:39.519 INFO:teuthology.orchestra.run.smithi121.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-25T09:00:39.544 INFO:teuthology.orchestra.run.smithi121.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 809 kB/s | 87 kB 00:00 2024-04-25T09:00:39.645 INFO:teuthology.orchestra.run.smithi121.stdout:(74/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.0 MB/s | 4.7 MB 00:00 2024-04-25T09:00:39.695 INFO:teuthology.orchestra.run.smithi121.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-04-25T09:00:39.729 INFO:teuthology.orchestra.run.smithi121.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-25T09:00:39.771 INFO:teuthology.orchestra.run.smithi121.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 344 kB/s | 26 kB 00:00 2024-04-25T09:00:39.813 INFO:teuthology.orchestra.run.smithi121.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 231 kB/s | 19 kB 00:00 2024-04-25T09:00:39.846 INFO:teuthology.orchestra.run.smithi121.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-25T09:00:39.897 INFO:teuthology.orchestra.run.smithi121.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 420 kB/s | 35 kB 00:00 2024-04-25T09:00:39.955 INFO:teuthology.orchestra.run.smithi121.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-25T09:00:40.005 INFO:teuthology.orchestra.run.smithi121.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T09:00:40.131 INFO:teuthology.orchestra.run.smithi121.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.8 MB/s | 6.0 MB 00:00 2024-04-25T09:00:40.156 INFO:teuthology.orchestra.run.smithi121.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 516 kB/s | 77 kB 00:00 2024-04-25T09:00:40.222 INFO:teuthology.orchestra.run.smithi121.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 465 kB/s | 42 kB 00:00 2024-04-25T09:00:40.256 INFO:teuthology.orchestra.run.smithi121.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-25T09:00:40.290 INFO:teuthology.orchestra.run.smithi121.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-25T09:00:40.323 INFO:teuthology.orchestra.run.smithi121.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T09:00:40.374 INFO:teuthology.orchestra.run.smithi121.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 615 kB/s | 51 kB 00:00 2024-04-25T09:00:40.424 INFO:teuthology.orchestra.run.smithi121.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-04-25T09:00:40.425 INFO:teuthology.orchestra.run.smithi062.stdout:(10/152): ceph-common-19.0.0-1628.g97ae6cc9.el9 2.3 MB/s | 20 MB 00:08 2024-04-25T09:00:40.541 INFO:teuthology.orchestra.run.smithi062.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 1.9 MB/s | 220 kB 00:00 2024-04-25T09:00:40.542 INFO:teuthology.orchestra.run.smithi121.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.3 MB/s | 5.1 MB 00:00 2024-04-25T09:00:40.591 INFO:teuthology.orchestra.run.smithi121.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 250 kB/s | 42 kB 00:00 2024-04-25T09:00:40.608 INFO:teuthology.orchestra.run.smithi062.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 413 kB/s | 27 kB 00:00 2024-04-25T09:00:40.642 INFO:teuthology.orchestra.run.smithi121.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 444 kB/s | 44 kB 00:00 2024-04-25T09:00:40.666 INFO:teuthology.orchestra.run.smithi062.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 546 kB/s | 31 kB 00:00 2024-04-25T09:00:40.701 INFO:teuthology.orchestra.run.smithi121.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-25T09:00:40.759 INFO:teuthology.orchestra.run.smithi121.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-04-25T09:00:40.810 INFO:teuthology.orchestra.run.smithi121.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 442 kB/s | 47 kB 00:00 2024-04-25T09:00:40.843 INFO:teuthology.orchestra.run.smithi121.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-25T09:00:40.893 INFO:teuthology.orchestra.run.smithi121.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 698 kB/s | 58 kB 00:00 2024-04-25T09:00:40.918 INFO:teuthology.orchestra.run.smithi121.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-25T09:00:40.934 INFO:teuthology.orchestra.run.smithi062.stdout:(14/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 3.6 MB/s | 17 MB 00:04 2024-04-25T09:00:40.969 INFO:teuthology.orchestra.run.smithi121.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 856 kB/s | 64 kB 00:00 2024-04-25T09:00:40.992 INFO:teuthology.orchestra.run.smithi062.stdout:(15/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 2.2 MB/s | 744 kB 00:00 2024-04-25T09:00:41.011 INFO:teuthology.orchestra.run.smithi121.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 484 kB/s | 45 kB 00:00 2024-04-25T09:00:41.042 INFO:teuthology.orchestra.run.smithi062.stdout:(16/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.5 MB/s | 163 kB 00:00 2024-04-25T09:00:41.076 INFO:teuthology.orchestra.run.smithi062.stdout:(17/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.5 MB/s | 127 kB 00:00 2024-04-25T09:00:41.170 INFO:teuthology.orchestra.run.smithi121.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 112 kB/s | 22 kB 00:00 2024-04-25T09:00:41.209 INFO:teuthology.orchestra.run.smithi062.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 2.9 MB/s | 501 kB 00:00 2024-04-25T09:00:41.276 INFO:teuthology.orchestra.run.smithi062.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 687 kB/s | 45 kB 00:00 2024-04-25T09:00:41.365 INFO:teuthology.orchestra.run.smithi062.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.5 MB/s | 131 kB 00:00 2024-04-25T09:00:41.427 INFO:teuthology.orchestra.run.smithi062.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.6 MB/s | 162 kB 00:00 2024-04-25T09:00:41.437 INFO:teuthology.orchestra.run.smithi121.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-04-25T09:00:41.527 INFO:teuthology.orchestra.run.smithi062.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.2 MB/s | 321 kB 00:00 2024-04-25T09:00:41.619 INFO:teuthology.orchestra.run.smithi062.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T09:00:41.678 INFO:teuthology.orchestra.run.smithi062.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.7 MB/s | 99 kB 00:00 2024-04-25T09:00:41.736 INFO:teuthology.orchestra.run.smithi062.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.5 MB/s | 85 kB 00:00 2024-04-25T09:00:41.854 INFO:teuthology.orchestra.run.smithi121.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-04-25T09:00:41.921 INFO:teuthology.orchestra.run.smithi121.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 731 kB/s | 548 kB 00:00 2024-04-25T09:00:41.954 INFO:teuthology.orchestra.run.smithi121.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-04-25T09:00:42.113 INFO:teuthology.orchestra.run.smithi121.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 164 kB/s | 26 kB 00:00 2024-04-25T09:00:42.163 INFO:teuthology.orchestra.run.smithi121.stdout:(108/152): grpc-data-1.46.7-10.el9.noarch.rpm 63 kB/s | 19 kB 00:00 2024-04-25T09:00:42.170 INFO:teuthology.orchestra.run.smithi062.stdout:(26/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 4.7 MB/s | 5.1 MB 00:01 2024-04-25T09:00:42.253 INFO:teuthology.orchestra.run.smithi062.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T09:00:42.267 INFO:teuthology.orchestra.run.smithi121.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 316 kB/s | 49 kB 00:00 2024-04-25T09:00:42.304 INFO:teuthology.orchestra.run.smithi062.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 642 kB/s | 31 kB 00:00 2024-04-25T09:00:42.331 INFO:teuthology.orchestra.run.smithi121.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 405 kB/s | 67 kB 00:00 2024-04-25T09:00:42.362 INFO:teuthology.orchestra.run.smithi062.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 2.4 MB/s | 138 kB 00:00 2024-04-25T09:00:42.397 INFO:teuthology.orchestra.run.smithi121.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-25T09:00:42.446 INFO:teuthology.orchestra.run.smithi062.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 4.4 MB/s | 3.1 MB 00:00 2024-04-25T09:00:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-25T09:00:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-25T09:00:42.740 INFO:teuthology.orchestra.run.smithi121.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 327 kB/s | 43 kB 00:00 2024-04-25T09:00:42.771 INFO:teuthology.orchestra.run.smithi062.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 4.4 MB/s | 1.8 MB 00:00 2024-04-25T09:00:42.807 INFO:teuthology.orchestra.run.smithi121.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 193 kB/s | 32 kB 00:00 2024-04-25T09:00:42.857 INFO:teuthology.orchestra.run.smithi121.stdout:(116/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 838 kB 00:00 2024-04-25T09:00:42.880 INFO:teuthology.orchestra.run.smithi062.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 2.3 MB/s | 248 kB 00:00 2024-04-25T09:00:42.899 INFO:teuthology.orchestra.run.smithi121.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 95 kB/s | 15 kB 00:00 2024-04-25T09:00:42.933 INFO:teuthology.orchestra.run.smithi121.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T09:00:42.938 INFO:teuthology.orchestra.run.smithi062.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 861 kB/s | 49 kB 00:00 2024-04-25T09:00:42.983 INFO:teuthology.orchestra.run.smithi121.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T09:00:42.997 INFO:teuthology.orchestra.run.smithi062.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 267 kB/s | 15 kB 00:00 2024-04-25T09:00:43.050 INFO:teuthology.orchestra.run.smithi121.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-25T09:00:43.117 INFO:teuthology.orchestra.run.smithi121.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-04-25T09:00:43.147 INFO:teuthology.orchestra.run.smithi062.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 1.7 MB/s | 264 kB 00:00 2024-04-25T09:00:43.151 INFO:teuthology.orchestra.run.smithi121.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.2 MB/s | 206 kB 00:00 2024-04-25T09:00:43.217 INFO:teuthology.orchestra.run.smithi121.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T09:00:43.268 INFO:teuthology.orchestra.run.smithi121.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-25T09:00:43.306 INFO:teuthology.orchestra.run.smithi062.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 4.6 MB/s | 748 kB 00:00 2024-04-25T09:00:43.365 INFO:teuthology.orchestra.run.smithi062.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 318 kB/s | 18 kB 00:00 2024-04-25T09:00:43.368 INFO:teuthology.orchestra.run.smithi121.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-04-25T09:00:43.410 INFO:teuthology.orchestra.run.smithi121.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 92 kB/s | 18 kB 00:00 2024-04-25T09:00:43.452 INFO:teuthology.orchestra.run.smithi121.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 126 kB/s | 23 kB 00:00 2024-04-25T09:00:43.527 INFO:teuthology.orchestra.run.smithi121.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 170 kB/s | 20 kB 00:00 2024-04-25T09:00:43.569 INFO:teuthology.orchestra.run.smithi121.stdout:(129/152): python3-jaraco-functools-3.5.0-2.el9 123 kB/s | 19 kB 00:00 2024-04-25T09:00:43.577 INFO:teuthology.orchestra.run.smithi062.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 380 kB/s | 80 kB 00:00 2024-04-25T09:00:43.645 INFO:teuthology.orchestra.run.smithi121.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2024-04-25T09:00:43.670 INFO:teuthology.orchestra.run.smithi121.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 788 kB/s | 79 kB 00:00 2024-04-25T09:00:43.712 INFO:teuthology.orchestra.run.smithi121.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi121.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 496 kB/s | 58 kB 00:00 2024-04-25T09:00:43.840 INFO:teuthology.orchestra.run.smithi062.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-04-25T09:00:43.854 INFO:teuthology.orchestra.run.smithi121.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-04-25T09:00:43.887 INFO:teuthology.orchestra.run.smithi121.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-04-25T09:00:43.890 INFO:teuthology.orchestra.run.smithi062.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-25T09:00:43.913 INFO:teuthology.orchestra.run.smithi121.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 600 kB/s | 90 kB 00:00 2024-04-25T09:00:43.971 INFO:teuthology.orchestra.run.smithi121.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 242 kB/s | 28 kB 00:00 2024-04-25T09:00:43.991 INFO:teuthology.orchestra.run.smithi062.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-25T09:00:44.013 INFO:teuthology.orchestra.run.smithi121.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-04-25T09:00:44.038 INFO:teuthology.orchestra.run.smithi121.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-25T09:00:44.097 INFO:teuthology.orchestra.run.smithi121.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 474 kB/s | 59 kB 00:00 2024-04-25T09:00:44.108 INFO:teuthology.orchestra.run.smithi062.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 734 kB/s | 195 kB 00:00 2024-04-25T09:00:44.139 INFO:teuthology.orchestra.run.smithi121.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-04-25T09:00:44.175 INFO:teuthology.orchestra.run.smithi062.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T09:00:44.181 INFO:teuthology.orchestra.run.smithi121.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 359 kB/s | 51 kB 00:00 2024-04-25T09:00:44.215 INFO:teuthology.orchestra.run.smithi121.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-25T09:00:44.233 INFO:teuthology.orchestra.run.smithi062.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-04-25T09:00:44.248 INFO:teuthology.orchestra.run.smithi121.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 829 kB/s | 90 kB 00:00 2024-04-25T09:00:44.267 INFO:teuthology.orchestra.run.smithi062.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 135 kB 00:00 2024-04-25T09:00:44.300 INFO:teuthology.orchestra.run.smithi062.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-04-25T09:00:44.328 INFO:teuthology.orchestra.run.smithi121.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-25T09:00:44.382 INFO:teuthology.orchestra.run.smithi121.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 120 kB/s | 20 kB 00:00 2024-04-25T09:00:44.384 INFO:teuthology.orchestra.run.smithi062.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-25T09:00:44.416 INFO:teuthology.orchestra.run.smithi121.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-25T09:00:44.555 INFO:teuthology.orchestra.run.smithi062.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-04-25T09:00:44.574 INFO:teuthology.orchestra.run.smithi121.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-04-25T09:00:44.635 INFO:teuthology.orchestra.run.smithi062.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-25T09:00:44.701 INFO:teuthology.orchestra.run.smithi062.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-04-25T09:00:44.727 INFO:teuthology.orchestra.run.smithi062.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 236 kB/s | 21 kB 00:00 2024-04-25T09:00:44.787 INFO:teuthology.orchestra.run.smithi062.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 568 kB/s | 33 kB 00:00 2024-04-25T09:00:44.827 INFO:teuthology.orchestra.run.smithi062.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-04-25T09:00:44.875 INFO:teuthology.orchestra.run.smithi121.stdout:(149/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 6.8 MB/s | 3.1 MB 00:00 2024-04-25T09:00:44.910 INFO:teuthology.orchestra.run.smithi062.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 224 kB/s | 18 kB 00:00 2024-04-25T09:00:44.970 INFO:teuthology.orchestra.run.smithi062.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-25T09:00:45.036 INFO:teuthology.orchestra.run.smithi062.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-25T09:00:45.050 INFO:teuthology.orchestra.run.smithi121.stdout:(150/152): librados2-19.0.0-1628.g97ae6cc9.el9. 5.1 MB/s | 3.4 MB 00:00 2024-04-25T09:00:45.095 INFO:teuthology.orchestra.run.smithi062.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 788 kB/s | 45 kB 00:00 2024-04-25T09:00:45.354 INFO:teuthology.orchestra.run.smithi062.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.3 MB/s | 3.0 MB 00:00 2024-04-25T09:00:45.401 INFO:teuthology.orchestra.run.smithi121.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 1.3 MB 00:00 2024-04-25T09:00:45.412 INFO:teuthology.orchestra.run.smithi062.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-04-25T09:00:45.942 INFO:teuthology.orchestra.run.smithi121.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-04-25T09:00:45.946 INFO:teuthology.orchestra.run.smithi062.stdout:(60/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 7.1 MB/s | 47 MB 00:06 2024-04-25T09:00:45.949 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:00:45.950 INFO:teuthology.orchestra.run.smithi121.stdout:Total 13 MB/s | 215 MB 00:16 2024-04-25T09:00:45.971 INFO:teuthology.orchestra.run.smithi062.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 46 kB/s | 25 kB 00:00 2024-04-25T09:00:46.013 INFO:teuthology.orchestra.run.smithi062.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 884 kB/s | 582 kB 00:00 2024-04-25T09:00:46.113 INFO:teuthology.orchestra.run.smithi062.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-25T09:00:46.139 INFO:teuthology.orchestra.run.smithi062.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-25T09:00:46.164 INFO:teuthology.orchestra.run.smithi062.stdout:(65/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 219 kB/s | 47 kB 00:00 2024-04-25T09:00:46.189 INFO:teuthology.orchestra.run.smithi062.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 472 kB/s | 35 kB 00:00 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi062.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 5.5 MB/s | 1.6 MB 00:00 2024-04-25T09:00:46.665 INFO:teuthology.orchestra.run.smithi062.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 9.0 MB/s | 4.7 MB 00:00 2024-04-25T09:00:46.715 INFO:teuthology.orchestra.run.smithi062.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 3.8 MB/s | 911 kB 00:00 2024-04-25T09:00:46.749 INFO:teuthology.orchestra.run.smithi062.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-25T09:00:46.874 INFO:teuthology.orchestra.run.smithi062.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.2 MB/s | 1.0 MB 00:00 2024-04-25T09:00:46.966 INFO:teuthology.orchestra.run.smithi062.stdout:(72/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-04-25T09:00:46.991 INFO:teuthology.orchestra.run.smithi062.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 753 kB/s | 87 kB 00:00 2024-04-25T09:00:47.058 INFO:teuthology.orchestra.run.smithi062.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T09:00:47.133 INFO:teuthology.orchestra.run.smithi062.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.3 MB/s | 248 kB 00:00 2024-04-25T09:00:47.200 INFO:teuthology.orchestra.run.smithi062.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 390 kB/s | 26 kB 00:00 2024-04-25T09:00:47.275 INFO:teuthology.orchestra.run.smithi062.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 258 kB/s | 19 kB 00:00 2024-04-25T09:00:47.409 INFO:teuthology.orchestra.run.smithi062.stdout:(78/152): postgresql-server-13.14-1.el9.x86_64. 8.3 MB/s | 5.8 MB 00:00 2024-04-25T09:00:47.442 INFO:teuthology.orchestra.run.smithi062.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-04-25T09:00:47.468 INFO:teuthology.orchestra.run.smithi062.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 604 kB/s | 35 kB 00:00 2024-04-25T09:00:47.535 INFO:teuthology.orchestra.run.smithi062.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T09:00:47.652 INFO:teuthology.orchestra.run.smithi062.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.7 MB/s | 6.0 MB 00:00 2024-04-25T09:00:47.704 INFO:teuthology.orchestra.run.smithi062.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 222 kB 00:00 2024-04-25T09:00:47.736 INFO:teuthology.orchestra.run.smithi062.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 935 kB/s | 77 kB 00:00 2024-04-25T09:00:47.761 INFO:teuthology.orchestra.run.smithi062.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 757 kB/s | 42 kB 00:00 2024-04-25T09:00:47.811 INFO:teuthology.orchestra.run.smithi062.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T09:00:47.845 INFO:teuthology.orchestra.run.smithi062.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-25T09:00:47.878 INFO:teuthology.orchestra.run.smithi062.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T09:00:47.904 INFO:teuthology.orchestra.run.smithi062.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 884 kB/s | 51 kB 00:00 2024-04-25T09:00:47.937 INFO:teuthology.orchestra.run.smithi062.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 931 kB/s | 54 kB 00:00 2024-04-25T09:00:48.063 INFO:teuthology.orchestra.run.smithi062.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 5.1 MB 00:00 2024-04-25T09:00:48.088 INFO:teuthology.orchestra.run.smithi062.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 279 kB/s | 42 kB 00:00 2024-04-25T09:00:48.121 INFO:teuthology.orchestra.run.smithi062.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 767 kB/s | 44 kB 00:00 2024-04-25T09:00:48.172 INFO:teuthology.orchestra.run.smithi062.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-25T09:00:48.197 INFO:teuthology.orchestra.run.smithi062.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 483 kB/s | 36 kB 00:00 2024-04-25T09:00:48.230 INFO:teuthology.orchestra.run.smithi062.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 823 kB/s | 47 kB 00:00 2024-04-25T09:00:48.272 INFO:teuthology.orchestra.run.smithi062.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T09:00:48.298 INFO:teuthology.orchestra.run.smithi062.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 874 kB/s | 58 kB 00:00 2024-04-25T09:00:48.331 INFO:teuthology.orchestra.run.smithi062.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T09:00:48.356 INFO:teuthology.orchestra.run.smithi062.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T09:00:48.399 INFO:teuthology.orchestra.run.smithi062.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-04-25T09:00:48.516 INFO:teuthology.orchestra.run.smithi062.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-04-25T09:00:48.719 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-25T09:00:48.941 INFO:teuthology.orchestra.run.smithi062.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-04-25T09:00:48.957 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-25T09:00:48.957 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-25T09:00:49.074 INFO:teuthology.orchestra.run.smithi062.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 983 kB/s | 548 kB 00:00 2024-04-25T09:00:49.208 INFO:teuthology.orchestra.run.smithi062.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-25T09:00:49.242 INFO:teuthology.orchestra.run.smithi062.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-04-25T09:00:49.275 INFO:teuthology.orchestra.run.smithi062.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 928 kB/s | 309 kB 00:00 2024-04-25T09:00:49.359 INFO:teuthology.orchestra.run.smithi062.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 585 kB/s | 49 kB 00:00 2024-04-25T09:00:49.418 INFO:teuthology.orchestra.run.smithi062.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 149 kB/s | 26 kB 00:00 2024-04-25T09:00:49.443 INFO:teuthology.orchestra.run.smithi062.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 815 kB/s | 67 kB 00:00 2024-04-25T09:00:49.551 INFO:teuthology.orchestra.run.smithi062.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-25T09:00:49.660 INFO:teuthology.orchestra.run.smithi062.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 627 kB/s | 151 kB 00:00 2024-04-25T09:00:49.702 INFO:teuthology.orchestra.run.smithi062.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-04-25T09:00:49.744 INFO:teuthology.orchestra.run.smithi062.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-25T09:00:49.769 INFO:teuthology.orchestra.run.smithi062.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-25T09:00:49.836 INFO:teuthology.orchestra.run.smithi062.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 242 kB/s | 32 kB 00:00 2024-04-25T09:00:49.954 INFO:teuthology.orchestra.run.smithi062.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 939 kB/s | 172 kB 00:00 2024-04-25T09:00:49.979 INFO:teuthology.orchestra.run.smithi062.stdout:(118/152): python3-certifi-2021.10.8-2.el9.noar 64 kB/s | 15 kB 00:00 2024-04-25T09:00:50.054 INFO:teuthology.orchestra.run.smithi062.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-25T09:00:50.088 INFO:teuthology.orchestra.run.smithi062.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-25T09:00:50.114 INFO:teuthology.orchestra.run.smithi062.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 265 kB/s | 35 kB 00:00 2024-04-25T09:00:50.146 INFO:teuthology.orchestra.run.smithi062.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-25T09:00:50.205 INFO:teuthology.orchestra.run.smithi062.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-04-25T09:00:50.230 INFO:teuthology.orchestra.run.smithi062.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-25T09:00:50.280 INFO:teuthology.orchestra.run.smithi062.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 472 kB/s | 23 kB 00:00 2024-04-25T09:00:50.305 INFO:teuthology.orchestra.run.smithi062.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-25T09:00:50.331 INFO:teuthology.orchestra.run.smithi062.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 396 kB/s | 19 kB 00:00 2024-04-25T09:00:50.390 INFO:teuthology.orchestra.run.smithi062.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-04-25T09:00:50.465 INFO:teuthology.orchestra.run.smithi062.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-04-25T09:00:50.490 INFO:teuthology.orchestra.run.smithi062.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 467 kB/s | 46 kB 00:00 2024-04-25T09:00:50.532 INFO:teuthology.orchestra.run.smithi062.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-25T09:00:50.574 INFO:teuthology.orchestra.run.smithi062.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-25T09:00:50.599 INFO:teuthology.orchestra.run.smithi062.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 532 kB/s | 58 kB 00:00 2024-04-25T09:00:50.641 INFO:teuthology.orchestra.run.smithi062.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-25T09:00:50.666 INFO:teuthology.orchestra.run.smithi062.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 180 kB/s | 16 kB 00:00 2024-04-25T09:00:50.692 INFO:teuthology.orchestra.run.smithi062.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 985 kB/s | 90 kB 00:00 2024-04-25T09:00:50.742 INFO:teuthology.orchestra.run.smithi062.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 413 kB/s | 31 kB 00:00 2024-04-25T09:00:50.767 INFO:teuthology.orchestra.run.smithi062.stdout:(138/152): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-04-25T09:00:50.826 INFO:teuthology.orchestra.run.smithi062.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-25T09:00:50.851 INFO:teuthology.orchestra.run.smithi062.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2024-04-25T09:00:50.876 INFO:teuthology.orchestra.run.smithi062.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-04-25T09:00:50.918 INFO:teuthology.orchestra.run.smithi062.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 559 kB/s | 51 kB 00:00 2024-04-25T09:00:50.952 INFO:teuthology.orchestra.run.smithi062.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T09:00:50.977 INFO:teuthology.orchestra.run.smithi062.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 899 kB/s | 90 kB 00:00 2024-04-25T09:00:51.011 INFO:teuthology.orchestra.run.smithi062.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-04-25T09:00:51.036 INFO:teuthology.orchestra.run.smithi062.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-04-25T09:00:51.069 INFO:teuthology.orchestra.run.smithi062.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-25T09:00:51.145 INFO:teuthology.orchestra.run.smithi062.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T09:00:51.512 INFO:teuthology.orchestra.run.smithi062.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.7 MB/s | 1.3 MB 00:00 2024-04-25T09:00:51.686 INFO:teuthology.orchestra.run.smithi062.stdout:(150/152): librados2-19.0.0-1628.g97ae6cc9.el9. 5.2 MB/s | 3.4 MB 00:00 2024-04-25T09:00:51.708 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-25T09:00:51.709 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-25T09:00:51.853 INFO:teuthology.orchestra.run.smithi062.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-04-25T09:00:51.954 INFO:teuthology.orchestra.run.smithi062.stdout:(152/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 3.6 MB/s | 3.1 MB 00:00 2024-04-25T09:00:51.960 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:00:51.960 INFO:teuthology.orchestra.run.smithi062.stdout:Total 10 MB/s | 215 MB 00:20 2024-04-25T09:00:54.900 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-25T09:00:55.159 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-25T09:00:55.159 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-25T09:00:57.891 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-25T09:00:57.891 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-25T09:00:58.278 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-25T09:00:58.400 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:00:58.497 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:00:58.631 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:00:58.690 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:00:58.724 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:00:58.854 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:00:59.298 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:00:59.405 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:00:59.582 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:00:59.608 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:00:59.734 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:00:59.774 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:01:00.100 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:01:00.162 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:01:00.187 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:01:00.330 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:01:00.355 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:01:00.441 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:01:00.489 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:01:00.554 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:01:00.865 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:01:00.925 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:01:01.114 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:01:01.250 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:01:01.298 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:01:01.347 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:01:01.450 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:01:01.519 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:01:01.590 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:01:01.640 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:01:01.680 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:01:01.773 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:01:01.850 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:01:01.889 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:01:01.944 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:01:02.059 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:01:02.101 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:01:02.155 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:01:02.192 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:01:02.305 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:01:02.363 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:01:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:01:02.650 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:01:02.723 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:01:03.381 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:01:03.487 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:01:03.523 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:01:03.727 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:01:03.783 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:01:03.858 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:01:04.148 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:01:04.212 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:01:04.293 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:01:04.389 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:01:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:01:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:01:04.541 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:01:04.581 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:01:04.652 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:01:04.710 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:01:04.776 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-25T09:01:04.886 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:01:04.906 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:01:04.963 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:01:04.977 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:01:05.000 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:01:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:01:05.108 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:01:05.173 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:01:05.196 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:01:05.207 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:01:05.242 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:01:05.291 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:01:05.323 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:01:05.339 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:01:05.356 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:01:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:01:05.727 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:01:05.780 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:01:05.788 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:01:05.827 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:01:05.876 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:01:05.903 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:01:05.943 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:01:06.001 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:01:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:01:06.104 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:01:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:01:06.129 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:01:06.190 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:01:06.261 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:01:06.266 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:01:06.295 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:01:06.320 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:01:06.473 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:01:06.535 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:01:06.577 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:01:06.648 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:01:06.648 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:01:06.710 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:01:06.733 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:01:06.807 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:01:06.882 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:01:06.918 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:01:06.951 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:01:07.057 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:01:07.085 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:01:07.167 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:01:07.494 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:01:07.538 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:01:07.718 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:01:07.840 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:01:07.855 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:01:07.877 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:01:07.906 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:01:07.925 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:01:07.939 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:01:08.029 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:01:08.061 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:01:08.089 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:01:08.126 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:01:08.154 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:01:08.211 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:01:08.230 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:01:08.250 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:01:08.274 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:01:08.375 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:01:08.443 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:01:08.486 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:01:08.504 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:01:08.546 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:01:08.547 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:01:08.665 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:01:08.668 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:01:08.715 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:01:08.772 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:01:08.804 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:01:08.931 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:01:08.991 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:01:09.105 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:01:09.298 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:01:09.370 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:01:09.521 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:01:09.669 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:01:10.087 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:01:10.201 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:01:10.243 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:01:10.298 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:01:10.347 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:01:10.441 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:01:10.719 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:01:10.797 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:01:10.902 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:01:11.011 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:01:11.055 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:01:11.096 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:01:11.184 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:01:11.228 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:01:11.321 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:01:11.330 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:01:11.373 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:01:11.444 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:01:11.489 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:01:11.540 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:01:11.572 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:01:11.624 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:01:11.661 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:01:11.736 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:01:11.878 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:01:11.919 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:01:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:01:11.968 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:01:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:01:12.001 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:01:12.034 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:01:12.163 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:01:12.197 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:01:12.243 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:01:12.262 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:01:12.429 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:01:12.474 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:01:12.525 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:01:12.600 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:01:12.682 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:01:12.740 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:01:12.788 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:01:12.843 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:01:12.879 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:01:12.893 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:01:12.918 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:01:12.968 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:01:13.019 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:01:13.042 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:01:13.045 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:01:13.199 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:01:13.276 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:01:13.322 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:01:13.396 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:01:13.555 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:01:13.626 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:01:14.623 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:01:14.682 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:01:14.715 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:01:14.846 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:01:14.920 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:01:15.034 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:01:15.075 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:01:15.324 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:01:15.364 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:01:15.488 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:01:15.738 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:01:15.784 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:01:16.418 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:01:16.425 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:01:16.470 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:01:16.574 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:01:16.621 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:01:16.622 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:01:16.622 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:01:16.622 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:16.663 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:01:18.263 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:01:18.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:01:18.435 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:01:18.484 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:01:18.897 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:01:18.948 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:01:19.083 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:01:19.123 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:01:19.183 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:01:19.753 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:01:19.785 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:01:19.886 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:01:19.911 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:01:22.617 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:01:22.657 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:01:23.368 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:01:23.419 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:01:23.587 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:01:23.587 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:01:23.587 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:01:23.587 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:23.636 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:01:29.848 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:01:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-04-25T09:01:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-04-25T09:01:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-04-25T09:01:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-04-25T09:01:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-04-25T09:01:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /root 2024-04-25T09:01:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-04-25T09:01:29.849 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:30.103 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:01:30.149 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:01:30.150 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:30.150 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T09:01:30.150 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:31.770 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:01:31.847 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:01:31.847 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:31.847 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T09:01:31.847 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:01:31.847 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:01:31.847 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:32.078 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:01:32.145 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:01:32.145 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:32.145 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T09:01:32.145 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:01:32.145 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:01:32.145 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:32.662 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:01:32.740 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:01:32.740 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:32.741 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T09:01:32.741 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:01:32.741 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:01:32.741 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:32.880 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T09:01:32.944 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T09:01:33.021 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T09:01:33.261 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T09:01:33.501 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T09:01:33.603 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T09:01:33.728 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:01:33.904 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:01:33.953 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T09:01:33.990 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T09:01:34.032 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T09:01:34.630 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:01:34.677 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:01:34.729 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T09:01:34.838 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T09:01:34.879 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:01:35.209 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:01:35.246 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:01:36.386 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:01:36.421 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:01:36.569 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:01:36.760 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T09:01:36.790 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:01:36.866 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:01:36.866 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:36.867 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T09:01:36.867 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:01:36.867 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:01:36.867 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:36.924 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:01:36.960 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:01:37.057 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T09:01:37.123 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-04-25T09:01:37.531 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:37.757 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:01:37.807 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:01:37.807 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:37.807 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T09:01:37.807 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T09:01:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:01:38.148 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:01:38.148 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:38.148 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T09:01:38.148 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:01:38.148 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:01:38.148 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:38.210 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:01:38.296 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:01:38.296 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:38.296 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T09:01:38.296 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:38.403 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:01:38.503 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:01:38.503 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:38.503 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T09:01:38.503 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:01:38.503 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:01:38.504 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:38.912 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:01:38.994 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:01:38.994 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:38.994 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T09:01:38.994 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:01:38.995 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:01:38.995 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:39.501 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:01:39.597 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:01:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T09:01:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:01:39.597 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:01:39.597 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:39.843 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:01:39.924 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:01:39.925 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:39.925 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T09:01:39.925 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:01:39.925 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:01:39.925 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:40.477 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:01:40.551 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:01:40.551 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:40.551 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T09:01:40.551 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:40.707 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T09:01:40.770 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T09:01:40.850 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T09:01:41.116 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T09:01:41.183 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T09:01:41.274 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T09:01:41.396 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:01:41.600 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:01:41.656 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T09:01:41.697 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T09:01:41.737 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T09:01:42.435 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:01:42.474 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:01:42.528 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T09:01:42.642 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T09:01:42.675 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:01:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:01:43.059 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:01:43.849 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T09:01:43.895 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T09:01:43.973 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T09:01:44.019 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T09:01:44.083 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T09:01:44.259 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:01:44.293 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:01:44.413 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T09:01:44.449 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:01:44.480 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T09:01:44.646 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T09:01:44.678 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:01:44.763 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:01:44.763 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:44.763 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T09:01:44.764 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:01:44.764 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:01:44.764 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:44.821 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:01:44.861 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:01:45.007 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T09:01:45.075 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T09:01:45.417 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:01:45.464 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:01:45.523 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:01:46.106 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T09:01:46.140 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:01:46.213 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:01:46.213 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:46.213 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T09:01:46.213 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:01:46.213 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:01:46.214 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:46.285 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:01:46.371 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:01:46.371 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:46.371 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T09:01:46.371 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:46.450 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:01:46.533 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:01:46.533 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:46.533 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T09:01:46.533 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:01:46.534 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:01:46.534 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:46.830 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T09:01:46.909 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:01:46.946 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T09:01:46.986 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:01:46.986 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:01:46.986 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T09:01:46.986 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:01:46.986 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:01:46.986 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:01:47.027 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T09:01:47.219 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T09:01:47.260 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T09:01:47.261 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:01:47.420 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:01:47.468 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:01:47.468 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:01:47.658 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:01:47.790 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:01:48.117 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:01:48.117 INFO:teuthology.orchestra.run.smithi121.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T09:01:48.118 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:52.117 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T09:01:52.171 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T09:01:52.260 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T09:01:52.299 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T09:01:52.342 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T09:01:52.673 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T09:01:52.726 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T09:01:52.754 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T09:01:53.420 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T09:01:53.420 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T09:01:53.420 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T09:01:53.420 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T09:01:53.420 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T09:01:53.420 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T09:01:53.420 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T09:01:53.421 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T09:01:53.422 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T09:01:53.434 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T09:01:53.434 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T09:01:53.434 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T09:01:53.434 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T09:01:53.435 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T09:01:53.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T09:01:53.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T09:01:53.439 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T09:01:53.440 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T09:01:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T09:01:53.442 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:01:53.443 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:01:53.773 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:01:53.833 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:01:55.310 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T09:01:55.425 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T09:01:55.518 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T09:01:55.742 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T09:01:55.791 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T09:01:55.791 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:01:55.909 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:01:55.955 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:01:55.956 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:01:55.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:01:55.994 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:55.994 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-04-25T09:01:55.994 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.994 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.995 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.996 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi121.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T09:01:55.999 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.000 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T09:01:56.001 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T09:01:56.002 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T09:01:56.003 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T09:01:56.004 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T09:01:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T09:01:56.006 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T09:01:56.006 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:01:56.006 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-25T09:01:56.203 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:01:56.339 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:01:56.658 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:56.659 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:01:56.659 INFO:teuthology.orchestra.run.smithi062.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T09:01:56.659 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:02.096 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T09:02:02.096 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T09:02:02.097 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T09:02:02.098 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T09:02:02.111 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T09:02:02.112 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T09:02:02.113 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T09:02:02.114 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T09:02:02.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T09:02:02.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T09:02:02.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T09:02:02.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T09:02:02.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T09:02:02.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T09:02:02.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T09:02:02.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T09:02:02.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T09:02:02.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T09:02:02.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T09:02:02.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T09:02:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T09:02:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T09:02:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T09:02:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T09:02:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T09:02:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T09:02:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:02:02.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T09:02:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T09:02:04.583 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T09:02:04.585 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T09:02:04.586 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T09:02:04.587 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.588 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T09:02:04.589 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T09:02:04.590 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T09:02:04.591 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T09:02:04.592 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T09:02:04.593 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T09:02:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T09:02:04.594 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:04.594 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-25T09:02:05.347 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:05.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:02:05.495 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T09:02:05.559 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:02:05.559 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:02:05.559 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T09:02:05.561 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:02:05.722 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T09:02:05.787 INFO:teuthology.orchestra.run.smithi121.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:02:05.787 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:02:05.787 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T09:02:05.789 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T09:02:05.790 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:05.790 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T09:02:05.860 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:05.860 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T09:02:05.950 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T09:02:05.950 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:05.950 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T09:02:06.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T09:02:06.157 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:06.157 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T09:02:06.236 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T09:02:06.303 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T09:02:06.304 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:06.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T09:02:06.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T09:02:06.473 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:06.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T09:02:06.547 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T09:02:06.659 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T09:02:06.659 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:06.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T09:02:06.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T09:02:06.839 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:06.839 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T09:02:06.920 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T09:02:07.038 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T09:02:07.128 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T09:02:07.128 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2024-04-25T09:02:07.132 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2024-04-25T09:02:07.205 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T09:02:07.205 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T09:02:07.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T09:02:07.276 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T09:02:07.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T09:02:07.280 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T09:02:07.356 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T09:02:07.356 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T09:02:07.356 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T09:02:07.357 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:07.357 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T09:02:07.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:07.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:07.643 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:07.643 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T09:02:07.694 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:07.912 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:08.004 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T09:02:08.004 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T09:02:08.004 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673006', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': '5787d42d10024ba410e822a8aba474331cb9af02'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5869, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '5787d42d10024ba410e822a8aba474331cb9af02', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi062.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHMnWXFNDmuTYiUV92lJ3TMit5J5Xs+Q/H2oIivYKoPZqJO/wt8rhPFUHuF/0d3myBhfP8KnLij1ATvxswCAaLc=', 'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMx7oyGubUIs9wXizG7gH/lHca55JUIEaB7eeEqpf3/AmudOgYHcnkAhKDWZKruOKEwWLL68EiRoDcftHhzZcDE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_08:15:18', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T09:02:08.005 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T09:02:08.013 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T09:02:08.066 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T09:02:08.070 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T09:02:08.146 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:08.146 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T09:02:08.183 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T09:02:08.184 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 09:01:57.195758257 +0000 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 08:59:34.014757296 +0000 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 08:59:34.014757296 +0000 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 08:59:34.014757296 +0000 2024-04-25T09:02:08.263 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T09:02:08.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:02:08.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:02:08.376 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000412489 s, 1.2 MB/s 2024-04-25T09:02:08.379 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T09:02:08.424 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 09:01:57.195758257 +0000 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 08:59:34.607747948 +0000 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 08:59:34.607747948 +0000 2024-04-25T09:02:08.513 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 08:59:34.607747948 +0000 2024-04-25T09:02:08.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T09:02:08.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:02:08.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:02:08.612 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000274345 s, 1.9 MB/s 2024-04-25T09:02:08.615 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T09:02:08.655 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-04-25T09:02:08.737 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T09:02:08.737 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:02:08.738 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-25T09:02:08.738 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:02:08.738 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:02:08.738 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 09:01:57.196758241 +0000 2024-04-25T09:02:08.738 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 08:59:35.227738175 +0000 2024-04-25T09:02:08.738 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 08:59:35.227738175 +0000 2024-04-25T09:02:08.738 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 08:59:35.227738175 +0000 2024-04-25T09:02:08.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T09:02:08.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:02:08.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:02:08.834 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000288926 s, 1.8 MB/s 2024-04-25T09:02:08.838 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T09:02:08.876 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-04-25T09:02:08.959 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T09:02:08.959 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:02:08.959 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-25T09:02:08.959 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:02:08.959 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:02:08.959 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 09:01:57.197758225 +0000 2024-04-25T09:02:08.959 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 08:59:35.966726526 +0000 2024-04-25T09:02:08.959 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 08:59:35.966726526 +0000 2024-04-25T09:02:08.960 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 08:59:35.966726526 +0000 2024-04-25T09:02:08.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T09:02:09.056 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:02:09.056 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:02:09.056 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000273486 s, 1.9 MB/s 2024-04-25T09:02:09.059 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T09:02:09.098 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:09.099 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:09.099 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T09:02:09.157 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T09:02:09.157 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-25 09:01:48.636582611 +0000 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-25 08:59:24.547194816 +0000 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-25 08:59:24.547194816 +0000 2024-04-25T09:02:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-25 08:59:24.547194816 +0000 2024-04-25T09:02:09.211 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T09:02:09.287 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-25T09:02:09.287 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-25T09:02:09.287 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000328898 s, 1.6 MB/s 2024-04-25T09:02:09.291 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T09:02:09.340 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-04-25T09:02:09.388 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T09:02:09.388 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:02:09.388 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-25T09:02:09.389 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:02:09.389 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:02:09.389 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-25 09:01:48.637582598 +0000 2024-04-25T09:02:09.389 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-25 08:59:25.114187679 +0000 2024-04-25T09:02:09.389 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-25 08:59:25.114187679 +0000 2024-04-25T09:02:09.389 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-25 08:59:25.114187679 +0000 2024-04-25T09:02:09.389 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T09:02:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-25T09:02:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-25T09:02:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000342474 s, 1.5 MB/s 2024-04-25T09:02:09.465 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T09:02:09.519 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-04-25T09:02:09.568 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T09:02:09.569 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:02:09.569 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-25T09:02:09.569 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:02:09.569 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:02:09.569 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-25 09:01:48.638582585 +0000 2024-04-25T09:02:09.569 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-25 08:59:25.708180202 +0000 2024-04-25T09:02:09.569 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-25 08:59:25.708180202 +0000 2024-04-25T09:02:09.569 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-25 08:59:25.708180202 +0000 2024-04-25T09:02:09.570 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T09:02:09.645 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-25T09:02:09.646 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-25T09:02:09.646 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000353882 s, 1.4 MB/s 2024-04-25T09:02:09.649 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T09:02:09.702 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-25 09:01:48.639582572 +0000 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-25 08:59:26.286172926 +0000 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-25 08:59:26.286172926 +0000 2024-04-25T09:02:09.751 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-25 08:59:26.286172926 +0000 2024-04-25T09:02:09.751 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T09:02:09.827 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-25T09:02:09.827 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-25T09:02:09.827 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000384631 s, 1.3 MB/s 2024-04-25T09:02:09.831 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T09:02:09.887 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:09.887 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T09:02:09.887 INFO:tasks.ceph:Generating config... 2024-04-25T09:02:09.889 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T09:02:09.889 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T09:02:09.889 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T09:02:09.889 INFO:tasks.ceph:[client] pid file = 2024-04-25T09:02:09.889 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T09:02:09.889 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T09:02:09.889 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T09:02:09.889 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T09:02:09.889 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T09:02:09.889 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T09:02:09.889 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T09:02:09.890 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T09:02:09.891 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T09:02:09.892 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T09:02:09.892 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T09:02:09.892 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T09:02:09.893 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T09:02:09.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T09:02:10.011 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T09:02:10.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T09:02:10.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:02:10.182 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.62'), ('mon.b', '172.21.15.121'), ('mon.c', '[v2:172.21.15.121:3301,v1:172.21.15.121:6790]')] 2024-04-25T09:02:10.183 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.62,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T09:02:10.184 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:10.184 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T09:02:10.223 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.62 --add b 172.21.15.121 --addv c '[v2:172.21.15.121:3301,v1:172.21.15.121:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:10.354 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:10.354 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: generated fsid 9e112875-3fc2-4963-90e0-2b56d5516d5c 2024-04-25T09:02:10.354 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = quincy 2024-04-25T09:02:10.354 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 0 2024-04-25T09:02:10.354 INFO:teuthology.orchestra.run.smithi062.stdout:fsid 9e112875-3fc2-4963-90e0-2b56d5516d5c 2024-04-25T09:02:10.354 INFO:teuthology.orchestra.run.smithi062.stdout:last_changed 2024-04-25T09:02:10.366637+0000 2024-04-25T09:02:10.355 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-04-25T09:02:10.366637+0000 2024-04-25T09:02:10.355 INFO:teuthology.orchestra.run.smithi062.stdout:min_mon_release 17 (quincy) 2024-04-25T09:02:10.355 INFO:teuthology.orchestra.run.smithi062.stdout:election_strategy: 1 2024-04-25T09:02:10.355 INFO:teuthology.orchestra.run.smithi062.stdout:0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-25T09:02:10.355 INFO:teuthology.orchestra.run.smithi062.stdout:1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-04-25T09:02:10.355 INFO:teuthology.orchestra.run.smithi062.stdout:2: [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] mon.c 2024-04-25T09:02:10.355 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T09:02:10.357 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T09:02:10.396 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9e112875-3fc2-4963-90e0-2b56d5516d5c... 2024-04-25T09:02:10.397 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:02:10.441 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2024-04-25T09:02:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2024-04-25T09:02:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.550 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.62,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]" 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on skipped message = False 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = 9e112875-3fc2-4963-90e0-2b56d5516d5c 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2024-04-25T09:02:10.551 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2024-04-25T09:02:10.552 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug trim objects = true 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.553 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: bdev async discard = True 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: bdev enable discard = True 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = bitmap 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 1/20 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 1/20 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 4/10 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2024-04-25T09:02:10.554 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2024-04-25T09:02:10.555 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2024-04-25T09:02:10.556 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: mon op complaint time = 120 2024-04-25T09:02:10.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: client mount timeout = 600 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = "" 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2024-04-25T09:02:10.558 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout:[mds] 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds = 20 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds balancer = 20 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout: mds cache max size = 1000 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug frag = True 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug scatterstat = True 2024-04-25T09:02:10.559 INFO:teuthology.orchestra.run.smithi062.stdout: mds log max segments = 1 2024-04-25T09:02:10.560 INFO:teuthology.orchestra.run.smithi062.stdout: mds op complaint time = 180 2024-04-25T09:02:10.560 INFO:teuthology.orchestra.run.smithi062.stdout: mds verify scatter = True 2024-04-25T09:02:10.560 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2024-04-25T09:02:10.560 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2024-04-25T09:02:10.560 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2024-04-25T09:02:10.586 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2024-04-25T09:02:10.586 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2024-04-25T09:02:10.586 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:02:10.586 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2024-04-25T09:02:10.587 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.587 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2024-04-25T09:02:10.587 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.587 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2024-04-25T09:02:10.587 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.587 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2024-04-25T09:02:10.587 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2024-04-25T09:02:10.587 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2024-04-25T09:02:10.588 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2024-04-25T09:02:10.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.590 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2024-04-25T09:02:10.590 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2024-04-25T09:02:10.590 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2024-04-25T09:02:10.590 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "172.21.15.62,172.21.15.121,[v2:172.21.15.121:3301,v1:172.21.15.121:6790]" 2024-04-25T09:02:10.590 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on skipped message = False 2024-04-25T09:02:10.590 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = 9e112875-3fc2-4963-90e0-2b56d5516d5c 2024-04-25T09:02:10.590 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.590 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2024-04-25T09:02:10.591 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug trim objects = true 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2024-04-25T09:02:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: bdev async discard = True 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: bdev enable discard = True 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore allocator = bitmap 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 1/20 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 1/20 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-25T09:02:10.593 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 4/10 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-04-25T09:02:10.594 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:02:10.595 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: mon op complaint time = 120 2024-04-25T09:02:10.596 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout: client mount timeout = 600 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2024-04-25T09:02:10.597 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = "" 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout:[mds] 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds = 20 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds balancer = 20 2024-04-25T09:02:10.598 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: mds cache max size = 1000 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug frag = True 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug scatterstat = True 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: mds log max segments = 1 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: mds op complaint time = 180 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: mds verify scatter = True 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2024-04-25T09:02:10.599 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2024-04-25T09:02:10.619 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T09:02:10.619 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T09:02:10.730 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T09:02:10.731 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:10.731 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T09:02:10.768 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:10.768 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T09:02:10.847 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:02:10.847 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:10.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T09:02:10.848 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:02:10.978 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:10.978 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:11.059 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:02:11.059 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:11.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T09:02:11.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:02:11.199 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:11.199 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:11.296 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T09:02:11.296 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T09:02:11.297 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T09:02:11.410 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T09:02:11.417 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T09:02:11.596 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T09:02:11.605 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T09:02:11.606 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T09:02:11.723 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T09:02:11.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T09:02:11.799 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T09:02:11.913 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T09:02:11.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T09:02:11.993 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T09:02:12.166 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T09:02:12.177 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T09:02:12.256 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T09:02:12.410 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T09:02:12.419 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T09:02:12.496 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T09:02:12.496 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T09:02:12.616 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T09:02:12.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T09:02:12.766 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T09:02:12.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T09:02:12.923 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T09:02:12.978 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T09:02:13.090 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T09:02:13.127 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T09:02:13.127 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T09:02:13.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T09:02:13.221 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:13.221 INFO:tasks.ceph:role: osd.0 2024-04-25T09:02:13.221 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:02:13.221 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:02:13.310 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:02:13.713 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-25T09:02:13.719 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-25T09:02:13.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T09:02:13.866 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T09:02:13.956 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T09:02:14.035 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:14.036 INFO:tasks.ceph:role: osd.1 2024-04-25T09:02:14.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:02:14.036 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:02:14.125 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:02:14.481 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-25T09:02:14.486 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-25T09:02:14.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T09:02:14.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T09:02:14.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T09:02:14.744 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:14.744 INFO:tasks.ceph:role: osd.2 2024-04-25T09:02:14.744 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:02:14.744 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T09:02:14.835 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:02:14.836 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:02:15.228 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-25T09:02:15.237 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-25T09:02:15.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T09:02:15.385 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T09:02:15.480 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T09:02:15.569 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:15.569 INFO:tasks.ceph:role: osd.3 2024-04-25T09:02:15.569 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:02:15.569 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T09:02:15.663 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:02:15.663 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:02:15.663 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:02:15.663 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:02:15.663 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:02:15.663 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:02:15.663 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:02:15.663 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:02:15.664 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:02:15.664 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:02:16.078 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-25T09:02:16.085 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-25T09:02:16.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T09:02:16.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T09:02:16.345 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:16.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:16.506+0000 7f95557dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T09:02:16.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:16.507+0000 7f95557dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T09:02:16.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:16.507+0000 7f95557dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:02:16.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:16.507+0000 7f95557dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:02:16.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:16.507+0000 7f95557dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:02:16.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:16.511+0000 7f95557dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T09:02:19.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T09:02:19.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:19.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:19.974+0000 7f4184c61640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T09:02:19.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:19.974+0000 7f4184c61640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T09:02:19.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:19.974+0000 7f4184c61640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:02:19.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:19.974+0000 7f4184c61640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:02:19.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:19.974+0000 7f4184c61640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:02:19.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:19.977+0000 7f4184c61640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T09:02:23.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T09:02:23.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:23.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:23.446+0000 7f3e85d14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T09:02:23.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:23.447+0000 7f3e85d14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T09:02:23.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:23.447+0000 7f3e85d14640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:02:23.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:23.447+0000 7f3e85d14640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:02:23.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:23.447+0000 7f3e85d14640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:02:23.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:23.449+0000 7f3e85d14640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T09:02:26.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T09:02:26.750 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:26.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:26.904+0000 7f07fd416640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T09:02:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:26.904+0000 7f07fd416640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T09:02:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:26.904+0000 7f07fd416640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:02:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:26.904+0000 7f07fd416640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:02:26.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:26.904+0000 7f07fd416640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:02:26.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:26.907+0000 7f07fd416640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T09:02:30.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T09:02:30.216 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T09:02:30.305 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:30.306 INFO:tasks.ceph:role: osd.4 2024-04-25T09:02:30.306 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:02:30.306 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T09:02:30.394 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:02:30.394 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:02:30.394 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:02:30.394 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:02:30.395 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:02:30.395 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:02:30.395 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:02:30.395 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:02:30.395 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:02:30.395 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:02:30.803 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-25T09:02:30.809 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-25T09:02:30.809 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T09:02:30.978 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T09:02:31.077 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T09:02:31.158 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:31.158 INFO:tasks.ceph:role: osd.5 2024-04-25T09:02:31.158 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:02:31.158 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T09:02:31.246 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:02:31.246 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:02:31.246 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:02:31.246 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:02:31.246 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:02:31.246 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:02:31.246 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:02:31.247 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:02:31.247 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:02:31.247 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:02:31.605 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-25T09:02:31.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-25T09:02:31.610 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T09:02:31.736 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T09:02:31.814 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T09:02:31.897 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:31.898 INFO:tasks.ceph:role: osd.6 2024-04-25T09:02:31.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:02:31.898 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T09:02:31.983 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:02:31.983 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:02:31.983 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:02:31.983 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:02:31.984 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:02:31.984 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:02:31.984 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:02:31.984 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:02:31.984 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:02:31.984 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:02:32.372 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-25T09:02:32.378 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-25T09:02:32.378 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T09:02:32.511 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T09:02:32.602 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T09:02:32.686 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:02:32.686 INFO:tasks.ceph:role: osd.7 2024-04-25T09:02:32.686 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:02:32.687 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T09:02:32.774 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:02:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:02:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:02:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:02:32.775 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:02:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:02:32.775 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:02:32.775 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:02:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:02:32.775 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:02:33.184 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-25T09:02:33.191 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-25T09:02:33.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T09:02:33.346 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T09:02:33.440 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:33.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:33.580+0000 7f05a1388640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T09:02:33.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:33.580+0000 7f05a1388640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T09:02:33.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:33.580+0000 7f05a1388640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:02:33.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:33.580+0000 7f05a1388640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:02:33.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:33.580+0000 7f05a1388640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:02:33.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:33.584+0000 7f05a1388640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T09:02:36.794 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T09:02:36.872 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:36.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:37.004+0000 7f41ce344640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T09:02:36.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:37.005+0000 7f41ce344640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T09:02:36.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:37.005+0000 7f41ce344640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:02:36.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:37.005+0000 7f41ce344640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:02:36.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:37.005+0000 7f41ce344640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:02:36.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:37.007+0000 7f41ce344640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T09:02:38.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:38.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:40.234 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T09:02:40.318 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:40.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:40.457+0000 7fbfaa211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T09:02:40.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:40.458+0000 7fbfaa211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T09:02:40.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:40.458+0000 7fbfaa211640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:02:40.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:40.458+0000 7fbfaa211640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:02:40.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:40.458+0000 7fbfaa211640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:02:40.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:40.461+0000 7fbfaa211640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T09:02:43.655 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T09:02:43.743 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:43.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:43.882+0000 7f4e81014640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T09:02:43.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:43.883+0000 7f4e81014640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T09:02:43.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:43.883+0000 7f4e81014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:02:43.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:43.883+0000 7f4e81014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:02:43.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:43.883+0000 7f4e81014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:02:43.875 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:43.886+0000 7f4e81014640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T09:02:47.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T09:02:47.181 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T09:02:47.182 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:47.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T09:02:47.277 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:47.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T09:02:47.366 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:47.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T09:02:47.454 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:47.454 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T09:02:47.541 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:47.541 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T09:02:47.630 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:47.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T09:02:47.714 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:47.714 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T09:02:47.796 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:47.796 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T09:02:47.883 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:47.883 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T09:02:47.963 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:47.963 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T09:02:48.047 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:48.047 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T09:02:48.130 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:48.130 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T09:02:48.214 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:48.214 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T09:02:48.295 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:48.295 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T09:02:48.373 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:48.373 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T09:02:48.436 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:48.436 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T09:02:48.491 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:48.491 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T09:02:48.547 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:48.547 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T09:02:48.599 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T09:02:48.599 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T09:02:48.607 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A== 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-04-25T09:02:48.687 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ== 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA== 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ== 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA== 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw== 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w== 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA== 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-04-25T09:02:48.688 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg== 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA== 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw== 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q== 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w== 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg== 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout:[client.2] 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A== 2024-04-25T09:02:48.689 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg== 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi062.stdout:[client.3] 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg== 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.y] 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A== 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ== 2024-04-25T09:02:48.690 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA== 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ== 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA== 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw== 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.3] 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w== 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA== 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg== 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2024-04-25T09:02:48.691 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA== 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.4] 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw== 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.5] 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q== 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.6] 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w== 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.7] 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg== 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout:[client.2] 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A== 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout:[client.1] 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg== 2024-04-25T09:02:48.692 INFO:teuthology.orchestra.run.smithi121.stdout:[client.3] 2024-04-25T09:02:48.693 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg== 2024-04-25T09:02:48.694 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:02:48.742 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:02:48.878 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:02:48.883 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:02:49.019 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:02:49.026 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:02:49.163 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:49.169 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:49.308 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:49.312 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:49.447 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:49.455 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:49.590 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:49.595 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:49.729 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:02:49.735 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:02:49.862 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:02:49.868 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:02:50.004 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:02:50.010 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:02:50.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:50.154 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:50.289 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:50.295 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:50.431 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:50.436 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:50.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:50.577 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:02:50.712 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:02:50.717 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:02:50.853 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:02:50.861 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:02:50.998 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:02:51.003 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:02:51.142 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:02:51.148 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:02:51.282 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T09:02:51.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T09:02:51.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:02:51.518 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T09:02:51.587 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T09:02:51.678 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:02:51.828 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T09:02:51.907 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T09:02:51.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:02:52.134 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T09:02:52.198 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:52.206 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:02:52.259 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T09:02:52.259 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T09:02:52.259 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T09:02:52.267 INFO:tasks.ceph.mon.a:Started 2024-04-25T09:02:52.267 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T09:02:52.267 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T09:02:52.294 INFO:tasks.ceph.mon.b:Started 2024-04-25T09:02:52.294 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T09:02:52.295 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T09:02:52.300 INFO:tasks.ceph.mon.c:Started 2024-04-25T09:02:52.300 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T09:02:52.300 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T09:02:52.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T09:02:52.305 INFO:tasks.ceph.mgr.y:Started 2024-04-25T09:02:52.306 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T09:02:52.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T09:02:52.310 INFO:tasks.ceph.mgr.x:Started 2024-04-25T09:02:52.310 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T09:02:52.310 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T09:02:52.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.494+0000 7f53ce400640 1 Processor -- start 2024-04-25T09:02:52.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.494+0000 7f53ce400640 1 -- start start 2024-04-25T09:02:52.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.495+0000 7f53ce400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 0x7f53c80cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.495+0000 7f53ce400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53c8162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.495+0000 7f53ce400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c80d1c90 con 0x7f53c815dd60 2024-04-25T09:02:52.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.495+0000 7f53ce400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80d1f70 con 0x7f53c80ce990 2024-04-25T09:02:52.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.495+0000 7f53ce400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80d2250 con 0x7f53c8161bf0 2024-04-25T09:02:52.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53cd000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f53c815dd60 0x7f53c80d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51766/0 (socket says 172.21.15.62:51766) 2024-04-25T09:02:52.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53cd000640 1 -- 172.21.15.62:0/3080264711 learned_addr learned my addr 172.21.15.62:0/3080264711 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53cda00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53c8162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 0x7f53c80cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53cda00640 1 -- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 msgr2=0x7f53c8162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53cda00640 1 -- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 msgr2=0x7f53c8162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53cda00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53c8162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53c7e00640 1 -- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 msgr2=0x7f53c80cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53c7e00640 1 -- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 msgr2=0x7f53c80cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 0x7f53c80cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:02:52.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53cda00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53c8162080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:02:52.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.496+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 0x7f53c80cee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:02:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.697+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 0x7f53c80cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.697+0000 7f53cda00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53c8162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.698+0000 7f53c7e00640 1 -- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 msgr2=0x7f53c80cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T09:02:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.698+0000 7f53c7e00640 1 -- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 msgr2=0x7f53c80cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:02:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.698+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 0x7f53c80cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:02:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.698+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 0x7f53c80cee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T09:02:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.698+0000 7f53cda00640 1 -- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 msgr2=0x7f53c8162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T09:02:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.698+0000 7f53cda00640 1 -- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 msgr2=0x7f53c8162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:02:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.698+0000 7f53cda00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53c8162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:02:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.698+0000 7f53cda00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53c8162080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T09:02:52.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.739+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743653035 0 0) 0x7f53c80d1c90 con 0x7f53c815dd60 2024-04-25T09:02:52.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.739+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b0003520 con 0x7f53c815dd60 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1016106928 0 0) 0x7f53b0003520 con 0x7f53c815dd60 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 msgr2=0x7f53c8162080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53c8162080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 msgr2=0x7f53c80cee20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 --2- 172.21.15.62:0/3080264711 >> v2:172.21.15.121:3300/0 conn(0x7f53c80ce990 0x7f53c80cee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80d2530 con 0x7f53c815dd60 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f53bc001670 con 0x7f53c815dd60 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 >> v1:172.21.15.62:6789/0 conn(0x7f53c815dd60 legacy=0x7f53c80d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.742+0000 7f53c7400640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.743+0000 7f53c7400640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53b0003fe0 0x7f53b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.743+0000 7f53c7400640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0004d40 0x7f53b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.743+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80d1f70 con 0x7f53b0003fe0 2024-04-25T09:02:52.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.743+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80d2250 con 0x7f53b0004d40 2024-04-25T09:02:52.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.743+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b000bb20 con 0x7f53c8161bf0 2024-04-25T09:02:52.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.743+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53b0003fe0 0x7f53b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.744+0000 7f53cda00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0004d40 0x7f53b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.744+0000 7f53cd000640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.744+0000 7f53c7e00640 1 -- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 msgr2=0x7f53b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.744+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53c8161bf0 0x7f53b00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.744+0000 7f53c7e00640 1 -- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0004d40 msgr2=0x7f53b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.744+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0004d40 0x7f53b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.744+0000 7f53c7e00640 1 -- 172.21.15.62:0/3080264711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80d2530 con 0x7f53b0003fe0 2024-04-25T09:02:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.746+0000 7f53c7e00640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53b0003fe0 0x7f53b0004800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f53b400daa0 tx=0x7f53b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9621e282bcef117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.748+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4038070 con 0x7f53b0003fe0 2024-04-25T09:02:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.748+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b4033410 con 0x7f53b0003fe0 2024-04-25T09:02:52.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.748+0000 7f53c7400640 1 -- 172.21.15.62:0/3080264711 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400d6a0 con 0x7f53b0003fe0 2024-04-25T09:02:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.749+0000 7f53ce400640 1 -- 172.21.15.62:0/3080264711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53b0003fe0 msgr2=0x7f53b0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.749+0000 7f53ce400640 1 --2- 172.21.15.62:0/3080264711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53b0003fe0 0x7f53b0004800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f53b400daa0 tx=0x7f53b0005830 comp rx=0 tx=0).stop 2024-04-25T09:02:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.749+0000 7f53cda00640 1 -- 172.21.15.62:0/3080264711 reap_dead start 2024-04-25T09:02:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.750+0000 7f53ce400640 1 -- 172.21.15.62:0/3080264711 shutdown_connections 2024-04-25T09:02:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.750+0000 7f53ce400640 1 -- 172.21.15.62:0/3080264711 >> 172.21.15.62:0/3080264711 conn(0x7f53c806b210 msgr2=0x7f53c80668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.750+0000 7f53ce400640 1 -- 172.21.15.62:0/3080264711 shutdown_connections 2024-04-25T09:02:52.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.750+0000 7f53ce400640 1 -- 172.21.15.62:0/3080264711 wait complete. 2024-04-25T09:02:52.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.752+0000 7f53ce400640 1 Processor -- start 2024-04-25T09:02:52.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.752+0000 7f53ce400640 1 -- start start 2024-04-25T09:02:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.753+0000 7f53ce400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53b0003fe0 0x7f53c814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.753+0000 7f53ce400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80cc7f0 0x7f53c80c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53ce400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53c80b53d0 0x7f53c80b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53ce400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8162200 con 0x7f53c80cc7f0 2024-04-25T09:02:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53ce400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8162080 con 0x7f53c80b53d0 2024-04-25T09:02:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53ce400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8162380 con 0x7f53b0003fe0 2024-04-25T09:02:52.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80cc7f0 0x7f53c80c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53cd000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53b0003fe0 0x7f53c814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80cc7f0 0x7f53c80c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45142/0 (socket says 172.21.15.62:45142) 2024-04-25T09:02:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53c7e00640 1 -- 172.21.15.62:0/1295973381 learned_addr learned my addr 172.21.15.62:0/1295973381 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:02:52.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.754+0000 7f53cda00640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53c80b53d0 0x7f53c80b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.755+0000 7f53cd000640 1 -- 172.21.15.62:0/1295973381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53c80b53d0 msgr2=0x7f53c80b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.755+0000 7f53cd000640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53c80b53d0 0x7f53c80b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.755+0000 7f53cd000640 1 -- 172.21.15.62:0/1295973381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80cc7f0 msgr2=0x7f53c80c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.755+0000 7f53cd000640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80cc7f0 0x7f53c80c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.755+0000 7f53cd000640 1 -- 172.21.15.62:0/1295973381 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c81504f0 con 0x7f53b0003fe0 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.755+0000 7f53c7e00640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80cc7f0 0x7f53c80c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.755+0000 7f53cd000640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53b0003fe0 0x7f53c814ffb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f53bc001050 tx=0x7f53bc00ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.756+0000 7f53c4c00640 1 -- 172.21.15.62:0/1295973381 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc019070 con 0x7f53b0003fe0 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.756+0000 7f53c4c00640 1 -- 172.21.15.62:0/1295973381 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc00c870 con 0x7f53b0003fe0 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.756+0000 7f53c4c00640 1 -- 172.21.15.62:0/1295973381 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00cc40 con 0x7f53b0003fe0 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.756+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c80c8c80 con 0x7f53b0003fe0 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.756+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c80c9240 con 0x7f53b0003fe0 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.756+0000 7f53c4c00640 1 -- 172.21.15.62:0/1295973381 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f53bc01c070 con 0x7f53b0003fe0 2024-04-25T09:02:52.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.756+0000 7f53c4c00640 1 -- 172.21.15.62:0/1295973381 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f53bc01e780 con 0x7f53b0003fe0 2024-04-25T09:02:52.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.758+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c80d3390 con 0x7f53b0003fe0 2024-04-25T09:02:52.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.762+0000 7f53c4c00640 1 -- 172.21.15.62:0/1295973381 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f53bc023020 con 0x7f53b0003fe0 2024-04-25T09:02:52.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.838+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f53c80750e0 con 0x7f53b0003fe0 2024-04-25T09:02:52.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.850+0000 7f53c4c00640 1 -- 172.21.15.62:0/1295973381 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f53c81504f0 con 0x7f53b0003fe0 2024-04-25T09:02:52.839 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-04-25T09:02:52.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53b0003fe0 msgr2=0x7f53c814ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53b0003fe0 0x7f53c814ffb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f53bc001050 tx=0x7f53bc00ed20 comp rx=0 tx=0).stop 2024-04-25T09:02:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 shutdown_connections 2024-04-25T09:02:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53c80cc7f0 0x7f53c80c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53c80b53d0 0x7f53c80b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 --2- 172.21.15.62:0/1295973381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53b0003fe0 0x7f53c814ffb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 >> 172.21.15.62:0/1295973381 conn(0x7f53c806b210 msgr2=0x7f53c80cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 shutdown_connections 2024-04-25T09:02:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:52.851+0000 7f53ce400640 1 -- 172.21.15.62:0/1295973381 wait complete. 2024-04-25T09:02:52.859 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T09:02:52.859 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T09:02:52.860 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:52.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T09:02:52.893 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:52.904+0000 7f4807723040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T09:02:52.935 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:52.935 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T09:02:52.936 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:52.947+0000 7f73337b4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T09:02:53.056 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:53.056 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T09:02:53.087 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:53.098+0000 7f4807723040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:02:53.131 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:53.142+0000 7f73337b4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:02:53.136 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:02:53.136 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T09:02:53.151 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:53.162+0000 7f4807723040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T09:02:53.197 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:53.208+0000 7f73337b4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T09:02:53.219 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:53.219 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T09:02:53.297 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:53.297 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T09:02:53.364 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:53.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T09:02:53.448 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:53.459+0000 7f4807723040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T09:02:53.487 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:02:53.487 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T09:02:53.504 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:53.515+0000 7f4807723040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T09:02:53.515 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:53.526+0000 7f73337b4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T09:02:53.573 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:53.584+0000 7f73337b4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T09:02:53.594 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 7d078b39-cff6-45ae-8b9c-9224dd5edb73 0 2024-04-25T09:02:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.791+0000 7f34c1600640 1 Processor -- start 2024-04-25T09:02:53.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.791+0000 7f34c1600640 1 -- start start 2024-04-25T09:02:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c1600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f34bc15dd60 0x7f34bc0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c1600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc162300 0x7f34bc0cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c1600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc0d1c90 con 0x7f34bc0ce990 2024-04-25T09:02:53.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c1600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34bc0d1f70 con 0x7f34bc15dd60 2024-04-25T09:02:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c1600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0d2250 con 0x7f34bc162300 2024-04-25T09:02:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c0c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc162300 0x7f34bc0cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c0c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc162300 0x7f34bc0cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:53992/0 (socket says 172.21.15.121:53992) 2024-04-25T09:02:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c0c00640 1 -- 172.21.15.121:0/492691144 learned_addr learned my addr 172.21.15.121:0/492691144 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c0c00640 1 -- 172.21.15.121:0/492691144 >> v2:172.21.15.62:3300/0 conn(0x7f34bc15dd60 msgr2=0x7f34bc0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c0c00640 1 --2- 172.21.15.121:0/492691144 >> v2:172.21.15.62:3300/0 conn(0x7f34bc15dd60 0x7f34bc0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c0c00640 1 -- 172.21.15.121:0/492691144 >> v1:172.21.15.62:6789/0 conn(0x7f34bc0ce990 legacy=0x7f34bc161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:53.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.793+0000 7f34c0c00640 1 -- 172.21.15.121:0/492691144 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc0d2530 con 0x7f34bc162300 2024-04-25T09:02:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.794+0000 7f34c0c00640 1 --2- 172.21.15.121:0/492691144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc162300 0x7f34bc0cee20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009e50 tx=0x7f34b402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198e8eedd65976aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.795+0000 7f34baa00640 1 -- 172.21.15.121:0/492691144 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4036070 con 0x7f34bc162300 2024-04-25T09:02:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.795+0000 7f34baa00640 1 -- 172.21.15.121:0/492691144 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b4031600 con 0x7f34bc162300 2024-04-25T09:02:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.795+0000 7f34baa00640 1 -- 172.21.15.121:0/492691144 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b40319d0 con 0x7f34bc162300 2024-04-25T09:02:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.795+0000 7f34c1600640 1 -- 172.21.15.121:0/492691144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc162300 msgr2=0x7f34bc0cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.795+0000 7f34c1600640 1 --2- 172.21.15.121:0/492691144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc162300 0x7f34bc0cee20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f34b4009e50 tx=0x7f34b402a6d0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.796+0000 7f34c1600640 1 -- 172.21.15.121:0/492691144 shutdown_connections 2024-04-25T09:02:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.796+0000 7f34c1600640 1 --2- 172.21.15.121:0/492691144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc162300 0x7f34bc0cee20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.796+0000 7f34c1600640 1 --2- 172.21.15.121:0/492691144 >> v2:172.21.15.62:3300/0 conn(0x7f34bc15dd60 0x7f34bc0d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.796+0000 7f34c1600640 1 -- 172.21.15.121:0/492691144 >> 172.21.15.121:0/492691144 conn(0x7f34bc06b210 msgr2=0x7f34bc0668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.796+0000 7f34c1600640 1 -- 172.21.15.121:0/492691144 shutdown_connections 2024-04-25T09:02:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.796+0000 7f34c1600640 1 -- 172.21.15.121:0/492691144 wait complete. 2024-04-25T09:02:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.797+0000 7f34c1600640 1 Processor -- start 2024-04-25T09:02:53.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.797+0000 7f34c1600640 1 -- start start 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.798+0000 7f34c1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc0ce990 0x7f34bc0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.798+0000 7f34c1600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc15dd60 0x7f34bc0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.798+0000 7f34c1600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34bc162300 0x7f34bc14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.798+0000 7f34c1600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0b6dd0 con 0x7f34bc0ce990 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.798+0000 7f34c1600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc15e140 con 0x7f34bc162300 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.798+0000 7f34c1600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0b6f50 con 0x7f34bc15dd60 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34bb400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc15dd60 0x7f34bc0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34bc162300 0x7f34bc14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34bc162300 0x7f34bc14f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46300/0 (socket says 172.21.15.121:46300) 2024-04-25T09:02:53.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 -- 172.21.15.121:0/1679096694 learned_addr learned my addr 172.21.15.121:0/1679096694 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34bb400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc15dd60 0x7f34bc0b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:54000/0 (socket says 172.21.15.121:54000) 2024-04-25T09:02:53.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34bbe00640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc0ce990 0x7f34bc0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 -- 172.21.15.121:0/1679096694 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc15dd60 msgr2=0x7f34bc0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc15dd60 0x7f34bc0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 -- 172.21.15.121:0/1679096694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc0ce990 msgr2=0x7f34bc0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc0ce990 0x7f34bc0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 -- 172.21.15.121:0/1679096694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc158740 con 0x7f34bc162300 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34bb400640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc15dd60 0x7f34bc0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.799+0000 7f34c0c00640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34bc162300 0x7f34bc14f530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f34b4034040 tx=0x7f34b4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.800+0000 7f34b3e00640 1 -- 172.21.15.121:0/1679096694 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4041070 con 0x7f34bc162300 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.800+0000 7f34b3e00640 1 -- 172.21.15.121:0/1679096694 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b403c870 con 0x7f34bc162300 2024-04-25T09:02:53.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.800+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34bc158a20 con 0x7f34bc162300 2024-04-25T09:02:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.800+0000 7f34b3e00640 1 -- 172.21.15.121:0/1679096694 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4036040 con 0x7f34bc162300 2024-04-25T09:02:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.800+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc158fe0 con 0x7f34bc162300 2024-04-25T09:02:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.800+0000 7f34b3e00640 1 -- 172.21.15.121:0/1679096694 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f34b4044070 con 0x7f34bc162300 2024-04-25T09:02:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.800+0000 7f34b3e00640 1 -- 172.21.15.121:0/1679096694 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f34b4046c60 con 0x7f34bc162300 2024-04-25T09:02:53.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.801+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34bc0b62a0 con 0x7f34bc162300 2024-04-25T09:02:53.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.804+0000 7f34b3e00640 1 -- 172.21.15.121:0/1679096694 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f34bc0b62a0 con 0x7f34bc162300 2024-04-25T09:02:53.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.882+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7d078b39-cff6-45ae-8b9c-9224dd5edb73", "id": 0} v 0) v1 -- 0x7f34bc0750e0 con 0x7f34bc162300 2024-04-25T09:02:53.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.890+0000 7f34b3e00640 1 -- 172.21.15.121:0/1679096694 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d078b39-cff6-45ae-8b9c-9224dd5edb73", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f34b40314c0 con 0x7f34bc162300 2024-04-25T09:02:53.879 INFO:teuthology.orchestra.run.smithi121.stdout:0 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34bc162300 msgr2=0x7f34bc14f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34bc162300 0x7f34bc14f530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f34b4034040 tx=0x7f34b4004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 shutdown_connections 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34bc162300 0x7f34bc14f530 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f34bc15dd60 0x7f34bc0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 --2- 172.21.15.121:0/1679096694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34bc0ce990 0x7f34bc0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 >> 172.21.15.121:0/1679096694 conn(0x7f34bc06b210 msgr2=0x7f34bc0d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 shutdown_connections 2024-04-25T09:02:53.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:53.892+0000 7f34c1600640 1 -- 172.21.15.121:0/1679096694 wait complete. 2024-04-25T09:02:53.896 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 232e7cd0-6bc1-4732-9f18-6cd44ea4338b 1 2024-04-25T09:02:54.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.082+0000 7fa268400640 1 Processor -- start 2024-04-25T09:02:54.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.082+0000 7fa268400640 1 -- start start 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa268400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2600b97b0 0x7fa2600b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa268400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa268400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2600d2270 con 0x7fa26015dc10 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa268400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2600b81a0 con 0x7fa2600b97b0 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa268400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600b7590 con 0x7fa2600b70d0 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa265000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2600b97b0 0x7fa2600b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa266400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa265000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2600b97b0 0x7fa2600b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46308/0 (socket says 172.21.15.121:46308) 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.084+0000 7fa265000640 1 -- 172.21.15.121:0/1320060984 learned_addr learned my addr 172.21.15.121:0/1320060984 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.085+0000 7fa265000640 1 -- 172.21.15.121:0/1320060984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 msgr2=0x7fa2600b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.085+0000 7fa265000640 1 --2- 172.21.15.121:0/1320060984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.085+0000 7fa265000640 1 -- 172.21.15.121:0/1320060984 >> v1:172.21.15.62:6789/0 conn(0x7fa26015dc10 legacy=0x7fa2600d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.085+0000 7fa265000640 1 -- 172.21.15.121:0/1320060984 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600b7870 con 0x7fa2600b97b0 2024-04-25T09:02:54.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.086+0000 7fa265000640 1 --2- 172.21.15.121:0/1320060984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa2600b6b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009870 tx=0x7fa24c02a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7fcadfd7c3afcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.086+0000 7fa25fe00640 1 -- 172.21.15.121:0/1320060984 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c037070 con 0x7fa2600b97b0 2024-04-25T09:02:54.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.086+0000 7fa25fe00640 1 -- 172.21.15.121:0/1320060984 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c032440 con 0x7fa2600b97b0 2024-04-25T09:02:54.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.086+0000 7fa25fe00640 1 -- 172.21.15.121:0/1320060984 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c032810 con 0x7fa2600b97b0 2024-04-25T09:02:54.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.086+0000 7fa268400640 1 -- 172.21.15.121:0/1320060984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 msgr2=0x7fa2600b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.086+0000 7fa268400640 1 --2- 172.21.15.121:0/1320060984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa2600b6b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009870 tx=0x7fa24c02a2a0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.087+0000 7fa268400640 1 -- 172.21.15.121:0/1320060984 shutdown_connections 2024-04-25T09:02:54.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.087+0000 7fa268400640 1 --2- 172.21.15.121:0/1320060984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.087+0000 7fa268400640 1 --2- 172.21.15.121:0/1320060984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa2600b6b90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.087+0000 7fa268400640 1 -- 172.21.15.121:0/1320060984 >> 172.21.15.121:0/1320060984 conn(0x7fa26006b000 msgr2=0x7fa26006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.087+0000 7fa268400640 1 -- 172.21.15.121:0/1320060984 shutdown_connections 2024-04-25T09:02:54.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.087+0000 7fa268400640 1 -- 172.21.15.121:0/1320060984 wait complete. 2024-04-25T09:02:54.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.088+0000 7fa268400640 1 Processor -- start 2024-04-25T09:02:54.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.089+0000 7fa268400640 1 -- start start 2024-04-25T09:02:54.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.089+0000 7fa268400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.089+0000 7fa268400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa26014d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.089+0000 7fa268400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26015dc10 0x7fa2601e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.089+0000 7fa268400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26015dff0 con 0x7fa26015dc10 2024-04-25T09:02:54.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.089+0000 7fa268400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600b7e30 con 0x7fa2600b97b0 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.089+0000 7fa268400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600b8b70 con 0x7fa2600b70d0 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa26014d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa26014d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46310/0 (socket says 172.21.15.121:46310) 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 -- 172.21.15.121:0/1189572994 learned_addr learned my addr 172.21.15.121:0/1189572994 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:54016/0 (socket says 172.21.15.121:54016) 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa266400640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26015dc10 0x7fa2601e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 -- 172.21.15.121:0/1189572994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 msgr2=0x7fa2600cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 -- 172.21.15.121:0/1189572994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26015dc10 msgr2=0x7fa2601e60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26015dc10 0x7fa2601e60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 -- 172.21.15.121:0/1189572994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2601e6830 con 0x7fa2600b97b0 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa266400640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26015dc10 0x7fa2601e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:54.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265a00640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.090+0000 7fa265000640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa26014d300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009870 tx=0x7fa24c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.091+0000 7fa25d600640 1 -- 172.21.15.121:0/1189572994 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c041020 con 0x7fa2600b97b0 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.091+0000 7fa25d600640 1 -- 172.21.15.121:0/1189572994 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c030640 con 0x7fa2600b97b0 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.091+0000 7fa25d600640 1 -- 172.21.15.121:0/1189572994 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c037040 con 0x7fa2600b97b0 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.091+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2601e6b10 con 0x7fa2600b97b0 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.091+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2601e70d0 con 0x7fa2600b97b0 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.091+0000 7fa25d600640 1 -- 172.21.15.121:0/1189572994 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa24c044020 con 0x7fa2600b97b0 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.091+0000 7fa25d600640 1 -- 172.21.15.121:0/1189572994 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa24c03dbf0 con 0x7fa2600b97b0 2024-04-25T09:02:54.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.092+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26004ef20 con 0x7fa2600b97b0 2024-04-25T09:02:54.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.094+0000 7fa25d600640 1 -- 172.21.15.121:0/1189572994 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa24c04b030 con 0x7fa2600b97b0 2024-04-25T09:02:54.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.170+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "232e7cd0-6bc1-4732-9f18-6cd44ea4338b", "id": 1} v 0) v1 -- 0x7fa2600c9f80 con 0x7fa2600b97b0 2024-04-25T09:02:54.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.178+0000 7fa25d600640 1 -- 172.21.15.121:0/1189572994 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "232e7cd0-6bc1-4732-9f18-6cd44ea4338b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa24c03ddf0 con 0x7fa2600b97b0 2024-04-25T09:02:54.166 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-25T09:02:54.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.179+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 msgr2=0x7fa26014d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.179+0000 7fa268400640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa26014d300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009870 tx=0x7fa24c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.179+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 shutdown_connections 2024-04-25T09:02:54.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.179+0000 7fa268400640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa26015dc10 0x7fa2601e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.179+0000 7fa268400640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2600b97b0 0x7fa26014d300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.179+0000 7fa268400640 1 --2- 172.21.15.121:0/1189572994 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2600b70d0 0x7fa2600cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.179+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 >> 172.21.15.121:0/1189572994 conn(0x7fa26006b000 msgr2=0x7fa2600d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.180+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 shutdown_connections 2024-04-25T09:02:54.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.180+0000 7fa268400640 1 -- 172.21.15.121:0/1189572994 wait complete. 2024-04-25T09:02:54.184 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 34dcda9a-65b6-4445-b6cb-489bdb58b737 2 2024-04-25T09:02:54.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.363+0000 7f0766600640 1 Processor -- start 2024-04-25T09:02:54.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.363+0000 7f0766600640 1 -- start start 2024-04-25T09:02:54.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f0766600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07600afef0 0x7f07600721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f0766600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f07600668f0 0x7f0760066da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f0766600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07600b3930 con 0x7f076015db60 2024-04-25T09:02:54.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f0766600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07600d1350 con 0x7f07600668f0 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f075f400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f076015db60 0x7f07600b57d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:32994/0 (socket says 172.21.15.121:32994) 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f075f400640 1 -- 172.21.15.121:0/281472847 learned_addr learned my addr 172.21.15.121:0/281472847 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f075ea00640 1 --2- 172.21.15.121:0/281472847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07600afef0 0x7f07600721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f075fe00640 1 --2- 172.21.15.121:0/281472847 >> v2:172.21.15.121:3300/0 conn(0x7f07600668f0 0x7f0760066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.365+0000 7f0766600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07600b3bd0 con 0x7f07600afef0 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.366+0000 7f075ea00640 1 -- 172.21.15.121:0/281472847 >> v2:172.21.15.121:3300/0 conn(0x7f07600668f0 msgr2=0x7f0760066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.366+0000 7f075ea00640 1 --2- 172.21.15.121:0/281472847 >> v2:172.21.15.121:3300/0 conn(0x7f07600668f0 0x7f0760066da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.366+0000 7f075ea00640 1 -- 172.21.15.121:0/281472847 >> v1:172.21.15.121:6789/0 conn(0x7f076015db60 legacy=0x7f07600b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.366+0000 7f075ea00640 1 -- 172.21.15.121:0/281472847 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600b3eb0 con 0x7f07600afef0 2024-04-25T09:02:54.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.366+0000 7f075fe00640 1 --2- 172.21.15.121:0/281472847 >> v2:172.21.15.121:3300/0 conn(0x7f07600668f0 0x7f0760066da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:54.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.367+0000 7f075ea00640 1 --2- 172.21.15.121:0/281472847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07600afef0 0x7f07600721f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f074c0097a0 tx=0x7f074c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d604ef6163ef82b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.367+0000 7f075e000640 1 -- 172.21.15.121:0/281472847 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c037070 con 0x7f07600afef0 2024-04-25T09:02:54.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.367+0000 7f075e000640 1 -- 172.21.15.121:0/281472847 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c032440 con 0x7f07600afef0 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.367+0000 7f075e000640 1 -- 172.21.15.121:0/281472847 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c032810 con 0x7f07600afef0 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.368+0000 7f0766600640 1 -- 172.21.15.121:0/281472847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07600afef0 msgr2=0x7f07600721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.368+0000 7f0766600640 1 --2- 172.21.15.121:0/281472847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07600afef0 0x7f07600721f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f074c0097a0 tx=0x7f074c02a180 comp rx=0 tx=0).stop 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.368+0000 7f0766600640 1 -- 172.21.15.121:0/281472847 shutdown_connections 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.368+0000 7f0766600640 1 --2- 172.21.15.121:0/281472847 >> v2:172.21.15.121:3300/0 conn(0x7f07600668f0 0x7f0760066da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.368+0000 7f0766600640 1 --2- 172.21.15.121:0/281472847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07600afef0 0x7f07600721f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.368+0000 7f0766600640 1 -- 172.21.15.121:0/281472847 >> 172.21.15.121:0/281472847 conn(0x7f076006abe0 msgr2=0x7f076006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.368+0000 7f0766600640 1 -- 172.21.15.121:0/281472847 shutdown_connections 2024-04-25T09:02:54.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.368+0000 7f0766600640 1 -- 172.21.15.121:0/281472847 wait complete. 2024-04-25T09:02:54.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.370+0000 7f0766600640 1 Processor -- start 2024-04-25T09:02:54.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.370+0000 7f0766600640 1 -- start start 2024-04-25T09:02:54.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f0766600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07600668f0 0x7f07601579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f0766600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07600afef0 0x7f0760146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f0766600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076015db60 0x7f07601470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f0766600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076015e0c0 con 0x7f07600afef0 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f0766600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076015df40 con 0x7f07600668f0 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f0766600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07600d0f70 con 0x7f076015db60 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07600668f0 0x7f07601579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07600668f0 0x7f07601579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46324/0 (socket says 172.21.15.121:46324) 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075f400640 1 -- 172.21.15.121:0/1054377901 learned_addr learned my addr 172.21.15.121:0/1054377901 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075fe00640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076015db60 0x7f07601470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075ea00640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07600afef0 0x7f0760146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075fe00640 1 -- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07600668f0 msgr2=0x7f07601579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075fe00640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07600668f0 0x7f07601579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075fe00640 1 -- 172.21.15.121:0/1054377901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07600afef0 msgr2=0x7f0760146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075fe00640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07600afef0 0x7f0760146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075fe00640 1 -- 172.21.15.121:0/1054377901 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0760145b60 con 0x7f076015db60 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.371+0000 7f075f400640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07600668f0 0x7f07601579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.372+0000 7f075fe00640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076015db60 0x7f07601470f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f075800efc0 tx=0x7f075800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.372+0000 7f0757400640 1 -- 172.21.15.121:0/1054377901 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758018030 con 0x7f076015db60 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.372+0000 7f0757400640 1 -- 172.21.15.121:0/1054377901 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075800cd30 con 0x7f076015db60 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.372+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0760145e40 con 0x7f076015db60 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.372+0000 7f0757400640 1 -- 172.21.15.121:0/1054377901 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758003bb0 con 0x7f076015db60 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.372+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0760146400 con 0x7f076015db60 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.372+0000 7f0757400640 1 -- 172.21.15.121:0/1054377901 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f075801b070 con 0x7f076015db60 2024-04-25T09:02:54.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.372+0000 7f0757400640 1 -- 172.21.15.121:0/1054377901 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f07580148a0 con 0x7f076015db60 2024-04-25T09:02:54.363 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.373+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076004ef20 con 0x7f076015db60 2024-04-25T09:02:54.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.376+0000 7f0757400640 1 -- 172.21.15.121:0/1054377901 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0758010040 con 0x7f076015db60 2024-04-25T09:02:54.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.454+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd new", "uuid": "34dcda9a-65b6-4445-b6cb-489bdb58b737", "id": 2} v 0) v1 -- 0x7f0760071ac0 con 0x7f076015db60 2024-04-25T09:02:54.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.462+0000 7f0757400640 1 -- 172.21.15.121:0/1054377901 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "34dcda9a-65b6-4445-b6cb-489bdb58b737", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0758014b80 con 0x7f076015db60 2024-04-25T09:02:54.451 INFO:teuthology.orchestra.run.smithi121.stdout:2 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076015db60 msgr2=0x7f07601470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076015db60 0x7f07601470f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f075800efc0 tx=0x7f075800c490 comp rx=0 tx=0).stop 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 shutdown_connections 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07600afef0 0x7f0760146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076015db60 0x7f07601470f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 --2- 172.21.15.121:0/1054377901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07600668f0 0x7f07601579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 >> 172.21.15.121:0/1054377901 conn(0x7f076006abe0 msgr2=0x7f07600b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 shutdown_connections 2024-04-25T09:02:54.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.464+0000 7f0766600640 1 -- 172.21.15.121:0/1054377901 wait complete. 2024-04-25T09:02:54.468 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 14caaf78-6122-44ff-9def-2f1db5cd9eb4 3 2024-04-25T09:02:54.517 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:54.527+0000 7f4807723040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T09:02:54.517 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:54.529+0000 7f73337b4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T09:02:54.627 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:54.638+0000 7f73337b4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T09:02:54.627 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:54.638+0000 7f4807723040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T09:02:54.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.653+0000 7f773fe00640 1 Processor -- start 2024-04-25T09:02:54.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.654+0000 7f773fe00640 1 -- start start 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.655+0000 7f773fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7740162300 0x7f77400cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.655+0000 7f773fe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77400d1c90 con 0x7f774015dd60 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.655+0000 7f773fe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77400d1f70 con 0x7f77400ce990 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.655+0000 7f773fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77400d2250 con 0x7f7740162300 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.655+0000 7f773e000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f77400ce990 0x7f7740161bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33008/0 (socket says 172.21.15.121:33008) 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.655+0000 7f773e000640 1 -- 172.21.15.121:0/1130628620 learned_addr learned my addr 172.21.15.121:0/1130628620 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773f400640 1 --2- 172.21.15.121:0/1130628620 >> v2:172.21.15.62:3300/0 conn(0x7f7740162300 0x7f77400cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588067912 0 0) 0x7f77400d1f70 con 0x7f77400ce990 2024-04-25T09:02:54.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7724003520 con 0x7f77400ce990 2024-04-25T09:02:54.646 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970837202 0 0) 0x7f7724003520 con 0x7f77400ce990 2024-04-25T09:02:54.646 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 >> v2:172.21.15.62:3300/0 conn(0x7f7740162300 msgr2=0x7f77400cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.646 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 --2- 172.21.15.121:0/1130628620 >> v2:172.21.15.62:3300/0 conn(0x7f7740162300 0x7f77400cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.646 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 >> v1:172.21.15.62:6789/0 conn(0x7f774015dd60 legacy=0x7f77400d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.646 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77400d2530 con 0x7f77400ce990 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 <== mon.1 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f772c001670 con 0x7f77400ce990 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 >> v1:172.21.15.121:6789/0 conn(0x7f77400ce990 legacy=0x7f7740161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77240039b0 0x7f7724003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7724004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77240051c0 0x7f772400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400d2250 con 0x7f77240039b0 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772400c160 con 0x7f77240051c0 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.656+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772400c360 con 0x7f7724004800 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773e000640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7724004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773f400640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77240051c0 0x7f772400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773ea00640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77240039b0 0x7f7724003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773e000640 1 -- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77240051c0 msgr2=0x7f772400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773e000640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77240051c0 0x7f772400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773e000640 1 -- 172.21.15.121:0/1130628620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77240039b0 msgr2=0x7f7724003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773e000640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77240039b0 0x7f7724003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773e000640 1 -- 172.21.15.121:0/1130628620 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724003520 con 0x7f7724004800 2024-04-25T09:02:54.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773f400640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77240051c0 0x7f772400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:54.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.657+0000 7f773f400640 1 -- 172.21.15.121:0/1130628620 reap_dead start 2024-04-25T09:02:54.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.658+0000 7f773e000640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7724004c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c210 tx=0x7f7724005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2fa59315804e196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.658+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c007d30 con 0x7f7724004800 2024-04-25T09:02:54.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.658+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c005af0 con 0x7f7724004800 2024-04-25T09:02:54.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.658+0000 7f773d600640 1 -- 172.21.15.121:0/1130628620 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c007070 con 0x7f7724004800 2024-04-25T09:02:54.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.659+0000 7f773fe00640 1 -- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 msgr2=0x7f7724004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.659+0000 7f773fe00640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7724004c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c210 tx=0x7f7724005830 comp rx=0 tx=0).stop 2024-04-25T09:02:54.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.659+0000 7f773fe00640 1 -- 172.21.15.121:0/1130628620 shutdown_connections 2024-04-25T09:02:54.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.659+0000 7f773fe00640 1 --2- 172.21.15.121:0/1130628620 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7724004c80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.659+0000 7f773fe00640 1 -- 172.21.15.121:0/1130628620 >> 172.21.15.121:0/1130628620 conn(0x7f774006b210 msgr2=0x7f77400668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.659+0000 7f773fe00640 1 -- 172.21.15.121:0/1130628620 shutdown_connections 2024-04-25T09:02:54.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.659+0000 7f773fe00640 1 -- 172.21.15.121:0/1130628620 wait complete. 2024-04-25T09:02:54.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.661+0000 7f773fe00640 1 Processor -- start 2024-04-25T09:02:54.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.661+0000 7f773fe00640 1 -- start start 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7740151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774015e350 0x7f77400c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f774015e7d0 0x7f77400c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400cefa0 con 0x7f774015e7d0 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400cee20 con 0x7f774015e350 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77400cf120 con 0x7f7724004800 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774015e350 0x7f77400c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7740151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774015e350 0x7f77400c79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46342/0 (socket says 172.21.15.121:46342) 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773e000640 1 -- 172.21.15.121:0/2548666130 learned_addr learned my addr 172.21.15.121:0/2548666130 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:54.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.662+0000 7f773ea00640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7740151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:54052/0 (socket says 172.21.15.121:54052) 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773f400640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f774015e7d0 0x7f77400c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773e000640 1 -- 172.21.15.121:0/2548666130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 msgr2=0x7f7740151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773e000640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7740151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773e000640 1 -- 172.21.15.121:0/2548666130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f774015e7d0 msgr2=0x7f77400c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773e000640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f774015e7d0 0x7f77400c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773e000640 1 -- 172.21.15.121:0/2548666130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7740161010 con 0x7f774015e350 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773e000640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774015e350 0x7f77400c79b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c210 tx=0x7f772c001a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f7737400640 1 -- 172.21.15.121:0/2548666130 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c041020 con 0x7f774015e350 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f7737400640 1 -- 172.21.15.121:0/2548666130 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c038040 con 0x7f774015e350 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f7737400640 1 -- 172.21.15.121:0/2548666130 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c007070 con 0x7f774015e350 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77401612f0 con 0x7f774015e350 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.663+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77400b3e60 con 0x7f774015e350 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.664+0000 7f7737400640 1 -- 172.21.15.121:0/2548666130 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f772c044020 con 0x7f774015e350 2024-04-25T09:02:54.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.664+0000 7f7737400640 1 -- 172.21.15.121:0/2548666130 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f772c005aa0 con 0x7f774015e350 2024-04-25T09:02:54.654 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.665+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77400750e0 con 0x7f774015e350 2024-04-25T09:02:54.655 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.667+0000 7f7737400640 1 -- 172.21.15.121:0/2548666130 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f772c0075a0 con 0x7f774015e350 2024-04-25T09:02:54.681 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:54.692+0000 7f4807723040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T09:02:54.700 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:54.711+0000 7f73337b4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T09:02:54.759 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.770+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "14caaf78-6122-44ff-9def-2f1db5cd9eb4", "id": 3} v 0) v1 -- 0x7f77400d18f0 con 0x7f774015e350 2024-04-25T09:02:54.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.777+0000 7f7737400640 1 -- 172.21.15.121:0/2548666130 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "14caaf78-6122-44ff-9def-2f1db5cd9eb4", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f772c032b60 con 0x7f774015e350 2024-04-25T09:02:54.766 INFO:teuthology.orchestra.run.smithi121.stdout:3 2024-04-25T09:02:54.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774015e350 msgr2=0x7f77400c79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774015e350 0x7f77400c79b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c210 tx=0x7f772c001a40 comp rx=0 tx=0).stop 2024-04-25T09:02:54.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 shutdown_connections 2024-04-25T09:02:54.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f774015e7d0 0x7f77400c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774015e350 0x7f77400c79b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 --2- 172.21.15.121:0/2548666130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7724004800 0x7f7740151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 >> 172.21.15.121:0/2548666130 conn(0x7f774006b210 msgr2=0x7f77400caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 shutdown_connections 2024-04-25T09:02:54.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.779+0000 7f773fe00640 1 -- 172.21.15.121:0/2548666130 wait complete. 2024-04-25T09:02:54.784 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 73c23f2f-c64e-4c72-8c29-3aece151a25f 4 2024-04-25T09:02:54.884 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:54.894+0000 7f4807723040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T09:02:54.918 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:54.929+0000 7f73337b4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T09:02:54.949 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:54.960+0000 7f4807723040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T09:02:54.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.968+0000 7f6a27000640 1 Processor -- start 2024-04-25T09:02:54.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.968+0000 7f6a27000640 1 -- start start 2024-04-25T09:02:54.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.969+0000 7f6a27000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6a200ce990 0x7f6a200cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.969+0000 7f6a27000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6a20161bf0 0x7f6a20162060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.969+0000 7f6a27000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a200d1c90 con 0x7f6a2015dd60 2024-04-25T09:02:54.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.969+0000 7f6a27000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a200d1f70 con 0x7f6a20161bf0 2024-04-25T09:02:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.969+0000 7f6a27000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a200d2250 con 0x7f6a200ce990 2024-04-25T09:02:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.970+0000 7f6a25200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6a200ce990 0x7f6a200cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.970+0000 7f6a25200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6a200ce990 0x7f6a200cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46358/0 (socket says 172.21.15.121:46358) 2024-04-25T09:02:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.970+0000 7f6a25200640 1 -- 172.21.15.121:0/3393448823 learned_addr learned my addr 172.21.15.121:0/3393448823 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.970+0000 7f6a26600640 1 --2- 172.21.15.121:0/3393448823 >> v2:172.21.15.62:3300/0 conn(0x7f6a20161bf0 0x7f6a20162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.970+0000 7f6a25200640 1 -- 172.21.15.121:0/3393448823 >> v2:172.21.15.62:3300/0 conn(0x7f6a20161bf0 msgr2=0x7f6a20162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.970+0000 7f6a25200640 1 --2- 172.21.15.121:0/3393448823 >> v2:172.21.15.62:3300/0 conn(0x7f6a20161bf0 0x7f6a20162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.970+0000 7f6a25200640 1 -- 172.21.15.121:0/3393448823 >> v1:172.21.15.121:6789/0 conn(0x7f6a2015dd60 legacy=0x7f6a200d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.970+0000 7f6a25200640 1 -- 172.21.15.121:0/3393448823 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200d2530 con 0x7f6a200ce990 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.971+0000 7f6a25200640 1 --2- 172.21.15.121:0/3393448823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200cee20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6a200cffb0 tx=0x7f6a0c02a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99620cf702fc8d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.971+0000 7f6a1fe00640 1 -- 172.21.15.121:0/3393448823 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c037070 con 0x7f6a200ce990 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.971+0000 7f6a1fe00640 1 -- 172.21.15.121:0/3393448823 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c032440 con 0x7f6a200ce990 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.971+0000 7f6a1fe00640 1 -- 172.21.15.121:0/3393448823 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c032810 con 0x7f6a200ce990 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.972+0000 7f6a27000640 1 -- 172.21.15.121:0/3393448823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 msgr2=0x7f6a200cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.972+0000 7f6a27000640 1 --2- 172.21.15.121:0/3393448823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200cee20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6a200cffb0 tx=0x7f6a0c02a1f0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.972+0000 7f6a27000640 1 -- 172.21.15.121:0/3393448823 shutdown_connections 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.972+0000 7f6a27000640 1 --2- 172.21.15.121:0/3393448823 >> v2:172.21.15.62:3300/0 conn(0x7f6a20161bf0 0x7f6a20162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.972+0000 7f6a27000640 1 --2- 172.21.15.121:0/3393448823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200cee20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.972+0000 7f6a27000640 1 -- 172.21.15.121:0/3393448823 >> 172.21.15.121:0/3393448823 conn(0x7f6a2006b210 msgr2=0x7f6a200668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.972+0000 7f6a27000640 1 -- 172.21.15.121:0/3393448823 shutdown_connections 2024-04-25T09:02:54.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.972+0000 7f6a27000640 1 -- 172.21.15.121:0/3393448823 wait complete. 2024-04-25T09:02:54.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.974+0000 7f6a27000640 1 Processor -- start 2024-04-25T09:02:54.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.974+0000 7f6a27000640 1 -- start start 2024-04-25T09:02:54.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a27000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a27000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2015dd60 0x7f6a200cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a27000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a20161bf0 0x7f6a200b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a27000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a20162720 con 0x7f6a2015dd60 2024-04-25T09:02:54.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a27000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a201625a0 con 0x7f6a200ce990 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a27000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a200b6dd0 con 0x7f6a20161bf0 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46372/0 (socket says 172.21.15.121:46372) 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25c00640 1 -- 172.21.15.121:0/3697192350 learned_addr learned my addr 172.21.15.121:0/3697192350 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a26600640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a20161bf0 0x7f6a200b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25200640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2015dd60 0x7f6a200cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25c00640 1 -- 172.21.15.121:0/3697192350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a20161bf0 msgr2=0x7f6a200b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25c00640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a20161bf0 0x7f6a200b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25c00640 1 -- 172.21.15.121:0/3697192350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2015dd60 msgr2=0x7f6a200cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25c00640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2015dd60 0x7f6a200cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.975+0000 7f6a25c00640 1 -- 172.21.15.121:0/3697192350 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200c7500 con 0x7f6a200ce990 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.976+0000 7f6a25c00640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200ccc80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c430 tx=0x7f6a1400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.976+0000 7f6a1d600640 1 -- 172.21.15.121:0/3697192350 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a140052d0 con 0x7f6a200ce990 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.976+0000 7f6a1d600640 1 -- 172.21.15.121:0/3697192350 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a14005490 con 0x7f6a200ce990 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.976+0000 7f6a1d600640 1 -- 172.21.15.121:0/3697192350 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14007260 con 0x7f6a200ce990 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.976+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a200c77e0 con 0x7f6a200ce990 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.976+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a200c7da0 con 0x7f6a200ce990 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.976+0000 7f6a1d600640 1 -- 172.21.15.121:0/3697192350 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6a14007400 con 0x7f6a200ce990 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.976+0000 7f6a1d600640 1 -- 172.21.15.121:0/3697192350 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f6a140035f0 con 0x7f6a200ce990 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.977+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a200750e0 con 0x7f6a200ce990 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:54.979+0000 7f6a1d600640 1 -- 172.21.15.121:0/3697192350 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6a14003a50 con 0x7f6a200ce990 2024-04-25T09:02:54.987 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:54.999+0000 7f73337b4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T09:02:55.013 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:55.024+0000 7f4807723040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T09:02:55.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.058+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "73c23f2f-c64e-4c72-8c29-3aece151a25f", "id": 4} v 0) v1 -- 0x7f6a200d18f0 con 0x7f6a200ce990 2024-04-25T09:02:55.050 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:55.062+0000 7f73337b4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T09:02:55.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.065+0000 7f6a1d600640 1 -- 172.21.15.121:0/3697192350 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73c23f2f-c64e-4c72-8c29-3aece151a25f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6a14003c30 con 0x7f6a200ce990 2024-04-25T09:02:55.053 INFO:teuthology.orchestra.run.smithi121.stdout:4 2024-04-25T09:02:55.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 msgr2=0x7f6a200ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200ccc80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400c430 tx=0x7f6a1400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 shutdown_connections 2024-04-25T09:02:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a2015dd60 0x7f6a200cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a20161bf0 0x7f6a200b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 --2- 172.21.15.121:0/3697192350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a200ce990 0x7f6a200ccc80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 >> 172.21.15.121:0/3697192350 conn(0x7f6a2006b210 msgr2=0x7f6a200cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 shutdown_connections 2024-04-25T09:02:55.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.066+0000 7f6a27000640 1 -- 172.21.15.121:0/3697192350 wait complete. 2024-04-25T09:02:55.071 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 28077425-23cd-49d3-84b5-e383d0aa1528 5 2024-04-25T09:02:55.147 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:55.157+0000 7f4807723040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:02:55.186 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:55.197+0000 7f73337b4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:02:55.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.253+0000 7f35d8c00640 1 Processor -- start 2024-04-25T09:02:55.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.254+0000 7f35d8c00640 1 -- start start 2024-04-25T09:02:55.244 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.255+0000 7f35d8c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35d415dd60 0x7f35d40d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.244 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.255+0000 7f35d8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d40ce990 0x7f35d40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.255+0000 7f35d8c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f35d4161bf0 0x7f35d4162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d8c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40cf330 con 0x7f35d4161bf0 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d8c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40d1c90 con 0x7f35d415dd60 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40d1f70 con 0x7f35d40ce990 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35d415dd60 0x7f35d40d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d40ce990 0x7f35d40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35d415dd60 0x7f35d40d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46386/0 (socket says 172.21.15.121:46386) 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3400640 1 -- 172.21.15.121:0/501241628 learned_addr learned my addr 172.21.15.121:0/501241628 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3e00640 1 --2- 172.21.15.121:0/501241628 >> v2:172.21.15.62:3300/0 conn(0x7f35d4161bf0 0x7f35d4162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3400640 1 -- 172.21.15.121:0/501241628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d40ce990 msgr2=0x7f35d40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3400640 1 --2- 172.21.15.121:0/501241628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d40ce990 0x7f35d40cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3400640 1 -- 172.21.15.121:0/501241628 >> v2:172.21.15.62:3300/0 conn(0x7f35d4161bf0 msgr2=0x7f35d4162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3400640 1 --2- 172.21.15.121:0/501241628 >> v2:172.21.15.62:3300/0 conn(0x7f35d4161bf0 0x7f35d4162080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.256+0000 7f35d3400640 1 -- 172.21.15.121:0/501241628 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40d2250 con 0x7f35d415dd60 2024-04-25T09:02:55.246 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.257+0000 7f35d3400640 1 --2- 172.21.15.121:0/501241628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d40d2a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f35d4067800 tx=0x7f35c402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f4b647ea05ea12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.246 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.257+0000 7f35d2a00640 1 -- 172.21.15.121:0/501241628 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4037070 con 0x7f35d415dd60 2024-04-25T09:02:55.246 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.258+0000 7f35d2a00640 1 -- 172.21.15.121:0/501241628 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c4032440 con 0x7f35d415dd60 2024-04-25T09:02:55.247 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.258+0000 7f35d2a00640 1 -- 172.21.15.121:0/501241628 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4032810 con 0x7f35d415dd60 2024-04-25T09:02:55.247 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.258+0000 7f35d8c00640 1 -- 172.21.15.121:0/501241628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 msgr2=0x7f35d40d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.247 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.258+0000 7f35d8c00640 1 --2- 172.21.15.121:0/501241628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d40d2a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f35d4067800 tx=0x7f35c402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.259+0000 7f35d8c00640 1 -- 172.21.15.121:0/501241628 shutdown_connections 2024-04-25T09:02:55.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.259+0000 7f35d8c00640 1 --2- 172.21.15.121:0/501241628 >> v2:172.21.15.62:3300/0 conn(0x7f35d4161bf0 0x7f35d4162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.259+0000 7f35d8c00640 1 --2- 172.21.15.121:0/501241628 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d40ce990 0x7f35d40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.259+0000 7f35d8c00640 1 --2- 172.21.15.121:0/501241628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d40d2a50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.259+0000 7f35d8c00640 1 -- 172.21.15.121:0/501241628 >> 172.21.15.121:0/501241628 conn(0x7f35d406b210 msgr2=0x7f35d40668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.259+0000 7f35d8c00640 1 -- 172.21.15.121:0/501241628 shutdown_connections 2024-04-25T09:02:55.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.259+0000 7f35d8c00640 1 -- 172.21.15.121:0/501241628 wait complete. 2024-04-25T09:02:55.249 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.260+0000 7f35d8c00640 1 Processor -- start 2024-04-25T09:02:55.249 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.261+0000 7f35d8c00640 1 -- start start 2024-04-25T09:02:55.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.261+0000 7f35d8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d40ce990 0x7f35d41501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.261+0000 7f35d8c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d41585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35d8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d4161bf0 0x7f35d4158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35d8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d415e140 con 0x7f35d40ce990 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35d8c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40d2f90 con 0x7f35d415dd60 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35d8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d41625c0 con 0x7f35d4161bf0 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d41585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d41585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46402/0 (socket says 172.21.15.121:46402) 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35cbe00640 1 -- 172.21.15.121:0/2639117364 learned_addr learned my addr 172.21.15.121:0/2639117364 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35d3e00640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d4161bf0 0x7f35d4158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35d3400640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d40ce990 0x7f35d41501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35cbe00640 1 -- 172.21.15.121:0/2639117364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d4161bf0 msgr2=0x7f35d4158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35cbe00640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d4161bf0 0x7f35d4158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35cbe00640 1 -- 172.21.15.121:0/2639117364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d40ce990 msgr2=0x7f35d41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35cbe00640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d40ce990 0x7f35d41501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.262+0000 7f35cbe00640 1 -- 172.21.15.121:0/2639117364 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40b5250 con 0x7f35d415dd60 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.263+0000 7f35cbe00640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d41585c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00e9b0 tx=0x7f35bc00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.263+0000 7f35cb400640 1 -- 172.21.15.121:0/2639117364 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc017070 con 0x7f35d415dd60 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.263+0000 7f35cb400640 1 -- 172.21.15.121:0/2639117364 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc00c9a0 con 0x7f35d415dd60 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.263+0000 7f35cb400640 1 -- 172.21.15.121:0/2639117364 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00cd90 con 0x7f35d415dd60 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.263+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d40b5530 con 0x7f35d415dd60 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.263+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d40b5af0 con 0x7f35d415dd60 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.263+0000 7f35cb400640 1 -- 172.21.15.121:0/2639117364 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f35bc01a070 con 0x7f35d415dd60 2024-04-25T09:02:55.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.263+0000 7f35cb400640 1 -- 172.21.15.121:0/2639117364 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f35bc0139f0 con 0x7f35d415dd60 2024-04-25T09:02:55.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.264+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d40750e0 con 0x7f35d415dd60 2024-04-25T09:02:55.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.266+0000 7f35cb400640 1 -- 172.21.15.121:0/2639117364 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f35bc013c90 con 0x7f35d415dd60 2024-04-25T09:02:55.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.341+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "28077425-23cd-49d3-84b5-e383d0aa1528", "id": 5} v 0) v1 -- 0x7f35d40cf9b0 con 0x7f35d415dd60 2024-04-25T09:02:55.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.348+0000 7f35cb400640 1 -- 172.21.15.121:0/2639117364 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "28077425-23cd-49d3-84b5-e383d0aa1528", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f35bc013470 con 0x7f35d415dd60 2024-04-25T09:02:55.337 INFO:teuthology.orchestra.run.smithi121.stdout:5 2024-04-25T09:02:55.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.349+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 msgr2=0x7f35d41585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.349+0000 7f35d8c00640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d41585c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00e9b0 tx=0x7f35bc00ee70 comp rx=0 tx=0).stop 2024-04-25T09:02:55.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.350+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 shutdown_connections 2024-04-25T09:02:55.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.350+0000 7f35d8c00640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35d4161bf0 0x7f35d4158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.350+0000 7f35d8c00640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35d415dd60 0x7f35d41585c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.350+0000 7f35d8c00640 1 --2- 172.21.15.121:0/2639117364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35d40ce990 0x7f35d41501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.350+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 >> 172.21.15.121:0/2639117364 conn(0x7f35d406b210 msgr2=0x7f35d40ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.350+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 shutdown_connections 2024-04-25T09:02:55.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.350+0000 7f35d8c00640 1 -- 172.21.15.121:0/2639117364 wait complete. 2024-04-25T09:02:55.354 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 916f3e21-559d-4401-bad6-f2b7713f7425 6 2024-04-25T09:02:55.531 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.543+0000 7f3befc00640 1 Processor -- start 2024-04-25T09:02:55.531 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.543+0000 7f3befc00640 1 -- start start 2024-04-25T09:02:55.533 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3befc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be815dc10 0x7f3be80d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.533 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3befc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3be80d20a0 0x7f3be80b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.533 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3befc00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be80b7590 con 0x7f3be80b70d0 2024-04-25T09:02:55.533 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3befc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be80b81a0 con 0x7f3be80d20a0 2024-04-25T09:02:55.533 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3befc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b7870 con 0x7f3be815dc10 2024-04-25T09:02:55.533 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3bed200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be815dc10 0x7f3be80d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3bedc00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3be80b70d0 0x7f3be80b8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:33028/0 (socket says 172.21.15.121:33028) 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3bedc00640 1 -- 172.21.15.121:0/3835632786 learned_addr learned my addr 172.21.15.121:0/3835632786 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.544+0000 7f3bed200640 1 --2- 172.21.15.121:0/3835632786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be815dc10 0x7f3be80d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:54094/0 (socket says 172.21.15.121:54094) 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.545+0000 7f3be7e00640 1 --2- 172.21.15.121:0/3835632786 >> v2:172.21.15.62:3300/0 conn(0x7f3be80d20a0 0x7f3be80b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.545+0000 7f3bed200640 1 -- 172.21.15.121:0/3835632786 >> v2:172.21.15.62:3300/0 conn(0x7f3be80d20a0 msgr2=0x7f3be80b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.545+0000 7f3bed200640 1 --2- 172.21.15.121:0/3835632786 >> v2:172.21.15.62:3300/0 conn(0x7f3be80d20a0 0x7f3be80b6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.545+0000 7f3bed200640 1 -- 172.21.15.121:0/3835632786 >> v1:172.21.15.121:6789/0 conn(0x7f3be80b70d0 legacy=0x7f3be80b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.545+0000 7f3bed200640 1 -- 172.21.15.121:0/3835632786 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be80b7b50 con 0x7f3be815dc10 2024-04-25T09:02:55.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.545+0000 7f3be7400640 1 -- 172.21.15.121:0/3835632786 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140318412 0 0) 0x7f3be80b7590 con 0x7f3be80b70d0 2024-04-25T09:02:55.535 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.546+0000 7f3bed200640 1 --2- 172.21.15.121:0/3835632786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be815dc10 0x7f3be80d1b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009d90 tx=0x7f3bdc02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16bdb801a8008f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.535 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.546+0000 7f3be7400640 1 -- 172.21.15.121:0/3835632786 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007740 con 0x7f3be815dc10 2024-04-25T09:02:55.535 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.546+0000 7f3be7400640 1 -- 172.21.15.121:0/3835632786 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc0078e0 con 0x7f3be815dc10 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.546+0000 7f3be7400640 1 -- 172.21.15.121:0/3835632786 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007cb0 con 0x7f3be815dc10 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.547+0000 7f3befc00640 1 -- 172.21.15.121:0/3835632786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be815dc10 msgr2=0x7f3be80d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.547+0000 7f3befc00640 1 --2- 172.21.15.121:0/3835632786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be815dc10 0x7f3be80d1b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009d90 tx=0x7f3bdc02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.547+0000 7f3befc00640 1 -- 172.21.15.121:0/3835632786 shutdown_connections 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.547+0000 7f3befc00640 1 --2- 172.21.15.121:0/3835632786 >> v2:172.21.15.62:3300/0 conn(0x7f3be80d20a0 0x7f3be80b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.547+0000 7f3befc00640 1 --2- 172.21.15.121:0/3835632786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be815dc10 0x7f3be80d1b60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.547+0000 7f3befc00640 1 -- 172.21.15.121:0/3835632786 >> 172.21.15.121:0/3835632786 conn(0x7f3be806b000 msgr2=0x7f3be806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.547+0000 7f3befc00640 1 -- 172.21.15.121:0/3835632786 shutdown_connections 2024-04-25T09:02:55.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.547+0000 7f3befc00640 1 -- 172.21.15.121:0/3835632786 wait complete. 2024-04-25T09:02:55.537 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.549+0000 7f3befc00640 1 Processor -- start 2024-04-25T09:02:55.537 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.549+0000 7f3befc00640 1 -- start start 2024-04-25T09:02:55.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3befc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be80b70d0 0x7f3be815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3befc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3be815dc10 0x7f3be8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3befc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be81583b0 0x7f3be80d2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3befc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b8d40 con 0x7f3be81583b0 2024-04-25T09:02:55.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3befc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be815dff0 con 0x7f3be815dc10 2024-04-25T09:02:55.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3befc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b8ec0 con 0x7f3be80b70d0 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3be815dc10 0x7f3be8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3bed200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be80b70d0 0x7f3be815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3be7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3be815dc10 0x7f3be8155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46406/0 (socket says 172.21.15.121:46406) 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3be7e00640 1 -- 172.21.15.121:0/1665812494 learned_addr learned my addr 172.21.15.121:0/1665812494 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3bedc00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be81583b0 0x7f3be80d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3be7e00640 1 -- 172.21.15.121:0/1665812494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be80b70d0 msgr2=0x7f3be815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3be7e00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be80b70d0 0x7f3be815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3be7e00640 1 -- 172.21.15.121:0/1665812494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be81583b0 msgr2=0x7f3be80d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3be7e00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be81583b0 0x7f3be80d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3be7e00640 1 -- 172.21.15.121:0/1665812494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be815b2d0 con 0x7f3be815dc10 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.550+0000 7f3bedc00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be81583b0 0x7f3be80d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.551+0000 7f3be7e00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3be815dc10 0x7f3be8155d90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400d830 tx=0x7f3bd400dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.551+0000 7f3be4c00640 1 -- 172.21.15.121:0/1665812494 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400b6c0 con 0x7f3be815dc10 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.551+0000 7f3be4c00640 1 -- 172.21.15.121:0/1665812494 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd4019070 con 0x7f3be815dc10 2024-04-25T09:02:55.540 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.551+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be8156870 con 0x7f3be815dc10 2024-04-25T09:02:55.541 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.551+0000 7f3be4c00640 1 -- 172.21.15.121:0/1665812494 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400ba80 con 0x7f3be815dc10 2024-04-25T09:02:55.541 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.551+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be8157110 con 0x7f3be815dc10 2024-04-25T09:02:55.541 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.551+0000 7f3be4c00640 1 -- 172.21.15.121:0/1665812494 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3bd401e460 con 0x7f3be815dc10 2024-04-25T09:02:55.541 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.551+0000 7f3be4c00640 1 -- 172.21.15.121:0/1665812494 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f3bd4014d40 con 0x7f3be815dc10 2024-04-25T09:02:55.541 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.552+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be80684f0 con 0x7f3be815dc10 2024-04-25T09:02:55.543 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.555+0000 7f3be4c00640 1 -- 172.21.15.121:0/1665812494 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3be80684f0 con 0x7f3be815dc10 2024-04-25T09:02:55.620 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:55.631+0000 7f4807723040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T09:02:55.621 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.633+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "916f3e21-559d-4401-bad6-f2b7713f7425", "id": 6} v 0) v1 -- 0x7f3be804ef20 con 0x7f3be815dc10 2024-04-25T09:02:55.628 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.640+0000 7f3be4c00640 1 -- 172.21.15.121:0/1665812494 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "916f3e21-559d-4401-bad6-f2b7713f7425", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3bd4014750 con 0x7f3be815dc10 2024-04-25T09:02:55.628 INFO:teuthology.orchestra.run.smithi121.stdout:6 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.641+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3be815dc10 msgr2=0x7f3be8155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.641+0000 7f3befc00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3be815dc10 0x7f3be8155d90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400d830 tx=0x7f3bd400dcf0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.641+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 shutdown_connections 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.641+0000 7f3befc00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3be81583b0 0x7f3be80d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.641+0000 7f3befc00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3be815dc10 0x7f3be8155d90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.641+0000 7f3befc00640 1 --2- 172.21.15.121:0/1665812494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3be80b70d0 0x7f3be815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.641+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 >> 172.21.15.121:0/1665812494 conn(0x7f3be806b000 msgr2=0x7f3be80b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.642+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 shutdown_connections 2024-04-25T09:02:55.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.642+0000 7f3befc00640 1 -- 172.21.15.121:0/1665812494 wait complete. 2024-04-25T09:02:55.646 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 70ddc55f-d707-4e42-912b-3b17d17aabb9 7 2024-04-25T09:02:55.684 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:55.695+0000 7f73337b4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T09:02:55.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.835+0000 7f57f6600640 1 Processor -- start 2024-04-25T09:02:55.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.836+0000 7f57f6600640 1 -- start start 2024-04-25T09:02:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.837+0000 7f57f6600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f57f015e180 0x7f57f0161b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.837+0000 7f57f6600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f57f00ce5c0 0x7f57f00b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.837+0000 7f57f6600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00b3310 con 0x7f57f00d0650 2024-04-25T09:02:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.837+0000 7f57f6600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f00d2210 con 0x7f57f00ce5c0 2024-04-25T09:02:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.837+0000 7f57f6600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f00b7600 con 0x7f57f015e180 2024-04-25T09:02:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.837+0000 7f57ef400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f57f015e180 0x7f57f0161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.837+0000 7f57ef400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f57f015e180 0x7f57f0161b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:46408/0 (socket says 172.21.15.121:46408) 2024-04-25T09:02:55.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.837+0000 7f57ef400640 1 -- 172.21.15.121:0/3170324059 learned_addr learned my addr 172.21.15.121:0/3170324059 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:55.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.838+0000 7f57e7e00640 1 --2- 172.21.15.121:0/3170324059 >> v2:172.21.15.62:3300/0 conn(0x7f57f00ce5c0 0x7f57f00b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.838+0000 7f57ef400640 1 -- 172.21.15.121:0/3170324059 >> v2:172.21.15.62:3300/0 conn(0x7f57f00ce5c0 msgr2=0x7f57f00b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.838+0000 7f57ef400640 1 --2- 172.21.15.121:0/3170324059 >> v2:172.21.15.62:3300/0 conn(0x7f57f00ce5c0 0x7f57f00b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.838+0000 7f57ef400640 1 -- 172.21.15.121:0/3170324059 >> v1:172.21.15.121:6789/0 conn(0x7f57f00d0650 legacy=0x7f57f00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.838+0000 7f57ef400640 1 -- 172.21.15.121:0/3170324059 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0162200 con 0x7f57f015e180 2024-04-25T09:02:55.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.838+0000 7f57e7e00640 1 --2- 172.21.15.121:0/3170324059 >> v2:172.21.15.62:3300/0 conn(0x7f57f00ce5c0 0x7f57f00b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:55.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.839+0000 7f57ef400640 1 --2- 172.21.15.121:0/3170324059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f015e180 0x7f57f0161b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f57f00678d0 tx=0x7f57e002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95c1edb187ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.839+0000 7f57eea00640 1 -- 172.21.15.121:0/3170324059 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0037070 con 0x7f57f015e180 2024-04-25T09:02:55.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.839+0000 7f57eea00640 1 -- 172.21.15.121:0/3170324059 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0032440 con 0x7f57f015e180 2024-04-25T09:02:55.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.839+0000 7f57eea00640 1 -- 172.21.15.121:0/3170324059 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0032810 con 0x7f57f015e180 2024-04-25T09:02:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.840+0000 7f57f6600640 1 -- 172.21.15.121:0/3170324059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f015e180 msgr2=0x7f57f0161b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.840+0000 7f57f6600640 1 --2- 172.21.15.121:0/3170324059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f015e180 0x7f57f0161b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f57f00678d0 tx=0x7f57e002a270 comp rx=0 tx=0).stop 2024-04-25T09:02:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.840+0000 7f57f6600640 1 -- 172.21.15.121:0/3170324059 shutdown_connections 2024-04-25T09:02:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.840+0000 7f57f6600640 1 --2- 172.21.15.121:0/3170324059 >> v2:172.21.15.62:3300/0 conn(0x7f57f00ce5c0 0x7f57f00b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.840+0000 7f57f6600640 1 --2- 172.21.15.121:0/3170324059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f015e180 0x7f57f0161b20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.840+0000 7f57f6600640 1 -- 172.21.15.121:0/3170324059 >> 172.21.15.121:0/3170324059 conn(0x7f57f006b210 msgr2=0x7f57f00668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.840+0000 7f57f6600640 1 -- 172.21.15.121:0/3170324059 shutdown_connections 2024-04-25T09:02:55.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.840+0000 7f57f6600640 1 -- 172.21.15.121:0/3170324059 wait complete. 2024-04-25T09:02:55.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.842+0000 7f57f6600640 1 Processor -- start 2024-04-25T09:02:55.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.842+0000 7f57f6600640 1 -- start start 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57f6600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f00ce5c0 0x7f57f01f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57f6600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f00d0650 0x7f57f01f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57f6600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f015e180 0x7f57f00cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57f6600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00d0ae0 con 0x7f57f015e180 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57f6600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01627c0 con 0x7f57f00d0650 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57f6600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00d0c60 con 0x7f57f00ce5c0 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57ef400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f00ce5c0 0x7f57f01f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57e7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f00d0650 0x7f57f01f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57ef400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f00ce5c0 0x7f57f01f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.121:54104/0 (socket says 172.21.15.121:54104) 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.843+0000 7f57ef400640 1 -- 172.21.15.121:0/4286584959 learned_addr learned my addr 172.21.15.121:0/4286584959 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-25T09:02:55.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57efe00640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f015e180 0x7f57f00cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7e00640 1 -- 172.21.15.121:0/4286584959 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f00ce5c0 msgr2=0x7f57f01f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7e00640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f00ce5c0 0x7f57f01f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7e00640 1 -- 172.21.15.121:0/4286584959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f015e180 msgr2=0x7f57f00cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7e00640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f015e180 0x7f57f00cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7e00640 1 -- 172.21.15.121:0/4286584959 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f01f77a0 con 0x7f57f00d0650 2024-04-25T09:02:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57ef400640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f00ce5c0 0x7f57f01f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:55.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57efe00640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f015e180 0x7f57f00cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7e00640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f00d0650 0x7f57f01f70c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f57d800c7f0 tx=0x7f57d800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7400640 1 -- 172.21.15.121:0/4286584959 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8015070 con 0x7f57f00d0650 2024-04-25T09:02:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7400640 1 -- 172.21.15.121:0/4286584959 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d8004220 con 0x7f57f00d0650 2024-04-25T09:02:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f01f7a80 con 0x7f57f00d0650 2024-04-25T09:02:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57e7400640 1 -- 172.21.15.121:0/4286584959 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8007620 con 0x7f57f00d0650 2024-04-25T09:02:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.844+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f00c8960 con 0x7f57f00d0650 2024-04-25T09:02:55.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.845+0000 7f57e7400640 1 -- 172.21.15.121:0/4286584959 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f57d80079d0 con 0x7f57f00d0650 2024-04-25T09:02:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.845+0000 7f57e7400640 1 -- 172.21.15.121:0/4286584959 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f57d8007cc0 con 0x7f57f00d0650 2024-04-25T09:02:55.835 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.846+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f00c8b60 con 0x7f57f00d0650 2024-04-25T09:02:55.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.850+0000 7f57e7400640 1 -- 172.21.15.121:0/4286584959 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f57f00c8b60 con 0x7f57f00d0650 2024-04-25T09:02:55.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.928+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd new", "uuid": "70ddc55f-d707-4e42-912b-3b17d17aabb9", "id": 7} v 0) v1 -- 0x7f57f004ef20 con 0x7f57f00d0650 2024-04-25T09:02:55.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.938+0000 7f57e7400640 1 -- 172.21.15.121:0/4286584959 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70ddc55f-d707-4e42-912b-3b17d17aabb9", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f57d8003610 con 0x7f57f00d0650 2024-04-25T09:02:55.927 INFO:teuthology.orchestra.run.smithi121.stdout:7 2024-04-25T09:02:55.927 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:55.937+0000 7f4807723040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T09:02:55.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.939+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f00d0650 msgr2=0x7f57f01f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.939+0000 7f57f6600640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f00d0650 0x7f57f01f70c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f57d800c7f0 tx=0x7f57d800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.940+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 shutdown_connections 2024-04-25T09:02:55.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.940+0000 7f57f6600640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f015e180 0x7f57f00cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.940+0000 7f57f6600640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f00d0650 0x7f57f01f70c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.940+0000 7f57f6600640 1 --2- 172.21.15.121:0/4286584959 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f00ce5c0 0x7f57f01f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.940+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 >> 172.21.15.121:0/4286584959 conn(0x7f57f006b210 msgr2=0x7f57f015c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.940+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 shutdown_connections 2024-04-25T09:02:55.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-25T09:02:55.940+0000 7f57f6600640 1 -- 172.21.15.121:0/4286584959 wait complete. 2024-04-25T09:02:55.944 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T09:02:55.944 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T09:02:55.951 INFO:tasks.ceph.osd.0:Started 2024-04-25T09:02:55.952 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T09:02:55.952 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T09:02:55.962 INFO:tasks.ceph.osd.1:Started 2024-04-25T09:02:55.962 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T09:02:55.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T09:02:55.967 INFO:tasks.ceph.osd.2:Started 2024-04-25T09:02:55.967 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T09:02:55.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T09:02:55.974 INFO:tasks.ceph.osd.3:Started 2024-04-25T09:02:55.974 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T09:02:55.974 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T09:02:55.991 INFO:tasks.ceph.osd.4:Started 2024-04-25T09:02:55.991 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T09:02:55.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T09:02:55.996 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:56.007+0000 7f4807723040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T09:02:55.998 INFO:tasks.ceph.osd.5:Started 2024-04-25T09:02:55.998 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T09:02:55.998 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T09:02:55.999 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:56.006+0000 7f73337b4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T09:02:56.004 INFO:tasks.ceph.osd.6:Started 2024-04-25T09:02:56.004 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T09:02:56.005 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T09:02:56.011 INFO:tasks.ceph.osd.7:Started 2024-04-25T09:02:56.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:56.071 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:56.082+0000 7f73337b4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T09:02:56.190 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:56.201+0000 7f4807723040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.221+0000 7fb57aa00640 1 Processor -- start 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.221+0000 7fb57aa00640 1 -- start start 2024-04-25T09:02:56.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.222+0000 7fb57aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb5740d0ae0 0x7fb5740d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.222+0000 7fb57aa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5740b6cf0 0x7fb5740b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.222+0000 7fb57aa00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5740d2620 con 0x7fb57415dc40 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.222+0000 7fb57aa00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740d23c0 con 0x7fb5740d0ae0 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.222+0000 7fb57aa00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740d2900 con 0x7fb5740b6cf0 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.223+0000 7fb56b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb5740d0ae0 0x7fb5740d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.223+0000 7fb578a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5740b6cf0 0x7fb5740b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.223+0000 7fb56b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb5740d0ae0 0x7fb5740d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32846/0 (socket says 172.21.15.62:32846) 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.223+0000 7fb56b400640 1 -- 172.21.15.62:0/1017385766 learned_addr learned my addr 172.21.15.62:0/1017385766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:02:56.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.223+0000 7fb578a00640 1 -- 172.21.15.62:0/1017385766 >> v2:172.21.15.62:3300/0 conn(0x7fb5740d0ae0 msgr2=0x7fb5740d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:56.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.223+0000 7fb578a00640 1 --2- 172.21.15.62:0/1017385766 >> v2:172.21.15.62:3300/0 conn(0x7fb5740d0ae0 0x7fb5740d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.223+0000 7fb578a00640 1 -- 172.21.15.62:0/1017385766 >> v1:172.21.15.62:6789/0 conn(0x7fb57415dc40 legacy=0x7fb5740d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.223+0000 7fb578a00640 1 -- 172.21.15.62:0/1017385766 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740d2be0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.224+0000 7fb578a00640 1 --2- 172.21.15.62:0/1017385766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740b6cf0 0x7fb5740b7180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009d90 tx=0x7fb56c02a6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=224c4c87f5381c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.224+0000 7fb573400640 1 -- 172.21.15.62:0/1017385766 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c007740 con 0x7fb5740b6cf0 2024-04-25T09:02:56.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.224+0000 7fb573400640 1 -- 172.21.15.62:0/1017385766 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c0078e0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.224+0000 7fb573400640 1 -- 172.21.15.62:0/1017385766 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c007cb0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.225+0000 7fb57aa00640 1 -- 172.21.15.62:0/1017385766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740b6cf0 msgr2=0x7fb5740b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.225+0000 7fb57aa00640 1 --2- 172.21.15.62:0/1017385766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740b6cf0 0x7fb5740b7180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009d90 tx=0x7fb56c02a6b0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.225+0000 7fb57aa00640 1 -- 172.21.15.62:0/1017385766 shutdown_connections 2024-04-25T09:02:56.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.225+0000 7fb57aa00640 1 --2- 172.21.15.62:0/1017385766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740b6cf0 0x7fb5740b7180 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.225+0000 7fb57aa00640 1 --2- 172.21.15.62:0/1017385766 >> v2:172.21.15.62:3300/0 conn(0x7fb5740d0ae0 0x7fb5740d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.225+0000 7fb57aa00640 1 -- 172.21.15.62:0/1017385766 >> 172.21.15.62:0/1017385766 conn(0x7fb57406b000 msgr2=0x7fb57406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:56.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.225+0000 7fb57aa00640 1 -- 172.21.15.62:0/1017385766 shutdown_connections 2024-04-25T09:02:56.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.225+0000 7fb57aa00640 1 -- 172.21.15.62:0/1017385766 wait complete. 2024-04-25T09:02:56.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.227+0000 7fb57aa00640 1 Processor -- start 2024-04-25T09:02:56.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.227+0000 7fb57aa00640 1 -- start start 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb57aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5740b6cf0 0x7fb574159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb57aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740d0ae0 0x7fb5740cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb57aa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57415dc40 0x7fb5740cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb57aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740d36c0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb57aa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740d31a0 con 0x7fb5740d0ae0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb57aa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5741527e0 con 0x7fb57415dc40 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb573e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5740b6cf0 0x7fb574159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb573e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5740b6cf0 0x7fb574159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32850/0 (socket says 172.21.15.62:32850) 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb573e00640 1 -- 172.21.15.62:0/2155773024 learned_addr learned my addr 172.21.15.62:0/2155773024 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb56b400640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740d0ae0 0x7fb5740cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.228+0000 7fb578a00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57415dc40 0x7fb5740cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb573e00640 1 -- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57415dc40 msgr2=0x7fb5740cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb573e00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57415dc40 0x7fb5740cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb573e00640 1 -- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740d0ae0 msgr2=0x7fb5740cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb573e00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740d0ae0 0x7fb5740cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb573e00640 1 -- 172.21.15.62:0/2155773024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57415e0d0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb578a00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57415dc40 0x7fb5740cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb573e00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5740b6cf0 0x7fb574159680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c3c0 tx=0x7fb56400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb570c00640 1 -- 172.21.15.62:0/2155773024 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640052d0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb570c00640 1 -- 172.21.15.62:0/2155773024 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb564005490 con 0x7fb5740b6cf0 2024-04-25T09:02:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb570c00640 1 -- 172.21.15.62:0/2155773024 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564007260 con 0x7fb5740b6cf0 2024-04-25T09:02:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57415a100 con 0x7fb5740b6cf0 2024-04-25T09:02:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.229+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57415a6c0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.230+0000 7fb570c00640 1 -- 172.21.15.62:0/2155773024 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb564007400 con 0x7fb5740b6cf0 2024-04-25T09:02:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.230+0000 7fb570c00640 1 -- 172.21.15.62:0/2155773024 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb564003a00 con 0x7fb5740b6cf0 2024-04-25T09:02:56.222 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:56.233+0000 7f73337b4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T09:02:56.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.231+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5740729c0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.233+0000 7fb570c00640 1 -- 172.21.15.62:0/2155773024 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb5640035f0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.298+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb57404f000 con 0x7fb5740b6cf0 2024-04-25T09:02:56.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.299+0000 7fb570c00640 1 -- 172.21.15.62:0/2155773024 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fb564003ce0 con 0x7fb5740b6cf0 2024-04-25T09:02:56.288 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:02:56.289 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:02:55.931188+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:56.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.300+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5740b6cf0 msgr2=0x7fb574159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.300+0000 7fb57aa00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5740b6cf0 0x7fb574159680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb56400c3c0 tx=0x7fb56400c880 comp rx=0 tx=0).stop 2024-04-25T09:02:56.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.300+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 shutdown_connections 2024-04-25T09:02:56.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.300+0000 7fb57aa00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb57415dc40 0x7fb5740cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.300+0000 7fb57aa00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5740d0ae0 0x7fb5740cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.300+0000 7fb57aa00640 1 --2- 172.21.15.62:0/2155773024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5740b6cf0 0x7fb574159680 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.300+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 >> 172.21.15.62:0/2155773024 conn(0x7fb57406b000 msgr2=0x7fb5740d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:56.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.300+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 shutdown_connections 2024-04-25T09:02:56.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:02:56.301+0000 7fb57aa00640 1 -- 172.21.15.62:0/2155773024 wait complete. 2024-04-25T09:02:56.305 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T09:02:56.306 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T09:02:56.548 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:56.559+0000 7f4807723040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T09:02:56.558 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:56.569+0000 7f73337b4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T09:02:56.607 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:56.618+0000 7f4807723040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T09:02:56.614 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:56.626+0000 7f73337b4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T09:02:56.669 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-25T09:02:56.680+0000 7f28866b5640 -1 Falling back to public interface 2024-04-25T09:02:56.671 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-04-25T09:02:56.682+0000 7fe9a8c84640 -1 Falling back to public interface 2024-04-25T09:02:56.675 INFO:tasks.ceph.osd.4.smithi121.stderr:2024-04-25T09:02:56.687+0000 7f9c0d526640 -1 Falling back to public interface 2024-04-25T09:02:56.678 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-04-25T09:02:56.689+0000 7f846dc16640 -1 Falling back to public interface 2024-04-25T09:02:56.688 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-04-25T09:02:56.699+0000 7fc3c6a19640 -1 Falling back to public interface 2024-04-25T09:02:56.688 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-04-25T09:02:56.699+0000 7f0242ddc640 -1 Falling back to public interface 2024-04-25T09:02:56.715 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-25T09:02:56.726+0000 7f18ef0d7640 -1 Falling back to public interface 2024-04-25T09:02:56.718 INFO:tasks.ceph.osd.7.smithi121.stderr:2024-04-25T09:02:56.730+0000 7f13ad6af640 -1 Falling back to public interface 2024-04-25T09:02:56.801 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:56.813+0000 7f73337b4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T09:02:56.802 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:56.813+0000 7f4807723040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T09:02:56.966 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:56.978+0000 7f73337b4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T09:02:56.977 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:56.988+0000 7f4807723040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T09:02:57.027 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:57.038+0000 7f73337b4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T09:02:57.038 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:57.049+0000 7f4807723040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T09:02:57.108 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:57.120+0000 7f73337b4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T09:02:57.111 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:57.122 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:57.133+0000 7f4807723040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T09:02:57.161 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:57.172+0000 7f73337b4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T09:02:57.179 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:57.189+0000 7f4807723040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T09:02:57.274 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.285+0000 7f5aa7200640 1 Processor -- start 2024-04-25T09:02:57.275 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.285+0000 7f5aa7200640 1 -- start start 2024-04-25T09:02:57.276 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.287+0000 7f5aa7200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5aa80b0d70 0x7f5aa80cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.277 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.287+0000 7f5aa7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 0x7f5aa806f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.277 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.287+0000 7f5aa7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa80cff10 con 0x7f5aa806f950 2024-04-25T09:02:57.277 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.287+0000 7f5aa7200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa806ef60 con 0x7f5aa80b0d70 2024-04-25T09:02:57.277 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.287+0000 7f5aa7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa806fd80 con 0x7f5aa815ed60 2024-04-25T09:02:57.277 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.287+0000 7f5aa5e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5aa80b0d70 0x7f5aa80cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.277 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.288+0000 7f5aa5e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5aa80b0d70 0x7f5aa80cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:58816/0 (socket says 172.21.15.62:58816) 2024-04-25T09:02:57.277 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.288+0000 7f5aa5e00640 1 -- 172.21.15.62:0/3947492791 learned_addr learned my addr 172.21.15.62:0/3947492791 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:02:57.277 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.288+0000 7f5aa5400640 1 --2- 172.21.15.62:0/3947492791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 0x7f5aa806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.278 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.288+0000 7f5aa5e00640 1 -- 172.21.15.62:0/3947492791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 msgr2=0x7f5aa806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:57.278 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.288+0000 7f5aa5e00640 1 --2- 172.21.15.62:0/3947492791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 0x7f5aa806f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.278 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.288+0000 7f5aa5e00640 1 -- 172.21.15.62:0/3947492791 >> v1:172.21.15.121:6789/0 conn(0x7f5aa806f950 legacy=0x7f5aa815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.278 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.288+0000 7f5aa5e00640 1 -- 172.21.15.62:0/3947492791 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa80d01f0 con 0x7f5aa80b0d70 2024-04-25T09:02:57.278 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.289+0000 7f5aa5e00640 1 --2- 172.21.15.62:0/3947492791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 0x7f5aa80cf930 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009870 tx=0x7f5a9802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81b3bc10775c26a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.279 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.289+0000 7f5aa4a00640 1 -- 172.21.15.62:0/3947492791 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98037070 con 0x7f5aa80b0d70 2024-04-25T09:02:57.279 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.289+0000 7f5aa4a00640 1 -- 172.21.15.62:0/3947492791 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a98032440 con 0x7f5aa80b0d70 2024-04-25T09:02:57.279 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.289+0000 7f5aa4a00640 1 -- 172.21.15.62:0/3947492791 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98032810 con 0x7f5aa80b0d70 2024-04-25T09:02:57.280 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.290+0000 7f5aa7200640 1 -- 172.21.15.62:0/3947492791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 msgr2=0x7f5aa80cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.280 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.290+0000 7f5aa7200640 1 --2- 172.21.15.62:0/3947492791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 0x7f5aa80cf930 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009870 tx=0x7f5a9802a250 comp rx=0 tx=0).stop 2024-04-25T09:02:57.280 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.290+0000 7f5aa7200640 1 -- 172.21.15.62:0/3947492791 shutdown_connections 2024-04-25T09:02:57.280 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.290+0000 7f5aa7200640 1 --2- 172.21.15.62:0/3947492791 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 0x7f5aa806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.280 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.290+0000 7f5aa7200640 1 --2- 172.21.15.62:0/3947492791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 0x7f5aa80cf930 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.280 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.290+0000 7f5aa7200640 1 -- 172.21.15.62:0/3947492791 >> 172.21.15.62:0/3947492791 conn(0x7f5aa80d17c0 msgr2=0x7f5aa80d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.280 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.291+0000 7f5aa7200640 1 -- 172.21.15.62:0/3947492791 shutdown_connections 2024-04-25T09:02:57.280 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.291+0000 7f5aa7200640 1 -- 172.21.15.62:0/3947492791 wait complete. 2024-04-25T09:02:57.281 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.293+0000 7f5aa7200640 1 Processor -- start 2024-04-25T09:02:57.282 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.293+0000 7f5aa7200640 1 -- start start 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa806f950 0x7f5aa81f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 0x7f5aa81f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 0x7f5aa80b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa80d0a90 con 0x7f5aa806f950 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa80d07b0 con 0x7f5aa80b0d70 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa815f1a0 con 0x7f5aa815ed60 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa806f950 0x7f5aa81f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa806f950 0x7f5aa81f5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32858/0 (socket says 172.21.15.62:32858) 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa5e00640 1 -- 172.21.15.62:0/346353436 learned_addr learned my addr 172.21.15.62:0/346353436 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:02:57.284 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa5400640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 0x7f5aa81f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.285 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.294+0000 7f5aa6800640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 0x7f5aa80b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.285 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5aa5e00640 1 -- 172.21.15.62:0/346353436 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 msgr2=0x7f5aa80b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.285 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5aa5e00640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 0x7f5aa80b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.285 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5aa5e00640 1 -- 172.21.15.62:0/346353436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 msgr2=0x7f5aa81f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.285 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5aa5e00640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 0x7f5aa81f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.285 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5aa5e00640 1 -- 172.21.15.62:0/346353436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa80b5430 con 0x7f5aa806f950 2024-04-25T09:02:57.285 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5aa5e00640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa806f950 0x7f5aa81f5360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5a9802a290 tx=0x7f5a980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.286 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5a9e000640 1 -- 172.21.15.62:0/346353436 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98037040 con 0x7f5aa806f950 2024-04-25T09:02:57.286 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5a9e000640 1 -- 172.21.15.62:0/346353436 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a98042070 con 0x7f5aa806f950 2024-04-25T09:02:57.286 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5a9e000640 1 -- 172.21.15.62:0/346353436 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98030880 con 0x7f5aa806f950 2024-04-25T09:02:57.286 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.295+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa80b5710 con 0x7f5aa806f950 2024-04-25T09:02:57.286 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.296+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa80b5cd0 con 0x7f5aa806f950 2024-04-25T09:02:57.286 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.296+0000 7f5a9e000640 1 -- 172.21.15.62:0/346353436 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5a98030a20 con 0x7f5aa806f950 2024-04-25T09:02:57.286 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.296+0000 7f5a9e000640 1 -- 172.21.15.62:0/346353436 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5a98047b40 con 0x7f5aa806f950 2024-04-25T09:02:57.286 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.297+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa8070000 con 0x7f5aa806f950 2024-04-25T09:02:57.290 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.301+0000 7f5a9e000640 1 -- 172.21.15.62:0/346353436 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5aa8070000 con 0x7f5aa806f950 2024-04-25T09:02:57.357 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.367+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aa80af3b0 con 0x7f5aa806f950 2024-04-25T09:02:57.357 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.369+0000 7f5a9e000640 1 -- 172.21.15.62:0/346353436 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f5aa80af3b0 con 0x7f5aa806f950 2024-04-25T09:02:57.358 INFO:teuthology.misc.health.smithi062.stdout: 2024-04-25T09:02:57.358 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":10,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:02:55.931188+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:57.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa806f950 msgr2=0x7f5aa81f5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa806f950 0x7f5aa81f5360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5a9802a290 tx=0x7f5a980046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 shutdown_connections 2024-04-25T09:02:57.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5aa815ed60 0x7f5aa80b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aa80b0d70 0x7f5aa81f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 --2- 172.21.15.62:0/346353436 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5aa806f950 0x7f5aa81f5360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.360 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 >> 172.21.15.62:0/346353436 conn(0x7f5aa80d17c0 msgr2=0x7f5aa806c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.360 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 shutdown_connections 2024-04-25T09:02:57.360 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:02:57.370+0000 7f5aa7200640 1 -- 172.21.15.62:0/346353436 wait complete. 2024-04-25T09:02:57.372 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T09:02:57.701 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:57.713+0000 7f73337b4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T09:02:57.746 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:57.757+0000 7f4807723040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T09:02:57.836 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:57.848+0000 7f73337b4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T09:02:57.886 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-25T09:02:57.897+0000 7f4807723040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T09:02:58.884 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:58.895+0000 7f72d4800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T09:02:58.885 INFO:tasks.ceph.mgr.x.smithi121.stderr:2024-04-25T09:02:58.897+0000 7f72d4800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T09:03:00.280 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-25T09:03:00.292+0000 7f28866b5640 -1 osd.6 0 log_to_monitors true 2024-04-25T09:03:00.323 INFO:tasks.ceph.osd.7.smithi121.stderr:2024-04-25T09:03:00.335+0000 7f13ad6af640 -1 osd.7 0 log_to_monitors true 2024-04-25T09:03:00.329 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-25T09:03:00.341+0000 7f18ef0d7640 -1 osd.5 0 log_to_monitors true 2024-04-25T09:03:00.336 INFO:tasks.ceph.osd.4.smithi121.stderr:2024-04-25T09:03:00.348+0000 7f9c0d526640 -1 osd.4 0 log_to_monitors true 2024-04-25T09:03:00.348 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-04-25T09:03:00.359+0000 7f0242ddc640 -1 osd.3 0 log_to_monitors true 2024-04-25T09:03:00.550 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-04-25T09:03:00.561+0000 7fe9a8c84640 -1 osd.0 0 log_to_monitors true 2024-04-25T09:03:00.602 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-04-25T09:03:00.613+0000 7f846dc16640 -1 osd.1 0 log_to_monitors true 2024-04-25T09:03:00.615 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-04-25T09:03:00.626+0000 7fc3c6a19640 -1 osd.2 0 log_to_monitors true 2024-04-25T09:03:04.179 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:04.181 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-04-25T09:03:04.192+0000 7fc3c0800640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T09:03:04.191 INFO:tasks.ceph.osd.7.smithi121.stderr:2024-04-25T09:03:04.203+0000 7f13a7c00640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T09:03:04.193 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-04-25T09:03:04.204+0000 7fc3b7200640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:03:04.201 INFO:tasks.ceph.osd.7.smithi121.stderr:2024-04-25T09:03:04.213+0000 7f139e600640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:03:04.225 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-25T09:03:04.236+0000 7f18e9600640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T09:03:04.228 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-25T09:03:04.240+0000 7f18e0000640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:03:04.235 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-25T09:03:04.247+0000 7f2880c00640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T09:03:04.240 INFO:tasks.ceph.osd.4.smithi121.stderr:2024-04-25T09:03:04.252+0000 7f9c09800640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T09:03:04.246 INFO:tasks.ceph.osd.6.smithi121.stderr:2024-04-25T09:03:04.258+0000 7f2877600640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:03:04.249 INFO:tasks.ceph.osd.4.smithi121.stderr:2024-04-25T09:03:04.261+0000 7f9bfe400640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:03:04.252 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-04-25T09:03:04.263+0000 7f8467c00640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T09:03:04.262 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-04-25T09:03:04.274+0000 7f845e600640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:03:04.268 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-04-25T09:03:04.279+0000 7f023d200640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T09:03:04.274 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-04-25T09:03:04.285+0000 7f0233c00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:03:04.286 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-04-25T09:03:04.297+0000 7fe9a5000640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T09:03:04.295 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-04-25T09:03:04.306+0000 7fe999c00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:03:04.348 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.359+0000 7fd5ab000640 1 Processor -- start 2024-04-25T09:03:04.349 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.359+0000 7fd5ab000640 1 -- start start 2024-04-25T09:03:04.350 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5ab000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5a40addb0 0x7fd5a40717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.350 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5ab000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a409dc10 0x7fd5a4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.350 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5ab000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a40b3a80 con 0x7fd5a4159f00 2024-04-25T09:03:04.350 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5ab000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a40b3d60 con 0x7fd5a40addb0 2024-04-25T09:03:04.350 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5ab000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40b4040 con 0x7fd5a409dc10 2024-04-25T09:03:04.351 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5aa600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a409dc10 0x7fd5a4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.351 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5aa600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a409dc10 0x7fd5a4071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:53820/0 (socket says 172.21.15.62:53820) 2024-04-25T09:03:04.351 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5a9200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5a40addb0 0x7fd5a40717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.351 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.361+0000 7fd5aa600640 1 -- 172.21.15.62:0/52102249 learned_addr learned my addr 172.21.15.62:0/52102249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:04.351 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.362+0000 7fd5aa600640 1 -- 172.21.15.62:0/52102249 >> v2:172.21.15.121:3300/0 conn(0x7fd5a40addb0 msgr2=0x7fd5a40717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.351 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.362+0000 7fd5aa600640 1 --2- 172.21.15.62:0/52102249 >> v2:172.21.15.121:3300/0 conn(0x7fd5a40addb0 0x7fd5a40717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.351 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.362+0000 7fd5aa600640 1 -- 172.21.15.62:0/52102249 >> v1:172.21.15.121:6789/0 conn(0x7fd5a4159f00 legacy=0x7fd5a40ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.351 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.362+0000 7fd5aa600640 1 -- 172.21.15.62:0/52102249 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a40b4320 con 0x7fd5a409dc10 2024-04-25T09:03:04.352 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.363+0000 7fd5aa600640 1 --2- 172.21.15.62:0/52102249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a409dc10 0x7fd5a4071d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00d150 tx=0x7fd59c02c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b771ee1fd3cbfe9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.352 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5a3e00640 1 -- 172.21.15.62:0/52102249 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c009600 con 0x7fd5a409dc10 2024-04-25T09:03:04.353 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5a3e00640 1 -- 172.21.15.62:0/52102249 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c037070 con 0x7fd5a409dc10 2024-04-25T09:03:04.353 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5a3e00640 1 -- 172.21.15.62:0/52102249 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0099c0 con 0x7fd5a409dc10 2024-04-25T09:03:04.354 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5ab000640 1 -- 172.21.15.62:0/52102249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a409dc10 msgr2=0x7fd5a4071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.354 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5ab000640 1 --2- 172.21.15.62:0/52102249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a409dc10 0x7fd5a4071d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00d150 tx=0x7fd59c02c930 comp rx=0 tx=0).stop 2024-04-25T09:03:04.354 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5ab000640 1 -- 172.21.15.62:0/52102249 shutdown_connections 2024-04-25T09:03:04.354 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5ab000640 1 --2- 172.21.15.62:0/52102249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a409dc10 0x7fd5a4071d00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.354 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5ab000640 1 --2- 172.21.15.62:0/52102249 >> v2:172.21.15.121:3300/0 conn(0x7fd5a40addb0 0x7fd5a40717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.354 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5ab000640 1 -- 172.21.15.62:0/52102249 >> 172.21.15.62:0/52102249 conn(0x7fd5a406ab20 msgr2=0x7fd5a406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.354 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5ab000640 1 -- 172.21.15.62:0/52102249 shutdown_connections 2024-04-25T09:03:04.354 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.364+0000 7fd5ab000640 1 -- 172.21.15.62:0/52102249 wait complete. 2024-04-25T09:03:04.355 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.366+0000 7fd5ab000640 1 Processor -- start 2024-04-25T09:03:04.355 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.366+0000 7fd5ab000640 1 -- start start 2024-04-25T09:03:04.357 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5ab000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a409dc10 0x7fd5a40b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.357 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5ab000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a40addb0 0x7fd5a415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.357 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5ab000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a4159f00 0x7fd5a415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.357 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5ab000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a415a2e0 con 0x7fd5a40addb0 2024-04-25T09:03:04.357 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5ab000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40b3750 con 0x7fd5a409dc10 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5ab000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a415a460 con 0x7fd5a4159f00 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5a9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a40addb0 0x7fd5a415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5a9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a40addb0 0x7fd5a415af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:32904/0 (socket says 172.21.15.62:32904) 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.367+0000 7fd5a9200640 1 -- 172.21.15.62:0/4004421863 learned_addr learned my addr 172.21.15.62:0/4004421863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a9c00640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a409dc10 0x7fd5a40b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5aa600640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a4159f00 0x7fd5a415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a9200640 1 -- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a4159f00 msgr2=0x7fd5a415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a9200640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a4159f00 0x7fd5a415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a9200640 1 -- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a409dc10 msgr2=0x7fd5a40b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a9200640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a409dc10 0x7fd5a40b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a9200640 1 -- 172.21.15.62:0/4004421863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a415b9a0 con 0x7fd5a40addb0 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a9200640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a40addb0 0x7fd5a415af20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e9b0 tx=0x7fd59000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a1600640 1 -- 172.21.15.62:0/4004421863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590017070 con 0x7fd5a40addb0 2024-04-25T09:03:04.358 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a1600640 1 -- 172.21.15.62:0/4004421863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59000c9a0 con 0x7fd5a40addb0 2024-04-25T09:03:04.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5a1600640 1 -- 172.21.15.62:0/4004421863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59000cd90 con 0x7fd5a40addb0 2024-04-25T09:03:04.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a41f77c0 con 0x7fd5a40addb0 2024-04-25T09:03:04.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.368+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a41f7d80 con 0x7fd5a40addb0 2024-04-25T09:03:04.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.370+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a409e0f0 con 0x7fd5a40addb0 2024-04-25T09:03:04.359 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.371+0000 7fd5a1600640 1 -- 172.21.15.62:0/4004421863 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fd59001a070 con 0x7fd5a40addb0 2024-04-25T09:03:04.361 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.372+0000 7fd5a1600640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd58406b440 0x7fd58406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.361 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.372+0000 7fd5a1600640 1 -- 172.21.15.62:0/4004421863 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fd590089430 con 0x7fd5a40addb0 2024-04-25T09:03:04.361 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.372+0000 7fd5a9c00640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd58406b440 0x7fd58406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.362 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.373+0000 7fd5a9c00640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd58406b440 0x7fd58406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd598007fd0 tx=0x7fd598009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.366 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.377+0000 7fd5a1600640 1 -- 172.21.15.62:0/4004421863 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd590059080 con 0x7fd5a40addb0 2024-04-25T09:03:04.560 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.571+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5a4075080 con 0x7fd5a40addb0 2024-04-25T09:03:04.562 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.573+0000 7fd5a1600640 1 -- 172.21.15.62:0/4004421863 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fd590058980 con 0x7fd5a40addb0 2024-04-25T09:03:04.562 INFO:teuthology.misc.health.smithi062.stdout: 2024-04-25T09:03:04.562 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":12,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:01.890119+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:04.564 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd58406b440 msgr2=0x7fd58406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd58406b440 0x7fd58406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd598007fd0 tx=0x7fd598009040 comp rx=0 tx=0).stop 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a40addb0 msgr2=0x7fd5a415af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a40addb0 0x7fd5a415af20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e9b0 tx=0x7fd59000ee70 comp rx=0 tx=0).stop 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 shutdown_connections 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd58406b440 0x7fd58406d900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a40addb0 0x7fd5a415af20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a4159f00 0x7fd5a415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 --2- 172.21.15.62:0/4004421863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a409dc10 0x7fd5a40b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 >> 172.21.15.62:0/4004421863 conn(0x7fd5a406ab20 msgr2=0x7fd5a406b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.575+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 shutdown_connections 2024-04-25T09:03:04.565 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:04.576+0000 7fd5ab000640 1 -- 172.21.15.62:0/4004421863 wait complete. 2024-04-25T09:03:04.578 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T09:03:08.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:08.183 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:11.385 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:11.582 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.593+0000 7f8757c00640 1 Processor -- start 2024-04-25T09:03:11.582 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.593+0000 7f8757c00640 1 -- start start 2024-04-25T09:03:11.584 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8757c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f875015eeb0 0x7f8750161900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.584 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8757c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f87500b35d0 0x7f87500b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.584 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8757c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8750162270 con 0x7f875015ead0 2024-04-25T09:03:11.584 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8757c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87500d1710 con 0x7f87500b35d0 2024-04-25T09:03:11.584 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8757c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87500d2860 con 0x7f875015eeb0 2024-04-25T09:03:11.585 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8755200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f875015ead0 0x7f87500d2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42144/0 (socket says 172.21.15.62:42144) 2024-04-25T09:03:11.585 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8755200640 1 -- 172.21.15.62:0/3419974885 learned_addr learned my addr 172.21.15.62:0/3419974885 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:11.585 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8755c00640 1 --2- 172.21.15.62:0/3419974885 >> v2:172.21.15.62:3300/0 conn(0x7f87500b35d0 0x7f87500b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.585 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.595+0000 7f8747e00640 1 --2- 172.21.15.62:0/3419974885 >> v2:172.21.15.121:3300/0 conn(0x7f875015eeb0 0x7f8750161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.586 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.596+0000 7f874fe00640 1 -- 172.21.15.62:0/3419974885 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204175753 0 0) 0x7f8750162270 con 0x7f875015ead0 2024-04-25T09:03:11.586 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.596+0000 7f874fe00640 1 -- 172.21.15.62:0/3419974885 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8734003520 con 0x7f875015ead0 2024-04-25T09:03:11.586 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.596+0000 7f8755c00640 1 -- 172.21.15.62:0/3419974885 >> v2:172.21.15.121:3300/0 conn(0x7f875015eeb0 msgr2=0x7f8750161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:11.586 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.596+0000 7f8755c00640 1 --2- 172.21.15.62:0/3419974885 >> v2:172.21.15.121:3300/0 conn(0x7f875015eeb0 0x7f8750161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.586 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.596+0000 7f8755c00640 1 -- 172.21.15.62:0/3419974885 >> v1:172.21.15.62:6789/0 conn(0x7f875015ead0 legacy=0x7f87500d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.586 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.596+0000 7f8755c00640 1 -- 172.21.15.62:0/3419974885 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8750162550 con 0x7f87500b35d0 2024-04-25T09:03:11.586 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.597+0000 7f8755c00640 1 --2- 172.21.15.62:0/3419974885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87500b35d0 0x7f87500b3a00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87500b1560 tx=0x7f874802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9a1bfdf6e0c80a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:11.587 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.597+0000 7f874fe00640 1 -- 172.21.15.62:0/3419974885 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748007740 con 0x7f87500b35d0 2024-04-25T09:03:11.587 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.597+0000 7f874fe00640 1 -- 172.21.15.62:0/3419974885 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87480078e0 con 0x7f87500b35d0 2024-04-25T09:03:11.587 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.597+0000 7f874fe00640 1 -- 172.21.15.62:0/3419974885 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748007cb0 con 0x7f87500b35d0 2024-04-25T09:03:11.587 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.598+0000 7f8757c00640 1 -- 172.21.15.62:0/3419974885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87500b35d0 msgr2=0x7f87500b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.588 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.598+0000 7f8757c00640 1 --2- 172.21.15.62:0/3419974885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87500b35d0 0x7f87500b3a00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87500b1560 tx=0x7f874802a680 comp rx=0 tx=0).stop 2024-04-25T09:03:11.588 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.598+0000 7f8757c00640 1 -- 172.21.15.62:0/3419974885 shutdown_connections 2024-04-25T09:03:11.588 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.598+0000 7f8757c00640 1 --2- 172.21.15.62:0/3419974885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87500b35d0 0x7f87500b3a00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.588 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.598+0000 7f8757c00640 1 --2- 172.21.15.62:0/3419974885 >> v2:172.21.15.121:3300/0 conn(0x7f875015eeb0 0x7f8750161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.588 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.598+0000 7f8757c00640 1 -- 172.21.15.62:0/3419974885 >> 172.21.15.62:0/3419974885 conn(0x7f875006af40 msgr2=0x7f8750072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:11.588 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.598+0000 7f8757c00640 1 -- 172.21.15.62:0/3419974885 shutdown_connections 2024-04-25T09:03:11.588 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.598+0000 7f8757c00640 1 -- 172.21.15.62:0/3419974885 wait complete. 2024-04-25T09:03:11.589 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.600+0000 7f8757c00640 1 Processor -- start 2024-04-25T09:03:11.589 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.600+0000 7f8757c00640 1 -- start start 2024-04-25T09:03:11.591 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.601+0000 7f8757c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87500b35d0 0x7f875014a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.591 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.601+0000 7f8757c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f875015ead0 0x7f8750158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.591 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.601+0000 7f8757c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f875015eeb0 0x7f8750152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.591 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.601+0000 7f8757c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87500d1c30 con 0x7f875015eeb0 2024-04-25T09:03:11.591 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.601+0000 7f8757c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87500b3f40 con 0x7f875015ead0 2024-04-25T09:03:11.591 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.601+0000 7f8757c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8750161e40 con 0x7f87500b35d0 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f875015eeb0 0x7f8750152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f875015eeb0 0x7f8750152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58984/0 (socket says 172.21.15.62:58984) 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 -- 172.21.15.62:0/3081460057 learned_addr learned my addr 172.21.15.62:0/3081460057 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8747e00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f875015ead0 0x7f8750158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755200640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87500b35d0 0x7f875014a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 -- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87500b35d0 msgr2=0x7f875014a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87500b35d0 0x7f875014a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 -- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f875015ead0 msgr2=0x7f8750158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f875015ead0 0x7f8750158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 -- 172.21.15.62:0/3081460057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875014a570 con 0x7f875015eeb0 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8747e00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f875015ead0 0x7f8750158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:11.592 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.602+0000 7f8755c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f875015eeb0 0x7f8750152cb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8748009e50 tx=0x7f8748004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:11.593 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.603+0000 7f874d600640 1 -- 172.21.15.62:0/3081460057 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748040070 con 0x7f875015eeb0 2024-04-25T09:03:11.593 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.603+0000 7f874d600640 1 -- 172.21.15.62:0/3081460057 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8748031bf0 con 0x7f875015eeb0 2024-04-25T09:03:11.593 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.603+0000 7f874d600640 1 -- 172.21.15.62:0/3081460057 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748009450 con 0x7f875015eeb0 2024-04-25T09:03:11.593 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.603+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8750151ef0 con 0x7f875015eeb0 2024-04-25T09:03:11.593 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.603+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87501524b0 con 0x7f875015eeb0 2024-04-25T09:03:11.594 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.604+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875004ef20 con 0x7f875015eeb0 2024-04-25T09:03:11.594 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.605+0000 7f874d600640 1 -- 172.21.15.62:0/3081460057 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8748039040 con 0x7f875015eeb0 2024-04-25T09:03:11.596 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.606+0000 7f874d600640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f872c06b4d0 0x7f872c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.596 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.607+0000 7f874d600640 1 -- 172.21.15.62:0/3081460057 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87480ab570 con 0x7f875015eeb0 2024-04-25T09:03:11.596 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.607+0000 7f8755200640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f872c06b4d0 0x7f872c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.597 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.608+0000 7f8755200640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f872c06b4d0 0x7f872c06d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8740000f60 tx=0x7f8740009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:11.601 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.612+0000 7f874d600640 1 -- 172.21.15.62:0/3081460057 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f874804c030 con 0x7f875015eeb0 2024-04-25T09:03:11.802 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.813+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f875006f7b0 con 0x7f875015eeb0 2024-04-25T09:03:11.804 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.815+0000 7f874d600640 1 -- 172.21.15.62:0/3081460057 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f875014a570 con 0x7f875015eeb0 2024-04-25T09:03:11.804 INFO:teuthology.misc.health.smithi062.stdout: 2024-04-25T09:03:11.804 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":16,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:08.227410+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:11.807 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f872c06b4d0 msgr2=0x7f872c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.807 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f872c06b4d0 0x7f872c06d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8740000f60 tx=0x7f8740009040 comp rx=0 tx=0).stop 2024-04-25T09:03:11.807 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f875015eeb0 msgr2=0x7f8750152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.807 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f875015eeb0 0x7f8750152cb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8748009e50 tx=0x7f8748004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:11.807 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 shutdown_connections 2024-04-25T09:03:11.807 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f872c06b4d0 0x7f872c06d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.808 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f875015eeb0 0x7f8750152cb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.808 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f875015ead0 0x7f8750158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.808 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 --2- 172.21.15.62:0/3081460057 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87500b35d0 0x7f875014a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.808 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 >> 172.21.15.62:0/3081460057 conn(0x7f875006af40 msgr2=0x7f8750157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:11.808 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 shutdown_connections 2024-04-25T09:03:11.808 INFO:teuthology.misc.health.smithi062.stderr:2024-04-25T09:03:11.818+0000 7f8757c00640 1 -- 172.21.15.62:0/3081460057 wait complete. 2024-04-25T09:03:11.822 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T09:03:11.822 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T09:03:11.823 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:03:11.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:03:11.828 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:03:11.828 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:03:11.829 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:03:11.833 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:03:11.834 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:03:11.834 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:03:11.842 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:03:11.842 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:03:11.843 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:03:11.851 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:03:11.851 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T09:03:11.852 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:11.977 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:03:11.985 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:03:12.001 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:03:12.002 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:03:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.045+0000 7f3a6fa00640 1 Processor -- start 2024-04-25T09:03:12.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.045+0000 7f3a6fa00640 1 -- start start 2024-04-25T09:03:12.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a6fa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3a6815dc40 0x7f3a680d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a6fa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a680d0ae0 0x7f3a680d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a6fa00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a680d2620 con 0x7f3a680d3ae0 2024-04-25T09:03:12.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a6fa00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680d2900 con 0x7f3a680d0ae0 2024-04-25T09:03:12.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a6fa00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680d2be0 con 0x7f3a6815dc40 2024-04-25T09:03:12.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a67e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a680d0ae0 0x7f3a680d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a6d000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3a6815dc40 0x7f3a680d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a67e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a680d0ae0 0x7f3a680d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59026/0 (socket says 172.21.15.62:59026) 2024-04-25T09:03:12.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.046+0000 7f3a67e00640 1 -- 172.21.15.62:0/2983888919 learned_addr learned my addr 172.21.15.62:0/2983888919 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:12.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.047+0000 7f3a67e00640 1 -- 172.21.15.62:0/2983888919 >> v2:172.21.15.121:3300/0 conn(0x7f3a6815dc40 msgr2=0x7f3a680d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:12.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.047+0000 7f3a67e00640 1 --2- 172.21.15.62:0/2983888919 >> v2:172.21.15.121:3300/0 conn(0x7f3a6815dc40 0x7f3a680d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.047+0000 7f3a67e00640 1 -- 172.21.15.62:0/2983888919 >> v1:172.21.15.121:6789/0 conn(0x7f3a680d3ae0 legacy=0x7f3a680d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.047+0000 7f3a67400640 1 -- 172.21.15.62:0/2983888919 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143502263 0 0) 0x7f3a680d2620 con 0x7f3a680d3ae0 2024-04-25T09:03:12.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.047+0000 7f3a67e00640 1 -- 172.21.15.62:0/2983888919 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680d2ec0 con 0x7f3a680d0ae0 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.048+0000 7f3a67e00640 1 --2- 172.21.15.62:0/2983888919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a680d0f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009870 tx=0x7f3a5402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b288d28b9fef57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.048+0000 7f3a67400640 1 -- 172.21.15.62:0/2983888919 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54037070 con 0x7f3a680d0ae0 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.048+0000 7f3a67400640 1 -- 172.21.15.62:0/2983888919 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a54032440 con 0x7f3a680d0ae0 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.048+0000 7f3a67400640 1 -- 172.21.15.62:0/2983888919 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54032810 con 0x7f3a680d0ae0 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.049+0000 7f3a6fa00640 1 -- 172.21.15.62:0/2983888919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 msgr2=0x7f3a680d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.049+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/2983888919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a680d0f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009870 tx=0x7f3a5402a2a0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.049+0000 7f3a6fa00640 1 -- 172.21.15.62:0/2983888919 shutdown_connections 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.049+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/2983888919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a680d0f70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.049+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/2983888919 >> v2:172.21.15.121:3300/0 conn(0x7f3a6815dc40 0x7f3a680d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.049+0000 7f3a6fa00640 1 -- 172.21.15.62:0/2983888919 >> 172.21.15.62:0/2983888919 conn(0x7f3a6806b000 msgr2=0x7f3a6806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:12.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.049+0000 7f3a6fa00640 1 -- 172.21.15.62:0/2983888919 shutdown_connections 2024-04-25T09:03:12.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.049+0000 7f3a6fa00640 1 -- 172.21.15.62:0/2983888919 wait complete. 2024-04-25T09:03:12.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.050+0000 7f3a6fa00640 1 Processor -- start 2024-04-25T09:03:12.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.051+0000 7f3a6fa00640 1 -- start start 2024-04-25T09:03:12.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.051+0000 7f3a6fa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a68153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.051+0000 7f3a6fa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a680d3ae0 0x7f3a680c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6fa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a6815dc40 0x7f3a6815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6fa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680d36c0 con 0x7f3a680d0ae0 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6fa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680b7d00 con 0x7f3a680d3ae0 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6fa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6815e3f0 con 0x7f3a6815dc40 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a68153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a68153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59028/0 (socket says 172.21.15.62:59028) 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 -- 172.21.15.62:0/3653094251 learned_addr learned my addr 172.21.15.62:0/3653094251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a67e00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a680d3ae0 0x7f3a680c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6da00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a6815dc40 0x7f3a6815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 -- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a6815dc40 msgr2=0x7f3a6815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a6815dc40 0x7f3a6815f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 -- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a680d3ae0 msgr2=0x7f3a680c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a680d3ae0 0x7f3a680c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 -- 172.21.15.62:0/3653094251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6815e6d0 con 0x7f3a680d0ae0 2024-04-25T09:03:12.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.052+0000 7f3a6d000640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a68153ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00e870 tx=0x7f3a5c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.053+0000 7f3a64c00640 1 -- 172.21.15.62:0/3653094251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c017070 con 0x7f3a680d0ae0 2024-04-25T09:03:12.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.053+0000 7f3a64c00640 1 -- 172.21.15.62:0/3653094251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c00c940 con 0x7f3a680d0ae0 2024-04-25T09:03:12.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.053+0000 7f3a64c00640 1 -- 172.21.15.62:0/3653094251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00cd30 con 0x7f3a680d0ae0 2024-04-25T09:03:12.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.053+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6815e9b0 con 0x7f3a680d0ae0 2024-04-25T09:03:12.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.053+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a680c8720 con 0x7f3a680d0ae0 2024-04-25T09:03:12.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.054+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a680729c0 con 0x7f3a680d0ae0 2024-04-25T09:03:12.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.055+0000 7f3a64c00640 1 -- 172.21.15.62:0/3653094251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a5c01a070 con 0x7f3a680d0ae0 2024-04-25T09:03:12.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.056+0000 7f3a64c00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a4806b4d0 0x7f3a4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.056+0000 7f3a64c00640 1 -- 172.21.15.62:0/3653094251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3a5c089d80 con 0x7f3a680d0ae0 2024-04-25T09:03:12.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.057+0000 7f3a67e00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a4806b4d0 0x7f3a4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.057+0000 7f3a67e00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a4806b4d0 0x7f3a4806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a54034000 tx=0x7f3a5402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.061+0000 7f3a64c00640 1 -- 172.21.15.62:0/3653094251 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a5c058f40 con 0x7f3a680d0ae0 2024-04-25T09:03:12.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.256+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a680c8a10 con 0x7f3a680d0ae0 2024-04-25T09:03:12.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.257+0000 7f3a64c00640 1 -- 172.21.15.62:0/3653094251 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f3a5c058840 con 0x7f3a680d0ae0 2024-04-25T09:03:12.247 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:12.247 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:08.227410+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:12.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a4806b4d0 msgr2=0x7f3a4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a4806b4d0 0x7f3a4806d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a54034000 tx=0x7f3a5402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:12.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 msgr2=0x7f3a68153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a68153ff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00e870 tx=0x7f3a5c00ed30 comp rx=0 tx=0).stop 2024-04-25T09:03:12.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 shutdown_connections 2024-04-25T09:03:12.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a4806b4d0 0x7f3a4806d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a6815dc40 0x7f3a6815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a680d3ae0 0x7f3a680c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 --2- 172.21.15.62:0/3653094251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a680d0ae0 0x7f3a68153ff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 >> 172.21.15.62:0/3653094251 conn(0x7f3a6806b000 msgr2=0x7f3a680b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:12.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 shutdown_connections 2024-04-25T09:03:12.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.260+0000 7f3a6fa00640 1 -- 172.21.15.62:0/3653094251 wait complete. 2024-04-25T09:03:12.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:12.266 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:12.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.479+0000 7fc32d800640 1 Processor -- start 2024-04-25T09:03:12.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.480+0000 7fc32d800640 1 -- start start 2024-04-25T09:03:12.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.481+0000 7fc32d800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc3280d3960 0x7fc3280d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.481+0000 7fc32d800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3280ce780 con 0x7fc32815dd50 2024-04-25T09:03:12.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.481+0000 7fc32d800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3280d42f0 con 0x7fc3280d2fb0 2024-04-25T09:03:12.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.481+0000 7fc32d800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3280d3440 con 0x7fc3280d3960 2024-04-25T09:03:12.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.482+0000 7fc327e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc32815dd50 0x7fc3280d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42188/0 (socket says 172.21.15.62:42188) 2024-04-25T09:03:12.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.482+0000 7fc327e00640 1 -- 172.21.15.62:0/2249650587 learned_addr learned my addr 172.21.15.62:0/2249650587 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:12.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.482+0000 7fc32ce00640 1 --2- 172.21.15.62:0/2249650587 >> v2:172.21.15.62:3300/0 conn(0x7fc3280d3960 0x7fc3280d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.482+0000 7fc326a00640 1 -- 172.21.15.62:0/2249650587 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562271743 0 0) 0x7fc3280ce780 con 0x7fc32815dd50 2024-04-25T09:03:12.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.482+0000 7fc326a00640 1 -- 172.21.15.62:0/2249650587 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc30c003520 con 0x7fc32815dd50 2024-04-25T09:03:12.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.482+0000 7fc32ce00640 1 -- 172.21.15.62:0/2249650587 >> v1:172.21.15.121:6789/0 conn(0x7fc3280d2fb0 legacy=0x7fc3280ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.482+0000 7fc32ce00640 1 -- 172.21.15.62:0/2249650587 >> v1:172.21.15.62:6789/0 conn(0x7fc32815dd50 legacy=0x7fc3280d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.482+0000 7fc32ce00640 1 -- 172.21.15.62:0/2249650587 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280ce1c0 con 0x7fc3280d3960 2024-04-25T09:03:12.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.483+0000 7fc32ce00640 1 --2- 172.21.15.62:0/2249650587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3280d3960 0x7fc3280d3db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3200097a0 tx=0x7fc32002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c98d6b9ba1107d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.484+0000 7fc326a00640 1 -- 172.21.15.62:0/2249650587 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320037070 con 0x7fc3280d3960 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.484+0000 7fc326a00640 1 -- 172.21.15.62:0/2249650587 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc320032440 con 0x7fc3280d3960 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.484+0000 7fc326a00640 1 -- 172.21.15.62:0/2249650587 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320032810 con 0x7fc3280d3960 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.484+0000 7fc32d800640 1 -- 172.21.15.62:0/2249650587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3280d3960 msgr2=0x7fc3280d3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.484+0000 7fc32d800640 1 --2- 172.21.15.62:0/2249650587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3280d3960 0x7fc3280d3db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3200097a0 tx=0x7fc32002a230 comp rx=0 tx=0).stop 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.484+0000 7fc32d800640 1 -- 172.21.15.62:0/2249650587 shutdown_connections 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.484+0000 7fc32d800640 1 --2- 172.21.15.62:0/2249650587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3280d3960 0x7fc3280d3db0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.484+0000 7fc32d800640 1 -- 172.21.15.62:0/2249650587 >> 172.21.15.62:0/2249650587 conn(0x7fc32806b000 msgr2=0x7fc32806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.485+0000 7fc32d800640 1 -- 172.21.15.62:0/2249650587 shutdown_connections 2024-04-25T09:03:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.485+0000 7fc32d800640 1 -- 172.21.15.62:0/2249650587 wait complete. 2024-04-25T09:03:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.486+0000 7fc32d800640 1 Processor -- start 2024-04-25T09:03:12.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.487+0000 7fc32d800640 1 -- start start 2024-04-25T09:03:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.487+0000 7fc32d800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3280d3960 0x7fc3280b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32d800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc32815dd50 0x7fc328157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3281f3430 0x7fc3280b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32d800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280b6c80 con 0x7fc3281f3430 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32d800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280b6b00 con 0x7fc3280d3960 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32d800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280b6e00 con 0x7fc32815dd50 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3281f3430 0x7fc3280b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3281f3430 0x7fc3280b8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59040/0 (socket says 172.21.15.62:59040) 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32ce00640 1 -- 172.21.15.62:0/225850821 learned_addr learned my addr 172.21.15.62:0/225850821 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc327400640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc32815dd50 0x7fc328157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc327e00640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3280d3960 0x7fc3280b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32ce00640 1 -- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc32815dd50 msgr2=0x7fc328157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32ce00640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc32815dd50 0x7fc328157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32ce00640 1 -- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3280d3960 msgr2=0x7fc3280b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32ce00640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3280d3960 0x7fc3280b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.488+0000 7fc32ce00640 1 -- 172.21.15.62:0/225850821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc328158550 con 0x7fc3281f3430 2024-04-25T09:03:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.489+0000 7fc32ce00640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3281f3430 0x7fc3280b8390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc320035040 tx=0x7fc3200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.489+0000 7fc31fe00640 1 -- 172.21.15.62:0/225850821 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320037040 con 0x7fc3281f3430 2024-04-25T09:03:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.489+0000 7fc31fe00640 1 -- 172.21.15.62:0/225850821 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc320042070 con 0x7fc3281f3430 2024-04-25T09:03:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.489+0000 7fc31fe00640 1 -- 172.21.15.62:0/225850821 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200308a0 con 0x7fc3281f3430 2024-04-25T09:03:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.489+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc328158830 con 0x7fc3281f3430 2024-04-25T09:03:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.489+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3280b3170 con 0x7fc3281f3430 2024-04-25T09:03:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.490+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32804ef20 con 0x7fc3281f3430 2024-04-25T09:03:12.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.491+0000 7fc31fe00640 1 -- 172.21.15.62:0/225850821 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc320030a40 con 0x7fc3281f3430 2024-04-25T09:03:12.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.492+0000 7fc31fe00640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc30406b500 0x7fc30406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.493+0000 7fc31fe00640 1 -- 172.21.15.62:0/225850821 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc3200aa830 con 0x7fc3281f3430 2024-04-25T09:03:12.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.493+0000 7fc327e00640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc30406b500 0x7fc30406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.494+0000 7fc327e00640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc30406b500 0x7fc30406d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc318001560 tx=0x7fc318009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.498+0000 7fc31fe00640 1 -- 172.21.15.62:0/225850821 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc320079960 con 0x7fc3281f3430 2024-04-25T09:03:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.696+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3280b9950 con 0x7fc3281f3430 2024-04-25T09:03:12.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.697+0000 7fc31fe00640 1 -- 172.21.15.62:0/225850821 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc320079260 con 0x7fc3281f3430 2024-04-25T09:03:12.686 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.699+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc30406b500 msgr2=0x7fc30406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.699+0000 7fc32d800640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc30406b500 0x7fc30406d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc318001560 tx=0x7fc318009040 comp rx=0 tx=0).stop 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3281f3430 msgr2=0x7fc3280b8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3281f3430 0x7fc3280b8390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc320035040 tx=0x7fc3200046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 shutdown_connections 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc30406b500 0x7fc30406d9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3281f3430 0x7fc3280b8390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc32815dd50 0x7fc328157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 --2- 172.21.15.62:0/225850821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3280d3960 0x7fc3280b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 >> 172.21.15.62:0/225850821 conn(0x7fc32806b000 msgr2=0x7fc3280cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 shutdown_connections 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.700+0000 7fc32d800640 1 -- 172.21.15.62:0/225850821 wait complete. 2024-04-25T09:03:12.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:12.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.928+0000 7fa9a4600640 1 Processor -- start 2024-04-25T09:03:12.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.929+0000 7fa9a4600640 1 -- start start 2024-04-25T09:03:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.930+0000 7fa9a4600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa99c15bd10 0x7fa99c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.930+0000 7fa9a4600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0aeed0 0x7fa99c0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.930+0000 7fa9a4600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c06ca70 con 0x7fa99c0cccf0 2024-04-25T09:03:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.930+0000 7fa9a4600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99c06d660 con 0x7fa99c15bd10 2024-04-25T09:03:12.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.930+0000 7fa9a4600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c15c0f0 con 0x7fa99c0aeed0 2024-04-25T09:03:12.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.931+0000 7fa9a1c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa99c15bd10 0x7fa99c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.931+0000 7fa9a1c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa99c15bd10 0x7fa99c0cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:49258/0 (socket says 172.21.15.62:49258) 2024-04-25T09:03:12.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.931+0000 7fa9a2600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0aeed0 0x7fa99c0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.931+0000 7fa9a1c00640 1 -- 172.21.15.62:0/1677398664 learned_addr learned my addr 172.21.15.62:0/1677398664 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.932+0000 7fa9a2600640 1 -- 172.21.15.62:0/1677398664 >> v2:172.21.15.121:3300/0 conn(0x7fa99c15bd10 msgr2=0x7fa99c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.932+0000 7fa9a2600640 1 --2- 172.21.15.62:0/1677398664 >> v2:172.21.15.121:3300/0 conn(0x7fa99c15bd10 0x7fa99c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.932+0000 7fa9a2600640 1 -- 172.21.15.62:0/1677398664 >> v1:172.21.15.121:6789/0 conn(0x7fa99c0cccf0 legacy=0x7fa99c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.932+0000 7fa9a2600640 1 -- 172.21.15.62:0/1677398664 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c06cd30 con 0x7fa99c0aeed0 2024-04-25T09:03:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.932+0000 7fa9a1c00640 1 --2- 172.21.15.62:0/1677398664 >> v2:172.21.15.121:3300/0 conn(0x7fa99c15bd10 0x7fa99c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:12.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.933+0000 7fa9a2600640 1 --2- 172.21.15.62:0/1677398664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0aeed0 0x7fa99c0cd6c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa99c06d7e0 tx=0x7fa99402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a63302f628b79c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa99be00640 1 -- 172.21.15.62:0/1677398664 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99402ae60 con 0x7fa99c0aeed0 2024-04-25T09:03:12.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa99be00640 1 -- 172.21.15.62:0/1677398664 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa994007740 con 0x7fa99c0aeed0 2024-04-25T09:03:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa99be00640 1 -- 172.21.15.62:0/1677398664 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994007b10 con 0x7fa99c0aeed0 2024-04-25T09:03:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa9a4600640 1 -- 172.21.15.62:0/1677398664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0aeed0 msgr2=0x7fa99c0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa9a4600640 1 --2- 172.21.15.62:0/1677398664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0aeed0 0x7fa99c0cd6c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa99c06d7e0 tx=0x7fa99402a640 comp rx=0 tx=0).stop 2024-04-25T09:03:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa9a4600640 1 -- 172.21.15.62:0/1677398664 shutdown_connections 2024-04-25T09:03:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa9a4600640 1 --2- 172.21.15.62:0/1677398664 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0aeed0 0x7fa99c0cd6c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa9a4600640 1 --2- 172.21.15.62:0/1677398664 >> v2:172.21.15.121:3300/0 conn(0x7fa99c15bd10 0x7fa99c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa9a4600640 1 -- 172.21.15.62:0/1677398664 >> 172.21.15.62:0/1677398664 conn(0x7fa99c0668f0 msgr2=0x7fa99c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa9a4600640 1 -- 172.21.15.62:0/1677398664 shutdown_connections 2024-04-25T09:03:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.934+0000 7fa9a4600640 1 -- 172.21.15.62:0/1677398664 wait complete. 2024-04-25T09:03:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.936+0000 7fa9a4600640 1 Processor -- start 2024-04-25T09:03:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.936+0000 7fa9a4600640 1 -- start start 2024-04-25T09:03:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.937+0000 7fa9a4600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0aeed0 0x7fa99c1f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.937+0000 7fa9a4600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0cccf0 0x7fa99c1452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.937+0000 7fa9a4600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99c15bd10 0x7fa99c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.937+0000 7fa9a4600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c0d01e0 con 0x7fa99c15bd10 2024-04-25T09:03:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.937+0000 7fa9a4600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c06d2f0 con 0x7fa99c0aeed0 2024-04-25T09:03:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.937+0000 7fa9a4600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c06daf0 con 0x7fa99c0cccf0 2024-04-25T09:03:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99c15bd10 0x7fa99c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99c15bd10 0x7fa99c1563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59048/0 (socket says 172.21.15.62:59048) 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 -- 172.21.15.62:0/2969462603 learned_addr learned my addr 172.21.15.62:0/2969462603 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a1c00640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0aeed0 0x7fa99c1f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a1200640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0cccf0 0x7fa99c1452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 -- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0cccf0 msgr2=0x7fa99c1452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0cccf0 0x7fa99c1452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 -- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0aeed0 msgr2=0x7fa99c1f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0aeed0 0x7fa99c1f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 -- 172.21.15.62:0/2969462603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c151a00 con 0x7fa99c15bd10 2024-04-25T09:03:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.938+0000 7fa9a2600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99c15bd10 0x7fa99c1563c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa99402ac30 tx=0x7fa994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.939+0000 7fa999600640 1 -- 172.21.15.62:0/2969462603 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99403f070 con 0x7fa99c15bd10 2024-04-25T09:03:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.939+0000 7fa999600640 1 -- 172.21.15.62:0/2969462603 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9940319c0 con 0x7fa99c15bd10 2024-04-25T09:03:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.939+0000 7fa999600640 1 -- 172.21.15.62:0/2969462603 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa994031b90 con 0x7fa99c15bd10 2024-04-25T09:03:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.939+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c154aa0 con 0x7fa99c15bd10 2024-04-25T09:03:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.939+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c151ca0 con 0x7fa99c15bd10 2024-04-25T09:03:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.940+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c072c70 con 0x7fa99c15bd10 2024-04-25T09:03:12.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.941+0000 7fa999600640 1 -- 172.21.15.62:0/2969462603 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa99403b410 con 0x7fa99c15bd10 2024-04-25T09:03:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.942+0000 7fa999600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa98006b4d0 0x7fa98006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.942+0000 7fa999600640 1 -- 172.21.15.62:0/2969462603 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa9940ab520 con 0x7fa99c15bd10 2024-04-25T09:03:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.943+0000 7fa9a1c00640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa98006b4d0 0x7fa98006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.944+0000 7fa9a1c00640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa98006b4d0 0x7fa98006d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa990009b70 tx=0x7fa990008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:12.946+0000 7fa999600640 1 -- 172.21.15.62:0/2969462603 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa99407a650 con 0x7fa99c15bd10 2024-04-25T09:03:13.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.196+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa99c0cb780 con 0x7fa99c15bd10 2024-04-25T09:03:13.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.197+0000 7fa999600640 1 -- 172.21.15.62:0/2969462603 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7fa994079f50 con 0x7fa99c15bd10 2024-04-25T09:03:13.186 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:13.187 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T09:03:13.187 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 3 2024-04-25T09:03:13.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa98006b4d0 msgr2=0x7fa98006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa98006b4d0 0x7fa98006d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa990009b70 tx=0x7fa990008040 comp rx=0 tx=0).stop 2024-04-25T09:03:13.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99c15bd10 msgr2=0x7fa99c1563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99c15bd10 0x7fa99c1563c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa99402ac30 tx=0x7fa994004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 shutdown_connections 2024-04-25T09:03:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa98006b4d0 0x7fa98006d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa99c15bd10 0x7fa99c1563c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa99c0cccf0 0x7fa99c1452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 --2- 172.21.15.62:0/2969462603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa99c0aeed0 0x7fa99c1f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.200+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 >> 172.21.15.62:0/2969462603 conn(0x7fa99c0668f0 msgr2=0x7fa99c14fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.201+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 shutdown_connections 2024-04-25T09:03:13.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.201+0000 7fa9a4600640 1 -- 172.21.15.62:0/2969462603 wait complete. 2024-04-25T09:03:13.208 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:13.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.417+0000 7f5d9e400640 1 Processor -- start 2024-04-25T09:03:13.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.418+0000 7f5d9e400640 1 -- start start 2024-04-25T09:03:13.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.419+0000 7f5d9e400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d9806d900 0x7f5d9815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.419+0000 7f5d9e400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d980cee40 0x7f5d980b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9e400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9815faa0 con 0x7f5d98144c10 2024-04-25T09:03:13.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9e400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d98144ff0 con 0x7f5d980cee40 2024-04-25T09:03:13.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9e400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d98160340 con 0x7f5d9806d900 2024-04-25T09:03:13.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d980cee40 0x7f5d980b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d980cee40 0x7f5d980b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59058/0 (socket says 172.21.15.62:59058) 2024-04-25T09:03:13.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9da00640 1 -- 172.21.15.62:0/80139197 learned_addr learned my addr 172.21.15.62:0/80139197 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:13.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d8fe00640 1 --2- 172.21.15.62:0/80139197 >> v2:172.21.15.121:3300/0 conn(0x7f5d9806d900 0x7f5d9815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9da00640 1 -- 172.21.15.62:0/80139197 >> v2:172.21.15.121:3300/0 conn(0x7f5d9806d900 msgr2=0x7f5d9815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9da00640 1 --2- 172.21.15.62:0/80139197 >> v2:172.21.15.121:3300/0 conn(0x7f5d9806d900 0x7f5d9815f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9da00640 1 -- 172.21.15.62:0/80139197 >> v1:172.21.15.121:6789/0 conn(0x7f5d98144c10 legacy=0x7f5d9806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.420+0000 7f5d9da00640 1 -- 172.21.15.62:0/80139197 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9815fd80 con 0x7f5d980cee40 2024-04-25T09:03:13.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.421+0000 7f5d9da00640 1 --2- 172.21.15.62:0/80139197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d980b4650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d9806e680 tx=0x7f5d9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca939a7416b6de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d97e00640 1 -- 172.21.15.62:0/80139197 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90037070 con 0x7f5d980cee40 2024-04-25T09:03:13.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d97e00640 1 -- 172.21.15.62:0/80139197 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d90032440 con 0x7f5d980cee40 2024-04-25T09:03:13.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d97e00640 1 -- 172.21.15.62:0/80139197 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90032810 con 0x7f5d980cee40 2024-04-25T09:03:13.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d9e400640 1 -- 172.21.15.62:0/80139197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 msgr2=0x7f5d980b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d9e400640 1 --2- 172.21.15.62:0/80139197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d980b4650 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d9806e680 tx=0x7f5d9002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d9e400640 1 -- 172.21.15.62:0/80139197 shutdown_connections 2024-04-25T09:03:13.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d9e400640 1 --2- 172.21.15.62:0/80139197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d980b4650 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d9e400640 1 --2- 172.21.15.62:0/80139197 >> v2:172.21.15.121:3300/0 conn(0x7f5d9806d900 0x7f5d9815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.422+0000 7f5d9e400640 1 -- 172.21.15.62:0/80139197 >> 172.21.15.62:0/80139197 conn(0x7f5d980b14d0 msgr2=0x7f5d980b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.423+0000 7f5d9e400640 1 -- 172.21.15.62:0/80139197 shutdown_connections 2024-04-25T09:03:13.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.423+0000 7f5d9e400640 1 -- 172.21.15.62:0/80139197 wait complete. 2024-04-25T09:03:13.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.424+0000 7f5d9e400640 1 Processor -- start 2024-04-25T09:03:13.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.424+0000 7f5d9e400640 1 -- start start 2024-04-25T09:03:13.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.425+0000 7f5d9e400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d9806d900 0x7f5d980cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.425+0000 7f5d9e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d9815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.425+0000 7f5d9e400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d98144c10 0x7f5d9815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.425+0000 7f5d9e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d980cf270 con 0x7f5d980cee40 2024-04-25T09:03:13.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.425+0000 7f5d9e400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d980b41c0 con 0x7f5d98144c10 2024-04-25T09:03:13.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.425+0000 7f5d9e400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d980b4b90 con 0x7f5d9806d900 2024-04-25T09:03:13.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d9815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d9815ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59060/0 (socket says 172.21.15.62:59060) 2024-04-25T09:03:13.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d8fe00640 1 -- 172.21.15.62:0/410516698 learned_addr learned my addr 172.21.15.62:0/410516698 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:13.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d8fe00640 1 -- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d9806d900 msgr2=0x7f5d980cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d9d000640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d9806d900 0x7f5d980cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d9da00640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d98144c10 0x7f5d9815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d8fe00640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d9806d900 0x7f5d980cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d8fe00640 1 -- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d98144c10 msgr2=0x7f5d9815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d8fe00640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d98144c10 0x7f5d9815b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.426+0000 7f5d8fe00640 1 -- 172.21.15.62:0/410516698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d980cb320 con 0x7f5d980cee40 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.427+0000 7f5d8fe00640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d9815ad60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000d800 tx=0x7f5d8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.427+0000 7f5d95600640 1 -- 172.21.15.62:0/410516698 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000b6c0 con 0x7f5d980cee40 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.427+0000 7f5d95600640 1 -- 172.21.15.62:0/410516698 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d80019070 con 0x7f5d980cee40 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.427+0000 7f5d95600640 1 -- 172.21.15.62:0/410516698 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000ba80 con 0x7f5d980cee40 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.427+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d980cb600 con 0x7f5d980cee40 2024-04-25T09:03:13.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.427+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d98141170 con 0x7f5d980cee40 2024-04-25T09:03:13.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.428+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9804ef20 con 0x7f5d980cee40 2024-04-25T09:03:13.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.429+0000 7f5d95600640 1 -- 172.21.15.62:0/410516698 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d8000bc20 con 0x7f5d980cee40 2024-04-25T09:03:13.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.430+0000 7f5d95600640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d7406b4d0 0x7f5d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.430+0000 7f5d95600640 1 -- 172.21.15.62:0/410516698 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d8008a400 con 0x7f5d980cee40 2024-04-25T09:03:13.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.431+0000 7f5d9d000640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d7406b4d0 0x7f5d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.432+0000 7f5d9d000640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d7406b4d0 0x7f5d7406d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007dc0 tx=0x7f5d880072f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.434+0000 7f5d95600640 1 -- 172.21.15.62:0/410516698 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d80059530 con 0x7f5d980cee40 2024-04-25T09:03:13.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.643+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d9815d1e0 con 0x7f5d980cee40 2024-04-25T09:03:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.645+0000 7f5d95600640 1 -- 172.21.15.62:0/410516698 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f5d80058e30 con 0x7f5d980cee40 2024-04-25T09:03:13.635 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:13.635 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:08.227410+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d7406b4d0 msgr2=0x7f5d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d7406b4d0 0x7f5d7406d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007dc0 tx=0x7f5d880072f0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 msgr2=0x7f5d9815ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d9815ad60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000d800 tx=0x7f5d8000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 shutdown_connections 2024-04-25T09:03:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d7406b4d0 0x7f5d7406d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d980cee40 0x7f5d9815ad60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d98144c10 0x7f5d9815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 --2- 172.21.15.62:0/410516698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d9806d900 0x7f5d980cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 >> 172.21.15.62:0/410516698 conn(0x7f5d980b14d0 msgr2=0x7f5d980b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 shutdown_connections 2024-04-25T09:03:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.648+0000 7f5d9e400640 1 -- 172.21.15.62:0/410516698 wait complete. 2024-04-25T09:03:13.656 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:13.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.872+0000 7f85d4a00640 1 Processor -- start 2024-04-25T09:03:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.873+0000 7f85d4a00640 1 -- start start 2024-04-25T09:03:13.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.874+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d006d2c0 0x7f85d00ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.874+0000 7f85d4a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f85d00b31a0 0x7f85d00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.874+0000 7f85d4a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d00cf000 con 0x7f85d006d6a0 2024-04-25T09:03:13.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.874+0000 7f85d4a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d00cf2e0 con 0x7f85d00b31a0 2024-04-25T09:03:13.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.874+0000 7f85d4a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00b26c0 con 0x7f85d006d2c0 2024-04-25T09:03:13.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.874+0000 7f85cea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f85d006d6a0 0x7f85d00ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42200/0 (socket says 172.21.15.62:42200) 2024-04-25T09:03:13.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.874+0000 7f85cea00640 1 -- 172.21.15.62:0/2332303835 learned_addr learned my addr 172.21.15.62:0/2332303835 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85cf400640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d006d2c0 0x7f85d00ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85cfe00640 1 --2- 172.21.15.62:0/2332303835 >> v2:172.21.15.121:3300/0 conn(0x7f85d00b31a0 0x7f85d00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484897152 0 0) 0x7f85d00cf000 con 0x7f85d006d6a0 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b4003520 con 0x7f85d006d6a0 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2748046106 0 0) 0x7f85b4003520 con 0x7f85d006d6a0 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d006d2c0 msgr2=0x7f85d00ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d006d2c0 0x7f85d00ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 >> v2:172.21.15.121:3300/0 conn(0x7f85d00b31a0 msgr2=0x7f85d00b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 --2- 172.21.15.62:0/2332303835 >> v2:172.21.15.121:3300/0 conn(0x7f85d00b31a0 0x7f85d00b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00b29a0 con 0x7f85d006d6a0 2024-04-25T09:03:13.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f85bc001670 con 0x7f85d006d6a0 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 >> v1:172.21.15.62:6789/0 conn(0x7f85d006d6a0 legacy=0x7f85d00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d006d2c0 0x7f85b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b4004d40 0x7f85b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00cf2e0 con 0x7f85b4003f30 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00b26c0 con 0x7f85b4004d40 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.875+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b400b9c0 con 0x7f85d006d2c0 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cea00640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cf400640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d006d2c0 0x7f85b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cfe00640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b4004d40 0x7f85b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cea00640 1 -- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d006d2c0 msgr2=0x7f85b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cea00640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d006d2c0 0x7f85b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cea00640 1 -- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b4004d40 msgr2=0x7f85b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cea00640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b4004d40 0x7f85b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cea00640 1 -- 172.21.15.62:0/2332303835 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4003520 con 0x7f85b4003f30 2024-04-25T09:03:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.876+0000 7f85cfe00640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b4004d40 0x7f85b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:13.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.877+0000 7f85cea00640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85b4004800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c290 tx=0x7f85b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11df6d28ff71dcd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.877+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc007d50 con 0x7f85b4003f30 2024-04-25T09:03:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.877+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc02e070 con 0x7f85b4003f30 2024-04-25T09:03:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.877+0000 7f85ce000640 1 -- 172.21.15.62:0/2332303835 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc005d70 con 0x7f85b4003f30 2024-04-25T09:03:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.878+0000 7f85d4a00640 1 -- 172.21.15.62:0/2332303835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 msgr2=0x7f85b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.878+0000 7f85d4a00640 1 --2- 172.21.15.62:0/2332303835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85b4004800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c290 tx=0x7f85b4005830 comp rx=0 tx=0).stop 2024-04-25T09:03:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.878+0000 7f85cfe00640 1 -- 172.21.15.62:0/2332303835 reap_dead start 2024-04-25T09:03:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.878+0000 7f85d4a00640 1 -- 172.21.15.62:0/2332303835 shutdown_connections 2024-04-25T09:03:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.878+0000 7f85d4a00640 1 -- 172.21.15.62:0/2332303835 >> 172.21.15.62:0/2332303835 conn(0x7f85d0066f20 msgr2=0x7f85d00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.878+0000 7f85d4a00640 1 -- 172.21.15.62:0/2332303835 shutdown_connections 2024-04-25T09:03:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.878+0000 7f85d4a00640 1 -- 172.21.15.62:0/2332303835 wait complete. 2024-04-25T09:03:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.880+0000 7f85d4a00640 1 Processor -- start 2024-04-25T09:03:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.880+0000 7f85d4a00640 1 -- start start 2024-04-25T09:03:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.881+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85d01f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.881+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d00c5b20 0x7f85d00c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.881+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85d00adbc0 0x7f85d00ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.881+0000 7f85d4a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d01448d0 con 0x7f85b4003f30 2024-04-25T09:03:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.881+0000 7f85d4a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d006db20 con 0x7f85d00adbc0 2024-04-25T09:03:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.881+0000 7f85d4a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0144a50 con 0x7f85d00c5b20 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.881+0000 7f85cf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85d01f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85d01f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59074/0 (socket says 172.21.15.62:59074) 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cf400640 1 -- 172.21.15.62:0/2460658474 learned_addr learned my addr 172.21.15.62:0/2460658474 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cea00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d00c5b20 0x7f85d00c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cf400640 1 -- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d00c5b20 msgr2=0x7f85d00c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cf400640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d00c5b20 0x7f85d00c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cf400640 1 -- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85d00adbc0 msgr2=0x7f85d00ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cf400640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85d00adbc0 0x7f85d00ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cf400640 1 -- 172.21.15.62:0/2460658474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00ae980 con 0x7f85b4003f30 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cea00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d00c5b20 0x7f85d00c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85cf400640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85d01f1080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007f60 tx=0x7f85c0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85c7400640 1 -- 172.21.15.62:0/2460658474 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c001a020 con 0x7f85b4003f30 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.882+0000 7f85c7400640 1 -- 172.21.15.62:0/2460658474 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c0016470 con 0x7f85b4003f30 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.883+0000 7f85c7400640 1 -- 172.21.15.62:0/2460658474 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0016840 con 0x7f85b4003f30 2024-04-25T09:03:13.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.883+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d00aec60 con 0x7f85b4003f30 2024-04-25T09:03:13.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.883+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d00aba20 con 0x7f85b4003f30 2024-04-25T09:03:13.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.884+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d006abe0 con 0x7f85b4003f30 2024-04-25T09:03:13.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.885+0000 7f85c7400640 1 -- 172.21.15.62:0/2460658474 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85c0005d10 con 0x7f85b4003f30 2024-04-25T09:03:13.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.886+0000 7f85c7400640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f85b006b500 0x7f85b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.886+0000 7f85c7400640 1 -- 172.21.15.62:0/2460658474 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f85c008af30 con 0x7f85b4003f30 2024-04-25T09:03:13.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.886+0000 7f85cea00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f85b006b500 0x7f85b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.887+0000 7f85cea00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f85b006b500 0x7f85b006d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85bc033000 tx=0x7f85bc009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:13.890+0000 7f85c7400640 1 -- 172.21.15.62:0/2460658474 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85c0024030 con 0x7f85b4003f30 2024-04-25T09:03:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.095+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85d004ef20 con 0x7f85b4003f30 2024-04-25T09:03:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.096+0000 7f85c7400640 1 -- 172.21.15.62:0/2460658474 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85c005a060 con 0x7f85b4003f30 2024-04-25T09:03:14.085 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.099+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f85b006b500 msgr2=0x7f85b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.099+0000 7f85d4a00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f85b006b500 0x7f85b006d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85bc033000 tx=0x7f85bc009030 comp rx=0 tx=0).stop 2024-04-25T09:03:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.099+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 msgr2=0x7f85d01f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.099+0000 7f85d4a00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85d01f1080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007f60 tx=0x7f85c0007f90 comp rx=0 tx=0).stop 2024-04-25T09:03:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.100+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 shutdown_connections 2024-04-25T09:03:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.100+0000 7f85d4a00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f85b006b500 0x7f85b006d9c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.100+0000 7f85d4a00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85d00adbc0 0x7f85d00ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.100+0000 7f85d4a00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f85d00c5b20 0x7f85d00c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.100+0000 7f85d4a00640 1 --2- 172.21.15.62:0/2460658474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85b4003f30 0x7f85d01f1080 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.100+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 >> 172.21.15.62:0/2460658474 conn(0x7f85d0066f20 msgr2=0x7f85d015bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:14.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.100+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 shutdown_connections 2024-04-25T09:03:14.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.100+0000 7f85d4a00640 1 -- 172.21.15.62:0/2460658474 wait complete. 2024-04-25T09:03:14.106 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:03:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.323+0000 7f1a7b200640 1 Processor -- start 2024-04-25T09:03:14.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.324+0000 7f1a7b200640 1 -- start start 2024-04-25T09:03:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.325+0000 7f1a7b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1a7c0b60b0 0x7f1a7c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.325+0000 7f1a7b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c0b85a0 con 0x7f1a7c15dca0 2024-04-25T09:03:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.325+0000 7f1a7b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7c0b8880 con 0x7f1a7c0d3070 2024-04-25T09:03:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.325+0000 7f1a7b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a7c0d1bd0 con 0x7f1a7c0b60b0 2024-04-25T09:03:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.326+0000 7f1a79e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1a7c15dca0 0x7f1a7c0b59a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42212/0 (socket says 172.21.15.62:42212) 2024-04-25T09:03:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.326+0000 7f1a79e00640 1 -- 172.21.15.62:0/4094771723 learned_addr learned my addr 172.21.15.62:0/4094771723 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.326+0000 7f1a79400640 1 --2- 172.21.15.62:0/4094771723 >> v2:172.21.15.121:3300/0 conn(0x7f1a7c0b60b0 0x7f1a7c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.326+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705535888 0 0) 0x7f1a7c0b85a0 con 0x7f1a7c15dca0 2024-04-25T09:03:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.326+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a60003520 con 0x7f1a7c15dca0 2024-04-25T09:03:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.326+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2069568539 0 0) 0x7f1a60003520 con 0x7f1a7c15dca0 2024-04-25T09:03:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.326+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 >> v2:172.21.15.121:3300/0 conn(0x7f1a7c0b60b0 msgr2=0x7f1a7c0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.326+0000 7f1a78a00640 1 --2- 172.21.15.62:0/4094771723 >> v2:172.21.15.121:3300/0 conn(0x7f1a7c0b60b0 0x7f1a7c0b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 >> v1:172.21.15.121:6789/0 conn(0x7f1a7c0d3070 legacy=0x7f1a7c0b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c0b8b60 con 0x7f1a7c15dca0 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1a6c0019f0 con 0x7f1a7c15dca0 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 >> v1:172.21.15.62:6789/0 conn(0x7f1a7c15dca0 legacy=0x7f1a7c0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a60003a70 0x7f1a60003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60004800 0x7f1a60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a60005210 0x7f1a6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0d1bd0 con 0x7f1a60005210 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6000c120 con 0x7f1a60003a70 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6000c320 con 0x7f1a60004800 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a7a800640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a60005210 0x7f1a6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a79400640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60004800 0x7f1a60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a79e00640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a60003a70 0x7f1a60003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a7a800640 1 -- 172.21.15.62:0/4094771723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60004800 msgr2=0x7f1a60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a7a800640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60004800 0x7f1a60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a7a800640 1 -- 172.21.15.62:0/4094771723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a60003a70 msgr2=0x7f1a60003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a7a800640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a60003a70 0x7f1a60003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.327+0000 7f1a7a800640 1 -- 172.21.15.62:0/4094771723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60003520 con 0x7f1a60005210 2024-04-25T09:03:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.328+0000 7f1a7a800640 1 -- 172.21.15.62:0/4094771723 reap_dead start 2024-04-25T09:03:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.328+0000 7f1a7a800640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a60005210 0x7f1a6000b9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a74001750 tx=0x7f1a60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7b6851380e4a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.329+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74001370 con 0x7f1a60005210 2024-04-25T09:03:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.329+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a74005310 con 0x7f1a60005210 2024-04-25T09:03:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.329+0000 7f1a78a00640 1 -- 172.21.15.62:0/4094771723 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7402abe0 con 0x7f1a60005210 2024-04-25T09:03:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.329+0000 7f1a7b200640 1 -- 172.21.15.62:0/4094771723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a60005210 msgr2=0x7f1a6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.329+0000 7f1a7b200640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a60005210 0x7f1a6000b9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1a74001750 tx=0x7f1a60005830 comp rx=0 tx=0).stop 2024-04-25T09:03:14.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.329+0000 7f1a7b200640 1 -- 172.21.15.62:0/4094771723 shutdown_connections 2024-04-25T09:03:14.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.329+0000 7f1a7b200640 1 --2- 172.21.15.62:0/4094771723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a60005210 0x7f1a6000b9c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.329+0000 7f1a7b200640 1 -- 172.21.15.62:0/4094771723 >> 172.21.15.62:0/4094771723 conn(0x7f1a7c06b000 msgr2=0x7f1a7c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:14.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.330+0000 7f1a7b200640 1 -- 172.21.15.62:0/4094771723 shutdown_connections 2024-04-25T09:03:14.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.330+0000 7f1a7b200640 1 -- 172.21.15.62:0/4094771723 wait complete. 2024-04-25T09:03:14.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.331+0000 7f1a7b200640 1 Processor -- start 2024-04-25T09:03:14.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.331+0000 7f1a7b200640 1 -- start start 2024-04-25T09:03:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.332+0000 7f1a7b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60005210 0x7f1a7c0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.332+0000 7f1a7b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c1f4060 0x7f1a7c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.332+0000 7f1a7b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a7c1f44e0 0x7f1a7c0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.332+0000 7f1a7b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0b5b20 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.332+0000 7f1a7b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0b59a0 con 0x7f1a7c1f4060 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.332+0000 7f1a7b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c0b5ca0 con 0x7f1a60005210 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a7c1f44e0 0x7f1a7c0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a7c1f44e0 0x7f1a7c0cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59084/0 (socket says 172.21.15.62:59084) 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 -- 172.21.15.62:0/369157643 learned_addr learned my addr 172.21.15.62:0/369157643 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a79e00640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60005210 0x7f1a7c0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a79400640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c1f4060 0x7f1a7c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 -- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60005210 msgr2=0x7f1a7c0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60005210 0x7f1a7c0cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 -- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c1f4060 msgr2=0x7f1a7c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c1f4060 0x7f1a7c0ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 -- 172.21.15.62:0/369157643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c15f1c0 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a79e00640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60005210 0x7f1a7c0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.333+0000 7f1a7a800640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a7c1f44e0 0x7f1a7c0cd290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a7402a590 tx=0x7f1a74001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.334+0000 7f1a72000640 1 -- 172.21.15.62:0/369157643 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74001370 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.334+0000 7f1a72000640 1 -- 172.21.15.62:0/369157643 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7402ae20 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.334+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c15e120 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.334+0000 7f1a72000640 1 -- 172.21.15.62:0/369157643 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74007a30 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.334+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c15e6e0 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.335+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a7c075070 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.336+0000 7f1a72000640 1 -- 172.21.15.62:0/369157643 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a74007030 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.337+0000 7f1a72000640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a5c06b4d0 0x7f1a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.337+0000 7f1a72000640 1 -- 172.21.15.62:0/369157643 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1a740a9180 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.338+0000 7f1a79e00640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a5c06b4d0 0x7f1a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.338+0000 7f1a79e00640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a5c06b4d0 0x7f1a5c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0ccce0 tx=0x7f1a6c001310 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.341+0000 7f1a72000640 1 -- 172.21.15.62:0/369157643 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a74078350 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.540+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1a7c15ad90 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.541+0000 7f1a72000640 1 -- 172.21.15.62:0/369157643 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1a740a7e20 con 0x7f1a7c1f44e0 2024-04-25T09:03:14.530 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:14.531 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:03:14.531 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:03:14.531 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:03:14.531 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:03:14.531 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:03:14.531 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:03:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a5c06b4d0 msgr2=0x7f1a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a5c06b4d0 0x7f1a5c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0ccce0 tx=0x7f1a6c001310 comp rx=0 tx=0).stop 2024-04-25T09:03:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a7c1f44e0 msgr2=0x7f1a7c0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a7c1f44e0 0x7f1a7c0cd290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1a7402a590 tx=0x7f1a74001120 comp rx=0 tx=0).stop 2024-04-25T09:03:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 shutdown_connections 2024-04-25T09:03:14.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a5c06b4d0 0x7f1a5c06d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a7c1f44e0 0x7f1a7c0cd290 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c1f4060 0x7f1a7c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 --2- 172.21.15.62:0/369157643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a60005210 0x7f1a7c0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 >> 172.21.15.62:0/369157643 conn(0x7f1a7c06b000 msgr2=0x7f1a7c0d4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 shutdown_connections 2024-04-25T09:03:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.544+0000 7f1a7b200640 1 -- 172.21.15.62:0/369157643 wait complete. 2024-04-25T09:03:14.551 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:03:14.551 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:03:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.777+0000 7fb826e00640 1 Processor -- start 2024-04-25T09:03:14.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.777+0000 7fb826e00640 1 -- start start 2024-04-25T09:03:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.779+0000 7fb826e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820161650 0x7fb8200d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.779+0000 7fb826e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8200d1480 con 0x7fb8200b5c70 2024-04-25T09:03:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.779+0000 7fb826e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8200d11a0 con 0x7fb820161270 2024-04-25T09:03:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.779+0000 7fb826e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200d3900 con 0x7fb820161650 2024-04-25T09:03:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.779+0000 7fb825000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb8200b5c70 0x7fb8200d1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40932/0 (socket says 172.21.15.62:40932) 2024-04-25T09:03:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.779+0000 7fb825000640 1 -- 172.21.15.62:0/2072753250 learned_addr learned my addr 172.21.15.62:0/2072753250 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb826400640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820161650 0x7fb8200d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256542377 0 0) 0x7fb8200d1480 con 0x7fb8200b5c70 2024-04-25T09:03:14.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb808003520 con 0x7fb8200b5c70 2024-04-25T09:03:14.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2639775023 0 0) 0x7fb808003520 con 0x7fb8200b5c70 2024-04-25T09:03:14.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820161650 msgr2=0x7fb8200d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820161650 0x7fb8200d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 >> v1:172.21.15.121:6789/0 conn(0x7fb820161270 legacy=0x7fb8200b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8200cee80 con 0x7fb8200b5c70 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb80c001670 con 0x7fb8200b5c70 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 >> v1:172.21.15.62:6789/0 conn(0x7fb8200b5c70 legacy=0x7fb8200d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8080039e0 0x7fb808003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820161650 0x7fb808004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb808004d40 0x7fb8080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200d3900 con 0x7fb808004d40 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80800bb80 con 0x7fb8080039e0 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.780+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80800bd80 con 0x7fb820161650 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.781+0000 7fb826400640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb808004d40 0x7fb8080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.781+0000 7fb825000640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820161650 0x7fb808004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.781+0000 7fb825a00640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8080039e0 0x7fb808003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.781+0000 7fb826400640 1 -- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820161650 msgr2=0x7fb808004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:14.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.781+0000 7fb826400640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820161650 0x7fb808004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.781+0000 7fb826400640 1 -- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8080039e0 msgr2=0x7fb808003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:14.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.781+0000 7fb826400640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8080039e0 0x7fb808003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.781+0000 7fb826400640 1 -- 172.21.15.62:0/2072753250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb808003520 con 0x7fb808004d40 2024-04-25T09:03:14.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.782+0000 7fb826400640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb808004d40 0x7fb8080051c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb818009f90 tx=0x7fb808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d008c90aa16de26a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.782+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818009c10 con 0x7fb808004d40 2024-04-25T09:03:14.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.782+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818036070 con 0x7fb808004d40 2024-04-25T09:03:14.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.782+0000 7fb81fe00640 1 -- 172.21.15.62:0/2072753250 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818031850 con 0x7fb808004d40 2024-04-25T09:03:14.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.783+0000 7fb826e00640 1 -- 172.21.15.62:0/2072753250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb808004d40 msgr2=0x7fb8080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.783+0000 7fb826e00640 1 --2- 172.21.15.62:0/2072753250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb808004d40 0x7fb8080051c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb818009f90 tx=0x7fb808005830 comp rx=0 tx=0).stop 2024-04-25T09:03:14.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.783+0000 7fb826400640 1 -- 172.21.15.62:0/2072753250 reap_dead start 2024-04-25T09:03:14.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.783+0000 7fb826e00640 1 -- 172.21.15.62:0/2072753250 shutdown_connections 2024-04-25T09:03:14.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.783+0000 7fb826e00640 1 -- 172.21.15.62:0/2072753250 >> 172.21.15.62:0/2072753250 conn(0x7fb820066d10 msgr2=0x7fb820067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:14.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.783+0000 7fb826e00640 1 -- 172.21.15.62:0/2072753250 shutdown_connections 2024-04-25T09:03:14.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.783+0000 7fb826e00640 1 -- 172.21.15.62:0/2072753250 wait complete. 2024-04-25T09:03:14.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.785+0000 7fb826e00640 1 Processor -- start 2024-04-25T09:03:14.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.785+0000 7fb826e00640 1 -- start start 2024-04-25T09:03:14.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb826e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb808004d40 0x7fb820157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb826e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb82015a610 0x7fb820159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb826e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820159b80 0x7fb8200cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb826e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200b2ed0 con 0x7fb82015a610 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb826e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200d3590 con 0x7fb808004d40 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb826e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200b3050 con 0x7fb820159b80 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb825000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb82015a610 0x7fb820159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb825000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb82015a610 0x7fb820159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59294/0 (socket says 172.21.15.62:59294) 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.786+0000 7fb825000640 1 -- 172.21.15.62:0/26725370 learned_addr learned my addr 172.21.15.62:0/26725370 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb825a00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb808004d40 0x7fb820157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb826400640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820159b80 0x7fb8200cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb825000640 1 -- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820159b80 msgr2=0x7fb8200cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb825000640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820159b80 0x7fb8200cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb825000640 1 -- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb808004d40 msgr2=0x7fb820157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb825000640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb808004d40 0x7fb820157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb825000640 1 -- 172.21.15.62:0/26725370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82015de20 con 0x7fb82015a610 2024-04-25T09:03:14.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb826400640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820159b80 0x7fb8200cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:14.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb825000640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb82015a610 0x7fb820159640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e780 tx=0x7fb80c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb81d600640 1 -- 172.21.15.62:0/26725370 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00ee30 con 0x7fb82015a610 2024-04-25T09:03:14.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb81d600640 1 -- 172.21.15.62:0/26725370 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c018070 con 0x7fb82015a610 2024-04-25T09:03:14.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8200ceb10 con 0x7fb82015a610 2024-04-25T09:03:14.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.787+0000 7fb81d600640 1 -- 172.21.15.62:0/26725370 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00cab0 con 0x7fb82015a610 2024-04-25T09:03:14.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.788+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb820158880 con 0x7fb82015a610 2024-04-25T09:03:14.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.789+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb820072910 con 0x7fb82015a610 2024-04-25T09:03:14.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.790+0000 7fb81d600640 1 -- 172.21.15.62:0/26725370 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb80c00cc50 con 0x7fb82015a610 2024-04-25T09:03:14.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.791+0000 7fb81d600640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb80006b500 0x7fb80006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.791+0000 7fb81d600640 1 -- 172.21.15.62:0/26725370 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb80c089a80 con 0x7fb82015a610 2024-04-25T09:03:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.792+0000 7fb825a00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb80006b500 0x7fb80006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.793+0000 7fb825a00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb80006b500 0x7fb80006d9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb820068120 tx=0x7fb814009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.796+0000 7fb81d600640 1 -- 172.21.15.62:0/26725370 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb80c08be20 con 0x7fb82015a610 2024-04-25T09:03:14.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:14.999+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb8200b63a0 con 0x7fb82015a610 2024-04-25T09:03:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.109+0000 7fb81d600640 1 -- 172.21.15.62:0/26725370 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb82015de20 con 0x7fb82015a610 2024-04-25T09:03:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:03:15.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb80006b500 msgr2=0x7fb80006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb80006b500 0x7fb80006d9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb820068120 tx=0x7fb814009040 comp rx=0 tx=0).stop 2024-04-25T09:03:15.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb82015a610 msgr2=0x7fb820159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb82015a610 0x7fb820159640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e780 tx=0x7fb80c00ec40 comp rx=0 tx=0).stop 2024-04-25T09:03:15.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 shutdown_connections 2024-04-25T09:03:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb80006b500 0x7fb80006d9c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb82015a610 0x7fb820159640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb820159b80 0x7fb8200cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 --2- 172.21.15.62:0/26725370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb808004d40 0x7fb820157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 >> 172.21.15.62:0/26725370 conn(0x7fb820066d10 msgr2=0x7fb8200b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 shutdown_connections 2024-04-25T09:03:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.112+0000 7fb826e00640 1 -- 172.21.15.62:0/26725370 wait complete. 2024-04-25T09:03:15.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:03:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.340+0000 7f7aa7200640 1 Processor -- start 2024-04-25T09:03:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.341+0000 7f7aa7200640 1 -- start start 2024-04-25T09:03:15.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.342+0000 7f7aa7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7aa815dd20 0x7f7aa80d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.342+0000 7f7aa7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa815e100 con 0x7f7aa80d34f0 2024-04-25T09:03:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.342+0000 7f7aa7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa80d2450 con 0x7f7aa80ceef0 2024-04-25T09:03:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.342+0000 7f7aa7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa80d2730 con 0x7f7aa815dd20 2024-04-25T09:03:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.343+0000 7f7aa5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7aa815dd20 0x7f7aa80d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.343+0000 7f7aa5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7aa815dd20 0x7f7aa80d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59310/0 (socket says 172.21.15.62:59310) 2024-04-25T09:03:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.343+0000 7f7aa5e00640 1 -- 172.21.15.62:0/2134330404 learned_addr learned my addr 172.21.15.62:0/2134330404 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.343+0000 7f7aa5e00640 1 -- 172.21.15.62:0/2134330404 >> v1:172.21.15.121:6789/0 conn(0x7f7aa80ceef0 legacy=0x7f7aa80b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.343+0000 7f7aa5e00640 1 -- 172.21.15.62:0/2134330404 >> v1:172.21.15.62:6789/0 conn(0x7f7aa80d34f0 legacy=0x7f7aa80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.343+0000 7f7aa5e00640 1 -- 172.21.15.62:0/2134330404 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa80d2a10 con 0x7f7aa815dd20 2024-04-25T09:03:15.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.344+0000 7f7aa5e00640 1 --2- 172.21.15.62:0/2134330404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa815dd20 0x7f7aa80d1d70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009870 tx=0x7f7a9802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f9219fca39a707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.344+0000 7f7aa4a00640 1 -- 172.21.15.62:0/2134330404 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98037070 con 0x7f7aa815dd20 2024-04-25T09:03:15.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.344+0000 7f7aa4a00640 1 -- 172.21.15.62:0/2134330404 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a98032440 con 0x7f7aa815dd20 2024-04-25T09:03:15.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.344+0000 7f7aa4a00640 1 -- 172.21.15.62:0/2134330404 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98032810 con 0x7f7aa815dd20 2024-04-25T09:03:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.345+0000 7f7aa7200640 1 -- 172.21.15.62:0/2134330404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa815dd20 msgr2=0x7f7aa80d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.345+0000 7f7aa7200640 1 --2- 172.21.15.62:0/2134330404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa815dd20 0x7f7aa80d1d70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009870 tx=0x7f7a9802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.345+0000 7f7aa7200640 1 -- 172.21.15.62:0/2134330404 shutdown_connections 2024-04-25T09:03:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.345+0000 7f7aa7200640 1 --2- 172.21.15.62:0/2134330404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa815dd20 0x7f7aa80d1d70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.345+0000 7f7aa7200640 1 -- 172.21.15.62:0/2134330404 >> 172.21.15.62:0/2134330404 conn(0x7f7aa806b000 msgr2=0x7f7aa806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.345+0000 7f7aa7200640 1 -- 172.21.15.62:0/2134330404 shutdown_connections 2024-04-25T09:03:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.345+0000 7f7aa7200640 1 -- 172.21.15.62:0/2134330404 wait complete. 2024-04-25T09:03:15.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.347+0000 7f7aa7200640 1 Processor -- start 2024-04-25T09:03:15.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.347+0000 7f7aa7200640 1 -- start start 2024-04-25T09:03:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aa80ceef0 0x7f7aa81f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa80d34f0 0x7f7aa80c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7aa815dd20 0x7f7aa8159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80b9660 con 0x7f7aa80d34f0 2024-04-25T09:03:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80b94e0 con 0x7f7aa80ceef0 2024-04-25T09:03:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80b97e0 con 0x7f7aa815dd20 2024-04-25T09:03:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa80d34f0 0x7f7aa80c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa80d34f0 0x7f7aa80c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59312/0 (socket says 172.21.15.62:59312) 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.348+0000 7f7aa5400640 1 -- 172.21.15.62:0/2161659034 learned_addr learned my addr 172.21.15.62:0/2161659034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa5e00640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aa80ceef0 0x7f7aa81f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa6800640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7aa815dd20 0x7f7aa8159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa5400640 1 -- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7aa815dd20 msgr2=0x7f7aa8159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa5400640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7aa815dd20 0x7f7aa8159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa5400640 1 -- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aa80ceef0 msgr2=0x7f7aa81f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa5400640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aa80ceef0 0x7f7aa81f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa5400640 1 -- 172.21.15.62:0/2161659034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa81f4430 con 0x7f7aa80d34f0 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa6800640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7aa815dd20 0x7f7aa8159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa5e00640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aa80ceef0 0x7f7aa81f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7aa5400640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa80d34f0 0x7f7aa80c8d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c440 tx=0x7f7a90001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.349+0000 7f7a9e000640 1 -- 172.21.15.62:0/2161659034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900052d0 con 0x7f7aa80d34f0 2024-04-25T09:03:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.350+0000 7f7a9e000640 1 -- 172.21.15.62:0/2161659034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a90005470 con 0x7f7aa80d34f0 2024-04-25T09:03:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.350+0000 7f7a9e000640 1 -- 172.21.15.62:0/2161659034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90007260 con 0x7f7aa80d34f0 2024-04-25T09:03:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.350+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa81f3e70 con 0x7f7aa80d34f0 2024-04-25T09:03:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.350+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa8159410 con 0x7f7aa80d34f0 2024-04-25T09:03:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.351+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa8075070 con 0x7f7aa80d34f0 2024-04-25T09:03:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.352+0000 7f7a9e000640 1 -- 172.21.15.62:0/2161659034 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a90007400 con 0x7f7aa80d34f0 2024-04-25T09:03:15.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.353+0000 7f7a9e000640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a8406b4d0 0x7f7a8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.353+0000 7f7a9e000640 1 -- 172.21.15.62:0/2161659034 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7a90089560 con 0x7f7aa80d34f0 2024-04-25T09:03:15.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.354+0000 7f7aa5e00640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a8406b4d0 0x7f7a8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.355+0000 7f7aa5e00640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a8406b4d0 0x7f7a8406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a98034000 tx=0x7f7a9802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.358+0000 7f7a9e000640 1 -- 172.21.15.62:0/2161659034 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a900585c0 con 0x7f7aa80d34f0 2024-04-25T09:03:15.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:15.566+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7aa8071600 con 0x7f7aa80d34f0 2024-04-25T09:03:16.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.123+0000 7f7a9e000640 1 -- 172.21.15.62:0/2161659034 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7a90057ec0 con 0x7f7aa80d34f0 2024-04-25T09:03:16.113 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:03:16.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a8406b4d0 msgr2=0x7f7a8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a8406b4d0 0x7f7a8406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a98034000 tx=0x7f7a9802e000 comp rx=0 tx=0).stop 2024-04-25T09:03:16.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa80d34f0 msgr2=0x7f7aa80c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa80d34f0 0x7f7aa80c8d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c440 tx=0x7f7a90001e70 comp rx=0 tx=0).stop 2024-04-25T09:03:16.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 shutdown_connections 2024-04-25T09:03:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a8406b4d0 0x7f7a8406d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa80d34f0 0x7f7aa80c8d80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7aa815dd20 0x7f7aa8159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 --2- 172.21.15.62:0/2161659034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aa80ceef0 0x7f7aa81f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.128+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 >> 172.21.15.62:0/2161659034 conn(0x7f7aa806b000 msgr2=0x7f7aa815e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.130+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 shutdown_connections 2024-04-25T09:03:16.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.131+0000 7f7aa7200640 1 -- 172.21.15.62:0/2161659034 wait complete. 2024-04-25T09:03:16.155 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:03:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.346+0000 7fb81da00640 1 Processor -- start 2024-04-25T09:03:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.346+0000 7fb81da00640 1 -- start start 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.348+0000 7fb81da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb8180cebc0 0x7fb81806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.348+0000 7fb81da00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180720c0 0x7fb8180703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.348+0000 7fb81da00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81806d070 con 0x7fb81815bdf0 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.348+0000 7fb81da00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81815c1d0 con 0x7fb8180cebc0 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.348+0000 7fb81da00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180ae8c0 con 0x7fb8180720c0 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.348+0000 7fb80e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb8180cebc0 0x7fb81806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.348+0000 7fb80e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb8180cebc0 0x7fb81806c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59324/0 (socket says 172.21.15.62:59324) 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.348+0000 7fb80e000640 1 -- 172.21.15.62:0/1625896838 learned_addr learned my addr 172.21.15.62:0/1625896838 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.349+0000 7fb817400640 1 --2- 172.21.15.62:0/1625896838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180720c0 0x7fb8180703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.349+0000 7fb80e000640 1 -- 172.21.15.62:0/1625896838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180720c0 msgr2=0x7fb8180703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.349+0000 7fb80e000640 1 --2- 172.21.15.62:0/1625896838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180720c0 0x7fb8180703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.349+0000 7fb80e000640 1 -- 172.21.15.62:0/1625896838 >> v1:172.21.15.121:6789/0 conn(0x7fb81815bdf0 legacy=0x7fb81806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.349+0000 7fb80e000640 1 -- 172.21.15.62:0/1625896838 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81806d350 con 0x7fb8180cebc0 2024-04-25T09:03:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.350+0000 7fb80e000640 1 --2- 172.21.15.62:0/1625896838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180cebc0 0x7fb81806c9b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb800009870 tx=0x7fb80002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895d8b6a936424bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.350+0000 7fb816000640 1 -- 172.21.15.62:0/1625896838 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800037070 con 0x7fb8180cebc0 2024-04-25T09:03:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.350+0000 7fb816000640 1 -- 172.21.15.62:0/1625896838 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800032440 con 0x7fb8180cebc0 2024-04-25T09:03:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.350+0000 7fb816000640 1 -- 172.21.15.62:0/1625896838 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800032810 con 0x7fb8180cebc0 2024-04-25T09:03:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.351+0000 7fb81da00640 1 -- 172.21.15.62:0/1625896838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180cebc0 msgr2=0x7fb81806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.351+0000 7fb81da00640 1 --2- 172.21.15.62:0/1625896838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180cebc0 0x7fb81806c9b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb800009870 tx=0x7fb80002a210 comp rx=0 tx=0).stop 2024-04-25T09:03:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.351+0000 7fb81da00640 1 -- 172.21.15.62:0/1625896838 shutdown_connections 2024-04-25T09:03:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.351+0000 7fb81da00640 1 --2- 172.21.15.62:0/1625896838 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180720c0 0x7fb8180703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.351+0000 7fb81da00640 1 --2- 172.21.15.62:0/1625896838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180cebc0 0x7fb81806c9b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.351+0000 7fb81da00640 1 -- 172.21.15.62:0/1625896838 >> 172.21.15.62:0/1625896838 conn(0x7fb8180b1410 msgr2=0x7fb8180b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.351+0000 7fb81da00640 1 -- 172.21.15.62:0/1625896838 shutdown_connections 2024-04-25T09:03:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.351+0000 7fb81da00640 1 -- 172.21.15.62:0/1625896838 wait complete. 2024-04-25T09:03:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.353+0000 7fb81da00640 1 Processor -- start 2024-04-25T09:03:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.353+0000 7fb81da00640 1 -- start start 2024-04-25T09:03:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb81da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180720c0 0x7fb818145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb81da00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180cebc0 0x7fb8181f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb81da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb81815bdf0 0x7fb8181536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb81da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180aedf0 con 0x7fb8180720c0 2024-04-25T09:03:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb81da00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818072550 con 0x7fb81815bdf0 2024-04-25T09:03:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb81da00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180708e0 con 0x7fb8180cebc0 2024-04-25T09:03:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb816a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180720c0 0x7fb818145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb816a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180720c0 0x7fb818145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59338/0 (socket says 172.21.15.62:59338) 2024-04-25T09:03:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.354+0000 7fb816a00640 1 -- 172.21.15.62:0/2909516371 learned_addr learned my addr 172.21.15.62:0/2909516371 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb817400640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb81815bdf0 0x7fb8181536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb816a00640 1 -- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180cebc0 msgr2=0x7fb8181f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb816a00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180cebc0 0x7fb8181f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb816a00640 1 -- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb81815bdf0 msgr2=0x7fb8181536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb816a00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb81815bdf0 0x7fb8181536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb816a00640 1 -- 172.21.15.62:0/2909516371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8181585a0 con 0x7fb8180720c0 2024-04-25T09:03:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb816a00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180720c0 0x7fb818145ff0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb80800ba50 tx=0x7fb80800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb80f400640 1 -- 172.21.15.62:0/2909516371 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808019070 con 0x7fb8180720c0 2024-04-25T09:03:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb80f400640 1 -- 172.21.15.62:0/2909516371 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80800cc20 con 0x7fb8180720c0 2024-04-25T09:03:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.355+0000 7fb80f400640 1 -- 172.21.15.62:0/2909516371 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808014680 con 0x7fb8180720c0 2024-04-25T09:03:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.356+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb818158880 con 0x7fb8180720c0 2024-04-25T09:03:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.356+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb818158e40 con 0x7fb8180720c0 2024-04-25T09:03:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.357+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81804ef20 con 0x7fb8180720c0 2024-04-25T09:03:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.358+0000 7fb80f400640 1 -- 172.21.15.62:0/2909516371 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb80801c070 con 0x7fb8180720c0 2024-04-25T09:03:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.359+0000 7fb80f400640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7f406b4d0 0x7fb7f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.359+0000 7fb80f400640 1 -- 172.21.15.62:0/2909516371 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb808089de0 con 0x7fb8180720c0 2024-04-25T09:03:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.360+0000 7fb80e000640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7f406b4d0 0x7fb7f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.361+0000 7fb80e000640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7f406b4d0 0x7fb7f406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb800034000 tx=0x7fb80002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.365+0000 7fb80f400640 1 -- 172.21.15.62:0/2909516371 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb808058700 con 0x7fb8180720c0 2024-04-25T09:03:16.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:16.605+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb818071c90 con 0x7fb8180720c0 2024-04-25T09:03:17.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.146+0000 7fb80f400640 1 -- 172.21.15.62:0/2909516371 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb80805c7a0 con 0x7fb8180720c0 2024-04-25T09:03:17.140 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T09:03:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.158+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7f406b4d0 msgr2=0x7fb7f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.158+0000 7fb81da00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7f406b4d0 0x7fb7f406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb800034000 tx=0x7fb80002c000 comp rx=0 tx=0).stop 2024-04-25T09:03:17.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.158+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180720c0 msgr2=0x7fb818145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.158+0000 7fb81da00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180720c0 0x7fb818145ff0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb80800ba50 tx=0x7fb80800bf10 comp rx=0 tx=0).stop 2024-04-25T09:03:17.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.159+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 shutdown_connections 2024-04-25T09:03:17.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.159+0000 7fb81da00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7f406b4d0 0x7fb7f406d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.159+0000 7fb81da00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb81815bdf0 0x7fb8181536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.159+0000 7fb81da00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8180cebc0 0x7fb8181f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.159+0000 7fb81da00640 1 --2- 172.21.15.62:0/2909516371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8180720c0 0x7fb818145ff0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.159+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 >> 172.21.15.62:0/2909516371 conn(0x7fb8180b1410 msgr2=0x7fb8180af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.161+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 shutdown_connections 2024-04-25T09:03:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.164+0000 7fb81da00640 1 -- 172.21.15.62:0/2909516371 wait complete. 2024-04-25T09:03:17.181 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.368+0000 7fc192e00640 1 Processor -- start 2024-04-25T09:03:17.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.368+0000 7fc192e00640 1 -- start start 2024-04-25T09:03:17.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.370+0000 7fc192e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc18c0cccf0 0x7fc18c0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.370+0000 7fc192e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c06ca70 con 0x7fc18c0aed00 2024-04-25T09:03:17.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.370+0000 7fc192e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c15c0f0 con 0x7fc18c15bd10 2024-04-25T09:03:17.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.370+0000 7fc192e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c06d660 con 0x7fc18c0cccf0 2024-04-25T09:03:17.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.370+0000 7fc190e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc18c0aed00 0x7fc18c0cd6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40956/0 (socket says 172.21.15.62:40956) 2024-04-25T09:03:17.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.370+0000 7fc190e00640 1 -- 172.21.15.62:0/808475840 learned_addr learned my addr 172.21.15.62:0/808475840 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:17.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.370+0000 7fc183e00640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc18c0cccf0 0x7fc18c0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.370+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227919098 0 0) 0x7fc18c06ca70 con 0x7fc18c0aed00 2024-04-25T09:03:17.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc170003520 con 0x7fc18c0aed00 2024-04-25T09:03:17.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3712475697 0 0) 0x7fc170003520 con 0x7fc18c0aed00 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc18c0cccf0 msgr2=0x7fc18c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc18c0cccf0 0x7fc18c0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 >> v1:172.21.15.121:6789/0 conn(0x7fc18c15bd10 legacy=0x7fc18c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c06cd30 con 0x7fc18c0aed00 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc1840019f0 con 0x7fc18c0aed00 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 >> v1:172.21.15.62:6789/0 conn(0x7fc18c0aed00 legacy=0x7fc18c0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc18c0cccf0 0x7fc1700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc170003fb0 0x7fc170004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc170004d40 0x7fc1700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c06d660 con 0x7fc170003fb0 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17000bb00 con 0x7fc170004d40 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17000bd00 con 0x7fc18c0cccf0 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.371+0000 7fc183e00640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc170003fb0 0x7fc170004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.373+0000 7fc18be00640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc18c0cccf0 0x7fc1700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.373+0000 7fc190e00640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc170004d40 0x7fc1700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.373+0000 7fc183e00640 1 -- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc18c0cccf0 msgr2=0x7fc1700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.373+0000 7fc183e00640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc18c0cccf0 0x7fc1700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.373+0000 7fc183e00640 1 -- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc170004d40 msgr2=0x7fc1700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.373+0000 7fc183e00640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc170004d40 0x7fc1700051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.373+0000 7fc183e00640 1 -- 172.21.15.62:0/808475840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc170003520 con 0x7fc170003fb0 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc183e00640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc170003fb0 0x7fc170004800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc174009ac0 tx=0x7fc170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfac8861f1e4efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174036070 con 0x7fc170003fb0 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc174031400 con 0x7fc170003fb0 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc18b400640 1 -- 172.21.15.62:0/808475840 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc174009690 con 0x7fc170003fb0 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc192e00640 1 -- 172.21.15.62:0/808475840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc170003fb0 msgr2=0x7fc170004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc192e00640 1 --2- 172.21.15.62:0/808475840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc170003fb0 0x7fc170004800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc174009ac0 tx=0x7fc170005830 comp rx=0 tx=0).stop 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc190e00640 1 -- 172.21.15.62:0/808475840 reap_dead start 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc192e00640 1 -- 172.21.15.62:0/808475840 shutdown_connections 2024-04-25T09:03:17.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc192e00640 1 -- 172.21.15.62:0/808475840 >> 172.21.15.62:0/808475840 conn(0x7fc18c0668f0 msgr2=0x7fc18c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc192e00640 1 -- 172.21.15.62:0/808475840 shutdown_connections 2024-04-25T09:03:17.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.374+0000 7fc192e00640 1 -- 172.21.15.62:0/808475840 wait complete. 2024-04-25T09:03:17.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.376+0000 7fc192e00640 1 Processor -- start 2024-04-25T09:03:17.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.376+0000 7fc192e00640 1 -- start start 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc192e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc170003fb0 0x7fc18c154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc192e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc18c158700 0x7fc18c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc192e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18c142e80 0x7fc18c143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc192e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c06daf0 con 0x7fc18c142e80 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc192e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c06d2f0 con 0x7fc18c158700 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc192e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c143840 con 0x7fc170003fb0 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc190e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18c142e80 0x7fc18c143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc190e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18c142e80 0x7fc18c143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59356/0 (socket says 172.21.15.62:59356) 2024-04-25T09:03:17.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc190e00640 1 -- 172.21.15.62:0/2225798515 learned_addr learned my addr 172.21.15.62:0/2225798515 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc18be00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc170003fb0 0x7fc18c154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.377+0000 7fc183e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc18c158700 0x7fc18c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc190e00640 1 -- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc170003fb0 msgr2=0x7fc18c154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc190e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc170003fb0 0x7fc18c154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc190e00640 1 -- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc18c158700 msgr2=0x7fc18c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc190e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc18c158700 0x7fc18c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc190e00640 1 -- 172.21.15.62:0/2225798515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c159270 con 0x7fc18c142e80 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc190e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18c142e80 0x7fc18c143300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc184001330 tx=0x7fc18400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc188c00640 1 -- 172.21.15.62:0/2225798515 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184004d60 con 0x7fc18c142e80 2024-04-25T09:03:17.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc188c00640 1 -- 172.21.15.62:0/2225798515 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18400bb80 con 0x7fc18c142e80 2024-04-25T09:03:17.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc188c00640 1 -- 172.21.15.62:0/2225798515 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184007070 con 0x7fc18c142e80 2024-04-25T09:03:17.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.378+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c159550 con 0x7fc18c142e80 2024-04-25T09:03:17.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.379+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c151f90 con 0x7fc18c142e80 2024-04-25T09:03:17.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.381+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18c072c70 con 0x7fc18c142e80 2024-04-25T09:03:17.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.382+0000 7fc188c00640 1 -- 172.21.15.62:0/2225798515 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc18401b060 con 0x7fc18c142e80 2024-04-25T09:03:17.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.383+0000 7fc188c00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc16806b500 0x7fc16806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.383+0000 7fc188c00640 1 -- 172.21.15.62:0/2225798515 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc18408a220 con 0x7fc18c142e80 2024-04-25T09:03:17.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.383+0000 7fc18be00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc16806b500 0x7fc16806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.384+0000 7fc18be00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc16806b500 0x7fc16806d9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc17c005490 tx=0x7fc17c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.386+0000 7fc188c00640 1 -- 172.21.15.62:0/2225798515 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc184059000 con 0x7fc18c142e80 2024-04-25T09:03:17.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.588+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc18c04f000 con 0x7fc18c142e80 2024-04-25T09:03:17.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.591+0000 7fc188c00640 1 -- 172.21.15.62:0/2225798515 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fc184058900 con 0x7fc18c142e80 2024-04-25T09:03:17.580 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:17.580 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":19,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:16.608055+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:17.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc16806b500 msgr2=0x7fc16806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc16806b500 0x7fc16806d9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc17c005490 tx=0x7fc17c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:17.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18c142e80 msgr2=0x7fc18c143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18c142e80 0x7fc18c143300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc184001330 tx=0x7fc18400b4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 shutdown_connections 2024-04-25T09:03:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc16806b500 0x7fc16806d9c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc18c142e80 0x7fc18c143300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc18c158700 0x7fc18c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 --2- 172.21.15.62:0/2225798515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc170003fb0 0x7fc18c154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 >> 172.21.15.62:0/2225798515 conn(0x7fc18c0668f0 msgr2=0x7fc18c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 shutdown_connections 2024-04-25T09:03:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.594+0000 7fc192e00640 1 -- 172.21.15.62:0/2225798515 wait complete. 2024-04-25T09:03:17.600 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.813+0000 7fda1fe00640 1 Processor -- start 2024-04-25T09:03:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.813+0000 7fda1fe00640 1 -- start start 2024-04-25T09:03:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fda2015f450 0x7fda200cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1fe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2015faa0 con 0x7fda2006d900 2024-04-25T09:03:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1fe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2015fd80 con 0x7fda20144c10 2024-04-25T09:03:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda20144ff0 con 0x7fda2015f450 2024-04-25T09:03:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fda2015f450 0x7fda200cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fda2015f450 0x7fda200cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59370/0 (socket says 172.21.15.62:59370) 2024-04-25T09:03:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1f400640 1 -- 172.21.15.62:0/3970536498 learned_addr learned my addr 172.21.15.62:0/3970536498 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1f400640 1 -- 172.21.15.62:0/3970536498 >> v1:172.21.15.121:6789/0 conn(0x7fda20144c10 legacy=0x7fda2006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1f400640 1 -- 172.21.15.62:0/3970536498 >> v1:172.21.15.62:6789/0 conn(0x7fda2006d900 legacy=0x7fda200b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.815+0000 7fda1f400640 1 -- 172.21.15.62:0/3970536498 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20160060 con 0x7fda2015f450 2024-04-25T09:03:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.816+0000 7fda1f400640 1 --2- 172.21.15.62:0/3970536498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda200cee40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda18009870 tx=0x7fda1802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e52f1827591610b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1d600640 1 -- 172.21.15.62:0/3970536498 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18037070 con 0x7fda2015f450 2024-04-25T09:03:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1d600640 1 -- 172.21.15.62:0/3970536498 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18032440 con 0x7fda2015f450 2024-04-25T09:03:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1d600640 1 -- 172.21.15.62:0/3970536498 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18032810 con 0x7fda2015f450 2024-04-25T09:03:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1fe00640 1 -- 172.21.15.62:0/3970536498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 msgr2=0x7fda200cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1fe00640 1 --2- 172.21.15.62:0/3970536498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda200cee40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda18009870 tx=0x7fda1802a210 comp rx=0 tx=0).stop 2024-04-25T09:03:17.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1fe00640 1 -- 172.21.15.62:0/3970536498 shutdown_connections 2024-04-25T09:03:17.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1fe00640 1 --2- 172.21.15.62:0/3970536498 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda200cee40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1fe00640 1 -- 172.21.15.62:0/3970536498 >> 172.21.15.62:0/3970536498 conn(0x7fda200b14d0 msgr2=0x7fda200b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1fe00640 1 -- 172.21.15.62:0/3970536498 shutdown_connections 2024-04-25T09:03:17.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.817+0000 7fda1fe00640 1 -- 172.21.15.62:0/3970536498 wait complete. 2024-04-25T09:03:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.819+0000 7fda1fe00640 1 Processor -- start 2024-04-25T09:03:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.819+0000 7fda1fe00640 1 -- start start 2024-04-25T09:03:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda2006d900 0x7fda20140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda20144c10 0x7fda2015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda20142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda200b5030 con 0x7fda2015f450 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2015f880 con 0x7fda2006d900 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda200b51b0 con 0x7fda20144c10 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda20142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda20142950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59386/0 (socket says 172.21.15.62:59386) 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.820+0000 7fda1f400640 1 -- 172.21.15.62:0/3321664059 learned_addr learned my addr 172.21.15.62:0/3321664059 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1ea00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda2006d900 0x7fda20140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1e000640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda20144c10 0x7fda2015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1f400640 1 -- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda20144c10 msgr2=0x7fda2015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1f400640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda20144c10 0x7fda2015b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1f400640 1 -- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda2006d900 msgr2=0x7fda20140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1f400640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda2006d900 0x7fda20140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1f400640 1 -- 172.21.15.62:0/3321664059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda201414e0 con 0x7fda2015f450 2024-04-25T09:03:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1ea00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda2006d900 0x7fda20140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1e000640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda20144c10 0x7fda2015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1f400640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda20142950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda1802a250 tx=0x7fda180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda16a00640 1 -- 172.21.15.62:0/3321664059 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18041020 con 0x7fda2015f450 2024-04-25T09:03:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda16a00640 1 -- 172.21.15.62:0/3321664059 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18030990 con 0x7fda2015f450 2024-04-25T09:03:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda16a00640 1 -- 172.21.15.62:0/3321664059 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18037040 con 0x7fda2015f450 2024-04-25T09:03:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.821+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda200cdf70 con 0x7fda2015f450 2024-04-25T09:03:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.822+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda200cd640 con 0x7fda2015f450 2024-04-25T09:03:17.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.823+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2004ef20 con 0x7fda2015f450 2024-04-25T09:03:17.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.824+0000 7fda16a00640 1 -- 172.21.15.62:0/3321664059 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda18044020 con 0x7fda2015f450 2024-04-25T09:03:17.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.825+0000 7fda16a00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda0006b4d0 0x7fda0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.825+0000 7fda16a00640 1 -- 172.21.15.62:0/3321664059 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda180aa8d0 con 0x7fda2015f450 2024-04-25T09:03:17.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.825+0000 7fda1ea00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda0006b4d0 0x7fda0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.826+0000 7fda1ea00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda0006b4d0 0x7fda0006d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda10003eb0 tx=0x7fda10009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:17.829+0000 7fda16a00640 1 -- 172.21.15.62:0/3321664059 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda1804b070 con 0x7fda2015f450 2024-04-25T09:03:18.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.021+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda200b22b0 con 0x7fda2015f450 2024-04-25T09:03:18.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.023+0000 7fda16a00640 1 -- 172.21.15.62:0/3321664059 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fda180796b0 con 0x7fda2015f450 2024-04-25T09:03:18.011 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:18.011 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":19,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:16.608055+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.025+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda0006b4d0 msgr2=0x7fda0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.025+0000 7fda1fe00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda0006b4d0 0x7fda0006d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda10003eb0 tx=0x7fda10009040 comp rx=0 tx=0).stop 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.025+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 msgr2=0x7fda20142950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.025+0000 7fda1fe00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda20142950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda1802a250 tx=0x7fda180046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.026+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 shutdown_connections 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.026+0000 7fda1fe00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda0006b4d0 0x7fda0006d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.026+0000 7fda1fe00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fda2015f450 0x7fda20142950 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.026+0000 7fda1fe00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fda20144c10 0x7fda2015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.026+0000 7fda1fe00640 1 --2- 172.21.15.62:0/3321664059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda2006d900 0x7fda20140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.026+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 >> 172.21.15.62:0/3321664059 conn(0x7fda200b14d0 msgr2=0x7fda2013f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.026+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 shutdown_connections 2024-04-25T09:03:18.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.026+0000 7fda1fe00640 1 -- 172.21.15.62:0/3321664059 wait complete. 2024-04-25T09:03:18.031 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:03:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.257+0000 7f7140600640 1 Processor -- start 2024-04-25T09:03:18.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.258+0000 7f7140600640 1 -- start start 2024-04-25T09:03:18.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.259+0000 7f7140600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f71380d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.259+0000 7f7140600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71380b97b0 0x7f71380b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.259+0000 7f7140600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71380b7590 con 0x7f71380d20a0 2024-04-25T09:03:18.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.259+0000 7f7140600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71380b81a0 con 0x7f71380b97b0 2024-04-25T09:03:18.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.259+0000 7f7140600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71380b7870 con 0x7f713815dc10 2024-04-25T09:03:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.260+0000 7f713dc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f71380d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.260+0000 7f713e600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71380b97b0 0x7f71380b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.260+0000 7f713dc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f71380d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41344/0 (socket says 172.21.15.62:41344) 2024-04-25T09:03:18.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.260+0000 7f713dc00640 1 -- 172.21.15.62:0/2872985980 learned_addr learned my addr 172.21.15.62:0/2872985980 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.261+0000 7f713dc00640 1 -- 172.21.15.62:0/2872985980 >> v2:172.21.15.121:3300/0 conn(0x7f71380b97b0 msgr2=0x7f71380b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.261+0000 7f713dc00640 1 --2- 172.21.15.62:0/2872985980 >> v2:172.21.15.121:3300/0 conn(0x7f71380b97b0 0x7f71380b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.261+0000 7f713dc00640 1 -- 172.21.15.62:0/2872985980 >> v1:172.21.15.121:6789/0 conn(0x7f71380d20a0 legacy=0x7f71380b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.261+0000 7f713dc00640 1 -- 172.21.15.62:0/2872985980 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380b7b50 con 0x7f713815dc10 2024-04-25T09:03:18.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.261+0000 7f713e600640 1 --2- 172.21.15.62:0/2872985980 >> v2:172.21.15.121:3300/0 conn(0x7f71380b97b0 0x7f71380b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.262+0000 7f713dc00640 1 --2- 172.21.15.62:0/2872985980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f71380d1b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7128009870 tx=0x7f712802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a54b06792c0a323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.262+0000 7f7137e00640 1 -- 172.21.15.62:0/2872985980 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128037070 con 0x7f713815dc10 2024-04-25T09:03:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7137e00640 1 -- 172.21.15.62:0/2872985980 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712802ae50 con 0x7f713815dc10 2024-04-25T09:03:18.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7137e00640 1 -- 172.21.15.62:0/2872985980 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128032860 con 0x7f713815dc10 2024-04-25T09:03:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7140600640 1 -- 172.21.15.62:0/2872985980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 msgr2=0x7f71380d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7140600640 1 --2- 172.21.15.62:0/2872985980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f71380d1b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7128009870 tx=0x7f712802a300 comp rx=0 tx=0).stop 2024-04-25T09:03:18.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7140600640 1 -- 172.21.15.62:0/2872985980 shutdown_connections 2024-04-25T09:03:18.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7140600640 1 --2- 172.21.15.62:0/2872985980 >> v2:172.21.15.121:3300/0 conn(0x7f71380b97b0 0x7f71380b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7140600640 1 --2- 172.21.15.62:0/2872985980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f71380d1b60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7140600640 1 -- 172.21.15.62:0/2872985980 >> 172.21.15.62:0/2872985980 conn(0x7f713806b000 msgr2=0x7f713806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7140600640 1 -- 172.21.15.62:0/2872985980 shutdown_connections 2024-04-25T09:03:18.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.263+0000 7f7140600640 1 -- 172.21.15.62:0/2872985980 wait complete. 2024-04-25T09:03:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.265+0000 7f7140600640 1 Processor -- start 2024-04-25T09:03:18.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.265+0000 7f7140600640 1 -- start start 2024-04-25T09:03:18.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.266+0000 7f7140600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380b97b0 0x7f71381f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.266+0000 7f7140600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f713814aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.266+0000 7f7140600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71381e8230 0x7f71381e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.266+0000 7f7140600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713815dff0 con 0x7f71381e8230 2024-04-25T09:03:18.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.266+0000 7f7140600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71380b72a0 con 0x7f71380b97b0 2024-04-25T09:03:18.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.266+0000 7f7140600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71380b8b70 con 0x7f713815dc10 2024-04-25T09:03:18.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.266+0000 7f713e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71381e8230 0x7f71381e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71381e8230 0x7f71381e86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59398/0 (socket says 172.21.15.62:59398) 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713e600640 1 -- 172.21.15.62:0/3272035089 learned_addr learned my addr 172.21.15.62:0/3272035089 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713dc00640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380b97b0 0x7f71381f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713d200640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f713814aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713e600640 1 -- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 msgr2=0x7f713814aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713e600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f713814aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713e600640 1 -- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380b97b0 msgr2=0x7f71381f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713e600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380b97b0 0x7f71381f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713e600640 1 -- 172.21.15.62:0/3272035089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713815fae0 con 0x7f71381e8230 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713d200640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f713814aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:18.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.267+0000 7f713e600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71381e8230 0x7f71381e86e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f713000efc0 tx=0x7f7130009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.268+0000 7f7135600640 1 -- 172.21.15.62:0/3272035089 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130018030 con 0x7f71381e8230 2024-04-25T09:03:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.268+0000 7f7135600640 1 -- 172.21.15.62:0/3272035089 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713000cd80 con 0x7f71381e8230 2024-04-25T09:03:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.268+0000 7f7135600640 1 -- 172.21.15.62:0/3272035089 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130003bb0 con 0x7f71381e8230 2024-04-25T09:03:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.268+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713815f800 con 0x7f71381e8230 2024-04-25T09:03:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.268+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71381e6850 con 0x7f71381e8230 2024-04-25T09:03:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.269+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713804ef20 con 0x7f71381e8230 2024-04-25T09:03:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.270+0000 7f7135600640 1 -- 172.21.15.62:0/3272035089 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f713001b070 con 0x7f71381e8230 2024-04-25T09:03:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.271+0000 7f7135600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711806b500 0x7f711806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.271+0000 7f7135600640 1 -- 172.21.15.62:0/3272035089 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f713008ac50 con 0x7f71381e8230 2024-04-25T09:03:18.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.271+0000 7f713dc00640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711806b500 0x7f711806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.272+0000 7f713dc00640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711806b500 0x7f711806d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7128034000 tx=0x7f712802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.275+0000 7f7135600640 1 -- 172.21.15.62:0/3272035089 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7130010040 con 0x7f71381e8230 2024-04-25T09:03:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:18.516+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f71380b2590 con 0x7f71381e8230 2024-04-25T09:03:19.006 INFO:tasks.ceph.mds.b.smithi121.stderr:2024-04-25T09:03:19.017+0000 7f4266e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:03:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.474+0000 7f7135600640 1 -- 172.21.15.62:0/3272035089 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7130059a30 con 0x7f71381e8230 2024-04-25T09:03:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711806b500 msgr2=0x7f711806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711806b500 0x7f711806d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7128034000 tx=0x7f712802e000 comp rx=0 tx=0).stop 2024-04-25T09:03:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71381e8230 msgr2=0x7f71381e86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71381e8230 0x7f71381e86e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f713000efc0 tx=0x7f7130009f90 comp rx=0 tx=0).stop 2024-04-25T09:03:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 shutdown_connections 2024-04-25T09:03:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711806b500 0x7f711806d9c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71381e8230 0x7f71381e86e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f713815dc10 0x7f713814aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 --2- 172.21.15.62:0/3272035089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71380b97b0 0x7f71381f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 >> 172.21.15.62:0/3272035089 conn(0x7f713806b000 msgr2=0x7f71380b6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 shutdown_connections 2024-04-25T09:03:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.478+0000 7f7140600640 1 -- 172.21.15.62:0/3272035089 wait complete. 2024-04-25T09:03:19.485 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:03:19.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:03:19.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.714+0000 7f11d1400640 1 Processor -- start 2024-04-25T09:03:19.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.714+0000 7f11d1400640 1 -- start start 2024-04-25T09:03:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11d1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f11cc0ceb50 0x7f11cc06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11d1400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11d1400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f11cc15e5f0 0x7f11cc0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11d1400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11cc06dc60 con 0x7f11cc0ceb50 2024-04-25T09:03:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11d1400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11cc15ee30 con 0x7f11cc15e5f0 2024-04-25T09:03:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11d1400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc1618b0 con 0x7f11cc06d800 2024-04-25T09:03:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11cbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f11cc0ceb50 0x7f11cc06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11cbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f11cc0ceb50 0x7f11cc06d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59404/0 (socket says 172.21.15.62:59404) 2024-04-25T09:03:19.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11cbe00640 1 -- 172.21.15.62:0/90721141 learned_addr learned my addr 172.21.15.62:0/90721141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:19.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11c3e00640 1 --2- 172.21.15.62:0/90721141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11cbe00640 1 -- 172.21.15.62:0/90721141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 msgr2=0x7f11cc15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11cbe00640 1 --2- 172.21.15.62:0/90721141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc15e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11cbe00640 1 -- 172.21.15.62:0/90721141 >> v2:172.21.15.121:3300/0 conn(0x7f11cc15e5f0 msgr2=0x7f11cc0ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.716+0000 7f11d0a00640 1 --2- 172.21.15.62:0/90721141 >> v2:172.21.15.121:3300/0 conn(0x7f11cc15e5f0 0x7f11cc0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.717+0000 7f11cbe00640 1 --2- 172.21.15.62:0/90721141 >> v2:172.21.15.121:3300/0 conn(0x7f11cc15e5f0 0x7f11cc0ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.717+0000 7f11cbe00640 1 -- 172.21.15.62:0/90721141 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc161b90 con 0x7f11cc0ceb50 2024-04-25T09:03:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.717+0000 7f11d0a00640 1 --2- 172.21.15.62:0/90721141 >> v2:172.21.15.121:3300/0 conn(0x7f11cc15e5f0 0x7f11cc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.718+0000 7f11cbe00640 1 --2- 172.21.15.62:0/90721141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc0ceb50 0x7f11cc06d2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0b5960 tx=0x7f11bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4991aea6d0a3baa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.718+0000 7f11cb400640 1 -- 172.21.15.62:0/90721141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc037070 con 0x7f11cc0ceb50 2024-04-25T09:03:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.718+0000 7f11cb400640 1 -- 172.21.15.62:0/90721141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc032440 con 0x7f11cc0ceb50 2024-04-25T09:03:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.718+0000 7f11cb400640 1 -- 172.21.15.62:0/90721141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc032810 con 0x7f11cc0ceb50 2024-04-25T09:03:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 -- 172.21.15.62:0/90721141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc0ceb50 msgr2=0x7f11cc06d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 --2- 172.21.15.62:0/90721141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc0ceb50 0x7f11cc06d2c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0b5960 tx=0x7f11bc02a140 comp rx=0 tx=0).stop 2024-04-25T09:03:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 -- 172.21.15.62:0/90721141 shutdown_connections 2024-04-25T09:03:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 --2- 172.21.15.62:0/90721141 >> v2:172.21.15.121:3300/0 conn(0x7f11cc15e5f0 0x7f11cc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 --2- 172.21.15.62:0/90721141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc15e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 --2- 172.21.15.62:0/90721141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc0ceb50 0x7f11cc06d2c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 -- 172.21.15.62:0/90721141 >> 172.21.15.62:0/90721141 conn(0x7f11cc066f20 msgr2=0x7f11cc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 -- 172.21.15.62:0/90721141 shutdown_connections 2024-04-25T09:03:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.719+0000 7f11d1400640 1 -- 172.21.15.62:0/90721141 wait complete. 2024-04-25T09:03:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.721+0000 7f11d1400640 1 Processor -- start 2024-04-25T09:03:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.721+0000 7f11d1400640 1 -- start start 2024-04-25T09:03:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d1400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc134a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0ceb50 0x7f11cc1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15e5f0 0x7f11cc156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d1400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0cf0b0 con 0x7f11cc15e5f0 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d1400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0cef30 con 0x7f11cc0ceb50 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d1400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc162390 con 0x7f11cc06d800 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15e5f0 0x7f11cc156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15e5f0 0x7f11cc156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59414/0 (socket says 172.21.15.62:59414) 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11d0a00640 1 -- 172.21.15.62:0/2570923247 learned_addr learned my addr 172.21.15.62:0/2570923247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11c3e00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0ceb50 0x7f11cc1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.722+0000 7f11cbe00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc134a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11d0a00640 1 -- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 msgr2=0x7f11cc134a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11d0a00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc134a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11d0a00640 1 -- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0ceb50 msgr2=0x7f11cc1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11d0a00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0ceb50 0x7f11cc1569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11d0a00640 1 -- 172.21.15.62:0/2570923247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc1575b0 con 0x7f11cc15e5f0 2024-04-25T09:03:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11cbe00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc134a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11d0a00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15e5f0 0x7f11cc156f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f11c400cd50 tx=0x7f11c400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11c8c00640 1 -- 172.21.15.62:0/2570923247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400f680 con 0x7f11cc15e5f0 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11c8c00640 1 -- 172.21.15.62:0/2570923247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c400f840 con 0x7f11cc15e5f0 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11c8c00640 1 -- 172.21.15.62:0/2570923247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400fc30 con 0x7f11cc15e5f0 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc157890 con 0x7f11cc15e5f0 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.723+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11cc1350b0 con 0x7f11cc15e5f0 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.725+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11cc04ef20 con 0x7f11cc15e5f0 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.726+0000 7f11c8c00640 1 -- 172.21.15.62:0/2570923247 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11c40213f0 con 0x7f11cc15e5f0 2024-04-25T09:03:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.727+0000 7f11c8c00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11a806b4d0 0x7f11a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.727+0000 7f11c8c00640 1 -- 172.21.15.62:0/2570923247 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11c408b0b0 con 0x7f11cc15e5f0 2024-04-25T09:03:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.728+0000 7f11cbe00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11a806b4d0 0x7f11a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.729+0000 7f11cbe00640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11a806b4d0 0x7f11a806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11bc034000 tx=0x7f11bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.732+0000 7f11c8c00640 1 -- 172.21.15.62:0/2570923247 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11c4014030 con 0x7f11cc15e5f0 2024-04-25T09:03:19.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:19.982+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f11cc1466e0 con 0x7f11cc15e5f0 2024-04-25T09:03:20.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.484+0000 7f11c8c00640 1 -- 172.21.15.62:0/2570923247 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f11c40597b0 con 0x7f11cc15e5f0 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.487+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11a806b4d0 msgr2=0x7f11a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.487+0000 7f11d1400640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11a806b4d0 0x7f11a806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11bc034000 tx=0x7f11bc02e000 comp rx=0 tx=0).stop 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.487+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15e5f0 msgr2=0x7f11cc156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.487+0000 7f11d1400640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15e5f0 0x7f11cc156f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f11c400cd50 tx=0x7f11c400a650 comp rx=0 tx=0).stop 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.488+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 shutdown_connections 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.488+0000 7f11d1400640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11a806b4d0 0x7f11a806d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.488+0000 7f11d1400640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15e5f0 0x7f11cc156f00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.488+0000 7f11d1400640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0ceb50 0x7f11cc1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.488+0000 7f11d1400640 1 --2- 172.21.15.62:0/2570923247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc06d800 0x7f11cc134a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.488+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 >> 172.21.15.62:0/2570923247 conn(0x7f11cc066f20 msgr2=0x7f11cc160af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.488+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 shutdown_connections 2024-04-25T09:03:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.488+0000 7f11d1400640 1 -- 172.21.15.62:0/2570923247 wait complete. 2024-04-25T09:03:20.497 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:03:20.497 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:20.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.712+0000 7fb2d3200640 1 Processor -- start 2024-04-25T09:03:20.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.713+0000 7fb2d3200640 1 -- start start 2024-04-25T09:03:20.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.714+0000 7fb2d3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d406d900 0x7fb2d4165370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.714+0000 7fb2d3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d40b1220 con 0x7fb2d415acf0 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.714+0000 7fb2d3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d415b220 con 0x7fb2d413d970 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.714+0000 7fb2d3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d415b500 con 0x7fb2d406d900 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.715+0000 7fb2d1400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb2d415acf0 0x7fb2d406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40980/0 (socket says 172.21.15.62:40980) 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.715+0000 7fb2d1400640 1 -- 172.21.15.62:0/2068989696 learned_addr learned my addr 172.21.15.62:0/2068989696 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.715+0000 7fb2d2800640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d406d900 0x7fb2d4165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.715+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078489058 0 0) 0x7fb2d40b1220 con 0x7fb2d415acf0 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.715+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b8003520 con 0x7fb2d415acf0 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.715+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 923754648 0 0) 0x7fb2b8003520 con 0x7fb2d415acf0 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d406d900 msgr2=0x7fb2d4165370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d406d900 0x7fb2d4165370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 >> v1:172.21.15.121:6789/0 conn(0x7fb2d413d970 legacy=0x7fb2d40b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d415b7e0 con 0x7fb2d415acf0 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb2c0001670 con 0x7fb2d415acf0 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 >> v1:172.21.15.62:6789/0 conn(0x7fb2d415acf0 legacy=0x7fb2d406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d406d900 0x7fb2b80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b8003f60 0x7fb2b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8004d40 0x7fb2b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d415b500 con 0x7fb2b8003f60 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b800baa0 con 0x7fb2b8004d40 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b800deb0 con 0x7fb2d406d900 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d1400640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b8003f60 0x7fb2b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d1e00640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d406d900 0x7fb2b80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.716+0000 7fb2d2800640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8004d40 0x7fb2b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.717+0000 7fb2d1400640 1 -- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d406d900 msgr2=0x7fb2b80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:20.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.717+0000 7fb2d1400640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d406d900 0x7fb2b80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.717+0000 7fb2d1400640 1 -- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8004d40 msgr2=0x7fb2b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:20.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.717+0000 7fb2d1400640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8004d40 0x7fb2b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.717+0000 7fb2d1400640 1 -- 172.21.15.62:0/2068989696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b8003520 con 0x7fb2b8003f60 2024-04-25T09:03:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.718+0000 7fb2d1400640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b8003f60 0x7fb2b8004800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c210 tx=0x7fb2b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88823c0e1aa9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.718+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0007cf0 con 0x7fb2b8003f60 2024-04-25T09:03:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.718+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0005af0 con 0x7fb2b8003f60 2024-04-25T09:03:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.718+0000 7fb2d0a00640 1 -- 172.21.15.62:0/2068989696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0007070 con 0x7fb2b8003f60 2024-04-25T09:03:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.718+0000 7fb2d3200640 1 -- 172.21.15.62:0/2068989696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b8003f60 msgr2=0x7fb2b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.718+0000 7fb2d3200640 1 --2- 172.21.15.62:0/2068989696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2b8003f60 0x7fb2b8004800 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c210 tx=0x7fb2b8005830 comp rx=0 tx=0).stop 2024-04-25T09:03:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.718+0000 7fb2d2800640 1 -- 172.21.15.62:0/2068989696 reap_dead start 2024-04-25T09:03:20.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.719+0000 7fb2d3200640 1 -- 172.21.15.62:0/2068989696 shutdown_connections 2024-04-25T09:03:20.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.719+0000 7fb2d3200640 1 -- 172.21.15.62:0/2068989696 >> 172.21.15.62:0/2068989696 conn(0x7fb2d40b14d0 msgr2=0x7fb2d40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:20.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.719+0000 7fb2d3200640 1 -- 172.21.15.62:0/2068989696 shutdown_connections 2024-04-25T09:03:20.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.719+0000 7fb2d3200640 1 -- 172.21.15.62:0/2068989696 wait complete. 2024-04-25T09:03:20.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.720+0000 7fb2d3200640 1 Processor -- start 2024-04-25T09:03:20.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.721+0000 7fb2d3200640 1 -- start start 2024-04-25T09:03:20.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.721+0000 7fb2d3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8003f60 0x7fb2d413a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.721+0000 7fb2d3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d413b8e0 0x7fb2d413bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.721+0000 7fb2d3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d40712b0 0x7fb2d4071730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.721+0000 7fb2d3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d413f3e0 con 0x7fb2d413b8e0 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.721+0000 7fb2d3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d413f260 con 0x7fb2b8003f60 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.721+0000 7fb2d3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d413f560 con 0x7fb2d40712b0 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d413b8e0 0x7fb2d413bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d413b8e0 0x7fb2d413bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59424/0 (socket says 172.21.15.62:59424) 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1400640 1 -- 172.21.15.62:0/3430536162 learned_addr learned my addr 172.21.15.62:0/3430536162 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1e00640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8003f60 0x7fb2d413a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d2800640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d40712b0 0x7fb2d4071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1400640 1 -- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d40712b0 msgr2=0x7fb2d4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1400640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d40712b0 0x7fb2d4071730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1400640 1 -- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8003f60 msgr2=0x7fb2d413a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1400640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8003f60 0x7fb2d413a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.722+0000 7fb2d1400640 1 -- 172.21.15.62:0/3430536162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d4072000 con 0x7fb2d413b8e0 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.723+0000 7fb2d1400640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d413b8e0 0x7fb2d413bd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c210 tx=0x7fb2d415b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:20.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.723+0000 7fb2ca000640 1 -- 172.21.15.62:0/3430536162 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0001630 con 0x7fb2d413b8e0 2024-04-25T09:03:20.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.723+0000 7fb2ca000640 1 -- 172.21.15.62:0/3430536162 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0042070 con 0x7fb2d413b8e0 2024-04-25T09:03:20.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.723+0000 7fb2ca000640 1 -- 172.21.15.62:0/3430536162 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0007270 con 0x7fb2d413b8e0 2024-04-25T09:03:20.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.723+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d40b2c70 con 0x7fb2d413b8e0 2024-04-25T09:03:20.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.723+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d40b3230 con 0x7fb2d413b8e0 2024-04-25T09:03:20.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.724+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d404ef20 con 0x7fb2d413b8e0 2024-04-25T09:03:20.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.725+0000 7fb2ca000640 1 -- 172.21.15.62:0/3430536162 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2c0007410 con 0x7fb2d413b8e0 2024-04-25T09:03:20.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.726+0000 7fb2ca000640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2b406b520 0x7fb2b406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.726+0000 7fb2ca000640 1 -- 172.21.15.62:0/3430536162 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2c00aa280 con 0x7fb2d413b8e0 2024-04-25T09:03:20.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.727+0000 7fb2d1e00640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2b406b520 0x7fb2b406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.728+0000 7fb2d1e00640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2b406b520 0x7fb2b406d9e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4007fd0 tx=0x7fb2c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:20.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.730+0000 7fb2ca000640 1 -- 172.21.15.62:0/3430536162 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2c00790e0 con 0x7fb2d413b8e0 2024-04-25T09:03:20.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.975+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2d413fcb0 con 0x7fb2d413b8e0 2024-04-25T09:03:20.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.976+0000 7fb2ca000640 1 -- 172.21.15.62:0/3430536162 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fb2c00789e0 con 0x7fb2d413b8e0 2024-04-25T09:03:20.964 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:20.964 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:16.609043+0000","modified":"2024-04-25T09:03:19.983991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:03:20.964 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.978+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2b406b520 msgr2=0x7fb2b406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.978+0000 7fb2d3200640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2b406b520 0x7fb2b406d9e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4007fd0 tx=0x7fb2c4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.978+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d413b8e0 msgr2=0x7fb2d413bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.978+0000 7fb2d3200640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d413b8e0 0x7fb2d413bd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c210 tx=0x7fb2d415b3f0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.978+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 shutdown_connections 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.978+0000 7fb2d3200640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2b406b520 0x7fb2b406d9e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.979+0000 7fb2d3200640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2d413b8e0 0x7fb2d413bd60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.979+0000 7fb2d3200640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2d40712b0 0x7fb2d4071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.979+0000 7fb2d3200640 1 --2- 172.21.15.62:0/3430536162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2b8003f60 0x7fb2d413a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.979+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 >> 172.21.15.62:0/3430536162 conn(0x7fb2d40b14d0 msgr2=0x7fb2d4140700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:20.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.979+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 shutdown_connections 2024-04-25T09:03:20.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:20.979+0000 7fb2d3200640 1 -- 172.21.15.62:0/3430536162 wait complete. 2024-04-25T09:03:20.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:21.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.206+0000 7f3641e00640 1 Processor -- start 2024-04-25T09:03:21.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.206+0000 7f3641e00640 1 -- start start 2024-04-25T09:03:21.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.207+0000 7f3641e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f363c06d900 0x7f363c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.207+0000 7f3641e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c0cee40 0x7f363c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.207+0000 7f3641e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0cf2b0 con 0x7f363c144c10 2024-04-25T09:03:21.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.207+0000 7f3641e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c144ff0 con 0x7f363c06d900 2024-04-25T09:03:21.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.207+0000 7f3641e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c160340 con 0x7f363c0cee40 2024-04-25T09:03:21.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f3640a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f363c144c10 0x7f363c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40990/0 (socket says 172.21.15.62:40990) 2024-04-25T09:03:21.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f3640a00640 1 -- 172.21.15.62:0/3817629638 learned_addr learned my addr 172.21.15.62:0/3817629638 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848226627 0 0) 0x7f363c0cf2b0 con 0x7f363c144c10 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3624003520 con 0x7f363c144c10 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363be00640 1 --2- 172.21.15.62:0/3817629638 >> v2:172.21.15.121:3300/0 conn(0x7f363c06d900 0x7f363c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f3641400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c0cee40 0x7f363c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 395897795 0 0) 0x7f3624003520 con 0x7f363c144c10 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c0cee40 msgr2=0x7f363c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c0cee40 0x7f363c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 >> v2:172.21.15.121:3300/0 conn(0x7f363c06d900 msgr2=0x7f363c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 --2- 172.21.15.62:0/3817629638 >> v2:172.21.15.121:3300/0 conn(0x7f363c06d900 0x7f363c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c15faa0 con 0x7f363c144c10 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.208+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f3630001670 con 0x7f363c144c10 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 >> v1:172.21.15.62:6789/0 conn(0x7f363c144c10 legacy=0x7f363c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f363b400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f363b400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c0cee40 0x7f3624004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f363b400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3624004d40 0x7f36240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c144ff0 con 0x7f3624004d40 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c160340 con 0x7f36240039b0 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362400bb50 con 0x7f363c0cee40 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f3641400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3624004d40 0x7f36240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f3640a00640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f363be00640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c0cee40 0x7f3624004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f3641400640 1 -- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c0cee40 msgr2=0x7f3624004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f3641400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c0cee40 0x7f3624004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f3641400640 1 -- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36240039b0 msgr2=0x7f3624003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.209+0000 7f3641400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.210+0000 7f3641400640 1 -- 172.21.15.62:0/3817629638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624003520 con 0x7f3624004d40 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.210+0000 7f3640a00640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36240039b0 0x7f3624003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.211+0000 7f3641400640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3624004d40 0x7f36240051c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3634007360 tx=0x7f3624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39524e1bb001b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.211+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634009be0 con 0x7f3624004d40 2024-04-25T09:03:21.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.211+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634036600 con 0x7f3624004d40 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.211+0000 7f363b400640 1 -- 172.21.15.62:0/3817629638 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340369d0 con 0x7f3624004d40 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.211+0000 7f3641e00640 1 -- 172.21.15.62:0/3817629638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3624004d40 msgr2=0x7f36240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.211+0000 7f3641e00640 1 --2- 172.21.15.62:0/3817629638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3624004d40 0x7f36240051c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3634007360 tx=0x7f3624005830 comp rx=0 tx=0).stop 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.211+0000 7f3641400640 1 -- 172.21.15.62:0/3817629638 reap_dead start 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.212+0000 7f3641e00640 1 -- 172.21.15.62:0/3817629638 shutdown_connections 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.212+0000 7f3641e00640 1 -- 172.21.15.62:0/3817629638 >> 172.21.15.62:0/3817629638 conn(0x7f363c0b14d0 msgr2=0x7f363c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.212+0000 7f3641e00640 1 -- 172.21.15.62:0/3817629638 shutdown_connections 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.212+0000 7f3641e00640 1 -- 172.21.15.62:0/3817629638 wait complete. 2024-04-25T09:03:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.213+0000 7f3641e00640 1 Processor -- start 2024-04-25T09:03:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.213+0000 7f3641e00640 1 -- start start 2024-04-25T09:03:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.214+0000 7f3641e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3624004d40 0x7f363c13f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.214+0000 7f3641e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c13f940 0x7f363c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.214+0000 7f3641e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c13fe40 0x7f363c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.214+0000 7f3641e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c160060 con 0x7f363c13fe40 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.214+0000 7f3641e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0b41c0 con 0x7f3624004d40 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.214+0000 7f3641e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0b4650 con 0x7f363c13f940 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c13fe40 0x7f363c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c13fe40 0x7f363c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59430/0 (socket says 172.21.15.62:59430) 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 -- 172.21.15.62:0/1234876551 learned_addr learned my addr 172.21.15.62:0/1234876551 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3640a00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3624004d40 0x7f363c13f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 -- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c13f940 msgr2=0x7f363c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f363be00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c13f940 0x7f363c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c13f940 0x7f363c0c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 -- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3624004d40 msgr2=0x7f363c13f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3624004d40 0x7f363c13f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 -- 172.21.15.62:0/1234876551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0c70c0 con 0x7f363c13fe40 2024-04-25T09:03:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3640a00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3624004d40 0x7f363c13f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.215+0000 7f3641400640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c13fe40 0x7f363c0c7b90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3634007480 tx=0x7f3634004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.216+0000 7f3638c00640 1 -- 172.21.15.62:0/1234876551 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634009be0 con 0x7f363c13fe40 2024-04-25T09:03:21.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.216+0000 7f3638c00640 1 -- 172.21.15.62:0/1234876551 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634036400 con 0x7f363c13fe40 2024-04-25T09:03:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.216+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c0b0f40 con 0x7f363c13fe40 2024-04-25T09:03:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.216+0000 7f3638c00640 1 -- 172.21.15.62:0/1234876551 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634038520 con 0x7f363c13fe40 2024-04-25T09:03:21.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.216+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c0b0c30 con 0x7f363c13fe40 2024-04-25T09:03:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.217+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c04ef20 con 0x7f363c13fe40 2024-04-25T09:03:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.218+0000 7f3638c00640 1 -- 172.21.15.62:0/1234876551 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f363402e620 con 0x7f363c13fe40 2024-04-25T09:03:21.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.220+0000 7f3638c00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f361c06b4d0 0x7f361c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.220+0000 7f3638c00640 1 -- 172.21.15.62:0/1234876551 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36340aafa0 con 0x7f363c13fe40 2024-04-25T09:03:21.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.220+0000 7f3640a00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f361c06b4d0 0x7f361c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.221+0000 7f3640a00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f361c06b4d0 0x7f361c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3630001eb0 tx=0x7f3630000f70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.224+0000 7f3638c00640 1 -- 172.21.15.62:0/1234876551 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3634048060 con 0x7f363c13fe40 2024-04-25T09:03:21.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.422+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f363c06b210 con 0x7f363c13fe40 2024-04-25T09:03:21.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.424+0000 7f3638c00640 1 -- 172.21.15.62:0/1234876551 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f3634032030 con 0x7f363c13fe40 2024-04-25T09:03:21.413 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:21.413 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:21.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f361c06b4d0 msgr2=0x7f361c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f361c06b4d0 0x7f361c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3630001eb0 tx=0x7f3630000f70 comp rx=0 tx=0).stop 2024-04-25T09:03:21.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c13fe40 msgr2=0x7f363c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c13fe40 0x7f363c0c7b90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3634007480 tx=0x7f3634004ab0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 shutdown_connections 2024-04-25T09:03:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f361c06b4d0 0x7f361c06d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f363c13fe40 0x7f363c0c7b90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f363c13f940 0x7f363c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 --2- 172.21.15.62:0/1234876551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3624004d40 0x7f363c13f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 >> 172.21.15.62:0/1234876551 conn(0x7f363c0b14d0 msgr2=0x7f363c15cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 shutdown_connections 2024-04-25T09:03:21.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.427+0000 7f3641e00640 1 -- 172.21.15.62:0/1234876551 wait complete. 2024-04-25T09:03:21.436 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:03:21.436 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.654+0000 7f8376800640 1 Processor -- start 2024-04-25T09:03:21.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.655+0000 7f8376800640 1 -- start start 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.656+0000 7f8376800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f837015b900 0x7f83700d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8376800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700d1cc0 0x7f83700b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8376800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f83700b6730 0x7f837015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8376800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83700d2120 con 0x7f83700b6730 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8376800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8370070a00 con 0x7f837015b900 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8376800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83700b6ba0 con 0x7f83700d1cc0 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8375e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f83700b6730 0x7f837015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8375e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f83700b6730 0x7f837015f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59432/0 (socket says 172.21.15.62:59432) 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8375e00640 1 -- 172.21.15.62:0/898324997 learned_addr learned my addr 172.21.15.62:0/898324997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:21.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8375400640 1 --2- 172.21.15.62:0/898324997 >> v2:172.21.15.121:3300/0 conn(0x7f837015b900 0x7f83700d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8374a00640 1 --2- 172.21.15.62:0/898324997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700d1cc0 0x7f83700b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.657+0000 7f8375e00640 1 -- 172.21.15.62:0/898324997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700d1cc0 msgr2=0x7f83700b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.658+0000 7f8375e00640 1 --2- 172.21.15.62:0/898324997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700d1cc0 0x7f83700b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.658+0000 7f8375e00640 1 -- 172.21.15.62:0/898324997 >> v2:172.21.15.121:3300/0 conn(0x7f837015b900 msgr2=0x7f83700d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.658+0000 7f8375e00640 1 --2- 172.21.15.62:0/898324997 >> v2:172.21.15.121:3300/0 conn(0x7f837015b900 0x7f83700d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.658+0000 7f8375e00640 1 -- 172.21.15.62:0/898324997 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83700cf890 con 0x7f83700b6730 2024-04-25T09:03:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.658+0000 7f8375400640 1 --2- 172.21.15.62:0/898324997 >> v2:172.21.15.121:3300/0 conn(0x7f837015b900 0x7f83700d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:21.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.658+0000 7f8374a00640 1 --2- 172.21.15.62:0/898324997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700d1cc0 0x7f83700b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.659+0000 7f8375e00640 1 --2- 172.21.15.62:0/898324997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700b6730 0x7f837015f5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f837006df60 tx=0x7f836802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecafa65c4d101e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.659+0000 7f836fe00640 1 -- 172.21.15.62:0/898324997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368037070 con 0x7f83700b6730 2024-04-25T09:03:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.659+0000 7f836fe00640 1 -- 172.21.15.62:0/898324997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8368032440 con 0x7f83700b6730 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.659+0000 7f836fe00640 1 -- 172.21.15.62:0/898324997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368032810 con 0x7f83700b6730 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 -- 172.21.15.62:0/898324997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700b6730 msgr2=0x7f837015f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 --2- 172.21.15.62:0/898324997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700b6730 0x7f837015f5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f837006df60 tx=0x7f836802a200 comp rx=0 tx=0).stop 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 -- 172.21.15.62:0/898324997 shutdown_connections 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 --2- 172.21.15.62:0/898324997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700b6730 0x7f837015f5e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 --2- 172.21.15.62:0/898324997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700d1cc0 0x7f83700b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 --2- 172.21.15.62:0/898324997 >> v2:172.21.15.121:3300/0 conn(0x7f837015b900 0x7f83700d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 -- 172.21.15.62:0/898324997 >> 172.21.15.62:0/898324997 conn(0x7f8370072340 msgr2=0x7f8370072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 -- 172.21.15.62:0/898324997 shutdown_connections 2024-04-25T09:03:21.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.660+0000 7f8376800640 1 -- 172.21.15.62:0/898324997 wait complete. 2024-04-25T09:03:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.661+0000 7f8376800640 1 Processor -- start 2024-04-25T09:03:21.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.662+0000 7f8376800640 1 -- start start 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.662+0000 7f8376800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700b6730 0x7f83700c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8376800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700d1cc0 0x7f83700c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8376800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f837015b900 0x7f8370154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8376800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83700703b0 con 0x7f83700d1cc0 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8376800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83700700d0 con 0x7f837015b900 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8376800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8370070690 con 0x7f83700b6730 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8374a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700d1cc0 0x7f83700c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8374a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700d1cc0 0x7f83700c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59434/0 (socket says 172.21.15.62:59434) 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8374a00640 1 -- 172.21.15.62:0/2085567556 learned_addr learned my addr 172.21.15.62:0/2085567556 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8375400640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700b6730 0x7f83700c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8375e00640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f837015b900 0x7f8370154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8374a00640 1 -- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700b6730 msgr2=0x7f83700c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8374a00640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700b6730 0x7f83700c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8374a00640 1 -- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f837015b900 msgr2=0x7f8370154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8374a00640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f837015b900 0x7f8370154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8374a00640 1 -- 172.21.15.62:0/2085567556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83700c6f40 con 0x7f83700d1cc0 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.663+0000 7f8375400640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700b6730 0x7f83700c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.664+0000 7f8374a00640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700d1cc0 0x7f83700c6a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c7c0 tx=0x7f835c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.664+0000 7f836d600640 1 -- 172.21.15.62:0/2085567556 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c015070 con 0x7f83700d1cc0 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.664+0000 7f836d600640 1 -- 172.21.15.62:0/2085567556 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c007a30 con 0x7f83700d1cc0 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.664+0000 7f836d600640 1 -- 172.21.15.62:0/2085567556 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c007e00 con 0x7f83700d1cc0 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.664+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83700c71e0 con 0x7f83700d1cc0 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.664+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83700c77a0 con 0x7f83700d1cc0 2024-04-25T09:03:21.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.665+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837006cd40 con 0x7f83700d1cc0 2024-04-25T09:03:21.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.666+0000 7f836d600640 1 -- 172.21.15.62:0/2085567556 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f835c018070 con 0x7f83700d1cc0 2024-04-25T09:03:21.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.667+0000 7f836d600640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f835006b500 0x7f835006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.667+0000 7f836d600640 1 -- 172.21.15.62:0/2085567556 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f835c08a0d0 con 0x7f83700d1cc0 2024-04-25T09:03:21.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.668+0000 7f8375400640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f835006b500 0x7f835006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.669+0000 7f8375400640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f835006b500 0x7f835006d9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8364007f40 tx=0x7f8364007490 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.672+0000 7f836d600640 1 -- 172.21.15.62:0/2085567556 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f835c058f50 con 0x7f83700d1cc0 2024-04-25T09:03:21.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.864+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f837015c4b0 con 0x7f835006b500 2024-04-25T09:03:21.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.885+0000 7f836d600640 1 -- 172.21.15.62:0/2085567556 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240067 (secure 0 0 0) 0x7f837015c4b0 con 0x7f835006b500 2024-04-25T09:03:21.876 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f835006b500 msgr2=0x7f835006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f835006b500 0x7f835006d9c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8364007f40 tx=0x7f8364007490 comp rx=0 tx=0).stop 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700d1cc0 msgr2=0x7f83700c6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700d1cc0 0x7f83700c6a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c7c0 tx=0x7f835c00cc80 comp rx=0 tx=0).stop 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 shutdown_connections 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f835006b500 0x7f835006d9c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83700d1cc0 0x7f83700c6a00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f837015b900 0x7f8370154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 --2- 172.21.15.62:0/2085567556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83700b6730 0x7f83700c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.889+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 >> 172.21.15.62:0/2085567556 conn(0x7f8370072340 msgr2=0x7f837015bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.890+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 shutdown_connections 2024-04-25T09:03:21.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:21.890+0000 7f8376800640 1 -- 172.21.15.62:0/2085567556 wait complete. 2024-04-25T09:03:21.897 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-25T09:03:19.873870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218632,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756088,"statfs":{"total":773094113280,"available":772870234112,"internally_reserved":0,"allocated":2531328,"data_stored":1593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.761229"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.127648+0000","last_change":"2024-04-25T09:03:15.127648+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.127648+0000","last_undegraded":"2024-04-25T09:03:15.127648+0000","last_fullsized":"2024-04-25T09:03:15.127648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.135452+0000","last_change":"2024-04-25T09:03:15.135452+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.135452+0000","last_undegraded":"2024-04-25T09:03:15.135452+0000","last_fullsized":"2024-04-25T09:03:15.135452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.136214+0000","last_change":"2024-04-25T09:03:15.136214+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.136214+0000","last_undegraded":"2024-04-25T09:03:15.136214+0000","last_fullsized":"2024-04-25T09:03:15.136214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.127950+0000","last_change":"2024-04-25T09:03:15.127950+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.127950+0000","last_undegraded":"2024-04-25T09:03:15.127950+0000","last_fullsized":"2024-04-25T09:03:15.127950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.133073+0000","last_change":"2024-04-25T09:03:15.133073+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.133073+0000","last_undegraded":"2024-04-25T09:03:15.133073+0000","last_fullsized":"2024-04-25T09:03:15.133073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.124544+0000","last_change":"2024-04-25T09:03:15.124544+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.124544+0000","last_undegraded":"2024-04-25T09:03:15.124544+0000","last_fullsized":"2024-04-25T09:03:15.124544+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.118410+0000","last_change":"2024-04-25T09:03:15.118410+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.118410+0000","last_undegraded":"2024-04-25T09:03:15.118410+0000","last_fullsized":"2024-04-25T09:03:15.118410+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.131113+0000","last_change":"2024-04-25T09:03:15.131113+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.131113+0000","last_undegraded":"2024-04-25T09:03:15.131113+0000","last_fullsized":"2024-04-25T09:03:15.131113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.119023+0000","last_change":"2024-04-25T09:03:15.119023+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.119023+0000","last_undegraded":"2024-04-25T09:03:15.119023+0000","last_fullsized":"2024-04-25T09:03:15.119023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.130746+0000","last_change":"2024-04-25T09:03:15.130746+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.130746+0000","last_undegraded":"2024-04-25T09:03:15.130746+0000","last_fullsized":"2024-04-25T09:03:15.130746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.124271+0000","last_change":"2024-04-25T09:03:15.124271+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.124271+0000","last_undegraded":"2024-04-25T09:03:15.124271+0000","last_fullsized":"2024-04-25T09:03:15.124271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.122632+0000","last_change":"2024-04-25T09:03:15.122632+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.122632+0000","last_undegraded":"2024-04-25T09:03:15.122632+0000","last_fullsized":"2024-04-25T09:03:15.122632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.124915+0000","last_change":"2024-04-25T09:03:15.124915+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.124915+0000","last_undegraded":"2024-04-25T09:03:15.124915+0000","last_fullsized":"2024-04-25T09:03:15.124915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.121908+0000","last_change":"2024-04-25T09:03:15.121908+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.121908+0000","last_undegraded":"2024-04-25T09:03:15.121908+0000","last_fullsized":"2024-04-25T09:03:15.121908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.127573+0000","last_change":"2024-04-25T09:03:15.127573+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.127573+0000","last_undegraded":"2024-04-25T09:03:15.127573+0000","last_fullsized":"2024-04-25T09:03:15.127573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.136267+0000","last_change":"2024-04-25T09:03:15.136267+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.136267+0000","last_undegraded":"2024-04-25T09:03:15.136267+0000","last_fullsized":"2024-04-25T09:03:15.136267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-25T09:03:15.135783+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:03:15.135783+0000","last_peered":"2024-04-25T09:03:15.135783+0000","last_clean":"2024-04-25T09:03:15.135783+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:03:15.135783+0000","last_undegraded":"2024-04-25T09:03:15.135783+0000","last_fullsized":"2024-04-25T09:03:15.135783+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.128075+0000","last_change":"2024-04-25T09:03:15.128075+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.128075+0000","last_undegraded":"2024-04-25T09:03:15.128075+0000","last_fullsized":"2024-04-25T09:03:15.128075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.126568+0000","last_change":"2024-04-25T09:03:15.126568+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.126568+0000","last_undegraded":"2024-04-25T09:03:15.126568+0000","last_fullsized":"2024-04-25T09:03:15.126568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.118437+0000","last_change":"2024-04-25T09:03:15.118437+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.118437+0000","last_undegraded":"2024-04-25T09:03:15.118437+0000","last_fullsized":"2024-04-25T09:03:15.118437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.128425+0000","last_change":"2024-04-25T09:03:15.128425+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.128425+0000","last_undegraded":"2024-04-25T09:03:15.128425+0000","last_fullsized":"2024-04-25T09:03:15.128425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.129140+0000","last_change":"2024-04-25T09:03:15.129140+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.129140+0000","last_undegraded":"2024-04-25T09:03:15.129140+0000","last_fullsized":"2024-04-25T09:03:15.129140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.129325+0000","last_change":"2024-04-25T09:03:15.129325+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.129325+0000","last_undegraded":"2024-04-25T09:03:15.129325+0000","last_fullsized":"2024-04-25T09:03:15.129325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.120129+0000","last_change":"2024-04-25T09:03:15.120129+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.120129+0000","last_undegraded":"2024-04-25T09:03:15.120129+0000","last_fullsized":"2024-04-25T09:03:15.120129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.129518+0000","last_change":"2024-04-25T09:03:15.129518+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.129518+0000","last_undegraded":"2024-04-25T09:03:15.129518+0000","last_fullsized":"2024-04-25T09:03:15.129518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.123638+0000","last_change":"2024-04-25T09:03:15.123638+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.123638+0000","last_undegraded":"2024-04-25T09:03:15.123638+0000","last_fullsized":"2024-04-25T09:03:15.123638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.120177+0000","last_change":"2024-04-25T09:03:15.120177+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.120177+0000","last_undegraded":"2024-04-25T09:03:15.120177+0000","last_fullsized":"2024-04-25T09:03:15.120177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.136340+0000","last_change":"2024-04-25T09:03:15.136340+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.136340+0000","last_undegraded":"2024-04-25T09:03:15.136340+0000","last_fullsized":"2024-04-25T09:03:15.136340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.130841+0000","last_change":"2024-04-25T09:03:15.130841+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.130841+0000","last_undegraded":"2024-04-25T09:03:15.130841+0000","last_fullsized":"2024-04-25T09:03:15.130841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.123849+0000","last_change":"2024-04-25T09:03:15.123849+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.123849+0000","last_undegraded":"2024-04-25T09:03:15.123849+0000","last_fullsized":"2024-04-25T09:03:15.123849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.124279+0000","last_change":"2024-04-25T09:03:15.124279+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.124279+0000","last_undegraded":"2024-04-25T09:03:15.124279+0000","last_fullsized":"2024-04-25T09:03:15.124279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.129013+0000","last_change":"2024-04-25T09:03:15.129013+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.129013+0000","last_undegraded":"2024-04-25T09:03:15.129013+0000","last_fullsized":"2024-04-25T09:03:15.129013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:03:16.111125+0000","last_change":"2024-04-25T09:03:16.111125+0000","last_active":"2024-04-25T09:03:16.111125+0000","last_peered":"2024-04-25T09:03:16.111125+0000","last_clean":"2024-04-25T09:03:16.111125+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:16.111125+0000","last_undegraded":"2024-04-25T09:03:16.111125+0000","last_fullsized":"2024-04-25T09:03:16.111125+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-25T09:03:15.120757+0000","last_change":"2024-04-25T09:03:15.120757+0000","last_active":"2024-04-25T09:03:15.105619+0000","last_peered":"2024-04-25T09:03:15.105619+0000","last_clean":"2024-04-25T09:03:15.105619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:03:15.120757+0000","last_undegraded":"2024-04-25T09:03:15.120757+0000","last_fullsized":"2024-04-25T09:03:15.120757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:21.901 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.163+0000 7ff30d800640 1 Processor -- start 2024-04-25T09:03:22.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.163+0000 7ff30d800640 1 -- start start 2024-04-25T09:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.165+0000 7ff30d800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff3080b56f0 0x7ff308160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.165+0000 7ff30d800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b6a90 0x7ff308161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.165+0000 7ff30d800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3080d3110 con 0x7ff3080b27a0 2024-04-25T09:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.165+0000 7ff30d800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff308161780 con 0x7ff3080b56f0 2024-04-25T09:03:22.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.165+0000 7ff30d800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080cedc0 con 0x7ff3080b6a90 2024-04-25T09:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.166+0000 7ff307400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b6a90 0x7ff308161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.166+0000 7ff307e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff3080b56f0 0x7ff308160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.166+0000 7ff307400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b6a90 0x7ff308161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41434/0 (socket says 172.21.15.62:41434) 2024-04-25T09:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.166+0000 7ff307400640 1 -- 172.21.15.62:0/488687770 learned_addr learned my addr 172.21.15.62:0/488687770 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.166+0000 7ff307400640 1 -- 172.21.15.62:0/488687770 >> v2:172.21.15.121:3300/0 conn(0x7ff3080b56f0 msgr2=0x7ff308160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.166+0000 7ff307400640 1 --2- 172.21.15.62:0/488687770 >> v2:172.21.15.121:3300/0 conn(0x7ff3080b56f0 0x7ff308160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.166+0000 7ff307400640 1 -- 172.21.15.62:0/488687770 >> v1:172.21.15.121:6789/0 conn(0x7ff3080b27a0 legacy=0x7ff308161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.166+0000 7ff307400640 1 -- 172.21.15.62:0/488687770 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080d33f0 con 0x7ff3080b6a90 2024-04-25T09:03:22.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.167+0000 7ff307400640 1 --2- 172.21.15.62:0/488687770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b6a90 0x7ff308161240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009870 tx=0x7ff2f402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96961201d29aa096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.168+0000 7ff306a00640 1 -- 172.21.15.62:0/488687770 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4037070 con 0x7ff3080b6a90 2024-04-25T09:03:22.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.168+0000 7ff306a00640 1 -- 172.21.15.62:0/488687770 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f4032440 con 0x7ff3080b6a90 2024-04-25T09:03:22.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.168+0000 7ff306a00640 1 -- 172.21.15.62:0/488687770 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4032810 con 0x7ff3080b6a90 2024-04-25T09:03:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.169+0000 7ff30d800640 1 -- 172.21.15.62:0/488687770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b6a90 msgr2=0x7ff308161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.169+0000 7ff30d800640 1 --2- 172.21.15.62:0/488687770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b6a90 0x7ff308161240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009870 tx=0x7ff2f402a250 comp rx=0 tx=0).stop 2024-04-25T09:03:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.169+0000 7ff30d800640 1 -- 172.21.15.62:0/488687770 shutdown_connections 2024-04-25T09:03:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.169+0000 7ff30d800640 1 --2- 172.21.15.62:0/488687770 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b6a90 0x7ff308161240 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.169+0000 7ff30d800640 1 --2- 172.21.15.62:0/488687770 >> v2:172.21.15.121:3300/0 conn(0x7ff3080b56f0 0x7ff308160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.169+0000 7ff30d800640 1 -- 172.21.15.62:0/488687770 >> 172.21.15.62:0/488687770 conn(0x7ff308066d10 msgr2=0x7ff308067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.169+0000 7ff30d800640 1 -- 172.21.15.62:0/488687770 shutdown_connections 2024-04-25T09:03:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.169+0000 7ff30d800640 1 -- 172.21.15.62:0/488687770 wait complete. 2024-04-25T09:03:22.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.171+0000 7ff30d800640 1 Processor -- start 2024-04-25T09:03:22.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.171+0000 7ff30d800640 1 -- start start 2024-04-25T09:03:22.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30d800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3080b27a0 0x7ff3080cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30d800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b56f0 0x7ff30815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3080b6a90 0x7ff3081560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30d800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080b2d60 con 0x7ff3080b6a90 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30d800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080cea50 con 0x7ff3080b27a0 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30d800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080b2ee0 con 0x7ff3080b56f0 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3080b6a90 0x7ff3081560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3080b6a90 0x7ff3081560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59440/0 (socket says 172.21.15.62:59440) 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff30ce00640 1 -- 172.21.15.62:0/317864031 learned_addr learned my addr 172.21.15.62:0/317864031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff307400640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b56f0 0x7ff30815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.173+0000 7ff307e00640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3080b27a0 0x7ff3080cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff30ce00640 1 -- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b56f0 msgr2=0x7ff30815b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff30ce00640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b56f0 0x7ff30815b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff30ce00640 1 -- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3080b27a0 msgr2=0x7ff3080cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff30ce00640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3080b27a0 0x7ff3080cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff30ce00640 1 -- 172.21.15.62:0/317864031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30815bb00 con 0x7ff3080b6a90 2024-04-25T09:03:22.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff307e00640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3080b27a0 0x7ff3080cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:22.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff30ce00640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3080b6a90 0x7ff3081560c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff30000c7a0 tx=0x7ff30000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff2ffe00640 1 -- 172.21.15.62:0/317864031 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300003b70 con 0x7ff3080b6a90 2024-04-25T09:03:22.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff2ffe00640 1 -- 172.21.15.62:0/317864031 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff300003d30 con 0x7ff3080b6a90 2024-04-25T09:03:22.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff2ffe00640 1 -- 172.21.15.62:0/317864031 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30000aa90 con 0x7ff3080b6a90 2024-04-25T09:03:22.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.174+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30815bde0 con 0x7ff3080b6a90 2024-04-25T09:03:22.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.175+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3080cc9a0 con 0x7ff3080b6a90 2024-04-25T09:03:22.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.176+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3080b9310 con 0x7ff3080b6a90 2024-04-25T09:03:22.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.177+0000 7ff2ffe00640 1 -- 172.21.15.62:0/317864031 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff30000ac30 con 0x7ff3080b6a90 2024-04-25T09:03:22.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.178+0000 7ff2ffe00640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2e806b500 0x7ff2e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.178+0000 7ff2ffe00640 1 -- 172.21.15.62:0/317864031 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff30008a0a0 con 0x7ff3080b6a90 2024-04-25T09:03:22.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.179+0000 7ff307e00640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2e806b500 0x7ff2e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.180+0000 7ff307e00640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2e806b500 0x7ff2e806d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009b90 tx=0x7ff2f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.182+0000 7ff2ffe00640 1 -- 172.21.15.62:0/317864031 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3080b9310 con 0x7ff3080b6a90 2024-04-25T09:03:22.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.378+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff30815b8a0 con 0x7ff2e806b500 2024-04-25T09:03:22.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.399+0000 7ff2ffe00640 1 -- 172.21.15.62:0/317864031 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251944 (secure 0 0 0) 0x7ff2f8011a60 con 0x7ff2e806b500 2024-04-25T09:03:22.388 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:22.392 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.405+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2e806b500 msgr2=0x7ff2e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.405+0000 7ff30d800640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2e806b500 0x7ff2e806d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009b90 tx=0x7ff2f8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.405+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3080b6a90 msgr2=0x7ff3081560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.405+0000 7ff30d800640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3080b6a90 0x7ff3081560c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff30000c7a0 tx=0x7ff30000cc60 comp rx=0 tx=0).stop 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.406+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 shutdown_connections 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.406+0000 7ff30d800640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2e806b500 0x7ff2e806d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.406+0000 7ff30d800640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3080b6a90 0x7ff3081560c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.406+0000 7ff30d800640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff3080b56f0 0x7ff30815b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.406+0000 7ff30d800640 1 --2- 172.21.15.62:0/317864031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3080b27a0 0x7ff3080cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.406+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 >> 172.21.15.62:0/317864031 conn(0x7ff308066d10 msgr2=0x7ff3080b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.406+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 shutdown_connections 2024-04-25T09:03:22.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.406+0000 7ff30d800640 1 -- 172.21.15.62:0/317864031 wait complete. 2024-04-25T09:03:22.415 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T09:03:21.884225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.011780"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172705+0000","last_change":"2024-04-25T09:03:17.179077+0000","last_active":"2024-04-25T09:03:18.172705+0000","last_peered":"2024-04-25T09:03:18.172705+0000","last_clean":"2024-04-25T09:03:18.172705+0000","last_became_active":"2024-04-25T09:03:17.178710+0000","last_became_peered":"2024-04-25T09:03:17.178710+0000","last_unstale":"2024-04-25T09:03:18.172705+0000","last_undegraded":"2024-04-25T09:03:18.172705+0000","last_fullsized":"2024-04-25T09:03:18.172705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565717+0000","last_change":"2024-04-25T09:03:17.164128+0000","last_active":"2024-04-25T09:03:18.565717+0000","last_peered":"2024-04-25T09:03:18.565717+0000","last_clean":"2024-04-25T09:03:18.565717+0000","last_became_active":"2024-04-25T09:03:17.163800+0000","last_became_peered":"2024-04-25T09:03:17.163800+0000","last_unstale":"2024-04-25T09:03:18.565717+0000","last_undegraded":"2024-04-25T09:03:18.565717+0000","last_fullsized":"2024-04-25T09:03:18.565717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:08.746742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565545+0000","last_change":"2024-04-25T09:03:17.167505+0000","last_active":"2024-04-25T09:03:18.565545+0000","last_peered":"2024-04-25T09:03:18.565545+0000","last_clean":"2024-04-25T09:03:18.565545+0000","last_became_active":"2024-04-25T09:03:17.167194+0000","last_became_peered":"2024-04-25T09:03:17.167194+0000","last_unstale":"2024-04-25T09:03:18.565545+0000","last_undegraded":"2024-04-25T09:03:18.565545+0000","last_fullsized":"2024-04-25T09:03:18.565545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:39.624252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562099+0000","last_change":"2024-04-25T09:03:17.162871+0000","last_active":"2024-04-25T09:03:18.562099+0000","last_peered":"2024-04-25T09:03:18.562099+0000","last_clean":"2024-04-25T09:03:18.562099+0000","last_became_active":"2024-04-25T09:03:17.162297+0000","last_became_peered":"2024-04-25T09:03:17.162297+0000","last_unstale":"2024-04-25T09:03:18.562099+0000","last_undegraded":"2024-04-25T09:03:18.562099+0000","last_fullsized":"2024-04-25T09:03:18.562099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171734+0000","last_change":"2024-04-25T09:03:17.172274+0000","last_active":"2024-04-25T09:03:18.171734+0000","last_peered":"2024-04-25T09:03:18.171734+0000","last_clean":"2024-04-25T09:03:18.171734+0000","last_became_active":"2024-04-25T09:03:17.171794+0000","last_became_peered":"2024-04-25T09:03:17.171794+0000","last_unstale":"2024-04-25T09:03:18.171734+0000","last_undegraded":"2024-04-25T09:03:18.171734+0000","last_fullsized":"2024-04-25T09:03:18.171734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562442+0000","last_change":"2024-04-25T09:03:17.158419+0000","last_active":"2024-04-25T09:03:18.562442+0000","last_peered":"2024-04-25T09:03:18.562442+0000","last_clean":"2024-04-25T09:03:18.562442+0000","last_became_active":"2024-04-25T09:03:17.155230+0000","last_became_peered":"2024-04-25T09:03:17.155230+0000","last_unstale":"2024-04-25T09:03:18.562442+0000","last_undegraded":"2024-04-25T09:03:18.562442+0000","last_fullsized":"2024-04-25T09:03:18.562442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172872+0000","last_change":"2024-04-25T09:03:17.165890+0000","last_active":"2024-04-25T09:03:18.172872+0000","last_peered":"2024-04-25T09:03:18.172872+0000","last_clean":"2024-04-25T09:03:18.172872+0000","last_became_active":"2024-04-25T09:03:17.164725+0000","last_became_peered":"2024-04-25T09:03:17.164725+0000","last_unstale":"2024-04-25T09:03:18.172872+0000","last_undegraded":"2024-04-25T09:03:18.172872+0000","last_fullsized":"2024-04-25T09:03:18.172872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:57.554441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563608+0000","last_change":"2024-04-25T09:03:17.162764+0000","last_active":"2024-04-25T09:03:18.563608+0000","last_peered":"2024-04-25T09:03:18.563608+0000","last_clean":"2024-04-25T09:03:18.563608+0000","last_became_active":"2024-04-25T09:03:17.162442+0000","last_became_peered":"2024-04-25T09:03:17.162442+0000","last_unstale":"2024-04-25T09:03:18.563608+0000","last_undegraded":"2024-04-25T09:03:18.563608+0000","last_fullsized":"2024-04-25T09:03:18.563608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:38.767433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151609+0000","last_change":"2024-04-25T09:03:17.156720+0000","last_active":"2024-04-25T09:03:18.151609+0000","last_peered":"2024-04-25T09:03:18.151609+0000","last_clean":"2024-04-25T09:03:18.151609+0000","last_became_active":"2024-04-25T09:03:17.156372+0000","last_became_peered":"2024-04-25T09:03:17.156372+0000","last_unstale":"2024-04-25T09:03:18.151609+0000","last_undegraded":"2024-04-25T09:03:18.151609+0000","last_fullsized":"2024-04-25T09:03:18.151609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172106+0000","last_change":"2024-04-25T09:03:17.160604+0000","last_active":"2024-04-25T09:03:18.172106+0000","last_peered":"2024-04-25T09:03:18.172106+0000","last_clean":"2024-04-25T09:03:18.172106+0000","last_became_active":"2024-04-25T09:03:17.157610+0000","last_became_peered":"2024-04-25T09:03:17.157610+0000","last_unstale":"2024-04-25T09:03:18.172106+0000","last_undegraded":"2024-04-25T09:03:18.172106+0000","last_fullsized":"2024-04-25T09:03:18.172106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:08.746742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172219+0000","last_change":"2024-04-25T09:03:17.172025+0000","last_active":"2024-04-25T09:03:18.172219+0000","last_peered":"2024-04-25T09:03:18.172219+0000","last_clean":"2024-04-25T09:03:18.172219+0000","last_became_active":"2024-04-25T09:03:17.171653+0000","last_became_peered":"2024-04-25T09:03:17.171653+0000","last_unstale":"2024-04-25T09:03:18.172219+0000","last_undegraded":"2024-04-25T09:03:18.172219+0000","last_fullsized":"2024-04-25T09:03:18.172219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:22.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172409+0000","last_change":"2024-04-25T09:03:17.155748+0000","last_active":"2024-04-25T09:03:18.172409+0000","last_peered":"2024-04-25T09:03:18.172409+0000","last_clean":"2024-04-25T09:03:18.172409+0000","last_became_active":"2024-04-25T09:03:17.155156+0000","last_became_peered":"2024-04-25T09:03:17.155156+0000","last_unstale":"2024-04-25T09:03:18.172409+0000","last_undegraded":"2024-04-25T09:03:18.172409+0000","last_fullsized":"2024-04-25T09:03:18.172409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:13.218030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563558+0000","last_change":"2024-04-25T09:03:17.162578+0000","last_active":"2024-04-25T09:03:18.563558+0000","last_peered":"2024-04-25T09:03:18.563558+0000","last_clean":"2024-04-25T09:03:18.563558+0000","last_became_active":"2024-04-25T09:03:17.162284+0000","last_became_peered":"2024-04-25T09:03:17.162284+0000","last_unstale":"2024-04-25T09:03:18.563558+0000","last_undegraded":"2024-04-25T09:03:18.563558+0000","last_fullsized":"2024-04-25T09:03:18.563558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:34.254072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563388+0000","last_change":"2024-04-25T09:03:17.150222+0000","last_active":"2024-04-25T09:03:18.563388+0000","last_peered":"2024-04-25T09:03:18.563388+0000","last_clean":"2024-04-25T09:03:18.563388+0000","last_became_active":"2024-04-25T09:03:17.149717+0000","last_became_peered":"2024-04-25T09:03:17.149717+0000","last_unstale":"2024-04-25T09:03:18.563388+0000","last_undegraded":"2024-04-25T09:03:18.563388+0000","last_fullsized":"2024-04-25T09:03:18.563388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564802+0000","last_change":"2024-04-25T09:03:17.156094+0000","last_active":"2024-04-25T09:03:18.564802+0000","last_peered":"2024-04-25T09:03:18.564802+0000","last_clean":"2024-04-25T09:03:18.564802+0000","last_became_active":"2024-04-25T09:03:17.155818+0000","last_became_peered":"2024-04-25T09:03:17.155818+0000","last_unstale":"2024-04-25T09:03:18.564802+0000","last_undegraded":"2024-04-25T09:03:18.564802+0000","last_fullsized":"2024-04-25T09:03:18.564802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.326298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561761+0000","last_change":"2024-04-25T09:03:17.164616+0000","last_active":"2024-04-25T09:03:18.561761+0000","last_peered":"2024-04-25T09:03:18.561761+0000","last_clean":"2024-04-25T09:03:18.561761+0000","last_became_active":"2024-04-25T09:03:17.158855+0000","last_became_peered":"2024-04-25T09:03:17.158855+0000","last_unstale":"2024-04-25T09:03:18.561761+0000","last_undegraded":"2024-04-25T09:03:18.561761+0000","last_fullsized":"2024-04-25T09:03:18.561761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565221+0000","last_change":"2024-04-25T09:03:17.163995+0000","last_active":"2024-04-25T09:03:18.565221+0000","last_peered":"2024-04-25T09:03:18.565221+0000","last_clean":"2024-04-25T09:03:18.565221+0000","last_became_active":"2024-04-25T09:03:17.162714+0000","last_became_peered":"2024-04-25T09:03:17.162714+0000","last_unstale":"2024-04-25T09:03:18.565221+0000","last_undegraded":"2024-04-25T09:03:18.565221+0000","last_fullsized":"2024-04-25T09:03:18.565221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:54.326567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567960+0000","last_change":"2024-04-25T09:03:17.159140+0000","last_active":"2024-04-25T09:03:18.567960+0000","last_peered":"2024-04-25T09:03:18.567960+0000","last_clean":"2024-04-25T09:03:18.567960+0000","last_became_active":"2024-04-25T09:03:17.158674+0000","last_became_peered":"2024-04-25T09:03:17.158674+0000","last_unstale":"2024-04-25T09:03:18.567960+0000","last_undegraded":"2024-04-25T09:03:18.567960+0000","last_fullsized":"2024-04-25T09:03:18.567960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564565+0000","last_change":"2024-04-25T09:03:17.153159+0000","last_active":"2024-04-25T09:03:18.564565+0000","last_peered":"2024-04-25T09:03:18.564565+0000","last_clean":"2024-04-25T09:03:18.564565+0000","last_became_active":"2024-04-25T09:03:17.152817+0000","last_became_peered":"2024-04-25T09:03:17.152817+0000","last_unstale":"2024-04-25T09:03:18.564565+0000","last_undegraded":"2024-04-25T09:03:18.564565+0000","last_fullsized":"2024-04-25T09:03:18.564565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:38.133534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172856+0000","last_change":"2024-04-25T09:03:17.155424+0000","last_active":"2024-04-25T09:03:18.172856+0000","last_peered":"2024-04-25T09:03:18.172856+0000","last_clean":"2024-04-25T09:03:18.172856+0000","last_became_active":"2024-04-25T09:03:17.154812+0000","last_became_peered":"2024-04-25T09:03:17.154812+0000","last_unstale":"2024-04-25T09:03:18.172856+0000","last_undegraded":"2024-04-25T09:03:18.172856+0000","last_fullsized":"2024-04-25T09:03:18.172856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:08.323326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154338+0000","last_change":"2024-04-25T09:03:17.156092+0000","last_active":"2024-04-25T09:03:18.154338+0000","last_peered":"2024-04-25T09:03:18.154338+0000","last_clean":"2024-04-25T09:03:18.154338+0000","last_became_active":"2024-04-25T09:03:17.153705+0000","last_became_peered":"2024-04-25T09:03:17.153705+0000","last_unstale":"2024-04-25T09:03:18.154338+0000","last_undegraded":"2024-04-25T09:03:18.154338+0000","last_fullsized":"2024-04-25T09:03:18.154338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:38.767433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150850+0000","last_change":"2024-04-25T09:03:17.155198+0000","last_active":"2024-04-25T09:03:18.150850+0000","last_peered":"2024-04-25T09:03:18.150850+0000","last_clean":"2024-04-25T09:03:18.150850+0000","last_became_active":"2024-04-25T09:03:17.154848+0000","last_became_peered":"2024-04-25T09:03:17.154848+0000","last_unstale":"2024-04-25T09:03:18.150850+0000","last_undegraded":"2024-04-25T09:03:18.150850+0000","last_fullsized":"2024-04-25T09:03:18.150850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:38.838938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564958+0000","last_change":"2024-04-25T09:03:17.161601+0000","last_active":"2024-04-25T09:03:18.564958+0000","last_peered":"2024-04-25T09:03:18.564958+0000","last_clean":"2024-04-25T09:03:18.564958+0000","last_became_active":"2024-04-25T09:03:17.161303+0000","last_became_peered":"2024-04-25T09:03:17.161303+0000","last_unstale":"2024-04-25T09:03:18.564958+0000","last_undegraded":"2024-04-25T09:03:18.564958+0000","last_fullsized":"2024-04-25T09:03:18.564958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:14.651686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562409+0000","last_change":"2024-04-25T09:03:17.168111+0000","last_active":"2024-04-25T09:03:18.562409+0000","last_peered":"2024-04-25T09:03:18.562409+0000","last_clean":"2024-04-25T09:03:18.562409+0000","last_became_active":"2024-04-25T09:03:17.165747+0000","last_became_peered":"2024-04-25T09:03:17.165747+0000","last_unstale":"2024-04-25T09:03:18.562409+0000","last_undegraded":"2024-04-25T09:03:18.562409+0000","last_fullsized":"2024-04-25T09:03:18.562409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:15.948906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565923+0000","last_change":"2024-04-25T09:03:17.161196+0000","last_active":"2024-04-25T09:03:18.565923+0000","last_peered":"2024-04-25T09:03:18.565923+0000","last_clean":"2024-04-25T09:03:18.565923+0000","last_became_active":"2024-04-25T09:03:17.158516+0000","last_became_peered":"2024-04-25T09:03:17.158516+0000","last_unstale":"2024-04-25T09:03:18.565923+0000","last_undegraded":"2024-04-25T09:03:18.565923+0000","last_fullsized":"2024-04-25T09:03:18.565923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:22.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563329+0000","last_change":"2024-04-25T09:03:17.164003+0000","last_active":"2024-04-25T09:03:18.563329+0000","last_peered":"2024-04-25T09:03:18.563329+0000","last_clean":"2024-04-25T09:03:18.563329+0000","last_became_active":"2024-04-25T09:03:17.162280+0000","last_became_peered":"2024-04-25T09:03:17.162280+0000","last_unstale":"2024-04-25T09:03:18.563329+0000","last_undegraded":"2024-04-25T09:03:18.563329+0000","last_fullsized":"2024-04-25T09:03:18.563329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:16.897576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564687+0000","last_change":"2024-04-25T09:03:17.155201+0000","last_active":"2024-04-25T09:03:18.564687+0000","last_peered":"2024-04-25T09:03:18.564687+0000","last_clean":"2024-04-25T09:03:18.564687+0000","last_became_active":"2024-04-25T09:03:17.154820+0000","last_became_peered":"2024-04-25T09:03:17.154820+0000","last_unstale":"2024-04-25T09:03:18.564687+0000","last_undegraded":"2024-04-25T09:03:18.564687+0000","last_fullsized":"2024-04-25T09:03:18.564687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:50.727941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563484+0000","last_change":"2024-04-25T09:03:17.155327+0000","last_active":"2024-04-25T09:03:18.563484+0000","last_peered":"2024-04-25T09:03:18.563484+0000","last_clean":"2024-04-25T09:03:18.563484+0000","last_became_active":"2024-04-25T09:03:17.155036+0000","last_became_peered":"2024-04-25T09:03:17.155036+0000","last_unstale":"2024-04-25T09:03:18.563484+0000","last_undegraded":"2024-04-25T09:03:18.563484+0000","last_fullsized":"2024-04-25T09:03:18.563484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:16.897576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565403+0000","last_change":"2024-04-25T09:03:17.157840+0000","last_active":"2024-04-25T09:03:18.565403+0000","last_peered":"2024-04-25T09:03:18.565403+0000","last_clean":"2024-04-25T09:03:18.565403+0000","last_became_active":"2024-04-25T09:03:17.157246+0000","last_became_peered":"2024-04-25T09:03:17.157246+0000","last_unstale":"2024-04-25T09:03:18.565403+0000","last_undegraded":"2024-04-25T09:03:18.565403+0000","last_fullsized":"2024-04-25T09:03:18.565403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563239+0000","last_change":"2024-04-25T09:03:17.170837+0000","last_active":"2024-04-25T09:03:18.563239+0000","last_peered":"2024-04-25T09:03:18.563239+0000","last_clean":"2024-04-25T09:03:18.563239+0000","last_became_active":"2024-04-25T09:03:17.166967+0000","last_became_peered":"2024-04-25T09:03:17.166967+0000","last_unstale":"2024-04-25T09:03:18.563239+0000","last_undegraded":"2024-04-25T09:03:18.563239+0000","last_fullsized":"2024-04-25T09:03:18.563239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:36.933011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567140+0000","last_change":"2024-04-25T09:03:17.145733+0000","last_active":"2024-04-25T09:03:18.567140+0000","last_peered":"2024-04-25T09:03:18.567140+0000","last_clean":"2024-04-25T09:03:18.567140+0000","last_became_active":"2024-04-25T09:03:17.145233+0000","last_became_peered":"2024-04-25T09:03:17.145233+0000","last_unstale":"2024-04-25T09:03:18.567140+0000","last_undegraded":"2024-04-25T09:03:18.567140+0000","last_fullsized":"2024-04-25T09:03:18.567140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:18.503932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561657+0000","last_change":"2024-04-25T09:03:17.157243+0000","last_active":"2024-04-25T09:03:18.561657+0000","last_peered":"2024-04-25T09:03:18.561657+0000","last_clean":"2024-04-25T09:03:18.561657+0000","last_became_active":"2024-04-25T09:03:17.156940+0000","last_became_peered":"2024-04-25T09:03:17.156940+0000","last_unstale":"2024-04-25T09:03:18.561657+0000","last_undegraded":"2024-04-25T09:03:18.561657+0000","last_fullsized":"2024-04-25T09:03:18.561657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150683+0000","last_change":"2024-04-25T09:03:17.156642+0000","last_active":"2024-04-25T09:03:18.150683+0000","last_peered":"2024-04-25T09:03:18.150683+0000","last_clean":"2024-04-25T09:03:18.150683+0000","last_became_active":"2024-04-25T09:03:17.155097+0000","last_became_peered":"2024-04-25T09:03:17.155097+0000","last_unstale":"2024-04-25T09:03:18.150683+0000","last_undegraded":"2024-04-25T09:03:18.150683+0000","last_fullsized":"2024-04-25T09:03:18.150683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565202+0000","last_change":"2024-04-25T09:03:16.166854+0000","last_active":"2024-04-25T09:03:18.565202+0000","last_peered":"2024-04-25T09:03:18.565202+0000","last_clean":"2024-04-25T09:03:18.565202+0000","last_became_active":"2024-04-25T09:03:16.166523+0000","last_became_peered":"2024-04-25T09:03:16.166523+0000","last_unstale":"2024-04-25T09:03:18.565202+0000","last_undegraded":"2024-04-25T09:03:18.565202+0000","last_fullsized":"2024-04-25T09:03:18.565202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154448+0000","last_change":"2024-04-25T09:03:17.158882+0000","last_active":"2024-04-25T09:03:18.154448+0000","last_peered":"2024-04-25T09:03:18.154448+0000","last_clean":"2024-04-25T09:03:18.154448+0000","last_became_active":"2024-04-25T09:03:17.158589+0000","last_became_peered":"2024-04-25T09:03:17.158589+0000","last_unstale":"2024-04-25T09:03:18.154448+0000","last_undegraded":"2024-04-25T09:03:18.154448+0000","last_fullsized":"2024-04-25T09:03:18.154448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:19.521616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563570+0000","last_change":"2024-04-25T09:03:16.168242+0000","last_active":"2024-04-25T09:03:18.563570+0000","last_peered":"2024-04-25T09:03:18.563570+0000","last_clean":"2024-04-25T09:03:18.563570+0000","last_became_active":"2024-04-25T09:03:16.167927+0000","last_became_peered":"2024-04-25T09:03:16.167927+0000","last_unstale":"2024-04-25T09:03:18.563570+0000","last_undegraded":"2024-04-25T09:03:18.563570+0000","last_fullsized":"2024-04-25T09:03:18.563570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:15.892070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562728+0000","last_change":"2024-04-25T09:03:16.163666+0000","last_active":"2024-04-25T09:03:18.562728+0000","last_peered":"2024-04-25T09:03:18.562728+0000","last_clean":"2024-04-25T09:03:18.562728+0000","last_became_active":"2024-04-25T09:03:16.163321+0000","last_became_peered":"2024-04-25T09:03:16.163321+0000","last_unstale":"2024-04-25T09:03:18.562728+0000","last_undegraded":"2024-04-25T09:03:18.562728+0000","last_fullsized":"2024-04-25T09:03:18.562728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564262+0000","last_change":"2024-04-25T09:03:17.163289+0000","last_active":"2024-04-25T09:03:18.564262+0000","last_peered":"2024-04-25T09:03:18.564262+0000","last_clean":"2024-04-25T09:03:18.564262+0000","last_became_active":"2024-04-25T09:03:17.162997+0000","last_became_peered":"2024-04-25T09:03:17.162997+0000","last_unstale":"2024-04-25T09:03:18.564262+0000","last_undegraded":"2024-04-25T09:03:18.564262+0000","last_fullsized":"2024-04-25T09:03:18.564262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:19.521616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561915+0000","last_change":"2024-04-25T09:03:16.146080+0000","last_active":"2024-04-25T09:03:18.561915+0000","last_peered":"2024-04-25T09:03:18.561915+0000","last_clean":"2024-04-25T09:03:18.561915+0000","last_became_active":"2024-04-25T09:03:16.145743+0000","last_became_peered":"2024-04-25T09:03:16.145743+0000","last_unstale":"2024-04-25T09:03:18.561915+0000","last_undegraded":"2024-04-25T09:03:18.561915+0000","last_fullsized":"2024-04-25T09:03:18.561915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:37.833432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154329+0000","last_change":"2024-04-25T09:03:17.146618+0000","last_active":"2024-04-25T09:03:18.154329+0000","last_peered":"2024-04-25T09:03:18.154329+0000","last_clean":"2024-04-25T09:03:18.154329+0000","last_became_active":"2024-04-25T09:03:17.146174+0000","last_became_peered":"2024-04-25T09:03:17.146174+0000","last_unstale":"2024-04-25T09:03:18.154329+0000","last_undegraded":"2024-04-25T09:03:18.154329+0000","last_fullsized":"2024-04-25T09:03:18.154329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:50.727941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151477+0000","last_change":"2024-04-25T09:03:16.159981+0000","last_active":"2024-04-25T09:03:18.151477+0000","last_peered":"2024-04-25T09:03:18.151477+0000","last_clean":"2024-04-25T09:03:18.151477+0000","last_became_active":"2024-04-25T09:03:16.159115+0000","last_became_peered":"2024-04-25T09:03:16.159115+0000","last_unstale":"2024-04-25T09:03:18.151477+0000","last_undegraded":"2024-04-25T09:03:18.151477+0000","last_fullsized":"2024-04-25T09:03:18.151477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563445+0000","last_change":"2024-04-25T09:03:17.163768+0000","last_active":"2024-04-25T09:03:18.563445+0000","last_peered":"2024-04-25T09:03:18.563445+0000","last_clean":"2024-04-25T09:03:18.563445+0000","last_became_active":"2024-04-25T09:03:17.162151+0000","last_became_peered":"2024-04-25T09:03:17.162151+0000","last_unstale":"2024-04-25T09:03:18.563445+0000","last_undegraded":"2024-04-25T09:03:18.563445+0000","last_fullsized":"2024-04-25T09:03:18.563445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:34.254072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172215+0000","last_change":"2024-04-25T09:03:16.169333+0000","last_active":"2024-04-25T09:03:18.172215+0000","last_peered":"2024-04-25T09:03:18.172215+0000","last_clean":"2024-04-25T09:03:18.172215+0000","last_became_active":"2024-04-25T09:03:16.167170+0000","last_became_peered":"2024-04-25T09:03:16.167170+0000","last_unstale":"2024-04-25T09:03:18.172215+0000","last_undegraded":"2024-04-25T09:03:18.172215+0000","last_fullsized":"2024-04-25T09:03:18.172215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:37.833432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566351+0000","last_change":"2024-04-25T09:03:17.168947+0000","last_active":"2024-04-25T09:03:18.566351+0000","last_peered":"2024-04-25T09:03:18.566351+0000","last_clean":"2024-04-25T09:03:18.566351+0000","last_became_active":"2024-04-25T09:03:17.168658+0000","last_became_peered":"2024-04-25T09:03:17.168658+0000","last_unstale":"2024-04-25T09:03:18.566351+0000","last_undegraded":"2024-04-25T09:03:18.566351+0000","last_fullsized":"2024-04-25T09:03:18.566351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567577+0000","last_change":"2024-04-25T09:03:16.160547+0000","last_active":"2024-04-25T09:03:18.567577+0000","last_peered":"2024-04-25T09:03:18.567577+0000","last_clean":"2024-04-25T09:03:18.567577+0000","last_became_active":"2024-04-25T09:03:16.160169+0000","last_became_peered":"2024-04-25T09:03:16.160169+0000","last_unstale":"2024-04-25T09:03:18.567577+0000","last_undegraded":"2024-04-25T09:03:18.567577+0000","last_fullsized":"2024-04-25T09:03:18.567577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564136+0000","last_change":"2024-04-25T09:03:17.161468+0000","last_active":"2024-04-25T09:03:18.564136+0000","last_peered":"2024-04-25T09:03:18.564136+0000","last_clean":"2024-04-25T09:03:18.564136+0000","last_became_active":"2024-04-25T09:03:17.159824+0000","last_became_peered":"2024-04-25T09:03:17.159824+0000","last_unstale":"2024-04-25T09:03:18.564136+0000","last_undegraded":"2024-04-25T09:03:18.564136+0000","last_fullsized":"2024-04-25T09:03:18.564136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:37.830284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562202+0000","last_change":"2024-04-25T09:03:16.155839+0000","last_active":"2024-04-25T09:03:18.562202+0000","last_peered":"2024-04-25T09:03:18.562202+0000","last_clean":"2024-04-25T09:03:18.562202+0000","last_became_active":"2024-04-25T09:03:16.155363+0000","last_became_peered":"2024-04-25T09:03:16.155363+0000","last_unstale":"2024-04-25T09:03:18.562202+0000","last_undegraded":"2024-04-25T09:03:18.562202+0000","last_fullsized":"2024-04-25T09:03:18.562202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:13.008112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172333+0000","last_change":"2024-04-25T09:03:17.158443+0000","last_active":"2024-04-25T09:03:18.172333+0000","last_peered":"2024-04-25T09:03:18.172333+0000","last_clean":"2024-04-25T09:03:18.172333+0000","last_became_active":"2024-04-25T09:03:17.158153+0000","last_became_peered":"2024-04-25T09:03:17.158153+0000","last_unstale":"2024-04-25T09:03:18.172333+0000","last_undegraded":"2024-04-25T09:03:18.172333+0000","last_fullsized":"2024-04-25T09:03:18.172333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.046413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567015+0000","last_change":"2024-04-25T09:03:16.170150+0000","last_active":"2024-04-25T09:03:18.567015+0000","last_peered":"2024-04-25T09:03:18.567015+0000","last_clean":"2024-04-25T09:03:18.567015+0000","last_became_active":"2024-04-25T09:03:16.169528+0000","last_became_peered":"2024-04-25T09:03:16.169528+0000","last_unstale":"2024-04-25T09:03:18.567015+0000","last_undegraded":"2024-04-25T09:03:18.567015+0000","last_fullsized":"2024-04-25T09:03:18.567015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563761+0000","last_change":"2024-04-25T09:03:17.161896+0000","last_active":"2024-04-25T09:03:18.563761+0000","last_peered":"2024-04-25T09:03:18.563761+0000","last_clean":"2024-04-25T09:03:18.563761+0000","last_became_active":"2024-04-25T09:03:17.161609+0000","last_became_peered":"2024-04-25T09:03:17.161609+0000","last_unstale":"2024-04-25T09:03:18.563761+0000","last_undegraded":"2024-04-25T09:03:18.563761+0000","last_fullsized":"2024-04-25T09:03:18.563761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563685+0000","last_change":"2024-04-25T09:03:16.150079+0000","last_active":"2024-04-25T09:03:18.563685+0000","last_peered":"2024-04-25T09:03:18.563685+0000","last_clean":"2024-04-25T09:03:18.563685+0000","last_became_active":"2024-04-25T09:03:16.149418+0000","last_became_peered":"2024-04-25T09:03:16.149418+0000","last_unstale":"2024-04-25T09:03:18.563685+0000","last_undegraded":"2024-04-25T09:03:18.563685+0000","last_fullsized":"2024-04-25T09:03:18.563685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:33.248566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564016+0000","last_change":"2024-04-25T09:03:17.169709+0000","last_active":"2024-04-25T09:03:18.564016+0000","last_peered":"2024-04-25T09:03:18.564016+0000","last_clean":"2024-04-25T09:03:18.564016+0000","last_became_active":"2024-04-25T09:03:17.169402+0000","last_became_peered":"2024-04-25T09:03:17.169402+0000","last_unstale":"2024-04-25T09:03:18.564016+0000","last_undegraded":"2024-04-25T09:03:18.564016+0000","last_fullsized":"2024-04-25T09:03:18.564016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:23.474225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171619+0000","last_change":"2024-04-25T09:03:16.164546+0000","last_active":"2024-04-25T09:03:18.171619+0000","last_peered":"2024-04-25T09:03:18.171619+0000","last_clean":"2024-04-25T09:03:18.171619+0000","last_became_active":"2024-04-25T09:03:16.162541+0000","last_became_peered":"2024-04-25T09:03:16.162541+0000","last_unstale":"2024-04-25T09:03:18.171619+0000","last_undegraded":"2024-04-25T09:03:18.171619+0000","last_fullsized":"2024-04-25T09:03:18.171619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:15.408869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150694+0000","last_change":"2024-04-25T09:03:17.156971+0000","last_active":"2024-04-25T09:03:18.150694+0000","last_peered":"2024-04-25T09:03:18.150694+0000","last_clean":"2024-04-25T09:03:18.150694+0000","last_became_active":"2024-04-25T09:03:17.156624+0000","last_became_peered":"2024-04-25T09:03:17.156624+0000","last_unstale":"2024-04-25T09:03:18.150694+0000","last_undegraded":"2024-04-25T09:03:18.150694+0000","last_fullsized":"2024-04-25T09:03:18.150694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:16.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153276+0000","last_change":"2024-04-25T09:03:16.155291+0000","last_active":"2024-04-25T09:03:18.153276+0000","last_peered":"2024-04-25T09:03:18.153276+0000","last_clean":"2024-04-25T09:03:18.153276+0000","last_became_active":"2024-04-25T09:03:16.154984+0000","last_became_peered":"2024-04-25T09:03:16.154984+0000","last_unstale":"2024-04-25T09:03:18.153276+0000","last_undegraded":"2024-04-25T09:03:18.153276+0000","last_fullsized":"2024-04-25T09:03:18.153276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562872+0000","last_change":"2024-04-25T09:03:17.163557+0000","last_active":"2024-04-25T09:03:18.562872+0000","last_peered":"2024-04-25T09:03:18.562872+0000","last_clean":"2024-04-25T09:03:18.562872+0000","last_became_active":"2024-04-25T09:03:17.163233+0000","last_became_peered":"2024-04-25T09:03:17.163233+0000","last_unstale":"2024-04-25T09:03:18.562872+0000","last_undegraded":"2024-04-25T09:03:18.562872+0000","last_fullsized":"2024-04-25T09:03:18.562872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:08.323326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172841+0000","last_change":"2024-04-25T09:03:16.168391+0000","last_active":"2024-04-25T09:03:18.172841+0000","last_peered":"2024-04-25T09:03:18.172841+0000","last_clean":"2024-04-25T09:03:18.172841+0000","last_became_active":"2024-04-25T09:03:16.168048+0000","last_became_peered":"2024-04-25T09:03:16.168048+0000","last_unstale":"2024-04-25T09:03:18.172841+0000","last_undegraded":"2024-04-25T09:03:18.172841+0000","last_fullsized":"2024-04-25T09:03:18.172841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:49.931470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563750+0000","last_change":"2024-04-25T09:03:17.173328+0000","last_active":"2024-04-25T09:03:18.563750+0000","last_peered":"2024-04-25T09:03:18.563750+0000","last_clean":"2024-04-25T09:03:18.563750+0000","last_became_active":"2024-04-25T09:03:17.165984+0000","last_became_peered":"2024-04-25T09:03:17.165984+0000","last_unstale":"2024-04-25T09:03:18.563750+0000","last_undegraded":"2024-04-25T09:03:18.563750+0000","last_fullsized":"2024-04-25T09:03:18.563750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153832+0000","last_change":"2024-04-25T09:03:16.167227+0000","last_active":"2024-04-25T09:03:18.153832+0000","last_peered":"2024-04-25T09:03:18.153832+0000","last_clean":"2024-04-25T09:03:18.153832+0000","last_became_active":"2024-04-25T09:03:16.166892+0000","last_became_peered":"2024-04-25T09:03:16.166892+0000","last_unstale":"2024-04-25T09:03:18.153832+0000","last_undegraded":"2024-04-25T09:03:18.153832+0000","last_fullsized":"2024-04-25T09:03:18.153832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172826+0000","last_change":"2024-04-25T09:03:17.163685+0000","last_active":"2024-04-25T09:03:18.172826+0000","last_peered":"2024-04-25T09:03:18.172826+0000","last_clean":"2024-04-25T09:03:18.172826+0000","last_became_active":"2024-04-25T09:03:17.163386+0000","last_became_peered":"2024-04-25T09:03:17.163386+0000","last_unstale":"2024-04-25T09:03:18.172826+0000","last_undegraded":"2024-04-25T09:03:18.172826+0000","last_fullsized":"2024-04-25T09:03:18.172826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:15.948906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565656+0000","last_change":"2024-04-25T09:03:16.160968+0000","last_active":"2024-04-25T09:03:18.565656+0000","last_peered":"2024-04-25T09:03:18.565656+0000","last_clean":"2024-04-25T09:03:18.565656+0000","last_became_active":"2024-04-25T09:03:16.158804+0000","last_became_peered":"2024-04-25T09:03:16.158804+0000","last_unstale":"2024-04-25T09:03:18.565656+0000","last_undegraded":"2024-04-25T09:03:18.565656+0000","last_fullsized":"2024-04-25T09:03:18.565656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.173017+0000","last_change":"2024-04-25T09:03:17.162364+0000","last_active":"2024-04-25T09:03:18.173017+0000","last_peered":"2024-04-25T09:03:18.173017+0000","last_clean":"2024-04-25T09:03:18.173017+0000","last_became_active":"2024-04-25T09:03:17.162093+0000","last_became_peered":"2024-04-25T09:03:17.162093+0000","last_unstale":"2024-04-25T09:03:18.173017+0000","last_undegraded":"2024-04-25T09:03:18.173017+0000","last_fullsized":"2024-04-25T09:03:18.173017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:36.933011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563994+0000","last_change":"2024-04-25T09:03:16.167573+0000","last_active":"2024-04-25T09:03:18.563994+0000","last_peered":"2024-04-25T09:03:18.563994+0000","last_clean":"2024-04-25T09:03:18.563994+0000","last_became_active":"2024-04-25T09:03:16.167279+0000","last_became_peered":"2024-04-25T09:03:16.167279+0000","last_unstale":"2024-04-25T09:03:18.563994+0000","last_undegraded":"2024-04-25T09:03:18.563994+0000","last_fullsized":"2024-04-25T09:03:18.563994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564569+0000","last_change":"2024-04-25T09:03:17.160049+0000","last_active":"2024-04-25T09:03:18.564569+0000","last_peered":"2024-04-25T09:03:18.564569+0000","last_clean":"2024-04-25T09:03:18.564569+0000","last_became_active":"2024-04-25T09:03:17.157508+0000","last_became_peered":"2024-04-25T09:03:17.157508+0000","last_unstale":"2024-04-25T09:03:18.564569+0000","last_undegraded":"2024-04-25T09:03:18.564569+0000","last_fullsized":"2024-04-25T09:03:18.564569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:25.867031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150373+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:03:18.150373+0000","last_peered":"2024-04-25T09:03:18.150373+0000","last_clean":"2024-04-25T09:03:18.150373+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:03:18.150373+0000","last_undegraded":"2024-04-25T09:03:18.150373+0000","last_fullsized":"2024-04-25T09:03:18.150373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564168+0000","last_change":"2024-04-25T09:03:16.152466+0000","last_active":"2024-04-25T09:03:18.564168+0000","last_peered":"2024-04-25T09:03:18.564168+0000","last_clean":"2024-04-25T09:03:18.564168+0000","last_became_active":"2024-04-25T09:03:16.152216+0000","last_became_peered":"2024-04-25T09:03:16.152216+0000","last_unstale":"2024-04-25T09:03:18.564168+0000","last_undegraded":"2024-04-25T09:03:18.564168+0000","last_fullsized":"2024-04-25T09:03:18.564168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:49.931470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154561+0000","last_change":"2024-04-25T09:03:17.162520+0000","last_active":"2024-04-25T09:03:18.154561+0000","last_peered":"2024-04-25T09:03:18.154561+0000","last_clean":"2024-04-25T09:03:18.154561+0000","last_became_active":"2024-04-25T09:03:17.160462+0000","last_became_peered":"2024-04-25T09:03:17.160462+0000","last_unstale":"2024-04-25T09:03:18.154561+0000","last_undegraded":"2024-04-25T09:03:18.154561+0000","last_fullsized":"2024-04-25T09:03:18.154561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.326298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562165+0000","last_change":"2024-04-25T09:03:16.167181+0000","last_active":"2024-04-25T09:03:18.562165+0000","last_peered":"2024-04-25T09:03:18.562165+0000","last_clean":"2024-04-25T09:03:18.562165+0000","last_became_active":"2024-04-25T09:03:16.166842+0000","last_became_peered":"2024-04-25T09:03:16.166842+0000","last_unstale":"2024-04-25T09:03:18.562165+0000","last_undegraded":"2024-04-25T09:03:18.562165+0000","last_fullsized":"2024-04-25T09:03:18.562165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:54.518583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563045+0000","last_change":"2024-04-25T09:03:17.158524+0000","last_active":"2024-04-25T09:03:18.563045+0000","last_peered":"2024-04-25T09:03:18.563045+0000","last_clean":"2024-04-25T09:03:18.563045+0000","last_became_active":"2024-04-25T09:03:17.157997+0000","last_became_peered":"2024-04-25T09:03:17.157997+0000","last_unstale":"2024-04-25T09:03:18.563045+0000","last_undegraded":"2024-04-25T09:03:18.563045+0000","last_fullsized":"2024-04-25T09:03:18.563045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566796+0000","last_change":"2024-04-25T09:03:16.169797+0000","last_active":"2024-04-25T09:03:18.566796+0000","last_peered":"2024-04-25T09:03:18.566796+0000","last_clean":"2024-04-25T09:03:18.566796+0000","last_became_active":"2024-04-25T09:03:16.169557+0000","last_became_peered":"2024-04-25T09:03:16.169557+0000","last_unstale":"2024-04-25T09:03:18.566796+0000","last_undegraded":"2024-04-25T09:03:18.566796+0000","last_fullsized":"2024-04-25T09:03:18.566796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:10.210895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563921+0000","last_change":"2024-04-25T09:03:17.157959+0000","last_active":"2024-04-25T09:03:18.563921+0000","last_peered":"2024-04-25T09:03:18.563921+0000","last_clean":"2024-04-25T09:03:18.563921+0000","last_became_active":"2024-04-25T09:03:17.157664+0000","last_became_peered":"2024-04-25T09:03:17.157664+0000","last_unstale":"2024-04-25T09:03:18.563921+0000","last_undegraded":"2024-04-25T09:03:18.563921+0000","last_fullsized":"2024-04-25T09:03:18.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.631418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563894+0000","last_change":"2024-04-25T09:03:16.157473+0000","last_active":"2024-04-25T09:03:18.563894+0000","last_peered":"2024-04-25T09:03:18.563894+0000","last_clean":"2024-04-25T09:03:18.563894+0000","last_became_active":"2024-04-25T09:03:16.157153+0000","last_became_peered":"2024-04-25T09:03:16.157153+0000","last_unstale":"2024-04-25T09:03:18.563894+0000","last_undegraded":"2024-04-25T09:03:18.563894+0000","last_fullsized":"2024-04-25T09:03:18.563894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:41.809030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172033+0000","last_change":"2024-04-25T09:03:17.164826+0000","last_active":"2024-04-25T09:03:18.172033+0000","last_peered":"2024-04-25T09:03:18.172033+0000","last_clean":"2024-04-25T09:03:18.172033+0000","last_became_active":"2024-04-25T09:03:17.161434+0000","last_became_peered":"2024-04-25T09:03:17.161434+0000","last_unstale":"2024-04-25T09:03:18.172033+0000","last_undegraded":"2024-04-25T09:03:18.172033+0000","last_fullsized":"2024-04-25T09:03:18.172033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:39.624252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565331+0000","last_change":"2024-04-25T09:03:16.171358+0000","last_active":"2024-04-25T09:03:18.565331+0000","last_peered":"2024-04-25T09:03:18.565331+0000","last_clean":"2024-04-25T09:03:18.565331+0000","last_became_active":"2024-04-25T09:03:16.171010+0000","last_became_peered":"2024-04-25T09:03:16.171010+0000","last_unstale":"2024-04-25T09:03:18.565331+0000","last_undegraded":"2024-04-25T09:03:18.565331+0000","last_fullsized":"2024-04-25T09:03:18.565331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:41.025423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562033+0000","last_change":"2024-04-25T09:03:17.161787+0000","last_active":"2024-04-25T09:03:18.562033+0000","last_peered":"2024-04-25T09:03:18.562033+0000","last_clean":"2024-04-25T09:03:18.562033+0000","last_became_active":"2024-04-25T09:03:17.157886+0000","last_became_peered":"2024-04-25T09:03:17.157886+0000","last_unstale":"2024-04-25T09:03:18.562033+0000","last_undegraded":"2024-04-25T09:03:18.562033+0000","last_fullsized":"2024-04-25T09:03:18.562033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:38.838938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564686+0000","last_change":"2024-04-25T09:03:16.157712+0000","last_active":"2024-04-25T09:03:18.564686+0000","last_peered":"2024-04-25T09:03:18.564686+0000","last_clean":"2024-04-25T09:03:18.564686+0000","last_became_active":"2024-04-25T09:03:16.157285+0000","last_became_peered":"2024-04-25T09:03:16.157285+0000","last_unstale":"2024-04-25T09:03:18.564686+0000","last_undegraded":"2024-04-25T09:03:18.564686+0000","last_fullsized":"2024-04-25T09:03:18.564686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:17.498426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563865+0000","last_change":"2024-04-25T09:03:17.169015+0000","last_active":"2024-04-25T09:03:18.563865+0000","last_peered":"2024-04-25T09:03:18.563865+0000","last_clean":"2024-04-25T09:03:18.563865+0000","last_became_active":"2024-04-25T09:03:17.168430+0000","last_became_peered":"2024-04-25T09:03:17.168430+0000","last_unstale":"2024-04-25T09:03:18.563865+0000","last_undegraded":"2024-04-25T09:03:18.563865+0000","last_fullsized":"2024-04-25T09:03:18.563865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:57.330616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154237+0000","last_change":"2024-04-25T09:03:16.167928+0000","last_active":"2024-04-25T09:03:18.154237+0000","last_peered":"2024-04-25T09:03:18.154237+0000","last_clean":"2024-04-25T09:03:18.154237+0000","last_became_active":"2024-04-25T09:03:16.167566+0000","last_became_peered":"2024-04-25T09:03:16.167566+0000","last_unstale":"2024-04-25T09:03:18.154237+0000","last_undegraded":"2024-04-25T09:03:18.154237+0000","last_fullsized":"2024-04-25T09:03:18.154237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:41.025423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154329+0000","last_change":"2024-04-25T09:03:17.157268+0000","last_active":"2024-04-25T09:03:18.154329+0000","last_peered":"2024-04-25T09:03:18.154329+0000","last_clean":"2024-04-25T09:03:18.154329+0000","last_became_active":"2024-04-25T09:03:17.156997+0000","last_became_peered":"2024-04-25T09:03:17.156997+0000","last_unstale":"2024-04-25T09:03:18.154329+0000","last_undegraded":"2024-04-25T09:03:18.154329+0000","last_fullsized":"2024-04-25T09:03:18.154329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.631418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565139+0000","last_change":"2024-04-25T09:03:16.169093+0000","last_active":"2024-04-25T09:03:18.565139+0000","last_peered":"2024-04-25T09:03:18.565139+0000","last_clean":"2024-04-25T09:03:18.565139+0000","last_became_active":"2024-04-25T09:03:16.168826+0000","last_became_peered":"2024-04-25T09:03:16.168826+0000","last_unstale":"2024-04-25T09:03:18.565139+0000","last_undegraded":"2024-04-25T09:03:18.565139+0000","last_fullsized":"2024-04-25T09:03:18.565139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.217267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154092+0000","last_change":"2024-04-25T09:03:17.136952+0000","last_active":"2024-04-25T09:03:18.154092+0000","last_peered":"2024-04-25T09:03:18.154092+0000","last_clean":"2024-04-25T09:03:18.154092+0000","last_became_active":"2024-04-25T09:03:17.136246+0000","last_became_peered":"2024-04-25T09:03:17.136246+0000","last_unstale":"2024-04-25T09:03:18.154092+0000","last_undegraded":"2024-04-25T09:03:18.154092+0000","last_fullsized":"2024-04-25T09:03:18.154092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:57.330616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171136+0000","last_change":"2024-04-25T09:03:16.156412+0000","last_active":"2024-04-25T09:03:18.171136+0000","last_peered":"2024-04-25T09:03:18.171136+0000","last_clean":"2024-04-25T09:03:18.171136+0000","last_became_active":"2024-04-25T09:03:16.151576+0000","last_became_peered":"2024-04-25T09:03:16.151576+0000","last_unstale":"2024-04-25T09:03:18.171136+0000","last_undegraded":"2024-04-25T09:03:18.171136+0000","last_fullsized":"2024-04-25T09:03:18.171136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.002913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562924+0000","last_change":"2024-04-25T09:03:17.154125+0000","last_active":"2024-04-25T09:03:18.562924+0000","last_peered":"2024-04-25T09:03:18.562924+0000","last_clean":"2024-04-25T09:03:18.562924+0000","last_became_active":"2024-04-25T09:03:17.153852+0000","last_became_peered":"2024-04-25T09:03:17.153852+0000","last_unstale":"2024-04-25T09:03:18.562924+0000","last_undegraded":"2024-04-25T09:03:18.562924+0000","last_fullsized":"2024-04-25T09:03:18.562924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:57.554441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153647+0000","last_change":"2024-04-25T09:03:16.167837+0000","last_active":"2024-04-25T09:03:18.153647+0000","last_peered":"2024-04-25T09:03:18.153647+0000","last_clean":"2024-04-25T09:03:18.153647+0000","last_became_active":"2024-04-25T09:03:16.167365+0000","last_became_peered":"2024-04-25T09:03:16.167365+0000","last_unstale":"2024-04-25T09:03:18.153647+0000","last_undegraded":"2024-04-25T09:03:18.153647+0000","last_fullsized":"2024-04-25T09:03:18.153647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:57.583736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153552+0000","last_change":"2024-04-25T09:03:17.137828+0000","last_active":"2024-04-25T09:03:18.153552+0000","last_peered":"2024-04-25T09:03:18.153552+0000","last_clean":"2024-04-25T09:03:18.153552+0000","last_became_active":"2024-04-25T09:03:17.137277+0000","last_became_peered":"2024-04-25T09:03:17.137277+0000","last_unstale":"2024-04-25T09:03:18.153552+0000","last_undegraded":"2024-04-25T09:03:18.153552+0000","last_fullsized":"2024-04-25T09:03:18.153552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564048+0000","last_change":"2024-04-25T09:03:16.165762+0000","last_active":"2024-04-25T09:03:18.564048+0000","last_peered":"2024-04-25T09:03:18.564048+0000","last_clean":"2024-04-25T09:03:18.564048+0000","last_became_active":"2024-04-25T09:03:16.165453+0000","last_became_peered":"2024-04-25T09:03:16.165453+0000","last_unstale":"2024-04-25T09:03:18.564048+0000","last_undegraded":"2024-04-25T09:03:18.564048+0000","last_fullsized":"2024-04-25T09:03:18.564048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567253+0000","last_change":"2024-04-25T09:03:17.152812+0000","last_active":"2024-04-25T09:03:18.567253+0000","last_peered":"2024-04-25T09:03:18.567253+0000","last_clean":"2024-04-25T09:03:18.567253+0000","last_became_active":"2024-04-25T09:03:17.152461+0000","last_became_peered":"2024-04-25T09:03:17.152461+0000","last_unstale":"2024-04-25T09:03:18.567253+0000","last_undegraded":"2024-04-25T09:03:18.567253+0000","last_fullsized":"2024-04-25T09:03:18.567253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:16.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563836+0000","last_change":"2024-04-25T09:03:16.156321+0000","last_active":"2024-04-25T09:03:18.563836+0000","last_peered":"2024-04-25T09:03:18.563836+0000","last_clean":"2024-04-25T09:03:18.563836+0000","last_became_active":"2024-04-25T09:03:16.155953+0000","last_became_peered":"2024-04-25T09:03:16.155953+0000","last_unstale":"2024-04-25T09:03:18.563836+0000","last_undegraded":"2024-04-25T09:03:18.563836+0000","last_fullsized":"2024-04-25T09:03:18.563836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:15.408869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563595+0000","last_change":"2024-04-25T09:03:17.158747+0000","last_active":"2024-04-25T09:03:18.563595+0000","last_peered":"2024-04-25T09:03:18.563595+0000","last_clean":"2024-04-25T09:03:18.563595+0000","last_became_active":"2024-04-25T09:03:17.158086+0000","last_became_peered":"2024-04-25T09:03:17.158086+0000","last_unstale":"2024-04-25T09:03:18.563595+0000","last_undegraded":"2024-04-25T09:03:18.563595+0000","last_fullsized":"2024-04-25T09:03:18.563595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:37.022619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564291+0000","last_change":"2024-04-25T09:03:17.167869+0000","last_active":"2024-04-25T09:03:18.564291+0000","last_peered":"2024-04-25T09:03:18.564291+0000","last_clean":"2024-04-25T09:03:18.564291+0000","last_became_active":"2024-04-25T09:03:17.164836+0000","last_became_peered":"2024-04-25T09:03:17.164836+0000","last_unstale":"2024-04-25T09:03:18.564291+0000","last_undegraded":"2024-04-25T09:03:18.564291+0000","last_fullsized":"2024-04-25T09:03:18.564291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:19.500716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172988+0000","last_change":"2024-04-25T09:03:16.162264+0000","last_active":"2024-04-25T09:03:18.172988+0000","last_peered":"2024-04-25T09:03:18.172988+0000","last_clean":"2024-04-25T09:03:18.172988+0000","last_became_active":"2024-04-25T09:03:16.161892+0000","last_became_peered":"2024-04-25T09:03:16.161892+0000","last_unstale":"2024-04-25T09:03:18.172988+0000","last_undegraded":"2024-04-25T09:03:18.172988+0000","last_fullsized":"2024-04-25T09:03:18.172988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:17.498426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151136+0000","last_change":"2024-04-25T09:03:17.156036+0000","last_active":"2024-04-25T09:03:18.151136+0000","last_peered":"2024-04-25T09:03:18.151136+0000","last_clean":"2024-04-25T09:03:18.151136+0000","last_became_active":"2024-04-25T09:03:17.154785+0000","last_became_peered":"2024-04-25T09:03:17.154785+0000","last_unstale":"2024-04-25T09:03:18.151136+0000","last_undegraded":"2024-04-25T09:03:18.151136+0000","last_fullsized":"2024-04-25T09:03:18.151136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172370+0000","last_change":"2024-04-25T09:03:16.161940+0000","last_active":"2024-04-25T09:03:18.172370+0000","last_peered":"2024-04-25T09:03:18.172370+0000","last_clean":"2024-04-25T09:03:18.172370+0000","last_became_active":"2024-04-25T09:03:16.161542+0000","last_became_peered":"2024-04-25T09:03:16.161542+0000","last_unstale":"2024-04-25T09:03:18.172370+0000","last_undegraded":"2024-04-25T09:03:18.172370+0000","last_fullsized":"2024-04-25T09:03:18.172370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:57.583736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566119+0000","last_change":"2024-04-25T09:03:17.157276+0000","last_active":"2024-04-25T09:03:18.566119+0000","last_peered":"2024-04-25T09:03:18.566119+0000","last_clean":"2024-04-25T09:03:18.566119+0000","last_became_active":"2024-04-25T09:03:17.156910+0000","last_became_peered":"2024-04-25T09:03:17.156910+0000","last_unstale":"2024-04-25T09:03:18.566119+0000","last_undegraded":"2024-04-25T09:03:18.566119+0000","last_fullsized":"2024-04-25T09:03:18.566119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:13.218030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566191+0000","last_change":"2024-04-25T09:03:16.167873+0000","last_active":"2024-04-25T09:03:18.566191+0000","last_peered":"2024-04-25T09:03:18.566191+0000","last_clean":"2024-04-25T09:03:18.566191+0000","last_became_active":"2024-04-25T09:03:16.166612+0000","last_became_peered":"2024-04-25T09:03:16.166612+0000","last_unstale":"2024-04-25T09:03:18.566191+0000","last_undegraded":"2024-04-25T09:03:18.566191+0000","last_fullsized":"2024-04-25T09:03:18.566191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566016+0000","last_change":"2024-04-25T09:03:17.156968+0000","last_active":"2024-04-25T09:03:18.566016+0000","last_peered":"2024-04-25T09:03:18.566016+0000","last_clean":"2024-04-25T09:03:18.566016+0000","last_became_active":"2024-04-25T09:03:17.156663+0000","last_became_peered":"2024-04-25T09:03:17.156663+0000","last_unstale":"2024-04-25T09:03:18.566016+0000","last_undegraded":"2024-04-25T09:03:18.566016+0000","last_fullsized":"2024-04-25T09:03:18.566016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.046413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154405+0000","last_change":"2024-04-25T09:03:16.157953+0000","last_active":"2024-04-25T09:03:18.154405+0000","last_peered":"2024-04-25T09:03:18.154405+0000","last_clean":"2024-04-25T09:03:18.154405+0000","last_became_active":"2024-04-25T09:03:16.157603+0000","last_became_peered":"2024-04-25T09:03:16.157603+0000","last_unstale":"2024-04-25T09:03:18.154405+0000","last_undegraded":"2024-04-25T09:03:18.154405+0000","last_fullsized":"2024-04-25T09:03:18.154405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.002913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":661394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":73218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:22.418 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:03:22.419 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T09:03:22.419 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T09:03:22.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T09:03:22.419 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T09:03:22.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.681+0000 7f45bbe00640 1 Processor -- start 2024-04-25T09:03:22.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.682+0000 7f45bbe00640 1 -- start start 2024-04-25T09:03:22.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.683+0000 7f45bbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f45bc06d900 0x7f45bc15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.683+0000 7f45bbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.683+0000 7f45bbe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc0cf2b0 con 0x7f45bc144c10 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.683+0000 7f45bbe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc144ff0 con 0x7f45bc06d900 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45bbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc160340 con 0x7f45bc0cee40 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45b2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f45bc06d900 0x7f45bc15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45b2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f45bc06d900 0x7f45bc15f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59444/0 (socket says 172.21.15.62:59444) 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45b2000640 1 -- 172.21.15.62:0/3205177725 learned_addr learned my addr 172.21.15.62:0/3205177725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45bb400640 1 --2- 172.21.15.62:0/3205177725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45b2000640 1 -- 172.21.15.62:0/3205177725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 msgr2=0x7f45bc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45b2000640 1 --2- 172.21.15.62:0/3205177725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45b2000640 1 -- 172.21.15.62:0/3205177725 >> v1:172.21.15.121:6789/0 conn(0x7f45bc144c10 legacy=0x7f45bc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45b2000640 1 -- 172.21.15.62:0/3205177725 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc15faa0 con 0x7f45bc06d900 2024-04-25T09:03:22.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.684+0000 7f45bb400640 1 --2- 172.21.15.62:0/3205177725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.686+0000 7f45b2000640 1 --2- 172.21.15.62:0/3205177725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc15f450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009870 tx=0x7f45a402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbadc982ee5dbab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.686+0000 7f45ba000640 1 -- 172.21.15.62:0/3205177725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4037070 con 0x7f45bc06d900 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.686+0000 7f45ba000640 1 -- 172.21.15.62:0/3205177725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a4032440 con 0x7f45bc06d900 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.686+0000 7f45ba000640 1 -- 172.21.15.62:0/3205177725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4032810 con 0x7f45bc06d900 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.687+0000 7f45bbe00640 1 -- 172.21.15.62:0/3205177725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 msgr2=0x7f45bc15f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.687+0000 7f45bbe00640 1 --2- 172.21.15.62:0/3205177725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc15f450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009870 tx=0x7f45a402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.687+0000 7f45bbe00640 1 -- 172.21.15.62:0/3205177725 shutdown_connections 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.687+0000 7f45bbe00640 1 --2- 172.21.15.62:0/3205177725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.687+0000 7f45bbe00640 1 --2- 172.21.15.62:0/3205177725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc15f450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.687+0000 7f45bbe00640 1 -- 172.21.15.62:0/3205177725 >> 172.21.15.62:0/3205177725 conn(0x7f45bc0b14d0 msgr2=0x7f45bc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:22.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.687+0000 7f45bbe00640 1 -- 172.21.15.62:0/3205177725 shutdown_connections 2024-04-25T09:03:22.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.687+0000 7f45bbe00640 1 -- 172.21.15.62:0/3205177725 wait complete. 2024-04-25T09:03:22.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.689+0000 7f45bbe00640 1 Processor -- start 2024-04-25T09:03:22.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.689+0000 7f45bbe00640 1 -- start start 2024-04-25T09:03:22.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45bbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45bbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45bc144c10 0x7f45bc13eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45bbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc160060 con 0x7f45bc06d900 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45bbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0b41c0 con 0x7f45bc144c10 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45bbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0b4b90 con 0x7f45bc0cee40 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc140210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59446/0 (socket says 172.21.15.62:59446) 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.690+0000 7f45baa00640 1 -- 172.21.15.62:0/1124647366 learned_addr learned my addr 172.21.15.62:0/1124647366 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45bb400640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45bc144c10 0x7f45bc13eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45b2000640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45baa00640 1 -- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 msgr2=0x7f45bc0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45baa00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45baa00640 1 -- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45bc144c10 msgr2=0x7f45bc13eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45baa00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45bc144c10 0x7f45bc13eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45baa00640 1 -- 172.21.15.62:0/1124647366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0cdcb0 con 0x7f45bc06d900 2024-04-25T09:03:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45baa00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc140210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00c400 tx=0x7f45ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45b3400640 1 -- 172.21.15.62:0/1124647366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac017070 con 0x7f45bc06d900 2024-04-25T09:03:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45b3400640 1 -- 172.21.15.62:0/1124647366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac0052d0 con 0x7f45bc06d900 2024-04-25T09:03:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc0cdf90 con 0x7f45bc06d900 2024-04-25T09:03:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.691+0000 7f45b3400640 1 -- 172.21.15.62:0/1124647366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac003830 con 0x7f45bc06d900 2024-04-25T09:03:22.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.692+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc141190 con 0x7f45bc06d900 2024-04-25T09:03:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.693+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc04ef20 con 0x7f45bc06d900 2024-04-25T09:03:22.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.694+0000 7f45b3400640 1 -- 172.21.15.62:0/1124647366 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45ac01a070 con 0x7f45bc06d900 2024-04-25T09:03:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.695+0000 7f45b3400640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f459806b4d0 0x7f459806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.695+0000 7f45b3400640 1 -- 172.21.15.62:0/1124647366 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45ac089560 con 0x7f45bc06d900 2024-04-25T09:03:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.695+0000 7f45b2000640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f459806b4d0 0x7f459806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.696+0000 7f45b2000640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f459806b4d0 0x7f459806d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45a4034000 tx=0x7f45a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.700+0000 7f45b3400640 1 -- 172.21.15.62:0/1124647366 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45ac0583e0 con 0x7f45bc06d900 2024-04-25T09:03:22.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.949+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f45bc0ce160 con 0x7f45bc06d900 2024-04-25T09:03:22.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.953+0000 7f45b3400640 1 -- 172.21.15.62:0/1124647366 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171787 (secure 0 0 0) 0x7f45ac057ce0 con 0x7f45bc06d900 2024-04-25T09:03:22.944 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f459806b4d0 msgr2=0x7f459806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f459806b4d0 0x7f459806d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45a4034000 tx=0x7f45a402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 msgr2=0x7f45bc140210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc140210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00c400 tx=0x7f45ac00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 shutdown_connections 2024-04-25T09:03:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f459806b4d0 0x7f459806d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45bc144c10 0x7f45bc13eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45bc0cee40 0x7f45bc0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 --2- 172.21.15.62:0/1124647366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc06d900 0x7f45bc140210 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.958+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 >> 172.21.15.62:0/1124647366 conn(0x7f45bc0b14d0 msgr2=0x7f45bc071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.959+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 shutdown_connections 2024-04-25T09:03:22.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:22.959+0000 7f45bbe00640 1 -- 172.21.15.62:0/1124647366 wait complete. 2024-04-25T09:03:22.965 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":568331749},{"type":"v1","addr":"172.21.15.121:6833","nonce":568331749}]},"active_addr":"172.21.15.121:6833/568331749","active_change":"2024-04-25T09:02:57.852561+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":4294159459}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":558414184}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":3347248262}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":357548860}]}]} 2024-04-25T09:03:22.969 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T09:03:22.969 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T09:03:22.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:23.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.190+0000 7f64b0a00640 1 Processor -- start 2024-04-25T09:03:23.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.190+0000 7f64b0a00640 1 -- start start 2024-04-25T09:03:23.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.191+0000 7f64b0a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64ac0b6a90 0x7f64ac0d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.191+0000 7f64b0a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f64ac0b27a0 0x7f64ac0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.191+0000 7f64b0a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac0d3110 con 0x7f64ac0b56f0 2024-04-25T09:03:23.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.191+0000 7f64b0a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0cedc0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.191+0000 7f64b0a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0d33f0 con 0x7f64ac0b27a0 2024-04-25T09:03:23.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.192+0000 7f64aaa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64ac0b6a90 0x7f64ac0d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.192+0000 7f64aaa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64ac0b6a90 0x7f64ac0d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59456/0 (socket says 172.21.15.62:59456) 2024-04-25T09:03:23.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.192+0000 7f64aaa00640 1 -- 172.21.15.62:0/3796560738 learned_addr learned my addr 172.21.15.62:0/3796560738 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:23.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.192+0000 7f64abe00640 1 --2- 172.21.15.62:0/3796560738 >> v2:172.21.15.121:3300/0 conn(0x7f64ac0b27a0 0x7f64ac0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.192+0000 7f64aaa00640 1 -- 172.21.15.62:0/3796560738 >> v2:172.21.15.121:3300/0 conn(0x7f64ac0b27a0 msgr2=0x7f64ac0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:23.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.192+0000 7f64aaa00640 1 --2- 172.21.15.62:0/3796560738 >> v2:172.21.15.121:3300/0 conn(0x7f64ac0b27a0 0x7f64ac0d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.192+0000 7f64aaa00640 1 -- 172.21.15.62:0/3796560738 >> v1:172.21.15.121:6789/0 conn(0x7f64ac0b56f0 legacy=0x7f64ac160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:23.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.192+0000 7f64aaa00640 1 -- 172.21.15.62:0/3796560738 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0d36d0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.193+0000 7f64aaa00640 1 --2- 172.21.15.62:0/3796560738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac0d1fe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0b2720 tx=0x7f649802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4918aef4c6119f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64aa000640 1 -- 172.21.15.62:0/3796560738 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498037070 con 0x7f64ac0b6a90 2024-04-25T09:03:23.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64aa000640 1 -- 172.21.15.62:0/3796560738 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6498032440 con 0x7f64ac0b6a90 2024-04-25T09:03:23.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64aa000640 1 -- 172.21.15.62:0/3796560738 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498032810 con 0x7f64ac0b6a90 2024-04-25T09:03:23.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64b0a00640 1 -- 172.21.15.62:0/3796560738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 msgr2=0x7f64ac0d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3796560738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac0d1fe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0b2720 tx=0x7f649802a230 comp rx=0 tx=0).stop 2024-04-25T09:03:23.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64b0a00640 1 -- 172.21.15.62:0/3796560738 shutdown_connections 2024-04-25T09:03:23.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3796560738 >> v2:172.21.15.121:3300/0 conn(0x7f64ac0b27a0 0x7f64ac0d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3796560738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac0d1fe0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64b0a00640 1 -- 172.21.15.62:0/3796560738 >> 172.21.15.62:0/3796560738 conn(0x7f64ac066d10 msgr2=0x7f64ac067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:23.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.194+0000 7f64b0a00640 1 -- 172.21.15.62:0/3796560738 shutdown_connections 2024-04-25T09:03:23.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.195+0000 7f64b0a00640 1 -- 172.21.15.62:0/3796560738 wait complete. 2024-04-25T09:03:23.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.197+0000 7f64b0a00640 1 Processor -- start 2024-04-25T09:03:23.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.197+0000 7f64b0a00640 1 -- start start 2024-04-25T09:03:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.198+0000 7f64b0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64ac0b27a0 0x7f64ac14eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.198+0000 7f64b0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64ac0b56f0 0x7f64ac159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64b0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64b0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0b2ee0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64b0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0b2d60 con 0x7f64ac0b56f0 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64b0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac0b5ad0 con 0x7f64ac0b27a0 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59472/0 (socket says 172.21.15.62:59472) 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64abe00640 1 -- 172.21.15.62:0/3488931969 learned_addr learned my addr 172.21.15.62:0/3488931969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64aaa00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64ac0b56f0 0x7f64ac159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64ab400640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64ac0b27a0 0x7f64ac14eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64abe00640 1 -- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64ac0b27a0 msgr2=0x7f64ac14eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64abe00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64ac0b27a0 0x7f64ac14eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64abe00640 1 -- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64ac0b56f0 msgr2=0x7f64ac159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64abe00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64ac0b56f0 0x7f64ac159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.199+0000 7f64abe00640 1 -- 172.21.15.62:0/3488931969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac158ce0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.200+0000 7f64abe00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac159b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d600 tx=0x7f64a400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.200+0000 7f64a3400640 1 -- 172.21.15.62:0/3488931969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a40125f0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.200+0000 7f64a3400640 1 -- 172.21.15.62:0/3488931969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a4019070 con 0x7f64ac0b6a90 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.200+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac158fc0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.200+0000 7f64a3400640 1 -- 172.21.15.62:0/3488931969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a40129d0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.200+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac0c7530 con 0x7f64ac0b6a90 2024-04-25T09:03:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.202+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac04ef20 con 0x7f64ac0b6a90 2024-04-25T09:03:23.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.203+0000 7f64a3400640 1 -- 172.21.15.62:0/3488931969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64a4012b70 con 0x7f64ac0b6a90 2024-04-25T09:03:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.204+0000 7f64a3400640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f648c06b4d0 0x7f648c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.204+0000 7f64a3400640 1 -- 172.21.15.62:0/3488931969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64a408a700 con 0x7f64ac0b6a90 2024-04-25T09:03:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.205+0000 7f64ab400640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f648c06b4d0 0x7f648c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.206+0000 7f64ab400640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f648c06b4d0 0x7f648c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f649c005520 tx=0x7f649c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.209+0000 7f64a3400640 1 -- 172.21.15.62:0/3488931969 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64a40594e0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.413+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64ac0b6180 con 0x7f64ac0b6a90 2024-04-25T09:03:23.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.416+0000 7f64a3400640 1 -- 172.21.15.62:0/3488931969 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f64a4058de0 con 0x7f64ac0b6a90 2024-04-25T09:03:23.404 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:23.404 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:23.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.418+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f648c06b4d0 msgr2=0x7f648c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.418+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f648c06b4d0 0x7f648c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f649c005520 tx=0x7f649c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:23.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 msgr2=0x7f64ac159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac159b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d600 tx=0x7f64a400dac0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 shutdown_connections 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f648c06b4d0 0x7f648c06d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64ac0b6a90 0x7f64ac159b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64ac0b56f0 0x7f64ac159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 --2- 172.21.15.62:0/3488931969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64ac0b27a0 0x7f64ac14eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 >> 172.21.15.62:0/3488931969 conn(0x7f64ac066d10 msgr2=0x7f64ac0b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 shutdown_connections 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.419+0000 7f64b0a00640 1 -- 172.21.15.62:0/3488931969 wait complete. 2024-04-25T09:03:23.427 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T09:03:23.427 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:23.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.652+0000 7f3edbe00640 1 Processor -- start 2024-04-25T09:03:23.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.652+0000 7f3edbe00640 1 -- start start 2024-04-25T09:03:23.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edbe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3edc13d970 0x7f3edc0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc0b1050 0x7f3edc06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3edc15acf0 0x7f3edc06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edbe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc15b220 con 0x7f3edc15acf0 2024-04-25T09:03:23.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edbe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc15b500 con 0x7f3edc13d970 2024-04-25T09:03:23.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc15b7e0 con 0x7f3edc0b1050 2024-04-25T09:03:23.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edb400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3edc15acf0 0x7f3edc06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edb400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3edc15acf0 0x7f3edc06d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59474/0 (socket says 172.21.15.62:59474) 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.654+0000 7f3edb400640 1 -- 172.21.15.62:0/4048223655 learned_addr learned my addr 172.21.15.62:0/4048223655 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.655+0000 7f3ed2000640 1 --2- 172.21.15.62:0/4048223655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc0b1050 0x7f3edc06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.655+0000 7f3edaa00640 1 --2- 172.21.15.62:0/4048223655 >> v2:172.21.15.121:3300/0 conn(0x7f3edc13d970 0x7f3edc0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.655+0000 7f3edb400640 1 -- 172.21.15.62:0/4048223655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc0b1050 msgr2=0x7f3edc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.655+0000 7f3edb400640 1 --2- 172.21.15.62:0/4048223655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc0b1050 0x7f3edc06d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.655+0000 7f3edb400640 1 -- 172.21.15.62:0/4048223655 >> v2:172.21.15.121:3300/0 conn(0x7f3edc13d970 msgr2=0x7f3edc0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.655+0000 7f3edb400640 1 --2- 172.21.15.62:0/4048223655 >> v2:172.21.15.121:3300/0 conn(0x7f3edc13d970 0x7f3edc0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.655+0000 7f3edb400640 1 -- 172.21.15.62:0/4048223655 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc15bac0 con 0x7f3edc15acf0 2024-04-25T09:03:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.655+0000 7f3edaa00640 1 --2- 172.21.15.62:0/4048223655 >> v2:172.21.15.121:3300/0 conn(0x7f3edc13d970 0x7f3edc0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:23.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.656+0000 7f3edb400640 1 --2- 172.21.15.62:0/4048223655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc15acf0 0x7f3edc06d730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3edc15b710 tx=0x7f3ed402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5588431377c60f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.656+0000 7f3eda000640 1 -- 172.21.15.62:0/4048223655 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4037070 con 0x7f3edc15acf0 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.656+0000 7f3eda000640 1 -- 172.21.15.62:0/4048223655 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed4032440 con 0x7f3edc15acf0 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.656+0000 7f3eda000640 1 -- 172.21.15.62:0/4048223655 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4032810 con 0x7f3edc15acf0 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 -- 172.21.15.62:0/4048223655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc15acf0 msgr2=0x7f3edc06d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 --2- 172.21.15.62:0/4048223655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc15acf0 0x7f3edc06d730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3edc15b710 tx=0x7f3ed402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 -- 172.21.15.62:0/4048223655 shutdown_connections 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 --2- 172.21.15.62:0/4048223655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc15acf0 0x7f3edc06d730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 --2- 172.21.15.62:0/4048223655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc0b1050 0x7f3edc06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 --2- 172.21.15.62:0/4048223655 >> v2:172.21.15.121:3300/0 conn(0x7f3edc13d970 0x7f3edc0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 -- 172.21.15.62:0/4048223655 >> 172.21.15.62:0/4048223655 conn(0x7f3edc0b14d0 msgr2=0x7f3edc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 -- 172.21.15.62:0/4048223655 shutdown_connections 2024-04-25T09:03:23.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.657+0000 7f3edbe00640 1 -- 172.21.15.62:0/4048223655 wait complete. 2024-04-25T09:03:23.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.659+0000 7f3edbe00640 1 Processor -- start 2024-04-25T09:03:23.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.659+0000 7f3edbe00640 1 -- start start 2024-04-25T09:03:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc0b1050 0x7f3edc138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3edc13d970 0x7f3edc154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc15acf0 0x7f3edc13c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc140290 con 0x7f3edc0b1050 2024-04-25T09:03:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc140110 con 0x7f3edc13d970 2024-04-25T09:03:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc140410 con 0x7f3edc15acf0 2024-04-25T09:03:23.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc0b1050 0x7f3edc138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc0b1050 0x7f3edc138ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59480/0 (socket says 172.21.15.62:59480) 2024-04-25T09:03:23.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.660+0000 7f3edaa00640 1 -- 172.21.15.62:0/185899430 learned_addr learned my addr 172.21.15.62:0/185899430 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edb400640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc15acf0 0x7f3edc13c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edaa00640 1 -- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc15acf0 msgr2=0x7f3edc13c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3ed2000640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3edc13d970 0x7f3edc154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edaa00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc15acf0 0x7f3edc13c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edaa00640 1 -- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3edc13d970 msgr2=0x7f3edc154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edaa00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3edc13d970 0x7f3edc154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edaa00640 1 -- 172.21.15.62:0/185899430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc137a10 con 0x7f3edc0b1050 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edaa00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc0b1050 0x7f3edc138ab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00cf60 tx=0x7f3ecc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3ed3e00640 1 -- 172.21.15.62:0/185899430 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc01a070 con 0x7f3edc0b1050 2024-04-25T09:03:23.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3ed3e00640 1 -- 172.21.15.62:0/185899430 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc00d840 con 0x7f3edc0b1050 2024-04-25T09:03:23.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3ed3e00640 1 -- 172.21.15.62:0/185899430 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00dc10 con 0x7f3edc0b1050 2024-04-25T09:03:23.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc137cf0 con 0x7f3edc0b1050 2024-04-25T09:03:23.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.661+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc1382b0 con 0x7f3edc0b1050 2024-04-25T09:03:23.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.662+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc04ef20 con 0x7f3edc0b1050 2024-04-25T09:03:23.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.663+0000 7f3ed3e00640 1 -- 172.21.15.62:0/185899430 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ecc01d070 con 0x7f3edc0b1050 2024-04-25T09:03:23.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.664+0000 7f3ed3e00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b500 0x7f3eb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.664+0000 7f3ed3e00640 1 -- 172.21.15.62:0/185899430 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ecc08aa00 con 0x7f3edc0b1050 2024-04-25T09:03:23.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.665+0000 7f3ed2000640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b500 0x7f3eb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.666+0000 7f3ed2000640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b500 0x7f3eb806d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4006fd0 tx=0x7f3ec4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.669+0000 7f3ed3e00640 1 -- 172.21.15.62:0/185899430 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ecc059190 con 0x7f3edc0b1050 2024-04-25T09:03:23.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.869+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3edc0cdc10 con 0x7f3edc0b1050 2024-04-25T09:03:23.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.871+0000 7f3ed3e00640 1 -- 172.21.15.62:0/185899430 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f3ecc05d230 con 0x7f3edc0b1050 2024-04-25T09:03:23.860 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:23.860 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b500 msgr2=0x7f3eb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b500 0x7f3eb806d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4006fd0 tx=0x7f3ec4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc0b1050 msgr2=0x7f3edc138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc0b1050 0x7f3edc138ab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00cf60 tx=0x7f3ecc00cf90 comp rx=0 tx=0).stop 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 shutdown_connections 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b500 0x7f3eb806d9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3edc15acf0 0x7f3edc13c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3edc13d970 0x7f3edc154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 --2- 172.21.15.62:0/185899430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3edc0b1050 0x7f3edc138ab0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 >> 172.21.15.62:0/185899430 conn(0x7f3edc0b14d0 msgr2=0x7f3edc0ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 shutdown_connections 2024-04-25T09:03:23.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:23.875+0000 7f3edbe00640 1 -- 172.21.15.62:0/185899430 wait complete. 2024-04-25T09:03:23.883 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T09:03:23.884 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T09:03:23.884 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T09:03:23.885 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T09:03:23.885 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T09:03:23.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T09:03:23.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T09:03:23.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T09:03:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.202+0000 7fb182600640 1 Processor -- start 2024-04-25T09:03:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.202+0000 7fb182600640 1 -- start start 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.203+0000 7f89efe00640 1 Processor -- start 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.203+0000 7f3f17e00640 1 Processor -- start 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.203+0000 7fb408a00640 1 Processor -- start 2024-04-25T09:03:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.203+0000 7f3f17e00640 1 -- start start 2024-04-25T09:03:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.203+0000 7f89efe00640 1 -- start start 2024-04-25T09:03:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.204+0000 7fb182600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb1740e30c0 0x7fb174057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.204+0000 7fb182600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb174054300 0x7fb1740068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.204+0000 7fb182600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1740582d0 con 0x7fb17403e840 2024-04-25T09:03:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.204+0000 7fb182600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb174058570 con 0x7fb1740e30c0 2024-04-25T09:03:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.204+0000 7fb182600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb174058850 con 0x7fb174054300 2024-04-25T09:03:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.204+0000 7fb408a00640 1 -- start start 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7fb17be00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb17403e840 0x7fb174053bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35304/0 (socket says 172.21.15.62:35304) 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7fb17be00640 1 -- 172.21.15.62:0/774830740 learned_addr learned my addr 172.21.15.62:0/774830740 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7fb181200640 1 --2- 172.21.15.62:0/774830740 >> v2:172.21.15.62:3300/0 conn(0x7fb1740e30c0 0x7fb174057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7fb181c00640 1 --2- 172.21.15.62:0/774830740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb174054300 0x7fb1740068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb181200640 1 -- 172.21.15.62:0/774830740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb174054300 msgr2=0x7fb1740068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb181200640 1 --2- 172.21.15.62:0/774830740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb174054300 0x7fb1740068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb408a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb404072020 0x7fb4040ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb408a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb404165e80 0x7fb404070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb408a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404071510 0x7fb4040ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7fa949000640 1 Processor -- start 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7fa949000640 1 -- start start 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fa949000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa94415dc50 0x7fa9440d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f18161430 0x7f3f180d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f3f17e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3f180d1130 0x7f3f180cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f3f17e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f180ce190 con 0x7f3f180b2740 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f3f17e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f180ce430 con 0x7f3f180d1130 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f3f17e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180ce710 con 0x7f3f18161430 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7f3f17400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3f180d1130 0x7f3f180cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7f3f17400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3f180d1130 0x7f3f180cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53464/0 (socket says 172.21.15.62:53464) 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7f3f17400640 1 -- 172.21.15.62:0/3269199937 learned_addr learned my addr 172.21.15.62:0/3269199937 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7f3f16a00640 1 --2- 172.21.15.62:0/3269199937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f18161430 0x7f3f180d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7f3f17400640 1 -- 172.21.15.62:0/3269199937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f18161430 msgr2=0x7f3f180d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7f3f17400640 1 --2- 172.21.15.62:0/3269199937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f18161430 0x7f3f180d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7f3f17400640 1 -- 172.21.15.62:0/3269199937 >> v1:172.21.15.62:6789/0 conn(0x7f3f180b2740 legacy=0x7f3f180d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7f3f17400640 1 -- 172.21.15.62:0/3269199937 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180ce9f0 con 0x7f3f180d1130 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fa949000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94415e030 con 0x7fa9440b9650 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fa949000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9440d2370 con 0x7fa9440cee60 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fa949000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9440d2650 con 0x7fa94415dc50 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f89efe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89e80ead00 0x7f89e804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f89efe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f89e8006ea0 0x7f89e8007310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f89efe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e8053720 con 0x7f89e804ffd0 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f89efe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e8007850 con 0x7f89e80ead00 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.205+0000 7f89efe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e8054330 con 0x7f89e8006ea0 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7f89ef400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f89e8006ea0 0x7f89e8007310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7f89ef400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f89e8006ea0 0x7f89e8007310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53472/0 (socket says 172.21.15.62:53472) 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7f89ef400640 1 -- 172.21.15.62:0/1331490112 learned_addr learned my addr 172.21.15.62:0/1331490112 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.206+0000 7f89eea00640 1 --2- 172.21.15.62:0/1331490112 >> v2:172.21.15.62:3300/0 conn(0x7f89e80ead00 0x7f89e804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7f89eea00640 1 -- 172.21.15.62:0/1331490112 >> v2:172.21.15.121:3300/0 conn(0x7f89e8006ea0 msgr2=0x7f89e8007310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7f89eea00640 1 --2- 172.21.15.62:0/1331490112 >> v2:172.21.15.121:3300/0 conn(0x7f89e8006ea0 0x7f89e8007310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7f89eea00640 1 -- 172.21.15.62:0/1331490112 >> v1:172.21.15.121:6789/0 conn(0x7f89e804ffd0 legacy=0x7f89e8037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7f89eea00640 1 -- 172.21.15.62:0/1331490112 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e8053a00 con 0x7f89e80ead00 2024-04-25T09:03:24.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb181200640 1 -- 172.21.15.62:0/774830740 >> v1:172.21.15.121:6789/0 conn(0x7fb17403e840 legacy=0x7fb174053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb181200640 1 -- 172.21.15.62:0/774830740 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174058b30 con 0x7fb1740e30c0 2024-04-25T09:03:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb408a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4041666d0 con 0x7fb404165e80 2024-04-25T09:03:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb408a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4041669b0 con 0x7fb404072020 2024-04-25T09:03:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.207+0000 7fb408a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb404166c90 con 0x7fb404071510 2024-04-25T09:03:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fa941400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa9440b9650 0x7fa9440ce750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41006/0 (socket says 172.21.15.62:41006) 2024-04-25T09:03:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fa941400640 1 -- 172.21.15.62:0/4258207301 learned_addr learned my addr 172.21.15.62:0/4258207301 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fa941e00640 1 --2- 172.21.15.62:0/4258207301 >> v2:172.21.15.62:3300/0 conn(0x7fa94415dc50 0x7fa9440d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fb402800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404071510 0x7fb4040ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fb402800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404071510 0x7fb4040ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41512/0 (socket says 172.21.15.62:41512) 2024-04-25T09:03:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.208+0000 7fb402800640 1 -- 172.21.15.62:0/1667593054 learned_addr learned my addr 172.21.15.62:0/1667593054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb401e00640 1 --2- 172.21.15.62:0/1667593054 >> v2:172.21.15.121:3300/0 conn(0x7fb404072020 0x7fb4040ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb181200640 1 --2- 172.21.15.62:0/774830740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1740e30c0 0x7fb174057d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0697c0 tx=0x7fb17c0a54c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3678293f3bdbfed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb401400640 1 --2- 172.21.15.62:0/1667593054 >> v2:172.21.15.62:3300/0 conn(0x7fb404165e80 0x7fb404070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.210+0000 7f3f17400640 1 --2- 172.21.15.62:0/3269199937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f180d1130 0x7f3f180cef10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009870 tx=0x7f3f0802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a484327c340ec1c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.210+0000 7fb17b400640 1 -- 172.21.15.62:0/774830740 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0aa7d0 con 0x7fb1740e30c0 2024-04-25T09:03:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb402800640 1 -- 172.21.15.62:0/1667593054 >> v2:172.21.15.121:3300/0 conn(0x7fb404072020 msgr2=0x7fb4040ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb402800640 1 --2- 172.21.15.62:0/1667593054 >> v2:172.21.15.121:3300/0 conn(0x7fb404072020 0x7fb4040ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb402800640 1 -- 172.21.15.62:0/1667593054 >> v2:172.21.15.62:3300/0 conn(0x7fb404165e80 msgr2=0x7fb404070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb402800640 1 --2- 172.21.15.62:0/1667593054 >> v2:172.21.15.62:3300/0 conn(0x7fb404165e80 0x7fb404070fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb402800640 1 -- 172.21.15.62:0/1667593054 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40406d000 con 0x7fb404071510 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.209+0000 7fb401400640 1 --2- 172.21.15.62:0/1667593054 >> v2:172.21.15.62:3300/0 conn(0x7fb404165e80 0x7fb404070fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.210+0000 7fa941e00640 1 -- 172.21.15.62:0/4258207301 >> v1:172.21.15.121:6789/0 conn(0x7fa9440cee60 legacy=0x7fa9440b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.210+0000 7fa941e00640 1 -- 172.21.15.62:0/4258207301 >> v1:172.21.15.62:6789/0 conn(0x7fa9440b9650 legacy=0x7fa9440ce750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.210+0000 7fa941e00640 1 -- 172.21.15.62:0/4258207301 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9440d2930 con 0x7fa94415dc50 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.210+0000 7fb17b400640 1 -- 172.21.15.62:0/774830740 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c0aa970 con 0x7fb1740e30c0 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.210+0000 7fb17b400640 1 -- 172.21.15.62:0/774830740 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c0aade0 con 0x7fb1740e30c0 2024-04-25T09:03:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.210+0000 7f89eea00640 1 --2- 172.21.15.62:0/1331490112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e80ead00 0x7f89e804fa90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89e80528a0 tx=0x7f89e002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58598a45dba52b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7f89ed600640 1 -- 172.21.15.62:0/1331490112 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0037070 con 0x7f89e80ead00 2024-04-25T09:03:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7f89ed600640 1 -- 172.21.15.62:0/1331490112 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e002ae40 con 0x7f89e80ead00 2024-04-25T09:03:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7f89ed600640 1 -- 172.21.15.62:0/1331490112 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0032660 con 0x7f89e80ead00 2024-04-25T09:03:24.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7f3f15600640 1 -- 172.21.15.62:0/3269199937 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08037070 con 0x7f3f180d1130 2024-04-25T09:03:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7fb182600640 1 -- 172.21.15.62:0/774830740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1740e30c0 msgr2=0x7fb174057d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7f3f15600640 1 -- 172.21.15.62:0/3269199937 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08032440 con 0x7f3f180d1130 2024-04-25T09:03:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7f3f15600640 1 -- 172.21.15.62:0/3269199937 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08032850 con 0x7f3f180d1130 2024-04-25T09:03:24.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7fb182600640 1 --2- 172.21.15.62:0/774830740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1740e30c0 0x7fb174057d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0697c0 tx=0x7fb17c0a54c0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7f89efe00640 1 -- 172.21.15.62:0/1331490112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e80ead00 msgr2=0x7f89e804fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.211+0000 7f89efe00640 1 --2- 172.21.15.62:0/1331490112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e80ead00 0x7f89e804fa90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89e80528a0 tx=0x7f89e002a0f0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7fb402800640 1 --2- 172.21.15.62:0/1667593054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404071510 0x7fb4040ce300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb40406e410 tx=0x7fb3fc02a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8858f29aee1b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f3f17e00640 1 -- 172.21.15.62:0/3269199937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f180d1130 msgr2=0x7f3f180cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f3f17e00640 1 --2- 172.21.15.62:0/3269199937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f180d1130 0x7f3f180cef10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009870 tx=0x7f3f0802a230 comp rx=0 tx=0).stop 2024-04-25T09:03:24.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7fa941e00640 1 --2- 172.21.15.62:0/4258207301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94415dc50 0x7fa9440d1a70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa9300097a0 tx=0x7fa93002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2726adcf9c2017f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f89efe00640 1 -- 172.21.15.62:0/1331490112 shutdown_connections 2024-04-25T09:03:24.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f89efe00640 1 --2- 172.21.15.62:0/1331490112 >> v2:172.21.15.121:3300/0 conn(0x7f89e8006ea0 0x7f89e8007310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f89efe00640 1 --2- 172.21.15.62:0/1331490112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e80ead00 0x7f89e804fa90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f89efe00640 1 -- 172.21.15.62:0/1331490112 >> 172.21.15.62:0/1331490112 conn(0x7f89e8009fb0 msgr2=0x7f89e800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7fb182600640 1 -- 172.21.15.62:0/774830740 shutdown_connections 2024-04-25T09:03:24.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7fb182600640 1 --2- 172.21.15.62:0/774830740 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb174054300 0x7fb1740068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7fb182600640 1 --2- 172.21.15.62:0/774830740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1740e30c0 0x7fb174057d90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f89efe00640 1 -- 172.21.15.62:0/1331490112 shutdown_connections 2024-04-25T09:03:24.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7fb182600640 1 -- 172.21.15.62:0/774830740 >> 172.21.15.62:0/774830740 conn(0x7fb1740559e0 msgr2=0x7fb174055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f89efe00640 1 -- 172.21.15.62:0/1331490112 wait complete. 2024-04-25T09:03:24.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7fa940a00640 1 -- 172.21.15.62:0/4258207301 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930037070 con 0x7fa94415dc50 2024-04-25T09:03:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f3f17e00640 1 -- 172.21.15.62:0/3269199937 shutdown_connections 2024-04-25T09:03:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f3f17e00640 1 --2- 172.21.15.62:0/3269199937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f180d1130 0x7f3f180cef10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f3f17e00640 1 --2- 172.21.15.62:0/3269199937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f18161430 0x7f3f180d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7f3f17e00640 1 -- 172.21.15.62:0/3269199937 >> 172.21.15.62:0/3269199937 conn(0x7f3f18066d10 msgr2=0x7f3f18067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.212+0000 7fb400a00640 1 -- 172.21.15.62:0/1667593054 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc036070 con 0x7fb404071510 2024-04-25T09:03:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.213+0000 7fb400a00640 1 -- 172.21.15.62:0/1667593054 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc031600 con 0x7fb404071510 2024-04-25T09:03:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.213+0000 7fb400a00640 1 -- 172.21.15.62:0/1667593054 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0319d0 con 0x7fb404071510 2024-04-25T09:03:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.213+0000 7fb182600640 1 -- 172.21.15.62:0/774830740 shutdown_connections 2024-04-25T09:03:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.213+0000 7fa940a00640 1 -- 172.21.15.62:0/4258207301 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930032440 con 0x7fa94415dc50 2024-04-25T09:03:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.213+0000 7fa940a00640 1 -- 172.21.15.62:0/4258207301 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930032810 con 0x7fa94415dc50 2024-04-25T09:03:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.213+0000 7fb182600640 1 -- 172.21.15.62:0/774830740 wait complete. 2024-04-25T09:03:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.214+0000 7fb408a00640 1 -- 172.21.15.62:0/1667593054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404071510 msgr2=0x7fb4040ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.214+0000 7fb408a00640 1 --2- 172.21.15.62:0/1667593054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404071510 0x7fb4040ce300 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb40406e410 tx=0x7fb3fc02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.214+0000 7fb408a00640 1 -- 172.21.15.62:0/1667593054 shutdown_connections 2024-04-25T09:03:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.214+0000 7fb408a00640 1 --2- 172.21.15.62:0/1667593054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404071510 0x7fb4040ce300 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.214+0000 7fb408a00640 1 --2- 172.21.15.62:0/1667593054 >> v2:172.21.15.62:3300/0 conn(0x7fb404165e80 0x7fb404070fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.214+0000 7fb408a00640 1 --2- 172.21.15.62:0/1667593054 >> v2:172.21.15.121:3300/0 conn(0x7fb404072020 0x7fb4040ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.214+0000 7fb408a00640 1 -- 172.21.15.62:0/1667593054 >> 172.21.15.62:0/1667593054 conn(0x7fb4040668f0 msgr2=0x7fb404066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fa949000640 1 -- 172.21.15.62:0/4258207301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94415dc50 msgr2=0x7fa9440d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fa949000640 1 --2- 172.21.15.62:0/4258207301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94415dc50 0x7fa9440d1a70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa9300097a0 tx=0x7fa93002a270 comp rx=0 tx=0).stop 2024-04-25T09:03:24.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fb22e800640 1 Processor -- start 2024-04-25T09:03:24.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fb408a00640 1 -- 172.21.15.62:0/1667593054 shutdown_connections 2024-04-25T09:03:24.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7f3f17e00640 1 -- 172.21.15.62:0/3269199937 shutdown_connections 2024-04-25T09:03:24.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7f3f17e00640 1 -- 172.21.15.62:0/3269199937 wait complete. 2024-04-25T09:03:24.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fa949000640 1 -- 172.21.15.62:0/4258207301 shutdown_connections 2024-04-25T09:03:24.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fa949000640 1 --2- 172.21.15.62:0/4258207301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94415dc50 0x7fa9440d1a70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fa949000640 1 -- 172.21.15.62:0/4258207301 >> 172.21.15.62:0/4258207301 conn(0x7fa94406afd0 msgr2=0x7fa94406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7f89efe00640 1 Processor -- start 2024-04-25T09:03:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fb182600640 1 Processor -- start 2024-04-25T09:03:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fa949000640 1 -- 172.21.15.62:0/4258207301 shutdown_connections 2024-04-25T09:03:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fa949000640 1 -- 172.21.15.62:0/4258207301 wait complete. 2024-04-25T09:03:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7fb22e800640 1 -- start start 2024-04-25T09:03:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.215+0000 7f89efe00640 1 -- start start 2024-04-25T09:03:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.216+0000 7fb182600640 1 -- start start 2024-04-25T09:03:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.216+0000 7faa77400640 1 Processor -- start 2024-04-25T09:03:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.216+0000 7faa77400640 1 -- start start 2024-04-25T09:03:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7f89efe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e8006ea0 0x7f89e817d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7f89efe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89e804ffd0 0x7f89e80e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7fb22e800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb2280d0ae0 0x7fb2280d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7fb22e800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2280d2620 con 0x7fb2280b6cf0 2024-04-25T09:03:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7fb22e800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2280d2900 con 0x7fb22815dc40 2024-04-25T09:03:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7fb22e800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2280d23c0 con 0x7fb2280d0ae0 2024-04-25T09:03:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7f89efe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e80ead00 0x7f89e80460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7f3f17e00640 1 Processor -- start 2024-04-25T09:03:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.217+0000 7f89efe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80547c0 con 0x7f89e8006ea0 2024-04-25T09:03:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7f89efe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8053fc0 con 0x7f89e80ead00 2024-04-25T09:03:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7f89efe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8054940 con 0x7f89e804ffd0 2024-04-25T09:03:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7f3f17e00640 1 -- start start 2024-04-25T09:03:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb182600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17403e840 0x7fb17404eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb182600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb174054300 0x7fb17417c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb182600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740e30c0 0x7fb174039320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7f89eea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e8006ea0 0x7f89e817d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7f89ee000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89e804ffd0 0x7f89e80e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fa949000640 1 Processor -- start 2024-04-25T09:03:24.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7f89ee000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89e804ffd0 0x7f89e80e8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41528/0 (socket says 172.21.15.62:41528) 2024-04-25T09:03:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.216+0000 7fb408a00640 1 -- 172.21.15.62:0/1667593054 wait complete. 2024-04-25T09:03:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb182600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740e34a0 con 0x7fb174054300 2024-04-25T09:03:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb182600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17403bfc0 con 0x7fb1740e30c0 2024-04-25T09:03:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb182600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb174006e10 con 0x7fb17403e840 2024-04-25T09:03:24.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7f89ee000640 1 -- 172.21.15.62:0/891756246 learned_addr learned my addr 172.21.15.62:0/891756246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7f89ef400640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e80ead00 0x7f89e80460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb408a00640 1 Processor -- start 2024-04-25T09:03:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb227400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb22815dc40 0x7fb2280d1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35334/0 (socket says 172.21.15.62:35334) 2024-04-25T09:03:24.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.218+0000 7fb227400640 1 -- 172.21.15.62:0/2514056137 learned_addr learned my addr 172.21.15.62:0/2514056137 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fb408a00640 1 -- start start 2024-04-25T09:03:24.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fa949000640 1 -- start start 2024-04-25T09:03:24.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fb181c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740e30c0 0x7fb174039320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fb181200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17403e840 0x7fb17404eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fb181c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740e30c0 0x7fb174039320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53512/0 (socket says 172.21.15.62:53512) 2024-04-25T09:03:24.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fb181c00640 1 -- 172.21.15.62:0/785638671 learned_addr learned my addr 172.21.15.62:0/785638671 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7faa77400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faa700d45a0 0x7faa700d3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fb21ea00640 1 --2- 172.21.15.62:0/2514056137 >> v2:172.21.15.121:3300/0 conn(0x7fb2280d0ae0 0x7fb2280d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f180b2740 0x7f3f181588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180d1130 0x7f3f180cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7faa77400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 0x7faa700d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7faa77400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa700d4980 con 0x7faa700d3760 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7faa77400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa700d1730 con 0x7faa700d45a0 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7faa77400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa700d1cf0 con 0x7faa700d2180 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fb181c00640 1 -- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17403e840 msgr2=0x7fb17404eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.219+0000 7fb181c00640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb17403e840 0x7fb17404eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7fb181c00640 1 -- 172.21.15.62:0/785638671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb174054300 msgr2=0x7fb17417c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7fb181c00640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb174054300 0x7fb17417c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7fb181c00640 1 -- 172.21.15.62:0/785638671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17417d0a0 con 0x7fb1740e30c0 2024-04-25T09:03:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f89eea00640 1 -- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89e804ffd0 msgr2=0x7f89e80e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f89eea00640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89e804ffd0 0x7f89e80e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f18161430 0x7f3f180cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f89eea00640 1 -- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e80ead00 msgr2=0x7f89e80460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f89eea00640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89e80ead00 0x7f89e80460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f89eea00640 1 -- 172.21.15.62:0/891756246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e804cbe0 con 0x7f89e8006ea0 2024-04-25T09:03:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f3f17e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180b2f20 con 0x7f3f180d1130 2024-04-25T09:03:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f3f17e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180b2da0 con 0x7f3f18161430 2024-04-25T09:03:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f3f17e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180d1670 con 0x7f3f180b2740 2024-04-25T09:03:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7faa6fe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7faa700d3760 0x7faa70162660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35350/0 (socket says 172.21.15.62:35350) 2024-04-25T09:03:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7faa6fe00640 1 -- 172.21.15.62:0/2779819904 learned_addr learned my addr 172.21.15.62:0/2779819904 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7faa74a00640 1 --2- 172.21.15.62:0/2779819904 >> v2:172.21.15.121:3300/0 conn(0x7faa700d45a0 0x7faa700d3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7fb181c00640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740e30c0 0x7fb174039320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb168004650 tx=0x7fb16800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f3f16000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180d1130 0x7f3f180cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f3f16000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180d1130 0x7f3f180cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59544/0 (socket says 172.21.15.62:59544) 2024-04-25T09:03:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.220+0000 7f3f16000640 1 -- 172.21.15.62:0/3065812247 learned_addr learned my addr 172.21.15.62:0/3065812247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7f3f17400640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f18161430 0x7f3f180cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7f89eea00640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e8006ea0 0x7f89e817d030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f89e0031000 tx=0x7f89e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fb408a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb404071510 0x7fb404164e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904420341 0 0) 0x7fb2280d2900 con 0x7fb22815dc40 2024-04-25T09:03:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7f3f16000640 1 -- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f180b2740 msgr2=0x7f3f181588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb20c003520 con 0x7fb22815dc40 2024-04-25T09:03:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7f3f16000640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f180b2740 0x7f3f181588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7f3f16000640 1 -- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f18161430 msgr2=0x7f3f180cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7f3f16000640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f18161430 0x7f3f180cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7f3f16000640 1 -- 172.21.15.62:0/3065812247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180cd540 con 0x7f3f180d1130 2024-04-25T09:03:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7faa75400640 1 --2- 172.21.15.62:0/2779819904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 0x7faa700d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168017070 con 0x7fb1740e30c0 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16800c940 con 0x7fb1740e30c0 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16800cd30 con 0x7fb1740e30c0 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7f3f17400640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f18161430 0x7f3f180cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fa949000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9440b9650 0x7fa9440c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fa949000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9440cee60 0x7fa94414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fa949000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94415dc50 0x7fa9441519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fa949000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440d1fb0 con 0x7fa94415dc50 2024-04-25T09:03:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7faa74a00640 1 -- 172.21.15.62:0/2779819904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 msgr2=0x7faa700d25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7faa74a00640 1 --2- 172.21.15.62:0/2779819904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 0x7faa700d25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7faa74a00640 1 -- 172.21.15.62:0/2779819904 >> v1:172.21.15.121:6789/0 conn(0x7faa700d3760 legacy=0x7faa70162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7faa74a00640 1 -- 172.21.15.62:0/2779819904 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa70163170 con 0x7faa700d45a0 2024-04-25T09:03:24.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17417d380 con 0x7fb1740e30c0 2024-04-25T09:03:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb408a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb404072020 0x7fb40415d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fa949000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440d2ef0 con 0x7fa9440b9650 2024-04-25T09:03:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.221+0000 7fa949000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440d2130 con 0x7fa9440cee60 2024-04-25T09:03:24.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb174051890 con 0x7fb1740e30c0 2024-04-25T09:03:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7f3f16000640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180d1130 0x7f3f180cab70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000c7c0 tx=0x7f3f1000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb408a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404165e80 0x7fb404163580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb401400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb404072020 0x7fb40415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fb408a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb404166460 con 0x7fb404071510 2024-04-25T09:03:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fb408a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4041662e0 con 0x7fb404072020 2024-04-25T09:03:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fb408a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb404072400 con 0x7fb404165e80 2024-04-25T09:03:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241891708 0 0) 0x7fb2280d2620 con 0x7fb2280b6cf0 2024-04-25T09:03:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2280d2900 con 0x7fb2280b6cf0 2024-04-25T09:03:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fa941e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9440b9650 0x7fa9440c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fa941e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9440b9650 0x7fa9440c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53532/0 (socket says 172.21.15.62:53532) 2024-04-25T09:03:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fa941e00640 1 -- 172.21.15.62:0/1289575231 learned_addr learned my addr 172.21.15.62:0/1289575231 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fa941400640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9440cee60 0x7fa94414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fb401400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb404072020 0x7fb40415d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53546/0 (socket says 172.21.15.62:53546) 2024-04-25T09:03:24.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fb401400640 1 -- 172.21.15.62:0/696294751 learned_addr learned my addr 172.21.15.62:0/696294751 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10016020 con 0x7f3f180d1130 2024-04-25T09:03:24.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb401e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb404071510 0x7fb404164e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f180cd820 con 0x7f3f180d1130 2024-04-25T09:03:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fb402800640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404165e80 0x7fb404163580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0041070 con 0x7f89e8006ea0 2024-04-25T09:03:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e003c670 con 0x7f89e8006ea0 2024-04-25T09:03:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0037040 con 0x7f89e8006ea0 2024-04-25T09:03:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e804cec0 con 0x7f89e8006ea0 2024-04-25T09:03:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1740057e0 con 0x7fb1740e30c0 2024-04-25T09:03:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fa942800640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94415dc50 0x7fa9441519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f180cdde0 con 0x7f3f180d1130 2024-04-25T09:03:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1000aa50 con 0x7f3f180d1130 2024-04-25T09:03:24.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10016020 con 0x7f3f180d1130 2024-04-25T09:03:24.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1134298256 0 0) 0x7fb20c003520 con 0x7fb22815dc40 2024-04-25T09:03:24.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 >> v2:172.21.15.121:3300/0 conn(0x7fb2280d0ae0 msgr2=0x7fb2280d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 --2- 172.21.15.62:0/2514056137 >> v2:172.21.15.121:3300/0 conn(0x7fb2280d0ae0 0x7fb2280d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 >> v1:172.21.15.62:6789/0 conn(0x7fb2280b6cf0 legacy=0x7fb2280b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280d2be0 con 0x7fb22815dc40 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 <== mon.1 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb2100015f0 con 0x7fb22815dc40 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 >> v1:172.21.15.121:6789/0 conn(0x7fb22815dc40 legacy=0x7fb2280d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb20c003b90 0x7fb20c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20c004800 0x7fb20c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb20c0051c0 0x7fb20c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280d23c0 con 0x7fb20c003b90 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c00c090 con 0x7fb20c004800 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.222+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c00c290 con 0x7fb20c0051c0 2024-04-25T09:03:24.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.223+0000 7fb21ea00640 1 --2- 172.21.15.62:0/2514056137 >> v2:172.21.15.121:3300/0 conn(0x7fb2280d0ae0 0x7fb2280d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7fa941400640 1 -- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9440b9650 msgr2=0x7fa9440c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7fa941400640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9440b9650 0x7fa9440c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7fa941400640 1 -- 172.21.15.62:0/1289575231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94415dc50 msgr2=0x7fa9441519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7fa941400640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa94415dc50 0x7fa9441519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7fb227400640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb20c003b90 0x7fb20c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7faa74a00640 1 --2- 172.21.15.62:0/2779819904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 0x7faa700d3220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa68009d90 tx=0x7faa6802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147c10ea688539f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e804d480 con 0x7f89e8006ea0 2024-04-25T09:03:24.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7fb227e00640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb20c0051c0 0x7fb20c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7fb402800640 1 -- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb404072020 msgr2=0x7fb40415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7fb402800640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb404072020 0x7fb40415d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7fb402800640 1 -- 172.21.15.62:0/696294751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb404071510 msgr2=0x7fb404164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7fb402800640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb404071510 0x7fb404164e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7fb402800640 1 -- 172.21.15.62:0/696294751 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4040b48c0 con 0x7fb404165e80 2024-04-25T09:03:24.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7fb401e00640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb404071510 0x7fb404164e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:24.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.224+0000 7fa941400640 1 -- 172.21.15.62:0/1289575231 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9441523a0 con 0x7fa9440cee60 2024-04-25T09:03:24.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7faa6f400640 1 -- 172.21.15.62:0/2779819904 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68007740 con 0x7faa700d45a0 2024-04-25T09:03:24.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7faa6f400640 1 -- 172.21.15.62:0/2779819904 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa680078e0 con 0x7faa700d45a0 2024-04-25T09:03:24.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7faa6f400640 1 -- 172.21.15.62:0/2779819904 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68007cb0 con 0x7faa700d45a0 2024-04-25T09:03:24.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89e8005390 con 0x7f89e8006ea0 2024-04-25T09:03:24.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7fb21ea00640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20c004800 0x7fb20c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.225+0000 7fa941400640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9440cee60 0x7fa94414fee0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa93c002260 tx=0x7fa93c00cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb402800640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404165e80 0x7fb404163580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc030000 tx=0x7fb3fc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb227400640 1 -- 172.21.15.62:0/2514056137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb20c0051c0 msgr2=0x7fb20c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb227400640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb20c0051c0 0x7fb20c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb227400640 1 -- 172.21.15.62:0/2514056137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20c004800 msgr2=0x7fb20c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb227400640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20c004800 0x7fb20c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb227400640 1 -- 172.21.15.62:0/2514056137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c003520 con 0x7fb20c003b90 2024-04-25T09:03:24.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb227e00640 1 -- 172.21.15.62:0/2514056137 reap_dead start 2024-04-25T09:03:24.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7faa77400640 1 -- 172.21.15.62:0/2779819904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 msgr2=0x7faa700d3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7faa77400640 1 --2- 172.21.15.62:0/2779819904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 0x7faa700d3220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa68009d90 tx=0x7faa6802a660 comp rx=0 tx=0).stop 2024-04-25T09:03:24.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ef0001030 con 0x7f3f180d1130 2024-04-25T09:03:24.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc036050 con 0x7fb404165e80 2024-04-25T09:03:24.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc042070 con 0x7fb404165e80 2024-04-25T09:03:24.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.226+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc009510 con 0x7fb404165e80 2024-04-25T09:03:24.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb404162aa0 con 0x7fb404165e80 2024-04-25T09:03:24.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb404163060 con 0x7fb404165e80 2024-04-25T09:03:24.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c017070 con 0x7fa9440cee60 2024-04-25T09:03:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c00ac30 con 0x7fa9440cee60 2024-04-25T09:03:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c012600 con 0x7fa9440cee60 2024-04-25T09:03:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7fa949000640 1 -- 172.21.15.62:0/1289575231 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa944152680 con 0x7fa9440cee60 2024-04-25T09:03:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7f4f24600640 1 Processor -- start 2024-04-25T09:03:24.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7fa949000640 1 -- 172.21.15.62:0/1289575231 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94414f6e0 con 0x7fa9440cee60 2024-04-25T09:03:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7f4f24600640 1 -- start start 2024-04-25T09:03:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.227+0000 7fb227400640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb20c003b90 0x7fb20c003f90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb210007f20 tx=0x7fb20c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87c3d0ac729c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.228+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210007b40 con 0x7fb20c003b90 2024-04-25T09:03:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.228+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb210005ad0 con 0x7fb20c003b90 2024-04-25T09:03:24.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.228+0000 7fb226a00640 1 -- 172.21.15.62:0/2514056137 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210007070 con 0x7fb20c003b90 2024-04-25T09:03:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7fa949000640 1 -- 172.21.15.62:0/1289575231 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa91c001030 con 0x7fa9440cee60 2024-04-25T09:03:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.228+0000 7faa77400640 1 -- 172.21.15.62:0/2779819904 shutdown_connections 2024-04-25T09:03:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7faa77400640 1 --2- 172.21.15.62:0/2779819904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 0x7faa700d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7faa77400640 1 --2- 172.21.15.62:0/2779819904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 0x7faa700d3220 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7faa77400640 1 -- 172.21.15.62:0/2779819904 >> 172.21.15.62:0/2779819904 conn(0x7faa7006b000 msgr2=0x7faa7006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f10019020 con 0x7f3f180d1130 2024-04-25T09:03:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb404072c70 con 0x7fb404165e80 2024-04-25T09:03:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7faa77400640 1 -- 172.21.15.62:0/2779819904 shutdown_connections 2024-04-25T09:03:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89e003a070 con 0x7f89e8006ea0 2024-04-25T09:03:24.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7faa77400640 1 -- 172.21.15.62:0/2779819904 wait complete. 2024-04-25T09:03:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7fb22e800640 1 -- 172.21.15.62:0/2514056137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb20c003b90 msgr2=0x7fb20c003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7fb22e800640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb20c003b90 0x7fb20c003f90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb210007f20 tx=0x7fb20c005830 comp rx=0 tx=0).stop 2024-04-25T09:03:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.229+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb16801a070 con 0x7fb1740e30c0 2024-04-25T09:03:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7fb22e800640 1 -- 172.21.15.62:0/2514056137 shutdown_connections 2024-04-25T09:03:24.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7f4f24600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f1c15def0 0x7f4f1c0b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7f4f24600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f1c0d3770 0x7f4f1c0b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7fb22e800640 1 --2- 172.21.15.62:0/2514056137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb20c003b90 0x7fb20c003f90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7fb22e800640 1 -- 172.21.15.62:0/2514056137 >> 172.21.15.62:0/2514056137 conn(0x7fb22806b000 msgr2=0x7fb22806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7f4f24600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0d2270 con 0x7f4f1c0d3310 2024-04-25T09:03:24.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7f3f0ea00640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef806b6d0 0x7f3ef806db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7f4f24600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c0d2550 con 0x7f4f1c15def0 2024-04-25T09:03:24.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7fb22e800640 1 -- 172.21.15.62:0/2514056137 shutdown_connections 2024-04-25T09:03:24.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7f4f21c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f1c15def0 0x7f4f1c0b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f10089ef0 con 0x7f3f180d1130 2024-04-25T09:03:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f21c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f1c15def0 0x7f4f1c0b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59576/0 (socket says 172.21.15.62:59576) 2024-04-25T09:03:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f21c00640 1 -- 172.21.15.62:0/2517545087 learned_addr learned my addr 172.21.15.62:0/2517545087 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f89e6a00640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89d006b5d0 0x7f89d006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f3f0ea00640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f3ef8073870 0x7f3ef8075c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 --> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f100079f0 con 0x7f3ef8073870 2024-04-25T09:03:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f1008c510 con 0x7f3f180d1130 2024-04-25T09:03:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89e00aab10 con 0x7f89e8006ea0 2024-04-25T09:03:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f24600640 1 -- 172.21.15.62:0/2517545087 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c0d2830 con 0x7f4f1c0d3770 2024-04-25T09:03:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f21200640 1 --1- 172.21.15.62:0/2517545087 >> v1:172.21.15.121:6789/0 conn(0x7f4f1c0d3310 0x7f4f1c0cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35352/0 (socket says 172.21.15.62:35352) 2024-04-25T09:03:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f22600640 1 --2- 172.21.15.62:0/2517545087 >> v2:172.21.15.121:3300/0 conn(0x7f4f1c0d3770 0x7f4f1c0b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f21c00640 1 -- 172.21.15.62:0/2517545087 >> v2:172.21.15.121:3300/0 conn(0x7f4f1c0d3770 msgr2=0x7f4f1c0b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f21c00640 1 --2- 172.21.15.62:0/2517545087 >> v2:172.21.15.121:3300/0 conn(0x7f4f1c0d3770 0x7f4f1c0b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f21c00640 1 -- 172.21.15.62:0/2517545087 >> v1:172.21.15.121:6789/0 conn(0x7f4f1c0d3310 legacy=0x7f4f1c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.231+0000 7f4f21c00640 1 -- 172.21.15.62:0/2517545087 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0d2b10 con 0x7f4f1c15def0 2024-04-25T09:03:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.232+0000 7f89e6a00640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f89d00737f0 0x7f89d0075c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.232+0000 7f3f16a00640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef806b6d0 0x7f3ef806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.232+0000 7f3f17400640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f3ef8073870 0x7f3ef8075c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.232+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 --> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89e0033e50 con 0x7f89d00737f0 2024-04-25T09:03:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.232+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e00aaf00 con 0x7f89e8006ea0 2024-04-25T09:03:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7f89ef400640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f89d00737f0 0x7f89d0075c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.232+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3fc007bf0 con 0x7fb404165e80 2024-04-25T09:03:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.232+0000 7faa77400640 1 Processor -- start 2024-04-25T09:03:24.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7faa77400640 1 -- start start 2024-04-25T09:03:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.232+0000 7f89ee000640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89d006b5d0 0x7f89d006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7f4f21c00640 1 --2- 172.21.15.62:0/2517545087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c15def0 0x7f4f1c0b95c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0ce1a0 tx=0x7f4f1402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfe25ea6ee061dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.230+0000 7fb22e800640 1 -- 172.21.15.62:0/2514056137 wait complete. 2024-04-25T09:03:24.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7f3f17400640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f3ef8073870 0x7f3ef8075c90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7fb22e800640 1 Processor -- start 2024-04-25T09:03:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7fb178c00640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb15c06b5a0 0x7fb15c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7fb3fa000640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3e406b4d0 0x7fb3e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3fc0ab9d0 con 0x7fb404165e80 2024-04-25T09:03:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.233+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa93c01a070 con 0x7fa9440cee60 2024-04-25T09:03:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.234+0000 7fb3fa000640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fb3e40736f0 0x7fb3e4075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.234+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 --> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3fc041ec0 con 0x7fb3e40736f0 2024-04-25T09:03:24.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.234+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3fc0abdc0 con 0x7fb404165e80 2024-04-25T09:03:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.234+0000 7f4f1be00640 1 -- 172.21.15.62:0/2517545087 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14036070 con 0x7f4f1c15def0 2024-04-25T09:03:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.234+0000 7f4f1be00640 1 -- 172.21.15.62:0/2517545087 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f14031600 con 0x7f4f1c15def0 2024-04-25T09:03:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.234+0000 7f4f1be00640 1 -- 172.21.15.62:0/2517545087 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140319d0 con 0x7f4f1c15def0 2024-04-25T09:03:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.234+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb16808a540 con 0x7fb1740e30c0 2024-04-25T09:03:24.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.234+0000 7f89ef400640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f89d00737f0 0x7f89d0075c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fb401e00640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3e406b4d0 0x7fb3e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fb401400640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fb3e40736f0 0x7fb3e4075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7f4f24600640 1 -- 172.21.15.62:0/2517545087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c15def0 msgr2=0x7f4f1c0b95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7f4f24600640 1 --2- 172.21.15.62:0/2517545087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c15def0 0x7f4f1c0b95c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0ce1a0 tx=0x7f4f1402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7faa77400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 0x7faa70148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fb22e800640 1 -- start start 2024-04-25T09:03:24.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7f3f16a00640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef806b6d0 0x7f3ef806db90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006a90 tx=0x7f3f04006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fb181200640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb15c06b5a0 0x7fb15c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7f89ee000640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89d006b5d0 0x7f89d006da90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89e80362b0 tx=0x7f89dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fa93a000640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa92406b5a0 0x7fa92406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa93c08a8e0 con 0x7fa9440cee60 2024-04-25T09:03:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fa93a000640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fa9240737c0 0x7fa924075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 --> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa924076290 con 0x7fa9240737c0 2024-04-25T09:03:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa93c08acd0 con 0x7fa9440cee60 2024-04-25T09:03:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7faa77400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa700d3760 0x7faa700b6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7faa77400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 0x7faa700b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7f4f24600640 1 -- 172.21.15.62:0/2517545087 shutdown_connections 2024-04-25T09:03:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7f4f24600640 1 --2- 172.21.15.62:0/2517545087 >> v2:172.21.15.121:3300/0 conn(0x7f4f1c0d3770 0x7f4f1c0b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7f4f24600640 1 --2- 172.21.15.62:0/2517545087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c15def0 0x7f4f1c0b95c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7f4f24600640 1 -- 172.21.15.62:0/2517545087 >> 172.21.15.62:0/2517545087 conn(0x7f4f1c06b000 msgr2=0x7f4f1c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 <== osd.1 v2:172.21.15.62:6808/2146620681 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3f100079f0 con 0x7f3ef8073870 2024-04-25T09:03:24.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.235+0000 7fb178c00640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fb15c0737c0 0x7fb15c075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 --> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb168023e50 con 0x7fb15c0737c0 2024-04-25T09:03:24.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7fb401400640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fb3e40736f0 0x7fb3e4075b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb16808a930 con 0x7fb1740e30c0 2024-04-25T09:03:24.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7fa942800640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fa9240737c0 0x7fa924075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa74a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 0x7faa70148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa74a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 0x7faa70148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41590/0 (socket says 172.21.15.62:41590) 2024-04-25T09:03:24.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa700d3760 0x7faa700b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7f4f24600640 1 -- 172.21.15.62:0/2517545087 shutdown_connections 2024-04-25T09:03:24.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa75400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 0x7faa700b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa75400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 0x7faa700b83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53586/0 (socket says 172.21.15.62:53586) 2024-04-25T09:03:24.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa75400640 1 -- 172.21.15.62:0/1492234189 learned_addr learned my addr 172.21.15.62:0/1492234189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 <== osd.2 v2:172.21.15.62:6816/385571314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f89e0033e50 con 0x7f89d00737f0 2024-04-25T09:03:24.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700d01e0 con 0x7faa700d3760 2024-04-25T09:03:24.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700d1980 con 0x7faa700d45a0 2024-04-25T09:03:24.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa70162d70 con 0x7faa700d2180 2024-04-25T09:03:24.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7fa941e00640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa92406b5a0 0x7fa92406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7fb181200640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb15c06b5a0 0x7fb15c06da60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb174056930 tx=0x7fb17c0aa000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7fb17be00640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fb15c0737c0 0x7fb15c075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7fb401e00640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3e406b4d0 0x7fb3e406d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400b880 tx=0x7fb3f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7fb22e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20c003b90 0x7fb2281f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7fb17be00640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fb15c0737c0 0x7fb15c075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7faa6fe00640 1 -- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 msgr2=0x7faa70148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7faa6fe00640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faa700d2180 0x7faa70148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7faa6fe00640 1 -- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 msgr2=0x7faa700b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7faa6fe00640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa700d45a0 0x7faa700b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7faa6fe00640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa701ebed0 con 0x7faa700d3760 2024-04-25T09:03:24.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb22e800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22815a540 0x7fb228148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.236+0000 7f4f24600640 1 -- 172.21.15.62:0/2517545087 wait complete. 2024-04-25T09:03:24.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7faa6fe00640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa700d3760 0x7faa700b6740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa5c000f60 tx=0x7faa5c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb22e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2281f4640 0x7fb22815f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb227e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2281f4640 0x7fb22815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb21ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22815a540 0x7fb228148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb227e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2281f4640 0x7fb22815f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59600/0 (socket says 172.21.15.62:59600) 2024-04-25T09:03:24.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb21ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22815a540 0x7fb228148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41596/0 (socket says 172.21.15.62:41596) 2024-04-25T09:03:24.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb21ea00640 1 -- 172.21.15.62:0/2036158990 learned_addr learned my addr 172.21.15.62:0/2036158990 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb227400640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20c003b90 0x7fb2281f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.239+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c017070 con 0x7faa700d3760 2024-04-25T09:03:24.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.239+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c005ce0 con 0x7faa700d3760 2024-04-25T09:03:24.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.237+0000 7fa942800640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fa9240737c0 0x7fa924075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fa941e00640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa92406b5a0 0x7fa92406da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa93002a2b0 tx=0x7fa93002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280d36c0 con 0x7fb2281f4640 2024-04-25T09:03:24.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2280d31a0 con 0x7fb20c003b90 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.238+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281f7b70 con 0x7fb22815a540 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.239+0000 7fb227e00640 1 -- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22815a540 msgr2=0x7fb228148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.239+0000 7fb227e00640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb22815a540 0x7fb228148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.239+0000 7fb227e00640 1 -- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20c003b90 msgr2=0x7fb2281f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.239+0000 7fb227e00640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20c003b90 0x7fb2281f7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.239+0000 7fb227e00640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2281f7e50 con 0x7fb2281f4640 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.240+0000 7f4f24600640 1 Processor -- start 2024-04-25T09:03:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.240+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c007310 con 0x7faa700d3760 2024-04-25T09:03:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.240+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 <== osd.4 v2:172.21.15.121:6808/3370771089 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa924076290 con 0x7fa9240737c0 2024-04-25T09:03:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.240+0000 7f4f24600640 1 -- start start 2024-04-25T09:03:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.240+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 <== osd.3 v2:172.21.15.62:6817/3104548384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb3fc041ec0 con 0x7fb3e40736f0 2024-04-25T09:03:24.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.240+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa700b9420 con 0x7faa700d3760 2024-04-25T09:03:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.240+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa701ec470 con 0x7faa700d3760 2024-04-25T09:03:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.241+0000 7fb227e00640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2281f4640 0x7fb22815f200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2280d4c10 tx=0x7fb22000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.241+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 <== osd.0 v2:172.21.15.62:6800/565545488 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb168023e50 con 0x7fb15c0737c0 2024-04-25T09:03:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.241+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa700729a0 con 0x7faa700d3760 2024-04-25T09:03:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.242+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220019070 con 0x7fb2281f4640 2024-04-25T09:03:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.242+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb220006e20 con 0x7fb2281f4640 2024-04-25T09:03:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.242+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220003bb0 con 0x7fb2281f4640 2024-04-25T09:03:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.243+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2281f8130 con 0x7fb2281f4640 2024-04-25T09:03:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.243+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22815a9a0 con 0x7fb2281f4640 2024-04-25T09:03:24.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.243+0000 7f4f24600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0d3310 0x7f4f1c14b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.243+0000 7f4f24600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15def0 0x7f4f1c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.243+0000 7f4f24600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b35b0 0x7f4f1c0b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.244+0000 7f4f21200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15def0 0x7f4f1c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.245+0000 7f4f21200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15def0 0x7f4f1c0b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53598/0 (socket says 172.21.15.62:53598) 2024-04-25T09:03:24.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.245+0000 7f4f21200640 1 -- 172.21.15.62:0/2138968327 learned_addr learned my addr 172.21.15.62:0/2138968327 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.245+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb200001030 con 0x7fb2281f4640 2024-04-25T09:03:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.244+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa5c01a070 con 0x7faa700d3760 2024-04-25T09:03:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7faa6cc00640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faa5006b520 0x7faa5006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7f4f22600640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b35b0 0x7f4f1c0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0d1e00 con 0x7f4f1c0d3310 2024-04-25T09:03:24.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b9b00 con 0x7f4f1c15def0 2024-04-25T09:03:24.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0d1f80 con 0x7f4f1c0b35b0 2024-04-25T09:03:24.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa5c0564c0 con 0x7faa700d3760 2024-04-25T09:03:24.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7faa6cc00640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7faa50074410 0x7faa500768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa50076f80 con 0x7faa50074410 2024-04-25T09:03:24.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.246+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa5c054e50 con 0x7faa700d3760 2024-04-25T09:03:24.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.247+0000 7f4f21c00640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0d3310 0x7f4f1c14b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.247+0000 7f4f21200640 1 -- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b35b0 msgr2=0x7f4f1c0b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.248+0000 7faa75400640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7faa50074410 0x7faa500768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.248+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2200050d0 con 0x7fb2281f4640 2024-04-25T09:03:24.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.247+0000 7f4f21200640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b35b0 0x7f4f1c0b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.247+0000 7f4f21200640 1 -- 172.21.15.62:0/2138968327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0d3310 msgr2=0x7f4f1c14b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.247+0000 7f4f21200640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0d3310 0x7f4f1c14b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.247+0000 7f4f21200640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c15f2f0 con 0x7f4f1c15def0 2024-04-25T09:03:24.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.248+0000 7faa74a00640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faa5006b520 0x7faa5006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.248+0000 7f4f21c00640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0d3310 0x7f4f1c14b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:24.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.248+0000 7f4f21200640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15def0 0x7f4f1c0b5010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c440 tx=0x7f4f08001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.249+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08017070 con 0x7f4f1c15def0 2024-04-25T09:03:24.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.249+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f08007500 con 0x7f4f1c15def0 2024-04-25T09:03:24.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.249+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f080059e0 con 0x7f4f1c15def0 2024-04-25T09:03:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.249+0000 7fb21fe00640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb20406b520 0x7fb20406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.249+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c15f5d0 con 0x7f4f1c15def0 2024-04-25T09:03:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.249+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c15fb90 con 0x7f4f1c15def0 2024-04-25T09:03:24.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.249+0000 7faa75400640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7faa50074410 0x7faa500768b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.249+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2200895d0 con 0x7fb2281f4640 2024-04-25T09:03:24.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.250+0000 7fb21fe00640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fb2040737a0 0x7fb204075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.250+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb204076270 con 0x7fb2040737a0 2024-04-25T09:03:24.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.250+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb22008e050 con 0x7fb2281f4640 2024-04-25T09:03:24.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.250+0000 7fb227400640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb20406b520 0x7fb20406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.250+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ef8001030 con 0x7f4f1c15def0 2024-04-25T09:03:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.250+0000 7fb21ea00640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fb2040737a0 0x7fb204075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.251+0000 7faa74a00640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faa5006b520 0x7faa5006d9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa68034000 tx=0x7faa68033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.251+0000 7fb227400640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb20406b520 0x7fb20406d9e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb210032000 tx=0x7fb210034040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.251+0000 7fb21ea00640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fb2040737a0 0x7fb204075bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.252+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 <== osd.5 v2:172.21.15.121:6816/3527509140 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faa50076f80 con 0x7faa50074410 2024-04-25T09:03:24.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.255+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f0801a070 con 0x7f4f1c15def0 2024-04-25T09:03:24.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.255+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 <== osd.6 v2:172.21.15.121:6800/3195802018 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb204076270 con 0x7fb2040737a0 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.256+0000 7f4f19600640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4efc06b520 0x7f4efc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.257+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f08089bb0 con 0x7f4f1c15def0 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.257+0000 7f4f19600640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7f4efc073740 0x7f4efc075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.257+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f08010e50 con 0x7f4efc073740 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.257+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f0808a090 con 0x7f4f1c15def0 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.257+0000 7f4f21c00640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4efc06b520 0x7f4efc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.258+0000 7f4f22600640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7f4efc073740 0x7f4efc075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.259+0000 7f4f22600640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7f4efc073740 0x7f4efc075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.259+0000 7f4f21c00640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4efc06b520 0x7f4efc06d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4f14033000 tx=0x7f4f14009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.261+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 <== osd.7 v2:172.21.15.121:6820/1260954068 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4f08010e50 con 0x7f4efc073740 2024-04-25T09:03:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.269+0000 7fa949000640 1 -- 172.21.15.62:0/1289575231 --> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa91c002dd0 con 0x7fa9240737c0 2024-04-25T09:03:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.270+0000 7fa93a000640 1 -- 172.21.15.62:0/1289575231 <== osd.4 v2:172.21.15.121:6808/3370771089 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa91c002dd0 con 0x7fa9240737c0 2024-04-25T09:03:24.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.273+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 --> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89e80531d0 con 0x7f89d00737f0 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.273+0000 7fa92f400640 1 -- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fa9240737c0 msgr2=0x7fa924075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.273+0000 7fa92f400640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fa9240737c0 0x7fa924075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa92f400640 1 -- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa92406b5a0 msgr2=0x7fa92406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa92f400640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa92406b5a0 0x7fa92406da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa93002a2b0 tx=0x7fa93002e000 comp rx=0 tx=0).stop 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa92f400640 1 -- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9440cee60 msgr2=0x7fa94414fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa92f400640 1 --2- 172.21.15.62:0/1289575231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9440cee60 0x7fa94414fee0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa93c002260 tx=0x7fa93c00cf70 comp rx=0 tx=0).stop 2024-04-25T09:03:24.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7f89e6a00640 1 -- 172.21.15.62:0/891756246 <== osd.2 v2:172.21.15.62:6816/385571314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f89e80531d0 con 0x7f89d00737f0 2024-04-25T09:03:24.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa942800640 1 -- 172.21.15.62:0/1289575231 reap_dead start 2024-04-25T09:03:24.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa92f400640 1 -- 172.21.15.62:0/1289575231 shutdown_connections 2024-04-25T09:03:24.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa92f400640 1 -- 172.21.15.62:0/1289575231 >> 172.21.15.62:0/1289575231 conn(0x7fa94406afd0 msgr2=0x7fa9440b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.273+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 --> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ef0002dd0 con 0x7f3ef8073870 2024-04-25T09:03:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7f3f0ea00640 1 -- 172.21.15.62:0/3065812247 <== osd.1 v2:172.21.15.62:6808/2146620681 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3ef0002dd0 con 0x7f3ef8073870 2024-04-25T09:03:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa92f400640 1 -- 172.21.15.62:0/1289575231 shutdown_connections 2024-04-25T09:03:24.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.274+0000 7fa92f400640 1 -- 172.21.15.62:0/1289575231 wait complete. 2024-04-25T09:03:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f3ef8073870 msgr2=0x7f3ef8075c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f3f17e00640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f3ef8073870 0x7f3ef8075c90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f89d00737f0 msgr2=0x7f89d0075c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f89efe00640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f89d00737f0 0x7f89d0075c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef806b6d0 msgr2=0x7f3ef806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f3f17e00640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef806b6d0 0x7f3ef806db90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006a90 tx=0x7f3f04006a20 comp rx=0 tx=0).stop 2024-04-25T09:03:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180d1130 msgr2=0x7f3f180cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f3f17e00640 1 --2- 172.21.15.62:0/3065812247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180d1130 0x7f3f180cab70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000c7c0 tx=0x7f3f1000cc80 comp rx=0 tx=0).stop 2024-04-25T09:03:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89d006b5d0 msgr2=0x7f89d006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f89efe00640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89d006b5d0 0x7f89d006da90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89e80362b0 tx=0x7f89dc009040 comp rx=0 tx=0).stop 2024-04-25T09:03:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e8006ea0 msgr2=0x7f89e817d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f89efe00640 1 --2- 172.21.15.62:0/891756246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89e8006ea0 0x7f89e817d030 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f89e0031000 tx=0x7f89e00046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f89ef400640 1 -- 172.21.15.62:0/891756246 reap_dead start 2024-04-25T09:03:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.275+0000 7f3f17400640 1 -- 172.21.15.62:0/3065812247 reap_dead start 2024-04-25T09:03:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.276+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 shutdown_connections 2024-04-25T09:03:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.276+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 >> 172.21.15.62:0/3065812247 conn(0x7f3f18066d10 msgr2=0x7f3f180b7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.276+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 shutdown_connections 2024-04-25T09:03:24.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.277+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 shutdown_connections 2024-04-25T09:03:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.277+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 >> 172.21.15.62:0/891756246 conn(0x7f89e8009fb0 msgr2=0x7f89e80e9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.277+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 shutdown_connections 2024-04-25T09:03:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.277+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 --> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb404165bf0 con 0x7fb3e40736f0 2024-04-25T09:03:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.277+0000 7f89efe00640 1 -- 172.21.15.62:0/891756246 wait complete. 2024-04-25T09:03:24.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.277+0000 7f3f17e00640 1 -- 172.21.15.62:0/3065812247 wait complete. 2024-04-25T09:03:24.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.278+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 --> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb17404f620 con 0x7fb15c0737c0 2024-04-25T09:03:24.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.278+0000 7fb3fa000640 1 -- 172.21.15.62:0/696294751 <== osd.3 v2:172.21.15.62:6817/3104548384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb404165bf0 con 0x7fb3e40736f0 2024-04-25T09:03:24.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb178c00640 1 -- 172.21.15.62:0/785638671 <== osd.0 v2:172.21.15.62:6800/565545488 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb17404f620 con 0x7fb15c0737c0 2024-04-25T09:03:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fb3e40736f0 msgr2=0x7fb3e4075b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fb3e40736f0 0x7fb3e4075b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3e406b4d0 msgr2=0x7fb3e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3e406b4d0 0x7fb3e406d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400b880 tx=0x7fb3f4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404165e80 msgr2=0x7fb404163580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 --2- 172.21.15.62:0/696294751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb404165e80 0x7fb404163580 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc030000 tx=0x7fb3fc004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb402800640 1 -- 172.21.15.62:0/696294751 reap_dead start 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 shutdown_connections 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 >> 172.21.15.62:0/696294751 conn(0x7fb4040668f0 msgr2=0x7fb4040ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 shutdown_connections 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb408a00640 1 -- 172.21.15.62:0/696294751 wait complete. 2024-04-25T09:03:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.279+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fb15c0737c0 msgr2=0x7fb15c075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.280+0000 7fb182600640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fb15c0737c0 0x7fb15c075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.280+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb15c06b5a0 msgr2=0x7fb15c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.280+0000 7fb182600640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb15c06b5a0 0x7fb15c06da60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb174056930 tx=0x7fb17c0aa000 comp rx=0 tx=0).stop 2024-04-25T09:03:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.280+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740e30c0 msgr2=0x7fb174039320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.280+0000 7fb182600640 1 --2- 172.21.15.62:0/785638671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1740e30c0 0x7fb174039320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb168004650 tx=0x7fb16800ece0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.280+0000 7fb181c00640 1 -- 172.21.15.62:0/785638671 reap_dead start 2024-04-25T09:03:24.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.281+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 shutdown_connections 2024-04-25T09:03:24.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.281+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 >> 172.21.15.62:0/785638671 conn(0x7fb1740559e0 msgr2=0x7fb1740e6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.281+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 shutdown_connections 2024-04-25T09:03:24.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.282+0000 7fb182600640 1 -- 172.21.15.62:0/785638671 wait complete. 2024-04-25T09:03:24.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.287+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 --> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa7004f000 con 0x7faa50074410 2024-04-25T09:03:24.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.288+0000 7faa6cc00640 1 -- 172.21.15.62:0/1492234189 <== osd.5 v2:172.21.15.121:6816/3527509140 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faa7004f000 con 0x7faa50074410 2024-04-25T09:03:24.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.288+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 --> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4ef8002dd0 con 0x7f4efc073740 2024-04-25T09:03:24.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.289+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7faa50074410 msgr2=0x7faa500768b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.289+0000 7faa77400640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7faa50074410 0x7faa500768b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.289+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faa5006b520 msgr2=0x7faa5006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.289+0000 7faa77400640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faa5006b520 0x7faa5006d9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa68034000 tx=0x7faa68033040 comp rx=0 tx=0).stop 2024-04-25T09:03:24.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.289+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa700d3760 msgr2=0x7faa700b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.289+0000 7faa77400640 1 --2- 172.21.15.62:0/1492234189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faa700d3760 0x7faa700b6740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa5c000f60 tx=0x7faa5c00c800 comp rx=0 tx=0).stop 2024-04-25T09:03:24.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.289+0000 7faa75400640 1 -- 172.21.15.62:0/1492234189 reap_dead start 2024-04-25T09:03:24.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.289+0000 7f4f19600640 1 -- 172.21.15.62:0/2138968327 <== osd.7 v2:172.21.15.121:6820/1260954068 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4ef8002dd0 con 0x7f4efc073740 2024-04-25T09:03:24.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 shutdown_connections 2024-04-25T09:03:24.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 >> 172.21.15.62:0/1492234189 conn(0x7faa7006b000 msgr2=0x7faa7006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7f4efc073740 msgr2=0x7f4efc075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7f4f24600640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7f4efc073740 0x7f4efc075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 shutdown_connections 2024-04-25T09:03:24.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7faa77400640 1 -- 172.21.15.62:0/1492234189 wait complete. 2024-04-25T09:03:24.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4efc06b520 msgr2=0x7f4efc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7f4f24600640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4efc06b520 0x7f4efc06d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4f14033000 tx=0x7f4f14009000 comp rx=0 tx=0).stop 2024-04-25T09:03:24.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15def0 msgr2=0x7f4f1c0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.290+0000 7f4f24600640 1 --2- 172.21.15.62:0/2138968327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15def0 0x7f4f1c0b5010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c440 tx=0x7f4f08001e70 comp rx=0 tx=0).stop 2024-04-25T09:03:24.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.292+0000 7f4f22600640 1 -- 172.21.15.62:0/2138968327 reap_dead start 2024-04-25T09:03:24.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.294+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 --> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb200002dd0 con 0x7fb2040737a0 2024-04-25T09:03:24.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.294+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 shutdown_connections 2024-04-25T09:03:24.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.294+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 >> 172.21.15.62:0/2138968327 conn(0x7f4f1c06b000 msgr2=0x7f4f1c0b8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.294+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 shutdown_connections 2024-04-25T09:03:24.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.294+0000 7f4f24600640 1 -- 172.21.15.62:0/2138968327 wait complete. 2024-04-25T09:03:24.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.295+0000 7fb21fe00640 1 -- 172.21.15.62:0/2036158990 <== osd.6 v2:172.21.15.121:6800/3195802018 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb200002dd0 con 0x7fb2040737a0 2024-04-25T09:03:24.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.296+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fb2040737a0 msgr2=0x7fb204075bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.296+0000 7fb22e800640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fb2040737a0 0x7fb204075bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.297+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb20406b520 msgr2=0x7fb20406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.297+0000 7fb22e800640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb20406b520 0x7fb20406d9e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb210032000 tx=0x7fb210034040 comp rx=0 tx=0).stop 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.297+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2281f4640 msgr2=0x7fb22815f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.297+0000 7fb22e800640 1 --2- 172.21.15.62:0/2036158990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2281f4640 0x7fb22815f200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2280d4c10 tx=0x7fb22000ff70 comp rx=0 tx=0).stop 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.297+0000 7fb227e00640 1 -- 172.21.15.62:0/2036158990 reap_dead start 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.297+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 shutdown_connections 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.297+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 >> 172.21.15.62:0/2036158990 conn(0x7fb22806b000 msgr2=0x7fb2280b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.297+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 shutdown_connections 2024-04-25T09:03:24.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.298+0000 7fb22e800640 1 -- 172.21.15.62:0/2036158990 wait complete. 2024-04-25T09:03:24.319 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-25T09:03:24.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T09:03:24.320 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-25T09:03:24.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T09:03:24.321 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-25T09:03:24.322 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T09:03:24.322 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-25T09:03:24.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T09:03:24.323 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-04-25T09:03:24.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T09:03:24.324 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-25T09:03:24.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T09:03:24.325 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-04-25T09:03:24.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T09:03:24.327 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-04-25T09:03:24.327 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T09:03:24.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.664+0000 7fec18c00640 1 Processor -- start 2024-04-25T09:03:24.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.664+0000 7fec18c00640 1 -- start start 2024-04-25T09:03:24.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.666+0000 7fec18c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec0c048100 0x7fec0c006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.666+0000 7fec18c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c007460 0x7fec0c053e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.666+0000 7fec18c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c005ec0 con 0x7fec0c054390 2024-04-25T09:03:24.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.666+0000 7fec18c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0c0e7b90 con 0x7fec0c048100 2024-04-25T09:03:24.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.666+0000 7fec18c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0547e0 con 0x7fec0c007460 2024-04-25T09:03:24.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.667+0000 7fec11400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c007460 0x7fec0c053e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.667+0000 7fec11400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c007460 0x7fec0c053e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41608/0 (socket says 172.21.15.62:41608) 2024-04-25T09:03:24.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.667+0000 7fec11400640 1 -- 172.21.15.62:0/3545196561 learned_addr learned my addr 172.21.15.62:0/3545196561 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.668+0000 7fec11e00640 1 --2- 172.21.15.62:0/3545196561 >> v2:172.21.15.121:3300/0 conn(0x7fec0c048100 0x7fec0c006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.669+0000 7f4cf3200640 1 Processor -- start 2024-04-25T09:03:24.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.670+0000 7f4cf3200640 1 -- start start 2024-04-25T09:03:24.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.669+0000 7fec11e00640 1 -- 172.21.15.62:0/3545196561 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c007460 msgr2=0x7fec0c053e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.669+0000 7fec11e00640 1 --2- 172.21.15.62:0/3545196561 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c007460 0x7fec0c053e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.669+0000 7fec11e00640 1 -- 172.21.15.62:0/3545196561 >> v1:172.21.15.121:6789/0 conn(0x7fec0c054390 legacy=0x7fec0c04f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.669+0000 7fec11e00640 1 -- 172.21.15.62:0/3545196561 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c053280 con 0x7fec0c048100 2024-04-25T09:03:24.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.671+0000 7fec11e00640 1 --2- 172.21.15.62:0/3545196561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c048100 0x7fec0c006f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7febfc0097a0 tx=0x7febfc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffdf61d0c5a41203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.671+0000 7f511ee00640 1 Processor -- start 2024-04-25T09:03:24.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.672+0000 7f4cf3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4ce80eb060 0x7f4ce8055300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.672+0000 7f4cf3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce80387e0 con 0x7f4ce80eb440 2024-04-25T09:03:24.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.672+0000 7f4cf3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce8056d90 con 0x7f4ce80061a0 2024-04-25T09:03:24.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.672+0000 7f4cf3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce8057e30 con 0x7f4ce80eb060 2024-04-25T09:03:24.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.672+0000 7f4cf1400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4ce80eb440 0x7f4ce80eb8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41024/0 (socket says 172.21.15.62:41024) 2024-04-25T09:03:24.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.672+0000 7f4cf1400640 1 -- 172.21.15.62:0/3872562492 learned_addr learned my addr 172.21.15.62:0/3872562492 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.672+0000 7f511ee00640 1 -- start start 2024-04-25T09:03:24.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.672+0000 7f4cf1e00640 1 --2- 172.21.15.62:0/3872562492 >> v2:172.21.15.121:3300/0 conn(0x7f4ce80eb060 0x7f4ce8055300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.673+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867128669 0 0) 0x7f4ce8056d90 con 0x7f4ce80061a0 2024-04-25T09:03:24.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.673+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd4003520 con 0x7f4ce80061a0 2024-04-25T09:03:24.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.673+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868307311 0 0) 0x7f4cec04fe10 con 0x7f4ce80eb440 2024-04-25T09:03:24.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.673+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce8056d90 con 0x7f4ce80eb440 2024-04-25T09:03:24.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f511ee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f5110049b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f511ee00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f511004a0a0 0x7f51100548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f511ee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51100e7e30 con 0x7f5110054de0 2024-04-25T09:03:24.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f511ee00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511003b5a0 con 0x7f511004a0a0 2024-04-25T09:03:24.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f511ee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511003b880 con 0x7f51100e7a50 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1295927923 0 0) 0x7f4ce8056d90 con 0x7f4ce80eb440 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 >> v2:172.21.15.121:3300/0 conn(0x7f4ce80eb060 msgr2=0x7f4ce8055300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 --2- 172.21.15.62:0/3872562492 >> v2:172.21.15.121:3300/0 conn(0x7f4ce80eb060 0x7f4ce8055300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 >> v1:172.21.15.121:6789/0 conn(0x7f4ce80061a0 legacy=0x7f4ce8006610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8057070 con 0x7f4ce80eb440 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f4cec04f770 con 0x7f4ce80eb440 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 >> v1:172.21.15.62:6789/0 conn(0x7f4ce80eb440 legacy=0x7f4ce80eb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cd4004ba0 0x7f4cd4004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd40037a0 0x7f4cd4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cd4004160 0x7f4cd400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7fec10a00640 1 -- 172.21.15.62:0/3545196561 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc037070 con 0x7fec0c048100 2024-04-25T09:03:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd400bfb0 con 0x7f4cd4004160 2024-04-25T09:03:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd400c130 con 0x7f4cd4004ba0 2024-04-25T09:03:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7fec10a00640 1 -- 172.21.15.62:0/3545196561 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc032440 con 0x7fec0c048100 2024-04-25T09:03:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.675+0000 7f511ce00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5110054de0 0x7f511003c360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35376/0 (socket says 172.21.15.62:35376) 2024-04-25T09:03:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.675+0000 7f511ce00640 1 -- 172.21.15.62:0/1203535513 learned_addr learned my addr 172.21.15.62:0/1203535513 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd400c330 con 0x7f4cd40037a0 2024-04-25T09:03:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.674+0000 7fec10a00640 1 -- 172.21.15.62:0/3545196561 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc032810 con 0x7fec0c048100 2024-04-25T09:03:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.675+0000 7f4cf2800640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cd4004160 0x7f4cd400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.675+0000 7f4cf1400640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd40037a0 0x7f4cd4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f4cf2800640 1 -- 172.21.15.62:0/3872562492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd40037a0 msgr2=0x7f4cd4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f4cf2800640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd40037a0 0x7f4cd4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f4cf2800640 1 -- 172.21.15.62:0/3872562492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cd4004ba0 msgr2=0x7f4cd4004fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:03:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f4cf2800640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cd4004ba0 0x7f4cd4004fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f4cf2800640 1 -- 172.21.15.62:0/3872562492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8057070 con 0x7f4cd4004160 2024-04-25T09:03:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7fec18c00640 1 -- 172.21.15.62:0/3545196561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c048100 msgr2=0x7fec0c006f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7fec18c00640 1 --2- 172.21.15.62:0/3545196561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c048100 0x7fec0c006f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7febfc0097a0 tx=0x7febfc02a270 comp rx=0 tx=0).stop 2024-04-25T09:03:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.675+0000 7f5117e00640 1 --2- 172.21.15.62:0/1203535513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f5110049b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.675+0000 7f5117400640 1 --2- 172.21.15.62:0/1203535513 >> v2:172.21.15.121:3300/0 conn(0x7f511004a0a0 0x7f51100548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f5116a00640 1 -- 172.21.15.62:0/1203535513 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367994708 0 0) 0x7f51100e7e30 con 0x7f5110054de0 2024-04-25T09:03:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7fec18c00640 1 -- 172.21.15.62:0/3545196561 shutdown_connections 2024-04-25T09:03:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f5116a00640 1 -- 172.21.15.62:0/1203535513 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5100003520 con 0x7f5110054de0 2024-04-25T09:03:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f5117e00640 1 -- 172.21.15.62:0/1203535513 >> v2:172.21.15.121:3300/0 conn(0x7f511004a0a0 msgr2=0x7f51100548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f5117e00640 1 --2- 172.21.15.62:0/1203535513 >> v2:172.21.15.121:3300/0 conn(0x7f511004a0a0 0x7f51100548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.676+0000 7f5117e00640 1 -- 172.21.15.62:0/1203535513 >> v1:172.21.15.121:6789/0 conn(0x7f5110054de0 legacy=0x7f511003c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7f5117e00640 1 -- 172.21.15.62:0/1203535513 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511003bb60 con 0x7f51100e7a50 2024-04-25T09:03:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7fec18c00640 1 --2- 172.21.15.62:0/3545196561 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c007460 0x7fec0c053e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7fec18c00640 1 --2- 172.21.15.62:0/3545196561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c048100 0x7fec0c006f20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7fec18c00640 1 -- 172.21.15.62:0/3545196561 >> 172.21.15.62:0/3545196561 conn(0x7fec0c009fe0 msgr2=0x7fec0c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7fec18c00640 1 -- 172.21.15.62:0/3545196561 shutdown_connections 2024-04-25T09:03:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7fb99ca00640 1 Processor -- start 2024-04-25T09:03:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7fb99ca00640 1 -- start start 2024-04-25T09:03:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7f4cf2800640 1 -- 172.21.15.62:0/3872562492 reap_dead start 2024-04-25T09:03:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.678+0000 7f5117e00640 1 --2- 172.21.15.62:0/1203535513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f5110049b60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f511802d370 tx=0x7f51180a53e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1cba1af38c5a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.679+0000 7f5116a00640 1 -- 172.21.15.62:0/1203535513 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180aa7d0 con 0x7f51100e7a50 2024-04-25T09:03:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.679+0000 7f5116a00640 1 -- 172.21.15.62:0/1203535513 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180aa9b0 con 0x7f51100e7a50 2024-04-25T09:03:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.677+0000 7fec18c00640 1 -- 172.21.15.62:0/3545196561 wait complete. 2024-04-25T09:03:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.679+0000 7fec18c00640 1 Processor -- start 2024-04-25T09:03:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.679+0000 7fb99ca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb99815dc40 0x7fb9980d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.679+0000 7fec18c00640 1 -- start start 2024-04-25T09:03:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.678+0000 7f4cf2800640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cd4004160 0x7f4cd400b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00018e0 tx=0x7f4cd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e4f0f6637e96a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0001500 con 0x7f4cd4004160 2024-04-25T09:03:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce002e070 con 0x7f4cd4004160 2024-04-25T09:03:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7f4cf0a00640 1 -- 172.21.15.62:0/3872562492 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0005070 con 0x7f4cd4004160 2024-04-25T09:03:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7f5116a00640 1 -- 172.21.15.62:0/1203535513 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180aae20 con 0x7f51100e7a50 2024-04-25T09:03:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7f511ee00640 1 -- 172.21.15.62:0/1203535513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 msgr2=0x7f5110049b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7f511ee00640 1 --2- 172.21.15.62:0/1203535513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f5110049b60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f511802d370 tx=0x7f51180a53e0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.679+0000 7fb99ca00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9980d4900 con 0x7fb9980d0f70 2024-04-25T09:03:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.679+0000 7fb99ca00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9980d1400 con 0x7fb9980d0ae0 2024-04-25T09:03:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.679+0000 7fb99ca00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9980d2620 con 0x7fb99815dc40 2024-04-25T09:03:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7f11d3c00640 1 Processor -- start 2024-04-25T09:03:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7fb995e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb99815dc40 0x7fb9980d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7fb995400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb9980d0ae0 0x7fb9980d3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35384/0 (socket says 172.21.15.62:35384) 2024-04-25T09:03:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7fb995400640 1 -- 172.21.15.62:0/739234908 learned_addr learned my addr 172.21.15.62:0/739234908 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f4cf3200640 1 -- 172.21.15.62:0/3872562492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cd4004160 msgr2=0x7f4cd400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f4cf3200640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cd4004160 0x7f4cd400b9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00018e0 tx=0x7f4cd4005830 comp rx=0 tx=0).stop 2024-04-25T09:03:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f4cf3200640 1 -- 172.21.15.62:0/3872562492 shutdown_connections 2024-04-25T09:03:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f4cf3200640 1 --2- 172.21.15.62:0/3872562492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cd4004160 0x7f4cd400b9c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f4cf3200640 1 -- 172.21.15.62:0/3872562492 >> 172.21.15.62:0/3872562492 conn(0x7f4ce80559e0 msgr2=0x7f4ce8055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7fb994a00640 1 -- 172.21.15.62:0/739234908 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580989022 0 0) 0x7fb9980d1400 con 0x7fb9980d0ae0 2024-04-25T09:03:24.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7fb994a00640 1 -- 172.21.15.62:0/739234908 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb97c003520 con 0x7fb9980d0ae0 2024-04-25T09:03:24.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f4cf3200640 1 -- 172.21.15.62:0/3872562492 shutdown_connections 2024-04-25T09:03:24.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7fec18c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0c007460 0x7fec0c048fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7fec18c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c048100 0x7fec0c04b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7fec18c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c054390 0x7fec0c04d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7fb995e00640 1 -- 172.21.15.62:0/739234908 >> v1:172.21.15.121:6789/0 conn(0x7fb9980d0ae0 legacy=0x7fb9980d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7fb995e00640 1 -- 172.21.15.62:0/739234908 >> v1:172.21.15.62:6789/0 conn(0x7fb9980d0f70 legacy=0x7fb9980d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7fb995e00640 1 -- 172.21.15.62:0/739234908 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9980d2900 con 0x7fb99815dc40 2024-04-25T09:03:24.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7f4cf3200640 1 -- 172.21.15.62:0/3872562492 wait complete. 2024-04-25T09:03:24.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.680+0000 7f11d3c00640 1 -- start start 2024-04-25T09:03:24.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7f11d3c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 0x7f11cc0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7f11d3c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11cc0d1400 con 0x7f11cc15dc40 2024-04-25T09:03:24.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7f11d3c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11cc0d2620 con 0x7f11cc0d0ae0 2024-04-25T09:03:24.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7f11d3c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0d23c0 con 0x7f11cc0d0f70 2024-04-25T09:03:24.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.682+0000 7fec11e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0c007460 0x7fec0c048fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7fec11e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0c007460 0x7fec0c048fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59642/0 (socket says 172.21.15.62:59642) 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f511ee00640 1 -- 172.21.15.62:0/1203535513 shutdown_connections 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f511ee00640 1 --2- 172.21.15.62:0/1203535513 >> v2:172.21.15.121:3300/0 conn(0x7f511004a0a0 0x7f51100548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f511ee00640 1 --2- 172.21.15.62:0/1203535513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f5110049b60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.681+0000 7f511ee00640 1 -- 172.21.15.62:0/1203535513 >> 172.21.15.62:0/1203535513 conn(0x7f5110056cd0 msgr2=0x7f51100570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7fec11400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c048100 0x7fec0c04b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7f511ee00640 1 -- 172.21.15.62:0/1203535513 shutdown_connections 2024-04-25T09:03:24.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7fec11400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c048100 0x7fec0c04b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41642/0 (socket says 172.21.15.62:41642) 2024-04-25T09:03:24.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7fec11e00640 1 -- 172.21.15.62:0/1448018570 learned_addr learned my addr 172.21.15.62:0/1448018570 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7f11d1c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 0x7f11cc0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7f511ee00640 1 -- 172.21.15.62:0/1203535513 wait complete. 2024-04-25T09:03:24.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7f11d1200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f11cc15dc40 0x7f11cc0d1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41048/0 (socket says 172.21.15.62:41048) 2024-04-25T09:03:24.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.684+0000 7f11d1200640 1 -- 172.21.15.62:0/2450957139 learned_addr learned my addr 172.21.15.62:0/2450957139 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.683+0000 7f11cbe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f11cc0d0ae0 0x7f11cc0b6cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35400/0 (socket says 172.21.15.62:35400) 2024-04-25T09:03:24.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.684+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c006440 con 0x7fec0c007460 2024-04-25T09:03:24.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.684+0000 7fec12800640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c054390 0x7fec0c04d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.684+0000 7fb995e00640 1 --2- 172.21.15.62:0/739234908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb99815dc40 0x7fb9980d1cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb9840097d0 tx=0x7fb98402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9629deced5ea8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.684+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c006a90 con 0x7fec0c054390 2024-04-25T09:03:24.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.684+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c006720 con 0x7fec0c048100 2024-04-25T09:03:24.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7fb994a00640 1 -- 172.21.15.62:0/739234908 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984037070 con 0x7fb99815dc40 2024-04-25T09:03:24.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7f11cb400640 1 -- 172.21.15.62:0/2450957139 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267024561 0 0) 0x7f11cc0d2620 con 0x7f11cc0d0ae0 2024-04-25T09:03:24.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7f11cb400640 1 -- 172.21.15.62:0/2450957139 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b4003520 con 0x7f11cc0d0ae0 2024-04-25T09:03:24.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7f4cf3200640 1 Processor -- start 2024-04-25T09:03:24.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7f11d1c00640 1 -- 172.21.15.62:0/2450957139 >> v1:172.21.15.121:6789/0 conn(0x7f11cc0d0ae0 legacy=0x7f11cc0b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7f11d1c00640 1 -- 172.21.15.62:0/2450957139 >> v1:172.21.15.62:6789/0 conn(0x7f11cc15dc40 legacy=0x7f11cc0d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7f11d1c00640 1 -- 172.21.15.62:0/2450957139 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc0d2900 con 0x7f11cc0d0f70 2024-04-25T09:03:24.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7f4cf3200640 1 -- start start 2024-04-25T09:03:24.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7fec11e00640 1 -- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c048100 msgr2=0x7fec0c04b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7fec11e00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c048100 0x7fec0c04b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7fec11e00640 1 -- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c054390 msgr2=0x7fec0c04d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7fec11e00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c054390 0x7fec0c04d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7fb994a00640 1 -- 172.21.15.62:0/739234908 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb984032440 con 0x7fb99815dc40 2024-04-25T09:03:24.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.685+0000 7fb994a00640 1 -- 172.21.15.62:0/739234908 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984032810 con 0x7fb99815dc40 2024-04-25T09:03:24.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.686+0000 7fec11e00640 1 -- 172.21.15.62:0/1448018570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c0494e0 con 0x7fec0c007460 2024-04-25T09:03:24.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.686+0000 7fec11400640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c048100 0x7fec0c04b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:24.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.686+0000 7f9480800640 1 Processor -- start 2024-04-25T09:03:24.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.686+0000 7fec11e00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0c007460 0x7fec0c048fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febfc0098c0 tx=0x7febfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.686+0000 7fb99ca00640 1 -- 172.21.15.62:0/739234908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb99815dc40 msgr2=0x7fb9980d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.686+0000 7fb99ca00640 1 --2- 172.21.15.62:0/739234908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb99815dc40 0x7fb9980d1cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb9840097d0 tx=0x7fb98402a250 comp rx=0 tx=0).stop 2024-04-25T09:03:24.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.686+0000 7f1b2e800640 1 Processor -- start 2024-04-25T09:03:24.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f511ee00640 1 Processor -- start 2024-04-25T09:03:24.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f4cf3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd4004160 0x7f4ce817e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f4cf3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce8009d50 0x7f4ce800a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f4cf3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ce800bc20 0x7f4ce800c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f9480800640 1 -- start start 2024-04-25T09:03:24.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7fec0a000640 1 -- 172.21.15.62:0/1448018570 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc037050 con 0x7fec0c007460 2024-04-25T09:03:24.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7fec0a000640 1 -- 172.21.15.62:0/1448018570 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc043070 con 0x7fec0c007460 2024-04-25T09:03:24.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f511ee00640 1 -- start start 2024-04-25T09:03:24.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f11d1c00640 1 --2- 172.21.15.62:0/2450957139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 0x7f11cc0b7400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009df0 tx=0x7f11c402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569015bf40c8a73f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7fb99ca00640 1 -- 172.21.15.62:0/739234908 shutdown_connections 2024-04-25T09:03:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7fb99ca00640 1 --2- 172.21.15.62:0/739234908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb99815dc40 0x7fb9980d1cb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7fb99ca00640 1 -- 172.21.15.62:0/739234908 >> 172.21.15.62:0/739234908 conn(0x7fb99806b000 msgr2=0x7fb99806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f4cf3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce8057630 con 0x7f4ce8009d50 2024-04-25T09:03:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f4cf3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce80579a0 con 0x7f4ce800bc20 2024-04-25T09:03:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f4cf3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce80e1e10 con 0x7f4cd4004160 2024-04-25T09:03:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f4cf1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd4004160 0x7f4ce817e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7f1b2e800640 1 -- start start 2024-04-25T09:03:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7f4cf1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd4004160 0x7f4ce817e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41660/0 (socket says 172.21.15.62:41660) 2024-04-25T09:03:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7f4cf1e00640 1 -- 172.21.15.62:0/3147436857 learned_addr learned my addr 172.21.15.62:0/3147436857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7fec0a000640 1 -- 172.21.15.62:0/1448018570 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc03ca50 con 0x7fec0c007460 2024-04-25T09:03:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.687+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c04b6f0 con 0x7fec0c007460 2024-04-25T09:03:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7fb99ca00640 1 -- 172.21.15.62:0/739234908 shutdown_connections 2024-04-25T09:03:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7f4cf1400640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce8009d50 0x7f4ce800a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7f4cf2800640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ce800bc20 0x7f4ce800c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7f4cf1e00640 1 -- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ce800bc20 msgr2=0x7f4ce800c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7f4cf1e00640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ce800bc20 0x7f4ce800c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7f4cf1e00640 1 -- 172.21.15.62:0/3147436857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce8009d50 msgr2=0x7f4ce800a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c04a3f0 con 0x7fec0c007460 2024-04-25T09:03:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.688+0000 7fb99ca00640 1 -- 172.21.15.62:0/739234908 wait complete. 2024-04-25T09:03:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f11cb400640 1 -- 172.21.15.62:0/2450957139 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4007740 con 0x7f11cc0d0f70 2024-04-25T09:03:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f11cb400640 1 -- 172.21.15.62:0/2450957139 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c40078e0 con 0x7f11cc0d0f70 2024-04-25T09:03:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f11cb400640 1 -- 172.21.15.62:0/2450957139 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4007740 con 0x7f11cc0d0f70 2024-04-25T09:03:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f1b2e800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b280d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f1b2e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1b280b6cf0 0x7f1b280b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f1b2e800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b280d2620 con 0x7f1b2815dc40 2024-04-25T09:03:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f1b2e800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b280d1480 con 0x7f1b280b6cf0 2024-04-25T09:03:24.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f1b2e800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280d23c0 con 0x7f1b280d0ae0 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f511ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f511004a0a0 0x7f51100dde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f511ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5110054de0 0x7f511004a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f511ee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f511000a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f511ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5110039cb0 con 0x7f5110054de0 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f511ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511004a5b0 con 0x7f511004a0a0 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f511ee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5110039e30 con 0x7f51100e7a50 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f5117e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f511004a0a0 0x7f51100dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f5117400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5110054de0 0x7f511004a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f4cf1e00640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce8009d50 0x7f4ce800a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7f4cf1e00640 1 -- 172.21.15.62:0/3147436857 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce800c900 con 0x7f4cd4004160 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f4cf1400640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce8009d50 0x7f4ce800a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f5117e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f511004a0a0 0x7f51100dde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53654/0 (socket says 172.21.15.62:53654) 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f5117e00640 1 -- 172.21.15.62:0/3783589285 learned_addr learned my addr 172.21.15.62:0/3783589285 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f511ce00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f511000a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.689+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c0053c0 con 0x7fec0c007460 2024-04-25T09:03:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f4cf1e00640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd4004160 0x7f4ce817e200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00d800 tx=0x7f4cdc00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f11d3c00640 1 -- 172.21.15.62:0/2450957139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 msgr2=0x7f11cc0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f11d3c00640 1 --2- 172.21.15.62:0/2450957139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 0x7f11cc0b7400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009df0 tx=0x7f11c402a710 comp rx=0 tx=0).stop 2024-04-25T09:03:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f1b26a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b280d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f1b26a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b280d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41678/0 (socket says 172.21.15.62:41678) 2024-04-25T09:03:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f1b26a00640 1 -- 172.21.15.62:0/2770844766 learned_addr learned my addr 172.21.15.62:0/2770844766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f9480800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94780d0ae0 0x7f94780d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f9480800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780d2620 con 0x7f94780b6cf0 2024-04-25T09:03:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f9480800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780d2900 con 0x7f947815dc40 2024-04-25T09:03:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f9480800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780d23c0 con 0x7f94780d0ae0 2024-04-25T09:03:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f5117e00640 1 -- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 msgr2=0x7f511000a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f5117e00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f511000a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f5117e00640 1 -- 172.21.15.62:0/3783589285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5110054de0 msgr2=0x7f511004a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.690+0000 7f5117e00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5110054de0 0x7f511004a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f5117e00640 1 -- 172.21.15.62:0/3783589285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511004f120 con 0x7f511004a0a0 2024-04-25T09:03:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f1b27e00640 1 --2- 172.21.15.62:0/2770844766 >> v2:172.21.15.62:3300/0 conn(0x7f1b280b6cf0 0x7f1b280b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7fb99ca00640 1 Processor -- start 2024-04-25T09:03:24.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f11d3c00640 1 -- 172.21.15.62:0/2450957139 shutdown_connections 2024-04-25T09:03:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f11d3c00640 1 --2- 172.21.15.62:0/2450957139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 0x7f11cc0b7400 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f11d3c00640 1 -- 172.21.15.62:0/2450957139 >> 172.21.15.62:0/2450957139 conn(0x7f11cc06b000 msgr2=0x7f11cc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f4ce6000640 1 -- 172.21.15.62:0/3147436857 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc018020 con 0x7f4cd4004160 2024-04-25T09:03:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f1b26a00640 1 -- 172.21.15.62:0/2770844766 >> v2:172.21.15.62:3300/0 conn(0x7f1b280b6cf0 msgr2=0x7f1b280b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f1b26a00640 1 --2- 172.21.15.62:0/2770844766 >> v2:172.21.15.62:3300/0 conn(0x7f1b280b6cf0 0x7f1b280b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f1b26a00640 1 -- 172.21.15.62:0/2770844766 >> v1:172.21.15.121:6789/0 conn(0x7f1b2815dc40 legacy=0x7f1b280d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f1b26a00640 1 -- 172.21.15.62:0/2770844766 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280d2900 con 0x7f1b280d0ae0 2024-04-25T09:03:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f4ce6000640 1 -- 172.21.15.62:0/3147436857 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0096c0 con 0x7f4cd4004160 2024-04-25T09:03:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f4ce6000640 1 -- 172.21.15.62:0/3147436857 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc009a90 con 0x7f4cd4004160 2024-04-25T09:03:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce800cbe0 con 0x7f4cd4004160 2024-04-25T09:03:24.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.691+0000 7f947d400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f94780d0ae0 0x7f94780d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.692+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce80e8ac0 con 0x7f4cd4004160 2024-04-25T09:03:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.692+0000 7fb99ca00640 1 -- start start 2024-04-25T09:03:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.692+0000 7f5117e00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f511004a0a0 0x7f51100dde20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f511807a660 tx=0x7f51180a7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7f947e800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f94780b6cf0 0x7f94780b7180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41056/0 (socket says 172.21.15.62:41056) 2024-04-25T09:03:24.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7f947e800640 1 -- 172.21.15.62:0/857955720 learned_addr learned my addr 172.21.15.62:0/857955720 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce80057e0 con 0x7f4cd4004160 2024-04-25T09:03:24.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7f1b26a00640 1 --2- 172.21.15.62:0/2770844766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b280d0f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009e50 tx=0x7f1b1802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160a195f89e78c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7f510be00640 1 -- 172.21.15.62:0/3783589285 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180acbe0 con 0x7f511004a0a0 2024-04-25T09:03:24.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7f510be00640 1 -- 172.21.15.62:0/3783589285 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180acd80 con 0x7f511004a0a0 2024-04-25T09:03:24.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7f510be00640 1 -- 172.21.15.62:0/3783589285 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180b6640 con 0x7f511004a0a0 2024-04-25T09:03:24.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511004f400 con 0x7f511004a0a0 2024-04-25T09:03:24.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7f1b26000640 1 -- 172.21.15.62:0/2770844766 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18036070 con 0x7f1b280d0ae0 2024-04-25T09:03:24.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7f1b26000640 1 -- 172.21.15.62:0/2770844766 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b18031600 con 0x7f1b280d0ae0 2024-04-25T09:03:24.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7f1b26000640 1 -- 172.21.15.62:0/2770844766 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b180319d0 con 0x7f1b280d0ae0 2024-04-25T09:03:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7fb99ca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9980d0ae0 0x7fb9981f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7fb99ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9980d0f70 0x7fb998154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7fb99ca00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb99815dc40 0x7fb9980b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7fb99ca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9980d31a0 con 0x7fb9980d0f70 2024-04-25T09:03:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7fb99ca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9980d2ec0 con 0x7fb9980d0ae0 2024-04-25T09:03:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7fb99ca00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9980b7d00 con 0x7fb99815dc40 2024-04-25T09:03:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7f11d3c00640 1 -- 172.21.15.62:0/2450957139 shutdown_connections 2024-04-25T09:03:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7f11d3c00640 1 -- 172.21.15.62:0/2450957139 wait complete. 2024-04-25T09:03:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.694+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511004f9c0 con 0x7f511004a0a0 2024-04-25T09:03:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7fb995e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9980d0ae0 0x7fb9981f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7fb995400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9980d0f70 0x7fb998154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7f1b2e800640 1 -- 172.21.15.62:0/2770844766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 msgr2=0x7f1b280d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7f1b2e800640 1 --2- 172.21.15.62:0/2770844766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b280d0f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009e50 tx=0x7f1b1802a6d0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7f4ce6000640 1 -- 172.21.15.62:0/3147436857 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cdc01b020 con 0x7f4cd4004160 2024-04-25T09:03:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7fb995400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9980d0f70 0x7fb998154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59682/0 (socket says 172.21.15.62:59682) 2024-04-25T09:03:24.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7fb995400640 1 -- 172.21.15.62:0/2226313154 learned_addr learned my addr 172.21.15.62:0/2226313154 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7f8265a00640 1 Processor -- start 2024-04-25T09:03:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7fb996800640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb99815dc40 0x7fb9980b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.693+0000 7fec0a000640 1 -- 172.21.15.62:0/1448018570 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febfc03cbf0 con 0x7fec0c007460 2024-04-25T09:03:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7fec0a000640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febf006b520 0x7febf006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.695+0000 7fec0a000640 1 -- 172.21.15.62:0/1448018570 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febfc0ab980 con 0x7fec0c007460 2024-04-25T09:03:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f947d400640 1 -- 172.21.15.62:0/857955720 >> v1:172.21.15.121:6789/0 conn(0x7f947815dc40 legacy=0x7f94780d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7fb995400640 1 -- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb99815dc40 msgr2=0x7fb9980b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7fb995400640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb99815dc40 0x7fb9980b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7fb995400640 1 -- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9980d0ae0 msgr2=0x7fb9981f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7fb995400640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9980d0ae0 0x7fb9981f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7fb995400640 1 -- 172.21.15.62:0/2226313154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9981557a0 con 0x7fb9980d0f70 2024-04-25T09:03:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f1b2e800640 1 -- 172.21.15.62:0/2770844766 shutdown_connections 2024-04-25T09:03:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f1b2e800640 1 --2- 172.21.15.62:0/2770844766 >> v2:172.21.15.62:3300/0 conn(0x7f1b280b6cf0 0x7f1b280b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f1b2e800640 1 --2- 172.21.15.62:0/2770844766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b280d0f40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f1b2e800640 1 -- 172.21.15.62:0/2770844766 >> 172.21.15.62:0/2770844766 conn(0x7f1b2806b000 msgr2=0x7f1b2806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f947d400640 1 -- 172.21.15.62:0/857955720 >> v1:172.21.15.62:6789/0 conn(0x7f94780b6cf0 legacy=0x7f94780b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f947d400640 1 -- 172.21.15.62:0/857955720 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780d2be0 con 0x7f94780d0ae0 2024-04-25T09:03:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f4ce6000640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd006b550 0x7f4cd006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7fb995400640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9980d0f70 0x7fb998154d00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb99000c780 tx=0x7fb99000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f4ce6000640 1 -- 172.21.15.62:0/3147436857 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cdc08a6e0 con 0x7f4cd4004160 2024-04-25T09:03:24.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f1b2e800640 1 -- 172.21.15.62:0/2770844766 shutdown_connections 2024-04-25T09:03:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7fec11400640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febf006b520 0x7febf006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511004c2b0 con 0x7f511004a0a0 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.696+0000 7f8265a00640 1 -- start start 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f8265a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82600d0f70 0x7f82600b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f8265a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600d1400 con 0x7f826015dc40 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f8265a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82600d2620 con 0x7f82600d0ae0 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f8265a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82600d23c0 con 0x7f82600d0f70 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f4cf1400640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd006b550 0x7f4cd006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.697+0000 7f1b2e800640 1 -- 172.21.15.62:0/2770844766 wait complete. 2024-04-25T09:03:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.698+0000 7f825ea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f826015dc40 0x7f82600d1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41072/0 (socket says 172.21.15.62:41072) 2024-04-25T09:03:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.698+0000 7f825ea00640 1 -- 172.21.15.62:0/597817725 learned_addr learned my addr 172.21.15.62:0/597817725 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.698+0000 7f947d400640 1 --2- 172.21.15.62:0/857955720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94780d0ae0 0x7f94780d0f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f946c0097d0 tx=0x7f946c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=994420ddaa8582ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.698+0000 7f825f400640 1 --2- 172.21.15.62:0/597817725 >> v2:172.21.15.62:3300/0 conn(0x7f82600d0f70 0x7f82600b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7fec11400640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febf006b520 0x7febf006d9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fec0c034830 tx=0x7fec04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7f4cf1400640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd006b550 0x7f4cd006da10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4cec073a80 tx=0x7f4cec0698e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7f947ca00640 1 -- 172.21.15.62:0/857955720 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c037070 con 0x7f94780d0ae0 2024-04-25T09:03:24.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7f825d600640 1 -- 172.21.15.62:0/597817725 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383813312 0 0) 0x7f82600d2620 con 0x7f82600d0ae0 2024-04-25T09:03:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7f825d600640 1 -- 172.21.15.62:0/597817725 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8244003520 con 0x7f82600d0ae0 2024-04-25T09:03:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7f11d3c00640 1 Processor -- start 2024-04-25T09:03:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7f947ca00640 1 -- 172.21.15.62:0/857955720 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c032440 con 0x7f94780d0ae0 2024-04-25T09:03:24.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7f510be00640 1 -- 172.21.15.62:0/3783589285 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51180c1060 con 0x7f511004a0a0 2024-04-25T09:03:24.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.700+0000 7f825f400640 1 -- 172.21.15.62:0/597817725 >> v1:172.21.15.121:6789/0 conn(0x7f82600d0ae0 legacy=0x7f82600b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.700+0000 7f825f400640 1 -- 172.21.15.62:0/597817725 >> v1:172.21.15.62:6789/0 conn(0x7f826015dc40 legacy=0x7f82600d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.700+0000 7f825f400640 1 -- 172.21.15.62:0/597817725 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600d2900 con 0x7f82600d0f70 2024-04-25T09:03:24.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.700+0000 7f1b2e800640 1 Processor -- start 2024-04-25T09:03:24.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.700+0000 7f1b2e800640 1 -- start start 2024-04-25T09:03:24.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.699+0000 7f11d3c00640 1 -- start start 2024-04-25T09:03:24.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.700+0000 7f947ca00640 1 -- 172.21.15.62:0/857955720 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c032850 con 0x7f94780d0ae0 2024-04-25T09:03:24.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.700+0000 7f9480800640 1 -- 172.21.15.62:0/857955720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94780d0ae0 msgr2=0x7f94780d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.700+0000 7f9480800640 1 --2- 172.21.15.62:0/857955720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94780d0ae0 0x7f94780d0f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f946c0097d0 tx=0x7f946c02a270 comp rx=0 tx=0).stop 2024-04-25T09:03:24.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f510be00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50f806b5a0 0x7f50f806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f510be00640 1 -- 172.21.15.62:0/3783589285 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51181250c0 con 0x7f511004a0a0 2024-04-25T09:03:24.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f1b2e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b6cf0 0x7f1b28160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f825f400640 1 --2- 172.21.15.62:0/597817725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82600d0f70 0x7f82600b7400 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82480097d0 tx=0x7f824802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3875e8d4a86d4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f1b2e800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b28160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f1b2e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b2815dc40 0x7f1b280cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f1b2e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280d31a0 con 0x7f1b2815dc40 2024-04-25T09:03:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f1b2e800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280d2ec0 con 0x7f1b280b6cf0 2024-04-25T09:03:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.701+0000 7f1b2e800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280d36c0 con 0x7f1b280d0ae0 2024-04-25T09:03:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f9480800640 1 -- 172.21.15.62:0/857955720 shutdown_connections 2024-04-25T09:03:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f9480800640 1 --2- 172.21.15.62:0/857955720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94780d0ae0 0x7f94780d0f70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f9480800640 1 -- 172.21.15.62:0/857955720 >> 172.21.15.62:0/857955720 conn(0x7f947806b000 msgr2=0x7f947806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f5117400640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50f806b5a0 0x7f50f806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f1b27400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b6cf0 0x7f1b28160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f9480800640 1 -- 172.21.15.62:0/857955720 shutdown_connections 2024-04-25T09:03:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f11d3c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0d0ae0 0x7f11cc1e7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f1b27400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b6cf0 0x7f1b28160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53690/0 (socket says 172.21.15.62:53690) 2024-04-25T09:03:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f1b27400640 1 -- 172.21.15.62:0/1786215957 learned_addr learned my addr 172.21.15.62:0/1786215957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7fb98e000640 1 -- 172.21.15.62:0/2226313154 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990018070 con 0x7fb9980d0f70 2024-04-25T09:03:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7fb98e000640 1 -- 172.21.15.62:0/2226313154 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99000a640 con 0x7fb9980d0f70 2024-04-25T09:03:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7fb98e000640 1 -- 172.21.15.62:0/2226313154 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99000aab0 con 0x7fb9980d0f70 2024-04-25T09:03:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb998155a80 con 0x7fb9980d0f70 2024-04-25T09:03:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f1b27e00640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b2815dc40 0x7f1b280cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f9480800640 1 -- 172.21.15.62:0/857955720 wait complete. 2024-04-25T09:03:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f825d600640 1 -- 172.21.15.62:0/597817725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248037070 con 0x7f82600d0f70 2024-04-25T09:03:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9980cb0a0 con 0x7fb9980d0f70 2024-04-25T09:03:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f825d600640 1 -- 172.21.15.62:0/597817725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8248032440 con 0x7f82600d0f70 2024-04-25T09:03:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7f825d600640 1 -- 172.21.15.62:0/597817725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248032810 con 0x7f82600d0f70 2024-04-25T09:03:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7f5117400640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50f806b5a0 0x7f50f806da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f510c006fd0 tx=0x7f510c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7f1b27e00640 1 -- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 msgr2=0x7f1b28160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7f1b27e00640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b28160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7f1b27e00640 1 -- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b6cf0 msgr2=0x7f1b28160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7f1b27e00640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b6cf0 0x7f1b28160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7f1b27e00640 1 -- 172.21.15.62:0/1786215957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2814fe80 con 0x7f1b2815dc40 2024-04-25T09:03:24.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.702+0000 7fec0a000640 1 -- 172.21.15.62:0/1448018570 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7febfc03d3b0 con 0x7fec0c007460 2024-04-25T09:03:24.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7f1b27e00640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b2815dc40 0x7f1b280cb660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000d530 tx=0x7f1b2000d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9980748e0 con 0x7fb9980d0f70 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f1b1f400640 1 -- 172.21.15.62:0/1786215957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b200125f0 con 0x7f1b2815dc40 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f1b1f400640 1 -- 172.21.15.62:0/1786215957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b20019070 con 0x7f1b2815dc40 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b28150160 con 0x7f1b2815dc40 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f1b1f400640 1 -- 172.21.15.62:0/1786215957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b200129d0 con 0x7f1b2815dc40 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f11d3c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 0x7f11cc14f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.703+0000 7f11d1200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0d0ae0 0x7f11cc1e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f11d3c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15dc40 0x7f11cc15bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f11d3c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0d31a0 con 0x7f11cc15dc40 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f11d3c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0d2ec0 con 0x7f11cc0d0ae0 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f11d3c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0d36c0 con 0x7f11cc0d0f70 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f11d1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15dc40 0x7f11cc15bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f11d1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15dc40 0x7f11cc15bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59694/0 (socket says 172.21.15.62:59694) 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f11d1c00640 1 -- 172.21.15.62:0/930925298 learned_addr learned my addr 172.21.15.62:0/930925298 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f8265a00640 1 -- 172.21.15.62:0/597817725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82600d0f70 msgr2=0x7f82600b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f8265a00640 1 --2- 172.21.15.62:0/597817725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82600d0f70 0x7f82600b7400 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82480097d0 tx=0x7f824802a250 comp rx=0 tx=0).stop 2024-04-25T09:03:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.704+0000 7f8265a00640 1 -- 172.21.15.62:0/597817725 shutdown_connections 2024-04-25T09:03:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.705+0000 7f8265a00640 1 --2- 172.21.15.62:0/597817725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82600d0f70 0x7f82600b7400 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.705+0000 7f8265a00640 1 -- 172.21.15.62:0/597817725 >> 172.21.15.62:0/597817725 conn(0x7f826006b000 msgr2=0x7f826006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.705+0000 7f11d1c00640 1 -- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 msgr2=0x7f11cc14f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.705+0000 7f11d1c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 0x7f11cc14f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.705+0000 7f11d1c00640 1 -- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0d0ae0 msgr2=0x7f11cc1e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.705+0000 7f11d1c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0d0ae0 0x7f11cc1e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.705+0000 7f11d1c00640 1 -- 172.21.15.62:0/930925298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc1e7a50 con 0x7f11cc15dc40 2024-04-25T09:03:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.705+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b280cb140 con 0x7f1b2815dc40 2024-04-25T09:03:24.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.706+0000 7f9480800640 1 Processor -- start 2024-04-25T09:03:24.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.706+0000 7f9480800640 1 -- start start 2024-04-25T09:03:24.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.706+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b28150680 con 0x7f1b2815dc40 2024-04-25T09:03:24.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.706+0000 7f8265a00640 1 -- 172.21.15.62:0/597817725 shutdown_connections 2024-04-25T09:03:24.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.707+0000 7f8265a00640 1 -- 172.21.15.62:0/597817725 wait complete. 2024-04-25T09:03:24.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.706+0000 7f11d1c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15dc40 0x7f11cc15bec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11c402ad00 tx=0x7f11c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.706+0000 7fb98e000640 1 -- 172.21.15.62:0/2226313154 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb99001b070 con 0x7fb9980d0f70 2024-04-25T09:03:24.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.707+0000 7fb98e000640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb97806b5a0 0x7fb97806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.707+0000 7fb98e000640 1 -- 172.21.15.62:0/2226313154 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9900898f0 con 0x7fb9980d0f70 2024-04-25T09:03:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f9480800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94780b6cf0 0x7f9478156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f9480800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94780d0ae0 0x7f947815f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f9480800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f947815dc40 0x7f9478153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f9480800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780d36c0 con 0x7f94780b6cf0 2024-04-25T09:03:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f9480800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780d31a0 con 0x7f947815dc40 2024-04-25T09:03:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f9480800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9478156f60 con 0x7f94780d0ae0 2024-04-25T09:03:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f947de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94780b6cf0 0x7f9478156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f947de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94780b6cf0 0x7f9478156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59706/0 (socket says 172.21.15.62:59706) 2024-04-25T09:03:24.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.708+0000 7f947de00640 1 -- 172.21.15.62:0/2089191821 learned_addr learned my addr 172.21.15.62:0/2089191821 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7f8265a00640 1 Processor -- start 2024-04-25T09:03:24.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7f947e800640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f947815dc40 0x7f9478153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7fb995e00640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb97806b5a0 0x7fb97806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7f947d400640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94780d0ae0 0x7f947815f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7f947de00640 1 -- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94780d0ae0 msgr2=0x7f947815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7f947de00640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94780d0ae0 0x7f947815f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7f947de00640 1 -- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f947815dc40 msgr2=0x7f9478153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7f947de00640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f947815dc40 0x7f9478153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.709+0000 7f8265a00640 1 -- start start 2024-04-25T09:03:24.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f8265a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82600d0ae0 0x7f8260160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f8265a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82600d0f70 0x7f82601605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f8265a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826015dc40 0x7f82601f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f8265a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600d31a0 con 0x7f826015dc40 2024-04-25T09:03:24.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f8265a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600d2ec0 con 0x7f82600d0f70 2024-04-25T09:03:24.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f8265a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600d36c0 con 0x7f82600d0ae0 2024-04-25T09:03:24.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f947de00640 1 -- 172.21.15.62:0/2089191821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9478157240 con 0x7f94780b6cf0 2024-04-25T09:03:24.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f1b1f400640 1 -- 172.21.15.62:0/1786215957 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b20012b70 con 0x7f1b2815dc40 2024-04-25T09:03:24.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f825f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826015dc40 0x7f82601f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7fb995e00640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb97806b5a0 0x7fb97806da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb984034000 tx=0x7fb98402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f825f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826015dc40 0x7f82601f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:59710/0 (socket says 172.21.15.62:59710) 2024-04-25T09:03:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.710+0000 7f825f400640 1 -- 172.21.15.62:0/2632763809 learned_addr learned my addr 172.21.15.62:0/2632763809 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f825e000640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82600d0f70 0x7f82601605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f825ea00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82600d0ae0 0x7f8260160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f947de00640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94780b6cf0 0x7f9478156960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f947400c730 tx=0x7f947400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f825f400640 1 -- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82600d0ae0 msgr2=0x7f8260160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f825f400640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82600d0ae0 0x7f8260160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f825f400640 1 -- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82600d0f70 msgr2=0x7f82601605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f825f400640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82600d0f70 0x7f82601605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f825f400640 1 -- 172.21.15.62:0/2632763809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826015f860 con 0x7f826015dc40 2024-04-25T09:03:24.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f825f400640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826015dc40 0x7f82601f7450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82480098c0 tx=0x7f82480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.711+0000 7f1b1f400640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b5a0 0x7f1b0406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.712+0000 7f1b1f400640 1 -- 172.21.15.62:0/1786215957 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b2008a520 con 0x7f1b2815dc40 2024-04-25T09:03:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.712+0000 7f1b27400640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b5a0 0x7f1b0406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.713+0000 7f1b27400640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b5a0 0x7f1b0406da60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009990 tx=0x7f1b10009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.713+0000 7f11c8c00640 1 -- 172.21.15.62:0/930925298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4009450 con 0x7f11cc15dc40 2024-04-25T09:03:24.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.713+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc1e7d30 con 0x7f11cc15dc40 2024-04-25T09:03:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.714+0000 7f9472000640 1 -- 172.21.15.62:0/2089191821 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474003b70 con 0x7f94780b6cf0 2024-04-25T09:03:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.714+0000 7f8256a00640 1 -- 172.21.15.62:0/2632763809 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248037050 con 0x7f826015dc40 2024-04-25T09:03:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.714+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8260159c70 con 0x7f826015dc40 2024-04-25T09:03:24.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.714+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9478157520 con 0x7f94780b6cf0 2024-04-25T09:03:24.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.714+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826015f2a0 con 0x7f826015dc40 2024-04-25T09:03:24.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.714+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11cc1e82f0 con 0x7f11cc15dc40 2024-04-25T09:03:24.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.714+0000 7fb98e000640 1 -- 172.21.15.62:0/2226313154 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9980748e0 con 0x7fb9980d0f70 2024-04-25T09:03:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.714+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94780cc3f0 con 0x7f94780b6cf0 2024-04-25T09:03:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.715+0000 7f4ce6000640 1 -- 172.21.15.62:0/3147436857 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4cdc022030 con 0x7f4cd4004160 2024-04-25T09:03:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.715+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82600729c0 con 0x7f826015dc40 2024-04-25T09:03:24.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.715+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11cc0729c0 con 0x7f11cc15dc40 2024-04-25T09:03:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.716+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94780729c0 con 0x7f94780b6cf0 2024-04-25T09:03:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.720+0000 7f510be00640 1 -- 172.21.15.62:0/3783589285 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f511004c2b0 con 0x7f511004a0a0 2024-04-25T09:03:24.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.721+0000 7f9472000640 1 -- 172.21.15.62:0/2089191821 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947400ce40 con 0x7f94780b6cf0 2024-04-25T09:03:24.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.721+0000 7f11c8c00640 1 -- 172.21.15.62:0/930925298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c4007840 con 0x7f11cc15dc40 2024-04-25T09:03:24.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.722+0000 7f11c8c00640 1 -- 172.21.15.62:0/930925298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4004440 con 0x7f11cc15dc40 2024-04-25T09:03:24.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.721+0000 7f9472000640 1 -- 172.21.15.62:0/2089191821 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947400a850 con 0x7f94780b6cf0 2024-04-25T09:03:24.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.722+0000 7f1b1f400640 1 -- 172.21.15.62:0/1786215957 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b28150680 con 0x7f1b2815dc40 2024-04-25T09:03:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.724+0000 7f9472000640 1 -- 172.21.15.62:0/2089191821 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f947400a9f0 con 0x7f94780b6cf0 2024-04-25T09:03:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.726+0000 7f9472000640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f945c06b4d0 0x7f945c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.726+0000 7f9472000640 1 -- 172.21.15.62:0/2089191821 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f947408aa70 con 0x7f94780b6cf0 2024-04-25T09:03:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.724+0000 7f11c8c00640 1 -- 172.21.15.62:0/930925298 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11c4039040 con 0x7f11cc15dc40 2024-04-25T09:03:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.726+0000 7f11c8c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.726+0000 7f11c8c00640 1 -- 172.21.15.62:0/930925298 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11c40abf00 con 0x7f11cc15dc40 2024-04-25T09:03:24.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.725+0000 7f8256a00640 1 -- 172.21.15.62:0/2632763809 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8248038040 con 0x7f826015dc40 2024-04-25T09:03:24.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.726+0000 7f8256a00640 1 -- 172.21.15.62:0/2632763809 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824803c8d0 con 0x7f826015dc40 2024-04-25T09:03:24.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.726+0000 7f8256a00640 1 -- 172.21.15.62:0/2632763809 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8248045020 con 0x7f826015dc40 2024-04-25T09:03:24.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.727+0000 7f11d1200640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.727+0000 7f947d400640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f945c06b4d0 0x7f945c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.727+0000 7f8256a00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f823c06b500 0x7f823c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.727+0000 7f8256a00640 1 -- 172.21.15.62:0/2632763809 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82480ab400 con 0x7f826015dc40 2024-04-25T09:03:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.728+0000 7f825ea00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f823c06b500 0x7f823c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.729+0000 7f947d400640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f945c06b4d0 0x7f945c06d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f946c034000 tx=0x7f946c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.729+0000 7f11d1200640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11b006b4d0 0x7f11b006d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f11c0005880 tx=0x7f11c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.731+0000 7f825ea00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f823c06b500 0x7f823c06d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8258005880 tx=0x7f8258009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.732+0000 7f11c8c00640 1 -- 172.21.15.62:0/930925298 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11c4052020 con 0x7f11cc15dc40 2024-04-25T09:03:24.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.741+0000 7f9472000640 1 -- 172.21.15.62:0/2089191821 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9474059310 con 0x7f94780b6cf0 2024-04-25T09:03:24.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:24.749+0000 7f8256a00640 1 -- 172.21.15.62:0/2632763809 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f824807a410 con 0x7f826015dc40 2024-04-25T09:03:25.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.137+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb9980729c0 con 0x7fb9980d0f70 2024-04-25T09:03:25.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.138+0000 7fb98e000640 1 -- 172.21.15.62:0/2226313154 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb990058100 con 0x7fb9980d0f70 2024-04-25T09:03:25.129 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-04-25T09:03:25.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.139+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fec0c052850 con 0x7fec0c007460 2024-04-25T09:03:25.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.139+0000 7fec0a000640 1 -- 172.21.15.62:0/1448018570 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7febfc07a7e0 con 0x7fec0c007460 2024-04-25T09:03:25.130 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-25T09:03:25.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.142+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb97806b5a0 msgr2=0x7fb97806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.142+0000 7fb99ca00640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb97806b5a0 0x7fb97806da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb984034000 tx=0x7fb98402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:25.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.142+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9980d0f70 msgr2=0x7fb998154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.142+0000 7fb99ca00640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9980d0f70 0x7fb998154d00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb99000c780 tx=0x7fb99000cc40 comp rx=0 tx=0).stop 2024-04-25T09:03:25.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.143+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 shutdown_connections 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.143+0000 7fb99ca00640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb97806b5a0 0x7fb97806da60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.143+0000 7fb99ca00640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9980d0f70 0x7fb998154d00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.143+0000 7fb99ca00640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb99815dc40 0x7fb9980b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.143+0000 7fb99ca00640 1 --2- 172.21.15.62:0/2226313154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9980d0ae0 0x7fb9981f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.143+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 >> 172.21.15.62:0/2226313154 conn(0x7fb99806b000 msgr2=0x7fb9980c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.143+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 shutdown_connections 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.143+0000 7fb99ca00640 1 -- 172.21.15.62:0/2226313154 wait complete. 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febf006b520 msgr2=0x7febf006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febf006b520 0x7febf006d9e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fec0c034830 tx=0x7fec04008040 comp rx=0 tx=0).stop 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0c007460 msgr2=0x7fec0c048fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0c007460 0x7fec0c048fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febfc0098c0 tx=0x7febfc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 shutdown_connections 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febf006b520 0x7febf006d9e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec0c054390 0x7fec0c04d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec0c048100 0x7fec0c04b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 --2- 172.21.15.62:0/1448018570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec0c007460 0x7fec0c048fa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 >> 172.21.15.62:0/1448018570 conn(0x7fec0c009fe0 msgr2=0x7fec0c0e5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.144+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 shutdown_connections 2024-04-25T09:03:25.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.145+0000 7fec18c00640 1 -- 172.21.15.62:0/1448018570 wait complete. 2024-04-25T09:03:25.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.153+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4ce804c7e0 con 0x7f4cd4004160 2024-04-25T09:03:25.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.155+0000 7f4ce6000640 1 -- 172.21.15.62:0/3147436857 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4cdc059640 con 0x7f4cd4004160 2024-04-25T09:03:25.150 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-25T09:03:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.171+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd006b550 msgr2=0x7f4cd006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.171+0000 7f4cf3200640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd006b550 0x7f4cd006da10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4cec073a80 tx=0x7f4cec0698e0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.171+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd4004160 msgr2=0x7f4ce817e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.171+0000 7f4cf3200640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd4004160 0x7f4ce817e200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00d800 tx=0x7f4cdc00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.172+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1b280729c0 con 0x7f1b2815dc40 2024-04-25T09:03:25.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.172+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 shutdown_connections 2024-04-25T09:03:25.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.173+0000 7f4cf3200640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd006b550 0x7f4cd006da10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.173+0000 7f4cf3200640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce8009d50 0x7f4ce800a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.173+0000 7f4cf3200640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ce800bc20 0x7f4ce800c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.173+0000 7f4cf3200640 1 --2- 172.21.15.62:0/3147436857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cd4004160 0x7f4ce817e200 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.173+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 >> 172.21.15.62:0/3147436857 conn(0x7f4ce80559e0 msgr2=0x7f4ce80e1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-25T09:03:25.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.173+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 shutdown_connections 2024-04-25T09:03:25.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.173+0000 7f4cf3200640 1 -- 172.21.15.62:0/3147436857 wait complete. 2024-04-25T09:03:25.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.173+0000 7f1b1f400640 1 -- 172.21.15.62:0/1786215957 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b20059380 con 0x7f1b2815dc40 2024-04-25T09:03:25.163 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-04-25T09:03:25.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.175+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f51100057e0 con 0x7f511004a0a0 2024-04-25T09:03:25.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.176+0000 7f510be00640 1 -- 172.21.15.62:0/3783589285 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51180f3f20 con 0x7f511004a0a0 2024-04-25T09:03:25.164 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-04-25T09:03:25.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.176+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b5a0 msgr2=0x7f1b0406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.176+0000 7f1b2e800640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b5a0 0x7f1b0406da60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009990 tx=0x7f1b10009110 comp rx=0 tx=0).stop 2024-04-25T09:03:25.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.176+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b2815dc40 msgr2=0x7f1b280cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.176+0000 7f1b2e800640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b2815dc40 0x7f1b280cb660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000d530 tx=0x7f1b2000d9f0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.177+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 shutdown_connections 2024-04-25T09:03:25.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.177+0000 7f1b2e800640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b5a0 0x7f1b0406da60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.177+0000 7f1b2e800640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b2815dc40 0x7f1b280cb660 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.177+0000 7f1b2e800640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280d0ae0 0x7f1b28160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.177+0000 7f1b2e800640 1 --2- 172.21.15.62:0/1786215957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b6cf0 0x7f1b28160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.177+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 >> 172.21.15.62:0/1786215957 conn(0x7f1b2806b000 msgr2=0x7f1b280d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-04-25T09:03:25.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.177+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 shutdown_connections 2024-04-25T09:03:25.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.177+0000 7f1b2e800640 1 -- 172.21.15.62:0/1786215957 wait complete. 2024-04-25T09:03:25.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50f806b5a0 msgr2=0x7f50f806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50f806b5a0 0x7f50f806da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f510c006fd0 tx=0x7f510c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:25.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f511004a0a0 msgr2=0x7f51100dde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f11cc0d1f60 con 0x7f11cc15dc40 2024-04-25T09:03:25.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f511004a0a0 0x7f51100dde20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f511807a660 tx=0x7f51180a7000 comp rx=0 tx=0).stop 2024-04-25T09:03:25.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 shutdown_connections 2024-04-25T09:03:25.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50f806b5a0 0x7f50f806da60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5110054de0 0x7f511004a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.180+0000 7f11c8c00640 1 -- 172.21.15.62:0/930925298 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11c4040030 con 0x7f11cc15dc40 2024-04-25T09:03:25.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51100e7a50 0x7f511000a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.169 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-04-25T09:03:25.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 --2- 172.21.15.62:0/3783589285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f511004a0a0 0x7f51100dde20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 >> 172.21.15.62:0/3783589285 conn(0x7f5110056cd0 msgr2=0x7f5110008e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 shutdown_connections 2024-04-25T09:03:25.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.179+0000 7f511ee00640 1 -- 172.21.15.62:0/3783589285 wait complete. 2024-04-25T09:03:25.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.182+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11b006b4d0 msgr2=0x7f11b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.182+0000 7f11d3c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11b006b4d0 0x7f11b006d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f11c0005880 tx=0x7f11c0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:25.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.182+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15dc40 msgr2=0x7f11cc15bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.182+0000 7f11d3c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15dc40 0x7f11cc15bec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11c402ad00 tx=0x7f11c4004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 shutdown_connections 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f11d3c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f11d3c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11cc15dc40 0x7f11cc15bec0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f11d3c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11cc0d0f70 0x7f11cc14f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f11d3c00640 1 --2- 172.21.15.62:0/930925298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f11cc0d0ae0 0x7f11cc1e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 >> 172.21.15.62:0/930925298 conn(0x7f11cc06b000 msgr2=0x7f11cc0c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 shutdown_connections 2024-04-25T09:03:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f11d3c00640 1 -- 172.21.15.62:0/930925298 wait complete. 2024-04-25T09:03:25.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.183+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f947804f000 con 0x7f94780b6cf0 2024-04-25T09:03:25.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.184+0000 7f9472000640 1 -- 172.21.15.62:0/2089191821 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f947405d3b0 con 0x7f94780b6cf0 2024-04-25T09:03:25.173 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2024-04-25T09:03:25.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f945c06b4d0 msgr2=0x7f945c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f945c06b4d0 0x7f945c06d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f946c034000 tx=0x7f946c02e000 comp rx=0 tx=0).stop 2024-04-25T09:03:25.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94780b6cf0 msgr2=0x7f9478156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94780b6cf0 0x7f9478156960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f947400c730 tx=0x7f947400cbf0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 shutdown_connections 2024-04-25T09:03:25.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f945c06b4d0 0x7f945c06d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f947815dc40 0x7f9478153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f94780d0ae0 0x7f947815f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 --2- 172.21.15.62:0/2089191821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94780b6cf0 0x7f9478156960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 >> 172.21.15.62:0/2089191821 conn(0x7f947806b000 msgr2=0x7f94780b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 shutdown_connections 2024-04-25T09:03:25.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.187+0000 7f9480800640 1 -- 172.21.15.62:0/2089191821 wait complete. 2024-04-25T09:03:25.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.194+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f826004f000 con 0x7f826015dc40 2024-04-25T09:03:25.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.195+0000 7f8256a00640 1 -- 172.21.15.62:0/2632763809 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8248079d10 con 0x7f826015dc40 2024-04-25T09:03:25.183 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-04-25T09:03:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f823c06b500 msgr2=0x7f823c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f823c06b500 0x7f823c06d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8258005880 tx=0x7f8258009040 comp rx=0 tx=0).stop 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826015dc40 msgr2=0x7f82601f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826015dc40 0x7f82601f7450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f82480098c0 tx=0x7f82480046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 shutdown_connections 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f823c06b500 0x7f823c06d9c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f826015dc40 0x7f82601f7450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82600d0f70 0x7f82601605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 --2- 172.21.15.62:0/2632763809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82600d0ae0 0x7f8260160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 >> 172.21.15.62:0/2632763809 conn(0x7f826006b000 msgr2=0x7f82600c7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.198+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 shutdown_connections 2024-04-25T09:03:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:25.199+0000 7f8265a00640 1 -- 172.21.15.62:0/2632763809 wait complete. 2024-04-25T09:03:25.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2024-04-25T09:03:25.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-04-25T09:03:25.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-25T09:03:25.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-04-25T09:03:25.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-04-25T09:03:25.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-04-25T09:03:26.163 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T09:03:26.167 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T09:03:26.191 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T09:03:26.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T09:03:26.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T09:03:26.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T09:03:26.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T09:03:26.211 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T09:03:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.471+0000 7fe137800640 1 Processor -- start 2024-04-25T09:03:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.472+0000 7fe137800640 1 -- start start 2024-04-25T09:03:26.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.474+0000 7fe137800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe1280e30c0 0x7fe128057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.475+0000 7fe137800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12803e840 0x7fe128053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.476+0000 7fe137800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1280582d0 con 0x7fe128054130 2024-04-25T09:03:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.476+0000 7fe137800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe128058570 con 0x7fe1280e30c0 2024-04-25T09:03:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.476+0000 7fe137800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe128058850 con 0x7fe12803e840 2024-04-25T09:03:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.476+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12803e840 0x7fe128053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.477+0000 7fe12fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12803e840 0x7fe128053bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:48632/0 (socket says 172.21.15.62:48632) 2024-04-25T09:03:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.477+0000 7fe12fe00640 1 -- 172.21.15.62:0/4283458429 learned_addr learned my addr 172.21.15.62:0/4283458429 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.478+0000 7fe12fe00640 1 -- 172.21.15.62:0/4283458429 >> v2:172.21.15.62:3300/0 conn(0x7fe1280e30c0 msgr2=0x7fe128057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.478+0000 7fe12fe00640 1 --2- 172.21.15.62:0/4283458429 >> v2:172.21.15.62:3300/0 conn(0x7fe1280e30c0 0x7fe128057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.478+0000 7fe12fe00640 1 -- 172.21.15.62:0/4283458429 >> v1:172.21.15.62:6789/0 conn(0x7fe128054130 legacy=0x7fe1280e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.478+0000 7fe12fe00640 1 -- 172.21.15.62:0/4283458429 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128058b30 con 0x7fe12803e840 2024-04-25T09:03:26.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.481+0000 7fe12fe00640 1 --2- 172.21.15.62:0/4283458429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12803e840 0x7fe128053bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe124009870 tx=0x7fe12402a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434f4e3389f24107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.481+0000 7fe12f400640 1 -- 172.21.15.62:0/4283458429 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124037070 con 0x7fe12803e840 2024-04-25T09:03:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.483+0000 7fe12f400640 1 -- 172.21.15.62:0/4283458429 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe124032440 con 0x7fe12803e840 2024-04-25T09:03:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.483+0000 7fe12f400640 1 -- 172.21.15.62:0/4283458429 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124032850 con 0x7fe12803e840 2024-04-25T09:03:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.484+0000 7fe137800640 1 -- 172.21.15.62:0/4283458429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12803e840 msgr2=0x7fe128053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.484+0000 7fe137800640 1 --2- 172.21.15.62:0/4283458429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12803e840 0x7fe128053bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe124009870 tx=0x7fe12402a2c0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.484+0000 7fe137800640 1 -- 172.21.15.62:0/4283458429 shutdown_connections 2024-04-25T09:03:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.484+0000 7fe137800640 1 --2- 172.21.15.62:0/4283458429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe12803e840 0x7fe128053bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.484+0000 7fe137800640 1 --2- 172.21.15.62:0/4283458429 >> v2:172.21.15.62:3300/0 conn(0x7fe1280e30c0 0x7fe128057d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.484+0000 7fe137800640 1 -- 172.21.15.62:0/4283458429 >> 172.21.15.62:0/4283458429 conn(0x7fe1280559e0 msgr2=0x7fe128055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.484+0000 7fe137800640 1 -- 172.21.15.62:0/4283458429 shutdown_connections 2024-04-25T09:03:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.484+0000 7fe137800640 1 -- 172.21.15.62:0/4283458429 wait complete. 2024-04-25T09:03:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.487+0000 7fe137800640 1 Processor -- start 2024-04-25T09:03:26.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.488+0000 7fe137800640 1 -- start start 2024-04-25T09:03:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.490+0000 7fe137800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128054130 0x7fe12817cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.490+0000 7fe137800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1280e30c0 0x7fe128039990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.490+0000 7fe137800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe128039ed0 0x7fe1280d1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.491+0000 7fe134e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128054130 0x7fe12817cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.491+0000 7fe134e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128054130 0x7fe12817cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:48638/0 (socket says 172.21.15.62:48638) 2024-04-25T09:03:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.492+0000 7fe135800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe128039ed0 0x7fe1280d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.492+0000 7fe135800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe128039ed0 0x7fe1280d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54920/0 (socket says 172.21.15.62:54920) 2024-04-25T09:03:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.492+0000 7fe134e00640 1 -- 172.21.15.62:0/2333799108 learned_addr learned my addr 172.21.15.62:0/2333799108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.492+0000 7fe137800640 1 -- 172.21.15.62:0/2333799108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280e34a0 con 0x7fe128039ed0 2024-04-25T09:03:26.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.492+0000 7fe137800640 1 -- 172.21.15.62:0/2333799108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12803bfc0 con 0x7fe1280e30c0 2024-04-25T09:03:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.492+0000 7fe12fe00640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1280e30c0 0x7fe128039990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.493+0000 7fe137800640 1 -- 172.21.15.62:0/2333799108 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280078e0 con 0x7fe128054130 2024-04-25T09:03:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.494+0000 7fe135800640 1 -- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128054130 msgr2=0x7fe12817cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.494+0000 7fe135800640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128054130 0x7fe12817cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.494+0000 7fe135800640 1 -- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1280e30c0 msgr2=0x7fe128039990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.494+0000 7fe135800640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1280e30c0 0x7fe128039990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.494+0000 7fe135800640 1 -- 172.21.15.62:0/2333799108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280dd2d0 con 0x7fe128039ed0 2024-04-25T09:03:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.494+0000 7fe12fe00640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1280e30c0 0x7fe128039990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.495+0000 7fe135800640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe128039ed0 0x7fe1280d1670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1300697c0 tx=0x7fe130068e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.495+0000 7fe12cc00640 1 -- 172.21.15.62:0/2333799108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130090070 con 0x7fe128039ed0 2024-04-25T09:03:26.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.496+0000 7fe137800640 1 -- 172.21.15.62:0/2333799108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1280dd5b0 con 0x7fe128039ed0 2024-04-25T09:03:26.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.496+0000 7fe137800640 1 -- 172.21.15.62:0/2333799108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1280ddb70 con 0x7fe128039ed0 2024-04-25T09:03:26.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.496+0000 7fe12cc00640 1 -- 172.21.15.62:0/2333799108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe130068030 con 0x7fe128039ed0 2024-04-25T09:03:26.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.496+0000 7fe12cc00640 1 -- 172.21.15.62:0/2333799108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130090070 con 0x7fe128039ed0 2024-04-25T09:03:26.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.500+0000 7fe137800640 1 -- 172.21.15.62:0/2333799108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10c005290 con 0x7fe128039ed0 2024-04-25T09:03:26.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.505+0000 7fe12cc00640 1 -- 172.21.15.62:0/2333799108 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe130093070 con 0x7fe128039ed0 2024-04-25T09:03:26.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.510+0000 7fe12cc00640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe11006b6d0 0x7fe11006db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.510+0000 7fe12cc00640 1 -- 172.21.15.62:0/2333799108 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe130104850 con 0x7fe128039ed0 2024-04-25T09:03:26.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.511+0000 7fe134e00640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe11006b6d0 0x7fe11006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.515+0000 7fabeec00640 1 Processor -- start 2024-04-25T09:03:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.515+0000 7fabeec00640 1 -- start start 2024-04-25T09:03:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.515+0000 7fe12cc00640 1 -- 172.21.15.62:0/2333799108 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1300d36c0 con 0x7fe128039ed0 2024-04-25T09:03:26.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.516+0000 7fe134e00640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe11006b6d0 0x7fe11006db90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009950 tx=0x7fe11c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.518+0000 7fabeec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe0054090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.518+0000 7fabeec00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabe004fce0 0x7fabe0050170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.519+0000 7fabe7e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fabe00df400 0x7fabe00073f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35780/0 (socket says 172.21.15.62:35780) 2024-04-25T09:03:26.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.519+0000 7fabe7e00640 1 -- 172.21.15.62:0/3660794396 learned_addr learned my addr 172.21.15.62:0/3660794396 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:26.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.519+0000 7fabed600640 1 --2- 172.21.15.62:0/3660794396 >> v2:172.21.15.62:3300/0 conn(0x7fabe004fce0 0x7fabe0050170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.520+0000 7fabeec00640 1 -- 172.21.15.62:0/3660794396 --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe0006070 con 0x7fabe00df400 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.520+0000 7fabeec00640 1 -- 172.21.15.62:0/3660794396 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe0006350 con 0x7fabe004fce0 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.520+0000 7fabeec00640 1 -- 172.21.15.62:0/3660794396 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe0006630 con 0x7fabe0053230 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.520+0000 7f2c6ac00640 1 Processor -- start 2024-04-25T09:03:26.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.521+0000 7f2c6ac00640 1 -- start start 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.522+0000 7fabe7400640 1 --2- 172.21.15.62:0/3660794396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe0054090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.522+0000 7fabed600640 1 -- 172.21.15.62:0/3660794396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 msgr2=0x7fabe0054090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.522+0000 7fabed600640 1 --2- 172.21.15.62:0/3660794396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe0054090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.522+0000 7fabed600640 1 -- 172.21.15.62:0/3660794396 >> v1:172.21.15.121:6789/0 conn(0x7fabe00df400 legacy=0x7fabe00073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.522+0000 7fabed600640 1 -- 172.21.15.62:0/3660794396 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe0006910 con 0x7fabe004fce0 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.523+0000 7f2c6ac00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c6415dc40 0x7f2c640d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.523+0000 7f2c6ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d0ae0 0x7f2c640d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.523+0000 7fabed600640 1 --2- 172.21.15.62:0/3660794396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe004fce0 0x7fabe0050170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fabe8062780 tx=0x7fabe80a5470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e17912cc21a5e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.523+0000 7f2c6ac00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c640d2620 con 0x7f2c640d3ae0 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.523+0000 7f2c6ac00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c640d1480 con 0x7f2c6415dc40 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.523+0000 7f2c6ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640d2900 con 0x7f2c640d0ae0 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.524+0000 7f2c68c00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2c640d3ae0 0x7f2c640d3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35792/0 (socket says 172.21.15.62:35792) 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.524+0000 7f2c68c00640 1 -- 172.21.15.62:0/1308052356 learned_addr learned my addr 172.21.15.62:0/1308052356 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.524+0000 7f2c63e00640 1 --2- 172.21.15.62:0/1308052356 >> v2:172.21.15.121:3300/0 conn(0x7f2c6415dc40 0x7f2c640d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.525+0000 7fabe6a00640 1 -- 172.21.15.62:0/3660794396 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80aa7d0 con 0x7fabe004fce0 2024-04-25T09:03:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.525+0000 7fabe6a00640 1 -- 172.21.15.62:0/3660794396 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe80aa9b0 con 0x7fabe004fce0 2024-04-25T09:03:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.525+0000 7f2c63400640 1 --2- 172.21.15.62:0/1308052356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d0ae0 0x7f2c640d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.525+0000 7fabe6a00640 1 -- 172.21.15.62:0/3660794396 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80aae20 con 0x7fabe004fce0 2024-04-25T09:03:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fabeec00640 1 -- 172.21.15.62:0/3660794396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe004fce0 msgr2=0x7fabe0050170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.525+0000 7f2c63e00640 1 -- 172.21.15.62:0/1308052356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d0ae0 msgr2=0x7f2c640d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.525+0000 7f2c63e00640 1 --2- 172.21.15.62:0/1308052356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d0ae0 0x7f2c640d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.525+0000 7f2c63e00640 1 -- 172.21.15.62:0/1308052356 >> v1:172.21.15.121:6789/0 conn(0x7f2c640d3ae0 legacy=0x7f2c640d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.525+0000 7f2c63e00640 1 -- 172.21.15.62:0/1308052356 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640d2be0 con 0x7f2c6415dc40 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fabeec00640 1 --2- 172.21.15.62:0/3660794396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe004fce0 0x7fabe0050170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fabe8062780 tx=0x7fabe80a5470 comp rx=0 tx=0).stop 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fabeec00640 1 -- 172.21.15.62:0/3660794396 shutdown_connections 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fabeec00640 1 --2- 172.21.15.62:0/3660794396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe004fce0 0x7fabe0050170 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fabeec00640 1 --2- 172.21.15.62:0/3660794396 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe0054090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fabeec00640 1 -- 172.21.15.62:0/3660794396 >> 172.21.15.62:0/3660794396 conn(0x7fabe0054ca0 msgr2=0x7fabe00550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fabeec00640 1 -- 172.21.15.62:0/3660794396 shutdown_connections 2024-04-25T09:03:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fabeec00640 1 -- 172.21.15.62:0/3660794396 wait complete. 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fae41800640 1 Processor -- start 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.526+0000 7fae41800640 1 -- start start 2024-04-25T09:03:26.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.528+0000 7f7875000640 1 Processor -- start 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.528+0000 7fae41800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae340e30f0 0x7fae34057dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.528+0000 7fae41800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fae34054330 0x7fae34006900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.528+0000 7fae41800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae34058300 con 0x7fae3403e870 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.528+0000 7fae41800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae340585a0 con 0x7fae34054330 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.528+0000 7fae41800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae34058880 con 0x7fae340e30f0 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.528+0000 7f7875000640 1 -- start start 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.528+0000 7f2c63e00640 1 --2- 172.21.15.62:0/1308052356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c6415dc40 0x7f2c640d1cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c500097a0 tx=0x7f2c5002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d143ef96c140edcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.529+0000 7fae3a000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fae3403e870 0x7fae34053c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35796/0 (socket says 172.21.15.62:35796) 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.529+0000 7fae3a000640 1 -- 172.21.15.62:0/126043133 learned_addr learned my addr 172.21.15.62:0/126043133 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.529+0000 7fae3aa00640 1 --2- 172.21.15.62:0/126043133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae340e30f0 0x7fae34057dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.529+0000 7fae3b400640 1 --2- 172.21.15.62:0/126043133 >> v2:172.21.15.62:3300/0 conn(0x7fae34054330 0x7fae34006900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7fae3b400640 1 -- 172.21.15.62:0/126043133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae340e30f0 msgr2=0x7fae34057dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7fae3b400640 1 --2- 172.21.15.62:0/126043133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae340e30f0 0x7fae34057dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7fae3b400640 1 -- 172.21.15.62:0/126043133 >> v1:172.21.15.121:6789/0 conn(0x7fae3403e870 legacy=0x7fae34053c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7fae3b400640 1 -- 172.21.15.62:0/126043133 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34058b60 con 0x7fae34054330 2024-04-25T09:03:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7f7875000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78700d1320 0x7f78700b4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7f7875000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f787015af30 0x7f78700b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7f7875000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78701592b0 con 0x7f78700d0f40 2024-04-25T09:03:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7f7875000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787015e550 con 0x7f787015af30 2024-04-25T09:03:26.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7f7875000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787015b4b0 con 0x7f78700d1320 2024-04-25T09:03:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.530+0000 7fabeec00640 1 Processor -- start 2024-04-25T09:03:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.531+0000 7f786f400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f78700d0f40 0x7f7870158ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35808/0 (socket says 172.21.15.62:35808) 2024-04-25T09:03:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.531+0000 7f786f400640 1 -- 172.21.15.62:0/1930184362 learned_addr learned my addr 172.21.15.62:0/1930184362 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.531+0000 7f2c62a00640 1 -- 172.21.15.62:0/1308052356 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50037070 con 0x7f2c6415dc40 2024-04-25T09:03:26.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.531+0000 7f2c62a00640 1 -- 172.21.15.62:0/1308052356 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c50032440 con 0x7f2c6415dc40 2024-04-25T09:03:26.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.531+0000 7fabeec00640 1 -- start start 2024-04-25T09:03:26.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.531+0000 7f2c62a00640 1 -- 172.21.15.62:0/1308052356 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50037070 con 0x7f2c6415dc40 2024-04-25T09:03:26.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.531+0000 7f786ea00640 1 --2- 172.21.15.62:0/1930184362 >> v2:172.21.15.121:3300/0 conn(0x7f78700d1320 0x7f78700b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.531+0000 7fae3b400640 1 --2- 172.21.15.62:0/126043133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34054330 0x7fae34006900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0697c0 tx=0x7fae3c0a5480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e10024c41daf22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f786e000640 1 -- 172.21.15.62:0/1930184362 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743798403 0 0) 0x7f78701592b0 con 0x7f78700d0f40 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f786e000640 1 -- 172.21.15.62:0/1930184362 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7854003520 con 0x7f78700d0f40 2024-04-25T09:03:26.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7fae39600640 1 -- 172.21.15.62:0/126043133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0aa7d0 con 0x7fae34054330 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f786fe00640 1 --2- 172.21.15.62:0/1930184362 >> v2:172.21.15.62:3300/0 conn(0x7f787015af30 0x7f78700b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f786ea00640 1 -- 172.21.15.62:0/1930184362 >> v2:172.21.15.62:3300/0 conn(0x7f787015af30 msgr2=0x7f78700b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f786ea00640 1 --2- 172.21.15.62:0/1930184362 >> v2:172.21.15.62:3300/0 conn(0x7f787015af30 0x7f78700b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f786ea00640 1 -- 172.21.15.62:0/1930184362 >> v1:172.21.15.121:6789/0 conn(0x7f78700d0f40 legacy=0x7f7870158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f786ea00640 1 -- 172.21.15.62:0/1930184362 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787015b790 con 0x7f78700d1320 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f2c6ac00640 1 -- 172.21.15.62:0/1308052356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c6415dc40 msgr2=0x7f2c640d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/1308052356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c6415dc40 0x7f2c640d1cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c500097a0 tx=0x7f2c5002a180 comp rx=0 tx=0).stop 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7fae39600640 1 -- 172.21.15.62:0/126043133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c0aa970 con 0x7fae34054330 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7f2c6ac00640 1 -- 172.21.15.62:0/1308052356 shutdown_connections 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/1308052356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d0ae0 0x7f2c640d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7fabeec00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe004fce0 0x7fabe0173340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7fabeec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe004dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/1308052356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c6415dc40 0x7f2c640d1cb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7f2c6ac00640 1 -- 172.21.15.62:0/1308052356 >> 172.21.15.62:0/1308052356 conn(0x7f2c6406b000 msgr2=0x7f2c6406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.532+0000 7fabeec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe00df400 0x7fabe004e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fabeec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00557c0 con 0x7fabe00df400 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fabeec00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe0006ed0 con 0x7fabe004fce0 2024-04-25T09:03:26.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fabeec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe0055940 con 0x7fabe0053230 2024-04-25T09:03:26.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7f2c6ac00640 1 -- 172.21.15.62:0/1308052356 shutdown_connections 2024-04-25T09:03:26.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7f2c6ac00640 1 -- 172.21.15.62:0/1308052356 wait complete. 2024-04-25T09:03:26.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fabe7400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe004dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fabed600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe00df400 0x7fabe004e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fabe7400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe004dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:48676/0 (socket says 172.21.15.62:48676) 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fabe7400640 1 -- 172.21.15.62:0/3892391622 learned_addr learned my addr 172.21.15.62:0/3892391622 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fabe7e00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe004fce0 0x7fabe0173340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fae41800640 1 -- 172.21.15.62:0/126043133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34054330 msgr2=0x7fae34006900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fae39600640 1 -- 172.21.15.62:0/126043133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0aade0 con 0x7fae34054330 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.533+0000 7fae41800640 1 --2- 172.21.15.62:0/126043133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34054330 0x7fae34006900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0697c0 tx=0x7fae3c0a5480 comp rx=0 tx=0).stop 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fabe7400640 1 -- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe004fce0 msgr2=0x7fabe0173340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fabe7400640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe004fce0 0x7fabe0173340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fabe7400640 1 -- 172.21.15.62:0/3892391622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe00df400 msgr2=0x7fabe004e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fabe7400640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe00df400 0x7fabe004e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fabe7400640 1 -- 172.21.15.62:0/3892391622 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe017e330 con 0x7fabe0053230 2024-04-25T09:03:26.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7f786ea00640 1 --2- 172.21.15.62:0/1930184362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d1320 0x7f78700b4380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f787006f3c0 tx=0x7f786002a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4da0ec3fc96d749 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fabed600640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe00df400 0x7fabe004e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:26.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.535+0000 7f786e000640 1 -- 172.21.15.62:0/1930184362 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860007740 con 0x7f78700d1320 2024-04-25T09:03:26.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.535+0000 7f786e000640 1 -- 172.21.15.62:0/1930184362 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78600078e0 con 0x7f78700d1320 2024-04-25T09:03:26.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.535+0000 7fabe7400640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe004dd80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00d5c0 tx=0x7fabdc00da80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fae41800640 1 -- 172.21.15.62:0/126043133 shutdown_connections 2024-04-25T09:03:26.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fae41800640 1 --2- 172.21.15.62:0/126043133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae34054330 0x7fae34006900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fae41800640 1 --2- 172.21.15.62:0/126043133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae340e30f0 0x7fae34057dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fae41800640 1 -- 172.21.15.62:0/126043133 >> 172.21.15.62:0/126043133 conn(0x7fae34055a10 msgr2=0x7fae34055e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.534+0000 7fae41800640 1 -- 172.21.15.62:0/126043133 shutdown_connections 2024-04-25T09:03:26.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.535+0000 7fae41800640 1 -- 172.21.15.62:0/126043133 wait complete. 2024-04-25T09:03:26.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7f2c6ac00640 1 Processor -- start 2024-04-25T09:03:26.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7fabd7e00640 1 -- 172.21.15.62:0/3892391622 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0125f0 con 0x7fabe0053230 2024-04-25T09:03:26.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7f7875000640 1 -- 172.21.15.62:0/1930184362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d1320 msgr2=0x7f78700b4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7f7875000640 1 --2- 172.21.15.62:0/1930184362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d1320 0x7f78700b4380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f787006f3c0 tx=0x7f786002a6d0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7f2c6ac00640 1 -- start start 2024-04-25T09:03:26.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7f7875000640 1 -- 172.21.15.62:0/1930184362 shutdown_connections 2024-04-25T09:03:26.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7f7875000640 1 --2- 172.21.15.62:0/1930184362 >> v2:172.21.15.62:3300/0 conn(0x7f787015af30 0x7f78700b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7f7875000640 1 --2- 172.21.15.62:0/1930184362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d1320 0x7f78700b4380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7f7875000640 1 -- 172.21.15.62:0/1930184362 >> 172.21.15.62:0/1930184362 conn(0x7f787006abe0 msgr2=0x7f787006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7f7875000640 1 -- 172.21.15.62:0/1930184362 shutdown_connections 2024-04-25T09:03:26.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7fae41800640 1 Processor -- start 2024-04-25T09:03:26.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7fabd7e00640 1 -- 172.21.15.62:0/3892391622 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc019070 con 0x7fabe0053230 2024-04-25T09:03:26.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.536+0000 7fabd7e00640 1 -- 172.21.15.62:0/3892391622 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0129d0 con 0x7fabe0053230 2024-04-25T09:03:26.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7f2c6ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640d0ae0 0x7f2c640cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7f2c6ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d3ae0 0x7f2c640c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7f2c6ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6415dc40 0x7f2c64154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.538+0000 7f2c6ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640b7d00 con 0x7f2c6415dc40 2024-04-25T09:03:26.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.538+0000 7f2c6ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640d31a0 con 0x7f2c640d0ae0 2024-04-25T09:03:26.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7fae41800640 1 -- start start 2024-04-25T09:03:26.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.538+0000 7f2c6ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c640d36c0 con 0x7f2c640d3ae0 2024-04-25T09:03:26.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe017e610 con 0x7fabe0053230 2024-04-25T09:03:26.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe017ebd0 con 0x7fabe0053230 2024-04-25T09:03:26.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.538+0000 7f2c68c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6415dc40 0x7f2c64154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.538+0000 7f551b400640 1 Processor -- start 2024-04-25T09:03:26.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7f551b400640 1 -- start start 2024-04-25T09:03:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.538+0000 7f2c68c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6415dc40 0x7f2c64154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54976/0 (socket says 172.21.15.62:54976) 2024-04-25T09:03:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7f2c68c00640 1 -- 172.21.15.62:0/2845153402 learned_addr learned my addr 172.21.15.62:0/2845153402 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7fae41800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3403e870 0x7fae340dc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7fae41800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae34054330 0x7fae34174510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7fae41800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae340e30f0 0x7fae34174a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7fae41800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae340e34d0 con 0x7fae3403e870 2024-04-25T09:03:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7fae41800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3403bff0 con 0x7fae340e30f0 2024-04-25T09:03:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7fae41800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae34006e40 con 0x7fae34054330 2024-04-25T09:03:26.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7fd5a2c00640 1 Processor -- start 2024-04-25T09:03:26.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7f2c63400640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d3ae0 0x7f2c640c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.537+0000 7f7875000640 1 -- 172.21.15.62:0/1930184362 wait complete. 2024-04-25T09:03:26.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe0008bb0 con 0x7fabe0053230 2024-04-25T09:03:26.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.539+0000 7f7875000640 1 Processor -- start 2024-04-25T09:03:26.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7fd5a2c00640 1 -- start start 2024-04-25T09:03:26.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7f7875000640 1 -- start start 2024-04-25T09:03:26.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7fae3aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3403e870 0x7fae340dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7f2c68c00640 1 -- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d3ae0 msgr2=0x7f2c640c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7f2c68c00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d3ae0 0x7f2c640c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7fae3aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3403e870 0x7fae340dc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54978/0 (socket says 172.21.15.62:54978) 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7fae3aa00640 1 -- 172.21.15.62:0/1252484083 learned_addr learned my addr 172.21.15.62:0/1252484083 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7f2c68c00640 1 -- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640d0ae0 msgr2=0x7f2c640cd2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7f2c68c00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640d0ae0 0x7f2c640cd2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7f2c68c00640 1 -- 172.21.15.62:0/2845153402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64150590 con 0x7f2c6415dc40 2024-04-25T09:03:26.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7fae3b400640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae340e30f0 0x7fae34174a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.540+0000 7f2c63400640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d3ae0 0x7f2c640c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7f2c68c00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6415dc40 0x7f2c64154140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00cbc0 tx=0x7f2c5c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7fae3aa00640 1 -- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae34054330 msgr2=0x7fae34174510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7fae3aa00640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae34054330 0x7fae34174510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7fae3aa00640 1 -- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae340e30f0 msgr2=0x7fae34174a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7fae3aa00640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae340e30f0 0x7fae34174a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7fae3aa00640 1 -- 172.21.15.62:0/1252484083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae340da0e0 con 0x7fae3403e870 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7f551b400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f55140cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7f551b400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55140aeed0 0x7f55140cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7f551b400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551406ca70 con 0x7f55140cccf0 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7f551b400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551406d660 con 0x7f55140aeed0 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7f551b400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551415c0f0 con 0x7f551415bd10 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7fc8b6200640 1 Processor -- start 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7fae3aa00640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3403e870 0x7fae340dc330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e870 tx=0x7fae2c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7f2c5be00640 1 -- 172.21.15.62:0/2845153402 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c017070 con 0x7f2c6415dc40 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7f2c5be00640 1 -- 172.21.15.62:0/2845153402 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c00ad40 con 0x7f2c6415dc40 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fd5a2c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd59c072180 0x7fd59c159c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fd5a2c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59c0cf240 con 0x7fd59c0ceda0 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fd5a2c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59c0b2900 con 0x7fd59c0b33e0 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fd5a2c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c0b3870 con 0x7fd59c072180 2024-04-25T09:03:26.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7f2c5be00640 1 -- 172.21.15.62:0/2845153402 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c012600 con 0x7f2c6415dc40 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fae32a00640 1 -- 172.21.15.62:0/1252484083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c017070 con 0x7fae3403e870 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fae32a00640 1 -- 172.21.15.62:0/1252484083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c00c940 con 0x7fae3403e870 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fd5a1800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd59c0ceda0 0x7fd59c15e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57586/0 (socket says 172.21.15.62:57586) 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fd5a1800640 1 -- 172.21.15.62:0/2565165734 learned_addr learned my addr 172.21.15.62:0/2565165734 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:26.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fae32a00640 1 -- 172.21.15.62:0/1252484083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00cd30 con 0x7fae3403e870 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae340da3c0 con 0x7fae3403e870 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae340d9be0 con 0x7fae3403e870 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7fd5a0e00640 1 --2- 172.21.15.62:0/2565165734 >> v2:172.21.15.62:3300/0 conn(0x7fd59c072180 0x7fd59c159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.541+0000 7fc8b6200640 1 -- start start 2024-04-25T09:03:26.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7fd5a0e00640 1 -- 172.21.15.62:0/2565165734 >> v1:172.21.15.121:6789/0 conn(0x7fd59c0b33e0 legacy=0x7fd59c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7fd5a0e00640 1 -- 172.21.15.62:0/2565165734 >> v1:172.21.15.62:6789/0 conn(0x7fd59c0ceda0 legacy=0x7fd59c15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7fd5a0e00640 1 -- 172.21.15.62:0/2565165734 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c0b2be0 con 0x7fd59c072180 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7f5518a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f55140cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7f5518a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f55140cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:48704/0 (socket says 172.21.15.62:48704) 2024-04-25T09:03:26.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.542+0000 7f5518a00640 1 -- 172.21.15.62:0/1126875071 learned_addr learned my addr 172.21.15.62:0/1126875071 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7f5519400640 1 --2- 172.21.15.62:0/1126875071 >> v2:172.21.15.121:3300/0 conn(0x7f55140aeed0 0x7f55140cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7f5518a00640 1 -- 172.21.15.62:0/1126875071 >> v2:172.21.15.121:3300/0 conn(0x7f55140aeed0 msgr2=0x7f55140cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7f5518a00640 1 --2- 172.21.15.62:0/1126875071 >> v2:172.21.15.121:3300/0 conn(0x7f55140aeed0 0x7f55140cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7f5518a00640 1 -- 172.21.15.62:0/1126875071 >> v1:172.21.15.62:6789/0 conn(0x7f55140cccf0 legacy=0x7f55140ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7f5518a00640 1 -- 172.21.15.62:0/1126875071 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551406cd30 con 0x7f551415bd10 2024-04-25T09:03:26.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.544+0000 7f5519400640 1 --2- 172.21.15.62:0/1126875071 >> v2:172.21.15.121:3300/0 conn(0x7f55140aeed0 0x7f55140cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:26.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.544+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c64150870 con 0x7f2c6415dc40 2024-04-25T09:03:26.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.544+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c640c5730 con 0x7f2c6415dc40 2024-04-25T09:03:26.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.544+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae34005810 con 0x7fae3403e870 2024-04-25T09:03:26.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7fc8b6200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc8b0067000 0x7fc8b0140160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.543+0000 7fc8b6200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc8b00714a0 0x7fc8b00cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.545+0000 7f7875000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d0f40 0x7f78700cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.545+0000 7f7875000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78700d1320 0x7f78702044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.545+0000 7fd5a0e00640 1 --2- 172.21.15.62:0/2565165734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c072180 0x7fd59c159c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5880097a0 tx=0x7fd58802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0ce9c4355c98cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.545+0000 7f7875000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f787015af30 0x7f7870148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.545+0000 7fabd7e00640 1 -- 172.21.15.62:0/3892391622 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabdc012b70 con 0x7fabe0053230 2024-04-25T09:03:26.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.545+0000 7fc8b6200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b00d0090 con 0x7fc8b00cfc70 2024-04-25T09:03:26.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.545+0000 7fc8af400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc8b0067000 0x7fc8b0140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.545+0000 7f786f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d0f40 0x7f78700cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f5518a00640 1 --2- 172.21.15.62:0/1126875071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f55140cfca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5500009870 tx=0x7f5500032a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca1836cccf1d810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f786f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d0f40 0x7f78700cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:52108/0 (socket says 172.21.15.62:52108) 2024-04-25T09:03:26.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f786f400640 1 -- 172.21.15.62:0/3077700777 learned_addr learned my addr 172.21.15.62:0/3077700777 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787006d630 con 0x7f78700d1320 2024-04-25T09:03:26.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7fc8afe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc8b00cfc70 0x7fc8b00ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57612/0 (socket says 172.21.15.62:57612) 2024-04-25T09:03:26.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7fc8afe00640 1 -- 172.21.15.62:0/426654286 learned_addr learned my addr 172.21.15.62:0/426654286 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:26.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f786fe00640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f787015af30 0x7f7870148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f786ea00640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78700d1320 0x7f78702044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7fc8aea00640 1 --2- 172.21.15.62:0/426654286 >> v2:172.21.15.62:3300/0 conn(0x7fc8b00714a0 0x7fc8b00cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787015bd50 con 0x7f78700d0f40 2024-04-25T09:03:26.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c640729c0 con 0x7f2c6415dc40 2024-04-25T09:03:26.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787006d7b0 con 0x7f787015af30 2024-04-25T09:03:26.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7fc8b6200640 1 -- 172.21.15.62:0/426654286 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b015da20 con 0x7fc8b00714a0 2024-04-25T09:03:26.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.546+0000 7fc8b6200640 1 -- 172.21.15.62:0/426654286 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b015dd00 con 0x7fc8b0067000 2024-04-25T09:03:26.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.547+0000 7f786ea00640 1 -- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f787015af30 msgr2=0x7f7870148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.547+0000 7fae32a00640 1 -- 172.21.15.62:0/1252484083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae2c01a070 con 0x7fae3403e870 2024-04-25T09:03:26.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.548+0000 7f786ea00640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f787015af30 0x7f7870148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.548+0000 7f786ea00640 1 -- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d0f40 msgr2=0x7f78700cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.548+0000 7f786ea00640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d0f40 0x7f78700cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.548+0000 7f786ea00640 1 -- 172.21.15.62:0/3077700777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7870203fe0 con 0x7f78700d1320 2024-04-25T09:03:26.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.548+0000 7f786f400640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d0f40 0x7f78700cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:26.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.547+0000 7fc8af400640 1 -- 172.21.15.62:0/426654286 >> v2:172.21.15.62:3300/0 conn(0x7fc8b00714a0 msgr2=0x7fc8b00cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.547+0000 7fc8af400640 1 --2- 172.21.15.62:0/426654286 >> v2:172.21.15.62:3300/0 conn(0x7fc8b00714a0 0x7fc8b00cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.547+0000 7fc8af400640 1 -- 172.21.15.62:0/426654286 >> v1:172.21.15.62:6789/0 conn(0x7fc8b00cfc70 legacy=0x7fc8b00ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.547+0000 7fc8af400640 1 -- 172.21.15.62:0/426654286 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b015dfe0 con 0x7fc8b0067000 2024-04-25T09:03:26.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.548+0000 7fabd7e00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabcc06b5a0 0x7fabcc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.549+0000 7f786ea00640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78700d1320 0x7f78702044c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7860031000 tx=0x7f7860004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.549+0000 7fc8af400640 1 --2- 172.21.15.62:0/426654286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 0x7fc8b0140160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00d0730 tx=0x7fc8a802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edb8b823bda32abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.549+0000 7fabd7e00640 1 -- 172.21.15.62:0/3892391622 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabdc08b040 con 0x7fabe0053230 2024-04-25T09:03:26.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.549+0000 7fae32a00640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae1c06b5d0 0x7fae1c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.549+0000 7fae32a00640 1 -- 172.21.15.62:0/1252484083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae2c08a770 con 0x7fae3403e870 2024-04-25T09:03:26.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.549+0000 7fabe7e00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabcc06b5a0 0x7fabcc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.550+0000 7f2c5be00640 1 -- 172.21.15.62:0/2845153402 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c5c01a070 con 0x7f2c6415dc40 2024-04-25T09:03:26.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.549+0000 7fc8ae000640 1 -- 172.21.15.62:0/426654286 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8007740 con 0x7fc8b0067000 2024-04-25T09:03:26.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.550+0000 7fc8ae000640 1 -- 172.21.15.62:0/426654286 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a80078e0 con 0x7fc8b0067000 2024-04-25T09:03:26.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.550+0000 7fc8ae000640 1 -- 172.21.15.62:0/426654286 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8007cb0 con 0x7fc8b0067000 2024-04-25T09:03:26.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.550+0000 7fae3a000640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae1c06b5d0 0x7fae1c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fabe7e00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabcc06b5a0 0x7fabcc06da60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabd8000f60 tx=0x7fabd8005e40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fc8b6200640 1 -- 172.21.15.62:0/426654286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 msgr2=0x7fc8b0140160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fc8b6200640 1 --2- 172.21.15.62:0/426654286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 0x7fc8b0140160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00d0730 tx=0x7fc8a802a710 comp rx=0 tx=0).stop 2024-04-25T09:03:26.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fc8b6200640 1 -- 172.21.15.62:0/426654286 shutdown_connections 2024-04-25T09:03:26.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fc8b6200640 1 --2- 172.21.15.62:0/426654286 >> v2:172.21.15.62:3300/0 conn(0x7fc8b00714a0 0x7fc8b00cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fc8b6200640 1 --2- 172.21.15.62:0/426654286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 0x7fc8b0140160 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fc8b6200640 1 -- 172.21.15.62:0/426654286 >> 172.21.15.62:0/426654286 conn(0x7fc8b0074290 msgr2=0x7fc8b00668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fae3a000640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae1c06b5d0 0x7fae1c06da90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fae340df2c0 tx=0x7fae28007a40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fc8b6200640 1 -- 172.21.15.62:0/426654286 shutdown_connections 2024-04-25T09:03:26.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7f2c5be00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c4406b520 0x7f2c4406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7f2c5be00640 1 -- 172.21.15.62:0/2845153402 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c5c08a3e0 con 0x7f2c6415dc40 2024-04-25T09:03:26.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.552+0000 7f2c63e00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c4406b520 0x7f2c4406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.551+0000 7fc8b6200640 1 -- 172.21.15.62:0/426654286 wait complete. 2024-04-25T09:03:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.552+0000 7fd59be00640 1 -- 172.21.15.62:0/2565165734 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588037070 con 0x7fd59c072180 2024-04-25T09:03:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.552+0000 7f7867400640 1 -- 172.21.15.62:0/3077700777 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860040070 con 0x7f78700d1320 2024-04-25T09:03:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.552+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78700d0290 con 0x7f78700d1320 2024-04-25T09:03:26.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.552+0000 7fae32a00640 1 -- 172.21.15.62:0/1252484083 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae2c059010 con 0x7fae3403e870 2024-04-25T09:03:26.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.552+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78700d0850 con 0x7f78700d1320 2024-04-25T09:03:26.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.553+0000 7f2c63e00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c4406b520 0x7f2c4406d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c50034000 tx=0x7f2c5002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.554+0000 7fc8b6200640 1 Processor -- start 2024-04-25T09:03:26.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.554+0000 7fc8b6200640 1 -- start start 2024-04-25T09:03:26.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.554+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7870072da0 con 0x7f78700d1320 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.555+0000 7fc8b6200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 0x7fc8b0143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.555+0000 7fc8b6200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b00714a0 0x7fc8b0157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.555+0000 7fc8b6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b00cfc70 0x7fc8b01e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.555+0000 7fc8b6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b00cf010 con 0x7fc8b00cfc70 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.555+0000 7fc8b6200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b01406a0 con 0x7fc8b0067000 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.555+0000 7fc8b6200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b00cf190 con 0x7fc8b00714a0 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b00cfc70 0x7fc8b01e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b00cfc70 0x7fc8b01e87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55006/0 (socket says 172.21.15.62:55006) 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8afe00640 1 -- 172.21.15.62:0/4090617103 learned_addr learned my addr 172.21.15.62:0/4090617103 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8af400640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 0x7fc8b0143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8aea00640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b00714a0 0x7fc8b0157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8afe00640 1 -- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b00714a0 msgr2=0x7fc8b0157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8afe00640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b00714a0 0x7fc8b0157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8afe00640 1 -- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 msgr2=0x7fc8b0143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8afe00640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 0x7fc8b0143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.556+0000 7fc8afe00640 1 -- 172.21.15.62:0/4090617103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b01ea0b0 con 0x7fc8b00cfc70 2024-04-25T09:03:26.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.557+0000 7fc8afe00640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b00cfc70 0x7fc8b01e87c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000c370 tx=0x7fc8a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.557+0000 7fd59be00640 1 -- 172.21.15.62:0/2565165734 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd588032440 con 0x7fd59c072180 2024-04-25T09:03:26.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.557+0000 7fd59be00640 1 -- 172.21.15.62:0/2565165734 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5880327f0 con 0x7fd59c072180 2024-04-25T09:03:26.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.557+0000 7fc8a7400640 1 -- 172.21.15.62:0/4090617103 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0017070 con 0x7fc8b00cfc70 2024-04-25T09:03:26.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.557+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b01e9010 con 0x7fc8b00cfc70 2024-04-25T09:03:26.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.558+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b01e95d0 con 0x7fc8b00cfc70 2024-04-25T09:03:26.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.558+0000 7fd5a2c00640 1 -- 172.21.15.62:0/2565165734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c072180 msgr2=0x7fd59c159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.558+0000 7fd5a2c00640 1 --2- 172.21.15.62:0/2565165734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c072180 0x7fd59c159c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd5880097a0 tx=0x7fd58802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.558+0000 7fd5a2c00640 1 -- 172.21.15.62:0/2565165734 shutdown_connections 2024-04-25T09:03:26.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.558+0000 7fd5a2c00640 1 --2- 172.21.15.62:0/2565165734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c072180 0x7fd59c159c40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.558+0000 7fd5a2c00640 1 -- 172.21.15.62:0/2565165734 >> 172.21.15.62:0/2565165734 conn(0x7fd59c0b14d0 msgr2=0x7fd59c0b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.559+0000 7fabd7e00640 1 -- 172.21.15.62:0/3892391622 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fabdc059850 con 0x7fabe0053230 2024-04-25T09:03:26.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.559+0000 7fd5a2c00640 1 -- 172.21.15.62:0/2565165734 shutdown_connections 2024-04-25T09:03:26.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.559+0000 7f5513400640 1 -- 172.21.15.62:0/1126875071 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550003b070 con 0x7f551415bd10 2024-04-25T09:03:26.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.559+0000 7f5513400640 1 -- 172.21.15.62:0/1126875071 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500037840 con 0x7f551415bd10 2024-04-25T09:03:26.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.559+0000 7f5513400640 1 -- 172.21.15.62:0/1126875071 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500037c10 con 0x7f551415bd10 2024-04-25T09:03:26.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.559+0000 7fd5a2c00640 1 -- 172.21.15.62:0/2565165734 wait complete. 2024-04-25T09:03:26.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.560+0000 7f551b400640 1 -- 172.21.15.62:0/1126875071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 msgr2=0x7f55140cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.559+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b0072c70 con 0x7fc8b00cfc70 2024-04-25T09:03:26.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.560+0000 7f551b400640 1 --2- 172.21.15.62:0/1126875071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f55140cfca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5500009870 tx=0x7f5500032a20 comp rx=0 tx=0).stop 2024-04-25T09:03:26.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.560+0000 7f551b400640 1 -- 172.21.15.62:0/1126875071 shutdown_connections 2024-04-25T09:03:26.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.560+0000 7f551b400640 1 --2- 172.21.15.62:0/1126875071 >> v2:172.21.15.121:3300/0 conn(0x7f55140aeed0 0x7f55140cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.560+0000 7f551b400640 1 --2- 172.21.15.62:0/1126875071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f55140cfca0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.560+0000 7f551b400640 1 -- 172.21.15.62:0/1126875071 >> 172.21.15.62:0/1126875071 conn(0x7f55140668f0 msgr2=0x7f5514066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.561+0000 7f551b400640 1 -- 172.21.15.62:0/1126875071 shutdown_connections 2024-04-25T09:03:26.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.561+0000 7f551b400640 1 -- 172.21.15.62:0/1126875071 wait complete. 2024-04-25T09:03:26.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.561+0000 7fd5a2c00640 1 Processor -- start 2024-04-25T09:03:26.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.562+0000 7f7867400640 1 -- 172.21.15.62:0/3077700777 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7860007e80 con 0x7f78700d1320 2024-04-25T09:03:26.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.562+0000 7f7867400640 1 -- 172.21.15.62:0/3077700777 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860007950 con 0x7f78700d1320 2024-04-25T09:03:26.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.562+0000 7fd5a2c00640 1 -- start start 2024-04-25T09:03:26.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.561+0000 7f2c5be00640 1 -- 172.21.15.62:0/2845153402 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c5c058c80 con 0x7f2c6415dc40 2024-04-25T09:03:26.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.563+0000 7f551b400640 1 Processor -- start 2024-04-25T09:03:26.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.563+0000 7f551b400640 1 -- start start 2024-04-25T09:03:26.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.564+0000 7fd5a2c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd59c072180 0x7fd59c1425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.564+0000 7fd5a2c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd59c0b33e0 0x7fd59c1477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.564+0000 7fd5a2c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c0ceda0 0x7fd59c1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.564+0000 7fd5a1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd59c072180 0x7fd59c1425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7fd5a1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd59c072180 0x7fd59c1425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:52136/0 (socket says 172.21.15.62:52136) 2024-04-25T09:03:26.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7f551b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55140aeed0 0x7f551414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7f551b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55140cccf0 0x7f5514152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7f551b400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f5514154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7fd5a2200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c0ceda0 0x7fd59c1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7fd5a1800640 1 -- 172.21.15.62:0/601009691 learned_addr learned my addr 172.21.15.62:0/601009691 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7fd5a0e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd59c0b33e0 0x7fd59c1477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7fd5a2c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0b0b70 con 0x7fd59c0ceda0 2024-04-25T09:03:26.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7fd5a2c00640 1 -- 172.21.15.62:0/601009691 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0cf520 con 0x7fd59c072180 2024-04-25T09:03:26.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7fd5a2c00640 1 -- 172.21.15.62:0/601009691 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c06d900 con 0x7fd59c0b33e0 2024-04-25T09:03:26.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7f5513e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55140cccf0 0x7f5514152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7f5513e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55140cccf0 0x7f5514152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55034/0 (socket says 172.21.15.62:55034) 2024-04-25T09:03:26.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.564+0000 7f7867400640 1 -- 172.21.15.62:0/3077700777 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7860039040 con 0x7f78700d1320 2024-04-25T09:03:26.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.565+0000 7f5518a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55140aeed0 0x7f551414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7f5518a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55140aeed0 0x7f551414eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:52142/0 (socket says 172.21.15.62:52142) 2024-04-25T09:03:26.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7f5518a00640 1 -- 172.21.15.62:0/2486012273 learned_addr learned my addr 172.21.15.62:0/2486012273 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:26.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7f7867400640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f785006b4d0 0x7f785006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55140d01e0 con 0x7f55140cccf0 2024-04-25T09:03:26.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551406d2f0 con 0x7f55140aeed0 2024-04-25T09:03:26.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551406daf0 con 0x7f551415bd10 2024-04-25T09:03:26.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7f5519400640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f5514154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7f7867400640 1 -- 172.21.15.62:0/3077700777 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78600ac0e0 con 0x7f78700d1320 2024-04-25T09:03:26.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7f786f400640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f785006b4d0 0x7f785006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7f5518a00640 1 -- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 msgr2=0x7f5514154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7fd5a1800640 1 -- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd59c0b33e0 msgr2=0x7fd59c1477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7fd5a1800640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd59c0b33e0 0x7fd59c1477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7fd5a1800640 1 -- 172.21.15.62:0/601009691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c0ceda0 msgr2=0x7fd59c1401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.566+0000 7fd5a1800640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c0ceda0 0x7fd59c1401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7fd5a1800640 1 -- 172.21.15.62:0/601009691 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c1275a0 con 0x7fd59c072180 2024-04-25T09:03:26.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7f5518a00640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f5514154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7fd5a2200640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c0ceda0 0x7fd59c1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:26.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7f5518a00640 1 -- 172.21.15.62:0/2486012273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55140cccf0 msgr2=0x7f5514152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7f5518a00640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55140cccf0 0x7f5514152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7f5518a00640 1 -- 172.21.15.62:0/2486012273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55142019f0 con 0x7f55140aeed0 2024-04-25T09:03:26.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.567+0000 7f5513e00640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55140cccf0 0x7f5514152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:26.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.568+0000 7fd5a1800640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd59c072180 0x7fd59c1425c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c390 tx=0x7fd59000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.568+0000 7f7867400640 1 -- 172.21.15.62:0/3077700777 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7860053020 con 0x7f78700d1320 2024-04-25T09:03:26.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.568+0000 7fd599600640 1 -- 172.21.15.62:0/601009691 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590017070 con 0x7fd59c072180 2024-04-25T09:03:26.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.568+0000 7f5518a00640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55140aeed0 0x7f551414eb60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5500009990 tx=0x7f55000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.569+0000 7f5510c00640 1 -- 172.21.15.62:0/2486012273 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550003b040 con 0x7f55140aeed0 2024-04-25T09:03:26.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.569+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55141519d0 con 0x7f55140aeed0 2024-04-25T09:03:26.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.569+0000 7fd599600640 1 -- 172.21.15.62:0/601009691 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590005ce0 con 0x7fd59c072180 2024-04-25T09:03:26.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.569+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5514151f90 con 0x7f55140aeed0 2024-04-25T09:03:26.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.569+0000 7f5510c00640 1 -- 172.21.15.62:0/2486012273 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500045070 con 0x7f55140aeed0 2024-04-25T09:03:26.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.569+0000 7f5510c00640 1 -- 172.21.15.62:0/2486012273 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500036cd0 con 0x7f55140aeed0 2024-04-25T09:03:26.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.569+0000 7f786f400640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f785006b4d0 0x7f785006d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f785c005880 tx=0x7f785c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.569+0000 7fd599600640 1 -- 172.21.15.62:0/601009691 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900072f0 con 0x7fd59c072180 2024-04-25T09:03:26.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.571+0000 7fc8a7400640 1 -- 172.21.15.62:0/4090617103 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a0005ce0 con 0x7fc8b00cfc70 2024-04-25T09:03:26.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.571+0000 7fc8a7400640 1 -- 172.21.15.62:0/4090617103 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0021060 con 0x7fc8b00cfc70 2024-04-25T09:03:26.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.571+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5514072c70 con 0x7f55140aeed0 2024-04-25T09:03:26.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.572+0000 7fd5a2c00640 1 -- 172.21.15.62:0/601009691 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c1438e0 con 0x7fd59c072180 2024-04-25T09:03:26.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.572+0000 7fd5a2c00640 1 -- 172.21.15.62:0/601009691 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c144890 con 0x7fd59c072180 2024-04-25T09:03:26.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.574+0000 7fd58ea00640 1 -- 172.21.15.62:0/601009691 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd570005290 con 0x7fd59c072180 2024-04-25T09:03:26.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.573+0000 7f5510c00640 1 -- 172.21.15.62:0/2486012273 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f550003e070 con 0x7f55140aeed0 2024-04-25T09:03:26.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.575+0000 7f5510c00640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f54f406b5a0 0x7f54f406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.575+0000 7f5510c00640 1 -- 172.21.15.62:0/2486012273 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55000ab580 con 0x7f55140aeed0 2024-04-25T09:03:26.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.576+0000 7fc8a7400640 1 -- 172.21.15.62:0/4090617103 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8a001a070 con 0x7fc8b00cfc70 2024-04-25T09:03:26.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.577+0000 7fd599600640 1 -- 172.21.15.62:0/601009691 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59001a070 con 0x7fd59c072180 2024-04-25T09:03:26.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.577+0000 7f5513e00640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f54f406b5a0 0x7f54f406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.578+0000 7fc8a7400640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89006b4d0 0x7fc89006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.578+0000 7fd599600640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.578+0000 7fc8a7400640 1 -- 172.21.15.62:0/4090617103 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8a00899a0 con 0x7fc8b00cfc70 2024-04-25T09:03:26.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.578+0000 7fc8a7400640 1 -- 172.21.15.62:0/4090617103 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8a00b68c0 con 0x7fc8b00cfc70 2024-04-25T09:03:26.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.579+0000 7fc8af400640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89006b4d0 0x7fc89006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.579+0000 7fd599600640 1 -- 172.21.15.62:0/601009691 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5900894d0 con 0x7fd59c072180 2024-04-25T09:03:26.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.579+0000 7f5513e00640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f54f406b5a0 0x7f54f406da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f551414f3b0 tx=0x7f5508009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.579+0000 7fd5a0e00640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.580+0000 7fd5a0e00640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd588034000 tx=0x7fd58802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.584+0000 7fc8af400640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89006b4d0 0x7fc89006d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8034000 tx=0x7fc8a802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.587+0000 7f5510c00640 1 -- 172.21.15.62:0/2486012273 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f550007a3e0 con 0x7f55140aeed0 2024-04-25T09:03:26.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.601+0000 7fd599600640 1 -- 172.21.15.62:0/601009691 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd590057d00 con 0x7fd59c072180 2024-04-25T09:03:26.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.953+0000 7fe137800640 1 -- 172.21.15.62:0/2333799108 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe10c005580 con 0x7fe128039ed0 2024-04-25T09:03:26.945 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-25T09:03:26.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.954+0000 7fe12cc00640 1 -- 172.21.15.62:0/2333799108 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1300d2fc0 con 0x7fe128039ed0 2024-04-25T09:03:26.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.961+0000 7fe122000640 1 -- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe11006b6d0 msgr2=0x7fe11006db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.961+0000 7fe122000640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe11006b6d0 0x7fe11006db90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009950 tx=0x7fe11c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:26.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.961+0000 7fe122000640 1 -- 172.21.15.62:0/2333799108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe128039ed0 msgr2=0x7fe1280d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.961+0000 7fe122000640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe128039ed0 0x7fe1280d1670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1300697c0 tx=0x7fe130068e40 comp rx=0 tx=0).stop 2024-04-25T09:03:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.962+0000 7fe122000640 1 -- 172.21.15.62:0/2333799108 shutdown_connections 2024-04-25T09:03:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.962+0000 7fe122000640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe11006b6d0 0x7fe11006db90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.962+0000 7fe122000640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe128039ed0 0x7fe1280d1670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.962+0000 7fe122000640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1280e30c0 0x7fe128039990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.962+0000 7fe122000640 1 --2- 172.21.15.62:0/2333799108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe128054130 0x7fe12817cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.962+0000 7fe122000640 1 -- 172.21.15.62:0/2333799108 >> 172.21.15.62:0/2333799108 conn(0x7fe1280559e0 msgr2=0x7fe1280389c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.963+0000 7fe122000640 1 -- 172.21.15.62:0/2333799108 shutdown_connections 2024-04-25T09:03:26.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.963+0000 7fe122000640 1 -- 172.21.15.62:0/2333799108 wait complete. 2024-04-25T09:03:26.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.978+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f78700b02e0 con 0x7f78700d1320 2024-04-25T09:03:26.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.979+0000 7f7867400640 1 -- 172.21.15.62:0/3077700777 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f786007a8f0 con 0x7f78700d1320 2024-04-25T09:03:26.968 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-25T09:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.984+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f785006b4d0 msgr2=0x7f785006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.984+0000 7f7875000640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f785006b4d0 0x7f785006d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f785c005880 tx=0x7f785c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.984+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78700d1320 msgr2=0x7f78702044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.985+0000 7f7875000640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78700d1320 0x7f78702044c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7860031000 tx=0x7f7860004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.986+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 shutdown_connections 2024-04-25T09:03:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.986+0000 7f7875000640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f785006b4d0 0x7f785006d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.986+0000 7f7875000640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78700d1320 0x7f78702044c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.986+0000 7f7875000640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f787015af30 0x7f7870148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.986+0000 7f7875000640 1 --2- 172.21.15.62:0/3077700777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78700d0f40 0x7f78700cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.986+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 >> 172.21.15.62:0/3077700777 conn(0x7f787006abe0 msgr2=0x7f787015a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.986+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 shutdown_connections 2024-04-25T09:03:26.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.987+0000 7f7875000640 1 -- 172.21.15.62:0/3077700777 wait complete. 2024-04-25T09:03:26.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-25T09:03:26.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.990+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fae340da8e0 con 0x7fae3403e870 2024-04-25T09:03:26.978 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:26.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.991+0000 7fae32a00640 1 -- 172.21.15.62:0/1252484083 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae2c05d0b0 con 0x7fae3403e870 2024-04-25T09:03:26.979 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-25T09:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.995+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae1c06b5d0 msgr2=0x7fae1c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.995+0000 7fae41800640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae1c06b5d0 0x7fae1c06da90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fae340df2c0 tx=0x7fae28007a40 comp rx=0 tx=0).stop 2024-04-25T09:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.995+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3403e870 msgr2=0x7fae340dc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.995+0000 7fae41800640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3403e870 0x7fae340dc330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00e870 tx=0x7fae2c00ed30 comp rx=0 tx=0).stop 2024-04-25T09:03:26.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.995+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fabe0005570 con 0x7fabe0053230 2024-04-25T09:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 shutdown_connections 2024-04-25T09:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fae41800640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae1c06b5d0 0x7fae1c06da90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fae41800640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae340e30f0 0x7fae34174a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fae41800640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae34054330 0x7fae34174510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fae41800640 1 --2- 172.21.15.62:0/1252484083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae3403e870 0x7fae340dc330 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 >> 172.21.15.62:0/1252484083 conn(0x7fae34055a10 msgr2=0x7fae340e19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 shutdown_connections 2024-04-25T09:03:26.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fae41800640 1 -- 172.21.15.62:0/1252484083 wait complete. 2024-04-25T09:03:26.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.996+0000 7fabd7e00640 1 -- 172.21.15.62:0/3892391622 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fabdc05d8f0 con 0x7fabe0053230 2024-04-25T09:03:26.985 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabcc06b5a0 msgr2=0x7fabcc06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabcc06b5a0 0x7fabcc06da60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fabd8000f60 tx=0x7fabd8005e40 comp rx=0 tx=0).stop 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 msgr2=0x7fabe004dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe004dd80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00d5c0 tx=0x7fabdc00da80 comp rx=0 tx=0).stop 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 shutdown_connections 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabcc06b5a0 0x7fabcc06da60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe00df400 0x7fabe004e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fabe0053230 0x7fabe004dd80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 --2- 172.21.15.62:0/3892391622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabe004fce0 0x7fabe0173340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 >> 172.21.15.62:0/3892391622 conn(0x7fabe0054ca0 msgr2=0x7fabe00487d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:26.999+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 shutdown_connections 2024-04-25T09:03:26.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.000+0000 7fabeec00640 1 -- 172.21.15.62:0/3892391622 wait complete. 2024-04-25T09:03:26.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.004+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2c6404f000 con 0x7f2c6415dc40 2024-04-25T09:03:26.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.005+0000 7f2c5be00640 1 -- 172.21.15.62:0/2845153402 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c5c05cd20 con 0x7f2c6415dc40 2024-04-25T09:03:26.993 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-25T09:03:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c4406b520 msgr2=0x7f2c4406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c4406b520 0x7f2c4406d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c50034000 tx=0x7f2c5002e000 comp rx=0 tx=0).stop 2024-04-25T09:03:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6415dc40 msgr2=0x7f2c64154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6415dc40 0x7f2c64154140 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00cbc0 tx=0x7f2c5c00a430 comp rx=0 tx=0).stop 2024-04-25T09:03:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 shutdown_connections 2024-04-25T09:03:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c4406b520 0x7f2c4406d9e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c6415dc40 0x7f2c64154140 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c640d3ae0 0x7f2c640c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 --2- 172.21.15.62:0/2845153402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c640d0ae0 0x7f2c640cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.009+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 >> 172.21.15.62:0/2845153402 conn(0x7f2c6406b000 msgr2=0x7f2c641516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.010+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 shutdown_connections 2024-04-25T09:03:26.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.010+0000 7f2c6ac00640 1 -- 172.21.15.62:0/2845153402 wait complete. 2024-04-25T09:03:27.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-25T09:03:27.003 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.016+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc8b0159dd0 con 0x7fc8b00cfc70 2024-04-25T09:03:27.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.016+0000 7fc8a7400640 1 -- 172.21.15.62:0/4090617103 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc8a00581c0 con 0x7fc8b00cfc70 2024-04-25T09:03:27.004 INFO:teuthology.orchestra.run.smithi062.stdout:55834574853 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.020+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89006b4d0 msgr2=0x7fc89006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.020+0000 7fc8b6200640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89006b4d0 0x7fc89006d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8034000 tx=0x7fc8a802f000 comp rx=0 tx=0).stop 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.020+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b00cfc70 msgr2=0x7fc8b01e87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.020+0000 7fc8b6200640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b00cfc70 0x7fc8b01e87c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000c370 tx=0x7fc8a000c830 comp rx=0 tx=0).stop 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.021+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 shutdown_connections 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.021+0000 7fc8b6200640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89006b4d0 0x7fc89006d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.021+0000 7fc8b6200640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b00cfc70 0x7fc8b01e87c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.021+0000 7fc8b6200640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b00714a0 0x7fc8b0157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.021+0000 7fc8b6200640 1 --2- 172.21.15.62:0/4090617103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b0067000 0x7fc8b0143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.021+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 >> 172.21.15.62:0/4090617103 conn(0x7fc8b0074290 msgr2=0x7fc8b01592a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.021+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 shutdown_connections 2024-04-25T09:03:27.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.022+0000 7fc8b6200640 1 -- 172.21.15.62:0/4090617103 wait complete. 2024-04-25T09:03:27.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-25T09:03:27.011 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.023+0000 7fd58ea00640 1 -- 172.21.15.62:0/601009691 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd570005850 con 0x7fd59c072180 2024-04-25T09:03:27.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.024+0000 7fd599600640 1 -- 172.21.15.62:0/601009691 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd590087e20 con 0x7fd59c072180 2024-04-25T09:03:27.012 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-04-25T09:03:27.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-04-25T09:03:27.014 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 -- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 msgr2=0x7fd57c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd588034000 tx=0x7fd58802e000 comp rx=0 tx=0).stop 2024-04-25T09:03:27.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 -- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd59c072180 msgr2=0x7fd59c1425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd59c072180 0x7fd59c1425c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c390 tx=0x7fd59000c850 comp rx=0 tx=0).stop 2024-04-25T09:03:27.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 -- 172.21.15.62:0/601009691 shutdown_connections 2024-04-25T09:03:27.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd59c0ceda0 0x7fd59c1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd59c0b33e0 0x7fd59c1477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 --2- 172.21.15.62:0/601009691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd59c072180 0x7fd59c1425c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 -- 172.21.15.62:0/601009691 >> 172.21.15.62:0/601009691 conn(0x7fd59c0b14d0 msgr2=0x7fd59c14bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 -- 172.21.15.62:0/601009691 shutdown_connections 2024-04-25T09:03:27.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.027+0000 7fd58ea00640 1 -- 172.21.15.62:0/601009691 wait complete. 2024-04-25T09:03:27.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.030+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f551404f000 con 0x7f55140aeed0 2024-04-25T09:03:27.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-04-25T09:03:27.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.031+0000 7f5510c00640 1 -- 172.21.15.62:0/2486012273 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5500079ce0 con 0x7f55140aeed0 2024-04-25T09:03:27.020 INFO:teuthology.orchestra.run.smithi062.stdout:55834574854 2024-04-25T09:03:27.020 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f54f406b5a0 msgr2=0x7f54f406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f54f406b5a0 0x7f54f406da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f551414f3b0 tx=0x7f5508009040 comp rx=0 tx=0).stop 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55140aeed0 msgr2=0x7f551414eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55140aeed0 0x7f551414eb60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5500009990 tx=0x7f55000046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 shutdown_connections 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f54f406b5a0 0x7f54f406da60 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f551414f3b0 tx=0x7f5508009040 comp rx=0 tx=0).stop 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55140cccf0 0x7f5514152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f551415bd10 0x7f5514154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 --2- 172.21.15.62:0/2486012273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55140aeed0 0x7f551414eb60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.033+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 >> 172.21.15.62:0/2486012273 conn(0x7f55140668f0 msgr2=0x7f55140cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.034+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 shutdown_connections 2024-04-25T09:03:27.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.034+0000 7f551b400640 1 -- 172.21.15.62:0/2486012273 wait complete. 2024-04-25T09:03:27.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-04-25T09:03:27.031 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.035 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-25T09:03:27.036 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-04-25T09:03:27.037 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.038 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T09:03:27.038 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:27.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.293+0000 7ff8c9000640 1 Processor -- start 2024-04-25T09:03:27.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.293+0000 7ff8c9000640 1 -- start start 2024-04-25T09:03:27.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c9000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8c40cfed0 0x7ff8c415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c9000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c4072340 0x7ff8c40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c9000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff8c40707f0 0x7ff8c415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c9000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c406c310 con 0x7ff8c40707f0 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c9000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c415f5f0 con 0x7ff8c40cfed0 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c9000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c406c5b0 con 0x7ff8c4072340 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c2800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff8c40707f0 0x7ff8c415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c2800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff8c40707f0 0x7ff8c415b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55048/0 (socket says 172.21.15.62:55048) 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c2800640 1 -- 172.21.15.62:0/634728347 learned_addr learned my addr 172.21.15.62:0/634728347 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c1e00640 1 --2- 172.21.15.62:0/634728347 >> v2:172.21.15.121:3300/0 conn(0x7ff8c40cfed0 0x7ff8c415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.295+0000 7ff8c1400640 1 --2- 172.21.15.62:0/634728347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c4072340 0x7ff8c40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.296+0000 7ff8c2800640 1 -- 172.21.15.62:0/634728347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c4072340 msgr2=0x7ff8c40702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.296+0000 7ff8c2800640 1 --2- 172.21.15.62:0/634728347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c4072340 0x7ff8c40702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.296+0000 7ff8c2800640 1 -- 172.21.15.62:0/634728347 >> v2:172.21.15.121:3300/0 conn(0x7ff8c40cfed0 msgr2=0x7ff8c415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.296+0000 7ff8c2800640 1 --2- 172.21.15.62:0/634728347 >> v2:172.21.15.121:3300/0 conn(0x7ff8c40cfed0 0x7ff8c415f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.296+0000 7ff8c2800640 1 -- 172.21.15.62:0/634728347 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c406c890 con 0x7ff8c40707f0 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.297+0000 7ff8c2800640 1 --2- 172.21.15.62:0/634728347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40707f0 0x7ff8c415b0f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40cd560 tx=0x7ff8bc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be78288a8b94be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.297+0000 7ff8c0a00640 1 -- 172.21.15.62:0/634728347 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc036070 con 0x7ff8c40707f0 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.297+0000 7ff8c0a00640 1 -- 172.21.15.62:0/634728347 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc031600 con 0x7ff8c40707f0 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.297+0000 7ff8c0a00640 1 -- 172.21.15.62:0/634728347 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0319d0 con 0x7ff8c40707f0 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 -- 172.21.15.62:0/634728347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40707f0 msgr2=0x7ff8c415b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 --2- 172.21.15.62:0/634728347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40707f0 0x7ff8c415b0f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40cd560 tx=0x7ff8bc02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 -- 172.21.15.62:0/634728347 shutdown_connections 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 --2- 172.21.15.62:0/634728347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40707f0 0x7ff8c415b0f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 --2- 172.21.15.62:0/634728347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c4072340 0x7ff8c40702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 --2- 172.21.15.62:0/634728347 >> v2:172.21.15.121:3300/0 conn(0x7ff8c40cfed0 0x7ff8c415f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 -- 172.21.15.62:0/634728347 >> 172.21.15.62:0/634728347 conn(0x7ff8c40668f0 msgr2=0x7ff8c4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 -- 172.21.15.62:0/634728347 shutdown_connections 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.298+0000 7ff8c9000640 1 -- 172.21.15.62:0/634728347 wait complete. 2024-04-25T09:03:27.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.299+0000 7ff8c9000640 1 Processor -- start 2024-04-25T09:03:27.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.300+0000 7ff8c9000640 1 -- start start 2024-04-25T09:03:27.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c9000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40707f0 0x7ff8c40af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40cfed0 0x7ff8c415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c41f4c20 0x7ff8c41f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c9000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c406d130 con 0x7ff8c40cfed0 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c9000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c406ce50 con 0x7ff8c41f4c20 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c9000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c415b630 con 0x7ff8c40707f0 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40cfed0 0x7ff8c415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40cfed0 0x7ff8c415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55058/0 (socket says 172.21.15.62:55058) 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1400640 1 -- 172.21.15.62:0/2975661867 learned_addr learned my addr 172.21.15.62:0/2975661867 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1e00640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40707f0 0x7ff8c40af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1400640 1 -- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40707f0 msgr2=0x7ff8c40af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1400640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40707f0 0x7ff8c40af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1400640 1 -- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c41f4c20 msgr2=0x7ff8c41f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1400640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c41f4c20 0x7ff8c41f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.301+0000 7ff8c1400640 1 -- 172.21.15.62:0/2975661867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c413d960 con 0x7ff8c40cfed0 2024-04-25T09:03:27.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.302+0000 7ff8c1400640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40cfed0 0x7ff8c415a020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000d830 tx=0x7ff8b000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.302+0000 7ff8ba000640 1 -- 172.21.15.62:0/2975661867 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0018020 con 0x7ff8c40cfed0 2024-04-25T09:03:27.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.302+0000 7ff8ba000640 1 -- 172.21.15.62:0/2975661867 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b00096a0 con 0x7ff8c40cfed0 2024-04-25T09:03:27.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.302+0000 7ff8ba000640 1 -- 172.21.15.62:0/2975661867 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0009a70 con 0x7ff8c40cfed0 2024-04-25T09:03:27.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.302+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c413dc20 con 0x7ff8c40cfed0 2024-04-25T09:03:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.302+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c40afce0 con 0x7ff8c40cfed0 2024-04-25T09:03:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.303+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c4072c70 con 0x7ff8c40cfed0 2024-04-25T09:03:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.304+0000 7ff8ba000640 1 -- 172.21.15.62:0/2975661867 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8b001b020 con 0x7ff8c40cfed0 2024-04-25T09:03:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.305+0000 7ff8ba000640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.305+0000 7ff8ba000640 1 -- 172.21.15.62:0/2975661867 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8b008a9e0 con 0x7ff8c40cfed0 2024-04-25T09:03:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.305+0000 7ff8c1e00640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.306+0000 7ff8c1e00640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff8a406b4d0 0x7ff8a406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.309+0000 7ff8ba000640 1 -- 172.21.15.62:0/2975661867 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8b0022020 con 0x7ff8c40cfed0 2024-04-25T09:03:27.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.495+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8c413e140 con 0x7ff8a406b4d0 2024-04-25T09:03:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.514+0000 7ff8ba000640 1 -- 172.21.15.62:0/2975661867 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251832 (secure 0 0 0) 0x7ff8c413e140 con 0x7ff8a406b4d0 2024-04-25T09:03:27.505 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:27.507 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:03:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.518+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff8a406b4d0 msgr2=0x7ff8a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.518+0000 7ff8c9000640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff8a406b4d0 0x7ff8a406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.518+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40cfed0 msgr2=0x7ff8c415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40cfed0 0x7ff8c415a020 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000d830 tx=0x7ff8b000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 shutdown_connections 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8c40cfed0 0x7ff8c415a020 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8c41f4c20 0x7ff8c41f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 --2- 172.21.15.62:0/2975661867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8c40707f0 0x7ff8c40af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 >> 172.21.15.62:0/2975661867 conn(0x7ff8c40668f0 msgr2=0x7ff8c4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 shutdown_connections 2024-04-25T09:03:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.519+0000 7ff8c9000640 1 -- 172.21.15.62:0/2975661867 wait complete. 2024-04-25T09:03:27.525 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-25T09:03:25.885837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.764755"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172705+0000","last_change":"2024-04-25T09:03:17.179077+0000","last_active":"2024-04-25T09:03:18.172705+0000","last_peered":"2024-04-25T09:03:18.172705+0000","last_clean":"2024-04-25T09:03:18.172705+0000","last_became_active":"2024-04-25T09:03:17.178710+0000","last_became_peered":"2024-04-25T09:03:17.178710+0000","last_unstale":"2024-04-25T09:03:18.172705+0000","last_undegraded":"2024-04-25T09:03:18.172705+0000","last_fullsized":"2024-04-25T09:03:18.172705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565717+0000","last_change":"2024-04-25T09:03:17.164128+0000","last_active":"2024-04-25T09:03:18.565717+0000","last_peered":"2024-04-25T09:03:18.565717+0000","last_clean":"2024-04-25T09:03:18.565717+0000","last_became_active":"2024-04-25T09:03:17.163800+0000","last_became_peered":"2024-04-25T09:03:17.163800+0000","last_unstale":"2024-04-25T09:03:18.565717+0000","last_undegraded":"2024-04-25T09:03:18.565717+0000","last_fullsized":"2024-04-25T09:03:18.565717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:08.746742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565545+0000","last_change":"2024-04-25T09:03:17.167505+0000","last_active":"2024-04-25T09:03:18.565545+0000","last_peered":"2024-04-25T09:03:18.565545+0000","last_clean":"2024-04-25T09:03:18.565545+0000","last_became_active":"2024-04-25T09:03:17.167194+0000","last_became_peered":"2024-04-25T09:03:17.167194+0000","last_unstale":"2024-04-25T09:03:18.565545+0000","last_undegraded":"2024-04-25T09:03:18.565545+0000","last_fullsized":"2024-04-25T09:03:18.565545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:39.624252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562099+0000","last_change":"2024-04-25T09:03:17.162871+0000","last_active":"2024-04-25T09:03:18.562099+0000","last_peered":"2024-04-25T09:03:18.562099+0000","last_clean":"2024-04-25T09:03:18.562099+0000","last_became_active":"2024-04-25T09:03:17.162297+0000","last_became_peered":"2024-04-25T09:03:17.162297+0000","last_unstale":"2024-04-25T09:03:18.562099+0000","last_undegraded":"2024-04-25T09:03:18.562099+0000","last_fullsized":"2024-04-25T09:03:18.562099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171734+0000","last_change":"2024-04-25T09:03:17.172274+0000","last_active":"2024-04-25T09:03:18.171734+0000","last_peered":"2024-04-25T09:03:18.171734+0000","last_clean":"2024-04-25T09:03:18.171734+0000","last_became_active":"2024-04-25T09:03:17.171794+0000","last_became_peered":"2024-04-25T09:03:17.171794+0000","last_unstale":"2024-04-25T09:03:18.171734+0000","last_undegraded":"2024-04-25T09:03:18.171734+0000","last_fullsized":"2024-04-25T09:03:18.171734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562442+0000","last_change":"2024-04-25T09:03:17.158419+0000","last_active":"2024-04-25T09:03:18.562442+0000","last_peered":"2024-04-25T09:03:18.562442+0000","last_clean":"2024-04-25T09:03:18.562442+0000","last_became_active":"2024-04-25T09:03:17.155230+0000","last_became_peered":"2024-04-25T09:03:17.155230+0000","last_unstale":"2024-04-25T09:03:18.562442+0000","last_undegraded":"2024-04-25T09:03:18.562442+0000","last_fullsized":"2024-04-25T09:03:18.562442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172872+0000","last_change":"2024-04-25T09:03:17.165890+0000","last_active":"2024-04-25T09:03:18.172872+0000","last_peered":"2024-04-25T09:03:18.172872+0000","last_clean":"2024-04-25T09:03:18.172872+0000","last_became_active":"2024-04-25T09:03:17.164725+0000","last_became_peered":"2024-04-25T09:03:17.164725+0000","last_unstale":"2024-04-25T09:03:18.172872+0000","last_undegraded":"2024-04-25T09:03:18.172872+0000","last_fullsized":"2024-04-25T09:03:18.172872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:57.554441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563608+0000","last_change":"2024-04-25T09:03:17.162764+0000","last_active":"2024-04-25T09:03:18.563608+0000","last_peered":"2024-04-25T09:03:18.563608+0000","last_clean":"2024-04-25T09:03:18.563608+0000","last_became_active":"2024-04-25T09:03:17.162442+0000","last_became_peered":"2024-04-25T09:03:17.162442+0000","last_unstale":"2024-04-25T09:03:18.563608+0000","last_undegraded":"2024-04-25T09:03:18.563608+0000","last_fullsized":"2024-04-25T09:03:18.563608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:38.767433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151609+0000","last_change":"2024-04-25T09:03:17.156720+0000","last_active":"2024-04-25T09:03:18.151609+0000","last_peered":"2024-04-25T09:03:18.151609+0000","last_clean":"2024-04-25T09:03:18.151609+0000","last_became_active":"2024-04-25T09:03:17.156372+0000","last_became_peered":"2024-04-25T09:03:17.156372+0000","last_unstale":"2024-04-25T09:03:18.151609+0000","last_undegraded":"2024-04-25T09:03:18.151609+0000","last_fullsized":"2024-04-25T09:03:18.151609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172106+0000","last_change":"2024-04-25T09:03:17.160604+0000","last_active":"2024-04-25T09:03:18.172106+0000","last_peered":"2024-04-25T09:03:18.172106+0000","last_clean":"2024-04-25T09:03:18.172106+0000","last_became_active":"2024-04-25T09:03:17.157610+0000","last_became_peered":"2024-04-25T09:03:17.157610+0000","last_unstale":"2024-04-25T09:03:18.172106+0000","last_undegraded":"2024-04-25T09:03:18.172106+0000","last_fullsized":"2024-04-25T09:03:18.172106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:08.746742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172219+0000","last_change":"2024-04-25T09:03:17.172025+0000","last_active":"2024-04-25T09:03:18.172219+0000","last_peered":"2024-04-25T09:03:18.172219+0000","last_clean":"2024-04-25T09:03:18.172219+0000","last_became_active":"2024-04-25T09:03:17.171653+0000","last_became_peered":"2024-04-25T09:03:17.171653+0000","last_unstale":"2024-04-25T09:03:18.172219+0000","last_undegraded":"2024-04-25T09:03:18.172219+0000","last_fullsized":"2024-04-25T09:03:18.172219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:22.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172409+0000","last_change":"2024-04-25T09:03:17.155748+0000","last_active":"2024-04-25T09:03:18.172409+0000","last_peered":"2024-04-25T09:03:18.172409+0000","last_clean":"2024-04-25T09:03:18.172409+0000","last_became_active":"2024-04-25T09:03:17.155156+0000","last_became_peered":"2024-04-25T09:03:17.155156+0000","last_unstale":"2024-04-25T09:03:18.172409+0000","last_undegraded":"2024-04-25T09:03:18.172409+0000","last_fullsized":"2024-04-25T09:03:18.172409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:13.218030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563558+0000","last_change":"2024-04-25T09:03:17.162578+0000","last_active":"2024-04-25T09:03:18.563558+0000","last_peered":"2024-04-25T09:03:18.563558+0000","last_clean":"2024-04-25T09:03:18.563558+0000","last_became_active":"2024-04-25T09:03:17.162284+0000","last_became_peered":"2024-04-25T09:03:17.162284+0000","last_unstale":"2024-04-25T09:03:18.563558+0000","last_undegraded":"2024-04-25T09:03:18.563558+0000","last_fullsized":"2024-04-25T09:03:18.563558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:34.254072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563388+0000","last_change":"2024-04-25T09:03:17.150222+0000","last_active":"2024-04-25T09:03:18.563388+0000","last_peered":"2024-04-25T09:03:18.563388+0000","last_clean":"2024-04-25T09:03:18.563388+0000","last_became_active":"2024-04-25T09:03:17.149717+0000","last_became_peered":"2024-04-25T09:03:17.149717+0000","last_unstale":"2024-04-25T09:03:18.563388+0000","last_undegraded":"2024-04-25T09:03:18.563388+0000","last_fullsized":"2024-04-25T09:03:18.563388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564802+0000","last_change":"2024-04-25T09:03:17.156094+0000","last_active":"2024-04-25T09:03:18.564802+0000","last_peered":"2024-04-25T09:03:18.564802+0000","last_clean":"2024-04-25T09:03:18.564802+0000","last_became_active":"2024-04-25T09:03:17.155818+0000","last_became_peered":"2024-04-25T09:03:17.155818+0000","last_unstale":"2024-04-25T09:03:18.564802+0000","last_undegraded":"2024-04-25T09:03:18.564802+0000","last_fullsized":"2024-04-25T09:03:18.564802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.326298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561761+0000","last_change":"2024-04-25T09:03:17.164616+0000","last_active":"2024-04-25T09:03:18.561761+0000","last_peered":"2024-04-25T09:03:18.561761+0000","last_clean":"2024-04-25T09:03:18.561761+0000","last_became_active":"2024-04-25T09:03:17.158855+0000","last_became_peered":"2024-04-25T09:03:17.158855+0000","last_unstale":"2024-04-25T09:03:18.561761+0000","last_undegraded":"2024-04-25T09:03:18.561761+0000","last_fullsized":"2024-04-25T09:03:18.561761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565221+0000","last_change":"2024-04-25T09:03:17.163995+0000","last_active":"2024-04-25T09:03:18.565221+0000","last_peered":"2024-04-25T09:03:18.565221+0000","last_clean":"2024-04-25T09:03:18.565221+0000","last_became_active":"2024-04-25T09:03:17.162714+0000","last_became_peered":"2024-04-25T09:03:17.162714+0000","last_unstale":"2024-04-25T09:03:18.565221+0000","last_undegraded":"2024-04-25T09:03:18.565221+0000","last_fullsized":"2024-04-25T09:03:18.565221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:54.326567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567960+0000","last_change":"2024-04-25T09:03:17.159140+0000","last_active":"2024-04-25T09:03:18.567960+0000","last_peered":"2024-04-25T09:03:18.567960+0000","last_clean":"2024-04-25T09:03:18.567960+0000","last_became_active":"2024-04-25T09:03:17.158674+0000","last_became_peered":"2024-04-25T09:03:17.158674+0000","last_unstale":"2024-04-25T09:03:18.567960+0000","last_undegraded":"2024-04-25T09:03:18.567960+0000","last_fullsized":"2024-04-25T09:03:18.567960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564565+0000","last_change":"2024-04-25T09:03:17.153159+0000","last_active":"2024-04-25T09:03:18.564565+0000","last_peered":"2024-04-25T09:03:18.564565+0000","last_clean":"2024-04-25T09:03:18.564565+0000","last_became_active":"2024-04-25T09:03:17.152817+0000","last_became_peered":"2024-04-25T09:03:17.152817+0000","last_unstale":"2024-04-25T09:03:18.564565+0000","last_undegraded":"2024-04-25T09:03:18.564565+0000","last_fullsized":"2024-04-25T09:03:18.564565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:38.133534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172856+0000","last_change":"2024-04-25T09:03:17.155424+0000","last_active":"2024-04-25T09:03:18.172856+0000","last_peered":"2024-04-25T09:03:18.172856+0000","last_clean":"2024-04-25T09:03:18.172856+0000","last_became_active":"2024-04-25T09:03:17.154812+0000","last_became_peered":"2024-04-25T09:03:17.154812+0000","last_unstale":"2024-04-25T09:03:18.172856+0000","last_undegraded":"2024-04-25T09:03:18.172856+0000","last_fullsized":"2024-04-25T09:03:18.172856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:08.323326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154338+0000","last_change":"2024-04-25T09:03:17.156092+0000","last_active":"2024-04-25T09:03:18.154338+0000","last_peered":"2024-04-25T09:03:18.154338+0000","last_clean":"2024-04-25T09:03:18.154338+0000","last_became_active":"2024-04-25T09:03:17.153705+0000","last_became_peered":"2024-04-25T09:03:17.153705+0000","last_unstale":"2024-04-25T09:03:18.154338+0000","last_undegraded":"2024-04-25T09:03:18.154338+0000","last_fullsized":"2024-04-25T09:03:18.154338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:38.767433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150850+0000","last_change":"2024-04-25T09:03:17.155198+0000","last_active":"2024-04-25T09:03:18.150850+0000","last_peered":"2024-04-25T09:03:18.150850+0000","last_clean":"2024-04-25T09:03:18.150850+0000","last_became_active":"2024-04-25T09:03:17.154848+0000","last_became_peered":"2024-04-25T09:03:17.154848+0000","last_unstale":"2024-04-25T09:03:18.150850+0000","last_undegraded":"2024-04-25T09:03:18.150850+0000","last_fullsized":"2024-04-25T09:03:18.150850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:38.838938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564958+0000","last_change":"2024-04-25T09:03:17.161601+0000","last_active":"2024-04-25T09:03:18.564958+0000","last_peered":"2024-04-25T09:03:18.564958+0000","last_clean":"2024-04-25T09:03:18.564958+0000","last_became_active":"2024-04-25T09:03:17.161303+0000","last_became_peered":"2024-04-25T09:03:17.161303+0000","last_unstale":"2024-04-25T09:03:18.564958+0000","last_undegraded":"2024-04-25T09:03:18.564958+0000","last_fullsized":"2024-04-25T09:03:18.564958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:14.651686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562409+0000","last_change":"2024-04-25T09:03:17.168111+0000","last_active":"2024-04-25T09:03:18.562409+0000","last_peered":"2024-04-25T09:03:18.562409+0000","last_clean":"2024-04-25T09:03:18.562409+0000","last_became_active":"2024-04-25T09:03:17.165747+0000","last_became_peered":"2024-04-25T09:03:17.165747+0000","last_unstale":"2024-04-25T09:03:18.562409+0000","last_undegraded":"2024-04-25T09:03:18.562409+0000","last_fullsized":"2024-04-25T09:03:18.562409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:15.948906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565923+0000","last_change":"2024-04-25T09:03:17.161196+0000","last_active":"2024-04-25T09:03:18.565923+0000","last_peered":"2024-04-25T09:03:18.565923+0000","last_clean":"2024-04-25T09:03:18.565923+0000","last_became_active":"2024-04-25T09:03:17.158516+0000","last_became_peered":"2024-04-25T09:03:17.158516+0000","last_unstale":"2024-04-25T09:03:18.565923+0000","last_undegraded":"2024-04-25T09:03:18.565923+0000","last_fullsized":"2024-04-25T09:03:18.565923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:22.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563329+0000","last_change":"2024-04-25T09:03:17.164003+0000","last_active":"2024-04-25T09:03:18.563329+0000","last_peered":"2024-04-25T09:03:18.563329+0000","last_clean":"2024-04-25T09:03:18.563329+0000","last_became_active":"2024-04-25T09:03:17.162280+0000","last_became_peered":"2024-04-25T09:03:17.162280+0000","last_unstale":"2024-04-25T09:03:18.563329+0000","last_undegraded":"2024-04-25T09:03:18.563329+0000","last_fullsized":"2024-04-25T09:03:18.563329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:16.897576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564687+0000","last_change":"2024-04-25T09:03:17.155201+0000","last_active":"2024-04-25T09:03:18.564687+0000","last_peered":"2024-04-25T09:03:18.564687+0000","last_clean":"2024-04-25T09:03:18.564687+0000","last_became_active":"2024-04-25T09:03:17.154820+0000","last_became_peered":"2024-04-25T09:03:17.154820+0000","last_unstale":"2024-04-25T09:03:18.564687+0000","last_undegraded":"2024-04-25T09:03:18.564687+0000","last_fullsized":"2024-04-25T09:03:18.564687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:50.727941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563484+0000","last_change":"2024-04-25T09:03:17.155327+0000","last_active":"2024-04-25T09:03:18.563484+0000","last_peered":"2024-04-25T09:03:18.563484+0000","last_clean":"2024-04-25T09:03:18.563484+0000","last_became_active":"2024-04-25T09:03:17.155036+0000","last_became_peered":"2024-04-25T09:03:17.155036+0000","last_unstale":"2024-04-25T09:03:18.563484+0000","last_undegraded":"2024-04-25T09:03:18.563484+0000","last_fullsized":"2024-04-25T09:03:18.563484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:16.897576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565403+0000","last_change":"2024-04-25T09:03:17.157840+0000","last_active":"2024-04-25T09:03:18.565403+0000","last_peered":"2024-04-25T09:03:18.565403+0000","last_clean":"2024-04-25T09:03:18.565403+0000","last_became_active":"2024-04-25T09:03:17.157246+0000","last_became_peered":"2024-04-25T09:03:17.157246+0000","last_unstale":"2024-04-25T09:03:18.565403+0000","last_undegraded":"2024-04-25T09:03:18.565403+0000","last_fullsized":"2024-04-25T09:03:18.565403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563239+0000","last_change":"2024-04-25T09:03:17.170837+0000","last_active":"2024-04-25T09:03:18.563239+0000","last_peered":"2024-04-25T09:03:18.563239+0000","last_clean":"2024-04-25T09:03:18.563239+0000","last_became_active":"2024-04-25T09:03:17.166967+0000","last_became_peered":"2024-04-25T09:03:17.166967+0000","last_unstale":"2024-04-25T09:03:18.563239+0000","last_undegraded":"2024-04-25T09:03:18.563239+0000","last_fullsized":"2024-04-25T09:03:18.563239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:36.933011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567140+0000","last_change":"2024-04-25T09:03:17.145733+0000","last_active":"2024-04-25T09:03:18.567140+0000","last_peered":"2024-04-25T09:03:18.567140+0000","last_clean":"2024-04-25T09:03:18.567140+0000","last_became_active":"2024-04-25T09:03:17.145233+0000","last_became_peered":"2024-04-25T09:03:17.145233+0000","last_unstale":"2024-04-25T09:03:18.567140+0000","last_undegraded":"2024-04-25T09:03:18.567140+0000","last_fullsized":"2024-04-25T09:03:18.567140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:18.503932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561657+0000","last_change":"2024-04-25T09:03:17.157243+0000","last_active":"2024-04-25T09:03:18.561657+0000","last_peered":"2024-04-25T09:03:18.561657+0000","last_clean":"2024-04-25T09:03:18.561657+0000","last_became_active":"2024-04-25T09:03:17.156940+0000","last_became_peered":"2024-04-25T09:03:17.156940+0000","last_unstale":"2024-04-25T09:03:18.561657+0000","last_undegraded":"2024-04-25T09:03:18.561657+0000","last_fullsized":"2024-04-25T09:03:18.561657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150683+0000","last_change":"2024-04-25T09:03:17.156642+0000","last_active":"2024-04-25T09:03:18.150683+0000","last_peered":"2024-04-25T09:03:18.150683+0000","last_clean":"2024-04-25T09:03:18.150683+0000","last_became_active":"2024-04-25T09:03:17.155097+0000","last_became_peered":"2024-04-25T09:03:17.155097+0000","last_unstale":"2024-04-25T09:03:18.150683+0000","last_undegraded":"2024-04-25T09:03:18.150683+0000","last_fullsized":"2024-04-25T09:03:18.150683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565202+0000","last_change":"2024-04-25T09:03:16.166854+0000","last_active":"2024-04-25T09:03:18.565202+0000","last_peered":"2024-04-25T09:03:18.565202+0000","last_clean":"2024-04-25T09:03:18.565202+0000","last_became_active":"2024-04-25T09:03:16.166523+0000","last_became_peered":"2024-04-25T09:03:16.166523+0000","last_unstale":"2024-04-25T09:03:18.565202+0000","last_undegraded":"2024-04-25T09:03:18.565202+0000","last_fullsized":"2024-04-25T09:03:18.565202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154448+0000","last_change":"2024-04-25T09:03:17.158882+0000","last_active":"2024-04-25T09:03:18.154448+0000","last_peered":"2024-04-25T09:03:18.154448+0000","last_clean":"2024-04-25T09:03:18.154448+0000","last_became_active":"2024-04-25T09:03:17.158589+0000","last_became_peered":"2024-04-25T09:03:17.158589+0000","last_unstale":"2024-04-25T09:03:18.154448+0000","last_undegraded":"2024-04-25T09:03:18.154448+0000","last_fullsized":"2024-04-25T09:03:18.154448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:19.521616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563570+0000","last_change":"2024-04-25T09:03:16.168242+0000","last_active":"2024-04-25T09:03:18.563570+0000","last_peered":"2024-04-25T09:03:18.563570+0000","last_clean":"2024-04-25T09:03:18.563570+0000","last_became_active":"2024-04-25T09:03:16.167927+0000","last_became_peered":"2024-04-25T09:03:16.167927+0000","last_unstale":"2024-04-25T09:03:18.563570+0000","last_undegraded":"2024-04-25T09:03:18.563570+0000","last_fullsized":"2024-04-25T09:03:18.563570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:15.892070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562728+0000","last_change":"2024-04-25T09:03:16.163666+0000","last_active":"2024-04-25T09:03:18.562728+0000","last_peered":"2024-04-25T09:03:18.562728+0000","last_clean":"2024-04-25T09:03:18.562728+0000","last_became_active":"2024-04-25T09:03:16.163321+0000","last_became_peered":"2024-04-25T09:03:16.163321+0000","last_unstale":"2024-04-25T09:03:18.562728+0000","last_undegraded":"2024-04-25T09:03:18.562728+0000","last_fullsized":"2024-04-25T09:03:18.562728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564262+0000","last_change":"2024-04-25T09:03:17.163289+0000","last_active":"2024-04-25T09:03:18.564262+0000","last_peered":"2024-04-25T09:03:18.564262+0000","last_clean":"2024-04-25T09:03:18.564262+0000","last_became_active":"2024-04-25T09:03:17.162997+0000","last_became_peered":"2024-04-25T09:03:17.162997+0000","last_unstale":"2024-04-25T09:03:18.564262+0000","last_undegraded":"2024-04-25T09:03:18.564262+0000","last_fullsized":"2024-04-25T09:03:18.564262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:19.521616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561915+0000","last_change":"2024-04-25T09:03:16.146080+0000","last_active":"2024-04-25T09:03:18.561915+0000","last_peered":"2024-04-25T09:03:18.561915+0000","last_clean":"2024-04-25T09:03:18.561915+0000","last_became_active":"2024-04-25T09:03:16.145743+0000","last_became_peered":"2024-04-25T09:03:16.145743+0000","last_unstale":"2024-04-25T09:03:18.561915+0000","last_undegraded":"2024-04-25T09:03:18.561915+0000","last_fullsized":"2024-04-25T09:03:18.561915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:37.833432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154329+0000","last_change":"2024-04-25T09:03:17.146618+0000","last_active":"2024-04-25T09:03:18.154329+0000","last_peered":"2024-04-25T09:03:18.154329+0000","last_clean":"2024-04-25T09:03:18.154329+0000","last_became_active":"2024-04-25T09:03:17.146174+0000","last_became_peered":"2024-04-25T09:03:17.146174+0000","last_unstale":"2024-04-25T09:03:18.154329+0000","last_undegraded":"2024-04-25T09:03:18.154329+0000","last_fullsized":"2024-04-25T09:03:18.154329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:50.727941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151477+0000","last_change":"2024-04-25T09:03:16.159981+0000","last_active":"2024-04-25T09:03:18.151477+0000","last_peered":"2024-04-25T09:03:18.151477+0000","last_clean":"2024-04-25T09:03:18.151477+0000","last_became_active":"2024-04-25T09:03:16.159115+0000","last_became_peered":"2024-04-25T09:03:16.159115+0000","last_unstale":"2024-04-25T09:03:18.151477+0000","last_undegraded":"2024-04-25T09:03:18.151477+0000","last_fullsized":"2024-04-25T09:03:18.151477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563445+0000","last_change":"2024-04-25T09:03:17.163768+0000","last_active":"2024-04-25T09:03:18.563445+0000","last_peered":"2024-04-25T09:03:18.563445+0000","last_clean":"2024-04-25T09:03:18.563445+0000","last_became_active":"2024-04-25T09:03:17.162151+0000","last_became_peered":"2024-04-25T09:03:17.162151+0000","last_unstale":"2024-04-25T09:03:18.563445+0000","last_undegraded":"2024-04-25T09:03:18.563445+0000","last_fullsized":"2024-04-25T09:03:18.563445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:34.254072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172215+0000","last_change":"2024-04-25T09:03:16.169333+0000","last_active":"2024-04-25T09:03:18.172215+0000","last_peered":"2024-04-25T09:03:18.172215+0000","last_clean":"2024-04-25T09:03:18.172215+0000","last_became_active":"2024-04-25T09:03:16.167170+0000","last_became_peered":"2024-04-25T09:03:16.167170+0000","last_unstale":"2024-04-25T09:03:18.172215+0000","last_undegraded":"2024-04-25T09:03:18.172215+0000","last_fullsized":"2024-04-25T09:03:18.172215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:37.833432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566351+0000","last_change":"2024-04-25T09:03:17.168947+0000","last_active":"2024-04-25T09:03:18.566351+0000","last_peered":"2024-04-25T09:03:18.566351+0000","last_clean":"2024-04-25T09:03:18.566351+0000","last_became_active":"2024-04-25T09:03:17.168658+0000","last_became_peered":"2024-04-25T09:03:17.168658+0000","last_unstale":"2024-04-25T09:03:18.566351+0000","last_undegraded":"2024-04-25T09:03:18.566351+0000","last_fullsized":"2024-04-25T09:03:18.566351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567577+0000","last_change":"2024-04-25T09:03:16.160547+0000","last_active":"2024-04-25T09:03:18.567577+0000","last_peered":"2024-04-25T09:03:18.567577+0000","last_clean":"2024-04-25T09:03:18.567577+0000","last_became_active":"2024-04-25T09:03:16.160169+0000","last_became_peered":"2024-04-25T09:03:16.160169+0000","last_unstale":"2024-04-25T09:03:18.567577+0000","last_undegraded":"2024-04-25T09:03:18.567577+0000","last_fullsized":"2024-04-25T09:03:18.567577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564136+0000","last_change":"2024-04-25T09:03:17.161468+0000","last_active":"2024-04-25T09:03:18.564136+0000","last_peered":"2024-04-25T09:03:18.564136+0000","last_clean":"2024-04-25T09:03:18.564136+0000","last_became_active":"2024-04-25T09:03:17.159824+0000","last_became_peered":"2024-04-25T09:03:17.159824+0000","last_unstale":"2024-04-25T09:03:18.564136+0000","last_undegraded":"2024-04-25T09:03:18.564136+0000","last_fullsized":"2024-04-25T09:03:18.564136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:37.830284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562202+0000","last_change":"2024-04-25T09:03:16.155839+0000","last_active":"2024-04-25T09:03:18.562202+0000","last_peered":"2024-04-25T09:03:18.562202+0000","last_clean":"2024-04-25T09:03:18.562202+0000","last_became_active":"2024-04-25T09:03:16.155363+0000","last_became_peered":"2024-04-25T09:03:16.155363+0000","last_unstale":"2024-04-25T09:03:18.562202+0000","last_undegraded":"2024-04-25T09:03:18.562202+0000","last_fullsized":"2024-04-25T09:03:18.562202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:13.008112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172333+0000","last_change":"2024-04-25T09:03:17.158443+0000","last_active":"2024-04-25T09:03:18.172333+0000","last_peered":"2024-04-25T09:03:18.172333+0000","last_clean":"2024-04-25T09:03:18.172333+0000","last_became_active":"2024-04-25T09:03:17.158153+0000","last_became_peered":"2024-04-25T09:03:17.158153+0000","last_unstale":"2024-04-25T09:03:18.172333+0000","last_undegraded":"2024-04-25T09:03:18.172333+0000","last_fullsized":"2024-04-25T09:03:18.172333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.046413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567015+0000","last_change":"2024-04-25T09:03:16.170150+0000","last_active":"2024-04-25T09:03:18.567015+0000","last_peered":"2024-04-25T09:03:18.567015+0000","last_clean":"2024-04-25T09:03:18.567015+0000","last_became_active":"2024-04-25T09:03:16.169528+0000","last_became_peered":"2024-04-25T09:03:16.169528+0000","last_unstale":"2024-04-25T09:03:18.567015+0000","last_undegraded":"2024-04-25T09:03:18.567015+0000","last_fullsized":"2024-04-25T09:03:18.567015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563761+0000","last_change":"2024-04-25T09:03:17.161896+0000","last_active":"2024-04-25T09:03:18.563761+0000","last_peered":"2024-04-25T09:03:18.563761+0000","last_clean":"2024-04-25T09:03:18.563761+0000","last_became_active":"2024-04-25T09:03:17.161609+0000","last_became_peered":"2024-04-25T09:03:17.161609+0000","last_unstale":"2024-04-25T09:03:18.563761+0000","last_undegraded":"2024-04-25T09:03:18.563761+0000","last_fullsized":"2024-04-25T09:03:18.563761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563685+0000","last_change":"2024-04-25T09:03:16.150079+0000","last_active":"2024-04-25T09:03:18.563685+0000","last_peered":"2024-04-25T09:03:18.563685+0000","last_clean":"2024-04-25T09:03:18.563685+0000","last_became_active":"2024-04-25T09:03:16.149418+0000","last_became_peered":"2024-04-25T09:03:16.149418+0000","last_unstale":"2024-04-25T09:03:18.563685+0000","last_undegraded":"2024-04-25T09:03:18.563685+0000","last_fullsized":"2024-04-25T09:03:18.563685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:33.248566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564016+0000","last_change":"2024-04-25T09:03:17.169709+0000","last_active":"2024-04-25T09:03:18.564016+0000","last_peered":"2024-04-25T09:03:18.564016+0000","last_clean":"2024-04-25T09:03:18.564016+0000","last_became_active":"2024-04-25T09:03:17.169402+0000","last_became_peered":"2024-04-25T09:03:17.169402+0000","last_unstale":"2024-04-25T09:03:18.564016+0000","last_undegraded":"2024-04-25T09:03:18.564016+0000","last_fullsized":"2024-04-25T09:03:18.564016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:23.474225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171619+0000","last_change":"2024-04-25T09:03:16.164546+0000","last_active":"2024-04-25T09:03:18.171619+0000","last_peered":"2024-04-25T09:03:18.171619+0000","last_clean":"2024-04-25T09:03:18.171619+0000","last_became_active":"2024-04-25T09:03:16.162541+0000","last_became_peered":"2024-04-25T09:03:16.162541+0000","last_unstale":"2024-04-25T09:03:18.171619+0000","last_undegraded":"2024-04-25T09:03:18.171619+0000","last_fullsized":"2024-04-25T09:03:18.171619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:15.408869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150694+0000","last_change":"2024-04-25T09:03:17.156971+0000","last_active":"2024-04-25T09:03:18.150694+0000","last_peered":"2024-04-25T09:03:18.150694+0000","last_clean":"2024-04-25T09:03:18.150694+0000","last_became_active":"2024-04-25T09:03:17.156624+0000","last_became_peered":"2024-04-25T09:03:17.156624+0000","last_unstale":"2024-04-25T09:03:18.150694+0000","last_undegraded":"2024-04-25T09:03:18.150694+0000","last_fullsized":"2024-04-25T09:03:18.150694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:16.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153276+0000","last_change":"2024-04-25T09:03:16.155291+0000","last_active":"2024-04-25T09:03:18.153276+0000","last_peered":"2024-04-25T09:03:18.153276+0000","last_clean":"2024-04-25T09:03:18.153276+0000","last_became_active":"2024-04-25T09:03:16.154984+0000","last_became_peered":"2024-04-25T09:03:16.154984+0000","last_unstale":"2024-04-25T09:03:18.153276+0000","last_undegraded":"2024-04-25T09:03:18.153276+0000","last_fullsized":"2024-04-25T09:03:18.153276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562872+0000","last_change":"2024-04-25T09:03:17.163557+0000","last_active":"2024-04-25T09:03:18.562872+0000","last_peered":"2024-04-25T09:03:18.562872+0000","last_clean":"2024-04-25T09:03:18.562872+0000","last_became_active":"2024-04-25T09:03:17.163233+0000","last_became_peered":"2024-04-25T09:03:17.163233+0000","last_unstale":"2024-04-25T09:03:18.562872+0000","last_undegraded":"2024-04-25T09:03:18.562872+0000","last_fullsized":"2024-04-25T09:03:18.562872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:08.323326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172841+0000","last_change":"2024-04-25T09:03:16.168391+0000","last_active":"2024-04-25T09:03:18.172841+0000","last_peered":"2024-04-25T09:03:18.172841+0000","last_clean":"2024-04-25T09:03:18.172841+0000","last_became_active":"2024-04-25T09:03:16.168048+0000","last_became_peered":"2024-04-25T09:03:16.168048+0000","last_unstale":"2024-04-25T09:03:18.172841+0000","last_undegraded":"2024-04-25T09:03:18.172841+0000","last_fullsized":"2024-04-25T09:03:18.172841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:49.931470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563750+0000","last_change":"2024-04-25T09:03:17.173328+0000","last_active":"2024-04-25T09:03:18.563750+0000","last_peered":"2024-04-25T09:03:18.563750+0000","last_clean":"2024-04-25T09:03:18.563750+0000","last_became_active":"2024-04-25T09:03:17.165984+0000","last_became_peered":"2024-04-25T09:03:17.165984+0000","last_unstale":"2024-04-25T09:03:18.563750+0000","last_undegraded":"2024-04-25T09:03:18.563750+0000","last_fullsized":"2024-04-25T09:03:18.563750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153832+0000","last_change":"2024-04-25T09:03:16.167227+0000","last_active":"2024-04-25T09:03:18.153832+0000","last_peered":"2024-04-25T09:03:18.153832+0000","last_clean":"2024-04-25T09:03:18.153832+0000","last_became_active":"2024-04-25T09:03:16.166892+0000","last_became_peered":"2024-04-25T09:03:16.166892+0000","last_unstale":"2024-04-25T09:03:18.153832+0000","last_undegraded":"2024-04-25T09:03:18.153832+0000","last_fullsized":"2024-04-25T09:03:18.153832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172826+0000","last_change":"2024-04-25T09:03:17.163685+0000","last_active":"2024-04-25T09:03:18.172826+0000","last_peered":"2024-04-25T09:03:18.172826+0000","last_clean":"2024-04-25T09:03:18.172826+0000","last_became_active":"2024-04-25T09:03:17.163386+0000","last_became_peered":"2024-04-25T09:03:17.163386+0000","last_unstale":"2024-04-25T09:03:18.172826+0000","last_undegraded":"2024-04-25T09:03:18.172826+0000","last_fullsized":"2024-04-25T09:03:18.172826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:15.948906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565656+0000","last_change":"2024-04-25T09:03:16.160968+0000","last_active":"2024-04-25T09:03:18.565656+0000","last_peered":"2024-04-25T09:03:18.565656+0000","last_clean":"2024-04-25T09:03:18.565656+0000","last_became_active":"2024-04-25T09:03:16.158804+0000","last_became_peered":"2024-04-25T09:03:16.158804+0000","last_unstale":"2024-04-25T09:03:18.565656+0000","last_undegraded":"2024-04-25T09:03:18.565656+0000","last_fullsized":"2024-04-25T09:03:18.565656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.173017+0000","last_change":"2024-04-25T09:03:17.162364+0000","last_active":"2024-04-25T09:03:18.173017+0000","last_peered":"2024-04-25T09:03:18.173017+0000","last_clean":"2024-04-25T09:03:18.173017+0000","last_became_active":"2024-04-25T09:03:17.162093+0000","last_became_peered":"2024-04-25T09:03:17.162093+0000","last_unstale":"2024-04-25T09:03:18.173017+0000","last_undegraded":"2024-04-25T09:03:18.173017+0000","last_fullsized":"2024-04-25T09:03:18.173017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:36.933011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563994+0000","last_change":"2024-04-25T09:03:16.167573+0000","last_active":"2024-04-25T09:03:18.563994+0000","last_peered":"2024-04-25T09:03:18.563994+0000","last_clean":"2024-04-25T09:03:18.563994+0000","last_became_active":"2024-04-25T09:03:16.167279+0000","last_became_peered":"2024-04-25T09:03:16.167279+0000","last_unstale":"2024-04-25T09:03:18.563994+0000","last_undegraded":"2024-04-25T09:03:18.563994+0000","last_fullsized":"2024-04-25T09:03:18.563994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564569+0000","last_change":"2024-04-25T09:03:17.160049+0000","last_active":"2024-04-25T09:03:18.564569+0000","last_peered":"2024-04-25T09:03:18.564569+0000","last_clean":"2024-04-25T09:03:18.564569+0000","last_became_active":"2024-04-25T09:03:17.157508+0000","last_became_peered":"2024-04-25T09:03:17.157508+0000","last_unstale":"2024-04-25T09:03:18.564569+0000","last_undegraded":"2024-04-25T09:03:18.564569+0000","last_fullsized":"2024-04-25T09:03:18.564569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:25.867031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150373+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:03:18.150373+0000","last_peered":"2024-04-25T09:03:18.150373+0000","last_clean":"2024-04-25T09:03:18.150373+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:03:18.150373+0000","last_undegraded":"2024-04-25T09:03:18.150373+0000","last_fullsized":"2024-04-25T09:03:18.150373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564168+0000","last_change":"2024-04-25T09:03:16.152466+0000","last_active":"2024-04-25T09:03:18.564168+0000","last_peered":"2024-04-25T09:03:18.564168+0000","last_clean":"2024-04-25T09:03:18.564168+0000","last_became_active":"2024-04-25T09:03:16.152216+0000","last_became_peered":"2024-04-25T09:03:16.152216+0000","last_unstale":"2024-04-25T09:03:18.564168+0000","last_undegraded":"2024-04-25T09:03:18.564168+0000","last_fullsized":"2024-04-25T09:03:18.564168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:49.931470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154561+0000","last_change":"2024-04-25T09:03:17.162520+0000","last_active":"2024-04-25T09:03:18.154561+0000","last_peered":"2024-04-25T09:03:18.154561+0000","last_clean":"2024-04-25T09:03:18.154561+0000","last_became_active":"2024-04-25T09:03:17.160462+0000","last_became_peered":"2024-04-25T09:03:17.160462+0000","last_unstale":"2024-04-25T09:03:18.154561+0000","last_undegraded":"2024-04-25T09:03:18.154561+0000","last_fullsized":"2024-04-25T09:03:18.154561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.326298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562165+0000","last_change":"2024-04-25T09:03:16.167181+0000","last_active":"2024-04-25T09:03:18.562165+0000","last_peered":"2024-04-25T09:03:18.562165+0000","last_clean":"2024-04-25T09:03:18.562165+0000","last_became_active":"2024-04-25T09:03:16.166842+0000","last_became_peered":"2024-04-25T09:03:16.166842+0000","last_unstale":"2024-04-25T09:03:18.562165+0000","last_undegraded":"2024-04-25T09:03:18.562165+0000","last_fullsized":"2024-04-25T09:03:18.562165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:54.518583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563045+0000","last_change":"2024-04-25T09:03:17.158524+0000","last_active":"2024-04-25T09:03:18.563045+0000","last_peered":"2024-04-25T09:03:18.563045+0000","last_clean":"2024-04-25T09:03:18.563045+0000","last_became_active":"2024-04-25T09:03:17.157997+0000","last_became_peered":"2024-04-25T09:03:17.157997+0000","last_unstale":"2024-04-25T09:03:18.563045+0000","last_undegraded":"2024-04-25T09:03:18.563045+0000","last_fullsized":"2024-04-25T09:03:18.563045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566796+0000","last_change":"2024-04-25T09:03:16.169797+0000","last_active":"2024-04-25T09:03:18.566796+0000","last_peered":"2024-04-25T09:03:18.566796+0000","last_clean":"2024-04-25T09:03:18.566796+0000","last_became_active":"2024-04-25T09:03:16.169557+0000","last_became_peered":"2024-04-25T09:03:16.169557+0000","last_unstale":"2024-04-25T09:03:18.566796+0000","last_undegraded":"2024-04-25T09:03:18.566796+0000","last_fullsized":"2024-04-25T09:03:18.566796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:10.210895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563921+0000","last_change":"2024-04-25T09:03:17.157959+0000","last_active":"2024-04-25T09:03:18.563921+0000","last_peered":"2024-04-25T09:03:18.563921+0000","last_clean":"2024-04-25T09:03:18.563921+0000","last_became_active":"2024-04-25T09:03:17.157664+0000","last_became_peered":"2024-04-25T09:03:17.157664+0000","last_unstale":"2024-04-25T09:03:18.563921+0000","last_undegraded":"2024-04-25T09:03:18.563921+0000","last_fullsized":"2024-04-25T09:03:18.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.631418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563894+0000","last_change":"2024-04-25T09:03:16.157473+0000","last_active":"2024-04-25T09:03:18.563894+0000","last_peered":"2024-04-25T09:03:18.563894+0000","last_clean":"2024-04-25T09:03:18.563894+0000","last_became_active":"2024-04-25T09:03:16.157153+0000","last_became_peered":"2024-04-25T09:03:16.157153+0000","last_unstale":"2024-04-25T09:03:18.563894+0000","last_undegraded":"2024-04-25T09:03:18.563894+0000","last_fullsized":"2024-04-25T09:03:18.563894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:41.809030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172033+0000","last_change":"2024-04-25T09:03:17.164826+0000","last_active":"2024-04-25T09:03:18.172033+0000","last_peered":"2024-04-25T09:03:18.172033+0000","last_clean":"2024-04-25T09:03:18.172033+0000","last_became_active":"2024-04-25T09:03:17.161434+0000","last_became_peered":"2024-04-25T09:03:17.161434+0000","last_unstale":"2024-04-25T09:03:18.172033+0000","last_undegraded":"2024-04-25T09:03:18.172033+0000","last_fullsized":"2024-04-25T09:03:18.172033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:39.624252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565331+0000","last_change":"2024-04-25T09:03:16.171358+0000","last_active":"2024-04-25T09:03:18.565331+0000","last_peered":"2024-04-25T09:03:18.565331+0000","last_clean":"2024-04-25T09:03:18.565331+0000","last_became_active":"2024-04-25T09:03:16.171010+0000","last_became_peered":"2024-04-25T09:03:16.171010+0000","last_unstale":"2024-04-25T09:03:18.565331+0000","last_undegraded":"2024-04-25T09:03:18.565331+0000","last_fullsized":"2024-04-25T09:03:18.565331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:41.025423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562033+0000","last_change":"2024-04-25T09:03:17.161787+0000","last_active":"2024-04-25T09:03:18.562033+0000","last_peered":"2024-04-25T09:03:18.562033+0000","last_clean":"2024-04-25T09:03:18.562033+0000","last_became_active":"2024-04-25T09:03:17.157886+0000","last_became_peered":"2024-04-25T09:03:17.157886+0000","last_unstale":"2024-04-25T09:03:18.562033+0000","last_undegraded":"2024-04-25T09:03:18.562033+0000","last_fullsized":"2024-04-25T09:03:18.562033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:38.838938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564686+0000","last_change":"2024-04-25T09:03:16.157712+0000","last_active":"2024-04-25T09:03:18.564686+0000","last_peered":"2024-04-25T09:03:18.564686+0000","last_clean":"2024-04-25T09:03:18.564686+0000","last_became_active":"2024-04-25T09:03:16.157285+0000","last_became_peered":"2024-04-25T09:03:16.157285+0000","last_unstale":"2024-04-25T09:03:18.564686+0000","last_undegraded":"2024-04-25T09:03:18.564686+0000","last_fullsized":"2024-04-25T09:03:18.564686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:17.498426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563865+0000","last_change":"2024-04-25T09:03:17.169015+0000","last_active":"2024-04-25T09:03:18.563865+0000","last_peered":"2024-04-25T09:03:18.563865+0000","last_clean":"2024-04-25T09:03:18.563865+0000","last_became_active":"2024-04-25T09:03:17.168430+0000","last_became_peered":"2024-04-25T09:03:17.168430+0000","last_unstale":"2024-04-25T09:03:18.563865+0000","last_undegraded":"2024-04-25T09:03:18.563865+0000","last_fullsized":"2024-04-25T09:03:18.563865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:57.330616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154237+0000","last_change":"2024-04-25T09:03:16.167928+0000","last_active":"2024-04-25T09:03:18.154237+0000","last_peered":"2024-04-25T09:03:18.154237+0000","last_clean":"2024-04-25T09:03:18.154237+0000","last_became_active":"2024-04-25T09:03:16.167566+0000","last_became_peered":"2024-04-25T09:03:16.167566+0000","last_unstale":"2024-04-25T09:03:18.154237+0000","last_undegraded":"2024-04-25T09:03:18.154237+0000","last_fullsized":"2024-04-25T09:03:18.154237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:41.025423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154329+0000","last_change":"2024-04-25T09:03:17.157268+0000","last_active":"2024-04-25T09:03:18.154329+0000","last_peered":"2024-04-25T09:03:18.154329+0000","last_clean":"2024-04-25T09:03:18.154329+0000","last_became_active":"2024-04-25T09:03:17.156997+0000","last_became_peered":"2024-04-25T09:03:17.156997+0000","last_unstale":"2024-04-25T09:03:18.154329+0000","last_undegraded":"2024-04-25T09:03:18.154329+0000","last_fullsized":"2024-04-25T09:03:18.154329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.631418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565139+0000","last_change":"2024-04-25T09:03:16.169093+0000","last_active":"2024-04-25T09:03:18.565139+0000","last_peered":"2024-04-25T09:03:18.565139+0000","last_clean":"2024-04-25T09:03:18.565139+0000","last_became_active":"2024-04-25T09:03:16.168826+0000","last_became_peered":"2024-04-25T09:03:16.168826+0000","last_unstale":"2024-04-25T09:03:18.565139+0000","last_undegraded":"2024-04-25T09:03:18.565139+0000","last_fullsized":"2024-04-25T09:03:18.565139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.217267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154092+0000","last_change":"2024-04-25T09:03:17.136952+0000","last_active":"2024-04-25T09:03:18.154092+0000","last_peered":"2024-04-25T09:03:18.154092+0000","last_clean":"2024-04-25T09:03:18.154092+0000","last_became_active":"2024-04-25T09:03:17.136246+0000","last_became_peered":"2024-04-25T09:03:17.136246+0000","last_unstale":"2024-04-25T09:03:18.154092+0000","last_undegraded":"2024-04-25T09:03:18.154092+0000","last_fullsized":"2024-04-25T09:03:18.154092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:57.330616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171136+0000","last_change":"2024-04-25T09:03:16.156412+0000","last_active":"2024-04-25T09:03:18.171136+0000","last_peered":"2024-04-25T09:03:18.171136+0000","last_clean":"2024-04-25T09:03:18.171136+0000","last_became_active":"2024-04-25T09:03:16.151576+0000","last_became_peered":"2024-04-25T09:03:16.151576+0000","last_unstale":"2024-04-25T09:03:18.171136+0000","last_undegraded":"2024-04-25T09:03:18.171136+0000","last_fullsized":"2024-04-25T09:03:18.171136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.002913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562924+0000","last_change":"2024-04-25T09:03:17.154125+0000","last_active":"2024-04-25T09:03:18.562924+0000","last_peered":"2024-04-25T09:03:18.562924+0000","last_clean":"2024-04-25T09:03:18.562924+0000","last_became_active":"2024-04-25T09:03:17.153852+0000","last_became_peered":"2024-04-25T09:03:17.153852+0000","last_unstale":"2024-04-25T09:03:18.562924+0000","last_undegraded":"2024-04-25T09:03:18.562924+0000","last_fullsized":"2024-04-25T09:03:18.562924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:57.554441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153647+0000","last_change":"2024-04-25T09:03:16.167837+0000","last_active":"2024-04-25T09:03:18.153647+0000","last_peered":"2024-04-25T09:03:18.153647+0000","last_clean":"2024-04-25T09:03:18.153647+0000","last_became_active":"2024-04-25T09:03:16.167365+0000","last_became_peered":"2024-04-25T09:03:16.167365+0000","last_unstale":"2024-04-25T09:03:18.153647+0000","last_undegraded":"2024-04-25T09:03:18.153647+0000","last_fullsized":"2024-04-25T09:03:18.153647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:57.583736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153552+0000","last_change":"2024-04-25T09:03:17.137828+0000","last_active":"2024-04-25T09:03:18.153552+0000","last_peered":"2024-04-25T09:03:18.153552+0000","last_clean":"2024-04-25T09:03:18.153552+0000","last_became_active":"2024-04-25T09:03:17.137277+0000","last_became_peered":"2024-04-25T09:03:17.137277+0000","last_unstale":"2024-04-25T09:03:18.153552+0000","last_undegraded":"2024-04-25T09:03:18.153552+0000","last_fullsized":"2024-04-25T09:03:18.153552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564048+0000","last_change":"2024-04-25T09:03:16.165762+0000","last_active":"2024-04-25T09:03:18.564048+0000","last_peered":"2024-04-25T09:03:18.564048+0000","last_clean":"2024-04-25T09:03:18.564048+0000","last_became_active":"2024-04-25T09:03:16.165453+0000","last_became_peered":"2024-04-25T09:03:16.165453+0000","last_unstale":"2024-04-25T09:03:18.564048+0000","last_undegraded":"2024-04-25T09:03:18.564048+0000","last_fullsized":"2024-04-25T09:03:18.564048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567253+0000","last_change":"2024-04-25T09:03:17.152812+0000","last_active":"2024-04-25T09:03:18.567253+0000","last_peered":"2024-04-25T09:03:18.567253+0000","last_clean":"2024-04-25T09:03:18.567253+0000","last_became_active":"2024-04-25T09:03:17.152461+0000","last_became_peered":"2024-04-25T09:03:17.152461+0000","last_unstale":"2024-04-25T09:03:18.567253+0000","last_undegraded":"2024-04-25T09:03:18.567253+0000","last_fullsized":"2024-04-25T09:03:18.567253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:16.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563836+0000","last_change":"2024-04-25T09:03:16.156321+0000","last_active":"2024-04-25T09:03:18.563836+0000","last_peered":"2024-04-25T09:03:18.563836+0000","last_clean":"2024-04-25T09:03:18.563836+0000","last_became_active":"2024-04-25T09:03:16.155953+0000","last_became_peered":"2024-04-25T09:03:16.155953+0000","last_unstale":"2024-04-25T09:03:18.563836+0000","last_undegraded":"2024-04-25T09:03:18.563836+0000","last_fullsized":"2024-04-25T09:03:18.563836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:15.408869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563595+0000","last_change":"2024-04-25T09:03:17.158747+0000","last_active":"2024-04-25T09:03:18.563595+0000","last_peered":"2024-04-25T09:03:18.563595+0000","last_clean":"2024-04-25T09:03:18.563595+0000","last_became_active":"2024-04-25T09:03:17.158086+0000","last_became_peered":"2024-04-25T09:03:17.158086+0000","last_unstale":"2024-04-25T09:03:18.563595+0000","last_undegraded":"2024-04-25T09:03:18.563595+0000","last_fullsized":"2024-04-25T09:03:18.563595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:37.022619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564291+0000","last_change":"2024-04-25T09:03:17.167869+0000","last_active":"2024-04-25T09:03:18.564291+0000","last_peered":"2024-04-25T09:03:18.564291+0000","last_clean":"2024-04-25T09:03:18.564291+0000","last_became_active":"2024-04-25T09:03:17.164836+0000","last_became_peered":"2024-04-25T09:03:17.164836+0000","last_unstale":"2024-04-25T09:03:18.564291+0000","last_undegraded":"2024-04-25T09:03:18.564291+0000","last_fullsized":"2024-04-25T09:03:18.564291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:19.500716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172988+0000","last_change":"2024-04-25T09:03:16.162264+0000","last_active":"2024-04-25T09:03:18.172988+0000","last_peered":"2024-04-25T09:03:18.172988+0000","last_clean":"2024-04-25T09:03:18.172988+0000","last_became_active":"2024-04-25T09:03:16.161892+0000","last_became_peered":"2024-04-25T09:03:16.161892+0000","last_unstale":"2024-04-25T09:03:18.172988+0000","last_undegraded":"2024-04-25T09:03:18.172988+0000","last_fullsized":"2024-04-25T09:03:18.172988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:17.498426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151136+0000","last_change":"2024-04-25T09:03:17.156036+0000","last_active":"2024-04-25T09:03:18.151136+0000","last_peered":"2024-04-25T09:03:18.151136+0000","last_clean":"2024-04-25T09:03:18.151136+0000","last_became_active":"2024-04-25T09:03:17.154785+0000","last_became_peered":"2024-04-25T09:03:17.154785+0000","last_unstale":"2024-04-25T09:03:18.151136+0000","last_undegraded":"2024-04-25T09:03:18.151136+0000","last_fullsized":"2024-04-25T09:03:18.151136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172370+0000","last_change":"2024-04-25T09:03:16.161940+0000","last_active":"2024-04-25T09:03:18.172370+0000","last_peered":"2024-04-25T09:03:18.172370+0000","last_clean":"2024-04-25T09:03:18.172370+0000","last_became_active":"2024-04-25T09:03:16.161542+0000","last_became_peered":"2024-04-25T09:03:16.161542+0000","last_unstale":"2024-04-25T09:03:18.172370+0000","last_undegraded":"2024-04-25T09:03:18.172370+0000","last_fullsized":"2024-04-25T09:03:18.172370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:57.583736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566119+0000","last_change":"2024-04-25T09:03:17.157276+0000","last_active":"2024-04-25T09:03:18.566119+0000","last_peered":"2024-04-25T09:03:18.566119+0000","last_clean":"2024-04-25T09:03:18.566119+0000","last_became_active":"2024-04-25T09:03:17.156910+0000","last_became_peered":"2024-04-25T09:03:17.156910+0000","last_unstale":"2024-04-25T09:03:18.566119+0000","last_undegraded":"2024-04-25T09:03:18.566119+0000","last_fullsized":"2024-04-25T09:03:18.566119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:13.218030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566191+0000","last_change":"2024-04-25T09:03:16.167873+0000","last_active":"2024-04-25T09:03:18.566191+0000","last_peered":"2024-04-25T09:03:18.566191+0000","last_clean":"2024-04-25T09:03:18.566191+0000","last_became_active":"2024-04-25T09:03:16.166612+0000","last_became_peered":"2024-04-25T09:03:16.166612+0000","last_unstale":"2024-04-25T09:03:18.566191+0000","last_undegraded":"2024-04-25T09:03:18.566191+0000","last_fullsized":"2024-04-25T09:03:18.566191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566016+0000","last_change":"2024-04-25T09:03:17.156968+0000","last_active":"2024-04-25T09:03:18.566016+0000","last_peered":"2024-04-25T09:03:18.566016+0000","last_clean":"2024-04-25T09:03:18.566016+0000","last_became_active":"2024-04-25T09:03:17.156663+0000","last_became_peered":"2024-04-25T09:03:17.156663+0000","last_unstale":"2024-04-25T09:03:18.566016+0000","last_undegraded":"2024-04-25T09:03:18.566016+0000","last_fullsized":"2024-04-25T09:03:18.566016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.046413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154405+0000","last_change":"2024-04-25T09:03:16.157953+0000","last_active":"2024-04-25T09:03:18.154405+0000","last_peered":"2024-04-25T09:03:18.154405+0000","last_clean":"2024-04-25T09:03:18.154405+0000","last_became_active":"2024-04-25T09:03:16.157603+0000","last_became_peered":"2024-04-25T09:03:16.157603+0000","last_unstale":"2024-04-25T09:03:18.154405+0000","last_undegraded":"2024-04-25T09:03:18.154405+0000","last_fullsized":"2024-04-25T09:03:18.154405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.002913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":661394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":73218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:27.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.787+0000 7f608b200640 1 Processor -- start 2024-04-25T09:03:27.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.787+0000 7f608b200640 1 -- start start 2024-04-25T09:03:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.789+0000 7f608b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f608c0b56f0 0x7f608c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.789+0000 7f608b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b27a0 0x7f608c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.789+0000 7f608b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0d3110 con 0x7f608c0b6a90 2024-04-25T09:03:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.789+0000 7f608b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c161780 con 0x7f608c0b56f0 2024-04-25T09:03:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.789+0000 7f608b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0cedc0 con 0x7f608c0b27a0 2024-04-25T09:03:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.789+0000 7f6089400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f608c0b6a90 0x7f608c161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57618/0 (socket says 172.21.15.62:57618) 2024-04-25T09:03:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.789+0000 7f6089400640 1 -- 172.21.15.62:0/2914969619 learned_addr learned my addr 172.21.15.62:0/2914969619 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.789+0000 7f6089e00640 1 --2- 172.21.15.62:0/2914969619 >> v2:172.21.15.62:3300/0 conn(0x7f608c0b56f0 0x7f608c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.790+0000 7f608a800640 1 --2- 172.21.15.62:0/2914969619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b27a0 0x7f608c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.790+0000 7f6088a00640 1 -- 172.21.15.62:0/2914969619 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619698538 0 0) 0x7f608c0d3110 con 0x7f608c0b6a90 2024-04-25T09:03:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.790+0000 7f6089e00640 1 -- 172.21.15.62:0/2914969619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b27a0 msgr2=0x7f608c161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.790+0000 7f6089e00640 1 --2- 172.21.15.62:0/2914969619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b27a0 0x7f608c161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.790+0000 7f6089e00640 1 -- 172.21.15.62:0/2914969619 >> v1:172.21.15.62:6789/0 conn(0x7f608c0b6a90 legacy=0x7f608c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.790+0000 7f6089e00640 1 -- 172.21.15.62:0/2914969619 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0d33f0 con 0x7f608c0b56f0 2024-04-25T09:03:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.791+0000 7f6089e00640 1 --2- 172.21.15.62:0/2914969619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b56f0 0x7f608c160d00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f607c0097a0 tx=0x7f607c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650038404edab258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.791+0000 7f6088a00640 1 -- 172.21.15.62:0/2914969619 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c037070 con 0x7f608c0b56f0 2024-04-25T09:03:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.791+0000 7f6088a00640 1 -- 172.21.15.62:0/2914969619 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c032440 con 0x7f608c0b56f0 2024-04-25T09:03:27.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.791+0000 7f6088a00640 1 -- 172.21.15.62:0/2914969619 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c032810 con 0x7f608c0b56f0 2024-04-25T09:03:27.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.792+0000 7f608b200640 1 -- 172.21.15.62:0/2914969619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b56f0 msgr2=0x7f608c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.792+0000 7f608b200640 1 --2- 172.21.15.62:0/2914969619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b56f0 0x7f608c160d00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f607c0097a0 tx=0x7f607c02a180 comp rx=0 tx=0).stop 2024-04-25T09:03:27.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.792+0000 7f608b200640 1 -- 172.21.15.62:0/2914969619 shutdown_connections 2024-04-25T09:03:27.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.792+0000 7f608b200640 1 --2- 172.21.15.62:0/2914969619 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b27a0 0x7f608c161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.792+0000 7f608b200640 1 --2- 172.21.15.62:0/2914969619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b56f0 0x7f608c160d00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.792+0000 7f608b200640 1 -- 172.21.15.62:0/2914969619 >> 172.21.15.62:0/2914969619 conn(0x7f608c066d10 msgr2=0x7f608c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.792+0000 7f608b200640 1 -- 172.21.15.62:0/2914969619 shutdown_connections 2024-04-25T09:03:27.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.792+0000 7f608b200640 1 -- 172.21.15.62:0/2914969619 wait complete. 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.794+0000 7f608b200640 1 Processor -- start 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.794+0000 7f608b200640 1 -- start start 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f608b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b27a0 0x7f608c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f608b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c0b56f0 0x7f608c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f608b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b6a90 0x7f608c158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f608b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0b2d60 con 0x7f608c0b27a0 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f608b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0cea50 con 0x7f608c0b56f0 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f608b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0b2ee0 con 0x7f608c0b6a90 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f6089e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b27a0 0x7f608c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f6089e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b27a0 0x7f608c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55084/0 (socket says 172.21.15.62:55084) 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.795+0000 7f6089e00640 1 -- 172.21.15.62:0/1234274266 learned_addr learned my addr 172.21.15.62:0/1234274266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f608a800640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b6a90 0x7f608c158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f6089e00640 1 -- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b6a90 msgr2=0x7f608c158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f6089e00640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b6a90 0x7f608c158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f6089400640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c0b56f0 0x7f608c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f6089e00640 1 -- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c0b56f0 msgr2=0x7f608c15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f6089e00640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c0b56f0 0x7f608c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f6089e00640 1 -- 172.21.15.62:0/1234274266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c15bfd0 con 0x7f608c0b27a0 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f608a800640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b6a90 0x7f608c158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f6089e00640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b27a0 0x7f608c0ce020 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f607c0098c0 tx=0x7f607c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.796+0000 7f6082000640 1 -- 172.21.15.62:0/1234274266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c037040 con 0x7f608c0b27a0 2024-04-25T09:03:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.797+0000 7f6082000640 1 -- 172.21.15.62:0/1234274266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c042070 con 0x7f608c0b27a0 2024-04-25T09:03:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.797+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c0c75d0 con 0x7f608c0b27a0 2024-04-25T09:03:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.797+0000 7f6082000640 1 -- 172.21.15.62:0/1234274266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c030bb0 con 0x7f608c0b27a0 2024-04-25T09:03:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.797+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c0ca1b0 con 0x7f608c0b27a0 2024-04-25T09:03:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.798+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c0b5bb0 con 0x7f608c0b27a0 2024-04-25T09:03:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.799+0000 7f6082000640 1 -- 172.21.15.62:0/1234274266 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f607c047460 con 0x7f608c0b27a0 2024-04-25T09:03:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.800+0000 7f6082000640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b500 0x7f606806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.800+0000 7f6082000640 1 -- 172.21.15.62:0/1234274266 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f607c0aae10 con 0x7f608c0b27a0 2024-04-25T09:03:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.800+0000 7f6089400640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b500 0x7f606806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.801+0000 7f6089400640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b500 0x7f606806d9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f608c159ef0 tx=0x7f6074009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.804+0000 7f6082000640 1 -- 172.21.15.62:0/1234274266 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f608c0b5bb0 con 0x7f608c0b27a0 2024-04-25T09:03:27.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:27.992+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f608c04ef20 con 0x7f606806b500 2024-04-25T09:03:27.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.011+0000 7f6082000640 1 -- 172.21.15.62:0/1234274266 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251833 (secure 0 0 0) 0x7f608c04ef20 con 0x7f606806b500 2024-04-25T09:03:28.002 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:28.003 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:03:28.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b500 msgr2=0x7f606806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b500 0x7f606806d9c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f608c159ef0 tx=0x7f6074009040 comp rx=0 tx=0).stop 2024-04-25T09:03:28.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b27a0 msgr2=0x7f608c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b27a0 0x7f608c0ce020 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f607c0098c0 tx=0x7f607c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 shutdown_connections 2024-04-25T09:03:28.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b500 0x7f606806d9c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0b6a90 0x7f608c158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c0b56f0 0x7f608c15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 --2- 172.21.15.62:0/1234274266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0b27a0 0x7f608c0ce020 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 >> 172.21.15.62:0/1234274266 conn(0x7f608c066d10 msgr2=0x7f608c0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 shutdown_connections 2024-04-25T09:03:28.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.017+0000 7f608b200640 1 -- 172.21.15.62:0/1234274266 wait complete. 2024-04-25T09:03:28.023 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T09:03:27.886821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.013738"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172705+0000","last_change":"2024-04-25T09:03:17.179077+0000","last_active":"2024-04-25T09:03:18.172705+0000","last_peered":"2024-04-25T09:03:18.172705+0000","last_clean":"2024-04-25T09:03:18.172705+0000","last_became_active":"2024-04-25T09:03:17.178710+0000","last_became_peered":"2024-04-25T09:03:17.178710+0000","last_unstale":"2024-04-25T09:03:18.172705+0000","last_undegraded":"2024-04-25T09:03:18.172705+0000","last_fullsized":"2024-04-25T09:03:18.172705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565717+0000","last_change":"2024-04-25T09:03:17.164128+0000","last_active":"2024-04-25T09:03:18.565717+0000","last_peered":"2024-04-25T09:03:18.565717+0000","last_clean":"2024-04-25T09:03:18.565717+0000","last_became_active":"2024-04-25T09:03:17.163800+0000","last_became_peered":"2024-04-25T09:03:17.163800+0000","last_unstale":"2024-04-25T09:03:18.565717+0000","last_undegraded":"2024-04-25T09:03:18.565717+0000","last_fullsized":"2024-04-25T09:03:18.565717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:08.746742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565545+0000","last_change":"2024-04-25T09:03:17.167505+0000","last_active":"2024-04-25T09:03:18.565545+0000","last_peered":"2024-04-25T09:03:18.565545+0000","last_clean":"2024-04-25T09:03:18.565545+0000","last_became_active":"2024-04-25T09:03:17.167194+0000","last_became_peered":"2024-04-25T09:03:17.167194+0000","last_unstale":"2024-04-25T09:03:18.565545+0000","last_undegraded":"2024-04-25T09:03:18.565545+0000","last_fullsized":"2024-04-25T09:03:18.565545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:39.624252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562099+0000","last_change":"2024-04-25T09:03:17.162871+0000","last_active":"2024-04-25T09:03:18.562099+0000","last_peered":"2024-04-25T09:03:18.562099+0000","last_clean":"2024-04-25T09:03:18.562099+0000","last_became_active":"2024-04-25T09:03:17.162297+0000","last_became_peered":"2024-04-25T09:03:17.162297+0000","last_unstale":"2024-04-25T09:03:18.562099+0000","last_undegraded":"2024-04-25T09:03:18.562099+0000","last_fullsized":"2024-04-25T09:03:18.562099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171734+0000","last_change":"2024-04-25T09:03:17.172274+0000","last_active":"2024-04-25T09:03:18.171734+0000","last_peered":"2024-04-25T09:03:18.171734+0000","last_clean":"2024-04-25T09:03:18.171734+0000","last_became_active":"2024-04-25T09:03:17.171794+0000","last_became_peered":"2024-04-25T09:03:17.171794+0000","last_unstale":"2024-04-25T09:03:18.171734+0000","last_undegraded":"2024-04-25T09:03:18.171734+0000","last_fullsized":"2024-04-25T09:03:18.171734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562442+0000","last_change":"2024-04-25T09:03:17.158419+0000","last_active":"2024-04-25T09:03:18.562442+0000","last_peered":"2024-04-25T09:03:18.562442+0000","last_clean":"2024-04-25T09:03:18.562442+0000","last_became_active":"2024-04-25T09:03:17.155230+0000","last_became_peered":"2024-04-25T09:03:17.155230+0000","last_unstale":"2024-04-25T09:03:18.562442+0000","last_undegraded":"2024-04-25T09:03:18.562442+0000","last_fullsized":"2024-04-25T09:03:18.562442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172872+0000","last_change":"2024-04-25T09:03:17.165890+0000","last_active":"2024-04-25T09:03:18.172872+0000","last_peered":"2024-04-25T09:03:18.172872+0000","last_clean":"2024-04-25T09:03:18.172872+0000","last_became_active":"2024-04-25T09:03:17.164725+0000","last_became_peered":"2024-04-25T09:03:17.164725+0000","last_unstale":"2024-04-25T09:03:18.172872+0000","last_undegraded":"2024-04-25T09:03:18.172872+0000","last_fullsized":"2024-04-25T09:03:18.172872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:57.554441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563608+0000","last_change":"2024-04-25T09:03:17.162764+0000","last_active":"2024-04-25T09:03:18.563608+0000","last_peered":"2024-04-25T09:03:18.563608+0000","last_clean":"2024-04-25T09:03:18.563608+0000","last_became_active":"2024-04-25T09:03:17.162442+0000","last_became_peered":"2024-04-25T09:03:17.162442+0000","last_unstale":"2024-04-25T09:03:18.563608+0000","last_undegraded":"2024-04-25T09:03:18.563608+0000","last_fullsized":"2024-04-25T09:03:18.563608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:38.767433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151609+0000","last_change":"2024-04-25T09:03:17.156720+0000","last_active":"2024-04-25T09:03:18.151609+0000","last_peered":"2024-04-25T09:03:18.151609+0000","last_clean":"2024-04-25T09:03:18.151609+0000","last_became_active":"2024-04-25T09:03:17.156372+0000","last_became_peered":"2024-04-25T09:03:17.156372+0000","last_unstale":"2024-04-25T09:03:18.151609+0000","last_undegraded":"2024-04-25T09:03:18.151609+0000","last_fullsized":"2024-04-25T09:03:18.151609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172106+0000","last_change":"2024-04-25T09:03:17.160604+0000","last_active":"2024-04-25T09:03:18.172106+0000","last_peered":"2024-04-25T09:03:18.172106+0000","last_clean":"2024-04-25T09:03:18.172106+0000","last_became_active":"2024-04-25T09:03:17.157610+0000","last_became_peered":"2024-04-25T09:03:17.157610+0000","last_unstale":"2024-04-25T09:03:18.172106+0000","last_undegraded":"2024-04-25T09:03:18.172106+0000","last_fullsized":"2024-04-25T09:03:18.172106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:08.746742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172219+0000","last_change":"2024-04-25T09:03:17.172025+0000","last_active":"2024-04-25T09:03:18.172219+0000","last_peered":"2024-04-25T09:03:18.172219+0000","last_clean":"2024-04-25T09:03:18.172219+0000","last_became_active":"2024-04-25T09:03:17.171653+0000","last_became_peered":"2024-04-25T09:03:17.171653+0000","last_unstale":"2024-04-25T09:03:18.172219+0000","last_undegraded":"2024-04-25T09:03:18.172219+0000","last_fullsized":"2024-04-25T09:03:18.172219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:22.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172409+0000","last_change":"2024-04-25T09:03:17.155748+0000","last_active":"2024-04-25T09:03:18.172409+0000","last_peered":"2024-04-25T09:03:18.172409+0000","last_clean":"2024-04-25T09:03:18.172409+0000","last_became_active":"2024-04-25T09:03:17.155156+0000","last_became_peered":"2024-04-25T09:03:17.155156+0000","last_unstale":"2024-04-25T09:03:18.172409+0000","last_undegraded":"2024-04-25T09:03:18.172409+0000","last_fullsized":"2024-04-25T09:03:18.172409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:13.218030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563558+0000","last_change":"2024-04-25T09:03:17.162578+0000","last_active":"2024-04-25T09:03:18.563558+0000","last_peered":"2024-04-25T09:03:18.563558+0000","last_clean":"2024-04-25T09:03:18.563558+0000","last_became_active":"2024-04-25T09:03:17.162284+0000","last_became_peered":"2024-04-25T09:03:17.162284+0000","last_unstale":"2024-04-25T09:03:18.563558+0000","last_undegraded":"2024-04-25T09:03:18.563558+0000","last_fullsized":"2024-04-25T09:03:18.563558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:34.254072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563388+0000","last_change":"2024-04-25T09:03:17.150222+0000","last_active":"2024-04-25T09:03:18.563388+0000","last_peered":"2024-04-25T09:03:18.563388+0000","last_clean":"2024-04-25T09:03:18.563388+0000","last_became_active":"2024-04-25T09:03:17.149717+0000","last_became_peered":"2024-04-25T09:03:17.149717+0000","last_unstale":"2024-04-25T09:03:18.563388+0000","last_undegraded":"2024-04-25T09:03:18.563388+0000","last_fullsized":"2024-04-25T09:03:18.563388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:00.222773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564802+0000","last_change":"2024-04-25T09:03:17.156094+0000","last_active":"2024-04-25T09:03:18.564802+0000","last_peered":"2024-04-25T09:03:18.564802+0000","last_clean":"2024-04-25T09:03:18.564802+0000","last_became_active":"2024-04-25T09:03:17.155818+0000","last_became_peered":"2024-04-25T09:03:17.155818+0000","last_unstale":"2024-04-25T09:03:18.564802+0000","last_undegraded":"2024-04-25T09:03:18.564802+0000","last_fullsized":"2024-04-25T09:03:18.564802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.326298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561761+0000","last_change":"2024-04-25T09:03:17.164616+0000","last_active":"2024-04-25T09:03:18.561761+0000","last_peered":"2024-04-25T09:03:18.561761+0000","last_clean":"2024-04-25T09:03:18.561761+0000","last_became_active":"2024-04-25T09:03:17.158855+0000","last_became_peered":"2024-04-25T09:03:17.158855+0000","last_unstale":"2024-04-25T09:03:18.561761+0000","last_undegraded":"2024-04-25T09:03:18.561761+0000","last_fullsized":"2024-04-25T09:03:18.561761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565221+0000","last_change":"2024-04-25T09:03:17.163995+0000","last_active":"2024-04-25T09:03:18.565221+0000","last_peered":"2024-04-25T09:03:18.565221+0000","last_clean":"2024-04-25T09:03:18.565221+0000","last_became_active":"2024-04-25T09:03:17.162714+0000","last_became_peered":"2024-04-25T09:03:17.162714+0000","last_unstale":"2024-04-25T09:03:18.565221+0000","last_undegraded":"2024-04-25T09:03:18.565221+0000","last_fullsized":"2024-04-25T09:03:18.565221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:54.326567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567960+0000","last_change":"2024-04-25T09:03:17.159140+0000","last_active":"2024-04-25T09:03:18.567960+0000","last_peered":"2024-04-25T09:03:18.567960+0000","last_clean":"2024-04-25T09:03:18.567960+0000","last_became_active":"2024-04-25T09:03:17.158674+0000","last_became_peered":"2024-04-25T09:03:17.158674+0000","last_unstale":"2024-04-25T09:03:18.567960+0000","last_undegraded":"2024-04-25T09:03:18.567960+0000","last_fullsized":"2024-04-25T09:03:18.567960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564565+0000","last_change":"2024-04-25T09:03:17.153159+0000","last_active":"2024-04-25T09:03:18.564565+0000","last_peered":"2024-04-25T09:03:18.564565+0000","last_clean":"2024-04-25T09:03:18.564565+0000","last_became_active":"2024-04-25T09:03:17.152817+0000","last_became_peered":"2024-04-25T09:03:17.152817+0000","last_unstale":"2024-04-25T09:03:18.564565+0000","last_undegraded":"2024-04-25T09:03:18.564565+0000","last_fullsized":"2024-04-25T09:03:18.564565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:38.133534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172856+0000","last_change":"2024-04-25T09:03:17.155424+0000","last_active":"2024-04-25T09:03:18.172856+0000","last_peered":"2024-04-25T09:03:18.172856+0000","last_clean":"2024-04-25T09:03:18.172856+0000","last_became_active":"2024-04-25T09:03:17.154812+0000","last_became_peered":"2024-04-25T09:03:17.154812+0000","last_unstale":"2024-04-25T09:03:18.172856+0000","last_undegraded":"2024-04-25T09:03:18.172856+0000","last_fullsized":"2024-04-25T09:03:18.172856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:08.323326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154338+0000","last_change":"2024-04-25T09:03:17.156092+0000","last_active":"2024-04-25T09:03:18.154338+0000","last_peered":"2024-04-25T09:03:18.154338+0000","last_clean":"2024-04-25T09:03:18.154338+0000","last_became_active":"2024-04-25T09:03:17.153705+0000","last_became_peered":"2024-04-25T09:03:17.153705+0000","last_unstale":"2024-04-25T09:03:18.154338+0000","last_undegraded":"2024-04-25T09:03:18.154338+0000","last_fullsized":"2024-04-25T09:03:18.154338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:38.767433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150850+0000","last_change":"2024-04-25T09:03:17.155198+0000","last_active":"2024-04-25T09:03:18.150850+0000","last_peered":"2024-04-25T09:03:18.150850+0000","last_clean":"2024-04-25T09:03:18.150850+0000","last_became_active":"2024-04-25T09:03:17.154848+0000","last_became_peered":"2024-04-25T09:03:17.154848+0000","last_unstale":"2024-04-25T09:03:18.150850+0000","last_undegraded":"2024-04-25T09:03:18.150850+0000","last_fullsized":"2024-04-25T09:03:18.150850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:38.838938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564958+0000","last_change":"2024-04-25T09:03:17.161601+0000","last_active":"2024-04-25T09:03:18.564958+0000","last_peered":"2024-04-25T09:03:18.564958+0000","last_clean":"2024-04-25T09:03:18.564958+0000","last_became_active":"2024-04-25T09:03:17.161303+0000","last_became_peered":"2024-04-25T09:03:17.161303+0000","last_unstale":"2024-04-25T09:03:18.564958+0000","last_undegraded":"2024-04-25T09:03:18.564958+0000","last_fullsized":"2024-04-25T09:03:18.564958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:14.651686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562409+0000","last_change":"2024-04-25T09:03:17.168111+0000","last_active":"2024-04-25T09:03:18.562409+0000","last_peered":"2024-04-25T09:03:18.562409+0000","last_clean":"2024-04-25T09:03:18.562409+0000","last_became_active":"2024-04-25T09:03:17.165747+0000","last_became_peered":"2024-04-25T09:03:17.165747+0000","last_unstale":"2024-04-25T09:03:18.562409+0000","last_undegraded":"2024-04-25T09:03:18.562409+0000","last_fullsized":"2024-04-25T09:03:18.562409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:15.948906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565923+0000","last_change":"2024-04-25T09:03:17.161196+0000","last_active":"2024-04-25T09:03:18.565923+0000","last_peered":"2024-04-25T09:03:18.565923+0000","last_clean":"2024-04-25T09:03:18.565923+0000","last_became_active":"2024-04-25T09:03:17.158516+0000","last_became_peered":"2024-04-25T09:03:17.158516+0000","last_unstale":"2024-04-25T09:03:18.565923+0000","last_undegraded":"2024-04-25T09:03:18.565923+0000","last_fullsized":"2024-04-25T09:03:18.565923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:22.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563329+0000","last_change":"2024-04-25T09:03:17.164003+0000","last_active":"2024-04-25T09:03:18.563329+0000","last_peered":"2024-04-25T09:03:18.563329+0000","last_clean":"2024-04-25T09:03:18.563329+0000","last_became_active":"2024-04-25T09:03:17.162280+0000","last_became_peered":"2024-04-25T09:03:17.162280+0000","last_unstale":"2024-04-25T09:03:18.563329+0000","last_undegraded":"2024-04-25T09:03:18.563329+0000","last_fullsized":"2024-04-25T09:03:18.563329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:16.897576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564687+0000","last_change":"2024-04-25T09:03:17.155201+0000","last_active":"2024-04-25T09:03:18.564687+0000","last_peered":"2024-04-25T09:03:18.564687+0000","last_clean":"2024-04-25T09:03:18.564687+0000","last_became_active":"2024-04-25T09:03:17.154820+0000","last_became_peered":"2024-04-25T09:03:17.154820+0000","last_unstale":"2024-04-25T09:03:18.564687+0000","last_undegraded":"2024-04-25T09:03:18.564687+0000","last_fullsized":"2024-04-25T09:03:18.564687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:50.727941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563484+0000","last_change":"2024-04-25T09:03:17.155327+0000","last_active":"2024-04-25T09:03:18.563484+0000","last_peered":"2024-04-25T09:03:18.563484+0000","last_clean":"2024-04-25T09:03:18.563484+0000","last_became_active":"2024-04-25T09:03:17.155036+0000","last_became_peered":"2024-04-25T09:03:17.155036+0000","last_unstale":"2024-04-25T09:03:18.563484+0000","last_undegraded":"2024-04-25T09:03:18.563484+0000","last_fullsized":"2024-04-25T09:03:18.563484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:16.897576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565403+0000","last_change":"2024-04-25T09:03:17.157840+0000","last_active":"2024-04-25T09:03:18.565403+0000","last_peered":"2024-04-25T09:03:18.565403+0000","last_clean":"2024-04-25T09:03:18.565403+0000","last_became_active":"2024-04-25T09:03:17.157246+0000","last_became_peered":"2024-04-25T09:03:17.157246+0000","last_unstale":"2024-04-25T09:03:18.565403+0000","last_undegraded":"2024-04-25T09:03:18.565403+0000","last_fullsized":"2024-04-25T09:03:18.565403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:03.908134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563239+0000","last_change":"2024-04-25T09:03:17.170837+0000","last_active":"2024-04-25T09:03:18.563239+0000","last_peered":"2024-04-25T09:03:18.563239+0000","last_clean":"2024-04-25T09:03:18.563239+0000","last_became_active":"2024-04-25T09:03:17.166967+0000","last_became_peered":"2024-04-25T09:03:17.166967+0000","last_unstale":"2024-04-25T09:03:18.563239+0000","last_undegraded":"2024-04-25T09:03:18.563239+0000","last_fullsized":"2024-04-25T09:03:18.563239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:36.933011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567140+0000","last_change":"2024-04-25T09:03:17.145733+0000","last_active":"2024-04-25T09:03:18.567140+0000","last_peered":"2024-04-25T09:03:18.567140+0000","last_clean":"2024-04-25T09:03:18.567140+0000","last_became_active":"2024-04-25T09:03:17.145233+0000","last_became_peered":"2024-04-25T09:03:17.145233+0000","last_unstale":"2024-04-25T09:03:18.567140+0000","last_undegraded":"2024-04-25T09:03:18.567140+0000","last_fullsized":"2024-04-25T09:03:18.567140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:18.503932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561657+0000","last_change":"2024-04-25T09:03:17.157243+0000","last_active":"2024-04-25T09:03:18.561657+0000","last_peered":"2024-04-25T09:03:18.561657+0000","last_clean":"2024-04-25T09:03:18.561657+0000","last_became_active":"2024-04-25T09:03:17.156940+0000","last_became_peered":"2024-04-25T09:03:17.156940+0000","last_unstale":"2024-04-25T09:03:18.561657+0000","last_undegraded":"2024-04-25T09:03:18.561657+0000","last_fullsized":"2024-04-25T09:03:18.561657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150683+0000","last_change":"2024-04-25T09:03:17.156642+0000","last_active":"2024-04-25T09:03:18.150683+0000","last_peered":"2024-04-25T09:03:18.150683+0000","last_clean":"2024-04-25T09:03:18.150683+0000","last_became_active":"2024-04-25T09:03:17.155097+0000","last_became_peered":"2024-04-25T09:03:17.155097+0000","last_unstale":"2024-04-25T09:03:18.150683+0000","last_undegraded":"2024-04-25T09:03:18.150683+0000","last_fullsized":"2024-04-25T09:03:18.150683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:50.936976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565202+0000","last_change":"2024-04-25T09:03:16.166854+0000","last_active":"2024-04-25T09:03:18.565202+0000","last_peered":"2024-04-25T09:03:18.565202+0000","last_clean":"2024-04-25T09:03:18.565202+0000","last_became_active":"2024-04-25T09:03:16.166523+0000","last_became_peered":"2024-04-25T09:03:16.166523+0000","last_unstale":"2024-04-25T09:03:18.565202+0000","last_undegraded":"2024-04-25T09:03:18.565202+0000","last_fullsized":"2024-04-25T09:03:18.565202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154448+0000","last_change":"2024-04-25T09:03:17.158882+0000","last_active":"2024-04-25T09:03:18.154448+0000","last_peered":"2024-04-25T09:03:18.154448+0000","last_clean":"2024-04-25T09:03:18.154448+0000","last_became_active":"2024-04-25T09:03:17.158589+0000","last_became_peered":"2024-04-25T09:03:17.158589+0000","last_unstale":"2024-04-25T09:03:18.154448+0000","last_undegraded":"2024-04-25T09:03:18.154448+0000","last_fullsized":"2024-04-25T09:03:18.154448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:19.521616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563570+0000","last_change":"2024-04-25T09:03:16.168242+0000","last_active":"2024-04-25T09:03:18.563570+0000","last_peered":"2024-04-25T09:03:18.563570+0000","last_clean":"2024-04-25T09:03:18.563570+0000","last_became_active":"2024-04-25T09:03:16.167927+0000","last_became_peered":"2024-04-25T09:03:16.167927+0000","last_unstale":"2024-04-25T09:03:18.563570+0000","last_undegraded":"2024-04-25T09:03:18.563570+0000","last_fullsized":"2024-04-25T09:03:18.563570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:15.892070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562728+0000","last_change":"2024-04-25T09:03:16.163666+0000","last_active":"2024-04-25T09:03:18.562728+0000","last_peered":"2024-04-25T09:03:18.562728+0000","last_clean":"2024-04-25T09:03:18.562728+0000","last_became_active":"2024-04-25T09:03:16.163321+0000","last_became_peered":"2024-04-25T09:03:16.163321+0000","last_unstale":"2024-04-25T09:03:18.562728+0000","last_undegraded":"2024-04-25T09:03:18.562728+0000","last_fullsized":"2024-04-25T09:03:18.562728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564262+0000","last_change":"2024-04-25T09:03:17.163289+0000","last_active":"2024-04-25T09:03:18.564262+0000","last_peered":"2024-04-25T09:03:18.564262+0000","last_clean":"2024-04-25T09:03:18.564262+0000","last_became_active":"2024-04-25T09:03:17.162997+0000","last_became_peered":"2024-04-25T09:03:17.162997+0000","last_unstale":"2024-04-25T09:03:18.564262+0000","last_undegraded":"2024-04-25T09:03:18.564262+0000","last_fullsized":"2024-04-25T09:03:18.564262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:19.521616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.561915+0000","last_change":"2024-04-25T09:03:16.146080+0000","last_active":"2024-04-25T09:03:18.561915+0000","last_peered":"2024-04-25T09:03:18.561915+0000","last_clean":"2024-04-25T09:03:18.561915+0000","last_became_active":"2024-04-25T09:03:16.145743+0000","last_became_peered":"2024-04-25T09:03:16.145743+0000","last_unstale":"2024-04-25T09:03:18.561915+0000","last_undegraded":"2024-04-25T09:03:18.561915+0000","last_fullsized":"2024-04-25T09:03:18.561915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:37.833432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154329+0000","last_change":"2024-04-25T09:03:17.146618+0000","last_active":"2024-04-25T09:03:18.154329+0000","last_peered":"2024-04-25T09:03:18.154329+0000","last_clean":"2024-04-25T09:03:18.154329+0000","last_became_active":"2024-04-25T09:03:17.146174+0000","last_became_peered":"2024-04-25T09:03:17.146174+0000","last_unstale":"2024-04-25T09:03:18.154329+0000","last_undegraded":"2024-04-25T09:03:18.154329+0000","last_fullsized":"2024-04-25T09:03:18.154329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:50.727941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151477+0000","last_change":"2024-04-25T09:03:16.159981+0000","last_active":"2024-04-25T09:03:18.151477+0000","last_peered":"2024-04-25T09:03:18.151477+0000","last_clean":"2024-04-25T09:03:18.151477+0000","last_became_active":"2024-04-25T09:03:16.159115+0000","last_became_peered":"2024-04-25T09:03:16.159115+0000","last_unstale":"2024-04-25T09:03:18.151477+0000","last_undegraded":"2024-04-25T09:03:18.151477+0000","last_fullsized":"2024-04-25T09:03:18.151477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563445+0000","last_change":"2024-04-25T09:03:17.163768+0000","last_active":"2024-04-25T09:03:18.563445+0000","last_peered":"2024-04-25T09:03:18.563445+0000","last_clean":"2024-04-25T09:03:18.563445+0000","last_became_active":"2024-04-25T09:03:17.162151+0000","last_became_peered":"2024-04-25T09:03:17.162151+0000","last_unstale":"2024-04-25T09:03:18.563445+0000","last_undegraded":"2024-04-25T09:03:18.563445+0000","last_fullsized":"2024-04-25T09:03:18.563445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:34.254072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172215+0000","last_change":"2024-04-25T09:03:16.169333+0000","last_active":"2024-04-25T09:03:18.172215+0000","last_peered":"2024-04-25T09:03:18.172215+0000","last_clean":"2024-04-25T09:03:18.172215+0000","last_became_active":"2024-04-25T09:03:16.167170+0000","last_became_peered":"2024-04-25T09:03:16.167170+0000","last_unstale":"2024-04-25T09:03:18.172215+0000","last_undegraded":"2024-04-25T09:03:18.172215+0000","last_fullsized":"2024-04-25T09:03:18.172215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:37.833432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566351+0000","last_change":"2024-04-25T09:03:17.168947+0000","last_active":"2024-04-25T09:03:18.566351+0000","last_peered":"2024-04-25T09:03:18.566351+0000","last_clean":"2024-04-25T09:03:18.566351+0000","last_became_active":"2024-04-25T09:03:17.168658+0000","last_became_peered":"2024-04-25T09:03:17.168658+0000","last_unstale":"2024-04-25T09:03:18.566351+0000","last_undegraded":"2024-04-25T09:03:18.566351+0000","last_fullsized":"2024-04-25T09:03:18.566351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:31.054318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567577+0000","last_change":"2024-04-25T09:03:16.160547+0000","last_active":"2024-04-25T09:03:18.567577+0000","last_peered":"2024-04-25T09:03:18.567577+0000","last_clean":"2024-04-25T09:03:18.567577+0000","last_became_active":"2024-04-25T09:03:16.160169+0000","last_became_peered":"2024-04-25T09:03:16.160169+0000","last_unstale":"2024-04-25T09:03:18.567577+0000","last_undegraded":"2024-04-25T09:03:18.567577+0000","last_fullsized":"2024-04-25T09:03:18.567577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564136+0000","last_change":"2024-04-25T09:03:17.161468+0000","last_active":"2024-04-25T09:03:18.564136+0000","last_peered":"2024-04-25T09:03:18.564136+0000","last_clean":"2024-04-25T09:03:18.564136+0000","last_became_active":"2024-04-25T09:03:17.159824+0000","last_became_peered":"2024-04-25T09:03:17.159824+0000","last_unstale":"2024-04-25T09:03:18.564136+0000","last_undegraded":"2024-04-25T09:03:18.564136+0000","last_fullsized":"2024-04-25T09:03:18.564136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:37.830284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562202+0000","last_change":"2024-04-25T09:03:16.155839+0000","last_active":"2024-04-25T09:03:18.562202+0000","last_peered":"2024-04-25T09:03:18.562202+0000","last_clean":"2024-04-25T09:03:18.562202+0000","last_became_active":"2024-04-25T09:03:16.155363+0000","last_became_peered":"2024-04-25T09:03:16.155363+0000","last_unstale":"2024-04-25T09:03:18.562202+0000","last_undegraded":"2024-04-25T09:03:18.562202+0000","last_fullsized":"2024-04-25T09:03:18.562202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:13.008112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172333+0000","last_change":"2024-04-25T09:03:17.158443+0000","last_active":"2024-04-25T09:03:18.172333+0000","last_peered":"2024-04-25T09:03:18.172333+0000","last_clean":"2024-04-25T09:03:18.172333+0000","last_became_active":"2024-04-25T09:03:17.158153+0000","last_became_peered":"2024-04-25T09:03:17.158153+0000","last_unstale":"2024-04-25T09:03:18.172333+0000","last_undegraded":"2024-04-25T09:03:18.172333+0000","last_fullsized":"2024-04-25T09:03:18.172333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.046413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567015+0000","last_change":"2024-04-25T09:03:16.170150+0000","last_active":"2024-04-25T09:03:18.567015+0000","last_peered":"2024-04-25T09:03:18.567015+0000","last_clean":"2024-04-25T09:03:18.567015+0000","last_became_active":"2024-04-25T09:03:16.169528+0000","last_became_peered":"2024-04-25T09:03:16.169528+0000","last_unstale":"2024-04-25T09:03:18.567015+0000","last_undegraded":"2024-04-25T09:03:18.567015+0000","last_fullsized":"2024-04-25T09:03:18.567015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563761+0000","last_change":"2024-04-25T09:03:17.161896+0000","last_active":"2024-04-25T09:03:18.563761+0000","last_peered":"2024-04-25T09:03:18.563761+0000","last_clean":"2024-04-25T09:03:18.563761+0000","last_became_active":"2024-04-25T09:03:17.161609+0000","last_became_peered":"2024-04-25T09:03:17.161609+0000","last_unstale":"2024-04-25T09:03:18.563761+0000","last_undegraded":"2024-04-25T09:03:18.563761+0000","last_fullsized":"2024-04-25T09:03:18.563761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563685+0000","last_change":"2024-04-25T09:03:16.150079+0000","last_active":"2024-04-25T09:03:18.563685+0000","last_peered":"2024-04-25T09:03:18.563685+0000","last_clean":"2024-04-25T09:03:18.563685+0000","last_became_active":"2024-04-25T09:03:16.149418+0000","last_became_peered":"2024-04-25T09:03:16.149418+0000","last_unstale":"2024-04-25T09:03:18.563685+0000","last_undegraded":"2024-04-25T09:03:18.563685+0000","last_fullsized":"2024-04-25T09:03:18.563685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:33.248566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564016+0000","last_change":"2024-04-25T09:03:17.169709+0000","last_active":"2024-04-25T09:03:18.564016+0000","last_peered":"2024-04-25T09:03:18.564016+0000","last_clean":"2024-04-25T09:03:18.564016+0000","last_became_active":"2024-04-25T09:03:17.169402+0000","last_became_peered":"2024-04-25T09:03:17.169402+0000","last_unstale":"2024-04-25T09:03:18.564016+0000","last_undegraded":"2024-04-25T09:03:18.564016+0000","last_fullsized":"2024-04-25T09:03:18.564016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:23.474225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171619+0000","last_change":"2024-04-25T09:03:16.164546+0000","last_active":"2024-04-25T09:03:18.171619+0000","last_peered":"2024-04-25T09:03:18.171619+0000","last_clean":"2024-04-25T09:03:18.171619+0000","last_became_active":"2024-04-25T09:03:16.162541+0000","last_became_peered":"2024-04-25T09:03:16.162541+0000","last_unstale":"2024-04-25T09:03:18.171619+0000","last_undegraded":"2024-04-25T09:03:18.171619+0000","last_fullsized":"2024-04-25T09:03:18.171619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:15.408869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150694+0000","last_change":"2024-04-25T09:03:17.156971+0000","last_active":"2024-04-25T09:03:18.150694+0000","last_peered":"2024-04-25T09:03:18.150694+0000","last_clean":"2024-04-25T09:03:18.150694+0000","last_became_active":"2024-04-25T09:03:17.156624+0000","last_became_peered":"2024-04-25T09:03:17.156624+0000","last_unstale":"2024-04-25T09:03:18.150694+0000","last_undegraded":"2024-04-25T09:03:18.150694+0000","last_fullsized":"2024-04-25T09:03:18.150694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:16.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153276+0000","last_change":"2024-04-25T09:03:16.155291+0000","last_active":"2024-04-25T09:03:18.153276+0000","last_peered":"2024-04-25T09:03:18.153276+0000","last_clean":"2024-04-25T09:03:18.153276+0000","last_became_active":"2024-04-25T09:03:16.154984+0000","last_became_peered":"2024-04-25T09:03:16.154984+0000","last_unstale":"2024-04-25T09:03:18.153276+0000","last_undegraded":"2024-04-25T09:03:18.153276+0000","last_fullsized":"2024-04-25T09:03:18.153276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562872+0000","last_change":"2024-04-25T09:03:17.163557+0000","last_active":"2024-04-25T09:03:18.562872+0000","last_peered":"2024-04-25T09:03:18.562872+0000","last_clean":"2024-04-25T09:03:18.562872+0000","last_became_active":"2024-04-25T09:03:17.163233+0000","last_became_peered":"2024-04-25T09:03:17.163233+0000","last_unstale":"2024-04-25T09:03:18.562872+0000","last_undegraded":"2024-04-25T09:03:18.562872+0000","last_fullsized":"2024-04-25T09:03:18.562872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:08.323326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172841+0000","last_change":"2024-04-25T09:03:16.168391+0000","last_active":"2024-04-25T09:03:18.172841+0000","last_peered":"2024-04-25T09:03:18.172841+0000","last_clean":"2024-04-25T09:03:18.172841+0000","last_became_active":"2024-04-25T09:03:16.168048+0000","last_became_peered":"2024-04-25T09:03:16.168048+0000","last_unstale":"2024-04-25T09:03:18.172841+0000","last_undegraded":"2024-04-25T09:03:18.172841+0000","last_fullsized":"2024-04-25T09:03:18.172841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:49.931470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563750+0000","last_change":"2024-04-25T09:03:17.173328+0000","last_active":"2024-04-25T09:03:18.563750+0000","last_peered":"2024-04-25T09:03:18.563750+0000","last_clean":"2024-04-25T09:03:18.563750+0000","last_became_active":"2024-04-25T09:03:17.165984+0000","last_became_peered":"2024-04-25T09:03:17.165984+0000","last_unstale":"2024-04-25T09:03:18.563750+0000","last_undegraded":"2024-04-25T09:03:18.563750+0000","last_fullsized":"2024-04-25T09:03:18.563750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153832+0000","last_change":"2024-04-25T09:03:16.167227+0000","last_active":"2024-04-25T09:03:18.153832+0000","last_peered":"2024-04-25T09:03:18.153832+0000","last_clean":"2024-04-25T09:03:18.153832+0000","last_became_active":"2024-04-25T09:03:16.166892+0000","last_became_peered":"2024-04-25T09:03:16.166892+0000","last_unstale":"2024-04-25T09:03:18.153832+0000","last_undegraded":"2024-04-25T09:03:18.153832+0000","last_fullsized":"2024-04-25T09:03:18.153832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172826+0000","last_change":"2024-04-25T09:03:17.163685+0000","last_active":"2024-04-25T09:03:18.172826+0000","last_peered":"2024-04-25T09:03:18.172826+0000","last_clean":"2024-04-25T09:03:18.172826+0000","last_became_active":"2024-04-25T09:03:17.163386+0000","last_became_peered":"2024-04-25T09:03:17.163386+0000","last_unstale":"2024-04-25T09:03:18.172826+0000","last_undegraded":"2024-04-25T09:03:18.172826+0000","last_fullsized":"2024-04-25T09:03:18.172826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:15.948906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565656+0000","last_change":"2024-04-25T09:03:16.160968+0000","last_active":"2024-04-25T09:03:18.565656+0000","last_peered":"2024-04-25T09:03:18.565656+0000","last_clean":"2024-04-25T09:03:18.565656+0000","last_became_active":"2024-04-25T09:03:16.158804+0000","last_became_peered":"2024-04-25T09:03:16.158804+0000","last_unstale":"2024-04-25T09:03:18.565656+0000","last_undegraded":"2024-04-25T09:03:18.565656+0000","last_fullsized":"2024-04-25T09:03:18.565656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:43.804933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.173017+0000","last_change":"2024-04-25T09:03:17.162364+0000","last_active":"2024-04-25T09:03:18.173017+0000","last_peered":"2024-04-25T09:03:18.173017+0000","last_clean":"2024-04-25T09:03:18.173017+0000","last_became_active":"2024-04-25T09:03:17.162093+0000","last_became_peered":"2024-04-25T09:03:17.162093+0000","last_unstale":"2024-04-25T09:03:18.173017+0000","last_undegraded":"2024-04-25T09:03:18.173017+0000","last_fullsized":"2024-04-25T09:03:18.173017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:36.933011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563994+0000","last_change":"2024-04-25T09:03:16.167573+0000","last_active":"2024-04-25T09:03:18.563994+0000","last_peered":"2024-04-25T09:03:18.563994+0000","last_clean":"2024-04-25T09:03:18.563994+0000","last_became_active":"2024-04-25T09:03:16.167279+0000","last_became_peered":"2024-04-25T09:03:16.167279+0000","last_unstale":"2024-04-25T09:03:18.563994+0000","last_undegraded":"2024-04-25T09:03:18.563994+0000","last_fullsized":"2024-04-25T09:03:18.563994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:36.089705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564569+0000","last_change":"2024-04-25T09:03:17.160049+0000","last_active":"2024-04-25T09:03:18.564569+0000","last_peered":"2024-04-25T09:03:18.564569+0000","last_clean":"2024-04-25T09:03:18.564569+0000","last_became_active":"2024-04-25T09:03:17.157508+0000","last_became_peered":"2024-04-25T09:03:17.157508+0000","last_unstale":"2024-04-25T09:03:18.564569+0000","last_undegraded":"2024-04-25T09:03:18.564569+0000","last_fullsized":"2024-04-25T09:03:18.564569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:25.867031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.150373+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:03:18.150373+0000","last_peered":"2024-04-25T09:03:18.150373+0000","last_clean":"2024-04-25T09:03:18.150373+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:03:18.150373+0000","last_undegraded":"2024-04-25T09:03:18.150373+0000","last_fullsized":"2024-04-25T09:03:18.150373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564168+0000","last_change":"2024-04-25T09:03:16.152466+0000","last_active":"2024-04-25T09:03:18.564168+0000","last_peered":"2024-04-25T09:03:18.564168+0000","last_clean":"2024-04-25T09:03:18.564168+0000","last_became_active":"2024-04-25T09:03:16.152216+0000","last_became_peered":"2024-04-25T09:03:16.152216+0000","last_unstale":"2024-04-25T09:03:18.564168+0000","last_undegraded":"2024-04-25T09:03:18.564168+0000","last_fullsized":"2024-04-25T09:03:18.564168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:49.931470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154561+0000","last_change":"2024-04-25T09:03:17.162520+0000","last_active":"2024-04-25T09:03:18.154561+0000","last_peered":"2024-04-25T09:03:18.154561+0000","last_clean":"2024-04-25T09:03:18.154561+0000","last_became_active":"2024-04-25T09:03:17.160462+0000","last_became_peered":"2024-04-25T09:03:17.160462+0000","last_unstale":"2024-04-25T09:03:18.154561+0000","last_undegraded":"2024-04-25T09:03:18.154561+0000","last_fullsized":"2024-04-25T09:03:18.154561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.326298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562165+0000","last_change":"2024-04-25T09:03:16.167181+0000","last_active":"2024-04-25T09:03:18.562165+0000","last_peered":"2024-04-25T09:03:18.562165+0000","last_clean":"2024-04-25T09:03:18.562165+0000","last_became_active":"2024-04-25T09:03:16.166842+0000","last_became_peered":"2024-04-25T09:03:16.166842+0000","last_unstale":"2024-04-25T09:03:18.562165+0000","last_undegraded":"2024-04-25T09:03:18.562165+0000","last_fullsized":"2024-04-25T09:03:18.562165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:54.518583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563045+0000","last_change":"2024-04-25T09:03:17.158524+0000","last_active":"2024-04-25T09:03:18.563045+0000","last_peered":"2024-04-25T09:03:18.563045+0000","last_clean":"2024-04-25T09:03:18.563045+0000","last_became_active":"2024-04-25T09:03:17.157997+0000","last_became_peered":"2024-04-25T09:03:17.157997+0000","last_unstale":"2024-04-25T09:03:18.563045+0000","last_undegraded":"2024-04-25T09:03:18.563045+0000","last_fullsized":"2024-04-25T09:03:18.563045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566796+0000","last_change":"2024-04-25T09:03:16.169797+0000","last_active":"2024-04-25T09:03:18.566796+0000","last_peered":"2024-04-25T09:03:18.566796+0000","last_clean":"2024-04-25T09:03:18.566796+0000","last_became_active":"2024-04-25T09:03:16.169557+0000","last_became_peered":"2024-04-25T09:03:16.169557+0000","last_unstale":"2024-04-25T09:03:18.566796+0000","last_undegraded":"2024-04-25T09:03:18.566796+0000","last_fullsized":"2024-04-25T09:03:18.566796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:10.210895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563921+0000","last_change":"2024-04-25T09:03:17.157959+0000","last_active":"2024-04-25T09:03:18.563921+0000","last_peered":"2024-04-25T09:03:18.563921+0000","last_clean":"2024-04-25T09:03:18.563921+0000","last_became_active":"2024-04-25T09:03:17.157664+0000","last_became_peered":"2024-04-25T09:03:17.157664+0000","last_unstale":"2024-04-25T09:03:18.563921+0000","last_undegraded":"2024-04-25T09:03:18.563921+0000","last_fullsized":"2024-04-25T09:03:18.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.631418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563894+0000","last_change":"2024-04-25T09:03:16.157473+0000","last_active":"2024-04-25T09:03:18.563894+0000","last_peered":"2024-04-25T09:03:18.563894+0000","last_clean":"2024-04-25T09:03:18.563894+0000","last_became_active":"2024-04-25T09:03:16.157153+0000","last_became_peered":"2024-04-25T09:03:16.157153+0000","last_unstale":"2024-04-25T09:03:18.563894+0000","last_undegraded":"2024-04-25T09:03:18.563894+0000","last_fullsized":"2024-04-25T09:03:18.563894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:41.809030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172033+0000","last_change":"2024-04-25T09:03:17.164826+0000","last_active":"2024-04-25T09:03:18.172033+0000","last_peered":"2024-04-25T09:03:18.172033+0000","last_clean":"2024-04-25T09:03:18.172033+0000","last_became_active":"2024-04-25T09:03:17.161434+0000","last_became_peered":"2024-04-25T09:03:17.161434+0000","last_unstale":"2024-04-25T09:03:18.172033+0000","last_undegraded":"2024-04-25T09:03:18.172033+0000","last_fullsized":"2024-04-25T09:03:18.172033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:39.624252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565331+0000","last_change":"2024-04-25T09:03:16.171358+0000","last_active":"2024-04-25T09:03:18.565331+0000","last_peered":"2024-04-25T09:03:18.565331+0000","last_clean":"2024-04-25T09:03:18.565331+0000","last_became_active":"2024-04-25T09:03:16.171010+0000","last_became_peered":"2024-04-25T09:03:16.171010+0000","last_unstale":"2024-04-25T09:03:18.565331+0000","last_undegraded":"2024-04-25T09:03:18.565331+0000","last_fullsized":"2024-04-25T09:03:18.565331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:41.025423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562033+0000","last_change":"2024-04-25T09:03:17.161787+0000","last_active":"2024-04-25T09:03:18.562033+0000","last_peered":"2024-04-25T09:03:18.562033+0000","last_clean":"2024-04-25T09:03:18.562033+0000","last_became_active":"2024-04-25T09:03:17.157886+0000","last_became_peered":"2024-04-25T09:03:17.157886+0000","last_unstale":"2024-04-25T09:03:18.562033+0000","last_undegraded":"2024-04-25T09:03:18.562033+0000","last_fullsized":"2024-04-25T09:03:18.562033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:38.838938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564686+0000","last_change":"2024-04-25T09:03:16.157712+0000","last_active":"2024-04-25T09:03:18.564686+0000","last_peered":"2024-04-25T09:03:18.564686+0000","last_clean":"2024-04-25T09:03:18.564686+0000","last_became_active":"2024-04-25T09:03:16.157285+0000","last_became_peered":"2024-04-25T09:03:16.157285+0000","last_unstale":"2024-04-25T09:03:18.564686+0000","last_undegraded":"2024-04-25T09:03:18.564686+0000","last_fullsized":"2024-04-25T09:03:18.564686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:17.498426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563865+0000","last_change":"2024-04-25T09:03:17.169015+0000","last_active":"2024-04-25T09:03:18.563865+0000","last_peered":"2024-04-25T09:03:18.563865+0000","last_clean":"2024-04-25T09:03:18.563865+0000","last_became_active":"2024-04-25T09:03:17.168430+0000","last_became_peered":"2024-04-25T09:03:17.168430+0000","last_unstale":"2024-04-25T09:03:18.563865+0000","last_undegraded":"2024-04-25T09:03:18.563865+0000","last_fullsized":"2024-04-25T09:03:18.563865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:57.330616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154237+0000","last_change":"2024-04-25T09:03:16.167928+0000","last_active":"2024-04-25T09:03:18.154237+0000","last_peered":"2024-04-25T09:03:18.154237+0000","last_clean":"2024-04-25T09:03:18.154237+0000","last_became_active":"2024-04-25T09:03:16.167566+0000","last_became_peered":"2024-04-25T09:03:16.167566+0000","last_unstale":"2024-04-25T09:03:18.154237+0000","last_undegraded":"2024-04-25T09:03:18.154237+0000","last_fullsized":"2024-04-25T09:03:18.154237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:41.025423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154329+0000","last_change":"2024-04-25T09:03:17.157268+0000","last_active":"2024-04-25T09:03:18.154329+0000","last_peered":"2024-04-25T09:03:18.154329+0000","last_clean":"2024-04-25T09:03:18.154329+0000","last_became_active":"2024-04-25T09:03:17.156997+0000","last_became_peered":"2024-04-25T09:03:17.156997+0000","last_unstale":"2024-04-25T09:03:18.154329+0000","last_undegraded":"2024-04-25T09:03:18.154329+0000","last_fullsized":"2024-04-25T09:03:18.154329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.631418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.565139+0000","last_change":"2024-04-25T09:03:16.169093+0000","last_active":"2024-04-25T09:03:18.565139+0000","last_peered":"2024-04-25T09:03:18.565139+0000","last_clean":"2024-04-25T09:03:18.565139+0000","last_became_active":"2024-04-25T09:03:16.168826+0000","last_became_peered":"2024-04-25T09:03:16.168826+0000","last_unstale":"2024-04-25T09:03:18.565139+0000","last_undegraded":"2024-04-25T09:03:18.565139+0000","last_fullsized":"2024-04-25T09:03:18.565139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:59.217267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154092+0000","last_change":"2024-04-25T09:03:17.136952+0000","last_active":"2024-04-25T09:03:18.154092+0000","last_peered":"2024-04-25T09:03:18.154092+0000","last_clean":"2024-04-25T09:03:18.154092+0000","last_became_active":"2024-04-25T09:03:17.136246+0000","last_became_peered":"2024-04-25T09:03:17.136246+0000","last_unstale":"2024-04-25T09:03:18.154092+0000","last_undegraded":"2024-04-25T09:03:18.154092+0000","last_fullsized":"2024-04-25T09:03:18.154092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:57.330616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.171136+0000","last_change":"2024-04-25T09:03:16.156412+0000","last_active":"2024-04-25T09:03:18.171136+0000","last_peered":"2024-04-25T09:03:18.171136+0000","last_clean":"2024-04-25T09:03:18.171136+0000","last_became_active":"2024-04-25T09:03:16.151576+0000","last_became_peered":"2024-04-25T09:03:16.151576+0000","last_unstale":"2024-04-25T09:03:18.171136+0000","last_undegraded":"2024-04-25T09:03:18.171136+0000","last_fullsized":"2024-04-25T09:03:18.171136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.002913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.562924+0000","last_change":"2024-04-25T09:03:17.154125+0000","last_active":"2024-04-25T09:03:18.562924+0000","last_peered":"2024-04-25T09:03:18.562924+0000","last_clean":"2024-04-25T09:03:18.562924+0000","last_became_active":"2024-04-25T09:03:17.153852+0000","last_became_peered":"2024-04-25T09:03:17.153852+0000","last_unstale":"2024-04-25T09:03:18.562924+0000","last_undegraded":"2024-04-25T09:03:18.562924+0000","last_fullsized":"2024-04-25T09:03:18.562924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:57.554441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153647+0000","last_change":"2024-04-25T09:03:16.167837+0000","last_active":"2024-04-25T09:03:18.153647+0000","last_peered":"2024-04-25T09:03:18.153647+0000","last_clean":"2024-04-25T09:03:18.153647+0000","last_became_active":"2024-04-25T09:03:16.167365+0000","last_became_peered":"2024-04-25T09:03:16.167365+0000","last_unstale":"2024-04-25T09:03:18.153647+0000","last_undegraded":"2024-04-25T09:03:18.153647+0000","last_fullsized":"2024-04-25T09:03:18.153647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:57.583736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.153552+0000","last_change":"2024-04-25T09:03:17.137828+0000","last_active":"2024-04-25T09:03:18.153552+0000","last_peered":"2024-04-25T09:03:18.153552+0000","last_clean":"2024-04-25T09:03:18.153552+0000","last_became_active":"2024-04-25T09:03:17.137277+0000","last_became_peered":"2024-04-25T09:03:17.137277+0000","last_unstale":"2024-04-25T09:03:18.153552+0000","last_undegraded":"2024-04-25T09:03:18.153552+0000","last_fullsized":"2024-04-25T09:03:18.153552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.814536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564048+0000","last_change":"2024-04-25T09:03:16.165762+0000","last_active":"2024-04-25T09:03:18.564048+0000","last_peered":"2024-04-25T09:03:18.564048+0000","last_clean":"2024-04-25T09:03:18.564048+0000","last_became_active":"2024-04-25T09:03:16.165453+0000","last_became_peered":"2024-04-25T09:03:16.165453+0000","last_unstale":"2024-04-25T09:03:18.564048+0000","last_undegraded":"2024-04-25T09:03:18.564048+0000","last_fullsized":"2024-04-25T09:03:18.564048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.567253+0000","last_change":"2024-04-25T09:03:17.152812+0000","last_active":"2024-04-25T09:03:18.567253+0000","last_peered":"2024-04-25T09:03:18.567253+0000","last_clean":"2024-04-25T09:03:18.567253+0000","last_became_active":"2024-04-25T09:03:17.152461+0000","last_became_peered":"2024-04-25T09:03:17.152461+0000","last_unstale":"2024-04-25T09:03:18.567253+0000","last_undegraded":"2024-04-25T09:03:18.567253+0000","last_fullsized":"2024-04-25T09:03:18.567253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:16.414375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563836+0000","last_change":"2024-04-25T09:03:16.156321+0000","last_active":"2024-04-25T09:03:18.563836+0000","last_peered":"2024-04-25T09:03:18.563836+0000","last_clean":"2024-04-25T09:03:18.563836+0000","last_became_active":"2024-04-25T09:03:16.155953+0000","last_became_peered":"2024-04-25T09:03:16.155953+0000","last_unstale":"2024-04-25T09:03:18.563836+0000","last_undegraded":"2024-04-25T09:03:18.563836+0000","last_fullsized":"2024-04-25T09:03:18.563836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:15.408869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.563595+0000","last_change":"2024-04-25T09:03:17.158747+0000","last_active":"2024-04-25T09:03:18.563595+0000","last_peered":"2024-04-25T09:03:18.563595+0000","last_clean":"2024-04-25T09:03:18.563595+0000","last_became_active":"2024-04-25T09:03:17.158086+0000","last_became_peered":"2024-04-25T09:03:17.158086+0000","last_unstale":"2024-04-25T09:03:18.563595+0000","last_undegraded":"2024-04-25T09:03:18.563595+0000","last_fullsized":"2024-04-25T09:03:18.563595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:37.022619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.564291+0000","last_change":"2024-04-25T09:03:17.167869+0000","last_active":"2024-04-25T09:03:18.564291+0000","last_peered":"2024-04-25T09:03:18.564291+0000","last_clean":"2024-04-25T09:03:18.564291+0000","last_became_active":"2024-04-25T09:03:17.164836+0000","last_became_peered":"2024-04-25T09:03:17.164836+0000","last_unstale":"2024-04-25T09:03:18.564291+0000","last_undegraded":"2024-04-25T09:03:18.564291+0000","last_fullsized":"2024-04-25T09:03:18.564291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:19.500716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172988+0000","last_change":"2024-04-25T09:03:16.162264+0000","last_active":"2024-04-25T09:03:18.172988+0000","last_peered":"2024-04-25T09:03:18.172988+0000","last_clean":"2024-04-25T09:03:18.172988+0000","last_became_active":"2024-04-25T09:03:16.161892+0000","last_became_peered":"2024-04-25T09:03:16.161892+0000","last_unstale":"2024-04-25T09:03:18.172988+0000","last_undegraded":"2024-04-25T09:03:18.172988+0000","last_fullsized":"2024-04-25T09:03:18.172988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:17.498426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.151136+0000","last_change":"2024-04-25T09:03:17.156036+0000","last_active":"2024-04-25T09:03:18.151136+0000","last_peered":"2024-04-25T09:03:18.151136+0000","last_clean":"2024-04-25T09:03:18.151136+0000","last_became_active":"2024-04-25T09:03:17.154785+0000","last_became_peered":"2024-04-25T09:03:17.154785+0000","last_unstale":"2024-04-25T09:03:18.151136+0000","last_undegraded":"2024-04-25T09:03:18.151136+0000","last_fullsized":"2024-04-25T09:03:18.151136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:55.524089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.172370+0000","last_change":"2024-04-25T09:03:16.161940+0000","last_active":"2024-04-25T09:03:18.172370+0000","last_peered":"2024-04-25T09:03:18.172370+0000","last_clean":"2024-04-25T09:03:18.172370+0000","last_became_active":"2024-04-25T09:03:16.161542+0000","last_became_peered":"2024-04-25T09:03:16.161542+0000","last_unstale":"2024-04-25T09:03:18.172370+0000","last_undegraded":"2024-04-25T09:03:18.172370+0000","last_fullsized":"2024-04-25T09:03:18.172370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:57.583736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566119+0000","last_change":"2024-04-25T09:03:17.157276+0000","last_active":"2024-04-25T09:03:18.566119+0000","last_peered":"2024-04-25T09:03:18.566119+0000","last_clean":"2024-04-25T09:03:18.566119+0000","last_became_active":"2024-04-25T09:03:17.156910+0000","last_became_peered":"2024-04-25T09:03:17.156910+0000","last_unstale":"2024-04-25T09:03:18.566119+0000","last_undegraded":"2024-04-25T09:03:18.566119+0000","last_fullsized":"2024-04-25T09:03:18.566119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:13.218030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566191+0000","last_change":"2024-04-25T09:03:16.167873+0000","last_active":"2024-04-25T09:03:18.566191+0000","last_peered":"2024-04-25T09:03:18.566191+0000","last_clean":"2024-04-25T09:03:18.566191+0000","last_became_active":"2024-04-25T09:03:16.166612+0000","last_became_peered":"2024-04-25T09:03:16.166612+0000","last_unstale":"2024-04-25T09:03:18.566191+0000","last_undegraded":"2024-04-25T09:03:18.566191+0000","last_fullsized":"2024-04-25T09:03:18.566191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:41.605742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.566016+0000","last_change":"2024-04-25T09:03:17.156968+0000","last_active":"2024-04-25T09:03:18.566016+0000","last_peered":"2024-04-25T09:03:18.566016+0000","last_clean":"2024-04-25T09:03:18.566016+0000","last_became_active":"2024-04-25T09:03:17.156663+0000","last_became_peered":"2024-04-25T09:03:17.156663+0000","last_unstale":"2024-04-25T09:03:18.566016+0000","last_undegraded":"2024-04-25T09:03:18.566016+0000","last_fullsized":"2024-04-25T09:03:18.566016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:16.111125+0000","last_clean_scrub_stamp":"2024-04-25T09:03:16.111125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:18.046413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T09:03:18.154405+0000","last_change":"2024-04-25T09:03:16.157953+0000","last_active":"2024-04-25T09:03:18.154405+0000","last_peered":"2024-04-25T09:03:18.154405+0000","last_clean":"2024-04-25T09:03:18.154405+0000","last_became_active":"2024-04-25T09:03:16.157603+0000","last_became_peered":"2024-04-25T09:03:16.157603+0000","last_unstale":"2024-04-25T09:03:18.154405+0000","last_undegraded":"2024-04-25T09:03:18.154405+0000","last_fullsized":"2024-04-25T09:03:18.154405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:15.105619+0000","last_clean_scrub_stamp":"2024-04-25T09:03:15.105619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:16.002913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":661394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":73218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:28.030 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T09:03:28.030 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T09:03:28.030 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T09:03:28.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:03:28.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.285+0000 7f90fde00640 1 Processor -- start 2024-04-25T09:03:28.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.285+0000 7f90fde00640 1 -- start start 2024-04-25T09:03:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.287+0000 7f90fde00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f90f815c120 0x7f90f80ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.287+0000 7f90fde00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f90f80b1da0 0x7f90f80cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.287+0000 7f90fde00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f815c670 con 0x7f90f80b19c0 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.287+0000 7f90fde00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f80cf5c0 con 0x7f90f815c120 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.287+0000 7f90fde00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f815c950 con 0x7f90f80b1da0 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.287+0000 7f90f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f90f815c120 0x7f90f80ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.287+0000 7f90f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f90f815c120 0x7f90f80ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55098/0 (socket says 172.21.15.62:55098) 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.287+0000 7f90f7e00640 1 -- 172.21.15.62:0/3595352432 learned_addr learned my addr 172.21.15.62:0/3595352432 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.288+0000 7f90f7e00640 1 -- 172.21.15.62:0/3595352432 >> v2:172.21.15.121:3300/0 conn(0x7f90f80b1da0 msgr2=0x7f90f80cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.288+0000 7f90f7e00640 1 --2- 172.21.15.62:0/3595352432 >> v2:172.21.15.121:3300/0 conn(0x7f90f80b1da0 0x7f90f80cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.288+0000 7f90f7e00640 1 -- 172.21.15.62:0/3595352432 >> v1:172.21.15.121:6789/0 conn(0x7f90f80b19c0 legacy=0x7f90f80ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.288+0000 7f90f7e00640 1 -- 172.21.15.62:0/3595352432 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f815cc30 con 0x7f90f815c120 2024-04-25T09:03:28.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.289+0000 7f90f7e00640 1 --2- 172.21.15.62:0/3595352432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f815c120 0x7f90f80ccbe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f90f80677b0 tx=0x7f90e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79fac16493a0b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.289+0000 7f90f7400640 1 -- 172.21.15.62:0/3595352432 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4037070 con 0x7f90f815c120 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.289+0000 7f90f7400640 1 -- 172.21.15.62:0/3595352432 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4032440 con 0x7f90f815c120 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.289+0000 7f90f7400640 1 -- 172.21.15.62:0/3595352432 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4032810 con 0x7f90f815c120 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.290+0000 7f90fde00640 1 -- 172.21.15.62:0/3595352432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f815c120 msgr2=0x7f90f80ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.290+0000 7f90fde00640 1 --2- 172.21.15.62:0/3595352432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f815c120 0x7f90f80ccbe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f90f80677b0 tx=0x7f90e402a230 comp rx=0 tx=0).stop 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.290+0000 7f90fde00640 1 -- 172.21.15.62:0/3595352432 shutdown_connections 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.290+0000 7f90fde00640 1 --2- 172.21.15.62:0/3595352432 >> v2:172.21.15.121:3300/0 conn(0x7f90f80b1da0 0x7f90f80cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.290+0000 7f90fde00640 1 --2- 172.21.15.62:0/3595352432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f815c120 0x7f90f80ccbe0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.290+0000 7f90fde00640 1 -- 172.21.15.62:0/3595352432 >> 172.21.15.62:0/3595352432 conn(0x7f90f80668f0 msgr2=0x7f90f8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.290+0000 7f90fde00640 1 -- 172.21.15.62:0/3595352432 shutdown_connections 2024-04-25T09:03:28.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.290+0000 7f90fde00640 1 -- 172.21.15.62:0/3595352432 wait complete. 2024-04-25T09:03:28.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.292+0000 7f90fde00640 1 Processor -- start 2024-04-25T09:03:28.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.292+0000 7f90fde00640 1 -- start start 2024-04-25T09:03:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.293+0000 7f90fde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f80b19c0 0x7f90f8156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f80b1da0 0x7f90f80b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fde00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f815c120 0x7f90f80b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fde00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80cd7e0 con 0x7f90f80b19c0 2024-04-25T09:03:28.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fde00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80cd660 con 0x7f90f80b1da0 2024-04-25T09:03:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fde00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80cb7d0 con 0x7f90f815c120 2024-04-25T09:03:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f80b19c0 0x7f90f8156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f80b19c0 0x7f90f8156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55106/0 (socket says 172.21.15.62:55106) 2024-04-25T09:03:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fca00640 1 -- 172.21.15.62:0/3270962429 learned_addr learned my addr 172.21.15.62:0/3270962429 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90f7e00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f80b1da0 0x7f90f80b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fd400640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f815c120 0x7f90f80b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fca00640 1 -- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f815c120 msgr2=0x7f90f80b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fca00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f815c120 0x7f90f80b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fca00640 1 -- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f80b1da0 msgr2=0x7f90f80b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fca00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f80b1da0 0x7f90f80b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90fca00640 1 -- 172.21.15.62:0/3270962429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f81571d0 con 0x7f90f80b19c0 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.294+0000 7f90f7e00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f80b1da0 0x7f90f80b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:28.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.295+0000 7f90fca00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f80b19c0 0x7f90f8156c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90e800c410 tx=0x7f90e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.295+0000 7f90f4c00640 1 -- 172.21.15.62:0/3270962429 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8017070 con 0x7f90f80b19c0 2024-04-25T09:03:28.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.295+0000 7f90f4c00640 1 -- 172.21.15.62:0/3270962429 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e80035f0 con 0x7f90f80b19c0 2024-04-25T09:03:28.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.295+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f80b3690 con 0x7f90f80b19c0 2024-04-25T09:03:28.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.295+0000 7f90f4c00640 1 -- 172.21.15.62:0/3270962429 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8003a20 con 0x7f90f80b19c0 2024-04-25T09:03:28.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.295+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f80b3c50 con 0x7f90f80b19c0 2024-04-25T09:03:28.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.296+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f8072b10 con 0x7f90f80b19c0 2024-04-25T09:03:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.297+0000 7f90f4c00640 1 -- 172.21.15.62:0/3270962429 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90e801a070 con 0x7f90f80b19c0 2024-04-25T09:03:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.298+0000 7f90f4c00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90d806b520 0x7f90d806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.298+0000 7f90f4c00640 1 -- 172.21.15.62:0/3270962429 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90e80896f0 con 0x7f90f80b19c0 2024-04-25T09:03:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.299+0000 7f90f7e00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90d806b520 0x7f90d806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.300+0000 7f90f7e00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90d806b520 0x7f90d806d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90e4034000 tx=0x7f90e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.303+0000 7f90f4c00640 1 -- 172.21.15.62:0/3270962429 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90e80585e0 con 0x7f90f80b19c0 2024-04-25T09:03:28.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.549+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f90f80750e0 con 0x7f90f80b19c0 2024-04-25T09:03:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.549+0000 7f90f4c00640 1 -- 172.21.15.62:0/3270962429 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f90e80010c0 con 0x7f90f80b19c0 2024-04-25T09:03:28.537 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:28.537 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T09:03:28.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90d806b520 msgr2=0x7f90d806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90d806b520 0x7f90d806d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90e4034000 tx=0x7f90e402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f80b19c0 msgr2=0x7f90f8156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f80b19c0 0x7f90f8156c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90e800c410 tx=0x7f90e800c8d0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 shutdown_connections 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90d806b520 0x7f90d806d9e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90f815c120 0x7f90f80b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90f80b1da0 0x7f90f80b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 --2- 172.21.15.62:0/3270962429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90f80b19c0 0x7f90f8156c90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 >> 172.21.15.62:0/3270962429 conn(0x7f90f80668f0 msgr2=0x7f90f815e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.552+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 shutdown_connections 2024-04-25T09:03:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.553+0000 7f90fde00640 1 -- 172.21.15.62:0/3270962429 wait complete. 2024-04-25T09:03:28.559 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T09:03:28.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:28.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.773+0000 7fb92be00640 1 Processor -- start 2024-04-25T09:03:28.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.774+0000 7fb92be00640 1 -- start start 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.775+0000 7fb92be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb92c161f00 0x7fb92c1622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.775+0000 7fb92be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb92c06d900 0x7fb92c0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.775+0000 7fb92be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92c0b2a20 con 0x7fb92c0cc9e0 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.775+0000 7fb92be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92c0b27d0 con 0x7fb92c06d900 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.775+0000 7fb92be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0b2d00 con 0x7fb92c161f00 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb923e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb92c0cc9e0 0x7fb92c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57628/0 (socket says 172.21.15.62:57628) 2024-04-25T09:03:28.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb923e00640 1 -- 172.21.15.62:0/1020016638 learned_addr learned my addr 172.21.15.62:0/1020016638 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92aa00640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb92c161f00 0x7fb92c1622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92b400640 1 --2- 172.21.15.62:0/1020016638 >> v2:172.21.15.121:3300/0 conn(0x7fb92c06d900 0x7fb92c0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035405892 0 0) 0x7fb92c0b2a20 con 0x7fb92c0cc9e0 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb910003520 con 0x7fb92c0cc9e0 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4022506236 0 0) 0x7fb910003520 con 0x7fb92c0cc9e0 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb92c161f00 msgr2=0x7fb92c1622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb92c161f00 0x7fb92c1622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 >> v2:172.21.15.121:3300/0 conn(0x7fb92c06d900 msgr2=0x7fb92c0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 --2- 172.21.15.62:0/1020016638 >> v2:172.21.15.121:3300/0 conn(0x7fb92c06d900 0x7fb92c0cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c0b2fe0 con 0x7fb92c0cc9e0 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.776+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb914001670 con 0x7fb92c0cc9e0 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 >> v1:172.21.15.62:6789/0 conn(0x7fb92c0cc9e0 legacy=0x7fb92c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92a000640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9100039b0 0x7fb910003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92a000640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb92c161f00 0x7fb910004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92a000640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb910004d40 0x7fb9100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0b27d0 con 0x7fb910004d40 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0b2d00 con 0x7fb9100039b0 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91000b9c0 con 0x7fb92c161f00 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92b400640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb910004d40 0x7fb9100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb923e00640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb92c161f00 0x7fb910004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92aa00640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9100039b0 0x7fb910003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92b400640 1 -- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb92c161f00 msgr2=0x7fb910004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92b400640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb92c161f00 0x7fb910004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92b400640 1 -- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9100039b0 msgr2=0x7fb910003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92b400640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9100039b0 0x7fb910003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.777+0000 7fb92b400640 1 -- 172.21.15.62:0/1020016638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb910003520 con 0x7fb910004d40 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.778+0000 7fb92b400640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb910004d40 0x7fb9100051c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb92c06ff50 tx=0x7fb910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759eb5bbb6e17d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.779+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924009c10 con 0x7fb910004d40 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.779+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb924036070 con 0x7fb910004d40 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.779+0000 7fb92a000640 1 -- 172.21.15.62:0/1020016638 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924031850 con 0x7fb910004d40 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.779+0000 7fb92be00640 1 -- 172.21.15.62:0/1020016638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb910004d40 msgr2=0x7fb9100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.779+0000 7fb92be00640 1 --2- 172.21.15.62:0/1020016638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb910004d40 0x7fb9100051c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb92c06ff50 tx=0x7fb910005830 comp rx=0 tx=0).stop 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.779+0000 7fb92b400640 1 -- 172.21.15.62:0/1020016638 reap_dead start 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.779+0000 7fb92be00640 1 -- 172.21.15.62:0/1020016638 shutdown_connections 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.780+0000 7fb92be00640 1 -- 172.21.15.62:0/1020016638 >> 172.21.15.62:0/1020016638 conn(0x7fb92c0b14d0 msgr2=0x7fb92c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.780+0000 7fb92be00640 1 -- 172.21.15.62:0/1020016638 shutdown_connections 2024-04-25T09:03:28.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.780+0000 7fb92be00640 1 -- 172.21.15.62:0/1020016638 wait complete. 2024-04-25T09:03:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.782+0000 7fb92be00640 1 Processor -- start 2024-04-25T09:03:28.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.782+0000 7fb92be00640 1 -- start start 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.783+0000 7fb92be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb910004d40 0x7fb92c152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.783+0000 7fb92be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb92c162820 0x7fb92c1ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.783+0000 7fb92be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb92c15d750 0x7fb92c14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.783+0000 7fb92be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c1622e0 con 0x7fb92c15d750 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.783+0000 7fb92be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0b35a0 con 0x7fb92c162820 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.783+0000 7fb92be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c162460 con 0x7fb910004d40 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.783+0000 7fb92b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb92c15d750 0x7fb92c14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.783+0000 7fb92b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb92c15d750 0x7fb92c14e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55128/0 (socket says 172.21.15.62:55128) 2024-04-25T09:03:28.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb92b400640 1 -- 172.21.15.62:0/1710802016 learned_addr learned my addr 172.21.15.62:0/1710802016 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:28.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb92aa00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb910004d40 0x7fb92c152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb923e00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb92c162820 0x7fb92c1ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb92b400640 1 -- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb910004d40 msgr2=0x7fb92c152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb92b400640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb910004d40 0x7fb92c152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb92b400640 1 -- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb92c162820 msgr2=0x7fb92c1ea6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb92b400640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb92c162820 0x7fb92c1ea6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb92b400640 1 -- 172.21.15.62:0/1710802016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c152da0 con 0x7fb92c15d750 2024-04-25T09:03:28.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.784+0000 7fb92b400640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb92c15d750 0x7fb92c14e9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb924033040 tx=0x7fb924009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.785+0000 7fb923400640 1 -- 172.21.15.62:0/1710802016 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924009c10 con 0x7fb92c15d750 2024-04-25T09:03:28.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.785+0000 7fb923400640 1 -- 172.21.15.62:0/1710802016 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb924003990 con 0x7fb92c15d750 2024-04-25T09:03:28.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.785+0000 7fb923400640 1 -- 172.21.15.62:0/1710802016 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924003d80 con 0x7fb92c15d750 2024-04-25T09:03:28.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.785+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92c1ec290 con 0x7fb92c15d750 2024-04-25T09:03:28.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.785+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c1ec850 con 0x7fb92c15d750 2024-04-25T09:03:28.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.786+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92c04ef20 con 0x7fb92c15d750 2024-04-25T09:03:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.787+0000 7fb923400640 1 -- 172.21.15.62:0/1710802016 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb924005d10 con 0x7fb92c15d750 2024-04-25T09:03:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.788+0000 7fb923400640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb90806b4d0 0x7fb90806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.788+0000 7fb923400640 1 -- 172.21.15.62:0/1710802016 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9240ab7f0 con 0x7fb92c15d750 2024-04-25T09:03:28.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.788+0000 7fb92aa00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb90806b4d0 0x7fb90806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:28.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.789+0000 7fb92aa00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb90806b4d0 0x7fb90806d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb92c1ed490 tx=0x7fb91c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.792+0000 7fb923400640 1 -- 172.21.15.62:0/1710802016 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb92404a060 con 0x7fb92c15d750 2024-04-25T09:03:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.993+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb92c1ecf00 con 0x7fb92c15d750 2024-04-25T09:03:28.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.994+0000 7fb923400640 1 -- 172.21.15.62:0/1710802016 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fb92402e050 con 0x7fb92c15d750 2024-04-25T09:03:28.983 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:28.983 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb90806b4d0 msgr2=0x7fb90806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb90806b4d0 0x7fb90806d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb92c1ed490 tx=0x7fb91c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:28.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb92c15d750 msgr2=0x7fb92c14e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb92c15d750 0x7fb92c14e9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb924033040 tx=0x7fb924009750 comp rx=0 tx=0).stop 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 shutdown_connections 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb90806b4d0 0x7fb90806d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb92c15d750 0x7fb92c14e9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb92c162820 0x7fb92c1ea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 --2- 172.21.15.62:0/1710802016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb910004d40 0x7fb92c152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 >> 172.21.15.62:0/1710802016 conn(0x7fb92c0b14d0 msgr2=0x7fb92c0b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 shutdown_connections 2024-04-25T09:03:28.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:28.997+0000 7fb92be00640 1 -- 172.21.15.62:0/1710802016 wait complete. 2024-04-25T09:03:29.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:29.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:29.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.232+0000 7fa532c00640 1 Processor -- start 2024-04-25T09:03:29.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.233+0000 7fa532c00640 1 -- start start 2024-04-25T09:03:29.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.234+0000 7fa532c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa52c0d3960 0x7fa52c0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.234+0000 7fa532c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0ce780 con 0x7fa52c0d2fb0 2024-04-25T09:03:29.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.234+0000 7fa532c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0d42f0 con 0x7fa52c15dd50 2024-04-25T09:03:29.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.234+0000 7fa532c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c0d3440 con 0x7fa52c0d3960 2024-04-25T09:03:29.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.234+0000 7fa530e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa52c0d2fb0 0x7fa52c0ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57630/0 (socket says 172.21.15.62:57630) 2024-04-25T09:03:29.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.234+0000 7fa530e00640 1 -- 172.21.15.62:0/2935276005 learned_addr learned my addr 172.21.15.62:0/2935276005 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:29.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.234+0000 7fa532200640 1 --2- 172.21.15.62:0/2935276005 >> v2:172.21.15.121:3300/0 conn(0x7fa52c0d3960 0x7fa52c0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324647974 0 0) 0x7fa52c0ce780 con 0x7fa52c0d2fb0 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa510003520 con 0x7fa52c0d2fb0 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2894326989 0 0) 0x7fa510003520 con 0x7fa52c0d2fb0 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 >> v2:172.21.15.121:3300/0 conn(0x7fa52c0d3960 msgr2=0x7fa52c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 --2- 172.21.15.62:0/2935276005 >> v2:172.21.15.121:3300/0 conn(0x7fa52c0d3960 0x7fa52c0d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 >> v1:172.21.15.121:6789/0 conn(0x7fa52c15dd50 legacy=0x7fa52c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0ce1c0 con 0x7fa52c0d2fb0 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fa518001670 con 0x7fa52c0d2fb0 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa532200640 1 --2- 172.21.15.62:0/2935276005 >> v2:172.21.15.121:3300/0 conn(0x7fa52c0d3960 0x7fa52c0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 >> v1:172.21.15.62:6789/0 conn(0x7fa52c0d2fb0 legacy=0x7fa52c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa510003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa510004800 0x7fa510004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5100051c0 0x7fa51000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0d3440 con 0x7fa510003a00 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51000c0c0 con 0x7fa5100051c0 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.235+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51000c2c0 con 0x7fa510004800 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa531800640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa510003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa530e00640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa510004800 0x7fa510004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa532200640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5100051c0 0x7fa51000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa531800640 1 -- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa510004800 msgr2=0x7fa510004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa531800640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa510004800 0x7fa510004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa531800640 1 -- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5100051c0 msgr2=0x7fa51000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa531800640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5100051c0 0x7fa51000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa531800640 1 -- 172.21.15.62:0/2935276005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa510003520 con 0x7fa510003a00 2024-04-25T09:03:29.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa530e00640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa510004800 0x7fa510004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa532200640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5100051c0 0x7fa51000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.236+0000 7fa532200640 1 -- 172.21.15.62:0/2935276005 reap_dead start 2024-04-25T09:03:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.237+0000 7fa531800640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa510003e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa51c001750 tx=0x7fa510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ed1c3ac3fe1729 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.237+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c001370 con 0x7fa510003a00 2024-04-25T09:03:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.237+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c0052d0 con 0x7fa510003a00 2024-04-25T09:03:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.237+0000 7fa52be00640 1 -- 172.21.15.62:0/2935276005 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c02ab50 con 0x7fa510003a00 2024-04-25T09:03:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.238+0000 7fa532c00640 1 -- 172.21.15.62:0/2935276005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 msgr2=0x7fa510003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.238+0000 7fa532c00640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa510003e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa51c001750 tx=0x7fa510005830 comp rx=0 tx=0).stop 2024-04-25T09:03:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.238+0000 7fa532c00640 1 -- 172.21.15.62:0/2935276005 shutdown_connections 2024-04-25T09:03:29.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.238+0000 7fa532c00640 1 --2- 172.21.15.62:0/2935276005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa510003e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.238+0000 7fa532c00640 1 -- 172.21.15.62:0/2935276005 >> 172.21.15.62:0/2935276005 conn(0x7fa52c06b000 msgr2=0x7fa52c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.238+0000 7fa532c00640 1 -- 172.21.15.62:0/2935276005 shutdown_connections 2024-04-25T09:03:29.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.238+0000 7fa532c00640 1 -- 172.21.15.62:0/2935276005 wait complete. 2024-04-25T09:03:29.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.240+0000 7fa532c00640 1 Processor -- start 2024-04-25T09:03:29.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.240+0000 7fa532c00640 1 -- start start 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa532c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa52c149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa532c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa52c0c8f20 0x7fa52c1e6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa532c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa52c0c93a0 0x7fa52c1e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa532c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0d3f30 con 0x7fa510003a00 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa532c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0d3db0 con 0x7fa52c0c93a0 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa532c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0d40b0 con 0x7fa52c0c8f20 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa531800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa52c149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa531800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa52c149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55132/0 (socket says 172.21.15.62:55132) 2024-04-25T09:03:29.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.241+0000 7fa531800640 1 -- 172.21.15.62:0/2070046916 learned_addr learned my addr 172.21.15.62:0/2070046916 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa532200640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa52c0c93a0 0x7fa52c1e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa530e00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa52c0c8f20 0x7fa52c1e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa531800640 1 -- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa52c0c8f20 msgr2=0x7fa52c1e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa531800640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa52c0c8f20 0x7fa52c1e6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa531800640 1 -- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa52c0c93a0 msgr2=0x7fa52c1e68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa531800640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa52c0c93a0 0x7fa52c1e68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa531800640 1 -- 172.21.15.62:0/2070046916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c14a310 con 0x7fa510003a00 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa532200640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa52c0c93a0 0x7fa52c1e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa531800640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa52c149dd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa51c001870 tx=0x7fa51c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa529600640 1 -- 172.21.15.62:0/2070046916 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c03f020 con 0x7fa510003a00 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa529600640 1 -- 172.21.15.62:0/2070046916 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c005480 con 0x7fa510003a00 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa529600640 1 -- 172.21.15.62:0/2070046916 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c007a30 con 0x7fa510003a00 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.242+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c0c9900 con 0x7fa510003a00 2024-04-25T09:03:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.243+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c0b3b30 con 0x7fa510003a00 2024-04-25T09:03:29.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.244+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c0c9e20 con 0x7fa510003a00 2024-04-25T09:03:29.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.245+0000 7fa529600640 1 -- 172.21.15.62:0/2070046916 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa51c042020 con 0x7fa510003a00 2024-04-25T09:03:29.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.245+0000 7fa529600640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.246+0000 7fa529600640 1 -- 172.21.15.62:0/2070046916 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa51c02be00 con 0x7fa510003a00 2024-04-25T09:03:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.246+0000 7fa530e00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.247+0000 7fa530e00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa50c06b4d0 0x7fa50c06d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5180017e0 tx=0x7fa518009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.250+0000 7fa529600640 1 -- 172.21.15.62:0/2070046916 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa52c0c9e20 con 0x7fa510003a00 2024-04-25T09:03:29.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.448+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa52c04ef20 con 0x7fa510003a00 2024-04-25T09:03:29.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.449+0000 7fa529600640 1 -- 172.21.15.62:0/2070046916 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa51c049020 con 0x7fa510003a00 2024-04-25T09:03:29.437 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:29.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa50c06b4d0 msgr2=0x7fa50c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa50c06b4d0 0x7fa50c06d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5180017e0 tx=0x7fa518009040 comp rx=0 tx=0).stop 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 msgr2=0x7fa52c149dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa52c149dd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa51c001870 tx=0x7fa51c001120 comp rx=0 tx=0).stop 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 shutdown_connections 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa52c0c93a0 0x7fa52c1e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa52c0c8f20 0x7fa52c1e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 --2- 172.21.15.62:0/2070046916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa510003a00 0x7fa52c149dd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 >> 172.21.15.62:0/2070046916 conn(0x7fa52c06b000 msgr2=0x7fa52c1601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 shutdown_connections 2024-04-25T09:03:29.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.452+0000 7fa532c00640 1 -- 172.21.15.62:0/2070046916 wait complete. 2024-04-25T09:03:29.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.691+0000 7fb872a00640 1 Processor -- start 2024-04-25T09:03:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.691+0000 7fb872a00640 1 -- start start 2024-04-25T09:03:29.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.692+0000 7fb872a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb86c06c9b0 0x7fb86c0720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.692+0000 7fb872a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c06ce00 con 0x7fb86c15bdf0 2024-04-25T09:03:29.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.693+0000 7fb872a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c06d070 con 0x7fb86c0cebc0 2024-04-25T09:03:29.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.693+0000 7fb872a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86c15c1d0 con 0x7fb86c06c9b0 2024-04-25T09:03:29.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.693+0000 7fb86be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb86c15bdf0 0x7fb86c06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57636/0 (socket says 172.21.15.62:57636) 2024-04-25T09:03:29.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.693+0000 7fb86be00640 1 -- 172.21.15.62:0/2460308684 learned_addr learned my addr 172.21.15.62:0/2460308684 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:29.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.693+0000 7fb870a00640 1 --2- 172.21.15.62:0/2460308684 >> v2:172.21.15.121:3300/0 conn(0x7fb86c06c9b0 0x7fb86c0720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.693+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668391352 0 0) 0x7fb86c06ce00 con 0x7fb86c15bdf0 2024-04-25T09:03:29.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.693+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb850003520 con 0x7fb86c15bdf0 2024-04-25T09:03:29.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2717628714 0 0) 0x7fb850003520 con 0x7fb86c15bdf0 2024-04-25T09:03:29.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 >> v2:172.21.15.121:3300/0 conn(0x7fb86c06c9b0 msgr2=0x7fb86c0720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 --2- 172.21.15.62:0/2460308684 >> v2:172.21.15.121:3300/0 conn(0x7fb86c06c9b0 0x7fb86c0720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 >> v1:172.21.15.121:6789/0 conn(0x7fb86c0cebc0 legacy=0x7fb86c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c06d350 con 0x7fb86c15bdf0 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb85c001670 con 0x7fb86c15bdf0 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 >> v1:172.21.15.62:6789/0 conn(0x7fb86c15bdf0 legacy=0x7fb86c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8500039b0 0x7fb850003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb850004800 0x7fb850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb85000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c15c1d0 con 0x7fb8500051c0 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85000c060 con 0x7fb8500039b0 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85000c260 con 0x7fb850004800 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb870a00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb85000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86be00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8500039b0 0x7fb850003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.694+0000 7fb86b400640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb850004800 0x7fb850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.695+0000 7fb870a00640 1 -- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb850004800 msgr2=0x7fb850004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.695+0000 7fb870a00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb850004800 0x7fb850004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.695+0000 7fb870a00640 1 -- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8500039b0 msgr2=0x7fb850003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.695+0000 7fb870a00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8500039b0 0x7fb850003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.695+0000 7fb870a00640 1 -- 172.21.15.62:0/2460308684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb850003520 con 0x7fb8500051c0 2024-04-25T09:03:29.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.695+0000 7fb86b400640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb850004800 0x7fb850004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.695+0000 7fb870a00640 1 -- 172.21.15.62:0/2460308684 reap_dead start 2024-04-25T09:03:29.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.696+0000 7fb870a00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb85000b9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb864009fc0 tx=0x7fb850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8934d361b1dc82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.696+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864035020 con 0x7fb8500051c0 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.696+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864003bd0 con 0x7fb8500051c0 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.696+0000 7fb86aa00640 1 -- 172.21.15.62:0/2460308684 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864009c10 con 0x7fb8500051c0 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.696+0000 7fb872a00640 1 -- 172.21.15.62:0/2460308684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 msgr2=0x7fb85000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.696+0000 7fb872a00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb85000b9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb864009fc0 tx=0x7fb850005830 comp rx=0 tx=0).stop 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.697+0000 7fb872a00640 1 -- 172.21.15.62:0/2460308684 shutdown_connections 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.697+0000 7fb872a00640 1 --2- 172.21.15.62:0/2460308684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb85000b9c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.697+0000 7fb872a00640 1 -- 172.21.15.62:0/2460308684 >> 172.21.15.62:0/2460308684 conn(0x7fb86c0b1410 msgr2=0x7fb86c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.697+0000 7fb872a00640 1 -- 172.21.15.62:0/2460308684 shutdown_connections 2024-04-25T09:03:29.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.697+0000 7fb872a00640 1 -- 172.21.15.62:0/2460308684 wait complete. 2024-04-25T09:03:29.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.699+0000 7fb872a00640 1 Processor -- start 2024-04-25T09:03:29.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.699+0000 7fb872a00640 1 -- start start 2024-04-25T09:03:29.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb872a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb86c153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb872a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb86c152060 0x7fb86c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb872a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c141260 0x7fb86c1416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb872a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0aedf0 con 0x7fb8500051c0 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb872a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0ae8c0 con 0x7fb86c141260 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb872a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0720c0 con 0x7fb86c152060 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb86be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb86c153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb86be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb86c153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55138/0 (socket says 172.21.15.62:55138) 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb86be00640 1 -- 172.21.15.62:0/1730430141 learned_addr learned my addr 172.21.15.62:0/1730430141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb870a00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c141260 0x7fb86c1416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb86b400640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb86c152060 0x7fb86c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb86be00640 1 -- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb86c152060 msgr2=0x7fb86c15e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb86be00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb86c152060 0x7fb86c15e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb86be00640 1 -- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c141260 msgr2=0x7fb86c1416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.700+0000 7fb86be00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c141260 0x7fb86c1416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.701+0000 7fb86be00640 1 -- 172.21.15.62:0/1730430141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c1401c0 con 0x7fb8500051c0 2024-04-25T09:03:29.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.701+0000 7fb86be00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb86c153220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e760 tx=0x7fb85c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.701+0000 7fb863e00640 1 -- 172.21.15.62:0/1730430141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00ee70 con 0x7fb8500051c0 2024-04-25T09:03:29.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.701+0000 7fb863e00640 1 -- 172.21.15.62:0/1730430141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c018070 con 0x7fb8500051c0 2024-04-25T09:03:29.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.701+0000 7fb863e00640 1 -- 172.21.15.62:0/1730430141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00cab0 con 0x7fb8500051c0 2024-04-25T09:03:29.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.701+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c1404a0 con 0x7fb8500051c0 2024-04-25T09:03:29.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.701+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c140a60 con 0x7fb8500051c0 2024-04-25T09:03:29.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.702+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c06cd90 con 0x7fb8500051c0 2024-04-25T09:03:29.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.703+0000 7fb863e00640 1 -- 172.21.15.62:0/1730430141 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb85c00cc50 con 0x7fb8500051c0 2024-04-25T09:03:29.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.704+0000 7fb863e00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb84c06b4d0 0x7fb84c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.704+0000 7fb863e00640 1 -- 172.21.15.62:0/1730430141 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb85c08a0a0 con 0x7fb8500051c0 2024-04-25T09:03:29.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.705+0000 7fb86b400640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb84c06b4d0 0x7fb84c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.706+0000 7fb86b400640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb84c06b4d0 0x7fb84c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0b12d0 tx=0x7fb858009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.708+0000 7fb863e00640 1 -- 172.21.15.62:0/1730430141 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb86c06cd90 con 0x7fb8500051c0 2024-04-25T09:03:29.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.909+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb86c04ef20 con 0x7fb8500051c0 2024-04-25T09:03:29.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.910+0000 7fb863e00640 1 -- 172.21.15.62:0/1730430141 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb85c058e80 con 0x7fb8500051c0 2024-04-25T09:03:29.898 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:03:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.912+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb84c06b4d0 msgr2=0x7fb84c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb84c06b4d0 0x7fb84c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0b12d0 tx=0x7fb858009040 comp rx=0 tx=0).stop 2024-04-25T09:03:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 msgr2=0x7fb86c153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb86c153220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e760 tx=0x7fb85c00ec20 comp rx=0 tx=0).stop 2024-04-25T09:03:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 shutdown_connections 2024-04-25T09:03:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb84c06b4d0 0x7fb84c06d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c141260 0x7fb86c1416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb86c152060 0x7fb86c15e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 --2- 172.21.15.62:0/1730430141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8500051c0 0x7fb86c153220 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 >> 172.21.15.62:0/1730430141 conn(0x7fb86c0b1410 msgr2=0x7fb86c0cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 shutdown_connections 2024-04-25T09:03:29.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:29.913+0000 7fb872a00640 1 -- 172.21.15.62:0/1730430141 wait complete. 2024-04-25T09:03:29.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.151+0000 7f180d200640 1 Processor -- start 2024-04-25T09:03:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.151+0000 7f180d200640 1 -- start start 2024-04-25T09:03:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.152+0000 7f180d200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18080ccdb0 0x7f18080cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.152+0000 7f180d200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f18080d0490 0x7f18080cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.152+0000 7f180d200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18080cd650 con 0x7f180815b8f0 2024-04-25T09:03:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.152+0000 7f180d200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18080cd370 con 0x7f18080ccdb0 2024-04-25T09:03:30.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.152+0000 7f180d200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18080cf470 con 0x7f18080d0490 2024-04-25T09:03:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f17fea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18080ccdb0 0x7f18080cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f17fea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18080ccdb0 0x7f18080cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55142/0 (socket says 172.21.15.62:55142) 2024-04-25T09:03:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f17fea00640 1 -- 172.21.15.62:0/2433267349 learned_addr learned my addr 172.21.15.62:0/2433267349 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f1807400640 1 --1- 172.21.15.62:0/2433267349 >> v1:172.21.15.62:6789/0 conn(0x7f180815b8f0 0x7f18080b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57652/0 (socket says 172.21.15.62:57652) 2024-04-25T09:03:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f1807e00640 1 --2- 172.21.15.62:0/2433267349 >> v2:172.21.15.121:3300/0 conn(0x7f18080d0490 0x7f18080cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f17fea00640 1 -- 172.21.15.62:0/2433267349 >> v2:172.21.15.121:3300/0 conn(0x7f18080d0490 msgr2=0x7f18080cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f17fea00640 1 --2- 172.21.15.62:0/2433267349 >> v2:172.21.15.121:3300/0 conn(0x7f18080d0490 0x7f18080cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f17fea00640 1 -- 172.21.15.62:0/2433267349 >> v1:172.21.15.62:6789/0 conn(0x7f180815b8f0 legacy=0x7f18080b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f1806a00640 1 -- 172.21.15.62:0/2433267349 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259283494 0 0) 0x7f18080cd650 con 0x7f180815b8f0 2024-04-25T09:03:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.153+0000 7f17fea00640 1 -- 172.21.15.62:0/2433267349 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18080cf750 con 0x7f18080ccdb0 2024-04-25T09:03:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.154+0000 7f17fea00640 1 --2- 172.21.15.62:0/2433267349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18080ccdb0 0x7f18080cff50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f18080d17a0 tx=0x7f17f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a63e301de0e2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f1806a00640 1 -- 172.21.15.62:0/2433267349 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0037070 con 0x7f18080ccdb0 2024-04-25T09:03:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f1806a00640 1 -- 172.21.15.62:0/2433267349 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f0032440 con 0x7f18080ccdb0 2024-04-25T09:03:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f1806a00640 1 -- 172.21.15.62:0/2433267349 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0032810 con 0x7f18080ccdb0 2024-04-25T09:03:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f180d200640 1 -- 172.21.15.62:0/2433267349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18080ccdb0 msgr2=0x7f18080cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f180d200640 1 --2- 172.21.15.62:0/2433267349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18080ccdb0 0x7f18080cff50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f18080d17a0 tx=0x7f17f002a270 comp rx=0 tx=0).stop 2024-04-25T09:03:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f180d200640 1 -- 172.21.15.62:0/2433267349 shutdown_connections 2024-04-25T09:03:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f180d200640 1 --2- 172.21.15.62:0/2433267349 >> v2:172.21.15.121:3300/0 conn(0x7f18080d0490 0x7f18080cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f180d200640 1 --2- 172.21.15.62:0/2433267349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18080ccdb0 0x7f18080cff50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f180d200640 1 -- 172.21.15.62:0/2433267349 >> 172.21.15.62:0/2433267349 conn(0x7f180809aa70 msgr2=0x7f18080b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f180d200640 1 -- 172.21.15.62:0/2433267349 shutdown_connections 2024-04-25T09:03:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.155+0000 7f180d200640 1 -- 172.21.15.62:0/2433267349 wait complete. 2024-04-25T09:03:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.157+0000 7f180d200640 1 Processor -- start 2024-04-25T09:03:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.157+0000 7f180d200640 1 -- start start 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.158+0000 7f180d200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18080ccdb0 0x7f1808156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.158+0000 7f180d200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18080d0490 0x7f1808157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.158+0000 7f180d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180815b8f0 0x7f1808157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.158+0000 7f180d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18080ce380 con 0x7f180815b8f0 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.158+0000 7f180d200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180815bcd0 con 0x7f18080d0490 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.158+0000 7f180d200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18080ce500 con 0x7f18080ccdb0 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180815b8f0 0x7f1808157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180815b8f0 0x7f1808157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55150/0 (socket says 172.21.15.62:55150) 2024-04-25T09:03:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 -- 172.21.15.62:0/1919248080 learned_addr learned my addr 172.21.15.62:0/1919248080 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f17fea00640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18080d0490 0x7f1808157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807400640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18080ccdb0 0x7f1808156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 -- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18080ccdb0 msgr2=0x7f1808156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18080ccdb0 0x7f1808156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 -- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18080d0490 msgr2=0x7f1808157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18080d0490 0x7f1808157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 -- 172.21.15.62:0/1919248080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180815e810 con 0x7f180815b8f0 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f17fea00640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18080d0490 0x7f1808157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.159+0000 7f1807e00640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180815b8f0 0x7f1808157dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f180000d800 tx=0x7f180000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.160+0000 7f17ffe00640 1 -- 172.21.15.62:0/1919248080 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180000b6c0 con 0x7f180815b8f0 2024-04-25T09:03:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.160+0000 7f17ffe00640 1 -- 172.21.15.62:0/1919248080 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1800019070 con 0x7f180815b8f0 2024-04-25T09:03:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.160+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180815e250 con 0x7f180815b8f0 2024-04-25T09:03:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.160+0000 7f17ffe00640 1 -- 172.21.15.62:0/1919248080 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180000bac0 con 0x7f180815b8f0 2024-04-25T09:03:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.160+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18080c6150 con 0x7f180815b8f0 2024-04-25T09:03:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.161+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180806c840 con 0x7f180815b8f0 2024-04-25T09:03:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.162+0000 7f17ffe00640 1 -- 172.21.15.62:0/1919248080 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f180000bc60 con 0x7f180815b8f0 2024-04-25T09:03:30.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.163+0000 7f17ffe00640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.163+0000 7f17ffe00640 1 -- 172.21.15.62:0/1919248080 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f180008a730 con 0x7f180815b8f0 2024-04-25T09:03:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.163+0000 7f1807400640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.164+0000 7f1807400640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17e406b4d0 0x7f17e406d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17f80012e0 tx=0x7f17f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.167+0000 7f17ffe00640 1 -- 172.21.15.62:0/1919248080 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1800059510 con 0x7f180815b8f0 2024-04-25T09:03:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.372+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18080750e0 con 0x7f180815b8f0 2024-04-25T09:03:30.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.373+0000 7f17ffe00640 1 -- 172.21.15.62:0/1919248080 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1800058e10 con 0x7f180815b8f0 2024-04-25T09:03:30.361 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:03:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17e406b4d0 msgr2=0x7f17e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17e406b4d0 0x7f17e406d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17f80012e0 tx=0x7f17f8009040 comp rx=0 tx=0).stop 2024-04-25T09:03:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180815b8f0 msgr2=0x7f1808157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180815b8f0 0x7f1808157dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f180000d800 tx=0x7f180000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 shutdown_connections 2024-04-25T09:03:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17e406b4d0 0x7f17e406d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f180815b8f0 0x7f1808157dc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18080d0490 0x7f1808157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 --2- 172.21.15.62:0/1919248080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18080ccdb0 0x7f1808156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 >> 172.21.15.62:0/1919248080 conn(0x7f180809aa70 msgr2=0x7f18080ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 shutdown_connections 2024-04-25T09:03:30.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.376+0000 7f180d200640 1 -- 172.21.15.62:0/1919248080 wait complete. 2024-04-25T09:03:30.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.612+0000 7ff2ce600640 1 Processor -- start 2024-04-25T09:03:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.613+0000 7ff2ce600640 1 -- start start 2024-04-25T09:03:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.614+0000 7ff2ce600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2c80b56f0 0x7ff2c8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.614+0000 7ff2ce600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff2c80b6a90 0x7ff2c8161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.614+0000 7ff2ce600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c80d3110 con 0x7ff2c80b27a0 2024-04-25T09:03:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.614+0000 7ff2ce600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c8161780 con 0x7ff2c80b56f0 2024-04-25T09:03:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.614+0000 7ff2ce600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c80cedc0 con 0x7ff2c80b6a90 2024-04-25T09:03:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.614+0000 7ff2cd200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2c80b56f0 0x7ff2c8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.615+0000 7ff2cd200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff2c80b56f0 0x7ff2c8160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55152/0 (socket says 172.21.15.62:55152) 2024-04-25T09:03:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.615+0000 7ff2cd200640 1 -- 172.21.15.62:0/3277666958 learned_addr learned my addr 172.21.15.62:0/3277666958 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.615+0000 7ff2cd200640 1 -- 172.21.15.62:0/3277666958 >> v2:172.21.15.121:3300/0 conn(0x7ff2c80b6a90 msgr2=0x7ff2c8161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.615+0000 7ff2cd200640 1 --2- 172.21.15.62:0/3277666958 >> v2:172.21.15.121:3300/0 conn(0x7ff2c80b6a90 0x7ff2c8161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.615+0000 7ff2cd200640 1 -- 172.21.15.62:0/3277666958 >> v1:172.21.15.121:6789/0 conn(0x7ff2c80b27a0 legacy=0x7ff2c8161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.615+0000 7ff2cd200640 1 -- 172.21.15.62:0/3277666958 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80d33f0 con 0x7ff2c80b56f0 2024-04-25T09:03:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.616+0000 7ff2cd200640 1 --2- 172.21.15.62:0/3277666958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8160d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2c808c570 tx=0x7ff2b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b6c3ff19258b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.616+0000 7ff2c7e00640 1 -- 172.21.15.62:0/3277666958 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8037070 con 0x7ff2c80b56f0 2024-04-25T09:03:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.616+0000 7ff2c7e00640 1 -- 172.21.15.62:0/3277666958 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b8032440 con 0x7ff2c80b56f0 2024-04-25T09:03:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.616+0000 7ff2c7e00640 1 -- 172.21.15.62:0/3277666958 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8032810 con 0x7ff2c80b56f0 2024-04-25T09:03:30.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.617+0000 7ff2ce600640 1 -- 172.21.15.62:0/3277666958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 msgr2=0x7ff2c8160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.617+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3277666958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8160d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff2c808c570 tx=0x7ff2b802a230 comp rx=0 tx=0).stop 2024-04-25T09:03:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.617+0000 7ff2ce600640 1 -- 172.21.15.62:0/3277666958 shutdown_connections 2024-04-25T09:03:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.617+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3277666958 >> v2:172.21.15.121:3300/0 conn(0x7ff2c80b6a90 0x7ff2c8161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.617+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3277666958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8160d00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.617+0000 7ff2ce600640 1 -- 172.21.15.62:0/3277666958 >> 172.21.15.62:0/3277666958 conn(0x7ff2c8066d10 msgr2=0x7ff2c8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.617+0000 7ff2ce600640 1 -- 172.21.15.62:0/3277666958 shutdown_connections 2024-04-25T09:03:30.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.617+0000 7ff2ce600640 1 -- 172.21.15.62:0/3277666958 wait complete. 2024-04-25T09:03:30.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.619+0000 7ff2ce600640 1 Processor -- start 2024-04-25T09:03:30.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.619+0000 7ff2ce600640 1 -- start start 2024-04-25T09:03:30.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.620+0000 7ff2ce600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b27a0 0x7ff2c815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.620+0000 7ff2ce600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.620+0000 7ff2ce600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c80b6a90 0x7ff2c80cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.620+0000 7ff2ce600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80b2d60 con 0x7ff2c80b56f0 2024-04-25T09:03:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.620+0000 7ff2ce600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80cea50 con 0x7ff2c80b6a90 2024-04-25T09:03:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.620+0000 7ff2ce600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80b2ee0 con 0x7ff2c80b27a0 2024-04-25T09:03:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.620+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55156/0 (socket says 172.21.15.62:55156) 2024-04-25T09:03:30.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2bfe00640 1 -- 172.21.15.62:0/3346568061 learned_addr learned my addr 172.21.15.62:0/3346568061 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2cdc00640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c80b6a90 0x7ff2c80cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2cd200640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b27a0 0x7ff2c815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2bfe00640 1 -- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b27a0 msgr2=0x7ff2c815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2bfe00640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b27a0 0x7ff2c815eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2bfe00640 1 -- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c80b6a90 msgr2=0x7ff2c80cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2bfe00640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c80b6a90 0x7ff2c80cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2bfe00640 1 -- 172.21.15.62:0/3346568061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c81e7d90 con 0x7ff2c80b56f0 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2cdc00640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c80b6a90 0x7ff2c80cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2bfe00640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8151f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000b440 tx=0x7ff2b000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.621+0000 7ff2c5600640 1 -- 172.21.15.62:0/3346568061 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0015020 con 0x7ff2c80b56f0 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.622+0000 7ff2c5600640 1 -- 172.21.15.62:0/3346568061 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b0011890 con 0x7ff2c80b56f0 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.622+0000 7ff2c5600640 1 -- 172.21.15.62:0/3346568061 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0011c80 con 0x7ff2c80b56f0 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.622+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c81e77d0 con 0x7ff2c80b56f0 2024-04-25T09:03:30.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.622+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c80cc5d0 con 0x7ff2c80b56f0 2024-04-25T09:03:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.623+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c804ef20 con 0x7ff2c80b56f0 2024-04-25T09:03:30.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.624+0000 7ff2c5600640 1 -- 172.21.15.62:0/3346568061 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2b0018020 con 0x7ff2c80b56f0 2024-04-25T09:03:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.625+0000 7ff2c5600640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2a406b4d0 0x7ff2a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.625+0000 7ff2c5600640 1 -- 172.21.15.62:0/3346568061 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2b008a2a0 con 0x7ff2c80b56f0 2024-04-25T09:03:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.626+0000 7ff2cd200640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2a406b4d0 0x7ff2a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.626+0000 7ff2cd200640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2a406b4d0 0x7ff2a406d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8034000 tx=0x7ff2b802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.629+0000 7ff2c5600640 1 -- 172.21.15.62:0/3346568061 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2b001f020 con 0x7ff2c80b56f0 2024-04-25T09:03:30.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.870+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2c80b6180 con 0x7ff2c80b56f0 2024-04-25T09:03:30.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.871+0000 7ff2c5600640 1 -- 172.21.15.62:0/3346568061 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7ff2b000d040 con 0x7ff2c80b56f0 2024-04-25T09:03:30.859 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:30.859 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:16.609043+0000","modified":"2024-04-25T09:03:19.983991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:03:30.860 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.873+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2a406b4d0 msgr2=0x7ff2a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.873+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2a406b4d0 0x7ff2a406d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8034000 tx=0x7ff2b802e000 comp rx=0 tx=0).stop 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.873+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 msgr2=0x7ff2c8151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.873+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8151f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000b440 tx=0x7ff2b000b900 comp rx=0 tx=0).stop 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.874+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 shutdown_connections 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.874+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff2a406b4d0 0x7ff2a406d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.874+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff2c80b56f0 0x7ff2c8151f10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.874+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2c80b6a90 0x7ff2c80cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.874+0000 7ff2ce600640 1 --2- 172.21.15.62:0/3346568061 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff2c80b27a0 0x7ff2c815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.874+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 >> 172.21.15.62:0/3346568061 conn(0x7ff2c8066d10 msgr2=0x7ff2c8160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.874+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 shutdown_connections 2024-04-25T09:03:30.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:30.874+0000 7ff2ce600640 1 -- 172.21.15.62:0/3346568061 wait complete. 2024-04-25T09:03:30.878 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:31.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.094+0000 7f81e6800640 1 Processor -- start 2024-04-25T09:03:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.094+0000 7f81e6800640 1 -- start start 2024-04-25T09:03:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e6800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81e0144c10 0x7f81e015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e6800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f81e00cee40 0x7f81e006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e6800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e00cf2d0 con 0x7f81e006d730 2024-04-25T09:03:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e6800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e0144ff0 con 0x7f81e00cee40 2024-04-25T09:03:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e6800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e006dba0 con 0x7f81e0144c10 2024-04-25T09:03:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e4a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f81e00cee40 0x7f81e006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e4a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f81e00cee40 0x7f81e006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55168/0 (socket says 172.21.15.62:55168) 2024-04-25T09:03:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e4a00640 1 -- 172.21.15.62:0/288620251 learned_addr learned my addr 172.21.15.62:0/288620251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e5400640 1 --2- 172.21.15.62:0/288620251 >> v2:172.21.15.121:3300/0 conn(0x7f81e0144c10 0x7f81e015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e4a00640 1 -- 172.21.15.62:0/288620251 >> v2:172.21.15.121:3300/0 conn(0x7f81e0144c10 msgr2=0x7f81e015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e4a00640 1 --2- 172.21.15.62:0/288620251 >> v2:172.21.15.121:3300/0 conn(0x7f81e0144c10 0x7f81e015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.096+0000 7f81e4a00640 1 -- 172.21.15.62:0/288620251 >> v1:172.21.15.121:6789/0 conn(0x7f81e006d730 legacy=0x7f81e00b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.097+0000 7f81e4a00640 1 -- 172.21.15.62:0/288620251 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e015faa0 con 0x7f81e00cee40 2024-04-25T09:03:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.098+0000 7f81e4a00640 1 --2- 172.21.15.62:0/288620251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e00cee40 0x7f81e006d1f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f81e00724a0 tx=0x7f81cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347bdf2a8f01faeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.098+0000 7f81dfe00640 1 -- 172.21.15.62:0/288620251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc037070 con 0x7f81e00cee40 2024-04-25T09:03:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.098+0000 7f81dfe00640 1 -- 172.21.15.62:0/288620251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc032440 con 0x7f81e00cee40 2024-04-25T09:03:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.098+0000 7f81dfe00640 1 -- 172.21.15.62:0/288620251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc032810 con 0x7f81e00cee40 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.098+0000 7f81e6800640 1 -- 172.21.15.62:0/288620251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e00cee40 msgr2=0x7f81e006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.098+0000 7f81e6800640 1 --2- 172.21.15.62:0/288620251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e00cee40 0x7f81e006d1f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f81e00724a0 tx=0x7f81cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.099+0000 7f81e6800640 1 -- 172.21.15.62:0/288620251 shutdown_connections 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.099+0000 7f81e6800640 1 --2- 172.21.15.62:0/288620251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e00cee40 0x7f81e006d1f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.099+0000 7f81e6800640 1 --2- 172.21.15.62:0/288620251 >> v2:172.21.15.121:3300/0 conn(0x7f81e0144c10 0x7f81e015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.099+0000 7f81e6800640 1 -- 172.21.15.62:0/288620251 >> 172.21.15.62:0/288620251 conn(0x7f81e00b14d0 msgr2=0x7f81e00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.099+0000 7f81e6800640 1 -- 172.21.15.62:0/288620251 shutdown_connections 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.099+0000 7f81e6800640 1 -- 172.21.15.62:0/288620251 wait complete. 2024-04-25T09:03:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.100+0000 7f81e6800640 1 Processor -- start 2024-04-25T09:03:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.101+0000 7f81e6800640 1 -- start start 2024-04-25T09:03:31.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.101+0000 7f81e6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81e006d730 0x7f81e014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.101+0000 7f81e6800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81e00cee40 0x7f81e014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.101+0000 7f81e6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e0144c10 0x7f81e01e5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.101+0000 7f81e6800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e00b41c0 con 0x7f81e0144c10 2024-04-25T09:03:31.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.101+0000 7f81e6800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e0160340 con 0x7f81e006d730 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.101+0000 7f81e6800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e0160060 con 0x7f81e00cee40 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e0144c10 0x7f81e01e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e0144c10 0x7f81e01e5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55172/0 (socket says 172.21.15.62:55172) 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 -- 172.21.15.62:0/1988649878 learned_addr learned my addr 172.21.15.62:0/1988649878 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5400640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81e006d730 0x7f81e014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e4a00640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81e00cee40 0x7f81e014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 -- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81e00cee40 msgr2=0x7f81e014df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81e00cee40 0x7f81e014df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 -- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81e006d730 msgr2=0x7f81e014da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81e006d730 0x7f81e014da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 -- 172.21.15.62:0/1988649878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e0139e10 con 0x7f81e0144c10 2024-04-25T09:03:31.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.102+0000 7f81e5e00640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e0144c10 0x7f81e01e5460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c780 tx=0x7f81d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.103+0000 7f81dd600640 1 -- 172.21.15.62:0/1988649878 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8017070 con 0x7f81e0144c10 2024-04-25T09:03:31.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.103+0000 7f81dd600640 1 -- 172.21.15.62:0/1988649878 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d8003da0 con 0x7f81e0144c10 2024-04-25T09:03:31.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.103+0000 7f81dd600640 1 -- 172.21.15.62:0/1988649878 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d80052b0 con 0x7f81e0144c10 2024-04-25T09:03:31.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.103+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e013a0f0 con 0x7f81e0144c10 2024-04-25T09:03:31.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.103+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e013b400 con 0x7f81e0144c10 2024-04-25T09:03:31.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.104+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e013a2c0 con 0x7f81e0144c10 2024-04-25T09:03:31.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.105+0000 7f81dd600640 1 -- 172.21.15.62:0/1988649878 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81d801a070 con 0x7f81e0144c10 2024-04-25T09:03:31.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.106+0000 7f81dd600640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f81c006b4d0 0x7f81c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.106+0000 7f81dd600640 1 -- 172.21.15.62:0/1988649878 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81d8089910 con 0x7f81e0144c10 2024-04-25T09:03:31.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.107+0000 7f81e5400640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f81c006b4d0 0x7f81c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.108+0000 7f81e5400640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f81c006b4d0 0x7f81c006d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f81d4006fd0 tx=0x7f81d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.110+0000 7f81dd600640 1 -- 172.21.15.62:0/1988649878 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81d80586f0 con 0x7f81e0144c10 2024-04-25T09:03:31.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.310+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81e004ef20 con 0x7f81e0144c10 2024-04-25T09:03:31.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.312+0000 7f81dd600640 1 -- 172.21.15.62:0/1988649878 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f81d8057ff0 con 0x7f81e0144c10 2024-04-25T09:03:31.300 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:31.300 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:31.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f81c006b4d0 msgr2=0x7f81c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f81c006b4d0 0x7f81c006d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f81d4006fd0 tx=0x7f81d4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:31.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e0144c10 msgr2=0x7f81e01e5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e0144c10 0x7f81e01e5460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81d800c780 tx=0x7f81d800cc40 comp rx=0 tx=0).stop 2024-04-25T09:03:31.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 shutdown_connections 2024-04-25T09:03:31.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f81c006b4d0 0x7f81c006d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81e0144c10 0x7f81e01e5460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81e00cee40 0x7f81e014df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 --2- 172.21.15.62:0/1988649878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81e006d730 0x7f81e014da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 >> 172.21.15.62:0/1988649878 conn(0x7f81e00b14d0 msgr2=0x7f81e00b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 shutdown_connections 2024-04-25T09:03:31.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.315+0000 7f81e6800640 1 -- 172.21.15.62:0/1988649878 wait complete. 2024-04-25T09:03:31.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:31.323 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.553+0000 7f4bb8000640 1 Processor -- start 2024-04-25T09:03:31.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.553+0000 7f4bb8000640 1 -- start start 2024-04-25T09:03:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.554+0000 7f4bb8000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.555+0000 7f4bb8000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb00d2270 con 0x7f4bb015dc10 2024-04-25T09:03:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.555+0000 7f4bb8000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb00b7590 con 0x7f4bb00b97b0 2024-04-25T09:03:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.555+0000 7f4bb8000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00b8fe0 con 0x7f4bb00b8630 2024-04-25T09:03:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.555+0000 7f4bb5600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4bb015dc10 0x7f4bb00d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57664/0 (socket says 172.21.15.62:57664) 2024-04-25T09:03:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.555+0000 7f4bb5600640 1 -- 172.21.15.62:0/202277875 learned_addr learned my addr 172.21.15.62:0/202277875 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.555+0000 7f4bb6000640 1 --2- 172.21.15.62:0/202277875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.555+0000 7f4bafe00640 1 -- 172.21.15.62:0/202277875 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787628083 0 0) 0x7f4bb00d2270 con 0x7f4bb015dc10 2024-04-25T09:03:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.555+0000 7f4bafe00640 1 -- 172.21.15.62:0/202277875 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b98003520 con 0x7f4bb015dc10 2024-04-25T09:03:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.556+0000 7f4bb6000640 1 -- 172.21.15.62:0/202277875 >> v1:172.21.15.121:6789/0 conn(0x7f4bb00b97b0 legacy=0x7f4bb00b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.556+0000 7f4bb6000640 1 -- 172.21.15.62:0/202277875 >> v1:172.21.15.62:6789/0 conn(0x7f4bb015dc10 legacy=0x7f4bb00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.556+0000 7f4bb6000640 1 -- 172.21.15.62:0/202277875 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00b7870 con 0x7f4bb00b8630 2024-04-25T09:03:31.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.557+0000 7f4bb6000640 1 --2- 172.21.15.62:0/202277875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb00b8aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009dc0 tx=0x7f4ba802a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9752cac31bb2cc21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.562+0000 7f4bafe00640 1 -- 172.21.15.62:0/202277875 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8007740 con 0x7f4bb00b8630 2024-04-25T09:03:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.562+0000 7f4bafe00640 1 -- 172.21.15.62:0/202277875 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba80078e0 con 0x7f4bb00b8630 2024-04-25T09:03:31.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.562+0000 7f4bafe00640 1 -- 172.21.15.62:0/202277875 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8007c90 con 0x7f4bb00b8630 2024-04-25T09:03:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.563+0000 7f4bb8000640 1 -- 172.21.15.62:0/202277875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 msgr2=0x7f4bb00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.563+0000 7f4bb8000640 1 --2- 172.21.15.62:0/202277875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb00b8aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009dc0 tx=0x7f4ba802a760 comp rx=0 tx=0).stop 2024-04-25T09:03:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.563+0000 7f4bb8000640 1 -- 172.21.15.62:0/202277875 shutdown_connections 2024-04-25T09:03:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.563+0000 7f4bb8000640 1 --2- 172.21.15.62:0/202277875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb00b8aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.563+0000 7f4bb8000640 1 -- 172.21.15.62:0/202277875 >> 172.21.15.62:0/202277875 conn(0x7f4bb006b000 msgr2=0x7f4bb006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.563+0000 7f4bb8000640 1 -- 172.21.15.62:0/202277875 shutdown_connections 2024-04-25T09:03:31.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.563+0000 7f4bb8000640 1 -- 172.21.15.62:0/202277875 wait complete. 2024-04-25T09:03:31.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.565+0000 7f4bb8000640 1 Processor -- start 2024-04-25T09:03:31.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.565+0000 7f4bb8000640 1 -- start start 2024-04-25T09:03:31.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb8000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb0152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb8000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bb00b97b0 0x7f4bb015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb8000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bb015dc10 0x7f4bb015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb8000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00b7e30 con 0x7f4bb00b97b0 2024-04-25T09:03:31.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb8000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00b81a0 con 0x7f4bb015dc10 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb8000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00b72a0 con 0x7f4bb00b8630 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bb00b97b0 0x7f4bb015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bb00b97b0 0x7f4bb015eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55186/0 (socket says 172.21.15.62:55186) 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb4c00640 1 -- 172.21.15.62:0/3008820044 learned_addr learned my addr 172.21.15.62:0/3008820044 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb5600640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb0152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.566+0000 7f4bb6000640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bb015dc10 0x7f4bb015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb4c00640 1 -- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 msgr2=0x7f4bb0152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb4c00640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb0152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb4c00640 1 -- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bb015dc10 msgr2=0x7f4bb015f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb4c00640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bb015dc10 0x7f4bb015f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb4c00640 1 -- 172.21.15.62:0/3008820044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb00c9ff0 con 0x7f4bb00b97b0 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb6000640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bb015dc10 0x7f4bb015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:31.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb4c00640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bb00b97b0 0x7f4bb015eb20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00c450 tx=0x7f4b9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bad600640 1 -- 172.21.15.62:0/3008820044 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0052d0 con 0x7f4bb00b97b0 2024-04-25T09:03:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bad600640 1 -- 172.21.15.62:0/3008820044 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c005470 con 0x7f4bb00b97b0 2024-04-25T09:03:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb00cbc30 con 0x7f4bb00b97b0 2024-04-25T09:03:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bad600640 1 -- 172.21.15.62:0/3008820044 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c012870 con 0x7f4bb00b97b0 2024-04-25T09:03:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.567+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb00cc1f0 con 0x7f4bb00b97b0 2024-04-25T09:03:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.569+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb00cfc70 con 0x7f4bb00b97b0 2024-04-25T09:03:31.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.569+0000 7f4bad600640 1 -- 172.21.15.62:0/3008820044 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b9c012a10 con 0x7f4bb00b97b0 2024-04-25T09:03:31.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.571+0000 7f4bad600640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4b9006b4d0 0x7f4b9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.571+0000 7f4bad600640 1 -- 172.21.15.62:0/3008820044 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b9c089ed0 con 0x7f4bb00b97b0 2024-04-25T09:03:31.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.571+0000 7f4bb5600640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4b9006b4d0 0x7f4b9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.572+0000 7f4bb5600640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4b9006b4d0 0x7f4b9006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4005880 tx=0x7f4ba4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.575+0000 7f4bad600640 1 -- 172.21.15.62:0/3008820044 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bb00cfc70 con 0x7f4bb00b97b0 2024-04-25T09:03:31.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.774+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bb004ef20 con 0x7f4bb00b97b0 2024-04-25T09:03:31.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.775+0000 7f4bad600640 1 -- 172.21.15.62:0/3008820044 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b9c058cb0 con 0x7f4bb00b97b0 2024-04-25T09:03:31.763 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4b9006b4d0 msgr2=0x7f4b9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4b9006b4d0 0x7f4b9006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4005880 tx=0x7f4ba4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bb00b97b0 msgr2=0x7f4bb015eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bb00b97b0 0x7f4bb015eb20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00c450 tx=0x7f4b9c00c910 comp rx=0 tx=0).stop 2024-04-25T09:03:31.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 shutdown_connections 2024-04-25T09:03:31.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4b9006b4d0 0x7f4b9006d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4bb00b97b0 0x7f4bb015eb20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bb015dc10 0x7f4bb015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 --2- 172.21.15.62:0/3008820044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4bb00b8630 0x7f4bb0152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 >> 172.21.15.62:0/3008820044 conn(0x7f4bb006b000 msgr2=0x7f4bb00b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 shutdown_connections 2024-04-25T09:03:31.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.778+0000 7f4bb8000640 1 -- 172.21.15.62:0/3008820044 wait complete. 2024-04-25T09:03:31.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:31.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.999+0000 7fe2eae00640 1 Processor -- start 2024-04-25T09:03:31.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:31.999+0000 7fe2eae00640 1 -- start start 2024-04-25T09:03:31.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.000+0000 7fe2eae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2e40d2fb0 0x7fe2e40ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.000+0000 7fe2eae00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e40ce780 con 0x7fe2e415dd50 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.000+0000 7fe2eae00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e40d3440 con 0x7fe2e40d3960 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.000+0000 7fe2eae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40ce1c0 con 0x7fe2e40d2fb0 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.001+0000 7fe2e9a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe2e415dd50 0x7fe2e40d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57672/0 (socket says 172.21.15.62:57672) 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.001+0000 7fe2e9a00640 1 -- 172.21.15.62:0/2317356305 learned_addr learned my addr 172.21.15.62:0/2317356305 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.001+0000 7fe2e9000640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2e40d2fb0 0x7fe2e40ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.001+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946587156 0 0) 0x7fe2e40ce780 con 0x7fe2e415dd50 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.001+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2cc003520 con 0x7fe2e415dd50 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3231037319 0 0) 0x7fe2cc003520 con 0x7fe2e415dd50 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2e40d2fb0 msgr2=0x7fe2e40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2e40d2fb0 0x7fe2e40ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 >> v1:172.21.15.121:6789/0 conn(0x7fe2e40d3960 legacy=0x7fe2e40d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e40ce4a0 con 0x7fe2e415dd50 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fe2d8001670 con 0x7fe2e415dd50 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 >> v1:172.21.15.62:6789/0 conn(0x7fe2e415dd50 legacy=0x7fe2e40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2cc0039e0 0x7fe2cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc004800 0x7fe2cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2cc005180 0x7fe2cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40ce1c0 con 0x7fe2cc005180 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc00bf00 con 0x7fe2cc0039e0 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc00c080 con 0x7fe2cc004800 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2ea400640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2cc005180 0x7fe2cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e9000640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc004800 0x7fe2cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2ea400640 1 -- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc004800 msgr2=0x7fe2cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2e9a00640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2cc0039e0 0x7fe2cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2ea400640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc004800 0x7fe2cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2ea400640 1 -- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2cc0039e0 msgr2=0x7fe2cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2ea400640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2cc0039e0 0x7fe2cc003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.002+0000 7fe2ea400640 1 -- 172.21.15.62:0/2317356305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc003520 con 0x7fe2cc005180 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.003+0000 7fe2e9000640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc004800 0x7fe2cc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.003+0000 7fe2ea400640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2cc005180 0x7fe2cc00b9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc001e50 tx=0x7fe2cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf198e0b513bf8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.004+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc001a70 con 0x7fe2cc005180 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.004+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc003970 con 0x7fe2cc005180 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.004+0000 7fe2e3e00640 1 -- 172.21.15.62:0/2317356305 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc02ae60 con 0x7fe2cc005180 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.004+0000 7fe2eae00640 1 -- 172.21.15.62:0/2317356305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2cc005180 msgr2=0x7fe2cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.004+0000 7fe2eae00640 1 --2- 172.21.15.62:0/2317356305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2cc005180 0x7fe2cc00b9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc001e50 tx=0x7fe2cc005830 comp rx=0 tx=0).stop 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.004+0000 7fe2ea400640 1 -- 172.21.15.62:0/2317356305 reap_dead start 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.004+0000 7fe2eae00640 1 -- 172.21.15.62:0/2317356305 shutdown_connections 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.004+0000 7fe2eae00640 1 -- 172.21.15.62:0/2317356305 >> 172.21.15.62:0/2317356305 conn(0x7fe2e406b000 msgr2=0x7fe2e406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.005+0000 7fe2eae00640 1 -- 172.21.15.62:0/2317356305 shutdown_connections 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.005+0000 7fe2eae00640 1 -- 172.21.15.62:0/2317356305 wait complete. 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.006+0000 7fe2eae00640 1 Processor -- start 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.006+0000 7fe2eae00640 1 -- start start 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.007+0000 7fe2eae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc005180 0x7fe2e414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.007+0000 7fe2eae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2e4160330 0x7fe2e41607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.007+0000 7fe2eae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2e41e6180 0x7fe2e41e6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.007+0000 7fe2eae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40ced50 con 0x7fe2e41e6180 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.007+0000 7fe2eae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40cebd0 con 0x7fe2e4160330 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.007+0000 7fe2eae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e40ceed0 con 0x7fe2cc005180 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2ea400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2e41e6180 0x7fe2e41e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2ea400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2e41e6180 0x7fe2e41e6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55198/0 (socket says 172.21.15.62:55198) 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2ea400640 1 -- 172.21.15.62:0/4180258937 learned_addr learned my addr 172.21.15.62:0/4180258937 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2e9a00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc005180 0x7fe2e414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2e9000640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2e4160330 0x7fe2e41607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2ea400640 1 -- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc005180 msgr2=0x7fe2e414ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2ea400640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc005180 0x7fe2e414ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2ea400640 1 -- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2e4160330 msgr2=0x7fe2e41607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2ea400640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2e4160330 0x7fe2e41607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.008+0000 7fe2ea400640 1 -- 172.21.15.62:0/4180258937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e414f2d0 con 0x7fe2e41e6180 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.009+0000 7fe2ea400640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2e41e6180 0x7fe2e41e6600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc001f70 tx=0x7fe2dc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.009+0000 7fe2e1600640 1 -- 172.21.15.62:0/4180258937 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc001a70 con 0x7fe2e41e6180 2024-04-25T09:03:31.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.009+0000 7fe2e1600640 1 -- 172.21.15.62:0/4180258937 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc041070 con 0x7fe2e41e6180 2024-04-25T09:03:31.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.009+0000 7fe2e1600640 1 -- 172.21.15.62:0/4180258937 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc007db0 con 0x7fe2e41e6180 2024-04-25T09:03:31.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.009+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e41e6e90 con 0x7fe2e41e6180 2024-04-25T09:03:31.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.009+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e41e7450 con 0x7fe2e41e6180 2024-04-25T09:03:31.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.010+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e404ef20 con 0x7fe2e41e6180 2024-04-25T09:03:31.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.011+0000 7fe2e1600640 1 -- 172.21.15.62:0/4180258937 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2dc03a070 con 0x7fe2e41e6180 2024-04-25T09:03:32.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.012+0000 7fe2e1600640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe2c406b500 0x7fe2c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.013+0000 7fe2e1600640 1 -- 172.21.15.62:0/4180258937 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2dc0a9bb0 con 0x7fe2e41e6180 2024-04-25T09:03:32.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.013+0000 7fe2e9a00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe2c406b500 0x7fe2c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.014+0000 7fe2e9a00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe2c406b500 0x7fe2c406d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8001eb0 tx=0x7fe2d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.016+0000 7fe2e1600640 1 -- 172.21.15.62:0/4180258937 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2dc078990 con 0x7fe2e41e6180 2024-04-25T09:03:32.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.207+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2e40b9950 con 0x7fe2e41e6180 2024-04-25T09:03:32.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.208+0000 7fe2e1600640 1 -- 172.21.15.62:0/4180258937 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2dc078290 con 0x7fe2e41e6180 2024-04-25T09:03:32.196 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:03:32.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.210+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe2c406b500 msgr2=0x7fe2c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.210+0000 7fe2eae00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe2c406b500 0x7fe2c406d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8001eb0 tx=0x7fe2d8009040 comp rx=0 tx=0).stop 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.210+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2e41e6180 msgr2=0x7fe2e41e6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.210+0000 7fe2eae00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2e41e6180 0x7fe2e41e6600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc001f70 tx=0x7fe2dc0014a0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.210+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 shutdown_connections 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.210+0000 7fe2eae00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe2c406b500 0x7fe2c406d9c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.210+0000 7fe2eae00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2e41e6180 0x7fe2e41e6600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.211+0000 7fe2eae00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2e4160330 0x7fe2e41607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.211+0000 7fe2eae00640 1 --2- 172.21.15.62:0/4180258937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2cc005180 0x7fe2e414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.211+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 >> 172.21.15.62:0/4180258937 conn(0x7fe2e406b000 msgr2=0x7fe2e40b8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.211+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 shutdown_connections 2024-04-25T09:03:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.211+0000 7fe2eae00640 1 -- 172.21.15.62:0/4180258937 wait complete. 2024-04-25T09:03:32.217 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:32.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.438+0000 7f8ff7e00640 1 Processor -- start 2024-04-25T09:03:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.439+0000 7f8ff7e00640 1 -- start start 2024-04-25T09:03:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.440+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80b31a0 0x7f8ff80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.440+0000 7f8ff7e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff80cd260 con 0x7f8ff806d2c0 2024-04-25T09:03:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.440+0000 7f8ff7e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff80cf000 con 0x7f8ff80ccde0 2024-04-25T09:03:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.440+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80cf2e0 con 0x7f8ff80b31a0 2024-04-25T09:03:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.441+0000 7f8ff6a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8ff806d2c0 0x7f8ff806d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57688/0 (socket says 172.21.15.62:57688) 2024-04-25T09:03:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.441+0000 7f8ff6a00640 1 -- 172.21.15.62:0/3892716553 learned_addr learned my addr 172.21.15.62:0/3892716553 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.441+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713374631 0 0) 0x7f8ff80cd260 con 0x7f8ff806d2c0 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.441+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fdc003520 con 0x7f8ff806d2c0 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.441+0000 7f8ff7400640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80b31a0 0x7f8ff80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2366415728 0 0) 0x7f8fdc003520 con 0x7f8ff806d2c0 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80b31a0 msgr2=0x7f8ff80b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80b31a0 0x7f8ff80b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 >> v1:172.21.15.121:6789/0 conn(0x7f8ff80ccde0 legacy=0x7f8ff80ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff80b26c0 con 0x7f8ff806d2c0 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f8fe8001670 con 0x7f8ff806d2c0 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 >> v1:172.21.15.62:6789/0 conn(0x7f8ff806d2c0 legacy=0x7f8ff806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fdc0039e0 0x7f8fdc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80b31a0 0x7f8fdc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8fdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80cf2e0 con 0x7f8fdc004d40 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc00bb50 con 0x7f8fdc0039e0 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc00bd50 con 0x7f8ff80b31a0 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff7400640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8fdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff6a00640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fdc0039e0 0x7f8fdc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8fefe00640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80b31a0 0x7f8fdc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff7400640 1 -- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80b31a0 msgr2=0x7f8fdc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.442+0000 7f8ff7400640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80b31a0 0x7f8fdc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.443+0000 7f8ff7400640 1 -- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fdc0039e0 msgr2=0x7f8fdc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.443+0000 7f8ff7400640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fdc0039e0 0x7f8fdc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.443+0000 7f8ff7400640 1 -- 172.21.15.62:0/3892716553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc003520 con 0x7f8fdc004d40 2024-04-25T09:03:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.444+0000 7f8ff7400640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8fdc0051c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0007360 tx=0x7f8fdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd55bff2bc05f6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.444+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0009be0 con 0x7f8fdc004d40 2024-04-25T09:03:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.444+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff0036600 con 0x7f8fdc004d40 2024-04-25T09:03:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.444+0000 7f8ff6000640 1 -- 172.21.15.62:0/3892716553 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00369d0 con 0x7f8fdc004d40 2024-04-25T09:03:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.444+0000 7f8ff7e00640 1 -- 172.21.15.62:0/3892716553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 msgr2=0x7f8fdc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.444+0000 7f8ff7e00640 1 --2- 172.21.15.62:0/3892716553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8fdc0051c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0007360 tx=0x7f8fdc005830 comp rx=0 tx=0).stop 2024-04-25T09:03:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.445+0000 7f8ff7400640 1 -- 172.21.15.62:0/3892716553 reap_dead start 2024-04-25T09:03:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.445+0000 7f8ff7e00640 1 -- 172.21.15.62:0/3892716553 shutdown_connections 2024-04-25T09:03:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.445+0000 7f8ff7e00640 1 -- 172.21.15.62:0/3892716553 >> 172.21.15.62:0/3892716553 conn(0x7f8ff8066f20 msgr2=0x7f8ff80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.445+0000 7f8ff7e00640 1 -- 172.21.15.62:0/3892716553 shutdown_connections 2024-04-25T09:03:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.445+0000 7f8ff7e00640 1 -- 172.21.15.62:0/3892716553 wait complete. 2024-04-25T09:03:32.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.446+0000 7f8ff7e00640 1 Processor -- start 2024-04-25T09:03:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.447+0000 7f8ff7e00640 1 -- start start 2024-04-25T09:03:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8ff8140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ff8070fe0 0x7f8ff8071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80acad0 0x7f8ff80acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff81448d0 con 0x7f8fdc004d40 2024-04-25T09:03:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80b2c80 con 0x7f8ff8070fe0 2024-04-25T09:03:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff8144a50 con 0x7f8ff80acad0 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8ff8140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8ff8140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55226/0 (socket says 172.21.15.62:55226) 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff6a00640 1 -- 172.21.15.62:0/2477807412 learned_addr learned my addr 172.21.15.62:0/2477807412 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8fefe00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ff8070fe0 0x7f8ff8071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff7400640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80acad0 0x7f8ff80acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff6a00640 1 -- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80acad0 msgr2=0x7f8ff80acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff6a00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80acad0 0x7f8ff80acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff6a00640 1 -- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ff8070fe0 msgr2=0x7f8ff8071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff6a00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ff8070fe0 0x7f8ff8071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8ff6a00640 1 -- 172.21.15.62:0/2477807412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff8071aa0 con 0x7f8fdc004d40 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.448+0000 7f8fefe00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ff8070fe0 0x7f8ff8071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.449+0000 7f8ff6a00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8ff8140090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000f60 tx=0x7f8fe800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.449+0000 7f8fef400640 1 -- 172.21.15.62:0/2477807412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800c870 con 0x7f8fdc004d40 2024-04-25T09:03:32.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.449+0000 7f8fef400640 1 -- 172.21.15.62:0/2477807412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8018070 con 0x7f8fdc004d40 2024-04-25T09:03:32.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.449+0000 7f8fef400640 1 -- 172.21.15.62:0/2477807412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800cc70 con 0x7f8fdc004d40 2024-04-25T09:03:32.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.449+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff8071d80 con 0x7f8fdc004d40 2024-04-25T09:03:32.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.449+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff80c8ae0 con 0x7f8fdc004d40 2024-04-25T09:03:32.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.450+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff806abe0 con 0x7f8fdc004d40 2024-04-25T09:03:32.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.451+0000 7f8fef400640 1 -- 172.21.15.62:0/2477807412 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fe801d460 con 0x7f8fdc004d40 2024-04-25T09:03:32.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.452+0000 7f8fef400640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.452+0000 7f8fef400640 1 -- 172.21.15.62:0/2477807412 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe808a2c0 con 0x7f8fdc004d40 2024-04-25T09:03:32.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.453+0000 7f8fefe00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.454+0000 7f8fefe00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8fd406b4d0 0x7f8fd406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80b1840 tx=0x7f8fe0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.456+0000 7f8fef400640 1 -- 172.21.15.62:0/2477807412 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fe8059090 con 0x7f8fdc004d40 2024-04-25T09:03:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.657+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ff804ef20 con 0x7f8fdc004d40 2024-04-25T09:03:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.658+0000 7f8fef400640 1 -- 172.21.15.62:0/2477807412 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8fe8058990 con 0x7f8fdc004d40 2024-04-25T09:03:32.646 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:03:32.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8fd406b4d0 msgr2=0x7f8fd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8fd406b4d0 0x7f8fd406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80b1840 tx=0x7f8fe0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 msgr2=0x7f8ff8140090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8ff8140090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000f60 tx=0x7f8fe800ecd0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 shutdown_connections 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8fd406b4d0 0x7f8fd406d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ff80acad0 0x7f8ff80acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ff8070fe0 0x7f8ff8071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 --2- 172.21.15.62:0/2477807412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fdc004d40 0x7f8ff8140090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.660+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 >> 172.21.15.62:0/2477807412 conn(0x7f8ff8066f20 msgr2=0x7f8ff806c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.661+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 shutdown_connections 2024-04-25T09:03:32.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.661+0000 7f8ff7e00640 1 -- 172.21.15.62:0/2477807412 wait complete. 2024-04-25T09:03:32.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:32.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.897+0000 7f1496a00640 1 Processor -- start 2024-04-25T09:03:32.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.898+0000 7f1496a00640 1 -- start start 2024-04-25T09:03:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.899+0000 7f1496a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14900b6a90 0x7f14900d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.899+0000 7f1496a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900d3110 con 0x7f14900b56f0 2024-04-25T09:03:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.899+0000 7f1496a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14900d33f0 con 0x7f14900b27a0 2024-04-25T09:03:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.899+0000 7f1496a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14900cedc0 con 0x7f14900b6a90 2024-04-25T09:03:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.899+0000 7f1495600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f14900b56f0 0x7f1490160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57696/0 (socket says 172.21.15.62:57696) 2024-04-25T09:03:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.899+0000 7f1495600640 1 -- 172.21.15.62:0/1690213332 learned_addr learned my addr 172.21.15.62:0/1690213332 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:32.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f1494c00640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14900b6a90 0x7f14900d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998141971 0 0) 0x7f14900d3110 con 0x7f14900b56f0 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1474003520 con 0x7f14900b56f0 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 463939425 0 0) 0x7f1474003520 con 0x7f14900b56f0 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14900b6a90 msgr2=0x7f14900d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14900b6a90 0x7f14900d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 >> v1:172.21.15.121:6789/0 conn(0x7f14900b27a0 legacy=0x7f14900d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900d36d0 con 0x7f14900b56f0 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1480001670 con 0x7f14900b56f0 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.900+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 >> v1:172.21.15.62:6789/0 conn(0x7f14900b56f0 legacy=0x7f1490160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f148fe00640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14740039e0 0x7f1474003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f148fe00640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1474004800 0x7f1474004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f148fe00640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14900b6a90 0x7f14740051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900cedc0 con 0x7f14740039e0 2024-04-25T09:03:32.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147400bad0 con 0x7f1474004800 2024-04-25T09:03:32.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147400bcd0 con 0x7f14900b6a90 2024-04-25T09:03:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f1495600640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14740039e0 0x7f1474003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f1494c00640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1474004800 0x7f1474004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f1496000640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14900b6a90 0x7f14740051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f1495600640 1 -- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14900b6a90 msgr2=0x7f14740051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f1495600640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14900b6a90 0x7f14740051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f1495600640 1 -- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1474004800 msgr2=0x7f1474004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f1495600640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1474004800 0x7f1474004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.901+0000 7f1495600640 1 -- 172.21.15.62:0/1690213332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1474003520 con 0x7f14740039e0 2024-04-25T09:03:32.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.902+0000 7f1495600640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14740039e0 0x7f1474003e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f148000c0e0 tx=0x7f1474005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da3bb727a43094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.903+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480007c20 con 0x7f14740039e0 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.903+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1480005af0 con 0x7f14740039e0 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.903+0000 7f148fe00640 1 -- 172.21.15.62:0/1690213332 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480007070 con 0x7f14740039e0 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.903+0000 7f1496a00640 1 -- 172.21.15.62:0/1690213332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14740039e0 msgr2=0x7f1474003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.903+0000 7f1496a00640 1 --2- 172.21.15.62:0/1690213332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14740039e0 0x7f1474003e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f148000c0e0 tx=0x7f1474005830 comp rx=0 tx=0).stop 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.903+0000 7f1496000640 1 -- 172.21.15.62:0/1690213332 reap_dead start 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.904+0000 7f1496a00640 1 -- 172.21.15.62:0/1690213332 shutdown_connections 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.904+0000 7f1496a00640 1 -- 172.21.15.62:0/1690213332 >> 172.21.15.62:0/1690213332 conn(0x7f1490066d10 msgr2=0x7f1490067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.904+0000 7f1496a00640 1 -- 172.21.15.62:0/1690213332 shutdown_connections 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.904+0000 7f1496a00640 1 -- 172.21.15.62:0/1690213332 wait complete. 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.906+0000 7f1496a00640 1 Processor -- start 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.906+0000 7f1496a00640 1 -- start start 2024-04-25T09:03:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14740039e0 0x7f149015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f149015dea0 0x7f14901f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14901e7280 0x7f14901e7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b2ee0 con 0x7f14901e7280 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b2d60 con 0x7f149015dea0 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14900d2c30 con 0x7f14740039e0 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14901e7280 0x7f14901e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14901e7280 0x7f14901e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55256/0 (socket says 172.21.15.62:55256) 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496000640 1 -- 172.21.15.62:0/3802267408 learned_addr learned my addr 172.21.15.62:0/3802267408 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496000640 1 -- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14740039e0 msgr2=0x7f149015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496000640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14740039e0 0x7f149015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496000640 1 -- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f149015dea0 msgr2=0x7f14901f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1494c00640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f149015dea0 0x7f14901f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496000640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f149015dea0 0x7f14901f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.907+0000 7f1496000640 1 -- 172.21.15.62:0/3802267408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14900cad20 con 0x7f14901e7280 2024-04-25T09:03:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.908+0000 7f1496000640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14901e7280 0x7f14901e7710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f14900b2750 tx=0x7f148800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.908+0000 7f148d600640 1 -- 172.21.15.62:0/3802267408 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488003970 con 0x7f14901e7280 2024-04-25T09:03:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.908+0000 7f148d600640 1 -- 172.21.15.62:0/3802267408 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1488018070 con 0x7f14901e7280 2024-04-25T09:03:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.908+0000 7f148d600640 1 -- 172.21.15.62:0/3802267408 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488003d50 con 0x7f14901e7280 2024-04-25T09:03:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.908+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14900cb000 con 0x7f14901e7280 2024-04-25T09:03:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.908+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149015dbc0 con 0x7f14901e7280 2024-04-25T09:03:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.909+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149004ef20 con 0x7f14901e7280 2024-04-25T09:03:32.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.910+0000 7f148d600640 1 -- 172.21.15.62:0/3802267408 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14880050d0 con 0x7f14901e7280 2024-04-25T09:03:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.911+0000 7f148d600640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147006b500 0x7f147006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.911+0000 7f148d600640 1 -- 172.21.15.62:0/3802267408 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1488089f20 con 0x7f14901e7280 2024-04-25T09:03:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.912+0000 7f1495600640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147006b500 0x7f147006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.913+0000 7f1495600640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147006b500 0x7f147006d9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1480032000 tx=0x7f1480007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:32.916+0000 7f148d600640 1 -- 172.21.15.62:0/3802267408 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1488058d90 con 0x7f14901e7280 2024-04-25T09:03:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.153+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14900ce580 con 0x7f14901e7280 2024-04-25T09:03:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.154+0000 7f148d600640 1 -- 172.21.15.62:0/3802267408 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f1488058690 con 0x7f14901e7280 2024-04-25T09:03:33.143 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:33.143 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:16.609043+0000","modified":"2024-04-25T09:03:19.983991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:03:33.143 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-25T09:03:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147006b500 msgr2=0x7f147006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147006b500 0x7f147006d9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1480032000 tx=0x7f1480007000 comp rx=0 tx=0).stop 2024-04-25T09:03:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14901e7280 msgr2=0x7f14901e7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14901e7280 0x7f14901e7710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f14900b2750 tx=0x7f148800ec10 comp rx=0 tx=0).stop 2024-04-25T09:03:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 shutdown_connections 2024-04-25T09:03:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147006b500 0x7f147006d9c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14901e7280 0x7f14901e7710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f149015dea0 0x7f14901f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 --2- 172.21.15.62:0/3802267408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14740039e0 0x7f149015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 >> 172.21.15.62:0/3802267408 conn(0x7f1490066d10 msgr2=0x7f14900c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 shutdown_connections 2024-04-25T09:03:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.158+0000 7f1496a00640 1 -- 172.21.15.62:0/3802267408 wait complete. 2024-04-25T09:03:33.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.390+0000 7f7210e00640 1 Processor -- start 2024-04-25T09:03:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.390+0000 7f7210e00640 1 -- start start 2024-04-25T09:03:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.392+0000 7f7210e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c13d970 0x7f720c0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.392+0000 7f7210e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f720c0b1050 0x7f720c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.392+0000 7f7210e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720c15b220 con 0x7f720c15acf0 2024-04-25T09:03:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.392+0000 7f7210e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720c15b500 con 0x7f720c0b1050 2024-04-25T09:03:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.392+0000 7f7210e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c15b7e0 con 0x7f720c13d970 2024-04-25T09:03:33.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.392+0000 7f720aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f720c0b1050 0x7f720c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.392+0000 7f720aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f720c0b1050 0x7f720c06d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55262/0 (socket says 172.21.15.62:55262) 2024-04-25T09:03:33.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.392+0000 7f720aa00640 1 -- 172.21.15.62:0/1998240187 learned_addr learned my addr 172.21.15.62:0/1998240187 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:33.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.393+0000 7f720aa00640 1 -- 172.21.15.62:0/1998240187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c13d970 msgr2=0x7f720c0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.393+0000 7f720b400640 1 --2- 172.21.15.62:0/1998240187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c13d970 0x7f720c0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.393+0000 7f720aa00640 1 --2- 172.21.15.62:0/1998240187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c13d970 0x7f720c0b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.393+0000 7f720aa00640 1 -- 172.21.15.62:0/1998240187 >> v1:172.21.15.121:6789/0 conn(0x7f720c15acf0 legacy=0x7f720c13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.393+0000 7f720aa00640 1 -- 172.21.15.62:0/1998240187 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c15bac0 con 0x7f720c0b1050 2024-04-25T09:03:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.394+0000 7f720aa00640 1 --2- 172.21.15.62:0/1998240187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c0b1050 0x7f720c06d1f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009870 tx=0x7f71f802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb7d1223b288441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.394+0000 7f720a000640 1 -- 172.21.15.62:0/1998240187 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8037070 con 0x7f720c0b1050 2024-04-25T09:03:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.394+0000 7f720a000640 1 -- 172.21.15.62:0/1998240187 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8032440 con 0x7f720c0b1050 2024-04-25T09:03:33.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.394+0000 7f720a000640 1 -- 172.21.15.62:0/1998240187 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8032810 con 0x7f720c0b1050 2024-04-25T09:03:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.395+0000 7f7210e00640 1 -- 172.21.15.62:0/1998240187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c0b1050 msgr2=0x7f720c06d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.395+0000 7f7210e00640 1 --2- 172.21.15.62:0/1998240187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c0b1050 0x7f720c06d1f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009870 tx=0x7f71f802a210 comp rx=0 tx=0).stop 2024-04-25T09:03:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.395+0000 7f7210e00640 1 -- 172.21.15.62:0/1998240187 shutdown_connections 2024-04-25T09:03:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.395+0000 7f7210e00640 1 --2- 172.21.15.62:0/1998240187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c0b1050 0x7f720c06d1f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.395+0000 7f7210e00640 1 --2- 172.21.15.62:0/1998240187 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c13d970 0x7f720c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.395+0000 7f7210e00640 1 -- 172.21.15.62:0/1998240187 >> 172.21.15.62:0/1998240187 conn(0x7f720c0b14d0 msgr2=0x7f720c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.395+0000 7f7210e00640 1 -- 172.21.15.62:0/1998240187 shutdown_connections 2024-04-25T09:03:33.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.395+0000 7f7210e00640 1 -- 172.21.15.62:0/1998240187 wait complete. 2024-04-25T09:03:33.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.397+0000 7f7210e00640 1 Processor -- start 2024-04-25T09:03:33.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.397+0000 7f7210e00640 1 -- start start 2024-04-25T09:03:33.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f7210e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c0b1050 0x7f720c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f7210e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f720c13d970 0x7f720c1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f7210e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c15acf0 0x7f720c1f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f7210e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c06d730 con 0x7f720c15acf0 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f7210e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c13f470 con 0x7f720c13d970 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f7210e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c06d8b0 con 0x7f720c0b1050 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f720be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c15acf0 0x7f720c1f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f720be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c15acf0 0x7f720c1f7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55264/0 (socket says 172.21.15.62:55264) 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f720be00640 1 -- 172.21.15.62:0/1688553615 learned_addr learned my addr 172.21.15.62:0/1688553615 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.398+0000 7f720aa00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f720c13d970 0x7f720c1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f720b400640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c0b1050 0x7f720c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f720be00640 1 -- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c0b1050 msgr2=0x7f720c159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f720be00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c0b1050 0x7f720c159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f720be00640 1 -- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f720c13d970 msgr2=0x7f720c1590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f720be00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f720c13d970 0x7f720c1590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f720be00640 1 -- 172.21.15.62:0/1688553615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c1595f0 con 0x7f720c15acf0 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f720aa00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f720c13d970 0x7f720c1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f720be00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c15acf0 0x7f720c1f7010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f720400c700 tx=0x7f720400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f7203400640 1 -- 172.21.15.62:0/1688553615 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204016020 con 0x7f720c15acf0 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f7203400640 1 -- 172.21.15.62:0/1688553615 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400aa50 con 0x7f720c15acf0 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c15a280 con 0x7f720c15acf0 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.399+0000 7f7203400640 1 -- 172.21.15.62:0/1688553615 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400ce10 con 0x7f720c15acf0 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.400+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c157600 con 0x7f720c15acf0 2024-04-25T09:03:33.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.401+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c157150 con 0x7f720c15acf0 2024-04-25T09:03:33.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.402+0000 7f7203400640 1 -- 172.21.15.62:0/1688553615 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7204019020 con 0x7f720c15acf0 2024-04-25T09:03:33.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.403+0000 7f7203400640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71ec06b4d0 0x7f71ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.403+0000 7f7203400640 1 -- 172.21.15.62:0/1688553615 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f720408a560 con 0x7f720c15acf0 2024-04-25T09:03:33.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.404+0000 7f720b400640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71ec06b4d0 0x7f71ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.405+0000 7f720b400640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71ec06b4d0 0x7f71ec06d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00a970 tx=0x7f71fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.407+0000 7f7203400640 1 -- 172.21.15.62:0/1688553615 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7204020020 con 0x7f720c15acf0 2024-04-25T09:03:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.609+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f720c04ef20 con 0x7f720c15acf0 2024-04-25T09:03:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.611+0000 7f7203400640 1 -- 172.21.15.62:0/1688553615 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f7204059340 con 0x7f720c15acf0 2024-04-25T09:03:33.599 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:33.600 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71ec06b4d0 msgr2=0x7f71ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71ec06b4d0 0x7f71ec06d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00a970 tx=0x7f71fc008040 comp rx=0 tx=0).stop 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c15acf0 msgr2=0x7f720c1f7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c15acf0 0x7f720c1f7010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f720400c700 tx=0x7f720400cbc0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 shutdown_connections 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71ec06b4d0 0x7f71ec06d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f720c15acf0 0x7f720c1f7010 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f720c13d970 0x7f720c1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 --2- 172.21.15.62:0/1688553615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f720c0b1050 0x7f720c159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.614+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 >> 172.21.15.62:0/1688553615 conn(0x7f720c0b14d0 msgr2=0x7f720c13e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.615+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 shutdown_connections 2024-04-25T09:03:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.615+0000 7f7210e00640 1 -- 172.21.15.62:0/1688553615 wait complete. 2024-04-25T09:03:33.624 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.843+0000 7f975e800640 1 Processor -- start 2024-04-25T09:03:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.843+0000 7f975e800640 1 -- start start 2024-04-25T09:03:33.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.844+0000 7f975e800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b56f0 0x7f9758160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.844+0000 7f975e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f97580b6a90 0x7f9758161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.844+0000 7f975e800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97580d3110 con 0x7f97580b27a0 2024-04-25T09:03:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.844+0000 7f975e800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9758161780 con 0x7f97580b6a90 2024-04-25T09:03:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.844+0000 7f975e800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580cedc0 con 0x7f97580b56f0 2024-04-25T09:03:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f975ca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f97580b6a90 0x7f9758161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f975ca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f97580b6a90 0x7f9758161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55280/0 (socket says 172.21.15.62:55280) 2024-04-25T09:03:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f975ca00640 1 -- 172.21.15.62:0/745229575 learned_addr learned my addr 172.21.15.62:0/745229575 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f975d400640 1 --2- 172.21.15.62:0/745229575 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b56f0 0x7f9758160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f975ca00640 1 -- 172.21.15.62:0/745229575 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b56f0 msgr2=0x7f9758160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f975ca00640 1 --2- 172.21.15.62:0/745229575 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b56f0 0x7f9758160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f975ca00640 1 -- 172.21.15.62:0/745229575 >> v1:172.21.15.62:6789/0 conn(0x7f97580b27a0 legacy=0x7f9758161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f975ca00640 1 -- 172.21.15.62:0/745229575 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580d33f0 con 0x7f97580b6a90 2024-04-25T09:03:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.845+0000 7f9757e00640 1 -- 172.21.15.62:0/745229575 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217389742 0 0) 0x7f97580d3110 con 0x7f97580b27a0 2024-04-25T09:03:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.846+0000 7f975ca00640 1 --2- 172.21.15.62:0/745229575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758161240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9744009870 tx=0x7f974402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53178c8e11bca6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.846+0000 7f9757e00640 1 -- 172.21.15.62:0/745229575 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744037070 con 0x7f97580b6a90 2024-04-25T09:03:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.846+0000 7f9757e00640 1 -- 172.21.15.62:0/745229575 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9744032440 con 0x7f97580b6a90 2024-04-25T09:03:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.846+0000 7f9757e00640 1 -- 172.21.15.62:0/745229575 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744032810 con 0x7f97580b6a90 2024-04-25T09:03:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.847+0000 7f975e800640 1 -- 172.21.15.62:0/745229575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 msgr2=0x7f9758161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.847+0000 7f975e800640 1 --2- 172.21.15.62:0/745229575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758161240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9744009870 tx=0x7f974402a250 comp rx=0 tx=0).stop 2024-04-25T09:03:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.847+0000 7f975e800640 1 -- 172.21.15.62:0/745229575 shutdown_connections 2024-04-25T09:03:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.847+0000 7f975e800640 1 --2- 172.21.15.62:0/745229575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758161240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.847+0000 7f975e800640 1 --2- 172.21.15.62:0/745229575 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b56f0 0x7f9758160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.847+0000 7f975e800640 1 -- 172.21.15.62:0/745229575 >> 172.21.15.62:0/745229575 conn(0x7f9758066d10 msgr2=0x7f9758067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.847+0000 7f975e800640 1 -- 172.21.15.62:0/745229575 shutdown_connections 2024-04-25T09:03:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.847+0000 7f975e800640 1 -- 172.21.15.62:0/745229575 wait complete. 2024-04-25T09:03:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.849+0000 7f975e800640 1 Processor -- start 2024-04-25T09:03:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.849+0000 7f975e800640 1 -- start start 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975e800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b27a0 0x7f97580cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97580b56f0 0x7f975815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580b2d60 con 0x7f97580b6a90 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975e800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580cea50 con 0x7f97580b56f0 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975e800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580b2ee0 con 0x7f97580b27a0 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55296/0 (socket says 172.21.15.62:55296) 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.850+0000 7f975de00640 1 -- 172.21.15.62:0/1178255930 learned_addr learned my addr 172.21.15.62:0/1178255930 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975d400640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b27a0 0x7f97580cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975ca00640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97580b56f0 0x7f975815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975de00640 1 -- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b27a0 msgr2=0x7f97580cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975de00640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b27a0 0x7f97580cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975de00640 1 -- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97580b56f0 msgr2=0x7f975815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975de00640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97580b56f0 0x7f975815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975de00640 1 -- 172.21.15.62:0/1178255930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975815ceb0 con 0x7f97580b6a90 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975de00640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758148c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f975000c760 tx=0x7f975000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f9755600640 1 -- 172.21.15.62:0/1178255930 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750003da0 con 0x7f97580b6a90 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f9755600640 1 -- 172.21.15.62:0/1178255930 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9750005070 con 0x7f97580b6a90 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f9755600640 1 -- 172.21.15.62:0/1178255930 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750005460 con 0x7f97580b6a90 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.851+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975815d190 con 0x7f97580b6a90 2024-04-25T09:03:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.852+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975815cbc0 con 0x7f97580b6a90 2024-04-25T09:03:33.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.853+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975804ef20 con 0x7f97580b6a90 2024-04-25T09:03:33.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.854+0000 7f9755600640 1 -- 172.21.15.62:0/1178255930 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9750005600 con 0x7f97580b6a90 2024-04-25T09:03:33.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.854+0000 7f9755600640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f973806b500 0x7f973806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.855+0000 7f9755600640 1 -- 172.21.15.62:0/1178255930 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f975008a190 con 0x7f97580b6a90 2024-04-25T09:03:33.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.855+0000 7f975d400640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f973806b500 0x7f973806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.856+0000 7f975d400640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f973806b500 0x7f973806d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f974c006fd0 tx=0x7f974c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:33.859+0000 7f9755600640 1 -- 172.21.15.62:0/1178255930 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9750058f70 con 0x7f97580b6a90 2024-04-25T09:03:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.098+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97580ce580 con 0x7f97580b6a90 2024-04-25T09:03:34.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.098+0000 7f9755600640 1 -- 172.21.15.62:0/1178255930 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f9750058870 con 0x7f97580b6a90 2024-04-25T09:03:34.086 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:34.087 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:16.609043+0000","modified":"2024-04-25T09:03:19.983991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:03:34.087 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f973806b500 msgr2=0x7f973806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f973806b500 0x7f973806d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f974c006fd0 tx=0x7f974c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 msgr2=0x7f9758148c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758148c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f975000c760 tx=0x7f975000cc20 comp rx=0 tx=0).stop 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 shutdown_connections 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f973806b500 0x7f973806d9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f97580b6a90 0x7f9758148c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97580b56f0 0x7f975815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 --2- 172.21.15.62:0/1178255930 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f97580b27a0 0x7f97580cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 >> 172.21.15.62:0/1178255930 conn(0x7f9758066d10 msgr2=0x7f97580b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 shutdown_connections 2024-04-25T09:03:34.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.101+0000 7f975e800640 1 -- 172.21.15.62:0/1178255930 wait complete. 2024-04-25T09:03:34.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:16.609043+0000', 'modified': '2024-04-25T09:03:19.983991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4257}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4257': {'gid': 4257, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6836/1669215157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 1669215157}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 1669215157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:03:34.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:03:34.510 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T09:03:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.744+0000 7f1c1be00640 1 Processor -- start 2024-04-25T09:03:34.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.744+0000 7f1c1be00640 1 -- start start 2024-04-25T09:03:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.745+0000 7f1c1be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c1c0b5c30 0x7f1c1c0b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.745+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c0d0fb0 0x7f1c1c161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.745+0000 7f1c1be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1c0cea70 con 0x7f1c1c161230 2024-04-25T09:03:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.745+0000 7f1c1be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1c0d2040 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.745+0000 7f1c1be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0ced50 con 0x7f1c1c0d0fb0 2024-04-25T09:03:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.746+0000 7f1c1a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c1c0b5c30 0x7f1c1c0b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.746+0000 7f1c1b400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c0d0fb0 0x7f1c1c161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.746+0000 7f1c1a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c1c0b5c30 0x7f1c1c0b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:52412/0 (socket says 172.21.15.62:52412) 2024-04-25T09:03:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.746+0000 7f1c1a000640 1 -- 172.21.15.62:0/2204744551 learned_addr learned my addr 172.21.15.62:0/2204744551 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.746+0000 7f1c1aa00640 1 --1- 172.21.15.62:0/2204744551 >> v1:172.21.15.121:6789/0 conn(0x7f1c1c161230 0x7f1c1c0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35914/0 (socket says 172.21.15.62:35914) 2024-04-25T09:03:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.747+0000 7f1c1a000640 1 -- 172.21.15.62:0/2204744551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c0d0fb0 msgr2=0x7f1c1c161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.747+0000 7f1c1a000640 1 --2- 172.21.15.62:0/2204744551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c0d0fb0 0x7f1c1c161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.747+0000 7f1c1a000640 1 -- 172.21.15.62:0/2204744551 >> v1:172.21.15.121:6789/0 conn(0x7f1c1c161230 legacy=0x7f1c1c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.747+0000 7f1c1a000640 1 -- 172.21.15.62:0/2204744551 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c0cf030 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.748+0000 7f1c1a000640 1 --2- 172.21.15.62:0/2204744551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c0b6c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009870 tx=0x7f1c0402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7e121373b3f450 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.749+0000 7f1c19600640 1 -- 172.21.15.62:0/2204744551 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c04037070 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.749+0000 7f1c19600640 1 -- 172.21.15.62:0/2204744551 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c04032440 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.749+0000 7f1c19600640 1 -- 172.21.15.62:0/2204744551 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c04032810 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.750+0000 7f1c1be00640 1 -- 172.21.15.62:0/2204744551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0b5c30 msgr2=0x7f1c1c0b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.750+0000 7f1c1be00640 1 --2- 172.21.15.62:0/2204744551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c0b6c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009870 tx=0x7f1c0402a250 comp rx=0 tx=0).stop 2024-04-25T09:03:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.750+0000 7f1c1be00640 1 -- 172.21.15.62:0/2204744551 shutdown_connections 2024-04-25T09:03:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.750+0000 7f1c1be00640 1 --2- 172.21.15.62:0/2204744551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c0d0fb0 0x7f1c1c161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.750+0000 7f1c1be00640 1 --2- 172.21.15.62:0/2204744551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c0b6c40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.750+0000 7f1c1be00640 1 -- 172.21.15.62:0/2204744551 >> 172.21.15.62:0/2204744551 conn(0x7f1c1c066d10 msgr2=0x7f1c1c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.750+0000 7f1c1be00640 1 -- 172.21.15.62:0/2204744551 shutdown_connections 2024-04-25T09:03:34.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.750+0000 7f1c1be00640 1 -- 172.21.15.62:0/2204744551 wait complete. 2024-04-25T09:03:34.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.752+0000 7f1c1be00640 1 Processor -- start 2024-04-25T09:03:34.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.752+0000 7f1c1be00640 1 -- start start 2024-04-25T09:03:34.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c14b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0d0fb0 0x7f1c1c0c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c161230 0x7f1c1c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c161610 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c0ca8d0 con 0x7f1c1c0d0fb0 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c161790 con 0x7f1c1c161230 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c14b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c14b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55300/0 (socket says 172.21.15.62:55300) 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.753+0000 7f1c1aa00640 1 -- 172.21.15.62:0/1083277350 learned_addr learned my addr 172.21.15.62:0/1083277350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1a000640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0d0fb0 0x7f1c1c0c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1b400640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c161230 0x7f1c1c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1aa00640 1 -- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c161230 msgr2=0x7f1c1c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c161230 0x7f1c1c0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1aa00640 1 -- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0d0fb0 msgr2=0x7f1c1c0c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0d0fb0 0x7f1c1c0c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1aa00640 1 -- 172.21.15.62:0/1083277350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c0c80f0 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c14b0d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00c800 tx=0x7f1c0c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c12a00640 1 -- 172.21.15.62:0/1083277350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c017070 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c12a00640 1 -- 172.21.15.62:0/1083277350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c00a640 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.754+0000 7f1c1be00640 1 -- 172.21.15.62:0/1083277350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c0c9040 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.755+0000 7f1c12a00640 1 -- 172.21.15.62:0/1083277350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00aa30 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.755+0000 7f1c1be00640 1 -- 172.21.15.62:0/1083277350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c0c9600 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.757+0000 7f1c12a00640 1 -- 172.21.15.62:0/1083277350 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c0c01a070 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.758+0000 7f1c12a00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bf806b4d0 0x7f1bf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.759+0000 7f1c12a00640 1 -- 172.21.15.62:0/1083277350 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c0c089770 con 0x7f1c1c0b5c30 2024-04-25T09:03:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.759+0000 7f1c1a000640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bf806b4d0 0x7f1bf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.759+0000 7f1c24405740 1 Processor -- start 2024-04-25T09:03:34.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.759+0000 7f1c24405740 1 -- start start 2024-04-25T09:03:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.760+0000 7f1c24405740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cc7b30 0x5616f7cef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.760+0000 7f1c24405740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cc7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.760+0000 7f1c1a000640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bf806b4d0 0x7f1bf806d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c04034000 tx=0x7f1c0402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.760+0000 7f1c24405740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7cc62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.760+0000 7f1c24405740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f7c2b620 con 0x5616f7cefeb0 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.760+0000 7f1c24405740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f7bd4ca0 con 0x5616f7cc7b30 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.760+0000 7f1c24405740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616f7c276c0 con 0x5616f7d107a0 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cc7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cc7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55306/0 (socket says 172.21.15.62:55306) 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1a000640 1 -- 172.21.15.62:0/4110497045 learned_addr learned my addr 172.21.15.62:0/4110497045 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1b400640 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7cc62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cc7b30 0x5616f7cef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1a000640 1 -- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 msgr2=0x5616f7cc62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1a000640 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7cc62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1a000640 1 -- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cc7b30 msgr2=0x5616f7cef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1a000640 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cc7b30 0x5616f7cef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1a000640 1 -- 172.21.15.62:0/4110497045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616f7bdbf40 con 0x5616f7cefeb0 2024-04-25T09:03:34.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.761+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cc7b30 0x5616f7cef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:34.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.762+0000 7f1c1a000640 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cc7f10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c04035040 tx=0x7f1c0402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a81178e5864cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.762+0000 7f1c0b000640 1 -- 172.21.15.62:0/4110497045 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0404e070 con 0x5616f7cefeb0 2024-04-25T09:03:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.762+0000 7f1c0b000640 1 -- 172.21.15.62:0/4110497045 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c04032d90 con 0x5616f7cefeb0 2024-04-25T09:03:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.762+0000 7f1c0b000640 1 -- 172.21.15.62:0/4110497045 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c04037040 con 0x5616f7cefeb0 2024-04-25T09:03:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.762+0000 7f1c24405740 1 -- 172.21.15.62:0/4110497045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 msgr2=0x5616f7cc7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.762+0000 7f1c24405740 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cc7f10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c04035040 tx=0x7f1c0402c000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.763+0000 7f1c24405740 1 -- 172.21.15.62:0/4110497045 shutdown_connections 2024-04-25T09:03:34.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.763+0000 7f1c24405740 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cc7f10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.763+0000 7f1c24405740 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7cc62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.763+0000 7f1c24405740 1 --2- 172.21.15.62:0/4110497045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cc7b30 0x5616f7cef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.763+0000 7f1c24405740 1 -- 172.21.15.62:0/4110497045 >> 172.21.15.62:0/4110497045 conn(0x5616f7cc4d50 msgr2=0x5616f7d0d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.763+0000 7f1c24405740 1 -- 172.21.15.62:0/4110497045 shutdown_connections 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.763+0000 7f1c24405740 1 -- 172.21.15.62:0/4110497045 wait complete. 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.763+0000 7f1c24405740 1 -- start start 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.764+0000 7f1c24405740 1 Processor -- start 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c24405740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cf1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c24405740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7d107a0 0x5616f7cf5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c24405740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7ce9220 0x5616f7cf63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c24405740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f7cef4e0 con 0x5616f7cefeb0 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c24405740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f7bca1f0 con 0x5616f7d107a0 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c24405740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616f7ceeed0 con 0x5616f7ce9220 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c24405740 10 client.0 _ll_register_callbacks cb 0x5616f7be7ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cf1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cf1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55316/0 (socket says 172.21.15.62:55316) 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1aa00640 1 -- 172.21.15.62:0/1626286043 learned_addr learned my addr 172.21.15.62:0/1626286043 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1a000640 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7d107a0 0x5616f7cf5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1aa00640 1 -- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7ce9220 msgr2=0x5616f7cf63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7ce9220 0x5616f7cf63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1aa00640 1 -- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7d107a0 msgr2=0x5616f7cf5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7d107a0 0x5616f7cf5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.765+0000 7f1c1aa00640 1 -- 172.21.15.62:0/1626286043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616f7be9c50 con 0x5616f7cefeb0 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cf1b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c001350 tx=0x7f1c0c058030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348caa58abd54048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1bff400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1bff400640 1 -- 172.21.15.62:0/1626286043 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c086060 con 0x5616f7cefeb0 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1bff400640 1 -- 172.21.15.62:0/1626286043 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c0c058490 con 0x5616f7cefeb0 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1c24405740 1 -- 172.21.15.62:0/1626286043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c04032d90 con 0x5616f7cefeb0 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1bff400640 1 -- 172.21.15.62:0/1626286043 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c0c074050 con 0x5616f7cefeb0 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1c24405740 10 client.4608 fetch_fsmap learned FSMap version 7 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1c24405740 1 -- 172.21.15.62:0/1626286043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5616f79402d0 con 0x5616f7cefeb0 2024-04-25T09:03:34.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1bff400640 1 -- 172.21.15.62:0/1626286043 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c076e50 con 0x5616f7cefeb0 2024-04-25T09:03:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1bff400640 1 -- 172.21.15.62:0/1626286043 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c0c060200 con 0x5616f7cefeb0 2024-04-25T09:03:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.766+0000 7f1c24405740 10 client.4608 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T09:03:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.767+0000 7f1c24405740 10 client.4608 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:03:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.767+0000 7f1c24405740 20 client.4608 populate_metadata read hostname 'smithi062' 2024-04-25T09:03:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.767+0000 7f1c24405740 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] conn(0x5616f7cf2cf0 0x5616f7d2ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.767+0000 7f1c24405740 4 client.4608 mds_command: new command op to 4257 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.767+0000 7f1c24405740 1 -- 172.21.15.62:0/1626286043 --> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616f7c38b60 con 0x5616f7cf2cf0 2024-04-25T09:03:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.767+0000 7f1c1a000640 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] conn(0x5616f7cf2cf0 0x5616f7d2ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.768+0000 7f1c1a000640 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] conn(0x5616f7cf2cf0 0x5616f7d2ae70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9827c48e5516244f server_cookie=96c32fcef5e52506 in_seq=0 out_seq=0 2024-04-25T09:03:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.768+0000 7f1bff400640 10 client.4608 ms_handle_connect on v2:172.21.15.121:6834/1669215157 2024-04-25T09:03:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1bff400640 1 -- 172.21.15.62:0/1626286043 <== mds.0 v2:172.21.15.121:6834/1669215157 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616f7c38b60 con 0x5616f7cf2cf0 2024-04-25T09:03:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1bff400640 10 client.4608 handle_command_reply: tid=0 2024-04-25T09:03:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 client.4608 shutdown 2024-04-25T09:03:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 -- 172.21.15.62:0/1626286043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 msgr2=0x5616f7cf1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cf1b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c001350 tx=0x7f1c0c058030 comp rx=0 tx=0).stop 2024-04-25T09:03:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 -- 172.21.15.62:0/1626286043 shutdown_connections 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] conn(0x5616f7cf2cf0 0x5616f7d2ae70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7ce9220 0x5616f7cf63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7d107a0 0x5616f7cf5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 --2- 172.21.15.62:0/1626286043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7cefeb0 0x5616f7cf1b50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1bff400640 0 client.4608 ms_handle_reset on v2:172.21.15.121:6834/1669215157 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 -- 172.21.15.62:0/1626286043 >> 172.21.15.62:0/1626286043 conn(0x5616f7cc7b30 msgr2=0x5616f7cc18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 -- 172.21.15.62:0/1626286043 shutdown_connections 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 1 -- 172.21.15.62:0/1626286043 wait complete. 2024-04-25T09:03:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.771+0000 7f1c24405740 20 client.4608 trim_cache size 0 max 16384 2024-04-25T09:03:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.789+0000 7f1c24405740 1 Processor -- start 2024-04-25T09:03:34.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.789+0000 7f1c24405740 1 -- start start 2024-04-25T09:03:34.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.789+0000 7f1c24405740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7ce9220 0x5616f7cc64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.789+0000 7f1c24405740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cefeb0 0x5616f7d1bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.789+0000 7f1c24405740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7d1c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.789+0000 7f1c24405740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c1565a0 con 0x5616f7ce9220 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.789+0000 7f1c24405740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c1f4a20 con 0x5616f7cefeb0 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.789+0000 7f1c24405740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c155fd0 con 0x5616f7d107a0 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7ce9220 0x5616f7cc64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7ce9220 0x5616f7cc64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36372/0 (socket says 172.21.15.62:36372) 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1aa00640 1 -- 172.21.15.62:0/2080200263 learned_addr learned my addr 172.21.15.62:0/2080200263 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1b400640 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7d1c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1aa00640 1 -- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 msgr2=0x5616f7d1c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1a000640 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cefeb0 0x5616f7d1bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7d1c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1aa00640 1 -- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cefeb0 msgr2=0x5616f7d1bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cefeb0 0x5616f7d1bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1aa00640 1 -- 172.21.15.62:0/2080200263 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616f7a0a6b0 con 0x5616f7ce9220 2024-04-25T09:03:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.790+0000 7f1c1b400640 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7d1c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7ce9220 0x5616f7cc64e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c057d30 tx=0x7f1c0c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4a34834652e798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c0a600640 1 -- 172.21.15.62:0/2080200263 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c07a030 con 0x5616f7ce9220 2024-04-25T09:03:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c0a600640 1 -- 172.21.15.62:0/2080200263 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c076e50 con 0x5616f7ce9220 2024-04-25T09:03:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c0a600640 1 -- 172.21.15.62:0/2080200263 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c086050 con 0x5616f7ce9220 2024-04-25T09:03:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c24405740 1 -- 172.21.15.62:0/2080200263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7ce9220 msgr2=0x5616f7cc64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c24405740 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7ce9220 0x5616f7cc64e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c057d30 tx=0x7f1c0c014000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c24405740 1 -- 172.21.15.62:0/2080200263 shutdown_connections 2024-04-25T09:03:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c24405740 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7d107a0 0x5616f7d1c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c24405740 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7cefeb0 0x5616f7d1bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c24405740 1 --2- 172.21.15.62:0/2080200263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7ce9220 0x5616f7cc64e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.791+0000 7f1c24405740 1 -- 172.21.15.62:0/2080200263 >> 172.21.15.62:0/2080200263 conn(0x5616f7cc7b30 msgr2=0x5616f7ce8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.792+0000 7f1c24405740 1 -- 172.21.15.62:0/2080200263 shutdown_connections 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.792+0000 7f1c24405740 1 -- 172.21.15.62:0/2080200263 wait complete. 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.792+0000 7f1c24405740 1 -- start start 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.792+0000 7f1c24405740 1 Processor -- start 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.793+0000 7f1c24405740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7ce9220 0x5616f7d0f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.793+0000 7f1c24405740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7cefeb0 0x5616f7d61430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.793+0000 7f1c24405740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7d107a0 0x5616f7d2f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.793+0000 7f1c24405740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f7cf2b70 con 0x5616f7d107a0 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.793+0000 7f1c24405740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f7ced280 con 0x5616f7ce9220 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.793+0000 7f1c24405740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616f7cf3100 con 0x5616f7cefeb0 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.793+0000 7f1c24405740 10 client.0 _ll_register_callbacks cb 0x7f1c0c054370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7d107a0 0x5616f7d2f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7d107a0 0x5616f7d2f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36374/0 (socket says 172.21.15.62:36374) 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 -- 172.21.15.62:0/855245322 learned_addr learned my addr 172.21.15.62:0/855245322 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7ce9220 0x5616f7d0f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1a000640 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7cefeb0 0x5616f7d61430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 -- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7cefeb0 msgr2=0x5616f7d61430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7cefeb0 0x5616f7d61430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 -- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7ce9220 msgr2=0x5616f7d0f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7ce9220 0x5616f7d0f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 -- 172.21.15.62:0/855245322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616f7cf59a0 con 0x5616f7d107a0 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1a000640 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7cefeb0 0x5616f7d61430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1c1b400640 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7d107a0 0x5616f7d2f270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5616f7ce9680 tx=0x7f1c140146c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48fd87e2d60cb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.794+0000 7f1bff400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:03:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.795+0000 7f1bff400640 1 -- 172.21.15.62:0/855245322 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1401c070 con 0x5616f7d107a0 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.795+0000 7f1bff400640 1 -- 172.21.15.62:0/855245322 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c14017670 con 0x5616f7d107a0 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.795+0000 7f1c24405740 1 -- 172.21.15.62:0/855245322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c0c076e50 con 0x5616f7d107a0 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.795+0000 7f1bff400640 1 -- 172.21.15.62:0/855245322 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c14027050 con 0x5616f7d107a0 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.795+0000 7f1c24405740 10 client.4614 fetch_fsmap learned FSMap version 7 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.795+0000 7f1c24405740 1 -- 172.21.15.62:0/855245322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5616f7d1c090 con 0x5616f7d107a0 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.795+0000 7f1bff400640 1 -- 172.21.15.62:0/855245322 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c076e50 con 0x5616f7d107a0 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.795+0000 7f1bff400640 1 -- 172.21.15.62:0/855245322 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c1402a020 con 0x5616f7d107a0 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.796+0000 7f1c24405740 10 client.4614 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.796+0000 7f1c24405740 10 client.4614 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.796+0000 7f1c24405740 20 client.4614 populate_metadata read hostname 'smithi062' 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.796+0000 7f1c24405740 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] conn(0x5616f7d11630 0x5616f7cc18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.796+0000 7f1c24405740 4 client.4614 mds_command: new command op to 4257 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.796+0000 7f1c24405740 1 -- 172.21.15.62:0/855245322 --> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5616f7cf3e40 con 0x5616f7d11630 2024-04-25T09:03:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.796+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] conn(0x5616f7d11630 0x5616f7cc18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.797+0000 7f1c1aa00640 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] conn(0x5616f7d11630 0x5616f7cc18b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d202692670848089 server_cookie=80f931ed4c970558 in_seq=0 out_seq=0 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.797+0000 7f1bff400640 10 client.4614 ms_handle_connect on v2:172.21.15.121:6834/1669215157 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1bff400640 1 -- 172.21.15.62:0/855245322 <== mds.0 v2:172.21.15.121:6834/1669215157 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5616f7cf3e40 con 0x5616f7d11630 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1bff400640 10 client.4614 handle_command_reply: tid=0 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 client.4614 shutdown 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 -- 172.21.15.62:0/855245322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7d107a0 msgr2=0x5616f7d2f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7d107a0 0x5616f7d2f270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5616f7ce9680 tx=0x7f1c140146c0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 -- 172.21.15.62:0/855245322 shutdown_connections 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:6834/1669215157,v1:172.21.15.121:6836/1669215157] conn(0x5616f7d11630 0x5616f7cc18b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5616f7d107a0 0x5616f7d2f270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5616f7cefeb0 0x5616f7d61430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 --2- 172.21.15.62:0/855245322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616f7ce9220 0x5616f7d0f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1bff400640 0 client.4614 ms_handle_reset on v2:172.21.15.121:6834/1669215157 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 -- 172.21.15.62:0/855245322 >> 172.21.15.62:0/855245322 conn(0x5616f7cc7b30 msgr2=0x5616f7ce8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 -- 172.21.15.62:0/855245322 shutdown_connections 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 1 -- 172.21.15.62:0/855245322 wait complete. 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.798+0000 7f1c24405740 20 client.4614 trim_cache size 0 max 16384 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:03:34.790 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4257, 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 17.647431301000001, 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 7, 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 20, 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 22.779368433999998 2024-04-25T09:03:34.791 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:03:34.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 -- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bf806b4d0 msgr2=0x7f1bf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bf806b4d0 0x7f1bf806d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c04034000 tx=0x7f1c0402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 -- 172.21.15.62:0/1083277350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c1c0b5c30 msgr2=0x7f1c1c14b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c14b0d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00c800 tx=0x7f1c0c00ccc0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 -- 172.21.15.62:0/1083277350 shutdown_connections 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bf806b4d0 0x7f1bf806d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c1c161230 0x7f1c1c0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1c0d0fb0 0x7f1c1c0c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 --2- 172.21.15.62:0/1083277350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c1c0b5c30 0x7f1c1c14b0d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 -- 172.21.15.62:0/1083277350 >> 172.21.15.62:0/1083277350 conn(0x7f1c1c066d10 msgr2=0x7f1c1c0b8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 -- 172.21.15.62:0/1083277350 shutdown_connections 2024-04-25T09:03:34.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:34.799+0000 7f1c1be00640 1 -- 172.21.15.62:0/1083277350 wait complete. 2024-04-25T09:03:34.805 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T09:03:34.833 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T09:03:34.833 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T09:03:34.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:34.833 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:03:34.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:34.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-25T09:03:34.980 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-04-25T09:03:34.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:34.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:34.984 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:03:35.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:35.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-25T09:03:35.126 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-04-25T09:03:35.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:35.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:35.130 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-25T09:03:35.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:35.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2024-04-25T09:03:35.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:35.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:35.291 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-25T09:03:35.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:35.343 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-25T09:03:35.421 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2024-04-25T09:03:35.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:35.425 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T09:03:35.425 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:03:35.425 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:35.425 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:03:35.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.657+0000 7fddd6400640 1 Processor -- start 2024-04-25T09:03:35.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.657+0000 7fddd6400640 1 -- start start 2024-04-25T09:03:35.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.658+0000 7fddd6400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fddd0140b90 0x7fddd00b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.658+0000 7fddd6400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd015ef30 con 0x7fddd015e9f0 2024-04-25T09:03:35.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.659+0000 7fddd6400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd015f210 con 0x7fddd00b30f0 2024-04-25T09:03:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.659+0000 7fddd6400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd00af080 con 0x7fddd0140b90 2024-04-25T09:03:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.659+0000 7fddcfe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fddd015e9f0 0x7fddd0141eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34722/0 (socket says 172.21.15.62:34722) 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.659+0000 7fddcfe00640 1 -- 172.21.15.62:0/3989360220 learned_addr learned my addr 172.21.15.62:0/3989360220 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.659+0000 7fddcf400640 1 --2- 172.21.15.62:0/3989360220 >> v2:172.21.15.121:3300/0 conn(0x7fddd0140b90 0x7fddd00b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.659+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716213757 0 0) 0x7fddd015ef30 con 0x7fddd015e9f0 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.659+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb4003520 con 0x7fddd015e9f0 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1824441866 0 0) 0x7fddb4003520 con 0x7fddd015e9f0 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 >> v2:172.21.15.121:3300/0 conn(0x7fddd0140b90 msgr2=0x7fddd00b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 --2- 172.21.15.62:0/3989360220 >> v2:172.21.15.121:3300/0 conn(0x7fddd0140b90 0x7fddd00b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 >> v1:172.21.15.121:6789/0 conn(0x7fddd00b30f0 legacy=0x7fddd015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd01413d0 con 0x7fddd015e9f0 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fddc0001670 con 0x7fddd015e9f0 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 >> v1:172.21.15.62:6789/0 conn(0x7fddd015e9f0 legacy=0x7fddd0141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb40039b0 0x7fddb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb4004800 0x7fddb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb400c030 con 0x7fddb40051c0 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb400c1e0 con 0x7fddb40039b0 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb400c3e0 con 0x7fddb4004800 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddcfe00640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddcf400640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb40039b0 0x7fddb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.660+0000 7fddcea00640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb4004800 0x7fddb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.661+0000 7fddcfe00640 1 -- 172.21.15.62:0/3989360220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb4004800 msgr2=0x7fddb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.661+0000 7fddcfe00640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddb4004800 0x7fddb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.661+0000 7fddcfe00640 1 -- 172.21.15.62:0/3989360220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb40039b0 msgr2=0x7fddb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.661+0000 7fddcfe00640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddb40039b0 0x7fddb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.661+0000 7fddcfe00640 1 -- 172.21.15.62:0/3989360220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb4003520 con 0x7fddb40051c0 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.662+0000 7fddcfe00640 1 -- 172.21.15.62:0/3989360220 reap_dead start 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.662+0000 7fddcfe00640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddb400b9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fddc000c240 tx=0x7fddb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b77eca80ee0914 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.662+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0007cf0 con 0x7fddb40051c0 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.662+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc00058d0 con 0x7fddb40051c0 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.662+0000 7fddce000640 1 -- 172.21.15.62:0/3989360220 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc0005ce0 con 0x7fddb40051c0 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.663+0000 7fddd6400640 1 -- 172.21.15.62:0/3989360220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 msgr2=0x7fddb400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.663+0000 7fddd6400640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddb400b9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fddc000c240 tx=0x7fddb4005830 comp rx=0 tx=0).stop 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.663+0000 7fddd6400640 1 -- 172.21.15.62:0/3989360220 shutdown_connections 2024-04-25T09:03:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.663+0000 7fddd6400640 1 --2- 172.21.15.62:0/3989360220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddb400b9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.663+0000 7fddd6400640 1 -- 172.21.15.62:0/3989360220 >> 172.21.15.62:0/3989360220 conn(0x7fddd00668f0 msgr2=0x7fddd0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.663+0000 7fddd6400640 1 -- 172.21.15.62:0/3989360220 shutdown_connections 2024-04-25T09:03:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.663+0000 7fddd6400640 1 -- 172.21.15.62:0/3989360220 wait complete. 2024-04-25T09:03:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.665+0000 7fddd6400640 1 Processor -- start 2024-04-25T09:03:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.665+0000 7fddd6400640 1 -- start start 2024-04-25T09:03:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddd6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddd015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddd6400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddd013d3a0 0x7fddd015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddd6400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd015d570 0x7fddd00b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddd6400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0140f70 con 0x7fddb40051c0 2024-04-25T09:03:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddd6400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0141990 con 0x7fddd015d570 2024-04-25T09:03:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddd6400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00b2bb0 con 0x7fddd013d3a0 2024-04-25T09:03:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddcf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddd015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddcf400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddd015b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36400/0 (socket says 172.21.15.62:36400) 2024-04-25T09:03:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.666+0000 7fddcf400640 1 -- 172.21.15.62:0/157818776 learned_addr learned my addr 172.21.15.62:0/157818776 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcea00640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddd013d3a0 0x7fddd015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcfe00640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd015d570 0x7fddd00b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcf400640 1 -- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddd013d3a0 msgr2=0x7fddd015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcf400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddd013d3a0 0x7fddd015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcf400640 1 -- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd015d570 msgr2=0x7fddd00b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcf400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd015d570 0x7fddd00b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcf400640 1 -- 172.21.15.62:0/157818776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd0130360 con 0x7fddb40051c0 2024-04-25T09:03:35.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcfe00640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd015d570 0x7fddd00b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddcf400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddd015b6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fddc8011e30 tx=0x7fddc800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddc7400640 1 -- 172.21.15.62:0/157818776 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8015630 con 0x7fddb40051c0 2024-04-25T09:03:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddc7400640 1 -- 172.21.15.62:0/157818776 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc80157d0 con 0x7fddb40051c0 2024-04-25T09:03:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd0130640 con 0x7fddb40051c0 2024-04-25T09:03:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.667+0000 7fddc7400640 1 -- 172.21.15.62:0/157818776 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8015bc0 con 0x7fddb40051c0 2024-04-25T09:03:35.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.668+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd0130c00 con 0x7fddb40051c0 2024-04-25T09:03:35.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.669+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd0073190 con 0x7fddb40051c0 2024-04-25T09:03:35.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.670+0000 7fddc7400640 1 -- 172.21.15.62:0/157818776 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddc801d400 con 0x7fddb40051c0 2024-04-25T09:03:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.671+0000 7fddc7400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fddac06b500 0x7fddac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.671+0000 7fddc7400640 1 -- 172.21.15.62:0/157818776 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddc808b3e0 con 0x7fddb40051c0 2024-04-25T09:03:35.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.671+0000 7fddcea00640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fddac06b500 0x7fddac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.672+0000 7fddcea00640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fddac06b500 0x7fddac06d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fddd00b1430 tx=0x7fddb8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.675+0000 7fddc7400640 1 -- 172.21.15.62:0/157818776 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddc8019030 con 0x7fddb40051c0 2024-04-25T09:03:35.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.921+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fddd004f000 con 0x7fddb40051c0 2024-04-25T09:03:35.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.922+0000 7fddc7400640 1 -- 172.21.15.62:0/157818776 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fddc8010040 con 0x7fddb40051c0 2024-04-25T09:03:35.909 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.924+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fddac06b500 msgr2=0x7fddac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.924+0000 7fddd6400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fddac06b500 0x7fddac06d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fddd00b1430 tx=0x7fddb8009040 comp rx=0 tx=0).stop 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.924+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 msgr2=0x7fddd015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.924+0000 7fddd6400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddd015b6e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fddc8011e30 tx=0x7fddc800f580 comp rx=0 tx=0).stop 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.925+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 shutdown_connections 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.925+0000 7fddd6400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fddac06b500 0x7fddac06d9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.925+0000 7fddd6400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddd015d570 0x7fddd00b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.925+0000 7fddd6400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddd013d3a0 0x7fddd015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.925+0000 7fddd6400640 1 --2- 172.21.15.62:0/157818776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddb40051c0 0x7fddd015b6e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.925+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 >> 172.21.15.62:0/157818776 conn(0x7fddd00668f0 msgr2=0x7fddd015fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.925+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 shutdown_connections 2024-04-25T09:03:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:35.925+0000 7fddd6400640 1 -- 172.21.15.62:0/157818776 wait complete. 2024-04-25T09:03:35.933 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:03:35.934 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:03:35.934 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:03:35.934 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:03:35.934 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:35.934 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:03:35.934 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:03:35.934 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:03:35.934 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:35.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:35.934 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:03:35.988 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:35.988 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:35.988 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:35.988 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:35.988 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:35.988 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:35.988 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42643sec preferred_lft 42643sec 2024-04-25T09:03:35.989 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:03:35.990 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:35.991 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:03:35.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:35.991 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:35.991 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:35.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:03:35.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-04-25T09:03:35.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-04-25T09:03:35.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:03:35.991 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:36.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:36.423 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:03:36.499 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:03:36.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:36.502 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:03:36.570 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:03:36.570 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:03:36.570 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:03:36.570 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:03:36.570 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:03:36.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:36.571 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:36.571 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:36.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:03:36.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:03:36.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:03:36.572 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:37.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:37.315 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:03:37.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:37.373 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-04-25T09:03:37.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:37.428 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:37.428 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:37.428 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:37.429 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T09:03:37.429 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:37.560 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T09:03:37.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:37.560 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:37.560 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:37.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T09:03:37.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:03:37.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T09:03:37.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T09:03:37.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T09:03:37.561 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:37.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:37.814 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:37.814 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:37.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:03:37.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:03:37.814 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:37.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:37.961 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:38.013 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:38.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:38.013 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:38.062 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:03:38.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:38.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:38.200 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:38.200 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:03:38.331 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:38.339 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:38.966 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:38.966 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:03:38.967 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:03:38.967 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:03:38.967 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:03:38.968 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:03:38.968 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:03:38.969 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:03:38.969 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:03:38.969 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:03:38.972 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:03:38.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:38.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:39.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.038 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:39.099 INFO:teuthology.orchestra.run.smithi062.stdout:client4623 2024-04-25T09:03:39.099 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:03:39.099 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:39.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.146 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:39.259 INFO:teuthology.orchestra.run.smithi062.stdout:client4623 2024-04-25T09:03:39.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.259 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:39.328 INFO:teuthology.orchestra.run.smithi062.stdout:client4623 2024-04-25T09:03:39.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.329 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9e112875-3fc2-4963-90e0-2b56d5516d5c.client4623/status') 2024-04-25T09:03:39.401 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.4623 (3)192.168.0.1:0/1423807735 2024-04-25T09:03:39.401 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-04-25T09:03:39.402 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-04-25T09:03:39.402 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-04-25T09:03:39.402 INFO:teuthology.orchestra.run.smithi062.stderr:69 bytes copied, 8.2571e-05 s, 836 kB/s 2024-04-25T09:03:39.405 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T09:03:39.405 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:03:39.405 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T09:03:39.405 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:03:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.606+0000 7f62ef400640 1 Processor -- start 2024-04-25T09:03:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.606+0000 7f62ef400640 1 -- start start 2024-04-25T09:03:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.608+0000 7f62ef400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62e80cfed0 0x7f62e815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.608+0000 7f62ef400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e806c310 con 0x7f62e8072340 2024-04-25T09:03:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.608+0000 7f62ef400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e815f5f0 con 0x7f62e80709c0 2024-04-25T09:03:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.608+0000 7f62ef400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e806c5b0 con 0x7f62e80cfed0 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.608+0000 7f62dfe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f62e8072340 0x7f62e80702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34738/0 (socket says 172.21.15.62:34738) 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.608+0000 7f62dfe00640 1 -- 172.21.15.62:0/945886069 learned_addr learned my addr 172.21.15.62:0/945886069 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.608+0000 7f62eca00640 1 --2- 172.21.15.62:0/945886069 >> v2:172.21.15.121:3300/0 conn(0x7f62e80cfed0 0x7f62e815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.609+0000 7f62e7e00640 1 -- 172.21.15.62:0/945886069 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022561859 0 0) 0x7f62e806c310 con 0x7f62e8072340 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.609+0000 7f62e7e00640 1 -- 172.21.15.62:0/945886069 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62cc005730 con 0x7f62e8072340 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.609+0000 7f62eca00640 1 -- 172.21.15.62:0/945886069 >> v1:172.21.15.121:6789/0 conn(0x7f62e80709c0 legacy=0x7f62e80b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.609+0000 7f62eca00640 1 -- 172.21.15.62:0/945886069 >> v1:172.21.15.62:6789/0 conn(0x7f62e8072340 legacy=0x7f62e80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.609+0000 7f62e7e00640 1 -- 172.21.15.62:0/945886069 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4089880959 0 0) 0x7f62cc005730 con 0x7f62e8072340 2024-04-25T09:03:39.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.609+0000 7f62eca00640 1 -- 172.21.15.62:0/945886069 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e806c890 con 0x7f62e80cfed0 2024-04-25T09:03:39.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.610+0000 7f62eca00640 1 --2- 172.21.15.62:0/945886069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e80cfed0 0x7f62e815f0b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009d90 tx=0x7f62d802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df0798295b23dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:39.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.611+0000 7f62e7e00640 1 -- 172.21.15.62:0/945886069 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0001670 con 0x7f62e80cfed0 2024-04-25T09:03:39.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.611+0000 7f62e7e00640 1 -- 172.21.15.62:0/945886069 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d8007540 con 0x7f62e80cfed0 2024-04-25T09:03:39.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.611+0000 7f62e7e00640 1 -- 172.21.15.62:0/945886069 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8007910 con 0x7f62e80cfed0 2024-04-25T09:03:39.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.611+0000 7f62ef400640 1 -- 172.21.15.62:0/945886069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e80cfed0 msgr2=0x7f62e815f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.611+0000 7f62ef400640 1 --2- 172.21.15.62:0/945886069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e80cfed0 0x7f62e815f0b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009d90 tx=0x7f62d802a680 comp rx=0 tx=0).stop 2024-04-25T09:03:39.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.612+0000 7f62ef400640 1 -- 172.21.15.62:0/945886069 shutdown_connections 2024-04-25T09:03:39.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.612+0000 7f62ef400640 1 --2- 172.21.15.62:0/945886069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e80cfed0 0x7f62e815f0b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.612+0000 7f62ef400640 1 -- 172.21.15.62:0/945886069 >> 172.21.15.62:0/945886069 conn(0x7f62e80668f0 msgr2=0x7f62e8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:39.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.612+0000 7f62ef400640 1 -- 172.21.15.62:0/945886069 shutdown_connections 2024-04-25T09:03:39.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.612+0000 7f62ef400640 1 -- 172.21.15.62:0/945886069 wait complete. 2024-04-25T09:03:39.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.614+0000 7f62ef400640 1 Processor -- start 2024-04-25T09:03:39.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.614+0000 7f62ef400640 1 -- start start 2024-04-25T09:03:39.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.615+0000 7f62ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e80709c0 0x7f62e815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.615+0000 7f62ef400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e80cfed0 0x7f62e81f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.615+0000 7f62ef400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e81f8720 0x7f62e81f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.615+0000 7f62ef400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e806d130 con 0x7f62e80709c0 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.615+0000 7f62ef400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e806ce50 con 0x7f62e81f8720 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.615+0000 7f62ef400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80b2f90 con 0x7f62e80cfed0 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.615+0000 7f62eca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e80709c0 0x7f62e815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62eca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e80709c0 0x7f62e815a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36412/0 (socket says 172.21.15.62:36412) 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62eca00640 1 -- 172.21.15.62:0/2514559709 learned_addr learned my addr 172.21.15.62:0/2514559709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62ed400640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e81f8720 0x7f62e81f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62dfe00640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e80cfed0 0x7f62e81f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62eca00640 1 -- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e80cfed0 msgr2=0x7f62e81f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62eca00640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e80cfed0 0x7f62e81f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62eca00640 1 -- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e81f8720 msgr2=0x7f62e81f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62eca00640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e81f8720 0x7f62e81f80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.616+0000 7f62eca00640 1 -- 172.21.15.62:0/2514559709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e815cc10 con 0x7f62e80709c0 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.617+0000 7f62eca00640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e80709c0 0x7f62e815a000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009d90 tx=0x7f62d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.617+0000 7f62e5600640 1 -- 172.21.15.62:0/2514559709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8040070 con 0x7f62e80709c0 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.617+0000 7f62e5600640 1 -- 172.21.15.62:0/2514559709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d8031bc0 con 0x7f62e80709c0 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.617+0000 7f62e5600640 1 -- 172.21.15.62:0/2514559709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8009450 con 0x7f62e80709c0 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.617+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e815cef0 con 0x7f62e80709c0 2024-04-25T09:03:39.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.617+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e815d4b0 con 0x7f62e80709c0 2024-04-25T09:03:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.618+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e8072c70 con 0x7f62e80709c0 2024-04-25T09:03:39.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.619+0000 7f62e5600640 1 -- 172.21.15.62:0/2514559709 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62d8039040 con 0x7f62e80709c0 2024-04-25T09:03:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.620+0000 7f62e5600640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62c406b4d0 0x7f62c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.620+0000 7f62e5600640 1 -- 172.21.15.62:0/2514559709 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62d80ab9b0 con 0x7f62e80709c0 2024-04-25T09:03:39.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.621+0000 7f62dfe00640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62c406b4d0 0x7f62c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.622+0000 7f62dfe00640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62c406b4d0 0x7f62c406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f62e81f7ae0 tx=0x7f62d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:39.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.625+0000 7f62e5600640 1 -- 172.21.15.62:0/2514559709 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62d807a790 con 0x7f62e80709c0 2024-04-25T09:03:39.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.866+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62e815ab70 con 0x7f62e80709c0 2024-04-25T09:03:39.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.867+0000 7f62e5600640 1 -- 172.21.15.62:0/2514559709 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62d807a090 con 0x7f62e80709c0 2024-04-25T09:03:39.854 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.869+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62c406b4d0 msgr2=0x7f62c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.869+0000 7f62ef400640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62c406b4d0 0x7f62c406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f62e81f7ae0 tx=0x7f62d0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e80709c0 msgr2=0x7f62e815a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e80709c0 0x7f62e815a000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009d90 tx=0x7f62d8004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 shutdown_connections 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62c406b4d0 0x7f62c406d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62e81f8720 0x7f62e81f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62e80cfed0 0x7f62e81f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 --2- 172.21.15.62:0/2514559709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e80709c0 0x7f62e815a000 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 >> 172.21.15.62:0/2514559709 conn(0x7f62e80668f0 msgr2=0x7f62e80cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 shutdown_connections 2024-04-25T09:03:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:39.870+0000 7f62ef400640 1 -- 172.21.15.62:0/2514559709 wait complete. 2024-04-25T09:03:39.874 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:03:39.875 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:03:39.875 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:03:39.875 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T09:03:39.875 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T09:03:39.875 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:03:39.875 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:03:39.875 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:03:39.875 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T09:03:39.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.875 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:03:39.924 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:39.924 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:39.924 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:39.924 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:03:39.925 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42639sec preferred_lft 42639sec 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.926 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:39.927 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:39.927 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:03:39.927 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.927 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:03:40.040 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:03:40.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.044 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:03:40.096 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:03:40.096 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:03:40.096 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:03:40.097 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:03:40.097 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:03:40.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.097 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:40.097 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:40.097 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:03:40.097 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:03:40.097 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:03:40.097 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:40.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.256 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:03:40.308 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:03:40.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.310 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-04-25T09:03:40.360 INFO:teuthology.orchestra.run.smithi062.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T09:03:40.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.361 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:40.361 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:40.361 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:40.361 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T09:03:40.361 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:40.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T09:03:40.570 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:40.570 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:40.570 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:40.570 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:40.570 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:40.570 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:40.570 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:40.570 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0a:62:10:f8:9d:bb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:03:40.571 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:03:40.571 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:40.571 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::862:10ff:fef8:9dbb/64 scope link 2024-04-25T09:03:40.571 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:40.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0a:62:10:f8:9d:bb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:40.661 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::862:10ff:fef8:9dbb/64 scope link 2024-04-25T09:03:40.662 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:40.665 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T09:03:40.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.665 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:40.665 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:40.665 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T09:03:40.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T09:03:40.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T09:03:40.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T09:03:40.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T09:03:40.666 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:40.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:40.899 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:03:40.899 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:03:40.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.1 up 2024-04-25T09:03:40.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T09:03:40.899 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:03:41.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:41.049 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T09:03:41.094 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T09:03:41.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:41.094 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T09:03:41.192 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T09:03:41.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.193 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:41.316 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:41.316 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:03:41.353 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:41.353 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:03:41.353 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:03:41.353 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 2@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:03:41.353 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:03:41.357 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T09:03:41.357 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:03:41.358 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:03:41.358 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:03:41.358 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:03:41.362 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:03:41.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T09:03:41.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.446 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:41.502 INFO:teuthology.orchestra.run.smithi062.stdout:client4448 2024-04-25T09:03:41.503 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:03:41.503 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:41.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.557 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:41.619 INFO:teuthology.orchestra.run.smithi062.stdout:client4448 2024-04-25T09:03:41.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.620 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:41.681 INFO:teuthology.orchestra.run.smithi062.stdout:client4448 2024-04-25T09:03:41.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.682 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9e112875-3fc2-4963-90e0-2b56d5516d5c.client4448/status') 2024-04-25T09:03:41.769 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.4448 (3)172.21.15.62:0/215227857 2024-04-25T09:03:41.769 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-04-25T09:03:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-04-25T09:03:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-04-25T09:03:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:69 bytes copied, 9.5226e-05 s, 725 kB/s 2024-04-25T09:03:41.773 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T09:03:41.773 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:03:41.773 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T09:03:41.773 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:03:41.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.995+0000 7f0fce800640 1 Processor -- start 2024-04-25T09:03:41.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.995+0000 7f0fce800640 1 -- start start 2024-04-25T09:03:41.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.997+0000 7f0fce800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0fc8144c10 0x7f0fc815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:41.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.997+0000 7f0fce800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0fc806d900 0x7f0fc80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:41.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.997+0000 7f0fce800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc80cf2d0 con 0x7f0fc80cee40 2024-04-25T09:03:41.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.997+0000 7f0fce800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc8144ff0 con 0x7f0fc8144c10 2024-04-25T09:03:41.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.997+0000 7f0fce800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc8160340 con 0x7f0fc806d900 2024-04-25T09:03:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.997+0000 7f0fcd400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0fc8144c10 0x7f0fc815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.997+0000 7f0fcd400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0fc8144c10 0x7f0fc815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36414/0 (socket says 172.21.15.62:36414) 2024-04-25T09:03:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.997+0000 7f0fcd400640 1 -- 172.21.15.62:0/330436021 learned_addr learned my addr 172.21.15.62:0/330436021 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.998+0000 7f0fcde00640 1 --2- 172.21.15.62:0/330436021 >> v2:172.21.15.121:3300/0 conn(0x7f0fc806d900 0x7f0fc80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.998+0000 7f0fcd400640 1 -- 172.21.15.62:0/330436021 >> v2:172.21.15.121:3300/0 conn(0x7f0fc806d900 msgr2=0x7f0fc80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.998+0000 7f0fcd400640 1 --2- 172.21.15.62:0/330436021 >> v2:172.21.15.121:3300/0 conn(0x7f0fc806d900 0x7f0fc80b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.998+0000 7f0fcd400640 1 -- 172.21.15.62:0/330436021 >> v1:172.21.15.121:6789/0 conn(0x7f0fc80cee40 legacy=0x7f0fc806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:41.998+0000 7f0fcd400640 1 -- 172.21.15.62:0/330436021 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc815faa0 con 0x7f0fc8144c10 2024-04-25T09:03:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.000+0000 7f0fcd400640 1 --2- 172.21.15.62:0/330436021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc8144c10 0x7f0fc815f450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8070100 tx=0x7f0fbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb6d6571a89d283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.000+0000 7f0fc7e00640 1 -- 172.21.15.62:0/330436021 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc037070 con 0x7f0fc8144c10 2024-04-25T09:03:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.000+0000 7f0fc7e00640 1 -- 172.21.15.62:0/330436021 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc032440 con 0x7f0fc8144c10 2024-04-25T09:03:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.000+0000 7f0fc7e00640 1 -- 172.21.15.62:0/330436021 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc032810 con 0x7f0fc8144c10 2024-04-25T09:03:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.001+0000 7f0fce800640 1 -- 172.21.15.62:0/330436021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc8144c10 msgr2=0x7f0fc815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.001+0000 7f0fce800640 1 --2- 172.21.15.62:0/330436021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc8144c10 0x7f0fc815f450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8070100 tx=0x7f0fbc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:03:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.001+0000 7f0fce800640 1 -- 172.21.15.62:0/330436021 shutdown_connections 2024-04-25T09:03:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.001+0000 7f0fce800640 1 --2- 172.21.15.62:0/330436021 >> v2:172.21.15.121:3300/0 conn(0x7f0fc806d900 0x7f0fc80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.001+0000 7f0fce800640 1 --2- 172.21.15.62:0/330436021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc8144c10 0x7f0fc815f450 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.001+0000 7f0fce800640 1 -- 172.21.15.62:0/330436021 >> 172.21.15.62:0/330436021 conn(0x7f0fc80b14d0 msgr2=0x7f0fc80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.001+0000 7f0fce800640 1 -- 172.21.15.62:0/330436021 shutdown_connections 2024-04-25T09:03:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.001+0000 7f0fce800640 1 -- 172.21.15.62:0/330436021 wait complete. 2024-04-25T09:03:41.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.003+0000 7f0fce800640 1 Processor -- start 2024-04-25T09:03:41.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.003+0000 7f0fce800640 1 -- start start 2024-04-25T09:03:41.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fce800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc806d900 0x7f0fc8132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:41.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fce800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc80cee40 0x7f0fc815e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fce800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc8144c10 0x7f0fc815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fce800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc8160060 con 0x7f0fc806d900 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fce800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80b41c0 con 0x7f0fc80cee40 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fce800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80b4b90 con 0x7f0fc8144c10 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fcd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc806d900 0x7f0fc8132eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fcd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc806d900 0x7f0fc8132eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36420/0 (socket says 172.21.15.62:36420) 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.004+0000 7f0fcd400640 1 -- 172.21.15.62:0/3438849591 learned_addr learned my addr 172.21.15.62:0/3438849591 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcde00640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc8144c10 0x7f0fc815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcca00640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc80cee40 0x7f0fc815e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcd400640 1 -- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc8144c10 msgr2=0x7f0fc815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcd400640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc8144c10 0x7f0fc815d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcd400640 1 -- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc80cee40 msgr2=0x7f0fc815e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcd400640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc80cee40 0x7f0fc815e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:41.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcd400640 1 -- 172.21.15.62:0/3438849591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc815ee80 con 0x7f0fc806d900 2024-04-25T09:03:41.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcca00640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc80cee40 0x7f0fc815e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:41.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fcd400640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc806d900 0x7f0fc8132eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc000c00 tx=0x7f0fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:41.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.005+0000 7f0fc5600640 1 -- 172.21.15.62:0/3438849591 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc041020 con 0x7f0fc806d900 2024-04-25T09:03:41.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.006+0000 7f0fc5600640 1 -- 172.21.15.62:0/3438849591 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc0306c0 con 0x7f0fc806d900 2024-04-25T09:03:41.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.006+0000 7f0fc5600640 1 -- 172.21.15.62:0/3438849591 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc037040 con 0x7f0fc806d900 2024-04-25T09:03:41.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.006+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc815c0e0 con 0x7f0fc806d900 2024-04-25T09:03:41.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.006+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc815c6a0 con 0x7f0fc806d900 2024-04-25T09:03:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.007+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc804ef20 con 0x7f0fc806d900 2024-04-25T09:03:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.008+0000 7f0fc5600640 1 -- 172.21.15.62:0/3438849591 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fbc044020 con 0x7f0fc806d900 2024-04-25T09:03:41.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.009+0000 7f0fc5600640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fa806b4d0 0x7f0fa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:41.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.009+0000 7f0fc5600640 1 -- 172.21.15.62:0/3438849591 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fbc0aaad0 con 0x7f0fc806d900 2024-04-25T09:03:41.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.009+0000 7f0fcca00640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fa806b4d0 0x7f0fa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:41.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.010+0000 7f0fcca00640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fa806b4d0 0x7f0fa806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80b07f0 tx=0x7f0fb4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:42.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.013+0000 7f0fc5600640 1 -- 172.21.15.62:0/3438849591 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fbc04b030 con 0x7f0fc806d900 2024-04-25T09:03:42.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.255+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0fc80b5050 con 0x7f0fc806d900 2024-04-25T09:03:42.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.256+0000 7f0fc5600640 1 -- 172.21.15.62:0/3438849591 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0fc815ee80 con 0x7f0fc806d900 2024-04-25T09:03:42.244 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:03:42.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fa806b4d0 msgr2=0x7f0fa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fa806b4d0 0x7f0fa806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80b07f0 tx=0x7f0fb4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc806d900 msgr2=0x7f0fc8132eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc806d900 0x7f0fc8132eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc000c00 tx=0x7f0fbc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 shutdown_connections 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fa806b4d0 0x7f0fa806d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fc8144c10 0x7f0fc815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc80cee40 0x7f0fc815e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 --2- 172.21.15.62:0/3438849591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fc806d900 0x7f0fc8132eb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 >> 172.21.15.62:0/3438849591 conn(0x7f0fc80b14d0 msgr2=0x7f0fc80c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 shutdown_connections 2024-04-25T09:03:42.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:42.259+0000 7f0fce800640 1 -- 172.21.15.62:0/3438849591 wait complete. 2024-04-25T09:03:42.265 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:03:42.266 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:03:42.266 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-25T09:03:42.266 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T09:03:42.266 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T09:03:42.266 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:03:42.266 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:03:42.266 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:03:42.266 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T09:03:42.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:42.266 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-25T09:03:42.324 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:42.324 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:03:42.325 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:42.326 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:03:42.326 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42616sec preferred_lft 42616sec 2024-04-25T09:03:42.326 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2024-04-25T09:03:42.326 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:42.327 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:03:42.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:42.327 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:42.327 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:42.327 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:03:42.327 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr flush dev ceph-brx 2024-04-25T09:03:42.327 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set ceph-brx up 2024-04-25T09:03:42.327 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:03:42.327 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:42.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:42.751 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:03:42.822 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-25T09:03:42.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:42.825 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-25T09:03:42.873 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-25T09:03:42.873 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:03:42.873 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:03:42.873 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:03:42.873 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:03:42.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:42.874 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:42.874 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:42.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:03:42.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:03:42.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:03:42.874 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:43.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:43.534 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-25T09:03:43.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:43.589 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2024-04-25T09:03:43.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:43.641 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:43.641 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:43.641 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:03:43.641 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T09:03:43.641 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:43.761 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T09:03:43.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:43.761 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:43.761 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:43.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T09:03:43.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:03:43.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T09:03:43.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T09:03:43.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T09:03:43.762 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:43.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:43.988 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:43.988 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:43.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-25T09:03:43.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:03:43.988 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:44.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:44.119 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T09:03:44.165 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T09:03:44.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:44.177 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T09:03:44.256 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T09:03:44.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:44.257 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:44.393 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:44.393 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:03:45.088 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:45.088 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:03:45.088 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:03:45.088 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 1@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:03:45.088 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:03:45.092 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T09:03:45.092 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:03:45.093 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:03:45.093 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:03:45.093 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:03:45.097 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:03:45.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:45.097 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T09:03:45.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:45.176 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi121.stdout:client4454 2024-04-25T09:03:45.230 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:03:45.230 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:45.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:45.284 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:45.337 INFO:teuthology.orchestra.run.smithi121.stdout:client4454 2024-04-25T09:03:45.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:45.338 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:45.391 INFO:teuthology.orchestra.run.smithi121.stdout:client4454 2024-04-25T09:03:45.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:45.392 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9e112875-3fc2-4963-90e0-2b56d5516d5c.client4454/status') 2024-04-25T09:03:45.468 INFO:teuthology.orchestra.run.smithi121.stdout:instance: client.4454 (3)192.168.0.1:0/669815156 2024-04-25T09:03:45.468 INFO:teuthology.orchestra.run.smithi121.stdout:blocklisted: false 2024-04-25T09:03:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:0+1 records in 2024-04-25T09:03:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:0+1 records out 2024-04-25T09:03:45.468 INFO:teuthology.orchestra.run.smithi121.stderr:68 bytes copied, 0.000115126 s, 591 kB/s 2024-04-25T09:03:45.471 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T09:03:45.471 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:03:45.472 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T09:03:45.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:03:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.702+0000 7fcfbb200640 1 Processor -- start 2024-04-25T09:03:45.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.702+0000 7fcfbb200640 1 -- start start 2024-04-25T09:03:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfbb200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcfb40b56f0 0x7fcfb4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfbb200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb41627a0 con 0x7fcfb40b27a0 2024-04-25T09:03:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfbb200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb40d3110 con 0x7fcfb40b6a90 2024-04-25T09:03:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfbb200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb40cedc0 con 0x7fcfb40b56f0 2024-04-25T09:03:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfb9e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcfb40b56f0 0x7fcfb4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfba800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fcfb40b27a0 0x7fcfb4162090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50364/0 (socket says 172.21.15.62:50364) 2024-04-25T09:03:45.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfba800640 1 -- 172.21.15.62:0/1749361692 learned_addr learned my addr 172.21.15.62:0/1749361692 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfb9e00640 1 -- 172.21.15.62:0/1749361692 >> v1:172.21.15.121:6789/0 conn(0x7fcfb40b6a90 legacy=0x7fcfb4161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.704+0000 7fcfb9e00640 1 -- 172.21.15.62:0/1749361692 >> v1:172.21.15.62:6789/0 conn(0x7fcfb40b27a0 legacy=0x7fcfb4162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.705+0000 7fcfb9e00640 1 -- 172.21.15.62:0/1749361692 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40d33f0 con 0x7fcfb40b56f0 2024-04-25T09:03:45.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.706+0000 7fcfb9e00640 1 --2- 172.21.15.62:0/1749361692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb4160d00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009d90 tx=0x7fcfac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce07eab99b7b587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.706+0000 7fcfb8a00640 1 -- 172.21.15.62:0/1749361692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac007740 con 0x7fcfb40b56f0 2024-04-25T09:03:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.706+0000 7fcfb8a00640 1 -- 172.21.15.62:0/1749361692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac0078e0 con 0x7fcfb40b56f0 2024-04-25T09:03:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.706+0000 7fcfb8a00640 1 -- 172.21.15.62:0/1749361692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac007cb0 con 0x7fcfb40b56f0 2024-04-25T09:03:45.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.707+0000 7fcfbb200640 1 -- 172.21.15.62:0/1749361692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 msgr2=0x7fcfb4160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.707+0000 7fcfbb200640 1 --2- 172.21.15.62:0/1749361692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb4160d00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009d90 tx=0x7fcfac02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.707+0000 7fcfbb200640 1 -- 172.21.15.62:0/1749361692 shutdown_connections 2024-04-25T09:03:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.707+0000 7fcfbb200640 1 --2- 172.21.15.62:0/1749361692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb4160d00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.707+0000 7fcfbb200640 1 -- 172.21.15.62:0/1749361692 >> 172.21.15.62:0/1749361692 conn(0x7fcfb4066d10 msgr2=0x7fcfb4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.707+0000 7fcfbb200640 1 -- 172.21.15.62:0/1749361692 shutdown_connections 2024-04-25T09:03:45.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.707+0000 7fcfbb200640 1 -- 172.21.15.62:0/1749361692 wait complete. 2024-04-25T09:03:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.709+0000 7fcfbb200640 1 Processor -- start 2024-04-25T09:03:45.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.709+0000 7fcfbb200640 1 -- start start 2024-04-25T09:03:45.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfbb200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b27a0 0x7fcfb415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfbb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb40d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfbb200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40b6a90 0x7fcfb41562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfbb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40b2d60 con 0x7fcfb40b56f0 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfbb200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40cea50 con 0x7fcfb40b6a90 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfbb200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40b2ee0 con 0x7fcfb40b27a0 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfb9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb40d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfb9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb40d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33976/0 (socket says 172.21.15.62:33976) 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.710+0000 7fcfb9400640 1 -- 172.21.15.62:0/1304918311 learned_addr learned my addr 172.21.15.62:0/1304918311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfba800640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40b6a90 0x7fcfb41562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfb9e00640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b27a0 0x7fcfb415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfb9400640 1 -- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b27a0 msgr2=0x7fcfb415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfb9400640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b27a0 0x7fcfb415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfb9400640 1 -- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40b6a90 msgr2=0x7fcfb41562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfb9400640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40b6a90 0x7fcfb41562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfb9400640 1 -- 172.21.15.62:0/1304918311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb415cf70 con 0x7fcfb40b56f0 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfb9e00640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b27a0 0x7fcfb415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfb9400640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb40d22d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400b3f0 tx=0x7fcfa400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.711+0000 7fcfaa000640 1 -- 172.21.15.62:0/1304918311 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4015020 con 0x7fcfb40b56f0 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.712+0000 7fcfaa000640 1 -- 172.21.15.62:0/1304918311 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa4011830 con 0x7fcfb40b56f0 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.712+0000 7fcfaa000640 1 -- 172.21.15.62:0/1304918311 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4011c20 con 0x7fcfb40b56f0 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.712+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb415d210 con 0x7fcfb40b56f0 2024-04-25T09:03:45.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.712+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb41e7bd0 con 0x7fcfb40b56f0 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.713+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb41e7700 con 0x7fcfb40b56f0 2024-04-25T09:03:45.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.714+0000 7fcfaa000640 1 -- 172.21.15.62:0/1304918311 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfa4018020 con 0x7fcfb40b56f0 2024-04-25T09:03:45.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.715+0000 7fcfaa000640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9806b500 0x7fcf9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.715+0000 7fcfaa000640 1 -- 172.21.15.62:0/1304918311 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfa408a170 con 0x7fcfb40b56f0 2024-04-25T09:03:45.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.716+0000 7fcfb9e00640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9806b500 0x7fcf9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.717+0000 7fcfb9e00640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9806b500 0x7fcf9806d9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfac034000 tx=0x7fcfac02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.720+0000 7fcfaa000640 1 -- 172.21.15.62:0/1304918311 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfa401f020 con 0x7fcfb40b56f0 2024-04-25T09:03:45.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.960+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcfb40ca860 con 0x7fcfb40b56f0 2024-04-25T09:03:45.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.961+0000 7fcfaa000640 1 -- 172.21.15.62:0/1304918311 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcfa40172c0 con 0x7fcfb40b56f0 2024-04-25T09:03:45.949 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9806b500 msgr2=0x7fcf9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9806b500 0x7fcf9806d9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfac034000 tx=0x7fcfac02f000 comp rx=0 tx=0).stop 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 msgr2=0x7fcfb40d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb40d22d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400b3f0 tx=0x7fcfa400b8b0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 shutdown_connections 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9806b500 0x7fcf9806d9c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40b56f0 0x7fcfb40d22d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb40b6a90 0x7fcfb41562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.964+0000 7fcfbb200640 1 --2- 172.21.15.62:0/1304918311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b27a0 0x7fcfb415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.965+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 >> 172.21.15.62:0/1304918311 conn(0x7fcfb4066d10 msgr2=0x7fcfb40b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.965+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 shutdown_connections 2024-04-25T09:03:45.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:45.965+0000 7fcfbb200640 1 -- 172.21.15.62:0/1304918311 wait complete. 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:03:45.973 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T09:03:45.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:45.974 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-25T09:03:46.032 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:46.032 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:46.032 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:46.032 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:03:46.033 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42613sec preferred_lft 42613sec 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 6e:af:d6:ac:53:1c brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::1856:1bff:fe3b:da10/64 scope link 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 6e:af:d6:ac:53:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::6caf:d6ff:feac:531c/64 scope link 2024-04-25T09:03:46.034 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.035 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:03:46.116 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-25T09:03:46.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.120 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-25T09:03:46.174 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-25T09:03:46.174 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:03:46.174 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:03:46.174 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:03:46.174 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:03:46.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.175 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:46.175 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:46.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:03:46.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:03:46.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:03:46.175 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:46.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.325 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-25T09:03:46.373 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T09:03:46.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.374 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2024-04-25T09:03:46.422 INFO:teuthology.orchestra.run.smithi121.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T09:03:46.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.423 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:46.424 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:46.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T09:03:46.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T09:03:46.424 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:46.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.542 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T09:03:46.621 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.622 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether fe:74:fa:d2:14:71 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:03:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:03:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::fc74:faff:fed2:1471/64 scope link 2024-04-25T09:03:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.625 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T09:03:46.707 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether fe:74:fa:d2:14:71 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::fc74:faff:fed2:1471/64 scope link 2024-04-25T09:03:46.708 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:46.712 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T09:03:46.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.712 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:46.712 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:46.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T09:03:46.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T09:03:46.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T09:03:46.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T09:03:46.713 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T09:03:46.713 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:46.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:46.929 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec bash -c ' 2024-04-25T09:03:46.929 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-25T09:03:46.929 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.1 up 2024-04-25T09:03:46.929 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T09:03:46.929 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-25T09:03:47.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:47.071 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T09:03:47.116 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T09:03:47.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:47.116 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T09:03:47.210 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T09:03:47.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:47.211 DEBUG:teuthology.orchestra.run.smithi121:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:47.336 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:47.336 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:03:47.368 INFO:teuthology.orchestra.run.smithi121.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:03:47.368 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:03:47.368 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:03:47.369 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: trying mount with new device syntax: 3@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:03:47.369 INFO:teuthology.orchestra.run.smithi121.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:03:47.370 INFO:teuthology.orchestra.run.smithi121.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T09:03:47.370 INFO:teuthology.orchestra.run.smithi121.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:03:47.370 INFO:teuthology.orchestra.run.smithi121.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:03:47.370 INFO:teuthology.orchestra.run.smithi121.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:03:47.370 INFO:teuthology.orchestra.run.smithi121.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:03:47.374 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:03:47.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:47.374 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T09:03:47.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:47.444 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:47.498 INFO:teuthology.orchestra.run.smithi121.stdout:client4457 2024-04-25T09:03:47.499 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:03:47.499 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:47.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:47.549 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:47.650 INFO:teuthology.orchestra.run.smithi121.stdout:client4457 2024-04-25T09:03:47.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:47.650 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:03:47.707 INFO:teuthology.orchestra.run.smithi121.stdout:client4457 2024-04-25T09:03:47.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:47.708 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9e112875-3fc2-4963-90e0-2b56d5516d5c.client4457/status') 2024-04-25T09:03:47.786 INFO:teuthology.orchestra.run.smithi121.stdout:instance: client.4457 (3)192.168.0.2:0/1948496898 2024-04-25T09:03:47.787 INFO:teuthology.orchestra.run.smithi121.stdout:blocklisted: false 2024-04-25T09:03:47.787 INFO:teuthology.orchestra.run.smithi121.stderr:0+1 records in 2024-04-25T09:03:47.787 INFO:teuthology.orchestra.run.smithi121.stderr:0+1 records out 2024-04-25T09:03:47.787 INFO:teuthology.orchestra.run.smithi121.stderr:69 bytes copied, 7.8064e-05 s, 884 kB/s 2024-04-25T09:03:47.791 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T09:03:47.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:48.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.032+0000 7fc4bf000640 1 Processor -- start 2024-04-25T09:03:48.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.032+0000 7fc4bf000640 1 -- start start 2024-04-25T09:03:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.033+0000 7fc4bf000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc4b80d1e80 0x7fc4b80d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4bf000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4b80d3450 0x7fc4b80d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4bf000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b80d23b0 con 0x7fc4b815dd70 2024-04-25T09:03:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4bf000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b80d0cc0 con 0x7fc4b80d3450 2024-04-25T09:03:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4bf000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b80d2690 con 0x7fc4b80d1e80 2024-04-25T09:03:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4bdc00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc4b815dd70 0x7fc4b80cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50376/0 (socket says 172.21.15.62:50376) 2024-04-25T09:03:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4bdc00640 1 -- 172.21.15.62:0/3776625235 learned_addr learned my addr 172.21.15.62:0/3776625235 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4be600640 1 --2- 172.21.15.62:0/3776625235 >> v2:172.21.15.62:3300/0 conn(0x7fc4b80d3450 0x7fc4b80d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4bd200640 1 --2- 172.21.15.62:0/3776625235 >> v2:172.21.15.121:3300/0 conn(0x7fc4b80d1e80 0x7fc4b80d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4b7e00640 1 -- 172.21.15.62:0/3776625235 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682422635 0 0) 0x7fc4b80d23b0 con 0x7fc4b815dd70 2024-04-25T09:03:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.034+0000 7fc4b7e00640 1 -- 172.21.15.62:0/3776625235 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a0003520 con 0x7fc4b815dd70 2024-04-25T09:03:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.035+0000 7fc4be600640 1 -- 172.21.15.62:0/3776625235 >> v2:172.21.15.121:3300/0 conn(0x7fc4b80d1e80 msgr2=0x7fc4b80d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.035+0000 7fc4be600640 1 --2- 172.21.15.62:0/3776625235 >> v2:172.21.15.121:3300/0 conn(0x7fc4b80d1e80 0x7fc4b80d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.035+0000 7fc4be600640 1 -- 172.21.15.62:0/3776625235 >> v1:172.21.15.62:6789/0 conn(0x7fc4b815dd70 legacy=0x7fc4b80cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.035+0000 7fc4be600640 1 -- 172.21.15.62:0/3776625235 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b80d2970 con 0x7fc4b80d3450 2024-04-25T09:03:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.036+0000 7fc4be600640 1 --2- 172.21.15.62:0/3776625235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b80d3450 0x7fc4b80d0780 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80c7320 tx=0x7fc4b002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5145c75036118d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.036+0000 7fc4b7e00640 1 -- 172.21.15.62:0/3776625235 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0007740 con 0x7fc4b80d3450 2024-04-25T09:03:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.036+0000 7fc4b7e00640 1 -- 172.21.15.62:0/3776625235 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b00078e0 con 0x7fc4b80d3450 2024-04-25T09:03:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.036+0000 7fc4b7e00640 1 -- 172.21.15.62:0/3776625235 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0007c90 con 0x7fc4b80d3450 2024-04-25T09:03:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.037+0000 7fc4bf000640 1 -- 172.21.15.62:0/3776625235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b80d3450 msgr2=0x7fc4b80d0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.037+0000 7fc4bf000640 1 --2- 172.21.15.62:0/3776625235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b80d3450 0x7fc4b80d0780 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80c7320 tx=0x7fc4b002a680 comp rx=0 tx=0).stop 2024-04-25T09:03:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.037+0000 7fc4bf000640 1 -- 172.21.15.62:0/3776625235 shutdown_connections 2024-04-25T09:03:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.037+0000 7fc4bf000640 1 --2- 172.21.15.62:0/3776625235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b80d3450 0x7fc4b80d0780 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.037+0000 7fc4bf000640 1 --2- 172.21.15.62:0/3776625235 >> v2:172.21.15.121:3300/0 conn(0x7fc4b80d1e80 0x7fc4b80d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.037+0000 7fc4bf000640 1 -- 172.21.15.62:0/3776625235 >> 172.21.15.62:0/3776625235 conn(0x7fc4b806b210 msgr2=0x7fc4b80668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.037+0000 7fc4bf000640 1 -- 172.21.15.62:0/3776625235 shutdown_connections 2024-04-25T09:03:48.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.037+0000 7fc4bf000640 1 -- 172.21.15.62:0/3776625235 wait complete. 2024-04-25T09:03:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.039+0000 7fc4bf000640 1 Processor -- start 2024-04-25T09:03:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.039+0000 7fc4bf000640 1 -- start start 2024-04-25T09:03:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4bf000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4b80d1e80 0x7fc4b815b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4bf000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4b80d3450 0x7fc4b80ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4bf000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b815dd70 0x7fc4b80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4bf000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b815e150 con 0x7fc4b815dd70 2024-04-25T09:03:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4bf000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80d2f30 con 0x7fc4b80d3450 2024-04-25T09:03:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4bf000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80b6c90 con 0x7fc4b80d1e80 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4be600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b815dd70 0x7fc4b80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4be600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b815dd70 0x7fc4b80acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33996/0 (socket says 172.21.15.62:33996) 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.040+0000 7fc4be600640 1 -- 172.21.15.62:0/2508235667 learned_addr learned my addr 172.21.15.62:0/2508235667 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4bd200640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4b80d3450 0x7fc4b80ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4bdc00640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4b80d1e80 0x7fc4b815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4be600640 1 -- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4b80d1e80 msgr2=0x7fc4b815b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4be600640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4b80d1e80 0x7fc4b815b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4be600640 1 -- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4b80d3450 msgr2=0x7fc4b80ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4be600640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4b80d3450 0x7fc4b80ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4be600640 1 -- 172.21.15.62:0/2508235667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b80ab210 con 0x7fc4b815dd70 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4bd200640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4b80d3450 0x7fc4b80ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4bdc00640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4b80d1e80 0x7fc4b815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4be600640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b815dd70 0x7fc4b80acd10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4b002ac70 tx=0x7fc4b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4b5600640 1 -- 172.21.15.62:0/2508235667 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0040070 con 0x7fc4b815dd70 2024-04-25T09:03:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4b5600640 1 -- 172.21.15.62:0/2508235667 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b0031c60 con 0x7fc4b815dd70 2024-04-25T09:03:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4b5600640 1 -- 172.21.15.62:0/2508235667 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0009450 con 0x7fc4b815dd70 2024-04-25T09:03:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.041+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b80ab4f0 con 0x7fc4b815dd70 2024-04-25T09:03:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.042+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b815e750 con 0x7fc4b815dd70 2024-04-25T09:03:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.043+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b804ef20 con 0x7fc4b815dd70 2024-04-25T09:03:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.044+0000 7fc4b5600640 1 -- 172.21.15.62:0/2508235667 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4b0039040 con 0x7fc4b815dd70 2024-04-25T09:03:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.044+0000 7fc4b5600640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc49806b4d0 0x7fc49806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.045+0000 7fc4b5600640 1 -- 172.21.15.62:0/2508235667 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4b00ab660 con 0x7fc4b815dd70 2024-04-25T09:03:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.045+0000 7fc4bdc00640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc49806b4d0 0x7fc49806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.046+0000 7fc4bdc00640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc49806b4d0 0x7fc49806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac005880 tx=0x7fc4ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.049+0000 7fc4b5600640 1 -- 172.21.15.62:0/2508235667 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4b007a4d0 con 0x7fc4b815dd70 2024-04-25T09:03:48.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.252+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4b815e920 con 0x7fc4b815dd70 2024-04-25T09:03:48.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.255+0000 7fc4b5600640 1 -- 172.21.15.62:0/2508235667 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fc4b0079dd0 con 0x7fc4b815dd70 2024-04-25T09:03:48.245 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:48.245 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc49806b4d0 msgr2=0x7fc49806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc49806b4d0 0x7fc49806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac005880 tx=0x7fc4ac009040 comp rx=0 tx=0).stop 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b815dd70 msgr2=0x7fc4b80acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b815dd70 0x7fc4b80acd10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4b002ac70 tx=0x7fc4b0004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 shutdown_connections 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc49806b4d0 0x7fc49806d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4b815dd70 0x7fc4b80acd10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4b80d3450 0x7fc4b80ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 --2- 172.21.15.62:0/2508235667 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4b80d1e80 0x7fc4b815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 >> 172.21.15.62:0/2508235667 conn(0x7fc4b806b210 msgr2=0x7fc4b80cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 shutdown_connections 2024-04-25T09:03:48.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.258+0000 7fc4bf000640 1 -- 172.21.15.62:0/2508235667 wait complete. 2024-04-25T09:03:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:48.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:48.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.489+0000 7f3fa6a00640 1 Processor -- start 2024-04-25T09:03:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.490+0000 7f3fa6a00640 1 -- start start 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.491+0000 7f3fa6a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.491+0000 7f3fa6a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3fa00d2fb0 0x7f3fa00ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.491+0000 7f3fa6a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa00ce780 con 0x7f3fa00d3960 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.491+0000 7f3fa6a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa00d3440 con 0x7f3fa00d2fb0 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.491+0000 7f3fa6a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00ce1c0 con 0x7f3fa015dd50 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.491+0000 7f3fa4c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3fa00d2fb0 0x7f3fa00ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.492+0000 7f3fa4c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3fa00d2fb0 0x7f3fa00ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34002/0 (socket says 172.21.15.62:34002) 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.492+0000 7f3fa4c00640 1 -- 172.21.15.62:0/3627866183 learned_addr learned my addr 172.21.15.62:0/3627866183 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.492+0000 7f3fa5600640 1 --2- 172.21.15.62:0/3627866183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.492+0000 7f3fa4c00640 1 -- 172.21.15.62:0/3627866183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 msgr2=0x7f3fa00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.492+0000 7f3fa4c00640 1 --2- 172.21.15.62:0/3627866183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.492+0000 7f3fa4c00640 1 -- 172.21.15.62:0/3627866183 >> v1:172.21.15.121:6789/0 conn(0x7f3fa00d3960 legacy=0x7f3fa00d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.492+0000 7f3fa4c00640 1 -- 172.21.15.62:0/3627866183 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00ce4a0 con 0x7f3fa00d2fb0 2024-04-25T09:03:48.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.492+0000 7f3fa5600640 1 --2- 172.21.15.62:0/3627866183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.493+0000 7f3fa4c00640 1 --2- 172.21.15.62:0/3627866183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d2fb0 0x7f3fa00ce9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009870 tx=0x7f3f8c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2297d613b27082eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.493+0000 7f3f9fe00640 1 -- 172.21.15.62:0/3627866183 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c037070 con 0x7f3fa00d2fb0 2024-04-25T09:03:48.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.493+0000 7f3f9fe00640 1 -- 172.21.15.62:0/3627866183 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c032440 con 0x7f3fa00d2fb0 2024-04-25T09:03:48.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.493+0000 7f3f9fe00640 1 -- 172.21.15.62:0/3627866183 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c032810 con 0x7f3fa00d2fb0 2024-04-25T09:03:48.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.494+0000 7f3fa6a00640 1 -- 172.21.15.62:0/3627866183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d2fb0 msgr2=0x7f3fa00ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.494+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/3627866183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d2fb0 0x7f3fa00ce9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009870 tx=0x7f3f8c02a210 comp rx=0 tx=0).stop 2024-04-25T09:03:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.494+0000 7f3fa6a00640 1 -- 172.21.15.62:0/3627866183 shutdown_connections 2024-04-25T09:03:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.494+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/3627866183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d2fb0 0x7f3fa00ce9c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.494+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/3627866183 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.494+0000 7f3fa6a00640 1 -- 172.21.15.62:0/3627866183 >> 172.21.15.62:0/3627866183 conn(0x7f3fa006b000 msgr2=0x7f3fa006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.494+0000 7f3fa6a00640 1 -- 172.21.15.62:0/3627866183 shutdown_connections 2024-04-25T09:03:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.494+0000 7f3fa6a00640 1 -- 172.21.15.62:0/3627866183 wait complete. 2024-04-25T09:03:48.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.496+0000 7f3fa6a00640 1 Processor -- start 2024-04-25T09:03:48.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.496+0000 7f3fa6a00640 1 -- start start 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d3960 0x7f3fa00aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa6a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00aa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa00b4730 0x7f3fa015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa6a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00b6b00 con 0x7f3fa00d3960 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa6a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00cef00 con 0x7f3fa00b4730 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa6a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00b6c80 con 0x7f3fa015dd50 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d3960 0x7f3fa00aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d3960 0x7f3fa00aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34014/0 (socket says 172.21.15.62:34014) 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.497+0000 7f3fa5600640 1 -- 172.21.15.62:0/1759571676 learned_addr learned my addr 172.21.15.62:0/1759571676 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa4c00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa6000640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa00b4730 0x7f3fa015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa5600640 1 -- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 msgr2=0x7f3fa00aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa5600640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa5600640 1 -- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa00b4730 msgr2=0x7f3fa015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa5600640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa00b4730 0x7f3fa015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa5600640 1 -- 172.21.15.62:0/1759571676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00b4c10 con 0x7f3fa00d3960 2024-04-25T09:03:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa4c00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa6000640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa00b4730 0x7f3fa015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa5600640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d3960 0x7f3fa00aa3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000ce00 tx=0x7f3f9000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3f9d600640 1 -- 172.21.15.62:0/1759571676 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9000f890 con 0x7f3fa00d3960 2024-04-25T09:03:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3f9d600640 1 -- 172.21.15.62:0/1759571676 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9000fa30 con 0x7f3fa00d3960 2024-04-25T09:03:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa00b4ef0 con 0x7f3fa00d3960 2024-04-25T09:03:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.498+0000 7f3f9d600640 1 -- 172.21.15.62:0/1759571676 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9000fe00 con 0x7f3fa00d3960 2024-04-25T09:03:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.499+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa00b54b0 con 0x7f3fa00d3960 2024-04-25T09:03:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.500+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa004ef20 con 0x7f3fa00d3960 2024-04-25T09:03:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.501+0000 7f3f9d600640 1 -- 172.21.15.62:0/1759571676 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f90018400 con 0x7f3fa00d3960 2024-04-25T09:03:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.502+0000 7f3f9d600640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3f8006b4d0 0x7f3f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.502+0000 7f3f9d600640 1 -- 172.21.15.62:0/1759571676 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f9008b210 con 0x7f3fa00d3960 2024-04-25T09:03:48.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.503+0000 7f3fa4c00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3f8006b4d0 0x7f3f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.504+0000 7f3fa4c00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3f8006b4d0 0x7f3f8006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c034000 tx=0x7f3f8c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.507+0000 7f3f9d600640 1 -- 172.21.15.62:0/1759571676 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f90014030 con 0x7f3fa00d3960 2024-04-25T09:03:48.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.698+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fa00b2d40 con 0x7f3fa00d3960 2024-04-25T09:03:48.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.699+0000 7f3f9d600640 1 -- 172.21.15.62:0/1759571676 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f9000ae20 con 0x7f3fa00d3960 2024-04-25T09:03:48.688 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.701+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3f8006b4d0 msgr2=0x7f3f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.701+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3f8006b4d0 0x7f3f8006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c034000 tx=0x7f3f8c02e000 comp rx=0 tx=0).stop 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.701+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d3960 msgr2=0x7f3fa00aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.701+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d3960 0x7f3fa00aa3c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000ce00 tx=0x7f3f9000a650 comp rx=0 tx=0).stop 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.702+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 shutdown_connections 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.702+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3f8006b4d0 0x7f3f8006d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.702+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa00b4730 0x7f3fa015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.702+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3fa015dd50 0x7f3fa00aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.702+0000 7f3fa6a00640 1 --2- 172.21.15.62:0/1759571676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3fa00d3960 0x7f3fa00aa3c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.702+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 >> 172.21.15.62:0/1759571676 conn(0x7f3fa006b000 msgr2=0x7f3fa006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.702+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 shutdown_connections 2024-04-25T09:03:48.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.702+0000 7f3fa6a00640 1 -- 172.21.15.62:0/1759571676 wait complete. 2024-04-25T09:03:48.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:48.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.931+0000 7fe5f9800640 1 Processor -- start 2024-04-25T09:03:48.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.931+0000 7fe5f9800640 1 -- start start 2024-04-25T09:03:48.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.933+0000 7fe5f9800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5f415e7c0 0x7fe5f4168800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.933+0000 7fe5f9800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f40b5560 con 0x7fe5f406d9d0 2024-04-25T09:03:48.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.933+0000 7fe5f9800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f415ee30 con 0x7fe5f40ceb50 2024-04-25T09:03:48.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.933+0000 7fe5f9800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f415ec10 con 0x7fe5f415e7c0 2024-04-25T09:03:48.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.934+0000 7fe5f8e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5f415e7c0 0x7fe5f4168800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.934+0000 7fe5f3400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe5f406d9d0 0x7fe5f415e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50390/0 (socket says 172.21.15.62:50390) 2024-04-25T09:03:48.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.934+0000 7fe5f3400640 1 -- 172.21.15.62:0/1787916290 learned_addr learned my addr 172.21.15.62:0/1787916290 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:48.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.934+0000 7fe5f8e00640 1 -- 172.21.15.62:0/1787916290 >> v1:172.21.15.121:6789/0 conn(0x7fe5f40ceb50 legacy=0x7fe5f406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:48.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.934+0000 7fe5f8e00640 1 -- 172.21.15.62:0/1787916290 >> v1:172.21.15.62:6789/0 conn(0x7fe5f406d9d0 legacy=0x7fe5f415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.934+0000 7fe5f8e00640 1 -- 172.21.15.62:0/1787916290 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f41618b0 con 0x7fe5f415e7c0 2024-04-25T09:03:48.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.934+0000 7fe5f2a00640 1 -- 172.21.15.62:0/1787916290 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497164541 0 0) 0x7fe5f40b5560 con 0x7fe5f406d9d0 2024-04-25T09:03:48.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.935+0000 7fe5f8e00640 1 --2- 172.21.15.62:0/1787916290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f415e7c0 0x7fe5f4168800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009d90 tx=0x7fe5ec02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19587c401b2947cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.936+0000 7fe5f2a00640 1 -- 172.21.15.62:0/1787916290 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec031600 con 0x7fe5f415e7c0 2024-04-25T09:03:48.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.936+0000 7fe5f2a00640 1 -- 172.21.15.62:0/1787916290 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec0317a0 con 0x7fe5f415e7c0 2024-04-25T09:03:48.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.936+0000 7fe5f2a00640 1 -- 172.21.15.62:0/1787916290 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec031b70 con 0x7fe5f415e7c0 2024-04-25T09:03:48.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.936+0000 7fe5f9800640 1 -- 172.21.15.62:0/1787916290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f415e7c0 msgr2=0x7fe5f4168800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.936+0000 7fe5f9800640 1 --2- 172.21.15.62:0/1787916290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f415e7c0 0x7fe5f4168800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009d90 tx=0x7fe5ec02a610 comp rx=0 tx=0).stop 2024-04-25T09:03:48.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.936+0000 7fe5f9800640 1 -- 172.21.15.62:0/1787916290 shutdown_connections 2024-04-25T09:03:48.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.936+0000 7fe5f9800640 1 --2- 172.21.15.62:0/1787916290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f415e7c0 0x7fe5f4168800 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.936+0000 7fe5f9800640 1 -- 172.21.15.62:0/1787916290 >> 172.21.15.62:0/1787916290 conn(0x7fe5f4066f20 msgr2=0x7fe5f40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:48.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.937+0000 7fe5f9800640 1 -- 172.21.15.62:0/1787916290 shutdown_connections 2024-04-25T09:03:48.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.937+0000 7fe5f9800640 1 -- 172.21.15.62:0/1787916290 wait complete. 2024-04-25T09:03:48.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.938+0000 7fe5f9800640 1 Processor -- start 2024-04-25T09:03:48.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.938+0000 7fe5f9800640 1 -- start start 2024-04-25T09:03:48.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.939+0000 7fe5f9800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f406d9d0 0x7fe5f40cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.939+0000 7fe5f9800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f40ceb50 0x7fe5f40b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f9800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5f415e7c0 0x7fe5f40b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f9800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40cf0b0 con 0x7fe5f406d9d0 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f9800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40cef30 con 0x7fe5f40ceb50 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f9800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4162390 con 0x7fe5f415e7c0 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f406d9d0 0x7fe5f40cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f406d9d0 0x7fe5f40cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34028/0 (socket says 172.21.15.62:34028) 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3e00640 1 -- 172.21.15.62:0/1304796350 learned_addr learned my addr 172.21.15.62:0/1304796350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3e00640 1 -- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5f415e7c0 msgr2=0x7fe5f40b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f8e00640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5f415e7c0 0x7fe5f40b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3400640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f40ceb50 0x7fe5f40b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3e00640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5f415e7c0 0x7fe5f40b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3e00640 1 -- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f40ceb50 msgr2=0x7fe5f40b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3e00640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f40ceb50 0x7fe5f40b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.940+0000 7fe5f3e00640 1 -- 172.21.15.62:0/1304796350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40cdd90 con 0x7fe5f406d9d0 2024-04-25T09:03:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.941+0000 7fe5f3e00640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f406d9d0 0x7fe5f40cd850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400c430 tx=0x7fe5e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.941+0000 7fe5ebe00640 1 -- 172.21.15.62:0/1304796350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4011790 con 0x7fe5f406d9d0 2024-04-25T09:03:48.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.941+0000 7fe5ebe00640 1 -- 172.21.15.62:0/1304796350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4017070 con 0x7fe5f406d9d0 2024-04-25T09:03:48.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.941+0000 7fe5ebe00640 1 -- 172.21.15.62:0/1304796350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4011bb0 con 0x7fe5f406d9d0 2024-04-25T09:03:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.941+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f41f82a0 con 0x7fe5f406d9d0 2024-04-25T09:03:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.941+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f41f8860 con 0x7fe5f406d9d0 2024-04-25T09:03:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.942+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f406abe0 con 0x7fe5f406d9d0 2024-04-25T09:03:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.943+0000 7fe5ebe00640 1 -- 172.21.15.62:0/1304796350 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5e401c460 con 0x7fe5f406d9d0 2024-04-25T09:03:48.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.945+0000 7fe5ebe00640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5d406b500 0x7fe5d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:48.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.945+0000 7fe5ebe00640 1 -- 172.21.15.62:0/1304796350 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5e4089ff0 con 0x7fe5f406d9d0 2024-04-25T09:03:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.945+0000 7fe5f3400640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5d406b500 0x7fe5d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.946+0000 7fe5f3400640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5d406b500 0x7fe5d406d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0007fd0 tx=0x7fe5e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:48.949+0000 7fe5ebe00640 1 -- 172.21.15.62:0/1304796350 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5e4058ea0 con 0x7fe5f406d9d0 2024-04-25T09:03:49.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.145+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5f41422a0 con 0x7fe5f406d9d0 2024-04-25T09:03:49.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.146+0000 7fe5ebe00640 1 -- 172.21.15.62:0/1304796350 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5e40587a0 con 0x7fe5f406d9d0 2024-04-25T09:03:49.135 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:03:49.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.148+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5d406b500 msgr2=0x7fe5d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.148+0000 7fe5f9800640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5d406b500 0x7fe5d406d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0007fd0 tx=0x7fe5e0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:49.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.148+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f406d9d0 msgr2=0x7fe5f40cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.148+0000 7fe5f9800640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f406d9d0 0x7fe5f40cd850 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400c430 tx=0x7fe5e400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.149+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 shutdown_connections 2024-04-25T09:03:49.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.149+0000 7fe5f9800640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5d406b500 0x7fe5d406d9c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.149+0000 7fe5f9800640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5f415e7c0 0x7fe5f40b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.149+0000 7fe5f9800640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5f40ceb50 0x7fe5f40b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.149+0000 7fe5f9800640 1 --2- 172.21.15.62:0/1304796350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5f406d9d0 0x7fe5f40cd850 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.149+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 >> 172.21.15.62:0/1304796350 conn(0x7fe5f4066f20 msgr2=0x7fe5f415f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.149+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 shutdown_connections 2024-04-25T09:03:49.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.149+0000 7fe5f9800640 1 -- 172.21.15.62:0/1304796350 wait complete. 2024-04-25T09:03:49.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:49.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.383+0000 7f0e3ee00640 1 Processor -- start 2024-04-25T09:03:49.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.383+0000 7f0e3ee00640 1 -- start start 2024-04-25T09:03:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.385+0000 7f0e3ee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e3806d2c0 0x7f0e380ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.385+0000 7f0e3ee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e380cf000 con 0x7f0e380b31a0 2024-04-25T09:03:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.385+0000 7f0e3ee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e380cf2e0 con 0x7f0e3806d6a0 2024-04-25T09:03:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.385+0000 7f0e3ee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380b26c0 con 0x7f0e3806d2c0 2024-04-25T09:03:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.385+0000 7f0e3e400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0e380b31a0 0x7f0e380b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50396/0 (socket says 172.21.15.62:50396) 2024-04-25T09:03:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.385+0000 7f0e3e400640 1 -- 172.21.15.62:0/2026782250 learned_addr learned my addr 172.21.15.62:0/2026782250 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e3da00640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e3806d2c0 0x7f0e380ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828289674 0 0) 0x7f0e380cf000 con 0x7f0e380b31a0 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e20003520 con 0x7f0e380b31a0 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189947273 0 0) 0x7f0e20003520 con 0x7f0e380b31a0 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e3806d2c0 msgr2=0x7f0e380ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e3806d2c0 0x7f0e380ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 >> v1:172.21.15.121:6789/0 conn(0x7f0e3806d6a0 legacy=0x7f0e380ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380b29a0 con 0x7f0e380b31a0 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f0e300019f0 con 0x7f0e380b31a0 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 >> v1:172.21.15.62:6789/0 conn(0x7f0e380b31a0 legacy=0x7f0e380b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.386+0000 7f0e37e00640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e3806d2c0 0x7f0e200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e37e00640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e20003fb0 0x7f0e20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e37e00640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e20004d40 0x7f0e200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380b26c0 con 0x7f0e20004d40 2024-04-25T09:03:49.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000bb00 con 0x7f0e20003fb0 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000bd00 con 0x7f0e3806d2c0 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3e400640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e20004d40 0x7f0e200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3da00640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e3806d2c0 0x7f0e200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3d000640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e20003fb0 0x7f0e20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3e400640 1 -- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e3806d2c0 msgr2=0x7f0e200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3e400640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e3806d2c0 0x7f0e200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3e400640 1 -- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e20003fb0 msgr2=0x7f0e20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3e400640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e20003fb0 0x7f0e20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3e400640 1 -- 172.21.15.62:0/2026782250 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20003520 con 0x7f0e20004d40 2024-04-25T09:03:49.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.387+0000 7f0e3d000640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e20003fb0 0x7f0e20004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.388+0000 7f0e3e400640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e20004d40 0x7f0e200051c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000a790 tx=0x7f0e20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcf1f15bac9bc30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.389+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3000a3b0 con 0x7f0e20004d40 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.389+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3002f070 con 0x7f0e20004d40 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.389+0000 7f0e37e00640 1 -- 172.21.15.62:0/2026782250 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3002bc40 con 0x7f0e20004d40 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.390+0000 7f0e3ee00640 1 -- 172.21.15.62:0/2026782250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e20004d40 msgr2=0x7f0e200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.390+0000 7f0e3ee00640 1 --2- 172.21.15.62:0/2026782250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e20004d40 0x7f0e200051c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000a790 tx=0x7f0e20005830 comp rx=0 tx=0).stop 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.390+0000 7f0e3e400640 1 -- 172.21.15.62:0/2026782250 reap_dead start 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.390+0000 7f0e3ee00640 1 -- 172.21.15.62:0/2026782250 shutdown_connections 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.390+0000 7f0e3ee00640 1 -- 172.21.15.62:0/2026782250 >> 172.21.15.62:0/2026782250 conn(0x7f0e38066f20 msgr2=0x7f0e380b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.390+0000 7f0e3ee00640 1 -- 172.21.15.62:0/2026782250 shutdown_connections 2024-04-25T09:03:49.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.390+0000 7f0e3ee00640 1 -- 172.21.15.62:0/2026782250 wait complete. 2024-04-25T09:03:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.391+0000 7f0e3ee00640 1 Processor -- start 2024-04-25T09:03:49.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.392+0000 7f0e3ee00640 1 -- start start 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3ee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e20004d40 0x7f0e380c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e3815b590 0x7f0e381e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e381e4e50 0x7f0e381e6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e381448d0 con 0x7f0e381e4e50 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3806db20 con 0x7f0e3815b590 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3ee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38144a50 con 0x7f0e20004d40 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e381e4e50 0x7f0e381e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e381e4e50 0x7f0e381e6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34054/0 (socket says 172.21.15.62:34054) 2024-04-25T09:03:49.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3e400640 1 -- 172.21.15.62:0/3739170891 learned_addr learned my addr 172.21.15.62:0/3739170891 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:49.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3da00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e20004d40 0x7f0e380c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3d000640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e3815b590 0x7f0e381e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3e400640 1 -- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e20004d40 msgr2=0x7f0e380c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3e400640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e20004d40 0x7f0e380c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3e400640 1 -- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e3815b590 msgr2=0x7f0e381e63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3e400640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e3815b590 0x7f0e381e63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.393+0000 7f0e3e400640 1 -- 172.21.15.62:0/3739170891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e381e53c0 con 0x7f0e381e4e50 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.394+0000 7f0e3e400640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e381e4e50 0x7f0e381e6920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0e30000f80 tx=0x7f0e380cf1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.394+0000 7f0e35600640 1 -- 172.21.15.62:0/3739170891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30044070 con 0x7f0e381e4e50 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.394+0000 7f0e35600640 1 -- 172.21.15.62:0/3739170891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e30001250 con 0x7f0e381e4e50 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.394+0000 7f0e35600640 1 -- 172.21.15.62:0/3739170891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3000a3b0 con 0x7f0e381e4e50 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.394+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e381e56a0 con 0x7f0e381e4e50 2024-04-25T09:03:49.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.394+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e381e5c60 con 0x7f0e381e4e50 2024-04-25T09:03:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.395+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3806abe0 con 0x7f0e381e4e50 2024-04-25T09:03:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.396+0000 7f0e35600640 1 -- 172.21.15.62:0/3739170891 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e30047070 con 0x7f0e381e4e50 2024-04-25T09:03:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.397+0000 7f0e35600640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e1c06b550 0x7f0e1c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.397+0000 7f0e35600640 1 -- 172.21.15.62:0/3739170891 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e300aa3a0 con 0x7f0e381e4e50 2024-04-25T09:03:49.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.398+0000 7f0e3da00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e1c06b550 0x7f0e1c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.399+0000 7f0e3da00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e1c06b550 0x7f0e1c06da10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0e38144430 tx=0x7f0e2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.403+0000 7f0e35600640 1 -- 172.21.15.62:0/3739170891 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e30079200 con 0x7f0e381e4e50 2024-04-25T09:03:49.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.599+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e3804ef20 con 0x7f0e381e4e50 2024-04-25T09:03:49.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.600+0000 7f0e35600640 1 -- 172.21.15.62:0/3739170891 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e30078b00 con 0x7f0e381e4e50 2024-04-25T09:03:49.589 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:03:49.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e1c06b550 msgr2=0x7f0e1c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e1c06b550 0x7f0e1c06da10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0e38144430 tx=0x7f0e2c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e381e4e50 msgr2=0x7f0e381e6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e381e4e50 0x7f0e381e6920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0e30000f80 tx=0x7f0e380cf1d0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 shutdown_connections 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e1c06b550 0x7f0e1c06da10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e381e4e50 0x7f0e381e6920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e3815b590 0x7f0e381e63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 --2- 172.21.15.62:0/3739170891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e20004d40 0x7f0e380c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.602+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 >> 172.21.15.62:0/3739170891 conn(0x7f0e38066f20 msgr2=0x7f0e3815c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.603+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 shutdown_connections 2024-04-25T09:03:49.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.603+0000 7f0e3ee00640 1 -- 172.21.15.62:0/3739170891 wait complete. 2024-04-25T09:03:49.609 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:49.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.815+0000 7f33a6200640 1 Processor -- start 2024-04-25T09:03:49.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.816+0000 7f33a6200640 1 -- start start 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.817+0000 7f33a6200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a015bd10 0x7f33a00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.817+0000 7f33a6200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f33a00cccf0 0x7f33a00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.817+0000 7f33a6200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a006ca70 con 0x7f33a00aed00 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.818+0000 7f33a6200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a006d660 con 0x7f33a00cccf0 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.818+0000 7f33a6200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00ce6d0 con 0x7f33a015bd10 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.818+0000 7f339fe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f33a00aed00 0x7f33a015f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50398/0 (socket says 172.21.15.62:50398) 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.818+0000 7f339fe00640 1 -- 172.21.15.62:0/1525891536 learned_addr learned my addr 172.21.15.62:0/1525891536 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.818+0000 7f339ea00640 1 --2- 172.21.15.62:0/1525891536 >> v2:172.21.15.62:3300/0 conn(0x7f33a00cccf0 0x7f33a00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.818+0000 7f339f400640 1 --2- 172.21.15.62:0/1525891536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a015bd10 0x7f33a00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.818+0000 7f339e000640 1 -- 172.21.15.62:0/1525891536 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601355713 0 0) 0x7f33a006ca70 con 0x7f33a00aed00 2024-04-25T09:03:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.818+0000 7f339e000640 1 -- 172.21.15.62:0/1525891536 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3380003520 con 0x7f33a00aed00 2024-04-25T09:03:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.819+0000 7f339ea00640 1 -- 172.21.15.62:0/1525891536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a015bd10 msgr2=0x7f33a00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.819+0000 7f339ea00640 1 --2- 172.21.15.62:0/1525891536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a015bd10 0x7f33a00cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.819+0000 7f339ea00640 1 -- 172.21.15.62:0/1525891536 >> v1:172.21.15.62:6789/0 conn(0x7f33a00aed00 legacy=0x7f33a015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.819+0000 7f339ea00640 1 -- 172.21.15.62:0/1525891536 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a015c0f0 con 0x7f33a00cccf0 2024-04-25T09:03:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.820+0000 7f339ea00640 1 --2- 172.21.15.62:0/1525891536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a00cccf0 0x7f33a00ae7c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33a006d7e0 tx=0x7f338802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9ce14f82a8a3b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.820+0000 7f339e000640 1 -- 172.21.15.62:0/1525891536 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388037070 con 0x7f33a00cccf0 2024-04-25T09:03:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.820+0000 7f339e000640 1 -- 172.21.15.62:0/1525891536 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3388032440 con 0x7f33a00cccf0 2024-04-25T09:03:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.820+0000 7f339e000640 1 -- 172.21.15.62:0/1525891536 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388032810 con 0x7f33a00cccf0 2024-04-25T09:03:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.821+0000 7f33a6200640 1 -- 172.21.15.62:0/1525891536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a00cccf0 msgr2=0x7f33a00ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.821+0000 7f33a6200640 1 --2- 172.21.15.62:0/1525891536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a00cccf0 0x7f33a00ae7c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33a006d7e0 tx=0x7f338802a180 comp rx=0 tx=0).stop 2024-04-25T09:03:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.821+0000 7f33a6200640 1 -- 172.21.15.62:0/1525891536 shutdown_connections 2024-04-25T09:03:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.821+0000 7f33a6200640 1 --2- 172.21.15.62:0/1525891536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a00cccf0 0x7f33a00ae7c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.821+0000 7f33a6200640 1 --2- 172.21.15.62:0/1525891536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a015bd10 0x7f33a00cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.821+0000 7f33a6200640 1 -- 172.21.15.62:0/1525891536 >> 172.21.15.62:0/1525891536 conn(0x7f33a00668f0 msgr2=0x7f33a0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.821+0000 7f33a6200640 1 -- 172.21.15.62:0/1525891536 shutdown_connections 2024-04-25T09:03:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.821+0000 7f33a6200640 1 -- 172.21.15.62:0/1525891536 wait complete. 2024-04-25T09:03:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.823+0000 7f33a6200640 1 Processor -- start 2024-04-25T09:03:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.823+0000 7f33a6200640 1 -- start start 2024-04-25T09:03:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.824+0000 7f33a6200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00aed00 0x7f33a0154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.824+0000 7f33a6200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a00cccf0 0x7f33a00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.824+0000 7f33a6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a015bd10 0x7f33a014a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.824+0000 7f33a6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a006d2f0 con 0x7f33a015bd10 2024-04-25T09:03:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.824+0000 7f33a6200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a006d010 con 0x7f33a00aed00 2024-04-25T09:03:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.824+0000 7f33a6200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00d01e0 con 0x7f33a00cccf0 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a015bd10 0x7f33a014a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a015bd10 0x7f33a014a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34078/0 (socket says 172.21.15.62:34078) 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 -- 172.21.15.62:0/3016606547 learned_addr learned my addr 172.21.15.62:0/3016606547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339ea00640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a00cccf0 0x7f33a00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339f400640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00aed00 0x7f33a0154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 -- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a00cccf0 msgr2=0x7f33a00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a00cccf0 0x7f33a00702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 -- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00aed00 msgr2=0x7f33a0154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00aed00 0x7f33a0154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 -- 172.21.15.62:0/3016606547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a0070940 con 0x7f33a015bd10 2024-04-25T09:03:49.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339ea00640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a00cccf0 0x7f33a00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:49.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.825+0000 7f339fe00640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a015bd10 0x7f33a014a200 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f339800cb40 tx=0x7f339800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.826+0000 7f3397400640 1 -- 172.21.15.62:0/3016606547 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398017070 con 0x7f33a015bd10 2024-04-25T09:03:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.826+0000 7f3397400640 1 -- 172.21.15.62:0/3016606547 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339800aca0 con 0x7f33a015bd10 2024-04-25T09:03:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.826+0000 7f3397400640 1 -- 172.21.15.62:0/3016606547 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398012620 con 0x7f33a015bd10 2024-04-25T09:03:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.826+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a0070c20 con 0x7f33a015bd10 2024-04-25T09:03:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.826+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a0157b00 con 0x7f33a015bd10 2024-04-25T09:03:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.827+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a0071010 con 0x7f33a015bd10 2024-04-25T09:03:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.828+0000 7f3397400640 1 -- 172.21.15.62:0/3016606547 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f339801a070 con 0x7f33a015bd10 2024-04-25T09:03:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.829+0000 7f3397400640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f337c06b4d0 0x7f337c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.829+0000 7f3397400640 1 -- 172.21.15.62:0/3016606547 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f339808a0a0 con 0x7f33a015bd10 2024-04-25T09:03:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.830+0000 7f339f400640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f337c06b4d0 0x7f337c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:49.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.831+0000 7f339f400640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f337c06b4d0 0x7f337c06d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33a01596c0 tx=0x7f3390008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:49.833+0000 7f3397400640 1 -- 172.21.15.62:0/3016606547 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3398058f10 con 0x7f33a015bd10 2024-04-25T09:03:50.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.037+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33a0072c70 con 0x7f33a015bd10 2024-04-25T09:03:50.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.038+0000 7f3397400640 1 -- 172.21.15.62:0/3016606547 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f3398058810 con 0x7f33a015bd10 2024-04-25T09:03:50.028 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:50.028 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:50.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f337c06b4d0 msgr2=0x7f337c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f337c06b4d0 0x7f337c06d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f33a01596c0 tx=0x7f3390008040 comp rx=0 tx=0).stop 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a015bd10 msgr2=0x7f33a014a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a015bd10 0x7f33a014a200 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f339800cb40 tx=0x7f339800a060 comp rx=0 tx=0).stop 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 shutdown_connections 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f337c06b4d0 0x7f337c06d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33a015bd10 0x7f33a014a200 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33a00cccf0 0x7f33a00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 --2- 172.21.15.62:0/3016606547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33a00aed00 0x7f33a0154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 >> 172.21.15.62:0/3016606547 conn(0x7f33a00668f0 msgr2=0x7f33a015d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 shutdown_connections 2024-04-25T09:03:50.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.041+0000 7f33a6200640 1 -- 172.21.15.62:0/3016606547 wait complete. 2024-04-25T09:03:50.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:50.050 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:50.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.276+0000 7fe97ce00640 1 Processor -- start 2024-04-25T09:03:50.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.276+0000 7fe97ce00640 1 -- start start 2024-04-25T09:03:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.278+0000 7fe97ce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe97815bdf0 0x7fe9780720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.278+0000 7fe97ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9780cebc0 0x7fe97806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.278+0000 7fe97ce00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97806d070 con 0x7fe97806c7e0 2024-04-25T09:03:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.278+0000 7fe97ce00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97815c1d0 con 0x7fe97815bdf0 2024-04-25T09:03:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.278+0000 7fe97ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780ae8c0 con 0x7fe9780cebc0 2024-04-25T09:03:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.279+0000 7fe976800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe97806c7e0 0x7fe9780cd960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50404/0 (socket says 172.21.15.62:50404) 2024-04-25T09:03:50.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.279+0000 7fe976800640 1 -- 172.21.15.62:0/2585618978 learned_addr learned my addr 172.21.15.62:0/2585618978 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.279+0000 7fe975400640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9780cebc0 0x7fe97806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.279+0000 7fe975e00640 1 --2- 172.21.15.62:0/2585618978 >> v2:172.21.15.121:3300/0 conn(0x7fe97815bdf0 0x7fe9780720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.279+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524677218 0 0) 0x7fe97806d070 con 0x7fe97806c7e0 2024-04-25T09:03:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.279+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe95c003520 con 0x7fe97806c7e0 2024-04-25T09:03:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.279+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654125485 0 0) 0x7fe95c003520 con 0x7fe97806c7e0 2024-04-25T09:03:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9780cebc0 msgr2=0x7fe97806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9780cebc0 0x7fe97806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 >> v2:172.21.15.121:3300/0 conn(0x7fe97815bdf0 msgr2=0x7fe9780720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 --2- 172.21.15.62:0/2585618978 >> v2:172.21.15.121:3300/0 conn(0x7fe97815bdf0 0x7fe9780720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97806d350 con 0x7fe97806c7e0 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fe9700019f0 con 0x7fe97806c7e0 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 >> v1:172.21.15.62:6789/0 conn(0x7fe97806c7e0 legacy=0x7fe9780cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe95c0039b0 0x7fe95c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe95c004800 0x7fe95c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0051c0 0x7fe95c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97815c1d0 con 0x7fe95c0039b0 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780ae8c0 con 0x7fe95c004800 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c00bf30 con 0x7fe95c0051c0 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe975400640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9780cebc0 0x7fe97806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe975e00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe95c0039b0 0x7fe95c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe976800640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0051c0 0x7fe95c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe975e00640 1 -- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0051c0 msgr2=0x7fe95c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe975e00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0051c0 0x7fe95c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe975e00640 1 -- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe95c004800 msgr2=0x7fe95c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe975e00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe95c004800 0x7fe95c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.280+0000 7fe975e00640 1 -- 172.21.15.62:0/2585618978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c003520 con 0x7fe95c0039b0 2024-04-25T09:03:50.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.281+0000 7fe975e00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe95c0039b0 0x7fe95c003dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe968009ac0 tx=0x7fe95c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a802a70bec57b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968036070 con 0x7fe95c0039b0 2024-04-25T09:03:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe968031400 con 0x7fe95c0039b0 2024-04-25T09:03:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe974a00640 1 -- 172.21.15.62:0/2585618978 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968009690 con 0x7fe95c0039b0 2024-04-25T09:03:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe97ce00640 1 -- 172.21.15.62:0/2585618978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe95c0039b0 msgr2=0x7fe95c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe97ce00640 1 --2- 172.21.15.62:0/2585618978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe95c0039b0 0x7fe95c003dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe968009ac0 tx=0x7fe95c005830 comp rx=0 tx=0).stop 2024-04-25T09:03:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe976800640 1 -- 172.21.15.62:0/2585618978 reap_dead start 2024-04-25T09:03:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe97ce00640 1 -- 172.21.15.62:0/2585618978 shutdown_connections 2024-04-25T09:03:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe97ce00640 1 -- 172.21.15.62:0/2585618978 >> 172.21.15.62:0/2585618978 conn(0x7fe9780b1410 msgr2=0x7fe9780b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.282+0000 7fe97ce00640 1 -- 172.21.15.62:0/2585618978 shutdown_connections 2024-04-25T09:03:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.283+0000 7fe97ce00640 1 -- 172.21.15.62:0/2585618978 wait complete. 2024-04-25T09:03:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.284+0000 7fe97ce00640 1 Processor -- start 2024-04-25T09:03:50.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.284+0000 7fe97ce00640 1 -- start start 2024-04-25T09:03:50.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.285+0000 7fe97ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0039b0 0x7fe9781e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.285+0000 7fe97ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9781f2620 0x7fe9781579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.285+0000 7fe97ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe978157f00 0x7fe97815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.285+0000 7fe97ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97806cc90 con 0x7fe9781f2620 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.285+0000 7fe97ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780aedf0 con 0x7fe978157f00 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.285+0000 7fe97ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97806ce10 con 0x7fe95c0039b0 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9781f2620 0x7fe9781579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9781f2620 0x7fe9781579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34092/0 (socket says 172.21.15.62:34092) 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 -- 172.21.15.62:0/2815854933 learned_addr learned my addr 172.21.15.62:0/2815854933 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975e00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0039b0 0x7fe9781e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe976800640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe978157f00 0x7fe97815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 -- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0039b0 msgr2=0x7fe9781e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0039b0 0x7fe9781e7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 -- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe978157f00 msgr2=0x7fe97815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe978157f00 0x7fe97815dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 -- 172.21.15.62:0/2815854933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97815c480 con 0x7fe9781f2620 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975e00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0039b0 0x7fe9781e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.286+0000 7fe975400640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9781f2620 0x7fe9781579c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe96000e590 tx=0x7fe96000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.287+0000 7fe96e000640 1 -- 172.21.15.62:0/2815854933 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960017070 con 0x7fe9781f2620 2024-04-25T09:03:50.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.287+0000 7fe96e000640 1 -- 172.21.15.62:0/2815854933 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960013610 con 0x7fe9781f2620 2024-04-25T09:03:50.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.287+0000 7fe96e000640 1 -- 172.21.15.62:0/2815854933 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000f040 con 0x7fe9781f2620 2024-04-25T09:03:50.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.287+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97815c760 con 0x7fe9781f2620 2024-04-25T09:03:50.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.287+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97815cd20 con 0x7fe9781f2620 2024-04-25T09:03:50.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.288+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9781582e0 con 0x7fe9781f2620 2024-04-25T09:03:50.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.289+0000 7fe96e000640 1 -- 172.21.15.62:0/2815854933 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe96001a070 con 0x7fe9781f2620 2024-04-25T09:03:50.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.290+0000 7fe96e000640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe95406b4d0 0x7fe95406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.290+0000 7fe96e000640 1 -- 172.21.15.62:0/2815854933 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe960089d60 con 0x7fe9781f2620 2024-04-25T09:03:50.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.291+0000 7fe975e00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe95406b4d0 0x7fe95406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.292+0000 7fe975e00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe95406b4d0 0x7fe95406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe968033000 tx=0x7fe968008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.294+0000 7fe96e000640 1 -- 172.21.15.62:0/2815854933 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe960058bd0 con 0x7fe9781f2620 2024-04-25T09:03:50.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.491+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe97815d210 con 0x7fe9781f2620 2024-04-25T09:03:50.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.492+0000 7fe96e000640 1 -- 172.21.15.62:0/2815854933 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9600584d0 con 0x7fe9781f2620 2024-04-25T09:03:50.481 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:50.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe95406b4d0 msgr2=0x7fe95406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe95406b4d0 0x7fe95406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe968033000 tx=0x7fe968008030 comp rx=0 tx=0).stop 2024-04-25T09:03:50.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9781f2620 msgr2=0x7fe9781579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9781f2620 0x7fe9781579c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe96000e590 tx=0x7fe96000ea50 comp rx=0 tx=0).stop 2024-04-25T09:03:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 shutdown_connections 2024-04-25T09:03:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe95406b4d0 0x7fe95406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9781f2620 0x7fe9781579c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe978157f00 0x7fe97815dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 --2- 172.21.15.62:0/2815854933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe95c0039b0 0x7fe9781e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 >> 172.21.15.62:0/2815854933 conn(0x7fe9780b1410 msgr2=0x7fe97815a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 shutdown_connections 2024-04-25T09:03:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.495+0000 7fe97ce00640 1 -- 172.21.15.62:0/2815854933 wait complete. 2024-04-25T09:03:50.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:50.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.733+0000 7fdd43200640 1 Processor -- start 2024-04-25T09:03:50.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.734+0000 7fdd43200640 1 -- start start 2024-04-25T09:03:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.735+0000 7fdd43200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 0x7fdd440ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.735+0000 7fdd43200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdd4406d6a0 0x7fdd440ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.735+0000 7fdd43200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd440b31a0 0x7fdd440b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.735+0000 7fdd43200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd440ced40 con 0x7fdd440b31a0 2024-04-25T09:03:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.735+0000 7fdd43200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd440cf000 con 0x7fdd4406d6a0 2024-04-25T09:03:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.735+0000 7fdd43200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440cf2e0 con 0x7fdd4406d2c0 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd42800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd440b31a0 0x7fdd440b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd42800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd440b31a0 0x7fdd440b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34104/0 (socket says 172.21.15.62:34104) 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd42800640 1 -- 172.21.15.62:0/1602707999 learned_addr learned my addr 172.21.15.62:0/1602707999 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd41400640 1 --2- 172.21.15.62:0/1602707999 >> v2:172.21.15.121:3300/0 conn(0x7fdd4406d6a0 0x7fdd440ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd41e00640 1 --2- 172.21.15.62:0/1602707999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 0x7fdd440ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd42800640 1 -- 172.21.15.62:0/1602707999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 msgr2=0x7fdd440ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd42800640 1 --2- 172.21.15.62:0/1602707999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 0x7fdd440ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd42800640 1 -- 172.21.15.62:0/1602707999 >> v2:172.21.15.121:3300/0 conn(0x7fdd4406d6a0 msgr2=0x7fdd440ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd42800640 1 --2- 172.21.15.62:0/1602707999 >> v2:172.21.15.121:3300/0 conn(0x7fdd4406d6a0 0x7fdd440ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.736+0000 7fdd42800640 1 -- 172.21.15.62:0/1602707999 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd440b26c0 con 0x7fdd440b31a0 2024-04-25T09:03:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.738+0000 7fdd42800640 1 --2- 172.21.15.62:0/1602707999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd440b31a0 0x7fdd440b3f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b2330 tx=0x7fdd3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec7737975c0fc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.738+0000 7fdd40a00640 1 -- 172.21.15.62:0/1602707999 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c036070 con 0x7fdd440b31a0 2024-04-25T09:03:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.738+0000 7fdd40a00640 1 -- 172.21.15.62:0/1602707999 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c031600 con 0x7fdd440b31a0 2024-04-25T09:03:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.738+0000 7fdd40a00640 1 -- 172.21.15.62:0/1602707999 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0319d0 con 0x7fdd440b31a0 2024-04-25T09:03:50.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 -- 172.21.15.62:0/1602707999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd440b31a0 msgr2=0x7fdd440b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 --2- 172.21.15.62:0/1602707999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd440b31a0 0x7fdd440b3f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b2330 tx=0x7fdd3c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 -- 172.21.15.62:0/1602707999 shutdown_connections 2024-04-25T09:03:50.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 --2- 172.21.15.62:0/1602707999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd440b31a0 0x7fdd440b3f40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 --2- 172.21.15.62:0/1602707999 >> v2:172.21.15.121:3300/0 conn(0x7fdd4406d6a0 0x7fdd440ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 --2- 172.21.15.62:0/1602707999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 0x7fdd440ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 -- 172.21.15.62:0/1602707999 >> 172.21.15.62:0/1602707999 conn(0x7fdd44066f20 msgr2=0x7fdd440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 -- 172.21.15.62:0/1602707999 shutdown_connections 2024-04-25T09:03:50.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.739+0000 7fdd43200640 1 -- 172.21.15.62:0/1602707999 wait complete. 2024-04-25T09:03:50.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.741+0000 7fdd43200640 1 Processor -- start 2024-04-25T09:03:50.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.741+0000 7fdd43200640 1 -- start start 2024-04-25T09:03:50.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd43200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 0x7fdd440ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd43200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4406d6a0 0x7fdd441e8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd43200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd440b31a0 0x7fdd4415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd43200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4406db20 con 0x7fdd4406d6a0 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd43200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440b2c80 con 0x7fdd440b31a0 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd43200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd441448d0 con 0x7fdd4406d2c0 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd41400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4406d6a0 0x7fdd441e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd41400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4406d6a0 0x7fdd441e8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34110/0 (socket says 172.21.15.62:34110) 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.742+0000 7fdd41400640 1 -- 172.21.15.62:0/1411526935 learned_addr learned my addr 172.21.15.62:0/1411526935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd42800640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd440b31a0 0x7fdd4415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd41e00640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 0x7fdd440ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd41400640 1 -- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 msgr2=0x7fdd440ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd41400640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 0x7fdd440ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd41400640 1 -- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd440b31a0 msgr2=0x7fdd4415dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd41400640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd440b31a0 0x7fdd4415dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd41400640 1 -- 172.21.15.62:0/1411526935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd441f74c0 con 0x7fdd4406d6a0 2024-04-25T09:03:50.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd41400640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4406d6a0 0x7fdd441e8830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000d5f0 tx=0x7fdd3000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.743+0000 7fdd3a000640 1 -- 172.21.15.62:0/1411526935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd300125f0 con 0x7fdd4406d6a0 2024-04-25T09:03:50.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.744+0000 7fdd3a000640 1 -- 172.21.15.62:0/1411526935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd30019070 con 0x7fdd4406d6a0 2024-04-25T09:03:50.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.744+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd441f77a0 con 0x7fdd4406d6a0 2024-04-25T09:03:50.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.744+0000 7fdd3a000640 1 -- 172.21.15.62:0/1411526935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd300129d0 con 0x7fdd4406d6a0 2024-04-25T09:03:50.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.744+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd441f7d60 con 0x7fdd4406d6a0 2024-04-25T09:03:50.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.745+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4406abe0 con 0x7fdd4406d6a0 2024-04-25T09:03:50.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.746+0000 7fdd3a000640 1 -- 172.21.15.62:0/1411526935 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd30012b70 con 0x7fdd4406d6a0 2024-04-25T09:03:50.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.747+0000 7fdd3a000640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b500 0x7fdd2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:50.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.747+0000 7fdd3a000640 1 -- 172.21.15.62:0/1411526935 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd3008a5e0 con 0x7fdd4406d6a0 2024-04-25T09:03:50.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.748+0000 7fdd41e00640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b500 0x7fdd2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.749+0000 7fdd41e00640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b500 0x7fdd2406d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd34006fd0 tx=0x7fdd34008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:50.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.751+0000 7fdd3a000640 1 -- 172.21.15.62:0/1411526935 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd30059450 con 0x7fdd4406d6a0 2024-04-25T09:03:50.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.955+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd4404ef20 con 0x7fdd4406d6a0 2024-04-25T09:03:50.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.955+0000 7fdd3a000640 1 -- 172.21.15.62:0/1411526935 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd30058d50 con 0x7fdd4406d6a0 2024-04-25T09:03:50.945 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b500 msgr2=0x7fdd2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b500 0x7fdd2406d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd34006fd0 tx=0x7fdd34008040 comp rx=0 tx=0).stop 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4406d6a0 msgr2=0x7fdd441e8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4406d6a0 0x7fdd441e8830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000d5f0 tx=0x7fdd3000dab0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 shutdown_connections 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b500 0x7fdd2406d9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4406d6a0 0x7fdd441e8830 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd440b31a0 0x7fdd4415dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.958+0000 7fdd43200640 1 --2- 172.21.15.62:0/1411526935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd4406d2c0 0x7fdd440ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.959+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 >> 172.21.15.62:0/1411526935 conn(0x7fdd44066f20 msgr2=0x7fdd440afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:50.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.959+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 shutdown_connections 2024-04-25T09:03:50.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:50.959+0000 7fdd43200640 1 -- 172.21.15.62:0/1411526935 wait complete. 2024-04-25T09:03:50.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:51.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.196+0000 7f6946e00640 1 Processor -- start 2024-04-25T09:03:51.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.196+0000 7f6946e00640 1 -- start start 2024-04-25T09:03:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.197+0000 7f6946e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 0x7f69400b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.197+0000 7f6946e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f69400ccdb0 0x7f69400cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6946e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69400d0490 0x7f69400cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6946e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400b4240 con 0x7f69400d0490 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6946e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400cd650 con 0x7f69400ccdb0 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6946e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69400cd370 con 0x7f694015b8f0 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6946400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69400d0490 0x7f69400cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6946400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69400d0490 0x7f69400cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34116/0 (socket says 172.21.15.62:34116) 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6946400640 1 -- 172.21.15.62:0/1309972264 learned_addr learned my addr 172.21.15.62:0/1309972264 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6945000640 1 --2- 172.21.15.62:0/1309972264 >> v2:172.21.15.121:3300/0 conn(0x7f69400ccdb0 0x7f69400cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.198+0000 7f6945a00640 1 --2- 172.21.15.62:0/1309972264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 0x7f69400b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.199+0000 7f6946400640 1 -- 172.21.15.62:0/1309972264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 msgr2=0x7f69400b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.199+0000 7f6946400640 1 --2- 172.21.15.62:0/1309972264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 0x7f69400b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.199+0000 7f6946400640 1 -- 172.21.15.62:0/1309972264 >> v2:172.21.15.121:3300/0 conn(0x7f69400ccdb0 msgr2=0x7f69400cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.199+0000 7f6946400640 1 --2- 172.21.15.62:0/1309972264 >> v2:172.21.15.121:3300/0 conn(0x7f69400ccdb0 0x7f69400cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.199+0000 7f6946400640 1 -- 172.21.15.62:0/1309972264 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400cf470 con 0x7f69400d0490 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.200+0000 7f6946400640 1 --2- 172.21.15.62:0/1309972264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400d0490 0x7f69400cde40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6938009e50 tx=0x7f693802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d23e94c8cc59fde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.200+0000 7f693fe00640 1 -- 172.21.15.62:0/1309972264 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938036070 con 0x7f69400d0490 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.200+0000 7f693fe00640 1 -- 172.21.15.62:0/1309972264 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6938031600 con 0x7f69400d0490 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.200+0000 7f693fe00640 1 -- 172.21.15.62:0/1309972264 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69380319d0 con 0x7f69400d0490 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 -- 172.21.15.62:0/1309972264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400d0490 msgr2=0x7f69400cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 --2- 172.21.15.62:0/1309972264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400d0490 0x7f69400cde40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6938009e50 tx=0x7f693802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 -- 172.21.15.62:0/1309972264 shutdown_connections 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 --2- 172.21.15.62:0/1309972264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400d0490 0x7f69400cde40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 --2- 172.21.15.62:0/1309972264 >> v2:172.21.15.121:3300/0 conn(0x7f69400ccdb0 0x7f69400cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 --2- 172.21.15.62:0/1309972264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 0x7f69400b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 -- 172.21.15.62:0/1309972264 >> 172.21.15.62:0/1309972264 conn(0x7f694009aa70 msgr2=0x7f69400b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 -- 172.21.15.62:0/1309972264 shutdown_connections 2024-04-25T09:03:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.201+0000 7f6946e00640 1 -- 172.21.15.62:0/1309972264 wait complete. 2024-04-25T09:03:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.203+0000 7f6946e00640 1 Processor -- start 2024-04-25T09:03:51.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.203+0000 7f6946e00640 1 -- start start 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6946e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400ccdb0 0x7f69400b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6946e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69400d0490 0x7f694015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6946e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 0x7f69400b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6946e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694015bcd0 con 0x7f69400ccdb0 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6946e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69400cfa30 con 0x7f69400d0490 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6946e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69400ce380 con 0x7f694015b8f0 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6945a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400ccdb0 0x7f69400b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6945a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400ccdb0 0x7f69400b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34120/0 (socket says 172.21.15.62:34120) 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.204+0000 7f6945a00640 1 -- 172.21.15.62:0/2351637368 learned_addr learned my addr 172.21.15.62:0/2351637368 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6945000640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69400d0490 0x7f694015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6946400640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 0x7f69400b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6945a00640 1 -- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 msgr2=0x7f69400b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6945a00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 0x7f69400b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6945a00640 1 -- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69400d0490 msgr2=0x7f694015c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6945a00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69400d0490 0x7f694015c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6945a00640 1 -- 172.21.15.62:0/2351637368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400b4950 con 0x7f69400ccdb0 2024-04-25T09:03:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6945000640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69400d0490 0x7f694015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6945a00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400ccdb0 0x7f69400b6520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f693400d620 tx=0x7f693400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f693d600640 1 -- 172.21.15.62:0/2351637368 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693400b620 con 0x7f69400ccdb0 2024-04-25T09:03:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f693d600640 1 -- 172.21.15.62:0/2351637368 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6934019070 con 0x7f69400ccdb0 2024-04-25T09:03:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69400b5490 con 0x7f69400ccdb0 2024-04-25T09:03:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.205+0000 7f693d600640 1 -- 172.21.15.62:0/2351637368 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693400ba00 con 0x7f69400ccdb0 2024-04-25T09:03:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.206+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69400b4ed0 con 0x7f69400ccdb0 2024-04-25T09:03:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.207+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69400b2a80 con 0x7f69400ccdb0 2024-04-25T09:03:51.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.208+0000 7f693d600640 1 -- 172.21.15.62:0/2351637368 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f693400bba0 con 0x7f69400ccdb0 2024-04-25T09:03:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.209+0000 7f693d600640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.209+0000 7f693d600640 1 -- 172.21.15.62:0/2351637368 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f693408a620 con 0x7f69400ccdb0 2024-04-25T09:03:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.210+0000 7f6945000640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.211+0000 7f6945000640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692406b4d0 0x7f692406d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f694015ea50 tx=0x7f6930008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.213+0000 7f693d600640 1 -- 172.21.15.62:0/2351637368 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69400b2a80 con 0x7f69400ccdb0 2024-04-25T09:03:51.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.412+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69400750e0 con 0x7f69400ccdb0 2024-04-25T09:03:51.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.413+0000 7f693d600640 1 -- 172.21.15.62:0/2351637368 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6934059400 con 0x7f69400ccdb0 2024-04-25T09:03:51.402 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:03:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692406b4d0 msgr2=0x7f692406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692406b4d0 0x7f692406d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f694015ea50 tx=0x7f6930008040 comp rx=0 tx=0).stop 2024-04-25T09:03:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400ccdb0 msgr2=0x7f69400b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400ccdb0 0x7f69400b6520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f693400d620 tx=0x7f693400dae0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 shutdown_connections 2024-04-25T09:03:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694015b8f0 0x7f69400b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69400d0490 0x7f694015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 --2- 172.21.15.62:0/2351637368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69400ccdb0 0x7f69400b6520 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 >> 172.21.15.62:0/2351637368 conn(0x7f694009aa70 msgr2=0x7f69400b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 shutdown_connections 2024-04-25T09:03:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.416+0000 7f6946e00640 1 -- 172.21.15.62:0/2351637368 wait complete. 2024-04-25T09:03:51.424 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.645+0000 7f681a200640 1 Processor -- start 2024-04-25T09:03:51.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.645+0000 7f681a200640 1 -- start start 2024-04-25T09:03:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.647+0000 7f681a200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f681415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.647+0000 7f681a200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f68140cee40 0x7f681406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.647+0000 7f681a200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681415faa0 con 0x7f681406d730 2024-04-25T09:03:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.647+0000 7f681a200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6814144ff0 con 0x7f68140cee40 2024-04-25T09:03:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.647+0000 7f681a200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6814160340 con 0x7f6814144c10 2024-04-25T09:03:51.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.648+0000 7f6818e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f681415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.648+0000 7f680be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f68140cee40 0x7f681406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.648+0000 7f6818e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f681415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:50182/0 (socket says 172.21.15.62:50182) 2024-04-25T09:03:51.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.648+0000 7f6818e00640 1 -- 172.21.15.62:0/2516083562 learned_addr learned my addr 172.21.15.62:0/2516083562 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:51.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.649+0000 7f680be00640 1 -- 172.21.15.62:0/2516083562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 msgr2=0x7f681415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.649+0000 7f680be00640 1 --2- 172.21.15.62:0/2516083562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f681415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.649+0000 7f680be00640 1 -- 172.21.15.62:0/2516083562 >> v1:172.21.15.121:6789/0 conn(0x7f681406d730 legacy=0x7f68140b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.649+0000 7f680be00640 1 -- 172.21.15.62:0/2516083562 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681415fd80 con 0x7f68140cee40 2024-04-25T09:03:51.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.650+0000 7f680be00640 1 --2- 172.21.15.62:0/2516083562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68140cee40 0x7f681406d1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0097a0 tx=0x7f67fc02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddad3bb839a620b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.651+0000 7f6813e00640 1 -- 172.21.15.62:0/2516083562 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc037070 con 0x7f68140cee40 2024-04-25T09:03:51.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.651+0000 7f6813e00640 1 -- 172.21.15.62:0/2516083562 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc02ae40 con 0x7f68140cee40 2024-04-25T09:03:51.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.651+0000 7f6813e00640 1 -- 172.21.15.62:0/2516083562 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc032660 con 0x7f68140cee40 2024-04-25T09:03:51.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.651+0000 7f681a200640 1 -- 172.21.15.62:0/2516083562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68140cee40 msgr2=0x7f681406d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.651+0000 7f681a200640 1 --2- 172.21.15.62:0/2516083562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68140cee40 0x7f681406d1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0097a0 tx=0x7f67fc02a0f0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.652+0000 7f681a200640 1 -- 172.21.15.62:0/2516083562 shutdown_connections 2024-04-25T09:03:51.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.652+0000 7f681a200640 1 --2- 172.21.15.62:0/2516083562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68140cee40 0x7f681406d1f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.652+0000 7f681a200640 1 --2- 172.21.15.62:0/2516083562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f681415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.652+0000 7f681a200640 1 -- 172.21.15.62:0/2516083562 >> 172.21.15.62:0/2516083562 conn(0x7f68140b14d0 msgr2=0x7f68140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:51.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.652+0000 7f681a200640 1 -- 172.21.15.62:0/2516083562 shutdown_connections 2024-04-25T09:03:51.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.652+0000 7f681a200640 1 -- 172.21.15.62:0/2516083562 wait complete. 2024-04-25T09:03:51.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.653+0000 7f681a200640 1 Processor -- start 2024-04-25T09:03:51.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.654+0000 7f681a200640 1 -- start start 2024-04-25T09:03:51.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f681a200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f681406d730 0x7f68141efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f681a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68140cee40 0x7f681414eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f681a200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f68140c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f681a200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68140cf2a0 con 0x7f68140cee40 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f681a200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68140b41c0 con 0x7f681406d730 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f681a200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68140b5660 con 0x7f6814144c10 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f680be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68140cee40 0x7f681414eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f680be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68140cee40 0x7f681414eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34126/0 (socket says 172.21.15.62:34126) 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f680be00640 1 -- 172.21.15.62:0/430692952 learned_addr learned my addr 172.21.15.62:0/430692952 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f6818e00640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f681406d730 0x7f68141efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.655+0000 7f6819800640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f68140c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f680be00640 1 -- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 msgr2=0x7f68140c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f680be00640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f68140c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f680be00640 1 -- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f681406d730 msgr2=0x7f68141efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f680be00640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f681406d730 0x7f68141efac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f680be00640 1 -- 172.21.15.62:0/430692952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6814142730 con 0x7f68140cee40 2024-04-25T09:03:51.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f680be00640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68140cee40 0x7f681414eaf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0098c0 tx=0x7f67fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f6811600640 1 -- 172.21.15.62:0/430692952 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc041030 con 0x7f68140cee40 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f6811600640 1 -- 172.21.15.62:0/430692952 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc02ac40 con 0x7f68140cee40 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6814142a10 con 0x7f68140cee40 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.656+0000 7f6811600640 1 -- 172.21.15.62:0/430692952 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc037040 con 0x7f68140cee40 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.657+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6814142fd0 con 0x7f68140cee40 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.658+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681404ef20 con 0x7f68140cee40 2024-04-25T09:03:51.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.659+0000 7f6811600640 1 -- 172.21.15.62:0/430692952 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67fc044070 con 0x7f68140cee40 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.660+0000 7f6811600640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67f006b4d0 0x7f67f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.660+0000 7f6811600640 1 -- 172.21.15.62:0/430692952 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67fc0aa990 con 0x7f68140cee40 2024-04-25T09:03:51.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.661+0000 7f6818e00640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67f006b4d0 0x7f67f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.662+0000 7f6818e00640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67f006b4d0 0x7f67f006d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6804009ce0 tx=0x7f680400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.665+0000 7f6811600640 1 -- 172.21.15.62:0/430692952 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67fc044e40 con 0x7f68140cee40 2024-04-25T09:03:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.870+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f681413dd10 con 0x7f68140cee40 2024-04-25T09:03:51.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.872+0000 7f6811600640 1 -- 172.21.15.62:0/430692952 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f67fc079890 con 0x7f68140cee40 2024-04-25T09:03:51.862 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:51.862 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67f006b4d0 msgr2=0x7f67f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67f006b4d0 0x7f67f006d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6804009ce0 tx=0x7f680400b040 comp rx=0 tx=0).stop 2024-04-25T09:03:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68140cee40 msgr2=0x7f681414eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68140cee40 0x7f681414eaf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0098c0 tx=0x7f67fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 shutdown_connections 2024-04-25T09:03:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67f006b4d0 0x7f67f006d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68140cee40 0x7f681414eaf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6814144c10 0x7f68140c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 --2- 172.21.15.62:0/430692952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f681406d730 0x7f68141efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 >> 172.21.15.62:0/430692952 conn(0x7f68140b14d0 msgr2=0x7f68140b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:51.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 shutdown_connections 2024-04-25T09:03:51.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:51.875+0000 7f681a200640 1 -- 172.21.15.62:0/430692952 wait complete. 2024-04-25T09:03:51.883 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:51.884 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:52.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.115+0000 7f096fe00640 1 Processor -- start 2024-04-25T09:03:52.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.116+0000 7f096fe00640 1 -- start start 2024-04-25T09:03:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.117+0000 7f096fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09700d2fb0 0x7f09700ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.117+0000 7f096fe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09700ce780 con 0x7f09700d3960 2024-04-25T09:03:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.117+0000 7f096fe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09700d3440 con 0x7f097015dd50 2024-04-25T09:03:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.117+0000 7f096fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09700ce1c0 con 0x7f09700d2fb0 2024-04-25T09:03:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.117+0000 7f096f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f09700d3960 0x7f09700d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50408/0 (socket says 172.21.15.62:50408) 2024-04-25T09:03:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.117+0000 7f096f400640 1 -- 172.21.15.62:0/3008590229 learned_addr learned my addr 172.21.15.62:0/3008590229 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:52.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.117+0000 7f096e000640 1 --2- 172.21.15.62:0/3008590229 >> v2:172.21.15.62:3300/0 conn(0x7f09700d2fb0 0x7f09700ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.118+0000 7f096d600640 1 -- 172.21.15.62:0/3008590229 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516724985 0 0) 0x7f09700ce780 con 0x7f09700d3960 2024-04-25T09:03:52.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.118+0000 7f096d600640 1 -- 172.21.15.62:0/3008590229 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0950003520 con 0x7f09700d3960 2024-04-25T09:03:52.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.118+0000 7f096e000640 1 -- 172.21.15.62:0/3008590229 >> v1:172.21.15.121:6789/0 conn(0x7f097015dd50 legacy=0x7f09700d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.118+0000 7f096e000640 1 -- 172.21.15.62:0/3008590229 >> v1:172.21.15.62:6789/0 conn(0x7f09700d3960 legacy=0x7f09700d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.118+0000 7f096e000640 1 -- 172.21.15.62:0/3008590229 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700ce4a0 con 0x7f09700d2fb0 2024-04-25T09:03:52.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.119+0000 7f096e000640 1 --2- 172.21.15.62:0/3008590229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d2fb0 0x7f09700ce9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0958009d90 tx=0x7f095802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b025a9203084827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.119+0000 7f096d600640 1 -- 172.21.15.62:0/3008590229 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958007740 con 0x7f09700d2fb0 2024-04-25T09:03:52.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.119+0000 7f096d600640 1 -- 172.21.15.62:0/3008590229 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09580078e0 con 0x7f09700d2fb0 2024-04-25T09:03:52.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.119+0000 7f096d600640 1 -- 172.21.15.62:0/3008590229 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958007cb0 con 0x7f09700d2fb0 2024-04-25T09:03:52.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.120+0000 7f096fe00640 1 -- 172.21.15.62:0/3008590229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d2fb0 msgr2=0x7f09700ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.120+0000 7f096fe00640 1 --2- 172.21.15.62:0/3008590229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d2fb0 0x7f09700ce9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0958009d90 tx=0x7f095802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.120+0000 7f096fe00640 1 -- 172.21.15.62:0/3008590229 shutdown_connections 2024-04-25T09:03:52.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.120+0000 7f096fe00640 1 --2- 172.21.15.62:0/3008590229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d2fb0 0x7f09700ce9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.120+0000 7f096fe00640 1 -- 172.21.15.62:0/3008590229 >> 172.21.15.62:0/3008590229 conn(0x7f097006b000 msgr2=0x7f097006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.120+0000 7f096fe00640 1 -- 172.21.15.62:0/3008590229 shutdown_connections 2024-04-25T09:03:52.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.120+0000 7f096fe00640 1 -- 172.21.15.62:0/3008590229 wait complete. 2024-04-25T09:03:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.122+0000 7f096fe00640 1 Processor -- start 2024-04-25T09:03:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.122+0000 7f096fe00640 1 -- start start 2024-04-25T09:03:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d3960 0x7f0970160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097015dd50 0x7f09701e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09701f3190 0x7f097014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700b6c80 con 0x7f09700d3960 2024-04-25T09:03:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09700b6b00 con 0x7f097015dd50 2024-04-25T09:03:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09700b6e00 con 0x7f09701f3190 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d3960 0x7f0970160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d3960 0x7f0970160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34136/0 (socket says 172.21.15.62:34136) 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.123+0000 7f096ea00640 1 -- 172.21.15.62:0/1675725686 learned_addr learned my addr 172.21.15.62:0/1675725686 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096e000640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097015dd50 0x7f09701e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096f400640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09701f3190 0x7f097014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096ea00640 1 -- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09701f3190 msgr2=0x7f097014a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096ea00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09701f3190 0x7f097014a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096ea00640 1 -- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097015dd50 msgr2=0x7f09701e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096ea00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097015dd50 0x7f09701e6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096ea00640 1 -- 172.21.15.62:0/1675725686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097014d450 con 0x7f09700d3960 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096e000640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097015dd50 0x7f09701e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096f400640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09701f3190 0x7f097014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096ea00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d3960 0x7f0970160b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f096000c400 tx=0x7f096000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f0966a00640 1 -- 172.21.15.62:0/1675725686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960017070 con 0x7f09700d3960 2024-04-25T09:03:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f0966a00640 1 -- 172.21.15.62:0/1675725686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09600052d0 con 0x7f09700d3960 2024-04-25T09:03:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f0966a00640 1 -- 172.21.15.62:0/1675725686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960003830 con 0x7f09700d3960 2024-04-25T09:03:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097015fad0 con 0x7f09700d3960 2024-04-25T09:03:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.124+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0970160090 con 0x7f09700d3960 2024-04-25T09:03:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.125+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097004ef20 con 0x7f09700d3960 2024-04-25T09:03:52.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.126+0000 7f0966a00640 1 -- 172.21.15.62:0/1675725686 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f096001a070 con 0x7f09700d3960 2024-04-25T09:03:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.127+0000 7f0966a00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094c06b4d0 0x7f094c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.127+0000 7f0966a00640 1 -- 172.21.15.62:0/1675725686 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0960089640 con 0x7f09700d3960 2024-04-25T09:03:52.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.128+0000 7f096e000640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094c06b4d0 0x7f094c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.129+0000 7f096e000640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094c06b4d0 0x7f094c06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0958034000 tx=0x7f095802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.132+0000 7f0966a00640 1 -- 172.21.15.62:0/1675725686 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0960058420 con 0x7f09700d3960 2024-04-25T09:03:52.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.329+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09700b9950 con 0x7f09700d3960 2024-04-25T09:03:52.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.330+0000 7f0966a00640 1 -- 172.21.15.62:0/1675725686 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0960057d20 con 0x7f09700d3960 2024-04-25T09:03:52.319 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:52.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094c06b4d0 msgr2=0x7f094c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094c06b4d0 0x7f094c06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0958034000 tx=0x7f095802f000 comp rx=0 tx=0).stop 2024-04-25T09:03:52.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d3960 msgr2=0x7f0970160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d3960 0x7f0970160b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f096000c400 tx=0x7f096000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 shutdown_connections 2024-04-25T09:03:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094c06b4d0 0x7f094c06d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09701f3190 0x7f097014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097015dd50 0x7f09701e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 --2- 172.21.15.62:0/1675725686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09700d3960 0x7f0970160b70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 >> 172.21.15.62:0/1675725686 conn(0x7f097006b000 msgr2=0x7f09700d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 shutdown_connections 2024-04-25T09:03:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.333+0000 7f096fe00640 1 -- 172.21.15.62:0/1675725686 wait complete. 2024-04-25T09:03:52.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:52.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.572+0000 7ff74d000640 1 Processor -- start 2024-04-25T09:03:52.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.572+0000 7ff74d000640 1 -- start start 2024-04-25T09:03:52.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.574+0000 7ff74d000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7480b2d80 0x7ff7480b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.574+0000 7ff74d000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7480b4430 con 0x7ff7480b36a0 2024-04-25T09:03:52.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.574+0000 7ff74d000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7480b4710 con 0x7ff7480b54d0 2024-04-25T09:03:52.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.574+0000 7ff74d000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7480b49f0 con 0x7ff7480b2d80 2024-04-25T09:03:52.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.574+0000 7ff73fe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff7480b36a0 0x7ff7480ce2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50412/0 (socket says 172.21.15.62:50412) 2024-04-25T09:03:52.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.574+0000 7ff73fe00640 1 -- 172.21.15.62:0/878078299 learned_addr learned my addr 172.21.15.62:0/878078299 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:52.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.574+0000 7ff747400640 1 --2- 172.21.15.62:0/878078299 >> v2:172.21.15.121:3300/0 conn(0x7ff7480b2d80 0x7ff7480b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543277843 0 0) 0x7ff7480b4430 con 0x7ff7480b36a0 2024-04-25T09:03:52.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff72c003520 con 0x7ff7480b36a0 2024-04-25T09:03:52.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 835297267 0 0) 0x7ff72c003520 con 0x7ff7480b36a0 2024-04-25T09:03:52.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 >> v2:172.21.15.121:3300/0 conn(0x7ff7480b2d80 msgr2=0x7ff7480b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 --2- 172.21.15.62:0/878078299 >> v2:172.21.15.121:3300/0 conn(0x7ff7480b2d80 0x7ff7480b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 >> v1:172.21.15.121:6789/0 conn(0x7ff7480b54d0 legacy=0x7ff748164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480b4cd0 con 0x7ff7480b36a0 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ff730001670 con 0x7ff7480b36a0 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 >> v1:172.21.15.62:6789/0 conn(0x7ff7480b36a0 legacy=0x7ff7480ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0039b0 0x7ff72c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c004800 0x7ff72c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c0051c0 0x7ff72c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480b49f0 con 0x7ff72c0051c0 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c00c040 con 0x7ff72c004800 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.575+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c00c240 con 0x7ff72c0039b0 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.576+0000 7ff747e00640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c0051c0 0x7ff72c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.576+0000 7ff747400640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0039b0 0x7ff72c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.576+0000 7ff73fe00640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c004800 0x7ff72c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.576+0000 7ff747e00640 1 -- 172.21.15.62:0/878078299 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0039b0 msgr2=0x7ff72c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.576+0000 7ff747e00640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0039b0 0x7ff72c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.576+0000 7ff747e00640 1 -- 172.21.15.62:0/878078299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c004800 msgr2=0x7ff72c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.576+0000 7ff747e00640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff72c004800 0x7ff72c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.576+0000 7ff747e00640 1 -- 172.21.15.62:0/878078299 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c003520 con 0x7ff72c0051c0 2024-04-25T09:03:52.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.577+0000 7ff747e00640 1 -- 172.21.15.62:0/878078299 reap_dead start 2024-04-25T09:03:52.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.577+0000 7ff747e00640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c0051c0 0x7ff72c00b9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7400044b0 tx=0x7ff72c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657709953e51b464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.578+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740037070 con 0x7ff72c0051c0 2024-04-25T09:03:52.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.578+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74002ad80 con 0x7ff72c0051c0 2024-04-25T09:03:52.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.578+0000 7ff746a00640 1 -- 172.21.15.62:0/878078299 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740004080 con 0x7ff72c0051c0 2024-04-25T09:03:52.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.578+0000 7ff74d000640 1 -- 172.21.15.62:0/878078299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c0051c0 msgr2=0x7ff72c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.578+0000 7ff74d000640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c0051c0 0x7ff72c00b9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7400044b0 tx=0x7ff72c005830 comp rx=0 tx=0).stop 2024-04-25T09:03:52.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.578+0000 7ff74d000640 1 -- 172.21.15.62:0/878078299 shutdown_connections 2024-04-25T09:03:52.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.578+0000 7ff74d000640 1 --2- 172.21.15.62:0/878078299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff72c0051c0 0x7ff72c00b9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.578+0000 7ff74d000640 1 -- 172.21.15.62:0/878078299 >> 172.21.15.62:0/878078299 conn(0x7ff748066f20 msgr2=0x7ff7480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.579+0000 7ff74d000640 1 -- 172.21.15.62:0/878078299 shutdown_connections 2024-04-25T09:03:52.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.579+0000 7ff74d000640 1 -- 172.21.15.62:0/878078299 wait complete. 2024-04-25T09:03:52.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.580+0000 7ff74d000640 1 Processor -- start 2024-04-25T09:03:52.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.580+0000 7ff74d000640 1 -- start start 2024-04-25T09:03:52.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.581+0000 7ff74d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0051c0 0x7ff7481541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.581+0000 7ff74d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7481ed5e0 0x7ff7481f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.581+0000 7ff74d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff748150a80 0x7ff748159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.581+0000 7ff74d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480b3160 con 0x7ff748150a80 2024-04-25T09:03:52.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.581+0000 7ff74d000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480b40e0 con 0x7ff7481ed5e0 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.581+0000 7ff74d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480b32e0 con 0x7ff72c0051c0 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff748150a80 0x7ff748159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff748150a80 0x7ff748159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34164/0 (socket says 172.21.15.62:34164) 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747e00640 1 -- 172.21.15.62:0/537590563 learned_addr learned my addr 172.21.15.62:0/537590563 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747e00640 1 -- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0051c0 msgr2=0x7ff7481541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747400640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0051c0 0x7ff7481541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff73fe00640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7481ed5e0 0x7ff7481f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747e00640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0051c0 0x7ff7481541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747e00640 1 -- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7481ed5e0 msgr2=0x7ff7481f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747e00640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7481ed5e0 0x7ff7481f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747e00640 1 -- 172.21.15.62:0/537590563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748154880 con 0x7ff748150a80 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.582+0000 7ff747400640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0051c0 0x7ff7481541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.583+0000 7ff747e00640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff748150a80 0x7ff748159090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff7400045d0 tx=0x7ff740003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.583+0000 7ff73f400640 1 -- 172.21.15.62:0/537590563 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740042070 con 0x7ff748150a80 2024-04-25T09:03:52.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.583+0000 7ff73f400640 1 -- 172.21.15.62:0/537590563 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff740006b50 con 0x7ff748150a80 2024-04-25T09:03:52.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.583+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff748154b60 con 0x7ff748150a80 2024-04-25T09:03:52.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.583+0000 7ff73f400640 1 -- 172.21.15.62:0/537590563 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740037040 con 0x7ff748150a80 2024-04-25T09:03:52.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.583+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff748155120 con 0x7ff748150a80 2024-04-25T09:03:52.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.584+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74806abe0 con 0x7ff748150a80 2024-04-25T09:03:52.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.585+0000 7ff73f400640 1 -- 172.21.15.62:0/537590563 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff740045070 con 0x7ff748150a80 2024-04-25T09:03:52.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.586+0000 7ff73f400640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff72406b4d0 0x7ff72406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.586+0000 7ff73f400640 1 -- 172.21.15.62:0/537590563 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7400a9460 con 0x7ff748150a80 2024-04-25T09:03:52.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.587+0000 7ff747400640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff72406b4d0 0x7ff72406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.588+0000 7ff747400640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff72406b4d0 0x7ff72406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff7480ae940 tx=0x7ff738008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.591+0000 7ff73f400640 1 -- 172.21.15.62:0/537590563 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff740077bf0 con 0x7ff748150a80 2024-04-25T09:03:52.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.794+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff74804ef20 con 0x7ff748150a80 2024-04-25T09:03:52.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.795+0000 7ff73f400640 1 -- 172.21.15.62:0/537590563 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff74007bc90 con 0x7ff748150a80 2024-04-25T09:03:52.784 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:03:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff72406b4d0 msgr2=0x7ff72406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff72406b4d0 0x7ff72406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff7480ae940 tx=0x7ff738008040 comp rx=0 tx=0).stop 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff748150a80 msgr2=0x7ff748159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff748150a80 0x7ff748159090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff7400045d0 tx=0x7ff740003a70 comp rx=0 tx=0).stop 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 shutdown_connections 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff72406b4d0 0x7ff72406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff748150a80 0x7ff748159090 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7481ed5e0 0x7ff7481f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 --2- 172.21.15.62:0/537590563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff72c0051c0 0x7ff7481541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 >> 172.21.15.62:0/537590563 conn(0x7ff748066f20 msgr2=0x7ff748156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.798+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 shutdown_connections 2024-04-25T09:03:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:52.799+0000 7ff74d000640 1 -- 172.21.15.62:0/537590563 wait complete. 2024-04-25T09:03:52.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:53.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.028+0000 7f7a46200640 1 Processor -- start 2024-04-25T09:03:53.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.028+0000 7f7a46200640 1 -- start start 2024-04-25T09:03:53.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a46200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7a4006d2c0 0x7f7a400ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a46200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4006d6a0 0x7f7a400ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a46200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a400ced40 con 0x7f7a400b31a0 2024-04-25T09:03:53.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a46200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a400cf000 con 0x7f7a4006d2c0 2024-04-25T09:03:53.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a46200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a400cf2e0 con 0x7f7a4006d6a0 2024-04-25T09:03:53.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a45800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7a400b31a0 0x7f7a400b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50418/0 (socket says 172.21.15.62:50418) 2024-04-25T09:03:53.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a45800640 1 -- 172.21.15.62:0/3509016891 learned_addr learned my addr 172.21.15.62:0/3509016891 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:53.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a3fe00640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4006d6a0 0x7f7a400ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.030+0000 7f7a44e00640 1 --2- 172.21.15.62:0/3509016891 >> v2:172.21.15.121:3300/0 conn(0x7f7a4006d2c0 0x7f7a400ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137479978 0 0) 0x7f7a400ced40 con 0x7f7a400b31a0 2024-04-25T09:03:53.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a28003520 con 0x7f7a400b31a0 2024-04-25T09:03:53.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927561096 0 0) 0x7f7a28003520 con 0x7f7a400b31a0 2024-04-25T09:03:53.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4006d6a0 msgr2=0x7f7a400ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4006d6a0 0x7f7a400ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 >> v2:172.21.15.121:3300/0 conn(0x7f7a4006d2c0 msgr2=0x7f7a400ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 --2- 172.21.15.62:0/3509016891 >> v2:172.21.15.121:3300/0 conn(0x7f7a4006d2c0 0x7f7a400ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a400b26c0 con 0x7f7a400b31a0 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f7a380019f0 con 0x7f7a400b31a0 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 >> v1:172.21.15.62:6789/0 conn(0x7f7a400b31a0 legacy=0x7f7a400b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4006d6a0 0x7f7a280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a28004d40 0x7f7a280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a400cf000 con 0x7f7a28003f30 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a400cf2e0 con 0x7f7a28004d40 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.031+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2800b9c0 con 0x7f7a4006d6a0 2024-04-25T09:03:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a3fe00640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a44e00640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4006d6a0 0x7f7a280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a45800640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a28004d40 0x7f7a280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a3fe00640 1 -- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4006d6a0 msgr2=0x7f7a280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a3fe00640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4006d6a0 0x7f7a280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a3fe00640 1 -- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a28004d40 msgr2=0x7f7a280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a3fe00640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a28004d40 0x7f7a280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a3fe00640 1 -- 172.21.15.62:0/3509016891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a28003520 con 0x7f7a28003f30 2024-04-25T09:03:53.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.032+0000 7f7a45800640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a28004d40 0x7f7a280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:53.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.033+0000 7f7a3fe00640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a28004800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009ac0 tx=0x7f7a28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d7a235aa599c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c036070 con 0x7f7a28003f30 2024-04-25T09:03:53.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c031400 con 0x7f7a28003f30 2024-04-25T09:03:53.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a3f400640 1 -- 172.21.15.62:0/3509016891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c009690 con 0x7f7a28003f30 2024-04-25T09:03:53.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a46200640 1 -- 172.21.15.62:0/3509016891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 msgr2=0x7f7a28004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a46200640 1 --2- 172.21.15.62:0/3509016891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a28004800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009ac0 tx=0x7f7a28005830 comp rx=0 tx=0).stop 2024-04-25T09:03:53.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a45800640 1 -- 172.21.15.62:0/3509016891 reap_dead start 2024-04-25T09:03:53.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a46200640 1 -- 172.21.15.62:0/3509016891 shutdown_connections 2024-04-25T09:03:53.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a46200640 1 -- 172.21.15.62:0/3509016891 >> 172.21.15.62:0/3509016891 conn(0x7f7a40066f20 msgr2=0x7f7a400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:53.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.034+0000 7f7a46200640 1 -- 172.21.15.62:0/3509016891 shutdown_connections 2024-04-25T09:03:53.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.035+0000 7f7a46200640 1 -- 172.21.15.62:0/3509016891 wait complete. 2024-04-25T09:03:53.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.036+0000 7f7a46200640 1 Processor -- start 2024-04-25T09:03:53.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.036+0000 7f7a46200640 1 -- start start 2024-04-25T09:03:53.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.037+0000 7f7a46200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a4015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.037+0000 7f7a46200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a4013e260 0x7f7a4013cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.037+0000 7f7a46200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4015a8d0 0x7f7a4013f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.037+0000 7f7a46200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4006db20 con 0x7f7a28003f30 2024-04-25T09:03:53.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.037+0000 7f7a46200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a400b2c80 con 0x7f7a4013e260 2024-04-25T09:03:53.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.037+0000 7f7a46200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a401448d0 con 0x7f7a4015a8d0 2024-04-25T09:03:53.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a4015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a4015af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34186/0 (socket says 172.21.15.62:34186) 2024-04-25T09:03:53.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 -- 172.21.15.62:0/526246945 learned_addr learned my addr 172.21.15.62:0/526246945 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:53.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a3fe00640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a4013e260 0x7f7a4013cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a45800640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4015a8d0 0x7f7a4013f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 -- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4015a8d0 msgr2=0x7f7a4013f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4015a8d0 0x7f7a4013f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 -- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a4013e260 msgr2=0x7f7a4013cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a4013e260 0x7f7a4013cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 -- 172.21.15.62:0/526246945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4015b7e0 con 0x7f7a28003f30 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.038+0000 7f7a44e00640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a4015af70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7a340116d0 tx=0x7f7a34011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.039+0000 7f7a3cc00640 1 -- 172.21.15.62:0/526246945 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400f960 con 0x7f7a28003f30 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.039+0000 7f7a3cc00640 1 -- 172.21.15.62:0/526246945 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3400fb00 con 0x7f7a28003f30 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.039+0000 7f7a3cc00640 1 -- 172.21.15.62:0/526246945 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34015460 con 0x7f7a28003f30 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.039+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4015ad60 con 0x7f7a28003f30 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.039+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4013d4a0 con 0x7f7a28003f30 2024-04-25T09:03:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.040+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4006abe0 con 0x7f7a28003f30 2024-04-25T09:03:53.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.041+0000 7f7a3cc00640 1 -- 172.21.15.62:0/526246945 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a34015600 con 0x7f7a28003f30 2024-04-25T09:03:53.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.042+0000 7f7a3cc00640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a2006b4a0 0x7f7a2006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.043+0000 7f7a3cc00640 1 -- 172.21.15.62:0/526246945 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a3408abc0 con 0x7f7a28003f30 2024-04-25T09:03:53.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.043+0000 7f7a3fe00640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a2006b4a0 0x7f7a2006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.044+0000 7f7a3fe00640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a2006b4a0 0x7f7a2006d960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c033000 tx=0x7f7a2c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.047+0000 7f7a3cc00640 1 -- 172.21.15.62:0/526246945 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a34019030 con 0x7f7a28003f30 2024-04-25T09:03:53.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.245+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a4004ef20 con 0x7f7a28003f30 2024-04-25T09:03:53.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.246+0000 7f7a3cc00640 1 -- 172.21.15.62:0/526246945 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a34059a30 con 0x7f7a28003f30 2024-04-25T09:03:53.235 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:03:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a2006b4a0 msgr2=0x7f7a2006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a2006b4a0 0x7f7a2006d960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c033000 tx=0x7f7a2c008030 comp rx=0 tx=0).stop 2024-04-25T09:03:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 msgr2=0x7f7a4015af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a4015af70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7a340116d0 tx=0x7f7a34011b90 comp rx=0 tx=0).stop 2024-04-25T09:03:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 shutdown_connections 2024-04-25T09:03:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a2006b4a0 0x7f7a2006d960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a4015a8d0 0x7f7a4013f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a4013e260 0x7f7a4013cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 --2- 172.21.15.62:0/526246945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a28003f30 0x7f7a4015af70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 >> 172.21.15.62:0/526246945 conn(0x7f7a40066f20 msgr2=0x7f7a4006ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:53.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 shutdown_connections 2024-04-25T09:03:53.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.249+0000 7f7a46200640 1 -- 172.21.15.62:0/526246945 wait complete. 2024-04-25T09:03:53.258 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.484+0000 7f95b4c00640 1 Processor -- start 2024-04-25T09:03:53.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.485+0000 7f95b4c00640 1 -- start start 2024-04-25T09:03:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.486+0000 7f95b4c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95b00b56f0 0x7f95b0160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.486+0000 7f95b4c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b6a90 0x7f95b0161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.486+0000 7f95b4c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f95b00b27a0 0x7f95b0161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.486+0000 7f95b4c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b0161780 con 0x7f95b00b56f0 2024-04-25T09:03:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.486+0000 7f95b4c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00cedc0 con 0x7f95b00b27a0 2024-04-25T09:03:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.486+0000 7f95b4c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00d3110 con 0x7f95b00b6a90 2024-04-25T09:03:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95af400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95b00b56f0 0x7f95b0160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95af400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95b00b56f0 0x7f95b0160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34200/0 (socket says 172.21.15.62:34200) 2024-04-25T09:03:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95af400640 1 -- 172.21.15.62:0/2539220207 learned_addr learned my addr 172.21.15.62:0/2539220207 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95afe00640 1 --2- 172.21.15.62:0/2539220207 >> v2:172.21.15.121:3300/0 conn(0x7f95b00b27a0 0x7f95b0161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95aea00640 1 --2- 172.21.15.62:0/2539220207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b6a90 0x7f95b0161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95af400640 1 -- 172.21.15.62:0/2539220207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b6a90 msgr2=0x7f95b0161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95af400640 1 --2- 172.21.15.62:0/2539220207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b6a90 0x7f95b0161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95af400640 1 -- 172.21.15.62:0/2539220207 >> v2:172.21.15.121:3300/0 conn(0x7f95b00b27a0 msgr2=0x7f95b0161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95af400640 1 --2- 172.21.15.62:0/2539220207 >> v2:172.21.15.121:3300/0 conn(0x7f95b00b27a0 0x7f95b0161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95af400640 1 -- 172.21.15.62:0/2539220207 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00d33f0 con 0x7f95b00b56f0 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.487+0000 7f95aea00640 1 --2- 172.21.15.62:0/2539220207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b6a90 0x7f95b0161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.488+0000 7f95af400640 1 --2- 172.21.15.62:0/2539220207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b56f0 0x7f95b0160d00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b30b0 tx=0x7f95a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bd3fab484b936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95ae000640 1 -- 172.21.15.62:0/2539220207 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0037070 con 0x7f95b00b56f0 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95ae000640 1 -- 172.21.15.62:0/2539220207 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a0032440 con 0x7f95b00b56f0 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95ae000640 1 -- 172.21.15.62:0/2539220207 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0032810 con 0x7f95b00b56f0 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95b4c00640 1 -- 172.21.15.62:0/2539220207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b56f0 msgr2=0x7f95b0160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2539220207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b56f0 0x7f95b0160d00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b30b0 tx=0x7f95a002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95b4c00640 1 -- 172.21.15.62:0/2539220207 shutdown_connections 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2539220207 >> v2:172.21.15.121:3300/0 conn(0x7f95b00b27a0 0x7f95b0161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2539220207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b6a90 0x7f95b0161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2539220207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b56f0 0x7f95b0160d00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.489+0000 7f95b4c00640 1 -- 172.21.15.62:0/2539220207 >> 172.21.15.62:0/2539220207 conn(0x7f95b0066d10 msgr2=0x7f95b0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.490+0000 7f95b4c00640 1 -- 172.21.15.62:0/2539220207 shutdown_connections 2024-04-25T09:03:53.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.490+0000 7f95b4c00640 1 -- 172.21.15.62:0/2539220207 wait complete. 2024-04-25T09:03:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.491+0000 7f95b4c00640 1 Processor -- start 2024-04-25T09:03:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.491+0000 7f95b4c00640 1 -- start start 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.492+0000 7f95b4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b27a0 0x7f95b0152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.492+0000 7f95b4c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b56f0 0x7f95b00cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.492+0000 7f95b4c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b00b6a90 0x7f95b00ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.492+0000 7f95b4c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b2d60 con 0x7f95b00b27a0 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.492+0000 7f95b4c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00cea50 con 0x7f95b00b6a90 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.492+0000 7f95b4c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b2ee0 con 0x7f95b00b56f0 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b27a0 0x7f95b0152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b27a0 0x7f95b0152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34210/0 (socket says 172.21.15.62:34210) 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 -- 172.21.15.62:0/2936808446 learned_addr learned my addr 172.21.15.62:0/2936808446 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 -- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b56f0 msgr2=0x7f95b00cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b56f0 0x7f95b00cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 -- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b00b6a90 msgr2=0x7f95b00ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b00b6a90 0x7f95b00ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 -- 172.21.15.62:0/2936808446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00aa610 con 0x7f95b00b27a0 2024-04-25T09:03:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.493+0000 7f95af400640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b27a0 0x7f95b0152e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95a002a220 tx=0x7f95a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.494+0000 7f95a7400640 1 -- 172.21.15.62:0/2936808446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0041020 con 0x7f95b00b27a0 2024-04-25T09:03:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.494+0000 7f95a7400640 1 -- 172.21.15.62:0/2936808446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a00306e0 con 0x7f95b00b27a0 2024-04-25T09:03:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.494+0000 7f95a7400640 1 -- 172.21.15.62:0/2936808446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0037040 con 0x7f95b00b27a0 2024-04-25T09:03:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.494+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b00aa8f0 con 0x7f95b00b27a0 2024-04-25T09:03:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.494+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b0153360 con 0x7f95b00b27a0 2024-04-25T09:03:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.495+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b004ef20 con 0x7f95b00b27a0 2024-04-25T09:03:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.496+0000 7f95a7400640 1 -- 172.21.15.62:0/2936808446 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95a0044020 con 0x7f95b00b27a0 2024-04-25T09:03:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.497+0000 7f95a7400640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f958c06b4d0 0x7f958c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.497+0000 7f95a7400640 1 -- 172.21.15.62:0/2936808446 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95a00aab00 con 0x7f95b00b27a0 2024-04-25T09:03:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.497+0000 7f95aea00640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f958c06b4d0 0x7f958c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.498+0000 7f95aea00640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f958c06b4d0 0x7f958c06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9598006fd0 tx=0x7f9598008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.501+0000 7f95a7400640 1 -- 172.21.15.62:0/2936808446 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95a004b030 con 0x7f95b00b27a0 2024-04-25T09:03:53.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.703+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95b00b6180 con 0x7f95b00b27a0 2024-04-25T09:03:53.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.705+0000 7f95a7400640 1 -- 172.21.15.62:0/2936808446 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f95a00aae20 con 0x7f95b00b27a0 2024-04-25T09:03:53.694 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:53.694 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.707+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f958c06b4d0 msgr2=0x7f958c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.707+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f958c06b4d0 0x7f958c06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9598006fd0 tx=0x7f9598008040 comp rx=0 tx=0).stop 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.707+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b27a0 msgr2=0x7f95b0152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.707+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b27a0 0x7f95b0152e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95a002a220 tx=0x7f95a00046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.707+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 shutdown_connections 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.708+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f958c06b4d0 0x7f958c06d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.708+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95b00b6a90 0x7f95b00ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.708+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95b00b56f0 0x7f95b00cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.708+0000 7f95b4c00640 1 --2- 172.21.15.62:0/2936808446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95b00b27a0 0x7f95b0152e20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.708+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 >> 172.21.15.62:0/2936808446 conn(0x7f95b0066d10 msgr2=0x7f95b00b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.708+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 shutdown_connections 2024-04-25T09:03:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.708+0000 7f95b4c00640 1 -- 172.21.15.62:0/2936808446 wait complete. 2024-04-25T09:03:53.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:53.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.946+0000 7f51de800640 1 Processor -- start 2024-04-25T09:03:53.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.946+0000 7f51de800640 1 -- start start 2024-04-25T09:03:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.948+0000 7f51de800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51d80cebc0 0x7f51d806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.948+0000 7f51de800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d806d070 con 0x7f51d80720c0 2024-04-25T09:03:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.948+0000 7f51de800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d806d350 con 0x7f51d815bdf0 2024-04-25T09:03:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.948+0000 7f51de800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8072570 con 0x7f51d80cebc0 2024-04-25T09:03:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.948+0000 7f51d7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f51d80720c0 0x7f51d80703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50420/0 (socket says 172.21.15.62:50420) 2024-04-25T09:03:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.948+0000 7f51d7e00640 1 -- 172.21.15.62:0/3705063454 learned_addr learned my addr 172.21.15.62:0/3705063454 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:53.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.948+0000 7f51d6a00640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51d80cebc0 0x7f51d806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739543875 0 0) 0x7f51d806d070 con 0x7f51d80720c0 2024-04-25T09:03:53.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51bc003520 con 0x7f51d80720c0 2024-04-25T09:03:53.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1598966482 0 0) 0x7f51bc003520 con 0x7f51d80720c0 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51d80cebc0 msgr2=0x7f51d806c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51d80cebc0 0x7f51d806c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 >> v1:172.21.15.121:6789/0 conn(0x7f51d815bdf0 legacy=0x7f51d806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d80aeb10 con 0x7f51d80720c0 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f51d0001670 con 0x7f51d80720c0 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 >> v1:172.21.15.62:6789/0 conn(0x7f51d80720c0 legacy=0x7f51d80703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51bc0039e0 0x7f51bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51d80cebc0 0x7f51bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51bc004d40 0x7f51bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc00ba70 con 0x7f51bc004d40 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc00bc20 con 0x7f51bc0039e0 2024-04-25T09:03:53.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.949+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc00be20 con 0x7f51d80cebc0 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d7e00640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51bc004d40 0x7f51bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d6a00640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51d80cebc0 0x7f51bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d7400640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51bc0039e0 0x7f51bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d7e00640 1 -- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51d80cebc0 msgr2=0x7f51bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d7e00640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51d80cebc0 0x7f51bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d7e00640 1 -- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51bc0039e0 msgr2=0x7f51bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d7e00640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51bc0039e0 0x7f51bc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d7e00640 1 -- 172.21.15.62:0/3705063454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc003520 con 0x7f51bc004d40 2024-04-25T09:03:53.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.950+0000 7f51d7400640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51bc0039e0 0x7f51bc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:53.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.951+0000 7f51d7e00640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51bc004d40 0x7f51bc0051c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c160 tx=0x7f51bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d64e1be3ec582d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.951+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0007c60 con 0x7f51bc004d40 2024-04-25T09:03:53.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.951+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0005af0 con 0x7f51bc004d40 2024-04-25T09:03:53.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.951+0000 7f51d6000640 1 -- 172.21.15.62:0/3705063454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0007070 con 0x7f51bc004d40 2024-04-25T09:03:53.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.952+0000 7f51de800640 1 -- 172.21.15.62:0/3705063454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51bc004d40 msgr2=0x7f51bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.952+0000 7f51de800640 1 --2- 172.21.15.62:0/3705063454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51bc004d40 0x7f51bc0051c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c160 tx=0x7f51bc005830 comp rx=0 tx=0).stop 2024-04-25T09:03:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.952+0000 7f51d7e00640 1 -- 172.21.15.62:0/3705063454 reap_dead start 2024-04-25T09:03:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.952+0000 7f51de800640 1 -- 172.21.15.62:0/3705063454 shutdown_connections 2024-04-25T09:03:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.952+0000 7f51de800640 1 -- 172.21.15.62:0/3705063454 >> 172.21.15.62:0/3705063454 conn(0x7f51d80b1410 msgr2=0x7f51d80b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.952+0000 7f51de800640 1 -- 172.21.15.62:0/3705063454 shutdown_connections 2024-04-25T09:03:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.952+0000 7f51de800640 1 -- 172.21.15.62:0/3705063454 wait complete. 2024-04-25T09:03:53.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.954+0000 7f51de800640 1 Processor -- start 2024-04-25T09:03:53.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.954+0000 7f51de800640 1 -- start start 2024-04-25T09:03:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.955+0000 7f51de800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51bc004d40 0x7f51d8157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.955+0000 7f51de800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d8154ee0 0x7f51d80b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.955+0000 7f51de800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d80b42e0 0x7f51d815c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.955+0000 7f51de800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80ae8c0 con 0x7f51d80b42e0 2024-04-25T09:03:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.955+0000 7f51de800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d815c1d0 con 0x7f51d8154ee0 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.955+0000 7f51de800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d806c9b0 con 0x7f51bc004d40 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d80b42e0 0x7f51d815c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d80b42e0 0x7f51d815c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34228/0 (socket says 172.21.15.62:34228) 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7e00640 1 -- 172.21.15.62:0/1911886197 learned_addr learned my addr 172.21.15.62:0/1911886197 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7400640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51bc004d40 0x7f51d8157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d6a00640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d8154ee0 0x7f51d80b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7e00640 1 -- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51bc004d40 msgr2=0x7f51d8157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7e00640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51bc004d40 0x7f51d8157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7e00640 1 -- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d8154ee0 msgr2=0x7f51d80b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7e00640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d8154ee0 0x7f51d80b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d7e00640 1 -- 172.21.15.62:0/1911886197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8155710 con 0x7f51d80b42e0 2024-04-25T09:03:53.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.956+0000 7f51d6a00640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d8154ee0 0x7f51d80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.957+0000 7f51d7e00640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d80b42e0 0x7f51d815c350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c280 tx=0x7f51d0001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.957+0000 7f51cf400640 1 -- 172.21.15.62:0/1911886197 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0037040 con 0x7f51d80b42e0 2024-04-25T09:03:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.957+0000 7f51cf400640 1 -- 172.21.15.62:0/1911886197 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0041070 con 0x7f51d80b42e0 2024-04-25T09:03:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.957+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d81559f0 con 0x7f51d80b42e0 2024-04-25T09:03:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.957+0000 7f51cf400640 1 -- 172.21.15.62:0/1911886197 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0007c60 con 0x7f51d80b42e0 2024-04-25T09:03:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.957+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d8155fb0 con 0x7f51d80b42e0 2024-04-25T09:03:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.958+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d806fd80 con 0x7f51d80b42e0 2024-04-25T09:03:53.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.959+0000 7f51cf400640 1 -- 172.21.15.62:0/1911886197 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51d0007070 con 0x7f51d80b42e0 2024-04-25T09:03:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.960+0000 7f51cf400640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b500 0x7f51b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.960+0000 7f51cf400640 1 -- 172.21.15.62:0/1911886197 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51d00a9ec0 con 0x7f51d80b42e0 2024-04-25T09:03:53.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.961+0000 7f51d7400640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b500 0x7f51b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:53.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.962+0000 7f51d7400640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b500 0x7f51b806d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f51c8001e40 tx=0x7f51c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:53.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:53.964+0000 7f51cf400640 1 -- 172.21.15.62:0/1911886197 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51d806fd80 con 0x7f51d80b42e0 2024-04-25T09:03:54.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.165+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51d80cd260 con 0x7f51d80b42e0 2024-04-25T09:03:54.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.166+0000 7f51cf400640 1 -- 172.21.15.62:0/1911886197 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51d80cd260 con 0x7f51d80b42e0 2024-04-25T09:03:54.155 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:54.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b500 msgr2=0x7f51b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b500 0x7f51b806d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f51c8001e40 tx=0x7f51c8009040 comp rx=0 tx=0).stop 2024-04-25T09:03:54.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d80b42e0 msgr2=0x7f51d815c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d80b42e0 0x7f51d815c350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c280 tx=0x7f51d0001970 comp rx=0 tx=0).stop 2024-04-25T09:03:54.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 shutdown_connections 2024-04-25T09:03:54.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b500 0x7f51b806d9c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d80b42e0 0x7f51d815c350 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d8154ee0 0x7f51d80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 --2- 172.21.15.62:0/1911886197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51bc004d40 0x7f51d8157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 >> 172.21.15.62:0/1911886197 conn(0x7f51d80b1410 msgr2=0x7f51d80cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:54.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 shutdown_connections 2024-04-25T09:03:54.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.169+0000 7f51de800640 1 -- 172.21.15.62:0/1911886197 wait complete. 2024-04-25T09:03:54.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.407+0000 7fb1d2c00640 1 Processor -- start 2024-04-25T09:03:54.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.407+0000 7fb1d2c00640 1 -- start start 2024-04-25T09:03:54.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.408+0000 7fb1d2c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb1cc0d2fb0 0x7fb1cc0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.408+0000 7fb1d2c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc0ce780 con 0x7fb1cc0d3960 2024-04-25T09:03:54.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.408+0000 7fb1d2c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc0d3440 con 0x7fb1cc15dd50 2024-04-25T09:03:54.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.408+0000 7fb1d2c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1cc0ce1c0 con 0x7fb1cc0d2fb0 2024-04-25T09:03:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.409+0000 7fb1d2200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb1cc0d3960 0x7fb1cc0d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50424/0 (socket says 172.21.15.62:50424) 2024-04-25T09:03:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.409+0000 7fb1d2200640 1 -- 172.21.15.62:0/4045363008 learned_addr learned my addr 172.21.15.62:0/4045363008 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.409+0000 7fb1d0e00640 1 --2- 172.21.15.62:0/4045363008 >> v2:172.21.15.121:3300/0 conn(0x7fb1cc0d2fb0 0x7fb1cc0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.409+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358919332 0 0) 0x7fb1cc0ce780 con 0x7fb1cc0d3960 2024-04-25T09:03:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.409+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1b0003520 con 0x7fb1cc0d3960 2024-04-25T09:03:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 907819843 0 0) 0x7fb1b0003520 con 0x7fb1cc0d3960 2024-04-25T09:03:54.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 >> v2:172.21.15.121:3300/0 conn(0x7fb1cc0d2fb0 msgr2=0x7fb1cc0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 --2- 172.21.15.62:0/4045363008 >> v2:172.21.15.121:3300/0 conn(0x7fb1cc0d2fb0 0x7fb1cc0ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 >> v1:172.21.15.121:6789/0 conn(0x7fb1cc15dd50 legacy=0x7fb1cc0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0ce4a0 con 0x7fb1cc0d3960 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb1c40019f0 con 0x7fb1cc0d3960 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 >> v1:172.21.15.62:6789/0 conn(0x7fb1cc0d3960 legacy=0x7fb1cc0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b0003a00 0x7fb1b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1b0004800 0x7fb1b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0ce1c0 con 0x7fb1b00051c0 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b000c0c0 con 0x7fb1b0003a00 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b000c2c0 con 0x7fb1b0004800 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.410+0000 7fb1d2200640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.411+0000 7fb1d1800640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b0003a00 0x7fb1b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.411+0000 7fb1d0e00640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1b0004800 0x7fb1b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.411+0000 7fb1d2200640 1 -- 172.21.15.62:0/4045363008 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1b0004800 msgr2=0x7fb1b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.411+0000 7fb1d2200640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1b0004800 0x7fb1b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.411+0000 7fb1d2200640 1 -- 172.21.15.62:0/4045363008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b0003a00 msgr2=0x7fb1b0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.411+0000 7fb1d2200640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1b0003a00 0x7fb1b0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.411+0000 7fb1d2200640 1 -- 172.21.15.62:0/4045363008 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b0003520 con 0x7fb1b00051c0 2024-04-25T09:03:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.412+0000 7fb1d2200640 1 -- 172.21.15.62:0/4045363008 reap_dead start 2024-04-25T09:03:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.412+0000 7fb1d2200640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1b000b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400a790 tx=0x7fb1b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ab2c36a31c5fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.412+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400a3b0 con 0x7fb1b00051c0 2024-04-25T09:03:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.412+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c402f070 con 0x7fb1b00051c0 2024-04-25T09:03:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.412+0000 7fb1cbe00640 1 -- 172.21.15.62:0/4045363008 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c402bc00 con 0x7fb1b00051c0 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.413+0000 7fb1d2c00640 1 -- 172.21.15.62:0/4045363008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 msgr2=0x7fb1b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.413+0000 7fb1d2c00640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1b000b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400a790 tx=0x7fb1b0005830 comp rx=0 tx=0).stop 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.413+0000 7fb1d2c00640 1 -- 172.21.15.62:0/4045363008 shutdown_connections 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.413+0000 7fb1d2c00640 1 --2- 172.21.15.62:0/4045363008 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1b000b9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.413+0000 7fb1d2c00640 1 -- 172.21.15.62:0/4045363008 >> 172.21.15.62:0/4045363008 conn(0x7fb1cc06b000 msgr2=0x7fb1cc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.413+0000 7fb1d2c00640 1 -- 172.21.15.62:0/4045363008 shutdown_connections 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.413+0000 7fb1d2c00640 1 -- 172.21.15.62:0/4045363008 wait complete. 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.415+0000 7fb1d2c00640 1 Processor -- start 2024-04-25T09:03:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.415+0000 7fb1d2c00640 1 -- start start 2024-04-25T09:03:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.416+0000 7fb1d2c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1cc14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.416+0000 7fb1d2c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc1586f0 0x7fb1cc0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.416+0000 7fb1d2c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc158b70 0x7fb1cc0b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.416+0000 7fb1d2c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0d4140 con 0x7fb1b00051c0 2024-04-25T09:03:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.416+0000 7fb1d2c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0d3fc0 con 0x7fb1cc158b70 2024-04-25T09:03:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.416+0000 7fb1d2c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0d42c0 con 0x7fb1cc1586f0 2024-04-25T09:03:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1cc14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1cc14ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34244/0 (socket says 172.21.15.62:34244) 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 -- 172.21.15.62:0/3697170010 learned_addr learned my addr 172.21.15.62:0/3697170010 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d0e00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc1586f0 0x7fb1cc0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d2200640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc158b70 0x7fb1cc0b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 -- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc1586f0 msgr2=0x7fb1cc0b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc1586f0 0x7fb1cc0b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 -- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc158b70 msgr2=0x7fb1cc0b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc158b70 0x7fb1cc0b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 -- 172.21.15.62:0/3697170010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc1599e0 con 0x7fb1b00051c0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d2200640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc158b70 0x7fb1cc0b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.417+0000 7fb1d1800640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1cc14ee00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00f5f0 tx=0x7fb1bc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.418+0000 7fb1c9600640 1 -- 172.21.15.62:0/3697170010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0052e0 con 0x7fb1b00051c0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.418+0000 7fb1c9600640 1 -- 172.21.15.62:0/3697170010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc018070 con 0x7fb1b00051c0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.418+0000 7fb1c9600640 1 -- 172.21.15.62:0/3697170010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc003810 con 0x7fb1b00051c0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.418+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc159140 con 0x7fb1b00051c0 2024-04-25T09:03:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.418+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc159700 con 0x7fb1b00051c0 2024-04-25T09:03:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.419+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc0cfe90 con 0x7fb1b00051c0 2024-04-25T09:03:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.420+0000 7fb1c9600640 1 -- 172.21.15.62:0/3697170010 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1bc0039b0 con 0x7fb1b00051c0 2024-04-25T09:03:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.421+0000 7fb1c9600640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb1ac06b500 0x7fb1ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.422+0000 7fb1c9600640 1 -- 172.21.15.62:0/3697170010 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1bc08a060 con 0x7fb1b00051c0 2024-04-25T09:03:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.422+0000 7fb1d0e00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb1ac06b500 0x7fb1ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.423+0000 7fb1d0e00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb1ac06b500 0x7fb1ac06d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc15ffa0 tx=0x7fb1b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.426+0000 7fb1c9600640 1 -- 172.21.15.62:0/3697170010 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1cc0cfe90 con 0x7fb1b00051c0 2024-04-25T09:03:54.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.622+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1cc04ef20 con 0x7fb1b00051c0 2024-04-25T09:03:54.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.623+0000 7fb1c9600640 1 -- 172.21.15.62:0/3697170010 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1bc058ed0 con 0x7fb1b00051c0 2024-04-25T09:03:54.612 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:03:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb1ac06b500 msgr2=0x7fb1ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb1ac06b500 0x7fb1ac06d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc15ffa0 tx=0x7fb1b8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 msgr2=0x7fb1cc14ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1cc14ee00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00f5f0 tx=0x7fb1bc00fab0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 shutdown_connections 2024-04-25T09:03:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb1ac06b500 0x7fb1ac06d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1cc158b70 0x7fb1cc0b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb1cc1586f0 0x7fb1cc0b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 --2- 172.21.15.62:0/3697170010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1b00051c0 0x7fb1cc14ee00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 >> 172.21.15.62:0/3697170010 conn(0x7fb1cc06b000 msgr2=0x7fb1cc0b8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 shutdown_connections 2024-04-25T09:03:54.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.626+0000 7fb1d2c00640 1 -- 172.21.15.62:0/3697170010 wait complete. 2024-04-25T09:03:54.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:54.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.861+0000 7fc45ea00640 1 Processor -- start 2024-04-25T09:03:54.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.861+0000 7fc45ea00640 1 -- start start 2024-04-25T09:03:54.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.862+0000 7fc45ea00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc45815c520 0x7fc4580b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.862+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 0x7fc4580ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.862+0000 7fc45ea00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580b4640 con 0x7fc4580b3360 2024-04-25T09:03:54.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.863+0000 7fc45ea00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc458165c40 con 0x7fc45815c520 2024-04-25T09:03:54.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.863+0000 7fc45ea00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc458165f20 con 0x7fc45806d9d0 2024-04-25T09:03:54.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.863+0000 7fc45cc00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc4580b3360 0x7fc45806d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40856/0 (socket says 172.21.15.62:40856) 2024-04-25T09:03:54.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.863+0000 7fc45cc00640 1 -- 172.21.15.62:0/2241762539 learned_addr learned my addr 172.21.15.62:0/2241762539 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:54.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.863+0000 7fc45e000640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 0x7fc4580ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.863+0000 7fc45d600640 1 --2- 172.21.15.62:0/2241762539 >> v2:172.21.15.121:3300/0 conn(0x7fc45815c520 0x7fc4580b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.863+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327847170 0 0) 0x7fc4580b4640 con 0x7fc4580b3360 2024-04-25T09:03:54.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.863+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc440003520 con 0x7fc4580b3360 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4089124138 0 0) 0x7fc440003520 con 0x7fc4580b3360 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 msgr2=0x7fc4580ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 0x7fc4580ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 >> v2:172.21.15.121:3300/0 conn(0x7fc45815c520 msgr2=0x7fc4580b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 --2- 172.21.15.62:0/2241762539 >> v2:172.21.15.121:3300/0 conn(0x7fc45815c520 0x7fc4580b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc458166200 con 0x7fc4580b3360 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc444001670 con 0x7fc4580b3360 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 >> v1:172.21.15.62:6789/0 conn(0x7fc4580b3360 legacy=0x7fc45806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 0x7fc4400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc440003f80 0x7fc440004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc440004d40 0x7fc4400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458165c40 con 0x7fc440003f80 2024-04-25T09:03:54.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc458165f20 con 0x7fc440004d40 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44000bba0 con 0x7fc45806d9d0 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc45cc00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc440003f80 0x7fc440004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.864+0000 7fc45e000640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc440004d40 0x7fc4400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.865+0000 7fc45d600640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 0x7fc4400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.865+0000 7fc45cc00640 1 -- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 msgr2=0x7fc4400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.865+0000 7fc45cc00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 0x7fc4400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.865+0000 7fc45cc00640 1 -- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc440004d40 msgr2=0x7fc4400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.865+0000 7fc45cc00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc440004d40 0x7fc4400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.865+0000 7fc45cc00640 1 -- 172.21.15.62:0/2241762539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440003520 con 0x7fc440003f80 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.865+0000 7fc45d600640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc45806d9d0 0x7fc4400039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:54.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.866+0000 7fc45cc00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc440003f80 0x7fc440004800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc44400c190 tx=0x7fc440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61df57389432cc78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.866+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444007c60 con 0x7fc440003f80 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.866+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc444005af0 con 0x7fc440003f80 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.866+0000 7fc457e00640 1 -- 172.21.15.62:0/2241762539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444007070 con 0x7fc440003f80 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.867+0000 7fc45ea00640 1 -- 172.21.15.62:0/2241762539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc440003f80 msgr2=0x7fc440004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.867+0000 7fc45ea00640 1 --2- 172.21.15.62:0/2241762539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc440003f80 0x7fc440004800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc44400c190 tx=0x7fc440005830 comp rx=0 tx=0).stop 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.867+0000 7fc45e000640 1 -- 172.21.15.62:0/2241762539 reap_dead start 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.867+0000 7fc45ea00640 1 -- 172.21.15.62:0/2241762539 shutdown_connections 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.867+0000 7fc45ea00640 1 -- 172.21.15.62:0/2241762539 >> 172.21.15.62:0/2241762539 conn(0x7fc458066f20 msgr2=0x7fc4580b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.867+0000 7fc45ea00640 1 -- 172.21.15.62:0/2241762539 shutdown_connections 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.867+0000 7fc45ea00640 1 -- 172.21.15.62:0/2241762539 wait complete. 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.869+0000 7fc45ea00640 1 Processor -- start 2024-04-25T09:03:54.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.869+0000 7fc45ea00640 1 -- start start 2024-04-25T09:03:54.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc440003f80 0x7fc4581f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc458160720 0x7fc45814df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4581ed810 0x7fc4581edc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580b4250 con 0x7fc4581ed810 2024-04-25T09:03:54.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45ea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45815c900 con 0x7fc458160720 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45ea00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4580b43d0 con 0x7fc440003f80 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4581ed810 0x7fc4581edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4581ed810 0x7fc4581edc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46308/0 (socket says 172.21.15.62:46308) 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45e000640 1 -- 172.21.15.62:0/2120374317 learned_addr learned my addr 172.21.15.62:0/2120374317 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45d600640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc440003f80 0x7fc4581f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45cc00640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc458160720 0x7fc45814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45e000640 1 -- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc440003f80 msgr2=0x7fc4581f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45e000640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc440003f80 0x7fc4581f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45e000640 1 -- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc458160720 msgr2=0x7fc45814df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45e000640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc458160720 0x7fc45814df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45e000640 1 -- 172.21.15.62:0/2120374317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45814e790 con 0x7fc4581ed810 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.870+0000 7fc45cc00640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc458160720 0x7fc45814df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.871+0000 7fc45e000640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4581ed810 0x7fc4581edc90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc45806e020 tx=0x7fc450009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.871+0000 7fc455600640 1 -- 172.21.15.62:0/2120374317 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450007d20 con 0x7fc4581ed810 2024-04-25T09:03:54.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.871+0000 7fc455600640 1 -- 172.21.15.62:0/2120374317 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc450019070 con 0x7fc4581ed810 2024-04-25T09:03:54.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.871+0000 7fc455600640 1 -- 172.21.15.62:0/2120374317 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450014650 con 0x7fc4581ed810 2024-04-25T09:03:54.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.871+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45814ea70 con 0x7fc4581ed810 2024-04-25T09:03:54.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.871+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45814f030 con 0x7fc4581ed810 2024-04-25T09:03:54.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.872+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45806abe0 con 0x7fc4581ed810 2024-04-25T09:03:54.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.873+0000 7fc455600640 1 -- 172.21.15.62:0/2120374317 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4500147f0 con 0x7fc4581ed810 2024-04-25T09:03:54.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.874+0000 7fc455600640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc43806b4d0 0x7fc43806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:54.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.874+0000 7fc455600640 1 -- 172.21.15.62:0/2120374317 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc45008a5a0 con 0x7fc4581ed810 2024-04-25T09:03:54.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.875+0000 7fc45d600640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc43806b4d0 0x7fc43806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:54.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.876+0000 7fc45d600640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc43806b4d0 0x7fc43806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009b20 tx=0x7fc44c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:54.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:54.878+0000 7fc455600640 1 -- 172.21.15.62:0/2120374317 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc450059410 con 0x7fc4581ed810 2024-04-25T09:03:55.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.085+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc45804ef20 con 0x7fc4581ed810 2024-04-25T09:03:55.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.086+0000 7fc455600640 1 -- 172.21.15.62:0/2120374317 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc450058d10 con 0x7fc4581ed810 2024-04-25T09:03:55.075 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:03:55.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc43806b4d0 msgr2=0x7fc43806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc43806b4d0 0x7fc43806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009b20 tx=0x7fc44c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:55.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4581ed810 msgr2=0x7fc4581edc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4581ed810 0x7fc4581edc90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc45806e020 tx=0x7fc450009cd0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 shutdown_connections 2024-04-25T09:03:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc43806b4d0 0x7fc43806d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4581ed810 0x7fc4581edc90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc458160720 0x7fc45814df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 --2- 172.21.15.62:0/2120374317 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc440003f80 0x7fc4581f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 >> 172.21.15.62:0/2120374317 conn(0x7fc458066f20 msgr2=0x7fc45806d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 shutdown_connections 2024-04-25T09:03:55.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.089+0000 7fc45ea00640 1 -- 172.21.15.62:0/2120374317 wait complete. 2024-04-25T09:03:55.109 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T09:03:55.109 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:03:55.109 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:55.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.334+0000 7f9afd000640 1 Processor -- start 2024-04-25T09:03:55.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.334+0000 7f9afd000640 1 -- start start 2024-04-25T09:03:55.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.336+0000 7f9afd000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af806d900 0x7f9af815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.336+0000 7f9afd000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80cf2d0 con 0x7f9af8144c10 2024-04-25T09:03:55.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.336+0000 7f9afd000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af815faa0 con 0x7f9af80cee40 2024-04-25T09:03:55.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.336+0000 7f9afd000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8144ff0 con 0x7f9af806d900 2024-04-25T09:03:55.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.336+0000 7f9af7400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9af8144c10 0x7f9af806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40862/0 (socket says 172.21.15.62:40862) 2024-04-25T09:03:55.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.336+0000 7f9af7400640 1 -- 172.21.15.62:0/3416335051 learned_addr learned my addr 172.21.15.62:0/3416335051 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6a00640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af806d900 0x7f9af815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697936000 0 0) 0x7f9af80cf2d0 con 0x7f9af8144c10 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9adc003520 con 0x7f9af8144c10 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862780643 0 0) 0x7f9adc003520 con 0x7f9af8144c10 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af806d900 msgr2=0x7f9af815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6000640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af806d900 0x7f9af815f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 >> v1:172.21.15.121:6789/0 conn(0x7f9af80cee40 legacy=0x7f9af80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af815fd80 con 0x7f9af8144c10 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.337+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f9ae8001670 con 0x7f9af8144c10 2024-04-25T09:03:55.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 >> v1:172.21.15.62:6789/0 conn(0x7f9af8144c10 legacy=0x7f9af806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af6000640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9adc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af6000640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9adc004800 0x7f9adc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af6000640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af806d900 0x7f9adc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8144ff0 con 0x7f9adc0039e0 2024-04-25T09:03:55.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc00bb50 con 0x7f9adc004800 2024-04-25T09:03:55.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc00bd20 con 0x7f9af806d900 2024-04-25T09:03:55.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af7400640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9adc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af7e00640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af806d900 0x7f9adc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.338+0000 7f9af6a00640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9adc004800 0x7f9adc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.339+0000 7f9af7400640 1 -- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af806d900 msgr2=0x7f9adc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.339+0000 7f9af7400640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af806d900 0x7f9adc005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.339+0000 7f9af7400640 1 -- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9adc004800 msgr2=0x7f9adc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.339+0000 7f9af7400640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9adc004800 0x7f9adc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.339+0000 7f9af7400640 1 -- 172.21.15.62:0/3416335051 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc003520 con 0x7f9adc0039e0 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.340+0000 7f9af7400640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9adc003e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c270 tx=0x7f9adc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bd94e4c9ffecf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.340+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8007cf0 con 0x7f9adc0039e0 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.340+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8005af0 con 0x7f9adc0039e0 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.340+0000 7f9af6000640 1 -- 172.21.15.62:0/3416335051 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8007070 con 0x7f9adc0039e0 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.340+0000 7f9afd000640 1 -- 172.21.15.62:0/3416335051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 msgr2=0x7f9adc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.340+0000 7f9afd000640 1 --2- 172.21.15.62:0/3416335051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9adc003e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c270 tx=0x7f9adc005830 comp rx=0 tx=0).stop 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.340+0000 7f9af7e00640 1 -- 172.21.15.62:0/3416335051 reap_dead start 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.341+0000 7f9afd000640 1 -- 172.21.15.62:0/3416335051 shutdown_connections 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.341+0000 7f9afd000640 1 -- 172.21.15.62:0/3416335051 >> 172.21.15.62:0/3416335051 conn(0x7f9af80b14d0 msgr2=0x7f9af80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.341+0000 7f9afd000640 1 -- 172.21.15.62:0/3416335051 shutdown_connections 2024-04-25T09:03:55.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.341+0000 7f9afd000640 1 -- 172.21.15.62:0/3416335051 wait complete. 2024-04-25T09:03:55.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.342+0000 7f9afd000640 1 Processor -- start 2024-04-25T09:03:55.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.343+0000 7f9afd000640 1 -- start start 2024-04-25T09:03:55.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9afd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9af81403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9afd000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af80cd0f0 0x7f9af8141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9afd000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af815b4c0 0x7f9af8142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9afd000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80b41c0 con 0x7f9adc0039e0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9afd000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8160340 con 0x7f9af815b4c0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9afd000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815f450 con 0x7f9af80cd0f0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9af7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9af81403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9af7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9af81403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46326/0 (socket says 172.21.15.62:46326) 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9af7400640 1 -- 172.21.15.62:0/4063463621 learned_addr learned my addr 172.21.15.62:0/4063463621 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9af7400640 1 -- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af80cd0f0 msgr2=0x7f9af8141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9af7400640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af80cd0f0 0x7f9af8141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9af7400640 1 -- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af815b4c0 msgr2=0x7f9af8142390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9af7400640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af815b4c0 0x7f9af8142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.344+0000 7f9af7400640 1 -- 172.21.15.62:0/4063463621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80cdbb0 con 0x7f9adc0039e0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.345+0000 7f9af7400640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9af81403b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c270 tx=0x7f9af815fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.345+0000 7f9aef400640 1 -- 172.21.15.62:0/4063463621 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8041030 con 0x7f9adc0039e0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.345+0000 7f9aef400640 1 -- 172.21.15.62:0/4063463621 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8000ed0 con 0x7f9adc0039e0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.345+0000 7f9aef400640 1 -- 172.21.15.62:0/4063463621 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8038040 con 0x7f9adc0039e0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.345+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af80cde90 con 0x7f9adc0039e0 2024-04-25T09:03:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.345+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af8140010 con 0x7f9adc0039e0 2024-04-25T09:03:55.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.346+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af804ef20 con 0x7f9adc0039e0 2024-04-25T09:03:55.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.348+0000 7f9aef400640 1 -- 172.21.15.62:0/4063463621 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ae8044070 con 0x7f9adc0039e0 2024-04-25T09:03:55.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.349+0000 7f9aef400640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ad806b4d0 0x7f9ad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.349+0000 7f9aef400640 1 -- 172.21.15.62:0/4063463621 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ae80a9eb0 con 0x7f9adc0039e0 2024-04-25T09:03:55.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.349+0000 7f9af6a00640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ad806b4d0 0x7f9ad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.350+0000 7f9af6a00640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ad806b4d0 0x7f9ad806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9af80cbc40 tx=0x7f9ae4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.353+0000 7f9aef400640 1 -- 172.21.15.62:0/4063463621 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ae8078d20 con 0x7f9adc0039e0 2024-04-25T09:03:55.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.556+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9af80cd860 con 0x7f9adc0039e0 2024-04-25T09:03:55.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.558+0000 7f9aef400640 1 -- 172.21.15.62:0/4063463621 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f9ae8078620 con 0x7f9adc0039e0 2024-04-25T09:03:55.548 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:55.548 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:18.143200+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ad806b4d0 msgr2=0x7f9ad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ad806b4d0 0x7f9ad806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9af80cbc40 tx=0x7f9ae4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 msgr2=0x7f9af81403b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9af81403b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c270 tx=0x7f9af815fc70 comp rx=0 tx=0).stop 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 shutdown_connections 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ad806b4d0 0x7f9ad806d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9af815b4c0 0x7f9af8142390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9af80cd0f0 0x7f9af8141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 --2- 172.21.15.62:0/4063463621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9adc0039e0 0x7f9af81403b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.561+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 >> 172.21.15.62:0/4063463621 conn(0x7f9af80b14d0 msgr2=0x7f9af80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.562+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 shutdown_connections 2024-04-25T09:03:55.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.562+0000 7f9afd000640 1 -- 172.21.15.62:0/4063463621 wait complete. 2024-04-25T09:03:55.572 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:55.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.804+0000 7fbca7e00640 1 Processor -- start 2024-04-25T09:03:55.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.804+0000 7fbca7e00640 1 -- start start 2024-04-25T09:03:55.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.806+0000 7fbca7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbca80ceb50 0x7fbca806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.806+0000 7fbca7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbca815e7c0 0x7fbca80b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.806+0000 7fbca7e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca815ee30 con 0x7fbca806d800 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.806+0000 7fbca7e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca81618b0 con 0x7fbca815e7c0 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.806+0000 7fbca7e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca8161b90 con 0x7fbca80ceb50 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.806+0000 7fbca7400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbca815e7c0 0x7fbca80b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.806+0000 7fbca7400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbca815e7c0 0x7fbca80b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46334/0 (socket says 172.21.15.62:46334) 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.806+0000 7fbca7400640 1 -- 172.21.15.62:0/1455213871 learned_addr learned my addr 172.21.15.62:0/1455213871 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.807+0000 7fbca6a00640 1 --2- 172.21.15.62:0/1455213871 >> v2:172.21.15.121:3300/0 conn(0x7fbca80ceb50 0x7fbca806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.807+0000 7fbca7400640 1 -- 172.21.15.62:0/1455213871 >> v2:172.21.15.121:3300/0 conn(0x7fbca80ceb50 msgr2=0x7fbca806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.807+0000 7fbca7400640 1 --2- 172.21.15.62:0/1455213871 >> v2:172.21.15.121:3300/0 conn(0x7fbca80ceb50 0x7fbca806d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.807+0000 7fbca7400640 1 -- 172.21.15.62:0/1455213871 >> v1:172.21.15.121:6789/0 conn(0x7fbca806d800 legacy=0x7fbca815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.807+0000 7fbca7400640 1 -- 172.21.15.62:0/1455213871 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80cef30 con 0x7fbca815e7c0 2024-04-25T09:03:55.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.808+0000 7fbca7400640 1 --2- 172.21.15.62:0/1455213871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca80b4550 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbca80af200 tx=0x7fbca002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12d90b00839ac66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.808+0000 7fbca6000640 1 -- 172.21.15.62:0/1455213871 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0037070 con 0x7fbca815e7c0 2024-04-25T09:03:55.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.808+0000 7fbca6000640 1 -- 172.21.15.62:0/1455213871 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca0032440 con 0x7fbca815e7c0 2024-04-25T09:03:55.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.808+0000 7fbca6000640 1 -- 172.21.15.62:0/1455213871 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0032810 con 0x7fbca815e7c0 2024-04-25T09:03:55.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.809+0000 7fbca7e00640 1 -- 172.21.15.62:0/1455213871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 msgr2=0x7fbca80b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.809+0000 7fbca7e00640 1 --2- 172.21.15.62:0/1455213871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca80b4550 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbca80af200 tx=0x7fbca002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.809+0000 7fbca7e00640 1 -- 172.21.15.62:0/1455213871 shutdown_connections 2024-04-25T09:03:55.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.809+0000 7fbca7e00640 1 --2- 172.21.15.62:0/1455213871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca80b4550 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.809+0000 7fbca7e00640 1 --2- 172.21.15.62:0/1455213871 >> v2:172.21.15.121:3300/0 conn(0x7fbca80ceb50 0x7fbca806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.809+0000 7fbca7e00640 1 -- 172.21.15.62:0/1455213871 >> 172.21.15.62:0/1455213871 conn(0x7fbca8066f20 msgr2=0x7fbca80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.809+0000 7fbca7e00640 1 -- 172.21.15.62:0/1455213871 shutdown_connections 2024-04-25T09:03:55.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.809+0000 7fbca7e00640 1 -- 172.21.15.62:0/1455213871 wait complete. 2024-04-25T09:03:55.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.811+0000 7fbca7e00640 1 Processor -- start 2024-04-25T09:03:55.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.811+0000 7fbca7e00640 1 -- start start 2024-04-25T09:03:55.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.812+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbca806d800 0x7fbca81f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.812+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbca80ceb50 0x7fbca8157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.812+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca8157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.812+0000 7fbca7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80b4a90 con 0x7fbca815e7c0 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.812+0000 7fbca7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca81625e0 con 0x7fbca806d800 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.812+0000 7fbca7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80b4c10 con 0x7fbca80ceb50 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.812+0000 7fbca7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca8157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.812+0000 7fbca7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca8157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46348/0 (socket says 172.21.15.62:46348) 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbca7400640 1 -- 172.21.15.62:0/651940180 learned_addr learned my addr 172.21.15.62:0/651940180 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbc9e000640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbca80ceb50 0x7fbca8157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbca6a00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbca806d800 0x7fbca81f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbca7400640 1 -- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbca80ceb50 msgr2=0x7fbca8157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbca7400640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbca80ceb50 0x7fbca8157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbca7400640 1 -- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbca806d800 msgr2=0x7fbca81f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbca7400640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbca806d800 0x7fbca81f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbca7400640 1 -- 172.21.15.62:0/651940180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8156990 con 0x7fbca815e7c0 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.813+0000 7fbca7400640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca8157f50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbca00090b0 tx=0x7fbca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.814+0000 7fbc9f400640 1 -- 172.21.15.62:0/651940180 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0041020 con 0x7fbca815e7c0 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.814+0000 7fbc9f400640 1 -- 172.21.15.62:0/651940180 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca0030990 con 0x7fbca815e7c0 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.814+0000 7fbc9f400640 1 -- 172.21.15.62:0/651940180 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0037040 con 0x7fbca815e7c0 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.814+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca8156c50 con 0x7fbca815e7c0 2024-04-25T09:03:55.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.814+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca8157210 con 0x7fbca815e7c0 2024-04-25T09:03:55.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.815+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca806abe0 con 0x7fbca815e7c0 2024-04-25T09:03:55.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.816+0000 7fbc9f400640 1 -- 172.21.15.62:0/651940180 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbca0044020 con 0x7fbca815e7c0 2024-04-25T09:03:55.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.817+0000 7fbc9f400640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc8406b4d0 0x7fbc8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.818+0000 7fbc9f400640 1 -- 172.21.15.62:0/651940180 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbca00aac30 con 0x7fbca815e7c0 2024-04-25T09:03:55.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.818+0000 7fbca6a00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc8406b4d0 0x7fbc8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.819+0000 7fbca6a00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc8406b4d0 0x7fbc8406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbca81564e0 tx=0x7fbc98008460 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:55.822+0000 7fbc9f400640 1 -- 172.21.15.62:0/651940180 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbca004b030 con 0x7fbca815e7c0 2024-04-25T09:03:56.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.024+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbca8147400 con 0x7fbca815e7c0 2024-04-25T09:03:56.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.025+0000 7fbc9f400640 1 -- 172.21.15.62:0/651940180 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbca0079a10 con 0x7fbca815e7c0 2024-04-25T09:03:56.014 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc8406b4d0 msgr2=0x7fbc8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc8406b4d0 0x7fbc8406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbca81564e0 tx=0x7fbc98008460 comp rx=0 tx=0).stop 2024-04-25T09:03:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 msgr2=0x7fbca8157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca8157f50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbca00090b0 tx=0x7fbca00046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 shutdown_connections 2024-04-25T09:03:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc8406b4d0 0x7fbc8406d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbca815e7c0 0x7fbca8157f50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbca80ceb50 0x7fbca8157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.027+0000 7fbca7e00640 1 --2- 172.21.15.62:0/651940180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbca806d800 0x7fbca81f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.028+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 >> 172.21.15.62:0/651940180 conn(0x7fbca8066f20 msgr2=0x7fbca8068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.028+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 shutdown_connections 2024-04-25T09:03:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.028+0000 7fbca7e00640 1 -- 172.21.15.62:0/651940180 wait complete. 2024-04-25T09:03:56.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.268+0000 7f5d62800640 1 Processor -- start 2024-04-25T09:03:56.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.269+0000 7f5d62800640 1 -- start start 2024-04-25T09:03:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.270+0000 7f5d62800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5d5c15dd50 0x7f5d5c0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.270+0000 7f5d62800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5c0ce780 con 0x7f5d5c0d2fb0 2024-04-25T09:03:56.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.270+0000 7f5d62800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5c0d3440 con 0x7f5d5c0d3960 2024-04-25T09:03:56.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.270+0000 7f5d62800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c0ce1c0 con 0x7f5d5c15dd50 2024-04-25T09:03:56.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.271+0000 7f5d60a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5d5c0d2fb0 0x7f5d5c0ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40868/0 (socket says 172.21.15.62:40868) 2024-04-25T09:03:56.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.271+0000 7f5d60a00640 1 -- 172.21.15.62:0/3755836949 learned_addr learned my addr 172.21.15.62:0/3755836949 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.271+0000 7f5d61400640 1 --2- 172.21.15.62:0/3755836949 >> v2:172.21.15.121:3300/0 conn(0x7f5d5c15dd50 0x7f5d5c0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.271+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695900888 0 0) 0x7f5d5c0ce780 con 0x7f5d5c0d2fb0 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.271+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d44003520 con 0x7f5d5c0d2fb0 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3124173180 0 0) 0x7f5d44003520 con 0x7f5d5c0d2fb0 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 >> v2:172.21.15.121:3300/0 conn(0x7f5d5c15dd50 msgr2=0x7f5d5c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 --2- 172.21.15.62:0/3755836949 >> v2:172.21.15.121:3300/0 conn(0x7f5d5c15dd50 0x7f5d5c0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 >> v1:172.21.15.121:6789/0 conn(0x7f5d5c0d3960 legacy=0x7f5d5c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c0ce4a0 con 0x7f5d5c0d2fb0 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f5d500019f0 con 0x7f5d5c0d2fb0 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 >> v1:172.21.15.62:6789/0 conn(0x7f5d5c0d2fb0 legacy=0x7f5d5c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d440039b0 0x7f5d44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d44004800 0x7f5d44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0ce1c0 con 0x7f5d440051c0 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4400c060 con 0x7f5d440039b0 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4400c260 con 0x7f5d44004800 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.272+0000 7f5d61e00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.273+0000 7f5d61400640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d440039b0 0x7f5d44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.273+0000 7f5d60a00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d44004800 0x7f5d44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.273+0000 7f5d61e00640 1 -- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d44004800 msgr2=0x7f5d44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.273+0000 7f5d61e00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d44004800 0x7f5d44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.273+0000 7f5d61e00640 1 -- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d440039b0 msgr2=0x7f5d44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.273+0000 7f5d61e00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d440039b0 0x7f5d44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.273+0000 7f5d61e00640 1 -- 172.21.15.62:0/3755836949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44003520 con 0x7f5d440051c0 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.273+0000 7f5d60a00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d44004800 0x7f5d44004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.274+0000 7f5d61e00640 1 -- 172.21.15.62:0/3755836949 reap_dead start 2024-04-25T09:03:56.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.274+0000 7f5d61e00640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d4400b9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d54001750 tx=0x7f5d44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ec075d460162f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.274+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54001370 con 0x7f5d440051c0 2024-04-25T09:03:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.274+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d54005310 con 0x7f5d440051c0 2024-04-25T09:03:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.274+0000 7f5d5be00640 1 -- 172.21.15.62:0/3755836949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5402abe0 con 0x7f5d440051c0 2024-04-25T09:03:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.275+0000 7f5d62800640 1 -- 172.21.15.62:0/3755836949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 msgr2=0x7f5d4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.275+0000 7f5d62800640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d4400b9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5d54001750 tx=0x7f5d44005830 comp rx=0 tx=0).stop 2024-04-25T09:03:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.275+0000 7f5d62800640 1 -- 172.21.15.62:0/3755836949 shutdown_connections 2024-04-25T09:03:56.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.275+0000 7f5d62800640 1 --2- 172.21.15.62:0/3755836949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d4400b9c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.275+0000 7f5d62800640 1 -- 172.21.15.62:0/3755836949 >> 172.21.15.62:0/3755836949 conn(0x7f5d5c06b000 msgr2=0x7f5d5c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.275+0000 7f5d62800640 1 -- 172.21.15.62:0/3755836949 shutdown_connections 2024-04-25T09:03:56.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.275+0000 7f5d62800640 1 -- 172.21.15.62:0/3755836949 wait complete. 2024-04-25T09:03:56.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.277+0000 7f5d62800640 1 Processor -- start 2024-04-25T09:03:56.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.277+0000 7f5d62800640 1 -- start start 2024-04-25T09:03:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d62800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d5c15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d62800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d5c15a300 0x7f5d5c0c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d62800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c1e8770 0x7f5d5c1e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d62800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0d4140 con 0x7f5d440051c0 2024-04-25T09:03:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d62800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0d3fc0 con 0x7f5d5c1e8770 2024-04-25T09:03:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d62800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c0d42c0 con 0x7f5d5c15a300 2024-04-25T09:03:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d61400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d5c15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d61400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d5c15b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46374/0 (socket says 172.21.15.62:46374) 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d61400640 1 -- 172.21.15.62:0/1871468490 learned_addr learned my addr 172.21.15.62:0/1871468490 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d60a00640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d5c15a300 0x7f5d5c0c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.278+0000 7f5d61400640 1 -- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d5c15a300 msgr2=0x7f5d5c0c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d61e00640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c1e8770 0x7f5d5c1e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d61400640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d5c15a300 0x7f5d5c0c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d61400640 1 -- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c1e8770 msgr2=0x7f5d5c1e8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d61400640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c1e8770 0x7f5d5c1e8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d61400640 1 -- 172.21.15.62:0/1871468490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c159560 con 0x7f5d440051c0 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d60a00640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d5c15a300 0x7f5d5c0c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d61400640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d5c15b8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0115a0 tx=0x7f5d4c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d59600640 1 -- 172.21.15.62:0/1871468490 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c015930 con 0x7f5d440051c0 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d59600640 1 -- 172.21.15.62:0/1871468490 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c015ad0 con 0x7f5d440051c0 2024-04-25T09:03:56.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d59600640 1 -- 172.21.15.62:0/1871468490 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00f480 con 0x7f5d440051c0 2024-04-25T09:03:56.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1e7eb0 con 0x7f5d440051c0 2024-04-25T09:03:56.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.279+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c159b00 con 0x7f5d440051c0 2024-04-25T09:03:56.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.281+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c074790 con 0x7f5d440051c0 2024-04-25T09:03:56.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.282+0000 7f5d59600640 1 -- 172.21.15.62:0/1871468490 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d4c00f620 con 0x7f5d440051c0 2024-04-25T09:03:56.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.283+0000 7f5d59600640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d4006b500 0x7f5d4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.283+0000 7f5d59600640 1 -- 172.21.15.62:0/1871468490 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d4c08a860 con 0x7f5d440051c0 2024-04-25T09:03:56.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.283+0000 7f5d60a00640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d4006b500 0x7f5d4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.285+0000 7f5d60a00640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d4006b500 0x7f5d4006d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d50001580 tx=0x7f5d50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.287+0000 7f5d59600640 1 -- 172.21.15.62:0/1871468490 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d5c074790 con 0x7f5d440051c0 2024-04-25T09:03:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.490+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d5c04ef20 con 0x7f5d440051c0 2024-04-25T09:03:56.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.491+0000 7f5d59600640 1 -- 172.21.15.62:0/1871468490 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d4c019030 con 0x7f5d440051c0 2024-04-25T09:03:56.480 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:03:56.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d4006b500 msgr2=0x7f5d4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d4006b500 0x7f5d4006d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d50001580 tx=0x7f5d50009040 comp rx=0 tx=0).stop 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 msgr2=0x7f5d5c15b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d5c15b8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0115a0 tx=0x7f5d4c011a60 comp rx=0 tx=0).stop 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 shutdown_connections 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d4006b500 0x7f5d4006d9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d5c1e8770 0x7f5d5c1e8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d5c15a300 0x7f5d5c0c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 --2- 172.21.15.62:0/1871468490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d440051c0 0x7f5d5c15b8b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 >> 172.21.15.62:0/1871468490 conn(0x7f5d5c06b000 msgr2=0x7f5d5c15be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 shutdown_connections 2024-04-25T09:03:56.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.494+0000 7f5d62800640 1 -- 172.21.15.62:0/1871468490 wait complete. 2024-04-25T09:03:56.501 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:56.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.728+0000 7f16b8e00640 1 Processor -- start 2024-04-25T09:03:56.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.728+0000 7f16b8e00640 1 -- start start 2024-04-25T09:03:56.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.729+0000 7f16b8e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16b40ce990 0x7f16b40cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.729+0000 7f16b8e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 0x7f16b4162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.729+0000 7f16b8e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b40d1c90 con 0x7f16b415dd60 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.729+0000 7f16b8e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b40d1f70 con 0x7f16b40ce990 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.729+0000 7f16b8e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b40d2250 con 0x7f16b4161bf0 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b3400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f16b415dd60 0x7f16b40d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40876/0 (socket says 172.21.15.62:40876) 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b3400640 1 -- 172.21.15.62:0/3857118475 learned_addr learned my addr 172.21.15.62:0/3857118475 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b2a00640 1 --2- 172.21.15.62:0/3857118475 >> v2:172.21.15.62:3300/0 conn(0x7f16b40ce990 0x7f16b40cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b3e00640 1 --2- 172.21.15.62:0/3857118475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 0x7f16b4162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b2000640 1 -- 172.21.15.62:0/3857118475 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691863059 0 0) 0x7f16b40d1c90 con 0x7f16b415dd60 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b2000640 1 -- 172.21.15.62:0/3857118475 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1698003520 con 0x7f16b415dd60 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b2a00640 1 -- 172.21.15.62:0/3857118475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 msgr2=0x7f16b4162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b2a00640 1 --2- 172.21.15.62:0/3857118475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 0x7f16b4162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b2a00640 1 -- 172.21.15.62:0/3857118475 >> v1:172.21.15.62:6789/0 conn(0x7f16b415dd60 legacy=0x7f16b40d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.730+0000 7f16b2a00640 1 -- 172.21.15.62:0/3857118475 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b40d2530 con 0x7f16b40ce990 2024-04-25T09:03:56.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.732+0000 7f16b2a00640 1 --2- 172.21.15.62:0/3857118475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b40ce990 0x7f16b40cee20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f169c0097a0 tx=0x7f169c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adad4d41e2a3da6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.732+0000 7f16b2000640 1 -- 172.21.15.62:0/3857118475 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c037070 con 0x7f16b40ce990 2024-04-25T09:03:56.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.732+0000 7f16b2000640 1 -- 172.21.15.62:0/3857118475 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c02ae50 con 0x7f16b40ce990 2024-04-25T09:03:56.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.732+0000 7f16b2000640 1 -- 172.21.15.62:0/3857118475 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c032660 con 0x7f16b40ce990 2024-04-25T09:03:56.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.732+0000 7f16b8e00640 1 -- 172.21.15.62:0/3857118475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b40ce990 msgr2=0x7f16b40cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.732+0000 7f16b8e00640 1 --2- 172.21.15.62:0/3857118475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b40ce990 0x7f16b40cee20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f169c0097a0 tx=0x7f169c02a100 comp rx=0 tx=0).stop 2024-04-25T09:03:56.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.733+0000 7f16b8e00640 1 -- 172.21.15.62:0/3857118475 shutdown_connections 2024-04-25T09:03:56.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.733+0000 7f16b8e00640 1 --2- 172.21.15.62:0/3857118475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 0x7f16b4162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.733+0000 7f16b8e00640 1 --2- 172.21.15.62:0/3857118475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b40ce990 0x7f16b40cee20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.733+0000 7f16b8e00640 1 -- 172.21.15.62:0/3857118475 >> 172.21.15.62:0/3857118475 conn(0x7f16b406b210 msgr2=0x7f16b40668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.733+0000 7f16b8e00640 1 -- 172.21.15.62:0/3857118475 shutdown_connections 2024-04-25T09:03:56.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.733+0000 7f16b8e00640 1 -- 172.21.15.62:0/3857118475 wait complete. 2024-04-25T09:03:56.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.734+0000 7f16b8e00640 1 Processor -- start 2024-04-25T09:03:56.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.735+0000 7f16b8e00640 1 -- start start 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.735+0000 7f16b8e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b40ce990 0x7f16b40b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.735+0000 7f16b8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b415dd60 0x7f16b41f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.735+0000 7f16b8e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 0x7f16b41e8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b8e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b4162740 con 0x7f16b415dd60 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b8e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b41625c0 con 0x7f16b40ce990 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b8e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b40b6dd0 con 0x7f16b4161bf0 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b415dd60 0x7f16b41f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b415dd60 0x7f16b41f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46388/0 (socket says 172.21.15.62:46388) 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b2a00640 1 -- 172.21.15.62:0/1578683253 learned_addr learned my addr 172.21.15.62:0/1578683253 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b3e00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 0x7f16b41e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b2a00640 1 -- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 msgr2=0x7f16b41e8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b3400640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b40ce990 0x7f16b40b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b2a00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 0x7f16b41e8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b2a00640 1 -- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b40ce990 msgr2=0x7f16b40b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b2a00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b40ce990 0x7f16b40b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.736+0000 7f16b2a00640 1 -- 172.21.15.62:0/1578683253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b40b5c20 con 0x7f16b415dd60 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.737+0000 7f16b2a00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b415dd60 0x7f16b41f4260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f169c035040 tx=0x7f169c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.737+0000 7f16ab400640 1 -- 172.21.15.62:0/1578683253 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c041070 con 0x7f16b415dd60 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.737+0000 7f16ab400640 1 -- 172.21.15.62:0/1578683253 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c03c460 con 0x7f16b415dd60 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.737+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b41f4d70 con 0x7f16b415dd60 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.737+0000 7f16ab400640 1 -- 172.21.15.62:0/1578683253 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c037040 con 0x7f16b415dd60 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.737+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b40b4ee0 con 0x7f16b415dd60 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.738+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b40b4cd0 con 0x7f16b415dd60 2024-04-25T09:03:56.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.739+0000 7f16ab400640 1 -- 172.21.15.62:0/1578683253 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f169c03a070 con 0x7f16b415dd60 2024-04-25T09:03:56.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.741+0000 7f16ab400640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.741+0000 7f16ab400640 1 -- 172.21.15.62:0/1578683253 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f169c0aa750 con 0x7f16b415dd60 2024-04-25T09:03:56.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.741+0000 7f16b3400640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.743+0000 7f16b3400640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f169006b4d0 0x7f169006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16a4000f60 tx=0x7f16a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.745+0000 7f16ab400640 1 -- 172.21.15.62:0/1578683253 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f169c079530 con 0x7f16b415dd60 2024-04-25T09:03:56.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.944+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16b40750e0 con 0x7f16b415dd60 2024-04-25T09:03:56.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.945+0000 7f16ab400640 1 -- 172.21.15.62:0/1578683253 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f169c078e30 con 0x7f16b415dd60 2024-04-25T09:03:56.934 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:03:56.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.947+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f169006b4d0 msgr2=0x7f169006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.947+0000 7f16b8e00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f169006b4d0 0x7f169006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f16a4000f60 tx=0x7f16a4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:56.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.947+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b415dd60 msgr2=0x7f16b41f4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.947+0000 7f16b8e00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b415dd60 0x7f16b41f4260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f169c035040 tx=0x7f169c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.948+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 shutdown_connections 2024-04-25T09:03:56.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.948+0000 7f16b8e00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.948+0000 7f16b8e00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16b415dd60 0x7f16b41f4260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.948+0000 7f16b8e00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16b4161bf0 0x7f16b41e8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.948+0000 7f16b8e00640 1 --2- 172.21.15.62:0/1578683253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16b40ce990 0x7f16b40b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.948+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 >> 172.21.15.62:0/1578683253 conn(0x7f16b406b210 msgr2=0x7f16b40cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.948+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 shutdown_connections 2024-04-25T09:03:56.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:56.948+0000 7f16b8e00640 1 -- 172.21.15.62:0/1578683253 wait complete. 2024-04-25T09:03:56.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T09:03:57.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.184+0000 7f8d58800640 1 Processor -- start 2024-04-25T09:03:57.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.184+0000 7f8d58800640 1 -- start start 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.185+0000 7f8d58800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d50072340 0x7f8d500702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d58800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8d500707f0 0x7f8d500b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d58800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5006c310 con 0x7f8d500cfed0 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d58800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5006c5b0 con 0x7f8d500707f0 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d58800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5006c890 con 0x7f8d50072340 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d55e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8d500cfed0 0x7f8d5015f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40884/0 (socket says 172.21.15.62:40884) 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d55e00640 1 -- 172.21.15.62:0/3921784992 learned_addr learned my addr 172.21.15.62:0/3921784992 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:57.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d56800640 1 --2- 172.21.15.62:0/3921784992 >> v2:172.21.15.62:3300/0 conn(0x7f8d500707f0 0x7f8d500b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d55400640 1 --2- 172.21.15.62:0/3921784992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d50072340 0x7f8d500702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d54a00640 1 -- 172.21.15.62:0/3921784992 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907513695 0 0) 0x7f8d5006c310 con 0x7f8d500cfed0 2024-04-25T09:03:57.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d54a00640 1 -- 172.21.15.62:0/3921784992 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d38003520 con 0x7f8d500cfed0 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d56800640 1 -- 172.21.15.62:0/3921784992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d50072340 msgr2=0x7f8d500702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.186+0000 7f8d56800640 1 --2- 172.21.15.62:0/3921784992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d50072340 0x7f8d500702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.187+0000 7f8d56800640 1 -- 172.21.15.62:0/3921784992 >> v1:172.21.15.62:6789/0 conn(0x7f8d500cfed0 legacy=0x7f8d5015f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.187+0000 7f8d56800640 1 -- 172.21.15.62:0/3921784992 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5006cb70 con 0x7f8d500707f0 2024-04-25T09:03:57.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d56800640 1 --2- 172.21.15.62:0/3921784992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d500707f0 0x7f8d500b2880 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009d90 tx=0x7f8d4c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75fab8b1673142d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d54a00640 1 -- 172.21.15.62:0/3921784992 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c007740 con 0x7f8d500707f0 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d54a00640 1 -- 172.21.15.62:0/3921784992 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c0078e0 con 0x7f8d500707f0 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d54a00640 1 -- 172.21.15.62:0/3921784992 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c007cb0 con 0x7f8d500707f0 2024-04-25T09:03:57.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d58800640 1 -- 172.21.15.62:0/3921784992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d500707f0 msgr2=0x7f8d500b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d58800640 1 --2- 172.21.15.62:0/3921784992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d500707f0 0x7f8d500b2880 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009d90 tx=0x7f8d4c02a660 comp rx=0 tx=0).stop 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d58800640 1 -- 172.21.15.62:0/3921784992 shutdown_connections 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d58800640 1 --2- 172.21.15.62:0/3921784992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d500707f0 0x7f8d500b2880 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d58800640 1 --2- 172.21.15.62:0/3921784992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d50072340 0x7f8d500702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.188+0000 7f8d58800640 1 -- 172.21.15.62:0/3921784992 >> 172.21.15.62:0/3921784992 conn(0x7f8d500668f0 msgr2=0x7f8d50066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.189+0000 7f8d58800640 1 -- 172.21.15.62:0/3921784992 shutdown_connections 2024-04-25T09:03:57.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.189+0000 7f8d58800640 1 -- 172.21.15.62:0/3921784992 wait complete. 2024-04-25T09:03:57.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.190+0000 7f8d58800640 1 Processor -- start 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.190+0000 7f8d58800640 1 -- start start 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.191+0000 7f8d58800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d500707f0 0x7f8d50142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.191+0000 7f8d58800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d500cfed0 0x7f8d5015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.191+0000 7f8d58800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5014fe10 0x7f8d50150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.191+0000 7f8d58800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500b2dc0 con 0x7f8d5014fe10 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.191+0000 7f8d58800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5006d130 con 0x7f8d500cfed0 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.191+0000 7f8d58800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500b2f40 con 0x7f8d500707f0 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d56800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5014fe10 0x7f8d50150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d56800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5014fe10 0x7f8d50150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46414/0 (socket says 172.21.15.62:46414) 2024-04-25T09:03:57.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d56800640 1 -- 172.21.15.62:0/2417478136 learned_addr learned my addr 172.21.15.62:0/2417478136 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d55e00640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d500707f0 0x7f8d50142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d55400640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d500cfed0 0x7f8d5015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d56800640 1 -- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d500707f0 msgr2=0x7f8d50142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d56800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d500707f0 0x7f8d50142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d56800640 1 -- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d500cfed0 msgr2=0x7f8d5015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d56800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d500cfed0 0x7f8d5015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d56800640 1 -- 172.21.15.62:0/2417478136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5015c940 con 0x7f8d5014fe10 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.192+0000 7f8d55400640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d500cfed0 0x7f8d5015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.193+0000 7f8d56800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5014fe10 0x7f8d50150260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009eb0 tx=0x7f8d4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.193+0000 7f8d4a000640 1 -- 172.21.15.62:0/2417478136 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c040070 con 0x7f8d5014fe10 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.193+0000 7f8d4a000640 1 -- 172.21.15.62:0/2417478136 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c031bc0 con 0x7f8d5014fe10 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.193+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5015cc20 con 0x7f8d5014fe10 2024-04-25T09:03:57.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.193+0000 7f8d4a000640 1 -- 172.21.15.62:0/2417478136 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c009450 con 0x7f8d5014fe10 2024-04-25T09:03:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.193+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5015d1e0 con 0x7f8d5014fe10 2024-04-25T09:03:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.194+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d50072c70 con 0x7f8d5014fe10 2024-04-25T09:03:57.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.195+0000 7f8d4a000640 1 -- 172.21.15.62:0/2417478136 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d4c039040 con 0x7f8d5014fe10 2024-04-25T09:03:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.196+0000 7f8d4a000640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d3406b4d0 0x7f8d3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.196+0000 7f8d4a000640 1 -- 172.21.15.62:0/2417478136 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d4c0abb60 con 0x7f8d5014fe10 2024-04-25T09:03:57.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.197+0000 7f8d55e00640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d3406b4d0 0x7f8d3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.198+0000 7f8d55e00640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d3406b4d0 0x7f8d3406d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005880 tx=0x7f8d44009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.200+0000 7f8d4a000640 1 -- 172.21.15.62:0/2417478136 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d4c07a940 con 0x7f8d5014fe10 2024-04-25T09:03:57.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.474+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f8d5006d950 con 0x7f8d5014fe10 2024-04-25T09:03:57.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.713+0000 7f8d4a000640 1 -- 172.21.15.62:0/2417478136 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v61) v1 ==== 151+0+0 (secure 0 0 0) 0x7f8d4c043e40 con 0x7f8d5014fe10 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.716+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d3406b4d0 msgr2=0x7f8d3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.716+0000 7f8d58800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d3406b4d0 0x7f8d3406d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005880 tx=0x7f8d44009040 comp rx=0 tx=0).stop 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.716+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5014fe10 msgr2=0x7f8d50150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.716+0000 7f8d58800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5014fe10 0x7f8d50150260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009eb0 tx=0x7f8d4c004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.717+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 shutdown_connections 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.717+0000 7f8d58800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d3406b4d0 0x7f8d3406d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.717+0000 7f8d58800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5014fe10 0x7f8d50150260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.717+0000 7f8d58800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d500cfed0 0x7f8d5015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.717+0000 7f8d58800640 1 --2- 172.21.15.62:0/2417478136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d500707f0 0x7f8d50142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.717+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 >> 172.21.15.62:0/2417478136 conn(0x7f8d500668f0 msgr2=0x7f8d500b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.717+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 shutdown_connections 2024-04-25T09:03:57.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.717+0000 7f8d58800640 1 -- 172.21.15.62:0/2417478136 wait complete. 2024-04-25T09:03:57.726 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:03:57.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.945+0000 7f8b07a00640 1 Processor -- start 2024-04-25T09:03:57.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.945+0000 7f8b07a00640 1 -- start start 2024-04-25T09:03:57.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8b07a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b000cebc0 0x7f8b0006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8b07a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8b000720c0 0x7f8b000703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8b07a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b00072510 con 0x7f8b0015bdf0 2024-04-25T09:03:57.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8b07a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0015c1d0 con 0x7f8b000720c0 2024-04-25T09:03:57.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8b07a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b000ae8c0 con 0x7f8b000cebc0 2024-04-25T09:03:57.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8b05000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8b0015bdf0 0x7f8b0006c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40890/0 (socket says 172.21.15.62:40890) 2024-04-25T09:03:57.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8b05000640 1 -- 172.21.15.62:0/177311971 learned_addr learned my addr 172.21.15.62:0/177311971 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:57.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8b05a00640 1 --2- 172.21.15.62:0/177311971 >> v2:172.21.15.62:3300/0 conn(0x7f8b000720c0 0x7f8b000703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.947+0000 7f8aff400640 1 -- 172.21.15.62:0/177311971 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282877203 0 0) 0x7f8b00072510 con 0x7f8b0015bdf0 2024-04-25T09:03:57.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.948+0000 7f8aff400640 1 -- 172.21.15.62:0/177311971 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae8003520 con 0x7f8b0015bdf0 2024-04-25T09:03:57.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.948+0000 7f8affe00640 1 --2- 172.21.15.62:0/177311971 >> v2:172.21.15.121:3300/0 conn(0x7f8b000cebc0 0x7f8b0006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.948+0000 7f8b05a00640 1 -- 172.21.15.62:0/177311971 >> v2:172.21.15.121:3300/0 conn(0x7f8b000cebc0 msgr2=0x7f8b0006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.948+0000 7f8b05a00640 1 --2- 172.21.15.62:0/177311971 >> v2:172.21.15.121:3300/0 conn(0x7f8b000cebc0 0x7f8b0006c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.948+0000 7f8b05a00640 1 -- 172.21.15.62:0/177311971 >> v1:172.21.15.62:6789/0 conn(0x7f8b0015bdf0 legacy=0x7f8b0006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.948+0000 7f8b05a00640 1 -- 172.21.15.62:0/177311971 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0006d070 con 0x7f8b000720c0 2024-04-25T09:03:57.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.949+0000 7f8b05a00640 1 --2- 172.21.15.62:0/177311971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b000720c0 0x7f8b000703a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b00068870 tx=0x7f8af802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a719c1727de6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.949+0000 7f8aff400640 1 -- 172.21.15.62:0/177311971 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8007740 con 0x7f8b000720c0 2024-04-25T09:03:57.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.949+0000 7f8aff400640 1 -- 172.21.15.62:0/177311971 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af80078e0 con 0x7f8b000720c0 2024-04-25T09:03:57.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.949+0000 7f8aff400640 1 -- 172.21.15.62:0/177311971 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8007c90 con 0x7f8b000720c0 2024-04-25T09:03:57.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.950+0000 7f8b07a00640 1 -- 172.21.15.62:0/177311971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b000720c0 msgr2=0x7f8b000703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.950+0000 7f8b07a00640 1 --2- 172.21.15.62:0/177311971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b000720c0 0x7f8b000703a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b00068870 tx=0x7f8af802a680 comp rx=0 tx=0).stop 2024-04-25T09:03:57.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.950+0000 7f8b07a00640 1 -- 172.21.15.62:0/177311971 shutdown_connections 2024-04-25T09:03:57.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.950+0000 7f8b07a00640 1 --2- 172.21.15.62:0/177311971 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b000720c0 0x7f8b000703a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.950+0000 7f8b07a00640 1 --2- 172.21.15.62:0/177311971 >> v2:172.21.15.121:3300/0 conn(0x7f8b000cebc0 0x7f8b0006c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.950+0000 7f8b07a00640 1 -- 172.21.15.62:0/177311971 >> 172.21.15.62:0/177311971 conn(0x7f8b000b1410 msgr2=0x7f8b000b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.950+0000 7f8b07a00640 1 -- 172.21.15.62:0/177311971 shutdown_connections 2024-04-25T09:03:57.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.950+0000 7f8b07a00640 1 -- 172.21.15.62:0/177311971 wait complete. 2024-04-25T09:03:57.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.952+0000 7f8b07a00640 1 Processor -- start 2024-04-25T09:03:57.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.952+0000 7f8b07a00640 1 -- start start 2024-04-25T09:03:57.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b07a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b000720c0 0x7f8b00156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b07a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b000cebc0 0x7f8b000b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b07a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b0015bdf0 0x7f8b00158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b07a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000aedf0 con 0x7f8b0015bdf0 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b07a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000aeb10 con 0x7f8b000cebc0 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b07a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000708e0 con 0x7f8b000720c0 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b05a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b0015bdf0 0x7f8b00158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b05a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b0015bdf0 0x7f8b00158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46430/0 (socket says 172.21.15.62:46430) 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8b05a00640 1 -- 172.21.15.62:0/2480067723 learned_addr learned my addr 172.21.15.62:0/2480067723 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.953+0000 7f8affe00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b000cebc0 0x7f8b000b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b05000640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b000720c0 0x7f8b00156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b05a00640 1 -- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b000720c0 msgr2=0x7f8b00156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b05a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b000720c0 0x7f8b00156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b05a00640 1 -- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b000cebc0 msgr2=0x7f8b000b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b05a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b000cebc0 0x7f8b000b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b05a00640 1 -- 172.21.15.62:0/2480067723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b000b39a0 con 0x7f8b0015bdf0 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b05000640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b000720c0 0x7f8b00156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:57.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b05a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b0015bdf0 0x7f8b00158840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8af802ac70 tx=0x7f8af8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8afcc00640 1 -- 172.21.15.62:0/2480067723 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8040070 con 0x7f8b0015bdf0 2024-04-25T09:03:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8afcc00640 1 -- 172.21.15.62:0/2480067723 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8031c60 con 0x7f8b0015bdf0 2024-04-25T09:03:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8afcc00640 1 -- 172.21.15.62:0/2480067723 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8009450 con 0x7f8b0015bdf0 2024-04-25T09:03:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b000b3c80 con 0x7f8b0015bdf0 2024-04-25T09:03:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.954+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b000b3510 con 0x7f8b0015bdf0 2024-04-25T09:03:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.955+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b000724a0 con 0x7f8b0015bdf0 2024-04-25T09:03:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.956+0000 7f8afcc00640 1 -- 172.21.15.62:0/2480067723 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8af8039040 con 0x7f8b0015bdf0 2024-04-25T09:03:57.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.957+0000 7f8afcc00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.957+0000 7f8afcc00640 1 -- 172.21.15.62:0/2480067723 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8af80ab3e0 con 0x7f8b0015bdf0 2024-04-25T09:03:57.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.958+0000 7f8b05000640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.959+0000 7f8b05000640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ae006b4d0 0x7f8ae006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8af4005880 tx=0x7f8af4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:57.962+0000 7f8afcc00640 1 -- 172.21.15.62:0/2480067723 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b000724a0 con 0x7f8b0015bdf0 2024-04-25T09:03:58.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.163+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8b0004ef20 con 0x7f8b0015bdf0 2024-04-25T09:03:58.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.168+0000 7f8afcc00640 1 -- 172.21.15.62:0/2480067723 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8af807a1c0 con 0x7f8b0015bdf0 2024-04-25T09:03:58.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ae006b4d0 msgr2=0x7f8ae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ae006b4d0 0x7f8ae006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8af4005880 tx=0x7f8af4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b0015bdf0 msgr2=0x7f8b00158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b0015bdf0 0x7f8b00158840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8af802ac70 tx=0x7f8af8004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 shutdown_connections 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b0015bdf0 0x7f8b00158840 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b000cebc0 0x7f8b000b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 --2- 172.21.15.62:0/2480067723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b000720c0 0x7f8b00156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 >> 172.21.15.62:0/2480067723 conn(0x7f8b000b1410 msgr2=0x7f8b000cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 shutdown_connections 2024-04-25T09:03:58.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.171+0000 7f8b07a00640 1 -- 172.21.15.62:0/2480067723 wait complete. 2024-04-25T09:03:58.180 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:03:58.180 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:58.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:58.396 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:03:58.397 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:03:58.397 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:58.457 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:03:58.457 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:58.511 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:03:58.511 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:58.565 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:58.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.784+0000 7f7cca400640 1 Processor -- start 2024-04-25T09:03:58.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.784+0000 7f7cca400640 1 -- start start 2024-04-25T09:03:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.785+0000 7f7cca400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc415dc40 0x7f7cc40d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.785+0000 7f7cca400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cc40d0ae0 0x7f7cc40d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.785+0000 7f7cca400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc40d2620 con 0x7f7cc40d3ae0 2024-04-25T09:03:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.785+0000 7f7cca400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc40d1480 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.785+0000 7f7cca400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40d2900 con 0x7f7cc415dc40 2024-04-25T09:03:58.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cc40d0ae0 0x7f7cc40d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cc40d0ae0 0x7f7cc40d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46438/0 (socket says 172.21.15.62:46438) 2024-04-25T09:03:58.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc2a00640 1 -- 172.21.15.62:0/1485201912 learned_addr learned my addr 172.21.15.62:0/1485201912 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:58.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc3400640 1 --2- 172.21.15.62:0/1485201912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc415dc40 0x7f7cc40d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc2a00640 1 -- 172.21.15.62:0/1485201912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc415dc40 msgr2=0x7f7cc40d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc2a00640 1 --2- 172.21.15.62:0/1485201912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc415dc40 0x7f7cc40d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc2a00640 1 -- 172.21.15.62:0/1485201912 >> v1:172.21.15.121:6789/0 conn(0x7f7cc40d3ae0 legacy=0x7f7cc40d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc2a00640 1 -- 172.21.15.62:0/1485201912 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40d2be0 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.786+0000 7f7cc3400640 1 --2- 172.21.15.62:0/1485201912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc415dc40 0x7f7cc40d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.787+0000 7f7cc2a00640 1 --2- 172.21.15.62:0/1485201912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc40d0f40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009870 tx=0x7f7cac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179d420dcc429bec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.788+0000 7f7cc2000640 1 -- 172.21.15.62:0/1485201912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac037070 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.788+0000 7f7cc2000640 1 -- 172.21.15.62:0/1485201912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac032440 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.788+0000 7f7cc2000640 1 -- 172.21.15.62:0/1485201912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac032810 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.789+0000 7f7cca400640 1 -- 172.21.15.62:0/1485201912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 msgr2=0x7f7cc40d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.789+0000 7f7cca400640 1 --2- 172.21.15.62:0/1485201912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc40d0f40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009870 tx=0x7f7cac02a210 comp rx=0 tx=0).stop 2024-04-25T09:03:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.789+0000 7f7cca400640 1 -- 172.21.15.62:0/1485201912 shutdown_connections 2024-04-25T09:03:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.789+0000 7f7cca400640 1 --2- 172.21.15.62:0/1485201912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc40d0f40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.789+0000 7f7cca400640 1 --2- 172.21.15.62:0/1485201912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc415dc40 0x7f7cc40d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.789+0000 7f7cca400640 1 -- 172.21.15.62:0/1485201912 >> 172.21.15.62:0/1485201912 conn(0x7f7cc406b000 msgr2=0x7f7cc406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.789+0000 7f7cca400640 1 -- 172.21.15.62:0/1485201912 shutdown_connections 2024-04-25T09:03:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.789+0000 7f7cca400640 1 -- 172.21.15.62:0/1485201912 wait complete. 2024-04-25T09:03:58.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.791+0000 7f7cca400640 1 Processor -- start 2024-04-25T09:03:58.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.791+0000 7f7cca400640 1 -- start start 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cca400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc41572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cca400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc40d3ae0 0x7f7cc41f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cca400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc415dc40 0x7f7cc4159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cca400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b7d00 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cca400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40d31a0 con 0x7f7cc415dc40 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cca400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40d36c0 con 0x7f7cc40d3ae0 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cc3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc41572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cc3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc41572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46442/0 (socket says 172.21.15.62:46442) 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.792+0000 7f7cc3400640 1 -- 172.21.15.62:0/4240471518 learned_addr learned my addr 172.21.15.62:0/4240471518 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc2a00640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc40d3ae0 0x7f7cc41f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc3e00640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc415dc40 0x7f7cc4159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc3400640 1 -- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc40d3ae0 msgr2=0x7f7cc41f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc3400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc40d3ae0 0x7f7cc41f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc3400640 1 -- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc415dc40 msgr2=0x7f7cc4159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc3400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc415dc40 0x7f7cc4159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc3400640 1 -- 172.21.15.62:0/4240471518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40cccc0 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc2a00640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc40d3ae0 0x7f7cc41f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cc3400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc41572e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400cf90 tx=0x7f7cb400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cbb400640 1 -- 172.21.15.62:0/4240471518 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb401a070 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cbb400640 1 -- 172.21.15.62:0/4240471518 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb400d840 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.793+0000 7f7cbb400640 1 -- 172.21.15.62:0/4240471518 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400dc10 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.794+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc40ccf60 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.794+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc41f4230 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.795+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc40cd350 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.796+0000 7f7cbb400640 1 -- 172.21.15.62:0/4240471518 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cb401d070 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.797+0000 7f7cbb400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ca006b4d0 0x7f7ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.797+0000 7f7cbb400640 1 -- 172.21.15.62:0/4240471518 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7cb408aa70 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.798+0000 7f7cc2a00640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ca006b4d0 0x7f7ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.799+0000 7f7cc2a00640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ca006b4d0 0x7f7ca006d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7cac034000 tx=0x7f7cac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:58.802+0000 7f7cbb400640 1 -- 172.21.15.62:0/4240471518 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cb40591e0 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.002+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cc40cd130 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.005+0000 7f7cbb400640 1 -- 172.21.15.62:0/4240471518 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f7cb405d280 con 0x7f7cc40d0ae0 2024-04-25T09:03:58.994 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:03:58.994 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":21,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:58.715037+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:58.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ca006b4d0 msgr2=0x7f7ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ca006b4d0 0x7f7ca006d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7cac034000 tx=0x7f7cac02e000 comp rx=0 tx=0).stop 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 msgr2=0x7f7cc41572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc41572e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400cf90 tx=0x7f7cb400a7d0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 shutdown_connections 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ca006b4d0 0x7f7ca006d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc415dc40 0x7f7cc4159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cc40d3ae0 0x7f7cc41f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 --2- 172.21.15.62:0/4240471518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cc40d0ae0 0x7f7cc41572e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.007+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 >> 172.21.15.62:0/4240471518 conn(0x7f7cc406b000 msgr2=0x7f7cc40b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.008+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 shutdown_connections 2024-04-25T09:03:58.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.008+0000 7f7cca400640 1 -- 172.21.15.62:0/4240471518 wait complete. 2024-04-25T09:03:59.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:59.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:59.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.238+0000 7f87d1000640 1 Processor -- start 2024-04-25T09:03:59.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.239+0000 7f87d1000640 1 -- start start 2024-04-25T09:03:59.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.240+0000 7f87d1000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f87cc0ceb50 0x7f87cc06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.240+0000 7f87d1000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc15ee30 con 0x7f87cc15e7c0 2024-04-25T09:03:59.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.240+0000 7f87d1000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc1618b0 con 0x7f87cc06d800 2024-04-25T09:03:59.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.240+0000 7f87d1000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc161b90 con 0x7f87cc0ceb50 2024-04-25T09:03:59.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.241+0000 7f87cb400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f87cc0ceb50 0x7f87cc06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.241+0000 7f87cbe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f87cc15e7c0 0x7f87cc0b4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40892/0 (socket says 172.21.15.62:40892) 2024-04-25T09:03:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.241+0000 7f87cbe00640 1 -- 172.21.15.62:0/267460572 learned_addr learned my addr 172.21.15.62:0/267460572 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:03:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.241+0000 7f87cb400640 1 -- 172.21.15.62:0/267460572 >> v1:172.21.15.121:6789/0 conn(0x7f87cc06d800 legacy=0x7f87cc15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.241+0000 7f87cb400640 1 -- 172.21.15.62:0/267460572 >> v1:172.21.15.62:6789/0 conn(0x7f87cc15e7c0 legacy=0x7f87cc0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.242+0000 7f87cb400640 1 -- 172.21.15.62:0/267460572 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0cef30 con 0x7f87cc0ceb50 2024-04-25T09:03:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.241+0000 7f87caa00640 1 -- 172.21.15.62:0/267460572 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043348085 0 0) 0x7f87cc15ee30 con 0x7f87cc15e7c0 2024-04-25T09:03:59.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.243+0000 7f87cb400640 1 --2- 172.21.15.62:0/267460572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc0ceb50 0x7f87cc06d2c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0097a0 tx=0x7f87bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d426c8132d408f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.243+0000 7f87caa00640 1 -- 172.21.15.62:0/267460572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc037070 con 0x7f87cc0ceb50 2024-04-25T09:03:59.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.243+0000 7f87caa00640 1 -- 172.21.15.62:0/267460572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc032440 con 0x7f87cc0ceb50 2024-04-25T09:03:59.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.243+0000 7f87caa00640 1 -- 172.21.15.62:0/267460572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc032810 con 0x7f87cc0ceb50 2024-04-25T09:03:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.244+0000 7f87d1000640 1 -- 172.21.15.62:0/267460572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc0ceb50 msgr2=0x7f87cc06d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.244+0000 7f87d1000640 1 --2- 172.21.15.62:0/267460572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc0ceb50 0x7f87cc06d2c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0097a0 tx=0x7f87bc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.244+0000 7f87d1000640 1 -- 172.21.15.62:0/267460572 shutdown_connections 2024-04-25T09:03:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.244+0000 7f87d1000640 1 --2- 172.21.15.62:0/267460572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc0ceb50 0x7f87cc06d2c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.244+0000 7f87d1000640 1 -- 172.21.15.62:0/267460572 >> 172.21.15.62:0/267460572 conn(0x7f87cc066f20 msgr2=0x7f87cc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.244+0000 7f87d1000640 1 -- 172.21.15.62:0/267460572 shutdown_connections 2024-04-25T09:03:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.244+0000 7f87d1000640 1 -- 172.21.15.62:0/267460572 wait complete. 2024-04-25T09:03:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.245+0000 7f87d1000640 1 Processor -- start 2024-04-25T09:03:59.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.246+0000 7f87d1000640 1 -- start start 2024-04-25T09:03:59.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.246+0000 7f87d1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc06d800 0x7f87cc1453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87d1000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87cc0ceb50 0x7f87cc134ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87d1000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87cc15e7c0 0x7f87cc157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87d1000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0b4c60 con 0x7f87cc06d800 2024-04-25T09:03:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87d1000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc1625e0 con 0x7f87cc0ceb50 2024-04-25T09:03:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87d1000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0b4de0 con 0x7f87cc15e7c0 2024-04-25T09:03:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87cb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc06d800 0x7f87cc1453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87cb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc06d800 0x7f87cc1453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46454/0 (socket says 172.21.15.62:46454) 2024-04-25T09:03:59.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87cb400640 1 -- 172.21.15.62:0/4285484246 learned_addr learned my addr 172.21.15.62:0/4285484246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87cb400640 1 -- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87cc15e7c0 msgr2=0x7f87cc157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87cb400640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87cc15e7c0 0x7f87cc157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87cb400640 1 -- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87cc0ceb50 msgr2=0x7f87cc134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87cb400640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87cc0ceb50 0x7f87cc134ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.247+0000 7f87cb400640 1 -- 172.21.15.62:0/4285484246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc158260 con 0x7f87cc06d800 2024-04-25T09:03:59.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.248+0000 7f87cb400640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc06d800 0x7f87cc1453b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0098c0 tx=0x7f87bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.248+0000 7f87c3400640 1 -- 172.21.15.62:0/4285484246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc037040 con 0x7f87cc06d800 2024-04-25T09:03:59.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.248+0000 7f87c3400640 1 -- 172.21.15.62:0/4285484246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc042070 con 0x7f87cc06d800 2024-04-25T09:03:59.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.248+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc1371a0 con 0x7f87cc06d800 2024-04-25T09:03:59.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.248+0000 7f87c3400640 1 -- 172.21.15.62:0/4285484246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc030a70 con 0x7f87cc06d800 2024-04-25T09:03:59.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.248+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc137760 con 0x7f87cc06d800 2024-04-25T09:03:59.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.249+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc06abe0 con 0x7f87cc06d800 2024-04-25T09:03:59.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.250+0000 7f87c3400640 1 -- 172.21.15.62:0/4285484246 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87bc030c10 con 0x7f87cc06d800 2024-04-25T09:03:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.251+0000 7f87c3400640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.251+0000 7f87c3400640 1 -- 172.21.15.62:0/4285484246 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f87bc0aad30 con 0x7f87cc06d800 2024-04-25T09:03:59.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.251+0000 7f87c3e00640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.252+0000 7f87c3e00640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f87a806b4d0 0x7f87a806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000f60 tx=0x7f87b40079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.255+0000 7f87c3400640 1 -- 172.21.15.62:0/4285484246 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87cc158260 con 0x7f87cc06d800 2024-04-25T09:03:59.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.455+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87cc15fee0 con 0x7f87cc06d800 2024-04-25T09:03:59.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.456+0000 7f87c3400640 1 -- 172.21.15.62:0/4285484246 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87bc045360 con 0x7f87cc06d800 2024-04-25T09:03:59.445 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:03:59.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f87a806b4d0 msgr2=0x7f87a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f87a806b4d0 0x7f87a806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000f60 tx=0x7f87b40079b0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc06d800 msgr2=0x7f87cc1453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc06d800 0x7f87cc1453b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0098c0 tx=0x7f87bc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 shutdown_connections 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87cc15e7c0 0x7f87cc157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87cc0ceb50 0x7f87cc134ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 --2- 172.21.15.62:0/4285484246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87cc06d800 0x7f87cc1453b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.459+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 >> 172.21.15.62:0/4285484246 conn(0x7f87cc066f20 msgr2=0x7f87cc0b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.460+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 shutdown_connections 2024-04-25T09:03:59.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.460+0000 7f87d1000640 1 -- 172.21.15.62:0/4285484246 wait complete. 2024-04-25T09:03:59.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:59.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.696+0000 7fd4e5800640 1 Processor -- start 2024-04-25T09:03:59.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.697+0000 7fd4e5800640 1 -- start start 2024-04-25T09:03:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.698+0000 7fd4e5800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 0x7fd4e00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.698+0000 7fd4e5800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd4e00b31a0 0x7fd4e00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.698+0000 7fd4e5800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e00ced40 con 0x7fd4e006d2c0 2024-04-25T09:03:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.698+0000 7fd4e5800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e00cd260 con 0x7fd4e00b31a0 2024-04-25T09:03:59.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.698+0000 7fd4e5800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e00cf000 con 0x7fd4e00ccde0 2024-04-25T09:03:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.699+0000 7fd4e4e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd4e00b31a0 0x7fd4e00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.699+0000 7fd4df400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 0x7fd4e00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.699+0000 7fd4e4e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd4e00b31a0 0x7fd4e00b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:46270/0 (socket says 172.21.15.62:46270) 2024-04-25T09:03:59.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.699+0000 7fd4e4e00640 1 -- 172.21.15.62:0/107883302 learned_addr learned my addr 172.21.15.62:0/107883302 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.700+0000 7fd4df400640 1 -- 172.21.15.62:0/107883302 >> v2:172.21.15.121:3300/0 conn(0x7fd4e00b31a0 msgr2=0x7fd4e00b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.700+0000 7fd4df400640 1 --2- 172.21.15.62:0/107883302 >> v2:172.21.15.121:3300/0 conn(0x7fd4e00b31a0 0x7fd4e00b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.700+0000 7fd4df400640 1 -- 172.21.15.62:0/107883302 >> v1:172.21.15.121:6789/0 conn(0x7fd4e006d2c0 legacy=0x7fd4e006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.700+0000 7fd4df400640 1 -- 172.21.15.62:0/107883302 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e00cf2e0 con 0x7fd4e00ccde0 2024-04-25T09:03:59.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.700+0000 7fd4dea00640 1 -- 172.21.15.62:0/107883302 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147116798 0 0) 0x7fd4e00ced40 con 0x7fd4e006d2c0 2024-04-25T09:03:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.702+0000 7fd4df400640 1 --2- 172.21.15.62:0/107883302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 0x7fd4e00ce800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80097a0 tx=0x7fd4c802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44b0b5e0485294c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.703+0000 7fd4dea00640 1 -- 172.21.15.62:0/107883302 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8037070 con 0x7fd4e00ccde0 2024-04-25T09:03:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.703+0000 7fd4dea00640 1 -- 172.21.15.62:0/107883302 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c8032440 con 0x7fd4e00ccde0 2024-04-25T09:03:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.703+0000 7fd4dea00640 1 -- 172.21.15.62:0/107883302 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8032810 con 0x7fd4e00ccde0 2024-04-25T09:03:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.703+0000 7fd4e5800640 1 -- 172.21.15.62:0/107883302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 msgr2=0x7fd4e00ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.703+0000 7fd4e5800640 1 --2- 172.21.15.62:0/107883302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 0x7fd4e00ce800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80097a0 tx=0x7fd4c802a180 comp rx=0 tx=0).stop 2024-04-25T09:03:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.704+0000 7fd4e5800640 1 -- 172.21.15.62:0/107883302 shutdown_connections 2024-04-25T09:03:59.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.704+0000 7fd4e5800640 1 --2- 172.21.15.62:0/107883302 >> v2:172.21.15.121:3300/0 conn(0x7fd4e00b31a0 0x7fd4e00b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.704+0000 7fd4e5800640 1 --2- 172.21.15.62:0/107883302 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 0x7fd4e00ce800 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.704+0000 7fd4e5800640 1 -- 172.21.15.62:0/107883302 >> 172.21.15.62:0/107883302 conn(0x7fd4e0066f20 msgr2=0x7fd4e00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.704+0000 7fd4e5800640 1 -- 172.21.15.62:0/107883302 shutdown_connections 2024-04-25T09:03:59.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.704+0000 7fd4e5800640 1 -- 172.21.15.62:0/107883302 wait complete. 2024-04-25T09:03:59.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.705+0000 7fd4e5800640 1 Processor -- start 2024-04-25T09:03:59.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.706+0000 7fd4e5800640 1 -- start start 2024-04-25T09:03:59.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.706+0000 7fd4e5800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4e006d2c0 0x7fd4e0138c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.706+0000 7fd4e5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e00b31a0 0x7fd4e00c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.706+0000 7fd4e5800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 0x7fd4e006c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4e5800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e00b2c80 con 0x7fd4e00b31a0 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4e5800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e00b29a0 con 0x7fd4e006d2c0 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4e5800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01448d0 con 0x7fd4e00ccde0 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4df400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e00b31a0 0x7fd4e00c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4df400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e00b31a0 0x7fd4e00c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46460/0 (socket says 172.21.15.62:46460) 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4df400640 1 -- 172.21.15.62:0/787538464 learned_addr learned my addr 172.21.15.62:0/787538464 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4df400640 1 -- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 msgr2=0x7fd4e006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4df400640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 0x7fd4e006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4dfe00640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4e006d2c0 0x7fd4e0138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4df400640 1 -- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4e006d2c0 msgr2=0x7fd4e0138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4df400640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4e006d2c0 0x7fd4e0138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.707+0000 7fd4df400640 1 -- 172.21.15.62:0/787538464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e00c4880 con 0x7fd4e00b31a0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.708+0000 7fd4df400640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e00b31a0 0x7fd4e00c5920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80098c0 tx=0x7fd4c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.708+0000 7fd4d7e00640 1 -- 172.21.15.62:0/787538464 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8037040 con 0x7fd4e00b31a0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.708+0000 7fd4d7e00640 1 -- 172.21.15.62:0/787538464 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c8030500 con 0x7fd4e00b31a0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.708+0000 7fd4d7e00640 1 -- 172.21.15.62:0/787538464 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80308d0 con 0x7fd4e00b31a0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.708+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e00c4b60 con 0x7fd4e00b31a0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.708+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e00c5120 con 0x7fd4e00b31a0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.709+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e006abe0 con 0x7fd4e00b31a0 2024-04-25T09:03:59.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.710+0000 7fd4d7e00640 1 -- 172.21.15.62:0/787538464 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4c8044020 con 0x7fd4e00b31a0 2024-04-25T09:03:59.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.711+0000 7fd4d7e00640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.711+0000 7fd4d7e00640 1 -- 172.21.15.62:0/787538464 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd4c80aa4b0 con 0x7fd4e00b31a0 2024-04-25T09:03:59.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.712+0000 7fd4dfe00640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.713+0000 7fd4dfe00640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00012e0 tx=0x7fd4d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.716+0000 7fd4d7e00640 1 -- 172.21.15.62:0/787538464 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4c8079300 con 0x7fd4e00b31a0 2024-04-25T09:03:59.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.918+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4e004ef20 con 0x7fd4e00b31a0 2024-04-25T09:03:59.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.919+0000 7fd4d7e00640 1 -- 172.21.15.62:0/787538464 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd4c8078c00 con 0x7fd4e00b31a0 2024-04-25T09:03:59.908 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:03:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd4bc06b4d0 msgr2=0x7fd4bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00012e0 tx=0x7fd4d0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e00b31a0 msgr2=0x7fd4e00c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e00b31a0 0x7fd4e00c5920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80098c0 tx=0x7fd4c80046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 shutdown_connections 2024-04-25T09:03:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd4bc06b4d0 0x7fd4bc06d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4e00b31a0 0x7fd4e00c5920 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd4e00ccde0 0x7fd4e006c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 --2- 172.21.15.62:0/787538464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4e006d2c0 0x7fd4e0138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 >> 172.21.15.62:0/787538464 conn(0x7fd4e0066f20 msgr2=0x7fd4e00b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 shutdown_connections 2024-04-25T09:03:59.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:03:59.922+0000 7fd4e5800640 1 -- 172.21.15.62:0/787538464 wait complete. 2024-04-25T09:03:59.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:00.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.160+0000 7f86f9c00640 1 Processor -- start 2024-04-25T09:04:00.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.161+0000 7f86f9c00640 1 -- start start 2024-04-25T09:04:00.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.162+0000 7f86f9c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86f415dd50 0x7f86f40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.162+0000 7f86f9c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86f40d3960 0x7f86f40d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.162+0000 7f86f9c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f40ce780 con 0x7f86f40d2fb0 2024-04-25T09:04:00.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.162+0000 7f86f9c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f40d42f0 con 0x7f86f415dd50 2024-04-25T09:04:00.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.162+0000 7f86f9c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f40d3440 con 0x7f86f40d3960 2024-04-25T09:04:00.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.163+0000 7f86f3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86f415dd50 0x7f86f40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.163+0000 7f86f3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f86f415dd50 0x7f86f40d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46474/0 (socket says 172.21.15.62:46474) 2024-04-25T09:04:00.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.163+0000 7f86f3e00640 1 -- 172.21.15.62:0/2115208092 learned_addr learned my addr 172.21.15.62:0/2115208092 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:00.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.163+0000 7f86f9200640 1 --2- 172.21.15.62:0/2115208092 >> v2:172.21.15.121:3300/0 conn(0x7f86f40d3960 0x7f86f40d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.164+0000 7f86f3e00640 1 -- 172.21.15.62:0/2115208092 >> v2:172.21.15.121:3300/0 conn(0x7f86f40d3960 msgr2=0x7f86f40d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.164+0000 7f86f3e00640 1 --2- 172.21.15.62:0/2115208092 >> v2:172.21.15.121:3300/0 conn(0x7f86f40d3960 0x7f86f40d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.164+0000 7f86f3e00640 1 -- 172.21.15.62:0/2115208092 >> v1:172.21.15.121:6789/0 conn(0x7f86f40d2fb0 legacy=0x7f86f40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.164+0000 7f86f3e00640 1 -- 172.21.15.62:0/2115208092 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40ce1c0 con 0x7f86f415dd50 2024-04-25T09:04:00.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.165+0000 7f86f3e00640 1 --2- 172.21.15.62:0/2115208092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f40d2300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86f40caa50 tx=0x7f86e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304083a67604f66c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.165+0000 7f86f3400640 1 -- 172.21.15.62:0/2115208092 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4037070 con 0x7f86f415dd50 2024-04-25T09:04:00.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f3400640 1 -- 172.21.15.62:0/2115208092 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4032440 con 0x7f86f415dd50 2024-04-25T09:04:00.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f3400640 1 -- 172.21.15.62:0/2115208092 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4032810 con 0x7f86f415dd50 2024-04-25T09:04:00.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f9c00640 1 -- 172.21.15.62:0/2115208092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 msgr2=0x7f86f40d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2115208092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f40d2300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86f40caa50 tx=0x7f86e402a230 comp rx=0 tx=0).stop 2024-04-25T09:04:00.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f9c00640 1 -- 172.21.15.62:0/2115208092 shutdown_connections 2024-04-25T09:04:00.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2115208092 >> v2:172.21.15.121:3300/0 conn(0x7f86f40d3960 0x7f86f40d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2115208092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f40d2300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f9c00640 1 -- 172.21.15.62:0/2115208092 >> 172.21.15.62:0/2115208092 conn(0x7f86f406b000 msgr2=0x7f86f406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f9c00640 1 -- 172.21.15.62:0/2115208092 shutdown_connections 2024-04-25T09:04:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.166+0000 7f86f9c00640 1 -- 172.21.15.62:0/2115208092 wait complete. 2024-04-25T09:04:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.168+0000 7f86f9c00640 1 Processor -- start 2024-04-25T09:04:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.168+0000 7f86f9c00640 1 -- start start 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.169+0000 7f86f9c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86f40d3960 0x7f86f4156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.169+0000 7f86f9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.169+0000 7f86f9c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f40c9060 0x7f86f40c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.169+0000 7f86f9c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40b6b00 con 0x7f86f415dd50 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.169+0000 7f86f9c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40d2840 con 0x7f86f40c9060 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.169+0000 7f86f9c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40b6c80 con 0x7f86f40d3960 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86ebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86ebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f415f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46476/0 (socket says 172.21.15.62:46476) 2024-04-25T09:04:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86ebe00640 1 -- 172.21.15.62:0/2907673193 learned_addr learned my addr 172.21.15.62:0/2907673193 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86f9200640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f40c9060 0x7f86f40c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86ebe00640 1 -- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86f40d3960 msgr2=0x7f86f4156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86f3e00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86f40d3960 0x7f86f4156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86ebe00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86f40d3960 0x7f86f4156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86ebe00640 1 -- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f40c9060 msgr2=0x7f86f40c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86ebe00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f40c9060 0x7f86f40c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86ebe00640 1 -- 172.21.15.62:0/2907673193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f415fe70 con 0x7f86f415dd50 2024-04-25T09:04:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.170+0000 7f86f9200640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f40c9060 0x7f86f40c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.171+0000 7f86ebe00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f415f7e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00c490 tx=0x7f86dc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.171+0000 7f86f0c00640 1 -- 172.21.15.62:0/2907673193 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc016020 con 0x7f86f415dd50 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.171+0000 7f86f0c00640 1 -- 172.21.15.62:0/2907673193 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc012830 con 0x7f86f415dd50 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.171+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f4160150 con 0x7f86f415dd50 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.171+0000 7f86f0c00640 1 -- 172.21.15.62:0/2907673193 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc012c20 con 0x7f86f415dd50 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.171+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f4160710 con 0x7f86f415dd50 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.172+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f404ef20 con 0x7f86f415dd50 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.173+0000 7f86f0c00640 1 -- 172.21.15.62:0/2907673193 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86dc019020 con 0x7f86f415dd50 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.174+0000 7f86f0c00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f86d006b500 0x7f86d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.175+0000 7f86f0c00640 1 -- 172.21.15.62:0/2907673193 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f86dc089f50 con 0x7f86f415dd50 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.175+0000 7f86f3e00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f86d006b500 0x7f86d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.176+0000 7f86f3e00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f86d006b500 0x7f86d006d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86e4034000 tx=0x7f86e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.179+0000 7f86f0c00640 1 -- 172.21.15.62:0/2907673193 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86dc020030 con 0x7f86f415dd50 2024-04-25T09:04:00.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.386+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86f40b9950 con 0x7f86f415dd50 2024-04-25T09:04:00.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.387+0000 7f86f0c00640 1 -- 172.21.15.62:0/2907673193 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86dc0182c0 con 0x7f86f415dd50 2024-04-25T09:04:00.376 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:04:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.389+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f86d006b500 msgr2=0x7f86d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.389+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f86d006b500 0x7f86d006d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f86e4034000 tx=0x7f86e402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.389+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 msgr2=0x7f86f415f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.389+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f415f7e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00c490 tx=0x7f86dc00c950 comp rx=0 tx=0).stop 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.390+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 shutdown_connections 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.390+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f86d006b500 0x7f86d006d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.390+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f86f415dd50 0x7f86f415f7e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.390+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f40c9060 0x7f86f40c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.390+0000 7f86f9c00640 1 --2- 172.21.15.62:0/2907673193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f86f40d3960 0x7f86f4156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.390+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 >> 172.21.15.62:0/2907673193 conn(0x7f86f406b000 msgr2=0x7f86f40cb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.390+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 shutdown_connections 2024-04-25T09:04:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.390+0000 7f86f9c00640 1 -- 172.21.15.62:0/2907673193 wait complete. 2024-04-25T09:04:00.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.625+0000 7f343bc00640 1 Processor -- start 2024-04-25T09:04:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.625+0000 7f343bc00640 1 -- start start 2024-04-25T09:04:00.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.627+0000 7f343bc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f34340ccce0 0x7f3434070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.627+0000 7f343bc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3434071510 0x7f34340ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.627+0000 7f343bc00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34341666d0 con 0x7f3434072020 2024-04-25T09:04:00.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.627+0000 7f343bc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34341669b0 con 0x7f34340ccce0 2024-04-25T09:04:00.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.627+0000 7f343bc00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3434166c90 con 0x7f3434071510 2024-04-25T09:04:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.627+0000 7f3439200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3434072020 0x7f3434165e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40904/0 (socket says 172.21.15.62:40904) 2024-04-25T09:04:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.627+0000 7f3433e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f34340ccce0 0x7f3434070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.627+0000 7f3439200640 1 -- 172.21.15.62:0/2242655187 learned_addr learned my addr 172.21.15.62:0/2242655187 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:00.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.628+0000 7f3439c00640 1 --2- 172.21.15.62:0/2242655187 >> v2:172.21.15.121:3300/0 conn(0x7f3434071510 0x7f34340ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.628+0000 7f3433e00640 1 -- 172.21.15.62:0/2242655187 >> v2:172.21.15.121:3300/0 conn(0x7f3434071510 msgr2=0x7f34340ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.628+0000 7f3433e00640 1 --2- 172.21.15.62:0/2242655187 >> v2:172.21.15.121:3300/0 conn(0x7f3434071510 0x7f34340ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.628+0000 7f3433e00640 1 -- 172.21.15.62:0/2242655187 >> v1:172.21.15.62:6789/0 conn(0x7f3434072020 legacy=0x7f3434165e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.628+0000 7f3433e00640 1 -- 172.21.15.62:0/2242655187 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343406d000 con 0x7f34340ccce0 2024-04-25T09:04:00.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.628+0000 7f3433400640 1 -- 172.21.15.62:0/2242655187 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236258538 0 0) 0x7f34341666d0 con 0x7f3434072020 2024-04-25T09:04:00.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.628+0000 7f3439c00640 1 --2- 172.21.15.62:0/2242655187 >> v2:172.21.15.121:3300/0 conn(0x7f3434071510 0x7f34340ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:00.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.629+0000 7f3433e00640 1 --2- 172.21.15.62:0/2242655187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f3434070fd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f34340d07a0 tx=0x7f342002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a6383690d3f15d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.630+0000 7f3433400640 1 -- 172.21.15.62:0/2242655187 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420037070 con 0x7f34340ccce0 2024-04-25T09:04:00.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.630+0000 7f3433400640 1 -- 172.21.15.62:0/2242655187 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3420032440 con 0x7f34340ccce0 2024-04-25T09:04:00.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.630+0000 7f3433400640 1 -- 172.21.15.62:0/2242655187 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420032810 con 0x7f34340ccce0 2024-04-25T09:04:00.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.631+0000 7f343bc00640 1 -- 172.21.15.62:0/2242655187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 msgr2=0x7f3434070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.631+0000 7f343bc00640 1 --2- 172.21.15.62:0/2242655187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f3434070fd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f34340d07a0 tx=0x7f342002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.631+0000 7f343bc00640 1 -- 172.21.15.62:0/2242655187 shutdown_connections 2024-04-25T09:04:00.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.631+0000 7f343bc00640 1 --2- 172.21.15.62:0/2242655187 >> v2:172.21.15.121:3300/0 conn(0x7f3434071510 0x7f34340ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.631+0000 7f343bc00640 1 --2- 172.21.15.62:0/2242655187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f3434070fd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.631+0000 7f343bc00640 1 -- 172.21.15.62:0/2242655187 >> 172.21.15.62:0/2242655187 conn(0x7f34340668f0 msgr2=0x7f3434066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.631+0000 7f343bc00640 1 -- 172.21.15.62:0/2242655187 shutdown_connections 2024-04-25T09:04:00.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.631+0000 7f343bc00640 1 -- 172.21.15.62:0/2242655187 wait complete. 2024-04-25T09:04:00.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.633+0000 7f343bc00640 1 Processor -- start 2024-04-25T09:04:00.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.634+0000 7f343bc00640 1 -- start start 2024-04-25T09:04:00.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f343bc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3434071510 0x7f34340b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f343bc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3434072020 0x7f343415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f343bc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f34341fc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f343bc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3434072580 con 0x7f34340ccce0 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f343bc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3434072400 con 0x7f3434072020 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f343bc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34340ce840 con 0x7f3434071510 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f3439c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f34341fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f3439c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f34341fc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46484/0 (socket says 172.21.15.62:46484) 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.635+0000 7f3439c00640 1 -- 172.21.15.62:0/1539082414 learned_addr learned my addr 172.21.15.62:0/1539082414 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3439200640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3434071510 0x7f34340b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3433e00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3434072020 0x7f343415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3439c00640 1 -- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3434071510 msgr2=0x7f34340b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3439c00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3434071510 0x7f34340b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3439c00640 1 -- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3434072020 msgr2=0x7f343415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3439c00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3434072020 0x7f343415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3439c00640 1 -- 172.21.15.62:0/1539082414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343415bcf0 con 0x7f34340ccce0 2024-04-25T09:04:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3439c00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f34341fc520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f342c00cd80 tx=0x7f342c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.636+0000 7f3430c00640 1 -- 172.21.15.62:0/1539082414 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c00f680 con 0x7f34340ccce0 2024-04-25T09:04:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.637+0000 7f3430c00640 1 -- 172.21.15.62:0/1539082414 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c00f840 con 0x7f34340ccce0 2024-04-25T09:04:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.637+0000 7f3430c00640 1 -- 172.21.15.62:0/1539082414 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c00fc30 con 0x7f34340ccce0 2024-04-25T09:04:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.637+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343415d900 con 0x7f34340ccce0 2024-04-25T09:04:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.637+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343415c270 con 0x7f34340ccce0 2024-04-25T09:04:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.638+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3434072c70 con 0x7f34340ccce0 2024-04-25T09:04:00.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.639+0000 7f3430c00640 1 -- 172.21.15.62:0/1539082414 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f342c00e3e0 con 0x7f34340ccce0 2024-04-25T09:04:00.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.640+0000 7f3430c00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f341406b4d0 0x7f341406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.640+0000 7f3430c00640 1 -- 172.21.15.62:0/1539082414 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f342c08af30 con 0x7f34340ccce0 2024-04-25T09:04:00.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.640+0000 7f3439200640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f341406b4d0 0x7f341406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.642+0000 7f3439200640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f341406b4d0 0x7f341406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3428000f60 tx=0x7f3428007b70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.644+0000 7f3430c00640 1 -- 172.21.15.62:0/1539082414 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f342c014030 con 0x7f34340ccce0 2024-04-25T09:04:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.860+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3434165bf0 con 0x7f34340ccce0 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.861+0000 7f3430c00640 1 -- 172.21.15.62:0/1539082414 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f342c059e20 con 0x7f34340ccce0 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f341406b4d0 msgr2=0x7f341406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f341406b4d0 0x7f341406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3428000f60 tx=0x7f3428007b70 comp rx=0 tx=0).stop 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 msgr2=0x7f34341fc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f34341fc520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f342c00cd80 tx=0x7f342c00a650 comp rx=0 tx=0).stop 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 shutdown_connections 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f341406b4d0 0x7f341406d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f34340ccce0 0x7f34341fc520 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3434072020 0x7f343415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 --2- 172.21.15.62:0/1539082414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3434071510 0x7f34340b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 >> 172.21.15.62:0/1539082414 conn(0x7f34340668f0 msgr2=0x7f34340cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 shutdown_connections 2024-04-25T09:04:00.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:00.864+0000 7f343bc00640 1 -- 172.21.15.62:0/1539082414 wait complete. 2024-04-25T09:04:00.872 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:04:00.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:00.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T09:04:00.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:04:00.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:00.959 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:04:01.019 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:04:01.019 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:01.073 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:04:01.073 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:01.122 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:04:01.122 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:01.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.451+0000 7ff61b200640 1 Processor -- start 2024-04-25T09:04:01.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.452+0000 7ff61b200640 1 -- start start 2024-04-25T09:04:01.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.453+0000 7ff61b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff6140b56f0 0x7ff614160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.453+0000 7ff61b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6140b6a90 0x7ff614161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.453+0000 7ff61b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6140d3110 con 0x7ff6140b27a0 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.453+0000 7ff61b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff614161780 con 0x7ff6140b56f0 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.453+0000 7ff61b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6140cedc0 con 0x7ff6140b6a90 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.453+0000 7ff61a800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff6140b27a0 0x7ff614161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40908/0 (socket says 172.21.15.62:40908) 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.453+0000 7ff61a800640 1 -- 172.21.15.62:0/416985466 learned_addr learned my addr 172.21.15.62:0/416985466 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316923815 0 0) 0x7ff6140d3110 con 0x7ff6140b27a0 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5fc003520 con 0x7ff6140b27a0 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff619400640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6140b6a90 0x7ff614161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff619e00640 1 --2- 172.21.15.62:0/416985466 >> v2:172.21.15.121:3300/0 conn(0x7ff6140b56f0 0x7ff614160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4150167605 0 0) 0x7ff5fc003520 con 0x7ff6140b27a0 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6140b6a90 msgr2=0x7ff614161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6140b6a90 0x7ff614161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 >> v2:172.21.15.121:3300/0 conn(0x7ff6140b56f0 msgr2=0x7ff614160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 --2- 172.21.15.62:0/416985466 >> v2:172.21.15.121:3300/0 conn(0x7ff6140b56f0 0x7ff614160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6140d33f0 con 0x7ff6140b27a0 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ff6100019f0 con 0x7ff6140b27a0 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff619e00640 1 --2- 172.21.15.62:0/416985466 >> v2:172.21.15.121:3300/0 conn(0x7ff6140b56f0 0x7ff614160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.454+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 >> v1:172.21.15.62:6789/0 conn(0x7ff6140b27a0 legacy=0x7ff614161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff618a00640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5fc0039b0 0x7ff5fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff618a00640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6140b6a90 0x7ff5fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff618a00640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc004d40 0x7ff5fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6140cedc0 con 0x7ff5fc0039b0 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc00baa0 con 0x7ff5fc004d40 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc00bca0 con 0x7ff6140b6a90 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff619e00640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5fc0039b0 0x7ff5fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff619400640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6140b6a90 0x7ff5fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff61a800640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc004d40 0x7ff5fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff619e00640 1 -- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6140b6a90 msgr2=0x7ff5fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff619e00640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff6140b6a90 0x7ff5fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff619e00640 1 -- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc004d40 msgr2=0x7ff5fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff619e00640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc004d40 0x7ff5fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.455+0000 7ff619e00640 1 -- 172.21.15.62:0/416985466 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc003520 con 0x7ff5fc0039b0 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.456+0000 7ff619e00640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5fc0039b0 0x7ff5fc003dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098a0 tx=0x7ff5fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815b0ad87ae03c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.457+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800f040 con 0x7ff5fc0039b0 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.457+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60802ad20 con 0x7ff5fc0039b0 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.457+0000 7ff618a00640 1 -- 172.21.15.62:0/416985466 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608004250 con 0x7ff5fc0039b0 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.457+0000 7ff61b200640 1 -- 172.21.15.62:0/416985466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5fc0039b0 msgr2=0x7ff5fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.457+0000 7ff61b200640 1 --2- 172.21.15.62:0/416985466 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5fc0039b0 0x7ff5fc003dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098a0 tx=0x7ff5fc005830 comp rx=0 tx=0).stop 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.457+0000 7ff61a800640 1 -- 172.21.15.62:0/416985466 reap_dead start 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.458+0000 7ff61b200640 1 -- 172.21.15.62:0/416985466 shutdown_connections 2024-04-25T09:04:01.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.458+0000 7ff61b200640 1 -- 172.21.15.62:0/416985466 >> 172.21.15.62:0/416985466 conn(0x7ff614066d10 msgr2=0x7ff614067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.458+0000 7ff61b200640 1 -- 172.21.15.62:0/416985466 shutdown_connections 2024-04-25T09:04:01.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.458+0000 7ff61b200640 1 -- 172.21.15.62:0/416985466 wait complete. 2024-04-25T09:04:01.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.459+0000 7ff61b200640 1 Processor -- start 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.460+0000 7ff61b200640 1 -- start start 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.460+0000 7ff61b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc0039b0 0x7ff61415d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.460+0000 7ff61b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff61415c0b0 0x7ff6140cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.460+0000 7ff61b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140cda00 0x7ff6140c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.460+0000 7ff61b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6140b2d60 con 0x7ff6140cda00 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.460+0000 7ff61b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6140cea50 con 0x7ff5fc0039b0 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.460+0000 7ff61b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6140b2ee0 con 0x7ff61415c0b0 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff61a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140cda00 0x7ff6140c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff61a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140cda00 0x7ff6140c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46500/0 (socket says 172.21.15.62:46500) 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff61a800640 1 -- 172.21.15.62:0/1737532380 learned_addr learned my addr 172.21.15.62:0/1737532380 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff61a800640 1 -- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff61415c0b0 msgr2=0x7ff6140cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff619400640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff61415c0b0 0x7ff6140cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff619e00640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc0039b0 0x7ff61415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff61a800640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff61415c0b0 0x7ff6140cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff61a800640 1 -- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc0039b0 msgr2=0x7ff61415d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff61a800640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc0039b0 0x7ff61415d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.461+0000 7ff61a800640 1 -- 172.21.15.62:0/1737532380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6140cac50 con 0x7ff6140cda00 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.462+0000 7ff61a800640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140cda00 0x7ff6140c7dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff6140d4ad0 tx=0x7ff610007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.462+0000 7ff60e000640 1 -- 172.21.15.62:0/1737532380 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000ea80 con 0x7ff6140cda00 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.462+0000 7ff60e000640 1 -- 172.21.15.62:0/1737532380 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610018070 con 0x7ff6140cda00 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.462+0000 7ff60e000640 1 -- 172.21.15.62:0/1737532380 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000ee60 con 0x7ff6140cda00 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.462+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6140caf10 con 0x7ff6140cda00 2024-04-25T09:04:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.462+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61415d6c0 con 0x7ff6140cda00 2024-04-25T09:04:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.463+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61404ef20 con 0x7ff6140cda00 2024-04-25T09:04:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.464+0000 7ff60e000640 1 -- 172.21.15.62:0/1737532380 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff61001d460 con 0x7ff6140cda00 2024-04-25T09:04:01.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.465+0000 7ff60e000640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5f806b4d0 0x7ff5f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.465+0000 7ff60e000640 1 -- 172.21.15.62:0/1737532380 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff61008a4a0 con 0x7ff6140cda00 2024-04-25T09:04:01.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.466+0000 7ff619e00640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5f806b4d0 0x7ff5f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.467+0000 7ff619e00640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5f806b4d0 0x7ff5f806d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff608033000 tx=0x7ff60802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.469+0000 7ff60e000640 1 -- 172.21.15.62:0/1737532380 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6100592f0 con 0x7ff6140cda00 2024-04-25T09:04:01.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.671+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6140ce580 con 0x7ff6140cda00 2024-04-25T09:04:01.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.673+0000 7ff60e000640 1 -- 172.21.15.62:0/1737532380 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16646 (secure 0 0 0) 0x7ff610058bf0 con 0x7ff6140cda00 2024-04-25T09:04:01.663 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:01.663 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:59.724537+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:01.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5f806b4d0 msgr2=0x7ff5f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5f806b4d0 0x7ff5f806d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff608033000 tx=0x7ff60802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:01.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140cda00 msgr2=0x7ff6140c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140cda00 0x7ff6140c7dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff6140d4ad0 tx=0x7ff610007bc0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 shutdown_connections 2024-04-25T09:04:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5f806b4d0 0x7ff5f806d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6140cda00 0x7ff6140c7dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff61415c0b0 0x7ff6140cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 --2- 172.21.15.62:0/1737532380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5fc0039b0 0x7ff61415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 >> 172.21.15.62:0/1737532380 conn(0x7ff614066d10 msgr2=0x7ff6140b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 shutdown_connections 2024-04-25T09:04:01.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.676+0000 7ff61b200640 1 -- 172.21.15.62:0/1737532380 wait complete. 2024-04-25T09:04:01.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:01.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:01.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.910+0000 7f6236800640 1 Processor -- start 2024-04-25T09:04:01.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.911+0000 7f6236800640 1 -- start start 2024-04-25T09:04:01.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.912+0000 7f6236800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f623006d2c0 0x7f62300ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.912+0000 7f6236800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62300cf000 con 0x7f62300b31a0 2024-04-25T09:04:01.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.912+0000 7f6236800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62300cf2e0 con 0x7f623006d6a0 2024-04-25T09:04:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.912+0000 7f6236800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300b26c0 con 0x7f623006d2c0 2024-04-25T09:04:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.913+0000 7f6235e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f62300b31a0 0x7f62300b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40910/0 (socket says 172.21.15.62:40910) 2024-04-25T09:04:01.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.913+0000 7f6235e00640 1 -- 172.21.15.62:0/3160584114 learned_addr learned my addr 172.21.15.62:0/3160584114 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.913+0000 7f6235400640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f623006d2c0 0x7f62300ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.913+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706369234 0 0) 0x7f62300cf000 con 0x7f62300b31a0 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.913+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6218003520 con 0x7f62300b31a0 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.913+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1397992179 0 0) 0x7f6218003520 con 0x7f62300b31a0 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f623006d2c0 msgr2=0x7f62300ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f623006d2c0 0x7f62300ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 >> v1:172.21.15.121:6789/0 conn(0x7f623006d6a0 legacy=0x7f62300ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300b29a0 con 0x7f62300b31a0 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f62280019f0 con 0x7f62300b31a0 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 >> v1:172.21.15.62:6789/0 conn(0x7f62300b31a0 legacy=0x7f62300b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62180039e0 0x7f6218003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f6218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f623006d2c0 0x7f62180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300b26c0 con 0x7f6218004800 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621800baa0 con 0x7f62180039e0 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621800bca0 con 0x7f623006d2c0 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f6234a00640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f6218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.914+0000 7f6235400640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62180039e0 0x7f6218003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.915+0000 7f6234a00640 1 -- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f623006d2c0 msgr2=0x7f62180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.915+0000 7f6234a00640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f623006d2c0 0x7f62180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.915+0000 7f6234a00640 1 -- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62180039e0 msgr2=0x7f6218003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.915+0000 7f6234a00640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62180039e0 0x7f6218003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.915+0000 7f6234a00640 1 -- 172.21.15.62:0/3160584114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6218003520 con 0x7f6218004800 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.916+0000 7f6234a00640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f6218004c80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f621c001720 tx=0x7f6218005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab930e09d9c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.916+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c001370 con 0x7f6218004800 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.916+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c0052d0 con 0x7f6218004800 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.916+0000 7f622fe00640 1 -- 172.21.15.62:0/3160584114 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c02ab90 con 0x7f6218004800 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.916+0000 7f6236800640 1 -- 172.21.15.62:0/3160584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 msgr2=0x7f6218004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.916+0000 7f6236800640 1 --2- 172.21.15.62:0/3160584114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f6218004c80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f621c001720 tx=0x7f6218005830 comp rx=0 tx=0).stop 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.917+0000 7f6235e00640 1 -- 172.21.15.62:0/3160584114 reap_dead start 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.917+0000 7f6236800640 1 -- 172.21.15.62:0/3160584114 shutdown_connections 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.917+0000 7f6236800640 1 -- 172.21.15.62:0/3160584114 >> 172.21.15.62:0/3160584114 conn(0x7f6230066f20 msgr2=0x7f62300b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.917+0000 7f6236800640 1 -- 172.21.15.62:0/3160584114 shutdown_connections 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.917+0000 7f6236800640 1 -- 172.21.15.62:0/3160584114 wait complete. 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.918+0000 7f6236800640 1 Processor -- start 2024-04-25T09:04:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.919+0000 7f6236800640 1 -- start start 2024-04-25T09:04:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6236800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f62300acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6236800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6230140dc0 0x7f623006ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6236800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6230141240 0x7f6230070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6236800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62301448d0 con 0x7f6218004800 2024-04-25T09:04:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6236800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623006db20 con 0x7f6230140dc0 2024-04-25T09:04:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6236800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6230144a50 con 0x7f6230141240 2024-04-25T09:04:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f62300acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f62300acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46510/0 (socket says 172.21.15.62:46510) 2024-04-25T09:04:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235400640 1 -- 172.21.15.62:0/584680305 learned_addr learned my addr 172.21.15.62:0/584680305 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235e00640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6230141240 0x7f6230070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235400640 1 -- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6230141240 msgr2=0x7f6230070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235400640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6230141240 0x7f6230070500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235400640 1 -- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6230140dc0 msgr2=0x7f623006ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235400640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6230140dc0 0x7f623006ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.920+0000 7f6235400640 1 -- 172.21.15.62:0/584680305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300abb00 con 0x7f6218004800 2024-04-25T09:04:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.921+0000 7f6235400640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f62300acba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6224011630 tx=0x7f6224011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.921+0000 7f622d600640 1 -- 172.21.15.62:0/584680305 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400f920 con 0x7f6218004800 2024-04-25T09:04:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.921+0000 7f622d600640 1 -- 172.21.15.62:0/584680305 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622400fac0 con 0x7f6218004800 2024-04-25T09:04:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.921+0000 7f622d600640 1 -- 172.21.15.62:0/584680305 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400fe70 con 0x7f6218004800 2024-04-25T09:04:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.921+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62300abde0 con 0x7f6218004800 2024-04-25T09:04:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.921+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62300ac3a0 con 0x7f6218004800 2024-04-25T09:04:01.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.922+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623006abe0 con 0x7f6218004800 2024-04-25T09:04:01.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.923+0000 7f622d600640 1 -- 172.21.15.62:0/584680305 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6224015460 con 0x7f6218004800 2024-04-25T09:04:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.924+0000 7f622d600640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621006b500 0x7f621006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.924+0000 7f622d600640 1 -- 172.21.15.62:0/584680305 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f622408ac70 con 0x7f6218004800 2024-04-25T09:04:01.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.925+0000 7f6234a00640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621006b500 0x7f621006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.926+0000 7f6234a00640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621006b500 0x7f621006d9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f621c032000 tx=0x7f621c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:01.928+0000 7f622d600640 1 -- 172.21.15.62:0/584680305 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6224019030 con 0x7f6218004800 2024-04-25T09:04:02.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.128+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f623004ef20 con 0x7f6218004800 2024-04-25T09:04:02.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.129+0000 7f622d600640 1 -- 172.21.15.62:0/584680305 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6224059a30 con 0x7f6218004800 2024-04-25T09:04:02.118 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621006b500 msgr2=0x7f621006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621006b500 0x7f621006d9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f621c032000 tx=0x7f621c02c000 comp rx=0 tx=0).stop 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 msgr2=0x7f62300acba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f62300acba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6224011630 tx=0x7f6224011af0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 shutdown_connections 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621006b500 0x7f621006d9c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6230141240 0x7f6230070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6230140dc0 0x7f623006ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 --2- 172.21.15.62:0/584680305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6218004800 0x7f62300acba0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.131+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 >> 172.21.15.62:0/584680305 conn(0x7f6230066f20 msgr2=0x7f62300ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.132+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 shutdown_connections 2024-04-25T09:04:02.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.132+0000 7f6236800640 1 -- 172.21.15.62:0/584680305 wait complete. 2024-04-25T09:04:02.140 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:02.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.367+0000 7f3871c00640 1 Processor -- start 2024-04-25T09:04:02.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.367+0000 7f3871c00640 1 -- start start 2024-04-25T09:04:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.369+0000 7f3871c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f386c15dc10 0x7f386c0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.369+0000 7f3871c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f386c0b97b0 0x7f386c0b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.369+0000 7f3871c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0b7590 con 0x7f386c0d20a0 2024-04-25T09:04:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.369+0000 7f3871c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0b81a0 con 0x7f386c0b97b0 2024-04-25T09:04:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.369+0000 7f3871c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0b7870 con 0x7f386c15dc10 2024-04-25T09:04:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.369+0000 7f386b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f386c0b97b0 0x7f386c0b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.369+0000 7f386b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f386c0b97b0 0x7f386c0b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46514/0 (socket says 172.21.15.62:46514) 2024-04-25T09:04:02.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.369+0000 7f386b400640 1 -- 172.21.15.62:0/3078099734 learned_addr learned my addr 172.21.15.62:0/3078099734 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.370+0000 7f386aa00640 1 --2- 172.21.15.62:0/3078099734 >> v2:172.21.15.121:3300/0 conn(0x7f386c15dc10 0x7f386c0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.370+0000 7f386b400640 1 -- 172.21.15.62:0/3078099734 >> v2:172.21.15.121:3300/0 conn(0x7f386c15dc10 msgr2=0x7f386c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.370+0000 7f386b400640 1 --2- 172.21.15.62:0/3078099734 >> v2:172.21.15.121:3300/0 conn(0x7f386c15dc10 0x7f386c0d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.370+0000 7f386b400640 1 -- 172.21.15.62:0/3078099734 >> v1:172.21.15.62:6789/0 conn(0x7f386c0d20a0 legacy=0x7f386c0b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.370+0000 7f386b400640 1 -- 172.21.15.62:0/3078099734 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0b7b50 con 0x7f386c0b97b0 2024-04-25T09:04:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.371+0000 7f386b400640 1 --2- 172.21.15.62:0/3078099734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c0b97b0 0x7f386c0b8630 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f386c08cd20 tx=0x7f386402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b33c94d7cfc8f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.371+0000 7f3869600640 1 -- 172.21.15.62:0/3078099734 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864036070 con 0x7f386c0b97b0 2024-04-25T09:04:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.371+0000 7f3869600640 1 -- 172.21.15.62:0/3078099734 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864031600 con 0x7f386c0b97b0 2024-04-25T09:04:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.371+0000 7f3869600640 1 -- 172.21.15.62:0/3078099734 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640319f0 con 0x7f386c0b97b0 2024-04-25T09:04:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.372+0000 7f3871c00640 1 -- 172.21.15.62:0/3078099734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c0b97b0 msgr2=0x7f386c0b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.372+0000 7f3871c00640 1 --2- 172.21.15.62:0/3078099734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c0b97b0 0x7f386c0b8630 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f386c08cd20 tx=0x7f386402a7c0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.372+0000 7f3871c00640 1 -- 172.21.15.62:0/3078099734 shutdown_connections 2024-04-25T09:04:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.372+0000 7f3871c00640 1 --2- 172.21.15.62:0/3078099734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c0b97b0 0x7f386c0b8630 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.372+0000 7f3871c00640 1 --2- 172.21.15.62:0/3078099734 >> v2:172.21.15.121:3300/0 conn(0x7f386c15dc10 0x7f386c0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.372+0000 7f3871c00640 1 -- 172.21.15.62:0/3078099734 >> 172.21.15.62:0/3078099734 conn(0x7f386c06b000 msgr2=0x7f386c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.372+0000 7f3871c00640 1 -- 172.21.15.62:0/3078099734 shutdown_connections 2024-04-25T09:04:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.372+0000 7f3871c00640 1 -- 172.21.15.62:0/3078099734 wait complete. 2024-04-25T09:04:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.374+0000 7f3871c00640 1 Processor -- start 2024-04-25T09:04:02.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.374+0000 7f3871c00640 1 -- start start 2024-04-25T09:04:02.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f3871c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f386c0b97b0 0x7f386c1f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f3871c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f386c15dc10 0x7f386c1f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f3871c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c1f7350 0x7f386c160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f3871c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c15dff0 con 0x7f386c1f7350 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f3871c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b72a0 con 0x7f386c15dc10 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f3871c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b8b70 con 0x7f386c0b97b0 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f386b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c1f7350 0x7f386c160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f386b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c1f7350 0x7f386c160340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46520/0 (socket says 172.21.15.62:46520) 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.375+0000 7f386b400640 1 -- 172.21.15.62:0/2556047156 learned_addr learned my addr 172.21.15.62:0/2556047156 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.376+0000 7f386aa00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f386c0b97b0 0x7f386c1f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.376+0000 7f386a000640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f386c15dc10 0x7f386c1f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.376+0000 7f386b400640 1 -- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f386c0b97b0 msgr2=0x7f386c1f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.376+0000 7f386b400640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f386c0b97b0 0x7f386c1f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.376+0000 7f386b400640 1 -- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f386c15dc10 msgr2=0x7f386c1f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.376+0000 7f386b400640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f386c15dc10 0x7f386c1f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.376+0000 7f386b400640 1 -- 172.21.15.62:0/2556047156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c1f78d0 con 0x7f386c1f7350 2024-04-25T09:04:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.376+0000 7f386b400640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c1f7350 0x7f386c160340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f386402a800 tx=0x7f3864004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.377+0000 7f3862a00640 1 -- 172.21.15.62:0/2556047156 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864036050 con 0x7f386c1f7350 2024-04-25T09:04:02.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.377+0000 7f3862a00640 1 -- 172.21.15.62:0/2556047156 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864042070 con 0x7f386c1f7350 2024-04-25T09:04:02.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.377+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c1f7bb0 con 0x7f386c1f7350 2024-04-25T09:04:02.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.377+0000 7f3862a00640 1 -- 172.21.15.62:0/2556047156 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864009510 con 0x7f386c1f7350 2024-04-25T09:04:02.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.377+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c150590 con 0x7f386c1f7350 2024-04-25T09:04:02.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.378+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c0b8f40 con 0x7f386c1f7350 2024-04-25T09:04:02.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.379+0000 7f3862a00640 1 -- 172.21.15.62:0/2556047156 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3864004450 con 0x7f386c1f7350 2024-04-25T09:04:02.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.380+0000 7f3862a00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f384c06b4d0 0x7f384c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.380+0000 7f3862a00640 1 -- 172.21.15.62:0/2556047156 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f38640aae80 con 0x7f386c1f7350 2024-04-25T09:04:02.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.381+0000 7f386aa00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f384c06b4d0 0x7f384c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.382+0000 7f386aa00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f384c06b4d0 0x7f384c06d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f385c006fd0 tx=0x7f385c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.384+0000 7f3862a00640 1 -- 172.21.15.62:0/2556047156 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f386c0b8f40 con 0x7f386c1f7350 2024-04-25T09:04:02.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.584+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f386c04ef20 con 0x7f386c1f7350 2024-04-25T09:04:02.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.584+0000 7f3862a00640 1 -- 172.21.15.62:0/2556047156 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f386404c020 con 0x7f386c1f7350 2024-04-25T09:04:02.574 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:04:02.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f384c06b4d0 msgr2=0x7f384c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f384c06b4d0 0x7f384c06d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f385c006fd0 tx=0x7f385c008040 comp rx=0 tx=0).stop 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c1f7350 msgr2=0x7f386c160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c1f7350 0x7f386c160340 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f386402a800 tx=0x7f3864004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 shutdown_connections 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f384c06b4d0 0x7f384c06d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386c1f7350 0x7f386c160340 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f386c15dc10 0x7f386c1f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 --2- 172.21.15.62:0/2556047156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f386c0b97b0 0x7f386c1f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 >> 172.21.15.62:0/2556047156 conn(0x7f386c06b000 msgr2=0x7f386c068b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.587+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 shutdown_connections 2024-04-25T09:04:02.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.588+0000 7f3871c00640 1 -- 172.21.15.62:0/2556047156 wait complete. 2024-04-25T09:04:02.596 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.819+0000 7f7d2f200640 1 Processor -- start 2024-04-25T09:04:02.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.819+0000 7f7d2f200640 1 -- start start 2024-04-25T09:04:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300ccde0 0x7f7d300ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d300b31a0 0x7f7d300b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d300ced40 con 0x7f7d3006d2c0 2024-04-25T09:04:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d300cd260 con 0x7f7d300b31a0 2024-04-25T09:04:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300cf000 con 0x7f7d300ccde0 2024-04-25T09:04:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d300b31a0 0x7f7d300b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7d300b31a0 0x7f7d300b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46532/0 (socket says 172.21.15.62:46532) 2024-04-25T09:04:02.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2e800640 1 -- 172.21.15.62:0/1621832598 learned_addr learned my addr 172.21.15.62:0/1621832598 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.821+0000 7f7d2d400640 1 --2- 172.21.15.62:0/1621832598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300ccde0 0x7f7d300ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.822+0000 7f7d2e800640 1 -- 172.21.15.62:0/1621832598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300ccde0 msgr2=0x7f7d300ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.822+0000 7f7d2e800640 1 --2- 172.21.15.62:0/1621832598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300ccde0 0x7f7d300ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.822+0000 7f7d2e800640 1 -- 172.21.15.62:0/1621832598 >> v1:172.21.15.121:6789/0 conn(0x7f7d3006d2c0 legacy=0x7f7d3006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.822+0000 7f7d2e800640 1 -- 172.21.15.62:0/1621832598 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300cf2e0 con 0x7f7d300b31a0 2024-04-25T09:04:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.823+0000 7f7d2e800640 1 --2- 172.21.15.62:0/1621832598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b31a0 0x7f7d300b3f40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009e50 tx=0x7f7d2802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b211753be80c5b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.823+0000 7f7d2ca00640 1 -- 172.21.15.62:0/1621832598 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28036070 con 0x7f7d300b31a0 2024-04-25T09:04:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.823+0000 7f7d2ca00640 1 -- 172.21.15.62:0/1621832598 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d28031600 con 0x7f7d300b31a0 2024-04-25T09:04:02.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.823+0000 7f7d2ca00640 1 -- 172.21.15.62:0/1621832598 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d280319d0 con 0x7f7d300b31a0 2024-04-25T09:04:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.824+0000 7f7d2f200640 1 -- 172.21.15.62:0/1621832598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b31a0 msgr2=0x7f7d300b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.824+0000 7f7d2f200640 1 --2- 172.21.15.62:0/1621832598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b31a0 0x7f7d300b3f40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009e50 tx=0x7f7d2802a710 comp rx=0 tx=0).stop 2024-04-25T09:04:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.824+0000 7f7d2f200640 1 -- 172.21.15.62:0/1621832598 shutdown_connections 2024-04-25T09:04:02.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.824+0000 7f7d2f200640 1 --2- 172.21.15.62:0/1621832598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d300b31a0 0x7f7d300b3f40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.824+0000 7f7d2f200640 1 --2- 172.21.15.62:0/1621832598 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300ccde0 0x7f7d300ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.824+0000 7f7d2f200640 1 -- 172.21.15.62:0/1621832598 >> 172.21.15.62:0/1621832598 conn(0x7f7d30066f20 msgr2=0x7f7d300b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.824+0000 7f7d2f200640 1 -- 172.21.15.62:0/1621832598 shutdown_connections 2024-04-25T09:04:02.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.824+0000 7f7d2f200640 1 -- 172.21.15.62:0/1621832598 wait complete. 2024-04-25T09:04:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.826+0000 7f7d2f200640 1 Processor -- start 2024-04-25T09:04:02.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.826+0000 7f7d2f200640 1 -- start start 2024-04-25T09:04:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.827+0000 7f7d2f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3006d2c0 0x7f7d3015b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.827+0000 7f7d2f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300b31a0 0x7f7d301e9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.827+0000 7f7d2f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d300ccde0 0x7f7d301e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.827+0000 7f7d2f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300b2c80 con 0x7f7d3006d2c0 2024-04-25T09:04:02.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.827+0000 7f7d2f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300b29a0 con 0x7f7d300ccde0 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.827+0000 7f7d2f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301448d0 con 0x7f7d300b31a0 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3006d2c0 0x7f7d3015b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3006d2c0 0x7f7d3015b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46538/0 (socket says 172.21.15.62:46538) 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2de00640 1 -- 172.21.15.62:0/482251540 learned_addr learned my addr 172.21.15.62:0/482251540 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2e800640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d300ccde0 0x7f7d301e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2d400640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300b31a0 0x7f7d301e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2de00640 1 -- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300b31a0 msgr2=0x7f7d301e9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2de00640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300b31a0 0x7f7d301e9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2de00640 1 -- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d300ccde0 msgr2=0x7f7d301e96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2de00640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d300ccde0 0x7f7d301e96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2de00640 1 -- 172.21.15.62:0/482251540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d301e8ad0 con 0x7f7d3006d2c0 2024-04-25T09:04:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.828+0000 7f7d2d400640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300b31a0 0x7f7d301e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:02.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.829+0000 7f7d2de00640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3006d2c0 0x7f7d3015b390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c3f0 tx=0x7f7d2000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.829+0000 7f7d26000640 1 -- 172.21.15.62:0/482251540 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200052d0 con 0x7f7d3006d2c0 2024-04-25T09:04:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.829+0000 7f7d26000640 1 -- 172.21.15.62:0/482251540 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20005490 con 0x7f7d3006d2c0 2024-04-25T09:04:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.829+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d301e7a30 con 0x7f7d3006d2c0 2024-04-25T09:04:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.829+0000 7f7d26000640 1 -- 172.21.15.62:0/482251540 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20007260 con 0x7f7d3006d2c0 2024-04-25T09:04:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.829+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d301e7ff0 con 0x7f7d3006d2c0 2024-04-25T09:04:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.830+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3006abe0 con 0x7f7d3006d2c0 2024-04-25T09:04:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.831+0000 7f7d26000640 1 -- 172.21.15.62:0/482251540 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d20007400 con 0x7f7d3006d2c0 2024-04-25T09:04:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.832+0000 7f7d26000640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d0c06b4d0 0x7f7d0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.832+0000 7f7d26000640 1 -- 172.21.15.62:0/482251540 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7d20089710 con 0x7f7d3006d2c0 2024-04-25T09:04:02.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.833+0000 7f7d2d400640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d0c06b4d0 0x7f7d0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.834+0000 7f7d2d400640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d0c06b4d0 0x7f7d0c06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7d301e6200 tx=0x7f7d18008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:02.837+0000 7f7d26000640 1 -- 172.21.15.62:0/482251540 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d20058560 con 0x7f7d3006d2c0 2024-04-25T09:04:03.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.037+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d300b2360 con 0x7f7d3006d2c0 2024-04-25T09:04:03.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.037+0000 7f7d26000640 1 -- 172.21.15.62:0/482251540 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d20057e60 con 0x7f7d3006d2c0 2024-04-25T09:04:03.026 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:04:03.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d0c06b4d0 msgr2=0x7f7d0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d0c06b4d0 0x7f7d0c06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7d301e6200 tx=0x7f7d18008040 comp rx=0 tx=0).stop 2024-04-25T09:04:03.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3006d2c0 msgr2=0x7f7d3015b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3006d2c0 0x7f7d3015b390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c3f0 tx=0x7f7d2000c8b0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 shutdown_connections 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d0c06b4d0 0x7f7d0c06d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d300ccde0 0x7f7d301e96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d300b31a0 0x7f7d301e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 --2- 172.21.15.62:0/482251540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d3006d2c0 0x7f7d3015b390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 >> 172.21.15.62:0/482251540 conn(0x7f7d30066f20 msgr2=0x7f7d3006c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 shutdown_connections 2024-04-25T09:04:03.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.040+0000 7f7d2f200640 1 -- 172.21.15.62:0/482251540 wait complete. 2024-04-25T09:04:03.049 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:03.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.273+0000 7fe9a4e00640 1 Processor -- start 2024-04-25T09:04:03.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.273+0000 7fe9a4e00640 1 -- start start 2024-04-25T09:04:03.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.275+0000 7fe9a4e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a015b9f0 0x7fe9a00d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.275+0000 7fe9a4e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe9a00d11d0 0x7fe9a0161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.275+0000 7fe9a4e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a015f2c0 con 0x7fe9a00b9470 2024-04-25T09:04:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.275+0000 7fe9a4e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a00b3420 con 0x7fe9a00d11d0 2024-04-25T09:04:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.275+0000 7fe9a4e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a015e6b0 con 0x7fe9a015b9f0 2024-04-25T09:04:03.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.275+0000 7fe99d400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe9a00b9470 0x7fe9a00d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40922/0 (socket says 172.21.15.62:40922) 2024-04-25T09:04:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.275+0000 7fe99d400640 1 -- 172.21.15.62:0/656361090 learned_addr learned my addr 172.21.15.62:0/656361090 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.275+0000 7fe99e800640 1 --2- 172.21.15.62:0/656361090 >> v2:172.21.15.62:3300/0 conn(0x7fe9a00d11d0 0x7fe9a0161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.276+0000 7fe99de00640 1 --2- 172.21.15.62:0/656361090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a015b9f0 0x7fe9a00d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.276+0000 7fe99ca00640 1 -- 172.21.15.62:0/656361090 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336659777 0 0) 0x7fe9a015f2c0 con 0x7fe9a00b9470 2024-04-25T09:04:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.276+0000 7fe99ca00640 1 -- 172.21.15.62:0/656361090 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe984003520 con 0x7fe9a00b9470 2024-04-25T09:04:03.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.276+0000 7fe99e800640 1 -- 172.21.15.62:0/656361090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a015b9f0 msgr2=0x7fe9a00d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.276+0000 7fe99e800640 1 --2- 172.21.15.62:0/656361090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a015b9f0 0x7fe9a00d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.276+0000 7fe99e800640 1 -- 172.21.15.62:0/656361090 >> v1:172.21.15.62:6789/0 conn(0x7fe9a00b9470 legacy=0x7fe9a00d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.276+0000 7fe99ca00640 1 -- 172.21.15.62:0/656361090 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3370402210 0 0) 0x7fe984003520 con 0x7fe9a00b9470 2024-04-25T09:04:03.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.276+0000 7fe99e800640 1 -- 172.21.15.62:0/656361090 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a015e990 con 0x7fe9a00d11d0 2024-04-25T09:04:03.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.277+0000 7fe99e800640 1 --2- 172.21.15.62:0/656361090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a00d11d0 0x7fe9a0161ba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe998009d90 tx=0x7fe99802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a8cc2241447aac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe99ca00640 1 -- 172.21.15.62:0/656361090 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c001670 con 0x7fe9a00d11d0 2024-04-25T09:04:03.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe99ca00640 1 -- 172.21.15.62:0/656361090 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998007540 con 0x7fe9a00d11d0 2024-04-25T09:04:03.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe99ca00640 1 -- 172.21.15.62:0/656361090 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998007910 con 0x7fe9a00d11d0 2024-04-25T09:04:03.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe9a4e00640 1 -- 172.21.15.62:0/656361090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a00d11d0 msgr2=0x7fe9a0161ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/656361090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a00d11d0 0x7fe9a0161ba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe998009d90 tx=0x7fe99802a5f0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe9a4e00640 1 -- 172.21.15.62:0/656361090 shutdown_connections 2024-04-25T09:04:03.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/656361090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a00d11d0 0x7fe9a0161ba0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/656361090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a015b9f0 0x7fe9a00d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe9a4e00640 1 -- 172.21.15.62:0/656361090 >> 172.21.15.62:0/656361090 conn(0x7fe9a006b000 msgr2=0x7fe9a006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:03.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe9a4e00640 1 -- 172.21.15.62:0/656361090 shutdown_connections 2024-04-25T09:04:03.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.278+0000 7fe9a4e00640 1 -- 172.21.15.62:0/656361090 wait complete. 2024-04-25T09:04:03.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.280+0000 7fe9a4e00640 1 Processor -- start 2024-04-25T09:04:03.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.280+0000 7fe9a4e00640 1 -- start start 2024-04-25T09:04:03.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.281+0000 7fe9a4e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9a00b9470 0x7fe9a01f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.281+0000 7fe9a4e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a00d11d0 0x7fe9a00cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.281+0000 7fe9a4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a015b9f0 0x7fe9a01f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.281+0000 7fe9a4e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a015e310 con 0x7fe9a015b9f0 2024-04-25T09:04:03.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.281+0000 7fe9a4e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a015ef50 con 0x7fe9a00b9470 2024-04-25T09:04:03.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.281+0000 7fe9a4e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a015e490 con 0x7fe9a00d11d0 2024-04-25T09:04:03.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a015b9f0 0x7fe9a01f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a015b9f0 0x7fe9a01f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46544/0 (socket says 172.21.15.62:46544) 2024-04-25T09:04:03.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 -- 172.21.15.62:0/279447494 learned_addr learned my addr 172.21.15.62:0/279447494 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:03.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 -- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a00d11d0 msgr2=0x7fe9a00cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a00d11d0 0x7fe9a00cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 -- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9a00b9470 msgr2=0x7fe9a01f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99de00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9a00b9470 0x7fe9a01f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9a00b9470 0x7fe9a01f87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 -- 172.21.15.62:0/279447494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00cc7b0 con 0x7fe9a015b9f0 2024-04-25T09:04:03.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.282+0000 7fe99e800640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a015b9f0 0x7fe9a01f3f20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe998009d90 tx=0x7fe998004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.283+0000 7fe996000640 1 -- 172.21.15.62:0/279447494 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998040070 con 0x7fe9a015b9f0 2024-04-25T09:04:03.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.283+0000 7fe996000640 1 -- 172.21.15.62:0/279447494 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998031b40 con 0x7fe9a015b9f0 2024-04-25T09:04:03.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.283+0000 7fe996000640 1 -- 172.21.15.62:0/279447494 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998007e90 con 0x7fe9a015b9f0 2024-04-25T09:04:03.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.283+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a00cca90 con 0x7fe9a015b9f0 2024-04-25T09:04:03.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.283+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a00cd050 con 0x7fe9a015b9f0 2024-04-25T09:04:03.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.284+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a00d3850 con 0x7fe9a015b9f0 2024-04-25T09:04:03.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.285+0000 7fe996000640 1 -- 172.21.15.62:0/279447494 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe998039040 con 0x7fe9a015b9f0 2024-04-25T09:04:03.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.286+0000 7fe996000640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.286+0000 7fe996000640 1 -- 172.21.15.62:0/279447494 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe9980ab5d0 con 0x7fe9a015b9f0 2024-04-25T09:04:03.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.286+0000 7fe99de00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.288+0000 7fe99de00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe98006b4d0 0x7fe98006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe9a015a9c0 tx=0x7fe990008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.290+0000 7fe996000640 1 -- 172.21.15.62:0/279447494 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9a00d3850 con 0x7fe9a015b9f0 2024-04-25T09:04:03.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.505+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe9a004f000 con 0x7fe9a015b9f0 2024-04-25T09:04:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.506+0000 7fe996000640 1 -- 172.21.15.62:0/279447494 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe99804b340 con 0x7fe9a015b9f0 2024-04-25T09:04:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:04:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe98006b4d0 msgr2=0x7fe98006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe98006b4d0 0x7fe98006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe9a015a9c0 tx=0x7fe990008040 comp rx=0 tx=0).stop 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a015b9f0 msgr2=0x7fe9a01f3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a015b9f0 0x7fe9a01f3f20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe998009d90 tx=0x7fe998004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 shutdown_connections 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe98006b4d0 0x7fe98006d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe9a015b9f0 0x7fe9a01f3f20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9a00d11d0 0x7fe9a00cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 --2- 172.21.15.62:0/279447494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9a00b9470 0x7fe9a01f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.509+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 >> 172.21.15.62:0/279447494 conn(0x7fe9a006b000 msgr2=0x7fe9a00d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.510+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 shutdown_connections 2024-04-25T09:04:03.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:03.510+0000 7fe9a4e00640 1 -- 172.21.15.62:0/279447494 wait complete. 2024-04-25T09:04:03.518 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T09:04:03.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:03.518 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T09:04:03.616 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:04:03.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:03.616 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:04:03.689 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:04:03.689 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:03.750 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:04:03.751 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:03.805 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:04:03.805 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:03.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:04.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.092+0000 7f02f5c00640 1 Processor -- start 2024-04-25T09:04:04.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.092+0000 7f02f5c00640 1 -- start start 2024-04-25T09:04:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.094+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02f013d970 0x7f02f00b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.094+0000 7f02f5c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f015b220 con 0x7f02f00b1050 2024-04-25T09:04:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.094+0000 7f02f5c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f015b500 con 0x7f02f006d900 2024-04-25T09:04:04.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.094+0000 7f02f5c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f015b7e0 con 0x7f02f013d970 2024-04-25T09:04:04.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.095+0000 7f02ef400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f02f00b1050 0x7f02f006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40934/0 (socket says 172.21.15.62:40934) 2024-04-25T09:04:04.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.095+0000 7f02ef400640 1 -- 172.21.15.62:0/2774528774 learned_addr learned my addr 172.21.15.62:0/2774528774 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:04.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.095+0000 7f02efe00640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02f013d970 0x7f02f00b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208504934 0 0) 0x7f02f015b220 con 0x7f02f00b1050 2024-04-25T09:04:04.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d4005730 con 0x7f02f00b1050 2024-04-25T09:04:04.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2768421671 0 0) 0x7f02d4005730 con 0x7f02f00b1050 2024-04-25T09:04:04.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02f013d970 msgr2=0x7f02f00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02f013d970 0x7f02f00b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 >> v1:172.21.15.121:6789/0 conn(0x7f02f006d900 legacy=0x7f02f013e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f015bac0 con 0x7f02f00b1050 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f02dc001670 con 0x7f02f00b1050 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 >> v1:172.21.15.62:6789/0 conn(0x7f02f00b1050 legacy=0x7f02f006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02f013d970 0x7f02d4005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d40061c0 0x7f02d4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d4006f50 0x7f02d40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f015b7e0 con 0x7f02d4006f50 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d400dd10 con 0x7f02d40061c0 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d400df10 con 0x7f02f013d970 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.096+0000 7f02f5200640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d4006f50 0x7f02d40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.097+0000 7f02efe00640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02f013d970 0x7f02d4005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.097+0000 7f02ef400640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d40061c0 0x7f02d4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.097+0000 7f02f5200640 1 -- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02f013d970 msgr2=0x7f02d4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.097+0000 7f02f5200640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02f013d970 0x7f02d4005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.097+0000 7f02f5200640 1 -- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d40061c0 msgr2=0x7f02d4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.097+0000 7f02f5200640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02d40061c0 0x7f02d4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.097+0000 7f02f5200640 1 -- 172.21.15.62:0/2774528774 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4005730 con 0x7f02d4006f50 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.098+0000 7f02f5200640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d4006f50 0x7f02d40073d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f02e8001750 tx=0x7f02d4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7540f4aaa1fbb94d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.098+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8001370 con 0x7f02d4006f50 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.098+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e80052d0 con 0x7f02d4006f50 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.098+0000 7f02eea00640 1 -- 172.21.15.62:0/2774528774 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e802abe0 con 0x7f02d4006f50 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.099+0000 7f02f5c00640 1 -- 172.21.15.62:0/2774528774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d4006f50 msgr2=0x7f02d40073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.099+0000 7f02f5c00640 1 --2- 172.21.15.62:0/2774528774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02d4006f50 0x7f02d40073d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f02e8001750 tx=0x7f02d4007a40 comp rx=0 tx=0).stop 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.099+0000 7f02f5200640 1 -- 172.21.15.62:0/2774528774 reap_dead start 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.099+0000 7f02f5c00640 1 -- 172.21.15.62:0/2774528774 shutdown_connections 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.099+0000 7f02f5c00640 1 -- 172.21.15.62:0/2774528774 >> 172.21.15.62:0/2774528774 conn(0x7f02f00b14d0 msgr2=0x7f02f00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.099+0000 7f02f5c00640 1 -- 172.21.15.62:0/2774528774 shutdown_connections 2024-04-25T09:04:04.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.099+0000 7f02f5c00640 1 -- 172.21.15.62:0/2774528774 wait complete. 2024-04-25T09:04:04.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.101+0000 7f02f5c00640 1 Processor -- start 2024-04-25T09:04:04.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.101+0000 7f02f5c00640 1 -- start start 2024-04-25T09:04:04.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.102+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02d4006f50 0x7f02f0138cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.102+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f014ef90 0x7f02f01f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.102+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02f01f3990 0x7f02f0159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.102+0000 7f02f5c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f00b0b10 con 0x7f02f014ef90 2024-04-25T09:04:04.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.102+0000 7f02f5c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f015af00 con 0x7f02f01f3990 2024-04-25T09:04:04.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.102+0000 7f02f5c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f00b0c90 con 0x7f02d4006f50 2024-04-25T09:04:04.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.102+0000 7f02ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f014ef90 0x7f02f01f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f014ef90 0x7f02f01f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46554/0 (socket says 172.21.15.62:46554) 2024-04-25T09:04:04.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02ef400640 1 -- 172.21.15.62:0/113911650 learned_addr learned my addr 172.21.15.62:0/113911650 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:04.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02efe00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02d4006f50 0x7f02f0138cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02ef400640 1 -- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02d4006f50 msgr2=0x7f02f0138cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02ef400640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02d4006f50 0x7f02f0138cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02f5200640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02f01f3990 0x7f02f0159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02ef400640 1 -- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02f01f3990 msgr2=0x7f02f0159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02ef400640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02f01f3990 0x7f02f0159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02ef400640 1 -- 172.21.15.62:0/113911650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0139300 con 0x7f02f014ef90 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.103+0000 7f02ef400640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f014ef90 0x7f02f01f3450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e890 tx=0x7f02dc00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.104+0000 7f02e7e00640 1 -- 172.21.15.62:0/113911650 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00c870 con 0x7f02f014ef90 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.104+0000 7f02e7e00640 1 -- 172.21.15.62:0/113911650 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc018070 con 0x7f02f014ef90 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.104+0000 7f02e7e00640 1 -- 172.21.15.62:0/113911650 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00cc50 con 0x7f02f014ef90 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.104+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f0133e10 con 0x7f02f014ef90 2024-04-25T09:04:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.104+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f01343d0 con 0x7f02f014ef90 2024-04-25T09:04:04.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.105+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f004ef20 con 0x7f02f014ef90 2024-04-25T09:04:04.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.106+0000 7f02e7e00640 1 -- 172.21.15.62:0/113911650 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02dc01d460 con 0x7f02f014ef90 2024-04-25T09:04:04.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.107+0000 7f02e7e00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02d006b4d0 0x7f02d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.107+0000 7f02e7e00640 1 -- 172.21.15.62:0/113911650 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f02dc08a170 con 0x7f02f014ef90 2024-04-25T09:04:04.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.108+0000 7f02efe00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02d006b4d0 0x7f02d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.109+0000 7f02efe00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02d006b4d0 0x7f02d006d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009fd0 tx=0x7f02e000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.111+0000 7f02e7e00640 1 -- 172.21.15.62:0/113911650 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02dc058f30 con 0x7f02f014ef90 2024-04-25T09:04:04.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.319+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02f015c570 con 0x7f02f014ef90 2024-04-25T09:04:04.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.322+0000 7f02e7e00640 1 -- 172.21.15.62:0/113911650 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f02dc058830 con 0x7f02f014ef90 2024-04-25T09:04:04.313 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:04.313 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:59.724537+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02d006b4d0 msgr2=0x7f02d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02d006b4d0 0x7f02d006d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02e0009fd0 tx=0x7f02e000b040 comp rx=0 tx=0).stop 2024-04-25T09:04:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f014ef90 msgr2=0x7f02f01f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f014ef90 0x7f02f01f3450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e890 tx=0x7f02dc00ed50 comp rx=0 tx=0).stop 2024-04-25T09:04:04.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 shutdown_connections 2024-04-25T09:04:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02d006b4d0 0x7f02d006d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02f014ef90 0x7f02f01f3450 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02f01f3990 0x7f02f0159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 --2- 172.21.15.62:0/113911650 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02d4006f50 0x7f02f0138cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 >> 172.21.15.62:0/113911650 conn(0x7f02f00b14d0 msgr2=0x7f02f013ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 shutdown_connections 2024-04-25T09:04:04.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.325+0000 7f02f5c00640 1 -- 172.21.15.62:0/113911650 wait complete. 2024-04-25T09:04:04.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:04.336 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:04.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.565+0000 7f17c6c00640 1 Processor -- start 2024-04-25T09:04:04.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.565+0000 7f17c6c00640 1 -- start start 2024-04-25T09:04:04.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.567+0000 7f17c6c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f17c00b31a0 0x7f17c00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.567+0000 7f17c6c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c00cd260 con 0x7f17c00ccde0 2024-04-25T09:04:04.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.567+0000 7f17c6c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c00cf000 con 0x7f17c006d2c0 2024-04-25T09:04:04.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.567+0000 7f17c6c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c00cf2e0 con 0x7f17c00b31a0 2024-04-25T09:04:04.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.567+0000 7f17c6200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f17c00b31a0 0x7f17c00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.567+0000 7f17c4e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f17c00ccde0 0x7f17c00ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40942/0 (socket says 172.21.15.62:40942) 2024-04-25T09:04:04.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.567+0000 7f17c4e00640 1 -- 172.21.15.62:0/873281243 learned_addr learned my addr 172.21.15.62:0/873281243 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:04.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.568+0000 7f17c6200640 1 -- 172.21.15.62:0/873281243 >> v1:172.21.15.121:6789/0 conn(0x7f17c006d2c0 legacy=0x7f17c006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.568+0000 7f17c6200640 1 -- 172.21.15.62:0/873281243 >> v1:172.21.15.62:6789/0 conn(0x7f17c00ccde0 legacy=0x7f17c00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.568+0000 7f17c6200640 1 -- 172.21.15.62:0/873281243 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00b26c0 con 0x7f17c00b31a0 2024-04-25T09:04:04.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.568+0000 7f17bfe00640 1 -- 172.21.15.62:0/873281243 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019315348 0 0) 0x7f17c00cd260 con 0x7f17c00ccde0 2024-04-25T09:04:04.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.569+0000 7f17c6200640 1 --2- 172.21.15.62:0/873281243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00b31a0 0x7f17c00b3f40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009d90 tx=0x7f17b802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5788d15a522974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.569+0000 7f17bfe00640 1 -- 172.21.15.62:0/873281243 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8007740 con 0x7f17c00b31a0 2024-04-25T09:04:04.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.569+0000 7f17bfe00640 1 -- 172.21.15.62:0/873281243 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b80078e0 con 0x7f17c00b31a0 2024-04-25T09:04:04.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.569+0000 7f17bfe00640 1 -- 172.21.15.62:0/873281243 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8007cb0 con 0x7f17c00b31a0 2024-04-25T09:04:04.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.570+0000 7f17c6c00640 1 -- 172.21.15.62:0/873281243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00b31a0 msgr2=0x7f17c00b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.570+0000 7f17c6c00640 1 --2- 172.21.15.62:0/873281243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00b31a0 0x7f17c00b3f40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009d90 tx=0x7f17b802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.570+0000 7f17c6c00640 1 -- 172.21.15.62:0/873281243 shutdown_connections 2024-04-25T09:04:04.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.570+0000 7f17c6c00640 1 --2- 172.21.15.62:0/873281243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00b31a0 0x7f17c00b3f40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.570+0000 7f17c6c00640 1 -- 172.21.15.62:0/873281243 >> 172.21.15.62:0/873281243 conn(0x7f17c0066f20 msgr2=0x7f17c00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.570+0000 7f17c6c00640 1 -- 172.21.15.62:0/873281243 shutdown_connections 2024-04-25T09:04:04.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.570+0000 7f17c6c00640 1 -- 172.21.15.62:0/873281243 wait complete. 2024-04-25T09:04:04.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.572+0000 7f17c6c00640 1 Processor -- start 2024-04-25T09:04:04.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.572+0000 7f17c6c00640 1 -- start start 2024-04-25T09:04:04.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17c006d2c0 0x7f17c013d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17c00b31a0 0x7f17c015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00ccde0 0x7f17c01f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c01448d0 con 0x7f17c00ccde0 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00b2c80 con 0x7f17c00b31a0 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0144a50 con 0x7f17c006d2c0 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00ccde0 0x7f17c01f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00ccde0 0x7f17c01f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46580/0 (socket says 172.21.15.62:46580) 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.573+0000 7f17c6200640 1 -- 172.21.15.62:0/1443614880 learned_addr learned my addr 172.21.15.62:0/1443614880 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17c5800640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17c006d2c0 0x7f17c013d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17c4e00640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17c00b31a0 0x7f17c015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17c6200640 1 -- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17c006d2c0 msgr2=0x7f17c013d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17c6200640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17c006d2c0 0x7f17c013d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17c6200640 1 -- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17c00b31a0 msgr2=0x7f17c015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17c6200640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17c00b31a0 0x7f17c015df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17c6200640 1 -- 172.21.15.62:0/1443614880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c013cf40 con 0x7f17c00ccde0 2024-04-25T09:04:04.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17c6200640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00ccde0 0x7f17c01f6f70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009eb0 tx=0x7f17b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.574+0000 7f17bd600640 1 -- 172.21.15.62:0/1443614880 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8040070 con 0x7f17c00ccde0 2024-04-25T09:04:04.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.575+0000 7f17bd600640 1 -- 172.21.15.62:0/1443614880 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8031c40 con 0x7f17c00ccde0 2024-04-25T09:04:04.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.575+0000 7f17bd600640 1 -- 172.21.15.62:0/1443614880 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b8003990 con 0x7f17c00ccde0 2024-04-25T09:04:04.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.575+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c00ade50 con 0x7f17c00ccde0 2024-04-25T09:04:04.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.575+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c00ae410 con 0x7f17c00ccde0 2024-04-25T09:04:04.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.576+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c00b2480 con 0x7f17c00ccde0 2024-04-25T09:04:04.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.577+0000 7f17bd600640 1 -- 172.21.15.62:0/1443614880 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17b8039040 con 0x7f17c00ccde0 2024-04-25T09:04:04.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.578+0000 7f17bd600640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17a006b4d0 0x7f17a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.578+0000 7f17bd600640 1 -- 172.21.15.62:0/1443614880 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f17b80ab560 con 0x7f17c00ccde0 2024-04-25T09:04:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.578+0000 7f17c5800640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17a006b4d0 0x7f17a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.579+0000 7f17c5800640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17a006b4d0 0x7f17a006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17c01f44a0 tx=0x7f17b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.582+0000 7f17bd600640 1 -- 172.21.15.62:0/1443614880 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17c00b2480 con 0x7f17c00ccde0 2024-04-25T09:04:04.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.785+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17c006b210 con 0x7f17c00ccde0 2024-04-25T09:04:04.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.786+0000 7f17bd600640 1 -- 172.21.15.62:0/1443614880 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17b807a3b0 con 0x7f17c00ccde0 2024-04-25T09:04:04.775 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:04:04.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17a006b4d0 msgr2=0x7f17a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17a006b4d0 0x7f17a006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17c01f44a0 tx=0x7f17b4009040 comp rx=0 tx=0).stop 2024-04-25T09:04:04.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00ccde0 msgr2=0x7f17c01f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00ccde0 0x7f17c01f6f70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17b8009eb0 tx=0x7f17b8004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 shutdown_connections 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f17a006b4d0 0x7f17a006d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f17c00ccde0 0x7f17c01f6f70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17c00b31a0 0x7f17c015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 --2- 172.21.15.62:0/1443614880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f17c006d2c0 0x7f17c013d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 >> 172.21.15.62:0/1443614880 conn(0x7f17c0066f20 msgr2=0x7f17c006cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 shutdown_connections 2024-04-25T09:04:04.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:04.789+0000 7f17c6c00640 1 -- 172.21.15.62:0/1443614880 wait complete. 2024-04-25T09:04:04.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:05.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.023+0000 7f37eae00640 1 Processor -- start 2024-04-25T09:04:05.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.024+0000 7f37eae00640 1 -- start start 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.025+0000 7f37eae00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f37e406d2c0 0x7f37e40ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.025+0000 7f37eae00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f37e40b31a0 0x7f37e40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.025+0000 7f37eae00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e40cf000 con 0x7f37e406d6a0 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.025+0000 7f37eae00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e40cf2e0 con 0x7f37e40b31a0 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.025+0000 7f37eae00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e40b26c0 con 0x7f37e406d2c0 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.026+0000 7f37ea400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f37e40b31a0 0x7f37e40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.026+0000 7f37ea400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f37e40b31a0 0x7f37e40b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50562/0 (socket says 172.21.15.62:50562) 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.026+0000 7f37ea400640 1 -- 172.21.15.62:0/544939837 learned_addr learned my addr 172.21.15.62:0/544939837 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:05.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.026+0000 7f37e9a00640 1 --2- 172.21.15.62:0/544939837 >> v2:172.21.15.121:3300/0 conn(0x7f37e406d2c0 0x7f37e40ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.027+0000 7f37ea400640 1 -- 172.21.15.62:0/544939837 >> v2:172.21.15.121:3300/0 conn(0x7f37e406d2c0 msgr2=0x7f37e40ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.027+0000 7f37ea400640 1 --2- 172.21.15.62:0/544939837 >> v2:172.21.15.121:3300/0 conn(0x7f37e406d2c0 0x7f37e40ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.027+0000 7f37ea400640 1 -- 172.21.15.62:0/544939837 >> v1:172.21.15.121:6789/0 conn(0x7f37e406d6a0 legacy=0x7f37e40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.027+0000 7f37ea400640 1 -- 172.21.15.62:0/544939837 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e40b29a0 con 0x7f37e40b31a0 2024-04-25T09:04:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.028+0000 7f37ea400640 1 --2- 172.21.15.62:0/544939837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e40b31a0 0x7f37e40b3f40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f37e40b20a0 tx=0x7f37dc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bfffec0976162e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37e3e00640 1 -- 172.21.15.62:0/544939837 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc036070 con 0x7f37e40b31a0 2024-04-25T09:04:05.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37e3e00640 1 -- 172.21.15.62:0/544939837 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc031600 con 0x7f37e40b31a0 2024-04-25T09:04:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37e3e00640 1 -- 172.21.15.62:0/544939837 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0319d0 con 0x7f37e40b31a0 2024-04-25T09:04:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37eae00640 1 -- 172.21.15.62:0/544939837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e40b31a0 msgr2=0x7f37e40b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37eae00640 1 --2- 172.21.15.62:0/544939837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e40b31a0 0x7f37e40b3f40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f37e40b20a0 tx=0x7f37dc02a730 comp rx=0 tx=0).stop 2024-04-25T09:04:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37eae00640 1 -- 172.21.15.62:0/544939837 shutdown_connections 2024-04-25T09:04:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37eae00640 1 --2- 172.21.15.62:0/544939837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e40b31a0 0x7f37e40b3f40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37eae00640 1 --2- 172.21.15.62:0/544939837 >> v2:172.21.15.121:3300/0 conn(0x7f37e406d2c0 0x7f37e40ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37eae00640 1 -- 172.21.15.62:0/544939837 >> 172.21.15.62:0/544939837 conn(0x7f37e4066f20 msgr2=0x7f37e40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.029+0000 7f37eae00640 1 -- 172.21.15.62:0/544939837 shutdown_connections 2024-04-25T09:04:05.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.030+0000 7f37eae00640 1 -- 172.21.15.62:0/544939837 wait complete. 2024-04-25T09:04:05.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.031+0000 7f37eae00640 1 Processor -- start 2024-04-25T09:04:05.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.032+0000 7f37eae00640 1 -- start start 2024-04-25T09:04:05.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.032+0000 7f37eae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e406d2c0 0x7f37e415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.032+0000 7f37eae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e406d6a0 0x7f37e415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37eae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37e40b31a0 0x7f37e413f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37eae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e41448d0 con 0x7f37e406d6a0 2024-04-25T09:04:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37eae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e406db20 con 0x7f37e406d2c0 2024-04-25T09:04:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37eae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e4144a50 con 0x7f37e40b31a0 2024-04-25T09:04:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e406d6a0 0x7f37e415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e406d6a0 0x7f37e415b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50564/0 (socket says 172.21.15.62:50564) 2024-04-25T09:04:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9000640 1 -- 172.21.15.62:0/719981006 learned_addr learned my addr 172.21.15.62:0/719981006 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9000640 1 -- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37e40b31a0 msgr2=0x7f37e413f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9a00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e406d2c0 0x7f37e415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9000640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37e40b31a0 0x7f37e413f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9000640 1 -- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e406d2c0 msgr2=0x7f37e415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9000640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e406d2c0 0x7f37e415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.033+0000 7f37e9000640 1 -- 172.21.15.62:0/719981006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e413e850 con 0x7f37e406d6a0 2024-04-25T09:04:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.034+0000 7f37e9000640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e406d6a0 0x7f37e415b290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c400 tx=0x7f37d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.034+0000 7f37e1600640 1 -- 172.21.15.62:0/719981006 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d00052d0 con 0x7f37e406d6a0 2024-04-25T09:04:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.034+0000 7f37e1600640 1 -- 172.21.15.62:0/719981006 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d0005490 con 0x7f37e406d6a0 2024-04-25T09:04:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.034+0000 7f37e1600640 1 -- 172.21.15.62:0/719981006 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0007260 con 0x7f37e406d6a0 2024-04-25T09:04:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.034+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e413eb30 con 0x7f37e406d6a0 2024-04-25T09:04:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.034+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e415c070 con 0x7f37e406d6a0 2024-04-25T09:04:05.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.035+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e406abe0 con 0x7f37e406d6a0 2024-04-25T09:04:05.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.036+0000 7f37e1600640 1 -- 172.21.15.62:0/719981006 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37d0007400 con 0x7f37e406d6a0 2024-04-25T09:04:05.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.038+0000 7f37e1600640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37c406b4d0 0x7f37c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.038+0000 7f37e1600640 1 -- 172.21.15.62:0/719981006 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f37d0089780 con 0x7f37e406d6a0 2024-04-25T09:04:05.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.038+0000 7f37e9a00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37c406b4d0 0x7f37c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.039+0000 7f37e9a00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37c406b4d0 0x7f37c406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009a10 tx=0x7f37d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.042+0000 7f37e1600640 1 -- 172.21.15.62:0/719981006 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37d00585d0 con 0x7f37e406d6a0 2024-04-25T09:04:05.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.248+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37e404ef20 con 0x7f37e406d6a0 2024-04-25T09:04:05.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.249+0000 7f37e1600640 1 -- 172.21.15.62:0/719981006 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37d0057ed0 con 0x7f37e406d6a0 2024-04-25T09:04:05.238 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:04:05.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37c406b4d0 msgr2=0x7f37c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37c406b4d0 0x7f37c406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009a10 tx=0x7f37d8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:05.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e406d6a0 msgr2=0x7f37e415b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e406d6a0 0x7f37e415b290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c400 tx=0x7f37d000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 shutdown_connections 2024-04-25T09:04:05.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37c406b4d0 0x7f37c406d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f37e406d6a0 0x7f37e415b290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f37e40b31a0 0x7f37e413f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 --2- 172.21.15.62:0/719981006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37e406d2c0 0x7f37e415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 >> 172.21.15.62:0/719981006 conn(0x7f37e4066f20 msgr2=0x7f37e4144bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 shutdown_connections 2024-04-25T09:04:05.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.252+0000 7f37eae00640 1 -- 172.21.15.62:0/719981006 wait complete. 2024-04-25T09:04:05.261 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.481+0000 7f8723200640 1 Processor -- start 2024-04-25T09:04:05.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.482+0000 7f8723200640 1 -- start start 2024-04-25T09:04:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.483+0000 7f8723200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f872406d2c0 0x7f87240ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.483+0000 7f8723200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f872406d6a0 0x7f87240ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.483+0000 7f8723200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240ced40 con 0x7f87240b31a0 2024-04-25T09:04:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.483+0000 7f8723200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240cf000 con 0x7f872406d6a0 2024-04-25T09:04:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.483+0000 7f8723200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240cf2e0 con 0x7f872406d2c0 2024-04-25T09:04:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.484+0000 7f871be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f872406d6a0 0x7f87240ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.484+0000 7f871be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f872406d6a0 0x7f87240ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50576/0 (socket says 172.21.15.62:50576) 2024-04-25T09:04:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.484+0000 7f871be00640 1 -- 172.21.15.62:0/513172785 learned_addr learned my addr 172.21.15.62:0/513172785 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:05.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.484+0000 7f8721e00640 1 --2- 172.21.15.62:0/513172785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f872406d2c0 0x7f87240ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.484+0000 7f871be00640 1 -- 172.21.15.62:0/513172785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f872406d2c0 msgr2=0x7f87240ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.484+0000 7f871be00640 1 --2- 172.21.15.62:0/513172785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f872406d2c0 0x7f87240ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.484+0000 7f871be00640 1 -- 172.21.15.62:0/513172785 >> v1:172.21.15.121:6789/0 conn(0x7f87240b31a0 legacy=0x7f87240b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.484+0000 7f871be00640 1 -- 172.21.15.62:0/513172785 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240b26c0 con 0x7f872406d6a0 2024-04-25T09:04:05.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.486+0000 7f871be00640 1 --2- 172.21.15.62:0/513172785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f87240ce800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f870c009870 tx=0x7f870c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37673c14d796d933 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.486+0000 7f8721400640 1 -- 172.21.15.62:0/513172785 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c037070 con 0x7f872406d6a0 2024-04-25T09:04:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.486+0000 7f8721400640 1 -- 172.21.15.62:0/513172785 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c032440 con 0x7f872406d6a0 2024-04-25T09:04:05.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.486+0000 7f8721400640 1 -- 172.21.15.62:0/513172785 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c032810 con 0x7f872406d6a0 2024-04-25T09:04:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.486+0000 7f8723200640 1 -- 172.21.15.62:0/513172785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 msgr2=0x7f87240ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.486+0000 7f8723200640 1 --2- 172.21.15.62:0/513172785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f87240ce800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f870c009870 tx=0x7f870c02a210 comp rx=0 tx=0).stop 2024-04-25T09:04:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.487+0000 7f8723200640 1 -- 172.21.15.62:0/513172785 shutdown_connections 2024-04-25T09:04:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.487+0000 7f8723200640 1 --2- 172.21.15.62:0/513172785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f87240ce800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.487+0000 7f8723200640 1 --2- 172.21.15.62:0/513172785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f872406d2c0 0x7f87240ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.487+0000 7f8723200640 1 -- 172.21.15.62:0/513172785 >> 172.21.15.62:0/513172785 conn(0x7f8724066f20 msgr2=0x7f87240b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.487+0000 7f8723200640 1 -- 172.21.15.62:0/513172785 shutdown_connections 2024-04-25T09:04:05.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.487+0000 7f8723200640 1 -- 172.21.15.62:0/513172785 wait complete. 2024-04-25T09:04:05.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.489+0000 7f8723200640 1 Processor -- start 2024-04-25T09:04:05.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.489+0000 7f8723200640 1 -- start start 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f8723200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872406d2c0 0x7f87241e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f8723200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f872415b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f8723200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87240b31a0 0x7f87241e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f8723200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872406db20 con 0x7f872406d6a0 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f8723200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b2c80 con 0x7f872406d2c0 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f8723200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87241448d0 con 0x7f87240b31a0 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f871be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f872415b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f871be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f872415b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50586/0 (socket says 172.21.15.62:50586) 2024-04-25T09:04:05.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f871be00640 1 -- 172.21.15.62:0/86837527 learned_addr learned my addr 172.21.15.62:0/86837527 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f8722800640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87240b31a0 0x7f87241e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f871be00640 1 -- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87240b31a0 msgr2=0x7f87241e68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.490+0000 7f8721e00640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872406d2c0 0x7f87241e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f871be00640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87240b31a0 0x7f87241e68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f871be00640 1 -- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872406d2c0 msgr2=0x7f87241e4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f871be00640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872406d2c0 0x7f87241e4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f871be00640 1 -- 172.21.15.62:0/86837527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87241e5380 con 0x7f872406d6a0 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f8722800640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87240b31a0 0x7f87241e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f8721e00640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872406d2c0 0x7f87241e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f871be00640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f872415b490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f870c000c00 tx=0x7f870c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f871a000640 1 -- 172.21.15.62:0/86837527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c041070 con 0x7f872406d6a0 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f871a000640 1 -- 172.21.15.62:0/86837527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c03c890 con 0x7f872406d6a0 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f871a000640 1 -- 172.21.15.62:0/86837527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c037040 con 0x7f872406d6a0 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87241e5660 con 0x7f872406d6a0 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.491+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87241e5c20 con 0x7f872406d6a0 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.493+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872404ef20 con 0x7f872406d6a0 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.493+0000 7f871a000640 1 -- 172.21.15.62:0/86837527 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f870c044070 con 0x7f872406d6a0 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.494+0000 7f871a000640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f870006b4d0 0x7f870006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.494+0000 7f871a000640 1 -- 172.21.15.62:0/86837527 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f870c0aadd0 con 0x7f872406d6a0 2024-04-25T09:04:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.495+0000 7f8721e00640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f870006b4d0 0x7f870006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.496+0000 7f8721e00640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f870006b4d0 0x7f870006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8714006fd0 tx=0x7f8714008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.499+0000 7f871a000640 1 -- 172.21.15.62:0/86837527 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f870c079d40 con 0x7f872406d6a0 2024-04-25T09:04:05.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.702+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87240cbc30 con 0x7f872406d6a0 2024-04-25T09:04:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.703+0000 7f871a000640 1 -- 172.21.15.62:0/86837527 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f870c079640 con 0x7f872406d6a0 2024-04-25T09:04:05.692 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f870006b4d0 msgr2=0x7f870006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f870006b4d0 0x7f870006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8714006fd0 tx=0x7f8714008040 comp rx=0 tx=0).stop 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 msgr2=0x7f872415b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f872415b490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f870c000c00 tx=0x7f870c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 shutdown_connections 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f870006b4d0 0x7f870006d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f872406d6a0 0x7f872415b490 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87240b31a0 0x7f87241e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 --2- 172.21.15.62:0/86837527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872406d2c0 0x7f87241e4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 >> 172.21.15.62:0/86837527 conn(0x7f8724066f20 msgr2=0x7f87240ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 shutdown_connections 2024-04-25T09:04:05.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.706+0000 7f8723200640 1 -- 172.21.15.62:0/86837527 wait complete. 2024-04-25T09:04:05.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:05.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.939+0000 7f096c600640 1 Processor -- start 2024-04-25T09:04:05.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.939+0000 7f096c600640 1 -- start start 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.940+0000 7f096c600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f096415bd10 0x7f09640cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.940+0000 7f096c600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 0x7f09640ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.940+0000 7f096c600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09640aed00 0x7f096415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.940+0000 7f096c600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096406d660 con 0x7f09640aed00 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.941+0000 7f096c600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096406ca70 con 0x7f096415bd10 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.941+0000 7f096c600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096415c0f0 con 0x7f09640cccf0 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.941+0000 7f096a600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09640aed00 0x7f096415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.941+0000 7f096a600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f09640aed00 0x7f096415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50588/0 (socket says 172.21.15.62:50588) 2024-04-25T09:04:05.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.941+0000 7f096a600640 1 -- 172.21.15.62:0/864389135 learned_addr learned my addr 172.21.15.62:0/864389135 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.941+0000 7f0969200640 1 --2- 172.21.15.62:0/864389135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 0x7f09640ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.941+0000 7f0969c00640 1 --2- 172.21.15.62:0/864389135 >> v2:172.21.15.121:3300/0 conn(0x7f096415bd10 0x7f09640cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.942+0000 7f096a600640 1 -- 172.21.15.62:0/864389135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 msgr2=0x7f09640ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.942+0000 7f096a600640 1 --2- 172.21.15.62:0/864389135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 0x7f09640ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.942+0000 7f096a600640 1 -- 172.21.15.62:0/864389135 >> v2:172.21.15.121:3300/0 conn(0x7f096415bd10 msgr2=0x7f09640cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.942+0000 7f096a600640 1 --2- 172.21.15.62:0/864389135 >> v2:172.21.15.121:3300/0 conn(0x7f096415bd10 0x7f09640cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.942+0000 7f096a600640 1 -- 172.21.15.62:0/864389135 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096406cd30 con 0x7f09640aed00 2024-04-25T09:04:05.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.943+0000 7f096a600640 1 --2- 172.21.15.62:0/864389135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09640aed00 0x7f096415f830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0964072720 tx=0x7f095c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8258dc52266ff6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.944+0000 7f0963e00640 1 -- 172.21.15.62:0/864389135 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c036070 con 0x7f09640aed00 2024-04-25T09:04:05.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.944+0000 7f0963e00640 1 -- 172.21.15.62:0/864389135 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c031600 con 0x7f09640aed00 2024-04-25T09:04:05.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.944+0000 7f0963e00640 1 -- 172.21.15.62:0/864389135 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0319d0 con 0x7f09640aed00 2024-04-25T09:04:05.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.944+0000 7f096c600640 1 -- 172.21.15.62:0/864389135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09640aed00 msgr2=0x7f096415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.944+0000 7f096c600640 1 --2- 172.21.15.62:0/864389135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09640aed00 0x7f096415f830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0964072720 tx=0x7f095c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.945+0000 7f096c600640 1 -- 172.21.15.62:0/864389135 shutdown_connections 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.945+0000 7f096c600640 1 --2- 172.21.15.62:0/864389135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09640aed00 0x7f096415f830 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.945+0000 7f096c600640 1 --2- 172.21.15.62:0/864389135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 0x7f09640ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.945+0000 7f096c600640 1 --2- 172.21.15.62:0/864389135 >> v2:172.21.15.121:3300/0 conn(0x7f096415bd10 0x7f09640cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.945+0000 7f096c600640 1 -- 172.21.15.62:0/864389135 >> 172.21.15.62:0/864389135 conn(0x7f09640668f0 msgr2=0x7f0964066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.945+0000 7f096c600640 1 -- 172.21.15.62:0/864389135 shutdown_connections 2024-04-25T09:04:05.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.945+0000 7f096c600640 1 -- 172.21.15.62:0/864389135 wait complete. 2024-04-25T09:04:05.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.946+0000 7f096c600640 1 Processor -- start 2024-04-25T09:04:05.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.947+0000 7f096c600640 1 -- start start 2024-04-25T09:04:05.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.947+0000 7f096c600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09640aed00 0x7f0964145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.947+0000 7f096c600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 0x7f0964154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.947+0000 7f096c600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096415bd10 0x7f0964156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.947+0000 7f096c600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09640d01e0 con 0x7f096415bd10 2024-04-25T09:04:05.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.947+0000 7f096c600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096406d2f0 con 0x7f09640aed00 2024-04-25T09:04:05.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.947+0000 7f096c600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096406daf0 con 0x7f09640cccf0 2024-04-25T09:04:05.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f096a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096415bd10 0x7f0964156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f096a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096415bd10 0x7f0964156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50592/0 (socket says 172.21.15.62:50592) 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f096a600640 1 -- 172.21.15.62:0/2357842026 learned_addr learned my addr 172.21.15.62:0/2357842026 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f0969200640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 0x7f0964154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f0969c00640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09640aed00 0x7f0964145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f096a600640 1 -- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 msgr2=0x7f0964154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f096a600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 0x7f0964154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f096a600640 1 -- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09640aed00 msgr2=0x7f0964145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f096a600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09640aed00 0x7f0964145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f096a600640 1 -- 172.21.15.62:0/2357842026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0964152590 con 0x7f096415bd10 2024-04-25T09:04:05.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.948+0000 7f0969c00640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09640aed00 0x7f0964145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:05.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.949+0000 7f096a600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096415bd10 0x7f0964156150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f095c009f40 tx=0x7f095c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.949+0000 7f0961600640 1 -- 172.21.15.62:0/2357842026 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c036050 con 0x7f096415bd10 2024-04-25T09:04:05.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.949+0000 7f0961600640 1 -- 172.21.15.62:0/2357842026 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c042070 con 0x7f096415bd10 2024-04-25T09:04:05.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.949+0000 7f0961600640 1 -- 172.21.15.62:0/2357842026 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c009510 con 0x7f096415bd10 2024-04-25T09:04:05.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.949+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0964152870 con 0x7f096415bd10 2024-04-25T09:04:05.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.949+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0964152e30 con 0x7f096415bd10 2024-04-25T09:04:05.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.951+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0964072c70 con 0x7f096415bd10 2024-04-25T09:04:05.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.952+0000 7f0961600640 1 -- 172.21.15.62:0/2357842026 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f095c007bd0 con 0x7f096415bd10 2024-04-25T09:04:05.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.953+0000 7f0961600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094406b4d0 0x7f094406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.953+0000 7f0961600640 1 -- 172.21.15.62:0/2357842026 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f095c0ab140 con 0x7f096415bd10 2024-04-25T09:04:05.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.953+0000 7f0969c00640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094406b4d0 0x7f094406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.954+0000 7f0969c00640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094406b4d0 0x7f094406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0958006fd0 tx=0x7f0958008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:05.957+0000 7f0961600640 1 -- 172.21.15.62:0/2357842026 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f095c045360 con 0x7f096415bd10 2024-04-25T09:04:06.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.168+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0964071c00 con 0x7f096415bd10 2024-04-25T09:04:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.169+0000 7f0961600640 1 -- 172.21.15.62:0/2357842026 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f095c079f90 con 0x7f096415bd10 2024-04-25T09:04:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.171+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094406b4d0 msgr2=0x7f094406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.171+0000 7f096c600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094406b4d0 0x7f094406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0958006fd0 tx=0x7f0958008040 comp rx=0 tx=0).stop 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.171+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096415bd10 msgr2=0x7f0964156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.171+0000 7f096c600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096415bd10 0x7f0964156150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f095c009f40 tx=0x7f095c004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.172+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 shutdown_connections 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.172+0000 7f096c600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f094406b4d0 0x7f094406d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.172+0000 7f096c600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f096415bd10 0x7f0964156150 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.172+0000 7f096c600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09640cccf0 0x7f0964154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.172+0000 7f096c600640 1 --2- 172.21.15.62:0/2357842026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09640aed00 0x7f0964145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.172+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 >> 172.21.15.62:0/2357842026 conn(0x7f09640668f0 msgr2=0x7f09640b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.172+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 shutdown_connections 2024-04-25T09:04:06.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.172+0000 7f096c600640 1 -- 172.21.15.62:0/2357842026 wait complete. 2024-04-25T09:04:06.180 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T09:04:06.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:06.180 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T09:04:06.268 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:04:06.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:06.269 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:04:06.324 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:04:06.324 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:06.381 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:04:06.382 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:06.432 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:04:06.432 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:06.482 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:06.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.713+0000 7f1173000640 1 Processor -- start 2024-04-25T09:04:06.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.713+0000 7f1173000640 1 -- start start 2024-04-25T09:04:06.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.715+0000 7f1173000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f116c0b56f0 0x7f116c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.715+0000 7f1173000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f116c0b27a0 0x7f116c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.715+0000 7f1173000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c0d3110 con 0x7f116c0b6a90 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.715+0000 7f1173000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c161780 con 0x7f116c0b27a0 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.715+0000 7f1173000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c0cedc0 con 0x7f116c0b56f0 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.715+0000 7f1172600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f116c0b27a0 0x7f116c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.715+0000 7f1172600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f116c0b27a0 0x7f116c161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50604/0 (socket says 172.21.15.62:50604) 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.715+0000 7f1172600640 1 -- 172.21.15.62:0/3320583374 learned_addr learned my addr 172.21.15.62:0/3320583374 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.716+0000 7f1171c00640 1 --2- 172.21.15.62:0/3320583374 >> v2:172.21.15.121:3300/0 conn(0x7f116c0b56f0 0x7f116c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.716+0000 7f1172600640 1 -- 172.21.15.62:0/3320583374 >> v2:172.21.15.121:3300/0 conn(0x7f116c0b56f0 msgr2=0x7f116c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.716+0000 7f1172600640 1 --2- 172.21.15.62:0/3320583374 >> v2:172.21.15.121:3300/0 conn(0x7f116c0b56f0 0x7f116c160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.716+0000 7f1172600640 1 -- 172.21.15.62:0/3320583374 >> v1:172.21.15.121:6789/0 conn(0x7f116c0b6a90 legacy=0x7f116c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.716+0000 7f1172600640 1 -- 172.21.15.62:0/3320583374 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c0d33f0 con 0x7f116c0b27a0 2024-04-25T09:04:06.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.716+0000 7f1171c00640 1 --2- 172.21.15.62:0/3320583374 >> v2:172.21.15.121:3300/0 conn(0x7f116c0b56f0 0x7f116c160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.717+0000 7f1172600640 1 --2- 172.21.15.62:0/3320583374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b27a0 0x7f116c161240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f116c08c310 tx=0x7f116402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3e0d1388a4dbbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.717+0000 7f116be00640 1 -- 172.21.15.62:0/3320583374 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164036070 con 0x7f116c0b27a0 2024-04-25T09:04:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.717+0000 7f116be00640 1 -- 172.21.15.62:0/3320583374 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1164031600 con 0x7f116c0b27a0 2024-04-25T09:04:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.717+0000 7f116be00640 1 -- 172.21.15.62:0/3320583374 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11640319d0 con 0x7f116c0b27a0 2024-04-25T09:04:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.718+0000 7f1173000640 1 -- 172.21.15.62:0/3320583374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b27a0 msgr2=0x7f116c161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.718+0000 7f1173000640 1 --2- 172.21.15.62:0/3320583374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b27a0 0x7f116c161240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f116c08c310 tx=0x7f116402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.718+0000 7f1173000640 1 -- 172.21.15.62:0/3320583374 shutdown_connections 2024-04-25T09:04:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.718+0000 7f1173000640 1 --2- 172.21.15.62:0/3320583374 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b27a0 0x7f116c161240 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.718+0000 7f1173000640 1 --2- 172.21.15.62:0/3320583374 >> v2:172.21.15.121:3300/0 conn(0x7f116c0b56f0 0x7f116c160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.718+0000 7f1173000640 1 -- 172.21.15.62:0/3320583374 >> 172.21.15.62:0/3320583374 conn(0x7f116c066d10 msgr2=0x7f116c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.718+0000 7f1173000640 1 -- 172.21.15.62:0/3320583374 shutdown_connections 2024-04-25T09:04:06.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.718+0000 7f1173000640 1 -- 172.21.15.62:0/3320583374 wait complete. 2024-04-25T09:04:06.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.720+0000 7f1173000640 1 Processor -- start 2024-04-25T09:04:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.720+0000 7f1173000640 1 -- start start 2024-04-25T09:04:06.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1173000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f116c0b27a0 0x7f116c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1173000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b56f0 0x7f116c1e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1173000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f116c0b6a90 0x7f116c0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1173000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c0b2d60 con 0x7f116c0b56f0 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1173000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c0cea50 con 0x7f116c0b6a90 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1173000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c0b2ee0 con 0x7f116c0b27a0 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1171200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b56f0 0x7f116c1e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1171200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b56f0 0x7f116c1e74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50616/0 (socket says 172.21.15.62:50616) 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.721+0000 7f1171200640 1 -- 172.21.15.62:0/3803733295 learned_addr learned my addr 172.21.15.62:0/3803733295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1171c00640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f116c0b27a0 0x7f116c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1172600640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f116c0b6a90 0x7f116c0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1171200640 1 -- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f116c0b27a0 msgr2=0x7f116c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1171200640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f116c0b27a0 0x7f116c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1171200640 1 -- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f116c0b6a90 msgr2=0x7f116c0b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1171200640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f116c0b6a90 0x7f116c0b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1171200640 1 -- 172.21.15.62:0/3803733295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c1fb590 con 0x7f116c0b56f0 2024-04-25T09:04:06.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1172600640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f116c0b6a90 0x7f116c0b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1171200640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b56f0 0x7f116c1e74d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f115800b3c0 tx=0x7f115800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1169600640 1 -- 172.21.15.62:0/3803733295 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158011830 con 0x7f116c0b56f0 2024-04-25T09:04:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1169600640 1 -- 172.21.15.62:0/3803733295 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158016070 con 0x7f116c0b56f0 2024-04-25T09:04:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c0ca890 con 0x7f116c0b56f0 2024-04-25T09:04:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.722+0000 7f1169600640 1 -- 172.21.15.62:0/3803733295 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158011bf0 con 0x7f116c0b56f0 2024-04-25T09:04:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.723+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c1fbae0 con 0x7f116c0b56f0 2024-04-25T09:04:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.724+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c04ef20 con 0x7f116c0b56f0 2024-04-25T09:04:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.725+0000 7f1169600640 1 -- 172.21.15.62:0/3803733295 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f115801a410 con 0x7f116c0b56f0 2024-04-25T09:04:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.725+0000 7f1169600640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f114c06b500 0x7f114c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.726+0000 7f1169600640 1 -- 172.21.15.62:0/3803733295 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f11580898f0 con 0x7f116c0b56f0 2024-04-25T09:04:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.726+0000 7f1171c00640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f114c06b500 0x7f114c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.727+0000 7f1171c00640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f114c06b500 0x7f114c06d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f116000a990 tx=0x7f1160008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.730+0000 7f1169600640 1 -- 172.21.15.62:0/3803733295 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f115801f020 con 0x7f116c0b56f0 2024-04-25T09:04:06.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.934+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f116c1e68d0 con 0x7f116c0b56f0 2024-04-25T09:04:06.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.936+0000 7f1169600640 1 -- 172.21.15.62:0/3803733295 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f11580586b0 con 0x7f116c0b56f0 2024-04-25T09:04:06.926 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:06.926 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:59.724537+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:06.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f114c06b500 msgr2=0x7f114c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f114c06b500 0x7f114c06d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f116000a990 tx=0x7f1160008040 comp rx=0 tx=0).stop 2024-04-25T09:04:06.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b56f0 msgr2=0x7f116c1e74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b56f0 0x7f116c1e74d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f115800b3c0 tx=0x7f115800b880 comp rx=0 tx=0).stop 2024-04-25T09:04:06.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 shutdown_connections 2024-04-25T09:04:06.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f114c06b500 0x7f114c06d9c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f116c0b56f0 0x7f116c1e74d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f116c0b6a90 0x7f116c0b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 --2- 172.21.15.62:0/3803733295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f116c0b27a0 0x7f116c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 >> 172.21.15.62:0/3803733295 conn(0x7f116c066d10 msgr2=0x7f116c0b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 shutdown_connections 2024-04-25T09:04:06.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:06.939+0000 7f1173000640 1 -- 172.21.15.62:0/3803733295 wait complete. 2024-04-25T09:04:06.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:06.950 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.175+0000 7f2607200640 1 Processor -- start 2024-04-25T09:04:07.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.175+0000 7f2607200640 1 -- start start 2024-04-25T09:04:07.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.177+0000 7f2607200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f260015dd50 0x7f26000d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.177+0000 7f2607200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000d2fb0 0x7f26000ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.177+0000 7f2607200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26000ce780 con 0x7f26000d3960 2024-04-25T09:04:07.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.177+0000 7f2607200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26000d3440 con 0x7f260015dd50 2024-04-25T09:04:07.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.177+0000 7f2607200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26000ce1c0 con 0x7f26000d2fb0 2024-04-25T09:04:07.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.178+0000 7f2605400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000d2fb0 0x7f26000ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.178+0000 7f2605e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f260015dd50 0x7f26000d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.178+0000 7f2605e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f260015dd50 0x7f26000d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:44224/0 (socket says 172.21.15.62:44224) 2024-04-25T09:04:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.178+0000 7f2606800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f26000d3960 0x7f26000d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:37870/0 (socket says 172.21.15.62:37870) 2024-04-25T09:04:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.178+0000 7f2606800640 1 -- 172.21.15.62:0/163994094 learned_addr learned my addr 172.21.15.62:0/163994094 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.178+0000 7f2605400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000d2fb0 0x7f26000ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:47740/0 (socket says 172.21.15.62:47740) 2024-04-25T09:04:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.179+0000 7f2605400640 1 -- 172.21.15.62:0/163994094 >> v2:172.21.15.121:3300/0 conn(0x7f260015dd50 msgr2=0x7f26000d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.179+0000 7f2605400640 1 --2- 172.21.15.62:0/163994094 >> v2:172.21.15.121:3300/0 conn(0x7f260015dd50 0x7f26000d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.179+0000 7f2605400640 1 -- 172.21.15.62:0/163994094 >> v1:172.21.15.121:6789/0 conn(0x7f26000d3960 legacy=0x7f26000d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.179+0000 7f2605400640 1 -- 172.21.15.62:0/163994094 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26000ce4a0 con 0x7f26000d2fb0 2024-04-25T09:04:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.179+0000 7f2604a00640 1 -- 172.21.15.62:0/163994094 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056323576 0 0) 0x7f26000ce780 con 0x7f26000d3960 2024-04-25T09:04:07.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.179+0000 7f2605e00640 1 --2- 172.21.15.62:0/163994094 >> v2:172.21.15.121:3300/0 conn(0x7f260015dd50 0x7f26000d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.181+0000 7f2605400640 1 --2- 172.21.15.62:0/163994094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000d2fb0 0x7f26000ce9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25f00097a0 tx=0x7f25f002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a4297738acdc57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.182+0000 7f2604a00640 1 -- 172.21.15.62:0/163994094 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0037070 con 0x7f26000d2fb0 2024-04-25T09:04:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.182+0000 7f2604a00640 1 -- 172.21.15.62:0/163994094 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f0032440 con 0x7f26000d2fb0 2024-04-25T09:04:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.182+0000 7f2604a00640 1 -- 172.21.15.62:0/163994094 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0032810 con 0x7f26000d2fb0 2024-04-25T09:04:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.182+0000 7f2607200640 1 -- 172.21.15.62:0/163994094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000d2fb0 msgr2=0x7f26000ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.182+0000 7f2607200640 1 --2- 172.21.15.62:0/163994094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000d2fb0 0x7f26000ce9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25f00097a0 tx=0x7f25f002a180 comp rx=0 tx=0).stop 2024-04-25T09:04:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.183+0000 7f2607200640 1 -- 172.21.15.62:0/163994094 shutdown_connections 2024-04-25T09:04:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.183+0000 7f2607200640 1 --2- 172.21.15.62:0/163994094 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000d2fb0 0x7f26000ce9c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.183+0000 7f2607200640 1 --2- 172.21.15.62:0/163994094 >> v2:172.21.15.121:3300/0 conn(0x7f260015dd50 0x7f26000d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.183+0000 7f2607200640 1 -- 172.21.15.62:0/163994094 >> 172.21.15.62:0/163994094 conn(0x7f260006b000 msgr2=0x7f260006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.183+0000 7f2607200640 1 -- 172.21.15.62:0/163994094 shutdown_connections 2024-04-25T09:04:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.183+0000 7f2607200640 1 -- 172.21.15.62:0/163994094 wait complete. 2024-04-25T09:04:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.185+0000 7f2607200640 1 Processor -- start 2024-04-25T09:04:07.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.185+0000 7f2607200640 1 -- start start 2024-04-25T09:04:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2607200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26000d3960 0x7f260014d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2607200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f260015dd50 0x7f26000ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2607200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000b3690 0x7f26000b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2607200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26000b6b00 con 0x7f260015dd50 2024-04-25T09:04:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2607200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26000cef00 con 0x7f26000d3960 2024-04-25T09:04:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2607200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26000b6c80 con 0x7f26000b3690 2024-04-25T09:04:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2605400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f260015dd50 0x7f26000ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2605400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f260015dd50 0x7f26000ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50628/0 (socket says 172.21.15.62:50628) 2024-04-25T09:04:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.186+0000 7f2605400640 1 -- 172.21.15.62:0/3557259542 learned_addr learned my addr 172.21.15.62:0/3557259542 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.187+0000 7f2605e00640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26000d3960 0x7f260014d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.187+0000 7f2606800640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000b3690 0x7f26000b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.187+0000 7f2605400640 1 -- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000b3690 msgr2=0x7f26000b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.187+0000 7f2605400640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000b3690 0x7f26000b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.187+0000 7f2605400640 1 -- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26000d3960 msgr2=0x7f260014d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.187+0000 7f2605400640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26000d3960 0x7f260014d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.187+0000 7f2605400640 1 -- 172.21.15.62:0/3557259542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26000c7790 con 0x7f260015dd50 2024-04-25T09:04:07.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.187+0000 7f2605400640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f260015dd50 0x7f26000ca050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f25f0035040 tx=0x7f25f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.188+0000 7f25fa000640 1 -- 172.21.15.62:0/3557259542 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f0037040 con 0x7f260015dd50 2024-04-25T09:04:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.188+0000 7f25fa000640 1 -- 172.21.15.62:0/3557259542 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f0030500 con 0x7f260015dd50 2024-04-25T09:04:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.188+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26000c7a70 con 0x7f260015dd50 2024-04-25T09:04:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.188+0000 7f25fa000640 1 -- 172.21.15.62:0/3557259542 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f00308d0 con 0x7f260015dd50 2024-04-25T09:04:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.188+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26000c8030 con 0x7f260015dd50 2024-04-25T09:04:07.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.189+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260004ef20 con 0x7f260015dd50 2024-04-25T09:04:07.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.190+0000 7f25fa000640 1 -- 172.21.15.62:0/3557259542 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25f0044020 con 0x7f260015dd50 2024-04-25T09:04:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.192+0000 7f25fa000640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f25e406b500 0x7f25e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.192+0000 7f25fa000640 1 -- 172.21.15.62:0/3557259542 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f25f00aa4f0 con 0x7f260015dd50 2024-04-25T09:04:07.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.192+0000 7f2605e00640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f25e406b500 0x7f25e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.193+0000 7f2605e00640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f25e406b500 0x7f25e406d9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009a30 tx=0x7f25f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.197+0000 7f25fa000640 1 -- 172.21.15.62:0/3557259542 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25f00792b0 con 0x7f260015dd50 2024-04-25T09:04:07.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.398+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f260015ce00 con 0x7f260015dd50 2024-04-25T09:04:07.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.399+0000 7f25fa000640 1 -- 172.21.15.62:0/3557259542 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25f0078bb0 con 0x7f260015dd50 2024-04-25T09:04:07.388 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:04:07.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f25e406b500 msgr2=0x7f25e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f25e406b500 0x7f25e406d9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009a30 tx=0x7f25f4008040 comp rx=0 tx=0).stop 2024-04-25T09:04:07.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f260015dd50 msgr2=0x7f26000ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f260015dd50 0x7f26000ca050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f25f0035040 tx=0x7f25f00046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 shutdown_connections 2024-04-25T09:04:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f25e406b500 0x7f25e406d9c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f260015dd50 0x7f26000ca050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26000b3690 0x7f26000b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 --2- 172.21.15.62:0/3557259542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26000d3960 0x7f260014d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 >> 172.21.15.62:0/3557259542 conn(0x7f260006b000 msgr2=0x7f2600068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.402+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 shutdown_connections 2024-04-25T09:04:07.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.403+0000 7f2607200640 1 -- 172.21.15.62:0/3557259542 wait complete. 2024-04-25T09:04:07.411 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:07.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.640+0000 7f1900a00640 1 Processor -- start 2024-04-25T09:04:07.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.640+0000 7f1900a00640 1 -- start start 2024-04-25T09:04:07.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.642+0000 7f1900a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18fc0ceb50 0x7f18fc06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.642+0000 7f1900a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc15e7c0 0x7f18fc0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.642+0000 7f1900a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc15ee30 con 0x7f18fc06d800 2024-04-25T09:04:07.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.642+0000 7f1900a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18fc15ec00 con 0x7f18fc0ceb50 2024-04-25T09:04:07.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.642+0000 7f1900a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc1618b0 con 0x7f18fc15e7c0 2024-04-25T09:04:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.642+0000 7f18fa800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18fc0ceb50 0x7f18fc06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.643+0000 7f18fa800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18fc0ceb50 0x7f18fc06d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50642/0 (socket says 172.21.15.62:50642) 2024-04-25T09:04:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.643+0000 7f18fa800640 1 -- 172.21.15.62:0/2716271010 learned_addr learned my addr 172.21.15.62:0/2716271010 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.643+0000 7f18fb200640 1 --2- 172.21.15.62:0/2716271010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc15e7c0 0x7f18fc0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.643+0000 7f18fa800640 1 -- 172.21.15.62:0/2716271010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc15e7c0 msgr2=0x7f18fc0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.643+0000 7f18fa800640 1 --2- 172.21.15.62:0/2716271010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc15e7c0 0x7f18fc0b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.643+0000 7f18fa800640 1 -- 172.21.15.62:0/2716271010 >> v1:172.21.15.121:6789/0 conn(0x7f18fc06d800 legacy=0x7f18fc15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.643+0000 7f18fa800640 1 -- 172.21.15.62:0/2716271010 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc161b90 con 0x7f18fc0ceb50 2024-04-25T09:04:07.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.645+0000 7f18fa800640 1 --2- 172.21.15.62:0/2716271010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc0ceb50 0x7f18fc06d2c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009870 tx=0x7f18ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fecb89c5d191ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.645+0000 7f18f9400640 1 -- 172.21.15.62:0/2716271010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec037070 con 0x7f18fc0ceb50 2024-04-25T09:04:07.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.645+0000 7f18f9400640 1 -- 172.21.15.62:0/2716271010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec032440 con 0x7f18fc0ceb50 2024-04-25T09:04:07.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.645+0000 7f18f9400640 1 -- 172.21.15.62:0/2716271010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ec032810 con 0x7f18fc0ceb50 2024-04-25T09:04:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.646+0000 7f1900a00640 1 -- 172.21.15.62:0/2716271010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc0ceb50 msgr2=0x7f18fc06d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.646+0000 7f1900a00640 1 --2- 172.21.15.62:0/2716271010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc0ceb50 0x7f18fc06d2c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009870 tx=0x7f18ec02a180 comp rx=0 tx=0).stop 2024-04-25T09:04:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.646+0000 7f1900a00640 1 -- 172.21.15.62:0/2716271010 shutdown_connections 2024-04-25T09:04:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.646+0000 7f1900a00640 1 --2- 172.21.15.62:0/2716271010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc15e7c0 0x7f18fc0b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.646+0000 7f1900a00640 1 --2- 172.21.15.62:0/2716271010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc0ceb50 0x7f18fc06d2c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.646+0000 7f1900a00640 1 -- 172.21.15.62:0/2716271010 >> 172.21.15.62:0/2716271010 conn(0x7f18fc066f20 msgr2=0x7f18fc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.646+0000 7f1900a00640 1 -- 172.21.15.62:0/2716271010 shutdown_connections 2024-04-25T09:04:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.646+0000 7f1900a00640 1 -- 172.21.15.62:0/2716271010 wait complete. 2024-04-25T09:04:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.648+0000 7f1900a00640 1 Processor -- start 2024-04-25T09:04:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.649+0000 7f1900a00640 1 -- start start 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f1900a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc06d800 0x7f18fc159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f1900a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc0ceb50 0x7f18fc15c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f1900a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc15e7c0 0x7f18fc0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f1900a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc162510 con 0x7f18fc15e7c0 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f1900a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc162390 con 0x7f18fc06d800 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f1900a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc0b4a90 con 0x7f18fc0ceb50 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f18fb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc15e7c0 0x7f18fc0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f18fb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc15e7c0 0x7f18fc0b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50654/0 (socket says 172.21.15.62:50654) 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f18fb200640 1 -- 172.21.15.62:0/711014690 learned_addr learned my addr 172.21.15.62:0/711014690 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.650+0000 7f18fa800640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc06d800 0x7f18fc159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18f9e00640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc0ceb50 0x7f18fc15c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18fb200640 1 -- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc0ceb50 msgr2=0x7f18fc15c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18fb200640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc0ceb50 0x7f18fc15c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18fb200640 1 -- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc06d800 msgr2=0x7f18fc159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18fb200640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc06d800 0x7f18fc159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18fb200640 1 -- 172.21.15.62:0/711014690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc15ce70 con 0x7f18fc15e7c0 2024-04-25T09:04:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18fb200640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc15e7c0 0x7f18fc0b0330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f18f400cc30 tx=0x7f18f400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18f2a00640 1 -- 172.21.15.62:0/711014690 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400ce40 con 0x7f18fc15e7c0 2024-04-25T09:04:07.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18f2a00640 1 -- 172.21.15.62:0/711014690 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f4003990 con 0x7f18fc15e7c0 2024-04-25T09:04:07.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.651+0000 7f18f2a00640 1 -- 172.21.15.62:0/711014690 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4003d40 con 0x7f18fc15e7c0 2024-04-25T09:04:07.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.652+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc15d150 con 0x7f18fc15e7c0 2024-04-25T09:04:07.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.652+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc1f4000 con 0x7f18fc15e7c0 2024-04-25T09:04:07.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.653+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc06abe0 con 0x7f18fc15e7c0 2024-04-25T09:04:07.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.654+0000 7f18f2a00640 1 -- 172.21.15.62:0/711014690 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18f4003ee0 con 0x7f18fc15e7c0 2024-04-25T09:04:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.655+0000 7f18f2a00640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f18dc06b4d0 0x7f18dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.655+0000 7f18f2a00640 1 -- 172.21.15.62:0/711014690 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f18f408a770 con 0x7f18fc15e7c0 2024-04-25T09:04:07.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.656+0000 7f18fa800640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f18dc06b4d0 0x7f18dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.657+0000 7f18fa800640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f18dc06b4d0 0x7f18dc06d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f18ec034000 tx=0x7f18ec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.660+0000 7f18f2a00640 1 -- 172.21.15.62:0/711014690 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18f4014030 con 0x7f18fc15e7c0 2024-04-25T09:04:07.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.867+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18fc04ef20 con 0x7f18fc15e7c0 2024-04-25T09:04:07.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.867+0000 7f18f2a00640 1 -- 172.21.15.62:0/711014690 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18f4059530 con 0x7f18fc15e7c0 2024-04-25T09:04:07.856 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:04:07.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f18dc06b4d0 msgr2=0x7f18dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f18dc06b4d0 0x7f18dc06d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f18ec034000 tx=0x7f18ec02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:07.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc15e7c0 msgr2=0x7f18fc0b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc15e7c0 0x7f18fc0b0330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f18f400cc30 tx=0x7f18f400a5c0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 shutdown_connections 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f18dc06b4d0 0x7f18dc06d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18fc15e7c0 0x7f18fc0b0330 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18fc0ceb50 0x7f18fc15c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 --2- 172.21.15.62:0/711014690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc06d800 0x7f18fc159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.870+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 >> 172.21.15.62:0/711014690 conn(0x7f18fc066f20 msgr2=0x7f18fc1601f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.871+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 shutdown_connections 2024-04-25T09:04:07.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:07.871+0000 7f1900a00640 1 -- 172.21.15.62:0/711014690 wait complete. 2024-04-25T09:04:07.881 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:08.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.113+0000 7f8663e00640 1 Processor -- start 2024-04-25T09:04:08.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.114+0000 7f8663e00640 1 -- start start 2024-04-25T09:04:08.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.115+0000 7f8663e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86640ceb50 0x7f866406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.115+0000 7f8663e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866415ee30 con 0x7f866406d800 2024-04-25T09:04:08.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.115+0000 7f8663e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86641618b0 con 0x7f866415e7c0 2024-04-25T09:04:08.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.115+0000 7f8663e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8664161b90 con 0x7f86640ceb50 2024-04-25T09:04:08.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.116+0000 7f8662000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f866406d800 0x7f866415e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51736/0 (socket says 172.21.15.62:51736) 2024-04-25T09:04:08.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.116+0000 7f8662000640 1 -- 172.21.15.62:0/2890612515 learned_addr learned my addr 172.21.15.62:0/2890612515 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:08.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.116+0000 7f8662a00640 1 --2- 172.21.15.62:0/2890612515 >> v2:172.21.15.121:3300/0 conn(0x7f86640ceb50 0x7f866406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.116+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249190967 0 0) 0x7f866415ee30 con 0x7f866406d800 2024-04-25T09:04:08.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8648003520 con 0x7f866406d800 2024-04-25T09:04:08.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 754737729 0 0) 0x7f8648003520 con 0x7f866406d800 2024-04-25T09:04:08.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 >> v2:172.21.15.121:3300/0 conn(0x7f86640ceb50 msgr2=0x7f866406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 --2- 172.21.15.62:0/2890612515 >> v2:172.21.15.121:3300/0 conn(0x7f86640ceb50 0x7f866406d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 >> v1:172.21.15.121:6789/0 conn(0x7f866415e7c0 legacy=0x7f86640b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640cef30 con 0x7f866406d800 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f8650001670 con 0x7f866406d800 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 >> v1:172.21.15.62:6789/0 conn(0x7f866406d800 legacy=0x7f866415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8648003a00 0x7f8648003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8648004800 0x7f8648004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8648005210 0x7f864800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8664161b90 con 0x7f8648003a00 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864800c160 con 0x7f8648004800 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864800c360 con 0x7f8648005210 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.117+0000 7f8662a00640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8648003a00 0x7f8648003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.118+0000 7f8662a00640 1 -- 172.21.15.62:0/2890612515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8648005210 msgr2=0x7f864800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.118+0000 7f8662a00640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8648005210 0x7f864800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.118+0000 7f8662a00640 1 -- 172.21.15.62:0/2890612515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8648004800 msgr2=0x7f8648004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.118+0000 7f8662a00640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8648004800 0x7f8648004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.118+0000 7f8662a00640 1 -- 172.21.15.62:0/2890612515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8648003520 con 0x7f8648003a00 2024-04-25T09:04:08.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.118+0000 7f8663400640 1 -- 172.21.15.62:0/2890612515 reap_dead start 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.119+0000 7f8662a00640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8648003a00 0x7f8648003e20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8654009a90 tx=0x7f8648005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fa305e6915b67c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.119+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654036070 con 0x7f8648003a00 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.119+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654031400 con 0x7f8648003a00 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.119+0000 7f8661600640 1 -- 172.21.15.62:0/2890612515 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654009690 con 0x7f8648003a00 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.120+0000 7f8663e00640 1 -- 172.21.15.62:0/2890612515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8648003a00 msgr2=0x7f8648003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.120+0000 7f8663e00640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8648003a00 0x7f8648003e20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8654009a90 tx=0x7f8648005830 comp rx=0 tx=0).stop 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.120+0000 7f8663e00640 1 -- 172.21.15.62:0/2890612515 shutdown_connections 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.120+0000 7f8663e00640 1 --2- 172.21.15.62:0/2890612515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8648003a00 0x7f8648003e20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.120+0000 7f8663e00640 1 -- 172.21.15.62:0/2890612515 >> 172.21.15.62:0/2890612515 conn(0x7f8664066f20 msgr2=0x7f86640b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.120+0000 7f8663e00640 1 -- 172.21.15.62:0/2890612515 shutdown_connections 2024-04-25T09:04:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.120+0000 7f8663e00640 1 -- 172.21.15.62:0/2890612515 wait complete. 2024-04-25T09:04:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.122+0000 7f8663e00640 1 Processor -- start 2024-04-25T09:04:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.122+0000 7f8663e00640 1 -- start start 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.123+0000 7f8663e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8648003a00 0x7f86640b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.123+0000 7f8663e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664157de0 0x7f866413b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.123+0000 7f8663e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8664158220 0x7f866413bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.123+0000 7f8663e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866406d2c0 con 0x7f8664157de0 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.123+0000 7f8663e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86641625a0 con 0x7f8664158220 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.123+0000 7f8663e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866406d440 con 0x7f8648003a00 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664157de0 0x7f866413b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664157de0 0x7f866413b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50678/0 (socket says 172.21.15.62:50678) 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662000640 1 -- 172.21.15.62:0/3158420932 learned_addr learned my addr 172.21.15.62:0/3158420932 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:08.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8663400640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8664158220 0x7f866413bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662a00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8648003a00 0x7f86640b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662000640 1 -- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8648003a00 msgr2=0x7f86640b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662000640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8648003a00 0x7f86640b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662000640 1 -- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8664158220 msgr2=0x7f866413bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662000640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8664158220 0x7f866413bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.124+0000 7f8662000640 1 -- 172.21.15.62:0/3158420932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8664140e60 con 0x7f8664157de0 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.125+0000 7f8662000640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664157de0 0x7f866413b770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f865000c790 tx=0x7f865000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.125+0000 7f865b400640 1 -- 172.21.15.62:0/3158420932 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650018070 con 0x7f8664157de0 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.125+0000 7f865b400640 1 -- 172.21.15.62:0/3158420932 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86500072d0 con 0x7f8664157de0 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.125+0000 7f865b400640 1 -- 172.21.15.62:0/3158420932 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86500047e0 con 0x7f8664157de0 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.125+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8664141140 con 0x7f8664157de0 2024-04-25T09:04:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.125+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866413e260 con 0x7f8664157de0 2024-04-25T09:04:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.126+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866406abe0 con 0x7f8664157de0 2024-04-25T09:04:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.127+0000 7f865b400640 1 -- 172.21.15.62:0/3158420932 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f865001b070 con 0x7f8664157de0 2024-04-25T09:04:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.128+0000 7f865b400640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f864406b4d0 0x7f864406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.128+0000 7f865b400640 1 -- 172.21.15.62:0/3158420932 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8650089830 con 0x7f8664157de0 2024-04-25T09:04:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.129+0000 7f8662a00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f864406b4d0 0x7f864406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.130+0000 7f8662a00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f864406b4d0 0x7f864406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8654033000 tx=0x7f8654008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.132+0000 7f865b400640 1 -- 172.21.15.62:0/3158420932 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8650058560 con 0x7f8664157de0 2024-04-25T09:04:08.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.330+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f866404ef20 con 0x7f8664157de0 2024-04-25T09:04:08.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.331+0000 7f865b400640 1 -- 172.21.15.62:0/3158420932 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8650057e60 con 0x7f8664157de0 2024-04-25T09:04:08.320 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.333+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f864406b4d0 msgr2=0x7f864406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.333+0000 7f8663e00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f864406b4d0 0x7f864406d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8654033000 tx=0x7f8654008030 comp rx=0 tx=0).stop 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.333+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664157de0 msgr2=0x7f866413b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.333+0000 7f8663e00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664157de0 0x7f866413b770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f865000c790 tx=0x7f865000cc50 comp rx=0 tx=0).stop 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.334+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 shutdown_connections 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.334+0000 7f8663e00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f864406b4d0 0x7f864406d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.334+0000 7f8663e00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8664157de0 0x7f866413b770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.334+0000 7f8663e00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8664158220 0x7f866413bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.334+0000 7f8663e00640 1 --2- 172.21.15.62:0/3158420932 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8648003a00 0x7f86640b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.334+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 >> 172.21.15.62:0/3158420932 conn(0x7f8664066f20 msgr2=0x7f86641463b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.334+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 shutdown_connections 2024-04-25T09:04:08.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.334+0000 7f8663e00640 1 -- 172.21.15.62:0/3158420932 wait complete. 2024-04-25T09:04:08.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:08.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:08.489 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:08.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.567+0000 7f6e7e400640 1 Processor -- start 2024-04-25T09:04:08.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.568+0000 7f6e7e400640 1 -- start start 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.569+0000 7f6e7e400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e7815f110 0x7f6e78070280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.569+0000 7f6e7e400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e7806e540 0x7f6e7806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.569+0000 7f6e7e400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7815f4f0 con 0x7f6e780cc2b0 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.569+0000 7f6e7e400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e780afe20 con 0x7f6e7806e540 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.569+0000 7f6e7e400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7806f310 con 0x7f6e7815f110 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.570+0000 7f6e77e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6e780cc2b0 0x7f6e7806f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:37898/0 (socket says 172.21.15.62:37898) 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.570+0000 7f6e77e00640 1 -- 172.21.15.62:0/2007510029 learned_addr learned my addr 172.21.15.62:0/2007510029 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.570+0000 7f6e7da00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e7806e540 0x7f6e7806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.570+0000 7f6e7d000640 1 --2- 172.21.15.62:0/2007510029 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e7815f110 0x7f6e78070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.570+0000 7f6e77400640 1 -- 172.21.15.62:0/2007510029 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262683590 0 0) 0x7f6e7815f4f0 con 0x7f6e780cc2b0 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.570+0000 7f6e77400640 1 -- 172.21.15.62:0/2007510029 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e60003520 con 0x7f6e780cc2b0 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.571+0000 7f6e7da00640 1 -- 172.21.15.62:0/2007510029 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e7815f110 msgr2=0x7f6e78070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.571+0000 7f6e7da00640 1 --2- 172.21.15.62:0/2007510029 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e7815f110 0x7f6e78070280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.571+0000 7f6e7da00640 1 -- 172.21.15.62:0/2007510029 >> v1:172.21.15.121:6789/0 conn(0x7f6e780cc2b0 legacy=0x7f6e7806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.571+0000 7f6e77400640 1 -- 172.21.15.62:0/2007510029 <== mon.1 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3508640973 0 0) 0x7f6e60003520 con 0x7f6e780cc2b0 2024-04-25T09:04:08.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.571+0000 7f6e7da00640 1 -- 172.21.15.62:0/2007510029 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780af7d0 con 0x7f6e7806e540 2024-04-25T09:04:08.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.573+0000 7f6e7da00640 1 --2- 172.21.15.62:0/2007510029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 0x7f6e7806e9d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009d90 tx=0x7f6e7002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=150e1237c9644370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.573+0000 7f6e77400640 1 -- 172.21.15.62:0/2007510029 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64001670 con 0x7f6e7806e540 2024-04-25T09:04:08.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.573+0000 7f6e77400640 1 -- 172.21.15.62:0/2007510029 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70007540 con 0x7f6e7806e540 2024-04-25T09:04:08.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.573+0000 7f6e77400640 1 -- 172.21.15.62:0/2007510029 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70007910 con 0x7f6e7806e540 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.574+0000 7f6e7e400640 1 -- 172.21.15.62:0/2007510029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 msgr2=0x7f6e7806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.574+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2007510029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 0x7f6e7806e9d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009d90 tx=0x7f6e7002a5f0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.574+0000 7f6e7e400640 1 -- 172.21.15.62:0/2007510029 shutdown_connections 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.574+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2007510029 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 0x7f6e7806e9d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.574+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2007510029 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e7815f110 0x7f6e78070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.574+0000 7f6e7e400640 1 -- 172.21.15.62:0/2007510029 >> 172.21.15.62:0/2007510029 conn(0x7f6e780b33d0 msgr2=0x7f6e780b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.574+0000 7f6e7e400640 1 -- 172.21.15.62:0/2007510029 shutdown_connections 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.574+0000 7f6e7e400640 1 -- 172.21.15.62:0/2007510029 wait complete. 2024-04-25T09:04:08.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.576+0000 7f6e7e400640 1 Processor -- start 2024-04-25T09:04:08.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.576+0000 7f6e7e400640 1 -- start start 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.577+0000 7f6e7e400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 0x7f6e780ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7e400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e780cc2b0 0x7f6e781f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e7815f110 0x7f6e781f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7806f090 con 0x7f6e7815f110 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7e400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7806ef10 con 0x7f6e7806e540 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7e400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780ae6b0 con 0x7f6e780cc2b0 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e7815f110 0x7f6e781f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e7815f110 0x7f6e781f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50682/0 (socket says 172.21.15.62:50682) 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7da00640 1 -- 172.21.15.62:0/2159272340 learned_addr learned my addr 172.21.15.62:0/2159272340 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7d000640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 0x7f6e780ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e77e00640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e780cc2b0 0x7f6e781f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7da00640 1 -- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e780cc2b0 msgr2=0x7f6e781f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7da00640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e780cc2b0 0x7f6e781f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7da00640 1 -- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 msgr2=0x7f6e780ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7da00640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 0x7f6e780ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.578+0000 7f6e7da00640 1 -- 172.21.15.62:0/2159272340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780cacb0 con 0x7f6e7815f110 2024-04-25T09:04:08.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.579+0000 7f6e7da00640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e7815f110 0x7f6e781f3af0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009d90 tx=0x7f6e70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.579+0000 7f6e74c00640 1 -- 172.21.15.62:0/2159272340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70040070 con 0x7f6e7815f110 2024-04-25T09:04:08.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.579+0000 7f6e74c00640 1 -- 172.21.15.62:0/2159272340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70031b40 con 0x7f6e7815f110 2024-04-25T09:04:08.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.579+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e780caf90 con 0x7f6e7815f110 2024-04-25T09:04:08.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.579+0000 7f6e74c00640 1 -- 172.21.15.62:0/2159272340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70007e90 con 0x7f6e7815f110 2024-04-25T09:04:08.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.579+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e780b5ef0 con 0x7f6e7815f110 2024-04-25T09:04:08.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.580+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e780cb160 con 0x7f6e7815f110 2024-04-25T09:04:08.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.581+0000 7f6e74c00640 1 -- 172.21.15.62:0/2159272340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e70039040 con 0x7f6e7815f110 2024-04-25T09:04:08.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.582+0000 7f6e74c00640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e5806b4d0 0x7f6e5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.582+0000 7f6e74c00640 1 -- 172.21.15.62:0/2159272340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6e700ab650 con 0x7f6e7815f110 2024-04-25T09:04:08.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.583+0000 7f6e7d000640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e5806b4d0 0x7f6e5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.584+0000 7f6e7d000640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e5806b4d0 0x7f6e5806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009a30 tx=0x7f6e6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.586+0000 7f6e74c00640 1 -- 172.21.15.62:0/2159272340 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e7004b340 con 0x7f6e7815f110 2024-04-25T09:04:08.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.806+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6e780b60f0 con 0x7f6e7815f110 2024-04-25T09:04:08.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.807+0000 7f6e74c00640 1 -- 172.21.15.62:0/2159272340 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6e7007a4a0 con 0x7f6e7815f110 2024-04-25T09:04:08.796 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:04:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e5806b4d0 msgr2=0x7f6e5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e5806b4d0 0x7f6e5806d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009a30 tx=0x7f6e6c008040 comp rx=0 tx=0).stop 2024-04-25T09:04:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e7815f110 msgr2=0x7f6e781f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e7815f110 0x7f6e781f3af0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009d90 tx=0x7f6e70004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 shutdown_connections 2024-04-25T09:04:08.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e5806b4d0 0x7f6e5806d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e7815f110 0x7f6e781f3af0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e780cc2b0 0x7f6e781f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 --2- 172.21.15.62:0/2159272340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e7806e540 0x7f6e780ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 >> 172.21.15.62:0/2159272340 conn(0x7f6e780b33d0 msgr2=0x7f6e7815c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 shutdown_connections 2024-04-25T09:04:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:08.810+0000 7f6e7e400640 1 -- 172.21.15.62:0/2159272340 wait complete. 2024-04-25T09:04:08.820 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T09:04:08.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:08.820 DEBUG:teuthology.orchestra.run.smithi121:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T09:04:08.917 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:04:08.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:08.917 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:04:08.961 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:04:08.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:09.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.194+0000 7f4f96000640 1 Processor -- start 2024-04-25T09:04:09.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.194+0000 7f4f96000640 1 -- start start 2024-04-25T09:04:09.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.195+0000 7f4f96000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f90144c10 0x7f4f9015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.195+0000 7f4f96000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f9006d900 0x7f4f900b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f96000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9015faa0 con 0x7f4f900cee40 2024-04-25T09:04:09.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f96000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f90144ff0 con 0x7f4f9006d900 2024-04-25T09:04:09.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f96000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90160340 con 0x7f4f90144c10 2024-04-25T09:04:09.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f95600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f9006d900 0x7f4f900b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f95600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f9006d900 0x7f4f900b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50686/0 (socket says 172.21.15.62:50686) 2024-04-25T09:04:09.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f95600640 1 -- 172.21.15.62:0/1379096733 learned_addr learned my addr 172.21.15.62:0/1379096733 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:09.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f94c00640 1 --2- 172.21.15.62:0/1379096733 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f90144c10 0x7f4f9015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f95600640 1 -- 172.21.15.62:0/1379096733 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f90144c10 msgr2=0x7f4f9015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f95600640 1 --2- 172.21.15.62:0/1379096733 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f90144c10 0x7f4f9015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f95600640 1 -- 172.21.15.62:0/1379096733 >> v1:172.21.15.121:6789/0 conn(0x7f4f900cee40 legacy=0x7f4f9006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.196+0000 7f4f95600640 1 -- 172.21.15.62:0/1379096733 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9015fd80 con 0x7f4f9006d900 2024-04-25T09:04:09.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.197+0000 7f4f95600640 1 --2- 172.21.15.62:0/1379096733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f900b4650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009e80 tx=0x7f4f8802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cf28ace5b3548a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f8f400640 1 -- 172.21.15.62:0/1379096733 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88036070 con 0x7f4f9006d900 2024-04-25T09:04:09.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f8f400640 1 -- 172.21.15.62:0/1379096733 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f88031600 con 0x7f4f9006d900 2024-04-25T09:04:09.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f8f400640 1 -- 172.21.15.62:0/1379096733 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f880319d0 con 0x7f4f9006d900 2024-04-25T09:04:09.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f96000640 1 -- 172.21.15.62:0/1379096733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 msgr2=0x7f4f900b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f96000640 1 --2- 172.21.15.62:0/1379096733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f900b4650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009e80 tx=0x7f4f8802a660 comp rx=0 tx=0).stop 2024-04-25T09:04:09.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f96000640 1 -- 172.21.15.62:0/1379096733 shutdown_connections 2024-04-25T09:04:09.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f96000640 1 --2- 172.21.15.62:0/1379096733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f900b4650 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f96000640 1 --2- 172.21.15.62:0/1379096733 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f90144c10 0x7f4f9015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.198+0000 7f4f96000640 1 -- 172.21.15.62:0/1379096733 >> 172.21.15.62:0/1379096733 conn(0x7f4f900b14d0 msgr2=0x7f4f900b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.199+0000 7f4f96000640 1 -- 172.21.15.62:0/1379096733 shutdown_connections 2024-04-25T09:04:09.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.199+0000 7f4f96000640 1 -- 172.21.15.62:0/1379096733 wait complete. 2024-04-25T09:04:09.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.200+0000 7f4f96000640 1 Processor -- start 2024-04-25T09:04:09.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.200+0000 7f4f96000640 1 -- start start 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.201+0000 7f4f96000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f90131790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.201+0000 7f4f96000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f900cee40 0x7f4f901f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.201+0000 7f4f96000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90144c10 0x7f4f900b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.201+0000 7f4f96000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900cf2a0 con 0x7f4f9006d900 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.201+0000 7f4f96000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900b41c0 con 0x7f4f90144c10 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.201+0000 7f4f96000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f900b4b90 con 0x7f4f900cee40 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f94c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f90131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f94c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f90131790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50688/0 (socket says 172.21.15.62:50688) 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f94c00640 1 -- 172.21.15.62:0/1547325308 learned_addr learned my addr 172.21.15.62:0/1547325308 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:09.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f95600640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90144c10 0x7f4f900b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f8fe00640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f900cee40 0x7f4f901f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f94c00640 1 -- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f900cee40 msgr2=0x7f4f901f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f94c00640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f900cee40 0x7f4f901f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f94c00640 1 -- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90144c10 msgr2=0x7f4f900b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f94c00640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90144c10 0x7f4f900b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f94c00640 1 -- 172.21.15.62:0/1547325308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f90131cd0 con 0x7f4f9006d900 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.202+0000 7f4f95600640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90144c10 0x7f4f900b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.203+0000 7f4f94c00640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f90131790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c7f0 tx=0x7f4f8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.203+0000 7f4f8cc00640 1 -- 172.21.15.62:0/1547325308 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84015070 con 0x7f4f9006d900 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.203+0000 7f4f8cc00640 1 -- 172.21.15.62:0/1547325308 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f84004220 con 0x7f4f9006d900 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.203+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f900ccf00 con 0x7f4f9006d900 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.203+0000 7f4f8cc00640 1 -- 172.21.15.62:0/1547325308 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f84007620 con 0x7f4f9006d900 2024-04-25T09:04:09.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.203+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f900cd4c0 con 0x7f4f9006d900 2024-04-25T09:04:09.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.204+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9004ef20 con 0x7f4f9006d900 2024-04-25T09:04:09.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.205+0000 7f4f8cc00640 1 -- 172.21.15.62:0/1547325308 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f840079d0 con 0x7f4f9006d900 2024-04-25T09:04:09.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.206+0000 7f4f8cc00640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f7006b4d0 0x7f4f7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.206+0000 7f4f8cc00640 1 -- 172.21.15.62:0/1547325308 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4f8408a550 con 0x7f4f9006d900 2024-04-25T09:04:09.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.207+0000 7f4f8fe00640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f7006b4d0 0x7f4f7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.208+0000 7f4f8fe00640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f7006b4d0 0x7f4f7006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f900701d0 tx=0x7f4f7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.211+0000 7f4f8cc00640 1 -- 172.21.15.62:0/1547325308 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f90131cd0 con 0x7f4f9006d900 2024-04-25T09:04:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.445+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f90140850 con 0x7f4f9006d900 2024-04-25T09:04:09.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.447+0000 7f4f8cc00640 1 -- 172.21.15.62:0/1547325308 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f4f840593b0 con 0x7f4f9006d900 2024-04-25T09:04:09.436 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:09.436 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:16.609043+0000","modified":"2024-04-25T09:03:19.983991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:04:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-25T09:04:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f7006b4d0 msgr2=0x7f4f7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f7006b4d0 0x7f4f7006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f900701d0 tx=0x7f4f7c009040 comp rx=0 tx=0).stop 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 msgr2=0x7f4f90131790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f90131790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c7f0 tx=0x7f4f8400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 shutdown_connections 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f7006b4d0 0x7f4f7006d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90144c10 0x7f4f900b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f900cee40 0x7f4f901f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 --2- 172.21.15.62:0/1547325308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f9006d900 0x7f4f90131790 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 >> 172.21.15.62:0/1547325308 conn(0x7f4f900b14d0 msgr2=0x7f4f9015bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 shutdown_connections 2024-04-25T09:04:09.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.450+0000 7f4f96000640 1 -- 172.21.15.62:0/1547325308 wait complete. 2024-04-25T09:04:09.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:09.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.675+0000 7f2d11a00640 1 Processor -- start 2024-04-25T09:04:09.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.676+0000 7f2d11a00640 1 -- start start 2024-04-25T09:04:09.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.677+0000 7f2d11a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2d0c154ac0 0x7f2d0c154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.677+0000 7f2d11a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2d0c067050 0x7f2d0c071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.677+0000 7f2d11a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c15e720 con 0x7f2d0c0aeb00 2024-04-25T09:04:09.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.677+0000 7f2d11a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c15ea00 con 0x7f2d0c067050 2024-04-25T09:04:09.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.677+0000 7f2d11a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c15ece0 con 0x7f2d0c154ac0 2024-04-25T09:04:09.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d0b400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2d0c0aeb00 0x7f2d0c066940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51744/0 (socket says 172.21.15.62:51744) 2024-04-25T09:04:09.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d11000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2d0c067050 0x7f2d0c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d0b400640 1 -- 172.21.15.62:0/2574637147 learned_addr learned my addr 172.21.15.62:0/2574637147 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:09.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d0be00640 1 --2- 172.21.15.62:0/2574637147 >> v2:172.21.15.121:3300/0 conn(0x7f2d0c154ac0 0x7f2d0c154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d0aa00640 1 -- 172.21.15.62:0/2574637147 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144448881 0 0) 0x7f2d0c15e720 con 0x7f2d0c0aeb00 2024-04-25T09:04:09.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d0aa00640 1 -- 172.21.15.62:0/2574637147 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf0003520 con 0x7f2d0c0aeb00 2024-04-25T09:04:09.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d11000640 1 -- 172.21.15.62:0/2574637147 >> v2:172.21.15.121:3300/0 conn(0x7f2d0c154ac0 msgr2=0x7f2d0c154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d11000640 1 --2- 172.21.15.62:0/2574637147 >> v2:172.21.15.121:3300/0 conn(0x7f2d0c154ac0 0x7f2d0c154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d11000640 1 -- 172.21.15.62:0/2574637147 >> v1:172.21.15.62:6789/0 conn(0x7f2d0c0aeb00 legacy=0x7f2d0c066940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.678+0000 7f2d11000640 1 -- 172.21.15.62:0/2574637147 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c15efc0 con 0x7f2d0c067050 2024-04-25T09:04:09.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.679+0000 7f2d11000640 1 --2- 172.21.15.62:0/2574637147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c067050 0x7f2d0c071470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009d90 tx=0x7f2d0402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bbcf4dd15fdde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.680+0000 7f2d0aa00640 1 -- 172.21.15.62:0/2574637147 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04031600 con 0x7f2d0c067050 2024-04-25T09:04:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.680+0000 7f2d0aa00640 1 -- 172.21.15.62:0/2574637147 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d040317a0 con 0x7f2d0c067050 2024-04-25T09:04:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.680+0000 7f2d0aa00640 1 -- 172.21.15.62:0/2574637147 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04031b70 con 0x7f2d0c067050 2024-04-25T09:04:09.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.680+0000 7f2d11a00640 1 -- 172.21.15.62:0/2574637147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c067050 msgr2=0x7f2d0c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.680+0000 7f2d11a00640 1 --2- 172.21.15.62:0/2574637147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c067050 0x7f2d0c071470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009d90 tx=0x7f2d0402a610 comp rx=0 tx=0).stop 2024-04-25T09:04:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.681+0000 7f2d11a00640 1 -- 172.21.15.62:0/2574637147 shutdown_connections 2024-04-25T09:04:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.681+0000 7f2d11a00640 1 --2- 172.21.15.62:0/2574637147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c067050 0x7f2d0c071470 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.681+0000 7f2d11a00640 1 --2- 172.21.15.62:0/2574637147 >> v2:172.21.15.121:3300/0 conn(0x7f2d0c154ac0 0x7f2d0c154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.681+0000 7f2d11a00640 1 -- 172.21.15.62:0/2574637147 >> 172.21.15.62:0/2574637147 conn(0x7f2d0c06abe0 msgr2=0x7f2d0c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.681+0000 7f2d11a00640 1 -- 172.21.15.62:0/2574637147 shutdown_connections 2024-04-25T09:04:09.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.681+0000 7f2d11a00640 1 -- 172.21.15.62:0/2574637147 wait complete. 2024-04-25T09:04:09.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.682+0000 7f2d11a00640 1 Processor -- start 2024-04-25T09:04:09.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.683+0000 7f2d11a00640 1 -- start start 2024-04-25T09:04:09.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.683+0000 7f2d11a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d0c067050 0x7f2d0c13ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.683+0000 7f2d11a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d0c0aeb00 0x7f2d0c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.683+0000 7f2d11a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c154ac0 0x7f2d0c152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.683+0000 7f2d11a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c071b30 con 0x7f2d0c154ac0 2024-04-25T09:04:09.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.683+0000 7f2d11a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0719b0 con 0x7f2d0c067050 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.683+0000 7f2d11a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c071cb0 con 0x7f2d0c0aeb00 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d11000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c154ac0 0x7f2d0c152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d11000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c154ac0 0x7f2d0c152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50700/0 (socket says 172.21.15.62:50700) 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d11000640 1 -- 172.21.15.62:0/4186166336 learned_addr learned my addr 172.21.15.62:0/4186166336 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d0be00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d0c067050 0x7f2d0c13ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d0b400640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d0c0aeb00 0x7f2d0c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d11000640 1 -- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d0c0aeb00 msgr2=0x7f2d0c1555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d11000640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d0c0aeb00 0x7f2d0c1555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d11000640 1 -- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d0c067050 msgr2=0x7f2d0c13ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d11000640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d0c067050 0x7f2d0c13ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.684+0000 7f2d11000640 1 -- 172.21.15.62:0/4186166336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c157100 con 0x7f2d0c154ac0 2024-04-25T09:04:09.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.685+0000 7f2d11000640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c154ac0 0x7f2d0c152da0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009eb0 tx=0x7f2d04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.685+0000 7f2d03e00640 1 -- 172.21.15.62:0/4186166336 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0403f070 con 0x7f2d0c154ac0 2024-04-25T09:04:09.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.685+0000 7f2d03e00640 1 -- 172.21.15.62:0/4186166336 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04030b60 con 0x7f2d0c154ac0 2024-04-25T09:04:09.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.685+0000 7f2d03e00640 1 -- 172.21.15.62:0/4186166336 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04009450 con 0x7f2d0c154ac0 2024-04-25T09:04:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.685+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1573a0 con 0x7f2d0c154ac0 2024-04-25T09:04:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.685+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c157960 con 0x7f2d0c154ac0 2024-04-25T09:04:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.686+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c04ef20 con 0x7f2d0c154ac0 2024-04-25T09:04:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.687+0000 7f2d03e00640 1 -- 172.21.15.62:0/4186166336 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d0403b410 con 0x7f2d0c154ac0 2024-04-25T09:04:09.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.688+0000 7f2d03e00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2ce806b4d0 0x7f2ce806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.688+0000 7f2d03e00640 1 -- 172.21.15.62:0/4186166336 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2d040ab7b0 con 0x7f2d0c154ac0 2024-04-25T09:04:09.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.689+0000 7f2d0be00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2ce806b4d0 0x7f2ce806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.690+0000 7f2d0be00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2ce806b4d0 0x7f2ce806d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006fd0 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.692+0000 7f2d03e00640 1 -- 172.21.15.62:0/4186166336 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d0407a570 con 0x7f2d0c154ac0 2024-04-25T09:04:09.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.895+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d0c0cf810 con 0x7f2d0c154ac0 2024-04-25T09:04:09.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.898+0000 7f2d03e00640 1 -- 172.21.15.62:0/4186166336 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f2d04079e70 con 0x7f2d0c154ac0 2024-04-25T09:04:09.887 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:09.888 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:59.724537+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.900+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2ce806b4d0 msgr2=0x7f2ce806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.900+0000 7f2d11a00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2ce806b4d0 0x7f2ce806d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006fd0 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.900+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c154ac0 msgr2=0x7f2d0c152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.900+0000 7f2d11a00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c154ac0 0x7f2d0c152da0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009eb0 tx=0x7f2d04004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.901+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 shutdown_connections 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.901+0000 7f2d11a00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2ce806b4d0 0x7f2ce806d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.901+0000 7f2d11a00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d0c154ac0 0x7f2d0c152da0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.901+0000 7f2d11a00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d0c0aeb00 0x7f2d0c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.901+0000 7f2d11a00640 1 --2- 172.21.15.62:0/4186166336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d0c067050 0x7f2d0c13ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.901+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 >> 172.21.15.62:0/4186166336 conn(0x7f2d0c06abe0 msgr2=0x7f2d0c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.901+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 shutdown_connections 2024-04-25T09:04:09.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:09.901+0000 7f2d11a00640 1 -- 172.21.15.62:0/4186166336 wait complete. 2024-04-25T09:04:09.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:15.055465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:15.619995+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:09.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:10.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.120+0000 7f454b200640 1 Processor -- start 2024-04-25T09:04:10.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.121+0000 7f454b200640 1 -- start start 2024-04-25T09:04:10.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.122+0000 7f454b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f454c0b2d80 0x7f454c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.122+0000 7f454b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f454c0b54d0 0x7f454c164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.122+0000 7f454b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454c0b4430 con 0x7f454c0b36a0 2024-04-25T09:04:10.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.122+0000 7f454b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c0b4710 con 0x7f454c0b2d80 2024-04-25T09:04:10.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.122+0000 7f454b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c0b49f0 con 0x7f454c0b54d0 2024-04-25T09:04:10.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f4549e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f454c0b2d80 0x7f454c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f4549e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f454c0b2d80 0x7f454c0b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50708/0 (socket says 172.21.15.62:50708) 2024-04-25T09:04:10.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f4549e00640 1 -- 172.21.15.62:0/3883194087 learned_addr learned my addr 172.21.15.62:0/3883194087 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f4549e00640 1 -- 172.21.15.62:0/3883194087 >> v2:172.21.15.121:3300/0 conn(0x7f454c0b54d0 msgr2=0x7f454c164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f454a800640 1 --2- 172.21.15.62:0/3883194087 >> v2:172.21.15.121:3300/0 conn(0x7f454c0b54d0 0x7f454c164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f4549e00640 1 --2- 172.21.15.62:0/3883194087 >> v2:172.21.15.121:3300/0 conn(0x7f454c0b54d0 0x7f454c164310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f4548a00640 1 -- 172.21.15.62:0/3883194087 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753933208 0 0) 0x7f454c0b4430 con 0x7f454c0b36a0 2024-04-25T09:04:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f4549e00640 1 -- 172.21.15.62:0/3883194087 >> v1:172.21.15.62:6789/0 conn(0x7f454c0b36a0 legacy=0x7f454c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f4549e00640 1 -- 172.21.15.62:0/3883194087 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0b4cd0 con 0x7f454c0b2d80 2024-04-25T09:04:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.123+0000 7f454a800640 1 --2- 172.21.15.62:0/3883194087 >> v2:172.21.15.121:3300/0 conn(0x7f454c0b54d0 0x7f454c164310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:10.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.124+0000 7f4549e00640 1 --2- 172.21.15.62:0/3883194087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b2d80 0x7f454c0b3160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f454c0aea60 tx=0x7f453c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4001bfe365b3e8de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.124+0000 7f4548a00640 1 -- 172.21.15.62:0/3883194087 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c037070 con 0x7f454c0b2d80 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f4548a00640 1 -- 172.21.15.62:0/3883194087 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c032440 con 0x7f454c0b2d80 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f4548a00640 1 -- 172.21.15.62:0/3883194087 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c032810 con 0x7f454c0b2d80 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f454b200640 1 -- 172.21.15.62:0/3883194087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b2d80 msgr2=0x7f454c0b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f454b200640 1 --2- 172.21.15.62:0/3883194087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b2d80 0x7f454c0b3160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f454c0aea60 tx=0x7f453c02a270 comp rx=0 tx=0).stop 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f454b200640 1 -- 172.21.15.62:0/3883194087 shutdown_connections 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f454b200640 1 --2- 172.21.15.62:0/3883194087 >> v2:172.21.15.121:3300/0 conn(0x7f454c0b54d0 0x7f454c164310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f454b200640 1 --2- 172.21.15.62:0/3883194087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b2d80 0x7f454c0b3160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f454b200640 1 -- 172.21.15.62:0/3883194087 >> 172.21.15.62:0/3883194087 conn(0x7f454c066f20 msgr2=0x7f454c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f454b200640 1 -- 172.21.15.62:0/3883194087 shutdown_connections 2024-04-25T09:04:10.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.125+0000 7f454b200640 1 -- 172.21.15.62:0/3883194087 wait complete. 2024-04-25T09:04:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.127+0000 7f454b200640 1 Processor -- start 2024-04-25T09:04:10.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.127+0000 7f454b200640 1 -- start start 2024-04-25T09:04:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f454b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f454c0b2d80 0x7f454c140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f454b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b36a0 0x7f454c1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f454b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f454c0b54d0 0x7f454c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f454b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0ce9c0 con 0x7f454c0b36a0 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f454b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0b40e0 con 0x7f454c0b54d0 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f454b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0ceb40 con 0x7f454c0b2d80 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f4549400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b36a0 0x7f454c1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f4549400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b36a0 0x7f454c1568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50720/0 (socket says 172.21.15.62:50720) 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.128+0000 7f4549400640 1 -- 172.21.15.62:0/2070701731 learned_addr learned my addr 172.21.15.62:0/2070701731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f454a800640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f454c0b54d0 0x7f454c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4549e00640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f454c0b2d80 0x7f454c140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4549400640 1 -- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f454c0b2d80 msgr2=0x7f454c140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4549400640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f454c0b2d80 0x7f454c140ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4549400640 1 -- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f454c0b54d0 msgr2=0x7f454c156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4549400640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f454c0b54d0 0x7f454c156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4549400640 1 -- 172.21.15.62:0/2070701731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c141000 con 0x7f454c0b36a0 2024-04-25T09:04:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4549400640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b36a0 0x7f454c1568a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f453400c520 tx=0x7f453400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4542000640 1 -- 172.21.15.62:0/2070701731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534017070 con 0x7f454c0b36a0 2024-04-25T09:04:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4542000640 1 -- 172.21.15.62:0/2070701731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45340035f0 con 0x7f454c0b36a0 2024-04-25T09:04:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f4542000640 1 -- 172.21.15.62:0/2070701731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45340039e0 con 0x7f454c0b36a0 2024-04-25T09:04:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.129+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c15cb50 con 0x7f454c0b36a0 2024-04-25T09:04:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.130+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c15d110 con 0x7f454c0b36a0 2024-04-25T09:04:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.131+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c04ef20 con 0x7f454c0b36a0 2024-04-25T09:04:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.132+0000 7f4542000640 1 -- 172.21.15.62:0/2070701731 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f453401a070 con 0x7f454c0b36a0 2024-04-25T09:04:10.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.133+0000 7f4542000640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f452806b4d0 0x7f452806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.133+0000 7f4542000640 1 -- 172.21.15.62:0/2070701731 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4534089620 con 0x7f454c0b36a0 2024-04-25T09:04:10.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.133+0000 7f4549e00640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f452806b4d0 0x7f452806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.134+0000 7f4549e00640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f452806b4d0 0x7f452806d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f453c034000 tx=0x7f453c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.137+0000 7f4542000640 1 -- 172.21.15.62:0/2070701731 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f454c141000 con 0x7f454c0b36a0 2024-04-25T09:04:10.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.335+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f454c1580f0 con 0x7f454c0b36a0 2024-04-25T09:04:10.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.336+0000 7f4542000640 1 -- 172.21.15.62:0/2070701731 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4534058480 con 0x7f454c0b36a0 2024-04-25T09:04:10.325 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:04:10.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f452806b4d0 msgr2=0x7f452806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f452806b4d0 0x7f452806d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f453c034000 tx=0x7f453c02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:10.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b36a0 msgr2=0x7f454c1568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b36a0 0x7f454c1568a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f453400c520 tx=0x7f453400c9e0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 shutdown_connections 2024-04-25T09:04:10.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f452806b4d0 0x7f452806d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f454c0b36a0 0x7f454c1568a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f454c0b54d0 0x7f454c156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 --2- 172.21.15.62:0/2070701731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f454c0b2d80 0x7f454c140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 >> 172.21.15.62:0/2070701731 conn(0x7f454c066f20 msgr2=0x7f454c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 shutdown_connections 2024-04-25T09:04:10.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.339+0000 7f454b200640 1 -- 172.21.15.62:0/2070701731 wait complete. 2024-04-25T09:04:10.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.572+0000 7f06c3e00640 1 Processor -- start 2024-04-25T09:04:10.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.572+0000 7f06c3e00640 1 -- start start 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06c3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 0x7f06c40ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06c3e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06c40ce7f0 0x7f06c4164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06c3e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c40b4430 con 0x7f06c40b2d80 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06c3e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c40b4710 con 0x7f06c40ce7f0 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06c3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40b49f0 con 0x7f06c40b54d0 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06c2a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f06c40b2d80 0x7f06c40b3160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51760/0 (socket says 172.21.15.62:51760) 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06c2a00640 1 -- 172.21.15.62:0/3888876223 learned_addr learned my addr 172.21.15.62:0/3888876223 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06bbe00640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 0x7f06c40ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.574+0000 7f06c3400640 1 --2- 172.21.15.62:0/3888876223 >> v2:172.21.15.121:3300/0 conn(0x7f06c40ce7f0 0x7f06c4164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574844973 0 0) 0x7f06c40b4430 con 0x7f06c40b2d80 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a8005730 con 0x7f06c40b2d80 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3338674219 0 0) 0x7f06a8005730 con 0x7f06c40b2d80 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 msgr2=0x7f06c40ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 0x7f06c40ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 >> v2:172.21.15.121:3300/0 conn(0x7f06c40ce7f0 msgr2=0x7f06c4164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 --2- 172.21.15.62:0/3888876223 >> v2:172.21.15.121:3300/0 conn(0x7f06c40ce7f0 0x7f06c4164310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c40b4cd0 con 0x7f06c40b2d80 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f06b4001670 con 0x7f06c40b2d80 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 >> v1:172.21.15.62:6789/0 conn(0x7f06c40b2d80 legacy=0x7f06c40b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 0x7f06a8005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06a8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06a8006f50 0x7f06a80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40b4710 con 0x7f06a8006140 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40b49f0 con 0x7f06a8006f50 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a800dbd0 con 0x7f06c40b54d0 2024-04-25T09:04:10.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.575+0000 7f06bbe00640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06a8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.576+0000 7f06c3400640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06a8006f50 0x7f06a80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.576+0000 7f06c2a00640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 0x7f06a8005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.576+0000 7f06bbe00640 1 -- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 msgr2=0x7f06a8005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.576+0000 7f06bbe00640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 0x7f06a8005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.576+0000 7f06bbe00640 1 -- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06a8006f50 msgr2=0x7f06a80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.576+0000 7f06bbe00640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06a8006f50 0x7f06a80073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.576+0000 7f06bbe00640 1 -- 172.21.15.62:0/3888876223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8005730 con 0x7f06a8006140 2024-04-25T09:04:10.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.576+0000 7f06c2a00640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c40b54d0 0x7f06a8005bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.577+0000 7f06bbe00640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06a8006a10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009a90 tx=0x7f06a8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408bc19d2c7a9730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.577+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac036070 con 0x7f06a8006140 2024-04-25T09:04:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.577+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac031400 con 0x7f06a8006140 2024-04-25T09:04:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.577+0000 7f06c2000640 1 -- 172.21.15.62:0/3888876223 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac009690 con 0x7f06a8006140 2024-04-25T09:04:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.578+0000 7f06c3e00640 1 -- 172.21.15.62:0/3888876223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 msgr2=0x7f06a8006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.578+0000 7f06c3e00640 1 --2- 172.21.15.62:0/3888876223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06a8006a10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009a90 tx=0x7f06a8007a40 comp rx=0 tx=0).stop 2024-04-25T09:04:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.578+0000 7f06c3400640 1 -- 172.21.15.62:0/3888876223 reap_dead start 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.578+0000 7f06c3e00640 1 -- 172.21.15.62:0/3888876223 shutdown_connections 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.578+0000 7f06c3e00640 1 -- 172.21.15.62:0/3888876223 >> 172.21.15.62:0/3888876223 conn(0x7f06c4066f20 msgr2=0x7f06c40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.578+0000 7f06c3e00640 1 -- 172.21.15.62:0/3888876223 shutdown_connections 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.578+0000 7f06c3e00640 1 -- 172.21.15.62:0/3888876223 wait complete. 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.580+0000 7f06c3e00640 1 Processor -- start 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.580+0000 7f06c3e00640 1 -- start start 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.581+0000 7f06c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06c41f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.581+0000 7f06c3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c414d3f0 0x7f06c4153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.581+0000 7f06c3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40ad8e0 0x7f06c40add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.581+0000 7f06c3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40b3870 con 0x7f06a8006140 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.581+0000 7f06c3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40b40e0 con 0x7f06c40ad8e0 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.581+0000 7f06c3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c40ce2b0 con 0x7f06c414d3f0 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.582+0000 7f06c2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06c41f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.582+0000 7f06c2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06c41f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50738/0 (socket says 172.21.15.62:50738) 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.582+0000 7f06c2a00640 1 -- 172.21.15.62:0/3079701141 learned_addr learned my addr 172.21.15.62:0/3079701141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.582+0000 7f06c2a00640 1 -- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c414d3f0 msgr2=0x7f06c4153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.582+0000 7f06c2a00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c414d3f0 0x7f06c4153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.582+0000 7f06c2a00640 1 -- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40ad8e0 msgr2=0x7f06c40add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.582+0000 7f06c2a00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40ad8e0 0x7f06c40add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.582+0000 7f06c2a00640 1 -- 172.21.15.62:0/3079701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c414e430 con 0x7f06a8006140 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.583+0000 7f06c2a00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06c41f4290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f06b400d810 tx=0x7f06b400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.583+0000 7f06bb400640 1 -- 172.21.15.62:0/3079701141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4019070 con 0x7f06a8006140 2024-04-25T09:04:10.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.583+0000 7f06bb400640 1 -- 172.21.15.62:0/3079701141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b40074e0 con 0x7f06a8006140 2024-04-25T09:04:10.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.583+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c414e710 con 0x7f06a8006140 2024-04-25T09:04:10.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.583+0000 7f06bb400640 1 -- 172.21.15.62:0/3079701141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4004a20 con 0x7f06a8006140 2024-04-25T09:04:10.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.583+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c41f3a90 con 0x7f06a8006140 2024-04-25T09:04:10.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.584+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c406abe0 con 0x7f06a8006140 2024-04-25T09:04:10.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.585+0000 7f06bb400640 1 -- 172.21.15.62:0/3079701141 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06b401c070 con 0x7f06a8006140 2024-04-25T09:04:10.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.586+0000 7f06bb400640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06a006b550 0x7f06a006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.586+0000 7f06bb400640 1 -- 172.21.15.62:0/3079701141 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f06b4089a50 con 0x7f06a8006140 2024-04-25T09:04:10.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.587+0000 7f06bbe00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06a006b550 0x7f06a006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.588+0000 7f06bbe00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06a006b550 0x7f06a006da10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f06ac033000 tx=0x7f06ac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.591+0000 7f06bb400640 1 -- 172.21.15.62:0/3079701141 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06b40582d0 con 0x7f06a8006140 2024-04-25T09:04:10.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.789+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06c404ef20 con 0x7f06a8006140 2024-04-25T09:04:10.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.790+0000 7f06bb400640 1 -- 172.21.15.62:0/3079701141 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06b405c370 con 0x7f06a8006140 2024-04-25T09:04:10.779 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06a006b550 msgr2=0x7f06a006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06a006b550 0x7f06a006da10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f06ac033000 tx=0x7f06ac02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 msgr2=0x7f06c41f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06c41f4290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f06b400d810 tx=0x7f06b400dcd0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 shutdown_connections 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06a006b550 0x7f06a006da10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06c40ad8e0 0x7f06c40add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06c414d3f0 0x7f06c4153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 --2- 172.21.15.62:0/3079701141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a8006140 0x7f06c41f4290 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 >> 172.21.15.62:0/3079701141 conn(0x7f06c4066f20 msgr2=0x7f06c40cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 shutdown_connections 2024-04-25T09:04:10.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:10.793+0000 7f06c3e00640 1 -- 172.21.15.62:0/3079701141 wait complete. 2024-04-25T09:04:10.804 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.031+0000 7f09aa800640 1 Processor -- start 2024-04-25T09:04:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.032+0000 7f09aa800640 1 -- start start 2024-04-25T09:04:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.033+0000 7f09aa800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a415bdf0 0x7f09a40720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.033+0000 7f09aa800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09a406c9b0 0x7f09a40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.033+0000 7f09aa800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a406d070 con 0x7f09a40cebc0 2024-04-25T09:04:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.033+0000 7f09aa800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a415c1d0 con 0x7f09a406c9b0 2024-04-25T09:04:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.034+0000 7f09aa800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40ae8c0 con 0x7f09a415bdf0 2024-04-25T09:04:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.034+0000 7f09a2a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f09a40cebc0 0x7f09a406c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51774/0 (socket says 172.21.15.62:51774) 2024-04-25T09:04:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.034+0000 7f09a2a00640 1 -- 172.21.15.62:0/1451595571 learned_addr learned my addr 172.21.15.62:0/1451595571 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:11.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.034+0000 7f09a3400640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a415bdf0 0x7f09a40720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.034+0000 7f09a3e00640 1 --2- 172.21.15.62:0/1451595571 >> v2:172.21.15.121:3300/0 conn(0x7f09a406c9b0 0x7f09a40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.034+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310582972 0 0) 0x7f09a406d070 con 0x7f09a40cebc0 2024-04-25T09:04:11.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0988003520 con 0x7f09a40cebc0 2024-04-25T09:04:11.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95206024 0 0) 0x7f0988003520 con 0x7f09a40cebc0 2024-04-25T09:04:11.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a415bdf0 msgr2=0x7f09a40720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a415bdf0 0x7f09a40720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 >> v2:172.21.15.121:3300/0 conn(0x7f09a406c9b0 msgr2=0x7f09a40703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 --2- 172.21.15.62:0/1451595571 >> v2:172.21.15.121:3300/0 conn(0x7f09a406c9b0 0x7f09a40703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a406d350 con 0x7f09a40cebc0 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f0990001670 con 0x7f09a40cebc0 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 >> v1:172.21.15.62:6789/0 conn(0x7f09a40cebc0 legacy=0x7f09a406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09880039b0 0x7f0988003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0988004800 0x7f0988004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a415bdf0 0x7f09880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a415c1d0 con 0x7f0988004800 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40ae8c0 con 0x7f09880039b0 2024-04-25T09:04:11.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098800b9c0 con 0x7f09a415bdf0 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.035+0000 7f09a2a00640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0988004800 0x7f0988004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.036+0000 7f09a3400640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09880039b0 0x7f0988003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.036+0000 7f09a3e00640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a415bdf0 0x7f09880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.036+0000 7f09a2a00640 1 -- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a415bdf0 msgr2=0x7f09880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.036+0000 7f09a2a00640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a415bdf0 0x7f09880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.036+0000 7f09a2a00640 1 -- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09880039b0 msgr2=0x7f0988003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.036+0000 7f09a2a00640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09880039b0 0x7f0988003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.036+0000 7f09a2a00640 1 -- 172.21.15.62:0/1451595571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0988003520 con 0x7f0988004800 2024-04-25T09:04:11.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.036+0000 7f09a3400640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09880039b0 0x7f0988003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:11.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.037+0000 7f09a2a00640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0988004800 0x7f0988004c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0990001050 tx=0x7f0988005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bc08f9ecf3e2c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990007d70 con 0x7f0988004800 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0990005af0 con 0x7f0988004800 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09a2000640 1 -- 172.21.15.62:0/1451595571 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990007070 con 0x7f0988004800 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09aa800640 1 -- 172.21.15.62:0/1451595571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0988004800 msgr2=0x7f0988004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09aa800640 1 --2- 172.21.15.62:0/1451595571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0988004800 0x7f0988004c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0990001050 tx=0x7f0988005830 comp rx=0 tx=0).stop 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09a3e00640 1 -- 172.21.15.62:0/1451595571 reap_dead start 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09aa800640 1 -- 172.21.15.62:0/1451595571 shutdown_connections 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09aa800640 1 -- 172.21.15.62:0/1451595571 >> 172.21.15.62:0/1451595571 conn(0x7f09a40b1410 msgr2=0x7f09a40b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09aa800640 1 -- 172.21.15.62:0/1451595571 shutdown_connections 2024-04-25T09:04:11.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.038+0000 7f09aa800640 1 -- 172.21.15.62:0/1451595571 wait complete. 2024-04-25T09:04:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.040+0000 7f09aa800640 1 Processor -- start 2024-04-25T09:04:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.040+0000 7f09aa800640 1 -- start start 2024-04-25T09:04:11.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.041+0000 7f09aa800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0988004800 0x7f09a40af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.041+0000 7f09aa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a414afb0 0x7f09a41f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.041+0000 7f09aa800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a414b3f0 0x7f09a41f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.041+0000 7f09aa800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40720c0 con 0x7f09a414afb0 2024-04-25T09:04:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.041+0000 7f09aa800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40aedf0 con 0x7f0988004800 2024-04-25T09:04:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.041+0000 7f09aa800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4072240 con 0x7f09a414b3f0 2024-04-25T09:04:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a414afb0 0x7f09a41f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a414afb0 0x7f09a41f18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50766/0 (socket says 172.21.15.62:50766) 2024-04-25T09:04:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a2a00640 1 -- 172.21.15.62:0/3888405020 learned_addr learned my addr 172.21.15.62:0/3888405020 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a3400640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0988004800 0x7f09a40af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a3e00640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a414b3f0 0x7f09a41f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a2a00640 1 -- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a414b3f0 msgr2=0x7f09a41f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a2a00640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a414b3f0 0x7f09a41f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a2a00640 1 -- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0988004800 msgr2=0x7f09a40af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a2a00640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0988004800 0x7f09a40af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.042+0000 7f09a2a00640 1 -- 172.21.15.62:0/3888405020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a414dbf0 con 0x7f09a414afb0 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.043+0000 7f09a2a00640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a414afb0 0x7f09a41f18c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f099000c2d0 tx=0x7f0990000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.043+0000 7f099b400640 1 -- 172.21.15.62:0/3888405020 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990042070 con 0x7f09a414afb0 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.043+0000 7f099b400640 1 -- 172.21.15.62:0/3888405020 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0990007d50 con 0x7f09a414afb0 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.043+0000 7f099b400640 1 -- 172.21.15.62:0/3888405020 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099003e450 con 0x7f09a414afb0 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.043+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a414ded0 con 0x7f09a414afb0 2024-04-25T09:04:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.043+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a414e490 con 0x7f09a414afb0 2024-04-25T09:04:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.044+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a404ef20 con 0x7f09a414afb0 2024-04-25T09:04:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.045+0000 7f099b400640 1 -- 172.21.15.62:0/3888405020 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0990045070 con 0x7f09a414afb0 2024-04-25T09:04:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.046+0000 7f099b400640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f098406b4d0 0x7f098406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.046+0000 7f099b400640 1 -- 172.21.15.62:0/3888405020 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f09900aac70 con 0x7f09a414afb0 2024-04-25T09:04:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.047+0000 7f09a3400640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f098406b4d0 0x7f098406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.048+0000 7f09a3400640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f098406b4d0 0x7f098406d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f09940098b0 tx=0x7f0994008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.051+0000 7f099b400640 1 -- 172.21.15.62:0/3888405020 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f099004c030 con 0x7f09a414afb0 2024-04-25T09:04:11.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.249+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09a4071c90 con 0x7f09a414afb0 2024-04-25T09:04:11.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.250+0000 7f099b400640 1 -- 172.21.15.62:0/3888405020 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0990079a30 con 0x7f09a414afb0 2024-04-25T09:04:11.238 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:04:11.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f098406b4d0 msgr2=0x7f098406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f098406b4d0 0x7f098406d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f09940098b0 tx=0x7f0994008040 comp rx=0 tx=0).stop 2024-04-25T09:04:11.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a414afb0 msgr2=0x7f09a41f18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a414afb0 0x7f09a41f18c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f099000c2d0 tx=0x7f0990000fe0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 shutdown_connections 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f098406b4d0 0x7f098406d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a414afb0 0x7f09a41f18c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f09a414b3f0 0x7f09a41f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 --2- 172.21.15.62:0/3888405020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0988004800 0x7f09a40af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 >> 172.21.15.62:0/3888405020 conn(0x7f09a40b1410 msgr2=0x7f09a4159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 shutdown_connections 2024-04-25T09:04:11.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.252+0000 7f09aa800640 1 -- 172.21.15.62:0/3888405020 wait complete. 2024-04-25T09:04:11.261 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:11.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.480+0000 7f4271c00640 1 Processor -- start 2024-04-25T09:04:11.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.481+0000 7f4271c00640 1 -- start start 2024-04-25T09:04:11.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.482+0000 7f4271c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f426c0b27a0 0x7f426c0d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.482+0000 7f4271c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426c0d3110 con 0x7f426c0b56f0 2024-04-25T09:04:11.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.482+0000 7f4271c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426c0d33f0 con 0x7f426c0b6a90 2024-04-25T09:04:11.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.482+0000 7f4271c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c0cedc0 con 0x7f426c0b27a0 2024-04-25T09:04:11.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.482+0000 7f426be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f426c0b56f0 0x7f426c160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51778/0 (socket says 172.21.15.62:51778) 2024-04-25T09:04:11.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.482+0000 7f426be00640 1 -- 172.21.15.62:0/38644445 learned_addr learned my addr 172.21.15.62:0/38644445 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:11.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f4271200640 1 --2- 172.21.15.62:0/38644445 >> v2:172.21.15.121:3300/0 conn(0x7f426c0b27a0 0x7f426c0d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727484165 0 0) 0x7f426c0d3110 con 0x7f426c0b56f0 2024-04-25T09:04:11.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4250003520 con 0x7f426c0b56f0 2024-04-25T09:04:11.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3566343111 0 0) 0x7f4250003520 con 0x7f426c0b56f0 2024-04-25T09:04:11.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 >> v2:172.21.15.121:3300/0 conn(0x7f426c0b27a0 msgr2=0x7f426c0d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 --2- 172.21.15.62:0/38644445 >> v2:172.21.15.121:3300/0 conn(0x7f426c0b27a0 0x7f426c0d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 >> v1:172.21.15.121:6789/0 conn(0x7f426c0b6a90 legacy=0x7f426c0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0d36d0 con 0x7f426c0b56f0 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f425c001670 con 0x7f426c0b56f0 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 >> v1:172.21.15.62:6789/0 conn(0x7f426c0b56f0 legacy=0x7f426c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42500039b0 0x7f4250003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4250004800 0x7f4250004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f42500051c0 0x7f425000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0cedc0 con 0x7f42500039b0 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425000c060 con 0x7f4250004800 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.483+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425000c260 con 0x7f42500051c0 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f426be00640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42500039b0 0x7f4250003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f426be00640 1 -- 172.21.15.62:0/38644445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f42500051c0 msgr2=0x7f425000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f426be00640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f42500051c0 0x7f425000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f4263e00640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4250004800 0x7f4250004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f426be00640 1 -- 172.21.15.62:0/38644445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4250004800 msgr2=0x7f4250004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f426be00640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4250004800 0x7f4250004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f4271200640 1 -- 172.21.15.62:0/38644445 reap_dead start 2024-04-25T09:04:11.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f426be00640 1 -- 172.21.15.62:0/38644445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4250003520 con 0x7f42500039b0 2024-04-25T09:04:11.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.484+0000 7f4263e00640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4250004800 0x7f4250004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:11.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.485+0000 7f426be00640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42500039b0 0x7f4250003dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c240 tx=0x7f4250005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdabeb1dcd3970c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.486+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c007cf0 con 0x7f42500039b0 2024-04-25T09:04:11.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.486+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c005af0 con 0x7f42500039b0 2024-04-25T09:04:11.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.486+0000 7f426b400640 1 -- 172.21.15.62:0/38644445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c007070 con 0x7f42500039b0 2024-04-25T09:04:11.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.486+0000 7f4271c00640 1 -- 172.21.15.62:0/38644445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42500039b0 msgr2=0x7f4250003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.486+0000 7f4271c00640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42500039b0 0x7f4250003dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c240 tx=0x7f4250005830 comp rx=0 tx=0).stop 2024-04-25T09:04:11.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.486+0000 7f4271c00640 1 -- 172.21.15.62:0/38644445 shutdown_connections 2024-04-25T09:04:11.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.486+0000 7f4271c00640 1 --2- 172.21.15.62:0/38644445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42500039b0 0x7f4250003dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.486+0000 7f4271c00640 1 -- 172.21.15.62:0/38644445 >> 172.21.15.62:0/38644445 conn(0x7f426c066d10 msgr2=0x7f426c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.487+0000 7f4271c00640 1 -- 172.21.15.62:0/38644445 shutdown_connections 2024-04-25T09:04:11.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.487+0000 7f4271c00640 1 -- 172.21.15.62:0/38644445 wait complete. 2024-04-25T09:04:11.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.488+0000 7f4271c00640 1 Processor -- start 2024-04-25T09:04:11.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.488+0000 7f4271c00640 1 -- start start 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.489+0000 7f4271c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42500039b0 0x7f426c0cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.489+0000 7f4271c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f426c14ec90 0x7f426c1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.489+0000 7f4271c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1596f0 0x7f426c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.489+0000 7f4271c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0b2ee0 con 0x7f426c1596f0 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.489+0000 7f4271c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0b2d60 con 0x7f42500039b0 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.489+0000 7f4271c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0b5ad0 con 0x7f426c14ec90 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4271200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1596f0 0x7f426c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4271200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1596f0 0x7f426c15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50796/0 (socket says 172.21.15.62:50796) 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4271200640 1 -- 172.21.15.62:0/3729746886 learned_addr learned my addr 172.21.15.62:0/3729746886 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f426be00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42500039b0 0x7f426c0cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4263e00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f426c14ec90 0x7f426c1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4271200640 1 -- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f426c14ec90 msgr2=0x7f426c1591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4271200640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f426c14ec90 0x7f426c1591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4271200640 1 -- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42500039b0 msgr2=0x7f426c0cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4271200640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42500039b0 0x7f426c0cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4271200640 1 -- 172.21.15.62:0/3729746886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c15b460 con 0x7f426c1596f0 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f426be00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42500039b0 0x7f426c0cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.490+0000 7f4263e00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f426c14ec90 0x7f426c1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.491+0000 7f4271200640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1596f0 0x7f426c15d7f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f426c0ced30 tx=0x7f426400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.491+0000 7f4268c00640 1 -- 172.21.15.62:0/3729746886 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426400bb10 con 0x7f426c1596f0 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.491+0000 7f4268c00640 1 -- 172.21.15.62:0/3729746886 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426401a070 con 0x7f426c1596f0 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.491+0000 7f4268c00640 1 -- 172.21.15.62:0/3729746886 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264014400 con 0x7f426c1596f0 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.491+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c15b6d0 con 0x7f426c1596f0 2024-04-25T09:04:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.491+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c15bc90 con 0x7f426c1596f0 2024-04-25T09:04:11.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.492+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c15be90 con 0x7f426c1596f0 2024-04-25T09:04:11.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.493+0000 7f4268c00640 1 -- 172.21.15.62:0/3729746886 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4264005d10 con 0x7f426c1596f0 2024-04-25T09:04:11.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.494+0000 7f4268c00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f424806b500 0x7f424806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.494+0000 7f4268c00640 1 -- 172.21.15.62:0/3729746886 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f426408a1a0 con 0x7f426c1596f0 2024-04-25T09:04:11.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.495+0000 7f426be00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f424806b500 0x7f424806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.496+0000 7f426be00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f424806b500 0x7f424806d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f425c032000 tx=0x7f425c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.498+0000 7f4268c00640 1 -- 172.21.15.62:0/3729746886 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f426c15be90 con 0x7f426c1596f0 2024-04-25T09:04:11.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.735+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f426c04ef20 con 0x7f426c1596f0 2024-04-25T09:04:11.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.736+0000 7f4268c00640 1 -- 172.21.15.62:0/3729746886 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f4264059000 con 0x7f426c1596f0 2024-04-25T09:04:11.725 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:11.725 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:16.609043+0000","modified":"2024-04-25T09:03:19.983991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:04:11.726 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f424806b500 msgr2=0x7f424806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f424806b500 0x7f424806d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f425c032000 tx=0x7f425c009030 comp rx=0 tx=0).stop 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1596f0 msgr2=0x7f426c15d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1596f0 0x7f426c15d7f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f426c0ced30 tx=0x7f426400ef30 comp rx=0 tx=0).stop 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 shutdown_connections 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f424806b500 0x7f424806d9c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426c1596f0 0x7f426c15d7f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f426c14ec90 0x7f426c1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 --2- 172.21.15.62:0/3729746886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42500039b0 0x7f426c0cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 >> 172.21.15.62:0/3729746886 conn(0x7f426c066d10 msgr2=0x7f426c0b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 shutdown_connections 2024-04-25T09:04:11.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.739+0000 7f4271c00640 1 -- 172.21.15.62:0/3729746886 wait complete. 2024-04-25T09:04:11.747 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:11.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.973+0000 7fbc5a400640 1 Processor -- start 2024-04-25T09:04:11.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.973+0000 7fbc5a400640 1 -- start start 2024-04-25T09:04:11.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.975+0000 7fbc5a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc5406d900 0x7fbc5415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.975+0000 7fbc5a400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbc540cee40 0x7fbc540b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.975+0000 7fbc5a400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc540cf2d0 con 0x7fbc54144c10 2024-04-25T09:04:11.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.975+0000 7fbc5a400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc54144ff0 con 0x7fbc540cee40 2024-04-25T09:04:11.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.975+0000 7fbc5a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54160340 con 0x7fbc5406d900 2024-04-25T09:04:11.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.975+0000 7fbc59000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbc54144c10 0x7fbc5406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51786/0 (socket says 172.21.15.62:51786) 2024-04-25T09:04:11.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.975+0000 7fbc59000640 1 -- 172.21.15.62:0/1494225455 learned_addr learned my addr 172.21.15.62:0/1494225455 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc59a00640 1 --2- 172.21.15.62:0/1494225455 >> v2:172.21.15.121:3300/0 conn(0x7fbc540cee40 0x7fbc540b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc4be00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc5406d900 0x7fbc5415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237765744 0 0) 0x7fbc540cf2d0 con 0x7fbc54144c10 2024-04-25T09:04:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc38003520 con 0x7fbc54144c10 2024-04-25T09:04:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3289104704 0 0) 0x7fbc38003520 con 0x7fbc54144c10 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc5406d900 msgr2=0x7fbc5415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc5406d900 0x7fbc5415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 >> v2:172.21.15.121:3300/0 conn(0x7fbc540cee40 msgr2=0x7fbc540b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 --2- 172.21.15.62:0/1494225455 >> v2:172.21.15.121:3300/0 conn(0x7fbc540cee40 0x7fbc540b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5415faa0 con 0x7fbc54144c10 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fbc44001670 con 0x7fbc54144c10 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 >> v1:172.21.15.62:6789/0 conn(0x7fbc54144c10 legacy=0x7fbc5406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc380039b0 0x7fbc38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc38004800 0x7fbc38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc5406d900 0x7fbc380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54144ff0 con 0x7fbc38004800 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54160340 con 0x7fbc380039b0 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.976+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3800b9f0 con 0x7fbc5406d900 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.977+0000 7fbc4be00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc38004800 0x7fbc38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.977+0000 7fbc59000640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc380039b0 0x7fbc38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.977+0000 7fbc59a00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc5406d900 0x7fbc380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.977+0000 7fbc4be00640 1 -- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc5406d900 msgr2=0x7fbc380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.977+0000 7fbc4be00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc5406d900 0x7fbc380051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.977+0000 7fbc4be00640 1 -- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc380039b0 msgr2=0x7fbc38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.977+0000 7fbc4be00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc380039b0 0x7fbc38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.977+0000 7fbc4be00640 1 -- 172.21.15.62:0/1494225455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38003520 con 0x7fbc38004800 2024-04-25T09:04:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.978+0000 7fbc4be00640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc38004800 0x7fbc38004c80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009ac0 tx=0x7fbc38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355bcc71e62828fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c036070 con 0x7fbc38004800 2024-04-25T09:04:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c031400 con 0x7fbc38004800 2024-04-25T09:04:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc53e00640 1 -- 172.21.15.62:0/1494225455 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c009690 con 0x7fbc38004800 2024-04-25T09:04:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc5a400640 1 -- 172.21.15.62:0/1494225455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc38004800 msgr2=0x7fbc38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc5a400640 1 --2- 172.21.15.62:0/1494225455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc38004800 0x7fbc38004c80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009ac0 tx=0x7fbc38005830 comp rx=0 tx=0).stop 2024-04-25T09:04:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc59a00640 1 -- 172.21.15.62:0/1494225455 reap_dead start 2024-04-25T09:04:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc5a400640 1 -- 172.21.15.62:0/1494225455 shutdown_connections 2024-04-25T09:04:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc5a400640 1 -- 172.21.15.62:0/1494225455 >> 172.21.15.62:0/1494225455 conn(0x7fbc540b14d0 msgr2=0x7fbc540b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.979+0000 7fbc5a400640 1 -- 172.21.15.62:0/1494225455 shutdown_connections 2024-04-25T09:04:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.980+0000 7fbc5a400640 1 -- 172.21.15.62:0/1494225455 wait complete. 2024-04-25T09:04:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.981+0000 7fbc5a400640 1 Processor -- start 2024-04-25T09:04:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.982+0000 7fbc5a400640 1 -- start start 2024-04-25T09:04:11.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.982+0000 7fbc5a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc38004800 0x7fbc5413d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc5a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc54138090 0x7fbc5415da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc5a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc5413e840 0x7fbc540714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc5a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54160060 con 0x7fbc5413e840 2024-04-25T09:04:11.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc5a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540b41c0 con 0x7fbc38004800 2024-04-25T09:04:11.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc5a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5415f450 con 0x7fbc54138090 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc5413e840 0x7fbc540714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc5413e840 0x7fbc540714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50814/0 (socket says 172.21.15.62:50814) 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59a00640 1 -- 172.21.15.62:0/3819988189 learned_addr learned my addr 172.21.15.62:0/3819988189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc4be00640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc54138090 0x7fbc5415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59000640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc38004800 0x7fbc5413d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59a00640 1 -- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc54138090 msgr2=0x7fbc5415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59a00640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc54138090 0x7fbc5415da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59a00640 1 -- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc38004800 msgr2=0x7fbc5413d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59a00640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc38004800 0x7fbc5413d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59a00640 1 -- 172.21.15.62:0/3819988189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc54138510 con 0x7fbc5413e840 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.983+0000 7fbc59000640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc38004800 0x7fbc5413d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.984+0000 7fbc59a00640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc5413e840 0x7fbc540714d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbc540c5580 tx=0x7fbc4c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.984+0000 7fbc51600640 1 -- 172.21.15.62:0/3819988189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c016420 con 0x7fbc5413e840 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.984+0000 7fbc51600640 1 -- 172.21.15.62:0/3819988189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c01b070 con 0x7fbc5413e840 2024-04-25T09:04:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.984+0000 7fbc51600640 1 -- 172.21.15.62:0/3819988189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c016820 con 0x7fbc5413e840 2024-04-25T09:04:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.984+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5413d7a0 con 0x7fbc5413e840 2024-04-25T09:04:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.984+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5413dd60 con 0x7fbc5413e840 2024-04-25T09:04:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.985+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5404ef20 con 0x7fbc5413e840 2024-04-25T09:04:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.986+0000 7fbc51600640 1 -- 172.21.15.62:0/3819988189 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc4c005d10 con 0x7fbc5413e840 2024-04-25T09:04:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.987+0000 7fbc51600640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc3006b500 0x7fbc3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.987+0000 7fbc51600640 1 -- 172.21.15.62:0/3819988189 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbc4c08ac30 con 0x7fbc5413e840 2024-04-25T09:04:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.988+0000 7fbc59000640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc3006b500 0x7fbc3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.989+0000 7fbc59000640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc3006b500 0x7fbc3006d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc44001200 tx=0x7fbc44001110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:11.992+0000 7fbc51600640 1 -- 172.21.15.62:0/3819988189 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc4c0599f0 con 0x7fbc5413e840 2024-04-25T09:04:12.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.189+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc5415df50 con 0x7fbc5413e840 2024-04-25T09:04:12.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.192+0000 7fbc51600640 1 -- 172.21.15.62:0/3819988189 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16646 (secure 0 0 0) 0x7fbc4c0592f0 con 0x7fbc5413e840 2024-04-25T09:04:12.181 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:12.181 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:59.724537+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:12.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.194+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc3006b500 msgr2=0x7fbc3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.194+0000 7fbc5a400640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc3006b500 0x7fbc3006d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc44001200 tx=0x7fbc44001110 comp rx=0 tx=0).stop 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.194+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc5413e840 msgr2=0x7fbc540714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.194+0000 7fbc5a400640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc5413e840 0x7fbc540714d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbc540c5580 tx=0x7fbc4c00f520 comp rx=0 tx=0).stop 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.194+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 shutdown_connections 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.194+0000 7fbc5a400640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbc3006b500 0x7fbc3006d9c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.195+0000 7fbc5a400640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc5413e840 0x7fbc540714d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.195+0000 7fbc5a400640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbc54138090 0x7fbc5415da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.195+0000 7fbc5a400640 1 --2- 172.21.15.62:0/3819988189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc38004800 0x7fbc5413d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.195+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 >> 172.21.15.62:0/3819988189 conn(0x7fbc540b14d0 msgr2=0x7fbc5406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.195+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 shutdown_connections 2024-04-25T09:04:12.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.195+0000 7fbc5a400640 1 -- 172.21.15.62:0/3819988189 wait complete. 2024-04-25T09:04:12.202 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:04:12.202 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.417+0000 7f4f24400640 1 Processor -- start 2024-04-25T09:04:12.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.418+0000 7f4f24400640 1 -- start start 2024-04-25T09:04:12.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.419+0000 7f4f24400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f1c15de70 0x7f4f1c0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.419+0000 7f4f24400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b9900 0x7f4f1c0b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.419+0000 7f4f24400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0b7780 con 0x7f4f1c0b2d40 2024-04-25T09:04:12.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.419+0000 7f4f24400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b8320 con 0x7f4f1c15de70 2024-04-25T09:04:12.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.419+0000 7f4f24400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b79f0 con 0x7f4f1c0b9900 2024-04-25T09:04:12.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.420+0000 7f4f21a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f1c15de70 0x7f4f1c0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.420+0000 7f4f21a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f1c15de70 0x7f4f1c0b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:44438/0 (socket says 172.21.15.62:44438) 2024-04-25T09:04:12.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.420+0000 7f4f21a00640 1 -- 172.21.15.62:0/3337600772 learned_addr learned my addr 172.21.15.62:0/3337600772 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:12.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.420+0000 7f4f21000640 1 --2- 172.21.15.62:0/3337600772 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b9900 0x7f4f1c0b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.421+0000 7f4f21a00640 1 -- 172.21.15.62:0/3337600772 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b9900 msgr2=0x7f4f1c0b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.421+0000 7f4f21a00640 1 --2- 172.21.15.62:0/3337600772 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b9900 0x7f4f1c0b87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.421+0000 7f4f21a00640 1 -- 172.21.15.62:0/3337600772 >> v1:172.21.15.121:6789/0 conn(0x7f4f1c0b2d40 legacy=0x7f4f1c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.421+0000 7f4f21a00640 1 -- 172.21.15.62:0/3337600772 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0b7cd0 con 0x7f4f1c15de70 2024-04-25T09:04:12.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.423+0000 7f4f21a00640 1 --2- 172.21.15.62:0/3337600772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 0x7f4f1c0b93c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009870 tx=0x7f4f1002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c28fa11cbcf97fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.423+0000 7f4f1be00640 1 -- 172.21.15.62:0/3337600772 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10037070 con 0x7f4f1c15de70 2024-04-25T09:04:12.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.423+0000 7f4f1be00640 1 -- 172.21.15.62:0/3337600772 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1002ae50 con 0x7f4f1c15de70 2024-04-25T09:04:12.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.423+0000 7f4f1be00640 1 -- 172.21.15.62:0/3337600772 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10032860 con 0x7f4f1c15de70 2024-04-25T09:04:12.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.424+0000 7f4f24400640 1 -- 172.21.15.62:0/3337600772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 msgr2=0x7f4f1c0b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.424+0000 7f4f24400640 1 --2- 172.21.15.62:0/3337600772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 0x7f4f1c0b93c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009870 tx=0x7f4f1002a300 comp rx=0 tx=0).stop 2024-04-25T09:04:12.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.424+0000 7f4f24400640 1 -- 172.21.15.62:0/3337600772 shutdown_connections 2024-04-25T09:04:12.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.424+0000 7f4f24400640 1 --2- 172.21.15.62:0/3337600772 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b9900 0x7f4f1c0b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.424+0000 7f4f24400640 1 --2- 172.21.15.62:0/3337600772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 0x7f4f1c0b93c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.424+0000 7f4f24400640 1 -- 172.21.15.62:0/3337600772 >> 172.21.15.62:0/3337600772 conn(0x7f4f1c06b000 msgr2=0x7f4f1c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.424+0000 7f4f24400640 1 -- 172.21.15.62:0/3337600772 shutdown_connections 2024-04-25T09:04:12.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.424+0000 7f4f24400640 1 -- 172.21.15.62:0/3337600772 wait complete. 2024-04-25T09:04:12.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.426+0000 7f4f24400640 1 Processor -- start 2024-04-25T09:04:12.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.426+0000 7f4f24400640 1 -- start start 2024-04-25T09:04:12.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.427+0000 7f4f24400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b2d40 0x7f4f1c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.427+0000 7f4f24400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0b9900 0x7f4f1c14f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.427+0000 7f4f24400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 0x7f4f1c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.427+0000 7f4f24400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0d2710 con 0x7f4f1c0b9900 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.427+0000 7f4f24400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0d2590 con 0x7f4f1c15de70 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.427+0000 7f4f24400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b8cf0 con 0x7f4f1c0b2d40 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0b9900 0x7f4f1c14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0b9900 0x7f4f1c14f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50830/0 (socket says 172.21.15.62:50830) 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 -- 172.21.15.62:0/1327548131 learned_addr learned my addr 172.21.15.62:0/1327548131 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21a00640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b2d40 0x7f4f1c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f22400640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 0x7f4f1c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 -- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b2d40 msgr2=0x7f4f1c15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b2d40 0x7f4f1c15f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 -- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 msgr2=0x7f4f1c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 0x7f4f1c0cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 -- 172.21.15.62:0/1327548131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c15e430 con 0x7f4f1c0b9900 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21a00640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b2d40 0x7f4f1c15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f22400640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 0x7f4f1c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.428+0000 7f4f21000640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0b9900 0x7f4f1c14f140 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00e9b0 tx=0x7f4f0c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.429+0000 7f4f19600640 1 -- 172.21.15.62:0/1327548131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c017070 con 0x7f4f1c0b9900 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.429+0000 7f4f19600640 1 -- 172.21.15.62:0/1327548131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c00c9a0 con 0x7f4f1c0b9900 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.429+0000 7f4f19600640 1 -- 172.21.15.62:0/1327548131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c00cd90 con 0x7f4f1c0b9900 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.429+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c15e710 con 0x7f4f1c0b9900 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.429+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c15ecd0 con 0x7f4f1c0b9900 2024-04-25T09:04:12.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.430+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c0729a0 con 0x7f4f1c0b9900 2024-04-25T09:04:12.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.431+0000 7f4f19600640 1 -- 172.21.15.62:0/1327548131 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f0c01a070 con 0x7f4f1c0b9900 2024-04-25T09:04:12.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.432+0000 7f4f19600640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f0006b4d0 0x7f4f0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.433+0000 7f4f19600640 1 -- 172.21.15.62:0/1327548131 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4f0c08a6a0 con 0x7f4f1c0b9900 2024-04-25T09:04:12.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.433+0000 7f4f21a00640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f0006b4d0 0x7f4f0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.435+0000 7f4f21a00640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f0006b4d0 0x7f4f0006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4f10034000 tx=0x7f4f1002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.437+0000 7f4f19600640 1 -- 172.21.15.62:0/1327548131 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f0c059460 con 0x7f4f1c0b9900 2024-04-25T09:04:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.675+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f1c0c72a0 con 0x7f4f1c0b9900 2024-04-25T09:04:12.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.676+0000 7f4f19600640 1 -- 172.21.15.62:0/1327548131 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f4f0c058d60 con 0x7f4f1c0b9900 2024-04-25T09:04:12.665 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:12.665 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:16.609043+0000","modified":"2024-04-25T09:03:19.983991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:04:12.666 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-25T09:04:12.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.678+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f0006b4d0 msgr2=0x7f4f0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.678+0000 7f4f24400640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f0006b4d0 0x7f4f0006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4f10034000 tx=0x7f4f1002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:12.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0b9900 msgr2=0x7f4f1c14f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0b9900 0x7f4f1c14f140 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00e9b0 tx=0x7f4f0c00ee70 comp rx=0 tx=0).stop 2024-04-25T09:04:12.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 shutdown_connections 2024-04-25T09:04:12.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f0006b4d0 0x7f4f0006d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f1c0b9900 0x7f4f1c14f140 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f1c15de70 0x7f4f1c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 --2- 172.21.15.62:0/1327548131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f1c0b2d40 0x7f4f1c15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 >> 172.21.15.62:0/1327548131 conn(0x7f4f1c06b000 msgr2=0x7f4f1c0b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 shutdown_connections 2024-04-25T09:04:12.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.679+0000 7f4f24400640 1 -- 172.21.15.62:0/1327548131 wait complete. 2024-04-25T09:04:12.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:12.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.909+0000 7fd8fb200640 1 Processor -- start 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.910+0000 7fd8fb200640 1 -- start start 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.911+0000 7fd8fb200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd8fc144c10 0x7fd8fc15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.911+0000 7fd8fb200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8fc06d900 0x7fd8fc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.911+0000 7fd8fb200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0cf2d0 con 0x7fd8fc0cee40 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.911+0000 7fd8fb200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc144ff0 con 0x7fd8fc06d900 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.911+0000 7fd8fb200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc160340 con 0x7fd8fc144c10 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8fa800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8fc06d900 0x7fd8fc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8fa800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8fc06d900 0x7fd8fc0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50832/0 (socket says 172.21.15.62:50832) 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8fa800640 1 -- 172.21.15.62:0/52366102 learned_addr learned my addr 172.21.15.62:0/52366102 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8f9e00640 1 --2- 172.21.15.62:0/52366102 >> v2:172.21.15.121:3300/0 conn(0x7fd8fc144c10 0x7fd8fc15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8fa800640 1 -- 172.21.15.62:0/52366102 >> v2:172.21.15.121:3300/0 conn(0x7fd8fc144c10 msgr2=0x7fd8fc15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8fa800640 1 --2- 172.21.15.62:0/52366102 >> v2:172.21.15.121:3300/0 conn(0x7fd8fc144c10 0x7fd8fc15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8fa800640 1 -- 172.21.15.62:0/52366102 >> v1:172.21.15.121:6789/0 conn(0x7fd8fc0cee40 legacy=0x7fd8fc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8fa800640 1 -- 172.21.15.62:0/52366102 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc15faa0 con 0x7fd8fc06d900 2024-04-25T09:04:12.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.912+0000 7fd8f9e00640 1 --2- 172.21.15.62:0/52366102 >> v2:172.21.15.121:3300/0 conn(0x7fd8fc144c10 0x7fd8fc15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.913+0000 7fd8fa800640 1 --2- 172.21.15.62:0/52366102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc06d900 0x7fd8fc0b4650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0724a0 tx=0x7fd8f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56640ca02b983c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8f8a00640 1 -- 172.21.15.62:0/52366102 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4036070 con 0x7fd8fc06d900 2024-04-25T09:04:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8f8a00640 1 -- 172.21.15.62:0/52366102 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4031600 con 0x7fd8fc06d900 2024-04-25T09:04:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8f8a00640 1 -- 172.21.15.62:0/52366102 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40319d0 con 0x7fd8fc06d900 2024-04-25T09:04:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8fb200640 1 -- 172.21.15.62:0/52366102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc06d900 msgr2=0x7fd8fc0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8fb200640 1 --2- 172.21.15.62:0/52366102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc06d900 0x7fd8fc0b4650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0724a0 tx=0x7fd8f402a660 comp rx=0 tx=0).stop 2024-04-25T09:04:12.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8fb200640 1 -- 172.21.15.62:0/52366102 shutdown_connections 2024-04-25T09:04:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8fb200640 1 --2- 172.21.15.62:0/52366102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc06d900 0x7fd8fc0b4650 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8fb200640 1 --2- 172.21.15.62:0/52366102 >> v2:172.21.15.121:3300/0 conn(0x7fd8fc144c10 0x7fd8fc15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.914+0000 7fd8fb200640 1 -- 172.21.15.62:0/52366102 >> 172.21.15.62:0/52366102 conn(0x7fd8fc0b14d0 msgr2=0x7fd8fc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.915+0000 7fd8fb200640 1 -- 172.21.15.62:0/52366102 shutdown_connections 2024-04-25T09:04:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.915+0000 7fd8fb200640 1 -- 172.21.15.62:0/52366102 wait complete. 2024-04-25T09:04:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.916+0000 7fd8fb200640 1 Processor -- start 2024-04-25T09:04:12.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.916+0000 7fd8fb200640 1 -- start start 2024-04-25T09:04:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.917+0000 7fd8fb200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8fc06d900 0x7fd8fc15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.917+0000 7fd8fb200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8fc0cee40 0x7fd8fc142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.917+0000 7fd8fb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc144c10 0x7fd8fc142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.917+0000 7fd8fb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc160060 con 0x7fd8fc144c10 2024-04-25T09:04:12.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.917+0000 7fd8fb200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0b41c0 con 0x7fd8fc06d900 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.917+0000 7fd8fb200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0b4b90 con 0x7fd8fc0cee40 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.917+0000 7fd8fa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc144c10 0x7fd8fc142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8fa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc144c10 0x7fd8fc142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50844/0 (socket says 172.21.15.62:50844) 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8fa800640 1 -- 172.21.15.62:0/3426914393 learned_addr learned my addr 172.21.15.62:0/3426914393 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8f9e00640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8fc06d900 0x7fd8fc15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8fa800640 1 -- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8fc0cee40 msgr2=0x7fd8fc142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8fa800640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8fc0cee40 0x7fd8fc142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8fa800640 1 -- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8fc06d900 msgr2=0x7fd8fc15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8fa800640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8fc06d900 0x7fd8fc15afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8fa800640 1 -- 172.21.15.62:0/3426914393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0cc520 con 0x7fd8fc144c10 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.918+0000 7fd8fa800640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc144c10 0x7fd8fc142590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009f40 tx=0x7fd8f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.919+0000 7fd8f2000640 1 -- 172.21.15.62:0/3426914393 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4036050 con 0x7fd8fc144c10 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.919+0000 7fd8f2000640 1 -- 172.21.15.62:0/3426914393 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4042070 con 0x7fd8fc144c10 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.919+0000 7fd8f2000640 1 -- 172.21.15.62:0/3426914393 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4009510 con 0x7fd8fc144c10 2024-04-25T09:04:12.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.919+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc142bc0 con 0x7fd8fc144c10 2024-04-25T09:04:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.919+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc143180 con 0x7fd8fc144c10 2024-04-25T09:04:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.920+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc04ef20 con 0x7fd8fc144c10 2024-04-25T09:04:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.921+0000 7fd8f2000640 1 -- 172.21.15.62:0/3426914393 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8f4007bd0 con 0x7fd8fc144c10 2024-04-25T09:04:12.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.922+0000 7fd8f2000640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.922+0000 7fd8f2000640 1 -- 172.21.15.62:0/3426914393 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd8f40ab1c0 con 0x7fd8fc144c10 2024-04-25T09:04:12.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.922+0000 7fd8f9e00640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.923+0000 7fd8f9e00640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b840 tx=0x7fd8ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:12.926+0000 7fd8f2000640 1 -- 172.21.15.62:0/3426914393 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8f407a020 con 0x7fd8fc144c10 2024-04-25T09:04:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.170+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8fc06d370 con 0x7fd8fc144c10 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.172+0000 7fd8f2000640 1 -- 172.21.15.62:0/3426914393 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fd8f4079920 con 0x7fd8fc144c10 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3027298478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3027298478},{"type":"v1","addr":"172.21.15.121:6837","nonce":3027298478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3497834481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3497834481},{"type":"v1","addr":"172.21.15.62:6833","nonce":3497834481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4197833323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4197833323},{"type":"v1","addr":"172.21.15.62:6835","nonce":4197833323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:16.609043+0000","modified":"2024-04-25T09:03:19.983991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/1669215157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1669215157},{"type":"v1","addr":"172.21.15.121:6836","nonce":1669215157}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8dc06b4d0 msgr2=0x7fd8dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b840 tx=0x7fd8ec008040 comp rx=0 tx=0).stop 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc144c10 msgr2=0x7fd8fc142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc144c10 0x7fd8fc142590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009f40 tx=0x7fd8f4004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 shutdown_connections 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8fc144c10 0x7fd8fc142590 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8fc0cee40 0x7fd8fc142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 --2- 172.21.15.62:0/3426914393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8fc06d900 0x7fd8fc15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 >> 172.21.15.62:0/3426914393 conn(0x7fd8fc0b14d0 msgr2=0x7fd8fc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 shutdown_connections 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.175+0000 7fd8fb200640 1 -- 172.21.15.62:0/3426914393 wait complete. 2024-04-25T09:04:13.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:13.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.388+0000 7f3ec0c00640 1 Processor -- start 2024-04-25T09:04:13.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.388+0000 7f3ec0c00640 1 -- start start 2024-04-25T09:04:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.389+0000 7f3ec0c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3ebc0b56f0 0x7f3ebc160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.389+0000 7f3ec0c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ebc0b27a0 0x7f3ebc161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.389+0000 7f3ec0c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc0d3110 con 0x7f3ebc0b6a90 2024-04-25T09:04:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.390+0000 7f3ec0c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ebc161780 con 0x7f3ebc0b56f0 2024-04-25T09:04:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.390+0000 7f3ec0c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0cedc0 con 0x7f3ebc0b27a0 2024-04-25T09:04:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.390+0000 7f3ebaa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3ebc0b6a90 0x7f3ebc161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51796/0 (socket says 172.21.15.62:51796) 2024-04-25T09:04:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.390+0000 7f3ebaa00640 1 -- 172.21.15.62:0/1415235719 learned_addr learned my addr 172.21.15.62:0/1415235719 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:13.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.390+0000 7f3ebbe00640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ebc0b27a0 0x7f3ebc161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.390+0000 7f3ebb400640 1 --2- 172.21.15.62:0/1415235719 >> v2:172.21.15.121:3300/0 conn(0x7f3ebc0b56f0 0x7f3ebc160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.390+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39147964 0 0) 0x7f3ebc0d3110 con 0x7f3ebc0b6a90 2024-04-25T09:04:13.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.390+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ea0003520 con 0x7f3ebc0b6a90 2024-04-25T09:04:13.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3961973422 0 0) 0x7f3ea0003520 con 0x7f3ebc0b6a90 2024-04-25T09:04:13.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ebc0b27a0 msgr2=0x7f3ebc161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ebc0b27a0 0x7f3ebc161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 >> v2:172.21.15.121:3300/0 conn(0x7f3ebc0b56f0 msgr2=0x7f3ebc160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 --2- 172.21.15.62:0/1415235719 >> v2:172.21.15.121:3300/0 conn(0x7f3ebc0b56f0 0x7f3ebc160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0d33f0 con 0x7f3ebc0b6a90 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f3ea8001670 con 0x7f3ebc0b6a90 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 >> v1:172.21.15.62:6789/0 conn(0x7f3ebc0b6a90 legacy=0x7f3ebc161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ebc0b27a0 0x7f3ea00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea0003f80 0x7f3ea0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea0004d40 0x7f3ea00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0cedc0 con 0x7f3ea0004d40 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea000b9c0 con 0x7f3ea0003f80 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea000bb40 con 0x7f3ebc0b27a0 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.391+0000 7f3ebbe00640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea0004d40 0x7f3ea00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.392+0000 7f3ebb400640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ebc0b27a0 0x7f3ea00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.392+0000 7f3ebaa00640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea0003f80 0x7f3ea0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.392+0000 7f3ebbe00640 1 -- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ebc0b27a0 msgr2=0x7f3ea00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.392+0000 7f3ebbe00640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ebc0b27a0 0x7f3ea00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.392+0000 7f3ebbe00640 1 -- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea0003f80 msgr2=0x7f3ea0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.392+0000 7f3ebbe00640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea0003f80 0x7f3ea0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.392+0000 7f3ebbe00640 1 -- 172.21.15.62:0/1415235719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0003520 con 0x7f3ea0004d40 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.393+0000 7f3ebbe00640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea0004d40 0x7f3ea00051c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009fc0 tx=0x7f3ea0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e0226fb06bc86d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.393+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4009c10 con 0x7f3ea0004d40 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.393+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb4036070 con 0x7f3ea0004d40 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.393+0000 7f3eba000640 1 -- 172.21.15.62:0/1415235719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4031840 con 0x7f3ea0004d40 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.394+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1415235719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea0004d40 msgr2=0x7f3ea00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.394+0000 7f3ec0c00640 1 --2- 172.21.15.62:0/1415235719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ea0004d40 0x7f3ea00051c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009fc0 tx=0x7f3ea0005830 comp rx=0 tx=0).stop 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.394+0000 7f3ebbe00640 1 -- 172.21.15.62:0/1415235719 reap_dead start 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.394+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1415235719 shutdown_connections 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.394+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1415235719 >> 172.21.15.62:0/1415235719 conn(0x7f3ebc066d10 msgr2=0x7f3ebc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.394+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1415235719 shutdown_connections 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.394+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1415235719 wait complete. 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.396+0000 7f3ec0c00640 1 Processor -- start 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.396+0000 7f3ec0c00640 1 -- start start 2024-04-25T09:04:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ec0c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ea0004d40 0x7f3ebc160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ec0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ebc1f4f90 0x7f3ebc158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ec0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ebc159140 0x7f3ebc1f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ec0c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0b2d60 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ec0c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0cea50 con 0x7f3ebc159140 2024-04-25T09:04:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ec0c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0b2ee0 con 0x7f3ea0004d40 2024-04-25T09:04:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ebaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ebc1f4f90 0x7f3ebc158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ebaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ebc1f4f90 0x7f3ebc158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50868/0 (socket says 172.21.15.62:50868) 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ebaa00640 1 -- 172.21.15.62:0/1366801633 learned_addr learned my addr 172.21.15.62:0/1366801633 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ebb400640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ea0004d40 0x7f3ebc160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.397+0000 7f3ebbe00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ebc159140 0x7f3ebc1f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3ebaa00640 1 -- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ea0004d40 msgr2=0x7f3ebc160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3ebaa00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ea0004d40 0x7f3ebc160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3ebaa00640 1 -- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ebc159140 msgr2=0x7f3ebc1f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3ebaa00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ebc159140 0x7f3ebc1f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3ebaa00640 1 -- 172.21.15.62:0/1366801633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc15e510 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3ebaa00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ebc1f4f90 0x7f3ebc158c00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800e700 tx=0x7f3ea800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3eb3400640 1 -- 172.21.15.62:0/1366801633 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800ee10 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3eb3400640 1 -- 172.21.15.62:0/1366801633 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea8018070 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3eb3400640 1 -- 172.21.15.62:0/1366801633 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800caf0 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc1f4420 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.398+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc1f49e0 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.400+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc04ef20 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.400+0000 7f3eb3400640 1 -- 172.21.15.62:0/1366801633 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ea800cc90 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.401+0000 7f3eb3400640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e9c06b4d0 0x7f3e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.402+0000 7f3eb3400640 1 -- 172.21.15.62:0/1366801633 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3ea808a1b0 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.402+0000 7f3ebb400640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e9c06b4d0 0x7f3e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.403+0000 7f3ebb400640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e9c06b4d0 0x7f3e9c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009a90 tx=0x7f3eac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.406+0000 7f3eb3400640 1 -- 172.21.15.62:0/1366801633 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ea8058f70 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.607+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ebc0ce580 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.609+0000 7f3eb3400640 1 -- 172.21.15.62:0/1366801633 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f3ea8058870 con 0x7f3ebc1f4f90 2024-04-25T09:04:13.598 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:13.598 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":22,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:03:59.724537+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:15.055465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:15.619995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e9c06b4d0 msgr2=0x7f3e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e9c06b4d0 0x7f3e9c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009a90 tx=0x7f3eac008040 comp rx=0 tx=0).stop 2024-04-25T09:04:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ebc1f4f90 msgr2=0x7f3ebc158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ebc1f4f90 0x7f3ebc158c00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800e700 tx=0x7f3ea800ebc0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 shutdown_connections 2024-04-25T09:04:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e9c06b4d0 0x7f3e9c06d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ebc1f4f90 0x7f3ebc158c00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ebc159140 0x7f3ebc1f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 --2- 172.21.15.62:0/1366801633 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ea0004d40 0x7f3ebc160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 >> 172.21.15.62:0/1366801633 conn(0x7f3ebc066d10 msgr2=0x7f3ebc1e57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 shutdown_connections 2024-04-25T09:04:13.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.612+0000 7f3ec0c00640 1 -- 172.21.15.62:0/1366801633 wait complete. 2024-04-25T09:04:13.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.826+0000 7f2052e00640 1 Processor -- start 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.826+0000 7f2052e00640 1 -- start start 2024-04-25T09:04:13.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.827+0000 7f2052e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f204c15bd10 0x7f204c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.828+0000 7f2052e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c06ca70 con 0x7f204c0cccf0 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.828+0000 7f2052e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c15c0f0 con 0x7f204c0aeed0 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.828+0000 7f2052e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c06d660 con 0x7f204c15bd10 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.828+0000 7f204b400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f204c0cccf0 0x7f204c0ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51810/0 (socket says 172.21.15.62:51810) 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.828+0000 7f204b400640 1 -- 172.21.15.62:0/2662118490 learned_addr learned my addr 172.21.15.62:0/2662118490 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.828+0000 7f204be00640 1 --2- 172.21.15.62:0/2662118490 >> v2:172.21.15.121:3300/0 conn(0x7f204c15bd10 0x7f204c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.828+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568725783 0 0) 0x7f204c06ca70 con 0x7f204c0cccf0 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.828+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2030003520 con 0x7f204c0cccf0 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1351007379 0 0) 0x7f2030003520 con 0x7f204c0cccf0 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 >> v2:172.21.15.121:3300/0 conn(0x7f204c15bd10 msgr2=0x7f204c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 --2- 172.21.15.62:0/2662118490 >> v2:172.21.15.121:3300/0 conn(0x7f204c15bd10 0x7f204c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 >> v1:172.21.15.121:6789/0 conn(0x7f204c0aeed0 legacy=0x7f204c0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c06cd30 con 0x7f204c0cccf0 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f2038001670 con 0x7f204c0cccf0 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 >> v1:172.21.15.62:6789/0 conn(0x7f204c0cccf0 legacy=0x7f204c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20300039b0 0x7f2030003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f2030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20300051c0 0x7f203000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c06d660 con 0x7f2030004800 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203000c040 con 0x7f20300039b0 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203000c240 con 0x7f20300051c0 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204b400640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f2030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204be00640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20300039b0 0x7f2030003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204b400640 1 -- 172.21.15.62:0/2662118490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20300051c0 msgr2=0x7f203000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204b400640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20300051c0 0x7f203000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204b400640 1 -- 172.21.15.62:0/2662118490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20300039b0 msgr2=0x7f2030003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204b400640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20300039b0 0x7f2030003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.829+0000 7f204b400640 1 -- 172.21.15.62:0/2662118490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030003520 con 0x7f2030004800 2024-04-25T09:04:13.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.830+0000 7f2050e00640 1 -- 172.21.15.62:0/2662118490 reap_dead start 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.830+0000 7f204b400640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f2030004c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f203800c0e0 tx=0x7f2030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64b75edaf2c5d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038007c00 con 0x7f2030004800 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038005ad0 con 0x7f2030004800 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f204aa00640 1 -- 172.21.15.62:0/2662118490 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038007070 con 0x7f2030004800 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f2052e00640 1 -- 172.21.15.62:0/2662118490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 msgr2=0x7f2030004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f2052e00640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f2030004c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f203800c0e0 tx=0x7f2030005830 comp rx=0 tx=0).stop 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f2052e00640 1 -- 172.21.15.62:0/2662118490 shutdown_connections 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f2052e00640 1 --2- 172.21.15.62:0/2662118490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f2030004c80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f2052e00640 1 -- 172.21.15.62:0/2662118490 >> 172.21.15.62:0/2662118490 conn(0x7f204c0668f0 msgr2=0x7f204c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f2052e00640 1 -- 172.21.15.62:0/2662118490 shutdown_connections 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.831+0000 7f2052e00640 1 -- 172.21.15.62:0/2662118490 wait complete. 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.833+0000 7f2052e00640 1 Processor -- start 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.833+0000 7f2052e00640 1 -- start start 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f2052e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f204c0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f2052e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f204c0adb50 0x7f204c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f2052e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f204c0adfd0 0x7f204c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f2052e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c06daf0 con 0x7f2030004800 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f2052e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c06d2f0 con 0x7f204c0adfd0 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f2052e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c071280 con 0x7f204c0adb50 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f204be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f204c0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f204be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f204c0b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50888/0 (socket says 172.21.15.62:50888) 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.834+0000 7f204be00640 1 -- 172.21.15.62:0/672082416 learned_addr learned my addr 172.21.15.62:0/672082416 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f2050e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f204c0adfd0 0x7f204c156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f204be00640 1 -- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f204c0adb50 msgr2=0x7f204c1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f204be00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f204c0adb50 0x7f204c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f204be00640 1 -- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f204c0adfd0 msgr2=0x7f204c156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f204be00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f204c0adfd0 0x7f204c156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f204be00640 1 -- 172.21.15.62:0/672082416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c070670 con 0x7f2030004800 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f2050e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f204c0adfd0 0x7f204c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f204be00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f204c0b1a80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f203c0108e0 tx=0x7f203c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f2043e00640 1 -- 172.21.15.62:0/672082416 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c00f040 con 0x7f2030004800 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f2043e00640 1 -- 172.21.15.62:0/672082416 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c00eb60 con 0x7f2030004800 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c070950 con 0x7f2030004800 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.835+0000 7f2043e00640 1 -- 172.21.15.62:0/672082416 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c01c400 con 0x7f2030004800 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.836+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c0b1fc0 con 0x7f2030004800 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.837+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c070d40 con 0x7f2030004800 2024-04-25T09:04:13.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.838+0000 7f2043e00640 1 -- 172.21.15.62:0/672082416 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f203c01c5a0 con 0x7f2030004800 2024-04-25T09:04:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.839+0000 7f2043e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f202c06b500 0x7f202c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.839+0000 7f2043e00640 1 -- 172.21.15.62:0/672082416 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f203c08aaa0 con 0x7f2030004800 2024-04-25T09:04:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.840+0000 7f204b400640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f202c06b500 0x7f202c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.841+0000 7f204b400640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f202c06b500 0x7f202c06d9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2038032000 tx=0x7f2038007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:13.843+0000 7f2043e00640 1 -- 172.21.15.62:0/672082416 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f203c018030 con 0x7f2030004800 2024-04-25T09:04:14.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.079+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f204c0685c0 con 0x7f2030004800 2024-04-25T09:04:14.069 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:04:14.080+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:04:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.084+0000 7f2043e00640 1 -- 172.21.15.62:0/672082416 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f204c0685c0 con 0x7f2030004800 2024-04-25T09:04:14.073 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.086+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f202c06b500 msgr2=0x7f202c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.086+0000 7f2052e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f202c06b500 0x7f202c06d9c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2038032000 tx=0x7f2038007000 comp rx=0 tx=0).stop 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.086+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 msgr2=0x7f204c0b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.086+0000 7f2052e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f204c0b1a80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f203c0108e0 tx=0x7f203c010da0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.087+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 shutdown_connections 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.087+0000 7f2052e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f202c06b500 0x7f202c06d9c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.087+0000 7f2052e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f204c0adfd0 0x7f204c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.087+0000 7f2052e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f204c0adb50 0x7f204c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.087+0000 7f2052e00640 1 --2- 172.21.15.62:0/672082416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2030004800 0x7f204c0b1a80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.087+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 >> 172.21.15.62:0/672082416 conn(0x7f204c0668f0 msgr2=0x7f204c0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.087+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 shutdown_connections 2024-04-25T09:04:14.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.088+0000 7f2052e00640 1 -- 172.21.15.62:0/672082416 wait complete. 2024-04-25T09:04:14.096 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:04:14.114 INFO:tasks.ceph.mds.b.smithi121.stderr: -7826> 2024-04-25T09:03:19.017+0000 7f4266e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:14.203 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:04:14.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.315+0000 7f9894600640 1 Processor -- start 2024-04-25T09:04:14.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.315+0000 7f9894600640 1 -- start start 2024-04-25T09:04:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9894600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c15e050 0x7f988c06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9894600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f988c06e330 0x7f988c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9894600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988c06d7f0 con 0x7f988c15dc70 2024-04-25T09:04:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9894600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988c06d230 con 0x7f988c06e330 2024-04-25T09:04:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9894600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c06d510 con 0x7f988c15e050 2024-04-25T09:04:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9891c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f988c15dc70 0x7f988c06dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51824/0 (socket says 172.21.15.62:51824) 2024-04-25T09:04:14.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9891c00640 1 -- 172.21.15.62:0/2935608762 learned_addr learned my addr 172.21.15.62:0/2935608762 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:14.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9891200640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c15e050 0x7f988c06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.317+0000 7f9892600640 1 --2- 172.21.15.62:0/2935608762 >> v2:172.21.15.121:3300/0 conn(0x7f988c06e330 0x7f988c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546750434 0 0) 0x7f988c06d7f0 con 0x7f988c15dc70 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9874003520 con 0x7f988c15dc70 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203349040 0 0) 0x7f9874003520 con 0x7f988c15dc70 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c15e050 msgr2=0x7f988c06c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c15e050 0x7f988c06c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 >> v2:172.21.15.121:3300/0 conn(0x7f988c06e330 msgr2=0x7f988c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 --2- 172.21.15.62:0/2935608762 >> v2:172.21.15.121:3300/0 conn(0x7f988c06e330 0x7f988c06c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c0b27b0 con 0x7f988c15dc70 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f9880001670 con 0x7f988c15dc70 2024-04-25T09:04:14.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 >> v1:172.21.15.62:6789/0 conn(0x7f988c15dc70 legacy=0x7f988c06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98740039b0 0x7f9874003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c15e050 0x7f9874004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874004d40 0x7f98740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c06d230 con 0x7f98740039b0 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c06d510 con 0x7f9874004d40 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987400ba40 con 0x7f988c15e050 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f9892600640 1 --2- 172.21.15.62:0/2935608762 >> v2:172.21.15.121:3300/0 conn(0x7f988c06e330 0x7f988c06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.318+0000 7f9891c00640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98740039b0 0x7f9874003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.319+0000 7f9891200640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c15e050 0x7f9874004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.319+0000 7f9891c00640 1 -- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c15e050 msgr2=0x7f9874004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.319+0000 7f9891c00640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f988c15e050 0x7f9874004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.319+0000 7f9891c00640 1 -- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874004d40 msgr2=0x7f98740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.319+0000 7f9891c00640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9874004d40 0x7f98740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.319+0000 7f9891c00640 1 -- 172.21.15.62:0/2935608762 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9874003520 con 0x7f98740039b0 2024-04-25T09:04:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.320+0000 7f9891c00640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98740039b0 0x7f9874003dd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f988000c190 tx=0x7f9874005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89a31af4401f5c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.320+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880007c60 con 0x7f98740039b0 2024-04-25T09:04:14.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.320+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880005ad0 con 0x7f98740039b0 2024-04-25T09:04:14.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.320+0000 7f988be00640 1 -- 172.21.15.62:0/2935608762 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880007070 con 0x7f98740039b0 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.321+0000 7f9894600640 1 -- 172.21.15.62:0/2935608762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98740039b0 msgr2=0x7f9874003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.321+0000 7f9894600640 1 --2- 172.21.15.62:0/2935608762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98740039b0 0x7f9874003dd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f988000c190 tx=0x7f9874005830 comp rx=0 tx=0).stop 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.321+0000 7f9892600640 1 -- 172.21.15.62:0/2935608762 reap_dead start 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.321+0000 7f9894600640 1 -- 172.21.15.62:0/2935608762 shutdown_connections 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.321+0000 7f9894600640 1 -- 172.21.15.62:0/2935608762 >> 172.21.15.62:0/2935608762 conn(0x7f988c0b1410 msgr2=0x7f988c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.321+0000 7f9894600640 1 -- 172.21.15.62:0/2935608762 shutdown_connections 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.321+0000 7f9894600640 1 -- 172.21.15.62:0/2935608762 wait complete. 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.323+0000 7f9894600640 1 Processor -- start 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.323+0000 7f9894600640 1 -- start start 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9894600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98740039b0 0x7f988c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9894600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f988c160e70 0x7f988c147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9894600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f988c152ec0 0x7f988c153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9894600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c06cfb0 con 0x7f988c160e70 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9894600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c06ce30 con 0x7f988c152ec0 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9894600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988c06c3b0 con 0x7f98740039b0 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f988c160e70 0x7f988c147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f988c160e70 0x7f988c147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50906/0 (socket says 172.21.15.62:50906) 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891200640 1 -- 172.21.15.62:0/4018152595 learned_addr learned my addr 172.21.15.62:0/4018152595 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891c00640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98740039b0 0x7f988c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891200640 1 -- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98740039b0 msgr2=0x7f988c151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891200640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98740039b0 0x7f988c151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891200640 1 -- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f988c152ec0 msgr2=0x7f988c153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891200640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f988c152ec0 0x7f988c153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.324+0000 7f9891200640 1 -- 172.21.15.62:0/4018152595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c1506d0 con 0x7f988c160e70 2024-04-25T09:04:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.325+0000 7f9891200640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f988c160e70 0x7f988c147070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9878010860 tx=0x7f9878010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.325+0000 7f9889600640 1 -- 172.21.15.62:0/4018152595 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987800f040 con 0x7f988c160e70 2024-04-25T09:04:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.325+0000 7f9889600640 1 -- 172.21.15.62:0/4018152595 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987800eb60 con 0x7f988c160e70 2024-04-25T09:04:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.325+0000 7f9889600640 1 -- 172.21.15.62:0/4018152595 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987801c400 con 0x7f988c160e70 2024-04-25T09:04:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.325+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c151af0 con 0x7f988c160e70 2024-04-25T09:04:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.325+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c1520b0 con 0x7f988c160e70 2024-04-25T09:04:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.326+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c04ef20 con 0x7f988c160e70 2024-04-25T09:04:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.327+0000 7f9889600640 1 -- 172.21.15.62:0/4018152595 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f987801c5a0 con 0x7f988c160e70 2024-04-25T09:04:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.328+0000 7f9889600640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f986c06b4d0 0x7f986c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.328+0000 7f9889600640 1 -- 172.21.15.62:0/4018152595 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f987808a880 con 0x7f988c160e70 2024-04-25T09:04:14.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.329+0000 7f9891c00640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f986c06b4d0 0x7f986c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.330+0000 7f9891c00640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f986c06b4d0 0x7f986c06d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9880032000 tx=0x7f9880007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.333+0000 7f9889600640 1 -- 172.21.15.62:0/4018152595 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9878018030 con 0x7f988c160e70 2024-04-25T09:04:14.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:14.559+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f988c0ae180 con 0x7f988c160e70 2024-04-25T09:04:15.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.087+0000 7f9889600640 1 -- 172.21.15.62:0/4018152595 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9878058fc0 con 0x7f988c160e70 2024-04-25T09:04:15.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f986c06b4d0 msgr2=0x7f986c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f986c06b4d0 0x7f986c06d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9880032000 tx=0x7f9880007000 comp rx=0 tx=0).stop 2024-04-25T09:04:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f988c160e70 msgr2=0x7f988c147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f988c160e70 0x7f988c147070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9878010860 tx=0x7f9878010d20 comp rx=0 tx=0).stop 2024-04-25T09:04:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 shutdown_connections 2024-04-25T09:04:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f986c06b4d0 0x7f986c06d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f988c160e70 0x7f988c147070 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f988c152ec0 0x7f988c153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 --2- 172.21.15.62:0/4018152595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98740039b0 0x7f988c151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 >> 172.21.15.62:0/4018152595 conn(0x7f988c0b1410 msgr2=0x7f988c0ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 shutdown_connections 2024-04-25T09:04:15.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.091+0000 7f9894600640 1 -- 172.21.15.62:0/4018152595 wait complete. 2024-04-25T09:04:15.102 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:04:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.331+0000 7f2e7a200640 1 Processor -- start 2024-04-25T09:04:15.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.332+0000 7f2e7a200640 1 -- start start 2024-04-25T09:04:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.333+0000 7f2e7a200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 0x7f2e74070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.333+0000 7f2e7a200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e7415bc00 0x7f2e7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.333+0000 7f2e7a200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e74071600 con 0x7f2e7406d8d0 2024-04-25T09:04:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e7a200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e740718e0 con 0x7f2e7415bc00 2024-04-25T09:04:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e7a200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7415c170 con 0x7f2e740aede0 2024-04-25T09:04:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e72a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e7415bc00 0x7f2e7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e72a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e7415bc00 0x7f2e7406d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53598/0 (socket says 172.21.15.62:53598) 2024-04-25T09:04:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e72a00640 1 -- 172.21.15.62:0/2138104218 learned_addr learned my addr 172.21.15.62:0/2138104218 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e73e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2e7406d8d0 0x7f2e740b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60538/0 (socket says 172.21.15.62:60538) 2024-04-25T09:04:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e73400640 1 --2- 172.21.15.62:0/2138104218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 0x7f2e74070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e72a00640 1 -- 172.21.15.62:0/2138104218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 msgr2=0x7f2e74070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e72a00640 1 --2- 172.21.15.62:0/2138104218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 0x7f2e74070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e72a00640 1 -- 172.21.15.62:0/2138104218 >> v1:172.21.15.62:6789/0 conn(0x7f2e7406d8d0 legacy=0x7f2e740b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e72a00640 1 -- 172.21.15.62:0/2138104218 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7415c450 con 0x7f2e7415bc00 2024-04-25T09:04:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.334+0000 7f2e72000640 1 -- 172.21.15.62:0/2138104218 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778919361 0 0) 0x7f2e74071600 con 0x7f2e7406d8d0 2024-04-25T09:04:15.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.336+0000 7f2e72a00640 1 --2- 172.21.15.62:0/2138104218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e7406d390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7c3dbae383fdc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.336+0000 7f2e72000640 1 -- 172.21.15.62:0/2138104218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c037070 con 0x7f2e7415bc00 2024-04-25T09:04:15.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.336+0000 7f2e72000640 1 -- 172.21.15.62:0/2138104218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c032440 con 0x7f2e7415bc00 2024-04-25T09:04:15.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.336+0000 7f2e72000640 1 -- 172.21.15.62:0/2138104218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c032810 con 0x7f2e7415bc00 2024-04-25T09:04:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.337+0000 7f2e7a200640 1 -- 172.21.15.62:0/2138104218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 msgr2=0x7f2e7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.337+0000 7f2e7a200640 1 --2- 172.21.15.62:0/2138104218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e7406d390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c02a270 comp rx=0 tx=0).stop 2024-04-25T09:04:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.337+0000 7f2e7a200640 1 -- 172.21.15.62:0/2138104218 shutdown_connections 2024-04-25T09:04:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.337+0000 7f2e7a200640 1 --2- 172.21.15.62:0/2138104218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e7406d390 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.337+0000 7f2e7a200640 1 --2- 172.21.15.62:0/2138104218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 0x7f2e74070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.337+0000 7f2e7a200640 1 -- 172.21.15.62:0/2138104218 >> 172.21.15.62:0/2138104218 conn(0x7f2e740668f0 msgr2=0x7f2e74066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.337+0000 7f2e7a200640 1 -- 172.21.15.62:0/2138104218 shutdown_connections 2024-04-25T09:04:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.337+0000 7f2e7a200640 1 -- 172.21.15.62:0/2138104218 wait complete. 2024-04-25T09:04:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.339+0000 7f2e7a200640 1 Processor -- start 2024-04-25T09:04:15.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.339+0000 7f2e7a200640 1 -- start start 2024-04-25T09:04:15.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e7a200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e7406d8d0 0x7f2e74151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e7a200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 0x7f2e74156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e7a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e740b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e7a200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e74071280 con 0x7f2e7415bc00 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e7a200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406dd20 con 0x7f2e7406d8d0 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e7a200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740af1c0 con 0x7f2e740aede0 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e740b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e740b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53600/0 (socket says 172.21.15.62:53600) 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.340+0000 7f2e73e00640 1 -- 172.21.15.62:0/1423194978 learned_addr learned my addr 172.21.15.62:0/1423194978 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e73400640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e7406d8d0 0x7f2e74151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e72a00640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 0x7f2e74156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e73e00640 1 -- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 msgr2=0x7f2e74156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e73e00640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 0x7f2e74156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e73e00640 1 -- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e7406d8d0 msgr2=0x7f2e74151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e73e00640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e7406d8d0 0x7f2e74151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e73e00640 1 -- 172.21.15.62:0/1423194978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e74151a60 con 0x7f2e7415bc00 2024-04-25T09:04:15.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e73e00640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e740b1200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c480 tx=0x7f2e6c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.341+0000 7f2e6b400640 1 -- 172.21.15.62:0/1423194978 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c018070 con 0x7f2e7415bc00 2024-04-25T09:04:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.342+0000 7f2e6b400640 1 -- 172.21.15.62:0/1423194978 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c007500 con 0x7f2e7415bc00 2024-04-25T09:04:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.342+0000 7f2e6b400640 1 -- 172.21.15.62:0/1423194978 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c004a20 con 0x7f2e7415bc00 2024-04-25T09:04:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.342+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e741578f0 con 0x7f2e7415bc00 2024-04-25T09:04:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.342+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e74157eb0 con 0x7f2e7415bc00 2024-04-25T09:04:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.343+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e74067c40 con 0x7f2e7415bc00 2024-04-25T09:04:15.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.344+0000 7f2e6b400640 1 -- 172.21.15.62:0/1423194978 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e6c004bc0 con 0x7f2e7415bc00 2024-04-25T09:04:15.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.345+0000 7f2e6b400640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e5006b4d0 0x7f2e5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.345+0000 7f2e6b400640 1 -- 172.21.15.62:0/1423194978 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f2e6c089520 con 0x7f2e7415bc00 2024-04-25T09:04:15.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.345+0000 7f2e73400640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e5006b4d0 0x7f2e5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.347+0000 7f2e73400640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e5006b4d0 0x7f2e5006d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e741f50d0 tx=0x7f2e6400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.349+0000 7f2e6b400640 1 -- 172.21.15.62:0/1423194978 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e6c058340 con 0x7f2e7415bc00 2024-04-25T09:04:15.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:15.551+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2e741528f0 con 0x7f2e7415bc00 2024-04-25T09:04:16.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.091+0000 7f2e6b400640 1 -- 172.21.15.62:0/1423194978 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2e6c057c40 con 0x7f2e7415bc00 2024-04-25T09:04:16.081 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:04:16.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e5006b4d0 msgr2=0x7f2e5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e5006b4d0 0x7f2e5006d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e741f50d0 tx=0x7f2e6400b040 comp rx=0 tx=0).stop 2024-04-25T09:04:16.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 msgr2=0x7f2e740b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e740b1200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c480 tx=0x7f2e6c00c940 comp rx=0 tx=0).stop 2024-04-25T09:04:16.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 shutdown_connections 2024-04-25T09:04:16.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e5006b4d0 0x7f2e5006d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e7415bc00 0x7f2e740b1200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e740aede0 0x7f2e74156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 --2- 172.21.15.62:0/1423194978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e7406d8d0 0x7f2e74151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 >> 172.21.15.62:0/1423194978 conn(0x7f2e740668f0 msgr2=0x7f2e740af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.095+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 shutdown_connections 2024-04-25T09:04:16.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.096+0000 7f2e7a200640 1 -- 172.21.15.62:0/1423194978 wait complete. 2024-04-25T09:04:16.108 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:04:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.334+0000 7f5db8a00640 1 Processor -- start 2024-04-25T09:04:16.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.334+0000 7f5db8a00640 1 -- start start 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.335+0000 7f5db8a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5db415dd90 0x7f5db40d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.335+0000 7f5db8a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40ceb10 0x7f5db40b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db8a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40d3770 con 0x7f5db40d1c50 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db8a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db415e170 con 0x7f5db415dd90 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db8a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40d3a50 con 0x7f5db40ceb10 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5db415dd90 0x7f5db40d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5db415dd90 0x7f5db40d3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53610/0 (socket says 172.21.15.62:53610) 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db1e00640 1 -- 172.21.15.62:0/3625733108 learned_addr learned my addr 172.21.15.62:0/3625733108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:16.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db1400640 1 --1- 172.21.15.62:0/3625733108 >> v1:172.21.15.62:6789/0 conn(0x7f5db40d1c50 0x7f5db40ce400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60548/0 (socket says 172.21.15.62:60548) 2024-04-25T09:04:16.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db1e00640 1 -- 172.21.15.62:0/3625733108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40ceb10 msgr2=0x7f5db40b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:16.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db1e00640 1 --2- 172.21.15.62:0/3625733108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40ceb10 0x7f5db40b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db1e00640 1 -- 172.21.15.62:0/3625733108 >> v1:172.21.15.62:6789/0 conn(0x7f5db40d1c50 legacy=0x7f5db40ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.336+0000 7f5db1e00640 1 -- 172.21.15.62:0/3625733108 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40d3d30 con 0x7f5db415dd90 2024-04-25T09:04:16.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.338+0000 7f5db1e00640 1 --2- 172.21.15.62:0/3625733108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db40d3070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009e50 tx=0x7f5da402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d4651c5d232adc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.338+0000 7f5db0a00640 1 -- 172.21.15.62:0/3625733108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4036070 con 0x7f5db415dd90 2024-04-25T09:04:16.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.338+0000 7f5db0a00640 1 -- 172.21.15.62:0/3625733108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da4031600 con 0x7f5db415dd90 2024-04-25T09:04:16.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.338+0000 7f5db0a00640 1 -- 172.21.15.62:0/3625733108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da40319d0 con 0x7f5db415dd90 2024-04-25T09:04:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.339+0000 7f5db8a00640 1 -- 172.21.15.62:0/3625733108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 msgr2=0x7f5db40d3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.339+0000 7f5db8a00640 1 --2- 172.21.15.62:0/3625733108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db40d3070 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009e50 tx=0x7f5da402a710 comp rx=0 tx=0).stop 2024-04-25T09:04:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.339+0000 7f5db8a00640 1 -- 172.21.15.62:0/3625733108 shutdown_connections 2024-04-25T09:04:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.339+0000 7f5db8a00640 1 --2- 172.21.15.62:0/3625733108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40ceb10 0x7f5db40b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.339+0000 7f5db8a00640 1 --2- 172.21.15.62:0/3625733108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db40d3070 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.339+0000 7f5db8a00640 1 -- 172.21.15.62:0/3625733108 >> 172.21.15.62:0/3625733108 conn(0x7f5db406b000 msgr2=0x7f5db406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.339+0000 7f5db8a00640 1 -- 172.21.15.62:0/3625733108 shutdown_connections 2024-04-25T09:04:16.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.339+0000 7f5db8a00640 1 -- 172.21.15.62:0/3625733108 wait complete. 2024-04-25T09:04:16.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.341+0000 7f5db8a00640 1 Processor -- start 2024-04-25T09:04:16.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.342+0000 7f5db8a00640 1 -- start start 2024-04-25T09:04:16.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.343+0000 7f5db8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5db40ceb10 0x7f5db41f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.343+0000 7f5db8a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40d1c50 0x7f5db40cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.343+0000 7f5db8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db415e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.343+0000 7f5db8a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40cf1c0 con 0x7f5db415dd90 2024-04-25T09:04:16.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.343+0000 7f5db8a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40d42f0 con 0x7f5db40ceb10 2024-04-25T09:04:16.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.343+0000 7f5db8a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b7370 con 0x7f5db40d1c50 2024-04-25T09:04:16.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db415e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53624/0 (socket says 172.21.15.62:53624) 2024-04-25T09:04:16.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db2800640 1 -- 172.21.15.62:0/4287930513 learned_addr learned my addr 172.21.15.62:0/4287930513 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db1e00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5db40ceb10 0x7f5db41f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db1400640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40d1c50 0x7f5db40cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db2800640 1 -- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40d1c50 msgr2=0x7f5db40cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db2800640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40d1c50 0x7f5db40cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db2800640 1 -- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5db40ceb10 msgr2=0x7f5db41f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db2800640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5db40ceb10 0x7f5db41f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db2800640 1 -- 172.21.15.62:0/4287930513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40cda80 con 0x7f5db415dd90 2024-04-25T09:04:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.344+0000 7f5db1400640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40d1c50 0x7f5db40cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.345+0000 7f5db2800640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db415e300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009fd0 tx=0x7f5dac00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.345+0000 7f5daa000640 1 -- 172.21.15.62:0/4287930513 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac011830 con 0x7f5db415dd90 2024-04-25T09:04:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.345+0000 7f5daa000640 1 -- 172.21.15.62:0/4287930513 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac016070 con 0x7f5db415dd90 2024-04-25T09:04:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.345+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db40ccbd0 con 0x7f5db415dd90 2024-04-25T09:04:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.345+0000 7f5daa000640 1 -- 172.21.15.62:0/4287930513 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac011bd0 con 0x7f5db415dd90 2024-04-25T09:04:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.345+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db40cc610 con 0x7f5db415dd90 2024-04-25T09:04:16.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.347+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db4067c40 con 0x7f5db415dd90 2024-04-25T09:04:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.348+0000 7f5daa000640 1 -- 172.21.15.62:0/4287930513 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5dac01b460 con 0x7f5db415dd90 2024-04-25T09:04:16.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.349+0000 7f5daa000640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d9006b520 0x7f5d9006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.349+0000 7f5daa000640 1 -- 172.21.15.62:0/4287930513 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f5dac089db0 con 0x7f5db415dd90 2024-04-25T09:04:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.349+0000 7f5db1e00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d9006b520 0x7f5d9006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.351+0000 7f5db1e00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d9006b520 0x7f5d9006d9e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5da4033000 tx=0x7f5da4009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.354+0000 7f5daa000640 1 -- 172.21.15.62:0/4287930513 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5dac08e070 con 0x7f5db415dd90 2024-04-25T09:04:16.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:16.554+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5db415a730 con 0x7f5db415dd90 2024-04-25T09:04:17.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.095+0000 7f5daa000640 1 -- 172.21.15.62:0/4287930513 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5dac058de0 con 0x7f5db415dd90 2024-04-25T09:04:17.084 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:04:17.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d9006b520 msgr2=0x7f5d9006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d9006b520 0x7f5d9006d9e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5da4033000 tx=0x7f5da4009000 comp rx=0 tx=0).stop 2024-04-25T09:04:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 msgr2=0x7f5db415e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db415e300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009fd0 tx=0x7f5dac00b8e0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 shutdown_connections 2024-04-25T09:04:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d9006b520 0x7f5d9006d9e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db415dd90 0x7f5db415e300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5db40d1c50 0x7f5db40cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 --2- 172.21.15.62:0/4287930513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5db40ceb10 0x7f5db41f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 >> 172.21.15.62:0/4287930513 conn(0x7f5db406b000 msgr2=0x7f5db40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 shutdown_connections 2024-04-25T09:04:17.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.099+0000 7f5db8a00640 1 -- 172.21.15.62:0/4287930513 wait complete. 2024-04-25T09:04:17.118 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:04:17.118 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:04:17.118 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:04:17.119 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:17.119 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:04:17.119 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:04:17.119 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:04:17.119 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:17.119 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:04:17.119 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:04:17.120 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:04:17.120 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:17.120 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:04:17.120 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:04:17.120 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:04:17.121 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:17.138 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:04:17.138 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:04:17.140 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:04:17.140 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:04:17.141 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:04:17.141 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:04:17.142 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:04:17.142 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:04:17.187 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:04:17.187 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:17.190 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:04:17.190 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:17.194 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:04:17.194 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:17.196 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:04:17.196 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:17.197 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:04:17.340 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:04:17.341 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:04:17.344 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:04:17.344 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:04:17.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.390+0000 7ff7df200640 1 Processor -- start 2024-04-25T09:04:17.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.391+0000 7ff7df200640 1 -- start start 2024-04-25T09:04:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.392+0000 7ff7df200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e0161230 0x7ff7e00b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.392+0000 7ff7df200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e00cea70 con 0x7ff7e00b5c30 2024-04-25T09:04:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.392+0000 7ff7df200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e00ced50 con 0x7ff7e00d0fb0 2024-04-25T09:04:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.392+0000 7ff7df200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e00d2040 con 0x7ff7e0161230 2024-04-25T09:04:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.392+0000 7ff7de800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff7e00b5c30 0x7ff7e0161a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60566/0 (socket says 172.21.15.62:60566) 2024-04-25T09:04:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.392+0000 7ff7de800640 1 -- 172.21.15.62:0/2640903589 learned_addr learned my addr 172.21.15.62:0/2640903589 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.392+0000 7ff7dde00640 1 --2- 172.21.15.62:0/2640903589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e0161230 0x7ff7e00b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.393+0000 7ff7dd400640 1 -- 172.21.15.62:0/2640903589 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702645731 0 0) 0x7ff7e00cea70 con 0x7ff7e00b5c30 2024-04-25T09:04:17.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.393+0000 7ff7dd400640 1 -- 172.21.15.62:0/2640903589 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c4003520 con 0x7ff7e00b5c30 2024-04-25T09:04:17.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.393+0000 7ff7dd400640 1 -- 172.21.15.62:0/2640903589 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081536905 0 0) 0x7ff7e00ced50 con 0x7ff7e00d0fb0 2024-04-25T09:04:17.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.393+0000 7ff7dd400640 1 -- 172.21.15.62:0/2640903589 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e00cea70 con 0x7ff7e00d0fb0 2024-04-25T09:04:17.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.393+0000 7ff7dde00640 1 -- 172.21.15.62:0/2640903589 >> v1:172.21.15.121:6789/0 conn(0x7ff7e00d0fb0 legacy=0x7ff7e00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.393+0000 7ff7dde00640 1 -- 172.21.15.62:0/2640903589 >> v1:172.21.15.62:6789/0 conn(0x7ff7e00b5c30 legacy=0x7ff7e0161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.393+0000 7ff7dde00640 1 -- 172.21.15.62:0/2640903589 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00cf030 con 0x7ff7e0161230 2024-04-25T09:04:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.394+0000 7ff7dde00640 1 --2- 172.21.15.62:0/2640903589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e0161230 0x7ff7e00b6c40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00097d0 tx=0x7ff7d002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2e4e260362d526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.394+0000 7ff7dd400640 1 -- 172.21.15.62:0/2640903589 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0037070 con 0x7ff7e0161230 2024-04-25T09:04:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.394+0000 7ff7dd400640 1 -- 172.21.15.62:0/2640903589 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d0032440 con 0x7ff7e0161230 2024-04-25T09:04:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.394+0000 7ff7dd400640 1 -- 172.21.15.62:0/2640903589 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0032810 con 0x7ff7e0161230 2024-04-25T09:04:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.395+0000 7ff7df200640 1 -- 172.21.15.62:0/2640903589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e0161230 msgr2=0x7ff7e00b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.395+0000 7ff7df200640 1 --2- 172.21.15.62:0/2640903589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e0161230 0x7ff7e00b6c40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00097d0 tx=0x7ff7d002a210 comp rx=0 tx=0).stop 2024-04-25T09:04:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.395+0000 7ff7df200640 1 -- 172.21.15.62:0/2640903589 shutdown_connections 2024-04-25T09:04:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.395+0000 7ff7df200640 1 --2- 172.21.15.62:0/2640903589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e0161230 0x7ff7e00b6c40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.395+0000 7ff7df200640 1 -- 172.21.15.62:0/2640903589 >> 172.21.15.62:0/2640903589 conn(0x7ff7e0066d10 msgr2=0x7ff7e0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.395+0000 7ff7df200640 1 -- 172.21.15.62:0/2640903589 shutdown_connections 2024-04-25T09:04:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.395+0000 7ff7df200640 1 -- 172.21.15.62:0/2640903589 wait complete. 2024-04-25T09:04:17.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.397+0000 7ff7df200640 1 Processor -- start 2024-04-25T09:04:17.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.397+0000 7ff7df200640 1 -- start start 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7df200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7e00b5c30 0x7ff7e00ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7df200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e00d0fb0 0x7ff7e00cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7df200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7e0161230 0x7ff7e014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7df200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e0161610 con 0x7ff7e0161230 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7df200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e00ca8d0 con 0x7ff7e00b5c30 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7df200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e0161790 con 0x7ff7e00d0fb0 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7de800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7e0161230 0x7ff7e014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7de800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7e0161230 0x7ff7e014cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53676/0 (socket says 172.21.15.62:53676) 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7de800640 1 -- 172.21.15.62:0/2020150898 learned_addr learned my addr 172.21.15.62:0/2020150898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7dde00640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7e00b5c30 0x7ff7e00ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.398+0000 7ff7d7e00640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e00d0fb0 0x7ff7e00cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7de800640 1 -- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e00d0fb0 msgr2=0x7ff7e00cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7de800640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e00d0fb0 0x7ff7e00cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7de800640 1 -- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7e00b5c30 msgr2=0x7ff7e00ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7de800640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7e00b5c30 0x7ff7e00ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7de800640 1 -- 172.21.15.62:0/2020150898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00cbee0 con 0x7ff7e0161230 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7de800640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7e0161230 0x7ff7e014cda0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800cba0 tx=0x7ff7d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7d6000640 1 -- 172.21.15.62:0/2020150898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8017070 con 0x7ff7e0161230 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7d6000640 1 -- 172.21.15.62:0/2020150898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d800ace0 con 0x7ff7e0161230 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.399+0000 7ff7d6000640 1 -- 172.21.15.62:0/2020150898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8012620 con 0x7ff7e0161230 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.400+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e00cc1c0 con 0x7ff7e0161230 2024-04-25T09:04:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.400+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e00cc780 con 0x7ff7e0161230 2024-04-25T09:04:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.401+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e004ef20 con 0x7ff7e0161230 2024-04-25T09:04:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.402+0000 7ff7d6000640 1 -- 172.21.15.62:0/2020150898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7d801a070 con 0x7ff7e0161230 2024-04-25T09:04:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.403+0000 7ff7d6000640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff7bc06b500 0x7ff7bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.403+0000 7ff7d6000640 1 -- 172.21.15.62:0/2020150898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff7d8089db0 con 0x7ff7e0161230 2024-04-25T09:04:17.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.403+0000 7ff7dde00640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff7bc06b500 0x7ff7bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.404+0000 7ff7dde00640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff7bc06b500 0x7ff7bc06d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0034000 tx=0x7ff7d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.407+0000 7ff7d6000640 1 -- 172.21.15.62:0/2020150898 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7d808e070 con 0x7ff7e0161230 2024-04-25T09:04:17.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:17.632+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff7e00ce640 con 0x7ff7e0161230 2024-04-25T09:04:18.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.135+0000 7ff7d6000640 1 -- 172.21.15.62:0/2020150898 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff7d80587b0 con 0x7ff7e0161230 2024-04-25T09:04:18.124 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:04:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff7bc06b500 msgr2=0x7ff7bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff7bc06b500 0x7ff7bc06d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0034000 tx=0x7ff7d002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7e0161230 msgr2=0x7ff7e014cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7e0161230 0x7ff7e014cda0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800cba0 tx=0x7ff7d800a430 comp rx=0 tx=0).stop 2024-04-25T09:04:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 shutdown_connections 2024-04-25T09:04:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff7bc06b500 0x7ff7bc06d9c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7e0161230 0x7ff7e014cda0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff7e00d0fb0 0x7ff7e00cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 --2- 172.21.15.62:0/2020150898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7e00b5c30 0x7ff7e00ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.138+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 >> 172.21.15.62:0/2020150898 conn(0x7ff7e0066d10 msgr2=0x7ff7e00b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.139+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 shutdown_connections 2024-04-25T09:04:18.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.139+0000 7ff7df200640 1 -- 172.21.15.62:0/2020150898 wait complete. 2024-04-25T09:04:18.150 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:04:18.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.366+0000 7f21d7200640 1 Processor -- start 2024-04-25T09:04:18.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.366+0000 7f21d7200640 1 -- start start 2024-04-25T09:04:18.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.368+0000 7f21d7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21d815b960 0x7f21d806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.368+0000 7f21d7200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f21d80b47e0 0x7f21d80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.368+0000 7f21d7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d806d4f0 con 0x7f21d80ae9d0 2024-04-25T09:04:18.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.368+0000 7f21d7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d815bd40 con 0x7f21d815b960 2024-04-25T09:04:18.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.368+0000 7f21d7200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80b3110 con 0x7f21d80b47e0 2024-04-25T09:04:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.368+0000 7f21d5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21d815b960 0x7f21d806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.368+0000 7f21d5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21d815b960 0x7f21d806ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53686/0 (socket says 172.21.15.62:53686) 2024-04-25T09:04:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.368+0000 7f21d5e00640 1 -- 172.21.15.62:0/1565170507 learned_addr learned my addr 172.21.15.62:0/1565170507 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.369+0000 7f21d5e00640 1 -- 172.21.15.62:0/1565170507 >> v2:172.21.15.121:3300/0 conn(0x7f21d80b47e0 msgr2=0x7f21d80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.369+0000 7f21d5e00640 1 --2- 172.21.15.62:0/1565170507 >> v2:172.21.15.121:3300/0 conn(0x7f21d80b47e0 0x7f21d80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.369+0000 7f21d5e00640 1 -- 172.21.15.62:0/1565170507 >> v1:172.21.15.121:6789/0 conn(0x7f21d80ae9d0 legacy=0x7f21d80b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.369+0000 7f21d5e00640 1 -- 172.21.15.62:0/1565170507 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80b2500 con 0x7f21d815b960 2024-04-25T09:04:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.370+0000 7f21d5e00640 1 --2- 172.21.15.62:0/1565170507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d815b960 0x7f21d806ca80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f21d80ae840 tx=0x7f21c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5707b52199ebe00d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.370+0000 7f21d4a00640 1 -- 172.21.15.62:0/1565170507 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8037070 con 0x7f21d815b960 2024-04-25T09:04:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.370+0000 7f21d4a00640 1 -- 172.21.15.62:0/1565170507 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c8032440 con 0x7f21d815b960 2024-04-25T09:04:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.370+0000 7f21d4a00640 1 -- 172.21.15.62:0/1565170507 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8032810 con 0x7f21d815b960 2024-04-25T09:04:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.371+0000 7f21d7200640 1 -- 172.21.15.62:0/1565170507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d815b960 msgr2=0x7f21d806ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.371+0000 7f21d7200640 1 --2- 172.21.15.62:0/1565170507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d815b960 0x7f21d806ca80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f21d80ae840 tx=0x7f21c802a230 comp rx=0 tx=0).stop 2024-04-25T09:04:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.371+0000 7f21d7200640 1 -- 172.21.15.62:0/1565170507 shutdown_connections 2024-04-25T09:04:18.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.371+0000 7f21d7200640 1 --2- 172.21.15.62:0/1565170507 >> v2:172.21.15.121:3300/0 conn(0x7f21d80b47e0 0x7f21d80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.371+0000 7f21d7200640 1 --2- 172.21.15.62:0/1565170507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d815b960 0x7f21d806ca80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.371+0000 7f21d7200640 1 -- 172.21.15.62:0/1565170507 >> 172.21.15.62:0/1565170507 conn(0x7f21d8074380 msgr2=0x7f21d80668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.371+0000 7f21d7200640 1 -- 172.21.15.62:0/1565170507 shutdown_connections 2024-04-25T09:04:18.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.371+0000 7f21d7200640 1 -- 172.21.15.62:0/1565170507 wait complete. 2024-04-25T09:04:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.374+0000 7f21d7200640 1 Processor -- start 2024-04-25T09:04:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.374+0000 7f21d7200640 1 -- start start 2024-04-25T09:04:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.375+0000 7f21d7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80ae9d0 0x7f21d815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.375+0000 7f21d7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d80b47e0 0x7f21d8159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.375+0000 7f21d7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d815b960 0x7f21d8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.375+0000 7f21d7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80b2da0 con 0x7f21d80ae9d0 2024-04-25T09:04:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.375+0000 7f21d7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80b2ac0 con 0x7f21d80b47e0 2024-04-25T09:04:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.375+0000 7f21d7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cdab0 con 0x7f21d815b960 2024-04-25T09:04:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80ae9d0 0x7f21d815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80ae9d0 0x7f21d815d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53688/0 (socket says 172.21.15.62:53688) 2024-04-25T09:04:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5e00640 1 -- 172.21.15.62:0/1403217357 learned_addr learned my addr 172.21.15.62:0/1403217357 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d6800640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d815b960 0x7f21d8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5400640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d80b47e0 0x7f21d8159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5e00640 1 -- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d815b960 msgr2=0x7f21d8154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5e00640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d815b960 0x7f21d8154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5e00640 1 -- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d80b47e0 msgr2=0x7f21d8159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5e00640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d80b47e0 0x7f21d8159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.376+0000 7f21d5e00640 1 -- 172.21.15.62:0/1403217357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8158bd0 con 0x7f21d80ae9d0 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.377+0000 7f21d5e00640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80ae9d0 0x7f21d815d2a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f21c802a270 tx=0x7f21c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.377+0000 7f21ce000640 1 -- 172.21.15.62:0/1403217357 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8041070 con 0x7f21d80ae9d0 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.377+0000 7f21ce000640 1 -- 172.21.15.62:0/1403217357 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c803c870 con 0x7f21d80ae9d0 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.377+0000 7f21ce000640 1 -- 172.21.15.62:0/1403217357 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8037040 con 0x7f21d80ae9d0 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.377+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d815d7e0 con 0x7f21d80ae9d0 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.377+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d8159130 con 0x7f21d80ae9d0 2024-04-25T09:04:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.378+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d8075070 con 0x7f21d80ae9d0 2024-04-25T09:04:18.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.379+0000 7f21ce000640 1 -- 172.21.15.62:0/1403217357 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21c8044070 con 0x7f21d80ae9d0 2024-04-25T09:04:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.380+0000 7f21ce000640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21b806b4d0 0x7f21b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.380+0000 7f21ce000640 1 -- 172.21.15.62:0/1403217357 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f21c80aaa90 con 0x7f21d80ae9d0 2024-04-25T09:04:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.381+0000 7f21d5400640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21b806b4d0 0x7f21b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.382+0000 7f21d5400640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21b806b4d0 0x7f21b806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21c4007fd0 tx=0x7f21c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.384+0000 7f21ce000640 1 -- 172.21.15.62:0/1403217357 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21d8158bd0 con 0x7f21d80ae9d0 2024-04-25T09:04:18.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.653+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f21d80b3eb0 con 0x7f21d80ae9d0 2024-04-25T09:04:18.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.655+0000 7f21ce000640 1 -- 172.21.15.62:0/1403217357 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f21c8079d70 con 0x7f21d80ae9d0 2024-04-25T09:04:18.643 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.644 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.646 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:18.647 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.648 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.649 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:04:18.650 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.651 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:04:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.653 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:04:18.654 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-mds", 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQA8HCpmhYtCKhAAsgwaoqXnNedi8fYTrQ+hbA==", 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.655 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQA8HCpmJqtCKhAArNcvR3rOatPdqUhEIKU0Uw==", 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-osd", 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQA8HCpmTsRCKhAAd4nKZXALrZhg+GwQH5nvDg==", 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T09:04:18.656 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQA8HCpmieFCKhAAtCLghRmpA2UYCVB1Nl/hHg==", 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQA8HCpmQPpCKhAAoUAIAWtpM3uJe1hJW2sSDw==", 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQA8HCpmfRRDKhAAUOXIcHA5bII4ZnNQsve8cA==", 2024-04-25T09:04:18.657 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:04:18.658 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:04:18.659 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:04:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21b806b4d0 msgr2=0x7f21b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21b806b4d0 0x7f21b806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21c4007fd0 tx=0x7f21c4009040 comp rx=0 tx=0).stop 2024-04-25T09:04:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80ae9d0 msgr2=0x7f21d815d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80ae9d0 0x7f21d815d2a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f21c802a270 tx=0x7f21c80046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 shutdown_connections 2024-04-25T09:04:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21b806b4d0 0x7f21b806d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21d815b960 0x7f21d8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21d80b47e0 0x7f21d8159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 --2- 172.21.15.62:0/1403217357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21d80ae9d0 0x7f21d815d2a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 >> 172.21.15.62:0/1403217357 conn(0x7f21d8074380 msgr2=0x7f21d80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 shutdown_connections 2024-04-25T09:04:18.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.657+0000 7f21d7200640 1 -- 172.21.15.62:0/1403217357 wait complete. 2024-04-25T09:04:18.665 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T09:04:18.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.889+0000 7fb161600640 1 Processor -- start 2024-04-25T09:04:18.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.890+0000 7fb161600640 1 -- start start 2024-04-25T09:04:18.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.891+0000 7fb161600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb15c0b96a0 0x7fb15c0d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.891+0000 7fb161600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c15e240 con 0x7fb15c15de60 2024-04-25T09:04:18.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.891+0000 7fb161600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c0b9ab0 con 0x7fb15c0d41f0 2024-04-25T09:04:18.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.891+0000 7fb161600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15c0d2390 con 0x7fb15c0b96a0 2024-04-25T09:04:18.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.891+0000 7fb15aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb15c15de60 0x7fb15c0d3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60574/0 (socket says 172.21.15.62:60574) 2024-04-25T09:04:18.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.891+0000 7fb15aa00640 1 -- 172.21.15.62:0/2186608460 learned_addr learned my addr 172.21.15.62:0/2186608460 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb15b400640 1 --2- 172.21.15.62:0/2186608460 >> v2:172.21.15.121:3300/0 conn(0x7fb15c0b96a0 0x7fb15c0d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817839213 0 0) 0x7fb15c15e240 con 0x7fb15c15de60 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb13c003520 con 0x7fb15c15de60 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096805745 0 0) 0x7fb13c003520 con 0x7fb15c15de60 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 >> v2:172.21.15.121:3300/0 conn(0x7fb15c0b96a0 msgr2=0x7fb15c0d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 --2- 172.21.15.62:0/2186608460 >> v2:172.21.15.121:3300/0 conn(0x7fb15c0b96a0 0x7fb15c0d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 >> v1:172.21.15.121:6789/0 conn(0x7fb15c0d41f0 legacy=0x7fb15c0b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c0d2670 con 0x7fb15c15de60 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb14c001670 con 0x7fb15c15de60 2024-04-25T09:04:18.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb15b400640 1 --2- 172.21.15.62:0/2186608460 >> v2:172.21.15.121:3300/0 conn(0x7fb15c0b96a0 0x7fb15c0d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 >> v1:172.21.15.62:6789/0 conn(0x7fb15c15de60 legacy=0x7fb15c0d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb13c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.892+0000 7fb159600640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb13c004800 0x7fb13c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb159600640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb13c005210 0x7fb13c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0d2390 con 0x7fb13c003a00 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c00c160 con 0x7fb13c004800 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c00c360 con 0x7fb13c005210 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb15aa00640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb13c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb15a000640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb13c004800 0x7fb13c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb15b400640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb13c005210 0x7fb13c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb15aa00640 1 -- 172.21.15.62:0/2186608460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb13c005210 msgr2=0x7fb13c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb15aa00640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb13c005210 0x7fb13c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb15aa00640 1 -- 172.21.15.62:0/2186608460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb13c004800 msgr2=0x7fb13c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb15aa00640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb13c004800 0x7fb13c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.894+0000 7fb15b400640 1 -- 172.21.15.62:0/2186608460 reap_dead start 2024-04-25T09:04:18.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.893+0000 7fb15aa00640 1 -- 172.21.15.62:0/2186608460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c003520 con 0x7fb13c003a00 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.895+0000 7fb15aa00640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb13c003e20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c0e0 tx=0x7fb13c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c189ff9b936444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.895+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c007c20 con 0x7fb13c003a00 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.895+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c005af0 con 0x7fb13c003a00 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.895+0000 7fb159600640 1 -- 172.21.15.62:0/2186608460 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c007070 con 0x7fb13c003a00 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.896+0000 7fb161600640 1 -- 172.21.15.62:0/2186608460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 msgr2=0x7fb13c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.896+0000 7fb161600640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb13c003e20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c0e0 tx=0x7fb13c005830 comp rx=0 tx=0).stop 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.896+0000 7fb161600640 1 -- 172.21.15.62:0/2186608460 shutdown_connections 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.896+0000 7fb161600640 1 --2- 172.21.15.62:0/2186608460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb13c003e20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.896+0000 7fb161600640 1 -- 172.21.15.62:0/2186608460 >> 172.21.15.62:0/2186608460 conn(0x7fb15c06b000 msgr2=0x7fb15c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.896+0000 7fb161600640 1 -- 172.21.15.62:0/2186608460 shutdown_connections 2024-04-25T09:04:18.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.896+0000 7fb161600640 1 -- 172.21.15.62:0/2186608460 wait complete. 2024-04-25T09:04:18.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.898+0000 7fb161600640 1 Processor -- start 2024-04-25T09:04:18.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.898+0000 7fb161600640 1 -- start start 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb161600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb15c1601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb161600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb15c149e00 0x7fb15c1f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb161600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb15c1f7220 0x7fb15c1f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb161600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0d2f10 con 0x7fb13c003a00 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb161600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0d2c30 con 0x7fb15c1f7220 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb161600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0d3430 con 0x7fb15c149e00 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb15aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb15c1601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb15aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb15c1601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53696/0 (socket says 172.21.15.62:53696) 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.899+0000 7fb15aa00640 1 -- 172.21.15.62:0/1010913441 learned_addr learned my addr 172.21.15.62:0/1010913441 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15b400640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb15c1f7220 0x7fb15c1f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15a000640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb15c149e00 0x7fb15c1f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15aa00640 1 -- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb15c149e00 msgr2=0x7fb15c1f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15aa00640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb15c149e00 0x7fb15c1f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15aa00640 1 -- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb15c1f7220 msgr2=0x7fb15c1f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15aa00640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb15c1f7220 0x7fb15c1f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15aa00640 1 -- 172.21.15.62:0/1010913441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c160880 con 0x7fb13c003a00 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15b400640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb15c1f7220 0x7fb15c1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb15aa00640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb15c1601c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c200 tx=0x7fb15c0d2840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb152a00640 1 -- 172.21.15.62:0/1010913441 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c041070 con 0x7fb13c003a00 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb152a00640 1 -- 172.21.15.62:0/1010913441 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c0047e0 con 0x7fb13c003a00 2024-04-25T09:04:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb152a00640 1 -- 172.21.15.62:0/1010913441 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c037040 con 0x7fb13c003a00 2024-04-25T09:04:18.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.900+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c160b60 con 0x7fb13c003a00 2024-04-25T09:04:18.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.901+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c161120 con 0x7fb13c003a00 2024-04-25T09:04:18.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.902+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c04ef20 con 0x7fb13c003a00 2024-04-25T09:04:18.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.903+0000 7fb152a00640 1 -- 172.21.15.62:0/1010913441 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb14c044070 con 0x7fb13c003a00 2024-04-25T09:04:18.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.904+0000 7fb152a00640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb13806b4d0 0x7fb13806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.904+0000 7fb152a00640 1 -- 172.21.15.62:0/1010913441 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb14c0a92a0 con 0x7fb13c003a00 2024-04-25T09:04:18.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.904+0000 7fb15a000640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb13806b4d0 0x7fb13806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.905+0000 7fb15a000640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb13806b4d0 0x7fb13806d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb144000f80 tx=0x7fb144009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:18.908+0000 7fb152a00640 1 -- 172.21.15.62:0/1010913441 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb14c077d80 con 0x7fb13c003a00 2024-04-25T09:04:19.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.191+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fb15c0c9160 con 0x7fb13c003a00 2024-04-25T09:04:19.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.196+0000 7fb152a00640 1 -- 172.21.15.62:0/1010913441 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb14c07be20 con 0x7fb13c003a00 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb13806b4d0 msgr2=0x7fb13806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb13806b4d0 0x7fb13806d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb144000f80 tx=0x7fb144009040 comp rx=0 tx=0).stop 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 msgr2=0x7fb15c1601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb15c1601c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c200 tx=0x7fb15c0d2840 comp rx=0 tx=0).stop 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 shutdown_connections 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb13806b4d0 0x7fb13806d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb15c1f7220 0x7fb15c1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb15c149e00 0x7fb15c1f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 --2- 172.21.15.62:0/1010913441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb13c003a00 0x7fb15c1601c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 >> 172.21.15.62:0/1010913441 conn(0x7fb15c06b000 msgr2=0x7fb15c15a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 shutdown_connections 2024-04-25T09:04:19.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.200+0000 7fb161600640 1 -- 172.21.15.62:0/1010913441 wait complete. 2024-04-25T09:04:19.208 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T09:04:19.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.423+0000 7f68fee00640 1 Processor -- start 2024-04-25T09:04:19.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.423+0000 7f68fee00640 1 -- start start 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.425+0000 7f68fee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f80b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.425+0000 7f68fee00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f68f80ccbe0 0x7f68f80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.425+0000 7f68fee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f80b34b0 con 0x7f68f81454c0 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.425+0000 7f68fee00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f8160530 con 0x7f68f80ccbe0 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.425+0000 7f68fee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80cd590 con 0x7f68f80b44e0 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.425+0000 7f68efe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f80b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.426+0000 7f68efe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f80b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:57574/0 (socket says 172.21.15.62:57574) 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.426+0000 7f68efe00640 1 -- 172.21.15.62:0/1167047539 learned_addr learned my addr 172.21.15.62:0/1167047539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.426+0000 7f68fce00640 1 --2- 172.21.15.62:0/1167047539 >> v2:172.21.15.121:3300/0 conn(0x7f68f80ccbe0 0x7f68f80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.426+0000 7f68efe00640 1 -- 172.21.15.62:0/1167047539 >> v2:172.21.15.121:3300/0 conn(0x7f68f80ccbe0 msgr2=0x7f68f80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.426+0000 7f68efe00640 1 --2- 172.21.15.62:0/1167047539 >> v2:172.21.15.121:3300/0 conn(0x7f68f80ccbe0 0x7f68f80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.426+0000 7f68efe00640 1 -- 172.21.15.62:0/1167047539 >> v1:172.21.15.121:6789/0 conn(0x7f68f81454c0 legacy=0x7f68f806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.426+0000 7f68efe00640 1 -- 172.21.15.62:0/1167047539 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80b3720 con 0x7f68f80b44e0 2024-04-25T09:04:19.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.428+0000 7f68efe00640 1 --2- 172.21.15.62:0/1167047539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f80b4940 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009870 tx=0x7f68e002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ded947d19aa39aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.428+0000 7f68f7400640 1 -- 172.21.15.62:0/1167047539 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0037070 con 0x7f68f80b44e0 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.428+0000 7f68f7400640 1 -- 172.21.15.62:0/1167047539 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e0032440 con 0x7f68f80b44e0 2024-04-25T09:04:19.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.428+0000 7f68f7400640 1 -- 172.21.15.62:0/1167047539 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0032810 con 0x7f68f80b44e0 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.429+0000 7f68fee00640 1 -- 172.21.15.62:0/1167047539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 msgr2=0x7f68f80b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.429+0000 7f68fee00640 1 --2- 172.21.15.62:0/1167047539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f80b4940 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009870 tx=0x7f68e002a250 comp rx=0 tx=0).stop 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.429+0000 7f68fee00640 1 -- 172.21.15.62:0/1167047539 shutdown_connections 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.429+0000 7f68fee00640 1 --2- 172.21.15.62:0/1167047539 >> v2:172.21.15.121:3300/0 conn(0x7f68f80ccbe0 0x7f68f80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.429+0000 7f68fee00640 1 --2- 172.21.15.62:0/1167047539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f80b4940 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.429+0000 7f68fee00640 1 -- 172.21.15.62:0/1167047539 >> 172.21.15.62:0/1167047539 conn(0x7f68f80668f0 msgr2=0x7f68f8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.429+0000 7f68fee00640 1 -- 172.21.15.62:0/1167047539 shutdown_connections 2024-04-25T09:04:19.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.429+0000 7f68fee00640 1 -- 172.21.15.62:0/1167047539 wait complete. 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.432+0000 7f68fee00640 1 Processor -- start 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.432+0000 7f68fee00640 1 -- start start 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.433+0000 7f68fee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f813c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.433+0000 7f68fee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68f80ccbe0 0x7f68f815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.433+0000 7f68fee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f81454c0 0x7f68f80b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.433+0000 7f68fee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80b3fc0 con 0x7f68f81454c0 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.433+0000 7f68fee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80b3ce0 con 0x7f68f80ccbe0 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.433+0000 7f68fee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f806ca80 con 0x7f68f80b44e0 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f81454c0 0x7f68f80b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f81454c0 0x7f68f80b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53706/0 (socket says 172.21.15.62:53706) 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 -- 172.21.15.62:0/835233218 learned_addr learned my addr 172.21.15.62:0/835233218 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68f7e00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f813c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68efe00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68f80ccbe0 0x7f68f815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 -- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 msgr2=0x7f68f813c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f813c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 -- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68f80ccbe0 msgr2=0x7f68f815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68f80ccbe0 0x7f68f815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 -- 172.21.15.62:0/835233218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80709e0 con 0x7f68f81454c0 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.434+0000 7f68fce00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f81454c0 0x7f68f80b16c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f68f000efc0 tx=0x7f68f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.435+0000 7f68f4c00640 1 -- 172.21.15.62:0/835233218 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0018020 con 0x7f68f81454c0 2024-04-25T09:04:19.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.435+0000 7f68f4c00640 1 -- 172.21.15.62:0/835233218 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f0014480 con 0x7f68f81454c0 2024-04-25T09:04:19.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.435+0000 7f68f4c00640 1 -- 172.21.15.62:0/835233218 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0014870 con 0x7f68f81454c0 2024-04-25T09:04:19.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.435+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f813a2c0 con 0x7f68f81454c0 2024-04-25T09:04:19.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.435+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f8070c80 con 0x7f68f81454c0 2024-04-25T09:04:19.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.436+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f804ef20 con 0x7f68f81454c0 2024-04-25T09:04:19.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.437+0000 7f68f4c00640 1 -- 172.21.15.62:0/835233218 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68f0005d10 con 0x7f68f81454c0 2024-04-25T09:04:19.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.438+0000 7f68f4c00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f68d406b500 0x7f68d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.439+0000 7f68f4c00640 1 -- 172.21.15.62:0/835233218 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f68f008aa20 con 0x7f68f81454c0 2024-04-25T09:04:19.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.439+0000 7f68f7e00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f68d406b500 0x7f68d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.440+0000 7f68f7e00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f68d406b500 0x7f68d406d9c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68f80aec10 tx=0x7f68e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.444+0000 7f68f4c00640 1 -- 172.21.15.62:0/835233218 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68f0022070 con 0x7f68f81454c0 2024-04-25T09:04:19.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.717+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f68f806b2f0 con 0x7f68f81454c0 2024-04-25T09:04:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.722+0000 7f68f4c00640 1 -- 172.21.15.62:0/835233218 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f68f0010040 con 0x7f68f81454c0 2024-04-25T09:04:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.725+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f68d406b500 msgr2=0x7f68d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.725+0000 7f68fee00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f68d406b500 0x7f68d406d9c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f68f80aec10 tx=0x7f68e8009040 comp rx=0 tx=0).stop 2024-04-25T09:04:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.725+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f81454c0 msgr2=0x7f68f80b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.725+0000 7f68fee00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f81454c0 0x7f68f80b16c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f68f000efc0 tx=0x7f68f000c490 comp rx=0 tx=0).stop 2024-04-25T09:04:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.726+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 shutdown_connections 2024-04-25T09:04:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.726+0000 7f68fee00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f68d406b500 0x7f68d406d9c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.726+0000 7f68fee00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68f81454c0 0x7f68f80b16c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.726+0000 7f68fee00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68f80ccbe0 0x7f68f815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.726+0000 7f68fee00640 1 --2- 172.21.15.62:0/835233218 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68f80b44e0 0x7f68f813c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.726+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 >> 172.21.15.62:0/835233218 conn(0x7f68f80668f0 msgr2=0x7f68f806c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.726+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 shutdown_connections 2024-04-25T09:04:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.726+0000 7f68fee00640 1 -- 172.21.15.62:0/835233218 wait complete. 2024-04-25T09:04:19.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T09:04:19.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.935+0000 7f8ab3200640 1 Processor -- start 2024-04-25T09:04:19.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.936+0000 7f8ab3200640 1 -- start start 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab3200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8aac0b60b0 0x7f8aac0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac0b85a0 con 0x7f8aac0d3070 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac0b8880 con 0x7f8aac15dca0 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab3200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0d1bd0 con 0x7f8aac0b60b0 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8aac0b60b0 0x7f8aac0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab2800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8aac0d3070 0x7f8aac0b6f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60588/0 (socket says 172.21.15.62:60588) 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab2800640 1 -- 172.21.15.62:0/3476074802 learned_addr learned my addr 172.21.15.62:0/3476074802 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab1400640 1 -- 172.21.15.62:0/3476074802 >> v1:172.21.15.121:6789/0 conn(0x7f8aac15dca0 legacy=0x7f8aac0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab1400640 1 -- 172.21.15.62:0/3476074802 >> v1:172.21.15.62:6789/0 conn(0x7f8aac0d3070 legacy=0x7f8aac0b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.937+0000 7f8ab1400640 1 -- 172.21.15.62:0/3476074802 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0b8b60 con 0x7f8aac0b60b0 2024-04-25T09:04:19.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.939+0000 7f8ab1400640 1 --2- 172.21.15.62:0/3476074802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0b60b0 0x7f8aac0b69d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0097a0 tx=0x7f8a9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6e7fb6214ec3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.939+0000 7f8ab0a00640 1 -- 172.21.15.62:0/3476074802 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c037070 con 0x7f8aac0b60b0 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.939+0000 7f8ab0a00640 1 -- 172.21.15.62:0/3476074802 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c032440 con 0x7f8aac0b60b0 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.939+0000 7f8ab0a00640 1 -- 172.21.15.62:0/3476074802 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c032810 con 0x7f8aac0b60b0 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.940+0000 7f8ab3200640 1 -- 172.21.15.62:0/3476074802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0b60b0 msgr2=0x7f8aac0b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.940+0000 7f8ab3200640 1 --2- 172.21.15.62:0/3476074802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0b60b0 0x7f8aac0b69d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0097a0 tx=0x7f8a9c02a270 comp rx=0 tx=0).stop 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.940+0000 7f8ab3200640 1 -- 172.21.15.62:0/3476074802 shutdown_connections 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.940+0000 7f8ab3200640 1 --2- 172.21.15.62:0/3476074802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0b60b0 0x7f8aac0b69d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.940+0000 7f8ab3200640 1 -- 172.21.15.62:0/3476074802 >> 172.21.15.62:0/3476074802 conn(0x7f8aac06b000 msgr2=0x7f8aac06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.940+0000 7f8ab3200640 1 -- 172.21.15.62:0/3476074802 shutdown_connections 2024-04-25T09:04:19.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.940+0000 7f8ab3200640 1 -- 172.21.15.62:0/3476074802 wait complete. 2024-04-25T09:04:19.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.941+0000 7f8ab3200640 1 Processor -- start 2024-04-25T09:04:19.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.941+0000 7f8ab3200640 1 -- start start 2024-04-25T09:04:19.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.942+0000 7f8ab3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac0b60b0 0x7f8aac150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.942+0000 7f8ab3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0d3070 0x7f8aac0aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.942+0000 7f8ab3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac15dca0 0x7f8aac0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.942+0000 7f8ab3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b77a0 con 0x7f8aac0d3070 2024-04-25T09:04:19.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.942+0000 7f8ab3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b7620 con 0x7f8aac0b60b0 2024-04-25T09:04:19.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.942+0000 7f8ab3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b7920 con 0x7f8aac15dca0 2024-04-25T09:04:19.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0d3070 0x7f8aac0aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0d3070 0x7f8aac0aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53726/0 (socket says 172.21.15.62:53726) 2024-04-25T09:04:19.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 -- 172.21.15.62:0/3653495471 learned_addr learned my addr 172.21.15.62:0/3653495471 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:19.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 -- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac15dca0 msgr2=0x7f8aac0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac15dca0 0x7f8aac0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 -- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac0b60b0 msgr2=0x7f8aac150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac0b60b0 0x7f8aac150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 -- 172.21.15.62:0/3653495471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac1508a0 con 0x7f8aac0d3070 2024-04-25T09:04:19.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.943+0000 7f8ab1400640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0d3070 0x7f8aac0aad30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c000c00 tx=0x7f8a9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.944+0000 7f8aa2000640 1 -- 172.21.15.62:0/3653495471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c037040 con 0x7f8aac0d3070 2024-04-25T09:04:19.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.944+0000 7f8aa2000640 1 -- 172.21.15.62:0/3653495471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c042070 con 0x7f8aac0d3070 2024-04-25T09:04:19.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.944+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac0ac6d0 con 0x7f8aac0d3070 2024-04-25T09:04:19.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.944+0000 7f8aa2000640 1 -- 172.21.15.62:0/3653495471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c030a70 con 0x7f8aac0d3070 2024-04-25T09:04:19.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.944+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac0ab910 con 0x7f8aac0d3070 2024-04-25T09:04:19.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.945+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac075070 con 0x7f8aac0d3070 2024-04-25T09:04:19.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.946+0000 7f8aa2000640 1 -- 172.21.15.62:0/3653495471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a9c030c10 con 0x7f8aac0d3070 2024-04-25T09:04:19.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.947+0000 7f8aa2000640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a9006b4d0 0x7f8a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.947+0000 7f8aa2000640 1 -- 172.21.15.62:0/3653495471 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8a9c0aa9c0 con 0x7f8aac0d3070 2024-04-25T09:04:19.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.948+0000 7f8ab1e00640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a9006b4d0 0x7f8a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.949+0000 7f8ab1e00640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a9006b4d0 0x7f8a9006d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400ab90 tx=0x7f8aa4009230 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:19.951+0000 7f8aa2000640 1 -- 172.21.15.62:0/3653495471 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a9c0ace20 con 0x7f8aac0d3070 2024-04-25T09:04:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.228+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f8aac0b5d20 con 0x7f8aac0d3070 2024-04-25T09:04:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.232+0000 7f8aa2000640 1 -- 172.21.15.62:0/3653495471 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8a9c045360 con 0x7f8aac0d3070 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a9006b4d0 msgr2=0x7f8a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a9006b4d0 0x7f8a9006d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400ab90 tx=0x7f8aa4009230 comp rx=0 tx=0).stop 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0d3070 msgr2=0x7f8aac0aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0d3070 0x7f8aac0aad30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c000c00 tx=0x7f8a9c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 shutdown_connections 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a9006b4d0 0x7f8a9006d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8aac0d3070 0x7f8aac0aad30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8aac15dca0 0x7f8aac0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 --2- 172.21.15.62:0/3653495471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aac0b60b0 0x7f8aac150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 >> 172.21.15.62:0/3653495471 conn(0x7f8aac06b000 msgr2=0x7f8aac15b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 shutdown_connections 2024-04-25T09:04:20.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.235+0000 7f8ab3200640 1 -- 172.21.15.62:0/3653495471 wait complete. 2024-04-25T09:04:20.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.462+0000 7f8229400640 1 Processor -- start 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.462+0000 7f8229400640 1 -- start start 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.464+0000 7f8229400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82240b47e0 0x7f822406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.464+0000 7f8229400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822406d4f0 con 0x7f822415b960 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.464+0000 7f8229400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82240b2500 con 0x7f82240ae9d0 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.464+0000 7f8229400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822415bd40 con 0x7f82240b47e0 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.464+0000 7f8223e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f822415b960 0x7f82240b40d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60600/0 (socket says 172.21.15.62:60600) 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.464+0000 7f8223e00640 1 -- 172.21.15.62:0/2834004163 learned_addr learned my addr 172.21.15.62:0/2834004163 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f821b400640 1 --2- 172.21.15.62:0/2834004163 >> v2:172.21.15.121:3300/0 conn(0x7f82240b47e0 0x7f822406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201297739 0 0) 0x7f822406d4f0 con 0x7f822415b960 2024-04-25T09:04:20.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8208003520 con 0x7f822415b960 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2730752693 0 0) 0x7f8208003520 con 0x7f822415b960 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 >> v2:172.21.15.121:3300/0 conn(0x7f82240b47e0 msgr2=0x7f822406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 --2- 172.21.15.62:0/2834004163 >> v2:172.21.15.121:3300/0 conn(0x7f82240b47e0 0x7f822406ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 >> v1:172.21.15.121:6789/0 conn(0x7f82240ae9d0 legacy=0x7f82240cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240b27e0 con 0x7f822415b960 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f82140019f0 con 0x7f822415b960 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 >> v1:172.21.15.62:6789/0 conn(0x7f822415b960 legacy=0x7f82240b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8208003a20 0x7f8208003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f8208004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82080051c0 0x7f820800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820800c0c0 con 0x7f8208004800 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820800c270 con 0x7f82080051c0 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.465+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820800c470 con 0x7f8208003a20 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f821b400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f8208004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f8228a00640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82080051c0 0x7f820800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f8223e00640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8208003a20 0x7f8208003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f821b400640 1 -- 172.21.15.62:0/2834004163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8208003a20 msgr2=0x7f8208003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:20.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f821b400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8208003a20 0x7f8208003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f821b400640 1 -- 172.21.15.62:0/2834004163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82080051c0 msgr2=0x7f820800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:20.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f821b400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82080051c0 0x7f820800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f821b400640 1 -- 172.21.15.62:0/2834004163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8208003520 con 0x7f8208004800 2024-04-25T09:04:20.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.466+0000 7f8228a00640 1 -- 172.21.15.62:0/2834004163 reap_dead start 2024-04-25T09:04:20.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.467+0000 7f821b400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f8208004c80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f820c009ac0 tx=0x7f8208005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c9bd907e5276b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:20.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.467+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c036070 con 0x7f8208004800 2024-04-25T09:04:20.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.467+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c031400 con 0x7f8208004800 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.467+0000 7f8223400640 1 -- 172.21.15.62:0/2834004163 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c009690 con 0x7f8208004800 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.468+0000 7f8229400640 1 -- 172.21.15.62:0/2834004163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 msgr2=0x7f8208004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.468+0000 7f8229400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f8208004c80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f820c009ac0 tx=0x7f8208005830 comp rx=0 tx=0).stop 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.468+0000 7f8229400640 1 -- 172.21.15.62:0/2834004163 shutdown_connections 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.468+0000 7f8229400640 1 --2- 172.21.15.62:0/2834004163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f8208004c80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.468+0000 7f8229400640 1 -- 172.21.15.62:0/2834004163 >> 172.21.15.62:0/2834004163 conn(0x7f8224074380 msgr2=0x7f82240668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.468+0000 7f8229400640 1 -- 172.21.15.62:0/2834004163 shutdown_connections 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.468+0000 7f8229400640 1 -- 172.21.15.62:0/2834004163 wait complete. 2024-04-25T09:04:20.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.470+0000 7f8229400640 1 Processor -- start 2024-04-25T09:04:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.470+0000 7f8229400640 1 -- start start 2024-04-25T09:04:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8229400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f82241507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8229400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822414e330 0x7f822414c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8229400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f822414f9f0 0x7f822414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8229400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240b2da0 con 0x7f8208004800 2024-04-25T09:04:20.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8229400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240b3110 con 0x7f822414f9f0 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8229400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822406ca80 con 0x7f822414e330 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8223e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f82241507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8223e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f82241507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53746/0 (socket says 172.21.15.62:53746) 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8223e00640 1 -- 172.21.15.62:0/2743100257 learned_addr learned my addr 172.21.15.62:0/2743100257 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.471+0000 7f8228a00640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f822414f9f0 0x7f822414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f821b400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822414e330 0x7f822414c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8223e00640 1 -- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822414e330 msgr2=0x7f822414c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8223e00640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822414e330 0x7f822414c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8223e00640 1 -- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f822414f9f0 msgr2=0x7f822414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8223e00640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f822414f9f0 0x7f822414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8223e00640 1 -- 172.21.15.62:0/2743100257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822414e8a0 con 0x7f8208004800 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8223e00640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f82241507d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8214002060 tx=0x7f821400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8220c00640 1 -- 172.21.15.62:0/2743100257 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214019070 con 0x7f8208004800 2024-04-25T09:04:20.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8220c00640 1 -- 172.21.15.62:0/2743100257 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214004d60 con 0x7f8208004800 2024-04-25T09:04:20.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8220c00640 1 -- 172.21.15.62:0/2743100257 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140072f0 con 0x7f8208004800 2024-04-25T09:04:20.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822414eb80 con 0x7f8208004800 2024-04-25T09:04:20.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.472+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8224150d10 con 0x7f8208004800 2024-04-25T09:04:20.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.474+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8224075070 con 0x7f8208004800 2024-04-25T09:04:20.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.474+0000 7f8220c00640 1 -- 172.21.15.62:0/2743100257 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8214007490 con 0x7f8208004800 2024-04-25T09:04:20.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.475+0000 7f8220c00640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f820006b500 0x7f820006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.475+0000 7f8220c00640 1 -- 172.21.15.62:0/2743100257 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8214089ec0 con 0x7f8208004800 2024-04-25T09:04:20.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.476+0000 7f821b400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f820006b500 0x7f820006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.477+0000 7f821b400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f820006b500 0x7f820006d9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f820c033000 tx=0x7f820c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:20.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.479+0000 7f8220c00640 1 -- 172.21.15.62:0/2743100257 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8214059080 con 0x7f8208004800 2024-04-25T09:04:20.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.754+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f82240b3eb0 con 0x7f8208004800 2024-04-25T09:04:20.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.758+0000 7f8220c00640 1 -- 172.21.15.62:0/2743100257 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8214058980 con 0x7f8208004800 2024-04-25T09:04:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f820006b500 msgr2=0x7f820006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f820006b500 0x7f820006d9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f820c033000 tx=0x7f820c008030 comp rx=0 tx=0).stop 2024-04-25T09:04:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 msgr2=0x7f82241507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f82241507d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8214002060 tx=0x7f821400b4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 shutdown_connections 2024-04-25T09:04:20.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f820006b500 0x7f820006d9c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f822414f9f0 0x7f822414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f822414e330 0x7f822414c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 --2- 172.21.15.62:0/2743100257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8208004800 0x7f82241507d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 >> 172.21.15.62:0/2743100257 conn(0x7f8224074380 msgr2=0x7f82240cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.761+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 shutdown_connections 2024-04-25T09:04:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.762+0000 7f8229400640 1 -- 172.21.15.62:0/2743100257 wait complete. 2024-04-25T09:04:20.767 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T09:04:20.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.980+0000 7f6c1d400640 1 Processor -- start 2024-04-25T09:04:20.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.980+0000 7f6c1d400640 1 -- start start 2024-04-25T09:04:20.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.982+0000 7f6c1d400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6c180b49e0 0x7f6c181668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.982+0000 7f6c1d400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c180b3720 con 0x7f6c180cf5f0 2024-04-25T09:04:20.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.982+0000 7f6c1d400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c180cff60 con 0x7f6c1815c4e0 2024-04-25T09:04:20.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.982+0000 7f6c1d400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c18070360 con 0x7f6c180b49e0 2024-04-25T09:04:20.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.983+0000 7f6c17400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6c180cf5f0 0x7f6c180b42d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60608/0 (socket says 172.21.15.62:60608) 2024-04-25T09:04:20.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.983+0000 7f6c17400640 1 -- 172.21.15.62:0/3915450500 learned_addr learned my addr 172.21.15.62:0/3915450500 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:20.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.983+0000 7f6c1ca00640 1 --2- 172.21.15.62:0/3915450500 >> v2:172.21.15.121:3300/0 conn(0x7f6c180b49e0 0x7f6c181668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.983+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432089417 0 0) 0x7f6c180b3720 con 0x7f6c180cf5f0 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.983+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bfc003520 con 0x7f6c180cf5f0 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.983+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1683483364 0 0) 0x7f6bfc003520 con 0x7f6c180cf5f0 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 >> v2:172.21.15.121:3300/0 conn(0x7f6c180b49e0 msgr2=0x7f6c181668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 --2- 172.21.15.62:0/3915450500 >> v2:172.21.15.121:3300/0 conn(0x7f6c180b49e0 0x7f6c181668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 >> v1:172.21.15.121:6789/0 conn(0x7f6c1815c4e0 legacy=0x7f6c1815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18070640 con 0x7f6c180cf5f0 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f6c04001670 con 0x7f6c180cf5f0 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 >> v1:172.21.15.62:6789/0 conn(0x7f6c180cf5f0 legacy=0x7f6c180b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bfc003a00 0x7f6bfc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6bfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bfc005210 0x7f6bfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c18070360 con 0x7f6bfc004800 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc00c160 con 0x7f6bfc005210 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc00c360 con 0x7f6bfc003a00 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c17400640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6bfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c17e00640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bfc003a00 0x7f6bfc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c1ca00640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bfc005210 0x7f6bfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c17400640 1 -- 172.21.15.62:0/3915450500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bfc003a00 msgr2=0x7f6bfc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c17400640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bfc003a00 0x7f6bfc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c17400640 1 -- 172.21.15.62:0/3915450500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bfc005210 msgr2=0x7f6bfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.984+0000 7f6c17400640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bfc005210 0x7f6bfc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.985+0000 7f6c17400640 1 -- 172.21.15.62:0/3915450500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc003520 con 0x7f6bfc004800 2024-04-25T09:04:20.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.985+0000 7f6c1ca00640 1 -- 172.21.15.62:0/3915450500 reap_dead start 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.985+0000 7f6c17400640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6bfc004c80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6c04007fd0 tx=0x7f6bfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d34354d285f771b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04007c20 con 0x7f6bfc004800 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c04005af0 con 0x7f6bfc004800 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c16a00640 1 -- 172.21.15.62:0/3915450500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04007070 con 0x7f6bfc004800 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c1d400640 1 -- 172.21.15.62:0/3915450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 msgr2=0x7f6bfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c1d400640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6bfc004c80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6c04007fd0 tx=0x7f6bfc005830 comp rx=0 tx=0).stop 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c1d400640 1 -- 172.21.15.62:0/3915450500 shutdown_connections 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c1d400640 1 --2- 172.21.15.62:0/3915450500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6bfc004c80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c1d400640 1 -- 172.21.15.62:0/3915450500 >> 172.21.15.62:0/3915450500 conn(0x7f6c180668f0 msgr2=0x7f6c18066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c1d400640 1 -- 172.21.15.62:0/3915450500 shutdown_connections 2024-04-25T09:04:20.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.986+0000 7f6c1d400640 1 -- 172.21.15.62:0/3915450500 wait complete. 2024-04-25T09:04:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.988+0000 7f6c1d400640 1 Processor -- start 2024-04-25T09:04:20.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.988+0000 7f6c1d400640 1 -- start start 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c1d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6c180b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c1d400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c180b19c0 0x7f6c18147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c1d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c18147940 0x7f6c18147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c1d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c18071120 con 0x7f6bfc004800 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c1d400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c18070c00 con 0x7f6c18147940 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c1d400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c180cfaa0 con 0x7f6c180b19c0 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c17e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6c180b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c17e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6c180b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53760/0 (socket says 172.21.15.62:53760) 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.989+0000 7f6c17e00640 1 -- 172.21.15.62:0/2747802834 learned_addr learned my addr 172.21.15.62:0/2747802834 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c1ca00640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c18147940 0x7f6c18147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c17e00640 1 -- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c180b19c0 msgr2=0x7f6c18147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c17e00640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c180b19c0 0x7f6c18147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c17e00640 1 -- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c18147940 msgr2=0x7f6c18147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c17e00640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c18147940 0x7f6c18147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c17e00640 1 -- 172.21.15.62:0/2747802834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c181484a0 con 0x7f6bfc004800 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c17e00640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6c180b1480 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800e480 tx=0x7f6c0800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c0fe00640 1 -- 172.21.15.62:0/2747802834 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c080052e0 con 0x7f6bfc004800 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c0fe00640 1 -- 172.21.15.62:0/2747802834 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c08018070 con 0x7f6bfc004800 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.990+0000 7f6c0fe00640 1 -- 172.21.15.62:0/2747802834 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08003810 con 0x7f6bfc004800 2024-04-25T09:04:20.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.991+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c18148780 con 0x7f6bfc004800 2024-04-25T09:04:20.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.991+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c18159820 con 0x7f6bfc004800 2024-04-25T09:04:20.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.992+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c181489a0 con 0x7f6bfc004800 2024-04-25T09:04:20.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.993+0000 7f6c0fe00640 1 -- 172.21.15.62:0/2747802834 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c080039b0 con 0x7f6bfc004800 2024-04-25T09:04:20.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.994+0000 7f6c0fe00640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6bf806b500 0x7f6bf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:20.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.994+0000 7f6c0fe00640 1 -- 172.21.15.62:0/2747802834 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6c08089bb0 con 0x7f6bfc004800 2024-04-25T09:04:20.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.994+0000 7f6c17400640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6bf806b500 0x7f6bf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:20.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.995+0000 7f6c17400640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6bf806b500 0x7f6bf806d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c04032000 tx=0x7f6c04007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:20.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:20.998+0000 7f6c0fe00640 1 -- 172.21.15.62:0/2747802834 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c08058d70 con 0x7f6bfc004800 2024-04-25T09:04:21.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.265+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f6c1804ef20 con 0x7f6bfc004800 2024-04-25T09:04:21.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.270+0000 7f6c0fe00640 1 -- 172.21.15.62:0/2747802834 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c08058670 con 0x7f6bfc004800 2024-04-25T09:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6bf806b500 msgr2=0x7f6bf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6bf806b500 0x7f6bf806d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c04032000 tx=0x7f6c04007000 comp rx=0 tx=0).stop 2024-04-25T09:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 msgr2=0x7f6c180b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6c180b1480 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800e480 tx=0x7f6c0800e940 comp rx=0 tx=0).stop 2024-04-25T09:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 shutdown_connections 2024-04-25T09:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6bf806b500 0x7f6bf806d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c18147940 0x7f6c18147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c180b19c0 0x7f6c18147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 --2- 172.21.15.62:0/2747802834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bfc004800 0x7f6c180b1480 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 >> 172.21.15.62:0/2747802834 conn(0x7f6c180668f0 msgr2=0x7f6c180b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 shutdown_connections 2024-04-25T09:04:21.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.273+0000 7f6c1d400640 1 -- 172.21.15.62:0/2747802834 wait complete. 2024-04-25T09:04:21.279 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T09:04:21.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.496+0000 7f571be00640 1 Processor -- start 2024-04-25T09:04:21.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.496+0000 7f571be00640 1 -- start start 2024-04-25T09:04:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.498+0000 7f571be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c15b960 0x7f571c06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.498+0000 7f571be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f571c0ae9d0 0x7f571c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.498+0000 7f571be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c06d4f0 con 0x7f571c0b4610 2024-04-25T09:04:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.498+0000 7f571be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571c15bd40 con 0x7f571c0ae9d0 2024-04-25T09:04:21.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.498+0000 7f571be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0b4a60 con 0x7f571c15b960 2024-04-25T09:04:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.499+0000 7f5712000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f571c0ae9d0 0x7f571c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.499+0000 7f5712000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f571c0ae9d0 0x7f571c0b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53768/0 (socket says 172.21.15.62:53768) 2024-04-25T09:04:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.499+0000 7f5712000640 1 -- 172.21.15.62:0/2166730143 learned_addr learned my addr 172.21.15.62:0/2166730143 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.499+0000 7f571aa00640 1 --2- 172.21.15.62:0/2166730143 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c15b960 0x7f571c06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.499+0000 7f5712000640 1 -- 172.21.15.62:0/2166730143 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c15b960 msgr2=0x7f571c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:21.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.499+0000 7f5712000640 1 --2- 172.21.15.62:0/2166730143 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c15b960 0x7f571c06ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.499+0000 7f5712000640 1 -- 172.21.15.62:0/2166730143 >> v1:172.21.15.121:6789/0 conn(0x7f571c0b4610 legacy=0x7f571c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.499+0000 7f5712000640 1 -- 172.21.15.62:0/2166730143 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0b2500 con 0x7f571c0ae9d0 2024-04-25T09:04:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.500+0000 7f5712000640 1 --2- 172.21.15.62:0/2166730143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c0ae9d0 0x7f571c0b40d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5704009870 tx=0x7f570402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30afb1db7acd7130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.500+0000 7f571a000640 1 -- 172.21.15.62:0/2166730143 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704037070 con 0x7f571c0ae9d0 2024-04-25T09:04:21.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571a000640 1 -- 172.21.15.62:0/2166730143 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704032440 con 0x7f571c0ae9d0 2024-04-25T09:04:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571a000640 1 -- 172.21.15.62:0/2166730143 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704032810 con 0x7f571c0ae9d0 2024-04-25T09:04:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571be00640 1 -- 172.21.15.62:0/2166730143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c0ae9d0 msgr2=0x7f571c0b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571be00640 1 --2- 172.21.15.62:0/2166730143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c0ae9d0 0x7f571c0b40d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5704009870 tx=0x7f570402a210 comp rx=0 tx=0).stop 2024-04-25T09:04:21.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571be00640 1 -- 172.21.15.62:0/2166730143 shutdown_connections 2024-04-25T09:04:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571be00640 1 --2- 172.21.15.62:0/2166730143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c0ae9d0 0x7f571c0b40d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571be00640 1 --2- 172.21.15.62:0/2166730143 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c15b960 0x7f571c06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571be00640 1 -- 172.21.15.62:0/2166730143 >> 172.21.15.62:0/2166730143 conn(0x7f571c074380 msgr2=0x7f571c0668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571be00640 1 -- 172.21.15.62:0/2166730143 shutdown_connections 2024-04-25T09:04:21.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.501+0000 7f571be00640 1 -- 172.21.15.62:0/2166730143 wait complete. 2024-04-25T09:04:21.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.504+0000 7f571be00640 1 Processor -- start 2024-04-25T09:04:21.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.504+0000 7f571be00640 1 -- start start 2024-04-25T09:04:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.505+0000 7f571be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f571c0ae9d0 0x7f571c13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.505+0000 7f571be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c0b4610 0x7f571c141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.505+0000 7f571be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c15b960 0x7f571c157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.505+0000 7f571be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0b2ac0 con 0x7f571c15b960 2024-04-25T09:04:21.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.505+0000 7f571be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0b3110 con 0x7f571c0ae9d0 2024-04-25T09:04:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.505+0000 7f571be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0b2da0 con 0x7f571c0b4610 2024-04-25T09:04:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.505+0000 7f571b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c15b960 0x7f571c157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.505+0000 7f571b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c15b960 0x7f571c157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53782/0 (socket says 172.21.15.62:53782) 2024-04-25T09:04:21.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f571b400640 1 -- 172.21.15.62:0/3316551100 learned_addr learned my addr 172.21.15.62:0/3316551100 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f571aa00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f571c0ae9d0 0x7f571c13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f5712000640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c0b4610 0x7f571c141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f571b400640 1 -- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c0b4610 msgr2=0x7f571c141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f571b400640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c0b4610 0x7f571c141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f571b400640 1 -- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f571c0ae9d0 msgr2=0x7f571c13e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f571b400640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f571c0ae9d0 0x7f571c13e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f571b400640 1 -- 172.21.15.62:0/3316551100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c159830 con 0x7f571c15b960 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.506+0000 7f571b400640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c15b960 0x7f571c157920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f571400c780 tx=0x7f571400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.507+0000 7f5713400640 1 -- 172.21.15.62:0/3316551100 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714004b60 con 0x7f571c15b960 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.507+0000 7f5713400640 1 -- 172.21.15.62:0/3316551100 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5714004d00 con 0x7f571c15b960 2024-04-25T09:04:21.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.507+0000 7f5713400640 1 -- 172.21.15.62:0/3316551100 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571400aa90 con 0x7f571c15b960 2024-04-25T09:04:21.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.507+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c1f3ca0 con 0x7f571c15b960 2024-04-25T09:04:21.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.507+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c159d90 con 0x7f571c15b960 2024-04-25T09:04:21.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.508+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c075070 con 0x7f571c15b960 2024-04-25T09:04:21.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.509+0000 7f5713400640 1 -- 172.21.15.62:0/3316551100 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f571400ac30 con 0x7f571c15b960 2024-04-25T09:04:21.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.510+0000 7f5713400640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56f806b4d0 0x7f56f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.510+0000 7f5713400640 1 -- 172.21.15.62:0/3316551100 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f571408a210 con 0x7f571c15b960 2024-04-25T09:04:21.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.511+0000 7f571aa00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56f806b4d0 0x7f56f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.512+0000 7f571aa00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56f806b4d0 0x7f56f806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f570c006fd0 tx=0x7f570c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.514+0000 7f5713400640 1 -- 172.21.15.62:0/3316551100 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5714020070 con 0x7f571c15b960 2024-04-25T09:04:21.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.793+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f571c0b3eb0 con 0x7f571c15b960 2024-04-25T09:04:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.798+0000 7f5713400640 1 -- 172.21.15.62:0/3316551100 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f571c159830 con 0x7f571c15b960 2024-04-25T09:04:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56f806b4d0 msgr2=0x7f56f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56f806b4d0 0x7f56f806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f570c006fd0 tx=0x7f570c008040 comp rx=0 tx=0).stop 2024-04-25T09:04:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c15b960 msgr2=0x7f571c157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c15b960 0x7f571c157920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f571400c780 tx=0x7f571400cc40 comp rx=0 tx=0).stop 2024-04-25T09:04:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 shutdown_connections 2024-04-25T09:04:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56f806b4d0 0x7f56f806d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f571c15b960 0x7f571c157920 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f571c0b4610 0x7f571c141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 --2- 172.21.15.62:0/3316551100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f571c0ae9d0 0x7f571c13e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 >> 172.21.15.62:0/3316551100 conn(0x7f571c074380 msgr2=0x7f571c0cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 shutdown_connections 2024-04-25T09:04:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:21.801+0000 7f571be00640 1 -- 172.21.15.62:0/3316551100 wait complete. 2024-04-25T09:04:21.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:22.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.020+0000 7f1dbe000640 1 Processor -- start 2024-04-25T09:04:22.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.020+0000 7f1dbe000640 1 -- start start 2024-04-25T09:04:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.021+0000 7f1dbe000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80d0ae0 0x7f1db80d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.021+0000 7f1dbe000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1db80b6cf0 0x7f1db80b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.021+0000 7f1dbe000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db80d2620 con 0x7f1db815dc40 2024-04-25T09:04:22.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.021+0000 7f1dbe000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db80d1480 con 0x7f1db80b6cf0 2024-04-25T09:04:22.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.021+0000 7f1dbe000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80d23c0 con 0x7f1db80d0ae0 2024-04-25T09:04:22.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.022+0000 7f1db7400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1db815dc40 0x7f1db80d1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60618/0 (socket says 172.21.15.62:60618) 2024-04-25T09:04:22.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.022+0000 7f1db7400640 1 -- 172.21.15.62:0/2441771709 learned_addr learned my addr 172.21.15.62:0/2441771709 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.022+0000 7f1dafe00640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80d0ae0 0x7f1db80d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.022+0000 7f1db7e00640 1 --2- 172.21.15.62:0/2441771709 >> v2:172.21.15.121:3300/0 conn(0x7f1db80b6cf0 0x7f1db80b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.022+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599922722 0 0) 0x7f1db80d2620 con 0x7f1db815dc40 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.022+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d9c003520 con 0x7f1db815dc40 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1199728005 0 0) 0x7f1d9c003520 con 0x7f1db815dc40 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80d0ae0 msgr2=0x7f1db80d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80d0ae0 0x7f1db80d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 >> v2:172.21.15.121:3300/0 conn(0x7f1db80b6cf0 msgr2=0x7f1db80b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 --2- 172.21.15.62:0/2441771709 >> v2:172.21.15.121:3300/0 conn(0x7f1db80b6cf0 0x7f1db80b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db80d2900 con 0x7f1db815dc40 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1da8001670 con 0x7f1db815dc40 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 >> v1:172.21.15.62:6789/0 conn(0x7f1db815dc40 legacy=0x7f1db80d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1d9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d9c004800 0x7f1d9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80d0ae0 0x7f1d9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80d23c0 con 0x7f1d9c0039e0 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c00b9c0 con 0x7f1d9c004800 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c00bb40 con 0x7f1db80d0ae0 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.023+0000 7f1db7400640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1d9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.024+0000 7f1db7e00640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80d0ae0 0x7f1d9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.024+0000 7f1dafe00640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d9c004800 0x7f1d9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.024+0000 7f1db7400640 1 -- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80d0ae0 msgr2=0x7f1d9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.024+0000 7f1db7400640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80d0ae0 0x7f1d9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.024+0000 7f1db7400640 1 -- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d9c004800 msgr2=0x7f1d9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.024+0000 7f1db7400640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d9c004800 0x7f1d9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.024+0000 7f1db7400640 1 -- 172.21.15.62:0/2441771709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c003520 con 0x7f1d9c0039e0 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.025+0000 7f1db7400640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1d9c003e00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1da800c240 tx=0x7f1d9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e4f24bea2e02ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8007d10 con 0x7f1d9c0039e0 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da8005ad0 con 0x7f1d9c0039e0 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1db6a00640 1 -- 172.21.15.62:0/2441771709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8007070 con 0x7f1d9c0039e0 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1dbe000640 1 -- 172.21.15.62:0/2441771709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 msgr2=0x7f1d9c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1dbe000640 1 --2- 172.21.15.62:0/2441771709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1d9c003e00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1da800c240 tx=0x7f1d9c005830 comp rx=0 tx=0).stop 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1db7e00640 1 -- 172.21.15.62:0/2441771709 reap_dead start 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1dbe000640 1 -- 172.21.15.62:0/2441771709 shutdown_connections 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1dbe000640 1 -- 172.21.15.62:0/2441771709 >> 172.21.15.62:0/2441771709 conn(0x7f1db806b000 msgr2=0x7f1db806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1dbe000640 1 -- 172.21.15.62:0/2441771709 shutdown_connections 2024-04-25T09:04:22.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.026+0000 7f1dbe000640 1 -- 172.21.15.62:0/2441771709 wait complete. 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.028+0000 7f1dbe000640 1 Processor -- start 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.028+0000 7f1dbe000640 1 -- start start 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.029+0000 7f1dbe000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1db80c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.029+0000 7f1dbe000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1db8158280 0x7f1db80cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.029+0000 7f1dbe000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80cc630 0x7f1db8157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.029+0000 7f1dbe000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80d31a0 con 0x7f1d9c0039e0 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.029+0000 7f1dbe000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80d2ec0 con 0x7f1db8158280 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.029+0000 7f1dbe000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80d36c0 con 0x7f1db80cc630 2024-04-25T09:04:22.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1db80c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1db80c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53806/0 (socket says 172.21.15.62:53806) 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7400640 1 -- 172.21.15.62:0/1227463176 learned_addr learned my addr 172.21.15.62:0/1227463176 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1dafe00640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1db8158280 0x7f1db80cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7e00640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80cc630 0x7f1db8157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7400640 1 -- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80cc630 msgr2=0x7f1db8157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7400640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80cc630 0x7f1db8157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7400640 1 -- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1db8158280 msgr2=0x7f1db80cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7400640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1db8158280 0x7f1db80cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.030+0000 7f1db7400640 1 -- 172.21.15.62:0/1227463176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db80ccb00 con 0x7f1d9c0039e0 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.031+0000 7f1db7400640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1db80c69d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1da800c240 tx=0x7f1db80d2ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.031+0000 7f1daf400640 1 -- 172.21.15.62:0/1227463176 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8042020 con 0x7f1d9c0039e0 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.031+0000 7f1daf400640 1 -- 172.21.15.62:0/1227463176 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da8004bf0 con 0x7f1d9c0039e0 2024-04-25T09:04:22.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.031+0000 7f1daf400640 1 -- 172.21.15.62:0/1227463176 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8007d10 con 0x7f1d9c0039e0 2024-04-25T09:04:22.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.031+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db80ccde0 con 0x7f1d9c0039e0 2024-04-25T09:04:22.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.031+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db80cd3a0 con 0x7f1d9c0039e0 2024-04-25T09:04:22.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.032+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db80729c0 con 0x7f1d9c0039e0 2024-04-25T09:04:22.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.033+0000 7f1daf400640 1 -- 172.21.15.62:0/1227463176 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1da8045020 con 0x7f1d9c0039e0 2024-04-25T09:04:22.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.034+0000 7f1daf400640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1d9406b520 0x7f1d9406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.034+0000 7f1daf400640 1 -- 172.21.15.62:0/1227463176 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1da80aa430 con 0x7f1d9c0039e0 2024-04-25T09:04:22.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.035+0000 7f1dafe00640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1d9406b520 0x7f1d9406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.036+0000 7f1dafe00640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1d9406b520 0x7f1d9406d9e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1db815e690 tx=0x7f1da0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.038+0000 7f1daf400640 1 -- 172.21.15.62:0/1227463176 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1da8079700 con 0x7f1d9c0039e0 2024-04-25T09:04:22.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.230+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1db804f000 con 0x7f1d9c0039e0 2024-04-25T09:04:22.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.232+0000 7f1daf400640 1 -- 172.21.15.62:0/1227463176 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f1da8079000 con 0x7f1d9c0039e0 2024-04-25T09:04:22.220 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:22.220 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:04:18.147285+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:22.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1d9406b520 msgr2=0x7f1d9406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1d9406b520 0x7f1d9406d9e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1db815e690 tx=0x7f1da0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:22.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 msgr2=0x7f1db80c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1db80c69d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1da800c240 tx=0x7f1db80d2ad0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 shutdown_connections 2024-04-25T09:04:22.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1d9406b520 0x7f1d9406d9e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1db80cc630 0x7f1db8157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1db8158280 0x7f1db80cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 --2- 172.21.15.62:0/1227463176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d9c0039e0 0x7f1db80c69d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.234+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 >> 172.21.15.62:0/1227463176 conn(0x7f1db806b000 msgr2=0x7f1db80c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.235+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 shutdown_connections 2024-04-25T09:04:22.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.235+0000 7f1dbe000640 1 -- 172.21.15.62:0/1227463176 wait complete. 2024-04-25T09:04:22.240 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:22.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:22.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.458+0000 7f4f73000640 1 Processor -- start 2024-04-25T09:04:22.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.458+0000 7f4f73000640 1 -- start start 2024-04-25T09:04:22.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.460+0000 7f4f73000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4f6c0b54d0 0x7f4f6c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.460+0000 7f4f73000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f6c0ce7f0 0x7f4f6c164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.460+0000 7f4f73000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c0b4430 con 0x7f4f6c0b2d80 2024-04-25T09:04:22.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.460+0000 7f4f73000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b4710 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.460+0000 7f4f73000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b49f0 con 0x7f4f6c0b54d0 2024-04-25T09:04:22.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.460+0000 7f4f71c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4f6c0b2d80 0x7f4f6c0b3160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60620/0 (socket says 172.21.15.62:60620) 2024-04-25T09:04:22.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.460+0000 7f4f71c00640 1 -- 172.21.15.62:0/2394610950 learned_addr learned my addr 172.21.15.62:0/2394610950 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:22.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.460+0000 7f4f72600640 1 --2- 172.21.15.62:0/2394610950 >> v2:172.21.15.62:3300/0 conn(0x7f4f6c0ce7f0 0x7f4f6c164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.461+0000 7f4f71200640 1 --2- 172.21.15.62:0/2394610950 >> v2:172.21.15.121:3300/0 conn(0x7f4f6c0b54d0 0x7f4f6c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.461+0000 7f4f6be00640 1 -- 172.21.15.62:0/2394610950 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926772524 0 0) 0x7f4f6c0b4430 con 0x7f4f6c0b2d80 2024-04-25T09:04:22.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.461+0000 7f4f6be00640 1 -- 172.21.15.62:0/2394610950 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f54003520 con 0x7f4f6c0b2d80 2024-04-25T09:04:22.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.461+0000 7f4f72600640 1 -- 172.21.15.62:0/2394610950 >> v2:172.21.15.121:3300/0 conn(0x7f4f6c0b54d0 msgr2=0x7f4f6c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.461+0000 7f4f72600640 1 --2- 172.21.15.62:0/2394610950 >> v2:172.21.15.121:3300/0 conn(0x7f4f6c0b54d0 0x7f4f6c0ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.461+0000 7f4f72600640 1 -- 172.21.15.62:0/2394610950 >> v1:172.21.15.62:6789/0 conn(0x7f4f6c0b2d80 legacy=0x7f4f6c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.461+0000 7f4f72600640 1 -- 172.21.15.62:0/2394610950 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0b4cd0 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.462+0000 7f4f72600640 1 --2- 172.21.15.62:0/2394610950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c164310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0cd4c0 tx=0x7f4f6402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad400eb9dc32085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.462+0000 7f4f6be00640 1 -- 172.21.15.62:0/2394610950 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64007740 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.462+0000 7f4f6be00640 1 -- 172.21.15.62:0/2394610950 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f640078e0 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.462+0000 7f4f6be00640 1 -- 172.21.15.62:0/2394610950 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64007cb0 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.463+0000 7f4f73000640 1 -- 172.21.15.62:0/2394610950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 msgr2=0x7f4f6c164310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.463+0000 7f4f73000640 1 --2- 172.21.15.62:0/2394610950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c164310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0cd4c0 tx=0x7f4f6402a680 comp rx=0 tx=0).stop 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.463+0000 7f4f73000640 1 -- 172.21.15.62:0/2394610950 shutdown_connections 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.463+0000 7f4f73000640 1 --2- 172.21.15.62:0/2394610950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c164310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.463+0000 7f4f73000640 1 --2- 172.21.15.62:0/2394610950 >> v2:172.21.15.121:3300/0 conn(0x7f4f6c0b54d0 0x7f4f6c0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.463+0000 7f4f73000640 1 -- 172.21.15.62:0/2394610950 >> 172.21.15.62:0/2394610950 conn(0x7f4f6c066f20 msgr2=0x7f4f6c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.463+0000 7f4f73000640 1 -- 172.21.15.62:0/2394610950 shutdown_connections 2024-04-25T09:04:22.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.463+0000 7f4f73000640 1 -- 172.21.15.62:0/2394610950 wait complete. 2024-04-25T09:04:22.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.464+0000 7f4f73000640 1 Processor -- start 2024-04-25T09:04:22.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.465+0000 7f4f73000640 1 -- start start 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.465+0000 7f4f73000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b2d80 0x7f4f6c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f73000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b54d0 0x7f4f6c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f73000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f73000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b3870 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f73000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b40e0 con 0x7f4f6c0b54d0 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f73000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c164850 con 0x7f4f6c0b2d80 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f72600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f72600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c0ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53822/0 (socket says 172.21.15.62:53822) 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f72600640 1 -- 172.21.15.62:0/2351164007 learned_addr learned my addr 172.21.15.62:0/2351164007 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f71200640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b54d0 0x7f4f6c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f71c00640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b2d80 0x7f4f6c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f72600640 1 -- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b2d80 msgr2=0x7f4f6c150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f72600640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b2d80 0x7f4f6c150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f72600640 1 -- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b54d0 msgr2=0x7f4f6c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f72600640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b54d0 0x7f4f6c15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f72600640 1 -- 172.21.15.62:0/2351164007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c1523c0 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.466+0000 7f4f71200640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b54d0 0x7f4f6c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.467+0000 7f4f72600640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c0ad8e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4f6402ac70 tx=0x7f4f64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.467+0000 7f4f69600640 1 -- 172.21.15.62:0/2351164007 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6403f070 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.467+0000 7f4f69600640 1 -- 172.21.15.62:0/2351164007 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f64031bc0 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.467+0000 7f4f69600640 1 -- 172.21.15.62:0/2351164007 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64009450 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.467+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c150a90 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.467+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c152940 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.468+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c06abe0 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.469+0000 7f4f69600640 1 -- 172.21.15.62:0/2351164007 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f6403b410 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.470+0000 7f4f69600640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.470+0000 7f4f69600640 1 -- 172.21.15.62:0/2351164007 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f640ab640 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.471+0000 7f4f71c00640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.472+0000 7f4f71c00640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f60000f60 tx=0x7f4f60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.474+0000 7f4f69600640 1 -- 172.21.15.62:0/2351164007 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f6407a770 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.664+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f6c04ef20 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.665+0000 7f4f69600640 1 -- 172.21.15.62:0/2351164007 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f6407a070 con 0x7f4f6c0ce7f0 2024-04-25T09:04:22.652 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:04:22.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4c06b4d0 msgr2=0x7f4f4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f60000f60 tx=0x7f4f60009040 comp rx=0 tx=0).stop 2024-04-25T09:04:22.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 msgr2=0x7f4f6c0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c0ad8e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4f6402ac70 tx=0x7f4f64004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 shutdown_connections 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4c06b4d0 0x7f4f4c06d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0ce7f0 0x7f4f6c0ad8e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b54d0 0x7f4f6c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 --2- 172.21.15.62:0/2351164007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b2d80 0x7f4f6c150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 >> 172.21.15.62:0/2351164007 conn(0x7f4f6c066f20 msgr2=0x7f4f6c15d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.667+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 shutdown_connections 2024-04-25T09:04:22.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.668+0000 7f4f73000640 1 -- 172.21.15.62:0/2351164007 wait complete. 2024-04-25T09:04:22.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:04:22.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.887+0000 7f5367000640 1 Processor -- start 2024-04-25T09:04:22.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.888+0000 7f5367000640 1 -- start start 2024-04-25T09:04:22.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.889+0000 7f5367000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f53600b60b0 0x7f53600b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.889+0000 7f5367000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600d3070 0x7f53600b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.889+0000 7f5367000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53600b85a0 con 0x7f536015dca0 2024-04-25T09:04:22.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.889+0000 7f5367000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53600d1bd0 con 0x7f53600b60b0 2024-04-25T09:04:22.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.889+0000 7f5367000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53600b8880 con 0x7f53600d3070 2024-04-25T09:04:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.890+0000 7f5365200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f53600b60b0 0x7f53600b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.890+0000 7f5365200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f53600b60b0 0x7f53600b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:35774/0 (socket says 172.21.15.62:35774) 2024-04-25T09:04:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.890+0000 7f5366600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600d3070 0x7f53600b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.890+0000 7f5365200640 1 -- 172.21.15.62:0/287369879 learned_addr learned my addr 172.21.15.62:0/287369879 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.891+0000 7f5365200640 1 -- 172.21.15.62:0/287369879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600d3070 msgr2=0x7f53600b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.891+0000 7f5365200640 1 --2- 172.21.15.62:0/287369879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600d3070 0x7f53600b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.891+0000 7f5365200640 1 -- 172.21.15.62:0/287369879 >> v1:172.21.15.121:6789/0 conn(0x7f536015dca0 legacy=0x7f53600b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.891+0000 7f5365200640 1 -- 172.21.15.62:0/287369879 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53600b8b60 con 0x7f53600b60b0 2024-04-25T09:04:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.891+0000 7f5366600640 1 --2- 172.21.15.62:0/287369879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600d3070 0x7f53600b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:22.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.892+0000 7f5365200640 1 --2- 172.21.15.62:0/287369879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53600b60b0 0x7f53600b69d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2f3edbf2d683ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.892+0000 7f535fe00640 1 -- 172.21.15.62:0/287369879 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c037070 con 0x7f53600b60b0 2024-04-25T09:04:22.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f535fe00640 1 -- 172.21.15.62:0/287369879 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c02ae50 con 0x7f53600b60b0 2024-04-25T09:04:22.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f535fe00640 1 -- 172.21.15.62:0/287369879 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c032860 con 0x7f53600b60b0 2024-04-25T09:04:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f5367000640 1 -- 172.21.15.62:0/287369879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53600b60b0 msgr2=0x7f53600b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f5367000640 1 --2- 172.21.15.62:0/287369879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53600b60b0 0x7f53600b69d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c02a300 comp rx=0 tx=0).stop 2024-04-25T09:04:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f5367000640 1 -- 172.21.15.62:0/287369879 shutdown_connections 2024-04-25T09:04:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f5367000640 1 --2- 172.21.15.62:0/287369879 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600d3070 0x7f53600b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f5367000640 1 --2- 172.21.15.62:0/287369879 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53600b60b0 0x7f53600b69d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f5367000640 1 -- 172.21.15.62:0/287369879 >> 172.21.15.62:0/287369879 conn(0x7f536006b000 msgr2=0x7f536006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.893+0000 7f5367000640 1 -- 172.21.15.62:0/287369879 shutdown_connections 2024-04-25T09:04:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.894+0000 7f5367000640 1 -- 172.21.15.62:0/287369879 wait complete. 2024-04-25T09:04:22.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.895+0000 7f5367000640 1 Processor -- start 2024-04-25T09:04:22.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.895+0000 7f5367000640 1 -- start start 2024-04-25T09:04:22.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.896+0000 7f5367000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600b60b0 0x7f53600cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.896+0000 7f5367000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53600d3070 0x7f536014bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.896+0000 7f5367000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f536015dca0 0x7f53601f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.896+0000 7f5367000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600b75d0 con 0x7f53600d3070 2024-04-25T09:04:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.896+0000 7f5367000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600b7450 con 0x7f536015dca0 2024-04-25T09:04:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.896+0000 7f5367000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53600b7750 con 0x7f53600b60b0 2024-04-25T09:04:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53600d3070 0x7f536014bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53600d3070 0x7f536014bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53830/0 (socket says 172.21.15.62:53830) 2024-04-25T09:04:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365200640 1 -- 172.21.15.62:0/3769680357 learned_addr learned my addr 172.21.15.62:0/3769680357 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:22.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365c00640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600b60b0 0x7f53600cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5366600640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f536015dca0 0x7f53601f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365200640 1 -- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600b60b0 msgr2=0x7f53600cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365200640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600b60b0 0x7f53600cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365200640 1 -- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f536015dca0 msgr2=0x7f53601f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365200640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f536015dca0 0x7f53601f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.897+0000 7f5365200640 1 -- 172.21.15.62:0/3769680357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53600cb990 con 0x7f53600d3070 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.898+0000 7f5365200640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53600d3070 0x7f536014bf80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f534c02a340 tx=0x7f534c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.898+0000 7f535d600640 1 -- 172.21.15.62:0/3769680357 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c041020 con 0x7f53600d3070 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.898+0000 7f535d600640 1 -- 172.21.15.62:0/3769680357 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c03c870 con 0x7f53600d3070 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.898+0000 7f535d600640 1 -- 172.21.15.62:0/3769680357 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c037040 con 0x7f53600d3070 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.898+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53600cbc70 con 0x7f53600d3070 2024-04-25T09:04:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.898+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53600cc230 con 0x7f53600d3070 2024-04-25T09:04:22.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.899+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5360075070 con 0x7f53600d3070 2024-04-25T09:04:22.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.900+0000 7f535d600640 1 -- 172.21.15.62:0/3769680357 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f534c044020 con 0x7f53600d3070 2024-04-25T09:04:22.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.901+0000 7f535d600640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.902+0000 7f535d600640 1 -- 172.21.15.62:0/3769680357 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f534c0aa760 con 0x7f53600d3070 2024-04-25T09:04:22.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.902+0000 7f5365c00640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.903+0000 7f5365c00640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f534006b4d0 0x7f534006d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5354001ca0 tx=0x7f5354009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:22.906+0000 7f535d600640 1 -- 172.21.15.62:0/3769680357 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f534c04b030 con 0x7f53600d3070 2024-04-25T09:04:23.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.106+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f53600716a0 con 0x7f53600d3070 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.107+0000 7f535d600640 1 -- 172.21.15.62:0/3769680357 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f534c079920 con 0x7f53600d3070 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:04:23.095 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:04:23.096 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.109+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f534006b4d0 msgr2=0x7f534006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.109+0000 7f5367000640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f534006b4d0 0x7f534006d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5354001ca0 tx=0x7f5354009040 comp rx=0 tx=0).stop 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.109+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53600d3070 msgr2=0x7f536014bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.109+0000 7f5367000640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53600d3070 0x7f536014bf80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f534c02a340 tx=0x7f534c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.110+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 shutdown_connections 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.110+0000 7f5367000640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.110+0000 7f5367000640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f53600d3070 0x7f536014bf80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.110+0000 7f5367000640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f536015dca0 0x7f53601f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.110+0000 7f5367000640 1 --2- 172.21.15.62:0/3769680357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53600b60b0 0x7f53600cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.110+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 >> 172.21.15.62:0/3769680357 conn(0x7f536006b000 msgr2=0x7f53600668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.110+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 shutdown_connections 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.110+0000 7f5367000640 1 -- 172.21.15.62:0/3769680357 wait complete. 2024-04-25T09:04:23.118 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:04:23.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:04:23.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.353+0000 7fd567e00640 1 Processor -- start 2024-04-25T09:04:23.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.353+0000 7fd567e00640 1 -- start start 2024-04-25T09:04:23.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.355+0000 7fd567e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5680b5c70 0x7fd5680d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.355+0000 7fd567e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5680d1480 con 0x7fd568161650 2024-04-25T09:04:23.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.355+0000 7fd567e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5680d11a0 con 0x7fd568161270 2024-04-25T09:04:23.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.355+0000 7fd567e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5680d3900 con 0x7fd5680b5c70 2024-04-25T09:04:23.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.355+0000 7fd567400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd568161650 0x7fd5680d1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60630/0 (socket says 172.21.15.62:60630) 2024-04-25T09:04:23.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.355+0000 7fd567400640 1 -- 172.21.15.62:0/838853190 learned_addr learned my addr 172.21.15.62:0/838853190 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:23.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.355+0000 7fd566000640 1 --2- 172.21.15.62:0/838853190 >> v2:172.21.15.121:3300/0 conn(0x7fd5680b5c70 0x7fd5680d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932710059 0 0) 0x7fd5680d1480 con 0x7fd568161650 2024-04-25T09:04:23.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd54c003520 con 0x7fd568161650 2024-04-25T09:04:23.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2213542819 0 0) 0x7fd54c003520 con 0x7fd568161650 2024-04-25T09:04:23.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 >> v2:172.21.15.121:3300/0 conn(0x7fd5680b5c70 msgr2=0x7fd5680d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 --2- 172.21.15.62:0/838853190 >> v2:172.21.15.121:3300/0 conn(0x7fd5680b5c70 0x7fd5680d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 >> v1:172.21.15.121:6789/0 conn(0x7fd568161270 legacy=0x7fd5680b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680cee80 con 0x7fd568161650 2024-04-25T09:04:23.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fd5600019f0 con 0x7fd568161650 2024-04-25T09:04:23.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 >> v1:172.21.15.62:6789/0 conn(0x7fd568161650 legacy=0x7fd5680d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c0039b0 0x7fd54c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c004800 0x7fd54c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd54c0052e0 0x7fd54c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680d3900 con 0x7fd54c004800 2024-04-25T09:04:23.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c00c1b0 con 0x7fd54c0039b0 2024-04-25T09:04:23.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.356+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c00c3b0 con 0x7fd54c0052e0 2024-04-25T09:04:23.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.357+0000 7fd566000640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c004800 0x7fd54c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.357+0000 7fd566a00640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c0039b0 0x7fd54c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.357+0000 7fd566000640 1 -- 172.21.15.62:0/838853190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd54c0052e0 msgr2=0x7fd54c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.357+0000 7fd566000640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd54c0052e0 0x7fd54c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.357+0000 7fd566000640 1 -- 172.21.15.62:0/838853190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c0039b0 msgr2=0x7fd54c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.357+0000 7fd566000640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c0039b0 0x7fd54c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.357+0000 7fd566000640 1 -- 172.21.15.62:0/838853190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c003520 con 0x7fd54c004800 2024-04-25T09:04:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.357+0000 7fd567400640 1 -- 172.21.15.62:0/838853190 reap_dead start 2024-04-25T09:04:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.358+0000 7fd566000640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c004800 0x7fd54c004c80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd554009a90 tx=0x7fd54c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631d078c01428061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.358+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554036070 con 0x7fd54c004800 2024-04-25T09:04:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.358+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd554031400 con 0x7fd54c004800 2024-04-25T09:04:23.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.358+0000 7fd565600640 1 -- 172.21.15.62:0/838853190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554009690 con 0x7fd54c004800 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.359+0000 7fd567e00640 1 -- 172.21.15.62:0/838853190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c004800 msgr2=0x7fd54c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.359+0000 7fd567e00640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c004800 0x7fd54c004c80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd554009a90 tx=0x7fd54c005830 comp rx=0 tx=0).stop 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.359+0000 7fd567e00640 1 -- 172.21.15.62:0/838853190 shutdown_connections 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.359+0000 7fd567e00640 1 --2- 172.21.15.62:0/838853190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd54c004800 0x7fd54c004c80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.359+0000 7fd567e00640 1 -- 172.21.15.62:0/838853190 >> 172.21.15.62:0/838853190 conn(0x7fd568066d10 msgr2=0x7fd568067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.359+0000 7fd567e00640 1 -- 172.21.15.62:0/838853190 shutdown_connections 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.359+0000 7fd567e00640 1 -- 172.21.15.62:0/838853190 wait complete. 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.361+0000 7fd567e00640 1 Processor -- start 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.361+0000 7fd567e00640 1 -- start start 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd567e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c004800 0x7fd5680c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd567e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5681589f0 0x7fd5680cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd567e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd568158e70 0x7fd56814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd567e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680b2ed0 con 0x7fd5681589f0 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd567e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680d3590 con 0x7fd54c004800 2024-04-25T09:04:23.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd567e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5680b3050 con 0x7fd568158e70 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd566000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5681589f0 0x7fd5680cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd566000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5681589f0 0x7fd5680cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53844/0 (socket says 172.21.15.62:53844) 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.362+0000 7fd566000640 1 -- 172.21.15.62:0/795963724 learned_addr learned my addr 172.21.15.62:0/795963724 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd566a00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c004800 0x7fd5680c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd567400640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd568158e70 0x7fd56814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd566000640 1 -- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd568158e70 msgr2=0x7fd56814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd566000640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd568158e70 0x7fd56814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd566000640 1 -- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c004800 msgr2=0x7fd5680c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd566000640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c004800 0x7fd5680c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd566000640 1 -- 172.21.15.62:0/795963724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680cc190 con 0x7fd5681589f0 2024-04-25T09:04:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd567400640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd568158e70 0x7fd56814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd566a00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c004800 0x7fd5680c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd566000640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5681589f0 0x7fd5680cd230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd554009bb0 tx=0x7fd554004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd55ea00640 1 -- 172.21.15.62:0/795963724 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55403f020 con 0x7fd5681589f0 2024-04-25T09:04:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd55ea00640 1 -- 172.21.15.62:0/795963724 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd554037040 con 0x7fd5681589f0 2024-04-25T09:04:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5680cc470 con 0x7fd5681589f0 2024-04-25T09:04:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.363+0000 7fd55ea00640 1 -- 172.21.15.62:0/795963724 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554036070 con 0x7fd5681589f0 2024-04-25T09:04:23.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.364+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5680cca30 con 0x7fd5681589f0 2024-04-25T09:04:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.365+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd568072910 con 0x7fd5681589f0 2024-04-25T09:04:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.366+0000 7fd55ea00640 1 -- 172.21.15.62:0/795963724 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd554042020 con 0x7fd5681589f0 2024-04-25T09:04:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.367+0000 7fd55ea00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd54806b500 0x7fd54806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.367+0000 7fd55ea00640 1 -- 172.21.15.62:0/795963724 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd5540a9cd0 con 0x7fd5681589f0 2024-04-25T09:04:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.367+0000 7fd566a00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd54806b500 0x7fd54806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.369+0000 7fd566a00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd54806b500 0x7fd54806d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5580016b0 tx=0x7fd558009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.371+0000 7fd55ea00640 1 -- 172.21.15.62:0/795963724 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd554049070 con 0x7fd5681589f0 2024-04-25T09:04:23.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:23.569+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd5680750e0 con 0x7fd5681589f0 2024-04-25T09:04:24.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.160+0000 7fd55ea00640 1 -- 172.21.15.62:0/795963724 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd554078e00 con 0x7fd5681589f0 2024-04-25T09:04:24.148 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:04:24.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd54806b500 msgr2=0x7fd54806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd54806b500 0x7fd54806d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5580016b0 tx=0x7fd558009040 comp rx=0 tx=0).stop 2024-04-25T09:04:24.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5681589f0 msgr2=0x7fd5680cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5681589f0 0x7fd5680cd230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd554009bb0 tx=0x7fd554004730 comp rx=0 tx=0).stop 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 shutdown_connections 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd54806b500 0x7fd54806d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5681589f0 0x7fd5680cd230 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd568158e70 0x7fd56814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 --2- 172.21.15.62:0/795963724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd54c004800 0x7fd5680c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.163+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 >> 172.21.15.62:0/795963724 conn(0x7fd568066d10 msgr2=0x7fd5680b7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.164+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 shutdown_connections 2024-04-25T09:04:24.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.164+0000 7fd567e00640 1 -- 172.21.15.62:0/795963724 wait complete. 2024-04-25T09:04:24.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:04:24.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.398+0000 7fb7df200640 1 Processor -- start 2024-04-25T09:04:24.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.398+0000 7fb7df200640 1 -- start start 2024-04-25T09:04:24.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.400+0000 7fb7df200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb7e00ce820 0x7fb7e00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.400+0000 7fb7df200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 0x7fb7e00d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.400+0000 7fb7df200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e00b6bf0 con 0x7fb7e015dc60 2024-04-25T09:04:24.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.400+0000 7fb7df200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e00d1b40 con 0x7fb7e00ce820 2024-04-25T09:04:24.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.400+0000 7fb7df200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e0162380 con 0x7fb7e00d2590 2024-04-25T09:04:24.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.400+0000 7fb7dde00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb7e015dc60 0x7fb7e01618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60638/0 (socket says 172.21.15.62:60638) 2024-04-25T09:04:24.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.400+0000 7fb7dde00640 1 -- 172.21.15.62:0/2687813929 learned_addr learned my addr 172.21.15.62:0/2687813929 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:24.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7de800640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 0x7fb7e00d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dd400640 1 --2- 172.21.15.62:0/2687813929 >> v2:172.21.15.121:3300/0 conn(0x7fb7e00ce820 0x7fb7e00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734016126 0 0) 0x7fb7d0001490 con 0x7fb7e015dc60 2024-04-25T09:04:24.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c4003520 con 0x7fb7e015dc60 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3081407755 0 0) 0x7fb7c4003520 con 0x7fb7e015dc60 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 msgr2=0x7fb7e00d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 0x7fb7e00d29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 >> v2:172.21.15.121:3300/0 conn(0x7fb7e00ce820 msgr2=0x7fb7e00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 --2- 172.21.15.62:0/2687813929 >> v2:172.21.15.121:3300/0 conn(0x7fb7e00ce820 0x7fb7e00cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0162660 con 0x7fb7e015dc60 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb7d0001bc0 con 0x7fb7e015dc60 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 >> v1:172.21.15.62:6789/0 conn(0x7fb7e015dc60 legacy=0x7fb7e01618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 0x7fb7c40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c4004d40 0x7fb7c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00d1b40 con 0x7fb7c4003f30 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e0162380 con 0x7fb7c4004d40 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.401+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c400ba10 con 0x7fb7e00d2590 2024-04-25T09:04:24.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7dd400640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7dde00640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 0x7fb7c40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7de800640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c4004d40 0x7fb7c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7dd400640 1 -- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 msgr2=0x7fb7c40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7dd400640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 0x7fb7c40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7dd400640 1 -- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c4004d40 msgr2=0x7fb7c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7dd400640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7c4004d40 0x7fb7c40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7dd400640 1 -- 172.21.15.62:0/2687813929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4003520 con 0x7fb7c4003f30 2024-04-25T09:04:24.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.402+0000 7fb7dde00640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00d2590 0x7fb7c40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:24.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.403+0000 7fb7dd400640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7c4004800 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009af0 tx=0x7fb7c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d3e10cbd69be42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8036070 con 0x7fb7c4003f30 2024-04-25T09:04:24.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c8031400 con 0x7fb7c4003f30 2024-04-25T09:04:24.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7dca00640 1 -- 172.21.15.62:0/2687813929 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8009690 con 0x7fb7c4003f30 2024-04-25T09:04:24.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7df200640 1 -- 172.21.15.62:0/2687813929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 msgr2=0x7fb7c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7df200640 1 --2- 172.21.15.62:0/2687813929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7c4004800 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009af0 tx=0x7fb7c4005830 comp rx=0 tx=0).stop 2024-04-25T09:04:24.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7de800640 1 -- 172.21.15.62:0/2687813929 reap_dead start 2024-04-25T09:04:24.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7df200640 1 -- 172.21.15.62:0/2687813929 shutdown_connections 2024-04-25T09:04:24.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7df200640 1 -- 172.21.15.62:0/2687813929 >> 172.21.15.62:0/2687813929 conn(0x7fb7e00668f0 msgr2=0x7fb7e0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.404+0000 7fb7df200640 1 -- 172.21.15.62:0/2687813929 shutdown_connections 2024-04-25T09:04:24.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.405+0000 7fb7df200640 1 -- 172.21.15.62:0/2687813929 wait complete. 2024-04-25T09:04:24.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.406+0000 7fb7df200640 1 Processor -- start 2024-04-25T09:04:24.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.406+0000 7fb7df200640 1 -- start start 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.407+0000 7fb7df200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7e00b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.407+0000 7fb7df200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e00b1840 0x7fb7e00b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.407+0000 7fb7df200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00b4540 0x7fb7e00b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.407+0000 7fb7df200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e0161fd0 con 0x7fb7c4003f30 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.407+0000 7fb7df200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00b9670 con 0x7fb7e00b1840 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.407+0000 7fb7df200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00b2200 con 0x7fb7e00b4540 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7e00b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7e00b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53858/0 (socket says 172.21.15.62:53858) 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 -- 172.21.15.62:0/620002654 learned_addr learned my addr 172.21.15.62:0/620002654 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:24.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 -- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00b4540 msgr2=0x7fb7e00b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00b4540 0x7fb7e00b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 -- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e00b1840 msgr2=0x7fb7e00b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e00b1840 0x7fb7e00b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 -- 172.21.15.62:0/620002654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00cabc0 con 0x7fb7c4003f30 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.408+0000 7fb7dde00640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7e00b56b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0001d60 tx=0x7fb7d000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.409+0000 7fb7d6000640 1 -- 172.21.15.62:0/620002654 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0019070 con 0x7fb7c4003f30 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.409+0000 7fb7d6000640 1 -- 172.21.15.62:0/620002654 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d0004d60 con 0x7fb7c4003f30 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.409+0000 7fb7d6000640 1 -- 172.21.15.62:0/620002654 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00072d0 con 0x7fb7c4003f30 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.409+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e00cae80 con 0x7fb7c4003f30 2024-04-25T09:04:24.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.409+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e00cb440 con 0x7fb7c4003f30 2024-04-25T09:04:24.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.410+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e0067c40 con 0x7fb7c4003f30 2024-04-25T09:04:24.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.411+0000 7fb7d6000640 1 -- 172.21.15.62:0/620002654 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7d001c070 con 0x7fb7c4003f30 2024-04-25T09:04:24.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.412+0000 7fb7d6000640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7bc06b500 0x7fb7bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.412+0000 7fb7d6000640 1 -- 172.21.15.62:0/620002654 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb7d008a200 con 0x7fb7c4003f30 2024-04-25T09:04:24.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.413+0000 7fb7dd400640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7bc06b500 0x7fb7bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.414+0000 7fb7dd400640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7bc06b500 0x7fb7bc06d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8033000 tx=0x7fb7c802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.417+0000 7fb7d6000640 1 -- 172.21.15.62:0/620002654 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7e0067c40 con 0x7fb7c4003f30 2024-04-25T09:04:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:24.615+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb7e00d1960 con 0x7fb7c4003f30 2024-04-25T09:04:25.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.166+0000 7fb7d6000640 1 -- 172.21.15.62:0/620002654 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb7e00d1960 con 0x7fb7c4003f30 2024-04-25T09:04:25.155 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:04:25.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7bc06b500 msgr2=0x7fb7bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7bc06b500 0x7fb7bc06d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8033000 tx=0x7fb7c802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:25.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 msgr2=0x7fb7e00b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7e00b56b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0001d60 tx=0x7fb7d000b560 comp rx=0 tx=0).stop 2024-04-25T09:04:25.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 shutdown_connections 2024-04-25T09:04:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7bc06b500 0x7fb7bc06d9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e00b4540 0x7fb7e00b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e00b1840 0x7fb7e00b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 --2- 172.21.15.62:0/620002654 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7c4003f30 0x7fb7e00b56b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.170+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 >> 172.21.15.62:0/620002654 conn(0x7fb7e00668f0 msgr2=0x7fb7e015cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.171+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 shutdown_connections 2024-04-25T09:04:25.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.171+0000 7fb7df200640 1 -- 172.21.15.62:0/620002654 wait complete. 2024-04-25T09:04:25.190 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:04:25.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.387+0000 7f7acbe00640 1 Processor -- start 2024-04-25T09:04:25.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.388+0000 7f7acbe00640 1 -- start start 2024-04-25T09:04:25.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7acbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0ce670 0x7f7acc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7acbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7acc0b2c10 0x7f7acc0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7acbe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7acc0cea50 0x7f7acc0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7acbe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc0b6a20 con 0x7f7acc0b2c10 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7acbe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7acc0b9610 con 0x7f7acc0cea50 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7acbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc06ac80 con 0x7f7acc0ce670 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7ac2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7acc0b2c10 0x7f7acc0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7ac2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7acc0b2c10 0x7f7acc0d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36128/0 (socket says 172.21.15.62:36128) 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.389+0000 7f7ac2000640 1 -- 172.21.15.62:0/1683991957 learned_addr learned my addr 172.21.15.62:0/1683991957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.390+0000 7f7acaa00640 1 --2- 172.21.15.62:0/1683991957 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0ce670 0x7f7acc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.390+0000 7f7acb400640 1 --2- 172.21.15.62:0/1683991957 >> v2:172.21.15.121:3300/0 conn(0x7f7acc0cea50 0x7f7acc0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.390+0000 7f7ac2000640 1 -- 172.21.15.62:0/1683991957 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0ce670 msgr2=0x7f7acc06b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:25.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.390+0000 7f7ac2000640 1 --2- 172.21.15.62:0/1683991957 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0ce670 0x7f7acc06b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.390+0000 7f7ac2000640 1 -- 172.21.15.62:0/1683991957 >> v2:172.21.15.121:3300/0 conn(0x7f7acc0cea50 msgr2=0x7f7acc0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:25.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.390+0000 7f7ac2000640 1 --2- 172.21.15.62:0/1683991957 >> v2:172.21.15.121:3300/0 conn(0x7f7acc0cea50 0x7f7acc0d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.390+0000 7f7ac2000640 1 -- 172.21.15.62:0/1683991957 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc06af60 con 0x7f7acc0b2c10 2024-04-25T09:04:25.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.391+0000 7f7ac2000640 1 --2- 172.21.15.62:0/1683991957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0b2c10 0x7f7acc0d33d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0d3280 tx=0x7f7ab402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe21e494cdcafb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.391+0000 7f7aca000640 1 -- 172.21.15.62:0/1683991957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4037070 con 0x7f7acc0b2c10 2024-04-25T09:04:25.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.391+0000 7f7aca000640 1 -- 172.21.15.62:0/1683991957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab4032440 con 0x7f7acc0b2c10 2024-04-25T09:04:25.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.391+0000 7f7aca000640 1 -- 172.21.15.62:0/1683991957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4032810 con 0x7f7acc0b2c10 2024-04-25T09:04:25.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 -- 172.21.15.62:0/1683991957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0b2c10 msgr2=0x7f7acc0d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 --2- 172.21.15.62:0/1683991957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0b2c10 0x7f7acc0d33d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0d3280 tx=0x7f7ab402a270 comp rx=0 tx=0).stop 2024-04-25T09:04:25.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 -- 172.21.15.62:0/1683991957 shutdown_connections 2024-04-25T09:04:25.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 --2- 172.21.15.62:0/1683991957 >> v2:172.21.15.121:3300/0 conn(0x7f7acc0cea50 0x7f7acc0d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 --2- 172.21.15.62:0/1683991957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0b2c10 0x7f7acc0d33d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 --2- 172.21.15.62:0/1683991957 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0ce670 0x7f7acc06b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 -- 172.21.15.62:0/1683991957 >> 172.21.15.62:0/1683991957 conn(0x7f7acc0d39c0 msgr2=0x7f7acc0d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:25.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 -- 172.21.15.62:0/1683991957 shutdown_connections 2024-04-25T09:04:25.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.392+0000 7f7acbe00640 1 -- 172.21.15.62:0/1683991957 wait complete. 2024-04-25T09:04:25.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.394+0000 7f7acbe00640 1 Processor -- start 2024-04-25T09:04:25.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.394+0000 7f7acbe00640 1 -- start start 2024-04-25T09:04:25.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.395+0000 7f7acbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc0b2c10 0x7f7acc15a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.395+0000 7f7acbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0ce670 0x7f7acc15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.395+0000 7f7acbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0cea50 0x7f7acc0c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.395+0000 7f7acbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0b92a0 con 0x7f7acc0ce670 2024-04-25T09:04:25.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.395+0000 7f7acbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc0ceee0 con 0x7f7acc0b2c10 2024-04-25T09:04:25.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.395+0000 7f7acbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc15ddd0 con 0x7f7acc0cea50 2024-04-25T09:04:25.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7ac2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0ce670 0x7f7acc15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7ac2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0ce670 0x7f7acc15eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36142/0 (socket says 172.21.15.62:36142) 2024-04-25T09:04:25.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7ac2000640 1 -- 172.21.15.62:0/4099008785 learned_addr learned my addr 172.21.15.62:0/4099008785 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:25.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7acb400640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0cea50 0x7f7acc0c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7acaa00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc0b2c10 0x7f7acc15a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7ac2000640 1 -- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0cea50 msgr2=0x7f7acc0c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7ac2000640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0cea50 0x7f7acc0c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7ac2000640 1 -- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc0b2c10 msgr2=0x7f7acc15a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7ac2000640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc0b2c10 0x7f7acc15a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7ac2000640 1 -- 172.21.15.62:0/4099008785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc160150 con 0x7f7acc0ce670 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.396+0000 7f7acaa00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc0b2c10 0x7f7acc15a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.397+0000 7f7ac2000640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0ce670 0x7f7acc15eaf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4000c00 tx=0x7f7ab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.397+0000 7f7ac3400640 1 -- 172.21.15.62:0/4099008785 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4041020 con 0x7f7acc0ce670 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.397+0000 7f7ac3400640 1 -- 172.21.15.62:0/4099008785 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab40306c0 con 0x7f7acc0ce670 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.397+0000 7f7ac3400640 1 -- 172.21.15.62:0/4099008785 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4037040 con 0x7f7acc0ce670 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.397+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc15f540 con 0x7f7acc0ce670 2024-04-25T09:04:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.398+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc15fde0 con 0x7f7acc0ce670 2024-04-25T09:04:25.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.399+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7acc15f930 con 0x7f7acc0ce670 2024-04-25T09:04:25.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.401+0000 7f7ac3400640 1 -- 172.21.15.62:0/4099008785 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ab4044020 con 0x7f7acc0ce670 2024-04-25T09:04:25.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.402+0000 7f7ac3400640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7aa806b4d0 0x7f7aa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.402+0000 7f7ac3400640 1 -- 172.21.15.62:0/4099008785 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7ab40aadd0 con 0x7f7acc0ce670 2024-04-25T09:04:25.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.403+0000 7f7acaa00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7aa806b4d0 0x7f7aa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.404+0000 7f7acaa00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7aa806b4d0 0x7f7aa806d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009aa0 tx=0x7f7abc0039f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.406+0000 7f7ac3400640 1 -- 172.21.15.62:0/4099008785 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ab404b030 con 0x7f7acc0ce670 2024-04-25T09:04:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:25.637+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7acc04ef20 con 0x7f7acc0ce670 2024-04-25T09:04:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.192+0000 7f7ac3400640 1 -- 172.21.15.62:0/4099008785 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7acc160150 con 0x7f7acc0ce670 2024-04-25T09:04:26.180 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T09:04:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7aa806b4d0 msgr2=0x7f7aa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7aa806b4d0 0x7f7aa806d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009aa0 tx=0x7f7abc0039f0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0ce670 msgr2=0x7f7acc15eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0ce670 0x7f7acc15eaf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4000c00 tx=0x7f7ab40046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 shutdown_connections 2024-04-25T09:04:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7aa806b4d0 0x7f7aa806d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7acc0ce670 0x7f7acc15eaf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7acc0cea50 0x7f7acc0c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 --2- 172.21.15.62:0/4099008785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc0b2c10 0x7f7acc15a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.196+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 >> 172.21.15.62:0/4099008785 conn(0x7f7acc0d39c0 msgr2=0x7f7acc0d1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.197+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 shutdown_connections 2024-04-25T09:04:26.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.197+0000 7f7acbe00640 1 -- 172.21.15.62:0/4099008785 wait complete. 2024-04-25T09:04:26.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:26.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.404+0000 7fc4ca000640 1 Processor -- start 2024-04-25T09:04:26.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.404+0000 7fc4ca000640 1 -- start start 2024-04-25T09:04:26.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.406+0000 7fc4ca000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4c415dd50 0x7fc4c406d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.406+0000 7fc4ca000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc4c40ae7e0 0x7fc4c4162270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.406+0000 7fc4ca000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c40aec20 con 0x7fc4c40ceac0 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.406+0000 7fc4ca000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c406e120 con 0x7fc4c415dd50 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.406+0000 7fc4ca000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c406db60 con 0x7fc4c40ae7e0 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.406+0000 7fc4c3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4c415dd50 0x7fc4c406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.406+0000 7fc4c3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc4c415dd50 0x7fc4c406d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36146/0 (socket says 172.21.15.62:36146) 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.406+0000 7fc4c3400640 1 -- 172.21.15.62:0/2667662832 learned_addr learned my addr 172.21.15.62:0/2667662832 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.407+0000 7fc4c3400640 1 -- 172.21.15.62:0/2667662832 >> v2:172.21.15.121:3300/0 conn(0x7fc4c40ae7e0 msgr2=0x7fc4c4162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.407+0000 7fc4c3400640 1 --2- 172.21.15.62:0/2667662832 >> v2:172.21.15.121:3300/0 conn(0x7fc4c40ae7e0 0x7fc4c4162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.407+0000 7fc4c3400640 1 -- 172.21.15.62:0/2667662832 >> v1:172.21.15.121:6789/0 conn(0x7fc4c40ceac0 legacy=0x7fc4c40cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.407+0000 7fc4c3400640 1 -- 172.21.15.62:0/2667662832 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c406de40 con 0x7fc4c415dd50 2024-04-25T09:04:26.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.408+0000 7fc4c3400640 1 --2- 172.21.15.62:0/2667662832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c406d530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40cd3c0 tx=0x7fc4b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455c535bcb5938b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.408+0000 7fc4c2000640 1 -- 172.21.15.62:0/2667662832 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4037070 con 0x7fc4c415dd50 2024-04-25T09:04:26.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.408+0000 7fc4c2000640 1 -- 172.21.15.62:0/2667662832 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b4032440 con 0x7fc4c415dd50 2024-04-25T09:04:26.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.408+0000 7fc4c2000640 1 -- 172.21.15.62:0/2667662832 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4032810 con 0x7fc4c415dd50 2024-04-25T09:04:26.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.409+0000 7fc4ca000640 1 -- 172.21.15.62:0/2667662832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 msgr2=0x7fc4c406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.409+0000 7fc4ca000640 1 --2- 172.21.15.62:0/2667662832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c406d530 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40cd3c0 tx=0x7fc4b402a230 comp rx=0 tx=0).stop 2024-04-25T09:04:26.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.409+0000 7fc4ca000640 1 -- 172.21.15.62:0/2667662832 shutdown_connections 2024-04-25T09:04:26.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.409+0000 7fc4ca000640 1 --2- 172.21.15.62:0/2667662832 >> v2:172.21.15.121:3300/0 conn(0x7fc4c40ae7e0 0x7fc4c4162270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.409+0000 7fc4ca000640 1 --2- 172.21.15.62:0/2667662832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c406d530 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.409+0000 7fc4ca000640 1 -- 172.21.15.62:0/2667662832 >> 172.21.15.62:0/2667662832 conn(0x7fc4c40668f0 msgr2=0x7fc4c4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.409+0000 7fc4ca000640 1 -- 172.21.15.62:0/2667662832 shutdown_connections 2024-04-25T09:04:26.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.409+0000 7fc4ca000640 1 -- 172.21.15.62:0/2667662832 wait complete. 2024-04-25T09:04:26.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.410+0000 7fc4ca000640 1 Processor -- start 2024-04-25T09:04:26.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.411+0000 7fc4ca000640 1 -- start start 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4ca000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c40ae7e0 0x7fc4c41515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4ca000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c40ceac0 0x7fc4c40c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c40c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4ca000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c415e130 con 0x7fc4c415dd50 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4ca000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4161d50 con 0x7fc4c40ceac0 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4ca000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c415e2b0 con 0x7fc4c40ae7e0 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c40c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c40c4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36152/0 (socket says 172.21.15.62:36152) 2024-04-25T09:04:26.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3e00640 1 -- 172.21.15.62:0/268481785 learned_addr learned my addr 172.21.15.62:0/268481785 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:26.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3400640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c40ae7e0 0x7fc4c41515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c2a00640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c40ceac0 0x7fc4c40c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3e00640 1 -- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c40ae7e0 msgr2=0x7fc4c41515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3e00640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c40ae7e0 0x7fc4c41515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3e00640 1 -- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c40ceac0 msgr2=0x7fc4c40c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3e00640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c40ceac0 0x7fc4c40c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c3e00640 1 -- 172.21.15.62:0/268481785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c40c5a90 con 0x7fc4c415dd50 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.412+0000 7fc4c2a00640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c40ceac0 0x7fc4c40c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.413+0000 7fc4c3e00640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c40c4bc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00c780 tx=0x7fc4bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.413+0000 7fc4bb400640 1 -- 172.21.15.62:0/268481785 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc017070 con 0x7fc4c415dd50 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.413+0000 7fc4bb400640 1 -- 172.21.15.62:0/268481785 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc003da0 con 0x7fc4c415dd50 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.413+0000 7fc4bb400640 1 -- 172.21.15.62:0/268481785 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc0052b0 con 0x7fc4c415dd50 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.413+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c4157e20 con 0x7fc4c415dd50 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.413+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c40c57b0 con 0x7fc4c415dd50 2024-04-25T09:04:26.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.414+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c4072c70 con 0x7fc4c415dd50 2024-04-25T09:04:26.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.415+0000 7fc4bb400640 1 -- 172.21.15.62:0/268481785 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4bc005450 con 0x7fc4c415dd50 2024-04-25T09:04:26.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.416+0000 7fc4bb400640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc4a406b4d0 0x7fc4a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.416+0000 7fc4bb400640 1 -- 172.21.15.62:0/268481785 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4bc089890 con 0x7fc4c415dd50 2024-04-25T09:04:26.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.417+0000 7fc4c3400640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc4a406b4d0 0x7fc4a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.418+0000 7fc4c3400640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc4a406b4d0 0x7fc4a406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4034000 tx=0x7fc4b402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.421+0000 7fc4bb400640 1 -- 172.21.15.62:0/268481785 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4bc058670 con 0x7fc4c415dd50 2024-04-25T09:04:26.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.612+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4c40ade60 con 0x7fc4c415dd50 2024-04-25T09:04:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.614+0000 7fc4bb400640 1 -- 172.21.15.62:0/268481785 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fc4bc057f70 con 0x7fc4c415dd50 2024-04-25T09:04:26.603 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:26.603 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:04:25.639037+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:23.626010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:24.669175+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc4a406b4d0 msgr2=0x7fc4a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc4a406b4d0 0x7fc4a406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4034000 tx=0x7fc4b402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 msgr2=0x7fc4c40c4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c40c4bc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00c780 tx=0x7fc4bc00cc40 comp rx=0 tx=0).stop 2024-04-25T09:04:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 shutdown_connections 2024-04-25T09:04:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc4a406b4d0 0x7fc4a406d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc4c415dd50 0x7fc4c40c4bc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4c40ceac0 0x7fc4c40c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 --2- 172.21.15.62:0/268481785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc4c40ae7e0 0x7fc4c41515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 >> 172.21.15.62:0/268481785 conn(0x7fc4c40668f0 msgr2=0x7fc4c415e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 shutdown_connections 2024-04-25T09:04:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.617+0000 7fc4ca000640 1 -- 172.21.15.62:0/268481785 wait complete. 2024-04-25T09:04:26.624 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.845+0000 7f132fe00640 1 Processor -- start 2024-04-25T09:04:26.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.845+0000 7f132fe00640 1 -- start start 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.847+0000 7f132fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f133006d900 0x7f133015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.847+0000 7f132fe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133015faa0 con 0x7f1330144c10 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.847+0000 7f132fe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133015fd80 con 0x7f13300cee40 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.847+0000 7f132fe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1330144ff0 con 0x7f133006d900 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.847+0000 7f132ea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1330144c10 0x7f133006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41398/0 (socket says 172.21.15.62:41398) 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.847+0000 7f132ea00640 1 -- 172.21.15.62:0/88728307 learned_addr learned my addr 172.21.15.62:0/88728307 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.847+0000 7f132e000640 1 --2- 172.21.15.62:0/88728307 >> v2:172.21.15.121:3300/0 conn(0x7f133006d900 0x7f133015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.847+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906465966 0 0) 0x7f133015faa0 con 0x7f1330144c10 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1314003520 con 0x7f1330144c10 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851647673 0 0) 0x7f1314003520 con 0x7f1330144c10 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 >> v2:172.21.15.121:3300/0 conn(0x7f133006d900 msgr2=0x7f133015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 --2- 172.21.15.62:0/88728307 >> v2:172.21.15.121:3300/0 conn(0x7f133006d900 0x7f133015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 >> v1:172.21.15.121:6789/0 conn(0x7f13300cee40 legacy=0x7f13300b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1330160060 con 0x7f1330144c10 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1320001670 con 0x7f1330144c10 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 >> v1:172.21.15.62:6789/0 conn(0x7f1330144c10 legacy=0x7f133006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13140039b0 0x7f1314003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314004800 0x7f1314004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13140052e0 0x7f131400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1330144ff0 con 0x7f1314004800 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131400c1b0 con 0x7f13140039b0 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131400c3b0 con 0x7f13140052e0 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.848+0000 7f132e000640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314004800 0x7f1314004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.849+0000 7f132ea00640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13140039b0 0x7f1314003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.849+0000 7f132f400640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13140052e0 0x7f131400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.849+0000 7f132e000640 1 -- 172.21.15.62:0/88728307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13140052e0 msgr2=0x7f131400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.849+0000 7f132e000640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f13140052e0 0x7f131400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.849+0000 7f132e000640 1 -- 172.21.15.62:0/88728307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13140039b0 msgr2=0x7f1314003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.849+0000 7f132e000640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13140039b0 0x7f1314003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.849+0000 7f132e000640 1 -- 172.21.15.62:0/88728307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1314003520 con 0x7f1314004800 2024-04-25T09:04:26.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.849+0000 7f132f400640 1 -- 172.21.15.62:0/88728307 reap_dead start 2024-04-25T09:04:26.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.850+0000 7f132e000640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314004800 0x7f1314004c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ba00 tx=0x7f1314005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe4e0921714a6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.850+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c030070 con 0x7f1314004800 2024-04-25T09:04:26.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.850+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c02ad70 con 0x7f1314004800 2024-04-25T09:04:26.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.850+0000 7f132d600640 1 -- 172.21.15.62:0/88728307 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00b600 con 0x7f1314004800 2024-04-25T09:04:26.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.851+0000 7f132fe00640 1 -- 172.21.15.62:0/88728307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314004800 msgr2=0x7f1314004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.851+0000 7f132fe00640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314004800 0x7f1314004c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ba00 tx=0x7f1314005830 comp rx=0 tx=0).stop 2024-04-25T09:04:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.851+0000 7f132fe00640 1 -- 172.21.15.62:0/88728307 shutdown_connections 2024-04-25T09:04:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.851+0000 7f132fe00640 1 --2- 172.21.15.62:0/88728307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1314004800 0x7f1314004c80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.851+0000 7f132fe00640 1 -- 172.21.15.62:0/88728307 >> 172.21.15.62:0/88728307 conn(0x7f13300b14d0 msgr2=0x7f13300b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.851+0000 7f132fe00640 1 -- 172.21.15.62:0/88728307 shutdown_connections 2024-04-25T09:04:26.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.851+0000 7f132fe00640 1 -- 172.21.15.62:0/88728307 wait complete. 2024-04-25T09:04:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.853+0000 7f132fe00640 1 Processor -- start 2024-04-25T09:04:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.853+0000 7f132fe00640 1 -- start start 2024-04-25T09:04:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1314004800 0x7f13301f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301f3310 0x7f133013e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133013edd0 0x7f133015ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13300b41c0 con 0x7f133013edd0 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1330160340 con 0x7f13301f3310 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133015f450 con 0x7f1314004800 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133013edd0 0x7f133015ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133013edd0 0x7f133015ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36166/0 (socket says 172.21.15.62:36166) 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.854+0000 7f132f400640 1 -- 172.21.15.62:0/4112330303 learned_addr learned my addr 172.21.15.62:0/4112330303 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132ea00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1314004800 0x7f13301f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132e000640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301f3310 0x7f133013e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132f400640 1 -- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1314004800 msgr2=0x7f13301f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132f400640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1314004800 0x7f13301f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132f400640 1 -- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301f3310 msgr2=0x7f133013e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132f400640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301f3310 0x7f133013e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132f400640 1 -- 172.21.15.62:0/4112330303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133013f5e0 con 0x7f133013edd0 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132ea00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1314004800 0x7f13301f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132e000640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301f3310 0x7f133013e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:26.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132f400640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133013edd0 0x7f133015ab50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f13300c5580 tx=0x7f132800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f1327400640 1 -- 172.21.15.62:0/4112330303 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328003970 con 0x7f133013edd0 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f1327400640 1 -- 172.21.15.62:0/4112330303 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1328018070 con 0x7f133013edd0 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133013f8c0 con 0x7f133013edd0 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.855+0000 7f1327400640 1 -- 172.21.15.62:0/4112330303 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328003d70 con 0x7f133013edd0 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.856+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133013fe80 con 0x7f133013edd0 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.857+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133004ef20 con 0x7f133013edd0 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.858+0000 7f1327400640 1 -- 172.21.15.62:0/4112330303 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13280050d0 con 0x7f133013edd0 2024-04-25T09:04:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.859+0000 7f1327400640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f131006b4d0 0x7f131006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.859+0000 7f1327400640 1 -- 172.21.15.62:0/4112330303 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1328089fe0 con 0x7f133013edd0 2024-04-25T09:04:26.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.859+0000 7f132ea00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f131006b4d0 0x7f131006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.860+0000 7f132ea00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f131006b4d0 0x7f131006d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1320001200 tx=0x7f1320001110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:26.863+0000 7f1327400640 1 -- 172.21.15.62:0/4112330303 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1328058de0 con 0x7f133013edd0 2024-04-25T09:04:27.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.062+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13301f36f0 con 0x7f133013edd0 2024-04-25T09:04:27.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.065+0000 7f1327400640 1 -- 172.21.15.62:0/4112330303 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f13280586e0 con 0x7f133013edd0 2024-04-25T09:04:27.053 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:27.053 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:04:25.639037+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:23.626010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:24.669175+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.067+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f131006b4d0 msgr2=0x7f131006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.067+0000 7f132fe00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f131006b4d0 0x7f131006d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1320001200 tx=0x7f1320001110 comp rx=0 tx=0).stop 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.067+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133013edd0 msgr2=0x7f133015ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.067+0000 7f132fe00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133013edd0 0x7f133015ab50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f13300c5580 tx=0x7f132800eb90 comp rx=0 tx=0).stop 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.068+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 shutdown_connections 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.068+0000 7f132fe00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f131006b4d0 0x7f131006d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.068+0000 7f132fe00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f133013edd0 0x7f133015ab50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.068+0000 7f132fe00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13301f3310 0x7f133013e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.068+0000 7f132fe00640 1 --2- 172.21.15.62:0/4112330303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1314004800 0x7f13301f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.068+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 >> 172.21.15.62:0/4112330303 conn(0x7f13300b14d0 msgr2=0x7f13300b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:27.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.068+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 shutdown_connections 2024-04-25T09:04:27.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.068+0000 7f132fe00640 1 -- 172.21.15.62:0/4112330303 wait complete. 2024-04-25T09:04:27.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:04:27.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.304+0000 7fe611200640 1 Processor -- start 2024-04-25T09:04:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.304+0000 7fe611200640 1 -- start start 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.305+0000 7fe611200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b2d80 0x7fe60c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe611200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe60c0b36a0 0x7fe60c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe611200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe60c0b54d0 0x7fe60c0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe611200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60c0b4430 con 0x7fe60c0b36a0 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe611200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60c0b4710 con 0x7fe60c0b54d0 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe611200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0b49f0 con 0x7fe60c0b2d80 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe60aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe60c0b36a0 0x7fe60c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe60aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe60c0b36a0 0x7fe60c0ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36180/0 (socket says 172.21.15.62:36180) 2024-04-25T09:04:27.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe60aa00640 1 -- 172.21.15.62:0/2337874520 learned_addr learned my addr 172.21.15.62:0/2337874520 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:27.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe60aa00640 1 -- 172.21.15.62:0/2337874520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b2d80 msgr2=0x7fe60c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe60b400640 1 --2- 172.21.15.62:0/2337874520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b2d80 0x7fe60c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe60be00640 1 --2- 172.21.15.62:0/2337874520 >> v2:172.21.15.121:3300/0 conn(0x7fe60c0b54d0 0x7fe60c0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.306+0000 7fe60aa00640 1 --2- 172.21.15.62:0/2337874520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b2d80 0x7fe60c0b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.307+0000 7fe60aa00640 1 -- 172.21.15.62:0/2337874520 >> v2:172.21.15.121:3300/0 conn(0x7fe60c0b54d0 msgr2=0x7fe60c0700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.307+0000 7fe60aa00640 1 --2- 172.21.15.62:0/2337874520 >> v2:172.21.15.121:3300/0 conn(0x7fe60c0b54d0 0x7fe60c0700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.307+0000 7fe60aa00640 1 -- 172.21.15.62:0/2337874520 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c0b4cd0 con 0x7fe60c0b36a0 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.308+0000 7fe60aa00640 1 --2- 172.21.15.62:0/2337874520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b36a0 0x7fe60c0ce2b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe60c072100 tx=0x7fe5f402c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbef0dea4742d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.308+0000 7fe60a000640 1 -- 172.21.15.62:0/2337874520 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4038070 con 0x7fe60c0b36a0 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.308+0000 7fe60a000640 1 -- 172.21.15.62:0/2337874520 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f4031710 con 0x7fe60c0b36a0 2024-04-25T09:04:27.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.308+0000 7fe60a000640 1 -- 172.21.15.62:0/2337874520 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4031ae0 con 0x7fe60c0b36a0 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 -- 172.21.15.62:0/2337874520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b36a0 msgr2=0x7fe60c0ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 --2- 172.21.15.62:0/2337874520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b36a0 0x7fe60c0ce2b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe60c072100 tx=0x7fe5f402c3f0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 -- 172.21.15.62:0/2337874520 shutdown_connections 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 --2- 172.21.15.62:0/2337874520 >> v2:172.21.15.121:3300/0 conn(0x7fe60c0b54d0 0x7fe60c0700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 --2- 172.21.15.62:0/2337874520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b36a0 0x7fe60c0ce2b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 --2- 172.21.15.62:0/2337874520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b2d80 0x7fe60c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 -- 172.21.15.62:0/2337874520 >> 172.21.15.62:0/2337874520 conn(0x7fe60c066f20 msgr2=0x7fe60c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 -- 172.21.15.62:0/2337874520 shutdown_connections 2024-04-25T09:04:27.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.309+0000 7fe611200640 1 -- 172.21.15.62:0/2337874520 wait complete. 2024-04-25T09:04:27.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.311+0000 7fe611200640 1 Processor -- start 2024-04-25T09:04:27.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.311+0000 7fe611200640 1 -- start start 2024-04-25T09:04:27.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.312+0000 7fe611200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe60c0b2d80 0x7fe60c0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.312+0000 7fe611200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b36a0 0x7fe60c14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.312+0000 7fe611200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b54d0 0x7fe60c14a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.312+0000 7fe611200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0ce7f0 con 0x7fe60c0b54d0 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.312+0000 7fe611200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0b40e0 con 0x7fe60c0b2d80 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.312+0000 7fe611200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60c0ce970 con 0x7fe60c0b36a0 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.312+0000 7fe60be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b54d0 0x7fe60c14a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b54d0 0x7fe60c14a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36184/0 (socket says 172.21.15.62:36184) 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60be00640 1 -- 172.21.15.62:0/1437402478 learned_addr learned my addr 172.21.15.62:0/1437402478 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60b400640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe60c0b2d80 0x7fe60c0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60aa00640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b36a0 0x7fe60c14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60be00640 1 -- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b36a0 msgr2=0x7fe60c14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60be00640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b36a0 0x7fe60c14a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60be00640 1 -- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe60c0b2d80 msgr2=0x7fe60c0b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60be00640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe60c0b2d80 0x7fe60c0b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60be00640 1 -- 172.21.15.62:0/1437402478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c153870 con 0x7fe60c0b54d0 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60b400640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe60c0b2d80 0x7fe60c0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:27.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60aa00640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b36a0 0x7fe60c14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:27.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.313+0000 7fe60be00640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b54d0 0x7fe60c14a620 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe60400c7f0 tx=0x7fe60400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.314+0000 7fe603400640 1 -- 172.21.15.62:0/1437402478 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604015070 con 0x7fe60c0b54d0 2024-04-25T09:04:27.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.314+0000 7fe603400640 1 -- 172.21.15.62:0/1437402478 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604004220 con 0x7fe60c0b54d0 2024-04-25T09:04:27.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.314+0000 7fe603400640 1 -- 172.21.15.62:0/1437402478 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604007620 con 0x7fe60c0b54d0 2024-04-25T09:04:27.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.314+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60c153b50 con 0x7fe60c0b54d0 2024-04-25T09:04:27.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.314+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60c154110 con 0x7fe60c0b54d0 2024-04-25T09:04:27.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.315+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60c06abe0 con 0x7fe60c0b54d0 2024-04-25T09:04:27.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.316+0000 7fe603400640 1 -- 172.21.15.62:0/1437402478 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6040079d0 con 0x7fe60c0b54d0 2024-04-25T09:04:27.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.317+0000 7fe603400640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5e806b4d0 0x7fe5e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.317+0000 7fe603400640 1 -- 172.21.15.62:0/1437402478 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe60408a370 con 0x7fe60c0b54d0 2024-04-25T09:04:27.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.318+0000 7fe60b400640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5e806b4d0 0x7fe5e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.319+0000 7fe60b400640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5e806b4d0 0x7fe5e806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe60c1f6d60 tx=0x7fe5fc007310 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.321+0000 7fe603400640 1 -- 172.21.15.62:0/1437402478 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe604059150 con 0x7fe60c0b54d0 2024-04-25T09:04:27.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:27.559+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe60c04ef20 con 0x7fe60c0b54d0 2024-04-25T09:04:28.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.225+0000 7fe603400640 1 -- 172.21.15.62:0/1437402478 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe604058a50 con 0x7fe60c0b54d0 2024-04-25T09:04:28.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5e806b4d0 msgr2=0x7fe5e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5e806b4d0 0x7fe5e806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe60c1f6d60 tx=0x7fe5fc007310 comp rx=0 tx=0).stop 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b54d0 msgr2=0x7fe60c14a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b54d0 0x7fe60c14a620 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe60400c7f0 tx=0x7fe60400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 shutdown_connections 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5e806b4d0 0x7fe5e806d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe60c0b54d0 0x7fe60c14a620 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe60c0b36a0 0x7fe60c14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 --2- 172.21.15.62:0/1437402478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe60c0b2d80 0x7fe60c0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.230+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 >> 172.21.15.62:0/1437402478 conn(0x7fe60c066f20 msgr2=0x7fe60c0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.231+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 shutdown_connections 2024-04-25T09:04:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.231+0000 7fe611200640 1 -- 172.21.15.62:0/1437402478 wait complete. 2024-04-25T09:04:28.243 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:04:28.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:04:28.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.471+0000 7ffa7f200640 1 Processor -- start 2024-04-25T09:04:28.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.472+0000 7ffa7f200640 1 -- start start 2024-04-25T09:04:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.473+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 0x7ffa780d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.473+0000 7ffa7f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780ce780 con 0x7ffa780d3960 2024-04-25T09:04:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.473+0000 7ffa7f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780d3440 con 0x7ffa780d2fb0 2024-04-25T09:04:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.473+0000 7ffa7f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780ce1c0 con 0x7ffa7815dd50 2024-04-25T09:04:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.473+0000 7ffa7e800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ffa780d3960 0x7ffa780d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41414/0 (socket says 172.21.15.62:41414) 2024-04-25T09:04:28.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.473+0000 7ffa7e800640 1 -- 172.21.15.62:0/2964962320 learned_addr learned my addr 172.21.15.62:0/2964962320 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.473+0000 7ffa7de00640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 0x7ffa780d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771719773 0 0) 0x7ffa780ce780 con 0x7ffa780d3960 2024-04-25T09:04:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa64003520 con 0x7ffa780d3960 2024-04-25T09:04:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468703044 0 0) 0x7ffa64003520 con 0x7ffa780d3960 2024-04-25T09:04:28.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 msgr2=0x7ffa780d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 0x7ffa780d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 >> v1:172.21.15.121:6789/0 conn(0x7ffa780d2fb0 legacy=0x7ffa780ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780ce4a0 con 0x7ffa780d3960 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ffa74001670 con 0x7ffa780d3960 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 >> v1:172.21.15.62:6789/0 conn(0x7ffa780d3960 legacy=0x7ffa780d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 0x7ffa640039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa64003fb0 0x7ffa64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa64004d40 0x7ffa640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780ce1c0 con 0x7ffa64004d40 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6400bb00 con 0x7ffa64003fb0 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6400bd00 con 0x7ffa7815dd50 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.474+0000 7ffa7e800640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa64004d40 0x7ffa640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.475+0000 7ffa7de00640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 0x7ffa640039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.475+0000 7ffa7d400640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa64003fb0 0x7ffa64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.475+0000 7ffa7e800640 1 -- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 msgr2=0x7ffa640039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.475+0000 7ffa7e800640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 0x7ffa640039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.475+0000 7ffa7e800640 1 -- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa64003fb0 msgr2=0x7ffa64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.475+0000 7ffa7e800640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa64003fb0 0x7ffa64004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.475+0000 7ffa7e800640 1 -- 172.21.15.62:0/2964962320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64003520 con 0x7ffa64004d40 2024-04-25T09:04:28.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.475+0000 7ffa7de00640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa7815dd50 0x7ffa640039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.476+0000 7ffa7e800640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa64004d40 0x7ffa640051c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c110 tx=0x7ffa64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c6578cf222d1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.476+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74007c40 con 0x7ffa64004d40 2024-04-25T09:04:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.476+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa74005ad0 con 0x7ffa64004d40 2024-04-25T09:04:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.476+0000 7ffa7ca00640 1 -- 172.21.15.62:0/2964962320 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74007070 con 0x7ffa64004d40 2024-04-25T09:04:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.477+0000 7ffa7f200640 1 -- 172.21.15.62:0/2964962320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa64004d40 msgr2=0x7ffa640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.477+0000 7ffa7f200640 1 --2- 172.21.15.62:0/2964962320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa64004d40 0x7ffa640051c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c110 tx=0x7ffa64005830 comp rx=0 tx=0).stop 2024-04-25T09:04:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.477+0000 7ffa7e800640 1 -- 172.21.15.62:0/2964962320 reap_dead start 2024-04-25T09:04:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.477+0000 7ffa7f200640 1 -- 172.21.15.62:0/2964962320 shutdown_connections 2024-04-25T09:04:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.477+0000 7ffa7f200640 1 -- 172.21.15.62:0/2964962320 >> 172.21.15.62:0/2964962320 conn(0x7ffa7806b000 msgr2=0x7ffa7806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.477+0000 7ffa7f200640 1 -- 172.21.15.62:0/2964962320 shutdown_connections 2024-04-25T09:04:28.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.477+0000 7ffa7f200640 1 -- 172.21.15.62:0/2964962320 wait complete. 2024-04-25T09:04:28.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.479+0000 7ffa7f200640 1 Processor -- start 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.479+0000 7ffa7f200640 1 -- start start 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa64004d40 0x7ffa780b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa7814cd90 0x7ffa781f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa781f72c0 0x7ffa7815f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780d26d0 con 0x7ffa781f72c0 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780d2550 con 0x7ffa7814cd90 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780d2850 con 0x7ffa64004d40 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa781f72c0 0x7ffa7815f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa781f72c0 0x7ffa7815f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36204/0 (socket says 172.21.15.62:36204) 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.480+0000 7ffa7e800640 1 -- 172.21.15.62:0/117876737 learned_addr learned my addr 172.21.15.62:0/117876737 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:28.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa7d400640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa7814cd90 0x7ffa781f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa7de00640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa64004d40 0x7ffa780b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa7e800640 1 -- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa64004d40 msgr2=0x7ffa780b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa7e800640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa64004d40 0x7ffa780b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa7e800640 1 -- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa7814cd90 msgr2=0x7ffa781f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa7e800640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa7814cd90 0x7ffa781f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa7e800640 1 -- 172.21.15.62:0/117876737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780c8f10 con 0x7ffa781f72c0 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa7e800640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa781f72c0 0x7ffa7815f1d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c230 tx=0x7ffa780d3610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa6e000640 1 -- 172.21.15.62:0/117876737 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74007c40 con 0x7ffa781f72c0 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa6e000640 1 -- 172.21.15.62:0/117876737 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa74042070 con 0x7ffa781f72c0 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.481+0000 7ffa6e000640 1 -- 172.21.15.62:0/117876737 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74007470 con 0x7ffa781f72c0 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.482+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa780c9190 con 0x7ffa781f72c0 2024-04-25T09:04:28.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.482+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa780c9750 con 0x7ffa781f72c0 2024-04-25T09:04:28.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.483+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7804ef20 con 0x7ffa781f72c0 2024-04-25T09:04:28.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.484+0000 7ffa6e000640 1 -- 172.21.15.62:0/117876737 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa74047460 con 0x7ffa781f72c0 2024-04-25T09:04:28.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.485+0000 7ffa6e000640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa5c06b500 0x7ffa5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.485+0000 7ffa6e000640 1 -- 172.21.15.62:0/117876737 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa740aa770 con 0x7ffa781f72c0 2024-04-25T09:04:28.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.486+0000 7ffa7de00640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa5c06b500 0x7ffa5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.487+0000 7ffa7de00640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa5c06b500 0x7ffa5c06d9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa7815a1e0 tx=0x7ffa7000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.490+0000 7ffa6e000640 1 -- 172.21.15.62:0/117876737 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa780c8f10 con 0x7ffa781f72c0 2024-04-25T09:04:28.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:28.735+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffa780b9950 con 0x7ffa781f72c0 2024-04-25T09:04:29.134 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-04-25T09:04:29.147+0000 7f06c6200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:29.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.236+0000 7ffa6e000640 1 -- 172.21.15.62:0/117876737 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffa74030380 con 0x7ffa781f72c0 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa5c06b500 msgr2=0x7ffa5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa5c06b500 0x7ffa5c06d9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa7815a1e0 tx=0x7ffa7000b040 comp rx=0 tx=0).stop 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa781f72c0 msgr2=0x7ffa7815f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa781f72c0 0x7ffa7815f1d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400c230 tx=0x7ffa780d3610 comp rx=0 tx=0).stop 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 shutdown_connections 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa5c06b500 0x7ffa5c06d9c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa781f72c0 0x7ffa7815f1d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa7814cd90 0x7ffa781f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 --2- 172.21.15.62:0/117876737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa64004d40 0x7ffa780b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 >> 172.21.15.62:0/117876737 conn(0x7ffa7806b000 msgr2=0x7ffa780cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:29.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 shutdown_connections 2024-04-25T09:04:29.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.240+0000 7ffa7f200640 1 -- 172.21.15.62:0/117876737 wait complete. 2024-04-25T09:04:29.248 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:04:29.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:29.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.471+0000 7f4758a00640 1 Processor -- start 2024-04-25T09:04:29.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.472+0000 7f4758a00640 1 -- start start 2024-04-25T09:04:29.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.473+0000 7f4758a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f47540b56f0 0x7f4754160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.473+0000 7f4758a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47541627a0 con 0x7f47540b27a0 2024-04-25T09:04:29.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.473+0000 7f4758a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47540d3110 con 0x7f47540b6a90 2024-04-25T09:04:29.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.473+0000 7f4758a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47540cedc0 con 0x7f47540b56f0 2024-04-25T09:04:29.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.473+0000 7f4753200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f47540b27a0 0x7f4754162090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41428/0 (socket says 172.21.15.62:41428) 2024-04-25T09:04:29.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.473+0000 7f4753200640 1 -- 172.21.15.62:0/3421597282 learned_addr learned my addr 172.21.15.62:0/3421597282 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4752800640 1 --2- 172.21.15.62:0/3421597282 >> v2:172.21.15.121:3300/0 conn(0x7f47540b56f0 0x7f4754160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393122011 0 0) 0x7f47541627a0 con 0x7f47540b27a0 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4738003520 con 0x7f47540b27a0 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642896358 0 0) 0x7f4738003520 con 0x7f47540b27a0 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 >> v2:172.21.15.121:3300/0 conn(0x7f47540b56f0 msgr2=0x7f4754160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 --2- 172.21.15.62:0/3421597282 >> v2:172.21.15.121:3300/0 conn(0x7f47540b56f0 0x7f4754160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 >> v1:172.21.15.121:6789/0 conn(0x7f47540b6a90 legacy=0x7f4754161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540d33f0 con 0x7f47540b27a0 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f474c0019f0 con 0x7f47540b27a0 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 >> v1:172.21.15.62:6789/0 conn(0x7f47540b27a0 legacy=0x7f4754162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47380039b0 0x7f4738003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4738004800 0x7f4738004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47380051c0 0x7f473800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.474+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47540cedc0 con 0x7f47380051c0 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473800c060 con 0x7f4738004800 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473800c260 con 0x7f47380039b0 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4753200640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47380051c0 0x7f473800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4749e00640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4738004800 0x7f4738004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4753200640 1 -- 172.21.15.62:0/3421597282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47380039b0 msgr2=0x7f4738003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:29.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4753200640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47380039b0 0x7f4738003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4753200640 1 -- 172.21.15.62:0/3421597282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4738004800 msgr2=0x7f4738004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:29.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4753200640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4738004800 0x7f4738004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4753200640 1 -- 172.21.15.62:0/3421597282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738003520 con 0x7f47380051c0 2024-04-25T09:04:29.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.475+0000 7f4749e00640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4738004800 0x7f4738004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.476+0000 7f4753200640 1 -- 172.21.15.62:0/3421597282 reap_dead start 2024-04-25T09:04:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.476+0000 7f4753200640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47380051c0 0x7f473800b9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f474c00a810 tx=0x7f4738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374216c6cac3269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.476+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c00a460 con 0x7f47380051c0 2024-04-25T09:04:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.476+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c02f070 con 0x7f47380051c0 2024-04-25T09:04:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.476+0000 7f4751e00640 1 -- 172.21.15.62:0/3421597282 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c02bc90 con 0x7f47380051c0 2024-04-25T09:04:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.477+0000 7f4758a00640 1 -- 172.21.15.62:0/3421597282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47380051c0 msgr2=0x7f473800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.477+0000 7f4758a00640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47380051c0 0x7f473800b9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f474c00a810 tx=0x7f4738005830 comp rx=0 tx=0).stop 2024-04-25T09:04:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.477+0000 7f4758a00640 1 -- 172.21.15.62:0/3421597282 shutdown_connections 2024-04-25T09:04:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.477+0000 7f4758a00640 1 --2- 172.21.15.62:0/3421597282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47380051c0 0x7f473800b9c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.477+0000 7f4758a00640 1 -- 172.21.15.62:0/3421597282 >> 172.21.15.62:0/3421597282 conn(0x7f4754066d10 msgr2=0x7f4754067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.477+0000 7f4758a00640 1 -- 172.21.15.62:0/3421597282 shutdown_connections 2024-04-25T09:04:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.477+0000 7f4758a00640 1 -- 172.21.15.62:0/3421597282 wait complete. 2024-04-25T09:04:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.479+0000 7f4758a00640 1 Processor -- start 2024-04-25T09:04:29.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.479+0000 7f4758a00640 1 -- start start 2024-04-25T09:04:29.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4758a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47380051c0 0x7f475415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4758a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47540cafa0 0x7f475415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4758a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4754153020 0x7f475415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4758a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47540b2d60 con 0x7f47540cafa0 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4758a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47540cea50 con 0x7f47380051c0 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4758a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47540b2ee0 con 0x7f4754153020 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4749e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47540cafa0 0x7f475415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4749e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47540cafa0 0x7f475415e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36214/0 (socket says 172.21.15.62:36214) 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.480+0000 7f4749e00640 1 -- 172.21.15.62:0/2905932219 learned_addr learned my addr 172.21.15.62:0/2905932219 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4753200640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4754153020 0x7f475415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4752800640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47380051c0 0x7f475415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4749e00640 1 -- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4754153020 msgr2=0x7f475415a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4749e00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4754153020 0x7f475415a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4749e00640 1 -- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47380051c0 msgr2=0x7f475415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4749e00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47380051c0 0x7f475415dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4749e00640 1 -- 172.21.15.62:0/2905932219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540cb430 con 0x7f47540cafa0 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4749e00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47540cafa0 0x7f475415e120 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f473c00d540 tx=0x7f473c00da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f474b400640 1 -- 172.21.15.62:0/2905932219 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c005480 con 0x7f47540cafa0 2024-04-25T09:04:29.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f474b400640 1 -- 172.21.15.62:0/2905932219 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c017070 con 0x7f47540cafa0 2024-04-25T09:04:29.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f474b400640 1 -- 172.21.15.62:0/2905932219 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c003850 con 0x7f47540cafa0 2024-04-25T09:04:29.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.481+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47540cb710 con 0x7f47540cafa0 2024-04-25T09:04:29.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.482+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47540cbcd0 con 0x7f47540cafa0 2024-04-25T09:04:29.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.483+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475404ef20 con 0x7f47540cafa0 2024-04-25T09:04:29.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.484+0000 7f474b400640 1 -- 172.21.15.62:0/2905932219 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f473c003ab0 con 0x7f47540cafa0 2024-04-25T09:04:29.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.485+0000 7f474b400640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f473006b4d0 0x7f473006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.485+0000 7f474b400640 1 -- 172.21.15.62:0/2905932219 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f473c08a0a0 con 0x7f47540cafa0 2024-04-25T09:04:29.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.485+0000 7f4752800640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f473006b4d0 0x7f473006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.486+0000 7f4752800640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f473006b4d0 0x7f473006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4744009970 tx=0x7f4744008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.489+0000 7f474b400640 1 -- 172.21.15.62:0/2905932219 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f473c019310 con 0x7f47540cafa0 2024-04-25T09:04:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.730+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4754153490 con 0x7f47540cafa0 2024-04-25T09:04:29.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.731+0000 7f474b400640 1 -- 172.21.15.62:0/2905932219 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f473c058e80 con 0x7f47540cafa0 2024-04-25T09:04:29.719 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:29.719 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2179586213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2179586213},{"type":"v1","addr":"172.21.15.121:6836","nonce":2179586213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1425308084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1425308084},{"type":"v1","addr":"172.21.15.121:6837","nonce":1425308084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/231681595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":231681595},{"type":"v1","addr":"172.21.15.62:6834","nonce":231681595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:25.639815+0000","modified":"2024-04-25T09:04:28.736856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/274650142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":274650142},{"type":"v1","addr":"172.21.15.62:6835","nonce":274650142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:04:29.719 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 16 2024-04-25T09:04:29.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f473006b4d0 msgr2=0x7f473006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f473006b4d0 0x7f473006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4744009970 tx=0x7f4744008040 comp rx=0 tx=0).stop 2024-04-25T09:04:29.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47540cafa0 msgr2=0x7f475415e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47540cafa0 0x7f475415e120 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f473c00d540 tx=0x7f473c00da00 comp rx=0 tx=0).stop 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 shutdown_connections 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f473006b4d0 0x7f473006d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47540cafa0 0x7f475415e120 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4754153020 0x7f475415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 --2- 172.21.15.62:0/2905932219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47380051c0 0x7f475415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 >> 172.21.15.62:0/2905932219 conn(0x7f4754066d10 msgr2=0x7f47540b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 shutdown_connections 2024-04-25T09:04:29.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.734+0000 7f4758a00640 1 -- 172.21.15.62:0/2905932219 wait complete. 2024-04-25T09:04:29.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.951+0000 7f39fda00640 1 Processor -- start 2024-04-25T09:04:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.951+0000 7f39fda00640 1 -- start start 2024-04-25T09:04:29.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.953+0000 7f39fda00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39f8144c10 0x7f39f815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.953+0000 7f39fda00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f80cf2d0 con 0x7f39f80cee40 2024-04-25T09:04:29.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.953+0000 7f39fda00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f815faa0 con 0x7f39f806d900 2024-04-25T09:04:29.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.953+0000 7f39fda00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f8144ff0 con 0x7f39f8144c10 2024-04-25T09:04:29.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.954+0000 7f39f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39f8144c10 0x7f39f815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.954+0000 7f39f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39f8144c10 0x7f39f815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36222/0 (socket says 172.21.15.62:36222) 2024-04-25T09:04:29.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.954+0000 7f39f7e00640 1 -- 172.21.15.62:0/2638161923 learned_addr learned my addr 172.21.15.62:0/2638161923 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:29.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.954+0000 7f39f7e00640 1 -- 172.21.15.62:0/2638161923 >> v1:172.21.15.121:6789/0 conn(0x7f39f806d900 legacy=0x7f39f80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:29.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.954+0000 7f39f7e00640 1 -- 172.21.15.62:0/2638161923 >> v1:172.21.15.62:6789/0 conn(0x7f39f80cee40 legacy=0x7f39f806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.954+0000 7f39f7e00640 1 -- 172.21.15.62:0/2638161923 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f815fd80 con 0x7f39f8144c10 2024-04-25T09:04:29.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.955+0000 7f39f7e00640 1 --2- 172.21.15.62:0/2638161923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f8144c10 0x7f39f815f450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009870 tx=0x7f39e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9e6eb2d6745a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.956+0000 7f39f6a00640 1 -- 172.21.15.62:0/2638161923 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8037070 con 0x7f39f8144c10 2024-04-25T09:04:29.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.956+0000 7f39f6a00640 1 -- 172.21.15.62:0/2638161923 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e8032440 con 0x7f39f8144c10 2024-04-25T09:04:29.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.956+0000 7f39f6a00640 1 -- 172.21.15.62:0/2638161923 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8032810 con 0x7f39f8144c10 2024-04-25T09:04:29.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.956+0000 7f39fda00640 1 -- 172.21.15.62:0/2638161923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f8144c10 msgr2=0x7f39f815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.956+0000 7f39fda00640 1 --2- 172.21.15.62:0/2638161923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f8144c10 0x7f39f815f450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009870 tx=0x7f39e802a210 comp rx=0 tx=0).stop 2024-04-25T09:04:29.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.957+0000 7f39fda00640 1 -- 172.21.15.62:0/2638161923 shutdown_connections 2024-04-25T09:04:29.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.957+0000 7f39fda00640 1 --2- 172.21.15.62:0/2638161923 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f8144c10 0x7f39f815f450 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.957+0000 7f39fda00640 1 -- 172.21.15.62:0/2638161923 >> 172.21.15.62:0/2638161923 conn(0x7f39f80b14d0 msgr2=0x7f39f80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:29.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.957+0000 7f39fda00640 1 -- 172.21.15.62:0/2638161923 shutdown_connections 2024-04-25T09:04:29.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.957+0000 7f39fda00640 1 -- 172.21.15.62:0/2638161923 wait complete. 2024-04-25T09:04:29.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.958+0000 7f39fda00640 1 Processor -- start 2024-04-25T09:04:29.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.959+0000 7f39fda00640 1 -- start start 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39fda00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f806d900 0x7f39f813f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39fda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80cee40 0x7f39f81e6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39fda00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39f8144c10 0x7f39f81e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39fda00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80b41c0 con 0x7f39f80cee40 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39fda00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8160340 con 0x7f39f806d900 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39fda00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f80b4d60 con 0x7f39f8144c10 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80cee40 0x7f39f81e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80cee40 0x7f39f81e6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36224/0 (socket says 172.21.15.62:36224) 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7400640 1 -- 172.21.15.62:0/3214790773 learned_addr learned my addr 172.21.15.62:0/3214790773 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7e00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f806d900 0x7f39f813f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7400640 1 -- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39f8144c10 msgr2=0x7f39f81e6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39fd000640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39f8144c10 0x7f39f81e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7400640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39f8144c10 0x7f39f81e6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7400640 1 -- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f806d900 msgr2=0x7f39f813f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7400640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f806d900 0x7f39f813f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7400640 1 -- 172.21.15.62:0/3214790773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f8140fb0 con 0x7f39f80cee40 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39fd000640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39f8144c10 0x7f39f81e6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.960+0000 7f39f7e00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f806d900 0x7f39f813f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.961+0000 7f39f7400640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80cee40 0x7f39f81e6250 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39e400c400 tx=0x7f39e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.961+0000 7f39efe00640 1 -- 172.21.15.62:0/3214790773 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40052d0 con 0x7f39f80cee40 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.961+0000 7f39efe00640 1 -- 172.21.15.62:0/3214790773 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e4005490 con 0x7f39f80cee40 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.961+0000 7f39efe00640 1 -- 172.21.15.62:0/3214790773 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4007260 con 0x7f39f80cee40 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.961+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8141290 con 0x7f39f80cee40 2024-04-25T09:04:29.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.961+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f8141d70 con 0x7f39f80cee40 2024-04-25T09:04:29.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.962+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f804ef20 con 0x7f39f80cee40 2024-04-25T09:04:29.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.963+0000 7f39efe00640 1 -- 172.21.15.62:0/3214790773 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39e4007400 con 0x7f39f80cee40 2024-04-25T09:04:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.964+0000 7f39efe00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f39d806b520 0x7f39d806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.964+0000 7f39efe00640 1 -- 172.21.15.62:0/3214790773 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39e4089610 con 0x7f39f80cee40 2024-04-25T09:04:29.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.965+0000 7f39f7e00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f39d806b520 0x7f39d806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.966+0000 7f39f7e00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f39d806b520 0x7f39d806d9e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f39e8034000 tx=0x7f39e802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:29.969+0000 7f39efe00640 1 -- 172.21.15.62:0/3214790773 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39e4058600 con 0x7f39f80cee40 2024-04-25T09:04:30.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.172+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39f80b22b0 con 0x7f39f80cee40 2024-04-25T09:04:30.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.174+0000 7f39efe00640 1 -- 172.21.15.62:0/3214790773 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f39e4057f00 con 0x7f39f80cee40 2024-04-25T09:04:30.162 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:30.162 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":33,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:04:27.212449+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:23.626010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:24.669175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.176+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f39d806b520 msgr2=0x7f39d806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.176+0000 7f39fda00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f39d806b520 0x7f39d806d9e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f39e8034000 tx=0x7f39e802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.176+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80cee40 msgr2=0x7f39f81e6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.176+0000 7f39fda00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80cee40 0x7f39f81e6250 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39e400c400 tx=0x7f39e400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.177+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 shutdown_connections 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.177+0000 7f39fda00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f39d806b520 0x7f39d806d9e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.177+0000 7f39fda00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39f80cee40 0x7f39f81e6250 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.177+0000 7f39fda00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39f8144c10 0x7f39f81e6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.177+0000 7f39fda00640 1 --2- 172.21.15.62:0/3214790773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f806d900 0x7f39f813f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.177+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 >> 172.21.15.62:0/3214790773 conn(0x7f39f80b14d0 msgr2=0x7f39f815be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.177+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 shutdown_connections 2024-04-25T09:04:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.177+0000 7f39fda00640 1 -- 172.21.15.62:0/3214790773 wait complete. 2024-04-25T09:04:30.185 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:04:30.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.409+0000 7f8961e00640 1 Processor -- start 2024-04-25T09:04:30.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.410+0000 7f8961e00640 1 -- start start 2024-04-25T09:04:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.411+0000 7f8961e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 0x7f895c0d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.411+0000 7f8961e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c0d3110 con 0x7f895c0b6a90 2024-04-25T09:04:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.411+0000 7f8961e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c0d33f0 con 0x7f895c0b56f0 2024-04-25T09:04:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.411+0000 7f8961e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0cedc0 con 0x7f895c0b27a0 2024-04-25T09:04:30.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f895be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f895c0b6a90 0x7f895c0d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41440/0 (socket says 172.21.15.62:41440) 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f895be00640 1 -- 172.21.15.62:0/2920047969 learned_addr learned my addr 172.21.15.62:0/2920047969 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327202754 0 0) 0x7f895c0d3110 con 0x7f895c0b6a90 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f8961400640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 0x7f895c0d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f893c003520 con 0x7f895c0b6a90 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 153449683 0 0) 0x7f893c003520 con 0x7f895c0b6a90 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 msgr2=0x7f895c0d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f895b400640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 0x7f895c0d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.412+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 >> v1:172.21.15.121:6789/0 conn(0x7f895c0b56f0 legacy=0x7f895c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0d36d0 con 0x7f895c0b6a90 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f8944001670 con 0x7f895c0b6a90 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 >> v1:172.21.15.62:6789/0 conn(0x7f895c0b6a90 legacy=0x7f895c0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f893c0039e0 0x7f893c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893c004800 0x7f893c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 0x7f893c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0cedc0 con 0x7f893c004800 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c00bb50 con 0x7f893c0039e0 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c00bd20 con 0x7f895c0b27a0 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895be00640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893c004800 0x7f893c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f8960a00640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f893c0039e0 0x7f893c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f8961400640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 0x7f893c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895be00640 1 -- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 msgr2=0x7f893c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895be00640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 0x7f893c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895be00640 1 -- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f893c0039e0 msgr2=0x7f893c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895be00640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f893c0039e0 0x7f893c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.413+0000 7f895be00640 1 -- 172.21.15.62:0/2920047969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c003520 con 0x7f893c004800 2024-04-25T09:04:30.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.414+0000 7f8961400640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f895c0b27a0 0x7f893c005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.415+0000 7f895be00640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893c004800 0x7f893c004c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f894400c240 tx=0x7f893c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cd9b007bc0b59e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.415+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944007cf0 con 0x7f893c004800 2024-04-25T09:04:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.415+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8944005ad0 con 0x7f893c004800 2024-04-25T09:04:30.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.415+0000 7f895b400640 1 -- 172.21.15.62:0/2920047969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944007070 con 0x7f893c004800 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.415+0000 7f8961e00640 1 -- 172.21.15.62:0/2920047969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893c004800 msgr2=0x7f893c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.415+0000 7f8961e00640 1 --2- 172.21.15.62:0/2920047969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f893c004800 0x7f893c004c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f894400c240 tx=0x7f893c005830 comp rx=0 tx=0).stop 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.415+0000 7f8961400640 1 -- 172.21.15.62:0/2920047969 reap_dead start 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.416+0000 7f8961e00640 1 -- 172.21.15.62:0/2920047969 shutdown_connections 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.416+0000 7f8961e00640 1 -- 172.21.15.62:0/2920047969 >> 172.21.15.62:0/2920047969 conn(0x7f895c066d10 msgr2=0x7f895c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.416+0000 7f8961e00640 1 -- 172.21.15.62:0/2920047969 shutdown_connections 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.416+0000 7f8961e00640 1 -- 172.21.15.62:0/2920047969 wait complete. 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.417+0000 7f8961e00640 1 Processor -- start 2024-04-25T09:04:30.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.418+0000 7f8961e00640 1 -- start start 2024-04-25T09:04:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.418+0000 7f8961e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c004800 0x7f895c15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f895c0ccb20 0x7f895c1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f895c15b880 0x7f895c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b2ee0 con 0x7f895c15b880 2024-04-25T09:04:30.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b2d60 con 0x7f895c0ccb20 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b5ad0 con 0x7f893c004800 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f895c15b880 0x7f895c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f895c15b880 0x7f895c15bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36248/0 (socket says 172.21.15.62:36248) 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961400640 1 -- 172.21.15.62:0/2054806502 learned_addr learned my addr 172.21.15.62:0/2054806502 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8960a00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c004800 0x7f895c15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f895be00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f895c0ccb20 0x7f895c1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961400640 1 -- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c004800 msgr2=0x7f895c15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961400640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c004800 0x7f895c15e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961400640 1 -- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f895c0ccb20 msgr2=0x7f895c1f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961400640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f895c0ccb20 0x7f895c1f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.419+0000 7f8961400640 1 -- 172.21.15.62:0/2054806502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0cde40 con 0x7f895c15b880 2024-04-25T09:04:30.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.420+0000 7f8961400640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f895c15b880 0x7f895c15bd30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f895c0d4ad0 tx=0x7f8954009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.420+0000 7f8958c00640 1 -- 172.21.15.62:0/2054806502 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954019070 con 0x7f895c15b880 2024-04-25T09:04:30.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.420+0000 7f8958c00640 1 -- 172.21.15.62:0/2054806502 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8954007d10 con 0x7f895c15b880 2024-04-25T09:04:30.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.420+0000 7f8958c00640 1 -- 172.21.15.62:0/2054806502 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954014640 con 0x7f895c15b880 2024-04-25T09:04:30.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.420+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c0cd2c0 con 0x7f895c15b880 2024-04-25T09:04:30.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.420+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c0cd880 con 0x7f895c15b880 2024-04-25T09:04:30.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.421+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c04ef20 con 0x7f895c15b880 2024-04-25T09:04:30.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.422+0000 7f8958c00640 1 -- 172.21.15.62:0/2054806502 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f895401c070 con 0x7f895c15b880 2024-04-25T09:04:30.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.423+0000 7f8958c00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f893806b4d0 0x7f893806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.423+0000 7f8958c00640 1 -- 172.21.15.62:0/2054806502 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8954089d80 con 0x7f895c15b880 2024-04-25T09:04:30.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.424+0000 7f8960a00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f893806b4d0 0x7f893806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.425+0000 7f8960a00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f893806b4d0 0x7f893806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f894c007fd0 tx=0x7f894c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.428+0000 7f8958c00640 1 -- 172.21.15.62:0/2054806502 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8954058bf0 con 0x7f895c15b880 2024-04-25T09:04:30.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.625+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f895c0ce580 con 0x7f893806b4d0 2024-04-25T09:04:30.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.644+0000 7f8958c00640 1 -- 172.21.15.62:0/2054806502 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246633 (secure 0 0 0) 0x7f895c0ce580 con 0x7f893806b4d0 2024-04-25T09:04:30.636 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:30.638 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:04:30.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.650+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f893806b4d0 msgr2=0x7f893806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.650+0000 7f8961e00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f893806b4d0 0x7f893806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f894c007fd0 tx=0x7f894c009040 comp rx=0 tx=0).stop 2024-04-25T09:04:30.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.650+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f895c15b880 msgr2=0x7f895c15bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.650+0000 7f8961e00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f895c15b880 0x7f895c15bd30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f895c0d4ad0 tx=0x7f8954009d30 comp rx=0 tx=0).stop 2024-04-25T09:04:30.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.651+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 shutdown_connections 2024-04-25T09:04:30.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.651+0000 7f8961e00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f893806b4d0 0x7f893806d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.651+0000 7f8961e00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f895c15b880 0x7f895c15bd30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.651+0000 7f8961e00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f895c0ccb20 0x7f895c1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.651+0000 7f8961e00640 1 --2- 172.21.15.62:0/2054806502 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f893c004800 0x7f895c15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.651+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 >> 172.21.15.62:0/2054806502 conn(0x7f895c066d10 msgr2=0x7f895c15f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:30.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.651+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 shutdown_connections 2024-04-25T09:04:30.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.651+0000 7f8961e00640 1 -- 172.21.15.62:0/2054806502 wait complete. 2024-04-25T09:04:30.659 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-25T09:04:29.909361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220060,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754660,"statfs":{"total":773094113280,"available":772868771840,"internally_reserved":0,"allocated":3973120,"data_stored":2245112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.745531"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195929+0000","last_change":"2024-04-25T09:04:25.195929+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195929+0000","last_undegraded":"2024-04-25T09:04:25.195929+0000","last_fullsized":"2024-04-25T09:04:25.195929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190558+0000","last_change":"2024-04-25T09:04:25.190558+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190558+0000","last_undegraded":"2024-04-25T09:04:25.190558+0000","last_fullsized":"2024-04-25T09:04:25.190558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184327+0000","last_change":"2024-04-25T09:04:25.184327+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184327+0000","last_undegraded":"2024-04-25T09:04:25.184327+0000","last_fullsized":"2024-04-25T09:04:25.184327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183027+0000","last_change":"2024-04-25T09:04:25.183027+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183027+0000","last_undegraded":"2024-04-25T09:04:25.183027+0000","last_fullsized":"2024-04-25T09:04:25.183027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.196108+0000","last_change":"2024-04-25T09:04:25.196108+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.196108+0000","last_undegraded":"2024-04-25T09:04:25.196108+0000","last_fullsized":"2024-04-25T09:04:25.196108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183693+0000","last_change":"2024-04-25T09:04:25.183693+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183693+0000","last_undegraded":"2024-04-25T09:04:25.183693+0000","last_fullsized":"2024-04-25T09:04:25.183693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195170+0000","last_change":"2024-04-25T09:04:25.195170+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195170+0000","last_undegraded":"2024-04-25T09:04:25.195170+0000","last_fullsized":"2024-04-25T09:04:25.195170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195276+0000","last_change":"2024-04-25T09:04:25.195276+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195276+0000","last_undegraded":"2024-04-25T09:04:25.195276+0000","last_fullsized":"2024-04-25T09:04:25.195276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184881+0000","last_change":"2024-04-25T09:04:25.184881+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184881+0000","last_undegraded":"2024-04-25T09:04:25.184881+0000","last_fullsized":"2024-04-25T09:04:25.184881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183151+0000","last_change":"2024-04-25T09:04:25.183151+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183151+0000","last_undegraded":"2024-04-25T09:04:25.183151+0000","last_fullsized":"2024-04-25T09:04:25.183151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189070+0000","last_change":"2024-04-25T09:04:25.189070+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189070+0000","last_undegraded":"2024-04-25T09:04:25.189070+0000","last_fullsized":"2024-04-25T09:04:25.189070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.182768+0000","last_change":"2024-04-25T09:04:25.182768+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.182768+0000","last_undegraded":"2024-04-25T09:04:25.182768+0000","last_fullsized":"2024-04-25T09:04:25.182768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192960+0000","last_change":"2024-04-25T09:04:25.192960+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192960+0000","last_undegraded":"2024-04-25T09:04:25.192960+0000","last_fullsized":"2024-04-25T09:04:25.192960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.175725+0000","last_change":"2024-04-25T09:04:25.175725+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.175725+0000","last_undegraded":"2024-04-25T09:04:25.175725+0000","last_fullsized":"2024-04-25T09:04:25.175725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189104+0000","last_change":"2024-04-25T09:04:25.189104+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189104+0000","last_undegraded":"2024-04-25T09:04:25.189104+0000","last_fullsized":"2024-04-25T09:04:25.189104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180464+0000","last_change":"2024-04-25T09:04:25.180464+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180464+0000","last_undegraded":"2024-04-25T09:04:25.180464+0000","last_fullsized":"2024-04-25T09:04:25.180464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192509+0000","last_change":"2024-04-25T09:04:25.192509+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192509+0000","last_undegraded":"2024-04-25T09:04:25.192509+0000","last_fullsized":"2024-04-25T09:04:25.192509+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186012+0000","last_change":"2024-04-25T09:04:25.186012+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186012+0000","last_undegraded":"2024-04-25T09:04:25.186012+0000","last_fullsized":"2024-04-25T09:04:25.186012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187585+0000","last_change":"2024-04-25T09:04:25.187585+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187585+0000","last_undegraded":"2024-04-25T09:04:25.187585+0000","last_fullsized":"2024-04-25T09:04:25.187585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.194517+0000","last_change":"2024-04-25T09:04:25.194517+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.194517+0000","last_undegraded":"2024-04-25T09:04:25.194517+0000","last_fullsized":"2024-04-25T09:04:25.194517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180248+0000","last_change":"2024-04-25T09:04:25.180248+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180248+0000","last_undegraded":"2024-04-25T09:04:25.180248+0000","last_fullsized":"2024-04-25T09:04:25.180248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193801+0000","last_change":"2024-04-25T09:04:25.193801+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193801+0000","last_undegraded":"2024-04-25T09:04:25.193801+0000","last_fullsized":"2024-04-25T09:04:25.193801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190403+0000","last_change":"2024-04-25T09:04:25.190403+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190403+0000","last_undegraded":"2024-04-25T09:04:25.190403+0000","last_fullsized":"2024-04-25T09:04:25.190403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192553+0000","last_change":"2024-04-25T09:04:25.192553+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192553+0000","last_undegraded":"2024-04-25T09:04:25.192553+0000","last_fullsized":"2024-04-25T09:04:25.192553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179859+0000","last_change":"2024-04-25T09:04:25.179859+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179859+0000","last_undegraded":"2024-04-25T09:04:25.179859+0000","last_fullsized":"2024-04-25T09:04:25.179859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178338+0000","last_change":"2024-04-25T09:04:25.178338+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178338+0000","last_undegraded":"2024-04-25T09:04:25.178338+0000","last_fullsized":"2024-04-25T09:04:25.178338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.188145+0000","last_change":"2024-04-25T09:04:25.188145+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.188145+0000","last_undegraded":"2024-04-25T09:04:25.188145+0000","last_fullsized":"2024-04-25T09:04:25.188145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190431+0000","last_change":"2024-04-25T09:04:25.190431+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190431+0000","last_undegraded":"2024-04-25T09:04:25.190431+0000","last_fullsized":"2024-04-25T09:04:25.190431+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179645+0000","last_change":"2024-04-25T09:04:25.179645+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179645+0000","last_undegraded":"2024-04-25T09:04:25.179645+0000","last_fullsized":"2024-04-25T09:04:25.179645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179586+0000","last_change":"2024-04-25T09:04:25.179586+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179586+0000","last_undegraded":"2024-04-25T09:04:25.179586+0000","last_fullsized":"2024-04-25T09:04:25.179586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185200+0000","last_change":"2024-04-25T09:04:25.185200+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185200+0000","last_undegraded":"2024-04-25T09:04:25.185200+0000","last_fullsized":"2024-04-25T09:04:25.185200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187138+0000","last_change":"2024-04-25T09:04:25.187138+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187138+0000","last_undegraded":"2024-04-25T09:04:25.187138+0000","last_fullsized":"2024-04-25T09:04:25.187138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191755+0000","last_change":"2024-04-25T09:04:25.191755+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191755+0000","last_undegraded":"2024-04-25T09:04:25.191755+0000","last_fullsized":"2024-04-25T09:04:25.191755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211702+0000","last_change":"2024-04-25T09:04:25.210460+0000","last_active":"2024-04-25T09:04:25.211702+0000","last_peered":"2024-04-25T09:04:25.211702+0000","last_clean":"2024-04-25T09:04:25.211702+0000","last_became_active":"2024-04-25T09:04:25.210130+0000","last_became_peered":"2024-04-25T09:04:25.210130+0000","last_unstale":"2024-04-25T09:04:25.211702+0000","last_undegraded":"2024-04-25T09:04:25.211702+0000","last_fullsized":"2024-04-25T09:04:25.211702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:20.178362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187891+0000","last_change":"2024-04-25T09:04:25.187891+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187891+0000","last_undegraded":"2024-04-25T09:04:25.187891+0000","last_fullsized":"2024-04-25T09:04:25.187891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.203300+0000","last_change":"2024-04-25T09:04:25.198907+0000","last_active":"2024-04-25T09:04:25.203300+0000","last_peered":"2024-04-25T09:04:25.203300+0000","last_clean":"2024-04-25T09:04:25.203300+0000","last_became_active":"2024-04-25T09:04:25.198368+0000","last_became_peered":"2024-04-25T09:04:25.198368+0000","last_unstale":"2024-04-25T09:04:25.203300+0000","last_undegraded":"2024-04-25T09:04:25.203300+0000","last_fullsized":"2024-04-25T09:04:25.203300+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:43.725847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189966+0000","last_change":"2024-04-25T09:04:25.189966+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189966+0000","last_undegraded":"2024-04-25T09:04:25.189966+0000","last_fullsized":"2024-04-25T09:04:25.189966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.217010+0000","last_change":"2024-04-25T09:04:25.211094+0000","last_active":"2024-04-25T09:04:25.217010+0000","last_peered":"2024-04-25T09:04:25.217010+0000","last_clean":"2024-04-25T09:04:25.217010+0000","last_became_active":"2024-04-25T09:04:25.210803+0000","last_became_peered":"2024-04-25T09:04:25.210803+0000","last_unstale":"2024-04-25T09:04:25.217010+0000","last_undegraded":"2024-04-25T09:04:25.217010+0000","last_fullsized":"2024-04-25T09:04:25.217010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:31.510668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.177047+0000","last_change":"2024-04-25T09:04:25.177047+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.177047+0000","last_undegraded":"2024-04-25T09:04:25.177047+0000","last_fullsized":"2024-04-25T09:04:25.177047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211824+0000","last_change":"2024-04-25T09:04:25.209064+0000","last_active":"2024-04-25T09:04:25.211824+0000","last_peered":"2024-04-25T09:04:25.211824+0000","last_clean":"2024-04-25T09:04:25.211824+0000","last_became_active":"2024-04-25T09:04:25.208733+0000","last_became_peered":"2024-04-25T09:04:25.208733+0000","last_unstale":"2024-04-25T09:04:25.211824+0000","last_undegraded":"2024-04-25T09:04:25.211824+0000","last_fullsized":"2024-04-25T09:04:25.211824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.182891+0000","last_change":"2024-04-25T09:04:25.182891+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.182891+0000","last_undegraded":"2024-04-25T09:04:25.182891+0000","last_fullsized":"2024-04-25T09:04:25.182891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212083+0000","last_change":"2024-04-25T09:04:25.207956+0000","last_active":"2024-04-25T09:04:25.212083+0000","last_peered":"2024-04-25T09:04:25.212083+0000","last_clean":"2024-04-25T09:04:25.212083+0000","last_became_active":"2024-04-25T09:04:25.207646+0000","last_became_peered":"2024-04-25T09:04:25.207646+0000","last_unstale":"2024-04-25T09:04:25.212083+0000","last_undegraded":"2024-04-25T09:04:25.212083+0000","last_fullsized":"2024-04-25T09:04:25.212083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185181+0000","last_change":"2024-04-25T09:04:25.185181+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185181+0000","last_undegraded":"2024-04-25T09:04:25.185181+0000","last_fullsized":"2024-04-25T09:04:25.185181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212097+0000","last_change":"2024-04-25T09:04:25.209033+0000","last_active":"2024-04-25T09:04:25.212097+0000","last_peered":"2024-04-25T09:04:25.212097+0000","last_clean":"2024-04-25T09:04:25.212097+0000","last_became_active":"2024-04-25T09:04:25.208734+0000","last_became_peered":"2024-04-25T09:04:25.208734+0000","last_unstale":"2024-04-25T09:04:25.212097+0000","last_undegraded":"2024-04-25T09:04:25.212097+0000","last_fullsized":"2024-04-25T09:04:25.212097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183410+0000","last_change":"2024-04-25T09:04:25.183410+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183410+0000","last_undegraded":"2024-04-25T09:04:25.183410+0000","last_fullsized":"2024-04-25T09:04:25.183410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.216919+0000","last_change":"2024-04-25T09:04:25.209807+0000","last_active":"2024-04-25T09:04:25.216919+0000","last_peered":"2024-04-25T09:04:25.216919+0000","last_clean":"2024-04-25T09:04:25.216919+0000","last_became_active":"2024-04-25T09:04:25.207401+0000","last_became_peered":"2024-04-25T09:04:25.207401+0000","last_unstale":"2024-04-25T09:04:25.216919+0000","last_undegraded":"2024-04-25T09:04:25.216919+0000","last_fullsized":"2024-04-25T09:04:25.216919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:45.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.176143+0000","last_change":"2024-04-25T09:04:25.176143+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.176143+0000","last_undegraded":"2024-04-25T09:04:25.176143+0000","last_fullsized":"2024-04-25T09:04:25.176143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202536+0000","last_change":"2024-04-25T09:04:25.200573+0000","last_active":"2024-04-25T09:04:25.202536+0000","last_peered":"2024-04-25T09:04:25.202536+0000","last_clean":"2024-04-25T09:04:25.202536+0000","last_became_active":"2024-04-25T09:04:25.200262+0000","last_became_peered":"2024-04-25T09:04:25.200262+0000","last_unstale":"2024-04-25T09:04:25.202536+0000","last_undegraded":"2024-04-25T09:04:25.202536+0000","last_fullsized":"2024-04-25T09:04:25.202536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:45.866728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191346+0000","last_change":"2024-04-25T09:04:25.191346+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191346+0000","last_undegraded":"2024-04-25T09:04:25.191346+0000","last_fullsized":"2024-04-25T09:04:25.191346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207779+0000","last_change":"2024-04-25T09:04:25.202054+0000","last_active":"2024-04-25T09:04:25.207779+0000","last_peered":"2024-04-25T09:04:25.207779+0000","last_clean":"2024-04-25T09:04:25.207779+0000","last_became_active":"2024-04-25T09:04:25.201654+0000","last_became_peered":"2024-04-25T09:04:25.201654+0000","last_unstale":"2024-04-25T09:04:25.207779+0000","last_undegraded":"2024-04-25T09:04:25.207779+0000","last_fullsized":"2024-04-25T09:04:25.207779+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.834376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.177649+0000","last_change":"2024-04-25T09:04:25.177649+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.177649+0000","last_undegraded":"2024-04-25T09:04:25.177649+0000","last_fullsized":"2024-04-25T09:04:25.177649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209791+0000","last_change":"2024-04-25T09:04:25.207756+0000","last_active":"2024-04-25T09:04:25.209791+0000","last_peered":"2024-04-25T09:04:25.209791+0000","last_clean":"2024-04-25T09:04:25.209791+0000","last_became_active":"2024-04-25T09:04:25.207460+0000","last_became_peered":"2024-04-25T09:04:25.207460+0000","last_unstale":"2024-04-25T09:04:25.209791+0000","last_undegraded":"2024-04-25T09:04:25.209791+0000","last_fullsized":"2024-04-25T09:04:25.209791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:21.753154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179742+0000","last_change":"2024-04-25T09:04:25.179742+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179742+0000","last_undegraded":"2024-04-25T09:04:25.179742+0000","last_fullsized":"2024-04-25T09:04:25.179742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209792+0000","last_change":"2024-04-25T09:04:25.207463+0000","last_active":"2024-04-25T09:04:25.209792+0000","last_peered":"2024-04-25T09:04:25.209792+0000","last_clean":"2024-04-25T09:04:25.209792+0000","last_became_active":"2024-04-25T09:04:25.207178+0000","last_became_peered":"2024-04-25T09:04:25.207178+0000","last_unstale":"2024-04-25T09:04:25.209792+0000","last_undegraded":"2024-04-25T09:04:25.209792+0000","last_fullsized":"2024-04-25T09:04:25.209792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:38.564239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193205+0000","last_change":"2024-04-25T09:04:25.193205+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193205+0000","last_undegraded":"2024-04-25T09:04:25.193205+0000","last_fullsized":"2024-04-25T09:04:25.193205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.216595+0000","last_change":"2024-04-25T09:04:25.211486+0000","last_active":"2024-04-25T09:04:25.216595+0000","last_peered":"2024-04-25T09:04:25.216595+0000","last_clean":"2024-04-25T09:04:25.216595+0000","last_became_active":"2024-04-25T09:04:25.210675+0000","last_became_peered":"2024-04-25T09:04:25.210675+0000","last_unstale":"2024-04-25T09:04:25.216595+0000","last_undegraded":"2024-04-25T09:04:25.216595+0000","last_fullsized":"2024-04-25T09:04:25.216595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:46.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193611+0000","last_change":"2024-04-25T09:04:25.193611+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193611+0000","last_undegraded":"2024-04-25T09:04:25.193611+0000","last_fullsized":"2024-04-25T09:04:25.193611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211408+0000","last_change":"2024-04-25T09:04:25.205276+0000","last_active":"2024-04-25T09:04:25.211408+0000","last_peered":"2024-04-25T09:04:25.211408+0000","last_clean":"2024-04-25T09:04:25.211408+0000","last_became_active":"2024-04-25T09:04:25.204962+0000","last_became_peered":"2024-04-25T09:04:25.204962+0000","last_unstale":"2024-04-25T09:04:25.211408+0000","last_undegraded":"2024-04-25T09:04:25.211408+0000","last_fullsized":"2024-04-25T09:04:25.211408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:33.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180283+0000","last_change":"2024-04-25T09:04:25.180283+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180283+0000","last_undegraded":"2024-04-25T09:04:25.180283+0000","last_fullsized":"2024-04-25T09:04:25.180283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202059+0000","last_change":"2024-04-25T09:04:25.197723+0000","last_active":"2024-04-25T09:04:25.202059+0000","last_peered":"2024-04-25T09:04:25.202059+0000","last_clean":"2024-04-25T09:04:25.202059+0000","last_became_active":"2024-04-25T09:04:25.197275+0000","last_became_peered":"2024-04-25T09:04:25.197275+0000","last_unstale":"2024-04-25T09:04:25.202059+0000","last_undegraded":"2024-04-25T09:04:25.202059+0000","last_fullsized":"2024-04-25T09:04:25.202059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:46.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184657+0000","last_change":"2024-04-25T09:04:25.184657+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184657+0000","last_undegraded":"2024-04-25T09:04:25.184657+0000","last_fullsized":"2024-04-25T09:04:25.184657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.200271+0000","last_change":"2024-04-25T09:04:25.191706+0000","last_active":"2024-04-25T09:04:25.200271+0000","last_peered":"2024-04-25T09:04:25.200271+0000","last_clean":"2024-04-25T09:04:25.200271+0000","last_became_active":"2024-04-25T09:04:25.190455+0000","last_became_peered":"2024-04-25T09:04:25.190455+0000","last_unstale":"2024-04-25T09:04:25.200271+0000","last_undegraded":"2024-04-25T09:04:25.200271+0000","last_fullsized":"2024-04-25T09:04:25.200271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189103+0000","last_change":"2024-04-25T09:04:25.189103+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189103+0000","last_undegraded":"2024-04-25T09:04:25.189103+0000","last_fullsized":"2024-04-25T09:04:25.189103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207113+0000","last_change":"2024-04-25T09:04:25.203759+0000","last_active":"2024-04-25T09:04:25.207113+0000","last_peered":"2024-04-25T09:04:25.207113+0000","last_clean":"2024-04-25T09:04:25.207113+0000","last_became_active":"2024-04-25T09:04:25.203310+0000","last_became_peered":"2024-04-25T09:04:25.203310+0000","last_unstale":"2024-04-25T09:04:25.207113+0000","last_undegraded":"2024-04-25T09:04:25.207113+0000","last_fullsized":"2024-04-25T09:04:25.207113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:05.590884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186060+0000","last_change":"2024-04-25T09:04:25.186060+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186060+0000","last_undegraded":"2024-04-25T09:04:25.186060+0000","last_fullsized":"2024-04-25T09:04:25.186060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.208472+0000","last_change":"2024-04-25T09:04:25.206521+0000","last_active":"2024-04-25T09:04:25.208472+0000","last_peered":"2024-04-25T09:04:25.208472+0000","last_clean":"2024-04-25T09:04:25.208472+0000","last_became_active":"2024-04-25T09:04:25.206196+0000","last_became_peered":"2024-04-25T09:04:25.206196+0000","last_unstale":"2024-04-25T09:04:25.208472+0000","last_undegraded":"2024-04-25T09:04:25.208472+0000","last_fullsized":"2024-04-25T09:04:25.208472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:44.969455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184247+0000","last_change":"2024-04-25T09:04:25.184247+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184247+0000","last_undegraded":"2024-04-25T09:04:25.184247+0000","last_fullsized":"2024-04-25T09:04:25.184247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211386+0000","last_change":"2024-04-25T09:04:25.205066+0000","last_active":"2024-04-25T09:04:25.211386+0000","last_peered":"2024-04-25T09:04:25.211386+0000","last_clean":"2024-04-25T09:04:25.211386+0000","last_became_active":"2024-04-25T09:04:25.204741+0000","last_became_peered":"2024-04-25T09:04:25.204741+0000","last_unstale":"2024-04-25T09:04:25.211386+0000","last_undegraded":"2024-04-25T09:04:25.211386+0000","last_fullsized":"2024-04-25T09:04:25.211386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:27.537160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.188188+0000","last_change":"2024-04-25T09:04:25.188188+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.188188+0000","last_undegraded":"2024-04-25T09:04:25.188188+0000","last_fullsized":"2024-04-25T09:04:25.188188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207049+0000","last_change":"2024-04-25T09:04:25.203672+0000","last_active":"2024-04-25T09:04:25.207049+0000","last_peered":"2024-04-25T09:04:25.207049+0000","last_clean":"2024-04-25T09:04:25.207049+0000","last_became_active":"2024-04-25T09:04:25.203329+0000","last_became_peered":"2024-04-25T09:04:25.203329+0000","last_unstale":"2024-04-25T09:04:25.207049+0000","last_undegraded":"2024-04-25T09:04:25.207049+0000","last_fullsized":"2024-04-25T09:04:25.207049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:12.815263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186433+0000","last_change":"2024-04-25T09:04:25.186433+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186433+0000","last_undegraded":"2024-04-25T09:04:25.186433+0000","last_fullsized":"2024-04-25T09:04:25.186433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202354+0000","last_change":"2024-04-25T09:04:25.198618+0000","last_active":"2024-04-25T09:04:25.202354+0000","last_peered":"2024-04-25T09:04:25.202354+0000","last_clean":"2024-04-25T09:04:25.202354+0000","last_became_active":"2024-04-25T09:04:25.198320+0000","last_became_peered":"2024-04-25T09:04:25.198320+0000","last_unstale":"2024-04-25T09:04:25.202354+0000","last_undegraded":"2024-04-25T09:04:25.202354+0000","last_fullsized":"2024-04-25T09:04:25.202354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:27.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191367+0000","last_change":"2024-04-25T09:04:25.191367+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191367+0000","last_undegraded":"2024-04-25T09:04:25.191367+0000","last_fullsized":"2024-04-25T09:04:25.191367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209901+0000","last_change":"2024-04-25T09:04:25.206110+0000","last_active":"2024-04-25T09:04:25.209901+0000","last_peered":"2024-04-25T09:04:25.209901+0000","last_clean":"2024-04-25T09:04:25.209901+0000","last_became_active":"2024-04-25T09:04:25.205671+0000","last_became_peered":"2024-04-25T09:04:25.205671+0000","last_unstale":"2024-04-25T09:04:25.209901+0000","last_undegraded":"2024-04-25T09:04:25.209901+0000","last_fullsized":"2024-04-25T09:04:25.209901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:37.157522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195435+0000","last_change":"2024-04-25T09:04:25.195435+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195435+0000","last_undegraded":"2024-04-25T09:04:25.195435+0000","last_fullsized":"2024-04-25T09:04:25.195435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211443+0000","last_change":"2024-04-25T09:04:25.205857+0000","last_active":"2024-04-25T09:04:25.211443+0000","last_peered":"2024-04-25T09:04:25.211443+0000","last_clean":"2024-04-25T09:04:25.211443+0000","last_became_active":"2024-04-25T09:04:25.204814+0000","last_became_peered":"2024-04-25T09:04:25.204814+0000","last_unstale":"2024-04-25T09:04:25.211443+0000","last_undegraded":"2024-04-25T09:04:25.211443+0000","last_fullsized":"2024-04-25T09:04:25.211443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:46.169978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189734+0000","last_change":"2024-04-25T09:04:25.189734+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189734+0000","last_undegraded":"2024-04-25T09:04:25.189734+0000","last_fullsized":"2024-04-25T09:04:25.189734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205881+0000","last_change":"2024-04-25T09:04:25.203994+0000","last_active":"2024-04-25T09:04:25.205881+0000","last_peered":"2024-04-25T09:04:25.205881+0000","last_clean":"2024-04-25T09:04:25.205881+0000","last_became_active":"2024-04-25T09:04:25.203665+0000","last_became_peered":"2024-04-25T09:04:25.203665+0000","last_unstale":"2024-04-25T09:04:25.205881+0000","last_undegraded":"2024-04-25T09:04:25.205881+0000","last_fullsized":"2024-04-25T09:04:25.205881+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:45.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.181957+0000","last_change":"2024-04-25T09:04:25.181957+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.181957+0000","last_undegraded":"2024-04-25T09:04:25.181957+0000","last_fullsized":"2024-04-25T09:04:25.181957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.204263+0000","last_change":"2024-04-25T09:04:25.201881+0000","last_active":"2024-04-25T09:04:25.204263+0000","last_peered":"2024-04-25T09:04:25.204263+0000","last_clean":"2024-04-25T09:04:25.204263+0000","last_became_active":"2024-04-25T09:04:25.201582+0000","last_became_peered":"2024-04-25T09:04:25.201582+0000","last_unstale":"2024-04-25T09:04:25.204263+0000","last_undegraded":"2024-04-25T09:04:25.204263+0000","last_fullsized":"2024-04-25T09:04:25.204263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:17.652744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183729+0000","last_change":"2024-04-25T09:04:25.183729+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183729+0000","last_undegraded":"2024-04-25T09:04:25.183729+0000","last_fullsized":"2024-04-25T09:04:25.183729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205257+0000","last_change":"2024-04-25T09:04:25.201894+0000","last_active":"2024-04-25T09:04:25.205257+0000","last_peered":"2024-04-25T09:04:25.205257+0000","last_clean":"2024-04-25T09:04:25.205257+0000","last_became_active":"2024-04-25T09:04:25.201583+0000","last_became_peered":"2024-04-25T09:04:25.201583+0000","last_unstale":"2024-04-25T09:04:25.205257+0000","last_undegraded":"2024-04-25T09:04:25.205257+0000","last_fullsized":"2024-04-25T09:04:25.205257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:46.169978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192016+0000","last_change":"2024-04-25T09:04:25.192016+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192016+0000","last_undegraded":"2024-04-25T09:04:25.192016+0000","last_fullsized":"2024-04-25T09:04:25.192016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209813+0000","last_change":"2024-04-25T09:04:25.207893+0000","last_active":"2024-04-25T09:04:25.209813+0000","last_peered":"2024-04-25T09:04:25.209813+0000","last_clean":"2024-04-25T09:04:25.209813+0000","last_became_active":"2024-04-25T09:04:25.207587+0000","last_became_peered":"2024-04-25T09:04:25.207587+0000","last_unstale":"2024-04-25T09:04:25.209813+0000","last_undegraded":"2024-04-25T09:04:25.209813+0000","last_fullsized":"2024-04-25T09:04:25.209813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:04.049549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185065+0000","last_change":"2024-04-25T09:04:25.185065+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185065+0000","last_undegraded":"2024-04-25T09:04:25.185065+0000","last_fullsized":"2024-04-25T09:04:25.185065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.208184+0000","last_change":"2024-04-25T09:04:25.206786+0000","last_active":"2024-04-25T09:04:25.208184+0000","last_peered":"2024-04-25T09:04:25.208184+0000","last_clean":"2024-04-25T09:04:25.208184+0000","last_became_active":"2024-04-25T09:04:25.206460+0000","last_became_peered":"2024-04-25T09:04:25.206460+0000","last_unstale":"2024-04-25T09:04:25.208184+0000","last_undegraded":"2024-04-25T09:04:25.208184+0000","last_fullsized":"2024-04-25T09:04:25.208184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178764+0000","last_change":"2024-04-25T09:04:25.178764+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178764+0000","last_undegraded":"2024-04-25T09:04:25.178764+0000","last_fullsized":"2024-04-25T09:04:25.178764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205405+0000","last_change":"2024-04-25T09:04:25.203588+0000","last_active":"2024-04-25T09:04:25.205405+0000","last_peered":"2024-04-25T09:04:25.205405+0000","last_clean":"2024-04-25T09:04:25.205405+0000","last_became_active":"2024-04-25T09:04:25.203279+0000","last_became_peered":"2024-04-25T09:04:25.203279+0000","last_unstale":"2024-04-25T09:04:25.205405+0000","last_undegraded":"2024-04-25T09:04:25.205405+0000","last_fullsized":"2024-04-25T09:04:25.205405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178917+0000","last_change":"2024-04-25T09:04:25.178917+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178917+0000","last_undegraded":"2024-04-25T09:04:25.178917+0000","last_fullsized":"2024-04-25T09:04:25.178917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209329+0000","last_change":"2024-04-25T09:04:25.205414+0000","last_active":"2024-04-25T09:04:25.209329+0000","last_peered":"2024-04-25T09:04:25.209329+0000","last_clean":"2024-04-25T09:04:25.209329+0000","last_became_active":"2024-04-25T09:04:25.205062+0000","last_became_peered":"2024-04-25T09:04:25.205062+0000","last_unstale":"2024-04-25T09:04:25.209329+0000","last_undegraded":"2024-04-25T09:04:25.209329+0000","last_fullsized":"2024-04-25T09:04:25.209329+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:56.631288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.175676+0000","last_change":"2024-04-25T09:04:25.175676+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.175676+0000","last_undegraded":"2024-04-25T09:04:25.175676+0000","last_fullsized":"2024-04-25T09:04:25.175676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212068+0000","last_change":"2024-04-25T09:04:25.207990+0000","last_active":"2024-04-25T09:04:25.212068+0000","last_peered":"2024-04-25T09:04:25.212068+0000","last_clean":"2024-04-25T09:04:25.212068+0000","last_became_active":"2024-04-25T09:04:25.207576+0000","last_became_peered":"2024-04-25T09:04:25.207576+0000","last_unstale":"2024-04-25T09:04:25.212068+0000","last_undegraded":"2024-04-25T09:04:25.212068+0000","last_fullsized":"2024-04-25T09:04:25.212068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:56.631288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187851+0000","last_change":"2024-04-25T09:04:25.187851+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187851+0000","last_undegraded":"2024-04-25T09:04:25.187851+0000","last_fullsized":"2024-04-25T09:04:25.187851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.203149+0000","last_change":"2024-04-25T09:04:25.198844+0000","last_active":"2024-04-25T09:04:25.203149+0000","last_peered":"2024-04-25T09:04:25.203149+0000","last_clean":"2024-04-25T09:04:25.203149+0000","last_became_active":"2024-04-25T09:04:25.198376+0000","last_became_peered":"2024-04-25T09:04:25.198376+0000","last_unstale":"2024-04-25T09:04:25.203149+0000","last_undegraded":"2024-04-25T09:04:25.203149+0000","last_fullsized":"2024-04-25T09:04:25.203149+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:57.417759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179814+0000","last_change":"2024-04-25T09:04:25.179814+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179814+0000","last_undegraded":"2024-04-25T09:04:25.179814+0000","last_fullsized":"2024-04-25T09:04:25.179814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209328+0000","last_change":"2024-04-25T09:04:25.202687+0000","last_active":"2024-04-25T09:04:25.209328+0000","last_peered":"2024-04-25T09:04:25.209328+0000","last_clean":"2024-04-25T09:04:25.209328+0000","last_became_active":"2024-04-25T09:04:25.202400+0000","last_became_peered":"2024-04-25T09:04:25.202400+0000","last_unstale":"2024-04-25T09:04:25.209328+0000","last_undegraded":"2024-04-25T09:04:25.209328+0000","last_fullsized":"2024-04-25T09:04:25.209328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:50.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.185425+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:04:25.185425+0000","last_peered":"2024-04-25T09:04:25.185425+0000","last_clean":"2024-04-25T09:04:25.185425+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:04:25.185425+0000","last_undegraded":"2024-04-25T09:04:25.185425+0000","last_fullsized":"2024-04-25T09:04:25.185425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:30.664 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:30.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.930+0000 7f365ce00640 1 Processor -- start 2024-04-25T09:04:30.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.930+0000 7f365ce00640 1 -- start start 2024-04-25T09:04:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f365ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f365815bd10 0x7f36580cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f365ce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f36580cccf0 0x7f36580ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f365ce00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365806ca70 con 0x7f36580aed00 2024-04-25T09:04:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f365ce00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365806d660 con 0x7f365815bd10 2024-04-25T09:04:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f365ce00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36580af140 con 0x7f36580cccf0 2024-04-25T09:04:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f3655e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f365815bd10 0x7f36580cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f3656800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f36580aed00 0x7f365815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41452/0 (socket says 172.21.15.62:41452) 2024-04-25T09:04:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f3656800640 1 -- 172.21.15.62:0/650008023 learned_addr learned my addr 172.21.15.62:0/650008023 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:30.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.932+0000 7f364d400640 1 --2- 172.21.15.62:0/650008023 >> v2:172.21.15.121:3300/0 conn(0x7f36580cccf0 0x7f36580ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.933+0000 7f3655e00640 1 -- 172.21.15.62:0/650008023 >> v2:172.21.15.121:3300/0 conn(0x7f36580cccf0 msgr2=0x7f36580ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:30.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.933+0000 7f3655e00640 1 --2- 172.21.15.62:0/650008023 >> v2:172.21.15.121:3300/0 conn(0x7f36580cccf0 0x7f36580ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.933+0000 7f3655e00640 1 -- 172.21.15.62:0/650008023 >> v1:172.21.15.62:6789/0 conn(0x7f36580aed00 legacy=0x7f365815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.933+0000 7f3655e00640 1 -- 172.21.15.62:0/650008023 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365815c0f0 con 0x7f365815bd10 2024-04-25T09:04:30.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.933+0000 7f3655400640 1 -- 172.21.15.62:0/650008023 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033799291 0 0) 0x7f365806ca70 con 0x7f36580aed00 2024-04-25T09:04:30.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.934+0000 7f3655e00640 1 --2- 172.21.15.62:0/650008023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365815bd10 0x7f36580cfca0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f365806d7e0 tx=0x7f364802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed8f31a8f209c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.934+0000 7f3655400640 1 -- 172.21.15.62:0/650008023 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648037070 con 0x7f365815bd10 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.934+0000 7f3655400640 1 -- 172.21.15.62:0/650008023 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648032440 con 0x7f365815bd10 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.934+0000 7f3655400640 1 -- 172.21.15.62:0/650008023 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648032810 con 0x7f365815bd10 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.935+0000 7f365ce00640 1 -- 172.21.15.62:0/650008023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365815bd10 msgr2=0x7f36580cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.935+0000 7f365ce00640 1 --2- 172.21.15.62:0/650008023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365815bd10 0x7f36580cfca0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f365806d7e0 tx=0x7f364802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.935+0000 7f365ce00640 1 -- 172.21.15.62:0/650008023 shutdown_connections 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.935+0000 7f365ce00640 1 --2- 172.21.15.62:0/650008023 >> v2:172.21.15.121:3300/0 conn(0x7f36580cccf0 0x7f36580ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.935+0000 7f365ce00640 1 --2- 172.21.15.62:0/650008023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365815bd10 0x7f36580cfca0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.935+0000 7f365ce00640 1 -- 172.21.15.62:0/650008023 >> 172.21.15.62:0/650008023 conn(0x7f36580668f0 msgr2=0x7f3658066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.935+0000 7f365ce00640 1 -- 172.21.15.62:0/650008023 shutdown_connections 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.935+0000 7f365ce00640 1 -- 172.21.15.62:0/650008023 wait complete. 2024-04-25T09:04:30.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.937+0000 7f365ce00640 1 Processor -- start 2024-04-25T09:04:30.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.937+0000 7f365ce00640 1 -- start start 2024-04-25T09:04:30.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f365ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36580aed00 0x7f36581f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f365ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36580cccf0 0x7f365814e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f365ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f365815bd10 0x7f3658152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f365ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365806d010 con 0x7f36580aed00 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f365ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36580ce6d0 con 0x7f36580cccf0 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f365ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365806d2f0 con 0x7f365815bd10 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f3655e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36580aed00 0x7f36581f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f3655e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36580aed00 0x7f36581f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36264/0 (socket says 172.21.15.62:36264) 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f3655e00640 1 -- 172.21.15.62:0/246186755 learned_addr learned my addr 172.21.15.62:0/246186755 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f364d400640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36580cccf0 0x7f365814e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.938+0000 7f3656800640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f365815bd10 0x7f3658152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f3655e00640 1 -- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f365815bd10 msgr2=0x7f3658152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f3655e00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f365815bd10 0x7f3658152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f3655e00640 1 -- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36580cccf0 msgr2=0x7f365814e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f3655e00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36580cccf0 0x7f365814e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f3655e00640 1 -- 172.21.15.62:0/246186755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658153c70 con 0x7f36580aed00 2024-04-25T09:04:30.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f3656800640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f365815bd10 0x7f3658152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:30.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f3655e00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36580aed00 0x7f36581f6fb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3648000c00 tx=0x7f36480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f364ea00640 1 -- 172.21.15.62:0/246186755 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648037040 con 0x7f36580aed00 2024-04-25T09:04:30.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f364ea00640 1 -- 172.21.15.62:0/246186755 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648042070 con 0x7f36580aed00 2024-04-25T09:04:30.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f364ea00640 1 -- 172.21.15.62:0/246186755 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648030bd0 con 0x7f36580aed00 2024-04-25T09:04:30.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.939+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36581f7800 con 0x7f36580aed00 2024-04-25T09:04:30.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.940+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3658153ee0 con 0x7f36580aed00 2024-04-25T09:04:30.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.941+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3658072c70 con 0x7f36580aed00 2024-04-25T09:04:30.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.942+0000 7f364ea00640 1 -- 172.21.15.62:0/246186755 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3648047460 con 0x7f36580aed00 2024-04-25T09:04:30.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.943+0000 7f364ea00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f363406b4d0 0x7f363406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.944+0000 7f364ea00640 1 -- 172.21.15.62:0/246186755 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36480aae80 con 0x7f36580aed00 2024-04-25T09:04:30.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.944+0000 7f364d400640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f363406b4d0 0x7f363406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.945+0000 7f364d400640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f363406b4d0 0x7f363406d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f365815dc30 tx=0x7f3640003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:30.948+0000 7f364ea00640 1 -- 172.21.15.62:0/246186755 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3648032e20 con 0x7f36580aed00 2024-04-25T09:04:31.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.141+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3658154120 con 0x7f363406b4d0 2024-04-25T09:04:31.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.160+0000 7f364ea00640 1 -- 172.21.15.62:0/246186755 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246633 (secure 0 0 0) 0x7f3640004750 con 0x7f363406b4d0 2024-04-25T09:04:31.151 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f363406b4d0 msgr2=0x7f363406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f363406b4d0 0x7f363406d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f365815dc30 tx=0x7f3640003990 comp rx=0 tx=0).stop 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36580aed00 msgr2=0x7f36581f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36580aed00 0x7f36581f6fb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3648000c00 tx=0x7f36480046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 shutdown_connections 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f363406b4d0 0x7f363406d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f365815bd10 0x7f3658152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36580cccf0 0x7f365814e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 --2- 172.21.15.62:0/246186755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36580aed00 0x7f36581f6fb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.165+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 >> 172.21.15.62:0/246186755 conn(0x7f36580668f0 msgr2=0x7f3658070630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.166+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 shutdown_connections 2024-04-25T09:04:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.166+0000 7f365ce00640 1 -- 172.21.15.62:0/246186755 wait complete. 2024-04-25T09:04:31.171 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-25T09:04:29.909361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220060,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754660,"statfs":{"total":773094113280,"available":772868771840,"internally_reserved":0,"allocated":3973120,"data_stored":2245112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.745531"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195929+0000","last_change":"2024-04-25T09:04:25.195929+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195929+0000","last_undegraded":"2024-04-25T09:04:25.195929+0000","last_fullsized":"2024-04-25T09:04:25.195929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190558+0000","last_change":"2024-04-25T09:04:25.190558+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190558+0000","last_undegraded":"2024-04-25T09:04:25.190558+0000","last_fullsized":"2024-04-25T09:04:25.190558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184327+0000","last_change":"2024-04-25T09:04:25.184327+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184327+0000","last_undegraded":"2024-04-25T09:04:25.184327+0000","last_fullsized":"2024-04-25T09:04:25.184327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183027+0000","last_change":"2024-04-25T09:04:25.183027+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183027+0000","last_undegraded":"2024-04-25T09:04:25.183027+0000","last_fullsized":"2024-04-25T09:04:25.183027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.196108+0000","last_change":"2024-04-25T09:04:25.196108+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.196108+0000","last_undegraded":"2024-04-25T09:04:25.196108+0000","last_fullsized":"2024-04-25T09:04:25.196108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183693+0000","last_change":"2024-04-25T09:04:25.183693+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183693+0000","last_undegraded":"2024-04-25T09:04:25.183693+0000","last_fullsized":"2024-04-25T09:04:25.183693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195170+0000","last_change":"2024-04-25T09:04:25.195170+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195170+0000","last_undegraded":"2024-04-25T09:04:25.195170+0000","last_fullsized":"2024-04-25T09:04:25.195170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195276+0000","last_change":"2024-04-25T09:04:25.195276+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195276+0000","last_undegraded":"2024-04-25T09:04:25.195276+0000","last_fullsized":"2024-04-25T09:04:25.195276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184881+0000","last_change":"2024-04-25T09:04:25.184881+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184881+0000","last_undegraded":"2024-04-25T09:04:25.184881+0000","last_fullsized":"2024-04-25T09:04:25.184881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183151+0000","last_change":"2024-04-25T09:04:25.183151+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183151+0000","last_undegraded":"2024-04-25T09:04:25.183151+0000","last_fullsized":"2024-04-25T09:04:25.183151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189070+0000","last_change":"2024-04-25T09:04:25.189070+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189070+0000","last_undegraded":"2024-04-25T09:04:25.189070+0000","last_fullsized":"2024-04-25T09:04:25.189070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.182768+0000","last_change":"2024-04-25T09:04:25.182768+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.182768+0000","last_undegraded":"2024-04-25T09:04:25.182768+0000","last_fullsized":"2024-04-25T09:04:25.182768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192960+0000","last_change":"2024-04-25T09:04:25.192960+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192960+0000","last_undegraded":"2024-04-25T09:04:25.192960+0000","last_fullsized":"2024-04-25T09:04:25.192960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.175725+0000","last_change":"2024-04-25T09:04:25.175725+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.175725+0000","last_undegraded":"2024-04-25T09:04:25.175725+0000","last_fullsized":"2024-04-25T09:04:25.175725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189104+0000","last_change":"2024-04-25T09:04:25.189104+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189104+0000","last_undegraded":"2024-04-25T09:04:25.189104+0000","last_fullsized":"2024-04-25T09:04:25.189104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180464+0000","last_change":"2024-04-25T09:04:25.180464+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180464+0000","last_undegraded":"2024-04-25T09:04:25.180464+0000","last_fullsized":"2024-04-25T09:04:25.180464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192509+0000","last_change":"2024-04-25T09:04:25.192509+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192509+0000","last_undegraded":"2024-04-25T09:04:25.192509+0000","last_fullsized":"2024-04-25T09:04:25.192509+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186012+0000","last_change":"2024-04-25T09:04:25.186012+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186012+0000","last_undegraded":"2024-04-25T09:04:25.186012+0000","last_fullsized":"2024-04-25T09:04:25.186012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187585+0000","last_change":"2024-04-25T09:04:25.187585+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187585+0000","last_undegraded":"2024-04-25T09:04:25.187585+0000","last_fullsized":"2024-04-25T09:04:25.187585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.194517+0000","last_change":"2024-04-25T09:04:25.194517+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.194517+0000","last_undegraded":"2024-04-25T09:04:25.194517+0000","last_fullsized":"2024-04-25T09:04:25.194517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180248+0000","last_change":"2024-04-25T09:04:25.180248+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180248+0000","last_undegraded":"2024-04-25T09:04:25.180248+0000","last_fullsized":"2024-04-25T09:04:25.180248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193801+0000","last_change":"2024-04-25T09:04:25.193801+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193801+0000","last_undegraded":"2024-04-25T09:04:25.193801+0000","last_fullsized":"2024-04-25T09:04:25.193801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190403+0000","last_change":"2024-04-25T09:04:25.190403+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190403+0000","last_undegraded":"2024-04-25T09:04:25.190403+0000","last_fullsized":"2024-04-25T09:04:25.190403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192553+0000","last_change":"2024-04-25T09:04:25.192553+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192553+0000","last_undegraded":"2024-04-25T09:04:25.192553+0000","last_fullsized":"2024-04-25T09:04:25.192553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179859+0000","last_change":"2024-04-25T09:04:25.179859+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179859+0000","last_undegraded":"2024-04-25T09:04:25.179859+0000","last_fullsized":"2024-04-25T09:04:25.179859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178338+0000","last_change":"2024-04-25T09:04:25.178338+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178338+0000","last_undegraded":"2024-04-25T09:04:25.178338+0000","last_fullsized":"2024-04-25T09:04:25.178338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.188145+0000","last_change":"2024-04-25T09:04:25.188145+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.188145+0000","last_undegraded":"2024-04-25T09:04:25.188145+0000","last_fullsized":"2024-04-25T09:04:25.188145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190431+0000","last_change":"2024-04-25T09:04:25.190431+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190431+0000","last_undegraded":"2024-04-25T09:04:25.190431+0000","last_fullsized":"2024-04-25T09:04:25.190431+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179645+0000","last_change":"2024-04-25T09:04:25.179645+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179645+0000","last_undegraded":"2024-04-25T09:04:25.179645+0000","last_fullsized":"2024-04-25T09:04:25.179645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179586+0000","last_change":"2024-04-25T09:04:25.179586+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179586+0000","last_undegraded":"2024-04-25T09:04:25.179586+0000","last_fullsized":"2024-04-25T09:04:25.179586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185200+0000","last_change":"2024-04-25T09:04:25.185200+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185200+0000","last_undegraded":"2024-04-25T09:04:25.185200+0000","last_fullsized":"2024-04-25T09:04:25.185200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187138+0000","last_change":"2024-04-25T09:04:25.187138+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187138+0000","last_undegraded":"2024-04-25T09:04:25.187138+0000","last_fullsized":"2024-04-25T09:04:25.187138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191755+0000","last_change":"2024-04-25T09:04:25.191755+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191755+0000","last_undegraded":"2024-04-25T09:04:25.191755+0000","last_fullsized":"2024-04-25T09:04:25.191755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211702+0000","last_change":"2024-04-25T09:04:25.210460+0000","last_active":"2024-04-25T09:04:25.211702+0000","last_peered":"2024-04-25T09:04:25.211702+0000","last_clean":"2024-04-25T09:04:25.211702+0000","last_became_active":"2024-04-25T09:04:25.210130+0000","last_became_peered":"2024-04-25T09:04:25.210130+0000","last_unstale":"2024-04-25T09:04:25.211702+0000","last_undegraded":"2024-04-25T09:04:25.211702+0000","last_fullsized":"2024-04-25T09:04:25.211702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:20.178362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187891+0000","last_change":"2024-04-25T09:04:25.187891+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187891+0000","last_undegraded":"2024-04-25T09:04:25.187891+0000","last_fullsized":"2024-04-25T09:04:25.187891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.203300+0000","last_change":"2024-04-25T09:04:25.198907+0000","last_active":"2024-04-25T09:04:25.203300+0000","last_peered":"2024-04-25T09:04:25.203300+0000","last_clean":"2024-04-25T09:04:25.203300+0000","last_became_active":"2024-04-25T09:04:25.198368+0000","last_became_peered":"2024-04-25T09:04:25.198368+0000","last_unstale":"2024-04-25T09:04:25.203300+0000","last_undegraded":"2024-04-25T09:04:25.203300+0000","last_fullsized":"2024-04-25T09:04:25.203300+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:43.725847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189966+0000","last_change":"2024-04-25T09:04:25.189966+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189966+0000","last_undegraded":"2024-04-25T09:04:25.189966+0000","last_fullsized":"2024-04-25T09:04:25.189966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.217010+0000","last_change":"2024-04-25T09:04:25.211094+0000","last_active":"2024-04-25T09:04:25.217010+0000","last_peered":"2024-04-25T09:04:25.217010+0000","last_clean":"2024-04-25T09:04:25.217010+0000","last_became_active":"2024-04-25T09:04:25.210803+0000","last_became_peered":"2024-04-25T09:04:25.210803+0000","last_unstale":"2024-04-25T09:04:25.217010+0000","last_undegraded":"2024-04-25T09:04:25.217010+0000","last_fullsized":"2024-04-25T09:04:25.217010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:31.510668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.177047+0000","last_change":"2024-04-25T09:04:25.177047+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.177047+0000","last_undegraded":"2024-04-25T09:04:25.177047+0000","last_fullsized":"2024-04-25T09:04:25.177047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211824+0000","last_change":"2024-04-25T09:04:25.209064+0000","last_active":"2024-04-25T09:04:25.211824+0000","last_peered":"2024-04-25T09:04:25.211824+0000","last_clean":"2024-04-25T09:04:25.211824+0000","last_became_active":"2024-04-25T09:04:25.208733+0000","last_became_peered":"2024-04-25T09:04:25.208733+0000","last_unstale":"2024-04-25T09:04:25.211824+0000","last_undegraded":"2024-04-25T09:04:25.211824+0000","last_fullsized":"2024-04-25T09:04:25.211824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.182891+0000","last_change":"2024-04-25T09:04:25.182891+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.182891+0000","last_undegraded":"2024-04-25T09:04:25.182891+0000","last_fullsized":"2024-04-25T09:04:25.182891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212083+0000","last_change":"2024-04-25T09:04:25.207956+0000","last_active":"2024-04-25T09:04:25.212083+0000","last_peered":"2024-04-25T09:04:25.212083+0000","last_clean":"2024-04-25T09:04:25.212083+0000","last_became_active":"2024-04-25T09:04:25.207646+0000","last_became_peered":"2024-04-25T09:04:25.207646+0000","last_unstale":"2024-04-25T09:04:25.212083+0000","last_undegraded":"2024-04-25T09:04:25.212083+0000","last_fullsized":"2024-04-25T09:04:25.212083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185181+0000","last_change":"2024-04-25T09:04:25.185181+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185181+0000","last_undegraded":"2024-04-25T09:04:25.185181+0000","last_fullsized":"2024-04-25T09:04:25.185181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212097+0000","last_change":"2024-04-25T09:04:25.209033+0000","last_active":"2024-04-25T09:04:25.212097+0000","last_peered":"2024-04-25T09:04:25.212097+0000","last_clean":"2024-04-25T09:04:25.212097+0000","last_became_active":"2024-04-25T09:04:25.208734+0000","last_became_peered":"2024-04-25T09:04:25.208734+0000","last_unstale":"2024-04-25T09:04:25.212097+0000","last_undegraded":"2024-04-25T09:04:25.212097+0000","last_fullsized":"2024-04-25T09:04:25.212097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183410+0000","last_change":"2024-04-25T09:04:25.183410+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183410+0000","last_undegraded":"2024-04-25T09:04:25.183410+0000","last_fullsized":"2024-04-25T09:04:25.183410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.216919+0000","last_change":"2024-04-25T09:04:25.209807+0000","last_active":"2024-04-25T09:04:25.216919+0000","last_peered":"2024-04-25T09:04:25.216919+0000","last_clean":"2024-04-25T09:04:25.216919+0000","last_became_active":"2024-04-25T09:04:25.207401+0000","last_became_peered":"2024-04-25T09:04:25.207401+0000","last_unstale":"2024-04-25T09:04:25.216919+0000","last_undegraded":"2024-04-25T09:04:25.216919+0000","last_fullsized":"2024-04-25T09:04:25.216919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:45.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.176143+0000","last_change":"2024-04-25T09:04:25.176143+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.176143+0000","last_undegraded":"2024-04-25T09:04:25.176143+0000","last_fullsized":"2024-04-25T09:04:25.176143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202536+0000","last_change":"2024-04-25T09:04:25.200573+0000","last_active":"2024-04-25T09:04:25.202536+0000","last_peered":"2024-04-25T09:04:25.202536+0000","last_clean":"2024-04-25T09:04:25.202536+0000","last_became_active":"2024-04-25T09:04:25.200262+0000","last_became_peered":"2024-04-25T09:04:25.200262+0000","last_unstale":"2024-04-25T09:04:25.202536+0000","last_undegraded":"2024-04-25T09:04:25.202536+0000","last_fullsized":"2024-04-25T09:04:25.202536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:45.866728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191346+0000","last_change":"2024-04-25T09:04:25.191346+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191346+0000","last_undegraded":"2024-04-25T09:04:25.191346+0000","last_fullsized":"2024-04-25T09:04:25.191346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207779+0000","last_change":"2024-04-25T09:04:25.202054+0000","last_active":"2024-04-25T09:04:25.207779+0000","last_peered":"2024-04-25T09:04:25.207779+0000","last_clean":"2024-04-25T09:04:25.207779+0000","last_became_active":"2024-04-25T09:04:25.201654+0000","last_became_peered":"2024-04-25T09:04:25.201654+0000","last_unstale":"2024-04-25T09:04:25.207779+0000","last_undegraded":"2024-04-25T09:04:25.207779+0000","last_fullsized":"2024-04-25T09:04:25.207779+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.834376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.177649+0000","last_change":"2024-04-25T09:04:25.177649+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.177649+0000","last_undegraded":"2024-04-25T09:04:25.177649+0000","last_fullsized":"2024-04-25T09:04:25.177649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209791+0000","last_change":"2024-04-25T09:04:25.207756+0000","last_active":"2024-04-25T09:04:25.209791+0000","last_peered":"2024-04-25T09:04:25.209791+0000","last_clean":"2024-04-25T09:04:25.209791+0000","last_became_active":"2024-04-25T09:04:25.207460+0000","last_became_peered":"2024-04-25T09:04:25.207460+0000","last_unstale":"2024-04-25T09:04:25.209791+0000","last_undegraded":"2024-04-25T09:04:25.209791+0000","last_fullsized":"2024-04-25T09:04:25.209791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:21.753154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179742+0000","last_change":"2024-04-25T09:04:25.179742+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179742+0000","last_undegraded":"2024-04-25T09:04:25.179742+0000","last_fullsized":"2024-04-25T09:04:25.179742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209792+0000","last_change":"2024-04-25T09:04:25.207463+0000","last_active":"2024-04-25T09:04:25.209792+0000","last_peered":"2024-04-25T09:04:25.209792+0000","last_clean":"2024-04-25T09:04:25.209792+0000","last_became_active":"2024-04-25T09:04:25.207178+0000","last_became_peered":"2024-04-25T09:04:25.207178+0000","last_unstale":"2024-04-25T09:04:25.209792+0000","last_undegraded":"2024-04-25T09:04:25.209792+0000","last_fullsized":"2024-04-25T09:04:25.209792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:38.564239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193205+0000","last_change":"2024-04-25T09:04:25.193205+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193205+0000","last_undegraded":"2024-04-25T09:04:25.193205+0000","last_fullsized":"2024-04-25T09:04:25.193205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.216595+0000","last_change":"2024-04-25T09:04:25.211486+0000","last_active":"2024-04-25T09:04:25.216595+0000","last_peered":"2024-04-25T09:04:25.216595+0000","last_clean":"2024-04-25T09:04:25.216595+0000","last_became_active":"2024-04-25T09:04:25.210675+0000","last_became_peered":"2024-04-25T09:04:25.210675+0000","last_unstale":"2024-04-25T09:04:25.216595+0000","last_undegraded":"2024-04-25T09:04:25.216595+0000","last_fullsized":"2024-04-25T09:04:25.216595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:46.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193611+0000","last_change":"2024-04-25T09:04:25.193611+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193611+0000","last_undegraded":"2024-04-25T09:04:25.193611+0000","last_fullsized":"2024-04-25T09:04:25.193611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211408+0000","last_change":"2024-04-25T09:04:25.205276+0000","last_active":"2024-04-25T09:04:25.211408+0000","last_peered":"2024-04-25T09:04:25.211408+0000","last_clean":"2024-04-25T09:04:25.211408+0000","last_became_active":"2024-04-25T09:04:25.204962+0000","last_became_peered":"2024-04-25T09:04:25.204962+0000","last_unstale":"2024-04-25T09:04:25.211408+0000","last_undegraded":"2024-04-25T09:04:25.211408+0000","last_fullsized":"2024-04-25T09:04:25.211408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:33.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180283+0000","last_change":"2024-04-25T09:04:25.180283+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180283+0000","last_undegraded":"2024-04-25T09:04:25.180283+0000","last_fullsized":"2024-04-25T09:04:25.180283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202059+0000","last_change":"2024-04-25T09:04:25.197723+0000","last_active":"2024-04-25T09:04:25.202059+0000","last_peered":"2024-04-25T09:04:25.202059+0000","last_clean":"2024-04-25T09:04:25.202059+0000","last_became_active":"2024-04-25T09:04:25.197275+0000","last_became_peered":"2024-04-25T09:04:25.197275+0000","last_unstale":"2024-04-25T09:04:25.202059+0000","last_undegraded":"2024-04-25T09:04:25.202059+0000","last_fullsized":"2024-04-25T09:04:25.202059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:46.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184657+0000","last_change":"2024-04-25T09:04:25.184657+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184657+0000","last_undegraded":"2024-04-25T09:04:25.184657+0000","last_fullsized":"2024-04-25T09:04:25.184657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.200271+0000","last_change":"2024-04-25T09:04:25.191706+0000","last_active":"2024-04-25T09:04:25.200271+0000","last_peered":"2024-04-25T09:04:25.200271+0000","last_clean":"2024-04-25T09:04:25.200271+0000","last_became_active":"2024-04-25T09:04:25.190455+0000","last_became_peered":"2024-04-25T09:04:25.190455+0000","last_unstale":"2024-04-25T09:04:25.200271+0000","last_undegraded":"2024-04-25T09:04:25.200271+0000","last_fullsized":"2024-04-25T09:04:25.200271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189103+0000","last_change":"2024-04-25T09:04:25.189103+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189103+0000","last_undegraded":"2024-04-25T09:04:25.189103+0000","last_fullsized":"2024-04-25T09:04:25.189103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207113+0000","last_change":"2024-04-25T09:04:25.203759+0000","last_active":"2024-04-25T09:04:25.207113+0000","last_peered":"2024-04-25T09:04:25.207113+0000","last_clean":"2024-04-25T09:04:25.207113+0000","last_became_active":"2024-04-25T09:04:25.203310+0000","last_became_peered":"2024-04-25T09:04:25.203310+0000","last_unstale":"2024-04-25T09:04:25.207113+0000","last_undegraded":"2024-04-25T09:04:25.207113+0000","last_fullsized":"2024-04-25T09:04:25.207113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:05.590884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186060+0000","last_change":"2024-04-25T09:04:25.186060+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186060+0000","last_undegraded":"2024-04-25T09:04:25.186060+0000","last_fullsized":"2024-04-25T09:04:25.186060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.208472+0000","last_change":"2024-04-25T09:04:25.206521+0000","last_active":"2024-04-25T09:04:25.208472+0000","last_peered":"2024-04-25T09:04:25.208472+0000","last_clean":"2024-04-25T09:04:25.208472+0000","last_became_active":"2024-04-25T09:04:25.206196+0000","last_became_peered":"2024-04-25T09:04:25.206196+0000","last_unstale":"2024-04-25T09:04:25.208472+0000","last_undegraded":"2024-04-25T09:04:25.208472+0000","last_fullsized":"2024-04-25T09:04:25.208472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:44.969455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184247+0000","last_change":"2024-04-25T09:04:25.184247+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184247+0000","last_undegraded":"2024-04-25T09:04:25.184247+0000","last_fullsized":"2024-04-25T09:04:25.184247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211386+0000","last_change":"2024-04-25T09:04:25.205066+0000","last_active":"2024-04-25T09:04:25.211386+0000","last_peered":"2024-04-25T09:04:25.211386+0000","last_clean":"2024-04-25T09:04:25.211386+0000","last_became_active":"2024-04-25T09:04:25.204741+0000","last_became_peered":"2024-04-25T09:04:25.204741+0000","last_unstale":"2024-04-25T09:04:25.211386+0000","last_undegraded":"2024-04-25T09:04:25.211386+0000","last_fullsized":"2024-04-25T09:04:25.211386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:27.537160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.188188+0000","last_change":"2024-04-25T09:04:25.188188+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.188188+0000","last_undegraded":"2024-04-25T09:04:25.188188+0000","last_fullsized":"2024-04-25T09:04:25.188188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207049+0000","last_change":"2024-04-25T09:04:25.203672+0000","last_active":"2024-04-25T09:04:25.207049+0000","last_peered":"2024-04-25T09:04:25.207049+0000","last_clean":"2024-04-25T09:04:25.207049+0000","last_became_active":"2024-04-25T09:04:25.203329+0000","last_became_peered":"2024-04-25T09:04:25.203329+0000","last_unstale":"2024-04-25T09:04:25.207049+0000","last_undegraded":"2024-04-25T09:04:25.207049+0000","last_fullsized":"2024-04-25T09:04:25.207049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:12.815263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186433+0000","last_change":"2024-04-25T09:04:25.186433+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186433+0000","last_undegraded":"2024-04-25T09:04:25.186433+0000","last_fullsized":"2024-04-25T09:04:25.186433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202354+0000","last_change":"2024-04-25T09:04:25.198618+0000","last_active":"2024-04-25T09:04:25.202354+0000","last_peered":"2024-04-25T09:04:25.202354+0000","last_clean":"2024-04-25T09:04:25.202354+0000","last_became_active":"2024-04-25T09:04:25.198320+0000","last_became_peered":"2024-04-25T09:04:25.198320+0000","last_unstale":"2024-04-25T09:04:25.202354+0000","last_undegraded":"2024-04-25T09:04:25.202354+0000","last_fullsized":"2024-04-25T09:04:25.202354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:27.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191367+0000","last_change":"2024-04-25T09:04:25.191367+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191367+0000","last_undegraded":"2024-04-25T09:04:25.191367+0000","last_fullsized":"2024-04-25T09:04:25.191367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209901+0000","last_change":"2024-04-25T09:04:25.206110+0000","last_active":"2024-04-25T09:04:25.209901+0000","last_peered":"2024-04-25T09:04:25.209901+0000","last_clean":"2024-04-25T09:04:25.209901+0000","last_became_active":"2024-04-25T09:04:25.205671+0000","last_became_peered":"2024-04-25T09:04:25.205671+0000","last_unstale":"2024-04-25T09:04:25.209901+0000","last_undegraded":"2024-04-25T09:04:25.209901+0000","last_fullsized":"2024-04-25T09:04:25.209901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:37.157522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195435+0000","last_change":"2024-04-25T09:04:25.195435+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195435+0000","last_undegraded":"2024-04-25T09:04:25.195435+0000","last_fullsized":"2024-04-25T09:04:25.195435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211443+0000","last_change":"2024-04-25T09:04:25.205857+0000","last_active":"2024-04-25T09:04:25.211443+0000","last_peered":"2024-04-25T09:04:25.211443+0000","last_clean":"2024-04-25T09:04:25.211443+0000","last_became_active":"2024-04-25T09:04:25.204814+0000","last_became_peered":"2024-04-25T09:04:25.204814+0000","last_unstale":"2024-04-25T09:04:25.211443+0000","last_undegraded":"2024-04-25T09:04:25.211443+0000","last_fullsized":"2024-04-25T09:04:25.211443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:46.169978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189734+0000","last_change":"2024-04-25T09:04:25.189734+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189734+0000","last_undegraded":"2024-04-25T09:04:25.189734+0000","last_fullsized":"2024-04-25T09:04:25.189734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205881+0000","last_change":"2024-04-25T09:04:25.203994+0000","last_active":"2024-04-25T09:04:25.205881+0000","last_peered":"2024-04-25T09:04:25.205881+0000","last_clean":"2024-04-25T09:04:25.205881+0000","last_became_active":"2024-04-25T09:04:25.203665+0000","last_became_peered":"2024-04-25T09:04:25.203665+0000","last_unstale":"2024-04-25T09:04:25.205881+0000","last_undegraded":"2024-04-25T09:04:25.205881+0000","last_fullsized":"2024-04-25T09:04:25.205881+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:45.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.181957+0000","last_change":"2024-04-25T09:04:25.181957+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.181957+0000","last_undegraded":"2024-04-25T09:04:25.181957+0000","last_fullsized":"2024-04-25T09:04:25.181957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.204263+0000","last_change":"2024-04-25T09:04:25.201881+0000","last_active":"2024-04-25T09:04:25.204263+0000","last_peered":"2024-04-25T09:04:25.204263+0000","last_clean":"2024-04-25T09:04:25.204263+0000","last_became_active":"2024-04-25T09:04:25.201582+0000","last_became_peered":"2024-04-25T09:04:25.201582+0000","last_unstale":"2024-04-25T09:04:25.204263+0000","last_undegraded":"2024-04-25T09:04:25.204263+0000","last_fullsized":"2024-04-25T09:04:25.204263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:17.652744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183729+0000","last_change":"2024-04-25T09:04:25.183729+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183729+0000","last_undegraded":"2024-04-25T09:04:25.183729+0000","last_fullsized":"2024-04-25T09:04:25.183729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205257+0000","last_change":"2024-04-25T09:04:25.201894+0000","last_active":"2024-04-25T09:04:25.205257+0000","last_peered":"2024-04-25T09:04:25.205257+0000","last_clean":"2024-04-25T09:04:25.205257+0000","last_became_active":"2024-04-25T09:04:25.201583+0000","last_became_peered":"2024-04-25T09:04:25.201583+0000","last_unstale":"2024-04-25T09:04:25.205257+0000","last_undegraded":"2024-04-25T09:04:25.205257+0000","last_fullsized":"2024-04-25T09:04:25.205257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:46.169978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192016+0000","last_change":"2024-04-25T09:04:25.192016+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192016+0000","last_undegraded":"2024-04-25T09:04:25.192016+0000","last_fullsized":"2024-04-25T09:04:25.192016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209813+0000","last_change":"2024-04-25T09:04:25.207893+0000","last_active":"2024-04-25T09:04:25.209813+0000","last_peered":"2024-04-25T09:04:25.209813+0000","last_clean":"2024-04-25T09:04:25.209813+0000","last_became_active":"2024-04-25T09:04:25.207587+0000","last_became_peered":"2024-04-25T09:04:25.207587+0000","last_unstale":"2024-04-25T09:04:25.209813+0000","last_undegraded":"2024-04-25T09:04:25.209813+0000","last_fullsized":"2024-04-25T09:04:25.209813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:04.049549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185065+0000","last_change":"2024-04-25T09:04:25.185065+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185065+0000","last_undegraded":"2024-04-25T09:04:25.185065+0000","last_fullsized":"2024-04-25T09:04:25.185065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.208184+0000","last_change":"2024-04-25T09:04:25.206786+0000","last_active":"2024-04-25T09:04:25.208184+0000","last_peered":"2024-04-25T09:04:25.208184+0000","last_clean":"2024-04-25T09:04:25.208184+0000","last_became_active":"2024-04-25T09:04:25.206460+0000","last_became_peered":"2024-04-25T09:04:25.206460+0000","last_unstale":"2024-04-25T09:04:25.208184+0000","last_undegraded":"2024-04-25T09:04:25.208184+0000","last_fullsized":"2024-04-25T09:04:25.208184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178764+0000","last_change":"2024-04-25T09:04:25.178764+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178764+0000","last_undegraded":"2024-04-25T09:04:25.178764+0000","last_fullsized":"2024-04-25T09:04:25.178764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205405+0000","last_change":"2024-04-25T09:04:25.203588+0000","last_active":"2024-04-25T09:04:25.205405+0000","last_peered":"2024-04-25T09:04:25.205405+0000","last_clean":"2024-04-25T09:04:25.205405+0000","last_became_active":"2024-04-25T09:04:25.203279+0000","last_became_peered":"2024-04-25T09:04:25.203279+0000","last_unstale":"2024-04-25T09:04:25.205405+0000","last_undegraded":"2024-04-25T09:04:25.205405+0000","last_fullsized":"2024-04-25T09:04:25.205405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178917+0000","last_change":"2024-04-25T09:04:25.178917+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178917+0000","last_undegraded":"2024-04-25T09:04:25.178917+0000","last_fullsized":"2024-04-25T09:04:25.178917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209329+0000","last_change":"2024-04-25T09:04:25.205414+0000","last_active":"2024-04-25T09:04:25.209329+0000","last_peered":"2024-04-25T09:04:25.209329+0000","last_clean":"2024-04-25T09:04:25.209329+0000","last_became_active":"2024-04-25T09:04:25.205062+0000","last_became_peered":"2024-04-25T09:04:25.205062+0000","last_unstale":"2024-04-25T09:04:25.209329+0000","last_undegraded":"2024-04-25T09:04:25.209329+0000","last_fullsized":"2024-04-25T09:04:25.209329+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:56.631288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.175676+0000","last_change":"2024-04-25T09:04:25.175676+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.175676+0000","last_undegraded":"2024-04-25T09:04:25.175676+0000","last_fullsized":"2024-04-25T09:04:25.175676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212068+0000","last_change":"2024-04-25T09:04:25.207990+0000","last_active":"2024-04-25T09:04:25.212068+0000","last_peered":"2024-04-25T09:04:25.212068+0000","last_clean":"2024-04-25T09:04:25.212068+0000","last_became_active":"2024-04-25T09:04:25.207576+0000","last_became_peered":"2024-04-25T09:04:25.207576+0000","last_unstale":"2024-04-25T09:04:25.212068+0000","last_undegraded":"2024-04-25T09:04:25.212068+0000","last_fullsized":"2024-04-25T09:04:25.212068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:56.631288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187851+0000","last_change":"2024-04-25T09:04:25.187851+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187851+0000","last_undegraded":"2024-04-25T09:04:25.187851+0000","last_fullsized":"2024-04-25T09:04:25.187851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.203149+0000","last_change":"2024-04-25T09:04:25.198844+0000","last_active":"2024-04-25T09:04:25.203149+0000","last_peered":"2024-04-25T09:04:25.203149+0000","last_clean":"2024-04-25T09:04:25.203149+0000","last_became_active":"2024-04-25T09:04:25.198376+0000","last_became_peered":"2024-04-25T09:04:25.198376+0000","last_unstale":"2024-04-25T09:04:25.203149+0000","last_undegraded":"2024-04-25T09:04:25.203149+0000","last_fullsized":"2024-04-25T09:04:25.203149+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:57.417759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179814+0000","last_change":"2024-04-25T09:04:25.179814+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179814+0000","last_undegraded":"2024-04-25T09:04:25.179814+0000","last_fullsized":"2024-04-25T09:04:25.179814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209328+0000","last_change":"2024-04-25T09:04:25.202687+0000","last_active":"2024-04-25T09:04:25.209328+0000","last_peered":"2024-04-25T09:04:25.209328+0000","last_clean":"2024-04-25T09:04:25.209328+0000","last_became_active":"2024-04-25T09:04:25.202400+0000","last_became_peered":"2024-04-25T09:04:25.202400+0000","last_unstale":"2024-04-25T09:04:25.209328+0000","last_undegraded":"2024-04-25T09:04:25.209328+0000","last_fullsized":"2024-04-25T09:04:25.209328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:50.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.185425+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:04:25.185425+0000","last_peered":"2024-04-25T09:04:25.185425+0000","last_clean":"2024-04-25T09:04:25.185425+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:04:25.185425+0000","last_undegraded":"2024-04-25T09:04:25.185425+0000","last_fullsized":"2024-04-25T09:04:25.185425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:31.175 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.434+0000 7ffb26c00640 1 Processor -- start 2024-04-25T09:04:31.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.434+0000 7ffb26c00640 1 -- start start 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.436+0000 7ffb26c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffb2015f450 0x7ffb200cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.436+0000 7ffb26c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2015faa0 con 0x7ffb2006d900 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.436+0000 7ffb26c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2015fd80 con 0x7ffb20144c10 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.436+0000 7ffb26c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb20144ff0 con 0x7ffb2015f450 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.436+0000 7ffb24e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ffb2006d900 0x7ffb200b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41466/0 (socket says 172.21.15.62:41466) 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.436+0000 7ffb24e00640 1 -- 172.21.15.62:0/2021081290 learned_addr learned my addr 172.21.15.62:0/2021081290 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.436+0000 7ffb26200640 1 --2- 172.21.15.62:0/2021081290 >> v2:172.21.15.62:3300/0 conn(0x7ffb2015f450 0x7ffb200cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.437+0000 7ffb1fe00640 1 -- 172.21.15.62:0/2021081290 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696395298 0 0) 0x7ffb2015faa0 con 0x7ffb2006d900 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.437+0000 7ffb1fe00640 1 -- 172.21.15.62:0/2021081290 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb08003520 con 0x7ffb2006d900 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.437+0000 7ffb26200640 1 -- 172.21.15.62:0/2021081290 >> v1:172.21.15.121:6789/0 conn(0x7ffb20144c10 legacy=0x7ffb2006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.437+0000 7ffb26200640 1 -- 172.21.15.62:0/2021081290 >> v1:172.21.15.62:6789/0 conn(0x7ffb2006d900 legacy=0x7ffb200b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.437+0000 7ffb26200640 1 -- 172.21.15.62:0/2021081290 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20160060 con 0x7ffb2015f450 2024-04-25T09:04:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.438+0000 7ffb26200640 1 --2- 172.21.15.62:0/2021081290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb2015f450 0x7ffb200cee40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009d90 tx=0x7ffb1802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9a8c7ce6914222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb1fe00640 1 -- 172.21.15.62:0/2021081290 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18007740 con 0x7ffb2015f450 2024-04-25T09:04:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb1fe00640 1 -- 172.21.15.62:0/2021081290 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb180078e0 con 0x7ffb2015f450 2024-04-25T09:04:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb1fe00640 1 -- 172.21.15.62:0/2021081290 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18007cb0 con 0x7ffb2015f450 2024-04-25T09:04:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb26c00640 1 -- 172.21.15.62:0/2021081290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb2015f450 msgr2=0x7ffb200cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb26c00640 1 --2- 172.21.15.62:0/2021081290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb2015f450 0x7ffb200cee40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009d90 tx=0x7ffb1802a660 comp rx=0 tx=0).stop 2024-04-25T09:04:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb26c00640 1 -- 172.21.15.62:0/2021081290 shutdown_connections 2024-04-25T09:04:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb26c00640 1 --2- 172.21.15.62:0/2021081290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb2015f450 0x7ffb200cee40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb26c00640 1 -- 172.21.15.62:0/2021081290 >> 172.21.15.62:0/2021081290 conn(0x7ffb200b14d0 msgr2=0x7ffb200b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.439+0000 7ffb26c00640 1 -- 172.21.15.62:0/2021081290 shutdown_connections 2024-04-25T09:04:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.440+0000 7ffb26c00640 1 -- 172.21.15.62:0/2021081290 wait complete. 2024-04-25T09:04:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.441+0000 7ffb26c00640 1 Processor -- start 2024-04-25T09:04:31.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.441+0000 7ffb26c00640 1 -- start start 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb26c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2006d900 0x7ffb2015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb26c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb20144c10 0x7ffb200c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb26c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2015f450 0x7ffb200c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb26c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200b5030 con 0x7ffb20144c10 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb26c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2015f880 con 0x7ffb2006d900 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb26c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200b51b0 con 0x7ffb2015f450 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb24e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb20144c10 0x7ffb200c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb24e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb20144c10 0x7ffb200c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36282/0 (socket says 172.21.15.62:36282) 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.442+0000 7ffb24e00640 1 -- 172.21.15.62:0/2101792832 learned_addr learned my addr 172.21.15.62:0/2101792832 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb25800640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2006d900 0x7ffb2015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb24e00640 1 -- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2015f450 msgr2=0x7ffb200c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb24e00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2015f450 0x7ffb200c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb24e00640 1 -- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2006d900 msgr2=0x7ffb2015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb24e00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2006d900 0x7ffb2015b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb24e00640 1 -- 172.21.15.62:0/2101792832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2015b6b0 con 0x7ffb20144c10 2024-04-25T09:04:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb24e00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb20144c10 0x7ffb200c7690 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c530 tx=0x7ffb0c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb1d600640 1 -- 172.21.15.62:0/2101792832 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c017070 con 0x7ffb20144c10 2024-04-25T09:04:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.443+0000 7ffb1d600640 1 -- 172.21.15.62:0/2101792832 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c005ce0 con 0x7ffb20144c10 2024-04-25T09:04:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.444+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb200c65f0 con 0x7ffb20144c10 2024-04-25T09:04:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.444+0000 7ffb1d600640 1 -- 172.21.15.62:0/2101792832 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0072b0 con 0x7ffb20144c10 2024-04-25T09:04:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.444+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb200c6e90 con 0x7ffb20144c10 2024-04-25T09:04:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.445+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2004ef20 con 0x7ffb20144c10 2024-04-25T09:04:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.446+0000 7ffb1d600640 1 -- 172.21.15.62:0/2101792832 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb0c01a070 con 0x7ffb20144c10 2024-04-25T09:04:31.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.447+0000 7ffb1d600640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb0006b500 0x7ffb0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.447+0000 7ffb1d600640 1 -- 172.21.15.62:0/2101792832 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb0c0895a0 con 0x7ffb20144c10 2024-04-25T09:04:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.447+0000 7ffb25800640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb0006b500 0x7ffb0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.449+0000 7ffb25800640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb0006b500 0x7ffb0006d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffb14000f60 tx=0x7ffb14009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.451+0000 7ffb1d600640 1 -- 172.21.15.62:0/2101792832 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb2015b6b0 con 0x7ffb20144c10 2024-04-25T09:04:31.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.640+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb200b22b0 con 0x7ffb0006b500 2024-04-25T09:04:31.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.659+0000 7ffb1d600640 1 -- 172.21.15.62:0/2101792832 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246633 (secure 0 0 0) 0x7ffb200b22b0 con 0x7ffb0006b500 2024-04-25T09:04:31.650 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:31.651 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:04:31.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb0006b500 msgr2=0x7ffb0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb0006b500 0x7ffb0006d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffb14000f60 tx=0x7ffb14009040 comp rx=0 tx=0).stop 2024-04-25T09:04:31.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb20144c10 msgr2=0x7ffb200c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb20144c10 0x7ffb200c7690 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c530 tx=0x7ffb0c00c9f0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 shutdown_connections 2024-04-25T09:04:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb0006b500 0x7ffb0006d9c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb20144c10 0x7ffb200c7690 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb2015f450 0x7ffb200c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 --2- 172.21.15.62:0/2101792832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb2006d900 0x7ffb2015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 >> 172.21.15.62:0/2101792832 conn(0x7ffb200b14d0 msgr2=0x7ffb200718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 shutdown_connections 2024-04-25T09:04:31.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.665+0000 7ffb26c00640 1 -- 172.21.15.62:0/2101792832 wait complete. 2024-04-25T09:04:31.671 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-25T09:04:29.909361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220060,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754660,"statfs":{"total":773094113280,"available":772868771840,"internally_reserved":0,"allocated":3973120,"data_stored":2245112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.745531"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195929+0000","last_change":"2024-04-25T09:04:25.195929+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195929+0000","last_undegraded":"2024-04-25T09:04:25.195929+0000","last_fullsized":"2024-04-25T09:04:25.195929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190558+0000","last_change":"2024-04-25T09:04:25.190558+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190558+0000","last_undegraded":"2024-04-25T09:04:25.190558+0000","last_fullsized":"2024-04-25T09:04:25.190558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184327+0000","last_change":"2024-04-25T09:04:25.184327+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184327+0000","last_undegraded":"2024-04-25T09:04:25.184327+0000","last_fullsized":"2024-04-25T09:04:25.184327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183027+0000","last_change":"2024-04-25T09:04:25.183027+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183027+0000","last_undegraded":"2024-04-25T09:04:25.183027+0000","last_fullsized":"2024-04-25T09:04:25.183027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.196108+0000","last_change":"2024-04-25T09:04:25.196108+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.196108+0000","last_undegraded":"2024-04-25T09:04:25.196108+0000","last_fullsized":"2024-04-25T09:04:25.196108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183693+0000","last_change":"2024-04-25T09:04:25.183693+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183693+0000","last_undegraded":"2024-04-25T09:04:25.183693+0000","last_fullsized":"2024-04-25T09:04:25.183693+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195170+0000","last_change":"2024-04-25T09:04:25.195170+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195170+0000","last_undegraded":"2024-04-25T09:04:25.195170+0000","last_fullsized":"2024-04-25T09:04:25.195170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195276+0000","last_change":"2024-04-25T09:04:25.195276+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195276+0000","last_undegraded":"2024-04-25T09:04:25.195276+0000","last_fullsized":"2024-04-25T09:04:25.195276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184881+0000","last_change":"2024-04-25T09:04:25.184881+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184881+0000","last_undegraded":"2024-04-25T09:04:25.184881+0000","last_fullsized":"2024-04-25T09:04:25.184881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183151+0000","last_change":"2024-04-25T09:04:25.183151+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183151+0000","last_undegraded":"2024-04-25T09:04:25.183151+0000","last_fullsized":"2024-04-25T09:04:25.183151+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189070+0000","last_change":"2024-04-25T09:04:25.189070+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189070+0000","last_undegraded":"2024-04-25T09:04:25.189070+0000","last_fullsized":"2024-04-25T09:04:25.189070+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.182768+0000","last_change":"2024-04-25T09:04:25.182768+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.182768+0000","last_undegraded":"2024-04-25T09:04:25.182768+0000","last_fullsized":"2024-04-25T09:04:25.182768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192960+0000","last_change":"2024-04-25T09:04:25.192960+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192960+0000","last_undegraded":"2024-04-25T09:04:25.192960+0000","last_fullsized":"2024-04-25T09:04:25.192960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.175725+0000","last_change":"2024-04-25T09:04:25.175725+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.175725+0000","last_undegraded":"2024-04-25T09:04:25.175725+0000","last_fullsized":"2024-04-25T09:04:25.175725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189104+0000","last_change":"2024-04-25T09:04:25.189104+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189104+0000","last_undegraded":"2024-04-25T09:04:25.189104+0000","last_fullsized":"2024-04-25T09:04:25.189104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180464+0000","last_change":"2024-04-25T09:04:25.180464+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180464+0000","last_undegraded":"2024-04-25T09:04:25.180464+0000","last_fullsized":"2024-04-25T09:04:25.180464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192509+0000","last_change":"2024-04-25T09:04:25.192509+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192509+0000","last_undegraded":"2024-04-25T09:04:25.192509+0000","last_fullsized":"2024-04-25T09:04:25.192509+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186012+0000","last_change":"2024-04-25T09:04:25.186012+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186012+0000","last_undegraded":"2024-04-25T09:04:25.186012+0000","last_fullsized":"2024-04-25T09:04:25.186012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187585+0000","last_change":"2024-04-25T09:04:25.187585+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187585+0000","last_undegraded":"2024-04-25T09:04:25.187585+0000","last_fullsized":"2024-04-25T09:04:25.187585+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.194517+0000","last_change":"2024-04-25T09:04:25.194517+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.194517+0000","last_undegraded":"2024-04-25T09:04:25.194517+0000","last_fullsized":"2024-04-25T09:04:25.194517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180248+0000","last_change":"2024-04-25T09:04:25.180248+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180248+0000","last_undegraded":"2024-04-25T09:04:25.180248+0000","last_fullsized":"2024-04-25T09:04:25.180248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193801+0000","last_change":"2024-04-25T09:04:25.193801+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193801+0000","last_undegraded":"2024-04-25T09:04:25.193801+0000","last_fullsized":"2024-04-25T09:04:25.193801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190403+0000","last_change":"2024-04-25T09:04:25.190403+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190403+0000","last_undegraded":"2024-04-25T09:04:25.190403+0000","last_fullsized":"2024-04-25T09:04:25.190403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192553+0000","last_change":"2024-04-25T09:04:25.192553+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192553+0000","last_undegraded":"2024-04-25T09:04:25.192553+0000","last_fullsized":"2024-04-25T09:04:25.192553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179859+0000","last_change":"2024-04-25T09:04:25.179859+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179859+0000","last_undegraded":"2024-04-25T09:04:25.179859+0000","last_fullsized":"2024-04-25T09:04:25.179859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178338+0000","last_change":"2024-04-25T09:04:25.178338+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178338+0000","last_undegraded":"2024-04-25T09:04:25.178338+0000","last_fullsized":"2024-04-25T09:04:25.178338+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.188145+0000","last_change":"2024-04-25T09:04:25.188145+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.188145+0000","last_undegraded":"2024-04-25T09:04:25.188145+0000","last_fullsized":"2024-04-25T09:04:25.188145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.190431+0000","last_change":"2024-04-25T09:04:25.190431+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.190431+0000","last_undegraded":"2024-04-25T09:04:25.190431+0000","last_fullsized":"2024-04-25T09:04:25.190431+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179645+0000","last_change":"2024-04-25T09:04:25.179645+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179645+0000","last_undegraded":"2024-04-25T09:04:25.179645+0000","last_fullsized":"2024-04-25T09:04:25.179645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179586+0000","last_change":"2024-04-25T09:04:25.179586+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179586+0000","last_undegraded":"2024-04-25T09:04:25.179586+0000","last_fullsized":"2024-04-25T09:04:25.179586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185200+0000","last_change":"2024-04-25T09:04:25.185200+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185200+0000","last_undegraded":"2024-04-25T09:04:25.185200+0000","last_fullsized":"2024-04-25T09:04:25.185200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187138+0000","last_change":"2024-04-25T09:04:25.187138+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187138+0000","last_undegraded":"2024-04-25T09:04:25.187138+0000","last_fullsized":"2024-04-25T09:04:25.187138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191755+0000","last_change":"2024-04-25T09:04:25.191755+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191755+0000","last_undegraded":"2024-04-25T09:04:25.191755+0000","last_fullsized":"2024-04-25T09:04:25.191755+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211702+0000","last_change":"2024-04-25T09:04:25.210460+0000","last_active":"2024-04-25T09:04:25.211702+0000","last_peered":"2024-04-25T09:04:25.211702+0000","last_clean":"2024-04-25T09:04:25.211702+0000","last_became_active":"2024-04-25T09:04:25.210130+0000","last_became_peered":"2024-04-25T09:04:25.210130+0000","last_unstale":"2024-04-25T09:04:25.211702+0000","last_undegraded":"2024-04-25T09:04:25.211702+0000","last_fullsized":"2024-04-25T09:04:25.211702+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:20.178362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187891+0000","last_change":"2024-04-25T09:04:25.187891+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187891+0000","last_undegraded":"2024-04-25T09:04:25.187891+0000","last_fullsized":"2024-04-25T09:04:25.187891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.203300+0000","last_change":"2024-04-25T09:04:25.198907+0000","last_active":"2024-04-25T09:04:25.203300+0000","last_peered":"2024-04-25T09:04:25.203300+0000","last_clean":"2024-04-25T09:04:25.203300+0000","last_became_active":"2024-04-25T09:04:25.198368+0000","last_became_peered":"2024-04-25T09:04:25.198368+0000","last_unstale":"2024-04-25T09:04:25.203300+0000","last_undegraded":"2024-04-25T09:04:25.203300+0000","last_fullsized":"2024-04-25T09:04:25.203300+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:43.725847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189966+0000","last_change":"2024-04-25T09:04:25.189966+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189966+0000","last_undegraded":"2024-04-25T09:04:25.189966+0000","last_fullsized":"2024-04-25T09:04:25.189966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.217010+0000","last_change":"2024-04-25T09:04:25.211094+0000","last_active":"2024-04-25T09:04:25.217010+0000","last_peered":"2024-04-25T09:04:25.217010+0000","last_clean":"2024-04-25T09:04:25.217010+0000","last_became_active":"2024-04-25T09:04:25.210803+0000","last_became_peered":"2024-04-25T09:04:25.210803+0000","last_unstale":"2024-04-25T09:04:25.217010+0000","last_undegraded":"2024-04-25T09:04:25.217010+0000","last_fullsized":"2024-04-25T09:04:25.217010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:31.510668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.177047+0000","last_change":"2024-04-25T09:04:25.177047+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.177047+0000","last_undegraded":"2024-04-25T09:04:25.177047+0000","last_fullsized":"2024-04-25T09:04:25.177047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211824+0000","last_change":"2024-04-25T09:04:25.209064+0000","last_active":"2024-04-25T09:04:25.211824+0000","last_peered":"2024-04-25T09:04:25.211824+0000","last_clean":"2024-04-25T09:04:25.211824+0000","last_became_active":"2024-04-25T09:04:25.208733+0000","last_became_peered":"2024-04-25T09:04:25.208733+0000","last_unstale":"2024-04-25T09:04:25.211824+0000","last_undegraded":"2024-04-25T09:04:25.211824+0000","last_fullsized":"2024-04-25T09:04:25.211824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.182891+0000","last_change":"2024-04-25T09:04:25.182891+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.182891+0000","last_undegraded":"2024-04-25T09:04:25.182891+0000","last_fullsized":"2024-04-25T09:04:25.182891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212083+0000","last_change":"2024-04-25T09:04:25.207956+0000","last_active":"2024-04-25T09:04:25.212083+0000","last_peered":"2024-04-25T09:04:25.212083+0000","last_clean":"2024-04-25T09:04:25.212083+0000","last_became_active":"2024-04-25T09:04:25.207646+0000","last_became_peered":"2024-04-25T09:04:25.207646+0000","last_unstale":"2024-04-25T09:04:25.212083+0000","last_undegraded":"2024-04-25T09:04:25.212083+0000","last_fullsized":"2024-04-25T09:04:25.212083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185181+0000","last_change":"2024-04-25T09:04:25.185181+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185181+0000","last_undegraded":"2024-04-25T09:04:25.185181+0000","last_fullsized":"2024-04-25T09:04:25.185181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212097+0000","last_change":"2024-04-25T09:04:25.209033+0000","last_active":"2024-04-25T09:04:25.212097+0000","last_peered":"2024-04-25T09:04:25.212097+0000","last_clean":"2024-04-25T09:04:25.212097+0000","last_became_active":"2024-04-25T09:04:25.208734+0000","last_became_peered":"2024-04-25T09:04:25.208734+0000","last_unstale":"2024-04-25T09:04:25.212097+0000","last_undegraded":"2024-04-25T09:04:25.212097+0000","last_fullsized":"2024-04-25T09:04:25.212097+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183410+0000","last_change":"2024-04-25T09:04:25.183410+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183410+0000","last_undegraded":"2024-04-25T09:04:25.183410+0000","last_fullsized":"2024-04-25T09:04:25.183410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.216919+0000","last_change":"2024-04-25T09:04:25.209807+0000","last_active":"2024-04-25T09:04:25.216919+0000","last_peered":"2024-04-25T09:04:25.216919+0000","last_clean":"2024-04-25T09:04:25.216919+0000","last_became_active":"2024-04-25T09:04:25.207401+0000","last_became_peered":"2024-04-25T09:04:25.207401+0000","last_unstale":"2024-04-25T09:04:25.216919+0000","last_undegraded":"2024-04-25T09:04:25.216919+0000","last_fullsized":"2024-04-25T09:04:25.216919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:45.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.176143+0000","last_change":"2024-04-25T09:04:25.176143+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.176143+0000","last_undegraded":"2024-04-25T09:04:25.176143+0000","last_fullsized":"2024-04-25T09:04:25.176143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202536+0000","last_change":"2024-04-25T09:04:25.200573+0000","last_active":"2024-04-25T09:04:25.202536+0000","last_peered":"2024-04-25T09:04:25.202536+0000","last_clean":"2024-04-25T09:04:25.202536+0000","last_became_active":"2024-04-25T09:04:25.200262+0000","last_became_peered":"2024-04-25T09:04:25.200262+0000","last_unstale":"2024-04-25T09:04:25.202536+0000","last_undegraded":"2024-04-25T09:04:25.202536+0000","last_fullsized":"2024-04-25T09:04:25.202536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:45.866728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191346+0000","last_change":"2024-04-25T09:04:25.191346+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191346+0000","last_undegraded":"2024-04-25T09:04:25.191346+0000","last_fullsized":"2024-04-25T09:04:25.191346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207779+0000","last_change":"2024-04-25T09:04:25.202054+0000","last_active":"2024-04-25T09:04:25.207779+0000","last_peered":"2024-04-25T09:04:25.207779+0000","last_clean":"2024-04-25T09:04:25.207779+0000","last_became_active":"2024-04-25T09:04:25.201654+0000","last_became_peered":"2024-04-25T09:04:25.201654+0000","last_unstale":"2024-04-25T09:04:25.207779+0000","last_undegraded":"2024-04-25T09:04:25.207779+0000","last_fullsized":"2024-04-25T09:04:25.207779+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.834376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.177649+0000","last_change":"2024-04-25T09:04:25.177649+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.177649+0000","last_undegraded":"2024-04-25T09:04:25.177649+0000","last_fullsized":"2024-04-25T09:04:25.177649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209791+0000","last_change":"2024-04-25T09:04:25.207756+0000","last_active":"2024-04-25T09:04:25.209791+0000","last_peered":"2024-04-25T09:04:25.209791+0000","last_clean":"2024-04-25T09:04:25.209791+0000","last_became_active":"2024-04-25T09:04:25.207460+0000","last_became_peered":"2024-04-25T09:04:25.207460+0000","last_unstale":"2024-04-25T09:04:25.209791+0000","last_undegraded":"2024-04-25T09:04:25.209791+0000","last_fullsized":"2024-04-25T09:04:25.209791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:21.753154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179742+0000","last_change":"2024-04-25T09:04:25.179742+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179742+0000","last_undegraded":"2024-04-25T09:04:25.179742+0000","last_fullsized":"2024-04-25T09:04:25.179742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209792+0000","last_change":"2024-04-25T09:04:25.207463+0000","last_active":"2024-04-25T09:04:25.209792+0000","last_peered":"2024-04-25T09:04:25.209792+0000","last_clean":"2024-04-25T09:04:25.209792+0000","last_became_active":"2024-04-25T09:04:25.207178+0000","last_became_peered":"2024-04-25T09:04:25.207178+0000","last_unstale":"2024-04-25T09:04:25.209792+0000","last_undegraded":"2024-04-25T09:04:25.209792+0000","last_fullsized":"2024-04-25T09:04:25.209792+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:38.564239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193205+0000","last_change":"2024-04-25T09:04:25.193205+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193205+0000","last_undegraded":"2024-04-25T09:04:25.193205+0000","last_fullsized":"2024-04-25T09:04:25.193205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.216595+0000","last_change":"2024-04-25T09:04:25.211486+0000","last_active":"2024-04-25T09:04:25.216595+0000","last_peered":"2024-04-25T09:04:25.216595+0000","last_clean":"2024-04-25T09:04:25.216595+0000","last_became_active":"2024-04-25T09:04:25.210675+0000","last_became_peered":"2024-04-25T09:04:25.210675+0000","last_unstale":"2024-04-25T09:04:25.216595+0000","last_undegraded":"2024-04-25T09:04:25.216595+0000","last_fullsized":"2024-04-25T09:04:25.216595+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:46.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.193611+0000","last_change":"2024-04-25T09:04:25.193611+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.193611+0000","last_undegraded":"2024-04-25T09:04:25.193611+0000","last_fullsized":"2024-04-25T09:04:25.193611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211408+0000","last_change":"2024-04-25T09:04:25.205276+0000","last_active":"2024-04-25T09:04:25.211408+0000","last_peered":"2024-04-25T09:04:25.211408+0000","last_clean":"2024-04-25T09:04:25.211408+0000","last_became_active":"2024-04-25T09:04:25.204962+0000","last_became_peered":"2024-04-25T09:04:25.204962+0000","last_unstale":"2024-04-25T09:04:25.211408+0000","last_undegraded":"2024-04-25T09:04:25.211408+0000","last_fullsized":"2024-04-25T09:04:25.211408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:33.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.180283+0000","last_change":"2024-04-25T09:04:25.180283+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.180283+0000","last_undegraded":"2024-04-25T09:04:25.180283+0000","last_fullsized":"2024-04-25T09:04:25.180283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202059+0000","last_change":"2024-04-25T09:04:25.197723+0000","last_active":"2024-04-25T09:04:25.202059+0000","last_peered":"2024-04-25T09:04:25.202059+0000","last_clean":"2024-04-25T09:04:25.202059+0000","last_became_active":"2024-04-25T09:04:25.197275+0000","last_became_peered":"2024-04-25T09:04:25.197275+0000","last_unstale":"2024-04-25T09:04:25.202059+0000","last_undegraded":"2024-04-25T09:04:25.202059+0000","last_fullsized":"2024-04-25T09:04:25.202059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:46.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184657+0000","last_change":"2024-04-25T09:04:25.184657+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184657+0000","last_undegraded":"2024-04-25T09:04:25.184657+0000","last_fullsized":"2024-04-25T09:04:25.184657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.200271+0000","last_change":"2024-04-25T09:04:25.191706+0000","last_active":"2024-04-25T09:04:25.200271+0000","last_peered":"2024-04-25T09:04:25.200271+0000","last_clean":"2024-04-25T09:04:25.200271+0000","last_became_active":"2024-04-25T09:04:25.190455+0000","last_became_peered":"2024-04-25T09:04:25.190455+0000","last_unstale":"2024-04-25T09:04:25.200271+0000","last_undegraded":"2024-04-25T09:04:25.200271+0000","last_fullsized":"2024-04-25T09:04:25.200271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189103+0000","last_change":"2024-04-25T09:04:25.189103+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189103+0000","last_undegraded":"2024-04-25T09:04:25.189103+0000","last_fullsized":"2024-04-25T09:04:25.189103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207113+0000","last_change":"2024-04-25T09:04:25.203759+0000","last_active":"2024-04-25T09:04:25.207113+0000","last_peered":"2024-04-25T09:04:25.207113+0000","last_clean":"2024-04-25T09:04:25.207113+0000","last_became_active":"2024-04-25T09:04:25.203310+0000","last_became_peered":"2024-04-25T09:04:25.203310+0000","last_unstale":"2024-04-25T09:04:25.207113+0000","last_undegraded":"2024-04-25T09:04:25.207113+0000","last_fullsized":"2024-04-25T09:04:25.207113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:05.590884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186060+0000","last_change":"2024-04-25T09:04:25.186060+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186060+0000","last_undegraded":"2024-04-25T09:04:25.186060+0000","last_fullsized":"2024-04-25T09:04:25.186060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.208472+0000","last_change":"2024-04-25T09:04:25.206521+0000","last_active":"2024-04-25T09:04:25.208472+0000","last_peered":"2024-04-25T09:04:25.208472+0000","last_clean":"2024-04-25T09:04:25.208472+0000","last_became_active":"2024-04-25T09:04:25.206196+0000","last_became_peered":"2024-04-25T09:04:25.206196+0000","last_unstale":"2024-04-25T09:04:25.208472+0000","last_undegraded":"2024-04-25T09:04:25.208472+0000","last_fullsized":"2024-04-25T09:04:25.208472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:44.969455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.184247+0000","last_change":"2024-04-25T09:04:25.184247+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.184247+0000","last_undegraded":"2024-04-25T09:04:25.184247+0000","last_fullsized":"2024-04-25T09:04:25.184247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211386+0000","last_change":"2024-04-25T09:04:25.205066+0000","last_active":"2024-04-25T09:04:25.211386+0000","last_peered":"2024-04-25T09:04:25.211386+0000","last_clean":"2024-04-25T09:04:25.211386+0000","last_became_active":"2024-04-25T09:04:25.204741+0000","last_became_peered":"2024-04-25T09:04:25.204741+0000","last_unstale":"2024-04-25T09:04:25.211386+0000","last_undegraded":"2024-04-25T09:04:25.211386+0000","last_fullsized":"2024-04-25T09:04:25.211386+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:27.537160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.188188+0000","last_change":"2024-04-25T09:04:25.188188+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.188188+0000","last_undegraded":"2024-04-25T09:04:25.188188+0000","last_fullsized":"2024-04-25T09:04:25.188188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.207049+0000","last_change":"2024-04-25T09:04:25.203672+0000","last_active":"2024-04-25T09:04:25.207049+0000","last_peered":"2024-04-25T09:04:25.207049+0000","last_clean":"2024-04-25T09:04:25.207049+0000","last_became_active":"2024-04-25T09:04:25.203329+0000","last_became_peered":"2024-04-25T09:04:25.203329+0000","last_unstale":"2024-04-25T09:04:25.207049+0000","last_undegraded":"2024-04-25T09:04:25.207049+0000","last_fullsized":"2024-04-25T09:04:25.207049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:12.815263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.186433+0000","last_change":"2024-04-25T09:04:25.186433+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.186433+0000","last_undegraded":"2024-04-25T09:04:25.186433+0000","last_fullsized":"2024-04-25T09:04:25.186433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.202354+0000","last_change":"2024-04-25T09:04:25.198618+0000","last_active":"2024-04-25T09:04:25.202354+0000","last_peered":"2024-04-25T09:04:25.202354+0000","last_clean":"2024-04-25T09:04:25.202354+0000","last_became_active":"2024-04-25T09:04:25.198320+0000","last_became_peered":"2024-04-25T09:04:25.198320+0000","last_unstale":"2024-04-25T09:04:25.202354+0000","last_undegraded":"2024-04-25T09:04:25.202354+0000","last_fullsized":"2024-04-25T09:04:25.202354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:27.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.191367+0000","last_change":"2024-04-25T09:04:25.191367+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.191367+0000","last_undegraded":"2024-04-25T09:04:25.191367+0000","last_fullsized":"2024-04-25T09:04:25.191367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209901+0000","last_change":"2024-04-25T09:04:25.206110+0000","last_active":"2024-04-25T09:04:25.209901+0000","last_peered":"2024-04-25T09:04:25.209901+0000","last_clean":"2024-04-25T09:04:25.209901+0000","last_became_active":"2024-04-25T09:04:25.205671+0000","last_became_peered":"2024-04-25T09:04:25.205671+0000","last_unstale":"2024-04-25T09:04:25.209901+0000","last_undegraded":"2024-04-25T09:04:25.209901+0000","last_fullsized":"2024-04-25T09:04:25.209901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:37.157522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.195435+0000","last_change":"2024-04-25T09:04:25.195435+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.195435+0000","last_undegraded":"2024-04-25T09:04:25.195435+0000","last_fullsized":"2024-04-25T09:04:25.195435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.211443+0000","last_change":"2024-04-25T09:04:25.205857+0000","last_active":"2024-04-25T09:04:25.211443+0000","last_peered":"2024-04-25T09:04:25.211443+0000","last_clean":"2024-04-25T09:04:25.211443+0000","last_became_active":"2024-04-25T09:04:25.204814+0000","last_became_peered":"2024-04-25T09:04:25.204814+0000","last_unstale":"2024-04-25T09:04:25.211443+0000","last_undegraded":"2024-04-25T09:04:25.211443+0000","last_fullsized":"2024-04-25T09:04:25.211443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:46.169978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.189734+0000","last_change":"2024-04-25T09:04:25.189734+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.189734+0000","last_undegraded":"2024-04-25T09:04:25.189734+0000","last_fullsized":"2024-04-25T09:04:25.189734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205881+0000","last_change":"2024-04-25T09:04:25.203994+0000","last_active":"2024-04-25T09:04:25.205881+0000","last_peered":"2024-04-25T09:04:25.205881+0000","last_clean":"2024-04-25T09:04:25.205881+0000","last_became_active":"2024-04-25T09:04:25.203665+0000","last_became_peered":"2024-04-25T09:04:25.203665+0000","last_unstale":"2024-04-25T09:04:25.205881+0000","last_undegraded":"2024-04-25T09:04:25.205881+0000","last_fullsized":"2024-04-25T09:04:25.205881+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:45.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.181957+0000","last_change":"2024-04-25T09:04:25.181957+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.181957+0000","last_undegraded":"2024-04-25T09:04:25.181957+0000","last_fullsized":"2024-04-25T09:04:25.181957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.204263+0000","last_change":"2024-04-25T09:04:25.201881+0000","last_active":"2024-04-25T09:04:25.204263+0000","last_peered":"2024-04-25T09:04:25.204263+0000","last_clean":"2024-04-25T09:04:25.204263+0000","last_became_active":"2024-04-25T09:04:25.201582+0000","last_became_peered":"2024-04-25T09:04:25.201582+0000","last_unstale":"2024-04-25T09:04:25.204263+0000","last_undegraded":"2024-04-25T09:04:25.204263+0000","last_fullsized":"2024-04-25T09:04:25.204263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:17.652744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.183729+0000","last_change":"2024-04-25T09:04:25.183729+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.183729+0000","last_undegraded":"2024-04-25T09:04:25.183729+0000","last_fullsized":"2024-04-25T09:04:25.183729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205257+0000","last_change":"2024-04-25T09:04:25.201894+0000","last_active":"2024-04-25T09:04:25.205257+0000","last_peered":"2024-04-25T09:04:25.205257+0000","last_clean":"2024-04-25T09:04:25.205257+0000","last_became_active":"2024-04-25T09:04:25.201583+0000","last_became_peered":"2024-04-25T09:04:25.201583+0000","last_unstale":"2024-04-25T09:04:25.205257+0000","last_undegraded":"2024-04-25T09:04:25.205257+0000","last_fullsized":"2024-04-25T09:04:25.205257+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:46.169978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.192016+0000","last_change":"2024-04-25T09:04:25.192016+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.192016+0000","last_undegraded":"2024-04-25T09:04:25.192016+0000","last_fullsized":"2024-04-25T09:04:25.192016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209813+0000","last_change":"2024-04-25T09:04:25.207893+0000","last_active":"2024-04-25T09:04:25.209813+0000","last_peered":"2024-04-25T09:04:25.209813+0000","last_clean":"2024-04-25T09:04:25.209813+0000","last_became_active":"2024-04-25T09:04:25.207587+0000","last_became_peered":"2024-04-25T09:04:25.207587+0000","last_unstale":"2024-04-25T09:04:25.209813+0000","last_undegraded":"2024-04-25T09:04:25.209813+0000","last_fullsized":"2024-04-25T09:04:25.209813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:04.049549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.185065+0000","last_change":"2024-04-25T09:04:25.185065+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.185065+0000","last_undegraded":"2024-04-25T09:04:25.185065+0000","last_fullsized":"2024-04-25T09:04:25.185065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.208184+0000","last_change":"2024-04-25T09:04:25.206786+0000","last_active":"2024-04-25T09:04:25.208184+0000","last_peered":"2024-04-25T09:04:25.208184+0000","last_clean":"2024-04-25T09:04:25.208184+0000","last_became_active":"2024-04-25T09:04:25.206460+0000","last_became_peered":"2024-04-25T09:04:25.206460+0000","last_unstale":"2024-04-25T09:04:25.208184+0000","last_undegraded":"2024-04-25T09:04:25.208184+0000","last_fullsized":"2024-04-25T09:04:25.208184+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:25.582921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178764+0000","last_change":"2024-04-25T09:04:25.178764+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178764+0000","last_undegraded":"2024-04-25T09:04:25.178764+0000","last_fullsized":"2024-04-25T09:04:25.178764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.205405+0000","last_change":"2024-04-25T09:04:25.203588+0000","last_active":"2024-04-25T09:04:25.205405+0000","last_peered":"2024-04-25T09:04:25.205405+0000","last_clean":"2024-04-25T09:04:25.205405+0000","last_became_active":"2024-04-25T09:04:25.203279+0000","last_became_peered":"2024-04-25T09:04:25.203279+0000","last_unstale":"2024-04-25T09:04:25.205405+0000","last_undegraded":"2024-04-25T09:04:25.205405+0000","last_fullsized":"2024-04-25T09:04:25.205405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:02.363010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.178917+0000","last_change":"2024-04-25T09:04:25.178917+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.178917+0000","last_undegraded":"2024-04-25T09:04:25.178917+0000","last_fullsized":"2024-04-25T09:04:25.178917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209329+0000","last_change":"2024-04-25T09:04:25.205414+0000","last_active":"2024-04-25T09:04:25.209329+0000","last_peered":"2024-04-25T09:04:25.209329+0000","last_clean":"2024-04-25T09:04:25.209329+0000","last_became_active":"2024-04-25T09:04:25.205062+0000","last_became_peered":"2024-04-25T09:04:25.205062+0000","last_unstale":"2024-04-25T09:04:25.209329+0000","last_undegraded":"2024-04-25T09:04:25.209329+0000","last_fullsized":"2024-04-25T09:04:25.209329+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:56.631288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.175676+0000","last_change":"2024-04-25T09:04:25.175676+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.175676+0000","last_undegraded":"2024-04-25T09:04:25.175676+0000","last_fullsized":"2024-04-25T09:04:25.175676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.212068+0000","last_change":"2024-04-25T09:04:25.207990+0000","last_active":"2024-04-25T09:04:25.212068+0000","last_peered":"2024-04-25T09:04:25.212068+0000","last_clean":"2024-04-25T09:04:25.212068+0000","last_became_active":"2024-04-25T09:04:25.207576+0000","last_became_peered":"2024-04-25T09:04:25.207576+0000","last_unstale":"2024-04-25T09:04:25.212068+0000","last_undegraded":"2024-04-25T09:04:25.212068+0000","last_fullsized":"2024-04-25T09:04:25.212068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:56.631288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.187851+0000","last_change":"2024-04-25T09:04:25.187851+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.187851+0000","last_undegraded":"2024-04-25T09:04:25.187851+0000","last_fullsized":"2024-04-25T09:04:25.187851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.203149+0000","last_change":"2024-04-25T09:04:25.198844+0000","last_active":"2024-04-25T09:04:25.203149+0000","last_peered":"2024-04-25T09:04:25.203149+0000","last_clean":"2024-04-25T09:04:25.203149+0000","last_became_active":"2024-04-25T09:04:25.198376+0000","last_became_peered":"2024-04-25T09:04:25.198376+0000","last_unstale":"2024-04-25T09:04:25.203149+0000","last_undegraded":"2024-04-25T09:04:25.203149+0000","last_fullsized":"2024-04-25T09:04:25.203149+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:57.417759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T09:04:25.179814+0000","last_change":"2024-04-25T09:04:25.179814+0000","last_active":"2024-04-25T09:04:25.160001+0000","last_peered":"2024-04-25T09:04:25.160001+0000","last_clean":"2024-04-25T09:04:25.160001+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:25.179814+0000","last_undegraded":"2024-04-25T09:04:25.179814+0000","last_fullsized":"2024-04-25T09:04:25.179814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.209328+0000","last_change":"2024-04-25T09:04:25.202687+0000","last_active":"2024-04-25T09:04:25.209328+0000","last_peered":"2024-04-25T09:04:25.209328+0000","last_clean":"2024-04-25T09:04:25.209328+0000","last_became_active":"2024-04-25T09:04:25.202400+0000","last_became_peered":"2024-04-25T09:04:25.202400+0000","last_unstale":"2024-04-25T09:04:25.209328+0000","last_undegraded":"2024-04-25T09:04:25.209328+0000","last_fullsized":"2024-04-25T09:04:25.209328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:50.157609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T09:04:25.185425+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:04:25.185425+0000","last_peered":"2024-04-25T09:04:25.185425+0000","last_clean":"2024-04-25T09:04:25.185425+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:04:25.185425+0000","last_undegraded":"2024-04-25T09:04:25.185425+0000","last_fullsized":"2024-04-25T09:04:25.185425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:31.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-25T09:04:31.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.195929+0000', 'last_change': '2024-04-25T09:04:25.195929+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.195929+0000', 'last_undegraded': '2024-04-25T09:04:25.195929+0000', 'last_fullsized': '2024-04-25T09:04:25.195929+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-25T09:04:31.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.190558+0000', 'last_change': '2024-04-25T09:04:25.190558+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.190558+0000', 'last_undegraded': '2024-04-25T09:04:25.190558+0000', 'last_fullsized': '2024-04-25T09:04:25.190558+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:31.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-25T09:04:31.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.184327+0000', 'last_change': '2024-04-25T09:04:25.184327+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.184327+0000', 'last_undegraded': '2024-04-25T09:04:25.184327+0000', 'last_fullsized': '2024-04-25T09:04:25.184327+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:31.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-25T09:04:31.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.183027+0000', 'last_change': '2024-04-25T09:04:25.183027+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.183027+0000', 'last_undegraded': '2024-04-25T09:04:25.183027+0000', 'last_fullsized': '2024-04-25T09:04:25.183027+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.196108+0000', 'last_change': '2024-04-25T09:04:25.196108+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.196108+0000', 'last_undegraded': '2024-04-25T09:04:25.196108+0000', 'last_fullsized': '2024-04-25T09:04:25.196108+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.183693+0000', 'last_change': '2024-04-25T09:04:25.183693+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.183693+0000', 'last_undegraded': '2024-04-25T09:04:25.183693+0000', 'last_fullsized': '2024-04-25T09:04:25.183693+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.195170+0000', 'last_change': '2024-04-25T09:04:25.195170+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.195170+0000', 'last_undegraded': '2024-04-25T09:04:25.195170+0000', 'last_fullsized': '2024-04-25T09:04:25.195170+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.195276+0000', 'last_change': '2024-04-25T09:04:25.195276+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.195276+0000', 'last_undegraded': '2024-04-25T09:04:25.195276+0000', 'last_fullsized': '2024-04-25T09:04:25.195276+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.184881+0000', 'last_change': '2024-04-25T09:04:25.184881+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.184881+0000', 'last_undegraded': '2024-04-25T09:04:25.184881+0000', 'last_fullsized': '2024-04-25T09:04:25.184881+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-25T09:04:31.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.183151+0000', 'last_change': '2024-04-25T09:04:25.183151+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.183151+0000', 'last_undegraded': '2024-04-25T09:04:25.183151+0000', 'last_fullsized': '2024-04-25T09:04:25.183151+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.189070+0000', 'last_change': '2024-04-25T09:04:25.189070+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.189070+0000', 'last_undegraded': '2024-04-25T09:04:25.189070+0000', 'last_fullsized': '2024-04-25T09:04:25.189070+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.182768+0000', 'last_change': '2024-04-25T09:04:25.182768+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.182768+0000', 'last_undegraded': '2024-04-25T09:04:25.182768+0000', 'last_fullsized': '2024-04-25T09:04:25.182768+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.192960+0000', 'last_change': '2024-04-25T09:04:25.192960+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.192960+0000', 'last_undegraded': '2024-04-25T09:04:25.192960+0000', 'last_fullsized': '2024-04-25T09:04:25.192960+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.175725+0000', 'last_change': '2024-04-25T09:04:25.175725+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.175725+0000', 'last_undegraded': '2024-04-25T09:04:25.175725+0000', 'last_fullsized': '2024-04-25T09:04:25.175725+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-25T09:04:31.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.189104+0000', 'last_change': '2024-04-25T09:04:25.189104+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.189104+0000', 'last_undegraded': '2024-04-25T09:04:25.189104+0000', 'last_fullsized': '2024-04-25T09:04:25.189104+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:31.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-25T09:04:31.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.180464+0000', 'last_change': '2024-04-25T09:04:25.180464+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.180464+0000', 'last_undegraded': '2024-04-25T09:04:25.180464+0000', 'last_fullsized': '2024-04-25T09:04:25.180464+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:31.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-25T09:04:31.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.192509+0000', 'last_change': '2024-04-25T09:04:25.192509+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.192509+0000', 'last_undegraded': '2024-04-25T09:04:25.192509+0000', 'last_fullsized': '2024-04-25T09:04:25.192509+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-25T09:04:31.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.186012+0000', 'last_change': '2024-04-25T09:04:25.186012+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.186012+0000', 'last_undegraded': '2024-04-25T09:04:25.186012+0000', 'last_fullsized': '2024-04-25T09:04:25.186012+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:31.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-25T09:04:31.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.187585+0000', 'last_change': '2024-04-25T09:04:25.187585+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.187585+0000', 'last_undegraded': '2024-04-25T09:04:25.187585+0000', 'last_fullsized': '2024-04-25T09:04:25.187585+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:31.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-25T09:04:31.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.194517+0000', 'last_change': '2024-04-25T09:04:25.194517+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.194517+0000', 'last_undegraded': '2024-04-25T09:04:25.194517+0000', 'last_fullsized': '2024-04-25T09:04:25.194517+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:31.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-25T09:04:31.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.180248+0000', 'last_change': '2024-04-25T09:04:25.180248+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.180248+0000', 'last_undegraded': '2024-04-25T09:04:25.180248+0000', 'last_fullsized': '2024-04-25T09:04:25.180248+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:31.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-25T09:04:31.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.193801+0000', 'last_change': '2024-04-25T09:04:25.193801+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.193801+0000', 'last_undegraded': '2024-04-25T09:04:25.193801+0000', 'last_fullsized': '2024-04-25T09:04:25.193801+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-25T09:04:31.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.190403+0000', 'last_change': '2024-04-25T09:04:25.190403+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.190403+0000', 'last_undegraded': '2024-04-25T09:04:25.190403+0000', 'last_fullsized': '2024-04-25T09:04:25.190403+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.192553+0000', 'last_change': '2024-04-25T09:04:25.192553+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.192553+0000', 'last_undegraded': '2024-04-25T09:04:25.192553+0000', 'last_fullsized': '2024-04-25T09:04:25.192553+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.179859+0000', 'last_change': '2024-04-25T09:04:25.179859+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.179859+0000', 'last_undegraded': '2024-04-25T09:04:25.179859+0000', 'last_fullsized': '2024-04-25T09:04:25.179859+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.178338+0000', 'last_change': '2024-04-25T09:04:25.178338+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.178338+0000', 'last_undegraded': '2024-04-25T09:04:25.178338+0000', 'last_fullsized': '2024-04-25T09:04:25.178338+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.188145+0000', 'last_change': '2024-04-25T09:04:25.188145+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.188145+0000', 'last_undegraded': '2024-04-25T09:04:25.188145+0000', 'last_fullsized': '2024-04-25T09:04:25.188145+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.190431+0000', 'last_change': '2024-04-25T09:04:25.190431+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.190431+0000', 'last_undegraded': '2024-04-25T09:04:25.190431+0000', 'last_fullsized': '2024-04-25T09:04:25.190431+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-25T09:04:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.179645+0000', 'last_change': '2024-04-25T09:04:25.179645+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.179645+0000', 'last_undegraded': '2024-04-25T09:04:25.179645+0000', 'last_fullsized': '2024-04-25T09:04:25.179645+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.179586+0000', 'last_change': '2024-04-25T09:04:25.179586+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.179586+0000', 'last_undegraded': '2024-04-25T09:04:25.179586+0000', 'last_fullsized': '2024-04-25T09:04:25.179586+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.185200+0000', 'last_change': '2024-04-25T09:04:25.185200+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.185200+0000', 'last_undegraded': '2024-04-25T09:04:25.185200+0000', 'last_fullsized': '2024-04-25T09:04:25.185200+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.187138+0000', 'last_change': '2024-04-25T09:04:25.187138+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.187138+0000', 'last_undegraded': '2024-04-25T09:04:25.187138+0000', 'last_fullsized': '2024-04-25T09:04:25.187138+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.191755+0000', 'last_change': '2024-04-25T09:04:25.191755+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.191755+0000', 'last_undegraded': '2024-04-25T09:04:25.191755+0000', 'last_fullsized': '2024-04-25T09:04:25.191755+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.187891+0000', 'last_change': '2024-04-25T09:04:25.187891+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.187891+0000', 'last_undegraded': '2024-04-25T09:04:25.187891+0000', 'last_fullsized': '2024-04-25T09:04:25.187891+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-25T09:04:31.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.189966+0000', 'last_change': '2024-04-25T09:04:25.189966+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.189966+0000', 'last_undegraded': '2024-04-25T09:04:25.189966+0000', 'last_fullsized': '2024-04-25T09:04:25.189966+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.177047+0000', 'last_change': '2024-04-25T09:04:25.177047+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.177047+0000', 'last_undegraded': '2024-04-25T09:04:25.177047+0000', 'last_fullsized': '2024-04-25T09:04:25.177047+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.182891+0000', 'last_change': '2024-04-25T09:04:25.182891+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.182891+0000', 'last_undegraded': '2024-04-25T09:04:25.182891+0000', 'last_fullsized': '2024-04-25T09:04:25.182891+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.185181+0000', 'last_change': '2024-04-25T09:04:25.185181+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.185181+0000', 'last_undegraded': '2024-04-25T09:04:25.185181+0000', 'last_fullsized': '2024-04-25T09:04:25.185181+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.183410+0000', 'last_change': '2024-04-25T09:04:25.183410+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.183410+0000', 'last_undegraded': '2024-04-25T09:04:25.183410+0000', 'last_fullsized': '2024-04-25T09:04:25.183410+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.176143+0000', 'last_change': '2024-04-25T09:04:25.176143+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.176143+0000', 'last_undegraded': '2024-04-25T09:04:25.176143+0000', 'last_fullsized': '2024-04-25T09:04:25.176143+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.191346+0000', 'last_change': '2024-04-25T09:04:25.191346+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.191346+0000', 'last_undegraded': '2024-04-25T09:04:25.191346+0000', 'last_fullsized': '2024-04-25T09:04:25.191346+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.177649+0000', 'last_change': '2024-04-25T09:04:25.177649+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.177649+0000', 'last_undegraded': '2024-04-25T09:04:25.177649+0000', 'last_fullsized': '2024-04-25T09:04:25.177649+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.179742+0000', 'last_change': '2024-04-25T09:04:25.179742+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.179742+0000', 'last_undegraded': '2024-04-25T09:04:25.179742+0000', 'last_fullsized': '2024-04-25T09:04:25.179742+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.193205+0000', 'last_change': '2024-04-25T09:04:25.193205+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.193205+0000', 'last_undegraded': '2024-04-25T09:04:25.193205+0000', 'last_fullsized': '2024-04-25T09:04:25.193205+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.193611+0000', 'last_change': '2024-04-25T09:04:25.193611+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.193611+0000', 'last_undegraded': '2024-04-25T09:04:25.193611+0000', 'last_fullsized': '2024-04-25T09:04:25.193611+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-25T09:04:31.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.180283+0000', 'last_change': '2024-04-25T09:04:25.180283+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.180283+0000', 'last_undegraded': '2024-04-25T09:04:25.180283+0000', 'last_fullsized': '2024-04-25T09:04:25.180283+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.184657+0000', 'last_change': '2024-04-25T09:04:25.184657+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.184657+0000', 'last_undegraded': '2024-04-25T09:04:25.184657+0000', 'last_fullsized': '2024-04-25T09:04:25.184657+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.189103+0000', 'last_change': '2024-04-25T09:04:25.189103+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.189103+0000', 'last_undegraded': '2024-04-25T09:04:25.189103+0000', 'last_fullsized': '2024-04-25T09:04:25.189103+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.186060+0000', 'last_change': '2024-04-25T09:04:25.186060+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.186060+0000', 'last_undegraded': '2024-04-25T09:04:25.186060+0000', 'last_fullsized': '2024-04-25T09:04:25.186060+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.184247+0000', 'last_change': '2024-04-25T09:04:25.184247+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.184247+0000', 'last_undegraded': '2024-04-25T09:04:25.184247+0000', 'last_fullsized': '2024-04-25T09:04:25.184247+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.188188+0000', 'last_change': '2024-04-25T09:04:25.188188+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.188188+0000', 'last_undegraded': '2024-04-25T09:04:25.188188+0000', 'last_fullsized': '2024-04-25T09:04:25.188188+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-25T09:04:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.186433+0000', 'last_change': '2024-04-25T09:04:25.186433+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.186433+0000', 'last_undegraded': '2024-04-25T09:04:25.186433+0000', 'last_fullsized': '2024-04-25T09:04:25.186433+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.191367+0000', 'last_change': '2024-04-25T09:04:25.191367+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.191367+0000', 'last_undegraded': '2024-04-25T09:04:25.191367+0000', 'last_fullsized': '2024-04-25T09:04:25.191367+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.195435+0000', 'last_change': '2024-04-25T09:04:25.195435+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.195435+0000', 'last_undegraded': '2024-04-25T09:04:25.195435+0000', 'last_fullsized': '2024-04-25T09:04:25.195435+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.189734+0000', 'last_change': '2024-04-25T09:04:25.189734+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.189734+0000', 'last_undegraded': '2024-04-25T09:04:25.189734+0000', 'last_fullsized': '2024-04-25T09:04:25.189734+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.181957+0000', 'last_change': '2024-04-25T09:04:25.181957+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.181957+0000', 'last_undegraded': '2024-04-25T09:04:25.181957+0000', 'last_fullsized': '2024-04-25T09:04:25.181957+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.183729+0000', 'last_change': '2024-04-25T09:04:25.183729+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.183729+0000', 'last_undegraded': '2024-04-25T09:04:25.183729+0000', 'last_fullsized': '2024-04-25T09:04:25.183729+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-25T09:04:31.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.192016+0000', 'last_change': '2024-04-25T09:04:25.192016+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.192016+0000', 'last_undegraded': '2024-04-25T09:04:25.192016+0000', 'last_fullsized': '2024-04-25T09:04:25.192016+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.185065+0000', 'last_change': '2024-04-25T09:04:25.185065+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.185065+0000', 'last_undegraded': '2024-04-25T09:04:25.185065+0000', 'last_fullsized': '2024-04-25T09:04:25.185065+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.178764+0000', 'last_change': '2024-04-25T09:04:25.178764+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.178764+0000', 'last_undegraded': '2024-04-25T09:04:25.178764+0000', 'last_fullsized': '2024-04-25T09:04:25.178764+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.178917+0000', 'last_change': '2024-04-25T09:04:25.178917+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.178917+0000', 'last_undegraded': '2024-04-25T09:04:25.178917+0000', 'last_fullsized': '2024-04-25T09:04:25.178917+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.175676+0000', 'last_change': '2024-04-25T09:04:25.175676+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.175676+0000', 'last_undegraded': '2024-04-25T09:04:25.175676+0000', 'last_fullsized': '2024-04-25T09:04:25.175676+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.187851+0000', 'last_change': '2024-04-25T09:04:25.187851+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.187851+0000', 'last_undegraded': '2024-04-25T09:04:25.187851+0000', 'last_fullsized': '2024-04-25T09:04:25.187851+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:31.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-25T09:04:31.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:25.179814+0000', 'last_change': '2024-04-25T09:04:25.179814+0000', 'last_active': '2024-04-25T09:04:25.160001+0000', 'last_peered': '2024-04-25T09:04:25.160001+0000', 'last_clean': '2024-04-25T09:04:25.160001+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:25.179814+0000', 'last_undegraded': '2024-04-25T09:04:25.179814+0000', 'last_fullsized': '2024-04-25T09:04:25.179814+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:25.160001+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:31.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:04:31.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.932+0000 7f2aa6e00640 1 Processor -- start 2024-04-25T09:04:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.933+0000 7f2aa6e00640 1 -- start start 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.934+0000 7f2aa6e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2aa015db60 0x7f2aa00d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.934+0000 7f2aa6e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa00d0a00 0x7f2aa00b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.934+0000 7f2aa6e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa00d2690 con 0x7f2aa015df40 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.934+0000 7f2aa6e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa00d2970 con 0x7f2aa015db60 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.934+0000 7f2aa6e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00d2c50 con 0x7f2aa00d0a00 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.935+0000 7f2aa5a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2aa015db60 0x7f2aa00d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.935+0000 7f2aa5a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2aa015db60 0x7f2aa00d1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36292/0 (socket says 172.21.15.62:36292) 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.935+0000 7f2aa5a00640 1 -- 172.21.15.62:0/214423785 learned_addr learned my addr 172.21.15.62:0/214423785 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.935+0000 7f2aa6400640 1 --2- 172.21.15.62:0/214423785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa00d0a00 0x7f2aa00b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.935+0000 7f2aa5a00640 1 -- 172.21.15.62:0/214423785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa00d0a00 msgr2=0x7f2aa00b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.935+0000 7f2aa5a00640 1 --2- 172.21.15.62:0/214423785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa00d0a00 0x7f2aa00b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.935+0000 7f2aa5a00640 1 -- 172.21.15.62:0/214423785 >> v1:172.21.15.121:6789/0 conn(0x7f2aa015df40 legacy=0x7f2aa00d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.935+0000 7f2aa5a00640 1 -- 172.21.15.62:0/214423785 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa00d2f30 con 0x7f2aa015db60 2024-04-25T09:04:31.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.937+0000 7f2aa5a00640 1 --2- 172.21.15.62:0/214423785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00d1e40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009870 tx=0x7f2a9402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bc3afa6e9aca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.937+0000 7f2a9fe00640 1 -- 172.21.15.62:0/214423785 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94037070 con 0x7f2aa015db60 2024-04-25T09:04:31.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.937+0000 7f2a9fe00640 1 -- 172.21.15.62:0/214423785 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94032440 con 0x7f2aa015db60 2024-04-25T09:04:31.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.937+0000 7f2a9fe00640 1 -- 172.21.15.62:0/214423785 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94032810 con 0x7f2aa015db60 2024-04-25T09:04:31.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.938+0000 7f2aa6e00640 1 -- 172.21.15.62:0/214423785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 msgr2=0x7f2aa00d1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.938+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/214423785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00d1e40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009870 tx=0x7f2a9402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.938+0000 7f2aa6e00640 1 -- 172.21.15.62:0/214423785 shutdown_connections 2024-04-25T09:04:31.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.938+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/214423785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa00d0a00 0x7f2aa00b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.938+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/214423785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00d1e40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.938+0000 7f2aa6e00640 1 -- 172.21.15.62:0/214423785 >> 172.21.15.62:0/214423785 conn(0x7f2aa006b210 msgr2=0x7f2aa00668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.938+0000 7f2aa6e00640 1 -- 172.21.15.62:0/214423785 shutdown_connections 2024-04-25T09:04:31.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.938+0000 7f2aa6e00640 1 -- 172.21.15.62:0/214423785 wait complete. 2024-04-25T09:04:31.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.940+0000 7f2aa6e00640 1 Processor -- start 2024-04-25T09:04:31.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.940+0000 7f2aa6e00640 1 -- start start 2024-04-25T09:04:31.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa6e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa00d0a00 0x7f2aa014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa6e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa6e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa015df40 0x7f2aa01f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa6e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00d3890 con 0x7f2aa015db60 2024-04-25T09:04:31.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa6e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00d3710 con 0x7f2aa00d0a00 2024-04-25T09:04:31.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa6e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00b9750 con 0x7f2aa015df40 2024-04-25T09:04:31.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00aa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36298/0 (socket says 172.21.15.62:36298) 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa5000640 1 -- 172.21.15.62:0/2997024350 learned_addr learned my addr 172.21.15.62:0/2997024350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa5a00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa00d0a00 0x7f2aa014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa5000640 1 -- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa015df40 msgr2=0x7f2aa01f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa5000640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa015df40 0x7f2aa01f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.941+0000 7f2aa5000640 1 -- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa00d0a00 msgr2=0x7f2aa014f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2aa5000640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa00d0a00 0x7f2aa014f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2aa5000640 1 -- 172.21.15.62:0/2997024350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa01e5300 con 0x7f2aa015db60 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2aa5a00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa00d0a00 0x7f2aa014f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2aa5000640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00aa440 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c3c0 tx=0x7f2a8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2a9d600640 1 -- 172.21.15.62:0/2997024350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c017070 con 0x7f2aa015db60 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2a9d600640 1 -- 172.21.15.62:0/2997024350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c0052d0 con 0x7f2aa015db60 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2a9d600640 1 -- 172.21.15.62:0/2997024350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c003830 con 0x7f2aa015db60 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa01e55e0 con 0x7f2aa015db60 2024-04-25T09:04:31.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.942+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa01e5ba0 con 0x7f2aa015db60 2024-04-25T09:04:31.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.943+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa0072910 con 0x7f2aa015db60 2024-04-25T09:04:31.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.944+0000 7f2a9d600640 1 -- 172.21.15.62:0/2997024350 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a8c01a070 con 0x7f2aa015db60 2024-04-25T09:04:31.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.945+0000 7f2a9d600640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2a8006b4d0 0x7f2a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.946+0000 7f2a9d600640 1 -- 172.21.15.62:0/2997024350 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a8c089500 con 0x7f2aa015db60 2024-04-25T09:04:31.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.946+0000 7f2aa5a00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2a8006b4d0 0x7f2a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.947+0000 7f2aa5a00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2a8006b4d0 0x7f2a8006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2a94034000 tx=0x7f2a9402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:31.949+0000 7f2a9d600640 1 -- 172.21.15.62:0/2997024350 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a8c0582e0 con 0x7f2aa015db60 2024-04-25T09:04:32.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.208+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2aa00750e0 con 0x7f2aa015db60 2024-04-25T09:04:32.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.209+0000 7f2a9d600640 1 -- 172.21.15.62:0/2997024350 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1194 (secure 0 0 0) 0x7f2a8c057be0 con 0x7f2aa015db60 2024-04-25T09:04:32.197 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:32.197 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":99,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225341440,"bytes_avail":772868771840,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-25T09:04:15.905336+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:04:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2a8006b4d0 msgr2=0x7f2a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2a8006b4d0 0x7f2a8006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2a94034000 tx=0x7f2a9402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 msgr2=0x7f2aa00aa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00aa440 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c3c0 tx=0x7f2a8c00c880 comp rx=0 tx=0).stop 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 shutdown_connections 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2a8006b4d0 0x7f2a8006d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa015db60 0x7f2aa00aa440 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2aa015df40 0x7f2aa01f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 --2- 172.21.15.62:0/2997024350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aa00d0a00 0x7f2aa014f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.211+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 >> 172.21.15.62:0/2997024350 conn(0x7f2aa006b210 msgr2=0x7f2aa00b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.212+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 shutdown_connections 2024-04-25T09:04:32.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.212+0000 7f2aa6e00640 1 -- 172.21.15.62:0/2997024350 wait complete. 2024-04-25T09:04:32.218 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:04:32.218 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.437+0000 7fcd0ac00640 1 Processor -- start 2024-04-25T09:04:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.437+0000 7fcd0ac00640 1 -- start start 2024-04-25T09:04:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.439+0000 7fcd0ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd04144c10 0x7fcd0415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.439+0000 7fcd0ac00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0415faa0 con 0x7fcd040cee40 2024-04-25T09:04:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.439+0000 7fcd0ac00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0415fd80 con 0x7fcd0406d900 2024-04-25T09:04:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.439+0000 7fcd0ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04144ff0 con 0x7fcd04144c10 2024-04-25T09:04:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.439+0000 7fcd08e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fcd040cee40 0x7fcd0406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41478/0 (socket says 172.21.15.62:41478) 2024-04-25T09:04:32.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.439+0000 7fcd08e00640 1 -- 172.21.15.62:0/1968320160 learned_addr learned my addr 172.21.15.62:0/1968320160 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:32.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507858201 0 0) 0x7fcd0415faa0 con 0x7fcd040cee40 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccec003520 con 0x7fcd040cee40 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd09800640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd04144c10 0x7fcd0415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 280538413 0 0) 0x7fccec003520 con 0x7fcd040cee40 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd04144c10 msgr2=0x7fcd0415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd04144c10 0x7fcd0415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 >> v1:172.21.15.121:6789/0 conn(0x7fcd0406d900 legacy=0x7fcd040b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd04160060 con 0x7fcd040cee40 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fccf0001670 con 0x7fcd040cee40 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.440+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 >> v1:172.21.15.62:6789/0 conn(0x7fcd040cee40 legacy=0x7fcd0406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd03e00640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccec0039e0 0x7fccec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd03e00640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccec004800 0x7fccec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd03e00640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd04144c10 0x7fccec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04144ff0 con 0x7fccec0039e0 2024-04-25T09:04:32.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0054f0 con 0x7fccec004800 2024-04-25T09:04:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec00bf00 con 0x7fcd04144c10 2024-04-25T09:04:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd09800640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccec0039e0 0x7fccec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd09800640 1 -- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd04144c10 msgr2=0x7fccec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd09800640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcd04144c10 0x7fccec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd09800640 1 -- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccec004800 msgr2=0x7fccec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd08e00640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccec004800 0x7fccec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd09800640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccec004800 0x7fccec004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.441+0000 7fcd09800640 1 -- 172.21.15.62:0/1968320160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec003520 con 0x7fccec0039e0 2024-04-25T09:04:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.442+0000 7fcd09800640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccec0039e0 0x7fccec003e00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fccf800ba10 tx=0x7fccec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e331a97c96196eab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.443+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8030070 con 0x7fccec0039e0 2024-04-25T09:04:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.443+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf802ae20 con 0x7fccec0039e0 2024-04-25T09:04:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.443+0000 7fcd03e00640 1 -- 172.21.15.62:0/1968320160 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf800b5e0 con 0x7fccec0039e0 2024-04-25T09:04:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.443+0000 7fcd0ac00640 1 -- 172.21.15.62:0/1968320160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccec0039e0 msgr2=0x7fccec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.443+0000 7fcd0ac00640 1 --2- 172.21.15.62:0/1968320160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fccec0039e0 0x7fccec003e00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fccf800ba10 tx=0x7fccec005830 comp rx=0 tx=0).stop 2024-04-25T09:04:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.443+0000 7fcd0a200640 1 -- 172.21.15.62:0/1968320160 reap_dead start 2024-04-25T09:04:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.444+0000 7fcd0ac00640 1 -- 172.21.15.62:0/1968320160 shutdown_connections 2024-04-25T09:04:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.444+0000 7fcd0ac00640 1 -- 172.21.15.62:0/1968320160 >> 172.21.15.62:0/1968320160 conn(0x7fcd040b14d0 msgr2=0x7fcd040b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.444+0000 7fcd0ac00640 1 -- 172.21.15.62:0/1968320160 shutdown_connections 2024-04-25T09:04:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.444+0000 7fcd0ac00640 1 -- 172.21.15.62:0/1968320160 wait complete. 2024-04-25T09:04:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.445+0000 7fcd0ac00640 1 Processor -- start 2024-04-25T09:04:32.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.446+0000 7fcd0ac00640 1 -- start start 2024-04-25T09:04:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd0ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccec0039e0 0x7fcd0415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd0ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd041f4a20 0x7fcd041e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd0ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd040b4760 0x7fcd040b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd0ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040b41c0 con 0x7fcd041f4a20 2024-04-25T09:04:32.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd0ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04160340 con 0x7fcd040b4760 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd0ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0415f450 con 0x7fccec0039e0 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd041f4a20 0x7fcd041e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd041f4a20 0x7fcd041e5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36316/0 (socket says 172.21.15.62:36316) 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd08e00640 1 -- 172.21.15.62:0/2413183504 learned_addr learned my addr 172.21.15.62:0/2413183504 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd0a200640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd040b4760 0x7fcd040b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd09800640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccec0039e0 0x7fcd0415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd08e00640 1 -- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccec0039e0 msgr2=0x7fcd0415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd08e00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccec0039e0 0x7fcd0415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd08e00640 1 -- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd040b4760 msgr2=0x7fcd040b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd08e00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd040b4760 0x7fcd040b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd08e00640 1 -- 172.21.15.62:0/2413183504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd041e5b00 con 0x7fcd041f4a20 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.447+0000 7fcd09800640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccec0039e0 0x7fcd0415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:32.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.448+0000 7fcd08e00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd041f4a20 0x7fcd041e5200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fccf000d7a0 tx=0x7fccf000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.448+0000 7fcd01600640 1 -- 172.21.15.62:0/2413183504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0019070 con 0x7fcd041f4a20 2024-04-25T09:04:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.448+0000 7fcd01600640 1 -- 172.21.15.62:0/2413183504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf0007560 con 0x7fcd041f4a20 2024-04-25T09:04:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.448+0000 7fcd01600640 1 -- 172.21.15.62:0/2413183504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0004a00 con 0x7fcd041f4a20 2024-04-25T09:04:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.448+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd041e5de0 con 0x7fcd041f4a20 2024-04-25T09:04:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.448+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd041e63a0 con 0x7fcd041f4a20 2024-04-25T09:04:32.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.449+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0415ba50 con 0x7fcd041f4a20 2024-04-25T09:04:32.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.450+0000 7fcd01600640 1 -- 172.21.15.62:0/2413183504 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccf001c070 con 0x7fcd041f4a20 2024-04-25T09:04:32.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.451+0000 7fcd01600640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcce406b4d0 0x7fcce406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.451+0000 7fcd01600640 1 -- 172.21.15.62:0/2413183504 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccf0089c80 con 0x7fcd041f4a20 2024-04-25T09:04:32.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.452+0000 7fcd09800640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcce406b4d0 0x7fcce406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.453+0000 7fcd09800640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcce406b4d0 0x7fcce406d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b9e0 tx=0x7fccf8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.455+0000 7fcd01600640 1 -- 172.21.15.62:0/2413183504 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccf004f8c0 con 0x7fcd041f4a20 2024-04-25T09:04:32.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.650+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd041e6780 con 0x7fcce406b4d0 2024-04-25T09:04:32.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.669+0000 7fcd01600640 1 -- 172.21.15.62:0/2413183504 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251954 (secure 0 0 0) 0x7fccf8034500 con 0x7fcce406b4d0 2024-04-25T09:04:32.661 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:32.662 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:04:32.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcce406b4d0 msgr2=0x7fcce406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcce406b4d0 0x7fcce406d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b9e0 tx=0x7fccf8030000 comp rx=0 tx=0).stop 2024-04-25T09:04:32.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd041f4a20 msgr2=0x7fcd041e5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd041f4a20 0x7fcd041e5200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fccf000d7a0 tx=0x7fccf000dc60 comp rx=0 tx=0).stop 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 shutdown_connections 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcce406b4d0 0x7fcce406d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcd041f4a20 0x7fcd041e5200 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd040b4760 0x7fcd040b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 --2- 172.21.15.62:0/2413183504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fccec0039e0 0x7fcd0415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 >> 172.21.15.62:0/2413183504 conn(0x7fcd040b14d0 msgr2=0x7fcd040b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.674+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 shutdown_connections 2024-04-25T09:04:32.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:32.675+0000 7fcd0ac00640 1 -- 172.21.15.62:0/2413183504 wait complete. 2024-04-25T09:04:32.682 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-25T09:04:31.909939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220312,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754408,"statfs":{"total":773094113280,"available":772868513792,"internally_reserved":0,"allocated":4235264,"data_stored":2354532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.740485"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368765+0000","last_change":"2024-04-25T09:04:26.241063+0000","last_active":"2024-04-25T09:04:28.368765+0000","last_peered":"2024-04-25T09:04:28.368765+0000","last_clean":"2024-04-25T09:04:28.368765+0000","last_became_active":"2024-04-25T09:04:26.238514+0000","last_became_peered":"2024-04-25T09:04:26.238514+0000","last_unstale":"2024-04-25T09:04:28.368765+0000","last_undegraded":"2024-04-25T09:04:28.368765+0000","last_fullsized":"2024-04-25T09:04:28.368765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:34.911589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076208+0000","last_change":"2024-04-25T09:04:26.237196+0000","last_active":"2024-04-25T09:04:28.076208+0000","last_peered":"2024-04-25T09:04:28.076208+0000","last_clean":"2024-04-25T09:04:28.076208+0000","last_became_active":"2024-04-25T09:04:26.236189+0000","last_became_peered":"2024-04-25T09:04:26.236189+0000","last_unstale":"2024-04-25T09:04:28.076208+0000","last_undegraded":"2024-04-25T09:04:28.076208+0000","last_fullsized":"2024-04-25T09:04:28.076208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:21.190794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369992+0000","last_change":"2024-04-25T09:04:26.223452+0000","last_active":"2024-04-25T09:04:28.369992+0000","last_peered":"2024-04-25T09:04:28.369992+0000","last_clean":"2024-04-25T09:04:28.369992+0000","last_became_active":"2024-04-25T09:04:26.223179+0000","last_became_peered":"2024-04-25T09:04:26.223179+0000","last_unstale":"2024-04-25T09:04:28.369992+0000","last_undegraded":"2024-04-25T09:04:28.369992+0000","last_fullsized":"2024-04-25T09:04:28.369992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:19.355095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232834+0000","last_change":"2024-04-25T09:04:26.226411+0000","last_active":"2024-04-25T09:04:27.232834+0000","last_peered":"2024-04-25T09:04:27.232834+0000","last_clean":"2024-04-25T09:04:27.232834+0000","last_became_active":"2024-04-25T09:04:26.226131+0000","last_became_peered":"2024-04-25T09:04:26.226131+0000","last_unstale":"2024-04-25T09:04:27.232834+0000","last_undegraded":"2024-04-25T09:04:27.232834+0000","last_fullsized":"2024-04-25T09:04:27.232834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.889201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.075837+0000","last_change":"2024-04-25T09:04:26.236909+0000","last_active":"2024-04-25T09:04:28.075837+0000","last_peered":"2024-04-25T09:04:28.075837+0000","last_clean":"2024-04-25T09:04:28.075837+0000","last_became_active":"2024-04-25T09:04:26.236362+0000","last_became_peered":"2024-04-25T09:04:26.236362+0000","last_unstale":"2024-04-25T09:04:28.075837+0000","last_undegraded":"2024-04-25T09:04:28.075837+0000","last_fullsized":"2024-04-25T09:04:28.075837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:13.827695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.211992+0000","last_change":"2024-04-25T09:04:26.237439+0000","last_active":"2024-04-25T09:04:28.211992+0000","last_peered":"2024-04-25T09:04:28.211992+0000","last_clean":"2024-04-25T09:04:28.211992+0000","last_became_active":"2024-04-25T09:04:26.236815+0000","last_became_peered":"2024-04-25T09:04:26.236815+0000","last_unstale":"2024-04-25T09:04:28.211992+0000","last_undegraded":"2024-04-25T09:04:28.211992+0000","last_fullsized":"2024-04-25T09:04:28.211992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:39.649661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367875+0000","last_change":"2024-04-25T09:04:26.231513+0000","last_active":"2024-04-25T09:04:28.367875+0000","last_peered":"2024-04-25T09:04:28.367875+0000","last_clean":"2024-04-25T09:04:28.367875+0000","last_became_active":"2024-04-25T09:04:26.231186+0000","last_became_peered":"2024-04-25T09:04:26.231186+0000","last_unstale":"2024-04-25T09:04:28.367875+0000","last_undegraded":"2024-04-25T09:04:28.367875+0000","last_fullsized":"2024-04-25T09:04:28.367875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:18.665177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367844+0000","last_change":"2024-04-25T09:04:26.237203+0000","last_active":"2024-04-25T09:04:28.367844+0000","last_peered":"2024-04-25T09:04:28.367844+0000","last_clean":"2024-04-25T09:04:28.367844+0000","last_became_active":"2024-04-25T09:04:26.236841+0000","last_became_peered":"2024-04-25T09:04:26.236841+0000","last_unstale":"2024-04-25T09:04:28.367844+0000","last_undegraded":"2024-04-25T09:04:28.367844+0000","last_fullsized":"2024-04-25T09:04:28.367844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.348176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369948+0000","last_change":"2024-04-25T09:04:26.232870+0000","last_active":"2024-04-25T09:04:28.369948+0000","last_peered":"2024-04-25T09:04:28.369948+0000","last_clean":"2024-04-25T09:04:28.369948+0000","last_became_active":"2024-04-25T09:04:26.232344+0000","last_became_peered":"2024-04-25T09:04:26.232344+0000","last_unstale":"2024-04-25T09:04:28.369948+0000","last_undegraded":"2024-04-25T09:04:28.369948+0000","last_fullsized":"2024-04-25T09:04:28.369948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:14.190501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233295+0000","last_change":"2024-04-25T09:04:26.227337+0000","last_active":"2024-04-25T09:04:27.233295+0000","last_peered":"2024-04-25T09:04:27.233295+0000","last_clean":"2024-04-25T09:04:27.233295+0000","last_became_active":"2024-04-25T09:04:26.226800+0000","last_became_peered":"2024-04-25T09:04:26.226800+0000","last_unstale":"2024-04-25T09:04:27.233295+0000","last_undegraded":"2024-04-25T09:04:27.233295+0000","last_fullsized":"2024-04-25T09:04:27.233295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:34.911589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367591+0000","last_change":"2024-04-25T09:04:26.224467+0000","last_active":"2024-04-25T09:04:28.367591+0000","last_peered":"2024-04-25T09:04:28.367591+0000","last_clean":"2024-04-25T09:04:28.367591+0000","last_became_active":"2024-04-25T09:04:26.224181+0000","last_became_peered":"2024-04-25T09:04:26.224181+0000","last_unstale":"2024-04-25T09:04:28.367591+0000","last_undegraded":"2024-04-25T09:04:28.367591+0000","last_fullsized":"2024-04-25T09:04:28.367591+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:33.128433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369536+0000","last_change":"2024-04-25T09:04:26.236743+0000","last_active":"2024-04-25T09:04:28.369536+0000","last_peered":"2024-04-25T09:04:28.369536+0000","last_clean":"2024-04-25T09:04:28.369536+0000","last_became_active":"2024-04-25T09:04:26.236311+0000","last_became_peered":"2024-04-25T09:04:26.236311+0000","last_unstale":"2024-04-25T09:04:28.369536+0000","last_undegraded":"2024-04-25T09:04:28.369536+0000","last_fullsized":"2024-04-25T09:04:28.369536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:44.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369519+0000","last_change":"2024-04-25T09:04:26.235258+0000","last_active":"2024-04-25T09:04:28.369519+0000","last_peered":"2024-04-25T09:04:28.369519+0000","last_clean":"2024-04-25T09:04:28.369519+0000","last_became_active":"2024-04-25T09:04:26.234312+0000","last_became_peered":"2024-04-25T09:04:26.234312+0000","last_unstale":"2024-04-25T09:04:28.369519+0000","last_undegraded":"2024-04-25T09:04:28.369519+0000","last_fullsized":"2024-04-25T09:04:28.369519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:26.452996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232145+0000","last_change":"2024-04-25T09:04:26.227707+0000","last_active":"2024-04-25T09:04:27.232145+0000","last_peered":"2024-04-25T09:04:27.232145+0000","last_clean":"2024-04-25T09:04:27.232145+0000","last_became_active":"2024-04-25T09:04:26.227283+0000","last_became_peered":"2024-04-25T09:04:26.227283+0000","last_unstale":"2024-04-25T09:04:27.232145+0000","last_undegraded":"2024-04-25T09:04:27.232145+0000","last_fullsized":"2024-04-25T09:04:27.232145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:21.190794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.226860+0000","last_change":"2024-04-25T09:04:26.225446+0000","last_active":"2024-04-25T09:04:27.226860+0000","last_peered":"2024-04-25T09:04:27.226860+0000","last_clean":"2024-04-25T09:04:27.226860+0000","last_became_active":"2024-04-25T09:04:26.225161+0000","last_became_peered":"2024-04-25T09:04:26.225161+0000","last_unstale":"2024-04-25T09:04:27.226860+0000","last_undegraded":"2024-04-25T09:04:27.226860+0000","last_fullsized":"2024-04-25T09:04:27.226860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.576671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.226604+0000","last_change":"2024-04-25T09:04:26.245986+0000","last_active":"2024-04-25T09:04:27.226604+0000","last_peered":"2024-04-25T09:04:27.226604+0000","last_clean":"2024-04-25T09:04:27.226604+0000","last_became_active":"2024-04-25T09:04:26.245517+0000","last_became_peered":"2024-04-25T09:04:26.245517+0000","last_unstale":"2024-04-25T09:04:27.226604+0000","last_undegraded":"2024-04-25T09:04:27.226604+0000","last_fullsized":"2024-04-25T09:04:27.226604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:51.170042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370762+0000","last_change":"2024-04-25T09:04:26.228748+0000","last_active":"2024-04-25T09:04:28.370762+0000","last_peered":"2024-04-25T09:04:28.370762+0000","last_clean":"2024-04-25T09:04:28.370762+0000","last_became_active":"2024-04-25T09:04:26.228225+0000","last_became_peered":"2024-04-25T09:04:26.228225+0000","last_unstale":"2024-04-25T09:04:28.370762+0000","last_undegraded":"2024-04-25T09:04:28.370762+0000","last_fullsized":"2024-04-25T09:04:28.370762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.893561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212359+0000","last_change":"2024-04-25T09:04:26.242396+0000","last_active":"2024-04-25T09:04:28.212359+0000","last_peered":"2024-04-25T09:04:28.212359+0000","last_clean":"2024-04-25T09:04:28.212359+0000","last_became_active":"2024-04-25T09:04:26.242112+0000","last_became_peered":"2024-04-25T09:04:26.242112+0000","last_unstale":"2024-04-25T09:04:28.212359+0000","last_undegraded":"2024-04-25T09:04:28.212359+0000","last_fullsized":"2024-04-25T09:04:28.212359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:38.169954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212023+0000","last_change":"2024-04-25T09:04:26.242548+0000","last_active":"2024-04-25T09:04:28.212023+0000","last_peered":"2024-04-25T09:04:28.212023+0000","last_clean":"2024-04-25T09:04:28.212023+0000","last_became_active":"2024-04-25T09:04:26.242269+0000","last_became_peered":"2024-04-25T09:04:26.242269+0000","last_unstale":"2024-04-25T09:04:28.212023+0000","last_undegraded":"2024-04-25T09:04:28.212023+0000","last_fullsized":"2024-04-25T09:04:28.212023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:19.355095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369921+0000","last_change":"2024-04-25T09:04:26.239456+0000","last_active":"2024-04-25T09:04:28.369921+0000","last_peered":"2024-04-25T09:04:28.369921+0000","last_clean":"2024-04-25T09:04:28.369921+0000","last_became_active":"2024-04-25T09:04:26.238831+0000","last_became_peered":"2024-04-25T09:04:26.238831+0000","last_unstale":"2024-04-25T09:04:28.369921+0000","last_undegraded":"2024-04-25T09:04:28.369921+0000","last_fullsized":"2024-04-25T09:04:28.369921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.348176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227059+0000","last_change":"2024-04-25T09:04:26.236502+0000","last_active":"2024-04-25T09:04:27.227059+0000","last_peered":"2024-04-25T09:04:27.227059+0000","last_clean":"2024-04-25T09:04:27.227059+0000","last_became_active":"2024-04-25T09:04:26.236194+0000","last_became_peered":"2024-04-25T09:04:26.236194+0000","last_unstale":"2024-04-25T09:04:27.227059+0000","last_undegraded":"2024-04-25T09:04:27.227059+0000","last_fullsized":"2024-04-25T09:04:27.227059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.889201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367923+0000","last_change":"2024-04-25T09:04:26.238295+0000","last_active":"2024-04-25T09:04:28.367923+0000","last_peered":"2024-04-25T09:04:28.367923+0000","last_clean":"2024-04-25T09:04:28.367923+0000","last_became_active":"2024-04-25T09:04:26.234544+0000","last_became_peered":"2024-04-25T09:04:26.234544+0000","last_unstale":"2024-04-25T09:04:28.367923+0000","last_undegraded":"2024-04-25T09:04:28.367923+0000","last_fullsized":"2024-04-25T09:04:28.367923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:13.827695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369089+0000","last_change":"2024-04-25T09:04:26.231385+0000","last_active":"2024-04-25T09:04:28.369089+0000","last_peered":"2024-04-25T09:04:28.369089+0000","last_clean":"2024-04-25T09:04:28.369089+0000","last_became_active":"2024-04-25T09:04:26.231103+0000","last_became_peered":"2024-04-25T09:04:26.231103+0000","last_unstale":"2024-04-25T09:04:28.369089+0000","last_undegraded":"2024-04-25T09:04:28.369089+0000","last_fullsized":"2024-04-25T09:04:28.369089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:29.346112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368121+0000","last_change":"2024-04-25T09:04:26.230878+0000","last_active":"2024-04-25T09:04:28.368121+0000","last_peered":"2024-04-25T09:04:28.368121+0000","last_clean":"2024-04-25T09:04:28.368121+0000","last_became_active":"2024-04-25T09:04:26.230312+0000","last_became_peered":"2024-04-25T09:04:26.230312+0000","last_unstale":"2024-04-25T09:04:28.368121+0000","last_undegraded":"2024-04-25T09:04:28.368121+0000","last_fullsized":"2024-04-25T09:04:28.368121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:21.190794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233894+0000","last_change":"2024-04-25T09:04:26.218543+0000","last_active":"2024-04-25T09:04:27.233894+0000","last_peered":"2024-04-25T09:04:27.233894+0000","last_clean":"2024-04-25T09:04:27.233894+0000","last_became_active":"2024-04-25T09:04:26.218256+0000","last_became_peered":"2024-04-25T09:04:26.218256+0000","last_unstale":"2024-04-25T09:04:27.233894+0000","last_undegraded":"2024-04-25T09:04:27.233894+0000","last_fullsized":"2024-04-25T09:04:27.233894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:36.346921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233751+0000","last_change":"2024-04-25T09:04:26.221255+0000","last_active":"2024-04-25T09:04:27.233751+0000","last_peered":"2024-04-25T09:04:27.233751+0000","last_clean":"2024-04-25T09:04:27.233751+0000","last_became_active":"2024-04-25T09:04:26.220969+0000","last_became_peered":"2024-04-25T09:04:26.220969+0000","last_unstale":"2024-04-25T09:04:27.233751+0000","last_undegraded":"2024-04-25T09:04:27.233751+0000","last_fullsized":"2024-04-25T09:04:27.233751+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:03.430439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076322+0000","last_change":"2024-04-25T09:04:26.238580+0000","last_active":"2024-04-25T09:04:28.076322+0000","last_peered":"2024-04-25T09:04:28.076322+0000","last_clean":"2024-04-25T09:04:28.076322+0000","last_became_active":"2024-04-25T09:04:26.238291+0000","last_became_peered":"2024-04-25T09:04:26.238291+0000","last_unstale":"2024-04-25T09:04:28.076322+0000","last_undegraded":"2024-04-25T09:04:28.076322+0000","last_fullsized":"2024-04-25T09:04:28.076322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.889201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367910+0000","last_change":"2024-04-25T09:04:26.238450+0000","last_active":"2024-04-25T09:04:28.367910+0000","last_peered":"2024-04-25T09:04:28.367910+0000","last_clean":"2024-04-25T09:04:28.367910+0000","last_became_active":"2024-04-25T09:04:26.238161+0000","last_became_peered":"2024-04-25T09:04:26.238161+0000","last_unstale":"2024-04-25T09:04:28.367910+0000","last_undegraded":"2024-04-25T09:04:28.367910+0000","last_fullsized":"2024-04-25T09:04:28.367910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:51.170042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227292+0000","last_change":"2024-04-25T09:04:26.232881+0000","last_active":"2024-04-25T09:04:27.227292+0000","last_peered":"2024-04-25T09:04:27.227292+0000","last_clean":"2024-04-25T09:04:27.227292+0000","last_became_active":"2024-04-25T09:04:26.232522+0000","last_became_peered":"2024-04-25T09:04:26.232522+0000","last_unstale":"2024-04-25T09:04:27.227292+0000","last_undegraded":"2024-04-25T09:04:27.227292+0000","last_fullsized":"2024-04-25T09:04:27.227292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:30.946720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227132+0000","last_change":"2024-04-25T09:04:26.234593+0000","last_active":"2024-04-25T09:04:27.227132+0000","last_peered":"2024-04-25T09:04:27.227132+0000","last_clean":"2024-04-25T09:04:27.227132+0000","last_became_active":"2024-04-25T09:04:26.233853+0000","last_became_peered":"2024-04-25T09:04:26.233853+0000","last_unstale":"2024-04-25T09:04:27.227132+0000","last_undegraded":"2024-04-25T09:04:27.227132+0000","last_fullsized":"2024-04-25T09:04:27.227132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368043+0000","last_change":"2024-04-25T09:04:26.241932+0000","last_active":"2024-04-25T09:04:28.368043+0000","last_peered":"2024-04-25T09:04:28.368043+0000","last_clean":"2024-04-25T09:04:28.368043+0000","last_became_active":"2024-04-25T09:04:26.235859+0000","last_became_peered":"2024-04-25T09:04:26.235859+0000","last_unstale":"2024-04-25T09:04:28.368043+0000","last_undegraded":"2024-04-25T09:04:28.368043+0000","last_fullsized":"2024-04-25T09:04:28.368043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.576671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367357+0000","last_change":"2024-04-25T09:04:26.222060+0000","last_active":"2024-04-25T09:04:28.367357+0000","last_peered":"2024-04-25T09:04:28.367357+0000","last_clean":"2024-04-25T09:04:28.367357+0000","last_became_active":"2024-04-25T09:04:26.221758+0000","last_became_peered":"2024-04-25T09:04:26.221758+0000","last_unstale":"2024-04-25T09:04:28.367357+0000","last_undegraded":"2024-04-25T09:04:28.367357+0000","last_fullsized":"2024-04-25T09:04:28.367357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:24.128100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369942+0000","last_change":"2024-04-25T09:04:26.233156+0000","last_active":"2024-04-25T09:04:28.369942+0000","last_peered":"2024-04-25T09:04:28.369942+0000","last_clean":"2024-04-25T09:04:28.369942+0000","last_became_active":"2024-04-25T09:04:26.232789+0000","last_became_peered":"2024-04-25T09:04:26.232789+0000","last_unstale":"2024-04-25T09:04:28.369942+0000","last_undegraded":"2024-04-25T09:04:28.369942+0000","last_fullsized":"2024-04-25T09:04:28.369942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:18.665177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.371104+0000","last_change":"2024-04-25T09:04:25.210460+0000","last_active":"2024-04-25T09:04:28.371104+0000","last_peered":"2024-04-25T09:04:28.371104+0000","last_clean":"2024-04-25T09:04:28.371104+0000","last_became_active":"2024-04-25T09:04:25.210130+0000","last_became_peered":"2024-04-25T09:04:25.210130+0000","last_unstale":"2024-04-25T09:04:28.371104+0000","last_undegraded":"2024-04-25T09:04:28.371104+0000","last_fullsized":"2024-04-25T09:04:28.371104+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:20.169291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368947+0000","last_change":"2024-04-25T09:04:26.234788+0000","last_active":"2024-04-25T09:04:28.368947+0000","last_peered":"2024-04-25T09:04:28.368947+0000","last_clean":"2024-04-25T09:04:28.368947+0000","last_became_active":"2024-04-25T09:04:26.234467+0000","last_became_peered":"2024-04-25T09:04:26.234467+0000","last_unstale":"2024-04-25T09:04:28.368947+0000","last_undegraded":"2024-04-25T09:04:28.368947+0000","last_fullsized":"2024-04-25T09:04:28.368947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:21.181723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.234070+0000","last_change":"2024-04-25T09:04:25.198907+0000","last_active":"2024-04-25T09:04:27.234070+0000","last_peered":"2024-04-25T09:04:27.234070+0000","last_clean":"2024-04-25T09:04:27.234070+0000","last_became_active":"2024-04-25T09:04:25.198368+0000","last_became_peered":"2024-04-25T09:04:25.198368+0000","last_unstale":"2024-04-25T09:04:27.234070+0000","last_undegraded":"2024-04-25T09:04:27.234070+0000","last_fullsized":"2024-04-25T09:04:27.234070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:21.753154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368307+0000","last_change":"2024-04-25T09:04:26.230893+0000","last_active":"2024-04-25T09:04:28.368307+0000","last_peered":"2024-04-25T09:04:28.368307+0000","last_clean":"2024-04-25T09:04:28.368307+0000","last_became_active":"2024-04-25T09:04:26.230660+0000","last_became_peered":"2024-04-25T09:04:26.230660+0000","last_unstale":"2024-04-25T09:04:28.368307+0000","last_undegraded":"2024-04-25T09:04:28.368307+0000","last_fullsized":"2024-04-25T09:04:28.368307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212528+0000","last_change":"2024-04-25T09:04:25.211094+0000","last_active":"2024-04-25T09:04:28.212528+0000","last_peered":"2024-04-25T09:04:28.212528+0000","last_clean":"2024-04-25T09:04:28.212528+0000","last_became_active":"2024-04-25T09:04:25.210803+0000","last_became_peered":"2024-04-25T09:04:25.210803+0000","last_unstale":"2024-04-25T09:04:28.212528+0000","last_undegraded":"2024-04-25T09:04:28.212528+0000","last_fullsized":"2024-04-25T09:04:28.212528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:46.169978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233974+0000","last_change":"2024-04-25T09:04:26.241136+0000","last_active":"2024-04-25T09:04:27.233974+0000","last_peered":"2024-04-25T09:04:27.233974+0000","last_clean":"2024-04-25T09:04:27.233974+0000","last_became_active":"2024-04-25T09:04:26.239497+0000","last_became_peered":"2024-04-25T09:04:26.239497+0000","last_unstale":"2024-04-25T09:04:27.233974+0000","last_undegraded":"2024-04-25T09:04:27.233974+0000","last_fullsized":"2024-04-25T09:04:27.233974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:43.508812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233279+0000","last_change":"2024-04-25T09:04:25.209064+0000","last_active":"2024-04-25T09:04:27.233279+0000","last_peered":"2024-04-25T09:04:27.233279+0000","last_clean":"2024-04-25T09:04:27.233279+0000","last_became_active":"2024-04-25T09:04:25.208733+0000","last_became_peered":"2024-04-25T09:04:25.208733+0000","last_unstale":"2024-04-25T09:04:27.233279+0000","last_undegraded":"2024-04-25T09:04:27.233279+0000","last_fullsized":"2024-04-25T09:04:27.233279+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:37.157522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369432+0000","last_change":"2024-04-25T09:04:26.228292+0000","last_active":"2024-04-25T09:04:28.369432+0000","last_peered":"2024-04-25T09:04:28.369432+0000","last_clean":"2024-04-25T09:04:28.369432+0000","last_became_active":"2024-04-25T09:04:26.227978+0000","last_became_peered":"2024-04-25T09:04:26.227978+0000","last_unstale":"2024-04-25T09:04:28.369432+0000","last_undegraded":"2024-04-25T09:04:28.369432+0000","last_fullsized":"2024-04-25T09:04:28.369432+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:58.430191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368448+0000","last_change":"2024-04-25T09:04:25.207956+0000","last_active":"2024-04-25T09:04:28.368448+0000","last_peered":"2024-04-25T09:04:28.368448+0000","last_clean":"2024-04-25T09:04:28.368448+0000","last_became_active":"2024-04-25T09:04:25.207646+0000","last_became_peered":"2024-04-25T09:04:25.207646+0000","last_unstale":"2024-04-25T09:04:28.368448+0000","last_undegraded":"2024-04-25T09:04:28.368448+0000","last_fullsized":"2024-04-25T09:04:28.368448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368292+0000","last_change":"2024-04-25T09:04:26.236804+0000","last_active":"2024-04-25T09:04:28.368292+0000","last_peered":"2024-04-25T09:04:28.368292+0000","last_clean":"2024-04-25T09:04:28.368292+0000","last_became_active":"2024-04-25T09:04:26.235872+0000","last_became_peered":"2024-04-25T09:04:26.235872+0000","last_unstale":"2024-04-25T09:04:28.368292+0000","last_undegraded":"2024-04-25T09:04:28.368292+0000","last_fullsized":"2024-04-25T09:04:28.368292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.889201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368082+0000","last_change":"2024-04-25T09:04:25.209033+0000","last_active":"2024-04-25T09:04:28.368082+0000","last_peered":"2024-04-25T09:04:28.368082+0000","last_clean":"2024-04-25T09:04:28.368082+0000","last_became_active":"2024-04-25T09:04:25.208734+0000","last_became_peered":"2024-04-25T09:04:25.208734+0000","last_unstale":"2024-04-25T09:04:28.368082+0000","last_undegraded":"2024-04-25T09:04:28.368082+0000","last_fullsized":"2024-04-25T09:04:28.368082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:57.417759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370812+0000","last_change":"2024-04-25T09:04:26.239390+0000","last_active":"2024-04-25T09:04:28.370812+0000","last_peered":"2024-04-25T09:04:28.370812+0000","last_clean":"2024-04-25T09:04:28.370812+0000","last_became_active":"2024-04-25T09:04:26.239103+0000","last_became_peered":"2024-04-25T09:04:26.239103+0000","last_unstale":"2024-04-25T09:04:28.370812+0000","last_undegraded":"2024-04-25T09:04:28.370812+0000","last_fullsized":"2024-04-25T09:04:28.370812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:36.846809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.211586+0000","last_change":"2024-04-25T09:04:25.209807+0000","last_active":"2024-04-25T09:04:28.211586+0000","last_peered":"2024-04-25T09:04:28.211586+0000","last_clean":"2024-04-25T09:04:28.211586+0000","last_became_active":"2024-04-25T09:04:25.207401+0000","last_became_peered":"2024-04-25T09:04:25.207401+0000","last_unstale":"2024-04-25T09:04:28.211586+0000","last_undegraded":"2024-04-25T09:04:28.211586+0000","last_fullsized":"2024-04-25T09:04:28.211586+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:45.866728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.231832+0000","last_change":"2024-04-25T09:04:26.212867+0000","last_active":"2024-04-25T09:04:27.231832+0000","last_peered":"2024-04-25T09:04:27.231832+0000","last_clean":"2024-04-25T09:04:27.231832+0000","last_became_active":"2024-04-25T09:04:26.212369+0000","last_became_peered":"2024-04-25T09:04:26.212369+0000","last_unstale":"2024-04-25T09:04:27.231832+0000","last_undegraded":"2024-04-25T09:04:27.231832+0000","last_fullsized":"2024-04-25T09:04:27.231832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:51.170042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368619+0000","last_change":"2024-04-25T09:04:25.200573+0000","last_active":"2024-04-25T09:04:28.368619+0000","last_peered":"2024-04-25T09:04:28.368619+0000","last_clean":"2024-04-25T09:04:28.368619+0000","last_became_active":"2024-04-25T09:04:25.200262+0000","last_became_peered":"2024-04-25T09:04:25.200262+0000","last_unstale":"2024-04-25T09:04:28.368619+0000","last_undegraded":"2024-04-25T09:04:28.368619+0000","last_fullsized":"2024-04-25T09:04:28.368619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:43.725847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367551+0000","last_change":"2024-04-25T09:04:26.240988+0000","last_active":"2024-04-25T09:04:28.367551+0000","last_peered":"2024-04-25T09:04:28.367551+0000","last_clean":"2024-04-25T09:04:28.367551+0000","last_became_active":"2024-04-25T09:04:26.240638+0000","last_became_peered":"2024-04-25T09:04:26.240638+0000","last_unstale":"2024-04-25T09:04:28.367551+0000","last_undegraded":"2024-04-25T09:04:28.367551+0000","last_fullsized":"2024-04-25T09:04:28.367551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:15.750677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076581+0000","last_change":"2024-04-25T09:04:25.202054+0000","last_active":"2024-04-25T09:04:28.076581+0000","last_peered":"2024-04-25T09:04:28.076581+0000","last_clean":"2024-04-25T09:04:28.076581+0000","last_became_active":"2024-04-25T09:04:25.201654+0000","last_became_peered":"2024-04-25T09:04:25.201654+0000","last_unstale":"2024-04-25T09:04:28.076581+0000","last_undegraded":"2024-04-25T09:04:28.076581+0000","last_fullsized":"2024-04-25T09:04:28.076581+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:28.333679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.234078+0000","last_change":"2024-04-25T09:04:26.237565+0000","last_active":"2024-04-25T09:04:27.234078+0000","last_peered":"2024-04-25T09:04:27.234078+0000","last_clean":"2024-04-25T09:04:27.234078+0000","last_became_active":"2024-04-25T09:04:26.237249+0000","last_became_peered":"2024-04-25T09:04:26.237249+0000","last_unstale":"2024-04-25T09:04:27.234078+0000","last_undegraded":"2024-04-25T09:04:27.234078+0000","last_fullsized":"2024-04-25T09:04:27.234078+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:36.700609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076072+0000","last_change":"2024-04-25T09:04:25.207756+0000","last_active":"2024-04-25T09:04:28.076072+0000","last_peered":"2024-04-25T09:04:28.076072+0000","last_clean":"2024-04-25T09:04:28.076072+0000","last_became_active":"2024-04-25T09:04:25.207460+0000","last_became_peered":"2024-04-25T09:04:25.207460+0000","last_unstale":"2024-04-25T09:04:28.076072+0000","last_undegraded":"2024-04-25T09:04:28.076072+0000","last_fullsized":"2024-04-25T09:04:28.076072+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:09.550143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232897+0000","last_change":"2024-04-25T09:04:26.226969+0000","last_active":"2024-04-25T09:04:27.232897+0000","last_peered":"2024-04-25T09:04:27.232897+0000","last_clean":"2024-04-25T09:04:27.232897+0000","last_became_active":"2024-04-25T09:04:26.226400+0000","last_became_peered":"2024-04-25T09:04:26.226400+0000","last_unstale":"2024-04-25T09:04:27.232897+0000","last_undegraded":"2024-04-25T09:04:27.232897+0000","last_fullsized":"2024-04-25T09:04:27.232897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:21.181723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367892+0000","last_change":"2024-04-25T09:04:25.207463+0000","last_active":"2024-04-25T09:04:28.367892+0000","last_peered":"2024-04-25T09:04:28.367892+0000","last_clean":"2024-04-25T09:04:28.367892+0000","last_became_active":"2024-04-25T09:04:25.207178+0000","last_became_peered":"2024-04-25T09:04:25.207178+0000","last_unstale":"2024-04-25T09:04:28.367892+0000","last_undegraded":"2024-04-25T09:04:28.367892+0000","last_fullsized":"2024-04-25T09:04:28.367892+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:33.899157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367344+0000","last_change":"2024-04-25T09:04:26.231308+0000","last_active":"2024-04-25T09:04:28.367344+0000","last_peered":"2024-04-25T09:04:28.367344+0000","last_clean":"2024-04-25T09:04:28.367344+0000","last_became_active":"2024-04-25T09:04:26.230107+0000","last_became_peered":"2024-04-25T09:04:26.230107+0000","last_unstale":"2024-04-25T09:04:28.367344+0000","last_undegraded":"2024-04-25T09:04:28.367344+0000","last_fullsized":"2024-04-25T09:04:28.367344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:47.725879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370180+0000","last_change":"2024-04-25T09:04:25.211486+0000","last_active":"2024-04-25T09:04:28.370180+0000","last_peered":"2024-04-25T09:04:28.370180+0000","last_clean":"2024-04-25T09:04:28.370180+0000","last_became_active":"2024-04-25T09:04:25.210675+0000","last_became_peered":"2024-04-25T09:04:25.210675+0000","last_unstale":"2024-04-25T09:04:28.370180+0000","last_undegraded":"2024-04-25T09:04:28.370180+0000","last_fullsized":"2024-04-25T09:04:28.370180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369160+0000","last_change":"2024-04-25T09:04:26.232458+0000","last_active":"2024-04-25T09:04:28.369160+0000","last_peered":"2024-04-25T09:04:28.369160+0000","last_clean":"2024-04-25T09:04:28.369160+0000","last_became_active":"2024-04-25T09:04:26.232152+0000","last_became_peered":"2024-04-25T09:04:26.232152+0000","last_unstale":"2024-04-25T09:04:28.369160+0000","last_undegraded":"2024-04-25T09:04:28.369160+0000","last_fullsized":"2024-04-25T09:04:28.369160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:30.510375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368619+0000","last_change":"2024-04-25T09:04:25.205276+0000","last_active":"2024-04-25T09:04:28.368619+0000","last_peered":"2024-04-25T09:04:28.368619+0000","last_clean":"2024-04-25T09:04:28.368619+0000","last_became_active":"2024-04-25T09:04:25.204962+0000","last_became_peered":"2024-04-25T09:04:25.204962+0000","last_unstale":"2024-04-25T09:04:28.368619+0000","last_undegraded":"2024-04-25T09:04:28.368619+0000","last_fullsized":"2024-04-25T09:04:28.368619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:38.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232923+0000","last_change":"2024-04-25T09:04:26.240130+0000","last_active":"2024-04-25T09:04:27.232923+0000","last_peered":"2024-04-25T09:04:27.232923+0000","last_clean":"2024-04-25T09:04:27.232923+0000","last_became_active":"2024-04-25T09:04:26.236257+0000","last_became_peered":"2024-04-25T09:04:26.236257+0000","last_unstale":"2024-04-25T09:04:27.232923+0000","last_undegraded":"2024-04-25T09:04:27.232923+0000","last_fullsized":"2024-04-25T09:04:27.232923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232385+0000","last_change":"2024-04-25T09:04:25.197723+0000","last_active":"2024-04-25T09:04:27.232385+0000","last_peered":"2024-04-25T09:04:27.232385+0000","last_clean":"2024-04-25T09:04:27.232385+0000","last_became_active":"2024-04-25T09:04:25.197275+0000","last_became_peered":"2024-04-25T09:04:25.197275+0000","last_unstale":"2024-04-25T09:04:27.232385+0000","last_undegraded":"2024-04-25T09:04:27.232385+0000","last_fullsized":"2024-04-25T09:04:27.232385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.834376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368340+0000","last_change":"2024-04-25T09:04:26.226130+0000","last_active":"2024-04-25T09:04:28.368340+0000","last_peered":"2024-04-25T09:04:28.368340+0000","last_clean":"2024-04-25T09:04:28.368340+0000","last_became_active":"2024-04-25T09:04:26.225532+0000","last_became_peered":"2024-04-25T09:04:26.225532+0000","last_unstale":"2024-04-25T09:04:28.368340+0000","last_undegraded":"2024-04-25T09:04:28.368340+0000","last_fullsized":"2024-04-25T09:04:28.368340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:30.946720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232108+0000","last_change":"2024-04-25T09:04:25.191706+0000","last_active":"2024-04-25T09:04:27.232108+0000","last_peered":"2024-04-25T09:04:27.232108+0000","last_clean":"2024-04-25T09:04:27.232108+0000","last_became_active":"2024-04-25T09:04:25.190455+0000","last_became_peered":"2024-04-25T09:04:25.190455+0000","last_unstale":"2024-04-25T09:04:27.232108+0000","last_undegraded":"2024-04-25T09:04:27.232108+0000","last_fullsized":"2024-04-25T09:04:27.232108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:18.342663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369381+0000","last_change":"2024-04-25T09:04:26.217025+0000","last_active":"2024-04-25T09:04:28.369381+0000","last_peered":"2024-04-25T09:04:28.369381+0000","last_clean":"2024-04-25T09:04:28.369381+0000","last_became_active":"2024-04-25T09:04:26.216110+0000","last_became_peered":"2024-04-25T09:04:26.216110+0000","last_unstale":"2024-04-25T09:04:28.369381+0000","last_undegraded":"2024-04-25T09:04:28.369381+0000","last_fullsized":"2024-04-25T09:04:28.369381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:19.936787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.211304+0000","last_change":"2024-04-25T09:04:25.203759+0000","last_active":"2024-04-25T09:04:28.211304+0000","last_peered":"2024-04-25T09:04:28.211304+0000","last_clean":"2024-04-25T09:04:28.211304+0000","last_became_active":"2024-04-25T09:04:25.203310+0000","last_became_peered":"2024-04-25T09:04:25.203310+0000","last_unstale":"2024-04-25T09:04:28.211304+0000","last_undegraded":"2024-04-25T09:04:28.211304+0000","last_fullsized":"2024-04-25T09:04:28.211304+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:27.537160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367763+0000","last_change":"2024-04-25T09:04:26.224528+0000","last_active":"2024-04-25T09:04:28.367763+0000","last_peered":"2024-04-25T09:04:28.367763+0000","last_clean":"2024-04-25T09:04:28.367763+0000","last_became_active":"2024-04-25T09:04:26.224221+0000","last_became_peered":"2024-04-25T09:04:26.224221+0000","last_unstale":"2024-04-25T09:04:28.367763+0000","last_undegraded":"2024-04-25T09:04:28.367763+0000","last_fullsized":"2024-04-25T09:04:28.367763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:27.543187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212375+0000","last_change":"2024-04-25T09:04:25.206521+0000","last_active":"2024-04-25T09:04:28.212375+0000","last_peered":"2024-04-25T09:04:28.212375+0000","last_clean":"2024-04-25T09:04:28.212375+0000","last_became_active":"2024-04-25T09:04:25.206196+0000","last_became_peered":"2024-04-25T09:04:25.206196+0000","last_unstale":"2024-04-25T09:04:28.212375+0000","last_undegraded":"2024-04-25T09:04:28.212375+0000","last_fullsized":"2024-04-25T09:04:28.212375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:33.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369968+0000","last_change":"2024-04-25T09:04:26.229046+0000","last_active":"2024-04-25T09:04:28.369968+0000","last_peered":"2024-04-25T09:04:28.369968+0000","last_clean":"2024-04-25T09:04:28.369968+0000","last_became_active":"2024-04-25T09:04:26.228781+0000","last_became_peered":"2024-04-25T09:04:26.228781+0000","last_unstale":"2024-04-25T09:04:28.369968+0000","last_undegraded":"2024-04-25T09:04:28.369968+0000","last_fullsized":"2024-04-25T09:04:28.369968+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:39.649661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368830+0000","last_change":"2024-04-25T09:04:25.205066+0000","last_active":"2024-04-25T09:04:28.368830+0000","last_peered":"2024-04-25T09:04:28.368830+0000","last_clean":"2024-04-25T09:04:28.368830+0000","last_became_active":"2024-04-25T09:04:25.204741+0000","last_became_peered":"2024-04-25T09:04:25.204741+0000","last_unstale":"2024-04-25T09:04:28.368830+0000","last_undegraded":"2024-04-25T09:04:28.368830+0000","last_fullsized":"2024-04-25T09:04:28.368830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:18.342663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.075954+0000","last_change":"2024-04-25T09:04:26.236052+0000","last_active":"2024-04-25T09:04:28.075954+0000","last_peered":"2024-04-25T09:04:28.075954+0000","last_clean":"2024-04-25T09:04:28.075954+0000","last_became_active":"2024-04-25T09:04:26.234393+0000","last_became_peered":"2024-04-25T09:04:26.234393+0000","last_unstale":"2024-04-25T09:04:28.075954+0000","last_undegraded":"2024-04-25T09:04:28.075954+0000","last_fullsized":"2024-04-25T09:04:28.075954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.576671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367732+0000","last_change":"2024-04-25T09:04:25.203672+0000","last_active":"2024-04-25T09:04:28.367732+0000","last_peered":"2024-04-25T09:04:28.367732+0000","last_clean":"2024-04-25T09:04:28.367732+0000","last_became_active":"2024-04-25T09:04:25.203329+0000","last_became_peered":"2024-04-25T09:04:25.203329+0000","last_unstale":"2024-04-25T09:04:28.367732+0000","last_undegraded":"2024-04-25T09:04:28.367732+0000","last_fullsized":"2024-04-25T09:04:28.367732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:29.934288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212153+0000","last_change":"2024-04-25T09:04:26.241160+0000","last_active":"2024-04-25T09:04:28.212153+0000","last_peered":"2024-04-25T09:04:28.212153+0000","last_clean":"2024-04-25T09:04:28.212153+0000","last_became_active":"2024-04-25T09:04:26.240856+0000","last_became_peered":"2024-04-25T09:04:26.240856+0000","last_unstale":"2024-04-25T09:04:28.212153+0000","last_undegraded":"2024-04-25T09:04:28.212153+0000","last_fullsized":"2024-04-25T09:04:28.212153+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:36.846809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227286+0000","last_change":"2024-04-25T09:04:25.198618+0000","last_active":"2024-04-25T09:04:27.227286+0000","last_peered":"2024-04-25T09:04:27.227286+0000","last_clean":"2024-04-25T09:04:27.227286+0000","last_became_active":"2024-04-25T09:04:25.198320+0000","last_became_peered":"2024-04-25T09:04:25.198320+0000","last_unstale":"2024-04-25T09:04:27.227286+0000","last_undegraded":"2024-04-25T09:04:27.227286+0000","last_fullsized":"2024-04-25T09:04:27.227286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:28.333679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369645+0000","last_change":"2024-04-25T09:04:26.239468+0000","last_active":"2024-04-25T09:04:28.369645+0000","last_peered":"2024-04-25T09:04:28.369645+0000","last_clean":"2024-04-25T09:04:28.369645+0000","last_became_active":"2024-04-25T09:04:26.237446+0000","last_became_peered":"2024-04-25T09:04:26.237446+0000","last_unstale":"2024-04-25T09:04:28.369645+0000","last_undegraded":"2024-04-25T09:04:28.369645+0000","last_fullsized":"2024-04-25T09:04:28.369645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:32.384380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076197+0000","last_change":"2024-04-25T09:04:25.206110+0000","last_active":"2024-04-25T09:04:28.076197+0000","last_peered":"2024-04-25T09:04:28.076197+0000","last_clean":"2024-04-25T09:04:28.076197+0000","last_became_active":"2024-04-25T09:04:25.205671+0000","last_became_peered":"2024-04-25T09:04:25.205671+0000","last_unstale":"2024-04-25T09:04:28.076197+0000","last_undegraded":"2024-04-25T09:04:28.076197+0000","last_fullsized":"2024-04-25T09:04:28.076197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:35.698063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.075890+0000","last_change":"2024-04-25T09:04:26.234944+0000","last_active":"2024-04-25T09:04:28.075890+0000","last_peered":"2024-04-25T09:04:28.075890+0000","last_clean":"2024-04-25T09:04:28.075890+0000","last_became_active":"2024-04-25T09:04:26.234649+0000","last_became_peered":"2024-04-25T09:04:26.234649+0000","last_unstale":"2024-04-25T09:04:28.075890+0000","last_undegraded":"2024-04-25T09:04:28.075890+0000","last_fullsized":"2024-04-25T09:04:28.075890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:51.170042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368239+0000","last_change":"2024-04-25T09:04:25.205857+0000","last_active":"2024-04-25T09:04:28.368239+0000","last_peered":"2024-04-25T09:04:28.368239+0000","last_clean":"2024-04-25T09:04:28.368239+0000","last_became_active":"2024-04-25T09:04:25.204814+0000","last_became_peered":"2024-04-25T09:04:25.204814+0000","last_unstale":"2024-04-25T09:04:28.368239+0000","last_undegraded":"2024-04-25T09:04:28.368239+0000","last_fullsized":"2024-04-25T09:04:28.368239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.834376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369041+0000","last_change":"2024-04-25T09:04:26.236022+0000","last_active":"2024-04-25T09:04:28.369041+0000","last_peered":"2024-04-25T09:04:28.369041+0000","last_clean":"2024-04-25T09:04:28.369041+0000","last_became_active":"2024-04-25T09:04:26.233069+0000","last_became_peered":"2024-04-25T09:04:26.233069+0000","last_unstale":"2024-04-25T09:04:28.369041+0000","last_undegraded":"2024-04-25T09:04:28.369041+0000","last_fullsized":"2024-04-25T09:04:28.369041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:17.441187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369502+0000","last_change":"2024-04-25T09:04:25.203994+0000","last_active":"2024-04-25T09:04:28.369502+0000","last_peered":"2024-04-25T09:04:28.369502+0000","last_clean":"2024-04-25T09:04:28.369502+0000","last_became_active":"2024-04-25T09:04:25.203665+0000","last_became_peered":"2024-04-25T09:04:25.203665+0000","last_unstale":"2024-04-25T09:04:28.369502+0000","last_undegraded":"2024-04-25T09:04:28.369502+0000","last_fullsized":"2024-04-25T09:04:28.369502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:31.510668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369363+0000","last_change":"2024-04-25T09:04:26.230309+0000","last_active":"2024-04-25T09:04:28.369363+0000","last_peered":"2024-04-25T09:04:28.369363+0000","last_clean":"2024-04-25T09:04:28.369363+0000","last_became_active":"2024-04-25T09:04:26.229935+0000","last_became_peered":"2024-04-25T09:04:26.229935+0000","last_unstale":"2024-04-25T09:04:28.369363+0000","last_undegraded":"2024-04-25T09:04:28.369363+0000","last_fullsized":"2024-04-25T09:04:28.369363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:47.487110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367079+0000","last_change":"2024-04-25T09:04:25.201881+0000","last_active":"2024-04-25T09:04:28.367079+0000","last_peered":"2024-04-25T09:04:28.367079+0000","last_clean":"2024-04-25T09:04:28.367079+0000","last_became_active":"2024-04-25T09:04:25.201582+0000","last_became_peered":"2024-04-25T09:04:25.201582+0000","last_unstale":"2024-04-25T09:04:28.367079+0000","last_undegraded":"2024-04-25T09:04:28.367079+0000","last_fullsized":"2024-04-25T09:04:28.367079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:41.953829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212421+0000","last_change":"2024-04-25T09:04:26.241389+0000","last_active":"2024-04-25T09:04:28.212421+0000","last_peered":"2024-04-25T09:04:28.212421+0000","last_clean":"2024-04-25T09:04:28.212421+0000","last_became_active":"2024-04-25T09:04:26.241141+0000","last_became_peered":"2024-04-25T09:04:26.241141+0000","last_unstale":"2024-04-25T09:04:28.212421+0000","last_undegraded":"2024-04-25T09:04:28.212421+0000","last_fullsized":"2024-04-25T09:04:28.212421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:22.765586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369451+0000","last_change":"2024-04-25T09:04:25.201894+0000","last_active":"2024-04-25T09:04:28.369451+0000","last_peered":"2024-04-25T09:04:28.369451+0000","last_clean":"2024-04-25T09:04:28.369451+0000","last_became_active":"2024-04-25T09:04:25.201583+0000","last_became_peered":"2024-04-25T09:04:25.201583+0000","last_unstale":"2024-04-25T09:04:28.369451+0000","last_undegraded":"2024-04-25T09:04:28.369451+0000","last_fullsized":"2024-04-25T09:04:28.369451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:57.417759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370974+0000","last_change":"2024-04-25T09:04:26.228528+0000","last_active":"2024-04-25T09:04:28.370974+0000","last_peered":"2024-04-25T09:04:28.370974+0000","last_clean":"2024-04-25T09:04:28.370974+0000","last_became_active":"2024-04-25T09:04:26.227833+0000","last_became_peered":"2024-04-25T09:04:26.227833+0000","last_unstale":"2024-04-25T09:04:28.370974+0000","last_undegraded":"2024-04-25T09:04:28.370974+0000","last_fullsized":"2024-04-25T09:04:28.370974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:21.229782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076447+0000","last_change":"2024-04-25T09:04:25.207893+0000","last_active":"2024-04-25T09:04:28.076447+0000","last_peered":"2024-04-25T09:04:28.076447+0000","last_clean":"2024-04-25T09:04:28.076447+0000","last_became_active":"2024-04-25T09:04:25.207587+0000","last_became_peered":"2024-04-25T09:04:25.207587+0000","last_unstale":"2024-04-25T09:04:28.076447+0000","last_undegraded":"2024-04-25T09:04:28.076447+0000","last_fullsized":"2024-04-25T09:04:28.076447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:27.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212473+0000","last_change":"2024-04-25T09:04:26.241946+0000","last_active":"2024-04-25T09:04:28.212473+0000","last_peered":"2024-04-25T09:04:28.212473+0000","last_clean":"2024-04-25T09:04:28.212473+0000","last_became_active":"2024-04-25T09:04:26.241637+0000","last_became_peered":"2024-04-25T09:04:26.241637+0000","last_unstale":"2024-04-25T09:04:28.212473+0000","last_undegraded":"2024-04-25T09:04:28.212473+0000","last_fullsized":"2024-04-25T09:04:28.212473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.061981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370214+0000","last_change":"2024-04-25T09:04:25.206786+0000","last_active":"2024-04-25T09:04:28.370214+0000","last_peered":"2024-04-25T09:04:28.370214+0000","last_clean":"2024-04-25T09:04:28.370214+0000","last_became_active":"2024-04-25T09:04:25.206460+0000","last_became_peered":"2024-04-25T09:04:25.206460+0000","last_unstale":"2024-04-25T09:04:28.370214+0000","last_undegraded":"2024-04-25T09:04:28.370214+0000","last_fullsized":"2024-04-25T09:04:28.370214+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:38.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.226938+0000","last_change":"2024-04-25T09:04:26.238638+0000","last_active":"2024-04-25T09:04:27.226938+0000","last_peered":"2024-04-25T09:04:27.226938+0000","last_clean":"2024-04-25T09:04:27.226938+0000","last_became_active":"2024-04-25T09:04:26.238351+0000","last_became_peered":"2024-04-25T09:04:26.238351+0000","last_unstale":"2024-04-25T09:04:27.226938+0000","last_undegraded":"2024-04-25T09:04:27.226938+0000","last_fullsized":"2024-04-25T09:04:27.226938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:21.190794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369651+0000","last_change":"2024-04-25T09:04:25.203588+0000","last_active":"2024-04-25T09:04:28.369651+0000","last_peered":"2024-04-25T09:04:28.369651+0000","last_clean":"2024-04-25T09:04:28.369651+0000","last_became_active":"2024-04-25T09:04:25.203279+0000","last_became_peered":"2024-04-25T09:04:25.203279+0000","last_unstale":"2024-04-25T09:04:28.369651+0000","last_undegraded":"2024-04-25T09:04:28.369651+0000","last_fullsized":"2024-04-25T09:04:28.369651+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:18.342663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233119+0000","last_change":"2024-04-25T09:04:26.237961+0000","last_active":"2024-04-25T09:04:27.233119+0000","last_peered":"2024-04-25T09:04:27.233119+0000","last_clean":"2024-04-25T09:04:27.233119+0000","last_became_active":"2024-04-25T09:04:26.235818+0000","last_became_peered":"2024-04-25T09:04:26.235818+0000","last_unstale":"2024-04-25T09:04:27.233119+0000","last_undegraded":"2024-04-25T09:04:27.233119+0000","last_fullsized":"2024-04-25T09:04:27.233119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:30.946720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369998+0000","last_change":"2024-04-25T09:04:25.205414+0000","last_active":"2024-04-25T09:04:28.369998+0000","last_peered":"2024-04-25T09:04:28.369998+0000","last_clean":"2024-04-25T09:04:28.369998+0000","last_became_active":"2024-04-25T09:04:25.205062+0000","last_became_peered":"2024-04-25T09:04:25.205062+0000","last_unstale":"2024-04-25T09:04:28.369998+0000","last_undegraded":"2024-04-25T09:04:28.369998+0000","last_fullsized":"2024-04-25T09:04:28.369998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:43.725847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232033+0000","last_change":"2024-04-25T09:04:26.221218+0000","last_active":"2024-04-25T09:04:27.232033+0000","last_peered":"2024-04-25T09:04:27.232033+0000","last_clean":"2024-04-25T09:04:27.232033+0000","last_became_active":"2024-04-25T09:04:26.217960+0000","last_became_peered":"2024-04-25T09:04:26.217960+0000","last_unstale":"2024-04-25T09:04:27.232033+0000","last_undegraded":"2024-04-25T09:04:27.232033+0000","last_fullsized":"2024-04-25T09:04:27.232033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.576671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368022+0000","last_change":"2024-04-25T09:04:25.207990+0000","last_active":"2024-04-25T09:04:28.368022+0000","last_peered":"2024-04-25T09:04:28.368022+0000","last_clean":"2024-04-25T09:04:28.368022+0000","last_became_active":"2024-04-25T09:04:25.207576+0000","last_became_peered":"2024-04-25T09:04:25.207576+0000","last_unstale":"2024-04-25T09:04:28.368022+0000","last_undegraded":"2024-04-25T09:04:28.368022+0000","last_fullsized":"2024-04-25T09:04:28.368022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:46.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369643+0000","last_change":"2024-04-25T09:04:26.233431+0000","last_active":"2024-04-25T09:04:28.369643+0000","last_peered":"2024-04-25T09:04:28.369643+0000","last_clean":"2024-04-25T09:04:28.369643+0000","last_became_active":"2024-04-25T09:04:26.233040+0000","last_became_peered":"2024-04-25T09:04:26.233040+0000","last_unstale":"2024-04-25T09:04:28.369643+0000","last_undegraded":"2024-04-25T09:04:28.369643+0000","last_fullsized":"2024-04-25T09:04:28.369643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:25.409619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232045+0000","last_change":"2024-04-25T09:04:25.198844+0000","last_active":"2024-04-25T09:04:27.232045+0000","last_peered":"2024-04-25T09:04:27.232045+0000","last_clean":"2024-04-25T09:04:27.232045+0000","last_became_active":"2024-04-25T09:04:25.198376+0000","last_became_peered":"2024-04-25T09:04:25.198376+0000","last_unstale":"2024-04-25T09:04:27.232045+0000","last_undegraded":"2024-04-25T09:04:27.232045+0000","last_fullsized":"2024-04-25T09:04:27.232045+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:38.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227542+0000","last_change":"2024-04-25T09:04:26.241339+0000","last_active":"2024-04-25T09:04:27.227542+0000","last_peered":"2024-04-25T09:04:27.227542+0000","last_clean":"2024-04-25T09:04:27.227542+0000","last_became_active":"2024-04-25T09:04:26.241051+0000","last_became_peered":"2024-04-25T09:04:26.241051+0000","last_unstale":"2024-04-25T09:04:27.227542+0000","last_undegraded":"2024-04-25T09:04:27.227542+0000","last_fullsized":"2024-04-25T09:04:27.227542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:34.911589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368027+0000","last_change":"2024-04-25T09:04:25.202687+0000","last_active":"2024-04-25T09:04:28.368027+0000","last_peered":"2024-04-25T09:04:28.368027+0000","last_clean":"2024-04-25T09:04:28.368027+0000","last_became_active":"2024-04-25T09:04:25.202400+0000","last_became_peered":"2024-04-25T09:04:25.202400+0000","last_unstale":"2024-04-25T09:04:28.368027+0000","last_undegraded":"2024-04-25T09:04:28.368027+0000","last_fullsized":"2024-04-25T09:04:28.368027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:02.418007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370931+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:04:28.370931+0000","last_peered":"2024-04-25T09:04:28.370931+0000","last_clean":"2024-04-25T09:04:28.370931+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:04:28.370931+0000","last_undegraded":"2024-04-25T09:04:28.370931+0000","last_fullsized":"2024-04-25T09:04:28.370931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":147734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":147756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":146898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":736560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:35.688 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:35.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.923+0000 7f756da00640 1 Processor -- start 2024-04-25T09:04:35.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.923+0000 7f756da00640 1 -- start start 2024-04-25T09:04:35.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.924+0000 7f756da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75680b56f0 0x7f7568160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.924+0000 7f756da00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f75680b6a90 0x7f7568161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.924+0000 7f756da00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b27a0 0x7f7568161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.924+0000 7f756da00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7568161780 con 0x7f75680b56f0 2024-04-25T09:04:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.924+0000 7f756da00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75680cedc0 con 0x7f75680b6a90 2024-04-25T09:04:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.924+0000 7f756da00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680d3110 con 0x7f75680b27a0 2024-04-25T09:04:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75680b56f0 0x7f7568160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75680b56f0 0x7f7568160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39728/0 (socket says 172.21.15.62:39728) 2024-04-25T09:04:35.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567e00640 1 -- 172.21.15.62:0/1709529642 learned_addr learned my addr 172.21.15.62:0/1709529642 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567400640 1 --2- 172.21.15.62:0/1709529642 >> v2:172.21.15.121:3300/0 conn(0x7f75680b6a90 0x7f7568161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f756d000640 1 --2- 172.21.15.62:0/1709529642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b27a0 0x7f7568161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567e00640 1 -- 172.21.15.62:0/1709529642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b27a0 msgr2=0x7f7568161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567e00640 1 --2- 172.21.15.62:0/1709529642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b27a0 0x7f7568161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567e00640 1 -- 172.21.15.62:0/1709529642 >> v2:172.21.15.121:3300/0 conn(0x7f75680b6a90 msgr2=0x7f7568161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567e00640 1 --2- 172.21.15.62:0/1709529642 >> v2:172.21.15.121:3300/0 conn(0x7f75680b6a90 0x7f7568161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567e00640 1 -- 172.21.15.62:0/1709529642 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680d33f0 con 0x7f75680b56f0 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.925+0000 7f7567400640 1 --2- 172.21.15.62:0/1709529642 >> v2:172.21.15.121:3300/0 conn(0x7f75680b6a90 0x7f7568161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.927+0000 7f7567e00640 1 --2- 172.21.15.62:0/1709529642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b56f0 0x7f7568160d00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f756808c570 tx=0x7f755802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43787740261502c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.927+0000 7f7566a00640 1 -- 172.21.15.62:0/1709529642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558036070 con 0x7f75680b56f0 2024-04-25T09:04:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.927+0000 7f7566a00640 1 -- 172.21.15.62:0/1709529642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7558031600 con 0x7f75680b56f0 2024-04-25T09:04:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.927+0000 7f7566a00640 1 -- 172.21.15.62:0/1709529642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75580319d0 con 0x7f75680b56f0 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 -- 172.21.15.62:0/1709529642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b56f0 msgr2=0x7f7568160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 --2- 172.21.15.62:0/1709529642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b56f0 0x7f7568160d00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f756808c570 tx=0x7f755802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 -- 172.21.15.62:0/1709529642 shutdown_connections 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 --2- 172.21.15.62:0/1709529642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b27a0 0x7f7568161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 --2- 172.21.15.62:0/1709529642 >> v2:172.21.15.121:3300/0 conn(0x7f75680b6a90 0x7f7568161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 --2- 172.21.15.62:0/1709529642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b56f0 0x7f7568160d00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 -- 172.21.15.62:0/1709529642 >> 172.21.15.62:0/1709529642 conn(0x7f7568066d10 msgr2=0x7f7568067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 -- 172.21.15.62:0/1709529642 shutdown_connections 2024-04-25T09:04:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.928+0000 7f756da00640 1 -- 172.21.15.62:0/1709529642 wait complete. 2024-04-25T09:04:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.930+0000 7f756da00640 1 Processor -- start 2024-04-25T09:04:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.931+0000 7f756da00640 1 -- start start 2024-04-25T09:04:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f756da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b27a0 0x7f75681f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f756da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75680b56f0 0x7f756814c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f756da00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b6a90 0x7f75681f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f756da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75680b2d60 con 0x7f75680b27a0 2024-04-25T09:04:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f756da00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75680cea50 con 0x7f75680b56f0 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f756da00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680b2ee0 con 0x7f75680b6a90 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f7567e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b27a0 0x7f75681f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f7567e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b27a0 0x7f75681f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39738/0 (socket says 172.21.15.62:39738) 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.932+0000 7f7567e00640 1 -- 172.21.15.62:0/3460218880 learned_addr learned my addr 172.21.15.62:0/3460218880 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f7567e00640 1 -- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b6a90 msgr2=0x7f75681f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f756d000640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b6a90 0x7f75681f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f7567e00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b6a90 0x7f75681f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f7567e00640 1 -- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75680b56f0 msgr2=0x7f756814c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f7567e00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75680b56f0 0x7f756814c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f7567e00640 1 -- 172.21.15.62:0/3460218880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756814ce30 con 0x7f75680b27a0 2024-04-25T09:04:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f7567e00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b27a0 0x7f75681f7a40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f755802ac90 tx=0x7f7558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f755fe00640 1 -- 172.21.15.62:0/3460218880 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558041020 con 0x7f75680b27a0 2024-04-25T09:04:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f755fe00640 1 -- 172.21.15.62:0/3460218880 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7558008790 con 0x7f75680b27a0 2024-04-25T09:04:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f755fe00640 1 -- 172.21.15.62:0/3460218880 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558036040 con 0x7f75680b27a0 2024-04-25T09:04:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.933+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756814d110 con 0x7f75680b27a0 2024-04-25T09:04:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.934+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75681f1390 con 0x7f75680b27a0 2024-04-25T09:04:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.935+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756804ef20 con 0x7f75680b27a0 2024-04-25T09:04:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.936+0000 7f755fe00640 1 -- 172.21.15.62:0/3460218880 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7558044020 con 0x7f75680b27a0 2024-04-25T09:04:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.937+0000 7f755fe00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f754406b4d0 0x7f754406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.937+0000 7f755fe00640 1 -- 172.21.15.62:0/3460218880 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75580ab060 con 0x7f75680b27a0 2024-04-25T09:04:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.937+0000 7f7567400640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f754406b4d0 0x7f754406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.939+0000 7f7567400640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f754406b4d0 0x7f754406d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7550006fd0 tx=0x7f7550008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:35.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:35.941+0000 7f755fe00640 1 -- 172.21.15.62:0/3460218880 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f755804b030 con 0x7f75680b27a0 2024-04-25T09:04:36.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.137+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f756814d630 con 0x7f754406b4d0 2024-04-25T09:04:36.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.156+0000 7f755fe00640 1 -- 172.21.15.62:0/3460218880 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251954 (secure 0 0 0) 0x7f756814d630 con 0x7f754406b4d0 2024-04-25T09:04:36.146 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:36.149 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:04:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f754406b4d0 msgr2=0x7f754406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f754406b4d0 0x7f754406d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7550006fd0 tx=0x7f7550008040 comp rx=0 tx=0).stop 2024-04-25T09:04:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b27a0 msgr2=0x7f75681f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b27a0 0x7f75681f7a40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f755802ac90 tx=0x7f7558004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 shutdown_connections 2024-04-25T09:04:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f754406b4d0 0x7f754406d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75680b6a90 0x7f75681f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75680b56f0 0x7f756814c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 --2- 172.21.15.62:0/3460218880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75680b27a0 0x7f75681f7a40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 >> 172.21.15.62:0/3460218880 conn(0x7f7568066d10 msgr2=0x7f75680c6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 shutdown_connections 2024-04-25T09:04:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.163+0000 7f756da00640 1 -- 172.21.15.62:0/3460218880 wait complete. 2024-04-25T09:04:36.173 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-04-25T09:04:35.911493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220312,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754408,"statfs":{"total":773094113280,"available":772868513792,"internally_reserved":0,"allocated":4235264,"data_stored":2354532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.715807"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368765+0000","last_change":"2024-04-25T09:04:26.241063+0000","last_active":"2024-04-25T09:04:28.368765+0000","last_peered":"2024-04-25T09:04:28.368765+0000","last_clean":"2024-04-25T09:04:28.368765+0000","last_became_active":"2024-04-25T09:04:26.238514+0000","last_became_peered":"2024-04-25T09:04:26.238514+0000","last_unstale":"2024-04-25T09:04:28.368765+0000","last_undegraded":"2024-04-25T09:04:28.368765+0000","last_fullsized":"2024-04-25T09:04:28.368765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:34.911589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076208+0000","last_change":"2024-04-25T09:04:26.237196+0000","last_active":"2024-04-25T09:04:28.076208+0000","last_peered":"2024-04-25T09:04:28.076208+0000","last_clean":"2024-04-25T09:04:28.076208+0000","last_became_active":"2024-04-25T09:04:26.236189+0000","last_became_peered":"2024-04-25T09:04:26.236189+0000","last_unstale":"2024-04-25T09:04:28.076208+0000","last_undegraded":"2024-04-25T09:04:28.076208+0000","last_fullsized":"2024-04-25T09:04:28.076208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:21.190794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369992+0000","last_change":"2024-04-25T09:04:26.223452+0000","last_active":"2024-04-25T09:04:28.369992+0000","last_peered":"2024-04-25T09:04:28.369992+0000","last_clean":"2024-04-25T09:04:28.369992+0000","last_became_active":"2024-04-25T09:04:26.223179+0000","last_became_peered":"2024-04-25T09:04:26.223179+0000","last_unstale":"2024-04-25T09:04:28.369992+0000","last_undegraded":"2024-04-25T09:04:28.369992+0000","last_fullsized":"2024-04-25T09:04:28.369992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:19.355095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232834+0000","last_change":"2024-04-25T09:04:26.226411+0000","last_active":"2024-04-25T09:04:27.232834+0000","last_peered":"2024-04-25T09:04:27.232834+0000","last_clean":"2024-04-25T09:04:27.232834+0000","last_became_active":"2024-04-25T09:04:26.226131+0000","last_became_peered":"2024-04-25T09:04:26.226131+0000","last_unstale":"2024-04-25T09:04:27.232834+0000","last_undegraded":"2024-04-25T09:04:27.232834+0000","last_fullsized":"2024-04-25T09:04:27.232834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.889201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.075837+0000","last_change":"2024-04-25T09:04:26.236909+0000","last_active":"2024-04-25T09:04:28.075837+0000","last_peered":"2024-04-25T09:04:28.075837+0000","last_clean":"2024-04-25T09:04:28.075837+0000","last_became_active":"2024-04-25T09:04:26.236362+0000","last_became_peered":"2024-04-25T09:04:26.236362+0000","last_unstale":"2024-04-25T09:04:28.075837+0000","last_undegraded":"2024-04-25T09:04:28.075837+0000","last_fullsized":"2024-04-25T09:04:28.075837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:13.827695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.211992+0000","last_change":"2024-04-25T09:04:26.237439+0000","last_active":"2024-04-25T09:04:28.211992+0000","last_peered":"2024-04-25T09:04:28.211992+0000","last_clean":"2024-04-25T09:04:28.211992+0000","last_became_active":"2024-04-25T09:04:26.236815+0000","last_became_peered":"2024-04-25T09:04:26.236815+0000","last_unstale":"2024-04-25T09:04:28.211992+0000","last_undegraded":"2024-04-25T09:04:28.211992+0000","last_fullsized":"2024-04-25T09:04:28.211992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:39.649661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367875+0000","last_change":"2024-04-25T09:04:26.231513+0000","last_active":"2024-04-25T09:04:28.367875+0000","last_peered":"2024-04-25T09:04:28.367875+0000","last_clean":"2024-04-25T09:04:28.367875+0000","last_became_active":"2024-04-25T09:04:26.231186+0000","last_became_peered":"2024-04-25T09:04:26.231186+0000","last_unstale":"2024-04-25T09:04:28.367875+0000","last_undegraded":"2024-04-25T09:04:28.367875+0000","last_fullsized":"2024-04-25T09:04:28.367875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:18.665177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367844+0000","last_change":"2024-04-25T09:04:26.237203+0000","last_active":"2024-04-25T09:04:28.367844+0000","last_peered":"2024-04-25T09:04:28.367844+0000","last_clean":"2024-04-25T09:04:28.367844+0000","last_became_active":"2024-04-25T09:04:26.236841+0000","last_became_peered":"2024-04-25T09:04:26.236841+0000","last_unstale":"2024-04-25T09:04:28.367844+0000","last_undegraded":"2024-04-25T09:04:28.367844+0000","last_fullsized":"2024-04-25T09:04:28.367844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.348176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369948+0000","last_change":"2024-04-25T09:04:26.232870+0000","last_active":"2024-04-25T09:04:28.369948+0000","last_peered":"2024-04-25T09:04:28.369948+0000","last_clean":"2024-04-25T09:04:28.369948+0000","last_became_active":"2024-04-25T09:04:26.232344+0000","last_became_peered":"2024-04-25T09:04:26.232344+0000","last_unstale":"2024-04-25T09:04:28.369948+0000","last_undegraded":"2024-04-25T09:04:28.369948+0000","last_fullsized":"2024-04-25T09:04:28.369948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:14.190501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233295+0000","last_change":"2024-04-25T09:04:26.227337+0000","last_active":"2024-04-25T09:04:27.233295+0000","last_peered":"2024-04-25T09:04:27.233295+0000","last_clean":"2024-04-25T09:04:27.233295+0000","last_became_active":"2024-04-25T09:04:26.226800+0000","last_became_peered":"2024-04-25T09:04:26.226800+0000","last_unstale":"2024-04-25T09:04:27.233295+0000","last_undegraded":"2024-04-25T09:04:27.233295+0000","last_fullsized":"2024-04-25T09:04:27.233295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:34.911589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367591+0000","last_change":"2024-04-25T09:04:26.224467+0000","last_active":"2024-04-25T09:04:28.367591+0000","last_peered":"2024-04-25T09:04:28.367591+0000","last_clean":"2024-04-25T09:04:28.367591+0000","last_became_active":"2024-04-25T09:04:26.224181+0000","last_became_peered":"2024-04-25T09:04:26.224181+0000","last_unstale":"2024-04-25T09:04:28.367591+0000","last_undegraded":"2024-04-25T09:04:28.367591+0000","last_fullsized":"2024-04-25T09:04:28.367591+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:33.128433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369536+0000","last_change":"2024-04-25T09:04:26.236743+0000","last_active":"2024-04-25T09:04:28.369536+0000","last_peered":"2024-04-25T09:04:28.369536+0000","last_clean":"2024-04-25T09:04:28.369536+0000","last_became_active":"2024-04-25T09:04:26.236311+0000","last_became_peered":"2024-04-25T09:04:26.236311+0000","last_unstale":"2024-04-25T09:04:28.369536+0000","last_undegraded":"2024-04-25T09:04:28.369536+0000","last_fullsized":"2024-04-25T09:04:28.369536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:44.738279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369519+0000","last_change":"2024-04-25T09:04:26.235258+0000","last_active":"2024-04-25T09:04:28.369519+0000","last_peered":"2024-04-25T09:04:28.369519+0000","last_clean":"2024-04-25T09:04:28.369519+0000","last_became_active":"2024-04-25T09:04:26.234312+0000","last_became_peered":"2024-04-25T09:04:26.234312+0000","last_unstale":"2024-04-25T09:04:28.369519+0000","last_undegraded":"2024-04-25T09:04:28.369519+0000","last_fullsized":"2024-04-25T09:04:28.369519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:26.452996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232145+0000","last_change":"2024-04-25T09:04:26.227707+0000","last_active":"2024-04-25T09:04:27.232145+0000","last_peered":"2024-04-25T09:04:27.232145+0000","last_clean":"2024-04-25T09:04:27.232145+0000","last_became_active":"2024-04-25T09:04:26.227283+0000","last_became_peered":"2024-04-25T09:04:26.227283+0000","last_unstale":"2024-04-25T09:04:27.232145+0000","last_undegraded":"2024-04-25T09:04:27.232145+0000","last_fullsized":"2024-04-25T09:04:27.232145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:21.190794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.226860+0000","last_change":"2024-04-25T09:04:26.225446+0000","last_active":"2024-04-25T09:04:27.226860+0000","last_peered":"2024-04-25T09:04:27.226860+0000","last_clean":"2024-04-25T09:04:27.226860+0000","last_became_active":"2024-04-25T09:04:26.225161+0000","last_became_peered":"2024-04-25T09:04:26.225161+0000","last_unstale":"2024-04-25T09:04:27.226860+0000","last_undegraded":"2024-04-25T09:04:27.226860+0000","last_fullsized":"2024-04-25T09:04:27.226860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.576671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.226604+0000","last_change":"2024-04-25T09:04:26.245986+0000","last_active":"2024-04-25T09:04:27.226604+0000","last_peered":"2024-04-25T09:04:27.226604+0000","last_clean":"2024-04-25T09:04:27.226604+0000","last_became_active":"2024-04-25T09:04:26.245517+0000","last_became_peered":"2024-04-25T09:04:26.245517+0000","last_unstale":"2024-04-25T09:04:27.226604+0000","last_undegraded":"2024-04-25T09:04:27.226604+0000","last_fullsized":"2024-04-25T09:04:27.226604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:51.170042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370762+0000","last_change":"2024-04-25T09:04:26.228748+0000","last_active":"2024-04-25T09:04:28.370762+0000","last_peered":"2024-04-25T09:04:28.370762+0000","last_clean":"2024-04-25T09:04:28.370762+0000","last_became_active":"2024-04-25T09:04:26.228225+0000","last_became_peered":"2024-04-25T09:04:26.228225+0000","last_unstale":"2024-04-25T09:04:28.370762+0000","last_undegraded":"2024-04-25T09:04:28.370762+0000","last_fullsized":"2024-04-25T09:04:28.370762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:32.893561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212359+0000","last_change":"2024-04-25T09:04:26.242396+0000","last_active":"2024-04-25T09:04:28.212359+0000","last_peered":"2024-04-25T09:04:28.212359+0000","last_clean":"2024-04-25T09:04:28.212359+0000","last_became_active":"2024-04-25T09:04:26.242112+0000","last_became_peered":"2024-04-25T09:04:26.242112+0000","last_unstale":"2024-04-25T09:04:28.212359+0000","last_undegraded":"2024-04-25T09:04:28.212359+0000","last_fullsized":"2024-04-25T09:04:28.212359+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:38.169954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212023+0000","last_change":"2024-04-25T09:04:26.242548+0000","last_active":"2024-04-25T09:04:28.212023+0000","last_peered":"2024-04-25T09:04:28.212023+0000","last_clean":"2024-04-25T09:04:28.212023+0000","last_became_active":"2024-04-25T09:04:26.242269+0000","last_became_peered":"2024-04-25T09:04:26.242269+0000","last_unstale":"2024-04-25T09:04:28.212023+0000","last_undegraded":"2024-04-25T09:04:28.212023+0000","last_fullsized":"2024-04-25T09:04:28.212023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:19.355095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369921+0000","last_change":"2024-04-25T09:04:26.239456+0000","last_active":"2024-04-25T09:04:28.369921+0000","last_peered":"2024-04-25T09:04:28.369921+0000","last_clean":"2024-04-25T09:04:28.369921+0000","last_became_active":"2024-04-25T09:04:26.238831+0000","last_became_peered":"2024-04-25T09:04:26.238831+0000","last_unstale":"2024-04-25T09:04:28.369921+0000","last_undegraded":"2024-04-25T09:04:28.369921+0000","last_fullsized":"2024-04-25T09:04:28.369921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:30.348176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227059+0000","last_change":"2024-04-25T09:04:26.236502+0000","last_active":"2024-04-25T09:04:27.227059+0000","last_peered":"2024-04-25T09:04:27.227059+0000","last_clean":"2024-04-25T09:04:27.227059+0000","last_became_active":"2024-04-25T09:04:26.236194+0000","last_became_peered":"2024-04-25T09:04:26.236194+0000","last_unstale":"2024-04-25T09:04:27.227059+0000","last_undegraded":"2024-04-25T09:04:27.227059+0000","last_fullsized":"2024-04-25T09:04:27.227059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.889201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367923+0000","last_change":"2024-04-25T09:04:26.238295+0000","last_active":"2024-04-25T09:04:28.367923+0000","last_peered":"2024-04-25T09:04:28.367923+0000","last_clean":"2024-04-25T09:04:28.367923+0000","last_became_active":"2024-04-25T09:04:26.234544+0000","last_became_peered":"2024-04-25T09:04:26.234544+0000","last_unstale":"2024-04-25T09:04:28.367923+0000","last_undegraded":"2024-04-25T09:04:28.367923+0000","last_fullsized":"2024-04-25T09:04:28.367923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:13.827695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369089+0000","last_change":"2024-04-25T09:04:26.231385+0000","last_active":"2024-04-25T09:04:28.369089+0000","last_peered":"2024-04-25T09:04:28.369089+0000","last_clean":"2024-04-25T09:04:28.369089+0000","last_became_active":"2024-04-25T09:04:26.231103+0000","last_became_peered":"2024-04-25T09:04:26.231103+0000","last_unstale":"2024-04-25T09:04:28.369089+0000","last_undegraded":"2024-04-25T09:04:28.369089+0000","last_fullsized":"2024-04-25T09:04:28.369089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:29.346112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368121+0000","last_change":"2024-04-25T09:04:26.230878+0000","last_active":"2024-04-25T09:04:28.368121+0000","last_peered":"2024-04-25T09:04:28.368121+0000","last_clean":"2024-04-25T09:04:28.368121+0000","last_became_active":"2024-04-25T09:04:26.230312+0000","last_became_peered":"2024-04-25T09:04:26.230312+0000","last_unstale":"2024-04-25T09:04:28.368121+0000","last_undegraded":"2024-04-25T09:04:28.368121+0000","last_fullsized":"2024-04-25T09:04:28.368121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:21.190794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233894+0000","last_change":"2024-04-25T09:04:26.218543+0000","last_active":"2024-04-25T09:04:27.233894+0000","last_peered":"2024-04-25T09:04:27.233894+0000","last_clean":"2024-04-25T09:04:27.233894+0000","last_became_active":"2024-04-25T09:04:26.218256+0000","last_became_peered":"2024-04-25T09:04:26.218256+0000","last_unstale":"2024-04-25T09:04:27.233894+0000","last_undegraded":"2024-04-25T09:04:27.233894+0000","last_fullsized":"2024-04-25T09:04:27.233894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:36.346921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233751+0000","last_change":"2024-04-25T09:04:26.221255+0000","last_active":"2024-04-25T09:04:27.233751+0000","last_peered":"2024-04-25T09:04:27.233751+0000","last_clean":"2024-04-25T09:04:27.233751+0000","last_became_active":"2024-04-25T09:04:26.220969+0000","last_became_peered":"2024-04-25T09:04:26.220969+0000","last_unstale":"2024-04-25T09:04:27.233751+0000","last_undegraded":"2024-04-25T09:04:27.233751+0000","last_fullsized":"2024-04-25T09:04:27.233751+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:03.430439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076322+0000","last_change":"2024-04-25T09:04:26.238580+0000","last_active":"2024-04-25T09:04:28.076322+0000","last_peered":"2024-04-25T09:04:28.076322+0000","last_clean":"2024-04-25T09:04:28.076322+0000","last_became_active":"2024-04-25T09:04:26.238291+0000","last_became_peered":"2024-04-25T09:04:26.238291+0000","last_unstale":"2024-04-25T09:04:28.076322+0000","last_undegraded":"2024-04-25T09:04:28.076322+0000","last_fullsized":"2024-04-25T09:04:28.076322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.889201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367910+0000","last_change":"2024-04-25T09:04:26.238450+0000","last_active":"2024-04-25T09:04:28.367910+0000","last_peered":"2024-04-25T09:04:28.367910+0000","last_clean":"2024-04-25T09:04:28.367910+0000","last_became_active":"2024-04-25T09:04:26.238161+0000","last_became_peered":"2024-04-25T09:04:26.238161+0000","last_unstale":"2024-04-25T09:04:28.367910+0000","last_undegraded":"2024-04-25T09:04:28.367910+0000","last_fullsized":"2024-04-25T09:04:28.367910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:51.170042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227292+0000","last_change":"2024-04-25T09:04:26.232881+0000","last_active":"2024-04-25T09:04:27.227292+0000","last_peered":"2024-04-25T09:04:27.227292+0000","last_clean":"2024-04-25T09:04:27.227292+0000","last_became_active":"2024-04-25T09:04:26.232522+0000","last_became_peered":"2024-04-25T09:04:26.232522+0000","last_unstale":"2024-04-25T09:04:27.227292+0000","last_undegraded":"2024-04-25T09:04:27.227292+0000","last_fullsized":"2024-04-25T09:04:27.227292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:30.946720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227132+0000","last_change":"2024-04-25T09:04:26.234593+0000","last_active":"2024-04-25T09:04:27.227132+0000","last_peered":"2024-04-25T09:04:27.227132+0000","last_clean":"2024-04-25T09:04:27.227132+0000","last_became_active":"2024-04-25T09:04:26.233853+0000","last_became_peered":"2024-04-25T09:04:26.233853+0000","last_unstale":"2024-04-25T09:04:27.227132+0000","last_undegraded":"2024-04-25T09:04:27.227132+0000","last_fullsized":"2024-04-25T09:04:27.227132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368043+0000","last_change":"2024-04-25T09:04:26.241932+0000","last_active":"2024-04-25T09:04:28.368043+0000","last_peered":"2024-04-25T09:04:28.368043+0000","last_clean":"2024-04-25T09:04:28.368043+0000","last_became_active":"2024-04-25T09:04:26.235859+0000","last_became_peered":"2024-04-25T09:04:26.235859+0000","last_unstale":"2024-04-25T09:04:28.368043+0000","last_undegraded":"2024-04-25T09:04:28.368043+0000","last_fullsized":"2024-04-25T09:04:28.368043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.576671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367357+0000","last_change":"2024-04-25T09:04:26.222060+0000","last_active":"2024-04-25T09:04:28.367357+0000","last_peered":"2024-04-25T09:04:28.367357+0000","last_clean":"2024-04-25T09:04:28.367357+0000","last_became_active":"2024-04-25T09:04:26.221758+0000","last_became_peered":"2024-04-25T09:04:26.221758+0000","last_unstale":"2024-04-25T09:04:28.367357+0000","last_undegraded":"2024-04-25T09:04:28.367357+0000","last_fullsized":"2024-04-25T09:04:28.367357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:24.128100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369942+0000","last_change":"2024-04-25T09:04:26.233156+0000","last_active":"2024-04-25T09:04:28.369942+0000","last_peered":"2024-04-25T09:04:28.369942+0000","last_clean":"2024-04-25T09:04:28.369942+0000","last_became_active":"2024-04-25T09:04:26.232789+0000","last_became_peered":"2024-04-25T09:04:26.232789+0000","last_unstale":"2024-04-25T09:04:28.369942+0000","last_undegraded":"2024-04-25T09:04:28.369942+0000","last_fullsized":"2024-04-25T09:04:28.369942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:18.665177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.371104+0000","last_change":"2024-04-25T09:04:25.210460+0000","last_active":"2024-04-25T09:04:28.371104+0000","last_peered":"2024-04-25T09:04:28.371104+0000","last_clean":"2024-04-25T09:04:28.371104+0000","last_became_active":"2024-04-25T09:04:25.210130+0000","last_became_peered":"2024-04-25T09:04:25.210130+0000","last_unstale":"2024-04-25T09:04:28.371104+0000","last_undegraded":"2024-04-25T09:04:28.371104+0000","last_fullsized":"2024-04-25T09:04:28.371104+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:20.169291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368947+0000","last_change":"2024-04-25T09:04:26.234788+0000","last_active":"2024-04-25T09:04:28.368947+0000","last_peered":"2024-04-25T09:04:28.368947+0000","last_clean":"2024-04-25T09:04:28.368947+0000","last_became_active":"2024-04-25T09:04:26.234467+0000","last_became_peered":"2024-04-25T09:04:26.234467+0000","last_unstale":"2024-04-25T09:04:28.368947+0000","last_undegraded":"2024-04-25T09:04:28.368947+0000","last_fullsized":"2024-04-25T09:04:28.368947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:21.181723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.234070+0000","last_change":"2024-04-25T09:04:25.198907+0000","last_active":"2024-04-25T09:04:27.234070+0000","last_peered":"2024-04-25T09:04:27.234070+0000","last_clean":"2024-04-25T09:04:27.234070+0000","last_became_active":"2024-04-25T09:04:25.198368+0000","last_became_peered":"2024-04-25T09:04:25.198368+0000","last_unstale":"2024-04-25T09:04:27.234070+0000","last_undegraded":"2024-04-25T09:04:27.234070+0000","last_fullsized":"2024-04-25T09:04:27.234070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:21.753154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368307+0000","last_change":"2024-04-25T09:04:26.230893+0000","last_active":"2024-04-25T09:04:28.368307+0000","last_peered":"2024-04-25T09:04:28.368307+0000","last_clean":"2024-04-25T09:04:28.368307+0000","last_became_active":"2024-04-25T09:04:26.230660+0000","last_became_peered":"2024-04-25T09:04:26.230660+0000","last_unstale":"2024-04-25T09:04:28.368307+0000","last_undegraded":"2024-04-25T09:04:28.368307+0000","last_fullsized":"2024-04-25T09:04:28.368307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212528+0000","last_change":"2024-04-25T09:04:25.211094+0000","last_active":"2024-04-25T09:04:28.212528+0000","last_peered":"2024-04-25T09:04:28.212528+0000","last_clean":"2024-04-25T09:04:28.212528+0000","last_became_active":"2024-04-25T09:04:25.210803+0000","last_became_peered":"2024-04-25T09:04:25.210803+0000","last_unstale":"2024-04-25T09:04:28.212528+0000","last_undegraded":"2024-04-25T09:04:28.212528+0000","last_fullsized":"2024-04-25T09:04:28.212528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:46.169978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233974+0000","last_change":"2024-04-25T09:04:26.241136+0000","last_active":"2024-04-25T09:04:27.233974+0000","last_peered":"2024-04-25T09:04:27.233974+0000","last_clean":"2024-04-25T09:04:27.233974+0000","last_became_active":"2024-04-25T09:04:26.239497+0000","last_became_peered":"2024-04-25T09:04:26.239497+0000","last_unstale":"2024-04-25T09:04:27.233974+0000","last_undegraded":"2024-04-25T09:04:27.233974+0000","last_fullsized":"2024-04-25T09:04:27.233974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:43.508812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233279+0000","last_change":"2024-04-25T09:04:25.209064+0000","last_active":"2024-04-25T09:04:27.233279+0000","last_peered":"2024-04-25T09:04:27.233279+0000","last_clean":"2024-04-25T09:04:27.233279+0000","last_became_active":"2024-04-25T09:04:25.208733+0000","last_became_peered":"2024-04-25T09:04:25.208733+0000","last_unstale":"2024-04-25T09:04:27.233279+0000","last_undegraded":"2024-04-25T09:04:27.233279+0000","last_fullsized":"2024-04-25T09:04:27.233279+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:37.157522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369432+0000","last_change":"2024-04-25T09:04:26.228292+0000","last_active":"2024-04-25T09:04:28.369432+0000","last_peered":"2024-04-25T09:04:28.369432+0000","last_clean":"2024-04-25T09:04:28.369432+0000","last_became_active":"2024-04-25T09:04:26.227978+0000","last_became_peered":"2024-04-25T09:04:26.227978+0000","last_unstale":"2024-04-25T09:04:28.369432+0000","last_undegraded":"2024-04-25T09:04:28.369432+0000","last_fullsized":"2024-04-25T09:04:28.369432+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:58.430191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368448+0000","last_change":"2024-04-25T09:04:25.207956+0000","last_active":"2024-04-25T09:04:28.368448+0000","last_peered":"2024-04-25T09:04:28.368448+0000","last_clean":"2024-04-25T09:04:28.368448+0000","last_became_active":"2024-04-25T09:04:25.207646+0000","last_became_peered":"2024-04-25T09:04:25.207646+0000","last_unstale":"2024-04-25T09:04:28.368448+0000","last_undegraded":"2024-04-25T09:04:28.368448+0000","last_fullsized":"2024-04-25T09:04:28.368448+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368292+0000","last_change":"2024-04-25T09:04:26.236804+0000","last_active":"2024-04-25T09:04:28.368292+0000","last_peered":"2024-04-25T09:04:28.368292+0000","last_clean":"2024-04-25T09:04:28.368292+0000","last_became_active":"2024-04-25T09:04:26.235872+0000","last_became_peered":"2024-04-25T09:04:26.235872+0000","last_unstale":"2024-04-25T09:04:28.368292+0000","last_undegraded":"2024-04-25T09:04:28.368292+0000","last_fullsized":"2024-04-25T09:04:28.368292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.889201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368082+0000","last_change":"2024-04-25T09:04:25.209033+0000","last_active":"2024-04-25T09:04:28.368082+0000","last_peered":"2024-04-25T09:04:28.368082+0000","last_clean":"2024-04-25T09:04:28.368082+0000","last_became_active":"2024-04-25T09:04:25.208734+0000","last_became_peered":"2024-04-25T09:04:25.208734+0000","last_unstale":"2024-04-25T09:04:28.368082+0000","last_undegraded":"2024-04-25T09:04:28.368082+0000","last_fullsized":"2024-04-25T09:04:28.368082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:57.417759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370812+0000","last_change":"2024-04-25T09:04:26.239390+0000","last_active":"2024-04-25T09:04:28.370812+0000","last_peered":"2024-04-25T09:04:28.370812+0000","last_clean":"2024-04-25T09:04:28.370812+0000","last_became_active":"2024-04-25T09:04:26.239103+0000","last_became_peered":"2024-04-25T09:04:26.239103+0000","last_unstale":"2024-04-25T09:04:28.370812+0000","last_undegraded":"2024-04-25T09:04:28.370812+0000","last_fullsized":"2024-04-25T09:04:28.370812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:36.846809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.211586+0000","last_change":"2024-04-25T09:04:25.209807+0000","last_active":"2024-04-25T09:04:28.211586+0000","last_peered":"2024-04-25T09:04:28.211586+0000","last_clean":"2024-04-25T09:04:28.211586+0000","last_became_active":"2024-04-25T09:04:25.207401+0000","last_became_peered":"2024-04-25T09:04:25.207401+0000","last_unstale":"2024-04-25T09:04:28.211586+0000","last_undegraded":"2024-04-25T09:04:28.211586+0000","last_fullsized":"2024-04-25T09:04:28.211586+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:45.866728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.231832+0000","last_change":"2024-04-25T09:04:26.212867+0000","last_active":"2024-04-25T09:04:27.231832+0000","last_peered":"2024-04-25T09:04:27.231832+0000","last_clean":"2024-04-25T09:04:27.231832+0000","last_became_active":"2024-04-25T09:04:26.212369+0000","last_became_peered":"2024-04-25T09:04:26.212369+0000","last_unstale":"2024-04-25T09:04:27.231832+0000","last_undegraded":"2024-04-25T09:04:27.231832+0000","last_fullsized":"2024-04-25T09:04:27.231832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:51.170042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368619+0000","last_change":"2024-04-25T09:04:25.200573+0000","last_active":"2024-04-25T09:04:28.368619+0000","last_peered":"2024-04-25T09:04:28.368619+0000","last_clean":"2024-04-25T09:04:28.368619+0000","last_became_active":"2024-04-25T09:04:25.200262+0000","last_became_peered":"2024-04-25T09:04:25.200262+0000","last_unstale":"2024-04-25T09:04:28.368619+0000","last_undegraded":"2024-04-25T09:04:28.368619+0000","last_fullsized":"2024-04-25T09:04:28.368619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:43.725847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367551+0000","last_change":"2024-04-25T09:04:26.240988+0000","last_active":"2024-04-25T09:04:28.367551+0000","last_peered":"2024-04-25T09:04:28.367551+0000","last_clean":"2024-04-25T09:04:28.367551+0000","last_became_active":"2024-04-25T09:04:26.240638+0000","last_became_peered":"2024-04-25T09:04:26.240638+0000","last_unstale":"2024-04-25T09:04:28.367551+0000","last_undegraded":"2024-04-25T09:04:28.367551+0000","last_fullsized":"2024-04-25T09:04:28.367551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:15.750677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076581+0000","last_change":"2024-04-25T09:04:25.202054+0000","last_active":"2024-04-25T09:04:28.076581+0000","last_peered":"2024-04-25T09:04:28.076581+0000","last_clean":"2024-04-25T09:04:28.076581+0000","last_became_active":"2024-04-25T09:04:25.201654+0000","last_became_peered":"2024-04-25T09:04:25.201654+0000","last_unstale":"2024-04-25T09:04:28.076581+0000","last_undegraded":"2024-04-25T09:04:28.076581+0000","last_fullsized":"2024-04-25T09:04:28.076581+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:28.333679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.234078+0000","last_change":"2024-04-25T09:04:26.237565+0000","last_active":"2024-04-25T09:04:27.234078+0000","last_peered":"2024-04-25T09:04:27.234078+0000","last_clean":"2024-04-25T09:04:27.234078+0000","last_became_active":"2024-04-25T09:04:26.237249+0000","last_became_peered":"2024-04-25T09:04:26.237249+0000","last_unstale":"2024-04-25T09:04:27.234078+0000","last_undegraded":"2024-04-25T09:04:27.234078+0000","last_fullsized":"2024-04-25T09:04:27.234078+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:36.700609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076072+0000","last_change":"2024-04-25T09:04:25.207756+0000","last_active":"2024-04-25T09:04:28.076072+0000","last_peered":"2024-04-25T09:04:28.076072+0000","last_clean":"2024-04-25T09:04:28.076072+0000","last_became_active":"2024-04-25T09:04:25.207460+0000","last_became_peered":"2024-04-25T09:04:25.207460+0000","last_unstale":"2024-04-25T09:04:28.076072+0000","last_undegraded":"2024-04-25T09:04:28.076072+0000","last_fullsized":"2024-04-25T09:04:28.076072+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:09.550143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232897+0000","last_change":"2024-04-25T09:04:26.226969+0000","last_active":"2024-04-25T09:04:27.232897+0000","last_peered":"2024-04-25T09:04:27.232897+0000","last_clean":"2024-04-25T09:04:27.232897+0000","last_became_active":"2024-04-25T09:04:26.226400+0000","last_became_peered":"2024-04-25T09:04:26.226400+0000","last_unstale":"2024-04-25T09:04:27.232897+0000","last_undegraded":"2024-04-25T09:04:27.232897+0000","last_fullsized":"2024-04-25T09:04:27.232897+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:21.181723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367892+0000","last_change":"2024-04-25T09:04:25.207463+0000","last_active":"2024-04-25T09:04:28.367892+0000","last_peered":"2024-04-25T09:04:28.367892+0000","last_clean":"2024-04-25T09:04:28.367892+0000","last_became_active":"2024-04-25T09:04:25.207178+0000","last_became_peered":"2024-04-25T09:04:25.207178+0000","last_unstale":"2024-04-25T09:04:28.367892+0000","last_undegraded":"2024-04-25T09:04:28.367892+0000","last_fullsized":"2024-04-25T09:04:28.367892+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:33.899157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367344+0000","last_change":"2024-04-25T09:04:26.231308+0000","last_active":"2024-04-25T09:04:28.367344+0000","last_peered":"2024-04-25T09:04:28.367344+0000","last_clean":"2024-04-25T09:04:28.367344+0000","last_became_active":"2024-04-25T09:04:26.230107+0000","last_became_peered":"2024-04-25T09:04:26.230107+0000","last_unstale":"2024-04-25T09:04:28.367344+0000","last_undegraded":"2024-04-25T09:04:28.367344+0000","last_fullsized":"2024-04-25T09:04:28.367344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:47.725879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370180+0000","last_change":"2024-04-25T09:04:25.211486+0000","last_active":"2024-04-25T09:04:28.370180+0000","last_peered":"2024-04-25T09:04:28.370180+0000","last_clean":"2024-04-25T09:04:28.370180+0000","last_became_active":"2024-04-25T09:04:25.210675+0000","last_became_peered":"2024-04-25T09:04:25.210675+0000","last_unstale":"2024-04-25T09:04:28.370180+0000","last_undegraded":"2024-04-25T09:04:28.370180+0000","last_fullsized":"2024-04-25T09:04:28.370180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369160+0000","last_change":"2024-04-25T09:04:26.232458+0000","last_active":"2024-04-25T09:04:28.369160+0000","last_peered":"2024-04-25T09:04:28.369160+0000","last_clean":"2024-04-25T09:04:28.369160+0000","last_became_active":"2024-04-25T09:04:26.232152+0000","last_became_peered":"2024-04-25T09:04:26.232152+0000","last_unstale":"2024-04-25T09:04:28.369160+0000","last_undegraded":"2024-04-25T09:04:28.369160+0000","last_fullsized":"2024-04-25T09:04:28.369160+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:30.510375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368619+0000","last_change":"2024-04-25T09:04:25.205276+0000","last_active":"2024-04-25T09:04:28.368619+0000","last_peered":"2024-04-25T09:04:28.368619+0000","last_clean":"2024-04-25T09:04:28.368619+0000","last_became_active":"2024-04-25T09:04:25.204962+0000","last_became_peered":"2024-04-25T09:04:25.204962+0000","last_unstale":"2024-04-25T09:04:28.368619+0000","last_undegraded":"2024-04-25T09:04:28.368619+0000","last_fullsized":"2024-04-25T09:04:28.368619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:38.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232923+0000","last_change":"2024-04-25T09:04:26.240130+0000","last_active":"2024-04-25T09:04:27.232923+0000","last_peered":"2024-04-25T09:04:27.232923+0000","last_clean":"2024-04-25T09:04:27.232923+0000","last_became_active":"2024-04-25T09:04:26.236257+0000","last_became_peered":"2024-04-25T09:04:26.236257+0000","last_unstale":"2024-04-25T09:04:27.232923+0000","last_undegraded":"2024-04-25T09:04:27.232923+0000","last_fullsized":"2024-04-25T09:04:27.232923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:42.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232385+0000","last_change":"2024-04-25T09:04:25.197723+0000","last_active":"2024-04-25T09:04:27.232385+0000","last_peered":"2024-04-25T09:04:27.232385+0000","last_clean":"2024-04-25T09:04:27.232385+0000","last_became_active":"2024-04-25T09:04:25.197275+0000","last_became_peered":"2024-04-25T09:04:25.197275+0000","last_unstale":"2024-04-25T09:04:27.232385+0000","last_undegraded":"2024-04-25T09:04:27.232385+0000","last_fullsized":"2024-04-25T09:04:27.232385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.834376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368340+0000","last_change":"2024-04-25T09:04:26.226130+0000","last_active":"2024-04-25T09:04:28.368340+0000","last_peered":"2024-04-25T09:04:28.368340+0000","last_clean":"2024-04-25T09:04:28.368340+0000","last_became_active":"2024-04-25T09:04:26.225532+0000","last_became_peered":"2024-04-25T09:04:26.225532+0000","last_unstale":"2024-04-25T09:04:28.368340+0000","last_undegraded":"2024-04-25T09:04:28.368340+0000","last_fullsized":"2024-04-25T09:04:28.368340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:30.946720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232108+0000","last_change":"2024-04-25T09:04:25.191706+0000","last_active":"2024-04-25T09:04:27.232108+0000","last_peered":"2024-04-25T09:04:27.232108+0000","last_clean":"2024-04-25T09:04:27.232108+0000","last_became_active":"2024-04-25T09:04:25.190455+0000","last_became_peered":"2024-04-25T09:04:25.190455+0000","last_unstale":"2024-04-25T09:04:27.232108+0000","last_undegraded":"2024-04-25T09:04:27.232108+0000","last_fullsized":"2024-04-25T09:04:27.232108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:18.342663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369381+0000","last_change":"2024-04-25T09:04:26.217025+0000","last_active":"2024-04-25T09:04:28.369381+0000","last_peered":"2024-04-25T09:04:28.369381+0000","last_clean":"2024-04-25T09:04:28.369381+0000","last_became_active":"2024-04-25T09:04:26.216110+0000","last_became_peered":"2024-04-25T09:04:26.216110+0000","last_unstale":"2024-04-25T09:04:28.369381+0000","last_undegraded":"2024-04-25T09:04:28.369381+0000","last_fullsized":"2024-04-25T09:04:28.369381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:19.936787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.211304+0000","last_change":"2024-04-25T09:04:25.203759+0000","last_active":"2024-04-25T09:04:28.211304+0000","last_peered":"2024-04-25T09:04:28.211304+0000","last_clean":"2024-04-25T09:04:28.211304+0000","last_became_active":"2024-04-25T09:04:25.203310+0000","last_became_peered":"2024-04-25T09:04:25.203310+0000","last_unstale":"2024-04-25T09:04:28.211304+0000","last_undegraded":"2024-04-25T09:04:28.211304+0000","last_fullsized":"2024-04-25T09:04:28.211304+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:27.537160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367763+0000","last_change":"2024-04-25T09:04:26.224528+0000","last_active":"2024-04-25T09:04:28.367763+0000","last_peered":"2024-04-25T09:04:28.367763+0000","last_clean":"2024-04-25T09:04:28.367763+0000","last_became_active":"2024-04-25T09:04:26.224221+0000","last_became_peered":"2024-04-25T09:04:26.224221+0000","last_unstale":"2024-04-25T09:04:28.367763+0000","last_undegraded":"2024-04-25T09:04:28.367763+0000","last_fullsized":"2024-04-25T09:04:28.367763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:27.543187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212375+0000","last_change":"2024-04-25T09:04:25.206521+0000","last_active":"2024-04-25T09:04:28.212375+0000","last_peered":"2024-04-25T09:04:28.212375+0000","last_clean":"2024-04-25T09:04:28.212375+0000","last_became_active":"2024-04-25T09:04:25.206196+0000","last_became_peered":"2024-04-25T09:04:25.206196+0000","last_unstale":"2024-04-25T09:04:28.212375+0000","last_undegraded":"2024-04-25T09:04:28.212375+0000","last_fullsized":"2024-04-25T09:04:28.212375+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:33.903475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369968+0000","last_change":"2024-04-25T09:04:26.229046+0000","last_active":"2024-04-25T09:04:28.369968+0000","last_peered":"2024-04-25T09:04:28.369968+0000","last_clean":"2024-04-25T09:04:28.369968+0000","last_became_active":"2024-04-25T09:04:26.228781+0000","last_became_peered":"2024-04-25T09:04:26.228781+0000","last_unstale":"2024-04-25T09:04:28.369968+0000","last_undegraded":"2024-04-25T09:04:28.369968+0000","last_fullsized":"2024-04-25T09:04:28.369968+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:39.649661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368830+0000","last_change":"2024-04-25T09:04:25.205066+0000","last_active":"2024-04-25T09:04:28.368830+0000","last_peered":"2024-04-25T09:04:28.368830+0000","last_clean":"2024-04-25T09:04:28.368830+0000","last_became_active":"2024-04-25T09:04:25.204741+0000","last_became_peered":"2024-04-25T09:04:25.204741+0000","last_unstale":"2024-04-25T09:04:28.368830+0000","last_undegraded":"2024-04-25T09:04:28.368830+0000","last_fullsized":"2024-04-25T09:04:28.368830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:18.342663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.075954+0000","last_change":"2024-04-25T09:04:26.236052+0000","last_active":"2024-04-25T09:04:28.075954+0000","last_peered":"2024-04-25T09:04:28.075954+0000","last_clean":"2024-04-25T09:04:28.075954+0000","last_became_active":"2024-04-25T09:04:26.234393+0000","last_became_peered":"2024-04-25T09:04:26.234393+0000","last_unstale":"2024-04-25T09:04:28.075954+0000","last_undegraded":"2024-04-25T09:04:28.075954+0000","last_fullsized":"2024-04-25T09:04:28.075954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.576671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367732+0000","last_change":"2024-04-25T09:04:25.203672+0000","last_active":"2024-04-25T09:04:28.367732+0000","last_peered":"2024-04-25T09:04:28.367732+0000","last_clean":"2024-04-25T09:04:28.367732+0000","last_became_active":"2024-04-25T09:04:25.203329+0000","last_became_peered":"2024-04-25T09:04:25.203329+0000","last_unstale":"2024-04-25T09:04:28.367732+0000","last_undegraded":"2024-04-25T09:04:28.367732+0000","last_fullsized":"2024-04-25T09:04:28.367732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:29.934288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212153+0000","last_change":"2024-04-25T09:04:26.241160+0000","last_active":"2024-04-25T09:04:28.212153+0000","last_peered":"2024-04-25T09:04:28.212153+0000","last_clean":"2024-04-25T09:04:28.212153+0000","last_became_active":"2024-04-25T09:04:26.240856+0000","last_became_peered":"2024-04-25T09:04:26.240856+0000","last_unstale":"2024-04-25T09:04:28.212153+0000","last_undegraded":"2024-04-25T09:04:28.212153+0000","last_fullsized":"2024-04-25T09:04:28.212153+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:36.846809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227286+0000","last_change":"2024-04-25T09:04:25.198618+0000","last_active":"2024-04-25T09:04:27.227286+0000","last_peered":"2024-04-25T09:04:27.227286+0000","last_clean":"2024-04-25T09:04:27.227286+0000","last_became_active":"2024-04-25T09:04:25.198320+0000","last_became_peered":"2024-04-25T09:04:25.198320+0000","last_unstale":"2024-04-25T09:04:27.227286+0000","last_undegraded":"2024-04-25T09:04:27.227286+0000","last_fullsized":"2024-04-25T09:04:27.227286+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:28.333679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369645+0000","last_change":"2024-04-25T09:04:26.239468+0000","last_active":"2024-04-25T09:04:28.369645+0000","last_peered":"2024-04-25T09:04:28.369645+0000","last_clean":"2024-04-25T09:04:28.369645+0000","last_became_active":"2024-04-25T09:04:26.237446+0000","last_became_peered":"2024-04-25T09:04:26.237446+0000","last_unstale":"2024-04-25T09:04:28.369645+0000","last_undegraded":"2024-04-25T09:04:28.369645+0000","last_fullsized":"2024-04-25T09:04:28.369645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:32.384380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076197+0000","last_change":"2024-04-25T09:04:25.206110+0000","last_active":"2024-04-25T09:04:28.076197+0000","last_peered":"2024-04-25T09:04:28.076197+0000","last_clean":"2024-04-25T09:04:28.076197+0000","last_became_active":"2024-04-25T09:04:25.205671+0000","last_became_peered":"2024-04-25T09:04:25.205671+0000","last_unstale":"2024-04-25T09:04:28.076197+0000","last_undegraded":"2024-04-25T09:04:28.076197+0000","last_fullsized":"2024-04-25T09:04:28.076197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:35.698063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.075890+0000","last_change":"2024-04-25T09:04:26.234944+0000","last_active":"2024-04-25T09:04:28.075890+0000","last_peered":"2024-04-25T09:04:28.075890+0000","last_clean":"2024-04-25T09:04:28.075890+0000","last_became_active":"2024-04-25T09:04:26.234649+0000","last_became_peered":"2024-04-25T09:04:26.234649+0000","last_unstale":"2024-04-25T09:04:28.075890+0000","last_undegraded":"2024-04-25T09:04:28.075890+0000","last_fullsized":"2024-04-25T09:04:28.075890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:51.170042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368239+0000","last_change":"2024-04-25T09:04:25.205857+0000","last_active":"2024-04-25T09:04:28.368239+0000","last_peered":"2024-04-25T09:04:28.368239+0000","last_clean":"2024-04-25T09:04:28.368239+0000","last_became_active":"2024-04-25T09:04:25.204814+0000","last_became_peered":"2024-04-25T09:04:25.204814+0000","last_unstale":"2024-04-25T09:04:28.368239+0000","last_undegraded":"2024-04-25T09:04:28.368239+0000","last_fullsized":"2024-04-25T09:04:28.368239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:35.834376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369041+0000","last_change":"2024-04-25T09:04:26.236022+0000","last_active":"2024-04-25T09:04:28.369041+0000","last_peered":"2024-04-25T09:04:28.369041+0000","last_clean":"2024-04-25T09:04:28.369041+0000","last_became_active":"2024-04-25T09:04:26.233069+0000","last_became_peered":"2024-04-25T09:04:26.233069+0000","last_unstale":"2024-04-25T09:04:28.369041+0000","last_undegraded":"2024-04-25T09:04:28.369041+0000","last_fullsized":"2024-04-25T09:04:28.369041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:17.441187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369502+0000","last_change":"2024-04-25T09:04:25.203994+0000","last_active":"2024-04-25T09:04:28.369502+0000","last_peered":"2024-04-25T09:04:28.369502+0000","last_clean":"2024-04-25T09:04:28.369502+0000","last_became_active":"2024-04-25T09:04:25.203665+0000","last_became_peered":"2024-04-25T09:04:25.203665+0000","last_unstale":"2024-04-25T09:04:28.369502+0000","last_undegraded":"2024-04-25T09:04:28.369502+0000","last_fullsized":"2024-04-25T09:04:28.369502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:31.510668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369363+0000","last_change":"2024-04-25T09:04:26.230309+0000","last_active":"2024-04-25T09:04:28.369363+0000","last_peered":"2024-04-25T09:04:28.369363+0000","last_clean":"2024-04-25T09:04:28.369363+0000","last_became_active":"2024-04-25T09:04:26.229935+0000","last_became_peered":"2024-04-25T09:04:26.229935+0000","last_unstale":"2024-04-25T09:04:28.369363+0000","last_undegraded":"2024-04-25T09:04:28.369363+0000","last_fullsized":"2024-04-25T09:04:28.369363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:47.487110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.367079+0000","last_change":"2024-04-25T09:04:25.201881+0000","last_active":"2024-04-25T09:04:28.367079+0000","last_peered":"2024-04-25T09:04:28.367079+0000","last_clean":"2024-04-25T09:04:28.367079+0000","last_became_active":"2024-04-25T09:04:25.201582+0000","last_became_peered":"2024-04-25T09:04:25.201582+0000","last_unstale":"2024-04-25T09:04:28.367079+0000","last_undegraded":"2024-04-25T09:04:28.367079+0000","last_fullsized":"2024-04-25T09:04:28.367079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:41.953829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212421+0000","last_change":"2024-04-25T09:04:26.241389+0000","last_active":"2024-04-25T09:04:28.212421+0000","last_peered":"2024-04-25T09:04:28.212421+0000","last_clean":"2024-04-25T09:04:28.212421+0000","last_became_active":"2024-04-25T09:04:26.241141+0000","last_became_peered":"2024-04-25T09:04:26.241141+0000","last_unstale":"2024-04-25T09:04:28.212421+0000","last_undegraded":"2024-04-25T09:04:28.212421+0000","last_fullsized":"2024-04-25T09:04:28.212421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:22.765586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369451+0000","last_change":"2024-04-25T09:04:25.201894+0000","last_active":"2024-04-25T09:04:28.369451+0000","last_peered":"2024-04-25T09:04:28.369451+0000","last_clean":"2024-04-25T09:04:28.369451+0000","last_became_active":"2024-04-25T09:04:25.201583+0000","last_became_peered":"2024-04-25T09:04:25.201583+0000","last_unstale":"2024-04-25T09:04:28.369451+0000","last_undegraded":"2024-04-25T09:04:28.369451+0000","last_fullsized":"2024-04-25T09:04:28.369451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:57.417759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370974+0000","last_change":"2024-04-25T09:04:26.228528+0000","last_active":"2024-04-25T09:04:28.370974+0000","last_peered":"2024-04-25T09:04:28.370974+0000","last_clean":"2024-04-25T09:04:28.370974+0000","last_became_active":"2024-04-25T09:04:26.227833+0000","last_became_peered":"2024-04-25T09:04:26.227833+0000","last_unstale":"2024-04-25T09:04:28.370974+0000","last_undegraded":"2024-04-25T09:04:28.370974+0000","last_fullsized":"2024-04-25T09:04:28.370974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:21.229782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.076447+0000","last_change":"2024-04-25T09:04:25.207893+0000","last_active":"2024-04-25T09:04:28.076447+0000","last_peered":"2024-04-25T09:04:28.076447+0000","last_clean":"2024-04-25T09:04:28.076447+0000","last_became_active":"2024-04-25T09:04:25.207587+0000","last_became_peered":"2024-04-25T09:04:25.207587+0000","last_unstale":"2024-04-25T09:04:28.076447+0000","last_undegraded":"2024-04-25T09:04:28.076447+0000","last_fullsized":"2024-04-25T09:04:28.076447+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:27.562675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.212473+0000","last_change":"2024-04-25T09:04:26.241946+0000","last_active":"2024-04-25T09:04:28.212473+0000","last_peered":"2024-04-25T09:04:28.212473+0000","last_clean":"2024-04-25T09:04:28.212473+0000","last_became_active":"2024-04-25T09:04:26.241637+0000","last_became_peered":"2024-04-25T09:04:26.241637+0000","last_unstale":"2024-04-25T09:04:28.212473+0000","last_undegraded":"2024-04-25T09:04:28.212473+0000","last_fullsized":"2024-04-25T09:04:28.212473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.061981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370214+0000","last_change":"2024-04-25T09:04:25.206786+0000","last_active":"2024-04-25T09:04:28.370214+0000","last_peered":"2024-04-25T09:04:28.370214+0000","last_clean":"2024-04-25T09:04:28.370214+0000","last_became_active":"2024-04-25T09:04:25.206460+0000","last_became_peered":"2024-04-25T09:04:25.206460+0000","last_unstale":"2024-04-25T09:04:28.370214+0000","last_undegraded":"2024-04-25T09:04:28.370214+0000","last_fullsized":"2024-04-25T09:04:28.370214+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:38.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.226938+0000","last_change":"2024-04-25T09:04:26.238638+0000","last_active":"2024-04-25T09:04:27.226938+0000","last_peered":"2024-04-25T09:04:27.226938+0000","last_clean":"2024-04-25T09:04:27.226938+0000","last_became_active":"2024-04-25T09:04:26.238351+0000","last_became_peered":"2024-04-25T09:04:26.238351+0000","last_unstale":"2024-04-25T09:04:27.226938+0000","last_undegraded":"2024-04-25T09:04:27.226938+0000","last_fullsized":"2024-04-25T09:04:27.226938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:21.190794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369651+0000","last_change":"2024-04-25T09:04:25.203588+0000","last_active":"2024-04-25T09:04:28.369651+0000","last_peered":"2024-04-25T09:04:28.369651+0000","last_clean":"2024-04-25T09:04:28.369651+0000","last_became_active":"2024-04-25T09:04:25.203279+0000","last_became_peered":"2024-04-25T09:04:25.203279+0000","last_unstale":"2024-04-25T09:04:28.369651+0000","last_undegraded":"2024-04-25T09:04:28.369651+0000","last_fullsized":"2024-04-25T09:04:28.369651+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:18.342663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.233119+0000","last_change":"2024-04-25T09:04:26.237961+0000","last_active":"2024-04-25T09:04:27.233119+0000","last_peered":"2024-04-25T09:04:27.233119+0000","last_clean":"2024-04-25T09:04:27.233119+0000","last_became_active":"2024-04-25T09:04:26.235818+0000","last_became_peered":"2024-04-25T09:04:26.235818+0000","last_unstale":"2024-04-25T09:04:27.233119+0000","last_undegraded":"2024-04-25T09:04:27.233119+0000","last_fullsized":"2024-04-25T09:04:27.233119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:30.946720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369998+0000","last_change":"2024-04-25T09:04:25.205414+0000","last_active":"2024-04-25T09:04:28.369998+0000","last_peered":"2024-04-25T09:04:28.369998+0000","last_clean":"2024-04-25T09:04:28.369998+0000","last_became_active":"2024-04-25T09:04:25.205062+0000","last_became_peered":"2024-04-25T09:04:25.205062+0000","last_unstale":"2024-04-25T09:04:28.369998+0000","last_undegraded":"2024-04-25T09:04:28.369998+0000","last_fullsized":"2024-04-25T09:04:28.369998+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:43.725847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232033+0000","last_change":"2024-04-25T09:04:26.221218+0000","last_active":"2024-04-25T09:04:27.232033+0000","last_peered":"2024-04-25T09:04:27.232033+0000","last_clean":"2024-04-25T09:04:27.232033+0000","last_became_active":"2024-04-25T09:04:26.217960+0000","last_became_peered":"2024-04-25T09:04:26.217960+0000","last_unstale":"2024-04-25T09:04:27.232033+0000","last_undegraded":"2024-04-25T09:04:27.232033+0000","last_fullsized":"2024-04-25T09:04:27.232033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.576671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368022+0000","last_change":"2024-04-25T09:04:25.207990+0000","last_active":"2024-04-25T09:04:28.368022+0000","last_peered":"2024-04-25T09:04:28.368022+0000","last_clean":"2024-04-25T09:04:28.368022+0000","last_became_active":"2024-04-25T09:04:25.207576+0000","last_became_peered":"2024-04-25T09:04:25.207576+0000","last_unstale":"2024-04-25T09:04:28.368022+0000","last_undegraded":"2024-04-25T09:04:28.368022+0000","last_fullsized":"2024-04-25T09:04:28.368022+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:46.474678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.369643+0000","last_change":"2024-04-25T09:04:26.233431+0000","last_active":"2024-04-25T09:04:28.369643+0000","last_peered":"2024-04-25T09:04:28.369643+0000","last_clean":"2024-04-25T09:04:28.369643+0000","last_became_active":"2024-04-25T09:04:26.233040+0000","last_became_peered":"2024-04-25T09:04:26.233040+0000","last_unstale":"2024-04-25T09:04:28.369643+0000","last_undegraded":"2024-04-25T09:04:28.369643+0000","last_fullsized":"2024-04-25T09:04:28.369643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:25.409619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.232045+0000","last_change":"2024-04-25T09:04:25.198844+0000","last_active":"2024-04-25T09:04:27.232045+0000","last_peered":"2024-04-25T09:04:27.232045+0000","last_clean":"2024-04-25T09:04:27.232045+0000","last_became_active":"2024-04-25T09:04:25.198376+0000","last_became_peered":"2024-04-25T09:04:25.198376+0000","last_unstale":"2024-04-25T09:04:27.232045+0000","last_undegraded":"2024-04-25T09:04:27.232045+0000","last_fullsized":"2024-04-25T09:04:27.232045+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:38.637229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:27.227542+0000","last_change":"2024-04-25T09:04:26.241339+0000","last_active":"2024-04-25T09:04:27.227542+0000","last_peered":"2024-04-25T09:04:27.227542+0000","last_clean":"2024-04-25T09:04:27.227542+0000","last_became_active":"2024-04-25T09:04:26.241051+0000","last_became_peered":"2024-04-25T09:04:26.241051+0000","last_unstale":"2024-04-25T09:04:27.227542+0000","last_undegraded":"2024-04-25T09:04:27.227542+0000","last_fullsized":"2024-04-25T09:04:27.227542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:25.160001+0000","last_clean_scrub_stamp":"2024-04-25T09:04:25.160001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:34.911589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.368027+0000","last_change":"2024-04-25T09:04:25.202687+0000","last_active":"2024-04-25T09:04:28.368027+0000","last_peered":"2024-04-25T09:04:28.368027+0000","last_clean":"2024-04-25T09:04:28.368027+0000","last_became_active":"2024-04-25T09:04:25.202400+0000","last_became_peered":"2024-04-25T09:04:25.202400+0000","last_unstale":"2024-04-25T09:04:28.368027+0000","last_undegraded":"2024-04-25T09:04:28.368027+0000","last_fullsized":"2024-04-25T09:04:28.368027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:24.147569+0000","last_clean_scrub_stamp":"2024-04-25T09:04:24.147569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:02.418007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T09:04:28.370931+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:04:28.370931+0000","last_peered":"2024-04-25T09:04:28.370931+0000","last_clean":"2024-04-25T09:04:28.370931+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:04:28.370931+0000","last_undegraded":"2024-04-25T09:04:28.370931+0000","last_fullsized":"2024-04-25T09:04:28.370931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":147734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":147756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":146898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":736560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:36.177 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:04:36.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:04:36.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.452+0000 7f7337200640 1 Processor -- start 2024-04-25T09:04:36.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.453+0000 7f7337200640 1 -- start start 2024-04-25T09:04:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.454+0000 7f7337200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f73380cc960 0x7f73380b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.454+0000 7f7337200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f73380b0eb0 0x7f73381656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.454+0000 7f7337200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733806d490 con 0x7f7338159680 2024-04-25T09:04:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.454+0000 7f7337200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733806d000 con 0x7f73380cc960 2024-04-25T09:04:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.454+0000 7f7337200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733806c3f0 con 0x7f73380b0eb0 2024-04-25T09:04:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f7335e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7338159680 0x7f733815cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42574/0 (socket says 172.21.15.62:42574) 2024-04-25T09:04:36.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f7335e00640 1 -- 172.21.15.62:0/3531293131 learned_addr learned my addr 172.21.15.62:0/3531293131 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:36.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f732fe00640 1 --2- 172.21.15.62:0/3531293131 >> v2:172.21.15.62:3300/0 conn(0x7f73380cc960 0x7f73380b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f7336800640 1 --2- 172.21.15.62:0/3531293131 >> v2:172.21.15.121:3300/0 conn(0x7f73380b0eb0 0x7f73381656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f7335400640 1 -- 172.21.15.62:0/3531293131 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581988975 0 0) 0x7f733806d490 con 0x7f7338159680 2024-04-25T09:04:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f7335400640 1 -- 172.21.15.62:0/3531293131 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f731c003520 con 0x7f7338159680 2024-04-25T09:04:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f732fe00640 1 -- 172.21.15.62:0/3531293131 >> v2:172.21.15.121:3300/0 conn(0x7f73380b0eb0 msgr2=0x7f73381656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f732fe00640 1 --2- 172.21.15.62:0/3531293131 >> v2:172.21.15.121:3300/0 conn(0x7f73380b0eb0 0x7f73381656f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f732fe00640 1 -- 172.21.15.62:0/3531293131 >> v1:172.21.15.62:6789/0 conn(0x7f7338159680 legacy=0x7f733815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.455+0000 7f732fe00640 1 -- 172.21.15.62:0/3531293131 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733806c6d0 con 0x7f73380cc960 2024-04-25T09:04:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.456+0000 7f732fe00640 1 --2- 172.21.15.62:0/3531293131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73380cc960 0x7f73380b0970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f733815c0d0 tx=0x7f732002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb2cb4b9fb923b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.457+0000 7f7335400640 1 -- 172.21.15.62:0/3531293131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320037070 con 0x7f73380cc960 2024-04-25T09:04:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.457+0000 7f7335400640 1 -- 172.21.15.62:0/3531293131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7320032440 con 0x7f73380cc960 2024-04-25T09:04:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.457+0000 7f7335400640 1 -- 172.21.15.62:0/3531293131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320032810 con 0x7f73380cc960 2024-04-25T09:04:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.458+0000 7f7337200640 1 -- 172.21.15.62:0/3531293131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73380cc960 msgr2=0x7f73380b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.458+0000 7f7337200640 1 --2- 172.21.15.62:0/3531293131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73380cc960 0x7f73380b0970 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f733815c0d0 tx=0x7f732002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.458+0000 7f7337200640 1 -- 172.21.15.62:0/3531293131 shutdown_connections 2024-04-25T09:04:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.458+0000 7f7337200640 1 --2- 172.21.15.62:0/3531293131 >> v2:172.21.15.121:3300/0 conn(0x7f73380b0eb0 0x7f73381656f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.458+0000 7f7337200640 1 --2- 172.21.15.62:0/3531293131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73380cc960 0x7f73380b0970 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.458+0000 7f7337200640 1 -- 172.21.15.62:0/3531293131 >> 172.21.15.62:0/3531293131 conn(0x7f73380668f0 msgr2=0x7f7338066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.458+0000 7f7337200640 1 -- 172.21.15.62:0/3531293131 shutdown_connections 2024-04-25T09:04:36.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.458+0000 7f7337200640 1 -- 172.21.15.62:0/3531293131 wait complete. 2024-04-25T09:04:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.460+0000 7f7337200640 1 Processor -- start 2024-04-25T09:04:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.461+0000 7f7337200640 1 -- start start 2024-04-25T09:04:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.462+0000 7f7337200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73380b0eb0 0x7f73381f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.462+0000 7f7337200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73380cc960 0x7f73380cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.462+0000 7f7337200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7338159680 0x7f73380b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.462+0000 7f7337200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7338159a60 con 0x7f7338159680 2024-04-25T09:04:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.462+0000 7f7337200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733806cc90 con 0x7f73380cc960 2024-04-25T09:04:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.462+0000 7f7337200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733815e2a0 con 0x7f73380b0eb0 2024-04-25T09:04:36.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.462+0000 7f7336800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7338159680 0x7f73380b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7336800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7338159680 0x7f73380b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39762/0 (socket says 172.21.15.62:39762) 2024-04-25T09:04:36.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7336800640 1 -- 172.21.15.62:0/1219270376 learned_addr learned my addr 172.21.15.62:0/1219270376 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7335e00640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73380b0eb0 0x7f73381f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f732fe00640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73380cc960 0x7f73380cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7336800640 1 -- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73380b0eb0 msgr2=0x7f73381f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7336800640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73380b0eb0 0x7f73381f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7336800640 1 -- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73380cc960 msgr2=0x7f73380cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7336800640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73380cc960 0x7f73380cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7336800640 1 -- 172.21.15.62:0/1219270376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73381f6690 con 0x7f7338159680 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.463+0000 7f7336800640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7338159680 0x7f73380b3a00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7330007e40 tx=0x7f733000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.464+0000 7f732e000640 1 -- 172.21.15.62:0/1219270376 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330014620 con 0x7f7338159680 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.464+0000 7f732e000640 1 -- 172.21.15.62:0/1219270376 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7330019070 con 0x7f7338159680 2024-04-25T09:04:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.464+0000 7f732e000640 1 -- 172.21.15.62:0/1219270376 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330014a20 con 0x7f7338159680 2024-04-25T09:04:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.464+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73380b2960 con 0x7f7338159680 2024-04-25T09:04:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.464+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73380b2f20 con 0x7f7338159680 2024-04-25T09:04:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.465+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7338075070 con 0x7f7338159680 2024-04-25T09:04:36.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.466+0000 7f732e000640 1 -- 172.21.15.62:0/1219270376 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7330014bc0 con 0x7f7338159680 2024-04-25T09:04:36.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.467+0000 7f732e000640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.467+0000 7f732e000640 1 -- 172.21.15.62:0/1219270376 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f733008aa40 con 0x7f7338159680 2024-04-25T09:04:36.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.468+0000 7f7335e00640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.469+0000 7f7335e00640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f731406b4d0 0x7f731406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7328000f60 tx=0x7f7328009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.472+0000 7f732e000640 1 -- 172.21.15.62:0/1219270376 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7330059950 con 0x7f7338159680 2024-04-25T09:04:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.741+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f733806b210 con 0x7f7338159680 2024-04-25T09:04:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.747+0000 7f732e000640 1 -- 172.21.15.62:0/1219270376 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f733008ae20 con 0x7f7338159680 2024-04-25T09:04:36.735 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:04:36.735 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:04:36.735 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:04:36.735 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:04:36.735 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:04:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:04:36.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.749+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f731406b4d0 msgr2=0x7f731406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.749+0000 7f7337200640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f731406b4d0 0x7f731406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7328000f60 tx=0x7f7328009040 comp rx=0 tx=0).stop 2024-04-25T09:04:36.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.749+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7338159680 msgr2=0x7f73380b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.749+0000 7f7337200640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7338159680 0x7f73380b3a00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7330007e40 tx=0x7f733000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.750+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 shutdown_connections 2024-04-25T09:04:36.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.750+0000 7f7337200640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.750+0000 7f7337200640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7338159680 0x7f73380b3a00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.750+0000 7f7337200640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73380cc960 0x7f73380cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.750+0000 7f7337200640 1 --2- 172.21.15.62:0/1219270376 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73380b0eb0 0x7f73381f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.750+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 >> 172.21.15.62:0/1219270376 conn(0x7f73380668f0 msgr2=0x7f73381588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.750+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 shutdown_connections 2024-04-25T09:04:36.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.750+0000 7f7337200640 1 -- 172.21.15.62:0/1219270376 wait complete. 2024-04-25T09:04:36.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:36.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.977+0000 7fa506600640 1 Processor -- start 2024-04-25T09:04:36.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.977+0000 7fa506600640 1 -- start start 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.978+0000 7fa506600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 0x7fa500160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.978+0000 7fa506600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa5000b27a0 0x7fa500161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.978+0000 7fa506600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5000d3110 con 0x7fa5000b6a90 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.978+0000 7fa506600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa500161780 con 0x7fa5000b27a0 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa506600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000cedc0 con 0x7fa5000b56f0 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa505c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa5000b27a0 0x7fa500161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa505c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa5000b27a0 0x7fa500161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39768/0 (socket says 172.21.15.62:39768) 2024-04-25T09:04:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa505c00640 1 -- 172.21.15.62:0/2563223609 learned_addr learned my addr 172.21.15.62:0/2563223609 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:36.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa505200640 1 --2- 172.21.15.62:0/2563223609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 0x7fa500160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa505c00640 1 -- 172.21.15.62:0/2563223609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 msgr2=0x7fa500160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa505c00640 1 --2- 172.21.15.62:0/2563223609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 0x7fa500160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa505c00640 1 -- 172.21.15.62:0/2563223609 >> v1:172.21.15.121:6789/0 conn(0x7fa5000b6a90 legacy=0x7fa500161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.979+0000 7fa505c00640 1 -- 172.21.15.62:0/2563223609 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000d33f0 con 0x7fa5000b27a0 2024-04-25T09:04:36.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.980+0000 7fa505c00640 1 --2- 172.21.15.62:0/2563223609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa500161240 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009e50 tx=0x7fa4f802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2043a21bef8192d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.981+0000 7fa4ff400640 1 -- 172.21.15.62:0/2563223609 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8036070 con 0x7fa5000b27a0 2024-04-25T09:04:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.981+0000 7fa4ff400640 1 -- 172.21.15.62:0/2563223609 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f8031600 con 0x7fa5000b27a0 2024-04-25T09:04:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.981+0000 7fa4ff400640 1 -- 172.21.15.62:0/2563223609 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80319d0 con 0x7fa5000b27a0 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.981+0000 7fa506600640 1 -- 172.21.15.62:0/2563223609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 msgr2=0x7fa500161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.981+0000 7fa506600640 1 --2- 172.21.15.62:0/2563223609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa500161240 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009e50 tx=0x7fa4f802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.982+0000 7fa506600640 1 -- 172.21.15.62:0/2563223609 shutdown_connections 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.982+0000 7fa506600640 1 --2- 172.21.15.62:0/2563223609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa500161240 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.982+0000 7fa506600640 1 --2- 172.21.15.62:0/2563223609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 0x7fa500160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.982+0000 7fa506600640 1 -- 172.21.15.62:0/2563223609 >> 172.21.15.62:0/2563223609 conn(0x7fa500066d10 msgr2=0x7fa500067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.982+0000 7fa506600640 1 -- 172.21.15.62:0/2563223609 shutdown_connections 2024-04-25T09:04:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.982+0000 7fa506600640 1 -- 172.21.15.62:0/2563223609 wait complete. 2024-04-25T09:04:36.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.983+0000 7fa506600640 1 Processor -- start 2024-04-25T09:04:36.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.984+0000 7fa506600640 1 -- start start 2024-04-25T09:04:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa506600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa50014dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa506600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 0x7fa50015ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa506600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5000b6a90 0x7fa50015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa506600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000b2d60 con 0x7fa5000b27a0 2024-04-25T09:04:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa506600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000cea50 con 0x7fa5000b6a90 2024-04-25T09:04:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa506600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000b2ee0 con 0x7fa5000b56f0 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa50014dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa50014dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39782/0 (socket says 172.21.15.62:39782) 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505200640 1 -- 172.21.15.62:0/1803836417 learned_addr learned my addr 172.21.15.62:0/1803836417 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa4ffe00640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 0x7fa50015ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505c00640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5000b6a90 0x7fa50015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505200640 1 -- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 msgr2=0x7fa50015ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505200640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 0x7fa50015ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505200640 1 -- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5000b6a90 msgr2=0x7fa50015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505200640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5000b6a90 0x7fa50015f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.985+0000 7fa505200640 1 -- 172.21.15.62:0/1803836417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000cc890 con 0x7fa5000b27a0 2024-04-25T09:04:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.986+0000 7fa505200640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa50014dd10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c7f0 tx=0x7fa4e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.986+0000 7fa4fcc00640 1 -- 172.21.15.62:0/1803836417 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8015070 con 0x7fa5000b27a0 2024-04-25T09:04:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.986+0000 7fa4fcc00640 1 -- 172.21.15.62:0/1803836417 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8004240 con 0x7fa5000b27a0 2024-04-25T09:04:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.986+0000 7fa4fcc00640 1 -- 172.21.15.62:0/1803836417 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8003610 con 0x7fa5000b27a0 2024-04-25T09:04:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.986+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5000ccb70 con 0x7fa5000b27a0 2024-04-25T09:04:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.986+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5000cd130 con 0x7fa5000b27a0 2024-04-25T09:04:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.988+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50004ef20 con 0x7fa5000b27a0 2024-04-25T09:04:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.989+0000 7fa4fcc00640 1 -- 172.21.15.62:0/1803836417 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4e80039c0 con 0x7fa5000b27a0 2024-04-25T09:04:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.990+0000 7fa4fcc00640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4dc06b500 0x7fa4dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.990+0000 7fa4fcc00640 1 -- 172.21.15.62:0/1803836417 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4e808a2a0 con 0x7fa5000b27a0 2024-04-25T09:04:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.991+0000 7fa4ffe00640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4dc06b500 0x7fa4dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.992+0000 7fa4ffe00640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4dc06b500 0x7fa4dc06d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0006fd0 tx=0x7fa4f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:36.995+0000 7fa4fcc00640 1 -- 172.21.15.62:0/1803836417 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4e8059110 con 0x7fa5000b27a0 2024-04-25T09:04:37.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.237+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5000ce580 con 0x7fa5000b27a0 2024-04-25T09:04:37.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.239+0000 7fa4fcc00640 1 -- 172.21.15.62:0/1803836417 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fa4e8058a10 con 0x7fa5000b27a0 2024-04-25T09:04:37.227 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:04:37.227 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2179586213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2179586213},{"type":"v1","addr":"172.21.15.121:6836","nonce":2179586213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1425308084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1425308084},{"type":"v1","addr":"172.21.15.121:6837","nonce":1425308084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/231681595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":231681595},{"type":"v1","addr":"172.21.15.62:6834","nonce":231681595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:25.639815+0000","modified":"2024-04-25T09:04:28.736856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/274650142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":274650142},{"type":"v1","addr":"172.21.15.62:6835","nonce":274650142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:04:37.227 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 16 2024-04-25T09:04:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4dc06b500 msgr2=0x7fa4dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4dc06b500 0x7fa4dc06d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0006fd0 tx=0x7fa4f0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 msgr2=0x7fa50014dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa50014dd10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c7f0 tx=0x7fa4e800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 shutdown_connections 2024-04-25T09:04:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4dc06b500 0x7fa4dc06d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa5000b6a90 0x7fa50015f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5000b56f0 0x7fa50015ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 --2- 172.21.15.62:0/1803836417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5000b27a0 0x7fa50014dd10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 >> 172.21.15.62:0/1803836417 conn(0x7fa500066d10 msgr2=0x7fa5000b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 shutdown_connections 2024-04-25T09:04:37.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.242+0000 7fa506600640 1 -- 172.21.15.62:0/1803836417 wait complete. 2024-04-25T09:04:37.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:04:25.639815+0000', 'modified': '2024-04-25T09:04:28.736856+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5043}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5043': {'gid': 5043, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6835/274650142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 274650142}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 274650142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:37.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:04:37.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:04:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.890+0000 7f1b07600640 1 Processor -- start 2024-04-25T09:04:37.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.890+0000 7f1b07600640 1 -- start start 2024-04-25T09:04:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.892+0000 7f1b07600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1b0015bd10 0x7f1b000cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.892+0000 7f1b07600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b000aeed0 0x7f1b000cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.892+0000 7f1b07600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0006ca70 con 0x7f1b000cccf0 2024-04-25T09:04:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.892+0000 7f1b07600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0006d660 con 0x7f1b0015bd10 2024-04-25T09:04:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.892+0000 7f1b07600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0015c0f0 con 0x7f1b000aeed0 2024-04-25T09:04:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.892+0000 7f1b04c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1b0015bd10 0x7f1b000cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.892+0000 7f1b04c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1b0015bd10 0x7f1b000cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39792/0 (socket says 172.21.15.62:39792) 2024-04-25T09:04:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.892+0000 7f1b04c00640 1 -- 172.21.15.62:0/3389151720 learned_addr learned my addr 172.21.15.62:0/3389151720 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.893+0000 7f1b04c00640 1 -- 172.21.15.62:0/3389151720 >> v2:172.21.15.121:3300/0 conn(0x7f1b000aeed0 msgr2=0x7f1b000cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.893+0000 7f1b05600640 1 --2- 172.21.15.62:0/3389151720 >> v2:172.21.15.121:3300/0 conn(0x7f1b000aeed0 0x7f1b000cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.893+0000 7f1b04c00640 1 --2- 172.21.15.62:0/3389151720 >> v2:172.21.15.121:3300/0 conn(0x7f1b000aeed0 0x7f1b000cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.893+0000 7f1b04c00640 1 -- 172.21.15.62:0/3389151720 >> v1:172.21.15.121:6789/0 conn(0x7f1b000cccf0 legacy=0x7f1b000ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.893+0000 7f1b04c00640 1 -- 172.21.15.62:0/3389151720 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0006cd30 con 0x7f1b0015bd10 2024-04-25T09:04:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.894+0000 7f1b04c00640 1 --2- 172.21.15.62:0/3389151720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000cfca0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1b000726c0 tx=0x7f1af002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49cd410b5dd6a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.894+0000 7f1aff400640 1 -- 172.21.15.62:0/3389151720 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0036070 con 0x7f1b0015bd10 2024-04-25T09:04:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.895+0000 7f1aff400640 1 -- 172.21.15.62:0/3389151720 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af0031600 con 0x7f1b0015bd10 2024-04-25T09:04:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.895+0000 7f1aff400640 1 -- 172.21.15.62:0/3389151720 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af00319d0 con 0x7f1b0015bd10 2024-04-25T09:04:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.895+0000 7f1b07600640 1 -- 172.21.15.62:0/3389151720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 msgr2=0x7f1b000cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.895+0000 7f1b07600640 1 --2- 172.21.15.62:0/3389151720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000cfca0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1b000726c0 tx=0x7f1af002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.895+0000 7f1b07600640 1 -- 172.21.15.62:0/3389151720 shutdown_connections 2024-04-25T09:04:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.895+0000 7f1b07600640 1 --2- 172.21.15.62:0/3389151720 >> v2:172.21.15.121:3300/0 conn(0x7f1b000aeed0 0x7f1b000cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.895+0000 7f1b07600640 1 --2- 172.21.15.62:0/3389151720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000cfca0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.895+0000 7f1b07600640 1 -- 172.21.15.62:0/3389151720 >> 172.21.15.62:0/3389151720 conn(0x7f1b000668f0 msgr2=0x7f1b00066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.896+0000 7f1b07600640 1 -- 172.21.15.62:0/3389151720 shutdown_connections 2024-04-25T09:04:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.896+0000 7f1b07600640 1 -- 172.21.15.62:0/3389151720 wait complete. 2024-04-25T09:04:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.898+0000 7f1b07600640 1 Processor -- start 2024-04-25T09:04:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.898+0000 7f1b07600640 1 -- start start 2024-04-25T09:04:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.899+0000 7f1b07600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b000aeed0 0x7f1b0014d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.899+0000 7f1b07600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b000cccf0 0x7f1b000b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.899+0000 7f1b07600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.899+0000 7f1b07600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b000d01e0 con 0x7f1b0015bd10 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.899+0000 7f1b07600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0006d2f0 con 0x7f1b000aeed0 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.899+0000 7f1b07600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0006daf0 con 0x7f1b000cccf0 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.899+0000 7f1b05600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b05600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39800/0 (socket says 172.21.15.62:39800) 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b05600640 1 -- 172.21.15.62:0/1771285540 learned_addr learned my addr 172.21.15.62:0/1771285540 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b04c00640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b000aeed0 0x7f1b0014d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1affe00640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b000cccf0 0x7f1b000b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b05600640 1 -- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b000cccf0 msgr2=0x7f1b000b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b05600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b000cccf0 0x7f1b000b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b05600640 1 -- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b000aeed0 msgr2=0x7f1b0014d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b05600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b000aeed0 0x7f1b0014d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b05600640 1 -- 172.21.15.62:0/1771285540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b000b2230 con 0x7f1b0015bd10 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.900+0000 7f1b05600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000ad8e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c7f0 tx=0x7f1af800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.901+0000 7f1afcc00640 1 -- 172.21.15.62:0/1771285540 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8015070 con 0x7f1b0015bd10 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.901+0000 7f1afcc00640 1 -- 172.21.15.62:0/1771285540 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af8004220 con 0x7f1b0015bd10 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.901+0000 7f1afcc00640 1 -- 172.21.15.62:0/1771285540 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8007620 con 0x7f1b0015bd10 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.901+0000 7f1b07600640 1 -- 172.21.15.62:0/1771285540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b00070450 con 0x7f1b0015bd10 2024-04-25T09:04:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.901+0000 7f1b07600640 1 -- 172.21.15.62:0/1771285540 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b00070a10 con 0x7f1b0015bd10 2024-04-25T09:04:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.903+0000 7f1afcc00640 1 -- 172.21.15.62:0/1771285540 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1af80079d0 con 0x7f1b0015bd10 2024-04-25T09:04:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.905+0000 7f1afcc00640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1adc06b4d0 0x7f1adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.905+0000 7f1afcc00640 1 -- 172.21.15.62:0/1771285540 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1af808a2e0 con 0x7f1b0015bd10 2024-04-25T09:04:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.905+0000 7f1b0a3e2740 1 Processor -- start 2024-04-25T09:04:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.905+0000 7f1b0a3e2740 1 -- start start 2024-04-25T09:04:37.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.905+0000 7f1b04c00640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1adc06b4d0 0x7f1adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893647950 0x55789366f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x557893647d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 0x557893646110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578935ab620 con 0x55789366fcd0 2024-04-25T09:04:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557893554ca0 con 0x5578936905c0 2024-04-25T09:04:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578935a76c0 con 0x557893647950 2024-04-25T09:04:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1affe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x557893647d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1affe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x557893647d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39802/0 (socket says 172.21.15.62:39802) 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.906+0000 7f1affe00640 1 -- 172.21.15.62:0/3135719438 learned_addr learned my addr 172.21.15.62:0/3135719438 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1b04c00640 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893647950 0x55789366f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1b05600640 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 0x557893646110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1b04c00640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1adc06b4d0 0x7f1adc06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1af0033000 tx=0x7f1af0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1affe00640 1 -- 172.21.15.62:0/3135719438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893647950 msgr2=0x55789366f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1affe00640 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893647950 0x55789366f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1affe00640 1 -- 172.21.15.62:0/3135719438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 msgr2=0x557893646110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1affe00640 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 0x557893646110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1affe00640 1 -- 172.21.15.62:0/3135719438 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55789355bf40 con 0x55789366fcd0 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1affe00640 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x557893647d30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8001580 tx=0x7f1ae8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bd5f5848888f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.907+0000 7f1af5200640 1 -- 172.21.15.62:0/3135719438 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8019070 con 0x55789366fcd0 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1af5200640 1 -- 172.21.15.62:0/3135719438 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae80050d0 con 0x55789366fcd0 2024-04-25T09:04:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1af5200640 1 -- 172.21.15.62:0/3135719438 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800e740 con 0x55789366fcd0 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3135719438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 msgr2=0x557893647d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x557893647d30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8001580 tx=0x7f1ae8001a40 comp rx=0 tx=0).stop 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3135719438 shutdown_connections 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x557893647d30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 0x557893646110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/3135719438 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893647950 0x55789366f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3135719438 >> 172.21.15.62:0/3135719438 conn(0x557893644b70 msgr2=0x55789368d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3135719438 shutdown_connections 2024-04-25T09:04:37.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.908+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3135719438 wait complete. 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.909+0000 7f1b0a3e2740 1 -- start start 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.909+0000 7f1b0a3e2740 1 Processor -- start 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.910+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789366fcd0 0x55789368f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.910+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578936905c0 0x557893655440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.910+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557893655980 0x557893666330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.910+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789366f300 con 0x557893655980 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.910+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789354a1f0 con 0x55789366fcd0 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.910+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55789366ecf0 con 0x5578936905c0 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.910+0000 7f1b0a3e2740 10 client.0 _ll_register_callbacks cb 0x557893567ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557893655980 0x557893666330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557893655980 0x557893666330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39808/0 (socket says 172.21.15.62:39808) 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 -- 172.21.15.62:0/2664591769 learned_addr learned my addr 172.21.15.62:0/2664591769 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b04c00640 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789366fcd0 0x55789368f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 -- 172.21.15.62:0/2664591769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578936905c0 msgr2=0x557893655440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578936905c0 0x557893655440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 -- 172.21.15.62:0/2664591769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789366fcd0 msgr2=0x55789368f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789366fcd0 0x55789368f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 -- 172.21.15.62:0/2664591769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557893569c50 con 0x557893655980 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1b05600640 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557893655980 0x557893666330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1af800afd0 tx=0x7f1af8058b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08ba13dbbe27aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1aed600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.911+0000 7f1aed600640 1 -- 172.21.15.62:0/2664591769 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af801e070 con 0x557893655980 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1aed600640 1 -- 172.21.15.62:0/2664591769 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1af8007cc0 con 0x557893655980 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/2664591769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ae80050d0 con 0x557893655980 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1aed600640 1 -- 172.21.15.62:0/2664591769 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1af8088b90 con 0x557893655980 2024-04-25T09:04:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 10 client.5238 fetch_fsmap learned FSMap version 16 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/2664591769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5578932c02d0 con 0x557893655980 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1aed600640 1 -- 172.21.15.62:0/2664591769 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae80050d0 con 0x557893655980 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1aed600640 1 -- 172.21.15.62:0/2664591769 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1af8057ea0 con 0x557893655980 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 10 client.5238 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 10 client.5238 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 20 client.5238 populate_metadata read hostname 'smithi062' 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557893666a70 0x55789366b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 4 client.5238 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:37.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.912+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/2664591769 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578935b8b60 con 0x557893666a70 2024-04-25T09:04:37.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.913+0000 7f1b04c00640 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557893666a70 0x55789366b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.913+0000 7f1b04c00640 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557893666a70 0x55789366b860 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34bb52b09976f1e8 server_cookie=19e6a0b300e668f6 in_seq=0 out_seq=0 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.913+0000 7f1aed600640 10 client.5238 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.914+0000 7f1aed600640 1 -- 172.21.15.62:0/2664591769 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578935b8b60 con 0x557893666a70 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.914+0000 7f1aed600640 10 client.5238 handle_command_reply: tid=0 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.914+0000 7f1b0a3e2740 1 client.5238 shutdown 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/2664591769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557893655980 msgr2=0x557893666330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557893655980 0x557893666330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1af800afd0 tx=0x7f1af8058b90 comp rx=0 tx=0).stop 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/2664591769 shutdown_connections 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557893666a70 0x55789366b860 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557893655980 0x557893666330 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578936905c0 0x557893655440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/2664591769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55789366fcd0 0x55789368f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1aed600640 0 client.5238 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:04:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/2664591769 >> 172.21.15.62:0/2664591769 conn(0x557893647950 msgr2=0x557893690a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/2664591769 shutdown_connections 2024-04-25T09:04:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/2664591769 wait complete. 2024-04-25T09:04:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.915+0000 7f1b0a3e2740 20 client.5238 trim_cache size 0 max 16384 2024-04-25T09:04:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.933+0000 7f1b0a3e2740 1 Processor -- start 2024-04-25T09:04:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.933+0000 7f1b0a3e2740 1 -- start start 2024-04-25T09:04:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.933+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893655980 0x55789366eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x55789366f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 0x55789366a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001ff220 con 0x55789366fcd0 2024-04-25T09:04:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001f84b0 con 0x5578936905c0 2024-04-25T09:04:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b001fec20 con 0x557893655980 2024-04-25T09:04:37.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1affe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x55789366f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1affe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x55789366f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39812/0 (socket says 172.21.15.62:39812) 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1affe00640 1 -- 172.21.15.62:0/3912328685 learned_addr learned my addr 172.21.15.62:0/3912328685 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1b04c00640 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893655980 0x55789366eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1b05600640 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 0x55789366a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1affe00640 1 -- 172.21.15.62:0/3912328685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893655980 msgr2=0x55789366eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1affe00640 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893655980 0x55789366eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.934+0000 7f1affe00640 1 -- 172.21.15.62:0/3912328685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 msgr2=0x55789366a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.935+0000 7f1affe00640 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 0x55789366a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.935+0000 7f1affe00640 1 -- 172.21.15.62:0/3912328685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55789338a6b0 con 0x55789366fcd0 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.935+0000 7f1affe00640 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x55789366f0c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8017040 tx=0x7f1ae8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3021cbab34fe1519 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.935+0000 7f1aefe00640 1 -- 172.21.15.62:0/3912328685 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8023070 con 0x55789366fcd0 2024-04-25T09:04:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.935+0000 7f1aefe00640 1 -- 172.21.15.62:0/3912328685 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae801a040 con 0x55789366fcd0 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.935+0000 7f1aefe00640 1 -- 172.21.15.62:0/3912328685 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8019070 con 0x55789366fcd0 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3912328685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 msgr2=0x55789366f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x55789366f0c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8017040 tx=0x7f1ae8011000 comp rx=0 tx=0).stop 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3912328685 shutdown_connections 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55789366fcd0 0x55789366f0c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578936905c0 0x55789366a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/3912328685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893655980 0x55789366eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3912328685 >> 172.21.15.62:0/3912328685 conn(0x557893647950 msgr2=0x55789366d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3912328685 shutdown_connections 2024-04-25T09:04:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.936+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/3912328685 wait complete. 2024-04-25T09:04:37.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.937+0000 7f1b0a3e2740 1 -- start start 2024-04-25T09:04:37.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.937+0000 7f1b0a3e2740 1 Processor -- start 2024-04-25T09:04:37.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557893655980 0x557893648740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578936905c0 0x557893672760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b0a3e2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893642270 0x557893672ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578936668f0 con 0x5578936905c0 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557893664fa0 con 0x557893655980 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b0a3e2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55789366ca80 con 0x557893642270 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b0a3e2740 10 client.0 _ll_register_callbacks cb 0x7f1af8054d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1affe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578936905c0 0x557893672760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1affe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578936905c0 0x557893672760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39828/0 (socket says 172.21.15.62:39828) 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1affe00640 1 -- 172.21.15.62:0/490089765 learned_addr learned my addr 172.21.15.62:0/490089765 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b04c00640 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557893655980 0x557893648740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.938+0000 7f1b05600640 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893642270 0x557893672ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1affe00640 1 -- 172.21.15.62:0/490089765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893642270 msgr2=0x557893672ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1affe00640 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893642270 0x557893672ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1affe00640 1 -- 172.21.15.62:0/490089765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557893655980 msgr2=0x557893648740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1affe00640 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557893655980 0x557893648740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1affe00640 1 -- 172.21.15.62:0/490089765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55789366b380 con 0x5578936905c0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1affe00640 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578936905c0 0x557893672760 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x557893643ee0 tx=0x7f1ae8000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82698ec3b693a053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1aed600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1aed600640 1 -- 172.21.15.62:0/490089765 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8019070 con 0x5578936905c0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/490089765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ae801a040 con 0x5578936905c0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.939+0000 7f1aed600640 1 -- 172.21.15.62:0/490089765 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ae8006ba0 con 0x5578936905c0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1aed600640 1 -- 172.21.15.62:0/490089765 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ae802f050 con 0x5578936905c0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b0a3e2740 10 client.5244 fetch_fsmap learned FSMap version 16 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/490089765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x557893648c80 con 0x5578936905c0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1aed600640 1 -- 172.21.15.62:0/490089765 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae801a040 con 0x5578936905c0 2024-04-25T09:04:37.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1aed600640 1 -- 172.21.15.62:0/490089765 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1ae8032020 con 0x5578936905c0 2024-04-25T09:04:37.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b0a3e2740 10 client.5244 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:04:37.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b0a3e2740 10 client.5244 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:04:37.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b0a3e2740 20 client.5244 populate_metadata read hostname 'smithi062' 2024-04-25T09:04:37.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557893701330 0x557893703720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b0a3e2740 4 client.5244 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:04:37.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/490089765 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5578935b2eb0 con 0x557893701330 2024-04-25T09:04:37.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.940+0000 7f1b04c00640 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557893701330 0x557893703720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.941+0000 7f1b04c00640 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557893701330 0x557893703720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6313037d5e49a700 server_cookie=8226006d91367932 in_seq=0 out_seq=0 2024-04-25T09:04:37.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.941+0000 7f1aed600640 10 client.5244 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:04:37.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.941+0000 7f1aed600640 1 -- 172.21.15.62:0/490089765 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5578935b2eb0 con 0x557893701330 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.941+0000 7f1aed600640 10 client.5244 handle_command_reply: tid=0 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.941+0000 7f1b0a3e2740 1 client.5244 shutdown 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.941+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/490089765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578936905c0 msgr2=0x557893672760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.941+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578936905c0 0x557893672760 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x557893643ee0 tx=0x7f1ae8000fb0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/490089765 shutdown_connections 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557893701330 0x557893703720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578936905c0 0x557893672760 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557893642270 0x557893672ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 1 --2- 172.21.15.62:0/490089765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557893655980 0x557893648740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1aed600640 0 client.5244 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/490089765 >> 172.21.15.62:0/490089765 conn(0x557893647950 msgr2=0x557893676200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/490089765 shutdown_connections 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 1 -- 172.21.15.62:0/490089765 wait complete. 2024-04-25T09:04:37.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b0a3e2740 20 client.5244 trim_cache size 0 max 16384 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 5043, 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 11.747223007000001, 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 16, 2024-04-25T09:04:37.934 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 33, 2024-04-25T09:04:37.935 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 33, 2024-04-25T09:04:37.935 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 20.580738677999999 2024-04-25T09:04:37.935 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:04:37.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b07600640 1 -- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1adc06b4d0 msgr2=0x7f1adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b07600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1adc06b4d0 0x7f1adc06d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1af0033000 tx=0x7f1af0009000 comp rx=0 tx=0).stop 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b07600640 1 -- 172.21.15.62:0/1771285540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 msgr2=0x7f1b000ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.942+0000 7f1b07600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000ad8e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c7f0 tx=0x7f1af800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.943+0000 7f1b07600640 1 -- 172.21.15.62:0/1771285540 shutdown_connections 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.943+0000 7f1b07600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1adc06b4d0 0x7f1adc06d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.943+0000 7f1b07600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0015bd10 0x7f1b000ad8e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.943+0000 7f1b07600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b000cccf0 0x7f1b000b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.943+0000 7f1b07600640 1 --2- 172.21.15.62:0/1771285540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b000aeed0 0x7f1b0014d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.943+0000 7f1b07600640 1 -- 172.21.15.62:0/1771285540 >> 172.21.15.62:0/1771285540 conn(0x7f1b000668f0 msgr2=0x7f1b000b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.943+0000 7f1b07600640 1 -- 172.21.15.62:0/1771285540 shutdown_connections 2024-04-25T09:04:37.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:37.943+0000 7f1b07600640 1 -- 172.21.15.62:0/1771285540 wait complete. 2024-04-25T09:04:37.948 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:04:38.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.170+0000 7f2e9de00640 1 Processor -- start 2024-04-25T09:04:38.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.170+0000 7f2e9de00640 1 -- start start 2024-04-25T09:04:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.171+0000 7f2e9de00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2e980b56f0 0x7f2e98160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.171+0000 7f2e9de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e980b6a90 0x7f2e98161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.172+0000 7f2e9de00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e980d3110 con 0x7f2e980b27a0 2024-04-25T09:04:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.172+0000 7f2e9de00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e98161780 con 0x7f2e980b6a90 2024-04-25T09:04:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.172+0000 7f2e9de00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e980cedc0 con 0x7f2e980b56f0 2024-04-25T09:04:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.172+0000 7f2e9d400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2e980b27a0 0x7f2e98161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42582/0 (socket says 172.21.15.62:42582) 2024-04-25T09:04:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.172+0000 7f2e9d400640 1 -- 172.21.15.62:0/1737093121 learned_addr learned my addr 172.21.15.62:0/1737093121 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.172+0000 7f2e97e00640 1 --2- 172.21.15.62:0/1737093121 >> v2:172.21.15.62:3300/0 conn(0x7f2e980b6a90 0x7f2e98161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.172+0000 7f2e9ca00640 1 --2- 172.21.15.62:0/1737093121 >> v2:172.21.15.121:3300/0 conn(0x7f2e980b56f0 0x7f2e98160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.172+0000 7f2e97400640 1 -- 172.21.15.62:0/1737093121 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058774875 0 0) 0x7f2e980d3110 con 0x7f2e980b27a0 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.173+0000 7f2e97400640 1 -- 172.21.15.62:0/1737093121 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e7c003520 con 0x7f2e980b27a0 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.173+0000 7f2e97e00640 1 -- 172.21.15.62:0/1737093121 >> v2:172.21.15.121:3300/0 conn(0x7f2e980b56f0 msgr2=0x7f2e98160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.173+0000 7f2e97e00640 1 --2- 172.21.15.62:0/1737093121 >> v2:172.21.15.121:3300/0 conn(0x7f2e980b56f0 0x7f2e98160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.173+0000 7f2e97e00640 1 -- 172.21.15.62:0/1737093121 >> v1:172.21.15.62:6789/0 conn(0x7f2e980b27a0 legacy=0x7f2e98161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.173+0000 7f2e97e00640 1 -- 172.21.15.62:0/1737093121 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980d33f0 con 0x7f2e980b6a90 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.174+0000 7f2e97e00640 1 --2- 172.21.15.62:0/1737093121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98161240 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2e9808c310 tx=0x7f2e8002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3884ab943cc99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.174+0000 7f2e97400640 1 -- 172.21.15.62:0/1737093121 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80037070 con 0x7f2e980b6a90 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.174+0000 7f2e97400640 1 -- 172.21.15.62:0/1737093121 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e80032440 con 0x7f2e980b6a90 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.174+0000 7f2e97400640 1 -- 172.21.15.62:0/1737093121 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80032810 con 0x7f2e980b6a90 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.175+0000 7f2e9de00640 1 -- 172.21.15.62:0/1737093121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 msgr2=0x7f2e98161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.175+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1737093121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98161240 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2e9808c310 tx=0x7f2e8002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.175+0000 7f2e9de00640 1 -- 172.21.15.62:0/1737093121 shutdown_connections 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.175+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1737093121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98161240 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.175+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1737093121 >> v2:172.21.15.121:3300/0 conn(0x7f2e980b56f0 0x7f2e98160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.175+0000 7f2e9de00640 1 -- 172.21.15.62:0/1737093121 >> 172.21.15.62:0/1737093121 conn(0x7f2e98066d10 msgr2=0x7f2e98067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.175+0000 7f2e9de00640 1 -- 172.21.15.62:0/1737093121 shutdown_connections 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.175+0000 7f2e9de00640 1 -- 172.21.15.62:0/1737093121 wait complete. 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.177+0000 7f2e9de00640 1 Processor -- start 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.177+0000 7f2e9de00640 1 -- start start 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e980b27a0 0x7f2e9815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e980b56f0 0x7f2e98158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b2d60 con 0x7f2e980b6a90 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9de00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980cea50 con 0x7f2e980b27a0 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9de00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980b2ee0 con 0x7f2e980b56f0 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39840/0 (socket says 172.21.15.62:39840) 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.178+0000 7f2e9d400640 1 -- 172.21.15.62:0/1298264006 learned_addr learned my addr 172.21.15.62:0/1298264006 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e9ca00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e980b27a0 0x7f2e9815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e97e00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e980b56f0 0x7f2e98158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e9d400640 1 -- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e980b56f0 msgr2=0x7f2e98158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e9d400640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e980b56f0 0x7f2e98158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e9d400640 1 -- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e980b27a0 msgr2=0x7f2e9815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e9d400640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e980b27a0 0x7f2e9815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e9d400640 1 -- 172.21.15.62:0/1298264006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e981f3910 con 0x7f2e980b6a90 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e9d400640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98158a40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000c430 tx=0x7f2e9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.179+0000 7f2e94c00640 1 -- 172.21.15.62:0/1298264006 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90017070 con 0x7f2e980b6a90 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.180+0000 7f2e94c00640 1 -- 172.21.15.62:0/1298264006 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e90005ce0 con 0x7f2e980b6a90 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.180+0000 7f2e94c00640 1 -- 172.21.15.62:0/1298264006 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900072d0 con 0x7f2e980b6a90 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.180+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9815d800 con 0x7f2e980b6a90 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.180+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9815d1b0 con 0x7f2e980b6a90 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.181+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9804ef20 con 0x7f2e980b6a90 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.182+0000 7f2e94c00640 1 -- 172.21.15.62:0/1298264006 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e9001a070 con 0x7f2e980b6a90 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.183+0000 7f2e94c00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e7406b4d0 0x7f2e7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.183+0000 7f2e94c00640 1 -- 172.21.15.62:0/1298264006 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e90089580 con 0x7f2e980b6a90 2024-04-25T09:04:38.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.183+0000 7f2e9ca00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e7406b4d0 0x7f2e7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.184+0000 7f2e9ca00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e7406b4d0 0x7f2e7406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e9815fdc0 tx=0x7f2e880039b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.187+0000 7f2e94c00640 1 -- 172.21.15.62:0/1298264006 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e90057d10 con 0x7f2e980b6a90 2024-04-25T09:04:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.433+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2e98146bc0 con 0x7f2e980b6a90 2024-04-25T09:04:38.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.434+0000 7f2e94c00640 1 -- 172.21.15.62:0/1298264006 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2e9005bdb0 con 0x7f2e980b6a90 2024-04-25T09:04:38.422 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:04:38.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e7406b4d0 msgr2=0x7f2e7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e7406b4d0 0x7f2e7406d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e9815fdc0 tx=0x7f2e880039b0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 msgr2=0x7f2e98158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98158a40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000c430 tx=0x7f2e9000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 shutdown_connections 2024-04-25T09:04:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e7406b4d0 0x7f2e7406d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e980b6a90 0x7f2e98158a40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e980b56f0 0x7f2e98158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 --2- 172.21.15.62:0/1298264006 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e980b27a0 0x7f2e9815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 >> 172.21.15.62:0/1298264006 conn(0x7f2e98066d10 msgr2=0x7f2e980b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 shutdown_connections 2024-04-25T09:04:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:38.437+0000 7f2e9de00640 1 -- 172.21.15.62:0/1298264006 wait complete. 2024-04-25T09:04:38.446 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:04:38.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:04:38.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:04:38.447 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:04:38.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:38.447 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:04:38.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:04:38.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:04:38.447 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:38.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:38.447 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:04:38.502 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:04:38.502 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:04:38.502 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:04:38.502 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:38.502 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42580sec preferred_lft 42580sec 2024-04-25T09:04:38.503 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:38.504 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:04:38.505 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:38.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:38.506 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:04:38.587 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:04:38.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:38.591 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:04:38.613 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:38.643 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:04:38.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:38.646 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:04:38.647 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:04:38.647 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:04:38.647 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:04:38.647 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:04:38.647 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:04:38.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:38.828 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:04:38.879 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:04:38.879 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:04:38.879 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:38.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:38.880 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:04:38.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:38.967 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:39.021 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:39.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:39.022 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:39.072 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:04:39.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:39.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:04:39.202 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:04:39.203 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:04:39.241 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:04:39.241 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:04:39.241 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:04:39.241 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:04:39.241 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:04:39.244 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:04:39.244 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:04:39.244 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:04:39.244 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:04:39.244 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:04:39.249 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:04:39.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:39.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:39.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:39.320 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:04:39.376 INFO:teuthology.orchestra.run.smithi062.stdout:client4936 2024-04-25T09:04:39.376 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:04:39.376 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:39.425 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T09:04:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.688+0000 7fdfabe00640 1 Processor -- start 2024-04-25T09:04:39.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.688+0000 7fdfabe00640 1 -- start start 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.689+0000 7fdfabe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdfac161270 0x7fdfac0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.689+0000 7fdfabe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdfac161650 0x7fdfac0b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.689+0000 7fdfabe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfac0d1480 con 0x7fdfac0b2980 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.689+0000 7fdfabe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfac0d3900 con 0x7fdfac161270 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.689+0000 7fdfabe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfac0d11a0 con 0x7fdfac161650 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfab400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdfac0b2980 0x7fdfac161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42588/0 (socket says 172.21.15.62:42588) 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfaaa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdfac161270 0x7fdfac0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfab400640 1 -- 172.21.15.62:0/3046272615 learned_addr learned my addr 172.21.15.62:0/3046272615 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:04:39.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfa3e00640 1 --2- 172.21.15.62:0/3046272615 >> v2:172.21.15.121:3300/0 conn(0x7fdfac161650 0x7fdfac0b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfaaa00640 1 -- 172.21.15.62:0/3046272615 >> v2:172.21.15.121:3300/0 conn(0x7fdfac161650 msgr2=0x7fdfac0b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfaaa00640 1 --2- 172.21.15.62:0/3046272615 >> v2:172.21.15.121:3300/0 conn(0x7fdfac161650 0x7fdfac0b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfaaa00640 1 -- 172.21.15.62:0/3046272615 >> v1:172.21.15.62:6789/0 conn(0x7fdfac0b2980 legacy=0x7fdfac161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfaaa00640 1 -- 172.21.15.62:0/3046272615 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac0cee80 con 0x7fdfac161270 2024-04-25T09:04:39.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.690+0000 7fdfaa000640 1 -- 172.21.15.62:0/3046272615 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100500337 0 0) 0x7fdfac0d1480 con 0x7fdfac0b2980 2024-04-25T09:04:39.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.692+0000 7fdfaaa00640 1 --2- 172.21.15.62:0/3046272615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161270 0x7fdfac0b5560 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdfac08c7d0 tx=0x7fdf9c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2a6b69c53aa722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.692+0000 7fdfaa000640 1 -- 172.21.15.62:0/3046272615 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c037070 con 0x7fdfac161270 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.692+0000 7fdfaa000640 1 -- 172.21.15.62:0/3046272615 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c02add0 con 0x7fdfac161270 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.692+0000 7fdfaa000640 1 -- 172.21.15.62:0/3046272615 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c032660 con 0x7fdfac161270 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.692+0000 7fdfabe00640 1 -- 172.21.15.62:0/3046272615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161270 msgr2=0x7fdfac0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.692+0000 7fdfabe00640 1 --2- 172.21.15.62:0/3046272615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161270 0x7fdfac0b5560 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdfac08c7d0 tx=0x7fdf9c02a080 comp rx=0 tx=0).stop 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.693+0000 7fdfabe00640 1 -- 172.21.15.62:0/3046272615 shutdown_connections 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.693+0000 7fdfabe00640 1 --2- 172.21.15.62:0/3046272615 >> v2:172.21.15.121:3300/0 conn(0x7fdfac161650 0x7fdfac0b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.693+0000 7fdfabe00640 1 --2- 172.21.15.62:0/3046272615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161270 0x7fdfac0b5560 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.693+0000 7fdfabe00640 1 -- 172.21.15.62:0/3046272615 >> 172.21.15.62:0/3046272615 conn(0x7fdfac066d10 msgr2=0x7fdfac067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.693+0000 7fdfabe00640 1 -- 172.21.15.62:0/3046272615 shutdown_connections 2024-04-25T09:04:39.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.693+0000 7fdfabe00640 1 -- 172.21.15.62:0/3046272615 wait complete. 2024-04-25T09:04:39.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.694+0000 7fdfabe00640 1 Processor -- start 2024-04-25T09:04:39.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.695+0000 7fdfabe00640 1 -- start start 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.695+0000 7fdfabe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfac0b2980 0x7fdfac0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.695+0000 7fdfabe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfac161270 0x7fdfac0d2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161650 0x7fdfac0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfabe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0b2ed0 con 0x7fdfac161650 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfabe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0d3590 con 0x7fdfac0b2980 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfabe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0b3050 con 0x7fdfac161270 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfab400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161650 0x7fdfac0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfab400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161650 0x7fdfac0d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39858/0 (socket says 172.21.15.62:39858) 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfab400640 1 -- 172.21.15.62:0/1468272252 learned_addr learned my addr 172.21.15.62:0/1468272252 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:04:39.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfaaa00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfac0b2980 0x7fdfac0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfa3e00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfac161270 0x7fdfac0d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfab400640 1 -- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfac161270 msgr2=0x7fdfac0d2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfab400640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfac161270 0x7fdfac0d2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfab400640 1 -- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfac0b2980 msgr2=0x7fdfac0cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.696+0000 7fdfab400640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfac0b2980 0x7fdfac0cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.697+0000 7fdfab400640 1 -- 172.21.15.62:0/1468272252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac15b0a0 con 0x7fdfac161650 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.697+0000 7fdfab400640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161650 0x7fdfac0d2590 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400c700 tx=0x7fdfa400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.697+0000 7fdfa2a00640 1 -- 172.21.15.62:0/1468272252 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4018070 con 0x7fdfac161650 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.697+0000 7fdfa2a00640 1 -- 172.21.15.62:0/1468272252 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa400ce10 con 0x7fdfac161650 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.697+0000 7fdfa2a00640 1 -- 172.21.15.62:0/1468272252 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400a860 con 0x7fdfac161650 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.697+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac15b380 con 0x7fdfac161650 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.697+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac15b940 con 0x7fdfac161650 2024-04-25T09:04:39.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.699+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac072910 con 0x7fdfac161650 2024-04-25T09:04:39.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.700+0000 7fdfa2a00640 1 -- 172.21.15.62:0/1468272252 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfa400aa00 con 0x7fdfac161650 2024-04-25T09:04:39.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.701+0000 7fdfa2a00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.701+0000 7fdfa2a00640 1 -- 172.21.15.62:0/1468272252 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfa4089810 con 0x7fdfac161650 2024-04-25T09:04:39.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.701+0000 7fdfaaa00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.703+0000 7fdfaaa00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf8806b4d0 0x7fdf8806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c034000 tx=0x7fdf9c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.705+0000 7fdfa2a00640 1 -- 172.21.15.62:0/1468272252 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfa4058680 con 0x7fdfac161650 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:39.951+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdfac0b63a0 con 0x7fdfac161650 2024-04-25T09:04:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.296+0000 7fdfa2a00640 1 -- 172.21.15.62:0/1468272252 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7fdfa4057f80 con 0x7fdfac161650 2024-04-25T09:04:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:enabled new snapshots 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf8806b4d0 msgr2=0x7fdf8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf8806b4d0 0x7fdf8806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c034000 tx=0x7fdf9c02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161650 msgr2=0x7fdfac0d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161650 0x7fdfac0d2590 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400c700 tx=0x7fdfa400cbc0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 shutdown_connections 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfac161650 0x7fdfac0d2590 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdfac161270 0x7fdfac0d2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 --2- 172.21.15.62:0/1468272252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfac0b2980 0x7fdfac0cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 >> 172.21.15.62:0/1468272252 conn(0x7fdfac066d10 msgr2=0x7fdfac0b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 shutdown_connections 2024-04-25T09:04:40.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:04:40.299+0000 7fdfabe00640 1 -- 172.21.15.62:0/1468272252 wait complete. 2024-04-25T09:04:40.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:40.306 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T09:04:40.364 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:04:40.365 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:40.409 DEBUG:teuthology.orchestra.run.smithi062:> stdin-killer -- python3 -c ' 2024-04-25T09:04:40.409 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> import uuid 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> n = 2000 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> create_hard_links = False 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> if 3 > 0: 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> create_hard_links = True 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> dpath = os.path.dirname(path) 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> fnameprefix = os.path.basename(path) 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> os.makedirs(dpath, exist_ok=True) 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:04:40.410 DEBUG:teuthology.orchestra.run.smithi062:> try: 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> for i in range(n): 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> f.write(f"{i}") 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> if False: 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> f.flush() 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> os.fsync(f.fileno()) 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> if False: 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> os.unlink(fpath) 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> if False: 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> os.fsync(dirfd) 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> if create_hard_links: 2024-04-25T09:04:40.411 DEBUG:teuthology.orchestra.run.smithi062:> for j in range(3): 2024-04-25T09:04:40.412 DEBUG:teuthology.orchestra.run.smithi062:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T09:04:40.412 DEBUG:teuthology.orchestra.run.smithi062:> if False: 2024-04-25T09:04:40.412 DEBUG:teuthology.orchestra.run.smithi062:> os.fsync(dirfd) 2024-04-25T09:04:40.412 DEBUG:teuthology.orchestra.run.smithi062:> finally: 2024-04-25T09:04:40.412 DEBUG:teuthology.orchestra.run.smithi062:> os.close(dirfd) 2024-04-25T09:04:40.412 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-04-25T09:05:08.787 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:08.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:30.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:05:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:05:30.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:30.808 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:05:30.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:05:30.917 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T09:05:38.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:38.928 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:06:09.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:06:09.048 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:06:32.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:06:32.207 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:06:32.288 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:06:32.288 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T09:06:32.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.505+0000 7fdac0c00640 1 Processor -- start 2024-04-25T09:06:32.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.506+0000 7fdac0c00640 1 -- start start 2024-04-25T09:06:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.508+0000 7fdac0c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdabc15c520 0x7fdabc0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.508+0000 7fdac0c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 0x7fdabc06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.508+0000 7fdac0c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdabc166720 0x7fdabc06d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.508+0000 7fdac0c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0b4640 con 0x7fdabc15c520 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.508+0000 7fdac0c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc165c40 con 0x7fdabc166720 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.508+0000 7fdac0c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc165f20 con 0x7fdabc0b3360 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.508+0000 7fdabb400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdabc15c520 0x7fdabc0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabb400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdabc15c520 0x7fdabc0b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42556/0 (socket says 172.21.15.62:42556) 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabb400640 1 -- 172.21.15.62:0/1509590390 learned_addr learned my addr 172.21.15.62:0/1509590390 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabaa00640 1 --2- 172.21.15.62:0/1509590390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 0x7fdabc06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabbe00640 1 --2- 172.21.15.62:0/1509590390 >> v2:172.21.15.121:3300/0 conn(0x7fdabc166720 0x7fdabc06d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabb400640 1 -- 172.21.15.62:0/1509590390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 msgr2=0x7fdabc06d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabb400640 1 --2- 172.21.15.62:0/1509590390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 0x7fdabc06d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabb400640 1 -- 172.21.15.62:0/1509590390 >> v2:172.21.15.121:3300/0 conn(0x7fdabc166720 msgr2=0x7fdabc06d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabb400640 1 --2- 172.21.15.62:0/1509590390 >> v2:172.21.15.121:3300/0 conn(0x7fdabc166720 0x7fdabc06d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.509+0000 7fdabb400640 1 -- 172.21.15.62:0/1509590390 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc166200 con 0x7fdabc15c520 2024-04-25T09:06:32.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.510+0000 7fdabb400640 1 --2- 172.21.15.62:0/1509590390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc0b3d10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdabc165e50 tx=0x7fdab402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9b41db3794baf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.511+0000 7fdaba000640 1 -- 172.21.15.62:0/1509590390 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4036070 con 0x7fdabc15c520 2024-04-25T09:06:32.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.511+0000 7fdaba000640 1 -- 172.21.15.62:0/1509590390 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab4031600 con 0x7fdabc15c520 2024-04-25T09:06:32.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.511+0000 7fdaba000640 1 -- 172.21.15.62:0/1509590390 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40319d0 con 0x7fdabc15c520 2024-04-25T09:06:32.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.511+0000 7fdac0c00640 1 -- 172.21.15.62:0/1509590390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 msgr2=0x7fdabc0b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.511+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1509590390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc0b3d10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdabc165e50 tx=0x7fdab402a5f0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.512+0000 7fdac0c00640 1 -- 172.21.15.62:0/1509590390 shutdown_connections 2024-04-25T09:06:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.512+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1509590390 >> v2:172.21.15.121:3300/0 conn(0x7fdabc166720 0x7fdabc06d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.512+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1509590390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 0x7fdabc06d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.512+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1509590390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc0b3d10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.512+0000 7fdac0c00640 1 -- 172.21.15.62:0/1509590390 >> 172.21.15.62:0/1509590390 conn(0x7fdabc066f20 msgr2=0x7fdabc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.512+0000 7fdac0c00640 1 -- 172.21.15.62:0/1509590390 shutdown_connections 2024-04-25T09:06:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.512+0000 7fdac0c00640 1 -- 172.21.15.62:0/1509590390 wait complete. 2024-04-25T09:06:32.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.513+0000 7fdac0c00640 1 Processor -- start 2024-04-25T09:06:32.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.514+0000 7fdac0c00640 1 -- start start 2024-04-25T09:06:32.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.514+0000 7fdac0c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 0x7fdabc14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.514+0000 7fdac0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.514+0000 7fdac0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdabc166720 0x7fdabc14d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdac0c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0b4250 con 0x7fdabc15c520 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdac0c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc15c900 con 0x7fdabc166720 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdac0c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0b43d0 con 0x7fdabc0b3360 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc14cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42564/0 (socket says 172.21.15.62:42564) 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabaa00640 1 -- 172.21.15.62:0/1041194425 learned_addr learned my addr 172.21.15.62:0/1041194425 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabb400640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 0x7fdabc14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabbe00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdabc166720 0x7fdabc14d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabaa00640 1 -- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 msgr2=0x7fdabc14b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabaa00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 0x7fdabc14b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabaa00640 1 -- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdabc166720 msgr2=0x7fdabc14d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabaa00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdabc166720 0x7fdabc14d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabaa00640 1 -- 172.21.15.62:0/1041194425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc14db80 con 0x7fdabc15c520 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.515+0000 7fdabbe00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdabc166720 0x7fdabc14d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.516+0000 7fdabaa00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc14cdc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400d830 tx=0x7fdaa400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.516+0000 7fdab3400640 1 -- 172.21.15.62:0/1041194425 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4018020 con 0x7fdabc15c520 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.516+0000 7fdab3400640 1 -- 172.21.15.62:0/1041194425 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa40096a0 con 0x7fdabc15c520 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.516+0000 7fdab3400640 1 -- 172.21.15.62:0/1041194425 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4009a70 con 0x7fdabc15c520 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.516+0000 7fdac0c00640 1 -- 172.21.15.62:0/1041194425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc14de60 con 0x7fdabc15c520 2024-04-25T09:06:32.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.516+0000 7fdac0c00640 1 -- 172.21.15.62:0/1041194425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc14e420 con 0x7fdabc15c520 2024-04-25T09:06:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.518+0000 7fdab3400640 1 -- 172.21.15.62:0/1041194425 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdaa401b020 con 0x7fdabc15c520 2024-04-25T09:06:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.519+0000 7fdab3400640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda9806b4d0 0x7fda9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.520+0000 7fdab3400640 1 -- 172.21.15.62:0/1041194425 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdaa408a7f0 con 0x7fdabc15c520 2024-04-25T09:06:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.520+0000 7fdac5163740 1 Processor -- start 2024-04-25T09:06:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.520+0000 7fdac5163740 1 -- start start 2024-04-25T09:06:32.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.520+0000 7fdabb400640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda9806b4d0 0x7fda9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdac5163740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a24c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdac5163740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a25290 0x55c0769f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdac5163740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0769f4fb0 0x55c076a1ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdac5163740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07695d870 con 0x55c076a24870 2024-04-25T09:06:32.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdac5163740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c076906d60 con 0x55c0769f4fb0 2024-04-25T09:06:32.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdac5163740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c076959910 con 0x55c076a25290 2024-04-25T09:06:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdabb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a24c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdabb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a24c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42566/0 (socket says 172.21.15.62:42566) 2024-04-25T09:06:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.521+0000 7fdabb400640 1 -- 172.21.15.62:0/1507718145 learned_addr learned my addr 172.21.15.62:0/1507718145 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabaa00640 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a25290 0x55c0769f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabbe00640 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0769f4fb0 0x55c076a1ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabb400640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda9806b4d0 0x7fda9806d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdab4033000 tx=0x7fdab402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabb400640 1 -- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a25290 msgr2=0x55c0769f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabb400640 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a25290 0x55c0769f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabb400640 1 -- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0769f4fb0 msgr2=0x55c076a1ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabb400640 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0769f4fb0 0x55c076a1ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabb400640 1 -- 172.21.15.62:0/1507718145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07673c7d0 con 0x55c076a24870 2024-04-25T09:06:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabaa00640 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a25290 0x55c0769f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:06:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdabb400640 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a24c50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000f80 tx=0x7fdab403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cc4a6f252713a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.522+0000 7fdaab400640 1 -- 172.21.15.62:0/1507718145 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4043070 con 0x55c076a24870 2024-04-25T09:06:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdaab400640 1 -- 172.21.15.62:0/1507718145 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab404c070 con 0x55c076a24870 2024-04-25T09:06:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdaab400640 1 -- 172.21.15.62:0/1507718145 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4036040 con 0x55c076a24870 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 -- 172.21.15.62:0/1507718145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 msgr2=0x55c076a24c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a24c50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000f80 tx=0x7fdab403c000 comp rx=0 tx=0).stop 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 -- 172.21.15.62:0/1507718145 shutdown_connections 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0769f4fb0 0x55c076a1ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a25290 0x55c0769f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 --2- 172.21.15.62:0/1507718145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a24c50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 -- 172.21.15.62:0/1507718145 >> 172.21.15.62:0/1507718145 conn(0x55c0769f6ef0 msgr2=0x55c076a3f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 -- 172.21.15.62:0/1507718145 shutdown_connections 2024-04-25T09:06:32.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.523+0000 7fdac5163740 1 -- 172.21.15.62:0/1507718145 wait complete. 2024-04-25T09:06:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.524+0000 7fdac5163740 1 -- start start 2024-04-25T09:06:32.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.524+0000 7fdac5163740 1 Processor -- start 2024-04-25T09:06:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.525+0000 7fdac5163740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a24870 0x55c076a19e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.525+0000 7fdac5163740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a26290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.525+0000 7fdac5163740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a23760 0x55c076a267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdac5163740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c076a20fc0 con 0x55c076a25290 2024-04-25T09:06:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdac5163740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07691a1c0 con 0x55c076a24870 2024-04-25T09:06:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdac5163740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c076a21260 con 0x55c076a23760 2024-04-25T09:06:32.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdac5163740 10 client.0 _ll_register_callbacks cb 0x55c0769588f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a26290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a26290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42568/0 (socket says 172.21.15.62:42568) 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabaa00640 1 -- 172.21.15.62:0/2952968563 learned_addr learned my addr 172.21.15.62:0/2952968563 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabb400640 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a24870 0x55c076a19e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabaa00640 1 -- 172.21.15.62:0/2952968563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a23760 msgr2=0x55c076a267d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabaa00640 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a23760 0x55c076a267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabaa00640 1 -- 172.21.15.62:0/2952968563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a24870 msgr2=0x55c076a19e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabaa00640 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a24870 0x55c076a19e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.526+0000 7fdabaa00640 1 -- 172.21.15.62:0/2952968563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0766320f0 con 0x55c076a25290 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fdabaa00640 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a26290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4004790 tx=0x7fdaa4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9a962ed58c4436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fda9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fda9fe00640 1 -- 172.21.15.62:0/2952968563 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4061560 con 0x55c076a25290 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fda9fe00640 1 -- 172.21.15.62:0/2952968563 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdaa4083c90 con 0x55c076a25290 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fdac5163740 1 -- 172.21.15.62:0/2952968563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdab404c070 con 0x55c076a25290 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fda9fe00640 1 -- 172.21.15.62:0/2952968563 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa405f0b0 con 0x55c076a25290 2024-04-25T09:06:32.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fdac5163740 10 client.5274 fetch_fsmap learned FSMap version 17 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fdac5163740 1 -- 172.21.15.62:0/2952968563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55c07679cf60 con 0x55c076a25290 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fda9fe00640 1 -- 172.21.15.62:0/2952968563 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab404c070 con 0x55c076a25290 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fda9fe00640 1 -- 172.21.15.62:0/2952968563 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdaa4013050 con 0x55c076a25290 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fdac5163740 10 client.5274 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.527+0000 7fdac5163740 10 client.5274 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.528+0000 7fdac5163740 20 client.5274 populate_metadata read hostname 'smithi062' 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.528+0000 7fdac5163740 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55c076a1eef0 0x55c076a1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.528+0000 7fdac5163740 4 client.5274 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.528+0000 7fdac5163740 1 -- 172.21.15.62:0/2952968563 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c076933690 con 0x55c076a1eef0 2024-04-25T09:06:32.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.528+0000 7fdabb400640 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55c076a1eef0 0x55c076a1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.528+0000 7fdabb400640 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55c076a1eef0 0x55c076a1f340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2a369c3427a552 server_cookie=8535da98906f8dc4 in_seq=0 out_seq=0 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.528+0000 7fda9fe00640 10 client.5274 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.529+0000 7fda9fe00640 1 -- 172.21.15.62:0/2952968563 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c076933690 con 0x55c076a1eef0 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.529+0000 7fda9fe00640 10 client.5274 handle_command_reply: tid=0 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.529+0000 7fdac5163740 1 client.5274 shutdown 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.529+0000 7fdac5163740 1 -- 172.21.15.62:0/2952968563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 msgr2=0x55c076a26290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.529+0000 7fdac5163740 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a26290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4004790 tx=0x7fdaa4009f60 comp rx=0 tx=0).stop 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 1 -- 172.21.15.62:0/2952968563 shutdown_connections 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55c076a1eef0 0x55c076a1f340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a26290 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a23760 0x55c076a267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 1 --2- 172.21.15.62:0/2952968563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a24870 0x55c076a19e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 1 -- 172.21.15.62:0/2952968563 >> 172.21.15.62:0/2952968563 conn(0x55c0769f4fb0 msgr2=0x55c0769f8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fda9fe00640 0 client.5274 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:32.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 1 -- 172.21.15.62:0/2952968563 shutdown_connections 2024-04-25T09:06:32.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 1 -- 172.21.15.62:0/2952968563 wait complete. 2024-04-25T09:06:32.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.530+0000 7fdac5163740 20 client.5274 trim_cache size 0 max 16384 2024-04-25T09:06:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.546+0000 7fdac5163740 1 Processor -- start 2024-04-25T09:06:32.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.546+0000 7fdac5163740 1 -- start start 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdac5163740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a23760 0x55c076a21cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdac5163740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdac5163740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a25290 0x55c0769f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdac5163740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc204310 con 0x55c076a24870 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdac5163740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc203510 con 0x55c076a25290 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdac5163740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc202940 con 0x55c076a23760 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdabaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdabaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a221f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42572/0 (socket says 172.21.15.62:42572) 2024-04-25T09:06:32.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdabaa00640 1 -- 172.21.15.62:0/2026767885 learned_addr learned my addr 172.21.15.62:0/2026767885 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdabaa00640 1 -- 172.21.15.62:0/2026767885 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a23760 msgr2=0x55c076a21cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdabaa00640 1 --2- 172.21.15.62:0/2026767885 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a23760 0x55c076a21cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdabaa00640 1 -- 172.21.15.62:0/2026767885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a25290 msgr2=0x55c0769f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdabaa00640 1 --2- 172.21.15.62:0/2026767885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a25290 0x55c0769f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.547+0000 7fdabaa00640 1 -- 172.21.15.62:0/2026767885 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c07660b1b0 con 0x55c076a24870 2024-04-25T09:06:32.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.548+0000 7fdabaa00640 1 --2- 172.21.15.62:0/2026767885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a221f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4057100 tx=0x7fdaa4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4de15526d1f2ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.548+0000 7fdaaaa00640 1 -- 172.21.15.62:0/2026767885 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa407c070 con 0x55c076a24870 2024-04-25T09:06:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.548+0000 7fdaaaa00640 1 -- 172.21.15.62:0/2026767885 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa405b050 con 0x55c076a24870 2024-04-25T09:06:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.548+0000 7fdaaaa00640 1 -- 172.21.15.62:0/2026767885 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4070e20 con 0x55c076a24870 2024-04-25T09:06:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 -- 172.21.15.62:0/2026767885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 msgr2=0x55c076a221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 --2- 172.21.15.62:0/2026767885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a221f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4057100 tx=0x7fdaa4016000 comp rx=0 tx=0).stop 2024-04-25T09:06:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 -- 172.21.15.62:0/2026767885 shutdown_connections 2024-04-25T09:06:32.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 --2- 172.21.15.62:0/2026767885 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a24870 0x55c076a221f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 --2- 172.21.15.62:0/2026767885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a25290 0x55c0769f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 --2- 172.21.15.62:0/2026767885 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a23760 0x55c076a21cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 -- 172.21.15.62:0/2026767885 >> 172.21.15.62:0/2026767885 conn(0x55c0769f4fb0 msgr2=0x55c0769f8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 -- 172.21.15.62:0/2026767885 shutdown_connections 2024-04-25T09:06:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 -- 172.21.15.62:0/2026767885 wait complete. 2024-04-25T09:06:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.549+0000 7fdac5163740 1 -- start start 2024-04-25T09:06:32.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.550+0000 7fdac5163740 1 Processor -- start 2024-04-25T09:06:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdac5163740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a23760 0x55c076a8e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdac5163740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a24870 0x55c076a21b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdac5163740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdac5163740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c076a52500 con 0x55c076a25290 2024-04-25T09:06:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdac5163740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc202f10 con 0x55c076a23760 2024-04-25T09:06:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdac5163740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c076a28610 con 0x55c076a24870 2024-04-25T09:06:32.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdac5163740 10 client.0 _ll_register_callbacks cb 0x7fdab4034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a220d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42584/0 (socket says 172.21.15.62:42584) 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabbe00640 1 -- 172.21.15.62:0/1059042482 learned_addr learned my addr 172.21.15.62:0/1059042482 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabaa00640 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a24870 0x55c076a21b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabb400640 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a23760 0x55c076a8e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabbe00640 1 -- 172.21.15.62:0/1059042482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a24870 msgr2=0x55c076a21b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabbe00640 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a24870 0x55c076a21b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabbe00640 1 -- 172.21.15.62:0/1059042482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a23760 msgr2=0x55c076a8e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabbe00640 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a23760 0x55c076a8e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.551+0000 7fdabbe00640 1 -- 172.21.15.62:0/1059042482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c076a8f330 con 0x55c076a25290 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.552+0000 7fdabbe00640 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a220d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55c076a20360 tx=0x7fdaac0105c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316934d49e72b8e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.552+0000 7fda9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.552+0000 7fda9fe00640 1 -- 172.21.15.62:0/1059042482 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac01a020 con 0x55c076a25290 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.552+0000 7fda9fe00640 1 -- 172.21.15.62:0/1059042482 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdaac015470 con 0x55c076a25290 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.552+0000 7fdac5163740 1 -- 172.21.15.62:0/1059042482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdaa405b050 con 0x55c076a25290 2024-04-25T09:06:32.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.552+0000 7fda9fe00640 1 -- 172.21.15.62:0/1059042482 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaac024050 con 0x55c076a25290 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.552+0000 7fdac5163740 10 client.5280 fetch_fsmap learned FSMap version 17 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.552+0000 7fdac5163740 1 -- 172.21.15.62:0/1059042482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55c076aaa010 con 0x55c076a25290 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fda9fe00640 1 -- 172.21.15.62:0/1059042482 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa405b050 con 0x55c076a25290 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fda9fe00640 1 -- 172.21.15.62:0/1059042482 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdaac004020 con 0x55c076a25290 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fdac5163740 10 client.5280 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fdac5163740 10 client.5280 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fdac5163740 20 client.5280 populate_metadata read hostname 'smithi062' 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fdac5163740 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55c076a8ebb0 0x55c076aa9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fdac5163740 4 client.5280 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:06:32.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fdac5163740 1 -- 172.21.15.62:0/1059042482 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55c07696adb0 con 0x55c076a8ebb0 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.553+0000 7fdabb400640 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55c076a8ebb0 0x55c076aa9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fdabb400640 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55c076a8ebb0 0x55c076aa9310 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc348663e7f766dc server_cookie=c3b2a90bfe295ab2 in_seq=0 out_seq=0 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fda9fe00640 10 client.5280 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fda9fe00640 1 -- 172.21.15.62:0/1059042482 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17928 (crc 0 0 0) 0x55c07696adb0 con 0x55c076a8ebb0 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fda9fe00640 10 client.5280 handle_command_reply: tid=0 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fdac5163740 1 client.5280 shutdown 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fdac5163740 1 -- 172.21.15.62:0/1059042482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 msgr2=0x55c076a220d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fdac5163740 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a220d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55c076a20360 tx=0x7fdaac0105c0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fdac5163740 1 -- 172.21.15.62:0/1059042482 shutdown_connections 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fdac5163740 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55c076a8ebb0 0x55c076aa9310 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.554+0000 7fdac5163740 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c076a25290 0x55c076a220d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.555+0000 7fdac5163740 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c076a24870 0x55c076a21b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.555+0000 7fdac5163740 1 --2- 172.21.15.62:0/1059042482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c076a23760 0x55c076a8e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.555+0000 7fdac5163740 1 -- 172.21.15.62:0/1059042482 >> 172.21.15.62:0/1059042482 conn(0x55c0769f4fb0 msgr2=0x55c076a19d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.555+0000 7fda9fe00640 0 client.5280 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.555+0000 7fdac5163740 1 -- 172.21.15.62:0/1059042482 shutdown_connections 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.555+0000 7fdac5163740 1 -- 172.21.15.62:0/1059042482 wait complete. 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.555+0000 7fdac5163740 20 client.5280 trim_cache size 0 max 16384 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 29710, 2024-04-25T09:06:32.545 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 50599, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 10656733, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 56379664, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 11, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 11.896547727, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 6.998151983, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 5.924533032, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.116502930, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 50597, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.768151414, 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000074473 2024-04-25T09:06:32.546 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 70241, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.041400967, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000589 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2934, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2928, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1022594, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 44082309, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-25T09:06:32.547 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.810198887, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.437316182, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.154718001, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.169795835, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2927, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.658453325, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000224958 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 122688, 2024-04-25T09:06:32.548 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 30064 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 1652, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 1679, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 418528, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 15497767, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.414486709, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.216250138, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.083593469, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.087984898, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1678, 2024-04-25T09:06:32.549 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.525483171, 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000313160 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 101040 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-25T09:06:32.550 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4162, 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 29.696609355, 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.007135177 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2056, 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.522406915, 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001226851 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.551 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 26010, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 26010, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 26010, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 511.642669716, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.019670998 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 12, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-25T09:06:32.552 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 19866, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 19866, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 19866, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 55, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 55, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 2, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 48676, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 48160, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:06:32.553 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 162, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T09:06:32.554 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 965, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 4022, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 3933, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:06:32.555 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 59, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 14053 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 2000, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 167, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 4127, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 1781, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 2230, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-25T09:06:32.556 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 25254, 2024-04-25T09:06:32.557 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 25254, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 36, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 36, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 4, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 95303107, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-04-25T09:06:32.558 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4081, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 133.776930336, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.032780428 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2014, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 2014, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 0, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 13, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 13, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-04-25T09:06:32.559 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 19866, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 19867, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 1, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 55, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 7613, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 7558, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 107596, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215396 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 29624, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 26010, 2024-04-25T09:06:32.560 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 7, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2000, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.096022981, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001548011 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 29, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.007911484, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000272809 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.561 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6000, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 14.487729938, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002414621 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 7738, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 16.053720950, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002074660 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.562 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.563 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000116263, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000116263 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.006928934, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006928934 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.564 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.017439030, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.017439030 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-25T09:06:32.565 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.185213532, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.023151691 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2230, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 443.213560512, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.198750475 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.566 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.037012200, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.037012200 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-25T09:06:32.567 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000453863, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000453863 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.568 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.569 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8000, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 34.536560029, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004317070 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.570 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 14342, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 14342, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 108, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-25T09:06:32.571 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-25T09:06:32.572 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-25T09:06:32.573 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-25T09:06:32.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-25T09:06:32.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:06:32.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:06:32.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:06:32.574 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:06:32.574 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 16373062, 2024-04-25T09:06:32.574 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 6103, 2024-04-25T09:06:32.574 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 440, 2024-04-25T09:06:32.577 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 5, 2024-04-25T09:06:32.577 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 5696, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 83, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 21573813, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 887045, 2024-04-25T09:06:32.578 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 6259, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 91514705, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 37, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 6259, 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-25T09:06:32.579 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6259, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 48.382228171, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.007730025 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6259, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 6420 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 6259, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 89, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 6170, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-25T09:06:32.580 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 11, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 89, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 4170, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 53, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 1781, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 33, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-25T09:06:32.581 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 22, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-25T09:06:32.582 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-25T09:06:32.583 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 35, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 3, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 8, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 8, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 215, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 46 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:06:32.584 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 32, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 1781, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 1781, 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-25T09:06:32.585 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 34292, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 9005654, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 34292, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 34292, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 9005654, 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:06:32.586 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 6259, 2024-04-25T09:06:32.587 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 91708834, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 6259, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 91708834, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T09:06:32.588 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 6259, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 6259, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 6259, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 6259, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:06:32.589 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 1781, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 194129, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 1781, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 89, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 194129, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.590 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x5652ab4246e0": { 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 84, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 25254, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 91108803, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 25254, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4081, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 91108719, 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:06:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:06:32.592 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:06:32.592 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:06:32.592 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.592 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:32.592 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:06:32.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 -- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda9806b4d0 msgr2=0x7fda9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda9806b4d0 0x7fda9806d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdab4033000 tx=0x7fdab402e000 comp rx=0 tx=0).stop 2024-04-25T09:06:32.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 -- 172.21.15.62:0/1041194425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 msgr2=0x7fdabc14cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc14cdc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400d830 tx=0x7fdaa400dcf0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 -- 172.21.15.62:0/1041194425 shutdown_connections 2024-04-25T09:06:32.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda9806b4d0 0x7fda9806d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdabc15c520 0x7fdabc14cdc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdabc166720 0x7fdabc14d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 --2- 172.21.15.62:0/1041194425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdabc0b3360 0x7fdabc14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 -- 172.21.15.62:0/1041194425 >> 172.21.15.62:0/1041194425 conn(0x7fdabc066f20 msgr2=0x7fdabc0af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 -- 172.21.15.62:0/1041194425 shutdown_connections 2024-04-25T09:06:32.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.556+0000 7fdac0c00640 1 -- 172.21.15.62:0/1041194425 wait complete. 2024-04-25T09:06:32.595 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T09:06:32.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:06:32.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T09:06:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.833+0000 7eff03e00640 1 Processor -- start 2024-04-25T09:06:32.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.833+0000 7eff03e00640 1 -- start start 2024-04-25T09:06:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.835+0000 7eff03e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415b520 0x7eff040ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.835+0000 7eff03e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff0406c440 0x7eff040adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.835+0000 7eff03e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0406e270 con 0x7eff0415eb90 2024-04-25T09:06:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.835+0000 7eff03e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff040b3fb0 con 0x7eff0406c440 2024-04-25T09:06:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.835+0000 7eff03e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0406c920 con 0x7eff0415b520 2024-04-25T09:06:32.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.836+0000 7efefa000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff0406c440 0x7eff040adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.836+0000 7eff02a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415b520 0x7eff040ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.836+0000 7efefa000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff0406c440 0x7eff040adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:60564/0 (socket says 172.21.15.62:60564) 2024-04-25T09:06:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.836+0000 7efefa000640 1 -- 172.21.15.62:0/4085868320 learned_addr learned my addr 172.21.15.62:0/4085868320 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.837+0000 7eff02000640 1 -- 172.21.15.62:0/4085868320 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176649011 0 0) 0x7eff0406e270 con 0x7eff0415eb90 2024-04-25T09:06:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.837+0000 7eff02000640 1 -- 172.21.15.62:0/4085868320 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee8003520 con 0x7eff0415eb90 2024-04-25T09:06:32.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.837+0000 7efefa000640 1 -- 172.21.15.62:0/4085868320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415b520 msgr2=0x7eff040ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.837+0000 7efefa000640 1 --2- 172.21.15.62:0/4085868320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415b520 0x7eff040ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.837+0000 7efefa000640 1 -- 172.21.15.62:0/4085868320 >> v1:172.21.15.121:6789/0 conn(0x7eff0415eb90 legacy=0x7eff0415f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.837+0000 7efefa000640 1 -- 172.21.15.62:0/4085868320 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0406cc00 con 0x7eff0406c440 2024-04-25T09:06:32.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.839+0000 7efefa000640 1 --2- 172.21.15.62:0/4085868320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 0x7eff040adea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efeec009870 tx=0x7efeec02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad805d08d6e2c9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.839+0000 7eff02000640 1 -- 172.21.15.62:0/4085868320 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec037070 con 0x7eff0406c440 2024-04-25T09:06:32.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.839+0000 7eff02000640 1 -- 172.21.15.62:0/4085868320 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec032440 con 0x7eff0406c440 2024-04-25T09:06:32.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.839+0000 7eff02000640 1 -- 172.21.15.62:0/4085868320 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec032810 con 0x7eff0406c440 2024-04-25T09:06:32.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.840+0000 7eff03e00640 1 -- 172.21.15.62:0/4085868320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 msgr2=0x7eff040adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.840+0000 7eff03e00640 1 --2- 172.21.15.62:0/4085868320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 0x7eff040adea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efeec009870 tx=0x7efeec02a270 comp rx=0 tx=0).stop 2024-04-25T09:06:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.840+0000 7eff03e00640 1 -- 172.21.15.62:0/4085868320 shutdown_connections 2024-04-25T09:06:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.840+0000 7eff03e00640 1 --2- 172.21.15.62:0/4085868320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 0x7eff040adea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.840+0000 7eff03e00640 1 --2- 172.21.15.62:0/4085868320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415b520 0x7eff040ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.840+0000 7eff03e00640 1 -- 172.21.15.62:0/4085868320 >> 172.21.15.62:0/4085868320 conn(0x7eff04066ad0 msgr2=0x7eff04066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.840+0000 7eff03e00640 1 -- 172.21.15.62:0/4085868320 shutdown_connections 2024-04-25T09:06:32.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.840+0000 7eff03e00640 1 -- 172.21.15.62:0/4085868320 wait complete. 2024-04-25T09:06:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.842+0000 7eff03e00640 1 Processor -- start 2024-04-25T09:06:32.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.842+0000 7eff03e00640 1 -- start start 2024-04-25T09:06:32.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7eff03e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 0x7eff04138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7eff03e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff0415b520 0x7eff041f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7eff03e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415eb90 0x7eff040cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7eff03e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0415b900 con 0x7eff0415b520 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7eff03e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0406d1c0 con 0x7eff0406c440 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7eff03e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0415ba80 con 0x7eff0415eb90 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff0415b520 0x7eff041f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff0415b520 0x7eff041f3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42586/0 (socket says 172.21.15.62:42586) 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.843+0000 7efefa000640 1 -- 172.21.15.62:0/2294415731 learned_addr learned my addr 172.21.15.62:0/2294415731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7eff02a00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 0x7eff04138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7efefa000640 1 -- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415eb90 msgr2=0x7eff040cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7eff03400640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415eb90 0x7eff040cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7efefa000640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415eb90 0x7eff040cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7efefa000640 1 -- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 msgr2=0x7eff04138200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7efefa000640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 0x7eff04138200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7efefa000640 1 -- 172.21.15.62:0/2294415731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff040cf140 con 0x7eff0415b520 2024-04-25T09:06:32.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7eff02a00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 0x7eff04138200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:06:32.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7eff03400640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415eb90 0x7eff040cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:06:32.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7efefa000640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff0415b520 0x7eff041f3a70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efeec000c00 tx=0x7efeec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.844+0000 7efefb400640 1 -- 172.21.15.62:0/2294415731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec041020 con 0x7eff0415b520 2024-04-25T09:06:32.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.845+0000 7efefb400640 1 -- 172.21.15.62:0/2294415731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec03d620 con 0x7eff0415b520 2024-04-25T09:06:32.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.845+0000 7efefb400640 1 -- 172.21.15.62:0/2294415731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec037040 con 0x7eff0415b520 2024-04-25T09:06:32.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.845+0000 7eff03e00640 1 -- 172.21.15.62:0/2294415731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff04144c60 con 0x7eff0415b520 2024-04-25T09:06:32.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.845+0000 7eff03e00640 1 -- 172.21.15.62:0/2294415731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff04143430 con 0x7eff0415b520 2024-04-25T09:06:32.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.847+0000 7efefb400640 1 -- 172.21.15.62:0/2294415731 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efeec044020 con 0x7eff0415b520 2024-04-25T09:06:32.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.848+0000 7efefb400640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efee006b4d0 0x7efee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.848+0000 7efefb400640 1 -- 172.21.15.62:0/2294415731 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efeec0aa9b0 con 0x7eff0415b520 2024-04-25T09:06:32.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.848+0000 7eff0c5c5740 1 Processor -- start 2024-04-25T09:06:32.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.849+0000 7eff02a00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efee006b4d0 0x7efee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.849+0000 7eff0c5c5740 1 -- start start 2024-04-25T09:06:32.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52e40 0x557524c057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c35f60 0x557524c352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35820 0x557524c0a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557524b6e9f0 con 0x557524c35f60 2024-04-25T09:06:32.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557524b17eb0 con 0x557524c35820 2024-04-25T09:06:32.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557524b6aa90 con 0x557524c52e40 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff02a00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efee006b4d0 0x7efee006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff041f6f40 tx=0x7efef4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c35f60 0x557524c352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c35f60 0x557524c352e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42598/0 (socket says 172.21.15.62:42598) 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7efefa000640 1 -- 172.21.15.62:0/2543066251 learned_addr learned my addr 172.21.15.62:0/2543066251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff02a00640 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52e40 0x557524c057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.850+0000 7eff03400640 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35820 0x557524c0a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efefa000640 1 -- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52e40 msgr2=0x557524c057d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efefa000640 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52e40 0x557524c057d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efefa000640 1 -- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35820 msgr2=0x557524c0a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efefa000640 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35820 0x557524c0a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efefa000640 1 -- 172.21.15.62:0/2543066251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55752494f940 con 0x557524c35f60 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7eff02a00640 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52e40 0x557524c057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:06:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efefa000640 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c35f60 0x557524c352e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efeec02a860 tx=0x7efeec078f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3d291da1ba024e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efef3000640 1 -- 172.21.15.62:0/2543066251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec07d160 con 0x557524c35f60 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efef3000640 1 -- 172.21.15.62:0/2543066251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec081720 con 0x557524c35f60 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.851+0000 7efef3000640 1 -- 172.21.15.62:0/2543066251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec075680 con 0x557524c35f60 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 -- 172.21.15.62:0/2543066251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c35f60 msgr2=0x557524c352e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c35f60 0x557524c352e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efeec02a860 tx=0x7efeec078f10 comp rx=0 tx=0).stop 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 -- 172.21.15.62:0/2543066251 shutdown_connections 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c35f60 0x557524c352e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35820 0x557524c0a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/2543066251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52e40 0x557524c057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 -- 172.21.15.62:0/2543066251 >> 172.21.15.62:0/2543066251 conn(0x557524c07a40 msgr2=0x557524c502e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 -- 172.21.15.62:0/2543066251 shutdown_connections 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.852+0000 7eff0c5c5740 1 -- 172.21.15.62:0/2543066251 wait complete. 2024-04-25T09:06:32.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.853+0000 7eff0c5c5740 1 -- start start 2024-04-25T09:06:32.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.853+0000 7eff0c5c5740 1 Processor -- start 2024-04-25T09:06:32.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c34770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c38450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c601b0 0x557524c25710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557524b6e9f0 con 0x557524c52e40 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557524b0d370 con 0x557524c35f60 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557524c09bf0 con 0x557524c601b0 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7eff0c5c5740 10 client.0 _ll_register_callbacks cb 0x557524882d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c38450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c38450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42612/0 (socket says 172.21.15.62:42612) 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.854+0000 7efefa000640 1 -- 172.21.15.62:0/1802284724 learned_addr learned my addr 172.21.15.62:0/1802284724 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7eff02a00640 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c34770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efefa000640 1 -- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c601b0 msgr2=0x557524c25710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7eff03400640 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c601b0 0x557524c25710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efefa000640 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c601b0 0x557524c25710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efefa000640 1 -- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 msgr2=0x557524c34770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efefa000640 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c34770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efefa000640 1 -- 172.21.15.62:0/1802284724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575249b00d0 con 0x557524c52e40 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7eff02a00640 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c34770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efefa000640 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c38450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efeec02a860 tx=0x7efeec02a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7454bb2cfc47e194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efee7400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efee7400640 1 -- 172.21.15.62:0/1802284724 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec081490 con 0x557524c52e40 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7efee7400640 1 -- 172.21.15.62:0/1802284724 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efeec031c00 con 0x557524c52e40 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.855+0000 7eff0c5c5740 1 -- 172.21.15.62:0/1802284724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efeec081720 con 0x557524c52e40 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7efee7400640 1 -- 172.21.15.62:0/1802284724 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeec031e00 con 0x557524c52e40 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff0c5c5740 10 client.5289 fetch_fsmap learned FSMap version 17 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff0c5c5740 1 -- 172.21.15.62:0/1802284724 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x557524843230 con 0x557524c52e40 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7efee7400640 1 -- 172.21.15.62:0/1802284724 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec081720 con 0x557524c52e40 2024-04-25T09:06:32.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7efee7400640 1 -- 172.21.15.62:0/1802284724 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efeec03a030 con 0x557524c52e40 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff0c5c5740 10 client.5289 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff0c5c5740 10 client.5289 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff0c5c5740 20 client.5289 populate_metadata read hostname 'smithi062' 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557524c38990 0x557524c63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff0c5c5740 4 client.5289 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff0c5c5740 1 -- 172.21.15.62:0/1802284724 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557524b7bf30 con 0x557524c38990 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.856+0000 7eff02a00640 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557524c38990 0x557524c63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.857+0000 7eff02a00640 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557524c38990 0x557524c63780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70467a1a851aa58 server_cookie=350ff3a280494e33 in_seq=0 out_seq=0 2024-04-25T09:06:32.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.857+0000 7efee7400640 10 client.5289 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:32.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7efee7400640 1 -- 172.21.15.62:0/1802284724 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557524b7bf30 con 0x557524c38990 2024-04-25T09:06:32.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7efee7400640 10 client.5289 handle_command_reply: tid=0 2024-04-25T09:06:32.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 client.5289 shutdown 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 -- 172.21.15.62:0/1802284724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 msgr2=0x557524c38450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c38450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efeec02a860 tx=0x7efeec02a7f0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 -- 172.21.15.62:0/1802284724 shutdown_connections 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557524c38990 0x557524c63780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c38450 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c601b0 0x557524c25710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/1802284724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c34770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7efee7400640 0 client.5289 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 -- 172.21.15.62:0/1802284724 >> 172.21.15.62:0/1802284724 conn(0x557524c35820 msgr2=0x557524c04e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 -- 172.21.15.62:0/1802284724 shutdown_connections 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.858+0000 7eff0c5c5740 1 -- 172.21.15.62:0/1802284724 wait complete. 2024-04-25T09:06:32.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.859+0000 7eff0c5c5740 20 client.5289 trim_cache size 0 max 16384 2024-04-25T09:06:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.875+0000 7eff0c5c5740 1 Processor -- start 2024-04-25T09:06:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.875+0000 7eff0c5c5740 1 -- start start 2024-04-25T09:06:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.876+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c4f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.876+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c31410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.876+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c32050 0x557524c37e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.876+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff041ffe30 con 0x557524c52e40 2024-04-25T09:06:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.876+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff041ff230 con 0x557524c35f60 2024-04-25T09:06:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.876+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff04143f80 con 0x557524c32050 2024-04-25T09:06:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c31410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c31410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42626/0 (socket says 172.21.15.62:42626) 2024-04-25T09:06:32.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 -- 172.21.15.62:0/67805953 learned_addr learned my addr 172.21.15.62:0/67805953 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 -- 172.21.15.62:0/67805953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c32050 msgr2=0x557524c37e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 --2- 172.21.15.62:0/67805953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c32050 0x557524c37e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 -- 172.21.15.62:0/67805953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 msgr2=0x557524c4f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 --2- 172.21.15.62:0/67805953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c4f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 -- 172.21.15.62:0/67805953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557524c387c0 con 0x557524c52e40 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.877+0000 7efefa000640 1 --2- 172.21.15.62:0/67805953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c31410 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efeec078810 tx=0x7efeec08e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1cfb0a74da62ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7efef2600640 1 -- 172.21.15.62:0/67805953 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec09b020 con 0x557524c52e40 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7efef2600640 1 -- 172.21.15.62:0/67805953 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec07d160 con 0x557524c52e40 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7efef2600640 1 -- 172.21.15.62:0/67805953 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec081290 con 0x557524c52e40 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7eff0c5c5740 1 -- 172.21.15.62:0/67805953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 msgr2=0x557524c31410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/67805953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c31410 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efeec078810 tx=0x7efeec08e070 comp rx=0 tx=0).stop 2024-04-25T09:06:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7eff0c5c5740 1 -- 172.21.15.62:0/67805953 shutdown_connections 2024-04-25T09:06:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/67805953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c52e40 0x557524c31410 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/67805953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c32050 0x557524c37e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/67805953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c4f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7eff0c5c5740 1 -- 172.21.15.62:0/67805953 >> 172.21.15.62:0/67805953 conn(0x557524c35820 msgr2=0x557524c52270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.878+0000 7eff0c5c5740 1 -- 172.21.15.62:0/67805953 shutdown_connections 2024-04-25T09:06:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.879+0000 7eff0c5c5740 1 -- 172.21.15.62:0/67805953 wait complete. 2024-04-25T09:06:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.880+0000 7eff0c5c5740 1 -- start start 2024-04-25T09:06:32.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.880+0000 7eff0c5c5740 1 Processor -- start 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c32050 0x557524cb4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c04420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff0c5c5740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52a00 0x557524c04960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557524c5fe10 con 0x557524c32050 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff04144580 con 0x557524c35f60 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff0c5c5740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557524c32860 con 0x557524c52a00 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff0c5c5740 10 client.0 _ll_register_callbacks cb 0x7efeec07f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff02a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c32050 0x557524cb4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff02a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c32050 0x557524cb4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42642/0 (socket says 172.21.15.62:42642) 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.881+0000 7eff02a00640 1 -- 172.21.15.62:0/826419940 learned_addr learned my addr 172.21.15.62:0/826419940 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:32.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7eff02a00640 1 -- 172.21.15.62:0/826419940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52a00 msgr2=0x557524c04960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7eff02a00640 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52a00 0x557524c04960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7efefa000640 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c04420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7eff02a00640 1 -- 172.21.15.62:0/826419940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 msgr2=0x557524c04420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:32.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7eff02a00640 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c04420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7eff02a00640 1 -- 172.21.15.62:0/826419940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557524c330e0 con 0x557524c32050 2024-04-25T09:06:32.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7eff02a00640 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c32050 0x557524cb4af0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x557524b6eb40 tx=0x7efef4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b8ceebf1525e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:32.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7efee7400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:06:32.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7efee7400640 1 -- 172.21.15.62:0/826419940 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4028020 con 0x557524c32050 2024-04-25T09:06:32.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7efee7400640 1 -- 172.21.15.62:0/826419940 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efef4024430 con 0x557524c32050 2024-04-25T09:06:32.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7eff0c5c5740 1 -- 172.21.15.62:0/826419940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efeec07d160 con 0x557524c32050 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.882+0000 7efee7400640 1 -- 172.21.15.62:0/826419940 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efef400f040 con 0x557524c32050 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7eff0c5c5740 10 client.5295 fetch_fsmap learned FSMap version 17 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7eff0c5c5740 1 -- 172.21.15.62:0/826419940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x557524c72fd0 con 0x557524c32050 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7efee7400640 1 -- 172.21.15.62:0/826419940 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec07d160 con 0x557524c32050 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7efee7400640 1 -- 172.21.15.62:0/826419940 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efef4035020 con 0x557524c32050 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7eff0c5c5740 10 client.5295 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7eff0c5c5740 10 client.5295 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7eff0c5c5740 20 client.5295 populate_metadata read hostname 'smithi062' 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557524c4c810 0x557524c45c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7eff0c5c5740 4 client.5295 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7eff0c5c5740 1 -- 172.21.15.62:0/826419940 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x557524b76280 con 0x557524c4c810 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.883+0000 7efefa000640 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557524c4c810 0x557524c45c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.884+0000 7efefa000640 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557524c4c810 0x557524c45c70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6bf664544d159f server_cookie=85d42f1fc375126c in_seq=0 out_seq=0 2024-04-25T09:06:32.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.884+0000 7efee7400640 10 client.5295 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:32.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.891+0000 7efee7400640 1 -- 172.21.15.62:0/826419940 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557524b76280 con 0x557524c4c810 2024-04-25T09:06:32.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.891+0000 7efee7400640 10 client.5295 handle_command_reply: tid=0 2024-04-25T09:06:32.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.891+0000 7eff0c5c5740 1 client.5295 shutdown 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 -- 172.21.15.62:0/826419940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c32050 msgr2=0x557524cb4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c32050 0x557524cb4af0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x557524b6eb40 tx=0x7efef4009190 comp rx=0 tx=0).stop 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 -- 172.21.15.62:0/826419940 shutdown_connections 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x557524c4c810 0x557524c45c70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557524c52a00 0x557524c04960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557524c35f60 0x557524c04420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 --2- 172.21.15.62:0/826419940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557524c32050 0x557524cb4af0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 -- 172.21.15.62:0/826419940 >> 172.21.15.62:0/826419940 conn(0x557524c35820 msgr2=0x557524c521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7efee7400640 0 client.5295 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 -- 172.21.15.62:0/826419940 shutdown_connections 2024-04-25T09:06:32.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.892+0000 7eff0c5c5740 1 -- 172.21.15.62:0/826419940 wait complete. 2024-04-25T09:06:32.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff0c5c5740 20 client.5295 trim_cache size 0 max 16384 2024-04-25T09:06:32.881 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:06:32.881 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:06:32.881 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "2d717523-943a-49c6-9715-fc393ba339e1", 2024-04-25T09:06:32.881 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:06:32.881 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:06:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 -- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efee006b4d0 msgr2=0x7efee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efee006b4d0 0x7efee006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff041f6f40 tx=0x7efef4008040 comp rx=0 tx=0).stop 2024-04-25T09:06:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 -- 172.21.15.62:0/2294415731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff0415b520 msgr2=0x7eff041f3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff0415b520 0x7eff041f3a70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efeec000c00 tx=0x7efeec0046c0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 -- 172.21.15.62:0/2294415731 shutdown_connections 2024-04-25T09:06:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efee006b4d0 0x7efee006d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff0415b520 0x7eff041f3a70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff0415eb90 0x7eff040cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 --2- 172.21.15.62:0/2294415731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff0406c440 0x7eff04138200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.893+0000 7eff03e00640 1 -- 172.21.15.62:0/2294415731 >> 172.21.15.62:0/2294415731 conn(0x7eff04066ad0 msgr2=0x7eff04147090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.894+0000 7eff03e00640 1 -- 172.21.15.62:0/2294415731 shutdown_connections 2024-04-25T09:06:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:32.894+0000 7eff03e00640 1 -- 172.21.15.62:0/2294415731 wait complete. 2024-04-25T09:06:32.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:06:32.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T09:06:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.127+0000 7f9b63200640 1 Processor -- start 2024-04-25T09:06:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.127+0000 7f9b63200640 1 -- start start 2024-04-25T09:06:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.129+0000 7f9b63200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b640ce990 0x7f9b640cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.129+0000 7f9b63200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9b64161bf0 0x7f9b64162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.129+0000 7f9b63200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b640d1c90 con 0x7f9b6415dd60 2024-04-25T09:06:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.129+0000 7f9b63200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b640cf330 con 0x7f9b64161bf0 2024-04-25T09:06:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.129+0000 7f9b63200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b640d1f70 con 0x7f9b640ce990 2024-04-25T09:06:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.129+0000 7f9b61e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9b6415dd60 0x7f9b640d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35024/0 (socket says 172.21.15.62:35024) 2024-04-25T09:06:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.129+0000 7f9b61e00640 1 -- 172.21.15.62:0/2636984521 learned_addr learned my addr 172.21.15.62:0/2636984521 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:06:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.129+0000 7f9b61400640 1 --2- 172.21.15.62:0/2636984521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b640ce990 0x7f9b640cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.130+0000 7f9b62800640 1 --2- 172.21.15.62:0/2636984521 >> v2:172.21.15.121:3300/0 conn(0x7f9b64161bf0 0x7f9b64162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.130+0000 7f9b60a00640 1 -- 172.21.15.62:0/2636984521 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134401083 0 0) 0x7f9b640d1c90 con 0x7f9b6415dd60 2024-04-25T09:06:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.130+0000 7f9b60a00640 1 -- 172.21.15.62:0/2636984521 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b48003520 con 0x7f9b6415dd60 2024-04-25T09:06:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.130+0000 7f9b60a00640 1 -- 172.21.15.62:0/2636984521 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3204299030 0 0) 0x7f9b48003520 con 0x7f9b6415dd60 2024-04-25T09:06:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.131+0000 7f9b61400640 1 -- 172.21.15.62:0/2636984521 >> v2:172.21.15.121:3300/0 conn(0x7f9b64161bf0 msgr2=0x7f9b64162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.131+0000 7f9b61400640 1 --2- 172.21.15.62:0/2636984521 >> v2:172.21.15.121:3300/0 conn(0x7f9b64161bf0 0x7f9b64162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.131+0000 7f9b61400640 1 -- 172.21.15.62:0/2636984521 >> v1:172.21.15.62:6789/0 conn(0x7f9b6415dd60 legacy=0x7f9b640d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.131+0000 7f9b61400640 1 -- 172.21.15.62:0/2636984521 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b640d2250 con 0x7f9b640ce990 2024-04-25T09:06:33.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.132+0000 7f9b61400640 1 --2- 172.21.15.62:0/2636984521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b640ce990 0x7f9b640cedf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b500097a0 tx=0x7f9b5002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a98f58315ec684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:33.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.133+0000 7f9b60a00640 1 -- 172.21.15.62:0/2636984521 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50037070 con 0x7f9b640ce990 2024-04-25T09:06:33.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.133+0000 7f9b60a00640 1 -- 172.21.15.62:0/2636984521 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5002ab10 con 0x7f9b640ce990 2024-04-25T09:06:33.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.133+0000 7f9b60a00640 1 -- 172.21.15.62:0/2636984521 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54001670 con 0x7f9b640ce990 2024-04-25T09:06:33.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.134+0000 7f9b63200640 1 -- 172.21.15.62:0/2636984521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b640ce990 msgr2=0x7f9b640cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.134+0000 7f9b63200640 1 --2- 172.21.15.62:0/2636984521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b640ce990 0x7f9b640cedf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9b500097a0 tx=0x7f9b5002a080 comp rx=0 tx=0).stop 2024-04-25T09:06:33.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.134+0000 7f9b63200640 1 -- 172.21.15.62:0/2636984521 shutdown_connections 2024-04-25T09:06:33.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.134+0000 7f9b63200640 1 --2- 172.21.15.62:0/2636984521 >> v2:172.21.15.121:3300/0 conn(0x7f9b64161bf0 0x7f9b64162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.134+0000 7f9b63200640 1 --2- 172.21.15.62:0/2636984521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b640ce990 0x7f9b640cedf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.134+0000 7f9b63200640 1 -- 172.21.15.62:0/2636984521 >> 172.21.15.62:0/2636984521 conn(0x7f9b6406b210 msgr2=0x7f9b640668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:33.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.134+0000 7f9b63200640 1 -- 172.21.15.62:0/2636984521 shutdown_connections 2024-04-25T09:06:33.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.134+0000 7f9b63200640 1 -- 172.21.15.62:0/2636984521 wait complete. 2024-04-25T09:06:33.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.137+0000 7f9b63200640 1 Processor -- start 2024-04-25T09:06:33.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.137+0000 7f9b63200640 1 -- start start 2024-04-25T09:06:33.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b63200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b640ce990 0x7f9b641f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b63200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b6415dd60 0x7f9b6415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b63200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64161bf0 0x7f9b6415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b63200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b641625c0 con 0x7f9b64161bf0 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b63200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6415e140 con 0x7f9b640ce990 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b63200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b64162740 con 0x7f9b6415dd60 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b62800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64161bf0 0x7f9b6415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b62800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64161bf0 0x7f9b6415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42652/0 (socket says 172.21.15.62:42652) 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b62800640 1 -- 172.21.15.62:0/2480813718 learned_addr learned my addr 172.21.15.62:0/2480813718 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.138+0000 7f9b61e00640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b640ce990 0x7f9b641f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.139+0000 7f9b61400640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b6415dd60 0x7f9b6415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.139+0000 7f9b61e00640 1 -- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b6415dd60 msgr2=0x7f9b6415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.139+0000 7f9b61e00640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b6415dd60 0x7f9b6415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.139+0000 7f9b61e00640 1 -- 172.21.15.62:0/2480813718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64161bf0 msgr2=0x7f9b6415c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.139+0000 7f9b61e00640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64161bf0 0x7f9b6415c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.139+0000 7f9b61e00640 1 -- 172.21.15.62:0/2480813718 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b640c75e0 con 0x7f9b640ce990 2024-04-25T09:06:33.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.139+0000 7f9b62800640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64161bf0 0x7f9b6415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:06:33.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.140+0000 7f9b61e00640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b640ce990 0x7f9b641f37d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c520 tx=0x7f9b5400c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:33.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.140+0000 7f9b5a000640 1 -- 172.21.15.62:0/2480813718 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54017070 con 0x7f9b640ce990 2024-04-25T09:06:33.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.141+0000 7f9b5a000640 1 -- 172.21.15.62:0/2480813718 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b54005d00 con 0x7f9b640ce990 2024-04-25T09:06:33.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.141+0000 7f9b5a000640 1 -- 172.21.15.62:0/2480813718 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b540072b0 con 0x7f9b640ce990 2024-04-25T09:06:33.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.141+0000 7f9b63200640 1 -- 172.21.15.62:0/2480813718 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b640b4070 con 0x7f9b640ce990 2024-04-25T09:06:33.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.141+0000 7f9b63200640 1 -- 172.21.15.62:0/2480813718 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b640c7b80 con 0x7f9b640ce990 2024-04-25T09:06:33.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.145+0000 7f9b5a000640 1 -- 172.21.15.62:0/2480813718 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b5401a070 con 0x7f9b640ce990 2024-04-25T09:06:33.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.145+0000 7f9b6b9aa740 1 Processor -- start 2024-04-25T09:06:33.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.145+0000 7f9b6b9aa740 1 -- start start 2024-04-25T09:06:33.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.146+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3537f40 0x562ef350f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.146+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef3538320 0x562ef3556980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.146+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350c450 0x562ef3556ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.146+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef3473c70 con 0x562ef350c450 2024-04-25T09:06:33.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.146+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef341ccf0 con 0x562ef3538320 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.146+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ef346fd10 con 0x562ef3537f40 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b5a000640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4406b4d0 0x7f9b4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b5a000640 1 -- 172.21.15.62:0/2480813718 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9b54089720 con 0x7f9b640ce990 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b61400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350c450 0x562ef3556ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b61400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350c450 0x562ef3556ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42658/0 (socket says 172.21.15.62:42658) 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b61400640 1 -- 172.21.15.62:0/2424084820 learned_addr learned my addr 172.21.15.62:0/2424084820 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b62800640 1 --2- 172.21.15.62:0/2424084820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3537f40 0x562ef350f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b61400640 1 -- 172.21.15.62:0/2424084820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3537f40 msgr2=0x562ef350f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:33.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b61400640 1 --2- 172.21.15.62:0/2424084820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3537f40 0x562ef350f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b61400640 1 -- 172.21.15.62:0/2424084820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef3538320 msgr2=0x562ef3556980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:06:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b61400640 1 --2- 172.21.15.62:0/2424084820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef3538320 0x562ef3556980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.147+0000 7f9b61400640 1 -- 172.21.15.62:0/2424084820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ef32527d0 con 0x562ef350c450 2024-04-25T09:06:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.148+0000 7f9b62800640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4406b4d0 0x7f9b4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.148+0000 7f9b61400640 1 --2- 172.21.15.62:0/2424084820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350c450 0x562ef3556ec0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9b500097a0 tx=0x7f9b5002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489fefb055e3a9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:33.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.149+0000 7f9b62800640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4406b4d0 0x7f9b4406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562ef3511360 tx=0x7f9b5c004a40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.149+0000 7f9b4e600640 1 -- 172.21.15.62:0/2424084820 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50037040 con 0x562ef350c450 2024-04-25T09:06:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.149+0000 7f9b4e600640 1 -- 172.21.15.62:0/2424084820 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5002ab10 con 0x562ef350c450 2024-04-25T09:06:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.149+0000 7f9b4e600640 1 -- 172.21.15.62:0/2424084820 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50032840 con 0x562ef350c450 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2424084820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350c450 msgr2=0x562ef3556ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2424084820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350c450 0x562ef3556ec0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9b500097a0 tx=0x7f9b5002c000 comp rx=0 tx=0).stop 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2424084820 shutdown_connections 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2424084820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350c450 0x562ef3556ec0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2424084820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef3538320 0x562ef3556980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2424084820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3537f40 0x562ef350f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2424084820 >> 172.21.15.62:0/2424084820 conn(0x562ef34d7a30 msgr2=0x562ef3555830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2424084820 shutdown_connections 2024-04-25T09:06:33.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.150+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2424084820 wait complete. 2024-04-25T09:06:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.151+0000 7f9b6b9aa740 1 -- start start 2024-04-25T09:06:33.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.152+0000 7f9b6b9aa740 1 Processor -- start 2024-04-25T09:06:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.153+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef354ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.153+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef3537f40 0x562ef35373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.153+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef356f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.153+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef3510cf0 con 0x562ef3537f40 2024-04-25T09:06:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.153+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef34123a0 con 0x562ef350c450 2024-04-25T09:06:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.153+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ef35562f0 con 0x562ef3538320 2024-04-25T09:06:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.154+0000 7f9b6b9aa740 10 client.0 _ll_register_callbacks cb 0x562ef342fb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:06:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.154+0000 7f9b61400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef3537f40 0x562ef35373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.154+0000 7f9b62800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef356f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.154+0000 7f9b61400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef3537f40 0x562ef35373e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42672/0 (socket says 172.21.15.62:42672) 2024-04-25T09:06:33.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.154+0000 7f9b61400640 1 -- 172.21.15.62:0/2383941567 learned_addr learned my addr 172.21.15.62:0/2383941567 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.155+0000 7f9b61400640 1 -- 172.21.15.62:0/2383941567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 msgr2=0x562ef356f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.155+0000 7f9b61400640 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef356f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.155+0000 7f9b61400640 1 -- 172.21.15.62:0/2383941567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 msgr2=0x562ef354ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.155+0000 7f9b61400640 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef354ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.155+0000 7f9b61400640 1 -- 172.21.15.62:0/2383941567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ef31883f0 con 0x562ef3537f40 2024-04-25T09:06:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.155+0000 7f9b62800640 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef356f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:06:33.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.155+0000 7f9b61400640 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef3537f40 0x562ef35373e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9b500097a0 tx=0x7f9b50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1523b71f630864f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.155+0000 7f9b42a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.156+0000 7f9b42a00640 1 -- 172.21.15.62:0/2383941567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50053060 con 0x562ef3537f40 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.156+0000 7f9b42a00640 1 -- 172.21.15.62:0/2383941567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9b50003820 con 0x562ef3537f40 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.156+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2383941567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b5002ab10 con 0x562ef3537f40 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.156+0000 7f9b42a00640 1 -- 172.21.15.62:0/2383941567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b50053060 con 0x562ef3537f40 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.156+0000 7f9b6b9aa740 10 client.5307 fetch_fsmap learned FSMap version 17 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.156+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2383941567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x562ef31480f0 con 0x562ef3537f40 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.157+0000 7f9b42a00640 1 -- 172.21.15.62:0/2383941567 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5002ab10 con 0x562ef3537f40 2024-04-25T09:06:33.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.157+0000 7f9b42a00640 1 -- 172.21.15.62:0/2383941567 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9b50003a20 con 0x562ef3537f40 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.157+0000 7f9b6b9aa740 10 client.5307 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.157+0000 7f9b6b9aa740 10 client.5307 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.157+0000 7f9b6b9aa740 20 client.5307 populate_metadata read hostname 'smithi062' 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.157+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x562ef356f800 0x562ef3557100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.157+0000 7f9b6b9aa740 4 client.5307 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.157+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2383941567 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ef34811b0 con 0x562ef356f800 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.158+0000 7f9b61e00640 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x562ef356f800 0x562ef3557100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.159+0000 7f9b61e00640 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x562ef356f800 0x562ef3557100 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36721b1e20dcebd1 server_cookie=bc71a8e2a249c91b in_seq=0 out_seq=0 2024-04-25T09:06:33.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.159+0000 7f9b42a00640 10 client.5307 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b42a00640 1 -- 172.21.15.62:0/2383941567 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ef34811b0 con 0x562ef356f800 2024-04-25T09:06:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b42a00640 10 client.5307 handle_command_reply: tid=0 2024-04-25T09:06:33.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b6b9aa740 1 client.5307 shutdown 2024-04-25T09:06:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2383941567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef3537f40 msgr2=0x562ef35373e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef3537f40 0x562ef35373e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9b500097a0 tx=0x7f9b50004730 comp rx=0 tx=0).stop 2024-04-25T09:06:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2383941567 shutdown_connections 2024-04-25T09:06:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x562ef356f800 0x562ef3557100 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef3537f40 0x562ef35373e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.160+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef356f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.161+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/2383941567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef354ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.161+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2383941567 >> 172.21.15.62:0/2383941567 conn(0x562ef34d7a30 msgr2=0x562ef3509bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.161+0000 7f9b42a00640 0 client.5307 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.161+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2383941567 shutdown_connections 2024-04-25T09:06:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.161+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/2383941567 wait complete. 2024-04-25T09:06:33.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.161+0000 7f9b6b9aa740 20 client.5307 trim_cache size 0 max 16384 2024-04-25T09:06:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.188+0000 7f9b6b9aa740 1 Processor -- start 2024-04-25T09:06:33.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.188+0000 7f9b6b9aa740 1 -- start start 2024-04-25T09:06:33.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.189+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef354bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.189+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3511f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.189+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef3509bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.189+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6406d890 con 0x562ef350b180 2024-04-25T09:06:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.189+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b64158c60 con 0x562ef350c450 2024-04-25T09:06:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.189+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b64158060 con 0x562ef3538320 2024-04-25T09:06:33.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.190+0000 7f9b61400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3511f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.190+0000 7f9b61e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef354bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.190+0000 7f9b61400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3511f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:51942/0 (socket says 172.21.15.62:51942) 2024-04-25T09:06:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.190+0000 7f9b61400640 1 -- 172.21.15.62:0/1058897479 learned_addr learned my addr 172.21.15.62:0/1058897479 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.190+0000 7f9b62800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef3509bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.191+0000 7f9b61400640 1 -- 172.21.15.62:0/1058897479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 msgr2=0x562ef354bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.191+0000 7f9b61400640 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef354bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.191+0000 7f9b61400640 1 -- 172.21.15.62:0/1058897479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 msgr2=0x562ef3509bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.191+0000 7f9b61400640 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef3509bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.191+0000 7f9b61400640 1 -- 172.21.15.62:0/1058897479 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ef350d1c0 con 0x562ef3538320 2024-04-25T09:06:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.191+0000 7f9b61e00640 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef354bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:06:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.191+0000 7f9b62800640 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef3509bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:06:33.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.192+0000 7f9b61400640 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3511f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b50001eb0 tx=0x7f9b50048070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd3942d7ffc4a9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:33.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.193+0000 7f9b4dc00640 1 -- 172.21.15.62:0/1058897479 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50053060 con 0x562ef3538320 2024-04-25T09:06:33.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.193+0000 7f9b4dc00640 1 -- 172.21.15.62:0/1058897479 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5002ab10 con 0x562ef3538320 2024-04-25T09:06:33.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.193+0000 7f9b4dc00640 1 -- 172.21.15.62:0/1058897479 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50037070 con 0x562ef3538320 2024-04-25T09:06:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/1058897479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 msgr2=0x562ef3511f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3511f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b50001eb0 tx=0x7f9b50048070 comp rx=0 tx=0).stop 2024-04-25T09:06:33.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/1058897479 shutdown_connections 2024-04-25T09:06:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef3509bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3511f40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/1058897479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef354bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/1058897479 >> 172.21.15.62:0/1058897479 conn(0x562ef34d7a30 msgr2=0x562ef356f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/1058897479 shutdown_connections 2024-04-25T09:06:33.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.194+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/1058897479 wait complete. 2024-04-25T09:06:33.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.195+0000 7f9b6b9aa740 1 -- start start 2024-04-25T09:06:33.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.195+0000 7f9b6b9aa740 1 Processor -- start 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef356fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef3550ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b6b9aa740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3551220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ef350b600 con 0x562ef350b180 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b64158660 con 0x562ef350c450 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b6b9aa740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ef35109b0 con 0x562ef3538320 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b6b9aa740 10 client.0 _ll_register_callbacks cb 0x7f9b50040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b61e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef356fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b61e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef356fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42684/0 (socket says 172.21.15.62:42684) 2024-04-25T09:06:33.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b61e00640 1 -- 172.21.15.62:0/3177513204 learned_addr learned my addr 172.21.15.62:0/3177513204 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:06:33.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.197+0000 7f9b61400640 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef3550ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.198+0000 7f9b62800640 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3551220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.198+0000 7f9b61e00640 1 -- 172.21.15.62:0/3177513204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 msgr2=0x562ef3551220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:33.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.198+0000 7f9b61e00640 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3551220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.198+0000 7f9b61e00640 1 -- 172.21.15.62:0/3177513204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 msgr2=0x562ef3550ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:06:33.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.198+0000 7f9b61e00640 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef3550ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.198+0000 7f9b61e00640 1 -- 172.21.15.62:0/3177513204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ef350df40 con 0x562ef350b180 2024-04-25T09:06:33.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.199+0000 7f9b61e00640 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef356fd60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9b54054290 tx=0x7f9b54001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247e7d9046d1a45b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:06:33.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.199+0000 7f9b42a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:06:33.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.199+0000 7f9b42a00640 1 -- 172.21.15.62:0/3177513204 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54085070 con 0x562ef350b180 2024-04-25T09:06:33.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.199+0000 7f9b42a00640 1 -- 172.21.15.62:0/3177513204 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9b54056130 con 0x562ef350b180 2024-04-25T09:06:33.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.199+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/3177513204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b5002ab10 con 0x562ef350b180 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b42a00640 1 -- 172.21.15.62:0/3177513204 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b5405e0b0 con 0x562ef350b180 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b6b9aa740 10 client.5313 fetch_fsmap learned FSMap version 17 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/3177513204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x562ef3556010 con 0x562ef350b180 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b42a00640 1 -- 172.21.15.62:0/3177513204 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5002ab10 con 0x562ef350b180 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b42a00640 1 -- 172.21.15.62:0/3177513204 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9b540544a0 con 0x562ef350b180 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b6b9aa740 10 client.5313 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b6b9aa740 10 client.5313 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b6b9aa740 20 client.5313 populate_metadata read hostname 'smithi062' 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x562ef3581740 0x562ef3534e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b6b9aa740 4 client.5313 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.200+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/3177513204 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ef347b500 con 0x562ef3581740 2024-04-25T09:06:33.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.201+0000 7f9b61400640 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x562ef3581740 0x562ef3534e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:06:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.202+0000 7f9b42a00640 10 client.5313 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.202+0000 7f9b61400640 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x562ef3581740 0x562ef3534e10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2423c95cc29bdef server_cookie=8558077097430c57 in_seq=0 out_seq=0 2024-04-25T09:06:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.202+0000 7f9b42a00640 1 -- 172.21.15.62:0/3177513204 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x562ef347b500 con 0x562ef3581740 2024-04-25T09:06:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.202+0000 7f9b42a00640 10 client.5313 handle_command_reply: tid=0 2024-04-25T09:06:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.202+0000 7f9b6b9aa740 1 client.5313 shutdown 2024-04-25T09:06:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/3177513204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 msgr2=0x562ef356fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef356fd60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9b54054290 tx=0x7f9b54001e70 comp rx=0 tx=0).stop 2024-04-25T09:06:33.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/3177513204 shutdown_connections 2024-04-25T09:06:33.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x562ef3581740 0x562ef3534e10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ef3538320 0x562ef3551220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ef350c450 0x562ef3550ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 --2- 172.21.15.62:0/3177513204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ef350b180 0x562ef356fd60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/3177513204 >> 172.21.15.62:0/3177513204 conn(0x562ef34d7a30 msgr2=0x562ef354b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b42a00640 0 client.5313 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.203+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/3177513204 shutdown_connections 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.204+0000 7f9b6b9aa740 1 -- 172.21.15.62:0/3177513204 wait complete. 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.204+0000 7f9b6b9aa740 20 client.5313 trim_cache size 0 max 16384 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "scrub active (761 inodes in the stack)", 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-25T09:06:33.192 INFO:teuthology.orchestra.run.smithi062.stdout: "2d717523-943a-49c6-9715-fc393ba339e1": { 2024-04-25T09:06:33.193 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-04-25T09:06:33.193 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "2d717523-943a-49c6-9715-fc393ba339e1", 2024-04-25T09:06:33.193 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T09:06:33.193 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:33.193 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:06:33.193 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:06:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.205+0000 7f9b63200640 1 -- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4406b4d0 msgr2=0x7f9b4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.205+0000 7f9b63200640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4406b4d0 0x7f9b4406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562ef3511360 tx=0x7f9b5c004a40 comp rx=0 tx=0).stop 2024-04-25T09:06:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.205+0000 7f9b63200640 1 -- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b640ce990 msgr2=0x7f9b641f37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:06:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.205+0000 7f9b63200640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b640ce990 0x7f9b641f37d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c520 tx=0x7f9b5400c9e0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.206+0000 7f9b63200640 1 -- 172.21.15.62:0/2480813718 shutdown_connections 2024-04-25T09:06:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.206+0000 7f9b63200640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4406b4d0 0x7f9b4406d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.206+0000 7f9b63200640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b64161bf0 0x7f9b6415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.206+0000 7f9b63200640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b6415dd60 0x7f9b6415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.206+0000 7f9b63200640 1 --2- 172.21.15.62:0/2480813718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b640ce990 0x7f9b641f37d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:06:33.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.206+0000 7f9b63200640 1 -- 172.21.15.62:0/2480813718 >> 172.21.15.62:0/2480813718 conn(0x7f9b6406b210 msgr2=0x7f9b640caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:06:33.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.206+0000 7f9b63200640 1 -- 172.21.15.62:0/2480813718 shutdown_connections 2024-04-25T09:06:33.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:06:33.206+0000 7f9b63200640 1 -- 172.21.15.62:0/2480813718 wait complete. 2024-04-25T09:06:33.219 INFO:tasks.cephfs.filesystem:scrub status for tag:2d717523-943a-49c6-9715-fc393ba339e1 - {'path': '/', 'tag': '2d717523-943a-49c6-9715-fc393ba339e1', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T09:06:39.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:06:39.167 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:03.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:03.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T09:07:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.455+0000 7ff19ae00640 1 Processor -- start 2024-04-25T09:07:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.455+0000 7ff19ae00640 1 -- start start 2024-04-25T09:07:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.456+0000 7ff19ae00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1940d2fb0 0x7ff1940ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.456+0000 7ff19ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1940d3960 0x7ff1940d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.456+0000 7ff19ae00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1940ce780 con 0x7ff19415dd50 2024-04-25T09:07:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.456+0000 7ff19ae00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1940d42f0 con 0x7ff1940d2fb0 2024-04-25T09:07:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.456+0000 7ff19ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940d3440 con 0x7ff1940d3960 2024-04-25T09:07:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.457+0000 7ff199a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff19415dd50 0x7ff1940d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34890/0 (socket says 172.21.15.62:34890) 2024-04-25T09:07:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.457+0000 7ff199000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1940d2fb0 0x7ff1940ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.457+0000 7ff199a00640 1 -- 172.21.15.62:0/1266488700 learned_addr learned my addr 172.21.15.62:0/1266488700 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.457+0000 7ff19a400640 1 --2- 172.21.15.62:0/1266488700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1940d3960 0x7ff1940d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.458+0000 7ff199000640 1 -- 172.21.15.62:0/1266488700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1940d3960 msgr2=0x7ff1940d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.458+0000 7ff199000640 1 --2- 172.21.15.62:0/1266488700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1940d3960 0x7ff1940d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.458+0000 7ff199000640 1 -- 172.21.15.62:0/1266488700 >> v1:172.21.15.62:6789/0 conn(0x7ff19415dd50 legacy=0x7ff1940d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.458+0000 7ff199000640 1 -- 172.21.15.62:0/1266488700 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940ce1c0 con 0x7ff1940d2fb0 2024-04-25T09:07:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.458+0000 7ff193e00640 1 -- 172.21.15.62:0/1266488700 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828241363 0 0) 0x7ff1940ce780 con 0x7ff19415dd50 2024-04-25T09:07:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.459+0000 7ff199000640 1 --2- 172.21.15.62:0/1266488700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d2fb0 0x7ff1940ce9c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff1800097a0 tx=0x7ff18002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6fec39d43fba67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.459+0000 7ff193e00640 1 -- 172.21.15.62:0/1266488700 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180037070 con 0x7ff1940d2fb0 2024-04-25T09:07:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.459+0000 7ff193e00640 1 -- 172.21.15.62:0/1266488700 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff180032440 con 0x7ff1940d2fb0 2024-04-25T09:07:03.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.459+0000 7ff193e00640 1 -- 172.21.15.62:0/1266488700 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180032810 con 0x7ff1940d2fb0 2024-04-25T09:07:03.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.460+0000 7ff19ae00640 1 -- 172.21.15.62:0/1266488700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d2fb0 msgr2=0x7ff1940ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.460+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1266488700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d2fb0 0x7ff1940ce9c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff1800097a0 tx=0x7ff18002a180 comp rx=0 tx=0).stop 2024-04-25T09:07:03.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.460+0000 7ff19ae00640 1 -- 172.21.15.62:0/1266488700 shutdown_connections 2024-04-25T09:07:03.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.460+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1266488700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1940d3960 0x7ff1940d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.460+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1266488700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d2fb0 0x7ff1940ce9c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.460+0000 7ff19ae00640 1 -- 172.21.15.62:0/1266488700 >> 172.21.15.62:0/1266488700 conn(0x7ff19406b000 msgr2=0x7ff19406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.460+0000 7ff19ae00640 1 -- 172.21.15.62:0/1266488700 shutdown_connections 2024-04-25T09:07:03.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.460+0000 7ff19ae00640 1 -- 172.21.15.62:0/1266488700 wait complete. 2024-04-25T09:07:03.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.462+0000 7ff19ae00640 1 Processor -- start 2024-04-25T09:07:03.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.462+0000 7ff19ae00640 1 -- start start 2024-04-25T09:07:03.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff19ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d3960 0x7ff1941e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff19ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff19415dd50 0x7ff1941605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff19ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1941f3660 0x7ff194160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff19ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940b6b00 con 0x7ff1940d3960 2024-04-25T09:07:03.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff19ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940cef00 con 0x7ff1941f3660 2024-04-25T09:07:03.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff19ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940b6c80 con 0x7ff19415dd50 2024-04-25T09:07:03.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff199a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d3960 0x7ff1941e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff199a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d3960 0x7ff1941e6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33266/0 (socket says 172.21.15.62:33266) 2024-04-25T09:07:03.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.463+0000 7ff199a00640 1 -- 172.21.15.62:0/1473682406 learned_addr learned my addr 172.21.15.62:0/1473682406 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff19a400640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1941f3660 0x7ff194160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff199000640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff19415dd50 0x7ff1941605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff199a00640 1 -- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff19415dd50 msgr2=0x7ff1941605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff199a00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff19415dd50 0x7ff1941605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff199a00640 1 -- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1941f3660 msgr2=0x7ff194160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff199a00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1941f3660 0x7ff194160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff199a00640 1 -- 172.21.15.62:0/1473682406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194161040 con 0x7ff1940d3960 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff199a00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d3960 0x7ff1941e6ee0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff18400c7c0 tx=0x7ff18400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff191600640 1 -- 172.21.15.62:0/1473682406 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184018070 con 0x7ff1940d3960 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff191600640 1 -- 172.21.15.62:0/1473682406 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18400a640 con 0x7ff1940d3960 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff191600640 1 -- 172.21.15.62:0/1473682406 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400aa30 con 0x7ff1940d3960 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.464+0000 7ff19ae00640 1 -- 172.21.15.62:0/1473682406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1940b42a0 con 0x7ff1940d3960 2024-04-25T09:07:03.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.465+0000 7ff19ae00640 1 -- 172.21.15.62:0/1473682406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1940b4b40 con 0x7ff1940d3960 2024-04-25T09:07:03.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.467+0000 7ff191600640 1 -- 172.21.15.62:0/1473682406 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff18401b070 con 0x7ff1940d3960 2024-04-25T09:07:03.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.468+0000 7ff191600640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff17406b520 0x7ff17406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.468+0000 7ff191600640 1 -- 172.21.15.62:0/1473682406 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff184089990 con 0x7ff1940d3960 2024-04-25T09:07:03.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.469+0000 7ff19f38e740 1 Processor -- start 2024-04-25T09:07:03.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.469+0000 7ff199000640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff17406b520 0x7ff17406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.469+0000 7ff19f38e740 1 -- start start 2024-04-25T09:07:03.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbdeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 0x55929cbdd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 0x55929cc07cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff19f38e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929cb40c70 con 0x55929cbdcd70 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff19f38e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929cae9cf0 con 0x55929cc19440 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff19f38e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55929cb3cd10 con 0x55929cc07830 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff199a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbdeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff199a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbdeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33280/0 (socket says 172.21.15.62:33280) 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff199a00640 1 -- 172.21.15.62:0/3930138912 learned_addr learned my addr 172.21.15.62:0/3930138912 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.470+0000 7ff199000640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff17406b520 0x7ff17406d9e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff180034000 tx=0x7ff18002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff199a00640 1 -- 172.21.15.62:0/3930138912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 msgr2=0x55929cc07cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff199a00640 1 --2- 172.21.15.62:0/3930138912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 0x55929cc07cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff199a00640 1 -- 172.21.15.62:0/3930138912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 msgr2=0x55929cbdd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff199000640 1 --2- 172.21.15.62:0/3930138912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 0x55929cbdd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff199a00640 1 --2- 172.21.15.62:0/3930138912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 0x55929cbdd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff199a00640 1 -- 172.21.15.62:0/3930138912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55929c91f7d0 con 0x55929cbdcd70 2024-04-25T09:07:03.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff199a00640 1 --2- 172.21.15.62:0/3930138912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbdeed0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff1840602d0 tx=0x7ff184054410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fc7dd3b962f63c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff18a000640 1 -- 172.21.15.62:0/3930138912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1840563f0 con 0x55929cbdcd70 2024-04-25T09:07:03.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff18a000640 1 -- 172.21.15.62:0/3930138912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff184054760 con 0x55929cbdcd70 2024-04-25T09:07:03.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.471+0000 7ff18a000640 1 -- 172.21.15.62:0/3930138912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184060570 con 0x55929cbdcd70 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 -- 172.21.15.62:0/3930138912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 msgr2=0x55929cbdeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3930138912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbdeed0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff1840602d0 tx=0x7ff184054410 comp rx=0 tx=0).stop 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 -- 172.21.15.62:0/3930138912 shutdown_connections 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3930138912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 0x55929cc07cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3930138912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 0x55929cbdd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3930138912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbdeed0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 -- 172.21.15.62:0/3930138912 >> 172.21.15.62:0/3930138912 conn(0x55929cba4a30 msgr2=0x55929cc22830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 -- 172.21.15.62:0/3930138912 shutdown_connections 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.472+0000 7ff19f38e740 1 -- 172.21.15.62:0/3930138912 wait complete. 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.473+0000 7ff19f38e740 1 -- start start 2024-04-25T09:07:03.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.473+0000 7ff19f38e740 1 Processor -- start 2024-04-25T09:07:03.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbf7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 0x55929cc08690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 0x55929cc03b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff19f38e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929cadf3a0 con 0x55929cbdcd70 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff19f38e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929cc05380 con 0x55929cc19440 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff19f38e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55929cc23cb0 con 0x55929cc07830 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff19f38e740 10 client.0 _ll_register_callbacks cb 0x55929cafcb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff199a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbf7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff199a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbf7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33288/0 (socket says 172.21.15.62:33288) 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.474+0000 7ff199a00640 1 -- 172.21.15.62:0/3304800190 learned_addr learned my addr 172.21.15.62:0/3304800190 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff19a400640 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 0x55929cc03b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff199000640 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 0x55929cc08690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff199a00640 1 -- 172.21.15.62:0/3304800190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 msgr2=0x55929cc08690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff199a00640 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 0x55929cc08690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff199a00640 1 -- 172.21.15.62:0/3304800190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 msgr2=0x55929cc03b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff199a00640 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 0x55929cc03b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff199a00640 1 -- 172.21.15.62:0/3304800190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55929c8553f0 con 0x55929cbdcd70 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff199a00640 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbf7050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff184058080 tx=0x7ff184001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511919df4983886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff17ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff17ea00640 1 -- 172.21.15.62:0/3304800190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184074060 con 0x55929cbdcd70 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff17ea00640 1 -- 172.21.15.62:0/3304800190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff18405e400 con 0x55929cbdcd70 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.475+0000 7ff19f38e740 1 -- 172.21.15.62:0/3304800190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff184054760 con 0x55929cbdcd70 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff17ea00640 1 -- 172.21.15.62:0/3304800190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff184057540 con 0x55929cbdcd70 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff19f38e740 10 client.5328 fetch_fsmap learned FSMap version 17 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff19f38e740 1 -- 172.21.15.62:0/3304800190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55929c8150f0 con 0x55929cbdcd70 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff17ea00640 1 -- 172.21.15.62:0/3304800190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff184054760 con 0x55929cbdcd70 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff17ea00640 1 -- 172.21.15.62:0/3304800190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1840576e0 con 0x55929cbdcd70 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff19f38e740 10 client.5328 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:07:03.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff19f38e740 10 client.5328 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:07:03.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff19f38e740 20 client.5328 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:03.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55929cc32480 0x55929cc04060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff19f38e740 4 client.5328 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:03.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.476+0000 7ff19f38e740 1 -- 172.21.15.62:0/3304800190 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55929cb4e1b0 con 0x55929cc32480 2024-04-25T09:07:03.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.477+0000 7ff199000640 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55929cc32480 0x55929cc04060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.477+0000 7ff199000640 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55929cc32480 0x55929cc04060 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e434f2cf111868 server_cookie=254d9ed580b7d3c7 in_seq=0 out_seq=0 2024-04-25T09:07:03.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.477+0000 7ff17ea00640 10 client.5328 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:07:03.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff17ea00640 1 -- 172.21.15.62:0/3304800190 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55929cb4e1b0 con 0x55929cc32480 2024-04-25T09:07:03.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff17ea00640 10 client.5328 handle_command_reply: tid=0 2024-04-25T09:07:03.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 client.5328 shutdown 2024-04-25T09:07:03.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 -- 172.21.15.62:0/3304800190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 msgr2=0x55929cbf7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbf7050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff184058080 tx=0x7ff184001250 comp rx=0 tx=0).stop 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 -- 172.21.15.62:0/3304800190 shutdown_connections 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55929cc32480 0x55929cc04060 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc19440 0x55929cc03b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc07830 0x55929cc08690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 --2- 172.21.15.62:0/3304800190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cbf7050 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 -- 172.21.15.62:0/3304800190 >> 172.21.15.62:0/3304800190 conn(0x55929cba4a30 msgr2=0x55929cc05560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff17ea00640 0 client.5328 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 -- 172.21.15.62:0/3304800190 shutdown_connections 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.478+0000 7ff19f38e740 1 -- 172.21.15.62:0/3304800190 wait complete. 2024-04-25T09:07:03.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.479+0000 7ff19f38e740 20 client.5328 trim_cache size 0 max 16384 2024-04-25T09:07:03.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.495+0000 7ff19f38e740 1 Processor -- start 2024-04-25T09:07:03.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.496+0000 7ff19f38e740 1 -- start start 2024-04-25T09:07:03.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.496+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cc22d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.496+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 0x55929cc0b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.496+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc19440 0x55929cc0b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.496+0000 7ff19f38e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941ffd60 con 0x55929cbdcd70 2024-04-25T09:07:03.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.496+0000 7ff19f38e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941fef40 con 0x55929cc07830 2024-04-25T09:07:03.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.496+0000 7ff19f38e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940b60b0 con 0x55929cc19440 2024-04-25T09:07:03.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cc22d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cc22d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33300/0 (socket says 172.21.15.62:33300) 2024-04-25T09:07:03.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 -- 172.21.15.62:0/2240895933 learned_addr learned my addr 172.21.15.62:0/2240895933 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 -- 172.21.15.62:0/2240895933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc19440 msgr2=0x55929cc0b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 --2- 172.21.15.62:0/2240895933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc19440 0x55929cc0b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 -- 172.21.15.62:0/2240895933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 msgr2=0x55929cc0b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 --2- 172.21.15.62:0/2240895933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 0x55929cc0b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 -- 172.21.15.62:0/2240895933 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55929cbf73c0 con 0x55929cbdcd70 2024-04-25T09:07:03.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.497+0000 7ff199a00640 1 --2- 172.21.15.62:0/2240895933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cc22d00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff18406ff20 tx=0x7ff184084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8dda25f7a53f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff189600640 1 -- 172.21.15.62:0/2240895933 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184074050 con 0x55929cbdcd70 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff189600640 1 -- 172.21.15.62:0/2240895933 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff184054760 con 0x55929cbdcd70 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff189600640 1 -- 172.21.15.62:0/2240895933 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18405e420 con 0x55929cbdcd70 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff19f38e740 1 -- 172.21.15.62:0/2240895933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 msgr2=0x55929cc22d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff19f38e740 1 --2- 172.21.15.62:0/2240895933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cc22d00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff18406ff20 tx=0x7ff184084040 comp rx=0 tx=0).stop 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff19f38e740 1 -- 172.21.15.62:0/2240895933 shutdown_connections 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff19f38e740 1 --2- 172.21.15.62:0/2240895933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cc19440 0x55929cc0b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff19f38e740 1 --2- 172.21.15.62:0/2240895933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 0x55929cc0b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff19f38e740 1 --2- 172.21.15.62:0/2240895933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cbdcd70 0x55929cc22d00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff19f38e740 1 -- 172.21.15.62:0/2240895933 >> 172.21.15.62:0/2240895933 conn(0x55929cba4a30 msgr2=0x55929cbd3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.498+0000 7ff19f38e740 1 -- 172.21.15.62:0/2240895933 shutdown_connections 2024-04-25T09:07:03.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.499+0000 7ff19f38e740 1 -- 172.21.15.62:0/2240895933 wait complete. 2024-04-25T09:07:03.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.499+0000 7ff19f38e740 1 -- start start 2024-04-25T09:07:03.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.499+0000 7ff19f38e740 1 Processor -- start 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.500+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cbdcd70 0x55929cc04030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.500+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 0x55929cbf5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.500+0000 7ff19f38e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cc19440 0x55929cc048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.500+0000 7ff19f38e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929cc257a0 con 0x55929cc19440 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.500+0000 7ff19f38e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941f75e0 con 0x55929cc07830 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.500+0000 7ff19f38e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55929cb4e1b0 con 0x55929cbdcd70 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.500+0000 7ff19f38e740 10 client.0 _ll_register_callbacks cb 0x7ff184015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff19a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cc19440 0x55929cc048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff19a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cc19440 0x55929cc048f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33314/0 (socket says 172.21.15.62:33314) 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff19a400640 1 -- 172.21.15.62:0/366720072 learned_addr learned my addr 172.21.15.62:0/366720072 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff199a00640 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cbdcd70 0x55929cc04030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff199000640 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 0x55929cbf5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff19a400640 1 -- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cbdcd70 msgr2=0x55929cc04030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff19a400640 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cbdcd70 0x55929cc04030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff19a400640 1 -- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 msgr2=0x55929cbf5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff19a400640 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 0x55929cbf5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff19a400640 1 -- 172.21.15.62:0/366720072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55929cc31d70 con 0x55929cc19440 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.501+0000 7ff199a00640 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cbdcd70 0x55929cc04030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff19a400640 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cc19440 0x55929cc048f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff1941f3a80 tx=0x7ff18c0105c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f5a0da5253cf22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff17ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff17ea00640 1 -- 172.21.15.62:0/366720072 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00f040 con 0x55929cc19440 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff17ea00640 1 -- 172.21.15.62:0/366720072 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff18c01c600 con 0x55929cc19440 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff19f38e740 1 -- 172.21.15.62:0/366720072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff184054760 con 0x55929cc19440 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff17ea00640 1 -- 172.21.15.62:0/366720072 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff18c024050 con 0x55929cc19440 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff19f38e740 10 client.5334 fetch_fsmap learned FSMap version 17 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff19f38e740 1 -- 172.21.15.62:0/366720072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55929cc71620 con 0x55929cc19440 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff17ea00640 1 -- 172.21.15.62:0/366720072 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff184054760 con 0x55929cc19440 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff17ea00640 1 -- 172.21.15.62:0/366720072 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff18c027020 con 0x55929cc19440 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.502+0000 7ff19f38e740 10 client.5334 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:07:03.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.503+0000 7ff19f38e740 10 client.5334 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:07:03.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.503+0000 7ff19f38e740 20 client.5334 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:03.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.503+0000 7ff19f38e740 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55929cc04e30 0x55929cc0aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.503+0000 7ff19f38e740 4 client.5334 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:07:03.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.503+0000 7ff19f38e740 1 -- 172.21.15.62:0/366720072 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55929cb48500 con 0x55929cc04e30 2024-04-25T09:07:03.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.503+0000 7ff199a00640 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55929cc04e30 0x55929cc0aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.503+0000 7ff199a00640 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55929cc04e30 0x55929cc0aee0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93a2504ebc3a158 server_cookie=dd0900457fc117ce in_seq=0 out_seq=0 2024-04-25T09:07:03.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.503+0000 7ff17ea00640 10 client.5334 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:07:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff17ea00640 1 -- 172.21.15.62:0/366720072 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55929cb48500 con 0x55929cc04e30 2024-04-25T09:07:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff17ea00640 10 client.5334 handle_command_reply: tid=0 2024-04-25T09:07:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 client.5334 shutdown 2024-04-25T09:07:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 -- 172.21.15.62:0/366720072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cc19440 msgr2=0x55929cc048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cc19440 0x55929cc048f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff1941f3a80 tx=0x7ff18c0105c0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 -- 172.21.15.62:0/366720072 shutdown_connections 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x55929cc04e30 0x55929cc0aee0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55929cc19440 0x55929cc048f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55929cc07830 0x55929cbf5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 --2- 172.21.15.62:0/366720072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55929cbdcd70 0x55929cc04030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 -- 172.21.15.62:0/366720072 >> 172.21.15.62:0/366720072 conn(0x55929cba4a30 msgr2=0x55929cc01660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff17ea00640 0 client.5334 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 -- 172.21.15.62:0/366720072 shutdown_connections 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 1 -- 172.21.15.62:0/366720072 wait complete. 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.504+0000 7ff19f38e740 20 client.5334 trim_cache size 0 max 16384 2024-04-25T09:07:03.496 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-25T09:07:03.497 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:07:03.497 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:03.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 -- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff17406b520 msgr2=0x7ff17406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff17406b520 0x7ff17406d9e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff180034000 tx=0x7ff18002e000 comp rx=0 tx=0).stop 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 -- 172.21.15.62:0/1473682406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d3960 msgr2=0x7ff1941e6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d3960 0x7ff1941e6ee0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff18400c7c0 tx=0x7ff18400cc80 comp rx=0 tx=0).stop 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 -- 172.21.15.62:0/1473682406 shutdown_connections 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff17406b520 0x7ff17406d9e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1941f3660 0x7ff194160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff19415dd50 0x7ff1941605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 --2- 172.21.15.62:0/1473682406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1940d3960 0x7ff1941e6ee0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 -- 172.21.15.62:0/1473682406 >> 172.21.15.62:0/1473682406 conn(0x7ff19406b000 msgr2=0x7ff1940cb120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 -- 172.21.15.62:0/1473682406 shutdown_connections 2024-04-25T09:07:03.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.505+0000 7ff19ae00640 1 -- 172.21.15.62:0/1473682406 wait complete. 2024-04-25T09:07:03.510 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:07:03.510 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:03.511 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T09:07:03.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.721+0000 7f50a7200640 1 Processor -- start 2024-04-25T09:07:03.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.721+0000 7f50a7200640 1 -- start start 2024-04-25T09:07:03.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.723+0000 7f50a7200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50a80b31a0 0x7f50a80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.723+0000 7f50a7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a80cd260 con 0x7f50a806d2c0 2024-04-25T09:07:03.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.723+0000 7f50a7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a80cf000 con 0x7f50a80ccde0 2024-04-25T09:07:03.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.723+0000 7f50a7200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a80cf2e0 con 0x7f50a80b31a0 2024-04-25T09:07:03.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.724+0000 7f50a5e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f50a806d2c0 0x7f50a806d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34896/0 (socket says 172.21.15.62:34896) 2024-04-25T09:07:03.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.724+0000 7f50a5e00640 1 -- 172.21.15.62:0/4029929148 learned_addr learned my addr 172.21.15.62:0/4029929148 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:03.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.724+0000 7f50a6800640 1 --2- 172.21.15.62:0/4029929148 >> v2:172.21.15.121:3300/0 conn(0x7f50a80b31a0 0x7f50a80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.724+0000 7f50a4a00640 1 -- 172.21.15.62:0/4029929148 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23626515 0 0) 0x7f50a80cd260 con 0x7f50a806d2c0 2024-04-25T09:07:03.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.724+0000 7f50a4a00640 1 -- 172.21.15.62:0/4029929148 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f508c003520 con 0x7f50a806d2c0 2024-04-25T09:07:03.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.725+0000 7f50a6800640 1 -- 172.21.15.62:0/4029929148 >> v1:172.21.15.121:6789/0 conn(0x7f50a80ccde0 legacy=0x7f50a80ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.725+0000 7f50a6800640 1 -- 172.21.15.62:0/4029929148 >> v1:172.21.15.62:6789/0 conn(0x7f50a806d2c0 legacy=0x7f50a806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.725+0000 7f50a6800640 1 -- 172.21.15.62:0/4029929148 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a80b26c0 con 0x7f50a80b31a0 2024-04-25T09:07:03.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.726+0000 7f50a6800640 1 --2- 172.21.15.62:0/4029929148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a80b31a0 0x7f50a80b3f40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009d90 tx=0x7f50a002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31223bfab8cac70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.726+0000 7f50a4a00640 1 -- 172.21.15.62:0/4029929148 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0007740 con 0x7f50a80b31a0 2024-04-25T09:07:03.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a4a00640 1 -- 172.21.15.62:0/4029929148 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a00078e0 con 0x7f50a80b31a0 2024-04-25T09:07:03.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a4a00640 1 -- 172.21.15.62:0/4029929148 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0007c90 con 0x7f50a80b31a0 2024-04-25T09:07:03.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a7200640 1 -- 172.21.15.62:0/4029929148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a80b31a0 msgr2=0x7f50a80b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a7200640 1 --2- 172.21.15.62:0/4029929148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a80b31a0 0x7f50a80b3f40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009d90 tx=0x7f50a002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a7200640 1 -- 172.21.15.62:0/4029929148 shutdown_connections 2024-04-25T09:07:03.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a7200640 1 --2- 172.21.15.62:0/4029929148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a80b31a0 0x7f50a80b3f40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a7200640 1 -- 172.21.15.62:0/4029929148 >> 172.21.15.62:0/4029929148 conn(0x7f50a8066f20 msgr2=0x7f50a80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a7200640 1 -- 172.21.15.62:0/4029929148 shutdown_connections 2024-04-25T09:07:03.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.727+0000 7f50a7200640 1 -- 172.21.15.62:0/4029929148 wait complete. 2024-04-25T09:07:03.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.729+0000 7f50a7200640 1 Processor -- start 2024-04-25T09:07:03.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.729+0000 7f50a7200640 1 -- start start 2024-04-25T09:07:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.730+0000 7f50a7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a806d2c0 0x7f50a81e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.730+0000 7f50a7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a80b31a0 0x7f50a815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.730+0000 7f50a7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a80ccde0 0x7f50a815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.730+0000 7f50a7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81448d0 con 0x7f50a806d2c0 2024-04-25T09:07:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.730+0000 7f50a7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80b2c80 con 0x7f50a80ccde0 2024-04-25T09:07:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.730+0000 7f50a7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8144a50 con 0x7f50a80b31a0 2024-04-25T09:07:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a806d2c0 0x7f50a81e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a806d2c0 0x7f50a81e83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33326/0 (socket says 172.21.15.62:33326) 2024-04-25T09:07:03.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5e00640 1 -- 172.21.15.62:0/2480833834 learned_addr learned my addr 172.21.15.62:0/2480833834 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5e00640 1 -- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a80b31a0 msgr2=0x7f50a815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5400640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a80b31a0 0x7f50a815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a80b31a0 0x7f50a815b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5e00640 1 -- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a80ccde0 msgr2=0x7f50a815b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a80ccde0 0x7f50a815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.731+0000 7f50a5e00640 1 -- 172.21.15.62:0/2480833834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a81e5730 con 0x7f50a806d2c0 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.732+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a806d2c0 0x7f50a81e83c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f509800c500 tx=0x7f509800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.732+0000 7f509e000640 1 -- 172.21.15.62:0/2480833834 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098017070 con 0x7f50a806d2c0 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.732+0000 7f509e000640 1 -- 172.21.15.62:0/2480833834 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5098005ce0 con 0x7f50a806d2c0 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.732+0000 7f509e000640 1 -- 172.21.15.62:0/2480833834 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098007330 con 0x7f50a806d2c0 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.732+0000 7f50a7200640 1 -- 172.21.15.62:0/2480833834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a81e4f10 con 0x7f50a806d2c0 2024-04-25T09:07:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.732+0000 7f50a7200640 1 -- 172.21.15.62:0/2480833834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a812fe90 con 0x7f50a806d2c0 2024-04-25T09:07:03.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.734+0000 7f509e000640 1 -- 172.21.15.62:0/2480833834 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f509801a070 con 0x7f50a806d2c0 2024-04-25T09:07:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.735+0000 7f509e000640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f508806b500 0x7f508806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.735+0000 7f509e000640 1 -- 172.21.15.62:0/2480833834 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5098089720 con 0x7f50a806d2c0 2024-04-25T09:07:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.736+0000 7f50a5400640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f508806b500 0x7f508806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.736+0000 7f50b0033740 1 Processor -- start 2024-04-25T09:07:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.736+0000 7f50b0033740 1 -- start start 2024-04-25T09:07:03.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50b0033740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 0x5590bcc51c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50b0033740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc21a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50b0033740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc21fb0 0x5590bcc49cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50b0033740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bcb8a870 con 0x5590bcc52290 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50b0033740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bcb33d60 con 0x5590bcc21fb0 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50b0033740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590bcb86910 con 0x5590bcc51870 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50a5400640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f508806b500 0x7f508806d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50a80b5b20 tx=0x7f5094009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc21a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc21a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33338/0 (socket says 172.21.15.62:33338) 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.737+0000 7f50a5400640 1 -- 172.21.15.62:0/3465368589 learned_addr learned my addr 172.21.15.62:0/3465368589 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a5e00640 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 0x5590bcc51c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a6800640 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc21fb0 0x5590bcc49cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a5400640 1 -- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 msgr2=0x5590bcc51c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a5400640 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 0x5590bcc51c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a5400640 1 -- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc21fb0 msgr2=0x5590bcc49cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a5400640 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc21fb0 0x5590bcc49cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a5400640 1 -- 172.21.15.62:0/3465368589 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590bc9697d0 con 0x5590bcc52290 2024-04-25T09:07:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a6800640 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc21fb0 0x5590bcc49cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.738+0000 7f50a5400640 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc21a70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f50940147b0 tx=0x7f5094010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcf33d3c56dfbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f5092a00640 1 -- 172.21.15.62:0/3465368589 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094024070 con 0x5590bcc52290 2024-04-25T09:07:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f5092a00640 1 -- 172.21.15.62:0/3465368589 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094012b50 con 0x5590bcc52290 2024-04-25T09:07:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f5092a00640 1 -- 172.21.15.62:0/3465368589 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509401f400 con 0x5590bcc52290 2024-04-25T09:07:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f50b0033740 1 -- 172.21.15.62:0/3465368589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 msgr2=0x5590bcc21a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f50b0033740 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc21a70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f50940147b0 tx=0x7f5094010000 comp rx=0 tx=0).stop 2024-04-25T09:07:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f50b0033740 1 -- 172.21.15.62:0/3465368589 shutdown_connections 2024-04-25T09:07:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f50b0033740 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc21a70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f50b0033740 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc21fb0 0x5590bcc49cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f50b0033740 1 --2- 172.21.15.62:0/3465368589 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 0x5590bcc51c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f50b0033740 1 -- 172.21.15.62:0/3465368589 >> 172.21.15.62:0/3465368589 conn(0x5590bcc23ef0 msgr2=0x5590bcc6c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.739+0000 7f50b0033740 1 -- 172.21.15.62:0/3465368589 shutdown_connections 2024-04-25T09:07:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.740+0000 7f50b0033740 1 -- 172.21.15.62:0/3465368589 wait complete. 2024-04-25T09:07:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.740+0000 7f50b0033740 1 -- start start 2024-04-25T09:07:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.740+0000 7f50b0033740 1 Processor -- start 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50b0033740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 0x5590bcc27a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50b0033740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 0x5590bcc6b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50b0033740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc6bb10 0x5590bcc26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50b0033740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bcc4dfc0 con 0x5590bcc6bb10 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50b0033740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bcb471c0 con 0x5590bcc52290 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50b0033740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590bcc4e260 con 0x5590bcc51870 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50b0033740 10 client.0 _ll_register_callbacks cb 0x5590bcb858f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc6bb10 0x5590bcc26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc6bb10 0x5590bcc26760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33340/0 (socket says 172.21.15.62:33340) 2024-04-25T09:07:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a6800640 1 -- 172.21.15.62:0/2530025978 learned_addr learned my addr 172.21.15.62:0/2530025978 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 0x5590bcc27a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a5400640 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 0x5590bcc6b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a6800640 1 -- 172.21.15.62:0/2530025978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 msgr2=0x5590bcc27a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a6800640 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 0x5590bcc27a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a6800640 1 -- 172.21.15.62:0/2530025978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 msgr2=0x5590bcc6b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a6800640 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 0x5590bcc6b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.742+0000 7f50a6800640 1 -- 172.21.15.62:0/2530025978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590bc85f0f0 con 0x5590bcc6bb10 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.743+0000 7f50a6800640 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc6bb10 0x5590bcc26760 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f50a002ace0 tx=0x7f50a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbaa722f17df77c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.743+0000 7f5087400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.743+0000 7f5087400640 1 -- 172.21.15.62:0/2530025978 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0009450 con 0x5590bcc6bb10 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.743+0000 7f5087400640 1 -- 172.21.15.62:0/2530025978 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f50a0004780 con 0x5590bcc6bb10 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.743+0000 7f50b0033740 1 -- 172.21.15.62:0/2530025978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5094012b50 con 0x5590bcc6bb10 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.743+0000 7f5087400640 1 -- 172.21.15.62:0/2530025978 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50a004c050 con 0x5590bcc6bb10 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.743+0000 7f50b0033740 10 client.5346 fetch_fsmap learned FSMap version 17 2024-04-25T09:07:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.743+0000 7f50b0033740 1 -- 172.21.15.62:0/2530025978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5590bc9c9f60 con 0x5590bcc6bb10 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f5087400640 1 -- 172.21.15.62:0/2530025978 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094012b50 con 0x5590bcc6bb10 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f5087400640 1 -- 172.21.15.62:0/2530025978 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f50a004f020 con 0x5590bcc6bb10 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f50b0033740 10 client.5346 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f50b0033740 10 client.5346 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f50b0033740 20 client.5346 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f50b0033740 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x5590bcc4f880 0x5590bcc4fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f50b0033740 4 client.5346 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f50b0033740 1 -- 172.21.15.62:0/2530025978 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590bcb60690 con 0x5590bcc4f880 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.744+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x5590bcc4f880 0x5590bcc4fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.745+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x5590bcc4f880 0x5590bcc4fc90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=181a3457605c422b server_cookie=b518bab380b29956 in_seq=0 out_seq=0 2024-04-25T09:07:03.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.745+0000 7f5087400640 10 client.5346 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:07:03.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.745+0000 7f5087400640 1 -- 172.21.15.62:0/2530025978 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590bcb60690 con 0x5590bcc4f880 2024-04-25T09:07:03.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.745+0000 7f5087400640 10 client.5346 handle_command_reply: tid=0 2024-04-25T09:07:03.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.745+0000 7f50b0033740 1 client.5346 shutdown 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 -- 172.21.15.62:0/2530025978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc6bb10 msgr2=0x5590bcc26760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc6bb10 0x5590bcc26760 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f50a002ace0 tx=0x7f50a0004a40 comp rx=0 tx=0).stop 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 -- 172.21.15.62:0/2530025978 shutdown_connections 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x5590bcc4f880 0x5590bcc4fc90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc6bb10 0x5590bcc26760 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 0x5590bcc6b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 --2- 172.21.15.62:0/2530025978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc51870 0x5590bcc27a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f5087400640 0 client.5346 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 -- 172.21.15.62:0/2530025978 >> 172.21.15.62:0/2530025978 conn(0x5590bcc21fb0 msgr2=0x5590bcc6c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 -- 172.21.15.62:0/2530025978 shutdown_connections 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 1 -- 172.21.15.62:0/2530025978 wait complete. 2024-04-25T09:07:03.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.746+0000 7f50b0033740 20 client.5346 trim_cache size 0 max 16384 2024-04-25T09:07:03.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.762+0000 7f50b0033740 1 Processor -- start 2024-04-25T09:07:03.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.762+0000 7f50b0033740 1 -- start start 2024-04-25T09:07:03.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.763+0000 7f50b0033740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc51870 0x5590bcccd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.763+0000 7f50b0033740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc25430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.763+0000 7f50b0033740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 0x5590bcc25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.763+0000 7f50b0033740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81ffe20 con 0x5590bcc52290 2024-04-25T09:07:03.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.763+0000 7f50b0033740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81fef40 con 0x5590bcc51870 2024-04-25T09:07:03.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.763+0000 7f50b0033740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8130fa0 con 0x5590bcc6bb10 2024-04-25T09:07:03.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc25430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc25430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33344/0 (socket says 172.21.15.62:33344) 2024-04-25T09:07:03.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5400640 1 -- 172.21.15.62:0/2102937958 learned_addr learned my addr 172.21.15.62:0/2102937958 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc51870 0x5590bcccd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a6800640 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 0x5590bcc25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5400640 1 -- 172.21.15.62:0/2102937958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 msgr2=0x5590bcc25970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5400640 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 0x5590bcc25970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5400640 1 -- 172.21.15.62:0/2102937958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc51870 msgr2=0x5590bcccd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5400640 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc51870 0x5590bcccd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.764+0000 7f50a5400640 1 -- 172.21.15.62:0/2102937958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590bcc26ad0 con 0x5590bcc52290 2024-04-25T09:07:03.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.765+0000 7f50a5400640 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc25430 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f50940127a0 tx=0x7f5094012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32f5b07e748e1d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.767+0000 7f5092000640 1 -- 172.21.15.62:0/2102937958 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094024040 con 0x5590bcc52290 2024-04-25T09:07:03.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.767+0000 7f5092000640 1 -- 172.21.15.62:0/2102937958 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094012b70 con 0x5590bcc52290 2024-04-25T09:07:03.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.767+0000 7f5092000640 1 -- 172.21.15.62:0/2102937958 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094028410 con 0x5590bcc52290 2024-04-25T09:07:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 -- 172.21.15.62:0/2102937958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 msgr2=0x5590bcc25430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc25430 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f50940127a0 tx=0x7f5094012000 comp rx=0 tx=0).stop 2024-04-25T09:07:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 -- 172.21.15.62:0/2102937958 shutdown_connections 2024-04-25T09:07:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc52290 0x5590bcc25430 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 0x5590bcc25970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 --2- 172.21.15.62:0/2102937958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc51870 0x5590bcccd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 -- 172.21.15.62:0/2102937958 >> 172.21.15.62:0/2102937958 conn(0x5590bcc21fb0 msgr2=0x5590bcc6b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 -- 172.21.15.62:0/2102937958 shutdown_connections 2024-04-25T09:07:03.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 -- 172.21.15.62:0/2102937958 wait complete. 2024-04-25T09:07:03.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.768+0000 7f50b0033740 1 -- start start 2024-04-25T09:07:03.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.769+0000 7f50b0033740 1 Processor -- start 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50b0033740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc51870 0x5590bcc6fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50b0033740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 0x5590bccc83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50b0033740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 0x5590bcc7b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50b0033740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bcc23350 con 0x5590bcc51870 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50b0033740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81315a0 con 0x5590bcc52290 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50b0033740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590bcc24af0 con 0x5590bcc6bb10 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50b0033740 10 client.0 _ll_register_callbacks cb 0x7f50940011e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50a5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc51870 0x5590bcc6fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50a5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc51870 0x5590bcc6fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33356/0 (socket says 172.21.15.62:33356) 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50a5e00640 1 -- 172.21.15.62:0/2145543850 learned_addr learned my addr 172.21.15.62:0/2145543850 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50a5e00640 1 -- 172.21.15.62:0/2145543850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 msgr2=0x5590bcc7b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 0x5590bcc7b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50a5e00640 1 -- 172.21.15.62:0/2145543850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 msgr2=0x5590bccc83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:03.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50a5400640 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 0x5590bccc83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.770+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 0x5590bccc83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f50a5e00640 1 -- 172.21.15.62:0/2145543850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590bcc51fc0 con 0x5590bcc51870 2024-04-25T09:07:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f50a5e00640 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc51870 0x5590bcc6fd90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f50980011e0 tx=0x7f509805e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c462da99da7316cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f5087400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f5087400640 1 -- 172.21.15.62:0/2145543850 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098074060 con 0x5590bcc51870 2024-04-25T09:07:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f5087400640 1 -- 172.21.15.62:0/2145543850 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5098058120 con 0x5590bcc51870 2024-04-25T09:07:03.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f50b0033740 1 -- 172.21.15.62:0/2145543850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5094012b70 con 0x5590bcc51870 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f5087400640 1 -- 172.21.15.62:0/2145543850 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5098058340 con 0x5590bcc51870 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f50b0033740 10 client.5352 fetch_fsmap learned FSMap version 17 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.771+0000 7f50b0033740 1 -- 172.21.15.62:0/2145543850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5590bc8381b0 con 0x5590bcc51870 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f5087400640 1 -- 172.21.15.62:0/2145543850 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094012b70 con 0x5590bcc51870 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f5087400640 1 -- 172.21.15.62:0/2145543850 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5098060140 con 0x5590bcc51870 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f50b0033740 10 client.5352 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f50b0033740 10 client.5352 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f50b0033740 20 client.5352 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f50b0033740 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x5590bcc20960 0x5590bcccae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f50b0033740 4 client.5352 mds_command: new command op to 5043 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f50b0033740 1 -- 172.21.15.62:0/2145543850 --> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5590bcb97db0 con 0x5590bcc20960 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.772+0000 7f50a5400640 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x5590bcc20960 0x5590bcccae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.773+0000 7f5087400640 10 client.5352 ms_handle_connect on v2:172.21.15.62:6833/274650142 2024-04-25T09:07:03.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.773+0000 7f50a5400640 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x5590bcc20960 0x5590bcccae50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86456ec8d7c4f4ba server_cookie=3fa2f7f84a009be8 in_seq=0 out_seq=0 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.773+0000 7f5087400640 1 -- 172.21.15.62:0/2145543850 <== mds.0 v2:172.21.15.62:6833/274650142 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17948 (crc 0 0 0) 0x5590bcb97db0 con 0x5590bcc20960 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.773+0000 7f5087400640 10 client.5352 handle_command_reply: tid=0 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.773+0000 7f50b0033740 1 client.5352 shutdown 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 -- 172.21.15.62:0/2145543850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc51870 msgr2=0x5590bcc6fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc51870 0x5590bcc6fd90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f50980011e0 tx=0x7f509805e040 comp rx=0 tx=0).stop 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 -- 172.21.15.62:0/2145543850 shutdown_connections 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.62:6833/274650142,v1:172.21.15.62:6835/274650142] conn(0x5590bcc20960 0x5590bcccae50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5590bcc6bb10 0x5590bcc7b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590bcc52290 0x5590bccc83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 --2- 172.21.15.62:0/2145543850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5590bcc51870 0x5590bcc6fd90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f5087400640 0 client.5352 ms_handle_reset on v2:172.21.15.62:6833/274650142 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 -- 172.21.15.62:0/2145543850 >> 172.21.15.62:0/2145543850 conn(0x5590bcc21fb0 msgr2=0x5590bcc25430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 -- 172.21.15.62:0/2145543850 shutdown_connections 2024-04-25T09:07:03.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 1 -- 172.21.15.62:0/2145543850 wait complete. 2024-04-25T09:07:03.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.774+0000 7f50b0033740 20 client.5352 trim_cache size 0 max 16384 2024-04-25T09:07:03.765 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:03.765 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:07:03.765 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 30758, 2024-04-25T09:07:03.765 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 51615, 2024-04-25T09:07:03.765 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 10929083, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 56806303, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 19, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 12.015534484, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 7.035570259, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 5.954775652, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.148734548, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 51613, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.969944342, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000076917 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 70285, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.041411096, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000589 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 4483, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 4477, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1618792, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 46003928, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 1.028734266, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.537799627, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.197738787, 2024-04-25T09:07:03.767 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.227940891, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4476, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.993082351, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000221868 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 126832, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 36912 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:07:03.768 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 3153, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 3186, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 800512, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 16002832, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.590451771, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.288585867, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.122501637, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.138283211, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3185, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.798536531, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000250717 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.769 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 111792 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-25T09:07:03.770 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 5797, 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 30.296445894, 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.005226228 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3850, 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.907691783, 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000755244 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-25T09:07:03.771 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 26010, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 26010, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 26010, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 511.642669716, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.019670998 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 12, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 17866, 2024-04-25T09:07:03.772 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 17866, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 17866, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 2, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 2, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 2, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 48677, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 48161, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 162, 2024-04-25T09:07:03.773 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:07:03.774 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 13, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 2000, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 965, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 4022, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 3933, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 60, 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 14053 2024-04-25T09:07:03.775 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 4127, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 3781, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 2230, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-25T09:07:03.776 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 27262, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-25T09:07:03.777 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 27262, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 39, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 39, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 4, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 96511015, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-25T09:07:03.778 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4087, 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 133.848301018, 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.032749767 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 14, 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 2016, 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 2002, 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 13, 2024-04-25T09:07:03.779 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 13, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 17866, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 19867, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 2001, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 2, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 7613, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 7611, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 97028, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215396 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 29624, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-25T09:07:03.780 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 26010, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 9, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2000, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.096022981, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001548011 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 29, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.007911484, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000272809 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.781 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6000, 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 14.487729938, 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002414621 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 7738, 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 16.053720950, 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002074660 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-25T09:07:03.782 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-25T09:07:03.783 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000116263, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000116263 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.784 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.006928934, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006928934 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.017439030, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.017439030 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.785 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.185213532, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.023151691 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2230, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 443.213560512, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.198750475 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.786 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.787 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.037012200, 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.037012200 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000453863, 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000453863 2024-04-25T09:07:03.788 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.789 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8000, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 34.536560029, 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004317070 2024-04-25T09:07:03.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-25T09:07:03.791 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 14342, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 14342, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 143, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:07:03.792 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:07:03.793 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 250661, 2024-04-25T09:07:03.794 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 92, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 440, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 5, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 5800, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 85, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-25T09:07:03.795 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 13453165, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 661217, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 10301, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 92937241, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 37, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 10301, 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-25T09:07:03.796 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 10301, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 65.426956424, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006351515 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 10301, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 10465 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 10301, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 2112, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 8189, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2024-04-25T09:07:03.797 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 11, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 97, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 4182, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 55, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 1781, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 2034, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 2015, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 22, 2024-04-25T09:07:03.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-25T09:07:03.799 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 37, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 5, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 8, 2024-04-25T09:07:03.800 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 8, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 220, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 48 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-25T09:07:03.801 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 32, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 3781, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 3781, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:07:03.802 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 38390, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 9886693, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 38390, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 38390, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 9886693, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-25T09:07:03.803 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 10301, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 93333370, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 10301, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 93333370, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.804 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 10301, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 10301, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 10301, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 10301, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.805 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 3781, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 396129, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 3781, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:07:03.806 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 95, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 396129, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x5652ab4246e0": { 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 27262, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 92316711, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:07:03.807 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 27262, 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4087, 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 92316711, 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:03.808 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 -- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f508806b500 msgr2=0x7f508806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f508806b500 0x7f508806d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50a80b5b20 tx=0x7f5094009040 comp rx=0 tx=0).stop 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 -- 172.21.15.62:0/2480833834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a806d2c0 msgr2=0x7f50a81e83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a806d2c0 0x7f50a81e83c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f509800c500 tx=0x7f509800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 -- 172.21.15.62:0/2480833834 shutdown_connections 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f508806b500 0x7f508806d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50a80ccde0 0x7f50a815b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50a80b31a0 0x7f50a815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 --2- 172.21.15.62:0/2480833834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50a806d2c0 0x7f50a81e83c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 -- 172.21.15.62:0/2480833834 >> 172.21.15.62:0/2480833834 conn(0x7f50a8066f20 msgr2=0x7f50a806d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 -- 172.21.15.62:0/2480833834 shutdown_connections 2024-04-25T09:07:03.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:03.775+0000 7f50a7200640 1 -- 172.21.15.62:0/2480833834 wait complete. 2024-04-25T09:07:03.811 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:07:03.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:07:03.926 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:07:04.027 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:07:04.027 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:07:04.027 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:04.088 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:07:04.088 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:04.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:04.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.371+0000 7f49a0a00640 1 Processor -- start 2024-04-25T09:07:04.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.371+0000 7f49a0a00640 1 -- start start 2024-04-25T09:07:04.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.373+0000 7f49a0a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f499c144c10 0x7f499c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.373+0000 7f49a0a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f499c0cee40 0x7f499c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.373+0000 7f49a0a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c0cf2d0 con 0x7f499c06d730 2024-04-25T09:07:04.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.373+0000 7f49a0a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c144ff0 con 0x7f499c144c10 2024-04-25T09:07:04.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.373+0000 7f49a0a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c06dba0 con 0x7f499c0cee40 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.373+0000 7f499a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f499c144c10 0x7f499c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.373+0000 7f499a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f499c144c10 0x7f499c15f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33366/0 (socket says 172.21.15.62:33366) 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.374+0000 7f499a800640 1 -- 172.21.15.62:0/279629774 learned_addr learned my addr 172.21.15.62:0/279629774 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.374+0000 7f4991e00640 1 --2- 172.21.15.62:0/279629774 >> v2:172.21.15.121:3300/0 conn(0x7f499c0cee40 0x7f499c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.374+0000 7f499a800640 1 -- 172.21.15.62:0/279629774 >> v2:172.21.15.121:3300/0 conn(0x7f499c0cee40 msgr2=0x7f499c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.374+0000 7f499a800640 1 --2- 172.21.15.62:0/279629774 >> v2:172.21.15.121:3300/0 conn(0x7f499c0cee40 0x7f499c06d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.374+0000 7f499a800640 1 -- 172.21.15.62:0/279629774 >> v1:172.21.15.121:6789/0 conn(0x7f499c06d730 legacy=0x7f499c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.374+0000 7f499a800640 1 -- 172.21.15.62:0/279629774 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c15faa0 con 0x7f499c144c10 2024-04-25T09:07:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.375+0000 7f499a800640 1 --2- 172.21.15.62:0/279629774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c15f450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f499c06e680 tx=0x7f498c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08dbd04d0b8acb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:04.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.376+0000 7f4999e00640 1 -- 172.21.15.62:0/279629774 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c036070 con 0x7f499c144c10 2024-04-25T09:07:04.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.376+0000 7f4999e00640 1 -- 172.21.15.62:0/279629774 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c031600 con 0x7f499c144c10 2024-04-25T09:07:04.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.376+0000 7f4999e00640 1 -- 172.21.15.62:0/279629774 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0319d0 con 0x7f499c144c10 2024-04-25T09:07:04.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.376+0000 7f49a0a00640 1 -- 172.21.15.62:0/279629774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 msgr2=0x7f499c15f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:04.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.376+0000 7f49a0a00640 1 --2- 172.21.15.62:0/279629774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c15f450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f499c06e680 tx=0x7f498c02a660 comp rx=0 tx=0).stop 2024-04-25T09:07:04.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.377+0000 7f49a0a00640 1 -- 172.21.15.62:0/279629774 shutdown_connections 2024-04-25T09:07:04.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.377+0000 7f49a0a00640 1 --2- 172.21.15.62:0/279629774 >> v2:172.21.15.121:3300/0 conn(0x7f499c0cee40 0x7f499c06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.377+0000 7f49a0a00640 1 --2- 172.21.15.62:0/279629774 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c15f450 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.377+0000 7f49a0a00640 1 -- 172.21.15.62:0/279629774 >> 172.21.15.62:0/279629774 conn(0x7f499c0b14d0 msgr2=0x7f499c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:04.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.377+0000 7f49a0a00640 1 -- 172.21.15.62:0/279629774 shutdown_connections 2024-04-25T09:07:04.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.377+0000 7f49a0a00640 1 -- 172.21.15.62:0/279629774 wait complete. 2024-04-25T09:07:04.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.378+0000 7f49a0a00640 1 Processor -- start 2024-04-25T09:07:04.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.379+0000 7f49a0a00640 1 -- start start 2024-04-25T09:07:04.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.379+0000 7f49a0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c06d730 0x7f499c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.379+0000 7f49a0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c0cee40 0x7f499c1f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f49a0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f49a0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0b41c0 con 0x7f499c144c10 2024-04-25T09:07:04.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f49a0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c160340 con 0x7f499c0cee40 2024-04-25T09:07:04.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f49a0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c160060 con 0x7f499c06d730 2024-04-25T09:07:04.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f499b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f499b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c1412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33368/0 (socket says 172.21.15.62:33368) 2024-04-25T09:07:04.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f499b200640 1 -- 172.21.15.62:0/1504457715 learned_addr learned my addr 172.21.15.62:0/1504457715 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:04.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f4991e00640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c0cee40 0x7f499c1f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.380+0000 7f499a800640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c06d730 0x7f499c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f499b200640 1 -- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c06d730 msgr2=0x7f499c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:04.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f499b200640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c06d730 0x7f499c15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f499b200640 1 -- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c0cee40 msgr2=0x7f499c1f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:04.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f499b200640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c0cee40 0x7f499c1f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f499b200640 1 -- 172.21.15.62:0/1504457715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c1f44b0 con 0x7f499c144c10 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f499a800640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c06d730 0x7f499c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f499b200640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c1412d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f499400c410 tx=0x7f499400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f4993400640 1 -- 172.21.15.62:0/1504457715 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994011790 con 0x7f499c144c10 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f4993400640 1 -- 172.21.15.62:0/1504457715 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4994017070 con 0x7f499c144c10 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f4993400640 1 -- 172.21.15.62:0/1504457715 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994011bb0 con 0x7f499c144c10 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c1f4790 con 0x7f499c144c10 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.381+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c0ca3f0 con 0x7f499c144c10 2024-04-25T09:07:04.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.383+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c04ef20 con 0x7f499c144c10 2024-04-25T09:07:04.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.384+0000 7f4993400640 1 -- 172.21.15.62:0/1504457715 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f499401c460 con 0x7f499c144c10 2024-04-25T09:07:04.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.385+0000 7f4993400640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f497806b4d0 0x7f497806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.385+0000 7f4993400640 1 -- 172.21.15.62:0/1504457715 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4994089ef0 con 0x7f499c144c10 2024-04-25T09:07:04.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.386+0000 7f499a800640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f497806b4d0 0x7f497806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.387+0000 7f499a800640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f497806b4d0 0x7f497806d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f498c033000 tx=0x7f498c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:04.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.390+0000 7f4993400640 1 -- 172.21.15.62:0/1504457715 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4994021020 con 0x7f499c144c10 2024-04-25T09:07:04.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.590+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f499c0ca5c0 con 0x7f499c144c10 2024-04-25T09:07:04.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.593+0000 7f4993400640 1 -- 172.21.15.62:0/1504457715 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16912 (secure 0 0 0) 0x7f4994058c40 con 0x7f499c144c10 2024-04-25T09:07:04.582 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:04.582 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":37,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:06:47.512613+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:23.626010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:24.669175+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.1d","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:04.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f497806b4d0 msgr2=0x7f497806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:04.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f497806b4d0 0x7f497806d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f498c033000 tx=0x7f498c009000 comp rx=0 tx=0).stop 2024-04-25T09:07:04.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 msgr2=0x7f499c1412d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c1412d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f499400c410 tx=0x7f499400c8d0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 shutdown_connections 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f497806b4d0 0x7f497806d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f499c144c10 0x7f499c1412d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f499c0cee40 0x7f499c1f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 --2- 172.21.15.62:0/1504457715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f499c06d730 0x7f499c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 >> 172.21.15.62:0/1504457715 conn(0x7f499c0b14d0 msgr2=0x7f499c0b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 shutdown_connections 2024-04-25T09:07:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.596+0000 7f49a0a00640 1 -- 172.21.15.62:0/1504457715 wait complete. 2024-04-25T09:07:04.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:04:23.626010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:04:24.669175+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:07:04.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:07:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.832+0000 7febe7200640 1 Processor -- start 2024-04-25T09:07:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.833+0000 7febe7200640 1 -- start start 2024-04-25T09:07:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.834+0000 7febe7200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7febe80ceb50 0x7febe806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.834+0000 7febe7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7febe806d800 0x7febe815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.834+0000 7febe7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe80b5560 con 0x7febe815e5f0 2024-04-25T09:07:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.834+0000 7febe7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe815ee30 con 0x7febe806d800 2024-04-25T09:07:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.834+0000 7febe7200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe81618b0 con 0x7febe80ceb50 2024-04-25T09:07:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.835+0000 7febdd400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7febe806d800 0x7febe815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.835+0000 7febdd400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7febe806d800 0x7febe815e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55200/0 (socket says 172.21.15.62:55200) 2024-04-25T09:07:04.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.835+0000 7febdd400640 1 -- 172.21.15.62:0/572254375 learned_addr learned my addr 172.21.15.62:0/572254375 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.835+0000 7febe5e00640 1 --2- 172.21.15.62:0/572254375 >> v2:172.21.15.121:3300/0 conn(0x7febe80ceb50 0x7febe806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.835+0000 7febdd400640 1 -- 172.21.15.62:0/572254375 >> v2:172.21.15.121:3300/0 conn(0x7febe80ceb50 msgr2=0x7febe806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.835+0000 7febdd400640 1 --2- 172.21.15.62:0/572254375 >> v2:172.21.15.121:3300/0 conn(0x7febe80ceb50 0x7febe806d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.835+0000 7febdd400640 1 -- 172.21.15.62:0/572254375 >> v1:172.21.15.121:6789/0 conn(0x7febe815e5f0 legacy=0x7febe80ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.835+0000 7febdd400640 1 -- 172.21.15.62:0/572254375 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8161b90 con 0x7febe806d800 2024-04-25T09:07:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.836+0000 7febdd400640 1 --2- 172.21.15.62:0/572254375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe806d800 0x7febe815e0b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7febe80af200 tx=0x7febd002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f86ab1ba6b66c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.837+0000 7febe5400640 1 -- 172.21.15.62:0/572254375 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0037070 con 0x7febe806d800 2024-04-25T09:07:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.837+0000 7febe5400640 1 -- 172.21.15.62:0/572254375 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd0032440 con 0x7febe806d800 2024-04-25T09:07:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.837+0000 7febe5400640 1 -- 172.21.15.62:0/572254375 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0032810 con 0x7febe806d800 2024-04-25T09:07:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.838+0000 7febe7200640 1 -- 172.21.15.62:0/572254375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe806d800 msgr2=0x7febe815e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.838+0000 7febe7200640 1 --2- 172.21.15.62:0/572254375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe806d800 0x7febe815e0b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7febe80af200 tx=0x7febd002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.838+0000 7febe7200640 1 -- 172.21.15.62:0/572254375 shutdown_connections 2024-04-25T09:07:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.838+0000 7febe7200640 1 --2- 172.21.15.62:0/572254375 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe806d800 0x7febe815e0b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.838+0000 7febe7200640 1 --2- 172.21.15.62:0/572254375 >> v2:172.21.15.121:3300/0 conn(0x7febe80ceb50 0x7febe806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.838+0000 7febe7200640 1 -- 172.21.15.62:0/572254375 >> 172.21.15.62:0/572254375 conn(0x7febe8066f20 msgr2=0x7febe80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.838+0000 7febe7200640 1 -- 172.21.15.62:0/572254375 shutdown_connections 2024-04-25T09:07:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.838+0000 7febe7200640 1 -- 172.21.15.62:0/572254375 wait complete. 2024-04-25T09:07:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.840+0000 7febe7200640 1 Processor -- start 2024-04-25T09:07:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.840+0000 7febe7200640 1 -- start start 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.841+0000 7febe7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe806d800 0x7febe8144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.841+0000 7febe7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe80ceb50 0x7febe81e9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.841+0000 7febe7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febe815e5f0 0x7febe8159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.841+0000 7febe7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80cf0b0 con 0x7febe80ceb50 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.841+0000 7febe7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe80cef30 con 0x7febe806d800 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.841+0000 7febe7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe8162390 con 0x7febe815e5f0 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febdd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe80ceb50 0x7febe81e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febdd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe80ceb50 0x7febe81e9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55216/0 (socket says 172.21.15.62:55216) 2024-04-25T09:07:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febdd400640 1 -- 172.21.15.62:0/3385557749 learned_addr learned my addr 172.21.15.62:0/3385557749 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febe6800640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febe815e5f0 0x7febe8159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febe5e00640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe806d800 0x7febe8144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febdd400640 1 -- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febe815e5f0 msgr2=0x7febe8159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febdd400640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febe815e5f0 0x7febe8159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febdd400640 1 -- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe806d800 msgr2=0x7febe8144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febdd400640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe806d800 0x7febe8144d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febdd400640 1 -- 172.21.15.62:0/3385557749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe81452d0 con 0x7febe80ceb50 2024-04-25T09:07:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febe6800640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febe815e5f0 0x7febe8159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.842+0000 7febe5e00640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe806d800 0x7febe8144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.843+0000 7febdd400640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe80ceb50 0x7febe81e9800 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7febd0000c00 tx=0x7febd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.843+0000 7febdea00640 1 -- 172.21.15.62:0/3385557749 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0041020 con 0x7febe80ceb50 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.843+0000 7febdea00640 1 -- 172.21.15.62:0/3385557749 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd00306c0 con 0x7febe80ceb50 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.843+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe81e8760 con 0x7febe80ceb50 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.843+0000 7febdea00640 1 -- 172.21.15.62:0/3385557749 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0037040 con 0x7febe80ceb50 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.843+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe81e8d20 con 0x7febe80ceb50 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.844+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe806abe0 con 0x7febe80ceb50 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.845+0000 7febdea00640 1 -- 172.21.15.62:0/3385557749 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febd0044020 con 0x7febe80ceb50 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.846+0000 7febdea00640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febc406b4d0 0x7febc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.846+0000 7febdea00640 1 -- 172.21.15.62:0/3385557749 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7febd00aabb0 con 0x7febe80ceb50 2024-04-25T09:07:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.847+0000 7febe5e00640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febc406b4d0 0x7febc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:04.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.848+0000 7febe5e00640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febc406b4d0 0x7febc406d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7febe80ae920 tx=0x7febd800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:04.850+0000 7febdea00640 1 -- 172.21.15.62:0/3385557749 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7febd004b030 con 0x7febe80ceb50 2024-04-25T09:07:05.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.050+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febe80ae390 con 0x7febe80ceb50 2024-04-25T09:07:05.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.052+0000 7febdea00640 1 -- 172.21.15.62:0/3385557749 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7febd0079900 con 0x7febe80ceb50 2024-04-25T09:07:05.040 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:07:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.054+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febc406b4d0 msgr2=0x7febc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.054+0000 7febe7200640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febc406b4d0 0x7febc406d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7febe80ae920 tx=0x7febd800b040 comp rx=0 tx=0).stop 2024-04-25T09:07:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.054+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe80ceb50 msgr2=0x7febe81e9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.054+0000 7febe7200640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe80ceb50 0x7febe81e9800 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7febd0000c00 tx=0x7febd00046c0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.055+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 shutdown_connections 2024-04-25T09:07:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.055+0000 7febe7200640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7febc406b4d0 0x7febc406d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.055+0000 7febe7200640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febe80ceb50 0x7febe81e9800 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.055+0000 7febe7200640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7febe815e5f0 0x7febe8159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.055+0000 7febe7200640 1 --2- 172.21.15.62:0/3385557749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe806d800 0x7febe8144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.055+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 >> 172.21.15.62:0/3385557749 conn(0x7febe8066f20 msgr2=0x7febe815f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.055+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 shutdown_connections 2024-04-25T09:07:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.055+0000 7febe7200640 1 -- 172.21.15.62:0/3385557749 wait complete. 2024-04-25T09:07:05.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:07:05.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.292+0000 7fe84fe00640 1 Processor -- start 2024-04-25T09:07:05.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.293+0000 7fe84fe00640 1 -- start start 2024-04-25T09:07:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.294+0000 7fe84fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe850163310 0x7fe850165700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.294+0000 7fe84fe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8500b4430 con 0x7fe8500b2d80 2024-04-25T09:07:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.294+0000 7fe84fe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8500b4710 con 0x7fe8500b54d0 2024-04-25T09:07:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.294+0000 7fe84fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500b49f0 con 0x7fe850163310 2024-04-25T09:07:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.295+0000 7fe84ea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe8500b2d80 0x7fe8500b3160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50890/0 (socket says 172.21.15.62:50890) 2024-04-25T09:07:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.295+0000 7fe84ea00640 1 -- 172.21.15.62:0/1914756112 learned_addr learned my addr 172.21.15.62:0/1914756112 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:05.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.295+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351348569 0 0) 0x7fe8500b4430 con 0x7fe8500b2d80 2024-04-25T09:07:05.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.295+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe834005730 con 0x7fe8500b2d80 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84f400640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe850163310 0x7fe850165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2888757603 0 0) 0x7fe834005730 con 0x7fe8500b2d80 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe850163310 msgr2=0x7fe850165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe850163310 0x7fe850165700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 >> v1:172.21.15.121:6789/0 conn(0x7fe8500b54d0 legacy=0x7fe8500ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8500b4cd0 con 0x7fe8500b2d80 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fe840001690 con 0x7fe8500b2d80 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 >> v1:172.21.15.62:6789/0 conn(0x7fe8500b2d80 legacy=0x7fe8500b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe850163310 0x7fe834005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe834006270 0x7fe834006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe834006f50 0x7fe8340073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500b49f0 con 0x7fe834006270 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83400de10 con 0x7fe834006f50 2024-04-25T09:07:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83400e010 con 0x7fe850163310 2024-04-25T09:07:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.296+0000 7fe84e000640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe834006270 0x7fe834006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.297+0000 7fe84ea00640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe850163310 0x7fe834005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.297+0000 7fe84e000640 1 -- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe850163310 msgr2=0x7fe834005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.297+0000 7fe84e000640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe850163310 0x7fe834005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.297+0000 7fe84e000640 1 -- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe834006f50 msgr2=0x7fe8340073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.297+0000 7fe84f400640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe834006f50 0x7fe8340073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.297+0000 7fe84e000640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe834006f50 0x7fe8340073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.297+0000 7fe84e000640 1 -- 172.21.15.62:0/1914756112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834005730 con 0x7fe834006270 2024-04-25T09:07:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.298+0000 7fe84e000640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe834006270 0x7fe834006a10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe83c001750 tx=0x7fe834007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff0aa7602cd632d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.298+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c001370 con 0x7fe834006270 2024-04-25T09:07:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.298+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c0052d0 con 0x7fe834006270 2024-04-25T09:07:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.298+0000 7fe84d600640 1 -- 172.21.15.62:0/1914756112 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c02abe0 con 0x7fe834006270 2024-04-25T09:07:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.299+0000 7fe84fe00640 1 -- 172.21.15.62:0/1914756112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe834006270 msgr2=0x7fe834006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.299+0000 7fe84fe00640 1 --2- 172.21.15.62:0/1914756112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe834006270 0x7fe834006a10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe83c001750 tx=0x7fe834007a40 comp rx=0 tx=0).stop 2024-04-25T09:07:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.299+0000 7fe84f400640 1 -- 172.21.15.62:0/1914756112 reap_dead start 2024-04-25T09:07:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.299+0000 7fe84fe00640 1 -- 172.21.15.62:0/1914756112 shutdown_connections 2024-04-25T09:07:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.299+0000 7fe84fe00640 1 -- 172.21.15.62:0/1914756112 >> 172.21.15.62:0/1914756112 conn(0x7fe850066f20 msgr2=0x7fe8500b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.299+0000 7fe84fe00640 1 -- 172.21.15.62:0/1914756112 shutdown_connections 2024-04-25T09:07:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.299+0000 7fe84fe00640 1 -- 172.21.15.62:0/1914756112 wait complete. 2024-04-25T09:07:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.301+0000 7fe84fe00640 1 Processor -- start 2024-04-25T09:07:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.301+0000 7fe84fe00640 1 -- start start 2024-04-25T09:07:05.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe834006270 0x7fe8501550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe85014a1c0 0x7fe85015d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8500ab390 0x7fe8500ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500b3870 con 0x7fe85014a1c0 2024-04-25T09:07:05.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500b40e0 con 0x7fe8500ab390 2024-04-25T09:07:05.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500ce9c0 con 0x7fe834006270 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe85014a1c0 0x7fe85015d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe85014a1c0 0x7fe85015d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55232/0 (socket says 172.21.15.62:55232) 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.302+0000 7fe84e000640 1 -- 172.21.15.62:0/3242709691 learned_addr learned my addr 172.21.15.62:0/3242709691 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.303+0000 7fe84ea00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe834006270 0x7fe8501550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.303+0000 7fe84f400640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8500ab390 0x7fe8500ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.303+0000 7fe84e000640 1 -- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe834006270 msgr2=0x7fe8501550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.303+0000 7fe84e000640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe834006270 0x7fe8501550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.303+0000 7fe84e000640 1 -- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8500ab390 msgr2=0x7fe8500ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.303+0000 7fe84e000640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8500ab390 0x7fe8500ab810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.303+0000 7fe84e000640 1 -- 172.21.15.62:0/3242709691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8501f7e00 con 0x7fe85014a1c0 2024-04-25T09:07:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.303+0000 7fe84e000640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe85014a1c0 0x7fe85015d3b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe83c008fd0 tx=0x7fe83c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.304+0000 7fe846a00640 1 -- 172.21.15.62:0/3242709691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c042070 con 0x7fe85014a1c0 2024-04-25T09:07:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.304+0000 7fe846a00640 1 -- 172.21.15.62:0/3242709691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c005480 con 0x7fe85014a1c0 2024-04-25T09:07:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.304+0000 7fe846a00640 1 -- 172.21.15.62:0/3242709691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c037040 con 0x7fe85014a1c0 2024-04-25T09:07:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.304+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8501f7040 con 0x7fe85014a1c0 2024-04-25T09:07:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.304+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8501f7600 con 0x7fe85014a1c0 2024-04-25T09:07:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.305+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85006abe0 con 0x7fe85014a1c0 2024-04-25T09:07:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.306+0000 7fe846a00640 1 -- 172.21.15.62:0/3242709691 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe83c045070 con 0x7fe85014a1c0 2024-04-25T09:07:05.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.307+0000 7fe846a00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe83006b500 0x7fe83006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.307+0000 7fe846a00640 1 -- 172.21.15.62:0/3242709691 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe83c0a9b70 con 0x7fe85014a1c0 2024-04-25T09:07:05.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.308+0000 7fe84ea00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe83006b500 0x7fe83006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.309+0000 7fe84ea00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe83006b500 0x7fe83006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe8500cd430 tx=0x7fe840000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:05.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.311+0000 7fe846a00640 1 -- 172.21.15.62:0/3242709691 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe83c078830 con 0x7fe85014a1c0 2024-04-25T09:07:05.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.511+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8501582d0 con 0x7fe85014a1c0 2024-04-25T09:07:05.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.512+0000 7fe846a00640 1 -- 172.21.15.62:0/3242709691 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe83c078130 con 0x7fe85014a1c0 2024-04-25T09:07:05.500 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.514+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe83006b500 msgr2=0x7fe83006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.514+0000 7fe84fe00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe83006b500 0x7fe83006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe8500cd430 tx=0x7fe840000ed0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.514+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe85014a1c0 msgr2=0x7fe85015d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.514+0000 7fe84fe00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe85014a1c0 0x7fe85015d3b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe83c008fd0 tx=0x7fe83c001120 comp rx=0 tx=0).stop 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.515+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 shutdown_connections 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.515+0000 7fe84fe00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe83006b500 0x7fe83006d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.515+0000 7fe84fe00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe85014a1c0 0x7fe85015d3b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.515+0000 7fe84fe00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8500ab390 0x7fe8500ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.515+0000 7fe84fe00640 1 --2- 172.21.15.62:0/3242709691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe834006270 0x7fe8501550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.515+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 >> 172.21.15.62:0/3242709691 conn(0x7fe850066f20 msgr2=0x7fe850159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:05.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.515+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 shutdown_connections 2024-04-25T09:07:05.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.515+0000 7fe84fe00640 1 -- 172.21.15.62:0/3242709691 wait complete. 2024-04-25T09:07:05.524 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:07:05.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.756+0000 7f645be00640 1 Processor -- start 2024-04-25T09:07:05.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.756+0000 7f645be00640 1 -- start start 2024-04-25T09:07:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.757+0000 7f645be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f645c0cde40 0x7f645c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.757+0000 7f645be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c0cd650 con 0x7f645c15b8f0 2024-04-25T09:07:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.758+0000 7f645be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c0cd370 con 0x7f645c0ccdb0 2024-04-25T09:07:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.758+0000 7f645be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0cf470 con 0x7f645c0cde40 2024-04-25T09:07:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.758+0000 7f645aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f645c15b8f0 0x7f645c0b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50900/0 (socket says 172.21.15.62:50900) 2024-04-25T09:07:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.758+0000 7f645aa00640 1 -- 172.21.15.62:0/672639133 learned_addr learned my addr 172.21.15.62:0/672639133 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.758+0000 7f645b400640 1 --2- 172.21.15.62:0/672639133 >> v2:172.21.15.121:3300/0 conn(0x7f645c0cde40 0x7f645c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.758+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993711630 0 0) 0x7f645c0cd650 con 0x7f645c15b8f0 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.758+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c003520 con 0x7f645c15b8f0 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3474559530 0 0) 0x7f643c003520 con 0x7f645c15b8f0 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 >> v2:172.21.15.121:3300/0 conn(0x7f645c0cde40 msgr2=0x7f645c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 --2- 172.21.15.62:0/672639133 >> v2:172.21.15.121:3300/0 conn(0x7f645c0cde40 0x7f645c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 >> v1:172.21.15.121:6789/0 conn(0x7f645c0ccdb0 legacy=0x7f645c0cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0cf750 con 0x7f645c15b8f0 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f644c001670 con 0x7f645c15b8f0 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 >> v1:172.21.15.62:6789/0 conn(0x7f645c15b8f0 legacy=0x7f645c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c003a00 0x7f643c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c004800 0x7f643c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f643c0051c0 0x7f643c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0cf470 con 0x7f643c004800 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c00c0c0 con 0x7f643c003a00 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c00c2c0 con 0x7f643c0051c0 2024-04-25T09:07:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.759+0000 7f645a000640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c004800 0x7f643c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.760+0000 7f645aa00640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c003a00 0x7f643c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.760+0000 7f645a000640 1 -- 172.21.15.62:0/672639133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f643c0051c0 msgr2=0x7f643c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.760+0000 7f645a000640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f643c0051c0 0x7f643c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.760+0000 7f645a000640 1 -- 172.21.15.62:0/672639133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c003a00 msgr2=0x7f643c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.760+0000 7f645a000640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c003a00 0x7f643c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.760+0000 7f645b400640 1 -- 172.21.15.62:0/672639133 reap_dead start 2024-04-25T09:07:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.760+0000 7f645a000640 1 -- 172.21.15.62:0/672639133 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c003520 con 0x7f643c004800 2024-04-25T09:07:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.761+0000 7f645a000640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c004800 0x7f643c004c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6444001e50 tx=0x7f643c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff303eeaf42848d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.761+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444001a70 con 0x7f643c004800 2024-04-25T09:07:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.761+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644402ae40 con 0x7f643c004800 2024-04-25T09:07:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.761+0000 7f6459600640 1 -- 172.21.15.62:0/672639133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444003bb0 con 0x7f643c004800 2024-04-25T09:07:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.762+0000 7f645be00640 1 -- 172.21.15.62:0/672639133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c004800 msgr2=0x7f643c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.762+0000 7f645be00640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c004800 0x7f643c004c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6444001e50 tx=0x7f643c005830 comp rx=0 tx=0).stop 2024-04-25T09:07:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.762+0000 7f645be00640 1 -- 172.21.15.62:0/672639133 shutdown_connections 2024-04-25T09:07:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.762+0000 7f645be00640 1 --2- 172.21.15.62:0/672639133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f643c004800 0x7f643c004c80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.762+0000 7f645be00640 1 -- 172.21.15.62:0/672639133 >> 172.21.15.62:0/672639133 conn(0x7f645c09aa70 msgr2=0x7f645c0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.762+0000 7f645be00640 1 -- 172.21.15.62:0/672639133 shutdown_connections 2024-04-25T09:07:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.762+0000 7f645be00640 1 -- 172.21.15.62:0/672639133 wait complete. 2024-04-25T09:07:05.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.764+0000 7f645be00640 1 Processor -- start 2024-04-25T09:07:05.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.764+0000 7f645be00640 1 -- start start 2024-04-25T09:07:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c004800 0x7f645c1f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f645c0c88d0 0x7f645c0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0ca880 0x7f645c0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0ce2d0 con 0x7f645c0ca880 2024-04-25T09:07:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c15bcd0 con 0x7f643c004800 2024-04-25T09:07:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0ce450 con 0x7f645c0c88d0 2024-04-25T09:07:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0ca880 0x7f645c0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0ca880 0x7f645c0cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55254/0 (socket says 172.21.15.62:55254) 2024-04-25T09:07:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645b400640 1 -- 172.21.15.62:0/644765149 learned_addr learned my addr 172.21.15.62:0/644765149 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645aa00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c004800 0x7f645c1f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645a000640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f645c0c88d0 0x7f645c0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.765+0000 7f645b400640 1 -- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f645c0c88d0 msgr2=0x7f645c0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645b400640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f645c0c88d0 0x7f645c0c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645b400640 1 -- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c004800 msgr2=0x7f645c1f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645b400640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c004800 0x7f645c1f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645b400640 1 -- 172.21.15.62:0/644765149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c1f4d90 con 0x7f645c0ca880 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645a000640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f645c0c88d0 0x7f645c0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645aa00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c004800 0x7f645c1f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645b400640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0ca880 0x7f645c0cad00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f645c0c50f0 tx=0x7f645400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f6452a00640 1 -- 172.21.15.62:0/644765149 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454017070 con 0x7f645c0ca880 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f6452a00640 1 -- 172.21.15.62:0/644765149 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645400c620 con 0x7f645c0ca880 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f6452a00640 1 -- 172.21.15.62:0/644765149 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400ca10 con 0x7f645c0ca880 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c0cb670 con 0x7f645c0ca880 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.766+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c15cfd0 con 0x7f645c0ca880 2024-04-25T09:07:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.768+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c0750e0 con 0x7f645c0ca880 2024-04-25T09:07:05.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.769+0000 7f6452a00640 1 -- 172.21.15.62:0/644765149 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f645401a070 con 0x7f645c0ca880 2024-04-25T09:07:05.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.770+0000 7f6452a00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f643806b500 0x7f643806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:05.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.770+0000 7f6452a00640 1 -- 172.21.15.62:0/644765149 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6454089d40 con 0x7f645c0ca880 2024-04-25T09:07:05.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.771+0000 7f645aa00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f643806b500 0x7f643806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:05.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.772+0000 7f645aa00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f643806b500 0x7f643806d9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f645c069350 tx=0x7f644c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:05.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.775+0000 7f6452a00640 1 -- 172.21.15.62:0/644765149 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6454058bc0 con 0x7f645c0ca880 2024-04-25T09:07:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.980+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f645c0cd650 con 0x7f645c0ca880 2024-04-25T09:07:05.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.981+0000 7f6452a00640 1 -- 172.21.15.62:0/644765149 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64540584c0 con 0x7f645c0ca880 2024-04-25T09:07:05.969 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:07:05.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.983+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f643806b500 msgr2=0x7f643806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.983+0000 7f645be00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f643806b500 0x7f643806d9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f645c069350 tx=0x7f644c009040 comp rx=0 tx=0).stop 2024-04-25T09:07:05.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.983+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0ca880 msgr2=0x7f645c0cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:05.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.983+0000 7f645be00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0ca880 0x7f645c0cad00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f645c0c50f0 tx=0x7f645400ec00 comp rx=0 tx=0).stop 2024-04-25T09:07:05.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.984+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 shutdown_connections 2024-04-25T09:07:05.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.984+0000 7f645be00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f643806b500 0x7f643806d9c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.984+0000 7f645be00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f645c0ca880 0x7f645c0cad00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.984+0000 7f645be00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f645c0c88d0 0x7f645c0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.984+0000 7f645be00640 1 --2- 172.21.15.62:0/644765149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f643c004800 0x7f645c1f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:05.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.984+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 >> 172.21.15.62:0/644765149 conn(0x7f645c09aa70 msgr2=0x7f645c0b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:05.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.984+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 shutdown_connections 2024-04-25T09:07:05.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:05.984+0000 7f645be00640 1 -- 172.21.15.62:0/644765149 wait complete. 2024-04-25T09:07:05.992 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:07:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.215+0000 7fd8cde00640 1 Processor -- start 2024-04-25T09:07:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.216+0000 7fd8cde00640 1 -- start start 2024-04-25T09:07:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.217+0000 7fd8cde00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd8c80cf5f0 0x7fd8c80b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.217+0000 7fd8cde00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c80cfaa0 0x7fd8c80b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.217+0000 7fd8cde00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c80cff60 con 0x7fd8c815c4e0 2024-04-25T09:07:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.217+0000 7fd8cde00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c80b3720 con 0x7fd8c80cf5f0 2024-04-25T09:07:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.217+0000 7fd8cde00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8070360 con 0x7fd8c80cfaa0 2024-04-25T09:07:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8cca00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd8c815c4e0 0x7fd8c815c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50902/0 (socket says 172.21.15.62:50902) 2024-04-25T09:07:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8cca00640 1 -- 172.21.15.62:0/4045044277 learned_addr learned my addr 172.21.15.62:0/4045044277 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151062357 0 0) 0x7fd8c80cff60 con 0x7fd8c815c4e0 2024-04-25T09:07:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b0003520 con 0x7fd8c815c4e0 2024-04-25T09:07:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8cd400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c80cfaa0 0x7fd8c80b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8c7e00640 1 --2- 172.21.15.62:0/4045044277 >> v2:172.21.15.121:3300/0 conn(0x7fd8c80cf5f0 0x7fd8c80b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1161025786 0 0) 0x7fd8b0003520 con 0x7fd8c815c4e0 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c80cfaa0 msgr2=0x7fd8c80b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8c7400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c80cfaa0 0x7fd8c80b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 >> v2:172.21.15.121:3300/0 conn(0x7fd8c80cf5f0 msgr2=0x7fd8c80b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.218+0000 7fd8c7400640 1 --2- 172.21.15.62:0/4045044277 >> v2:172.21.15.121:3300/0 conn(0x7fd8c80cf5f0 0x7fd8c80b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8070640 con 0x7fd8c815c4e0 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fd8bc001670 con 0x7fd8c815c4e0 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 >> v1:172.21.15.62:6789/0 conn(0x7fd8c815c4e0 legacy=0x7fd8c815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8b00039b0 0x7fd8b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c80cfaa0 0x7fd8b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80b3720 con 0x7fd8b0004d40 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8070360 con 0x7fd8b00039b0 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b000baf0 con 0x7fd8c80cfaa0 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8cd400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8cca00640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8b00039b0 0x7fd8b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.219+0000 7fd8c7e00640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c80cfaa0 0x7fd8b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.220+0000 7fd8cd400640 1 -- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c80cfaa0 msgr2=0x7fd8b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:06.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.220+0000 7fd8cd400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c80cfaa0 0x7fd8b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.220+0000 7fd8cd400640 1 -- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8b00039b0 msgr2=0x7fd8b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:06.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.220+0000 7fd8cd400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8b00039b0 0x7fd8b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.220+0000 7fd8cd400640 1 -- 172.21.15.62:0/4045044277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0003520 con 0x7fd8b0004d40 2024-04-25T09:07:06.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.221+0000 7fd8cd400640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8b00051c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007330 tx=0x7fd8b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aff1cbaf973b50d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:06.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.221+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0033070 con 0x7fd8b0004d40 2024-04-25T09:07:06.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.221+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c002f650 con 0x7fd8b0004d40 2024-04-25T09:07:06.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.221+0000 7fd8c7400640 1 -- 172.21.15.62:0/4045044277 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0009be0 con 0x7fd8b0004d40 2024-04-25T09:07:06.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.222+0000 7fd8cde00640 1 -- 172.21.15.62:0/4045044277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 msgr2=0x7fd8b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.222+0000 7fd8cde00640 1 --2- 172.21.15.62:0/4045044277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8b00051c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007330 tx=0x7fd8b0005830 comp rx=0 tx=0).stop 2024-04-25T09:07:06.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.222+0000 7fd8cd400640 1 -- 172.21.15.62:0/4045044277 reap_dead start 2024-04-25T09:07:06.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.222+0000 7fd8cde00640 1 -- 172.21.15.62:0/4045044277 shutdown_connections 2024-04-25T09:07:06.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.222+0000 7fd8cde00640 1 -- 172.21.15.62:0/4045044277 >> 172.21.15.62:0/4045044277 conn(0x7fd8c80668f0 msgr2=0x7fd8c8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:06.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.222+0000 7fd8cde00640 1 -- 172.21.15.62:0/4045044277 shutdown_connections 2024-04-25T09:07:06.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.222+0000 7fd8cde00640 1 -- 172.21.15.62:0/4045044277 wait complete. 2024-04-25T09:07:06.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.224+0000 7fd8cde00640 1 Processor -- start 2024-04-25T09:07:06.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.224+0000 7fd8cde00640 1 -- start start 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8c8145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cde00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c8148440 0x7fd8c8145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c81470a0 0x7fd8c80ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cde00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8071120 con 0x7fd8b0004d40 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cde00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8070c00 con 0x7fd8c81470a0 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cde00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80b4810 con 0x7fd8c8148440 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8c8145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8c8145900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55278/0 (socket says 172.21.15.62:55278) 2024-04-25T09:07:06.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.225+0000 7fd8cca00640 1 -- 172.21.15.62:0/415192235 learned_addr learned my addr 172.21.15.62:0/415192235 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.226+0000 7fd8c7e00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c8148440 0x7fd8c8145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.226+0000 7fd8cca00640 1 -- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c8148440 msgr2=0x7fd8c8145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.226+0000 7fd8cd400640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c81470a0 0x7fd8c80ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.226+0000 7fd8cca00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c8148440 0x7fd8c8145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.226+0000 7fd8cca00640 1 -- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c81470a0 msgr2=0x7fd8c80ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.226+0000 7fd8cca00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c81470a0 0x7fd8c80ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.226+0000 7fd8cca00640 1 -- 172.21.15.62:0/415192235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c81488c0 con 0x7fd8b0004d40 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.226+0000 7fd8cca00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8c8145900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00e7b0 tx=0x7fd8bc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.227+0000 7fd8c4c00640 1 -- 172.21.15.62:0/415192235 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00c870 con 0x7fd8b0004d40 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.227+0000 7fd8c4c00640 1 -- 172.21.15.62:0/415192235 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc018070 con 0x7fd8b0004d40 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.227+0000 7fd8c4c00640 1 -- 172.21.15.62:0/415192235 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00cc30 con 0x7fd8b0004d40 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.227+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c8146000 con 0x7fd8b0004d40 2024-04-25T09:07:06.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.227+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c81465c0 con 0x7fd8b0004d40 2024-04-25T09:07:06.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.229+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c80b4bf0 con 0x7fd8b0004d40 2024-04-25T09:07:06.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.229+0000 7fd8c4c00640 1 -- 172.21.15.62:0/415192235 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8bc01d460 con 0x7fd8b0004d40 2024-04-25T09:07:06.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.230+0000 7fd8c4c00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a806b520 0x7fd8a806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.231+0000 7fd8c4c00640 1 -- 172.21.15.62:0/415192235 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd8bc08a190 con 0x7fd8b0004d40 2024-04-25T09:07:06.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.231+0000 7fd8c7e00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a806b520 0x7fd8a806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.233+0000 7fd8c7e00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a806b520 0x7fd8a806d9e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80b3ec0 tx=0x7fd8b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:06.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.235+0000 7fd8c4c00640 1 -- 172.21.15.62:0/415192235 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8c80b4bf0 con 0x7fd8b0004d40 2024-04-25T09:07:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.454+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd8c804ef20 con 0x7fd8b0004d40 2024-04-25T09:07:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.454+0000 7fd8c4c00640 1 -- 172.21.15.62:0/415192235 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd8bc058f60 con 0x7fd8b0004d40 2024-04-25T09:07:06.443 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:07:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.457+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a806b520 msgr2=0x7fd8a806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.457+0000 7fd8cde00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a806b520 0x7fd8a806d9e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80b3ec0 tx=0x7fd8b4008040 comp rx=0 tx=0).stop 2024-04-25T09:07:06.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.457+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 msgr2=0x7fd8c8145900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8c8145900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00e7b0 tx=0x7fd8bc00ec70 comp rx=0 tx=0).stop 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 shutdown_connections 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a806b520 0x7fd8a806d9e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c81470a0 0x7fd8c80ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c8148440 0x7fd8c8145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 --2- 172.21.15.62:0/415192235 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8b0004d40 0x7fd8c8145900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 >> 172.21.15.62:0/415192235 conn(0x7fd8c80668f0 msgr2=0x7fd8c8156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 shutdown_connections 2024-04-25T09:07:06.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.458+0000 7fd8cde00640 1 -- 172.21.15.62:0/415192235 wait complete. 2024-04-25T09:07:06.464 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:07:06.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:06.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:07:06.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:07:06.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:06.571 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:07:06.621 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:07:06.622 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:06.663 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:07:06.663 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:07:06.663 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:06.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:07:06.704 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:07:06.705 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:06.757 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:07:06.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:06.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.978+0000 7fdf07000640 1 Processor -- start 2024-04-25T09:07:06.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.979+0000 7fdf07000640 1 -- start start 2024-04-25T09:07:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.980+0000 7fdf07000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf000b6a90 0x7fdf000d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.980+0000 7fdf07000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf000d3110 con 0x7fdf000b27a0 2024-04-25T09:07:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.980+0000 7fdf07000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf000d33f0 con 0x7fdf000b56f0 2024-04-25T09:07:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.980+0000 7fdf07000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000cedc0 con 0x7fdf000b6a90 2024-04-25T09:07:06.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.981+0000 7fdf06600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdf000b27a0 0x7fdf000d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50916/0 (socket says 172.21.15.62:50916) 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.981+0000 7fdf06600640 1 -- 172.21.15.62:0/1858734344 learned_addr learned my addr 172.21.15.62:0/1858734344 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.981+0000 7fdf05200640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf000b6a90 0x7fdf000d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.981+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380564095 0 0) 0x7fdf000d3110 con 0x7fdf000b27a0 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.981+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee8003520 con 0x7fdf000b27a0 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3915852776 0 0) 0x7fdee8003520 con 0x7fdf000b27a0 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf000b6a90 msgr2=0x7fdf000d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf000b6a90 0x7fdf000d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 >> v1:172.21.15.121:6789/0 conn(0x7fdf000b56f0 legacy=0x7fdf00160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf000d36d0 con 0x7fdf000b27a0 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fdef80019f0 con 0x7fdf000b27a0 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 >> v1:172.21.15.62:6789/0 conn(0x7fdf000b27a0 legacy=0x7fdf000d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf000b6a90 0x7fdee80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdee8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8004d40 0x7fdee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000cedc0 con 0x7fdee8003fb0 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee800bb00 con 0x7fdee8004d40 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee800bd00 con 0x7fdf000b6a90 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.982+0000 7fdf05200640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdee8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.983+0000 7fdf05c00640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf000b6a90 0x7fdee80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.983+0000 7fdf06600640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8004d40 0x7fdee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.983+0000 7fdf05200640 1 -- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf000b6a90 msgr2=0x7fdee80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.983+0000 7fdf05200640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf000b6a90 0x7fdee80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.983+0000 7fdf05200640 1 -- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8004d40 msgr2=0x7fdee80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.983+0000 7fdf05200640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8004d40 0x7fdee80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.983+0000 7fdf05200640 1 -- 172.21.15.62:0/1858734344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee8003520 con 0x7fdee8003fb0 2024-04-25T09:07:06.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.984+0000 7fdf05200640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdee8004800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009ac0 tx=0x7fdee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b13b9820bc57a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.984+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0036070 con 0x7fdee8003fb0 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.984+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0031400 con 0x7fdee8003fb0 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.984+0000 7fdeffe00640 1 -- 172.21.15.62:0/1858734344 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0009690 con 0x7fdee8003fb0 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.985+0000 7fdf07000640 1 -- 172.21.15.62:0/1858734344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 msgr2=0x7fdee8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.985+0000 7fdf07000640 1 --2- 172.21.15.62:0/1858734344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdee8004800 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009ac0 tx=0x7fdee8005830 comp rx=0 tx=0).stop 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.985+0000 7fdf06600640 1 -- 172.21.15.62:0/1858734344 reap_dead start 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.985+0000 7fdf07000640 1 -- 172.21.15.62:0/1858734344 shutdown_connections 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.985+0000 7fdf07000640 1 -- 172.21.15.62:0/1858734344 >> 172.21.15.62:0/1858734344 conn(0x7fdf00066d10 msgr2=0x7fdf00067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.985+0000 7fdf07000640 1 -- 172.21.15.62:0/1858734344 shutdown_connections 2024-04-25T09:07:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.985+0000 7fdf07000640 1 -- 172.21.15.62:0/1858734344 wait complete. 2024-04-25T09:07:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.987+0000 7fdf07000640 1 Processor -- start 2024-04-25T09:07:06.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.987+0000 7fdf07000640 1 -- start start 2024-04-25T09:07:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf07000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdf0014ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf07000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf0015bcf0 0x7fdf0015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf07000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf001f6be0 0x7fdf001f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf07000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000b2ee0 con 0x7fdee8003fb0 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf07000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000b2d60 con 0x7fdf001f6be0 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf07000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000d2c30 con 0x7fdf0015bcf0 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf05c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdf0014ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf05c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdf0014ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55306/0 (socket says 172.21.15.62:55306) 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf05c00640 1 -- 172.21.15.62:0/165913072 learned_addr learned my addr 172.21.15.62:0/165913072 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf05c00640 1 -- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf0015bcf0 msgr2=0x7fdf0015c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf05c00640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf0015bcf0 0x7fdf0015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf05c00640 1 -- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf001f6be0 msgr2=0x7fdf001f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf05c00640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf001f6be0 0x7fdf001f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:06.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.988+0000 7fdf05c00640 1 -- 172.21.15.62:0/165913072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0015c860 con 0x7fdee8003fb0 2024-04-25T09:07:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.989+0000 7fdf05c00640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdf0014ce00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e580 tx=0x7fdef400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.989+0000 7fdefd600640 1 -- 172.21.15.62:0/165913072 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40052e0 con 0x7fdee8003fb0 2024-04-25T09:07:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.989+0000 7fdefd600640 1 -- 172.21.15.62:0/165913072 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4018070 con 0x7fdee8003fb0 2024-04-25T09:07:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.989+0000 7fdefd600640 1 -- 172.21.15.62:0/165913072 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4003810 con 0x7fdee8003fb0 2024-04-25T09:07:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.989+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0015cb40 con 0x7fdee8003fb0 2024-04-25T09:07:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.989+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0014ad50 con 0x7fdee8003fb0 2024-04-25T09:07:06.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.991+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0004ef20 con 0x7fdee8003fb0 2024-04-25T09:07:06.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.991+0000 7fdefd600640 1 -- 172.21.15.62:0/165913072 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdef40039b0 con 0x7fdee8003fb0 2024-04-25T09:07:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.993+0000 7fdefd600640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdee406b4d0 0x7fdee406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.993+0000 7fdefd600640 1 -- 172.21.15.62:0/165913072 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdef4089fe0 con 0x7fdee8003fb0 2024-04-25T09:07:06.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.993+0000 7fdf05200640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdee406b4d0 0x7fdee406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:06.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.995+0000 7fdf05200640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdee406b4d0 0x7fdee406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdef0033000 tx=0x7fdef002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:06.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:06.997+0000 7fdefd600640 1 -- 172.21.15.62:0/165913072 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdef4058d40 con 0x7fdee8003fb0 2024-04-25T09:07:07.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.234+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf000ce580 con 0x7fdee8003fb0 2024-04-25T09:07:07.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.235+0000 7fdefd600640 1 -- 172.21.15.62:0/165913072 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fdef4058640 con 0x7fdee8003fb0 2024-04-25T09:07:07.224 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:07.224 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2179586213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2179586213},{"type":"v1","addr":"172.21.15.121:6836","nonce":2179586213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1425308084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1425308084},{"type":"v1","addr":"172.21.15.121:6837","nonce":1425308084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/231681595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":231681595},{"type":"v1","addr":"172.21.15.62:6834","nonce":231681595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:04:25.639815+0000","modified":"2024-04-25T09:04:39.952254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/274650142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":274650142},{"type":"v1","addr":"172.21.15.62:6835","nonce":274650142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:07:07.224 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdee406b4d0 msgr2=0x7fdee406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdee406b4d0 0x7fdee406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdef0033000 tx=0x7fdef002e000 comp rx=0 tx=0).stop 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 msgr2=0x7fdf0014ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdf0014ce00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e580 tx=0x7fdef400ea40 comp rx=0 tx=0).stop 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 shutdown_connections 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdee406b4d0 0x7fdee406d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf001f6be0 0x7fdf001f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf0015bcf0 0x7fdf0015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 --2- 172.21.15.62:0/165913072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee8003fb0 0x7fdf0014ce00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 >> 172.21.15.62:0/165913072 conn(0x7fdf00066d10 msgr2=0x7fdf000d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 shutdown_connections 2024-04-25T09:07:07.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.238+0000 7fdf07000640 1 -- 172.21.15.62:0/165913072 wait complete. 2024-04-25T09:07:07.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:07.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.471+0000 7ffbad200640 1 Processor -- start 2024-04-25T09:07:07.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.471+0000 7ffbad200640 1 -- start start 2024-04-25T09:07:07.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.472+0000 7ffbad200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba80b56f0 0x7ffba8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.472+0000 7ffbad200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffba80b6a90 0x7ffba8161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.472+0000 7ffbad200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba80d3110 con 0x7ffba80b27a0 2024-04-25T09:07:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.472+0000 7ffbad200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba8161780 con 0x7ffba80b6a90 2024-04-25T09:07:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.472+0000 7ffbad200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80cedc0 con 0x7ffba80b56f0 2024-04-25T09:07:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ffba80b27a0 0x7ffba8161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50926/0 (socket says 172.21.15.62:50926) 2024-04-25T09:07:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba7e00640 1 -- 172.21.15.62:0/2892318624 learned_addr learned my addr 172.21.15.62:0/2892318624 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba7400640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba80b56f0 0x7ffba8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6a00640 1 --2- 172.21.15.62:0/2892318624 >> v2:172.21.15.121:3300/0 conn(0x7ffba80b6a90 0x7ffba8161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914049517 0 0) 0x7ffba80d3110 con 0x7ffba80b27a0 2024-04-25T09:07:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb8c003520 con 0x7ffba80b27a0 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3585767674 0 0) 0x7ffb8c003520 con 0x7ffba80b27a0 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba80b56f0 msgr2=0x7ffba8160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6000640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba80b56f0 0x7ffba8160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 >> v2:172.21.15.121:3300/0 conn(0x7ffba80b6a90 msgr2=0x7ffba8161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6000640 1 --2- 172.21.15.62:0/2892318624 >> v2:172.21.15.121:3300/0 conn(0x7ffba80b6a90 0x7ffba8161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.473+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80d33f0 con 0x7ffba80b27a0 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ffba00019f0 con 0x7ffba80b27a0 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 >> v1:172.21.15.62:6789/0 conn(0x7ffba80b27a0 legacy=0x7ffba8161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6000640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c0039b0 0x7ffb8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6000640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb8c004800 0x7ffb8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6000640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba80b56f0 0x7ffb8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80cedc0 con 0x7ffb8c004800 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c00b9c0 con 0x7ffb8c0039b0 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c00bb40 con 0x7ffba80b56f0 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6a00640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb8c004800 0x7ffb8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba7400640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c0039b0 0x7ffb8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba7e00640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba80b56f0 0x7ffb8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6a00640 1 -- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba80b56f0 msgr2=0x7ffb8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.474+0000 7ffba6a00640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba80b56f0 0x7ffb8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.475+0000 7ffba6a00640 1 -- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c0039b0 msgr2=0x7ffb8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.475+0000 7ffba6a00640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c0039b0 0x7ffb8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.475+0000 7ffba6a00640 1 -- 172.21.15.62:0/2892318624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c003520 con 0x7ffb8c004800 2024-04-25T09:07:07.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.476+0000 7ffba6a00640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb8c004800 0x7ffb8c004c80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009ac0 tx=0x7ffb8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37f6373f2bad706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:07.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.476+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94036070 con 0x7ffb8c004800 2024-04-25T09:07:07.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.476+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94031400 con 0x7ffb8c004800 2024-04-25T09:07:07.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.476+0000 7ffba6000640 1 -- 172.21.15.62:0/2892318624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94009690 con 0x7ffb8c004800 2024-04-25T09:07:07.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.476+0000 7ffbad200640 1 -- 172.21.15.62:0/2892318624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb8c004800 msgr2=0x7ffb8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.476+0000 7ffbad200640 1 --2- 172.21.15.62:0/2892318624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb8c004800 0x7ffb8c004c80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009ac0 tx=0x7ffb8c005830 comp rx=0 tx=0).stop 2024-04-25T09:07:07.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.476+0000 7ffba7e00640 1 -- 172.21.15.62:0/2892318624 reap_dead start 2024-04-25T09:07:07.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.477+0000 7ffbad200640 1 -- 172.21.15.62:0/2892318624 shutdown_connections 2024-04-25T09:07:07.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.477+0000 7ffbad200640 1 -- 172.21.15.62:0/2892318624 >> 172.21.15.62:0/2892318624 conn(0x7ffba8066d10 msgr2=0x7ffba8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:07.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.477+0000 7ffbad200640 1 -- 172.21.15.62:0/2892318624 shutdown_connections 2024-04-25T09:07:07.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.477+0000 7ffbad200640 1 -- 172.21.15.62:0/2892318624 wait complete. 2024-04-25T09:07:07.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.478+0000 7ffbad200640 1 Processor -- start 2024-04-25T09:07:07.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.479+0000 7ffbad200640 1 -- start start 2024-04-25T09:07:07.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.479+0000 7ffbad200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c004800 0x7ffba80ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffbad200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba8158770 0x7ffba815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffbad200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba814ea40 0x7ffba814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffbad200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80b2d60 con 0x7ffba814ea40 2024-04-25T09:07:07.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffbad200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80cea50 con 0x7ffb8c004800 2024-04-25T09:07:07.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffbad200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80b2ee0 con 0x7ffba8158770 2024-04-25T09:07:07.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba814ea40 0x7ffba814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba814ea40 0x7ffba814eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55330/0 (socket says 172.21.15.62:55330) 2024-04-25T09:07:07.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7e00640 1 -- 172.21.15.62:0/3062193337 learned_addr learned my addr 172.21.15.62:0/3062193337 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:07.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba6a00640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba8158770 0x7ffba815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7400640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c004800 0x7ffba80ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7e00640 1 -- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba8158770 msgr2=0x7ffba815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7e00640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba8158770 0x7ffba815a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7e00640 1 -- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c004800 msgr2=0x7ffba80ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7e00640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c004800 0x7ffba80ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba7e00640 1 -- 172.21.15.62:0/3062193337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba8158f30 con 0x7ffba814ea40 2024-04-25T09:07:07.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.480+0000 7ffba6a00640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba8158770 0x7ffba815a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:07.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.481+0000 7ffba7e00640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba814ea40 0x7ffba814eec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ffba808c570 tx=0x7ffba0007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:07.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.481+0000 7ffb9f400640 1 -- 172.21.15.62:0/3062193337 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000ebc0 con 0x7ffba814ea40 2024-04-25T09:07:07.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.481+0000 7ffb9f400640 1 -- 172.21.15.62:0/3062193337 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba0018070 con 0x7ffba814ea40 2024-04-25T09:07:07.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.481+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba8159210 con 0x7ffba814ea40 2024-04-25T09:07:07.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.481+0000 7ffb9f400640 1 -- 172.21.15.62:0/3062193337 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0013420 con 0x7ffba814ea40 2024-04-25T09:07:07.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.481+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba80c8650 con 0x7ffba814ea40 2024-04-25T09:07:07.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.482+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba80c8440 con 0x7ffba814ea40 2024-04-25T09:07:07.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.483+0000 7ffb9f400640 1 -- 172.21.15.62:0/3062193337 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffba00135c0 con 0x7ffba814ea40 2024-04-25T09:07:07.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.484+0000 7ffb9f400640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb8806b500 0x7ffb8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.484+0000 7ffb9f400640 1 -- 172.21.15.62:0/3062193337 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ffba008a5f0 con 0x7ffba814ea40 2024-04-25T09:07:07.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.485+0000 7ffba7400640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb8806b500 0x7ffb8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.486+0000 7ffba7400640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb8806b500 0x7ffb8806d9c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009870 tx=0x7ffb98008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:07.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.489+0000 7ffb9f400640 1 -- 172.21.15.62:0/3062193337 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffba0059340 con 0x7ffba814ea40 2024-04-25T09:07:07.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.696+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffba804ef20 con 0x7ffba814ea40 2024-04-25T09:07:07.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.698+0000 7ffb9f400640 1 -- 172.21.15.62:0/3062193337 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16912 (secure 0 0 0) 0x7ffba0058c40 con 0x7ffba814ea40 2024-04-25T09:07:07.687 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:07.687 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":37,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:06:47.512613+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:23.626010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:24.669175+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.1d","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb8806b500 msgr2=0x7ffb8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb8806b500 0x7ffb8806d9c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009870 tx=0x7ffb98008040 comp rx=0 tx=0).stop 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba814ea40 msgr2=0x7ffba814eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba814ea40 0x7ffba814eec0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ffba808c570 tx=0x7ffba0007bc0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 shutdown_connections 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb8806b500 0x7ffb8806d9c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffba814ea40 0x7ffba814eec0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffba8158770 0x7ffba815a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 --2- 172.21.15.62:0/3062193337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb8c004800 0x7ffba80ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 >> 172.21.15.62:0/3062193337 conn(0x7ffba8066d10 msgr2=0x7ffba80d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 shutdown_connections 2024-04-25T09:07:07.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.701+0000 7ffbad200640 1 -- 172.21.15.62:0/3062193337 wait complete. 2024-04-25T09:07:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:04:23.626010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:04:24.669175+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 36, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:07:07.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:07:07.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.927+0000 7fa0d3200640 1 Processor -- start 2024-04-25T09:07:07.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.928+0000 7fa0d3200640 1 -- start start 2024-04-25T09:07:07.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0d3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 0x7fa0d40d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0d3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa0d40d2fb0 0x7fa0d40ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0d3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d40ce780 con 0x7fa0d40d3960 2024-04-25T09:07:07.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0d3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d40d3440 con 0x7fa0d40d2fb0 2024-04-25T09:07:07.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0d3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40ce1c0 con 0x7fa0d415dd50 2024-04-25T09:07:07.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0d2800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa0d40d3960 0x7fa0d40d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50932/0 (socket says 172.21.15.62:50932) 2024-04-25T09:07:07.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0d2800640 1 -- 172.21.15.62:0/3408659788 learned_addr learned my addr 172.21.15.62:0/3408659788 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0d1e00640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 0x7fa0d40d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.929+0000 7fa0c9400640 1 --2- 172.21.15.62:0/3408659788 >> v2:172.21.15.121:3300/0 conn(0x7fa0d40d2fb0 0x7fa0d40ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579638668 0 0) 0x7fa0d40ce780 con 0x7fa0d40d3960 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b8003520 con 0x7fa0d40d3960 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1738491564 0 0) 0x7fa0b8003520 con 0x7fa0d40d3960 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 msgr2=0x7fa0d40d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 0x7fa0d40d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 >> v2:172.21.15.121:3300/0 conn(0x7fa0d40d2fb0 msgr2=0x7fa0d40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 --2- 172.21.15.62:0/3408659788 >> v2:172.21.15.121:3300/0 conn(0x7fa0d40d2fb0 0x7fa0d40ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40ce4a0 con 0x7fa0d40d3960 2024-04-25T09:07:07.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fa0cc0019f0 con 0x7fa0d40d3960 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 >> v1:172.21.15.62:6789/0 conn(0x7fa0d40d3960 legacy=0x7fa0d40d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 0x7fa0b80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0b8003f30 0x7fa0b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40d3440 con 0x7fa0b8004d40 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40ce1c0 con 0x7fa0b8003f30 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.930+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b800b9c0 con 0x7fa0d415dd50 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0d2800640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0c9400640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0b8003f30 0x7fa0b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0d1e00640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 0x7fa0b80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0d2800640 1 -- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 msgr2=0x7fa0b80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0d2800640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 0x7fa0b80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0d2800640 1 -- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0b8003f30 msgr2=0x7fa0b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0d2800640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0b8003f30 0x7fa0b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0d2800640 1 -- 172.21.15.62:0/3408659788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8003520 con 0x7fa0b8004d40 2024-04-25T09:07:07.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.931+0000 7fa0d1e00640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415dd50 0x7fa0b80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.932+0000 7fa0d2800640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0b80051c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00a920 tx=0x7fa0b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374dfa38e3963b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.932+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00a4c0 con 0x7fa0b8004d40 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.932+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc02f070 con 0x7fa0b8004d40 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.932+0000 7fa0d1400640 1 -- 172.21.15.62:0/3408659788 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc02bd40 con 0x7fa0b8004d40 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.933+0000 7fa0d3200640 1 -- 172.21.15.62:0/3408659788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 msgr2=0x7fa0b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.933+0000 7fa0d3200640 1 --2- 172.21.15.62:0/3408659788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0b80051c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00a920 tx=0x7fa0b8005830 comp rx=0 tx=0).stop 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.933+0000 7fa0d2800640 1 -- 172.21.15.62:0/3408659788 reap_dead start 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.933+0000 7fa0d3200640 1 -- 172.21.15.62:0/3408659788 shutdown_connections 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.933+0000 7fa0d3200640 1 -- 172.21.15.62:0/3408659788 >> 172.21.15.62:0/3408659788 conn(0x7fa0d406b000 msgr2=0x7fa0d406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.933+0000 7fa0d3200640 1 -- 172.21.15.62:0/3408659788 shutdown_connections 2024-04-25T09:07:07.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.933+0000 7fa0d3200640 1 -- 172.21.15.62:0/3408659788 wait complete. 2024-04-25T09:07:07.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.935+0000 7fa0d3200640 1 Processor -- start 2024-04-25T09:07:07.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.935+0000 7fa0d3200640 1 -- start start 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0d41e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0d40c9390 0x7fa0d414cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415f1f0 0x7fa0d415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40d2480 con 0x7fa0b8004d40 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40d2300 con 0x7fa0d40c9390 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40d2600 con 0x7fa0d415f1f0 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0d41e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0d41e7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55340/0 (socket says 172.21.15.62:55340) 2024-04-25T09:07:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.936+0000 7fa0d1e00640 1 -- 172.21.15.62:0/2835002865 learned_addr learned my addr 172.21.15.62:0/2835002865 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:07.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0c9400640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0d40c9390 0x7fa0d414cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0d2800640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415f1f0 0x7fa0d415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0d1e00640 1 -- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415f1f0 msgr2=0x7fa0d415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0d1e00640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415f1f0 0x7fa0d415f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0d1e00640 1 -- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0d40c9390 msgr2=0x7fa0d414cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0d1e00640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0d40c9390 0x7fa0d414cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0d1e00640 1 -- 172.21.15.62:0/2835002865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d4159ef0 con 0x7fa0b8004d40 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0c9400640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0d40c9390 0x7fa0d414cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.937+0000 7fa0d1e00640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0d41e7c60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40108e0 tx=0x7fa0c4010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.938+0000 7fa0caa00640 1 -- 172.21.15.62:0/2835002865 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400f040 con 0x7fa0b8004d40 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.938+0000 7fa0caa00640 1 -- 172.21.15.62:0/2835002865 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c400ec20 con 0x7fa0b8004d40 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.938+0000 7fa0caa00640 1 -- 172.21.15.62:0/2835002865 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c401c400 con 0x7fa0b8004d40 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.938+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d40c9870 con 0x7fa0b8004d40 2024-04-25T09:07:07.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.938+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d40c9e30 con 0x7fa0b8004d40 2024-04-25T09:07:07.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.939+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d404ef20 con 0x7fa0b8004d40 2024-04-25T09:07:07.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.940+0000 7fa0caa00640 1 -- 172.21.15.62:0/2835002865 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0c401c5a0 con 0x7fa0b8004d40 2024-04-25T09:07:07.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.941+0000 7fa0caa00640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa0b006b500 0x7fa0b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:07.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.942+0000 7fa0caa00640 1 -- 172.21.15.62:0/2835002865 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa0c408aa80 con 0x7fa0b8004d40 2024-04-25T09:07:07.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.942+0000 7fa0c9400640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa0b006b500 0x7fa0b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:07.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.943+0000 7fa0c9400640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa0b006b500 0x7fa0b006d9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40b68f0 tx=0x7fa0bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:07.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:07.947+0000 7fa0caa00640 1 -- 172.21.15.62:0/2835002865 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0c408ce20 con 0x7fa0b8004d40 2024-04-25T09:07:08.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.146+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0d40b9950 con 0x7fa0b8004d40 2024-04-25T09:07:08.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.147+0000 7fa0caa00640 1 -- 172.21.15.62:0/2835002865 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0c4018030 con 0x7fa0b8004d40 2024-04-25T09:07:08.135 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:07:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.149+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa0b006b500 msgr2=0x7fa0b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.149+0000 7fa0d3200640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa0b006b500 0x7fa0b006d9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40b68f0 tx=0x7fa0bc008040 comp rx=0 tx=0).stop 2024-04-25T09:07:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.149+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 msgr2=0x7fa0d41e7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.149+0000 7fa0d3200640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0d41e7c60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40108e0 tx=0x7fa0c4010da0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.150+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 shutdown_connections 2024-04-25T09:07:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.150+0000 7fa0d3200640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa0b006b500 0x7fa0b006d9c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.150+0000 7fa0d3200640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa0d415f1f0 0x7fa0d415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.150+0000 7fa0d3200640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0d40c9390 0x7fa0d414cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.150+0000 7fa0d3200640 1 --2- 172.21.15.62:0/2835002865 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa0b8004d40 0x7fa0d41e7c60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.150+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 >> 172.21.15.62:0/2835002865 conn(0x7fa0d406b000 msgr2=0x7fa0d40cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.150+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 shutdown_connections 2024-04-25T09:07:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.150+0000 7fa0d3200640 1 -- 172.21.15.62:0/2835002865 wait complete. 2024-04-25T09:07:08.158 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:07:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.373+0000 7f74c6600640 1 Processor -- start 2024-04-25T09:07:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.373+0000 7f74c6600640 1 -- start start 2024-04-25T09:07:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.375+0000 7f74c6600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f74c006d2c0 0x7f74c00ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.375+0000 7f74c6600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 0x7f74c00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.375+0000 7f74c6600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c00cf000 con 0x7f74c006d6a0 2024-04-25T09:07:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.375+0000 7f74c6600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c00cf2e0 con 0x7f74c006d2c0 2024-04-25T09:07:08.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.375+0000 7f74c6600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00b26c0 con 0x7f74c00b31a0 2024-04-25T09:07:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.376+0000 7f74c5200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f74c006d2c0 0x7f74c00ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.376+0000 7f74c5c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 0x7f74c00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.376+0000 7f74c5200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f74c006d2c0 0x7f74c00ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:32940/0 (socket says 172.21.15.62:32940) 2024-04-25T09:07:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.376+0000 7f74c5200640 1 -- 172.21.15.62:0/2406202535 learned_addr learned my addr 172.21.15.62:0/2406202535 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.377+0000 7f74c5200640 1 -- 172.21.15.62:0/2406202535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 msgr2=0x7f74c00b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.377+0000 7f74c5200640 1 --2- 172.21.15.62:0/2406202535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 0x7f74c00b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.377+0000 7f74c5200640 1 -- 172.21.15.62:0/2406202535 >> v1:172.21.15.121:6789/0 conn(0x7f74c006d6a0 legacy=0x7f74c00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.377+0000 7f74c5200640 1 -- 172.21.15.62:0/2406202535 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00b29a0 con 0x7f74c006d2c0 2024-04-25T09:07:08.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.378+0000 7f74c5200640 1 --2- 172.21.15.62:0/2406202535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d2c0 0x7f74c00ccde0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009870 tx=0x7f74b402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8016b9c527a3d0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:08.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.378+0000 7f74bf400640 1 -- 172.21.15.62:0/2406202535 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4037070 con 0x7f74c006d2c0 2024-04-25T09:07:08.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.378+0000 7f74bf400640 1 -- 172.21.15.62:0/2406202535 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b4032440 con 0x7f74c006d2c0 2024-04-25T09:07:08.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.378+0000 7f74bf400640 1 -- 172.21.15.62:0/2406202535 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4032810 con 0x7f74c006d2c0 2024-04-25T09:07:08.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.379+0000 7f74c6600640 1 -- 172.21.15.62:0/2406202535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d2c0 msgr2=0x7f74c00ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.379+0000 7f74c6600640 1 --2- 172.21.15.62:0/2406202535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d2c0 0x7f74c00ccde0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009870 tx=0x7f74b402a250 comp rx=0 tx=0).stop 2024-04-25T09:07:08.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.379+0000 7f74c6600640 1 -- 172.21.15.62:0/2406202535 shutdown_connections 2024-04-25T09:07:08.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.379+0000 7f74c6600640 1 --2- 172.21.15.62:0/2406202535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 0x7f74c00b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.379+0000 7f74c6600640 1 --2- 172.21.15.62:0/2406202535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d2c0 0x7f74c00ccde0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.379+0000 7f74c6600640 1 -- 172.21.15.62:0/2406202535 >> 172.21.15.62:0/2406202535 conn(0x7f74c0066f20 msgr2=0x7f74c00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:08.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.379+0000 7f74c6600640 1 -- 172.21.15.62:0/2406202535 shutdown_connections 2024-04-25T09:07:08.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.379+0000 7f74c6600640 1 -- 172.21.15.62:0/2406202535 wait complete. 2024-04-25T09:07:08.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.381+0000 7f74c6600640 1 Processor -- start 2024-04-25T09:07:08.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.381+0000 7f74c6600640 1 -- start start 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c6600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006d2c0 0x7f74c013b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c6600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d6a0 0x7f74c013ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c6600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 0x7f74c015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c6600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c01448d0 con 0x7f74c006d2c0 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c6600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c006db20 con 0x7f74c006d6a0 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c6600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c0144a50 con 0x7f74c00b31a0 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006d2c0 0x7f74c013b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006d2c0 0x7f74c013b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55354/0 (socket says 172.21.15.62:55354) 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.382+0000 7f74c5200640 1 -- 172.21.15.62:0/888032107 learned_addr learned my addr 172.21.15.62:0/888032107 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.383+0000 7f74bfe00640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d6a0 0x7f74c013ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.383+0000 7f74c5c00640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 0x7f74c015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.383+0000 7f74c5200640 1 -- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 msgr2=0x7f74c015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.383+0000 7f74c5200640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 0x7f74c015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.383+0000 7f74c5200640 1 -- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d6a0 msgr2=0x7f74c013ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.383+0000 7f74c5200640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d6a0 0x7f74c013ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.383+0000 7f74c5200640 1 -- 172.21.15.62:0/888032107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c013b580 con 0x7f74c006d2c0 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.383+0000 7f74c5200640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006d2c0 0x7f74c013b040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f74b4000c00 tx=0x7f74b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.384+0000 7f74bcc00640 1 -- 172.21.15.62:0/888032107 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4037040 con 0x7f74c006d2c0 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.384+0000 7f74bcc00640 1 -- 172.21.15.62:0/888032107 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b4042070 con 0x7f74c006d2c0 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.384+0000 7f74bcc00640 1 -- 172.21.15.62:0/888032107 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4030880 con 0x7f74c006d2c0 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.384+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c013f6e0 con 0x7f74c006d2c0 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.384+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c013fca0 con 0x7f74c006d2c0 2024-04-25T09:07:08.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.385+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c006abe0 con 0x7f74c006d2c0 2024-04-25T09:07:08.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.386+0000 7f74bcc00640 1 -- 172.21.15.62:0/888032107 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74b4030a20 con 0x7f74c006d2c0 2024-04-25T09:07:08.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.387+0000 7f74bcc00640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f74a006b500 0x7f74a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.387+0000 7f74bcc00640 1 -- 172.21.15.62:0/888032107 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f74b40aaa70 con 0x7f74c006d2c0 2024-04-25T09:07:08.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.388+0000 7f74bfe00640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f74a006b500 0x7f74a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.389+0000 7f74bfe00640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f74a006b500 0x7f74a006d9c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74c0068130 tx=0x7f74ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:08.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.391+0000 7f74bcc00640 1 -- 172.21.15.62:0/888032107 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74c013b580 con 0x7f74c006d2c0 2024-04-25T09:07:08.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.591+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74c015e770 con 0x7f74c006d2c0 2024-04-25T09:07:08.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.592+0000 7f74bcc00640 1 -- 172.21.15.62:0/888032107 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74b4079850 con 0x7f74c006d2c0 2024-04-25T09:07:08.582 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f74a006b500 msgr2=0x7f74a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f74a006b500 0x7f74a006d9c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f74c0068130 tx=0x7f74ac009040 comp rx=0 tx=0).stop 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006d2c0 msgr2=0x7f74c013b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006d2c0 0x7f74c013b040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f74b4000c00 tx=0x7f74b40046c0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 shutdown_connections 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f74a006b500 0x7f74a006d9c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74c00b31a0 0x7f74c015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74c006d6a0 0x7f74c013ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 --2- 172.21.15.62:0/888032107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74c006d2c0 0x7f74c013b040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 >> 172.21.15.62:0/888032107 conn(0x7f74c0066f20 msgr2=0x7f74c00cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:08.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 shutdown_connections 2024-04-25T09:07:08.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.595+0000 7f74c6600640 1 -- 172.21.15.62:0/888032107 wait complete. 2024-04-25T09:07:08.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:07:08.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.833+0000 7f6f44a00640 1 Processor -- start 2024-04-25T09:07:08.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.833+0000 7f6f44a00640 1 -- start start 2024-04-25T09:07:08.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.835+0000 7f6f44a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6f4006d9d0 0x7f6f4015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.835+0000 7f6f44a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f4015e5f0 0x7f6f400b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.835+0000 7f6f44a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4015ee30 con 0x7f6f400ceb50 2024-04-25T09:07:08.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.835+0000 7f6f44a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f401618b0 con 0x7f6f4006d9d0 2024-04-25T09:07:08.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.835+0000 7f6f44a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f40161b90 con 0x7f6f4015e5f0 2024-04-25T09:07:08.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.835+0000 7f6f3de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6f4006d9d0 0x7f6f4015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.835+0000 7f6f3e800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6f400ceb50 0x7f6f4006d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50940/0 (socket says 172.21.15.62:50940) 2024-04-25T09:07:08.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.835+0000 7f6f3e800640 1 -- 172.21.15.62:0/3090355461 learned_addr learned my addr 172.21.15.62:0/3090355461 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:08.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.836+0000 7f6f3f200640 1 --2- 172.21.15.62:0/3090355461 >> v2:172.21.15.121:3300/0 conn(0x7f6f4015e5f0 0x7f6f400b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.836+0000 7f6f3d400640 1 -- 172.21.15.62:0/3090355461 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516733566 0 0) 0x7f6f4015ee30 con 0x7f6f400ceb50 2024-04-25T09:07:08.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.836+0000 7f6f3de00640 1 -- 172.21.15.62:0/3090355461 >> v2:172.21.15.121:3300/0 conn(0x7f6f4015e5f0 msgr2=0x7f6f400b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:08.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.836+0000 7f6f3de00640 1 --2- 172.21.15.62:0/3090355461 >> v2:172.21.15.121:3300/0 conn(0x7f6f4015e5f0 0x7f6f400b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.836+0000 7f6f3de00640 1 -- 172.21.15.62:0/3090355461 >> v1:172.21.15.62:6789/0 conn(0x7f6f400ceb50 legacy=0x7f6f4006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.836+0000 7f6f3de00640 1 -- 172.21.15.62:0/3090355461 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4015ea40 con 0x7f6f4006d9d0 2024-04-25T09:07:08.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.836+0000 7f6f3f200640 1 --2- 172.21.15.62:0/3090355461 >> v2:172.21.15.121:3300/0 conn(0x7f6f4015e5f0 0x7f6f400b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:08.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.837+0000 7f6f3de00640 1 --2- 172.21.15.62:0/3090355461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4006d9d0 0x7f6f4015e0b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6f400b3ad0 tx=0x7f6f2c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd36917a1e7817d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:08.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.837+0000 7f6f3d400640 1 -- 172.21.15.62:0/3090355461 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c037070 con 0x7f6f4006d9d0 2024-04-25T09:07:08.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.837+0000 7f6f3d400640 1 -- 172.21.15.62:0/3090355461 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c032440 con 0x7f6f4006d9d0 2024-04-25T09:07:08.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.837+0000 7f6f3d400640 1 -- 172.21.15.62:0/3090355461 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c02ae60 con 0x7f6f4006d9d0 2024-04-25T09:07:08.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.838+0000 7f6f44a00640 1 -- 172.21.15.62:0/3090355461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4006d9d0 msgr2=0x7f6f4015e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.838+0000 7f6f44a00640 1 --2- 172.21.15.62:0/3090355461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4006d9d0 0x7f6f4015e0b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6f400b3ad0 tx=0x7f6f2c02a110 comp rx=0 tx=0).stop 2024-04-25T09:07:08.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.838+0000 7f6f44a00640 1 -- 172.21.15.62:0/3090355461 shutdown_connections 2024-04-25T09:07:08.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.838+0000 7f6f44a00640 1 --2- 172.21.15.62:0/3090355461 >> v2:172.21.15.121:3300/0 conn(0x7f6f4015e5f0 0x7f6f400b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.838+0000 7f6f44a00640 1 --2- 172.21.15.62:0/3090355461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4006d9d0 0x7f6f4015e0b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.838+0000 7f6f44a00640 1 -- 172.21.15.62:0/3090355461 >> 172.21.15.62:0/3090355461 conn(0x7f6f40066f20 msgr2=0x7f6f400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:08.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.838+0000 7f6f44a00640 1 -- 172.21.15.62:0/3090355461 shutdown_connections 2024-04-25T09:07:08.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.838+0000 7f6f44a00640 1 -- 172.21.15.62:0/3090355461 wait complete. 2024-04-25T09:07:08.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.840+0000 7f6f44a00640 1 Processor -- start 2024-04-25T09:07:08.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.840+0000 7f6f44a00640 1 -- start start 2024-04-25T09:07:08.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f44a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4006d9d0 0x7f6f4013af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f44a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f400ceb50 0x7f6f40158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f44a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4015e5f0 0x7f6f4015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f44a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40162510 con 0x7f6f4015e5f0 2024-04-25T09:07:08.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f44a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40162390 con 0x7f6f400ceb50 2024-04-25T09:07:08.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f44a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400b4a90 con 0x7f6f4006d9d0 2024-04-25T09:07:08.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f3f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4015e5f0 0x7f6f4015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f3f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4015e5f0 0x7f6f4015b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55372/0 (socket says 172.21.15.62:55372) 2024-04-25T09:07:08.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f3f200640 1 -- 172.21.15.62:0/1682905966 learned_addr learned my addr 172.21.15.62:0/1682905966 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:08.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.841+0000 7f6f3de00640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f400ceb50 0x7f6f40158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f3e800640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4006d9d0 0x7f6f4013af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f3f200640 1 -- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4006d9d0 msgr2=0x7f6f4013af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f3f200640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4006d9d0 0x7f6f4013af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f3f200640 1 -- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f400ceb50 msgr2=0x7f6f40158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:08.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f3f200640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f400ceb50 0x7f6f40158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:08.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f3f200640 1 -- 172.21.15.62:0/1682905966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40156df0 con 0x7f6f4015e5f0 2024-04-25T09:07:08.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f3f200640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4015e5f0 0x7f6f4015b1f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800cd50 tx=0x7f6f3800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:08.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f36a00640 1 -- 172.21.15.62:0/1682905966 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800f680 con 0x7f6f4015e5f0 2024-04-25T09:07:08.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f36a00640 1 -- 172.21.15.62:0/1682905966 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3800f840 con 0x7f6f4015e5f0 2024-04-25T09:07:08.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f36a00640 1 -- 172.21.15.62:0/1682905966 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3800fc30 con 0x7f6f4015e5f0 2024-04-25T09:07:08.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.842+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f401570d0 con 0x7f6f4015e5f0 2024-04-25T09:07:08.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.843+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f40157690 con 0x7f6f4015e5f0 2024-04-25T09:07:08.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.844+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4006abe0 con 0x7f6f4015e5f0 2024-04-25T09:07:08.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.845+0000 7f6f36a00640 1 -- 172.21.15.62:0/1682905966 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f380213f0 con 0x7f6f4015e5f0 2024-04-25T09:07:08.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.846+0000 7f6f36a00640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f2006b4d0 0x7f6f2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:08.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.846+0000 7f6f36a00640 1 -- 172.21.15.62:0/1682905966 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6f3808b160 con 0x7f6f4015e5f0 2024-04-25T09:07:08.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.846+0000 7f6f3e800640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f2006b4d0 0x7f6f2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:08.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.848+0000 7f6f3e800640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f2006b4d0 0x7f6f2006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6f30000f60 tx=0x7f6f30007b70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:08.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:08.850+0000 7f6f36a00640 1 -- 172.21.15.62:0/1682905966 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f38014030 con 0x7f6f4015e5f0 2024-04-25T09:07:09.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.048+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f4004ef20 con 0x7f6f4015e5f0 2024-04-25T09:07:09.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.049+0000 7f6f36a00640 1 -- 172.21.15.62:0/1682905966 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f38059eb0 con 0x7f6f4015e5f0 2024-04-25T09:07:09.038 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f2006b4d0 msgr2=0x7f6f2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f2006b4d0 0x7f6f2006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6f30000f60 tx=0x7f6f30007b70 comp rx=0 tx=0).stop 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4015e5f0 msgr2=0x7f6f4015b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4015e5f0 0x7f6f4015b1f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800cd50 tx=0x7f6f3800a650 comp rx=0 tx=0).stop 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 shutdown_connections 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f2006b4d0 0x7f6f2006d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f4015e5f0 0x7f6f4015b1f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f400ceb50 0x7f6f40158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 --2- 172.21.15.62:0/1682905966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f4006d9d0 0x7f6f4013af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 >> 172.21.15.62:0/1682905966 conn(0x7f6f40066f20 msgr2=0x7f6f400b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 shutdown_connections 2024-04-25T09:07:09.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.052+0000 7f6f44a00640 1 -- 172.21.15.62:0/1682905966 wait complete. 2024-04-25T09:07:09.060 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:09.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.273+0000 7f91afe00640 1 Processor -- start 2024-04-25T09:07:09.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.274+0000 7f91afe00640 1 -- start start 2024-04-25T09:07:09.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.275+0000 7f91afe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b56f0 0x7f91b0160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.275+0000 7f91afe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91b00b6a90 0x7f91b0161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.275+0000 7f91afe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00d3110 con 0x7f91b00b27a0 2024-04-25T09:07:09.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.275+0000 7f91afe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b0161780 con 0x7f91b00b6a90 2024-04-25T09:07:09.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.275+0000 7f91afe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00cedc0 con 0x7f91b00b56f0 2024-04-25T09:07:09.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91af400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f91b00b27a0 0x7f91b0161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50944/0 (socket says 172.21.15.62:50944) 2024-04-25T09:07:09.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91af400640 1 -- 172.21.15.62:0/688877148 learned_addr learned my addr 172.21.15.62:0/688877148 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:09.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91ae000640 1 --2- 172.21.15.62:0/688877148 >> v2:172.21.15.62:3300/0 conn(0x7f91b00b6a90 0x7f91b0161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91aea00640 1 --2- 172.21.15.62:0/688877148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b56f0 0x7f91b0160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91ad600640 1 -- 172.21.15.62:0/688877148 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573563854 0 0) 0x7f91b00d3110 con 0x7f91b00b27a0 2024-04-25T09:07:09.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91ae000640 1 -- 172.21.15.62:0/688877148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b56f0 msgr2=0x7f91b0160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:09.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91ae000640 1 --2- 172.21.15.62:0/688877148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b56f0 0x7f91b0160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91ae000640 1 -- 172.21.15.62:0/688877148 >> v1:172.21.15.62:6789/0 conn(0x7f91b00b27a0 legacy=0x7f91b0161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.276+0000 7f91ae000640 1 -- 172.21.15.62:0/688877148 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00d33f0 con 0x7f91b00b6a90 2024-04-25T09:07:09.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.277+0000 7f91ae000640 1 --2- 172.21.15.62:0/688877148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b6a90 0x7f91b0161240 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009d90 tx=0x7f91a002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269e2d631af37287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:09.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91ad600640 1 -- 172.21.15.62:0/688877148 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0007740 con 0x7f91b00b6a90 2024-04-25T09:07:09.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91ad600640 1 -- 172.21.15.62:0/688877148 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a00078e0 con 0x7f91b00b6a90 2024-04-25T09:07:09.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91ad600640 1 -- 172.21.15.62:0/688877148 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0007cb0 con 0x7f91b00b6a90 2024-04-25T09:07:09.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91afe00640 1 -- 172.21.15.62:0/688877148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b6a90 msgr2=0x7f91b0161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91afe00640 1 --2- 172.21.15.62:0/688877148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b6a90 0x7f91b0161240 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009d90 tx=0x7f91a002a660 comp rx=0 tx=0).stop 2024-04-25T09:07:09.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91afe00640 1 -- 172.21.15.62:0/688877148 shutdown_connections 2024-04-25T09:07:09.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91afe00640 1 --2- 172.21.15.62:0/688877148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b6a90 0x7f91b0161240 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91afe00640 1 --2- 172.21.15.62:0/688877148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b56f0 0x7f91b0160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.278+0000 7f91afe00640 1 -- 172.21.15.62:0/688877148 >> 172.21.15.62:0/688877148 conn(0x7f91b0066d10 msgr2=0x7f91b0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:09.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.279+0000 7f91afe00640 1 -- 172.21.15.62:0/688877148 shutdown_connections 2024-04-25T09:07:09.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.279+0000 7f91afe00640 1 -- 172.21.15.62:0/688877148 wait complete. 2024-04-25T09:07:09.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.280+0000 7f91afe00640 1 Processor -- start 2024-04-25T09:07:09.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.280+0000 7f91afe00640 1 -- start start 2024-04-25T09:07:09.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.281+0000 7f91afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b27a0 0x7f91b00cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.281+0000 7f91afe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b00b56f0 0x7f91b01fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.281+0000 7f91afe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b6a90 0x7f91b01f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.281+0000 7f91afe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00b2d60 con 0x7f91b00b27a0 2024-04-25T09:07:09.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.281+0000 7f91afe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00cea50 con 0x7f91b00b56f0 2024-04-25T09:07:09.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.281+0000 7f91afe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00b2ee0 con 0x7f91b00b6a90 2024-04-25T09:07:09.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91aea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b27a0 0x7f91b00cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91aea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b27a0 0x7f91b00cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55396/0 (socket says 172.21.15.62:55396) 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91aea00640 1 -- 172.21.15.62:0/3131100572 learned_addr learned my addr 172.21.15.62:0/3131100572 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91af400640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b6a90 0x7f91b01f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91ae000640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b00b56f0 0x7f91b01fbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91aea00640 1 -- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b6a90 msgr2=0x7f91b01f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91aea00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b6a90 0x7f91b01f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91aea00640 1 -- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b00b56f0 msgr2=0x7f91b01fbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91aea00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b00b56f0 0x7f91b01fbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.282+0000 7f91aea00640 1 -- 172.21.15.62:0/3131100572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b01f8810 con 0x7f91b00b27a0 2024-04-25T09:07:09.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.283+0000 7f91aea00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b27a0 0x7f91b00cd7b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f919800d800 tx=0x7f919800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:09.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.283+0000 7f91a7400640 1 -- 172.21.15.62:0/3131100572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919800b6c0 con 0x7f91b00b27a0 2024-04-25T09:07:09.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.283+0000 7f91a7400640 1 -- 172.21.15.62:0/3131100572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9198019070 con 0x7f91b00b27a0 2024-04-25T09:07:09.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.283+0000 7f91a7400640 1 -- 172.21.15.62:0/3131100572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919800ba80 con 0x7f91b00b27a0 2024-04-25T09:07:09.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.283+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b01f8af0 con 0x7f91b00b27a0 2024-04-25T09:07:09.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.283+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b01fb5c0 con 0x7f91b00b27a0 2024-04-25T09:07:09.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.284+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b004ef20 con 0x7f91b00b27a0 2024-04-25T09:07:09.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.285+0000 7f91a7400640 1 -- 172.21.15.62:0/3131100572 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f919800bc20 con 0x7f91b00b27a0 2024-04-25T09:07:09.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.286+0000 7f91a7400640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f918c06b500 0x7f918c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.286+0000 7f91a7400640 1 -- 172.21.15.62:0/3131100572 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f919808a7e0 con 0x7f91b00b27a0 2024-04-25T09:07:09.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.287+0000 7f91ae000640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f918c06b500 0x7f918c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.288+0000 7f91ae000640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f918c06b500 0x7f918c06d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f91a0034000 tx=0x7f91a0033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:09.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.291+0000 7f91a7400640 1 -- 172.21.15.62:0/3131100572 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9198059530 con 0x7f91b00b27a0 2024-04-25T09:07:09.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.528+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91b00b6180 con 0x7f91b00b27a0 2024-04-25T09:07:09.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.530+0000 7f91a7400640 1 -- 172.21.15.62:0/3131100572 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f9198058e30 con 0x7f91b00b27a0 2024-04-25T09:07:09.518 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:09.519 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2179586213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2179586213},{"type":"v1","addr":"172.21.15.121:6836","nonce":2179586213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1425308084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1425308084},{"type":"v1","addr":"172.21.15.121:6837","nonce":1425308084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/231681595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":231681595},{"type":"v1","addr":"172.21.15.62:6834","nonce":231681595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:04:25.639815+0000","modified":"2024-04-25T09:04:39.952254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/274650142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":274650142},{"type":"v1","addr":"172.21.15.62:6835","nonce":274650142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:07:09.519 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-04-25T09:07:09.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.532+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f918c06b500 msgr2=0x7f918c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.532+0000 7f91afe00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f918c06b500 0x7f918c06d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f91a0034000 tx=0x7f91a0033040 comp rx=0 tx=0).stop 2024-04-25T09:07:09.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.532+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b27a0 msgr2=0x7f91b00cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.532+0000 7f91afe00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b27a0 0x7f91b00cd7b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f919800d800 tx=0x7f919800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.533+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 shutdown_connections 2024-04-25T09:07:09.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.533+0000 7f91afe00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f918c06b500 0x7f918c06d9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.533+0000 7f91afe00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91b00b6a90 0x7f91b01f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.533+0000 7f91afe00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91b00b56f0 0x7f91b01fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.533+0000 7f91afe00640 1 --2- 172.21.15.62:0/3131100572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91b00b27a0 0x7f91b00cd7b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.533+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 >> 172.21.15.62:0/3131100572 conn(0x7f91b0066d10 msgr2=0x7f91b0160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:09.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.533+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 shutdown_connections 2024-04-25T09:07:09.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.533+0000 7f91afe00640 1 -- 172.21.15.62:0/3131100572 wait complete. 2024-04-25T09:07:09.540 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:09.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.759+0000 7fdef5a00640 1 Processor -- start 2024-04-25T09:07:09.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.759+0000 7fdef5a00640 1 -- start start 2024-04-25T09:07:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.760+0000 7fdef5a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 0x7fdef015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.760+0000 7fdef5a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef00cf2d0 con 0x7fdef0144c10 2024-04-25T09:07:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.761+0000 7fdef5a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef015faa0 con 0x7fdef00cee40 2024-04-25T09:07:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.761+0000 7fdef5a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0144ff0 con 0x7fdef006d900 2024-04-25T09:07:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.761+0000 7fdeefe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdef0144c10 0x7fdef006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50948/0 (socket says 172.21.15.62:50948) 2024-04-25T09:07:09.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.761+0000 7fdeefe00640 1 -- 172.21.15.62:0/2792993926 learned_addr learned my addr 172.21.15.62:0/2792993926 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.761+0000 7fdeef400640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 0x7fdef015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.761+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452417876 0 0) 0x7fdef00cf2d0 con 0x7fdef0144c10 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.761+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded4003520 con 0x7fdef0144c10 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2063562205 0 0) 0x7fded4003520 con 0x7fdef0144c10 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 msgr2=0x7fdef015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 0x7fdef015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 >> v1:172.21.15.121:6789/0 conn(0x7fdef00cee40 legacy=0x7fdef00b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef015fd80 con 0x7fdef0144c10 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fdee0001670 con 0x7fdef0144c10 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 >> v1:172.21.15.62:6789/0 conn(0x7fdef0144c10 legacy=0x7fdef006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded40039e0 0x7fded4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4004800 0x7fded4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 0x7fded4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0144ff0 con 0x7fded4004800 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded400bb50 con 0x7fded40039e0 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded400bd20 con 0x7fdef006d900 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.762+0000 7fdeef400640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4004800 0x7fded4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.763+0000 7fdef5000640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 0x7fded4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.763+0000 7fdeef400640 1 -- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 msgr2=0x7fded4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.763+0000 7fdeef400640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 0x7fded4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.763+0000 7fdeef400640 1 -- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded40039e0 msgr2=0x7fded4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.763+0000 7fdeefe00640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded40039e0 0x7fded4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.763+0000 7fdeef400640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded40039e0 0x7fded4003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.763+0000 7fdeef400640 1 -- 172.21.15.62:0/2792993926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded4003520 con 0x7fded4004800 2024-04-25T09:07:09.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.763+0000 7fdef5000640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef006d900 0x7fded4005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:09.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.764+0000 7fdeef400640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4004800 0x7fded4004c80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009af0 tx=0x7fded4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83e5bf20b172a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:09.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.764+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc036070 con 0x7fded4004800 2024-04-25T09:07:09.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.764+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc031400 con 0x7fded4004800 2024-04-25T09:07:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.764+0000 7fdeeea00640 1 -- 172.21.15.62:0/2792993926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc009690 con 0x7fded4004800 2024-04-25T09:07:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.765+0000 7fdef5a00640 1 -- 172.21.15.62:0/2792993926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4004800 msgr2=0x7fded4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.765+0000 7fdef5a00640 1 --2- 172.21.15.62:0/2792993926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fded4004800 0x7fded4004c80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009af0 tx=0x7fded4005830 comp rx=0 tx=0).stop 2024-04-25T09:07:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.765+0000 7fdef5000640 1 -- 172.21.15.62:0/2792993926 reap_dead start 2024-04-25T09:07:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.765+0000 7fdef5a00640 1 -- 172.21.15.62:0/2792993926 shutdown_connections 2024-04-25T09:07:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.765+0000 7fdef5a00640 1 -- 172.21.15.62:0/2792993926 >> 172.21.15.62:0/2792993926 conn(0x7fdef00b14d0 msgr2=0x7fdef00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.765+0000 7fdef5a00640 1 -- 172.21.15.62:0/2792993926 shutdown_connections 2024-04-25T09:07:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.765+0000 7fdef5a00640 1 -- 172.21.15.62:0/2792993926 wait complete. 2024-04-25T09:07:09.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.767+0000 7fdef5a00640 1 Processor -- start 2024-04-25T09:07:09.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.767+0000 7fdef5a00640 1 -- start start 2024-04-25T09:07:09.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdef5a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded4004800 0x7fdef015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdef5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdef015b1d0 0x7fdef013d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdef5a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef013d660 0x7fdef013f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdef5a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef00b41c0 con 0x7fdef015b1d0 2024-04-25T09:07:09.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdef5a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0160340 con 0x7fded4004800 2024-04-25T09:07:09.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdef5a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef015f450 con 0x7fdef013d660 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdeef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdef015b1d0 0x7fdef013d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdeef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdef015b1d0 0x7fdef013d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55416/0 (socket says 172.21.15.62:55416) 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.768+0000 7fdeef400640 1 -- 172.21.15.62:0/165753154 learned_addr learned my addr 172.21.15.62:0/165753154 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdef5000640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef013d660 0x7fdef013f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdeefe00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded4004800 0x7fdef015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdeef400640 1 -- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef013d660 msgr2=0x7fdef013f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdeef400640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef013d660 0x7fdef013f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdeef400640 1 -- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded4004800 msgr2=0x7fdef015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdeef400640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded4004800 0x7fdef015d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdeef400640 1 -- 172.21.15.62:0/165753154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef015dc90 con 0x7fdef015b1d0 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdeefe00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded4004800 0x7fdef015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:09.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdeef400640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdef015b1d0 0x7fdef013d120 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdedc034040 tx=0x7fdedc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:09.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdee7e00640 1 -- 172.21.15.62:0/165753154 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc040020 con 0x7fdef015b1d0 2024-04-25T09:07:09.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdee7e00640 1 -- 172.21.15.62:0/165753154 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc003b40 con 0x7fdef015b1d0 2024-04-25T09:07:09.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdee7e00640 1 -- 172.21.15.62:0/165753154 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc036040 con 0x7fdef015b1d0 2024-04-25T09:07:09.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.769+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef0153570 con 0x7fdef015b1d0 2024-04-25T09:07:09.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.770+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef015e1f0 con 0x7fdef015b1d0 2024-04-25T09:07:09.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.771+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef004ef20 con 0x7fdef015b1d0 2024-04-25T09:07:09.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.772+0000 7fdee7e00640 1 -- 172.21.15.62:0/165753154 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdedc043020 con 0x7fdef015b1d0 2024-04-25T09:07:09.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.773+0000 7fdee7e00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:09.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.773+0000 7fdee7e00640 1 -- 172.21.15.62:0/165753154 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdedc0aa740 con 0x7fdef015b1d0 2024-04-25T09:07:09.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.773+0000 7fdeefe00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:09.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.775+0000 7fdeefe00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fded006b4d0 0x7fded006d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdee0001eb0 tx=0x7fdee0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:09.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.777+0000 7fdee7e00640 1 -- 172.21.15.62:0/165753154 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdedc04a030 con 0x7fdef015b1d0 2024-04-25T09:07:09.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.975+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdef013da40 con 0x7fdef015b1d0 2024-04-25T09:07:09.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.978+0000 7fdee7e00640 1 -- 172.21.15.62:0/165753154 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16912 (secure 0 0 0) 0x7fdef015dc90 con 0x7fdef015b1d0 2024-04-25T09:07:09.967 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:09.967 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":37,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:06:47.512613+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:23.626010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:24.669175+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.1d","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fded006b4d0 msgr2=0x7fded006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fded006b4d0 0x7fded006d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdee0001eb0 tx=0x7fdee0009040 comp rx=0 tx=0).stop 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdef015b1d0 msgr2=0x7fdef013d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdef015b1d0 0x7fdef013d120 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdedc034040 tx=0x7fdedc004730 comp rx=0 tx=0).stop 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 shutdown_connections 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdef015b1d0 0x7fdef013d120 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdef013d660 0x7fdef013f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 --2- 172.21.15.62:0/165753154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fded4004800 0x7fdef015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 >> 172.21.15.62:0/165753154 conn(0x7fdef00b14d0 msgr2=0x7fdef00b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:09.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 shutdown_connections 2024-04-25T09:07:09.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:09.981+0000 7fdef5a00640 1 -- 172.21.15.62:0/165753154 wait complete. 2024-04-25T09:07:09.988 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:07:09.988 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:10.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.206+0000 7f4721600640 1 Processor -- start 2024-04-25T09:07:10.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.206+0000 7f4721600640 1 -- start start 2024-04-25T09:07:10.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.208+0000 7f4721600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 0x7f471c0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.208+0000 7f4721600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f471c0aed00 0x7f471c0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.208+0000 7f4721600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c06ca70 con 0x7f471c15bd10 2024-04-25T09:07:10.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.208+0000 7f4721600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471c06d660 con 0x7f471c0aed00 2024-04-25T09:07:10.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.208+0000 7f4721600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c15c0f0 con 0x7f471c0cccf0 2024-04-25T09:07:10.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.208+0000 7f471aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f471c15bd10 0x7f471c0cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50962/0 (socket says 172.21.15.62:50962) 2024-04-25T09:07:10.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.208+0000 7f471aa00640 1 -- 172.21.15.62:0/3524050463 learned_addr learned my addr 172.21.15.62:0/3524050463 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:10.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.208+0000 7f471b400640 1 --2- 172.21.15.62:0/3524050463 >> v2:172.21.15.62:3300/0 conn(0x7f471c0aed00 0x7f471c0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.209+0000 7f471a000640 1 --2- 172.21.15.62:0/3524050463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 0x7f471c0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.209+0000 7f4719600640 1 -- 172.21.15.62:0/3524050463 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831671294 0 0) 0x7f471c06ca70 con 0x7f471c15bd10 2024-04-25T09:07:10.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.209+0000 7f4719600640 1 -- 172.21.15.62:0/3524050463 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4700003520 con 0x7f471c15bd10 2024-04-25T09:07:10.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.209+0000 7f471b400640 1 -- 172.21.15.62:0/3524050463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 msgr2=0x7f471c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:10.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.209+0000 7f471b400640 1 --2- 172.21.15.62:0/3524050463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 0x7f471c0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.209+0000 7f471b400640 1 -- 172.21.15.62:0/3524050463 >> v1:172.21.15.62:6789/0 conn(0x7f471c15bd10 legacy=0x7f471c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.209+0000 7f4719600640 1 -- 172.21.15.62:0/3524050463 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108716361 0 0) 0x7f4700003520 con 0x7f471c15bd10 2024-04-25T09:07:10.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.209+0000 7f471b400640 1 -- 172.21.15.62:0/3524050463 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c06cd30 con 0x7f471c0aed00 2024-04-25T09:07:10.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.211+0000 7f471b400640 1 --2- 172.21.15.62:0/3524050463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c0aed00 0x7f471c0cd6c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f471c06d7e0 tx=0x7f471402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a5feb3bd5484a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:10.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.211+0000 7f4719600640 1 -- 172.21.15.62:0/3524050463 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c001670 con 0x7f471c0aed00 2024-04-25T09:07:10.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.211+0000 7f4719600640 1 -- 172.21.15.62:0/3524050463 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714007540 con 0x7f471c0aed00 2024-04-25T09:07:10.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.211+0000 7f4719600640 1 -- 172.21.15.62:0/3524050463 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714007910 con 0x7f471c0aed00 2024-04-25T09:07:10.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.212+0000 7f4721600640 1 -- 172.21.15.62:0/3524050463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c0aed00 msgr2=0x7f471c0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.212+0000 7f4721600640 1 --2- 172.21.15.62:0/3524050463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c0aed00 0x7f471c0cd6c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f471c06d7e0 tx=0x7f471402a5d0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.212+0000 7f4721600640 1 -- 172.21.15.62:0/3524050463 shutdown_connections 2024-04-25T09:07:10.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.212+0000 7f4721600640 1 --2- 172.21.15.62:0/3524050463 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c0aed00 0x7f471c0cd6c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.212+0000 7f4721600640 1 --2- 172.21.15.62:0/3524050463 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 0x7f471c0ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.212+0000 7f4721600640 1 -- 172.21.15.62:0/3524050463 >> 172.21.15.62:0/3524050463 conn(0x7f471c0668f0 msgr2=0x7f471c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:10.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.212+0000 7f4721600640 1 -- 172.21.15.62:0/3524050463 shutdown_connections 2024-04-25T09:07:10.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.212+0000 7f4721600640 1 -- 172.21.15.62:0/3524050463 wait complete. 2024-04-25T09:07:10.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.214+0000 7f4721600640 1 Processor -- start 2024-04-25T09:07:10.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.214+0000 7f4721600640 1 -- start start 2024-04-25T09:07:10.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f4721600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f471c0aed00 0x7f471c141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f4721600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 0x7f471c0af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f4721600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c15bd10 0x7f471c0ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f4721600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c06daf0 con 0x7f471c15bd10 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f4721600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c06d2f0 con 0x7f471c0aed00 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f4721600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c142700 con 0x7f471c0cccf0 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f471b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c15bd10 0x7f471c0ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f471b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c15bd10 0x7f471c0ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55436/0 (socket says 172.21.15.62:55436) 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.215+0000 7f471b400640 1 -- 172.21.15.62:0/413586427 learned_addr learned my addr 172.21.15.62:0/413586427 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f471a000640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 0x7f471c0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f471aa00640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f471c0aed00 0x7f471c141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f471b400640 1 -- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 msgr2=0x7f471c0af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f471b400640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 0x7f471c0af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f471b400640 1 -- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f471c0aed00 msgr2=0x7f471c141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f471b400640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f471c0aed00 0x7f471c141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f471b400640 1 -- 172.21.15.62:0/413586427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c1449c0 con 0x7f471c15bd10 2024-04-25T09:07:10.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f471b400640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c15bd10 0x7f471c0ad910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4714009e80 tx=0x7f4714004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:10.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.216+0000 7f4712a00640 1 -- 172.21.15.62:0/413586427 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714040070 con 0x7f471c15bd10 2024-04-25T09:07:10.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.217+0000 7f4712a00640 1 -- 172.21.15.62:0/413586427 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714031b40 con 0x7f471c15bd10 2024-04-25T09:07:10.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.217+0000 7f4712a00640 1 -- 172.21.15.62:0/413586427 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714009450 con 0x7f471c15bd10 2024-04-25T09:07:10.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.217+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c144ca0 con 0x7f471c15bd10 2024-04-25T09:07:10.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.217+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c145260 con 0x7f471c15bd10 2024-04-25T09:07:10.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.218+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c072c70 con 0x7f471c15bd10 2024-04-25T09:07:10.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.219+0000 7f4712a00640 1 -- 172.21.15.62:0/413586427 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4714039040 con 0x7f471c15bd10 2024-04-25T09:07:10.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.220+0000 7f4712a00640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f46fc06b520 0x7f46fc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.220+0000 7f4712a00640 1 -- 172.21.15.62:0/413586427 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f47140aba40 con 0x7f471c15bd10 2024-04-25T09:07:10.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.220+0000 7f471aa00640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f46fc06b520 0x7f46fc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.221+0000 7f471aa00640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f46fc06b520 0x7f46fc06d9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f470c005880 tx=0x7f470c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:10.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.224+0000 7f4712a00640 1 -- 172.21.15.62:0/413586427 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f471407a810 con 0x7f471c15bd10 2024-04-25T09:07:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.456+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f471c04f000 con 0x7f471c15bd10 2024-04-25T09:07:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.457+0000 7f4712a00640 1 -- 172.21.15.62:0/413586427 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f471407a110 con 0x7f471c15bd10 2024-04-25T09:07:10.446 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:10.446 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2179586213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2179586213},{"type":"v1","addr":"172.21.15.121:6836","nonce":2179586213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1425308084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1425308084},{"type":"v1","addr":"172.21.15.121:6837","nonce":1425308084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/231681595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":231681595},{"type":"v1","addr":"172.21.15.62:6834","nonce":231681595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:04:25.639815+0000","modified":"2024-04-25T09:04:39.952254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/274650142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":274650142},{"type":"v1","addr":"172.21.15.62:6835","nonce":274650142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:07:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f46fc06b520 msgr2=0x7f46fc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f46fc06b520 0x7f46fc06d9e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f470c005880 tx=0x7f470c009040 comp rx=0 tx=0).stop 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c15bd10 msgr2=0x7f471c0ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c15bd10 0x7f471c0ad910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4714009e80 tx=0x7f4714004a40 comp rx=0 tx=0).stop 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 shutdown_connections 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f46fc06b520 0x7f46fc06d9e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f471c15bd10 0x7f471c0ad910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f471c0cccf0 0x7f471c0af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 --2- 172.21.15.62:0/413586427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f471c0aed00 0x7f471c141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 >> 172.21.15.62:0/413586427 conn(0x7f471c0668f0 msgr2=0x7f471c0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 shutdown_connections 2024-04-25T09:07:10.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.460+0000 7f4721600640 1 -- 172.21.15.62:0/413586427 wait complete. 2024-04-25T09:07:10.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:10.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:10.517 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:10.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.663+0000 7f9d85e00640 1 Processor -- start 2024-04-25T09:07:10.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.664+0000 7f9d85e00640 1 -- start start 2024-04-25T09:07:10.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.665+0000 7f9d85e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80143500 0x7f9d80143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.665+0000 7f9d85e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9d8006d160 0x7f9d8006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.665+0000 7f9d85e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8006d700 con 0x7f9d80071140 2024-04-25T09:07:10.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.665+0000 7f9d85e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8006d9e0 con 0x7f9d8006d160 2024-04-25T09:07:10.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.665+0000 7f9d85e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8006dcc0 con 0x7f9d80143500 2024-04-25T09:07:10.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.666+0000 7f9d85400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9d80071140 0x7f9d8015ae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50964/0 (socket says 172.21.15.62:50964) 2024-04-25T09:07:10.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.666+0000 7f9d85400640 1 -- 172.21.15.62:0/1264884840 learned_addr learned my addr 172.21.15.62:0/1264884840 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:10.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.666+0000 7f9d84a00640 1 --2- 172.21.15.62:0/1264884840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80143500 0x7f9d80143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.666+0000 7f9d7fe00640 1 --2- 172.21.15.62:0/1264884840 >> v2:172.21.15.121:3300/0 conn(0x7f9d8006d160 0x7f9d8006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.666+0000 7f9d7f400640 1 -- 172.21.15.62:0/1264884840 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446375192 0 0) 0x7f9d8006d700 con 0x7f9d80071140 2024-04-25T09:07:10.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.666+0000 7f9d7f400640 1 -- 172.21.15.62:0/1264884840 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d64003520 con 0x7f9d80071140 2024-04-25T09:07:10.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.667+0000 7f9d84a00640 1 -- 172.21.15.62:0/1264884840 >> v2:172.21.15.121:3300/0 conn(0x7f9d8006d160 msgr2=0x7f9d8006e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:10.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.667+0000 7f9d84a00640 1 --2- 172.21.15.62:0/1264884840 >> v2:172.21.15.121:3300/0 conn(0x7f9d8006d160 0x7f9d8006e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.667+0000 7f9d7f400640 1 -- 172.21.15.62:0/1264884840 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 561952294 0 0) 0x7f9d64003520 con 0x7f9d80071140 2024-04-25T09:07:10.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.667+0000 7f9d84a00640 1 -- 172.21.15.62:0/1264884840 >> v1:172.21.15.62:6789/0 conn(0x7f9d80071140 legacy=0x7f9d8015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.667+0000 7f9d84a00640 1 -- 172.21.15.62:0/1264884840 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8006dfa0 con 0x7f9d80143500 2024-04-25T09:07:10.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.668+0000 7f9d84a00640 1 --2- 172.21.15.62:0/1264884840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80143500 0x7f9d80143ee0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000bfa0 tx=0x7f9d7002c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4234fe988466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:10.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.668+0000 7f9d7f400640 1 -- 172.21.15.62:0/1264884840 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70038070 con 0x7f9d80143500 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.668+0000 7f9d7f400640 1 -- 172.21.15.62:0/1264884840 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7000b2e0 con 0x7f9d80143500 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.668+0000 7f9d7f400640 1 -- 172.21.15.62:0/1264884840 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78001670 con 0x7f9d80143500 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.669+0000 7f9d85e00640 1 -- 172.21.15.62:0/1264884840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80143500 msgr2=0x7f9d80143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.669+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1264884840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80143500 0x7f9d80143ee0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000bfa0 tx=0x7f9d7002c7f0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.669+0000 7f9d85e00640 1 -- 172.21.15.62:0/1264884840 shutdown_connections 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.669+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1264884840 >> v2:172.21.15.121:3300/0 conn(0x7f9d8006d160 0x7f9d8006e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.669+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1264884840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80143500 0x7f9d80143ee0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.669+0000 7f9d85e00640 1 -- 172.21.15.62:0/1264884840 >> 172.21.15.62:0/1264884840 conn(0x7f9d80074290 msgr2=0x7f9d8006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.669+0000 7f9d85e00640 1 -- 172.21.15.62:0/1264884840 shutdown_connections 2024-04-25T09:07:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.669+0000 7f9d85e00640 1 -- 172.21.15.62:0/1264884840 wait complete. 2024-04-25T09:07:10.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.671+0000 7f9d85e00640 1 Processor -- start 2024-04-25T09:07:10.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.671+0000 7f9d85e00640 1 -- start start 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8006d160 0x7f9d8013c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80071140 0x7f9d80158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d80143500 0x7f9d80159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d801438e0 con 0x7f9d80143500 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80144420 con 0x7f9d8006d160 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80143a60 con 0x7f9d80071140 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d80143500 0x7f9d80159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d80143500 0x7f9d80159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55448/0 (socket says 172.21.15.62:55448) 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d85400640 1 -- 172.21.15.62:0/1623829451 learned_addr learned my addr 172.21.15.62:0/1623829451 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d7fe00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80071140 0x7f9d80158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.672+0000 7f9d84a00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8006d160 0x7f9d8013c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d85400640 1 -- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80071140 msgr2=0x7f9d80158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d85400640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80071140 0x7f9d80158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d85400640 1 -- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8006d160 msgr2=0x7f9d8013c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d85400640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8006d160 0x7f9d8013c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d85400640 1 -- 172.21.15.62:0/1623829451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8013c5a0 con 0x7f9d80143500 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d85400640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d80143500 0x7f9d80159300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c560 tx=0x7f9d7800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d7cc00640 1 -- 172.21.15.62:0/1623829451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78017070 con 0x7f9d80143500 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d7cc00640 1 -- 172.21.15.62:0/1623829451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78005ce0 con 0x7f9d80143500 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d7cc00640 1 -- 172.21.15.62:0/1623829451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d780072b0 con 0x7f9d80143500 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.673+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d801f3d90 con 0x7f9d80143500 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.674+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d801f4350 con 0x7f9d80143500 2024-04-25T09:07:10.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.675+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8004ef20 con 0x7f9d80143500 2024-04-25T09:07:10.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.676+0000 7f9d7cc00640 1 -- 172.21.15.62:0/1623829451 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d7801a070 con 0x7f9d80143500 2024-04-25T09:07:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.677+0000 7f9d7cc00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.677+0000 7f9d7cc00640 1 -- 172.21.15.62:0/1623829451 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9d78089740 con 0x7f9d80143500 2024-04-25T09:07:10.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.677+0000 7f9d84a00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:10.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.678+0000 7f9d84a00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d6006b4d0 0x7f9d6006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d70035000 tx=0x7f9d7000b000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.681+0000 7f9d7cc00640 1 -- 172.21.15.62:0/1623829451 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d8013c5a0 con 0x7f9d80143500 2024-04-25T09:07:10.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.923+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d8015a930 con 0x7f9d80143500 2024-04-25T09:07:10.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.924+0000 7f9d7cc00640 1 -- 172.21.15.62:0/1623829451 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f9d7805ce20 con 0x7f9d80143500 2024-04-25T09:07:10.913 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:10.913 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2179586213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2179586213},{"type":"v1","addr":"172.21.15.121:6836","nonce":2179586213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1425308084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1425308084},{"type":"v1","addr":"172.21.15.121:6837","nonce":1425308084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/231681595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":231681595},{"type":"v1","addr":"172.21.15.62:6834","nonce":231681595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:04:25.639815+0000","modified":"2024-04-25T09:04:39.952254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5043":{"gid":5043,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6835/274650142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":274650142},{"type":"v1","addr":"172.21.15.62:6835","nonce":274650142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:07:10.913 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-04-25T09:07:10.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d6006b4d0 msgr2=0x7f9d6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d6006b4d0 0x7f9d6006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d70035000 tx=0x7f9d7000b000 comp rx=0 tx=0).stop 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d80143500 msgr2=0x7f9d80159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d80143500 0x7f9d80159300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c560 tx=0x7f9d7800ca20 comp rx=0 tx=0).stop 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 shutdown_connections 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d80143500 0x7f9d80159300 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d80071140 0x7f9d80158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 --2- 172.21.15.62:0/1623829451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d8006d160 0x7f9d8013c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 >> 172.21.15.62:0/1623829451 conn(0x7f9d80074290 msgr2=0x7f9d8006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 shutdown_connections 2024-04-25T09:07:10.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:10.927+0000 7f9d85e00640 1 -- 172.21.15.62:0/1623829451 wait complete. 2024-04-25T09:07:10.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:11.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.133+0000 7f192f200640 1 Processor -- start 2024-04-25T09:07:11.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.133+0000 7f192f200640 1 -- start start 2024-04-25T09:07:11.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.134+0000 7f192f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f192815b9f0 0x7f19280d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.134+0000 7f192f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19280b9470 0x7f19280d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.134+0000 7f192f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192815f2c0 con 0x7f19280d1000 2024-04-25T09:07:11.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.134+0000 7f192f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19280b3420 con 0x7f19280b9470 2024-04-25T09:07:11.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.134+0000 7f192f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19280d1430 con 0x7f192815b9f0 2024-04-25T09:07:11.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f1927400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19280b9470 0x7f19280d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f192d200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f19280d1000 0x7f1928161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50968/0 (socket says 172.21.15.62:50968) 2024-04-25T09:07:11.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f192d200640 1 -- 172.21.15.62:0/1998360101 learned_addr learned my addr 172.21.15.62:0/1998360101 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:11.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f1927e00640 1 --2- 172.21.15.62:0/1998360101 >> v2:172.21.15.121:3300/0 conn(0x7f192815b9f0 0x7f19280d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f1927400640 1 -- 172.21.15.62:0/1998360101 >> v2:172.21.15.121:3300/0 conn(0x7f192815b9f0 msgr2=0x7f19280d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:11.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f1927400640 1 --2- 172.21.15.62:0/1998360101 >> v2:172.21.15.121:3300/0 conn(0x7f192815b9f0 0x7f19280d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f1927400640 1 -- 172.21.15.62:0/1998360101 >> v1:172.21.15.62:6789/0 conn(0x7f19280d1000 legacy=0x7f1928161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f1927400640 1 -- 172.21.15.62:0/1998360101 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192815e6b0 con 0x7f19280b9470 2024-04-25T09:07:11.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.135+0000 7f1926a00640 1 -- 172.21.15.62:0/1998360101 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761380555 0 0) 0x7f192815f2c0 con 0x7f19280d1000 2024-04-25T09:07:11.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.136+0000 7f1927400640 1 --2- 172.21.15.62:0/1998360101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19280b9470 0x7f19280d0ac0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f19280b63a0 tx=0x7f191002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b347bb9107ba3000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:11.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.137+0000 7f1926a00640 1 -- 172.21.15.62:0/1998360101 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910037070 con 0x7f19280b9470 2024-04-25T09:07:11.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.137+0000 7f1926a00640 1 -- 172.21.15.62:0/1998360101 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1910032440 con 0x7f19280b9470 2024-04-25T09:07:11.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.137+0000 7f1926a00640 1 -- 172.21.15.62:0/1998360101 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910032810 con 0x7f19280b9470 2024-04-25T09:07:11.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.137+0000 7f192f200640 1 -- 172.21.15.62:0/1998360101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19280b9470 msgr2=0x7f19280d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.137+0000 7f192f200640 1 --2- 172.21.15.62:0/1998360101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19280b9470 0x7f19280d0ac0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f19280b63a0 tx=0x7f191002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.138+0000 7f192f200640 1 -- 172.21.15.62:0/1998360101 shutdown_connections 2024-04-25T09:07:11.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.138+0000 7f192f200640 1 --2- 172.21.15.62:0/1998360101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19280b9470 0x7f19280d0ac0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.138+0000 7f192f200640 1 --2- 172.21.15.62:0/1998360101 >> v2:172.21.15.121:3300/0 conn(0x7f192815b9f0 0x7f19280d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.138+0000 7f192f200640 1 -- 172.21.15.62:0/1998360101 >> 172.21.15.62:0/1998360101 conn(0x7f192806b000 msgr2=0x7f192806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:11.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.138+0000 7f192f200640 1 -- 172.21.15.62:0/1998360101 shutdown_connections 2024-04-25T09:07:11.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.138+0000 7f192f200640 1 -- 172.21.15.62:0/1998360101 wait complete. 2024-04-25T09:07:11.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.140+0000 7f192f200640 1 Processor -- start 2024-04-25T09:07:11.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.140+0000 7f192f200640 1 -- start start 2024-04-25T09:07:11.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.141+0000 7f192f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19280b9470 0x7f1928161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.141+0000 7f192f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19280d1000 0x7f19281e9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.141+0000 7f192f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f192815b9f0 0x7f19280b8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.141+0000 7f192f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192815ef50 con 0x7f192815b9f0 2024-04-25T09:07:11.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.141+0000 7f192f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192815ec70 con 0x7f19280b9470 2024-04-25T09:07:11.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.141+0000 7f192f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192815e310 con 0x7f19280d1000 2024-04-25T09:07:11.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f192d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f192815b9f0 0x7f19280b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f192d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f192815b9f0 0x7f19280b8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55460/0 (socket says 172.21.15.62:55460) 2024-04-25T09:07:11.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f192d200640 1 -- 172.21.15.62:0/1526835486 learned_addr learned my addr 172.21.15.62:0/1526835486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:11.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f1927e00640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19280b9470 0x7f1928161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f1927400640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19280d1000 0x7f19281e9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f192d200640 1 -- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19280d1000 msgr2=0x7f19281e9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f192d200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19280d1000 0x7f19281e9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f192d200640 1 -- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19280b9470 msgr2=0x7f1928161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f192d200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19280b9470 0x7f1928161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f192d200640 1 -- 172.21.15.62:0/1526835486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280cac20 con 0x7f192815b9f0 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.142+0000 7f1927400640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19280d1000 0x7f19281e9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.143+0000 7f192d200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f192815b9f0 0x7f19280b8150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f192000c7a0 tx=0x7f192000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.143+0000 7f191fe00640 1 -- 172.21.15.62:0/1526835486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920018070 con 0x7f192815b9f0 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.143+0000 7f191fe00640 1 -- 172.21.15.62:0/1526835486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192000a640 con 0x7f192815b9f0 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.143+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192815fa20 con 0x7f192815b9f0 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.143+0000 7f191fe00640 1 -- 172.21.15.62:0/1526835486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192000aab0 con 0x7f192815b9f0 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.143+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192815ffe0 con 0x7f192815b9f0 2024-04-25T09:07:11.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.144+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19280729c0 con 0x7f192815b9f0 2024-04-25T09:07:11.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.145+0000 7f191fe00640 1 -- 172.21.15.62:0/1526835486 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f192001b070 con 0x7f192815b9f0 2024-04-25T09:07:11.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.147+0000 7f191fe00640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f190406b4d0 0x7f190406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.147+0000 7f191fe00640 1 -- 172.21.15.62:0/1526835486 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f19200899b0 con 0x7f192815b9f0 2024-04-25T09:07:11.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.147+0000 7f1927e00640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f190406b4d0 0x7f190406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.149+0000 7f1927e00640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f190406b4d0 0x7f190406d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1918006fd0 tx=0x7f1918008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:11.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.151+0000 7f191fe00640 1 -- 172.21.15.62:0/1526835486 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1920058670 con 0x7f192815b9f0 2024-04-25T09:07:11.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.352+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f192804f000 con 0x7f192815b9f0 2024-04-25T09:07:11.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.355+0000 7f191fe00640 1 -- 172.21.15.62:0/1526835486 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16912 (secure 0 0 0) 0x7f1920057f70 con 0x7f192815b9f0 2024-04-25T09:07:11.344 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:11.344 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":37,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:06:47.512613+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:23.626010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:24.669175+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":36,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.19","mappings":[{"from":4,"to":5}]},{"pgid":"5.1d","mappings":[{"from":4,"to":5}]},{"pgid":"5.28","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:11.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.358+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f190406b4d0 msgr2=0x7f190406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.358+0000 7f192f200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f190406b4d0 0x7f190406d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1918006fd0 tx=0x7f1918008040 comp rx=0 tx=0).stop 2024-04-25T09:07:11.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.358+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f192815b9f0 msgr2=0x7f19280b8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.358+0000 7f192f200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f192815b9f0 0x7f19280b8150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f192000c7a0 tx=0x7f192000cc60 comp rx=0 tx=0).stop 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.359+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 shutdown_connections 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.359+0000 7f192f200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f190406b4d0 0x7f190406d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.359+0000 7f192f200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f192815b9f0 0x7f19280b8150 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.359+0000 7f192f200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19280d1000 0x7f19281e9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.359+0000 7f192f200640 1 --2- 172.21.15.62:0/1526835486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19280b9470 0x7f1928161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.359+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 >> 172.21.15.62:0/1526835486 conn(0x7f192806b000 msgr2=0x7f19280b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.359+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 shutdown_connections 2024-04-25T09:07:11.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.359+0000 7f192f200640 1 -- 172.21.15.62:0/1526835486 wait complete. 2024-04-25T09:07:11.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:07:11.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.588+0000 7f7d02c00640 1 Processor -- start 2024-04-25T09:07:11.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.589+0000 7f7d02c00640 1 -- start start 2024-04-25T09:07:11.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.590+0000 7f7d02c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 0x7f7cfc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.590+0000 7f7d02c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cfc0ae7e0 0x7f7cfc162270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.590+0000 7f7d02c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0aec20 con 0x7f7cfc0ceac0 2024-04-25T09:07:11.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.590+0000 7f7d02c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc06e120 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.590+0000 7f7d02c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc06db60 con 0x7f7cfc15dd50 2024-04-25T09:07:11.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.591+0000 7f7cf3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cfc0ae7e0 0x7f7cfc162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.591+0000 7f7d00c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7cfc0ceac0 0x7f7cfc0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50978/0 (socket says 172.21.15.62:50978) 2024-04-25T09:07:11.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.591+0000 7f7d00c00640 1 -- 172.21.15.62:0/682314992 learned_addr learned my addr 172.21.15.62:0/682314992 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:11.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.591+0000 7f7cfbe00640 1 --2- 172.21.15.62:0/682314992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 0x7f7cfc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.591+0000 7f7cf3e00640 1 -- 172.21.15.62:0/682314992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 msgr2=0x7f7cfc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:11.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.591+0000 7f7cf3e00640 1 --2- 172.21.15.62:0/682314992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 0x7f7cfc06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.591+0000 7f7cf3e00640 1 -- 172.21.15.62:0/682314992 >> v1:172.21.15.62:6789/0 conn(0x7f7cfc0ceac0 legacy=0x7f7cfc0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.591+0000 7f7cf3e00640 1 -- 172.21.15.62:0/682314992 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc06de40 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.592+0000 7f7cf3e00640 1 --2- 172.21.15.62:0/682314992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc162270 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22963b3edfbec257 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:11.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.593+0000 7f7cfb400640 1 -- 172.21.15.62:0/682314992 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4037070 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.593+0000 7f7cfb400640 1 -- 172.21.15.62:0/682314992 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce402ae50 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.593+0000 7f7cfb400640 1 -- 172.21.15.62:0/682314992 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4032660 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.593+0000 7f7d02c00640 1 -- 172.21.15.62:0/682314992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 msgr2=0x7f7cfc162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.593+0000 7f7d02c00640 1 --2- 172.21.15.62:0/682314992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc162270 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce402a100 comp rx=0 tx=0).stop 2024-04-25T09:07:11.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.594+0000 7f7d02c00640 1 -- 172.21.15.62:0/682314992 shutdown_connections 2024-04-25T09:07:11.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.594+0000 7f7d02c00640 1 --2- 172.21.15.62:0/682314992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc162270 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.594+0000 7f7d02c00640 1 --2- 172.21.15.62:0/682314992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 0x7f7cfc06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.594+0000 7f7d02c00640 1 -- 172.21.15.62:0/682314992 >> 172.21.15.62:0/682314992 conn(0x7f7cfc0668f0 msgr2=0x7f7cfc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:11.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.594+0000 7f7d02c00640 1 -- 172.21.15.62:0/682314992 shutdown_connections 2024-04-25T09:07:11.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.594+0000 7f7d02c00640 1 -- 172.21.15.62:0/682314992 wait complete. 2024-04-25T09:07:11.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.595+0000 7f7d02c00640 1 Processor -- start 2024-04-25T09:07:11.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.596+0000 7f7d02c00640 1 -- start start 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.596+0000 7f7d02c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc1e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.596+0000 7f7d02c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cfc0ceac0 0x7f7cfc0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.596+0000 7f7d02c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 0x7f7cfc0b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.596+0000 7f7d02c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15e130 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.596+0000 7f7d02c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc161d50 con 0x7f7cfc0ceac0 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.596+0000 7f7d02c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15e2b0 con 0x7f7cfc15dd50 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cfbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc1e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cfbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc1e82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55478/0 (socket says 172.21.15.62:55478) 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cfbe00640 1 -- 172.21.15.62:0/3892546692 learned_addr learned my addr 172.21.15.62:0/3892546692 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cfbe00640 1 -- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 msgr2=0x7f7cfc0b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cfbe00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 0x7f7cfc0b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cf3e00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cfc0ceac0 0x7f7cfc0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cfbe00640 1 -- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cfc0ceac0 msgr2=0x7f7cfc0b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cfbe00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cfc0ceac0 0x7f7cfc0b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.597+0000 7f7cfbe00640 1 -- 172.21.15.62:0/3892546692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0b2c20 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.598+0000 7f7cfbe00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc1e82e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00d5f0 tx=0x7f7cec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:11.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.598+0000 7f7cf8c00640 1 -- 172.21.15.62:0/3892546692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0125f0 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.598+0000 7f7cf8c00640 1 -- 172.21.15.62:0/3892546692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec019070 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.598+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc0b2ef0 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.598+0000 7f7cf8c00640 1 -- 172.21.15.62:0/3892546692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0129d0 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.598+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc14ab00 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.599+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc072c70 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.600+0000 7f7cf8c00640 1 -- 172.21.15.62:0/3892546692 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cec012b70 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.601+0000 7f7cf8c00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd806b4d0 0x7f7cd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:11.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.601+0000 7f7cf8c00640 1 -- 172.21.15.62:0/3892546692 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7cec08a660 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.601+0000 7f7cf3e00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd806b4d0 0x7f7cd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:11.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.603+0000 7f7cf3e00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd806b4d0 0x7f7cd806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4034000 tx=0x7f7ce402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:11.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.605+0000 7f7cf8c00640 1 -- 172.21.15.62:0/3892546692 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cec059440 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.846+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7cfc04f000 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.835 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:07:11.847+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:07:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.851+0000 7f7cf8c00640 1 -- 172.21.15.62:0/3892546692 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7cec058d40 con 0x7f7cfc0ae7e0 2024-04-25T09:07:11.840 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd806b4d0 msgr2=0x7f7cd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd806b4d0 0x7f7cd806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4034000 tx=0x7f7ce402e000 comp rx=0 tx=0).stop 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 msgr2=0x7f7cfc1e82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc1e82e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00d5f0 tx=0x7f7cec00dab0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 shutdown_connections 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd806b4d0 0x7f7cd806d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cfc15dd50 0x7f7cfc0b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cfc0ceac0 0x7f7cfc0b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 --2- 172.21.15.62:0/3892546692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc0ae7e0 0x7f7cfc1e82e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 >> 172.21.15.62:0/3892546692 conn(0x7f7cfc0668f0 msgr2=0x7f7cfc068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 shutdown_connections 2024-04-25T09:07:11.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:11.854+0000 7f7d02c00640 1 -- 172.21.15.62:0/3892546692 wait complete. 2024-04-25T09:07:11.869 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:07:11.930 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:07:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.074+0000 7ff24aa00640 1 Processor -- start 2024-04-25T09:07:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.074+0000 7ff24aa00640 1 -- start start 2024-04-25T09:07:12.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.076+0000 7ff24aa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff24406e700 0x7ff2440b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.076+0000 7ff24aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff244073c60 0x7ff244070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.076+0000 7ff24aa00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2440b3200 con 0x7ff24415dda0 2024-04-25T09:07:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.076+0000 7ff24aa00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24415e180 con 0x7ff244073c60 2024-04-25T09:07:12.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.076+0000 7ff24aa00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440ae700 con 0x7ff24406e700 2024-04-25T09:07:12.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.077+0000 7ff248a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff244073c60 0x7ff244070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:12.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.077+0000 7ff248a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff244073c60 0x7ff244070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55496/0 (socket says 172.21.15.62:55496) 2024-04-25T09:07:12.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.077+0000 7ff248a00640 1 -- 172.21.15.62:0/3803348030 learned_addr learned my addr 172.21.15.62:0/3803348030 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:12.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.077+0000 7ff23b400640 1 --2- 172.21.15.62:0/3803348030 >> v2:172.21.15.121:3300/0 conn(0x7ff24406e700 0x7ff2440b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:12.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.077+0000 7ff248a00640 1 -- 172.21.15.62:0/3803348030 >> v2:172.21.15.121:3300/0 conn(0x7ff24406e700 msgr2=0x7ff2440b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:12.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.077+0000 7ff248a00640 1 --2- 172.21.15.62:0/3803348030 >> v2:172.21.15.121:3300/0 conn(0x7ff24406e700 0x7ff2440b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.077+0000 7ff248a00640 1 -- 172.21.15.62:0/3803348030 >> v1:172.21.15.62:6789/0 conn(0x7ff24415dda0 legacy=0x7ff24406dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:12.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.077+0000 7ff248a00640 1 -- 172.21.15.62:0/3803348030 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440b34e0 con 0x7ff244073c60 2024-04-25T09:07:12.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.079+0000 7ff248a00640 1 --2- 172.21.15.62:0/3803348030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff244073c60 0x7ff244070b30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff2440d0050 tx=0x7ff23c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa2a15dab999266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:12.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.079+0000 7ff243400640 1 -- 172.21.15.62:0/3803348030 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c037070 con 0x7ff244073c60 2024-04-25T09:07:12.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.079+0000 7ff243400640 1 -- 172.21.15.62:0/3803348030 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c032440 con 0x7ff244073c60 2024-04-25T09:07:12.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.079+0000 7ff243400640 1 -- 172.21.15.62:0/3803348030 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c032810 con 0x7ff244073c60 2024-04-25T09:07:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.080+0000 7ff24aa00640 1 -- 172.21.15.62:0/3803348030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff244073c60 msgr2=0x7ff244070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.080+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3803348030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff244073c60 0x7ff244070b30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff2440d0050 tx=0x7ff23c02a160 comp rx=0 tx=0).stop 2024-04-25T09:07:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.080+0000 7ff24aa00640 1 -- 172.21.15.62:0/3803348030 shutdown_connections 2024-04-25T09:07:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.080+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3803348030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff244073c60 0x7ff244070b30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.080+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3803348030 >> v2:172.21.15.121:3300/0 conn(0x7ff24406e700 0x7ff2440b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.080+0000 7ff24aa00640 1 -- 172.21.15.62:0/3803348030 >> 172.21.15.62:0/3803348030 conn(0x7ff2440d00a0 msgr2=0x7ff2440d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.080+0000 7ff24aa00640 1 -- 172.21.15.62:0/3803348030 shutdown_connections 2024-04-25T09:07:12.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.080+0000 7ff24aa00640 1 -- 172.21.15.62:0/3803348030 wait complete. 2024-04-25T09:07:12.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.082+0000 7ff24aa00640 1 Processor -- start 2024-04-25T09:07:12.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.082+0000 7ff24aa00640 1 -- start start 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.083+0000 7ff24aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff24406e700 0x7ff2440c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.083+0000 7ff24aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff244073c60 0x7ff244161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.083+0000 7ff24aa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff24415dda0 0x7ff24414a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.083+0000 7ff24aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24406f4a0 con 0x7ff24406e700 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.083+0000 7ff24aa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff244071070 con 0x7ff244073c60 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.083+0000 7ff24aa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440b37c0 con 0x7ff24415dda0 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff24406e700 0x7ff2440c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff24406e700 0x7ff2440c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55498/0 (socket says 172.21.15.62:55498) 2024-04-25T09:07:12.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 -- 172.21.15.62:0/3303495025 learned_addr learned my addr 172.21.15.62:0/3303495025 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:12.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff23b400640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff244073c60 0x7ff244161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff248a00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff24415dda0 0x7ff24414a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 -- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff24415dda0 msgr2=0x7ff24414a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff24415dda0 0x7ff24414a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 -- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff244073c60 msgr2=0x7ff244161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff244073c60 0x7ff244161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 -- 172.21.15.62:0/3303495025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440c62a0 con 0x7ff24406e700 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.084+0000 7ff243e00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff24406e700 0x7ff2440c5a00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff23400c790 tx=0x7ff23400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.085+0000 7ff240c00640 1 -- 172.21.15.62:0/3303495025 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234016020 con 0x7ff24406e700 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.085+0000 7ff240c00640 1 -- 172.21.15.62:0/3303495025 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff234006d50 con 0x7ff24406e700 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.085+0000 7ff240c00640 1 -- 172.21.15.62:0/3303495025 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234003bb0 con 0x7ff24406e700 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.085+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2440c6580 con 0x7ff24406e700 2024-04-25T09:07:12.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.085+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440c6b40 con 0x7ff24406e700 2024-04-25T09:07:12.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.086+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff244072550 con 0x7ff24406e700 2024-04-25T09:07:12.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.088+0000 7ff240c00640 1 -- 172.21.15.62:0/3303495025 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff23400a850 con 0x7ff24406e700 2024-04-25T09:07:12.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.089+0000 7ff240c00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff22006b550 0x7ff22006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:12.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.089+0000 7ff240c00640 1 -- 172.21.15.62:0/3303495025 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5911+0+0 (secure 0 0 0) 0x7ff234089f80 con 0x7ff24406e700 2024-04-25T09:07:12.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.089+0000 7ff23b400640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff22006b550 0x7ff22006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:12.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.091+0000 7ff23b400640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff22006b550 0x7ff22006da10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2440b4fa0 tx=0x7ff22c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.105+0000 7ff240c00640 1 -- 172.21.15.62:0/3303495025 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff244072550 con 0x7ff24406e700 2024-04-25T09:07:12.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.346+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff24404ef20 con 0x7ff24406e700 2024-04-25T09:07:12.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.863+0000 7ff240c00640 1 -- 172.21.15.62:0/3303495025 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff234020020 con 0x7ff24406e700 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff22006b550 msgr2=0x7ff22006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff22006b550 0x7ff22006da10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2440b4fa0 tx=0x7ff22c008040 comp rx=0 tx=0).stop 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff24406e700 msgr2=0x7ff2440c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff24406e700 0x7ff2440c5a00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff23400c790 tx=0x7ff23400cc50 comp rx=0 tx=0).stop 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 shutdown_connections 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff22006b550 0x7ff22006da10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff24415dda0 0x7ff24414a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff244073c60 0x7ff244161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 --2- 172.21.15.62:0/3303495025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff24406e700 0x7ff2440c5a00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 >> 172.21.15.62:0/3303495025 conn(0x7ff2440d00a0 msgr2=0x7ff2440b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:12.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 shutdown_connections 2024-04-25T09:07:12.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:12.867+0000 7ff24aa00640 1 -- 172.21.15.62:0/3303495025 wait complete. 2024-04-25T09:07:12.877 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:07:13.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.092+0000 7fce5f200640 1 Processor -- start 2024-04-25T09:07:13.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.092+0000 7fce5f200640 1 -- start start 2024-04-25T09:07:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.094+0000 7fce5f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fce600b34e0 0x7fce600cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.094+0000 7fce5f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce600b32d0 con 0x7fce600cc8e0 2024-04-25T09:07:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.094+0000 7fce5f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce600cf610 con 0x7fce60071600 2024-04-25T09:07:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.094+0000 7fce5f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce60071ae0 con 0x7fce600b34e0 2024-04-25T09:07:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.094+0000 7fce5e800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fce600cc8e0 0x7fce6016bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50986/0 (socket says 172.21.15.62:50986) 2024-04-25T09:07:13.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.094+0000 7fce5e800640 1 -- 172.21.15.62:0/3486719750 learned_addr learned my addr 172.21.15.62:0/3486719750 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5de00640 1 --2- 172.21.15.62:0/3486719750 >> v2:172.21.15.121:3300/0 conn(0x7fce600b34e0 0x7fce600cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198630269 0 0) 0x7fce600b32d0 con 0x7fce600cc8e0 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce44003520 con 0x7fce600cc8e0 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3891105031 0 0) 0x7fce44003520 con 0x7fce600cc8e0 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 >> v2:172.21.15.121:3300/0 conn(0x7fce600b34e0 msgr2=0x7fce600cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 --2- 172.21.15.62:0/3486719750 >> v2:172.21.15.121:3300/0 conn(0x7fce600b34e0 0x7fce600cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 >> v1:172.21.15.121:6789/0 conn(0x7fce60071600 legacy=0x7fce600b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60071dc0 con 0x7fce600cc8e0 2024-04-25T09:07:13.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fce580019f0 con 0x7fce600cc8e0 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 >> v1:172.21.15.62:6789/0 conn(0x7fce600cc8e0 legacy=0x7fce6016bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce44003a00 0x7fce44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.095+0000 7fce5ca00640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5ca00640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce440051c0 0x7fce4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce60071ae0 con 0x7fce44004800 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4400c0c0 con 0x7fce44003a00 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4400c2c0 con 0x7fce440051c0 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5d400640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5de00640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce44003a00 0x7fce44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5e800640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce440051c0 0x7fce4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5d400640 1 -- 172.21.15.62:0/3486719750 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce440051c0 msgr2=0x7fce4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:13.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5d400640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce440051c0 0x7fce4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5d400640 1 -- 172.21.15.62:0/3486719750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce44003a00 msgr2=0x7fce44003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:13.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5d400640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce44003a00 0x7fce44003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.096+0000 7fce5d400640 1 -- 172.21.15.62:0/3486719750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44003520 con 0x7fce44004800 2024-04-25T09:07:13.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.097+0000 7fce5e800640 1 -- 172.21.15.62:0/3486719750 reap_dead start 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.098+0000 7fce5d400640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce44004c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce4c001750 tx=0x7fce44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e8dc1ad5f3d204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.098+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c001370 con 0x7fce44004800 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.098+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c0052d0 con 0x7fce44004800 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.098+0000 7fce5ca00640 1 -- 172.21.15.62:0/3486719750 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c02abc0 con 0x7fce44004800 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.099+0000 7fce5f200640 1 -- 172.21.15.62:0/3486719750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 msgr2=0x7fce44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.099+0000 7fce5f200640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce44004c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce4c001750 tx=0x7fce44005830 comp rx=0 tx=0).stop 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.099+0000 7fce5f200640 1 -- 172.21.15.62:0/3486719750 shutdown_connections 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.099+0000 7fce5f200640 1 --2- 172.21.15.62:0/3486719750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce44004c80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.099+0000 7fce5f200640 1 -- 172.21.15.62:0/3486719750 >> 172.21.15.62:0/3486719750 conn(0x7fce60066ad0 msgr2=0x7fce60066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:13.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.099+0000 7fce5f200640 1 -- 172.21.15.62:0/3486719750 shutdown_connections 2024-04-25T09:07:13.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.099+0000 7fce5f200640 1 -- 172.21.15.62:0/3486719750 wait complete. 2024-04-25T09:07:13.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.101+0000 7fce5f200640 1 Processor -- start 2024-04-25T09:07:13.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.101+0000 7fce5f200640 1 -- start start 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.102+0000 7fce5f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce601548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.102+0000 7fce5f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce60162fe0 0x7fce60163440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.102+0000 7fce5f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce601647d0 0x7fce60164c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.102+0000 7fce5f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6006ca00 con 0x7fce44004800 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.102+0000 7fce5f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6006c720 con 0x7fce60162fe0 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.102+0000 7fce5f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce600cf0d0 con 0x7fce601647d0 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce601548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce601548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55512/0 (socket says 172.21.15.62:55512) 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5de00640 1 -- 172.21.15.62:0/1650246433 learned_addr learned my addr 172.21.15.62:0/1650246433 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5d400640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce60162fe0 0x7fce60163440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5de00640 1 -- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce601647d0 msgr2=0x7fce60164c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:07:13.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5de00640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce601647d0 0x7fce60164c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5de00640 1 -- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce60162fe0 msgr2=0x7fce60163440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:13.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5de00640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce60162fe0 0x7fce60163440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5de00640 1 -- 172.21.15.62:0/1650246433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60163a60 con 0x7fce44004800 2024-04-25T09:07:13.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.103+0000 7fce5d400640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce60162fe0 0x7fce60163440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:13.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.104+0000 7fce5de00640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce601548b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce500114e0 tx=0x7fce500119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:13.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.104+0000 7fce56000640 1 -- 172.21.15.62:0/1650246433 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50015870 con 0x7fce44004800 2024-04-25T09:07:13.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.104+0000 7fce56000640 1 -- 172.21.15.62:0/1650246433 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50015a10 con 0x7fce44004800 2024-04-25T09:07:13.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.104+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce60163d40 con 0x7fce44004800 2024-04-25T09:07:13.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.104+0000 7fce56000640 1 -- 172.21.15.62:0/1650246433 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50015dc0 con 0x7fce44004800 2024-04-25T09:07:13.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.104+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce60158b90 con 0x7fce44004800 2024-04-25T09:07:13.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.106+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce60072b30 con 0x7fce44004800 2024-04-25T09:07:13.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.107+0000 7fce56000640 1 -- 172.21.15.62:0/1650246433 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce50013400 con 0x7fce44004800 2024-04-25T09:07:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.108+0000 7fce56000640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fce4006b500 0x7fce4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.108+0000 7fce56000640 1 -- 172.21.15.62:0/1650246433 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fce5008ab60 con 0x7fce44004800 2024-04-25T09:07:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.108+0000 7fce5d400640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fce4006b500 0x7fce4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.110+0000 7fce5d400640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fce4006b500 0x7fce4006d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fce4c032000 tx=0x7fce4c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:13.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.113+0000 7fce56000640 1 -- 172.21.15.62:0/1650246433 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce50019030 con 0x7fce44004800 2024-04-25T09:07:13.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.316+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce600cd490 con 0x7fce44004800 2024-04-25T09:07:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.867+0000 7fce56000640 1 -- 172.21.15.62:0/1650246433 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce500592c0 con 0x7fce44004800 2024-04-25T09:07:13.856 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:07:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.872+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fce4006b500 msgr2=0x7fce4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.872+0000 7fce5f200640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fce4006b500 0x7fce4006d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fce4c032000 tx=0x7fce4c02e000 comp rx=0 tx=0).stop 2024-04-25T09:07:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.872+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 msgr2=0x7fce601548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.872+0000 7fce5f200640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce601548b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce500114e0 tx=0x7fce500119a0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.873+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 shutdown_connections 2024-04-25T09:07:13.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.873+0000 7fce5f200640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fce4006b500 0x7fce4006d9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.873+0000 7fce5f200640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fce601647d0 0x7fce60164c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.873+0000 7fce5f200640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce60162fe0 0x7fce60163440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.873+0000 7fce5f200640 1 --2- 172.21.15.62:0/1650246433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce44004800 0x7fce601548b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.873+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 >> 172.21.15.62:0/1650246433 conn(0x7fce60066ad0 msgr2=0x7fce600b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.873+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 shutdown_connections 2024-04-25T09:07:13.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:13.873+0000 7fce5f200640 1 -- 172.21.15.62:0/1650246433 wait complete. 2024-04-25T09:07:13.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:07:14.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.101+0000 7f9398800640 1 Processor -- start 2024-04-25T09:07:14.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.101+0000 7f9398800640 1 -- start start 2024-04-25T09:07:14.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9398800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f939015b850 0x7f939006c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:14.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9398800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f93900aebe0 0x7f93900cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:14.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9398800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93900703f0 con 0x7f939015bc30 2024-04-25T09:07:14.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9398800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9390071490 con 0x7f939015b850 2024-04-25T09:07:14.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9398800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93900706d0 con 0x7f93900aebe0 2024-04-25T09:07:14.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9395e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f939015b850 0x7f939006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:14.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9395e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f939015b850 0x7f939006c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55522/0 (socket says 172.21.15.62:55522) 2024-04-25T09:07:14.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9395e00640 1 -- 172.21.15.62:0/476955731 learned_addr learned my addr 172.21.15.62:0/476955731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:14.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.103+0000 7f9396800640 1 --2- 172.21.15.62:0/476955731 >> v2:172.21.15.121:3300/0 conn(0x7f93900aebe0 0x7f93900cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:14.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.104+0000 7f9395e00640 1 -- 172.21.15.62:0/476955731 >> v2:172.21.15.121:3300/0 conn(0x7f93900aebe0 msgr2=0x7f93900cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:14.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.104+0000 7f9395e00640 1 --2- 172.21.15.62:0/476955731 >> v2:172.21.15.121:3300/0 conn(0x7f93900aebe0 0x7f93900cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.104+0000 7f9395e00640 1 -- 172.21.15.62:0/476955731 >> v1:172.21.15.121:6789/0 conn(0x7f939015bc30 legacy=0x7f939006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:14.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.104+0000 7f9395e00640 1 -- 172.21.15.62:0/476955731 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93900709b0 con 0x7f939015b850 2024-04-25T09:07:14.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.104+0000 7f9396800640 1 --2- 172.21.15.62:0/476955731 >> v2:172.21.15.121:3300/0 conn(0x7f93900aebe0 0x7f93900cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:14.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.105+0000 7f9395e00640 1 --2- 172.21.15.62:0/476955731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f939015b850 0x7f939006c280 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f939006e750 tx=0x7f938402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22993cb5ea5d13c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.106+0000 7f9394a00640 1 -- 172.21.15.62:0/476955731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384037070 con 0x7f939015b850 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.106+0000 7f9394a00640 1 -- 172.21.15.62:0/476955731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384032440 con 0x7f939015b850 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.106+0000 7f9394a00640 1 -- 172.21.15.62:0/476955731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384032810 con 0x7f939015b850 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.106+0000 7f9398800640 1 -- 172.21.15.62:0/476955731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f939015b850 msgr2=0x7f939006c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.106+0000 7f9398800640 1 --2- 172.21.15.62:0/476955731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f939015b850 0x7f939006c280 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f939006e750 tx=0x7f938402a270 comp rx=0 tx=0).stop 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.107+0000 7f9398800640 1 -- 172.21.15.62:0/476955731 shutdown_connections 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.107+0000 7f9398800640 1 --2- 172.21.15.62:0/476955731 >> v2:172.21.15.121:3300/0 conn(0x7f93900aebe0 0x7f93900cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.107+0000 7f9398800640 1 --2- 172.21.15.62:0/476955731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f939015b850 0x7f939006c280 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.107+0000 7f9398800640 1 -- 172.21.15.62:0/476955731 >> 172.21.15.62:0/476955731 conn(0x7f93900668f0 msgr2=0x7f9390066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:14.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.107+0000 7f9398800640 1 -- 172.21.15.62:0/476955731 shutdown_connections 2024-04-25T09:07:14.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.107+0000 7f9398800640 1 -- 172.21.15.62:0/476955731 wait complete. 2024-04-25T09:07:14.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.109+0000 7f9398800640 1 Processor -- start 2024-04-25T09:07:14.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.109+0000 7f9398800640 1 -- start start 2024-04-25T09:07:14.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9398800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93900aebe0 0x7f93901e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:14.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9398800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f939015b850 0x7f93901e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:14.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9398800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f939015bc30 0x7f939015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:14.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9398800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939006ced0 con 0x7f93900aebe0 2024-04-25T09:07:14.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9398800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9390070f70 con 0x7f939015b850 2024-04-25T09:07:14.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9398800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939006d050 con 0x7f939015bc30 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9395e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93900aebe0 0x7f93901e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9395e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93900aebe0 0x7f93901e5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55526/0 (socket says 172.21.15.62:55526) 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9395e00640 1 -- 172.21.15.62:0/1038016402 learned_addr learned my addr 172.21.15.62:0/1038016402 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9396800640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f939015bc30 0x7f939015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9395400640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f939015b850 0x7f93901e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.110+0000 7f9395e00640 1 -- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f939015bc30 msgr2=0x7f939015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9395e00640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f939015bc30 0x7f939015de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9395e00640 1 -- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f939015b850 msgr2=0x7f93901e59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9395e00640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f939015b850 0x7f93901e59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9395e00640 1 -- 172.21.15.62:0/1038016402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9390145bf0 con 0x7f93900aebe0 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9396800640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f939015bc30 0x7f939015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9395400640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f939015b850 0x7f93901e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:14.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9395e00640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93900aebe0 0x7f93901e5470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f938402a2b0 tx=0x7f93840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:14.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f938a000640 1 -- 172.21.15.62:0/1038016402 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384041020 con 0x7f93900aebe0 2024-04-25T09:07:14.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f938a000640 1 -- 172.21.15.62:0/1038016402 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93840306c0 con 0x7f93900aebe0 2024-04-25T09:07:14.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f938a000640 1 -- 172.21.15.62:0/1038016402 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384037040 con 0x7f93900aebe0 2024-04-25T09:07:14.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9390145ed0 con 0x7f93900aebe0 2024-04-25T09:07:14.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.111+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9390146490 con 0x7f93900aebe0 2024-04-25T09:07:14.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.113+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9390067c40 con 0x7f93900aebe0 2024-04-25T09:07:14.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.114+0000 7f938a000640 1 -- 172.21.15.62:0/1038016402 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9384044020 con 0x7f93900aebe0 2024-04-25T09:07:14.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.115+0000 7f938a000640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f937406b4d0 0x7f937406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:14.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.115+0000 7f938a000640 1 -- 172.21.15.62:0/1038016402 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f93840aaa50 con 0x7f93900aebe0 2024-04-25T09:07:14.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.115+0000 7f9395400640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f937406b4d0 0x7f937406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:14.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.117+0000 7f9395400640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f937406b4d0 0x7f937406d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f939006dad0 tx=0x7f9380009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:14.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.119+0000 7f938a000640 1 -- 172.21.15.62:0/1038016402 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f938404b030 con 0x7f93900aebe0 2024-04-25T09:07:14.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.326+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f939006d790 con 0x7f93900aebe0 2024-04-25T09:07:14.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.877+0000 7f938a000640 1 -- 172.21.15.62:0/1038016402 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9384079880 con 0x7f93900aebe0 2024-04-25T09:07:14.866 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:07:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.883+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f937406b4d0 msgr2=0x7f937406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.883+0000 7f9398800640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f937406b4d0 0x7f937406d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f939006dad0 tx=0x7f9380009040 comp rx=0 tx=0).stop 2024-04-25T09:07:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.883+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93900aebe0 msgr2=0x7f93901e5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:14.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.883+0000 7f9398800640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93900aebe0 0x7f93901e5470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f938402a2b0 tx=0x7f93840046c0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.885+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 shutdown_connections 2024-04-25T09:07:14.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.885+0000 7f9398800640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f937406b4d0 0x7f937406d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.885+0000 7f9398800640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f939015bc30 0x7f939015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.885+0000 7f9398800640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f939015b850 0x7f93901e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.885+0000 7f9398800640 1 --2- 172.21.15.62:0/1038016402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93900aebe0 0x7f93901e5470 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:14.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.885+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 >> 172.21.15.62:0/1038016402 conn(0x7f93900668f0 msgr2=0x7f939006c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.885+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 shutdown_connections 2024-04-25T09:07:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:14.885+0000 7f9398800640 1 -- 172.21.15.62:0/1038016402 wait complete. 2024-04-25T09:07:14.903 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:07:15.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.104+0000 7f8a7d200640 1 Processor -- start 2024-04-25T09:07:15.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.104+0000 7f8a7d200640 1 -- start start 2024-04-25T09:07:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.106+0000 7f8a7d200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a7806d9d0 0x7f8a7815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.106+0000 7f8a7d200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8a7815e5f0 0x7f8a780b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.106+0000 7f8a7d200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7815ee30 con 0x7f8a780ceb50 2024-04-25T09:07:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.106+0000 7f8a7d200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a781618b0 con 0x7f8a7806d9d0 2024-04-25T09:07:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.106+0000 7f8a7d200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a78161b90 con 0x7f8a7815e5f0 2024-04-25T09:07:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.106+0000 7f8a76a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a7806d9d0 0x7f8a7815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.106+0000 7f8a76a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a7806d9d0 0x7f8a7815e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49810/0 (socket says 172.21.15.62:49810) 2024-04-25T09:07:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.106+0000 7f8a76a00640 1 -- 172.21.15.62:0/3805688750 learned_addr learned my addr 172.21.15.62:0/3805688750 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.107+0000 7f8a77e00640 1 --2- 172.21.15.62:0/3805688750 >> v2:172.21.15.121:3300/0 conn(0x7f8a7815e5f0 0x7f8a780b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.107+0000 7f8a76a00640 1 -- 172.21.15.62:0/3805688750 >> v2:172.21.15.121:3300/0 conn(0x7f8a7815e5f0 msgr2=0x7f8a780b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.107+0000 7f8a76a00640 1 --2- 172.21.15.62:0/3805688750 >> v2:172.21.15.121:3300/0 conn(0x7f8a7815e5f0 0x7f8a780b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.107+0000 7f8a76a00640 1 -- 172.21.15.62:0/3805688750 >> v1:172.21.15.121:6789/0 conn(0x7f8a780ceb50 legacy=0x7f8a7806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:15.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.107+0000 7f8a76a00640 1 -- 172.21.15.62:0/3805688750 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7815ea40 con 0x7f8a7806d9d0 2024-04-25T09:07:15.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.108+0000 7f8a76a00640 1 --2- 172.21.15.62:0/3805688750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7806d9d0 0x7f8a7815e0b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a780b3ad0 tx=0x7f8a6002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2b0cd6e20625db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.108+0000 7f8a76000640 1 -- 172.21.15.62:0/3805688750 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60037070 con 0x7f8a7806d9d0 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.108+0000 7f8a76000640 1 -- 172.21.15.62:0/3805688750 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60032440 con 0x7f8a7806d9d0 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.108+0000 7f8a76000640 1 -- 172.21.15.62:0/3805688750 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60032810 con 0x7f8a7806d9d0 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.109+0000 7f8a7d200640 1 -- 172.21.15.62:0/3805688750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7806d9d0 msgr2=0x7f8a7815e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.109+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3805688750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7806d9d0 0x7f8a7815e0b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8a780b3ad0 tx=0x7f8a6002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.109+0000 7f8a7d200640 1 -- 172.21.15.62:0/3805688750 shutdown_connections 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.109+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3805688750 >> v2:172.21.15.121:3300/0 conn(0x7f8a7815e5f0 0x7f8a780b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.109+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3805688750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7806d9d0 0x7f8a7815e0b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.109+0000 7f8a7d200640 1 -- 172.21.15.62:0/3805688750 >> 172.21.15.62:0/3805688750 conn(0x7f8a78066f20 msgr2=0x7f8a780b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:15.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.109+0000 7f8a7d200640 1 -- 172.21.15.62:0/3805688750 shutdown_connections 2024-04-25T09:07:15.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.109+0000 7f8a7d200640 1 -- 172.21.15.62:0/3805688750 wait complete. 2024-04-25T09:07:15.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.111+0000 7f8a7d200640 1 Processor -- start 2024-04-25T09:07:15.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.111+0000 7f8a7d200640 1 -- start start 2024-04-25T09:07:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.112+0000 7f8a7d200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a7806d9d0 0x7f8a78157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.112+0000 7f8a7d200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a780ceb50 0x7f8a78155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.112+0000 7f8a7d200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7815e5f0 0x7f8a78153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.112+0000 7f8a7d200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a78162510 con 0x7f8a7815e5f0 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.112+0000 7f8a7d200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a78162390 con 0x7f8a7806d9d0 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.112+0000 7f8a7d200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780b4a90 con 0x7f8a780ceb50 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.112+0000 7f8a77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7815e5f0 0x7f8a78153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7815e5f0 0x7f8a78153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49812/0 (socket says 172.21.15.62:49812) 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77e00640 1 -- 172.21.15.62:0/3878580347 learned_addr learned my addr 172.21.15.62:0/3878580347 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77400640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a7806d9d0 0x7f8a78157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a76a00640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a780ceb50 0x7f8a78155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77e00640 1 -- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a780ceb50 msgr2=0x7f8a78155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77e00640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a780ceb50 0x7f8a78155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77e00640 1 -- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a7806d9d0 msgr2=0x7f8a78157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77e00640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a7806d9d0 0x7f8a78157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77e00640 1 -- 172.21.15.62:0/3878580347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a781374b0 con 0x7f8a7815e5f0 2024-04-25T09:07:15.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a76a00640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a780ceb50 0x7f8a78155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:15.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.113+0000 7f8a77e00640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7815e5f0 0x7f8a78153f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a70007e60 tx=0x7f8a7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:15.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.114+0000 7f8a6f400640 1 -- 172.21.15.62:0/3878580347 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70014420 con 0x7f8a7815e5f0 2024-04-25T09:07:15.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.114+0000 7f8a6f400640 1 -- 172.21.15.62:0/3878580347 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a70019070 con 0x7f8a7815e5f0 2024-04-25T09:07:15.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.114+0000 7f8a6f400640 1 -- 172.21.15.62:0/3878580347 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70014820 con 0x7f8a7815e5f0 2024-04-25T09:07:15.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.114+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a78134800 con 0x7f8a7815e5f0 2024-04-25T09:07:15.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.114+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a78134dc0 con 0x7f8a7815e5f0 2024-04-25T09:07:15.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.115+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7806abe0 con 0x7f8a7815e5f0 2024-04-25T09:07:15.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.116+0000 7f8a6f400640 1 -- 172.21.15.62:0/3878580347 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a700149c0 con 0x7f8a7815e5f0 2024-04-25T09:07:15.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.117+0000 7f8a6f400640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a5406b4d0 0x7f8a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.117+0000 7f8a6f400640 1 -- 172.21.15.62:0/3878580347 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a7008a800 con 0x7f8a7815e5f0 2024-04-25T09:07:15.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.118+0000 7f8a77400640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a5406b4d0 0x7f8a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.119+0000 7f8a77400640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a5406b4d0 0x7f8a5406d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8a68004ea0 tx=0x7f8a68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:15.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.121+0000 7f8a6f400640 1 -- 172.21.15.62:0/3878580347 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a781374b0 con 0x7f8a7815e5f0 2024-04-25T09:07:15.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.329+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8a7804ef20 con 0x7f8a7815e5f0 2024-04-25T09:07:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.335+0000 7f8a6f400640 1 -- 172.21.15.62:0/3878580347 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8a700598e0 con 0x7f8a7815e5f0 2024-04-25T09:07:15.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.337+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a5406b4d0 msgr2=0x7f8a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.337+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a5406b4d0 0x7f8a5406d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8a68004ea0 tx=0x7f8a68009040 comp rx=0 tx=0).stop 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.337+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7815e5f0 msgr2=0x7f8a78153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.337+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7815e5f0 0x7f8a78153f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8a70007e60 tx=0x7f8a7000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.338+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 shutdown_connections 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.338+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a5406b4d0 0x7f8a5406d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.338+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a7815e5f0 0x7f8a78153f10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.338+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a780ceb50 0x7f8a78155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.338+0000 7f8a7d200640 1 --2- 172.21.15.62:0/3878580347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a7806d9d0 0x7f8a78157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.338+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 >> 172.21.15.62:0/3878580347 conn(0x7f8a78066f20 msgr2=0x7f8a780ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.338+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 shutdown_connections 2024-04-25T09:07:15.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.338+0000 7f8a7d200640 1 -- 172.21.15.62:0/3878580347 wait complete. 2024-04-25T09:07:15.345 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T09:07:15.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.577+0000 7f0227e00640 1 Processor -- start 2024-04-25T09:07:15.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.577+0000 7f0227e00640 1 -- start start 2024-04-25T09:07:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.579+0000 7f0227e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02280b56f0 0x7f0228160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.579+0000 7f0227e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 0x7f0228161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.579+0000 7f0227e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02280d3110 con 0x7f02280b27a0 2024-04-25T09:07:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.579+0000 7f0227e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0228161780 con 0x7f02280b56f0 2024-04-25T09:07:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.579+0000 7f0227e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cedc0 con 0x7f02280b6a90 2024-04-25T09:07:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.580+0000 7f0226a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02280b56f0 0x7f0228160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.580+0000 7f0226a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02280b56f0 0x7f0228160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49824/0 (socket says 172.21.15.62:49824) 2024-04-25T09:07:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.580+0000 7f0226a00640 1 -- 172.21.15.62:0/3809529606 learned_addr learned my addr 172.21.15.62:0/3809529606 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:15.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.580+0000 7f0226a00640 1 -- 172.21.15.62:0/3809529606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 msgr2=0x7f0228161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:15.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.580+0000 7f0226a00640 1 --2- 172.21.15.62:0/3809529606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 0x7f0228161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.580+0000 7f0226a00640 1 -- 172.21.15.62:0/3809529606 >> v1:172.21.15.62:6789/0 conn(0x7f02280b27a0 legacy=0x7f0228161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.580+0000 7f0226a00640 1 -- 172.21.15.62:0/3809529606 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02280d33f0 con 0x7f02280b56f0 2024-04-25T09:07:15.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.581+0000 7f0226a00640 1 --2- 172.21.15.62:0/3809529606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f0228160d00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0218009870 tx=0x7f021802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e507e76d3fbc3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:15.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0226000640 1 -- 172.21.15.62:0/3809529606 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218037070 con 0x7f02280b56f0 2024-04-25T09:07:15.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0226000640 1 -- 172.21.15.62:0/3809529606 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0218032440 con 0x7f02280b56f0 2024-04-25T09:07:15.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0226000640 1 -- 172.21.15.62:0/3809529606 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218032810 con 0x7f02280b56f0 2024-04-25T09:07:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0227e00640 1 -- 172.21.15.62:0/3809529606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 msgr2=0x7f0228160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0227e00640 1 --2- 172.21.15.62:0/3809529606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f0228160d00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0218009870 tx=0x7f021802a250 comp rx=0 tx=0).stop 2024-04-25T09:07:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0227e00640 1 -- 172.21.15.62:0/3809529606 shutdown_connections 2024-04-25T09:07:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0227e00640 1 --2- 172.21.15.62:0/3809529606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 0x7f0228161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0227e00640 1 --2- 172.21.15.62:0/3809529606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f0228160d00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0227e00640 1 -- 172.21.15.62:0/3809529606 >> 172.21.15.62:0/3809529606 conn(0x7f0228066d10 msgr2=0x7f0228067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0227e00640 1 -- 172.21.15.62:0/3809529606 shutdown_connections 2024-04-25T09:07:15.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.582+0000 7f0227e00640 1 -- 172.21.15.62:0/3809529606 wait complete. 2024-04-25T09:07:15.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.584+0000 7f0227e00640 1 Processor -- start 2024-04-25T09:07:15.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.584+0000 7f0227e00640 1 -- start start 2024-04-25T09:07:15.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.585+0000 7f0227e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b27a0 0x7f02281f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.585+0000 7f0227e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f02280c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.585+0000 7f0227e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 0x7f02280c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.585+0000 7f0227e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280b2d60 con 0x7f02280b56f0 2024-04-25T09:07:15.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.585+0000 7f0227e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cea50 con 0x7f02280b27a0 2024-04-25T09:07:15.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.585+0000 7f0227e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280b2ee0 con 0x7f02280b6a90 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f02280c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f02280c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49834/0 (socket says 172.21.15.62:49834) 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 -- 172.21.15.62:0/2236581545 learned_addr learned my addr 172.21.15.62:0/2236581545 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f0226a00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b27a0 0x7f02281f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 -- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 msgr2=0x7f02280c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f0227400640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 0x7f02280c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 0x7f02280c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 -- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b27a0 msgr2=0x7f02281f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b27a0 0x7f02281f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 -- 172.21.15.62:0/2236581545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02281f4cf0 con 0x7f02280b56f0 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f0226a00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b27a0 0x7f02281f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.586+0000 7f021fe00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f02280c95c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f021000b410 tx=0x7f021000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.587+0000 7f021ea00640 1 -- 172.21.15.62:0/2236581545 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210015020 con 0x7f02280b56f0 2024-04-25T09:07:15.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.587+0000 7f021ea00640 1 -- 172.21.15.62:0/2236581545 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210011850 con 0x7f02280b56f0 2024-04-25T09:07:15.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.587+0000 7f021ea00640 1 -- 172.21.15.62:0/2236581545 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210011c40 con 0x7f02280b56f0 2024-04-25T09:07:15.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.587+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02281f40e0 con 0x7f02280b56f0 2024-04-25T09:07:15.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.587+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02281f46a0 con 0x7f02280b56f0 2024-04-25T09:07:15.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.588+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022804ef20 con 0x7f02280b56f0 2024-04-25T09:07:15.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.589+0000 7f021ea00640 1 -- 172.21.15.62:0/2236581545 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0210018020 con 0x7f02280b56f0 2024-04-25T09:07:15.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.590+0000 7f021ea00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020406b520 0x7f020406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:15.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.590+0000 7f021ea00640 1 -- 172.21.15.62:0/2236581545 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0210089d60 con 0x7f02280b56f0 2024-04-25T09:07:15.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.591+0000 7f0226a00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020406b520 0x7f020406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:15.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.592+0000 7f0226a00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020406b520 0x7f020406d9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0218034000 tx=0x7f021802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:15.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.595+0000 7f021ea00640 1 -- 172.21.15.62:0/2236581545 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f021001f020 con 0x7f02280b56f0 2024-04-25T09:07:15.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:15.877+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f02280ce580 con 0x7f02280b56f0 2024-04-25T09:07:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.890+0000 7f021ea00640 1 -- 172.21.15.62:0/2236581545 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v187) v1 ==== 148+0+0 (secure 0 0 0) 0x7f02100172c0 con 0x7f02280b56f0 2024-04-25T09:07:16.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020406b520 msgr2=0x7f020406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:16.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020406b520 0x7f020406d9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0218034000 tx=0x7f021802e000 comp rx=0 tx=0).stop 2024-04-25T09:07:16.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 msgr2=0x7f02280c95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:16.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f02280c95c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f021000b410 tx=0x7f021000b8d0 comp rx=0 tx=0).stop 2024-04-25T09:07:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 shutdown_connections 2024-04-25T09:07:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020406b520 0x7f020406d9e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f02280c95c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b6a90 0x7f02280c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 --2- 172.21.15.62:0/2236581545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b27a0 0x7f02281f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.894+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 >> 172.21.15.62:0/2236581545 conn(0x7f0228066d10 msgr2=0x7f02280b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.895+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 shutdown_connections 2024-04-25T09:07:16.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:16.895+0000 7f0227e00640 1 -- 172.21.15.62:0/2236581545 wait complete. 2024-04-25T09:07:16.905 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T09:07:16.905 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:07:16.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:17.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.129+0000 7f500d400640 1 Processor -- start 2024-04-25T09:07:17.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.129+0000 7f500d400640 1 -- start start 2024-04-25T09:07:17.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.131+0000 7f500d400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5008144c10 0x7f500815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.131+0000 7f500d400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f50080cee40 0x7f500806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.131+0000 7f500d400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500815faa0 con 0x7f500806d730 2024-04-25T09:07:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.131+0000 7f500d400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5008144ff0 con 0x7f50080cee40 2024-04-25T09:07:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.131+0000 7f500d400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5008160340 con 0x7f5008144c10 2024-04-25T09:07:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.131+0000 7f500ca00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f500806d730 0x7f50080b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36870/0 (socket says 172.21.15.62:36870) 2024-04-25T09:07:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.131+0000 7f500ca00640 1 -- 172.21.15.62:0/3865875097 learned_addr learned my addr 172.21.15.62:0/3865875097 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.131+0000 7f5007400640 1 --2- 172.21.15.62:0/3865875097 >> v2:172.21.15.62:3300/0 conn(0x7f50080cee40 0x7f500806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.132+0000 7f5007e00640 1 --2- 172.21.15.62:0/3865875097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5008144c10 0x7f500815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.132+0000 7f5007400640 1 -- 172.21.15.62:0/3865875097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5008144c10 msgr2=0x7f500815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:17.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.132+0000 7f5007400640 1 --2- 172.21.15.62:0/3865875097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5008144c10 0x7f500815f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.132+0000 7f5006a00640 1 -- 172.21.15.62:0/3865875097 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390203991 0 0) 0x7f500815faa0 con 0x7f500806d730 2024-04-25T09:07:17.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.132+0000 7f5007400640 1 -- 172.21.15.62:0/3865875097 >> v1:172.21.15.62:6789/0 conn(0x7f500806d730 legacy=0x7f50080b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.132+0000 7f5007400640 1 -- 172.21.15.62:0/3865875097 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500815fd80 con 0x7f50080cee40 2024-04-25T09:07:17.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.133+0000 7f5007400640 1 --2- 172.21.15.62:0/3865875097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50080cee40 0x7f500806d1f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40097a0 tx=0x7f4ff402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255139de5a17fdbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.133+0000 7f5006a00640 1 -- 172.21.15.62:0/3865875097 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4037070 con 0x7f50080cee40 2024-04-25T09:07:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.133+0000 7f5006a00640 1 -- 172.21.15.62:0/3865875097 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff402ae50 con 0x7f50080cee40 2024-04-25T09:07:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.133+0000 7f5006a00640 1 -- 172.21.15.62:0/3865875097 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4032660 con 0x7f50080cee40 2024-04-25T09:07:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.134+0000 7f500d400640 1 -- 172.21.15.62:0/3865875097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50080cee40 msgr2=0x7f500806d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.134+0000 7f500d400640 1 --2- 172.21.15.62:0/3865875097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50080cee40 0x7f500806d1f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40097a0 tx=0x7f4ff402a100 comp rx=0 tx=0).stop 2024-04-25T09:07:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.134+0000 7f500d400640 1 -- 172.21.15.62:0/3865875097 shutdown_connections 2024-04-25T09:07:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.134+0000 7f500d400640 1 --2- 172.21.15.62:0/3865875097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50080cee40 0x7f500806d1f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.134+0000 7f500d400640 1 --2- 172.21.15.62:0/3865875097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5008144c10 0x7f500815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.134+0000 7f500d400640 1 -- 172.21.15.62:0/3865875097 >> 172.21.15.62:0/3865875097 conn(0x7f50080b14d0 msgr2=0x7f50080b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:17.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.134+0000 7f500d400640 1 -- 172.21.15.62:0/3865875097 shutdown_connections 2024-04-25T09:07:17.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.134+0000 7f500d400640 1 -- 172.21.15.62:0/3865875097 wait complete. 2024-04-25T09:07:17.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.136+0000 7f500d400640 1 Processor -- start 2024-04-25T09:07:17.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.136+0000 7f500d400640 1 -- start start 2024-04-25T09:07:17.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500d400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f500806d730 0x7f50081f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50080cee40 0x7f50081f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5008144c10 0x7f50080c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50080cf2a0 con 0x7f5008144c10 2024-04-25T09:07:17.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500d400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50080b41c0 con 0x7f50080cee40 2024-04-25T09:07:17.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500d400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50080b5660 con 0x7f500806d730 2024-04-25T09:07:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5008144c10 0x7f50080c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5008144c10 0x7f50080c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49860/0 (socket says 172.21.15.62:49860) 2024-04-25T09:07:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.137+0000 7f500ca00640 1 -- 172.21.15.62:0/1607932912 learned_addr learned my addr 172.21.15.62:0/1607932912 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f5007e00640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f500806d730 0x7f50081f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f5007400640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50080cee40 0x7f50081f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f500ca00640 1 -- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f500806d730 msgr2=0x7f50081f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f500ca00640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f500806d730 0x7f50081f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f500ca00640 1 -- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50080cee40 msgr2=0x7f50081f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f500ca00640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50080cee40 0x7f50081f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f500ca00640 1 -- 172.21.15.62:0/1607932912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50081425d0 con 0x7f5008144c10 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f5007400640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50080cee40 0x7f50081f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f500ca00640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5008144c10 0x7f50080c6ae0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f500000c7a0 tx=0x7f500000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f4fffe00640 1 -- 172.21.15.62:0/1607932912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000018070 con 0x7f5008144c10 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f4fffe00640 1 -- 172.21.15.62:0/1607932912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500000a640 con 0x7f5008144c10 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f4fffe00640 1 -- 172.21.15.62:0/1607932912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500000aab0 con 0x7f5008144c10 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.138+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50081428b0 con 0x7f5008144c10 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.139+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5008142e70 con 0x7f5008144c10 2024-04-25T09:07:17.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.140+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500804ef20 con 0x7f5008144c10 2024-04-25T09:07:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.141+0000 7f4fffe00640 1 -- 172.21.15.62:0/1607932912 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f500001b070 con 0x7f5008144c10 2024-04-25T09:07:17.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.141+0000 7f4fffe00640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4fe806b520 0x7f4fe806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.142+0000 7f4fffe00640 1 -- 172.21.15.62:0/1607932912 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5000089660 con 0x7f5008144c10 2024-04-25T09:07:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.142+0000 7f5007e00640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4fe806b520 0x7f4fe806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.143+0000 7f5007e00640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4fe806b520 0x7f4fe806d9e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8006fd0 tx=0x7f4ff8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:17.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.146+0000 7f4fffe00640 1 -- 172.21.15.62:0/1607932912 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5000058850 con 0x7f5008144c10 2024-04-25T09:07:17.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.350+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50080b3770 con 0x7f5008144c10 2024-04-25T09:07:17.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.352+0000 7f4fffe00640 1 -- 172.21.15.62:0/1607932912 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13006 (secure 0 0 0) 0x7f5000058150 con 0x7f5008144c10 2024-04-25T09:07:17.341 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:17.341 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":40,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:07:14.869450+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/274650142":"2024-04-26T09:07:11.848802+0000","172.21.15.62:6833/274650142":"2024-04-26T09:07:11.848802+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4fe806b520 msgr2=0x7f4fe806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4fe806b520 0x7f4fe806d9e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8006fd0 tx=0x7f4ff8008040 comp rx=0 tx=0).stop 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5008144c10 msgr2=0x7f50080c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5008144c10 0x7f50080c6ae0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f500000c7a0 tx=0x7f500000cc60 comp rx=0 tx=0).stop 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 shutdown_connections 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4fe806b520 0x7f4fe806d9e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5008144c10 0x7f50080c6ae0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50080cee40 0x7f50081f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 --2- 172.21.15.62:0/1607932912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f500806d730 0x7f50081f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 >> 172.21.15.62:0/1607932912 conn(0x7f50080b14d0 msgr2=0x7f5008070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 shutdown_connections 2024-04-25T09:07:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.355+0000 7f500d400640 1 -- 172.21.15.62:0/1607932912 wait complete. 2024-04-25T09:07:17.364 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:07:17.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:07:17.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.592+0000 7f5a5ea00640 1 Processor -- start 2024-04-25T09:07:17.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.593+0000 7f5a5ea00640 1 -- start start 2024-04-25T09:07:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.594+0000 7f5a5ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a5815db80 0x7f5a580b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.594+0000 7f5a5ea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a58161810 0x7f5a580ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.594+0000 7f5a5ea00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a58161f80 con 0x7f5a580ced30 2024-04-25T09:07:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.594+0000 7f5a5ea00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a580d1db0 con 0x7f5a58161810 2024-04-25T09:07:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.594+0000 7f5a5ea00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a58162260 con 0x7f5a5815db80 2024-04-25T09:07:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a4fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a58161810 0x7f5a580ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a4fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a58161810 0x7f5a580ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49872/0 (socket says 172.21.15.62:49872) 2024-04-25T09:07:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a4fe00640 1 -- 172.21.15.62:0/2991282556 learned_addr learned my addr 172.21.15.62:0/2991282556 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a57e00640 1 --2- 172.21.15.62:0/2991282556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a5815db80 0x7f5a580b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a4fe00640 1 -- 172.21.15.62:0/2991282556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a5815db80 msgr2=0x7f5a580b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a4fe00640 1 --2- 172.21.15.62:0/2991282556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a5815db80 0x7f5a580b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a4fe00640 1 -- 172.21.15.62:0/2991282556 >> v1:172.21.15.62:6789/0 conn(0x7f5a580ced30 legacy=0x7f5a580d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a4fe00640 1 -- 172.21.15.62:0/2991282556 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a58162540 con 0x7f5a58161810 2024-04-25T09:07:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.595+0000 7f5a57400640 1 -- 172.21.15.62:0/2991282556 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637626102 0 0) 0x7f5a58161f80 con 0x7f5a580ced30 2024-04-25T09:07:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a4fe00640 1 --2- 172.21.15.62:0/2991282556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580ce7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009870 tx=0x7f5a4002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9607601ba4523fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:17.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a57400640 1 -- 172.21.15.62:0/2991282556 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40037070 con 0x7f5a58161810 2024-04-25T09:07:17.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a57400640 1 -- 172.21.15.62:0/2991282556 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a40032440 con 0x7f5a58161810 2024-04-25T09:07:17.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a57400640 1 -- 172.21.15.62:0/2991282556 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40032810 con 0x7f5a58161810 2024-04-25T09:07:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2991282556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 msgr2=0x7f5a580ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2991282556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580ce7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009870 tx=0x7f5a4002a250 comp rx=0 tx=0).stop 2024-04-25T09:07:17.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2991282556 shutdown_connections 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2991282556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580ce7f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2991282556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a5815db80 0x7f5a580b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.597+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2991282556 >> 172.21.15.62:0/2991282556 conn(0x7f5a580668f0 msgr2=0x7f5a58066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.598+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2991282556 shutdown_connections 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.598+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2991282556 wait complete. 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.599+0000 7f5a5ea00640 1 Processor -- start 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.599+0000 7f5a5ea00640 1 -- start start 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.600+0000 7f5a5ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a580ced30 0x7f5a580c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.600+0000 7f5a5ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a5815db80 0x7f5a581499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.600+0000 7f5a5ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.600+0000 7f5a5ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5815e0e0 con 0x7f5a58161810 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.600+0000 7f5a5ea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5815df60 con 0x7f5a5815db80 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.600+0000 7f5a5ea00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580d3250 con 0x7f5a580ced30 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a5ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a5ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580b8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49886/0 (socket says 172.21.15.62:49886) 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a5ca00640 1 -- 172.21.15.62:0/2509726979 learned_addr learned my addr 172.21.15.62:0/2509726979 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a57e00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a580ced30 0x7f5a580c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a4fe00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a5815db80 0x7f5a581499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a5ca00640 1 -- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a580ced30 msgr2=0x7f5a580c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a5ca00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a580ced30 0x7f5a580c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a5ca00640 1 -- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a5815db80 msgr2=0x7f5a581499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a5ca00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a5815db80 0x7f5a581499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a5ca00640 1 -- 172.21.15.62:0/2509726979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a580cac90 con 0x7f5a58161810 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.601+0000 7f5a57e00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a580ced30 0x7f5a580c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.602+0000 7f5a5ca00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580b8760 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000c760 tx=0x7f5a5000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.602+0000 7f5a54c00640 1 -- 172.21.15.62:0/2509726979 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50003da0 con 0x7f5a58161810 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.602+0000 7f5a54c00640 1 -- 172.21.15.62:0/2509726979 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a50005070 con 0x7f5a58161810 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.602+0000 7f5a54c00640 1 -- 172.21.15.62:0/2509726979 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50005460 con 0x7f5a58161810 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.602+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a580caf70 con 0x7f5a58161810 2024-04-25T09:07:17.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.602+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a580cb530 con 0x7f5a58161810 2024-04-25T09:07:17.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.603+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5804ef20 con 0x7f5a58161810 2024-04-25T09:07:17.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.604+0000 7f5a54c00640 1 -- 172.21.15.62:0/2509726979 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a50005600 con 0x7f5a58161810 2024-04-25T09:07:17.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.605+0000 7f5a54c00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5a3406b520 0x7f5a3406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:17.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.605+0000 7f5a54c00640 1 -- 172.21.15.62:0/2509726979 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a50089e10 con 0x7f5a58161810 2024-04-25T09:07:17.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.606+0000 7f5a57e00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5a3406b520 0x7f5a3406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:17.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.607+0000 7f5a57e00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5a3406b520 0x7f5a3406d9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a48006fd0 tx=0x7f5a48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:17.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.610+0000 7f5a54c00640 1 -- 172.21.15.62:0/2509726979 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a5008d030 con 0x7f5a58161810 2024-04-25T09:07:17.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.810+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a58159ce0 con 0x7f5a58161810 2024-04-25T09:07:17.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.811+0000 7f5a54c00640 1 -- 172.21.15.62:0/2509726979 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a50059000 con 0x7f5a58161810 2024-04-25T09:07:17.800 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5a3406b520 msgr2=0x7f5a3406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5a3406b520 0x7f5a3406d9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5a48006fd0 tx=0x7f5a48008040 comp rx=0 tx=0).stop 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 msgr2=0x7f5a580b8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580b8760 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000c760 tx=0x7f5a5000cc20 comp rx=0 tx=0).stop 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 shutdown_connections 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5a3406b520 0x7f5a3406d9e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a58161810 0x7f5a580b8760 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a5815db80 0x7f5a581499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 --2- 172.21.15.62:0/2509726979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a580ced30 0x7f5a580c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 >> 172.21.15.62:0/2509726979 conn(0x7f5a580668f0 msgr2=0x7f5a580d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 shutdown_connections 2024-04-25T09:07:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:17.814+0000 7f5a5ea00640 1 -- 172.21.15.62:0/2509726979 wait complete. 2024-04-25T09:07:17.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T09:07:18.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.044+0000 7fc6cac00640 1 Processor -- start 2024-04-25T09:07:18.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.045+0000 7fc6cac00640 1 -- start start 2024-04-25T09:07:18.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.047+0000 7fc6cac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c4144c10 0x7fc6c415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.047+0000 7fc6cac00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc6c406d900 0x7fc6c40b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.047+0000 7fc6cac00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c415faa0 con 0x7fc6c40cee40 2024-04-25T09:07:18.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.047+0000 7fc6cac00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c4144ff0 con 0x7fc6c406d900 2024-04-25T09:07:18.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.047+0000 7fc6cac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4160340 con 0x7fc6c4144c10 2024-04-25T09:07:18.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.047+0000 7fc6c8e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc6c40cee40 0x7fc6c406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36894/0 (socket says 172.21.15.62:36894) 2024-04-25T09:07:18.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.047+0000 7fc6c8e00640 1 -- 172.21.15.62:0/1852342096 learned_addr learned my addr 172.21.15.62:0/1852342096 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:18.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.047+0000 7fc6c9800640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c4144c10 0x7fc6c415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6ca200640 1 --2- 172.21.15.62:0/1852342096 >> v2:172.21.15.121:3300/0 conn(0x7fc6c406d900 0x7fc6c40b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304509782 0 0) 0x7fc6c415faa0 con 0x7fc6c40cee40 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6ac003520 con 0x7fc6c40cee40 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 404827102 0 0) 0x7fc6ac003520 con 0x7fc6c40cee40 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c4144c10 msgr2=0x7fc6c415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c4144c10 0x7fc6c415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 >> v2:172.21.15.121:3300/0 conn(0x7fc6c406d900 msgr2=0x7fc6c40b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 --2- 172.21.15.62:0/1852342096 >> v2:172.21.15.121:3300/0 conn(0x7fc6c406d900 0x7fc6c40b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c415fd80 con 0x7fc6c40cee40 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc6b0001670 con 0x7fc6c40cee40 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 >> v1:172.21.15.62:6789/0 conn(0x7fc6c40cee40 legacy=0x7fc6c406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6ac0039b0 0x7fc6ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c4144c10 0x7fc6ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6ac004d40 0x7fc6ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4144ff0 con 0x7fc6ac0039b0 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4160340 con 0x7fc6ac004d40 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.048+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac00b9f0 con 0x7fc6c4144c10 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.049+0000 7fc6c9800640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6ac0039b0 0x7fc6ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.049+0000 7fc6c8e00640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c4144c10 0x7fc6ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.049+0000 7fc6c9800640 1 -- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c4144c10 msgr2=0x7fc6ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.049+0000 7fc6c9800640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c4144c10 0x7fc6ac004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.049+0000 7fc6ca200640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6ac004d40 0x7fc6ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.049+0000 7fc6c9800640 1 -- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6ac004d40 msgr2=0x7fc6ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.049+0000 7fc6c9800640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6ac004d40 0x7fc6ac0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.049+0000 7fc6c9800640 1 -- 172.21.15.62:0/1852342096 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac003520 con 0x7fc6ac0039b0 2024-04-25T09:07:18.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.050+0000 7fc6c9800640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6ac0039b0 0x7fc6ac003dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009fc0 tx=0x7fc6ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93e8d1dee4ee60f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.050+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8009c10 con 0x7fc6ac0039b0 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.050+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b8036070 con 0x7fc6ac0039b0 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.050+0000 7fc6c3e00640 1 -- 172.21.15.62:0/1852342096 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8031880 con 0x7fc6ac0039b0 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.051+0000 7fc6cac00640 1 -- 172.21.15.62:0/1852342096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6ac0039b0 msgr2=0x7fc6ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.051+0000 7fc6cac00640 1 --2- 172.21.15.62:0/1852342096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6ac0039b0 0x7fc6ac003dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009fc0 tx=0x7fc6ac005830 comp rx=0 tx=0).stop 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.051+0000 7fc6ca200640 1 -- 172.21.15.62:0/1852342096 reap_dead start 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.051+0000 7fc6cac00640 1 -- 172.21.15.62:0/1852342096 shutdown_connections 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.051+0000 7fc6cac00640 1 -- 172.21.15.62:0/1852342096 >> 172.21.15.62:0/1852342096 conn(0x7fc6c40b14d0 msgr2=0x7fc6c40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.051+0000 7fc6cac00640 1 -- 172.21.15.62:0/1852342096 shutdown_connections 2024-04-25T09:07:18.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.051+0000 7fc6cac00640 1 -- 172.21.15.62:0/1852342096 wait complete. 2024-04-25T09:07:18.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.053+0000 7fc6cac00640 1 Processor -- start 2024-04-25T09:07:18.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.053+0000 7fc6cac00640 1 -- start start 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.054+0000 7fc6cac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6ac0039b0 0x7fc6c415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.054+0000 7fc6cac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c41e5520 0x7fc6c4133d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.054+0000 7fc6cac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c40caf10 0x7fc6c40cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.054+0000 7fc6cac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40cf2a0 con 0x7fc6c40caf10 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.054+0000 7fc6cac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40b41c0 con 0x7fc6c41e5520 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.054+0000 7fc6cac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c415f450 con 0x7fc6ac0039b0 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6ca200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c40caf10 0x7fc6c40cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6ca200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c40caf10 0x7fc6c40cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49906/0 (socket says 172.21.15.62:49906) 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6ca200640 1 -- 172.21.15.62:0/3763019556 learned_addr learned my addr 172.21.15.62:0/3763019556 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6c8e00640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c41e5520 0x7fc6c4133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6c9800640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6ac0039b0 0x7fc6c415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6ca200640 1 -- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6ac0039b0 msgr2=0x7fc6c415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:18.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6ca200640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6ac0039b0 0x7fc6c415b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6ca200640 1 -- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c41e5520 msgr2=0x7fc6c4133d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6ca200640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c41e5520 0x7fc6c4133d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6ca200640 1 -- 172.21.15.62:0/3763019556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c415b640 con 0x7fc6c40caf10 2024-04-25T09:07:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.055+0000 7fc6c8e00640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c41e5520 0x7fc6c4133d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:18.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.056+0000 7fc6ca200640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c40caf10 0x7fc6c40cb390 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40b4130 tx=0x7fc6bc010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.056+0000 7fc6c1600640 1 -- 172.21.15.62:0/3763019556 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00f040 con 0x7fc6c40caf10 2024-04-25T09:07:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.056+0000 7fc6c1600640 1 -- 172.21.15.62:0/3763019556 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc01c070 con 0x7fc6c40caf10 2024-04-25T09:07:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.056+0000 7fc6c1600640 1 -- 172.21.15.62:0/3763019556 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00eb00 con 0x7fc6c40caf10 2024-04-25T09:07:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.056+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c4142ba0 con 0x7fc6c40caf10 2024-04-25T09:07:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.056+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c4134910 con 0x7fc6c40caf10 2024-04-25T09:07:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.057+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c404ef20 con 0x7fc6c40caf10 2024-04-25T09:07:18.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.058+0000 7fc6c1600640 1 -- 172.21.15.62:0/3763019556 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6bc00eca0 con 0x7fc6c40caf10 2024-04-25T09:07:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.059+0000 7fc6c1600640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a406b500 0x7fc6a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:18.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.059+0000 7fc6c1600640 1 -- 172.21.15.62:0/3763019556 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc6bc0896f0 con 0x7fc6c40caf10 2024-04-25T09:07:18.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.060+0000 7fc6c9800640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a406b500 0x7fc6a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:18.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.061+0000 7fc6c9800640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a406b500 0x7fc6a406d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8035000 tx=0x7fc6b802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:18.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.063+0000 7fc6c1600640 1 -- 172.21.15.62:0/3763019556 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6bc025020 con 0x7fc6c40caf10 2024-04-25T09:07:18.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.334+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7fc6c40b22b0 con 0x7fc6c40caf10 2024-04-25T09:07:18.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.903+0000 7fc6c1600640 1 -- 172.21.15.62:0/3763019556 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v189) v1 ==== 146+0+0 (secure 0 0 0) 0x7fc6c415b640 con 0x7fc6c40caf10 2024-04-25T09:07:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a406b500 msgr2=0x7fc6a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:18.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a406b500 0x7fc6a406d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8035000 tx=0x7fc6b802f000 comp rx=0 tx=0).stop 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c40caf10 msgr2=0x7fc6c40cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c40caf10 0x7fc6c40cb390 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40b4130 tx=0x7fc6bc010c20 comp rx=0 tx=0).stop 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 shutdown_connections 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a406b500 0x7fc6a406d9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c40caf10 0x7fc6c40cb390 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6c41e5520 0x7fc6c4133d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 --2- 172.21.15.62:0/3763019556 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6ac0039b0 0x7fc6c415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 >> 172.21.15.62:0/3763019556 conn(0x7fc6c40b14d0 msgr2=0x7fc6c4071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 shutdown_connections 2024-04-25T09:07:18.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:18.907+0000 7fc6cac00640 1 -- 172.21.15.62:0/3763019556 wait complete. 2024-04-25T09:07:18.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:07:19.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.149+0000 7f8f8ee00640 1 Processor -- start 2024-04-25T09:07:19.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.149+0000 7f8f8ee00640 1 -- start start 2024-04-25T09:07:19.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.150+0000 7f8f8ee00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f8806d2c0 0x7f8f880ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:19.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.150+0000 7f8f8ee00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8f8806d6a0 0x7f8f880ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:19.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8ee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f880ced40 con 0x7f8f880b31a0 2024-04-25T09:07:19.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8ee00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f880cf000 con 0x7f8f8806d2c0 2024-04-25T09:07:19.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8ee00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f880cf2e0 con 0x7f8f8806d6a0 2024-04-25T09:07:19.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f8806d2c0 0x7f8f880ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:19.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f8806d2c0 0x7f8f880ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49910/0 (socket says 172.21.15.62:49910) 2024-04-25T09:07:19.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8da00640 1 -- 172.21.15.62:0/2709144104 learned_addr learned my addr 172.21.15.62:0/2709144104 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:19.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8d000640 1 --2- 172.21.15.62:0/2709144104 >> v2:172.21.15.121:3300/0 conn(0x7f8f8806d6a0 0x7f8f880ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:19.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8da00640 1 -- 172.21.15.62:0/2709144104 >> v2:172.21.15.121:3300/0 conn(0x7f8f8806d6a0 msgr2=0x7f8f880ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:19.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8da00640 1 --2- 172.21.15.62:0/2709144104 >> v2:172.21.15.121:3300/0 conn(0x7f8f8806d6a0 0x7f8f880ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8da00640 1 -- 172.21.15.62:0/2709144104 >> v1:172.21.15.121:6789/0 conn(0x7f8f880b31a0 legacy=0x7f8f880b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:19.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8da00640 1 -- 172.21.15.62:0/2709144104 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f880b26c0 con 0x7f8f8806d2c0 2024-04-25T09:07:19.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.151+0000 7f8f8d000640 1 --2- 172.21.15.62:0/2709144104 >> v2:172.21.15.121:3300/0 conn(0x7f8f8806d6a0 0x7f8f880ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:19.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.152+0000 7f8f8da00640 1 --2- 172.21.15.62:0/2709144104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d2c0 0x7f8f880ccde0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f880722f0 tx=0x7f8f7c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d8f8a4f5f612e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:19.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.153+0000 7f8f87e00640 1 -- 172.21.15.62:0/2709144104 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c037070 con 0x7f8f8806d2c0 2024-04-25T09:07:19.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.153+0000 7f8f87e00640 1 -- 172.21.15.62:0/2709144104 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c032440 con 0x7f8f8806d2c0 2024-04-25T09:07:19.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.153+0000 7f8f87e00640 1 -- 172.21.15.62:0/2709144104 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c032810 con 0x7f8f8806d2c0 2024-04-25T09:07:19.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.154+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2709144104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d2c0 msgr2=0x7f8f880ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:19.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.154+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2709144104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d2c0 0x7f8f880ccde0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f880722f0 tx=0x7f8f7c02a230 comp rx=0 tx=0).stop 2024-04-25T09:07:19.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.154+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2709144104 shutdown_connections 2024-04-25T09:07:19.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.154+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2709144104 >> v2:172.21.15.121:3300/0 conn(0x7f8f8806d6a0 0x7f8f880ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.154+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2709144104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d2c0 0x7f8f880ccde0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.154+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2709144104 >> 172.21.15.62:0/2709144104 conn(0x7f8f88066f20 msgr2=0x7f8f880b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:19.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.154+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2709144104 shutdown_connections 2024-04-25T09:07:19.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.154+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2709144104 wait complete. 2024-04-25T09:07:19.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.156+0000 7f8f8ee00640 1 Processor -- start 2024-04-25T09:07:19.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.156+0000 7f8f8ee00640 1 -- start start 2024-04-25T09:07:19.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8ee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f8806d2c0 0x7f8f881e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:19.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d6a0 0x7f8f880ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:19.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f880b31a0 0x7f8f88135d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8806db20 con 0x7f8f8806d6a0 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f880b2c80 con 0x7f8f880b31a0 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8ee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f881448d0 con 0x7f8f8806d2c0 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d6a0 0x7f8f880ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d6a0 0x7f8f880ab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49916/0 (socket says 172.21.15.62:49916) 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8d000640 1 -- 172.21.15.62:0/2666347428 learned_addr learned my addr 172.21.15.62:0/2666347428 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8da00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f8806d2c0 0x7f8f881e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8d000640 1 -- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f8806d2c0 msgr2=0x7f8f881e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8d000640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f8806d2c0 0x7f8f881e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8d000640 1 -- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f880b31a0 msgr2=0x7f8f88135d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8d000640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f880b31a0 0x7f8f88135d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.157+0000 7f8f8d000640 1 -- 172.21.15.62:0/2666347428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f88136280 con 0x7f8f8806d6a0 2024-04-25T09:07:19.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.158+0000 7f8f8d000640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d6a0 0x7f8f880ab120 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c7f0 tx=0x7f8f7400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:19.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.158+0000 7f8f85600640 1 -- 172.21.15.62:0/2666347428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74015070 con 0x7f8f8806d6a0 2024-04-25T09:07:19.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.158+0000 7f8f85600640 1 -- 172.21.15.62:0/2666347428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f74007a30 con 0x7f8f8806d6a0 2024-04-25T09:07:19.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.158+0000 7f8f85600640 1 -- 172.21.15.62:0/2666347428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74007e00 con 0x7f8f8806d6a0 2024-04-25T09:07:19.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.158+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f880ab8a0 con 0x7f8f8806d6a0 2024-04-25T09:07:19.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.158+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f880abe60 con 0x7f8f8806d6a0 2024-04-25T09:07:19.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.159+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8806abe0 con 0x7f8f8806d6a0 2024-04-25T09:07:19.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.160+0000 7f8f85600640 1 -- 172.21.15.62:0/2666347428 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f74018070 con 0x7f8f8806d6a0 2024-04-25T09:07:19.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.161+0000 7f8f85600640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f6806b500 0x7f8f6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:19.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.161+0000 7f8f85600640 1 -- 172.21.15.62:0/2666347428 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8f74089eb0 con 0x7f8f8806d6a0 2024-04-25T09:07:19.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.162+0000 7f8f8da00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f6806b500 0x7f8f6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:19.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.163+0000 7f8f8da00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f6806b500 0x7f8f6806d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c034000 tx=0x7f8f7c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:19.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.166+0000 7f8f85600640 1 -- 172.21.15.62:0/2666347428 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f74058f90 con 0x7f8f8806d6a0 2024-04-25T09:07:19.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.371+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8f8804ef20 con 0x7f8f8806d6a0 2024-04-25T09:07:19.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.379+0000 7f8f85600640 1 -- 172.21.15.62:0/2666347428 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8f74058890 con 0x7f8f8806d6a0 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f6806b500 msgr2=0x7f8f6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f6806b500 0x7f8f6806d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c034000 tx=0x7f8f7c02e000 comp rx=0 tx=0).stop 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d6a0 msgr2=0x7f8f880ab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d6a0 0x7f8f880ab120 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400c7f0 tx=0x7f8f7400cbf0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 shutdown_connections 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f6806b500 0x7f8f6806d9c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f8806d6a0 0x7f8f880ab120 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f880b31a0 0x7f8f88135d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 --2- 172.21.15.62:0/2666347428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f8806d2c0 0x7f8f881e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 >> 172.21.15.62:0/2666347428 conn(0x7f8f88066f20 msgr2=0x7f8f8815bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:19.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 shutdown_connections 2024-04-25T09:07:19.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:19.382+0000 7f8f8ee00640 1 -- 172.21.15.62:0/2666347428 wait complete. 2024-04-25T09:07:19.391 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:07:19.391 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:07:19.497 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:07:19.610 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:07:19.610 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:07:19.610 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:19.670 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:07:19.670 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:07:19.670 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:19.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:07:19.727 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:07:19.727 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:19.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:07:19.787 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:07:19.787 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:19.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:07:19.844 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:07:19.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.065+0000 7f3179200640 1 Processor -- start 2024-04-25T09:07:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.065+0000 7f3179200640 1 -- start start 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3179200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3174161f00 0x7f31741622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3179200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f31740cc9e0 0x7f31740cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3179200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f317406d1f0 0x7f317406d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3179200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31740b27d0 con 0x7f31740cc9e0 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3179200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31740b2a20 con 0x7f317406d1f0 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3179200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31740b2d00 con 0x7f3174161f00 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3172a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f31740cc9e0 0x7f31740cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3172a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f31740cc9e0 0x7f31740cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49918/0 (socket says 172.21.15.62:49918) 2024-04-25T09:07:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.067+0000 7f3172a00640 1 -- 172.21.15.62:0/765789065 learned_addr learned my addr 172.21.15.62:0/765789065 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.068+0000 7f3172a00640 1 -- 172.21.15.62:0/765789065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3174161f00 msgr2=0x7f31741622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.068+0000 7f3172a00640 1 --2- 172.21.15.62:0/765789065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3174161f00 0x7f31741622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.068+0000 7f3172a00640 1 -- 172.21.15.62:0/765789065 >> v2:172.21.15.121:3300/0 conn(0x7f317406d1f0 msgr2=0x7f317406d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.068+0000 7f3173e00640 1 --2- 172.21.15.62:0/765789065 >> v2:172.21.15.121:3300/0 conn(0x7f317406d1f0 0x7f317406d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.068+0000 7f3172a00640 1 --2- 172.21.15.62:0/765789065 >> v2:172.21.15.121:3300/0 conn(0x7f317406d1f0 0x7f317406d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.068+0000 7f3172a00640 1 -- 172.21.15.62:0/765789065 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31740b2fe0 con 0x7f31740cc9e0 2024-04-25T09:07:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.068+0000 7f3173e00640 1 --2- 172.21.15.62:0/765789065 >> v2:172.21.15.121:3300/0 conn(0x7f317406d1f0 0x7f317406d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.069+0000 7f3172a00640 1 --2- 172.21.15.62:0/765789065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31740cc9e0 0x7f31740cce40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3174070100 tx=0x7f315c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecea8e2482c8011 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.069+0000 7f3172000640 1 -- 172.21.15.62:0/765789065 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c037070 con 0x7f31740cc9e0 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3172000640 1 -- 172.21.15.62:0/765789065 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c032440 con 0x7f31740cc9e0 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3172000640 1 -- 172.21.15.62:0/765789065 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c032810 con 0x7f31740cc9e0 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3179200640 1 -- 172.21.15.62:0/765789065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31740cc9e0 msgr2=0x7f31740cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3179200640 1 --2- 172.21.15.62:0/765789065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31740cc9e0 0x7f31740cce40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3174070100 tx=0x7f315c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3179200640 1 -- 172.21.15.62:0/765789065 shutdown_connections 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3179200640 1 --2- 172.21.15.62:0/765789065 >> v2:172.21.15.121:3300/0 conn(0x7f317406d1f0 0x7f317406d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3179200640 1 --2- 172.21.15.62:0/765789065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31740cc9e0 0x7f31740cce40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3179200640 1 --2- 172.21.15.62:0/765789065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3174161f00 0x7f31741622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3179200640 1 -- 172.21.15.62:0/765789065 >> 172.21.15.62:0/765789065 conn(0x7f31740b14d0 msgr2=0x7f31740b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.070+0000 7f3179200640 1 -- 172.21.15.62:0/765789065 shutdown_connections 2024-04-25T09:07:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.071+0000 7f3179200640 1 -- 172.21.15.62:0/765789065 wait complete. 2024-04-25T09:07:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.072+0000 7f3179200640 1 Processor -- start 2024-04-25T09:07:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.072+0000 7f3179200640 1 -- start start 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.073+0000 7f3179200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f317406d1f0 0x7f31741631d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.073+0000 7f3179200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31740cc9e0 0x7f31741594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3179200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174161f00 0x7f317414a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3179200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31740b3ac0 con 0x7f3174161f00 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3179200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31740b35a0 con 0x7f317406d1f0 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3179200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31740b3c40 con 0x7f31740cc9e0 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174161f00 0x7f317414a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174161f00 0x7f317414a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49926/0 (socket says 172.21.15.62:49926) 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173e00640 1 -- 172.21.15.62:0/3736747132 learned_addr learned my addr 172.21.15.62:0/3736747132 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173400640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f317406d1f0 0x7f31741631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173e00640 1 -- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31740cc9e0 msgr2=0x7f31741594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173e00640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31740cc9e0 0x7f31741594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173e00640 1 -- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f317406d1f0 msgr2=0x7f31741631d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173e00640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f317406d1f0 0x7f31741631d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.074+0000 7f3173e00640 1 -- 172.21.15.62:0/3736747132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3174163b00 con 0x7f3174161f00 2024-04-25T09:07:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.075+0000 7f3173e00640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174161f00 0x7f317414a330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f316c00cf20 tx=0x7f316c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.075+0000 7f316b400640 1 -- 172.21.15.62:0/3736747132 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c01a070 con 0x7f3174161f00 2024-04-25T09:07:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.075+0000 7f316b400640 1 -- 172.21.15.62:0/3736747132 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c00d840 con 0x7f3174161f00 2024-04-25T09:07:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.075+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3174163e40 con 0x7f3174161f00 2024-04-25T09:07:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.075+0000 7f316b400640 1 -- 172.21.15.62:0/3736747132 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c00dc30 con 0x7f3174161f00 2024-04-25T09:07:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.075+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3174164680 con 0x7f3174161f00 2024-04-25T09:07:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.076+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f317404ef20 con 0x7f3174161f00 2024-04-25T09:07:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.077+0000 7f316b400640 1 -- 172.21.15.62:0/3736747132 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f316c01d070 con 0x7f3174161f00 2024-04-25T09:07:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.079+0000 7f316b400640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f315006b4d0 0x7f315006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.079+0000 7f316b400640 1 -- 172.21.15.62:0/3736747132 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f316c08a8f0 con 0x7f3174161f00 2024-04-25T09:07:20.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.079+0000 7f3173400640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f315006b4d0 0x7f315006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.081+0000 7f3173400640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f315006b4d0 0x7f315006d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f316400b810 tx=0x7f3164008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:20.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.083+0000 7f316b400640 1 -- 172.21.15.62:0/3736747132 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3174163e40 con 0x7f3174161f00 2024-04-25T09:07:20.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.323+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3174164dd0 con 0x7f3174161f00 2024-04-25T09:07:20.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.324+0000 7f316b400640 1 -- 172.21.15.62:0/3736747132 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f316c059940 con 0x7f3174161f00 2024-04-25T09:07:20.313 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:20.313 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/2179586213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2179586213},{"type":"v1","addr":"172.21.15.121:6836","nonce":2179586213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4798,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1425308084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1425308084},{"type":"v1","addr":"172.21.15.121:6837","nonce":1425308084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5037,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/231681595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":231681595},{"type":"v1","addr":"172.21.15.62:6834","nonce":231681595}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3181168188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3181168188},{"type":"v1","addr":"172.21.15.62:6835","nonce":3181168188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T09:07:20.313 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 19 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f315006b4d0 msgr2=0x7f315006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f315006b4d0 0x7f315006d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f316400b810 tx=0x7f3164008040 comp rx=0 tx=0).stop 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174161f00 msgr2=0x7f317414a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174161f00 0x7f317414a330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f316c00cf20 tx=0x7f316c00a7d0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 shutdown_connections 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f315006b4d0 0x7f315006d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3174161f00 0x7f317414a330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f31740cc9e0 0x7f31741594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 --2- 172.21.15.62:0/3736747132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f317406d1f0 0x7f31741631d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 >> 172.21.15.62:0/3736747132 conn(0x7f31740b14d0 msgr2=0x7f31740b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 shutdown_connections 2024-04-25T09:07:20.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.327+0000 7f3179200640 1 -- 172.21.15.62:0/3736747132 wait complete. 2024-04-25T09:07:20.335 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:07:20.336 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:07:20.336 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:07:20.336 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:07:20.336 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:07:20.336 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:07:20.336 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:07:20.337 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:07:20.337 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:07:20.337 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:07:20.337 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:07:20.337 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:07:20.337 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:07:20.338 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:07:20.338 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:07:20.338 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:07:20.353 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:07:20.354 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:07:20.356 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:07:20.356 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:07:20.359 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:07:20.360 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:07:20.362 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:07:20.362 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:07:20.403 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:07:20.403 DEBUG:teuthology.parallel:result is None 2024-04-25T09:07:20.408 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:07:20.408 DEBUG:teuthology.parallel:result is None 2024-04-25T09:07:20.409 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:07:20.410 DEBUG:teuthology.parallel:result is None 2024-04-25T09:07:20.414 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:07:20.414 DEBUG:teuthology.parallel:result is None 2024-04-25T09:07:20.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:07:20.559 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:07:20.559 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:07:20.566 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:07:20.570 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:07:20.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.610+0000 7f025b200640 1 Processor -- start 2024-04-25T09:07:20.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.610+0000 7f025b200640 1 -- start start 2024-04-25T09:07:20.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.611+0000 7f025b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f025c0b5c30 0x7f025c0b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.611+0000 7f025b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f025c0d0fb0 0x7f025c161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.611+0000 7f025b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c0cea70 con 0x7f025c161230 2024-04-25T09:07:20.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.611+0000 7f025b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c0d2040 con 0x7f025c0d0fb0 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.611+0000 7f025b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c0ced50 con 0x7f025c0b5c30 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f025a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f025c0d0fb0 0x7f025c161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f025a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f025c0d0fb0 0x7f025c161a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49948/0 (socket says 172.21.15.62:49948) 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f025a800640 1 -- 172.21.15.62:0/27797038 learned_addr learned my addr 172.21.15.62:0/27797038 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f0259400640 1 --2- 172.21.15.62:0/27797038 >> v2:172.21.15.121:3300/0 conn(0x7f025c0b5c30 0x7f025c0b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f025a800640 1 -- 172.21.15.62:0/27797038 >> v2:172.21.15.121:3300/0 conn(0x7f025c0b5c30 msgr2=0x7f025c0b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f025a800640 1 --2- 172.21.15.62:0/27797038 >> v2:172.21.15.121:3300/0 conn(0x7f025c0b5c30 0x7f025c0b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f025a800640 1 -- 172.21.15.62:0/27797038 >> v1:172.21.15.121:6789/0 conn(0x7f025c161230 legacy=0x7f025c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f025a800640 1 -- 172.21.15.62:0/27797038 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0cf030 con 0x7f025c0d0fb0 2024-04-25T09:07:20.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.612+0000 7f0259400640 1 --2- 172.21.15.62:0/27797038 >> v2:172.21.15.121:3300/0 conn(0x7f025c0b5c30 0x7f025c0b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:20.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.613+0000 7f025a800640 1 --2- 172.21.15.62:0/27797038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c0d0fb0 0x7f025c161a20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f025c0d4b80 tx=0x7f025402a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c61e3d9ac085e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:20.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.613+0000 7f0258a00640 1 -- 172.21.15.62:0/27797038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254036070 con 0x7f025c0d0fb0 2024-04-25T09:07:20.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.613+0000 7f0258a00640 1 -- 172.21.15.62:0/27797038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0254031600 con 0x7f025c0d0fb0 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.613+0000 7f0258a00640 1 -- 172.21.15.62:0/27797038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02540319d0 con 0x7f025c0d0fb0 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.614+0000 7f025b200640 1 -- 172.21.15.62:0/27797038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c0d0fb0 msgr2=0x7f025c161a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.614+0000 7f025b200640 1 --2- 172.21.15.62:0/27797038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c0d0fb0 0x7f025c161a20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f025c0d4b80 tx=0x7f025402a740 comp rx=0 tx=0).stop 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.614+0000 7f025b200640 1 -- 172.21.15.62:0/27797038 shutdown_connections 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.614+0000 7f025b200640 1 --2- 172.21.15.62:0/27797038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c0d0fb0 0x7f025c161a20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.614+0000 7f025b200640 1 --2- 172.21.15.62:0/27797038 >> v2:172.21.15.121:3300/0 conn(0x7f025c0b5c30 0x7f025c0b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.614+0000 7f025b200640 1 -- 172.21.15.62:0/27797038 >> 172.21.15.62:0/27797038 conn(0x7f025c066d10 msgr2=0x7f025c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.614+0000 7f025b200640 1 -- 172.21.15.62:0/27797038 shutdown_connections 2024-04-25T09:07:20.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.614+0000 7f025b200640 1 -- 172.21.15.62:0/27797038 wait complete. 2024-04-25T09:07:20.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.616+0000 7f025b200640 1 Processor -- start 2024-04-25T09:07:20.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.616+0000 7f025b200640 1 -- start start 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f025c0b5c30 0x7f025c0cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f025c0d0fb0 0x7f025c0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c161230 0x7f025c0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c161610 con 0x7f025c161230 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0ca8d0 con 0x7f025c0d0fb0 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c161790 con 0x7f025c0b5c30 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c161230 0x7f025c0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c161230 0x7f025c0c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49962/0 (socket says 172.21.15.62:49962) 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f025a800640 1 -- 172.21.15.62:0/211424631 learned_addr learned my addr 172.21.15.62:0/211424631 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f0259e00640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f025c0b5c30 0x7f025c0cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.617+0000 7f0259400640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f025c0d0fb0 0x7f025c0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f025a800640 1 -- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f025c0b5c30 msgr2=0x7f025c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f025a800640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f025c0b5c30 0x7f025c0cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f025a800640 1 -- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f025c0d0fb0 msgr2=0x7f025c0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f025a800640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f025c0d0fb0 0x7f025c0c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f025a800640 1 -- 172.21.15.62:0/211424631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0c8870 con 0x7f025c161230 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f0259400640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f025c0d0fb0 0x7f025c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f025a800640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c161230 0x7f025c0c8330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0254009f40 tx=0x7f0254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f0252000640 1 -- 172.21.15.62:0/211424631 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254036050 con 0x7f025c161230 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f0252000640 1 -- 172.21.15.62:0/211424631 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0254042070 con 0x7f025c161230 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f0252000640 1 -- 172.21.15.62:0/211424631 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254009510 con 0x7f025c161230 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.618+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c0c8b20 con 0x7f025c161230 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.619+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c146c30 con 0x7f025c161230 2024-04-25T09:07:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.620+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c04ef20 con 0x7f025c161230 2024-04-25T09:07:20.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.621+0000 7f0252000640 1 -- 172.21.15.62:0/211424631 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0254007bd0 con 0x7f025c161230 2024-04-25T09:07:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.622+0000 7f0252000640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f023c06b4d0 0x7f023c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.622+0000 7f0252000640 1 -- 172.21.15.62:0/211424631 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f02540aacf0 con 0x7f025c161230 2024-04-25T09:07:20.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.623+0000 7f0259e00640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f023c06b4d0 0x7f023c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.623+0000 7f0259e00640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f023c06b4d0 0x7f023c06d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f024c0016e0 tx=0x7f024c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.627+0000 7f0252000640 1 -- 172.21.15.62:0/211424631 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0254079e60 con 0x7f025c161230 2024-04-25T09:07:20.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.834+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f025c0b6200 con 0x7f025c161230 2024-04-25T09:07:20.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.926+0000 7f0252000640 1 -- 172.21.15.62:0/211424631 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0254079760 con 0x7f025c161230 2024-04-25T09:07:20.915 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:07:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f023c06b4d0 msgr2=0x7f023c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f023c06b4d0 0x7f023c06d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f024c0016e0 tx=0x7f024c009040 comp rx=0 tx=0).stop 2024-04-25T09:07:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c161230 msgr2=0x7f025c0c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c161230 0x7f025c0c8330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0254009f40 tx=0x7f0254004a40 comp rx=0 tx=0).stop 2024-04-25T09:07:20.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 shutdown_connections 2024-04-25T09:07:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f023c06b4d0 0x7f023c06d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f025c161230 0x7f025c0c8330 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f025c0d0fb0 0x7f025c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 --2- 172.21.15.62:0/211424631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f025c0b5c30 0x7f025c0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 >> 172.21.15.62:0/211424631 conn(0x7f025c066d10 msgr2=0x7f025c15f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 shutdown_connections 2024-04-25T09:07:20.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:20.929+0000 7f025b200640 1 -- 172.21.15.62:0/211424631 wait complete. 2024-04-25T09:07:20.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:07:21.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.152+0000 7fc1eaa00640 1 Processor -- start 2024-04-25T09:07:21.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.152+0000 7fc1eaa00640 1 -- start start 2024-04-25T09:07:21.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.154+0000 7fc1eaa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e415b960 0x7fc1e406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.154+0000 7fc1eaa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc1e40ae9d0 0x7fc1e40b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.154+0000 7fc1eaa00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e406d4f0 con 0x7fc1e40b4610 2024-04-25T09:07:21.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.154+0000 7fc1eaa00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e415bd40 con 0x7fc1e40ae9d0 2024-04-25T09:07:21.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.154+0000 7fc1eaa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e40b4a60 con 0x7fc1e415b960 2024-04-25T09:07:21.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.155+0000 7fc1e9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e415b960 0x7fc1e406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.155+0000 7fc1e9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e415b960 0x7fc1e406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:43438/0 (socket says 172.21.15.62:43438) 2024-04-25T09:07:21.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.155+0000 7fc1e9600640 1 -- 172.21.15.62:0/3578851790 learned_addr learned my addr 172.21.15.62:0/3578851790 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:21.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.155+0000 7fc1ea000640 1 --1- 172.21.15.62:0/3578851790 >> v1:172.21.15.121:6789/0 conn(0x7fc1e40b4610 0x7fc1e415f450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:38900/0 (socket says 172.21.15.62:38900) 2024-04-25T09:07:21.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.155+0000 7fc1e8c00640 1 --2- 172.21.15.62:0/3578851790 >> v2:172.21.15.121:3300/0 conn(0x7fc1e40ae9d0 0x7fc1e40b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.156+0000 7fc1e9600640 1 -- 172.21.15.62:0/3578851790 >> v2:172.21.15.121:3300/0 conn(0x7fc1e40ae9d0 msgr2=0x7fc1e40b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.156+0000 7fc1e9600640 1 --2- 172.21.15.62:0/3578851790 >> v2:172.21.15.121:3300/0 conn(0x7fc1e40ae9d0 0x7fc1e40b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.156+0000 7fc1e9600640 1 -- 172.21.15.62:0/3578851790 >> v1:172.21.15.121:6789/0 conn(0x7fc1e40b4610 legacy=0x7fc1e415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.156+0000 7fc1e9600640 1 -- 172.21.15.62:0/3578851790 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40b2500 con 0x7fc1e415b960 2024-04-25T09:07:21.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.156+0000 7fc1e3e00640 1 -- 172.21.15.62:0/3578851790 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563316680 0 0) 0x7fc1e406d4f0 con 0x7fc1e40b4610 2024-04-25T09:07:21.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.157+0000 7fc1e9600640 1 --2- 172.21.15.62:0/3578851790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e415b960 0x7fc1e406ca80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009870 tx=0x7fc1d802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232be8e0a2dcd9c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.157+0000 7fc1e3e00640 1 -- 172.21.15.62:0/3578851790 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8037070 con 0x7fc1e415b960 2024-04-25T09:07:21.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.157+0000 7fc1e3e00640 1 -- 172.21.15.62:0/3578851790 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d8032440 con 0x7fc1e415b960 2024-04-25T09:07:21.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.157+0000 7fc1e3e00640 1 -- 172.21.15.62:0/3578851790 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8032810 con 0x7fc1e415b960 2024-04-25T09:07:21.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.158+0000 7fc1eaa00640 1 -- 172.21.15.62:0/3578851790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e415b960 msgr2=0x7fc1e406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.158+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/3578851790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e415b960 0x7fc1e406ca80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009870 tx=0x7fc1d802a270 comp rx=0 tx=0).stop 2024-04-25T09:07:21.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.158+0000 7fc1eaa00640 1 -- 172.21.15.62:0/3578851790 shutdown_connections 2024-04-25T09:07:21.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.158+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/3578851790 >> v2:172.21.15.121:3300/0 conn(0x7fc1e40ae9d0 0x7fc1e40b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.158+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/3578851790 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e415b960 0x7fc1e406ca80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.158+0000 7fc1eaa00640 1 -- 172.21.15.62:0/3578851790 >> 172.21.15.62:0/3578851790 conn(0x7fc1e4074380 msgr2=0x7fc1e40668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.158+0000 7fc1eaa00640 1 -- 172.21.15.62:0/3578851790 shutdown_connections 2024-04-25T09:07:21.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.158+0000 7fc1eaa00640 1 -- 172.21.15.62:0/3578851790 wait complete. 2024-04-25T09:07:21.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.161+0000 7fc1eaa00640 1 Processor -- start 2024-04-25T09:07:21.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.161+0000 7fc1eaa00640 1 -- start start 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1eaa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e40ae9d0 0x7fc1e414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1e40b4610 0x7fc1e415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1eaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e415b960 0x7fc1e4159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1eaa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e40b2ac0 con 0x7fc1e40b4610 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1eaa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e40b3110 con 0x7fc1e415b960 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1eaa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e40b2da0 con 0x7fc1e40ae9d0 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1e8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1e40b4610 0x7fc1e415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1e8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1e40b4610 0x7fc1e415e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49974/0 (socket says 172.21.15.62:49974) 2024-04-25T09:07:21.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.162+0000 7fc1e8c00640 1 -- 172.21.15.62:0/2920600853 learned_addr learned my addr 172.21.15.62:0/2920600853 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e9600640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e40ae9d0 0x7fc1e414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1ea000640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e415b960 0x7fc1e4159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e8c00640 1 -- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e40ae9d0 msgr2=0x7fc1e414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e8c00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e40ae9d0 0x7fc1e414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e8c00640 1 -- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e415b960 msgr2=0x7fc1e4159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e8c00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e415b960 0x7fc1e4159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e8c00640 1 -- 172.21.15.62:0/2920600853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e415ea20 con 0x7fc1e40b4610 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e8c00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1e40b4610 0x7fc1e415e4e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000e9e0 tx=0x7fc1d000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e1600640 1 -- 172.21.15.62:0/2920600853 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0017070 con 0x7fc1e40b4610 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.163+0000 7fc1e1600640 1 -- 172.21.15.62:0/2920600853 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d000c9d0 con 0x7fc1e40b4610 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.164+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e415ecc0 con 0x7fc1e40b4610 2024-04-25T09:07:21.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.164+0000 7fc1e1600640 1 -- 172.21.15.62:0/2920600853 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d000cdc0 con 0x7fc1e40b4610 2024-04-25T09:07:21.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.164+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e4158d60 con 0x7fc1e40b4610 2024-04-25T09:07:21.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.165+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e40cc2f0 con 0x7fc1e40b4610 2024-04-25T09:07:21.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.166+0000 7fc1e1600640 1 -- 172.21.15.62:0/2920600853 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1d001a070 con 0x7fc1e40b4610 2024-04-25T09:07:21.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.167+0000 7fc1e1600640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1c406b4d0 0x7fc1c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.167+0000 7fc1e1600640 1 -- 172.21.15.62:0/2920600853 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1d008a1f0 con 0x7fc1e40b4610 2024-04-25T09:07:21.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.168+0000 7fc1e9600640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1c406b4d0 0x7fc1c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.169+0000 7fc1e9600640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1c406b4d0 0x7fc1c406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8034000 tx=0x7fc1d802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.172+0000 7fc1e1600640 1 -- 172.21.15.62:0/2920600853 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1e40cc2f0 con 0x7fc1e40b4610 2024-04-25T09:07:21.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.433+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc1e4075070 con 0x7fc1e40b4610 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.433+0000 7fc1e1600640 1 -- 172.21.15.62:0/2920600853 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc1d0059450 con 0x7fc1e40b4610 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:07:21.422 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.423 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:07:21.424 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:07:21.425 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:07:21.426 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.428 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.429 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-25T09:07:21.430 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:07:21.431 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:07:21.432 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:07:21.433 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:07:21.434 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.435 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:21.435 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:07:21.435 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1c406b4d0 msgr2=0x7fc1c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1c406b4d0 0x7fc1c406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8034000 tx=0x7fc1d802e000 comp rx=0 tx=0).stop 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1e40b4610 msgr2=0x7fc1e415e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1e40b4610 0x7fc1e415e4e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000e9e0 tx=0x7fc1d000eea0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 shutdown_connections 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1c406b4d0 0x7fc1c406d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1e40b4610 0x7fc1e415e4e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1e415b960 0x7fc1e4159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 --2- 172.21.15.62:0/2920600853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1e40ae9d0 0x7fc1e414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 >> 172.21.15.62:0/2920600853 conn(0x7fc1e4074380 msgr2=0x7fc1e40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 shutdown_connections 2024-04-25T09:07:21.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.436+0000 7fc1eaa00640 1 -- 172.21.15.62:0/2920600853 wait complete. 2024-04-25T09:07:21.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.660+0000 7fb174a00640 1 Processor -- start 2024-04-25T09:07:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.660+0000 7fb174a00640 1 -- start start 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb174a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb17006d900 0x7fb17015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb174a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17015faa0 con 0x7fb170144c10 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb174a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17015fd80 con 0x7fb1700cee40 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb174a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb170144ff0 con 0x7fb17006d900 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb167e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb17006d900 0x7fb17015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb167e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb17006d900 0x7fb17015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49984/0 (socket says 172.21.15.62:49984) 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb167e00640 1 -- 172.21.15.62:0/1677854108 learned_addr learned my addr 172.21.15.62:0/1677854108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb16f400640 1 --1- 172.21.15.62:0/1677854108 >> v1:172.21.15.62:6789/0 conn(0x7fb170144c10 0x7fb17006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36908/0 (socket says 172.21.15.62:36908) 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb167e00640 1 -- 172.21.15.62:0/1677854108 >> v1:172.21.15.121:6789/0 conn(0x7fb1700cee40 legacy=0x7fb1700b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.662+0000 7fb167e00640 1 -- 172.21.15.62:0/1677854108 >> v1:172.21.15.62:6789/0 conn(0x7fb170144c10 legacy=0x7fb17006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.663+0000 7fb167e00640 1 -- 172.21.15.62:0/1677854108 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170160060 con 0x7fb17006d900 2024-04-25T09:07:21.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.663+0000 7fb16ea00640 1 -- 172.21.15.62:0/1677854108 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230904826 0 0) 0x7fb17015faa0 con 0x7fb170144c10 2024-04-25T09:07:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.664+0000 7fb167e00640 1 --2- 172.21.15.62:0/1677854108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17015f450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb158009870 tx=0x7fb15802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb5522a2977ee20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.664+0000 7fb16ea00640 1 -- 172.21.15.62:0/1677854108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158037070 con 0x7fb17006d900 2024-04-25T09:07:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.664+0000 7fb16ea00640 1 -- 172.21.15.62:0/1677854108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb158032440 con 0x7fb17006d900 2024-04-25T09:07:21.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.664+0000 7fb16ea00640 1 -- 172.21.15.62:0/1677854108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158032810 con 0x7fb17006d900 2024-04-25T09:07:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.665+0000 7fb174a00640 1 -- 172.21.15.62:0/1677854108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 msgr2=0x7fb17015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.665+0000 7fb174a00640 1 --2- 172.21.15.62:0/1677854108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17015f450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb158009870 tx=0x7fb15802a210 comp rx=0 tx=0).stop 2024-04-25T09:07:21.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.665+0000 7fb174a00640 1 -- 172.21.15.62:0/1677854108 shutdown_connections 2024-04-25T09:07:21.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.665+0000 7fb174a00640 1 --2- 172.21.15.62:0/1677854108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17015f450 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.665+0000 7fb174a00640 1 -- 172.21.15.62:0/1677854108 >> 172.21.15.62:0/1677854108 conn(0x7fb1700b14d0 msgr2=0x7fb1700b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.665+0000 7fb174a00640 1 -- 172.21.15.62:0/1677854108 shutdown_connections 2024-04-25T09:07:21.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.665+0000 7fb174a00640 1 -- 172.21.15.62:0/1677854108 wait complete. 2024-04-25T09:07:21.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.667+0000 7fb174a00640 1 Processor -- start 2024-04-25T09:07:21.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.667+0000 7fb174a00640 1 -- start start 2024-04-25T09:07:21.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb174a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17013a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb174a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1700cee40 0x7fb170139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb174a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb170144c10 0x7fb17015e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb174a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700b41c0 con 0x7fb17006d900 2024-04-25T09:07:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb174a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170160340 con 0x7fb1700cee40 2024-04-25T09:07:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb174a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700b4d60 con 0x7fb170144c10 2024-04-25T09:07:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb16f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17013a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb16f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17013a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:49988/0 (socket says 172.21.15.62:49988) 2024-04-25T09:07:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb16f400640 1 -- 172.21.15.62:0/3657628492 learned_addr learned my addr 172.21.15.62:0/3657628492 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb167e00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1700cee40 0x7fb170139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb16f400640 1 -- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb170144c10 msgr2=0x7fb17015e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb16fe00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb170144c10 0x7fb17015e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb16f400640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb170144c10 0x7fb17015e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb16f400640 1 -- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1700cee40 msgr2=0x7fb170139900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.668+0000 7fb16f400640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1700cee40 0x7fb170139900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.669+0000 7fb16f400640 1 -- 172.21.15.62:0/3657628492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17013e080 con 0x7fb17006d900 2024-04-25T09:07:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.669+0000 7fb16f400640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17013a160 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c430 tx=0x7fb16000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.669+0000 7fb167400640 1 -- 172.21.15.62:0/3657628492 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1600074e0 con 0x7fb17006d900 2024-04-25T09:07:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.669+0000 7fb167400640 1 -- 172.21.15.62:0/3657628492 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1600047e0 con 0x7fb17006d900 2024-04-25T09:07:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.669+0000 7fb167400640 1 -- 172.21.15.62:0/3657628492 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160004bd0 con 0x7fb17006d900 2024-04-25T09:07:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.669+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17013a740 con 0x7fb17006d900 2024-04-25T09:07:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.669+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17013e620 con 0x7fb17006d900 2024-04-25T09:07:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.671+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17004ef20 con 0x7fb17006d900 2024-04-25T09:07:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.672+0000 7fb167400640 1 -- 172.21.15.62:0/3657628492 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb160011070 con 0x7fb17006d900 2024-04-25T09:07:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.672+0000 7fb167400640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb14c06b4d0 0x7fb14c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.673+0000 7fb167400640 1 -- 172.21.15.62:0/3657628492 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb160089bc0 con 0x7fb17006d900 2024-04-25T09:07:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.673+0000 7fb167e00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb14c06b4d0 0x7fb14c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:21.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.674+0000 7fb167e00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb14c06b4d0 0x7fb14c06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb158034000 tx=0x7fb15802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:21.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.677+0000 7fb167400640 1 -- 172.21.15.62:0/3657628492 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb160058cf0 con 0x7fb17006d900 2024-04-25T09:07:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.880+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1700b22b0 con 0x7fb17006d900 2024-04-25T09:07:21.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.883+0000 7fb167400640 1 -- 172.21.15.62:0/3657628492 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+12879 (secure 0 0 0) 0x7fb1600585f0 con 0x7fb17006d900 2024-04-25T09:07:21.871 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:21.871 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":44,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:07:20.939110+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:21.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.885+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb14c06b4d0 msgr2=0x7fb14c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.885+0000 7fb174a00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb14c06b4d0 0x7fb14c06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb158034000 tx=0x7fb15802e000 comp rx=0 tx=0).stop 2024-04-25T09:07:21.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 msgr2=0x7fb17013a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:21.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17013a160 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb16000c430 tx=0x7fb16000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 shutdown_connections 2024-04-25T09:07:21.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb14c06b4d0 0x7fb14c06d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb170144c10 0x7fb17015e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1700cee40 0x7fb170139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 --2- 172.21.15.62:0/3657628492 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb17006d900 0x7fb17013a160 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:21.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 >> 172.21.15.62:0/3657628492 conn(0x7fb1700b14d0 msgr2=0x7fb17006c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:21.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 shutdown_connections 2024-04-25T09:07:21.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:21.886+0000 7fb174a00640 1 -- 172.21.15.62:0/3657628492 wait complete. 2024-04-25T09:07:21.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:07:21.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:07:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.113+0000 7fe02a800640 1 Processor -- start 2024-04-25T09:07:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.113+0000 7fe02a800640 1 -- start start 2024-04-25T09:07:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.115+0000 7fe02a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0240ccde0 0x7fe0240ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.115+0000 7fe02a800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0240ced40 con 0x7fe0240b31a0 2024-04-25T09:07:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.115+0000 7fe02a800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0240cd260 con 0x7fe02406d2c0 2024-04-25T09:07:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.115+0000 7fe02a800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0240cf000 con 0x7fe0240ccde0 2024-04-25T09:07:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.115+0000 7fe028a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0240ccde0 0x7fe0240ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.115+0000 7fe028a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0240ccde0 0x7fe0240ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50000/0 (socket says 172.21.15.62:50000) 2024-04-25T09:07:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.115+0000 7fe028a00640 1 -- 172.21.15.62:0/223806723 learned_addr learned my addr 172.21.15.62:0/223806723 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.115+0000 7fe029e00640 1 --1- 172.21.15.62:0/223806723 >> v1:172.21.15.62:6789/0 conn(0x7fe0240b31a0 0x7fe0240b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36912/0 (socket says 172.21.15.62:36912) 2024-04-25T09:07:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.116+0000 7fe028a00640 1 -- 172.21.15.62:0/223806723 >> v1:172.21.15.121:6789/0 conn(0x7fe02406d2c0 legacy=0x7fe02406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.116+0000 7fe028a00640 1 -- 172.21.15.62:0/223806723 >> v1:172.21.15.62:6789/0 conn(0x7fe0240b31a0 legacy=0x7fe0240b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.116+0000 7fe028a00640 1 -- 172.21.15.62:0/223806723 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0240cf2e0 con 0x7fe0240ccde0 2024-04-25T09:07:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.117+0000 7fe028a00640 1 --2- 172.21.15.62:0/223806723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe0240ce800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe010009870 tx=0x7fe01002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83427edcf8261511 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.117+0000 7fe023e00640 1 -- 172.21.15.62:0/223806723 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010037070 con 0x7fe0240ccde0 2024-04-25T09:07:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.117+0000 7fe023e00640 1 -- 172.21.15.62:0/223806723 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01002ae50 con 0x7fe0240ccde0 2024-04-25T09:07:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.117+0000 7fe023e00640 1 -- 172.21.15.62:0/223806723 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010032860 con 0x7fe0240ccde0 2024-04-25T09:07:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.118+0000 7fe02a800640 1 -- 172.21.15.62:0/223806723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 msgr2=0x7fe0240ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.118+0000 7fe02a800640 1 --2- 172.21.15.62:0/223806723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe0240ce800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe010009870 tx=0x7fe01002a300 comp rx=0 tx=0).stop 2024-04-25T09:07:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.118+0000 7fe02a800640 1 -- 172.21.15.62:0/223806723 shutdown_connections 2024-04-25T09:07:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.118+0000 7fe02a800640 1 --2- 172.21.15.62:0/223806723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe0240ce800 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.118+0000 7fe02a800640 1 -- 172.21.15.62:0/223806723 >> 172.21.15.62:0/223806723 conn(0x7fe024066f20 msgr2=0x7fe0240b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.118+0000 7fe02a800640 1 -- 172.21.15.62:0/223806723 shutdown_connections 2024-04-25T09:07:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.118+0000 7fe02a800640 1 -- 172.21.15.62:0/223806723 wait complete. 2024-04-25T09:07:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.120+0000 7fe02a800640 1 Processor -- start 2024-04-25T09:07:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.120+0000 7fe02a800640 1 -- start start 2024-04-25T09:07:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe02a800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02406d2c0 0x7fe0241e6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe02a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0240b31a0 0x7fe024070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe02a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe02415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe02a800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0240b2c80 con 0x7fe0240ccde0 2024-04-25T09:07:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe02a800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0240b29a0 con 0x7fe0240b31a0 2024-04-25T09:07:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe02a800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0241448d0 con 0x7fe02406d2c0 2024-04-25T09:07:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe029e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe02415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe029e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe02415af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50012/0 (socket says 172.21.15.62:50012) 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe029e00640 1 -- 172.21.15.62:0/1091346089 learned_addr learned my addr 172.21.15.62:0/1091346089 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe029e00640 1 -- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02406d2c0 msgr2=0x7fe0241e6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe028a00640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0240b31a0 0x7fe024070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.121+0000 7fe029e00640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02406d2c0 0x7fe0241e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe029e00640 1 -- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0240b31a0 msgr2=0x7fe024070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe029e00640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0240b31a0 0x7fe024070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe029e00640 1 -- 172.21.15.62:0/1091346089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0241303a0 con 0x7fe0240ccde0 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe028a00640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0240b31a0 0x7fe024070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe029e00640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe02415af00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00c780 tx=0x7fe01c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe021600640 1 -- 172.21.15.62:0/1091346089 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c017070 con 0x7fe0240ccde0 2024-04-25T09:07:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe021600640 1 -- 172.21.15.62:0/1091346089 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c00a640 con 0x7fe0240ccde0 2024-04-25T09:07:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe021600640 1 -- 172.21.15.62:0/1091346089 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00aa30 con 0x7fe0240ccde0 2024-04-25T09:07:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe024130680 con 0x7fe0240ccde0 2024-04-25T09:07:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.122+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe024130c40 con 0x7fe0240ccde0 2024-04-25T09:07:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.124+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe024074830 con 0x7fe0240ccde0 2024-04-25T09:07:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.124+0000 7fe021600640 1 -- 172.21.15.62:0/1091346089 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe01c01a070 con 0x7fe0240ccde0 2024-04-25T09:07:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.125+0000 7fe021600640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe00406b500 0x7fe00406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.125+0000 7fe021600640 1 -- 172.21.15.62:0/1091346089 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe01c089520 con 0x7fe0240ccde0 2024-04-25T09:07:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.126+0000 7fe029400640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe00406b500 0x7fe00406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.128+0000 7fe029400640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe00406b500 0x7fe00406d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe018000f60 tx=0x7fe0180079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:22.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.130+0000 7fe021600640 1 -- 172.21.15.62:0/1091346089 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe024074830 con 0x7fe0240ccde0 2024-04-25T09:07:22.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.328+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0240aed90 con 0x7fe0240ccde0 2024-04-25T09:07:22.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.329+0000 7fe021600640 1 -- 172.21.15.62:0/1091346089 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe01c057f70 con 0x7fe0240ccde0 2024-04-25T09:07:22.318 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:07:22.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.332+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe00406b500 msgr2=0x7fe00406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.332+0000 7fe02a800640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe00406b500 0x7fe00406d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe018000f60 tx=0x7fe0180079b0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.332+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 msgr2=0x7fe02415af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.332+0000 7fe02a800640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe02415af00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00c780 tx=0x7fe01c00cc40 comp rx=0 tx=0).stop 2024-04-25T09:07:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.332+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 shutdown_connections 2024-04-25T09:07:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.332+0000 7fe02a800640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe00406b500 0x7fe00406d9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.332+0000 7fe02a800640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0240ccde0 0x7fe02415af00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.333+0000 7fe02a800640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0240b31a0 0x7fe024070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.333+0000 7fe02a800640 1 --2- 172.21.15.62:0/1091346089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02406d2c0 0x7fe0241e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.333+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 >> 172.21.15.62:0/1091346089 conn(0x7fe024066f20 msgr2=0x7fe02406d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.333+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 shutdown_connections 2024-04-25T09:07:22.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.333+0000 7fe02a800640 1 -- 172.21.15.62:0/1091346089 wait complete. 2024-04-25T09:07:22.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:07:22.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.561+0000 7f4d0b600640 1 Processor -- start 2024-04-25T09:07:22.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.562+0000 7f4d0b600640 1 -- start start 2024-04-25T09:07:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.563+0000 7f4d0b600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d040d0550 0x7f4d040d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.563+0000 7f4d0b600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d041622e0 con 0x7f4d040d2500 2024-04-25T09:07:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.563+0000 7f4d0b600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d041625c0 con 0x7f4d04161140 2024-04-25T09:07:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.563+0000 7f4d0b600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d040d1d00 con 0x7f4d040d0550 2024-04-25T09:07:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.564+0000 7f4d03e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d040d0550 0x7f4d040d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.564+0000 7f4d03e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d040d0550 0x7f4d040d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50026/0 (socket says 172.21.15.62:50026) 2024-04-25T09:07:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.564+0000 7f4d03e00640 1 -- 172.21.15.62:0/1225048237 learned_addr learned my addr 172.21.15.62:0/1225048237 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.564+0000 7f4d09600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4d040d2500 0x7f4d040d2910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36918/0 (socket says 172.21.15.62:36918) 2024-04-25T09:07:22.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.564+0000 7f4d03e00640 1 -- 172.21.15.62:0/1225048237 >> v1:172.21.15.121:6789/0 conn(0x7f4d04161140 legacy=0x7f4d041618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.564+0000 7f4d03e00640 1 -- 172.21.15.62:0/1225048237 >> v1:172.21.15.62:6789/0 conn(0x7f4d040d2500 legacy=0x7f4d040d2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.564+0000 7f4d03e00640 1 -- 172.21.15.62:0/1225048237 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040d1fe0 con 0x7f4d040d0550 2024-04-25T09:07:22.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.565+0000 7f4d03e00640 1 --2- 172.21.15.62:0/1225048237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d0550 0x7f4d040d09e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009870 tx=0x7f4cf002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb2a0db47f5c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:22.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d03400640 1 -- 172.21.15.62:0/1225048237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0037070 con 0x7f4d040d0550 2024-04-25T09:07:22.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d03400640 1 -- 172.21.15.62:0/1225048237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf002ae50 con 0x7f4d040d0550 2024-04-25T09:07:22.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d03400640 1 -- 172.21.15.62:0/1225048237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0032860 con 0x7f4d040d0550 2024-04-25T09:07:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d0b600640 1 -- 172.21.15.62:0/1225048237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d0550 msgr2=0x7f4d040d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d0b600640 1 --2- 172.21.15.62:0/1225048237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d0550 0x7f4d040d09e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009870 tx=0x7f4cf002a300 comp rx=0 tx=0).stop 2024-04-25T09:07:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d0b600640 1 -- 172.21.15.62:0/1225048237 shutdown_connections 2024-04-25T09:07:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d0b600640 1 --2- 172.21.15.62:0/1225048237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d0550 0x7f4d040d09e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d0b600640 1 -- 172.21.15.62:0/1225048237 >> 172.21.15.62:0/1225048237 conn(0x7f4d040668f0 msgr2=0x7f4d04066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.566+0000 7f4d0b600640 1 -- 172.21.15.62:0/1225048237 shutdown_connections 2024-04-25T09:07:22.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.567+0000 7f4d0b600640 1 -- 172.21.15.62:0/1225048237 wait complete. 2024-04-25T09:07:22.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.568+0000 7f4d0b600640 1 Processor -- start 2024-04-25T09:07:22.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.568+0000 7f4d0b600640 1 -- start start 2024-04-25T09:07:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d0b600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d040d0550 0x7f4d0415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d0b600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d2500 0x7f4d040c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d0b600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d04161140 0x7f4d0414ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d0b600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041616a0 con 0x7f4d040d2500 2024-04-25T09:07:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d0b600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04161520 con 0x7f4d040d0550 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d0b600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040b9970 con 0x7f4d04161140 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d03e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d2500 0x7f4d040c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d03e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d2500 0x7f4d040c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50032/0 (socket says 172.21.15.62:50032) 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.569+0000 7f4d03e00640 1 -- 172.21.15.62:0/144390593 learned_addr learned my addr 172.21.15.62:0/144390593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d09600640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d04161140 0x7f4d0414ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d08c00640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d040d0550 0x7f4d0415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d03e00640 1 -- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d04161140 msgr2=0x7f4d0414ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d03e00640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d04161140 0x7f4d0414ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d03e00640 1 -- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d040d0550 msgr2=0x7f4d0415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d03e00640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d040d0550 0x7f4d0415b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d03e00640 1 -- 172.21.15.62:0/144390593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040c9810 con 0x7f4d040d2500 2024-04-25T09:07:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d03e00640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d2500 0x7f4d040c9070 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0000c00 tx=0x7f4cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.570+0000 7f4d00c00640 1 -- 172.21.15.62:0/144390593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0041070 con 0x7f4d040d2500 2024-04-25T09:07:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.571+0000 7f4d00c00640 1 -- 172.21.15.62:0/144390593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf003c890 con 0x7f4d040d2500 2024-04-25T09:07:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.571+0000 7f4d00c00640 1 -- 172.21.15.62:0/144390593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0037040 con 0x7f4d040d2500 2024-04-25T09:07:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.571+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d040c9af0 con 0x7f4d040d2500 2024-04-25T09:07:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.571+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d040ca0b0 con 0x7f4d040d2500 2024-04-25T09:07:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.572+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040b3cd0 con 0x7f4d040d2500 2024-04-25T09:07:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.573+0000 7f4d00c00640 1 -- 172.21.15.62:0/144390593 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cf0044070 con 0x7f4d040d2500 2024-04-25T09:07:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.574+0000 7f4d00c00640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ce406b500 0x7f4ce406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.574+0000 7f4d00c00640 1 -- 172.21.15.62:0/144390593 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4cf00aa880 con 0x7f4d040d2500 2024-04-25T09:07:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.574+0000 7f4d08c00640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ce406b500 0x7f4ce406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.576+0000 7f4d08c00640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ce406b500 0x7f4ce406d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4007fd0 tx=0x7f4cf4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.578+0000 7f4d00c00640 1 -- 172.21.15.62:0/144390593 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4cf0079920 con 0x7f4d040d2500 2024-04-25T09:07:22.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.779+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4d0404ef20 con 0x7f4d040d2500 2024-04-25T09:07:22.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.780+0000 7f4d00c00640 1 -- 172.21.15.62:0/144390593 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4cf0079220 con 0x7f4d040d2500 2024-04-25T09:07:22.768 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:22.768 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:07:22.769 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:07:22.769 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:07:22.769 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:07:22.769 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:22.769 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ce406b500 msgr2=0x7f4ce406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ce406b500 0x7f4ce406d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4007fd0 tx=0x7f4cf4009040 comp rx=0 tx=0).stop 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d2500 msgr2=0x7f4d040c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d2500 0x7f4d040c9070 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0000c00 tx=0x7f4cf00046c0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 shutdown_connections 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ce406b500 0x7f4ce406d9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d040d2500 0x7f4d040c9070 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d04161140 0x7f4d0414ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 --2- 172.21.15.62:0/144390593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d040d0550 0x7f4d0415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:22.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 >> 172.21.15.62:0/144390593 conn(0x7f4d040668f0 msgr2=0x7f4d040d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.782+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 shutdown_connections 2024-04-25T09:07:22.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:22.783+0000 7f4d0b600640 1 -- 172.21.15.62:0/144390593 wait complete. 2024-04-25T09:07:22.789 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:07:22.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:07:23.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.020+0000 7f58a9200640 1 Processor -- start 2024-04-25T09:07:23.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.020+0000 7f58a9200640 1 -- start start 2024-04-25T09:07:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.022+0000 7f58a9200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58a415b520 0x7f58a40ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.022+0000 7f58a9200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58a406c440 0x7f58a40adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.022+0000 7f58a9200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a406e270 con 0x7f58a415eb90 2024-04-25T09:07:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.022+0000 7f58a9200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40b3fb0 con 0x7f58a406c440 2024-04-25T09:07:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.022+0000 7f58a9200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a406c920 con 0x7f58a415b520 2024-04-25T09:07:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.022+0000 7f58a2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58a406c440 0x7f58a40adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.022+0000 7f58a2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58a406c440 0x7f58a40adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50046/0 (socket says 172.21.15.62:50046) 2024-04-25T09:07:23.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.022+0000 7f58a2a00640 1 -- 172.21.15.62:0/3069695989 learned_addr learned my addr 172.21.15.62:0/3069695989 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.023+0000 7f58a3400640 1 --2- 172.21.15.62:0/3069695989 >> v2:172.21.15.121:3300/0 conn(0x7f58a415b520 0x7f58a40ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.023+0000 7f58a2a00640 1 -- 172.21.15.62:0/3069695989 >> v2:172.21.15.121:3300/0 conn(0x7f58a415b520 msgr2=0x7f58a40ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.023+0000 7f58a2a00640 1 --2- 172.21.15.62:0/3069695989 >> v2:172.21.15.121:3300/0 conn(0x7f58a415b520 0x7f58a40ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.023+0000 7f58a2a00640 1 -- 172.21.15.62:0/3069695989 >> v1:172.21.15.121:6789/0 conn(0x7f58a415eb90 legacy=0x7f58a415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.023+0000 7f58a2a00640 1 -- 172.21.15.62:0/3069695989 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a406cc00 con 0x7f58a406c440 2024-04-25T09:07:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.024+0000 7f58a2a00640 1 --2- 172.21.15.62:0/3069695989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a406c440 0x7f58a40adea0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f588c009870 tx=0x7f588c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a70e94e6c07ff8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.025+0000 7f58a2000640 1 -- 172.21.15.62:0/3069695989 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c037070 con 0x7f58a406c440 2024-04-25T09:07:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.025+0000 7f58a2000640 1 -- 172.21.15.62:0/3069695989 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c032440 con 0x7f58a406c440 2024-04-25T09:07:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.025+0000 7f58a2000640 1 -- 172.21.15.62:0/3069695989 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c032810 con 0x7f58a406c440 2024-04-25T09:07:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.026+0000 7f58a9200640 1 -- 172.21.15.62:0/3069695989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a406c440 msgr2=0x7f58a40adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.026+0000 7f58a9200640 1 --2- 172.21.15.62:0/3069695989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a406c440 0x7f58a40adea0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f588c009870 tx=0x7f588c02a270 comp rx=0 tx=0).stop 2024-04-25T09:07:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.026+0000 7f58a9200640 1 -- 172.21.15.62:0/3069695989 shutdown_connections 2024-04-25T09:07:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.026+0000 7f58a9200640 1 --2- 172.21.15.62:0/3069695989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a406c440 0x7f58a40adea0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.026+0000 7f58a9200640 1 --2- 172.21.15.62:0/3069695989 >> v2:172.21.15.121:3300/0 conn(0x7f58a415b520 0x7f58a40ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.026+0000 7f58a9200640 1 -- 172.21.15.62:0/3069695989 >> 172.21.15.62:0/3069695989 conn(0x7f58a4066ad0 msgr2=0x7f58a4066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:23.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.026+0000 7f58a9200640 1 -- 172.21.15.62:0/3069695989 shutdown_connections 2024-04-25T09:07:23.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.026+0000 7f58a9200640 1 -- 172.21.15.62:0/3069695989 wait complete. 2024-04-25T09:07:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.028+0000 7f58a9200640 1 Processor -- start 2024-04-25T09:07:23.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.028+0000 7f58a9200640 1 -- start start 2024-04-25T09:07:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a9200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58a406c440 0x7f58a413d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a415b520 0x7f58a4141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a9200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58a415eb90 0x7f58a40ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a9200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a415b900 con 0x7f58a415b520 2024-04-25T09:07:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a9200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a406d1c0 con 0x7f58a415eb90 2024-04-25T09:07:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a9200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a415ba80 con 0x7f58a406c440 2024-04-25T09:07:23.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a415b520 0x7f58a4141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a415b520 0x7f58a4141170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50058/0 (socket says 172.21.15.62:50058) 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a2a00640 1 -- 172.21.15.62:0/1334830658 learned_addr learned my addr 172.21.15.62:0/1334830658 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a3400640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58a406c440 0x7f58a413d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a3e00640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58a415eb90 0x7f58a40ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a2a00640 1 -- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58a406c440 msgr2=0x7f58a413d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a2a00640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58a406c440 0x7f58a413d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a2a00640 1 -- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58a415eb90 msgr2=0x7f58a40ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a2a00640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58a415eb90 0x7f58a40ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.029+0000 7f58a2a00640 1 -- 172.21.15.62:0/1334830658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a413d670 con 0x7f58a415b520 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.030+0000 7f58a3e00640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58a415eb90 0x7f58a40ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:23.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.030+0000 7f58a2a00640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a415b520 0x7f58a4141170 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f588c009870 tx=0x7f588c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.030+0000 7f589b400640 1 -- 172.21.15.62:0/1334830658 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c037040 con 0x7f58a415b520 2024-04-25T09:07:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.030+0000 7f589b400640 1 -- 172.21.15.62:0/1334830658 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c004230 con 0x7f58a415b520 2024-04-25T09:07:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.030+0000 7f589b400640 1 -- 172.21.15.62:0/1334830658 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c003610 con 0x7f58a415b520 2024-04-25T09:07:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.030+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a41419c0 con 0x7f58a415b520 2024-04-25T09:07:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.030+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a4141f80 con 0x7f58a415b520 2024-04-25T09:07:23.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.032+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a4072b30 con 0x7f58a415b520 2024-04-25T09:07:23.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.033+0000 7f589b400640 1 -- 172.21.15.62:0/1334830658 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f588c0037b0 con 0x7f58a415b520 2024-04-25T09:07:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.034+0000 7f589b400640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f588006b500 0x7f588006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:23.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.034+0000 7f589b400640 1 -- 172.21.15.62:0/1334830658 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f588c0aad00 con 0x7f58a415b520 2024-04-25T09:07:23.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.035+0000 7f58a3400640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f588006b500 0x7f588006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:23.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.036+0000 7f58a3400640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f588006b500 0x7f588006d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5894009a60 tx=0x7f5894003990 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:23.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.038+0000 7f589b400640 1 -- 172.21.15.62:0/1334830658 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58a413d670 con 0x7f58a415b520 2024-04-25T09:07:23.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.244+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f58a40750e0 con 0x7f58a415b520 2024-04-25T09:07:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.958+0000 7f589b400640 1 -- 172.21.15.62:0/1334830658 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7f588c079ec0 con 0x7f58a415b520 2024-04-25T09:07:23.947 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:07:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f588006b500 msgr2=0x7f588006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f588006b500 0x7f588006d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5894009a60 tx=0x7f5894003990 comp rx=0 tx=0).stop 2024-04-25T09:07:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a415b520 msgr2=0x7f58a4141170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a415b520 0x7f58a4141170 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f588c009870 tx=0x7f588c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 shutdown_connections 2024-04-25T09:07:23.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f588006b500 0x7f588006d9c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58a415b520 0x7f58a4141170 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58a415eb90 0x7f58a40ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 --2- 172.21.15.62:0/1334830658 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58a406c440 0x7f58a413d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.963+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 >> 172.21.15.62:0/1334830658 conn(0x7f58a4066ad0 msgr2=0x7f58a40aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.964+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 shutdown_connections 2024-04-25T09:07:23.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:23.964+0000 7f58a9200640 1 -- 172.21.15.62:0/1334830658 wait complete. 2024-04-25T09:07:23.980 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:07:24.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.193+0000 7f12c0a00640 1 Processor -- start 2024-04-25T09:07:24.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.193+0000 7f12c0a00640 1 -- start start 2024-04-25T09:07:24.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.194+0000 7f12c0a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f12bc0b1080 0x7f12bc166700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:24.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.194+0000 7f12c0a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12bc06d490 con 0x7f12bc159680 2024-04-25T09:07:24.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.194+0000 7f12c0a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12bc06c3f0 con 0x7f12bc0cc960 2024-04-25T09:07:24.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.194+0000 7f12c0a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12bc06d000 con 0x7f12bc0b1080 2024-04-25T09:07:24.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.195+0000 7f12ba800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f12bc159680 0x7f12bc15cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36928/0 (socket says 172.21.15.62:36928) 2024-04-25T09:07:24.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.195+0000 7f12ba800640 1 -- 172.21.15.62:0/1181848207 learned_addr learned my addr 172.21.15.62:0/1181848207 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:24.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.195+0000 7f12bb200640 1 --2- 172.21.15.62:0/1181848207 >> v2:172.21.15.121:3300/0 conn(0x7f12bc0b1080 0x7f12bc166700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:24.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.195+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622834004 0 0) 0x7f12bc06d490 con 0x7f12bc159680 2024-04-25T09:07:24.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.195+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12a0003520 con 0x7f12bc159680 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2646768597 0 0) 0x7f12a0003520 con 0x7f12bc159680 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 >> v2:172.21.15.121:3300/0 conn(0x7f12bc0b1080 msgr2=0x7f12bc166700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 --2- 172.21.15.62:0/1181848207 >> v2:172.21.15.121:3300/0 conn(0x7f12bc0b1080 0x7f12bc166700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 >> v1:172.21.15.121:6789/0 conn(0x7f12bc0cc960 legacy=0x7f12bc0b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc06c6d0 con 0x7f12bc159680 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f12ac001670 con 0x7f12bc159680 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 >> v1:172.21.15.62:6789/0 conn(0x7f12bc159680 legacy=0x7f12bc15cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12a00039b0 0x7f12a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:24.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12a00051c0 0x7f12a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc06d000 con 0x7f12a0004800 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a000c060 con 0x7f12a00051c0 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a000c260 con 0x7f12a00039b0 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9e00640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12ba800640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12a00039b0 0x7f12a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9e00640 1 -- 172.21.15.62:0/1181848207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12a00039b0 msgr2=0x7f12a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.196+0000 7f12b9e00640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12a00039b0 0x7f12a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.197+0000 7f12b9e00640 1 -- 172.21.15.62:0/1181848207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12a00051c0 msgr2=0x7f12a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.197+0000 7f12b9e00640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12a00051c0 0x7f12a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.197+0000 7f12b9e00640 1 -- 172.21.15.62:0/1181848207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a0003520 con 0x7f12a0004800 2024-04-25T09:07:24.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.197+0000 7f12bb200640 1 -- 172.21.15.62:0/1181848207 reap_dead start 2024-04-25T09:07:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.198+0000 7f12b9e00640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12a0004c80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f12a4001720 tx=0x7f12a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aa5f37affa3dad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.198+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4001370 con 0x7f12a0004800 2024-04-25T09:07:24.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.198+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a40052d0 con 0x7f12a0004800 2024-04-25T09:07:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.198+0000 7f12b9400640 1 -- 172.21.15.62:0/1181848207 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a402abc0 con 0x7f12a0004800 2024-04-25T09:07:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.198+0000 7f12c0a00640 1 -- 172.21.15.62:0/1181848207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 msgr2=0x7f12a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.198+0000 7f12c0a00640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12a0004c80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f12a4001720 tx=0x7f12a0005830 comp rx=0 tx=0).stop 2024-04-25T09:07:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.199+0000 7f12c0a00640 1 -- 172.21.15.62:0/1181848207 shutdown_connections 2024-04-25T09:07:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.199+0000 7f12c0a00640 1 --2- 172.21.15.62:0/1181848207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12a0004c80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.199+0000 7f12c0a00640 1 -- 172.21.15.62:0/1181848207 >> 172.21.15.62:0/1181848207 conn(0x7f12bc0668f0 msgr2=0x7f12bc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.199+0000 7f12c0a00640 1 -- 172.21.15.62:0/1181848207 shutdown_connections 2024-04-25T09:07:24.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.199+0000 7f12c0a00640 1 -- 172.21.15.62:0/1181848207 wait complete. 2024-04-25T09:07:24.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.200+0000 7f12c0a00640 1 Processor -- start 2024-04-25T09:07:24.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.201+0000 7f12c0a00640 1 -- start start 2024-04-25T09:07:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.201+0000 7f12c0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12bc0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12c0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12bc144e60 0x7f12bc1452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12c0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12bc15cbe0 0x7f12bc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12c0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc159a60 con 0x7f12a0004800 2024-04-25T09:07:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12c0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc06cc90 con 0x7f12bc15cbe0 2024-04-25T09:07:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12c0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12bc0b0970 con 0x7f12bc144e60 2024-04-25T09:07:24.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12ba800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12bc0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:24.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12ba800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12bc0cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50072/0 (socket says 172.21.15.62:50072) 2024-04-25T09:07:24.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12ba800640 1 -- 172.21.15.62:0/4084036078 learned_addr learned my addr 172.21.15.62:0/4084036078 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12b9e00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12bc144e60 0x7f12bc1452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12ba800640 1 -- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12bc144e60 msgr2=0x7f12bc1452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12ba800640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12bc144e60 0x7f12bc1452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12ba800640 1 -- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12bc15cbe0 msgr2=0x7f12bc15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12bb200640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12bc15cbe0 0x7f12bc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12ba800640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12bc15cbe0 0x7f12bc15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.202+0000 7f12ba800640 1 -- 172.21.15.62:0/4084036078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc145820 con 0x7f12a0004800 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.203+0000 7f12ba800640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12bc0cbd00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00e760 tx=0x7f12ac00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.203+0000 7f12b2a00640 1 -- 172.21.15.62:0/4084036078 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac00ee70 con 0x7f12a0004800 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.203+0000 7f12b2a00640 1 -- 172.21.15.62:0/4084036078 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac018070 con 0x7f12a0004800 2024-04-25T09:07:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.203+0000 7f12b2a00640 1 -- 172.21.15.62:0/4084036078 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac00ca70 con 0x7f12a0004800 2024-04-25T09:07:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.203+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc15d9a0 con 0x7f12a0004800 2024-04-25T09:07:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.203+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12bc142f60 con 0x7f12a0004800 2024-04-25T09:07:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.204+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12bc075070 con 0x7f12a0004800 2024-04-25T09:07:24.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.205+0000 7f12b2a00640 1 -- 172.21.15.62:0/4084036078 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12ac00cc10 con 0x7f12a0004800 2024-04-25T09:07:24.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.207+0000 7f12b2a00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f129806b4d0 0x7f129806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:24.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.207+0000 7f12b2a00640 1 -- 172.21.15.62:0/4084036078 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f12ac089e20 con 0x7f12a0004800 2024-04-25T09:07:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.208+0000 7f12b9e00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f129806b4d0 0x7f129806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:24.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.209+0000 7f12b9e00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f129806b4d0 0x7f129806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f12a4032000 tx=0x7f12a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:24.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.212+0000 7f12b2a00640 1 -- 172.21.15.62:0/4084036078 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12ac058e70 con 0x7f12a0004800 2024-04-25T09:07:24.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.416+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f12bc06e020 con 0x7f12a0004800 2024-04-25T09:07:24.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.975+0000 7f12b2a00640 1 -- 172.21.15.62:0/4084036078 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7f12ac058770 con 0x7f12a0004800 2024-04-25T09:07:24.964 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:07:24.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f129806b4d0 msgr2=0x7f129806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:24.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f129806b4d0 0x7f129806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f12a4032000 tx=0x7f12a402e000 comp rx=0 tx=0).stop 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 msgr2=0x7f12bc0cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12bc0cbd00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00e760 tx=0x7f12ac00ec20 comp rx=0 tx=0).stop 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 shutdown_connections 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f129806b4d0 0x7f129806d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12bc15cbe0 0x7f12bc15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f12bc144e60 0x7f12bc1452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 --2- 172.21.15.62:0/4084036078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12a0004800 0x7f12bc0cbd00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 >> 172.21.15.62:0/4084036078 conn(0x7f12bc0668f0 msgr2=0x7f12bc0cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 shutdown_connections 2024-04-25T09:07:24.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:24.979+0000 7f12c0a00640 1 -- 172.21.15.62:0/4084036078 wait complete. 2024-04-25T09:07:25.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:07:25.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.196+0000 7fca5d600640 1 Processor -- start 2024-04-25T09:07:25.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.196+0000 7fca5d600640 1 -- start start 2024-04-25T09:07:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.198+0000 7fca5d600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca580ccde0 0x7fca580ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.198+0000 7fca5d600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca580ced40 con 0x7fca580b31a0 2024-04-25T09:07:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.198+0000 7fca5d600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca580cd260 con 0x7fca5806d2c0 2024-04-25T09:07:25.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.198+0000 7fca5d600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca580cf000 con 0x7fca580ccde0 2024-04-25T09:07:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.198+0000 7fca5cc00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fca580b31a0 0x7fca580b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44688/0 (socket says 172.21.15.62:44688) 2024-04-25T09:07:25.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.198+0000 7fca5cc00640 1 -- 172.21.15.62:0/264588521 learned_addr learned my addr 172.21.15.62:0/264588521 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:25.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca4fe00640 1 --2- 172.21.15.62:0/264588521 >> v2:172.21.15.121:3300/0 conn(0x7fca580ccde0 0x7fca580ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660244898 0 0) 0x7fca580ced40 con 0x7fca580b31a0 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3c003520 con 0x7fca580b31a0 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2685156638 0 0) 0x7fca3c003520 con 0x7fca580b31a0 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 >> v2:172.21.15.121:3300/0 conn(0x7fca580ccde0 msgr2=0x7fca580ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 --2- 172.21.15.62:0/264588521 >> v2:172.21.15.121:3300/0 conn(0x7fca580ccde0 0x7fca580ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 >> v1:172.21.15.121:6789/0 conn(0x7fca5806d2c0 legacy=0x7fca5806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580cf2e0 con 0x7fca580b31a0 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fca500019f0 con 0x7fca580b31a0 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 >> v1:172.21.15.62:6789/0 conn(0x7fca580b31a0 legacy=0x7fca580b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca3c0051c0 0x7fca3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:25.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580cf000 con 0x7fca3c004800 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c00c060 con 0x7fca3c0051c0 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.199+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c00c260 con 0x7fca3c0039b0 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca4fe00640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca5cc00640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca3c0051c0 0x7fca3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca57e00640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca4fe00640 1 -- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c0039b0 msgr2=0x7fca3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca4fe00640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca4fe00640 1 -- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca3c0051c0 msgr2=0x7fca3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca4fe00640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca3c0051c0 0x7fca3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca4fe00640 1 -- 172.21.15.62:0/264588521 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c003520 con 0x7fca3c004800 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca57e00640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c0039b0 0x7fca3c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca5cc00640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca3c0051c0 0x7fca3c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:25.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.200+0000 7fca5cc00640 1 -- 172.21.15.62:0/264588521 reap_dead start 2024-04-25T09:07:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.201+0000 7fca4fe00640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fca40009ac0 tx=0x7fca3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38abed62dc2d050c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40036070 con 0x7fca3c004800 2024-04-25T09:07:25.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca40031400 con 0x7fca3c004800 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca57400640 1 -- 172.21.15.62:0/264588521 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40009690 con 0x7fca3c004800 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca5d600640 1 -- 172.21.15.62:0/264588521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca3c004800 msgr2=0x7fca3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca5d600640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fca40009ac0 tx=0x7fca3c005830 comp rx=0 tx=0).stop 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca5d600640 1 -- 172.21.15.62:0/264588521 shutdown_connections 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca5d600640 1 --2- 172.21.15.62:0/264588521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca3c004800 0x7fca3c004c80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca5d600640 1 -- 172.21.15.62:0/264588521 >> 172.21.15.62:0/264588521 conn(0x7fca58066f20 msgr2=0x7fca580b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca5d600640 1 -- 172.21.15.62:0/264588521 shutdown_connections 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.202+0000 7fca5d600640 1 -- 172.21.15.62:0/264588521 wait complete. 2024-04-25T09:07:25.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.204+0000 7fca5d600640 1 Processor -- start 2024-04-25T09:07:25.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.204+0000 7fca5d600640 1 -- start start 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.205+0000 7fca5d600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c004800 0x7fca58140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.205+0000 7fca5d600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca581f3790 0x7fca58071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.205+0000 7fca5d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca58071eb0 0x7fca580af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.205+0000 7fca5d600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580b2c80 con 0x7fca58071eb0 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.205+0000 7fca5d600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580b29a0 con 0x7fca581f3790 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.205+0000 7fca5d600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca581448d0 con 0x7fca3c004800 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca5cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca58071eb0 0x7fca580af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca5cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca58071eb0 0x7fca580af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33062/0 (socket says 172.21.15.62:33062) 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca5cc00640 1 -- 172.21.15.62:0/173597876 learned_addr learned my addr 172.21.15.62:0/173597876 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca57e00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c004800 0x7fca58140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca4fe00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca581f3790 0x7fca58071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:25.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca5cc00640 1 -- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c004800 msgr2=0x7fca58140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca5cc00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c004800 0x7fca58140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca5cc00640 1 -- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca581f3790 msgr2=0x7fca58071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca5cc00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca581f3790 0x7fca58071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca5cc00640 1 -- 172.21.15.62:0/173597876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca581f3c20 con 0x7fca58071eb0 2024-04-25T09:07:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.206+0000 7fca4fe00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca581f3790 0x7fca58071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:25.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.207+0000 7fca5cc00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca58071eb0 0x7fca580af330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fca580cc6f0 tx=0x7fca50007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.207+0000 7fca54c00640 1 -- 172.21.15.62:0/173597876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000e9c0 con 0x7fca58071eb0 2024-04-25T09:07:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.207+0000 7fca54c00640 1 -- 172.21.15.62:0/173597876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca50018070 con 0x7fca58071eb0 2024-04-25T09:07:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.207+0000 7fca54c00640 1 -- 172.21.15.62:0/173597876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000eda0 con 0x7fca58071eb0 2024-04-25T09:07:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.207+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca581f3f00 con 0x7fca58071eb0 2024-04-25T09:07:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.207+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca581f44c0 con 0x7fca58071eb0 2024-04-25T09:07:25.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.208+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5806abe0 con 0x7fca58071eb0 2024-04-25T09:07:25.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.210+0000 7fca54c00640 1 -- 172.21.15.62:0/173597876 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca5001d460 con 0x7fca58071eb0 2024-04-25T09:07:25.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.211+0000 7fca54c00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca3406b4d0 0x7fca3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:25.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.211+0000 7fca54c00640 1 -- 172.21.15.62:0/173597876 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fca5008a440 con 0x7fca58071eb0 2024-04-25T09:07:25.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.211+0000 7fca57e00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca3406b4d0 0x7fca3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:25.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.213+0000 7fca57e00640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca3406b4d0 0x7fca3406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca580aff40 tx=0x7fca48009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:25.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.216+0000 7fca54c00640 1 -- 172.21.15.62:0/173597876 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca50059280 con 0x7fca58071eb0 2024-04-25T09:07:25.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.451+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fca580b0070 con 0x7fca58071eb0 2024-04-25T09:07:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.991+0000 7fca54c00640 1 -- 172.21.15.62:0/173597876 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fca50058b80 con 0x7fca58071eb0 2024-04-25T09:07:25.983 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T09:07:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.999+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca3406b4d0 msgr2=0x7fca3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.999+0000 7fca5d600640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca3406b4d0 0x7fca3406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca580aff40 tx=0x7fca48009040 comp rx=0 tx=0).stop 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.999+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca58071eb0 msgr2=0x7fca580af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:25.999+0000 7fca5d600640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca58071eb0 0x7fca580af330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fca580cc6f0 tx=0x7fca50007af0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.001+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 shutdown_connections 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.001+0000 7fca5d600640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca3406b4d0 0x7fca3406d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.001+0000 7fca5d600640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca58071eb0 0x7fca580af330 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.001+0000 7fca5d600640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca581f3790 0x7fca58071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.001+0000 7fca5d600640 1 --2- 172.21.15.62:0/173597876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca3c004800 0x7fca58140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:25.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.001+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 >> 172.21.15.62:0/173597876 conn(0x7fca58066f20 msgr2=0x7fca5815bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:25.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.002+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 shutdown_connections 2024-04-25T09:07:25.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.002+0000 7fca5d600640 1 -- 172.21.15.62:0/173597876 wait complete. 2024-04-25T09:07:26.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.206+0000 7f3ee0400640 1 Processor -- start 2024-04-25T09:07:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.207+0000 7f3ee0400640 1 -- start start 2024-04-25T09:07:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.208+0000 7f3ee0400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3ed80cfed0 0x7f3ed815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.208+0000 7f3ee0400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed8072340 0x7f3ed80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.208+0000 7f3ee0400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed806c310 con 0x7f3ed80707f0 2024-04-25T09:07:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.208+0000 7f3ee0400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed815f5f0 con 0x7f3ed80cfed0 2024-04-25T09:07:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.208+0000 7f3ee0400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed806c5b0 con 0x7f3ed8072340 2024-04-25T09:07:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.209+0000 7f3edd000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed8072340 0x7f3ed80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.209+0000 7f3edd000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed8072340 0x7f3ed80702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:51292/0 (socket says 172.21.15.62:51292) 2024-04-25T09:07:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.209+0000 7f3edd000640 1 -- 172.21.15.62:0/769641497 learned_addr learned my addr 172.21.15.62:0/769641497 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.209+0000 7f3ede400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3ed80707f0 0x7f3ed815b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35814/0 (socket says 172.21.15.62:35814) 2024-04-25T09:07:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.209+0000 7f3edda00640 1 --2- 172.21.15.62:0/769641497 >> v2:172.21.15.121:3300/0 conn(0x7f3ed80cfed0 0x7f3ed815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.210+0000 7f3edda00640 1 -- 172.21.15.62:0/769641497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed8072340 msgr2=0x7f3ed80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.210+0000 7f3edda00640 1 --2- 172.21.15.62:0/769641497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed8072340 0x7f3ed80702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.210+0000 7f3edda00640 1 -- 172.21.15.62:0/769641497 >> v1:172.21.15.121:6789/0 conn(0x7f3ed80707f0 legacy=0x7f3ed815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.210+0000 7f3edda00640 1 -- 172.21.15.62:0/769641497 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed806c890 con 0x7f3ed80cfed0 2024-04-25T09:07:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.211+0000 7f3edda00640 1 --2- 172.21.15.62:0/769641497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 0x7f3ed815f0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ed806c4e0 tx=0x7f3ecc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd6c80ca876756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:26.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ed7e00640 1 -- 172.21.15.62:0/769641497 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc037070 con 0x7f3ed80cfed0 2024-04-25T09:07:26.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ed7e00640 1 -- 172.21.15.62:0/769641497 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc032440 con 0x7f3ed80cfed0 2024-04-25T09:07:26.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ed7e00640 1 -- 172.21.15.62:0/769641497 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc032810 con 0x7f3ed80cfed0 2024-04-25T09:07:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ee0400640 1 -- 172.21.15.62:0/769641497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 msgr2=0x7f3ed815f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ee0400640 1 --2- 172.21.15.62:0/769641497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 0x7f3ed815f0b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ed806c4e0 tx=0x7f3ecc02a180 comp rx=0 tx=0).stop 2024-04-25T09:07:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ee0400640 1 -- 172.21.15.62:0/769641497 shutdown_connections 2024-04-25T09:07:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ee0400640 1 --2- 172.21.15.62:0/769641497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed8072340 0x7f3ed80702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ee0400640 1 --2- 172.21.15.62:0/769641497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 0x7f3ed815f0b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.212+0000 7f3ee0400640 1 -- 172.21.15.62:0/769641497 >> 172.21.15.62:0/769641497 conn(0x7f3ed80668f0 msgr2=0x7f3ed8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.213+0000 7f3ee0400640 1 -- 172.21.15.62:0/769641497 shutdown_connections 2024-04-25T09:07:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.213+0000 7f3ee0400640 1 -- 172.21.15.62:0/769641497 wait complete. 2024-04-25T09:07:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.215+0000 7f3ee0400640 1 Processor -- start 2024-04-25T09:07:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.215+0000 7f3ee0400640 1 -- start start 2024-04-25T09:07:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.216+0000 7f3ee0400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed80707f0 0x7f3ed815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.216+0000 7f3ee0400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 0x7f3ed81438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.216+0000 7f3ee0400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed815a630 0x7f3ed8143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.216+0000 7f3ee0400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed806d130 con 0x7f3ed815a630 2024-04-25T09:07:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.216+0000 7f3ee0400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed806ce50 con 0x7f3ed80cfed0 2024-04-25T09:07:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.216+0000 7f3ee0400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed80b3890 con 0x7f3ed80707f0 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.216+0000 7f3ede400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed815a630 0x7f3ed8143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ede400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed815a630 0x7f3ed8143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33072/0 (socket says 172.21.15.62:33072) 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ede400640 1 -- 172.21.15.62:0/2338441876 learned_addr learned my addr 172.21.15.62:0/2338441876 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3edd000640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 0x7f3ed81438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3edda00640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed80707f0 0x7f3ed815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ede400640 1 -- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed80707f0 msgr2=0x7f3ed815b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ede400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed80707f0 0x7f3ed815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ede400640 1 -- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 msgr2=0x7f3ed81438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ede400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 0x7f3ed81438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ede400640 1 -- 172.21.15.62:0/2338441876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed8142850 con 0x7f3ed815a630 2024-04-25T09:07:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ede400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed815a630 0x7f3ed8143e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c780 tx=0x7f3ed000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.217+0000 7f3ed5600640 1 -- 172.21.15.62:0/2338441876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0017070 con 0x7f3ed815a630 2024-04-25T09:07:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.218+0000 7f3ed5600640 1 -- 172.21.15.62:0/2338441876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed000a640 con 0x7f3ed815a630 2024-04-25T09:07:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.218+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed8142b30 con 0x7f3ed815a630 2024-04-25T09:07:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.218+0000 7f3ed5600640 1 -- 172.21.15.62:0/2338441876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed000aa30 con 0x7f3ed815a630 2024-04-25T09:07:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.218+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed81430f0 con 0x7f3ed815a630 2024-04-25T09:07:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.219+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed8072c70 con 0x7f3ed815a630 2024-04-25T09:07:26.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.220+0000 7f3ed5600640 1 -- 172.21.15.62:0/2338441876 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ed001a070 con 0x7f3ed815a630 2024-04-25T09:07:26.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.221+0000 7f3ed5600640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b4d0 0x7f3eb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.221+0000 7f3ed5600640 1 -- 172.21.15.62:0/2338441876 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ed00897d0 con 0x7f3ed815a630 2024-04-25T09:07:26.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.222+0000 7f3edda00640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b4d0 0x7f3eb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.223+0000 7f3edda00640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b4d0 0x7f3eb806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc034000 tx=0x7f3ecc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:26.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.226+0000 7f3ed5600640 1 -- 172.21.15.62:0/2338441876 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ed0057f60 con 0x7f3ed815a630 2024-04-25T09:07:26.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.420+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ed804f000 con 0x7f3ed815a630 2024-04-25T09:07:26.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.422+0000 7f3ed5600640 1 -- 172.21.15.62:0/2338441876 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f3ed005c000 con 0x7f3ed815a630 2024-04-25T09:07:26.411 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:26.411 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":47,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:07:25.453408+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:07:23.299693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T09:07:24.475140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:26.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b4d0 msgr2=0x7f3eb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b4d0 0x7f3eb806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc034000 tx=0x7f3ecc02e000 comp rx=0 tx=0).stop 2024-04-25T09:07:26.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed815a630 msgr2=0x7f3ed8143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed815a630 0x7f3ed8143e30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c780 tx=0x7f3ed000cc40 comp rx=0 tx=0).stop 2024-04-25T09:07:26.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 shutdown_connections 2024-04-25T09:07:26.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3eb806b4d0 0x7f3eb806d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3ed815a630 0x7f3ed8143e30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ed80cfed0 0x7f3ed81438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 --2- 172.21.15.62:0/2338441876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3ed80707f0 0x7f3ed815b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 >> 172.21.15.62:0/2338441876 conn(0x7f3ed80668f0 msgr2=0x7f3ed80cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:26.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 shutdown_connections 2024-04-25T09:07:26.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.426+0000 7f3ee0400640 1 -- 172.21.15.62:0/2338441876 wait complete. 2024-04-25T09:07:26.432 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.648+0000 7f78e1e00640 1 Processor -- start 2024-04-25T09:07:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.648+0000 7f78e1e00640 1 -- start start 2024-04-25T09:07:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.649+0000 7f78e1e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78dc06d900 0x7f78dc1685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.650+0000 7f78e1e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78dc0cf030 con 0x7f78dc15d810 2024-04-25T09:07:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.650+0000 7f78e1e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78dc0cf310 con 0x7f78dc15dbf0 2024-04-25T09:07:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.650+0000 7f78e1e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc0c8cd0 con 0x7f78dc06d900 2024-04-25T09:07:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.650+0000 7f78e0a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f78dc15d810 0x7f78dc0ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44694/0 (socket says 172.21.15.62:44694) 2024-04-25T09:07:26.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.650+0000 7f78e0a00640 1 -- 172.21.15.62:0/21065009 learned_addr learned my addr 172.21.15.62:0/21065009 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.650+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978897368 0 0) 0x7f78dc0cf030 con 0x7f78dc15d810 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.650+0000 7f78e1400640 1 --2- 172.21.15.62:0/21065009 >> v2:172.21.15.121:3300/0 conn(0x7f78dc06d900 0x7f78dc1685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.650+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c0003520 con 0x7f78dc15d810 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1090971313 0 0) 0x7f78c0003520 con 0x7f78dc15d810 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 >> v2:172.21.15.121:3300/0 conn(0x7f78dc06d900 msgr2=0x7f78dc1685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 --2- 172.21.15.62:0/21065009 >> v2:172.21.15.121:3300/0 conn(0x7f78dc06d900 0x7f78dc1685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 >> v1:172.21.15.121:6789/0 conn(0x7f78dc15dbf0 legacy=0x7f78dc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0c8fb0 con 0x7f78dc15d810 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f78cc001670 con 0x7f78dc15d810 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78e1400640 1 --2- 172.21.15.62:0/21065009 >> v2:172.21.15.121:3300/0 conn(0x7f78dc06d900 0x7f78dc1685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:26.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 >> v1:172.21.15.62:6789/0 conn(0x7f78dc15d810 legacy=0x7f78dc0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78c0003a00 0x7f78c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78c0004800 0x7f78c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0c8cd0 con 0x7f78c0005210 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c000c160 con 0x7f78c0004800 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.651+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c000c360 con 0x7f78c0003a00 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.652+0000 7f78e1400640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.652+0000 7f78e0a00640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78c0003a00 0x7f78c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.652+0000 7f78dbe00640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78c0004800 0x7f78c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.652+0000 7f78e1400640 1 -- 172.21.15.62:0/21065009 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78c0003a00 msgr2=0x7f78c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.652+0000 7f78e1400640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78c0003a00 0x7f78c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.652+0000 7f78e1400640 1 -- 172.21.15.62:0/21065009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78c0004800 msgr2=0x7f78c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.652+0000 7f78e1400640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78c0004800 0x7f78c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.652+0000 7f78e1400640 1 -- 172.21.15.62:0/21065009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c0003520 con 0x7f78c0005210 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.653+0000 7f78e1400640 1 -- 172.21.15.62:0/21065009 reap_dead start 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.653+0000 7f78e1400640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78c000b9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f78d40073b0 tx=0x7f78c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40212efe84a51972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d4009c80 con 0x7f78c0005210 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d402c650 con 0x7f78c0005210 2024-04-25T09:07:26.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78db400640 1 -- 172.21.15.62:0/21065009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d402ca40 con 0x7f78c0005210 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78e1e00640 1 -- 172.21.15.62:0/21065009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 msgr2=0x7f78c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78e1e00640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78c000b9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f78d40073b0 tx=0x7f78c0005830 comp rx=0 tx=0).stop 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78e1e00640 1 -- 172.21.15.62:0/21065009 shutdown_connections 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78e1e00640 1 --2- 172.21.15.62:0/21065009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78c000b9c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78e1e00640 1 -- 172.21.15.62:0/21065009 >> 172.21.15.62:0/21065009 conn(0x7f78dc0b14d0 msgr2=0x7f78dc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78e1e00640 1 -- 172.21.15.62:0/21065009 shutdown_connections 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.654+0000 7f78e1e00640 1 -- 172.21.15.62:0/21065009 wait complete. 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.656+0000 7f78e1e00640 1 Processor -- start 2024-04-25T09:07:26.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.656+0000 7f78e1e00640 1 -- start start 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.657+0000 7f78e1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78dc144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.657+0000 7f78e1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78dc0b4930 0x7f78dc14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.657+0000 7f78e1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78dc0b2ed0 0x7f78dc14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.657+0000 7f78e1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0ce830 con 0x7f78c0005210 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.657+0000 7f78e1e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0b10d0 con 0x7f78dc0b2ed0 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.657+0000 7f78e1e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0ce9b0 con 0x7f78dc0b4930 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78dc144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78dc144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33082/0 (socket says 172.21.15.62:33082) 2024-04-25T09:07:26.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 -- 172.21.15.62:0/3009216504 learned_addr learned my addr 172.21.15.62:0/3009216504 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e1400640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78dc0b2ed0 0x7f78dc14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78dbe00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78dc0b4930 0x7f78dc14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 -- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78dc0b4930 msgr2=0x7f78dc14d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78dc0b4930 0x7f78dc14d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 -- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78dc0b2ed0 msgr2=0x7f78dc14fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78dc0b2ed0 0x7f78dc14fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 -- 172.21.15.62:0/3009216504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0b3720 con 0x7f78c0005210 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e1400640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78dc0b2ed0 0x7f78dc14fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.658+0000 7f78e0a00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78dc144870 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f78cc001050 tx=0x7f78cc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:26.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.659+0000 7f78d8c00640 1 -- 172.21.15.62:0/3009216504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00c660 con 0x7f78c0005210 2024-04-25T09:07:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.659+0000 7f78d8c00640 1 -- 172.21.15.62:0/3009216504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc018070 con 0x7f78c0005210 2024-04-25T09:07:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.659+0000 7f78d8c00640 1 -- 172.21.15.62:0/3009216504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00ca60 con 0x7f78c0005210 2024-04-25T09:07:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.659+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc14da50 con 0x7f78c0005210 2024-04-25T09:07:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.659+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc14d670 con 0x7f78c0005210 2024-04-25T09:07:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.660+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc14e060 con 0x7f78c0005210 2024-04-25T09:07:26.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.661+0000 7f78d8c00640 1 -- 172.21.15.62:0/3009216504 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78cc00cc00 con 0x7f78c0005210 2024-04-25T09:07:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.662+0000 7f78d8c00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78bc06b4d0 0x7f78bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:26.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.662+0000 7f78d8c00640 1 -- 172.21.15.62:0/3009216504 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78cc089f60 con 0x7f78c0005210 2024-04-25T09:07:26.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.663+0000 7f78dbe00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78bc06b4d0 0x7f78bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:26.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.664+0000 7f78dbe00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78bc06b4d0 0x7f78bc06d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f78c8007fd0 tx=0x7f78c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:26.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.667+0000 7f78d8c00640 1 -- 172.21.15.62:0/3009216504 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78cc058e70 con 0x7f78c0005210 2024-04-25T09:07:26.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.869+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78dc0b4d10 con 0x7f78c0005210 2024-04-25T09:07:26.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.871+0000 7f78d8c00640 1 -- 172.21.15.62:0/3009216504 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f78dc0b4d10 con 0x7f78c0005210 2024-04-25T09:07:26.860 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:26.860 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":47,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:07:25.453408+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:07:23.299693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T09:07:24.475140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:26.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78bc06b4d0 msgr2=0x7f78bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78bc06b4d0 0x7f78bc06d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f78c8007fd0 tx=0x7f78c8009040 comp rx=0 tx=0).stop 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 msgr2=0x7f78dc144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78dc144870 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f78cc001050 tx=0x7f78cc00eb50 comp rx=0 tx=0).stop 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 shutdown_connections 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78bc06b4d0 0x7f78bc06d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78dc0b2ed0 0x7f78dc14fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78dc0b4930 0x7f78dc14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 --2- 172.21.15.62:0/3009216504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78c0005210 0x7f78dc144870 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 >> 172.21.15.62:0/3009216504 conn(0x7f78dc0b14d0 msgr2=0x7f78dc070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 shutdown_connections 2024-04-25T09:07:26.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:26.874+0000 7f78e1e00640 1 -- 172.21.15.62:0/3009216504 wait complete. 2024-04-25T09:07:26.881 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:07:27.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.112+0000 7fb568600640 1 Processor -- start 2024-04-25T09:07:27.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.112+0000 7fb568600640 1 -- start start 2024-04-25T09:07:27.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.113+0000 7fb568600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb5600d05a0 0x7fb56015f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:27.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.113+0000 7fb568600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb56015fd50 0x7fb56006e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:27.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.113+0000 7fb568600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb560160160 con 0x7fb56015bde0 2024-04-25T09:07:27.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.114+0000 7fb568600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5600d0a10 con 0x7fb56015fd50 2024-04-25T09:07:27.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.114+0000 7fb568600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5600cf500 con 0x7fb5600d05a0 2024-04-25T09:07:27.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.114+0000 7fb566600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb56015fd50 0x7fb56006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.114+0000 7fb566600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb56015fd50 0x7fb56006e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33088/0 (socket says 172.21.15.62:33088) 2024-04-25T09:07:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.114+0000 7fb566600640 1 -- 172.21.15.62:0/2958253263 learned_addr learned my addr 172.21.15.62:0/2958253263 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.114+0000 7fb565200640 1 --2- 172.21.15.62:0/2958253263 >> v2:172.21.15.121:3300/0 conn(0x7fb5600d05a0 0x7fb56015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.114+0000 7fb566600640 1 -- 172.21.15.62:0/2958253263 >> v2:172.21.15.121:3300/0 conn(0x7fb5600d05a0 msgr2=0x7fb56015f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.114+0000 7fb566600640 1 --2- 172.21.15.62:0/2958253263 >> v2:172.21.15.121:3300/0 conn(0x7fb5600d05a0 0x7fb56015f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.115+0000 7fb566600640 1 -- 172.21.15.62:0/2958253263 >> v1:172.21.15.62:6789/0 conn(0x7fb56015bde0 legacy=0x7fb5600cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.115+0000 7fb566600640 1 -- 172.21.15.62:0/2958253263 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5600cf7e0 con 0x7fb56015fd50 2024-04-25T09:07:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.115+0000 7fb55fe00640 1 -- 172.21.15.62:0/2958253263 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703150953 0 0) 0x7fb550001110 con 0x7fb56015bde0 2024-04-25T09:07:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.116+0000 7fb566600640 1 --2- 172.21.15.62:0/2958253263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015fd50 0x7fb56006e590 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb5600cc010 tx=0x7fb55802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1967d6bd7ee0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.116+0000 7fb55fe00640 1 -- 172.21.15.62:0/2958253263 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558037070 con 0x7fb56015fd50 2024-04-25T09:07:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.116+0000 7fb55fe00640 1 -- 172.21.15.62:0/2958253263 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55802ae50 con 0x7fb56015fd50 2024-04-25T09:07:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.116+0000 7fb55fe00640 1 -- 172.21.15.62:0/2958253263 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558032860 con 0x7fb56015fd50 2024-04-25T09:07:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.116+0000 7fb568600640 1 -- 172.21.15.62:0/2958253263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015fd50 msgr2=0x7fb56006e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.116+0000 7fb568600640 1 --2- 172.21.15.62:0/2958253263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015fd50 0x7fb56006e590 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb5600cc010 tx=0x7fb55802a300 comp rx=0 tx=0).stop 2024-04-25T09:07:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.117+0000 7fb568600640 1 -- 172.21.15.62:0/2958253263 shutdown_connections 2024-04-25T09:07:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.117+0000 7fb568600640 1 --2- 172.21.15.62:0/2958253263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015fd50 0x7fb56006e590 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.117+0000 7fb568600640 1 --2- 172.21.15.62:0/2958253263 >> v2:172.21.15.121:3300/0 conn(0x7fb5600d05a0 0x7fb56015f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.117+0000 7fb568600640 1 -- 172.21.15.62:0/2958253263 >> 172.21.15.62:0/2958253263 conn(0x7fb5600b3440 msgr2=0x7fb5600b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:27.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.117+0000 7fb568600640 1 -- 172.21.15.62:0/2958253263 shutdown_connections 2024-04-25T09:07:27.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.117+0000 7fb568600640 1 -- 172.21.15.62:0/2958253263 wait complete. 2024-04-25T09:07:27.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.118+0000 7fb568600640 1 Processor -- start 2024-04-25T09:07:27.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.119+0000 7fb568600640 1 -- start start 2024-04-25T09:07:27.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.119+0000 7fb568600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5600d05a0 0x7fb56015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:27.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.119+0000 7fb568600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015bde0 0x7fb5600b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.119+0000 7fb568600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb56015fd50 0x7fb5600c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.119+0000 7fb568600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600d0080 con 0x7fb56015bde0 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.119+0000 7fb568600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600cfda0 con 0x7fb56015fd50 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.119+0000 7fb568600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb560160570 con 0x7fb5600d05a0 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015bde0 0x7fb5600b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015bde0 0x7fb5600b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33100/0 (socket says 172.21.15.62:33100) 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565200640 1 -- 172.21.15.62:0/3413518748 learned_addr learned my addr 172.21.15.62:0/3413518748 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb566600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb56015fd50 0x7fb5600c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565200640 1 -- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5600d05a0 msgr2=0x7fb56015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565c00640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5600d05a0 0x7fb56015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565200640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5600d05a0 0x7fb56015ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565200640 1 -- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb56015fd50 msgr2=0x7fb5600c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565200640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb56015fd50 0x7fb5600c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb565200640 1 -- 172.21.15.62:0/3413518748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56015c1c0 con 0x7fb56015bde0 2024-04-25T09:07:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.120+0000 7fb566600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb56015fd50 0x7fb5600c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.121+0000 7fb565200640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015bde0 0x7fb5600b5870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb554007e70 tx=0x7fb55400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.121+0000 7fb55d600640 1 -- 172.21.15.62:0/3413518748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554014620 con 0x7fb56015bde0 2024-04-25T09:07:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.121+0000 7fb55d600640 1 -- 172.21.15.62:0/3413518748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb554019070 con 0x7fb56015bde0 2024-04-25T09:07:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.121+0000 7fb55d600640 1 -- 172.21.15.62:0/3413518748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554014a00 con 0x7fb56015bde0 2024-04-25T09:07:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.121+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56015c4a0 con 0x7fb56015bde0 2024-04-25T09:07:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.121+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56015cd40 con 0x7fb56015bde0 2024-04-25T09:07:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.122+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56004ef20 con 0x7fb56015bde0 2024-04-25T09:07:27.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.124+0000 7fb55d600640 1 -- 172.21.15.62:0/3413518748 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb554014ba0 con 0x7fb56015bde0 2024-04-25T09:07:27.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.125+0000 7fb55d600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb54406b4d0 0x7fb54406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:27.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.125+0000 7fb55d600640 1 -- 172.21.15.62:0/3413518748 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb55408aac0 con 0x7fb56015bde0 2024-04-25T09:07:27.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.126+0000 7fb565c00640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb54406b4d0 0x7fb54406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:27.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.127+0000 7fb565c00640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb54406b4d0 0x7fb54406d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb550000f60 tx=0x7fb550009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:27.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.130+0000 7fb55d600640 1 -- 172.21.15.62:0/3413518748 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb554059940 con 0x7fb56015bde0 2024-04-25T09:07:27.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:27.364+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb5600b3820 con 0x7fb56015bde0 2024-04-25T09:07:27.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.003+0000 7fb55d600640 1 -- 172.21.15.62:0/3413518748 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb554059240 con 0x7fb56015bde0 2024-04-25T09:07:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb54406b4d0 msgr2=0x7fb54406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb54406b4d0 0x7fb54406d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb550000f60 tx=0x7fb550009040 comp rx=0 tx=0).stop 2024-04-25T09:07:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015bde0 msgr2=0x7fb5600b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015bde0 0x7fb5600b5870 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb554007e70 tx=0x7fb55400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 shutdown_connections 2024-04-25T09:07:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb54406b4d0 0x7fb54406d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb56015bde0 0x7fb5600b5870 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb56015fd50 0x7fb5600c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 --2- 172.21.15.62:0/3413518748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb5600d05a0 0x7fb56015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.006+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 >> 172.21.15.62:0/3413518748 conn(0x7fb5600b3440 msgr2=0x7fb56006f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.007+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 shutdown_connections 2024-04-25T09:07:27.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.007+0000 7fb568600640 1 -- 172.21.15.62:0/3413518748 wait complete. 2024-04-25T09:07:28.015 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:07:28.015 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:07:28.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.246+0000 7f4afe800640 1 Processor -- start 2024-04-25T09:07:28.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.246+0000 7f4afe800640 1 -- start start 2024-04-25T09:07:28.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.248+0000 7f4afe800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4af80ceb50 0x7f4af806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:28.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.248+0000 7f4afe800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af815e7c0 0x7f4af80b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:28.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.248+0000 7f4afe800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af815ee30 con 0x7f4af806d800 2024-04-25T09:07:28.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.248+0000 7f4afe800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af815ec00 con 0x7f4af80ceb50 2024-04-25T09:07:28.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.248+0000 7f4afe800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81618b0 con 0x7f4af815e7c0 2024-04-25T09:07:28.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.248+0000 7f4afd400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4af80ceb50 0x7f4af806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:28.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.248+0000 7f4afd400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4af80ceb50 0x7f4af806d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33108/0 (socket says 172.21.15.62:33108) 2024-04-25T09:07:28.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.249+0000 7f4afd400640 1 -- 172.21.15.62:0/2289203044 learned_addr learned my addr 172.21.15.62:0/2289203044 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:28.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.249+0000 7f4afde00640 1 --2- 172.21.15.62:0/2289203044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af815e7c0 0x7f4af80b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:28.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.249+0000 7f4afd400640 1 -- 172.21.15.62:0/2289203044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af815e7c0 msgr2=0x7f4af80b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:28.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.249+0000 7f4afd400640 1 --2- 172.21.15.62:0/2289203044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af815e7c0 0x7f4af80b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.249+0000 7f4afd400640 1 -- 172.21.15.62:0/2289203044 >> v1:172.21.15.121:6789/0 conn(0x7f4af806d800 legacy=0x7f4af815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:28.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.249+0000 7f4afd400640 1 -- 172.21.15.62:0/2289203044 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8161b90 con 0x7f4af80ceb50 2024-04-25T09:07:28.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.250+0000 7f4afd400640 1 --2- 172.21.15.62:0/2289203044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af806d2c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009e50 tx=0x7f4aec02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c510da8a96b7662a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:28.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.251+0000 7f4af7e00640 1 -- 172.21.15.62:0/2289203044 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec036070 con 0x7f4af80ceb50 2024-04-25T09:07:28.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.251+0000 7f4af7e00640 1 -- 172.21.15.62:0/2289203044 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec031600 con 0x7f4af80ceb50 2024-04-25T09:07:28.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.251+0000 7f4af7e00640 1 -- 172.21.15.62:0/2289203044 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0319d0 con 0x7f4af80ceb50 2024-04-25T09:07:28.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.251+0000 7f4afe800640 1 -- 172.21.15.62:0/2289203044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 msgr2=0x7f4af806d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:28.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.251+0000 7f4afe800640 1 --2- 172.21.15.62:0/2289203044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af806d2c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009e50 tx=0x7f4aec02a630 comp rx=0 tx=0).stop 2024-04-25T09:07:28.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.252+0000 7f4afe800640 1 -- 172.21.15.62:0/2289203044 shutdown_connections 2024-04-25T09:07:28.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.252+0000 7f4afe800640 1 --2- 172.21.15.62:0/2289203044 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af815e7c0 0x7f4af80b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.252+0000 7f4afe800640 1 --2- 172.21.15.62:0/2289203044 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af806d2c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.252+0000 7f4afe800640 1 -- 172.21.15.62:0/2289203044 >> 172.21.15.62:0/2289203044 conn(0x7f4af8066f20 msgr2=0x7f4af80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:28.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.252+0000 7f4afe800640 1 -- 172.21.15.62:0/2289203044 shutdown_connections 2024-04-25T09:07:28.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.252+0000 7f4afe800640 1 -- 172.21.15.62:0/2289203044 wait complete. 2024-04-25T09:07:28.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.253+0000 7f4afe800640 1 Processor -- start 2024-04-25T09:07:28.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.254+0000 7f4afe800640 1 -- start start 2024-04-25T09:07:28.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.254+0000 7f4afe800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af806d800 0x7f4af80b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:28.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.254+0000 7f4afe800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af8144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:28.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.254+0000 7f4afe800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af815e7c0 0x7f4af81e9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:28.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.254+0000 7f4afe800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af8162510 con 0x7f4af80ceb50 2024-04-25T09:07:28.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.254+0000 7f4afe800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af8162390 con 0x7f4af815e7c0 2024-04-25T09:07:28.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.254+0000 7f4afe800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80b4a90 con 0x7f4af806d800 2024-04-25T09:07:28.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.255+0000 7f4afca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af8144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.255+0000 7f4afca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af8144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33116/0 (socket says 172.21.15.62:33116) 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.255+0000 7f4afca00640 1 -- 172.21.15.62:0/2882424645 learned_addr learned my addr 172.21.15.62:0/2882424645 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.255+0000 7f4afde00640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af815e7c0 0x7f4af81e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.255+0000 7f4afd400640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af806d800 0x7f4af80b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.255+0000 7f4afca00640 1 -- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af806d800 msgr2=0x7f4af80b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.255+0000 7f4afca00640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af806d800 0x7f4af80b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4afca00640 1 -- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af815e7c0 msgr2=0x7f4af81e9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4afca00640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af815e7c0 0x7f4af81e9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4afca00640 1 -- 172.21.15.62:0/2882424645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af81e8070 con 0x7f4af80ceb50 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4afca00640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af8144490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400c430 tx=0x7f4ae400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4af5600640 1 -- 172.21.15.62:0/2882424645 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4011790 con 0x7f4af80ceb50 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4af5600640 1 -- 172.21.15.62:0/2882424645 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4017070 con 0x7f4af80ceb50 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af81e8350 con 0x7f4af80ceb50 2024-04-25T09:07:28.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4af5600640 1 -- 172.21.15.62:0/2882424645 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4011bb0 con 0x7f4af80ceb50 2024-04-25T09:07:28.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.256+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af81e8910 con 0x7f4af80ceb50 2024-04-25T09:07:28.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.258+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af804ef20 con 0x7f4af80ceb50 2024-04-25T09:07:28.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.259+0000 7f4af5600640 1 -- 172.21.15.62:0/2882424645 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ae401c460 con 0x7f4af80ceb50 2024-04-25T09:07:28.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.260+0000 7f4af5600640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ad806b4d0 0x7f4ad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:28.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.260+0000 7f4af5600640 1 -- 172.21.15.62:0/2882424645 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ae408a180 con 0x7f4af80ceb50 2024-04-25T09:07:28.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.260+0000 7f4afd400640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ad806b4d0 0x7f4ad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:28.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.262+0000 7f4afd400640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ad806b4d0 0x7f4ad806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4aec033000 tx=0x7f4aec009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:28.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.265+0000 7f4af5600640 1 -- 172.21.15.62:0/2882424645 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ae4058f60 con 0x7f4af80ceb50 2024-04-25T09:07:28.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:28.503+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4af806b210 con 0x7f4af80ceb50 2024-04-25T09:07:28.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.006+0000 7f4af5600640 1 -- 172.21.15.62:0/2882424645 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4ae4058860 con 0x7f4af80ceb50 2024-04-25T09:07:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.010+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ad806b4d0 msgr2=0x7f4ad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ad806b4d0 0x7f4ad806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4aec033000 tx=0x7f4aec009000 comp rx=0 tx=0).stop 2024-04-25T09:07:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 msgr2=0x7f4af8144490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af8144490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400c430 tx=0x7f4ae400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 shutdown_connections 2024-04-25T09:07:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ad806b4d0 0x7f4ad806d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4af80ceb50 0x7f4af8144490 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4af815e7c0 0x7f4af81e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 --2- 172.21.15.62:0/2882424645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4af806d800 0x7f4af80b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 >> 172.21.15.62:0/2882424645 conn(0x7f4af8066f20 msgr2=0x7f4af8070590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 shutdown_connections 2024-04-25T09:07:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.011+0000 7f4afe800640 1 -- 172.21.15.62:0/2882424645 wait complete. 2024-04-25T09:07:29.021 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:07:29.021 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.244+0000 7fa677200640 1 Processor -- start 2024-04-25T09:07:29.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.244+0000 7fa677200640 1 -- start start 2024-04-25T09:07:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.246+0000 7fa677200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa67815d810 0x7fa6780ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.246+0000 7fa677200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67806d900 0x7fa6781675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.246+0000 7fa677200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6780ced70 con 0x7fa67815dbf0 2024-04-25T09:07:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.246+0000 7fa677200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780cf030 con 0x7fa67815d810 2024-04-25T09:07:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.246+0000 7fa677200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780cf310 con 0x7fa67806d900 2024-04-25T09:07:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.247+0000 7fa675e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa67815d810 0x7fa6780ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.247+0000 7fa675e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa67815d810 0x7fa6780ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33130/0 (socket says 172.21.15.62:33130) 2024-04-25T09:07:29.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.247+0000 7fa675e00640 1 -- 172.21.15.62:0/3506929849 learned_addr learned my addr 172.21.15.62:0/3506929849 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:29.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.247+0000 7fa675e00640 1 -- 172.21.15.62:0/3506929849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67806d900 msgr2=0x7fa6781675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:29.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.247+0000 7fa675e00640 1 --2- 172.21.15.62:0/3506929849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67806d900 0x7fa6781675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.247+0000 7fa675e00640 1 -- 172.21.15.62:0/3506929849 >> v1:172.21.15.121:6789/0 conn(0x7fa67815dbf0 legacy=0x7fa67806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:29.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.247+0000 7fa675e00640 1 -- 172.21.15.62:0/3506929849 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780c8cd0 con 0x7fa67815d810 2024-04-25T09:07:29.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.248+0000 7fa675e00640 1 --2- 172.21.15.62:0/3506929849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815d810 0x7fa6780ce830 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa668009870 tx=0x7fa66802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865edf5de3bbbee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:29.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.249+0000 7fa675400640 1 -- 172.21.15.62:0/3506929849 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668037070 con 0x7fa67815d810 2024-04-25T09:07:29.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.249+0000 7fa675400640 1 -- 172.21.15.62:0/3506929849 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668032440 con 0x7fa67815d810 2024-04-25T09:07:29.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.249+0000 7fa675400640 1 -- 172.21.15.62:0/3506929849 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668032810 con 0x7fa67815d810 2024-04-25T09:07:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.249+0000 7fa677200640 1 -- 172.21.15.62:0/3506929849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815d810 msgr2=0x7fa6780ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.249+0000 7fa677200640 1 --2- 172.21.15.62:0/3506929849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815d810 0x7fa6780ce830 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa668009870 tx=0x7fa66802a210 comp rx=0 tx=0).stop 2024-04-25T09:07:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.250+0000 7fa677200640 1 -- 172.21.15.62:0/3506929849 shutdown_connections 2024-04-25T09:07:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.250+0000 7fa677200640 1 --2- 172.21.15.62:0/3506929849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67806d900 0x7fa6781675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.250+0000 7fa677200640 1 --2- 172.21.15.62:0/3506929849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815d810 0x7fa6780ce830 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.250+0000 7fa677200640 1 -- 172.21.15.62:0/3506929849 >> 172.21.15.62:0/3506929849 conn(0x7fa6780b14d0 msgr2=0x7fa6780b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.250+0000 7fa677200640 1 -- 172.21.15.62:0/3506929849 shutdown_connections 2024-04-25T09:07:29.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.250+0000 7fa677200640 1 -- 172.21.15.62:0/3506929849 wait complete. 2024-04-25T09:07:29.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.252+0000 7fa677200640 1 Processor -- start 2024-04-25T09:07:29.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.252+0000 7fa677200640 1 -- start start 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.253+0000 7fa677200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa67806d900 0x7fa678155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.253+0000 7fa677200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67815d810 0x7fa67814e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.253+0000 7fa677200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815dbf0 0x7fa67815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.253+0000 7fa677200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780ca660 con 0x7fa67815dbf0 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.253+0000 7fa677200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780ca4e0 con 0x7fa67806d900 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.253+0000 7fa677200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780ca7e0 con 0x7fa67815d810 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa676800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815dbf0 0x7fa67815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa676800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815dbf0 0x7fa67815bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33142/0 (socket says 172.21.15.62:33142) 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa676800640 1 -- 172.21.15.62:0/2437274536 learned_addr learned my addr 172.21.15.62:0/2437274536 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa675e00640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa67806d900 0x7fa678155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa66d400640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67815d810 0x7fa67814e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa676800640 1 -- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67815d810 msgr2=0x7fa67814e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa676800640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67815d810 0x7fa67814e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa676800640 1 -- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa67806d900 msgr2=0x7fa678155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa676800640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa67806d900 0x7fa678155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.254+0000 7fa676800640 1 -- 172.21.15.62:0/2437274536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678152010 con 0x7fa67815dbf0 2024-04-25T09:07:29.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.255+0000 7fa676800640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815dbf0 0x7fa67815bff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa67000b970 tx=0x7fa67000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:29.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.255+0000 7fa66ea00640 1 -- 172.21.15.62:0/2437274536 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670019070 con 0x7fa67815dbf0 2024-04-25T09:07:29.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.255+0000 7fa66ea00640 1 -- 172.21.15.62:0/2437274536 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67000cb60 con 0x7fa67815dbf0 2024-04-25T09:07:29.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.255+0000 7fa66ea00640 1 -- 172.21.15.62:0/2437274536 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670014440 con 0x7fa67815dbf0 2024-04-25T09:07:29.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.255+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6781548b0 con 0x7fa67815dbf0 2024-04-25T09:07:29.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.255+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67815b7f0 con 0x7fa67815dbf0 2024-04-25T09:07:29.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.257+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67804ef20 con 0x7fa67815dbf0 2024-04-25T09:07:29.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.257+0000 7fa66ea00640 1 -- 172.21.15.62:0/2437274536 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa67001c070 con 0x7fa67815dbf0 2024-04-25T09:07:29.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.258+0000 7fa66ea00640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa65406b500 0x7fa65406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.258+0000 7fa66ea00640 1 -- 172.21.15.62:0/2437274536 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa670089dc0 con 0x7fa67815dbf0 2024-04-25T09:07:29.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.259+0000 7fa675e00640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa65406b500 0x7fa65406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.260+0000 7fa675e00640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa65406b500 0x7fa65406d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa668034000 tx=0x7fa66802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:29.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.263+0000 7fa66ea00640 1 -- 172.21.15.62:0/2437274536 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6781548b0 con 0x7fa67815dbf0 2024-04-25T09:07:29.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.504+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa67806ea80 con 0x7fa67815dbf0 2024-04-25T09:07:29.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.505+0000 7fa66ea00640 1 -- 172.21.15.62:0/2437274536 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fa670058680 con 0x7fa67815dbf0 2024-04-25T09:07:29.492 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:29.492 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:28.504417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:07:29.492 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-04-25T09:07:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa65406b500 msgr2=0x7fa65406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa65406b500 0x7fa65406d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa668034000 tx=0x7fa66802e000 comp rx=0 tx=0).stop 2024-04-25T09:07:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815dbf0 msgr2=0x7fa67815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815dbf0 0x7fa67815bff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa67000b970 tx=0x7fa67000be30 comp rx=0 tx=0).stop 2024-04-25T09:07:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 shutdown_connections 2024-04-25T09:07:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa65406b500 0x7fa65406d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa67815dbf0 0x7fa67815bff0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa67815d810 0x7fa67814e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 --2- 172.21.15.62:0/2437274536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa67806d900 0x7fa678155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.507+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 >> 172.21.15.62:0/2437274536 conn(0x7fa6780b14d0 msgr2=0x7fa6780cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:29.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.508+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 shutdown_connections 2024-04-25T09:07:29.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.508+0000 7fa677200640 1 -- 172.21.15.62:0/2437274536 wait complete. 2024-04-25T09:07:29.515 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:07:29.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.742+0000 7f2e91000640 1 Processor -- start 2024-04-25T09:07:29.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.742+0000 7f2e91000640 1 -- start start 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e91000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2e8c15d810 0x7f2e8c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e91000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e8c15dbf0 0x7f2e8c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e91000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c0ced70 con 0x7f2e8c0b0ec0 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e91000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c0cf030 con 0x7f2e8c15dbf0 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e91000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c0cf310 con 0x7f2e8c15d810 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e8aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e8c15dbf0 0x7f2e8c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e8be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2e8c0b0ec0 0x7f2e8c0c94d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44710/0 (socket says 172.21.15.62:44710) 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e8be00640 1 -- 172.21.15.62:0/3438141335 learned_addr learned my addr 172.21.15.62:0/3438141335 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.744+0000 7f2e8b400640 1 --2- 172.21.15.62:0/3438141335 >> v2:172.21.15.121:3300/0 conn(0x7f2e8c15d810 0x7f2e8c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.745+0000 7f2e8aa00640 1 -- 172.21.15.62:0/3438141335 >> v2:172.21.15.121:3300/0 conn(0x7f2e8c15d810 msgr2=0x7f2e8c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.745+0000 7f2e8aa00640 1 --2- 172.21.15.62:0/3438141335 >> v2:172.21.15.121:3300/0 conn(0x7f2e8c15d810 0x7f2e8c0ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.745+0000 7f2e8aa00640 1 -- 172.21.15.62:0/3438141335 >> v1:172.21.15.62:6789/0 conn(0x7f2e8c0b0ec0 legacy=0x7f2e8c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.745+0000 7f2e8aa00640 1 -- 172.21.15.62:0/3438141335 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0c8cd0 con 0x7f2e8c15dbf0 2024-04-25T09:07:29.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.746+0000 7f2e8aa00640 1 --2- 172.21.15.62:0/3438141335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15dbf0 0x7f2e8c06d1f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0b5a90 tx=0x7f2e7802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46c590d58e710c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:29.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.746+0000 7f2e8a000640 1 -- 172.21.15.62:0/3438141335 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78037070 con 0x7f2e8c15dbf0 2024-04-25T09:07:29.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.746+0000 7f2e8a000640 1 -- 172.21.15.62:0/3438141335 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78032440 con 0x7f2e8c15dbf0 2024-04-25T09:07:29.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.746+0000 7f2e8a000640 1 -- 172.21.15.62:0/3438141335 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78032810 con 0x7f2e8c15dbf0 2024-04-25T09:07:29.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.747+0000 7f2e91000640 1 -- 172.21.15.62:0/3438141335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15dbf0 msgr2=0x7f2e8c06d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.747+0000 7f2e91000640 1 --2- 172.21.15.62:0/3438141335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15dbf0 0x7f2e8c06d1f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0b5a90 tx=0x7f2e7802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.747+0000 7f2e91000640 1 -- 172.21.15.62:0/3438141335 shutdown_connections 2024-04-25T09:07:29.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.747+0000 7f2e91000640 1 --2- 172.21.15.62:0/3438141335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15dbf0 0x7f2e8c06d1f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.747+0000 7f2e91000640 1 --2- 172.21.15.62:0/3438141335 >> v2:172.21.15.121:3300/0 conn(0x7f2e8c15d810 0x7f2e8c0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.747+0000 7f2e91000640 1 -- 172.21.15.62:0/3438141335 >> 172.21.15.62:0/3438141335 conn(0x7f2e8c0b14d0 msgr2=0x7f2e8c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.747+0000 7f2e91000640 1 -- 172.21.15.62:0/3438141335 shutdown_connections 2024-04-25T09:07:29.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.747+0000 7f2e91000640 1 -- 172.21.15.62:0/3438141335 wait complete. 2024-04-25T09:07:29.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.748+0000 7f2e91000640 1 Processor -- start 2024-04-25T09:07:29.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.749+0000 7f2e91000640 1 -- start start 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e91000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e8c0b0ec0 0x7f2e8c150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e91000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15d810 0x7f2e8c140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e91000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8c15dbf0 0x7f2e8c140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e91000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c06d8b0 con 0x7f2e8c15d810 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e91000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c06d730 con 0x7f2e8c0b0ec0 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e91000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c06da30 con 0x7f2e8c15dbf0 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15d810 0x7f2e8c140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15d810 0x7f2e8c140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33160/0 (socket says 172.21.15.62:33160) 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8aa00640 1 -- 172.21.15.62:0/1874921012 learned_addr learned my addr 172.21.15.62:0/1874921012 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8b400640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e8c0b0ec0 0x7f2e8c150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8aa00640 1 -- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8c15dbf0 msgr2=0x7f2e8c140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8aa00640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8c15dbf0 0x7f2e8c140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8aa00640 1 -- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e8c0b0ec0 msgr2=0x7f2e8c150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8aa00640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e8c0b0ec0 0x7f2e8c150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.750+0000 7f2e8aa00640 1 -- 172.21.15.62:0/1874921012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c1f4710 con 0x7f2e8c15d810 2024-04-25T09:07:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.751+0000 7f2e8aa00640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15d810 0x7f2e8c140730 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2e78035040 tx=0x7f2e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:29.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.751+0000 7f2e83400640 1 -- 172.21.15.62:0/1874921012 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78037040 con 0x7f2e8c15d810 2024-04-25T09:07:29.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.751+0000 7f2e83400640 1 -- 172.21.15.62:0/1874921012 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78042070 con 0x7f2e8c15d810 2024-04-25T09:07:29.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.751+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c144190 con 0x7f2e8c15d810 2024-04-25T09:07:29.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.751+0000 7f2e83400640 1 -- 172.21.15.62:0/1874921012 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78030bd0 con 0x7f2e8c15d810 2024-04-25T09:07:29.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.751+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c0ca940 con 0x7f2e8c15d810 2024-04-25T09:07:29.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.752+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c04ef20 con 0x7f2e8c15d810 2024-04-25T09:07:29.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.753+0000 7f2e83400640 1 -- 172.21.15.62:0/1874921012 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e78047460 con 0x7f2e8c15d810 2024-04-25T09:07:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.754+0000 7f2e83400640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.754+0000 7f2e83400640 1 -- 172.21.15.62:0/1874921012 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e780aae60 con 0x7f2e8c15d810 2024-04-25T09:07:29.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.755+0000 7f2e8b400640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.756+0000 7f2e8b400640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c006fd0 tx=0x7f2e7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.759+0000 7f2e83400640 1 -- 172.21.15.62:0/1874921012 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e78032e20 con 0x7f2e8c15d810 2024-04-25T09:07:29.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.959+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e8c157880 con 0x7f2e8c15d810 2024-04-25T09:07:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.962+0000 7f2e83400640 1 -- 172.21.15.62:0/1874921012 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f2e78079c40 con 0x7f2e8c15d810 2024-04-25T09:07:29.950 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:29.950 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":48,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:07:26.990806+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:07:23.299693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T09:07:24.475140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e6c06b4d0 msgr2=0x7f2e6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c006fd0 tx=0x7f2e7c008040 comp rx=0 tx=0).stop 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15d810 msgr2=0x7f2e8c140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15d810 0x7f2e8c140730 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2e78035040 tx=0x7f2e780046c0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 shutdown_connections 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e8c15d810 0x7f2e8c140730 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e8c15dbf0 0x7f2e8c140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 --2- 172.21.15.62:0/1874921012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e8c0b0ec0 0x7f2e8c150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 >> 172.21.15.62:0/1874921012 conn(0x7f2e8c0b14d0 msgr2=0x7f2e8c15b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 shutdown_connections 2024-04-25T09:07:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:29.965+0000 7f2e91000640 1 -- 172.21.15.62:0/1874921012 wait complete. 2024-04-25T09:07:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:07:29.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:07:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.193+0000 7f60dfe00640 1 Processor -- start 2024-04-25T09:07:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.193+0000 7f60dfe00640 1 -- start start 2024-04-25T09:07:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.194+0000 7f60dfe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60e015dd70 0x7f60e00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.194+0000 7f60dfe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e00d0950 0x7f60e00b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dfe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e00d23b0 con 0x7f60e00d1e80 2024-04-25T09:07:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dfe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e00d2690 con 0x7f60e015dd70 2024-04-25T09:07:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dfe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e00d2970 con 0x7f60e00d0950 2024-04-25T09:07:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60e015dd70 0x7f60e00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60e015dd70 0x7f60e00cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33164/0 (socket says 172.21.15.62:33164) 2024-04-25T09:07:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dea00640 1 -- 172.21.15.62:0/583612550 learned_addr learned my addr 172.21.15.62:0/583612550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60de000640 1 --1- 172.21.15.62:0/583612550 >> v1:172.21.15.62:6789/0 conn(0x7f60e00d1e80 0x7f60e00d0240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44726/0 (socket says 172.21.15.62:44726) 2024-04-25T09:07:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dea00640 1 -- 172.21.15.62:0/583612550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e00d0950 msgr2=0x7f60e00b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dea00640 1 --2- 172.21.15.62:0/583612550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e00d0950 0x7f60e00b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dea00640 1 -- 172.21.15.62:0/583612550 >> v1:172.21.15.62:6789/0 conn(0x7f60e00d1e80 legacy=0x7f60e00d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.195+0000 7f60dea00640 1 -- 172.21.15.62:0/583612550 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e00d2c50 con 0x7f60e015dd70 2024-04-25T09:07:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.196+0000 7f60dea00640 1 --2- 172.21.15.62:0/583612550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e015dd70 0x7f60e00cec60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009e50 tx=0x7f60d002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0925b13ef28d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.197+0000 7f60dd600640 1 -- 172.21.15.62:0/583612550 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0036070 con 0x7f60e015dd70 2024-04-25T09:07:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.197+0000 7f60dd600640 1 -- 172.21.15.62:0/583612550 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d0031600 con 0x7f60e015dd70 2024-04-25T09:07:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.197+0000 7f60dd600640 1 -- 172.21.15.62:0/583612550 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00319d0 con 0x7f60e015dd70 2024-04-25T09:07:30.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.198+0000 7f60dfe00640 1 -- 172.21.15.62:0/583612550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e015dd70 msgr2=0x7f60e00cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.198+0000 7f60dfe00640 1 --2- 172.21.15.62:0/583612550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e015dd70 0x7f60e00cec60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009e50 tx=0x7f60d002a710 comp rx=0 tx=0).stop 2024-04-25T09:07:30.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.198+0000 7f60dfe00640 1 -- 172.21.15.62:0/583612550 shutdown_connections 2024-04-25T09:07:30.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.198+0000 7f60dfe00640 1 --2- 172.21.15.62:0/583612550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e00d0950 0x7f60e00b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.198+0000 7f60dfe00640 1 --2- 172.21.15.62:0/583612550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e015dd70 0x7f60e00cec60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.198+0000 7f60dfe00640 1 -- 172.21.15.62:0/583612550 >> 172.21.15.62:0/583612550 conn(0x7f60e006b210 msgr2=0x7f60e00668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:30.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.198+0000 7f60dfe00640 1 -- 172.21.15.62:0/583612550 shutdown_connections 2024-04-25T09:07:30.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.198+0000 7f60dfe00640 1 -- 172.21.15.62:0/583612550 wait complete. 2024-04-25T09:07:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.200+0000 7f60dfe00640 1 Processor -- start 2024-04-25T09:07:30.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.200+0000 7f60dfe00640 1 -- start start 2024-04-25T09:07:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e00d0950 0x7f60e00b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60e00d1e80 0x7f60e014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dfe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e015dd70 0x7f60e00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dfe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e00cf1a0 con 0x7f60e00d0950 2024-04-25T09:07:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dfe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e015e150 con 0x7f60e00d1e80 2024-04-25T09:07:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dfe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e00cf320 con 0x7f60e015dd70 2024-04-25T09:07:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e00d0950 0x7f60e00b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e00d0950 0x7f60e00b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33176/0 (socket says 172.21.15.62:33176) 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dea00640 1 -- 172.21.15.62:0/1969321974 learned_addr learned my addr 172.21.15.62:0/1969321974 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60de000640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60e00d1e80 0x7f60e014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dea00640 1 -- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e015dd70 msgr2=0x7f60e00cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60df400640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e015dd70 0x7f60e00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dea00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e015dd70 0x7f60e00cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dea00640 1 -- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60e00d1e80 msgr2=0x7f60e014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dea00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60e00d1e80 0x7f60e014be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.201+0000 7f60dea00640 1 -- 172.21.15.62:0/1969321974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e015f180 con 0x7f60e00d0950 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.202+0000 7f60de000640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60e00d1e80 0x7f60e014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:30.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.202+0000 7f60dea00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e00d0950 0x7f60e00b3da0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009e50 tx=0x7f60d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:30.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.202+0000 7f60d7400640 1 -- 172.21.15.62:0/1969321974 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0036050 con 0x7f60e00d0950 2024-04-25T09:07:30.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.202+0000 7f60d7400640 1 -- 172.21.15.62:0/1969321974 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d0042070 con 0x7f60e00d0950 2024-04-25T09:07:30.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.202+0000 7f60d7400640 1 -- 172.21.15.62:0/1969321974 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0009510 con 0x7f60e00d0950 2024-04-25T09:07:30.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.202+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e014c3c0 con 0x7f60e00d0950 2024-04-25T09:07:30.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.202+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e00b47b0 con 0x7f60e00d0950 2024-04-25T09:07:30.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.204+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e004ef20 con 0x7f60e00d0950 2024-04-25T09:07:30.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.205+0000 7f60d7400640 1 -- 172.21.15.62:0/1969321974 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60d0007c50 con 0x7f60e00d0950 2024-04-25T09:07:30.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.205+0000 7f60d7400640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f60bc06b500 0x7f60bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.206+0000 7f60d7400640 1 -- 172.21.15.62:0/1969321974 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60d00ab260 con 0x7f60e00d0950 2024-04-25T09:07:30.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.206+0000 7f60de000640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f60bc06b500 0x7f60bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.207+0000 7f60de000640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f60bc06b500 0x7f60bc06d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f60e00b80a0 tx=0x7f60c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.210+0000 7f60d7400640 1 -- 172.21.15.62:0/1969321974 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60d007a040 con 0x7f60e00d0950 2024-04-25T09:07:30.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.407+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60e00c7070 con 0x7f60bc06b500 2024-04-25T09:07:30.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.427+0000 7f60d7400640 1 -- 172.21.15.62:0/1969321974 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246639 (secure 0 0 0) 0x7f60e00c7070 con 0x7f60bc06b500 2024-04-25T09:07:30.418 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:30.420 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:07:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f60bc06b500 msgr2=0x7f60bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f60bc06b500 0x7f60bc06d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f60e00b80a0 tx=0x7f60c8009040 comp rx=0 tx=0).stop 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e00d0950 msgr2=0x7f60e00b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e00d0950 0x7f60e00b3da0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009e50 tx=0x7f60d0004a40 comp rx=0 tx=0).stop 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 shutdown_connections 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f60bc06b500 0x7f60bc06d9c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60e015dd70 0x7f60e00cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60e00d1e80 0x7f60e014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 --2- 172.21.15.62:0/1969321974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60e00d0950 0x7f60e00b3da0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.432+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 >> 172.21.15.62:0/1969321974 conn(0x7f60e006b210 msgr2=0x7f60e00b8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.433+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 shutdown_connections 2024-04-25T09:07:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.433+0000 7f60dfe00640 1 -- 172.21.15.62:0/1969321974 wait complete. 2024-04-25T09:07:30.439 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-25T09:07:29.983331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300592,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754674128,"statfs":{"total":773094113280,"available":772786307072,"internally_reserved":0,"allocated":5513216,"data_stored":2951216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.006202"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996639+0000","last_change":"2024-04-25T09:07:24.996639+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996639+0000","last_undegraded":"2024-04-25T09:07:24.996639+0000","last_fullsized":"2024-04-25T09:07:24.996639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990524+0000","last_change":"2024-04-25T09:07:24.990524+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990524+0000","last_undegraded":"2024-04-25T09:07:24.990524+0000","last_fullsized":"2024-04-25T09:07:24.990524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995074+0000","last_change":"2024-04-25T09:07:24.995074+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995074+0000","last_undegraded":"2024-04-25T09:07:24.995074+0000","last_fullsized":"2024-04-25T09:07:24.995074+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997350+0000","last_change":"2024-04-25T09:07:24.997350+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997350+0000","last_undegraded":"2024-04-25T09:07:24.997350+0000","last_fullsized":"2024-04-25T09:07:24.997350+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991559+0000","last_change":"2024-04-25T09:07:24.991559+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991559+0000","last_undegraded":"2024-04-25T09:07:24.991559+0000","last_fullsized":"2024-04-25T09:07:24.991559+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.002084+0000","last_change":"2024-04-25T09:07:25.002084+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.002084+0000","last_undegraded":"2024-04-25T09:07:25.002084+0000","last_fullsized":"2024-04-25T09:07:25.002084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990527+0000","last_change":"2024-04-25T09:07:24.990527+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990527+0000","last_undegraded":"2024-04-25T09:07:24.990527+0000","last_fullsized":"2024-04-25T09:07:24.990527+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998301+0000","last_change":"2024-04-25T09:07:24.998301+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998301+0000","last_undegraded":"2024-04-25T09:07:24.998301+0000","last_fullsized":"2024-04-25T09:07:24.998301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997181+0000","last_change":"2024-04-25T09:07:24.997181+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997181+0000","last_undegraded":"2024-04-25T09:07:24.997181+0000","last_fullsized":"2024-04-25T09:07:24.997181+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991564+0000","last_change":"2024-04-25T09:07:24.991564+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991564+0000","last_undegraded":"2024-04-25T09:07:24.991564+0000","last_fullsized":"2024-04-25T09:07:24.991564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.005646+0000","last_change":"2024-04-25T09:07:25.005646+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.005646+0000","last_undegraded":"2024-04-25T09:07:25.005646+0000","last_fullsized":"2024-04-25T09:07:25.005646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997243+0000","last_change":"2024-04-25T09:07:24.997243+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997243+0000","last_undegraded":"2024-04-25T09:07:24.997243+0000","last_fullsized":"2024-04-25T09:07:24.997243+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.002332+0000","last_change":"2024-04-25T09:07:25.002332+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.002332+0000","last_undegraded":"2024-04-25T09:07:25.002332+0000","last_fullsized":"2024-04-25T09:07:25.002332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994369+0000","last_change":"2024-04-25T09:07:24.994369+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994369+0000","last_undegraded":"2024-04-25T09:07:24.994369+0000","last_fullsized":"2024-04-25T09:07:24.994369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998804+0000","last_change":"2024-04-25T09:07:24.998804+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998804+0000","last_undegraded":"2024-04-25T09:07:24.998804+0000","last_fullsized":"2024-04-25T09:07:24.998804+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987193+0000","last_change":"2024-04-25T09:07:24.987193+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987193+0000","last_undegraded":"2024-04-25T09:07:24.987193+0000","last_fullsized":"2024-04-25T09:07:24.987193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991010+0000","last_change":"2024-04-25T09:07:24.991010+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991010+0000","last_undegraded":"2024-04-25T09:07:24.991010+0000","last_fullsized":"2024-04-25T09:07:24.991010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998751+0000","last_change":"2024-04-25T09:07:24.998751+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998751+0000","last_undegraded":"2024-04-25T09:07:24.998751+0000","last_fullsized":"2024-04-25T09:07:24.998751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989327+0000","last_change":"2024-04-25T09:07:24.989327+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989327+0000","last_undegraded":"2024-04-25T09:07:24.989327+0000","last_fullsized":"2024-04-25T09:07:24.989327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993017+0000","last_change":"2024-04-25T09:07:24.993017+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993017+0000","last_undegraded":"2024-04-25T09:07:24.993017+0000","last_fullsized":"2024-04-25T09:07:24.993017+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.003843+0000","last_change":"2024-04-25T09:07:25.003843+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.003843+0000","last_undegraded":"2024-04-25T09:07:25.003843+0000","last_fullsized":"2024-04-25T09:07:25.003843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987628+0000","last_change":"2024-04-25T09:07:24.987628+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987628+0000","last_undegraded":"2024-04-25T09:07:24.987628+0000","last_fullsized":"2024-04-25T09:07:24.987628+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987608+0000","last_change":"2024-04-25T09:07:24.987608+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987608+0000","last_undegraded":"2024-04-25T09:07:24.987608+0000","last_fullsized":"2024-04-25T09:07:24.987608+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989974+0000","last_change":"2024-04-25T09:07:24.989974+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989974+0000","last_undegraded":"2024-04-25T09:07:24.989974+0000","last_fullsized":"2024-04-25T09:07:24.989974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995869+0000","last_change":"2024-04-25T09:07:24.995869+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995869+0000","last_undegraded":"2024-04-25T09:07:24.995869+0000","last_fullsized":"2024-04-25T09:07:24.995869+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.007025+0000","last_change":"2024-04-25T09:07:25.007025+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.007025+0000","last_undegraded":"2024-04-25T09:07:25.007025+0000","last_fullsized":"2024-04-25T09:07:25.007025+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993999+0000","last_change":"2024-04-25T09:07:24.993999+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993999+0000","last_undegraded":"2024-04-25T09:07:24.993999+0000","last_fullsized":"2024-04-25T09:07:24.993999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996751+0000","last_change":"2024-04-25T09:07:24.996751+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996751+0000","last_undegraded":"2024-04-25T09:07:24.996751+0000","last_fullsized":"2024-04-25T09:07:24.996751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990286+0000","last_change":"2024-04-25T09:07:24.990286+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990286+0000","last_undegraded":"2024-04-25T09:07:24.990286+0000","last_fullsized":"2024-04-25T09:07:24.990286+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990066+0000","last_change":"2024-04-25T09:07:24.990066+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990066+0000","last_undegraded":"2024-04-25T09:07:24.990066+0000","last_fullsized":"2024-04-25T09:07:24.990066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996794+0000","last_change":"2024-04-25T09:07:24.996794+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996794+0000","last_undegraded":"2024-04-25T09:07:24.996794+0000","last_fullsized":"2024-04-25T09:07:24.996794+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997739+0000","last_change":"2024-04-25T09:07:24.997739+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997739+0000","last_undegraded":"2024-04-25T09:07:24.997739+0000","last_fullsized":"2024-04-25T09:07:24.997739+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987393+0000","last_change":"2024-04-25T09:07:24.987393+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987393+0000","last_undegraded":"2024-04-25T09:07:24.987393+0000","last_fullsized":"2024-04-25T09:07:24.987393+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013200+0000","last_change":"2024-04-25T09:07:25.010479+0000","last_active":"2024-04-25T09:07:25.013200+0000","last_peered":"2024-04-25T09:07:25.013200+0000","last_clean":"2024-04-25T09:07:25.013200+0000","last_became_active":"2024-04-25T09:07:25.009971+0000","last_became_peered":"2024-04-25T09:07:25.009971+0000","last_unstale":"2024-04-25T09:07:25.013200+0000","last_undegraded":"2024-04-25T09:07:25.013200+0000","last_fullsized":"2024-04-25T09:07:25.013200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:58.222873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995032+0000","last_change":"2024-04-25T09:07:24.995032+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995032+0000","last_undegraded":"2024-04-25T09:07:24.995032+0000","last_fullsized":"2024-04-25T09:07:24.995032+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017081+0000","last_change":"2024-04-25T09:07:25.012693+0000","last_active":"2024-04-25T09:07:25.017081+0000","last_peered":"2024-04-25T09:07:25.017081+0000","last_clean":"2024-04-25T09:07:25.017081+0000","last_became_active":"2024-04-25T09:07:25.012366+0000","last_became_peered":"2024-04-25T09:07:25.012366+0000","last_unstale":"2024-04-25T09:07:25.017081+0000","last_undegraded":"2024-04-25T09:07:25.017081+0000","last_fullsized":"2024-04-25T09:07:25.017081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:42.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996781+0000","last_change":"2024-04-25T09:07:24.996781+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996781+0000","last_undegraded":"2024-04-25T09:07:24.996781+0000","last_fullsized":"2024-04-25T09:07:24.996781+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013434+0000","last_change":"2024-04-25T09:07:25.011946+0000","last_active":"2024-04-25T09:07:25.013434+0000","last_peered":"2024-04-25T09:07:25.013434+0000","last_clean":"2024-04-25T09:07:25.013434+0000","last_became_active":"2024-04-25T09:07:25.011630+0000","last_became_peered":"2024-04-25T09:07:25.011630+0000","last_unstale":"2024-04-25T09:07:25.013434+0000","last_undegraded":"2024-04-25T09:07:25.013434+0000","last_fullsized":"2024-04-25T09:07:25.013434+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:45.527811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994342+0000","last_change":"2024-04-25T09:07:24.994342+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994342+0000","last_undegraded":"2024-04-25T09:07:24.994342+0000","last_fullsized":"2024-04-25T09:07:24.994342+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.014473+0000","last_change":"2024-04-25T09:07:25.010814+0000","last_active":"2024-04-25T09:07:25.014473+0000","last_peered":"2024-04-25T09:07:25.014473+0000","last_clean":"2024-04-25T09:07:25.014473+0000","last_became_active":"2024-04-25T09:07:25.008979+0000","last_became_peered":"2024-04-25T09:07:25.008979+0000","last_unstale":"2024-04-25T09:07:25.014473+0000","last_undegraded":"2024-04-25T09:07:25.014473+0000","last_fullsized":"2024-04-25T09:07:25.014473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:50.056920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992079+0000","last_change":"2024-04-25T09:07:24.992079+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992079+0000","last_undegraded":"2024-04-25T09:07:24.992079+0000","last_fullsized":"2024-04-25T09:07:24.992079+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013153+0000","last_change":"2024-04-25T09:07:25.010295+0000","last_active":"2024-04-25T09:07:25.013153+0000","last_peered":"2024-04-25T09:07:25.013153+0000","last_clean":"2024-04-25T09:07:25.013153+0000","last_became_active":"2024-04-25T09:07:25.009984+0000","last_became_peered":"2024-04-25T09:07:25.009984+0000","last_unstale":"2024-04-25T09:07:25.013153+0000","last_undegraded":"2024-04-25T09:07:25.013153+0000","last_fullsized":"2024-04-25T09:07:25.013153+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:35.643627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.999831+0000","last_change":"2024-04-25T09:07:24.999831+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.999831+0000","last_undegraded":"2024-04-25T09:07:24.999831+0000","last_fullsized":"2024-04-25T09:07:24.999831+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010479+0000","last_change":"2024-04-25T09:07:25.005618+0000","last_active":"2024-04-25T09:07:25.010479+0000","last_peered":"2024-04-25T09:07:25.010479+0000","last_clean":"2024-04-25T09:07:25.010479+0000","last_became_active":"2024-04-25T09:07:25.005276+0000","last_became_peered":"2024-04-25T09:07:25.005276+0000","last_unstale":"2024-04-25T09:07:25.010479+0000","last_undegraded":"2024-04-25T09:07:25.010479+0000","last_fullsized":"2024-04-25T09:07:25.010479+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:39.155136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991651+0000","last_change":"2024-04-25T09:07:24.991651+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991651+0000","last_undegraded":"2024-04-25T09:07:24.991651+0000","last_fullsized":"2024-04-25T09:07:24.991651+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017467+0000","last_change":"2024-04-25T09:07:25.014784+0000","last_active":"2024-04-25T09:07:25.017467+0000","last_peered":"2024-04-25T09:07:25.017467+0000","last_clean":"2024-04-25T09:07:25.017467+0000","last_became_active":"2024-04-25T09:07:25.012907+0000","last_became_peered":"2024-04-25T09:07:25.012907+0000","last_unstale":"2024-04-25T09:07:25.017467+0000","last_undegraded":"2024-04-25T09:07:25.017467+0000","last_fullsized":"2024-04-25T09:07:25.017467+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.030750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.988372+0000","last_change":"2024-04-25T09:07:24.988372+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.988372+0000","last_undegraded":"2024-04-25T09:07:24.988372+0000","last_fullsized":"2024-04-25T09:07:24.988372+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015362+0000","last_change":"2024-04-25T09:07:25.009763+0000","last_active":"2024-04-25T09:07:25.015362+0000","last_peered":"2024-04-25T09:07:25.015362+0000","last_clean":"2024-04-25T09:07:25.015362+0000","last_became_active":"2024-04-25T09:07:25.009474+0000","last_became_peered":"2024-04-25T09:07:25.009474+0000","last_unstale":"2024-04-25T09:07:25.015362+0000","last_undegraded":"2024-04-25T09:07:25.015362+0000","last_fullsized":"2024-04-25T09:07:25.015362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:51.123149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996639+0000","last_change":"2024-04-25T09:07:24.996639+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996639+0000","last_undegraded":"2024-04-25T09:07:24.996639+0000","last_fullsized":"2024-04-25T09:07:24.996639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015127+0000","last_change":"2024-04-25T09:07:25.007040+0000","last_active":"2024-04-25T09:07:25.015127+0000","last_peered":"2024-04-25T09:07:25.015127+0000","last_clean":"2024-04-25T09:07:25.015127+0000","last_became_active":"2024-04-25T09:07:25.006742+0000","last_became_peered":"2024-04-25T09:07:25.006742+0000","last_unstale":"2024-04-25T09:07:25.015127+0000","last_undegraded":"2024-04-25T09:07:25.015127+0000","last_fullsized":"2024-04-25T09:07:25.015127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:07.492703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993569+0000","last_change":"2024-04-25T09:07:24.993569+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993569+0000","last_undegraded":"2024-04-25T09:07:24.993569+0000","last_fullsized":"2024-04-25T09:07:24.993569+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018281+0000","last_change":"2024-04-25T09:07:25.016674+0000","last_active":"2024-04-25T09:07:25.018281+0000","last_peered":"2024-04-25T09:07:25.018281+0000","last_clean":"2024-04-25T09:07:25.018281+0000","last_became_active":"2024-04-25T09:07:25.016317+0000","last_became_peered":"2024-04-25T09:07:25.016317+0000","last_unstale":"2024-04-25T09:07:25.018281+0000","last_undegraded":"2024-04-25T09:07:25.018281+0000","last_fullsized":"2024-04-25T09:07:25.018281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:19.859538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996661+0000","last_change":"2024-04-25T09:07:24.996661+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996661+0000","last_undegraded":"2024-04-25T09:07:24.996661+0000","last_fullsized":"2024-04-25T09:07:24.996661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013680+0000","last_change":"2024-04-25T09:07:25.008303+0000","last_active":"2024-04-25T09:07:25.013680+0000","last_peered":"2024-04-25T09:07:25.013680+0000","last_clean":"2024-04-25T09:07:25.013680+0000","last_became_active":"2024-04-25T09:07:25.006381+0000","last_became_peered":"2024-04-25T09:07:25.006381+0000","last_unstale":"2024-04-25T09:07:25.013680+0000","last_undegraded":"2024-04-25T09:07:25.013680+0000","last_fullsized":"2024-04-25T09:07:25.013680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993591+0000","last_change":"2024-04-25T09:07:24.993591+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993591+0000","last_undegraded":"2024-04-25T09:07:24.993591+0000","last_fullsized":"2024-04-25T09:07:24.993591+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007328+0000","last_change":"2024-04-25T09:07:25.003604+0000","last_active":"2024-04-25T09:07:25.007328+0000","last_peered":"2024-04-25T09:07:25.007328+0000","last_clean":"2024-04-25T09:07:25.007328+0000","last_became_active":"2024-04-25T09:07:25.003266+0000","last_became_peered":"2024-04-25T09:07:25.003266+0000","last_unstale":"2024-04-25T09:07:25.007328+0000","last_undegraded":"2024-04-25T09:07:25.007328+0000","last_fullsized":"2024-04-25T09:07:25.007328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998031+0000","last_change":"2024-04-25T09:07:24.998031+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998031+0000","last_undegraded":"2024-04-25T09:07:24.998031+0000","last_fullsized":"2024-04-25T09:07:24.998031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010140+0000","last_change":"2024-04-25T09:07:25.006123+0000","last_active":"2024-04-25T09:07:25.010140+0000","last_peered":"2024-04-25T09:07:25.010140+0000","last_clean":"2024-04-25T09:07:25.010140+0000","last_became_active":"2024-04-25T09:07:25.004723+0000","last_became_peered":"2024-04-25T09:07:25.004723+0000","last_unstale":"2024-04-25T09:07:25.010140+0000","last_undegraded":"2024-04-25T09:07:25.010140+0000","last_fullsized":"2024-04-25T09:07:25.010140+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:07.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986042+0000","last_change":"2024-04-25T09:07:24.986042+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986042+0000","last_undegraded":"2024-04-25T09:07:24.986042+0000","last_fullsized":"2024-04-25T09:07:24.986042+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007877+0000","last_change":"2024-04-25T09:07:25.004430+0000","last_active":"2024-04-25T09:07:25.007877+0000","last_peered":"2024-04-25T09:07:25.007877+0000","last_clean":"2024-04-25T09:07:25.007877+0000","last_became_active":"2024-04-25T09:07:25.000012+0000","last_became_peered":"2024-04-25T09:07:25.000012+0000","last_unstale":"2024-04-25T09:07:25.007877+0000","last_undegraded":"2024-04-25T09:07:25.007877+0000","last_fullsized":"2024-04-25T09:07:25.007877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:13.587386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.985571+0000","last_change":"2024-04-25T09:07:24.985571+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.985571+0000","last_undegraded":"2024-04-25T09:07:24.985571+0000","last_fullsized":"2024-04-25T09:07:24.985571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017723+0000","last_change":"2024-04-25T09:07:25.013988+0000","last_active":"2024-04-25T09:07:25.017723+0000","last_peered":"2024-04-25T09:07:25.017723+0000","last_clean":"2024-04-25T09:07:25.017723+0000","last_became_active":"2024-04-25T09:07:25.013681+0000","last_became_peered":"2024-04-25T09:07:25.013681+0000","last_unstale":"2024-04-25T09:07:25.017723+0000","last_undegraded":"2024-04-25T09:07:25.017723+0000","last_fullsized":"2024-04-25T09:07:25.017723+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.565341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998624+0000","last_change":"2024-04-25T09:07:24.998624+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998624+0000","last_undegraded":"2024-04-25T09:07:24.998624+0000","last_fullsized":"2024-04-25T09:07:24.998624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010072+0000","last_change":"2024-04-25T09:07:25.006626+0000","last_active":"2024-04-25T09:07:25.010072+0000","last_peered":"2024-04-25T09:07:25.010072+0000","last_clean":"2024-04-25T09:07:25.010072+0000","last_became_active":"2024-04-25T09:07:25.006323+0000","last_became_peered":"2024-04-25T09:07:25.006323+0000","last_unstale":"2024-04-25T09:07:25.010072+0000","last_undegraded":"2024-04-25T09:07:25.010072+0000","last_fullsized":"2024-04-25T09:07:25.010072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:36.534602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986282+0000","last_change":"2024-04-25T09:07:24.986282+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986282+0000","last_undegraded":"2024-04-25T09:07:24.986282+0000","last_fullsized":"2024-04-25T09:07:24.986282+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018288+0000","last_change":"2024-04-25T09:07:25.016157+0000","last_active":"2024-04-25T09:07:25.018288+0000","last_peered":"2024-04-25T09:07:25.018288+0000","last_clean":"2024-04-25T09:07:25.018288+0000","last_became_active":"2024-04-25T09:07:25.015731+0000","last_became_peered":"2024-04-25T09:07:25.015731+0000","last_unstale":"2024-04-25T09:07:25.018288+0000","last_undegraded":"2024-04-25T09:07:25.018288+0000","last_fullsized":"2024-04-25T09:07:25.018288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:30.570535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986852+0000","last_change":"2024-04-25T09:07:24.986852+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986852+0000","last_undegraded":"2024-04-25T09:07:24.986852+0000","last_fullsized":"2024-04-25T09:07:24.986852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015226+0000","last_change":"2024-04-25T09:07:25.010677+0000","last_active":"2024-04-25T09:07:25.015226+0000","last_peered":"2024-04-25T09:07:25.015226+0000","last_clean":"2024-04-25T09:07:25.015226+0000","last_became_active":"2024-04-25T09:07:25.010337+0000","last_became_peered":"2024-04-25T09:07:25.010337+0000","last_unstale":"2024-04-25T09:07:25.015226+0000","last_undegraded":"2024-04-25T09:07:25.015226+0000","last_fullsized":"2024-04-25T09:07:25.015226+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:37.745468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992003+0000","last_change":"2024-04-25T09:07:24.992003+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992003+0000","last_undegraded":"2024-04-25T09:07:24.992003+0000","last_fullsized":"2024-04-25T09:07:24.992003+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015417+0000","last_change":"2024-04-25T09:07:25.012576+0000","last_active":"2024-04-25T09:07:25.015417+0000","last_peered":"2024-04-25T09:07:25.015417+0000","last_clean":"2024-04-25T09:07:25.015417+0000","last_became_active":"2024-04-25T09:07:25.010949+0000","last_became_peered":"2024-04-25T09:07:25.010949+0000","last_unstale":"2024-04-25T09:07:25.015417+0000","last_undegraded":"2024-04-25T09:07:25.015417+0000","last_fullsized":"2024-04-25T09:07:25.015417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:07.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994462+0000","last_change":"2024-04-25T09:07:24.994462+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994462+0000","last_undegraded":"2024-04-25T09:07:24.994462+0000","last_fullsized":"2024-04-25T09:07:24.994462+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017671+0000","last_change":"2024-04-25T09:07:25.013972+0000","last_active":"2024-04-25T09:07:25.017671+0000","last_peered":"2024-04-25T09:07:25.017671+0000","last_clean":"2024-04-25T09:07:25.017671+0000","last_became_active":"2024-04-25T09:07:25.013633+0000","last_became_peered":"2024-04-25T09:07:25.013633+0000","last_unstale":"2024-04-25T09:07:25.017671+0000","last_undegraded":"2024-04-25T09:07:25.017671+0000","last_fullsized":"2024-04-25T09:07:25.017671+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:47.797851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993010+0000","last_change":"2024-04-25T09:07:24.993010+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993010+0000","last_undegraded":"2024-04-25T09:07:24.993010+0000","last_fullsized":"2024-04-25T09:07:24.993010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018264+0000","last_change":"2024-04-25T09:07:25.016649+0000","last_active":"2024-04-25T09:07:25.018264+0000","last_peered":"2024-04-25T09:07:25.018264+0000","last_clean":"2024-04-25T09:07:25.018264+0000","last_became_active":"2024-04-25T09:07:25.016387+0000","last_became_peered":"2024-04-25T09:07:25.016387+0000","last_unstale":"2024-04-25T09:07:25.018264+0000","last_undegraded":"2024-04-25T09:07:25.018264+0000","last_fullsized":"2024-04-25T09:07:25.018264+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:09.601456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998397+0000","last_change":"2024-04-25T09:07:24.998397+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998397+0000","last_undegraded":"2024-04-25T09:07:24.998397+0000","last_fullsized":"2024-04-25T09:07:24.998397+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015086+0000","last_change":"2024-04-25T09:07:25.009598+0000","last_active":"2024-04-25T09:07:25.015086+0000","last_peered":"2024-04-25T09:07:25.015086+0000","last_clean":"2024-04-25T09:07:25.015086+0000","last_became_active":"2024-04-25T09:07:25.009294+0000","last_became_peered":"2024-04-25T09:07:25.009294+0000","last_unstale":"2024-04-25T09:07:25.015086+0000","last_undegraded":"2024-04-25T09:07:25.015086+0000","last_fullsized":"2024-04-25T09:07:25.015086+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:31.801129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997245+0000","last_change":"2024-04-25T09:07:24.997245+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997245+0000","last_undegraded":"2024-04-25T09:07:24.997245+0000","last_fullsized":"2024-04-25T09:07:24.997245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007711+0000","last_change":"2024-04-25T09:07:25.005217+0000","last_active":"2024-04-25T09:07:25.007711+0000","last_peered":"2024-04-25T09:07:25.007711+0000","last_clean":"2024-04-25T09:07:25.007711+0000","last_became_active":"2024-04-25T09:07:25.003714+0000","last_became_peered":"2024-04-25T09:07:25.003714+0000","last_unstale":"2024-04-25T09:07:25.007711+0000","last_undegraded":"2024-04-25T09:07:25.007711+0000","last_fullsized":"2024-04-25T09:07:25.007711+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:55.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998226+0000","last_change":"2024-04-25T09:07:24.998226+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998226+0000","last_undegraded":"2024-04-25T09:07:24.998226+0000","last_fullsized":"2024-04-25T09:07:24.998226+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.005589+0000","last_change":"2024-04-25T09:07:25.003186+0000","last_active":"2024-04-25T09:07:25.005589+0000","last_peered":"2024-04-25T09:07:25.005589+0000","last_clean":"2024-04-25T09:07:25.005589+0000","last_became_active":"2024-04-25T09:07:25.002435+0000","last_became_peered":"2024-04-25T09:07:25.002435+0000","last_unstale":"2024-04-25T09:07:25.005589+0000","last_undegraded":"2024-04-25T09:07:25.005589+0000","last_fullsized":"2024-04-25T09:07:25.005589+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:22.212413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994941+0000","last_change":"2024-04-25T09:07:24.994941+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994941+0000","last_undegraded":"2024-04-25T09:07:24.994941+0000","last_fullsized":"2024-04-25T09:07:24.994941+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018229+0000","last_change":"2024-04-25T09:07:25.015721+0000","last_active":"2024-04-25T09:07:25.018229+0000","last_peered":"2024-04-25T09:07:25.018229+0000","last_clean":"2024-04-25T09:07:25.018229+0000","last_became_active":"2024-04-25T09:07:25.015220+0000","last_became_peered":"2024-04-25T09:07:25.015220+0000","last_unstale":"2024-04-25T09:07:25.018229+0000","last_undegraded":"2024-04-25T09:07:25.018229+0000","last_fullsized":"2024-04-25T09:07:25.018229+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:31.801129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991242+0000","last_change":"2024-04-25T09:07:24.991242+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991242+0000","last_undegraded":"2024-04-25T09:07:24.991242+0000","last_fullsized":"2024-04-25T09:07:24.991242+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015029+0000","last_change":"2024-04-25T09:07:25.010776+0000","last_active":"2024-04-25T09:07:25.015029+0000","last_peered":"2024-04-25T09:07:25.015029+0000","last_clean":"2024-04-25T09:07:25.015029+0000","last_became_active":"2024-04-25T09:07:25.010280+0000","last_became_peered":"2024-04-25T09:07:25.010280+0000","last_unstale":"2024-04-25T09:07:25.015029+0000","last_undegraded":"2024-04-25T09:07:25.015029+0000","last_fullsized":"2024-04-25T09:07:25.015029+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:30.570535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993158+0000","last_change":"2024-04-25T09:07:24.993158+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993158+0000","last_undegraded":"2024-04-25T09:07:24.993158+0000","last_fullsized":"2024-04-25T09:07:24.993158+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.012361+0000","last_change":"2024-04-25T09:07:25.007229+0000","last_active":"2024-04-25T09:07:25.012361+0000","last_peered":"2024-04-25T09:07:25.012361+0000","last_clean":"2024-04-25T09:07:25.012361+0000","last_became_active":"2024-04-25T09:07:25.006847+0000","last_became_peered":"2024-04-25T09:07:25.006847+0000","last_unstale":"2024-04-25T09:07:25.012361+0000","last_undegraded":"2024-04-25T09:07:25.012361+0000","last_fullsized":"2024-04-25T09:07:25.012361+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:26.558822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989296+0000","last_change":"2024-04-25T09:07:24.989296+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989296+0000","last_undegraded":"2024-04-25T09:07:24.989296+0000","last_fullsized":"2024-04-25T09:07:24.989296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.016762+0000","last_change":"2024-04-25T09:07:25.011689+0000","last_active":"2024-04-25T09:07:25.016762+0000","last_peered":"2024-04-25T09:07:25.016762+0000","last_clean":"2024-04-25T09:07:25.016762+0000","last_became_active":"2024-04-25T09:07:25.011388+0000","last_became_peered":"2024-04-25T09:07:25.011388+0000","last_unstale":"2024-04-25T09:07:25.016762+0000","last_undegraded":"2024-04-25T09:07:25.016762+0000","last_fullsized":"2024-04-25T09:07:25.016762+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:31.406996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.007613+0000","last_change":"2024-04-25T09:07:25.007613+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.007613+0000","last_undegraded":"2024-04-25T09:07:25.007613+0000","last_fullsized":"2024-04-25T09:07:25.007613+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.005467+0000","last_change":"2024-04-25T09:07:25.004275+0000","last_active":"2024-04-25T09:07:25.005467+0000","last_peered":"2024-04-25T09:07:25.005467+0000","last_clean":"2024-04-25T09:07:25.005467+0000","last_became_active":"2024-04-25T09:07:25.004002+0000","last_became_peered":"2024-04-25T09:07:25.004002+0000","last_unstale":"2024-04-25T09:07:25.005467+0000","last_undegraded":"2024-04-25T09:07:25.005467+0000","last_fullsized":"2024-04-25T09:07:25.005467+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:33.025712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993341+0000","last_change":"2024-04-25T09:07:24.993341+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993341+0000","last_undegraded":"2024-04-25T09:07:24.993341+0000","last_fullsized":"2024-04-25T09:07:24.993341+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013092+0000","last_change":"2024-04-25T09:07:25.007837+0000","last_active":"2024-04-25T09:07:25.013092+0000","last_peered":"2024-04-25T09:07:25.013092+0000","last_clean":"2024-04-25T09:07:25.013092+0000","last_became_active":"2024-04-25T09:07:25.007537+0000","last_became_peered":"2024-04-25T09:07:25.007537+0000","last_unstale":"2024-04-25T09:07:25.013092+0000","last_undegraded":"2024-04-25T09:07:25.013092+0000","last_fullsized":"2024-04-25T09:07:25.013092+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.053815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992440+0000","last_change":"2024-04-25T09:07:24.992440+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992440+0000","last_undegraded":"2024-04-25T09:07:24.992440+0000","last_fullsized":"2024-04-25T09:07:24.992440+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.006431+0000","last_change":"2024-04-25T09:07:25.002676+0000","last_active":"2024-04-25T09:07:25.006431+0000","last_peered":"2024-04-25T09:07:25.006431+0000","last_clean":"2024-04-25T09:07:25.006431+0000","last_became_active":"2024-04-25T09:07:25.001512+0000","last_became_peered":"2024-04-25T09:07:25.001512+0000","last_unstale":"2024-04-25T09:07:25.006431+0000","last_undegraded":"2024-04-25T09:07:25.006431+0000","last_fullsized":"2024-04-25T09:07:25.006431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:37.745468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.004957+0000","last_change":"2024-04-25T09:07:25.004957+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.004957+0000","last_undegraded":"2024-04-25T09:07:25.004957+0000","last_fullsized":"2024-04-25T09:07:25.004957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.019379+0000","last_change":"2024-04-25T09:07:25.014785+0000","last_active":"2024-04-25T09:07:25.019379+0000","last_peered":"2024-04-25T09:07:25.019379+0000","last_clean":"2024-04-25T09:07:25.019379+0000","last_became_active":"2024-04-25T09:07:25.013321+0000","last_became_peered":"2024-04-25T09:07:25.013321+0000","last_unstale":"2024-04-25T09:07:25.019379+0000","last_undegraded":"2024-04-25T09:07:25.019379+0000","last_fullsized":"2024-04-25T09:07:25.019379+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:09.368176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.004963+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:07:25.004963+0000","last_peered":"2024-04-25T09:07:25.004963+0000","last_clean":"2024-04-25T09:07:25.004963+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:07:25.004963+0000","last_undegraded":"2024-04-25T09:07:25.004963+0000","last_fullsized":"2024-04-25T09:07:25.004963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1134592,"data_stored":811678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1134592,"data_stored":811678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:07:30.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:07:30.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.711+0000 7f038f200640 1 Processor -- start 2024-04-25T09:07:30.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.712+0000 7f038f200640 1 -- start start 2024-04-25T09:07:30.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.713+0000 7f038f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0390144c10 0x7f039015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.713+0000 7f038f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03900cee40 0x7f039006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.713+0000 7f038f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f039006d730 0x7f03900b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.713+0000 7f038f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0390144ff0 con 0x7f039006d730 2024-04-25T09:07:30.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.713+0000 7f038f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0390160340 con 0x7f0390144c10 2024-04-25T09:07:30.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.713+0000 7f038f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03900b41c0 con 0x7f03900cee40 2024-04-25T09:07:30.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f039006d730 0x7f03900b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f039006d730 0x7f03900b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33186/0 (socket says 172.21.15.62:33186) 2024-04-25T09:07:30.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038e800640 1 -- 172.21.15.62:0/2236574883 learned_addr learned my addr 172.21.15.62:0/2236574883 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:30.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038de00640 1 --2- 172.21.15.62:0/2236574883 >> v2:172.21.15.121:3300/0 conn(0x7f0390144c10 0x7f039015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f0385400640 1 --2- 172.21.15.62:0/2236574883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03900cee40 0x7f039006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038e800640 1 -- 172.21.15.62:0/2236574883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03900cee40 msgr2=0x7f039006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:30.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038e800640 1 --2- 172.21.15.62:0/2236574883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03900cee40 0x7f039006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038e800640 1 -- 172.21.15.62:0/2236574883 >> v2:172.21.15.121:3300/0 conn(0x7f0390144c10 msgr2=0x7f039015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:30.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038e800640 1 --2- 172.21.15.62:0/2236574883 >> v2:172.21.15.121:3300/0 conn(0x7f0390144c10 0x7f039015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038e800640 1 -- 172.21.15.62:0/2236574883 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039015faa0 con 0x7f039006d730 2024-04-25T09:07:30.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.714+0000 7f038de00640 1 --2- 172.21.15.62:0/2236574883 >> v2:172.21.15.121:3300/0 conn(0x7f0390144c10 0x7f039015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:30.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.715+0000 7f038e800640 1 --2- 172.21.15.62:0/2236574883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f039006d730 0x7f03900b2da0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f03900724a0 tx=0x7f038802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ede2c5e7d04d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:30.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.716+0000 7f038d400640 1 -- 172.21.15.62:0/2236574883 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388036070 con 0x7f039006d730 2024-04-25T09:07:30.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.716+0000 7f038d400640 1 -- 172.21.15.62:0/2236574883 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0388031600 con 0x7f039006d730 2024-04-25T09:07:30.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.716+0000 7f038d400640 1 -- 172.21.15.62:0/2236574883 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03880319d0 con 0x7f039006d730 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.716+0000 7f038f200640 1 -- 172.21.15.62:0/2236574883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f039006d730 msgr2=0x7f03900b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.716+0000 7f038f200640 1 --2- 172.21.15.62:0/2236574883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f039006d730 0x7f03900b2da0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f03900724a0 tx=0x7f038802a610 comp rx=0 tx=0).stop 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.717+0000 7f038f200640 1 -- 172.21.15.62:0/2236574883 shutdown_connections 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.717+0000 7f038f200640 1 --2- 172.21.15.62:0/2236574883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f039006d730 0x7f03900b2da0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.717+0000 7f038f200640 1 --2- 172.21.15.62:0/2236574883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03900cee40 0x7f039006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.717+0000 7f038f200640 1 --2- 172.21.15.62:0/2236574883 >> v2:172.21.15.121:3300/0 conn(0x7f0390144c10 0x7f039015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.717+0000 7f038f200640 1 -- 172.21.15.62:0/2236574883 >> 172.21.15.62:0/2236574883 conn(0x7f03900b14d0 msgr2=0x7f03900b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.717+0000 7f038f200640 1 -- 172.21.15.62:0/2236574883 shutdown_connections 2024-04-25T09:07:30.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.717+0000 7f038f200640 1 -- 172.21.15.62:0/2236574883 wait complete. 2024-04-25T09:07:30.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.718+0000 7f038f200640 1 Processor -- start 2024-04-25T09:07:30.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.719+0000 7f038f200640 1 -- start start 2024-04-25T09:07:30.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f039006d730 0x7f03901f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03900cee40 0x7f03901f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0390144c10 0x7f039012fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03900cf2a0 con 0x7f03900cee40 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390160060 con 0x7f0390144c10 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03900b32e0 con 0x7f039006d730 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f0385400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03900cee40 0x7f03901f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f0385400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03900cee40 0x7f03901f1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33202/0 (socket says 172.21.15.62:33202) 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f0385400640 1 -- 172.21.15.62:0/1655441897 learned_addr learned my addr 172.21.15.62:0/1655441897 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038de00640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f039006d730 0x7f03901f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038e800640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0390144c10 0x7f039012fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f0385400640 1 -- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f039006d730 msgr2=0x7f03901f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f0385400640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f039006d730 0x7f03901f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f0385400640 1 -- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0390144c10 msgr2=0x7f039012fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f0385400640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0390144c10 0x7f039012fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f0385400640 1 -- 172.21.15.62:0/1655441897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390130170 con 0x7f03900cee40 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038de00640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f039006d730 0x7f03901f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.720+0000 7f038e800640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0390144c10 0x7f039012fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.721+0000 7f0385400640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03900cee40 0x7f03901f1cc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f037800d800 tx=0x7f037800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.721+0000 7f0386a00640 1 -- 172.21.15.62:0/1655441897 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378018020 con 0x7f03900cee40 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.721+0000 7f0386a00640 1 -- 172.21.15.62:0/1655441897 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378009640 con 0x7f03900cee40 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.721+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03901f2430 con 0x7f03900cee40 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.721+0000 7f0386a00640 1 -- 172.21.15.62:0/1655441897 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378009a10 con 0x7f03900cee40 2024-04-25T09:07:30.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.721+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03901f29f0 con 0x7f03900cee40 2024-04-25T09:07:30.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.722+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039004ef20 con 0x7f03900cee40 2024-04-25T09:07:30.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.723+0000 7f0386a00640 1 -- 172.21.15.62:0/1655441897 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f037801b020 con 0x7f03900cee40 2024-04-25T09:07:30.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.724+0000 7f0386a00640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f036c06b4d0 0x7f036c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:30.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.725+0000 7f0386a00640 1 -- 172.21.15.62:0/1655441897 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f037808a400 con 0x7f03900cee40 2024-04-25T09:07:30.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.725+0000 7f038de00640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f036c06b4d0 0x7f036c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:30.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.726+0000 7f038de00640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f036c06b4d0 0x7f036c06d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0380006fd0 tx=0x7f0380008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:30.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.729+0000 7f0386a00640 1 -- 172.21.15.62:0/1655441897 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0378022030 con 0x7f03900cee40 2024-04-25T09:07:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.928+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03900c8c00 con 0x7f036c06b4d0 2024-04-25T09:07:30.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.948+0000 7f0386a00640 1 -- 172.21.15.62:0/1655441897 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246639 (secure 0 0 0) 0x7f03900c8c00 con 0x7f036c06b4d0 2024-04-25T09:07:30.938 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:30.940 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f036c06b4d0 msgr2=0x7f036c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f036c06b4d0 0x7f036c06d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0380006fd0 tx=0x7f0380008040 comp rx=0 tx=0).stop 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03900cee40 msgr2=0x7f03901f1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03900cee40 0x7f03901f1cc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f037800d800 tx=0x7f037800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 shutdown_connections 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f036c06b4d0 0x7f036c06d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03900cee40 0x7f03901f1cc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0390144c10 0x7f039012fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 --2- 172.21.15.62:0/1655441897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f039006d730 0x7f03901f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.953+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 >> 172.21.15.62:0/1655441897 conn(0x7f03900b14d0 msgr2=0x7f03900b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.954+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 shutdown_connections 2024-04-25T09:07:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:30.954+0000 7f038f200640 1 -- 172.21.15.62:0/1655441897 wait complete. 2024-04-25T09:07:30.963 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-25T09:07:29.983331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300592,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754674128,"statfs":{"total":773094113280,"available":772786307072,"internally_reserved":0,"allocated":5513216,"data_stored":2951216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.006202"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996639+0000","last_change":"2024-04-25T09:07:24.996639+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996639+0000","last_undegraded":"2024-04-25T09:07:24.996639+0000","last_fullsized":"2024-04-25T09:07:24.996639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990524+0000","last_change":"2024-04-25T09:07:24.990524+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990524+0000","last_undegraded":"2024-04-25T09:07:24.990524+0000","last_fullsized":"2024-04-25T09:07:24.990524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995074+0000","last_change":"2024-04-25T09:07:24.995074+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995074+0000","last_undegraded":"2024-04-25T09:07:24.995074+0000","last_fullsized":"2024-04-25T09:07:24.995074+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997350+0000","last_change":"2024-04-25T09:07:24.997350+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997350+0000","last_undegraded":"2024-04-25T09:07:24.997350+0000","last_fullsized":"2024-04-25T09:07:24.997350+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991559+0000","last_change":"2024-04-25T09:07:24.991559+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991559+0000","last_undegraded":"2024-04-25T09:07:24.991559+0000","last_fullsized":"2024-04-25T09:07:24.991559+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.002084+0000","last_change":"2024-04-25T09:07:25.002084+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.002084+0000","last_undegraded":"2024-04-25T09:07:25.002084+0000","last_fullsized":"2024-04-25T09:07:25.002084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990527+0000","last_change":"2024-04-25T09:07:24.990527+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990527+0000","last_undegraded":"2024-04-25T09:07:24.990527+0000","last_fullsized":"2024-04-25T09:07:24.990527+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998301+0000","last_change":"2024-04-25T09:07:24.998301+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998301+0000","last_undegraded":"2024-04-25T09:07:24.998301+0000","last_fullsized":"2024-04-25T09:07:24.998301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997181+0000","last_change":"2024-04-25T09:07:24.997181+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997181+0000","last_undegraded":"2024-04-25T09:07:24.997181+0000","last_fullsized":"2024-04-25T09:07:24.997181+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991564+0000","last_change":"2024-04-25T09:07:24.991564+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991564+0000","last_undegraded":"2024-04-25T09:07:24.991564+0000","last_fullsized":"2024-04-25T09:07:24.991564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.005646+0000","last_change":"2024-04-25T09:07:25.005646+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.005646+0000","last_undegraded":"2024-04-25T09:07:25.005646+0000","last_fullsized":"2024-04-25T09:07:25.005646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997243+0000","last_change":"2024-04-25T09:07:24.997243+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997243+0000","last_undegraded":"2024-04-25T09:07:24.997243+0000","last_fullsized":"2024-04-25T09:07:24.997243+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.002332+0000","last_change":"2024-04-25T09:07:25.002332+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.002332+0000","last_undegraded":"2024-04-25T09:07:25.002332+0000","last_fullsized":"2024-04-25T09:07:25.002332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994369+0000","last_change":"2024-04-25T09:07:24.994369+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994369+0000","last_undegraded":"2024-04-25T09:07:24.994369+0000","last_fullsized":"2024-04-25T09:07:24.994369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998804+0000","last_change":"2024-04-25T09:07:24.998804+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998804+0000","last_undegraded":"2024-04-25T09:07:24.998804+0000","last_fullsized":"2024-04-25T09:07:24.998804+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987193+0000","last_change":"2024-04-25T09:07:24.987193+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987193+0000","last_undegraded":"2024-04-25T09:07:24.987193+0000","last_fullsized":"2024-04-25T09:07:24.987193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991010+0000","last_change":"2024-04-25T09:07:24.991010+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991010+0000","last_undegraded":"2024-04-25T09:07:24.991010+0000","last_fullsized":"2024-04-25T09:07:24.991010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998751+0000","last_change":"2024-04-25T09:07:24.998751+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998751+0000","last_undegraded":"2024-04-25T09:07:24.998751+0000","last_fullsized":"2024-04-25T09:07:24.998751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989327+0000","last_change":"2024-04-25T09:07:24.989327+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989327+0000","last_undegraded":"2024-04-25T09:07:24.989327+0000","last_fullsized":"2024-04-25T09:07:24.989327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993017+0000","last_change":"2024-04-25T09:07:24.993017+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993017+0000","last_undegraded":"2024-04-25T09:07:24.993017+0000","last_fullsized":"2024-04-25T09:07:24.993017+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.003843+0000","last_change":"2024-04-25T09:07:25.003843+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.003843+0000","last_undegraded":"2024-04-25T09:07:25.003843+0000","last_fullsized":"2024-04-25T09:07:25.003843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987628+0000","last_change":"2024-04-25T09:07:24.987628+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987628+0000","last_undegraded":"2024-04-25T09:07:24.987628+0000","last_fullsized":"2024-04-25T09:07:24.987628+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987608+0000","last_change":"2024-04-25T09:07:24.987608+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987608+0000","last_undegraded":"2024-04-25T09:07:24.987608+0000","last_fullsized":"2024-04-25T09:07:24.987608+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989974+0000","last_change":"2024-04-25T09:07:24.989974+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989974+0000","last_undegraded":"2024-04-25T09:07:24.989974+0000","last_fullsized":"2024-04-25T09:07:24.989974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995869+0000","last_change":"2024-04-25T09:07:24.995869+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995869+0000","last_undegraded":"2024-04-25T09:07:24.995869+0000","last_fullsized":"2024-04-25T09:07:24.995869+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.007025+0000","last_change":"2024-04-25T09:07:25.007025+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.007025+0000","last_undegraded":"2024-04-25T09:07:25.007025+0000","last_fullsized":"2024-04-25T09:07:25.007025+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993999+0000","last_change":"2024-04-25T09:07:24.993999+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993999+0000","last_undegraded":"2024-04-25T09:07:24.993999+0000","last_fullsized":"2024-04-25T09:07:24.993999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996751+0000","last_change":"2024-04-25T09:07:24.996751+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996751+0000","last_undegraded":"2024-04-25T09:07:24.996751+0000","last_fullsized":"2024-04-25T09:07:24.996751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990286+0000","last_change":"2024-04-25T09:07:24.990286+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990286+0000","last_undegraded":"2024-04-25T09:07:24.990286+0000","last_fullsized":"2024-04-25T09:07:24.990286+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990066+0000","last_change":"2024-04-25T09:07:24.990066+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990066+0000","last_undegraded":"2024-04-25T09:07:24.990066+0000","last_fullsized":"2024-04-25T09:07:24.990066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996794+0000","last_change":"2024-04-25T09:07:24.996794+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996794+0000","last_undegraded":"2024-04-25T09:07:24.996794+0000","last_fullsized":"2024-04-25T09:07:24.996794+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997739+0000","last_change":"2024-04-25T09:07:24.997739+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997739+0000","last_undegraded":"2024-04-25T09:07:24.997739+0000","last_fullsized":"2024-04-25T09:07:24.997739+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987393+0000","last_change":"2024-04-25T09:07:24.987393+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987393+0000","last_undegraded":"2024-04-25T09:07:24.987393+0000","last_fullsized":"2024-04-25T09:07:24.987393+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013200+0000","last_change":"2024-04-25T09:07:25.010479+0000","last_active":"2024-04-25T09:07:25.013200+0000","last_peered":"2024-04-25T09:07:25.013200+0000","last_clean":"2024-04-25T09:07:25.013200+0000","last_became_active":"2024-04-25T09:07:25.009971+0000","last_became_peered":"2024-04-25T09:07:25.009971+0000","last_unstale":"2024-04-25T09:07:25.013200+0000","last_undegraded":"2024-04-25T09:07:25.013200+0000","last_fullsized":"2024-04-25T09:07:25.013200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:58.222873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995032+0000","last_change":"2024-04-25T09:07:24.995032+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995032+0000","last_undegraded":"2024-04-25T09:07:24.995032+0000","last_fullsized":"2024-04-25T09:07:24.995032+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017081+0000","last_change":"2024-04-25T09:07:25.012693+0000","last_active":"2024-04-25T09:07:25.017081+0000","last_peered":"2024-04-25T09:07:25.017081+0000","last_clean":"2024-04-25T09:07:25.017081+0000","last_became_active":"2024-04-25T09:07:25.012366+0000","last_became_peered":"2024-04-25T09:07:25.012366+0000","last_unstale":"2024-04-25T09:07:25.017081+0000","last_undegraded":"2024-04-25T09:07:25.017081+0000","last_fullsized":"2024-04-25T09:07:25.017081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:42.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996781+0000","last_change":"2024-04-25T09:07:24.996781+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996781+0000","last_undegraded":"2024-04-25T09:07:24.996781+0000","last_fullsized":"2024-04-25T09:07:24.996781+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013434+0000","last_change":"2024-04-25T09:07:25.011946+0000","last_active":"2024-04-25T09:07:25.013434+0000","last_peered":"2024-04-25T09:07:25.013434+0000","last_clean":"2024-04-25T09:07:25.013434+0000","last_became_active":"2024-04-25T09:07:25.011630+0000","last_became_peered":"2024-04-25T09:07:25.011630+0000","last_unstale":"2024-04-25T09:07:25.013434+0000","last_undegraded":"2024-04-25T09:07:25.013434+0000","last_fullsized":"2024-04-25T09:07:25.013434+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:45.527811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994342+0000","last_change":"2024-04-25T09:07:24.994342+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994342+0000","last_undegraded":"2024-04-25T09:07:24.994342+0000","last_fullsized":"2024-04-25T09:07:24.994342+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.014473+0000","last_change":"2024-04-25T09:07:25.010814+0000","last_active":"2024-04-25T09:07:25.014473+0000","last_peered":"2024-04-25T09:07:25.014473+0000","last_clean":"2024-04-25T09:07:25.014473+0000","last_became_active":"2024-04-25T09:07:25.008979+0000","last_became_peered":"2024-04-25T09:07:25.008979+0000","last_unstale":"2024-04-25T09:07:25.014473+0000","last_undegraded":"2024-04-25T09:07:25.014473+0000","last_fullsized":"2024-04-25T09:07:25.014473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:50.056920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992079+0000","last_change":"2024-04-25T09:07:24.992079+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992079+0000","last_undegraded":"2024-04-25T09:07:24.992079+0000","last_fullsized":"2024-04-25T09:07:24.992079+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013153+0000","last_change":"2024-04-25T09:07:25.010295+0000","last_active":"2024-04-25T09:07:25.013153+0000","last_peered":"2024-04-25T09:07:25.013153+0000","last_clean":"2024-04-25T09:07:25.013153+0000","last_became_active":"2024-04-25T09:07:25.009984+0000","last_became_peered":"2024-04-25T09:07:25.009984+0000","last_unstale":"2024-04-25T09:07:25.013153+0000","last_undegraded":"2024-04-25T09:07:25.013153+0000","last_fullsized":"2024-04-25T09:07:25.013153+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:35.643627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.999831+0000","last_change":"2024-04-25T09:07:24.999831+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.999831+0000","last_undegraded":"2024-04-25T09:07:24.999831+0000","last_fullsized":"2024-04-25T09:07:24.999831+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010479+0000","last_change":"2024-04-25T09:07:25.005618+0000","last_active":"2024-04-25T09:07:25.010479+0000","last_peered":"2024-04-25T09:07:25.010479+0000","last_clean":"2024-04-25T09:07:25.010479+0000","last_became_active":"2024-04-25T09:07:25.005276+0000","last_became_peered":"2024-04-25T09:07:25.005276+0000","last_unstale":"2024-04-25T09:07:25.010479+0000","last_undegraded":"2024-04-25T09:07:25.010479+0000","last_fullsized":"2024-04-25T09:07:25.010479+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:39.155136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991651+0000","last_change":"2024-04-25T09:07:24.991651+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991651+0000","last_undegraded":"2024-04-25T09:07:24.991651+0000","last_fullsized":"2024-04-25T09:07:24.991651+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017467+0000","last_change":"2024-04-25T09:07:25.014784+0000","last_active":"2024-04-25T09:07:25.017467+0000","last_peered":"2024-04-25T09:07:25.017467+0000","last_clean":"2024-04-25T09:07:25.017467+0000","last_became_active":"2024-04-25T09:07:25.012907+0000","last_became_peered":"2024-04-25T09:07:25.012907+0000","last_unstale":"2024-04-25T09:07:25.017467+0000","last_undegraded":"2024-04-25T09:07:25.017467+0000","last_fullsized":"2024-04-25T09:07:25.017467+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.030750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.988372+0000","last_change":"2024-04-25T09:07:24.988372+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.988372+0000","last_undegraded":"2024-04-25T09:07:24.988372+0000","last_fullsized":"2024-04-25T09:07:24.988372+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015362+0000","last_change":"2024-04-25T09:07:25.009763+0000","last_active":"2024-04-25T09:07:25.015362+0000","last_peered":"2024-04-25T09:07:25.015362+0000","last_clean":"2024-04-25T09:07:25.015362+0000","last_became_active":"2024-04-25T09:07:25.009474+0000","last_became_peered":"2024-04-25T09:07:25.009474+0000","last_unstale":"2024-04-25T09:07:25.015362+0000","last_undegraded":"2024-04-25T09:07:25.015362+0000","last_fullsized":"2024-04-25T09:07:25.015362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:51.123149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996639+0000","last_change":"2024-04-25T09:07:24.996639+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996639+0000","last_undegraded":"2024-04-25T09:07:24.996639+0000","last_fullsized":"2024-04-25T09:07:24.996639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015127+0000","last_change":"2024-04-25T09:07:25.007040+0000","last_active":"2024-04-25T09:07:25.015127+0000","last_peered":"2024-04-25T09:07:25.015127+0000","last_clean":"2024-04-25T09:07:25.015127+0000","last_became_active":"2024-04-25T09:07:25.006742+0000","last_became_peered":"2024-04-25T09:07:25.006742+0000","last_unstale":"2024-04-25T09:07:25.015127+0000","last_undegraded":"2024-04-25T09:07:25.015127+0000","last_fullsized":"2024-04-25T09:07:25.015127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:07.492703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993569+0000","last_change":"2024-04-25T09:07:24.993569+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993569+0000","last_undegraded":"2024-04-25T09:07:24.993569+0000","last_fullsized":"2024-04-25T09:07:24.993569+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018281+0000","last_change":"2024-04-25T09:07:25.016674+0000","last_active":"2024-04-25T09:07:25.018281+0000","last_peered":"2024-04-25T09:07:25.018281+0000","last_clean":"2024-04-25T09:07:25.018281+0000","last_became_active":"2024-04-25T09:07:25.016317+0000","last_became_peered":"2024-04-25T09:07:25.016317+0000","last_unstale":"2024-04-25T09:07:25.018281+0000","last_undegraded":"2024-04-25T09:07:25.018281+0000","last_fullsized":"2024-04-25T09:07:25.018281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:19.859538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996661+0000","last_change":"2024-04-25T09:07:24.996661+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996661+0000","last_undegraded":"2024-04-25T09:07:24.996661+0000","last_fullsized":"2024-04-25T09:07:24.996661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013680+0000","last_change":"2024-04-25T09:07:25.008303+0000","last_active":"2024-04-25T09:07:25.013680+0000","last_peered":"2024-04-25T09:07:25.013680+0000","last_clean":"2024-04-25T09:07:25.013680+0000","last_became_active":"2024-04-25T09:07:25.006381+0000","last_became_peered":"2024-04-25T09:07:25.006381+0000","last_unstale":"2024-04-25T09:07:25.013680+0000","last_undegraded":"2024-04-25T09:07:25.013680+0000","last_fullsized":"2024-04-25T09:07:25.013680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993591+0000","last_change":"2024-04-25T09:07:24.993591+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993591+0000","last_undegraded":"2024-04-25T09:07:24.993591+0000","last_fullsized":"2024-04-25T09:07:24.993591+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007328+0000","last_change":"2024-04-25T09:07:25.003604+0000","last_active":"2024-04-25T09:07:25.007328+0000","last_peered":"2024-04-25T09:07:25.007328+0000","last_clean":"2024-04-25T09:07:25.007328+0000","last_became_active":"2024-04-25T09:07:25.003266+0000","last_became_peered":"2024-04-25T09:07:25.003266+0000","last_unstale":"2024-04-25T09:07:25.007328+0000","last_undegraded":"2024-04-25T09:07:25.007328+0000","last_fullsized":"2024-04-25T09:07:25.007328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998031+0000","last_change":"2024-04-25T09:07:24.998031+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998031+0000","last_undegraded":"2024-04-25T09:07:24.998031+0000","last_fullsized":"2024-04-25T09:07:24.998031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010140+0000","last_change":"2024-04-25T09:07:25.006123+0000","last_active":"2024-04-25T09:07:25.010140+0000","last_peered":"2024-04-25T09:07:25.010140+0000","last_clean":"2024-04-25T09:07:25.010140+0000","last_became_active":"2024-04-25T09:07:25.004723+0000","last_became_peered":"2024-04-25T09:07:25.004723+0000","last_unstale":"2024-04-25T09:07:25.010140+0000","last_undegraded":"2024-04-25T09:07:25.010140+0000","last_fullsized":"2024-04-25T09:07:25.010140+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:07.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986042+0000","last_change":"2024-04-25T09:07:24.986042+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986042+0000","last_undegraded":"2024-04-25T09:07:24.986042+0000","last_fullsized":"2024-04-25T09:07:24.986042+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007877+0000","last_change":"2024-04-25T09:07:25.004430+0000","last_active":"2024-04-25T09:07:25.007877+0000","last_peered":"2024-04-25T09:07:25.007877+0000","last_clean":"2024-04-25T09:07:25.007877+0000","last_became_active":"2024-04-25T09:07:25.000012+0000","last_became_peered":"2024-04-25T09:07:25.000012+0000","last_unstale":"2024-04-25T09:07:25.007877+0000","last_undegraded":"2024-04-25T09:07:25.007877+0000","last_fullsized":"2024-04-25T09:07:25.007877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:13.587386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.985571+0000","last_change":"2024-04-25T09:07:24.985571+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.985571+0000","last_undegraded":"2024-04-25T09:07:24.985571+0000","last_fullsized":"2024-04-25T09:07:24.985571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017723+0000","last_change":"2024-04-25T09:07:25.013988+0000","last_active":"2024-04-25T09:07:25.017723+0000","last_peered":"2024-04-25T09:07:25.017723+0000","last_clean":"2024-04-25T09:07:25.017723+0000","last_became_active":"2024-04-25T09:07:25.013681+0000","last_became_peered":"2024-04-25T09:07:25.013681+0000","last_unstale":"2024-04-25T09:07:25.017723+0000","last_undegraded":"2024-04-25T09:07:25.017723+0000","last_fullsized":"2024-04-25T09:07:25.017723+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.565341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998624+0000","last_change":"2024-04-25T09:07:24.998624+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998624+0000","last_undegraded":"2024-04-25T09:07:24.998624+0000","last_fullsized":"2024-04-25T09:07:24.998624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010072+0000","last_change":"2024-04-25T09:07:25.006626+0000","last_active":"2024-04-25T09:07:25.010072+0000","last_peered":"2024-04-25T09:07:25.010072+0000","last_clean":"2024-04-25T09:07:25.010072+0000","last_became_active":"2024-04-25T09:07:25.006323+0000","last_became_peered":"2024-04-25T09:07:25.006323+0000","last_unstale":"2024-04-25T09:07:25.010072+0000","last_undegraded":"2024-04-25T09:07:25.010072+0000","last_fullsized":"2024-04-25T09:07:25.010072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:36.534602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986282+0000","last_change":"2024-04-25T09:07:24.986282+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986282+0000","last_undegraded":"2024-04-25T09:07:24.986282+0000","last_fullsized":"2024-04-25T09:07:24.986282+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018288+0000","last_change":"2024-04-25T09:07:25.016157+0000","last_active":"2024-04-25T09:07:25.018288+0000","last_peered":"2024-04-25T09:07:25.018288+0000","last_clean":"2024-04-25T09:07:25.018288+0000","last_became_active":"2024-04-25T09:07:25.015731+0000","last_became_peered":"2024-04-25T09:07:25.015731+0000","last_unstale":"2024-04-25T09:07:25.018288+0000","last_undegraded":"2024-04-25T09:07:25.018288+0000","last_fullsized":"2024-04-25T09:07:25.018288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:30.570535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986852+0000","last_change":"2024-04-25T09:07:24.986852+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986852+0000","last_undegraded":"2024-04-25T09:07:24.986852+0000","last_fullsized":"2024-04-25T09:07:24.986852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015226+0000","last_change":"2024-04-25T09:07:25.010677+0000","last_active":"2024-04-25T09:07:25.015226+0000","last_peered":"2024-04-25T09:07:25.015226+0000","last_clean":"2024-04-25T09:07:25.015226+0000","last_became_active":"2024-04-25T09:07:25.010337+0000","last_became_peered":"2024-04-25T09:07:25.010337+0000","last_unstale":"2024-04-25T09:07:25.015226+0000","last_undegraded":"2024-04-25T09:07:25.015226+0000","last_fullsized":"2024-04-25T09:07:25.015226+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:37.745468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992003+0000","last_change":"2024-04-25T09:07:24.992003+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992003+0000","last_undegraded":"2024-04-25T09:07:24.992003+0000","last_fullsized":"2024-04-25T09:07:24.992003+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015417+0000","last_change":"2024-04-25T09:07:25.012576+0000","last_active":"2024-04-25T09:07:25.015417+0000","last_peered":"2024-04-25T09:07:25.015417+0000","last_clean":"2024-04-25T09:07:25.015417+0000","last_became_active":"2024-04-25T09:07:25.010949+0000","last_became_peered":"2024-04-25T09:07:25.010949+0000","last_unstale":"2024-04-25T09:07:25.015417+0000","last_undegraded":"2024-04-25T09:07:25.015417+0000","last_fullsized":"2024-04-25T09:07:25.015417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:07.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994462+0000","last_change":"2024-04-25T09:07:24.994462+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994462+0000","last_undegraded":"2024-04-25T09:07:24.994462+0000","last_fullsized":"2024-04-25T09:07:24.994462+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017671+0000","last_change":"2024-04-25T09:07:25.013972+0000","last_active":"2024-04-25T09:07:25.017671+0000","last_peered":"2024-04-25T09:07:25.017671+0000","last_clean":"2024-04-25T09:07:25.017671+0000","last_became_active":"2024-04-25T09:07:25.013633+0000","last_became_peered":"2024-04-25T09:07:25.013633+0000","last_unstale":"2024-04-25T09:07:25.017671+0000","last_undegraded":"2024-04-25T09:07:25.017671+0000","last_fullsized":"2024-04-25T09:07:25.017671+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:47.797851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993010+0000","last_change":"2024-04-25T09:07:24.993010+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993010+0000","last_undegraded":"2024-04-25T09:07:24.993010+0000","last_fullsized":"2024-04-25T09:07:24.993010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018264+0000","last_change":"2024-04-25T09:07:25.016649+0000","last_active":"2024-04-25T09:07:25.018264+0000","last_peered":"2024-04-25T09:07:25.018264+0000","last_clean":"2024-04-25T09:07:25.018264+0000","last_became_active":"2024-04-25T09:07:25.016387+0000","last_became_peered":"2024-04-25T09:07:25.016387+0000","last_unstale":"2024-04-25T09:07:25.018264+0000","last_undegraded":"2024-04-25T09:07:25.018264+0000","last_fullsized":"2024-04-25T09:07:25.018264+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:09.601456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998397+0000","last_change":"2024-04-25T09:07:24.998397+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998397+0000","last_undegraded":"2024-04-25T09:07:24.998397+0000","last_fullsized":"2024-04-25T09:07:24.998397+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015086+0000","last_change":"2024-04-25T09:07:25.009598+0000","last_active":"2024-04-25T09:07:25.015086+0000","last_peered":"2024-04-25T09:07:25.015086+0000","last_clean":"2024-04-25T09:07:25.015086+0000","last_became_active":"2024-04-25T09:07:25.009294+0000","last_became_peered":"2024-04-25T09:07:25.009294+0000","last_unstale":"2024-04-25T09:07:25.015086+0000","last_undegraded":"2024-04-25T09:07:25.015086+0000","last_fullsized":"2024-04-25T09:07:25.015086+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:31.801129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997245+0000","last_change":"2024-04-25T09:07:24.997245+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997245+0000","last_undegraded":"2024-04-25T09:07:24.997245+0000","last_fullsized":"2024-04-25T09:07:24.997245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007711+0000","last_change":"2024-04-25T09:07:25.005217+0000","last_active":"2024-04-25T09:07:25.007711+0000","last_peered":"2024-04-25T09:07:25.007711+0000","last_clean":"2024-04-25T09:07:25.007711+0000","last_became_active":"2024-04-25T09:07:25.003714+0000","last_became_peered":"2024-04-25T09:07:25.003714+0000","last_unstale":"2024-04-25T09:07:25.007711+0000","last_undegraded":"2024-04-25T09:07:25.007711+0000","last_fullsized":"2024-04-25T09:07:25.007711+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:55.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998226+0000","last_change":"2024-04-25T09:07:24.998226+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998226+0000","last_undegraded":"2024-04-25T09:07:24.998226+0000","last_fullsized":"2024-04-25T09:07:24.998226+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.005589+0000","last_change":"2024-04-25T09:07:25.003186+0000","last_active":"2024-04-25T09:07:25.005589+0000","last_peered":"2024-04-25T09:07:25.005589+0000","last_clean":"2024-04-25T09:07:25.005589+0000","last_became_active":"2024-04-25T09:07:25.002435+0000","last_became_peered":"2024-04-25T09:07:25.002435+0000","last_unstale":"2024-04-25T09:07:25.005589+0000","last_undegraded":"2024-04-25T09:07:25.005589+0000","last_fullsized":"2024-04-25T09:07:25.005589+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:22.212413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994941+0000","last_change":"2024-04-25T09:07:24.994941+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994941+0000","last_undegraded":"2024-04-25T09:07:24.994941+0000","last_fullsized":"2024-04-25T09:07:24.994941+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018229+0000","last_change":"2024-04-25T09:07:25.015721+0000","last_active":"2024-04-25T09:07:25.018229+0000","last_peered":"2024-04-25T09:07:25.018229+0000","last_clean":"2024-04-25T09:07:25.018229+0000","last_became_active":"2024-04-25T09:07:25.015220+0000","last_became_peered":"2024-04-25T09:07:25.015220+0000","last_unstale":"2024-04-25T09:07:25.018229+0000","last_undegraded":"2024-04-25T09:07:25.018229+0000","last_fullsized":"2024-04-25T09:07:25.018229+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:31.801129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991242+0000","last_change":"2024-04-25T09:07:24.991242+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991242+0000","last_undegraded":"2024-04-25T09:07:24.991242+0000","last_fullsized":"2024-04-25T09:07:24.991242+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015029+0000","last_change":"2024-04-25T09:07:25.010776+0000","last_active":"2024-04-25T09:07:25.015029+0000","last_peered":"2024-04-25T09:07:25.015029+0000","last_clean":"2024-04-25T09:07:25.015029+0000","last_became_active":"2024-04-25T09:07:25.010280+0000","last_became_peered":"2024-04-25T09:07:25.010280+0000","last_unstale":"2024-04-25T09:07:25.015029+0000","last_undegraded":"2024-04-25T09:07:25.015029+0000","last_fullsized":"2024-04-25T09:07:25.015029+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:30.570535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993158+0000","last_change":"2024-04-25T09:07:24.993158+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993158+0000","last_undegraded":"2024-04-25T09:07:24.993158+0000","last_fullsized":"2024-04-25T09:07:24.993158+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.012361+0000","last_change":"2024-04-25T09:07:25.007229+0000","last_active":"2024-04-25T09:07:25.012361+0000","last_peered":"2024-04-25T09:07:25.012361+0000","last_clean":"2024-04-25T09:07:25.012361+0000","last_became_active":"2024-04-25T09:07:25.006847+0000","last_became_peered":"2024-04-25T09:07:25.006847+0000","last_unstale":"2024-04-25T09:07:25.012361+0000","last_undegraded":"2024-04-25T09:07:25.012361+0000","last_fullsized":"2024-04-25T09:07:25.012361+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:26.558822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989296+0000","last_change":"2024-04-25T09:07:24.989296+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989296+0000","last_undegraded":"2024-04-25T09:07:24.989296+0000","last_fullsized":"2024-04-25T09:07:24.989296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.016762+0000","last_change":"2024-04-25T09:07:25.011689+0000","last_active":"2024-04-25T09:07:25.016762+0000","last_peered":"2024-04-25T09:07:25.016762+0000","last_clean":"2024-04-25T09:07:25.016762+0000","last_became_active":"2024-04-25T09:07:25.011388+0000","last_became_peered":"2024-04-25T09:07:25.011388+0000","last_unstale":"2024-04-25T09:07:25.016762+0000","last_undegraded":"2024-04-25T09:07:25.016762+0000","last_fullsized":"2024-04-25T09:07:25.016762+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:31.406996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.007613+0000","last_change":"2024-04-25T09:07:25.007613+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.007613+0000","last_undegraded":"2024-04-25T09:07:25.007613+0000","last_fullsized":"2024-04-25T09:07:25.007613+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.005467+0000","last_change":"2024-04-25T09:07:25.004275+0000","last_active":"2024-04-25T09:07:25.005467+0000","last_peered":"2024-04-25T09:07:25.005467+0000","last_clean":"2024-04-25T09:07:25.005467+0000","last_became_active":"2024-04-25T09:07:25.004002+0000","last_became_peered":"2024-04-25T09:07:25.004002+0000","last_unstale":"2024-04-25T09:07:25.005467+0000","last_undegraded":"2024-04-25T09:07:25.005467+0000","last_fullsized":"2024-04-25T09:07:25.005467+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:33.025712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993341+0000","last_change":"2024-04-25T09:07:24.993341+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993341+0000","last_undegraded":"2024-04-25T09:07:24.993341+0000","last_fullsized":"2024-04-25T09:07:24.993341+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013092+0000","last_change":"2024-04-25T09:07:25.007837+0000","last_active":"2024-04-25T09:07:25.013092+0000","last_peered":"2024-04-25T09:07:25.013092+0000","last_clean":"2024-04-25T09:07:25.013092+0000","last_became_active":"2024-04-25T09:07:25.007537+0000","last_became_peered":"2024-04-25T09:07:25.007537+0000","last_unstale":"2024-04-25T09:07:25.013092+0000","last_undegraded":"2024-04-25T09:07:25.013092+0000","last_fullsized":"2024-04-25T09:07:25.013092+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.053815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992440+0000","last_change":"2024-04-25T09:07:24.992440+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992440+0000","last_undegraded":"2024-04-25T09:07:24.992440+0000","last_fullsized":"2024-04-25T09:07:24.992440+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.006431+0000","last_change":"2024-04-25T09:07:25.002676+0000","last_active":"2024-04-25T09:07:25.006431+0000","last_peered":"2024-04-25T09:07:25.006431+0000","last_clean":"2024-04-25T09:07:25.006431+0000","last_became_active":"2024-04-25T09:07:25.001512+0000","last_became_peered":"2024-04-25T09:07:25.001512+0000","last_unstale":"2024-04-25T09:07:25.006431+0000","last_undegraded":"2024-04-25T09:07:25.006431+0000","last_fullsized":"2024-04-25T09:07:25.006431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:37.745468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.004957+0000","last_change":"2024-04-25T09:07:25.004957+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.004957+0000","last_undegraded":"2024-04-25T09:07:25.004957+0000","last_fullsized":"2024-04-25T09:07:25.004957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.019379+0000","last_change":"2024-04-25T09:07:25.014785+0000","last_active":"2024-04-25T09:07:25.019379+0000","last_peered":"2024-04-25T09:07:25.019379+0000","last_clean":"2024-04-25T09:07:25.019379+0000","last_became_active":"2024-04-25T09:07:25.013321+0000","last_became_peered":"2024-04-25T09:07:25.013321+0000","last_unstale":"2024-04-25T09:07:25.019379+0000","last_undegraded":"2024-04-25T09:07:25.019379+0000","last_fullsized":"2024-04-25T09:07:25.019379+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:09.368176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.004963+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:07:25.004963+0000","last_peered":"2024-04-25T09:07:25.004963+0000","last_clean":"2024-04-25T09:07:25.004963+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:07:25.004963+0000","last_undegraded":"2024-04-25T09:07:25.004963+0000","last_fullsized":"2024-04-25T09:07:25.004963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1134592,"data_stored":811678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1134592,"data_stored":811678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:07:30.966 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:07:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.224+0000 7f2fe6a00640 1 Processor -- start 2024-04-25T09:07:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.224+0000 7f2fe6a00640 1 -- start start 2024-04-25T09:07:31.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.225+0000 7f2fe6a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2fe00b6a90 0x7f2fe00d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.225+0000 7f2fe6a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2fe00b27a0 0x7f2fe00d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.225+0000 7f2fe6a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00d3110 con 0x7f2fe00b56f0 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.225+0000 7f2fe6a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe00cedc0 con 0x7f2fe00b27a0 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.225+0000 7f2fe6a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe00d33f0 con 0x7f2fe00b6a90 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2fe00b27a0 0x7f2fe00d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2fe00b27a0 0x7f2fe00d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33210/0 (socket says 172.21.15.62:33210) 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe6000640 1 -- 172.21.15.62:0/1510220242 learned_addr learned my addr 172.21.15.62:0/1510220242 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe4c00640 1 --2- 172.21.15.62:0/1510220242 >> v2:172.21.15.121:3300/0 conn(0x7f2fe00b6a90 0x7f2fe00d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe6000640 1 -- 172.21.15.62:0/1510220242 >> v2:172.21.15.121:3300/0 conn(0x7f2fe00b6a90 msgr2=0x7f2fe00d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe6000640 1 --2- 172.21.15.62:0/1510220242 >> v2:172.21.15.121:3300/0 conn(0x7f2fe00b6a90 0x7f2fe00d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe6000640 1 -- 172.21.15.62:0/1510220242 >> v1:172.21.15.121:6789/0 conn(0x7f2fe00b56f0 legacy=0x7f2fe0160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe6000640 1 -- 172.21.15.62:0/1510220242 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00d36d0 con 0x7f2fe00b27a0 2024-04-25T09:07:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.226+0000 7f2fe4c00640 1 --2- 172.21.15.62:0/1510220242 >> v2:172.21.15.121:3300/0 conn(0x7f2fe00b6a90 0x7f2fe00d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:31.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.227+0000 7f2fe6000640 1 --2- 172.21.15.62:0/1510220242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b27a0 0x7f2fe00d2520 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2fe008c570 tx=0x7f2fd802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38647624357f9132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:31.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fdfe00640 1 -- 172.21.15.62:0/1510220242 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8036070 con 0x7f2fe00b27a0 2024-04-25T09:07:31.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fdfe00640 1 -- 172.21.15.62:0/1510220242 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8031600 con 0x7f2fe00b27a0 2024-04-25T09:07:31.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fdfe00640 1 -- 172.21.15.62:0/1510220242 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80319d0 con 0x7f2fe00b27a0 2024-04-25T09:07:31.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fe6a00640 1 -- 172.21.15.62:0/1510220242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b27a0 msgr2=0x7f2fe00d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:31.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/1510220242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b27a0 0x7f2fe00d2520 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2fe008c570 tx=0x7f2fd802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fe6a00640 1 -- 172.21.15.62:0/1510220242 shutdown_connections 2024-04-25T09:07:31.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/1510220242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b27a0 0x7f2fe00d2520 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/1510220242 >> v2:172.21.15.121:3300/0 conn(0x7f2fe00b6a90 0x7f2fe00d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fe6a00640 1 -- 172.21.15.62:0/1510220242 >> 172.21.15.62:0/1510220242 conn(0x7f2fe0066d10 msgr2=0x7f2fe0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:31.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fe6a00640 1 -- 172.21.15.62:0/1510220242 shutdown_connections 2024-04-25T09:07:31.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.228+0000 7f2fe6a00640 1 -- 172.21.15.62:0/1510220242 wait complete. 2024-04-25T09:07:31.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.230+0000 7f2fe6a00640 1 Processor -- start 2024-04-25T09:07:31.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.230+0000 7f2fe6a00640 1 -- start start 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.231+0000 7f2fe6a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fe00b27a0 0x7f2fe01f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.231+0000 7f2fe6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fe00b56f0 0x7f2fe00c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.231+0000 7f2fe6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b6a90 0x7f2fe015dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.231+0000 7f2fe6a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b2ee0 con 0x7f2fe00b6a90 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.231+0000 7f2fe6a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b2d60 con 0x7f2fe00b56f0 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.231+0000 7f2fe6a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00b5ad0 con 0x7f2fe00b27a0 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b6a90 0x7f2fe015dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b6a90 0x7f2fe015dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33216/0 (socket says 172.21.15.62:33216) 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe6000640 1 -- 172.21.15.62:0/494377605 learned_addr learned my addr 172.21.15.62:0/494377605 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe4c00640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fe00b56f0 0x7f2fe00c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe5600640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fe00b27a0 0x7f2fe01f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe6000640 1 -- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fe00b27a0 msgr2=0x7f2fe01f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:31.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe6000640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fe00b27a0 0x7f2fe01f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe6000640 1 -- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fe00b56f0 msgr2=0x7f2fe00c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:31.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe6000640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fe00b56f0 0x7f2fe00c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe6000640 1 -- 172.21.15.62:0/494377605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe01f77e0 con 0x7f2fe00b6a90 2024-04-25T09:07:31.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe4c00640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fe00b56f0 0x7f2fe00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:31.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.232+0000 7f2fe5600640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fe00b27a0 0x7f2fe01f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:31.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.233+0000 7f2fe6000640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b6a90 0x7f2fe015dfd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009f40 tx=0x7f2fd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:31.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.233+0000 7f2fdd600640 1 -- 172.21.15.62:0/494377605 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8036050 con 0x7f2fe00b6a90 2024-04-25T09:07:31.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.233+0000 7f2fdd600640 1 -- 172.21.15.62:0/494377605 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8042070 con 0x7f2fe00b6a90 2024-04-25T09:07:31.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.233+0000 7f2fdd600640 1 -- 172.21.15.62:0/494377605 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8009510 con 0x7f2fe00b6a90 2024-04-25T09:07:31.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.233+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe01f7ac0 con 0x7f2fe00b6a90 2024-04-25T09:07:31.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.233+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe01f8080 con 0x7f2fe00b6a90 2024-04-25T09:07:31.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.234+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe004ef20 con 0x7f2fe00b6a90 2024-04-25T09:07:31.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.235+0000 7f2fdd600640 1 -- 172.21.15.62:0/494377605 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fd8007bd0 con 0x7f2fe00b6a90 2024-04-25T09:07:31.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.236+0000 7f2fdd600640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2fc006b4d0 0x7f2fc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.236+0000 7f2fdd600640 1 -- 172.21.15.62:0/494377605 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fd80ab200 con 0x7f2fe00b6a90 2024-04-25T09:07:31.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.237+0000 7f2fe5600640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2fc006b4d0 0x7f2fc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.238+0000 7f2fe5600640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2fc006b4d0 0x7f2fc006d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4003eb0 tx=0x7f2fd4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:31.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.240+0000 7f2fdd600640 1 -- 172.21.15.62:0/494377605 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fd807a080 con 0x7f2fe00b6a90 2024-04-25T09:07:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.437+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2fe00ce580 con 0x7f2fc006b4d0 2024-04-25T09:07:31.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.455+0000 7f2fdd600640 1 -- 172.21.15.62:0/494377605 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246639 (secure 0 0 0) 0x7f2fd4054380 con 0x7f2fc006b4d0 2024-04-25T09:07:31.446 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:31.447 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:07:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2fc006b4d0 msgr2=0x7f2fc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2fc006b4d0 0x7f2fc006d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4003eb0 tx=0x7f2fd4009040 comp rx=0 tx=0).stop 2024-04-25T09:07:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b6a90 msgr2=0x7f2fe015dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b6a90 0x7f2fe015dfd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009f40 tx=0x7f2fd8004a40 comp rx=0 tx=0).stop 2024-04-25T09:07:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 shutdown_connections 2024-04-25T09:07:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2fc006b4d0 0x7f2fc006d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fe00b6a90 0x7f2fe015dfd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fe00b56f0 0x7f2fe00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 --2- 172.21.15.62:0/494377605 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2fe00b27a0 0x7f2fe01f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.461+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 >> 172.21.15.62:0/494377605 conn(0x7f2fe0066d10 msgr2=0x7f2fe00d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.462+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 shutdown_connections 2024-04-25T09:07:31.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.462+0000 7f2fe6a00640 1 -- 172.21.15.62:0/494377605 wait complete. 2024-04-25T09:07:31.468 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-25T09:07:29.983331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300592,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754674128,"statfs":{"total":773094113280,"available":772786307072,"internally_reserved":0,"allocated":5513216,"data_stored":2951216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.006202"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996639+0000","last_change":"2024-04-25T09:07:24.996639+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996639+0000","last_undegraded":"2024-04-25T09:07:24.996639+0000","last_fullsized":"2024-04-25T09:07:24.996639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990524+0000","last_change":"2024-04-25T09:07:24.990524+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990524+0000","last_undegraded":"2024-04-25T09:07:24.990524+0000","last_fullsized":"2024-04-25T09:07:24.990524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995074+0000","last_change":"2024-04-25T09:07:24.995074+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995074+0000","last_undegraded":"2024-04-25T09:07:24.995074+0000","last_fullsized":"2024-04-25T09:07:24.995074+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997350+0000","last_change":"2024-04-25T09:07:24.997350+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997350+0000","last_undegraded":"2024-04-25T09:07:24.997350+0000","last_fullsized":"2024-04-25T09:07:24.997350+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991559+0000","last_change":"2024-04-25T09:07:24.991559+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991559+0000","last_undegraded":"2024-04-25T09:07:24.991559+0000","last_fullsized":"2024-04-25T09:07:24.991559+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.002084+0000","last_change":"2024-04-25T09:07:25.002084+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.002084+0000","last_undegraded":"2024-04-25T09:07:25.002084+0000","last_fullsized":"2024-04-25T09:07:25.002084+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990527+0000","last_change":"2024-04-25T09:07:24.990527+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990527+0000","last_undegraded":"2024-04-25T09:07:24.990527+0000","last_fullsized":"2024-04-25T09:07:24.990527+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998301+0000","last_change":"2024-04-25T09:07:24.998301+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998301+0000","last_undegraded":"2024-04-25T09:07:24.998301+0000","last_fullsized":"2024-04-25T09:07:24.998301+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997181+0000","last_change":"2024-04-25T09:07:24.997181+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997181+0000","last_undegraded":"2024-04-25T09:07:24.997181+0000","last_fullsized":"2024-04-25T09:07:24.997181+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991564+0000","last_change":"2024-04-25T09:07:24.991564+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991564+0000","last_undegraded":"2024-04-25T09:07:24.991564+0000","last_fullsized":"2024-04-25T09:07:24.991564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.005646+0000","last_change":"2024-04-25T09:07:25.005646+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.005646+0000","last_undegraded":"2024-04-25T09:07:25.005646+0000","last_fullsized":"2024-04-25T09:07:25.005646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997243+0000","last_change":"2024-04-25T09:07:24.997243+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997243+0000","last_undegraded":"2024-04-25T09:07:24.997243+0000","last_fullsized":"2024-04-25T09:07:24.997243+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.002332+0000","last_change":"2024-04-25T09:07:25.002332+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.002332+0000","last_undegraded":"2024-04-25T09:07:25.002332+0000","last_fullsized":"2024-04-25T09:07:25.002332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994369+0000","last_change":"2024-04-25T09:07:24.994369+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994369+0000","last_undegraded":"2024-04-25T09:07:24.994369+0000","last_fullsized":"2024-04-25T09:07:24.994369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998804+0000","last_change":"2024-04-25T09:07:24.998804+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998804+0000","last_undegraded":"2024-04-25T09:07:24.998804+0000","last_fullsized":"2024-04-25T09:07:24.998804+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987193+0000","last_change":"2024-04-25T09:07:24.987193+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987193+0000","last_undegraded":"2024-04-25T09:07:24.987193+0000","last_fullsized":"2024-04-25T09:07:24.987193+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991010+0000","last_change":"2024-04-25T09:07:24.991010+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991010+0000","last_undegraded":"2024-04-25T09:07:24.991010+0000","last_fullsized":"2024-04-25T09:07:24.991010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998751+0000","last_change":"2024-04-25T09:07:24.998751+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998751+0000","last_undegraded":"2024-04-25T09:07:24.998751+0000","last_fullsized":"2024-04-25T09:07:24.998751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989327+0000","last_change":"2024-04-25T09:07:24.989327+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989327+0000","last_undegraded":"2024-04-25T09:07:24.989327+0000","last_fullsized":"2024-04-25T09:07:24.989327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993017+0000","last_change":"2024-04-25T09:07:24.993017+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993017+0000","last_undegraded":"2024-04-25T09:07:24.993017+0000","last_fullsized":"2024-04-25T09:07:24.993017+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.003843+0000","last_change":"2024-04-25T09:07:25.003843+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.003843+0000","last_undegraded":"2024-04-25T09:07:25.003843+0000","last_fullsized":"2024-04-25T09:07:25.003843+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987628+0000","last_change":"2024-04-25T09:07:24.987628+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987628+0000","last_undegraded":"2024-04-25T09:07:24.987628+0000","last_fullsized":"2024-04-25T09:07:24.987628+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987608+0000","last_change":"2024-04-25T09:07:24.987608+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987608+0000","last_undegraded":"2024-04-25T09:07:24.987608+0000","last_fullsized":"2024-04-25T09:07:24.987608+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989974+0000","last_change":"2024-04-25T09:07:24.989974+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989974+0000","last_undegraded":"2024-04-25T09:07:24.989974+0000","last_fullsized":"2024-04-25T09:07:24.989974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995869+0000","last_change":"2024-04-25T09:07:24.995869+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995869+0000","last_undegraded":"2024-04-25T09:07:24.995869+0000","last_fullsized":"2024-04-25T09:07:24.995869+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.007025+0000","last_change":"2024-04-25T09:07:25.007025+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.007025+0000","last_undegraded":"2024-04-25T09:07:25.007025+0000","last_fullsized":"2024-04-25T09:07:25.007025+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993999+0000","last_change":"2024-04-25T09:07:24.993999+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993999+0000","last_undegraded":"2024-04-25T09:07:24.993999+0000","last_fullsized":"2024-04-25T09:07:24.993999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996751+0000","last_change":"2024-04-25T09:07:24.996751+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996751+0000","last_undegraded":"2024-04-25T09:07:24.996751+0000","last_fullsized":"2024-04-25T09:07:24.996751+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990286+0000","last_change":"2024-04-25T09:07:24.990286+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990286+0000","last_undegraded":"2024-04-25T09:07:24.990286+0000","last_fullsized":"2024-04-25T09:07:24.990286+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.990066+0000","last_change":"2024-04-25T09:07:24.990066+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.990066+0000","last_undegraded":"2024-04-25T09:07:24.990066+0000","last_fullsized":"2024-04-25T09:07:24.990066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996794+0000","last_change":"2024-04-25T09:07:24.996794+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996794+0000","last_undegraded":"2024-04-25T09:07:24.996794+0000","last_fullsized":"2024-04-25T09:07:24.996794+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997739+0000","last_change":"2024-04-25T09:07:24.997739+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997739+0000","last_undegraded":"2024-04-25T09:07:24.997739+0000","last_fullsized":"2024-04-25T09:07:24.997739+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.987393+0000","last_change":"2024-04-25T09:07:24.987393+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.987393+0000","last_undegraded":"2024-04-25T09:07:24.987393+0000","last_fullsized":"2024-04-25T09:07:24.987393+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013200+0000","last_change":"2024-04-25T09:07:25.010479+0000","last_active":"2024-04-25T09:07:25.013200+0000","last_peered":"2024-04-25T09:07:25.013200+0000","last_clean":"2024-04-25T09:07:25.013200+0000","last_became_active":"2024-04-25T09:07:25.009971+0000","last_became_peered":"2024-04-25T09:07:25.009971+0000","last_unstale":"2024-04-25T09:07:25.013200+0000","last_undegraded":"2024-04-25T09:07:25.013200+0000","last_fullsized":"2024-04-25T09:07:25.013200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:58.222873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.995032+0000","last_change":"2024-04-25T09:07:24.995032+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.995032+0000","last_undegraded":"2024-04-25T09:07:24.995032+0000","last_fullsized":"2024-04-25T09:07:24.995032+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017081+0000","last_change":"2024-04-25T09:07:25.012693+0000","last_active":"2024-04-25T09:07:25.017081+0000","last_peered":"2024-04-25T09:07:25.017081+0000","last_clean":"2024-04-25T09:07:25.017081+0000","last_became_active":"2024-04-25T09:07:25.012366+0000","last_became_peered":"2024-04-25T09:07:25.012366+0000","last_unstale":"2024-04-25T09:07:25.017081+0000","last_undegraded":"2024-04-25T09:07:25.017081+0000","last_fullsized":"2024-04-25T09:07:25.017081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:42.922687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996781+0000","last_change":"2024-04-25T09:07:24.996781+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996781+0000","last_undegraded":"2024-04-25T09:07:24.996781+0000","last_fullsized":"2024-04-25T09:07:24.996781+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013434+0000","last_change":"2024-04-25T09:07:25.011946+0000","last_active":"2024-04-25T09:07:25.013434+0000","last_peered":"2024-04-25T09:07:25.013434+0000","last_clean":"2024-04-25T09:07:25.013434+0000","last_became_active":"2024-04-25T09:07:25.011630+0000","last_became_peered":"2024-04-25T09:07:25.011630+0000","last_unstale":"2024-04-25T09:07:25.013434+0000","last_undegraded":"2024-04-25T09:07:25.013434+0000","last_fullsized":"2024-04-25T09:07:25.013434+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:45.527811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994342+0000","last_change":"2024-04-25T09:07:24.994342+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994342+0000","last_undegraded":"2024-04-25T09:07:24.994342+0000","last_fullsized":"2024-04-25T09:07:24.994342+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.014473+0000","last_change":"2024-04-25T09:07:25.010814+0000","last_active":"2024-04-25T09:07:25.014473+0000","last_peered":"2024-04-25T09:07:25.014473+0000","last_clean":"2024-04-25T09:07:25.014473+0000","last_became_active":"2024-04-25T09:07:25.008979+0000","last_became_peered":"2024-04-25T09:07:25.008979+0000","last_unstale":"2024-04-25T09:07:25.014473+0000","last_undegraded":"2024-04-25T09:07:25.014473+0000","last_fullsized":"2024-04-25T09:07:25.014473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:50.056920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992079+0000","last_change":"2024-04-25T09:07:24.992079+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992079+0000","last_undegraded":"2024-04-25T09:07:24.992079+0000","last_fullsized":"2024-04-25T09:07:24.992079+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013153+0000","last_change":"2024-04-25T09:07:25.010295+0000","last_active":"2024-04-25T09:07:25.013153+0000","last_peered":"2024-04-25T09:07:25.013153+0000","last_clean":"2024-04-25T09:07:25.013153+0000","last_became_active":"2024-04-25T09:07:25.009984+0000","last_became_peered":"2024-04-25T09:07:25.009984+0000","last_unstale":"2024-04-25T09:07:25.013153+0000","last_undegraded":"2024-04-25T09:07:25.013153+0000","last_fullsized":"2024-04-25T09:07:25.013153+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:35.643627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.999831+0000","last_change":"2024-04-25T09:07:24.999831+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.999831+0000","last_undegraded":"2024-04-25T09:07:24.999831+0000","last_fullsized":"2024-04-25T09:07:24.999831+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010479+0000","last_change":"2024-04-25T09:07:25.005618+0000","last_active":"2024-04-25T09:07:25.010479+0000","last_peered":"2024-04-25T09:07:25.010479+0000","last_clean":"2024-04-25T09:07:25.010479+0000","last_became_active":"2024-04-25T09:07:25.005276+0000","last_became_peered":"2024-04-25T09:07:25.005276+0000","last_unstale":"2024-04-25T09:07:25.010479+0000","last_undegraded":"2024-04-25T09:07:25.010479+0000","last_fullsized":"2024-04-25T09:07:25.010479+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:39.155136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991651+0000","last_change":"2024-04-25T09:07:24.991651+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991651+0000","last_undegraded":"2024-04-25T09:07:24.991651+0000","last_fullsized":"2024-04-25T09:07:24.991651+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017467+0000","last_change":"2024-04-25T09:07:25.014784+0000","last_active":"2024-04-25T09:07:25.017467+0000","last_peered":"2024-04-25T09:07:25.017467+0000","last_clean":"2024-04-25T09:07:25.017467+0000","last_became_active":"2024-04-25T09:07:25.012907+0000","last_became_peered":"2024-04-25T09:07:25.012907+0000","last_unstale":"2024-04-25T09:07:25.017467+0000","last_undegraded":"2024-04-25T09:07:25.017467+0000","last_fullsized":"2024-04-25T09:07:25.017467+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.030750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.988372+0000","last_change":"2024-04-25T09:07:24.988372+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.988372+0000","last_undegraded":"2024-04-25T09:07:24.988372+0000","last_fullsized":"2024-04-25T09:07:24.988372+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015362+0000","last_change":"2024-04-25T09:07:25.009763+0000","last_active":"2024-04-25T09:07:25.015362+0000","last_peered":"2024-04-25T09:07:25.015362+0000","last_clean":"2024-04-25T09:07:25.015362+0000","last_became_active":"2024-04-25T09:07:25.009474+0000","last_became_peered":"2024-04-25T09:07:25.009474+0000","last_unstale":"2024-04-25T09:07:25.015362+0000","last_undegraded":"2024-04-25T09:07:25.015362+0000","last_fullsized":"2024-04-25T09:07:25.015362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:51.123149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996639+0000","last_change":"2024-04-25T09:07:24.996639+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996639+0000","last_undegraded":"2024-04-25T09:07:24.996639+0000","last_fullsized":"2024-04-25T09:07:24.996639+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015127+0000","last_change":"2024-04-25T09:07:25.007040+0000","last_active":"2024-04-25T09:07:25.015127+0000","last_peered":"2024-04-25T09:07:25.015127+0000","last_clean":"2024-04-25T09:07:25.015127+0000","last_became_active":"2024-04-25T09:07:25.006742+0000","last_became_peered":"2024-04-25T09:07:25.006742+0000","last_unstale":"2024-04-25T09:07:25.015127+0000","last_undegraded":"2024-04-25T09:07:25.015127+0000","last_fullsized":"2024-04-25T09:07:25.015127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:07.492703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993569+0000","last_change":"2024-04-25T09:07:24.993569+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993569+0000","last_undegraded":"2024-04-25T09:07:24.993569+0000","last_fullsized":"2024-04-25T09:07:24.993569+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018281+0000","last_change":"2024-04-25T09:07:25.016674+0000","last_active":"2024-04-25T09:07:25.018281+0000","last_peered":"2024-04-25T09:07:25.018281+0000","last_clean":"2024-04-25T09:07:25.018281+0000","last_became_active":"2024-04-25T09:07:25.016317+0000","last_became_peered":"2024-04-25T09:07:25.016317+0000","last_unstale":"2024-04-25T09:07:25.018281+0000","last_undegraded":"2024-04-25T09:07:25.018281+0000","last_fullsized":"2024-04-25T09:07:25.018281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:19.859538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.996661+0000","last_change":"2024-04-25T09:07:24.996661+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.996661+0000","last_undegraded":"2024-04-25T09:07:24.996661+0000","last_fullsized":"2024-04-25T09:07:24.996661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013680+0000","last_change":"2024-04-25T09:07:25.008303+0000","last_active":"2024-04-25T09:07:25.013680+0000","last_peered":"2024-04-25T09:07:25.013680+0000","last_clean":"2024-04-25T09:07:25.013680+0000","last_became_active":"2024-04-25T09:07:25.006381+0000","last_became_peered":"2024-04-25T09:07:25.006381+0000","last_unstale":"2024-04-25T09:07:25.013680+0000","last_undegraded":"2024-04-25T09:07:25.013680+0000","last_fullsized":"2024-04-25T09:07:25.013680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993591+0000","last_change":"2024-04-25T09:07:24.993591+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993591+0000","last_undegraded":"2024-04-25T09:07:24.993591+0000","last_fullsized":"2024-04-25T09:07:24.993591+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007328+0000","last_change":"2024-04-25T09:07:25.003604+0000","last_active":"2024-04-25T09:07:25.007328+0000","last_peered":"2024-04-25T09:07:25.007328+0000","last_clean":"2024-04-25T09:07:25.007328+0000","last_became_active":"2024-04-25T09:07:25.003266+0000","last_became_peered":"2024-04-25T09:07:25.003266+0000","last_unstale":"2024-04-25T09:07:25.007328+0000","last_undegraded":"2024-04-25T09:07:25.007328+0000","last_fullsized":"2024-04-25T09:07:25.007328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998031+0000","last_change":"2024-04-25T09:07:24.998031+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998031+0000","last_undegraded":"2024-04-25T09:07:24.998031+0000","last_fullsized":"2024-04-25T09:07:24.998031+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010140+0000","last_change":"2024-04-25T09:07:25.006123+0000","last_active":"2024-04-25T09:07:25.010140+0000","last_peered":"2024-04-25T09:07:25.010140+0000","last_clean":"2024-04-25T09:07:25.010140+0000","last_became_active":"2024-04-25T09:07:25.004723+0000","last_became_peered":"2024-04-25T09:07:25.004723+0000","last_unstale":"2024-04-25T09:07:25.010140+0000","last_undegraded":"2024-04-25T09:07:25.010140+0000","last_fullsized":"2024-04-25T09:07:25.010140+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:07.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986042+0000","last_change":"2024-04-25T09:07:24.986042+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986042+0000","last_undegraded":"2024-04-25T09:07:24.986042+0000","last_fullsized":"2024-04-25T09:07:24.986042+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007877+0000","last_change":"2024-04-25T09:07:25.004430+0000","last_active":"2024-04-25T09:07:25.007877+0000","last_peered":"2024-04-25T09:07:25.007877+0000","last_clean":"2024-04-25T09:07:25.007877+0000","last_became_active":"2024-04-25T09:07:25.000012+0000","last_became_peered":"2024-04-25T09:07:25.000012+0000","last_unstale":"2024-04-25T09:07:25.007877+0000","last_undegraded":"2024-04-25T09:07:25.007877+0000","last_fullsized":"2024-04-25T09:07:25.007877+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:13.587386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.985571+0000","last_change":"2024-04-25T09:07:24.985571+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.985571+0000","last_undegraded":"2024-04-25T09:07:24.985571+0000","last_fullsized":"2024-04-25T09:07:24.985571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017723+0000","last_change":"2024-04-25T09:07:25.013988+0000","last_active":"2024-04-25T09:07:25.017723+0000","last_peered":"2024-04-25T09:07:25.017723+0000","last_clean":"2024-04-25T09:07:25.017723+0000","last_became_active":"2024-04-25T09:07:25.013681+0000","last_became_peered":"2024-04-25T09:07:25.013681+0000","last_unstale":"2024-04-25T09:07:25.017723+0000","last_undegraded":"2024-04-25T09:07:25.017723+0000","last_fullsized":"2024-04-25T09:07:25.017723+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:48.565341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998624+0000","last_change":"2024-04-25T09:07:24.998624+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998624+0000","last_undegraded":"2024-04-25T09:07:24.998624+0000","last_fullsized":"2024-04-25T09:07:24.998624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.010072+0000","last_change":"2024-04-25T09:07:25.006626+0000","last_active":"2024-04-25T09:07:25.010072+0000","last_peered":"2024-04-25T09:07:25.010072+0000","last_clean":"2024-04-25T09:07:25.010072+0000","last_became_active":"2024-04-25T09:07:25.006323+0000","last_became_peered":"2024-04-25T09:07:25.006323+0000","last_unstale":"2024-04-25T09:07:25.010072+0000","last_undegraded":"2024-04-25T09:07:25.010072+0000","last_fullsized":"2024-04-25T09:07:25.010072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:36.534602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986282+0000","last_change":"2024-04-25T09:07:24.986282+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986282+0000","last_undegraded":"2024-04-25T09:07:24.986282+0000","last_fullsized":"2024-04-25T09:07:24.986282+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018288+0000","last_change":"2024-04-25T09:07:25.016157+0000","last_active":"2024-04-25T09:07:25.018288+0000","last_peered":"2024-04-25T09:07:25.018288+0000","last_clean":"2024-04-25T09:07:25.018288+0000","last_became_active":"2024-04-25T09:07:25.015731+0000","last_became_peered":"2024-04-25T09:07:25.015731+0000","last_unstale":"2024-04-25T09:07:25.018288+0000","last_undegraded":"2024-04-25T09:07:25.018288+0000","last_fullsized":"2024-04-25T09:07:25.018288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:30.570535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.986852+0000","last_change":"2024-04-25T09:07:24.986852+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.986852+0000","last_undegraded":"2024-04-25T09:07:24.986852+0000","last_fullsized":"2024-04-25T09:07:24.986852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015226+0000","last_change":"2024-04-25T09:07:25.010677+0000","last_active":"2024-04-25T09:07:25.015226+0000","last_peered":"2024-04-25T09:07:25.015226+0000","last_clean":"2024-04-25T09:07:25.015226+0000","last_became_active":"2024-04-25T09:07:25.010337+0000","last_became_peered":"2024-04-25T09:07:25.010337+0000","last_unstale":"2024-04-25T09:07:25.015226+0000","last_undegraded":"2024-04-25T09:07:25.015226+0000","last_fullsized":"2024-04-25T09:07:25.015226+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:37.745468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992003+0000","last_change":"2024-04-25T09:07:24.992003+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992003+0000","last_undegraded":"2024-04-25T09:07:24.992003+0000","last_fullsized":"2024-04-25T09:07:24.992003+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015417+0000","last_change":"2024-04-25T09:07:25.012576+0000","last_active":"2024-04-25T09:07:25.015417+0000","last_peered":"2024-04-25T09:07:25.015417+0000","last_clean":"2024-04-25T09:07:25.015417+0000","last_became_active":"2024-04-25T09:07:25.010949+0000","last_became_peered":"2024-04-25T09:07:25.010949+0000","last_unstale":"2024-04-25T09:07:25.015417+0000","last_undegraded":"2024-04-25T09:07:25.015417+0000","last_fullsized":"2024-04-25T09:07:25.015417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:07.269907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994462+0000","last_change":"2024-04-25T09:07:24.994462+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994462+0000","last_undegraded":"2024-04-25T09:07:24.994462+0000","last_fullsized":"2024-04-25T09:07:24.994462+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.017671+0000","last_change":"2024-04-25T09:07:25.013972+0000","last_active":"2024-04-25T09:07:25.017671+0000","last_peered":"2024-04-25T09:07:25.017671+0000","last_clean":"2024-04-25T09:07:25.017671+0000","last_became_active":"2024-04-25T09:07:25.013633+0000","last_became_peered":"2024-04-25T09:07:25.013633+0000","last_unstale":"2024-04-25T09:07:25.017671+0000","last_undegraded":"2024-04-25T09:07:25.017671+0000","last_fullsized":"2024-04-25T09:07:25.017671+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:47.797851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993010+0000","last_change":"2024-04-25T09:07:24.993010+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993010+0000","last_undegraded":"2024-04-25T09:07:24.993010+0000","last_fullsized":"2024-04-25T09:07:24.993010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018264+0000","last_change":"2024-04-25T09:07:25.016649+0000","last_active":"2024-04-25T09:07:25.018264+0000","last_peered":"2024-04-25T09:07:25.018264+0000","last_clean":"2024-04-25T09:07:25.018264+0000","last_became_active":"2024-04-25T09:07:25.016387+0000","last_became_peered":"2024-04-25T09:07:25.016387+0000","last_unstale":"2024-04-25T09:07:25.018264+0000","last_undegraded":"2024-04-25T09:07:25.018264+0000","last_fullsized":"2024-04-25T09:07:25.018264+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:09.601456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998397+0000","last_change":"2024-04-25T09:07:24.998397+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998397+0000","last_undegraded":"2024-04-25T09:07:24.998397+0000","last_fullsized":"2024-04-25T09:07:24.998397+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015086+0000","last_change":"2024-04-25T09:07:25.009598+0000","last_active":"2024-04-25T09:07:25.015086+0000","last_peered":"2024-04-25T09:07:25.015086+0000","last_clean":"2024-04-25T09:07:25.015086+0000","last_became_active":"2024-04-25T09:07:25.009294+0000","last_became_peered":"2024-04-25T09:07:25.009294+0000","last_unstale":"2024-04-25T09:07:25.015086+0000","last_undegraded":"2024-04-25T09:07:25.015086+0000","last_fullsized":"2024-04-25T09:07:25.015086+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:31.801129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.997245+0000","last_change":"2024-04-25T09:07:24.997245+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.997245+0000","last_undegraded":"2024-04-25T09:07:24.997245+0000","last_fullsized":"2024-04-25T09:07:24.997245+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.007711+0000","last_change":"2024-04-25T09:07:25.005217+0000","last_active":"2024-04-25T09:07:25.007711+0000","last_peered":"2024-04-25T09:07:25.007711+0000","last_clean":"2024-04-25T09:07:25.007711+0000","last_became_active":"2024-04-25T09:07:25.003714+0000","last_became_peered":"2024-04-25T09:07:25.003714+0000","last_unstale":"2024-04-25T09:07:25.007711+0000","last_undegraded":"2024-04-25T09:07:25.007711+0000","last_fullsized":"2024-04-25T09:07:25.007711+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:55.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.998226+0000","last_change":"2024-04-25T09:07:24.998226+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.998226+0000","last_undegraded":"2024-04-25T09:07:24.998226+0000","last_fullsized":"2024-04-25T09:07:24.998226+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.005589+0000","last_change":"2024-04-25T09:07:25.003186+0000","last_active":"2024-04-25T09:07:25.005589+0000","last_peered":"2024-04-25T09:07:25.005589+0000","last_clean":"2024-04-25T09:07:25.005589+0000","last_became_active":"2024-04-25T09:07:25.002435+0000","last_became_peered":"2024-04-25T09:07:25.002435+0000","last_unstale":"2024-04-25T09:07:25.005589+0000","last_undegraded":"2024-04-25T09:07:25.005589+0000","last_fullsized":"2024-04-25T09:07:25.005589+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:22.212413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.994941+0000","last_change":"2024-04-25T09:07:24.994941+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.994941+0000","last_undegraded":"2024-04-25T09:07:24.994941+0000","last_fullsized":"2024-04-25T09:07:24.994941+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.018229+0000","last_change":"2024-04-25T09:07:25.015721+0000","last_active":"2024-04-25T09:07:25.018229+0000","last_peered":"2024-04-25T09:07:25.018229+0000","last_clean":"2024-04-25T09:07:25.018229+0000","last_became_active":"2024-04-25T09:07:25.015220+0000","last_became_peered":"2024-04-25T09:07:25.015220+0000","last_unstale":"2024-04-25T09:07:25.018229+0000","last_undegraded":"2024-04-25T09:07:25.018229+0000","last_fullsized":"2024-04-25T09:07:25.018229+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:31.801129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.991242+0000","last_change":"2024-04-25T09:07:24.991242+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.991242+0000","last_undegraded":"2024-04-25T09:07:24.991242+0000","last_fullsized":"2024-04-25T09:07:24.991242+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.015029+0000","last_change":"2024-04-25T09:07:25.010776+0000","last_active":"2024-04-25T09:07:25.015029+0000","last_peered":"2024-04-25T09:07:25.015029+0000","last_clean":"2024-04-25T09:07:25.015029+0000","last_became_active":"2024-04-25T09:07:25.010280+0000","last_became_peered":"2024-04-25T09:07:25.010280+0000","last_unstale":"2024-04-25T09:07:25.015029+0000","last_undegraded":"2024-04-25T09:07:25.015029+0000","last_fullsized":"2024-04-25T09:07:25.015029+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:30.570535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993158+0000","last_change":"2024-04-25T09:07:24.993158+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993158+0000","last_undegraded":"2024-04-25T09:07:24.993158+0000","last_fullsized":"2024-04-25T09:07:24.993158+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.012361+0000","last_change":"2024-04-25T09:07:25.007229+0000","last_active":"2024-04-25T09:07:25.012361+0000","last_peered":"2024-04-25T09:07:25.012361+0000","last_clean":"2024-04-25T09:07:25.012361+0000","last_became_active":"2024-04-25T09:07:25.006847+0000","last_became_peered":"2024-04-25T09:07:25.006847+0000","last_unstale":"2024-04-25T09:07:25.012361+0000","last_undegraded":"2024-04-25T09:07:25.012361+0000","last_fullsized":"2024-04-25T09:07:25.012361+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:26.558822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.989296+0000","last_change":"2024-04-25T09:07:24.989296+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.989296+0000","last_undegraded":"2024-04-25T09:07:24.989296+0000","last_fullsized":"2024-04-25T09:07:24.989296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.016762+0000","last_change":"2024-04-25T09:07:25.011689+0000","last_active":"2024-04-25T09:07:25.016762+0000","last_peered":"2024-04-25T09:07:25.016762+0000","last_clean":"2024-04-25T09:07:25.016762+0000","last_became_active":"2024-04-25T09:07:25.011388+0000","last_became_peered":"2024-04-25T09:07:25.011388+0000","last_unstale":"2024-04-25T09:07:25.016762+0000","last_undegraded":"2024-04-25T09:07:25.016762+0000","last_fullsized":"2024-04-25T09:07:25.016762+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:31.406996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.007613+0000","last_change":"2024-04-25T09:07:25.007613+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.007613+0000","last_undegraded":"2024-04-25T09:07:25.007613+0000","last_fullsized":"2024-04-25T09:07:25.007613+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.005467+0000","last_change":"2024-04-25T09:07:25.004275+0000","last_active":"2024-04-25T09:07:25.005467+0000","last_peered":"2024-04-25T09:07:25.005467+0000","last_clean":"2024-04-25T09:07:25.005467+0000","last_became_active":"2024-04-25T09:07:25.004002+0000","last_became_peered":"2024-04-25T09:07:25.004002+0000","last_unstale":"2024-04-25T09:07:25.005467+0000","last_undegraded":"2024-04-25T09:07:25.005467+0000","last_fullsized":"2024-04-25T09:07:25.005467+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:33.025712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.993341+0000","last_change":"2024-04-25T09:07:24.993341+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.993341+0000","last_undegraded":"2024-04-25T09:07:24.993341+0000","last_fullsized":"2024-04-25T09:07:24.993341+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.013092+0000","last_change":"2024-04-25T09:07:25.007837+0000","last_active":"2024-04-25T09:07:25.013092+0000","last_peered":"2024-04-25T09:07:25.013092+0000","last_clean":"2024-04-25T09:07:25.013092+0000","last_became_active":"2024-04-25T09:07:25.007537+0000","last_became_peered":"2024-04-25T09:07:25.007537+0000","last_unstale":"2024-04-25T09:07:25.013092+0000","last_undegraded":"2024-04-25T09:07:25.013092+0000","last_fullsized":"2024-04-25T09:07:25.013092+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.053815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:24.992440+0000","last_change":"2024-04-25T09:07:24.992440+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:24.992440+0000","last_undegraded":"2024-04-25T09:07:24.992440+0000","last_fullsized":"2024-04-25T09:07:24.992440+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.006431+0000","last_change":"2024-04-25T09:07:25.002676+0000","last_active":"2024-04-25T09:07:25.006431+0000","last_peered":"2024-04-25T09:07:25.006431+0000","last_clean":"2024-04-25T09:07:25.006431+0000","last_became_active":"2024-04-25T09:07:25.001512+0000","last_became_peered":"2024-04-25T09:07:25.001512+0000","last_unstale":"2024-04-25T09:07:25.006431+0000","last_undegraded":"2024-04-25T09:07:25.006431+0000","last_fullsized":"2024-04-25T09:07:25.006431+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:37.745468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-04-25T09:07:25.004957+0000","last_change":"2024-04-25T09:07:25.004957+0000","last_active":"2024-04-25T09:07:24.961079+0000","last_peered":"2024-04-25T09:07:24.961079+0000","last_clean":"2024-04-25T09:07:24.961079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:07:25.004957+0000","last_undegraded":"2024-04-25T09:07:25.004957+0000","last_fullsized":"2024-04-25T09:07:25.004957+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.019379+0000","last_change":"2024-04-25T09:07:25.014785+0000","last_active":"2024-04-25T09:07:25.019379+0000","last_peered":"2024-04-25T09:07:25.019379+0000","last_clean":"2024-04-25T09:07:25.019379+0000","last_became_active":"2024-04-25T09:07:25.013321+0000","last_became_peered":"2024-04-25T09:07:25.013321+0000","last_unstale":"2024-04-25T09:07:25.019379+0000","last_undegraded":"2024-04-25T09:07:25.019379+0000","last_fullsized":"2024-04-25T09:07:25.019379+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:09.368176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T09:07:25.004963+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:07:25.004963+0000","last_peered":"2024-04-25T09:07:25.004963+0000","last_clean":"2024-04-25T09:07:25.004963+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:07:25.004963+0000","last_undegraded":"2024-04-25T09:07:25.004963+0000","last_fullsized":"2024-04-25T09:07:25.004963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35868,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335972,"statfs":{"total":96636764160,"available":96600035328,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35872,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335968,"statfs":{"total":96636764160,"available":96600031232,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1134592,"data_stored":811678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40028,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331812,"statfs":{"total":96636764160,"available":96595775488,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36452,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335388,"statfs":{"total":96636764160,"available":96599437312,"internally_reserved":0,"allocated":1134592,"data_stored":811678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":540672,"data_stored":221310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:07:31.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-25T09:07:31.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.996639+0000', 'last_change': '2024-04-25T09:07:24.996639+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.996639+0000', 'last_undegraded': '2024-04-25T09:07:24.996639+0000', 'last_fullsized': '2024-04-25T09:07:24.996639+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:07:31.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-25T09:07:31.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.990524+0000', 'last_change': '2024-04-25T09:07:24.990524+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.990524+0000', 'last_undegraded': '2024-04-25T09:07:24.990524+0000', 'last_fullsized': '2024-04-25T09:07:24.990524+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-25T09:07:31.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.995074+0000', 'last_change': '2024-04-25T09:07:24.995074+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.995074+0000', 'last_undegraded': '2024-04-25T09:07:24.995074+0000', 'last_fullsized': '2024-04-25T09:07:24.995074+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.997350+0000', 'last_change': '2024-04-25T09:07:24.997350+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.997350+0000', 'last_undegraded': '2024-04-25T09:07:24.997350+0000', 'last_fullsized': '2024-04-25T09:07:24.997350+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.991559+0000', 'last_change': '2024-04-25T09:07:24.991559+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.991559+0000', 'last_undegraded': '2024-04-25T09:07:24.991559+0000', 'last_fullsized': '2024-04-25T09:07:24.991559+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:25.002084+0000', 'last_change': '2024-04-25T09:07:25.002084+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:25.002084+0000', 'last_undegraded': '2024-04-25T09:07:25.002084+0000', 'last_fullsized': '2024-04-25T09:07:25.002084+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.990527+0000', 'last_change': '2024-04-25T09:07:24.990527+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.990527+0000', 'last_undegraded': '2024-04-25T09:07:24.990527+0000', 'last_fullsized': '2024-04-25T09:07:24.990527+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.998301+0000', 'last_change': '2024-04-25T09:07:24.998301+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.998301+0000', 'last_undegraded': '2024-04-25T09:07:24.998301+0000', 'last_fullsized': '2024-04-25T09:07:24.998301+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-25T09:07:31.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.997181+0000', 'last_change': '2024-04-25T09:07:24.997181+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.997181+0000', 'last_undegraded': '2024-04-25T09:07:24.997181+0000', 'last_fullsized': '2024-04-25T09:07:24.997181+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.991564+0000', 'last_change': '2024-04-25T09:07:24.991564+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.991564+0000', 'last_undegraded': '2024-04-25T09:07:24.991564+0000', 'last_fullsized': '2024-04-25T09:07:24.991564+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:25.005646+0000', 'last_change': '2024-04-25T09:07:25.005646+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:25.005646+0000', 'last_undegraded': '2024-04-25T09:07:25.005646+0000', 'last_fullsized': '2024-04-25T09:07:25.005646+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.997243+0000', 'last_change': '2024-04-25T09:07:24.997243+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.997243+0000', 'last_undegraded': '2024-04-25T09:07:24.997243+0000', 'last_fullsized': '2024-04-25T09:07:24.997243+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:25.002332+0000', 'last_change': '2024-04-25T09:07:25.002332+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:25.002332+0000', 'last_undegraded': '2024-04-25T09:07:25.002332+0000', 'last_fullsized': '2024-04-25T09:07:25.002332+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.994369+0000', 'last_change': '2024-04-25T09:07:24.994369+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.994369+0000', 'last_undegraded': '2024-04-25T09:07:24.994369+0000', 'last_fullsized': '2024-04-25T09:07:24.994369+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.998804+0000', 'last_change': '2024-04-25T09:07:24.998804+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.998804+0000', 'last_undegraded': '2024-04-25T09:07:24.998804+0000', 'last_fullsized': '2024-04-25T09:07:24.998804+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.987193+0000', 'last_change': '2024-04-25T09:07:24.987193+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.987193+0000', 'last_undegraded': '2024-04-25T09:07:24.987193+0000', 'last_fullsized': '2024-04-25T09:07:24.987193+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.991010+0000', 'last_change': '2024-04-25T09:07:24.991010+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.991010+0000', 'last_undegraded': '2024-04-25T09:07:24.991010+0000', 'last_fullsized': '2024-04-25T09:07:24.991010+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.998751+0000', 'last_change': '2024-04-25T09:07:24.998751+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.998751+0000', 'last_undegraded': '2024-04-25T09:07:24.998751+0000', 'last_fullsized': '2024-04-25T09:07:24.998751+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.989327+0000', 'last_change': '2024-04-25T09:07:24.989327+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.989327+0000', 'last_undegraded': '2024-04-25T09:07:24.989327+0000', 'last_fullsized': '2024-04-25T09:07:24.989327+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.993017+0000', 'last_change': '2024-04-25T09:07:24.993017+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.993017+0000', 'last_undegraded': '2024-04-25T09:07:24.993017+0000', 'last_fullsized': '2024-04-25T09:07:24.993017+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:07:31.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:25.003843+0000', 'last_change': '2024-04-25T09:07:25.003843+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:25.003843+0000', 'last_undegraded': '2024-04-25T09:07:25.003843+0000', 'last_fullsized': '2024-04-25T09:07:25.003843+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.987628+0000', 'last_change': '2024-04-25T09:07:24.987628+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.987628+0000', 'last_undegraded': '2024-04-25T09:07:24.987628+0000', 'last_fullsized': '2024-04-25T09:07:24.987628+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.987608+0000', 'last_change': '2024-04-25T09:07:24.987608+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.987608+0000', 'last_undegraded': '2024-04-25T09:07:24.987608+0000', 'last_fullsized': '2024-04-25T09:07:24.987608+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.989974+0000', 'last_change': '2024-04-25T09:07:24.989974+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.989974+0000', 'last_undegraded': '2024-04-25T09:07:24.989974+0000', 'last_fullsized': '2024-04-25T09:07:24.989974+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.995869+0000', 'last_change': '2024-04-25T09:07:24.995869+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.995869+0000', 'last_undegraded': '2024-04-25T09:07:24.995869+0000', 'last_fullsized': '2024-04-25T09:07:24.995869+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-25T09:07:31.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:25.007025+0000', 'last_change': '2024-04-25T09:07:25.007025+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:25.007025+0000', 'last_undegraded': '2024-04-25T09:07:25.007025+0000', 'last_fullsized': '2024-04-25T09:07:25.007025+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.993999+0000', 'last_change': '2024-04-25T09:07:24.993999+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.993999+0000', 'last_undegraded': '2024-04-25T09:07:24.993999+0000', 'last_fullsized': '2024-04-25T09:07:24.993999+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.996751+0000', 'last_change': '2024-04-25T09:07:24.996751+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.996751+0000', 'last_undegraded': '2024-04-25T09:07:24.996751+0000', 'last_fullsized': '2024-04-25T09:07:24.996751+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.990286+0000', 'last_change': '2024-04-25T09:07:24.990286+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.990286+0000', 'last_undegraded': '2024-04-25T09:07:24.990286+0000', 'last_fullsized': '2024-04-25T09:07:24.990286+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.990066+0000', 'last_change': '2024-04-25T09:07:24.990066+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.990066+0000', 'last_undegraded': '2024-04-25T09:07:24.990066+0000', 'last_fullsized': '2024-04-25T09:07:24.990066+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.996794+0000', 'last_change': '2024-04-25T09:07:24.996794+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.996794+0000', 'last_undegraded': '2024-04-25T09:07:24.996794+0000', 'last_fullsized': '2024-04-25T09:07:24.996794+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-25T09:07:31.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.997739+0000', 'last_change': '2024-04-25T09:07:24.997739+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.997739+0000', 'last_undegraded': '2024-04-25T09:07:24.997739+0000', 'last_fullsized': '2024-04-25T09:07:24.997739+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.987393+0000', 'last_change': '2024-04-25T09:07:24.987393+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.987393+0000', 'last_undegraded': '2024-04-25T09:07:24.987393+0000', 'last_fullsized': '2024-04-25T09:07:24.987393+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.995032+0000', 'last_change': '2024-04-25T09:07:24.995032+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.995032+0000', 'last_undegraded': '2024-04-25T09:07:24.995032+0000', 'last_fullsized': '2024-04-25T09:07:24.995032+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.996781+0000', 'last_change': '2024-04-25T09:07:24.996781+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.996781+0000', 'last_undegraded': '2024-04-25T09:07:24.996781+0000', 'last_fullsized': '2024-04-25T09:07:24.996781+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.994342+0000', 'last_change': '2024-04-25T09:07:24.994342+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.994342+0000', 'last_undegraded': '2024-04-25T09:07:24.994342+0000', 'last_fullsized': '2024-04-25T09:07:24.994342+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.992079+0000', 'last_change': '2024-04-25T09:07:24.992079+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.992079+0000', 'last_undegraded': '2024-04-25T09:07:24.992079+0000', 'last_fullsized': '2024-04-25T09:07:24.992079+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-25T09:07:31.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.999831+0000', 'last_change': '2024-04-25T09:07:24.999831+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.999831+0000', 'last_undegraded': '2024-04-25T09:07:24.999831+0000', 'last_fullsized': '2024-04-25T09:07:24.999831+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.991651+0000', 'last_change': '2024-04-25T09:07:24.991651+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.991651+0000', 'last_undegraded': '2024-04-25T09:07:24.991651+0000', 'last_fullsized': '2024-04-25T09:07:24.991651+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.988372+0000', 'last_change': '2024-04-25T09:07:24.988372+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.988372+0000', 'last_undegraded': '2024-04-25T09:07:24.988372+0000', 'last_fullsized': '2024-04-25T09:07:24.988372+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.996639+0000', 'last_change': '2024-04-25T09:07:24.996639+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.996639+0000', 'last_undegraded': '2024-04-25T09:07:24.996639+0000', 'last_fullsized': '2024-04-25T09:07:24.996639+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.993569+0000', 'last_change': '2024-04-25T09:07:24.993569+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.993569+0000', 'last_undegraded': '2024-04-25T09:07:24.993569+0000', 'last_fullsized': '2024-04-25T09:07:24.993569+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T09:07:31.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.996661+0000', 'last_change': '2024-04-25T09:07:24.996661+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.996661+0000', 'last_undegraded': '2024-04-25T09:07:24.996661+0000', 'last_fullsized': '2024-04-25T09:07:24.996661+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.993591+0000', 'last_change': '2024-04-25T09:07:24.993591+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.993591+0000', 'last_undegraded': '2024-04-25T09:07:24.993591+0000', 'last_fullsized': '2024-04-25T09:07:24.993591+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.998031+0000', 'last_change': '2024-04-25T09:07:24.998031+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.998031+0000', 'last_undegraded': '2024-04-25T09:07:24.998031+0000', 'last_fullsized': '2024-04-25T09:07:24.998031+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.986042+0000', 'last_change': '2024-04-25T09:07:24.986042+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.986042+0000', 'last_undegraded': '2024-04-25T09:07:24.986042+0000', 'last_fullsized': '2024-04-25T09:07:24.986042+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.985571+0000', 'last_change': '2024-04-25T09:07:24.985571+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.985571+0000', 'last_undegraded': '2024-04-25T09:07:24.985571+0000', 'last_fullsized': '2024-04-25T09:07:24.985571+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.998624+0000', 'last_change': '2024-04-25T09:07:24.998624+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.998624+0000', 'last_undegraded': '2024-04-25T09:07:24.998624+0000', 'last_fullsized': '2024-04-25T09:07:24.998624+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-25T09:07:31.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.986282+0000', 'last_change': '2024-04-25T09:07:24.986282+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.986282+0000', 'last_undegraded': '2024-04-25T09:07:24.986282+0000', 'last_fullsized': '2024-04-25T09:07:24.986282+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.986852+0000', 'last_change': '2024-04-25T09:07:24.986852+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.986852+0000', 'last_undegraded': '2024-04-25T09:07:24.986852+0000', 'last_fullsized': '2024-04-25T09:07:24.986852+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.992003+0000', 'last_change': '2024-04-25T09:07:24.992003+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.992003+0000', 'last_undegraded': '2024-04-25T09:07:24.992003+0000', 'last_fullsized': '2024-04-25T09:07:24.992003+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.994462+0000', 'last_change': '2024-04-25T09:07:24.994462+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.994462+0000', 'last_undegraded': '2024-04-25T09:07:24.994462+0000', 'last_fullsized': '2024-04-25T09:07:24.994462+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.993010+0000', 'last_change': '2024-04-25T09:07:24.993010+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.993010+0000', 'last_undegraded': '2024-04-25T09:07:24.993010+0000', 'last_fullsized': '2024-04-25T09:07:24.993010+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.998397+0000', 'last_change': '2024-04-25T09:07:24.998397+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.998397+0000', 'last_undegraded': '2024-04-25T09:07:24.998397+0000', 'last_fullsized': '2024-04-25T09:07:24.998397+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-25T09:07:31.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.997245+0000', 'last_change': '2024-04-25T09:07:24.997245+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.997245+0000', 'last_undegraded': '2024-04-25T09:07:24.997245+0000', 'last_fullsized': '2024-04-25T09:07:24.997245+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.998226+0000', 'last_change': '2024-04-25T09:07:24.998226+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.998226+0000', 'last_undegraded': '2024-04-25T09:07:24.998226+0000', 'last_fullsized': '2024-04-25T09:07:24.998226+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.994941+0000', 'last_change': '2024-04-25T09:07:24.994941+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.994941+0000', 'last_undegraded': '2024-04-25T09:07:24.994941+0000', 'last_fullsized': '2024-04-25T09:07:24.994941+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.991242+0000', 'last_change': '2024-04-25T09:07:24.991242+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.991242+0000', 'last_undegraded': '2024-04-25T09:07:24.991242+0000', 'last_fullsized': '2024-04-25T09:07:24.991242+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.993158+0000', 'last_change': '2024-04-25T09:07:24.993158+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.993158+0000', 'last_undegraded': '2024-04-25T09:07:24.993158+0000', 'last_fullsized': '2024-04-25T09:07:24.993158+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.989296+0000', 'last_change': '2024-04-25T09:07:24.989296+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.989296+0000', 'last_undegraded': '2024-04-25T09:07:24.989296+0000', 'last_fullsized': '2024-04-25T09:07:24.989296+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T09:07:31.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:25.007613+0000', 'last_change': '2024-04-25T09:07:25.007613+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:25.007613+0000', 'last_undegraded': '2024-04-25T09:07:25.007613+0000', 'last_fullsized': '2024-04-25T09:07:25.007613+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-25T09:07:31.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.993341+0000', 'last_change': '2024-04-25T09:07:24.993341+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.993341+0000', 'last_undegraded': '2024-04-25T09:07:24.993341+0000', 'last_fullsized': '2024-04-25T09:07:24.993341+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:07:31.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-25T09:07:31.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:24.992440+0000', 'last_change': '2024-04-25T09:07:24.992440+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:24.992440+0000', 'last_undegraded': '2024-04-25T09:07:24.992440+0000', 'last_fullsized': '2024-04-25T09:07:24.992440+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:07:31.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-25T09:07:31.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:07:25.004957+0000', 'last_change': '2024-04-25T09:07:25.004957+0000', 'last_active': '2024-04-25T09:07:24.961079+0000', 'last_peered': '2024-04-25T09:07:24.961079+0000', 'last_clean': '2024-04-25T09:07:24.961079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:07:25.004957+0000', 'last_undegraded': '2024-04-25T09:07:25.004957+0000', 'last_fullsized': '2024-04-25T09:07:25.004957+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'last_clean_scrub_stamp': '2024-04-25T09:07:24.961079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:07:31.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:07:31.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.738+0000 7fdd47200640 1 Processor -- start 2024-04-25T09:07:31.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.739+0000 7fdd47200640 1 -- start start 2024-04-25T09:07:31.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.740+0000 7fdd47200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdd4815df40 0x7fdd480d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.740+0000 7fdd47200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd480b9750 0x7fdd480d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.740+0000 7fdd47200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd480d2690 con 0x7fdd4815db60 2024-04-25T09:07:31.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.740+0000 7fdd47200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd480b7c20 con 0x7fdd480b9750 2024-04-25T09:07:31.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.740+0000 7fdd47200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd480d2970 con 0x7fdd4815df40 2024-04-25T09:07:31.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.740+0000 7fdd46800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd480b9750 0x7fdd480d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.740+0000 7fdd46800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd480b9750 0x7fdd480d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33220/0 (socket says 172.21.15.62:33220) 2024-04-25T09:07:31.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.740+0000 7fdd46800640 1 -- 172.21.15.62:0/3696515298 learned_addr learned my addr 172.21.15.62:0/3696515298 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:31.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.741+0000 7fdd3d400640 1 --2- 172.21.15.62:0/3696515298 >> v2:172.21.15.121:3300/0 conn(0x7fdd4815df40 0x7fdd480d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.741+0000 7fdd46800640 1 -- 172.21.15.62:0/3696515298 >> v2:172.21.15.121:3300/0 conn(0x7fdd4815df40 msgr2=0x7fdd480d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:31.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.741+0000 7fdd46800640 1 --2- 172.21.15.62:0/3696515298 >> v2:172.21.15.121:3300/0 conn(0x7fdd4815df40 0x7fdd480d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.741+0000 7fdd46800640 1 -- 172.21.15.62:0/3696515298 >> v1:172.21.15.121:6789/0 conn(0x7fdd4815db60 legacy=0x7fdd480d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:31.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.741+0000 7fdd46800640 1 -- 172.21.15.62:0/3696515298 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd480d2c50 con 0x7fdd480b9750 2024-04-25T09:07:31.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.741+0000 7fdd3d400640 1 --2- 172.21.15.62:0/3696515298 >> v2:172.21.15.121:3300/0 conn(0x7fdd4815df40 0x7fdd480d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:31.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.742+0000 7fdd46800640 1 --2- 172.21.15.62:0/3696515298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd480b9750 0x7fdd480d0830 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdd480cdf30 tx=0x7fdd4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202215c03afc2dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:31.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.742+0000 7fdd45400640 1 -- 172.21.15.62:0/3696515298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40036070 con 0x7fdd480b9750 2024-04-25T09:07:31.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.742+0000 7fdd45400640 1 -- 172.21.15.62:0/3696515298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd40031600 con 0x7fdd480b9750 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.742+0000 7fdd45400640 1 -- 172.21.15.62:0/3696515298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd400319d0 con 0x7fdd480b9750 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.743+0000 7fdd47200640 1 -- 172.21.15.62:0/3696515298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd480b9750 msgr2=0x7fdd480d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.743+0000 7fdd47200640 1 --2- 172.21.15.62:0/3696515298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd480b9750 0x7fdd480d0830 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdd480cdf30 tx=0x7fdd4002a660 comp rx=0 tx=0).stop 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.743+0000 7fdd47200640 1 -- 172.21.15.62:0/3696515298 shutdown_connections 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.743+0000 7fdd47200640 1 --2- 172.21.15.62:0/3696515298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd480b9750 0x7fdd480d0830 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.743+0000 7fdd47200640 1 --2- 172.21.15.62:0/3696515298 >> v2:172.21.15.121:3300/0 conn(0x7fdd4815df40 0x7fdd480d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.743+0000 7fdd47200640 1 -- 172.21.15.62:0/3696515298 >> 172.21.15.62:0/3696515298 conn(0x7fdd4806b210 msgr2=0x7fdd480668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.743+0000 7fdd47200640 1 -- 172.21.15.62:0/3696515298 shutdown_connections 2024-04-25T09:07:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.743+0000 7fdd47200640 1 -- 172.21.15.62:0/3696515298 wait complete. 2024-04-25T09:07:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.745+0000 7fdd47200640 1 Processor -- start 2024-04-25T09:07:31.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.745+0000 7fdd47200640 1 -- start start 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.746+0000 7fdd47200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd480b9750 0x7fdd481f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.746+0000 7fdd47200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4815db60 0x7fdd480c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.746+0000 7fdd47200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd4815df40 0x7fdd480c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.746+0000 7fdd47200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480d3890 con 0x7fdd4815db60 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.746+0000 7fdd47200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480d3710 con 0x7fdd4815df40 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.746+0000 7fdd47200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480b6c10 con 0x7fdd480b9750 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd3d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4815db60 0x7fdd480c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd3d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4815db60 0x7fdd480c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33232/0 (socket says 172.21.15.62:33232) 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd3d400640 1 -- 172.21.15.62:0/2647737000 learned_addr learned my addr 172.21.15.62:0/2647737000 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd45e00640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd480b9750 0x7fdd481f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd46800640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd4815df40 0x7fdd480c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd3d400640 1 -- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd480b9750 msgr2=0x7fdd481f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd3d400640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd480b9750 0x7fdd481f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd3d400640 1 -- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd4815df40 msgr2=0x7fdd480c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd3d400640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd4815df40 0x7fdd480c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.747+0000 7fdd3d400640 1 -- 172.21.15.62:0/2647737000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd481e3e00 con 0x7fdd4815db60 2024-04-25T09:07:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.748+0000 7fdd3d400640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4815db60 0x7fdd480c8750 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000c820 tx=0x7fdd3000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:31.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.748+0000 7fdd3ea00640 1 -- 172.21.15.62:0/2647737000 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30015070 con 0x7fdd4815db60 2024-04-25T09:07:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.748+0000 7fdd3ea00640 1 -- 172.21.15.62:0/2647737000 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd30007a30 con 0x7fdd4815db60 2024-04-25T09:07:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.748+0000 7fdd3ea00640 1 -- 172.21.15.62:0/2647737000 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30007e20 con 0x7fdd4815db60 2024-04-25T09:07:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.748+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd481e40e0 con 0x7fdd4815db60 2024-04-25T09:07:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.748+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd48146b20 con 0x7fdd4815db60 2024-04-25T09:07:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.749+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd48072910 con 0x7fdd4815db60 2024-04-25T09:07:31.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.750+0000 7fdd3ea00640 1 -- 172.21.15.62:0/2647737000 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd30018070 con 0x7fdd4815db60 2024-04-25T09:07:31.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.751+0000 7fdd3ea00640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b4d0 0x7fdd2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:31.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.751+0000 7fdd3ea00640 1 -- 172.21.15.62:0/2647737000 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd30089d10 con 0x7fdd4815db60 2024-04-25T09:07:31.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.752+0000 7fdd45e00640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b4d0 0x7fdd2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:31.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.753+0000 7fdd45e00640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b4d0 0x7fdd2406d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd38004ea0 tx=0x7fdd38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:31.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:31.757+0000 7fdd3ea00640 1 -- 172.21.15.62:0/2647737000 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd30058ba0 con 0x7fdd4815db60 2024-04-25T09:07:31.930 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-04-25T09:07:31.942+0000 7f670cc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:07:32.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.036+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdd480750e0 con 0x7fdd4815db60 2024-04-25T09:07:32.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.037+0000 7fdd3ea00640 1 -- 172.21.15.62:0/2647737000 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1013 (secure 0 0 0) 0x7fdd300584a0 con 0x7fdd4815db60 2024-04-25T09:07:32.025 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:32.025 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":279,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":308092928,"bytes_avail":772786020352,"bytes_total":773094113280,"write_bytes_sec":2046,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5559}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-25T09:07:11.972752+0000","services":{}},"progress_events":{}} 2024-04-25T09:07:32.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b4d0 msgr2=0x7fdd2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:32.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b4d0 0x7fdd2406d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd38004ea0 tx=0x7fdd38009040 comp rx=0 tx=0).stop 2024-04-25T09:07:32.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4815db60 msgr2=0x7fdd480c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:32.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4815db60 0x7fdd480c8750 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000c820 tx=0x7fdd3000cce0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 shutdown_connections 2024-04-25T09:07:32.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdd2406b4d0 0x7fdd2406d990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd4815db60 0x7fdd480c8750 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd4815df40 0x7fdd480c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 --2- 172.21.15.62:0/2647737000 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdd480b9750 0x7fdd481f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.040+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 >> 172.21.15.62:0/2647737000 conn(0x7fdd4806b210 msgr2=0x7fdd480cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:32.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.041+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 shutdown_connections 2024-04-25T09:07:32.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.041+0000 7fdd47200640 1 -- 172.21.15.62:0/2647737000 wait complete. 2024-04-25T09:07:32.048 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:07:32.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:07:32.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.271+0000 7f1093000640 1 Processor -- start 2024-04-25T09:07:32.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.271+0000 7f1093000640 1 -- start start 2024-04-25T09:07:32.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.273+0000 7f1093000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c0cc490 0x7f108c15af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:32.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.273+0000 7f1093000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f108c06e310 0x7f108c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:32.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.273+0000 7f1093000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c070ed0 con 0x7f108c070980 2024-04-25T09:07:32.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.273+0000 7f1093000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108c0b50c0 con 0x7f108c06e310 2024-04-25T09:07:32.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.273+0000 7f1093000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0b4140 con 0x7f108c0cc490 2024-04-25T09:07:32.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.273+0000 7f1091c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c0cc490 0x7f108c15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:32.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.273+0000 7f1091c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c0cc490 0x7f108c15af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:51456/0 (socket says 172.21.15.62:51456) 2024-04-25T09:07:32.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.273+0000 7f1091c00640 1 -- 172.21.15.62:0/130453631 learned_addr learned my addr 172.21.15.62:0/130453631 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:32.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.274+0000 7f1092600640 1 --2- 172.21.15.62:0/130453631 >> v2:172.21.15.121:3300/0 conn(0x7f108c06e310 0x7f108c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:32.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.274+0000 7f1091c00640 1 -- 172.21.15.62:0/130453631 >> v2:172.21.15.121:3300/0 conn(0x7f108c06e310 msgr2=0x7f108c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:32.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.274+0000 7f1091c00640 1 --2- 172.21.15.62:0/130453631 >> v2:172.21.15.121:3300/0 conn(0x7f108c06e310 0x7f108c15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.274+0000 7f1091c00640 1 -- 172.21.15.62:0/130453631 >> v1:172.21.15.121:6789/0 conn(0x7f108c070980 legacy=0x7f108c06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:32.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.274+0000 7f1091c00640 1 -- 172.21.15.62:0/130453631 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c0711b0 con 0x7f108c0cc490 2024-04-25T09:07:32.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.274+0000 7f1092600640 1 --2- 172.21.15.62:0/130453631 >> v2:172.21.15.121:3300/0 conn(0x7f108c06e310 0x7f108c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:32.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.276+0000 7f1091c00640 1 --2- 172.21.15.62:0/130453631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c0cc490 0x7f108c15af40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f107c009e50 tx=0x7f107c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8b7262e7d7e95c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:32.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.276+0000 7f108be00640 1 -- 172.21.15.62:0/130453631 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c037070 con 0x7f108c0cc490 2024-04-25T09:07:32.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.276+0000 7f108be00640 1 -- 172.21.15.62:0/130453631 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c032600 con 0x7f108c0cc490 2024-04-25T09:07:32.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.276+0000 7f108be00640 1 -- 172.21.15.62:0/130453631 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0329d0 con 0x7f108c0cc490 2024-04-25T09:07:32.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.277+0000 7f1093000640 1 -- 172.21.15.62:0/130453631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c0cc490 msgr2=0x7f108c15af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:32.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.277+0000 7f1093000640 1 --2- 172.21.15.62:0/130453631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c0cc490 0x7f108c15af40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f107c009e50 tx=0x7f107c02b790 comp rx=0 tx=0).stop 2024-04-25T09:07:32.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.277+0000 7f1093000640 1 -- 172.21.15.62:0/130453631 shutdown_connections 2024-04-25T09:07:32.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.277+0000 7f1093000640 1 --2- 172.21.15.62:0/130453631 >> v2:172.21.15.121:3300/0 conn(0x7f108c06e310 0x7f108c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.277+0000 7f1093000640 1 --2- 172.21.15.62:0/130453631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c0cc490 0x7f108c15af40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.277+0000 7f1093000640 1 -- 172.21.15.62:0/130453631 >> 172.21.15.62:0/130453631 conn(0x7f108c0b1bf0 msgr2=0x7f108c0b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:32.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.277+0000 7f1093000640 1 -- 172.21.15.62:0/130453631 shutdown_connections 2024-04-25T09:07:32.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.277+0000 7f1093000640 1 -- 172.21.15.62:0/130453631 wait complete. 2024-04-25T09:07:32.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.280+0000 7f1093000640 1 Processor -- start 2024-04-25T09:07:32.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.280+0000 7f1093000640 1 -- start start 2024-04-25T09:07:32.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.281+0000 7f1093000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c06e310 0x7f108c140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:32.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.281+0000 7f1093000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c070980 0x7f108c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:32.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.281+0000 7f1093000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c0cc490 0x7f108c1410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.281+0000 7f1093000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c15b890 con 0x7f108c06e310 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.281+0000 7f1093000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c071770 con 0x7f108c0cc490 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.281+0000 7f1093000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0cc870 con 0x7f108c070980 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.281+0000 7f1091c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c06e310 0x7f108c140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.282+0000 7f1091c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c06e310 0x7f108c140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:33240/0 (socket says 172.21.15.62:33240) 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.282+0000 7f1091c00640 1 -- 172.21.15.62:0/702686196 learned_addr learned my addr 172.21.15.62:0/702686196 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.282+0000 7f1091c00640 1 -- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c070980 msgr2=0x7f108c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.282+0000 7f1091c00640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c070980 0x7f108c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.282+0000 7f1091c00640 1 -- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c0cc490 msgr2=0x7f108c1410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.282+0000 7f1091c00640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c0cc490 0x7f108c1410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.282+0000 7f1091c00640 1 -- 172.21.15.62:0/702686196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c152dd0 con 0x7f108c06e310 2024-04-25T09:07:32.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.282+0000 7f1091c00640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c06e310 0x7f108c140290 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f107c035040 tx=0x7f107c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:32.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.283+0000 7f1089600640 1 -- 172.21.15.62:0/702686196 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c037050 con 0x7f108c06e310 2024-04-25T09:07:32.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.283+0000 7f1089600640 1 -- 172.21.15.62:0/702686196 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c043070 con 0x7f108c06e310 2024-04-25T09:07:32.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.283+0000 7f1089600640 1 -- 172.21.15.62:0/702686196 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c009510 con 0x7f108c06e310 2024-04-25T09:07:32.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.283+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c152810 con 0x7f108c06e310 2024-04-25T09:07:32.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.283+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c142b20 con 0x7f108c06e310 2024-04-25T09:07:32.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.284+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c04ef20 con 0x7f108c06e310 2024-04-25T09:07:32.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.285+0000 7f1089600640 1 -- 172.21.15.62:0/702686196 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f107c007c50 con 0x7f108c06e310 2024-04-25T09:07:32.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.286+0000 7f1089600640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106c06b4d0 0x7f106c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:32.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.286+0000 7f1089600640 1 -- 172.21.15.62:0/702686196 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f107c0ab220 con 0x7f108c06e310 2024-04-25T09:07:32.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.286+0000 7f1091200640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106c06b4d0 0x7f106c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:32.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.288+0000 7f1091200640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106c06b4d0 0x7f106c06d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1078000f60 tx=0x7f10780079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:32.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.290+0000 7f1089600640 1 -- 172.21.15.62:0/702686196 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f107c07a120 con 0x7f108c06e310 2024-04-25T09:07:32.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.493+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f108c0726d0 con 0x7f106c06b4d0 2024-04-25T09:07:32.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.513+0000 7f1089600640 1 -- 172.21.15.62:0/702686196 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251959 (secure 0 0 0) 0x7f108c0726d0 con 0x7f106c06b4d0 2024-04-25T09:07:32.504 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:32.506 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:07:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106c06b4d0 msgr2=0x7f106c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106c06b4d0 0x7f106c06d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1078000f60 tx=0x7f10780079d0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c06e310 msgr2=0x7f108c140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c06e310 0x7f108c140290 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f107c035040 tx=0x7f107c004ab0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 shutdown_connections 2024-04-25T09:07:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106c06b4d0 0x7f106c06d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c0cc490 0x7f108c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f108c070980 0x7f108c06d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 --2- 172.21.15.62:0/702686196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f108c06e310 0x7f108c140290 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 >> 172.21.15.62:0/702686196 conn(0x7f108c0b1bf0 msgr2=0x7f108c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 shutdown_connections 2024-04-25T09:07:32.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:32.519+0000 7f1093000640 1 -- 172.21.15.62:0/702686196 wait complete. 2024-04-25T09:07:32.528 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-04-25T09:07:31.983929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300872,"kb_used_data":5640,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754673848,"statfs":{"total":773094113280,"available":772786020352,"internally_reserved":0,"allocated":5775360,"data_stored":3060636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.004205"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627354+0000","last_change":"2024-04-25T09:07:26.016714+0000","last_active":"2024-04-25T09:07:27.627354+0000","last_peered":"2024-04-25T09:07:27.627354+0000","last_clean":"2024-04-25T09:07:27.627354+0000","last_became_active":"2024-04-25T09:07:26.016368+0000","last_became_peered":"2024-04-25T09:07:26.016368+0000","last_unstale":"2024-04-25T09:07:27.627354+0000","last_undegraded":"2024-04-25T09:07:27.627354+0000","last_fullsized":"2024-04-25T09:07:27.627354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:11.998769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095724+0000","last_change":"2024-04-25T09:07:26.004394+0000","last_active":"2024-04-25T09:07:27.095724+0000","last_peered":"2024-04-25T09:07:27.095724+0000","last_clean":"2024-04-25T09:07:27.095724+0000","last_became_active":"2024-04-25T09:07:26.003907+0000","last_became_peered":"2024-04-25T09:07:26.003907+0000","last_unstale":"2024-04-25T09:07:27.095724+0000","last_undegraded":"2024-04-25T09:07:27.095724+0000","last_fullsized":"2024-04-25T09:07:27.095724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:25.097739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005043+0000","last_change":"2024-04-25T09:07:26.007063+0000","last_active":"2024-04-25T09:07:27.005043+0000","last_peered":"2024-04-25T09:07:27.005043+0000","last_clean":"2024-04-25T09:07:27.005043+0000","last_became_active":"2024-04-25T09:07:26.006193+0000","last_became_peered":"2024-04-25T09:07:26.006193+0000","last_unstale":"2024-04-25T09:07:27.005043+0000","last_undegraded":"2024-04-25T09:07:27.005043+0000","last_fullsized":"2024-04-25T09:07:27.005043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.581450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.013311+0000","last_change":"2024-04-25T09:07:26.019340+0000","last_active":"2024-04-25T09:07:27.013311+0000","last_peered":"2024-04-25T09:07:27.013311+0000","last_clean":"2024-04-25T09:07:27.013311+0000","last_became_active":"2024-04-25T09:07:26.018933+0000","last_became_peered":"2024-04-25T09:07:26.018933+0000","last_unstale":"2024-04-25T09:07:27.013311+0000","last_undegraded":"2024-04-25T09:07:27.013311+0000","last_fullsized":"2024-04-25T09:07:27.013311+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.794953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627210+0000","last_change":"2024-04-25T09:07:25.999364+0000","last_active":"2024-04-25T09:07:27.627210+0000","last_peered":"2024-04-25T09:07:27.627210+0000","last_clean":"2024-04-25T09:07:27.627210+0000","last_became_active":"2024-04-25T09:07:25.999056+0000","last_became_peered":"2024-04-25T09:07:25.999056+0000","last_unstale":"2024-04-25T09:07:27.627210+0000","last_undegraded":"2024-04-25T09:07:27.627210+0000","last_fullsized":"2024-04-25T09:07:27.627210+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:54.734490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629870+0000","last_change":"2024-04-25T09:07:26.010154+0000","last_active":"2024-04-25T09:07:27.629870+0000","last_peered":"2024-04-25T09:07:27.629870+0000","last_clean":"2024-04-25T09:07:27.629870+0000","last_became_active":"2024-04-25T09:07:26.009860+0000","last_became_peered":"2024-04-25T09:07:26.009860+0000","last_unstale":"2024-04-25T09:07:27.629870+0000","last_undegraded":"2024-04-25T09:07:27.629870+0000","last_fullsized":"2024-04-25T09:07:27.629870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:52.139257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017599+0000","last_change":"2024-04-25T09:07:26.021448+0000","last_active":"2024-04-25T09:07:27.017599+0000","last_peered":"2024-04-25T09:07:27.017599+0000","last_clean":"2024-04-25T09:07:27.017599+0000","last_became_active":"2024-04-25T09:07:26.021127+0000","last_became_peered":"2024-04-25T09:07:26.021127+0000","last_unstale":"2024-04-25T09:07:27.017599+0000","last_undegraded":"2024-04-25T09:07:27.017599+0000","last_fullsized":"2024-04-25T09:07:27.017599+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:00.152037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012009+0000","last_change":"2024-04-25T09:07:26.015054+0000","last_active":"2024-04-25T09:07:27.012009+0000","last_peered":"2024-04-25T09:07:27.012009+0000","last_clean":"2024-04-25T09:07:27.012009+0000","last_became_active":"2024-04-25T09:07:26.014723+0000","last_became_peered":"2024-04-25T09:07:26.014723+0000","last_unstale":"2024-04-25T09:07:27.012009+0000","last_undegraded":"2024-04-25T09:07:27.012009+0000","last_fullsized":"2024-04-25T09:07:27.012009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:11.998769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629344+0000","last_change":"2024-04-25T09:07:26.002430+0000","last_active":"2024-04-25T09:07:27.629344+0000","last_peered":"2024-04-25T09:07:27.629344+0000","last_clean":"2024-04-25T09:07:27.629344+0000","last_became_active":"2024-04-25T09:07:26.000950+0000","last_became_peered":"2024-04-25T09:07:26.000950+0000","last_unstale":"2024-04-25T09:07:27.629344+0000","last_undegraded":"2024-04-25T09:07:27.629344+0000","last_fullsized":"2024-04-25T09:07:27.629344+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:37.550711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.008472+0000","last_change":"2024-04-25T09:07:26.001972+0000","last_active":"2024-04-25T09:07:27.008472+0000","last_peered":"2024-04-25T09:07:27.008472+0000","last_clean":"2024-04-25T09:07:27.008472+0000","last_became_active":"2024-04-25T09:07:26.001620+0000","last_became_peered":"2024-04-25T09:07:26.001620+0000","last_unstale":"2024-04-25T09:07:27.008472+0000","last_undegraded":"2024-04-25T09:07:27.008472+0000","last_fullsized":"2024-04-25T09:07:27.008472+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.794953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012899+0000","last_change":"2024-04-25T09:07:26.006192+0000","last_active":"2024-04-25T09:07:27.012899+0000","last_peered":"2024-04-25T09:07:27.012899+0000","last_clean":"2024-04-25T09:07:27.012899+0000","last_became_active":"2024-04-25T09:07:26.005914+0000","last_became_peered":"2024-04-25T09:07:26.005914+0000","last_unstale":"2024-04-25T09:07:27.012899+0000","last_undegraded":"2024-04-25T09:07:27.012899+0000","last_fullsized":"2024-04-25T09:07:27.012899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:43.938796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095873+0000","last_change":"2024-04-25T09:07:26.006885+0000","last_active":"2024-04-25T09:07:27.095873+0000","last_peered":"2024-04-25T09:07:27.095873+0000","last_clean":"2024-04-25T09:07:27.095873+0000","last_became_active":"2024-04-25T09:07:26.006574+0000","last_became_peered":"2024-04-25T09:07:26.006574+0000","last_unstale":"2024-04-25T09:07:27.095873+0000","last_undegraded":"2024-04-25T09:07:27.095873+0000","last_fullsized":"2024-04-25T09:07:27.095873+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:11.998769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628608+0000","last_change":"2024-04-25T09:07:26.008467+0000","last_active":"2024-04-25T09:07:27.628608+0000","last_peered":"2024-04-25T09:07:27.628608+0000","last_clean":"2024-04-25T09:07:27.628608+0000","last_became_active":"2024-04-25T09:07:26.008184+0000","last_became_peered":"2024-04-25T09:07:26.008184+0000","last_unstale":"2024-04-25T09:07:27.628608+0000","last_undegraded":"2024-04-25T09:07:27.628608+0000","last_fullsized":"2024-04-25T09:07:27.628608+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:08.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012036+0000","last_change":"2024-04-25T09:07:26.010106+0000","last_active":"2024-04-25T09:07:27.012036+0000","last_peered":"2024-04-25T09:07:27.012036+0000","last_clean":"2024-04-25T09:07:27.012036+0000","last_became_active":"2024-04-25T09:07:26.009780+0000","last_became_peered":"2024-04-25T09:07:26.009780+0000","last_unstale":"2024-04-25T09:07:27.012036+0000","last_undegraded":"2024-04-25T09:07:27.012036+0000","last_fullsized":"2024-04-25T09:07:27.012036+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:59.375662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.011739+0000","last_change":"2024-04-25T09:07:26.017867+0000","last_active":"2024-04-25T09:07:27.011739+0000","last_peered":"2024-04-25T09:07:27.011739+0000","last_clean":"2024-04-25T09:07:27.011739+0000","last_became_active":"2024-04-25T09:07:26.017165+0000","last_became_peered":"2024-04-25T09:07:26.017165+0000","last_unstale":"2024-04-25T09:07:27.011739+0000","last_undegraded":"2024-04-25T09:07:27.011739+0000","last_fullsized":"2024-04-25T09:07:27.011739+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:25.097739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627371+0000","last_change":"2024-04-25T09:07:26.013143+0000","last_active":"2024-04-25T09:07:27.627371+0000","last_peered":"2024-04-25T09:07:27.627371+0000","last_clean":"2024-04-25T09:07:27.627371+0000","last_became_active":"2024-04-25T09:07:26.012393+0000","last_became_peered":"2024-04-25T09:07:26.012393+0000","last_unstale":"2024-04-25T09:07:27.627371+0000","last_undegraded":"2024-04-25T09:07:27.627371+0000","last_fullsized":"2024-04-25T09:07:27.627371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:59.375662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095886+0000","last_change":"2024-04-25T09:07:26.019563+0000","last_active":"2024-04-25T09:07:27.095886+0000","last_peered":"2024-04-25T09:07:27.095886+0000","last_clean":"2024-04-25T09:07:27.095886+0000","last_became_active":"2024-04-25T09:07:26.019237+0000","last_became_peered":"2024-04-25T09:07:26.019237+0000","last_unstale":"2024-04-25T09:07:27.095886+0000","last_undegraded":"2024-04-25T09:07:27.095886+0000","last_fullsized":"2024-04-25T09:07:27.095886+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:59.375662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005303+0000","last_change":"2024-04-25T09:07:26.015812+0000","last_active":"2024-04-25T09:07:27.005303+0000","last_peered":"2024-04-25T09:07:27.005303+0000","last_clean":"2024-04-25T09:07:27.005303+0000","last_became_active":"2024-04-25T09:07:26.014351+0000","last_became_peered":"2024-04-25T09:07:26.014351+0000","last_unstale":"2024-04-25T09:07:27.005303+0000","last_undegraded":"2024-04-25T09:07:27.005303+0000","last_fullsized":"2024-04-25T09:07:27.005303+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:42.685590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627161+0000","last_change":"2024-04-25T09:07:26.005702+0000","last_active":"2024-04-25T09:07:27.627161+0000","last_peered":"2024-04-25T09:07:27.627161+0000","last_clean":"2024-04-25T09:07:27.627161+0000","last_became_active":"2024-04-25T09:07:26.005377+0000","last_became_peered":"2024-04-25T09:07:26.005377+0000","last_unstale":"2024-04-25T09:07:27.627161+0000","last_undegraded":"2024-04-25T09:07:27.627161+0000","last_fullsized":"2024-04-25T09:07:27.627161+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:25.097739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007601+0000","last_change":"2024-04-25T09:07:25.992649+0000","last_active":"2024-04-25T09:07:27.007601+0000","last_peered":"2024-04-25T09:07:27.007601+0000","last_clean":"2024-04-25T09:07:27.007601+0000","last_became_active":"2024-04-25T09:07:25.992349+0000","last_became_peered":"2024-04-25T09:07:25.992349+0000","last_unstale":"2024-04-25T09:07:27.007601+0000","last_undegraded":"2024-04-25T09:07:27.007601+0000","last_fullsized":"2024-04-25T09:07:27.007601+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:43.938796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012682+0000","last_change":"2024-04-25T09:07:26.010761+0000","last_active":"2024-04-25T09:07:27.012682+0000","last_peered":"2024-04-25T09:07:27.012682+0000","last_clean":"2024-04-25T09:07:27.012682+0000","last_became_active":"2024-04-25T09:07:26.010211+0000","last_became_peered":"2024-04-25T09:07:26.010211+0000","last_unstale":"2024-04-25T09:07:27.012682+0000","last_undegraded":"2024-04-25T09:07:27.012682+0000","last_fullsized":"2024-04-25T09:07:27.012682+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:34.041821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627122+0000","last_change":"2024-04-25T09:07:26.004057+0000","last_active":"2024-04-25T09:07:27.627122+0000","last_peered":"2024-04-25T09:07:27.627122+0000","last_clean":"2024-04-25T09:07:27.627122+0000","last_became_active":"2024-04-25T09:07:26.003763+0000","last_became_peered":"2024-04-25T09:07:26.003763+0000","last_unstale":"2024-04-25T09:07:27.627122+0000","last_undegraded":"2024-04-25T09:07:27.627122+0000","last_fullsized":"2024-04-25T09:07:27.627122+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.581450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017863+0000","last_change":"2024-04-25T09:07:26.007824+0000","last_active":"2024-04-25T09:07:27.017863+0000","last_peered":"2024-04-25T09:07:27.017863+0000","last_clean":"2024-04-25T09:07:27.017863+0000","last_became_active":"2024-04-25T09:07:26.007220+0000","last_became_peered":"2024-04-25T09:07:26.007220+0000","last_unstale":"2024-04-25T09:07:27.017863+0000","last_undegraded":"2024-04-25T09:07:27.017863+0000","last_fullsized":"2024-04-25T09:07:27.017863+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.096221+0000","last_change":"2024-04-25T09:07:26.015631+0000","last_active":"2024-04-25T09:07:27.096221+0000","last_peered":"2024-04-25T09:07:27.096221+0000","last_clean":"2024-04-25T09:07:27.096221+0000","last_became_active":"2024-04-25T09:07:26.015348+0000","last_became_peered":"2024-04-25T09:07:26.015348+0000","last_unstale":"2024-04-25T09:07:27.096221+0000","last_undegraded":"2024-04-25T09:07:27.096221+0000","last_fullsized":"2024-04-25T09:07:27.096221+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:34.041821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627447+0000","last_change":"2024-04-25T09:07:26.006204+0000","last_active":"2024-04-25T09:07:27.627447+0000","last_peered":"2024-04-25T09:07:27.627447+0000","last_clean":"2024-04-25T09:07:27.627447+0000","last_became_active":"2024-04-25T09:07:26.004871+0000","last_became_peered":"2024-04-25T09:07:26.004871+0000","last_unstale":"2024-04-25T09:07:27.627447+0000","last_undegraded":"2024-04-25T09:07:27.627447+0000","last_fullsized":"2024-04-25T09:07:27.627447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:16.317310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012850+0000","last_change":"2024-04-25T09:07:26.009652+0000","last_active":"2024-04-25T09:07:27.012850+0000","last_peered":"2024-04-25T09:07:27.012850+0000","last_clean":"2024-04-25T09:07:27.012850+0000","last_became_active":"2024-04-25T09:07:26.007525+0000","last_became_peered":"2024-04-25T09:07:26.007525+0000","last_unstale":"2024-04-25T09:07:27.012850+0000","last_undegraded":"2024-04-25T09:07:27.012850+0000","last_fullsized":"2024-04-25T09:07:27.012850+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:40.171244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017771+0000","last_change":"2024-04-25T09:07:26.022588+0000","last_active":"2024-04-25T09:07:27.017771+0000","last_peered":"2024-04-25T09:07:27.017771+0000","last_clean":"2024-04-25T09:07:27.017771+0000","last_became_active":"2024-04-25T09:07:26.022302+0000","last_became_peered":"2024-04-25T09:07:26.022302+0000","last_unstale":"2024-04-25T09:07:27.017771+0000","last_undegraded":"2024-04-25T09:07:27.017771+0000","last_fullsized":"2024-04-25T09:07:27.017771+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:37.208922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018323+0000","last_change":"2024-04-25T09:07:26.016684+0000","last_active":"2024-04-25T09:07:27.018323+0000","last_peered":"2024-04-25T09:07:27.018323+0000","last_clean":"2024-04-25T09:07:27.018323+0000","last_became_active":"2024-04-25T09:07:26.009349+0000","last_became_peered":"2024-04-25T09:07:26.009349+0000","last_unstale":"2024-04-25T09:07:27.018323+0000","last_undegraded":"2024-04-25T09:07:27.018323+0000","last_fullsized":"2024-04-25T09:07:27.018323+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:28.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626586+0000","last_change":"2024-04-25T09:07:26.000007+0000","last_active":"2024-04-25T09:07:27.626586+0000","last_peered":"2024-04-25T09:07:27.626586+0000","last_clean":"2024-04-25T09:07:27.626586+0000","last_became_active":"2024-04-25T09:07:25.997357+0000","last_became_peered":"2024-04-25T09:07:25.997357+0000","last_unstale":"2024-04-25T09:07:27.626586+0000","last_undegraded":"2024-04-25T09:07:27.626586+0000","last_fullsized":"2024-04-25T09:07:27.626586+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:00.300113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095599+0000","last_change":"2024-04-25T09:07:26.007780+0000","last_active":"2024-04-25T09:07:27.095599+0000","last_peered":"2024-04-25T09:07:27.095599+0000","last_clean":"2024-04-25T09:07:27.095599+0000","last_became_active":"2024-04-25T09:07:26.006868+0000","last_became_peered":"2024-04-25T09:07:26.006868+0000","last_unstale":"2024-04-25T09:07:27.095599+0000","last_undegraded":"2024-04-25T09:07:27.095599+0000","last_fullsized":"2024-04-25T09:07:27.095599+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:00.300113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629903+0000","last_change":"2024-04-25T09:07:26.002598+0000","last_active":"2024-04-25T09:07:27.629903+0000","last_peered":"2024-04-25T09:07:27.629903+0000","last_clean":"2024-04-25T09:07:27.629903+0000","last_became_active":"2024-04-25T09:07:26.002300+0000","last_became_peered":"2024-04-25T09:07:26.002300+0000","last_unstale":"2024-04-25T09:07:27.629903+0000","last_undegraded":"2024-04-25T09:07:27.629903+0000","last_fullsized":"2024-04-25T09:07:27.629903+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:10.617565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012230+0000","last_change":"2024-04-25T09:07:26.007946+0000","last_active":"2024-04-25T09:07:27.012230+0000","last_peered":"2024-04-25T09:07:27.012230+0000","last_clean":"2024-04-25T09:07:27.012230+0000","last_became_active":"2024-04-25T09:07:26.007517+0000","last_became_peered":"2024-04-25T09:07:26.007517+0000","last_unstale":"2024-04-25T09:07:27.012230+0000","last_undegraded":"2024-04-25T09:07:27.012230+0000","last_fullsized":"2024-04-25T09:07:27.012230+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.619139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627665+0000","last_change":"2024-04-25T09:07:25.995876+0000","last_active":"2024-04-25T09:07:27.627665+0000","last_peered":"2024-04-25T09:07:27.627665+0000","last_clean":"2024-04-25T09:07:27.627665+0000","last_became_active":"2024-04-25T09:07:25.995428+0000","last_became_peered":"2024-04-25T09:07:25.995428+0000","last_unstale":"2024-04-25T09:07:27.627665+0000","last_undegraded":"2024-04-25T09:07:27.627665+0000","last_fullsized":"2024-04-25T09:07:27.627665+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:48.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629501+0000","last_change":"2024-04-25T09:07:25.010479+0000","last_active":"2024-04-25T09:07:27.629501+0000","last_peered":"2024-04-25T09:07:27.629501+0000","last_clean":"2024-04-25T09:07:27.629501+0000","last_became_active":"2024-04-25T09:07:25.009971+0000","last_became_peered":"2024-04-25T09:07:25.009971+0000","last_unstale":"2024-04-25T09:07:27.629501+0000","last_undegraded":"2024-04-25T09:07:27.629501+0000","last_fullsized":"2024-04-25T09:07:27.629501+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.789498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627733+0000","last_change":"2024-04-25T09:07:26.000285+0000","last_active":"2024-04-25T09:07:27.627733+0000","last_peered":"2024-04-25T09:07:27.627733+0000","last_clean":"2024-04-25T09:07:27.627733+0000","last_became_active":"2024-04-25T09:07:25.999478+0000","last_became_peered":"2024-04-25T09:07:25.999478+0000","last_unstale":"2024-04-25T09:07:27.627733+0000","last_undegraded":"2024-04-25T09:07:27.627733+0000","last_fullsized":"2024-04-25T09:07:27.627733+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.619139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018093+0000","last_change":"2024-04-25T09:07:25.012693+0000","last_active":"2024-04-25T09:07:27.018093+0000","last_peered":"2024-04-25T09:07:27.018093+0000","last_clean":"2024-04-25T09:07:27.018093+0000","last_became_active":"2024-04-25T09:07:25.012366+0000","last_became_peered":"2024-04-25T09:07:25.012366+0000","last_unstale":"2024-04-25T09:07:27.018093+0000","last_undegraded":"2024-04-25T09:07:27.018093+0000","last_fullsized":"2024-04-25T09:07:27.018093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:34.864604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.096087+0000","last_change":"2024-04-25T09:07:26.008674+0000","last_active":"2024-04-25T09:07:27.096087+0000","last_peered":"2024-04-25T09:07:27.096087+0000","last_clean":"2024-04-25T09:07:27.096087+0000","last_became_active":"2024-04-25T09:07:26.008177+0000","last_became_peered":"2024-04-25T09:07:26.008177+0000","last_unstale":"2024-04-25T09:07:27.096087+0000","last_undegraded":"2024-04-25T09:07:27.096087+0000","last_fullsized":"2024-04-25T09:07:27.096087+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:23.228521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629122+0000","last_change":"2024-04-25T09:07:25.011946+0000","last_active":"2024-04-25T09:07:27.629122+0000","last_peered":"2024-04-25T09:07:27.629122+0000","last_clean":"2024-04-25T09:07:27.629122+0000","last_became_active":"2024-04-25T09:07:25.011630+0000","last_became_peered":"2024-04-25T09:07:25.011630+0000","last_unstale":"2024-04-25T09:07:27.629122+0000","last_undegraded":"2024-04-25T09:07:27.629122+0000","last_fullsized":"2024-04-25T09:07:27.629122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:33.841946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.013009+0000","last_change":"2024-04-25T09:07:26.022576+0000","last_active":"2024-04-25T09:07:27.013009+0000","last_peered":"2024-04-25T09:07:27.013009+0000","last_clean":"2024-04-25T09:07:27.013009+0000","last_became_active":"2024-04-25T09:07:26.022271+0000","last_became_peered":"2024-04-25T09:07:26.022271+0000","last_unstale":"2024-04-25T09:07:27.013009+0000","last_undegraded":"2024-04-25T09:07:27.013009+0000","last_fullsized":"2024-04-25T09:07:27.013009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:10.384284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018370+0000","last_change":"2024-04-25T09:07:25.010814+0000","last_active":"2024-04-25T09:07:27.018370+0000","last_peered":"2024-04-25T09:07:27.018370+0000","last_clean":"2024-04-25T09:07:27.018370+0000","last_became_active":"2024-04-25T09:07:25.008979+0000","last_became_peered":"2024-04-25T09:07:25.008979+0000","last_unstale":"2024-04-25T09:07:27.018370+0000","last_undegraded":"2024-04-25T09:07:27.018370+0000","last_fullsized":"2024-04-25T09:07:27.018370+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:50.778845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018088+0000","last_change":"2024-04-25T09:07:26.020848+0000","last_active":"2024-04-25T09:07:27.018088+0000","last_peered":"2024-04-25T09:07:27.018088+0000","last_clean":"2024-04-25T09:07:27.018088+0000","last_became_active":"2024-04-25T09:07:26.020534+0000","last_became_peered":"2024-04-25T09:07:26.020534+0000","last_unstale":"2024-04-25T09:07:27.018088+0000","last_undegraded":"2024-04-25T09:07:27.018088+0000","last_fullsized":"2024-04-25T09:07:27.018088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:06.982333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629181+0000","last_change":"2024-04-25T09:07:25.010295+0000","last_active":"2024-04-25T09:07:27.629181+0000","last_peered":"2024-04-25T09:07:27.629181+0000","last_clean":"2024-04-25T09:07:27.629181+0000","last_became_active":"2024-04-25T09:07:25.009984+0000","last_became_peered":"2024-04-25T09:07:25.009984+0000","last_unstale":"2024-04-25T09:07:27.629181+0000","last_undegraded":"2024-04-25T09:07:27.629181+0000","last_fullsized":"2024-04-25T09:07:27.629181+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:13.587386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007143+0000","last_change":"2024-04-25T09:07:26.008439+0000","last_active":"2024-04-25T09:07:27.007143+0000","last_peered":"2024-04-25T09:07:27.007143+0000","last_clean":"2024-04-25T09:07:27.007143+0000","last_became_active":"2024-04-25T09:07:26.007383+0000","last_became_peered":"2024-04-25T09:07:26.007383+0000","last_unstale":"2024-04-25T09:07:27.007143+0000","last_undegraded":"2024-04-25T09:07:27.007143+0000","last_fullsized":"2024-04-25T09:07:27.007143+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:40.715423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017754+0000","last_change":"2024-04-25T09:07:25.005618+0000","last_active":"2024-04-25T09:07:27.017754+0000","last_peered":"2024-04-25T09:07:27.017754+0000","last_clean":"2024-04-25T09:07:27.017754+0000","last_became_active":"2024-04-25T09:07:25.005276+0000","last_became_peered":"2024-04-25T09:07:25.005276+0000","last_unstale":"2024-04-25T09:07:27.017754+0000","last_undegraded":"2024-04-25T09:07:27.017754+0000","last_fullsized":"2024-04-25T09:07:27.017754+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.993720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.006839+0000","last_change":"2024-04-25T09:07:26.009183+0000","last_active":"2024-04-25T09:07:27.006839+0000","last_peered":"2024-04-25T09:07:27.006839+0000","last_clean":"2024-04-25T09:07:27.006839+0000","last_became_active":"2024-04-25T09:07:26.008849+0000","last_became_peered":"2024-04-25T09:07:26.008849+0000","last_unstale":"2024-04-25T09:07:27.006839+0000","last_undegraded":"2024-04-25T09:07:27.006839+0000","last_fullsized":"2024-04-25T09:07:27.006839+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:34.041821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626727+0000","last_change":"2024-04-25T09:07:25.014784+0000","last_active":"2024-04-25T09:07:27.626727+0000","last_peered":"2024-04-25T09:07:27.626727+0000","last_clean":"2024-04-25T09:07:27.626727+0000","last_became_active":"2024-04-25T09:07:25.012907+0000","last_became_peered":"2024-04-25T09:07:25.012907+0000","last_unstale":"2024-04-25T09:07:27.626727+0000","last_undegraded":"2024-04-25T09:07:27.626727+0000","last_fullsized":"2024-04-25T09:07:27.626727+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017720+0000","last_change":"2024-04-25T09:07:26.008053+0000","last_active":"2024-04-25T09:07:27.017720+0000","last_peered":"2024-04-25T09:07:27.017720+0000","last_clean":"2024-04-25T09:07:27.017720+0000","last_became_active":"2024-04-25T09:07:26.007557+0000","last_became_peered":"2024-04-25T09:07:26.007557+0000","last_unstale":"2024-04-25T09:07:27.017720+0000","last_undegraded":"2024-04-25T09:07:27.017720+0000","last_fullsized":"2024-04-25T09:07:27.017720+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:53.818889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007098+0000","last_change":"2024-04-25T09:07:25.009763+0000","last_active":"2024-04-25T09:07:27.007098+0000","last_peered":"2024-04-25T09:07:27.007098+0000","last_clean":"2024-04-25T09:07:27.007098+0000","last_became_active":"2024-04-25T09:07:25.009474+0000","last_became_peered":"2024-04-25T09:07:25.009474+0000","last_unstale":"2024-04-25T09:07:27.007098+0000","last_undegraded":"2024-04-25T09:07:27.007098+0000","last_fullsized":"2024-04-25T09:07:27.007098+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:25.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005544+0000","last_change":"2024-04-25T09:07:26.006903+0000","last_active":"2024-04-25T09:07:27.005544+0000","last_peered":"2024-04-25T09:07:27.005544+0000","last_clean":"2024-04-25T09:07:27.005544+0000","last_became_active":"2024-04-25T09:07:26.003814+0000","last_became_peered":"2024-04-25T09:07:26.003814+0000","last_unstale":"2024-04-25T09:07:27.005544+0000","last_undegraded":"2024-04-25T09:07:27.005544+0000","last_fullsized":"2024-04-25T09:07:27.005544+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:28.513205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005540+0000","last_change":"2024-04-25T09:07:25.007040+0000","last_active":"2024-04-25T09:07:27.005540+0000","last_peered":"2024-04-25T09:07:27.005540+0000","last_clean":"2024-04-25T09:07:27.005540+0000","last_became_active":"2024-04-25T09:07:25.006742+0000","last_became_peered":"2024-04-25T09:07:25.006742+0000","last_unstale":"2024-04-25T09:07:27.005540+0000","last_undegraded":"2024-04-25T09:07:27.005540+0000","last_fullsized":"2024-04-25T09:07:27.005540+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:29.835765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005689+0000","last_change":"2024-04-25T09:07:26.002649+0000","last_active":"2024-04-25T09:07:27.005689+0000","last_peered":"2024-04-25T09:07:27.005689+0000","last_clean":"2024-04-25T09:07:27.005689+0000","last_became_active":"2024-04-25T09:07:26.002339+0000","last_became_peered":"2024-04-25T09:07:26.002339+0000","last_unstale":"2024-04-25T09:07:27.005689+0000","last_undegraded":"2024-04-25T09:07:27.005689+0000","last_fullsized":"2024-04-25T09:07:27.005689+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:52.805606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.008483+0000","last_change":"2024-04-25T09:07:25.016674+0000","last_active":"2024-04-25T09:07:27.008483+0000","last_peered":"2024-04-25T09:07:27.008483+0000","last_clean":"2024-04-25T09:07:27.008483+0000","last_became_active":"2024-04-25T09:07:25.016317+0000","last_became_peered":"2024-04-25T09:07:25.016317+0000","last_unstale":"2024-04-25T09:07:27.008483+0000","last_undegraded":"2024-04-25T09:07:27.008483+0000","last_fullsized":"2024-04-25T09:07:27.008483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:55.318929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012865+0000","last_change":"2024-04-25T09:07:26.012586+0000","last_active":"2024-04-25T09:07:27.012865+0000","last_peered":"2024-04-25T09:07:27.012865+0000","last_clean":"2024-04-25T09:07:27.012865+0000","last_became_active":"2024-04-25T09:07:26.012196+0000","last_became_peered":"2024-04-25T09:07:26.012196+0000","last_unstale":"2024-04-25T09:07:27.012865+0000","last_undegraded":"2024-04-25T09:07:27.012865+0000","last_fullsized":"2024-04-25T09:07:27.012865+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:51.073028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095743+0000","last_change":"2024-04-25T09:07:25.008303+0000","last_active":"2024-04-25T09:07:27.095743+0000","last_peered":"2024-04-25T09:07:27.095743+0000","last_clean":"2024-04-25T09:07:27.095743+0000","last_became_active":"2024-04-25T09:07:25.006381+0000","last_became_peered":"2024-04-25T09:07:25.006381+0000","last_unstale":"2024-04-25T09:07:27.095743+0000","last_undegraded":"2024-04-25T09:07:27.095743+0000","last_fullsized":"2024-04-25T09:07:27.095743+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:55.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017944+0000","last_change":"2024-04-25T09:07:26.020831+0000","last_active":"2024-04-25T09:07:27.017944+0000","last_peered":"2024-04-25T09:07:27.017944+0000","last_clean":"2024-04-25T09:07:27.017944+0000","last_became_active":"2024-04-25T09:07:26.020522+0000","last_became_peered":"2024-04-25T09:07:26.020522+0000","last_unstale":"2024-04-25T09:07:27.017944+0000","last_undegraded":"2024-04-25T09:07:27.017944+0000","last_fullsized":"2024-04-25T09:07:27.017944+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:53.955549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007893+0000","last_change":"2024-04-25T09:07:25.003604+0000","last_active":"2024-04-25T09:07:27.007893+0000","last_peered":"2024-04-25T09:07:27.007893+0000","last_clean":"2024-04-25T09:07:27.007893+0000","last_became_active":"2024-04-25T09:07:25.003266+0000","last_became_peered":"2024-04-25T09:07:25.003266+0000","last_unstale":"2024-04-25T09:07:27.007893+0000","last_undegraded":"2024-04-25T09:07:27.007893+0000","last_fullsized":"2024-04-25T09:07:27.007893+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:39.609538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.004758+0000","last_change":"2024-04-25T09:07:26.014052+0000","last_active":"2024-04-25T09:07:27.004758+0000","last_peered":"2024-04-25T09:07:27.004758+0000","last_clean":"2024-04-25T09:07:27.004758+0000","last_became_active":"2024-04-25T09:07:26.013779+0000","last_became_peered":"2024-04-25T09:07:26.013779+0000","last_unstale":"2024-04-25T09:07:27.004758+0000","last_undegraded":"2024-04-25T09:07:27.004758+0000","last_fullsized":"2024-04-25T09:07:27.004758+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:46.543920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626797+0000","last_change":"2024-04-25T09:07:25.006123+0000","last_active":"2024-04-25T09:07:27.626797+0000","last_peered":"2024-04-25T09:07:27.626797+0000","last_clean":"2024-04-25T09:07:27.626797+0000","last_became_active":"2024-04-25T09:07:25.004723+0000","last_became_peered":"2024-04-25T09:07:25.004723+0000","last_unstale":"2024-04-25T09:07:27.626797+0000","last_undegraded":"2024-04-25T09:07:27.626797+0000","last_fullsized":"2024-04-25T09:07:27.626797+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.526901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626806+0000","last_change":"2024-04-25T09:07:25.998673+0000","last_active":"2024-04-25T09:07:27.626806+0000","last_peered":"2024-04-25T09:07:27.626806+0000","last_clean":"2024-04-25T09:07:27.626806+0000","last_became_active":"2024-04-25T09:07:25.998396+0000","last_became_peered":"2024-04-25T09:07:25.998396+0000","last_unstale":"2024-04-25T09:07:27.626806+0000","last_undegraded":"2024-04-25T09:07:27.626806+0000","last_fullsized":"2024-04-25T09:07:27.626806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:46.543920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626256+0000","last_change":"2024-04-25T09:07:25.004430+0000","last_active":"2024-04-25T09:07:27.626256+0000","last_peered":"2024-04-25T09:07:27.626256+0000","last_clean":"2024-04-25T09:07:27.626256+0000","last_became_active":"2024-04-25T09:07:25.000012+0000","last_became_peered":"2024-04-25T09:07:25.000012+0000","last_unstale":"2024-04-25T09:07:27.626256+0000","last_undegraded":"2024-04-25T09:07:27.626256+0000","last_fullsized":"2024-04-25T09:07:27.626256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:55.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628814+0000","last_change":"2024-04-25T09:07:25.992553+0000","last_active":"2024-04-25T09:07:27.628814+0000","last_peered":"2024-04-25T09:07:27.628814+0000","last_clean":"2024-04-25T09:07:27.628814+0000","last_became_active":"2024-04-25T09:07:25.992105+0000","last_became_peered":"2024-04-25T09:07:25.992105+0000","last_unstale":"2024-04-25T09:07:27.628814+0000","last_undegraded":"2024-04-25T09:07:27.628814+0000","last_fullsized":"2024-04-25T09:07:27.628814+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:52.805606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628811+0000","last_change":"2024-04-25T09:07:25.013988+0000","last_active":"2024-04-25T09:07:27.628811+0000","last_peered":"2024-04-25T09:07:27.628811+0000","last_clean":"2024-04-25T09:07:27.628811+0000","last_became_active":"2024-04-25T09:07:25.013681+0000","last_became_peered":"2024-04-25T09:07:25.013681+0000","last_unstale":"2024-04-25T09:07:27.628811+0000","last_undegraded":"2024-04-25T09:07:27.628811+0000","last_fullsized":"2024-04-25T09:07:27.628811+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:41.669482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095796+0000","last_change":"2024-04-25T09:07:26.001821+0000","last_active":"2024-04-25T09:07:27.095796+0000","last_peered":"2024-04-25T09:07:27.095796+0000","last_clean":"2024-04-25T09:07:27.095796+0000","last_became_active":"2024-04-25T09:07:26.001396+0000","last_became_peered":"2024-04-25T09:07:26.001396+0000","last_unstale":"2024-04-25T09:07:27.095796+0000","last_undegraded":"2024-04-25T09:07:27.095796+0000","last_fullsized":"2024-04-25T09:07:27.095796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:54.734490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":22,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007349+0000","last_change":"2024-04-25T09:07:25.006626+0000","last_active":"2024-04-25T09:07:27.007349+0000","last_peered":"2024-04-25T09:07:27.007349+0000","last_clean":"2024-04-25T09:07:27.007349+0000","last_became_active":"2024-04-25T09:07:25.006323+0000","last_became_peered":"2024-04-25T09:07:25.006323+0000","last_unstale":"2024-04-25T09:07:27.007349+0000","last_undegraded":"2024-04-25T09:07:27.007349+0000","last_fullsized":"2024-04-25T09:07:27.007349+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:02.442135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628115+0000","last_change":"2024-04-25T09:07:26.005948+0000","last_active":"2024-04-25T09:07:27.628115+0000","last_peered":"2024-04-25T09:07:27.628115+0000","last_clean":"2024-04-25T09:07:27.628115+0000","last_became_active":"2024-04-25T09:07:26.004879+0000","last_became_peered":"2024-04-25T09:07:26.004879+0000","last_unstale":"2024-04-25T09:07:27.628115+0000","last_undegraded":"2024-04-25T09:07:27.628115+0000","last_fullsized":"2024-04-25T09:07:27.628115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:42.685590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012918+0000","last_change":"2024-04-25T09:07:25.016157+0000","last_active":"2024-04-25T09:07:27.012918+0000","last_peered":"2024-04-25T09:07:27.012918+0000","last_clean":"2024-04-25T09:07:27.012918+0000","last_became_active":"2024-04-25T09:07:25.015731+0000","last_became_peered":"2024-04-25T09:07:25.015731+0000","last_unstale":"2024-04-25T09:07:27.012918+0000","last_undegraded":"2024-04-25T09:07:27.012918+0000","last_fullsized":"2024-04-25T09:07:27.012918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.030750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628224+0000","last_change":"2024-04-25T09:07:26.000165+0000","last_active":"2024-04-25T09:07:27.628224+0000","last_peered":"2024-04-25T09:07:27.628224+0000","last_clean":"2024-04-25T09:07:27.628224+0000","last_became_active":"2024-04-25T09:07:25.999872+0000","last_became_peered":"2024-04-25T09:07:25.999872+0000","last_unstale":"2024-04-25T09:07:27.628224+0000","last_undegraded":"2024-04-25T09:07:27.628224+0000","last_fullsized":"2024-04-25T09:07:27.628224+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:28.513205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005532+0000","last_change":"2024-04-25T09:07:25.010677+0000","last_active":"2024-04-25T09:07:27.005532+0000","last_peered":"2024-04-25T09:07:27.005532+0000","last_clean":"2024-04-25T09:07:27.005532+0000","last_became_active":"2024-04-25T09:07:25.010337+0000","last_became_peered":"2024-04-25T09:07:25.010337+0000","last_unstale":"2024-04-25T09:07:27.005532+0000","last_undegraded":"2024-04-25T09:07:27.005532+0000","last_fullsized":"2024-04-25T09:07:27.005532+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:26.558822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095960+0000","last_change":"2024-04-25T09:07:26.015023+0000","last_active":"2024-04-25T09:07:27.095960+0000","last_peered":"2024-04-25T09:07:27.095960+0000","last_clean":"2024-04-25T09:07:27.095960+0000","last_became_active":"2024-04-25T09:07:26.014632+0000","last_became_peered":"2024-04-25T09:07:26.014632+0000","last_unstale":"2024-04-25T09:07:27.095960+0000","last_undegraded":"2024-04-25T09:07:27.095960+0000","last_fullsized":"2024-04-25T09:07:27.095960+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:16.317310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005217+0000","last_change":"2024-04-25T09:07:25.012576+0000","last_active":"2024-04-25T09:07:27.005217+0000","last_peered":"2024-04-25T09:07:27.005217+0000","last_clean":"2024-04-25T09:07:27.005217+0000","last_became_active":"2024-04-25T09:07:25.010949+0000","last_became_peered":"2024-04-25T09:07:25.010949+0000","last_unstale":"2024-04-25T09:07:27.005217+0000","last_undegraded":"2024-04-25T09:07:27.005217+0000","last_fullsized":"2024-04-25T09:07:27.005217+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005121+0000","last_change":"2024-04-25T09:07:26.006235+0000","last_active":"2024-04-25T09:07:27.005121+0000","last_peered":"2024-04-25T09:07:27.005121+0000","last_clean":"2024-04-25T09:07:27.005121+0000","last_became_active":"2024-04-25T09:07:26.005829+0000","last_became_peered":"2024-04-25T09:07:26.005829+0000","last_unstale":"2024-04-25T09:07:27.005121+0000","last_undegraded":"2024-04-25T09:07:27.005121+0000","last_fullsized":"2024-04-25T09:07:27.005121+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:48.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629476+0000","last_change":"2024-04-25T09:07:25.013972+0000","last_active":"2024-04-25T09:07:27.629476+0000","last_peered":"2024-04-25T09:07:27.629476+0000","last_clean":"2024-04-25T09:07:27.629476+0000","last_became_active":"2024-04-25T09:07:25.013633+0000","last_became_peered":"2024-04-25T09:07:25.013633+0000","last_unstale":"2024-04-25T09:07:27.629476+0000","last_undegraded":"2024-04-25T09:07:27.629476+0000","last_fullsized":"2024-04-25T09:07:27.629476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:54.158011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007150+0000","last_change":"2024-04-25T09:07:25.996907+0000","last_active":"2024-04-25T09:07:27.007150+0000","last_peered":"2024-04-25T09:07:27.007150+0000","last_clean":"2024-04-25T09:07:27.007150+0000","last_became_active":"2024-04-25T09:07:25.996559+0000","last_became_peered":"2024-04-25T09:07:25.996559+0000","last_unstale":"2024-04-25T09:07:27.007150+0000","last_undegraded":"2024-04-25T09:07:27.007150+0000","last_fullsized":"2024-04-25T09:07:27.007150+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:40.171244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007079+0000","last_change":"2024-04-25T09:07:25.016649+0000","last_active":"2024-04-25T09:07:27.007079+0000","last_peered":"2024-04-25T09:07:27.007079+0000","last_clean":"2024-04-25T09:07:27.007079+0000","last_became_active":"2024-04-25T09:07:25.016387+0000","last_became_peered":"2024-04-25T09:07:25.016387+0000","last_unstale":"2024-04-25T09:07:27.007079+0000","last_undegraded":"2024-04-25T09:07:27.007079+0000","last_fullsized":"2024-04-25T09:07:27.007079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:45.873510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628913+0000","last_change":"2024-04-25T09:07:26.008218+0000","last_active":"2024-04-25T09:07:27.628913+0000","last_peered":"2024-04-25T09:07:27.628913+0000","last_clean":"2024-04-25T09:07:27.628913+0000","last_became_active":"2024-04-25T09:07:26.007932+0000","last_became_peered":"2024-04-25T09:07:26.007932+0000","last_unstale":"2024-04-25T09:07:27.628913+0000","last_undegraded":"2024-04-25T09:07:27.628913+0000","last_fullsized":"2024-04-25T09:07:27.628913+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:56.817088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626189+0000","last_change":"2024-04-25T09:07:25.009598+0000","last_active":"2024-04-25T09:07:27.626189+0000","last_peered":"2024-04-25T09:07:27.626189+0000","last_clean":"2024-04-25T09:07:27.626189+0000","last_became_active":"2024-04-25T09:07:25.009294+0000","last_became_peered":"2024-04-25T09:07:25.009294+0000","last_unstale":"2024-04-25T09:07:27.626189+0000","last_undegraded":"2024-04-25T09:07:27.626189+0000","last_fullsized":"2024-04-25T09:07:27.626189+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:18.169959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012554+0000","last_change":"2024-04-25T09:07:26.018073+0000","last_active":"2024-04-25T09:07:27.012554+0000","last_peered":"2024-04-25T09:07:27.012554+0000","last_clean":"2024-04-25T09:07:27.012554+0000","last_became_active":"2024-04-25T09:07:26.017151+0000","last_became_peered":"2024-04-25T09:07:26.017151+0000","last_unstale":"2024-04-25T09:07:27.012554+0000","last_undegraded":"2024-04-25T09:07:27.012554+0000","last_fullsized":"2024-04-25T09:07:27.012554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:23.228521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.006638+0000","last_change":"2024-04-25T09:07:25.005217+0000","last_active":"2024-04-25T09:07:27.006638+0000","last_peered":"2024-04-25T09:07:27.006638+0000","last_clean":"2024-04-25T09:07:27.006638+0000","last_became_active":"2024-04-25T09:07:25.003714+0000","last_became_peered":"2024-04-25T09:07:25.003714+0000","last_unstale":"2024-04-25T09:07:27.006638+0000","last_undegraded":"2024-04-25T09:07:27.006638+0000","last_fullsized":"2024-04-25T09:07:27.006638+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.284005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.011958+0000","last_change":"2024-04-25T09:07:26.012395+0000","last_active":"2024-04-25T09:07:27.011958+0000","last_peered":"2024-04-25T09:07:27.011958+0000","last_clean":"2024-04-25T09:07:27.011958+0000","last_became_active":"2024-04-25T09:07:26.012075+0000","last_became_peered":"2024-04-25T09:07:26.012075+0000","last_unstale":"2024-04-25T09:07:27.011958+0000","last_undegraded":"2024-04-25T09:07:27.011958+0000","last_fullsized":"2024-04-25T09:07:27.011958+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:54.734490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017579+0000","last_change":"2024-04-25T09:07:25.003186+0000","last_active":"2024-04-25T09:07:27.017579+0000","last_peered":"2024-04-25T09:07:27.017579+0000","last_clean":"2024-04-25T09:07:27.017579+0000","last_became_active":"2024-04-25T09:07:25.002435+0000","last_became_peered":"2024-04-25T09:07:25.002435+0000","last_unstale":"2024-04-25T09:07:27.017579+0000","last_undegraded":"2024-04-25T09:07:27.017579+0000","last_fullsized":"2024-04-25T09:07:27.017579+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:39.699314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018286+0000","last_change":"2024-04-25T09:07:26.005884+0000","last_active":"2024-04-25T09:07:27.018286+0000","last_peered":"2024-04-25T09:07:27.018286+0000","last_clean":"2024-04-25T09:07:27.018286+0000","last_became_active":"2024-04-25T09:07:26.005516+0000","last_became_peered":"2024-04-25T09:07:26.005516+0000","last_unstale":"2024-04-25T09:07:27.018286+0000","last_undegraded":"2024-04-25T09:07:27.018286+0000","last_fullsized":"2024-04-25T09:07:27.018286+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:58.338563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.013242+0000","last_change":"2024-04-25T09:07:25.015721+0000","last_active":"2024-04-25T09:07:27.013242+0000","last_peered":"2024-04-25T09:07:27.013242+0000","last_clean":"2024-04-25T09:07:27.013242+0000","last_became_active":"2024-04-25T09:07:25.015220+0000","last_became_peered":"2024-04-25T09:07:25.015220+0000","last_unstale":"2024-04-25T09:07:27.013242+0000","last_undegraded":"2024-04-25T09:07:27.013242+0000","last_fullsized":"2024-04-25T09:07:27.013242+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:31.406996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007758+0000","last_change":"2024-04-25T09:07:26.019229+0000","last_active":"2024-04-25T09:07:27.007758+0000","last_peered":"2024-04-25T09:07:27.007758+0000","last_clean":"2024-04-25T09:07:27.007758+0000","last_became_active":"2024-04-25T09:07:26.018552+0000","last_became_peered":"2024-04-25T09:07:26.018552+0000","last_unstale":"2024-04-25T09:07:27.007758+0000","last_undegraded":"2024-04-25T09:07:27.007758+0000","last_fullsized":"2024-04-25T09:07:27.007758+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:10.384284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627176+0000","last_change":"2024-04-25T09:07:25.010776+0000","last_active":"2024-04-25T09:07:27.627176+0000","last_peered":"2024-04-25T09:07:27.627176+0000","last_clean":"2024-04-25T09:07:27.627176+0000","last_became_active":"2024-04-25T09:07:25.010280+0000","last_became_peered":"2024-04-25T09:07:25.010280+0000","last_unstale":"2024-04-25T09:07:27.627176+0000","last_undegraded":"2024-04-25T09:07:27.627176+0000","last_fullsized":"2024-04-25T09:07:27.627176+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:36.534602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018146+0000","last_change":"2024-04-25T09:07:26.020850+0000","last_active":"2024-04-25T09:07:27.018146+0000","last_peered":"2024-04-25T09:07:27.018146+0000","last_clean":"2024-04-25T09:07:27.018146+0000","last_became_active":"2024-04-25T09:07:26.020043+0000","last_became_peered":"2024-04-25T09:07:26.020043+0000","last_unstale":"2024-04-25T09:07:27.018146+0000","last_undegraded":"2024-04-25T09:07:27.018146+0000","last_fullsized":"2024-04-25T09:07:27.018146+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:22.528760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627651+0000","last_change":"2024-04-25T09:07:25.007229+0000","last_active":"2024-04-25T09:07:27.627651+0000","last_peered":"2024-04-25T09:07:27.627651+0000","last_clean":"2024-04-25T09:07:27.627651+0000","last_became_active":"2024-04-25T09:07:25.006847+0000","last_became_peered":"2024-04-25T09:07:25.006847+0000","last_unstale":"2024-04-25T09:07:27.627651+0000","last_undegraded":"2024-04-25T09:07:27.627651+0000","last_fullsized":"2024-04-25T09:07:27.627651+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:07.376346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007258+0000","last_change":"2024-04-25T09:07:25.998518+0000","last_active":"2024-04-25T09:07:27.007258+0000","last_peered":"2024-04-25T09:07:27.007258+0000","last_clean":"2024-04-25T09:07:27.007258+0000","last_became_active":"2024-04-25T09:07:25.998260+0000","last_became_peered":"2024-04-25T09:07:25.998260+0000","last_unstale":"2024-04-25T09:07:27.007258+0000","last_undegraded":"2024-04-25T09:07:27.007258+0000","last_fullsized":"2024-04-25T09:07:27.007258+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:51.073028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626859+0000","last_change":"2024-04-25T09:07:25.011689+0000","last_active":"2024-04-25T09:07:27.626859+0000","last_peered":"2024-04-25T09:07:27.626859+0000","last_clean":"2024-04-25T09:07:27.626859+0000","last_became_active":"2024-04-25T09:07:25.011388+0000","last_became_peered":"2024-04-25T09:07:25.011388+0000","last_unstale":"2024-04-25T09:07:27.626859+0000","last_undegraded":"2024-04-25T09:07:27.626859+0000","last_fullsized":"2024-04-25T09:07:27.626859+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:51.123149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012108+0000","last_change":"2024-04-25T09:07:26.009528+0000","last_active":"2024-04-25T09:07:27.012108+0000","last_peered":"2024-04-25T09:07:27.012108+0000","last_clean":"2024-04-25T09:07:27.012108+0000","last_became_active":"2024-04-25T09:07:26.009266+0000","last_became_peered":"2024-04-25T09:07:26.009266+0000","last_unstale":"2024-04-25T09:07:27.012108+0000","last_undegraded":"2024-04-25T09:07:27.012108+0000","last_fullsized":"2024-04-25T09:07:27.012108+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:16.317310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017918+0000","last_change":"2024-04-25T09:07:25.004275+0000","last_active":"2024-04-25T09:07:27.017918+0000","last_peered":"2024-04-25T09:07:27.017918+0000","last_clean":"2024-04-25T09:07:27.017918+0000","last_became_active":"2024-04-25T09:07:25.004002+0000","last_became_peered":"2024-04-25T09:07:25.004002+0000","last_unstale":"2024-04-25T09:07:27.017918+0000","last_undegraded":"2024-04-25T09:07:27.017918+0000","last_fullsized":"2024-04-25T09:07:27.017918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:10.218708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.004278+0000","last_change":"2024-04-25T09:07:26.015332+0000","last_active":"2024-04-25T09:07:27.004278+0000","last_peered":"2024-04-25T09:07:27.004278+0000","last_clean":"2024-04-25T09:07:27.004278+0000","last_became_active":"2024-04-25T09:07:26.014764+0000","last_became_peered":"2024-04-25T09:07:26.014764+0000","last_unstale":"2024-04-25T09:07:27.004278+0000","last_undegraded":"2024-04-25T09:07:27.004278+0000","last_fullsized":"2024-04-25T09:07:27.004278+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:59.238982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629076+0000","last_change":"2024-04-25T09:07:25.007837+0000","last_active":"2024-04-25T09:07:27.629076+0000","last_peered":"2024-04-25T09:07:27.629076+0000","last_clean":"2024-04-25T09:07:27.629076+0000","last_became_active":"2024-04-25T09:07:25.007537+0000","last_became_peered":"2024-04-25T09:07:25.007537+0000","last_unstale":"2024-04-25T09:07:27.629076+0000","last_undegraded":"2024-04-25T09:07:27.629076+0000","last_fullsized":"2024-04-25T09:07:27.629076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:27.497096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.096029+0000","last_change":"2024-04-25T09:07:26.005327+0000","last_active":"2024-04-25T09:07:27.096029+0000","last_peered":"2024-04-25T09:07:27.096029+0000","last_clean":"2024-04-25T09:07:27.096029+0000","last_became_active":"2024-04-25T09:07:26.004944+0000","last_became_peered":"2024-04-25T09:07:26.004944+0000","last_unstale":"2024-04-25T09:07:27.096029+0000","last_undegraded":"2024-04-25T09:07:27.096029+0000","last_fullsized":"2024-04-25T09:07:27.096029+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.619139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627862+0000","last_change":"2024-04-25T09:07:25.002676+0000","last_active":"2024-04-25T09:07:27.627862+0000","last_peered":"2024-04-25T09:07:27.627862+0000","last_clean":"2024-04-25T09:07:27.627862+0000","last_became_active":"2024-04-25T09:07:25.001512+0000","last_became_peered":"2024-04-25T09:07:25.001512+0000","last_unstale":"2024-04-25T09:07:27.627862+0000","last_undegraded":"2024-04-25T09:07:27.627862+0000","last_fullsized":"2024-04-25T09:07:27.627862+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:29.835765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.013236+0000","last_change":"2024-04-25T09:07:26.015182+0000","last_active":"2024-04-25T09:07:27.013236+0000","last_peered":"2024-04-25T09:07:27.013236+0000","last_clean":"2024-04-25T09:07:27.013236+0000","last_became_active":"2024-04-25T09:07:26.014868+0000","last_became_peered":"2024-04-25T09:07:26.014868+0000","last_unstale":"2024-04-25T09:07:27.013236+0000","last_undegraded":"2024-04-25T09:07:27.013236+0000","last_fullsized":"2024-04-25T09:07:27.013236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:40.715423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018440+0000","last_change":"2024-04-25T09:07:25.014785+0000","last_active":"2024-04-25T09:07:27.018440+0000","last_peered":"2024-04-25T09:07:27.018440+0000","last_clean":"2024-04-25T09:07:27.018440+0000","last_became_active":"2024-04-25T09:07:25.013321+0000","last_became_peered":"2024-04-25T09:07:25.013321+0000","last_unstale":"2024-04-25T09:07:27.018440+0000","last_undegraded":"2024-04-25T09:07:27.018440+0000","last_fullsized":"2024-04-25T09:07:27.018440+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:47.161339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.004256+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:07:27.004256+0000","last_peered":"2024-04-25T09:07:27.004256+0000","last_clean":"2024-04-25T09:07:27.004256+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:07:27.004256+0000","last_undegraded":"2024-04-25T09:07:27.004256+0000","last_fullsized":"2024-04-25T09:07:27.004256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":569344,"data_stored":234419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":573440,"data_stored":235001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40644,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331196,"statfs":{"total":96636764160,"available":96595144704,"internally_reserved":0,"allocated":1163264,"data_stored":824775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40080,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331760,"statfs":{"total":96636764160,"available":96595722240,"internally_reserved":0,"allocated":585728,"data_stored":236089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40072,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331768,"statfs":{"total":96636764160,"available":96595730432,"internally_reserved":0,"allocated":577536,"data_stored":235997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40064,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331776,"statfs":{"total":96636764160,"available":96595738624,"internally_reserved":0,"allocated":569344,"data_stored":234475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335364,"statfs":{"total":96636764160,"available":96599412736,"internally_reserved":0,"allocated":1163264,"data_stored":824823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":573440,"data_stored":235057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:07:35.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:07:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.769+0000 7f063ae00640 1 Processor -- start 2024-04-25T09:07:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.769+0000 7f063ae00640 1 -- start start 2024-04-25T09:07:35.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f063ae00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f063415bd10 0x7f06340cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f063ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340cccf0 0x7f06340ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f063ae00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06340aed00 0x7f063415f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f063ae00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063406d660 con 0x7f063415bd10 2024-04-25T09:07:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f063ae00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063406ca70 con 0x7f06340aed00 2024-04-25T09:07:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f063ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063415c0f0 con 0x7f06340cccf0 2024-04-25T09:07:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f0633e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f063415bd10 0x7f06340cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f0633e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f063415bd10 0x7f06340cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58074/0 (socket says 172.21.15.62:58074) 2024-04-25T09:07:35.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.771+0000 7f0633e00640 1 -- 172.21.15.62:0/1015871327 learned_addr learned my addr 172.21.15.62:0/1015871327 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.772+0000 7f0638e00640 1 --2- 172.21.15.62:0/1015871327 >> v2:172.21.15.121:3300/0 conn(0x7f06340aed00 0x7f063415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.772+0000 7f0633e00640 1 -- 172.21.15.62:0/1015871327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340cccf0 msgr2=0x7f06340ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.772+0000 7f0633400640 1 --2- 172.21.15.62:0/1015871327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340cccf0 0x7f06340ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.772+0000 7f0633e00640 1 --2- 172.21.15.62:0/1015871327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340cccf0 0x7f06340ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.772+0000 7f0633e00640 1 -- 172.21.15.62:0/1015871327 >> v2:172.21.15.121:3300/0 conn(0x7f06340aed00 msgr2=0x7f063415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.772+0000 7f0633e00640 1 --2- 172.21.15.62:0/1015871327 >> v2:172.21.15.121:3300/0 conn(0x7f06340aed00 0x7f063415f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.772+0000 7f0633e00640 1 -- 172.21.15.62:0/1015871327 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063406cd30 con 0x7f063415bd10 2024-04-25T09:07:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.772+0000 7f0638e00640 1 --2- 172.21.15.62:0/1015871327 >> v2:172.21.15.121:3300/0 conn(0x7f06340aed00 0x7f063415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:35.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.773+0000 7f0633e00640 1 --2- 172.21.15.62:0/1015871327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f063415bd10 0x7f06340cfca0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f06340b4d90 tx=0x7f062402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1de3f0da485eca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.773+0000 7f0632a00640 1 -- 172.21.15.62:0/1015871327 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624037070 con 0x7f063415bd10 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.773+0000 7f0632a00640 1 -- 172.21.15.62:0/1015871327 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0624032440 con 0x7f063415bd10 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.773+0000 7f0632a00640 1 -- 172.21.15.62:0/1015871327 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624032810 con 0x7f063415bd10 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 -- 172.21.15.62:0/1015871327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f063415bd10 msgr2=0x7f06340cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 --2- 172.21.15.62:0/1015871327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f063415bd10 0x7f06340cfca0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f06340b4d90 tx=0x7f062402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 -- 172.21.15.62:0/1015871327 shutdown_connections 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 --2- 172.21.15.62:0/1015871327 >> v2:172.21.15.121:3300/0 conn(0x7f06340aed00 0x7f063415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 --2- 172.21.15.62:0/1015871327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340cccf0 0x7f06340ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 --2- 172.21.15.62:0/1015871327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f063415bd10 0x7f06340cfca0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 -- 172.21.15.62:0/1015871327 >> 172.21.15.62:0/1015871327 conn(0x7f06340668f0 msgr2=0x7f0634066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 -- 172.21.15.62:0/1015871327 shutdown_connections 2024-04-25T09:07:35.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.774+0000 7f063ae00640 1 -- 172.21.15.62:0/1015871327 wait complete. 2024-04-25T09:07:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.776+0000 7f063ae00640 1 Processor -- start 2024-04-25T09:07:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.776+0000 7f063ae00640 1 -- start start 2024-04-25T09:07:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.777+0000 7f063ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340aed00 0x7f06341f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.777+0000 7f063ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06340cccf0 0x7f06340af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:35.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.777+0000 7f063ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f063415bd10 0x7f06340af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.777+0000 7f063ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340d01e0 con 0x7f06340cccf0 2024-04-25T09:07:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.777+0000 7f063ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063406d2f0 con 0x7f063415bd10 2024-04-25T09:07:35.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.777+0000 7f063ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063406daf0 con 0x7f06340aed00 2024-04-25T09:07:35.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.777+0000 7f0633400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06340cccf0 0x7f06340af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06340cccf0 0x7f06340af170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58090/0 (socket says 172.21.15.62:58090) 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633400640 1 -- 172.21.15.62:0/123037293 learned_addr learned my addr 172.21.15.62:0/123037293 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633e00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340aed00 0x7f06341f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0638e00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f063415bd10 0x7f06340af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633400640 1 -- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340aed00 msgr2=0x7f06341f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633400640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340aed00 0x7f06341f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633400640 1 -- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f063415bd10 msgr2=0x7f06340af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633400640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f063415bd10 0x7f06340af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633400640 1 -- 172.21.15.62:0/123037293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06341f7f90 con 0x7f06340cccf0 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.778+0000 7f0633400640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06340cccf0 0x7f06340af170 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f062000c820 tx=0x7f062000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.779+0000 7f062be00640 1 -- 172.21.15.62:0/123037293 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620015070 con 0x7f06340cccf0 2024-04-25T09:07:35.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.779+0000 7f062be00640 1 -- 172.21.15.62:0/123037293 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0620004240 con 0x7f06340cccf0 2024-04-25T09:07:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.779+0000 7f062be00640 1 -- 172.21.15.62:0/123037293 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062000ce10 con 0x7f06340cccf0 2024-04-25T09:07:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.779+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0634155380 con 0x7f06340cccf0 2024-04-25T09:07:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.779+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634155940 con 0x7f06340cccf0 2024-04-25T09:07:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.780+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0634072c70 con 0x7f06340cccf0 2024-04-25T09:07:35.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.781+0000 7f062be00640 1 -- 172.21.15.62:0/123037293 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0620007830 con 0x7f06340cccf0 2024-04-25T09:07:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.782+0000 7f062be00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f061406b4d0 0x7f061406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:35.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.782+0000 7f062be00640 1 -- 172.21.15.62:0/123037293 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f062008a0c0 con 0x7f06340cccf0 2024-04-25T09:07:35.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.783+0000 7f0633e00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f061406b4d0 0x7f061406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:35.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.784+0000 7f0633e00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f061406b4d0 0x7f061406d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0624034000 tx=0x7f062402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:35.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.787+0000 7f062be00640 1 -- 172.21.15.62:0/123037293 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0620058ea0 con 0x7f06340cccf0 2024-04-25T09:07:35.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:35.985+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06340cb780 con 0x7f061406b4d0 2024-04-25T09:07:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.005+0000 7f062be00640 1 -- 172.21.15.62:0/123037293 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251848 (secure 0 0 0) 0x7f06340cb780 con 0x7f061406b4d0 2024-04-25T09:07:35.996 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:35.997 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f061406b4d0 msgr2=0x7f061406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f061406b4d0 0x7f061406d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0624034000 tx=0x7f062402e000 comp rx=0 tx=0).stop 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06340cccf0 msgr2=0x7f06340af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06340cccf0 0x7f06340af170 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f062000c820 tx=0x7f062000cce0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 shutdown_connections 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f061406b4d0 0x7f061406d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06340cccf0 0x7f06340af170 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f063415bd10 0x7f06340af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 --2- 172.21.15.62:0/123037293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06340aed00 0x7f06341f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 >> 172.21.15.62:0/123037293 conn(0x7f06340668f0 msgr2=0x7f06340ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 shutdown_connections 2024-04-25T09:07:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.011+0000 7f063ae00640 1 -- 172.21.15.62:0/123037293 wait complete. 2024-04-25T09:07:36.018 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-04-25T09:07:35.985549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":300872,"kb_used_data":5640,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754673848,"statfs":{"total":773094113280,"available":772786020352,"internally_reserved":0,"allocated":5775360,"data_stored":3060636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":302173744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.007650"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627354+0000","last_change":"2024-04-25T09:07:26.016714+0000","last_active":"2024-04-25T09:07:27.627354+0000","last_peered":"2024-04-25T09:07:27.627354+0000","last_clean":"2024-04-25T09:07:27.627354+0000","last_became_active":"2024-04-25T09:07:26.016368+0000","last_became_peered":"2024-04-25T09:07:26.016368+0000","last_unstale":"2024-04-25T09:07:27.627354+0000","last_undegraded":"2024-04-25T09:07:27.627354+0000","last_fullsized":"2024-04-25T09:07:27.627354+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:11.998769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095724+0000","last_change":"2024-04-25T09:07:26.004394+0000","last_active":"2024-04-25T09:07:27.095724+0000","last_peered":"2024-04-25T09:07:27.095724+0000","last_clean":"2024-04-25T09:07:27.095724+0000","last_became_active":"2024-04-25T09:07:26.003907+0000","last_became_peered":"2024-04-25T09:07:26.003907+0000","last_unstale":"2024-04-25T09:07:27.095724+0000","last_undegraded":"2024-04-25T09:07:27.095724+0000","last_fullsized":"2024-04-25T09:07:27.095724+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:25.097739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005043+0000","last_change":"2024-04-25T09:07:26.007063+0000","last_active":"2024-04-25T09:07:27.005043+0000","last_peered":"2024-04-25T09:07:27.005043+0000","last_clean":"2024-04-25T09:07:27.005043+0000","last_became_active":"2024-04-25T09:07:26.006193+0000","last_became_peered":"2024-04-25T09:07:26.006193+0000","last_unstale":"2024-04-25T09:07:27.005043+0000","last_undegraded":"2024-04-25T09:07:27.005043+0000","last_fullsized":"2024-04-25T09:07:27.005043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.581450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.013311+0000","last_change":"2024-04-25T09:07:26.019340+0000","last_active":"2024-04-25T09:07:27.013311+0000","last_peered":"2024-04-25T09:07:27.013311+0000","last_clean":"2024-04-25T09:07:27.013311+0000","last_became_active":"2024-04-25T09:07:26.018933+0000","last_became_peered":"2024-04-25T09:07:26.018933+0000","last_unstale":"2024-04-25T09:07:27.013311+0000","last_undegraded":"2024-04-25T09:07:27.013311+0000","last_fullsized":"2024-04-25T09:07:27.013311+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.794953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627210+0000","last_change":"2024-04-25T09:07:25.999364+0000","last_active":"2024-04-25T09:07:27.627210+0000","last_peered":"2024-04-25T09:07:27.627210+0000","last_clean":"2024-04-25T09:07:27.627210+0000","last_became_active":"2024-04-25T09:07:25.999056+0000","last_became_peered":"2024-04-25T09:07:25.999056+0000","last_unstale":"2024-04-25T09:07:27.627210+0000","last_undegraded":"2024-04-25T09:07:27.627210+0000","last_fullsized":"2024-04-25T09:07:27.627210+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:54.734490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629870+0000","last_change":"2024-04-25T09:07:26.010154+0000","last_active":"2024-04-25T09:07:27.629870+0000","last_peered":"2024-04-25T09:07:27.629870+0000","last_clean":"2024-04-25T09:07:27.629870+0000","last_became_active":"2024-04-25T09:07:26.009860+0000","last_became_peered":"2024-04-25T09:07:26.009860+0000","last_unstale":"2024-04-25T09:07:27.629870+0000","last_undegraded":"2024-04-25T09:07:27.629870+0000","last_fullsized":"2024-04-25T09:07:27.629870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:52.139257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017599+0000","last_change":"2024-04-25T09:07:26.021448+0000","last_active":"2024-04-25T09:07:27.017599+0000","last_peered":"2024-04-25T09:07:27.017599+0000","last_clean":"2024-04-25T09:07:27.017599+0000","last_became_active":"2024-04-25T09:07:26.021127+0000","last_became_peered":"2024-04-25T09:07:26.021127+0000","last_unstale":"2024-04-25T09:07:27.017599+0000","last_undegraded":"2024-04-25T09:07:27.017599+0000","last_fullsized":"2024-04-25T09:07:27.017599+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:00.152037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012009+0000","last_change":"2024-04-25T09:07:26.015054+0000","last_active":"2024-04-25T09:07:27.012009+0000","last_peered":"2024-04-25T09:07:27.012009+0000","last_clean":"2024-04-25T09:07:27.012009+0000","last_became_active":"2024-04-25T09:07:26.014723+0000","last_became_peered":"2024-04-25T09:07:26.014723+0000","last_unstale":"2024-04-25T09:07:27.012009+0000","last_undegraded":"2024-04-25T09:07:27.012009+0000","last_fullsized":"2024-04-25T09:07:27.012009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:11.998769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629344+0000","last_change":"2024-04-25T09:07:26.002430+0000","last_active":"2024-04-25T09:07:27.629344+0000","last_peered":"2024-04-25T09:07:27.629344+0000","last_clean":"2024-04-25T09:07:27.629344+0000","last_became_active":"2024-04-25T09:07:26.000950+0000","last_became_peered":"2024-04-25T09:07:26.000950+0000","last_unstale":"2024-04-25T09:07:27.629344+0000","last_undegraded":"2024-04-25T09:07:27.629344+0000","last_fullsized":"2024-04-25T09:07:27.629344+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:37.550711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.008472+0000","last_change":"2024-04-25T09:07:26.001972+0000","last_active":"2024-04-25T09:07:27.008472+0000","last_peered":"2024-04-25T09:07:27.008472+0000","last_clean":"2024-04-25T09:07:27.008472+0000","last_became_active":"2024-04-25T09:07:26.001620+0000","last_became_peered":"2024-04-25T09:07:26.001620+0000","last_unstale":"2024-04-25T09:07:27.008472+0000","last_undegraded":"2024-04-25T09:07:27.008472+0000","last_fullsized":"2024-04-25T09:07:27.008472+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:51.794953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012899+0000","last_change":"2024-04-25T09:07:26.006192+0000","last_active":"2024-04-25T09:07:27.012899+0000","last_peered":"2024-04-25T09:07:27.012899+0000","last_clean":"2024-04-25T09:07:27.012899+0000","last_became_active":"2024-04-25T09:07:26.005914+0000","last_became_peered":"2024-04-25T09:07:26.005914+0000","last_unstale":"2024-04-25T09:07:27.012899+0000","last_undegraded":"2024-04-25T09:07:27.012899+0000","last_fullsized":"2024-04-25T09:07:27.012899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:43.938796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095873+0000","last_change":"2024-04-25T09:07:26.006885+0000","last_active":"2024-04-25T09:07:27.095873+0000","last_peered":"2024-04-25T09:07:27.095873+0000","last_clean":"2024-04-25T09:07:27.095873+0000","last_became_active":"2024-04-25T09:07:26.006574+0000","last_became_peered":"2024-04-25T09:07:26.006574+0000","last_unstale":"2024-04-25T09:07:27.095873+0000","last_undegraded":"2024-04-25T09:07:27.095873+0000","last_fullsized":"2024-04-25T09:07:27.095873+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:11.998769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628608+0000","last_change":"2024-04-25T09:07:26.008467+0000","last_active":"2024-04-25T09:07:27.628608+0000","last_peered":"2024-04-25T09:07:27.628608+0000","last_clean":"2024-04-25T09:07:27.628608+0000","last_became_active":"2024-04-25T09:07:26.008184+0000","last_became_peered":"2024-04-25T09:07:26.008184+0000","last_unstale":"2024-04-25T09:07:27.628608+0000","last_undegraded":"2024-04-25T09:07:27.628608+0000","last_fullsized":"2024-04-25T09:07:27.628608+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:08.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012036+0000","last_change":"2024-04-25T09:07:26.010106+0000","last_active":"2024-04-25T09:07:27.012036+0000","last_peered":"2024-04-25T09:07:27.012036+0000","last_clean":"2024-04-25T09:07:27.012036+0000","last_became_active":"2024-04-25T09:07:26.009780+0000","last_became_peered":"2024-04-25T09:07:26.009780+0000","last_unstale":"2024-04-25T09:07:27.012036+0000","last_undegraded":"2024-04-25T09:07:27.012036+0000","last_fullsized":"2024-04-25T09:07:27.012036+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:59.375662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.011739+0000","last_change":"2024-04-25T09:07:26.017867+0000","last_active":"2024-04-25T09:07:27.011739+0000","last_peered":"2024-04-25T09:07:27.011739+0000","last_clean":"2024-04-25T09:07:27.011739+0000","last_became_active":"2024-04-25T09:07:26.017165+0000","last_became_peered":"2024-04-25T09:07:26.017165+0000","last_unstale":"2024-04-25T09:07:27.011739+0000","last_undegraded":"2024-04-25T09:07:27.011739+0000","last_fullsized":"2024-04-25T09:07:27.011739+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:25.097739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627371+0000","last_change":"2024-04-25T09:07:26.013143+0000","last_active":"2024-04-25T09:07:27.627371+0000","last_peered":"2024-04-25T09:07:27.627371+0000","last_clean":"2024-04-25T09:07:27.627371+0000","last_became_active":"2024-04-25T09:07:26.012393+0000","last_became_peered":"2024-04-25T09:07:26.012393+0000","last_unstale":"2024-04-25T09:07:27.627371+0000","last_undegraded":"2024-04-25T09:07:27.627371+0000","last_fullsized":"2024-04-25T09:07:27.627371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:59.375662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095886+0000","last_change":"2024-04-25T09:07:26.019563+0000","last_active":"2024-04-25T09:07:27.095886+0000","last_peered":"2024-04-25T09:07:27.095886+0000","last_clean":"2024-04-25T09:07:27.095886+0000","last_became_active":"2024-04-25T09:07:26.019237+0000","last_became_peered":"2024-04-25T09:07:26.019237+0000","last_unstale":"2024-04-25T09:07:27.095886+0000","last_undegraded":"2024-04-25T09:07:27.095886+0000","last_fullsized":"2024-04-25T09:07:27.095886+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:59.375662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005303+0000","last_change":"2024-04-25T09:07:26.015812+0000","last_active":"2024-04-25T09:07:27.005303+0000","last_peered":"2024-04-25T09:07:27.005303+0000","last_clean":"2024-04-25T09:07:27.005303+0000","last_became_active":"2024-04-25T09:07:26.014351+0000","last_became_peered":"2024-04-25T09:07:26.014351+0000","last_unstale":"2024-04-25T09:07:27.005303+0000","last_undegraded":"2024-04-25T09:07:27.005303+0000","last_fullsized":"2024-04-25T09:07:27.005303+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:42.685590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627161+0000","last_change":"2024-04-25T09:07:26.005702+0000","last_active":"2024-04-25T09:07:27.627161+0000","last_peered":"2024-04-25T09:07:27.627161+0000","last_clean":"2024-04-25T09:07:27.627161+0000","last_became_active":"2024-04-25T09:07:26.005377+0000","last_became_peered":"2024-04-25T09:07:26.005377+0000","last_unstale":"2024-04-25T09:07:27.627161+0000","last_undegraded":"2024-04-25T09:07:27.627161+0000","last_fullsized":"2024-04-25T09:07:27.627161+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:25.097739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007601+0000","last_change":"2024-04-25T09:07:25.992649+0000","last_active":"2024-04-25T09:07:27.007601+0000","last_peered":"2024-04-25T09:07:27.007601+0000","last_clean":"2024-04-25T09:07:27.007601+0000","last_became_active":"2024-04-25T09:07:25.992349+0000","last_became_peered":"2024-04-25T09:07:25.992349+0000","last_unstale":"2024-04-25T09:07:27.007601+0000","last_undegraded":"2024-04-25T09:07:27.007601+0000","last_fullsized":"2024-04-25T09:07:27.007601+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:43.938796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012682+0000","last_change":"2024-04-25T09:07:26.010761+0000","last_active":"2024-04-25T09:07:27.012682+0000","last_peered":"2024-04-25T09:07:27.012682+0000","last_clean":"2024-04-25T09:07:27.012682+0000","last_became_active":"2024-04-25T09:07:26.010211+0000","last_became_peered":"2024-04-25T09:07:26.010211+0000","last_unstale":"2024-04-25T09:07:27.012682+0000","last_undegraded":"2024-04-25T09:07:27.012682+0000","last_fullsized":"2024-04-25T09:07:27.012682+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:34.041821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627122+0000","last_change":"2024-04-25T09:07:26.004057+0000","last_active":"2024-04-25T09:07:27.627122+0000","last_peered":"2024-04-25T09:07:27.627122+0000","last_clean":"2024-04-25T09:07:27.627122+0000","last_became_active":"2024-04-25T09:07:26.003763+0000","last_became_peered":"2024-04-25T09:07:26.003763+0000","last_unstale":"2024-04-25T09:07:27.627122+0000","last_undegraded":"2024-04-25T09:07:27.627122+0000","last_fullsized":"2024-04-25T09:07:27.627122+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:49.581450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017863+0000","last_change":"2024-04-25T09:07:26.007824+0000","last_active":"2024-04-25T09:07:27.017863+0000","last_peered":"2024-04-25T09:07:27.017863+0000","last_clean":"2024-04-25T09:07:27.017863+0000","last_became_active":"2024-04-25T09:07:26.007220+0000","last_became_peered":"2024-04-25T09:07:26.007220+0000","last_unstale":"2024-04-25T09:07:27.017863+0000","last_undegraded":"2024-04-25T09:07:27.017863+0000","last_fullsized":"2024-04-25T09:07:27.017863+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:35.491071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.096221+0000","last_change":"2024-04-25T09:07:26.015631+0000","last_active":"2024-04-25T09:07:27.096221+0000","last_peered":"2024-04-25T09:07:27.096221+0000","last_clean":"2024-04-25T09:07:27.096221+0000","last_became_active":"2024-04-25T09:07:26.015348+0000","last_became_peered":"2024-04-25T09:07:26.015348+0000","last_unstale":"2024-04-25T09:07:27.096221+0000","last_undegraded":"2024-04-25T09:07:27.096221+0000","last_fullsized":"2024-04-25T09:07:27.096221+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:34.041821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627447+0000","last_change":"2024-04-25T09:07:26.006204+0000","last_active":"2024-04-25T09:07:27.627447+0000","last_peered":"2024-04-25T09:07:27.627447+0000","last_clean":"2024-04-25T09:07:27.627447+0000","last_became_active":"2024-04-25T09:07:26.004871+0000","last_became_peered":"2024-04-25T09:07:26.004871+0000","last_unstale":"2024-04-25T09:07:27.627447+0000","last_undegraded":"2024-04-25T09:07:27.627447+0000","last_fullsized":"2024-04-25T09:07:27.627447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:16.317310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012850+0000","last_change":"2024-04-25T09:07:26.009652+0000","last_active":"2024-04-25T09:07:27.012850+0000","last_peered":"2024-04-25T09:07:27.012850+0000","last_clean":"2024-04-25T09:07:27.012850+0000","last_became_active":"2024-04-25T09:07:26.007525+0000","last_became_peered":"2024-04-25T09:07:26.007525+0000","last_unstale":"2024-04-25T09:07:27.012850+0000","last_undegraded":"2024-04-25T09:07:27.012850+0000","last_fullsized":"2024-04-25T09:07:27.012850+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:40.171244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017771+0000","last_change":"2024-04-25T09:07:26.022588+0000","last_active":"2024-04-25T09:07:27.017771+0000","last_peered":"2024-04-25T09:07:27.017771+0000","last_clean":"2024-04-25T09:07:27.017771+0000","last_became_active":"2024-04-25T09:07:26.022302+0000","last_became_peered":"2024-04-25T09:07:26.022302+0000","last_unstale":"2024-04-25T09:07:27.017771+0000","last_undegraded":"2024-04-25T09:07:27.017771+0000","last_fullsized":"2024-04-25T09:07:27.017771+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:37.208922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018323+0000","last_change":"2024-04-25T09:07:26.016684+0000","last_active":"2024-04-25T09:07:27.018323+0000","last_peered":"2024-04-25T09:07:27.018323+0000","last_clean":"2024-04-25T09:07:27.018323+0000","last_became_active":"2024-04-25T09:07:26.009349+0000","last_became_peered":"2024-04-25T09:07:26.009349+0000","last_unstale":"2024-04-25T09:07:27.018323+0000","last_undegraded":"2024-04-25T09:07:27.018323+0000","last_fullsized":"2024-04-25T09:07:27.018323+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:28.370111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626586+0000","last_change":"2024-04-25T09:07:26.000007+0000","last_active":"2024-04-25T09:07:27.626586+0000","last_peered":"2024-04-25T09:07:27.626586+0000","last_clean":"2024-04-25T09:07:27.626586+0000","last_became_active":"2024-04-25T09:07:25.997357+0000","last_became_peered":"2024-04-25T09:07:25.997357+0000","last_unstale":"2024-04-25T09:07:27.626586+0000","last_undegraded":"2024-04-25T09:07:27.626586+0000","last_fullsized":"2024-04-25T09:07:27.626586+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:00.300113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095599+0000","last_change":"2024-04-25T09:07:26.007780+0000","last_active":"2024-04-25T09:07:27.095599+0000","last_peered":"2024-04-25T09:07:27.095599+0000","last_clean":"2024-04-25T09:07:27.095599+0000","last_became_active":"2024-04-25T09:07:26.006868+0000","last_became_peered":"2024-04-25T09:07:26.006868+0000","last_unstale":"2024-04-25T09:07:27.095599+0000","last_undegraded":"2024-04-25T09:07:27.095599+0000","last_fullsized":"2024-04-25T09:07:27.095599+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:00.300113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629903+0000","last_change":"2024-04-25T09:07:26.002598+0000","last_active":"2024-04-25T09:07:27.629903+0000","last_peered":"2024-04-25T09:07:27.629903+0000","last_clean":"2024-04-25T09:07:27.629903+0000","last_became_active":"2024-04-25T09:07:26.002300+0000","last_became_peered":"2024-04-25T09:07:26.002300+0000","last_unstale":"2024-04-25T09:07:27.629903+0000","last_undegraded":"2024-04-25T09:07:27.629903+0000","last_fullsized":"2024-04-25T09:07:27.629903+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:10.617565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012230+0000","last_change":"2024-04-25T09:07:26.007946+0000","last_active":"2024-04-25T09:07:27.012230+0000","last_peered":"2024-04-25T09:07:27.012230+0000","last_clean":"2024-04-25T09:07:27.012230+0000","last_became_active":"2024-04-25T09:07:26.007517+0000","last_became_peered":"2024-04-25T09:07:26.007517+0000","last_unstale":"2024-04-25T09:07:27.012230+0000","last_undegraded":"2024-04-25T09:07:27.012230+0000","last_fullsized":"2024-04-25T09:07:27.012230+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.619139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627665+0000","last_change":"2024-04-25T09:07:25.995876+0000","last_active":"2024-04-25T09:07:27.627665+0000","last_peered":"2024-04-25T09:07:27.627665+0000","last_clean":"2024-04-25T09:07:27.627665+0000","last_became_active":"2024-04-25T09:07:25.995428+0000","last_became_peered":"2024-04-25T09:07:25.995428+0000","last_unstale":"2024-04-25T09:07:27.627665+0000","last_undegraded":"2024-04-25T09:07:27.627665+0000","last_fullsized":"2024-04-25T09:07:27.627665+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:48.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629501+0000","last_change":"2024-04-25T09:07:25.010479+0000","last_active":"2024-04-25T09:07:27.629501+0000","last_peered":"2024-04-25T09:07:27.629501+0000","last_clean":"2024-04-25T09:07:27.629501+0000","last_became_active":"2024-04-25T09:07:25.009971+0000","last_became_peered":"2024-04-25T09:07:25.009971+0000","last_unstale":"2024-04-25T09:07:27.629501+0000","last_undegraded":"2024-04-25T09:07:27.629501+0000","last_fullsized":"2024-04-25T09:07:27.629501+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:51.789498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627733+0000","last_change":"2024-04-25T09:07:26.000285+0000","last_active":"2024-04-25T09:07:27.627733+0000","last_peered":"2024-04-25T09:07:27.627733+0000","last_clean":"2024-04-25T09:07:27.627733+0000","last_became_active":"2024-04-25T09:07:25.999478+0000","last_became_peered":"2024-04-25T09:07:25.999478+0000","last_unstale":"2024-04-25T09:07:27.627733+0000","last_undegraded":"2024-04-25T09:07:27.627733+0000","last_fullsized":"2024-04-25T09:07:27.627733+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.619139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018093+0000","last_change":"2024-04-25T09:07:25.012693+0000","last_active":"2024-04-25T09:07:27.018093+0000","last_peered":"2024-04-25T09:07:27.018093+0000","last_clean":"2024-04-25T09:07:27.018093+0000","last_became_active":"2024-04-25T09:07:25.012366+0000","last_became_peered":"2024-04-25T09:07:25.012366+0000","last_unstale":"2024-04-25T09:07:27.018093+0000","last_undegraded":"2024-04-25T09:07:27.018093+0000","last_fullsized":"2024-04-25T09:07:27.018093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:34.864604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.096087+0000","last_change":"2024-04-25T09:07:26.008674+0000","last_active":"2024-04-25T09:07:27.096087+0000","last_peered":"2024-04-25T09:07:27.096087+0000","last_clean":"2024-04-25T09:07:27.096087+0000","last_became_active":"2024-04-25T09:07:26.008177+0000","last_became_peered":"2024-04-25T09:07:26.008177+0000","last_unstale":"2024-04-25T09:07:27.096087+0000","last_undegraded":"2024-04-25T09:07:27.096087+0000","last_fullsized":"2024-04-25T09:07:27.096087+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:23.228521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629122+0000","last_change":"2024-04-25T09:07:25.011946+0000","last_active":"2024-04-25T09:07:27.629122+0000","last_peered":"2024-04-25T09:07:27.629122+0000","last_clean":"2024-04-25T09:07:27.629122+0000","last_became_active":"2024-04-25T09:07:25.011630+0000","last_became_peered":"2024-04-25T09:07:25.011630+0000","last_unstale":"2024-04-25T09:07:27.629122+0000","last_undegraded":"2024-04-25T09:07:27.629122+0000","last_fullsized":"2024-04-25T09:07:27.629122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:33.841946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.013009+0000","last_change":"2024-04-25T09:07:26.022576+0000","last_active":"2024-04-25T09:07:27.013009+0000","last_peered":"2024-04-25T09:07:27.013009+0000","last_clean":"2024-04-25T09:07:27.013009+0000","last_became_active":"2024-04-25T09:07:26.022271+0000","last_became_peered":"2024-04-25T09:07:26.022271+0000","last_unstale":"2024-04-25T09:07:27.013009+0000","last_undegraded":"2024-04-25T09:07:27.013009+0000","last_fullsized":"2024-04-25T09:07:27.013009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:10.384284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018370+0000","last_change":"2024-04-25T09:07:25.010814+0000","last_active":"2024-04-25T09:07:27.018370+0000","last_peered":"2024-04-25T09:07:27.018370+0000","last_clean":"2024-04-25T09:07:27.018370+0000","last_became_active":"2024-04-25T09:07:25.008979+0000","last_became_peered":"2024-04-25T09:07:25.008979+0000","last_unstale":"2024-04-25T09:07:27.018370+0000","last_undegraded":"2024-04-25T09:07:27.018370+0000","last_fullsized":"2024-04-25T09:07:27.018370+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:50.778845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018088+0000","last_change":"2024-04-25T09:07:26.020848+0000","last_active":"2024-04-25T09:07:27.018088+0000","last_peered":"2024-04-25T09:07:27.018088+0000","last_clean":"2024-04-25T09:07:27.018088+0000","last_became_active":"2024-04-25T09:07:26.020534+0000","last_became_peered":"2024-04-25T09:07:26.020534+0000","last_unstale":"2024-04-25T09:07:27.018088+0000","last_undegraded":"2024-04-25T09:07:27.018088+0000","last_fullsized":"2024-04-25T09:07:27.018088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:06.982333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629181+0000","last_change":"2024-04-25T09:07:25.010295+0000","last_active":"2024-04-25T09:07:27.629181+0000","last_peered":"2024-04-25T09:07:27.629181+0000","last_clean":"2024-04-25T09:07:27.629181+0000","last_became_active":"2024-04-25T09:07:25.009984+0000","last_became_peered":"2024-04-25T09:07:25.009984+0000","last_unstale":"2024-04-25T09:07:27.629181+0000","last_undegraded":"2024-04-25T09:07:27.629181+0000","last_fullsized":"2024-04-25T09:07:27.629181+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:13.587386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007143+0000","last_change":"2024-04-25T09:07:26.008439+0000","last_active":"2024-04-25T09:07:27.007143+0000","last_peered":"2024-04-25T09:07:27.007143+0000","last_clean":"2024-04-25T09:07:27.007143+0000","last_became_active":"2024-04-25T09:07:26.007383+0000","last_became_peered":"2024-04-25T09:07:26.007383+0000","last_unstale":"2024-04-25T09:07:27.007143+0000","last_undegraded":"2024-04-25T09:07:27.007143+0000","last_fullsized":"2024-04-25T09:07:27.007143+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:40.715423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017754+0000","last_change":"2024-04-25T09:07:25.005618+0000","last_active":"2024-04-25T09:07:27.017754+0000","last_peered":"2024-04-25T09:07:27.017754+0000","last_clean":"2024-04-25T09:07:27.017754+0000","last_became_active":"2024-04-25T09:07:25.005276+0000","last_became_peered":"2024-04-25T09:07:25.005276+0000","last_unstale":"2024-04-25T09:07:27.017754+0000","last_undegraded":"2024-04-25T09:07:27.017754+0000","last_fullsized":"2024-04-25T09:07:27.017754+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.993720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.006839+0000","last_change":"2024-04-25T09:07:26.009183+0000","last_active":"2024-04-25T09:07:27.006839+0000","last_peered":"2024-04-25T09:07:27.006839+0000","last_clean":"2024-04-25T09:07:27.006839+0000","last_became_active":"2024-04-25T09:07:26.008849+0000","last_became_peered":"2024-04-25T09:07:26.008849+0000","last_unstale":"2024-04-25T09:07:27.006839+0000","last_undegraded":"2024-04-25T09:07:27.006839+0000","last_fullsized":"2024-04-25T09:07:27.006839+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:34.041821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626727+0000","last_change":"2024-04-25T09:07:25.014784+0000","last_active":"2024-04-25T09:07:27.626727+0000","last_peered":"2024-04-25T09:07:27.626727+0000","last_clean":"2024-04-25T09:07:27.626727+0000","last_became_active":"2024-04-25T09:07:25.012907+0000","last_became_peered":"2024-04-25T09:07:25.012907+0000","last_unstale":"2024-04-25T09:07:27.626727+0000","last_undegraded":"2024-04-25T09:07:27.626727+0000","last_fullsized":"2024-04-25T09:07:27.626727+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.575417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017720+0000","last_change":"2024-04-25T09:07:26.008053+0000","last_active":"2024-04-25T09:07:27.017720+0000","last_peered":"2024-04-25T09:07:27.017720+0000","last_clean":"2024-04-25T09:07:27.017720+0000","last_became_active":"2024-04-25T09:07:26.007557+0000","last_became_peered":"2024-04-25T09:07:26.007557+0000","last_unstale":"2024-04-25T09:07:27.017720+0000","last_undegraded":"2024-04-25T09:07:27.017720+0000","last_fullsized":"2024-04-25T09:07:27.017720+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:53.818889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007098+0000","last_change":"2024-04-25T09:07:25.009763+0000","last_active":"2024-04-25T09:07:27.007098+0000","last_peered":"2024-04-25T09:07:27.007098+0000","last_clean":"2024-04-25T09:07:27.007098+0000","last_became_active":"2024-04-25T09:07:25.009474+0000","last_became_peered":"2024-04-25T09:07:25.009474+0000","last_unstale":"2024-04-25T09:07:27.007098+0000","last_undegraded":"2024-04-25T09:07:27.007098+0000","last_fullsized":"2024-04-25T09:07:27.007098+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:25.750332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005544+0000","last_change":"2024-04-25T09:07:26.006903+0000","last_active":"2024-04-25T09:07:27.005544+0000","last_peered":"2024-04-25T09:07:27.005544+0000","last_clean":"2024-04-25T09:07:27.005544+0000","last_became_active":"2024-04-25T09:07:26.003814+0000","last_became_peered":"2024-04-25T09:07:26.003814+0000","last_unstale":"2024-04-25T09:07:27.005544+0000","last_undegraded":"2024-04-25T09:07:27.005544+0000","last_fullsized":"2024-04-25T09:07:27.005544+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:28.513205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005540+0000","last_change":"2024-04-25T09:07:25.007040+0000","last_active":"2024-04-25T09:07:27.005540+0000","last_peered":"2024-04-25T09:07:27.005540+0000","last_clean":"2024-04-25T09:07:27.005540+0000","last_became_active":"2024-04-25T09:07:25.006742+0000","last_became_peered":"2024-04-25T09:07:25.006742+0000","last_unstale":"2024-04-25T09:07:27.005540+0000","last_undegraded":"2024-04-25T09:07:27.005540+0000","last_fullsized":"2024-04-25T09:07:27.005540+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:29.835765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005689+0000","last_change":"2024-04-25T09:07:26.002649+0000","last_active":"2024-04-25T09:07:27.005689+0000","last_peered":"2024-04-25T09:07:27.005689+0000","last_clean":"2024-04-25T09:07:27.005689+0000","last_became_active":"2024-04-25T09:07:26.002339+0000","last_became_peered":"2024-04-25T09:07:26.002339+0000","last_unstale":"2024-04-25T09:07:27.005689+0000","last_undegraded":"2024-04-25T09:07:27.005689+0000","last_fullsized":"2024-04-25T09:07:27.005689+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:52.805606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.008483+0000","last_change":"2024-04-25T09:07:25.016674+0000","last_active":"2024-04-25T09:07:27.008483+0000","last_peered":"2024-04-25T09:07:27.008483+0000","last_clean":"2024-04-25T09:07:27.008483+0000","last_became_active":"2024-04-25T09:07:25.016317+0000","last_became_peered":"2024-04-25T09:07:25.016317+0000","last_unstale":"2024-04-25T09:07:27.008483+0000","last_undegraded":"2024-04-25T09:07:27.008483+0000","last_fullsized":"2024-04-25T09:07:27.008483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:55.318929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012865+0000","last_change":"2024-04-25T09:07:26.012586+0000","last_active":"2024-04-25T09:07:27.012865+0000","last_peered":"2024-04-25T09:07:27.012865+0000","last_clean":"2024-04-25T09:07:27.012865+0000","last_became_active":"2024-04-25T09:07:26.012196+0000","last_became_peered":"2024-04-25T09:07:26.012196+0000","last_unstale":"2024-04-25T09:07:27.012865+0000","last_undegraded":"2024-04-25T09:07:27.012865+0000","last_fullsized":"2024-04-25T09:07:27.012865+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:51.073028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095743+0000","last_change":"2024-04-25T09:07:25.008303+0000","last_active":"2024-04-25T09:07:27.095743+0000","last_peered":"2024-04-25T09:07:27.095743+0000","last_clean":"2024-04-25T09:07:27.095743+0000","last_became_active":"2024-04-25T09:07:25.006381+0000","last_became_peered":"2024-04-25T09:07:25.006381+0000","last_unstale":"2024-04-25T09:07:27.095743+0000","last_undegraded":"2024-04-25T09:07:27.095743+0000","last_fullsized":"2024-04-25T09:07:27.095743+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:55.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017944+0000","last_change":"2024-04-25T09:07:26.020831+0000","last_active":"2024-04-25T09:07:27.017944+0000","last_peered":"2024-04-25T09:07:27.017944+0000","last_clean":"2024-04-25T09:07:27.017944+0000","last_became_active":"2024-04-25T09:07:26.020522+0000","last_became_peered":"2024-04-25T09:07:26.020522+0000","last_unstale":"2024-04-25T09:07:27.017944+0000","last_undegraded":"2024-04-25T09:07:27.017944+0000","last_fullsized":"2024-04-25T09:07:27.017944+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:53.955549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":20,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007893+0000","last_change":"2024-04-25T09:07:25.003604+0000","last_active":"2024-04-25T09:07:27.007893+0000","last_peered":"2024-04-25T09:07:27.007893+0000","last_clean":"2024-04-25T09:07:27.007893+0000","last_became_active":"2024-04-25T09:07:25.003266+0000","last_became_peered":"2024-04-25T09:07:25.003266+0000","last_unstale":"2024-04-25T09:07:27.007893+0000","last_undegraded":"2024-04-25T09:07:27.007893+0000","last_fullsized":"2024-04-25T09:07:27.007893+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:39.609538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.004758+0000","last_change":"2024-04-25T09:07:26.014052+0000","last_active":"2024-04-25T09:07:27.004758+0000","last_peered":"2024-04-25T09:07:27.004758+0000","last_clean":"2024-04-25T09:07:27.004758+0000","last_became_active":"2024-04-25T09:07:26.013779+0000","last_became_peered":"2024-04-25T09:07:26.013779+0000","last_unstale":"2024-04-25T09:07:27.004758+0000","last_undegraded":"2024-04-25T09:07:27.004758+0000","last_fullsized":"2024-04-25T09:07:27.004758+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:46.543920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626797+0000","last_change":"2024-04-25T09:07:25.006123+0000","last_active":"2024-04-25T09:07:27.626797+0000","last_peered":"2024-04-25T09:07:27.626797+0000","last_clean":"2024-04-25T09:07:27.626797+0000","last_became_active":"2024-04-25T09:07:25.004723+0000","last_became_peered":"2024-04-25T09:07:25.004723+0000","last_unstale":"2024-04-25T09:07:27.626797+0000","last_undegraded":"2024-04-25T09:07:27.626797+0000","last_fullsized":"2024-04-25T09:07:27.626797+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:25.526901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626806+0000","last_change":"2024-04-25T09:07:25.998673+0000","last_active":"2024-04-25T09:07:27.626806+0000","last_peered":"2024-04-25T09:07:27.626806+0000","last_clean":"2024-04-25T09:07:27.626806+0000","last_became_active":"2024-04-25T09:07:25.998396+0000","last_became_peered":"2024-04-25T09:07:25.998396+0000","last_unstale":"2024-04-25T09:07:27.626806+0000","last_undegraded":"2024-04-25T09:07:27.626806+0000","last_fullsized":"2024-04-25T09:07:27.626806+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:46.543920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626256+0000","last_change":"2024-04-25T09:07:25.004430+0000","last_active":"2024-04-25T09:07:27.626256+0000","last_peered":"2024-04-25T09:07:27.626256+0000","last_clean":"2024-04-25T09:07:27.626256+0000","last_became_active":"2024-04-25T09:07:25.000012+0000","last_became_peered":"2024-04-25T09:07:25.000012+0000","last_unstale":"2024-04-25T09:07:27.626256+0000","last_undegraded":"2024-04-25T09:07:27.626256+0000","last_fullsized":"2024-04-25T09:07:27.626256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:55.800979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628814+0000","last_change":"2024-04-25T09:07:25.992553+0000","last_active":"2024-04-25T09:07:27.628814+0000","last_peered":"2024-04-25T09:07:27.628814+0000","last_clean":"2024-04-25T09:07:27.628814+0000","last_became_active":"2024-04-25T09:07:25.992105+0000","last_became_peered":"2024-04-25T09:07:25.992105+0000","last_unstale":"2024-04-25T09:07:27.628814+0000","last_undegraded":"2024-04-25T09:07:27.628814+0000","last_fullsized":"2024-04-25T09:07:27.628814+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:52.805606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628811+0000","last_change":"2024-04-25T09:07:25.013988+0000","last_active":"2024-04-25T09:07:27.628811+0000","last_peered":"2024-04-25T09:07:27.628811+0000","last_clean":"2024-04-25T09:07:27.628811+0000","last_became_active":"2024-04-25T09:07:25.013681+0000","last_became_peered":"2024-04-25T09:07:25.013681+0000","last_unstale":"2024-04-25T09:07:27.628811+0000","last_undegraded":"2024-04-25T09:07:27.628811+0000","last_fullsized":"2024-04-25T09:07:27.628811+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:41.669482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095796+0000","last_change":"2024-04-25T09:07:26.001821+0000","last_active":"2024-04-25T09:07:27.095796+0000","last_peered":"2024-04-25T09:07:27.095796+0000","last_clean":"2024-04-25T09:07:27.095796+0000","last_became_active":"2024-04-25T09:07:26.001396+0000","last_became_peered":"2024-04-25T09:07:26.001396+0000","last_unstale":"2024-04-25T09:07:27.095796+0000","last_undegraded":"2024-04-25T09:07:27.095796+0000","last_fullsized":"2024-04-25T09:07:27.095796+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:54.734490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":22,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007349+0000","last_change":"2024-04-25T09:07:25.006626+0000","last_active":"2024-04-25T09:07:27.007349+0000","last_peered":"2024-04-25T09:07:27.007349+0000","last_clean":"2024-04-25T09:07:27.007349+0000","last_became_active":"2024-04-25T09:07:25.006323+0000","last_became_peered":"2024-04-25T09:07:25.006323+0000","last_unstale":"2024-04-25T09:07:27.007349+0000","last_undegraded":"2024-04-25T09:07:27.007349+0000","last_fullsized":"2024-04-25T09:07:27.007349+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:02.442135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628115+0000","last_change":"2024-04-25T09:07:26.005948+0000","last_active":"2024-04-25T09:07:27.628115+0000","last_peered":"2024-04-25T09:07:27.628115+0000","last_clean":"2024-04-25T09:07:27.628115+0000","last_became_active":"2024-04-25T09:07:26.004879+0000","last_became_peered":"2024-04-25T09:07:26.004879+0000","last_unstale":"2024-04-25T09:07:27.628115+0000","last_undegraded":"2024-04-25T09:07:27.628115+0000","last_fullsized":"2024-04-25T09:07:27.628115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:42.685590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012918+0000","last_change":"2024-04-25T09:07:25.016157+0000","last_active":"2024-04-25T09:07:27.012918+0000","last_peered":"2024-04-25T09:07:27.012918+0000","last_clean":"2024-04-25T09:07:27.012918+0000","last_became_active":"2024-04-25T09:07:25.015731+0000","last_became_peered":"2024-04-25T09:07:25.015731+0000","last_unstale":"2024-04-25T09:07:27.012918+0000","last_undegraded":"2024-04-25T09:07:27.012918+0000","last_fullsized":"2024-04-25T09:07:27.012918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:08.030750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628224+0000","last_change":"2024-04-25T09:07:26.000165+0000","last_active":"2024-04-25T09:07:27.628224+0000","last_peered":"2024-04-25T09:07:27.628224+0000","last_clean":"2024-04-25T09:07:27.628224+0000","last_became_active":"2024-04-25T09:07:25.999872+0000","last_became_peered":"2024-04-25T09:07:25.999872+0000","last_unstale":"2024-04-25T09:07:27.628224+0000","last_undegraded":"2024-04-25T09:07:27.628224+0000","last_fullsized":"2024-04-25T09:07:27.628224+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:28.513205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005532+0000","last_change":"2024-04-25T09:07:25.010677+0000","last_active":"2024-04-25T09:07:27.005532+0000","last_peered":"2024-04-25T09:07:27.005532+0000","last_clean":"2024-04-25T09:07:27.005532+0000","last_became_active":"2024-04-25T09:07:25.010337+0000","last_became_peered":"2024-04-25T09:07:25.010337+0000","last_unstale":"2024-04-25T09:07:27.005532+0000","last_undegraded":"2024-04-25T09:07:27.005532+0000","last_fullsized":"2024-04-25T09:07:27.005532+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:26.558822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.095960+0000","last_change":"2024-04-25T09:07:26.015023+0000","last_active":"2024-04-25T09:07:27.095960+0000","last_peered":"2024-04-25T09:07:27.095960+0000","last_clean":"2024-04-25T09:07:27.095960+0000","last_became_active":"2024-04-25T09:07:26.014632+0000","last_became_peered":"2024-04-25T09:07:26.014632+0000","last_unstale":"2024-04-25T09:07:27.095960+0000","last_undegraded":"2024-04-25T09:07:27.095960+0000","last_fullsized":"2024-04-25T09:07:27.095960+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:16.317310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005217+0000","last_change":"2024-04-25T09:07:25.012576+0000","last_active":"2024-04-25T09:07:27.005217+0000","last_peered":"2024-04-25T09:07:27.005217+0000","last_clean":"2024-04-25T09:07:27.005217+0000","last_became_active":"2024-04-25T09:07:25.010949+0000","last_became_peered":"2024-04-25T09:07:25.010949+0000","last_unstale":"2024-04-25T09:07:27.005217+0000","last_undegraded":"2024-04-25T09:07:27.005217+0000","last_fullsized":"2024-04-25T09:07:27.005217+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.785650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.005121+0000","last_change":"2024-04-25T09:07:26.006235+0000","last_active":"2024-04-25T09:07:27.005121+0000","last_peered":"2024-04-25T09:07:27.005121+0000","last_clean":"2024-04-25T09:07:27.005121+0000","last_became_active":"2024-04-25T09:07:26.005829+0000","last_became_peered":"2024-04-25T09:07:26.005829+0000","last_unstale":"2024-04-25T09:07:27.005121+0000","last_undegraded":"2024-04-25T09:07:27.005121+0000","last_fullsized":"2024-04-25T09:07:27.005121+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:48.813959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629476+0000","last_change":"2024-04-25T09:07:25.013972+0000","last_active":"2024-04-25T09:07:27.629476+0000","last_peered":"2024-04-25T09:07:27.629476+0000","last_clean":"2024-04-25T09:07:27.629476+0000","last_became_active":"2024-04-25T09:07:25.013633+0000","last_became_peered":"2024-04-25T09:07:25.013633+0000","last_unstale":"2024-04-25T09:07:27.629476+0000","last_undegraded":"2024-04-25T09:07:27.629476+0000","last_fullsized":"2024-04-25T09:07:27.629476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:54.158011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007150+0000","last_change":"2024-04-25T09:07:25.996907+0000","last_active":"2024-04-25T09:07:27.007150+0000","last_peered":"2024-04-25T09:07:27.007150+0000","last_clean":"2024-04-25T09:07:27.007150+0000","last_became_active":"2024-04-25T09:07:25.996559+0000","last_became_peered":"2024-04-25T09:07:25.996559+0000","last_unstale":"2024-04-25T09:07:27.007150+0000","last_undegraded":"2024-04-25T09:07:27.007150+0000","last_fullsized":"2024-04-25T09:07:27.007150+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:40.171244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007079+0000","last_change":"2024-04-25T09:07:25.016649+0000","last_active":"2024-04-25T09:07:27.007079+0000","last_peered":"2024-04-25T09:07:27.007079+0000","last_clean":"2024-04-25T09:07:27.007079+0000","last_became_active":"2024-04-25T09:07:25.016387+0000","last_became_peered":"2024-04-25T09:07:25.016387+0000","last_unstale":"2024-04-25T09:07:27.007079+0000","last_undegraded":"2024-04-25T09:07:27.007079+0000","last_fullsized":"2024-04-25T09:07:27.007079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:45.873510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.628913+0000","last_change":"2024-04-25T09:07:26.008218+0000","last_active":"2024-04-25T09:07:27.628913+0000","last_peered":"2024-04-25T09:07:27.628913+0000","last_clean":"2024-04-25T09:07:27.628913+0000","last_became_active":"2024-04-25T09:07:26.007932+0000","last_became_peered":"2024-04-25T09:07:26.007932+0000","last_unstale":"2024-04-25T09:07:27.628913+0000","last_undegraded":"2024-04-25T09:07:27.628913+0000","last_fullsized":"2024-04-25T09:07:27.628913+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:56.817088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626189+0000","last_change":"2024-04-25T09:07:25.009598+0000","last_active":"2024-04-25T09:07:27.626189+0000","last_peered":"2024-04-25T09:07:27.626189+0000","last_clean":"2024-04-25T09:07:27.626189+0000","last_became_active":"2024-04-25T09:07:25.009294+0000","last_became_peered":"2024-04-25T09:07:25.009294+0000","last_unstale":"2024-04-25T09:07:27.626189+0000","last_undegraded":"2024-04-25T09:07:27.626189+0000","last_fullsized":"2024-04-25T09:07:27.626189+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:18.169959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012554+0000","last_change":"2024-04-25T09:07:26.018073+0000","last_active":"2024-04-25T09:07:27.012554+0000","last_peered":"2024-04-25T09:07:27.012554+0000","last_clean":"2024-04-25T09:07:27.012554+0000","last_became_active":"2024-04-25T09:07:26.017151+0000","last_became_peered":"2024-04-25T09:07:26.017151+0000","last_unstale":"2024-04-25T09:07:27.012554+0000","last_undegraded":"2024-04-25T09:07:27.012554+0000","last_fullsized":"2024-04-25T09:07:27.012554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:23.228521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.006638+0000","last_change":"2024-04-25T09:07:25.005217+0000","last_active":"2024-04-25T09:07:27.006638+0000","last_peered":"2024-04-25T09:07:27.006638+0000","last_clean":"2024-04-25T09:07:27.006638+0000","last_became_active":"2024-04-25T09:07:25.003714+0000","last_became_peered":"2024-04-25T09:07:25.003714+0000","last_unstale":"2024-04-25T09:07:27.006638+0000","last_undegraded":"2024-04-25T09:07:27.006638+0000","last_fullsized":"2024-04-25T09:07:27.006638+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.284005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.011958+0000","last_change":"2024-04-25T09:07:26.012395+0000","last_active":"2024-04-25T09:07:27.011958+0000","last_peered":"2024-04-25T09:07:27.011958+0000","last_clean":"2024-04-25T09:07:27.011958+0000","last_became_active":"2024-04-25T09:07:26.012075+0000","last_became_peered":"2024-04-25T09:07:26.012075+0000","last_unstale":"2024-04-25T09:07:27.011958+0000","last_undegraded":"2024-04-25T09:07:27.011958+0000","last_fullsized":"2024-04-25T09:07:27.011958+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:54.734490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017579+0000","last_change":"2024-04-25T09:07:25.003186+0000","last_active":"2024-04-25T09:07:27.017579+0000","last_peered":"2024-04-25T09:07:27.017579+0000","last_clean":"2024-04-25T09:07:27.017579+0000","last_became_active":"2024-04-25T09:07:25.002435+0000","last_became_peered":"2024-04-25T09:07:25.002435+0000","last_unstale":"2024-04-25T09:07:27.017579+0000","last_undegraded":"2024-04-25T09:07:27.017579+0000","last_fullsized":"2024-04-25T09:07:27.017579+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:39.699314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018286+0000","last_change":"2024-04-25T09:07:26.005884+0000","last_active":"2024-04-25T09:07:27.018286+0000","last_peered":"2024-04-25T09:07:27.018286+0000","last_clean":"2024-04-25T09:07:27.018286+0000","last_became_active":"2024-04-25T09:07:26.005516+0000","last_became_peered":"2024-04-25T09:07:26.005516+0000","last_unstale":"2024-04-25T09:07:27.018286+0000","last_undegraded":"2024-04-25T09:07:27.018286+0000","last_fullsized":"2024-04-25T09:07:27.018286+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:58.338563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.013242+0000","last_change":"2024-04-25T09:07:25.015721+0000","last_active":"2024-04-25T09:07:27.013242+0000","last_peered":"2024-04-25T09:07:27.013242+0000","last_clean":"2024-04-25T09:07:27.013242+0000","last_became_active":"2024-04-25T09:07:25.015220+0000","last_became_peered":"2024-04-25T09:07:25.015220+0000","last_unstale":"2024-04-25T09:07:27.013242+0000","last_undegraded":"2024-04-25T09:07:27.013242+0000","last_fullsized":"2024-04-25T09:07:27.013242+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:31.406996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007758+0000","last_change":"2024-04-25T09:07:26.019229+0000","last_active":"2024-04-25T09:07:27.007758+0000","last_peered":"2024-04-25T09:07:27.007758+0000","last_clean":"2024-04-25T09:07:27.007758+0000","last_became_active":"2024-04-25T09:07:26.018552+0000","last_became_peered":"2024-04-25T09:07:26.018552+0000","last_unstale":"2024-04-25T09:07:27.007758+0000","last_undegraded":"2024-04-25T09:07:27.007758+0000","last_fullsized":"2024-04-25T09:07:27.007758+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:10.384284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627176+0000","last_change":"2024-04-25T09:07:25.010776+0000","last_active":"2024-04-25T09:07:27.627176+0000","last_peered":"2024-04-25T09:07:27.627176+0000","last_clean":"2024-04-25T09:07:27.627176+0000","last_became_active":"2024-04-25T09:07:25.010280+0000","last_became_peered":"2024-04-25T09:07:25.010280+0000","last_unstale":"2024-04-25T09:07:27.627176+0000","last_undegraded":"2024-04-25T09:07:27.627176+0000","last_fullsized":"2024-04-25T09:07:27.627176+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:36.534602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018146+0000","last_change":"2024-04-25T09:07:26.020850+0000","last_active":"2024-04-25T09:07:27.018146+0000","last_peered":"2024-04-25T09:07:27.018146+0000","last_clean":"2024-04-25T09:07:27.018146+0000","last_became_active":"2024-04-25T09:07:26.020043+0000","last_became_peered":"2024-04-25T09:07:26.020043+0000","last_unstale":"2024-04-25T09:07:27.018146+0000","last_undegraded":"2024-04-25T09:07:27.018146+0000","last_fullsized":"2024-04-25T09:07:27.018146+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:22.528760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627651+0000","last_change":"2024-04-25T09:07:25.007229+0000","last_active":"2024-04-25T09:07:27.627651+0000","last_peered":"2024-04-25T09:07:27.627651+0000","last_clean":"2024-04-25T09:07:27.627651+0000","last_became_active":"2024-04-25T09:07:25.006847+0000","last_became_peered":"2024-04-25T09:07:25.006847+0000","last_unstale":"2024-04-25T09:07:27.627651+0000","last_undegraded":"2024-04-25T09:07:27.627651+0000","last_fullsized":"2024-04-25T09:07:27.627651+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:07.376346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.007258+0000","last_change":"2024-04-25T09:07:25.998518+0000","last_active":"2024-04-25T09:07:27.007258+0000","last_peered":"2024-04-25T09:07:27.007258+0000","last_clean":"2024-04-25T09:07:27.007258+0000","last_became_active":"2024-04-25T09:07:25.998260+0000","last_became_peered":"2024-04-25T09:07:25.998260+0000","last_unstale":"2024-04-25T09:07:27.007258+0000","last_undegraded":"2024-04-25T09:07:27.007258+0000","last_fullsized":"2024-04-25T09:07:27.007258+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:51.073028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.626859+0000","last_change":"2024-04-25T09:07:25.011689+0000","last_active":"2024-04-25T09:07:27.626859+0000","last_peered":"2024-04-25T09:07:27.626859+0000","last_clean":"2024-04-25T09:07:27.626859+0000","last_became_active":"2024-04-25T09:07:25.011388+0000","last_became_peered":"2024-04-25T09:07:25.011388+0000","last_unstale":"2024-04-25T09:07:27.626859+0000","last_undegraded":"2024-04-25T09:07:27.626859+0000","last_fullsized":"2024-04-25T09:07:27.626859+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:51.123149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.012108+0000","last_change":"2024-04-25T09:07:26.009528+0000","last_active":"2024-04-25T09:07:27.012108+0000","last_peered":"2024-04-25T09:07:27.012108+0000","last_clean":"2024-04-25T09:07:27.012108+0000","last_became_active":"2024-04-25T09:07:26.009266+0000","last_became_peered":"2024-04-25T09:07:26.009266+0000","last_unstale":"2024-04-25T09:07:27.012108+0000","last_undegraded":"2024-04-25T09:07:27.012108+0000","last_fullsized":"2024-04-25T09:07:27.012108+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:16.317310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.017918+0000","last_change":"2024-04-25T09:07:25.004275+0000","last_active":"2024-04-25T09:07:27.017918+0000","last_peered":"2024-04-25T09:07:27.017918+0000","last_clean":"2024-04-25T09:07:27.017918+0000","last_became_active":"2024-04-25T09:07:25.004002+0000","last_became_peered":"2024-04-25T09:07:25.004002+0000","last_unstale":"2024-04-25T09:07:27.017918+0000","last_undegraded":"2024-04-25T09:07:27.017918+0000","last_fullsized":"2024-04-25T09:07:27.017918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:10.218708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.004278+0000","last_change":"2024-04-25T09:07:26.015332+0000","last_active":"2024-04-25T09:07:27.004278+0000","last_peered":"2024-04-25T09:07:27.004278+0000","last_clean":"2024-04-25T09:07:27.004278+0000","last_became_active":"2024-04-25T09:07:26.014764+0000","last_became_peered":"2024-04-25T09:07:26.014764+0000","last_unstale":"2024-04-25T09:07:27.004278+0000","last_undegraded":"2024-04-25T09:07:27.004278+0000","last_fullsized":"2024-04-25T09:07:27.004278+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:59.238982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.629076+0000","last_change":"2024-04-25T09:07:25.007837+0000","last_active":"2024-04-25T09:07:27.629076+0000","last_peered":"2024-04-25T09:07:27.629076+0000","last_clean":"2024-04-25T09:07:27.629076+0000","last_became_active":"2024-04-25T09:07:25.007537+0000","last_became_peered":"2024-04-25T09:07:25.007537+0000","last_unstale":"2024-04-25T09:07:27.629076+0000","last_undegraded":"2024-04-25T09:07:27.629076+0000","last_fullsized":"2024-04-25T09:07:27.629076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:27.497096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.096029+0000","last_change":"2024-04-25T09:07:26.005327+0000","last_active":"2024-04-25T09:07:27.096029+0000","last_peered":"2024-04-25T09:07:27.096029+0000","last_clean":"2024-04-25T09:07:27.096029+0000","last_became_active":"2024-04-25T09:07:26.004944+0000","last_became_peered":"2024-04-25T09:07:26.004944+0000","last_unstale":"2024-04-25T09:07:27.096029+0000","last_undegraded":"2024-04-25T09:07:27.096029+0000","last_fullsized":"2024-04-25T09:07:27.096029+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.619139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.627862+0000","last_change":"2024-04-25T09:07:25.002676+0000","last_active":"2024-04-25T09:07:27.627862+0000","last_peered":"2024-04-25T09:07:27.627862+0000","last_clean":"2024-04-25T09:07:27.627862+0000","last_became_active":"2024-04-25T09:07:25.001512+0000","last_became_peered":"2024-04-25T09:07:25.001512+0000","last_unstale":"2024-04-25T09:07:27.627862+0000","last_undegraded":"2024-04-25T09:07:27.627862+0000","last_fullsized":"2024-04-25T09:07:27.627862+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:29.835765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.013236+0000","last_change":"2024-04-25T09:07:26.015182+0000","last_active":"2024-04-25T09:07:27.013236+0000","last_peered":"2024-04-25T09:07:27.013236+0000","last_clean":"2024-04-25T09:07:27.013236+0000","last_became_active":"2024-04-25T09:07:26.014868+0000","last_became_peered":"2024-04-25T09:07:26.014868+0000","last_unstale":"2024-04-25T09:07:27.013236+0000","last_undegraded":"2024-04-25T09:07:27.013236+0000","last_fullsized":"2024-04-25T09:07:27.013236+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:24.961079+0000","last_clean_scrub_stamp":"2024-04-25T09:07:24.961079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:40.715423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.018440+0000","last_change":"2024-04-25T09:07:25.014785+0000","last_active":"2024-04-25T09:07:27.018440+0000","last_peered":"2024-04-25T09:07:27.018440+0000","last_clean":"2024-04-25T09:07:27.018440+0000","last_became_active":"2024-04-25T09:07:25.013321+0000","last_became_peered":"2024-04-25T09:07:25.013321+0000","last_unstale":"2024-04-25T09:07:27.018440+0000","last_undegraded":"2024-04-25T09:07:27.018440+0000","last_fullsized":"2024-04-25T09:07:27.018440+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:07:23.944971+0000","last_clean_scrub_stamp":"2024-04-25T09:07:23.944971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:47.161339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T09:07:27.004256+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:07:27.004256+0000","last_peered":"2024-04-25T09:07:27.004256+0000","last_clean":"2024-04-25T09:07:27.004256+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:07:27.004256+0000","last_undegraded":"2024-04-25T09:07:27.004256+0000","last_fullsized":"2024-04-25T09:07:27.004256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35896,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335944,"statfs":{"total":96636764160,"available":96600006656,"internally_reserved":0,"allocated":569344,"data_stored":234419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":573440,"data_stored":235001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40644,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331196,"statfs":{"total":96636764160,"available":96595144704,"internally_reserved":0,"allocated":1163264,"data_stored":824775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40080,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331760,"statfs":{"total":96636764160,"available":96595722240,"internally_reserved":0,"allocated":585728,"data_stored":236089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40072,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331768,"statfs":{"total":96636764160,"available":96595730432,"internally_reserved":0,"allocated":577536,"data_stored":235997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40064,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331776,"statfs":{"total":96636764160,"available":96595738624,"internally_reserved":0,"allocated":569344,"data_stored":234475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335364,"statfs":{"total":96636764160,"available":96599412736,"internally_reserved":0,"allocated":1163264,"data_stored":824823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":573440,"data_stored":235057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:07:36.023 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:07:36.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:07:36.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.299+0000 7f024be00640 1 Processor -- start 2024-04-25T09:07:36.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.299+0000 7f024be00640 1 -- start start 2024-04-25T09:07:36.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f024c15dd20 0x7f024c0d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0d34f0 0x7f024c0b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f024c0ce7e0 0x7f024c0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024c0b9a20 con 0x7f024c15dd20 2024-04-25T09:07:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024c15e100 con 0x7f024c0ce7e0 2024-04-25T09:07:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d2450 con 0x7f024c0d34f0 2024-04-25T09:07:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f024c15dd20 0x7f024c0d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f024c15dd20 0x7f024c0d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58106/0 (socket says 172.21.15.62:58106) 2024-04-25T09:07:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024aa00640 1 -- 172.21.15.62:0/1636347747 learned_addr learned my addr 172.21.15.62:0/1636347747 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f024b400640 1 --2- 172.21.15.62:0/1636347747 >> v2:172.21.15.121:3300/0 conn(0x7f024c0ce7e0 0x7f024c0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.301+0000 7f0243e00640 1 --2- 172.21.15.62:0/1636347747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0d34f0 0x7f024c0b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.302+0000 7f024aa00640 1 -- 172.21.15.62:0/1636347747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0d34f0 msgr2=0x7f024c0b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.302+0000 7f024aa00640 1 --2- 172.21.15.62:0/1636347747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0d34f0 0x7f024c0b94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.302+0000 7f024aa00640 1 -- 172.21.15.62:0/1636347747 >> v2:172.21.15.121:3300/0 conn(0x7f024c0ce7e0 msgr2=0x7f024c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.302+0000 7f024aa00640 1 --2- 172.21.15.62:0/1636347747 >> v2:172.21.15.121:3300/0 conn(0x7f024c0ce7e0 0x7f024c0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.302+0000 7f024aa00640 1 -- 172.21.15.62:0/1636347747 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0d2730 con 0x7f024c15dd20 2024-04-25T09:07:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.302+0000 7f0243e00640 1 --2- 172.21.15.62:0/1636347747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0d34f0 0x7f024c0b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:36.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.303+0000 7f024aa00640 1 --2- 172.21.15.62:0/1636347747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0d1d70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f024c0d1c90 tx=0x7f023c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9f27e1236d5c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:36.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.303+0000 7f024a000640 1 -- 172.21.15.62:0/1636347747 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c037070 con 0x7f024c15dd20 2024-04-25T09:07:36.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.303+0000 7f024a000640 1 -- 172.21.15.62:0/1636347747 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c032440 con 0x7f024c15dd20 2024-04-25T09:07:36.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.303+0000 7f024a000640 1 -- 172.21.15.62:0/1636347747 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c032810 con 0x7f024c15dd20 2024-04-25T09:07:36.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 -- 172.21.15.62:0/1636347747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 msgr2=0x7f024c0d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 --2- 172.21.15.62:0/1636347747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0d1d70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f024c0d1c90 tx=0x7f023c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 -- 172.21.15.62:0/1636347747 shutdown_connections 2024-04-25T09:07:36.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 --2- 172.21.15.62:0/1636347747 >> v2:172.21.15.121:3300/0 conn(0x7f024c0ce7e0 0x7f024c0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 --2- 172.21.15.62:0/1636347747 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0d34f0 0x7f024c0b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 --2- 172.21.15.62:0/1636347747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0d1d70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 -- 172.21.15.62:0/1636347747 >> 172.21.15.62:0/1636347747 conn(0x7f024c06b000 msgr2=0x7f024c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:36.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 -- 172.21.15.62:0/1636347747 shutdown_connections 2024-04-25T09:07:36.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.304+0000 7f024be00640 1 -- 172.21.15.62:0/1636347747 wait complete. 2024-04-25T09:07:36.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.306+0000 7f024be00640 1 Processor -- start 2024-04-25T09:07:36.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.306+0000 7f024be00640 1 -- start start 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.307+0000 7f024be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0ce7e0 0x7f024c158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.307+0000 7f024be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f024c0d34f0 0x7f024c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.307+0000 7f024be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.307+0000 7f024be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d39a0 con 0x7f024c15dd20 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.307+0000 7f024be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d2cf0 con 0x7f024c0d34f0 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.307+0000 7f024be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d3b20 con 0x7f024c0ce7e0 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.307+0000 7f024b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.308+0000 7f024b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58108/0 (socket says 172.21.15.62:58108) 2024-04-25T09:07:36.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.308+0000 7f024b400640 1 -- 172.21.15.62:0/2182950429 learned_addr learned my addr 172.21.15.62:0/2182950429 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:36.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.308+0000 7f024aa00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0ce7e0 0x7f024c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.308+0000 7f024b400640 1 -- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0ce7e0 msgr2=0x7f024c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.308+0000 7f024b400640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0ce7e0 0x7f024c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.308+0000 7f024b400640 1 -- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f024c0d34f0 msgr2=0x7f024c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.308+0000 7f024b400640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f024c0d34f0 0x7f024c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.308+0000 7f024b400640 1 -- 172.21.15.62:0/2182950429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0ccc40 con 0x7f024c15dd20 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.309+0000 7f024b400640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0cc400 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0244007e60 tx=0x7f024400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.309+0000 7f0242a00640 1 -- 172.21.15.62:0/2182950429 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244014620 con 0x7f024c15dd20 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.309+0000 7f0242a00640 1 -- 172.21.15.62:0/2182950429 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0244019070 con 0x7f024c15dd20 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.309+0000 7f0242a00640 1 -- 172.21.15.62:0/2182950429 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244014a20 con 0x7f024c15dd20 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.309+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c156560 con 0x7f024c15dd20 2024-04-25T09:07:36.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.309+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024c156ae0 con 0x7f024c15dd20 2024-04-25T09:07:36.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.311+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024c075070 con 0x7f024c15dd20 2024-04-25T09:07:36.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.312+0000 7f0242a00640 1 -- 172.21.15.62:0/2182950429 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0244014bc0 con 0x7f024c15dd20 2024-04-25T09:07:36.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.313+0000 7f0242a00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f022806b4d0 0x7f022806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.313+0000 7f0242a00640 1 -- 172.21.15.62:0/2182950429 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f024408aa30 con 0x7f024c15dd20 2024-04-25T09:07:36.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.313+0000 7f024aa00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f022806b4d0 0x7f022806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.315+0000 7f024aa00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f022806b4d0 0x7f022806d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f023c034000 tx=0x7f023c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:36.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.318+0000 7f0242a00640 1 -- 172.21.15.62:0/2182950429 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0244059940 con 0x7f024c15dd20 2024-04-25T09:07:36.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.586+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f024c15d290 con 0x7f024c15dd20 2024-04-25T09:07:36.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.591+0000 7f0242a00640 1 -- 172.21.15.62:0/2182950429 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0244059240 con 0x7f024c15dd20 2024-04-25T09:07:36.579 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:07:36.579 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:07:36.579 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:07:36.579 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:07:36.579 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:07:36.580 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:07:36.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.594+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f022806b4d0 msgr2=0x7f022806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.594+0000 7f024be00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f022806b4d0 0x7f022806d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f023c034000 tx=0x7f023c02e000 comp rx=0 tx=0).stop 2024-04-25T09:07:36.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.594+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 msgr2=0x7f024c0cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.594+0000 7f024be00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0cc400 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0244007e60 tx=0x7f024400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.595+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 shutdown_connections 2024-04-25T09:07:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.595+0000 7f024be00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f022806b4d0 0x7f022806d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.595+0000 7f024be00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f024c15dd20 0x7f024c0cc400 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.595+0000 7f024be00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f024c0d34f0 0x7f024c1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.595+0000 7f024be00640 1 --2- 172.21.15.62:0/2182950429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f024c0ce7e0 0x7f024c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.595+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 >> 172.21.15.62:0/2182950429 conn(0x7f024c06b000 msgr2=0x7f024c0cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.595+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 shutdown_connections 2024-04-25T09:07:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.595+0000 7f024be00640 1 -- 172.21.15.62:0/2182950429 wait complete. 2024-04-25T09:07:36.601 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:36.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.813+0000 7fc992a00640 1 Processor -- start 2024-04-25T09:07:36.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.813+0000 7fc992a00640 1 -- start start 2024-04-25T09:07:36.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.815+0000 7fc992a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc98c15f450 0x7fc98c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.815+0000 7fc992a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98c15faa0 con 0x7fc98c06d900 2024-04-25T09:07:36.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.815+0000 7fc992a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98c15fd80 con 0x7fc98c144c10 2024-04-25T09:07:36.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.815+0000 7fc992a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98c144ff0 con 0x7fc98c15f450 2024-04-25T09:07:36.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.815+0000 7fc990c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc98c06d900 0x7fc98c0b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39784/0 (socket says 172.21.15.62:39784) 2024-04-25T09:07:36.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.815+0000 7fc990c00640 1 -- 172.21.15.62:0/2709353272 learned_addr learned my addr 172.21.15.62:0/2709353272 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.815+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567216373 0 0) 0x7fc98c15faa0 con 0x7fc98c06d900 2024-04-25T09:07:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.815+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc974003520 con 0x7fc98c06d900 2024-04-25T09:07:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc992000640 1 --2- 172.21.15.62:0/2709353272 >> v2:172.21.15.121:3300/0 conn(0x7fc98c15f450 0x7fc98c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1852938701 0 0) 0x7fc974003520 con 0x7fc98c06d900 2024-04-25T09:07:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 >> v2:172.21.15.121:3300/0 conn(0x7fc98c15f450 msgr2=0x7fc98c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:36.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 --2- 172.21.15.62:0/2709353272 >> v2:172.21.15.121:3300/0 conn(0x7fc98c15f450 0x7fc98c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 >> v1:172.21.15.121:6789/0 conn(0x7fc98c144c10 legacy=0x7fc98c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c160060 con 0x7fc98c06d900 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc978001670 con 0x7fc98c06d900 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 >> v1:172.21.15.62:6789/0 conn(0x7fc98c06d900 legacy=0x7fc98c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9740039b0 0x7fc974003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc974004800 0x7fc974004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9740053c0 0x7fc97400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c144ff0 con 0x7fc9740053c0 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97400c260 con 0x7fc974004800 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97400c460 con 0x7fc9740039b0 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc992000640 1 --2- 172.21.15.62:0/2709353272 >> v2:172.21.15.121:3300/0 conn(0x7fc98c15f450 0x7fc98c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.816+0000 7fc992000640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9740053c0 0x7fc97400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.817+0000 7fc991600640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9740039b0 0x7fc974003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.817+0000 7fc990c00640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc974004800 0x7fc974004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.817+0000 7fc992000640 1 -- 172.21.15.62:0/2709353272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9740039b0 msgr2=0x7fc974003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.817+0000 7fc992000640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9740039b0 0x7fc974003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.817+0000 7fc992000640 1 -- 172.21.15.62:0/2709353272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc974004800 msgr2=0x7fc974004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.817+0000 7fc992000640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc974004800 0x7fc974004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.817+0000 7fc992000640 1 -- 172.21.15.62:0/2709353272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc974003520 con 0x7fc9740053c0 2024-04-25T09:07:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.818+0000 7fc992000640 1 -- 172.21.15.62:0/2709353272 reap_dead start 2024-04-25T09:07:36.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.818+0000 7fc992000640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9740053c0 0x7fc97400b9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc98400c410 tx=0x7fc974005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45274e2a9a5da551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.818+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984038070 con 0x7fc9740053c0 2024-04-25T09:07:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.818+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98402bb70 con 0x7fc9740053c0 2024-04-25T09:07:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.818+0000 7fc98be00640 1 -- 172.21.15.62:0/2709353272 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984007c00 con 0x7fc9740053c0 2024-04-25T09:07:36.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.819+0000 7fc992a00640 1 -- 172.21.15.62:0/2709353272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9740053c0 msgr2=0x7fc97400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.819+0000 7fc992a00640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9740053c0 0x7fc97400b9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc98400c410 tx=0x7fc974005830 comp rx=0 tx=0).stop 2024-04-25T09:07:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.819+0000 7fc992a00640 1 -- 172.21.15.62:0/2709353272 shutdown_connections 2024-04-25T09:07:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.819+0000 7fc992a00640 1 --2- 172.21.15.62:0/2709353272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9740053c0 0x7fc97400b9c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.819+0000 7fc992a00640 1 -- 172.21.15.62:0/2709353272 >> 172.21.15.62:0/2709353272 conn(0x7fc98c0b14d0 msgr2=0x7fc98c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.819+0000 7fc992a00640 1 -- 172.21.15.62:0/2709353272 shutdown_connections 2024-04-25T09:07:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.819+0000 7fc992a00640 1 -- 172.21.15.62:0/2709353272 wait complete. 2024-04-25T09:07:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.821+0000 7fc992a00640 1 Processor -- start 2024-04-25T09:07:36.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.821+0000 7fc992a00640 1 -- start start 2024-04-25T09:07:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc992a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9740053c0 0x7fc98c15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc992a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98c15e270 0x7fc98c1f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc992a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98c1306a0 0x7fc98c130b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc992a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0cf050 con 0x7fc98c15e270 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc992a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c15f880 con 0x7fc9740053c0 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc992a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0cf1d0 con 0x7fc98c1306a0 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc990c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98c15e270 0x7fc98c1f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc990c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98c15e270 0x7fc98c1f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58122/0 (socket says 172.21.15.62:58122) 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc990c00640 1 -- 172.21.15.62:0/3164691408 learned_addr learned my addr 172.21.15.62:0/3164691408 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.822+0000 7fc991600640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9740053c0 0x7fc98c15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc992000640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98c1306a0 0x7fc98c130b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc990c00640 1 -- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98c1306a0 msgr2=0x7fc98c130b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc990c00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98c1306a0 0x7fc98c130b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc990c00640 1 -- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9740053c0 msgr2=0x7fc98c15caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc990c00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9740053c0 0x7fc98c15caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc990c00640 1 -- 172.21.15.62:0/3164691408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c15d1d0 con 0x7fc98c15e270 2024-04-25T09:07:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc990c00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98c15e270 0x7fc98c1f48c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc97800e700 tx=0x7fc97800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc989600640 1 -- 172.21.15.62:0/3164691408 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800c6c0 con 0x7fc98c15e270 2024-04-25T09:07:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc989600640 1 -- 172.21.15.62:0/3164691408 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978018070 con 0x7fc98c15e270 2024-04-25T09:07:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc989600640 1 -- 172.21.15.62:0/3164691408 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800ca80 con 0x7fc98c15e270 2024-04-25T09:07:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.823+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98c15d4b0 con 0x7fc98c15e270 2024-04-25T09:07:36.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.824+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98c15da70 con 0x7fc98c15e270 2024-04-25T09:07:36.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.825+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98c06ea80 con 0x7fc98c15e270 2024-04-25T09:07:36.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.826+0000 7fc989600640 1 -- 172.21.15.62:0/3164691408 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc97800cc20 con 0x7fc98c15e270 2024-04-25T09:07:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.827+0000 7fc989600640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96c06b500 0x7fc96c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.827+0000 7fc989600640 1 -- 172.21.15.62:0/3164691408 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc978089f70 con 0x7fc98c15e270 2024-04-25T09:07:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.828+0000 7fc991600640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96c06b500 0x7fc96c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:36.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.829+0000 7fc991600640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96c06b500 0x7fc96c06d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc98c15e780 tx=0x7fc980009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:36.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:36.832+0000 7fc989600640 1 -- 172.21.15.62:0/3164691408 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc98c06ea80 con 0x7fc98c15e270 2024-04-25T09:07:37.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.072+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc98c04ef20 con 0x7fc98c15e270 2024-04-25T09:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.073+0000 7fc989600640 1 -- 172.21.15.62:0/3164691408 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fc978058de0 con 0x7fc98c15e270 2024-04-25T09:07:37.061 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:37.061 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:28.504417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:07:37.061 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-04-25T09:07:37.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96c06b500 msgr2=0x7fc96c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96c06b500 0x7fc96c06d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc98c15e780 tx=0x7fc980009040 comp rx=0 tx=0).stop 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98c15e270 msgr2=0x7fc98c1f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98c15e270 0x7fc98c1f48c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc97800e700 tx=0x7fc97800ebc0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 shutdown_connections 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96c06b500 0x7fc96c06d9c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc98c15e270 0x7fc98c1f48c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc98c1306a0 0x7fc98c130b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 --2- 172.21.15.62:0/3164691408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9740053c0 0x7fc98c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 >> 172.21.15.62:0/3164691408 conn(0x7fc98c0b14d0 msgr2=0x7fc98c070130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 shutdown_connections 2024-04-25T09:07:37.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.076+0000 7fc992a00640 1 -- 172.21.15.62:0/3164691408 wait complete. 2024-04-25T09:07:37.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:07:25.454273+0000', 'modified': '2024-04-25T09:07:28.504417+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5559}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5559': {'gid': 5559, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/752515489', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 752515489}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 752515489}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:07:37.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:07:37.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:07:37.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.721+0000 7f21d1800640 1 Processor -- start 2024-04-25T09:07:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.721+0000 7f21d1800640 1 -- start start 2024-04-25T09:07:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21d1800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21cc15bd10 0x7f21cc0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21d1800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f21cc0cccf0 0x7f21cc0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21d1800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc06ca70 con 0x7f21cc0aed00 2024-04-25T09:07:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21d1800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc06d660 con 0x7f21cc15bd10 2024-04-25T09:07:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21d1800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc0af140 con 0x7f21cc0cccf0 2024-04-25T09:07:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21caa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21cc15bd10 0x7f21cc0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21caa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21cc15bd10 0x7f21cc0cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58134/0 (socket says 172.21.15.62:58134) 2024-04-25T09:07:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21caa00640 1 -- 172.21.15.62:0/3710898012 learned_addr learned my addr 172.21.15.62:0/3710898012 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21cb400640 1 --1- 172.21.15.62:0/3710898012 >> v1:172.21.15.62:6789/0 conn(0x7f21cc0aed00 0x7f21cc15f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39796/0 (socket says 172.21.15.62:39796) 2024-04-25T09:07:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.723+0000 7f21ca000640 1 --2- 172.21.15.62:0/3710898012 >> v2:172.21.15.121:3300/0 conn(0x7f21cc0cccf0 0x7f21cc0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.724+0000 7f21caa00640 1 -- 172.21.15.62:0/3710898012 >> v2:172.21.15.121:3300/0 conn(0x7f21cc0cccf0 msgr2=0x7f21cc0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.724+0000 7f21caa00640 1 --2- 172.21.15.62:0/3710898012 >> v2:172.21.15.121:3300/0 conn(0x7f21cc0cccf0 0x7f21cc0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.724+0000 7f21caa00640 1 -- 172.21.15.62:0/3710898012 >> v1:172.21.15.62:6789/0 conn(0x7f21cc0aed00 legacy=0x7f21cc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.724+0000 7f21c9600640 1 -- 172.21.15.62:0/3710898012 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189077320 0 0) 0x7f21cc06ca70 con 0x7f21cc0aed00 2024-04-25T09:07:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.724+0000 7f21caa00640 1 -- 172.21.15.62:0/3710898012 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc15c0f0 con 0x7f21cc15bd10 2024-04-25T09:07:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.725+0000 7f21caa00640 1 --2- 172.21.15.62:0/3710898012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc15bd10 0x7f21cc0cfca0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f21cc06d7e0 tx=0x7f21bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc9fe2dd9776048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.725+0000 7f21c9600640 1 -- 172.21.15.62:0/3710898012 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc037070 con 0x7f21cc15bd10 2024-04-25T09:07:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.725+0000 7f21c9600640 1 -- 172.21.15.62:0/3710898012 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc032440 con 0x7f21cc15bd10 2024-04-25T09:07:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.725+0000 7f21c9600640 1 -- 172.21.15.62:0/3710898012 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc032810 con 0x7f21cc15bd10 2024-04-25T09:07:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.726+0000 7f21d1800640 1 -- 172.21.15.62:0/3710898012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc15bd10 msgr2=0x7f21cc0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.726+0000 7f21d1800640 1 --2- 172.21.15.62:0/3710898012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc15bd10 0x7f21cc0cfca0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f21cc06d7e0 tx=0x7f21bc02a270 comp rx=0 tx=0).stop 2024-04-25T09:07:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.726+0000 7f21d1800640 1 -- 172.21.15.62:0/3710898012 shutdown_connections 2024-04-25T09:07:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.726+0000 7f21d1800640 1 --2- 172.21.15.62:0/3710898012 >> v2:172.21.15.121:3300/0 conn(0x7f21cc0cccf0 0x7f21cc0ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.726+0000 7f21d1800640 1 --2- 172.21.15.62:0/3710898012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc15bd10 0x7f21cc0cfca0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.726+0000 7f21d1800640 1 -- 172.21.15.62:0/3710898012 >> 172.21.15.62:0/3710898012 conn(0x7f21cc0668f0 msgr2=0x7f21cc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.726+0000 7f21d1800640 1 -- 172.21.15.62:0/3710898012 shutdown_connections 2024-04-25T09:07:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.726+0000 7f21d1800640 1 -- 172.21.15.62:0/3710898012 wait complete. 2024-04-25T09:07:37.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.728+0000 7f21d1800640 1 Processor -- start 2024-04-25T09:07:37.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.728+0000 7f21d1800640 1 -- start start 2024-04-25T09:07:37.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21d1800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21cc0aed00 0x7f21cc1f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21d1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc0cccf0 0x7f21cc15ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21d1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21cc15bd10 0x7f21cc154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21d1800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc06d010 con 0x7f21cc0cccf0 2024-04-25T09:07:37.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21d1800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0ce6d0 con 0x7f21cc15bd10 2024-04-25T09:07:37.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21d1800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc06d2f0 con 0x7f21cc0aed00 2024-04-25T09:07:37.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc0cccf0 0x7f21cc15ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc0cccf0 0x7f21cc15ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58144/0 (socket says 172.21.15.62:58144) 2024-04-25T09:07:37.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.729+0000 7f21ca000640 1 -- 172.21.15.62:0/3556422872 learned_addr learned my addr 172.21.15.62:0/3556422872 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:37.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21ca000640 1 -- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21cc0aed00 msgr2=0x7f21cc1f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21ca000640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21cc0aed00 0x7f21cc1f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21ca000640 1 -- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21cc15bd10 msgr2=0x7f21cc154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21ca000640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21cc15bd10 0x7f21cc154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21ca000640 1 -- 172.21.15.62:0/3556422872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc15b180 con 0x7f21cc0cccf0 2024-04-25T09:07:37.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21ca000640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc0cccf0 0x7f21cc15ac40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d800 tx=0x7f21b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21c2a00640 1 -- 172.21.15.62:0/3556422872 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b800b6c0 con 0x7f21cc0cccf0 2024-04-25T09:07:37.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21c2a00640 1 -- 172.21.15.62:0/3556422872 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8019070 con 0x7f21cc0cccf0 2024-04-25T09:07:37.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21c2a00640 1 -- 172.21.15.62:0/3556422872 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b800ba80 con 0x7f21cc0cccf0 2024-04-25T09:07:37.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21d1800640 1 -- 172.21.15.62:0/3556422872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc1f40f0 con 0x7f21cc0cccf0 2024-04-25T09:07:37.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.730+0000 7f21d1800640 1 -- 172.21.15.62:0/3556422872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc1f46b0 con 0x7f21cc0cccf0 2024-04-25T09:07:37.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.733+0000 7f21c2a00640 1 -- 172.21.15.62:0/3556422872 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21b800bc20 con 0x7f21cc0cccf0 2024-04-25T09:07:37.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.734+0000 7f21c2a00640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21ac06b4d0 0x7f21ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.734+0000 7f21c2a00640 1 -- 172.21.15.62:0/3556422872 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21b808a750 con 0x7f21cc0cccf0 2024-04-25T09:07:37.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.734+0000 7f21caa00640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21ac06b4d0 0x7f21ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.734+0000 7f21d4632740 1 Processor -- start 2024-04-25T09:07:37.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.735+0000 7f21d4632740 1 -- start start 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.735+0000 7f21d4632740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d7521950 0x5578d7549790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.735+0000 7f21d4632740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d7549cd0 0x5578d7521d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.735+0000 7f21caa00640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21ac06b4d0 0x7f21ac06d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f21bc034000 tx=0x7f21bc02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.735+0000 7f21d4632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d756a5c0 0x5578d7520110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.735+0000 7f21d4632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d7485620 con 0x5578d756a5c0 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.735+0000 7f21d4632740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d742eca0 con 0x5578d7521950 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.735+0000 7f21d4632740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578d74816c0 con 0x5578d7549cd0 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d756a5c0 0x5578d7520110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d756a5c0 0x5578d7520110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58146/0 (socket says 172.21.15.62:58146) 2024-04-25T09:07:37.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 -- 172.21.15.62:0/156447246 learned_addr learned my addr 172.21.15.62:0/156447246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21ca000640 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d7549cd0 0x5578d7521d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21caa00640 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d7521950 0x5578d7549790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 -- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d7549cd0 msgr2=0x5578d7521d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d7549cd0 0x5578d7521d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 -- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d7521950 msgr2=0x5578d7549790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d7521950 0x5578d7549790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 -- 172.21.15.62:0/156447246 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578d7435f40 con 0x5578d756a5c0 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21caa00640 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d7521950 0x5578d7549790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.736+0000 7f21cb400640 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d756a5c0 0x5578d7520110 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f21c4001cd0 tx=0x7f21c4002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe76642c23ba096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.737+0000 7f21b7400640 1 -- 172.21.15.62:0/156447246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400c840 con 0x5578d756a5c0 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.737+0000 7f21b7400640 1 -- 172.21.15.62:0/156447246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c400ca00 con 0x5578d756a5c0 2024-04-25T09:07:37.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.737+0000 7f21b7400640 1 -- 172.21.15.62:0/156447246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400ce30 con 0x5578d756a5c0 2024-04-25T09:07:37.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.737+0000 7f21d4632740 1 -- 172.21.15.62:0/156447246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d756a5c0 msgr2=0x5578d7520110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.737+0000 7f21d4632740 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d756a5c0 0x5578d7520110 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f21c4001cd0 tx=0x7f21c4002190 comp rx=0 tx=0).stop 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.738+0000 7f21d4632740 1 -- 172.21.15.62:0/156447246 shutdown_connections 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.738+0000 7f21d4632740 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d756a5c0 0x5578d7520110 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.738+0000 7f21d4632740 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d7549cd0 0x5578d7521d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.738+0000 7f21d4632740 1 --2- 172.21.15.62:0/156447246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d7521950 0x5578d7549790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.738+0000 7f21d4632740 1 -- 172.21.15.62:0/156447246 >> 172.21.15.62:0/156447246 conn(0x5578d751eb70 msgr2=0x5578d7567550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.738+0000 7f21d4632740 1 -- 172.21.15.62:0/156447246 shutdown_connections 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.738+0000 7f21d4632740 1 -- 172.21.15.62:0/156447246 wait complete. 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.738+0000 7f21d4632740 1 -- start start 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.739+0000 7f21d4632740 1 Processor -- start 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21d4632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d7544f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21d4632740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d754dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21d4632740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d751b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21d4632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d7549300 con 0x5578d7549cd0 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21d4632740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d74241f0 con 0x5578d754e150 2024-04-25T09:07:37.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21d4632740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578d7548cf0 con 0x5578d756a5c0 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21d4632740 10 client.0 _ll_register_callbacks cb 0x5578d7441ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d7544f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d7544f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58156/0 (socket says 172.21.15.62:58156) 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21caa00640 1 -- 172.21.15.62:0/304687371 learned_addr learned my addr 172.21.15.62:0/304687371 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21ca000640 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d754dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21caa00640 1 -- 172.21.15.62:0/304687371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 msgr2=0x5578d754dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21caa00640 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d754dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21caa00640 1 -- 172.21.15.62:0/304687371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 msgr2=0x5578d751b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21caa00640 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d751b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.740+0000 7f21caa00640 1 -- 172.21.15.62:0/304687371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578d7443c50 con 0x5578d7549cd0 2024-04-25T09:07:37.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21caa00640 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d7544f80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f21bc035040 tx=0x7f21bc02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be2dfd64820ecc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21abe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21abe00640 1 -- 172.21.15.62:0/304687371 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc037040 con 0x5578d7549cd0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21abe00640 1 -- 172.21.15.62:0/304687371 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21bc03d640 con 0x5578d7549cd0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21d4632740 1 -- 172.21.15.62:0/304687371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21c400ca00 con 0x5578d7549cd0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21abe00640 1 -- 172.21.15.62:0/304687371 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21bc055050 con 0x5578d7549cd0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21d4632740 10 client.5721 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21d4632740 1 -- 172.21.15.62:0/304687371 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5578d719a2d0 con 0x5578d7549cd0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.741+0000 7f21abe00640 1 -- 172.21.15.62:0/304687371 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c400ca00 con 0x5578d7549cd0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.742+0000 7f21abe00640 1 -- 172.21.15.62:0/304687371 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21bc058020 con 0x5578d7549cd0 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.742+0000 7f21d4632740 10 client.5721 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.742+0000 7f21d4632740 10 client.5721 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.742+0000 7f21d4632740 20 client.5721 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.742+0000 7f21d4632740 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5578d7577b70 0x5578d757b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.742+0000 7f21d4632740 4 client.5721 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.742+0000 7f21d4632740 1 -- 172.21.15.62:0/304687371 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578d7492b60 con 0x5578d7577b70 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.742+0000 7f21ca000640 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5578d7577b70 0x5578d757b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.743+0000 7f21ca000640 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5578d7577b70 0x5578d757b0b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03a0b0d24719ab6 server_cookie=419c8d48138df8b6 in_seq=0 out_seq=0 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.743+0000 7f21abe00640 10 client.5721 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21abe00640 1 -- 172.21.15.62:0/304687371 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578d7492b60 con 0x5578d7577b70 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21abe00640 10 client.5721 handle_command_reply: tid=0 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 client.5721 shutdown 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 -- 172.21.15.62:0/304687371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 msgr2=0x5578d7544f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d7544f80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f21bc035040 tx=0x7f21bc02c070 comp rx=0 tx=0).stop 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 -- 172.21.15.62:0/304687371 shutdown_connections 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5578d7577b70 0x5578d757b0b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d751b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d754dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21abe00640 0 client.5721 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:37.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 --2- 172.21.15.62:0/304687371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d7544f80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 -- 172.21.15.62:0/304687371 >> 172.21.15.62:0/304687371 conn(0x5578d7521950 msgr2=0x5578d7520110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:37.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 -- 172.21.15.62:0/304687371 shutdown_connections 2024-04-25T09:07:37.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 1 -- 172.21.15.62:0/304687371 wait complete. 2024-04-25T09:07:37.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.744+0000 7f21d4632740 20 client.5721 trim_cache size 0 max 16384 2024-04-25T09:07:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.762+0000 7f21d4632740 1 Processor -- start 2024-04-25T09:07:37.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.762+0000 7f21d4632740 1 -- start start 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.763+0000 7f21d4632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d754f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.763+0000 7f21d4632740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d757a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.763+0000 7f21d4632740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d75810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.763+0000 7f21d4632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1f7c50 con 0x5578d7549cd0 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.763+0000 7f21d4632740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1f7050 con 0x5578d754e150 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.763+0000 7f21d4632740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1f7650 con 0x5578d756a5c0 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.764+0000 7f21caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d754f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.764+0000 7f21caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d754f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58168/0 (socket says 172.21.15.62:58168) 2024-04-25T09:07:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.764+0000 7f21caa00640 1 -- 172.21.15.62:0/1335739088 learned_addr learned my addr 172.21.15.62:0/1335739088 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.764+0000 7f21caa00640 1 -- 172.21.15.62:0/1335739088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 msgr2=0x5578d75810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.764+0000 7f21caa00640 1 --2- 172.21.15.62:0/1335739088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d75810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.764+0000 7f21caa00640 1 -- 172.21.15.62:0/1335739088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 msgr2=0x5578d757a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.764+0000 7f21caa00640 1 --2- 172.21.15.62:0/1335739088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d757a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.764+0000 7f21caa00640 1 -- 172.21.15.62:0/1335739088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578d72646b0 con 0x5578d7549cd0 2024-04-25T09:07:37.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.765+0000 7f21caa00640 1 --2- 172.21.15.62:0/1335739088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d754f200 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0090b0 tx=0x7f21bc03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa624ab8af647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.765+0000 7f21b6a00640 1 -- 172.21.15.62:0/1335739088 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc05e070 con 0x5578d7549cd0 2024-04-25T09:07:37.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.765+0000 7f21b6a00640 1 -- 172.21.15.62:0/1335739088 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc04d070 con 0x5578d7549cd0 2024-04-25T09:07:37.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.765+0000 7f21b6a00640 1 -- 172.21.15.62:0/1335739088 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc037070 con 0x5578d7549cd0 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 -- 172.21.15.62:0/1335739088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 msgr2=0x5578d754f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 --2- 172.21.15.62:0/1335739088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d754f200 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0090b0 tx=0x7f21bc03c000 comp rx=0 tx=0).stop 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 -- 172.21.15.62:0/1335739088 shutdown_connections 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 --2- 172.21.15.62:0/1335739088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d75810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 --2- 172.21.15.62:0/1335739088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d757a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 --2- 172.21.15.62:0/1335739088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d754f200 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 -- 172.21.15.62:0/1335739088 >> 172.21.15.62:0/1335739088 conn(0x5578d7521950 msgr2=0x5578d7541910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 -- 172.21.15.62:0/1335739088 shutdown_connections 2024-04-25T09:07:37.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.766+0000 7f21d4632740 1 -- 172.21.15.62:0/1335739088 wait complete. 2024-04-25T09:07:37.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.767+0000 7f21d4632740 1 -- start start 2024-04-25T09:07:37.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.767+0000 7f21d4632740 1 Processor -- start 2024-04-25T09:07:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21d4632740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d75d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21d4632740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d754f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21d4632740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d75810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21d4632740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d75779f0 con 0x5578d7549cd0 2024-04-25T09:07:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21d4632740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d754bfb0 con 0x5578d754e150 2024-04-25T09:07:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21d4632740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5578d754c800 con 0x5578d756a5c0 2024-04-25T09:07:37.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21d4632740 10 client.0 _ll_register_callbacks cb 0x7f21bc035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d75d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d75d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58176/0 (socket says 172.21.15.62:58176) 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.768+0000 7f21caa00640 1 -- 172.21.15.62:0/3007951621 learned_addr learned my addr 172.21.15.62:0/3007951621 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21caa00640 1 -- 172.21.15.62:0/3007951621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 msgr2=0x5578d75810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21caa00640 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d75810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21caa00640 1 -- 172.21.15.62:0/3007951621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 msgr2=0x5578d754f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21caa00640 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d754f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21caa00640 1 -- 172.21.15.62:0/3007951621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578d754d230 con 0x5578d7549cd0 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21caa00640 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d75d1530 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f21bc02a860 tx=0x7f21bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b2c2ebb56de2c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21abe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:37.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21abe00640 1 -- 172.21.15.62:0/3007951621 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc05e070 con 0x5578d7549cd0 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21abe00640 1 -- 172.21.15.62:0/3007951621 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21bc004230 con 0x5578d7549cd0 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.769+0000 7f21d4632740 1 -- 172.21.15.62:0/3007951621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21bc04d070 con 0x5578d7549cd0 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21abe00640 1 -- 172.21.15.62:0/3007951621 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21bc037040 con 0x5578d7549cd0 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21d4632740 10 client.5727 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21d4632740 1 -- 172.21.15.62:0/3007951621 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5578d75d2a60 con 0x5578d7549cd0 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21abe00640 1 -- 172.21.15.62:0/3007951621 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc04d070 con 0x5578d7549cd0 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21abe00640 1 -- 172.21.15.62:0/3007951621 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21bc06f020 con 0x5578d7549cd0 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21d4632740 10 client.5727 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21d4632740 10 client.5727 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21d4632740 20 client.5727 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21d4632740 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5578d7581880 0x5578d7521d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21d4632740 4 client.5727 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21d4632740 1 -- 172.21.15.62:0/3007951621 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5578d751f2d0 con 0x5578d7581880 2024-04-25T09:07:37.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.770+0000 7f21ca000640 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5578d7581880 0x5578d7521d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21ca000640 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5578d7581880 0x5578d7521d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb870e5c516649c9 server_cookie=3e1b0c1bcb049719 in_seq=0 out_seq=0 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21abe00640 10 client.5727 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21abe00640 1 -- 172.21.15.62:0/3007951621 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x5578d751f2d0 con 0x5578d7581880 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21abe00640 10 client.5727 handle_command_reply: tid=0 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 client.5727 shutdown 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 -- 172.21.15.62:0/3007951621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 msgr2=0x5578d75d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d75d1530 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f21bc02a860 tx=0x7f21bc004730 comp rx=0 tx=0).stop 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 -- 172.21.15.62:0/3007951621 shutdown_connections 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5578d7581880 0x5578d7521d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5578d756a5c0 0x5578d75810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578d754e150 0x5578d754f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 --2- 172.21.15.62:0/3007951621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5578d7549cd0 0x5578d75d1530 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21abe00640 0 client.5727 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:37.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.771+0000 7f21d4632740 1 -- 172.21.15.62:0/3007951621 >> 172.21.15.62:0/3007951621 conn(0x5578d7521950 msgr2=0x5578d754d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d4632740 1 -- 172.21.15.62:0/3007951621 shutdown_connections 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d4632740 1 -- 172.21.15.62:0/3007951621 wait complete. 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d4632740 20 client.5727 trim_cache size 0 max 16384 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 5559, 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:07:37.762 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:07:37.763 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 11.777272534, 2024-04-25T09:07:37.763 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 26, 2024-04-25T09:07:37.763 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 48, 2024-04-25T09:07:37.763 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 48, 2024-04-25T09:07:37.763 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 17.194782905 2024-04-25T09:07:37.763 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 -- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21ac06b4d0 msgr2=0x7f21ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21ac06b4d0 0x7f21ac06d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f21bc034000 tx=0x7f21bc02c000 comp rx=0 tx=0).stop 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 -- 172.21.15.62:0/3556422872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc0cccf0 msgr2=0x7f21cc15ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc0cccf0 0x7f21cc15ac40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f21b800d800 tx=0x7f21b800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 -- 172.21.15.62:0/3556422872 shutdown_connections 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f21ac06b4d0 0x7f21ac06d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21cc0cccf0 0x7f21cc15ac40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21cc15bd10 0x7f21cc154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 --2- 172.21.15.62:0/3556422872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21cc0aed00 0x7f21cc1f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.772+0000 7f21d1800640 1 -- 172.21.15.62:0/3556422872 >> 172.21.15.62:0/3556422872 conn(0x7f21cc0668f0 msgr2=0x7f21cc155120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:37.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.773+0000 7f21d1800640 1 -- 172.21.15.62:0/3556422872 shutdown_connections 2024-04-25T09:07:37.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.773+0000 7f21d1800640 1 -- 172.21.15.62:0/3556422872 wait complete. 2024-04-25T09:07:37.777 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:07:37.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.995+0000 7f0b1ac00640 1 Processor -- start 2024-04-25T09:07:37.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.996+0000 7f0b1ac00640 1 -- start start 2024-04-25T09:07:37.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.998+0000 7f0b1ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 0x7f0b140cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.998+0000 7f0b1ac00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0b140b6270 0x7f0b1406d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.998+0000 7f0b1ac00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b14145790 con 0x7f0b140d0200 2024-04-25T09:07:37.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.998+0000 7f0b1ac00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1415c110 con 0x7f0b140b6270 2024-04-25T09:07:37.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.998+0000 7f0b1ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1415c3b0 con 0x7f0b141453b0 2024-04-25T09:07:37.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.998+0000 7f0b19800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 0x7f0b140cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.999+0000 7f0b1a200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0b140b6270 0x7f0b1406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.999+0000 7f0b19800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 0x7f0b140cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:44640/0 (socket says 172.21.15.62:44640) 2024-04-25T09:07:37.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.999+0000 7f0b18e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0b140d0200 0x7f0b140b5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:40848/0 (socket says 172.21.15.62:40848) 2024-04-25T09:07:37.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.999+0000 7f0b19800640 1 -- 172.21.15.62:0/443742253 learned_addr learned my addr 172.21.15.62:0/443742253 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:37.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.999+0000 7f0b1a200640 1 -- 172.21.15.62:0/443742253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 msgr2=0x7f0b140cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:37.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.999+0000 7f0b1a200640 1 --2- 172.21.15.62:0/443742253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 0x7f0b140cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:37.999+0000 7f0b1a200640 1 -- 172.21.15.62:0/443742253 >> v1:172.21.15.121:6789/0 conn(0x7f0b140d0200 legacy=0x7f0b140b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.000+0000 7f0b1a200640 1 -- 172.21.15.62:0/443742253 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1415c690 con 0x7f0b140b6270 2024-04-25T09:07:37.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.001+0000 7f0b1a200640 1 --2- 172.21.15.62:0/443742253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b140b6270 0x7f0b1406d410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0097a0 tx=0x7f0b0c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5449da522845b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.001+0000 7f0b13e00640 1 -- 172.21.15.62:0/443742253 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c037070 con 0x7f0b140b6270 2024-04-25T09:07:37.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.001+0000 7f0b13e00640 1 -- 172.21.15.62:0/443742253 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c02ae50 con 0x7f0b140b6270 2024-04-25T09:07:37.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.001+0000 7f0b13e00640 1 -- 172.21.15.62:0/443742253 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c032660 con 0x7f0b140b6270 2024-04-25T09:07:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.002+0000 7f0b1ac00640 1 -- 172.21.15.62:0/443742253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b140b6270 msgr2=0x7f0b1406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.002+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/443742253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b140b6270 0x7f0b1406d410 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0097a0 tx=0x7f0b0c02a100 comp rx=0 tx=0).stop 2024-04-25T09:07:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.002+0000 7f0b1ac00640 1 -- 172.21.15.62:0/443742253 shutdown_connections 2024-04-25T09:07:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.002+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/443742253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b140b6270 0x7f0b1406d410 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.002+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/443742253 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 0x7f0b140cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.002+0000 7f0b1ac00640 1 -- 172.21.15.62:0/443742253 >> 172.21.15.62:0/443742253 conn(0x7f0b1406abe0 msgr2=0x7f0b1406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.002+0000 7f0b1ac00640 1 -- 172.21.15.62:0/443742253 shutdown_connections 2024-04-25T09:07:37.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.002+0000 7f0b1ac00640 1 -- 172.21.15.62:0/443742253 wait complete. 2024-04-25T09:07:37.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.004+0000 7f0b1ac00640 1 Processor -- start 2024-04-25T09:07:37.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.005+0000 7f0b1ac00640 1 -- start start 2024-04-25T09:07:37.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b1ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b140b6270 0x7f0b140c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b1ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b140d0200 0x7f0b1413d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b1ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 0x7f0b1413dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b1ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14072130 con 0x7f0b140b6270 2024-04-25T09:07:37.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b1ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1415cc50 con 0x7f0b140d0200 2024-04-25T09:07:37.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b1ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1406d950 con 0x7f0b141453b0 2024-04-25T09:07:37.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b19800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b140b6270 0x7f0b140c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:37.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b19800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b140b6270 0x7f0b140c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:58178/0 (socket says 172.21.15.62:58178) 2024-04-25T09:07:37.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.006+0000 7f0b19800640 1 -- 172.21.15.62:0/2851617285 learned_addr learned my addr 172.21.15.62:0/2851617285 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:37.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b19800640 1 -- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 msgr2=0x7f0b1413dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b19800640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 0x7f0b1413dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b19800640 1 -- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b140d0200 msgr2=0x7f0b1413d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:37.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b19800640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b140d0200 0x7f0b1413d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:37.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b19800640 1 -- 172.21.15.62:0/2851617285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b141f2970 con 0x7f0b140b6270 2024-04-25T09:07:37.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b19800640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b140b6270 0x7f0b140c79a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009cf0 tx=0x7f0b0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:37.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b11600640 1 -- 172.21.15.62:0/2851617285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800cda0 con 0x7f0b140b6270 2024-04-25T09:07:37.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b11600640 1 -- 172.21.15.62:0/2851617285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b08019070 con 0x7f0b140b6270 2024-04-25T09:07:37.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b11600640 1 -- 172.21.15.62:0/2851617285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080146a0 con 0x7f0b140b6270 2024-04-25T09:07:37.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b141f2690 con 0x7f0b140b6270 2024-04-25T09:07:37.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.007+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b140c71a0 con 0x7f0b140b6270 2024-04-25T09:07:37.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.009+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1404ef20 con 0x7f0b140b6270 2024-04-25T09:07:37.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.010+0000 7f0b11600640 1 -- 172.21.15.62:0/2851617285 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b08005d10 con 0x7f0b140b6270 2024-04-25T09:07:37.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.011+0000 7f0b11600640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0af406b4d0 0x7f0af406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:37.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.011+0000 7f0b11600640 1 -- 172.21.15.62:0/2851617285 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b0808a6f0 con 0x7f0b140b6270 2024-04-25T09:07:37.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.012+0000 7f0b18e00640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0af406b4d0 0x7f0af406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:38.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.013+0000 7f0b18e00640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0af406b4d0 0x7f0af406d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b00000f60 tx=0x7f0b00007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:38.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.016+0000 7f0b11600640 1 -- 172.21.15.62:0/2851617285 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b08010040 con 0x7f0b140b6270 2024-04-25T09:07:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.256+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0b140b5d00 con 0x7f0b140b6270 2024-04-25T09:07:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.257+0000 7f0b11600640 1 -- 172.21.15.62:0/2851617285 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0b08059570 con 0x7f0b140b6270 2024-04-25T09:07:38.244 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:07:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0af406b4d0 msgr2=0x7f0af406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0af406b4d0 0x7f0af406d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b00000f60 tx=0x7f0b00007a00 comp rx=0 tx=0).stop 2024-04-25T09:07:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b140b6270 msgr2=0x7f0b140c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b140b6270 0x7f0b140c79a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009cf0 tx=0x7f0b0800c490 comp rx=0 tx=0).stop 2024-04-25T09:07:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 shutdown_connections 2024-04-25T09:07:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0af406b4d0 0x7f0af406d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:38.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b141453b0 0x7f0b1413dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b140d0200 0x7f0b1413d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.259+0000 7f0b1ac00640 1 --2- 172.21.15.62:0/2851617285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b140b6270 0x7f0b140c79a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.260+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 >> 172.21.15.62:0/2851617285 conn(0x7f0b1406abe0 msgr2=0x7f0b140c5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.260+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 shutdown_connections 2024-04-25T09:07:38.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:38.260+0000 7f0b1ac00640 1 -- 172.21.15.62:0/2851617285 wait complete. 2024-04-25T09:07:38.267 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:07:38.267 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:07:38.267 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:07:38.267 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:07:38.267 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:07:38.268 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:07:38.268 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:07:38.268 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:07:38.268 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:07:38.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:38.268 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:07:38.320 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42401sec preferred_lft 42401sec 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:07:38.321 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:07:38.322 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:07:38.322 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:07:38.322 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:07:38.322 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:07:38.322 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:07:38.322 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:07:38.322 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:07:38.322 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:07:38.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:38.323 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:07:38.405 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:07:38.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:38.409 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:07:38.463 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:07:38.463 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:07:38.463 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:07:38.463 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:07:38.463 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:07:38.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:38.464 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:07:38.464 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:07:38.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:07:38.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:07:38.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:07:38.464 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:07:38.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:38.623 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:07:38.676 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:07:38.676 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:07:38.677 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:07:38.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:38.678 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:07:38.764 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:07:38.764 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:07:38.822 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:07:38.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:07:38.823 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:07:38.877 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:07:38.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:38.877 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:07:39.012 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:07:39.012 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:07:39.047 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:07:39.047 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:07:39.047 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:07:39.047 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:07:39.047 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:07:39.049 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:07:39.050 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:07:39.050 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:07:39.050 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:07:39.050 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:07:39.055 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:07:39.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:39.056 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:07:39.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:39.136 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:07:39.205 INFO:teuthology.orchestra.run.smithi062.stdout:client5733 2024-04-25T09:07:39.206 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:07:39.206 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:39.259 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:07:39.260 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T09:07:39.260 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T09:07:39.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:39.260 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> stdin-killer -- python3 -c ' 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> import stat 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> import json 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> import sys 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> try: 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> except OSError as e: 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> sys.exit(e.errno) 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:07:39.325 DEBUG:teuthology.orchestra.run.smithi062:> print(json.dumps( 2024-04-25T09:07:39.326 DEBUG:teuthology.orchestra.run.smithi062:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:07:39.326 DEBUG:teuthology.orchestra.run.smithi062:> indent=2)) 2024-04-25T09:07:39.326 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-04-25T09:07:39.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:39 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T09:07:39.489 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T09:07:39.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:39.490 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T09:07:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T09:07:39.914 INFO:teuthology.orchestra.run.smithi062.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T09:07:40.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:40.659 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:07:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:Updating files: 19% (269/1398) Updating files: 20% (280/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 38% (537/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 46% (644/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 54% (759/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 64% (905/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 78% (1103/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 93% (1305/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T09:07:47.807 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T09:07:47.807 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T09:07:47.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:48.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.031+0000 7f5b03e00640 1 Processor -- start 2024-04-25T09:07:48.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.031+0000 7f5b03e00640 1 -- start start 2024-04-25T09:07:48.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.032+0000 7f5b03e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b04144c10 0x7f5b0415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.032+0000 7f5b03e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b040cee40 0x7f5b0406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.033+0000 7f5b03e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0415faa0 con 0x7f5b0406d730 2024-04-25T09:07:48.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.033+0000 7f5b03e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b04144ff0 con 0x7f5b040cee40 2024-04-25T09:07:48.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.033+0000 7f5b03e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b04160340 con 0x7f5b04144c10 2024-04-25T09:07:48.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.033+0000 7f5b03400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5b0406d730 0x7f5b040b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53588/0 (socket says 172.21.15.62:53588) 2024-04-25T09:07:48.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.033+0000 7f5b03400640 1 -- 172.21.15.62:0/1981110688 learned_addr learned my addr 172.21.15.62:0/1981110688 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:48.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.033+0000 7f5b02a00640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b04144c10 0x7f5b0415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.033+0000 7f5b02000640 1 --2- 172.21.15.62:0/1981110688 >> v2:172.21.15.121:3300/0 conn(0x7f5b040cee40 0x7f5b0406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297274038 0 0) 0x7f5b0415faa0 con 0x7f5b0406d730 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae8003520 con 0x7f5b0406d730 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3850625901 0 0) 0x7f5ae8003520 con 0x7f5b0406d730 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b04144c10 msgr2=0x7f5b0415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b04144c10 0x7f5b0415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 >> v2:172.21.15.121:3300/0 conn(0x7f5b040cee40 msgr2=0x7f5b0406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 --2- 172.21.15.62:0/1981110688 >> v2:172.21.15.121:3300/0 conn(0x7f5b040cee40 0x7f5b0406d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0415fd80 con 0x7f5b0406d730 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f5afc0019f0 con 0x7f5b0406d730 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 >> v1:172.21.15.62:6789/0 conn(0x7f5b0406d730 legacy=0x7f5b040b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b04144c10 0x7f5ae80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5ae8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ae8004d40 0x7f5ae80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b04144ff0 con 0x7f5ae8003f30 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b04160340 con 0x7f5ae8004d40 2024-04-25T09:07:48.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae800ba10 con 0x7f5b04144c10 2024-04-25T09:07:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.034+0000 7f5b02000640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5ae8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.035+0000 7f5b02a00640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b04144c10 0x7f5ae80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.035+0000 7f5b02000640 1 -- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b04144c10 msgr2=0x7f5ae80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.035+0000 7f5b02000640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b04144c10 0x7f5ae80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.035+0000 7f5b02000640 1 -- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ae8004d40 msgr2=0x7f5ae80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.035+0000 7f5b02000640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ae8004d40 0x7f5ae80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.035+0000 7f5b02000640 1 -- 172.21.15.62:0/1981110688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8003520 con 0x7f5ae8003f30 2024-04-25T09:07:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.036+0000 7f5b02000640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5ae8004800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009af0 tx=0x7f5ae8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd01217eb6b5b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.036+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0036070 con 0x7f5ae8003f30 2024-04-25T09:07:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.036+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0031400 con 0x7f5ae8003f30 2024-04-25T09:07:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.036+0000 7f5b01600640 1 -- 172.21.15.62:0/1981110688 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0009690 con 0x7f5ae8003f30 2024-04-25T09:07:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.037+0000 7f5b03e00640 1 -- 172.21.15.62:0/1981110688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 msgr2=0x7f5ae8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.037+0000 7f5b03e00640 1 --2- 172.21.15.62:0/1981110688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5ae8004800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009af0 tx=0x7f5ae8005830 comp rx=0 tx=0).stop 2024-04-25T09:07:48.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.037+0000 7f5b03400640 1 -- 172.21.15.62:0/1981110688 reap_dead start 2024-04-25T09:07:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.037+0000 7f5b03e00640 1 -- 172.21.15.62:0/1981110688 shutdown_connections 2024-04-25T09:07:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.037+0000 7f5b03e00640 1 -- 172.21.15.62:0/1981110688 >> 172.21.15.62:0/1981110688 conn(0x7f5b040b14d0 msgr2=0x7f5b040b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.037+0000 7f5b03e00640 1 -- 172.21.15.62:0/1981110688 shutdown_connections 2024-04-25T09:07:48.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.037+0000 7f5b03e00640 1 -- 172.21.15.62:0/1981110688 wait complete. 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.039+0000 7f5b03e00640 1 Processor -- start 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.039+0000 7f5b03e00640 1 -- start start 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.040+0000 7f5b03e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5b0415b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.040+0000 7f5b03e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b0413f8d0 0x7f5b0413fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.040+0000 7f5b03e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b0415bfb0 0x7f5b0415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.040+0000 7f5b03e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040cf2a0 con 0x7f5ae8003f30 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.040+0000 7f5b03e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040b41c0 con 0x7f5b0413f8d0 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.040+0000 7f5b03e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040b5660 con 0x7f5b0415bfb0 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.040+0000 7f5b02a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5b0415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5b0415b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53498/0 (socket says 172.21.15.62:53498) 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02a00640 1 -- 172.21.15.62:0/427463725 learned_addr learned my addr 172.21.15.62:0/427463725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02000640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b0413f8d0 0x7f5b0413fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b03400640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b0415bfb0 0x7f5b0415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02a00640 1 -- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b0415bfb0 msgr2=0x7f5b0415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02a00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b0415bfb0 0x7f5b0415c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02a00640 1 -- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b0413f8d0 msgr2=0x7f5b0413fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02a00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b0413f8d0 0x7f5b0413fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02a00640 1 -- 172.21.15.62:0/427463725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b041ef9b0 con 0x7f5ae8003f30 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b03400640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b0415bfb0 0x7f5b0415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.041+0000 7f5b02000640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b0413f8d0 0x7f5b0413fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:48.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.042+0000 7f5b02a00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5b0415b450 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5af4010890 tx=0x7f5af4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.042+0000 7f5afaa00640 1 -- 172.21.15.62:0/427463725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af400f040 con 0x7f5ae8003f30 2024-04-25T09:07:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.042+0000 7f5afaa00640 1 -- 172.21.15.62:0/427463725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af400eb60 con 0x7f5ae8003f30 2024-04-25T09:07:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.042+0000 7f5afaa00640 1 -- 172.21.15.62:0/427463725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af401c400 con 0x7f5ae8003f30 2024-04-25T09:07:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.042+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0415cd10 con 0x7f5ae8003f30 2024-04-25T09:07:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.042+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b04134d30 con 0x7f5ae8003f30 2024-04-25T09:07:48.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.043+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0404ef20 con 0x7f5ae8003f30 2024-04-25T09:07:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.044+0000 7f5afaa00640 1 -- 172.21.15.62:0/427463725 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5af401c5a0 con 0x7f5ae8003f30 2024-04-25T09:07:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.045+0000 7f5afaa00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5ae406b4d0 0x7f5ae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.045+0000 7f5afaa00640 1 -- 172.21.15.62:0/427463725 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5af408a780 con 0x7f5ae8003f30 2024-04-25T09:07:48.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.046+0000 7f5b02000640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5ae406b4d0 0x7f5ae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.047+0000 7f5b02000640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5ae406b4d0 0x7f5ae406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5af0033000 tx=0x7f5af0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.050+0000 7f5afaa00640 1 -- 172.21.15.62:0/427463725 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5af4018030 con 0x7f5ae8003f30 2024-04-25T09:07:48.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.290+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b0415e150 con 0x7f5ae8003f30 2024-04-25T09:07:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.291+0000 7f5afaa00640 1 -- 172.21.15.62:0/427463725 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f5af4058fa0 con 0x7f5ae8003f30 2024-04-25T09:07:48.279 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:48.279 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:28.504417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:07:48.279 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5ae406b4d0 msgr2=0x7f5ae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5ae406b4d0 0x7f5ae406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5af0033000 tx=0x7f5af0008030 comp rx=0 tx=0).stop 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 msgr2=0x7f5b0415b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5b0415b450 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5af4010890 tx=0x7f5af4010d50 comp rx=0 tx=0).stop 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 shutdown_connections 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5ae406b4d0 0x7f5ae406d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b0415bfb0 0x7f5b0415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b0413f8d0 0x7f5b0413fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 --2- 172.21.15.62:0/427463725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ae8003f30 0x7f5b0415b450 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 >> 172.21.15.62:0/427463725 conn(0x7f5b040b14d0 msgr2=0x7f5b040b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 shutdown_connections 2024-04-25T09:07:48.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.294+0000 7f5b03e00640 1 -- 172.21.15.62:0/427463725 wait complete. 2024-04-25T09:07:48.301 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T09:07:48.492 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:48.492 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": -2 2024-04-25T09:07:48.492 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:48.507 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T09:07:48.507 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T09:07:48.507 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:48.507 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T09:07:48.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.732+0000 7f1083200640 1 Processor -- start 2024-04-25T09:07:48.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.732+0000 7f1083200640 1 -- start start 2024-04-25T09:07:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.734+0000 7f1083200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f10840ccde0 0x7f10840ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.734+0000 7f1083200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10840b31a0 0x7f10840b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.734+0000 7f1083200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10840ced40 con 0x7f108406d2c0 2024-04-25T09:07:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.734+0000 7f1083200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10840cd260 con 0x7f10840ccde0 2024-04-25T09:07:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.734+0000 7f1083200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10840cf000 con 0x7f10840b31a0 2024-04-25T09:07:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1081400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f10840ccde0 0x7f10840ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1081400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f10840ccde0 0x7f10840ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53502/0 (socket says 172.21.15.62:53502) 2024-04-25T09:07:48.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1081400640 1 -- 172.21.15.62:0/1510508596 learned_addr learned my addr 172.21.15.62:0/1510508596 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:48.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1081400640 1 -- 172.21.15.62:0/1510508596 >> v2:172.21.15.121:3300/0 conn(0x7f10840b31a0 msgr2=0x7f10840b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1082800640 1 --2- 172.21.15.62:0/1510508596 >> v2:172.21.15.121:3300/0 conn(0x7f10840b31a0 0x7f10840b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1081400640 1 --2- 172.21.15.62:0/1510508596 >> v2:172.21.15.121:3300/0 conn(0x7f10840b31a0 0x7f10840b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1081400640 1 -- 172.21.15.62:0/1510508596 >> v1:172.21.15.121:6789/0 conn(0x7f108406d2c0 legacy=0x7f108406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1081400640 1 -- 172.21.15.62:0/1510508596 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840cf2e0 con 0x7f10840ccde0 2024-04-25T09:07:48.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.735+0000 7f1082800640 1 --2- 172.21.15.62:0/1510508596 >> v2:172.21.15.121:3300/0 conn(0x7f10840b31a0 0x7f10840b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:48.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.736+0000 7f1081400640 1 --2- 172.21.15.62:0/1510508596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f10840ce800 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1084145620 tx=0x7f107002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7441b5573b0041a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.737+0000 7f1080a00640 1 -- 172.21.15.62:0/1510508596 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070037070 con 0x7f10840ccde0 2024-04-25T09:07:48.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.737+0000 7f1080a00640 1 -- 172.21.15.62:0/1510508596 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1070032440 con 0x7f10840ccde0 2024-04-25T09:07:48.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.737+0000 7f1080a00640 1 -- 172.21.15.62:0/1510508596 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070032810 con 0x7f10840ccde0 2024-04-25T09:07:48.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.737+0000 7f1083200640 1 -- 172.21.15.62:0/1510508596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 msgr2=0x7f10840ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.737+0000 7f1083200640 1 --2- 172.21.15.62:0/1510508596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f10840ce800 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1084145620 tx=0x7f107002a230 comp rx=0 tx=0).stop 2024-04-25T09:07:48.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.738+0000 7f1083200640 1 -- 172.21.15.62:0/1510508596 shutdown_connections 2024-04-25T09:07:48.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.738+0000 7f1083200640 1 --2- 172.21.15.62:0/1510508596 >> v2:172.21.15.121:3300/0 conn(0x7f10840b31a0 0x7f10840b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.738+0000 7f1083200640 1 --2- 172.21.15.62:0/1510508596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f10840ce800 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.738+0000 7f1083200640 1 -- 172.21.15.62:0/1510508596 >> 172.21.15.62:0/1510508596 conn(0x7f1084066f20 msgr2=0x7f10840b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:48.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.738+0000 7f1083200640 1 -- 172.21.15.62:0/1510508596 shutdown_connections 2024-04-25T09:07:48.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.738+0000 7f1083200640 1 -- 172.21.15.62:0/1510508596 wait complete. 2024-04-25T09:07:48.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.739+0000 7f1083200640 1 Processor -- start 2024-04-25T09:07:48.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.740+0000 7f1083200640 1 -- start start 2024-04-25T09:07:48.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.740+0000 7f1083200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108406d2c0 0x7f108415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1083200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10840b31a0 0x7f108406c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1083200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f108406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1083200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b2c80 con 0x7f10840ccde0 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1083200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b29a0 con 0x7f108406d2c0 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1083200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10841448d0 con 0x7f10840b31a0 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1082800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f108406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1082800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f108406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53518/0 (socket says 172.21.15.62:53518) 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1082800640 1 -- 172.21.15.62:0/2200886090 learned_addr learned my addr 172.21.15.62:0/2200886090 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1081400640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10840b31a0 0x7f108406c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1081e00640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108406d2c0 0x7f108415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1082800640 1 -- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10840b31a0 msgr2=0x7f108406c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1082800640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10840b31a0 0x7f108406c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1082800640 1 -- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108406d2c0 msgr2=0x7f108415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1082800640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108406d2c0 0x7f108415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1082800640 1 -- 172.21.15.62:0/2200886090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108413d840 con 0x7f10840ccde0 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.741+0000 7f1081400640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10840b31a0 0x7f108406c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.742+0000 7f1082800640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f108406c7c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f107c00cf20 tx=0x7f107c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.742+0000 7f107a000640 1 -- 172.21.15.62:0/2200886090 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c01a070 con 0x7f10840ccde0 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.742+0000 7f107a000640 1 -- 172.21.15.62:0/2200886090 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c00d840 con 0x7f10840ccde0 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.742+0000 7f107a000640 1 -- 172.21.15.62:0/2200886090 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c00dc30 con 0x7f10840ccde0 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.742+0000 7f1083200640 1 -- 172.21.15.62:0/2200886090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108413db20 con 0x7f10840ccde0 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.742+0000 7f1083200640 1 -- 172.21.15.62:0/2200886090 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108413e0e0 con 0x7f10840ccde0 2024-04-25T09:07:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.744+0000 7f107a000640 1 -- 172.21.15.62:0/2200886090 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f107c01d070 con 0x7f10840ccde0 2024-04-25T09:07:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.745+0000 7f107a000640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106406b500 0x7f106406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.746+0000 7f107a000640 1 -- 172.21.15.62:0/2200886090 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f107c08ab50 con 0x7f10840ccde0 2024-04-25T09:07:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.746+0000 7f1081e00640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106406b500 0x7f106406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.746+0000 7f108b766740 1 Processor -- start 2024-04-25T09:07:48.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.746+0000 7f108b766740 1 -- start start 2024-04-25T09:07:48.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.747+0000 7f1081e00640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106406b500 0x7f106406d9c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1074004ea0 tx=0x7f1074009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.747+0000 7f108b766740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 0x55acff1c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.747+0000 7f108b766740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff1c4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.747+0000 7f108b766740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1eec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.747+0000 7f108b766740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff126b90 con 0x55acff1ee7e0 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.747+0000 7f108b766740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff0d00b0 con 0x55acff1c3d20 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.747+0000 7f108b766740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acff122c30 con 0x55acff2003f0 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1eec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53530/0 (socket says 172.21.15.62:53530) 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 -- 172.21.15.62:0/1467157828 learned_addr learned my addr 172.21.15.62:0/1467157828 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1081400640 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff1c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1081e00640 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 0x55acff1c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 -- 172.21.15.62:0/1467157828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 msgr2=0x55acff1c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff1c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 -- 172.21.15.62:0/1467157828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 msgr2=0x55acff1c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 0x55acff1c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 -- 172.21.15.62:0/1467157828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acfef079f0 con 0x55acff1ee7e0 2024-04-25T09:07:48.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.748+0000 7f1082800640 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1eec60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f107c00cfd0 tx=0x7f107c0599f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97021dfdbc32c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f106e600640 1 -- 172.21.15.62:0/1467157828 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c05d170 con 0x55acff1ee7e0 2024-04-25T09:07:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f106e600640 1 -- 172.21.15.62:0/1467157828 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c061940 con 0x55acff1ee7e0 2024-04-25T09:07:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f106e600640 1 -- 172.21.15.62:0/1467157828 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c057530 con 0x55acff1ee7e0 2024-04-25T09:07:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f108b766740 1 -- 172.21.15.62:0/1467157828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 msgr2=0x55acff1eec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f108b766740 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1eec60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f107c00cfd0 tx=0x7f107c0599f0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f108b766740 1 -- 172.21.15.62:0/1467157828 shutdown_connections 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f108b766740 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1eec60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f108b766740 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff1c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f108b766740 1 --2- 172.21.15.62:0/1467157828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 0x55acff1c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.749+0000 7f108b766740 1 -- 172.21.15.62:0/1467157828 >> 172.21.15.62:0/1467157828 conn(0x55acff18bfb0 msgr2=0x55acff2097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.750+0000 7f108b766740 1 -- 172.21.15.62:0/1467157828 shutdown_connections 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.750+0000 7f108b766740 1 -- 172.21.15.62:0/1467157828 wait complete. 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.751+0000 7f108b766740 1 -- start start 2024-04-25T09:07:48.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.751+0000 7f108b766740 1 Processor -- start 2024-04-25T09:07:48.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f108b766740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 0x55acff1f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f108b766740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f108b766740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff1e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f108b766740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff0c5570 con 0x55acff1ee7e0 2024-04-25T09:07:48.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f108b766740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff1ec330 con 0x55acff1c3d20 2024-04-25T09:07:48.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f108b766740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acff21b190 con 0x55acff2003f0 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f108b766740 10 client.0 _ll_register_callbacks cb 0x55acfee3add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f1081400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f1081400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1f0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53536/0 (socket says 172.21.15.62:53536) 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f1081400640 1 -- 172.21.15.62:0/2201700313 learned_addr learned my addr 172.21.15.62:0/2201700313 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f1081e00640 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 0x55acff1f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.753+0000 7f1082800640 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff1e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.754+0000 7f1081400640 1 -- 172.21.15.62:0/2201700313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 msgr2=0x55acff1e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.754+0000 7f1081400640 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff1e82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.754+0000 7f1081400640 1 -- 172.21.15.62:0/2201700313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 msgr2=0x55acff1f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.754+0000 7f1081400640 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 0x55acff1f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.754+0000 7f1081400640 1 -- 172.21.15.62:0/2201700313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acfedfb2e0 con 0x55acff1ee7e0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.754+0000 7f1081400640 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1f0d10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f107002a820 tx=0x7f1070004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62410175aa8b397e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.754+0000 7f1062a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f1062a00640 1 -- 172.21.15.62:0/2201700313 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070037040 con 0x55acff1ee7e0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f1062a00640 1 -- 172.21.15.62:0/2201700313 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f107003c640 con 0x55acff1ee7e0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f108b766740 1 -- 172.21.15.62:0/2201700313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f107c061940 con 0x55acff1ee7e0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f1062a00640 1 -- 172.21.15.62:0/2201700313 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f107004d050 con 0x55acff1ee7e0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f108b766740 10 client.5751 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f108b766740 1 -- 172.21.15.62:0/2201700313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55acfef68180 con 0x55acff1ee7e0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f1062a00640 1 -- 172.21.15.62:0/2201700313 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c061940 con 0x55acff1ee7e0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f1062a00640 1 -- 172.21.15.62:0/2201700313 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1070050020 con 0x55acff1ee7e0 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f108b766740 10 client.5751 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f108b766740 10 client.5751 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f108b766740 20 client.5751 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:48.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.755+0000 7f108b766740 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55acff200a00 0x55acff200e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.756+0000 7f108b766740 4 client.5751 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:48.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.756+0000 7f108b766740 1 -- 172.21.15.62:0/2201700313 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acff1340d0 con 0x55acff200a00 2024-04-25T09:07:48.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.756+0000 7f1081e00640 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55acff200a00 0x55acff200e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.756+0000 7f1081e00640 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55acff200a00 0x55acff200e00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=809b0197fea18d66 server_cookie=d64d9f4baaf75c48 in_seq=0 out_seq=0 2024-04-25T09:07:48.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.756+0000 7f1062a00640 10 client.5751 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f1062a00640 1 -- 172.21.15.62:0/2201700313 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acff1340d0 con 0x55acff200a00 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f1062a00640 10 client.5751 handle_command_reply: tid=0 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 client.5751 shutdown 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 -- 172.21.15.62:0/2201700313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 msgr2=0x55acff1f0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1f0d10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f107002a820 tx=0x7f1070004730 comp rx=0 tx=0).stop 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 -- 172.21.15.62:0/2201700313 shutdown_connections 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55acff200a00 0x55acff200e00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1ee7e0 0x55acff1f0d10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff1e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 --2- 172.21.15.62:0/2201700313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1c3d20 0x55acff1f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f1062a00640 0 client.5751 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.757+0000 7f108b766740 1 -- 172.21.15.62:0/2201700313 >> 172.21.15.62:0/2201700313 conn(0x55acff18bfb0 msgr2=0x55acff20cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.758+0000 7f108b766740 1 -- 172.21.15.62:0/2201700313 shutdown_connections 2024-04-25T09:07:48.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.758+0000 7f108b766740 1 -- 172.21.15.62:0/2201700313 wait complete. 2024-04-25T09:07:48.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.758+0000 7f108b766740 20 client.5751 trim_cache size 0 max 16384 2024-04-25T09:07:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.775+0000 7f108b766740 1 Processor -- start 2024-04-25T09:07:48.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.775+0000 7f108b766740 1 -- start start 2024-04-25T09:07:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f108b766740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff2191a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f108b766740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff1ee7e0 0x55acff1e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f108b766740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff2003f0 0x55acff1e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f108b766740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841ffdf0 con 0x55acff1c3d20 2024-04-25T09:07:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f108b766740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841fef50 con 0x55acff2003f0 2024-04-25T09:07:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f108b766740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10841f40b0 con 0x55acff1ee7e0 2024-04-25T09:07:48.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff2191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff2191a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53550/0 (socket says 172.21.15.62:53550) 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 -- 172.21.15.62:0/2230904616 learned_addr learned my addr 172.21.15.62:0/2230904616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 -- 172.21.15.62:0/2230904616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff1ee7e0 msgr2=0x55acff1e7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 --2- 172.21.15.62:0/2230904616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff1ee7e0 0x55acff1e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 -- 172.21.15.62:0/2230904616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff2003f0 msgr2=0x55acff1e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 --2- 172.21.15.62:0/2230904616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff2003f0 0x55acff1e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 -- 172.21.15.62:0/2230904616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acfedd43a0 con 0x55acff1c3d20 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.776+0000 7f1081e00640 1 --2- 172.21.15.62:0/2230904616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff2191a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f107400e530 tx=0x7f1074014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d141d432aca0953b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f106dc00640 1 -- 172.21.15.62:0/2230904616 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074027e60 con 0x55acff1c3d20 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f106dc00640 1 -- 172.21.15.62:0/2230904616 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074007020 con 0x55acff1c3d20 2024-04-25T09:07:48.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f106dc00640 1 -- 172.21.15.62:0/2230904616 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400eea0 con 0x55acff1c3d20 2024-04-25T09:07:48.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 -- 172.21.15.62:0/2230904616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 msgr2=0x55acff2191a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 --2- 172.21.15.62:0/2230904616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff2191a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f107400e530 tx=0x7f1074014000 comp rx=0 tx=0).stop 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 -- 172.21.15.62:0/2230904616 shutdown_connections 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 --2- 172.21.15.62:0/2230904616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff2003f0 0x55acff1e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 --2- 172.21.15.62:0/2230904616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff1ee7e0 0x55acff1e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 --2- 172.21.15.62:0/2230904616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff2191a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 -- 172.21.15.62:0/2230904616 >> 172.21.15.62:0/2230904616 conn(0x55acff18bfb0 msgr2=0x55acff1c4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 -- 172.21.15.62:0/2230904616 shutdown_connections 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.777+0000 7f108b766740 1 -- 172.21.15.62:0/2230904616 wait complete. 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.778+0000 7f108b766740 1 -- start start 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.778+0000 7f108b766740 1 Processor -- start 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.779+0000 7f108b766740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff1ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.779+0000 7f108b766740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1ee7e0 0x55acff1e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.779+0000 7f108b766740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff218780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.779+0000 7f108b766740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10841fe9e0 con 0x55acff1c3d20 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.779+0000 7f108b766740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff0f5030 con 0x55acff1ee7e0 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.779+0000 7f108b766740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acff1c23a0 con 0x55acff2003f0 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.779+0000 7f108b766740 10 client.0 _ll_register_callbacks cb 0x7f107c0554d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff1ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff1ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53552/0 (socket says 172.21.15.62:53552) 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 -- 172.21.15.62:0/3919129852 learned_addr learned my addr 172.21.15.62:0/3919129852 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 -- 172.21.15.62:0/3919129852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 msgr2=0x55acff218780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081400640 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1ee7e0 0x55acff1e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff218780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 -- 172.21.15.62:0/3919129852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1ee7e0 msgr2=0x55acff1e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1ee7e0 0x55acff1e6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 -- 172.21.15.62:0/3919129852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acff1e6890 con 0x55acff1c3d20 2024-04-25T09:07:48.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1081e00640 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff1ffcf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f107400ebf0 tx=0x7f10740036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8343a985115061a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:48.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.780+0000 7f1062a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:48.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f1062a00640 1 -- 172.21.15.62:0/3919129852 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740074a0 con 0x55acff1c3d20 2024-04-25T09:07:48.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f1062a00640 1 -- 172.21.15.62:0/3919129852 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f107401e040 con 0x55acff1c3d20 2024-04-25T09:07:48.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 1 -- 172.21.15.62:0/3919129852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1074007020 con 0x55acff1c3d20 2024-04-25T09:07:48.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f1062a00640 1 -- 172.21.15.62:0/3919129852 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1074007640 con 0x55acff1c3d20 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 10 client.5757 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 1 -- 172.21.15.62:0/3919129852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55acff1f23f0 con 0x55acff1c3d20 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f1062a00640 1 -- 172.21.15.62:0/3919129852 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074007020 con 0x55acff1c3d20 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f1062a00640 1 -- 172.21.15.62:0/3919129852 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1074014070 con 0x55acff1c3d20 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 10 client.5757 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 10 client.5757 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 20 client.5757 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55acff1c32b0 0x55acff222fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 4 client.5757 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.781+0000 7f108b766740 1 -- 172.21.15.62:0/3919129852 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55acff12e420 con 0x55acff1c32b0 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.782+0000 7f1081400640 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55acff1c32b0 0x55acff222fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.782+0000 7f1081400640 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55acff1c32b0 0x55acff222fc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c9d6feed6217a server_cookie=f3ab337d0ad9a536 in_seq=0 out_seq=0 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.782+0000 7f1062a00640 10 client.5757 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f1062a00640 1 -- 172.21.15.62:0/3919129852 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55acff12e420 con 0x55acff1c32b0 2024-04-25T09:07:48.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f1062a00640 10 client.5757 handle_command_reply: tid=0 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 client.5757 shutdown 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 -- 172.21.15.62:0/3919129852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 msgr2=0x55acff1ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff1ffcf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f107400ebf0 tx=0x7f10740036f0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 -- 172.21.15.62:0/3919129852 shutdown_connections 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55acff1c32b0 0x55acff222fc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55acff2003f0 0x55acff218780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55acff1ee7e0 0x55acff1e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 --2- 172.21.15.62:0/3919129852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55acff1c3d20 0x55acff1ffcf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f1062a00640 0 client.5757 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 -- 172.21.15.62:0/3919129852 >> 172.21.15.62:0/3919129852 conn(0x55acff18bfb0 msgr2=0x55acff2191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 -- 172.21.15.62:0/3919129852 shutdown_connections 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 1 -- 172.21.15.62:0/3919129852 wait complete. 2024-04-25T09:07:48.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.783+0000 7f108b766740 20 client.5757 trim_cache size 0 max 16384 2024-04-25T09:07:48.775 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:48.775 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": -2 2024-04-25T09:07:48.775 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 -- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106406b500 msgr2=0x7f106406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106406b500 0x7f106406d9c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1074004ea0 tx=0x7f1074009040 comp rx=0 tx=0).stop 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 -- 172.21.15.62:0/2200886090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 msgr2=0x7f108406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f108406c7c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f107c00cf20 tx=0x7f107c00a7d0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 -- 172.21.15.62:0/2200886090 shutdown_connections 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f106406b500 0x7f106406d9c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10840ccde0 0x7f108406c7c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10840b31a0 0x7f108406c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 --2- 172.21.15.62:0/2200886090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108406d2c0 0x7f108415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 -- 172.21.15.62:0/2200886090 >> 172.21.15.62:0/2200886090 conn(0x7f1084066f20 msgr2=0x7f108406d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 -- 172.21.15.62:0/2200886090 shutdown_connections 2024-04-25T09:07:48.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:48.784+0000 7f1083200640 1 -- 172.21.15.62:0/2200886090 wait complete. 2024-04-25T09:07:48.790 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T09:07:48.790 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:07:48.790 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:07:48.790 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:48.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.009+0000 7fb3a8c00640 1 Processor -- start 2024-04-25T09:07:48.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.009+0000 7fb3a8c00640 1 -- start start 2024-04-25T09:07:48.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.011+0000 7fb3a8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a40b7fe0 0x7fb3a40d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:48.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.011+0000 7fb3a8c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a40d20c0 con 0x7fb3a40b5b30 2024-04-25T09:07:48.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.011+0000 7fb3a8c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a40d23a0 con 0x7fb3a415da70 2024-04-25T09:07:48.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.011+0000 7fb3a8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40d2960 con 0x7fb3a40b7fe0 2024-04-25T09:07:48.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.011+0000 7fb399400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb3a40b5b30 0x7fb3a40b2410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53600/0 (socket says 172.21.15.62:53600) 2024-04-25T09:07:48.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.011+0000 7fb399400640 1 -- 172.21.15.62:0/3652711415 learned_addr learned my addr 172.21.15.62:0/3652711415 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.011+0000 7fb3a2800640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a40b7fe0 0x7fb3a40d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731942039 0 0) 0x7fb3a40d20c0 con 0x7fb3a40b5b30 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb388003520 con 0x7fb3a40b5b30 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012805461 0 0) 0x7fb388003520 con 0x7fb3a40b5b30 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a40b7fe0 msgr2=0x7fb3a40d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a40b7fe0 0x7fb3a40d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 >> v1:172.21.15.121:6789/0 conn(0x7fb3a415da70 legacy=0x7fb3a40b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40d2680 con 0x7fb3a40b5b30 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb38c001670 con 0x7fb3a40b5b30 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 >> v1:172.21.15.62:6789/0 conn(0x7fb3a40b5b30 legacy=0x7fb3a40b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3880039e0 0x7fb388003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a40b7fe0 0x7fb3880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40d2960 con 0x7fb388004800 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38800ba70 con 0x7fb3880039e0 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.012+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38800bc70 con 0x7fb3a40b7fe0 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.013+0000 7fb399400640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.013+0000 7fb3a1e00640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3880039e0 0x7fb388003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.013+0000 7fb3a2800640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a40b7fe0 0x7fb3880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.013+0000 7fb399400640 1 -- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a40b7fe0 msgr2=0x7fb3880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.013+0000 7fb399400640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a40b7fe0 0x7fb3880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.013+0000 7fb399400640 1 -- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3880039e0 msgr2=0x7fb388003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.013+0000 7fb399400640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3880039e0 0x7fb388003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.013+0000 7fb399400640 1 -- 172.21.15.62:0/3652711415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb388003520 con 0x7fb388004800 2024-04-25T09:07:49.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.015+0000 7fb399400640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb388004c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb38c001eb0 tx=0x7fb388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d0b5c9f35c44e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.015+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c007d30 con 0x7fb388004800 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.015+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c005af0 con 0x7fb388004800 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.015+0000 7fb3a1400640 1 -- 172.21.15.62:0/3652711415 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c007070 con 0x7fb388004800 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.015+0000 7fb3a8c00640 1 -- 172.21.15.62:0/3652711415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 msgr2=0x7fb388004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.015+0000 7fb3a8c00640 1 --2- 172.21.15.62:0/3652711415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb388004c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb38c001eb0 tx=0x7fb388005830 comp rx=0 tx=0).stop 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.015+0000 7fb3a2800640 1 -- 172.21.15.62:0/3652711415 reap_dead start 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.016+0000 7fb3a8c00640 1 -- 172.21.15.62:0/3652711415 shutdown_connections 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.016+0000 7fb3a8c00640 1 -- 172.21.15.62:0/3652711415 >> 172.21.15.62:0/3652711415 conn(0x7fb3a4066d10 msgr2=0x7fb3a4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.016+0000 7fb3a8c00640 1 -- 172.21.15.62:0/3652711415 shutdown_connections 2024-04-25T09:07:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.016+0000 7fb3a8c00640 1 -- 172.21.15.62:0/3652711415 wait complete. 2024-04-25T09:07:49.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.018+0000 7fb3a8c00640 1 Processor -- start 2024-04-25T09:07:49.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.018+0000 7fb3a8c00640 1 -- start start 2024-04-25T09:07:49.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb3a41f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:49.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a8c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3a40caa70 0x7fb3a40caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:49.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a41591d0 0x7fb3a4159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:49.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a415dfd0 con 0x7fb388004800 2024-04-25T09:07:49.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a8c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a415de50 con 0x7fb3a40caa70 2024-04-25T09:07:49.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40d1a00 con 0x7fb3a41591d0 2024-04-25T09:07:49.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb3a41f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb3a41f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53568/0 (socket says 172.21.15.62:53568) 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a1e00640 1 -- 172.21.15.62:0/2754085284 learned_addr learned my addr 172.21.15.62:0/2754085284 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb399400640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3a40caa70 0x7fb3a40caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.019+0000 7fb3a2800640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a41591d0 0x7fb3a4159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb3a1e00640 1 -- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a41591d0 msgr2=0x7fb3a4159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb3a1e00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a41591d0 0x7fb3a4159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb3a1e00640 1 -- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3a40caa70 msgr2=0x7fb3a40caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb3a1e00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3a40caa70 0x7fb3a40caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb3a1e00640 1 -- 172.21.15.62:0/2754085284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a414c810 con 0x7fb388004800 2024-04-25T09:07:49.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb3a1e00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb3a41f3250 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb39400e8c0 tx=0x7fb39400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb39aa00640 1 -- 172.21.15.62:0/2754085284 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394003b80 con 0x7fb388004800 2024-04-25T09:07:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb39aa00640 1 -- 172.21.15.62:0/2754085284 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394018070 con 0x7fb388004800 2024-04-25T09:07:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb39aa00640 1 -- 172.21.15.62:0/2754085284 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940050d0 con 0x7fb388004800 2024-04-25T09:07:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.020+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a40cb5e0 con 0x7fb388004800 2024-04-25T09:07:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.021+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a40cbba0 con 0x7fb388004800 2024-04-25T09:07:49.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.022+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a404ef20 con 0x7fb388004800 2024-04-25T09:07:49.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.023+0000 7fb39aa00640 1 -- 172.21.15.62:0/2754085284 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb394005270 con 0x7fb388004800 2024-04-25T09:07:49.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.024+0000 7fb39aa00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb38006b4d0 0x7fb38006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:49.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.024+0000 7fb39aa00640 1 -- 172.21.15.62:0/2754085284 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb39408a2a0 con 0x7fb388004800 2024-04-25T09:07:49.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.024+0000 7fb399400640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb38006b4d0 0x7fb38006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:49.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.025+0000 7fb399400640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb38006b4d0 0x7fb38006d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb38c032000 tx=0x7fb38c02d000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:49.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.028+0000 7fb39aa00640 1 -- 172.21.15.62:0/2754085284 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb39401a310 con 0x7fb388004800 2024-04-25T09:07:49.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.265+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3a40b7ba0 con 0x7fb388004800 2024-04-25T09:07:49.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.266+0000 7fb39aa00640 1 -- 172.21.15.62:0/2754085284 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fb394059110 con 0x7fb388004800 2024-04-25T09:07:49.254 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:49.254 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:28.504417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:07:49.254 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb38006b4d0 msgr2=0x7fb38006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb38006b4d0 0x7fb38006d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb38c032000 tx=0x7fb38c02d000 comp rx=0 tx=0).stop 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 msgr2=0x7fb3a41f3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb3a41f3250 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb39400e8c0 tx=0x7fb39400ed80 comp rx=0 tx=0).stop 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 shutdown_connections 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb38006b4d0 0x7fb38006d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3a41591d0 0x7fb3a4159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3a40caa70 0x7fb3a40caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 --2- 172.21.15.62:0/2754085284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb388004800 0x7fb3a41f3250 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 >> 172.21.15.62:0/2754085284 conn(0x7fb3a4066d10 msgr2=0x7fb3a40b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 shutdown_connections 2024-04-25T09:07:49.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:49.269+0000 7fb3a8c00640 1 -- 172.21.15.62:0/2754085284 wait complete. 2024-04-25T09:07:49.275 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:07:49.828 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:49.829 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:07:49.829 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:49.844 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:07:49.844 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:07:49.844 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:49.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:07:50.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.064+0000 7f5bc3000640 1 Processor -- start 2024-04-25T09:07:50.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.064+0000 7f5bc3000640 1 -- start start 2024-04-25T09:07:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.066+0000 7f5bc3000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 0x7f5bbc0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.066+0000 7f5bc3000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5bbc0d2fb0 0x7f5bbc0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.066+0000 7f5bc3000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc0ce780 con 0x7f5bbc0d3960 2024-04-25T09:07:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.066+0000 7f5bc3000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0d3440 con 0x7f5bbc0d2fb0 2024-04-25T09:07:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.066+0000 7f5bc3000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0ce1c0 con 0x7f5bbc15dd50 2024-04-25T09:07:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.066+0000 7f5bc2600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5bbc0d3960 0x7f5bbc0d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53616/0 (socket says 172.21.15.62:53616) 2024-04-25T09:07:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.066+0000 7f5bc2600640 1 -- 172.21.15.62:0/2548575051 learned_addr learned my addr 172.21.15.62:0/2548575051 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:50.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.066+0000 7f5bc1200640 1 --2- 172.21.15.62:0/2548575051 >> v2:172.21.15.62:3300/0 conn(0x7f5bbc0d2fb0 0x7f5bbc0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.067+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2548575051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 0x7f5bbc0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.067+0000 7f5bbbe00640 1 -- 172.21.15.62:0/2548575051 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724969230 0 0) 0x7f5bbc0ce780 con 0x7f5bbc0d3960 2024-04-25T09:07:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.067+0000 7f5bbbe00640 1 -- 172.21.15.62:0/2548575051 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba4003520 con 0x7f5bbc0d3960 2024-04-25T09:07:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.067+0000 7f5bc1200640 1 -- 172.21.15.62:0/2548575051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 msgr2=0x7f5bbc0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.067+0000 7f5bc1200640 1 --2- 172.21.15.62:0/2548575051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 0x7f5bbc0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.067+0000 7f5bc1200640 1 -- 172.21.15.62:0/2548575051 >> v1:172.21.15.62:6789/0 conn(0x7f5bbc0d3960 legacy=0x7f5bbc0d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.067+0000 7f5bc1200640 1 -- 172.21.15.62:0/2548575051 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0ce4a0 con 0x7f5bbc0d2fb0 2024-04-25T09:07:50.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.068+0000 7f5bc1200640 1 --2- 172.21.15.62:0/2548575051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0d2fb0 0x7f5bbc0ce9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80097a0 tx=0x7f5ba802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbd9eec95244e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.068+0000 7f5bbbe00640 1 -- 172.21.15.62:0/2548575051 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8037070 con 0x7f5bbc0d2fb0 2024-04-25T09:07:50.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.068+0000 7f5bbbe00640 1 -- 172.21.15.62:0/2548575051 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba8032440 con 0x7f5bbc0d2fb0 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.068+0000 7f5bbbe00640 1 -- 172.21.15.62:0/2548575051 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8032810 con 0x7f5bbc0d2fb0 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.069+0000 7f5bc3000640 1 -- 172.21.15.62:0/2548575051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0d2fb0 msgr2=0x7f5bbc0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.069+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2548575051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0d2fb0 0x7f5bbc0ce9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80097a0 tx=0x7f5ba802a180 comp rx=0 tx=0).stop 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.069+0000 7f5bc3000640 1 -- 172.21.15.62:0/2548575051 shutdown_connections 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.069+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2548575051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0d2fb0 0x7f5bbc0ce9c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.069+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2548575051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 0x7f5bbc0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.069+0000 7f5bc3000640 1 -- 172.21.15.62:0/2548575051 >> 172.21.15.62:0/2548575051 conn(0x7f5bbc06b000 msgr2=0x7f5bbc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.069+0000 7f5bc3000640 1 -- 172.21.15.62:0/2548575051 shutdown_connections 2024-04-25T09:07:50.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.069+0000 7f5bc3000640 1 -- 172.21.15.62:0/2548575051 wait complete. 2024-04-25T09:07:50.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.071+0000 7f5bc3000640 1 Processor -- start 2024-04-25T09:07:50.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.071+0000 7f5bc3000640 1 -- start start 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc3000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bbc0d3960 0x7f5bbc0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc3000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 0x7f5bbc14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc3000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0b3d30 0x7f5bbc0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc3000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b6b00 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc3000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0cef00 con 0x7f5bbc0d3960 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc3000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b6c80 con 0x7f5bbc15dd50 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0b3d30 0x7f5bbc0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0b3d30 0x7f5bbc0cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53576/0 (socket says 172.21.15.62:53576) 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.072+0000 7f5bc2600640 1 -- 172.21.15.62:0/2657338565 learned_addr learned my addr 172.21.15.62:0/2657338565 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bc1200640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 0x7f5bbc14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bbc0d3960 0x7f5bbc0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bc2600640 1 -- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 msgr2=0x7f5bbc14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 0x7f5bbc14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bc2600640 1 -- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bbc0d3960 msgr2=0x7f5bbc0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bbc0d3960 0x7f5bbc0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bc2600640 1 -- 172.21.15.62:0/2657338565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc150240 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0b3d30 0x7f5bbc0cb040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400cb40 tx=0x7f5bb400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bb9600640 1 -- 172.21.15.62:0/2657338565 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4017070 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.073+0000 7f5bb9600640 1 -- 172.21.15.62:0/2657338565 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb400aca0 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.074+0000 7f5bc3000640 1 -- 172.21.15.62:0/2657338565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc0c8d10 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.074+0000 7f5bb9600640 1 -- 172.21.15.62:0/2657338565 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4012620 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.074+0000 7f5bc3000640 1 -- 172.21.15.62:0/2657338565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc0b4d60 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.076+0000 7f5bb9600640 1 -- 172.21.15.62:0/2657338565 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bb401a070 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.077+0000 7f5bb9600640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.078+0000 7f5bb9600640 1 -- 172.21.15.62:0/2657338565 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb4089fe0 con 0x7f5bbc0b3d30 2024-04-25T09:07:50.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.078+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.078+0000 7f5bc75d6740 1 Processor -- start 2024-04-25T09:07:50.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.078+0000 7f5bc75d6740 1 -- start start 2024-04-25T09:07:50.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.079+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577b670 0x55bdc5796c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.079+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc577ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.079+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577bf90 0x55bdc5778250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.079+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc56b4c40 con 0x55bdc577bf90 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.079+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc565e050 con 0x55bdc578d090 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.079+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdc56b0ce0 con 0x55bdc577b670 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.079+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006fd0 tx=0x7f5bb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.079+0000 7f5bc2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577bf90 0x55bdc5778250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577bf90 0x55bdc5778250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53584/0 (socket says 172.21.15.62:53584) 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc2600640 1 -- 172.21.15.62:0/2084747496 learned_addr learned my addr 172.21.15.62:0/2084747496 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577b670 0x55bdc5796c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc1200640 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc577ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc2600640 1 -- 172.21.15.62:0/2084747496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577b670 msgr2=0x55bdc5796c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577b670 0x55bdc5796c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc2600640 1 -- 172.21.15.62:0/2084747496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 msgr2=0x55bdc577ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc577ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc2600640 1 -- 172.21.15.62:0/2084747496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdc5495b00 con 0x55bdc577bf90 2024-04-25T09:07:50.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.080+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577bf90 0x55bdc5778250 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4060990 tx=0x7f5bb4058870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3008643a2db9ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bae000640 1 -- 172.21.15.62:0/2084747496 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4056ab0 con 0x55bdc577bf90 2024-04-25T09:07:50.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bae000640 1 -- 172.21.15.62:0/2084747496 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4054e20 con 0x55bdc577bf90 2024-04-25T09:07:50.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bae000640 1 -- 172.21.15.62:0/2084747496 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4060c50 con 0x55bdc577bf90 2024-04-25T09:07:50.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2084747496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577bf90 msgr2=0x55bdc5778250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577bf90 0x55bdc5778250 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4060990 tx=0x7f5bb4058870 comp rx=0 tx=0).stop 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2084747496 shutdown_connections 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577bf90 0x55bdc5778250 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc577ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2084747496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577b670 0x55bdc5796c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2084747496 >> 172.21.15.62:0/2084747496 conn(0x55bdc57185e0 msgr2=0x55bdc5796480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2084747496 shutdown_connections 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.081+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2084747496 wait complete. 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.082+0000 7f5bc75d6740 1 -- start start 2024-04-25T09:07:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.082+0000 7f5bc75d6740 1 Processor -- start 2024-04-25T09:07:50.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.083+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.083+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc577e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc577ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc57529b0 con 0x55bdc578d090 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc56535d0 con 0x55bdc577b670 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdc574d230 con 0x55bdc577bf90 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc75d6740 10 client.0 _ll_register_callbacks cb 0x55bdc56aca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc577ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc577ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53592/0 (socket says 172.21.15.62:53592) 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc2600640 1 -- 172.21.15.62:0/2359462202 learned_addr learned my addr 172.21.15.62:0/2359462202 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc1200640 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc577e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc2600640 1 -- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 msgr2=0x55bdc577e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc577e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc2600640 1 -- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 msgr2=0x55bdc577cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc2600640 1 -- 172.21.15.62:0/2359462202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdc53893f0 con 0x55bdc578d090 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.084+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc577ec00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4088840 tx=0x7f5bb4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ff139f90ca0388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5ba2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5ba2a00640 1 -- 172.21.15.62:0/2359462202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4056b70 con 0x55bdc578d090 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5ba2a00640 1 -- 172.21.15.62:0/2359462202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb4004b60 con 0x55bdc578d090 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2359462202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bb4054e20 con 0x55bdc578d090 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5ba2a00640 1 -- 172.21.15.62:0/2359462202 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb4004da0 con 0x55bdc578d090 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5bc75d6740 10 client.5778 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2359462202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55bdc54f6290 con 0x55bdc578d090 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5ba2a00640 1 -- 172.21.15.62:0/2359462202 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4054e20 con 0x55bdc578d090 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5ba2a00640 1 -- 172.21.15.62:0/2359462202 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5bb4057c00 con 0x55bdc578d090 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5bc75d6740 10 client.5778 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5bc75d6740 10 client.5778 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:50.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5bc75d6740 20 client.5778 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.085+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55bdc577d590 0x55bdc5795c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.086+0000 7f5bc75d6740 4 client.5778 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.086+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2359462202 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdc56c2180 con 0x55bdc577d590 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.086+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55bdc577d590 0x55bdc5795c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.086+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55bdc577d590 0x55bdc5795c00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4840a8a278fd372f server_cookie=d21e6102486ffbf in_seq=0 out_seq=0 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.086+0000 7f5ba2a00640 10 client.5778 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.087+0000 7f5ba2a00640 1 -- 172.21.15.62:0/2359462202 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdc56c2180 con 0x55bdc577d590 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.087+0000 7f5ba2a00640 10 client.5778 handle_command_reply: tid=0 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.087+0000 7f5bc75d6740 1 client.5778 shutdown 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2359462202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 msgr2=0x55bdc577ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc577ec00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4088840 tx=0x7f5bb4001250 comp rx=0 tx=0).stop 2024-04-25T09:07:50.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2359462202 shutdown_connections 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55bdc577d590 0x55bdc5795c00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc577ec00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc577e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2359462202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5ba2a00640 0 client.5778 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2359462202 >> 172.21.15.62:0/2359462202 conn(0x55bdc57185e0 msgr2=0x55bdc5773670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2359462202 shutdown_connections 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2359462202 wait complete. 2024-04-25T09:07:50.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.088+0000 7f5bc75d6740 20 client.5778 trim_cache size 0 max 16384 2024-04-25T09:07:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.106+0000 7f5bc75d6740 1 Processor -- start 2024-04-25T09:07:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.106+0000 7f5bc75d6740 1 -- start start 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.106+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.106+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc5747150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.106+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc574adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc1ffb70 con 0x55bdc578d090 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0c8a80 con 0x55bdc577b670 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0c7e80 con 0x55bdc577bf90 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc574adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc574adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53604/0 (socket says 172.21.15.62:53604) 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc2600640 1 -- 172.21.15.62:0/2179980958 learned_addr learned my addr 172.21.15.62:0/2179980958 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc1200640 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc5747150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc2600640 1 -- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 msgr2=0x55bdc5747150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc5747150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc2600640 1 -- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 msgr2=0x55bdc577af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.107+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.108+0000 7f5bc2600640 1 -- 172.21.15.62:0/2179980958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdc53624b0 con 0x55bdc578d090 2024-04-25T09:07:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.108+0000 7f5bc1200640 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc5747150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.108+0000 7f5bc2600640 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc574adc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4054c10 tx=0x7f5bb406e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1d45cf0f317827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.109+0000 7f5bad600640 1 -- 172.21.15.62:0/2179980958 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4004b60 con 0x55bdc578d090 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.109+0000 7f5bad600640 1 -- 172.21.15.62:0/2179980958 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4070280 con 0x55bdc578d090 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.109+0000 7f5bad600640 1 -- 172.21.15.62:0/2179980958 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4054e20 con 0x55bdc578d090 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2179980958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 msgr2=0x55bdc574adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc574adc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4054c10 tx=0x7f5bb406e040 comp rx=0 tx=0).stop 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2179980958 shutdown_connections 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc578d090 0x55bdc574adc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc5747150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/2179980958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc577b670 0x55bdc577af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2179980958 >> 172.21.15.62:0/2179980958 conn(0x55bdc57185e0 msgr2=0x55bdc574c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2179980958 shutdown_connections 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.110+0000 7f5bc75d6740 1 -- 172.21.15.62:0/2179980958 wait complete. 2024-04-25T09:07:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.111+0000 7f5bc75d6740 1 -- start start 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.111+0000 7f5bc75d6740 1 Processor -- start 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577b670 0x55bdc5795920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc57ff670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc75d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc5794720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdc577bac0 con 0x55bdc577b670 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0c8480 con 0x55bdc578d090 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc75d6740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdc5793540 con 0x55bdc577bf90 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc75d6740 10 client.0 _ll_register_callbacks cb 0x7f5bb400af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:50.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577b670 0x55bdc5795920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577b670 0x55bdc5795920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53620/0 (socket says 172.21.15.62:53620) 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc1c00640 1 -- 172.21.15.62:0/3070705270 learned_addr learned my addr 172.21.15.62:0/3070705270 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc2600640 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc5794720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.112+0000 7f5bc1200640 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc57ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5bc1c00640 1 -- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 msgr2=0x55bdc57ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc57ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5bc1c00640 1 -- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 msgr2=0x55bdc5794720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc5794720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5bc1c00640 1 -- 172.21.15.62:0/3070705270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdc57b81c0 con 0x55bdc577b670 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5bc2600640 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc5794720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5bc1c00640 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577b670 0x55bdc5795920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55bdc54f6340 tx=0x7f5bb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427ffd8737bca8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5ba2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5ba2a00640 1 -- 172.21.15.62:0/3070705270 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb001b030 con 0x55bdc577b670 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5ba2a00640 1 -- 172.21.15.62:0/3070705270 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb0025d40 con 0x55bdc577b670 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.113+0000 7f5bc75d6740 1 -- 172.21.15.62:0/3070705270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bb4070280 con 0x55bdc577b670 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5ba2a00640 1 -- 172.21.15.62:0/3070705270 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb00270b0 con 0x55bdc577b670 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc75d6740 10 client.5784 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:50.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc75d6740 1 -- 172.21.15.62:0/3070705270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55bdc577aa70 con 0x55bdc577b670 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5ba2a00640 1 -- 172.21.15.62:0/3070705270 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4070280 con 0x55bdc577b670 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5ba2a00640 1 -- 172.21.15.62:0/3070705270 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5bb003b080 con 0x55bdc577b670 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc75d6740 10 client.5784 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc75d6740 10 client.5784 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc75d6740 20 client.5784 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55bdc5794c60 0x55bdc57a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc75d6740 4 client.5784 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc75d6740 1 -- 172.21.15.62:0/3070705270 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55bdc577bdc0 con 0x55bdc5794c60 2024-04-25T09:07:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.114+0000 7f5bc1200640 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55bdc5794c60 0x55bdc57a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.115+0000 7f5bc1200640 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55bdc5794c60 0x55bdc57a8670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47421a226c7c852a server_cookie=d806deb4f5f66d66 in_seq=0 out_seq=0 2024-04-25T09:07:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.115+0000 7f5ba2a00640 10 client.5784 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5ba2a00640 1 -- 172.21.15.62:0/3070705270 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bdc577bdc0 con 0x55bdc5794c60 2024-04-25T09:07:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5ba2a00640 10 client.5784 handle_command_reply: tid=0 2024-04-25T09:07:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 client.5784 shutdown 2024-04-25T09:07:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 -- 172.21.15.62:0/3070705270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577b670 msgr2=0x55bdc5795920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577b670 0x55bdc5795920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55bdc54f6340 tx=0x7f5bb0004730 comp rx=0 tx=0).stop 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 -- 172.21.15.62:0/3070705270 shutdown_connections 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55bdc5794c60 0x55bdc57a8670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdc578d090 0x55bdc5794720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdc577bf90 0x55bdc57ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 --2- 172.21.15.62:0/3070705270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdc577b670 0x55bdc5795920 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5ba2a00640 0 client.5784 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 -- 172.21.15.62:0/3070705270 >> 172.21.15.62:0/3070705270 conn(0x55bdc57185e0 msgr2=0x55bdc57999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 -- 172.21.15.62:0/3070705270 shutdown_connections 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 1 -- 172.21.15.62:0/3070705270 wait complete. 2024-04-25T09:07:50.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.116+0000 7f5bc75d6740 20 client.5784 trim_cache size 0 max 16384 2024-04-25T09:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "08d4fc58-457c-4afc-baad-27faadb6e74a", 2024-04-25T09:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:07:50.109 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:50.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 -- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b9c06b4d0 msgr2=0x7f5b9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006fd0 tx=0x7f5bb0008040 comp rx=0 tx=0).stop 2024-04-25T09:07:50.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 -- 172.21.15.62:0/2657338565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0b3d30 msgr2=0x7f5bbc0cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0b3d30 0x7f5bbc0cb040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400cb40 tx=0x7f5bb400a060 comp rx=0 tx=0).stop 2024-04-25T09:07:50.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 -- 172.21.15.62:0/2657338565 shutdown_connections 2024-04-25T09:07:50.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5bbc0b3d30 0x7f5bbc0cb040 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5bbc15dd50 0x7f5bbc14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 --2- 172.21.15.62:0/2657338565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bbc0d3960 0x7f5bbc0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 -- 172.21.15.62:0/2657338565 >> 172.21.15.62:0/2657338565 conn(0x7f5bbc06b000 msgr2=0x7f5bbc06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:50.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 -- 172.21.15.62:0/2657338565 shutdown_connections 2024-04-25T09:07:50.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.117+0000 7f5bc3000640 1 -- 172.21.15.62:0/2657338565 wait complete. 2024-04-25T09:07:50.121 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '08d4fc58-457c-4afc-baad-27faadb6e74a', 'mode': 'asynchronous'}' 2024-04-25T09:07:50.121 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:07:50.121 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:07:50.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:50.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.338+0000 7f0033200640 1 Processor -- start 2024-04-25T09:07:50.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.338+0000 7f0033200640 1 -- start start 2024-04-25T09:07:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.339+0000 7f0033200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f003415dd70 0x7f00340cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.339+0000 7f0033200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f00340d1e80 0x7f00340d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.339+0000 7f0033200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00340d23b0 con 0x7f00340d3450 2024-04-25T09:07:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.339+0000 7f0033200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00340d2690 con 0x7f00340d1e80 2024-04-25T09:07:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.339+0000 7f0033200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340d2970 con 0x7f003415dd70 2024-04-25T09:07:50.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.340+0000 7f0031e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f003415dd70 0x7f00340cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.340+0000 7f0031e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f003415dd70 0x7f00340cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:52392/0 (socket says 172.21.15.62:52392) 2024-04-25T09:07:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.340+0000 7f0031400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f00340d1e80 0x7f00340d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.340+0000 7f0031e00640 1 -- 172.21.15.62:0/1463733547 learned_addr learned my addr 172.21.15.62:0/1463733547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.340+0000 7f0032800640 1 --1- 172.21.15.62:0/1463733547 >> v1:172.21.15.121:6789/0 conn(0x7f00340d3450 0x7f00340b53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:55090/0 (socket says 172.21.15.62:55090) 2024-04-25T09:07:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.341+0000 7f0031400640 1 -- 172.21.15.62:0/1463733547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f003415dd70 msgr2=0x7f00340cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.341+0000 7f0031400640 1 --2- 172.21.15.62:0/1463733547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f003415dd70 0x7f00340cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.341+0000 7f0031400640 1 -- 172.21.15.62:0/1463733547 >> v1:172.21.15.121:6789/0 conn(0x7f00340d3450 legacy=0x7f00340b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.341+0000 7f0031400640 1 -- 172.21.15.62:0/1463733547 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00340d2c50 con 0x7f00340d1e80 2024-04-25T09:07:50.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.342+0000 7f0031400640 1 --2- 172.21.15.62:0/1463733547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00340d1e80 0x7f00340d0240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0020009d90 tx=0x7f002002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d48fb19efa1959 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0030a00640 1 -- 172.21.15.62:0/1463733547 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020007740 con 0x7f00340d1e80 2024-04-25T09:07:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0030a00640 1 -- 172.21.15.62:0/1463733547 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00200078e0 con 0x7f00340d1e80 2024-04-25T09:07:50.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0030a00640 1 -- 172.21.15.62:0/1463733547 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020007cb0 con 0x7f00340d1e80 2024-04-25T09:07:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0033200640 1 -- 172.21.15.62:0/1463733547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00340d1e80 msgr2=0x7f00340d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0033200640 1 --2- 172.21.15.62:0/1463733547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00340d1e80 0x7f00340d0240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0020009d90 tx=0x7f002002a660 comp rx=0 tx=0).stop 2024-04-25T09:07:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0033200640 1 -- 172.21.15.62:0/1463733547 shutdown_connections 2024-04-25T09:07:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0033200640 1 --2- 172.21.15.62:0/1463733547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00340d1e80 0x7f00340d0240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0033200640 1 --2- 172.21.15.62:0/1463733547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f003415dd70 0x7f00340cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0033200640 1 -- 172.21.15.62:0/1463733547 >> 172.21.15.62:0/1463733547 conn(0x7f003406b210 msgr2=0x7f00340668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0033200640 1 -- 172.21.15.62:0/1463733547 shutdown_connections 2024-04-25T09:07:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.343+0000 7f0033200640 1 -- 172.21.15.62:0/1463733547 wait complete. 2024-04-25T09:07:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.345+0000 7f0033200640 1 Processor -- start 2024-04-25T09:07:50.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.345+0000 7f0033200640 1 -- start start 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.346+0000 7f0033200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340d1e80 0x7f003415ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.346+0000 7f0033200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00340d3450 0x7f003415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.346+0000 7f0033200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f003415dd70 0x7f00341f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.346+0000 7f0033200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00340cf1a0 con 0x7f00340d1e80 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.346+0000 7f0033200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003415e150 con 0x7f003415dd70 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.346+0000 7f0033200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340cf320 con 0x7f00340d3450 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340d1e80 0x7f003415ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340d1e80 0x7f003415ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53636/0 (socket says 172.21.15.62:53636) 2024-04-25T09:07:50.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 -- 172.21.15.62:0/4031692612 learned_addr learned my addr 172.21.15.62:0/4031692612 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:50.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031400640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00340d3450 0x7f003415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0032800640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f003415dd70 0x7f00341f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 -- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00340d3450 msgr2=0x7f003415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00340d3450 0x7f003415ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 -- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f003415dd70 msgr2=0x7f00341f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f003415dd70 0x7f00341f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 -- 172.21.15.62:0/4031692612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003415f770 con 0x7f00340d1e80 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0032800640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f003415dd70 0x7f00341f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.347+0000 7f0031e00640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340d1e80 0x7f003415ba30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f002400e870 tx=0x7f002400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.348+0000 7f002a000640 1 -- 172.21.15.62:0/4031692612 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024017070 con 0x7f00340d1e80 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.348+0000 7f002a000640 1 -- 172.21.15.62:0/4031692612 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002400c8e0 con 0x7f00340d1e80 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.348+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00340b35f0 con 0x7f00340d1e80 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.348+0000 7f002a000640 1 -- 172.21.15.62:0/4031692612 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400ccd0 con 0x7f00340d1e80 2024-04-25T09:07:50.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.348+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003415f340 con 0x7f00340d1e80 2024-04-25T09:07:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.349+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003404ef20 con 0x7f00340d1e80 2024-04-25T09:07:50.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.350+0000 7f002a000640 1 -- 172.21.15.62:0/4031692612 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f002401a070 con 0x7f00340d1e80 2024-04-25T09:07:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.351+0000 7f002a000640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f001406b4d0 0x7f001406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.351+0000 7f002a000640 1 -- 172.21.15.62:0/4031692612 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f002408a420 con 0x7f00340d1e80 2024-04-25T09:07:50.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.351+0000 7f0031400640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f001406b4d0 0x7f001406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:50.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.353+0000 7f0031400640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f001406b4d0 0x7f001406d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0020034000 tx=0x7f002002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:50.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.356+0000 7f002a000640 1 -- 172.21.15.62:0/4031692612 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00240593f0 con 0x7f00340d1e80 2024-04-25T09:07:50.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.597+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00340c7510 con 0x7f00340d1e80 2024-04-25T09:07:50.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.599+0000 7f002a000640 1 -- 172.21.15.62:0/4031692612 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0024058cf0 con 0x7f00340d1e80 2024-04-25T09:07:50.587 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:50.587 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:28.504417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:07:50.587 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.602+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f001406b4d0 msgr2=0x7f001406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.602+0000 7f0033200640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f001406b4d0 0x7f001406d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0020034000 tx=0x7f002002f000 comp rx=0 tx=0).stop 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.602+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340d1e80 msgr2=0x7f003415ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.602+0000 7f0033200640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340d1e80 0x7f003415ba30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f002400e870 tx=0x7f002400ed30 comp rx=0 tx=0).stop 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.603+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 shutdown_connections 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.603+0000 7f0033200640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f001406b4d0 0x7f001406d990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.603+0000 7f0033200640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f003415dd70 0x7f00341f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.603+0000 7f0033200640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00340d3450 0x7f003415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.603+0000 7f0033200640 1 --2- 172.21.15.62:0/4031692612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00340d1e80 0x7f003415ba30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.603+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 >> 172.21.15.62:0/4031692612 conn(0x7f003406b210 msgr2=0x7f003415cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:50.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.604+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 shutdown_connections 2024-04-25T09:07:50.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:50.604+0000 7f0033200640 1 -- 172.21.15.62:0/4031692612 wait complete. 2024-04-25T09:07:50.617 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:07:50.831 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:50.831 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:07:50.831 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:50.846 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:07:50.846 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:07:50.846 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:50.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:07:51.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.065+0000 7f8f43200640 1 Processor -- start 2024-04-25T09:07:51.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.065+0000 7f8f43200640 1 -- start start 2024-04-25T09:07:51.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.067+0000 7f8f43200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8f3c0ceb50 0x7f8f3c06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.067+0000 7f8f43200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f3c15e7c0 0x7f8f3c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.067+0000 7f8f43200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c15ee30 con 0x7f8f3c06d800 2024-04-25T09:07:51.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.067+0000 7f8f43200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c1618b0 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.067+0000 7f8f43200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c161b90 con 0x7f8f3c0ceb50 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.067+0000 7f8f42800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f3c15e7c0 0x7f8f3c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.067+0000 7f8f42800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8f3c15e7c0 0x7f8f3c0b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53640/0 (socket says 172.21.15.62:53640) 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.067+0000 7f8f42800640 1 -- 172.21.15.62:0/3316004178 learned_addr learned my addr 172.21.15.62:0/3316004178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.068+0000 7f8f41e00640 1 --2- 172.21.15.62:0/3316004178 >> v2:172.21.15.121:3300/0 conn(0x7f8f3c0ceb50 0x7f8f3c06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.068+0000 7f8f42800640 1 -- 172.21.15.62:0/3316004178 >> v2:172.21.15.121:3300/0 conn(0x7f8f3c0ceb50 msgr2=0x7f8f3c06d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.068+0000 7f8f42800640 1 --2- 172.21.15.62:0/3316004178 >> v2:172.21.15.121:3300/0 conn(0x7f8f3c0ceb50 0x7f8f3c06d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.068+0000 7f8f42800640 1 -- 172.21.15.62:0/3316004178 >> v1:172.21.15.62:6789/0 conn(0x7f8f3c06d800 legacy=0x7f8f3c15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.068+0000 7f8f42800640 1 -- 172.21.15.62:0/3316004178 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0cef30 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.068+0000 7f8f41e00640 1 --2- 172.21.15.62:0/3316004178 >> v2:172.21.15.121:3300/0 conn(0x7f8f3c0ceb50 0x7f8f3c06d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:51.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.069+0000 7f8f42800640 1 --2- 172.21.15.62:0/3316004178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c0b4550 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0af200 tx=0x7f8f3802a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649d7f01ad862a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.069+0000 7f8f40a00640 1 -- 172.21.15.62:0/3316004178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38036070 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.069+0000 7f8f40a00640 1 -- 172.21.15.62:0/3316004178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38031600 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.069+0000 7f8f40a00640 1 -- 172.21.15.62:0/3316004178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380319d0 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.070+0000 7f8f43200640 1 -- 172.21.15.62:0/3316004178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 msgr2=0x7f8f3c0b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.070+0000 7f8f43200640 1 --2- 172.21.15.62:0/3316004178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c0b4550 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0af200 tx=0x7f8f3802a6b0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.070+0000 7f8f43200640 1 -- 172.21.15.62:0/3316004178 shutdown_connections 2024-04-25T09:07:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.070+0000 7f8f43200640 1 --2- 172.21.15.62:0/3316004178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c0b4550 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.070+0000 7f8f43200640 1 --2- 172.21.15.62:0/3316004178 >> v2:172.21.15.121:3300/0 conn(0x7f8f3c0ceb50 0x7f8f3c06d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.070+0000 7f8f43200640 1 -- 172.21.15.62:0/3316004178 >> 172.21.15.62:0/3316004178 conn(0x7f8f3c066f20 msgr2=0x7f8f3c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.070+0000 7f8f43200640 1 -- 172.21.15.62:0/3316004178 shutdown_connections 2024-04-25T09:07:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.070+0000 7f8f43200640 1 -- 172.21.15.62:0/3316004178 wait complete. 2024-04-25T09:07:51.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.073+0000 7f8f43200640 1 Processor -- start 2024-04-25T09:07:51.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.073+0000 7f8f43200640 1 -- start start 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.074+0000 7f8f43200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f3c06d800 0x7f8f3c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.074+0000 7f8f43200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f3c0ceb50 0x7f8f3c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.074+0000 7f8f43200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.074+0000 7f8f43200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0b4a90 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.074+0000 7f8f43200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1625e0 con 0x7f8f3c0ceb50 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.074+0000 7f8f43200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0b4c10 con 0x7f8f3c06d800 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f42800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f42800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53644/0 (socket says 172.21.15.62:53644) 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f42800640 1 -- 172.21.15.62:0/2160599993 learned_addr learned my addr 172.21.15.62:0/2160599993 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f41e00640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f3c06d800 0x7f8f3c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f41400640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f3c0ceb50 0x7f8f3c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f42800640 1 -- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f3c06d800 msgr2=0x7f8f3c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f42800640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f3c06d800 0x7f8f3c0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f42800640 1 -- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f3c0ceb50 msgr2=0x7f8f3c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f42800640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f3c0ceb50 0x7f8f3c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f42800640 1 -- 172.21.15.62:0/2160599993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c1f4110 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.075+0000 7f8f41400640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f3c0ceb50 0x7f8f3c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:51.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.076+0000 7f8f42800640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c156190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8f38034040 tx=0x7f8f38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.076+0000 7f8f36000640 1 -- 172.21.15.62:0/2160599993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38036050 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.076+0000 7f8f36000640 1 -- 172.21.15.62:0/2160599993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38042070 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.076+0000 7f8f43200640 1 -- 172.21.15.62:0/2160599993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c1f6be0 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.076+0000 7f8f36000640 1 -- 172.21.15.62:0/2160599993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38009510 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.076+0000 7f8f43200640 1 -- 172.21.15.62:0/2160599993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c1f71a0 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.078+0000 7f8f36000640 1 -- 172.21.15.62:0/2160599993 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f38007bd0 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.079+0000 7f8f36000640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f2006b4d0 0x7f8f2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.080+0000 7f8f36000640 1 -- 172.21.15.62:0/2160599993 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f380ab1f0 con 0x7f8f3c15e7c0 2024-04-25T09:07:51.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.080+0000 7f8f4776d740 1 Processor -- start 2024-04-25T09:07:51.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.080+0000 7f8f41e00640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f2006b4d0 0x7f8f2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.080+0000 7f8f4776d740 1 -- start start 2024-04-25T09:07:51.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.081+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.081+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893ddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.081+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893de1f0 0x55ec893b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.081+0000 7f8f4776d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec89316cf0 con 0x55ec893fb810 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.081+0000 7f8f4776d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec892c01d0 con 0x55ec893de930 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.081+0000 7f8f4776d740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec89312d90 con 0x55ec893de1f0 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.081+0000 7f8f41e00640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f2006b4d0 0x7f8f2006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000a990 tx=0x7f8f30008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53654/0 (socket says 172.21.15.62:53654) 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 -- 172.21.15.62:0/3246284260 learned_addr learned my addr 172.21.15.62:0/3246284260 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41400640 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893ddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f42800640 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893de1f0 0x55ec893b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 -- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893de1f0 msgr2=0x55ec893b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893de1f0 0x55ec893b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 -- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 msgr2=0x55ec893ddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893ddcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 -- 172.21.15.62:0/3246284260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec890f7f00 con 0x55ec893fb810 2024-04-25T09:07:51.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41400640 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893ddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:51.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f42800640 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893de1f0 0x55ec893b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:51.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.082+0000 7f8f41e00640 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893ae1a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000fae0 tx=0x7f8f30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cfd02f519d41dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f2a600640 1 -- 172.21.15.62:0/3246284260 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3001b050 con 0x55ec893fb810 2024-04-25T09:07:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f2a600640 1 -- 172.21.15.62:0/3246284260 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f30025070 con 0x55ec893fb810 2024-04-25T09:07:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f2a600640 1 -- 172.21.15.62:0/3246284260 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30020440 con 0x55ec893fb810 2024-04-25T09:07:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f4776d740 1 -- 172.21.15.62:0/3246284260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 msgr2=0x55ec893ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f4776d740 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893ae1a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000fae0 tx=0x7f8f30012000 comp rx=0 tx=0).stop 2024-04-25T09:07:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f4776d740 1 -- 172.21.15.62:0/3246284260 shutdown_connections 2024-04-25T09:07:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f4776d740 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893de1f0 0x55ec893b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f4776d740 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893ddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f4776d740 1 --2- 172.21.15.62:0/3246284260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893ae1a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.083+0000 7f8f4776d740 1 -- 172.21.15.62:0/3246284260 >> 172.21.15.62:0/3246284260 conn(0x55ec893b0410 msgr2=0x55ec893f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.084+0000 7f8f4776d740 1 -- 172.21.15.62:0/3246284260 shutdown_connections 2024-04-25T09:07:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.084+0000 7f8f4776d740 1 -- 172.21.15.62:0/3246284260 wait complete. 2024-04-25T09:07:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.084+0000 7f8f4776d740 1 -- start start 2024-04-25T09:07:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.084+0000 7f8f4776d740 1 Processor -- start 2024-04-25T09:07:51.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.085+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893de930 0x55ec893f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.085+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893fb810 0x55ec893f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.085+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893cb0b0 0x55ec893cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f4776d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec893b25c0 con 0x55ec893de930 2024-04-25T09:07:51.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f4776d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec892b5b50 con 0x55ec893cb0b0 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f4776d740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec893b19b0 con 0x55ec893fb810 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f4776d740 10 client.0 _ll_register_callbacks cb 0x55ec892d7370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f41e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893de930 0x55ec893f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f41e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893de930 0x55ec893f90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53664/0 (socket says 172.21.15.62:53664) 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f41e00640 1 -- 172.21.15.62:0/1889539043 learned_addr learned my addr 172.21.15.62:0/1889539043 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f42800640 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893cb0b0 0x55ec893cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f41e00640 1 -- 172.21.15.62:0/1889539043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893fb810 msgr2=0x55ec893f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f41e00640 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893fb810 0x55ec893f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f41e00640 1 -- 172.21.15.62:0/1889539043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893cb0b0 msgr2=0x55ec893cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f41e00640 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893cb0b0 0x55ec893cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.086+0000 7f8f41e00640 1 -- 172.21.15.62:0/1889539043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec88feb7f0 con 0x55ec893de930 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f41e00640 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893de930 0x55ec893f90b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000afd0 tx=0x7f8f30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba5424eceb45851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f1ea00640 1 -- 172.21.15.62:0/1889539043 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30020850 con 0x55ec893de930 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f1ea00640 1 -- 172.21.15.62:0/1889539043 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f300209f0 con 0x55ec893de930 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f4776d740 1 -- 172.21.15.62:0/1889539043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f30025070 con 0x55ec893de930 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f1ea00640 1 -- 172.21.15.62:0/1889539043 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f3001b020 con 0x55ec893de930 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f4776d740 10 client.5802 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f4776d740 1 -- 172.21.15.62:0/1889539043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55ec89158690 con 0x55ec893de930 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f1ea00640 1 -- 172.21.15.62:0/1889539043 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f30025070 con 0x55ec893de930 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.087+0000 7f8f1ea00640 1 -- 172.21.15.62:0/1889539043 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f3003e080 con 0x55ec893de930 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.088+0000 7f8f4776d740 10 client.5802 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.088+0000 7f8f4776d740 10 client.5802 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.088+0000 7f8f4776d740 20 client.5802 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.088+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ec8940c880 0x55ec8940ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.088+0000 7f8f4776d740 4 client.5802 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:51.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.088+0000 7f8f4776d740 1 -- 172.21.15.62:0/1889539043 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec89324230 con 0x55ec8940c880 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.088+0000 7f8f41400640 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ec8940c880 0x55ec8940ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.089+0000 7f8f1ea00640 10 client.5802 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.089+0000 7f8f41400640 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ec8940c880 0x55ec8940ccb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae65333e32868bf server_cookie=55a8e83db1ac3a63 in_seq=0 out_seq=0 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.089+0000 7f8f1ea00640 1 -- 172.21.15.62:0/1889539043 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec89324230 con 0x55ec8940c880 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.089+0000 7f8f1ea00640 10 client.5802 handle_command_reply: tid=0 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.089+0000 7f8f4776d740 1 client.5802 shutdown 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.089+0000 7f8f4776d740 1 -- 172.21.15.62:0/1889539043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893de930 msgr2=0x55ec893f90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.089+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893de930 0x55ec893f90b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000afd0 tx=0x7f8f30004730 comp rx=0 tx=0).stop 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 1 -- 172.21.15.62:0/1889539043 shutdown_connections 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ec8940c880 0x55ec8940ccb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893cb0b0 0x55ec893cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893fb810 0x55ec893f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1889539043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893de930 0x55ec893f90b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f1ea00640 0 client.5802 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 1 -- 172.21.15.62:0/1889539043 >> 172.21.15.62:0/1889539043 conn(0x55ec893de1f0 msgr2=0x55ec893ad830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 1 -- 172.21.15.62:0/1889539043 shutdown_connections 2024-04-25T09:07:51.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 1 -- 172.21.15.62:0/1889539043 wait complete. 2024-04-25T09:07:51.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.090+0000 7f8f4776d740 20 client.5802 trim_cache size 0 max 16384 2024-04-25T09:07:51.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.107+0000 7f8f4776d740 1 Processor -- start 2024-04-25T09:07:51.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.107+0000 7f8f4776d740 1 -- start start 2024-04-25T09:07:51.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.108+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893cb0b0 0x55ec8940bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.108+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.108+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.108+0000 7f8f4776d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec893aeba0 con 0x55ec893fb810 2024-04-25T09:07:51.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.108+0000 7f8f4776d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1ffc60 con 0x55ec893de930 2024-04-25T09:07:51.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.108+0000 7f8f4776d740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1fee70 con 0x55ec893cb0b0 2024-04-25T09:07:51.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893da950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53666/0 (socket says 172.21.15.62:53666) 2024-04-25T09:07:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 -- 172.21.15.62:0/1878831515 learned_addr learned my addr 172.21.15.62:0/1878831515 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f41e00640 1 --2- 172.21.15.62:0/1878831515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893cb0b0 0x55ec8940bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 -- 172.21.15.62:0/1878831515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893cb0b0 msgr2=0x55ec8940bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 --2- 172.21.15.62:0/1878831515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893cb0b0 0x55ec8940bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 -- 172.21.15.62:0/1878831515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 msgr2=0x55ec893da410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 --2- 172.21.15.62:0/1878831515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 -- 172.21.15.62:0/1878831515 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec893cb880 con 0x55ec893fb810 2024-04-25T09:07:51.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.109+0000 7f8f42800640 1 --2- 172.21.15.62:0/1878831515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893da950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8f3802aca0 tx=0x7f8f38079550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6caef38571316cd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.112+0000 7f8f29c00640 1 -- 172.21.15.62:0/1878831515 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38079a10 con 0x55ec893fb810 2024-04-25T09:07:51.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.112+0000 7f8f29c00640 1 -- 172.21.15.62:0/1878831515 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38037040 con 0x55ec893fb810 2024-04-25T09:07:51.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.112+0000 7f8f29c00640 1 -- 172.21.15.62:0/1878831515 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3807dab0 con 0x55ec893fb810 2024-04-25T09:07:51.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 -- 172.21.15.62:0/1878831515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 msgr2=0x55ec893da950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1878831515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893da950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8f3802aca0 tx=0x7f8f38079550 comp rx=0 tx=0).stop 2024-04-25T09:07:51.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 -- 172.21.15.62:0/1878831515 shutdown_connections 2024-04-25T09:07:51.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1878831515 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893fb810 0x55ec893da950 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1878831515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893da410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 --2- 172.21.15.62:0/1878831515 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893cb0b0 0x55ec8940bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 -- 172.21.15.62:0/1878831515 >> 172.21.15.62:0/1878831515 conn(0x55ec893de1f0 msgr2=0x55ec893a96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:51.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 -- 172.21.15.62:0/1878831515 shutdown_connections 2024-04-25T09:07:51.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.113+0000 7f8f4776d740 1 -- 172.21.15.62:0/1878831515 wait complete. 2024-04-25T09:07:51.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.114+0000 7f8f4776d740 1 -- start start 2024-04-25T09:07:51.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.114+0000 7f8f4776d740 1 Processor -- start 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893cb0b0 0x55ec89463660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f4776d740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893fb810 0x55ec893ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f4776d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c15d270 con 0x55ec893cb0b0 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f4776d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1f3d60 con 0x55ec893de930 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f4776d740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec893d9710 con 0x55ec893fb810 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f4776d740 10 client.0 _ll_register_callbacks cb 0x7f8f3000a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f41e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893cb0b0 0x55ec89463660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f41e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893cb0b0 0x55ec89463660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53678/0 (socket says 172.21.15.62:53678) 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.115+0000 7f8f41e00640 1 -- 172.21.15.62:0/2045521453 learned_addr learned my addr 172.21.15.62:0/2045521453 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f41e00640 1 -- 172.21.15.62:0/2045521453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893fb810 msgr2=0x55ec893ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f41e00640 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893fb810 0x55ec893ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f41e00640 1 -- 172.21.15.62:0/2045521453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 msgr2=0x55ec893acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f41e00640 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f41e00640 1 -- 172.21.15.62:0/2045521453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec88fc48b0 con 0x55ec893cb0b0 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f41e00640 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893cb0b0 0x55ec89463660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000a510 tx=0x7f8f30015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75a5e37fe595193 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f1ea00640 1 -- 172.21.15.62:0/2045521453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30044060 con 0x55ec893cb0b0 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f1ea00640 1 -- 172.21.15.62:0/2045521453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f30005da0 con 0x55ec893cb0b0 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.116+0000 7f8f4776d740 1 -- 172.21.15.62:0/2045521453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f38037040 con 0x55ec893cb0b0 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f1ea00640 1 -- 172.21.15.62:0/2045521453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f30006030 con 0x55ec893cb0b0 2024-04-25T09:07:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f4776d740 10 client.5808 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f4776d740 1 -- 172.21.15.62:0/2045521453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55ec8940c010 con 0x55ec893cb0b0 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f1ea00640 1 -- 172.21.15.62:0/2045521453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f38037040 con 0x55ec893cb0b0 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f1ea00640 1 -- 172.21.15.62:0/2045521453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f3003e070 con 0x55ec893cb0b0 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f4776d740 10 client.5808 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f4776d740 10 client.5808 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f4776d740 20 client.5808 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f4776d740 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ec89463e40 0x55ec894642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f4776d740 4 client.5808 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f4776d740 1 -- 172.21.15.62:0/2045521453 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55ec893f2580 con 0x55ec89463e40 2024-04-25T09:07:51.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.117+0000 7f8f41400640 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ec89463e40 0x55ec894642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.118+0000 7f8f41400640 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ec89463e40 0x55ec894642a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34886727fc3802bf server_cookie=5eb17795af7fbb17 in_seq=0 out_seq=0 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.118+0000 7f8f1ea00640 10 client.5808 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.118+0000 7f8f1ea00640 1 -- 172.21.15.62:0/2045521453 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec893f2580 con 0x55ec89463e40 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f1ea00640 10 client.5808 handle_command_reply: tid=0 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 client.5808 shutdown 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 -- 172.21.15.62:0/2045521453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893cb0b0 msgr2=0x55ec89463660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893cb0b0 0x55ec89463660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000a510 tx=0x7f8f30015000 comp rx=0 tx=0).stop 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 -- 172.21.15.62:0/2045521453 shutdown_connections 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ec89463e40 0x55ec894642a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ec893fb810 0x55ec893ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec893de930 0x55ec893acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 --2- 172.21.15.62:0/2045521453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ec893cb0b0 0x55ec89463660 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f1ea00640 0 client.5808 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 -- 172.21.15.62:0/2045521453 >> 172.21.15.62:0/2045521453 conn(0x55ec893de1f0 msgr2=0x55ec893da3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 -- 172.21.15.62:0/2045521453 shutdown_connections 2024-04-25T09:07:51.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 1 -- 172.21.15.62:0/2045521453 wait complete. 2024-04-25T09:07:51.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.119+0000 7f8f4776d740 20 client.5808 trim_cache size 0 max 16384 2024-04-25T09:07:51.110 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:07:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "dad1f5dc-b1a1-4567-b5ca-71fb342e55d3", 2024-04-25T09:07:51.110 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:07:51.110 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 -- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f2006b4d0 msgr2=0x7f8f2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f2006b4d0 0x7f8f2006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000a990 tx=0x7f8f30008040 comp rx=0 tx=0).stop 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 -- 172.21.15.62:0/2160599993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 msgr2=0x7f8f3c156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c156190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8f38034040 tx=0x7f8f38004a40 comp rx=0 tx=0).stop 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 -- 172.21.15.62:0/2160599993 shutdown_connections 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f2006b4d0 0x7f8f2006d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f3c15e7c0 0x7f8f3c156190 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f3c0ceb50 0x7f8f3c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 --2- 172.21.15.62:0/2160599993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8f3c06d800 0x7f8f3c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 -- 172.21.15.62:0/2160599993 >> 172.21.15.62:0/2160599993 conn(0x7f8f3c066f20 msgr2=0x7f8f3c0b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:51.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 -- 172.21.15.62:0/2160599993 shutdown_connections 2024-04-25T09:07:51.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.120+0000 7f8f43200640 1 -- 172.21.15.62:0/2160599993 wait complete. 2024-04-25T09:07:51.126 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'dad1f5dc-b1a1-4567-b5ca-71fb342e55d3', 'mode': 'asynchronous'}' 2024-04-25T09:07:51.126 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T09:07:51.126 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:07:51.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.335+0000 7f3f1a200640 1 Processor -- start 2024-04-25T09:07:51.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.335+0000 7f3f1a200640 1 -- start start 2024-04-25T09:07:51.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.337+0000 7f3f1a200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f1415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.337+0000 7f3f1a200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3f140cee40 0x7f3f140b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.337+0000 7f3f1a200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f140cf2d0 con 0x7f3f14144c10 2024-04-25T09:07:51.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.337+0000 7f3f1a200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f14144ff0 con 0x7f3f140cee40 2024-04-25T09:07:51.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.337+0000 7f3f1a200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14160340 con 0x7f3f1406d900 2024-04-25T09:07:51.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.337+0000 7f3f18e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3f14144c10 0x7f3f1406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53634/0 (socket says 172.21.15.62:53634) 2024-04-25T09:07:51.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.337+0000 7f3f18e00640 1 -- 172.21.15.62:0/3092985149 learned_addr learned my addr 172.21.15.62:0/3092985149 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.338+0000 7f3f13e00640 1 --2- 172.21.15.62:0/3092985149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f1415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.338+0000 7f3f19800640 1 --2- 172.21.15.62:0/3092985149 >> v2:172.21.15.121:3300/0 conn(0x7f3f140cee40 0x7f3f140b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.338+0000 7f3f13400640 1 -- 172.21.15.62:0/3092985149 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307023099 0 0) 0x7f3f140cf2d0 con 0x7f3f14144c10 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.338+0000 7f3f13400640 1 -- 172.21.15.62:0/3092985149 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3efc003520 con 0x7f3f14144c10 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.338+0000 7f3f13e00640 1 -- 172.21.15.62:0/3092985149 >> v2:172.21.15.121:3300/0 conn(0x7f3f140cee40 msgr2=0x7f3f140b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.338+0000 7f3f13e00640 1 --2- 172.21.15.62:0/3092985149 >> v2:172.21.15.121:3300/0 conn(0x7f3f140cee40 0x7f3f140b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.339+0000 7f3f13400640 1 -- 172.21.15.62:0/3092985149 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1479610933 0 0) 0x7f3efc003520 con 0x7f3f14144c10 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.339+0000 7f3f13e00640 1 -- 172.21.15.62:0/3092985149 >> v1:172.21.15.62:6789/0 conn(0x7f3f14144c10 legacy=0x7f3f1406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.339+0000 7f3f13e00640 1 -- 172.21.15.62:0/3092985149 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1415faa0 con 0x7f3f1406d900 2024-04-25T09:07:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.340+0000 7f3f13e00640 1 --2- 172.21.15.62:0/3092985149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f1415f450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f000097a0 tx=0x7f3f0002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc3ae369cb686d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f13400640 1 -- 172.21.15.62:0/3092985149 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00037070 con 0x7f3f1406d900 2024-04-25T09:07:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f13400640 1 -- 172.21.15.62:0/3092985149 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0002ab10 con 0x7f3f1406d900 2024-04-25T09:07:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f13400640 1 -- 172.21.15.62:0/3092985149 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08001670 con 0x7f3f1406d900 2024-04-25T09:07:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f1a200640 1 -- 172.21.15.62:0/3092985149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 msgr2=0x7f3f1415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3092985149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f1415f450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f000097a0 tx=0x7f3f0002a080 comp rx=0 tx=0).stop 2024-04-25T09:07:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f1a200640 1 -- 172.21.15.62:0/3092985149 shutdown_connections 2024-04-25T09:07:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3092985149 >> v2:172.21.15.121:3300/0 conn(0x7f3f140cee40 0x7f3f140b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3092985149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f1415f450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f1a200640 1 -- 172.21.15.62:0/3092985149 >> 172.21.15.62:0/3092985149 conn(0x7f3f140b14d0 msgr2=0x7f3f140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.341+0000 7f3f1a200640 1 -- 172.21.15.62:0/3092985149 shutdown_connections 2024-04-25T09:07:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.342+0000 7f3f1a200640 1 -- 172.21.15.62:0/3092985149 wait complete. 2024-04-25T09:07:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.343+0000 7f3f1a200640 1 Processor -- start 2024-04-25T09:07:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.344+0000 7f3f1a200640 1 -- start start 2024-04-25T09:07:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f1a200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f141f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f1a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f140cee40 0x7f3f14141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f1a200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f14144c10 0x7f3f141433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f1a200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14160060 con 0x7f3f140cee40 2024-04-25T09:07:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f1a200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140b41c0 con 0x7f3f14144c10 2024-04-25T09:07:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f1a200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140b4b90 con 0x7f3f1406d900 2024-04-25T09:07:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f13e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f140cee40 0x7f3f14141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f13e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f140cee40 0x7f3f14141550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53692/0 (socket says 172.21.15.62:53692) 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f13e00640 1 -- 172.21.15.62:0/3266249444 learned_addr learned my addr 172.21.15.62:0/3266249444 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f18e00640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f141f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f19800640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f14144c10 0x7f3f141433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f13e00640 1 -- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 msgr2=0x7f3f141f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f13e00640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f141f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f13e00640 1 -- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f14144c10 msgr2=0x7f3f141433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f13e00640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f14144c10 0x7f3f141433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.345+0000 7f3f13e00640 1 -- 172.21.15.62:0/3266249444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14141fa0 con 0x7f3f140cee40 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.346+0000 7f3f19800640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f14144c10 0x7f3f141433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:51.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.346+0000 7f3f13e00640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f140cee40 0x7f3f14141550 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3f00035040 tx=0x7f3f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.346+0000 7f3f10c00640 1 -- 172.21.15.62:0/3266249444 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00041070 con 0x7f3f140cee40 2024-04-25T09:07:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.346+0000 7f3f10c00640 1 -- 172.21.15.62:0/3266249444 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0003c660 con 0x7f3f140cee40 2024-04-25T09:07:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.346+0000 7f3f10c00640 1 -- 172.21.15.62:0/3266249444 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00037040 con 0x7f3f140cee40 2024-04-25T09:07:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.346+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f14142280 con 0x7f3f140cee40 2024-04-25T09:07:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.347+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f140c5f10 con 0x7f3f140cee40 2024-04-25T09:07:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.348+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f140c53a0 con 0x7f3f140cee40 2024-04-25T09:07:51.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.349+0000 7f3f10c00640 1 -- 172.21.15.62:0/3266249444 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f00044070 con 0x7f3f140cee40 2024-04-25T09:07:51.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.350+0000 7f3f10c00640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef406b4d0 0x7f3ef406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:51.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.350+0000 7f3f10c00640 1 -- 172.21.15.62:0/3266249444 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f000aa9a0 con 0x7f3f140cee40 2024-04-25T09:07:51.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.351+0000 7f3f18e00640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef406b4d0 0x7f3ef406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.352+0000 7f3f18e00640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef406b4d0 0x7f3ef406d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f08005880 tx=0x7f3f08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:51.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.355+0000 7f3f10c00640 1 -- 172.21.15.62:0/3266249444 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f140c53a0 con 0x7f3f140cee40 2024-04-25T09:07:51.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.596+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f1404ef20 con 0x7f3f140cee40 2024-04-25T09:07:51.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.597+0000 7f3f10c00640 1 -- 172.21.15.62:0/3266249444 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f3f00079810 con 0x7f3f140cee40 2024-04-25T09:07:51.585 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:51.585 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:28.504417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:07:51.585 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef406b4d0 msgr2=0x7f3ef406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef406b4d0 0x7f3ef406d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f08005880 tx=0x7f3f08009040 comp rx=0 tx=0).stop 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f140cee40 msgr2=0x7f3f14141550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f140cee40 0x7f3f14141550 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3f00035040 tx=0x7f3f000046c0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 shutdown_connections 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3ef406b4d0 0x7f3ef406d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f140cee40 0x7f3f14141550 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f14144c10 0x7f3f141433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 --2- 172.21.15.62:0/3266249444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3f1406d900 0x7f3f141f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:51.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.600+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 >> 172.21.15.62:0/3266249444 conn(0x7f3f140b14d0 msgr2=0x7f3f14070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:51.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.601+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 shutdown_connections 2024-04-25T09:07:51.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:51.601+0000 7f3f1a200640 1 -- 172.21.15.62:0/3266249444 wait complete. 2024-04-25T09:07:51.607 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-25T09:07:51.815 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:51.815 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:07:51.816 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:51.831 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:07:51.831 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T09:07:51.831 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:51.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T09:07:52.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.054+0000 7f6d81600640 1 Processor -- start 2024-04-25T09:07:52.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.054+0000 7f6d81600640 1 -- start start 2024-04-25T09:07:52.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.056+0000 7f6d81600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ccde0 0x7f6d7c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.056+0000 7f6d81600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7c0ced40 con 0x7f6d7c06d2c0 2024-04-25T09:07:52.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.056+0000 7f6d81600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7c0cd260 con 0x7f6d7c0b31a0 2024-04-25T09:07:52.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.056+0000 7f6d81600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0cf000 con 0x7f6d7c0ccde0 2024-04-25T09:07:52.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.056+0000 7f6d7be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6d7c06d2c0 0x7f6d7c06d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53640/0 (socket says 172.21.15.62:53640) 2024-04-25T09:07:52.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.057+0000 7f6d7be00640 1 -- 172.21.15.62:0/983698820 learned_addr learned my addr 172.21.15.62:0/983698820 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:52.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.057+0000 7f6d73e00640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ccde0 0x7f6d7c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.057+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268616768 0 0) 0x7f6d7c0ced40 con 0x7f6d7c06d2c0 2024-04-25T09:07:52.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.057+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d60003520 con 0x7f6d7c06d2c0 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3471320170 0 0) 0x7f6d60003520 con 0x7f6d7c06d2c0 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ccde0 msgr2=0x7f6d7c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ccde0 0x7f6d7c0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 >> v1:172.21.15.121:6789/0 conn(0x7f6d7c0b31a0 legacy=0x7f6d7c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c0cf2e0 con 0x7f6d7c06d2c0 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f6d6c001670 con 0x7f6d7c06d2c0 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 >> v1:172.21.15.62:6789/0 conn(0x7f6d7c06d2c0 legacy=0x7f6d7c06d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d600039e0 0x7f6d60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ccde0 0x7f6d60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d60004d40 0x7f6d600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0cf000 con 0x7f6d600039e0 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6000ba70 con 0x7f6d60004d40 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6000bc70 con 0x7f6d7c0ccde0 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7be00640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d600039e0 0x7f6d60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d7be00640 1 -- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ccde0 msgr2=0x7f6d60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.059+0000 7f6d73e00640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ccde0 0x7f6d60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.058+0000 7f6d80c00640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d60004d40 0x7f6d600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.059+0000 7f6d7be00640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d7c0ccde0 0x7f6d60004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.059+0000 7f6d7be00640 1 -- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d60004d40 msgr2=0x7f6d600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.059+0000 7f6d7be00640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d60004d40 0x7f6d600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.059+0000 7f6d7be00640 1 -- 172.21.15.62:0/983698820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d60003520 con 0x7f6d600039e0 2024-04-25T09:07:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.059+0000 7f6d80c00640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d60004d40 0x7f6d600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.060+0000 7f6d7be00640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d600039e0 0x7f6d60003e00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c001eb0 tx=0x7f6d60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7975d189e649f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.060+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c007d30 con 0x7f6d600039e0 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.060+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c0058d0 con 0x7f6d600039e0 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.060+0000 7f6d7b400640 1 -- 172.21.15.62:0/983698820 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c005ce0 con 0x7f6d600039e0 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.061+0000 7f6d81600640 1 -- 172.21.15.62:0/983698820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d600039e0 msgr2=0x7f6d60003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.061+0000 7f6d81600640 1 --2- 172.21.15.62:0/983698820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d600039e0 0x7f6d60003e00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c001eb0 tx=0x7f6d60005830 comp rx=0 tx=0).stop 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.061+0000 7f6d80c00640 1 -- 172.21.15.62:0/983698820 reap_dead start 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.061+0000 7f6d81600640 1 -- 172.21.15.62:0/983698820 shutdown_connections 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.061+0000 7f6d81600640 1 -- 172.21.15.62:0/983698820 >> 172.21.15.62:0/983698820 conn(0x7f6d7c066f20 msgr2=0x7f6d7c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.061+0000 7f6d81600640 1 -- 172.21.15.62:0/983698820 shutdown_connections 2024-04-25T09:07:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.061+0000 7f6d81600640 1 -- 172.21.15.62:0/983698820 wait complete. 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.063+0000 7f6d81600640 1 Processor -- start 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.063+0000 7f6d81600640 1 -- start start 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d81600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d600039e0 0x7f6d7c133210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d81600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c131740 0x7f6d7c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d81600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c131db0 0x7f6d7c132230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d81600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0b2c80 con 0x7f6d7c131740 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d81600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c0b29a0 con 0x7f6d7c131db0 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d81600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c1448d0 con 0x7f6d600039e0 2024-04-25T09:07:52.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c131740 0x7f6d7c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c131740 0x7f6d7c0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53716/0 (socket says 172.21.15.62:53716) 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.064+0000 7f6d73e00640 1 -- 172.21.15.62:0/2305041679 learned_addr learned my addr 172.21.15.62:0/2305041679 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.065+0000 7f6d7be00640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d600039e0 0x7f6d7c133210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.065+0000 7f6d80c00640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c131db0 0x7f6d7c132230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.065+0000 7f6d73e00640 1 -- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d600039e0 msgr2=0x7f6d7c133210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.065+0000 7f6d73e00640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d600039e0 0x7f6d7c133210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.065+0000 7f6d73e00640 1 -- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c131db0 msgr2=0x7f6d7c132230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.065+0000 7f6d73e00640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c131db0 0x7f6d7c132230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.065+0000 7f6d73e00640 1 -- 172.21.15.62:0/2305041679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c1436c0 con 0x7f6d7c131740 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.065+0000 7f6d73e00640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c131740 0x7f6d7c0cadc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6d64010860 tx=0x7f6d64010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.066+0000 7f6d78c00640 1 -- 172.21.15.62:0/2305041679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400f040 con 0x7f6d7c131740 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.066+0000 7f6d78c00640 1 -- 172.21.15.62:0/2305041679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400eb00 con 0x7f6d7c131740 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.066+0000 7f6d81600640 1 -- 172.21.15.62:0/2305041679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c1337f0 con 0x7f6d7c131740 2024-04-25T09:07:52.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.066+0000 7f6d78c00640 1 -- 172.21.15.62:0/2305041679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6401c400 con 0x7f6d7c131740 2024-04-25T09:07:52.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.066+0000 7f6d81600640 1 -- 172.21.15.62:0/2305041679 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c133db0 con 0x7f6d7c131740 2024-04-25T09:07:52.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.068+0000 7f6d78c00640 1 -- 172.21.15.62:0/2305041679 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d6401c5a0 con 0x7f6d7c131740 2024-04-25T09:07:52.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.070+0000 7f6d78c00640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6d5806b500 0x7f6d5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.070+0000 7f6d78c00640 1 -- 172.21.15.62:0/2305041679 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d6408a7a0 con 0x7f6d7c131740 2024-04-25T09:07:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.071+0000 7f6d7be00640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6d5806b500 0x7f6d5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.071+0000 7f6d85bba740 1 Processor -- start 2024-04-25T09:07:52.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.071+0000 7f6d85bba740 1 -- start start 2024-04-25T09:07:52.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526d3b80 0x563452686510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6560 0x56345268b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d85bba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634525ef880 con 0x5634526d3b80 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d85bba740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563452598d00 con 0x5634526b6560 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d85bba740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634525eb920 con 0x5634526b6ca0 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6d5806b500 0x7f6d5806d9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c033000 tx=0x7f6d6c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526d3b80 0x563452686510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526d3b80 0x563452686510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53728/0 (socket says 172.21.15.62:53728) 2024-04-25T09:07:52.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 -- 172.21.15.62:0/1519761535 learned_addr learned my addr 172.21.15.62:0/1519761535 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 -- 172.21.15.62:0/1519761535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 msgr2=0x5634526b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 --2- 172.21.15.62:0/1519761535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 -- 172.21.15.62:0/1519761535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6560 msgr2=0x56345268b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d80c00640 1 --2- 172.21.15.62:0/1519761535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6560 0x56345268b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 --2- 172.21.15.62:0/1519761535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6560 0x56345268b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.072+0000 7f6d7be00640 1 -- 172.21.15.62:0/1519761535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634525a51f0 con 0x5634526d3b80 2024-04-25T09:07:52.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.073+0000 7f6d7be00640 1 --2- 172.21.15.62:0/1519761535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526d3b80 0x563452686510 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c2d0 tx=0x7f6d6c009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a60a1b892b18c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.073+0000 7f6d6be00640 1 -- 172.21.15.62:0/1519761535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c043030 con 0x5634526d3b80 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.073+0000 7f6d6be00640 1 -- 172.21.15.62:0/1519761535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c039040 con 0x5634526d3b80 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.073+0000 7f6d6be00640 1 -- 172.21.15.62:0/1519761535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c004c70 con 0x5634526d3b80 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 -- 172.21.15.62:0/1519761535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526d3b80 msgr2=0x563452686510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 --2- 172.21.15.62:0/1519761535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526d3b80 0x563452686510 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c2d0 tx=0x7f6d6c009030 comp rx=0 tx=0).stop 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 -- 172.21.15.62:0/1519761535 shutdown_connections 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 --2- 172.21.15.62:0/1519761535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6560 0x56345268b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 --2- 172.21.15.62:0/1519761535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 --2- 172.21.15.62:0/1519761535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526d3b80 0x563452686510 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 -- 172.21.15.62:0/1519761535 >> 172.21.15.62:0/1519761535 conn(0x563452688780 msgr2=0x5634526d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 -- 172.21.15.62:0/1519761535 shutdown_connections 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.074+0000 7f6d85bba740 1 -- 172.21.15.62:0/1519761535 wait complete. 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.075+0000 7f6d85bba740 1 -- start start 2024-04-25T09:07:52.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.075+0000 7f6d85bba740 1 Processor -- start 2024-04-25T09:07:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 0x5634526a4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d85bba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345268a930 con 0x5634526b9030 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d85bba740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56345258e1c0 con 0x5634526d3b80 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d85bba740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x563452689d20 con 0x5634526b6ca0 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d85bba740 10 client.0 _ll_register_callbacks cb 0x5634525abb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d80c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d80c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53742/0 (socket says 172.21.15.62:53742) 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.076+0000 7f6d80c00640 1 -- 172.21.15.62:0/3510550 learned_addr learned my addr 172.21.15.62:0/3510550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d7be00640 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d73e00640 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 0x5634526a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d80c00640 1 -- 172.21.15.62:0/3510550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 msgr2=0x5634526a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d80c00640 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d80c00640 1 -- 172.21.15.62:0/3510550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 msgr2=0x5634526a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d80c00640 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 0x5634526a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d80c00640 1 -- 172.21.15.62:0/3510550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634523d0690 con 0x5634526b9030 2024-04-25T09:07:52.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d80c00640 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526c8730 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c06e560 tx=0x7f6d74010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04199c724aa9e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d68c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.077+0000 7f6d68c00640 1 -- 172.21.15.62:0/3510550 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74003d80 con 0x5634526b9030 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d68c00640 1 -- 172.21.15.62:0/3510550 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d740050d0 con 0x5634526b9030 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 1 -- 172.21.15.62:0/3510550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d6c039040 con 0x5634526b9030 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d68c00640 1 -- 172.21.15.62:0/3510550 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d74021050 con 0x5634526b9030 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 10 client.5826 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 1 -- 172.21.15.62:0/3510550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x563452430e20 con 0x5634526b9030 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d68c00640 1 -- 172.21.15.62:0/3510550 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c039040 con 0x5634526b9030 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d68c00640 1 -- 172.21.15.62:0/3510550 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d74024020 con 0x5634526b9030 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 10 client.5826 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 10 client.5826 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 20 client.5826 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5634526d23a0 0x5634526d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 4 client.5826 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.078+0000 7f6d85bba740 1 -- 172.21.15.62:0/3510550 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634525fcdc0 con 0x5634526d23a0 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.079+0000 7f6d7be00640 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5634526d23a0 0x5634526d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.079+0000 7f6d7be00640 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5634526d23a0 0x5634526d27b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=992e0e9058c68ff0 server_cookie=85fd7b1df14845a8 in_seq=0 out_seq=0 2024-04-25T09:07:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.079+0000 7f6d68c00640 10 client.5826 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d68c00640 1 -- 172.21.15.62:0/3510550 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634525fcdc0 con 0x5634526d23a0 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d68c00640 10 client.5826 handle_command_reply: tid=0 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 client.5826 shutdown 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 -- 172.21.15.62:0/3510550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 msgr2=0x5634526c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526c8730 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c06e560 tx=0x7f6d74010980 comp rx=0 tx=0).stop 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 -- 172.21.15.62:0/3510550 shutdown_connections 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5634526d23a0 0x5634526d27b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526c8730 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 0x5634526a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3510550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d68c00640 0 client.5826 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.080+0000 7f6d85bba740 1 -- 172.21.15.62:0/3510550 >> 172.21.15.62:0/3510550 conn(0x5634526b6560 msgr2=0x563452687250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.081+0000 7f6d85bba740 1 -- 172.21.15.62:0/3510550 shutdown_connections 2024-04-25T09:07:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.081+0000 7f6d85bba740 1 -- 172.21.15.62:0/3510550 wait complete. 2024-04-25T09:07:52.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.081+0000 7f6d85bba740 20 client.5826 trim_cache size 0 max 16384 2024-04-25T09:07:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.097+0000 7f6d85bba740 1 Processor -- start 2024-04-25T09:07:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.098+0000 7f6d85bba740 1 -- start start 2024-04-25T09:07:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.098+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.098+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526df720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 0x5634526dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d85bba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c200680 con 0x5634526b9030 2024-04-25T09:07:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d85bba740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c1357c0 con 0x5634526d3b80 2024-04-25T09:07:52.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d85bba740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c1e8b90 con 0x5634526b6ca0 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526df720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526df720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53756/0 (socket says 172.21.15.62:53756) 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d73e00640 1 -- 172.21.15.62:0/3667426527 learned_addr learned my addr 172.21.15.62:0/3667426527 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d80c00640 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 0x5634526dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d7be00640 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d73e00640 1 -- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 msgr2=0x5634526b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d73e00640 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d73e00640 1 -- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 msgr2=0x5634526dfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d73e00640 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 0x5634526dfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d73e00640 1 -- 172.21.15.62:0/3667426527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56345268d8c0 con 0x5634526b9030 2024-04-25T09:07:52.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.099+0000 7f6d7be00640 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.100+0000 7f6d73e00640 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526df720 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d640552e0 tx=0x7f6d64058a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56ab1118976f16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.100+0000 7f6d6b400640 1 -- 172.21.15.62:0/3667426527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64087070 con 0x5634526b9030 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.100+0000 7f6d6b400640 1 -- 172.21.15.62:0/3667426527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d64058e80 con 0x5634526b9030 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.100+0000 7f6d6b400640 1 -- 172.21.15.62:0/3667426527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64070070 con 0x5634526b9030 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 -- 172.21.15.62:0/3667426527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 msgr2=0x5634526df720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526df720 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6d640552e0 tx=0x7f6d64058a90 comp rx=0 tx=0).stop 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 -- 172.21.15.62:0/3667426527 shutdown_connections 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x5634526df720 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526d3b80 0x5634526dfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 --2- 172.21.15.62:0/3667426527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526b6ca0 0x5634526b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 -- 172.21.15.62:0/3667426527 >> 172.21.15.62:0/3667426527 conn(0x5634526b6560 msgr2=0x563452687250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 -- 172.21.15.62:0/3667426527 shutdown_connections 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 -- 172.21.15.62:0/3667426527 wait complete. 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.101+0000 7f6d85bba740 1 -- start start 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.102+0000 7f6d85bba740 1 Processor -- start 2024-04-25T09:07:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6ca0 0x5634527363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x56345273c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d85bba740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526d3b80 0x5634526b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d85bba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634526b1440 con 0x5634526b9030 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d85bba740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c1351f0 con 0x5634526b6ca0 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d85bba740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634526f3ac0 con 0x5634526d3b80 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d85bba740 10 client.0 _ll_register_callbacks cb 0x7f6d74007ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x56345273c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x56345273c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53768/0 (socket says 172.21.15.62:53768) 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d73e00640 1 -- 172.21.15.62:0/4256252926 learned_addr learned my addr 172.21.15.62:0/4256252926 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d73e00640 1 -- 172.21.15.62:0/4256252926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526d3b80 msgr2=0x5634526b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d80c00640 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526d3b80 0x5634526b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d73e00640 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526d3b80 0x5634526b7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d7be00640 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6ca0 0x5634527363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d73e00640 1 -- 172.21.15.62:0/4256252926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6ca0 msgr2=0x5634527363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d73e00640 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6ca0 0x5634527363a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.103+0000 7f6d73e00640 1 -- 172.21.15.62:0/4256252926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634522c3f80 con 0x5634526b9030 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.104+0000 7f6d73e00640 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x56345273c2b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e540 tx=0x7f6d64010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66e6b273cc3fed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.104+0000 7f6d68c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.104+0000 7f6d68c00640 1 -- 172.21.15.62:0/4256252926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64074020 con 0x5634526b9030 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.104+0000 7f6d68c00640 1 -- 172.21.15.62:0/4256252926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d64061100 con 0x5634526b9030 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.104+0000 7f6d85bba740 1 -- 172.21.15.62:0/4256252926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d64058e80 con 0x5634526b9030 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.104+0000 7f6d68c00640 1 -- 172.21.15.62:0/4256252926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d6407e050 con 0x5634526b9030 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.104+0000 7f6d85bba740 10 client.5832 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.104+0000 7f6d85bba740 1 -- 172.21.15.62:0/4256252926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x5634526a3180 con 0x5634526b9030 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d68c00640 1 -- 172.21.15.62:0/4256252926 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d64058e80 con 0x5634526b9030 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d68c00640 1 -- 172.21.15.62:0/4256252926 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d64081020 con 0x5634526b9030 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d85bba740 10 client.5832 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:52.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d85bba740 10 client.5832 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d85bba740 20 client.5832 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d85bba740 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5634526b76d0 0x5634526c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d85bba740 4 client.5832 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d85bba740 1 -- 172.21.15.62:0/4256252926 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x5634525f7110 con 0x5634526b76d0 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.105+0000 7f6d7be00640 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5634526b76d0 0x5634526c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d7be00640 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5634526b76d0 0x5634526c6f70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e960250f28c524db server_cookie=dcc8c30e0ddc40a7 in_seq=0 out_seq=0 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d68c00640 10 client.5832 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d68c00640 1 -- 172.21.15.62:0/4256252926 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5634525f7110 con 0x5634526b76d0 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d68c00640 10 client.5832 handle_command_reply: tid=0 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 client.5832 shutdown 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 -- 172.21.15.62:0/4256252926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 msgr2=0x56345273c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x56345273c2b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e540 tx=0x7f6d64010fd0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 -- 172.21.15.62:0/4256252926 shutdown_connections 2024-04-25T09:07:52.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5634526b76d0 0x5634526c6f70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5634526b9030 0x56345273c2b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5634526d3b80 0x5634526b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 --2- 172.21.15.62:0/4256252926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5634526b6ca0 0x5634527363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d85bba740 1 -- 172.21.15.62:0/4256252926 >> 172.21.15.62:0/4256252926 conn(0x5634526b6560 msgr2=0x563452685a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.106+0000 7f6d68c00640 0 client.5832 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d85bba740 1 -- 172.21.15.62:0/4256252926 shutdown_connections 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d85bba740 1 -- 172.21.15.62:0/4256252926 wait complete. 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d85bba740 20 client.5832 trim_cache size 0 max 16384 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "0819cef4-bb62-488e-99eb-f775ed42a119", 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:07:52.101 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:52.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d81600640 1 -- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6d5806b500 msgr2=0x7f6d5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d81600640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6d5806b500 0x7f6d5806d9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c033000 tx=0x7f6d6c02e000 comp rx=0 tx=0).stop 2024-04-25T09:07:52.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d81600640 1 -- 172.21.15.62:0/2305041679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c131740 msgr2=0x7f6d7c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d81600640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c131740 0x7f6d7c0cadc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6d64010860 tx=0x7f6d64010d20 comp rx=0 tx=0).stop 2024-04-25T09:07:52.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d81600640 1 -- 172.21.15.62:0/2305041679 shutdown_connections 2024-04-25T09:07:52.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.107+0000 7f6d81600640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6d5806b500 0x7f6d5806d9c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.108+0000 7f6d81600640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d7c131740 0x7f6d7c0cadc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.108+0000 7f6d81600640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c131db0 0x7f6d7c132230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.108+0000 7f6d81600640 1 --2- 172.21.15.62:0/2305041679 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6d600039e0 0x7f6d7c133210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.108+0000 7f6d81600640 1 -- 172.21.15.62:0/2305041679 >> 172.21.15.62:0/2305041679 conn(0x7f6d7c066f20 msgr2=0x7f6d7c06c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.108+0000 7f6d81600640 1 -- 172.21.15.62:0/2305041679 shutdown_connections 2024-04-25T09:07:52.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.108+0000 7f6d81600640 1 -- 172.21.15.62:0/2305041679 wait complete. 2024-04-25T09:07:52.114 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '0819cef4-bb62-488e-99eb-f775ed42a119', 'mode': 'asynchronous'}' 2024-04-25T09:07:52.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:52.114 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T09:07:52.172 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T09:07:52.172 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.394+0000 7f2e69200640 1 Processor -- start 2024-04-25T09:07:52.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.394+0000 7f2e69200640 1 -- start start 2024-04-25T09:07:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.396+0000 7f2e69200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b56f0 0x7f2e64160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.396+0000 7f2e69200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e640b27a0 0x7f2e64161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.396+0000 7f2e69200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e640d3110 con 0x7f2e640b6a90 2024-04-25T09:07:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.396+0000 7f2e69200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e64161780 con 0x7f2e640b27a0 2024-04-25T09:07:52.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.396+0000 7f2e69200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640cedc0 con 0x7f2e640b56f0 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e5aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2e640b6a90 0x7f2e64161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53648/0 (socket says 172.21.15.62:53648) 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e5aa00640 1 -- 172.21.15.62:0/278722907 learned_addr learned my addr 172.21.15.62:0/278722907 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e63e00640 1 --2- 172.21.15.62:0/278722907 >> v2:172.21.15.62:3300/0 conn(0x7f2e640b27a0 0x7f2e64161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e63400640 1 --2- 172.21.15.62:0/278722907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b56f0 0x7f2e64160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e62a00640 1 -- 172.21.15.62:0/278722907 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142825122 0 0) 0x7f2e640d3110 con 0x7f2e640b6a90 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e62a00640 1 -- 172.21.15.62:0/278722907 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e48003520 con 0x7f2e640b6a90 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e63e00640 1 -- 172.21.15.62:0/278722907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b56f0 msgr2=0x7f2e64160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e63e00640 1 --2- 172.21.15.62:0/278722907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b56f0 0x7f2e64160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e63e00640 1 -- 172.21.15.62:0/278722907 >> v1:172.21.15.62:6789/0 conn(0x7f2e640b6a90 legacy=0x7f2e64161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e62a00640 1 -- 172.21.15.62:0/278722907 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892966227 0 0) 0x7f2e48003520 con 0x7f2e640b6a90 2024-04-25T09:07:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.397+0000 7f2e63e00640 1 -- 172.21.15.62:0/278722907 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e640d33f0 con 0x7f2e640b27a0 2024-04-25T09:07:52.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.399+0000 7f2e63e00640 1 --2- 172.21.15.62:0/278722907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e64161240 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0097a0 tx=0x7f2e5c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbbd38e618dd343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.399+0000 7f2e62a00640 1 -- 172.21.15.62:0/278722907 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c037070 con 0x7f2e640b27a0 2024-04-25T09:07:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.399+0000 7f2e62a00640 1 -- 172.21.15.62:0/278722907 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c02aba0 con 0x7f2e640b27a0 2024-04-25T09:07:52.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.399+0000 7f2e62a00640 1 -- 172.21.15.62:0/278722907 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c001670 con 0x7f2e640b27a0 2024-04-25T09:07:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.400+0000 7f2e69200640 1 -- 172.21.15.62:0/278722907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 msgr2=0x7f2e64161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.400+0000 7f2e69200640 1 --2- 172.21.15.62:0/278722907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e64161240 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0097a0 tx=0x7f2e5c02a110 comp rx=0 tx=0).stop 2024-04-25T09:07:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.400+0000 7f2e69200640 1 -- 172.21.15.62:0/278722907 shutdown_connections 2024-04-25T09:07:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.400+0000 7f2e69200640 1 --2- 172.21.15.62:0/278722907 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e64161240 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.400+0000 7f2e69200640 1 --2- 172.21.15.62:0/278722907 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b56f0 0x7f2e64160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.400+0000 7f2e69200640 1 -- 172.21.15.62:0/278722907 >> 172.21.15.62:0/278722907 conn(0x7f2e64066d10 msgr2=0x7f2e64067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.400+0000 7f2e69200640 1 -- 172.21.15.62:0/278722907 shutdown_connections 2024-04-25T09:07:52.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.400+0000 7f2e69200640 1 -- 172.21.15.62:0/278722907 wait complete. 2024-04-25T09:07:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.402+0000 7f2e69200640 1 Processor -- start 2024-04-25T09:07:52.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.402+0000 7f2e69200640 1 -- start start 2024-04-25T09:07:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e69200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e6414ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e69200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e640b56f0 0x7f2e64158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e69200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b6a90 0x7f2e640c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e69200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640b2d60 con 0x7f2e640b27a0 2024-04-25T09:07:52.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e69200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640cea50 con 0x7f2e640b56f0 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e69200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640b2ee0 con 0x7f2e640b6a90 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e63400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e6414ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e63400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e6414ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53774/0 (socket says 172.21.15.62:53774) 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.403+0000 7f2e63400640 1 -- 172.21.15.62:0/839275395 learned_addr learned my addr 172.21.15.62:0/839275395 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.404+0000 7f2e63400640 1 -- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b6a90 msgr2=0x7f2e640c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.404+0000 7f2e63400640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b6a90 0x7f2e640c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.404+0000 7f2e63400640 1 -- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e640b56f0 msgr2=0x7f2e64158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.404+0000 7f2e63400640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e640b56f0 0x7f2e64158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.404+0000 7f2e63400640 1 -- 172.21.15.62:0/839275395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64158bc0 con 0x7f2e640b27a0 2024-04-25T09:07:52.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.404+0000 7f2e63400640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e6414ce30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009db0 tx=0x7f2e5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.404+0000 7f2e5be00640 1 -- 172.21.15.62:0/839275395 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54018020 con 0x7f2e640b27a0 2024-04-25T09:07:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.405+0000 7f2e5be00640 1 -- 172.21.15.62:0/839275395 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e54014480 con 0x7f2e640b27a0 2024-04-25T09:07:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.405+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e64158e90 con 0x7f2e640b27a0 2024-04-25T09:07:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.405+0000 7f2e5be00640 1 -- 172.21.15.62:0/839275395 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54014850 con 0x7f2e640b27a0 2024-04-25T09:07:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.405+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e64159450 con 0x7f2e640b27a0 2024-04-25T09:07:52.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.406+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6404ef20 con 0x7f2e640b27a0 2024-04-25T09:07:52.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.407+0000 7f2e5be00640 1 -- 172.21.15.62:0/839275395 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e54005d10 con 0x7f2e640b27a0 2024-04-25T09:07:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.408+0000 7f2e5be00640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e4006b500 0x7f2e4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.408+0000 7f2e5be00640 1 -- 172.21.15.62:0/839275395 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e5408ae00 con 0x7f2e640b27a0 2024-04-25T09:07:52.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.408+0000 7f2e5aa00640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e4006b500 0x7f2e4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:52.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.409+0000 7f2e5aa00640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e4006b500 0x7f2e4006d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2e6415a7f0 tx=0x7f2e4c001180 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.412+0000 7f2e5be00640 1 -- 172.21.15.62:0/839275395 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e54022070 con 0x7f2e640b27a0 2024-04-25T09:07:52.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.651+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e64074110 con 0x7f2e640b27a0 2024-04-25T09:07:52.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.652+0000 7f2e5be00640 1 -- 172.21.15.62:0/839275395 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f2e54010040 con 0x7f2e640b27a0 2024-04-25T09:07:52.640 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:52.640 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:28.504417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:07:52.641 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-04-25T09:07:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e4006b500 msgr2=0x7f2e4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e4006b500 0x7f2e4006d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2e6415a7f0 tx=0x7f2e4c001180 comp rx=0 tx=0).stop 2024-04-25T09:07:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 msgr2=0x7f2e6414ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e6414ce30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009db0 tx=0x7f2e5400c490 comp rx=0 tx=0).stop 2024-04-25T09:07:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 shutdown_connections 2024-04-25T09:07:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e4006b500 0x7f2e4006d9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e640b6a90 0x7f2e640c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e640b56f0 0x7f2e64158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 --2- 172.21.15.62:0/839275395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e640b27a0 0x7f2e6414ce30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 >> 172.21.15.62:0/839275395 conn(0x7f2e64066d10 msgr2=0x7f2e640d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 shutdown_connections 2024-04-25T09:07:52.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:52.655+0000 7f2e69200640 1 -- 172.21.15.62:0/839275395 wait complete. 2024-04-25T09:07:52.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T09:07:52.875 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:52.875 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:07:52.876 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:52.891 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:07:52.891 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:07:52.892 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:07:52.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:07:52.946 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T09:07:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:07:53.029 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:07:53.030 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294926 s, 35.6 MB/s 2024-04-25T09:07:53.031 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T09:07:53.031 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:07:53.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.254+0000 7f457f200640 1 Processor -- start 2024-04-25T09:07:53.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.255+0000 7f457f200640 1 -- start start 2024-04-25T09:07:53.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.256+0000 7f457f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f45800b56f0 0x7f4580160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.256+0000 7f457f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f45800b27a0 0x7f4580161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.256+0000 7f457f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45800d3110 con 0x7f45800b6a90 2024-04-25T09:07:53.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.256+0000 7f457f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4580161780 con 0x7f45800b56f0 2024-04-25T09:07:53.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.256+0000 7f457f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45800cedc0 con 0x7f45800b27a0 2024-04-25T09:07:53.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.256+0000 7f457de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f45800b56f0 0x7f4580160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.256+0000 7f457d400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f45800b6a90 0x7f4580161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53650/0 (socket says 172.21.15.62:53650) 2024-04-25T09:07:53.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.256+0000 7f457d400640 1 -- 172.21.15.62:0/3740786664 learned_addr learned my addr 172.21.15.62:0/3740786664 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:53.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.257+0000 7f457e800640 1 --2- 172.21.15.62:0/3740786664 >> v2:172.21.15.121:3300/0 conn(0x7f45800b27a0 0x7f4580161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.257+0000 7f457ca00640 1 -- 172.21.15.62:0/3740786664 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768266756 0 0) 0x7f45800d3110 con 0x7f45800b6a90 2024-04-25T09:07:53.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.257+0000 7f457ca00640 1 -- 172.21.15.62:0/3740786664 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4564003520 con 0x7f45800b6a90 2024-04-25T09:07:53.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.257+0000 7f457de00640 1 -- 172.21.15.62:0/3740786664 >> v2:172.21.15.121:3300/0 conn(0x7f45800b27a0 msgr2=0x7f4580161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:53.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.257+0000 7f457de00640 1 --2- 172.21.15.62:0/3740786664 >> v2:172.21.15.121:3300/0 conn(0x7f45800b27a0 0x7f4580161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.257+0000 7f457de00640 1 -- 172.21.15.62:0/3740786664 >> v1:172.21.15.62:6789/0 conn(0x7f45800b6a90 legacy=0x7f4580161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.257+0000 7f457de00640 1 -- 172.21.15.62:0/3740786664 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45800d33f0 con 0x7f45800b56f0 2024-04-25T09:07:53.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.259+0000 7f457de00640 1 --2- 172.21.15.62:0/3740786664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b56f0 0x7f4580160d00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f45800b2720 tx=0x7f457002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce27ce41f6321f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.259+0000 7f457ca00640 1 -- 172.21.15.62:0/3740786664 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570007740 con 0x7f45800b56f0 2024-04-25T09:07:53.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.259+0000 7f457ca00640 1 -- 172.21.15.62:0/3740786664 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45700078e0 con 0x7f45800b56f0 2024-04-25T09:07:53.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.259+0000 7f457ca00640 1 -- 172.21.15.62:0/3740786664 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570007cb0 con 0x7f45800b56f0 2024-04-25T09:07:53.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.260+0000 7f457f200640 1 -- 172.21.15.62:0/3740786664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b56f0 msgr2=0x7f4580160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.260+0000 7f457f200640 1 --2- 172.21.15.62:0/3740786664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b56f0 0x7f4580160d00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f45800b2720 tx=0x7f457002a680 comp rx=0 tx=0).stop 2024-04-25T09:07:53.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.260+0000 7f457f200640 1 -- 172.21.15.62:0/3740786664 shutdown_connections 2024-04-25T09:07:53.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.260+0000 7f457f200640 1 --2- 172.21.15.62:0/3740786664 >> v2:172.21.15.121:3300/0 conn(0x7f45800b27a0 0x7f4580161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.260+0000 7f457f200640 1 --2- 172.21.15.62:0/3740786664 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b56f0 0x7f4580160d00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.260+0000 7f457f200640 1 -- 172.21.15.62:0/3740786664 >> 172.21.15.62:0/3740786664 conn(0x7f4580066d10 msgr2=0x7f4580067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:53.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.260+0000 7f457f200640 1 -- 172.21.15.62:0/3740786664 shutdown_connections 2024-04-25T09:07:53.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.260+0000 7f457f200640 1 -- 172.21.15.62:0/3740786664 wait complete. 2024-04-25T09:07:53.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.262+0000 7f457f200640 1 Processor -- start 2024-04-25T09:07:53.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.262+0000 7f457f200640 1 -- start start 2024-04-25T09:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b27a0 0x7f45801f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45800b56f0 0x7f45800c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45800b6a90 0x7f458015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45800b2d60 con 0x7f45800b27a0 2024-04-25T09:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45800cea50 con 0x7f45800b56f0 2024-04-25T09:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45800b2ee0 con 0x7f45800b6a90 2024-04-25T09:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b27a0 0x7f45801f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b27a0 0x7f45801f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53802/0 (socket says 172.21.15.62:53802) 2024-04-25T09:07:53.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.263+0000 7f457de00640 1 -- 172.21.15.62:0/2583304638 learned_addr learned my addr 172.21.15.62:0/2583304638 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457e800640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45800b6a90 0x7f458015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457de00640 1 -- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45800b6a90 msgr2=0x7f458015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457d400640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45800b56f0 0x7f45800c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457de00640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45800b6a90 0x7f458015bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457de00640 1 -- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45800b56f0 msgr2=0x7f45800c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457de00640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45800b56f0 0x7f45800c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457de00640 1 -- 172.21.15.62:0/2583304638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45800c9670 con 0x7f45800b27a0 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457e800640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45800b6a90 0x7f458015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457de00640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b27a0 0x7f45801f4bf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4570009e80 tx=0x7f4570004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f4576000640 1 -- 172.21.15.62:0/2583304638 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570040070 con 0x7f45800b27a0 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f4576000640 1 -- 172.21.15.62:0/2583304638 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4570031c40 con 0x7f45800b27a0 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45801f3be0 con 0x7f45800b27a0 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.264+0000 7f4576000640 1 -- 172.21.15.62:0/2583304638 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570009450 con 0x7f45800b27a0 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.265+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45801f41a0 con 0x7f45800b27a0 2024-04-25T09:07:53.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.266+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458004ef20 con 0x7f45800b27a0 2024-04-25T09:07:53.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.267+0000 7f4576000640 1 -- 172.21.15.62:0/2583304638 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4570039040 con 0x7f45800b27a0 2024-04-25T09:07:53.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.268+0000 7f4576000640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f455c06b4d0 0x7f455c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:53.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.268+0000 7f4576000640 1 -- 172.21.15.62:0/2583304638 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45700ab750 con 0x7f45800b27a0 2024-04-25T09:07:53.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.268+0000 7f457d400640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f455c06b4d0 0x7f455c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:53.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.269+0000 7f457d400640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f455c06b4d0 0x7f455c06d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f45800b7140 tx=0x7f4568009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:53.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.272+0000 7f4576000640 1 -- 172.21.15.62:0/2583304638 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f457003a3b0 con 0x7f45800b27a0 2024-04-25T09:07:53.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.518+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45800ce580 con 0x7f45800b27a0 2024-04-25T09:07:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.519+0000 7f4576000640 1 -- 172.21.15.62:0/2583304638 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f457007a5c0 con 0x7f45800b27a0 2024-04-25T09:07:53.507 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:07:53.507 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:28.504417+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:07:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 26 2024-04-25T09:07:53.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f455c06b4d0 msgr2=0x7f455c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f455c06b4d0 0x7f455c06d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f45800b7140 tx=0x7f4568009040 comp rx=0 tx=0).stop 2024-04-25T09:07:53.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b27a0 msgr2=0x7f45801f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b27a0 0x7f45801f4bf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4570009e80 tx=0x7f4570004a40 comp rx=0 tx=0).stop 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 shutdown_connections 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f455c06b4d0 0x7f455c06d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f45800b6a90 0x7f458015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45800b56f0 0x7f45800c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 --2- 172.21.15.62:0/2583304638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45800b27a0 0x7f45801f4bf0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 >> 172.21.15.62:0/2583304638 conn(0x7f4580066d10 msgr2=0x7f45800d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 shutdown_connections 2024-04-25T09:07:53.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53.522+0000 7f457f200640 1 -- 172.21.15.62:0/2583304638 wait complete. 2024-04-25T09:07:53.528 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T09:07:53.745 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:53.746 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:07:53.746 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:53.762 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:07:53.762 DEBUG:teuthology.orchestra.run.smithi062:> stdin-killer -- python3 -c ' 2024-04-25T09:07:53.763 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-04-25T09:07:53.763 DEBUG:teuthology.orchestra.run.smithi062:> import stat 2024-04-25T09:07:53.763 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:07:53.763 DEBUG:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T09:07:53.763 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-04-25T09:07:53.913 INFO:teuthology.orchestra.run.smithi062.stdout:1099511630188 2024-04-25T09:07:53.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:07:53.927 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:53.928 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:07:54.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.201+0000 7fd65f200640 1 Processor -- start 2024-04-25T09:07:54.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.201+0000 7fd65f200640 1 -- start start 2024-04-25T09:07:54.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.203+0000 7fd65f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6580ceb50 0x7fd65806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.203+0000 7fd65f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65815ee30 con 0x7fd65815e7c0 2024-04-25T09:07:54.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.203+0000 7fd65f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6581618b0 con 0x7fd65806d800 2024-04-25T09:07:54.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.203+0000 7fd65f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd658161b90 con 0x7fd6580ceb50 2024-04-25T09:07:54.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.203+0000 7fd65e800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd65815e7c0 0x7fd6580b4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53660/0 (socket says 172.21.15.62:53660) 2024-04-25T09:07:54.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.203+0000 7fd65e800640 1 -- 172.21.15.62:0/3389137691 learned_addr learned my addr 172.21.15.62:0/3389137691 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65de00640 1 --2- 172.21.15.62:0/3389137691 >> v2:172.21.15.121:3300/0 conn(0x7fd6580ceb50 0x7fd65806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535631821 0 0) 0x7fd65815ee30 con 0x7fd65815e7c0 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd640003520 con 0x7fd65815e7c0 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036577671 0 0) 0x7fd640003520 con 0x7fd65815e7c0 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 >> v2:172.21.15.121:3300/0 conn(0x7fd6580ceb50 msgr2=0x7fd65806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 --2- 172.21.15.62:0/3389137691 >> v2:172.21.15.121:3300/0 conn(0x7fd6580ceb50 0x7fd65806d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 >> v1:172.21.15.121:6789/0 conn(0x7fd65806d800 legacy=0x7fd65815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580cef30 con 0x7fd65815e7c0 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fd6540019f0 con 0x7fd65815e7c0 2024-04-25T09:07:54.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 >> v1:172.21.15.62:6789/0 conn(0x7fd65815e7c0 legacy=0x7fd6580b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6400039b0 0x7fd640003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd640004800 0x7fd640004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400052e0 0x7fd64000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd658161b90 con 0x7fd6400039b0 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64000c1b0 con 0x7fd6400052e0 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64000c3b0 con 0x7fd640004800 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.204+0000 7fd65de00640 1 --2- 172.21.15.62:0/3389137691 >> v2:172.21.15.121:3300/0 conn(0x7fd6580ceb50 0x7fd65806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65de00640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6400039b0 0x7fd640003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65e800640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400052e0 0x7fd64000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65d400640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd640004800 0x7fd640004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65de00640 1 -- 172.21.15.62:0/3389137691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd640004800 msgr2=0x7fd640004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65de00640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd640004800 0x7fd640004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65de00640 1 -- 172.21.15.62:0/3389137691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400052e0 msgr2=0x7fd64000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65de00640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400052e0 0x7fd64000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65e800640 1 -- 172.21.15.62:0/3389137691 reap_dead start 2024-04-25T09:07:54.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.205+0000 7fd65de00640 1 -- 172.21.15.62:0/3389137691 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd640003520 con 0x7fd6400039b0 2024-04-25T09:07:54.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.206+0000 7fd65de00640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6400039b0 0x7fd640003dd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd640005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b4e3b48cd193c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.207+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c037070 con 0x7fd6400039b0 2024-04-25T09:07:54.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.207+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c02ac70 con 0x7fd6400039b0 2024-04-25T09:07:54.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.207+0000 7fd65ca00640 1 -- 172.21.15.62:0/3389137691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00f040 con 0x7fd6400039b0 2024-04-25T09:07:54.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.207+0000 7fd65f200640 1 -- 172.21.15.62:0/3389137691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6400039b0 msgr2=0x7fd640003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.207+0000 7fd65f200640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6400039b0 0x7fd640003dd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd640005830 comp rx=0 tx=0).stop 2024-04-25T09:07:54.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.207+0000 7fd65f200640 1 -- 172.21.15.62:0/3389137691 shutdown_connections 2024-04-25T09:07:54.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.207+0000 7fd65f200640 1 --2- 172.21.15.62:0/3389137691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6400039b0 0x7fd640003dd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.207+0000 7fd65f200640 1 -- 172.21.15.62:0/3389137691 >> 172.21.15.62:0/3389137691 conn(0x7fd658066f20 msgr2=0x7fd6580b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.208+0000 7fd65f200640 1 -- 172.21.15.62:0/3389137691 shutdown_connections 2024-04-25T09:07:54.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.208+0000 7fd65f200640 1 -- 172.21.15.62:0/3389137691 wait complete. 2024-04-25T09:07:54.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.209+0000 7fd65f200640 1 Processor -- start 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.210+0000 7fd65f200640 1 -- start start 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.210+0000 7fd65f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400039b0 0x7fd658154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6580b0060 0x7fd6580b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65813cbc0 0x7fd65813d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65806d2c0 con 0x7fd6580b0060 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581625a0 con 0x7fd6400039b0 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65806d440 con 0x7fd65813cbc0 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6580b0060 0x7fd6580b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6580b0060 0x7fd6580b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53818/0 (socket says 172.21.15.62:53818) 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65d400640 1 -- 172.21.15.62:0/891920271 learned_addr learned my addr 172.21.15.62:0/891920271 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65de00640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400039b0 0x7fd658154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65e800640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65813cbc0 0x7fd65813d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65d400640 1 -- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65813cbc0 msgr2=0x7fd65813d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65d400640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65813cbc0 0x7fd65813d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65d400640 1 -- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400039b0 msgr2=0x7fd658154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65d400640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400039b0 0x7fd658154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65d400640 1 -- 172.21.15.62:0/891920271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580b0b50 con 0x7fd6580b0060 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.211+0000 7fd65e800640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65813cbc0 0x7fd65813d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.212+0000 7fd65d400640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6580b0060 0x7fd6580b04e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e440 tx=0x7fd64800e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.212+0000 7fd652000640 1 -- 172.21.15.62:0/891920271 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648019070 con 0x7fd6580b0060 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.212+0000 7fd652000640 1 -- 172.21.15.62:0/891920271 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6480052e0 con 0x7fd6580b0060 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.212+0000 7fd652000640 1 -- 172.21.15.62:0/891920271 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648003830 con 0x7fd6580b0060 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.212+0000 7fd65f200640 1 -- 172.21.15.62:0/891920271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6580b0e30 con 0x7fd6580b0060 2024-04-25T09:07:54.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.212+0000 7fd65f200640 1 -- 172.21.15.62:0/891920271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6581559d0 con 0x7fd6580b0060 2024-04-25T09:07:54.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.214+0000 7fd652000640 1 -- 172.21.15.62:0/891920271 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd64801c070 con 0x7fd6580b0060 2024-04-25T09:07:54.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.215+0000 7fd652000640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd63c06b500 0x7fd63c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.215+0000 7fd652000640 1 -- 172.21.15.62:0/891920271 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd648089720 con 0x7fd6580b0060 2024-04-25T09:07:54.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.216+0000 7fd663612740 1 Processor -- start 2024-04-25T09:07:54.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.216+0000 7fd65de00640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd63c06b500 0x7fd63c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.216+0000 7fd663612740 1 -- start start 2024-04-25T09:07:54.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd663612740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a4f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd663612740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a6ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd65de00640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd63c06b500 0x7fd63c06d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd64c034000 tx=0x7fd64c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd663612740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a24cf0 0x55f472a6af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd663612740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f472987d00 con 0x55f472a6c8b0 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd663612740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f472930f50 con 0x55f472a4f010 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd663612740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f472983da0 con 0x55f472a24cf0 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd65d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a6ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd65d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a6ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53830/0 (socket says 172.21.15.62:53830) 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.217+0000 7fd65d400640 1 -- 172.21.15.62:0/2725885224 learned_addr learned my addr 172.21.15.62:0/2725885224 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65e800640 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a24cf0 0x55f472a6af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65de00640 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a4f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65d400640 1 -- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a24cf0 msgr2=0x55f472a6af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65d400640 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a24cf0 0x55f472a6af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65d400640 1 -- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 msgr2=0x55f472a4f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65d400640 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a4f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65d400640 1 -- 172.21.15.62:0/2725885224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f472954130 con 0x55f472a6c8b0 2024-04-25T09:07:54.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65de00640 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a4f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:54.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd65d400640 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a6ce00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd648057ed0 tx=0x7fd6480579d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea3b3a5a04147a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd646600640 1 -- 172.21.15.62:0/2725885224 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64806f020 con 0x55f472a6c8b0 2024-04-25T09:07:54.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd646600640 1 -- 172.21.15.62:0/2725885224 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64806be50 con 0x55f472a6c8b0 2024-04-25T09:07:54.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.218+0000 7fd646600640 1 -- 172.21.15.62:0/2725885224 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6480605a0 con 0x55f472a6c8b0 2024-04-25T09:07:54.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 -- 172.21.15.62:0/2725885224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 msgr2=0x55f472a6ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a6ce00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd648057ed0 tx=0x7fd6480579d0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 -- 172.21.15.62:0/2725885224 shutdown_connections 2024-04-25T09:07:54.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a6ce00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a24cf0 0x55f472a6af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 --2- 172.21.15.62:0/2725885224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a4f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 -- 172.21.15.62:0/2725885224 >> 172.21.15.62:0/2725885224 conn(0x55f472a21620 msgr2=0x55f472a69e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 -- 172.21.15.62:0/2725885224 shutdown_connections 2024-04-25T09:07:54.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.219+0000 7fd663612740 1 -- 172.21.15.62:0/2725885224 wait complete. 2024-04-25T09:07:54.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.220+0000 7fd663612740 1 -- start start 2024-04-25T09:07:54.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.220+0000 7fd663612740 1 Processor -- start 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.221+0000 7fd663612740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a4f010 0x55f472a529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.221+0000 7fd663612740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 0x55f472a60ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.221+0000 7fd663612740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 0x55f472a4b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd663612740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f472a25a90 con 0x55f472a4f010 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd663612740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f472926530 con 0x55f472a611e0 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd663612740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f472a1f550 con 0x55f472a6c8b0 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd663612740 10 client.0 _ll_register_callbacks cb 0x55f47297f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a4f010 0x55f472a529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a4f010 0x55f472a529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53836/0 (socket says 172.21.15.62:53836) 2024-04-25T09:07:54.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65de00640 1 -- 172.21.15.62:0/888131350 learned_addr learned my addr 172.21.15.62:0/888131350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65d400640 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 0x55f472a60ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65de00640 1 -- 172.21.15.62:0/888131350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 msgr2=0x55f472a60ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65de00640 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 0x55f472a60ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65de00640 1 -- 172.21.15.62:0/888131350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 msgr2=0x55f472a4b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65de00640 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 0x55f472a4b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.222+0000 7fd65de00640 1 -- 172.21.15.62:0/888131350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f47294edd0 con 0x55f472a4f010 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd65de00640 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a4f010 0x55f472a529d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009960 tx=0x7fd64c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1c2034f89dc50a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd63aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd63aa00640 1 -- 172.21.15.62:0/888131350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c037040 con 0x55f472a4f010 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd63aa00640 1 -- 172.21.15.62:0/888131350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd64c049670 con 0x55f472a4f010 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd663612740 1 -- 172.21.15.62:0/888131350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd64806be50 con 0x55f472a4f010 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd63aa00640 1 -- 172.21.15.62:0/888131350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64c056050 con 0x55f472a4f010 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd663612740 10 client.5862 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd663612740 1 -- 172.21.15.62:0/888131350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55f47269c3f0 con 0x55f472a4f010 2024-04-25T09:07:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.223+0000 7fd63aa00640 1 -- 172.21.15.62:0/888131350 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64806be50 con 0x55f472a4f010 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.224+0000 7fd63aa00640 1 -- 172.21.15.62:0/888131350 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd64c059020 con 0x55f472a4f010 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.224+0000 7fd663612740 10 client.5862 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.224+0000 7fd663612740 10 client.5862 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.224+0000 7fd663612740 20 client.5862 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.224+0000 7fd663612740 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f472a50eb0 0x55f472a512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.224+0000 7fd663612740 4 client.5862 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.224+0000 7fd663612740 1 -- 172.21.15.62:0/888131350 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f472995240 con 0x55f472a50eb0 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.224+0000 7fd65d400640 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f472a50eb0 0x55f472a512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.225+0000 7fd65d400640 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f472a50eb0 0x55f472a512c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea12bdf6c0a3cefd server_cookie=6d2b1f0f9f5da134 in_seq=0 out_seq=0 2024-04-25T09:07:54.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.225+0000 7fd63aa00640 10 client.5862 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.225+0000 7fd63aa00640 1 -- 172.21.15.62:0/888131350 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f472995240 con 0x55f472a50eb0 2024-04-25T09:07:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.225+0000 7fd63aa00640 10 client.5862 handle_command_reply: tid=0 2024-04-25T09:07:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.225+0000 7fd663612740 1 client.5862 shutdown 2024-04-25T09:07:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.225+0000 7fd663612740 1 -- 172.21.15.62:0/888131350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a4f010 msgr2=0x55f472a529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.225+0000 7fd663612740 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a4f010 0x55f472a529d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009960 tx=0x7fd64c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.225+0000 7fd663612740 1 -- 172.21.15.62:0/888131350 shutdown_connections 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd663612740 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f472a50eb0 0x55f472a512c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd663612740 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 0x55f472a4b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd663612740 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 0x55f472a60ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd63aa00640 0 client.5862 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd663612740 1 --2- 172.21.15.62:0/888131350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a4f010 0x55f472a529d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd663612740 1 -- 172.21.15.62:0/888131350 >> 172.21.15.62:0/888131350 conn(0x55f472a24cf0 msgr2=0x55f472a4e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd663612740 1 -- 172.21.15.62:0/888131350 shutdown_connections 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd663612740 1 -- 172.21.15.62:0/888131350 wait complete. 2024-04-25T09:07:54.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.226+0000 7fd663612740 20 client.5862 trim_cache size 0 max 16384 2024-04-25T09:07:54.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.239+0000 7fd663612740 1 Processor -- start 2024-04-25T09:07:54.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.239+0000 7fd663612740 1 -- start start 2024-04-25T09:07:54.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd663612740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd663612740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 0x55f472a1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd663612740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a4de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd663612740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6582005d0 con 0x55f472a6c8b0 2024-04-25T09:07:54.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd663612740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581ff7d0 con 0x55f472a611e0 2024-04-25T09:07:54.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd663612740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd658156ef0 con 0x55f472a4f010 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd65e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a4de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd65e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a4de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53846/0 (socket says 172.21.15.62:53846) 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.240+0000 7fd65e800640 1 -- 172.21.15.62:0/766757284 learned_addr learned my addr 172.21.15.62:0/766757284 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd65de00640 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd65d400640 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 0x55f472a1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd65e800640 1 -- 172.21.15.62:0/766757284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a4f010 msgr2=0x55f472a84730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd65e800640 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd65e800640 1 -- 172.21.15.62:0/766757284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 msgr2=0x55f472a1ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd65e800640 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 0x55f472a1ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd65e800640 1 -- 172.21.15.62:0/766757284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4727667d0 con 0x55f472a6c8b0 2024-04-25T09:07:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd65e800640 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a4de20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd65806e160 tx=0x7fd654001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b6e14a74290284 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd645c00640 1 -- 172.21.15.62:0/766757284 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540076b0 con 0x55f472a6c8b0 2024-04-25T09:07:54.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd645c00640 1 -- 172.21.15.62:0/766757284 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654019070 con 0x55f472a6c8b0 2024-04-25T09:07:54.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.241+0000 7fd645c00640 1 -- 172.21.15.62:0/766757284 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654007ab0 con 0x55f472a6c8b0 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 -- 172.21.15.62:0/766757284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 msgr2=0x55f472a4de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a4de20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd65806e160 tx=0x7fd654001b20 comp rx=0 tx=0).stop 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 -- 172.21.15.62:0/766757284 shutdown_connections 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a6c8b0 0x55f472a4de20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a611e0 0x55f472a1ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 --2- 172.21.15.62:0/766757284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 -- 172.21.15.62:0/766757284 >> 172.21.15.62:0/766757284 conn(0x55f472a24cf0 msgr2=0x55f472a250d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 -- 172.21.15.62:0/766757284 shutdown_connections 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.242+0000 7fd663612740 1 -- 172.21.15.62:0/766757284 wait complete. 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.243+0000 7fd663612740 1 -- start start 2024-04-25T09:07:54.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.243+0000 7fd663612740 1 Processor -- start 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd663612740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd663612740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a611e0 0x55f472abd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd663612740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 0x55f472ad3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd663612740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f472a1b180 con 0x55f472a611e0 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd663612740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581574f0 con 0x55f472a4f010 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd663612740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f472a4b6f0 con 0x55f472a6c8b0 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd663612740 10 client.0 _ll_register_callbacks cb 0x55f472a1fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd65d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a611e0 0x55f472abd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd65d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a611e0 0x55f472abd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53852/0 (socket says 172.21.15.62:53852) 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.244+0000 7fd65d400640 1 -- 172.21.15.62:0/2814384815 learned_addr learned my addr 172.21.15.62:0/2814384815 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd65de00640 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd65d400640 1 -- 172.21.15.62:0/2814384815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 msgr2=0x55f472ad3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd65d400640 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 0x55f472ad3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd65d400640 1 -- 172.21.15.62:0/2814384815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 msgr2=0x55f472a84730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd65d400640 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd65d400640 1 -- 172.21.15.62:0/2814384815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f472a6d390 con 0x55f472a611e0 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd65de00640 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd65d400640 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a611e0 0x55f472abd050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd6480543a0 tx=0x7fd64805e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b95a7636deba164 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd63aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd63aa00640 1 -- 172.21.15.62:0/2814384815 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64806f040 con 0x55f472a611e0 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd63aa00640 1 -- 172.21.15.62:0/2814384815 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6480545b0 con 0x55f472a611e0 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.245+0000 7fd663612740 1 -- 172.21.15.62:0/2814384815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd654019070 con 0x55f472a611e0 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd63aa00640 1 -- 172.21.15.62:0/2814384815 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64806be50 con 0x55f472a611e0 2024-04-25T09:07:54.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd663612740 10 client.5868 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd663612740 1 -- 172.21.15.62:0/2814384815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55f472a6cf80 con 0x55f472a611e0 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd63aa00640 1 -- 172.21.15.62:0/2814384815 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654019070 con 0x55f472a611e0 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd63aa00640 1 -- 172.21.15.62:0/2814384815 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd64805e070 con 0x55f472a611e0 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd663612740 10 client.5868 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd663612740 10 client.5868 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd663612740 20 client.5868 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd663612740 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f472a67230 0x55f472a67610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd663612740 4 client.5868 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd663612740 1 -- 172.21.15.62:0/2814384815 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f472a64620 con 0x55f472a67230 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.246+0000 7fd65de00640 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f472a67230 0x55f472a67610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd65de00640 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f472a67230 0x55f472a67610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd270750e626bc6 server_cookie=cac86c9b2d86164c in_seq=0 out_seq=0 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd63aa00640 10 client.5868 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd63aa00640 1 -- 172.21.15.62:0/2814384815 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f472a64620 con 0x55f472a67230 2024-04-25T09:07:54.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd63aa00640 10 client.5868 handle_command_reply: tid=0 2024-04-25T09:07:54.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 client.5868 shutdown 2024-04-25T09:07:54.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 -- 172.21.15.62:0/2814384815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a611e0 msgr2=0x55f472abd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a611e0 0x55f472abd050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd6480543a0 tx=0x7fd64805e000 comp rx=0 tx=0).stop 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 -- 172.21.15.62:0/2814384815 shutdown_connections 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f472a67230 0x55f472a67610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f472a611e0 0x55f472abd050 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f472a6c8b0 0x55f472ad3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 --2- 172.21.15.62:0/2814384815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f472a4f010 0x55f472a84730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd63aa00640 0 client.5868 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.247+0000 7fd663612740 1 -- 172.21.15.62:0/2814384815 >> 172.21.15.62:0/2814384815 conn(0x55f472a24cf0 msgr2=0x55f472a4de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd663612740 1 -- 172.21.15.62:0/2814384815 shutdown_connections 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd663612740 1 -- 172.21.15.62:0/2814384815 wait complete. 2024-04-25T09:07:54.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd663612740 20 client.5868 trim_cache size 0 max 16384 2024-04-25T09:07:54.239 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-25T09:07:54.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 -- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd63c06b500 msgr2=0x7fd63c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd63c06b500 0x7fd63c06d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd64c034000 tx=0x7fd64c02e000 comp rx=0 tx=0).stop 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 -- 172.21.15.62:0/891920271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6580b0060 msgr2=0x7fd6580b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6580b0060 0x7fd6580b04e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e440 tx=0x7fd64800e900 comp rx=0 tx=0).stop 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 -- 172.21.15.62:0/891920271 shutdown_connections 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd63c06b500 0x7fd63c06d9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6580b0060 0x7fd6580b04e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd65813cbc0 0x7fd65813d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 --2- 172.21.15.62:0/891920271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6400039b0 0x7fd658154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.248+0000 7fd65f200640 1 -- 172.21.15.62:0/891920271 >> 172.21.15.62:0/891920271 conn(0x7fd658066f20 msgr2=0x7fd658145b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.249+0000 7fd65f200640 1 -- 172.21.15.62:0/891920271 shutdown_connections 2024-04-25T09:07:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.249+0000 7fd65f200640 1 -- 172.21.15.62:0/891920271 wait complete. 2024-04-25T09:07:54.253 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096c.00000000 parent 2024-04-25T09:07:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.377+0000 7fad46540f40 1 Processor -- start 2024-04-25T09:07:54.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.377+0000 7fad46540f40 1 -- start start 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.378+0000 7fad46540f40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 0x561bd53e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.378+0000 7fad46540f40 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 0x561bd53e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.378+0000 7fad46540f40 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd54febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.378+0000 7fad46540f40 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd533fdd0 con 0x561bd54fe740 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.378+0000 7fad46540f40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd5343d30 con 0x561bd54610b0 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.378+0000 7fad46540f40 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bd54707e0 con 0x561bd5472320 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd54febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd54febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53880/0 (socket says 172.21.15.62:53880) 2024-04-25T09:07:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 -- 172.21.15.62:0/1606196065 learned_addr learned my addr 172.21.15.62:0/1606196065 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad43800640 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 0x561bd53e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad42e00640 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 0x561bd53e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 -- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 msgr2=0x561bd53e19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 0x561bd53e19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 -- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 msgr2=0x561bd53e14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 0x561bd53e14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 -- 172.21.15.62:0/1606196065 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd5471650 con 0x561bd54fe740 2024-04-25T09:07:54.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad42e00640 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 0x561bd53e19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.379+0000 7fad44200640 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd54febc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fad3c001910 tx=0x7fad3c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec04cdb4aeaeb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad41a00640 1 -- 172.21.15.62:0/1606196065 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00f450 con 0x561bd54fe740 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad41a00640 1 -- 172.21.15.62:0/1606196065 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c00f5f0 con 0x561bd54fe740 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad41a00640 1 -- 172.21.15.62:0/1606196065 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00f9e0 con 0x561bd54fe740 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 -- 172.21.15.62:0/1606196065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 msgr2=0x561bd54febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd54febc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fad3c001910 tx=0x7fad3c0048f0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 -- 172.21.15.62:0/1606196065 shutdown_connections 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd54febc0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 0x561bd53e19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 --2- 172.21.15.62:0/1606196065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 0x561bd53e14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 -- 172.21.15.62:0/1606196065 >> 172.21.15.62:0/1606196065 conn(0x561bd53c3f20 msgr2=0x561bd53c91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 -- 172.21.15.62:0/1606196065 shutdown_connections 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.380+0000 7fad46540f40 1 -- 172.21.15.62:0/1606196065 wait complete. 2024-04-25T09:07:54.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.381+0000 7fad46540f40 1 Processor -- start 2024-04-25T09:07:54.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.381+0000 7fad46540f40 1 -- start start 2024-04-25T09:07:54.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.382+0000 7fad46540f40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 0x561bd545c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.382+0000 7fad46540f40 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 0x561bd54ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.382+0000 7fad46540f40 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd5459730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.382+0000 7fad46540f40 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd5471910 con 0x561bd54fe740 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.382+0000 7fad46540f40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd5461490 con 0x561bd54610b0 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.382+0000 7fad46540f40 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bd5471bf0 con 0x561bd5472320 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd5459730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd5459730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53896/0 (socket says 172.21.15.62:53896) 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 -- 172.21.15.62:0/3584338339 learned_addr learned my addr 172.21.15.62:0/3584338339 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad43800640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 0x561bd545c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad42e00640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 0x561bd54ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 -- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 msgr2=0x561bd54ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561bd5472320 0x561bd54ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 -- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 msgr2=0x561bd545c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561bd54610b0 0x561bd545c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 -- 172.21.15.62:0/3584338339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd545af90 con 0x561bd54fe740 2024-04-25T09:07:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.383+0000 7fad44200640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd5459730 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad3c014670 tx=0x561bd53def50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.384+0000 7fad3be00640 1 -- 172.21.15.62:0/3584338339 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c023070 con 0x561bd54fe740 2024-04-25T09:07:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.384+0000 7fad3be00640 1 -- 172.21.15.62:0/3584338339 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c00fe00 con 0x561bd54fe740 2024-04-25T09:07:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.384+0000 7fad3be00640 1 -- 172.21.15.62:0/3584338339 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00f450 con 0x561bd54fe740 2024-04-25T09:07:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.384+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561bd545b270 con 0x561bd54fe740 2024-04-25T09:07:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.384+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bd545b830 con 0x561bd54fe740 2024-04-25T09:07:54.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.386+0000 7fad3be00640 1 -- 172.21.15.62:0/3584338339 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad3c026070 con 0x561bd54fe740 2024-04-25T09:07:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.387+0000 7fad3be00640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fad24069f00 0x7fad2406c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.387+0000 7fad3be00640 1 -- 172.21.15.62:0/3584338339 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad3c089b30 con 0x561bd54fe740 2024-04-25T09:07:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.387+0000 7fad46540f40 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x561bd5459c70 0x561bd53e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.387+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 --> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] -- osd_op(unknown.0.0:1 7.2b 7:d716f75e:::1000000096c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48) v9 -- 0x561bd5507300 con 0x561bd5459c70 2024-04-25T09:07:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.388+0000 7fad43800640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fad24069f00 0x7fad2406c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.388+0000 7fad42e00640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x561bd5459c70 0x561bd53e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.389+0000 7fad43800640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fad24069f00 0x7fad2406c3c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fad34009520 tx=0x7fad3400c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.389+0000 7fad42e00640 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x561bd5459c70 0x561bd53e17b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.391+0000 7fad42e00640 1 -- 172.21.15.62:0/3584338339 <== osd.5 v2:172.21.15.121:6816/3527509140 1 ==== osd_op_reply(1 1000000096c.00000000 [rmxattr] v48'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fad300547a0 con 0x561bd5459c70 2024-04-25T09:07:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.391+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x561bd5459c70 msgr2=0x561bd53e17b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.391+0000 7fad46540f40 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x561bd5459c70 0x561bd53e17b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.391+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fad24069f00 msgr2=0x7fad2406c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.391+0000 7fad46540f40 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fad24069f00 0x7fad2406c3c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fad34009520 tx=0x7fad3400c040 comp rx=0 tx=0).stop 2024-04-25T09:07:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.392+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 msgr2=0x561bd5459730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.392+0000 7fad46540f40 1 --2- 172.21.15.62:0/3584338339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561bd54fe740 0x561bd5459730 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad3c014670 tx=0x561bd53def50 comp rx=0 tx=0).stop 2024-04-25T09:07:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.392+0000 7fad44200640 1 -- 172.21.15.62:0/3584338339 reap_dead start 2024-04-25T09:07:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.392+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 shutdown_connections 2024-04-25T09:07:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.392+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 >> 172.21.15.62:0/3584338339 conn(0x561bd53c3f20 msgr2=0x561bd53e0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.392+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 shutdown_connections 2024-04-25T09:07:54.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.392+0000 7fad46540f40 1 -- 172.21.15.62:0/3584338339 wait complete. 2024-04-25T09:07:54.386 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T09:07:54.386 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:07:54.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T09:07:54.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.612+0000 7f02e6e00640 1 Processor -- start 2024-04-25T09:07:54.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.612+0000 7f02e6e00640 1 -- start start 2024-04-25T09:07:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.614+0000 7f02e6e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02e00ccde0 0x7f02e00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.614+0000 7f02e6e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e00ced40 con 0x7f02e00b31a0 2024-04-25T09:07:54.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.614+0000 7f02e6e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e00cd260 con 0x7f02e006d2c0 2024-04-25T09:07:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.614+0000 7f02e6e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e00cf000 con 0x7f02e00ccde0 2024-04-25T09:07:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.614+0000 7f02e5000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02e00ccde0 0x7f02e00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.614+0000 7f02e5000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02e00ccde0 0x7f02e00ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53904/0 (socket says 172.21.15.62:53904) 2024-04-25T09:07:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.614+0000 7f02e5000640 1 -- 172.21.15.62:0/143793363 learned_addr learned my addr 172.21.15.62:0/143793363 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.614+0000 7f02e5000640 1 -- 172.21.15.62:0/143793363 >> v1:172.21.15.121:6789/0 conn(0x7f02e006d2c0 legacy=0x7f02e006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.615+0000 7f02e5000640 1 -- 172.21.15.62:0/143793363 >> v1:172.21.15.62:6789/0 conn(0x7f02e00b31a0 legacy=0x7f02e00b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.615+0000 7f02e5000640 1 -- 172.21.15.62:0/143793363 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e00cf2e0 con 0x7f02e00ccde0 2024-04-25T09:07:54.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.616+0000 7f02e5000640 1 --2- 172.21.15.62:0/143793363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e00ccde0 0x7f02e00ce800 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009870 tx=0x7f02d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd6f1ee1df26e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.616+0000 7f02dfe00640 1 -- 172.21.15.62:0/143793363 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0037070 con 0x7f02e00ccde0 2024-04-25T09:07:54.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.616+0000 7f02dfe00640 1 -- 172.21.15.62:0/143793363 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0032440 con 0x7f02e00ccde0 2024-04-25T09:07:54.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.616+0000 7f02dfe00640 1 -- 172.21.15.62:0/143793363 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0032810 con 0x7f02e00ccde0 2024-04-25T09:07:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.617+0000 7f02e6e00640 1 -- 172.21.15.62:0/143793363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e00ccde0 msgr2=0x7f02e00ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.617+0000 7f02e6e00640 1 --2- 172.21.15.62:0/143793363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e00ccde0 0x7f02e00ce800 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009870 tx=0x7f02d002a270 comp rx=0 tx=0).stop 2024-04-25T09:07:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.617+0000 7f02e6e00640 1 -- 172.21.15.62:0/143793363 shutdown_connections 2024-04-25T09:07:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.617+0000 7f02e6e00640 1 --2- 172.21.15.62:0/143793363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e00ccde0 0x7f02e00ce800 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.617+0000 7f02e6e00640 1 -- 172.21.15.62:0/143793363 >> 172.21.15.62:0/143793363 conn(0x7f02e0066f20 msgr2=0x7f02e00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.617+0000 7f02e6e00640 1 -- 172.21.15.62:0/143793363 shutdown_connections 2024-04-25T09:07:54.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.617+0000 7f02e6e00640 1 -- 172.21.15.62:0/143793363 wait complete. 2024-04-25T09:07:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.619+0000 7f02e6e00640 1 Processor -- start 2024-04-25T09:07:54.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.619+0000 7f02e6e00640 1 -- start start 2024-04-25T09:07:54.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e6e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e006d2c0 0x7f02e00aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e6e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e00b31a0 0x7f02e015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e6e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e00ccde0 0x7f02e00c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e6e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00b2c80 con 0x7f02e006d2c0 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e6e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00b29a0 con 0x7f02e00ccde0 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e6e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01448d0 con 0x7f02e00b31a0 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e006d2c0 0x7f02e00aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e006d2c0 0x7f02e00aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53918/0 (socket says 172.21.15.62:53918) 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5a00640 1 -- 172.21.15.62:0/4011309900 learned_addr learned my addr 172.21.15.62:0/4011309900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e6400640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e00ccde0 0x7f02e00c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5000640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e00b31a0 0x7f02e015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5a00640 1 -- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e00b31a0 msgr2=0x7f02e015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5a00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e00b31a0 0x7f02e015b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5a00640 1 -- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e00ccde0 msgr2=0x7f02e00c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5a00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e00ccde0 0x7f02e00c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.620+0000 7f02e5a00640 1 -- 172.21.15.62:0/4011309900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e00adaf0 con 0x7f02e006d2c0 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.621+0000 7f02e5a00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e006d2c0 0x7f02e00aeb90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02d400b3f0 tx=0x7f02d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.621+0000 7f02dd600640 1 -- 172.21.15.62:0/4011309900 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4015020 con 0x7f02e006d2c0 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.621+0000 7f02dd600640 1 -- 172.21.15.62:0/4011309900 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d4011830 con 0x7f02e006d2c0 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.621+0000 7f02e6e00640 1 -- 172.21.15.62:0/4011309900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e00addd0 con 0x7f02e006d2c0 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.621+0000 7f02dd600640 1 -- 172.21.15.62:0/4011309900 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4011c20 con 0x7f02e006d2c0 2024-04-25T09:07:54.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.621+0000 7f02e6e00640 1 -- 172.21.15.62:0/4011309900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e00ae390 con 0x7f02e006d2c0 2024-04-25T09:07:54.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.624+0000 7f02dd600640 1 -- 172.21.15.62:0/4011309900 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02d4018020 con 0x7f02e006d2c0 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.625+0000 7f02dd600640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02c406b500 0x7f02c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.625+0000 7f02dd600640 1 -- 172.21.15.62:0/4011309900 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02d4089da0 con 0x7f02e006d2c0 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.625+0000 7f02eb240740 1 Processor -- start 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.625+0000 7f02eb240740 1 -- start start 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.626+0000 7f02e5000640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02c406b500 0x7f02c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.626+0000 7f02eb240740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee78140 0x55f7aee78520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.626+0000 7f02eb240740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee73050 0x55f7aee76d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.626+0000 7f02eb240740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee77280 0x55f7aee776c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.626+0000 7f02eb240740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7aeddbd80 con 0x55f7aee77280 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.626+0000 7f02eb240740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7aed84cc0 con 0x55f7aee78140 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.626+0000 7f02eb240740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7aedd7e20 con 0x55f7aee73050 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.626+0000 7f02e6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee77280 0x55f7aee776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee77280 0x55f7aee776c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53924/0 (socket says 172.21.15.62:53924) 2024-04-25T09:07:54.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e6400640 1 -- 172.21.15.62:0/3911970833 learned_addr learned my addr 172.21.15.62:0/3911970833 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e5a00640 1 --2- 172.21.15.62:0/3911970833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee78140 0x55f7aee78520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e5000640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02c406b500 0x7f02c406d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f02d0034000 tx=0x7f02d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e6400640 1 -- 172.21.15.62:0/3911970833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee73050 msgr2=0x55f7aee76d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e6400640 1 --2- 172.21.15.62:0/3911970833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee73050 0x55f7aee76d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e6400640 1 -- 172.21.15.62:0/3911970833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee78140 msgr2=0x55f7aee78520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e6400640 1 --2- 172.21.15.62:0/3911970833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee78140 0x55f7aee78520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e6400640 1 -- 172.21.15.62:0/3911970833 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7aebbcc00 con 0x55f7aee77280 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02e6400640 1 --2- 172.21.15.62:0/3911970833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee77280 0x55f7aee776c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f02d8001d60 tx=0x7f02d800d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a924ef3ecb3019f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.627+0000 7f02ce000640 1 -- 172.21.15.62:0/3911970833 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8018070 con 0x55f7aee77280 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02ce000640 1 -- 172.21.15.62:0/3911970833 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800dd90 con 0x55f7aee77280 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02ce000640 1 -- 172.21.15.62:0/3911970833 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8003b90 con 0x55f7aee77280 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 -- 172.21.15.62:0/3911970833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee77280 msgr2=0x55f7aee776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 --2- 172.21.15.62:0/3911970833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee77280 0x55f7aee776c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f02d8001d60 tx=0x7f02d800d150 comp rx=0 tx=0).stop 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 -- 172.21.15.62:0/3911970833 shutdown_connections 2024-04-25T09:07:54.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 --2- 172.21.15.62:0/3911970833 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee77280 0x55f7aee776c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 --2- 172.21.15.62:0/3911970833 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee73050 0x55f7aee76d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 --2- 172.21.15.62:0/3911970833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee78140 0x55f7aee78520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 -- 172.21.15.62:0/3911970833 >> 172.21.15.62:0/3911970833 conn(0x55f7aee754d0 msgr2=0x55f7aeebdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 -- 172.21.15.62:0/3911970833 shutdown_connections 2024-04-25T09:07:54.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.628+0000 7f02eb240740 1 -- 172.21.15.62:0/3911970833 wait complete. 2024-04-25T09:07:54.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.629+0000 7f02eb240740 1 -- start start 2024-04-25T09:07:54.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.629+0000 7f02eb240740 1 Processor -- start 2024-04-25T09:07:54.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.630+0000 7f02eb240740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aeec5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.630+0000 7f02eb240740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee78140 0x55f7aeebafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.630+0000 7f02eb240740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee8d9a0 0x55f7aeec3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.630+0000 7f02eb240740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7aeea3cd0 con 0x55f7aee78140 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.630+0000 7f02eb240740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7aed7a240 con 0x55f7aee77280 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.630+0000 7f02eb240740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7aeea3150 con 0x55f7aee8d9a0 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.630+0000 7f02eb240740 10 client.0 _ll_register_callbacks cb 0x55f7aedd3280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee78140 0x55f7aeebafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee78140 0x55f7aeebafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53936/0 (socket says 172.21.15.62:53936) 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 -- 172.21.15.62:0/805046482 learned_addr learned my addr 172.21.15.62:0/805046482 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5a00640 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aeec5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 -- 172.21.15.62:0/805046482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee8d9a0 msgr2=0x55f7aeec3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e6400640 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee8d9a0 0x55f7aeec3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee8d9a0 0x55f7aeec3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 -- 172.21.15.62:0/805046482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 msgr2=0x55f7aeec5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aeec5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 -- 172.21.15.62:0/805046482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7aeab04f0 con 0x55f7aee78140 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02e5000640 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee78140 0x55f7aeebafd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f02d00090b0 tx=0x7f02d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183e4768e08bb9df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.631+0000 7f02c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02c2a00640 1 -- 172.21.15.62:0/805046482 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d004d060 con 0x55f7aee78140 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02eb240740 1 -- 172.21.15.62:0/805046482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02d800dd90 con 0x55f7aee78140 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02c2a00640 1 -- 172.21.15.62:0/805046482 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02d003e9f0 con 0x55f7aee78140 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02c2a00640 1 -- 172.21.15.62:0/805046482 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d0053050 con 0x55f7aee78140 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02eb240740 10 client.5889 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02eb240740 1 -- 172.21.15.62:0/805046482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55f7aec1d390 con 0x55f7aee78140 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02c2a00640 1 -- 172.21.15.62:0/805046482 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800dd90 con 0x55f7aee78140 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02c2a00640 1 -- 172.21.15.62:0/805046482 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02d0056020 con 0x55f7aee78140 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02eb240740 10 client.5889 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02eb240740 10 client.5889 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02eb240740 20 client.5889 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.632+0000 7f02eb240740 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f7aeec4110 0x55f7aeec23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.633+0000 7f02eb240740 4 client.5889 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.633+0000 7f02eb240740 1 -- 172.21.15.62:0/805046482 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7aede92c0 con 0x55f7aeec4110 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.633+0000 7f02e5a00640 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f7aeec4110 0x55f7aeec23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.633+0000 7f02e5a00640 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f7aeec4110 0x55f7aeec23d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f3273ae35d45c5 server_cookie=7616c6644a1c87ed in_seq=0 out_seq=0 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.633+0000 7f02c2a00640 10 client.5889 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02c2a00640 1 -- 172.21.15.62:0/805046482 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7aede92c0 con 0x55f7aeec4110 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02c2a00640 10 client.5889 handle_command_reply: tid=0 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 client.5889 shutdown 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 -- 172.21.15.62:0/805046482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee78140 msgr2=0x55f7aeebafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee78140 0x55f7aeebafd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f02d00090b0 tx=0x7f02d0004730 comp rx=0 tx=0).stop 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 -- 172.21.15.62:0/805046482 shutdown_connections 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f7aeec4110 0x55f7aeec23d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee78140 0x55f7aeebafd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee8d9a0 0x55f7aeec3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 --2- 172.21.15.62:0/805046482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aeec5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02c2a00640 0 client.5889 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.634+0000 7f02eb240740 1 -- 172.21.15.62:0/805046482 >> 172.21.15.62:0/805046482 conn(0x55f7aee73050 msgr2=0x55f7aee92960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.635+0000 7f02eb240740 1 -- 172.21.15.62:0/805046482 shutdown_connections 2024-04-25T09:07:54.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.635+0000 7f02eb240740 1 -- 172.21.15.62:0/805046482 wait complete. 2024-04-25T09:07:54.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.635+0000 7f02eb240740 20 client.5889 trim_cache size 0 max 16384 2024-04-25T09:07:54.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.652+0000 7f02eb240740 1 Processor -- start 2024-04-25T09:07:54.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.652+0000 7f02eb240740 1 -- start start 2024-04-25T09:07:54.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.652+0000 7f02eb240740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aee722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.652+0000 7f02eb240740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 0x55f7aeebe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.652+0000 7f02eb240740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeebe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.652+0000 7f02eb240740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00ab7d0 con 0x55f7aee8d9a0 2024-04-25T09:07:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.652+0000 7f02eb240740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00aa9d0 con 0x55f7aee77280 2024-04-25T09:07:54.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.652+0000 7f02eb240740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e00716b0 con 0x55f7aee78140 2024-04-25T09:07:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.653+0000 7f02e6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeebe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.653+0000 7f02e6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeebe7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53944/0 (socket says 172.21.15.62:53944) 2024-04-25T09:07:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.653+0000 7f02e6400640 1 -- 172.21.15.62:0/267041241 learned_addr learned my addr 172.21.15.62:0/267041241 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.653+0000 7f02e5000640 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 0x55f7aeebe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.653+0000 7f02e5a00640 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aee722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.653+0000 7f02e6400640 1 -- 172.21.15.62:0/267041241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 msgr2=0x55f7aeebe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.653+0000 7f02e6400640 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 0x55f7aeebe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.654+0000 7f02e6400640 1 -- 172.21.15.62:0/267041241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 msgr2=0x55f7aee722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.654+0000 7f02e6400640 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aee722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.654+0000 7f02e6400640 1 -- 172.21.15.62:0/267041241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7aea89670 con 0x55f7aee8d9a0 2024-04-25T09:07:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.654+0000 7f02e6400640 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeebe7a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f02d8016040 tx=0x7f02d800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba32bc41c91a94f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.654+0000 7f02cd600640 1 -- 172.21.15.62:0/267041241 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8024070 con 0x55f7aee8d9a0 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.654+0000 7f02cd600640 1 -- 172.21.15.62:0/267041241 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800db90 con 0x55f7aee8d9a0 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.654+0000 7f02cd600640 1 -- 172.21.15.62:0/267041241 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8018030 con 0x55f7aee8d9a0 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 -- 172.21.15.62:0/267041241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 msgr2=0x55f7aeebe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeebe7a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f02d8016040 tx=0x7f02d800b000 comp rx=0 tx=0).stop 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 -- 172.21.15.62:0/267041241 shutdown_connections 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeebe7a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 0x55f7aeebe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 --2- 172.21.15.62:0/267041241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aee722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 -- 172.21.15.62:0/267041241 >> 172.21.15.62:0/267041241 conn(0x55f7aee73050 msgr2=0x55f7aeec5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 -- 172.21.15.62:0/267041241 shutdown_connections 2024-04-25T09:07:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.655+0000 7f02eb240740 1 -- 172.21.15.62:0/267041241 wait complete. 2024-04-25T09:07:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.656+0000 7f02eb240740 1 -- start start 2024-04-25T09:07:54.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.656+0000 7f02eb240740 1 Processor -- start 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02eb240740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aeea2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02eb240740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 0x55f7aeeb5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02eb240740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeeb5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02eb240740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e0071cb0 con 0x55f7aee8d9a0 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02eb240740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7aeeb4b50 con 0x55f7aee77280 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02eb240740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7aef310c0 con 0x55f7aee78140 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02eb240740 10 client.0 _ll_register_callbacks cb 0x7f02d8001560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02e6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeeb5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02e6400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeeb5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53950/0 (socket says 172.21.15.62:53950) 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.657+0000 7f02e6400640 1 -- 172.21.15.62:0/2410115175 learned_addr learned my addr 172.21.15.62:0/2410115175 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02e5a00640 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aeea2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02e6400640 1 -- 172.21.15.62:0/2410115175 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 msgr2=0x55f7aeeb5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02e6400640 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 0x55f7aeeb5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02e6400640 1 -- 172.21.15.62:0/2410115175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 msgr2=0x55f7aeea2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02e6400640 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aeea2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02e6400640 1 -- 172.21.15.62:0/2410115175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7aeee4520 con 0x55f7aee8d9a0 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02e6400640 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeeb5db0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f02d8007fd0 tx=0x7f02d8001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef9b142b9e4f4fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02c2a00640 1 -- 172.21.15.62:0/2410115175 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8024040 con 0x55f7aee8d9a0 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02c2a00640 1 -- 172.21.15.62:0/2410115175 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02d80050d0 con 0x55f7aee8d9a0 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.658+0000 7f02eb240740 1 -- 172.21.15.62:0/2410115175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02d800db90 con 0x55f7aee8d9a0 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02c2a00640 1 -- 172.21.15.62:0/2410115175 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02d8035050 con 0x55f7aee8d9a0 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02eb240740 10 client.5895 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:54.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02eb240740 1 -- 172.21.15.62:0/2410115175 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55f7aee8ec40 con 0x55f7aee8d9a0 2024-04-25T09:07:54.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02c2a00640 1 -- 172.21.15.62:0/2410115175 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800db90 con 0x55f7aee8d9a0 2024-04-25T09:07:54.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02c2a00640 1 -- 172.21.15.62:0/2410115175 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02d8038020 con 0x55f7aee8d9a0 2024-04-25T09:07:54.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02eb240740 10 client.5895 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:54.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02eb240740 10 client.5895 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:54.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02eb240740 20 client.5895 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:54.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02eb240740 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f7aeebdf60 0x55f7aeebe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02eb240740 4 client.5895 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T09:07:54.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02eb240740 1 -- 172.21.15.62:0/2410115175 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x55f7aede3610 con 0x55f7aeebdf60 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.659+0000 7f02e5a00640 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f7aeebdf60 0x55f7aeebe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.660+0000 7f02e5a00640 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f7aeebdf60 0x55f7aeebe370 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b64a7d1ecf95674 server_cookie=f376ea1bba2fcbf7 in_seq=0 out_seq=0 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.660+0000 7f02c2a00640 10 client.5895 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.660+0000 7f02c2a00640 1 -- 172.21.15.62:0/2410115175 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f7aede3610 con 0x55f7aeebdf60 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.660+0000 7f02c2a00640 10 client.5895 handle_command_reply: tid=0 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.660+0000 7f02eb240740 1 client.5895 shutdown 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 -- 172.21.15.62:0/2410115175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 msgr2=0x55f7aeeb5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeeb5db0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f02d8007fd0 tx=0x7f02d8001330 comp rx=0 tx=0).stop 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 -- 172.21.15.62:0/2410115175 shutdown_connections 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55f7aeebdf60 0x55f7aeebe370 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f7aee8d9a0 0x55f7aeeb5db0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f7aee78140 0x55f7aeeb5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 --2- 172.21.15.62:0/2410115175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7aee77280 0x55f7aeea2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 -- 172.21.15.62:0/2410115175 >> 172.21.15.62:0/2410115175 conn(0x55f7aee73050 msgr2=0x55f7aee722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02c2a00640 0 client.5895 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 -- 172.21.15.62:0/2410115175 shutdown_connections 2024-04-25T09:07:54.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 1 -- 172.21.15.62:0/2410115175 wait complete. 2024-04-25T09:07:54.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f02eb240740 20 client.5895 trim_cache size 0 max 16384 2024-04-25T09:07:54.651 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:54.651 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:07:54.651 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "f7654421-4864-46b2-b0f5-f404fef1b802", 2024-04-25T09:07:54.651 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:07:54.651 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:54.651 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-04-25T09:07:54.661+0000 7f6707c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096c [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v658 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5733=0-4194304@1} caps={5733=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5733 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55e2444c1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1000000096c:[<0x10000000001/new_file_0 v658>,<0x10000000000/ceph-qa-suite v65>,<0x1/scrub_test_path v28>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T09:07:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 -- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02c406b500 msgr2=0x7f02c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02c406b500 0x7f02c406d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f02d0034000 tx=0x7f02d002e000 comp rx=0 tx=0).stop 2024-04-25T09:07:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 -- 172.21.15.62:0/4011309900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e006d2c0 msgr2=0x7f02e00aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e006d2c0 0x7f02e00aeb90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02d400b3f0 tx=0x7f02d400b8b0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 -- 172.21.15.62:0/4011309900 shutdown_connections 2024-04-25T09:07:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f02c406b500 0x7f02c406d9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02e00ccde0 0x7f02e00c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02e00b31a0 0x7f02e015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 --2- 172.21.15.62:0/4011309900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02e006d2c0 0x7f02e00aeb90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 -- 172.21.15.62:0/4011309900 >> 172.21.15.62:0/4011309900 conn(0x7f02e0066f20 msgr2=0x7f02e006cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 -- 172.21.15.62:0/4011309900 shutdown_connections 2024-04-25T09:07:54.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.662+0000 7f02e6e00640 1 -- 172.21.15.62:0/4011309900 wait complete. 2024-04-25T09:07:54.666 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': 'f7654421-4864-46b2-b0f5-f404fef1b802', 'mode': 'asynchronous'}' 2024-04-25T09:07:54.666 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:07:54.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:07:54.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.877+0000 7fa3cbe00640 1 Processor -- start 2024-04-25T09:07:54.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.877+0000 7fa3cbe00640 1 -- start start 2024-04-25T09:07:54.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.878+0000 7fa3cbe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa3cc15dd60 0x7fa3cc0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.878+0000 7fa3cbe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0d1c90 con 0x7fa3cc162300 2024-04-25T09:07:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.878+0000 7fa3cbe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0d1f70 con 0x7fa3cc0ce990 2024-04-25T09:07:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.878+0000 7fa3cbe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc0d2250 con 0x7fa3cc15dd60 2024-04-25T09:07:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.878+0000 7fa3cb400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa3cc162300 0x7fa3cc0b6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50650/0 (socket says 172.21.15.62:50650) 2024-04-25T09:07:54.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.878+0000 7fa3cb400640 1 -- 172.21.15.62:0/1342438775 learned_addr learned my addr 172.21.15.62:0/1342438775 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:07:54.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1342438775 >> v2:172.21.15.121:3300/0 conn(0x7fa3cc15dd60 0x7fa3cc0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556040601 0 0) 0x7fa3cc0d1c90 con 0x7fa3cc162300 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b0003520 con 0x7fa3cc162300 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228365976 0 0) 0x7fa3b0003520 con 0x7fa3cc162300 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 >> v2:172.21.15.121:3300/0 conn(0x7fa3cc15dd60 msgr2=0x7fa3cc0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 --2- 172.21.15.62:0/1342438775 >> v2:172.21.15.121:3300/0 conn(0x7fa3cc15dd60 0x7fa3cc0d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 >> v1:172.21.15.121:6789/0 conn(0x7fa3cc0ce990 legacy=0x7fa3cc161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0d2530 con 0x7fa3cc162300 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fa3c4001670 con 0x7fa3cc162300 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 >> v1:172.21.15.62:6789/0 conn(0x7fa3cc162300 legacy=0x7fa3cc0b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.879+0000 7fa3c9600640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b0004800 0x7fa3b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3c9600640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0d2250 con 0x7fa3b0003a00 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b000c0c0 con 0x7fa3b0004800 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b000c2c0 con 0x7fa3b00051c0 2024-04-25T09:07:54.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3ca000640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b0004800 0x7fa3b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3cb400640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3caa00640 1 -- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b00051c0 msgr2=0x7fa3b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3caa00640 1 -- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b0004800 msgr2=0x7fa3b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3b0004800 0x7fa3b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.880+0000 7fa3caa00640 1 -- 172.21.15.62:0/1342438775 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0003520 con 0x7fa3b0003a00 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.881+0000 7fa3cb400640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3b00051c0 0x7fa3b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:54.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.881+0000 7fa3cb400640 1 -- 172.21.15.62:0/1342438775 reap_dead start 2024-04-25T09:07:54.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.882+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3b0003e20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009ac0 tx=0x7fa3b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb3f9ad5acdc8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.882+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4036070 con 0x7fa3b0003a00 2024-04-25T09:07:54.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.882+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b4031400 con 0x7fa3b0003a00 2024-04-25T09:07:54.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.882+0000 7fa3c9600640 1 -- 172.21.15.62:0/1342438775 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4009690 con 0x7fa3b0003a00 2024-04-25T09:07:54.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.882+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1342438775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 msgr2=0x7fa3b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.882+0000 7fa3cbe00640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3b0003e20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009ac0 tx=0x7fa3b0005830 comp rx=0 tx=0).stop 2024-04-25T09:07:54.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.883+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1342438775 shutdown_connections 2024-04-25T09:07:54.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.883+0000 7fa3cbe00640 1 --2- 172.21.15.62:0/1342438775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3b0003e20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.883+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1342438775 >> 172.21.15.62:0/1342438775 conn(0x7fa3cc06b210 msgr2=0x7fa3cc0668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.883+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1342438775 shutdown_connections 2024-04-25T09:07:54.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.883+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1342438775 wait complete. 2024-04-25T09:07:54.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.884+0000 7fa3cbe00640 1 Processor -- start 2024-04-25T09:07:54.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.885+0000 7fa3cbe00640 1 -- start start 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3cc1f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0b58e0 0x7fa3cc0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc1f44a0 0x7fa3cc1f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0cefa0 con 0x7fa3b0003a00 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0cee20 con 0x7fa3cc1f44a0 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0cf120 con 0x7fa3cc0b58e0 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3cc1f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3caa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3cc1f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45224/0 (socket says 172.21.15.62:45224) 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3caa00640 1 -- 172.21.15.62:0/1812830007 learned_addr learned my addr 172.21.15.62:0/1812830007 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3caa00640 1 -- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0b58e0 msgr2=0x7fa3cc0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0b58e0 0x7fa3cc0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3caa00640 1 -- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc1f44a0 msgr2=0x7fa3cc1f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc1f44a0 0x7fa3cc1f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.886+0000 7fa3caa00640 1 -- 172.21.15.62:0/1812830007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0b62a0 con 0x7fa3b0003a00 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.887+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3cc1f7ad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009be0 tx=0x7fa3b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.887+0000 7fa3c2a00640 1 -- 172.21.15.62:0/1812830007 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b403f020 con 0x7fa3b0003a00 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.887+0000 7fa3c2a00640 1 -- 172.21.15.62:0/1812830007 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b4037040 con 0x7fa3b0003a00 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.887+0000 7fa3c2a00640 1 -- 172.21.15.62:0/1812830007 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4036070 con 0x7fa3b0003a00 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.887+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1812830007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc160380 con 0x7fa3b0003a00 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.887+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1812830007 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc160940 con 0x7fa3b0003a00 2024-04-25T09:07:54.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.889+0000 7fa3c2a00640 1 -- 172.21.15.62:0/1812830007 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3b4042020 con 0x7fa3b0003a00 2024-04-25T09:07:54.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.891+0000 7fa3c2a00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa3a806b4d0 0x7fa3a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.891+0000 7fa3c2a00640 1 -- 172.21.15.62:0/1812830007 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3b40a9fc0 con 0x7fa3b0003a00 2024-04-25T09:07:54.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.892+0000 7fa3ca000640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa3a806b4d0 0x7fa3a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.892+0000 7fa3d4bb8740 1 Processor -- start 2024-04-25T09:07:54.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.892+0000 7fa3d4bb8740 1 -- start start 2024-04-25T09:07:54.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3ca000640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa3a806b4d0 0x7fa3a806d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ce2e0 tx=0x7fa3bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c206d70 0x56168c208ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c243440 0x56168c207150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c231cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168c16ac70 con 0x56168c231830 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168c113cf0 con 0x56168c206d70 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56168c166d10 con 0x56168c243440 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3cb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c231cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3cb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c231cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45228/0 (socket says 172.21.15.62:45228) 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.893+0000 7fa3cb400640 1 -- 172.21.15.62:0/2804511360 learned_addr learned my addr 172.21.15.62:0/2804511360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3caa00640 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c206d70 0x56168c208ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3ca000640 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c243440 0x56168c207150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3cb400640 1 -- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c243440 msgr2=0x56168c207150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3cb400640 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c243440 0x56168c207150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3cb400640 1 -- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c206d70 msgr2=0x56168c208ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3cb400640 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c206d70 0x56168c208ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3cb400640 1 -- 172.21.15.62:0/2804511360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56168bf497d0 con 0x56168c231830 2024-04-25T09:07:54.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3caa00640 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c206d70 0x56168c208ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:07:54.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3cb400640 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c231cb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0cdf10 tx=0x7fa3c4007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6045af417e599c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3bb400640 1 -- 172.21.15.62:0/2804511360 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4007440 con 0x56168c231830 2024-04-25T09:07:54.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3bb400640 1 -- 172.21.15.62:0/2804511360 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c4007620 con 0x56168c231830 2024-04-25T09:07:54.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.894+0000 7fa3bb400640 1 -- 172.21.15.62:0/2804511360 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4004a20 con 0x56168c231830 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/2804511360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 msgr2=0x56168c231cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c231cb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0cdf10 tx=0x7fa3c4007d50 comp rx=0 tx=0).stop 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/2804511360 shutdown_connections 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c231cb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c243440 0x56168c207150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/2804511360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c206d70 0x56168c208ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/2804511360 >> 172.21.15.62:0/2804511360 conn(0x56168c1cea30 msgr2=0x56168c24c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/2804511360 shutdown_connections 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.895+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/2804511360 wait complete. 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.896+0000 7fa3d4bb8740 1 -- start start 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.896+0000 7fa3d4bb8740 1 Processor -- start 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c2210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c232a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c245dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168c1093a0 con 0x56168c231830 2024-04-25T09:07:54.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168c22f380 con 0x56168c243440 2024-04-25T09:07:54.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56168c25c6b0 con 0x56168c206d70 2024-04-25T09:07:54.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3d4bb8740 10 client.0 _ll_register_callbacks cb 0x56168c126b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:54.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c232a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c232a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45244/0 (socket says 172.21.15.62:45244) 2024-04-25T09:07:54.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.897+0000 7fa3ca000640 1 -- 172.21.15.62:0/1610633221 learned_addr learned my addr 172.21.15.62:0/1610633221 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c2210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3cb400640 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c245dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3ca000640 1 -- 172.21.15.62:0/1610633221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 msgr2=0x56168c2210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3ca000640 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c2210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3ca000640 1 -- 172.21.15.62:0/1610633221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 msgr2=0x56168c245dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3ca000640 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c245dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3ca000640 1 -- 172.21.15.62:0/1610633221 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56168be7f3f0 con 0x56168c231830 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3ca000640 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c232a80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e810 tx=0x7fa3bc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b881721b92d8422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.898+0000 7fa3afe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3afe00640 1 -- 172.21.15.62:0/1610633221 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc027060 con 0x56168c231830 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3afe00640 1 -- 172.21.15.62:0/1610633221 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3bc010040 con 0x56168c231830 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/1610633221 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3c4007620 con 0x56168c231830 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3afe00640 1 -- 172.21.15.62:0/1610633221 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3bc005650 con 0x56168c231830 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 10 client.5907 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/1610633221 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x56168be3f0f0 con 0x56168c231830 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3afe00640 1 -- 172.21.15.62:0/1610633221 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c4007620 con 0x56168c231830 2024-04-25T09:07:54.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3afe00640 1 -- 172.21.15.62:0/1610633221 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa3bc013420 con 0x56168c231830 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 10 client.5907 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 10 client.5907 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 20 client.5907 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56168c233690 0x56168c25faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 4 client.5907 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.899+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/1610633221 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56168c1781b0 con 0x56168c233690 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.900+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56168c233690 0x56168c25faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.900+0000 7fa3caa00640 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56168c233690 0x56168c25faf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c8c5b1c733340db server_cookie=6b0404c3fd9b62ac in_seq=0 out_seq=0 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.900+0000 7fa3afe00640 10 client.5907 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3afe00640 1 -- 172.21.15.62:0/1610633221 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56168c1781b0 con 0x56168c233690 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3afe00640 10 client.5907 handle_command_reply: tid=0 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 client.5907 shutdown 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/1610633221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 msgr2=0x56168c232a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c232a80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e810 tx=0x7fa3bc000f40 comp rx=0 tx=0).stop 2024-04-25T09:07:54.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/1610633221 shutdown_connections 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56168c233690 0x56168c25faf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c232a80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c245dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/1610633221 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c2210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3afe00640 0 client.5907 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/1610633221 >> 172.21.15.62:0/1610633221 conn(0x56168c1cea30 msgr2=0x56168c24fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.901+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/1610633221 shutdown_connections 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.902+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/1610633221 wait complete. 2024-04-25T09:07:54.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.902+0000 7fa3d4bb8740 20 client.5907 trim_cache size 0 max 16384 2024-04-25T09:07:54.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.918+0000 7fa3d4bb8740 1 Processor -- start 2024-04-25T09:07:54.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.919+0000 7fa3d4bb8740 1 -- start start 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.919+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c232c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.919+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c205530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.919+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c205a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.919+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc2005f0 con 0x56168c231830 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.919+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc1ff9f0 con 0x56168c243440 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.919+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc14bdc0 con 0x56168c206d70 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c205530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c205530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45250/0 (socket says 172.21.15.62:45250) 2024-04-25T09:07:54.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 -- 172.21.15.62:0/3345026457 learned_addr learned my addr 172.21.15.62:0/3345026457 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 -- 172.21.15.62:0/3345026457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 msgr2=0x56168c232c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 --2- 172.21.15.62:0/3345026457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c232c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 -- 172.21.15.62:0/3345026457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 msgr2=0x56168c205a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 --2- 172.21.15.62:0/3345026457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c205a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 -- 172.21.15.62:0/3345026457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56168c203050 con 0x56168c231830 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.920+0000 7fa3ca000640 1 --2- 172.21.15.62:0/3345026457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c205530 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e7e0 tx=0x7fa3bc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405c8dca7750a301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3baa00640 1 -- 172.21.15.62:0/3345026457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0057f0 con 0x56168c231830 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3baa00640 1 -- 172.21.15.62:0/3345026457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc007e50 con 0x56168c231830 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3baa00640 1 -- 172.21.15.62:0/3345026457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc013420 con 0x56168c231830 2024-04-25T09:07:54.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3345026457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 msgr2=0x56168c205530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3345026457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c205530 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00e7e0 tx=0x7fa3bc01f000 comp rx=0 tx=0).stop 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3345026457 shutdown_connections 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3345026457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c205530 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3345026457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c205a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3345026457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c232c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3345026457 >> 172.21.15.62:0/3345026457 conn(0x56168c1cea30 msgr2=0x56168c220ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3345026457 shutdown_connections 2024-04-25T09:07:54.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.921+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3345026457 wait complete. 2024-04-25T09:07:54.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.922+0000 7fa3d4bb8740 1 -- start start 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.922+0000 7fa3d4bb8740 1 Processor -- start 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.923+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c2b9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.923+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c21f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.923+0000 7fa3d4bb8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c1fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.923+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0cccb0 con 0x56168c231830 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.923+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168c2015b0 con 0x56168c243440 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.923+0000 7fa3d4bb8740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56168c220080 con 0x56168c206d70 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.923+0000 7fa3d4bb8740 10 client.0 _ll_register_callbacks cb 0x7fa3bc001870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c21f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c21f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45258/0 (socket says 172.21.15.62:45258) 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 -- 172.21.15.62:0/3467326198 learned_addr learned my addr 172.21.15.62:0/3467326198 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3caa00640 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c2b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 -- 172.21.15.62:0/3467326198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 msgr2=0x56168c2b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c2b9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 -- 172.21.15.62:0/3467326198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 msgr2=0x56168c1fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c1fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 -- 172.21.15.62:0/3467326198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56168c22fae0 con 0x56168c231830 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3ca000640 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c21f6b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc02f040 tx=0x7fa3bc005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ee30bb41cb197c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.924+0000 7fa3afe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3afe00640 1 -- 172.21.15.62:0/3467326198 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc013bb0 con 0x56168c231830 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3afe00640 1 -- 172.21.15.62:0/3467326198 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3bc013d50 con 0x56168c231830 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3467326198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3bc007e50 con 0x56168c231830 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3afe00640 1 -- 172.21.15.62:0/3467326198 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3bc046050 con 0x56168c231830 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 10 client.5913 fetch_fsmap learned FSMap version 26 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3467326198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x56168c25f4c0 con 0x56168c231830 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3afe00640 1 -- 172.21.15.62:0/3467326198 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc007e50 con 0x56168c231830 2024-04-25T09:07:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3afe00640 1 -- 172.21.15.62:0/3467326198 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa3bc049020 con 0x56168c231830 2024-04-25T09:07:54.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 10 client.5913 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T09:07:54.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 10 client.5913 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:07:54.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 20 client.5913 populate_metadata read hostname 'smithi062' 2024-04-25T09:07:54.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56168c22ffe0 0x56168c2303e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:07:54.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 4 client.5913 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:07:54.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.925+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3467326198 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56168c172500 con 0x56168c22ffe0 2024-04-25T09:07:54.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.926+0000 7fa3caa00640 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56168c22ffe0 0x56168c2303e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:07:54.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.926+0000 7fa3caa00640 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56168c22ffe0 0x56168c2303e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a807ded9995270eb server_cookie=945f7e14c12194cf in_seq=0 out_seq=0 2024-04-25T09:07:54.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.926+0000 7fa3afe00640 10 client.5913 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3afe00640 1 -- 172.21.15.62:0/3467326198 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+696 (crc 0 0 0) 0x56168c172500 con 0x56168c22ffe0 2024-04-25T09:07:54.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3afe00640 10 client.5913 handle_command_reply: tid=0 2024-04-25T09:07:54.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 client.5913 shutdown 2024-04-25T09:07:54.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3467326198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 msgr2=0x56168c21f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c21f6b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc02f040 tx=0x7fa3bc005000 comp rx=0 tx=0).stop 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3467326198 shutdown_connections 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56168c22ffe0 0x56168c2303e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56168c231830 0x56168c21f6b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3afe00640 0 client.5913 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56168c243440 0x56168c1fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 --2- 172.21.15.62:0/3467326198 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56168c206d70 0x56168c2b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3467326198 >> 172.21.15.62:0/3467326198 conn(0x56168c1cea30 msgr2=0x56168c220ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3467326198 shutdown_connections 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 1 -- 172.21.15.62:0/3467326198 wait complete. 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.927+0000 7fa3d4bb8740 20 client.5913 trim_cache size 0 max 16384 2024-04-25T09:07:54.917 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "0819cef4-bb62-488e-99eb-f775ed42a119": { 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/", 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "0819cef4-bb62-488e-99eb-f775ed42a119", 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "" 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "dad1f5dc-b1a1-4567-b5ca-71fb342e55d3": { 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "dad1f5dc-b1a1-4567-b5ca-71fb342e55d3", 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "" 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "f7654421-4864-46b2-b0f5-f404fef1b802": { 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "f7654421-4864-46b2-b0f5-f404fef1b802", 2024-04-25T09:07:54.918 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "" 2024-04-25T09:07:54.919 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:54.919 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:07:54.919 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:07:54.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa3a806b4d0 msgr2=0x7fa3a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa3a806b4d0 0x7fa3a806d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ce2e0 tx=0x7fa3bc009040 comp rx=0 tx=0).stop 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1812830007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 msgr2=0x7fa3cc1f7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3cc1f7ad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009be0 tx=0x7fa3b4004730 comp rx=0 tx=0).stop 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1812830007 shutdown_connections 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa3a806b4d0 0x7fa3a806d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3cc1f44a0 0x7fa3cc1f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa3cc0b58e0 0x7fa3cc0b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 --2- 172.21.15.62:0/1812830007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3b0003a00 0x7fa3cc1f7ad0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1812830007 >> 172.21.15.62:0/1812830007 conn(0x7fa3cc06b210 msgr2=0x7fa3cc0caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1812830007 shutdown_connections 2024-04-25T09:07:54.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:07:54.928+0000 7fa3cbe00640 1 -- 172.21.15.62:0/1812830007 wait complete. 2024-04-25T09:07:58.139 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:07:58.152+0000 7f47c3e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:07:59.934 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:07:59.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:08:00.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.170+0000 7f425f200640 1 Processor -- start 2024-04-25T09:08:00.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.170+0000 7f425f200640 1 -- start start 2024-04-25T09:08:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.171+0000 7f425f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 0x7f42580d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.171+0000 7f425f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f42580d2fb0 0x7f42580ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.171+0000 7f425f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42580ce780 con 0x7f42580d3960 2024-04-25T09:08:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.171+0000 7f425f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42580d3440 con 0x7f42580d2fb0 2024-04-25T09:08:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.171+0000 7f425f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42580ce1c0 con 0x7f425815dd50 2024-04-25T09:08:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.171+0000 7f425d400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f42580d2fb0 0x7f42580ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.172+0000 7f425d400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f42580d2fb0 0x7f42580ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45270/0 (socket says 172.21.15.62:45270) 2024-04-25T09:08:00.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.172+0000 7f425d400640 1 -- 172.21.15.62:0/3048313983 learned_addr learned my addr 172.21.15.62:0/3048313983 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.172+0000 7f425de00640 1 --2- 172.21.15.62:0/3048313983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 0x7f42580d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.172+0000 7f425d400640 1 -- 172.21.15.62:0/3048313983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 msgr2=0x7f42580d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.172+0000 7f425d400640 1 --2- 172.21.15.62:0/3048313983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 0x7f42580d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.172+0000 7f425d400640 1 -- 172.21.15.62:0/3048313983 >> v1:172.21.15.121:6789/0 conn(0x7f42580d3960 legacy=0x7f42580d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.172+0000 7f425d400640 1 -- 172.21.15.62:0/3048313983 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42580ce4a0 con 0x7f42580d2fb0 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.173+0000 7f425d400640 1 --2- 172.21.15.62:0/3048313983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42580d2fb0 0x7f42580ce9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4248009870 tx=0x7f424802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3f33d9cbcff7db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425ca00640 1 -- 172.21.15.62:0/3048313983 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248037070 con 0x7f42580d2fb0 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425ca00640 1 -- 172.21.15.62:0/3048313983 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4248032440 con 0x7f42580d2fb0 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425ca00640 1 -- 172.21.15.62:0/3048313983 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248032810 con 0x7f42580d2fb0 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425f200640 1 -- 172.21.15.62:0/3048313983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42580d2fb0 msgr2=0x7f42580ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425f200640 1 --2- 172.21.15.62:0/3048313983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42580d2fb0 0x7f42580ce9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4248009870 tx=0x7f424802a210 comp rx=0 tx=0).stop 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425f200640 1 -- 172.21.15.62:0/3048313983 shutdown_connections 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425f200640 1 --2- 172.21.15.62:0/3048313983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42580d2fb0 0x7f42580ce9c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425f200640 1 --2- 172.21.15.62:0/3048313983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 0x7f42580d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425f200640 1 -- 172.21.15.62:0/3048313983 >> 172.21.15.62:0/3048313983 conn(0x7f425806b000 msgr2=0x7f425806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425f200640 1 -- 172.21.15.62:0/3048313983 shutdown_connections 2024-04-25T09:08:00.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.174+0000 7f425f200640 1 -- 172.21.15.62:0/3048313983 wait complete. 2024-04-25T09:08:00.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.176+0000 7f425f200640 1 Processor -- start 2024-04-25T09:08:00.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.176+0000 7f425f200640 1 -- start start 2024-04-25T09:08:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.177+0000 7f425f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42580d3960 0x7f42580c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.177+0000 7f425f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 0x7f42581f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.177+0000 7f425f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4258150160 0x7f42581f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.177+0000 7f425f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42580b6b00 con 0x7f4258150160 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.177+0000 7f425f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42580cef00 con 0x7f42580d3960 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.177+0000 7f425f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42580b6c80 con 0x7f425815dd50 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4258150160 0x7f42581f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4258150160 0x7f42581f7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45280/0 (socket says 172.21.15.62:45280) 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425e800640 1 -- 172.21.15.62:0/1594953527 learned_addr learned my addr 172.21.15.62:0/1594953527 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425d400640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 0x7f42581f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425de00640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42580d3960 0x7f42580c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425e800640 1 -- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 msgr2=0x7f42581f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425e800640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 0x7f42581f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425e800640 1 -- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42580d3960 msgr2=0x7f42580c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425e800640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42580d3960 0x7f42580c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.178+0000 7f425e800640 1 -- 172.21.15.62:0/1594953527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42580b2580 con 0x7f4258150160 2024-04-25T09:08:00.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.179+0000 7f425e800640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4258150160 0x7f42581f7720 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f425400c7e0 tx=0x7f425400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.179+0000 7f4252000640 1 -- 172.21.15.62:0/1594953527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254004b60 con 0x7f4258150160 2024-04-25T09:08:00.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.179+0000 7f4252000640 1 -- 172.21.15.62:0/1594953527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4254004d00 con 0x7f4258150160 2024-04-25T09:08:00.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.179+0000 7f425f200640 1 -- 172.21.15.62:0/1594953527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42581506d0 con 0x7f4258150160 2024-04-25T09:08:00.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.179+0000 7f4252000640 1 -- 172.21.15.62:0/1594953527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425400aa30 con 0x7f4258150160 2024-04-25T09:08:00.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.179+0000 7f425f200640 1 -- 172.21.15.62:0/1594953527 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4258150c90 con 0x7f4258150160 2024-04-25T09:08:00.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.182+0000 7f4252000640 1 -- 172.21.15.62:0/1594953527 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f425400abd0 con 0x7f4258150160 2024-04-25T09:08:00.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.183+0000 7f4252000640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f423c06b500 0x7f423c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.183+0000 7f4252000640 1 -- 172.21.15.62:0/1594953527 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f425408a500 con 0x7f4258150160 2024-04-25T09:08:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.184+0000 7f4263641740 1 Processor -- start 2024-04-25T09:08:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.184+0000 7f4263641740 1 -- start start 2024-04-25T09:08:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.184+0000 7f425de00640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f423c06b500 0x7f423c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f4263641740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 0x55fe78eb5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f4263641740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78eb4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f4263641740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78edecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f4263641740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78e17c70 con 0x55fe78ede830 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f4263641740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78dc0cf0 con 0x55fe78ef0440 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f4263641740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe78e13d10 con 0x55fe78eb3d70 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78edecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78edecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45286/0 (socket says 172.21.15.62:45286) 2024-04-25T09:08:00.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425e800640 1 -- 172.21.15.62:0/97226910 learned_addr learned my addr 172.21.15.62:0/97226910 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425de00640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f423c06b500 0x7f423c06d9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f424c006fd0 tx=0x7f424c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425e800640 1 -- 172.21.15.62:0/97226910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 msgr2=0x55fe78eb5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425e800640 1 --2- 172.21.15.62:0/97226910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 0x55fe78eb5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425e800640 1 -- 172.21.15.62:0/97226910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 msgr2=0x55fe78eb4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425e800640 1 --2- 172.21.15.62:0/97226910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78eb4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.185+0000 7f425e800640 1 -- 172.21.15.62:0/97226910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe78bf67d0 con 0x55fe78ede830 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.186+0000 7f425e800640 1 --2- 172.21.15.62:0/97226910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78edecb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f42540888d0 tx=0x7f4254058850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec109b06bc4ef60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.186+0000 7f4246600640 1 -- 172.21.15.62:0/97226910 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425405cd30 con 0x55fe78ede830 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.186+0000 7f4246600640 1 -- 172.21.15.62:0/97226910 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540612f0 con 0x55fe78ede830 2024-04-25T09:08:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.186+0000 7f4246600640 1 -- 172.21.15.62:0/97226910 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254056ee0 con 0x55fe78ede830 2024-04-25T09:08:00.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.186+0000 7f4263641740 1 -- 172.21.15.62:0/97226910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 msgr2=0x55fe78edecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.186+0000 7f4263641740 1 --2- 172.21.15.62:0/97226910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78edecb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f42540888d0 tx=0x7f4254058850 comp rx=0 tx=0).stop 2024-04-25T09:08:00.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.187+0000 7f4263641740 1 -- 172.21.15.62:0/97226910 shutdown_connections 2024-04-25T09:08:00.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.187+0000 7f4263641740 1 --2- 172.21.15.62:0/97226910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78edecb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.187+0000 7f4263641740 1 --2- 172.21.15.62:0/97226910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78eb4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.187+0000 7f4263641740 1 --2- 172.21.15.62:0/97226910 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 0x55fe78eb5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.187+0000 7f4263641740 1 -- 172.21.15.62:0/97226910 >> 172.21.15.62:0/97226910 conn(0x55fe78e7ba30 msgr2=0x55fe78ef9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.187+0000 7f4263641740 1 -- 172.21.15.62:0/97226910 shutdown_connections 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.187+0000 7f4263641740 1 -- 172.21.15.62:0/97226910 wait complete. 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.187+0000 7f4263641740 1 -- start start 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.188+0000 7f4263641740 1 Processor -- start 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f4263641740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78eb3d70 0x55fe78ee1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f4263641740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 0x55fe78edb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f4263641740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78f092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f4263641740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78e17c70 con 0x55fe78eb3d70 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f4263641740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78db63a0 con 0x55fe78ef0440 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f4263641740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe78f0a0f0 con 0x55fe78ede830 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f4263641740 10 client.0 _ll_register_callbacks cb 0x55fe78dd3b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f425de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78eb3d70 0x55fe78ee1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f425de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78eb3d70 0x55fe78ee1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45292/0 (socket says 172.21.15.62:45292) 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f425de00640 1 -- 172.21.15.62:0/1886673677 learned_addr learned my addr 172.21.15.62:0/1886673677 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f425d400640 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 0x55fe78edb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.189+0000 7f425e800640 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78f092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f425de00640 1 -- 172.21.15.62:0/1886673677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 msgr2=0x55fe78edb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f425de00640 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 0x55fe78edb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f425de00640 1 -- 172.21.15.62:0/1886673677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 msgr2=0x55fe78f092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f425de00640 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78f092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f425de00640 1 -- 172.21.15.62:0/1886673677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe78b2c3f0 con 0x55fe78eb3d70 2024-04-25T09:08:00.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f425de00640 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78eb3d70 0x55fe78ee1250 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f424c0095d0 tx=0x7f424c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a9c72a6bfeffd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f423aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:00.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f423aa00640 1 -- 172.21.15.62:0/1886673677 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c025060 con 0x55fe78eb3d70 2024-04-25T09:08:00.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f423aa00640 1 -- 172.21.15.62:0/1886673677 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f424c015d50 con 0x55fe78eb3d70 2024-04-25T09:08:00.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.190+0000 7f4263641740 1 -- 172.21.15.62:0/1886673677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42540612f0 con 0x55fe78eb3d70 2024-04-25T09:08:00.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f423aa00640 1 -- 172.21.15.62:0/1886673677 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f424c02b050 con 0x55fe78eb3d70 2024-04-25T09:08:00.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f4263641740 10 client.5925 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f4263641740 1 -- 172.21.15.62:0/1886673677 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55fe78aec0f0 con 0x55fe78eb3d70 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f423aa00640 1 -- 172.21.15.62:0/1886673677 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540612f0 con 0x55fe78eb3d70 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f423aa00640 1 -- 172.21.15.62:0/1886673677 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f424c02e020 con 0x55fe78eb3d70 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f4263641740 10 client.5925 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f4263641740 10 client.5925 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f4263641740 20 client.5925 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f4263641740 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55fe78f09820 0x55fe78ef6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f4263641740 4 client.5925 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.191+0000 7f4263641740 1 -- 172.21.15.62:0/1886673677 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe78e251b0 con 0x55fe78f09820 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.192+0000 7f425d400640 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55fe78f09820 0x55fe78ef6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.192+0000 7f425d400640 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55fe78f09820 0x55fe78ef6470 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b9354575cc192eb server_cookie=714a60364ebd6fb5 in_seq=0 out_seq=0 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.192+0000 7f423aa00640 10 client.5925 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f423aa00640 1 -- 172.21.15.62:0/1886673677 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe78e251b0 con 0x55fe78f09820 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f423aa00640 10 client.5925 handle_command_reply: tid=0 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 client.5925 shutdown 2024-04-25T09:08:00.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 -- 172.21.15.62:0/1886673677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78eb3d70 msgr2=0x55fe78ee1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78eb3d70 0x55fe78ee1250 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f424c0095d0 tx=0x7f424c00ed20 comp rx=0 tx=0).stop 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 -- 172.21.15.62:0/1886673677 shutdown_connections 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55fe78f09820 0x55fe78ef6470 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78f092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 0x55fe78edb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 --2- 172.21.15.62:0/1886673677 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78eb3d70 0x55fe78ee1250 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f423aa00640 0 client.5925 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 -- 172.21.15.62:0/1886673677 >> 172.21.15.62:0/1886673677 conn(0x55fe78e7ba30 msgr2=0x55fe78efccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 -- 172.21.15.62:0/1886673677 shutdown_connections 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.193+0000 7f4263641740 1 -- 172.21.15.62:0/1886673677 wait complete. 2024-04-25T09:08:00.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.194+0000 7f4263641740 20 client.5925 trim_cache size 0 max 16384 2024-04-25T09:08:00.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.211+0000 7f4263641740 1 Processor -- start 2024-04-25T09:08:00.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.211+0000 7f4263641740 1 -- start start 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f4263641740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78eb3d70 0x55fe78edb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f4263641740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 0x55fe78eda720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f4263641740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f4263641740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42581fff60 con 0x55fe78ef0440 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f4263641740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258151960 con 0x55fe78eb3d70 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f4263641740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42580c9280 con 0x55fe78ede830 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f425e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f425e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45308/0 (socket says 172.21.15.62:45308) 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.212+0000 7f425e800640 1 -- 172.21.15.62:0/2164660385 learned_addr learned my addr 172.21.15.62:0/2164660385 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f425e800640 1 -- 172.21.15.62:0/2164660385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 msgr2=0x55fe78eda720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f425de00640 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78eb3d70 0x55fe78edb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f425d400640 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 0x55fe78eda720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f425e800640 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 0x55fe78eda720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f425e800640 1 -- 172.21.15.62:0/2164660385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78eb3d70 msgr2=0x55fe78edb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f425e800640 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78eb3d70 0x55fe78edb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f425e800640 1 -- 172.21.15.62:0/2164660385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe78f09650 con 0x55fe78ef0440 2024-04-25T09:08:00.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f425e800640 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f425400c7b0 tx=0x7f425408a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b12bb38c9395b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f4245c00640 1 -- 172.21.15.62:0/2164660385 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254076070 con 0x55fe78ef0440 2024-04-25T09:08:00.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f4245c00640 1 -- 172.21.15.62:0/2164660385 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540612f0 con 0x55fe78ef0440 2024-04-25T09:08:00.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.213+0000 7f4245c00640 1 -- 172.21.15.62:0/2164660385 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425405cd30 con 0x55fe78ef0440 2024-04-25T09:08:00.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 -- 172.21.15.62:0/2164660385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ef0440 msgr2=0x55fe78edac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f425400c7b0 tx=0x7f425408a050 comp rx=0 tx=0).stop 2024-04-25T09:08:00.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 -- 172.21.15.62:0/2164660385 shutdown_connections 2024-04-25T09:08:00.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78ede830 0x55fe78eda720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 --2- 172.21.15.62:0/2164660385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78eb3d70 0x55fe78edb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 -- 172.21.15.62:0/2164660385 >> 172.21.15.62:0/2164660385 conn(0x55fe78e7ba30 msgr2=0x55fe78ef5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 -- 172.21.15.62:0/2164660385 shutdown_connections 2024-04-25T09:08:00.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.214+0000 7f4263641740 1 -- 172.21.15.62:0/2164660385 wait complete. 2024-04-25T09:08:00.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.215+0000 7f4263641740 1 -- start start 2024-04-25T09:08:00.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.215+0000 7f4263641740 1 Processor -- start 2024-04-25T09:08:00.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.216+0000 7f4263641740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 0x55fe78eadbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.216+0000 7f4263641740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78eda720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.216+0000 7f4263641740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.216+0000 7f4263641740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe78ee17b0 con 0x55fe78ede830 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.216+0000 7f4263641740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42580b60e0 con 0x55fe78ef0440 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.216+0000 7f4263641740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe78f1afb0 con 0x55fe78eb3d70 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.216+0000 7f4263641740 10 client.0 _ll_register_callbacks cb 0x7f424c009180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78eda720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78eda720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45312/0 (socket says 172.21.15.62:45312) 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425d400640 1 -- 172.21.15.62:0/2398992083 learned_addr learned my addr 172.21.15.62:0/2398992083 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425de00640 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 0x55fe78eadbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425e800640 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425d400640 1 -- 172.21.15.62:0/2398992083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 msgr2=0x55fe78eadbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425d400640 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 0x55fe78eadbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425d400640 1 -- 172.21.15.62:0/2398992083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 msgr2=0x55fe78edac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425d400640 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.217+0000 7f425d400640 1 -- 172.21.15.62:0/2398992083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe78f18920 con 0x55fe78ede830 2024-04-25T09:08:00.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.218+0000 7f425d400640 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78eda720 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4248009630 tx=0x7f42480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2943b03791a281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.218+0000 7f423aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.221+0000 7f423aa00640 1 -- 172.21.15.62:0/2398992083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248037040 con 0x55fe78ede830 2024-04-25T09:08:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.221+0000 7f423aa00640 1 -- 172.21.15.62:0/2398992083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f42480422e0 con 0x55fe78ede830 2024-04-25T09:08:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.221+0000 7f4263641740 1 -- 172.21.15.62:0/2398992083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42540612f0 con 0x55fe78ede830 2024-04-25T09:08:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.221+0000 7f423aa00640 1 -- 172.21.15.62:0/2398992083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f424804d050 con 0x55fe78ede830 2024-04-25T09:08:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.221+0000 7f4263641740 10 client.5931 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.221+0000 7f4263641740 1 -- 172.21.15.62:0/2398992083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55fe78eb3910 con 0x55fe78ede830 2024-04-25T09:08:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.221+0000 7f423aa00640 1 -- 172.21.15.62:0/2398992083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540612f0 con 0x55fe78ede830 2024-04-25T09:08:00.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.222+0000 7f423aa00640 1 -- 172.21.15.62:0/2398992083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4248050020 con 0x55fe78ede830 2024-04-25T09:08:00.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.222+0000 7f4263641740 10 client.5931 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:00.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.222+0000 7f4263641740 10 client.5931 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:00.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.222+0000 7f4263641740 20 client.5931 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:00.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.222+0000 7f4263641740 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55fe78f46f00 0x55fe78f19ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.222+0000 7f4263641740 4 client.5931 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:08:00.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.222+0000 7f4263641740 1 -- 172.21.15.62:0/2398992083 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe78eb4fc0 con 0x55fe78f46f00 2024-04-25T09:08:00.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.222+0000 7f425de00640 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55fe78f46f00 0x55fe78f19ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f423aa00640 10 client.5931 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f425de00640 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55fe78f46f00 0x55fe78f19ea0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24eac364ce175f0 server_cookie=7f0b35ca75c5f698 in_seq=0 out_seq=0 2024-04-25T09:08:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f423aa00640 1 -- 172.21.15.62:0/2398992083 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fe78eb4fc0 con 0x55fe78f46f00 2024-04-25T09:08:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f423aa00640 10 client.5931 handle_command_reply: tid=0 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 client.5931 shutdown 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 -- 172.21.15.62:0/2398992083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 msgr2=0x55fe78eda720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78eda720 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4248009630 tx=0x7f42480046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 -- 172.21.15.62:0/2398992083 shutdown_connections 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55fe78f46f00 0x55fe78f19ea0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fe78ede830 0x55fe78eda720 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe78ef0440 0x55fe78edac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 --2- 172.21.15.62:0/2398992083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fe78eb3d70 0x55fe78eadbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f423aa00640 0 client.5931 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 -- 172.21.15.62:0/2398992083 >> 172.21.15.62:0/2398992083 conn(0x55fe78e7ba30 msgr2=0x55fe78f58650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 -- 172.21.15.62:0/2398992083 shutdown_connections 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 1 -- 172.21.15.62:0/2398992083 wait complete. 2024-04-25T09:08:00.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.223+0000 7f4263641740 20 client.5931 trim_cache size 0 max 16384 2024-04-25T09:08:00.214 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:00.214 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-25T09:08:00.214 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:08:00.214 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 -- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f423c06b500 msgr2=0x7f423c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f423c06b500 0x7f423c06d9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f424c006fd0 tx=0x7f424c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 -- 172.21.15.62:0/1594953527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4258150160 msgr2=0x7f42581f7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4258150160 0x7f42581f7720 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f425400c7e0 tx=0x7f425400cca0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 -- 172.21.15.62:0/1594953527 shutdown_connections 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f423c06b500 0x7f423c06d9c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4258150160 0x7f42581f7720 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f425815dd50 0x7f42581f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 --2- 172.21.15.62:0/1594953527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42580d3960 0x7f42580c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 -- 172.21.15.62:0/1594953527 >> 172.21.15.62:0/1594953527 conn(0x7f425806b000 msgr2=0x7f425815cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 -- 172.21.15.62:0/1594953527 shutdown_connections 2024-04-25T09:08:00.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.224+0000 7f425f200640 1 -- 172.21.15.62:0/1594953527 wait complete. 2024-04-25T09:08:00.230 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:08:00.230 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:00.230 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:08:00.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.438+0000 7f8897200640 1 Processor -- start 2024-04-25T09:08:00.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.438+0000 7f8897200640 1 -- start start 2024-04-25T09:08:00.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.440+0000 7f8897200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8890072b70 0x7f88900b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.440+0000 7f8897200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f88900b6790 0x7f88900cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.440+0000 7f8897200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889015f650 con 0x7f889015b870 2024-04-25T09:08:00.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.440+0000 7f8897200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889015f400 con 0x7f88900b6790 2024-04-25T09:08:00.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.440+0000 7f8897200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88900b6c00 con 0x7f8890072b70 2024-04-25T09:08:00.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.440+0000 7f8896800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f88900b6790 0x7f88900cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.440+0000 7f8896800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f88900b6790 0x7f88900cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45318/0 (socket says 172.21.15.62:45318) 2024-04-25T09:08:00.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.440+0000 7f8896800640 1 -- 172.21.15.62:0/2222789310 learned_addr learned my addr 172.21.15.62:0/2222789310 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.441+0000 7f8895400640 1 --2- 172.21.15.62:0/2222789310 >> v2:172.21.15.121:3300/0 conn(0x7f8890072b70 0x7f88900b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.441+0000 7f8896800640 1 -- 172.21.15.62:0/2222789310 >> v2:172.21.15.121:3300/0 conn(0x7f8890072b70 msgr2=0x7f88900b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.441+0000 7f8896800640 1 --2- 172.21.15.62:0/2222789310 >> v2:172.21.15.121:3300/0 conn(0x7f8890072b70 0x7f88900b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.441+0000 7f8896800640 1 -- 172.21.15.62:0/2222789310 >> v1:172.21.15.121:6789/0 conn(0x7f889015b870 legacy=0x7f8890072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.441+0000 7f8896800640 1 -- 172.21.15.62:0/2222789310 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889015f930 con 0x7f88900b6790 2024-04-25T09:08:00.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.442+0000 7f8896800640 1 --2- 172.21.15.62:0/2222789310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88900b6790 0x7f88900cfb80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f88900701f0 tx=0x7f888c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15fa2ac35e6d371 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.442+0000 7f8894a00640 1 -- 172.21.15.62:0/2222789310 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c036070 con 0x7f88900b6790 2024-04-25T09:08:00.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.442+0000 7f8894a00640 1 -- 172.21.15.62:0/2222789310 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c031600 con 0x7f88900b6790 2024-04-25T09:08:00.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.442+0000 7f8894a00640 1 -- 172.21.15.62:0/2222789310 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c0319d0 con 0x7f88900b6790 2024-04-25T09:08:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.443+0000 7f8897200640 1 -- 172.21.15.62:0/2222789310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88900b6790 msgr2=0x7f88900cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.443+0000 7f8897200640 1 --2- 172.21.15.62:0/2222789310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88900b6790 0x7f88900cfb80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f88900701f0 tx=0x7f888c02a660 comp rx=0 tx=0).stop 2024-04-25T09:08:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.443+0000 7f8897200640 1 -- 172.21.15.62:0/2222789310 shutdown_connections 2024-04-25T09:08:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.443+0000 7f8897200640 1 --2- 172.21.15.62:0/2222789310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88900b6790 0x7f88900cfb80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.443+0000 7f8897200640 1 --2- 172.21.15.62:0/2222789310 >> v2:172.21.15.121:3300/0 conn(0x7f8890072b70 0x7f88900b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.443+0000 7f8897200640 1 -- 172.21.15.62:0/2222789310 >> 172.21.15.62:0/2222789310 conn(0x7f88900672d0 msgr2=0x7f889006e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.443+0000 7f8897200640 1 -- 172.21.15.62:0/2222789310 shutdown_connections 2024-04-25T09:08:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.443+0000 7f8897200640 1 -- 172.21.15.62:0/2222789310 wait complete. 2024-04-25T09:08:00.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.445+0000 7f8897200640 1 Processor -- start 2024-04-25T09:08:00.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.445+0000 7f8897200640 1 -- start start 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.446+0000 7f8897200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8890072b70 0x7f88900aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.446+0000 7f8897200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88900b6790 0x7f8890142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.446+0000 7f8897200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f889015b870 0x7f8890143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.446+0000 7f8897200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88901601d0 con 0x7f889015b870 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.446+0000 7f8897200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889015fef0 con 0x7f88900b6790 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.446+0000 7f8897200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889015bc50 con 0x7f8890072b70 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f889015b870 0x7f8890143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f889015b870 0x7f8890143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45332/0 (socket says 172.21.15.62:45332) 2024-04-25T09:08:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 -- 172.21.15.62:0/425571931 learned_addr learned my addr 172.21.15.62:0/425571931 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8895400640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88900b6790 0x7f8890142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8895e00640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8890072b70 0x7f88900aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 -- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8890072b70 msgr2=0x7f88900aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8890072b70 0x7f88900aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 -- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88900b6790 msgr2=0x7f8890142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88900b6790 0x7f8890142d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 -- 172.21.15.62:0/425571931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88901e8330 con 0x7f889015b870 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8895e00640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8890072b70 0x7f88900aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.447+0000 7f8896800640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f889015b870 0x7f8890143240 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f888c009f40 tx=0x7f888c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.448+0000 7f888a000640 1 -- 172.21.15.62:0/425571931 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c036050 con 0x7f889015b870 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.448+0000 7f888a000640 1 -- 172.21.15.62:0/425571931 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c042070 con 0x7f889015b870 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.448+0000 7f888a000640 1 -- 172.21.15.62:0/425571931 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c009510 con 0x7f889015b870 2024-04-25T09:08:00.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.448+0000 7f8897200640 1 -- 172.21.15.62:0/425571931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88901e8610 con 0x7f889015b870 2024-04-25T09:08:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.448+0000 7f8897200640 1 -- 172.21.15.62:0/425571931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88901e8bd0 con 0x7f889015b870 2024-04-25T09:08:00.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.450+0000 7f888a000640 1 -- 172.21.15.62:0/425571931 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f888c007bd0 con 0x7f889015b870 2024-04-25T09:08:00.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.452+0000 7f888a000640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f887406b4d0 0x7f887406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.452+0000 7f888a000640 1 -- 172.21.15.62:0/425571931 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f888c0ab2a0 con 0x7f889015b870 2024-04-25T09:08:00.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.452+0000 7f8895e00640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f887406b4d0 0x7f887406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.452+0000 7f889b779740 1 Processor -- start 2024-04-25T09:08:00.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.453+0000 7f889b779740 1 -- start start 2024-04-25T09:08:00.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.453+0000 7f889b779740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bbb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.453+0000 7f889b779740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 0x55d158bd8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.453+0000 7f889b779740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158b90cf0 0x55d158bd6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.453+0000 7f889b779740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158af3d00 con 0x55d158bbb010 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.453+0000 7f889b779740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158a9cf50 con 0x55d158b90cf0 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.453+0000 7f889b779740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d158aefda0 con 0x55d158bd88b0 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f887406b4d0 0x7f887406d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f888400a830 tx=0x7f8884008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bbb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bbb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45338/0 (socket says 172.21.15.62:45338) 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 -- 172.21.15.62:0/556506479 learned_addr learned my addr 172.21.15.62:0/556506479 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895400640 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 0x55d158bd8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8896800640 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158b90cf0 0x55d158bd6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 -- 172.21.15.62:0/556506479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 msgr2=0x55d158bd8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 0x55d158bd8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 -- 172.21.15.62:0/556506479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158b90cf0 msgr2=0x55d158bd6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158b90cf0 0x55d158bd6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.454+0000 7f8895e00640 1 -- 172.21.15.62:0/556506479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d158ac0130 con 0x55d158bbb010 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.455+0000 7f8895e00640 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bbb3f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f888400ca60 tx=0x7f888400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea05925cff85a811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.455+0000 7f887e600640 1 -- 172.21.15.62:0/556506479 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884013db0 con 0x55d158bbb010 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.455+0000 7f887e600640 1 -- 172.21.15.62:0/556506479 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888400ae60 con 0x55d158bbb010 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.455+0000 7f887e600640 1 -- 172.21.15.62:0/556506479 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884004260 con 0x55d158bbb010 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 -- 172.21.15.62:0/556506479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 msgr2=0x55d158bbb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bbb3f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f888400ca60 tx=0x7f888400f000 comp rx=0 tx=0).stop 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 -- 172.21.15.62:0/556506479 shutdown_connections 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158b90cf0 0x55d158bd6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 0x55d158bd8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 --2- 172.21.15.62:0/556506479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bbb3f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 -- 172.21.15.62:0/556506479 >> 172.21.15.62:0/556506479 conn(0x55d158b8d620 msgr2=0x55d158bd5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 -- 172.21.15.62:0/556506479 shutdown_connections 2024-04-25T09:08:00.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.456+0000 7f889b779740 1 -- 172.21.15.62:0/556506479 wait complete. 2024-04-25T09:08:00.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.457+0000 7f889b779740 1 -- start start 2024-04-25T09:08:00.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.457+0000 7f889b779740 1 Processor -- start 2024-04-25T09:08:00.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f889b779740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f889b779740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 0x55d158baa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f889b779740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bb6c70 0x55d158baa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f889b779740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158b91a90 con 0x55d158bbb010 2024-04-25T09:08:00.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f889b779740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158a92530 con 0x55d158bd88b0 2024-04-25T09:08:00.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f889b779740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d158b8b550 con 0x55d158bb6c70 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f889b779740 10 client.0 _ll_register_callbacks cb 0x55d158aeb430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f8895e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f8895e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bba1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45344/0 (socket says 172.21.15.62:45344) 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.458+0000 7f8895e00640 1 -- 172.21.15.62:0/3940174147 learned_addr learned my addr 172.21.15.62:0/3940174147 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8895e00640 1 -- 172.21.15.62:0/3940174147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bb6c70 msgr2=0x55d158baa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8895e00640 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bb6c70 0x55d158baa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8895e00640 1 -- 172.21.15.62:0/3940174147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 msgr2=0x55d158baa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8895e00640 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 0x55d158baa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8895e00640 1 -- 172.21.15.62:0/3940174147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d158abadd0 con 0x55d158bbb010 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8895e00640 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bba1d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f888400a440 tx=0x7f8884004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7b2d2aefbb5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8872a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8872a00640 1 -- 172.21.15.62:0/3940174147 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88840035f0 con 0x55d158bbb010 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f8872a00640 1 -- 172.21.15.62:0/3940174147 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8884003790 con 0x55d158bbb010 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.459+0000 7f889b779740 1 -- 172.21.15.62:0/3940174147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f888400ae60 con 0x55d158bbb010 2024-04-25T09:08:00.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f8872a00640 1 -- 172.21.15.62:0/3940174147 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8884003990 con 0x55d158bbb010 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f889b779740 10 client.5943 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f889b779740 1 -- 172.21.15.62:0/3940174147 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55d1588083f0 con 0x55d158bbb010 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f8872a00640 1 -- 172.21.15.62:0/3940174147 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888400ae60 con 0x55d158bbb010 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f8872a00640 1 -- 172.21.15.62:0/3940174147 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8884003b30 con 0x55d158bbb010 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f889b779740 10 client.5943 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f889b779740 10 client.5943 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f889b779740 20 client.5943 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f889b779740 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55d158be66b0 0x55d158bbcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f889b779740 4 client.5943 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:00.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f889b779740 1 -- 172.21.15.62:0/3940174147 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d158b01240 con 0x55d158be66b0 2024-04-25T09:08:00.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.460+0000 7f8895400640 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55d158be66b0 0x55d158bbcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.461+0000 7f8895400640 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55d158be66b0 0x55d158bbcf10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a11f565f9c448891 server_cookie=4b132a1a82170ec4 in_seq=0 out_seq=0 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.461+0000 7f8872a00640 10 client.5943 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f8872a00640 1 -- 172.21.15.62:0/3940174147 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d158b01240 con 0x55d158be66b0 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f8872a00640 10 client.5943 handle_command_reply: tid=0 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 client.5943 shutdown 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 -- 172.21.15.62:0/3940174147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 msgr2=0x55d158bba1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bba1d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f888400a440 tx=0x7f8884004680 comp rx=0 tx=0).stop 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 -- 172.21.15.62:0/3940174147 shutdown_connections 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55d158be66b0 0x55d158bbcf10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bb6c70 0x55d158baa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 0x55d158baa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 --2- 172.21.15.62:0/3940174147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158bba1d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 -- 172.21.15.62:0/3940174147 >> 172.21.15.62:0/3940174147 conn(0x55d158b90cf0 msgr2=0x55d158bd93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f8872a00640 0 client.5943 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:00.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 -- 172.21.15.62:0/3940174147 shutdown_connections 2024-04-25T09:08:00.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.462+0000 7f889b779740 1 -- 172.21.15.62:0/3940174147 wait complete. 2024-04-25T09:08:00.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.463+0000 7f889b779740 20 client.5943 trim_cache size 0 max 16384 2024-04-25T09:08:00.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.476+0000 7f889b779740 1 Processor -- start 2024-04-25T09:08:00.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.476+0000 7f889b779740 1 -- start start 2024-04-25T09:08:00.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f889b779740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bb6c70 0x55d158bd4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f889b779740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bbb010 0x55d158be8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f889b779740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 0x55d158be9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f889b779740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88901fffc0 con 0x55d158bb6c70 2024-04-25T09:08:00.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f889b779740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88901556e0 con 0x55d158bd88b0 2024-04-25T09:08:00.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f889b779740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889015cbc0 con 0x55d158bbb010 2024-04-25T09:08:00.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f8895e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bb6c70 0x55d158bd4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f8895e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bb6c70 0x55d158bd4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45346/0 (socket says 172.21.15.62:45346) 2024-04-25T09:08:00.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.477+0000 7f8895e00640 1 -- 172.21.15.62:0/3252139323 learned_addr learned my addr 172.21.15.62:0/3252139323 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f8895400640 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bbb010 0x55d158be8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f8896800640 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 0x55d158be9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f8895e00640 1 -- 172.21.15.62:0/3252139323 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bbb010 msgr2=0x55d158be8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f8895e00640 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bbb010 0x55d158be8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f8895e00640 1 -- 172.21.15.62:0/3252139323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 msgr2=0x55d158be9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f8895e00640 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 0x55d158be9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f8895e00640 1 -- 172.21.15.62:0/3252139323 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1588d27d0 con 0x55d158bb6c70 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f8895e00640 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bb6c70 0x55d158bd4ca0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8884013c50 tx=0x7f8884026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cab1bad38bccff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f887dc00640 1 -- 172.21.15.62:0/3252139323 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884040070 con 0x55d158bb6c70 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f887dc00640 1 -- 172.21.15.62:0/3252139323 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888400ae70 con 0x55d158bb6c70 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.478+0000 7f887dc00640 1 -- 172.21.15.62:0/3252139323 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888403b480 con 0x55d158bb6c70 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 -- 172.21.15.62:0/3252139323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bb6c70 msgr2=0x55d158bd4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bb6c70 0x55d158bd4ca0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8884013c50 tx=0x7f8884026000 comp rx=0 tx=0).stop 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 -- 172.21.15.62:0/3252139323 shutdown_connections 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bd88b0 0x55d158be9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bbb010 0x55d158be8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 --2- 172.21.15.62:0/3252139323 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bb6c70 0x55d158bd4ca0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 -- 172.21.15.62:0/3252139323 >> 172.21.15.62:0/3252139323 conn(0x55d158b90cf0 msgr2=0x55d158b910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 -- 172.21.15.62:0/3252139323 shutdown_connections 2024-04-25T09:08:00.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.479+0000 7f889b779740 1 -- 172.21.15.62:0/3252139323 wait complete. 2024-04-25T09:08:00.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.480+0000 7f889b779740 1 -- start start 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.480+0000 7f889b779740 1 Processor -- start 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f889b779740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bb6c70 0x55d158bd45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f889b779740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158c39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f889b779740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 0x55d158be8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f889b779740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158b8e490 con 0x55d158bbb010 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f889b779740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88901550e0 con 0x55d158bb6c70 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f889b779740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d158bd4ae0 con 0x55d158bd88b0 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f889b779740 10 client.0 _ll_register_callbacks cb 0x55d158b8ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f8895400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158c39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f8895400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158c39ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45354/0 (socket says 172.21.15.62:45354) 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.481+0000 7f8895400640 1 -- 172.21.15.62:0/4152609880 learned_addr learned my addr 172.21.15.62:0/4152609880 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8895400640 1 -- 172.21.15.62:0/4152609880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 msgr2=0x55d158be8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8895400640 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 0x55d158be8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8895e00640 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bb6c70 0x55d158bd45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8895400640 1 -- 172.21.15.62:0/4152609880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bb6c70 msgr2=0x55d158bd45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8895400640 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bb6c70 0x55d158bd45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8895400640 1 -- 172.21.15.62:0/4152609880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d158c3a220 con 0x55d158bbb010 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8895e00640 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bb6c70 0x55d158bd45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8895400640 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158c39ce0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8880004770 tx=0x7f8880017440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba8b47e53214617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8872a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8872a00640 1 -- 172.21.15.62:0/4152609880 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880017b40 con 0x55d158bbb010 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8872a00640 1 -- 172.21.15.62:0/4152609880 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8880017ce0 con 0x55d158bbb010 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f889b779740 1 -- 172.21.15.62:0/4152609880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f888400ae70 con 0x55d158bbb010 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f8872a00640 1 -- 172.21.15.62:0/4152609880 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888002a050 con 0x55d158bbb010 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f889b779740 10 client.5949 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.482+0000 7f889b779740 1 -- 172.21.15.62:0/4152609880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55d158bcc600 con 0x55d158bbb010 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f8872a00640 1 -- 172.21.15.62:0/4152609880 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888400ae70 con 0x55d158bbb010 2024-04-25T09:08:00.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f8872a00640 1 -- 172.21.15.62:0/4152609880 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f888002d020 con 0x55d158bbb010 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f889b779740 10 client.5949 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f889b779740 10 client.5949 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f889b779740 20 client.5949 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f889b779740 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55d158be9500 0x55d158bcb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f889b779740 4 client.5949 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f889b779740 1 -- 172.21.15.62:0/4152609880 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d158afb590 con 0x55d158be9500 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.483+0000 7f8895e00640 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55d158be9500 0x55d158bcb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f8895e00640 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55d158be9500 0x55d158bcb7b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eec5b0622e86dd1 server_cookie=6a372850ed971e8 in_seq=0 out_seq=0 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f8872a00640 10 client.5949 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f8872a00640 1 -- 172.21.15.62:0/4152609880 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55d158afb590 con 0x55d158be9500 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f8872a00640 10 client.5949 handle_command_reply: tid=0 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 client.5949 shutdown 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 -- 172.21.15.62:0/4152609880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 msgr2=0x55d158c39ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158c39ce0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8880004770 tx=0x7f8880017440 comp rx=0 tx=0).stop 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 -- 172.21.15.62:0/4152609880 shutdown_connections 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55d158be9500 0x55d158bcb7b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d158bbb010 0x55d158c39ce0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d158bd88b0 0x55d158be8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 --2- 172.21.15.62:0/4152609880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d158bb6c70 0x55d158bd45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f8872a00640 0 client.5949 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.484+0000 7f889b779740 1 -- 172.21.15.62:0/4152609880 >> 172.21.15.62:0/4152609880 conn(0x55d158b90cf0 msgr2=0x55d158bd9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f889b779740 1 -- 172.21.15.62:0/4152609880 shutdown_connections 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f889b779740 1 -- 172.21.15.62:0/4152609880 wait complete. 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f889b779740 20 client.5949 trim_cache size 0 max 16384 2024-04-25T09:08:00.477 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:08:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-25T09:08:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 1671593002, 2024-04-25T09:08:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511630188, 2024-04-25T09:08:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T09:08:00.478 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:00.478 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 -- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f887406b4d0 msgr2=0x7f887406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f887406b4d0 0x7f887406d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f888400a830 tx=0x7f8884008040 comp rx=0 tx=0).stop 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 -- 172.21.15.62:0/425571931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f889015b870 msgr2=0x7f8890143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f889015b870 0x7f8890143240 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f888c009f40 tx=0x7f888c004a40 comp rx=0 tx=0).stop 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 -- 172.21.15.62:0/425571931 shutdown_connections 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f887406b4d0 0x7f887406d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f889015b870 0x7f8890143240 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88900b6790 0x7f8890142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 --2- 172.21.15.62:0/425571931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8890072b70 0x7f88900aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.485+0000 7f8897200640 1 -- 172.21.15.62:0/425571931 >> 172.21.15.62:0/425571931 conn(0x7f88900672d0 msgr2=0x7f8890158550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.486+0000 7f8897200640 1 -- 172.21.15.62:0/425571931 shutdown_connections 2024-04-25T09:08:00.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.486+0000 7f8897200640 1 -- 172.21.15.62:0/425571931 wait complete. 2024-04-25T09:08:00.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:00.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.712+0000 7f8a6be00640 1 Processor -- start 2024-04-25T09:08:00.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.712+0000 7f8a6be00640 1 -- start start 2024-04-25T09:08:00.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.714+0000 7f8a6be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a6c06d900 0x7f8a6c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.714+0000 7f8a6be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c15faa0 con 0x7f8a6c144c10 2024-04-25T09:08:00.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.714+0000 7f8a6be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c15fd80 con 0x7f8a6c0cee40 2024-04-25T09:08:00.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.714+0000 7f8a6be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6c144ff0 con 0x7f8a6c06d900 2024-04-25T09:08:00.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.714+0000 7f8a6a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a6c06d900 0x7f8a6c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.714+0000 7f8a6aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8a6c144c10 0x7f8a6c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50652/0 (socket says 172.21.15.62:50652) 2024-04-25T09:08:00.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.714+0000 7f8a6aa00640 1 -- 172.21.15.62:0/1145223610 learned_addr learned my addr 172.21.15.62:0/1145223610 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:00.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.714+0000 7f8a6a000640 1 --2- 172.21.15.62:0/1145223610 >> v2:172.21.15.62:3300/0 conn(0x7f8a6c06d900 0x7f8a6c15f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45368/0 (socket says 172.21.15.62:45368) 2024-04-25T09:08:00.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.715+0000 7f8a6a000640 1 -- 172.21.15.62:0/1145223610 >> v1:172.21.15.121:6789/0 conn(0x7f8a6c0cee40 legacy=0x7f8a6c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:00.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.715+0000 7f8a6a000640 1 -- 172.21.15.62:0/1145223610 >> v1:172.21.15.62:6789/0 conn(0x7f8a6c144c10 legacy=0x7f8a6c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.715+0000 7f8a6a000640 1 -- 172.21.15.62:0/1145223610 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c160060 con 0x7f8a6c06d900 2024-04-25T09:08:00.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.716+0000 7f8a6a000640 1 --2- 172.21.15.62:0/1145223610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c06d900 0x7f8a6c15f450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8a580097a0 tx=0x7f8a5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6740a101eb1442bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.716+0000 7f8a69600640 1 -- 172.21.15.62:0/1145223610 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58037070 con 0x7f8a6c06d900 2024-04-25T09:08:00.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.716+0000 7f8a69600640 1 -- 172.21.15.62:0/1145223610 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5802ae50 con 0x7f8a6c06d900 2024-04-25T09:08:00.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.716+0000 7f8a69600640 1 -- 172.21.15.62:0/1145223610 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58032660 con 0x7f8a6c06d900 2024-04-25T09:08:00.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.717+0000 7f8a6be00640 1 -- 172.21.15.62:0/1145223610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c06d900 msgr2=0x7f8a6c15f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.717+0000 7f8a6be00640 1 --2- 172.21.15.62:0/1145223610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c06d900 0x7f8a6c15f450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8a580097a0 tx=0x7f8a5802a140 comp rx=0 tx=0).stop 2024-04-25T09:08:00.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.717+0000 7f8a6be00640 1 -- 172.21.15.62:0/1145223610 shutdown_connections 2024-04-25T09:08:00.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.717+0000 7f8a6be00640 1 --2- 172.21.15.62:0/1145223610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c06d900 0x7f8a6c15f450 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.717+0000 7f8a6be00640 1 -- 172.21.15.62:0/1145223610 >> 172.21.15.62:0/1145223610 conn(0x7f8a6c0b14d0 msgr2=0x7f8a6c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.717+0000 7f8a6be00640 1 -- 172.21.15.62:0/1145223610 shutdown_connections 2024-04-25T09:08:00.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.717+0000 7f8a6be00640 1 -- 172.21.15.62:0/1145223610 wait complete. 2024-04-25T09:08:00.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.718+0000 7f8a6be00640 1 Processor -- start 2024-04-25T09:08:00.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.719+0000 7f8a6be00640 1 -- start start 2024-04-25T09:08:00.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a6c06d900 0x7f8a6c13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a6c0cee40 0x7f8a6c1303d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c144c10 0x7f8a6c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0b41c0 con 0x7f8a6c144c10 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c160340 con 0x7f8a6c0cee40 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0b4d60 con 0x7f8a6c06d900 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c144c10 0x7f8a6c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c144c10 0x7f8a6c0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45384/0 (socket says 172.21.15.62:45384) 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6b400640 1 -- 172.21.15.62:0/2809334755 learned_addr learned my addr 172.21.15.62:0/2809334755 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6a000640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a6c0cee40 0x7f8a6c1303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6aa00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a6c06d900 0x7f8a6c13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6b400640 1 -- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a6c06d900 msgr2=0x7f8a6c13fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6b400640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a6c06d900 0x7f8a6c13fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6b400640 1 -- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a6c0cee40 msgr2=0x7f8a6c1303d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6b400640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a6c0cee40 0x7f8a6c1303d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6b400640 1 -- 172.21.15.62:0/2809334755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c130910 con 0x7f8a6c144c10 2024-04-25T09:08:00.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.720+0000 7f8a6a000640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a6c0cee40 0x7f8a6c1303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:00.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.721+0000 7f8a6b400640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c144c10 0x7f8a6c0cd920 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c780 tx=0x7f8a6400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.721+0000 7f8a62a00640 1 -- 172.21.15.62:0/2809334755 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64004b60 con 0x7f8a6c144c10 2024-04-25T09:08:00.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.721+0000 7f8a62a00640 1 -- 172.21.15.62:0/2809334755 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a64004d00 con 0x7f8a6c144c10 2024-04-25T09:08:00.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.721+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c143600 con 0x7f8a6c144c10 2024-04-25T09:08:00.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.721+0000 7f8a62a00640 1 -- 172.21.15.62:0/2809334755 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400aa90 con 0x7f8a6c144c10 2024-04-25T09:08:00.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.721+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c143bc0 con 0x7f8a6c144c10 2024-04-25T09:08:00.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.723+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c04ef20 con 0x7f8a6c144c10 2024-04-25T09:08:00.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.724+0000 7f8a62a00640 1 -- 172.21.15.62:0/2809334755 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a6400ac30 con 0x7f8a6c144c10 2024-04-25T09:08:00.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.725+0000 7f8a62a00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a4c06b520 0x7f8a4c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:00.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.725+0000 7f8a62a00640 1 -- 172.21.15.62:0/2809334755 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8a6408a470 con 0x7f8a6c144c10 2024-04-25T09:08:00.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.725+0000 7f8a6aa00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a4c06b520 0x7f8a4c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:00.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.727+0000 7f8a6aa00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a4c06b520 0x7f8a4c06d9e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c143530 tx=0x7f8a5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:00.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.730+0000 7f8a62a00640 1 -- 172.21.15.62:0/2809334755 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a64020070 con 0x7f8a6c144c10 2024-04-25T09:08:00.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.968+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a6c15e130 con 0x7f8a6c144c10 2024-04-25T09:08:00.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.969+0000 7f8a62a00640 1 -- 172.21.15.62:0/2809334755 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f8a6c130910 con 0x7f8a6c144c10 2024-04-25T09:08:00.958 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:00.958 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:58.031468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:00.958 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-25T09:08:00.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a4c06b520 msgr2=0x7f8a4c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a4c06b520 0x7f8a4c06d9e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c143530 tx=0x7f8a5c009040 comp rx=0 tx=0).stop 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c144c10 msgr2=0x7f8a6c0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c144c10 0x7f8a6c0cd920 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400c780 tx=0x7f8a6400cc40 comp rx=0 tx=0).stop 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 shutdown_connections 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8a4c06b520 0x7f8a4c06d9e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a6c144c10 0x7f8a6c0cd920 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a6c0cee40 0x7f8a6c1303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 --2- 172.21.15.62:0/2809334755 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8a6c06d900 0x7f8a6c13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 >> 172.21.15.62:0/2809334755 conn(0x7f8a6c0b14d0 msgr2=0x7f8a6c15c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 shutdown_connections 2024-04-25T09:08:00.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:00.972+0000 7f8a6be00640 1 -- 172.21.15.62:0/2809334755 wait complete. 2024-04-25T09:08:00.980 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 1671593002 2024-04-25T09:08:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.208+0000 7f1de3a00640 1 Processor -- start 2024-04-25T09:08:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.209+0000 7f1de3a00640 1 -- start start 2024-04-25T09:08:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1de3a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc15bdf0 0x7f1ddc0720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1de3a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ddc0cebc0 0x7f1ddc06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1de3a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ddc06d070 con 0x7f1ddc06c7e0 2024-04-25T09:08:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1de3a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ddc15c1d0 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1de3a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0ae8c0 con 0x7f1ddc15bdf0 2024-04-25T09:08:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1ddbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ddc0cebc0 0x7f1ddc06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1de1a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1ddc06c7e0 0x7f1ddc0cd960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50658/0 (socket says 172.21.15.62:50658) 2024-04-25T09:08:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1ddbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ddc0cebc0 0x7f1ddc06c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45386/0 (socket says 172.21.15.62:45386) 2024-04-25T09:08:01.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.210+0000 7f1de1a00640 1 -- 172.21.15.62:0/2091682729 learned_addr learned my addr 172.21.15.62:0/2091682729 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:01.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.211+0000 7f1de1000640 1 --2- 172.21.15.62:0/2091682729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc15bdf0 0x7f1ddc0720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.211+0000 7f1ddbe00640 1 -- 172.21.15.62:0/2091682729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc15bdf0 msgr2=0x7f1ddc0720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.211+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/2091682729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc15bdf0 0x7f1ddc0720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.211+0000 7f1ddbe00640 1 -- 172.21.15.62:0/2091682729 >> v1:172.21.15.62:6789/0 conn(0x7f1ddc06c7e0 legacy=0x7f1ddc0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.211+0000 7f1ddbe00640 1 -- 172.21.15.62:0/2091682729 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc06d350 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.212+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/2091682729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc06c2a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80097a0 tx=0x7f1dc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230a518568b346af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.212+0000 7f1ddb400640 1 -- 172.21.15.62:0/2091682729 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8037070 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.212+0000 7f1ddb400640 1 -- 172.21.15.62:0/2091682729 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc8032440 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.212+0000 7f1ddb400640 1 -- 172.21.15.62:0/2091682729 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8032810 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.213+0000 7f1de3a00640 1 -- 172.21.15.62:0/2091682729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 msgr2=0x7f1ddc06c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.213+0000 7f1de3a00640 1 --2- 172.21.15.62:0/2091682729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc06c2a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80097a0 tx=0x7f1dc802a180 comp rx=0 tx=0).stop 2024-04-25T09:08:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.213+0000 7f1de3a00640 1 -- 172.21.15.62:0/2091682729 shutdown_connections 2024-04-25T09:08:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.213+0000 7f1de3a00640 1 --2- 172.21.15.62:0/2091682729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc06c2a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.213+0000 7f1de3a00640 1 --2- 172.21.15.62:0/2091682729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc15bdf0 0x7f1ddc0720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.213+0000 7f1de3a00640 1 -- 172.21.15.62:0/2091682729 >> 172.21.15.62:0/2091682729 conn(0x7f1ddc0b1410 msgr2=0x7f1ddc0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.213+0000 7f1de3a00640 1 -- 172.21.15.62:0/2091682729 shutdown_connections 2024-04-25T09:08:01.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.213+0000 7f1de3a00640 1 -- 172.21.15.62:0/2091682729 wait complete. 2024-04-25T09:08:01.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.215+0000 7f1de3a00640 1 Processor -- start 2024-04-25T09:08:01.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.215+0000 7f1de3a00640 1 -- start start 2024-04-25T09:08:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1de3a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc06c7e0 0x7f1ddc1e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1de3a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc1e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1de3a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ddc15bdf0 0x7f1ddc0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1de3a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc06cc90 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1de3a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc0aedf0 con 0x7f1ddc15bdf0 2024-04-25T09:08:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1de3a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc06ce10 con 0x7f1ddc06c7e0 2024-04-25T09:08:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1ddbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc1e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1ddbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc1e4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45396/0 (socket says 172.21.15.62:45396) 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.216+0000 7f1ddbe00640 1 -- 172.21.15.62:0/137810098 learned_addr learned my addr 172.21.15.62:0/137810098 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1de1000640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc06c7e0 0x7f1ddc1e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1de1a00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ddc15bdf0 0x7f1ddc0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1ddbe00640 1 -- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc06c7e0 msgr2=0x7f1ddc1e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc06c7e0 0x7f1ddc1e75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1ddbe00640 1 -- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ddc15bdf0 msgr2=0x7f1ddc0ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ddc15bdf0 0x7f1ddc0ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1ddbe00640 1 -- 172.21.15.62:0/137810098 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc1e5750 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1de1a00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ddc15bdf0 0x7f1ddc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.217+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc1e4f20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80097a0 tx=0x7f1dc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.218+0000 7f1dd8c00640 1 -- 172.21.15.62:0/137810098 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8041070 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.218+0000 7f1dd8c00640 1 -- 172.21.15.62:0/137810098 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc803c870 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.218+0000 7f1dd8c00640 1 -- 172.21.15.62:0/137810098 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8037040 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.218+0000 7f1de3a00640 1 -- 172.21.15.62:0/137810098 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc1e5a30 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.218+0000 7f1de3a00640 1 -- 172.21.15.62:0/137810098 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc1e5ff0 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.220+0000 7f1dd8c00640 1 -- 172.21.15.62:0/137810098 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1dc8044070 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.222+0000 7f1dd8c00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1dbc06b500 0x7f1dbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.222+0000 7f1de6963740 1 Processor -- start 2024-04-25T09:08:01.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.222+0000 7f1dd8c00640 1 -- 172.21.15.62:0/137810098 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1dc80aae00 con 0x7f1ddc0cebc0 2024-04-25T09:08:01.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.222+0000 7f1de6963740 1 -- start start 2024-04-25T09:08:01.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.223+0000 7f1de1000640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1dbc06b500 0x7f1dbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.223+0000 7f1de6963740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 0x560641203c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.223+0000 7f1de6963740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5606411fe4e0 0x560641202460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.223+0000 7f1de6963740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x560641202de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.223+0000 7f1de6963740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560641167820 con 0x5606412029a0 2024-04-25T09:08:01.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.223+0000 7f1de6963740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560641110d00 con 0x560641203860 2024-04-25T09:08:01.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.223+0000 7f1de6963740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606411638c0 con 0x5606411fe4e0 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x560641202de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x560641202de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45406/0 (socket says 172.21.15.62:45406) 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 -- 172.21.15.62:0/2653518060 learned_addr learned my addr 172.21.15.62:0/2653518060 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1000640 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 0x560641203c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5606411fe4e0 0x560641202460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 -- 172.21.15.62:0/2653518060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5606411fe4e0 msgr2=0x560641202460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5606411fe4e0 0x560641202460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 -- 172.21.15.62:0/2653518060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 msgr2=0x560641203c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 0x560641203c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 -- 172.21.15.62:0/2653518060 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560640f48710 con 0x5606412029a0 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1000640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1dbc06b500 0x7f1dbc06d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0006fd0 tx=0x7f1dd000acf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.224+0000 7f1de1a00640 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x560641202de0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40021a0 tx=0x7f1dd400d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f476cd477578ad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.225+0000 7f1dcd600640 1 -- 172.21.15.62:0/2653518060 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4018070 con 0x5606412029a0 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.225+0000 7f1dcd600640 1 -- 172.21.15.62:0/2653518060 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd4013670 con 0x5606412029a0 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.225+0000 7f1dcd600640 1 -- 172.21.15.62:0/2653518060 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4013a40 con 0x5606412029a0 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.225+0000 7f1de6963740 1 -- 172.21.15.62:0/2653518060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 msgr2=0x560641202de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x560641202de0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40021a0 tx=0x7f1dd400d5e0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 -- 172.21.15.62:0/2653518060 shutdown_connections 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x560641202de0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5606411fe4e0 0x560641202460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 --2- 172.21.15.62:0/2653518060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 0x560641203c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 -- 172.21.15.62:0/2653518060 >> 172.21.15.62:0/2653518060 conn(0x560641200960 msgr2=0x560641249420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 -- 172.21.15.62:0/2653518060 shutdown_connections 2024-04-25T09:08:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 -- 172.21.15.62:0/2653518060 wait complete. 2024-04-25T09:08:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.226+0000 7f1de6963740 1 -- start start 2024-04-25T09:08:01.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.227+0000 7f1de6963740 1 Processor -- start 2024-04-25T09:08:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de6963740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606412029a0 0x56064124efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de6963740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641203860 0x56064124c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de6963740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641224dc0 0x56064124cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de6963740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064122f3f0 con 0x560641224dc0 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de6963740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560641106360 con 0x5606412029a0 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de6963740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56064122e870 con 0x560641203860 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de6963740 10 client.0 _ll_register_callbacks cb 0x560641112c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de1a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641224dc0 0x56064124cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de1a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641224dc0 0x56064124cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45422/0 (socket says 172.21.15.62:45422) 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.228+0000 7f1de1a00640 1 -- 172.21.15.62:0/735641320 learned_addr learned my addr 172.21.15.62:0/735641320 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1de1000640 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606412029a0 0x56064124efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1de1a00640 1 -- 172.21.15.62:0/735641320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641203860 msgr2=0x56064124c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1de1a00640 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641203860 0x56064124c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1de1a00640 1 -- 172.21.15.62:0/735641320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606412029a0 msgr2=0x56064124efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1de1a00640 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606412029a0 0x56064124efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1de1a00640 1 -- 172.21.15.62:0/735641320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560640fa8ea0 con 0x560641224dc0 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1de1000640 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606412029a0 0x56064124efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:01.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1de1a00640 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641224dc0 0x56064124cd30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400d620 tx=0x7f1dd4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc63e06630b671a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.229+0000 7f1dc2000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:01.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.232+0000 7f1dc2000640 1 -- 172.21.15.62:0/735641320 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4023020 con 0x560641224dc0 2024-04-25T09:08:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.232+0000 7f1de6963740 1 -- 172.21.15.62:0/735641320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dd4013670 con 0x560641224dc0 2024-04-25T09:08:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.232+0000 7f1dc2000640 1 -- 172.21.15.62:0/735641320 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1dd401f480 con 0x560641224dc0 2024-04-25T09:08:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.232+0000 7f1dc2000640 1 -- 172.21.15.62:0/735641320 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dd402d050 con 0x560641224dc0 2024-04-25T09:08:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.232+0000 7f1de6963740 10 client.5973 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.232+0000 7f1de6963740 1 -- 172.21.15.62:0/735641320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x560640e3c000 con 0x560641224dc0 2024-04-25T09:08:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1dc2000640 1 -- 172.21.15.62:0/735641320 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd4013670 con 0x560641224dc0 2024-04-25T09:08:01.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1dc2000640 1 -- 172.21.15.62:0/735641320 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1dd4030020 con 0x560641224dc0 2024-04-25T09:08:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1de6963740 10 client.5973 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1de6963740 10 client.5973 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:08:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1de6963740 20 client.5973 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1de6963740 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56064124e0a0 0x56064124e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1de6963740 4 client.5973 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1de6963740 1 -- 172.21.15.62:0/735641320 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560641174d60 con 0x56064124e0a0 2024-04-25T09:08:01.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.233+0000 7f1de1000640 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56064124e0a0 0x56064124e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.234+0000 7f1de1000640 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56064124e0a0 0x56064124e4a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f96d91d32b6f836 server_cookie=a3f061ad657a785b in_seq=0 out_seq=0 2024-04-25T09:08:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.234+0000 7f1dc2000640 10 client.5973 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.235+0000 7f1dc2000640 1 -- 172.21.15.62:0/735641320 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560641174d60 con 0x56064124e0a0 2024-04-25T09:08:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.235+0000 7f1dc2000640 10 client.5973 handle_command_reply: tid=0 2024-04-25T09:08:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.235+0000 7f1de6963740 1 client.5973 shutdown 2024-04-25T09:08:01.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 -- 172.21.15.62:0/735641320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641224dc0 msgr2=0x56064124cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641224dc0 0x56064124cd30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400d620 tx=0x7f1dd4001250 comp rx=0 tx=0).stop 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 -- 172.21.15.62:0/735641320 shutdown_connections 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56064124e0a0 0x56064124e4a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641224dc0 0x56064124cd30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641203860 0x56064124c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 --2- 172.21.15.62:0/735641320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5606412029a0 0x56064124efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 -- 172.21.15.62:0/735641320 >> 172.21.15.62:0/735641320 conn(0x5606411fe4e0 msgr2=0x5606411fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1dc2000640 0 client.5973 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.236+0000 7f1de6963740 1 -- 172.21.15.62:0/735641320 shutdown_connections 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.237+0000 7f1de6963740 1 -- 172.21.15.62:0/735641320 wait complete. 2024-04-25T09:08:01.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.237+0000 7f1de6963740 20 client.5973 trim_cache size 0 max 16384 2024-04-25T09:08:01.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.252+0000 7f1de6963740 1 Processor -- start 2024-04-25T09:08:01.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.252+0000 7f1de6963740 1 -- start start 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de6963740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x5606412ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de6963740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 0x56064122da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de6963740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 0x56064122dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de6963740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1ff460 con 0x5606412029a0 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de6963740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1f4a40 con 0x560641203860 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de6963740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1f5040 con 0x560641224dc0 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x5606412ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x5606412ac3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45424/0 (socket says 172.21.15.62:45424) 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.253+0000 7f1de1000640 1 -- 172.21.15.62:0/695570440 learned_addr learned my addr 172.21.15.62:0/695570440 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:01.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1de1000640 1 -- 172.21.15.62:0/695570440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 msgr2=0x56064122dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1de1000640 1 --2- 172.21.15.62:0/695570440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 0x56064122dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1de1000640 1 -- 172.21.15.62:0/695570440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 msgr2=0x56064122da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1de1000640 1 --2- 172.21.15.62:0/695570440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 0x56064122da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1de1000640 1 -- 172.21.15.62:0/695570440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56064124f0d0 con 0x5606412029a0 2024-04-25T09:08:01.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1de1000640 1 --2- 172.21.15.62:0/695570440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x5606412ac3d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x5606412270b0 tx=0x7f1dd001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bda5bb32a0d75bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1dccc00640 1 -- 172.21.15.62:0/695570440 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd001b040 con 0x5606412029a0 2024-04-25T09:08:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1dccc00640 1 -- 172.21.15.62:0/695570440 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd001c040 con 0x5606412029a0 2024-04-25T09:08:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.254+0000 7f1dccc00640 1 -- 172.21.15.62:0/695570440 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0015e20 con 0x5606412029a0 2024-04-25T09:08:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 -- 172.21.15.62:0/695570440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 msgr2=0x5606412ac3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 --2- 172.21.15.62:0/695570440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x5606412ac3d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x5606412270b0 tx=0x7f1dd001a000 comp rx=0 tx=0).stop 2024-04-25T09:08:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 -- 172.21.15.62:0/695570440 shutdown_connections 2024-04-25T09:08:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 --2- 172.21.15.62:0/695570440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 0x56064122dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 --2- 172.21.15.62:0/695570440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560641203860 0x56064122da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 --2- 172.21.15.62:0/695570440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5606412029a0 0x5606412ac3d0 secure :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0x5606412270b0 tx=0x7f1dd001a000 comp rx=0 tx=0).stop 2024-04-25T09:08:01.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 -- 172.21.15.62:0/695570440 >> 172.21.15.62:0/695570440 conn(0x5606411fe4e0 msgr2=0x56064122ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 -- 172.21.15.62:0/695570440 shutdown_connections 2024-04-25T09:08:01.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.255+0000 7f1de6963740 1 -- 172.21.15.62:0/695570440 wait complete. 2024-04-25T09:08:01.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.256+0000 7f1de6963740 1 -- start start 2024-04-25T09:08:01.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.256+0000 7f1de6963740 1 Processor -- start 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de6963740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641203860 0x5606412a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de6963740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 0x560641262ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de6963740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064124e480 0x5606412631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de6963740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064122a310 con 0x560641203860 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de6963740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560641250220 con 0x56064124e480 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de6963740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x560641229ba0 con 0x560641224dc0 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de6963740 10 client.0 _ll_register_callbacks cb 0x7f1dd4001a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641203860 0x5606412a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641203860 0x5606412a91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45436/0 (socket says 172.21.15.62:45436) 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.257+0000 7f1de1000640 1 -- 172.21.15.62:0/3605265410 learned_addr learned my addr 172.21.15.62:0/3605265410 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 0x560641262ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1de1a00640 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064124e480 0x5606412631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1de1000640 1 -- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 msgr2=0x560641262ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1de1000640 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 0x560641262ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1de1000640 1 -- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064124e480 msgr2=0x5606412631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1de1000640 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064124e480 0x5606412631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1de1000640 1 -- 172.21.15.62:0/3605265410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560641225480 con 0x560641203860 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1de1a00640 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064124e480 0x5606412631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1de1000640 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641203860 0x5606412a91f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0029040 tx=0x7f1dd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9750e3b0259428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1dc2000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.258+0000 7f1dc2000640 1 -- 172.21.15.62:0/3605265410 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0034070 con 0x560641203860 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1dc2000640 1 -- 172.21.15.62:0/3605265410 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1dd0026570 con 0x560641203860 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 1 -- 172.21.15.62:0/3605265410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dd001c040 con 0x560641203860 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1dc2000640 1 -- 172.21.15.62:0/3605265410 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dd003f050 con 0x560641203860 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 10 client.5979 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 1 -- 172.21.15.62:0/3605265410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x560641226000 con 0x560641203860 2024-04-25T09:08:01.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1dc2000640 1 -- 172.21.15.62:0/3605265410 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd001c040 con 0x560641203860 2024-04-25T09:08:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1dc2000640 1 -- 172.21.15.62:0/3605265410 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1dd0042020 con 0x560641203860 2024-04-25T09:08:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 10 client.5979 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 10 client.5979 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:08:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 20 client.5979 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5606411fdf90 0x5606412ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 4 client.5979 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1671593002}] 2024-04-25T09:08:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.259+0000 7f1de6963740 1 -- 172.21.15.62:0/3605265410 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1671593002}) v1 -- 0x56064116f0b0 con 0x5606411fdf90 2024-04-25T09:08:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.260+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5606411fdf90 0x5606412ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.260+0000 7f1ddbe00640 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5606411fdf90 0x5606412ad3f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c24beedf05328140 server_cookie=a9cc43561b833438 in_seq=0 out_seq=0 2024-04-25T09:08:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.260+0000 7f1dc2000640 10 client.5979 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.260+0000 7f1dc2000640 1 -- 172.21.15.62:0/3605265410 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x56064116f0b0 con 0x5606411fdf90 2024-04-25T09:08:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.260+0000 7f1dc2000640 10 client.5979 handle_command_reply: tid=0 2024-04-25T09:08:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.260+0000 7f1de6963740 1 client.5979 shutdown 2024-04-25T09:08:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.260+0000 7f1de6963740 1 -- 172.21.15.62:0/3605265410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641203860 msgr2=0x5606412a91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.260+0000 7f1de6963740 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641203860 0x5606412a91f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0029040 tx=0x7f1dd0004730 comp rx=0 tx=0).stop 2024-04-25T09:08:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 1 -- 172.21.15.62:0/3605265410 shutdown_connections 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5606411fdf90 0x5606412ad3f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064124e480 0x5606412631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560641224dc0 0x560641262ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 1 --2- 172.21.15.62:0/3605265410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560641203860 0x5606412a91f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1dc2000640 0 client.5979 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 1 -- 172.21.15.62:0/3605265410 >> 172.21.15.62:0/3605265410 conn(0x5606411fe4e0 msgr2=0x56064122da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 1 -- 172.21.15.62:0/3605265410 shutdown_connections 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 1 -- 172.21.15.62:0/3605265410 wait complete. 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.261+0000 7f1de6963740 20 client.5979 trim_cache size 0 max 16384 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 -- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1dbc06b500 msgr2=0x7f1dbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1dbc06b500 0x7f1dbc06d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0006fd0 tx=0x7f1dd000acf0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 -- 172.21.15.62:0/137810098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 msgr2=0x7f1ddc1e4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc1e4f20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80097a0 tx=0x7f1dc80046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 -- 172.21.15.62:0/137810098 shutdown_connections 2024-04-25T09:08:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1dbc06b500 0x7f1dbc06d9c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ddc0cebc0 0x7f1ddc1e4f20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ddc15bdf0 0x7f1ddc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 --2- 172.21.15.62:0/137810098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1ddc06c7e0 0x7f1ddc1e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 -- 172.21.15.62:0/137810098 >> 172.21.15.62:0/137810098 conn(0x7f1ddc0b1410 msgr2=0x7f1ddc15a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 -- 172.21.15.62:0/137810098 shutdown_connections 2024-04-25T09:08:01.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.262+0000 7f1de3a00640 1 -- 172.21.15.62:0/137810098 wait complete. 2024-04-25T09:08:01.267 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:08:01.268 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.462+0000 7f51f1c00640 1 Processor -- start 2024-04-25T09:08:01.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.462+0000 7f51f1c00640 1 -- start start 2024-04-25T09:08:01.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.464+0000 7f51f1c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f51ec0b56f0 0x7f51ec160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.464+0000 7f51f1c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0b6a90 0x7f51ec161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.464+0000 7f51f1c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec0d3110 con 0x7f51ec0b27a0 2024-04-25T09:08:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.464+0000 7f51f1c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec161780 con 0x7f51ec0b56f0 2024-04-25T09:08:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.464+0000 7f51f1c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0cedc0 con 0x7f51ec0b6a90 2024-04-25T09:08:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.464+0000 7f51f1200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f51ec0b27a0 0x7f51ec161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50660/0 (socket says 172.21.15.62:50660) 2024-04-25T09:08:01.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.464+0000 7f51f1200640 1 -- 172.21.15.62:0/688625688 learned_addr learned my addr 172.21.15.62:0/688625688 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:01.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51e3e00640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0b6a90 0x7f51ec161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51ebe00640 1 --2- 172.21.15.62:0/688625688 >> v2:172.21.15.121:3300/0 conn(0x7f51ec0b56f0 0x7f51ec160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254674012 0 0) 0x7f51ec0d3110 con 0x7f51ec0b27a0 2024-04-25T09:08:01.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d0003520 con 0x7f51ec0b27a0 2024-04-25T09:08:01.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 743084636 0 0) 0x7f51d0003520 con 0x7f51ec0b27a0 2024-04-25T09:08:01.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0b6a90 msgr2=0x7f51ec161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51eb400640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0b6a90 0x7f51ec161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 >> v2:172.21.15.121:3300/0 conn(0x7f51ec0b56f0 msgr2=0x7f51ec160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.465+0000 7f51eb400640 1 --2- 172.21.15.62:0/688625688 >> v2:172.21.15.121:3300/0 conn(0x7f51ec0b56f0 0x7f51ec160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0d33f0 con 0x7f51ec0b27a0 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f51e40019f0 con 0x7f51ec0b27a0 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 >> v1:172.21.15.62:6789/0 conn(0x7f51ec0b27a0 legacy=0x7f51ec161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0b6a90 0x7f51d00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d0003f80 0x7f51d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0004d40 0x7f51d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0cedc0 con 0x7f51d0003f80 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d000b9c0 con 0x7f51d0004d40 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d000bb40 con 0x7f51ec0b6a90 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51e3e00640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d0003f80 0x7f51d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51ebe00640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0b6a90 0x7f51d00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.466+0000 7f51f1200640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0004d40 0x7f51d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.467+0000 7f51e3e00640 1 -- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0b6a90 msgr2=0x7f51d00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.467+0000 7f51e3e00640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0b6a90 0x7f51d00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.467+0000 7f51e3e00640 1 -- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0004d40 msgr2=0x7f51d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:01.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.467+0000 7f51e3e00640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0004d40 0x7f51d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.467+0000 7f51e3e00640 1 -- 172.21.15.62:0/688625688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0003520 con 0x7f51d0003f80 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.468+0000 7f51e3e00640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d0003f80 0x7f51d0004800 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009af0 tx=0x7f51d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad37ce5da0f2d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.468+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4036070 con 0x7f51d0003f80 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.468+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4031400 con 0x7f51d0003f80 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.468+0000 7f51eb400640 1 -- 172.21.15.62:0/688625688 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4009690 con 0x7f51d0003f80 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.468+0000 7f51f1c00640 1 -- 172.21.15.62:0/688625688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d0003f80 msgr2=0x7f51d0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.468+0000 7f51f1c00640 1 --2- 172.21.15.62:0/688625688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51d0003f80 0x7f51d0004800 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009af0 tx=0x7f51d0005830 comp rx=0 tx=0).stop 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.469+0000 7f51f1200640 1 -- 172.21.15.62:0/688625688 reap_dead start 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.469+0000 7f51f1c00640 1 -- 172.21.15.62:0/688625688 shutdown_connections 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.469+0000 7f51f1c00640 1 -- 172.21.15.62:0/688625688 >> 172.21.15.62:0/688625688 conn(0x7f51ec066d10 msgr2=0x7f51ec067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.469+0000 7f51f1c00640 1 -- 172.21.15.62:0/688625688 shutdown_connections 2024-04-25T09:08:01.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.469+0000 7f51f1c00640 1 -- 172.21.15.62:0/688625688 wait complete. 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.471+0000 7f51f1c00640 1 Processor -- start 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.471+0000 7f51f1c00640 1 -- start start 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51f1c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0003f80 0x7f51ec0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51f1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec1f7fc0 0x7f51ec157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51f1c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0c7540 0x7f51ec1e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51f1c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0b2d60 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51f1c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0cea50 con 0x7f51d0003f80 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51f1c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0b2ee0 con 0x7f51ec0c7540 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec1f7fc0 0x7f51ec157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec1f7fc0 0x7f51ec157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45468/0 (socket says 172.21.15.62:45468) 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51e3e00640 1 -- 172.21.15.62:0/2881505977 learned_addr learned my addr 172.21.15.62:0/2881505977 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51ebe00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0003f80 0x7f51ec0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51e3e00640 1 -- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0c7540 msgr2=0x7f51ec1e6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0c7540 0x7f51ec1e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51e3e00640 1 -- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0003f80 msgr2=0x7f51ec0c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0003f80 0x7f51ec0c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51e3e00640 1 -- 172.21.15.62:0/2881505977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec1f7520 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.472+0000 7f51ebe00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0003f80 0x7f51ec0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.473+0000 7f51e3e00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec1f7fc0 0x7f51ec157060 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f51d4034040 tx=0x7f51d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.473+0000 7f51e8c00640 1 -- 172.21.15.62:0/2881505977 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4040020 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.473+0000 7f51e8c00640 1 -- 172.21.15.62:0/2881505977 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d40038d0 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.473+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec0ca710 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.473+0000 7f51e8c00640 1 -- 172.21.15.62:0/2881505977 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4036040 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.473+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec1f77c0 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.474+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec04ef20 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.475+0000 7f51e8c00640 1 -- 172.21.15.62:0/2881505977 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51d4043020 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.476+0000 7f51e8c00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51c806b4d0 0x7f51c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.476+0000 7f51e8c00640 1 -- 172.21.15.62:0/2881505977 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f51d40aa300 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.477+0000 7f51ebe00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51c806b4d0 0x7f51c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.478+0000 7f51ebe00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51c806b4d0 0x7f51c806d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009a80 tx=0x7f51dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.481+0000 7f51e8c00640 1 -- 172.21.15.62:0/2881505977 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51d404a070 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.717+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51ec0ce580 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.718+0000 7f51e8c00640 1 -- 172.21.15.62:0/2881505977 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f51d40790e0 con 0x7f51ec1f7fc0 2024-04-25T09:08:01.707 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:01.707 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:58.031468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:01.707 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51c806b4d0 msgr2=0x7f51c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51c806b4d0 0x7f51c806d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009a80 tx=0x7f51dc008040 comp rx=0 tx=0).stop 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec1f7fc0 msgr2=0x7f51ec157060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec1f7fc0 0x7f51ec157060 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f51d4034040 tx=0x7f51d4004730 comp rx=0 tx=0).stop 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 shutdown_connections 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51c806b4d0 0x7f51c806d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51ec1f7fc0 0x7f51ec157060 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51ec0c7540 0x7f51ec1e6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 --2- 172.21.15.62:0/2881505977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d0003f80 0x7f51ec0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 >> 172.21.15.62:0/2881505977 conn(0x7f51ec066d10 msgr2=0x7f51ec0b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:01.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 shutdown_connections 2024-04-25T09:08:01.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:01.721+0000 7f51f1c00640 1 -- 172.21.15.62:0/2881505977 wait complete. 2024-04-25T09:08:01.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-25T09:08:01.938 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:01.939 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:08:01.939 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:01.953 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:08:01.953 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:01.953 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T09:08:01.954 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T09:08:01.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:01.954 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T09:08:02.006 DEBUG:teuthology.orchestra.run.smithi062:> stdin-killer -- python3 -c ' 2024-04-25T09:08:02.006 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-04-25T09:08:02.006 DEBUG:teuthology.orchestra.run.smithi062:> import stat 2024-04-25T09:08:02.006 DEBUG:teuthology.orchestra.run.smithi062:> import json 2024-04-25T09:08:02.006 DEBUG:teuthology.orchestra.run.smithi062:> import sys 2024-04-25T09:08:02.006 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:08:02.006 DEBUG:teuthology.orchestra.run.smithi062:> try: 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> except OSError as e: 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> sys.exit(e.errno) 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> print(json.dumps( 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> indent=2)) 2024-04-25T09:08:02.007 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-04-25T09:08:02.157 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:02.157 INFO:teuthology.orchestra.run.smithi062.stdout: "st_mode": 16877, 2024-04-25T09:08:02.157 INFO:teuthology.orchestra.run.smithi062.stdout: "st_ino": 1099511627777, 2024-04-25T09:08:02.157 INFO:teuthology.orchestra.run.smithi062.stdout: "st_dev": 40, 2024-04-25T09:08:02.157 INFO:teuthology.orchestra.run.smithi062.stdout: "st_nlink": 16, 2024-04-25T09:08:02.157 INFO:teuthology.orchestra.run.smithi062.stdout: "st_uid": 1000, 2024-04-25T09:08:02.158 INFO:teuthology.orchestra.run.smithi062.stdout: "st_gid": 1316, 2024-04-25T09:08:02.158 INFO:teuthology.orchestra.run.smithi062.stdout: "st_size": 18, 2024-04-25T09:08:02.158 INFO:teuthology.orchestra.run.smithi062.stdout: "st_atime": 1714036059.607298, 2024-04-25T09:08:02.158 INFO:teuthology.orchestra.run.smithi062.stdout: "st_mtime": 1714036073.0110857, 2024-04-25T09:08:02.158 INFO:teuthology.orchestra.run.smithi062.stdout: "st_ctime": 1714036073.0110857 2024-04-25T09:08:02.158 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:08:02.171 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T09:08:02.171 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T09:08:02.171 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.430+0000 7f57f3200640 1 Processor -- start 2024-04-25T09:08:02.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.430+0000 7f57f3200640 1 -- start start 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.432+0000 7f57f3200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f57f415d810 0x7f57f40ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.432+0000 7f57f3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f57f406d900 0x7f57f41675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.432+0000 7f57f3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f40ced70 con 0x7f57f415dbf0 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.432+0000 7f57f3200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f40cf030 con 0x7f57f415d810 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.432+0000 7f57f3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f40cf310 con 0x7f57f406d900 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.432+0000 7f57f1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f57f415d810 0x7f57f40ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.432+0000 7f57f1400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f57f415dbf0 0x7f57f406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50666/0 (socket says 172.21.15.62:50666) 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.432+0000 7f57f1400640 1 -- 172.21.15.62:0/3619621194 learned_addr learned my addr 172.21.15.62:0/3619621194 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:02.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.433+0000 7f57f2800640 1 --2- 172.21.15.62:0/3619621194 >> v2:172.21.15.121:3300/0 conn(0x7f57f406d900 0x7f57f41675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.433+0000 7f57f1e00640 1 -- 172.21.15.62:0/3619621194 >> v2:172.21.15.121:3300/0 conn(0x7f57f406d900 msgr2=0x7f57f41675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:02.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.433+0000 7f57f1e00640 1 --2- 172.21.15.62:0/3619621194 >> v2:172.21.15.121:3300/0 conn(0x7f57f406d900 0x7f57f41675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.433+0000 7f57f1e00640 1 -- 172.21.15.62:0/3619621194 >> v1:172.21.15.62:6789/0 conn(0x7f57f415dbf0 legacy=0x7f57f406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.433+0000 7f57f1e00640 1 -- 172.21.15.62:0/3619621194 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40c8cd0 con 0x7f57f415d810 2024-04-25T09:08:02.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.434+0000 7f57f1e00640 1 --2- 172.21.15.62:0/3619621194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40ce830 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f57f40cd540 tx=0x7f57e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41298cc2190b3f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.434+0000 7f57f0a00640 1 -- 172.21.15.62:0/3619621194 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4037070 con 0x7f57f415d810 2024-04-25T09:08:02.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.434+0000 7f57f0a00640 1 -- 172.21.15.62:0/3619621194 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e4032440 con 0x7f57f415d810 2024-04-25T09:08:02.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.434+0000 7f57f0a00640 1 -- 172.21.15.62:0/3619621194 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4032810 con 0x7f57f415d810 2024-04-25T09:08:02.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.435+0000 7f57f3200640 1 -- 172.21.15.62:0/3619621194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 msgr2=0x7f57f40ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.435+0000 7f57f3200640 1 --2- 172.21.15.62:0/3619621194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40ce830 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f57f40cd540 tx=0x7f57e402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.435+0000 7f57f3200640 1 -- 172.21.15.62:0/3619621194 shutdown_connections 2024-04-25T09:08:02.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.435+0000 7f57f3200640 1 --2- 172.21.15.62:0/3619621194 >> v2:172.21.15.121:3300/0 conn(0x7f57f406d900 0x7f57f41675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.435+0000 7f57f3200640 1 --2- 172.21.15.62:0/3619621194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40ce830 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.435+0000 7f57f3200640 1 -- 172.21.15.62:0/3619621194 >> 172.21.15.62:0/3619621194 conn(0x7f57f40b14d0 msgr2=0x7f57f40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:02.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.435+0000 7f57f3200640 1 -- 172.21.15.62:0/3619621194 shutdown_connections 2024-04-25T09:08:02.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.435+0000 7f57f3200640 1 -- 172.21.15.62:0/3619621194 wait complete. 2024-04-25T09:08:02.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.437+0000 7f57f3200640 1 Processor -- start 2024-04-25T09:08:02.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.437+0000 7f57f3200640 1 -- start start 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f406d900 0x7f57f4139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f415dbf0 0x7f57f415b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ca660 con 0x7f57f415d810 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ca4e0 con 0x7f57f406d900 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ca7e0 con 0x7f57f415dbf0 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45488/0 (socket says 172.21.15.62:45488) 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.438+0000 7f57f1400640 1 -- 172.21.15.62:0/593879651 learned_addr learned my addr 172.21.15.62:0/593879651 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:02.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f1e00640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f406d900 0x7f57f4139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f2800640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f415dbf0 0x7f57f415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f1400640 1 -- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f415dbf0 msgr2=0x7f57f415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f1400640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f415dbf0 0x7f57f415b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f1400640 1 -- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f406d900 msgr2=0x7f57f4139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f1400640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f406d900 0x7f57f4139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f1400640 1 -- 172.21.15.62:0/593879651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40b4650 con 0x7f57f415d810 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f1400640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40b2820 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c3f0 tx=0x7f57e000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57ea000640 1 -- 172.21.15.62:0/593879651 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0017070 con 0x7f57f415d810 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57ea000640 1 -- 172.21.15.62:0/593879651 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0005ce0 con 0x7f57f415d810 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57ea000640 1 -- 172.21.15.62:0/593879651 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e00072d0 con 0x7f57f415d810 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.439+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f40b48f0 con 0x7f57f415d810 2024-04-25T09:08:02.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.440+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f40b4eb0 con 0x7f57f415d810 2024-04-25T09:08:02.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.441+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f404ef20 con 0x7f57f415d810 2024-04-25T09:08:02.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.442+0000 7f57ea000640 1 -- 172.21.15.62:0/593879651 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57e001a070 con 0x7f57f415d810 2024-04-25T09:08:02.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.443+0000 7f57ea000640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d406b4d0 0x7f57d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:02.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.443+0000 7f57ea000640 1 -- 172.21.15.62:0/593879651 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f57e00894f0 con 0x7f57f415d810 2024-04-25T09:08:02.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.443+0000 7f57f1e00640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d406b4d0 0x7f57d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:02.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.444+0000 7f57f1e00640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d406b4d0 0x7f57d406d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f57e4034000 tx=0x7f57e402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:02.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.447+0000 7f57ea000640 1 -- 172.21.15.62:0/593879651 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57e0057c90 con 0x7f57f415d810 2024-04-25T09:08:02.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.690+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57f4071200 con 0x7f57f415d810 2024-04-25T09:08:02.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.691+0000 7f57ea000640 1 -- 172.21.15.62:0/593879651 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f57e005bd30 con 0x7f57f415d810 2024-04-25T09:08:02.680 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:02.680 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:58.031468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:02.680 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-25T09:08:02.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d406b4d0 msgr2=0x7f57d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d406b4d0 0x7f57d406d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f57e4034000 tx=0x7f57e402e000 comp rx=0 tx=0).stop 2024-04-25T09:08:02.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 msgr2=0x7f57f40b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:02.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40b2820 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c3f0 tx=0x7f57e000c8b0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 shutdown_connections 2024-04-25T09:08:02.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d406b4d0 0x7f57d406d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57f415d810 0x7f57f40b2820 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57f415dbf0 0x7f57f415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 --2- 172.21.15.62:0/593879651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57f406d900 0x7f57f4139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:02.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 >> 172.21.15.62:0/593879651 conn(0x7f57f40b14d0 msgr2=0x7f57f40717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:02.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 shutdown_connections 2024-04-25T09:08:02.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:02.694+0000 7f57f3200640 1 -- 172.21.15.62:0/593879651 wait complete. 2024-04-25T09:08:02.702 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T09:08:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:02.896 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": -2 2024-04-25T09:08:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:02.912 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T09:08:02.912 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T09:08:02.912 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:02.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T09:08:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.139+0000 7f250b200640 1 Processor -- start 2024-04-25T09:08:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.139+0000 7f250b200640 1 -- start start 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.140+0000 7f250b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f250c0b2d80 0x7f250c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b36a0 0x7f250c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f250c0b54d0 0x7f250c0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250c0b4430 con 0x7f250c0b54d0 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250c0b4710 con 0x7f250c0b2d80 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0b49f0 con 0x7f250c0b36a0 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f250c0b54d0 0x7f250c0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f250c0b54d0 0x7f250c0700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45496/0 (socket says 172.21.15.62:45496) 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250a800640 1 -- 172.21.15.62:0/705828765 learned_addr learned my addr 172.21.15.62:0/705828765 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f2501400640 1 --2- 172.21.15.62:0/705828765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b36a0 0x7f250c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f2509e00640 1 --2- 172.21.15.62:0/705828765 >> v2:172.21.15.121:3300/0 conn(0x7f250c0b2d80 0x7f250c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250a800640 1 -- 172.21.15.62:0/705828765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b36a0 msgr2=0x7f250c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250a800640 1 --2- 172.21.15.62:0/705828765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b36a0 0x7f250c0ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250a800640 1 -- 172.21.15.62:0/705828765 >> v2:172.21.15.121:3300/0 conn(0x7f250c0b2d80 msgr2=0x7f250c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250a800640 1 --2- 172.21.15.62:0/705828765 >> v2:172.21.15.121:3300/0 conn(0x7f250c0b2d80 0x7f250c0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f250a800640 1 -- 172.21.15.62:0/705828765 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c0b4cd0 con 0x7f250c0b54d0 2024-04-25T09:08:03.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.141+0000 7f2509e00640 1 --2- 172.21.15.62:0/705828765 >> v2:172.21.15.121:3300/0 conn(0x7f250c0b2d80 0x7f250c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.143+0000 7f250a800640 1 --2- 172.21.15.62:0/705828765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b54d0 0x7f250c0700c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f250c0721d0 tx=0x7f250402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c273df8400e2801b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.143+0000 7f2509400640 1 -- 172.21.15.62:0/705828765 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504037070 con 0x7f250c0b54d0 2024-04-25T09:08:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.143+0000 7f2509400640 1 -- 172.21.15.62:0/705828765 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504032440 con 0x7f250c0b54d0 2024-04-25T09:08:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.143+0000 7f2509400640 1 -- 172.21.15.62:0/705828765 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504032810 con 0x7f250c0b54d0 2024-04-25T09:08:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.143+0000 7f250b200640 1 -- 172.21.15.62:0/705828765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b54d0 msgr2=0x7f250c0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.143+0000 7f250b200640 1 --2- 172.21.15.62:0/705828765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b54d0 0x7f250c0700c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f250c0721d0 tx=0x7f250402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.144+0000 7f250b200640 1 -- 172.21.15.62:0/705828765 shutdown_connections 2024-04-25T09:08:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.144+0000 7f250b200640 1 --2- 172.21.15.62:0/705828765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b54d0 0x7f250c0700c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.144+0000 7f250b200640 1 --2- 172.21.15.62:0/705828765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b36a0 0x7f250c0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.144+0000 7f250b200640 1 --2- 172.21.15.62:0/705828765 >> v2:172.21.15.121:3300/0 conn(0x7f250c0b2d80 0x7f250c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.144+0000 7f250b200640 1 -- 172.21.15.62:0/705828765 >> 172.21.15.62:0/705828765 conn(0x7f250c066f20 msgr2=0x7f250c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.144+0000 7f250b200640 1 -- 172.21.15.62:0/705828765 shutdown_connections 2024-04-25T09:08:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.144+0000 7f250b200640 1 -- 172.21.15.62:0/705828765 wait complete. 2024-04-25T09:08:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.145+0000 7f250b200640 1 Processor -- start 2024-04-25T09:08:03.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.145+0000 7f250b200640 1 -- start start 2024-04-25T09:08:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.146+0000 7f250b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b2d80 0x7f250c140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.146+0000 7f250b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f250c0b36a0 0x7f250c154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.146+0000 7f250b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b54d0 0x7f250c1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.146+0000 7f250b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0ce7f0 con 0x7f250c0b2d80 2024-04-25T09:08:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.146+0000 7f250b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0b40e0 con 0x7f250c0b36a0 2024-04-25T09:08:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.146+0000 7f250b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0ce970 con 0x7f250c0b54d0 2024-04-25T09:08:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b2d80 0x7f250c140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b2d80 0x7f250c140f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45504/0 (socket says 172.21.15.62:45504) 2024-04-25T09:08:03.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 -- 172.21.15.62:0/1747778709 learned_addr learned my addr 172.21.15.62:0/1747778709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2501400640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f250c0b36a0 0x7f250c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 -- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b54d0 msgr2=0x7f250c1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b54d0 0x7f250c1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 -- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f250c0b36a0 msgr2=0x7f250c154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f250c0b36a0 0x7f250c154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 -- 172.21.15.62:0/1747778709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c141450 con 0x7f250c0b2d80 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2501400640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f250c0b36a0 0x7f250c154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.147+0000 7f2509e00640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b2d80 0x7f250c140f10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00ce60 tx=0x7f24fc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.148+0000 7f2502a00640 1 -- 172.21.15.62:0/1747778709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00f890 con 0x7f250c0b2d80 2024-04-25T09:08:03.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.148+0000 7f2502a00640 1 -- 172.21.15.62:0/1747778709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc00fa30 con 0x7f250c0b2d80 2024-04-25T09:08:03.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.148+0000 7f2502a00640 1 -- 172.21.15.62:0/1747778709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00fe20 con 0x7f250c0b2d80 2024-04-25T09:08:03.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.148+0000 7f250b200640 1 -- 172.21.15.62:0/1747778709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250c154e00 con 0x7f250c0b2d80 2024-04-25T09:08:03.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.148+0000 7f250b200640 1 -- 172.21.15.62:0/1747778709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c1553c0 con 0x7f250c0b2d80 2024-04-25T09:08:03.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.150+0000 7f2502a00640 1 -- 172.21.15.62:0/1747778709 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24fc018400 con 0x7f250c0b2d80 2024-04-25T09:08:03.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.151+0000 7f2502a00640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f24e806b4d0 0x7f24e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.151+0000 7f2502a00640 1 -- 172.21.15.62:0/1747778709 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f24fc08b1a0 con 0x7f250c0b2d80 2024-04-25T09:08:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.152+0000 7f2513bfb740 1 Processor -- start 2024-04-25T09:08:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.152+0000 7f2513bfb740 1 -- start start 2024-04-25T09:08:03.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.152+0000 7f2501400640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f24e806b4d0 0x7f24e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.152+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da00d20 0x56501da02e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.152+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da01100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da2b7e0 0x56501da2bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2513bfb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501d963b90 con 0x56501da00d20 2024-04-25T09:08:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2513bfb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501d90d0b0 con 0x56501da2b7e0 2024-04-25T09:08:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2513bfb740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56501d95fc30 con 0x56501da3d3f0 2024-04-25T09:08:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2509e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da00d20 0x56501da02e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2509e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da00d20 0x56501da02e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45514/0 (socket says 172.21.15.62:45514) 2024-04-25T09:08:03.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2509e00640 1 -- 172.21.15.62:0/682409201 learned_addr learned my addr 172.21.15.62:0/682409201 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2501400640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f24e806b4d0 0x7f24e806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f250c1f6b00 tx=0x7f24f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2509e00640 1 -- 172.21.15.62:0/682409201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 msgr2=0x56501da01100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2509e00640 1 --2- 172.21.15.62:0/682409201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da01100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2509e00640 1 -- 172.21.15.62:0/682409201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da2b7e0 msgr2=0x56501da2bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2509e00640 1 --2- 172.21.15.62:0/682409201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da2b7e0 0x56501da2bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.153+0000 7f2509e00640 1 -- 172.21.15.62:0/682409201 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56501d7449f0 con 0x56501da00d20 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.154+0000 7f2509e00640 1 --2- 172.21.15.62:0/682409201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da00d20 0x56501da02e80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0619d0 tx=0x7f24fc00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb62e52065d93e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.154+0000 7f24faa00640 1 -- 172.21.15.62:0/682409201 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc01d040 con 0x56501da00d20 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.154+0000 7f24faa00640 1 -- 172.21.15.62:0/682409201 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc061f00 con 0x56501da00d20 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.154+0000 7f24faa00640 1 -- 172.21.15.62:0/682409201 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc055e80 con 0x56501da00d20 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.154+0000 7f2513bfb740 1 -- 172.21.15.62:0/682409201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da00d20 msgr2=0x56501da02e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.154+0000 7f2513bfb740 1 --2- 172.21.15.62:0/682409201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da00d20 0x56501da02e80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0619d0 tx=0x7f24fc00af90 comp rx=0 tx=0).stop 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.154+0000 7f2513bfb740 1 -- 172.21.15.62:0/682409201 shutdown_connections 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.154+0000 7f2513bfb740 1 --2- 172.21.15.62:0/682409201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da2b7e0 0x56501da2bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.155+0000 7f2513bfb740 1 --2- 172.21.15.62:0/682409201 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da01100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.155+0000 7f2513bfb740 1 --2- 172.21.15.62:0/682409201 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da00d20 0x56501da02e80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.155+0000 7f2513bfb740 1 -- 172.21.15.62:0/682409201 >> 172.21.15.62:0/682409201 conn(0x56501d9c8fb0 msgr2=0x56501da467e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.155+0000 7f2513bfb740 1 -- 172.21.15.62:0/682409201 shutdown_connections 2024-04-25T09:08:03.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.155+0000 7f2513bfb740 1 -- 172.21.15.62:0/682409201 wait complete. 2024-04-25T09:08:03.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.155+0000 7f2513bfb740 1 -- start start 2024-04-25T09:08:03.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.155+0000 7f2513bfb740 1 Processor -- start 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.156+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.156+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.156+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da25c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.156+0000 7f2513bfb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501d902570 con 0x56501da2b7e0 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.156+0000 7f2513bfb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501da29330 con 0x56501da00d20 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.156+0000 7f2513bfb740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56501da2a4e0 con 0x56501da3d3f0 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.156+0000 7f2513bfb740 10 client.0 _ll_register_callbacks cb 0x56501d677dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2501400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da56810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2501400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da56810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45520/0 (socket says 172.21.15.62:45520) 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2501400640 1 -- 172.21.15.62:0/2883371284 learned_addr learned my addr 172.21.15.62:0/2883371284 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2509e00640 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2501400640 1 -- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 msgr2=0x56501da25c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f250a800640 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da25c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2501400640 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da25c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2501400640 1 -- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 msgr2=0x56501da2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2501400640 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2501400640 1 -- 172.21.15.62:0/2883371284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56501d6382e0 con 0x56501da2b7e0 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.157+0000 7f2509e00640 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f2501400640 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da56810 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f24f4006fd0 tx=0x7f24f400ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9951bf42a682fb2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f24ef400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:03.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f24ef400640 1 -- 172.21.15.62:0/2883371284 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f400f040 con 0x56501da2b7e0 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f24ef400640 1 -- 172.21.15.62:0/2883371284 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f24f40091c0 con 0x56501da2b7e0 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f2513bfb740 1 -- 172.21.15.62:0/2883371284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24fc061f00 con 0x56501da2b7e0 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f24ef400640 1 -- 172.21.15.62:0/2883371284 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24f4003ca0 con 0x56501da2b7e0 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f2513bfb740 10 client.6006 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f2513bfb740 1 -- 172.21.15.62:0/2883371284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x56501d7a5180 con 0x56501da2b7e0 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.158+0000 7f24ef400640 1 -- 172.21.15.62:0/2883371284 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc061f00 con 0x56501da2b7e0 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f24ef400640 1 -- 172.21.15.62:0/2883371284 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24f4012070 con 0x56501da2b7e0 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f2513bfb740 10 client.6006 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f2513bfb740 10 client.6006 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f2513bfb740 20 client.6006 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:03.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f2513bfb740 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56501da294b0 0x56501da29910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f2513bfb740 4 client.6006 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f2513bfb740 1 -- 172.21.15.62:0/2883371284 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56501d9710d0 con 0x56501da294b0 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f2509e00640 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56501da294b0 0x56501da29910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f2509e00640 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56501da294b0 0x56501da29910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9045f4ab806edb9b server_cookie=9108691cba3a580c in_seq=0 out_seq=0 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.159+0000 7f24ef400640 10 client.6006 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f24ef400640 1 -- 172.21.15.62:0/2883371284 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56501d9710d0 con 0x56501da294b0 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f24ef400640 10 client.6006 handle_command_reply: tid=0 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 client.6006 shutdown 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 -- 172.21.15.62:0/2883371284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 msgr2=0x56501da56810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da56810 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f24f4006fd0 tx=0x7f24f400ee10 comp rx=0 tx=0).stop 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 -- 172.21.15.62:0/2883371284 shutdown_connections 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56501da294b0 0x56501da29910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da56810 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da25c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f24ef400640 0 client.6006 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 --2- 172.21.15.62:0/2883371284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 -- 172.21.15.62:0/2883371284 >> 172.21.15.62:0/2883371284 conn(0x56501d9c8fb0 msgr2=0x56501da49880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.160+0000 7f2513bfb740 1 -- 172.21.15.62:0/2883371284 shutdown_connections 2024-04-25T09:08:03.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.161+0000 7f2513bfb740 1 -- 172.21.15.62:0/2883371284 wait complete. 2024-04-25T09:08:03.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.161+0000 7f2513bfb740 20 client.6006 trim_cache size 0 max 16384 2024-04-25T09:08:03.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.177+0000 7f2513bfb740 1 Processor -- start 2024-04-25T09:08:03.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.177+0000 7f2513bfb740 1 -- start start 2024-04-25T09:08:03.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da28ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da2a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da2e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2513bfb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c200150 con 0x56501da2b7e0 2024-04-25T09:08:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2513bfb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c1ff270 con 0x56501da00d20 2024-04-25T09:08:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2513bfb740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c15c080 con 0x56501da3d3f0 2024-04-25T09:08:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2501400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da2a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2501400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da2a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45524/0 (socket says 172.21.15.62:45524) 2024-04-25T09:08:03.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.178+0000 7f2501400640 1 -- 172.21.15.62:0/3266526055 learned_addr learned my addr 172.21.15.62:0/3266526055 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f2501400640 1 -- 172.21.15.62:0/3266526055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 msgr2=0x56501da2e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f2501400640 1 --2- 172.21.15.62:0/3266526055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da2e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f2501400640 1 -- 172.21.15.62:0/3266526055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 msgr2=0x56501da28ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f2501400640 1 --2- 172.21.15.62:0/3266526055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da28ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f2501400640 1 -- 172.21.15.62:0/3266526055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56501d6113a0 con 0x56501da2b7e0 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f2501400640 1 --2- 172.21.15.62:0/3266526055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da2a750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f24f4008010 tx=0x7f24f401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d71e108c917e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f24fa000640 1 -- 172.21.15.62:0/3266526055 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f40091c0 con 0x56501da2b7e0 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f24fa000640 1 -- 172.21.15.62:0/3266526055 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f40136b0 con 0x56501da2b7e0 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.179+0000 7f24fa000640 1 -- 172.21.15.62:0/3266526055 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4013aa0 con 0x56501da2b7e0 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 -- 172.21.15.62:0/3266526055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 msgr2=0x56501da2a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 --2- 172.21.15.62:0/3266526055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da2a750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f24f4008010 tx=0x7f24f401d000 comp rx=0 tx=0).stop 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 -- 172.21.15.62:0/3266526055 shutdown_connections 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 --2- 172.21.15.62:0/3266526055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da2b7e0 0x56501da2a750 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 --2- 172.21.15.62:0/3266526055 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da3d3f0 0x56501da2e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 --2- 172.21.15.62:0/3266526055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 -- 172.21.15.62:0/3266526055 >> 172.21.15.62:0/3266526055 conn(0x56501d9c8fb0 msgr2=0x56501d9f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 -- 172.21.15.62:0/3266526055 shutdown_connections 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 -- 172.21.15.62:0/3266526055 wait complete. 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.180+0000 7f2513bfb740 1 -- start start 2024-04-25T09:08:03.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.181+0000 7f2513bfb740 1 Processor -- start 2024-04-25T09:08:03.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da2b7e0 0x56501da9a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f2513bfb740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da3d3f0 0x56501da2a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f2513bfb740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501da68e40 con 0x56501da3d3f0 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f2513bfb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c15c680 con 0x56501da00d20 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f2513bfb740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56501da290e0 con 0x56501da2b7e0 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f2513bfb740 10 client.0 _ll_register_callbacks cb 0x7f24fc022f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f250a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da3d3f0 0x56501da2a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f250a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da3d3f0 0x56501da2a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45534/0 (socket says 172.21.15.62:45534) 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f250a800640 1 -- 172.21.15.62:0/4107418958 learned_addr learned my addr 172.21.15.62:0/4107418958 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f250a800640 1 -- 172.21.15.62:0/4107418958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da2b7e0 msgr2=0x56501da9a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f2509e00640 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f250a800640 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da2b7e0 0x56501da9a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f250a800640 1 -- 172.21.15.62:0/4107418958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 msgr2=0x56501da2e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f250a800640 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.182+0000 7f250a800640 1 -- 172.21.15.62:0/4107418958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56501da9c940 con 0x56501da3d3f0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f250a800640 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da3d3f0 0x56501da2a4c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f250402a7d0 tx=0x7f2504032a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faab0fb27e7eb45b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f24ef400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f24ef400640 1 -- 172.21.15.62:0/4107418958 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504037040 con 0x56501da3d3f0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f24ef400640 1 -- 172.21.15.62:0/4107418958 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f250403d630 con 0x56501da3d3f0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f2513bfb740 1 -- 172.21.15.62:0/4107418958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24f40136b0 con 0x56501da3d3f0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f24ef400640 1 -- 172.21.15.62:0/4107418958 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2504049050 con 0x56501da3d3f0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f2513bfb740 10 client.6012 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f2513bfb740 1 -- 172.21.15.62:0/4107418958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x56501da9d700 con 0x56501da3d3f0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.183+0000 7f24ef400640 1 -- 172.21.15.62:0/4107418958 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f40136b0 con 0x56501da3d3f0 2024-04-25T09:08:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f24ef400640 1 -- 172.21.15.62:0/4107418958 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f250404c020 con 0x56501da3d3f0 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f2513bfb740 10 client.6012 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f2513bfb740 10 client.6012 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f2513bfb740 20 client.6012 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f2513bfb740 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56501da2aa00 0x56501da3c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f2513bfb740 4 client.6012 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f2513bfb740 1 -- 172.21.15.62:0/4107418958 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x56501d96b420 con 0x56501da2aa00 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f2509e00640 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56501da2aa00 0x56501da3c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f2509e00640 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56501da2aa00 0x56501da3c170 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3cb8d74823c3661 server_cookie=aa3908d2e40341ca in_seq=0 out_seq=0 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.184+0000 7f24ef400640 10 client.6012 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f24ef400640 1 -- 172.21.15.62:0/4107418958 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x56501d96b420 con 0x56501da2aa00 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f24ef400640 10 client.6012 handle_command_reply: tid=0 2024-04-25T09:08:03.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 client.6012 shutdown 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 -- 172.21.15.62:0/4107418958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da3d3f0 msgr2=0x56501da2a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da3d3f0 0x56501da2a4c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f250402a7d0 tx=0x7f2504032a90 comp rx=0 tx=0).stop 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 -- 172.21.15.62:0/4107418958 shutdown_connections 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56501da2aa00 0x56501da3c170 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56501da3d3f0 0x56501da2a4c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56501da2b7e0 0x56501da9a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 --2- 172.21.15.62:0/4107418958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56501da00d20 0x56501da2e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f24ef400640 0 client.6012 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 -- 172.21.15.62:0/4107418958 >> 172.21.15.62:0/4107418958 conn(0x56501d9c8fb0 msgr2=0x56501da28820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 -- 172.21.15.62:0/4107418958 shutdown_connections 2024-04-25T09:08:03.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 1 -- 172.21.15.62:0/4107418958 wait complete. 2024-04-25T09:08:03.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.185+0000 7f2513bfb740 20 client.6012 trim_cache size 0 max 16384 2024-04-25T09:08:03.179 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:03.179 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": -2 2024-04-25T09:08:03.179 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 -- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f24e806b4d0 msgr2=0x7f24e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f24e806b4d0 0x7f24e806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f250c1f6b00 tx=0x7f24f4008040 comp rx=0 tx=0).stop 2024-04-25T09:08:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 -- 172.21.15.62:0/1747778709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b2d80 msgr2=0x7f250c140f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b2d80 0x7f250c140f10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00ce60 tx=0x7f24fc00a650 comp rx=0 tx=0).stop 2024-04-25T09:08:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 -- 172.21.15.62:0/1747778709 shutdown_connections 2024-04-25T09:08:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f24e806b4d0 0x7f24e806d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f250c0b54d0 0x7f250c1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f250c0b36a0 0x7f250c154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 --2- 172.21.15.62:0/1747778709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f250c0b2d80 0x7f250c140f10 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 -- 172.21.15.62:0/1747778709 >> 172.21.15.62:0/1747778709 conn(0x7f250c066f20 msgr2=0x7f250c159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 -- 172.21.15.62:0/1747778709 shutdown_connections 2024-04-25T09:08:03.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.186+0000 7f250b200640 1 -- 172.21.15.62:0/1747778709 wait complete. 2024-04-25T09:08:03.190 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T09:08:03.190 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:08:03.191 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:03.191 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:08:03.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.400+0000 7f559a600640 1 Processor -- start 2024-04-25T09:08:03.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.400+0000 7f559a600640 1 -- start start 2024-04-25T09:08:03.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.402+0000 7f559a600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f559406d2c0 0x7f55940ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.402+0000 7f559a600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f559406d6a0 0x7f55940ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.402+0000 7f559a600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55940ced40 con 0x7f55940b31a0 2024-04-25T09:08:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.402+0000 7f559a600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55940cf000 con 0x7f559406d2c0 2024-04-25T09:08:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.402+0000 7f559a600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55940cf2e0 con 0x7f559406d6a0 2024-04-25T09:08:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.402+0000 7f5599200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f559406d2c0 0x7f55940ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.402+0000 7f5599c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f55940b31a0 0x7f55940b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50678/0 (socket says 172.21.15.62:50678) 2024-04-25T09:08:03.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.402+0000 7f5599c00640 1 -- 172.21.15.62:0/2340355103 learned_addr learned my addr 172.21.15.62:0/2340355103 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.403+0000 7f5599200640 1 -- 172.21.15.62:0/2340355103 >> v2:172.21.15.121:3300/0 conn(0x7f559406d6a0 msgr2=0x7f55940ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.403+0000 7f5593e00640 1 --2- 172.21.15.62:0/2340355103 >> v2:172.21.15.121:3300/0 conn(0x7f559406d6a0 0x7f55940ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.403+0000 7f5599200640 1 --2- 172.21.15.62:0/2340355103 >> v2:172.21.15.121:3300/0 conn(0x7f559406d6a0 0x7f55940ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.403+0000 7f5599200640 1 -- 172.21.15.62:0/2340355103 >> v1:172.21.15.62:6789/0 conn(0x7f55940b31a0 legacy=0x7f55940b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.403+0000 7f5599200640 1 -- 172.21.15.62:0/2340355103 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55940b26c0 con 0x7f559406d2c0 2024-04-25T09:08:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.404+0000 7f5599200640 1 --2- 172.21.15.62:0/2340355103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f559406d2c0 0x7f55940ccde0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f55940722f0 tx=0x7f558802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0f195d565a9926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.404+0000 7f5593400640 1 -- 172.21.15.62:0/2340355103 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588037070 con 0x7f559406d2c0 2024-04-25T09:08:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f5593400640 1 -- 172.21.15.62:0/2340355103 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5588032440 con 0x7f559406d2c0 2024-04-25T09:08:03.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f5593400640 1 -- 172.21.15.62:0/2340355103 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588032810 con 0x7f559406d2c0 2024-04-25T09:08:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f559a600640 1 -- 172.21.15.62:0/2340355103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f559406d2c0 msgr2=0x7f55940ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f559a600640 1 --2- 172.21.15.62:0/2340355103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f559406d2c0 0x7f55940ccde0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f55940722f0 tx=0x7f558802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f559a600640 1 -- 172.21.15.62:0/2340355103 shutdown_connections 2024-04-25T09:08:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f559a600640 1 --2- 172.21.15.62:0/2340355103 >> v2:172.21.15.121:3300/0 conn(0x7f559406d6a0 0x7f55940ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f559a600640 1 --2- 172.21.15.62:0/2340355103 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f559406d2c0 0x7f55940ccde0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f559a600640 1 -- 172.21.15.62:0/2340355103 >> 172.21.15.62:0/2340355103 conn(0x7f5594066f20 msgr2=0x7f55940b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f559a600640 1 -- 172.21.15.62:0/2340355103 shutdown_connections 2024-04-25T09:08:03.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.405+0000 7f559a600640 1 -- 172.21.15.62:0/2340355103 wait complete. 2024-04-25T09:08:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.407+0000 7f559a600640 1 Processor -- start 2024-04-25T09:08:03.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.407+0000 7f559a600640 1 -- start start 2024-04-25T09:08:03.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.408+0000 7f559a600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f559406d2c0 0x7f55940cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.408+0000 7f559a600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f559406d6a0 0x7f55940ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.408+0000 7f559a600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55940b31a0 0x7f55940ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.408+0000 7f559a600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559406db20 con 0x7f55940b31a0 2024-04-25T09:08:03.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.408+0000 7f559a600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55940b2c80 con 0x7f559406d6a0 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.408+0000 7f559a600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55941448d0 con 0x7f559406d2c0 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.408+0000 7f5599c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55940b31a0 0x7f55940ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55940b31a0 0x7f55940ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45560/0 (socket says 172.21.15.62:45560) 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599c00640 1 -- 172.21.15.62:0/304716185 learned_addr learned my addr 172.21.15.62:0/304716185 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599c00640 1 -- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f559406d2c0 msgr2=0x7f55940cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599200640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f559406d2c0 0x7f55940cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5593e00640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f559406d6a0 0x7f55940ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599c00640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f559406d2c0 0x7f55940cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599c00640 1 -- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f559406d6a0 msgr2=0x7f55940ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599c00640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f559406d6a0 0x7f55940ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599c00640 1 -- 172.21.15.62:0/304716185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55941e6700 con 0x7f55940b31a0 2024-04-25T09:08:03.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5593e00640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f559406d6a0 0x7f55940ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.409+0000 7f5599c00640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55940b31a0 0x7f55940ade20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f558c00ca80 tx=0x7f558c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.410+0000 7f5590c00640 1 -- 172.21.15.62:0/304716185 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c017070 con 0x7f55940b31a0 2024-04-25T09:08:03.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.410+0000 7f5590c00640 1 -- 172.21.15.62:0/304716185 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c00ac30 con 0x7f55940b31a0 2024-04-25T09:08:03.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.410+0000 7f5590c00640 1 -- 172.21.15.62:0/304716185 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c012600 con 0x7f55940b31a0 2024-04-25T09:08:03.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.410+0000 7f559a600640 1 -- 172.21.15.62:0/304716185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55941e69e0 con 0x7f55940b31a0 2024-04-25T09:08:03.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.410+0000 7f559a600640 1 -- 172.21.15.62:0/304716185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559412fc90 con 0x7f55940b31a0 2024-04-25T09:08:03.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.412+0000 7f5590c00640 1 -- 172.21.15.62:0/304716185 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f558c01a070 con 0x7f55940b31a0 2024-04-25T09:08:03.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.413+0000 7f5590c00640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f557406b4d0 0x7f557406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.413+0000 7f5590c00640 1 -- 172.21.15.62:0/304716185 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f558c089e90 con 0x7f55940b31a0 2024-04-25T09:08:03.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.414+0000 7f559eb82740 1 Processor -- start 2024-04-25T09:08:03.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.414+0000 7f5599200640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f557406b4d0 0x7f557406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.414+0000 7f559eb82740 1 -- start start 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.415+0000 7f5599200640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f557406b4d0 0x7f557406d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5588034000 tx=0x7f558802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.415+0000 7f559eb82740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3c1ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.415+0000 7f559eb82740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c03a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.415+0000 7f559eb82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 0x557fc3c00250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.415+0000 7f559eb82740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc3b3cc40 con 0x557fc3c15090 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.415+0000 7f559eb82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc3ae6050 con 0x557fc3c03f90 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.415+0000 7f559eb82740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fc3b38ce0 con 0x557fc3c03670 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.415+0000 7f5593e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c03a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5593e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c03a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45576/0 (socket says 172.21.15.62:45576) 2024-04-25T09:08:03.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5593e00640 1 -- 172.21.15.62:0/384339730 learned_addr learned my addr 172.21.15.62:0/384339730 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5599200640 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3c1ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5599c00640 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 0x557fc3c00250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5593e00640 1 -- 172.21.15.62:0/384339730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 msgr2=0x557fc3c1ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5593e00640 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3c1ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5593e00640 1 -- 172.21.15.62:0/384339730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 msgr2=0x557fc3c00250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5593e00640 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 0x557fc3c00250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5593e00640 1 -- 172.21.15.62:0/384339730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fc391db00 con 0x557fc3c15090 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.416+0000 7f5593e00640 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c03a50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f558000b9a0 tx=0x7f558000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8032001e2485b1b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f5585600640 1 -- 172.21.15.62:0/384339730 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580016070 con 0x557fc3c15090 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f5585600640 1 -- 172.21.15.62:0/384339730 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558000dc10 con 0x557fc3c15090 2024-04-25T09:08:03.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f5585600640 1 -- 172.21.15.62:0/384339730 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580012460 con 0x557fc3c15090 2024-04-25T09:08:03.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f559eb82740 1 -- 172.21.15.62:0/384339730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 msgr2=0x557fc3c03a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f559eb82740 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c03a50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f558000b9a0 tx=0x7f558000be60 comp rx=0 tx=0).stop 2024-04-25T09:08:03.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f559eb82740 1 -- 172.21.15.62:0/384339730 shutdown_connections 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f559eb82740 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c03a50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f559eb82740 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 0x557fc3c00250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f559eb82740 1 --2- 172.21.15.62:0/384339730 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3c1ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.417+0000 7f559eb82740 1 -- 172.21.15.62:0/384339730 >> 172.21.15.62:0/384339730 conn(0x557fc3ba05e0 msgr2=0x557fc3c1e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.418+0000 7f559eb82740 1 -- 172.21.15.62:0/384339730 shutdown_connections 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.418+0000 7f559eb82740 1 -- 172.21.15.62:0/384339730 wait complete. 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.418+0000 7f559eb82740 1 -- start start 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.418+0000 7f559eb82740 1 Processor -- start 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.419+0000 7f559eb82740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3c328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.419+0000 7f559eb82740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03f90 0x557fc3c2e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.419+0000 7f559eb82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 0x557fc3c19c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.419+0000 7f559eb82740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc3bda9b0 con 0x557fc3c03f90 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.419+0000 7f559eb82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc3adb5d0 con 0x557fc3c15090 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.419+0000 7f559eb82740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fc3bd5230 con 0x557fc3c03670 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.419+0000 7f559eb82740 10 client.0 _ll_register_callbacks cb 0x557fc3b34a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03f90 0x557fc3c2e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03f90 0x557fc3c2e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45584/0 (socket says 172.21.15.62:45584) 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 -- 172.21.15.62:0/2640899848 learned_addr learned my addr 172.21.15.62:0/2640899848 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 -- 172.21.15.62:0/2640899848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 msgr2=0x557fc3c328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3c328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 -- 172.21.15.62:0/2640899848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 msgr2=0x557fc3c19c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 0x557fc3c19c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 -- 172.21.15.62:0/2640899848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fc38113f0 con 0x557fc3c03f90 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f5593e00640 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03f90 0x557fc3c2e230 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5580008010 tx=0x7f5580004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2076ea8e3c679f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.420+0000 7f557a000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f557a000640 1 -- 172.21.15.62:0/2640899848 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580016040 con 0x557fc3c03f90 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f557a000640 1 -- 172.21.15.62:0/2640899848 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f558000ddc0 con 0x557fc3c03f90 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 1 -- 172.21.15.62:0/2640899848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f558000dc10 con 0x557fc3c03f90 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f557a000640 1 -- 172.21.15.62:0/2640899848 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5580012cc0 con 0x557fc3c03f90 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 10 client.6027 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 1 -- 172.21.15.62:0/2640899848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x557fc397e290 con 0x557fc3c03f90 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f557a000640 1 -- 172.21.15.62:0/2640899848 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558000dc10 con 0x557fc3c03f90 2024-04-25T09:08:03.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f557a000640 1 -- 172.21.15.62:0/2640899848 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f558001b450 con 0x557fc3c03f90 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 10 client.6027 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 10 client.6027 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 20 client.6027 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x557fc3c1d550 0x557fc3c05b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 4 client.6027 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.421+0000 7f559eb82740 1 -- 172.21.15.62:0/2640899848 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fc3b4a180 con 0x557fc3c1d550 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.422+0000 7f5599200640 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x557fc3c1d550 0x557fc3c05b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.422+0000 7f5599200640 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x557fc3c1d550 0x557fc3c05b50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81bf28e6536c8ea server_cookie=4e98fc4681525719 in_seq=0 out_seq=0 2024-04-25T09:08:03.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.422+0000 7f557a000640 10 client.6027 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f557a000640 1 -- 172.21.15.62:0/2640899848 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fc3b4a180 con 0x557fc3c1d550 2024-04-25T09:08:03.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f557a000640 10 client.6027 handle_command_reply: tid=0 2024-04-25T09:08:03.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 client.6027 shutdown 2024-04-25T09:08:03.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 -- 172.21.15.62:0/2640899848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03f90 msgr2=0x557fc3c2e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03f90 0x557fc3c2e230 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5580008010 tx=0x7f5580004730 comp rx=0 tx=0).stop 2024-04-25T09:08:03.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 -- 172.21.15.62:0/2640899848 shutdown_connections 2024-04-25T09:08:03.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x557fc3c1d550 0x557fc3c05b50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03f90 0x557fc3c2e230 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 0x557fc3c19c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 --2- 172.21.15.62:0/2640899848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3c328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f557a000640 0 client.6027 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.423+0000 7f559eb82740 1 -- 172.21.15.62:0/2640899848 >> 172.21.15.62:0/2640899848 conn(0x557fc3ba05e0 msgr2=0x557fc3bfb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.424+0000 7f559eb82740 1 -- 172.21.15.62:0/2640899848 shutdown_connections 2024-04-25T09:08:03.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.424+0000 7f559eb82740 1 -- 172.21.15.62:0/2640899848 wait complete. 2024-04-25T09:08:03.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.424+0000 7f559eb82740 20 client.6027 trim_cache size 0 max 16384 2024-04-25T09:08:03.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.441+0000 7f559eb82740 1 Processor -- start 2024-04-25T09:08:03.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.441+0000 7f559eb82740 1 -- start start 2024-04-25T09:08:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f559eb82740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3bcf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f559eb82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 0x557fc3c21e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f559eb82740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c19c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f559eb82740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55941ffdc0 con 0x557fc3c15090 2024-04-25T09:08:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f559eb82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55941fefd0 con 0x557fc3c03f90 2024-04-25T09:08:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f559eb82740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5594130ec0 con 0x557fc3c03670 2024-04-25T09:08:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f5599c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c19c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f5599c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c19c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45594/0 (socket says 172.21.15.62:45594) 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f5599c00640 1 -- 172.21.15.62:0/46479350 learned_addr learned my addr 172.21.15.62:0/46479350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f5593e00640 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 0x557fc3c21e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.442+0000 7f5599200640 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3bcf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5599c00640 1 -- 172.21.15.62:0/46479350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 msgr2=0x557fc3bcf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5599c00640 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3bcf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5599c00640 1 -- 172.21.15.62:0/46479350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 msgr2=0x557fc3c21e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5599c00640 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 0x557fc3c21e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5599c00640 1 -- 172.21.15.62:0/46479350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fc37ea4b0 con 0x557fc3c15090 2024-04-25T09:08:03.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5599c00640 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c19c60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f558c0012e0 tx=0x7f558c05c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4517fa853e4e65c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5584c00640 1 -- 172.21.15.62:0/46479350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c070020 con 0x557fc3c15090 2024-04-25T09:08:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5584c00640 1 -- 172.21.15.62:0/46479350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c054c90 con 0x557fc3c15090 2024-04-25T09:08:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.443+0000 7f5584c00640 1 -- 172.21.15.62:0/46479350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c060a30 con 0x557fc3c15090 2024-04-25T09:08:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 -- 172.21.15.62:0/46479350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 msgr2=0x557fc3c19c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c19c60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f558c0012e0 tx=0x7f558c05c6c0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 -- 172.21.15.62:0/46479350 shutdown_connections 2024-04-25T09:08:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c15090 0x557fc3c19c60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c03f90 0x557fc3c21e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 --2- 172.21.15.62:0/46479350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03670 0x557fc3bcf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 -- 172.21.15.62:0/46479350 >> 172.21.15.62:0/46479350 conn(0x557fc3ba05e0 msgr2=0x557fc3c02c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 -- 172.21.15.62:0/46479350 shutdown_connections 2024-04-25T09:08:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.444+0000 7f559eb82740 1 -- 172.21.15.62:0/46479350 wait complete. 2024-04-25T09:08:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.445+0000 7f559eb82740 1 -- start start 2024-04-25T09:08:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.445+0000 7f559eb82740 1 Processor -- start 2024-04-25T09:08:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f559eb82740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03670 0x557fc3c1c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f559eb82740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03f90 0x557fc3c002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f559eb82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 0x557fc3c00820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f559eb82740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fc3c053e0 con 0x557fc3c03670 2024-04-25T09:08:03.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f559eb82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55941314c0 con 0x557fc3c15090 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f559eb82740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fc3c1cea0 con 0x557fc3c03f90 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f559eb82740 10 client.0 _ll_register_callbacks cb 0x7f558000b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f5599200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03670 0x557fc3c1c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f5599200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03670 0x557fc3c1c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45606/0 (socket says 172.21.15.62:45606) 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.446+0000 7f5599200640 1 -- 172.21.15.62:0/975256319 learned_addr learned my addr 172.21.15.62:0/975256319 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f5599200640 1 -- 172.21.15.62:0/975256319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03f90 msgr2=0x557fc3c002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f5599c00640 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 0x557fc3c00820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f5599200640 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03f90 0x557fc3c002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f5599200640 1 -- 172.21.15.62:0/975256319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 msgr2=0x557fc3c00820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f5599200640 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 0x557fc3c00820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f5599200640 1 -- 172.21.15.62:0/975256319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fc3c317f0 con 0x557fc3c03670 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f5599200640 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03670 0x557fc3c1c960 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5588009560 tx=0x7f5588004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b37fdf08522dd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f557a000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f557a000640 1 -- 172.21.15.62:0/975256319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558804d030 con 0x557fc3c03670 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f557a000640 1 -- 172.21.15.62:0/975256319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5588033b10 con 0x557fc3c03670 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.447+0000 7f559eb82740 1 -- 172.21.15.62:0/975256319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f558c054c90 con 0x557fc3c03670 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f557a000640 1 -- 172.21.15.62:0/975256319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5588059050 con 0x557fc3c03670 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f559eb82740 10 client.6033 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f559eb82740 1 -- 172.21.15.62:0/975256319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x557fc3c1c310 con 0x557fc3c03670 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f557a000640 1 -- 172.21.15.62:0/975256319 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c054c90 con 0x557fc3c03670 2024-04-25T09:08:03.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f557a000640 1 -- 172.21.15.62:0/975256319 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f558805c020 con 0x557fc3c03670 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f559eb82740 10 client.6033 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f559eb82740 10 client.6033 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f559eb82740 20 client.6033 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f559eb82740 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x557fc3c1bec0 0x557fc3c1f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f559eb82740 4 client.6033 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f559eb82740 1 -- 172.21.15.62:0/975256319 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x557fc3b444d0 con 0x557fc3c1bec0 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.448+0000 7f5593e00640 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x557fc3c1bec0 0x557fc3c1f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.449+0000 7f557a000640 10 client.6033 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.449+0000 7f5593e00640 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x557fc3c1bec0 0x557fc3c1f5b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2fdd59621af4abe server_cookie=ac65958150e139f6 in_seq=0 out_seq=0 2024-04-25T09:08:03.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f557a000640 1 -- 172.21.15.62:0/975256319 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557fc3b444d0 con 0x557fc3c1bec0 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f557a000640 10 client.6033 handle_command_reply: tid=0 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 client.6033 shutdown 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 -- 172.21.15.62:0/975256319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03670 msgr2=0x557fc3c1c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03670 0x557fc3c1c960 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5588009560 tx=0x7f5588004730 comp rx=0 tx=0).stop 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 -- 172.21.15.62:0/975256319 shutdown_connections 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x557fc3c1bec0 0x557fc3c1f5b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc3c15090 0x557fc3c00820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557fc3c03f90 0x557fc3c002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 --2- 172.21.15.62:0/975256319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557fc3c03670 0x557fc3c1c960 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f557a000640 0 client.6033 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 -- 172.21.15.62:0/975256319 >> 172.21.15.62:0/975256319 conn(0x557fc3ba05e0 msgr2=0x557fc3bced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 -- 172.21.15.62:0/975256319 shutdown_connections 2024-04-25T09:08:03.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 1 -- 172.21.15.62:0/975256319 wait complete. 2024-04-25T09:08:03.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.450+0000 7f559eb82740 20 client.6033 trim_cache size 0 max 16384 2024-04-25T09:08:03.444 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:03.444 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:08:03.444 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "97a7a95b-55f2-40d8-b6b0-d78549928951", 2024-04-25T09:08:03.445 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:08:03.445 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 -- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f557406b4d0 msgr2=0x7f557406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f557406b4d0 0x7f557406d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5588034000 tx=0x7f558802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 -- 172.21.15.62:0/304716185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55940b31a0 msgr2=0x7f55940ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55940b31a0 0x7f55940ade20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f558c00ca80 tx=0x7f558c00cf40 comp rx=0 tx=0).stop 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 -- 172.21.15.62:0/304716185 shutdown_connections 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f557406b4d0 0x7f557406d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55940b31a0 0x7f55940ade20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f559406d6a0 0x7f55940ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 --2- 172.21.15.62:0/304716185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f559406d2c0 0x7f55940cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 -- 172.21.15.62:0/304716185 >> 172.21.15.62:0/304716185 conn(0x7f5594066f20 msgr2=0x7f559415c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 -- 172.21.15.62:0/304716185 shutdown_connections 2024-04-25T09:08:03.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.451+0000 7f559a600640 1 -- 172.21.15.62:0/304716185 wait complete. 2024-04-25T09:08:03.458 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '97a7a95b-55f2-40d8-b6b0-d78549928951', 'mode': 'asynchronous'}' 2024-04-25T09:08:03.458 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:08:03.458 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:03.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:08:03.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.685+0000 7f780b200640 1 Processor -- start 2024-04-25T09:08:03.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.685+0000 7f780b200640 1 -- start start 2024-04-25T09:08:03.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.687+0000 7f780b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f780c06d2c0 0x7f780c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.687+0000 7f780b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0b31a0 0x7f780c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.688+0000 7f780b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c0cf000 con 0x7f780c06d6a0 2024-04-25T09:08:03.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.688+0000 7f780b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c0cf2e0 con 0x7f780c06d2c0 2024-04-25T09:08:03.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.688+0000 7f780b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0b26c0 con 0x7f780c0b31a0 2024-04-25T09:08:03.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.688+0000 7f7809e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f780c06d2c0 0x7f780c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.688+0000 7f7809400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f780c06d6a0 0x7f780c0ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:57992/0 (socket says 172.21.15.62:57992) 2024-04-25T09:08:03.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.688+0000 7f7809400640 1 -- 172.21.15.62:0/4286421554 learned_addr learned my addr 172.21.15.62:0/4286421554 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:03.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.688+0000 7f780a800640 1 --2- 172.21.15.62:0/4286421554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0b31a0 0x7f780c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.689+0000 7f7809e00640 1 -- 172.21.15.62:0/4286421554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0b31a0 msgr2=0x7f780c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.689+0000 7f7809e00640 1 --2- 172.21.15.62:0/4286421554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0b31a0 0x7f780c0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.689+0000 7f7809e00640 1 -- 172.21.15.62:0/4286421554 >> v1:172.21.15.121:6789/0 conn(0x7f780c06d6a0 legacy=0x7f780c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.689+0000 7f7809e00640 1 -- 172.21.15.62:0/4286421554 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c0b29a0 con 0x7f780c06d2c0 2024-04-25T09:08:03.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.691+0000 7f7809e00640 1 --2- 172.21.15.62:0/4286421554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 0x7f780c0ccde0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0097a0 tx=0x7f77fc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1617ddf3997a1b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.692+0000 7f7808a00640 1 -- 172.21.15.62:0/4286421554 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc037070 con 0x7f780c06d2c0 2024-04-25T09:08:03.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.692+0000 7f7808a00640 1 -- 172.21.15.62:0/4286421554 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc032440 con 0x7f780c06d2c0 2024-04-25T09:08:03.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.692+0000 7f7808a00640 1 -- 172.21.15.62:0/4286421554 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc032810 con 0x7f780c06d2c0 2024-04-25T09:08:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.692+0000 7f780b200640 1 -- 172.21.15.62:0/4286421554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 msgr2=0x7f780c0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.693+0000 7f780b200640 1 --2- 172.21.15.62:0/4286421554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 0x7f780c0ccde0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0097a0 tx=0x7f77fc02a180 comp rx=0 tx=0).stop 2024-04-25T09:08:03.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.693+0000 7f780b200640 1 -- 172.21.15.62:0/4286421554 shutdown_connections 2024-04-25T09:08:03.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.693+0000 7f780b200640 1 --2- 172.21.15.62:0/4286421554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0b31a0 0x7f780c0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.693+0000 7f780b200640 1 --2- 172.21.15.62:0/4286421554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 0x7f780c0ccde0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.693+0000 7f780b200640 1 -- 172.21.15.62:0/4286421554 >> 172.21.15.62:0/4286421554 conn(0x7f780c066f20 msgr2=0x7f780c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.693+0000 7f780b200640 1 -- 172.21.15.62:0/4286421554 shutdown_connections 2024-04-25T09:08:03.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.693+0000 7f780b200640 1 -- 172.21.15.62:0/4286421554 wait complete. 2024-04-25T09:08:03.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.695+0000 7f780b200640 1 Processor -- start 2024-04-25T09:08:03.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.695+0000 7f780b200640 1 -- start start 2024-04-25T09:08:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.696+0000 7f780b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 0x7f780c0ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.696+0000 7f780b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c06d6a0 0x7f780c139d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.696+0000 7f780b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f780c0b31a0 0x7f780c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.696+0000 7f780b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1448d0 con 0x7f780c0b31a0 2024-04-25T09:08:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.696+0000 7f780b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c06db20 con 0x7f780c06d2c0 2024-04-25T09:08:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.696+0000 7f780b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c144a50 con 0x7f780c06d6a0 2024-04-25T09:08:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f780a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f780c0b31a0 0x7f780c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f780a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f780c0b31a0 0x7f780c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45620/0 (socket says 172.21.15.62:45620) 2024-04-25T09:08:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f780a800640 1 -- 172.21.15.62:0/658506861 learned_addr learned my addr 172.21.15.62:0/658506861 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f7809400640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c06d6a0 0x7f780c139d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f7809e00640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 0x7f780c0ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f780a800640 1 -- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c06d6a0 msgr2=0x7f780c139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f780a800640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c06d6a0 0x7f780c139d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f780a800640 1 -- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 msgr2=0x7f780c0ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f780a800640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 0x7f780c0ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.697+0000 7f780a800640 1 -- 172.21.15.62:0/658506861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c0700c0 con 0x7f780c0b31a0 2024-04-25T09:08:03.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.698+0000 7f780a800640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f780c0b31a0 0x7f780c0ca700 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f780400efc0 tx=0x7f780400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.698+0000 7f7802000640 1 -- 172.21.15.62:0/658506861 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804018020 con 0x7f780c0b31a0 2024-04-25T09:08:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.698+0000 7f7802000640 1 -- 172.21.15.62:0/658506861 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7804014480 con 0x7f780c0b31a0 2024-04-25T09:08:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.698+0000 7f780b200640 1 -- 172.21.15.62:0/658506861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c0703a0 con 0x7f780c0b31a0 2024-04-25T09:08:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.698+0000 7f7802000640 1 -- 172.21.15.62:0/658506861 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804014870 con 0x7f780c0b31a0 2024-04-25T09:08:03.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.698+0000 7f780b200640 1 -- 172.21.15.62:0/658506861 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c070960 con 0x7f780c0b31a0 2024-04-25T09:08:03.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.702+0000 7f7802000640 1 -- 172.21.15.62:0/658506861 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7804005d10 con 0x7f780c0b31a0 2024-04-25T09:08:03.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.703+0000 7f7802000640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.703+0000 7f7802000640 1 -- 172.21.15.62:0/658506861 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f780408adc0 con 0x7f780c0b31a0 2024-04-25T09:08:03.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.703+0000 7f781382b740 1 Processor -- start 2024-04-25T09:08:03.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.703+0000 7f781382b740 1 -- start start 2024-04-25T09:08:03.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.704+0000 7f7809e00640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.704+0000 7f781382b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09d02810 0x55ba09cb51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.704+0000 7f781382b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 0x55ba09ce4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.704+0000 7f781382b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09ce51f0 0x55ba09cba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.704+0000 7f781382b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09c1dcf0 con 0x55ba09d02810 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.704+0000 7f781382b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09bc71d0 con 0x55ba09ce51f0 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.704+0000 7f781382b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba09c19d90 con 0x55ba09ce5930 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09d02810 0x55ba09cb51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09d02810 0x55ba09cb51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45634/0 (socket says 172.21.15.62:45634) 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 -- 172.21.15.62:0/3142775428 learned_addr learned my addr 172.21.15.62:0/3142775428 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f77fc034000 tx=0x7f77fc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f780a800640 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09ce51f0 0x55ba09cba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809400640 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 0x55ba09ce4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 -- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 msgr2=0x55ba09ce4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 0x55ba09ce4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 -- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09ce51f0 msgr2=0x55ba09cba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09ce51f0 0x55ba09cba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 -- 172.21.15.62:0/3142775428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba099fef00 con 0x55ba09d02810 2024-04-25T09:08:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f780a800640 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09ce51f0 0x55ba09cba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.705+0000 7f7809e00640 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09d02810 0x55ba09cb51a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009890 tx=0x7f77fc008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be35960967470b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.706+0000 7f77f6600640 1 -- 172.21.15.62:0/3142775428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc04d070 con 0x55ba09d02810 2024-04-25T09:08:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.706+0000 7f77f6600640 1 -- 172.21.15.62:0/3142775428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc03e660 con 0x55ba09d02810 2024-04-25T09:08:03.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.706+0000 7f77f6600640 1 -- 172.21.15.62:0/3142775428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc037040 con 0x55ba09d02810 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.706+0000 7f781382b740 1 -- 172.21.15.62:0/3142775428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09d02810 msgr2=0x55ba09cb51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.706+0000 7f781382b740 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09d02810 0x55ba09cb51a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009890 tx=0x7f77fc008030 comp rx=0 tx=0).stop 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.707+0000 7f781382b740 1 -- 172.21.15.62:0/3142775428 shutdown_connections 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.707+0000 7f781382b740 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09ce51f0 0x55ba09cba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.707+0000 7f781382b740 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 0x55ba09ce4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.707+0000 7f781382b740 1 --2- 172.21.15.62:0/3142775428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09d02810 0x55ba09cb51a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.707+0000 7f781382b740 1 -- 172.21.15.62:0/3142775428 >> 172.21.15.62:0/3142775428 conn(0x55ba09cb7410 msgr2=0x55ba09cffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.707+0000 7f781382b740 1 -- 172.21.15.62:0/3142775428 shutdown_connections 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.707+0000 7f781382b740 1 -- 172.21.15.62:0/3142775428 wait complete. 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.707+0000 7f781382b740 1 -- start start 2024-04-25T09:08:03.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.708+0000 7f781382b740 1 Processor -- start 2024-04-25T09:08:03.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f781382b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cd5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f781382b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cf6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f781382b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09cf8820 0x55ba09d00de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f781382b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09cb95c0 con 0x55ba09ce5930 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f781382b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09bbcb50 con 0x55ba09d02810 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f781382b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba09cb89b0 con 0x55ba09cf8820 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f781382b740 10 client.0 _ll_register_callbacks cb 0x55ba09bde370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cd5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cd5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45636/0 (socket says 172.21.15.62:45636) 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809e00640 1 -- 172.21.15.62:0/702101574 learned_addr learned my addr 172.21.15.62:0/702101574 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809400640 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cf6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f780a800640 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09cf8820 0x55ba09d00de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809e00640 1 -- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09cf8820 msgr2=0x55ba09d00de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809e00640 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09cf8820 0x55ba09d00de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809e00640 1 -- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 msgr2=0x55ba09cf6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809e00640 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cf6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809e00640 1 -- 172.21.15.62:0/702101574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba098f27f0 con 0x55ba09ce5930 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.709+0000 7f7809400640 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cf6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.710+0000 7f7809e00640 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cd5760 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f77fc04b040 tx=0x7f77fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ba1fbeee2339d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.710+0000 7f77eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.710+0000 7f77eaa00640 1 -- 172.21.15.62:0/702101574 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc057070 con 0x55ba09ce5930 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.710+0000 7f77eaa00640 1 -- 172.21.15.62:0/702101574 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f77fc03ec60 con 0x55ba09ce5930 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.710+0000 7f781382b740 1 -- 172.21.15.62:0/702101574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77fc03e660 con 0x55ba09ce5930 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.710+0000 7f77eaa00640 1 -- 172.21.15.62:0/702101574 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77fc05f050 con 0x55ba09ce5930 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.710+0000 7f781382b740 10 client.6042 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.710+0000 7f781382b740 1 -- 172.21.15.62:0/702101574 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55ba09a5f690 con 0x55ba09ce5930 2024-04-25T09:08:03.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f77eaa00640 1 -- 172.21.15.62:0/702101574 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc03e660 con 0x55ba09ce5930 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f77eaa00640 1 -- 172.21.15.62:0/702101574 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f77fc062020 con 0x55ba09ce5930 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f781382b740 10 client.6042 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f781382b740 10 client.6042 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f781382b740 20 client.6042 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f781382b740 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ba09d0fb80 0x55ba09d0ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f781382b740 4 client.6042 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f781382b740 1 -- 172.21.15.62:0/702101574 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba09c2b230 con 0x55ba09d0fb80 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.711+0000 7f7809400640 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ba09d0fb80 0x55ba09d0ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.712+0000 7f7809400640 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ba09d0fb80 0x55ba09d0ffc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc04a98f3c333a1 server_cookie=59dec9c37ba6bfcb in_seq=0 out_seq=0 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.712+0000 7f77eaa00640 10 client.6042 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.712+0000 7f77eaa00640 1 -- 172.21.15.62:0/702101574 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba09c2b230 con 0x55ba09d0fb80 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.712+0000 7f77eaa00640 10 client.6042 handle_command_reply: tid=0 2024-04-25T09:08:03.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 client.6042 shutdown 2024-04-25T09:08:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 -- 172.21.15.62:0/702101574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 msgr2=0x55ba09cd5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cd5760 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f77fc04b040 tx=0x7f77fc004730 comp rx=0 tx=0).stop 2024-04-25T09:08:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 -- 172.21.15.62:0/702101574 shutdown_connections 2024-04-25T09:08:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ba09d0fb80 0x55ba09d0ffc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09cf8820 0x55ba09d00de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cf6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f77eaa00640 0 client.6042 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 --2- 172.21.15.62:0/702101574 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cd5760 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 -- 172.21.15.62:0/702101574 >> 172.21.15.62:0/702101574 conn(0x55ba09ce51f0 msgr2=0x55ba09d01d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 -- 172.21.15.62:0/702101574 shutdown_connections 2024-04-25T09:08:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 1 -- 172.21.15.62:0/702101574 wait complete. 2024-04-25T09:08:03.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.713+0000 7f781382b740 20 client.6042 trim_cache size 0 max 16384 2024-04-25T09:08:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.731+0000 7f781382b740 1 Processor -- start 2024-04-25T09:08:03.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.731+0000 7f781382b740 1 -- start start 2024-04-25T09:08:03.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f781382b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 0x55ba09d64c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f781382b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09cf8820 0x55ba09cb5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f781382b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cb6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f781382b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c200040 con 0x55ba09cf8820 2024-04-25T09:08:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f781382b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1ff200 con 0x55ba09d02810 2024-04-25T09:08:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f781382b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1fe800 con 0x55ba09ce5930 2024-04-25T09:08:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f7809400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09cf8820 0x55ba09cb5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f7809400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09cf8820 0x55ba09cb5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45652/0 (socket says 172.21.15.62:45652) 2024-04-25T09:08:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f7809400640 1 -- 172.21.15.62:0/2604985238 learned_addr learned my addr 172.21.15.62:0/2604985238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f780a800640 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cb6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.732+0000 7f7809e00640 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 0x55ba09d64c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f7809400640 1 -- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 msgr2=0x55ba09d64c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f7809400640 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 0x55ba09d64c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f7809400640 1 -- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 msgr2=0x55ba09cb6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f7809400640 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cb6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f7809400640 1 -- 172.21.15.62:0/2604985238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba098cb8b0 con 0x55ba09cf8820 2024-04-25T09:08:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f780a800640 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cb6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f7809400640 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09cf8820 0x55ba09cb5ee0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005880 tx=0x7f77f8001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68345f266bd5e7c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f77f5c00640 1 -- 172.21.15.62:0/2604985238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8018890 con 0x55ba09cf8820 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f77f5c00640 1 -- 172.21.15.62:0/2604985238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f8018e50 con 0x55ba09cf8820 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.733+0000 7f77f5c00640 1 -- 172.21.15.62:0/2604985238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8005af0 con 0x55ba09cf8820 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 -- 172.21.15.62:0/2604985238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09cf8820 msgr2=0x55ba09cb5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09cf8820 0x55ba09cb5ee0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005880 tx=0x7f77f8001d10 comp rx=0 tx=0).stop 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 -- 172.21.15.62:0/2604985238 shutdown_connections 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09cf8820 0x55ba09cb5ee0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09d02810 0x55ba09cb6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 --2- 172.21.15.62:0/2604985238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09ce5930 0x55ba09d64c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 -- 172.21.15.62:0/2604985238 >> 172.21.15.62:0/2604985238 conn(0x55ba09ce51f0 msgr2=0x55ba09d01c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 -- 172.21.15.62:0/2604985238 shutdown_connections 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.734+0000 7f781382b740 1 -- 172.21.15.62:0/2604985238 wait complete. 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.735+0000 7f781382b740 1 -- start start 2024-04-25T09:08:03.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.735+0000 7f781382b740 1 Processor -- start 2024-04-25T09:08:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.736+0000 7f781382b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cf5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.736+0000 7f781382b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09cf8820 0x55ba09d53550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.736+0000 7f781382b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09d02810 0x55ba09cb5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.736+0000 7f781382b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1fec80 con 0x55ba09ce5930 2024-04-25T09:08:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.736+0000 7f781382b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09cfe960 con 0x55ba09cf8820 2024-04-25T09:08:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.736+0000 7f781382b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba09cf64c0 con 0x55ba09d02810 2024-04-25T09:08:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.736+0000 7f781382b740 10 client.0 _ll_register_callbacks cb 0x7f77fc035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:03.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.736+0000 7f7809e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cf5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cf5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45654/0 (socket says 172.21.15.62:45654) 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809e00640 1 -- 172.21.15.62:0/1108621457 learned_addr learned my addr 172.21.15.62:0/1108621457 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f780a800640 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09d02810 0x55ba09cb5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809400640 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09cf8820 0x55ba09d53550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809e00640 1 -- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09d02810 msgr2=0x55ba09cb5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809e00640 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09d02810 0x55ba09cb5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809e00640 1 -- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09cf8820 msgr2=0x55ba09d53550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809e00640 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09cf8820 0x55ba09d53550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809e00640 1 -- 172.21.15.62:0/1108621457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba09ce5fc0 con 0x55ba09ce5930 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809400640 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09cf8820 0x55ba09d53550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f7809e00640 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cf5dc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f77fc04b040 tx=0x7f77fc03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c3b6b7671749c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f77eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:03.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.737+0000 7f77eaa00640 1 -- 172.21.15.62:0/1108621457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc057040 con 0x55ba09ce5930 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f77eaa00640 1 -- 172.21.15.62:0/1108621457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f77fc03cc80 con 0x55ba09ce5930 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 1 -- 172.21.15.62:0/1108621457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77f8018e50 con 0x55ba09ce5930 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f77eaa00640 1 -- 172.21.15.62:0/1108621457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77fc04d040 con 0x55ba09ce5930 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 10 client.6048 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 1 -- 172.21.15.62:0/1108621457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55ba09ce6de0 con 0x55ba09ce5930 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f77eaa00640 1 -- 172.21.15.62:0/1108621457 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f8018e50 con 0x55ba09ce5930 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f77eaa00640 1 -- 172.21.15.62:0/1108621457 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f77fc062040 con 0x55ba09ce5930 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 10 client.6048 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 10 client.6048 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 20 client.6048 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ba09cb6420 0x55ba09ce7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 4 client.6048 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.738+0000 7f781382b740 1 -- 172.21.15.62:0/1108621457 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55ba09c25580 con 0x55ba09cb6420 2024-04-25T09:08:03.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.739+0000 7f7809400640 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ba09cb6420 0x55ba09ce7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.739+0000 7f7809400640 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ba09cb6420 0x55ba09ce7dc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96004b1b82b3ccb server_cookie=ad804798cbbabc63 in_seq=0 out_seq=0 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.739+0000 7f77eaa00640 10 client.6048 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f77eaa00640 1 -- 172.21.15.62:0/1108621457 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ba09c25580 con 0x55ba09cb6420 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f77eaa00640 10 client.6048 handle_command_reply: tid=0 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 client.6048 shutdown 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 -- 172.21.15.62:0/1108621457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 msgr2=0x55ba09cf5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cf5dc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f77fc04b040 tx=0x7f77fc03d000 comp rx=0 tx=0).stop 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 -- 172.21.15.62:0/1108621457 shutdown_connections 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55ba09cb6420 0x55ba09ce7dc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba09d02810 0x55ba09cb5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba09cf8820 0x55ba09d53550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 --2- 172.21.15.62:0/1108621457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba09ce5930 0x55ba09cf5dc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 -- 172.21.15.62:0/1108621457 >> 172.21.15.62:0/1108621457 conn(0x55ba09ce51f0 msgr2=0x55ba09cff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f77eaa00640 0 client.6048 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 -- 172.21.15.62:0/1108621457 shutdown_connections 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.740+0000 7f781382b740 1 -- 172.21.15.62:0/1108621457 wait complete. 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f781382b740 20 client.6048 trim_cache size 0 max 16384 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "3e8e373d-f899-4aba-8475-56ce734d4220", 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:08:03.732 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 -- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 msgr2=0x7f77ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f77fc034000 tx=0x7f77fc02e000 comp rx=0 tx=0).stop 2024-04-25T09:08:03.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 -- 172.21.15.62:0/658506861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f780c0b31a0 msgr2=0x7f780c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f780c0b31a0 0x7f780c0ca700 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f780400efc0 tx=0x7f780400c490 comp rx=0 tx=0).stop 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 -- 172.21.15.62:0/658506861 shutdown_connections 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f780c0b31a0 0x7f780c0ca700 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c06d6a0 0x7f780c139d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 --2- 172.21.15.62:0/658506861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c06d2c0 0x7f780c0ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 -- 172.21.15.62:0/658506861 >> 172.21.15.62:0/658506861 conn(0x7f780c066f20 msgr2=0x7f780c15c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.741+0000 7f780b200640 1 -- 172.21.15.62:0/658506861 shutdown_connections 2024-04-25T09:08:03.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.742+0000 7f780b200640 1 -- 172.21.15.62:0/658506861 wait complete. 2024-04-25T09:08:03.747 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '3e8e373d-f899-4aba-8475-56ce734d4220', 'mode': 'asynchronous'}' 2024-04-25T09:08:03.747 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T09:08:03.747 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:03.748 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T09:08:03.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.954+0000 7f82d1e00640 1 Processor -- start 2024-04-25T09:08:03.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.954+0000 7f82d1e00640 1 -- start start 2024-04-25T09:08:03.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.956+0000 7f82d1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82cc06d2c0 0x7f82cc0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.956+0000 7f82d1e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82cc06d6a0 0x7f82cc0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.956+0000 7f82d1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.956+0000 7f82d1e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc0ced40 con 0x7f82cc06d2c0 2024-04-25T09:08:03.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.956+0000 7f82d1e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc0cf000 con 0x7f82cc06d6a0 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.956+0000 7f82d1e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0cf2e0 con 0x7f82cc0b31a0 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.956+0000 7f82d0a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82cc06d2c0 0x7f82cc0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d0a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82cc06d2c0 0x7f82cc0ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45670/0 (socket says 172.21.15.62:45670) 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d0a00640 1 -- 172.21.15.62:0/1819285933 learned_addr learned my addr 172.21.15.62:0/1819285933 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d1400640 1 --2- 172.21.15.62:0/1819285933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82c3e00640 1 --2- 172.21.15.62:0/1819285933 >> v2:172.21.15.121:3300/0 conn(0x7f82cc06d6a0 0x7f82cc0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d0a00640 1 -- 172.21.15.62:0/1819285933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 msgr2=0x7f82cc0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d0a00640 1 --2- 172.21.15.62:0/1819285933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d0a00640 1 -- 172.21.15.62:0/1819285933 >> v2:172.21.15.121:3300/0 conn(0x7f82cc06d6a0 msgr2=0x7f82cc0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d0a00640 1 --2- 172.21.15.62:0/1819285933 >> v2:172.21.15.121:3300/0 conn(0x7f82cc06d6a0 0x7f82cc0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d0a00640 1 -- 172.21.15.62:0/1819285933 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0b26c0 con 0x7f82cc06d2c0 2024-04-25T09:08:03.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.957+0000 7f82d1400640 1 --2- 172.21.15.62:0/1819285933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.958+0000 7f82d0a00640 1 --2- 172.21.15.62:0/1819285933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d2c0 0x7f82cc0ccde0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0722f0 tx=0x7f82bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3983244cbb7ca38d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.958+0000 7f82cbe00640 1 -- 172.21.15.62:0/1819285933 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc037070 con 0x7f82cc06d2c0 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82cbe00640 1 -- 172.21.15.62:0/1819285933 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc032440 con 0x7f82cc06d2c0 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82cbe00640 1 -- 172.21.15.62:0/1819285933 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc032810 con 0x7f82cc06d2c0 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82d1e00640 1 -- 172.21.15.62:0/1819285933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d2c0 msgr2=0x7f82cc0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82d1e00640 1 --2- 172.21.15.62:0/1819285933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d2c0 0x7f82cc0ccde0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0722f0 tx=0x7f82bc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82d1e00640 1 -- 172.21.15.62:0/1819285933 shutdown_connections 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82d1e00640 1 --2- 172.21.15.62:0/1819285933 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82d1e00640 1 --2- 172.21.15.62:0/1819285933 >> v2:172.21.15.121:3300/0 conn(0x7f82cc06d6a0 0x7f82cc0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82d1e00640 1 --2- 172.21.15.62:0/1819285933 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d2c0 0x7f82cc0ccde0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.959+0000 7f82d1e00640 1 -- 172.21.15.62:0/1819285933 >> 172.21.15.62:0/1819285933 conn(0x7f82cc066f20 msgr2=0x7f82cc0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.960+0000 7f82d1e00640 1 -- 172.21.15.62:0/1819285933 shutdown_connections 2024-04-25T09:08:03.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.960+0000 7f82d1e00640 1 -- 172.21.15.62:0/1819285933 wait complete. 2024-04-25T09:08:03.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.962+0000 7f82d1e00640 1 Processor -- start 2024-04-25T09:08:03.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.962+0000 7f82d1e00640 1 -- start start 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82d1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc06d2c0 0x7f82cc15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82d1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d6a0 0x7f82cc1e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82d1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc1e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82d1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc06db20 con 0x7f82cc06d6a0 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82d1e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0b2c80 con 0x7f82cc06d2c0 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82d1e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1448d0 con 0x7f82cc0b31a0 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d6a0 0x7f82cc1e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d6a0 0x7f82cc1e6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45676/0 (socket says 172.21.15.62:45676) 2024-04-25T09:08:03.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82c3e00640 1 -- 172.21.15.62:0/553402437 learned_addr learned my addr 172.21.15.62:0/553402437 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82d0a00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc06d2c0 0x7f82cc15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.964+0000 7f82d1400640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc1e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c3e00640 1 -- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 msgr2=0x7f82cc1e73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c3e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc1e73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c3e00640 1 -- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc06d2c0 msgr2=0x7f82cc15b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c3e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc06d2c0 0x7f82cc15b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c3e00640 1 -- 172.21.15.62:0/553402437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc15e740 con 0x7f82cc06d6a0 2024-04-25T09:08:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c3e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d6a0 0x7f82cc1e6ea0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f82b400d800 tx=0x7f82b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c9600640 1 -- 172.21.15.62:0/553402437 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4018030 con 0x7f82cc06d6a0 2024-04-25T09:08:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c9600640 1 -- 172.21.15.62:0/553402437 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b40096a0 con 0x7f82cc06d6a0 2024-04-25T09:08:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82c9600640 1 -- 172.21.15.62:0/553402437 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4009870 con 0x7f82cc06d6a0 2024-04-25T09:08:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.965+0000 7f82d1e00640 1 -- 172.21.15.62:0/553402437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc1e4890 con 0x7f82cc06d6a0 2024-04-25T09:08:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.966+0000 7f82d1e00640 1 -- 172.21.15.62:0/553402437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc1e4e50 con 0x7f82cc06d6a0 2024-04-25T09:08:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.968+0000 7f82c9600640 1 -- 172.21.15.62:0/553402437 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82b401b070 con 0x7f82cc06d6a0 2024-04-25T09:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.968+0000 7f82c9600640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.969+0000 7f82c9600640 1 -- 172.21.15.62:0/553402437 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f82b408a5e0 con 0x7f82cc06d6a0 2024-04-25T09:08:03.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.969+0000 7f82d0a00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.970+0000 7f82d63c0740 1 Processor -- start 2024-04-25T09:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.970+0000 7f82d63c0740 1 -- start start 2024-04-25T09:08:03.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.970+0000 7f82d0a00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f82bc034000 tx=0x7f82bc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc9098510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc90c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8560 0x555cc909d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d63c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc9001880 con 0x555cc90c8ca0 2024-04-25T09:08:03.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d63c0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc8faad00 con 0x555cc90c8560 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d63c0740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc8ffd920 con 0x555cc90e5b80 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc90c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc90c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45688/0 (socket says 172.21.15.62:45688) 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82c3e00640 1 -- 172.21.15.62:0/3763087656 learned_addr learned my addr 172.21.15.62:0/3763087656 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d0a00640 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc9098510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d1400640 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8560 0x555cc909d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82c3e00640 1 -- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 msgr2=0x555cc9098510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc9098510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82c3e00640 1 -- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8560 msgr2=0x555cc909d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8560 0x555cc909d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82c3e00640 1 -- 172.21.15.62:0/3763087656 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc8fb71f0 con 0x555cc90c8ca0 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d0a00640 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc9098510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.971+0000 7f82d1400640 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8560 0x555cc909d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.972+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc90c8020 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b40593f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5f4764b516209c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.972+0000 7f82c1200640 1 -- 172.21.15.62:0/3763087656 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b405f030 con 0x555cc90c8ca0 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.972+0000 7f82c1200640 1 -- 172.21.15.62:0/3763087656 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4061340 con 0x555cc90c8ca0 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.972+0000 7f82c1200640 1 -- 172.21.15.62:0/3763087656 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b40552a0 con 0x555cc90c8ca0 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 -- 172.21.15.62:0/3763087656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 msgr2=0x555cc90c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc90c8020 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b40593f0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 -- 172.21.15.62:0/3763087656 shutdown_connections 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc90c8020 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8560 0x555cc909d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3763087656 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc9098510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 -- 172.21.15.62:0/3763087656 >> 172.21.15.62:0/3763087656 conn(0x555cc909a780 msgr2=0x555cc90e3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 -- 172.21.15.62:0/3763087656 shutdown_connections 2024-04-25T09:08:03.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 -- 172.21.15.62:0/3763087656 wait complete. 2024-04-25T09:08:03.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.973+0000 7f82d63c0740 1 -- start start 2024-04-25T09:08:03.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.974+0000 7f82d63c0740 1 Processor -- start 2024-04-25T09:08:03.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc90c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e1b60 0x555cc90cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d63c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc909c930 con 0x555cc90e1b60 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d63c0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc8fa01c0 con 0x555cc90c8ca0 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d63c0740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc909bd20 con 0x555cc90e5b80 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d63c0740 10 client.0 _ll_register_callbacks cb 0x555cc8fbdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e1b60 0x555cc90cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e1b60 0x555cc90cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45690/0 (socket says 172.21.15.62:45690) 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d1400640 1 -- 172.21.15.62:0/3158508453 learned_addr learned my addr 172.21.15.62:0/3158508453 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d0a00640 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc90c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d1400640 1 -- 172.21.15.62:0/3158508453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 msgr2=0x555cc90c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d1400640 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc90c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d1400640 1 -- 172.21.15.62:0/3158508453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 msgr2=0x555cc90e49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.975+0000 7f82d1400640 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90e49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82d1400640 1 -- 172.21.15.62:0/3158508453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cc8de2690 con 0x555cc90e1b60 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82d1400640 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e1b60 0x555cc90cbb40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f82c400eae0 tx=0x7f82c400c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f43ec0a4b6d02da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82b9600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82b9600640 1 -- 172.21.15.62:0/3158508453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c401c060 con 0x555cc90e1b60 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82b9600640 1 -- 172.21.15.62:0/3158508453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f82c400f450 con 0x555cc90e1b60 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82d63c0740 1 -- 172.21.15.62:0/3158508453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82b4061340 con 0x555cc90e1b60 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82b9600640 1 -- 172.21.15.62:0/3158508453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c4022050 con 0x555cc90e1b60 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82d63c0740 10 client.6060 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.976+0000 7f82d63c0740 1 -- 172.21.15.62:0/3158508453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x555cc8e42e20 con 0x555cc90e1b60 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82b9600640 1 -- 172.21.15.62:0/3158508453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4061340 con 0x555cc90e1b60 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82b9600640 1 -- 172.21.15.62:0/3158508453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82c4025020 con 0x555cc90e1b60 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82d63c0740 10 client.6060 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82d63c0740 10 client.6060 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:03.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82d63c0740 20 client.6060 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555cc90c3240 0x555cc90f6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82d63c0740 4 client.6060 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82d63c0740 1 -- 172.21.15.62:0/3158508453 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cc900edc0 con 0x555cc90c3240 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.977+0000 7f82d0a00640 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555cc90c3240 0x555cc90f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.978+0000 7f82d0a00640 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555cc90c3240 0x555cc90f6790 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc5107a9e68825a server_cookie=d62e5d76233dd24b in_seq=0 out_seq=0 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.978+0000 7f82b9600640 10 client.6060 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.978+0000 7f82b9600640 1 -- 172.21.15.62:0/3158508453 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555cc900edc0 con 0x555cc90c3240 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.978+0000 7f82b9600640 10 client.6060 handle_command_reply: tid=0 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 client.6060 shutdown 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 -- 172.21.15.62:0/3158508453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e1b60 msgr2=0x555cc90cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e1b60 0x555cc90cbb40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f82c400eae0 tx=0x7f82c400c360 comp rx=0 tx=0).stop 2024-04-25T09:08:03.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 -- 172.21.15.62:0/3158508453 shutdown_connections 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555cc90c3240 0x555cc90f6790 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e1b60 0x555cc90cbb40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc90c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 --2- 172.21.15.62:0/3158508453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82b9600640 0 client.6060 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 -- 172.21.15.62:0/3158508453 >> 172.21.15.62:0/3158508453 conn(0x555cc90c8560 msgr2=0x555cc9099250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 -- 172.21.15.62:0/3158508453 shutdown_connections 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 1 -- 172.21.15.62:0/3158508453 wait complete. 2024-04-25T09:08:03.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.979+0000 7f82d63c0740 20 client.6060 trim_cache size 0 max 16384 2024-04-25T09:08:03.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.996+0000 7f82d63c0740 1 Processor -- start 2024-04-25T09:08:03.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.996+0000 7f82d63c0740 1 -- start start 2024-04-25T09:08:03.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc909dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90e1b60 0x555cc909e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc90e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d63c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1fff30 con 0x555cc90c8ca0 2024-04-25T09:08:03.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d63c0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1ff170 con 0x555cc90e1b60 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d63c0740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1e5c80 con 0x555cc90e5b80 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc909dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc909dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45702/0 (socket says 172.21.15.62:45702) 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.997+0000 7f82d0a00640 1 -- 172.21.15.62:0/1516824516 learned_addr learned my addr 172.21.15.62:0/1516824516 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82c3e00640 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90e1b60 0x555cc909e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82d1400640 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc90e2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82d0a00640 1 -- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 msgr2=0x555cc90e2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82d0a00640 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc90e2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82d0a00640 1 -- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90e1b60 msgr2=0x555cc909e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82d0a00640 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90e1b60 0x555cc909e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82d0a00640 1 -- 172.21.15.62:0/1516824516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc909b8c0 con 0x555cc90c8ca0 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82c3e00640 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90e1b60 0x555cc909e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82d0a00640 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc909dd30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009960 tx=0x7f82bc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662e394d30b0a850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82bbe00640 1 -- 172.21.15.62:0/1516824516 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc037040 con 0x555cc90c8ca0 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82bbe00640 1 -- 172.21.15.62:0/1516824516 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc051030 con 0x555cc90c8ca0 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.998+0000 7f82bbe00640 1 -- 172.21.15.62:0/1516824516 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc04d4d0 con 0x555cc90c8ca0 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 -- 172.21.15.62:0/1516824516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 msgr2=0x555cc909dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc909dd30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009960 tx=0x7f82bc02c000 comp rx=0 tx=0).stop 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 -- 172.21.15.62:0/1516824516 shutdown_connections 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e5b80 0x555cc90e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90e1b60 0x555cc909e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 --2- 172.21.15.62:0/1516824516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90c8ca0 0x555cc909dd30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 -- 172.21.15.62:0/1516824516 >> 172.21.15.62:0/1516824516 conn(0x555cc90c8560 msgr2=0x555cc9099250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 -- 172.21.15.62:0/1516824516 shutdown_connections 2024-04-25T09:08:03.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:03.999+0000 7f82d63c0740 1 -- 172.21.15.62:0/1516824516 wait complete. 2024-04-25T09:08:03.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.000+0000 7f82d63c0740 1 -- start start 2024-04-25T09:08:03.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.000+0000 7f82d63c0740 1 Processor -- start 2024-04-25T09:08:03.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.001+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.001+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e1b60 0x555cc90c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.001+0000 7f82d63c0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e5b80 0x555cc90c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.001+0000 7f82d63c0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1e6280 con 0x555cc90e5b80 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.001+0000 7f82d63c0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc9094390 con 0x555cc90c8ca0 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.001+0000 7f82d63c0740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc90c9ec0 con 0x555cc90e1b60 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.001+0000 7f82d63c0740 10 client.0 _ll_register_callbacks cb 0x7f82c40086a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.001+0000 7f82d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e5b80 0x555cc90c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e5b80 0x555cc90c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45708/0 (socket says 172.21.15.62:45708) 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d1400640 1 -- 172.21.15.62:0/2508112857 learned_addr learned my addr 172.21.15.62:0/2508112857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d0a00640 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82c3e00640 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e1b60 0x555cc90c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d1400640 1 -- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e1b60 msgr2=0x555cc90c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d1400640 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e1b60 0x555cc90c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d1400640 1 -- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 msgr2=0x555cc90c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d1400640 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d1400640 1 -- 172.21.15.62:0/2508112857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cc8cd5f80 con 0x555cc90e5b80 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d0a00640 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82d1400640 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e5b80 0x555cc90c4ac0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f82c400ff80 tx=0x7f82c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7638d957468cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.002+0000 7f82b9600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82b9600640 1 -- 172.21.15.62:0/2508112857 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400c860 con 0x555cc90e5b80 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82b9600640 1 -- 172.21.15.62:0/2508112857 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f82c40082f0 con 0x555cc90e5b80 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 1 -- 172.21.15.62:0/2508112857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82bc051030 con 0x555cc90e5b80 2024-04-25T09:08:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82b9600640 1 -- 172.21.15.62:0/2508112857 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c402f470 con 0x555cc90e5b80 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 10 client.6066 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 1 -- 172.21.15.62:0/2508112857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x555cc90f2700 con 0x555cc90e5b80 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82b9600640 1 -- 172.21.15.62:0/2508112857 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc051030 con 0x555cc90e5b80 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82b9600640 1 -- 172.21.15.62:0/2508112857 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f82c4025050 con 0x555cc90e5b80 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 10 client.6066 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 10 client.6066 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 20 client.6066 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555cc90f2c90 0x555cc914d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 4 client.6066 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.003+0000 7f82d63c0740 1 -- 172.21.15.62:0/2508112857 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x555cc9009110 con 0x555cc90f2c90 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.004+0000 7f82d0a00640 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555cc90f2c90 0x555cc914d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.004+0000 7f82d0a00640 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555cc90f2c90 0x555cc914d1e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b9c96ec7c90e6e server_cookie=48defee12ce42162 in_seq=0 out_seq=0 2024-04-25T09:08:03.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.004+0000 7f82b9600640 10 client.6066 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82b9600640 1 -- 172.21.15.62:0/2508112857 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555cc9009110 con 0x555cc90f2c90 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82b9600640 10 client.6066 handle_command_reply: tid=0 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 client.6066 shutdown 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 -- 172.21.15.62:0/2508112857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e5b80 msgr2=0x555cc90c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e5b80 0x555cc90c4ac0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f82c400ff80 tx=0x7f82c4004a40 comp rx=0 tx=0).stop 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 -- 172.21.15.62:0/2508112857 shutdown_connections 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555cc90f2c90 0x555cc914d1e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555cc90e5b80 0x555cc90c4ac0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555cc90e1b60 0x555cc90c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82b9600640 0 client.6066 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 --2- 172.21.15.62:0/2508112857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555cc90c8ca0 0x555cc90c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 -- 172.21.15.62:0/2508112857 >> 172.21.15.62:0/2508112857 conn(0x555cc90c8560 msgr2=0x555cc90e3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 -- 172.21.15.62:0/2508112857 shutdown_connections 2024-04-25T09:08:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 1 -- 172.21.15.62:0/2508112857 wait complete. 2024-04-25T09:08:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.005+0000 7f82d63c0740 20 client.6066 trim_cache size 0 max 16384 2024-04-25T09:08:03.998 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:03.998 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:08:03.998 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "13cc506a-bab1-44ae-b7ba-d64ae97954f8", 2024-04-25T09:08:03.998 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:08:03.998 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 -- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 msgr2=0x7f82a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f82bc034000 tx=0x7f82bc02e000 comp rx=0 tx=0).stop 2024-04-25T09:08:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 -- 172.21.15.62:0/553402437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d6a0 msgr2=0x7f82cc1e6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d6a0 0x7f82cc1e6ea0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f82b400d800 tx=0x7f82b400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:08:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 -- 172.21.15.62:0/553402437 shutdown_connections 2024-04-25T09:08:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82cc06d6a0 0x7f82cc1e6ea0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82cc0b31a0 0x7f82cc1e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 --2- 172.21.15.62:0/553402437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82cc06d2c0 0x7f82cc15b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 -- 172.21.15.62:0/553402437 >> 172.21.15.62:0/553402437 conn(0x7f82cc066f20 msgr2=0x7f82cc0aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 -- 172.21.15.62:0/553402437 shutdown_connections 2024-04-25T09:08:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.006+0000 7f82d1e00640 1 -- 172.21.15.62:0/553402437 wait complete. 2024-04-25T09:08:04.013 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '13cc506a-bab1-44ae-b7ba-d64ae97954f8', 'mode': 'asynchronous'}' 2024-04-25T09:08:04.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:04.013 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T09:08:04.068 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T09:08:04.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:04.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.294+0000 7fbf75000640 1 Processor -- start 2024-04-25T09:08:04.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.294+0000 7fbf75000640 1 -- start start 2024-04-25T09:08:04.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.296+0000 7fbf75000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbf7013d970 0x7fbf700b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.296+0000 7fbf75000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbf7006d900 0x7fbf7013e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.296+0000 7fbf75000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7015b220 con 0x7fbf700b1050 2024-04-25T09:08:04.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.296+0000 7fbf75000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7015b500 con 0x7fbf7013d970 2024-04-25T09:08:04.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.296+0000 7fbf75000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7015b7e0 con 0x7fbf7006d900 2024-04-25T09:08:04.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.296+0000 7fbf6f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbf7013d970 0x7fbf700b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.296+0000 7fbf6f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbf7013d970 0x7fbf700b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45724/0 (socket says 172.21.15.62:45724) 2024-04-25T09:08:04.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.296+0000 7fbf6f400640 1 -- 172.21.15.62:0/2453918890 learned_addr learned my addr 172.21.15.62:0/2453918890 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:04.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.297+0000 7fbf6f400640 1 -- 172.21.15.62:0/2453918890 >> v2:172.21.15.121:3300/0 conn(0x7fbf7006d900 msgr2=0x7fbf7013e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:04.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.297+0000 7fbf6f400640 1 --2- 172.21.15.62:0/2453918890 >> v2:172.21.15.121:3300/0 conn(0x7fbf7006d900 0x7fbf7013e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.297+0000 7fbf6f400640 1 -- 172.21.15.62:0/2453918890 >> v1:172.21.15.121:6789/0 conn(0x7fbf700b1050 legacy=0x7fbf7006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:04.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.297+0000 7fbf6f400640 1 -- 172.21.15.62:0/2453918890 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7015bac0 con 0x7fbf7013d970 2024-04-25T09:08:04.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.298+0000 7fbf6f400640 1 --2- 172.21.15.62:0/2453918890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7013d970 0x7fbf700b0b10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbf700b2590 tx=0x7fbf6002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9c2f0b053a044b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:04.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.298+0000 7fbf6e000640 1 -- 172.21.15.62:0/2453918890 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60037070 con 0x7fbf7013d970 2024-04-25T09:08:04.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.298+0000 7fbf6e000640 1 -- 172.21.15.62:0/2453918890 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60032440 con 0x7fbf7013d970 2024-04-25T09:08:04.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.298+0000 7fbf6e000640 1 -- 172.21.15.62:0/2453918890 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60032810 con 0x7fbf7013d970 2024-04-25T09:08:04.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.298+0000 7fbf75000640 1 -- 172.21.15.62:0/2453918890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7013d970 msgr2=0x7fbf700b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.298+0000 7fbf75000640 1 --2- 172.21.15.62:0/2453918890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7013d970 0x7fbf700b0b10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbf700b2590 tx=0x7fbf6002a230 comp rx=0 tx=0).stop 2024-04-25T09:08:04.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.299+0000 7fbf75000640 1 -- 172.21.15.62:0/2453918890 shutdown_connections 2024-04-25T09:08:04.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.299+0000 7fbf75000640 1 --2- 172.21.15.62:0/2453918890 >> v2:172.21.15.121:3300/0 conn(0x7fbf7006d900 0x7fbf7013e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.299+0000 7fbf75000640 1 --2- 172.21.15.62:0/2453918890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf7013d970 0x7fbf700b0b10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.299+0000 7fbf75000640 1 -- 172.21.15.62:0/2453918890 >> 172.21.15.62:0/2453918890 conn(0x7fbf700b14d0 msgr2=0x7fbf700b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:04.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.299+0000 7fbf75000640 1 -- 172.21.15.62:0/2453918890 shutdown_connections 2024-04-25T09:08:04.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.299+0000 7fbf75000640 1 -- 172.21.15.62:0/2453918890 wait complete. 2024-04-25T09:08:04.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.300+0000 7fbf75000640 1 Processor -- start 2024-04-25T09:08:04.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.301+0000 7fbf75000640 1 -- start start 2024-04-25T09:08:04.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.301+0000 7fbf75000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7006d900 0x7fbf701f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf75000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf700b1050 0x7fbf7015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf75000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7013d970 0x7fbf7013bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf75000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7013e790 con 0x7fbf700b1050 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf75000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7015af00 con 0x7fbf7006d900 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf75000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7013e910 con 0x7fbf7013d970 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf700b1050 0x7fbf7015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf700b1050 0x7fbf7015d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45726/0 (socket says 172.21.15.62:45726) 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6ea00640 1 -- 172.21.15.62:0/16573545 learned_addr learned my addr 172.21.15.62:0/16573545 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6f400640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7006d900 0x7fbf701f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6fe00640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7013d970 0x7fbf7013bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6ea00640 1 -- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7013d970 msgr2=0x7fbf7013bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6ea00640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7013d970 0x7fbf7013bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6ea00640 1 -- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7006d900 msgr2=0x7fbf701f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6ea00640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7006d900 0x7fbf701f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6ea00640 1 -- 172.21.15.62:0/16573545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7015bee0 con 0x7fbf700b1050 2024-04-25T09:08:04.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.302+0000 7fbf6fe00640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7013d970 0x7fbf7013bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:04.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.303+0000 7fbf6ea00640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf700b1050 0x7fbf7015d9b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c440 tx=0x7fbf5800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:04.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.303+0000 7fbf67400640 1 -- 172.21.15.62:0/16573545 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58017070 con 0x7fbf700b1050 2024-04-25T09:08:04.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.303+0000 7fbf67400640 1 -- 172.21.15.62:0/16573545 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf580035f0 con 0x7fbf700b1050 2024-04-25T09:08:04.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.303+0000 7fbf67400640 1 -- 172.21.15.62:0/16573545 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf580039e0 con 0x7fbf700b1050 2024-04-25T09:08:04.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.303+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7015def0 con 0x7fbf700b1050 2024-04-25T09:08:04.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.303+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf701f43e0 con 0x7fbf700b1050 2024-04-25T09:08:04.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.304+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7004ef20 con 0x7fbf700b1050 2024-04-25T09:08:04.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.305+0000 7fbf67400640 1 -- 172.21.15.62:0/16573545 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf58003b80 con 0x7fbf700b1050 2024-04-25T09:08:04.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.306+0000 7fbf67400640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:04.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.306+0000 7fbf67400640 1 -- 172.21.15.62:0/16573545 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbf580895d0 con 0x7fbf700b1050 2024-04-25T09:08:04.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.307+0000 7fbf6f400640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:04.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.308+0000 7fbf6f400640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbf60034000 tx=0x7fbf6002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:04.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.311+0000 7fbf67400640 1 -- 172.21.15.62:0/16573545 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf58058320 con 0x7fbf700b1050 2024-04-25T09:08:04.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.547+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf7006d6e0 con 0x7fbf700b1050 2024-04-25T09:08:04.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.549+0000 7fbf67400640 1 -- 172.21.15.62:0/16573545 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fbf580010c0 con 0x7fbf700b1050 2024-04-25T09:08:04.537 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:04.537 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:58.031468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:04.537 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf4c06b4d0 msgr2=0x7fbf4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbf60034000 tx=0x7fbf6002e000 comp rx=0 tx=0).stop 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf700b1050 msgr2=0x7fbf7015d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf700b1050 0x7fbf7015d9b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c440 tx=0x7fbf5800c900 comp rx=0 tx=0).stop 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 shutdown_connections 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbf700b1050 0x7fbf7015d9b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbf7013d970 0x7fbf7013bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 --2- 172.21.15.62:0/16573545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf7006d900 0x7fbf701f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 >> 172.21.15.62:0/16573545 conn(0x7fbf700b14d0 msgr2=0x7fbf70070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.551+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 shutdown_connections 2024-04-25T09:08:04.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:04.552+0000 7fbf75000640 1 -- 172.21.15.62:0/16573545 wait complete. 2024-04-25T09:08:04.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T09:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:08:04.787 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:04.804 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:08:04.804 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:04.804 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:04.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:04.859 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T09:08:04.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:08:04.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:08:04.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339987 s, 30.8 MB/s 2024-04-25T09:08:04.952 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T09:08:04.953 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.174+0000 7f165d000640 1 Processor -- start 2024-04-25T09:08:05.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.174+0000 7f165d000640 1 -- start start 2024-04-25T09:08:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.176+0000 7f165d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f165806d530 0x7f16580ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.176+0000 7f165d000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16580ceac0 0x7f16580cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.176+0000 7f165d000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f165806e120 con 0x7f165815dd50 2024-04-25T09:08:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.176+0000 7f165d000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1658162980 con 0x7f16580ceac0 2024-04-25T09:08:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.176+0000 7f165d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165806db60 con 0x7f165806d530 2024-04-25T09:08:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.177+0000 7f1656800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16580ceac0 0x7f16580cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.177+0000 7f1656800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f16580ceac0 0x7f16580cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60164/0 (socket says 172.21.15.62:60164) 2024-04-25T09:08:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.177+0000 7f1656800640 1 -- 172.21.15.62:0/3148323010 learned_addr learned my addr 172.21.15.62:0/3148323010 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.177+0000 7f1655400640 1 --2- 172.21.15.62:0/3148323010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f165806d530 0x7f16580ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.177+0000 7f1656800640 1 -- 172.21.15.62:0/3148323010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f165806d530 msgr2=0x7f16580ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.177+0000 7f1656800640 1 --2- 172.21.15.62:0/3148323010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f165806d530 0x7f16580ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.177+0000 7f1656800640 1 -- 172.21.15.62:0/3148323010 >> v1:172.21.15.121:6789/0 conn(0x7f165815dd50 legacy=0x7f1658162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:05.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.177+0000 7f1656800640 1 -- 172.21.15.62:0/3148323010 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165806de40 con 0x7f16580ceac0 2024-04-25T09:08:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.178+0000 7f1656800640 1 --2- 172.21.15.62:0/3148323010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16580ceac0 0x7f16580cef50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1650009e50 tx=0x7f165002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2017596ab981fc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.179+0000 7f1654a00640 1 -- 172.21.15.62:0/3148323010 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650036070 con 0x7f16580ceac0 2024-04-25T09:08:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.179+0000 7f1654a00640 1 -- 172.21.15.62:0/3148323010 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1650031600 con 0x7f16580ceac0 2024-04-25T09:08:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.179+0000 7f1654a00640 1 -- 172.21.15.62:0/3148323010 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500319d0 con 0x7f16580ceac0 2024-04-25T09:08:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.179+0000 7f165d000640 1 -- 172.21.15.62:0/3148323010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16580ceac0 msgr2=0x7f16580cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.179+0000 7f165d000640 1 --2- 172.21.15.62:0/3148323010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16580ceac0 0x7f16580cef50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1650009e50 tx=0x7f165002a6d0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.180+0000 7f165d000640 1 -- 172.21.15.62:0/3148323010 shutdown_connections 2024-04-25T09:08:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.180+0000 7f165d000640 1 --2- 172.21.15.62:0/3148323010 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16580ceac0 0x7f16580cef50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.180+0000 7f165d000640 1 --2- 172.21.15.62:0/3148323010 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f165806d530 0x7f16580ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.180+0000 7f165d000640 1 -- 172.21.15.62:0/3148323010 >> 172.21.15.62:0/3148323010 conn(0x7f16580668f0 msgr2=0x7f1658066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.180+0000 7f165d000640 1 -- 172.21.15.62:0/3148323010 shutdown_connections 2024-04-25T09:08:05.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.180+0000 7f165d000640 1 -- 172.21.15.62:0/3148323010 wait complete. 2024-04-25T09:08:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.182+0000 7f165d000640 1 Processor -- start 2024-04-25T09:08:05.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.182+0000 7f165d000640 1 -- start start 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.183+0000 7f165d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f165806d530 0x7f16581585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.183+0000 7f165d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16580ceac0 0x7f1658148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f165d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165815dd50 0x7f1658149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f165d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16580cf490 con 0x7f165815dd50 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f165d000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1658161d50 con 0x7f165806d530 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f165d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165815e130 con 0x7f16580ceac0 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1656800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165815dd50 0x7f1658149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1656800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165815dd50 0x7f1658149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60170/0 (socket says 172.21.15.62:60170) 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1656800640 1 -- 172.21.15.62:0/2980269792 learned_addr learned my addr 172.21.15.62:0/2980269792 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1655e00640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f165806d530 0x7f16581585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1655400640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16580ceac0 0x7f1658148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1656800640 1 -- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16580ceac0 msgr2=0x7f1658148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1656800640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16580ceac0 0x7f1658148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1656800640 1 -- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f165806d530 msgr2=0x7f16581585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1656800640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f165806d530 0x7f16581585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.184+0000 7f1656800640 1 -- 172.21.15.62:0/2980269792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16580af100 con 0x7f165815dd50 2024-04-25T09:08:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.185+0000 7f1656800640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165815dd50 0x7f1658149400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1650009f70 tx=0x7f1650004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.185+0000 7f164e000640 1 -- 172.21.15.62:0/2980269792 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650036050 con 0x7f165815dd50 2024-04-25T09:08:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.185+0000 7f164e000640 1 -- 172.21.15.62:0/2980269792 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1650042070 con 0x7f165815dd50 2024-04-25T09:08:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.185+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16580af3e0 con 0x7f165815dd50 2024-04-25T09:08:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.185+0000 7f164e000640 1 -- 172.21.15.62:0/2980269792 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650009510 con 0x7f165815dd50 2024-04-25T09:08:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.185+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16580af9a0 con 0x7f165815dd50 2024-04-25T09:08:05.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.186+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1658072c70 con 0x7f165815dd50 2024-04-25T09:08:05.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.187+0000 7f164e000640 1 -- 172.21.15.62:0/2980269792 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1650007c30 con 0x7f165815dd50 2024-04-25T09:08:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.189+0000 7f164e000640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f163406b4d0 0x7f163406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.189+0000 7f164e000640 1 -- 172.21.15.62:0/2980269792 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f16500ab260 con 0x7f165815dd50 2024-04-25T09:08:05.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.190+0000 7f1655e00640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f163406b4d0 0x7f163406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:05.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.191+0000 7f1655e00640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f163406b4d0 0x7f163406d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f16580b4340 tx=0x7f1648009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:05.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.193+0000 7f164e000640 1 -- 172.21.15.62:0/2980269792 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1650079fc0 con 0x7f165815dd50 2024-04-25T09:08:05.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.437+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16580ade60 con 0x7f165815dd50 2024-04-25T09:08:05.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.438+0000 7f164e000640 1 -- 172.21.15.62:0/2980269792 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f16500798c0 con 0x7f165815dd50 2024-04-25T09:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:05.427 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:07:58.031468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:05.427 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 27 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f163406b4d0 msgr2=0x7f163406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f163406b4d0 0x7f163406d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f16580b4340 tx=0x7f1648009040 comp rx=0 tx=0).stop 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165815dd50 msgr2=0x7f1658149400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165815dd50 0x7f1658149400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1650009f70 tx=0x7f1650004a40 comp rx=0 tx=0).stop 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 shutdown_connections 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f163406b4d0 0x7f163406d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f165815dd50 0x7f1658149400 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16580ceac0 0x7f1658148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 --2- 172.21.15.62:0/2980269792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f165806d530 0x7f16581585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 >> 172.21.15.62:0/2980269792 conn(0x7f16580668f0 msgr2=0x7f165815c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 shutdown_connections 2024-04-25T09:08:05.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05.441+0000 7f165d000640 1 -- 172.21.15.62:0/2980269792 wait complete. 2024-04-25T09:08:05.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T09:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:08:05.671 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:05.687 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:08:05.688 DEBUG:teuthology.orchestra.run.smithi062:> stdin-killer -- python3 -c ' 2024-04-25T09:08:05.688 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-04-25T09:08:05.688 DEBUG:teuthology.orchestra.run.smithi062:> import stat 2024-04-25T09:08:05.688 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:08:05.688 DEBUG:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T09:08:05.688 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-04-25T09:08:05.838 INFO:teuthology.orchestra.run.smithi062.stdout:1099511630190 2024-04-25T09:08:05.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:08:05.854 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:05.854 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:08:06.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.130+0000 7ff06a000640 1 Processor -- start 2024-04-25T09:08:06.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.130+0000 7ff06a000640 1 -- start start 2024-04-25T09:08:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.132+0000 7ff06a000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 0x7ff0640d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.132+0000 7ff06a000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640ce780 con 0x7ff0640d2fb0 2024-04-25T09:08:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.132+0000 7ff06a000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640d42f0 con 0x7ff06415dd50 2024-04-25T09:08:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.132+0000 7ff06a000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640d3440 con 0x7ff0640d3960 2024-04-25T09:08:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.132+0000 7ff063e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff0640d2fb0 0x7ff0640ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53232/0 (socket says 172.21.15.62:53232) 2024-04-25T09:08:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.132+0000 7ff063e00640 1 -- 172.21.15.62:0/703406891 learned_addr learned my addr 172.21.15.62:0/703406891 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:06.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff069600640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 0x7ff0640d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648382098 0 0) 0x7ff0640ce780 con 0x7ff0640d2fb0 2024-04-25T09:08:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff04c003520 con 0x7ff0640d2fb0 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1993137197 0 0) 0x7ff04c003520 con 0x7ff0640d2fb0 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 msgr2=0x7ff0640d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 0x7ff0640d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 >> v1:172.21.15.121:6789/0 conn(0x7ff06415dd50 legacy=0x7ff0640d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640ce1c0 con 0x7ff0640d2fb0 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ff050001670 con 0x7ff0640d2fb0 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 >> v1:172.21.15.62:6789/0 conn(0x7ff0640d2fb0 legacy=0x7ff0640ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff04c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 0x7ff04c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c004d40 0x7ff04c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640d3440 con 0x7ff04c0039e0 2024-04-25T09:08:06.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c00b9f0 con 0x7ff04c004d40 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.133+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c00bbf0 con 0x7ff0640d3960 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff068c00640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff04c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff063e00640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 0x7ff04c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff069600640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c004d40 0x7ff04c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff068c00640 1 -- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 msgr2=0x7ff04c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff068c00640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 0x7ff04c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff068c00640 1 -- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c004d40 msgr2=0x7ff04c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff068c00640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff04c004d40 0x7ff04c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff068c00640 1 -- 172.21.15.62:0/703406891 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c003520 con 0x7ff04c0039e0 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.134+0000 7ff063e00640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640d3960 0x7ff04c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.135+0000 7ff068c00640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff04c003dc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff058001910 tx=0x7ff04c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828e085eeb591846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.135+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058037070 con 0x7ff04c0039e0 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05802aba0 con 0x7ff04c0039e0 2024-04-25T09:08:06.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff063400640 1 -- 172.21.15.62:0/703406891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580014e0 con 0x7ff04c0039e0 2024-04-25T09:08:06.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff06a000640 1 -- 172.21.15.62:0/703406891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 msgr2=0x7ff04c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff06a000640 1 --2- 172.21.15.62:0/703406891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff04c003dc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff058001910 tx=0x7ff04c005830 comp rx=0 tx=0).stop 2024-04-25T09:08:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff069600640 1 -- 172.21.15.62:0/703406891 reap_dead start 2024-04-25T09:08:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff06a000640 1 -- 172.21.15.62:0/703406891 shutdown_connections 2024-04-25T09:08:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff06a000640 1 -- 172.21.15.62:0/703406891 >> 172.21.15.62:0/703406891 conn(0x7ff06406b000 msgr2=0x7ff06406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff06a000640 1 -- 172.21.15.62:0/703406891 shutdown_connections 2024-04-25T09:08:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.136+0000 7ff06a000640 1 -- 172.21.15.62:0/703406891 wait complete. 2024-04-25T09:08:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.138+0000 7ff06a000640 1 Processor -- start 2024-04-25T09:08:06.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.138+0000 7ff06a000640 1 -- start start 2024-04-25T09:08:06.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff06a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff0640b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff06a000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640b4ea0 0x7ff0640b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff06a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff064152df0 0x7ff064153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff06a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640d3f30 con 0x7ff04c0039e0 2024-04-25T09:08:06.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff06a000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640d3db0 con 0x7ff064152df0 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff06a000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640d40b0 con 0x7ff0640b4ea0 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff068c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff0640b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff068c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff0640b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60188/0 (socket says 172.21.15.62:60188) 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.139+0000 7ff068c00640 1 -- 172.21.15.62:0/1365824338 learned_addr learned my addr 172.21.15.62:0/1365824338 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.140+0000 7ff068c00640 1 -- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640b4ea0 msgr2=0x7ff0640b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.140+0000 7ff068c00640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640b4ea0 0x7ff0640b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.140+0000 7ff068c00640 1 -- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff064152df0 msgr2=0x7ff064153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.140+0000 7ff068c00640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff064152df0 0x7ff064153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.140+0000 7ff068c00640 1 -- 172.21.15.62:0/1365824338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06415e210 con 0x7ff04c0039e0 2024-04-25T09:08:06.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.140+0000 7ff068c00640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff0640b4490 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff058001910 tx=0x7ff0580012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.140+0000 7ff060c00640 1 -- 172.21.15.62:0/1365824338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058037040 con 0x7ff04c0039e0 2024-04-25T09:08:06.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.141+0000 7ff060c00640 1 -- 172.21.15.62:0/1365824338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0580052e0 con 0x7ff04c0039e0 2024-04-25T09:08:06.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.141+0000 7ff060c00640 1 -- 172.21.15.62:0/1365824338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058007a70 con 0x7ff04c0039e0 2024-04-25T09:08:06.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.141+0000 7ff06a000640 1 -- 172.21.15.62:0/1365824338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06415e4f0 con 0x7ff04c0039e0 2024-04-25T09:08:06.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.141+0000 7ff06a000640 1 -- 172.21.15.62:0/1365824338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06415eab0 con 0x7ff04c0039e0 2024-04-25T09:08:06.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.143+0000 7ff060c00640 1 -- 172.21.15.62:0/1365824338 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff05803a070 con 0x7ff04c0039e0 2024-04-25T09:08:06.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.144+0000 7ff060c00640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff04406b500 0x7ff04406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.144+0000 7ff060c00640 1 -- 172.21.15.62:0/1365824338 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff0580aa190 con 0x7ff04c0039e0 2024-04-25T09:08:06.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.145+0000 7ff063e00640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff04406b500 0x7ff04406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.145+0000 7ff06e38f740 1 Processor -- start 2024-04-25T09:08:06.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.145+0000 7ff06e38f740 1 -- start start 2024-04-25T09:08:06.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff063e00640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff04406b500 0x7ff04406d9c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff050001eb0 tx=0x7ff050009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c6114010 0x5584c61143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c6131e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c60e9cf0 0x5584c612ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff06e38f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c604cd00 con 0x5584c60e9cf0 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff06e38f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c5ff5f50 con 0x5584c61318b0 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff06e38f740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584c6048da0 con 0x5584c6114010 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff069600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c60e9cf0 0x5584c612ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff069600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c60e9cf0 0x5584c612ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60204/0 (socket says 172.21.15.62:60204) 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.146+0000 7ff069600640 1 -- 172.21.15.62:0/45479952 learned_addr learned my addr 172.21.15.62:0/45479952 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.147+0000 7ff068c00640 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c6114010 0x5584c61143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.147+0000 7ff063e00640 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c6131e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.147+0000 7ff069600640 1 -- 172.21.15.62:0/45479952 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c6114010 msgr2=0x5584c61143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.147+0000 7ff069600640 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c6114010 0x5584c61143f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.147+0000 7ff069600640 1 -- 172.21.15.62:0/45479952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 msgr2=0x5584c6131e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.147+0000 7ff069600640 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c6131e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.147+0000 7ff069600640 1 -- 172.21.15.62:0/45479952 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584c6019130 con 0x5584c60e9cf0 2024-04-25T09:08:06.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.147+0000 7ff069600640 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c60e9cf0 0x5584c612ff70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff064161620 tx=0x7ff05c0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f556f77c6f945e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff055200640 1 -- 172.21.15.62:0/45479952 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c01a070 con 0x5584c60e9cf0 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff055200640 1 -- 172.21.15.62:0/45479952 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c003b90 con 0x5584c60e9cf0 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff055200640 1 -- 172.21.15.62:0/45479952 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c004030 con 0x5584c60e9cf0 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff06e38f740 1 -- 172.21.15.62:0/45479952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c60e9cf0 msgr2=0x5584c612ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff06e38f740 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c60e9cf0 0x5584c612ff70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff064161620 tx=0x7ff05c0075c0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff06e38f740 1 -- 172.21.15.62:0/45479952 shutdown_connections 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff06e38f740 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c60e9cf0 0x5584c612ff70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff06e38f740 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c6131e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff06e38f740 1 --2- 172.21.15.62:0/45479952 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c6114010 0x5584c61143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.148+0000 7ff06e38f740 1 -- 172.21.15.62:0/45479952 >> 172.21.15.62:0/45479952 conn(0x5584c60e6620 msgr2=0x5584c612ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.149+0000 7ff06e38f740 1 -- 172.21.15.62:0/45479952 shutdown_connections 2024-04-25T09:08:06.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.149+0000 7ff06e38f740 1 -- 172.21.15.62:0/45479952 wait complete. 2024-04-25T09:08:06.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.149+0000 7ff06e38f740 1 -- start start 2024-04-25T09:08:06.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.150+0000 7ff06e38f740 1 Processor -- start 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c612dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c61039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 0x5584c6101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff06e38f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c60eaa90 con 0x5584c6114010 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff06e38f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c5feb530 con 0x5584c61318b0 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff06e38f740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584c60e4550 con 0x5584c61169b0 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff06e38f740 10 client.0 _ll_register_callbacks cb 0x5584c6044430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff068c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c612dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff068c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c612dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60214/0 (socket says 172.21.15.62:60214) 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff068c00640 1 -- 172.21.15.62:0/61781666 learned_addr learned my addr 172.21.15.62:0/61781666 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff063e00640 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c61039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff069600640 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 0x5584c6101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff068c00640 1 -- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 msgr2=0x5584c6101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff068c00640 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 0x5584c6101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff068c00640 1 -- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 msgr2=0x5584c61039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff068c00640 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c61039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff068c00640 1 -- 172.21.15.62:0/61781666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584c6013dd0 con 0x5584c6114010 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.151+0000 7ff069600640 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 0x5584c6101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.152+0000 7ff068c00640 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c612dca0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff058007d20 tx=0x7ff058078290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cc956edd9d123a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.152+0000 7ff049600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.152+0000 7ff049600640 1 -- 172.21.15.62:0/61781666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05807c9c0 con 0x5584c6114010 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.152+0000 7ff049600640 1 -- 172.21.15.62:0/61781666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff058090180 con 0x5584c6114010 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.152+0000 7ff06e38f740 1 -- 172.21.15.62:0/61781666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff05c003b90 con 0x5584c6114010 2024-04-25T09:08:06.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.152+0000 7ff049600640 1 -- 172.21.15.62:0/61781666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff058080f80 con 0x5584c6114010 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.152+0000 7ff06e38f740 10 client.6090 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.152+0000 7ff06e38f740 1 -- 172.21.15.62:0/61781666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5584c5d613f0 con 0x5584c6114010 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff049600640 1 -- 172.21.15.62:0/61781666 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c003b90 con 0x5584c6114010 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff049600640 1 -- 172.21.15.62:0/61781666 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff05804c060 con 0x5584c6114010 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff06e38f740 10 client.6090 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff06e38f740 10 client.6090 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff06e38f740 20 client.6090 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff06e38f740 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5584c60e32e0 0x5584c60e3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff06e38f740 4 client.6090 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff06e38f740 1 -- 172.21.15.62:0/61781666 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584c605a240 con 0x5584c60e32e0 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.153+0000 7ff063e00640 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5584c60e32e0 0x5584c60e3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.154+0000 7ff063e00640 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5584c60e32e0 0x5584c60e3710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53814d774e30375 server_cookie=e39b62c64e799ad6 in_seq=0 out_seq=0 2024-04-25T09:08:06.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.154+0000 7ff049600640 10 client.6090 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.154+0000 7ff049600640 1 -- 172.21.15.62:0/61781666 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584c605a240 con 0x5584c60e32e0 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.154+0000 7ff049600640 10 client.6090 handle_command_reply: tid=0 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.154+0000 7ff06e38f740 1 client.6090 shutdown 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 -- 172.21.15.62:0/61781666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 msgr2=0x5584c612dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c612dca0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff058007d20 tx=0x7ff058078290 comp rx=0 tx=0).stop 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 -- 172.21.15.62:0/61781666 shutdown_connections 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5584c60e32e0 0x5584c60e3710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 0x5584c6101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c61039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 --2- 172.21.15.62:0/61781666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c612dca0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff049600640 0 client.6090 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 -- 172.21.15.62:0/61781666 >> 172.21.15.62:0/61781666 conn(0x5584c60e9cf0 msgr2=0x5584c61323a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 -- 172.21.15.62:0/61781666 shutdown_connections 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 1 -- 172.21.15.62:0/61781666 wait complete. 2024-04-25T09:08:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.155+0000 7ff06e38f740 20 client.6090 trim_cache size 0 max 16384 2024-04-25T09:08:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.168+0000 7ff06e38f740 1 Processor -- start 2024-04-25T09:08:06.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.168+0000 7ff06e38f740 1 -- start start 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.169+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.169+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c61169b0 0x5584c60dfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.169+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61318b0 0x5584c612f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.169+0000 7ff06e38f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c6149d60 con 0x5584c61169b0 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.169+0000 7ff06e38f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff064200490 con 0x5584c6114010 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.169+0000 7ff06e38f740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641ff680 con 0x5584c61318b0 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c61169b0 0x5584c60dfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c61169b0 0x5584c60dfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60216/0 (socket says 172.21.15.62:60216) 2024-04-25T09:08:06.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 -- 172.21.15.62:0/126272082 learned_addr learned my addr 172.21.15.62:0/126272082 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff068c00640 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff069600640 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61318b0 0x5584c612f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 -- 172.21.15.62:0/126272082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61318b0 msgr2=0x5584c612f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61318b0 0x5584c612f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 -- 172.21.15.62:0/126272082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c6114010 msgr2=0x5584c60eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 -- 172.21.15.62:0/126272082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584c5e2b7d0 con 0x5584c61169b0 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.170+0000 7ff063e00640 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c61169b0 0x5584c60dfdb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff050000fd0 tx=0x7ff050011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be33a8496a2c0c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff04be00640 1 -- 172.21.15.62:0/126272082 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0500247a0 con 0x5584c61169b0 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff04be00640 1 -- 172.21.15.62:0/126272082 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff050005cc0 con 0x5584c61169b0 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff04be00640 1 -- 172.21.15.62:0/126272082 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05000d9a0 con 0x5584c61169b0 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff06e38f740 1 -- 172.21.15.62:0/126272082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c61169b0 msgr2=0x5584c60dfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff06e38f740 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c61169b0 0x5584c60dfdb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff050000fd0 tx=0x7ff050011000 comp rx=0 tx=0).stop 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff06e38f740 1 -- 172.21.15.62:0/126272082 shutdown_connections 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff06e38f740 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c61169b0 0x5584c60dfdb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff06e38f740 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61318b0 0x5584c612f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff06e38f740 1 --2- 172.21.15.62:0/126272082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.171+0000 7ff06e38f740 1 -- 172.21.15.62:0/126272082 >> 172.21.15.62:0/126272082 conn(0x5584c60e9cf0 msgr2=0x5584c60ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.172+0000 7ff06e38f740 1 -- 172.21.15.62:0/126272082 shutdown_connections 2024-04-25T09:08:06.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.172+0000 7ff06e38f740 1 -- 172.21.15.62:0/126272082 wait complete. 2024-04-25T09:08:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.172+0000 7ff06e38f740 1 -- start start 2024-04-25T09:08:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.172+0000 7ff06e38f740 1 Processor -- start 2024-04-25T09:08:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.173+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.173+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 0x5584c6127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.173+0000 7ff06e38f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c6132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.173+0000 7ff06e38f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641607c0 con 0x5584c6114010 2024-04-25T09:08:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff06e38f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641601c0 con 0x5584c61318b0 2024-04-25T09:08:06.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff06e38f740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584c6101490 con 0x5584c61169b0 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff06e38f740 10 client.0 _ll_register_callbacks cb 0x5584c60e4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60222/0 (socket says 172.21.15.62:60222) 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 -- 172.21.15.62:0/1487426657 learned_addr learned my addr 172.21.15.62:0/1487426657 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 -- 172.21.15.62:0/1487426657 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 msgr2=0x5584c6127d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 0x5584c6127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 -- 172.21.15.62:0/1487426657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 msgr2=0x5584c6132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c6132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 -- 172.21.15.62:0/1487426657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584c61251c0 con 0x5584c6114010 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff068c00640 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff058074cd0 tx=0x7ff058046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950fc00525d274e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.174+0000 7ff049600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff049600640 1 -- 172.21.15.62:0/1487426657 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05807c9c0 con 0x5584c6114010 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff049600640 1 -- 172.21.15.62:0/1487426657 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff058080d00 con 0x5584c6114010 2024-04-25T09:08:06.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 1 -- 172.21.15.62:0/1487426657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff050005cc0 con 0x5584c6114010 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff049600640 1 -- 172.21.15.62:0/1487426657 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff058097340 con 0x5584c6114010 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 10 client.6096 fetch_fsmap learned FSMap version 27 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 1 -- 172.21.15.62:0/1487426657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5584c61251c0 con 0x5584c6114010 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff049600640 1 -- 172.21.15.62:0/1487426657 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff050005cc0 con 0x5584c6114010 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff049600640 1 -- 172.21.15.62:0/1487426657 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff058078ad0 con 0x5584c6114010 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 10 client.6096 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 10 client.6096 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 20 client.6096 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5584c61024d0 0x5584c6102960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 4 client.6096 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.175+0000 7ff06e38f740 1 -- 172.21.15.62:0/1487426657 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584c6101630 con 0x5584c61024d0 2024-04-25T09:08:06.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.176+0000 7ff063e00640 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5584c61024d0 0x5584c6102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.176+0000 7ff063e00640 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5584c61024d0 0x5584c6102960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736858e741a9e80b server_cookie=b08b1e16389eb6a3 in_seq=0 out_seq=0 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.176+0000 7ff049600640 10 client.6096 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.176+0000 7ff049600640 1 -- 172.21.15.62:0/1487426657 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584c6101630 con 0x5584c61024d0 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.176+0000 7ff049600640 10 client.6096 handle_command_reply: tid=0 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.176+0000 7ff06e38f740 1 client.6096 shutdown 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 -- 172.21.15.62:0/1487426657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 msgr2=0x5584c60eb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff058074cd0 tx=0x7ff058046040 comp rx=0 tx=0).stop 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 -- 172.21.15.62:0/1487426657 shutdown_connections 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5584c61024d0 0x5584c6102960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584c61318b0 0x5584c6132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5584c61169b0 0x5584c6127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 --2- 172.21.15.62:0/1487426657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5584c6114010 0x5584c60eb0c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff049600640 0 client.6096 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 -- 172.21.15.62:0/1487426657 >> 172.21.15.62:0/1487426657 conn(0x5584c60e9cf0 msgr2=0x5584c612f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 -- 172.21.15.62:0/1487426657 shutdown_connections 2024-04-25T09:08:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 1 -- 172.21.15.62:0/1487426657 wait complete. 2024-04-25T09:08:06.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.177+0000 7ff06e38f740 20 client.6096 trim_cache size 0 max 16384 2024-04-25T09:08:06.169 INFO:teuthology.orchestra.run.smithi062.stdout:[] 2024-04-25T09:08:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 -- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff04406b500 msgr2=0x7ff04406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff04406b500 0x7ff04406d9c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff050001eb0 tx=0x7ff050009040 comp rx=0 tx=0).stop 2024-04-25T09:08:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 -- 172.21.15.62:0/1365824338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 msgr2=0x7ff0640b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff0640b4490 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff058001910 tx=0x7ff0580012b0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 -- 172.21.15.62:0/1365824338 shutdown_connections 2024-04-25T09:08:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff04406b500 0x7ff04406d9c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff064152df0 0x7ff064153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff0640b4ea0 0x7ff0640b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 --2- 172.21.15.62:0/1365824338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff04c0039e0 0x7ff0640b4490 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 -- 172.21.15.62:0/1365824338 >> 172.21.15.62:0/1365824338 conn(0x7ff06406b000 msgr2=0x7ff0640d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 -- 172.21.15.62:0/1365824338 shutdown_connections 2024-04-25T09:08:06.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.178+0000 7ff06a000640 1 -- 172.21.15.62:0/1365824338 wait complete. 2024-04-25T09:08:06.185 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096e.00000000 parent 2024-04-25T09:08:06.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.314+0000 7faf50645f40 1 Processor -- start 2024-04-25T09:08:06.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.314+0000 7faf50645f40 1 -- start start 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.315+0000 7faf50645f40 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611881ca4a0 0x56118829fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.315+0000 7faf50645f40 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 0x5611882a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.315+0000 7faf50645f40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611882a3f70 0x5611882167e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.315+0000 7faf50645f40 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611881cadf0 con 0x5611881ca4a0 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.315+0000 7faf50645f40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561188171dd0 con 0x5611882a3f70 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.315+0000 7faf50645f40 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x561188217d90 con 0x5611882a2d40 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.315+0000 7faf4d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611881ca4a0 0x56118829fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611881ca4a0 0x56118829fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60236/0 (socket says 172.21.15.62:60236) 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4d800640 1 -- 172.21.15.62:0/811976046 learned_addr learned my addr 172.21.15.62:0/811976046 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4d800640 1 -- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 msgr2=0x5611882a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4ce00640 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 0x5611882a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4e200640 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611882a3f70 0x5611882167e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4d800640 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 0x5611882a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4d800640 1 -- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611882a3f70 msgr2=0x5611882167e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4d800640 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611882a3f70 0x5611882167e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4d800640 1 -- 172.21.15.62:0/811976046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611881f7e80 con 0x5611881ca4a0 2024-04-25T09:08:06.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4e200640 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611882a3f70 0x5611882167e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.316+0000 7faf4d800640 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611881ca4a0 0x56118829fdd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7faf40001e30 tx=0x7faf4000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbd69e419f2f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf47400640 1 -- 172.21.15.62:0/811976046 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40012020 con 0x5611881ca4a0 2024-04-25T09:08:06.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf47400640 1 -- 172.21.15.62:0/811976046 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4000e670 con 0x5611881ca4a0 2024-04-25T09:08:06.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf47400640 1 -- 172.21.15.62:0/811976046 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4000ea40 con 0x5611881ca4a0 2024-04-25T09:08:06.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf50645f40 1 -- 172.21.15.62:0/811976046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611881ca4a0 msgr2=0x56118829fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf50645f40 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611881ca4a0 0x56118829fdd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7faf40001e30 tx=0x7faf4000b780 comp rx=0 tx=0).stop 2024-04-25T09:08:06.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf50645f40 1 -- 172.21.15.62:0/811976046 shutdown_connections 2024-04-25T09:08:06.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf50645f40 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611882a3f70 0x5611882167e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf50645f40 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 0x5611882a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf50645f40 1 --2- 172.21.15.62:0/811976046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611881ca4a0 0x56118829fdd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf50645f40 1 -- 172.21.15.62:0/811976046 >> 172.21.15.62:0/811976046 conn(0x5611881fa2b0 msgr2=0x5611881c87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.317+0000 7faf50645f40 1 -- 172.21.15.62:0/811976046 shutdown_connections 2024-04-25T09:08:06.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.318+0000 7faf50645f40 1 -- 172.21.15.62:0/811976046 wait complete. 2024-04-25T09:08:06.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.318+0000 7faf50645f40 1 Processor -- start 2024-04-25T09:08:06.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.319+0000 7faf50645f40 1 -- start start 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.319+0000 7faf50645f40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611881ca4a0 0x561188281ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.319+0000 7faf50645f40 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 0x561188286980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.319+0000 7faf50645f40 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611882a3f70 0x5611882a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf50645f40 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561188331b20 con 0x5611882a3f70 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf50645f40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611881cac00 con 0x5611881ca4a0 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf50645f40 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611881f8790 con 0x5611882a2d40 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611882a3f70 0x5611882a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611882a3f70 0x5611882a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60242/0 (socket says 172.21.15.62:60242) 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4e200640 1 -- 172.21.15.62:0/3598480405 learned_addr learned my addr 172.21.15.62:0/3598480405 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4d800640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611881ca4a0 0x561188281ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4ce00640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 0x561188286980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4e200640 1 -- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 msgr2=0x561188286980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4e200640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5611882a2d40 0x561188286980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4e200640 1 -- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611881ca4a0 msgr2=0x561188281ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4e200640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611881ca4a0 0x561188281ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.320+0000 7faf4e200640 1 -- 172.21.15.62:0/3598480405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56118829d490 con 0x5611882a3f70 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.321+0000 7faf4e200640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611882a3f70 0x5611882a0e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faf4800a470 tx=0x7faf4800a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.321+0000 7faf46000640 1 -- 172.21.15.62:0/3598480405 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48015070 con 0x5611882a3f70 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.321+0000 7faf46000640 1 -- 172.21.15.62:0/3598480405 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48011870 con 0x5611882a3f70 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.321+0000 7faf46000640 1 -- 172.21.15.62:0/3598480405 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48011c40 con 0x5611882a3f70 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.321+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56118829d770 con 0x5611882a3f70 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.321+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56118829dd30 con 0x5611882a3f70 2024-04-25T09:08:06.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.323+0000 7faf46000640 1 -- 172.21.15.62:0/3598480405 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf480103c0 con 0x5611882a3f70 2024-04-25T09:08:06.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.324+0000 7faf46000640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf30069eb0 0x7faf3006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.324+0000 7faf46000640 1 -- 172.21.15.62:0/3598480405 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7faf48089190 con 0x5611882a3f70 2024-04-25T09:08:06.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.325+0000 7faf50645f40 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x561188286ec0 0x5611882845e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.325+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 --> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] -- osd_op(unknown.0.0:1 7.27 7:e5ebf85a:::1000000096e.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50) v9 -- 0x561188284b20 con 0x561188286ec0 2024-04-25T09:08:06.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.325+0000 7faf4d800640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf30069eb0 0x7faf3006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.325+0000 7faf4ce00640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x561188286ec0 0x5611882845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.326+0000 7faf4ce00640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x561188286ec0 0x5611882845e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.326+0000 7faf4d800640 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf30069eb0 0x7faf3006c370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5611881c8890 tx=0x7faf40009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.327+0000 7faf4ce00640 1 -- 172.21.15.62:0/3598480405 <== osd.0 v2:172.21.15.62:6800/565545488 1 ==== osd_op_reply(1 1000000096e.00000000 [rmxattr] v50'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7faf3c054710 con 0x561188286ec0 2024-04-25T09:08:06.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x561188286ec0 msgr2=0x5611882845e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x561188286ec0 0x5611882845e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf30069eb0 msgr2=0x7faf3006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf30069eb0 0x7faf3006c370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5611881c8890 tx=0x7faf40009000 comp rx=0 tx=0).stop 2024-04-25T09:08:06.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611882a3f70 msgr2=0x5611882a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 --2- 172.21.15.62:0/3598480405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5611882a3f70 0x5611882a0e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faf4800a470 tx=0x7faf4800a930 comp rx=0 tx=0).stop 2024-04-25T09:08:06.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf4e200640 1 -- 172.21.15.62:0/3598480405 reap_dead start 2024-04-25T09:08:06.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 shutdown_connections 2024-04-25T09:08:06.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 >> 172.21.15.62:0/3598480405 conn(0x5611881fa2b0 msgr2=0x5611881fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 shutdown_connections 2024-04-25T09:08:06.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.328+0000 7faf50645f40 1 -- 172.21.15.62:0/3598480405 wait complete. 2024-04-25T09:08:06.321 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T09:08:06.321 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:06.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T09:08:06.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.530+0000 7f71f5400640 1 Processor -- start 2024-04-25T09:08:06.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.530+0000 7f71f5400640 1 -- start start 2024-04-25T09:08:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.531+0000 7f71f5400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71f00cebc0 0x7f71f006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.531+0000 7f71f5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71f00720c0 0x7f71f00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.531+0000 7f71f5400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f0072510 con 0x7f71f015bdf0 2024-04-25T09:08:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.531+0000 7f71f5400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f015c1d0 con 0x7f71f00720c0 2024-04-25T09:08:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.531+0000 7f71f5400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f00ae8c0 con 0x7f71f00cebc0 2024-04-25T09:08:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.532+0000 7f71ef200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71f00720c0 0x7f71f00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.532+0000 7f71ef200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71f00720c0 0x7f71f00703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60248/0 (socket says 172.21.15.62:60248) 2024-04-25T09:08:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.532+0000 7f71ef200640 1 -- 172.21.15.62:0/3455937432 learned_addr learned my addr 172.21.15.62:0/3455937432 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.532+0000 7f71ef200640 1 -- 172.21.15.62:0/3455937432 >> v2:172.21.15.121:3300/0 conn(0x7f71f00cebc0 msgr2=0x7f71f006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.532+0000 7f71ef200640 1 --2- 172.21.15.62:0/3455937432 >> v2:172.21.15.121:3300/0 conn(0x7f71f00cebc0 0x7f71f006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.532+0000 7f71ef200640 1 -- 172.21.15.62:0/3455937432 >> v1:172.21.15.121:6789/0 conn(0x7f71f015bdf0 legacy=0x7f71f006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.532+0000 7f71ef200640 1 -- 172.21.15.62:0/3455937432 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f006d070 con 0x7f71f00720c0 2024-04-25T09:08:06.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.533+0000 7f71ef200640 1 --2- 172.21.15.62:0/3455937432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00720c0 0x7f71f00703a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f71f0068870 tx=0x7f71e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597f7811994b0bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.534+0000 7f71ede00640 1 -- 172.21.15.62:0/3455937432 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8036070 con 0x7f71f00720c0 2024-04-25T09:08:06.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.534+0000 7f71ede00640 1 -- 172.21.15.62:0/3455937432 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e8031600 con 0x7f71f00720c0 2024-04-25T09:08:06.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.534+0000 7f71ede00640 1 -- 172.21.15.62:0/3455937432 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80319d0 con 0x7f71f00720c0 2024-04-25T09:08:06.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.534+0000 7f71f5400640 1 -- 172.21.15.62:0/3455937432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00720c0 msgr2=0x7f71f00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.534+0000 7f71f5400640 1 --2- 172.21.15.62:0/3455937432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00720c0 0x7f71f00703a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f71f0068870 tx=0x7f71e802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.534+0000 7f71f5400640 1 -- 172.21.15.62:0/3455937432 shutdown_connections 2024-04-25T09:08:06.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.535+0000 7f71f5400640 1 --2- 172.21.15.62:0/3455937432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00720c0 0x7f71f00703a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.535+0000 7f71f5400640 1 --2- 172.21.15.62:0/3455937432 >> v2:172.21.15.121:3300/0 conn(0x7f71f00cebc0 0x7f71f006c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.535+0000 7f71f5400640 1 -- 172.21.15.62:0/3455937432 >> 172.21.15.62:0/3455937432 conn(0x7f71f00b1410 msgr2=0x7f71f00b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.535+0000 7f71f5400640 1 -- 172.21.15.62:0/3455937432 shutdown_connections 2024-04-25T09:08:06.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.535+0000 7f71f5400640 1 -- 172.21.15.62:0/3455937432 wait complete. 2024-04-25T09:08:06.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.536+0000 7f71f5400640 1 Processor -- start 2024-04-25T09:08:06.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.537+0000 7f71f5400640 1 -- start start 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71f5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f00720c0 0x7f71f00adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00cebc0 0x7f71f00b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71f5400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015bdf0 0x7f71f0144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71f5400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00aedf0 con 0x7f71f00cebc0 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71f5400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00aeb10 con 0x7f71f00720c0 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71f5400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00708e0 con 0x7f71f015bdf0 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00cebc0 0x7f71f00b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00cebc0 0x7f71f00b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60262/0 (socket says 172.21.15.62:60262) 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71e5e00640 1 -- 172.21.15.62:0/1285776748 learned_addr learned my addr 172.21.15.62:0/1285776748 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71ee800640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f00720c0 0x7f71f00adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71ef200640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015bdf0 0x7f71f0144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71e5e00640 1 -- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015bdf0 msgr2=0x7f71f0144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71e5e00640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015bdf0 0x7f71f0144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71e5e00640 1 -- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f00720c0 msgr2=0x7f71f00adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71e5e00640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f00720c0 0x7f71f00adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.538+0000 7f71e5e00640 1 -- 172.21.15.62:0/1285776748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f01f5070 con 0x7f71f00cebc0 2024-04-25T09:08:06.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.539+0000 7f71ee800640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f00720c0 0x7f71f00adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:06.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.539+0000 7f71e5e00640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00cebc0 0x7f71f00b40e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f71d800b4a0 tx=0x7f71d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.539+0000 7f71e7400640 1 -- 172.21.15.62:0/1285776748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8014070 con 0x7f71f00cebc0 2024-04-25T09:08:06.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.539+0000 7f71e7400640 1 -- 172.21.15.62:0/1285776748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8010610 con 0x7f71f00cebc0 2024-04-25T09:08:06.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.539+0000 7f71e7400640 1 -- 172.21.15.62:0/1285776748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8010a00 con 0x7f71f00cebc0 2024-04-25T09:08:06.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.539+0000 7f71f5400640 1 -- 172.21.15.62:0/1285776748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f00b3600 con 0x7f71f00cebc0 2024-04-25T09:08:06.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.539+0000 7f71f5400640 1 -- 172.21.15.62:0/1285776748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f00b3bc0 con 0x7f71f00cebc0 2024-04-25T09:08:06.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.542+0000 7f71e7400640 1 -- 172.21.15.62:0/1285776748 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71d8017070 con 0x7f71f00cebc0 2024-04-25T09:08:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.543+0000 7f71e7400640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71cc06b4d0 0x7f71cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.543+0000 7f71e7400640 1 -- 172.21.15.62:0/1285776748 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f71d8089d70 con 0x7f71f00cebc0 2024-04-25T09:08:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.543+0000 7f71f8367740 1 Processor -- start 2024-04-25T09:08:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.543+0000 7f71ee800640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71cc06b4d0 0x7f71cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.544+0000 7f71f8367740 1 -- start start 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71f8367740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f4c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71f8367740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f46eb0 0x55b889f4aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71ee800640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71cc06b4d0 0x7f71cc06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f71e0006fd0 tx=0x7f71e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71f8367740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 0x55b889f4b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71f8367740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b889eafd80 con 0x55b889f46eb0 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71f8367740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b889e58cc0 con 0x55b889f4bfa0 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71f8367740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b889eabe20 con 0x55b889f4b0e0 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f46eb0 0x55b889f4aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71e5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f46eb0 0x55b889f4aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60272/0 (socket says 172.21.15.62:60272) 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71e5e00640 1 -- 172.21.15.62:0/2898826597 learned_addr learned my addr 172.21.15.62:0/2898826597 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71ee800640 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f4c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71e5e00640 1 -- 172.21.15.62:0/2898826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 msgr2=0x55b889f4b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71e5e00640 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 0x55b889f4b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71e5e00640 1 -- 172.21.15.62:0/2898826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 msgr2=0x55b889f4c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71e5e00640 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f4c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71e5e00640 1 -- 172.21.15.62:0/2898826597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b889c90c00 con 0x55b889f46eb0 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.545+0000 7f71ee800640 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f4c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.546+0000 7f71e5e00640 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f46eb0 0x55b889f4aba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f71d80880b0 tx=0x7f71d800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc40fd99acbf0cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.546+0000 7f71df000640 1 -- 172.21.15.62:0/2898826597 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d801e040 con 0x55b889f46eb0 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.546+0000 7f71df000640 1 -- 172.21.15.62:0/2898826597 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8054a30 con 0x55b889f46eb0 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.546+0000 7f71df000640 1 -- 172.21.15.62:0/2898826597 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8010ba0 con 0x55b889f46eb0 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.546+0000 7f71f8367740 1 -- 172.21.15.62:0/2898826597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f46eb0 msgr2=0x55b889f4aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.546+0000 7f71f8367740 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f46eb0 0x55b889f4aba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f71d80880b0 tx=0x7f71d800bf90 comp rx=0 tx=0).stop 2024-04-25T09:08:06.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.547+0000 7f71f8367740 1 -- 172.21.15.62:0/2898826597 shutdown_connections 2024-04-25T09:08:06.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.547+0000 7f71f8367740 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f46eb0 0x55b889f4aba0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.547+0000 7f71f8367740 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 0x55b889f4b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.547+0000 7f71f8367740 1 --2- 172.21.15.62:0/2898826597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f4c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.547+0000 7f71f8367740 1 -- 172.21.15.62:0/2898826597 >> 172.21.15.62:0/2898826597 conn(0x55b889f49330 msgr2=0x55b889f91b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.547+0000 7f71f8367740 1 -- 172.21.15.62:0/2898826597 shutdown_connections 2024-04-25T09:08:06.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.547+0000 7f71f8367740 1 -- 172.21.15.62:0/2898826597 wait complete. 2024-04-25T09:08:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.547+0000 7f71f8367740 1 -- start start 2024-04-25T09:08:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.548+0000 7f71f8367740 1 Processor -- start 2024-04-25T09:08:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71f8367740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 0x55b889f6b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71f8367740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71f8367740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f952b0 0x55b889f95730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71f8367740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b889f77b30 con 0x55b889f952b0 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71f8367740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b889e4e240 con 0x55b889f4bfa0 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71f8367740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b889f76fb0 con 0x55b889f4b0e0 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71f8367740 10 client.0 _ll_register_callbacks cb 0x55b889ea7280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ef200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f952b0 0x55b889f95730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ef200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f952b0 0x55b889f95730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60274/0 (socket says 172.21.15.62:60274) 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ef200640 1 -- 172.21.15.62:0/917840406 learned_addr learned my addr 172.21.15.62:0/917840406 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71e5e00640 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ee800640 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 0x55b889f6b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ef200640 1 -- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 msgr2=0x55b889f6b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ef200640 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 0x55b889f6b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ef200640 1 -- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 msgr2=0x55b889f707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ef200640 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.549+0000 7f71ef200640 1 -- 172.21.15.62:0/917840406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b889b844f0 con 0x55b889f952b0 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71ee800640 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 0x55b889f6b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71ef200640 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f952b0 0x55b889f95730 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f71e8034040 tx=0x7f71e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde4176b1bee9e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71d3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71d3400640 1 -- 172.21.15.62:0/917840406 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8042070 con 0x55b889f952b0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71d3400640 1 -- 172.21.15.62:0/917840406 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f71e8031d40 con 0x55b889f952b0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71f8367740 1 -- 172.21.15.62:0/917840406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71d8054a30 con 0x55b889f952b0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71d3400640 1 -- 172.21.15.62:0/917840406 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e804c050 con 0x55b889f952b0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71f8367740 10 client.6120 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.550+0000 7f71f8367740 1 -- 172.21.15.62:0/917840406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55b889cf1390 con 0x55b889f952b0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71d3400640 1 -- 172.21.15.62:0/917840406 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8054a30 con 0x55b889f952b0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71d3400640 1 -- 172.21.15.62:0/917840406 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71e804f020 con 0x55b889f952b0 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71f8367740 10 client.6120 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71f8367740 10 client.6120 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71f8367740 20 client.6120 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71f8367740 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55b889f96700 0x55b889f96ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71f8367740 4 client.6120 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71f8367740 1 -- 172.21.15.62:0/917840406 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b889ebd2c0 con 0x55b889f96700 2024-04-25T09:08:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.551+0000 7f71ee800640 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55b889f96700 0x55b889f96ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.552+0000 7f71ee800640 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55b889f96700 0x55b889f96ae0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6bfe4f185bfc6e server_cookie=6821a7e6d2a72f2f in_seq=0 out_seq=0 2024-04-25T09:08:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.552+0000 7f71d3400640 10 client.6120 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.552+0000 7f71d3400640 1 -- 172.21.15.62:0/917840406 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b889ebd2c0 con 0x55b889f96700 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.552+0000 7f71d3400640 10 client.6120 handle_command_reply: tid=0 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 client.6120 shutdown 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 -- 172.21.15.62:0/917840406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f952b0 msgr2=0x55b889f95730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f952b0 0x55b889f95730 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f71e8034040 tx=0x7f71e8004a40 comp rx=0 tx=0).stop 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 -- 172.21.15.62:0/917840406 shutdown_connections 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55b889f96700 0x55b889f96ae0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f952b0 0x55b889f95730 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889f707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71d3400640 0 client.6120 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 --2- 172.21.15.62:0/917840406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4b0e0 0x55b889f6b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 -- 172.21.15.62:0/917840406 >> 172.21.15.62:0/917840406 conn(0x55b889f46eb0 msgr2=0x55b889f466b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 -- 172.21.15.62:0/917840406 shutdown_connections 2024-04-25T09:08:06.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 1 -- 172.21.15.62:0/917840406 wait complete. 2024-04-25T09:08:06.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.553+0000 7f71f8367740 20 client.6120 trim_cache size 0 max 16384 2024-04-25T09:08:06.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.570+0000 7f71f8367740 1 Processor -- start 2024-04-25T09:08:06.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.570+0000 7f71f8367740 1 -- start start 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71f8367740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f61bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71f8367740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4bfa0 0x55b889f45e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71f8367740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f952b0 0x55b889f46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71f8367740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0200340 con 0x55b889f4b0e0 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71f8367740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0159a80 con 0x55b889f952b0 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71f8367740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0158e80 con 0x55b889f4bfa0 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71ee800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f61bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71ee800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f61bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60276/0 (socket says 172.21.15.62:60276) 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.571+0000 7f71ee800640 1 -- 172.21.15.62:0/1882013709 learned_addr learned my addr 172.21.15.62:0/1882013709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71ee800640 1 -- 172.21.15.62:0/1882013709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4bfa0 msgr2=0x55b889f45e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71ee800640 1 --2- 172.21.15.62:0/1882013709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4bfa0 0x55b889f45e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71ee800640 1 -- 172.21.15.62:0/1882013709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f952b0 msgr2=0x55b889f46360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71ee800640 1 --2- 172.21.15.62:0/1882013709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f952b0 0x55b889f46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71ee800640 1 -- 172.21.15.62:0/1882013709 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b889b5d670 con 0x55b889f4b0e0 2024-04-25T09:08:06.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71ee800640 1 --2- 172.21.15.62:0/1882013709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f61bd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f71e000dba0 tx=0x7f71e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af1c76082eb6747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71de600640 1 -- 172.21.15.62:0/1882013709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e002a070 con 0x55b889f4b0e0 2024-04-25T09:08:06.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71de600640 1 -- 172.21.15.62:0/1882013709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e0014b70 con 0x55b889f4b0e0 2024-04-25T09:08:06.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.572+0000 7f71de600640 1 -- 172.21.15.62:0/1882013709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0024150 con 0x55b889f4b0e0 2024-04-25T09:08:06.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 -- 172.21.15.62:0/1882013709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 msgr2=0x55b889f61bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 --2- 172.21.15.62:0/1882013709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f61bd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f71e000dba0 tx=0x7f71e0013000 comp rx=0 tx=0).stop 2024-04-25T09:08:06.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 -- 172.21.15.62:0/1882013709 shutdown_connections 2024-04-25T09:08:06.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 --2- 172.21.15.62:0/1882013709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f952b0 0x55b889f46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 --2- 172.21.15.62:0/1882013709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f4bfa0 0x55b889f45e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 --2- 172.21.15.62:0/1882013709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f61bd0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 -- 172.21.15.62:0/1882013709 >> 172.21.15.62:0/1882013709 conn(0x55b889f46eb0 msgr2=0x55b889f6b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 -- 172.21.15.62:0/1882013709 shutdown_connections 2024-04-25T09:08:06.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.573+0000 7f71f8367740 1 -- 172.21.15.62:0/1882013709 wait complete. 2024-04-25T09:08:06.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.574+0000 7f71f8367740 1 -- start start 2024-04-25T09:08:06.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.574+0000 7f71f8367740 1 Processor -- start 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71f8367740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f46610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71f8367740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889fad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71f8367740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f952b0 0x55b889f70030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71f8367740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0159480 con 0x55b889f4b0e0 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71f8367740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b889f96e20 con 0x55b889f4bfa0 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71f8367740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b889f91ab0 con 0x55b889f952b0 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71f8367740 10 client.0 _ll_register_callbacks cb 0x7f71d8058210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71ee800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f46610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71ee800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f46610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60290/0 (socket says 172.21.15.62:60290) 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.575+0000 7f71ee800640 1 -- 172.21.15.62:0/3501117059 learned_addr learned my addr 172.21.15.62:0/3501117059 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71e5e00640 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889fad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71ef200640 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f952b0 0x55b889f70030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71ee800640 1 -- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f952b0 msgr2=0x55b889f70030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71ee800640 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f952b0 0x55b889f70030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71ee800640 1 -- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 msgr2=0x55b889fad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71ee800640 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889fad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71ee800640 1 -- 172.21.15.62:0/3501117059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b889f99920 con 0x55b889f4b0e0 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71ef200640 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f952b0 0x55b889f70030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71ee800640 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f46610 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f71e000d9c0 tx=0x7f71e0004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e6685745ffab68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71d3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.576+0000 7f71d3400640 1 -- 172.21.15.62:0/3501117059 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e002a040 con 0x55b889f4b0e0 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71d3400640 1 -- 172.21.15.62:0/3501117059 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f71e0005d10 con 0x55b889f4b0e0 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 1 -- 172.21.15.62:0/3501117059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e0014b70 con 0x55b889f4b0e0 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71d3400640 1 -- 172.21.15.62:0/3501117059 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e0006040 con 0x55b889f4b0e0 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 10 client.6126 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 1 -- 172.21.15.62:0/3501117059 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55b889f6bc00 con 0x55b889f4b0e0 2024-04-25T09:08:06.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71d3400640 1 -- 172.21.15.62:0/3501117059 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e0014b70 con 0x55b889f4b0e0 2024-04-25T09:08:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71d3400640 1 -- 172.21.15.62:0/3501117059 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71e002d070 con 0x55b889f4b0e0 2024-04-25T09:08:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 10 client.6126 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 10 client.6126 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 20 client.6126 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55b889fad7b0 0x55b889f70570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 4 client.6126 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T09:08:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.577+0000 7f71f8367740 1 -- 172.21.15.62:0/3501117059 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55b889f4af30 con 0x55b889fad7b0 2024-04-25T09:08:06.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.578+0000 7f71e5e00640 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55b889fad7b0 0x55b889f70570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.578+0000 7f71e5e00640 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55b889fad7b0 0x55b889f70570 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96d16eb8c4681ad server_cookie=97958b275771d6f3 in_seq=0 out_seq=0 2024-04-25T09:08:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.578+0000 7f71d3400640 10 client.6126 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71d3400640 1 -- 172.21.15.62:0/3501117059 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b889f4af30 con 0x55b889fad7b0 2024-04-25T09:08:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71d3400640 10 client.6126 handle_command_reply: tid=0 2024-04-25T09:08:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 client.6126 shutdown 2024-04-25T09:08:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 -- 172.21.15.62:0/3501117059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 msgr2=0x55b889f46610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f46610 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f71e000d9c0 tx=0x7f71e0004640 comp rx=0 tx=0).stop 2024-04-25T09:08:06.569 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f6707c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096e [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v673 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5733=0-4194304@1} caps={5733=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5733 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55e2444c2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1000000096e:[<0x10000000001/new_file_1 v673>,<0x10000000000/ceph-qa-suite v84>,<0x1/scrub_test_path v40>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T09:08:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 -- 172.21.15.62:0/3501117059 shutdown_connections 2024-04-25T09:08:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55b889fad7b0 0x55b889f70570 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55b889f952b0 0x55b889f70030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b889f4bfa0 0x55b889fad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 --2- 172.21.15.62:0/3501117059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55b889f4b0e0 0x55b889f46610 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 -- 172.21.15.62:0/3501117059 >> 172.21.15.62:0/3501117059 conn(0x55b889f46eb0 msgr2=0x55b889f6f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71d3400640 0 client.6126 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 -- 172.21.15.62:0/3501117059 shutdown_connections 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 1 -- 172.21.15.62:0/3501117059 wait complete. 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.579+0000 7f71f8367740 20 client.6126 trim_cache size 0 max 16384 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "5bdcf376-7436-491e-bc15-fe5771c09c59", 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:08:06.571 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:06.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 -- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71cc06b4d0 msgr2=0x7f71cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71cc06b4d0 0x7f71cc06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f71e0006fd0 tx=0x7f71e0008040 comp rx=0 tx=0).stop 2024-04-25T09:08:06.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 -- 172.21.15.62:0/1285776748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00cebc0 msgr2=0x7f71f00b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00cebc0 0x7f71f00b40e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f71d800b4a0 tx=0x7f71d800b960 comp rx=0 tx=0).stop 2024-04-25T09:08:06.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 -- 172.21.15.62:0/1285776748 shutdown_connections 2024-04-25T09:08:06.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f71cc06b4d0 0x7f71cc06d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71f00cebc0 0x7f71f00b40e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71f015bdf0 0x7f71f0144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 --2- 172.21.15.62:0/1285776748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71f00720c0 0x7f71f00adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 -- 172.21.15.62:0/1285776748 >> 172.21.15.62:0/1285776748 conn(0x7f71f00b1410 msgr2=0x7f71f00ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 -- 172.21.15.62:0/1285776748 shutdown_connections 2024-04-25T09:08:06.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.580+0000 7f71f5400640 1 -- 172.21.15.62:0/1285776748 wait complete. 2024-04-25T09:08:06.586 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '5bdcf376-7436-491e-bc15-fe5771c09c59', 'mode': 'asynchronous'}' 2024-04-25T09:08:06.586 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:08:06.586 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:08:06.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.800+0000 7f657ec00640 1 Processor -- start 2024-04-25T09:08:06.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.800+0000 7f657ec00640 1 -- start start 2024-04-25T09:08:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.802+0000 7f657ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f657815dc70 0x7f657806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.802+0000 7f657ec00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657806d7f0 con 0x7f657806e160 2024-04-25T09:08:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.802+0000 7f657ec00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657806d230 con 0x7f657815e050 2024-04-25T09:08:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.802+0000 7f657ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657806d510 con 0x7f657815dc70 2024-04-25T09:08:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f657cc00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f657806e160 0x7f65780b2fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53250/0 (socket says 172.21.15.62:53250) 2024-04-25T09:08:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f657cc00640 1 -- 172.21.15.62:0/2185748455 learned_addr learned my addr 172.21.15.62:0/2185748455 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:06.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6577e00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f657815dc70 0x7f657806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936987850 0 0) 0x7f657806d7f0 con 0x7f657806e160 2024-04-25T09:08:06.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f655c003520 con 0x7f657806e160 2024-04-25T09:08:06.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3545388820 0 0) 0x7f655c003520 con 0x7f657806e160 2024-04-25T09:08:06.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f657815dc70 msgr2=0x7f657806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6576a00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f657815dc70 0x7f657806dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 >> v1:172.21.15.121:6789/0 conn(0x7f657815e050 legacy=0x7f657806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65780b27b0 con 0x7f657806e160 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.803+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f65700019f0 con 0x7f657806e160 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 >> v1:172.21.15.62:6789/0 conn(0x7f657806e160 legacy=0x7f65780b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6576a00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f655c0039e0 0x7f655c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6576a00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f657815dc70 0x7f655c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6576a00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f655c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657806d510 con 0x7f655c004d40 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c00baa0 con 0x7f655c0039e0 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c00bca0 con 0x7f657815dc70 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f657cc00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f655c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6577400640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f657815dc70 0x7f655c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f6577e00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f655c0039e0 0x7f655c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.804+0000 7f657cc00640 1 -- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f657815dc70 msgr2=0x7f655c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.805+0000 7f657cc00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f657815dc70 0x7f655c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.805+0000 7f657cc00640 1 -- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f655c0039e0 msgr2=0x7f655c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.805+0000 7f657cc00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f655c0039e0 0x7f655c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.805+0000 7f657cc00640 1 -- 172.21.15.62:0/2185748455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c003520 con 0x7f655c004d40 2024-04-25T09:08:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f657cc00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f655c0051c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f657000a740 tx=0x7f655c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103ccb584da16364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000a390 con 0x7f655c004d40 2024-04-25T09:08:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657002f070 con 0x7f655c004d40 2024-04-25T09:08:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f6576a00640 1 -- 172.21.15.62:0/2185748455 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657002bbc0 con 0x7f655c004d40 2024-04-25T09:08:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f657ec00640 1 -- 172.21.15.62:0/2185748455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 msgr2=0x7f655c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f657ec00640 1 --2- 172.21.15.62:0/2185748455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f655c0051c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f657000a740 tx=0x7f655c005830 comp rx=0 tx=0).stop 2024-04-25T09:08:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f657cc00640 1 -- 172.21.15.62:0/2185748455 reap_dead start 2024-04-25T09:08:06.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f657ec00640 1 -- 172.21.15.62:0/2185748455 shutdown_connections 2024-04-25T09:08:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.806+0000 7f657ec00640 1 -- 172.21.15.62:0/2185748455 >> 172.21.15.62:0/2185748455 conn(0x7f65780b1410 msgr2=0x7f65780b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.807+0000 7f657ec00640 1 -- 172.21.15.62:0/2185748455 shutdown_connections 2024-04-25T09:08:06.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.807+0000 7f657ec00640 1 -- 172.21.15.62:0/2185748455 wait complete. 2024-04-25T09:08:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.808+0000 7f657ec00640 1 Processor -- start 2024-04-25T09:08:06.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.808+0000 7f657ec00640 1 -- start start 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.809+0000 7f657ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f6578147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.809+0000 7f657ec00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6578156250 0x7f65780afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.809+0000 7f657ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65781566d0 0x7f65780b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.809+0000 7f657ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657806dda0 con 0x7f655c004d40 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.809+0000 7f657ec00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657806dc20 con 0x7f6578156250 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.809+0000 7f657ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657806df20 con 0x7f65781566d0 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f6578147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f6578147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60302/0 (socket says 172.21.15.62:60302) 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577e00640 1 -- 172.21.15.62:0/2766675846 learned_addr learned my addr 172.21.15.62:0/2766675846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f657cc00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65781566d0 0x7f65780b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577400640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6578156250 0x7f65780afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577e00640 1 -- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65781566d0 msgr2=0x7f65780b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577e00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65781566d0 0x7f65780b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577e00640 1 -- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6578156250 msgr2=0x7f65780afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577e00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6578156250 0x7f65780afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.810+0000 7f6577e00640 1 -- 172.21.15.62:0/2766675846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657815f2b0 con 0x7f655c004d40 2024-04-25T09:08:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.811+0000 7f6577e00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f6578147210 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6568011600 tx=0x7f6568011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.811+0000 7f656fe00640 1 -- 172.21.15.62:0/2766675846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568015870 con 0x7f655c004d40 2024-04-25T09:08:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.811+0000 7f656fe00640 1 -- 172.21.15.62:0/2766675846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568015a10 con 0x7f655c004d40 2024-04-25T09:08:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.811+0000 7f657ec00640 1 -- 172.21.15.62:0/2766675846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657815f590 con 0x7f655c004d40 2024-04-25T09:08:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.811+0000 7f656fe00640 1 -- 172.21.15.62:0/2766675846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568015dc0 con 0x7f655c004d40 2024-04-25T09:08:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.811+0000 7f657ec00640 1 -- 172.21.15.62:0/2766675846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657815fb50 con 0x7f655c004d40 2024-04-25T09:08:06.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.813+0000 7f656fe00640 1 -- 172.21.15.62:0/2766675846 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f656800f450 con 0x7f655c004d40 2024-04-25T09:08:06.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.814+0000 7f656fe00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f655806b500 0x7f655806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.815+0000 7f656fe00640 1 -- 172.21.15.62:0/2766675846 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f656808ab70 con 0x7f655c004d40 2024-04-25T09:08:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.815+0000 7f6577400640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f655806b500 0x7f655806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.815+0000 7f6581a34740 1 Processor -- start 2024-04-25T09:08:06.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.815+0000 7f6581a34740 1 -- start start 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.816+0000 7f6577400640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f655806b500 0x7f655806d9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f657815d5d0 tx=0x7f6564009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.816+0000 7f6581a34740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626175ecbb0 0x5626175eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.816+0000 7f6581a34740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617629280 0x5626175ecf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.816+0000 7f6581a34740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 0x562617617af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.816+0000 7f6581a34740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562617550c70 con 0x562617629280 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.816+0000 7f6581a34740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626174f9cf0 con 0x5626175ecbb0 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.816+0000 7f6581a34740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56261754cd10 con 0x562617617670 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617629280 0x5626175ecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617629280 0x5626175ecf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60318/0 (socket says 172.21.15.62:60318) 2024-04-25T09:08:06.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 -- 172.21.15.62:0/3733301381 learned_addr learned my addr 172.21.15.62:0/3733301381 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f657cc00640 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 0x562617617af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577e00640 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626175ecbb0 0x5626175eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 -- 172.21.15.62:0/3733301381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 msgr2=0x562617617af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 0x562617617af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 -- 172.21.15.62:0/3733301381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626175ecbb0 msgr2=0x5626175eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626175ecbb0 0x5626175eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 -- 172.21.15.62:0/3733301381 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56261732f7d0 con 0x562617629280 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.817+0000 7f6577400640 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617629280 0x5626175ecf90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f65640015a0 tx=0x7f6564013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea782c3b458975a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6563e00640 1 -- 172.21.15.62:0/3733301381 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564027070 con 0x562617629280 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6563e00640 1 -- 172.21.15.62:0/3733301381 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564014740 con 0x562617629280 2024-04-25T09:08:06.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6563e00640 1 -- 172.21.15.62:0/3733301381 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564014af0 con 0x562617629280 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6581a34740 1 -- 172.21.15.62:0/3733301381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617629280 msgr2=0x5626175ecf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6581a34740 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617629280 0x5626175ecf90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f65640015a0 tx=0x7f6564013000 comp rx=0 tx=0).stop 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6581a34740 1 -- 172.21.15.62:0/3733301381 shutdown_connections 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6581a34740 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617629280 0x5626175ecf90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6581a34740 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 0x562617617af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6581a34740 1 --2- 172.21.15.62:0/3733301381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5626175ecbb0 0x5626175eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.818+0000 7f6581a34740 1 -- 172.21.15.62:0/3733301381 >> 172.21.15.62:0/3733301381 conn(0x5626175b4a30 msgr2=0x562617632670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.819+0000 7f6581a34740 1 -- 172.21.15.62:0/3733301381 shutdown_connections 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.819+0000 7f6581a34740 1 -- 172.21.15.62:0/3733301381 wait complete. 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.819+0000 7f6581a34740 1 -- start start 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.819+0000 7f6581a34740 1 Processor -- start 2024-04-25T09:08:06.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.820+0000 7f6581a34740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626175ecbb0 0x56261760de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.820+0000 7f6581a34740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 0x562617619d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.820+0000 7f6581a34740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x562617611ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.820+0000 7f6581a34740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626174ef3a0 con 0x5626175ecbb0 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.820+0000 7f6581a34740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626176151c0 con 0x562617629280 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.820+0000 7f6581a34740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562617642aa0 con 0x562617617670 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.820+0000 7f6581a34740 10 client.0 _ll_register_callbacks cb 0x56261750cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626175ecbb0 0x56261760de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626175ecbb0 0x56261760de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60322/0 (socket says 172.21.15.62:60322) 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 -- 172.21.15.62:0/3810502795 learned_addr learned my addr 172.21.15.62:0/3810502795 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577400640 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 0x562617619d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f657cc00640 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x562617611ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 -- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 msgr2=0x562617619d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 0x562617619d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 -- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 msgr2=0x562617611ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x562617611ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 -- 172.21.15.62:0/3810502795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626172653f0 con 0x5626175ecbb0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f657cc00640 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x562617611ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6577e00640 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626175ecbb0 0x56261760de90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f65680591e0 tx=0x7f6568059310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fc503195f9408f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.821+0000 7f6560c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6560c00640 1 -- 172.21.15.62:0/3810502795 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656805d160 con 0x5626175ecbb0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6560c00640 1 -- 172.21.15.62:0/3810502795 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f65680614d0 con 0x5626175ecbb0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 1 -- 172.21.15.62:0/3810502795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6564014740 con 0x5626175ecbb0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6560c00640 1 -- 172.21.15.62:0/3810502795 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6568061960 con 0x5626175ecbb0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 10 client.6138 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 1 -- 172.21.15.62:0/3810502795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5626172250f0 con 0x5626175ecbb0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6560c00640 1 -- 172.21.15.62:0/3810502795 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564014740 con 0x5626175ecbb0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6560c00640 1 -- 172.21.15.62:0/3810502795 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f65680586a0 con 0x5626175ecbb0 2024-04-25T09:08:06.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 10 client.6138 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:06.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 10 client.6138 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:06.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 20 client.6138 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:06.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x562617612010 0x5626176100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 4 client.6138 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:06.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.822+0000 7f6581a34740 1 -- 172.21.15.62:0/3810502795 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56261755e1b0 con 0x562617612010 2024-04-25T09:08:06.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.823+0000 7f6577400640 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x562617612010 0x5626176100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.823+0000 7f6577400640 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x562617612010 0x5626176100e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2870a68755c7e4cd server_cookie=c58a884c7dcde6bb in_seq=0 out_seq=0 2024-04-25T09:08:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.823+0000 7f6560c00640 10 client.6138 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.824+0000 7f6560c00640 1 -- 172.21.15.62:0/3810502795 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56261755e1b0 con 0x562617612010 2024-04-25T09:08:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.824+0000 7f6560c00640 10 client.6138 handle_command_reply: tid=0 2024-04-25T09:08:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.824+0000 7f6581a34740 1 client.6138 shutdown 2024-04-25T09:08:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.824+0000 7f6581a34740 1 -- 172.21.15.62:0/3810502795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626175ecbb0 msgr2=0x56261760de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.824+0000 7f6581a34740 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626175ecbb0 0x56261760de90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f65680591e0 tx=0x7f6568059310 comp rx=0 tx=0).stop 2024-04-25T09:08:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 1 -- 172.21.15.62:0/3810502795 shutdown_connections 2024-04-25T09:08:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x562617612010 0x5626176100e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x562617611ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562617617670 0x562617619d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 1 --2- 172.21.15.62:0/3810502795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5626175ecbb0 0x56261760de90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 1 -- 172.21.15.62:0/3810502795 >> 172.21.15.62:0/3810502795 conn(0x5626175b4a30 msgr2=0x5626176155c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6560c00640 0 client.6138 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 1 -- 172.21.15.62:0/3810502795 shutdown_connections 2024-04-25T09:08:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 1 -- 172.21.15.62:0/3810502795 wait complete. 2024-04-25T09:08:06.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.825+0000 7f6581a34740 20 client.6138 trim_cache size 0 max 16384 2024-04-25T09:08:06.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.844+0000 7f6581a34740 1 Processor -- start 2024-04-25T09:08:06.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.844+0000 7f6581a34740 1 -- start start 2024-04-25T09:08:06.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.845+0000 7f6581a34740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 0x562617654f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.845+0000 7f6581a34740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261760d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.845+0000 7f6581a34740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x562617632ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.845+0000 7f6581a34740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65782009b0 con 0x562617617670 2024-04-25T09:08:06.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.845+0000 7f6581a34740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781ffbb0 con 0x562617629280 2024-04-25T09:08:06.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.845+0000 7f6581a34740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6578152830 con 0x5626175ecbb0 2024-04-25T09:08:06.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261760d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261760d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60328/0 (socket says 172.21.15.62:60328) 2024-04-25T09:08:06.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 -- 172.21.15.62:0/2305937416 learned_addr learned my addr 172.21.15.62:0/2305937416 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577e00640 1 --2- 172.21.15.62:0/2305937416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 0x562617654f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 -- 172.21.15.62:0/2305937416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 msgr2=0x562617654f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 --2- 172.21.15.62:0/2305937416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 0x562617654f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 -- 172.21.15.62:0/2305937416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 msgr2=0x562617632ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 --2- 172.21.15.62:0/2305937416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x562617632ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 -- 172.21.15.62:0/2305937416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626175e8a60 con 0x562617617670 2024-04-25T09:08:06.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.846+0000 7f6577400640 1 --2- 172.21.15.62:0/2305937416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261760d530 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6564001570 tx=0x7f6564014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dcb90fc48313e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.847+0000 7f6563400640 1 -- 172.21.15.62:0/2305937416 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564027040 con 0x562617617670 2024-04-25T09:08:06.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.848+0000 7f6563400640 1 -- 172.21.15.62:0/2305937416 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564039070 con 0x562617617670 2024-04-25T09:08:06.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.848+0000 7f6563400640 1 -- 172.21.15.62:0/2305937416 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564014d70 con 0x562617617670 2024-04-25T09:08:06.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.848+0000 7f6581a34740 1 -- 172.21.15.62:0/2305937416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 msgr2=0x56261760d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.848+0000 7f6581a34740 1 --2- 172.21.15.62:0/2305937416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261760d530 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6564001570 tx=0x7f6564014000 comp rx=0 tx=0).stop 2024-04-25T09:08:06.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.849+0000 7f6581a34740 1 -- 172.21.15.62:0/2305937416 shutdown_connections 2024-04-25T09:08:06.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.849+0000 7f6581a34740 1 --2- 172.21.15.62:0/2305937416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261760d530 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.849+0000 7f6581a34740 1 --2- 172.21.15.62:0/2305937416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x562617632ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.849+0000 7f6581a34740 1 --2- 172.21.15.62:0/2305937416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 0x562617654f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.849+0000 7f6581a34740 1 -- 172.21.15.62:0/2305937416 >> 172.21.15.62:0/2305937416 conn(0x5626175b4a30 msgr2=0x56261761aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.849+0000 7f6581a34740 1 -- 172.21.15.62:0/2305937416 shutdown_connections 2024-04-25T09:08:06.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.849+0000 7f6581a34740 1 -- 172.21.15.62:0/2305937416 wait complete. 2024-04-25T09:08:06.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.849+0000 7f6581a34740 1 -- start start 2024-04-25T09:08:06.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.850+0000 7f6581a34740 1 Processor -- start 2024-04-25T09:08:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6581a34740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 0x5626175ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6581a34740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261764f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6581a34740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x56261764f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6581a34740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261750e020 con 0x562617617670 2024-04-25T09:08:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6581a34740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781ff5b0 con 0x562617629280 2024-04-25T09:08:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6581a34740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56261760e800 con 0x5626175ecbb0 2024-04-25T09:08:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6581a34740 10 client.0 _ll_register_callbacks cb 0x7f6568059020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6577400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261764f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6577400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261764f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60330/0 (socket says 172.21.15.62:60330) 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6577400640 1 -- 172.21.15.62:0/4008501190 learned_addr learned my addr 172.21.15.62:0/4008501190 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6577400640 1 -- 172.21.15.62:0/4008501190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 msgr2=0x5626175ec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6577400640 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 0x5626175ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6577400640 1 -- 172.21.15.62:0/4008501190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 msgr2=0x56261764f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6577400640 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x56261764f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.851+0000 7f6577400640 1 -- 172.21.15.62:0/4008501190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562617635eb0 con 0x562617617670 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6577400640 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261764f070 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6564036040 tx=0x7f6564015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635081346764a325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6560c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6560c00640 1 -- 172.21.15.62:0/4008501190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564042060 con 0x562617617670 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6560c00640 1 -- 172.21.15.62:0/4008501190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6564014900 con 0x562617617670 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6581a34740 1 -- 172.21.15.62:0/4008501190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6564039070 con 0x562617617670 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6560c00640 1 -- 172.21.15.62:0/4008501190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6564048050 con 0x562617617670 2024-04-25T09:08:06.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6581a34740 10 client.6144 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6581a34740 1 -- 172.21.15.62:0/4008501190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x56261764fb40 con 0x562617617670 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.852+0000 7f6560c00640 1 -- 172.21.15.62:0/4008501190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6564039070 con 0x562617617670 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.853+0000 7f6560c00640 1 -- 172.21.15.62:0/4008501190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f656404b020 con 0x562617617670 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.853+0000 7f6581a34740 10 client.6144 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.853+0000 7f6581a34740 10 client.6144 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.853+0000 7f6581a34740 20 client.6144 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.853+0000 7f6581a34740 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56261761b060 0x56261764ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.853+0000 7f6581a34740 4 client.6144 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.853+0000 7f6581a34740 1 -- 172.21.15.62:0/4008501190 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562617558500 con 0x56261761b060 2024-04-25T09:08:06.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.853+0000 7f6577e00640 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56261761b060 0x56261764ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:06.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6577e00640 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56261761b060 0x56261764ac70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea6aa2cc543f1d server_cookie=1d8a2c22be870b12 in_seq=0 out_seq=0 2024-04-25T09:08:06.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6560c00640 10 client.6144 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6560c00640 1 -- 172.21.15.62:0/4008501190 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x562617558500 con 0x56261761b060 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6560c00640 10 client.6144 handle_command_reply: tid=0 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 client.6144 shutdown 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 -- 172.21.15.62:0/4008501190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 msgr2=0x56261764f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261764f070 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6564036040 tx=0x7f6564015000 comp rx=0 tx=0).stop 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 -- 172.21.15.62:0/4008501190 shutdown_connections 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x56261761b060 0x56261764ac70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562617617670 0x56261764f070 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562617629280 0x56261764f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 --2- 172.21.15.62:0/4008501190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5626175ecbb0 0x5626175ec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6560c00640 0 client.6144 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 -- 172.21.15.62:0/4008501190 >> 172.21.15.62:0/4008501190 conn(0x5626175b4a30 msgr2=0x562617615120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 -- 172.21.15.62:0/4008501190 shutdown_connections 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.854+0000 7f6581a34740 1 -- 172.21.15.62:0/4008501190 wait complete. 2024-04-25T09:08:06.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f6581a34740 20 client.6144 trim_cache size 0 max 16384 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "5bdcf376-7436-491e-bc15-fe5771c09c59": { 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "5bdcf376-7436-491e-bc15-fe5771c09c59", 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "" 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:06.846 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:06.847 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:06.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 -- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f655806b500 msgr2=0x7f655806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f655806b500 0x7f655806d9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f657815d5d0 tx=0x7f6564009040 comp rx=0 tx=0).stop 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 -- 172.21.15.62:0/2766675846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 msgr2=0x7f6578147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f6578147210 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6568011600 tx=0x7f6568011ac0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 -- 172.21.15.62:0/2766675846 shutdown_connections 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f655806b500 0x7f655806d9c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65781566d0 0x7f65780b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6578156250 0x7f65780afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 --2- 172.21.15.62:0/2766675846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f655c004d40 0x7f6578147210 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 -- 172.21.15.62:0/2766675846 >> 172.21.15.62:0/2766675846 conn(0x7f65780b1410 msgr2=0x7f65780b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 -- 172.21.15.62:0/2766675846 shutdown_connections 2024-04-25T09:08:06.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:06.855+0000 7f657ec00640 1 -- 172.21.15.62:0/2766675846 wait complete. 2024-04-25T09:08:10.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:08:10.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:08:11.861 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:08:11.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:08:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.099+0000 7f8d2b200640 1 Processor -- start 2024-04-25T09:08:12.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.099+0000 7f8d2b200640 1 -- start start 2024-04-25T09:08:12.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.101+0000 7f8d2b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8d2c0ccde0 0x7f8d2c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.101+0000 7f8d2b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0ced40 con 0x7f8d2c06d2c0 2024-04-25T09:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.101+0000 7f8d2b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0cd260 con 0x7f8d2c0b31a0 2024-04-25T09:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.101+0000 7f8d2b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0cf000 con 0x7f8d2c0ccde0 2024-04-25T09:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.101+0000 7f8d29e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8d2c06d2c0 0x7f8d2c06d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53262/0 (socket says 172.21.15.62:53262) 2024-04-25T09:08:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.101+0000 7f8d29e00640 1 -- 172.21.15.62:0/3880238412 learned_addr learned my addr 172.21.15.62:0/3880238412 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036024915 0 0) 0x7f8d2c0ced40 con 0x7f8d2c06d2c0 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d10003520 con 0x7f8d2c06d2c0 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d29400640 1 --2- 172.21.15.62:0/3880238412 >> v2:172.21.15.121:3300/0 conn(0x7f8d2c0ccde0 0x7f8d2c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1441375139 0 0) 0x7f8d10003520 con 0x7f8d2c06d2c0 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 >> v2:172.21.15.121:3300/0 conn(0x7f8d2c0ccde0 msgr2=0x7f8d2c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 --2- 172.21.15.62:0/3880238412 >> v2:172.21.15.121:3300/0 conn(0x7f8d2c0ccde0 0x7f8d2c0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 >> v1:172.21.15.121:6789/0 conn(0x7f8d2c0b31a0 legacy=0x7f8d2c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0cf2e0 con 0x7f8d2c06d2c0 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f8d1c001670 con 0x7f8d2c06d2c0 2024-04-25T09:08:12.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 >> v1:172.21.15.62:6789/0 conn(0x7f8d2c06d2c0 legacy=0x7f8d2c06d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d10003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d10004800 0x7f8d10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.102+0000 7f8d28a00640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d10005210 0x7f8d1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0cf000 con 0x7f8d10003a00 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1000c140 con 0x7f8d10005210 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1000c340 con 0x7f8d10004800 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d29e00640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d10003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d2a800640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d10005210 0x7f8d1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d29e00640 1 -- 172.21.15.62:0/3880238412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d10004800 msgr2=0x7f8d10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d29e00640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d10004800 0x7f8d10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d29e00640 1 -- 172.21.15.62:0/3880238412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d10005210 msgr2=0x7f8d1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d29e00640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d10005210 0x7f8d1000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d29e00640 1 -- 172.21.15.62:0/3880238412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d10003520 con 0x7f8d10003a00 2024-04-25T09:08:12.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.103+0000 7f8d2a800640 1 -- 172.21.15.62:0/3880238412 reap_dead start 2024-04-25T09:08:12.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.104+0000 7f8d29e00640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d10003e00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c160 tx=0x7f8d10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c01fd327d266341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c007c60 con 0x7f8d10003a00 2024-04-25T09:08:12.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c005ad0 con 0x7f8d10003a00 2024-04-25T09:08:12.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d28a00640 1 -- 172.21.15.62:0/3880238412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c007070 con 0x7f8d10003a00 2024-04-25T09:08:12.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d2b200640 1 -- 172.21.15.62:0/3880238412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 msgr2=0x7f8d10003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d2b200640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d10003e00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00c160 tx=0x7f8d10005830 comp rx=0 tx=0).stop 2024-04-25T09:08:12.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d2b200640 1 -- 172.21.15.62:0/3880238412 shutdown_connections 2024-04-25T09:08:12.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d2b200640 1 --2- 172.21.15.62:0/3880238412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d10003e00 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d2b200640 1 -- 172.21.15.62:0/3880238412 >> 172.21.15.62:0/3880238412 conn(0x7f8d2c066f20 msgr2=0x7f8d2c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.105+0000 7f8d2b200640 1 -- 172.21.15.62:0/3880238412 shutdown_connections 2024-04-25T09:08:12.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.106+0000 7f8d2b200640 1 -- 172.21.15.62:0/3880238412 wait complete. 2024-04-25T09:08:12.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.107+0000 7f8d2b200640 1 Processor -- start 2024-04-25T09:08:12.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.108+0000 7f8d2b200640 1 -- start start 2024-04-25T09:08:12.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d2b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d2c13aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d2b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d2c1417a0 0x7f8d2c141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d2b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2c14bed0 0x7f8d2c14c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d2b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b2c80 con 0x7f8d10003a00 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d2b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b29a0 con 0x7f8d2c1417a0 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d2b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1448d0 con 0x7f8d2c14bed0 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d2c13aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d2c13aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60354/0 (socket says 172.21.15.62:60354) 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d29e00640 1 -- 172.21.15.62:0/3013174969 learned_addr learned my addr 172.21.15.62:0/3013174969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d29e00640 1 -- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2c14bed0 msgr2=0x7f8d2c14c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d29e00640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2c14bed0 0x7f8d2c14c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d29e00640 1 -- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d2c1417a0 msgr2=0x7f8d2c141c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d29e00640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d2c1417a0 0x7f8d2c141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.109+0000 7f8d29e00640 1 -- 172.21.15.62:0/3013174969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c1f27c0 con 0x7f8d10003a00 2024-04-25T09:08:12.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.110+0000 7f8d29e00640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d2c13aed0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c02ad30 tx=0x7f8d2c0cf4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.110+0000 7f8d22000640 1 -- 172.21.15.62:0/3013174969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c043070 con 0x7f8d10003a00 2024-04-25T09:08:12.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.110+0000 7f8d22000640 1 -- 172.21.15.62:0/3013174969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c038040 con 0x7f8d10003a00 2024-04-25T09:08:12.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.110+0000 7f8d22000640 1 -- 172.21.15.62:0/3013174969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c007270 con 0x7f8d10003a00 2024-04-25T09:08:12.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.110+0000 7f8d2b200640 1 -- 172.21.15.62:0/3013174969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c1f2aa0 con 0x7f8d10003a00 2024-04-25T09:08:12.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.110+0000 7f8d2b200640 1 -- 172.21.15.62:0/3013174969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c13b410 con 0x7f8d10003a00 2024-04-25T09:08:12.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.112+0000 7f8d22000640 1 -- 172.21.15.62:0/3013174969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d1c007410 con 0x7f8d10003a00 2024-04-25T09:08:12.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.113+0000 7f8d22000640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d0c06b500 0x7f8d0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.114+0000 7f8d22000640 1 -- 172.21.15.62:0/3013174969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8d1c0a9ed0 con 0x7f8d10003a00 2024-04-25T09:08:12.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.114+0000 7f8d33b8a740 1 Processor -- start 2024-04-25T09:08:12.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.114+0000 7f8d29400640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d0c06b500 0x7f8d0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.114+0000 7f8d33b8a740 1 -- start start 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb07d70 0x55c2fdb09ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d29400640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d0c06b500 0x7f8d0c06d9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c1441d0 tx=0x7f8d18008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb44440 0x55c2fdb08150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2fda6bc70 con 0x55c2fdb32830 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2fda14cf0 con 0x55c2fdb07d70 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2fda67d10 con 0x55c2fdb44440 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d2a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d2a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60362/0 (socket says 172.21.15.62:60362) 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d2a800640 1 -- 172.21.15.62:0/6319979 learned_addr learned my addr 172.21.15.62:0/6319979 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d29400640 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb44440 0x55c2fdb08150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d29e00640 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb07d70 0x55c2fdb09ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d2a800640 1 -- 172.21.15.62:0/6319979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb44440 msgr2=0x55c2fdb08150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d2a800640 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb44440 0x55c2fdb08150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d2a800640 1 -- 172.21.15.62:0/6319979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb07d70 msgr2=0x55c2fdb09ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d2a800640 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb07d70 0x55c2fdb09ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.116+0000 7f8d2a800640 1 -- 172.21.15.62:0/6319979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2fd84a7d0 con 0x55c2fdb32830 2024-04-25T09:08:12.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.117+0000 7f8d2a800640 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32cb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0cc6f0 tx=0x7f8d24004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72065f91b8c11369 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.117+0000 7f8d16600640 1 -- 172.21.15.62:0/6319979 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24016020 con 0x55c2fdb32830 2024-04-25T09:08:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.117+0000 7f8d16600640 1 -- 172.21.15.62:0/6319979 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400dab0 con 0x55c2fdb32830 2024-04-25T09:08:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.117+0000 7f8d16600640 1 -- 172.21.15.62:0/6319979 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400dea0 con 0x55c2fdb32830 2024-04-25T09:08:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 -- 172.21.15.62:0/6319979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 msgr2=0x55c2fdb32cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32cb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0cc6f0 tx=0x7f8d24004660 comp rx=0 tx=0).stop 2024-04-25T09:08:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 -- 172.21.15.62:0/6319979 shutdown_connections 2024-04-25T09:08:12.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32cb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb44440 0x55c2fdb08150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/6319979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb07d70 0x55c2fdb09ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 -- 172.21.15.62:0/6319979 >> 172.21.15.62:0/6319979 conn(0x55c2fdacfa30 msgr2=0x55c2fdb4d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 -- 172.21.15.62:0/6319979 shutdown_connections 2024-04-25T09:08:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.118+0000 7f8d33b8a740 1 -- 172.21.15.62:0/6319979 wait complete. 2024-04-25T09:08:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.119+0000 7f8d33b8a740 1 -- start start 2024-04-25T09:08:12.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.119+0000 7f8d33b8a740 1 Processor -- start 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 0x55c2fdb25160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb50340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2fda0a3a0 con 0x55c2fdb32830 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2fdb30380 con 0x55c2fdb44440 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2fdb256a0 con 0x55c2fdb07d70 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d33b8a740 10 client.0 _ll_register_callbacks cb 0x55c2fda27b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d29400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d29400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60376/0 (socket says 172.21.15.62:60376) 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d29400640 1 -- 172.21.15.62:0/2851673578 learned_addr learned my addr 172.21.15.62:0/2851673578 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d29e00640 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 0x55c2fdb25160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d29400640 1 -- 172.21.15.62:0/2851673578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 msgr2=0x55c2fdb25160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d29400640 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 0x55c2fdb25160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d29400640 1 -- 172.21.15.62:0/2851673578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 msgr2=0x55c2fdb50340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.120+0000 7f8d2a800640 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb50340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d29400640 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb50340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d29400640 1 -- 172.21.15.62:0/2851673578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2fd7803f0 con 0x55c2fdb32830 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d29400640 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32c30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c062780 tx=0x7f8d18006a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eea435a902fcd24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d0aa00640 1 -- 172.21.15.62:0/2851673578 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18014d40 con 0x55c2fdb32830 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d33b8a740 1 -- 172.21.15.62:0/2851673578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d2400dab0 con 0x55c2fdb32830 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d0aa00640 1 -- 172.21.15.62:0/2851673578 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8d1801f410 con 0x55c2fdb32830 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d0aa00640 1 -- 172.21.15.62:0/2851673578 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d18005990 con 0x55c2fdb32830 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.121+0000 7f8d33b8a740 10 client.6156 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d33b8a740 1 -- 172.21.15.62:0/2851673578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55c2fd7400f0 con 0x55c2fdb32830 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d0aa00640 1 -- 172.21.15.62:0/2851673578 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1802c070 con 0x55c2fdb32830 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d0aa00640 1 -- 172.21.15.62:0/2851673578 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d18013070 con 0x55c2fdb32830 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d33b8a740 10 client.6156 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d33b8a740 10 client.6156 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:12.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d33b8a740 20 client.6156 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:12.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55c2fdb61080 0x55c2fdb614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d33b8a740 4 client.6156 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:12.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d33b8a740 1 -- 172.21.15.62:0/2851673578 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2fda791b0 con 0x55c2fdb61080 2024-04-25T09:08:12.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.122+0000 7f8d29e00640 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55c2fdb61080 0x55c2fdb614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.123+0000 7f8d29e00640 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55c2fdb61080 0x55c2fdb614c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8919bcf5cd2497a3 server_cookie=b6402b3304aa5de0 in_seq=0 out_seq=0 2024-04-25T09:08:12.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.123+0000 7f8d0aa00640 10 client.6156 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:12.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.123+0000 7f8d0aa00640 1 -- 172.21.15.62:0/2851673578 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2fda791b0 con 0x55c2fdb61080 2024-04-25T09:08:12.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.123+0000 7f8d0aa00640 10 client.6156 handle_command_reply: tid=0 2024-04-25T09:08:12.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 client.6156 shutdown 2024-04-25T09:08:12.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 -- 172.21.15.62:0/2851673578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 msgr2=0x55c2fdb32c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32c30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c062780 tx=0x7f8d18006a20 comp rx=0 tx=0).stop 2024-04-25T09:08:12.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 -- 172.21.15.62:0/2851673578 shutdown_connections 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55c2fdb61080 0x55c2fdb614c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb32c30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb50340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/2851673578 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 0x55c2fdb25160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 -- 172.21.15.62:0/2851673578 >> 172.21.15.62:0/2851673578 conn(0x55c2fdacfa30 msgr2=0x55c2fdb30780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d0aa00640 0 client.6156 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 -- 172.21.15.62:0/2851673578 shutdown_connections 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 1 -- 172.21.15.62:0/2851673578 wait complete. 2024-04-25T09:08:12.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.124+0000 7f8d33b8a740 20 client.6156 trim_cache size 0 max 16384 2024-04-25T09:08:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.141+0000 7f8d33b8a740 1 Processor -- start 2024-04-25T09:08:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.141+0000 7f8d33b8a740 1 -- start start 2024-04-25T09:08:12.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 0x55c2fdb2f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb67710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb2fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1fc140 con 0x55c2fdb32830 2024-04-25T09:08:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1fb2a0 con 0x55c2fdb44440 2024-04-25T09:08:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1fa740 con 0x55c2fdb07d70 2024-04-25T09:08:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d29400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb67710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d29400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb67710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60392/0 (socket says 172.21.15.62:60392) 2024-04-25T09:08:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.142+0000 7f8d29400640 1 -- 172.21.15.62:0/833426978 learned_addr learned my addr 172.21.15.62:0/833426978 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.143+0000 7f8d29e00640 1 --2- 172.21.15.62:0/833426978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 0x55c2fdb2f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.143+0000 7f8d29400640 1 -- 172.21.15.62:0/833426978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 msgr2=0x55c2fdb2f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.143+0000 7f8d29400640 1 --2- 172.21.15.62:0/833426978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 0x55c2fdb2f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.143+0000 7f8d29400640 1 -- 172.21.15.62:0/833426978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 msgr2=0x55c2fdb2fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.143+0000 7f8d29400640 1 --2- 172.21.15.62:0/833426978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb2fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.143+0000 7f8d29400640 1 -- 172.21.15.62:0/833426978 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2fdb70d90 con 0x55c2fdb32830 2024-04-25T09:08:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.143+0000 7f8d29400640 1 --2- 172.21.15.62:0/833426978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb67710 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8d18008010 tx=0x7f8d1801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964f951bac50d3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.144+0000 7f8d15c00640 1 -- 172.21.15.62:0/833426978 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1801f410 con 0x55c2fdb32830 2024-04-25T09:08:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.144+0000 7f8d15c00640 1 -- 172.21.15.62:0/833426978 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1802c070 con 0x55c2fdb32830 2024-04-25T09:08:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.144+0000 7f8d15c00640 1 -- 172.21.15.62:0/833426978 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1801f7f0 con 0x55c2fdb32830 2024-04-25T09:08:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.144+0000 7f8d33b8a740 1 -- 172.21.15.62:0/833426978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 msgr2=0x55c2fdb67710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.144+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/833426978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb67710 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8d18008010 tx=0x7f8d1801d000 comp rx=0 tx=0).stop 2024-04-25T09:08:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.145+0000 7f8d33b8a740 1 -- 172.21.15.62:0/833426978 shutdown_connections 2024-04-25T09:08:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.145+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/833426978 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb32830 0x55c2fdb67710 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.145+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/833426978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb2fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.145+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/833426978 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb07d70 0x55c2fdb2f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.145+0000 7f8d33b8a740 1 -- 172.21.15.62:0/833426978 >> 172.21.15.62:0/833426978 conn(0x55c2fdacfa30 msgr2=0x55c2fdb47f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.145+0000 7f8d33b8a740 1 -- 172.21.15.62:0/833426978 shutdown_connections 2024-04-25T09:08:12.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.145+0000 7f8d33b8a740 1 -- 172.21.15.62:0/833426978 wait complete. 2024-04-25T09:08:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.146+0000 7f8d33b8a740 1 -- start start 2024-04-25T09:08:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.146+0000 7f8d33b8a740 1 Processor -- start 2024-04-25T09:08:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb07d70 0x55c2fdb06f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb32830 0x55c2fdb07480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d33b8a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1faca0 con 0x55c2fdb07d70 2024-04-25T09:08:12.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2fda28fb0 con 0x55c2fdb44440 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d33b8a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2fdb478d0 con 0x55c2fdb32830 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d33b8a740 10 client.0 _ll_register_callbacks cb 0x7f8d1800b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb07d70 0x55c2fdb06f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb07d70 0x55c2fdb06f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60398/0 (socket says 172.21.15.62:60398) 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.147+0000 7f8d29e00640 1 -- 172.21.15.62:0/1940558645 learned_addr learned my addr 172.21.15.62:0/1940558645 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d29e00640 1 -- 172.21.15.62:0/1940558645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb32830 msgr2=0x55c2fdb07480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d29e00640 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb32830 0x55c2fdb07480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d29e00640 1 -- 172.21.15.62:0/1940558645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 msgr2=0x55c2fdb4e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d29e00640 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d29e00640 1 -- 172.21.15.62:0/1940558645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2fdbbb640 con 0x55c2fdb07d70 2024-04-25T09:08:12.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d29e00640 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb07d70 0x55c2fdb06f40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c074a30 tx=0x7f8d1c0784c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee2a564c828dc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d0aa00640 1 -- 172.21.15.62:0/1940558645 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0a6060 con 0x55c2fdb07d70 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d0aa00640 1 -- 172.21.15.62:0/1940558645 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8d1c0768d0 con 0x55c2fdb07d70 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.148+0000 7f8d33b8a740 1 -- 172.21.15.62:0/1940558645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d1802c070 con 0x55c2fdb07d70 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d0aa00640 1 -- 172.21.15.62:0/1940558645 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d1c096050 con 0x55c2fdb07d70 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d33b8a740 10 client.6162 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d33b8a740 1 -- 172.21.15.62:0/1940558645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55c2fdb43b10 con 0x55c2fdb07d70 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d0aa00640 1 -- 172.21.15.62:0/1940558645 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1802c070 con 0x55c2fdb07d70 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d0aa00640 1 -- 172.21.15.62:0/1940558645 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d1c074c40 con 0x55c2fdb07d70 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d33b8a740 10 client.6162 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d33b8a740 10 client.6162 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d33b8a740 20 client.6162 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:12.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55c2fdb471c0 0x55c2fdb4eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d33b8a740 4 client.6162 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:08:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d33b8a740 1 -- 172.21.15.62:0/1940558645 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c2fdb36160 con 0x55c2fdb471c0 2024-04-25T09:08:12.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.149+0000 7f8d29400640 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55c2fdb471c0 0x55c2fdb4eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.150+0000 7f8d29400640 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55c2fdb471c0 0x55c2fdb4eea0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258d0d6a08a3c875 server_cookie=27743be0e23e1dcf in_seq=0 out_seq=0 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.150+0000 7f8d0aa00640 10 client.6162 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.150+0000 7f8d0aa00640 1 -- 172.21.15.62:0/1940558645 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c2fdb36160 con 0x55c2fdb471c0 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.150+0000 7f8d0aa00640 10 client.6162 handle_command_reply: tid=0 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.150+0000 7f8d33b8a740 1 client.6162 shutdown 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.150+0000 7f8d33b8a740 1 -- 172.21.15.62:0/1940558645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb07d70 msgr2=0x55c2fdb06f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.150+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb07d70 0x55c2fdb06f40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c074a30 tx=0x7f8d1c0784c0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 1 -- 172.21.15.62:0/1940558645 shutdown_connections 2024-04-25T09:08:12.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x55c2fdb471c0 0x55c2fdb4eea0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c2fdb44440 0x55c2fdb4e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c2fdb32830 0x55c2fdb07480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 1 --2- 172.21.15.62:0/1940558645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c2fdb07d70 0x55c2fdb06f40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d0aa00640 0 client.6162 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 1 -- 172.21.15.62:0/1940558645 >> 172.21.15.62:0/1940558645 conn(0x55c2fdacfa30 msgr2=0x55c2fdb2efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 1 -- 172.21.15.62:0/1940558645 shutdown_connections 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 1 -- 172.21.15.62:0/1940558645 wait complete. 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d33b8a740 20 client.6162 trim_cache size 0 max 16384 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:12.142 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-25T09:08:12.143 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:08:12.143 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:12.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.151+0000 7f8d2b200640 1 -- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d0c06b500 msgr2=0x7f8d0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d0c06b500 0x7f8d0c06d9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c1441d0 tx=0x7f8d18008040 comp rx=0 tx=0).stop 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 -- 172.21.15.62:0/3013174969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 msgr2=0x7f8d2c13aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d2c13aed0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c02ad30 tx=0x7f8d2c0cf4b0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 -- 172.21.15.62:0/3013174969 shutdown_connections 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d0c06b500 0x7f8d0c06d9c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d2c14bed0 0x7f8d2c14c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d2c1417a0 0x7f8d2c141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 --2- 172.21.15.62:0/3013174969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d10003a00 0x7f8d2c13aed0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 -- 172.21.15.62:0/3013174969 >> 172.21.15.62:0/3013174969 conn(0x7f8d2c066f20 msgr2=0x7f8d2c06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 -- 172.21.15.62:0/3013174969 shutdown_connections 2024-04-25T09:08:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.152+0000 7f8d2b200640 1 -- 172.21.15.62:0/3013174969 wait complete. 2024-04-25T09:08:12.158 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:08:12.158 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:12.158 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:08:12.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.365+0000 7f0272c00640 1 Processor -- start 2024-04-25T09:08:12.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.366+0000 7f0272c00640 1 -- start start 2024-04-25T09:08:12.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.367+0000 7f0272c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f026c0d2fb0 0x7f026c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.367+0000 7f0272c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f026c0d3960 0x7f026c0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.367+0000 7f0272c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026c0ce780 con 0x7f026c15dd50 2024-04-25T09:08:12.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.367+0000 7f0272c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026c0d42f0 con 0x7f026c0d2fb0 2024-04-25T09:08:12.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.367+0000 7f0272c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026c0d3440 con 0x7f026c0d3960 2024-04-25T09:08:12.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.367+0000 7f0270e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f026c0d2fb0 0x7f026c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.367+0000 7f0270e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f026c0d2fb0 0x7f026c0ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60412/0 (socket says 172.21.15.62:60412) 2024-04-25T09:08:12.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.367+0000 7f0270e00640 1 -- 172.21.15.62:0/3932042745 learned_addr learned my addr 172.21.15.62:0/3932042745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.368+0000 7f0270e00640 1 -- 172.21.15.62:0/3932042745 >> v2:172.21.15.121:3300/0 conn(0x7f026c0d3960 msgr2=0x7f026c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.368+0000 7f0270e00640 1 --2- 172.21.15.62:0/3932042745 >> v2:172.21.15.121:3300/0 conn(0x7f026c0d3960 0x7f026c0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.368+0000 7f0270e00640 1 -- 172.21.15.62:0/3932042745 >> v1:172.21.15.121:6789/0 conn(0x7f026c15dd50 legacy=0x7f026c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.368+0000 7f0270e00640 1 -- 172.21.15.62:0/3932042745 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c0ce1c0 con 0x7f026c0d2fb0 2024-04-25T09:08:12.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.369+0000 7f0270e00640 1 --2- 172.21.15.62:0/3932042745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d2fb0 0x7f026c0ce9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f026c0caa50 tx=0x7f025802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f435e0fba9b787d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.369+0000 7f026be00640 1 -- 172.21.15.62:0/3932042745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258037070 con 0x7f026c0d2fb0 2024-04-25T09:08:12.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.369+0000 7f026be00640 1 -- 172.21.15.62:0/3932042745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0258032440 con 0x7f026c0d2fb0 2024-04-25T09:08:12.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.369+0000 7f026be00640 1 -- 172.21.15.62:0/3932042745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258032810 con 0x7f026c0d2fb0 2024-04-25T09:08:12.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.370+0000 7f0272c00640 1 -- 172.21.15.62:0/3932042745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d2fb0 msgr2=0x7f026c0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.370+0000 7f0272c00640 1 --2- 172.21.15.62:0/3932042745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d2fb0 0x7f026c0ce9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f026c0caa50 tx=0x7f025802a230 comp rx=0 tx=0).stop 2024-04-25T09:08:12.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.370+0000 7f0272c00640 1 -- 172.21.15.62:0/3932042745 shutdown_connections 2024-04-25T09:08:12.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.370+0000 7f0272c00640 1 --2- 172.21.15.62:0/3932042745 >> v2:172.21.15.121:3300/0 conn(0x7f026c0d3960 0x7f026c0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.370+0000 7f0272c00640 1 --2- 172.21.15.62:0/3932042745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d2fb0 0x7f026c0ce9c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.370+0000 7f0272c00640 1 -- 172.21.15.62:0/3932042745 >> 172.21.15.62:0/3932042745 conn(0x7f026c06b000 msgr2=0x7f026c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.370+0000 7f0272c00640 1 -- 172.21.15.62:0/3932042745 shutdown_connections 2024-04-25T09:08:12.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.370+0000 7f0272c00640 1 -- 172.21.15.62:0/3932042745 wait complete. 2024-04-25T09:08:12.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.372+0000 7f0272c00640 1 Processor -- start 2024-04-25T09:08:12.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.372+0000 7f0272c00640 1 -- start start 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0272c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d3960 0x7f026c15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0272c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f026c15dd50 0x7f026c150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0272c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f026c151860 0x7f026c1601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0272c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0b6b00 con 0x7f026c0d3960 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0272c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0cef00 con 0x7f026c15dd50 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0272c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0b6c80 con 0x7f026c151860 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0271800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d3960 0x7f026c15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0271800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d3960 0x7f026c15eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60428/0 (socket says 172.21.15.62:60428) 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0271800640 1 -- 172.21.15.62:0/4098922088 learned_addr learned my addr 172.21.15.62:0/4098922088 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0271800640 1 -- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f026c151860 msgr2=0x7f026c1601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0271800640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f026c151860 0x7f026c1601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0271800640 1 -- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f026c15dd50 msgr2=0x7f026c150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0271800640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f026c15dd50 0x7f026c150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.373+0000 7f0271800640 1 -- 172.21.15.62:0/4098922088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c160af0 con 0x7f026c0d3960 2024-04-25T09:08:12.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.374+0000 7f0271800640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d3960 0x7f026c15eaf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f026000b3f0 tx=0x7f026000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.374+0000 7f0269600640 1 -- 172.21.15.62:0/4098922088 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260015020 con 0x7f026c0d3960 2024-04-25T09:08:12.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.374+0000 7f0269600640 1 -- 172.21.15.62:0/4098922088 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0260011830 con 0x7f026c0d3960 2024-04-25T09:08:12.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.374+0000 7f0272c00640 1 -- 172.21.15.62:0/4098922088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c160dd0 con 0x7f026c0d3960 2024-04-25T09:08:12.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.374+0000 7f0269600640 1 -- 172.21.15.62:0/4098922088 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260011c20 con 0x7f026c0d3960 2024-04-25T09:08:12.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.374+0000 7f0272c00640 1 -- 172.21.15.62:0/4098922088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c15e5d0 con 0x7f026c0d3960 2024-04-25T09:08:12.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.377+0000 7f0269600640 1 -- 172.21.15.62:0/4098922088 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0260018020 con 0x7f026c0d3960 2024-04-25T09:08:12.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.379+0000 7f0269600640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f024c06b4d0 0x7f024c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.379+0000 7f0269600640 1 -- 172.21.15.62:0/4098922088 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0260089f10 con 0x7f026c0d3960 2024-04-25T09:08:12.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.379+0000 7f0276f9f740 1 Processor -- start 2024-04-25T09:08:12.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.379+0000 7f0276f9f740 1 -- start start 2024-04-25T09:08:12.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.379+0000 7f0270e00640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f024c06b4d0 0x7f024c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 0x555c3ca653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca828b0 0x555c3ca82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca3acf0 0x555c3ca80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0276f9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3c99dd00 con 0x555c3ca3acf0 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0276f9f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3c946f50 con 0x555c3ca65010 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0276f9f740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c3c999da0 con 0x555c3ca828b0 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0272200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca3acf0 0x555c3ca80f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0272200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca3acf0 0x555c3ca80f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60430/0 (socket says 172.21.15.62:60430) 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.380+0000 7f0272200640 1 -- 172.21.15.62:0/721030143 learned_addr learned my addr 172.21.15.62:0/721030143 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0271800640 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 0x555c3ca653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0270e00640 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca828b0 0x555c3ca82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0272200640 1 -- 172.21.15.62:0/721030143 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca828b0 msgr2=0x555c3ca82e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0272200640 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca828b0 0x555c3ca82e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0272200640 1 -- 172.21.15.62:0/721030143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 msgr2=0x555c3ca653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0272200640 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 0x555c3ca653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0272200640 1 -- 172.21.15.62:0/721030143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c3c96a130 con 0x555c3ca3acf0 2024-04-25T09:08:12.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0270e00640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f024c06b4d0 0x7f024c06d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0258034000 tx=0x7f025802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f0272200640 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca3acf0 0x555c3ca80f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0264009d00 tx=0x7f026400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7200d27ddff50f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.381+0000 7f025e000640 1 -- 172.21.15.62:0/721030143 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026400bd40 con 0x555c3ca3acf0 2024-04-25T09:08:12.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f025e000640 1 -- 172.21.15.62:0/721030143 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264017070 con 0x555c3ca3acf0 2024-04-25T09:08:12.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f025e000640 1 -- 172.21.15.62:0/721030143 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264012670 con 0x555c3ca3acf0 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 -- 172.21.15.62:0/721030143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca3acf0 msgr2=0x555c3ca80f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca3acf0 0x555c3ca80f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0264009d00 tx=0x7f026400b420 comp rx=0 tx=0).stop 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 -- 172.21.15.62:0/721030143 shutdown_connections 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca3acf0 0x555c3ca80f70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca828b0 0x555c3ca82e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 --2- 172.21.15.62:0/721030143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 0x555c3ca653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 -- 172.21.15.62:0/721030143 >> 172.21.15.62:0/721030143 conn(0x555c3ca37620 msgr2=0x555c3ca7fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 -- 172.21.15.62:0/721030143 shutdown_connections 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.382+0000 7f0276f9f740 1 -- 172.21.15.62:0/721030143 wait complete. 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.383+0000 7f0276f9f740 1 -- start start 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.383+0000 7f0276f9f740 1 Processor -- start 2024-04-25T09:08:12.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.384+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca65010 0x555c3ca342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.384+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 0x555c3ca34820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.384+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca7ed20 0x555c3ca66fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.384+0000 7f0276f9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3ca3ba90 con 0x555c3ca65010 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.384+0000 7f0276f9f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3c93c530 con 0x555c3ca828b0 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.384+0000 7f0276f9f740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c3ca35550 con 0x555c3ca7ed20 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.384+0000 7f0276f9f740 10 client.0 _ll_register_callbacks cb 0x555c3c995430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca65010 0x555c3ca342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca65010 0x555c3ca342e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60442/0 (socket says 172.21.15.62:60442) 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 -- 172.21.15.62:0/1532010514 learned_addr learned my addr 172.21.15.62:0/1532010514 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 -- 172.21.15.62:0/1532010514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca7ed20 msgr2=0x555c3ca66fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca7ed20 0x555c3ca66fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 -- 172.21.15.62:0/1532010514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 msgr2=0x555c3ca34820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 0x555c3ca34820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 -- 172.21.15.62:0/1532010514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c3c964dd0 con 0x555c3ca65010 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0271800640 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca65010 0x555c3ca342e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f026000bdb0 tx=0x7f026005c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b30a3577fbb485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.385+0000 7f0252a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0252a00640 1 -- 172.21.15.62:0/1532010514 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260086060 con 0x555c3ca65010 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0252a00640 1 -- 172.21.15.62:0/1532010514 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0260054c60 con 0x555c3ca65010 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 1 -- 172.21.15.62:0/1532010514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0264017070 con 0x555c3ca65010 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0252a00640 1 -- 172.21.15.62:0/1532010514 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0260057a40 con 0x555c3ca65010 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 10 client.6174 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 1 -- 172.21.15.62:0/1532010514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x555c3c6b23f0 con 0x555c3ca65010 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0252a00640 1 -- 172.21.15.62:0/1532010514 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264017070 con 0x555c3ca65010 2024-04-25T09:08:12.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0252a00640 1 -- 172.21.15.62:0/1532010514 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0260057be0 con 0x555c3ca65010 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 10 client.6174 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 10 client.6174 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 20 client.6174 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555c3ca679b0 0x555c3ca773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 4 client.6174 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.386+0000 7f0276f9f740 1 -- 172.21.15.62:0/1532010514 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c3c9ab240 con 0x555c3ca679b0 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.387+0000 7f0270e00640 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555c3ca679b0 0x555c3ca773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.387+0000 7f0270e00640 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555c3ca679b0 0x555c3ca773a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5163d6419daa16f server_cookie=f62064e8b90434f4 in_seq=0 out_seq=0 2024-04-25T09:08:12.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.387+0000 7f0252a00640 10 client.6174 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0252a00640 1 -- 172.21.15.62:0/1532010514 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c3c9ab240 con 0x555c3ca679b0 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0252a00640 10 client.6174 handle_command_reply: tid=0 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 client.6174 shutdown 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 -- 172.21.15.62:0/1532010514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca65010 msgr2=0x555c3ca342e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca65010 0x555c3ca342e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f026000bdb0 tx=0x7f026005c800 comp rx=0 tx=0).stop 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 -- 172.21.15.62:0/1532010514 shutdown_connections 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555c3ca679b0 0x555c3ca773a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca7ed20 0x555c3ca66fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 0x555c3ca34820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 --2- 172.21.15.62:0/1532010514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca65010 0x555c3ca342e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0252a00640 0 client.6174 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:12.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.388+0000 7f0276f9f740 1 -- 172.21.15.62:0/1532010514 >> 172.21.15.62:0/1532010514 conn(0x555c3ca3acf0 msgr2=0x555c3ca833a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.389+0000 7f0276f9f740 1 -- 172.21.15.62:0/1532010514 shutdown_connections 2024-04-25T09:08:12.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.389+0000 7f0276f9f740 1 -- 172.21.15.62:0/1532010514 wait complete. 2024-04-25T09:08:12.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.389+0000 7f0276f9f740 20 client.6174 trim_cache size 0 max 16384 2024-04-25T09:08:12.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.402+0000 7f0276f9f740 1 Processor -- start 2024-04-25T09:08:12.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.402+0000 7f0276f9f740 1 -- start start 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 0x555c3ca77ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca7ed20 0x555c3caa31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca828b0 0x555c3ca3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0276f9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c2001c0 con 0x555c3ca828b0 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0276f9f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c1ff3f0 con 0x555c3ca65010 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0276f9f740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026c155750 con 0x555c3ca7ed20 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0272200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca828b0 0x555c3ca3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0272200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca828b0 0x555c3ca3a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60458/0 (socket says 172.21.15.62:60458) 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.403+0000 7f0272200640 1 -- 172.21.15.62:0/984994826 learned_addr learned my addr 172.21.15.62:0/984994826 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f0272200640 1 -- 172.21.15.62:0/984994826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca7ed20 msgr2=0x555c3caa31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f0272200640 1 --2- 172.21.15.62:0/984994826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca7ed20 0x555c3caa31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f0272200640 1 -- 172.21.15.62:0/984994826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 msgr2=0x555c3ca77ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f0272200640 1 --2- 172.21.15.62:0/984994826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 0x555c3ca77ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f0272200640 1 -- 172.21.15.62:0/984994826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c3c77c7d0 con 0x555c3ca828b0 2024-04-25T09:08:12.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f0272200640 1 --2- 172.21.15.62:0/984994826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca828b0 0x555c3ca3a370 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0264013000 tx=0x7f026400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99df2a50ad2c17cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f025d600640 1 -- 172.21.15.62:0/984994826 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264022020 con 0x555c3ca828b0 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f025d600640 1 -- 172.21.15.62:0/984994826 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264017040 con 0x555c3ca828b0 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.404+0000 7f025d600640 1 -- 172.21.15.62:0/984994826 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640128b0 con 0x555c3ca828b0 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 -- 172.21.15.62:0/984994826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca828b0 msgr2=0x555c3ca3a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 --2- 172.21.15.62:0/984994826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca828b0 0x555c3ca3a370 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0264013000 tx=0x7f026400e000 comp rx=0 tx=0).stop 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 -- 172.21.15.62:0/984994826 shutdown_connections 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 --2- 172.21.15.62:0/984994826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca828b0 0x555c3ca3a370 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 --2- 172.21.15.62:0/984994826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca7ed20 0x555c3caa31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 --2- 172.21.15.62:0/984994826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca65010 0x555c3ca77ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 -- 172.21.15.62:0/984994826 >> 172.21.15.62:0/984994826 conn(0x555c3ca3acf0 msgr2=0x555c3ca7f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 -- 172.21.15.62:0/984994826 shutdown_connections 2024-04-25T09:08:12.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.405+0000 7f0276f9f740 1 -- 172.21.15.62:0/984994826 wait complete. 2024-04-25T09:08:12.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.406+0000 7f0276f9f740 1 -- start start 2024-04-25T09:08:12.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.406+0000 7f0276f9f740 1 Processor -- start 2024-04-25T09:08:12.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca65010 0x555c3ca82d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca7ed20 0x555c3ca778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0276f9f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 0x555c3ca99610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0276f9f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c3ca67120 con 0x555c3ca7ed20 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0276f9f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0b3e90 con 0x555c3ca828b0 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0276f9f740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c3ca82440 con 0x555c3ca65010 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0276f9f740 10 client.0 _ll_register_callbacks cb 0x555c3ca35a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0270e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca7ed20 0x555c3ca778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0270e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca7ed20 0x555c3ca778b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60460/0 (socket says 172.21.15.62:60460) 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.407+0000 7f0270e00640 1 -- 172.21.15.62:0/3277255178 learned_addr learned my addr 172.21.15.62:0/3277255178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0272200640 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 0x555c3ca99610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0270e00640 1 -- 172.21.15.62:0/3277255178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca65010 msgr2=0x555c3ca82d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0270e00640 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca65010 0x555c3ca82d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0270e00640 1 -- 172.21.15.62:0/3277255178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 msgr2=0x555c3ca99610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0270e00640 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 0x555c3ca99610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0270e00640 1 -- 172.21.15.62:0/3277255178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c3ca80b30 con 0x555c3ca7ed20 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0270e00640 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca7ed20 0x555c3ca778b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f02580090b0 tx=0x7f0258004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2253ed1ede99ccd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0252a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0252a00640 1 -- 172.21.15.62:0/3277255178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258037040 con 0x555c3ca7ed20 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0252a00640 1 -- 172.21.15.62:0/3277255178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0258032bc0 con 0x555c3ca7ed20 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.408+0000 7f0276f9f740 1 -- 172.21.15.62:0/3277255178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0264017040 con 0x555c3ca7ed20 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0252a00640 1 -- 172.21.15.62:0/3277255178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f025805a050 con 0x555c3ca7ed20 2024-04-25T09:08:12.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0276f9f740 10 client.6180 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0276f9f740 1 -- 172.21.15.62:0/3277255178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x555c3ca83580 con 0x555c3ca7ed20 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0252a00640 1 -- 172.21.15.62:0/3277255178 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264017040 con 0x555c3ca7ed20 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0252a00640 1 -- 172.21.15.62:0/3277255178 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f025805d020 con 0x555c3ca7ed20 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0276f9f740 10 client.6180 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0276f9f740 10 client.6180 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0276f9f740 20 client.6180 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0276f9f740 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555c3ca65db0 0x555c3ca99b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0276f9f740 4 client.6180 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.409+0000 7f0276f9f740 1 -- 172.21.15.62:0/3277255178 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c3ca3c430 con 0x555c3ca65db0 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.410+0000 7f0271800640 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555c3ca65db0 0x555c3ca99b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.410+0000 7f0271800640 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555c3ca65db0 0x555c3ca99b50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1919315725b9a1c server_cookie=f5bcc684a22afe00 in_seq=0 out_seq=0 2024-04-25T09:08:12.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.410+0000 7f0252a00640 10 client.6180 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.410+0000 7f0252a00640 1 -- 172.21.15.62:0/3277255178 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x555c3ca3c430 con 0x555c3ca65db0 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.410+0000 7f0252a00640 10 client.6180 handle_command_reply: tid=0 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.410+0000 7f0276f9f740 1 client.6180 shutdown 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.410+0000 7f0276f9f740 1 -- 172.21.15.62:0/3277255178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca7ed20 msgr2=0x555c3ca778b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.410+0000 7f0276f9f740 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca7ed20 0x555c3ca778b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f02580090b0 tx=0x7f0258004170 comp rx=0 tx=0).stop 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 1 -- 172.21.15.62:0/3277255178 shutdown_connections 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x555c3ca65db0 0x555c3ca99b50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555c3ca7ed20 0x555c3ca778b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555c3ca828b0 0x555c3ca99610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 1 --2- 172.21.15.62:0/3277255178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555c3ca65010 0x555c3ca82d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0252a00640 0 client.6180 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 1 -- 172.21.15.62:0/3277255178 >> 172.21.15.62:0/3277255178 conn(0x555c3ca3acf0 msgr2=0x555c3ca3b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 1 -- 172.21.15.62:0/3277255178 shutdown_connections 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 1 -- 172.21.15.62:0/3277255178 wait complete. 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0276f9f740 20 client.6180 trim_cache size 0 max 16384 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi062.stdout: "damage_type": "backtrace", 2024-04-25T09:08:12.404 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 2770711430, 2024-04-25T09:08:12.405 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 1099511630190, 2024-04-25T09:08:12.405 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T09:08:12.405 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:12.405 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0272c00640 1 -- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f024c06b4d0 msgr2=0x7f024c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0272c00640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f024c06b4d0 0x7f024c06d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0258034000 tx=0x7f025802c000 comp rx=0 tx=0).stop 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0272c00640 1 -- 172.21.15.62:0/4098922088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d3960 msgr2=0x7f026c15eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.411+0000 7f0272c00640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d3960 0x7f026c15eaf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f026000b3f0 tx=0x7f026000b8b0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.412+0000 7f0272c00640 1 -- 172.21.15.62:0/4098922088 shutdown_connections 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.412+0000 7f0272c00640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f024c06b4d0 0x7f024c06d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.412+0000 7f0272c00640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f026c151860 0x7f026c1601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.412+0000 7f0272c00640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f026c15dd50 0x7f026c150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.412+0000 7f0272c00640 1 --2- 172.21.15.62:0/4098922088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f026c0d3960 0x7f026c15eaf0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.412+0000 7f0272c00640 1 -- 172.21.15.62:0/4098922088 >> 172.21.15.62:0/4098922088 conn(0x7f026c06b000 msgr2=0x7f026c0b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.412+0000 7f0272c00640 1 -- 172.21.15.62:0/4098922088 shutdown_connections 2024-04-25T09:08:12.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.412+0000 7f0272c00640 1 -- 172.21.15.62:0/4098922088 wait complete. 2024-04-25T09:08:12.417 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:12.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.632+0000 7f0b05600640 1 Processor -- start 2024-04-25T09:08:12.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.632+0000 7f0b05600640 1 -- start start 2024-04-25T09:08:12.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.633+0000 7f0b05600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0b0006d900 0x7f0b0015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.633+0000 7f0b05600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b000cee40 0x7f0b000b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.633+0000 7f0b05600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b000cf2b0 con 0x7f0b00144c10 2024-04-25T09:08:12.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.633+0000 7f0b05600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b00144ff0 con 0x7f0b0006d900 2024-04-25T09:08:12.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.633+0000 7f0b05600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00160340 con 0x7f0b000cee40 2024-04-25T09:08:12.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.634+0000 7f0affe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0b00144c10 0x7f0b0006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53278/0 (socket says 172.21.15.62:53278) 2024-04-25T09:08:12.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.634+0000 7f0affe00640 1 -- 172.21.15.62:0/1613192397 learned_addr learned my addr 172.21.15.62:0/1613192397 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.634+0000 7f0af7400640 1 --2- 172.21.15.62:0/1613192397 >> v2:172.21.15.121:3300/0 conn(0x7f0b0006d900 0x7f0b0015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.634+0000 7f0b04c00640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b000cee40 0x7f0b000b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637809031 0 0) 0x7f0b000cf2b0 con 0x7f0b00144c10 2024-04-25T09:08:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae4003520 con 0x7f0b00144c10 2024-04-25T09:08:12.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515015524 0 0) 0x7f0ae4003520 con 0x7f0b00144c10 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b000cee40 msgr2=0x7f0b000b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b000cee40 0x7f0b000b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 >> v2:172.21.15.121:3300/0 conn(0x7f0b0006d900 msgr2=0x7f0b0015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 --2- 172.21.15.62:0/1613192397 >> v2:172.21.15.121:3300/0 conn(0x7f0b0006d900 0x7f0b0015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0015faa0 con 0x7f0b00144c10 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f0af00019f0 con 0x7f0b00144c10 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 >> v1:172.21.15.62:6789/0 conn(0x7f0b00144c10 legacy=0x7f0b0006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b000cee40 0x7f0ae40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0ae4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ae4004d40 0x7f0ae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00144ff0 con 0x7f0ae4003f80 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00160340 con 0x7f0ae4004d40 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.635+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae400ba70 con 0x7f0b000cee40 2024-04-25T09:08:12.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.636+0000 7f0af7400640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0ae4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.636+0000 7f0af7400640 1 -- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b000cee40 msgr2=0x7f0ae40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.636+0000 7f0af7400640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b000cee40 0x7f0ae40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.636+0000 7f0af7400640 1 -- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ae4004d40 msgr2=0x7f0ae40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:12.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.636+0000 7f0af7400640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ae4004d40 0x7f0ae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.636+0000 7f0af7400640 1 -- 172.21.15.62:0/1613192397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae4003520 con 0x7f0ae4003f80 2024-04-25T09:08:12.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.637+0000 7f0af7400640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0ae4004800 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009ac0 tx=0x7f0ae4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79a1fcdf0d76ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.637+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8036070 con 0x7f0ae4003f80 2024-04-25T09:08:12.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.637+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae8031400 con 0x7f0ae4003f80 2024-04-25T09:08:12.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.637+0000 7f0aff400640 1 -- 172.21.15.62:0/1613192397 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8009690 con 0x7f0ae4003f80 2024-04-25T09:08:12.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.638+0000 7f0b05600640 1 -- 172.21.15.62:0/1613192397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 msgr2=0x7f0ae4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.638+0000 7f0b05600640 1 --2- 172.21.15.62:0/1613192397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0ae4004800 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009ac0 tx=0x7f0ae4005830 comp rx=0 tx=0).stop 2024-04-25T09:08:12.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.638+0000 7f0b04c00640 1 -- 172.21.15.62:0/1613192397 reap_dead start 2024-04-25T09:08:12.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.638+0000 7f0b05600640 1 -- 172.21.15.62:0/1613192397 shutdown_connections 2024-04-25T09:08:12.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.638+0000 7f0b05600640 1 -- 172.21.15.62:0/1613192397 >> 172.21.15.62:0/1613192397 conn(0x7f0b000b14d0 msgr2=0x7f0b000b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.638+0000 7f0b05600640 1 -- 172.21.15.62:0/1613192397 shutdown_connections 2024-04-25T09:08:12.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.638+0000 7f0b05600640 1 -- 172.21.15.62:0/1613192397 wait complete. 2024-04-25T09:08:12.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.640+0000 7f0b05600640 1 Processor -- start 2024-04-25T09:08:12.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.640+0000 7f0b05600640 1 -- start start 2024-04-25T09:08:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0b05600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0b001f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0b05600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b001f5a70 0x7f0b001411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0b05600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b001416f0 0x7f0b000cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0b05600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00160060 con 0x7f0ae4003f80 2024-04-25T09:08:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0b05600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000b41c0 con 0x7f0b001f5a70 2024-04-25T09:08:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0b05600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000b4650 con 0x7f0b001416f0 2024-04-25T09:08:12.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0affe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0b001f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0affe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0b001f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60486/0 (socket says 172.21.15.62:60486) 2024-04-25T09:08:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.641+0000 7f0affe00640 1 -- 172.21.15.62:0/1258649649 learned_addr learned my addr 172.21.15.62:0/1258649649 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0affe00640 1 -- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b001416f0 msgr2=0x7f0b000cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0affe00640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b001416f0 0x7f0b000cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0affe00640 1 -- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b001f5a70 msgr2=0x7f0b001411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0affe00640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b001f5a70 0x7f0b001411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0affe00640 1 -- 172.21.15.62:0/1258649649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b001427c0 con 0x7f0ae4003f80 2024-04-25T09:08:12.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0affe00640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0b001f5530 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0af000caa0 tx=0x7f0af000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0afcc00640 1 -- 172.21.15.62:0/1258649649 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0018070 con 0x7f0ae4003f80 2024-04-25T09:08:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0afcc00640 1 -- 172.21.15.62:0/1258649649 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af000a710 con 0x7f0ae4003f80 2024-04-25T09:08:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0afcc00640 1 -- 172.21.15.62:0/1258649649 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af000ab40 con 0x7f0ae4003f80 2024-04-25T09:08:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.642+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b00141f20 con 0x7f0ae4003f80 2024-04-25T09:08:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.643+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b001424e0 con 0x7f0ae4003f80 2024-04-25T09:08:12.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.644+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0004ef20 con 0x7f0ae4003f80 2024-04-25T09:08:12.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.645+0000 7f0afcc00640 1 -- 172.21.15.62:0/1258649649 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0af000ace0 con 0x7f0ae4003f80 2024-04-25T09:08:12.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.646+0000 7f0afcc00640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0adc06b4d0 0x7f0adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:12.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.646+0000 7f0afcc00640 1 -- 172.21.15.62:0/1258649649 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0af0089b80 con 0x7f0ae4003f80 2024-04-25T09:08:12.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.646+0000 7f0af7400640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0adc06b4d0 0x7f0adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:12.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.647+0000 7f0af7400640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0adc06b4d0 0x7f0adc06d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8033000 tx=0x7f0ae802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:12.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.651+0000 7f0afcc00640 1 -- 172.21.15.62:0/1258649649 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0af0058a20 con 0x7f0ae4003f80 2024-04-25T09:08:12.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.895+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b000cec20 con 0x7f0ae4003f80 2024-04-25T09:08:12.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.896+0000 7f0afcc00640 1 -- 172.21.15.62:0/1258649649 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f0af0058320 con 0x7f0ae4003f80 2024-04-25T09:08:12.884 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:12.884 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:08:06.031370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:12.885 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0adc06b4d0 msgr2=0x7f0adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0adc06b4d0 0x7f0adc06d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8033000 tx=0x7f0ae802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 msgr2=0x7f0b001f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0b001f5530 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0af000caa0 tx=0x7f0af000cf60 comp rx=0 tx=0).stop 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 shutdown_connections 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0adc06b4d0 0x7f0adc06d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0b001416f0 0x7f0b000cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b001f5a70 0x7f0b001411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 --2- 172.21.15.62:0/1258649649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae4003f80 0x7f0b001f5530 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 >> 172.21.15.62:0/1258649649 conn(0x7f0b000b14d0 msgr2=0x7f0b0015be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 shutdown_connections 2024-04-25T09:08:12.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:12.899+0000 7f0b05600640 1 -- 172.21.15.62:0/1258649649 wait complete. 2024-04-25T09:08:12.907 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage rm 2770711430 2024-04-25T09:08:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.136+0000 7fe11d600640 1 Processor -- start 2024-04-25T09:08:13.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.136+0000 7fe11d600640 1 -- start start 2024-04-25T09:08:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.137+0000 7fe11d600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe11815dab0 0x7fe1180d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.137+0000 7fe11d600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe1180b5b50 0x7fe1180cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.137+0000 7fe11d600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1180d36d0 con 0x7fe11815de90 2024-04-25T09:08:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.138+0000 7fe11d600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1180b6ab0 con 0x7fe1180b5b50 2024-04-25T09:08:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.138+0000 7fe11d600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe118161830 con 0x7fe11815dab0 2024-04-25T09:08:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.138+0000 7fe117e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe11815dab0 0x7fe1180d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.138+0000 7fe117e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe11815dab0 0x7fe1180d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:33596/0 (socket says 172.21.15.62:33596) 2024-04-25T09:08:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.138+0000 7fe117e00640 1 -- 172.21.15.62:0/3663773112 learned_addr learned my addr 172.21.15.62:0/3663773112 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:13.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.138+0000 7fe11cc00640 1 --2- 172.21.15.62:0/3663773112 >> v2:172.21.15.121:3300/0 conn(0x7fe1180b5b50 0x7fe1180cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.139+0000 7fe11cc00640 1 -- 172.21.15.62:0/3663773112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe11815dab0 msgr2=0x7fe1180d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.139+0000 7fe11cc00640 1 --2- 172.21.15.62:0/3663773112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe11815dab0 0x7fe1180d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.139+0000 7fe11cc00640 1 -- 172.21.15.62:0/3663773112 >> v1:172.21.15.121:6789/0 conn(0x7fe11815de90 legacy=0x7fe1180b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.139+0000 7fe11cc00640 1 -- 172.21.15.62:0/3663773112 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe118161b10 con 0x7fe1180b5b50 2024-04-25T09:08:13.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.139+0000 7fe117e00640 1 --2- 172.21.15.62:0/3663773112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe11815dab0 0x7fe1180d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:08:13.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.141+0000 7fe11cc00640 1 --2- 172.21.15.62:0/3663773112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1180b5b50 0x7fe1180cdf80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe1100097a0 tx=0x7fe11002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf9a2b27e6dc1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.141+0000 7fe116a00640 1 -- 172.21.15.62:0/3663773112 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110037070 con 0x7fe1180b5b50 2024-04-25T09:08:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe116a00640 1 -- 172.21.15.62:0/3663773112 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe110032440 con 0x7fe1180b5b50 2024-04-25T09:08:13.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe116a00640 1 -- 172.21.15.62:0/3663773112 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110032810 con 0x7fe1180b5b50 2024-04-25T09:08:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe11d600640 1 -- 172.21.15.62:0/3663773112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1180b5b50 msgr2=0x7fe1180cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe11d600640 1 --2- 172.21.15.62:0/3663773112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1180b5b50 0x7fe1180cdf80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe1100097a0 tx=0x7fe11002a180 comp rx=0 tx=0).stop 2024-04-25T09:08:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe11d600640 1 -- 172.21.15.62:0/3663773112 shutdown_connections 2024-04-25T09:08:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe11d600640 1 --2- 172.21.15.62:0/3663773112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1180b5b50 0x7fe1180cdf80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe11d600640 1 --2- 172.21.15.62:0/3663773112 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe11815dab0 0x7fe1180d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe11d600640 1 -- 172.21.15.62:0/3663773112 >> 172.21.15.62:0/3663773112 conn(0x7fe1180d39c0 msgr2=0x7fe1180d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe11d600640 1 -- 172.21.15.62:0/3663773112 shutdown_connections 2024-04-25T09:08:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.142+0000 7fe11d600640 1 -- 172.21.15.62:0/3663773112 wait complete. 2024-04-25T09:08:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.144+0000 7fe11d600640 1 Processor -- start 2024-04-25T09:08:13.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.144+0000 7fe11d600640 1 -- start start 2024-04-25T09:08:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.145+0000 7fe11d600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1180b5b50 0x7fe1181f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.145+0000 7fe11d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11815dab0 0x7fe11815f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.145+0000 7fe11d600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe11815de90 0x7fe1181f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.145+0000 7fe11d600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180d2700 con 0x7fe11815dab0 2024-04-25T09:08:13.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.145+0000 7fe11d600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181620d0 con 0x7fe11815de90 2024-04-25T09:08:13.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.145+0000 7fe11d600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180d2880 con 0x7fe1180b5b50 2024-04-25T09:08:13.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11815dab0 0x7fe11815f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11815dab0 0x7fe11815f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60494/0 (socket says 172.21.15.62:60494) 2024-04-25T09:08:13.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117400640 1 -- 172.21.15.62:0/3548876508 learned_addr learned my addr 172.21.15.62:0/3548876508 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117e00640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1180b5b50 0x7fe1181f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe11cc00640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe11815de90 0x7fe1181f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117400640 1 -- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1180b5b50 msgr2=0x7fe1181f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117400640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1180b5b50 0x7fe1181f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117400640 1 -- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe11815de90 msgr2=0x7fe1181f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117400640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe11815de90 0x7fe1181f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.146+0000 7fe117400640 1 -- 172.21.15.62:0/3548876508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1181f6dd0 con 0x7fe11815dab0 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.147+0000 7fe117400640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11815dab0 0x7fe11815f890 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c730 tx=0x7fe10000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.147+0000 7fe10fe00640 1 -- 172.21.15.62:0/3548876508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100016020 con 0x7fe11815dab0 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.147+0000 7fe10fe00640 1 -- 172.21.15.62:0/3548876508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100006d50 con 0x7fe11815dab0 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.147+0000 7fe10fe00640 1 -- 172.21.15.62:0/3548876508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100003bb0 con 0x7fe11815dab0 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.147+0000 7fe11d600640 1 -- 172.21.15.62:0/3548876508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1181602c0 con 0x7fe11815dab0 2024-04-25T09:08:13.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.147+0000 7fe11d600640 1 -- 172.21.15.62:0/3548876508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe118160880 con 0x7fe11815dab0 2024-04-25T09:08:13.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.150+0000 7fe10fe00640 1 -- 172.21.15.62:0/3548876508 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe10000a850 con 0x7fe11815dab0 2024-04-25T09:08:13.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.150+0000 7fe10fe00640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0f406b4d0 0x7fe0f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.151+0000 7fe10fe00640 1 -- 172.21.15.62:0/3548876508 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe100089eb0 con 0x7fe11815dab0 2024-04-25T09:08:13.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.151+0000 7fe1219f7740 1 Processor -- start 2024-04-25T09:08:13.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.151+0000 7fe1219f7740 1 -- start start 2024-04-25T09:08:13.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.151+0000 7fe117e00640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0f406b4d0 0x7fe0f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 0x5640bcc30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2b700 0x5640bcc2f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc2fbc0 0x5640bcc30000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe1219f7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640bcb94820 con 0x5640bcc2fbc0 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe1219f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640bcb3dd00 con 0x5640bcc2b700 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe1219f7740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640bcb908c0 con 0x5640bcc30a80 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe117e00640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0f406b4d0 0x7fe0f406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe108009b70 tx=0x7fe108008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe11cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc2fbc0 0x5640bcc30000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe11cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc2fbc0 0x5640bcc30000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60504/0 (socket says 172.21.15.62:60504) 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.152+0000 7fe11cc00640 1 -- 172.21.15.62:0/2797556262 learned_addr learned my addr 172.21.15.62:0/2797556262 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe117e00640 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 0x5640bcc30e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe117400640 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2b700 0x5640bcc2f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe11cc00640 1 -- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 msgr2=0x5640bcc30e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe11cc00640 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 0x5640bcc30e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe11cc00640 1 -- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2b700 msgr2=0x5640bcc2f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe11cc00640 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2b700 0x5640bcc2f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe11cc00640 1 -- 172.21.15.62:0/2797556262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640bc975710 con 0x5640bcc2fbc0 2024-04-25T09:08:13.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe117400640 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2b700 0x5640bcc2f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:13.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.153+0000 7fe11cc00640 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc2fbc0 0x5640bcc30000 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe1100097a0 tx=0x7fe11002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c5f7fd43227beb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe107e00640 1 -- 172.21.15.62:0/2797556262 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110037040 con 0x5640bcc2fbc0 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe107e00640 1 -- 172.21.15.62:0/2797556262 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11003e660 con 0x5640bcc2fbc0 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe107e00640 1 -- 172.21.15.62:0/2797556262 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11003ea10 con 0x5640bcc2fbc0 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe1219f7740 1 -- 172.21.15.62:0/2797556262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc2fbc0 msgr2=0x5640bcc30000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc2fbc0 0x5640bcc30000 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe1100097a0 tx=0x7fe11002c000 comp rx=0 tx=0).stop 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe1219f7740 1 -- 172.21.15.62:0/2797556262 shutdown_connections 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc2fbc0 0x5640bcc30000 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2b700 0x5640bcc2f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2797556262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 0x5640bcc30e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe1219f7740 1 -- 172.21.15.62:0/2797556262 >> 172.21.15.62:0/2797556262 conn(0x5640bcc2db80 msgr2=0x5640bcc76640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.154+0000 7fe1219f7740 1 -- 172.21.15.62:0/2797556262 shutdown_connections 2024-04-25T09:08:13.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.155+0000 7fe1219f7740 1 -- 172.21.15.62:0/2797556262 wait complete. 2024-04-25T09:08:13.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.155+0000 7fe1219f7740 1 -- start start 2024-04-25T09:08:13.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.155+0000 7fe1219f7740 1 Processor -- start 2024-04-25T09:08:13.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 0x5640bcc51c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc30a80 0x5640bcc42b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc54cc0 0x5640bcc79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe1219f7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640bcc5c610 con 0x5640bcc30a80 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe1219f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640bcb33360 con 0x5640bcc2fbc0 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe1219f7740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640bcc5ba90 con 0x5640bcc54cc0 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe1219f7740 10 client.0 _ll_register_callbacks cb 0x5640bcb3fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc30a80 0x5640bcc42b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc30a80 0x5640bcc42b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60512/0 (socket says 172.21.15.62:60512) 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117400640 1 -- 172.21.15.62:0/2404509542 learned_addr learned my addr 172.21.15.62:0/2404509542 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117e00640 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 0x5640bcc51c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117400640 1 -- 172.21.15.62:0/2404509542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc54cc0 msgr2=0x5640bcc79a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117400640 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc54cc0 0x5640bcc79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117400640 1 -- 172.21.15.62:0/2404509542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 msgr2=0x5640bcc51c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117400640 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 0x5640bcc51c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.157+0000 7fe117400640 1 -- 172.21.15.62:0/2404509542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640bc9d5ea0 con 0x5640bcc30a80 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe117400640 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc30a80 0x5640bcc42b80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe100060810 tx=0x7fe1000193d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9233e127cf38ef13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe104c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe104c00640 1 -- 172.21.15.62:0/2404509542 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100086060 con 0x5640bcc30a80 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe104c00640 1 -- 172.21.15.62:0/2404509542 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe10000ab40 con 0x5640bcc30a80 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe1219f7740 1 -- 172.21.15.62:0/2404509542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe11003e660 con 0x5640bcc30a80 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe104c00640 1 -- 172.21.15.62:0/2404509542 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe10005c780 con 0x5640bcc30a80 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe1219f7740 10 client.6195 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe1219f7740 1 -- 172.21.15.62:0/2404509542 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5640bc869000 con 0x5640bcc30a80 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.158+0000 7fe104c00640 1 -- 172.21.15.62:0/2404509542 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11003e660 con 0x5640bcc30a80 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.159+0000 7fe104c00640 1 -- 172.21.15.62:0/2404509542 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe100057a80 con 0x5640bcc30a80 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.159+0000 7fe1219f7740 10 client.6195 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.159+0000 7fe1219f7740 10 client.6195 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.159+0000 7fe1219f7740 20 client.6195 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:13.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.159+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5640bcc7b2e0 0x5640bcc7b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.159+0000 7fe1219f7740 4 client.6195 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:13.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.159+0000 7fe1219f7740 1 -- 172.21.15.62:0/2404509542 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640bcba1d60 con 0x5640bcc7b2e0 2024-04-25T09:08:13.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.159+0000 7fe117e00640 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5640bcc7b2e0 0x5640bcc7b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.160+0000 7fe117e00640 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5640bcc7b2e0 0x5640bcc7b6c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e48e99df0a0b8a server_cookie=48bd379ada27735e in_seq=0 out_seq=0 2024-04-25T09:08:13.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.160+0000 7fe104c00640 10 client.6195 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:13.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.160+0000 7fe104c00640 1 -- 172.21.15.62:0/2404509542 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640bcba1d60 con 0x5640bcc7b2e0 2024-04-25T09:08:13.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.160+0000 7fe104c00640 10 client.6195 handle_command_reply: tid=0 2024-04-25T09:08:13.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.160+0000 7fe1219f7740 1 client.6195 shutdown 2024-04-25T09:08:13.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 -- 172.21.15.62:0/2404509542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc30a80 msgr2=0x5640bcc42b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc30a80 0x5640bcc42b80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe100060810 tx=0x7fe1000193d0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 -- 172.21.15.62:0/2404509542 shutdown_connections 2024-04-25T09:08:13.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5640bcc7b2e0 0x5640bcc7b6c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc30a80 0x5640bcc42b80 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc54cc0 0x5640bcc79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 --2- 172.21.15.62:0/2404509542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 0x5640bcc51c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe104c00640 0 client.6195 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:13.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 -- 172.21.15.62:0/2404509542 >> 172.21.15.62:0/2404509542 conn(0x5640bcc2b700 msgr2=0x5640bcc2ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 -- 172.21.15.62:0/2404509542 shutdown_connections 2024-04-25T09:08:13.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 1 -- 172.21.15.62:0/2404509542 wait complete. 2024-04-25T09:08:13.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.161+0000 7fe1219f7740 20 client.6195 trim_cache size 0 max 16384 2024-04-25T09:08:13.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.174+0000 7fe1219f7740 1 Processor -- start 2024-04-25T09:08:13.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.174+0000 7fe1219f7740 1 -- start start 2024-04-25T09:08:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.175+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 0x5640bcc5ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.175+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 0x5640bcc5bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.175+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc85d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.175+0000 7fe1219f7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180cd3c0 con 0x5640bcc54cc0 2024-04-25T09:08:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.175+0000 7fe1219f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180cc7c0 con 0x5640bcc2fbc0 2024-04-25T09:08:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.175+0000 7fe1219f7740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180ccdc0 con 0x5640bcc30a80 2024-04-25T09:08:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.175+0000 7fe11cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc85d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe11cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc85d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60516/0 (socket says 172.21.15.62:60516) 2024-04-25T09:08:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe11cc00640 1 -- 172.21.15.62:0/4015497965 learned_addr learned my addr 172.21.15.62:0/4015497965 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe11cc00640 1 -- 172.21.15.62:0/4015497965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 msgr2=0x5640bcc5bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe11cc00640 1 --2- 172.21.15.62:0/4015497965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 0x5640bcc5bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe11cc00640 1 -- 172.21.15.62:0/4015497965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 msgr2=0x5640bcc5ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe11cc00640 1 --2- 172.21.15.62:0/4015497965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 0x5640bcc5ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe11cc00640 1 -- 172.21.15.62:0/4015497965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640bcc2f9f0 con 0x5640bcc54cc0 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe11cc00640 1 --2- 172.21.15.62:0/4015497965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc85d80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe11002a770 tx=0x7fe11002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5fb344e2678545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe107400640 1 -- 172.21.15.62:0/4015497965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110037070 con 0x5640bcc54cc0 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe107400640 1 -- 172.21.15.62:0/4015497965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe110004250 con 0x5640bcc54cc0 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.176+0000 7fe107400640 1 -- 172.21.15.62:0/4015497965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110003610 con 0x5640bcc54cc0 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 -- 172.21.15.62:0/4015497965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 msgr2=0x5640bcc85d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 --2- 172.21.15.62:0/4015497965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc85d80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe11002a770 tx=0x7fe11002c070 comp rx=0 tx=0).stop 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 -- 172.21.15.62:0/4015497965 shutdown_connections 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 --2- 172.21.15.62:0/4015497965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc85d80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 --2- 172.21.15.62:0/4015497965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc30a80 0x5640bcc5bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 --2- 172.21.15.62:0/4015497965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc2fbc0 0x5640bcc5ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 -- 172.21.15.62:0/4015497965 >> 172.21.15.62:0/4015497965 conn(0x5640bcc2b700 msgr2=0x5640bcc7b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 -- 172.21.15.62:0/4015497965 shutdown_connections 2024-04-25T09:08:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.177+0000 7fe1219f7740 1 -- 172.21.15.62:0/4015497965 wait complete. 2024-04-25T09:08:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.178+0000 7fe1219f7740 1 -- start start 2024-04-25T09:08:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.178+0000 7fe1219f7740 1 Processor -- start 2024-04-25T09:08:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.179+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc2fbc0 0x5640bcc503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.179+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc30a80 0x5640bcce1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.179+0000 7fe1219f7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.179+0000 7fe1219f7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640bcc58690 con 0x5640bcc54cc0 2024-04-25T09:08:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.179+0000 7fe1219f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640bcc56e10 con 0x5640bcc30a80 2024-04-25T09:08:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.179+0000 7fe1219f7740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640bcc7e860 con 0x5640bcc2fbc0 2024-04-25T09:08:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe1219f7740 10 client.0 _ll_register_callbacks cb 0x7fe110035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe11cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe11cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc850e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60520/0 (socket says 172.21.15.62:60520) 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe11cc00640 1 -- 172.21.15.62:0/1181247847 learned_addr learned my addr 172.21.15.62:0/1181247847 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe117e00640 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc2fbc0 0x5640bcc503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe11cc00640 1 -- 172.21.15.62:0/1181247847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc2fbc0 msgr2=0x5640bcc503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe11cc00640 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc2fbc0 0x5640bcc503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe11cc00640 1 -- 172.21.15.62:0/1181247847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc30a80 msgr2=0x5640bcce1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe11cc00640 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc30a80 0x5640bcce1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.180+0000 7fe11cc00640 1 -- 172.21.15.62:0/1181247847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640bcc6af30 con 0x5640bcc54cc0 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe11cc00640 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc850e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1a59528556ceb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe104c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe104c00640 1 -- 172.21.15.62:0/1181247847 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110037040 con 0x5640bcc54cc0 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe1219f7740 1 -- 172.21.15.62:0/1181247847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe110004250 con 0x5640bcc54cc0 2024-04-25T09:08:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe104c00640 1 -- 172.21.15.62:0/1181247847 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe110003be0 con 0x5640bcc54cc0 2024-04-25T09:08:13.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe104c00640 1 -- 172.21.15.62:0/1181247847 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe110003de0 con 0x5640bcc54cc0 2024-04-25T09:08:13.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe1219f7740 10 client.6201 fetch_fsmap learned FSMap version 28 2024-04-25T09:08:13.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe1219f7740 1 -- 172.21.15.62:0/1181247847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5640bcc53810 con 0x5640bcc54cc0 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe104c00640 1 -- 172.21.15.62:0/1181247847 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe110004250 con 0x5640bcc54cc0 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.181+0000 7fe104c00640 1 -- 172.21.15.62:0/1181247847 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe11003e670 con 0x5640bcc54cc0 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe1219f7740 10 client.6201 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe1219f7740 10 client.6201 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe1219f7740 20 client.6201 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe1219f7740 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5640bcc41100 0x5640bcc414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe1219f7740 4 client.6201 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2770711430}] 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe1219f7740 1 -- 172.21.15.62:0/1181247847 --> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2770711430}) v1 -- 0x5640bcc560e0 con 0x5640bcc41100 2024-04-25T09:08:13.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe117e00640 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5640bcc41100 0x5640bcc414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe117e00640 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5640bcc41100 0x5640bcc414e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d2eaafe30ffc3c8 server_cookie=f197141624f6da4e in_seq=0 out_seq=0 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.182+0000 7fe104c00640 10 client.6201 ms_handle_connect on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe104c00640 1 -- 172.21.15.62:0/1181247847 <== mds.0 v2:172.21.15.62:6833/752515489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x5640bcc560e0 con 0x5640bcc41100 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe104c00640 10 client.6201 handle_command_reply: tid=0 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 client.6201 shutdown 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 -- 172.21.15.62:0/1181247847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 msgr2=0x5640bcc850e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc850e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110004730 comp rx=0 tx=0).stop 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 -- 172.21.15.62:0/1181247847 shutdown_connections 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.62:6833/752515489,v1:172.21.15.62:6835/752515489] conn(0x5640bcc41100 0x5640bcc414e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5640bcc54cc0 0x5640bcc850e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640bcc30a80 0x5640bcce1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 --2- 172.21.15.62:0/1181247847 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5640bcc2fbc0 0x5640bcc503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 -- 172.21.15.62:0/1181247847 >> 172.21.15.62:0/1181247847 conn(0x5640bcc2b700 msgr2=0x5640bcc509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe104c00640 0 client.6201 ms_handle_reset on v2:172.21.15.62:6833/752515489 2024-04-25T09:08:13.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 -- 172.21.15.62:0/1181247847 shutdown_connections 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 1 -- 172.21.15.62:0/1181247847 wait complete. 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.183+0000 7fe1219f7740 20 client.6201 trim_cache size 0 max 16384 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 -- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0f406b4d0 msgr2=0x7fe0f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0f406b4d0 0x7fe0f406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe108009b70 tx=0x7fe108008040 comp rx=0 tx=0).stop 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 -- 172.21.15.62:0/3548876508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11815dab0 msgr2=0x7fe11815f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11815dab0 0x7fe11815f890 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c730 tx=0x7fe10000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 -- 172.21.15.62:0/3548876508 shutdown_connections 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0f406b4d0 0x7fe0f406d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe11815dab0 0x7fe11815f890 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe11815de90 0x7fe1181f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 --2- 172.21.15.62:0/3548876508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1180b5b50 0x7fe1181f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 -- 172.21.15.62:0/3548876508 >> 172.21.15.62:0/3548876508 conn(0x7fe1180d39c0 msgr2=0x7fe1180b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 -- 172.21.15.62:0/3548876508 shutdown_connections 2024-04-25T09:08:13.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.184+0000 7fe11d600640 1 -- 172.21.15.62:0/3548876508 wait complete. 2024-04-25T09:08:13.189 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:08:13.189 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:13.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.385+0000 7f7551c00640 1 Processor -- start 2024-04-25T09:08:13.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.386+0000 7f7551c00640 1 -- start start 2024-04-25T09:08:13.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.387+0000 7f7551c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f754c0b56f0 0x7f754c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.387+0000 7f7551c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c1627a0 con 0x7f754c0b27a0 2024-04-25T09:08:13.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.387+0000 7f7551c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c0d3110 con 0x7f754c0b6a90 2024-04-25T09:08:13.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.387+0000 7f7551c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0cedc0 con 0x7f754c0b56f0 2024-04-25T09:08:13.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.387+0000 7f7551200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f754c0b27a0 0x7f754c162090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53286/0 (socket says 172.21.15.62:53286) 2024-04-25T09:08:13.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.387+0000 7f7551200640 1 -- 172.21.15.62:0/2497256944 learned_addr learned my addr 172.21.15.62:0/2497256944 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754be00640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f754c0b56f0 0x7f754c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362022041 0 0) 0x7f754c1627a0 con 0x7f754c0b27a0 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7530003520 con 0x7f754c0b27a0 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2649128678 0 0) 0x7f7530003520 con 0x7f754c0b27a0 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f754c0b56f0 msgr2=0x7f754c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f754c0b56f0 0x7f754c160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 >> v1:172.21.15.121:6789/0 conn(0x7f754c0b6a90 legacy=0x7f754c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0d33f0 con 0x7f754c0b27a0 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f7544001670 con 0x7f754c0b27a0 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 >> v1:172.21.15.62:6789/0 conn(0x7f754c0b27a0 legacy=0x7f754c162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.388+0000 7f754aa00640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75300039e0 0x7f7530003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f754aa00640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f754c0b56f0 0x7f7530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f754aa00640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7530004d40 0x7f75300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0cedc0 con 0x7f7530004d40 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753000baa0 con 0x7f75300039e0 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753000bca0 con 0x7f754c0b56f0 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f7551200640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7530004d40 0x7f75300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f754be00640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75300039e0 0x7f7530003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f754b400640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f754c0b56f0 0x7f7530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f7551200640 1 -- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f754c0b56f0 msgr2=0x7f7530004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f7551200640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f754c0b56f0 0x7f7530004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f7551200640 1 -- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75300039e0 msgr2=0x7f7530003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f7551200640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75300039e0 0x7f7530003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.389+0000 7f7551200640 1 -- 172.21.15.62:0/2497256944 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7530003520 con 0x7f7530004d40 2024-04-25T09:08:13.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f7551200640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7530004d40 0x7f75300051c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7544007fd0 tx=0x7f7530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5805189a10b9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544007c20 con 0x7f7530004d40 2024-04-25T09:08:13.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7544005af0 con 0x7f7530004d40 2024-04-25T09:08:13.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f754aa00640 1 -- 172.21.15.62:0/2497256944 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544007070 con 0x7f7530004d40 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f7551c00640 1 -- 172.21.15.62:0/2497256944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7530004d40 msgr2=0x7f75300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f7551c00640 1 --2- 172.21.15.62:0/2497256944 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7530004d40 0x7f75300051c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7544007fd0 tx=0x7f7530005830 comp rx=0 tx=0).stop 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f7551200640 1 -- 172.21.15.62:0/2497256944 reap_dead start 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f7551c00640 1 -- 172.21.15.62:0/2497256944 shutdown_connections 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.391+0000 7f7551c00640 1 -- 172.21.15.62:0/2497256944 >> 172.21.15.62:0/2497256944 conn(0x7f754c066d10 msgr2=0x7f754c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.392+0000 7f7551c00640 1 -- 172.21.15.62:0/2497256944 shutdown_connections 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.392+0000 7f7551c00640 1 -- 172.21.15.62:0/2497256944 wait complete. 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.393+0000 7f7551c00640 1 Processor -- start 2024-04-25T09:08:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.394+0000 7f7551c00640 1 -- start start 2024-04-25T09:08:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.394+0000 7f7551c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7530004d40 0x7f754c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.394+0000 7f7551c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f754c152cd0 0x7f754c0b8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.394+0000 7f7551c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f754c0b8e90 0x7f754c0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.394+0000 7f7551c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0b2d60 con 0x7f754c0b8e90 2024-04-25T09:08:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.394+0000 7f7551c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0cea50 con 0x7f754c152cd0 2024-04-25T09:08:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.394+0000 7f7551c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0b2ee0 con 0x7f7530004d40 2024-04-25T09:08:13.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f7551200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f754c0b8e90 0x7f754c0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f7551200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f754c0b8e90 0x7f754c0c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60548/0 (socket says 172.21.15.62:60548) 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f7551200640 1 -- 172.21.15.62:0/2713760935 learned_addr learned my addr 172.21.15.62:0/2713760935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f754be00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7530004d40 0x7f754c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f7551200640 1 -- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7530004d40 msgr2=0x7f754c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f7551200640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7530004d40 0x7f754c0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f754b400640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f754c152cd0 0x7f754c0b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f7551200640 1 -- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f754c152cd0 msgr2=0x7f754c0b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f7551200640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f754c152cd0 0x7f754c0b8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f7551200640 1 -- 172.21.15.62:0/2713760935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0ca720 con 0x7f754c0b8e90 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.395+0000 7f754be00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7530004d40 0x7f754c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.396+0000 7f7551200640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f754c0b8e90 0x7f754c0c8580 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f754402acf0 tx=0x7f754c0d35c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.396+0000 7f7543e00640 1 -- 172.21.15.62:0/2713760935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544001640 con 0x7f754c0b8e90 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.396+0000 7f7543e00640 1 -- 172.21.15.62:0/2713760935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7544040070 con 0x7f754c0b8e90 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.396+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c1e7b70 con 0x7f754c0b8e90 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.396+0000 7f7543e00640 1 -- 172.21.15.62:0/2713760935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544004a00 con 0x7f754c0b8e90 2024-04-25T09:08:13.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.396+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c1e80f0 con 0x7f754c0b8e90 2024-04-25T09:08:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.397+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c04ef20 con 0x7f754c0b8e90 2024-04-25T09:08:13.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.398+0000 7f7543e00640 1 -- 172.21.15.62:0/2713760935 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f754403a070 con 0x7f754c0b8e90 2024-04-25T09:08:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.400+0000 7f7543e00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f752c06b4d0 0x7f752c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.400+0000 7f7543e00640 1 -- 172.21.15.62:0/2713760935 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f75440a9c90 con 0x7f754c0b8e90 2024-04-25T09:08:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.400+0000 7f754be00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f752c06b4d0 0x7f752c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:13.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.402+0000 7f754be00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f752c06b4d0 0x7f752c06d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f753c007440 tx=0x7f753c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:13.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.404+0000 7f7543e00640 1 -- 172.21.15.62:0/2713760935 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f754c0ca720 con 0x7f754c0b8e90 2024-04-25T09:08:13.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.642+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f754c0b96b0 con 0x7f754c0b8e90 2024-04-25T09:08:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.643+0000 7f7543e00640 1 -- 172.21.15.62:0/2713760935 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f75440789e0 con 0x7f754c0b8e90 2024-04-25T09:08:13.631 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:13.631 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:08:06.031370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 28 2024-04-25T09:08:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f752c06b4d0 msgr2=0x7f752c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f752c06b4d0 0x7f752c06d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f753c007440 tx=0x7f753c00b040 comp rx=0 tx=0).stop 2024-04-25T09:08:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f754c0b8e90 msgr2=0x7f754c0c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f754c0b8e90 0x7f754c0c8580 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f754402acf0 tx=0x7f754c0d35c0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 shutdown_connections 2024-04-25T09:08:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f752c06b4d0 0x7f752c06d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f754c0b8e90 0x7f754c0c8580 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f754c152cd0 0x7f754c0b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 --2- 172.21.15.62:0/2713760935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7530004d40 0x7f754c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.645+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 >> 172.21.15.62:0/2713760935 conn(0x7f754c066d10 msgr2=0x7f754c0b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.646+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 shutdown_connections 2024-04-25T09:08:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:13.646+0000 7f7551c00640 1 -- 172.21.15.62:0/2713760935 wait complete. 2024-04-25T09:08:13.652 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok flush_path / 2024-04-25T09:08:13.870 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:13.870 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:08:13.870 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:13.885 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:08:13.885 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:08:13.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:08:13.995 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:08:14.121 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:08:14.121 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:14.121 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:14.181 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:14.181 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:14.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:14.276 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:08:14.288+0000 7f47c3e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:08:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.458+0000 7feecf200640 1 Processor -- start 2024-04-25T09:08:14.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.459+0000 7feecf200640 1 -- start start 2024-04-25T09:08:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.460+0000 7feecf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed00b5870 0x7feed0166720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.460+0000 7feecf200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed00cf890 con 0x7feed015b900 2024-04-25T09:08:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.460+0000 7feecf200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed006fdf0 con 0x7feed00d1e90 2024-04-25T09:08:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.460+0000 7feecf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed0070a00 con 0x7feed00b5870 2024-04-25T09:08:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.461+0000 7feecde00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7feed015b900 0x7feed00d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53296/0 (socket says 172.21.15.62:53296) 2024-04-25T09:08:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.461+0000 7feecde00640 1 -- 172.21.15.62:0/2533084602 learned_addr learned my addr 172.21.15.62:0/2533084602 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.461+0000 7feece800640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed00b5870 0x7feed0166720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.461+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069473160 0 0) 0x7feec0001110 con 0x7feed015b900 2024-04-25T09:08:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.461+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb0003520 con 0x7feed015b900 2024-04-25T09:08:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2940414127 0 0) 0x7feeb0003520 con 0x7feed015b900 2024-04-25T09:08:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed00b5870 msgr2=0x7feed0166720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed00b5870 0x7feed0166720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 >> v1:172.21.15.121:6789/0 conn(0x7feed00d1e90 legacy=0x7feed00b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00700d0 con 0x7feed015b900 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7feec0001840 con 0x7feed015b900 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 >> v1:172.21.15.62:6789/0 conn(0x7feed015b900 legacy=0x7feed00d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed00b5870 0x7feeb00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feeb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb0004d40 0x7feeb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0070a00 con 0x7feeb0003fb0 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb000bb00 con 0x7feeb0004d40 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb000bd00 con 0x7feed00b5870 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecd400640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feeb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.462+0000 7feecde00640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed00b5870 0x7feeb00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.463+0000 7feece800640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb0004d40 0x7feeb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.463+0000 7feecd400640 1 -- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed00b5870 msgr2=0x7feeb00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.463+0000 7feecd400640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed00b5870 0x7feeb00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.463+0000 7feecd400640 1 -- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb0004d40 msgr2=0x7feeb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:14.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.463+0000 7feecd400640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb0004d40 0x7feeb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.463+0000 7feecd400640 1 -- 172.21.15.62:0/2533084602 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0003520 con 0x7feeb0003fb0 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.464+0000 7feecd400640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feeb0004800 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feeb8001e50 tx=0x7feeb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7799760319c168ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.464+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8001a70 con 0x7feeb0003fb0 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.464+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb802ada0 con 0x7feeb0003fb0 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.464+0000 7feecca00640 1 -- 172.21.15.62:0/2533084602 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8003bb0 con 0x7feeb0003fb0 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.465+0000 7feecf200640 1 -- 172.21.15.62:0/2533084602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 msgr2=0x7feeb0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.465+0000 7feecf200640 1 --2- 172.21.15.62:0/2533084602 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feeb0004800 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7feeb8001e50 tx=0x7feeb0005830 comp rx=0 tx=0).stop 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.465+0000 7feece800640 1 -- 172.21.15.62:0/2533084602 reap_dead start 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.465+0000 7feecf200640 1 -- 172.21.15.62:0/2533084602 shutdown_connections 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.465+0000 7feecf200640 1 -- 172.21.15.62:0/2533084602 >> 172.21.15.62:0/2533084602 conn(0x7feed0072340 msgr2=0x7feed0072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.465+0000 7feecf200640 1 -- 172.21.15.62:0/2533084602 shutdown_connections 2024-04-25T09:08:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.465+0000 7feecf200640 1 -- 172.21.15.62:0/2533084602 wait complete. 2024-04-25T09:08:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.467+0000 7feecf200640 1 Processor -- start 2024-04-25T09:08:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.467+0000 7feecf200640 1 -- start start 2024-04-25T09:08:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feed00aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed01f3980 0x7feed01f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed00af230 0x7feed00b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed015bce0 con 0x7feeb0003fb0 2024-04-25T09:08:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecf200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0070690 con 0x7feed00af230 2024-04-25T09:08:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed015be60 con 0x7feed01f3980 2024-04-25T09:08:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feed00aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feed00aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60576/0 (socket says 172.21.15.62:60576) 2024-04-25T09:08:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecde00640 1 -- 172.21.15.62:0/806120137 learned_addr learned my addr 172.21.15.62:0/806120137 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecd400640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed01f3980 0x7feed01f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feece800640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed00af230 0x7feed00b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.468+0000 7feecde00640 1 -- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed01f3980 msgr2=0x7feed01f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feecde00640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed01f3980 0x7feed01f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feecde00640 1 -- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed00af230 msgr2=0x7feed00b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feecde00640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed00af230 0x7feed00b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feecde00640 1 -- 172.21.15.62:0/806120137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed01f4950 con 0x7feeb0003fb0 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feece800640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed00af230 0x7feed00b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feecde00640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feed00aff60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7feec000e700 tx=0x7feec000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feec6000640 1 -- 172.21.15.62:0/806120137 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec000c640 con 0x7feeb0003fb0 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feec6000640 1 -- 172.21.15.62:0/806120137 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0018070 con 0x7feeb0003fb0 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feec6000640 1 -- 172.21.15.62:0/806120137 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec000ca00 con 0x7feeb0003fb0 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed01f3e00 con 0x7feeb0003fb0 2024-04-25T09:08:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.469+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed01f4390 con 0x7feeb0003fb0 2024-04-25T09:08:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.471+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed004ef20 con 0x7feeb0003fb0 2024-04-25T09:08:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.471+0000 7feec6000640 1 -- 172.21.15.62:0/806120137 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feec000cba0 con 0x7feeb0003fb0 2024-04-25T09:08:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.472+0000 7feec6000640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feeac06b500 0x7feeac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.472+0000 7feec6000640 1 -- 172.21.15.62:0/806120137 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7feec008a120 con 0x7feeb0003fb0 2024-04-25T09:08:14.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.473+0000 7feecd400640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feeac06b500 0x7feeac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.474+0000 7feecd400640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feeac06b500 0x7feeac06d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feeb8032000 tx=0x7feeb802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.477+0000 7feec6000640 1 -- 172.21.15.62:0/806120137 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feed00cf890 con 0x7feeb0003fb0 2024-04-25T09:08:14.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.677+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feed00adb90 con 0x7feeb0003fb0 2024-04-25T09:08:14.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.679+0000 7feec6000640 1 -- 172.21.15.62:0/806120137 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16889 (secure 0 0 0) 0x7feec0058e70 con 0x7feeb0003fb0 2024-04-25T09:08:14.668 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:14.668 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":50,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:01.219227+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:07:23.299693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T09:07:24.475140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":0}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.681+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feeac06b500 msgr2=0x7feeac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.681+0000 7feecf200640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feeac06b500 0x7feeac06d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feeb8032000 tx=0x7feeb802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.681+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 msgr2=0x7feed00aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.681+0000 7feecf200640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feed00aff60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7feec000e700 tx=0x7feec000ebc0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.682+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 shutdown_connections 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.682+0000 7feecf200640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feeac06b500 0x7feeac06d9c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.682+0000 7feecf200640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feed00af230 0x7feed00b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.682+0000 7feecf200640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feed01f3980 0x7feed01f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.682+0000 7feecf200640 1 --2- 172.21.15.62:0/806120137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feeb0003fb0 0x7feed00aff60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.682+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 >> 172.21.15.62:0/806120137 conn(0x7feed0072340 msgr2=0x7feed00d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.682+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 shutdown_connections 2024-04-25T09:08:14.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.682+0000 7feecf200640 1 -- 172.21.15.62:0/806120137 wait complete. 2024-04-25T09:08:14.691 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:07:23.299693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:07:24.475140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:14.691 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:14.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.967+0000 7fb847e00640 1 Processor -- start 2024-04-25T09:08:14.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.967+0000 7fb847e00640 1 -- start start 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.969+0000 7fb847e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb8480ccde0 0x7fb8480ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.969+0000 7fb847e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480b31a0 0x7fb8480b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.969+0000 7fb847e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8480ced40 con 0x7fb84806d2c0 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.969+0000 7fb847e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8480cd260 con 0x7fb8480ccde0 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.969+0000 7fb847e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480cf000 con 0x7fb8480b31a0 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.969+0000 7fb83e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb8480ccde0 0x7fb8480ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.969+0000 7fb846a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb84806d2c0 0x7fb84806d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48600/0 (socket says 172.21.15.62:48600) 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.969+0000 7fb846a00640 1 -- 172.21.15.62:0/221000831 learned_addr learned my addr 172.21.15.62:0/221000831 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:14.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.970+0000 7fb847400640 1 --2- 172.21.15.62:0/221000831 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480b31a0 0x7fb8480b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.970+0000 7fb83e000640 1 -- 172.21.15.62:0/221000831 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480b31a0 msgr2=0x7fb8480b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:14.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.970+0000 7fb83e000640 1 --2- 172.21.15.62:0/221000831 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480b31a0 0x7fb8480b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.970+0000 7fb83e000640 1 -- 172.21.15.62:0/221000831 >> v1:172.21.15.62:6789/0 conn(0x7fb84806d2c0 legacy=0x7fb84806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.970+0000 7fb83e000640 1 -- 172.21.15.62:0/221000831 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8480cf2e0 con 0x7fb8480ccde0 2024-04-25T09:08:14.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.971+0000 7fb83e000640 1 --2- 172.21.15.62:0/221000831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480ccde0 0x7fb8480ce800 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb8300097a0 tx=0x7fb83002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bf720574b07229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.972+0000 7fb846000640 1 -- 172.21.15.62:0/221000831 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830037070 con 0x7fb8480ccde0 2024-04-25T09:08:14.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.972+0000 7fb846000640 1 -- 172.21.15.62:0/221000831 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb830032440 con 0x7fb8480ccde0 2024-04-25T09:08:14.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.972+0000 7fb846000640 1 -- 172.21.15.62:0/221000831 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830032810 con 0x7fb8480ccde0 2024-04-25T09:08:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.972+0000 7fb847e00640 1 -- 172.21.15.62:0/221000831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480ccde0 msgr2=0x7fb8480ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.972+0000 7fb847e00640 1 --2- 172.21.15.62:0/221000831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480ccde0 0x7fb8480ce800 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb8300097a0 tx=0x7fb83002a180 comp rx=0 tx=0).stop 2024-04-25T09:08:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.973+0000 7fb847e00640 1 -- 172.21.15.62:0/221000831 shutdown_connections 2024-04-25T09:08:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.973+0000 7fb847e00640 1 --2- 172.21.15.62:0/221000831 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480b31a0 0x7fb8480b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.973+0000 7fb847e00640 1 --2- 172.21.15.62:0/221000831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480ccde0 0x7fb8480ce800 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.973+0000 7fb847e00640 1 -- 172.21.15.62:0/221000831 >> 172.21.15.62:0/221000831 conn(0x7fb848066f20 msgr2=0x7fb8480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.973+0000 7fb847e00640 1 -- 172.21.15.62:0/221000831 shutdown_connections 2024-04-25T09:08:14.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.973+0000 7fb847e00640 1 -- 172.21.15.62:0/221000831 wait complete. 2024-04-25T09:08:14.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.975+0000 7fb847e00640 1 Processor -- start 2024-04-25T09:08:14.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.975+0000 7fb847e00640 1 -- start start 2024-04-25T09:08:14.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb847e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb84806d2c0 0x7fb84815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb847e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8480b31a0 0x7fb8480ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb847e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480ccde0 0x7fb8480c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb847e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b2c80 con 0x7fb84806d2c0 2024-04-25T09:08:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb847e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480b29a0 con 0x7fb8480b31a0 2024-04-25T09:08:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb847e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8481448d0 con 0x7fb8480ccde0 2024-04-25T09:08:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb846a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb84806d2c0 0x7fb84815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb846a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb84806d2c0 0x7fb84815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60334/0 (socket says 172.21.15.62:60334) 2024-04-25T09:08:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.976+0000 7fb846a00640 1 -- 172.21.15.62:0/767767851 learned_addr learned my addr 172.21.15.62:0/767767851 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb83e000640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8480b31a0 0x7fb8480ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb847400640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480ccde0 0x7fb8480c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb846a00640 1 -- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480ccde0 msgr2=0x7fb8480c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb846a00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480ccde0 0x7fb8480c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb846a00640 1 -- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8480b31a0 msgr2=0x7fb8480ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb846a00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8480b31a0 0x7fb8480ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:14.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb846a00640 1 -- 172.21.15.62:0/767767851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8480ac660 con 0x7fb84806d2c0 2024-04-25T09:08:14.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb846a00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb84806d2c0 0x7fb84815b0d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb83800c3c0 tx=0x7fb83800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb83fe00640 1 -- 172.21.15.62:0/767767851 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838017070 con 0x7fb84806d2c0 2024-04-25T09:08:14.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb83fe00640 1 -- 172.21.15.62:0/767767851 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb838005ce0 con 0x7fb84806d2c0 2024-04-25T09:08:14.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8480ac940 con 0x7fb84806d2c0 2024-04-25T09:08:14.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.977+0000 7fb83fe00640 1 -- 172.21.15.62:0/767767851 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380072f0 con 0x7fb84806d2c0 2024-04-25T09:08:14.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.978+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8480acf00 con 0x7fb84806d2c0 2024-04-25T09:08:14.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.979+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84804ef20 con 0x7fb84806d2c0 2024-04-25T09:08:14.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.980+0000 7fb83fe00640 1 -- 172.21.15.62:0/767767851 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb83801a070 con 0x7fb84806d2c0 2024-04-25T09:08:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.981+0000 7fb83fe00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb82406b4d0 0x7fb82406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.981+0000 7fb83fe00640 1 -- 172.21.15.62:0/767767851 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb8380895e0 con 0x7fb84806d2c0 2024-04-25T09:08:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.981+0000 7fb83e000640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb82406b4d0 0x7fb82406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:14.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.983+0000 7fb83e000640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb82406b4d0 0x7fb82406d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb830034000 tx=0x7fb83002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:14.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:14.985+0000 7fb83fe00640 1 -- 172.21.15.62:0/767767851 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb838058460 con 0x7fb84806d2c0 2024-04-25T09:08:15.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.187+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb84806b210 con 0x7fb84806d2c0 2024-04-25T09:08:15.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.188+0000 7fb83fe00640 1 -- 172.21.15.62:0/767767851 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb838057d60 con 0x7fb84806d2c0 2024-04-25T09:08:15.176 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb82406b4d0 msgr2=0x7fb82406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb82406b4d0 0x7fb82406d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb830034000 tx=0x7fb83002e000 comp rx=0 tx=0).stop 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb84806d2c0 msgr2=0x7fb84815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb84806d2c0 0x7fb84815b0d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb83800c3c0 tx=0x7fb83800c880 comp rx=0 tx=0).stop 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 shutdown_connections 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb82406b4d0 0x7fb82406d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8480ccde0 0x7fb8480c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8480b31a0 0x7fb8480ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 --2- 172.21.15.62:0/767767851 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb84806d2c0 0x7fb84815b0d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 >> 172.21.15.62:0/767767851 conn(0x7fb848066f20 msgr2=0x7fb84806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 shutdown_connections 2024-04-25T09:08:15.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.191+0000 7fb847e00640 1 -- 172.21.15.62:0/767767851 wait complete. 2024-04-25T09:08:15.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:08:15.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.417+0000 7fd7c6000640 1 Processor -- start 2024-04-25T09:08:15.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.418+0000 7fd7c6000640 1 -- start start 2024-04-25T09:08:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.419+0000 7fd7c6000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd7c015dd50 0x7fd7c00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.419+0000 7fd7c6000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d2fb0 0x7fd7c00ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.419+0000 7fd7c6000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00ce780 con 0x7fd7c00d3960 2024-04-25T09:08:15.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.419+0000 7fd7c6000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c00d3440 con 0x7fd7c015dd50 2024-04-25T09:08:15.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.419+0000 7fd7c6000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00ce1c0 con 0x7fd7c00d2fb0 2024-04-25T09:08:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.420+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d2fb0 0x7fd7c00ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.420+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d2fb0 0x7fd7c00ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:46840/0 (socket says 172.21.15.62:46840) 2024-04-25T09:08:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.420+0000 7fd7bfe00640 1 -- 172.21.15.62:0/3716658570 learned_addr learned my addr 172.21.15.62:0/3716658570 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.420+0000 7fd7c4c00640 1 --2- 172.21.15.62:0/3716658570 >> v2:172.21.15.121:3300/0 conn(0x7fd7c015dd50 0x7fd7c00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.421+0000 7fd7bfe00640 1 -- 172.21.15.62:0/3716658570 >> v2:172.21.15.121:3300/0 conn(0x7fd7c015dd50 msgr2=0x7fd7c00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.421+0000 7fd7bfe00640 1 --2- 172.21.15.62:0/3716658570 >> v2:172.21.15.121:3300/0 conn(0x7fd7c015dd50 0x7fd7c00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.421+0000 7fd7bfe00640 1 -- 172.21.15.62:0/3716658570 >> v1:172.21.15.121:6789/0 conn(0x7fd7c00d3960 legacy=0x7fd7c00d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.421+0000 7fd7bfe00640 1 -- 172.21.15.62:0/3716658570 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00ce4a0 con 0x7fd7c00d2fb0 2024-04-25T09:08:15.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.423+0000 7fd7bfe00640 1 --2- 172.21.15.62:0/3716658570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d2fb0 0x7fd7c00ce9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009870 tx=0x7fd7ac02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332f3a257668b724 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:15.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.424+0000 7fd7bf400640 1 -- 172.21.15.62:0/3716658570 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac037070 con 0x7fd7c00d2fb0 2024-04-25T09:08:15.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.424+0000 7fd7bf400640 1 -- 172.21.15.62:0/3716658570 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac032440 con 0x7fd7c00d2fb0 2024-04-25T09:08:15.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.424+0000 7fd7bf400640 1 -- 172.21.15.62:0/3716658570 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac032810 con 0x7fd7c00d2fb0 2024-04-25T09:08:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.425+0000 7fd7c6000640 1 -- 172.21.15.62:0/3716658570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d2fb0 msgr2=0x7fd7c00ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.425+0000 7fd7c6000640 1 --2- 172.21.15.62:0/3716658570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d2fb0 0x7fd7c00ce9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009870 tx=0x7fd7ac02a270 comp rx=0 tx=0).stop 2024-04-25T09:08:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.425+0000 7fd7c6000640 1 -- 172.21.15.62:0/3716658570 shutdown_connections 2024-04-25T09:08:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.425+0000 7fd7c6000640 1 --2- 172.21.15.62:0/3716658570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d2fb0 0x7fd7c00ce9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.425+0000 7fd7c6000640 1 --2- 172.21.15.62:0/3716658570 >> v2:172.21.15.121:3300/0 conn(0x7fd7c015dd50 0x7fd7c00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.425+0000 7fd7c6000640 1 -- 172.21.15.62:0/3716658570 >> 172.21.15.62:0/3716658570 conn(0x7fd7c006b000 msgr2=0x7fd7c006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.425+0000 7fd7c6000640 1 -- 172.21.15.62:0/3716658570 shutdown_connections 2024-04-25T09:08:15.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.425+0000 7fd7c6000640 1 -- 172.21.15.62:0/3716658570 wait complete. 2024-04-25T09:08:15.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.427+0000 7fd7c6000640 1 Processor -- start 2024-04-25T09:08:15.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.427+0000 7fd7c6000640 1 -- start start 2024-04-25T09:08:15.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7c6000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d3960 0x7fd7c00b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7c6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c015dd50 0x7fd7c00b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7c6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c00b40c0 0x7fd7c014f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7c6000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00b6b00 con 0x7fd7c015dd50 2024-04-25T09:08:15.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7c6000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cef00 con 0x7fd7c00b40c0 2024-04-25T09:08:15.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7c6000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00b6c80 con 0x7fd7c00d3960 2024-04-25T09:08:15.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c015dd50 0x7fd7c00b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c015dd50 0x7fd7c00b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60350/0 (socket says 172.21.15.62:60350) 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.428+0000 7fd7bfe00640 1 -- 172.21.15.62:0/2590976794 learned_addr learned my addr 172.21.15.62:0/2590976794 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.429+0000 7fd7c4c00640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d3960 0x7fd7c00b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.429+0000 7fd7c5600640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c00b40c0 0x7fd7c014f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.429+0000 7fd7bfe00640 1 -- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d3960 msgr2=0x7fd7c00b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.429+0000 7fd7bfe00640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d3960 0x7fd7c00b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.429+0000 7fd7bfe00640 1 -- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c00b40c0 msgr2=0x7fd7c014f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.429+0000 7fd7bfe00640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c00b40c0 0x7fd7c014f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.429+0000 7fd7bfe00640 1 -- 172.21.15.62:0/2590976794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c015d0c0 con 0x7fd7c015dd50 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.429+0000 7fd7bfe00640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c015dd50 0x7fd7c00b3b80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac02a2b0 tx=0x7fd7ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.430+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2590976794 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac037040 con 0x7fd7c015dd50 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.430+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2590976794 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac042070 con 0x7fd7c015dd50 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.430+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2590976794 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac030880 con 0x7fd7c015dd50 2024-04-25T09:08:15.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.430+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c00b46f0 con 0x7fd7c015dd50 2024-04-25T09:08:15.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.430+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c01e65b0 con 0x7fd7c015dd50 2024-04-25T09:08:15.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.431+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c004ef20 con 0x7fd7c015dd50 2024-04-25T09:08:15.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.432+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2590976794 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7ac030a20 con 0x7fd7c015dd50 2024-04-25T09:08:15.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.433+0000 7fd7bcc00640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd7a006b4d0 0x7fd7a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.433+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2590976794 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd7ac0aaaa0 con 0x7fd7c015dd50 2024-04-25T09:08:15.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.434+0000 7fd7c4c00640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd7a006b4d0 0x7fd7a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.435+0000 7fd7c4c00640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd7a006b4d0 0x7fd7a006d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009a30 tx=0x7fd7b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.437+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2590976794 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7ac079920 con 0x7fd7c015dd50 2024-04-25T09:08:15.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.767+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7c00b9950 con 0x7fd7c015dd50 2024-04-25T09:08:15.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.768+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2590976794 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7ac079220 con 0x7fd7c015dd50 2024-04-25T09:08:15.756 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd7a006b4d0 msgr2=0x7fd7a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd7a006b4d0 0x7fd7a006d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009a30 tx=0x7fd7b4008040 comp rx=0 tx=0).stop 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c015dd50 msgr2=0x7fd7c00b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c015dd50 0x7fd7c00b3b80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac02a2b0 tx=0x7fd7ac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 shutdown_connections 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd7a006b4d0 0x7fd7a006d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7c015dd50 0x7fd7c00b3b80 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7c00b40c0 0x7fd7c014f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 --2- 172.21.15.62:0/2590976794 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7c00d3960 0x7fd7c00b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 >> 172.21.15.62:0/2590976794 conn(0x7fd7c006b000 msgr2=0x7fd7c00caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 shutdown_connections 2024-04-25T09:08:15.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.771+0000 7fd7c6000640 1 -- 172.21.15.62:0/2590976794 wait complete. 2024-04-25T09:08:15.779 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:08:15.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.981+0000 7fd749a00640 1 Processor -- start 2024-04-25T09:08:15.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.982+0000 7fd749a00640 1 -- start start 2024-04-25T09:08:15.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.983+0000 7fd749a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd74406b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.983+0000 7fd749a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7440b2c10 0x7fd7440d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.983+0000 7fd749a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7440b9610 con 0x7fd7440cea50 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.983+0000 7fd749a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7440b6a20 con 0x7fd7440b2c10 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.983+0000 7fd749a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74406ac80 con 0x7fd7440ce670 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.984+0000 7fd743400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7440b2c10 0x7fd7440d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.984+0000 7fd743400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7440b2c10 0x7fd7440d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60364/0 (socket says 172.21.15.62:60364) 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.984+0000 7fd743400640 1 -- 172.21.15.62:0/1760165013 learned_addr learned my addr 172.21.15.62:0/1760165013 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.984+0000 7fd743e00640 1 --2- 172.21.15.62:0/1760165013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd74406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.984+0000 7fd743400640 1 -- 172.21.15.62:0/1760165013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 msgr2=0x7fd74406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.984+0000 7fd743400640 1 --2- 172.21.15.62:0/1760165013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd74406b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.984+0000 7fd743400640 1 -- 172.21.15.62:0/1760165013 >> v1:172.21.15.121:6789/0 conn(0x7fd7440cea50 legacy=0x7fd7440d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:15.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.984+0000 7fd743400640 1 -- 172.21.15.62:0/1760165013 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74406af60 con 0x7fd7440b2c10 2024-04-25T09:08:15.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.985+0000 7fd743400640 1 --2- 172.21.15.62:0/1760165013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7440d33d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd730009870 tx=0x7fd73002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7207c56a70464d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:15.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.986+0000 7fd742a00640 1 -- 172.21.15.62:0/1760165013 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730037070 con 0x7fd7440b2c10 2024-04-25T09:08:15.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.986+0000 7fd742a00640 1 -- 172.21.15.62:0/1760165013 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730032440 con 0x7fd7440b2c10 2024-04-25T09:08:15.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.986+0000 7fd742a00640 1 -- 172.21.15.62:0/1760165013 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730032810 con 0x7fd7440b2c10 2024-04-25T09:08:15.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.987+0000 7fd749a00640 1 -- 172.21.15.62:0/1760165013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 msgr2=0x7fd7440d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.987+0000 7fd749a00640 1 --2- 172.21.15.62:0/1760165013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7440d33d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd730009870 tx=0x7fd73002a2a0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.987+0000 7fd749a00640 1 -- 172.21.15.62:0/1760165013 shutdown_connections 2024-04-25T09:08:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.987+0000 7fd749a00640 1 --2- 172.21.15.62:0/1760165013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7440d33d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.987+0000 7fd749a00640 1 --2- 172.21.15.62:0/1760165013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd74406b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.987+0000 7fd749a00640 1 -- 172.21.15.62:0/1760165013 >> 172.21.15.62:0/1760165013 conn(0x7fd7440d39c0 msgr2=0x7fd7440d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.988+0000 7fd749a00640 1 -- 172.21.15.62:0/1760165013 shutdown_connections 2024-04-25T09:08:15.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.988+0000 7fd749a00640 1 -- 172.21.15.62:0/1760165013 wait complete. 2024-04-25T09:08:15.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.991+0000 7fd749a00640 1 Processor -- start 2024-04-25T09:08:15.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.991+0000 7fd749a00640 1 -- start start 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.992+0000 7fd749a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7441e8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.992+0000 7fd749a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd7440c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.992+0000 7fd749a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7440cea50 0x7fd7440c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.992+0000 7fd749a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440b92a0 con 0x7fd7440b2c10 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.992+0000 7fd749a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440ceee0 con 0x7fd7440cea50 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.992+0000 7fd749a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74415ddd0 con 0x7fd7440ce670 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7441e8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7441e8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60366/0 (socket says 172.21.15.62:60366) 2024-04-25T09:08:15.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743e00640 1 -- 172.21.15.62:0/3071352635 learned_addr learned my addr 172.21.15.62:0/3071352635 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743400640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd7440c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd749000640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7440cea50 0x7fd7440c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743e00640 1 -- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 msgr2=0x7fd7440c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743e00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd7440c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743e00640 1 -- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7440cea50 msgr2=0x7fd7440c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743e00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7440cea50 0x7fd7440c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743e00640 1 -- 172.21.15.62:0/3071352635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7441e8670 con 0x7fd7440b2c10 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd749000640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7440cea50 0x7fd7440c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.993+0000 7fd743400640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd7440c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.994+0000 7fd743e00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7441e8130 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd73400d800 tx=0x7fd73400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.994+0000 7fd73be00640 1 -- 172.21.15.62:0/3071352635 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73400b6c0 con 0x7fd7440b2c10 2024-04-25T09:08:15.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.994+0000 7fd73be00640 1 -- 172.21.15.62:0/3071352635 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd734019070 con 0x7fd7440b2c10 2024-04-25T09:08:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.994+0000 7fd73be00640 1 -- 172.21.15.62:0/3071352635 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73400ba80 con 0x7fd7440b2c10 2024-04-25T09:08:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.994+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7441f49b0 con 0x7fd7440b2c10 2024-04-25T09:08:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.994+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7441e7930 con 0x7fd7440b2c10 2024-04-25T09:08:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.995+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74404ef20 con 0x7fd7440b2c10 2024-04-25T09:08:15.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.996+0000 7fd73be00640 1 -- 172.21.15.62:0/3071352635 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd73400bc20 con 0x7fd7440b2c10 2024-04-25T09:08:15.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.998+0000 7fd73be00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd72406b4d0 0x7fd72406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:15.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.998+0000 7fd73be00640 1 -- 172.21.15.62:0/3071352635 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd73408a740 con 0x7fd7440b2c10 2024-04-25T09:08:15.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:15.998+0000 7fd743400640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd72406b4d0 0x7fd72406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:15.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.000+0000 7fd743400640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd72406b4d0 0x7fd72406d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd730034000 tx=0x7fd73002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:15.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.002+0000 7fd73be00640 1 -- 172.21.15.62:0/3071352635 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd734059530 con 0x7fd7440b2c10 2024-04-25T09:08:16.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.204+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7440d1670 con 0x7fd7440b2c10 2024-04-25T09:08:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.205+0000 7fd73be00640 1 -- 172.21.15.62:0/3071352635 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd734058e30 con 0x7fd7440b2c10 2024-04-25T09:08:16.193 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:08:16.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd72406b4d0 msgr2=0x7fd72406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:16.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd72406b4d0 0x7fd72406d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd730034000 tx=0x7fd73002e000 comp rx=0 tx=0).stop 2024-04-25T09:08:16.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 msgr2=0x7fd7441e8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:16.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7441e8130 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd73400d800 tx=0x7fd73400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 shutdown_connections 2024-04-25T09:08:16.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd72406b4d0 0x7fd72406d990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7440cea50 0x7fd7440c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7440ce670 0x7fd7440c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 --2- 172.21.15.62:0/3071352635 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7440b2c10 0x7fd7441e8130 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.207+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 >> 172.21.15.62:0/3071352635 conn(0x7fd7440d39c0 msgr2=0x7fd74415c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.208+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 shutdown_connections 2024-04-25T09:08:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.208+0000 7fd749a00640 1 -- 172.21.15.62:0/3071352635 wait complete. 2024-04-25T09:08:16.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:08:16.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.406+0000 7f2198c00640 1 Processor -- start 2024-04-25T09:08:16.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.406+0000 7f2198c00640 1 -- start start 2024-04-25T09:08:16.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.407+0000 7f2198c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f219415c4e0 0x7f219415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:16.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.408+0000 7f2198c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940b49e0 0x7f21940b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:16.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.408+0000 7f2198c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21940cff60 con 0x7f21940cf5f0 2024-04-25T09:08:16.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.408+0000 7f2198c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2194070360 con 0x7f219415c4e0 2024-04-25T09:08:16.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.408+0000 7f2198c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2194070640 con 0x7f21940b49e0 2024-04-25T09:08:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.408+0000 7f2193e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940b49e0 0x7f21940b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.408+0000 7f2193400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f219415c4e0 0x7f219415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.408+0000 7f218be00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f21940cf5f0 0x7f21940b42d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:33280/0 (socket says 172.21.15.62:33280) 2024-04-25T09:08:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.409+0000 7f218be00640 1 -- 172.21.15.62:0/4008749728 learned_addr learned my addr 172.21.15.62:0/4008749728 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:16.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.409+0000 7f2193e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940b49e0 0x7f21940b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:46876/0 (socket says 172.21.15.62:46876) 2024-04-25T09:08:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.409+0000 7f2193e00640 1 -- 172.21.15.62:0/4008749728 >> v2:172.21.15.121:3300/0 conn(0x7f219415c4e0 msgr2=0x7f219415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.409+0000 7f2193e00640 1 --2- 172.21.15.62:0/4008749728 >> v2:172.21.15.121:3300/0 conn(0x7f219415c4e0 0x7f219415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.409+0000 7f2193e00640 1 -- 172.21.15.62:0/4008749728 >> v1:172.21.15.121:6789/0 conn(0x7f21940cf5f0 legacy=0x7f21940b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.409+0000 7f2193e00640 1 -- 172.21.15.62:0/4008749728 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2194070920 con 0x7f21940b49e0 2024-04-25T09:08:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.411+0000 7f2193e00640 1 --2- 172.21.15.62:0/4008749728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940b49e0 0x7f21940b2710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f218c009d90 tx=0x7f218c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd90d2742791e6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:16.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.411+0000 7f2192a00640 1 -- 172.21.15.62:0/4008749728 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c007740 con 0x7f21940b49e0 2024-04-25T09:08:16.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.411+0000 7f2192a00640 1 -- 172.21.15.62:0/4008749728 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c0078e0 con 0x7f21940b49e0 2024-04-25T09:08:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.411+0000 7f2192a00640 1 -- 172.21.15.62:0/4008749728 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c007cb0 con 0x7f21940b49e0 2024-04-25T09:08:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.412+0000 7f2198c00640 1 -- 172.21.15.62:0/4008749728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940b49e0 msgr2=0x7f21940b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.412+0000 7f2198c00640 1 --2- 172.21.15.62:0/4008749728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940b49e0 0x7f21940b2710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f218c009d90 tx=0x7f218c02a660 comp rx=0 tx=0).stop 2024-04-25T09:08:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.412+0000 7f2198c00640 1 -- 172.21.15.62:0/4008749728 shutdown_connections 2024-04-25T09:08:16.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.412+0000 7f2198c00640 1 --2- 172.21.15.62:0/4008749728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940b49e0 0x7f21940b2710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.412+0000 7f2198c00640 1 --2- 172.21.15.62:0/4008749728 >> v2:172.21.15.121:3300/0 conn(0x7f219415c4e0 0x7f219415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.412+0000 7f2198c00640 1 -- 172.21.15.62:0/4008749728 >> 172.21.15.62:0/4008749728 conn(0x7f21940668f0 msgr2=0x7f2194066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:16.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.412+0000 7f2198c00640 1 -- 172.21.15.62:0/4008749728 shutdown_connections 2024-04-25T09:08:16.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.412+0000 7f2198c00640 1 -- 172.21.15.62:0/4008749728 wait complete. 2024-04-25T09:08:16.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.413+0000 7f2198c00640 1 Processor -- start 2024-04-25T09:08:16.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.414+0000 7f2198c00640 1 -- start start 2024-04-25T09:08:16.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.414+0000 7f2198c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940b49e0 0x7f2194156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:16.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.414+0000 7f2198c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940cf5f0 0x7f21941f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:16.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.414+0000 7f2198c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f219415c4e0 0x7f21940adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:16.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.414+0000 7f2198c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2194071120 con 0x7f21940b49e0 2024-04-25T09:08:16.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.414+0000 7f2198c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219415ce00 con 0x7f219415c4e0 2024-04-25T09:08:16.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.414+0000 7f2198c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21940cfaa0 con 0x7f21940cf5f0 2024-04-25T09:08:16.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940b49e0 0x7f2194156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:16.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940b49e0 0x7f2194156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60378/0 (socket says 172.21.15.62:60378) 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 -- 172.21.15.62:0/3774047782 learned_addr learned my addr 172.21.15.62:0/3774047782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f218be00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940cf5f0 0x7f21941f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193e00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f219415c4e0 0x7f21940adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 -- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940cf5f0 msgr2=0x7f21941f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940cf5f0 0x7f21941f4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 -- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f219415c4e0 msgr2=0x7f21940adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f219415c4e0 0x7f21940adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 -- 172.21.15.62:0/3774047782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21941f5330 con 0x7f21940b49e0 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.415+0000 7f2193400640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940b49e0 0x7f2194156260 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f218400e9b0 tx=0x7f218400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.416+0000 7f218b400640 1 -- 172.21.15.62:0/3774047782 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184017070 con 0x7f21940b49e0 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.416+0000 7f218b400640 1 -- 172.21.15.62:0/3774047782 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218400c9a0 con 0x7f21940b49e0 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.416+0000 7f218b400640 1 -- 172.21.15.62:0/3774047782 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218400cd90 con 0x7f21940b49e0 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.416+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21941f5610 con 0x7f21940b49e0 2024-04-25T09:08:16.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.416+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21941f1d70 con 0x7f21940b49e0 2024-04-25T09:08:16.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.417+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219404ef20 con 0x7f21940b49e0 2024-04-25T09:08:16.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.418+0000 7f218b400640 1 -- 172.21.15.62:0/3774047782 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f218401a070 con 0x7f21940b49e0 2024-04-25T09:08:16.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.419+0000 7f218b400640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f217006b520 0x7f217006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:16.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.419+0000 7f218b400640 1 -- 172.21.15.62:0/3774047782 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f218408a510 con 0x7f21940b49e0 2024-04-25T09:08:16.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.420+0000 7f218be00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f217006b520 0x7f217006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:16.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.421+0000 7f218be00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f217006b520 0x7f217006d9e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f21940d0710 tx=0x7f217c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:16.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.423+0000 7f218b400640 1 -- 172.21.15.62:0/3774047782 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2184059470 con 0x7f21940b49e0 2024-04-25T09:08:16.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.625+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2194071b00 con 0x7f21940b49e0 2024-04-25T09:08:16.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.626+0000 7f218b400640 1 -- 172.21.15.62:0/3774047782 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2184058d70 con 0x7f21940b49e0 2024-04-25T09:08:16.614 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:08:16.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f217006b520 msgr2=0x7f217006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f217006b520 0x7f217006d9e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f21940d0710 tx=0x7f217c009040 comp rx=0 tx=0).stop 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940b49e0 msgr2=0x7f2194156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940b49e0 0x7f2194156260 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f218400e9b0 tx=0x7f218400ee70 comp rx=0 tx=0).stop 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 shutdown_connections 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f217006b520 0x7f217006d9e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f219415c4e0 0x7f21940adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f21940cf5f0 0x7f21941f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 --2- 172.21.15.62:0/3774047782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940b49e0 0x7f2194156260 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 >> 172.21.15.62:0/3774047782 conn(0x7f21940668f0 msgr2=0x7f21940cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 shutdown_connections 2024-04-25T09:08:16.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:16.629+0000 7f2198c00640 1 -- 172.21.15.62:0/3774047782 wait complete. 2024-04-25T09:08:16.635 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:08:16.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:16.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:16.987 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:08:16.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:16.988 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:08:17.028 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:08:17.028 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:17.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:08:17.080 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:17.080 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:17.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:08:17.127 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:08:17.127 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:17.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:08:17.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:17.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.408+0000 7f26e3e00640 1 Processor -- start 2024-04-25T09:08:17.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.409+0000 7f26e3e00640 1 -- start start 2024-04-25T09:08:17.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.410+0000 7f26e3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26dc0cfed0 0x7f26dc15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc0709c0 0x7f26dc0b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e3e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26dc06c310 con 0x7f26dc072340 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e3e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26dc15f5f0 con 0x7f26dc0cfed0 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc06c5b0 con 0x7f26dc0709c0 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26dc0cfed0 0x7f26dc15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26dc0cfed0 0x7f26dc15f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60394/0 (socket says 172.21.15.62:60394) 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e1400640 1 -- 172.21.15.62:0/2028053989 learned_addr learned my addr 172.21.15.62:0/2028053989 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e1400640 1 -- 172.21.15.62:0/2028053989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc0709c0 msgr2=0x7f26dc0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e1400640 1 --2- 172.21.15.62:0/2028053989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc0709c0 0x7f26dc0b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e1400640 1 -- 172.21.15.62:0/2028053989 >> v1:172.21.15.121:6789/0 conn(0x7f26dc072340 legacy=0x7f26dc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:17.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.411+0000 7f26e1400640 1 -- 172.21.15.62:0/2028053989 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc06c890 con 0x7f26dc0cfed0 2024-04-25T09:08:17.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.413+0000 7f26e1400640 1 --2- 172.21.15.62:0/2028053989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc15f0b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009870 tx=0x7f26d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dac8c6ebfbe79b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.413+0000 7f26dbe00640 1 -- 172.21.15.62:0/2028053989 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0037070 con 0x7f26dc0cfed0 2024-04-25T09:08:17.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.413+0000 7f26dbe00640 1 -- 172.21.15.62:0/2028053989 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d0032440 con 0x7f26dc0cfed0 2024-04-25T09:08:17.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.413+0000 7f26dbe00640 1 -- 172.21.15.62:0/2028053989 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0032810 con 0x7f26dc0cfed0 2024-04-25T09:08:17.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.414+0000 7f26e3e00640 1 -- 172.21.15.62:0/2028053989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 msgr2=0x7f26dc15f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.414+0000 7f26e3e00640 1 --2- 172.21.15.62:0/2028053989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc15f0b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009870 tx=0x7f26d002a210 comp rx=0 tx=0).stop 2024-04-25T09:08:17.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.414+0000 7f26e3e00640 1 -- 172.21.15.62:0/2028053989 shutdown_connections 2024-04-25T09:08:17.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.414+0000 7f26e3e00640 1 --2- 172.21.15.62:0/2028053989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc0709c0 0x7f26dc0b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.414+0000 7f26e3e00640 1 --2- 172.21.15.62:0/2028053989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc15f0b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.414+0000 7f26e3e00640 1 -- 172.21.15.62:0/2028053989 >> 172.21.15.62:0/2028053989 conn(0x7f26dc0668f0 msgr2=0x7f26dc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.414+0000 7f26e3e00640 1 -- 172.21.15.62:0/2028053989 shutdown_connections 2024-04-25T09:08:17.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.414+0000 7f26e3e00640 1 -- 172.21.15.62:0/2028053989 wait complete. 2024-04-25T09:08:17.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.416+0000 7f26e3e00640 1 Processor -- start 2024-04-25T09:08:17.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.417+0000 7f26e3e00640 1 -- start start 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26dc0709c0 0x7f26dc141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc13f3e0 0x7f26dc13f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc06d130 con 0x7f26dc0cfed0 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc06ce50 con 0x7f26dc0709c0 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc0b2dc0 con 0x7f26dc13f3e0 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc0cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60406/0 (socket says 172.21.15.62:60406) 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e0a00640 1 -- 172.21.15.62:0/1117660889 learned_addr learned my addr 172.21.15.62:0/1117660889 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.418+0000 7f26e1400640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26dc0709c0 0x7f26dc141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.419+0000 7f26e1e00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc13f3e0 0x7f26dc13f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.419+0000 7f26e0a00640 1 -- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc13f3e0 msgr2=0x7f26dc13f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.419+0000 7f26e0a00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc13f3e0 0x7f26dc13f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.419+0000 7f26e0a00640 1 -- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26dc0709c0 msgr2=0x7f26dc141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.419+0000 7f26e0a00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26dc0709c0 0x7f26dc141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.419+0000 7f26e0a00640 1 -- 172.21.15.62:0/1117660889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc0ce5f0 con 0x7f26dc0cfed0 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.419+0000 7f26e1400640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26dc0709c0 0x7f26dc141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.419+0000 7f26e0a00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc0cd6c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c410 tx=0x7f26c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.420+0000 7f26d9600640 1 -- 172.21.15.62:0/1117660889 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8017070 con 0x7f26dc0cfed0 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.420+0000 7f26d9600640 1 -- 172.21.15.62:0/1117660889 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c80035f0 con 0x7f26dc0cfed0 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.420+0000 7f26d9600640 1 -- 172.21.15.62:0/1117660889 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80039e0 con 0x7f26dc0cfed0 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.420+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc0ce030 con 0x7f26dc0cfed0 2024-04-25T09:08:17.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.420+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc13c7a0 con 0x7f26dc0cfed0 2024-04-25T09:08:17.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.421+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc072c70 con 0x7f26dc0cfed0 2024-04-25T09:08:17.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.422+0000 7f26d9600640 1 -- 172.21.15.62:0/1117660889 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26c801a070 con 0x7f26dc0cfed0 2024-04-25T09:08:17.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.423+0000 7f26d9600640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26bc06b4d0 0x7f26bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.423+0000 7f26d9600640 1 -- 172.21.15.62:0/1117660889 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f26c80897a0 con 0x7f26dc0cfed0 2024-04-25T09:08:17.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.424+0000 7f26e1400640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26bc06b4d0 0x7f26bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.425+0000 7f26e1400640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26bc06b4d0 0x7f26bc06d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f26d0034000 tx=0x7f26d002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.428+0000 7f26d9600640 1 -- 172.21.15.62:0/1117660889 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26c80584f0 con 0x7f26dc0cfed0 2024-04-25T09:08:17.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.668+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26dc04f000 con 0x7f26dc0cfed0 2024-04-25T09:08:17.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.669+0000 7f26d9600640 1 -- 172.21.15.62:0/1117660889 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f26c80010c0 con 0x7f26dc0cfed0 2024-04-25T09:08:17.658 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:17.658 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:08:14.032985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:17.658 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 29 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.672+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26bc06b4d0 msgr2=0x7f26bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.672+0000 7f26e3e00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26bc06b4d0 0x7f26bc06d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f26d0034000 tx=0x7f26d002e000 comp rx=0 tx=0).stop 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.672+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 msgr2=0x7f26dc0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.672+0000 7f26e3e00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc0cd6c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c410 tx=0x7f26c800c8d0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.672+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 shutdown_connections 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.672+0000 7f26e3e00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26bc06b4d0 0x7f26bc06d990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.672+0000 7f26e3e00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26dc0cfed0 0x7f26dc0cd6c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.673+0000 7f26e3e00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26dc13f3e0 0x7f26dc13f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.673+0000 7f26e3e00640 1 --2- 172.21.15.62:0/1117660889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26dc0709c0 0x7f26dc141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.673+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 >> 172.21.15.62:0/1117660889 conn(0x7f26dc0668f0 msgr2=0x7f26dc0b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.673+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 shutdown_connections 2024-04-25T09:08:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.673+0000 7f26e3e00640 1 -- 172.21.15.62:0/1117660889 wait complete. 2024-04-25T09:08:17.681 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.904+0000 7ff006600640 1 Processor -- start 2024-04-25T09:08:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.904+0000 7ff006600640 1 -- start start 2024-04-25T09:08:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.906+0000 7ff006600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff00006d900 0x7ff00015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.906+0000 7ff006600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00015faa0 con 0x7ff000144c10 2024-04-25T09:08:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.906+0000 7ff006600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00015fd80 con 0x7ff0000cee40 2024-04-25T09:08:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.906+0000 7ff006600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff000144ff0 con 0x7ff00006d900 2024-04-25T09:08:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.906+0000 7fefffe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff00006d900 0x7ff00015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.907+0000 7fefffe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff00006d900 0x7ff00015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60414/0 (socket says 172.21.15.62:60414) 2024-04-25T09:08:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.907+0000 7ff005200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff000144c10 0x7ff00006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48610/0 (socket says 172.21.15.62:48610) 2024-04-25T09:08:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.907+0000 7fefffe00640 1 -- 172.21.15.62:0/2337638527 learned_addr learned my addr 172.21.15.62:0/2337638527 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.907+0000 7fefffe00640 1 -- 172.21.15.62:0/2337638527 >> v1:172.21.15.121:6789/0 conn(0x7ff0000cee40 legacy=0x7ff0000b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.907+0000 7fefffe00640 1 -- 172.21.15.62:0/2337638527 >> v1:172.21.15.62:6789/0 conn(0x7ff000144c10 legacy=0x7ff00006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.907+0000 7fefffe00640 1 -- 172.21.15.62:0/2337638527 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000160060 con 0x7ff00006d900 2024-04-25T09:08:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.907+0000 7fefff400640 1 -- 172.21.15.62:0/2337638527 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38870463 0 0) 0x7ff00015faa0 con 0x7ff000144c10 2024-04-25T09:08:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.908+0000 7fefffe00640 1 --2- 172.21.15.62:0/2337638527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff00015f450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fefec009870 tx=0x7fefec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf64a57cf3ca325b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7fefff400640 1 -- 172.21.15.62:0/2337638527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec037070 con 0x7ff00006d900 2024-04-25T09:08:17.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7fefff400640 1 -- 172.21.15.62:0/2337638527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec032440 con 0x7ff00006d900 2024-04-25T09:08:17.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7fefff400640 1 -- 172.21.15.62:0/2337638527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec032810 con 0x7ff00006d900 2024-04-25T09:08:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7ff006600640 1 -- 172.21.15.62:0/2337638527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 msgr2=0x7ff00015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7ff006600640 1 --2- 172.21.15.62:0/2337638527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff00015f450 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fefec009870 tx=0x7fefec02a270 comp rx=0 tx=0).stop 2024-04-25T09:08:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7ff006600640 1 -- 172.21.15.62:0/2337638527 shutdown_connections 2024-04-25T09:08:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7ff006600640 1 --2- 172.21.15.62:0/2337638527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff00015f450 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7ff006600640 1 -- 172.21.15.62:0/2337638527 >> 172.21.15.62:0/2337638527 conn(0x7ff0000b14d0 msgr2=0x7ff0000b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.909+0000 7ff006600640 1 -- 172.21.15.62:0/2337638527 shutdown_connections 2024-04-25T09:08:17.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.910+0000 7ff006600640 1 -- 172.21.15.62:0/2337638527 wait complete. 2024-04-25T09:08:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.911+0000 7ff006600640 1 Processor -- start 2024-04-25T09:08:17.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.911+0000 7ff006600640 1 -- start start 2024-04-25T09:08:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.912+0000 7ff006600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff000143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.912+0000 7ff006600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0000cee40 0x7ff00015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.912+0000 7ff006600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff000144c10 0x7ff0000c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.912+0000 7ff006600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000b41c0 con 0x7ff00006d900 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.912+0000 7ff006600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff000160340 con 0x7ff0000cee40 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.912+0000 7ff006600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000b4d60 con 0x7ff000144c10 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff000143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff000143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60416/0 (socket says 172.21.15.62:60416) 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 -- 172.21.15.62:0/2223242683 learned_addr learned my addr 172.21.15.62:0/2223242683 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7fefffe00640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0000cee40 0x7ff00015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 -- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff000144c10 msgr2=0x7ff0000c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005c00640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff000144c10 0x7ff0000c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff000144c10 0x7ff0000c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 -- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0000cee40 msgr2=0x7ff00015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0000cee40 0x7ff00015afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:17.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 -- 172.21.15.62:0/2223242683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000c6c80 con 0x7ff00006d900 2024-04-25T09:08:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7fefffe00640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0000cee40 0x7ff00015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.913+0000 7ff005200640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff000143f20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7feff400c3f0 tx=0x7feff400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.914+0000 7feffcc00640 1 -- 172.21.15.62:0/2223242683 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40074e0 con 0x7ff00006d900 2024-04-25T09:08:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.914+0000 7feffcc00640 1 -- 172.21.15.62:0/2223242683 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff40047e0 con 0x7ff00006d900 2024-04-25T09:08:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.914+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff000144460 con 0x7ff00006d900 2024-04-25T09:08:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.914+0000 7feffcc00640 1 -- 172.21.15.62:0/2223242683 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4004c10 con 0x7ff00006d900 2024-04-25T09:08:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.914+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0000c6350 con 0x7ff00006d900 2024-04-25T09:08:17.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.915+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00004ef20 con 0x7ff00006d900 2024-04-25T09:08:17.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.916+0000 7feffcc00640 1 -- 172.21.15.62:0/2223242683 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feff4011070 con 0x7ff00006d900 2024-04-25T09:08:17.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.918+0000 7feffcc00640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fefe006b4d0 0x7fefe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:17.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.918+0000 7feffcc00640 1 -- 172.21.15.62:0/2223242683 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7feff4089e40 con 0x7ff00006d900 2024-04-25T09:08:17.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.918+0000 7fefffe00640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fefe006b4d0 0x7fefe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:17.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.920+0000 7fefffe00640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fefe006b4d0 0x7fefe006d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fefec034000 tx=0x7fefec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:17.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:17.922+0000 7feffcc00640 1 -- 172.21.15.62:0/2223242683 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feff401a2b0 con 0x7ff00006d900 2024-04-25T09:08:18.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.132+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0000b22b0 con 0x7ff00006d900 2024-04-25T09:08:18.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.135+0000 7feffcc00640 1 -- 172.21.15.62:0/2223242683 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16889 (secure 0 0 0) 0x7feff4058c20 con 0x7ff00006d900 2024-04-25T09:08:18.124 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:18.124 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":50,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:01.219227+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:07:23.299693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T09:07:24.475140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":0}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:18.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fefe006b4d0 msgr2=0x7fefe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fefe006b4d0 0x7fefe006d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fefec034000 tx=0x7fefec02e000 comp rx=0 tx=0).stop 2024-04-25T09:08:18.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 msgr2=0x7ff000143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff000143f20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7feff400c3f0 tx=0x7feff400c8b0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 shutdown_connections 2024-04-25T09:08:18.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fefe006b4d0 0x7fefe006d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff000144c10 0x7ff0000c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0000cee40 0x7ff00015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 --2- 172.21.15.62:0/2223242683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff00006d900 0x7ff000143f20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.138+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 >> 172.21.15.62:0/2223242683 conn(0x7ff0000b14d0 msgr2=0x7ff000071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.139+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 shutdown_connections 2024-04-25T09:08:18.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.139+0000 7ff006600640 1 -- 172.21.15.62:0/2223242683 wait complete. 2024-04-25T09:08:18.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:07:23.299693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:07:24.475140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:18.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:18.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.375+0000 7f525be00640 1 Processor -- start 2024-04-25T09:08:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.375+0000 7f525be00640 1 -- start start 2024-04-25T09:08:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.377+0000 7f525be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f525c0d2fb0 0x7f525c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.377+0000 7f525be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f525c0d3960 0x7f525c0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.377+0000 7f525be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0ce780 con 0x7f525c15dd50 2024-04-25T09:08:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.377+0000 7f525be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525c0d42f0 con 0x7f525c0d2fb0 2024-04-25T09:08:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.377+0000 7f525be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525c0d3440 con 0x7f525c0d3960 2024-04-25T09:08:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.377+0000 7f525aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f525c15dd50 0x7f525c0d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48616/0 (socket says 172.21.15.62:48616) 2024-04-25T09:08:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.377+0000 7f525aa00640 1 -- 172.21.15.62:0/334040777 learned_addr learned my addr 172.21.15.62:0/334040777 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.377+0000 7f5253e00640 1 --2- 172.21.15.62:0/334040777 >> v2:172.21.15.62:3300/0 conn(0x7f525c0d2fb0 0x7f525c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.378+0000 7f525b400640 1 --2- 172.21.15.62:0/334040777 >> v2:172.21.15.121:3300/0 conn(0x7f525c0d3960 0x7f525c0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.378+0000 7f525a000640 1 -- 172.21.15.62:0/334040777 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748209919 0 0) 0x7f525c0ce780 con 0x7f525c15dd50 2024-04-25T09:08:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.378+0000 7f525a000640 1 -- 172.21.15.62:0/334040777 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5240003520 con 0x7f525c15dd50 2024-04-25T09:08:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.378+0000 7f5253e00640 1 -- 172.21.15.62:0/334040777 >> v2:172.21.15.121:3300/0 conn(0x7f525c0d3960 msgr2=0x7f525c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.378+0000 7f5253e00640 1 --2- 172.21.15.62:0/334040777 >> v2:172.21.15.121:3300/0 conn(0x7f525c0d3960 0x7f525c0d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.378+0000 7f5253e00640 1 -- 172.21.15.62:0/334040777 >> v1:172.21.15.62:6789/0 conn(0x7f525c15dd50 legacy=0x7f525c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.378+0000 7f5253e00640 1 -- 172.21.15.62:0/334040777 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0ce1c0 con 0x7f525c0d2fb0 2024-04-25T09:08:18.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.379+0000 7f5253e00640 1 --2- 172.21.15.62:0/334040777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0d2fb0 0x7f525c0ce9c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f525c0caa50 tx=0x7f524402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ee8def87985a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.379+0000 7f525a000640 1 -- 172.21.15.62:0/334040777 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244037070 con 0x7f525c0d2fb0 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.379+0000 7f525a000640 1 -- 172.21.15.62:0/334040777 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5244032440 con 0x7f525c0d2fb0 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.379+0000 7f525a000640 1 -- 172.21.15.62:0/334040777 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440327f0 con 0x7f525c0d2fb0 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.380+0000 7f525be00640 1 -- 172.21.15.62:0/334040777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0d2fb0 msgr2=0x7f525c0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.380+0000 7f525be00640 1 --2- 172.21.15.62:0/334040777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0d2fb0 0x7f525c0ce9c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f525c0caa50 tx=0x7f524402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.380+0000 7f525be00640 1 -- 172.21.15.62:0/334040777 shutdown_connections 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.380+0000 7f525be00640 1 --2- 172.21.15.62:0/334040777 >> v2:172.21.15.121:3300/0 conn(0x7f525c0d3960 0x7f525c0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.380+0000 7f525be00640 1 --2- 172.21.15.62:0/334040777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0d2fb0 0x7f525c0ce9c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.380+0000 7f525be00640 1 -- 172.21.15.62:0/334040777 >> 172.21.15.62:0/334040777 conn(0x7f525c06b000 msgr2=0x7f525c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.380+0000 7f525be00640 1 -- 172.21.15.62:0/334040777 shutdown_connections 2024-04-25T09:08:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.380+0000 7f525be00640 1 -- 172.21.15.62:0/334040777 wait complete. 2024-04-25T09:08:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.382+0000 7f525be00640 1 Processor -- start 2024-04-25T09:08:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.382+0000 7f525be00640 1 -- start start 2024-04-25T09:08:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f525be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0d3960 0x7f525c152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f525be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c15dd50 0x7f525c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f525be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15fca0 0x7f525c160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f525be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0b6b00 con 0x7f525c15dd50 2024-04-25T09:08:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f525be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0cef00 con 0x7f525c15fca0 2024-04-25T09:08:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f525be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0b6c80 con 0x7f525c0d3960 2024-04-25T09:08:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f5253e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c15dd50 0x7f525c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f5253e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c15dd50 0x7f525c1602f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60434/0 (socket says 172.21.15.62:60434) 2024-04-25T09:08:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.383+0000 7f5253e00640 1 -- 172.21.15.62:0/3049756639 learned_addr learned my addr 172.21.15.62:0/3049756639 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.384+0000 7f5253e00640 1 -- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0d3960 msgr2=0x7f525c152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.384+0000 7f525aa00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0d3960 0x7f525c152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.384+0000 7f525b400640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15fca0 0x7f525c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.384+0000 7f5253e00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0d3960 0x7f525c152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.384+0000 7f5253e00640 1 -- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15fca0 msgr2=0x7f525c160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.384+0000 7f5253e00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15fca0 0x7f525c160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.384+0000 7f5253e00640 1 -- 172.21.15.62:0/3049756639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c158ab0 con 0x7f525c15dd50 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.384+0000 7f5253e00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c15dd50 0x7f525c1602f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f524402a1e0 tx=0x7f52440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.385+0000 7f5252a00640 1 -- 172.21.15.62:0/3049756639 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244037040 con 0x7f525c15dd50 2024-04-25T09:08:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.385+0000 7f5252a00640 1 -- 172.21.15.62:0/3049756639 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52440304c0 con 0x7f525c15dd50 2024-04-25T09:08:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.385+0000 7f5252a00640 1 -- 172.21.15.62:0/3049756639 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5244030890 con 0x7f525c15dd50 2024-04-25T09:08:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.385+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c157f30 con 0x7f525c15dd50 2024-04-25T09:08:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.385+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c1584f0 con 0x7f525c15dd50 2024-04-25T09:08:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.386+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c0cf3f0 con 0x7f525c15dd50 2024-04-25T09:08:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.387+0000 7f5252a00640 1 -- 172.21.15.62:0/3049756639 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5244044020 con 0x7f525c15dd50 2024-04-25T09:08:18.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.388+0000 7f5252a00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523806b4d0 0x7f523806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.388+0000 7f5252a00640 1 -- 172.21.15.62:0/3049756639 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f52440aa6b0 con 0x7f525c15dd50 2024-04-25T09:08:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.389+0000 7f525aa00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523806b4d0 0x7f523806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.390+0000 7f525aa00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523806b4d0 0x7f523806d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f524c000f60 tx=0x7f524c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.392+0000 7f5252a00640 1 -- 172.21.15.62:0/3049756639 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f525c0cf3f0 con 0x7f525c15dd50 2024-04-25T09:08:18.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.593+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f525c04ef20 con 0x7f525c15dd50 2024-04-25T09:08:18.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.593+0000 7f5252a00640 1 -- 172.21.15.62:0/3049756639 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5244079400 con 0x7f525c15dd50 2024-04-25T09:08:18.582 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523806b4d0 msgr2=0x7f523806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523806b4d0 0x7f523806d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f524c000f60 tx=0x7f524c009040 comp rx=0 tx=0).stop 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c15dd50 msgr2=0x7f525c1602f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c15dd50 0x7f525c1602f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f524402a1e0 tx=0x7f52440046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 shutdown_connections 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523806b4d0 0x7f523806d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c15dd50 0x7f525c1602f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15fca0 0x7f525c160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 --2- 172.21.15.62:0/3049756639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0d3960 0x7f525c152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 >> 172.21.15.62:0/3049756639 conn(0x7f525c06b000 msgr2=0x7f525c0b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 shutdown_connections 2024-04-25T09:08:18.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.596+0000 7f525be00640 1 -- 172.21.15.62:0/3049756639 wait complete. 2024-04-25T09:08:18.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:08:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.829+0000 7f4280c00640 1 Processor -- start 2024-04-25T09:08:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.829+0000 7f4280c00640 1 -- start start 2024-04-25T09:08:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.830+0000 7f4280c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0b31a0 0x7f427c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.830+0000 7f4280c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427c0cd260 con 0x7f427c0ccde0 2024-04-25T09:08:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.830+0000 7f4280c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427c0cf000 con 0x7f427c06d2c0 2024-04-25T09:08:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.830+0000 7f4280c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0cf2e0 con 0x7f427c0b31a0 2024-04-25T09:08:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.831+0000 7f427aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f427c0ccde0 0x7f427c0ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48618/0 (socket says 172.21.15.62:48618) 2024-04-25T09:08:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.831+0000 7f427aa00640 1 -- 172.21.15.62:0/3627544591 learned_addr learned my addr 172.21.15.62:0/3627544591 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.831+0000 7f427be00640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0b31a0 0x7f427c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.831+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384119203 0 0) 0x7f427c0cd260 con 0x7f427c0ccde0 2024-04-25T09:08:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.831+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4260003520 con 0x7f427c0ccde0 2024-04-25T09:08:18.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 415875482 0 0) 0x7f4260003520 con 0x7f427c0ccde0 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0b31a0 msgr2=0x7f427c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0b31a0 0x7f427c0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 >> v1:172.21.15.121:6789/0 conn(0x7f427c06d2c0 legacy=0x7f427c06d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0b26c0 con 0x7f427c0ccde0 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f4268001670 con 0x7f427c0ccde0 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 >> v1:172.21.15.62:6789/0 conn(0x7f427c0ccde0 legacy=0x7f427c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42600039e0 0x7f4260003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4260004800 0x7f4260004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0b31a0 0x7f42600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0cf2e0 con 0x7f4260004800 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426000bad0 con 0x7f42600039e0 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426000bcd0 con 0x7f427c0b31a0 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427aa00640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4260004800 0x7f4260004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.832+0000 7f427b400640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42600039e0 0x7f4260003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.833+0000 7f427aa00640 1 -- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0b31a0 msgr2=0x7f42600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.833+0000 7f427aa00640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f427c0b31a0 0x7f42600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.833+0000 7f427aa00640 1 -- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42600039e0 msgr2=0x7f4260003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.833+0000 7f427aa00640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42600039e0 0x7f4260003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.833+0000 7f427aa00640 1 -- 172.21.15.62:0/3627544591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260003520 con 0x7f4260004800 2024-04-25T09:08:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.833+0000 7f427b400640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f42600039e0 0x7f4260003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.834+0000 7f427aa00640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4260004800 0x7f4260004c80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f426800c160 tx=0x7f4260005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263f26e8df2ae33c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.834+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268007c60 con 0x7f4260004800 2024-04-25T09:08:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.834+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4268005ad0 con 0x7f4260004800 2024-04-25T09:08:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.834+0000 7f427a000640 1 -- 172.21.15.62:0/3627544591 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268007070 con 0x7f4260004800 2024-04-25T09:08:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.835+0000 7f4280c00640 1 -- 172.21.15.62:0/3627544591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4260004800 msgr2=0x7f4260004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.835+0000 7f4280c00640 1 --2- 172.21.15.62:0/3627544591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4260004800 0x7f4260004c80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f426800c160 tx=0x7f4260005830 comp rx=0 tx=0).stop 2024-04-25T09:08:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.835+0000 7f427be00640 1 -- 172.21.15.62:0/3627544591 reap_dead start 2024-04-25T09:08:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.835+0000 7f4280c00640 1 -- 172.21.15.62:0/3627544591 shutdown_connections 2024-04-25T09:08:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.835+0000 7f4280c00640 1 -- 172.21.15.62:0/3627544591 >> 172.21.15.62:0/3627544591 conn(0x7f427c066f20 msgr2=0x7f427c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.835+0000 7f4280c00640 1 -- 172.21.15.62:0/3627544591 shutdown_connections 2024-04-25T09:08:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.835+0000 7f4280c00640 1 -- 172.21.15.62:0/3627544591 wait complete. 2024-04-25T09:08:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.837+0000 7f4280c00640 1 Processor -- start 2024-04-25T09:08:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.837+0000 7f4280c00640 1 -- start start 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f4280c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4260004800 0x7f427c0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f4280c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f427c0cbb70 0x7f427c0a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f4280c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c141c50 0x7f427c1420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f4280c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c1448d0 con 0x7f427c0cbb70 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f4280c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0b2c80 con 0x7f427c141c50 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f4280c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427c144a50 con 0x7f4260004800 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f427aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f427c0cbb70 0x7f427c0a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f427aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f427c0cbb70 0x7f427c0a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60440/0 (socket says 172.21.15.62:60440) 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f427aa00640 1 -- 172.21.15.62:0/269133825 learned_addr learned my addr 172.21.15.62:0/269133825 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.838+0000 7f427b400640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4260004800 0x7f427c0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f427be00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c141c50 0x7f427c1420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f427aa00640 1 -- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4260004800 msgr2=0x7f427c0a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f427aa00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4260004800 0x7f427c0a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f427aa00640 1 -- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c141c50 msgr2=0x7f427c1420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f427aa00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c141c50 0x7f427c1420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f427aa00640 1 -- 172.21.15.62:0/269133825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0aa040 con 0x7f427c0cbb70 2024-04-25T09:08:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f427b400640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4260004800 0x7f427c0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f427aa00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f427c0cbb70 0x7f427c0a9a10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f426802ad20 tx=0x7f4268001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f4273400640 1 -- 172.21.15.62:0/269133825 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268040020 con 0x7f427c0cbb70 2024-04-25T09:08:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f4273400640 1 -- 172.21.15.62:0/269133825 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42680013b0 con 0x7f427c0cbb70 2024-04-25T09:08:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.839+0000 7f4273400640 1 -- 172.21.15.62:0/269133825 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268037040 con 0x7f427c0cbb70 2024-04-25T09:08:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.840+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427c0aa320 con 0x7f427c0cbb70 2024-04-25T09:08:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.840+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c12fbb0 con 0x7f427c0cbb70 2024-04-25T09:08:18.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.841+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427c06abe0 con 0x7f427c0cbb70 2024-04-25T09:08:18.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.842+0000 7f4273400640 1 -- 172.21.15.62:0/269133825 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4268043020 con 0x7f427c0cbb70 2024-04-25T09:08:18.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.843+0000 7f4273400640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f425c06b4d0 0x7f425c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:18.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.843+0000 7f4273400640 1 -- 172.21.15.62:0/269133825 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f42680a9ee0 con 0x7f427c0cbb70 2024-04-25T09:08:18.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.844+0000 7f427b400640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f425c06b4d0 0x7f425c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:18.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.845+0000 7f427b400640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f425c06b4d0 0x7f425c06d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f426c000c00 tx=0x7f426c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:18.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:18.848+0000 7f4273400640 1 -- 172.21.15.62:0/269133825 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4268078d80 con 0x7f427c0cbb70 2024-04-25T09:08:19.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.045+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f427c04ef20 con 0x7f427c0cbb70 2024-04-25T09:08:19.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.046+0000 7f4273400640 1 -- 172.21.15.62:0/269133825 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4268078680 con 0x7f427c0cbb70 2024-04-25T09:08:19.034 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:08:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f425c06b4d0 msgr2=0x7f425c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f425c06b4d0 0x7f425c06d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f426c000c00 tx=0x7f426c009040 comp rx=0 tx=0).stop 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f427c0cbb70 msgr2=0x7f427c0a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f427c0cbb70 0x7f427c0a9a10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f426802ad20 tx=0x7f4268001970 comp rx=0 tx=0).stop 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 shutdown_connections 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f425c06b4d0 0x7f425c06d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f427c0cbb70 0x7f427c0a9a10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c141c50 0x7f427c1420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 --2- 172.21.15.62:0/269133825 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4260004800 0x7f427c0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 >> 172.21.15.62:0/269133825 conn(0x7f427c066f20 msgr2=0x7f427c0aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 shutdown_connections 2024-04-25T09:08:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.049+0000 7f4280c00640 1 -- 172.21.15.62:0/269133825 wait complete. 2024-04-25T09:08:19.057 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:08:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.278+0000 7f73ce200640 1 Processor -- start 2024-04-25T09:08:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.278+0000 7f73ce200640 1 -- start start 2024-04-25T09:08:19.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.280+0000 7f73ce200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f73c806d2c0 0x7f73c80ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.280+0000 7f73ce200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f73c80b31a0 0x7f73c80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.280+0000 7f73ce200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c80cf000 con 0x7f73c806d6a0 2024-04-25T09:08:19.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.280+0000 7f73ce200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c80cf2e0 con 0x7f73c80b31a0 2024-04-25T09:08:19.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.280+0000 7f73ce200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c80b26c0 con 0x7f73c806d2c0 2024-04-25T09:08:19.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.280+0000 7f73c7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f73c806d6a0 0x7f73c80ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48624/0 (socket says 172.21.15.62:48624) 2024-04-25T09:08:19.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.280+0000 7f73c7e00640 1 -- 172.21.15.62:0/1688931500 learned_addr learned my addr 172.21.15.62:0/1688931500 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:19.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.280+0000 7f73cd800640 1 --2- 172.21.15.62:0/1688931500 >> v2:172.21.15.62:3300/0 conn(0x7f73c80b31a0 0x7f73c80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.281+0000 7f73cce00640 1 --2- 172.21.15.62:0/1688931500 >> v2:172.21.15.121:3300/0 conn(0x7f73c806d2c0 0x7f73c80ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.281+0000 7f73c7400640 1 -- 172.21.15.62:0/1688931500 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825047982 0 0) 0x7f73c80cf000 con 0x7f73c806d6a0 2024-04-25T09:08:19.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.281+0000 7f73cd800640 1 -- 172.21.15.62:0/1688931500 >> v2:172.21.15.121:3300/0 conn(0x7f73c806d2c0 msgr2=0x7f73c80ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.281+0000 7f73cd800640 1 --2- 172.21.15.62:0/1688931500 >> v2:172.21.15.121:3300/0 conn(0x7f73c806d2c0 0x7f73c80ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.281+0000 7f73cd800640 1 -- 172.21.15.62:0/1688931500 >> v1:172.21.15.62:6789/0 conn(0x7f73c806d6a0 legacy=0x7f73c80ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.281+0000 7f73cd800640 1 -- 172.21.15.62:0/1688931500 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80b29a0 con 0x7f73c80b31a0 2024-04-25T09:08:19.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.282+0000 7f73cd800640 1 --2- 172.21.15.62:0/1688931500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c80b31a0 0x7f73c80b3f40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f73c80b20a0 tx=0x7f73c002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbeaad4d68a27bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.282+0000 7f73c7400640 1 -- 172.21.15.62:0/1688931500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0007740 con 0x7f73c80b31a0 2024-04-25T09:08:19.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.282+0000 7f73c7400640 1 -- 172.21.15.62:0/1688931500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c00078e0 con 0x7f73c80b31a0 2024-04-25T09:08:19.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.282+0000 7f73c7400640 1 -- 172.21.15.62:0/1688931500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c0007cb0 con 0x7f73c80b31a0 2024-04-25T09:08:19.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.283+0000 7f73ce200640 1 -- 172.21.15.62:0/1688931500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c80b31a0 msgr2=0x7f73c80b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.283+0000 7f73ce200640 1 --2- 172.21.15.62:0/1688931500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c80b31a0 0x7f73c80b3f40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f73c80b20a0 tx=0x7f73c002a680 comp rx=0 tx=0).stop 2024-04-25T09:08:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.283+0000 7f73ce200640 1 -- 172.21.15.62:0/1688931500 shutdown_connections 2024-04-25T09:08:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.283+0000 7f73ce200640 1 --2- 172.21.15.62:0/1688931500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c80b31a0 0x7f73c80b3f40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.283+0000 7f73ce200640 1 --2- 172.21.15.62:0/1688931500 >> v2:172.21.15.121:3300/0 conn(0x7f73c806d2c0 0x7f73c80ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.283+0000 7f73ce200640 1 -- 172.21.15.62:0/1688931500 >> 172.21.15.62:0/1688931500 conn(0x7f73c8066f20 msgr2=0x7f73c80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.283+0000 7f73ce200640 1 -- 172.21.15.62:0/1688931500 shutdown_connections 2024-04-25T09:08:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.283+0000 7f73ce200640 1 -- 172.21.15.62:0/1688931500 wait complete. 2024-04-25T09:08:19.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.285+0000 7f73ce200640 1 Processor -- start 2024-04-25T09:08:19.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.285+0000 7f73ce200640 1 -- start start 2024-04-25T09:08:19.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73ce200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c806d2c0 0x7f73c815b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73ce200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c806d6a0 0x7f73c81354c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73ce200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73c80b31a0 0x7f73c81e8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73ce200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81448d0 con 0x7f73c806d6a0 2024-04-25T09:08:19.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73ce200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c806db20 con 0x7f73c806d2c0 2024-04-25T09:08:19.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73ce200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8144a50 con 0x7f73c80b31a0 2024-04-25T09:08:19.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c806d6a0 0x7f73c81354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c806d6a0 0x7f73c81354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60462/0 (socket says 172.21.15.62:60462) 2024-04-25T09:08:19.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.286+0000 7f73c7e00640 1 -- 172.21.15.62:0/3579398987 learned_addr learned my addr 172.21.15.62:0/3579398987 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:19.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73cce00640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c806d2c0 0x7f73c815b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73cd800640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73c80b31a0 0x7f73c81e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73c7e00640 1 -- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73c80b31a0 msgr2=0x7f73c81e8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73c7e00640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73c80b31a0 0x7f73c81e8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73c7e00640 1 -- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c806d2c0 msgr2=0x7f73c815b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73c7e00640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c806d2c0 0x7f73c815b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73c7e00640 1 -- 172.21.15.62:0/3579398987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c8134420 con 0x7f73c806d6a0 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73cd800640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73c80b31a0 0x7f73c81e8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73cce00640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c806d2c0 0x7f73c815b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.287+0000 7f73c7e00640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c806d6a0 0x7f73c81354c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f73b400c4d0 tx=0x7f73b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.288+0000 7f73c4c00640 1 -- 172.21.15.62:0/3579398987 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4018070 con 0x7f73c806d6a0 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.288+0000 7f73c4c00640 1 -- 172.21.15.62:0/3579398987 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b4007500 con 0x7f73c806d6a0 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.288+0000 7f73c4c00640 1 -- 172.21.15.62:0/3579398987 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4004a20 con 0x7f73c806d6a0 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.288+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c8134700 con 0x7f73c806d6a0 2024-04-25T09:08:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.288+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c8134cc0 con 0x7f73c806d6a0 2024-04-25T09:08:19.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.289+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c804ef20 con 0x7f73c806d6a0 2024-04-25T09:08:19.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.290+0000 7f73c4c00640 1 -- 172.21.15.62:0/3579398987 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73b4004bc0 con 0x7f73c806d6a0 2024-04-25T09:08:19.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.292+0000 7f73c4c00640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73a806b4d0 0x7f73a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.292+0000 7f73c4c00640 1 -- 172.21.15.62:0/3579398987 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f73b40895b0 con 0x7f73c806d6a0 2024-04-25T09:08:19.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.292+0000 7f73cce00640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73a806b4d0 0x7f73a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.293+0000 7f73cce00640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73a806b4d0 0x7f73a806d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73bc006fd0 tx=0x7f73bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.297+0000 7f73c4c00640 1 -- 172.21.15.62:0/3579398987 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73b4058390 con 0x7f73c806d6a0 2024-04-25T09:08:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.501+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73c806b210 con 0x7f73c806d6a0 2024-04-25T09:08:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.501+0000 7f73c4c00640 1 -- 172.21.15.62:0/3579398987 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7f73b4057c90 con 0x7f73c806d6a0 2024-04-25T09:08:19.490 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:08:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73a806b4d0 msgr2=0x7f73a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73a806b4d0 0x7f73a806d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73bc006fd0 tx=0x7f73bc008040 comp rx=0 tx=0).stop 2024-04-25T09:08:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c806d6a0 msgr2=0x7f73c81354c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c806d6a0 0x7f73c81354c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f73b400c4d0 tx=0x7f73b400c990 comp rx=0 tx=0).stop 2024-04-25T09:08:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 shutdown_connections 2024-04-25T09:08:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73a806b4d0 0x7f73a806d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73c806d6a0 0x7f73c81354c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73c80b31a0 0x7f73c81e8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 --2- 172.21.15.62:0/3579398987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73c806d2c0 0x7f73c815b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 >> 172.21.15.62:0/3579398987 conn(0x7f73c8066f20 msgr2=0x7f73c80afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 shutdown_connections 2024-04-25T09:08:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.504+0000 7f73ce200640 1 -- 172.21.15.62:0/3579398987 wait complete. 2024-04-25T09:08:19.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:19.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.717+0000 7f3430c00640 1 Processor -- start 2024-04-25T09:08:19.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.717+0000 7f3430c00640 1 -- start start 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.719+0000 7f3430c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f342c1584d0 0x7f342c15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.719+0000 7f3430c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f342c0b5190 0x7f342c070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.719+0000 7f3430c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0b30e0 con 0x7f342c0b4510 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.719+0000 7f3430c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c0cfc40 con 0x7f342c1584d0 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.719+0000 7f3430c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c0b33c0 con 0x7f342c0b5190 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.719+0000 7f342b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f342c1584d0 0x7f342c15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.719+0000 7f342b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f342c1584d0 0x7f342c15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60472/0 (socket says 172.21.15.62:60472) 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.719+0000 7f342b400640 1 -- 172.21.15.62:0/3407385422 learned_addr learned my addr 172.21.15.62:0/3407385422 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.720+0000 7f342be00640 1 --2- 172.21.15.62:0/3407385422 >> v2:172.21.15.121:3300/0 conn(0x7f342c0b5190 0x7f342c070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.720+0000 7f342b400640 1 -- 172.21.15.62:0/3407385422 >> v2:172.21.15.121:3300/0 conn(0x7f342c0b5190 msgr2=0x7f342c070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:19.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.720+0000 7f342b400640 1 --2- 172.21.15.62:0/3407385422 >> v2:172.21.15.121:3300/0 conn(0x7f342c0b5190 0x7f342c070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.720+0000 7f342b400640 1 -- 172.21.15.62:0/3407385422 >> v1:172.21.15.62:6789/0 conn(0x7f342c0b4510 legacy=0x7f342c0b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.720+0000 7f342b400640 1 -- 172.21.15.62:0/3407385422 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0b36a0 con 0x7f342c1584d0 2024-04-25T09:08:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.720+0000 7f342be00640 1 --2- 172.21.15.62:0/3407385422 >> v2:172.21.15.121:3300/0 conn(0x7f342c0b5190 0x7f342c070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.720+0000 7f342aa00640 1 -- 172.21.15.62:0/3407385422 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203098487 0 0) 0x7f342c0b30e0 con 0x7f342c0b4510 2024-04-25T09:08:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.721+0000 7f342b400640 1 --2- 172.21.15.62:0/3407385422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c1584d0 0x7f342c15b7b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f342c0744a0 tx=0x7f341c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199e05174448db39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.721+0000 7f342aa00640 1 -- 172.21.15.62:0/3407385422 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c037070 con 0x7f342c1584d0 2024-04-25T09:08:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.721+0000 7f342aa00640 1 -- 172.21.15.62:0/3407385422 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c032440 con 0x7f342c1584d0 2024-04-25T09:08:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.721+0000 7f342aa00640 1 -- 172.21.15.62:0/3407385422 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c032810 con 0x7f342c1584d0 2024-04-25T09:08:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.721+0000 7f3430c00640 1 -- 172.21.15.62:0/3407385422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c1584d0 msgr2=0x7f342c15b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.721+0000 7f3430c00640 1 --2- 172.21.15.62:0/3407385422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c1584d0 0x7f342c15b7b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f342c0744a0 tx=0x7f341c02a200 comp rx=0 tx=0).stop 2024-04-25T09:08:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.722+0000 7f3430c00640 1 -- 172.21.15.62:0/3407385422 shutdown_connections 2024-04-25T09:08:19.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.722+0000 7f3430c00640 1 --2- 172.21.15.62:0/3407385422 >> v2:172.21.15.121:3300/0 conn(0x7f342c0b5190 0x7f342c070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.722+0000 7f3430c00640 1 --2- 172.21.15.62:0/3407385422 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c1584d0 0x7f342c15b7b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.722+0000 7f3430c00640 1 -- 172.21.15.62:0/3407385422 >> 172.21.15.62:0/3407385422 conn(0x7f342c06b350 msgr2=0x7f342c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.722+0000 7f3430c00640 1 -- 172.21.15.62:0/3407385422 shutdown_connections 2024-04-25T09:08:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.722+0000 7f3430c00640 1 -- 172.21.15.62:0/3407385422 wait complete. 2024-04-25T09:08:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.723+0000 7f3430c00640 1 Processor -- start 2024-04-25T09:08:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.724+0000 7f3430c00640 1 -- start start 2024-04-25T09:08:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.724+0000 7f3430c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f342c0b4510 0x7f342c0aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.724+0000 7f3430c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c0b5190 0x7f342c0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.724+0000 7f3430c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f342c148f00 0x7f342c071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3430c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c071010 con 0x7f342c0b5190 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3430c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0b3c60 con 0x7f342c0b4510 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3430c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c071190 con 0x7f342c148f00 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3423e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c0b5190 0x7f342c0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3423e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c0b5190 0x7f342c0714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60478/0 (socket says 172.21.15.62:60478) 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3423e00640 1 -- 172.21.15.62:0/165175941 learned_addr learned my addr 172.21.15.62:0/165175941 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f342b400640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f342c0b4510 0x7f342c0aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f342be00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f342c148f00 0x7f342c071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3423e00640 1 -- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f342c148f00 msgr2=0x7f342c071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3423e00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f342c148f00 0x7f342c071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3423e00640 1 -- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f342c0b4510 msgr2=0x7f342c0aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3423e00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f342c0b4510 0x7f342c0aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f3423e00640 1 -- 172.21.15.62:0/165175941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c14c430 con 0x7f342c0b5190 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.725+0000 7f342b400640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f342c0b4510 0x7f342c0aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.726+0000 7f3423e00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c0b5190 0x7f342c0714c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f341400c4b0 tx=0x7f341400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.726+0000 7f3423400640 1 -- 172.21.15.62:0/165175941 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414017070 con 0x7f342c0b5190 2024-04-25T09:08:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.726+0000 7f3423400640 1 -- 172.21.15.62:0/165175941 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3414005480 con 0x7f342c0b5190 2024-04-25T09:08:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.726+0000 7f3423400640 1 -- 172.21.15.62:0/165175941 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414003830 con 0x7f342c0b5190 2024-04-25T09:08:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.726+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c14b670 con 0x7f342c0b5190 2024-04-25T09:08:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.726+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c14bc30 con 0x7f342c0b5190 2024-04-25T09:08:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.727+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c04ef20 con 0x7f342c0b5190 2024-04-25T09:08:19.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.728+0000 7f3423400640 1 -- 172.21.15.62:0/165175941 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f341401a070 con 0x7f342c0b5190 2024-04-25T09:08:19.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.730+0000 7f3423400640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.731+0000 7f3423400640 1 -- 172.21.15.62:0/165175941 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f34140896b0 con 0x7f342c0b5190 2024-04-25T09:08:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.731+0000 7f342b400640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:19.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.732+0000 7f342b400640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f340806b4d0 0x7f340806d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f341c034000 tx=0x7f341c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:19.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.734+0000 7f3423400640 1 -- 172.21.15.62:0/165175941 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3414058400 con 0x7f342c0b5190 2024-04-25T09:08:19.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.961+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f342c1f7640 con 0x7f342c0b5190 2024-04-25T09:08:19.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.962+0000 7f3423400640 1 -- 172.21.15.62:0/165175941 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f3414057d00 con 0x7f342c0b5190 2024-04-25T09:08:19.951 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:19.951 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:08:18.032398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:19.951 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-04-25T09:08:19.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.964+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f340806b4d0 msgr2=0x7f340806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.964+0000 7f3430c00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f340806b4d0 0x7f340806d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f341c034000 tx=0x7f341c02e000 comp rx=0 tx=0).stop 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.964+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c0b5190 msgr2=0x7f342c0714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.964+0000 7f3430c00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c0b5190 0x7f342c0714c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f341400c4b0 tx=0x7f341400c970 comp rx=0 tx=0).stop 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.965+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 shutdown_connections 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.965+0000 7f3430c00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.965+0000 7f3430c00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f342c0b5190 0x7f342c0714c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.965+0000 7f3430c00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f342c148f00 0x7f342c071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.965+0000 7f3430c00640 1 --2- 172.21.15.62:0/165175941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f342c0b4510 0x7f342c0aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.965+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 >> 172.21.15.62:0/165175941 conn(0x7f342c06b350 msgr2=0x7f342c15e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.965+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 shutdown_connections 2024-04-25T09:08:19.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:19.965+0000 7f3430c00640 1 -- 172.21.15.62:0/165175941 wait complete. 2024-04-25T09:08:19.971 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:20.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.176+0000 7f3b62400640 1 Processor -- start 2024-04-25T09:08:20.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.176+0000 7f3b62400640 1 -- start start 2024-04-25T09:08:20.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.178+0000 7f3b62400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b5c144c10 0x7f3b5c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.178+0000 7f3b62400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 0x7f3b5c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.178+0000 7f3b62400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5c15faa0 con 0x7f3b5c06d730 2024-04-25T09:08:20.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.178+0000 7f3b62400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5c144ff0 con 0x7f3b5c144c10 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.178+0000 7f3b62400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c160340 con 0x7f3b5c0cee40 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.178+0000 7f3b61000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b5c144c10 0x7f3b5c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.178+0000 7f3b61000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b5c144c10 0x7f3b5c15f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60492/0 (socket says 172.21.15.62:60492) 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.178+0000 7f3b61000640 1 -- 172.21.15.62:0/2179047922 learned_addr learned my addr 172.21.15.62:0/2179047922 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.179+0000 7f3b53e00640 1 --2- 172.21.15.62:0/2179047922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 0x7f3b5c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.179+0000 7f3b61000640 1 -- 172.21.15.62:0/2179047922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 msgr2=0x7f3b5c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.179+0000 7f3b61000640 1 --2- 172.21.15.62:0/2179047922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 0x7f3b5c06d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.179+0000 7f3b61000640 1 -- 172.21.15.62:0/2179047922 >> v1:172.21.15.121:6789/0 conn(0x7f3b5c06d730 legacy=0x7f3b5c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.179+0000 7f3b61000640 1 -- 172.21.15.62:0/2179047922 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c15fd80 con 0x7f3b5c144c10 2024-04-25T09:08:20.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.179+0000 7f3b53e00640 1 --2- 172.21.15.62:0/2179047922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 0x7f3b5c06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:20.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.180+0000 7f3b61000640 1 --2- 172.21.15.62:0/2179047922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c144c10 0x7f3b5c15f450 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009e50 tx=0x7f3b4c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1c82aa231e7501 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:20.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.180+0000 7f3b5be00640 1 -- 172.21.15.62:0/2179047922 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c036070 con 0x7f3b5c144c10 2024-04-25T09:08:20.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.180+0000 7f3b5be00640 1 -- 172.21.15.62:0/2179047922 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c031600 con 0x7f3b5c144c10 2024-04-25T09:08:20.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.180+0000 7f3b5be00640 1 -- 172.21.15.62:0/2179047922 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0319d0 con 0x7f3b5c144c10 2024-04-25T09:08:20.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.181+0000 7f3b62400640 1 -- 172.21.15.62:0/2179047922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c144c10 msgr2=0x7f3b5c15f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.181+0000 7f3b62400640 1 --2- 172.21.15.62:0/2179047922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c144c10 0x7f3b5c15f450 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009e50 tx=0x7f3b4c02a640 comp rx=0 tx=0).stop 2024-04-25T09:08:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.181+0000 7f3b62400640 1 -- 172.21.15.62:0/2179047922 shutdown_connections 2024-04-25T09:08:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.181+0000 7f3b62400640 1 --2- 172.21.15.62:0/2179047922 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 0x7f3b5c06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.181+0000 7f3b62400640 1 --2- 172.21.15.62:0/2179047922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c144c10 0x7f3b5c15f450 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.181+0000 7f3b62400640 1 -- 172.21.15.62:0/2179047922 >> 172.21.15.62:0/2179047922 conn(0x7f3b5c0b14d0 msgr2=0x7f3b5c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.181+0000 7f3b62400640 1 -- 172.21.15.62:0/2179047922 shutdown_connections 2024-04-25T09:08:20.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.181+0000 7f3b62400640 1 -- 172.21.15.62:0/2179047922 wait complete. 2024-04-25T09:08:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.183+0000 7f3b62400640 1 Processor -- start 2024-04-25T09:08:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.183+0000 7f3b62400640 1 -- start start 2024-04-25T09:08:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b62400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c06d730 0x7f3b5c15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b62400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 0x7f3b5c15c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b62400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b5c144c10 0x7f3b5c1f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b62400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0cf2a0 con 0x7f3b5c06d730 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b62400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0b41c0 con 0x7f3b5c144c10 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b62400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0b5660 con 0x7f3b5c0cee40 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b61000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c06d730 0x7f3b5c15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b61000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c06d730 0x7f3b5c15be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60508/0 (socket says 172.21.15.62:60508) 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b61000640 1 -- 172.21.15.62:0/3966719696 learned_addr learned my addr 172.21.15.62:0/3966719696 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b61000640 1 -- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 msgr2=0x7f3b5c15c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b61000640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 0x7f3b5c15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b61000640 1 -- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b5c144c10 msgr2=0x7f3b5c1f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b61000640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b5c144c10 0x7f3b5c1f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.184+0000 7f3b61000640 1 -- 172.21.15.62:0/3966719696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c1435f0 con 0x7f3b5c06d730 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.185+0000 7f3b61000640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c06d730 0x7f3b5c15be90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c034040 tx=0x7f3b4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:20.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.185+0000 7f3b59600640 1 -- 172.21.15.62:0/3966719696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c036050 con 0x7f3b5c06d730 2024-04-25T09:08:20.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.185+0000 7f3b59600640 1 -- 172.21.15.62:0/3966719696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c042070 con 0x7f3b5c06d730 2024-04-25T09:08:20.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.185+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c138d30 con 0x7f3b5c06d730 2024-04-25T09:08:20.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.185+0000 7f3b59600640 1 -- 172.21.15.62:0/3966719696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c009510 con 0x7f3b5c06d730 2024-04-25T09:08:20.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.185+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c1392f0 con 0x7f3b5c06d730 2024-04-25T09:08:20.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.186+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c04ef20 con 0x7f3b5c06d730 2024-04-25T09:08:20.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.187+0000 7f3b59600640 1 -- 172.21.15.62:0/3966719696 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b4c007c50 con 0x7f3b5c06d730 2024-04-25T09:08:20.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.188+0000 7f3b59600640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b3806b4d0 0x7f3b3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.188+0000 7f3b59600640 1 -- 172.21.15.62:0/3966719696 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3b4c0ab2e0 con 0x7f3b5c06d730 2024-04-25T09:08:20.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.189+0000 7f3b53e00640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b3806b4d0 0x7f3b3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.190+0000 7f3b53e00640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b3806b4d0 0x7f3b3806d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3b44006fd0 tx=0x7f3b44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:20.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.194+0000 7f3b59600640 1 -- 172.21.15.62:0/3966719696 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b4c02ae20 con 0x7f3b5c06d730 2024-04-25T09:08:20.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.393+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b5c140640 con 0x7f3b5c06d730 2024-04-25T09:08:20.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.396+0000 7f3b59600640 1 -- 172.21.15.62:0/3966719696 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f3b4c045360 con 0x7f3b5c06d730 2024-04-25T09:08:20.385 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:20.385 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":50,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:01.219227+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:07:23.299693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T09:07:24.475140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":0}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b3806b4d0 msgr2=0x7f3b3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b3806b4d0 0x7f3b3806d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3b44006fd0 tx=0x7f3b44008040 comp rx=0 tx=0).stop 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c06d730 msgr2=0x7f3b5c15be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c06d730 0x7f3b5c15be90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c034040 tx=0x7f3b4c004a40 comp rx=0 tx=0).stop 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 shutdown_connections 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b3806b4d0 0x7f3b3806d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b5c144c10 0x7f3b5c1f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b5c0cee40 0x7f3b5c15c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 --2- 172.21.15.62:0/3966719696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b5c06d730 0x7f3b5c15be90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.399+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 >> 172.21.15.62:0/3966719696 conn(0x7f3b5c0b14d0 msgr2=0x7f3b5c0cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:20.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.400+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 shutdown_connections 2024-04-25T09:08:20.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.400+0000 7f3b62400640 1 -- 172.21.15.62:0/3966719696 wait complete. 2024-04-25T09:08:20.408 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:08:20.408 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.622+0000 7fbd9f400640 1 Processor -- start 2024-04-25T09:08:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.622+0000 7fbd9f400640 1 -- start start 2024-04-25T09:08:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd9f400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd9815bd10 0x7fbd980cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd9f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbd980cccf0 0x7fbd980ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd9f400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9806ca70 con 0x7fbd980aed00 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd9f400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9806d660 con 0x7fbd980cccf0 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd9f400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980ce6d0 con 0x7fbd9815bd10 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd97e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbd980cccf0 0x7fbd980ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd97e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbd980cccf0 0x7fbd980ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60514/0 (socket says 172.21.15.62:60514) 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd9d400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbd980aed00 0x7fbd9815f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48646/0 (socket says 172.21.15.62:48646) 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.624+0000 7fbd97e00640 1 -- 172.21.15.62:0/2173865111 learned_addr learned my addr 172.21.15.62:0/2173865111 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:20.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.625+0000 7fbd9ca00640 1 --2- 172.21.15.62:0/2173865111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd9815bd10 0x7fbd980cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.625+0000 7fbd97e00640 1 -- 172.21.15.62:0/2173865111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd9815bd10 msgr2=0x7fbd980cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:20.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.625+0000 7fbd97e00640 1 --2- 172.21.15.62:0/2173865111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd9815bd10 0x7fbd980cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.625+0000 7fbd97e00640 1 -- 172.21.15.62:0/2173865111 >> v1:172.21.15.62:6789/0 conn(0x7fbd980aed00 legacy=0x7fbd9815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.625+0000 7fbd97e00640 1 -- 172.21.15.62:0/2173865111 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9815c0f0 con 0x7fbd980cccf0 2024-04-25T09:08:20.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.625+0000 7fbd97400640 1 -- 172.21.15.62:0/2173865111 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260184223 0 0) 0x7fbd9806ca70 con 0x7fbd980aed00 2024-04-25T09:08:20.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.626+0000 7fbd97e00640 1 --2- 172.21.15.62:0/2173865111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd980cccf0 0x7fbd980ae7c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbd9806d7e0 tx=0x7fbd8402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f162090c70fbccd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.627+0000 7fbd97400640 1 -- 172.21.15.62:0/2173865111 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84037070 con 0x7fbd980cccf0 2024-04-25T09:08:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.627+0000 7fbd97400640 1 -- 172.21.15.62:0/2173865111 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd84032440 con 0x7fbd980cccf0 2024-04-25T09:08:20.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.627+0000 7fbd97400640 1 -- 172.21.15.62:0/2173865111 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84032810 con 0x7fbd980cccf0 2024-04-25T09:08:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.627+0000 7fbd9f400640 1 -- 172.21.15.62:0/2173865111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd980cccf0 msgr2=0x7fbd980ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.627+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2173865111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd980cccf0 0x7fbd980ae7c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbd9806d7e0 tx=0x7fbd8402a250 comp rx=0 tx=0).stop 2024-04-25T09:08:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.628+0000 7fbd9f400640 1 -- 172.21.15.62:0/2173865111 shutdown_connections 2024-04-25T09:08:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.628+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2173865111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd980cccf0 0x7fbd980ae7c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.628+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2173865111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd9815bd10 0x7fbd980cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.628+0000 7fbd9f400640 1 -- 172.21.15.62:0/2173865111 >> 172.21.15.62:0/2173865111 conn(0x7fbd980668f0 msgr2=0x7fbd98066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.628+0000 7fbd9f400640 1 -- 172.21.15.62:0/2173865111 shutdown_connections 2024-04-25T09:08:20.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.628+0000 7fbd9f400640 1 -- 172.21.15.62:0/2173865111 wait complete. 2024-04-25T09:08:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.629+0000 7fbd9f400640 1 Processor -- start 2024-04-25T09:08:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.629+0000 7fbd9f400640 1 -- start start 2024-04-25T09:08:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.630+0000 7fbd9f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd980aed00 0x7fbd9814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.630+0000 7fbd9f400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd980cccf0 0x7fbd980aa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.630+0000 7fbd9f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd9815bd10 0x7fbd980aa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.630+0000 7fbd9f400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9806d2f0 con 0x7fbd9815bd10 2024-04-25T09:08:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.630+0000 7fbd9f400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9806d010 con 0x7fbd980aed00 2024-04-25T09:08:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.630+0000 7fbd9f400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980d01e0 con 0x7fbd980cccf0 2024-04-25T09:08:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd9815bd10 0x7fbd980aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd9815bd10 0x7fbd980aa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60530/0 (socket says 172.21.15.62:60530) 2024-04-25T09:08:20.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9d400640 1 -- 172.21.15.62:0/2350390845 learned_addr learned my addr 172.21.15.62:0/2350390845 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd97e00640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd980cccf0 0x7fbd980aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9ca00640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd980aed00 0x7fbd9814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9d400640 1 -- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd980cccf0 msgr2=0x7fbd980aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9d400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd980cccf0 0x7fbd980aa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9d400640 1 -- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd980aed00 msgr2=0x7fbd9814d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9d400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd980aed00 0x7fbd9814d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.631+0000 7fbd9d400640 1 -- 172.21.15.62:0/2350390845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9814b1e0 con 0x7fbd9815bd10 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.632+0000 7fbd9d400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd9815bd10 0x7fbd980aa790 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000c7a0 tx=0x7fbd9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.632+0000 7fbd94c00640 1 -- 172.21.15.62:0/2350390845 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90018070 con 0x7fbd9815bd10 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.632+0000 7fbd94c00640 1 -- 172.21.15.62:0/2350390845 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9000a640 con 0x7fbd9815bd10 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.632+0000 7fbd94c00640 1 -- 172.21.15.62:0/2350390845 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000aab0 con 0x7fbd9815bd10 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.632+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9814b4c0 con 0x7fbd9815bd10 2024-04-25T09:08:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.632+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd980aacd0 con 0x7fbd9815bd10 2024-04-25T09:08:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.633+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9814b8b0 con 0x7fbd9815bd10 2024-04-25T09:08:20.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.634+0000 7fbd94c00640 1 -- 172.21.15.62:0/2350390845 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd9001b070 con 0x7fbd9815bd10 2024-04-25T09:08:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.635+0000 7fbd94c00640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7806b4d0 0x7fbd7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.635+0000 7fbd94c00640 1 -- 172.21.15.62:0/2350390845 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbd90089970 con 0x7fbd9815bd10 2024-04-25T09:08:20.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.636+0000 7fbd9ca00640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7806b4d0 0x7fbd7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:20.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.637+0000 7fbd9ca00640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7806b4d0 0x7fbd7806d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c006fd0 tx=0x7fbd8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:20.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.639+0000 7fbd94c00640 1 -- 172.21.15.62:0/2350390845 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd900586c0 con 0x7fbd9815bd10 2024-04-25T09:08:20.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.878+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd98072c70 con 0x7fbd9815bd10 2024-04-25T09:08:20.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.879+0000 7fbd94c00640 1 -- 172.21.15.62:0/2350390845 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fbd90057fc0 con 0x7fbd9815bd10 2024-04-25T09:08:20.868 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:20.868 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:08:18.032398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:20.868 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7806b4d0 msgr2=0x7fbd7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7806b4d0 0x7fbd7806d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c006fd0 tx=0x7fbd8c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd9815bd10 msgr2=0x7fbd980aa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd9815bd10 0x7fbd980aa790 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000c7a0 tx=0x7fbd9000cc60 comp rx=0 tx=0).stop 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 shutdown_connections 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7806b4d0 0x7fbd7806d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd9815bd10 0x7fbd980aa790 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd980cccf0 0x7fbd980aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 --2- 172.21.15.62:0/2350390845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd980aed00 0x7fbd9814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.882+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 >> 172.21.15.62:0/2350390845 conn(0x7fbd980668f0 msgr2=0x7fbd9815e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.883+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 shutdown_connections 2024-04-25T09:08:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:20.883+0000 7fbd9f400640 1 -- 172.21.15.62:0/2350390845 wait complete. 2024-04-25T09:08:20.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:21.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.111+0000 7fd82ec00640 1 Processor -- start 2024-04-25T09:08:21.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.111+0000 7fd82ec00640 1 -- start start 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.113+0000 7fd82ec00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd828144c10 0x7fd82815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.113+0000 7fd82ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 0x7fd8280b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.113+0000 7fd82ec00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8280cf2d0 con 0x7fd8280cee40 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.113+0000 7fd82ec00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd828144ff0 con 0x7fd828144c10 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.113+0000 7fd82ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd828160340 con 0x7fd82806d900 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.113+0000 7fd82d800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd828144c10 0x7fd82815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.113+0000 7fd82d800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd828144c10 0x7fd82815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60540/0 (socket says 172.21.15.62:60540) 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.113+0000 7fd82d800640 1 -- 172.21.15.62:0/3449029006 learned_addr learned my addr 172.21.15.62:0/3449029006 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:21.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.114+0000 7fd82e200640 1 --2- 172.21.15.62:0/3449029006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 0x7fd8280b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.114+0000 7fd82d800640 1 -- 172.21.15.62:0/3449029006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 msgr2=0x7fd8280b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.114+0000 7fd82d800640 1 --2- 172.21.15.62:0/3449029006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 0x7fd8280b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.114+0000 7fd82d800640 1 -- 172.21.15.62:0/3449029006 >> v1:172.21.15.121:6789/0 conn(0x7fd8280cee40 legacy=0x7fd82806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.114+0000 7fd82d800640 1 -- 172.21.15.62:0/3449029006 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82815faa0 con 0x7fd828144c10 2024-04-25T09:08:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.115+0000 7fd82d800640 1 --2- 172.21.15.62:0/3449029006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd82815f450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009870 tx=0x7fd81c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7800b10c0ace14ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:21.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.115+0000 7fd827e00640 1 -- 172.21.15.62:0/3449029006 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c037070 con 0x7fd828144c10 2024-04-25T09:08:21.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.115+0000 7fd827e00640 1 -- 172.21.15.62:0/3449029006 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c032440 con 0x7fd828144c10 2024-04-25T09:08:21.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.115+0000 7fd827e00640 1 -- 172.21.15.62:0/3449029006 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c032810 con 0x7fd828144c10 2024-04-25T09:08:21.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.116+0000 7fd82ec00640 1 -- 172.21.15.62:0/3449029006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 msgr2=0x7fd82815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.116+0000 7fd82ec00640 1 --2- 172.21.15.62:0/3449029006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd82815f450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009870 tx=0x7fd81c02a180 comp rx=0 tx=0).stop 2024-04-25T09:08:21.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.116+0000 7fd82ec00640 1 -- 172.21.15.62:0/3449029006 shutdown_connections 2024-04-25T09:08:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.116+0000 7fd82ec00640 1 --2- 172.21.15.62:0/3449029006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 0x7fd8280b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.116+0000 7fd82ec00640 1 --2- 172.21.15.62:0/3449029006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd82815f450 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.116+0000 7fd82ec00640 1 -- 172.21.15.62:0/3449029006 >> 172.21.15.62:0/3449029006 conn(0x7fd8280b14d0 msgr2=0x7fd8280b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.116+0000 7fd82ec00640 1 -- 172.21.15.62:0/3449029006 shutdown_connections 2024-04-25T09:08:21.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.116+0000 7fd82ec00640 1 -- 172.21.15.62:0/3449029006 wait complete. 2024-04-25T09:08:21.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.118+0000 7fd82ec00640 1 Processor -- start 2024-04-25T09:08:21.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.119+0000 7fd82ec00640 1 -- start start 2024-04-25T09:08:21.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 0x7fd828135060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82ec00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8280cee40 0x7fd8281419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd828141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828160060 con 0x7fd828144c10 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82ec00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280b41c0 con 0x7fd8280cee40 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280b4b90 con 0x7fd82806d900 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd828141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd828141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60542/0 (socket says 172.21.15.62:60542) 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82e200640 1 -- 172.21.15.62:0/2253586392 learned_addr learned my addr 172.21.15.62:0/2253586392 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.120+0000 7fd82d800640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 0x7fd828135060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.121+0000 7fd82ce00640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8280cee40 0x7fd8281419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.121+0000 7fd82e200640 1 -- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 msgr2=0x7fd828135060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.121+0000 7fd82e200640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 0x7fd828135060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.121+0000 7fd82e200640 1 -- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8280cee40 msgr2=0x7fd8281419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.121+0000 7fd82e200640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8280cee40 0x7fd8281419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.121+0000 7fd82e200640 1 -- 172.21.15.62:0/2253586392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280cabc0 con 0x7fd828144c10 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.121+0000 7fd82e200640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd828141f20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd820007e60 tx=0x7fd82000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.121+0000 7fd825600640 1 -- 172.21.15.62:0/2253586392 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820014620 con 0x7fd828144c10 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.122+0000 7fd825600640 1 -- 172.21.15.62:0/2253586392 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd820019070 con 0x7fd828144c10 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.122+0000 7fd825600640 1 -- 172.21.15.62:0/2253586392 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820014a00 con 0x7fd828144c10 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.122+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8280caea0 con 0x7fd828144c10 2024-04-25T09:08:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.122+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8280cb460 con 0x7fd828144c10 2024-04-25T09:08:21.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.123+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8280cb840 con 0x7fd828144c10 2024-04-25T09:08:21.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.124+0000 7fd825600640 1 -- 172.21.15.62:0/2253586392 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd820014ba0 con 0x7fd828144c10 2024-04-25T09:08:21.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.126+0000 7fd825600640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd80806b4d0 0x7fd80806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.126+0000 7fd825600640 1 -- 172.21.15.62:0/2253586392 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd82008ab40 con 0x7fd828144c10 2024-04-25T09:08:21.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.127+0000 7fd82d800640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd80806b4d0 0x7fd80806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.128+0000 7fd82d800640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd80806b4d0 0x7fd80806d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd81c034000 tx=0x7fd81c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:21.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.131+0000 7fd825600640 1 -- 172.21.15.62:0/2253586392 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd820059920 con 0x7fd828144c10 2024-04-25T09:08:21.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.373+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd82804ef20 con 0x7fd828144c10 2024-04-25T09:08:21.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.374+0000 7fd825600640 1 -- 172.21.15.62:0/2253586392 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fd820059220 con 0x7fd828144c10 2024-04-25T09:08:21.363 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:21.363 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:07:25.454273+0000","modified":"2024-04-25T09:08:18.032398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.62:6835/752515489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":752515489},{"type":"v1","addr":"172.21.15.62:6835","nonce":752515489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:08:21.363 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 30 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd80806b4d0 msgr2=0x7fd80806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd80806b4d0 0x7fd80806d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd81c034000 tx=0x7fd81c02e000 comp rx=0 tx=0).stop 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 msgr2=0x7fd828141f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd828141f20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd820007e60 tx=0x7fd82000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 shutdown_connections 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd80806b4d0 0x7fd80806d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd828144c10 0x7fd828141f20 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8280cee40 0x7fd8281419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 --2- 172.21.15.62:0/2253586392 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd82806d900 0x7fd828135060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 >> 172.21.15.62:0/2253586392 conn(0x7fd8280b14d0 msgr2=0x7fd828071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 shutdown_connections 2024-04-25T09:08:21.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.377+0000 7fd82ec00640 1 -- 172.21.15.62:0/2253586392 wait complete. 2024-04-25T09:08:21.385 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:21.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.613+0000 7f9b03e00640 1 Processor -- start 2024-04-25T09:08:21.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.613+0000 7f9b03e00640 1 -- start start 2024-04-25T09:08:21.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.615+0000 7f9b03e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 0x7f9b0415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.615+0000 7f9b03e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b040cf2d0 con 0x7f9b04144c10 2024-04-25T09:08:21.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.615+0000 7f9b03e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0415faa0 con 0x7f9b040cee40 2024-04-25T09:08:21.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.615+0000 7f9b03e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04144ff0 con 0x7f9b0406d900 2024-04-25T09:08:21.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.615+0000 7f9b02a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9b04144c10 0x7f9b0406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48648/0 (socket says 172.21.15.62:48648) 2024-04-25T09:08:21.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.615+0000 7f9b02a00640 1 -- 172.21.15.62:0/1411061341 learned_addr learned my addr 172.21.15.62:0/1411061341 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:21.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b02000640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 0x7f9b0415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29010997 0 0) 0x7f9b040cf2d0 con 0x7f9b04144c10 2024-04-25T09:08:21.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae8003520 con 0x7f9b04144c10 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270170924 0 0) 0x7f9ae8003520 con 0x7f9b04144c10 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 msgr2=0x7f9b0415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 0x7f9b0415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 >> v1:172.21.15.121:6789/0 conn(0x7f9b040cee40 legacy=0x7f9b040b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0415fd80 con 0x7f9b04144c10 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f9af4001670 con 0x7f9b04144c10 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 >> v1:172.21.15.62:6789/0 conn(0x7f9b04144c10 legacy=0x7f9b0406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 0x7f9ae80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ae8003f60 0x7f9ae8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.616+0000 7f9b01600640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9ae80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04144ff0 con 0x7f9ae8004d40 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800bb50 con 0x7f9ae8003f60 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae800bd50 con 0x7f9b0406d900 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b03400640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9ae80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b02000640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ae8003f60 0x7f9ae8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b03400640 1 -- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 msgr2=0x7f9ae80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b02a00640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 0x7f9ae80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b03400640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 0x7f9ae80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b03400640 1 -- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ae8003f60 msgr2=0x7f9ae8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b03400640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ae8003f60 0x7f9ae8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b03400640 1 -- 172.21.15.62:0/1411061341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8003520 con 0x7f9ae8004d40 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b02000640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ae8003f60 0x7f9ae8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:21.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.617+0000 7f9b02a00640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b0406d900 0x7f9ae80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.618+0000 7f9b03400640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9ae80051c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9afc001e50 tx=0x7f9ae8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3faf5d7525aca8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc001a70 con 0x7f9ae8004d40 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc003970 con 0x7f9ae8004d40 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b01600640 1 -- 172.21.15.62:0/1411061341 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc02ae60 con 0x7f9ae8004d40 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b03e00640 1 -- 172.21.15.62:0/1411061341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 msgr2=0x7f9ae80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b03e00640 1 --2- 172.21.15.62:0/1411061341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9ae80051c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9afc001e50 tx=0x7f9ae8005830 comp rx=0 tx=0).stop 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b03400640 1 -- 172.21.15.62:0/1411061341 reap_dead start 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b03e00640 1 -- 172.21.15.62:0/1411061341 shutdown_connections 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b03e00640 1 -- 172.21.15.62:0/1411061341 >> 172.21.15.62:0/1411061341 conn(0x7f9b040b14d0 msgr2=0x7f9b040b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b03e00640 1 -- 172.21.15.62:0/1411061341 shutdown_connections 2024-04-25T09:08:21.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.619+0000 7f9b03e00640 1 -- 172.21.15.62:0/1411061341 wait complete. 2024-04-25T09:08:21.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.621+0000 7f9b03e00640 1 Processor -- start 2024-04-25T09:08:21.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.621+0000 7f9b03e00640 1 -- start start 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.622+0000 7f9b03e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9b040cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.622+0000 7f9b03e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b040cdbe0 0x7f9b04140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.622+0000 7f9b03e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b04141a80 0x7f9b04141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.622+0000 7f9b03e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040b41c0 con 0x7f9ae8004d40 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.622+0000 7f9b03e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04160340 con 0x7f9b04141a80 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.622+0000 7f9b03e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0415f450 con 0x7f9b040cdbe0 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.622+0000 7f9b02a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9b040cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9b040cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60562/0 (socket says 172.21.15.62:60562) 2024-04-25T09:08:21.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02a00640 1 -- 172.21.15.62:0/4014035251 learned_addr learned my addr 172.21.15.62:0/4014035251 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:21.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02000640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b040cdbe0 0x7f9b04140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02a00640 1 -- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b040cdbe0 msgr2=0x7f9b04140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b03400640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b04141a80 0x7f9b04141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02a00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b040cdbe0 0x7f9b04140c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02a00640 1 -- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b04141a80 msgr2=0x7f9b04141f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02a00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b04141a80 0x7f9b04141f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02a00640 1 -- 172.21.15.62:0/4014035251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b04135620 con 0x7f9ae8004d40 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02000640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b040cdbe0 0x7f9b04140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.623+0000 7f9b02a00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9b040cb280 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9af400d8c0 tx=0x7f9af400dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.624+0000 7f9afb400640 1 -- 172.21.15.62:0/4014035251 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4019070 con 0x7f9ae8004d40 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.624+0000 7f9afb400640 1 -- 172.21.15.62:0/4014035251 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af40074e0 con 0x7f9ae8004d40 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.624+0000 7f9afb400640 1 -- 172.21.15.62:0/4014035251 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4007690 con 0x7f9ae8004d40 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.624+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b040c9f00 con 0x7f9ae8004d40 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.624+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b040ca4c0 con 0x7f9ae8004d40 2024-04-25T09:08:21.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.625+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0404ef20 con 0x7f9ae8004d40 2024-04-25T09:08:21.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.626+0000 7f9afb400640 1 -- 172.21.15.62:0/4014035251 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9af401c070 con 0x7f9ae8004d40 2024-04-25T09:08:21.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.627+0000 7f9afb400640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ae406b500 0x7f9ae406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:21.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.627+0000 7f9afb400640 1 -- 172.21.15.62:0/4014035251 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9af4089e30 con 0x7f9ae8004d40 2024-04-25T09:08:21.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.628+0000 7f9b02000640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ae406b500 0x7f9ae406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:21.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.629+0000 7f9b02000640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ae406b500 0x7f9ae406d9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9b04140770 tx=0x7f9af0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:21.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.631+0000 7f9afb400640 1 -- 172.21.15.62:0/4014035251 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9af4058b80 con 0x7f9ae8004d40 2024-04-25T09:08:21.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.835+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b04141860 con 0x7f9ae8004d40 2024-04-25T09:08:21.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.838+0000 7f9afb400640 1 -- 172.21.15.62:0/4014035251 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f9af4058480 con 0x7f9ae8004d40 2024-04-25T09:08:21.827 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:21.827 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":50,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:01.219227+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:07:23.299693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T09:07:24.475140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.8","mappings":[{"from":5,"to":0}]},{"pgid":"7.26","mappings":[{"from":5,"to":7}]},{"pgid":"7.33","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]},{"pgid":"7.3c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ae406b500 msgr2=0x7f9ae406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ae406b500 0x7f9ae406d9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9b04140770 tx=0x7f9af0008040 comp rx=0 tx=0).stop 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 msgr2=0x7f9b040cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9b040cb280 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9af400d8c0 tx=0x7f9af400dd80 comp rx=0 tx=0).stop 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 shutdown_connections 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9ae406b500 0x7f9ae406d9c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b04141a80 0x7f9b04141f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b040cdbe0 0x7f9b04140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 --2- 172.21.15.62:0/4014035251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ae8004d40 0x7f9b040cb280 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:21.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 >> 172.21.15.62:0/4014035251 conn(0x7f9b040b14d0 msgr2=0x7f9b040719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:21.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 shutdown_connections 2024-04-25T09:08:21.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:21.841+0000 7f9b03e00640 1 -- 172.21.15.62:0/4014035251 wait complete. 2024-04-25T09:08:21.851 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:08:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.074+0000 7f392a400640 1 Processor -- start 2024-04-25T09:08:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.074+0000 7f392a400640 1 -- start start 2024-04-25T09:08:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.076+0000 7f392a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f392415c4e0 0x7f392415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.076+0000 7f392a400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f39240b49e0 0x7f39240b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.076+0000 7f392a400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39240cff60 con 0x7f39240cf5f0 2024-04-25T09:08:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.076+0000 7f392a400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3924070360 con 0x7f39240b49e0 2024-04-25T09:08:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.076+0000 7f392a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3924070640 con 0x7f392415c4e0 2024-04-25T09:08:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.076+0000 7f3929000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f392415c4e0 0x7f392415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.077+0000 7f3929000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f392415c4e0 0x7f392415c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:47036/0 (socket says 172.21.15.62:47036) 2024-04-25T09:08:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.077+0000 7f3929000640 1 -- 172.21.15.62:0/3388191416 learned_addr learned my addr 172.21.15.62:0/3388191416 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.077+0000 7f3923e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f39240cf5f0 0x7f39240b42d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:33354/0 (socket says 172.21.15.62:33354) 2024-04-25T09:08:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.077+0000 7f3929a00640 1 --2- 172.21.15.62:0/3388191416 >> v2:172.21.15.121:3300/0 conn(0x7f39240b49e0 0x7f39240b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.077+0000 7f3923400640 1 -- 172.21.15.62:0/3388191416 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329951285 0 0) 0x7f39240cff60 con 0x7f39240cf5f0 2024-04-25T09:08:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.077+0000 7f3923400640 1 -- 172.21.15.62:0/3388191416 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f390c003520 con 0x7f39240cf5f0 2024-04-25T09:08:22.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.078+0000 7f3929000640 1 -- 172.21.15.62:0/3388191416 >> v2:172.21.15.121:3300/0 conn(0x7f39240b49e0 msgr2=0x7f39240b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.078+0000 7f3929000640 1 --2- 172.21.15.62:0/3388191416 >> v2:172.21.15.121:3300/0 conn(0x7f39240b49e0 0x7f39240b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.078+0000 7f3929000640 1 -- 172.21.15.62:0/3388191416 >> v1:172.21.15.121:6789/0 conn(0x7f39240cf5f0 legacy=0x7f39240b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.078+0000 7f3929000640 1 -- 172.21.15.62:0/3388191416 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924070920 con 0x7f392415c4e0 2024-04-25T09:08:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.078+0000 7f3929a00640 1 --2- 172.21.15.62:0/3388191416 >> v2:172.21.15.121:3300/0 conn(0x7f39240b49e0 0x7f39240b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:08:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.079+0000 7f3929000640 1 --2- 172.21.15.62:0/3388191416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f392415c4e0 0x7f392415c8c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3918009870 tx=0x7f391802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46062f2568125c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.080+0000 7f3923400640 1 -- 172.21.15.62:0/3388191416 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918037070 con 0x7f392415c4e0 2024-04-25T09:08:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.080+0000 7f3923400640 1 -- 172.21.15.62:0/3388191416 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3918032440 con 0x7f392415c4e0 2024-04-25T09:08:22.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.080+0000 7f3923400640 1 -- 172.21.15.62:0/3388191416 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918032810 con 0x7f392415c4e0 2024-04-25T09:08:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.080+0000 7f392a400640 1 -- 172.21.15.62:0/3388191416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f392415c4e0 msgr2=0x7f392415c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.080+0000 7f392a400640 1 --2- 172.21.15.62:0/3388191416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f392415c4e0 0x7f392415c8c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3918009870 tx=0x7f391802a250 comp rx=0 tx=0).stop 2024-04-25T09:08:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.081+0000 7f392a400640 1 -- 172.21.15.62:0/3388191416 shutdown_connections 2024-04-25T09:08:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.081+0000 7f392a400640 1 --2- 172.21.15.62:0/3388191416 >> v2:172.21.15.121:3300/0 conn(0x7f39240b49e0 0x7f39240b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.081+0000 7f392a400640 1 --2- 172.21.15.62:0/3388191416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f392415c4e0 0x7f392415c8c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.081+0000 7f392a400640 1 -- 172.21.15.62:0/3388191416 >> 172.21.15.62:0/3388191416 conn(0x7f39240668f0 msgr2=0x7f3924066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.081+0000 7f392a400640 1 -- 172.21.15.62:0/3388191416 shutdown_connections 2024-04-25T09:08:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.081+0000 7f392a400640 1 -- 172.21.15.62:0/3388191416 wait complete. 2024-04-25T09:08:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.082+0000 7f392a400640 1 Processor -- start 2024-04-25T09:08:22.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.083+0000 7f392a400640 1 -- start start 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.083+0000 7f392a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39240b49e0 0x7f39241f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.083+0000 7f392a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39240cf5f0 0x7f39241451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.083+0000 7f392a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392415c4e0 0x7f39240b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f392a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924071120 con 0x7f392415c4e0 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f392a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392415ce00 con 0x7f39240b49e0 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f392a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39240cfa70 con 0x7f39240cf5f0 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392415c4e0 0x7f39240b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392415c4e0 0x7f39240b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60570/0 (socket says 172.21.15.62:60570) 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929a00640 1 -- 172.21.15.62:0/3221833817 learned_addr learned my addr 172.21.15.62:0/3221833817 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3923e00640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39240cf5f0 0x7f39241451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929000640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39240b49e0 0x7f39241f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929a00640 1 -- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39240cf5f0 msgr2=0x7f39241451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929a00640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39240cf5f0 0x7f39241451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929a00640 1 -- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39240b49e0 msgr2=0x7f39241f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929a00640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39240b49e0 0x7f39241f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.084+0000 7f3929a00640 1 -- 172.21.15.62:0/3221833817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39241f42c0 con 0x7f392415c4e0 2024-04-25T09:08:22.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.085+0000 7f3929a00640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392415c4e0 0x7f39240b2a10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f391c00efb0 tx=0x7f391c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.085+0000 7f3920c00640 1 -- 172.21.15.62:0/3221833817 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c018020 con 0x7f392415c4e0 2024-04-25T09:08:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.085+0000 7f3920c00640 1 -- 172.21.15.62:0/3221833817 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c014480 con 0x7f392415c4e0 2024-04-25T09:08:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.085+0000 7f3920c00640 1 -- 172.21.15.62:0/3221833817 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c014850 con 0x7f392415c4e0 2024-04-25T09:08:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.085+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39240b05d0 con 0x7f392415c4e0 2024-04-25T09:08:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.085+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39240b0b90 con 0x7f392415c4e0 2024-04-25T09:08:22.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.086+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392404ef20 con 0x7f392415c4e0 2024-04-25T09:08:22.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.087+0000 7f3920c00640 1 -- 172.21.15.62:0/3221833817 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f391c005d10 con 0x7f392415c4e0 2024-04-25T09:08:22.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.088+0000 7f3920c00640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f390406b4d0 0x7f390406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.088+0000 7f3920c00640 1 -- 172.21.15.62:0/3221833817 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f391c08b180 con 0x7f392415c4e0 2024-04-25T09:08:22.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.089+0000 7f3929000640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f390406b4d0 0x7f390406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.090+0000 7f3929000640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f390406b4d0 0x7f390406d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3918034000 tx=0x7f391802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:22.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.093+0000 7f3920c00640 1 -- 172.21.15.62:0/3221833817 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f391c022030 con 0x7f392415c4e0 2024-04-25T09:08:22.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.336+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3924071b00 con 0x7f392415c4e0 2024-04-25T09:08:22.324 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:08:22.337+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:08:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.346+0000 7f3920c00640 1 -- 172.21.15.62:0/3221833817 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f39241f42c0 con 0x7f392415c4e0 2024-04-25T09:08:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:08:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f390406b4d0 msgr2=0x7f390406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f390406b4d0 0x7f390406d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3918034000 tx=0x7f391802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392415c4e0 msgr2=0x7f39240b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392415c4e0 0x7f39240b2a10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f391c00efb0 tx=0x7f391c00c490 comp rx=0 tx=0).stop 2024-04-25T09:08:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 shutdown_connections 2024-04-25T09:08:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f390406b4d0 0x7f390406d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392415c4e0 0x7f39240b2a10 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39240cf5f0 0x7f39241451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 --2- 172.21.15.62:0/3221833817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39240b49e0 0x7f39241f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 >> 172.21.15.62:0/3221833817 conn(0x7f39240668f0 msgr2=0x7f392415d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:22.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.349+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 shutdown_connections 2024-04-25T09:08:22.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.350+0000 7f392a400640 1 -- 172.21.15.62:0/3221833817 wait complete. 2024-04-25T09:08:22.359 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:08:22.444 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:08:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.570+0000 7f734de00640 1 Processor -- start 2024-04-25T09:08:22.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.570+0000 7f734de00640 1 -- start start 2024-04-25T09:08:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.571+0000 7f734de00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f734815dd50 0x7f73480d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.571+0000 7f734de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.571+0000 7f734de00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73480ce780 con 0x7f73480d2fb0 2024-04-25T09:08:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.571+0000 7f734de00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73480d42f0 con 0x7f734815dd50 2024-04-25T09:08:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.572+0000 7f734de00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480d3440 con 0x7f73480d3960 2024-04-25T09:08:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.572+0000 7f734d400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.572+0000 7f7347e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f73480d2fb0 0x7f73480ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:33378/0 (socket says 172.21.15.62:33378) 2024-04-25T09:08:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.572+0000 7f7347e00640 1 -- 172.21.15.62:0/616515945 learned_addr learned my addr 172.21.15.62:0/616515945 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.572+0000 7f734ca00640 1 --2- 172.21.15.62:0/616515945 >> v2:172.21.15.121:3300/0 conn(0x7f734815dd50 0x7f73480d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.573+0000 7f734d400640 1 -- 172.21.15.62:0/616515945 >> v2:172.21.15.121:3300/0 conn(0x7f734815dd50 msgr2=0x7f73480d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.573+0000 7f734d400640 1 --2- 172.21.15.62:0/616515945 >> v2:172.21.15.121:3300/0 conn(0x7f734815dd50 0x7f73480d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.573+0000 7f734d400640 1 -- 172.21.15.62:0/616515945 >> v1:172.21.15.121:6789/0 conn(0x7f73480d2fb0 legacy=0x7f73480ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.573+0000 7f734d400640 1 -- 172.21.15.62:0/616515945 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480ce1c0 con 0x7f73480d3960 2024-04-25T09:08:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.573+0000 7f734ca00640 1 --2- 172.21.15.62:0/616515945 >> v2:172.21.15.121:3300/0 conn(0x7f734815dd50 0x7f73480d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:08:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.575+0000 7f734d400640 1 --2- 172.21.15.62:0/616515945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480d3db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7340009d90 tx=0x7f734002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf33430880218b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.575+0000 7f7347400640 1 -- 172.21.15.62:0/616515945 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340007740 con 0x7f73480d3960 2024-04-25T09:08:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f7347400640 1 -- 172.21.15.62:0/616515945 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73400078e0 con 0x7f73480d3960 2024-04-25T09:08:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f7347400640 1 -- 172.21.15.62:0/616515945 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340007cb0 con 0x7f73480d3960 2024-04-25T09:08:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f734de00640 1 -- 172.21.15.62:0/616515945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 msgr2=0x7f73480d3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f734de00640 1 --2- 172.21.15.62:0/616515945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480d3db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7340009d90 tx=0x7f734002a660 comp rx=0 tx=0).stop 2024-04-25T09:08:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f734de00640 1 -- 172.21.15.62:0/616515945 shutdown_connections 2024-04-25T09:08:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f734de00640 1 --2- 172.21.15.62:0/616515945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480d3db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f734de00640 1 --2- 172.21.15.62:0/616515945 >> v2:172.21.15.121:3300/0 conn(0x7f734815dd50 0x7f73480d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f734de00640 1 -- 172.21.15.62:0/616515945 >> 172.21.15.62:0/616515945 conn(0x7f734806b000 msgr2=0x7f734806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.576+0000 7f734de00640 1 -- 172.21.15.62:0/616515945 shutdown_connections 2024-04-25T09:08:22.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.577+0000 7f734de00640 1 -- 172.21.15.62:0/616515945 wait complete. 2024-04-25T09:08:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.578+0000 7f734de00640 1 Processor -- start 2024-04-25T09:08:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.579+0000 7f734de00640 1 -- start start 2024-04-25T09:08:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f734de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f734de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f734815dd50 0x7f73480c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f734de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73480b48f0 0x7f734814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f734de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480b6b00 con 0x7f734815dd50 2024-04-25T09:08:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f734de00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480d2840 con 0x7f73480b48f0 2024-04-25T09:08:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f734de00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480b6c80 con 0x7f73480d3960 2024-04-25T09:08:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f7347e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f734815dd50 0x7f73480c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f7347e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f734815dd50 0x7f73480c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60592/0 (socket says 172.21.15.62:60592) 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f7347e00640 1 -- 172.21.15.62:0/3755712128 learned_addr learned my addr 172.21.15.62:0/3755712128 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.580+0000 7f734ca00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f734d400640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73480b48f0 0x7f734814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7347e00640 1 -- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 msgr2=0x7f73480b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7347e00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7347e00640 1 -- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73480b48f0 msgr2=0x7f734814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7347e00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73480b48f0 0x7f734814bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7347e00640 1 -- 172.21.15.62:0/3755712128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734814c310 con 0x7f734815dd50 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f734ca00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7347e00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f734815dd50 0x7f73480c95f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f733400c400 tx=0x7f733400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7344c00640 1 -- 172.21.15.62:0/3755712128 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334017070 con 0x7f734815dd50 2024-04-25T09:08:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7344c00640 1 -- 172.21.15.62:0/3755712128 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334005ce0 con 0x7f734815dd50 2024-04-25T09:08:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f7344c00640 1 -- 172.21.15.62:0/3755712128 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340072f0 con 0x7f734815dd50 2024-04-25T09:08:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.581+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73480b3850 con 0x7f734815dd50 2024-04-25T09:08:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.582+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73480b3e10 con 0x7f734815dd50 2024-04-25T09:08:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.583+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734804ef20 con 0x7f734815dd50 2024-04-25T09:08:22.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.584+0000 7f7344c00640 1 -- 172.21.15.62:0/3755712128 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f733401a070 con 0x7f734815dd50 2024-04-25T09:08:22.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.585+0000 7f7344c00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:22.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.585+0000 7f7344c00640 1 -- 172.21.15.62:0/3755712128 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f73340897a0 con 0x7f734815dd50 2024-04-25T09:08:22.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.585+0000 7f734ca00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:22.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.587+0000 7f734ca00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f732806b4d0 0x7f732806d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f733c009af0 tx=0x7f733c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:22.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.590+0000 7f7344c00640 1 -- 172.21.15.62:0/3755712128 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7334058490 con 0x7f734815dd50 2024-04-25T09:08:22.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:22.826+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7348161040 con 0x7f734815dd50 2024-04-25T09:08:23.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.351+0000 7f7344c00640 1 -- 172.21.15.62:0/3755712128 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7334057d90 con 0x7f734815dd50 2024-04-25T09:08:23.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f732806b4d0 msgr2=0x7f732806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:23.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f732806b4d0 0x7f732806d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f733c009af0 tx=0x7f733c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:23.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f734815dd50 msgr2=0x7f73480c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:23.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f734815dd50 0x7f73480c95f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f733400c400 tx=0x7f733400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 shutdown_connections 2024-04-25T09:08:23.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f734815dd50 0x7f73480c95f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73480b48f0 0x7f734814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 --2- 172.21.15.62:0/3755712128 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f73480d3960 0x7f73480b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 >> 172.21.15.62:0/3755712128 conn(0x7f734806b000 msgr2=0x7f73480b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:23.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.355+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 shutdown_connections 2024-04-25T09:08:23.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.356+0000 7f734de00640 1 -- 172.21.15.62:0/3755712128 wait complete. 2024-04-25T09:08:23.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:08:23.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.592+0000 7fbae5a00640 1 Processor -- start 2024-04-25T09:08:23.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.592+0000 7fbae5a00640 1 -- start start 2024-04-25T09:08:23.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.594+0000 7fbae5a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae015b520 0x7fbae00ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.594+0000 7fbae5a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbae015f2a0 0x7fbae00adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.594+0000 7fbae5a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae006e270 con 0x7fbae006c440 2024-04-25T09:08:23.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.594+0000 7fbae5a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae00b3fb0 con 0x7fbae015f2a0 2024-04-25T09:08:23.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.594+0000 7fbae5a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae006c920 con 0x7fbae015b520 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.595+0000 7fbadf400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbae006c440 0x7fbae015eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48660/0 (socket says 172.21.15.62:48660) 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.595+0000 7fbadf400640 1 -- 172.21.15.62:0/2815426293 learned_addr learned my addr 172.21.15.62:0/2815426293 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.595+0000 7fbae5000640 1 --2- 172.21.15.62:0/2815426293 >> v2:172.21.15.121:3300/0 conn(0x7fbae015f2a0 0x7fbae00adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.595+0000 7fbadfe00640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae015b520 0x7fbae00ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.595+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296137164 0 0) 0x7fbae006e270 con 0x7fbae006c440 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.595+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac4003520 con 0x7fbae006c440 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2099165760 0 0) 0x7fbac4003520 con 0x7fbae006c440 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae015b520 msgr2=0x7fbae00ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:23.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae015b520 0x7fbae00ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 >> v2:172.21.15.121:3300/0 conn(0x7fbae015f2a0 msgr2=0x7fbae00adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 --2- 172.21.15.62:0/2815426293 >> v2:172.21.15.121:3300/0 conn(0x7fbae015f2a0 0x7fbae00adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae006cc00 con 0x7fbae006c440 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fbac8001670 con 0x7fbae006c440 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 >> v1:172.21.15.62:6789/0 conn(0x7fbae006c440 legacy=0x7fbae015eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbac40039b0 0x7fbac4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae015b520 0x7fbac4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbac40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00b3fb0 con 0x7fbac4004d40 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae006c920 con 0x7fbac40039b0 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac400b9c0 con 0x7fbae015b520 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbae5000640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbac40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadf400640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae015b520 0x7fbac4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.596+0000 7fbadfe00640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbac40039b0 0x7fbac4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.597+0000 7fbae5000640 1 -- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae015b520 msgr2=0x7fbac4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:23.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.597+0000 7fbae5000640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae015b520 0x7fbac4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.597+0000 7fbae5000640 1 -- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbac40039b0 msgr2=0x7fbac4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:23.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.597+0000 7fbae5000640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbac40039b0 0x7fbac4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.597+0000 7fbae5000640 1 -- 172.21.15.62:0/2815426293 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac4003520 con 0x7fbac4004d40 2024-04-25T09:08:23.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.598+0000 7fbae5000640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbac40051c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009fc0 tx=0x7fbac4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9d4e90e4b0adc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:23.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.598+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8009c10 con 0x7fbac4004d40 2024-04-25T09:08:23.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.598+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad8036070 con 0x7fbac4004d40 2024-04-25T09:08:23.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.598+0000 7fbadea00640 1 -- 172.21.15.62:0/2815426293 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad802d070 con 0x7fbac4004d40 2024-04-25T09:08:23.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.599+0000 7fbae5a00640 1 -- 172.21.15.62:0/2815426293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 msgr2=0x7fbac40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:23.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.599+0000 7fbae5a00640 1 --2- 172.21.15.62:0/2815426293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbac40051c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009fc0 tx=0x7fbac4005830 comp rx=0 tx=0).stop 2024-04-25T09:08:23.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.599+0000 7fbae5000640 1 -- 172.21.15.62:0/2815426293 reap_dead start 2024-04-25T09:08:23.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.599+0000 7fbae5a00640 1 -- 172.21.15.62:0/2815426293 shutdown_connections 2024-04-25T09:08:23.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.599+0000 7fbae5a00640 1 -- 172.21.15.62:0/2815426293 >> 172.21.15.62:0/2815426293 conn(0x7fbae0066ad0 msgr2=0x7fbae0066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:23.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.599+0000 7fbae5a00640 1 -- 172.21.15.62:0/2815426293 shutdown_connections 2024-04-25T09:08:23.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.599+0000 7fbae5a00640 1 -- 172.21.15.62:0/2815426293 wait complete. 2024-04-25T09:08:23.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.601+0000 7fbae5a00640 1 Processor -- start 2024-04-25T09:08:23.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.601+0000 7fbae5a00640 1 -- start start 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbae5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbae00c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbae5a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae013b1c0 0x7fbae00cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbae5a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae0160420 0x7fbae01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbae5a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae015b900 con 0x7fbac4004d40 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbae5a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae006d1c0 con 0x7fbae0160420 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbae5a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae015ba80 con 0x7fbae013b1c0 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbadfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbae00c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbadfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbae00c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60608/0 (socket says 172.21.15.62:60608) 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbadfe00640 1 -- 172.21.15.62:0/1465828969 learned_addr learned my addr 172.21.15.62:0/1465828969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbadf400640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae013b1c0 0x7fbae00cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbae5000640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae0160420 0x7fbae01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbadfe00640 1 -- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae013b1c0 msgr2=0x7fbae00cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:23.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbadfe00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae013b1c0 0x7fbae00cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.602+0000 7fbadfe00640 1 -- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae0160420 msgr2=0x7fbae01608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:23.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.603+0000 7fbadfe00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae0160420 0x7fbae01608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:23.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.603+0000 7fbadfe00640 1 -- 172.21.15.62:0/1465828969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00cdf20 con 0x7fbac4004d40 2024-04-25T09:08:23.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.603+0000 7fbadfe00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbae00c5840 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbad0010860 tx=0x7fbad0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:23.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.603+0000 7fbad7e00640 1 -- 172.21.15.62:0/1465828969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad000f040 con 0x7fbac4004d40 2024-04-25T09:08:23.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.603+0000 7fbad7e00640 1 -- 172.21.15.62:0/1465828969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad000eb60 con 0x7fbac4004d40 2024-04-25T09:08:23.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.603+0000 7fbad7e00640 1 -- 172.21.15.62:0/1465828969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad001c400 con 0x7fbac4004d40 2024-04-25T09:08:23.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.603+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae00ce1c0 con 0x7fbac4004d40 2024-04-25T09:08:23.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.603+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae00ce780 con 0x7fbac4004d40 2024-04-25T09:08:23.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.604+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae0072b30 con 0x7fbac4004d40 2024-04-25T09:08:23.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.605+0000 7fbad7e00640 1 -- 172.21.15.62:0/1465828969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbad001c5a0 con 0x7fbac4004d40 2024-04-25T09:08:23.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.607+0000 7fbad7e00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbabc06b4d0 0x7fbabc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:23.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.607+0000 7fbad7e00640 1 -- 172.21.15.62:0/1465828969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fbad008aa20 con 0x7fbac4004d40 2024-04-25T09:08:23.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.607+0000 7fbadf400640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbabc06b4d0 0x7fbabc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:23.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.609+0000 7fbadf400640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbabc06b4d0 0x7fbabc06d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbac8001eb0 tx=0x7fbac8001da0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:23.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.612+0000 7fbad7e00640 1 -- 172.21.15.62:0/1465828969 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbad0018030 con 0x7fbac4004d40 2024-04-25T09:08:23.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:23.813+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbae015df50 con 0x7fbac4004d40 2024-04-25T09:08:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.355+0000 7fbad7e00640 1 -- 172.21.15.62:0/1465828969 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbad0059710 con 0x7fbac4004d40 2024-04-25T09:08:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:08:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbabc06b4d0 msgr2=0x7fbabc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbabc06b4d0 0x7fbabc06d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbac8001eb0 tx=0x7fbac8001da0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 msgr2=0x7fbae00c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbae00c5840 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbad0010860 tx=0x7fbad0010d20 comp rx=0 tx=0).stop 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 shutdown_connections 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbabc06b4d0 0x7fbabc06d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae0160420 0x7fbae01608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbae013b1c0 0x7fbae00cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 --2- 172.21.15.62:0/1465828969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbac4004d40 0x7fbae00c5840 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 >> 172.21.15.62:0/1465828969 conn(0x7fbae0066ad0 msgr2=0x7fbae01465c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 shutdown_connections 2024-04-25T09:08:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.359+0000 7fbae5a00640 1 -- 172.21.15.62:0/1465828969 wait complete. 2024-04-25T09:08:24.370 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:08:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.601+0000 7f586b400640 1 Processor -- start 2024-04-25T09:08:24.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.601+0000 7f586b400640 1 -- start start 2024-04-25T09:08:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.603+0000 7f586b400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586415eaf0 0x7f5864168b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.603+0000 7f586b400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586415f110 con 0x7f58640cfe50 2024-04-25T09:08:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.603+0000 7f586b400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5864161840 con 0x7f5864147470 2024-04-25T09:08:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.603+0000 7f586b400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5864161b20 con 0x7f586415eaf0 2024-04-25T09:08:24.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.603+0000 7f5863e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f58640cfe50 0x7f586415e3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:48664/0 (socket says 172.21.15.62:48664) 2024-04-25T09:08:24.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.603+0000 7f5863e00640 1 -- 172.21.15.62:0/480486403 learned_addr learned my addr 172.21.15.62:0/480486403 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.603+0000 7f5869400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586415eaf0 0x7f5864168b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348891394 0 0) 0x7f586415f110 con 0x7f58640cfe50 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f584c003520 con 0x7f58640cfe50 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840935115 0 0) 0x7f584c003520 con 0x7f58640cfe50 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586415eaf0 msgr2=0x7f5864168b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586415eaf0 0x7f5864168b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 >> v1:172.21.15.121:6789/0 conn(0x7f5864147470 legacy=0x7f58640cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5864161e00 con 0x7f58640cfe50 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f5850001670 con 0x7f58640cfe50 2024-04-25T09:08:24.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 >> v1:172.21.15.62:6789/0 conn(0x7f58640cfe50 legacy=0x7f586415e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f584c0039e0 0x7f584c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586415eaf0 0x7f584c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f584c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864161b20 con 0x7f584c004d40 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c00baa0 con 0x7f584c0039e0 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.604+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c00bca0 con 0x7f586415eaf0 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5869400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f584c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5863e00640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586415eaf0 0x7f584c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:24.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5868a00640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f584c0039e0 0x7f584c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5869400640 1 -- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586415eaf0 msgr2=0x7f584c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5869400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586415eaf0 0x7f584c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5869400640 1 -- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f584c0039e0 msgr2=0x7f584c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5869400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f584c0039e0 0x7f584c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5869400640 1 -- 172.21.15.62:0/480486403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c003520 con 0x7f584c004d40 2024-04-25T09:08:24.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.605+0000 7f5868a00640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f584c0039e0 0x7f584c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.606+0000 7f5869400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f584c0051c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f585c009f90 tx=0x7f584c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc68e1a0a1e6e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:24.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c009c10 con 0x7f584c004d40 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c036070 con 0x7f584c004d40 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f5863400640 1 -- 172.21.15.62:0/480486403 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c031850 con 0x7f584c004d40 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f586b400640 1 -- 172.21.15.62:0/480486403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 msgr2=0x7f584c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f586b400640 1 --2- 172.21.15.62:0/480486403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f584c0051c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f585c009f90 tx=0x7f584c005830 comp rx=0 tx=0).stop 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f5869400640 1 -- 172.21.15.62:0/480486403 reap_dead start 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f586b400640 1 -- 172.21.15.62:0/480486403 shutdown_connections 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f586b400640 1 -- 172.21.15.62:0/480486403 >> 172.21.15.62:0/480486403 conn(0x7f58640668f0 msgr2=0x7f5864066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.607+0000 7f586b400640 1 -- 172.21.15.62:0/480486403 shutdown_connections 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.608+0000 7f586b400640 1 -- 172.21.15.62:0/480486403 wait complete. 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.609+0000 7f586b400640 1 Processor -- start 2024-04-25T09:08:24.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.609+0000 7f586b400640 1 -- start start 2024-04-25T09:08:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.610+0000 7f586b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f58640b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.610+0000 7f586b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58640af8f0 0x7f58640ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.610+0000 7f586b400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586406d650 0x7f586406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.610+0000 7f586b400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864162600 con 0x7f584c004d40 2024-04-25T09:08:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.610+0000 7f586b400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864147850 con 0x7f58640af8f0 2024-04-25T09:08:24.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.610+0000 7f586b400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5864071360 con 0x7f586406d650 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f58640b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f58640b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60624/0 (socket says 172.21.15.62:60624) 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 -- 172.21.15.62:0/3373956603 learned_addr learned my addr 172.21.15.62:0/3373956603 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 -- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586406d650 msgr2=0x7f586406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586406d650 0x7f586406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 -- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58640af8f0 msgr2=0x7f58640ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58640af8f0 0x7f58640ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 -- 172.21.15.62:0/3373956603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640b1920 con 0x7f584c004d40 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.611+0000 7f5868a00640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f58640b12d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f585800d800 tx=0x7f585800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.612+0000 7f5860c00640 1 -- 172.21.15.62:0/3373956603 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858005070 con 0x7f584c004d40 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.612+0000 7f5860c00640 1 -- 172.21.15.62:0/3373956603 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5858005210 con 0x7f584c004d40 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.612+0000 7f5860c00640 1 -- 172.21.15.62:0/3373956603 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858007030 con 0x7f584c004d40 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.612+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58640b1c00 con 0x7f584c004d40 2024-04-25T09:08:24.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.612+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58640b21c0 con 0x7f584c004d40 2024-04-25T09:08:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.613+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586406e010 con 0x7f584c004d40 2024-04-25T09:08:24.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.614+0000 7f5860c00640 1 -- 172.21.15.62:0/3373956603 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f585801a020 con 0x7f584c004d40 2024-04-25T09:08:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.615+0000 7f5860c00640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:24.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.615+0000 7f5860c00640 1 -- 172.21.15.62:0/3373956603 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f58580898a0 con 0x7f584c004d40 2024-04-25T09:08:24.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.616+0000 7f5863e00640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.617+0000 7f5863e00640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f584406b4d0 0x7f584406d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f586406d3e0 tx=0x7f5850001d90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:24.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.620+0000 7f5860c00640 1 -- 172.21.15.62:0/3373956603 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58580587f0 con 0x7f584c004d40 2024-04-25T09:08:24.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:24.820+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58640aefd0 con 0x7f584c004d40 2024-04-25T09:08:25.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.372+0000 7f5860c00640 1 -- 172.21.15.62:0/3373956603 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f58580580f0 con 0x7f584c004d40 2024-04-25T09:08:25.361 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:08:25.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f584406b4d0 msgr2=0x7f584406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f584406b4d0 0x7f584406d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f586406d3e0 tx=0x7f5850001d90 comp rx=0 tx=0).stop 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 msgr2=0x7f58640b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f58640b12d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f585800d800 tx=0x7f585800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 shutdown_connections 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f586406d650 0x7f586406dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58640af8f0 0x7f58640ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 --2- 172.21.15.62:0/3373956603 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f584c004d40 0x7f58640b12d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 >> 172.21.15.62:0/3373956603 conn(0x7f58640668f0 msgr2=0x7f58640ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 shutdown_connections 2024-04-25T09:08:25.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.376+0000 7f586b400640 1 -- 172.21.15.62:0/3373956603 wait complete. 2024-04-25T09:08:25.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:08:25.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.616+0000 7faf49600640 1 Processor -- start 2024-04-25T09:08:25.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.617+0000 7faf49600640 1 -- start start 2024-04-25T09:08:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.618+0000 7faf49600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440cebc0 0x7faf4406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.618+0000 7faf49600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faf440720c0 0x7faf440703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.618+0000 7faf49600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4406d070 con 0x7faf4415bdf0 2024-04-25T09:08:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.618+0000 7faf49600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf44072570 con 0x7faf440720c0 2024-04-25T09:08:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.618+0000 7faf49600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4415c1d0 con 0x7faf440cebc0 2024-04-25T09:08:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.619+0000 7faf43400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faf440720c0 0x7faf440703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.619+0000 7faf43400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faf440720c0 0x7faf440703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35004/0 (socket says 172.21.15.62:35004) 2024-04-25T09:08:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.619+0000 7faf43400640 1 -- 172.21.15.62:0/2795473979 learned_addr learned my addr 172.21.15.62:0/2795473979 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.619+0000 7faf43400640 1 -- 172.21.15.62:0/2795473979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440cebc0 msgr2=0x7faf4406c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.619+0000 7faf43400640 1 --2- 172.21.15.62:0/2795473979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440cebc0 0x7faf4406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.619+0000 7faf43400640 1 -- 172.21.15.62:0/2795473979 >> v1:172.21.15.121:6789/0 conn(0x7faf4415bdf0 legacy=0x7faf4406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.619+0000 7faf43400640 1 -- 172.21.15.62:0/2795473979 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4406d350 con 0x7faf440720c0 2024-04-25T09:08:25.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.620+0000 7faf43400640 1 --2- 172.21.15.62:0/2795473979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440720c0 0x7faf440703a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009e50 tx=0x7faf3c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1307a0706ab6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:25.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf42000640 1 -- 172.21.15.62:0/2795473979 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c036070 con 0x7faf440720c0 2024-04-25T09:08:25.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf42000640 1 -- 172.21.15.62:0/2795473979 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c031600 con 0x7faf440720c0 2024-04-25T09:08:25.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf42000640 1 -- 172.21.15.62:0/2795473979 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0319d0 con 0x7faf440720c0 2024-04-25T09:08:25.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf49600640 1 -- 172.21.15.62:0/2795473979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440720c0 msgr2=0x7faf440703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:25.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf49600640 1 --2- 172.21.15.62:0/2795473979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440720c0 0x7faf440703a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009e50 tx=0x7faf3c02a6c0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf49600640 1 -- 172.21.15.62:0/2795473979 shutdown_connections 2024-04-25T09:08:25.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf49600640 1 --2- 172.21.15.62:0/2795473979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440720c0 0x7faf440703a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf49600640 1 --2- 172.21.15.62:0/2795473979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440cebc0 0x7faf4406c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.621+0000 7faf49600640 1 -- 172.21.15.62:0/2795473979 >> 172.21.15.62:0/2795473979 conn(0x7faf440b1410 msgr2=0x7faf440b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:25.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.622+0000 7faf49600640 1 -- 172.21.15.62:0/2795473979 shutdown_connections 2024-04-25T09:08:25.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.622+0000 7faf49600640 1 -- 172.21.15.62:0/2795473979 wait complete. 2024-04-25T09:08:25.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.624+0000 7faf49600640 1 Processor -- start 2024-04-25T09:08:25.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.624+0000 7faf49600640 1 -- start start 2024-04-25T09:08:25.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf49600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440720c0 0x7faf4414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:25.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf49600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440cebc0 0x7faf44157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:25.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf49600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf4415bdf0 0x7faf44158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:25.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf49600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf440aedf0 con 0x7faf440cebc0 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf49600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf440ae8c0 con 0x7faf4415bdf0 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf49600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf440708e0 con 0x7faf440720c0 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf3be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440cebc0 0x7faf44157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf3be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440cebc0 0x7faf44157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35012/0 (socket says 172.21.15.62:35012) 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf3be00640 1 -- 172.21.15.62:0/1074689722 learned_addr learned my addr 172.21.15.62:0/1074689722 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.625+0000 7faf42a00640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440720c0 0x7faf4414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf43400640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf4415bdf0 0x7faf44158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3be00640 1 -- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440720c0 msgr2=0x7faf4414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3be00640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440720c0 0x7faf4414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3be00640 1 -- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf4415bdf0 msgr2=0x7faf44158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3be00640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf4415bdf0 0x7faf44158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3be00640 1 -- 172.21.15.62:0/1074689722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf441e5200 con 0x7faf440cebc0 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3be00640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440cebc0 0x7faf44157ec0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00b450 tx=0x7faf2c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:25.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3aa00640 1 -- 172.21.15.62:0/1074689722 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c015020 con 0x7faf440cebc0 2024-04-25T09:08:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3aa00640 1 -- 172.21.15.62:0/1074689722 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c011890 con 0x7faf440cebc0 2024-04-25T09:08:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf3aa00640 1 -- 172.21.15.62:0/1074689722 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c011c80 con 0x7faf440cebc0 2024-04-25T09:08:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.626+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf441e54e0 con 0x7faf440cebc0 2024-04-25T09:08:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.627+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf441e5aa0 con 0x7faf440cebc0 2024-04-25T09:08:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.628+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4404ef20 con 0x7faf440cebc0 2024-04-25T09:08:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.629+0000 7faf3aa00640 1 -- 172.21.15.62:0/1074689722 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf2c018020 con 0x7faf440cebc0 2024-04-25T09:08:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.630+0000 7faf3aa00640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf2006b4d0 0x7faf2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.630+0000 7faf3aa00640 1 -- 172.21.15.62:0/1074689722 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faf2c08a190 con 0x7faf440cebc0 2024-04-25T09:08:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.630+0000 7faf42a00640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf2006b4d0 0x7faf2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:25.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.631+0000 7faf42a00640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf2006b4d0 0x7faf2006d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faf34006fd0 tx=0x7faf34008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:25.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.634+0000 7faf3aa00640 1 -- 172.21.15.62:0/1074689722 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf2c01f020 con 0x7faf440cebc0 2024-04-25T09:08:25.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.866+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7faf440b0c30 con 0x7faf440cebc0 2024-04-25T09:08:25.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.872+0000 7faf3aa00640 1 -- 172.21.15.62:0/1074689722 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7faf2c00d040 con 0x7faf440cebc0 2024-04-25T09:08:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.874+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf2006b4d0 msgr2=0x7faf2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.874+0000 7faf49600640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf2006b4d0 0x7faf2006d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faf34006fd0 tx=0x7faf34008040 comp rx=0 tx=0).stop 2024-04-25T09:08:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.874+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440cebc0 msgr2=0x7faf44157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.874+0000 7faf49600640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440cebc0 0x7faf44157ec0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00b450 tx=0x7faf2c00b910 comp rx=0 tx=0).stop 2024-04-25T09:08:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.875+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 shutdown_connections 2024-04-25T09:08:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.875+0000 7faf49600640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf2006b4d0 0x7faf2006d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.875+0000 7faf49600640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf440cebc0 0x7faf44157ec0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.875+0000 7faf49600640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf4415bdf0 0x7faf44158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.875+0000 7faf49600640 1 --2- 172.21.15.62:0/1074689722 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf440720c0 0x7faf4414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.875+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 >> 172.21.15.62:0/1074689722 conn(0x7faf440b1410 msgr2=0x7faf4415b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.875+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 shutdown_connections 2024-04-25T09:08:25.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:25.875+0000 7faf49600640 1 -- 172.21.15.62:0/1074689722 wait complete. 2024-04-25T09:08:25.883 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T09:08:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.090+0000 7f1b2ce00640 1 Processor -- start 2024-04-25T09:08:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.090+0000 7f1b2ce00640 1 -- start start 2024-04-25T09:08:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.092+0000 7f1b2ce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b2815b900 0x7f1b280d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.092+0000 7f1b2ce00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b280cf890 con 0x7f1b280b6900 2024-04-25T09:08:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.092+0000 7f1b2ce00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2806fdf0 con 0x7f1b280d1cc0 2024-04-25T09:08:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.092+0000 7f1b2ce00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b28070a00 con 0x7f1b2815b900 2024-04-25T09:08:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.092+0000 7f1b27e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1b280b6900 0x7f1b280b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51906/0 (socket says 172.21.15.62:51906) 2024-04-25T09:08:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.092+0000 7f1b27e00640 1 -- 172.21.15.62:0/669474224 learned_addr learned my addr 172.21.15.62:0/669474224 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:26.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.092+0000 7f1b27400640 1 --2- 172.21.15.62:0/669474224 >> v2:172.21.15.121:3300/0 conn(0x7f1b2815b900 0x7f1b280d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:26.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833087027 0 0) 0x7f1b10001110 con 0x7f1b280b6900 2024-04-25T09:08:26.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b0c003520 con 0x7f1b280b6900 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1524633364 0 0) 0x7f1b0c003520 con 0x7f1b280b6900 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 >> v2:172.21.15.121:3300/0 conn(0x7f1b2815b900 msgr2=0x7f1b280d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 --2- 172.21.15.62:0/669474224 >> v2:172.21.15.121:3300/0 conn(0x7f1b2815b900 0x7f1b280d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 >> v1:172.21.15.121:6789/0 conn(0x7f1b280d1cc0 legacy=0x7f1b280b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280700d0 con 0x7f1b280b6900 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1b10001840 con 0x7f1b280b6900 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 >> v1:172.21.15.62:6789/0 conn(0x7f1b280b6900 legacy=0x7f1b280b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b0c0039b0 0x7f1b0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b0c004800 0x7f1b0c004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28070a00 con 0x7f1b0c0051a0 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c00c040 con 0x7f1b0c0039b0 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c00c240 con 0x7f1b0c004800 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.093+0000 7f1b27e00640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.094+0000 7f1b27400640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b0c0039b0 0x7f1b0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.094+0000 7f1b26a00640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b0c004800 0x7f1b0c004c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.094+0000 7f1b27e00640 1 -- 172.21.15.62:0/669474224 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b0c004800 msgr2=0x7f1b0c004c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.094+0000 7f1b27e00640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b0c004800 0x7f1b0c004c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.094+0000 7f1b27e00640 1 -- 172.21.15.62:0/669474224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b0c0039b0 msgr2=0x7f1b0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.094+0000 7f1b27e00640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b0c0039b0 0x7f1b0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.094+0000 7f1b27e00640 1 -- 172.21.15.62:0/669474224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c003520 con 0x7f1b0c0051a0 2024-04-25T09:08:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.095+0000 7f1b27e00640 1 -- 172.21.15.62:0/669474224 reap_dead start 2024-04-25T09:08:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.095+0000 7f1b27e00640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b0c00b9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c490 tx=0x7f1b0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16b3dbc499a633b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.095+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000c0e0 con 0x7f1b0c0051a0 2024-04-25T09:08:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.095+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b100058d0 con 0x7f1b0c0051a0 2024-04-25T09:08:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.095+0000 7f1b26000640 1 -- 172.21.15.62:0/669474224 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10005cc0 con 0x7f1b0c0051a0 2024-04-25T09:08:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.096+0000 7f1b2ce00640 1 -- 172.21.15.62:0/669474224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 msgr2=0x7f1b0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.096+0000 7f1b2ce00640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b0c00b9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c490 tx=0x7f1b0c005830 comp rx=0 tx=0).stop 2024-04-25T09:08:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.096+0000 7f1b2ce00640 1 -- 172.21.15.62:0/669474224 shutdown_connections 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.096+0000 7f1b2ce00640 1 --2- 172.21.15.62:0/669474224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b0c00b9c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.096+0000 7f1b2ce00640 1 -- 172.21.15.62:0/669474224 >> 172.21.15.62:0/669474224 conn(0x7f1b28072340 msgr2=0x7f1b28072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.096+0000 7f1b2ce00640 1 -- 172.21.15.62:0/669474224 shutdown_connections 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.096+0000 7f1b2ce00640 1 -- 172.21.15.62:0/669474224 wait complete. 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.098+0000 7f1b2ce00640 1 Processor -- start 2024-04-25T09:08:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.098+0000 7f1b2ce00640 1 -- start start 2024-04-25T09:08:26.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b2ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b2814b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b2ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b58e0 0x7f1b280b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b2ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280ab240 0x7f1b280ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b2ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2815bce0 con 0x7f1b0c0051a0 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b2ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28070690 con 0x7f1b280b58e0 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b2ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2815be60 con 0x7f1b280ab240 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b27400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b2814b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b27400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b2814b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35040/0 (socket says 172.21.15.62:35040) 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.099+0000 7f1b27400640 1 -- 172.21.15.62:0/3808779761 learned_addr learned my addr 172.21.15.62:0/3808779761 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b26a00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b58e0 0x7f1b280b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b27e00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280ab240 0x7f1b280ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b27400640 1 -- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280ab240 msgr2=0x7f1b280ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b27400640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280ab240 0x7f1b280ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b27400640 1 -- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b58e0 msgr2=0x7f1b280b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:26.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b27400640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b58e0 0x7f1b280b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b27400640 1 -- 172.21.15.62:0/3808779761 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280a9670 con 0x7f1b0c0051a0 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b27e00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280ab240 0x7f1b280ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b27400640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b2814b220 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1b20007f00 tx=0x7f1b200115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b1f400640 1 -- 172.21.15.62:0/3808779761 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2001a020 con 0x7f1b0c0051a0 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b1f400640 1 -- 172.21.15.62:0/3808779761 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b20016470 con 0x7f1b0c0051a0 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b1f400640 1 -- 172.21.15.62:0/3808779761 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20016860 con 0x7f1b0c0051a0 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.100+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b280a9950 con 0x7f1b0c0051a0 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.101+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b280a9f10 con 0x7f1b0c0051a0 2024-04-25T09:08:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.102+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2804ef20 con 0x7f1b0c0051a0 2024-04-25T09:08:26.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.103+0000 7f1b1f400640 1 -- 172.21.15.62:0/3808779761 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b20005d10 con 0x7f1b0c0051a0 2024-04-25T09:08:26.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.104+0000 7f1b1f400640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b500 0x7f1b0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:26.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.104+0000 7f1b1f400640 1 -- 172.21.15.62:0/3808779761 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1b2008ad80 con 0x7f1b0c0051a0 2024-04-25T09:08:26.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.105+0000 7f1b26a00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b500 0x7f1b0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:26.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.106+0000 7f1b26a00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b500 0x7f1b0406d9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1b281f71e0 tx=0x7f1b18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:26.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.109+0000 7f1b1f400640 1 -- 172.21.15.62:0/3808779761 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b20024030 con 0x7f1b0c0051a0 2024-04-25T09:08:26.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:26.390+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f1b2815a260 con 0x7f1b0c0051a0 2024-04-25T09:08:27.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.379+0000 7f1b1f400640 1 -- 172.21.15.62:0/3808779761 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v247) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1b20059e60 con 0x7f1b0c0051a0 2024-04-25T09:08:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.382+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b500 msgr2=0x7f1b0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.382+0000 7f1b2ce00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b500 0x7f1b0406d9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1b281f71e0 tx=0x7f1b18009040 comp rx=0 tx=0).stop 2024-04-25T09:08:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.382+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 msgr2=0x7f1b2814b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.382+0000 7f1b2ce00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b2814b220 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1b20007f00 tx=0x7f1b200115a0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.383+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 shutdown_connections 2024-04-25T09:08:27.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.383+0000 7f1b2ce00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1b0406b500 0x7f1b0406d9c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.383+0000 7f1b2ce00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b280ab240 0x7f1b280ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.383+0000 7f1b2ce00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b280b58e0 0x7f1b280b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.383+0000 7f1b2ce00640 1 --2- 172.21.15.62:0/3808779761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b0c0051a0 0x7f1b2814b220 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.383+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 >> 172.21.15.62:0/3808779761 conn(0x7f1b28072340 msgr2=0x7f1b280b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:27.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.383+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 shutdown_connections 2024-04-25T09:08:27.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.383+0000 7f1b2ce00640 1 -- 172.21.15.62:0/3808779761 wait complete. 2024-04-25T09:08:27.393 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T09:08:27.393 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:08:27.393 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:27.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.621+0000 7f629a000640 1 Processor -- start 2024-04-25T09:08:27.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.621+0000 7f629a000640 1 -- start start 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.622+0000 7f629a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62940b56f0 0x7f6294160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.622+0000 7f629a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62940b6a90 0x7f6294161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.622+0000 7f629a000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f6294161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.622+0000 7f629a000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6294161780 con 0x7f62940b6a90 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.622+0000 7f629a000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62940cedc0 con 0x7f62940b56f0 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.622+0000 7f629a000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62940d3110 con 0x7f62940b27a0 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6293e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62940b6a90 0x7f6294161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6293e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62940b6a90 0x7f6294161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35048/0 (socket says 172.21.15.62:35048) 2024-04-25T09:08:27.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6293e00640 1 -- 172.21.15.62:0/3493829647 learned_addr learned my addr 172.21.15.62:0/3493829647 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:27.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6298c00640 1 --2- 172.21.15.62:0/3493829647 >> v2:172.21.15.121:3300/0 conn(0x7f62940b56f0 0x7f6294160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:27.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6299600640 1 --2- 172.21.15.62:0/3493829647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f6294161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:27.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6293e00640 1 -- 172.21.15.62:0/3493829647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 msgr2=0x7f6294161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:27.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6293e00640 1 --2- 172.21.15.62:0/3493829647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f6294161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6293e00640 1 -- 172.21.15.62:0/3493829647 >> v2:172.21.15.121:3300/0 conn(0x7f62940b56f0 msgr2=0x7f6294160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:27.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6293e00640 1 --2- 172.21.15.62:0/3493829647 >> v2:172.21.15.121:3300/0 conn(0x7f62940b56f0 0x7f6294160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6293e00640 1 -- 172.21.15.62:0/3493829647 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62940d33f0 con 0x7f62940b6a90 2024-04-25T09:08:27.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.623+0000 7f6298c00640 1 --2- 172.21.15.62:0/3493829647 >> v2:172.21.15.121:3300/0 conn(0x7f62940b56f0 0x7f6294160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:27.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.624+0000 7f6293e00640 1 --2- 172.21.15.62:0/3493829647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f6294161240 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f629408c310 tx=0x7f628002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0d088cdb2606d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f6293400640 1 -- 172.21.15.62:0/3493829647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280037070 con 0x7f62940b6a90 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f6293400640 1 -- 172.21.15.62:0/3493829647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6280032440 con 0x7f62940b6a90 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f6293400640 1 -- 172.21.15.62:0/3493829647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280032810 con 0x7f62940b6a90 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f629a000640 1 -- 172.21.15.62:0/3493829647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 msgr2=0x7f6294161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f629a000640 1 --2- 172.21.15.62:0/3493829647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f6294161240 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f629408c310 tx=0x7f628002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f629a000640 1 -- 172.21.15.62:0/3493829647 shutdown_connections 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f629a000640 1 --2- 172.21.15.62:0/3493829647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f6294161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f629a000640 1 --2- 172.21.15.62:0/3493829647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f6294161240 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f629a000640 1 --2- 172.21.15.62:0/3493829647 >> v2:172.21.15.121:3300/0 conn(0x7f62940b56f0 0x7f6294160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.625+0000 7f629a000640 1 -- 172.21.15.62:0/3493829647 >> 172.21.15.62:0/3493829647 conn(0x7f6294066d10 msgr2=0x7f6294067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:27.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.626+0000 7f629a000640 1 -- 172.21.15.62:0/3493829647 shutdown_connections 2024-04-25T09:08:27.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.626+0000 7f629a000640 1 -- 172.21.15.62:0/3493829647 wait complete. 2024-04-25T09:08:27.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.627+0000 7f629a000640 1 Processor -- start 2024-04-25T09:08:27.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.627+0000 7f629a000640 1 -- start start 2024-04-25T09:08:27.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.628+0000 7f629a000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f62940b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:27.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.628+0000 7f629a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62940b56f0 0x7f6294153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.628+0000 7f629a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f62940cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.628+0000 7f629a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62940b2d60 con 0x7f62940b6a90 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.628+0000 7f629a000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62940cea50 con 0x7f62940b56f0 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.628+0000 7f629a000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62940b2ee0 con 0x7f62940b27a0 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6299600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f62940cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6299600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f62940cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35058/0 (socket says 172.21.15.62:35058) 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6299600640 1 -- 172.21.15.62:0/56498256 learned_addr learned my addr 172.21.15.62:0/56498256 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6298c00640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f62940b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6293e00640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62940b56f0 0x7f6294153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6299600640 1 -- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 msgr2=0x7f62940b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6299600640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f62940b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6299600640 1 -- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62940b56f0 msgr2=0x7f6294153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:27.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6299600640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62940b56f0 0x7f6294153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6299600640 1 -- 172.21.15.62:0/56498256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6294157280 con 0x7f62940b6a90 2024-04-25T09:08:27.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.629+0000 7f6298c00640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f62940b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:27.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.630+0000 7f6299600640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f62940cc8b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f628c00ccf0 tx=0x7f628c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:27.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.630+0000 7f6290c00640 1 -- 172.21.15.62:0/56498256 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c003b90 con 0x7f62940b6a90 2024-04-25T09:08:27.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.630+0000 7f6290c00640 1 -- 172.21.15.62:0/56498256 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c003d30 con 0x7f62940b6a90 2024-04-25T09:08:27.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.630+0000 7f6290c00640 1 -- 172.21.15.62:0/56498256 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c004260 con 0x7f62940b6a90 2024-04-25T09:08:27.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.630+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6294157560 con 0x7f62940b6a90 2024-04-25T09:08:27.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.630+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6294157b20 con 0x7f62940b6a90 2024-04-25T09:08:27.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.631+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629404ef20 con 0x7f62940b6a90 2024-04-25T09:08:27.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.632+0000 7f6290c00640 1 -- 172.21.15.62:0/56498256 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f628c004400 con 0x7f62940b6a90 2024-04-25T09:08:27.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.634+0000 7f6290c00640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f627406b4d0 0x7f627406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:27.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.634+0000 7f6290c00640 1 -- 172.21.15.62:0/56498256 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f628c08aa50 con 0x7f62940b6a90 2024-04-25T09:08:27.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.634+0000 7f6298c00640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f627406b4d0 0x7f627406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:27.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.636+0000 7f6298c00640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f627406b4d0 0x7f627406d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6284006fd0 tx=0x7f6284008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:27.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.638+0000 7f6290c00640 1 -- 172.21.15.62:0/56498256 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f628c059b40 con 0x7f62940b6a90 2024-04-25T09:08:27.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.843+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62940ce580 con 0x7f62940b6a90 2024-04-25T09:08:27.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.845+0000 7f6290c00640 1 -- 172.21.15.62:0/56498256 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+13006 (secure 0 0 0) 0x7f628c059440 con 0x7f62940b6a90 2024-04-25T09:08:27.833 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:27.833 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":53,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:25.370093+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/752515489":"2024-04-26T09:08:22.338100+0000","172.21.15.62:6833/752515489":"2024-04-26T09:08:22.338100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:27.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f627406b4d0 msgr2=0x7f627406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f627406b4d0 0x7f627406d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6284006fd0 tx=0x7f6284008040 comp rx=0 tx=0).stop 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 msgr2=0x7f62940cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f62940cc8b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f628c00ccf0 tx=0x7f628c00a660 comp rx=0 tx=0).stop 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 shutdown_connections 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f627406b4d0 0x7f627406d990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62940b6a90 0x7f62940cc8b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62940b56f0 0x7f6294153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 --2- 172.21.15.62:0/56498256 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62940b27a0 0x7f62940b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 >> 172.21.15.62:0/56498256 conn(0x7f6294066d10 msgr2=0x7f62940b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 shutdown_connections 2024-04-25T09:08:27.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:27.848+0000 7f629a000640 1 -- 172.21.15.62:0/56498256 wait complete. 2024-04-25T09:08:27.856 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:27.857 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.087+0000 7f4361600640 1 Processor -- start 2024-04-25T09:08:28.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.087+0000 7f4361600640 1 -- start start 2024-04-25T09:08:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.089+0000 7f4361600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f435c0ceb50 0x7f435c06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.089+0000 7f4361600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f435c15e7c0 0x7f435c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.089+0000 7f4361600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c15ee30 con 0x7f435c06d800 2024-04-25T09:08:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.089+0000 7f4361600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c06dc60 con 0x7f435c15e7c0 2024-04-25T09:08:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.089+0000 7f4361600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1618b0 con 0x7f435c0ceb50 2024-04-25T09:08:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f4353e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f435c06d800 0x7f435c15e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51916/0 (socket says 172.21.15.62:51916) 2024-04-25T09:08:28.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f4353e00640 1 -- 172.21.15.62:0/1015956053 learned_addr learned my addr 172.21.15.62:0/1015956053 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f4360c00640 1 --2- 172.21.15.62:0/1015956053 >> v2:172.21.15.121:3300/0 conn(0x7f435c15e7c0 0x7f435c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435be00640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f435c0ceb50 0x7f435c06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702069602 0 0) 0x7f435c15ee30 con 0x7f435c06d800 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4340003520 con 0x7f435c06d800 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3600595251 0 0) 0x7f4340003520 con 0x7f435c06d800 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f435c0ceb50 msgr2=0x7f435c06d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f435c0ceb50 0x7f435c06d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 >> v2:172.21.15.121:3300/0 conn(0x7f435c15e7c0 msgr2=0x7f435c0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 --2- 172.21.15.62:0/1015956053 >> v2:172.21.15.121:3300/0 conn(0x7f435c15e7c0 0x7f435c0b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c161b90 con 0x7f435c06d800 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.090+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f4344001670 con 0x7f435c06d800 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 >> v1:172.21.15.62:6789/0 conn(0x7f435c06d800 legacy=0x7f435c15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f435b400640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43400039b0 0x7f4340003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f435b400640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f435c0ceb50 0x7f4340004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f435b400640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340004d40 0x7f43400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1618b0 con 0x7f4340004d40 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434000b9c0 con 0x7f43400039b0 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434000bb70 con 0x7f435c0ceb50 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f4360c00640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340004d40 0x7f43400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f4353e00640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f435c0ceb50 0x7f4340004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f435be00640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43400039b0 0x7f4340003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f4360c00640 1 -- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f435c0ceb50 msgr2=0x7f4340004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f4360c00640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f435c0ceb50 0x7f4340004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f4360c00640 1 -- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43400039b0 msgr2=0x7f4340003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f4360c00640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43400039b0 0x7f4340003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.091+0000 7f4360c00640 1 -- 172.21.15.62:0/1015956053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340003520 con 0x7f4340004d40 2024-04-25T09:08:28.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f4360c00640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340004d40 0x7f43400051c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4354009fc0 tx=0x7f4340005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e134d09d20125dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:28.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354009c10 con 0x7f4340004d40 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4354036070 con 0x7f4340004d40 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f435b400640 1 -- 172.21.15.62:0/1015956053 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435402d070 con 0x7f4340004d40 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f4361600640 1 -- 172.21.15.62:0/1015956053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340004d40 msgr2=0x7f43400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f4361600640 1 --2- 172.21.15.62:0/1015956053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4340004d40 0x7f43400051c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4354009fc0 tx=0x7f4340005830 comp rx=0 tx=0).stop 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f4360c00640 1 -- 172.21.15.62:0/1015956053 reap_dead start 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f4361600640 1 -- 172.21.15.62:0/1015956053 shutdown_connections 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.093+0000 7f4361600640 1 -- 172.21.15.62:0/1015956053 >> 172.21.15.62:0/1015956053 conn(0x7f435c066f20 msgr2=0x7f435c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.094+0000 7f4361600640 1 -- 172.21.15.62:0/1015956053 shutdown_connections 2024-04-25T09:08:28.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.094+0000 7f4361600640 1 -- 172.21.15.62:0/1015956053 wait complete. 2024-04-25T09:08:28.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.095+0000 7f4361600640 1 Processor -- start 2024-04-25T09:08:28.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.096+0000 7f4361600640 1 -- start start 2024-04-25T09:08:28.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.096+0000 7f4361600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4340004d40 0x7f435c1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.096+0000 7f4361600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c156060 0x7f435c1564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.096+0000 7f4361600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c0b0380 0x7f435c0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.096+0000 7f4361600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c162510 con 0x7f435c0b0380 2024-04-25T09:08:28.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.096+0000 7f4361600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c162390 con 0x7f435c156060 2024-04-25T09:08:28.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4361600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c06d2c0 con 0x7f4340004d40 2024-04-25T09:08:28.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4360c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c0b0380 0x7f435c0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4360c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c0b0380 0x7f435c0b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35080/0 (socket says 172.21.15.62:35080) 2024-04-25T09:08:28.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4360c00640 1 -- 172.21.15.62:0/2592025717 learned_addr learned my addr 172.21.15.62:0/2592025717 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:28.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f435be00640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4340004d40 0x7f435c1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4353e00640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c156060 0x7f435c1564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4360c00640 1 -- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4340004d40 msgr2=0x7f435c1f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4360c00640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4340004d40 0x7f435c1f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4360c00640 1 -- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c156060 msgr2=0x7f435c1564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4360c00640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c156060 0x7f435c1564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.097+0000 7f4360c00640 1 -- 172.21.15.62:0/2592025717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c141e70 con 0x7f435c0b0380 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.098+0000 7f4360c00640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c0b0380 0x7f435c0b0800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4354033040 tx=0x7f4354009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.098+0000 7f4358c00640 1 -- 172.21.15.62:0/2592025717 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354009c10 con 0x7f435c0b0380 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.098+0000 7f4358c00640 1 -- 172.21.15.62:0/2592025717 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4354003990 con 0x7f435c0b0380 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.098+0000 7f4358c00640 1 -- 172.21.15.62:0/2592025717 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354003d80 con 0x7f435c0b0380 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.098+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c142150 con 0x7f435c0b0380 2024-04-25T09:08:28.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.098+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c142710 con 0x7f435c0b0380 2024-04-25T09:08:28.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.100+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c142a40 con 0x7f435c0b0380 2024-04-25T09:08:28.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.100+0000 7f4358c00640 1 -- 172.21.15.62:0/2592025717 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4354005d10 con 0x7f435c0b0380 2024-04-25T09:08:28.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.101+0000 7f4358c00640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f433806b4d0 0x7f433806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.101+0000 7f4358c00640 1 -- 172.21.15.62:0/2592025717 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f43540ab3d0 con 0x7f435c0b0380 2024-04-25T09:08:28.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.102+0000 7f435be00640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f433806b4d0 0x7f433806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.103+0000 7f435be00640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f433806b4d0 0x7f433806d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f434c009990 tx=0x7f434c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:28.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.106+0000 7f4358c00640 1 -- 172.21.15.62:0/2592025717 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f435407a660 con 0x7f435c0b0380 2024-04-25T09:08:28.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.304+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f435c06abe0 con 0x7f435c0b0380 2024-04-25T09:08:28.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.305+0000 7f4358c00640 1 -- 172.21.15.62:0/2592025717 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4354079f60 con 0x7f435c0b0380 2024-04-25T09:08:28.294 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:08:28.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f433806b4d0 msgr2=0x7f433806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f433806b4d0 0x7f433806d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f434c009990 tx=0x7f434c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c0b0380 msgr2=0x7f435c0b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c0b0380 0x7f435c0b0800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4354033040 tx=0x7f4354009750 comp rx=0 tx=0).stop 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 shutdown_connections 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f433806b4d0 0x7f433806d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f435c0b0380 0x7f435c0b0800 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f435c156060 0x7f435c1564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 --2- 172.21.15.62:0/2592025717 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4340004d40 0x7f435c1f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 >> 172.21.15.62:0/2592025717 conn(0x7f435c066f20 msgr2=0x7f435c146050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 shutdown_connections 2024-04-25T09:08:28.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.308+0000 7f4361600640 1 -- 172.21.15.62:0/2592025717 wait complete. 2024-04-25T09:08:28.314 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T09:08:28.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.541+0000 7f022e200640 1 Processor -- start 2024-04-25T09:08:28.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.541+0000 7f022e200640 1 -- start start 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.542+0000 7f022e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02280b56f0 0x7f0228160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.542+0000 7f022e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f0228161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.542+0000 7f022e200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02280d3110 con 0x7f02280b6a90 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.542+0000 7f022e200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0228161780 con 0x7f02280b56f0 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.542+0000 7f022e200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cedc0 con 0x7f02280b27a0 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02280b56f0 0x7f0228160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f02280b56f0 0x7f0228160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35088/0 (socket says 172.21.15.62:35088) 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022ce00640 1 -- 172.21.15.62:0/1360245408 learned_addr learned my addr 172.21.15.62:0/1360245408 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:28.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f0227e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f02280b6a90 0x7f0228161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51928/0 (socket says 172.21.15.62:51928) 2024-04-25T09:08:28.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022d800640 1 --2- 172.21.15.62:0/1360245408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f0228161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022ce00640 1 -- 172.21.15.62:0/1360245408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 msgr2=0x7f0228161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:28.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022ce00640 1 --2- 172.21.15.62:0/1360245408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f0228161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022ce00640 1 -- 172.21.15.62:0/1360245408 >> v1:172.21.15.62:6789/0 conn(0x7f02280b6a90 legacy=0x7f0228161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022ce00640 1 -- 172.21.15.62:0/1360245408 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02280d33f0 con 0x7f02280b56f0 2024-04-25T09:08:28.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f0227400640 1 -- 172.21.15.62:0/1360245408 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255750911 0 0) 0x7f02280d3110 con 0x7f02280b6a90 2024-04-25T09:08:28.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.543+0000 7f022d800640 1 --2- 172.21.15.62:0/1360245408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f0228161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.544+0000 7f022ce00640 1 --2- 172.21.15.62:0/1360245408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f0228160d00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0218009870 tx=0x7f021802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97ab8eb8a1d55bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.545+0000 7f0227400640 1 -- 172.21.15.62:0/1360245408 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218037070 con 0x7f02280b56f0 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.545+0000 7f0227400640 1 -- 172.21.15.62:0/1360245408 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0218032440 con 0x7f02280b56f0 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.545+0000 7f0227400640 1 -- 172.21.15.62:0/1360245408 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218032810 con 0x7f02280b56f0 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.545+0000 7f022e200640 1 -- 172.21.15.62:0/1360245408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 msgr2=0x7f0228160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.545+0000 7f022e200640 1 --2- 172.21.15.62:0/1360245408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f0228160d00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0218009870 tx=0x7f021802a250 comp rx=0 tx=0).stop 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.546+0000 7f022e200640 1 -- 172.21.15.62:0/1360245408 shutdown_connections 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.546+0000 7f022e200640 1 --2- 172.21.15.62:0/1360245408 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f0228161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.546+0000 7f022e200640 1 --2- 172.21.15.62:0/1360245408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b56f0 0x7f0228160d00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.546+0000 7f022e200640 1 -- 172.21.15.62:0/1360245408 >> 172.21.15.62:0/1360245408 conn(0x7f0228066d10 msgr2=0x7f0228067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.546+0000 7f022e200640 1 -- 172.21.15.62:0/1360245408 shutdown_connections 2024-04-25T09:08:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.546+0000 7f022e200640 1 -- 172.21.15.62:0/1360245408 wait complete. 2024-04-25T09:08:28.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.547+0000 7f022e200640 1 Processor -- start 2024-04-25T09:08:28.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.547+0000 7f022e200640 1 -- start start 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.548+0000 7f022e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f022814db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.548+0000 7f022e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b56f0 0x7f022814b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.548+0000 7f022e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b6a90 0x7f022814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.548+0000 7f022e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280b2d60 con 0x7f02280b6a90 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.548+0000 7f022e200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cea50 con 0x7f02280b56f0 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.548+0000 7f022e200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280b2ee0 con 0x7f02280b27a0 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b6a90 0x7f022814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b6a90 0x7f022814c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35102/0 (socket says 172.21.15.62:35102) 2024-04-25T09:08:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022d800640 1 -- 172.21.15.62:0/1093796017 learned_addr learned my addr 172.21.15.62:0/1093796017 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:28.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f0227e00640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b56f0 0x7f022814b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022ce00640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f022814db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022d800640 1 -- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 msgr2=0x7f022814db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022d800640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f022814db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022d800640 1 -- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b56f0 msgr2=0x7f022814b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022d800640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b56f0 0x7f022814b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022d800640 1 -- 172.21.15.62:0/1093796017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022814cad0 con 0x7f02280b6a90 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f0227e00640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b56f0 0x7f022814b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.549+0000 7f022ce00640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f022814db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.550+0000 7f022d800640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b6a90 0x7f022814c180 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f022000ced0 tx=0x7f022000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.550+0000 7f0224c00640 1 -- 172.21.15.62:0/1093796017 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220003b90 con 0x7f02280b6a90 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.550+0000 7f0224c00640 1 -- 172.21.15.62:0/1093796017 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0220003d50 con 0x7f02280b6a90 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.550+0000 7f0224c00640 1 -- 172.21.15.62:0/1093796017 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220004260 con 0x7f02280b6a90 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.550+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022814cdb0 con 0x7f02280b6a90 2024-04-25T09:08:28.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.550+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022814d370 con 0x7f02280b6a90 2024-04-25T09:08:28.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.551+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022804ef20 con 0x7f02280b6a90 2024-04-25T09:08:28.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.552+0000 7f0224c00640 1 -- 172.21.15.62:0/1093796017 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0220004400 con 0x7f02280b6a90 2024-04-25T09:08:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.553+0000 7f0224c00640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020806b500 0x7f020806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:28.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.553+0000 7f0224c00640 1 -- 172.21.15.62:0/1093796017 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f022008a940 con 0x7f02280b6a90 2024-04-25T09:08:28.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.554+0000 7f022ce00640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020806b500 0x7f020806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:28.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.555+0000 7f022ce00640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020806b500 0x7f020806d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0218034000 tx=0x7f021802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:28.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.558+0000 7f0224c00640 1 -- 172.21.15.62:0/1093796017 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0220059990 con 0x7f02280b6a90 2024-04-25T09:08:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:28.830+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f02280b7300 con 0x7f02280b6a90 2024-04-25T09:08:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.393+0000 7f0224c00640 1 -- 172.21.15.62:0/1093796017 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v249) v1 ==== 146+0+0 (secure 0 0 0) 0x7f0220059290 con 0x7f02280b6a90 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.397+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020806b500 msgr2=0x7f020806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.397+0000 7f022e200640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020806b500 0x7f020806d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0218034000 tx=0x7f021802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.397+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b6a90 msgr2=0x7f022814c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.397+0000 7f022e200640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b6a90 0x7f022814c180 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f022000ced0 tx=0x7f022000a710 comp rx=0 tx=0).stop 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.398+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 shutdown_connections 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.398+0000 7f022e200640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f020806b500 0x7f020806d9c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.398+0000 7f022e200640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02280b6a90 0x7f022814c180 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.398+0000 7f022e200640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02280b56f0 0x7f022814b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.398+0000 7f022e200640 1 --2- 172.21.15.62:0/1093796017 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02280b27a0 0x7f022814db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.398+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 >> 172.21.15.62:0/1093796017 conn(0x7f0228066d10 msgr2=0x7f02280b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:29.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.398+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 shutdown_connections 2024-04-25T09:08:29.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.398+0000 7f022e200640 1 -- 172.21.15.62:0/1093796017 wait complete. 2024-04-25T09:08:29.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:08:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.630+0000 7f0946000640 1 Processor -- start 2024-04-25T09:08:29.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.630+0000 7f0946000640 1 -- start start 2024-04-25T09:08:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0946000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f094015c520 0x7f09400b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0946000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09400b3360 0x7f094006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0946000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0940166720 0x7f094006d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0946000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09400b4640 con 0x7f094015c520 2024-04-25T09:08:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0946000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0940165c40 con 0x7f09400b3360 2024-04-25T09:08:29.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0946000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0940165f20 con 0x7f0940166720 2024-04-25T09:08:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0944c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f094015c520 0x7f09400b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0944c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f094015c520 0x7f09400b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35104/0 (socket says 172.21.15.62:35104) 2024-04-25T09:08:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.632+0000 7f0944c00640 1 -- 172.21.15.62:0/1054453361 learned_addr learned my addr 172.21.15.62:0/1054453361 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.633+0000 7f093fe00640 1 --2- 172.21.15.62:0/1054453361 >> v2:172.21.15.121:3300/0 conn(0x7f09400b3360 0x7f094006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.633+0000 7f0945600640 1 --2- 172.21.15.62:0/1054453361 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0940166720 0x7f094006d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.633+0000 7f0944c00640 1 -- 172.21.15.62:0/1054453361 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0940166720 msgr2=0x7f094006d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.633+0000 7f0944c00640 1 --2- 172.21.15.62:0/1054453361 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0940166720 0x7f094006d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.633+0000 7f0944c00640 1 -- 172.21.15.62:0/1054453361 >> v2:172.21.15.121:3300/0 conn(0x7f09400b3360 msgr2=0x7f094006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.633+0000 7f0944c00640 1 --2- 172.21.15.62:0/1054453361 >> v2:172.21.15.121:3300/0 conn(0x7f09400b3360 0x7f094006d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.633+0000 7f0944c00640 1 -- 172.21.15.62:0/1054453361 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940166200 con 0x7f094015c520 2024-04-25T09:08:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.634+0000 7f0944c00640 1 --2- 172.21.15.62:0/1054453361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f094015c520 0x7f09400b3d10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f09400b3c90 tx=0x7f093002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2117020856e6273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.634+0000 7f093f400640 1 -- 172.21.15.62:0/1054453361 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930036070 con 0x7f094015c520 2024-04-25T09:08:29.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.634+0000 7f093f400640 1 -- 172.21.15.62:0/1054453361 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0930031600 con 0x7f094015c520 2024-04-25T09:08:29.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.634+0000 7f093f400640 1 -- 172.21.15.62:0/1054453361 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09300319d0 con 0x7f094015c520 2024-04-25T09:08:29.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 -- 172.21.15.62:0/1054453361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f094015c520 msgr2=0x7f09400b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:29.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 --2- 172.21.15.62:0/1054453361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f094015c520 0x7f09400b3d10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f09400b3c90 tx=0x7f093002a610 comp rx=0 tx=0).stop 2024-04-25T09:08:29.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 -- 172.21.15.62:0/1054453361 shutdown_connections 2024-04-25T09:08:29.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 --2- 172.21.15.62:0/1054453361 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0940166720 0x7f094006d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 --2- 172.21.15.62:0/1054453361 >> v2:172.21.15.121:3300/0 conn(0x7f09400b3360 0x7f094006d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 --2- 172.21.15.62:0/1054453361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f094015c520 0x7f09400b3d10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 -- 172.21.15.62:0/1054453361 >> 172.21.15.62:0/1054453361 conn(0x7f0940066f20 msgr2=0x7f09400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:29.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 -- 172.21.15.62:0/1054453361 shutdown_connections 2024-04-25T09:08:29.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.635+0000 7f0946000640 1 -- 172.21.15.62:0/1054453361 wait complete. 2024-04-25T09:08:29.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.637+0000 7f0946000640 1 Processor -- start 2024-04-25T09:08:29.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.637+0000 7f0946000640 1 -- start start 2024-04-25T09:08:29.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.638+0000 7f0946000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09400b3360 0x7f0940156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.638+0000 7f0946000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f094015c520 0x7f09401fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.638+0000 7f0946000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0940166720 0x7f0940155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.638+0000 7f0946000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400b4250 con 0x7f09400b3360 2024-04-25T09:08:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.638+0000 7f0946000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094015c900 con 0x7f0940166720 2024-04-25T09:08:29.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.638+0000 7f0946000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09400b43d0 con 0x7f094015c520 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0944c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09400b3360 0x7f0940156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0944c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09400b3360 0x7f0940156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35106/0 (socket says 172.21.15.62:35106) 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0944c00640 1 -- 172.21.15.62:0/1718017141 learned_addr learned my addr 172.21.15.62:0/1718017141 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f093fe00640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f094015c520 0x7f09401fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0945600640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0940166720 0x7f0940155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0944c00640 1 -- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f094015c520 msgr2=0x7f09401fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0944c00640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f094015c520 0x7f09401fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0944c00640 1 -- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0940166720 msgr2=0x7f0940155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0944c00640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0940166720 0x7f0940155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0944c00640 1 -- 172.21.15.62:0/1718017141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09401fc680 con 0x7f09400b3360 2024-04-25T09:08:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.639+0000 7f0945600640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0940166720 0x7f0940155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.640+0000 7f0944c00640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09400b3360 0x7f0940156710 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0930034040 tx=0x7f0930004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.640+0000 7f093cc00640 1 -- 172.21.15.62:0/1718017141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930036050 con 0x7f09400b3360 2024-04-25T09:08:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.640+0000 7f093cc00640 1 -- 172.21.15.62:0/1718017141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0930042070 con 0x7f09400b3360 2024-04-25T09:08:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.640+0000 7f093cc00640 1 -- 172.21.15.62:0/1718017141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930009510 con 0x7f09400b3360 2024-04-25T09:08:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.640+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09400cd780 con 0x7f09400b3360 2024-04-25T09:08:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.640+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09400a98b0 con 0x7f09400b3360 2024-04-25T09:08:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.641+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094004ef20 con 0x7f09400b3360 2024-04-25T09:08:29.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.642+0000 7f093cc00640 1 -- 172.21.15.62:0/1718017141 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0930007c30 con 0x7f09400b3360 2024-04-25T09:08:29.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.643+0000 7f093cc00640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f092006b4d0 0x7f092006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:29.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.643+0000 7f093cc00640 1 -- 172.21.15.62:0/1718017141 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f09300aae80 con 0x7f09400b3360 2024-04-25T09:08:29.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.644+0000 7f093fe00640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f092006b4d0 0x7f092006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:29.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.645+0000 7f093fe00640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f092006b4d0 0x7f092006d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09400cf320 tx=0x7f092c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:29.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.647+0000 7f093cc00640 1 -- 172.21.15.62:0/1718017141 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09401fc680 con 0x7f09400b3360 2024-04-25T09:08:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.850+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0940161f00 con 0x7f09400b3360 2024-04-25T09:08:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.854+0000 7f093cc00640 1 -- 172.21.15.62:0/1718017141 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0930079ff0 con 0x7f09400b3360 2024-04-25T09:08:29.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f092006b4d0 msgr2=0x7f092006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:29.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f092006b4d0 0x7f092006d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09400cf320 tx=0x7f092c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:29.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09400b3360 msgr2=0x7f0940156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09400b3360 0x7f0940156710 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0930034040 tx=0x7f0930004a40 comp rx=0 tx=0).stop 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 shutdown_connections 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f092006b4d0 0x7f092006d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0940166720 0x7f0940155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f094015c520 0x7f09401fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 --2- 172.21.15.62:0/1718017141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09400b3360 0x7f0940156710 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 >> 172.21.15.62:0/1718017141 conn(0x7f0940066f20 msgr2=0x7f0940164ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 shutdown_connections 2024-04-25T09:08:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:29.858+0000 7f0946000640 1 -- 172.21.15.62:0/1718017141 wait complete. 2024-04-25T09:08:29.866 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:08:29.866 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:08:29.975 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:08:30.081 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:08:30.081 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:30.081 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:30.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:08:30.142 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:08:30.142 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:30.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:08:30.199 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:30.199 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:30.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:08:30.258 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:08:30.258 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:30.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:08:30.316 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:08:30.316 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.548+0000 7f5262800640 1 Processor -- start 2024-04-25T09:08:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.548+0000 7f5262800640 1 -- start start 2024-04-25T09:08:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.549+0000 7f5262800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f525c15e2c0 0x7f525c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f5262800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f525c0b2d30 0x7f525c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f5262800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c0b4450 con 0x7f525c0cecd0 2024-04-25T09:08:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f5262800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525c0cc020 con 0x7f525c0b2d30 2024-04-25T09:08:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f5262800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525c0cc300 con 0x7f525c15e2c0 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f525c0b2d30 0x7f525c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f525c0b2d30 0x7f525c0ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35110/0 (socket says 172.21.15.62:35110) 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525aa00640 1 -- 172.21.15.62:0/253946264 learned_addr learned my addr 172.21.15.62:0/253946264 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525b400640 1 --2- 172.21.15.62:0/253946264 >> v2:172.21.15.121:3300/0 conn(0x7f525c15e2c0 0x7f525c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525aa00640 1 -- 172.21.15.62:0/253946264 >> v2:172.21.15.121:3300/0 conn(0x7f525c15e2c0 msgr2=0x7f525c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525aa00640 1 --2- 172.21.15.62:0/253946264 >> v2:172.21.15.121:3300/0 conn(0x7f525c15e2c0 0x7f525c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525aa00640 1 -- 172.21.15.62:0/253946264 >> v1:172.21.15.121:6789/0 conn(0x7f525c0cecd0 legacy=0x7f525c06adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525aa00640 1 -- 172.21.15.62:0/253946264 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c15e6a0 con 0x7f525c0b2d30 2024-04-25T09:08:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.551+0000 7f525b400640 1 --2- 172.21.15.62:0/253946264 >> v2:172.21.15.121:3300/0 conn(0x7f525c15e2c0 0x7f525c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.552+0000 7f525aa00640 1 --2- 172.21.15.62:0/253946264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0b2d30 0x7f525c0ce790 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f525c0d19a0 tx=0x7f524802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eca7314df70407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.552+0000 7f525a000640 1 -- 172.21.15.62:0/253946264 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248037070 con 0x7f525c0b2d30 2024-04-25T09:08:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.552+0000 7f525a000640 1 -- 172.21.15.62:0/253946264 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5248032440 con 0x7f525c0b2d30 2024-04-25T09:08:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.552+0000 7f525a000640 1 -- 172.21.15.62:0/253946264 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248032810 con 0x7f525c0b2d30 2024-04-25T09:08:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.553+0000 7f5262800640 1 -- 172.21.15.62:0/253946264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0b2d30 msgr2=0x7f525c0ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.553+0000 7f5262800640 1 --2- 172.21.15.62:0/253946264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0b2d30 0x7f525c0ce790 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f525c0d19a0 tx=0x7f524802a230 comp rx=0 tx=0).stop 2024-04-25T09:08:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.553+0000 7f5262800640 1 -- 172.21.15.62:0/253946264 shutdown_connections 2024-04-25T09:08:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.553+0000 7f5262800640 1 --2- 172.21.15.62:0/253946264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0b2d30 0x7f525c0ce790 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.553+0000 7f5262800640 1 --2- 172.21.15.62:0/253946264 >> v2:172.21.15.121:3300/0 conn(0x7f525c15e2c0 0x7f525c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.553+0000 7f5262800640 1 -- 172.21.15.62:0/253946264 >> 172.21.15.62:0/253946264 conn(0x7f525c0b3540 msgr2=0x7f525c0b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.553+0000 7f5262800640 1 -- 172.21.15.62:0/253946264 shutdown_connections 2024-04-25T09:08:30.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.553+0000 7f5262800640 1 -- 172.21.15.62:0/253946264 wait complete. 2024-04-25T09:08:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.555+0000 7f5262800640 1 Processor -- start 2024-04-25T09:08:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.555+0000 7f5262800640 1 -- start start 2024-04-25T09:08:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f5262800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0b2d30 0x7f525c14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:30.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f5262800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0cecd0 0x7f525c148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f5262800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15e2c0 0x7f525c0c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f5262800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0d0ea0 con 0x7f525c0cecd0 2024-04-25T09:08:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f5262800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0d0bc0 con 0x7f525c15e2c0 2024-04-25T09:08:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f5262800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0cd7a0 con 0x7f525c0b2d30 2024-04-25T09:08:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f525aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0cecd0 0x7f525c148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f525aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0cecd0 0x7f525c148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35126/0 (socket says 172.21.15.62:35126) 2024-04-25T09:08:30.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.556+0000 7f525aa00640 1 -- 172.21.15.62:0/1877630101 learned_addr learned my addr 172.21.15.62:0/1877630101 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:30.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.557+0000 7f525b400640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0b2d30 0x7f525c14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:30.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.557+0000 7f525aa00640 1 -- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0b2d30 msgr2=0x7f525c14b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:30.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.557+0000 7f525aa00640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0b2d30 0x7f525c14b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.557+0000 7f525aa00640 1 -- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15e2c0 msgr2=0x7f525c0c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.557+0000 7f525aa00640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15e2c0 0x7f525c0c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.557+0000 7f525aa00640 1 -- 172.21.15.62:0/1877630101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c1f3900 con 0x7f525c0cecd0 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.557+0000 7f525aa00640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0cecd0 0x7f525c148e00 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5248000c00 tx=0x7f52480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.558+0000 7f5253400640 1 -- 172.21.15.62:0/1877630101 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248041070 con 0x7f525c0cecd0 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.558+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c0c8360 con 0x7f525c0cecd0 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.558+0000 7f5253400640 1 -- 172.21.15.62:0/1877630101 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524803c870 con 0x7f525c0cecd0 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.558+0000 7f5253400640 1 -- 172.21.15.62:0/1877630101 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248037040 con 0x7f525c0cecd0 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.558+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c15bc20 con 0x7f525c0cecd0 2024-04-25T09:08:30.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.559+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c09fb30 con 0x7f525c0cecd0 2024-04-25T09:08:30.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.560+0000 7f5253400640 1 -- 172.21.15.62:0/1877630101 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5248044070 con 0x7f525c0cecd0 2024-04-25T09:08:30.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.563+0000 7f5253400640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523c06b4d0 0x7f523c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:30.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.563+0000 7f5253400640 1 -- 172.21.15.62:0/1877630101 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f52480aabd0 con 0x7f525c0cecd0 2024-04-25T09:08:30.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.563+0000 7f525b400640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523c06b4d0 0x7f523c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:30.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.563+0000 7f525b400640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523c06b4d0 0x7f523c06d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f524c008ee0 tx=0x7f524c008430 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:30.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.567+0000 7f5253400640 1 -- 172.21.15.62:0/1877630101 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f525c0b4450 con 0x7f525c0cecd0 2024-04-25T09:08:30.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.807+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f525c04f000 con 0x7f525c0cecd0 2024-04-25T09:08:30.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.808+0000 7f5253400640 1 -- 172.21.15.62:0/1877630101 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f5248079cb0 con 0x7f525c0cecd0 2024-04-25T09:08:30.797 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:30.797 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4805,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3643356404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3643356404},{"type":"v1","addr":"172.21.15.121:6835","nonce":3643356404}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/4121110303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":4121110303},{"type":"v1","addr":"172.21.15.121:6837","nonce":4121110303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/2907936690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2907936690},{"type":"v1","addr":"172.21.15.62:6834","nonce":2907936690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6321,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3846812213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3846812213},{"type":"v1","addr":"172.21.15.62:6835","nonce":3846812213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[]} 2024-04-25T09:08:30.797 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 32 2024-04-25T09:08:30.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523c06b4d0 msgr2=0x7f523c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:30.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523c06b4d0 0x7f523c06d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f524c008ee0 tx=0x7f524c008430 comp rx=0 tx=0).stop 2024-04-25T09:08:30.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0cecd0 msgr2=0x7f525c148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0cecd0 0x7f525c148e00 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5248000c00 tx=0x7f52480046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 shutdown_connections 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f523c06b4d0 0x7f523c06d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525c0cecd0 0x7f525c148e00 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f525c15e2c0 0x7f525c0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 --2- 172.21.15.62:0/1877630101 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f525c0b2d30 0x7f525c14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 >> 172.21.15.62:0/1877630101 conn(0x7f525c0b3540 msgr2=0x7f525c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 shutdown_connections 2024-04-25T09:08:30.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:30.812+0000 7f5262800640 1 -- 172.21.15.62:0/1877630101 wait complete. 2024-04-25T09:08:30.819 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:08:30.819 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:08:30.819 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:08:30.819 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:30.820 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:08:30.820 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:08:30.820 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:08:30.820 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:30.820 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:08:30.820 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:08:30.820 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:08:30.820 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:30.821 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:08:30.821 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:08:30.821 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:08:30.821 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:30.838 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:08:30.838 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:08:30.840 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:08:30.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:08:30.848 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:08:30.848 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:08:30.850 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:08:30.850 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:08:30.885 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:08:30.885 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:30.891 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:08:30.891 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:30.897 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:08:30.897 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:30.903 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:08:30.904 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:30.904 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:08:31.031 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:08:31.033 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:08:31.053 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:08:31.055 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:08:31.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.091+0000 7fb664000640 1 Processor -- start 2024-04-25T09:08:31.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.091+0000 7fb664000640 1 -- start start 2024-04-25T09:08:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb664000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb65c0cfed0 0x7fb65c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb664000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 0x7fb65c0b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb664000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c06c310 con 0x7fb65c072340 2024-04-25T09:08:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb664000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c15f5f0 con 0x7fb65c0cfed0 2024-04-25T09:08:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb664000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c06c5b0 con 0x7fb65c0709c0 2024-04-25T09:08:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb661600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb65c0cfed0 0x7fb65c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb661600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb65c0cfed0 0x7fb65c15f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35170/0 (socket says 172.21.15.62:35170) 2024-04-25T09:08:31.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb661600640 1 -- 172.21.15.62:0/1243738759 learned_addr learned my addr 172.21.15.62:0/1243738759 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:31.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb661600640 1 -- 172.21.15.62:0/1243738759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 msgr2=0x7fb65c0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:31.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb661600640 1 --2- 172.21.15.62:0/1243738759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 0x7fb65c0b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:31.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb661600640 1 -- 172.21.15.62:0/1243738759 >> v1:172.21.15.62:6789/0 conn(0x7fb65c072340 legacy=0x7fb65c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:31.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.093+0000 7fb661600640 1 -- 172.21.15.62:0/1243738759 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c06c890 con 0x7fb65c0cfed0 2024-04-25T09:08:31.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.094+0000 7fb661600640 1 --2- 172.21.15.62:0/1243738759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c0cfed0 0x7fb65c15f0b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb650009870 tx=0x7fb65002c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311ee89783f577fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:31.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb65be00640 1 -- 172.21.15.62:0/1243738759 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650038070 con 0x7fb65c0cfed0 2024-04-25T09:08:31.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb65be00640 1 -- 172.21.15.62:0/1243738759 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb650031710 con 0x7fb65c0cfed0 2024-04-25T09:08:31.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb65be00640 1 -- 172.21.15.62:0/1243738759 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650031ae0 con 0x7fb65c0cfed0 2024-04-25T09:08:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb664000640 1 -- 172.21.15.62:0/1243738759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c0cfed0 msgr2=0x7fb65c15f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb664000640 1 --2- 172.21.15.62:0/1243738759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c0cfed0 0x7fb65c15f0b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb650009870 tx=0x7fb65002c460 comp rx=0 tx=0).stop 2024-04-25T09:08:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb664000640 1 -- 172.21.15.62:0/1243738759 shutdown_connections 2024-04-25T09:08:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb664000640 1 --2- 172.21.15.62:0/1243738759 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 0x7fb65c0b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb664000640 1 --2- 172.21.15.62:0/1243738759 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c0cfed0 0x7fb65c15f0b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb664000640 1 -- 172.21.15.62:0/1243738759 >> 172.21.15.62:0/1243738759 conn(0x7fb65c0668f0 msgr2=0x7fb65c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb664000640 1 -- 172.21.15.62:0/1243738759 shutdown_connections 2024-04-25T09:08:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.095+0000 7fb664000640 1 -- 172.21.15.62:0/1243738759 wait complete. 2024-04-25T09:08:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.097+0000 7fb664000640 1 Processor -- start 2024-04-25T09:08:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.097+0000 7fb664000640 1 -- start start 2024-04-25T09:08:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb664000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 0x7fb65c15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb664000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb65c0cfed0 0x7fb65c134f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb664000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c15a790 0x7fb65c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb664000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c06d130 con 0x7fb65c15a790 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb664000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c06ce50 con 0x7fb65c0cfed0 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb664000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0b2dc0 con 0x7fb65c0709c0 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb662000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c15a790 0x7fb65c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb662000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c15a790 0x7fb65c0ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35178/0 (socket says 172.21.15.62:35178) 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb662000640 1 -- 172.21.15.62:0/401609192 learned_addr learned my addr 172.21.15.62:0/401609192 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb661600640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 0x7fb65c15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.098+0000 7fb660c00640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb65c0cfed0 0x7fb65c134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb662000640 1 -- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 msgr2=0x7fb65c15c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb662000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 0x7fb65c15c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb662000640 1 -- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb65c0cfed0 msgr2=0x7fb65c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb662000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb65c0cfed0 0x7fb65c134f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb662000640 1 -- 172.21.15.62:0/401609192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c1354c0 con 0x7fb65c15a790 2024-04-25T09:08:31.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb662000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c15a790 0x7fb65c0ad8e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb65400b9e0 tx=0x7fb65400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb659600640 1 -- 172.21.15.62:0/401609192 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654019070 con 0x7fb65c15a790 2024-04-25T09:08:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb659600640 1 -- 172.21.15.62:0/401609192 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65400cb40 con 0x7fb65c15a790 2024-04-25T09:08:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb659600640 1 -- 172.21.15.62:0/401609192 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654014440 con 0x7fb65c15a790 2024-04-25T09:08:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.099+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c15c760 con 0x7fb65c15a790 2024-04-25T09:08:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.100+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c15cd20 con 0x7fb65c15a790 2024-04-25T09:08:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.101+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c072c70 con 0x7fb65c15a790 2024-04-25T09:08:31.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.101+0000 7fb659600640 1 -- 172.21.15.62:0/401609192 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb65401c070 con 0x7fb65c15a790 2024-04-25T09:08:31.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.102+0000 7fb659600640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb63c06b4d0 0x7fb63c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:31.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.103+0000 7fb659600640 1 -- 172.21.15.62:0/401609192 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb654089c20 con 0x7fb65c15a790 2024-04-25T09:08:31.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.103+0000 7fb661600640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb63c06b4d0 0x7fb63c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:31.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.104+0000 7fb661600640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb63c06b4d0 0x7fb63c06d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb650035000 tx=0x7fb650008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:31.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.106+0000 7fb659600640 1 -- 172.21.15.62:0/401609192 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb65c1354c0 con 0x7fb65c15a790 2024-04-25T09:08:31.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:31.318+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb65c04f000 con 0x7fb65c15a790 2024-04-25T09:08:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.440+0000 7fb659600640 1 -- 172.21.15.62:0/401609192 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb6540586b0 con 0x7fb65c15a790 2024-04-25T09:08:32.428 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.443+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb63c06b4d0 msgr2=0x7fb63c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.443+0000 7fb664000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb63c06b4d0 0x7fb63c06d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb650035000 tx=0x7fb650008030 comp rx=0 tx=0).stop 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.443+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c15a790 msgr2=0x7fb65c0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.443+0000 7fb664000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c15a790 0x7fb65c0ad8e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb65400b9e0 tx=0x7fb65400bea0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.444+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 shutdown_connections 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.444+0000 7fb664000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb63c06b4d0 0x7fb63c06d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.444+0000 7fb664000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb65c15a790 0x7fb65c0ad8e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.444+0000 7fb664000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb65c0cfed0 0x7fb65c134f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.444+0000 7fb664000640 1 --2- 172.21.15.62:0/401609192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb65c0709c0 0x7fb65c15c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.444+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 >> 172.21.15.62:0/401609192 conn(0x7fb65c0668f0 msgr2=0x7fb65c0b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.444+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 shutdown_connections 2024-04-25T09:08:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.444+0000 7fb664000640 1 -- 172.21.15.62:0/401609192 wait complete. 2024-04-25T09:08:32.454 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:08:32.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.680+0000 7fb881000640 1 Processor -- start 2024-04-25T09:08:32.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.680+0000 7fb881000640 1 -- start start 2024-04-25T09:08:32.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.681+0000 7fb881000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb87c144100 0x7fb87c0b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.681+0000 7fb881000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 0x7fb87c0add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.681+0000 7fb881000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c0b3b70 con 0x7fb87c0ad8e0 2024-04-25T09:08:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.681+0000 7fb881000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87c0b4460 con 0x7fb87c144100 2024-04-25T09:08:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.681+0000 7fb881000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c06d110 con 0x7fb87c06caf0 2024-04-25T09:08:32.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.682+0000 7fb879400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb87c0ad8e0 0x7fb87c06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51964/0 (socket says 172.21.15.62:51964) 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.682+0000 7fb879400640 1 -- 172.21.15.62:0/4165664499 learned_addr learned my addr 172.21.15.62:0/4165664499 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.682+0000 7fb87a800640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 0x7fb87c0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.682+0000 7fb879e00640 1 --2- 172.21.15.62:0/4165664499 >> v2:172.21.15.121:3300/0 conn(0x7fb87c144100 0x7fb87c0b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.682+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129424486 0 0) 0x7fb87c0b3b70 con 0x7fb87c0ad8e0 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.682+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb860003520 con 0x7fb87c0ad8e0 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2138094905 0 0) 0x7fb860003520 con 0x7fb87c0ad8e0 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 msgr2=0x7fb87c0add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 0x7fb87c0add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 >> v2:172.21.15.121:3300/0 conn(0x7fb87c144100 msgr2=0x7fb87c0b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 --2- 172.21.15.62:0/4165664499 >> v2:172.21.15.121:3300/0 conn(0x7fb87c144100 0x7fb87c0b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0b3e10 con 0x7fb87c0ad8e0 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb8640019f0 con 0x7fb87c0ad8e0 2024-04-25T09:08:32.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 >> v1:172.21.15.62:6789/0 conn(0x7fb87c0ad8e0 legacy=0x7fb87c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 0x7fb8600039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb860003fa0 0x7fb860004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb860004d40 0x7fb8600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0b4460 con 0x7fb860004d40 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c06d110 con 0x7fb860003fa0 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86000b9f0 con 0x7fb87c06caf0 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.683+0000 7fb87a800640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb860004d40 0x7fb8600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.684+0000 7fb879400640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb860003fa0 0x7fb860004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.684+0000 7fb87a800640 1 -- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 msgr2=0x7fb8600039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.684+0000 7fb879e00640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 0x7fb8600039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.684+0000 7fb87a800640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 0x7fb8600039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.684+0000 7fb87a800640 1 -- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb860003fa0 msgr2=0x7fb860004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.684+0000 7fb87a800640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb860003fa0 0x7fb860004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.684+0000 7fb87a800640 1 -- 172.21.15.62:0/4165664499 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb860003520 con 0x7fb860004d40 2024-04-25T09:08:32.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.684+0000 7fb879e00640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb87c06caf0 0x7fb8600039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.685+0000 7fb87a800640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb860004d40 0x7fb8600051c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb874009ac0 tx=0x7fb860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f28176085b91d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.685+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874036070 con 0x7fb860004d40 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.685+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874031400 con 0x7fb860004d40 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.685+0000 7fb878a00640 1 -- 172.21.15.62:0/4165664499 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874009690 con 0x7fb860004d40 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.686+0000 7fb881000640 1 -- 172.21.15.62:0/4165664499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb860004d40 msgr2=0x7fb8600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.686+0000 7fb881000640 1 --2- 172.21.15.62:0/4165664499 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb860004d40 0x7fb8600051c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb874009ac0 tx=0x7fb860005830 comp rx=0 tx=0).stop 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.686+0000 7fb87a800640 1 -- 172.21.15.62:0/4165664499 reap_dead start 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.686+0000 7fb881000640 1 -- 172.21.15.62:0/4165664499 shutdown_connections 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.686+0000 7fb881000640 1 -- 172.21.15.62:0/4165664499 >> 172.21.15.62:0/4165664499 conn(0x7fb87c0668f0 msgr2=0x7fb87c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.686+0000 7fb881000640 1 -- 172.21.15.62:0/4165664499 shutdown_connections 2024-04-25T09:08:32.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.686+0000 7fb881000640 1 -- 172.21.15.62:0/4165664499 wait complete. 2024-04-25T09:08:32.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.688+0000 7fb881000640 1 Processor -- start 2024-04-25T09:08:32.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.688+0000 7fb881000640 1 -- start start 2024-04-25T09:08:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb881000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb860004d40 0x7fb87c1f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb881000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb87c140620 0x7fb87c160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb881000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb87c1f4120 0x7fb87c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb881000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c144660 con 0x7fb87c1f4120 2024-04-25T09:08:32.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb881000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c1444e0 con 0x7fb87c140620 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb881000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0add70 con 0x7fb860004d40 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb87a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb87c1f4120 0x7fb87c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb87a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb87c1f4120 0x7fb87c15d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35204/0 (socket says 172.21.15.62:35204) 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.689+0000 7fb87a800640 1 -- 172.21.15.62:0/4033970921 learned_addr learned my addr 172.21.15.62:0/4033970921 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb879e00640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb860004d40 0x7fb87c1f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb879400640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb87c140620 0x7fb87c160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb87a800640 1 -- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb860004d40 msgr2=0x7fb87c1f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb87a800640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb860004d40 0x7fb87c1f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb87a800640 1 -- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb87c140620 msgr2=0x7fb87c160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb87a800640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb87c140620 0x7fb87c160610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb87a800640 1 -- 172.21.15.62:0/4033970921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c1f7400 con 0x7fb87c1f4120 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb879e00640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb860004d40 0x7fb87c1f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:32.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.690+0000 7fb87a800640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb87c1f4120 0x7fb87c15d070 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb874009ac0 tx=0x7fb874004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:32.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.691+0000 7fb872000640 1 -- 172.21.15.62:0/4033970921 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874036040 con 0x7fb87c1f4120 2024-04-25T09:08:32.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.691+0000 7fb872000640 1 -- 172.21.15.62:0/4033970921 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874041070 con 0x7fb87c1f4120 2024-04-25T09:08:32.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.691+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c1f76a0 con 0x7fb87c1f4120 2024-04-25T09:08:32.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.691+0000 7fb872000640 1 -- 172.21.15.62:0/4033970921 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874009690 con 0x7fb87c1f4120 2024-04-25T09:08:32.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.691+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c1f7c60 con 0x7fb87c1f4120 2024-04-25T09:08:32.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.692+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c04ef20 con 0x7fb87c1f4120 2024-04-25T09:08:32.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.693+0000 7fb872000640 1 -- 172.21.15.62:0/4033970921 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb874039070 con 0x7fb87c1f4120 2024-04-25T09:08:32.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.695+0000 7fb872000640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb85806b4d0 0x7fb85806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:32.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.695+0000 7fb872000640 1 -- 172.21.15.62:0/4033970921 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb8740a9680 con 0x7fb87c1f4120 2024-04-25T09:08:32.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.695+0000 7fb879e00640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb85806b4d0 0x7fb85806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:32.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.697+0000 7fb879e00640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb85806b4d0 0x7fb85806d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009b20 tx=0x7fb86c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:32.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.700+0000 7fb872000640 1 -- 172.21.15.62:0/4033970921 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb87404a020 con 0x7fb87c1f4120 2024-04-25T09:08:32.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.977+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb87c070e20 con 0x7fb87c1f4120 2024-04-25T09:08:32.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.978+0000 7fb872000640 1 -- 172.21.15.62:0/4033970921 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb8740787b0 con 0x7fb87c1f4120 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:08:32.966 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:08:32.967 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.968 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:08:32.969 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.970 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.971 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.972 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.973 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.974 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.975 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:08:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:08:32.977 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:32.978 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:08:32.979 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:32.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.981+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb85806b4d0 msgr2=0x7fb85806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.981+0000 7fb881000640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb85806b4d0 0x7fb85806d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009b20 tx=0x7fb86c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.981+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb87c1f4120 msgr2=0x7fb87c15d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.981+0000 7fb881000640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb87c1f4120 0x7fb87c15d070 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb874009ac0 tx=0x7fb874004730 comp rx=0 tx=0).stop 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.982+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 shutdown_connections 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.982+0000 7fb881000640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb85806b4d0 0x7fb85806d990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.982+0000 7fb881000640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb87c1f4120 0x7fb87c15d070 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.982+0000 7fb881000640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb87c140620 0x7fb87c160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.982+0000 7fb881000640 1 --2- 172.21.15.62:0/4033970921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb860004d40 0x7fb87c1f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.982+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 >> 172.21.15.62:0/4033970921 conn(0x7fb87c0668f0 msgr2=0x7fb87c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.982+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 shutdown_connections 2024-04-25T09:08:32.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:32.982+0000 7fb881000640 1 -- 172.21.15.62:0/4033970921 wait complete. 2024-04-25T09:08:32.989 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:33.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.206+0000 7f4074a00640 1 Processor -- start 2024-04-25T09:08:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.206+0000 7f4074a00640 1 -- start start 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.208+0000 7f4074a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4070144c10 0x7f407015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.208+0000 7f4074a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f407006d900 0x7f40700b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.208+0000 7f4074a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407015faa0 con 0x7f40700cee40 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.208+0000 7f4074a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4070144ff0 con 0x7f407006d900 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.208+0000 7f4074a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4070160340 con 0x7f4070144c10 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.208+0000 7f406fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f407006d900 0x7f40700b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.209+0000 7f406fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f407006d900 0x7f40700b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35218/0 (socket says 172.21.15.62:35218) 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.209+0000 7f406fe00640 1 -- 172.21.15.62:0/4209321031 learned_addr learned my addr 172.21.15.62:0/4209321031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.209+0000 7f406f400640 1 --2- 172.21.15.62:0/4209321031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4070144c10 0x7f407015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.209+0000 7f406fe00640 1 -- 172.21.15.62:0/4209321031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4070144c10 msgr2=0x7f407015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.209+0000 7f406fe00640 1 --2- 172.21.15.62:0/4209321031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4070144c10 0x7f407015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.209+0000 7f406fe00640 1 -- 172.21.15.62:0/4209321031 >> v1:172.21.15.121:6789/0 conn(0x7f40700cee40 legacy=0x7f407006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.209+0000 7f406fe00640 1 -- 172.21.15.62:0/4209321031 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407015fd80 con 0x7f407006d900 2024-04-25T09:08:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.211+0000 7f406fe00640 1 --2- 172.21.15.62:0/4209321031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f407006d900 0x7f40700b4650 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4068009e80 tx=0x7f406802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3958acd57413f456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:33.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.211+0000 7f406e000640 1 -- 172.21.15.62:0/4209321031 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068036070 con 0x7f407006d900 2024-04-25T09:08:33.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.211+0000 7f406e000640 1 -- 172.21.15.62:0/4209321031 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4068031600 con 0x7f407006d900 2024-04-25T09:08:33.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.211+0000 7f406e000640 1 -- 172.21.15.62:0/4209321031 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40680319d0 con 0x7f407006d900 2024-04-25T09:08:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.212+0000 7f4074a00640 1 -- 172.21.15.62:0/4209321031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f407006d900 msgr2=0x7f40700b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.212+0000 7f4074a00640 1 --2- 172.21.15.62:0/4209321031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f407006d900 0x7f40700b4650 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4068009e80 tx=0x7f406802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.212+0000 7f4074a00640 1 -- 172.21.15.62:0/4209321031 shutdown_connections 2024-04-25T09:08:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.212+0000 7f4074a00640 1 --2- 172.21.15.62:0/4209321031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f407006d900 0x7f40700b4650 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.212+0000 7f4074a00640 1 --2- 172.21.15.62:0/4209321031 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4070144c10 0x7f407015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.212+0000 7f4074a00640 1 -- 172.21.15.62:0/4209321031 >> 172.21.15.62:0/4209321031 conn(0x7f40700b14d0 msgr2=0x7f40700b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.212+0000 7f4074a00640 1 -- 172.21.15.62:0/4209321031 shutdown_connections 2024-04-25T09:08:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.212+0000 7f4074a00640 1 -- 172.21.15.62:0/4209321031 wait complete. 2024-04-25T09:08:33.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.214+0000 7f4074a00640 1 Processor -- start 2024-04-25T09:08:33.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.214+0000 7f4074a00640 1 -- start start 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f4074a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f407006d900 0x7f40701f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f4074a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40700cee40 0x7f40701f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f4074a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4070144c10 0x7f40701f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f4074a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40700cf2a0 con 0x7f4070144c10 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f4074a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40700b41c0 con 0x7f40700cee40 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f4074a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40700b4b90 con 0x7f407006d900 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4070144c10 0x7f40701f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4070144c10 0x7f40701f5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35230/0 (socket says 172.21.15.62:35230) 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406fe00640 1 -- 172.21.15.62:0/3782576179 learned_addr learned my addr 172.21.15.62:0/3782576179 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406ea00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40700cee40 0x7f40701f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406f400640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f407006d900 0x7f40701f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406fe00640 1 -- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f407006d900 msgr2=0x7f40701f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406fe00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f407006d900 0x7f40701f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406fe00640 1 -- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40700cee40 msgr2=0x7f40701f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406fe00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40700cee40 0x7f40701f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406fe00640 1 -- 172.21.15.62:0/3782576179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40701f5430 con 0x7f4070144c10 2024-04-25T09:08:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406ea00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40700cee40 0x7f40701f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.215+0000 7f406f400640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f407006d900 0x7f40701f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.216+0000 7f406fe00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4070144c10 0x7f40701f5630 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4068034040 tx=0x7f4068004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.216+0000 7f4067400640 1 -- 172.21.15.62:0/3782576179 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068036050 con 0x7f4070144c10 2024-04-25T09:08:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.216+0000 7f4067400640 1 -- 172.21.15.62:0/3782576179 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4068042070 con 0x7f4070144c10 2024-04-25T09:08:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.216+0000 7f4067400640 1 -- 172.21.15.62:0/3782576179 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068009510 con 0x7f4070144c10 2024-04-25T09:08:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.216+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40701f1ea0 con 0x7f4070144c10 2024-04-25T09:08:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.216+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4070139e40 con 0x7f4070144c10 2024-04-25T09:08:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.217+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40700c4870 con 0x7f4070144c10 2024-04-25T09:08:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.218+0000 7f4067400640 1 -- 172.21.15.62:0/3782576179 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4068007bd0 con 0x7f4070144c10 2024-04-25T09:08:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.219+0000 7f4067400640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f405006b4d0 0x7f405006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.219+0000 7f4067400640 1 -- 172.21.15.62:0/3782576179 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40680aae40 con 0x7f4070144c10 2024-04-25T09:08:33.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.220+0000 7f406f400640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f405006b4d0 0x7f405006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.221+0000 7f406f400640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f405006b4d0 0x7f405006d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4060006fd0 tx=0x7f4060008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:33.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.223+0000 7f4067400640 1 -- 172.21.15.62:0/3782576179 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40700c4870 con 0x7f4070144c10 2024-04-25T09:08:33.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.415+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40700ca130 con 0x7f4070144c10 2024-04-25T09:08:33.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.416+0000 7f4067400640 1 -- 172.21.15.62:0/3782576179 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f40701f5430 con 0x7f4070144c10 2024-04-25T09:08:33.404 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:33.404 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":58,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:32.436837+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:33.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.418+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f405006b4d0 msgr2=0x7f405006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.418+0000 7f4074a00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f405006b4d0 0x7f405006d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4060006fd0 tx=0x7f4060008040 comp rx=0 tx=0).stop 2024-04-25T09:08:33.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.418+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4070144c10 msgr2=0x7f40701f5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.418+0000 7f4074a00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4070144c10 0x7f40701f5630 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4068034040 tx=0x7f4068004a40 comp rx=0 tx=0).stop 2024-04-25T09:08:33.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.419+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 shutdown_connections 2024-04-25T09:08:33.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.419+0000 7f4074a00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f405006b4d0 0x7f405006d990 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.419+0000 7f4074a00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4070144c10 0x7f40701f5630 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.419+0000 7f4074a00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40700cee40 0x7f40701f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.419+0000 7f4074a00640 1 --2- 172.21.15.62:0/3782576179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f407006d900 0x7f40701f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.419+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 >> 172.21.15.62:0/3782576179 conn(0x7f40700b14d0 msgr2=0x7f40700b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:33.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.419+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 shutdown_connections 2024-04-25T09:08:33.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.419+0000 7f4074a00640 1 -- 172.21.15.62:0/3782576179 wait complete. 2024-04-25T09:08:33.423 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:33.424 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:33.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.639+0000 7f3613200640 1 Processor -- start 2024-04-25T09:08:33.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.640+0000 7f3613200640 1 -- start start 2024-04-25T09:08:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.641+0000 7f3613200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f361415c520 0x7f36140b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.641+0000 7f3613200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f361406d9d0 0x7f36140ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.641+0000 7f3613200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36140b4640 con 0x7f36140b3360 2024-04-25T09:08:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.641+0000 7f3613200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3614165c40 con 0x7f361406d9d0 2024-04-25T09:08:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.642+0000 7f3613200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3614165f20 con 0x7f361415c520 2024-04-25T09:08:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.642+0000 7f3612800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f361406d9d0 0x7f36140ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.642+0000 7f3611e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f361415c520 0x7f36140b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.642+0000 7f3612800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f361406d9d0 0x7f36140ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:38356/0 (socket says 172.21.15.62:38356) 2024-04-25T09:08:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.642+0000 7f3612800640 1 -- 172.21.15.62:0/435333688 learned_addr learned my addr 172.21.15.62:0/435333688 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.643+0000 7f3612800640 1 -- 172.21.15.62:0/435333688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f361415c520 msgr2=0x7f36140b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.643+0000 7f3612800640 1 --2- 172.21.15.62:0/435333688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f361415c520 0x7f36140b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.643+0000 7f3612800640 1 -- 172.21.15.62:0/435333688 >> v1:172.21.15.121:6789/0 conn(0x7f36140b3360 legacy=0x7f361406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.643+0000 7f3612800640 1 -- 172.21.15.62:0/435333688 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614166200 con 0x7f361406d9d0 2024-04-25T09:08:33.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.643+0000 7f3611e00640 1 --2- 172.21.15.62:0/435333688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f361415c520 0x7f36140b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:08:33.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.644+0000 7f3612800640 1 --2- 172.21.15.62:0/435333688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361406d9d0 0x7f36140ce6f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f360c009e50 tx=0x7f360c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46c85c8cfd79f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:33.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.645+0000 7f3611400640 1 -- 172.21.15.62:0/435333688 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c036070 con 0x7f361406d9d0 2024-04-25T09:08:33.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.645+0000 7f3611400640 1 -- 172.21.15.62:0/435333688 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c031600 con 0x7f361406d9d0 2024-04-25T09:08:33.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.645+0000 7f3611400640 1 -- 172.21.15.62:0/435333688 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0319d0 con 0x7f361406d9d0 2024-04-25T09:08:33.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.646+0000 7f3613200640 1 -- 172.21.15.62:0/435333688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361406d9d0 msgr2=0x7f36140ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.646+0000 7f3613200640 1 --2- 172.21.15.62:0/435333688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361406d9d0 0x7f36140ce6f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f360c009e50 tx=0x7f360c02a5f0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.646+0000 7f3613200640 1 -- 172.21.15.62:0/435333688 shutdown_connections 2024-04-25T09:08:33.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.646+0000 7f3613200640 1 --2- 172.21.15.62:0/435333688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361406d9d0 0x7f36140ce6f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.646+0000 7f3613200640 1 --2- 172.21.15.62:0/435333688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f361415c520 0x7f36140b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.646+0000 7f3613200640 1 -- 172.21.15.62:0/435333688 >> 172.21.15.62:0/435333688 conn(0x7f3614066f20 msgr2=0x7f36140b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.646+0000 7f3613200640 1 -- 172.21.15.62:0/435333688 shutdown_connections 2024-04-25T09:08:33.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.646+0000 7f3613200640 1 -- 172.21.15.62:0/435333688 wait complete. 2024-04-25T09:08:33.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.648+0000 7f3613200640 1 Processor -- start 2024-04-25T09:08:33.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.648+0000 7f3613200640 1 -- start start 2024-04-25T09:08:33.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3613200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f361406d9d0 0x7f361414e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3613200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36140b3360 0x7f36141ec690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3613200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361415c520 0x7f36140ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3613200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140b4250 con 0x7f361406d9d0 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3613200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361415c900 con 0x7f361415c520 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3613200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36140b43d0 con 0x7f36140b3360 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3611e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f361406d9d0 0x7f361414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3611e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f361406d9d0 0x7f361414e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35236/0 (socket says 172.21.15.62:35236) 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3611e00640 1 -- 172.21.15.62:0/1633639951 learned_addr learned my addr 172.21.15.62:0/1633639951 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3612800640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361415c520 0x7f36140ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3611e00640 1 -- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36140b3360 msgr2=0x7f36141ec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3611e00640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36140b3360 0x7f36141ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3611e00640 1 -- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361415c520 msgr2=0x7f36140ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3611e00640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361415c520 0x7f36140ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.650+0000 7f3611e00640 1 -- 172.21.15.62:0/1633639951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361414ecb0 con 0x7f361406d9d0 2024-04-25T09:08:33.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.651+0000 7f3611e00640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f361406d9d0 0x7f361414e770 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f360400e960 tx=0x7f360400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:33.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.651+0000 7f360a000640 1 -- 172.21.15.62:0/1633639951 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604017070 con 0x7f361406d9d0 2024-04-25T09:08:33.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.651+0000 7f360a000640 1 -- 172.21.15.62:0/1633639951 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360400ca00 con 0x7f361406d9d0 2024-04-25T09:08:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.651+0000 7f360a000640 1 -- 172.21.15.62:0/1633639951 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360400cdd0 con 0x7f361406d9d0 2024-04-25T09:08:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.651+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36141ecfe0 con 0x7f361406d9d0 2024-04-25T09:08:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.651+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36141ed5a0 con 0x7f361406d9d0 2024-04-25T09:08:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.652+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361404ef20 con 0x7f361406d9d0 2024-04-25T09:08:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.653+0000 7f360a000640 1 -- 172.21.15.62:0/1633639951 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f360401a070 con 0x7f361406d9d0 2024-04-25T09:08:33.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.654+0000 7f360a000640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f35f006b4d0 0x7f35f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:33.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.654+0000 7f360a000640 1 -- 172.21.15.62:0/1633639951 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3604089e00 con 0x7f361406d9d0 2024-04-25T09:08:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.655+0000 7f360be00640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f35f006b4d0 0x7f35f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:33.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.656+0000 7f360be00640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f35f006b4d0 0x7f35f006d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f361415b800 tx=0x7f35fc007970 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.659+0000 7f360a000640 1 -- 172.21.15.62:0/1633639951 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f361414ecb0 con 0x7f361406d9d0 2024-04-25T09:08:33.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.855+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36140b1110 con 0x7f361406d9d0 2024-04-25T09:08:33.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.856+0000 7f360a000640 1 -- 172.21.15.62:0/1633639951 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3604058f30 con 0x7f361406d9d0 2024-04-25T09:08:33.843 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:08:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.858+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f35f006b4d0 msgr2=0x7f35f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.858+0000 7f3613200640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f35f006b4d0 0x7f35f006d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f361415b800 tx=0x7f35fc007970 comp rx=0 tx=0).stop 2024-04-25T09:08:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.858+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f361406d9d0 msgr2=0x7f361414e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.858+0000 7f3613200640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f361406d9d0 0x7f361414e770 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f360400e960 tx=0x7f360400ee20 comp rx=0 tx=0).stop 2024-04-25T09:08:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.859+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 shutdown_connections 2024-04-25T09:08:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.859+0000 7f3613200640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f35f006b4d0 0x7f35f006d990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.859+0000 7f3613200640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f361415c520 0x7f36140ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.859+0000 7f3613200640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f36140b3360 0x7f36141ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.859+0000 7f3613200640 1 --2- 172.21.15.62:0/1633639951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f361406d9d0 0x7f361414e770 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:33.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.859+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 >> 172.21.15.62:0/1633639951 conn(0x7f3614066f20 msgr2=0x7f3614163f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:33.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.859+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 shutdown_connections 2024-04-25T09:08:33.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:33.859+0000 7f3613200640 1 -- 172.21.15.62:0/1633639951 wait complete. 2024-04-25T09:08:33.864 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:08:34.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.079+0000 7fb01ee00640 1 Processor -- start 2024-04-25T09:08:34.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.080+0000 7fb01ee00640 1 -- start start 2024-04-25T09:08:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.081+0000 7fb01ee00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb0180d09e0 0x7fb0180d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.081+0000 7fb01ee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0181622e0 con 0x7fb018161140 2024-04-25T09:08:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.081+0000 7fb01ee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0181625c0 con 0x7fb0180d0550 2024-04-25T09:08:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.081+0000 7fb01ee00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0180d1d00 con 0x7fb0180d09e0 2024-04-25T09:08:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.082+0000 7fb01ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb0180d09e0 0x7fb0180d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.082+0000 7fb017e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb018161140 0x7fb0181618b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51978/0 (socket says 172.21.15.62:51978) 2024-04-25T09:08:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.082+0000 7fb017e00640 1 -- 172.21.15.62:0/3449744115 learned_addr learned my addr 172.21.15.62:0/3449744115 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:34.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.082+0000 7fb01ce00640 1 -- 172.21.15.62:0/3449744115 >> v1:172.21.15.121:6789/0 conn(0x7fb0180d0550 legacy=0x7fb0180d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:34.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.082+0000 7fb01ce00640 1 -- 172.21.15.62:0/3449744115 >> v1:172.21.15.62:6789/0 conn(0x7fb018161140 legacy=0x7fb0181618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.082+0000 7fb016a00640 1 -- 172.21.15.62:0/3449744115 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399474075 0 0) 0x7fb0181622e0 con 0x7fb018161140 2024-04-25T09:08:34.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.082+0000 7fb01ce00640 1 -- 172.21.15.62:0/3449744115 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180d1fe0 con 0x7fb0180d09e0 2024-04-25T09:08:34.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.083+0000 7fb01ce00640 1 --2- 172.21.15.62:0/3449744115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d09e0 0x7fb0180d2c10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb0100097a0 tx=0x7fb01002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dd37083b1b9903 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:34.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb016a00640 1 -- 172.21.15.62:0/3449744115 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010037070 con 0x7fb0180d09e0 2024-04-25T09:08:34.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb016a00640 1 -- 172.21.15.62:0/3449744115 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb010032440 con 0x7fb0180d09e0 2024-04-25T09:08:34.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb016a00640 1 -- 172.21.15.62:0/3449744115 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010032810 con 0x7fb0180d09e0 2024-04-25T09:08:34.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb01ee00640 1 -- 172.21.15.62:0/3449744115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d09e0 msgr2=0x7fb0180d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb01ee00640 1 --2- 172.21.15.62:0/3449744115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d09e0 0x7fb0180d2c10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb0100097a0 tx=0x7fb01002a230 comp rx=0 tx=0).stop 2024-04-25T09:08:34.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb01ee00640 1 -- 172.21.15.62:0/3449744115 shutdown_connections 2024-04-25T09:08:34.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb01ee00640 1 --2- 172.21.15.62:0/3449744115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d09e0 0x7fb0180d2c10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb01ee00640 1 -- 172.21.15.62:0/3449744115 >> 172.21.15.62:0/3449744115 conn(0x7fb0180668f0 msgr2=0x7fb018066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:34.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb01ee00640 1 -- 172.21.15.62:0/3449744115 shutdown_connections 2024-04-25T09:08:34.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.084+0000 7fb01ee00640 1 -- 172.21.15.62:0/3449744115 wait complete. 2024-04-25T09:08:34.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.086+0000 7fb01ee00640 1 Processor -- start 2024-04-25T09:08:34.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.086+0000 7fb01ee00640 1 -- start start 2024-04-25T09:08:34.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.087+0000 7fb01ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d0550 0x7fb018153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.087+0000 7fb01ee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0180d09e0 0x7fb0180cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.087+0000 7fb01ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb018161140 0x7fb0180c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.087+0000 7fb01ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018161520 con 0x7fb0180d0550 2024-04-25T09:08:34.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.087+0000 7fb01ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180d0df0 con 0x7fb018161140 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.087+0000 7fb01ee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0181616a0 con 0x7fb0180d09e0 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d0550 0x7fb018153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d0550 0x7fb018153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35244/0 (socket says 172.21.15.62:35244) 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017e00640 1 -- 172.21.15.62:0/3087503593 learned_addr learned my addr 172.21.15.62:0/3087503593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017400640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0180d09e0 0x7fb0180cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb01ce00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb018161140 0x7fb0180c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017e00640 1 -- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0180d09e0 msgr2=0x7fb0180cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017e00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0180d09e0 0x7fb0180cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017e00640 1 -- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb018161140 msgr2=0x7fb0180c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017e00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb018161140 0x7fb0180c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb017e00640 1 -- 172.21.15.62:0/3087503593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180cc7b0 con 0x7fb0180d0550 2024-04-25T09:08:34.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.088+0000 7fb01ce00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb018161140 0x7fb0180c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:34.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.089+0000 7fb017e00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d0550 0x7fb018153960 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c420 tx=0x7fb00800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:34.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.089+0000 7fb00fe00640 1 -- 172.21.15.62:0/3087503593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0080074e0 con 0x7fb0180d0550 2024-04-25T09:08:34.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.089+0000 7fb00fe00640 1 -- 172.21.15.62:0/3087503593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0080047e0 con 0x7fb0180d0550 2024-04-25T09:08:34.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.089+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0180cca90 con 0x7fb0180d0550 2024-04-25T09:08:34.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.089+0000 7fb00fe00640 1 -- 172.21.15.62:0/3087503593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008004c10 con 0x7fb0180d0550 2024-04-25T09:08:34.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.089+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0181e82e0 con 0x7fb0180d0550 2024-04-25T09:08:34.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.090+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018068530 con 0x7fb0180d0550 2024-04-25T09:08:34.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.091+0000 7fb00fe00640 1 -- 172.21.15.62:0/3087503593 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb008011070 con 0x7fb0180d0550 2024-04-25T09:08:34.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.092+0000 7fb00fe00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faff406b520 0x7faff406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.092+0000 7fb00fe00640 1 -- 172.21.15.62:0/3087503593 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb0080899f0 con 0x7fb0180d0550 2024-04-25T09:08:34.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.092+0000 7fb017400640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faff406b520 0x7faff406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.094+0000 7fb017400640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faff406b520 0x7faff406d9e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb0180728d0 tx=0x7fb000009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:34.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.096+0000 7fb00fe00640 1 -- 172.21.15.62:0/3087503593 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb018068530 con 0x7fb0180d0550 2024-04-25T09:08:34.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.298+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb01804ef20 con 0x7fb0180d0550 2024-04-25T09:08:34.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.299+0000 7fb00fe00640 1 -- 172.21.15.62:0/3087503593 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb00800e280 con 0x7fb0180d0550 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:08:34.287 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:08:34.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faff406b520 msgr2=0x7faff406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faff406b520 0x7faff406d9e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb0180728d0 tx=0x7fb000009040 comp rx=0 tx=0).stop 2024-04-25T09:08:34.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d0550 msgr2=0x7fb018153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d0550 0x7fb018153960 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c420 tx=0x7fb00800c8e0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 shutdown_connections 2024-04-25T09:08:34.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faff406b520 0x7faff406d9e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb018161140 0x7fb0180c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0180d09e0 0x7fb0180cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 --2- 172.21.15.62:0/3087503593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0180d0550 0x7fb018153960 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 >> 172.21.15.62:0/3087503593 conn(0x7fb0180668f0 msgr2=0x7fb0180b8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:34.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 shutdown_connections 2024-04-25T09:08:34.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.302+0000 7fb01ee00640 1 -- 172.21.15.62:0/3087503593 wait complete. 2024-04-25T09:08:34.310 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:08:34.310 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:08:34.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.548+0000 7f7ec2a00640 1 Processor -- start 2024-04-25T09:08:34.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.548+0000 7f7ec2a00640 1 -- start start 2024-04-25T09:08:34.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.550+0000 7f7ec2a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7ebc15b850 0x7f7ebc06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.550+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc0aebe0 0x7f7ebc0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.550+0000 7f7ec2a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc0703f0 con 0x7f7ebc15bc30 2024-04-25T09:08:34.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.550+0000 7f7ec2a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc071490 con 0x7f7ebc15b850 2024-04-25T09:08:34.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.550+0000 7f7ec2a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0706d0 con 0x7f7ebc0aebe0 2024-04-25T09:08:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.550+0000 7f7ebb400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7ebc15bc30 0x7f7ebc06c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51980/0 (socket says 172.21.15.62:51980) 2024-04-25T09:08:34.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.550+0000 7f7ebb400640 1 -- 172.21.15.62:0/3056622227 learned_addr learned my addr 172.21.15.62:0/3056622227 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebbe00640 1 --2- 172.21.15.62:0/3056622227 >> v2:172.21.15.121:3300/0 conn(0x7f7ebc15b850 0x7f7ebc06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ec0a00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc0aebe0 0x7f7ebc0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24961526 0 0) 0x7f7ebc0703f0 con 0x7f7ebc15bc30 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea0003520 con 0x7f7ebc15bc30 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 476480919 0 0) 0x7f7ea0003520 con 0x7f7ebc15bc30 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc0aebe0 msgr2=0x7f7ebc0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc0aebe0 0x7f7ebc0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 >> v2:172.21.15.121:3300/0 conn(0x7f7ebc15b850 msgr2=0x7f7ebc06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 --2- 172.21.15.62:0/3056622227 >> v2:172.21.15.121:3300/0 conn(0x7f7ebc15b850 0x7f7ebc06c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0709b0 con 0x7f7ebc15bc30 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f7ea8001670 con 0x7f7ebc15bc30 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 >> v1:172.21.15.62:6789/0 conn(0x7f7ebc15bc30 legacy=0x7f7ebc06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc0aebe0 0x7f7ea00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.551+0000 7f7ebaa00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ea0003f30 0x7f7ea0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ebaa00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ea0004d40 0x7f7ea00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc071490 con 0x7f7ea0004d40 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0706d0 con 0x7f7ea0003f30 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea000b9c0 con 0x7f7ebc0aebe0 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ec0a00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ea0004d40 0x7f7ea00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ebbe00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc0aebe0 0x7f7ea00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ec0a00640 1 -- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc0aebe0 msgr2=0x7f7ea00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ec0a00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc0aebe0 0x7f7ea00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ec0a00640 1 -- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ea0003f30 msgr2=0x7f7ea0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ebb400640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ea0003f30 0x7f7ea0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ec0a00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ea0003f30 0x7f7ea0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ec0a00640 1 -- 172.21.15.62:0/3056622227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0003520 con 0x7f7ea0004d40 2024-04-25T09:08:34.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.552+0000 7f7ebb400640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ea0003f30 0x7f7ea0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:34.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.553+0000 7f7ec0a00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ea0004d40 0x7f7ea00051c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009f90 tx=0x7f7ea0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feeef1c2f6cc8300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:34.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4009c10 con 0x7f7ea0004d40 2024-04-25T09:08:34.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb4036070 con 0x7f7ea0004d40 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ebaa00640 1 -- 172.21.15.62:0/3056622227 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4031630 con 0x7f7ea0004d40 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ec2a00640 1 -- 172.21.15.62:0/3056622227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ea0004d40 msgr2=0x7f7ea00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ec2a00640 1 --2- 172.21.15.62:0/3056622227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ea0004d40 0x7f7ea00051c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009f90 tx=0x7f7ea0005830 comp rx=0 tx=0).stop 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ec0a00640 1 -- 172.21.15.62:0/3056622227 reap_dead start 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ec2a00640 1 -- 172.21.15.62:0/3056622227 shutdown_connections 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ec2a00640 1 -- 172.21.15.62:0/3056622227 >> 172.21.15.62:0/3056622227 conn(0x7f7ebc0668f0 msgr2=0x7f7ebc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ec2a00640 1 -- 172.21.15.62:0/3056622227 shutdown_connections 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.554+0000 7f7ec2a00640 1 -- 172.21.15.62:0/3056622227 wait complete. 2024-04-25T09:08:34.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.556+0000 7f7ec2a00640 1 Processor -- start 2024-04-25T09:08:34.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.556+0000 7f7ec2a00640 1 -- start start 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc0adb50 0x7f7ebc1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ebc0adf30 0x7f7ebc0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc1596d0 0x7f7ebc157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ec2a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc06ced0 con 0x7f7ebc0adb50 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ec2a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc070f70 con 0x7f7ebc0adf30 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ec2a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc06d050 con 0x7f7ebc1596d0 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ebbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc0adb50 0x7f7ebc1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ebbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc0adb50 0x7f7ebc1f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35262/0 (socket says 172.21.15.62:35262) 2024-04-25T09:08:34.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.557+0000 7f7ebbe00640 1 -- 172.21.15.62:0/1047528921 learned_addr learned my addr 172.21.15.62:0/1047528921 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:34.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7ebbe00640 1 -- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc1596d0 msgr2=0x7f7ebc157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7ebbe00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc1596d0 0x7f7ebc157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7ebbe00640 1 -- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ebc0adf30 msgr2=0x7f7ebc0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:34.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7ebbe00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ebc0adf30 0x7f7ebc0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:34.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7ebbe00640 1 -- 172.21.15.62:0/1047528921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc1f4790 con 0x7f7ebc0adb50 2024-04-25T09:08:34.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7ebbe00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc0adb50 0x7f7ebc1f41b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7eac011700 tx=0x7f7eac011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:34.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7eb3e00640 1 -- 172.21.15.62:0/1047528921 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00f8e0 con 0x7f7ebc0adb50 2024-04-25T09:08:34.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7eb3e00640 1 -- 172.21.15.62:0/1047528921 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac00fa80 con 0x7f7ebc0adb50 2024-04-25T09:08:34.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7eb3e00640 1 -- 172.21.15.62:0/1047528921 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00fe30 con 0x7f7ebc0adb50 2024-04-25T09:08:34.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.558+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc1f4a70 con 0x7f7ebc0adb50 2024-04-25T09:08:34.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.559+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc1f5030 con 0x7f7ebc0adb50 2024-04-25T09:08:34.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.560+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc067c40 con 0x7f7ebc0adb50 2024-04-25T09:08:34.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.561+0000 7f7eb3e00640 1 -- 172.21.15.62:0/1047528921 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7eac015460 con 0x7f7ebc0adb50 2024-04-25T09:08:34.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.562+0000 7f7eb3e00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e9c06b4d0 0x7f7e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:34.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.562+0000 7f7eb3e00640 1 -- 172.21.15.62:0/1047528921 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7eac08a820 con 0x7f7ebc0adb50 2024-04-25T09:08:34.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.562+0000 7f7ebb400640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e9c06b4d0 0x7f7e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:34.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.563+0000 7f7ebb400640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e9c06b4d0 0x7f7e9c06d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8001eb0 tx=0x7f7ea8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:34.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.566+0000 7f7eb3e00640 1 -- 172.21.15.62:0/1047528921 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7eac019030 con 0x7f7ebc0adb50 2024-04-25T09:08:34.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:34.763+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7ebc06d790 con 0x7f7ebc0adb50 2024-04-25T09:08:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.468+0000 7f7eb3e00640 1 -- 172.21.15.62:0/1047528921 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7eac059300 con 0x7f7ebc0adb50 2024-04-25T09:08:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:08:35.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.471+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e9c06b4d0 msgr2=0x7f7e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:35.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.471+0000 7f7ec2a00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e9c06b4d0 0x7f7e9c06d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8001eb0 tx=0x7f7ea8009040 comp rx=0 tx=0).stop 2024-04-25T09:08:35.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.471+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc0adb50 msgr2=0x7f7ebc1f41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:35.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.471+0000 7f7ec2a00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc0adb50 0x7f7ebc1f41b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7eac011700 tx=0x7f7eac011bc0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.472+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 shutdown_connections 2024-04-25T09:08:35.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.472+0000 7f7ec2a00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e9c06b4d0 0x7f7e9c06d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.472+0000 7f7ec2a00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ebc1596d0 0x7f7ebc157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.472+0000 7f7ec2a00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ebc0adf30 0x7f7ebc0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.472+0000 7f7ec2a00640 1 --2- 172.21.15.62:0/1047528921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ebc0adb50 0x7f7ebc1f41b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.472+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 >> 172.21.15.62:0/1047528921 conn(0x7f7ebc0668f0 msgr2=0x7f7ebc0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:35.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.472+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 shutdown_connections 2024-04-25T09:08:35.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.472+0000 7f7ec2a00640 1 -- 172.21.15.62:0/1047528921 wait complete. 2024-04-25T09:08:35.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:08:35.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.718+0000 7f9297e00640 1 Processor -- start 2024-04-25T09:08:35.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.719+0000 7f9297e00640 1 -- start start 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.720+0000 7f9297e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 0x7f92980cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.720+0000 7f9297e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92980d0030 0x7f92980afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.720+0000 7f9297e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92980aff40 0x7f92980b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.720+0000 7f9297e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92980d0470 con 0x7f92980d0030 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.720+0000 7f9297e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92980ae960 con 0x7f92980aff40 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.720+0000 7f9297e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92980aec40 con 0x7f929815ef30 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92980d0030 0x7f92980afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f92980d0030 0x7f92980afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35068/0 (socket says 172.21.15.62:35068) 2024-04-25T09:08:35.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296000640 1 -- 172.21.15.62:0/2245958159 learned_addr learned my addr 172.21.15.62:0/2245958159 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:35.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296a00640 1 --2- 172.21.15.62:0/2245958159 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 0x7f92980cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:35.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9297400640 1 --2- 172.21.15.62:0/2245958159 >> v2:172.21.15.121:3300/0 conn(0x7f92980aff40 0x7f92980b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:35.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296000640 1 -- 172.21.15.62:0/2245958159 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 msgr2=0x7f92980cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:35.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296000640 1 --2- 172.21.15.62:0/2245958159 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 0x7f92980cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296000640 1 -- 172.21.15.62:0/2245958159 >> v2:172.21.15.121:3300/0 conn(0x7f92980aff40 msgr2=0x7f92980b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:35.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296000640 1 --2- 172.21.15.62:0/2245958159 >> v2:172.21.15.121:3300/0 conn(0x7f92980aff40 0x7f92980b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.721+0000 7f9296000640 1 -- 172.21.15.62:0/2245958159 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92980aef20 con 0x7f92980d0030 2024-04-25T09:08:35.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.722+0000 7f9296000640 1 --2- 172.21.15.62:0/2245958159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980d0030 0x7f92980afa00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f92980d0890 tx=0x7f928002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c219397f067bac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:35.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.723+0000 7f9295600640 1 -- 172.21.15.62:0/2245958159 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280037070 con 0x7f92980d0030 2024-04-25T09:08:35.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.723+0000 7f9295600640 1 -- 172.21.15.62:0/2245958159 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9280032440 con 0x7f92980d0030 2024-04-25T09:08:35.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.723+0000 7f9295600640 1 -- 172.21.15.62:0/2245958159 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280032810 con 0x7f92980d0030 2024-04-25T09:08:35.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 -- 172.21.15.62:0/2245958159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980d0030 msgr2=0x7f92980afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:35.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 --2- 172.21.15.62:0/2245958159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980d0030 0x7f92980afa00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f92980d0890 tx=0x7f928002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 -- 172.21.15.62:0/2245958159 shutdown_connections 2024-04-25T09:08:35.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 --2- 172.21.15.62:0/2245958159 >> v2:172.21.15.121:3300/0 conn(0x7f92980aff40 0x7f92980b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 --2- 172.21.15.62:0/2245958159 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980d0030 0x7f92980afa00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 --2- 172.21.15.62:0/2245958159 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 0x7f92980cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 -- 172.21.15.62:0/2245958159 >> 172.21.15.62:0/2245958159 conn(0x7f929809aa70 msgr2=0x7f92980d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:35.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 -- 172.21.15.62:0/2245958159 shutdown_connections 2024-04-25T09:08:35.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.724+0000 7f9297e00640 1 -- 172.21.15.62:0/2245958159 wait complete. 2024-04-25T09:08:35.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.725+0000 7f9297e00640 1 Processor -- start 2024-04-25T09:08:35.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.726+0000 7f9297e00640 1 -- start start 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9297e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980aff40 0x7f929814cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9297e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92980d0030 0x7f92980c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9297e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 0x7f92980caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9297e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92980ae5a0 con 0x7f92980aff40 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9297e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92980af4e0 con 0x7f92980d0030 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9297e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92980ae720 con 0x7f929815ef30 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9296a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980aff40 0x7f929814cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9296a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980aff40 0x7f929814cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35074/0 (socket says 172.21.15.62:35074) 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9296a00640 1 -- 172.21.15.62:0/3842458475 learned_addr learned my addr 172.21.15.62:0/3842458475 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9297400640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 0x7f92980caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.727+0000 7f9296000640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92980d0030 0x7f92980c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9296a00640 1 -- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 msgr2=0x7f92980caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9296a00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 0x7f92980caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9296a00640 1 -- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92980d0030 msgr2=0x7f92980c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9296a00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92980d0030 0x7f92980c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9296a00640 1 -- 172.21.15.62:0/3842458475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92980b0470 con 0x7f92980aff40 2024-04-25T09:08:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9296000640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92980d0030 0x7f92980c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:35.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9296a00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980aff40 0x7f929814cbf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f928800c820 tx=0x7f928800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:35.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f928ea00640 1 -- 172.21.15.62:0/3842458475 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9288015070 con 0x7f92980aff40 2024-04-25T09:08:35.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f928ea00640 1 -- 172.21.15.62:0/3842458475 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9288004020 con 0x7f92980aff40 2024-04-25T09:08:35.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92980b0710 con 0x7f92980aff40 2024-04-25T09:08:35.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f928ea00640 1 -- 172.21.15.62:0/3842458475 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92880043f0 con 0x7f92980aff40 2024-04-25T09:08:35.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.728+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92980b0fb0 con 0x7f92980aff40 2024-04-25T09:08:35.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.729+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92980b08e0 con 0x7f92980aff40 2024-04-25T09:08:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.732+0000 7f928ea00640 1 -- 172.21.15.62:0/3842458475 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9288007830 con 0x7f92980aff40 2024-04-25T09:08:35.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.733+0000 7f928ea00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f927406b4d0 0x7f927406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:35.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.733+0000 7f928ea00640 1 -- 172.21.15.62:0/3842458475 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f928808a170 con 0x7f92980aff40 2024-04-25T09:08:35.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.733+0000 7f9296000640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f927406b4d0 0x7f927406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:35.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.735+0000 7f9296000640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f927406b4d0 0x7f927406d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9280034000 tx=0x7f928002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.736+0000 7f928ea00640 1 -- 172.21.15.62:0/3842458475 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9288059340 con 0x7f92980aff40 2024-04-25T09:08:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:35.939+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9298075070 con 0x7f92980aff40 2024-04-25T09:08:36.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.474+0000 7f928ea00640 1 -- 172.21.15.62:0/3842458475 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9288058c40 con 0x7f92980aff40 2024-04-25T09:08:36.462 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:08:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.477+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f927406b4d0 msgr2=0x7f927406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.477+0000 7f9297e00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f927406b4d0 0x7f927406d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9280034000 tx=0x7f928002e000 comp rx=0 tx=0).stop 2024-04-25T09:08:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.477+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980aff40 msgr2=0x7f929814cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.477+0000 7f9297e00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980aff40 0x7f929814cbf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f928800c820 tx=0x7f928800cce0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.478+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 shutdown_connections 2024-04-25T09:08:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.478+0000 7f9297e00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f927406b4d0 0x7f927406d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.478+0000 7f9297e00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f929815ef30 0x7f92980caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.478+0000 7f9297e00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92980d0030 0x7f92980c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.478+0000 7f9297e00640 1 --2- 172.21.15.62:0/3842458475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f92980aff40 0x7f929814cbf0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.478+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 >> 172.21.15.62:0/3842458475 conn(0x7f929809aa70 msgr2=0x7f929806fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.478+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 shutdown_connections 2024-04-25T09:08:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.478+0000 7f9297e00640 1 -- 172.21.15.62:0/3842458475 wait complete. 2024-04-25T09:08:36.498 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:08:36.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.693+0000 7f948e200640 1 Processor -- start 2024-04-25T09:08:36.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.693+0000 7f948e200640 1 -- start start 2024-04-25T09:08:36.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.695+0000 7f948e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f948815bdf0 0x7f94880720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:36.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.695+0000 7f948e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 0x7f94880703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:36.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.695+0000 7f948e200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948806d070 con 0x7f94880cebc0 2024-04-25T09:08:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.695+0000 7f948e200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948806ce40 con 0x7f948815bdf0 2024-04-25T09:08:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.695+0000 7f948e200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948815c1d0 con 0x7f948806c9b0 2024-04-25T09:08:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.695+0000 7f9487400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f948815bdf0 0x7f94880720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.695+0000 7f9487400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f948815bdf0 0x7f94880720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35082/0 (socket says 172.21.15.62:35082) 2024-04-25T09:08:36.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.695+0000 7f9487400640 1 -- 172.21.15.62:0/1760258626 learned_addr learned my addr 172.21.15.62:0/1760258626 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.696+0000 7f9487e00640 1 --2- 172.21.15.62:0/1760258626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 0x7f94880703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.696+0000 7f9487400640 1 -- 172.21.15.62:0/1760258626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 msgr2=0x7f94880703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.696+0000 7f9487400640 1 --2- 172.21.15.62:0/1760258626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 0x7f94880703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.696+0000 7f9487400640 1 -- 172.21.15.62:0/1760258626 >> v1:172.21.15.62:6789/0 conn(0x7f94880cebc0 legacy=0x7f948806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:36.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.696+0000 7f9487400640 1 -- 172.21.15.62:0/1760258626 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948806d350 con 0x7f948815bdf0 2024-04-25T09:08:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.697+0000 7f9487400640 1 --2- 172.21.15.62:0/1760258626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948815bdf0 0x7f94880720c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9478009870 tx=0x7f947802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10b6c9a65e3eb7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.697+0000 7f9486000640 1 -- 172.21.15.62:0/1760258626 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478037070 con 0x7f948815bdf0 2024-04-25T09:08:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.697+0000 7f9486000640 1 -- 172.21.15.62:0/1760258626 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9478032440 con 0x7f948815bdf0 2024-04-25T09:08:36.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.697+0000 7f9486000640 1 -- 172.21.15.62:0/1760258626 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478032810 con 0x7f948815bdf0 2024-04-25T09:08:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.698+0000 7f948e200640 1 -- 172.21.15.62:0/1760258626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948815bdf0 msgr2=0x7f94880720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.698+0000 7f948e200640 1 --2- 172.21.15.62:0/1760258626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948815bdf0 0x7f94880720c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9478009870 tx=0x7f947802a250 comp rx=0 tx=0).stop 2024-04-25T09:08:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.698+0000 7f948e200640 1 -- 172.21.15.62:0/1760258626 shutdown_connections 2024-04-25T09:08:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.698+0000 7f948e200640 1 --2- 172.21.15.62:0/1760258626 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 0x7f94880703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.698+0000 7f948e200640 1 --2- 172.21.15.62:0/1760258626 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f948815bdf0 0x7f94880720c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.698+0000 7f948e200640 1 -- 172.21.15.62:0/1760258626 >> 172.21.15.62:0/1760258626 conn(0x7f94880b1410 msgr2=0x7f94880b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.698+0000 7f948e200640 1 -- 172.21.15.62:0/1760258626 shutdown_connections 2024-04-25T09:08:36.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.698+0000 7f948e200640 1 -- 172.21.15.62:0/1760258626 wait complete. 2024-04-25T09:08:36.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.700+0000 7f948e200640 1 Processor -- start 2024-04-25T09:08:36.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.700+0000 7f948e200640 1 -- start start 2024-04-25T09:08:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.701+0000 7f948e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 0x7f948814b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.701+0000 7f948e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94880cebc0 0x7f94881f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.701+0000 7f948e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f948815bdf0 0x7f94881f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.701+0000 7f948e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94880aedf0 con 0x7f94880cebc0 2024-04-25T09:08:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.701+0000 7f948e200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94880ae8c0 con 0x7f948815bdf0 2024-04-25T09:08:36.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.701+0000 7f948e200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94880708e0 con 0x7f948806c9b0 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.702+0000 7f9486a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94880cebc0 0x7f94881f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.702+0000 7f9487400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 0x7f948814b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.702+0000 7f9486a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94880cebc0 0x7f94881f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35090/0 (socket says 172.21.15.62:35090) 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.702+0000 7f9486a00640 1 -- 172.21.15.62:0/4200648185 learned_addr learned my addr 172.21.15.62:0/4200648185 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.702+0000 7f9487e00640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f948815bdf0 0x7f94881f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f9486a00640 1 -- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 msgr2=0x7f948814b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f9486a00640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 0x7f948814b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f9486a00640 1 -- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f948815bdf0 msgr2=0x7f94881f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f9486a00640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f948815bdf0 0x7f94881f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:36.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f9486a00640 1 -- 172.21.15.62:0/4200648185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94881e5e00 con 0x7f94880cebc0 2024-04-25T09:08:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f9486a00640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94880cebc0 0x7f94881f38f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f947400c450 tx=0x7f947400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f947f400640 1 -- 172.21.15.62:0/4200648185 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740052d0 con 0x7f94880cebc0 2024-04-25T09:08:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f947f400640 1 -- 172.21.15.62:0/4200648185 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9474005470 con 0x7f94880cebc0 2024-04-25T09:08:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94881e60e0 con 0x7f94880cebc0 2024-04-25T09:08:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.703+0000 7f947f400640 1 -- 172.21.15.62:0/4200648185 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474012870 con 0x7f94880cebc0 2024-04-25T09:08:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.704+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94881e66a0 con 0x7f94880cebc0 2024-04-25T09:08:36.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.705+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948804ef20 con 0x7f94880cebc0 2024-04-25T09:08:36.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.706+0000 7f947f400640 1 -- 172.21.15.62:0/4200648185 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9474012a10 con 0x7f94880cebc0 2024-04-25T09:08:36.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.707+0000 7f947f400640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f946806b4d0 0x7f946806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:36.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.707+0000 7f947f400640 1 -- 172.21.15.62:0/4200648185 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9474089d00 con 0x7f94880cebc0 2024-04-25T09:08:36.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.708+0000 7f9487400640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f946806b4d0 0x7f946806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:36.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.709+0000 7f9487400640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f946806b4d0 0x7f946806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9478034000 tx=0x7f947802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:36.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.712+0000 7f947f400640 1 -- 172.21.15.62:0/4200648185 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94740584f0 con 0x7f94880cebc0 2024-04-25T09:08:36.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:36.957+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f948814b720 con 0x7f94880cebc0 2024-04-25T09:08:37.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.494+0000 7f947f400640 1 -- 172.21.15.62:0/4200648185 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v34) v1 ==== 172+0+0 (secure 0 0 0) 0x7f947405c590 con 0x7f94880cebc0 2024-04-25T09:08:37.482 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T09:08:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.497+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f946806b4d0 msgr2=0x7f946806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.497+0000 7f948e200640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f946806b4d0 0x7f946806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9478034000 tx=0x7f947802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.497+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94880cebc0 msgr2=0x7f94881f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.497+0000 7f948e200640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94880cebc0 0x7f94881f38f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f947400c450 tx=0x7f947400c910 comp rx=0 tx=0).stop 2024-04-25T09:08:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.498+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 shutdown_connections 2024-04-25T09:08:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.498+0000 7f948e200640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f946806b4d0 0x7f946806d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.498+0000 7f948e200640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94880cebc0 0x7f94881f38f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.498+0000 7f948e200640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f948815bdf0 0x7f94881f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.498+0000 7f948e200640 1 --2- 172.21.15.62:0/4200648185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f948806c9b0 0x7f948814b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.498+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 >> 172.21.15.62:0/4200648185 conn(0x7f94880b1410 msgr2=0x7f948815a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.498+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 shutdown_connections 2024-04-25T09:08:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.498+0000 7f948e200640 1 -- 172.21.15.62:0/4200648185 wait complete. 2024-04-25T09:08:37.523 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:37.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.713+0000 7f905d000640 1 Processor -- start 2024-04-25T09:08:37.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.713+0000 7f905d000640 1 -- start start 2024-04-25T09:08:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.714+0000 7f905d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9058072020 0x7f90580ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.714+0000 7f905d000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f90580ce300 0x7f90580ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.714+0000 7f905d000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90581666d0 con 0x7f9058165e80 2024-04-25T09:08:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.714+0000 7f905d000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90580ceca0 con 0x7f90580ce300 2024-04-25T09:08:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.714+0000 7f905d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90581669b0 con 0x7f9058072020 2024-04-25T09:08:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.714+0000 7f904d400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9058165e80 0x7f9058070fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44260/0 (socket says 172.21.15.62:44260) 2024-04-25T09:08:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.714+0000 7f904d400640 1 -- 172.21.15.62:0/1523514806 learned_addr learned my addr 172.21.15.62:0/1523514806 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9056800640 1 --2- 172.21.15.62:0/1523514806 >> v2:172.21.15.62:3300/0 conn(0x7f90580ce300 0x7f90580ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:37.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9055e00640 1 --2- 172.21.15.62:0/1523514806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9058072020 0x7f90580ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:37.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9055400640 1 -- 172.21.15.62:0/1523514806 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537009935 0 0) 0x7f90581666d0 con 0x7f9058165e80 2024-04-25T09:08:37.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9055400640 1 -- 172.21.15.62:0/1523514806 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f903c003520 con 0x7f9058165e80 2024-04-25T09:08:37.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9056800640 1 -- 172.21.15.62:0/1523514806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9058072020 msgr2=0x7f90580ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:37.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9056800640 1 --2- 172.21.15.62:0/1523514806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9058072020 0x7f90580ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9056800640 1 -- 172.21.15.62:0/1523514806 >> v1:172.21.15.62:6789/0 conn(0x7f9058165e80 legacy=0x7f9058070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:37.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9056800640 1 -- 172.21.15.62:0/1523514806 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058166c90 con 0x7f90580ce300 2024-04-25T09:08:37.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.715+0000 7f9055e00640 1 --2- 172.21.15.62:0/1523514806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9058072020 0x7f90580ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:37.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.717+0000 7f9056800640 1 --2- 172.21.15.62:0/1523514806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90580ce300 0x7f90580ce760 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9050009d90 tx=0x7f905002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961c305550e41ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:37.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.717+0000 7f9055400640 1 -- 172.21.15.62:0/1523514806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050007740 con 0x7f90580ce300 2024-04-25T09:08:37.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.717+0000 7f9055400640 1 -- 172.21.15.62:0/1523514806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90500078e0 con 0x7f90580ce300 2024-04-25T09:08:37.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.717+0000 7f9055400640 1 -- 172.21.15.62:0/1523514806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050007cb0 con 0x7f90580ce300 2024-04-25T09:08:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.717+0000 7f905d000640 1 -- 172.21.15.62:0/1523514806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90580ce300 msgr2=0x7f90580ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.717+0000 7f905d000640 1 --2- 172.21.15.62:0/1523514806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90580ce300 0x7f90580ce760 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9050009d90 tx=0x7f905002a660 comp rx=0 tx=0).stop 2024-04-25T09:08:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.718+0000 7f905d000640 1 -- 172.21.15.62:0/1523514806 shutdown_connections 2024-04-25T09:08:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.718+0000 7f905d000640 1 --2- 172.21.15.62:0/1523514806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90580ce300 0x7f90580ce760 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.718+0000 7f905d000640 1 --2- 172.21.15.62:0/1523514806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9058072020 0x7f90580ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.718+0000 7f905d000640 1 -- 172.21.15.62:0/1523514806 >> 172.21.15.62:0/1523514806 conn(0x7f90580668f0 msgr2=0x7f9058066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.718+0000 7f905d000640 1 -- 172.21.15.62:0/1523514806 shutdown_connections 2024-04-25T09:08:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.718+0000 7f905d000640 1 -- 172.21.15.62:0/1523514806 wait complete. 2024-04-25T09:08:37.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.719+0000 7f905d000640 1 Processor -- start 2024-04-25T09:08:37.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.720+0000 7f905d000640 1 -- start start 2024-04-25T09:08:37.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f905d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9058072020 0x7f90580b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f905d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90580ce300 0x7f90581ec690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f905d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9058165e80 0x7f90581ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f905d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90581662e0 con 0x7f9058165e80 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f905d000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905806d2e0 con 0x7f9058072020 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f905d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9058166460 con 0x7f90580ce300 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f9056800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9058165e80 0x7f90581ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f9056800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9058165e80 0x7f90581ecbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35108/0 (socket says 172.21.15.62:35108) 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f9056800640 1 -- 172.21.15.62:0/1170110160 learned_addr learned my addr 172.21.15.62:0/1170110160 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f904d400640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90580ce300 0x7f90581ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.721+0000 7f9055e00640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9058072020 0x7f90580b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f9056800640 1 -- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90580ce300 msgr2=0x7f90581ec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f9056800640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90580ce300 0x7f90581ec690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f9056800640 1 -- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9058072020 msgr2=0x7f90580b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:37.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f9056800640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9058072020 0x7f90580b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f9056800640 1 -- 172.21.15.62:0/1170110160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058164750 con 0x7f9058165e80 2024-04-25T09:08:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f9056800640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9058165e80 0x7f90581ecbd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f905002ac50 tx=0x7f9050004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f904ea00640 1 -- 172.21.15.62:0/1170110160 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050040070 con 0x7f9058165e80 2024-04-25T09:08:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f904ea00640 1 -- 172.21.15.62:0/1170110160 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050031bc0 con 0x7f9058165e80 2024-04-25T09:08:37.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f904ea00640 1 -- 172.21.15.62:0/1170110160 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050009450 con 0x7f9058165e80 2024-04-25T09:08:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.722+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90580a9870 con 0x7f9058165e80 2024-04-25T09:08:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.723+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90581ed7c0 con 0x7f9058165e80 2024-04-25T09:08:37.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.724+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90580a9a40 con 0x7f9058165e80 2024-04-25T09:08:37.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.725+0000 7f904ea00640 1 -- 172.21.15.62:0/1170110160 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9050039040 con 0x7f9058165e80 2024-04-25T09:08:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.726+0000 7f904ea00640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f903406b4d0 0x7f903406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.726+0000 7f904ea00640 1 -- 172.21.15.62:0/1170110160 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90500abad0 con 0x7f9058165e80 2024-04-25T09:08:37.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.726+0000 7f9055e00640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f903406b4d0 0x7f903406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:37.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.728+0000 7f9055e00640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f903406b4d0 0x7f903406d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9048006fd0 tx=0x7f9048008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:37.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.730+0000 7f904ea00640 1 -- 172.21.15.62:0/1170110160 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f905007a9d0 con 0x7f9058165e80 2024-04-25T09:08:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.925+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9058072c70 con 0x7f9058165e80 2024-04-25T09:08:37.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.926+0000 7f904ea00640 1 -- 172.21.15.62:0/1170110160 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f905007a2d0 con 0x7f9058165e80 2024-04-25T09:08:37.914 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:37.914 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":61,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:36.958072+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:34.817879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:35.997553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.928+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f903406b4d0 msgr2=0x7f903406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.928+0000 7f905d000640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f903406b4d0 0x7f903406d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9048006fd0 tx=0x7f9048008040 comp rx=0 tx=0).stop 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.928+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9058165e80 msgr2=0x7f90581ecbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.928+0000 7f905d000640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9058165e80 0x7f90581ecbd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f905002ac50 tx=0x7f9050004a40 comp rx=0 tx=0).stop 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.929+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 shutdown_connections 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.929+0000 7f905d000640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f903406b4d0 0x7f903406d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.929+0000 7f905d000640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9058165e80 0x7f90581ecbd0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.929+0000 7f905d000640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90580ce300 0x7f90581ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.929+0000 7f905d000640 1 --2- 172.21.15.62:0/1170110160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9058072020 0x7f90580b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.929+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 >> 172.21.15.62:0/1170110160 conn(0x7f90580668f0 msgr2=0x7f905806d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:37.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.929+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 shutdown_connections 2024-04-25T09:08:37.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:37.929+0000 7f905d000640 1 -- 172.21.15.62:0/1170110160 wait complete. 2024-04-25T09:08:37.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:38.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.146+0000 7f1fe5000640 1 Processor -- start 2024-04-25T09:08:38.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.146+0000 7f1fe5000640 1 -- start start 2024-04-25T09:08:38.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.148+0000 7f1fe5000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe015bd10 0x7f1fe00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.148+0000 7f1fe5000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe006ca70 con 0x7f1fe00cccf0 2024-04-25T09:08:38.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.148+0000 7f1fe5000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe015c0f0 con 0x7f1fe00aeed0 2024-04-25T09:08:38.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.148+0000 7f1fe5000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe006d660 con 0x7f1fe015bd10 2024-04-25T09:08:38.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.148+0000 7f1fdd400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1fe00cccf0 0x7f1fe00ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44272/0 (socket says 172.21.15.62:44272) 2024-04-25T09:08:38.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.148+0000 7f1fdd400640 1 -- 172.21.15.62:0/3788394046 learned_addr learned my addr 172.21.15.62:0/3788394046 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:38.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.148+0000 7f1fdde00640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe015bd10 0x7f1fe00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915777836 0 0) 0x7f1fe006ca70 con 0x7f1fe00cccf0 2024-04-25T09:08:38.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc4003520 con 0x7f1fe00cccf0 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 217767982 0 0) 0x7f1fc4003520 con 0x7f1fe00cccf0 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe015bd10 msgr2=0x7f1fe00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe015bd10 0x7f1fe00cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 >> v1:172.21.15.121:6789/0 conn(0x7f1fe00aeed0 legacy=0x7f1fe00cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe006cd30 con 0x7f1fe00cccf0 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1fc8001670 con 0x7f1fe00cccf0 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 >> v1:172.21.15.62:6789/0 conn(0x7f1fe00cccf0 legacy=0x7f1fe00ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40039e0 0x7f1fc4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc40043d0 0x7f1fc40048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc40050a0 0x7f1fc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe006d660 con 0x7f1fc40050a0 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc400bfb0 con 0x7f1fc40043d0 2024-04-25T09:08:38.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc400c1b0 con 0x7f1fc40039e0 2024-04-25T09:08:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.149+0000 7f1fde800640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc40050a0 0x7f1fc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.150+0000 7f1fdde00640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40039e0 0x7f1fc4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.150+0000 7f1fdd400640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc40043d0 0x7f1fc40048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.150+0000 7f1fde800640 1 -- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40039e0 msgr2=0x7f1fc4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.150+0000 7f1fde800640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40039e0 0x7f1fc4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.150+0000 7f1fde800640 1 -- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc40043d0 msgr2=0x7f1fc40048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.150+0000 7f1fde800640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc40043d0 0x7f1fc40048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.150+0000 7f1fde800640 1 -- 172.21.15.62:0/3788394046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4003520 con 0x7f1fc40050a0 2024-04-25T09:08:38.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.151+0000 7f1fde800640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc40050a0 0x7f1fc400b9c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8001720 tx=0x7f1fc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35f4388f91cdf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:38.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.151+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8001370 con 0x7f1fc40050a0 2024-04-25T09:08:38.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.151+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd80052d0 con 0x7f1fc40050a0 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.151+0000 7f1fdca00640 1 -- 172.21.15.62:0/3788394046 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd802abc0 con 0x7f1fc40050a0 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.152+0000 7f1fe5000640 1 -- 172.21.15.62:0/3788394046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc40050a0 msgr2=0x7f1fc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.152+0000 7f1fe5000640 1 --2- 172.21.15.62:0/3788394046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc40050a0 0x7f1fc400b9c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8001720 tx=0x7f1fc4005830 comp rx=0 tx=0).stop 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.152+0000 7f1fde800640 1 -- 172.21.15.62:0/3788394046 reap_dead start 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.152+0000 7f1fe5000640 1 -- 172.21.15.62:0/3788394046 shutdown_connections 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.152+0000 7f1fe5000640 1 -- 172.21.15.62:0/3788394046 >> 172.21.15.62:0/3788394046 conn(0x7f1fe00668f0 msgr2=0x7f1fe0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.152+0000 7f1fe5000640 1 -- 172.21.15.62:0/3788394046 shutdown_connections 2024-04-25T09:08:38.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.152+0000 7f1fe5000640 1 -- 172.21.15.62:0/3788394046 wait complete. 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.154+0000 7f1fe5000640 1 Processor -- start 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.154+0000 7f1fe5000640 1 -- start start 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fe5000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40050a0 0x7f1fe01f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fe5000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0071dc0 0x7f1fe0152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fe5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe0151e00 0x7f1fe00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fe5000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe006daf0 con 0x7f1fe0151e00 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fe5000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe006d2f0 con 0x7f1fe0071dc0 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fe5000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe0070350 con 0x7f1fc40050a0 2024-04-25T09:08:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fde800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe0151e00 0x7f1fe00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fde800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe0151e00 0x7f1fe00ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35130/0 (socket says 172.21.15.62:35130) 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fde800640 1 -- 172.21.15.62:0/3093102904 learned_addr learned my addr 172.21.15.62:0/3093102904 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fdde00640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40050a0 0x7f1fe01f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.155+0000 7f1fdd400640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0071dc0 0x7f1fe0152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fde800640 1 -- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40050a0 msgr2=0x7f1fe01f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fde800640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40050a0 0x7f1fe01f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fde800640 1 -- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0071dc0 msgr2=0x7f1fe0152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fde800640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0071dc0 0x7f1fe0152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fde800640 1 -- 172.21.15.62:0/3093102904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe01f42b0 con 0x7f1fe0151e00 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fdd400640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0071dc0 0x7f1fe0152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fde800640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe0151e00 0x7f1fe00ad8e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1fd802a570 tx=0x7f1fd8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fd6000640 1 -- 172.21.15.62:0/3093102904 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8037040 con 0x7f1fe0151e00 2024-04-25T09:08:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fd6000640 1 -- 172.21.15.62:0/3093102904 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8040070 con 0x7f1fe0151e00 2024-04-25T09:08:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fd6000640 1 -- 172.21.15.62:0/3093102904 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8001370 con 0x7f1fe0151e00 2024-04-25T09:08:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe01f4590 con 0x7f1fe0151e00 2024-04-25T09:08:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.156+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe0153460 con 0x7f1fe0151e00 2024-04-25T09:08:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.157+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe0072c70 con 0x7f1fe0151e00 2024-04-25T09:08:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.158+0000 7f1fd6000640 1 -- 172.21.15.62:0/3093102904 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fd80052d0 con 0x7f1fe0151e00 2024-04-25T09:08:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.159+0000 7f1fd6000640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.159+0000 7f1fd6000640 1 -- 172.21.15.62:0/3093102904 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fd80a9ab0 con 0x7f1fe0151e00 2024-04-25T09:08:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.160+0000 7f1fdde00640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.161+0000 7f1fdde00640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009fd0 tx=0x7f1fd000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:38.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.163+0000 7f1fd6000640 1 -- 172.21.15.62:0/3093102904 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1fd80788a0 con 0x7f1fe0151e00 2024-04-25T09:08:38.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.363+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fe004f000 con 0x7f1fe0151e00 2024-04-25T09:08:38.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.366+0000 7f1fd6000640 1 -- 172.21.15.62:0/3093102904 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f1fd80781a0 con 0x7f1fe0151e00 2024-04-25T09:08:38.354 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:38.354 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":61,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:36.958072+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:34.817879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:35.997553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:38.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 msgr2=0x7f1fbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009fd0 tx=0x7f1fd000b040 comp rx=0 tx=0).stop 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe0151e00 msgr2=0x7f1fe00ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe0151e00 0x7f1fe00ad8e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1fd802a570 tx=0x7f1fd8001120 comp rx=0 tx=0).stop 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 shutdown_connections 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fe0151e00 0x7f1fe00ad8e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0071dc0 0x7f1fe0152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 --2- 172.21.15.62:0/3093102904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fc40050a0 0x7f1fe01f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 >> 172.21.15.62:0/3093102904 conn(0x7f1fe00668f0 msgr2=0x7f1fe0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 shutdown_connections 2024-04-25T09:08:38.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.369+0000 7f1fe5000640 1 -- 172.21.15.62:0/3093102904 wait complete. 2024-04-25T09:08:38.375 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:08:38.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.595+0000 7fd9c3e00640 1 Processor -- start 2024-04-25T09:08:38.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.596+0000 7fd9c3e00640 1 -- start start 2024-04-25T09:08:38.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.597+0000 7fd9c3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9c406d2c0 0x7fd9c40ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.597+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 0x7fd9c40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.597+0000 7fd9c3e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c40ced40 con 0x7fd9c40b31a0 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.597+0000 7fd9c3e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c40cf000 con 0x7fd9c406d2c0 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.597+0000 7fd9c3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c40cf2e0 con 0x7fd9c406d6a0 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.598+0000 7fd9c2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9c406d2c0 0x7fd9c40ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.598+0000 7fd9c2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9c406d2c0 0x7fd9c40ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35134/0 (socket says 172.21.15.62:35134) 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.598+0000 7fd9c2a00640 1 -- 172.21.15.62:0/4029503281 learned_addr learned my addr 172.21.15.62:0/4029503281 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.598+0000 7fd9bbe00640 1 --2- 172.21.15.62:0/4029503281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 0x7fd9c40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.598+0000 7fd9c2a00640 1 -- 172.21.15.62:0/4029503281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 msgr2=0x7fd9c40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.598+0000 7fd9c2a00640 1 --2- 172.21.15.62:0/4029503281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 0x7fd9c40ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.598+0000 7fd9c2a00640 1 -- 172.21.15.62:0/4029503281 >> v1:172.21.15.121:6789/0 conn(0x7fd9c40b31a0 legacy=0x7fd9c40b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:38.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.598+0000 7fd9c2a00640 1 -- 172.21.15.62:0/4029503281 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c40b26c0 con 0x7fd9c406d2c0 2024-04-25T09:08:38.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.599+0000 7fd9c2a00640 1 --2- 172.21.15.62:0/4029503281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40ccde0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009870 tx=0x7fd9b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe4fe0b6632bac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:38.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.600+0000 7fd9c2000640 1 -- 172.21.15.62:0/4029503281 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4037070 con 0x7fd9c406d2c0 2024-04-25T09:08:38.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.600+0000 7fd9c2000640 1 -- 172.21.15.62:0/4029503281 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b4032440 con 0x7fd9c406d2c0 2024-04-25T09:08:38.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.600+0000 7fd9c2000640 1 -- 172.21.15.62:0/4029503281 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4032810 con 0x7fd9c406d2c0 2024-04-25T09:08:38.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.600+0000 7fd9c3e00640 1 -- 172.21.15.62:0/4029503281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 msgr2=0x7fd9c40ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.600+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/4029503281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40ccde0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009870 tx=0x7fd9b402a210 comp rx=0 tx=0).stop 2024-04-25T09:08:38.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.601+0000 7fd9c3e00640 1 -- 172.21.15.62:0/4029503281 shutdown_connections 2024-04-25T09:08:38.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.601+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/4029503281 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 0x7fd9c40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.601+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/4029503281 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40ccde0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.601+0000 7fd9c3e00640 1 -- 172.21.15.62:0/4029503281 >> 172.21.15.62:0/4029503281 conn(0x7fd9c4066f20 msgr2=0x7fd9c40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.601+0000 7fd9c3e00640 1 -- 172.21.15.62:0/4029503281 shutdown_connections 2024-04-25T09:08:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.601+0000 7fd9c3e00640 1 -- 172.21.15.62:0/4029503281 wait complete. 2024-04-25T09:08:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.602+0000 7fd9c3e00640 1 Processor -- start 2024-04-25T09:08:38.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.602+0000 7fd9c3e00640 1 -- start start 2024-04-25T09:08:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.603+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.603+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 0x7fd9c4070af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.603+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c40b31a0 0x7fd9c415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.603+0000 7fd9c3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c406db20 con 0x7fd9c406d2c0 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.603+0000 7fd9c3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c40b2c80 con 0x7fd9c40b31a0 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.603+0000 7fd9c3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c41448d0 con 0x7fd9c406d6a0 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35142/0 (socket says 172.21.15.62:35142) 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 -- 172.21.15.62:0/3418511961 learned_addr learned my addr 172.21.15.62:0/3418511961 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 -- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 msgr2=0x7fd9c4070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9bbe00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 0x7fd9c4070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 0x7fd9c4070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 -- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c40b31a0 msgr2=0x7fd9c415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c3400640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c40b31a0 0x7fd9c415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c40b31a0 0x7fd9c415b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 -- 172.21.15.62:0/3418511961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4071150 con 0x7fd9c406d2c0 2024-04-25T09:08:38.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.604+0000 7fd9c2a00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40705b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009870 tx=0x7fd9b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.605+0000 7fd9baa00640 1 -- 172.21.15.62:0/3418511961 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4041070 con 0x7fd9c406d2c0 2024-04-25T09:08:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.605+0000 7fd9baa00640 1 -- 172.21.15.62:0/3418511961 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b403c890 con 0x7fd9c406d2c0 2024-04-25T09:08:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.605+0000 7fd9baa00640 1 -- 172.21.15.62:0/3418511961 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4037040 con 0x7fd9c406d2c0 2024-04-25T09:08:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.605+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c4071430 con 0x7fd9c406d2c0 2024-04-25T09:08:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.605+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c40ac530 con 0x7fd9c406d2c0 2024-04-25T09:08:38.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.606+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c406abe0 con 0x7fd9c406d2c0 2024-04-25T09:08:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.607+0000 7fd9baa00640 1 -- 172.21.15.62:0/3418511961 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9b4044070 con 0x7fd9c406d2c0 2024-04-25T09:08:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.608+0000 7fd9baa00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd9a006b4d0 0x7fd9a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:38.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.608+0000 7fd9baa00640 1 -- 172.21.15.62:0/3418511961 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9b40aae80 con 0x7fd9c406d2c0 2024-04-25T09:08:38.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.609+0000 7fd9bbe00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd9a006b4d0 0x7fd9a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.610+0000 7fd9bbe00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd9a006b4d0 0x7fd9a006d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40cd6f0 tx=0x7fd9ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:38.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.613+0000 7fd9baa00640 1 -- 172.21.15.62:0/3418511961 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9b4079cf0 con 0x7fd9c406d2c0 2024-04-25T09:08:38.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:38.840+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd9c404ef20 con 0x7fd9c406d2c0 2024-04-25T09:08:39.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.534+0000 7fd9baa00640 1 -- 172.21.15.62:0/3418511961 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd9b40795f0 con 0x7fd9c406d2c0 2024-04-25T09:08:39.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd9a006b4d0 msgr2=0x7fd9a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd9a006b4d0 0x7fd9a006d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40cd6f0 tx=0x7fd9ac008040 comp rx=0 tx=0).stop 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 msgr2=0x7fd9c40705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40705b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009870 tx=0x7fd9b40046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 shutdown_connections 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd9a006b4d0 0x7fd9a006d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c40b31a0 0x7fd9c415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9c406d6a0 0x7fd9c4070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 --2- 172.21.15.62:0/3418511961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9c406d2c0 0x7fd9c40705b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 >> 172.21.15.62:0/3418511961 conn(0x7fd9c4066f20 msgr2=0x7fd9c415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 shutdown_connections 2024-04-25T09:08:39.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.539+0000 7fd9c3e00640 1 -- 172.21.15.62:0/3418511961 wait complete. 2024-04-25T09:08:39.547 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:08:39.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:08:39.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.761+0000 7f82ec600640 1 Processor -- start 2024-04-25T09:08:39.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.762+0000 7f82ec600640 1 -- start start 2024-04-25T09:08:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.763+0000 7f82ec600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 0x7f82e40b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.763+0000 7f82ec600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82e40b70d0 0x7f82e40b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.763+0000 7f82ec600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e40d2270 con 0x7f82e415dc10 2024-04-25T09:08:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.763+0000 7f82ec600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e40b81a0 con 0x7f82e40b70d0 2024-04-25T09:08:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.763+0000 7f82ec600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40b7590 con 0x7f82e40b97b0 2024-04-25T09:08:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82e9c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f82e415dc10 0x7f82e40d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44284/0 (socket says 172.21.15.62:44284) 2024-04-25T09:08:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82e9c00640 1 -- 172.21.15.62:0/2996691230 learned_addr learned my addr 172.21.15.62:0/2996691230 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:39.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82ea600640 1 --2- 172.21.15.62:0/2996691230 >> v2:172.21.15.62:3300/0 conn(0x7f82e40b70d0 0x7f82e40b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:39.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82e9200640 1 --2- 172.21.15.62:0/2996691230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 0x7f82e40b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:39.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82e3e00640 1 -- 172.21.15.62:0/2996691230 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091592729 0 0) 0x7f82e40d2270 con 0x7f82e415dc10 2024-04-25T09:08:39.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82ea600640 1 -- 172.21.15.62:0/2996691230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 msgr2=0x7f82e40b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:39.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82ea600640 1 --2- 172.21.15.62:0/2996691230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 0x7f82e40b6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82ea600640 1 -- 172.21.15.62:0/2996691230 >> v1:172.21.15.62:6789/0 conn(0x7f82e415dc10 legacy=0x7f82e40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:39.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.764+0000 7f82ea600640 1 -- 172.21.15.62:0/2996691230 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40b7870 con 0x7f82e40b70d0 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.766+0000 7f82ea600640 1 --2- 172.21.15.62:0/2996691230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e40b8630 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009dc0 tx=0x7f82dc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43360f0317502b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.766+0000 7f82e3e00640 1 -- 172.21.15.62:0/2996691230 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc007740 con 0x7f82e40b70d0 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.766+0000 7f82e3e00640 1 -- 172.21.15.62:0/2996691230 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc0078e0 con 0x7f82e40b70d0 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.766+0000 7f82e3e00640 1 -- 172.21.15.62:0/2996691230 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc007cb0 con 0x7f82e40b70d0 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.766+0000 7f82ec600640 1 -- 172.21.15.62:0/2996691230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 msgr2=0x7f82e40b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.766+0000 7f82ec600640 1 --2- 172.21.15.62:0/2996691230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e40b8630 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009dc0 tx=0x7f82dc02a710 comp rx=0 tx=0).stop 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.767+0000 7f82ec600640 1 -- 172.21.15.62:0/2996691230 shutdown_connections 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.767+0000 7f82ec600640 1 --2- 172.21.15.62:0/2996691230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e40b8630 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.767+0000 7f82ec600640 1 --2- 172.21.15.62:0/2996691230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 0x7f82e40b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.767+0000 7f82ec600640 1 -- 172.21.15.62:0/2996691230 >> 172.21.15.62:0/2996691230 conn(0x7f82e406b000 msgr2=0x7f82e406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:39.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.767+0000 7f82ec600640 1 -- 172.21.15.62:0/2996691230 shutdown_connections 2024-04-25T09:08:39.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.767+0000 7f82ec600640 1 -- 172.21.15.62:0/2996691230 wait complete. 2024-04-25T09:08:39.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.768+0000 7f82ec600640 1 Processor -- start 2024-04-25T09:08:39.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.768+0000 7f82ec600640 1 -- start start 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.769+0000 7f82ec600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e41e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.769+0000 7f82ec600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 0x7f82e40cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.769+0000 7f82ec600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82e415dc10 0x7f82e414aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.769+0000 7f82ec600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e415dff0 con 0x7f82e40b70d0 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.769+0000 7f82ec600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40b7e30 con 0x7f82e415dc10 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.769+0000 7f82ec600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40b8b70 con 0x7f82e40b97b0 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e41e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e41e60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35162/0 (socket says 172.21.15.62:35162) 2024-04-25T09:08:39.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9c00640 1 -- 172.21.15.62:0/110642403 learned_addr learned my addr 172.21.15.62:0/110642403 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82ea600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82e415dc10 0x7f82e414aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9200640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 0x7f82e40cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9c00640 1 -- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 msgr2=0x7f82e40cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9c00640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 0x7f82e40cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9c00640 1 -- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82e415dc10 msgr2=0x7f82e414aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9c00640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82e415dc10 0x7f82e414aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82e9c00640 1 -- 172.21.15.62:0/110642403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e41e69c0 con 0x7f82e40b70d0 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.770+0000 7f82ea600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82e415dc10 0x7f82e414aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.771+0000 7f82e9c00640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e41e60c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f82d400c430 tx=0x7f82d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:39.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.771+0000 7f82e1600640 1 -- 172.21.15.62:0/110642403 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d40074e0 con 0x7f82e40b70d0 2024-04-25T09:08:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.771+0000 7f82e1600640 1 -- 172.21.15.62:0/110642403 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d40047e0 con 0x7f82e40b70d0 2024-04-25T09:08:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.771+0000 7f82e1600640 1 -- 172.21.15.62:0/110642403 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4004bd0 con 0x7f82e40b70d0 2024-04-25T09:08:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.771+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e41e6ca0 con 0x7f82e40b70d0 2024-04-25T09:08:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.771+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e41e7260 con 0x7f82e40b70d0 2024-04-25T09:08:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.772+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e40cf990 con 0x7f82e40b70d0 2024-04-25T09:08:39.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.773+0000 7f82e1600640 1 -- 172.21.15.62:0/110642403 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82d4011070 con 0x7f82e40b70d0 2024-04-25T09:08:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.774+0000 7f82e1600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82c406b500 0x7f82c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.774+0000 7f82e1600640 1 -- 172.21.15.62:0/110642403 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82d4089e70 con 0x7f82e40b70d0 2024-04-25T09:08:39.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.775+0000 7f82e9200640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82c406b500 0x7f82c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:39.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.776+0000 7f82e9200640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82c406b500 0x7f82c406d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f82e40c6f50 tx=0x7f82d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:39.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:39.778+0000 7f82e1600640 1 -- 172.21.15.62:0/110642403 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82e40cf990 con 0x7f82e40b70d0 2024-04-25T09:08:40.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.022+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f82e404ef20 con 0x7f82e40b70d0 2024-04-25T09:08:40.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.545+0000 7f82e1600640 1 -- 172.21.15.62:0/110642403 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f82d4058cf0 con 0x7f82e40b70d0 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82c406b500 msgr2=0x7f82c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82c406b500 0x7f82c406d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f82e40c6f50 tx=0x7f82d0008040 comp rx=0 tx=0).stop 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 msgr2=0x7f82e41e60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e41e60c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f82d400c430 tx=0x7f82d400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 shutdown_connections 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82c406b500 0x7f82c406d9c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82e415dc10 0x7f82e414aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82e40b97b0 0x7f82e40cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 --2- 172.21.15.62:0/110642403 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82e40b70d0 0x7f82e41e60c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.548+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 >> 172.21.15.62:0/110642403 conn(0x7f82e406b000 msgr2=0x7f82e40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.549+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 shutdown_connections 2024-04-25T09:08:40.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.549+0000 7f82ec600640 1 -- 172.21.15.62:0/110642403 wait complete. 2024-04-25T09:08:40.557 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:08:40.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:40.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.777+0000 7fd332600640 1 Processor -- start 2024-04-25T09:08:40.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.777+0000 7fd332600640 1 -- start start 2024-04-25T09:08:40.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.778+0000 7fd332600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd32c154ac0 0x7fd32c154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.778+0000 7fd332600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c0aeb00 0x7fd32c066940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.778+0000 7fd332600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c15e720 con 0x7fd32c066e80 2024-04-25T09:08:40.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.778+0000 7fd332600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32c0672f0 con 0x7fd32c154ac0 2024-04-25T09:08:40.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.778+0000 7fd332600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c15ea00 con 0x7fd32c0aeb00 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd331200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd32c154ac0 0x7fd32c154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd331c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd32c066e80 0x7fd32c071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44298/0 (socket says 172.21.15.62:44298) 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd331c00640 1 -- 172.21.15.62:0/147274214 learned_addr learned my addr 172.21.15.62:0/147274214 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd32be00640 1 --2- 172.21.15.62:0/147274214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c0aeb00 0x7fd32c066940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd331200640 1 -- 172.21.15.62:0/147274214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c0aeb00 msgr2=0x7fd32c066940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd331200640 1 --2- 172.21.15.62:0/147274214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c0aeb00 0x7fd32c066940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd331200640 1 -- 172.21.15.62:0/147274214 >> v1:172.21.15.62:6789/0 conn(0x7fd32c066e80 legacy=0x7fd32c071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd331200640 1 -- 172.21.15.62:0/147274214 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c15ece0 con 0x7fd32c154ac0 2024-04-25T09:08:40.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.779+0000 7fd32b400640 1 -- 172.21.15.62:0/147274214 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740694895 0 0) 0x7fd32c15e720 con 0x7fd32c066e80 2024-04-25T09:08:40.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.780+0000 7fd331200640 1 --2- 172.21.15.62:0/147274214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c154ac0 0x7fd32c154ea0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097a0 tx=0x7fd31402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92084c96f599c9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd32b400640 1 -- 172.21.15.62:0/147274214 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314037070 con 0x7fd32c154ac0 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd32b400640 1 -- 172.21.15.62:0/147274214 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd314032440 con 0x7fd32c154ac0 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd32b400640 1 -- 172.21.15.62:0/147274214 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314032810 con 0x7fd32c154ac0 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd332600640 1 -- 172.21.15.62:0/147274214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c154ac0 msgr2=0x7fd32c154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd332600640 1 --2- 172.21.15.62:0/147274214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c154ac0 0x7fd32c154ea0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd3140097a0 tx=0x7fd31402a180 comp rx=0 tx=0).stop 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd332600640 1 -- 172.21.15.62:0/147274214 shutdown_connections 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd332600640 1 --2- 172.21.15.62:0/147274214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c0aeb00 0x7fd32c066940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd332600640 1 --2- 172.21.15.62:0/147274214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c154ac0 0x7fd32c154ea0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd332600640 1 -- 172.21.15.62:0/147274214 >> 172.21.15.62:0/147274214 conn(0x7fd32c06abe0 msgr2=0x7fd32c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.781+0000 7fd332600640 1 -- 172.21.15.62:0/147274214 shutdown_connections 2024-04-25T09:08:40.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.782+0000 7fd332600640 1 -- 172.21.15.62:0/147274214 wait complete. 2024-04-25T09:08:40.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.783+0000 7fd332600640 1 Processor -- start 2024-04-25T09:08:40.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.783+0000 7fd332600640 1 -- start start 2024-04-25T09:08:40.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.784+0000 7fd332600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c066e80 0x7fd32c13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.784+0000 7fd332600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c0aeb00 0x7fd32c13c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.784+0000 7fd332600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c154ac0 0x7fd32c157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.784+0000 7fd332600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c071b80 con 0x7fd32c066e80 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.784+0000 7fd332600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c15e410 con 0x7fd32c0aeb00 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.784+0000 7fd332600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c071d00 con 0x7fd32c154ac0 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c066e80 0x7fd32c13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c066e80 0x7fd32c13bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35192/0 (socket says 172.21.15.62:35192) 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 -- 172.21.15.62:0/2453422043 learned_addr learned my addr 172.21.15.62:0/2453422043 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 -- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c154ac0 msgr2=0x7fd32c157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c154ac0 0x7fd32c157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 -- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c0aeb00 msgr2=0x7fd32c13c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c0aeb00 0x7fd32c13c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 -- 172.21.15.62:0/2453422043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0b22f0 con 0x7fd32c066e80 2024-04-25T09:08:40.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.785+0000 7fd331200640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c066e80 0x7fd32c13bf80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd314000c00 tx=0x7fd3140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.786+0000 7fd328c00640 1 -- 172.21.15.62:0/2453422043 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314037040 con 0x7fd32c066e80 2024-04-25T09:08:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.786+0000 7fd328c00640 1 -- 172.21.15.62:0/2453422043 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd314042070 con 0x7fd32c066e80 2024-04-25T09:08:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.786+0000 7fd328c00640 1 -- 172.21.15.62:0/2453422043 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314030bb0 con 0x7fd32c066e80 2024-04-25T09:08:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.786+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c0b25d0 con 0x7fd32c066e80 2024-04-25T09:08:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.786+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c0b2b90 con 0x7fd32c066e80 2024-04-25T09:08:40.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.787+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c04ef20 con 0x7fd32c066e80 2024-04-25T09:08:40.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.788+0000 7fd328c00640 1 -- 172.21.15.62:0/2453422043 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd314047460 con 0x7fd32c066e80 2024-04-25T09:08:40.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.789+0000 7fd328c00640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd30806b4d0 0x7fd30806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:40.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.789+0000 7fd328c00640 1 -- 172.21.15.62:0/2453422043 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3140aae20 con 0x7fd32c066e80 2024-04-25T09:08:40.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.790+0000 7fd32be00640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd30806b4d0 0x7fd30806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:40.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.791+0000 7fd32be00640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd30806b4d0 0x7fd30806d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0070b0 tx=0x7fd31c007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:40.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:40.793+0000 7fd328c00640 1 -- 172.21.15.62:0/2453422043 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd314032e20 con 0x7fd32c066e80 2024-04-25T09:08:40.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:08:40.934 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:08:41.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.032+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd32c0b68d0 con 0x7fd32c066e80 2024-04-25T09:08:41.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.033+0000 7fd328c00640 1 -- 172.21.15.62:0/2453422043 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fd314079c90 con 0x7fd32c066e80 2024-04-25T09:08:41.021 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:41.021 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3737740461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3737740461},{"type":"v1","addr":"172.21.15.62:6835","nonce":3737740461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":5099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/4172870608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4172870608},{"type":"v1","addr":"172.21.15.121:6836","nonce":4172870608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2112486228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":2112486228},{"type":"v1","addr":"172.21.15.121:6837","nonce":2112486228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:40.023899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/3491368038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3491368038},{"type":"v1","addr":"172.21.15.62:6834","nonce":3491368038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:08:41.021 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-04-25T09:08:41.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd30806b4d0 msgr2=0x7fd30806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd30806b4d0 0x7fd30806d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0070b0 tx=0x7fd31c007040 comp rx=0 tx=0).stop 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c066e80 msgr2=0x7fd32c13bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c066e80 0x7fd32c13bf80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd314000c00 tx=0x7fd3140046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 shutdown_connections 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd30806b4d0 0x7fd30806d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd32c154ac0 0x7fd32c157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c0aeb00 0x7fd32c13c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 --2- 172.21.15.62:0/2453422043 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd32c066e80 0x7fd32c13bf80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 >> 172.21.15.62:0/2453422043 conn(0x7fd32c06abe0 msgr2=0x7fd32c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 shutdown_connections 2024-04-25T09:08:41.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.036+0000 7fd332600640 1 -- 172.21.15.62:0/2453422043 wait complete. 2024-04-25T09:08:41.043 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.311+0000 7f2bb1e00640 1 Processor -- start 2024-04-25T09:08:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.311+0000 7f2bb1e00640 1 -- start start 2024-04-25T09:08:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.313+0000 7f2bb1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac0cfed0 0x7f2bac15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.313+0000 7f2bb1e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bac06c310 con 0x7f2bac072340 2024-04-25T09:08:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.313+0000 7f2bb1e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bac15f5f0 con 0x7f2bac0709c0 2024-04-25T09:08:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.313+0000 7f2bb1e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac06c5b0 con 0x7f2bac0cfed0 2024-04-25T09:08:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.313+0000 7f2baa000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2bac072340 0x7f2bac0702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44312/0 (socket says 172.21.15.62:44312) 2024-04-25T09:08:41.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.313+0000 7f2baa000640 1 -- 172.21.15.62:0/4027073049 learned_addr learned my addr 172.21.15.62:0/4027073049 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.313+0000 7f2baaa00640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac0cfed0 0x7f2bac15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.313+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697214008 0 0) 0x7f2bac06c310 con 0x7f2bac072340 2024-04-25T09:08:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b90003520 con 0x7f2bac072340 2024-04-25T09:08:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1917198024 0 0) 0x7f2b90003520 con 0x7f2bac072340 2024-04-25T09:08:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac0cfed0 msgr2=0x7f2bac15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac0cfed0 0x7f2bac15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 >> v1:172.21.15.121:6789/0 conn(0x7f2bac0709c0 legacy=0x7f2bac0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac06c890 con 0x7f2bac072340 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f2b98001690 con 0x7f2bac072340 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 >> v1:172.21.15.62:6789/0 conn(0x7f2bac072340 legacy=0x7f2bac0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b900039e0 0x7f2b90003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2b90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b900051c0 0x7f2b9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac06c5b0 con 0x7f2b90004800 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9000c060 con 0x7f2b900039e0 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9000c260 con 0x7f2b900051c0 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2baaa00640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac0cfed0 0x7f2bac15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.314+0000 7f2baa000640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2b90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.315+0000 7f2bab400640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b900051c0 0x7f2b9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.315+0000 7f2baa000640 1 -- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b900051c0 msgr2=0x7f2b9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.315+0000 7f2baa000640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2b900051c0 0x7f2b9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.315+0000 7f2baa000640 1 -- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b900039e0 msgr2=0x7f2b90003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:41.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.315+0000 7f2baa000640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b900039e0 0x7f2b90003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.315+0000 7f2baa000640 1 -- 172.21.15.62:0/4027073049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90003520 con 0x7f2b90004800 2024-04-25T09:08:41.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.316+0000 7f2baa000640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2b90004c80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c210 tx=0x7f2b90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293e9685731e80b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.316+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98007d30 con 0x7f2b90004800 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.316+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b980058d0 con 0x7f2b90004800 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.316+0000 7f2ba9600640 1 -- 172.21.15.62:0/4027073049 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98005ca0 con 0x7f2b90004800 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.317+0000 7f2bb1e00640 1 -- 172.21.15.62:0/4027073049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 msgr2=0x7f2b90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.317+0000 7f2bb1e00640 1 --2- 172.21.15.62:0/4027073049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2b90004c80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800c210 tx=0x7f2b90005830 comp rx=0 tx=0).stop 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.317+0000 7f2bab400640 1 -- 172.21.15.62:0/4027073049 reap_dead start 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.317+0000 7f2bb1e00640 1 -- 172.21.15.62:0/4027073049 shutdown_connections 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.317+0000 7f2bb1e00640 1 -- 172.21.15.62:0/4027073049 >> 172.21.15.62:0/4027073049 conn(0x7f2bac0668f0 msgr2=0x7f2bac066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.317+0000 7f2bb1e00640 1 -- 172.21.15.62:0/4027073049 shutdown_connections 2024-04-25T09:08:41.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.317+0000 7f2bb1e00640 1 -- 172.21.15.62:0/4027073049 wait complete. 2024-04-25T09:08:41.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.319+0000 7f2bb1e00640 1 Processor -- start 2024-04-25T09:08:41.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.319+0000 7f2bb1e00640 1 -- start start 2024-04-25T09:08:41.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2bb1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2bac15d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2bb1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bac15a700 0x7f2bac13a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2bb1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac13ac40 0x7f2bac13d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2bb1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac06d130 con 0x7f2b90004800 2024-04-25T09:08:41.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2bb1e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac06ce50 con 0x7f2bac15a700 2024-04-25T09:08:41.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2bb1e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac15f0b0 con 0x7f2bac13ac40 2024-04-25T09:08:41.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2baaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2bac15d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2baaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2bac15d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35212/0 (socket says 172.21.15.62:35212) 2024-04-25T09:08:41.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2baaa00640 1 -- 172.21.15.62:0/866171600 learned_addr learned my addr 172.21.15.62:0/866171600 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.320+0000 7f2bab400640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac13ac40 0x7f2bac13d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2baaa00640 1 -- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac13ac40 msgr2=0x7f2bac13d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2baa000640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bac15a700 0x7f2bac13a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2baaa00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac13ac40 0x7f2bac13d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2baaa00640 1 -- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bac15a700 msgr2=0x7f2bac13a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2baaa00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bac15a700 0x7f2bac13a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2baaa00640 1 -- 172.21.15.62:0/866171600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac15c3f0 con 0x7f2b90004800 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2baaa00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2bac15d490 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006d40 tx=0x7f2b9c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2ba3400640 1 -- 172.21.15.62:0/866171600 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c004230 con 0x7f2b90004800 2024-04-25T09:08:41.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2ba3400640 1 -- 172.21.15.62:0/866171600 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c0043f0 con 0x7f2b90004800 2024-04-25T09:08:41.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2ba3400640 1 -- 172.21.15.62:0/866171600 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c00ab00 con 0x7f2b90004800 2024-04-25T09:08:41.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac15c6d0 con 0x7f2b90004800 2024-04-25T09:08:41.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.321+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac15cc90 con 0x7f2b90004800 2024-04-25T09:08:41.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.323+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac072c70 con 0x7f2b90004800 2024-04-25T09:08:41.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.323+0000 7f2ba3400640 1 -- 172.21.15.62:0/866171600 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b9c00aca0 con 0x7f2b90004800 2024-04-25T09:08:41.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.324+0000 7f2ba3400640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2b8c06b4d0 0x7f2b8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.324+0000 7f2ba3400640 1 -- 172.21.15.62:0/866171600 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b9c08a340 con 0x7f2b90004800 2024-04-25T09:08:41.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.325+0000 7f2baa000640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2b8c06b4d0 0x7f2b8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.326+0000 7f2baa000640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2b8c06b4d0 0x7f2b8c06d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b98033000 tx=0x7f2b9802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.329+0000 7f2ba3400640 1 -- 172.21.15.62:0/866171600 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b9c015070 con 0x7f2b90004800 2024-04-25T09:08:41.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.534+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bac04f000 con 0x7f2b90004800 2024-04-25T09:08:41.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.537+0000 7f2ba3400640 1 -- 172.21.15.62:0/866171600 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f2b9c0591c0 con 0x7f2b90004800 2024-04-25T09:08:41.524 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:41.524 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":62,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:38.509961+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:34.817879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:35.997553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2b8c06b4d0 msgr2=0x7f2b8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2b8c06b4d0 0x7f2b8c06d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b98033000 tx=0x7f2b9802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:41.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 msgr2=0x7f2bac15d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2bac15d490 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006d40 tx=0x7f2b9c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 shutdown_connections 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2b8c06b4d0 0x7f2b8c06d990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bac13ac40 0x7f2bac13d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bac15a700 0x7f2bac13a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 --2- 172.21.15.62:0/866171600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b90004800 0x7f2bac15d490 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 >> 172.21.15.62:0/866171600 conn(0x7f2bac0668f0 msgr2=0x7f2bac15e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.540+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 shutdown_connections 2024-04-25T09:08:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.541+0000 7f2bb1e00640 1 -- 172.21.15.62:0/866171600 wait complete. 2024-04-25T09:08:41.548 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:08:41.548 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:08:41.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.770+0000 7ffbb7200640 1 Processor -- start 2024-04-25T09:08:41.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.771+0000 7ffbb7200640 1 -- start start 2024-04-25T09:08:41.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.772+0000 7ffbb7200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffbb8144c10 0x7ffbb815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.772+0000 7ffbb7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb80cf2d0 con 0x7ffbb806d900 2024-04-25T09:08:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.773+0000 7ffbb7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb815faa0 con 0x7ffbb80cee40 2024-04-25T09:08:41.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.773+0000 7ffbb7200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb8144ff0 con 0x7ffbb8144c10 2024-04-25T09:08:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.773+0000 7ffbb6800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ffbb806d900 0x7ffbb80b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44324/0 (socket says 172.21.15.62:44324) 2024-04-25T09:08:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.773+0000 7ffbb6800640 1 -- 172.21.15.62:0/1638685481 learned_addr learned my addr 172.21.15.62:0/1638685481 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.773+0000 7ffbb5e00640 1 --2- 172.21.15.62:0/1638685481 >> v2:172.21.15.121:3300/0 conn(0x7ffbb8144c10 0x7ffbb815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.774+0000 7ffbb5400640 1 -- 172.21.15.62:0/1638685481 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688245870 0 0) 0x7ffbb80cf2d0 con 0x7ffbb806d900 2024-04-25T09:08:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.774+0000 7ffbb5400640 1 -- 172.21.15.62:0/1638685481 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb9c003520 con 0x7ffbb806d900 2024-04-25T09:08:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.774+0000 7ffbb5e00640 1 -- 172.21.15.62:0/1638685481 >> v1:172.21.15.121:6789/0 conn(0x7ffbb80cee40 legacy=0x7ffbb806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.774+0000 7ffbb5e00640 1 -- 172.21.15.62:0/1638685481 >> v1:172.21.15.62:6789/0 conn(0x7ffbb806d900 legacy=0x7ffbb80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.774+0000 7ffbb5e00640 1 -- 172.21.15.62:0/1638685481 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb815fd80 con 0x7ffbb8144c10 2024-04-25T09:08:41.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.775+0000 7ffbb5e00640 1 --2- 172.21.15.62:0/1638685481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb8144c10 0x7ffbb815f450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffba80097a0 tx=0x7ffba802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67b977bbe3e382fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.776+0000 7ffbb5400640 1 -- 172.21.15.62:0/1638685481 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8037070 con 0x7ffbb8144c10 2024-04-25T09:08:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.776+0000 7ffbb5400640 1 -- 172.21.15.62:0/1638685481 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba8032440 con 0x7ffbb8144c10 2024-04-25T09:08:41.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.776+0000 7ffbb5400640 1 -- 172.21.15.62:0/1638685481 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8032810 con 0x7ffbb8144c10 2024-04-25T09:08:41.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.776+0000 7ffbb7200640 1 -- 172.21.15.62:0/1638685481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb8144c10 msgr2=0x7ffbb815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.776+0000 7ffbb7200640 1 --2- 172.21.15.62:0/1638685481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb8144c10 0x7ffbb815f450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffba80097a0 tx=0x7ffba802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.777+0000 7ffbb7200640 1 -- 172.21.15.62:0/1638685481 shutdown_connections 2024-04-25T09:08:41.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.777+0000 7ffbb7200640 1 --2- 172.21.15.62:0/1638685481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb8144c10 0x7ffbb815f450 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.777+0000 7ffbb7200640 1 -- 172.21.15.62:0/1638685481 >> 172.21.15.62:0/1638685481 conn(0x7ffbb80b14d0 msgr2=0x7ffbb80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:41.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.777+0000 7ffbb7200640 1 -- 172.21.15.62:0/1638685481 shutdown_connections 2024-04-25T09:08:41.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.777+0000 7ffbb7200640 1 -- 172.21.15.62:0/1638685481 wait complete. 2024-04-25T09:08:41.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.778+0000 7ffbb7200640 1 Processor -- start 2024-04-25T09:08:41.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.779+0000 7ffbb7200640 1 -- start start 2024-04-25T09:08:41.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb806d900 0x7ffbb81f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb80cee40 0x7ffbb80cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb8144c10 0x7ffbb8134c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80b41c0 con 0x7ffbb806d900 2024-04-25T09:08:41.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8160340 con 0x7ffbb80cee40 2024-04-25T09:08:41.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80b4d60 con 0x7ffbb8144c10 2024-04-25T09:08:41.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb806d900 0x7ffbb81f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb806d900 0x7ffbb81f7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35228/0 (socket says 172.21.15.62:35228) 2024-04-25T09:08:41.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb5e00640 1 -- 172.21.15.62:0/517673257 learned_addr learned my addr 172.21.15.62:0/517673257 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb5e00640 1 -- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb8144c10 msgr2=0x7ffbb8134c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb5e00640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb8144c10 0x7ffbb8134c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb5e00640 1 -- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb80cee40 msgr2=0x7ffbb80cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb5e00640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb80cee40 0x7ffbb80cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.780+0000 7ffbb5e00640 1 -- 172.21.15.62:0/517673257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80cdc30 con 0x7ffbb806d900 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.781+0000 7ffbb5e00640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb806d900 0x7ffbb81f7460 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ffba80098c0 tx=0x7ffba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.781+0000 7ffbaea00640 1 -- 172.21.15.62:0/517673257 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8037040 con 0x7ffbb806d900 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.781+0000 7ffbaea00640 1 -- 172.21.15.62:0/517673257 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba8042070 con 0x7ffbb806d900 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.781+0000 7ffbaea00640 1 -- 172.21.15.62:0/517673257 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8030940 con 0x7ffbb806d900 2024-04-25T09:08:41.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.781+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb80cdf10 con 0x7ffbb806d900 2024-04-25T09:08:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.781+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb80c9fa0 con 0x7ffbb806d900 2024-04-25T09:08:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.783+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb804ef20 con 0x7ffbb806d900 2024-04-25T09:08:41.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.784+0000 7ffbaea00640 1 -- 172.21.15.62:0/517673257 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffba8030ae0 con 0x7ffbb806d900 2024-04-25T09:08:41.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.784+0000 7ffbaea00640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b4d0 0x7ffb9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:41.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.785+0000 7ffbaea00640 1 -- 172.21.15.62:0/517673257 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffba80aabe0 con 0x7ffbb806d900 2024-04-25T09:08:41.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.785+0000 7ffbad400640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b4d0 0x7ffb9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:41.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.787+0000 7ffbad400640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b4d0 0x7ffb9406d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffba0000f60 tx=0x7ffba00079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:41.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.789+0000 7ffbaea00640 1 -- 172.21.15.62:0/517673257 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffba8045360 con 0x7ffbb806d900 2024-04-25T09:08:41.863 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-04-25T09:08:41.876+0000 7f6a6f000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:08:41.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:41.988+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbb80b22b0 con 0x7ffb9406b4d0 2024-04-25T09:08:41.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.006+0000 7ffbaea00640 1 -- 172.21.15.62:0/517673257 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240078 (secure 0 0 0) 0x7ffbb80b22b0 con 0x7ffb9406b4d0 2024-04-25T09:08:41.997 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:41.998 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b4d0 msgr2=0x7ffb9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b4d0 0x7ffb9406d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffba0000f60 tx=0x7ffba00079b0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb806d900 msgr2=0x7ffbb81f7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb806d900 0x7ffbb81f7460 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ffba80098c0 tx=0x7ffba80046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 shutdown_connections 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b4d0 0x7ffb9406d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb8144c10 0x7ffbb8134c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb80cee40 0x7ffbb80cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 --2- 172.21.15.62:0/517673257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb806d900 0x7ffbb81f7460 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.011+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 >> 172.21.15.62:0/517673257 conn(0x7ffbb80b14d0 msgr2=0x7ffbb815cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.012+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 shutdown_connections 2024-04-25T09:08:42.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.012+0000 7ffbb7200640 1 -- 172.21.15.62:0/517673257 wait complete. 2024-04-25T09:08:42.020 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-04-25T09:08:40.010062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":332404,"kb_used_data":6660,"kb_used_omap":12,"kb_used_meta":325619,"kb_avail":754642316,"statfs":{"total":773094113280,"available":772753731584,"internally_reserved":0,"allocated":6819840,"data_stored":3532901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":333434416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.538751"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.475390+0000","last_change":"2024-04-25T09:08:35.475390+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.475390+0000","last_undegraded":"2024-04-25T09:08:35.475390+0000","last_fullsized":"2024-04-25T09:08:35.475390+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.476223+0000","last_change":"2024-04-25T09:08:35.476223+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.476223+0000","last_undegraded":"2024-04-25T09:08:35.476223+0000","last_fullsized":"2024-04-25T09:08:35.476223+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.487809+0000","last_change":"2024-04-25T09:08:35.487809+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.487809+0000","last_undegraded":"2024-04-25T09:08:35.487809+0000","last_fullsized":"2024-04-25T09:08:35.487809+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.488299+0000","last_change":"2024-04-25T09:08:35.488299+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.488299+0000","last_undegraded":"2024-04-25T09:08:35.488299+0000","last_fullsized":"2024-04-25T09:08:35.488299+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.485569+0000","last_change":"2024-04-25T09:08:35.485569+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.485569+0000","last_undegraded":"2024-04-25T09:08:35.485569+0000","last_fullsized":"2024-04-25T09:08:35.485569+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.486921+0000","last_change":"2024-04-25T09:08:35.486921+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.486921+0000","last_undegraded":"2024-04-25T09:08:35.486921+0000","last_fullsized":"2024-04-25T09:08:35.486921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.478759+0000","last_change":"2024-04-25T09:08:35.478759+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.478759+0000","last_undegraded":"2024-04-25T09:08:35.478759+0000","last_fullsized":"2024-04-25T09:08:35.478759+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.480769+0000","last_change":"2024-04-25T09:08:35.480769+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.480769+0000","last_undegraded":"2024-04-25T09:08:35.480769+0000","last_fullsized":"2024-04-25T09:08:35.480769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.478706+0000","last_change":"2024-04-25T09:08:35.478706+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.478706+0000","last_undegraded":"2024-04-25T09:08:35.478706+0000","last_fullsized":"2024-04-25T09:08:35.478706+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.485020+0000","last_change":"2024-04-25T09:08:35.485020+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.485020+0000","last_undegraded":"2024-04-25T09:08:35.485020+0000","last_fullsized":"2024-04-25T09:08:35.485020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.488363+0000","last_change":"2024-04-25T09:08:35.488363+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.488363+0000","last_undegraded":"2024-04-25T09:08:35.488363+0000","last_fullsized":"2024-04-25T09:08:35.488363+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.479373+0000","last_change":"2024-04-25T09:08:35.479373+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.479373+0000","last_undegraded":"2024-04-25T09:08:35.479373+0000","last_fullsized":"2024-04-25T09:08:35.479373+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.482523+0000","last_change":"2024-04-25T09:08:35.482523+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.482523+0000","last_undegraded":"2024-04-25T09:08:35.482523+0000","last_fullsized":"2024-04-25T09:08:35.482523+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.488362+0000","last_change":"2024-04-25T09:08:35.488362+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.488362+0000","last_undegraded":"2024-04-25T09:08:35.488362+0000","last_fullsized":"2024-04-25T09:08:35.488362+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.488295+0000","last_change":"2024-04-25T09:08:35.488295+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.488295+0000","last_undegraded":"2024-04-25T09:08:35.488295+0000","last_fullsized":"2024-04-25T09:08:35.488295+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.488317+0000","last_change":"2024-04-25T09:08:35.488317+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.488317+0000","last_undegraded":"2024-04-25T09:08:35.488317+0000","last_fullsized":"2024-04-25T09:08:35.488317+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.478335+0000","last_change":"2024-04-25T09:08:35.478335+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.478335+0000","last_undegraded":"2024-04-25T09:08:35.478335+0000","last_fullsized":"2024-04-25T09:08:35.478335+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.482504+0000","last_change":"2024-04-25T09:08:35.482504+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.482504+0000","last_undegraded":"2024-04-25T09:08:35.482504+0000","last_fullsized":"2024-04-25T09:08:35.482504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.483239+0000","last_change":"2024-04-25T09:08:35.483239+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.483239+0000","last_undegraded":"2024-04-25T09:08:35.483239+0000","last_fullsized":"2024-04-25T09:08:35.483239+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.483194+0000","last_change":"2024-04-25T09:08:35.483194+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.483194+0000","last_undegraded":"2024-04-25T09:08:35.483194+0000","last_fullsized":"2024-04-25T09:08:35.483194+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.475476+0000","last_change":"2024-04-25T09:08:35.475476+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.475476+0000","last_undegraded":"2024-04-25T09:08:35.475476+0000","last_fullsized":"2024-04-25T09:08:35.475476+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.482449+0000","last_change":"2024-04-25T09:08:35.482449+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.482449+0000","last_undegraded":"2024-04-25T09:08:35.482449+0000","last_fullsized":"2024-04-25T09:08:35.482449+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.478707+0000","last_change":"2024-04-25T09:08:35.478707+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.478707+0000","last_undegraded":"2024-04-25T09:08:35.478707+0000","last_fullsized":"2024-04-25T09:08:35.478707+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.476425+0000","last_change":"2024-04-25T09:08:35.476425+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.476425+0000","last_undegraded":"2024-04-25T09:08:35.476425+0000","last_fullsized":"2024-04-25T09:08:35.476425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.485856+0000","last_change":"2024-04-25T09:08:35.485856+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.485856+0000","last_undegraded":"2024-04-25T09:08:35.485856+0000","last_fullsized":"2024-04-25T09:08:35.485856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.479421+0000","last_change":"2024-04-25T09:08:35.479421+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.479421+0000","last_undegraded":"2024-04-25T09:08:35.479421+0000","last_fullsized":"2024-04-25T09:08:35.479421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.484244+0000","last_change":"2024-04-25T09:08:35.484244+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.484244+0000","last_undegraded":"2024-04-25T09:08:35.484244+0000","last_fullsized":"2024-04-25T09:08:35.484244+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.476216+0000","last_change":"2024-04-25T09:08:35.476216+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.476216+0000","last_undegraded":"2024-04-25T09:08:35.476216+0000","last_fullsized":"2024-04-25T09:08:35.476216+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.484368+0000","last_change":"2024-04-25T09:08:35.484368+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.484368+0000","last_undegraded":"2024-04-25T09:08:35.484368+0000","last_fullsized":"2024-04-25T09:08:35.484368+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.485157+0000","last_change":"2024-04-25T09:08:35.485157+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.485157+0000","last_undegraded":"2024-04-25T09:08:35.485157+0000","last_fullsized":"2024-04-25T09:08:35.485157+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.482434+0000","last_change":"2024-04-25T09:08:35.482434+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.482434+0000","last_undegraded":"2024-04-25T09:08:35.482434+0000","last_fullsized":"2024-04-25T09:08:35.482434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:08:36.468102+0000","last_change":"2024-04-25T09:08:36.468102+0000","last_active":"2024-04-25T09:08:36.468102+0000","last_peered":"2024-04-25T09:08:36.468102+0000","last_clean":"2024-04-25T09:08:36.468102+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:36.468102+0000","last_undegraded":"2024-04-25T09:08:36.468102+0000","last_fullsized":"2024-04-25T09:08:36.468102+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":59,"state":"creating+peering","last_fresh":"2024-04-25T09:08:35.481224+0000","last_change":"2024-04-25T09:08:35.481224+0000","last_active":"2024-04-25T09:08:35.453249+0000","last_peered":"2024-04-25T09:08:35.453249+0000","last_clean":"2024-04-25T09:08:35.453249+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:08:35.481224+0000","last_undegraded":"2024-04-25T09:08:35.481224+0000","last_fullsized":"2024-04-25T09:08:35.481224+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":240,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-25T09:08:35.480218+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:08:35.480218+0000","last_peered":"2024-04-25T09:08:35.480218+0000","last_clean":"2024-04-25T09:08:35.480218+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:08:35.480218+0000","last_undegraded":"2024-04-25T09:08:35.480218+0000","last_fullsized":"2024-04-25T09:08:35.480218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36032,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335808,"statfs":{"total":96636764160,"available":96599867392,"internally_reserved":0,"allocated":708608,"data_stored":296202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40180,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331660,"statfs":{"total":96636764160,"available":96595619840,"internally_reserved":0,"allocated":696320,"data_stored":290385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40768,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331072,"statfs":{"total":96636764160,"available":96595017728,"internally_reserved":0,"allocated":1290240,"data_stored":880753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":290385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40192,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331648,"statfs":{"total":96636764160,"available":96595607552,"internally_reserved":0,"allocated":708608,"data_stored":296202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58240,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313600,"statfs":{"total":96636764160,"available":96577126400,"internally_reserved":0,"allocated":708608,"data_stored":296202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40772,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331068,"statfs":{"total":96636764160,"available":96595013632,"internally_reserved":0,"allocated":1302528,"data_stored":886570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574917,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36032,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335808,"statfs":{"total":96636764160,"available":96599867392,"internally_reserved":0,"allocated":708608,"data_stored":296202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:08:42.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:08:42.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.290+0000 7f84b7e00640 1 Processor -- start 2024-04-25T09:08:42.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.291+0000 7f84b7e00640 1 -- start start 2024-04-25T09:08:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.292+0000 7f84b7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84b806d900 0x7f84b815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f84b80cee40 0x7f84b80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b7e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b815faa0 con 0x7f84b8144c10 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b7e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b8144ff0 con 0x7f84b806d900 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b7e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b8160340 con 0x7f84b80cee40 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84b806d900 0x7f84b815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84b806d900 0x7f84b815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35238/0 (socket says 172.21.15.62:35238) 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b6000640 1 -- 172.21.15.62:0/1439775799 learned_addr learned my addr 172.21.15.62:0/1439775799 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b7400640 1 --2- 172.21.15.62:0/1439775799 >> v2:172.21.15.121:3300/0 conn(0x7f84b80cee40 0x7f84b80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b6000640 1 -- 172.21.15.62:0/1439775799 >> v2:172.21.15.121:3300/0 conn(0x7f84b80cee40 msgr2=0x7f84b80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b6000640 1 --2- 172.21.15.62:0/1439775799 >> v2:172.21.15.121:3300/0 conn(0x7f84b80cee40 0x7f84b80b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b6000640 1 -- 172.21.15.62:0/1439775799 >> v1:172.21.15.121:6789/0 conn(0x7f84b8144c10 legacy=0x7f84b806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b6000640 1 -- 172.21.15.62:0/1439775799 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b815fd80 con 0x7f84b806d900 2024-04-25T09:08:42.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.293+0000 7f84b7400640 1 --2- 172.21.15.62:0/1439775799 >> v2:172.21.15.121:3300/0 conn(0x7f84b80cee40 0x7f84b80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:42.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.294+0000 7f84b6000640 1 --2- 172.21.15.62:0/1439775799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b806d900 0x7f84b815f450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f84b815f390 tx=0x7f84a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db8096204009eef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:42.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.295+0000 7f84b5600640 1 -- 172.21.15.62:0/1439775799 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4037070 con 0x7f84b806d900 2024-04-25T09:08:42.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.295+0000 7f84b5600640 1 -- 172.21.15.62:0/1439775799 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a4032440 con 0x7f84b806d900 2024-04-25T09:08:42.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.295+0000 7f84b5600640 1 -- 172.21.15.62:0/1439775799 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4032810 con 0x7f84b806d900 2024-04-25T09:08:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.295+0000 7f84b7e00640 1 -- 172.21.15.62:0/1439775799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b806d900 msgr2=0x7f84b815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.295+0000 7f84b7e00640 1 --2- 172.21.15.62:0/1439775799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b806d900 0x7f84b815f450 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f84b815f390 tx=0x7f84a402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.296+0000 7f84b7e00640 1 -- 172.21.15.62:0/1439775799 shutdown_connections 2024-04-25T09:08:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.296+0000 7f84b7e00640 1 --2- 172.21.15.62:0/1439775799 >> v2:172.21.15.121:3300/0 conn(0x7f84b80cee40 0x7f84b80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.296+0000 7f84b7e00640 1 --2- 172.21.15.62:0/1439775799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b806d900 0x7f84b815f450 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.296+0000 7f84b7e00640 1 -- 172.21.15.62:0/1439775799 >> 172.21.15.62:0/1439775799 conn(0x7f84b80b14d0 msgr2=0x7f84b80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:42.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.296+0000 7f84b7e00640 1 -- 172.21.15.62:0/1439775799 shutdown_connections 2024-04-25T09:08:42.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.296+0000 7f84b7e00640 1 -- 172.21.15.62:0/1439775799 wait complete. 2024-04-25T09:08:42.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.298+0000 7f84b7e00640 1 Processor -- start 2024-04-25T09:08:42.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.298+0000 7f84b7e00640 1 -- start start 2024-04-25T09:08:42.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.299+0000 7f84b7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84b806d900 0x7f84b815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.299+0000 7f84b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b80cee40 0x7f84b8142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84b8144c10 0x7f84b80cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80cf270 con 0x7f84b80cee40 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80b41c0 con 0x7f84b8144c10 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b80b4b90 con 0x7f84b806d900 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b80cee40 0x7f84b8142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b80cee40 0x7f84b8142760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35246/0 (socket says 172.21.15.62:35246) 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b6000640 1 -- 172.21.15.62:0/3728892931 learned_addr learned my addr 172.21.15.62:0/3728892931 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b6000640 1 -- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84b806d900 msgr2=0x7f84b815b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b6000640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84b806d900 0x7f84b815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b6000640 1 -- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84b8144c10 msgr2=0x7f84b80cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b6000640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84b8144c10 0x7f84b80cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.300+0000 7f84b6000640 1 -- 172.21.15.62:0/3728892931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b81430d0 con 0x7f84b80cee40 2024-04-25T09:08:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.301+0000 7f84b6000640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b80cee40 0x7f84b8142760 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f84a4000c00 tx=0x7f84a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:42.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.301+0000 7f84aea00640 1 -- 172.21.15.62:0/3728892931 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4041020 con 0x7f84b80cee40 2024-04-25T09:08:42.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.301+0000 7f84aea00640 1 -- 172.21.15.62:0/3728892931 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a4030970 con 0x7f84b80cee40 2024-04-25T09:08:42.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.301+0000 7f84aea00640 1 -- 172.21.15.62:0/3728892931 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4037040 con 0x7f84b80cee40 2024-04-25T09:08:42.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.301+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b80cc2c0 con 0x7f84b80cee40 2024-04-25T09:08:42.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.301+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b80ccb60 con 0x7f84b80cee40 2024-04-25T09:08:42.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.302+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b804ef20 con 0x7f84b80cee40 2024-04-25T09:08:42.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.303+0000 7f84aea00640 1 -- 172.21.15.62:0/3728892931 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84a4044020 con 0x7f84b80cee40 2024-04-25T09:08:42.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.304+0000 7f84aea00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f849806b4d0 0x7f849806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.304+0000 7f84aea00640 1 -- 172.21.15.62:0/3728892931 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84a40aac70 con 0x7f84b80cee40 2024-04-25T09:08:42.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.305+0000 7f84b6a00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f849806b4d0 0x7f849806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.306+0000 7f84b6a00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f849806b4d0 0x7f849806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f84a8006fd0 tx=0x7f84a8006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:42.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.308+0000 7f84aea00640 1 -- 172.21.15.62:0/3728892931 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84a404b030 con 0x7f84b80cee40 2024-04-25T09:08:42.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.502+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84b806d270 con 0x7f849806b4d0 2024-04-25T09:08:42.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.522+0000 7f84aea00640 1 -- 172.21.15.62:0/3728892931 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251933 (secure 0 0 0) 0x7f84b806d270 con 0x7f849806b4d0 2024-04-25T09:08:42.513 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:42.514 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f849806b4d0 msgr2=0x7f849806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f849806b4d0 0x7f849806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f84a8006fd0 tx=0x7f84a8006bc0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b80cee40 msgr2=0x7f84b8142760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b80cee40 0x7f84b8142760 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f84a4000c00 tx=0x7f84a40046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 shutdown_connections 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f849806b4d0 0x7f849806d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84b80cee40 0x7f84b8142760 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84b8144c10 0x7f84b80cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 --2- 172.21.15.62:0/3728892931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84b806d900 0x7f84b815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.527+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 >> 172.21.15.62:0/3728892931 conn(0x7f84b80b14d0 msgr2=0x7f84b815e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:42.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.528+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 shutdown_connections 2024-04-25T09:08:42.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.528+0000 7f84b7e00640 1 -- 172.21.15.62:0/3728892931 wait complete. 2024-04-25T09:08:42.534 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-04-25T09:08:42.010655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":332764,"kb_used_data":7012,"kb_used_omap":12,"kb_used_meta":325619,"kb_avail":754641956,"statfs":{"total":773094113280,"available":772753362944,"internally_reserved":0,"allocated":7180288,"data_stored":3691072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":333434416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002080"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090541+0000","last_change":"2024-04-25T09:08:37.544716+0000","last_active":"2024-04-25T09:08:39.090541+0000","last_peered":"2024-04-25T09:08:39.090541+0000","last_clean":"2024-04-25T09:08:39.090541+0000","last_became_active":"2024-04-25T09:08:37.538910+0000","last_became_peered":"2024-04-25T09:08:37.538910+0000","last_unstale":"2024-04-25T09:08:39.090541+0000","last_undegraded":"2024-04-25T09:08:39.090541+0000","last_fullsized":"2024-04-25T09:08:39.090541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:43.227235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.528659+0000","last_change":"2024-04-25T09:08:37.544273+0000","last_active":"2024-04-25T09:08:38.528659+0000","last_peered":"2024-04-25T09:08:38.528659+0000","last_clean":"2024-04-25T09:08:38.528659+0000","last_became_active":"2024-04-25T09:08:37.543932+0000","last_became_peered":"2024-04-25T09:08:37.543932+0000","last_unstale":"2024-04-25T09:08:38.528659+0000","last_undegraded":"2024-04-25T09:08:38.528659+0000","last_fullsized":"2024-04-25T09:08:38.528659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:11.659059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.515871+0000","last_change":"2024-04-25T09:08:37.522046+0000","last_active":"2024-04-25T09:08:39.515871+0000","last_peered":"2024-04-25T09:08:39.515871+0000","last_clean":"2024-04-25T09:08:39.515871+0000","last_became_active":"2024-04-25T09:08:37.521553+0000","last_became_peered":"2024-04-25T09:08:37.521553+0000","last_unstale":"2024-04-25T09:08:39.515871+0000","last_undegraded":"2024-04-25T09:08:39.515871+0000","last_fullsized":"2024-04-25T09:08:39.515871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.074048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457927+0000","last_change":"2024-04-25T09:08:37.546261+0000","last_active":"2024-04-25T09:08:39.457927+0000","last_peered":"2024-04-25T09:08:39.457927+0000","last_clean":"2024-04-25T09:08:39.457927+0000","last_became_active":"2024-04-25T09:08:37.545850+0000","last_became_peered":"2024-04-25T09:08:37.545850+0000","last_unstale":"2024-04-25T09:08:39.457927+0000","last_undegraded":"2024-04-25T09:08:39.457927+0000","last_fullsized":"2024-04-25T09:08:39.457927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:44.814713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.089731+0000","last_change":"2024-04-25T09:08:37.545290+0000","last_active":"2024-04-25T09:08:39.089731+0000","last_peered":"2024-04-25T09:08:39.089731+0000","last_clean":"2024-04-25T09:08:39.089731+0000","last_became_active":"2024-04-25T09:08:37.544476+0000","last_became_peered":"2024-04-25T09:08:37.544476+0000","last_unstale":"2024-04-25T09:08:39.089731+0000","last_undegraded":"2024-04-25T09:08:39.089731+0000","last_fullsized":"2024-04-25T09:08:39.089731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:46.327191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.089517+0000","last_change":"2024-04-25T09:08:37.542965+0000","last_active":"2024-04-25T09:08:39.089517+0000","last_peered":"2024-04-25T09:08:39.089517+0000","last_clean":"2024-04-25T09:08:39.089517+0000","last_became_active":"2024-04-25T09:08:37.542487+0000","last_became_peered":"2024-04-25T09:08:37.542487+0000","last_unstale":"2024-04-25T09:08:39.089517+0000","last_undegraded":"2024-04-25T09:08:39.089517+0000","last_fullsized":"2024-04-25T09:08:39.089517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:37.904023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457146+0000","last_change":"2024-04-25T09:08:37.524130+0000","last_active":"2024-04-25T09:08:39.457146+0000","last_peered":"2024-04-25T09:08:39.457146+0000","last_clean":"2024-04-25T09:08:39.457146+0000","last_became_active":"2024-04-25T09:08:37.523756+0000","last_became_peered":"2024-04-25T09:08:37.523756+0000","last_unstale":"2024-04-25T09:08:39.457146+0000","last_undegraded":"2024-04-25T09:08:39.457146+0000","last_fullsized":"2024-04-25T09:08:39.457146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:54.725361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.537078+0000","last_change":"2024-04-25T09:08:37.529811+0000","last_active":"2024-04-25T09:08:37.537078+0000","last_peered":"2024-04-25T09:08:37.537078+0000","last_clean":"2024-04-25T09:08:37.537078+0000","last_became_active":"2024-04-25T09:08:37.529449+0000","last_became_peered":"2024-04-25T09:08:37.529449+0000","last_unstale":"2024-04-25T09:08:37.537078+0000","last_undegraded":"2024-04-25T09:08:37.537078+0000","last_fullsized":"2024-04-25T09:08:37.537078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:43.227235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.547605+0000","last_change":"2024-04-25T09:08:37.545148+0000","last_active":"2024-04-25T09:08:37.547605+0000","last_peered":"2024-04-25T09:08:37.547605+0000","last_clean":"2024-04-25T09:08:37.547605+0000","last_became_active":"2024-04-25T09:08:37.544745+0000","last_became_peered":"2024-04-25T09:08:37.544745+0000","last_unstale":"2024-04-25T09:08:37.547605+0000","last_undegraded":"2024-04-25T09:08:37.547605+0000","last_fullsized":"2024-04-25T09:08:37.547605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:59.684469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090191+0000","last_change":"2024-04-25T09:08:37.538002+0000","last_active":"2024-04-25T09:08:39.090191+0000","last_peered":"2024-04-25T09:08:39.090191+0000","last_clean":"2024-04-25T09:08:39.090191+0000","last_became_active":"2024-04-25T09:08:37.537643+0000","last_became_peered":"2024-04-25T09:08:37.537643+0000","last_unstale":"2024-04-25T09:08:39.090191+0000","last_undegraded":"2024-04-25T09:08:39.090191+0000","last_fullsized":"2024-04-25T09:08:39.090191+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:04.737876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.530488+0000","last_change":"2024-04-25T09:08:37.543406+0000","last_active":"2024-04-25T09:08:38.530488+0000","last_peered":"2024-04-25T09:08:38.530488+0000","last_clean":"2024-04-25T09:08:38.530488+0000","last_became_active":"2024-04-25T09:08:37.543096+0000","last_became_peered":"2024-04-25T09:08:37.543096+0000","last_unstale":"2024-04-25T09:08:38.530488+0000","last_undegraded":"2024-04-25T09:08:38.530488+0000","last_fullsized":"2024-04-25T09:08:38.530488+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:04.737876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.527149+0000","last_change":"2024-04-25T09:08:37.522774+0000","last_active":"2024-04-25T09:08:37.527149+0000","last_peered":"2024-04-25T09:08:37.527149+0000","last_clean":"2024-04-25T09:08:37.527149+0000","last_became_active":"2024-04-25T09:08:37.522447+0000","last_became_peered":"2024-04-25T09:08:37.522447+0000","last_unstale":"2024-04-25T09:08:37.527149+0000","last_undegraded":"2024-04-25T09:08:37.527149+0000","last_fullsized":"2024-04-25T09:08:37.527149+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:52.222445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.534959+0000","last_change":"2024-04-25T09:08:37.529624+0000","last_active":"2024-04-25T09:08:37.534959+0000","last_peered":"2024-04-25T09:08:37.534959+0000","last_clean":"2024-04-25T09:08:37.534959+0000","last_became_active":"2024-04-25T09:08:37.529317+0000","last_became_peered":"2024-04-25T09:08:37.529317+0000","last_unstale":"2024-04-25T09:08:37.534959+0000","last_undegraded":"2024-04-25T09:08:37.534959+0000","last_fullsized":"2024-04-25T09:08:37.534959+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:38.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.515869+0000","last_change":"2024-04-25T09:08:37.537992+0000","last_active":"2024-04-25T09:08:39.515869+0000","last_peered":"2024-04-25T09:08:39.515869+0000","last_clean":"2024-04-25T09:08:39.515869+0000","last_became_active":"2024-04-25T09:08:37.537291+0000","last_became_peered":"2024-04-25T09:08:37.537291+0000","last_unstale":"2024-04-25T09:08:39.515869+0000","last_undegraded":"2024-04-25T09:08:39.515869+0000","last_fullsized":"2024-04-25T09:08:39.515869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:58.433170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.540684+0000","last_change":"2024-04-25T09:08:37.536914+0000","last_active":"2024-04-25T09:08:37.540684+0000","last_peered":"2024-04-25T09:08:37.540684+0000","last_clean":"2024-04-25T09:08:37.540684+0000","last_became_active":"2024-04-25T09:08:37.535748+0000","last_became_peered":"2024-04-25T09:08:37.535748+0000","last_unstale":"2024-04-25T09:08:37.540684+0000","last_undegraded":"2024-04-25T09:08:37.540684+0000","last_fullsized":"2024-04-25T09:08:37.540684+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:18.926327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.089518+0000","last_change":"2024-04-25T09:08:37.540569+0000","last_active":"2024-04-25T09:08:39.089518+0000","last_peered":"2024-04-25T09:08:39.089518+0000","last_clean":"2024-04-25T09:08:39.089518+0000","last_became_active":"2024-04-25T09:08:37.534447+0000","last_became_peered":"2024-04-25T09:08:37.534447+0000","last_unstale":"2024-04-25T09:08:39.089518+0000","last_undegraded":"2024-04-25T09:08:39.089518+0000","last_fullsized":"2024-04-25T09:08:39.089518+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:04.256504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090757+0000","last_change":"2024-04-25T09:08:37.537442+0000","last_active":"2024-04-25T09:08:39.090757+0000","last_peered":"2024-04-25T09:08:39.090757+0000","last_clean":"2024-04-25T09:08:39.090757+0000","last_became_active":"2024-04-25T09:08:37.537033+0000","last_became_peered":"2024-04-25T09:08:37.537033+0000","last_unstale":"2024-04-25T09:08:39.090757+0000","last_undegraded":"2024-04-25T09:08:39.090757+0000","last_fullsized":"2024-04-25T09:08:39.090757+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:15.657509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.516592+0000","last_change":"2024-04-25T09:08:37.533971+0000","last_active":"2024-04-25T09:08:39.516592+0000","last_peered":"2024-04-25T09:08:39.516592+0000","last_clean":"2024-04-25T09:08:39.516592+0000","last_became_active":"2024-04-25T09:08:37.533667+0000","last_became_peered":"2024-04-25T09:08:37.533667+0000","last_unstale":"2024-04-25T09:08:39.516592+0000","last_undegraded":"2024-04-25T09:08:39.516592+0000","last_fullsized":"2024-04-25T09:08:39.516592+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:24.099039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.528434+0000","last_change":"2024-04-25T09:08:37.541472+0000","last_active":"2024-04-25T09:08:38.528434+0000","last_peered":"2024-04-25T09:08:38.528434+0000","last_clean":"2024-04-25T09:08:38.528434+0000","last_became_active":"2024-04-25T09:08:37.541153+0000","last_became_peered":"2024-04-25T09:08:37.541153+0000","last_unstale":"2024-04-25T09:08:38.528434+0000","last_undegraded":"2024-04-25T09:08:38.528434+0000","last_fullsized":"2024-04-25T09:08:38.528434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:43.397392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.530500+0000","last_change":"2024-04-25T09:08:37.546654+0000","last_active":"2024-04-25T09:08:38.530500+0000","last_peered":"2024-04-25T09:08:38.530500+0000","last_clean":"2024-04-25T09:08:38.530500+0000","last_became_active":"2024-04-25T09:08:37.546341+0000","last_became_peered":"2024-04-25T09:08:37.546341+0000","last_unstale":"2024-04-25T09:08:38.530500+0000","last_undegraded":"2024-04-25T09:08:38.530500+0000","last_fullsized":"2024-04-25T09:08:38.530500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:38.375919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.512187+0000","last_change":"2024-04-25T09:08:37.510818+0000","last_active":"2024-04-25T09:08:37.512187+0000","last_peered":"2024-04-25T09:08:37.512187+0000","last_clean":"2024-04-25T09:08:37.512187+0000","last_became_active":"2024-04-25T09:08:37.510538+0000","last_became_peered":"2024-04-25T09:08:37.510538+0000","last_unstale":"2024-04-25T09:08:37.512187+0000","last_undegraded":"2024-04-25T09:08:37.512187+0000","last_fullsized":"2024-04-25T09:08:37.512187+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:39.877133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.530742+0000","last_change":"2024-04-25T09:08:37.522121+0000","last_active":"2024-04-25T09:08:38.530742+0000","last_peered":"2024-04-25T09:08:38.530742+0000","last_clean":"2024-04-25T09:08:38.530742+0000","last_became_active":"2024-04-25T09:08:37.521749+0000","last_became_peered":"2024-04-25T09:08:37.521749+0000","last_unstale":"2024-04-25T09:08:38.530742+0000","last_undegraded":"2024-04-25T09:08:38.530742+0000","last_fullsized":"2024-04-25T09:08:38.530742+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:43.227235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457455+0000","last_change":"2024-04-25T09:08:37.535212+0000","last_active":"2024-04-25T09:08:39.457455+0000","last_peered":"2024-04-25T09:08:39.457455+0000","last_clean":"2024-04-25T09:08:39.457455+0000","last_became_active":"2024-04-25T09:08:37.534668+0000","last_became_peered":"2024-04-25T09:08:37.534668+0000","last_unstale":"2024-04-25T09:08:39.457455+0000","last_undegraded":"2024-04-25T09:08:39.457455+0000","last_fullsized":"2024-04-25T09:08:39.457455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:56.633159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.547581+0000","last_change":"2024-04-25T09:08:37.544898+0000","last_active":"2024-04-25T09:08:37.547581+0000","last_peered":"2024-04-25T09:08:37.547581+0000","last_clean":"2024-04-25T09:08:37.547581+0000","last_became_active":"2024-04-25T09:08:37.544599+0000","last_became_peered":"2024-04-25T09:08:37.544599+0000","last_unstale":"2024-04-25T09:08:37.547581+0000","last_undegraded":"2024-04-25T09:08:37.547581+0000","last_fullsized":"2024-04-25T09:08:37.547581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:28.516850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.528654+0000","last_change":"2024-04-25T09:08:37.544031+0000","last_active":"2024-04-25T09:08:38.528654+0000","last_peered":"2024-04-25T09:08:38.528654+0000","last_clean":"2024-04-25T09:08:38.528654+0000","last_became_active":"2024-04-25T09:08:37.543676+0000","last_became_peered":"2024-04-25T09:08:37.543676+0000","last_unstale":"2024-04-25T09:08:38.528654+0000","last_undegraded":"2024-04-25T09:08:38.528654+0000","last_fullsized":"2024-04-25T09:08:38.528654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:33.051357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.528809+0000","last_change":"2024-04-25T09:08:37.524261+0000","last_active":"2024-04-25T09:08:37.528809+0000","last_peered":"2024-04-25T09:08:37.528809+0000","last_clean":"2024-04-25T09:08:37.528809+0000","last_became_active":"2024-04-25T09:08:37.524006+0000","last_became_peered":"2024-04-25T09:08:37.524006+0000","last_unstale":"2024-04-25T09:08:37.528809+0000","last_undegraded":"2024-04-25T09:08:37.528809+0000","last_fullsized":"2024-04-25T09:08:37.528809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:09.845585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.528813+0000","last_change":"2024-04-25T09:08:37.537689+0000","last_active":"2024-04-25T09:08:38.528813+0000","last_peered":"2024-04-25T09:08:38.528813+0000","last_clean":"2024-04-25T09:08:38.528813+0000","last_became_active":"2024-04-25T09:08:37.536544+0000","last_became_peered":"2024-04-25T09:08:37.536544+0000","last_unstale":"2024-04-25T09:08:38.528813+0000","last_undegraded":"2024-04-25T09:08:38.528813+0000","last_fullsized":"2024-04-25T09:08:38.528813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:05.325911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.538678+0000","last_change":"2024-04-25T09:08:37.534948+0000","last_active":"2024-04-25T09:08:37.538678+0000","last_peered":"2024-04-25T09:08:37.538678+0000","last_clean":"2024-04-25T09:08:37.538678+0000","last_became_active":"2024-04-25T09:08:37.534559+0000","last_became_peered":"2024-04-25T09:08:37.534559+0000","last_unstale":"2024-04-25T09:08:37.538678+0000","last_undegraded":"2024-04-25T09:08:37.538678+0000","last_fullsized":"2024-04-25T09:08:37.538678+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.513023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.537794+0000","last_change":"2024-04-25T09:08:37.530659+0000","last_active":"2024-04-25T09:08:37.537794+0000","last_peered":"2024-04-25T09:08:37.537794+0000","last_clean":"2024-04-25T09:08:37.537794+0000","last_became_active":"2024-04-25T09:08:37.530327+0000","last_became_peered":"2024-04-25T09:08:37.530327+0000","last_unstale":"2024-04-25T09:08:37.537794+0000","last_undegraded":"2024-04-25T09:08:37.537794+0000","last_fullsized":"2024-04-25T09:08:37.537794+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:15.657509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457203+0000","last_change":"2024-04-25T09:08:37.521960+0000","last_active":"2024-04-25T09:08:39.457203+0000","last_peered":"2024-04-25T09:08:39.457203+0000","last_clean":"2024-04-25T09:08:39.457203+0000","last_became_active":"2024-04-25T09:08:37.521662+0000","last_became_peered":"2024-04-25T09:08:37.521662+0000","last_unstale":"2024-04-25T09:08:39.457203+0000","last_undegraded":"2024-04-25T09:08:39.457203+0000","last_fullsized":"2024-04-25T09:08:39.457203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:48.063497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.527367+0000","last_change":"2024-04-25T09:08:37.533396+0000","last_active":"2024-04-25T09:08:38.527367+0000","last_peered":"2024-04-25T09:08:38.527367+0000","last_clean":"2024-04-25T09:08:38.527367+0000","last_became_active":"2024-04-25T09:08:37.532697+0000","last_became_peered":"2024-04-25T09:08:37.532697+0000","last_unstale":"2024-04-25T09:08:38.527367+0000","last_undegraded":"2024-04-25T09:08:38.527367+0000","last_fullsized":"2024-04-25T09:08:38.527367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:31.246016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.529981+0000","last_change":"2024-04-25T09:08:37.525442+0000","last_active":"2024-04-25T09:08:38.529981+0000","last_peered":"2024-04-25T09:08:38.529981+0000","last_clean":"2024-04-25T09:08:38.529981+0000","last_became_active":"2024-04-25T09:08:37.517257+0000","last_became_peered":"2024-04-25T09:08:37.517257+0000","last_unstale":"2024-04-25T09:08:38.529981+0000","last_undegraded":"2024-04-25T09:08:38.529981+0000","last_fullsized":"2024-04-25T09:08:38.529981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:20.010867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090468+0000","last_change":"2024-04-25T09:08:37.537314+0000","last_active":"2024-04-25T09:08:39.090468+0000","last_peered":"2024-04-25T09:08:39.090468+0000","last_clean":"2024-04-25T09:08:39.090468+0000","last_became_active":"2024-04-25T09:08:37.537017+0000","last_became_peered":"2024-04-25T09:08:37.537017+0000","last_unstale":"2024-04-25T09:08:39.090468+0000","last_undegraded":"2024-04-25T09:08:39.090468+0000","last_fullsized":"2024-04-25T09:08:39.090468+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:38.375919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.531141+0000","last_change":"2024-04-25T09:08:36.500544+0000","last_active":"2024-04-25T09:08:38.531141+0000","last_peered":"2024-04-25T09:08:38.531141+0000","last_clean":"2024-04-25T09:08:38.531141+0000","last_became_active":"2024-04-25T09:08:36.499491+0000","last_became_peered":"2024-04-25T09:08:36.499491+0000","last_unstale":"2024-04-25T09:08:38.531141+0000","last_undegraded":"2024-04-25T09:08:38.531141+0000","last_fullsized":"2024-04-25T09:08:38.531141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:37.129703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090811+0000","last_change":"2024-04-25T09:08:37.548451+0000","last_active":"2024-04-25T09:08:39.090811+0000","last_peered":"2024-04-25T09:08:39.090811+0000","last_clean":"2024-04-25T09:08:39.090811+0000","last_became_active":"2024-04-25T09:08:37.548168+0000","last_became_peered":"2024-04-25T09:08:37.548168+0000","last_unstale":"2024-04-25T09:08:39.090811+0000","last_undegraded":"2024-04-25T09:08:39.090811+0000","last_fullsized":"2024-04-25T09:08:39.090811+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:24.649676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.528564+0000","last_change":"2024-04-25T09:08:36.508508+0000","last_active":"2024-04-25T09:08:38.528564+0000","last_peered":"2024-04-25T09:08:38.528564+0000","last_clean":"2024-04-25T09:08:38.528564+0000","last_became_active":"2024-04-25T09:08:36.508011+0000","last_became_peered":"2024-04-25T09:08:36.508011+0000","last_unstale":"2024-04-25T09:08:38.528564+0000","last_undegraded":"2024-04-25T09:08:38.528564+0000","last_fullsized":"2024-04-25T09:08:38.528564+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:02.287123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.549007+0000","last_change":"2024-04-25T09:08:37.547529+0000","last_active":"2024-04-25T09:08:37.549007+0000","last_peered":"2024-04-25T09:08:37.549007+0000","last_clean":"2024-04-25T09:08:37.549007+0000","last_became_active":"2024-04-25T09:08:37.545384+0000","last_became_peered":"2024-04-25T09:08:37.545384+0000","last_unstale":"2024-04-25T09:08:37.549007+0000","last_undegraded":"2024-04-25T09:08:37.549007+0000","last_fullsized":"2024-04-25T09:08:37.549007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:33.794566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090073+0000","last_change":"2024-04-25T09:08:36.513694+0000","last_active":"2024-04-25T09:08:39.090073+0000","last_peered":"2024-04-25T09:08:39.090073+0000","last_clean":"2024-04-25T09:08:39.090073+0000","last_became_active":"2024-04-25T09:08:36.513388+0000","last_became_peered":"2024-04-25T09:08:36.513388+0000","last_unstale":"2024-04-25T09:08:39.090073+0000","last_undegraded":"2024-04-25T09:08:39.090073+0000","last_fullsized":"2024-04-25T09:08:39.090073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:47.701092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.089999+0000","last_change":"2024-04-25T09:08:37.545084+0000","last_active":"2024-04-25T09:08:39.089999+0000","last_peered":"2024-04-25T09:08:39.089999+0000","last_clean":"2024-04-25T09:08:39.089999+0000","last_became_active":"2024-04-25T09:08:37.544206+0000","last_became_peered":"2024-04-25T09:08:37.544206+0000","last_unstale":"2024-04-25T09:08:39.089999+0000","last_undegraded":"2024-04-25T09:08:39.089999+0000","last_fullsized":"2024-04-25T09:08:39.089999+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:20.010867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457715+0000","last_change":"2024-04-25T09:08:36.496813+0000","last_active":"2024-04-25T09:08:39.457715+0000","last_peered":"2024-04-25T09:08:39.457715+0000","last_clean":"2024-04-25T09:08:39.457715+0000","last_became_active":"2024-04-25T09:08:36.496467+0000","last_became_peered":"2024-04-25T09:08:36.496467+0000","last_unstale":"2024-04-25T09:08:39.457715+0000","last_undegraded":"2024-04-25T09:08:39.457715+0000","last_fullsized":"2024-04-25T09:08:39.457715+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:26.107164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.538637+0000","last_change":"2024-04-25T09:08:37.535602+0000","last_active":"2024-04-25T09:08:37.538637+0000","last_peered":"2024-04-25T09:08:37.538637+0000","last_clean":"2024-04-25T09:08:37.538637+0000","last_became_active":"2024-04-25T09:08:37.535245+0000","last_became_peered":"2024-04-25T09:08:37.535245+0000","last_unstale":"2024-04-25T09:08:37.538637+0000","last_undegraded":"2024-04-25T09:08:37.538637+0000","last_fullsized":"2024-04-25T09:08:37.538637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:24.099039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.515595+0000","last_change":"2024-04-25T09:08:36.500593+0000","last_active":"2024-04-25T09:08:39.515595+0000","last_peered":"2024-04-25T09:08:39.515595+0000","last_clean":"2024-04-25T09:08:39.515595+0000","last_became_active":"2024-04-25T09:08:36.500302+0000","last_became_peered":"2024-04-25T09:08:36.500302+0000","last_unstale":"2024-04-25T09:08:39.515595+0000","last_undegraded":"2024-04-25T09:08:39.515595+0000","last_fullsized":"2024-04-25T09:08:39.515595+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:44.678969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.530788+0000","last_change":"2024-04-25T09:08:37.538047+0000","last_active":"2024-04-25T09:08:38.530788+0000","last_peered":"2024-04-25T09:08:38.530788+0000","last_clean":"2024-04-25T09:08:38.530788+0000","last_became_active":"2024-04-25T09:08:37.536066+0000","last_became_peered":"2024-04-25T09:08:37.536066+0000","last_unstale":"2024-04-25T09:08:38.530788+0000","last_undegraded":"2024-04-25T09:08:38.530788+0000","last_fullsized":"2024-04-25T09:08:38.530788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:15.657509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090618+0000","last_change":"2024-04-25T09:08:36.512217+0000","last_active":"2024-04-25T09:08:39.090618+0000","last_peered":"2024-04-25T09:08:39.090618+0000","last_clean":"2024-04-25T09:08:39.090618+0000","last_became_active":"2024-04-25T09:08:36.510668+0000","last_became_peered":"2024-04-25T09:08:36.510668+0000","last_unstale":"2024-04-25T09:08:39.090618+0000","last_undegraded":"2024-04-25T09:08:39.090618+0000","last_fullsized":"2024-04-25T09:08:39.090618+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:42.382540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.515245+0000","last_change":"2024-04-25T09:08:37.530118+0000","last_active":"2024-04-25T09:08:39.515245+0000","last_peered":"2024-04-25T09:08:39.515245+0000","last_clean":"2024-04-25T09:08:39.515245+0000","last_became_active":"2024-04-25T09:08:37.529851+0000","last_became_peered":"2024-04-25T09:08:37.529851+0000","last_unstale":"2024-04-25T09:08:39.515245+0000","last_undegraded":"2024-04-25T09:08:39.515245+0000","last_fullsized":"2024-04-25T09:08:39.515245+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:15.657509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.508712+0000","last_change":"2024-04-25T09:08:36.506944+0000","last_active":"2024-04-25T09:08:37.508712+0000","last_peered":"2024-04-25T09:08:37.508712+0000","last_clean":"2024-04-25T09:08:37.508712+0000","last_became_active":"2024-04-25T09:08:36.506321+0000","last_became_peered":"2024-04-25T09:08:36.506321+0000","last_unstale":"2024-04-25T09:08:37.508712+0000","last_undegraded":"2024-04-25T09:08:37.508712+0000","last_fullsized":"2024-04-25T09:08:37.508712+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:44.533991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.089413+0000","last_change":"2024-04-25T09:08:37.532582+0000","last_active":"2024-04-25T09:08:39.089413+0000","last_peered":"2024-04-25T09:08:39.089413+0000","last_clean":"2024-04-25T09:08:39.089413+0000","last_became_active":"2024-04-25T09:08:37.531982+0000","last_became_peered":"2024-04-25T09:08:37.531982+0000","last_unstale":"2024-04-25T09:08:39.089413+0000","last_undegraded":"2024-04-25T09:08:39.089413+0000","last_fullsized":"2024-04-25T09:08:39.089413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.903986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.506961+0000","last_change":"2024-04-25T09:08:36.508213+0000","last_active":"2024-04-25T09:08:37.506961+0000","last_peered":"2024-04-25T09:08:37.506961+0000","last_clean":"2024-04-25T09:08:37.506961+0000","last_became_active":"2024-04-25T09:08:36.507922+0000","last_became_peered":"2024-04-25T09:08:36.507922+0000","last_unstale":"2024-04-25T09:08:37.506961+0000","last_undegraded":"2024-04-25T09:08:37.506961+0000","last_fullsized":"2024-04-25T09:08:37.506961+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:33.020930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.528836+0000","last_change":"2024-04-25T09:08:37.525744+0000","last_active":"2024-04-25T09:08:37.528836+0000","last_peered":"2024-04-25T09:08:37.528836+0000","last_clean":"2024-04-25T09:08:37.528836+0000","last_became_active":"2024-04-25T09:08:37.525435+0000","last_became_peered":"2024-04-25T09:08:37.525435+0000","last_unstale":"2024-04-25T09:08:37.528836+0000","last_undegraded":"2024-04-25T09:08:37.528836+0000","last_fullsized":"2024-04-25T09:08:37.528836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.693822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.512178+0000","last_change":"2024-04-25T09:08:36.508278+0000","last_active":"2024-04-25T09:08:37.512178+0000","last_peered":"2024-04-25T09:08:37.512178+0000","last_clean":"2024-04-25T09:08:37.512178+0000","last_became_active":"2024-04-25T09:08:36.508007+0000","last_became_peered":"2024-04-25T09:08:36.508007+0000","last_unstale":"2024-04-25T09:08:37.512178+0000","last_undegraded":"2024-04-25T09:08:37.512178+0000","last_fullsized":"2024-04-25T09:08:37.512178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:54.430965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.516553+0000","last_change":"2024-04-25T09:08:37.536774+0000","last_active":"2024-04-25T09:08:39.516553+0000","last_peered":"2024-04-25T09:08:39.516553+0000","last_clean":"2024-04-25T09:08:39.516553+0000","last_became_active":"2024-04-25T09:08:37.536450+0000","last_became_peered":"2024-04-25T09:08:37.536450+0000","last_unstale":"2024-04-25T09:08:39.516553+0000","last_undegraded":"2024-04-25T09:08:39.516553+0000","last_fullsized":"2024-04-25T09:08:39.516553+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.513023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.527216+0000","last_change":"2024-04-25T09:08:36.515555+0000","last_active":"2024-04-25T09:08:37.527216+0000","last_peered":"2024-04-25T09:08:37.527216+0000","last_clean":"2024-04-25T09:08:37.527216+0000","last_became_active":"2024-04-25T09:08:36.515254+0000","last_became_peered":"2024-04-25T09:08:36.515254+0000","last_unstale":"2024-04-25T09:08:37.527216+0000","last_undegraded":"2024-04-25T09:08:37.527216+0000","last_fullsized":"2024-04-25T09:08:37.527216+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:03.723024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.548488+0000","last_change":"2024-04-25T09:08:37.545345+0000","last_active":"2024-04-25T09:08:37.548488+0000","last_peered":"2024-04-25T09:08:37.548488+0000","last_clean":"2024-04-25T09:08:37.548488+0000","last_became_active":"2024-04-25T09:08:37.545065+0000","last_became_peered":"2024-04-25T09:08:37.545065+0000","last_unstale":"2024-04-25T09:08:37.548488+0000","last_undegraded":"2024-04-25T09:08:37.548488+0000","last_fullsized":"2024-04-25T09:08:37.548488+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:27.916058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457519+0000","last_change":"2024-04-25T09:08:36.497005+0000","last_active":"2024-04-25T09:08:39.457519+0000","last_peered":"2024-04-25T09:08:39.457519+0000","last_clean":"2024-04-25T09:08:39.457519+0000","last_became_active":"2024-04-25T09:08:36.496569+0000","last_became_peered":"2024-04-25T09:08:36.496569+0000","last_unstale":"2024-04-25T09:08:39.457519+0000","last_undegraded":"2024-04-25T09:08:39.457519+0000","last_fullsized":"2024-04-25T09:08:39.457519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:08.128437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457263+0000","last_change":"2024-04-25T09:08:37.536701+0000","last_active":"2024-04-25T09:08:39.457263+0000","last_peered":"2024-04-25T09:08:39.457263+0000","last_clean":"2024-04-25T09:08:39.457263+0000","last_became_active":"2024-04-25T09:08:37.536368+0000","last_became_peered":"2024-04-25T09:08:37.536368+0000","last_unstale":"2024-04-25T09:08:39.457263+0000","last_undegraded":"2024-04-25T09:08:39.457263+0000","last_fullsized":"2024-04-25T09:08:39.457263+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:54.684674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.531355+0000","last_change":"2024-04-25T09:08:36.505886+0000","last_active":"2024-04-25T09:08:37.531355+0000","last_peered":"2024-04-25T09:08:37.531355+0000","last_clean":"2024-04-25T09:08:37.531355+0000","last_became_active":"2024-04-25T09:08:36.505598+0000","last_became_peered":"2024-04-25T09:08:36.505598+0000","last_unstale":"2024-04-25T09:08:37.531355+0000","last_undegraded":"2024-04-25T09:08:37.531355+0000","last_fullsized":"2024-04-25T09:08:37.531355+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:04.447718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.539724+0000","last_change":"2024-04-25T09:08:37.537466+0000","last_active":"2024-04-25T09:08:37.539724+0000","last_peered":"2024-04-25T09:08:37.539724+0000","last_clean":"2024-04-25T09:08:37.539724+0000","last_became_active":"2024-04-25T09:08:37.537184+0000","last_became_peered":"2024-04-25T09:08:37.537184+0000","last_unstale":"2024-04-25T09:08:37.539724+0000","last_undegraded":"2024-04-25T09:08:37.539724+0000","last_fullsized":"2024-04-25T09:08:37.539724+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:03.301975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.541480+0000","last_change":"2024-04-25T09:08:36.512515+0000","last_active":"2024-04-25T09:08:37.541480+0000","last_peered":"2024-04-25T09:08:37.541480+0000","last_clean":"2024-04-25T09:08:37.541480+0000","last_became_active":"2024-04-25T09:08:36.511991+0000","last_became_peered":"2024-04-25T09:08:36.511991+0000","last_unstale":"2024-04-25T09:08:37.541480+0000","last_undegraded":"2024-04-25T09:08:37.541480+0000","last_fullsized":"2024-04-25T09:08:37.541480+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:45.312339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.515743+0000","last_change":"2024-04-25T09:08:37.518223+0000","last_active":"2024-04-25T09:08:39.515743+0000","last_peered":"2024-04-25T09:08:39.515743+0000","last_clean":"2024-04-25T09:08:39.515743+0000","last_became_active":"2024-04-25T09:08:37.516318+0000","last_became_peered":"2024-04-25T09:08:37.516318+0000","last_unstale":"2024-04-25T09:08:39.515743+0000","last_undegraded":"2024-04-25T09:08:39.515743+0000","last_fullsized":"2024-04-25T09:08:39.515743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:24.649676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.456803+0000","last_change":"2024-04-25T09:08:36.503609+0000","last_active":"2024-04-25T09:08:39.456803+0000","last_peered":"2024-04-25T09:08:39.456803+0000","last_clean":"2024-04-25T09:08:39.456803+0000","last_became_active":"2024-04-25T09:08:36.503095+0000","last_became_peered":"2024-04-25T09:08:36.503095+0000","last_unstale":"2024-04-25T09:08:39.456803+0000","last_undegraded":"2024-04-25T09:08:39.456803+0000","last_fullsized":"2024-04-25T09:08:39.456803+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:01.637523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.515863+0000","last_change":"2024-04-25T09:08:37.514053+0000","last_active":"2024-04-25T09:08:37.515863+0000","last_peered":"2024-04-25T09:08:37.515863+0000","last_clean":"2024-04-25T09:08:37.515863+0000","last_became_active":"2024-04-25T09:08:37.513436+0000","last_became_peered":"2024-04-25T09:08:37.513436+0000","last_unstale":"2024-04-25T09:08:37.515863+0000","last_undegraded":"2024-04-25T09:08:37.515863+0000","last_fullsized":"2024-04-25T09:08:37.515863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:21.891306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457670+0000","last_change":"2024-04-25T09:08:36.494637+0000","last_active":"2024-04-25T09:08:39.457670+0000","last_peered":"2024-04-25T09:08:39.457670+0000","last_clean":"2024-04-25T09:08:39.457670+0000","last_became_active":"2024-04-25T09:08:36.494256+0000","last_became_peered":"2024-04-25T09:08:36.494256+0000","last_unstale":"2024-04-25T09:08:39.457670+0000","last_undegraded":"2024-04-25T09:08:39.457670+0000","last_fullsized":"2024-04-25T09:08:39.457670+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:30.631972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457606+0000","last_change":"2024-04-25T09:08:37.535307+0000","last_active":"2024-04-25T09:08:39.457606+0000","last_peered":"2024-04-25T09:08:39.457606+0000","last_clean":"2024-04-25T09:08:39.457606+0000","last_became_active":"2024-04-25T09:08:37.534964+0000","last_became_peered":"2024-04-25T09:08:37.534964+0000","last_unstale":"2024-04-25T09:08:39.457606+0000","last_undegraded":"2024-04-25T09:08:39.457606+0000","last_fullsized":"2024-04-25T09:08:39.457606+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:54.822610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":22,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.456889+0000","last_change":"2024-04-25T09:08:36.503639+0000","last_active":"2024-04-25T09:08:39.456889+0000","last_peered":"2024-04-25T09:08:39.456889+0000","last_clean":"2024-04-25T09:08:39.456889+0000","last_became_active":"2024-04-25T09:08:36.503201+0000","last_became_peered":"2024-04-25T09:08:36.503201+0000","last_unstale":"2024-04-25T09:08:39.456889+0000","last_undegraded":"2024-04-25T09:08:39.456889+0000","last_fullsized":"2024-04-25T09:08:39.456889+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:12.622134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.516753+0000","last_change":"2024-04-25T09:08:37.512915+0000","last_active":"2024-04-25T09:08:39.516753+0000","last_peered":"2024-04-25T09:08:39.516753+0000","last_clean":"2024-04-25T09:08:39.516753+0000","last_became_active":"2024-04-25T09:08:37.512611+0000","last_became_peered":"2024-04-25T09:08:37.512611+0000","last_unstale":"2024-04-25T09:08:39.516753+0000","last_undegraded":"2024-04-25T09:08:39.516753+0000","last_fullsized":"2024-04-25T09:08:39.516753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:26.865276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.528369+0000","last_change":"2024-04-25T09:08:36.515975+0000","last_active":"2024-04-25T09:08:38.528369+0000","last_peered":"2024-04-25T09:08:38.528369+0000","last_clean":"2024-04-25T09:08:38.528369+0000","last_became_active":"2024-04-25T09:08:36.515757+0000","last_became_peered":"2024-04-25T09:08:36.515757+0000","last_unstale":"2024-04-25T09:08:38.528369+0000","last_undegraded":"2024-04-25T09:08:38.528369+0000","last_fullsized":"2024-04-25T09:08:38.528369+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:54.430965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.516926+0000","last_change":"2024-04-25T09:08:37.538458+0000","last_active":"2024-04-25T09:08:39.516926+0000","last_peered":"2024-04-25T09:08:39.516926+0000","last_clean":"2024-04-25T09:08:39.516926+0000","last_became_active":"2024-04-25T09:08:37.538153+0000","last_became_peered":"2024-04-25T09:08:37.538153+0000","last_unstale":"2024-04-25T09:08:39.516926+0000","last_undegraded":"2024-04-25T09:08:39.516926+0000","last_fullsized":"2024-04-25T09:08:39.516926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:18.290938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.541492+0000","last_change":"2024-04-25T09:08:36.514255+0000","last_active":"2024-04-25T09:08:37.541492+0000","last_peered":"2024-04-25T09:08:37.541492+0000","last_clean":"2024-04-25T09:08:37.541492+0000","last_became_active":"2024-04-25T09:08:36.513963+0000","last_became_peered":"2024-04-25T09:08:36.513963+0000","last_unstale":"2024-04-25T09:08:37.541492+0000","last_undegraded":"2024-04-25T09:08:37.541492+0000","last_fullsized":"2024-04-25T09:08:37.541492+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:37.361067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.528121+0000","last_change":"2024-04-25T09:08:37.535650+0000","last_active":"2024-04-25T09:08:38.528121+0000","last_peered":"2024-04-25T09:08:38.528121+0000","last_clean":"2024-04-25T09:08:38.528121+0000","last_became_active":"2024-04-25T09:08:37.535041+0000","last_became_peered":"2024-04-25T09:08:37.535041+0000","last_unstale":"2024-04-25T09:08:38.528121+0000","last_undegraded":"2024-04-25T09:08:38.528121+0000","last_fullsized":"2024-04-25T09:08:38.528121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:40.285121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.516060+0000","last_change":"2024-04-25T09:08:36.512778+0000","last_active":"2024-04-25T09:08:39.516060+0000","last_peered":"2024-04-25T09:08:39.516060+0000","last_clean":"2024-04-25T09:08:39.516060+0000","last_became_active":"2024-04-25T09:08:36.512527+0000","last_became_peered":"2024-04-25T09:08:36.512527+0000","last_unstale":"2024-04-25T09:08:39.516060+0000","last_undegraded":"2024-04-25T09:08:39.516060+0000","last_fullsized":"2024-04-25T09:08:39.516060+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:38.862281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.457870+0000","last_change":"2024-04-25T09:08:37.546410+0000","last_active":"2024-04-25T09:08:39.457870+0000","last_peered":"2024-04-25T09:08:39.457870+0000","last_clean":"2024-04-25T09:08:39.457870+0000","last_became_active":"2024-04-25T09:08:37.546001+0000","last_became_peered":"2024-04-25T09:08:37.546001+0000","last_unstale":"2024-04-25T09:08:39.457870+0000","last_undegraded":"2024-04-25T09:08:39.457870+0000","last_fullsized":"2024-04-25T09:08:39.457870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.874101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.520866+0000","last_change":"2024-04-25T09:08:36.514407+0000","last_active":"2024-04-25T09:08:37.520866+0000","last_peered":"2024-04-25T09:08:37.520866+0000","last_clean":"2024-04-25T09:08:37.520866+0000","last_became_active":"2024-04-25T09:08:36.514096+0000","last_became_peered":"2024-04-25T09:08:36.514096+0000","last_unstale":"2024-04-25T09:08:37.520866+0000","last_undegraded":"2024-04-25T09:08:37.520866+0000","last_fullsized":"2024-04-25T09:08:37.520866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:36.889170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.547624+0000","last_change":"2024-04-25T09:08:37.545250+0000","last_active":"2024-04-25T09:08:37.547624+0000","last_peered":"2024-04-25T09:08:37.547624+0000","last_clean":"2024-04-25T09:08:37.547624+0000","last_became_active":"2024-04-25T09:08:37.544950+0000","last_became_peered":"2024-04-25T09:08:37.544950+0000","last_unstale":"2024-04-25T09:08:37.547624+0000","last_undegraded":"2024-04-25T09:08:37.547624+0000","last_fullsized":"2024-04-25T09:08:37.547624+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:47.387734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.530498+0000","last_change":"2024-04-25T09:08:36.509235+0000","last_active":"2024-04-25T09:08:38.530498+0000","last_peered":"2024-04-25T09:08:38.530498+0000","last_clean":"2024-04-25T09:08:38.530498+0000","last_became_active":"2024-04-25T09:08:36.508901+0000","last_became_peered":"2024-04-25T09:08:36.508901+0000","last_unstale":"2024-04-25T09:08:38.530498+0000","last_undegraded":"2024-04-25T09:08:38.530498+0000","last_fullsized":"2024-04-25T09:08:38.530498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:44.678969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.537835+0000","last_change":"2024-04-25T09:08:37.532273+0000","last_active":"2024-04-25T09:08:37.537835+0000","last_peered":"2024-04-25T09:08:37.537835+0000","last_clean":"2024-04-25T09:08:37.537835+0000","last_became_active":"2024-04-25T09:08:37.531792+0000","last_became_peered":"2024-04-25T09:08:37.531792+0000","last_unstale":"2024-04-25T09:08:37.537835+0000","last_undegraded":"2024-04-25T09:08:37.537835+0000","last_fullsized":"2024-04-25T09:08:37.537835+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:24.649676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.527121+0000","last_change":"2024-04-25T09:08:36.514938+0000","last_active":"2024-04-25T09:08:37.527121+0000","last_peered":"2024-04-25T09:08:37.527121+0000","last_clean":"2024-04-25T09:08:37.527121+0000","last_became_active":"2024-04-25T09:08:36.514601+0000","last_became_peered":"2024-04-25T09:08:36.514601+0000","last_unstale":"2024-04-25T09:08:37.527121+0000","last_undegraded":"2024-04-25T09:08:37.527121+0000","last_fullsized":"2024-04-25T09:08:37.527121+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:18.996015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.538669+0000","last_change":"2024-04-25T09:08:37.534286+0000","last_active":"2024-04-25T09:08:37.538669+0000","last_peered":"2024-04-25T09:08:37.538669+0000","last_clean":"2024-04-25T09:08:37.538669+0000","last_became_active":"2024-04-25T09:08:37.533969+0000","last_became_peered":"2024-04-25T09:08:37.533969+0000","last_unstale":"2024-04-25T09:08:37.538669+0000","last_undegraded":"2024-04-25T09:08:37.538669+0000","last_fullsized":"2024-04-25T09:08:37.538669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.074048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.510344+0000","last_change":"2024-04-25T09:08:36.513857+0000","last_active":"2024-04-25T09:08:37.510344+0000","last_peered":"2024-04-25T09:08:37.510344+0000","last_clean":"2024-04-25T09:08:37.510344+0000","last_became_active":"2024-04-25T09:08:36.513416+0000","last_became_peered":"2024-04-25T09:08:36.513416+0000","last_unstale":"2024-04-25T09:08:37.510344+0000","last_undegraded":"2024-04-25T09:08:37.510344+0000","last_fullsized":"2024-04-25T09:08:37.510344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:50.663414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.517070+0000","last_change":"2024-04-25T09:08:37.532633+0000","last_active":"2024-04-25T09:08:39.517070+0000","last_peered":"2024-04-25T09:08:39.517070+0000","last_clean":"2024-04-25T09:08:39.517070+0000","last_became_active":"2024-04-25T09:08:37.532285+0000","last_became_peered":"2024-04-25T09:08:37.532285+0000","last_unstale":"2024-04-25T09:08:39.517070+0000","last_undegraded":"2024-04-25T09:08:39.517070+0000","last_fullsized":"2024-04-25T09:08:39.517070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:33.794566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.528683+0000","last_change":"2024-04-25T09:08:36.510736+0000","last_active":"2024-04-25T09:08:38.528683+0000","last_peered":"2024-04-25T09:08:38.528683+0000","last_clean":"2024-04-25T09:08:38.528683+0000","last_became_active":"2024-04-25T09:08:36.510419+0000","last_became_peered":"2024-04-25T09:08:36.510419+0000","last_unstale":"2024-04-25T09:08:38.528683+0000","last_undegraded":"2024-04-25T09:08:38.528683+0000","last_fullsized":"2024-04-25T09:08:38.528683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:20.876454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.517032+0000","last_change":"2024-04-25T09:08:37.529083+0000","last_active":"2024-04-25T09:08:39.517032+0000","last_peered":"2024-04-25T09:08:39.517032+0000","last_clean":"2024-04-25T09:08:39.517032+0000","last_became_active":"2024-04-25T09:08:37.526802+0000","last_became_peered":"2024-04-25T09:08:37.526802+0000","last_unstale":"2024-04-25T09:08:39.517032+0000","last_undegraded":"2024-04-25T09:08:39.517032+0000","last_fullsized":"2024-04-25T09:08:39.517032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:27.916058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"62'1","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090403+0000","last_change":"2024-04-25T09:08:36.510464+0000","last_active":"2024-04-25T09:08:39.090403+0000","last_peered":"2024-04-25T09:08:39.090403+0000","last_clean":"2024-04-25T09:08:39.090403+0000","last_became_active":"2024-04-25T09:08:36.510181+0000","last_became_peered":"2024-04-25T09:08:36.510181+0000","last_unstale":"2024-04-25T09:08:39.090403+0000","last_undegraded":"2024-04-25T09:08:39.090403+0000","last_fullsized":"2024-04-25T09:08:39.090403+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:04.447718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.529237+0000","last_change":"2024-04-25T09:08:37.535829+0000","last_active":"2024-04-25T09:08:38.529237+0000","last_peered":"2024-04-25T09:08:38.529237+0000","last_clean":"2024-04-25T09:08:38.529237+0000","last_became_active":"2024-04-25T09:08:37.535543+0000","last_became_peered":"2024-04-25T09:08:37.535543+0000","last_unstale":"2024-04-25T09:08:38.529237+0000","last_undegraded":"2024-04-25T09:08:38.529237+0000","last_fullsized":"2024-04-25T09:08:38.529237+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:48.715944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.511408+0000","last_change":"2024-04-25T09:08:36.494162+0000","last_active":"2024-04-25T09:08:37.511408+0000","last_peered":"2024-04-25T09:08:37.511408+0000","last_clean":"2024-04-25T09:08:37.511408+0000","last_became_active":"2024-04-25T09:08:36.493892+0000","last_became_peered":"2024-04-25T09:08:36.493892+0000","last_unstale":"2024-04-25T09:08:37.511408+0000","last_undegraded":"2024-04-25T09:08:37.511408+0000","last_fullsized":"2024-04-25T09:08:37.511408+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:01.565198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.542636+0000","last_change":"2024-04-25T09:08:37.537372+0000","last_active":"2024-04-25T09:08:37.542636+0000","last_peered":"2024-04-25T09:08:37.542636+0000","last_clean":"2024-04-25T09:08:37.542636+0000","last_became_active":"2024-04-25T09:08:37.537028+0000","last_became_peered":"2024-04-25T09:08:37.537028+0000","last_unstale":"2024-04-25T09:08:37.542636+0000","last_undegraded":"2024-04-25T09:08:37.542636+0000","last_fullsized":"2024-04-25T09:08:37.542636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:26.865276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.515482+0000","last_change":"2024-04-25T09:08:36.503258+0000","last_active":"2024-04-25T09:08:39.515482+0000","last_peered":"2024-04-25T09:08:39.515482+0000","last_clean":"2024-04-25T09:08:39.515482+0000","last_became_active":"2024-04-25T09:08:36.502865+0000","last_became_peered":"2024-04-25T09:08:36.502865+0000","last_unstale":"2024-04-25T09:08:39.515482+0000","last_undegraded":"2024-04-25T09:08:39.515482+0000","last_fullsized":"2024-04-25T09:08:39.515482+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:08.830733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.090919+0000","last_change":"2024-04-25T09:08:37.542854+0000","last_active":"2024-04-25T09:08:39.090919+0000","last_peered":"2024-04-25T09:08:39.090919+0000","last_clean":"2024-04-25T09:08:39.090919+0000","last_became_active":"2024-04-25T09:08:37.542150+0000","last_became_peered":"2024-04-25T09:08:37.542150+0000","last_unstale":"2024-04-25T09:08:39.090919+0000","last_undegraded":"2024-04-25T09:08:39.090919+0000","last_fullsized":"2024-04-25T09:08:39.090919+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:50.074048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.527856+0000","last_change":"2024-04-25T09:08:36.516827+0000","last_active":"2024-04-25T09:08:38.527856+0000","last_peered":"2024-04-25T09:08:38.527856+0000","last_clean":"2024-04-25T09:08:38.527856+0000","last_became_active":"2024-04-25T09:08:36.516429+0000","last_became_peered":"2024-04-25T09:08:36.516429+0000","last_unstale":"2024-04-25T09:08:38.527856+0000","last_undegraded":"2024-04-25T09:08:38.527856+0000","last_fullsized":"2024-04-25T09:08:38.527856+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:51.207593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.540744+0000","last_change":"2024-04-25T09:08:37.538645+0000","last_active":"2024-04-25T09:08:37.540744+0000","last_peered":"2024-04-25T09:08:37.540744+0000","last_clean":"2024-04-25T09:08:37.540744+0000","last_became_active":"2024-04-25T09:08:37.537131+0000","last_became_peered":"2024-04-25T09:08:37.537131+0000","last_unstale":"2024-04-25T09:08:37.540744+0000","last_undegraded":"2024-04-25T09:08:37.540744+0000","last_fullsized":"2024-04-25T09:08:37.540744+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.903986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.515340+0000","last_change":"2024-04-25T09:08:36.512804+0000","last_active":"2024-04-25T09:08:39.515340+0000","last_peered":"2024-04-25T09:08:39.515340+0000","last_clean":"2024-04-25T09:08:39.515340+0000","last_became_active":"2024-04-25T09:08:36.512575+0000","last_became_peered":"2024-04-25T09:08:36.512575+0000","last_unstale":"2024-04-25T09:08:39.515340+0000","last_undegraded":"2024-04-25T09:08:39.515340+0000","last_fullsized":"2024-04-25T09:08:39.515340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:33.020930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.538430+0000","last_change":"2024-04-25T09:08:37.534338+0000","last_active":"2024-04-25T09:08:37.538430+0000","last_peered":"2024-04-25T09:08:37.538430+0000","last_clean":"2024-04-25T09:08:37.538430+0000","last_became_active":"2024-04-25T09:08:37.534045+0000","last_became_peered":"2024-04-25T09:08:37.534045+0000","last_unstale":"2024-04-25T09:08:37.538430+0000","last_undegraded":"2024-04-25T09:08:37.538430+0000","last_fullsized":"2024-04-25T09:08:37.538430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:58.433170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:39.089190+0000","last_change":"2024-04-25T09:08:36.514608+0000","last_active":"2024-04-25T09:08:39.089190+0000","last_peered":"2024-04-25T09:08:39.089190+0000","last_clean":"2024-04-25T09:08:39.089190+0000","last_became_active":"2024-04-25T09:08:36.513338+0000","last_became_peered":"2024-04-25T09:08:36.513338+0000","last_unstale":"2024-04-25T09:08:39.089190+0000","last_undegraded":"2024-04-25T09:08:39.089190+0000","last_fullsized":"2024-04-25T09:08:39.089190+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:39.270268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-25T09:08:38.527197+0000","last_change":"2024-04-25T09:08:37.533988+0000","last_active":"2024-04-25T09:08:38.527197+0000","last_peered":"2024-04-25T09:08:38.527197+0000","last_clean":"2024-04-25T09:08:38.527197+0000","last_became_active":"2024-04-25T09:08:37.533611+0000","last_became_peered":"2024-04-25T09:08:37.533611+0000","last_unstale":"2024-04-25T09:08:38.527197+0000","last_undegraded":"2024-04-25T09:08:38.527197+0000","last_fullsized":"2024-04-25T09:08:38.527197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:04.900228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.519709+0000","last_change":"2024-04-25T09:08:36.510900+0000","last_active":"2024-04-25T09:08:37.519709+0000","last_peered":"2024-04-25T09:08:37.519709+0000","last_clean":"2024-04-25T09:08:37.519709+0000","last_became_active":"2024-04-25T09:08:36.510571+0000","last_became_peered":"2024-04-25T09:08:36.510571+0000","last_unstale":"2024-04-25T09:08:37.519709+0000","last_undegraded":"2024-04-25T09:08:37.519709+0000","last_fullsized":"2024-04-25T09:08:37.519709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.241651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.544977+0000","last_change":"2024-04-25T09:08:37.540006+0000","last_active":"2024-04-25T09:08:37.544977+0000","last_peered":"2024-04-25T09:08:37.544977+0000","last_clean":"2024-04-25T09:08:37.544977+0000","last_became_active":"2024-04-25T09:08:37.539754+0000","last_became_peered":"2024-04-25T09:08:37.539754+0000","last_unstale":"2024-04-25T09:08:37.544977+0000","last_undegraded":"2024-04-25T09:08:37.544977+0000","last_fullsized":"2024-04-25T09:08:37.544977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:36.468102+0000","last_clean_scrub_stamp":"2024-04-25T09:08:36.468102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:18.290938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.505895+0000","last_change":"2024-04-25T09:08:36.502652+0000","last_active":"2024-04-25T09:08:37.505895+0000","last_peered":"2024-04-25T09:08:37.505895+0000","last_clean":"2024-04-25T09:08:37.505895+0000","last_became_active":"2024-04-25T09:08:36.502345+0000","last_became_peered":"2024-04-25T09:08:36.502345+0000","last_unstale":"2024-04-25T09:08:37.505895+0000","last_undegraded":"2024-04-25T09:08:37.505895+0000","last_fullsized":"2024-04-25T09:08:37.505895+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:08:35.453249+0000","last_clean_scrub_stamp":"2024-04-25T09:08:35.453249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:17.474502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":244,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T09:08:37.510446+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:08:37.510446+0000","last_peered":"2024-04-25T09:08:37.510446+0000","last_clean":"2024-04-25T09:08:37.510446+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:08:37.510446+0000","last_undegraded":"2024-04-25T09:08:37.510446+0000","last_fullsized":"2024-04-25T09:08:37.510446+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36084,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":753664,"data_stored":315644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40232,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331608,"statfs":{"total":96636764160,"available":96595566592,"internally_reserved":0,"allocated":749568,"data_stored":315574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40800,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331040,"statfs":{"total":96636764160,"available":96594984960,"internally_reserved":0,"allocated":1331200,"data_stored":899654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40240,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331600,"statfs":{"total":96636764160,"available":96595558400,"internally_reserved":0,"allocated":753664,"data_stored":316168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40240,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331600,"statfs":{"total":96636764160,"available":96595558400,"internally_reserved":0,"allocated":749568,"data_stored":315642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58296,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313544,"statfs":{"total":96636764160,"available":96577069056,"internally_reserved":0,"allocated":765952,"data_stored":317324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40804,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331036,"statfs":{"total":96636764160,"available":96594980864,"internally_reserved":0,"allocated":1331200,"data_stored":900202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36068,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335772,"statfs":{"total":96636764160,"available":96599830528,"internally_reserved":0,"allocated":745472,"data_stored":310864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:08:42.538 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:08:42.538 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:08:42.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.812+0000 7f2809e00640 1 Processor -- start 2024-04-25T09:08:42.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.812+0000 7f2809e00640 1 -- start start 2024-04-25T09:08:42.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.814+0000 7f2809e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f280415dd20 0x7f28040d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.814+0000 7f2809e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28040ceef0 0x7f28040b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.814+0000 7f2809e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280415e100 con 0x7f28040d34f0 2024-04-25T09:08:42.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.814+0000 7f2809e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28040b9a20 con 0x7f28040ceef0 2024-04-25T09:08:42.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.814+0000 7f2809e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040b7b50 con 0x7f280415dd20 2024-04-25T09:08:42.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.814+0000 7f2809400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28040ceef0 0x7f28040b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.814+0000 7f2808a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f280415dd20 0x7f28040d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.815+0000 7f2809400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f28040ceef0 0x7f28040b94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:48514/0 (socket says 172.21.15.62:48514) 2024-04-25T09:08:42.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.815+0000 7f2808a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f280415dd20 0x7f28040d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:60702/0 (socket says 172.21.15.62:60702) 2024-04-25T09:08:42.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.815+0000 7f2809400640 1 -- 172.21.15.62:0/1678564435 learned_addr learned my addr 172.21.15.62:0/1678564435 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:42.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.815+0000 7f2808a00640 1 -- 172.21.15.62:0/1678564435 >> v2:172.21.15.121:3300/0 conn(0x7f28040ceef0 msgr2=0x7f28040b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:42.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.815+0000 7f2808a00640 1 --2- 172.21.15.62:0/1678564435 >> v2:172.21.15.121:3300/0 conn(0x7f28040ceef0 0x7f28040b94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.815+0000 7f2808a00640 1 -- 172.21.15.62:0/1678564435 >> v1:172.21.15.121:6789/0 conn(0x7f28040d34f0 legacy=0x7f28040ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.815+0000 7f2808a00640 1 -- 172.21.15.62:0/1678564435 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040d2450 con 0x7f280415dd20 2024-04-25T09:08:42.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.815+0000 7f2809400640 1 --2- 172.21.15.62:0/1678564435 >> v2:172.21.15.121:3300/0 conn(0x7f28040ceef0 0x7f28040b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:08:42.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.817+0000 7f2808a00640 1 --2- 172.21.15.62:0/1678564435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f280415dd20 0x7f28040d1d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27f80097a0 tx=0x7f27f802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87a4f6b69ff10aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:42.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.817+0000 7f2803400640 1 -- 172.21.15.62:0/1678564435 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8037070 con 0x7f280415dd20 2024-04-25T09:08:42.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.817+0000 7f2803400640 1 -- 172.21.15.62:0/1678564435 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f8032440 con 0x7f280415dd20 2024-04-25T09:08:42.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2803400640 1 -- 172.21.15.62:0/1678564435 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8032810 con 0x7f280415dd20 2024-04-25T09:08:42.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2809e00640 1 -- 172.21.15.62:0/1678564435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f280415dd20 msgr2=0x7f28040d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2809e00640 1 --2- 172.21.15.62:0/1678564435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f280415dd20 0x7f28040d1d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27f80097a0 tx=0x7f27f802a180 comp rx=0 tx=0).stop 2024-04-25T09:08:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2809e00640 1 -- 172.21.15.62:0/1678564435 shutdown_connections 2024-04-25T09:08:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2809e00640 1 --2- 172.21.15.62:0/1678564435 >> v2:172.21.15.121:3300/0 conn(0x7f28040ceef0 0x7f28040b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2809e00640 1 --2- 172.21.15.62:0/1678564435 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f280415dd20 0x7f28040d1d70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2809e00640 1 -- 172.21.15.62:0/1678564435 >> 172.21.15.62:0/1678564435 conn(0x7f280406b000 msgr2=0x7f280406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2809e00640 1 -- 172.21.15.62:0/1678564435 shutdown_connections 2024-04-25T09:08:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.818+0000 7f2809e00640 1 -- 172.21.15.62:0/1678564435 wait complete. 2024-04-25T09:08:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.820+0000 7f2809e00640 1 Processor -- start 2024-04-25T09:08:42.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.820+0000 7f2809e00640 1 -- start start 2024-04-25T09:08:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.821+0000 7f2809e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28040ceef0 0x7f28041f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.821+0000 7f2809e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040d34f0 0x7f28040cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.821+0000 7f2809e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f280415dd20 0x7f2804158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.821+0000 7f2809e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040d2cf0 con 0x7f28040d34f0 2024-04-25T09:08:42.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.821+0000 7f2809e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040d2a10 con 0x7f280415dd20 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.821+0000 7f2809e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040b6b40 con 0x7f28040ceef0 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2803e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040d34f0 0x7f28040cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2803e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040d34f0 0x7f28040cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35256/0 (socket says 172.21.15.62:35256) 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2803e00640 1 -- 172.21.15.62:0/2624414566 learned_addr learned my addr 172.21.15.62:0/2624414566 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2808a00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28040ceef0 0x7f28041f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2809400640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f280415dd20 0x7f2804158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2803e00640 1 -- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28040ceef0 msgr2=0x7f28041f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2803e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28040ceef0 0x7f28041f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2803e00640 1 -- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f280415dd20 msgr2=0x7f2804158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2803e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f280415dd20 0x7f2804158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2803e00640 1 -- 172.21.15.62:0/2624414566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28041f6cb0 con 0x7f28040d34f0 2024-04-25T09:08:42.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.822+0000 7f2808a00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28040ceef0 0x7f28041f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.823+0000 7f2803e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040d34f0 0x7f28040cd300 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c470 tx=0x7f27f0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.823+0000 7f2800c00640 1 -- 172.21.15.62:0/2624414566 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f00052d0 con 0x7f28040d34f0 2024-04-25T09:08:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.823+0000 7f2800c00640 1 -- 172.21.15.62:0/2624414566 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f0005470 con 0x7f28040d34f0 2024-04-25T09:08:42.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.823+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28040cc260 con 0x7f28040d34f0 2024-04-25T09:08:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.823+0000 7f2800c00640 1 -- 172.21.15.62:0/2624414566 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0007260 con 0x7f28040d34f0 2024-04-25T09:08:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.823+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28040cc820 con 0x7f28040d34f0 2024-04-25T09:08:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.824+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2804075070 con 0x7f28040d34f0 2024-04-25T09:08:42.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.825+0000 7f2800c00640 1 -- 172.21.15.62:0/2624414566 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27f0007400 con 0x7f28040d34f0 2024-04-25T09:08:42.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.826+0000 7f2800c00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f27e406b500 0x7f27e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:42.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.826+0000 7f2800c00640 1 -- 172.21.15.62:0/2624414566 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27f00897c0 con 0x7f28040d34f0 2024-04-25T09:08:42.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.827+0000 7f2808a00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f27e406b500 0x7f27e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:42.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.828+0000 7f2808a00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f27e406b500 0x7f27e406d9c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f27f8034000 tx=0x7f27f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:42.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:42.831+0000 7f2800c00640 1 -- 172.21.15.62:0/2624414566 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27f00585b0 con 0x7f28040d34f0 2024-04-25T09:08:43.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.113+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f28040b5cb0 con 0x7f28040d34f0 2024-04-25T09:08:43.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.118+0000 7f2800c00640 1 -- 172.21.15.62:0/2624414566 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f27f0057eb0 con 0x7f28040d34f0 2024-04-25T09:08:43.106 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:08:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:08:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:08:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:08:43.106 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:08:43.106 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f27e406b500 msgr2=0x7f27e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f27e406b500 0x7f27e406d9c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f27f8034000 tx=0x7f27f802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040d34f0 msgr2=0x7f28040cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040d34f0 0x7f28040cd300 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c470 tx=0x7f27f0001e70 comp rx=0 tx=0).stop 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 shutdown_connections 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f27e406b500 0x7f27e406d9c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040d34f0 0x7f28040cd300 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f280415dd20 0x7f2804158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 --2- 172.21.15.62:0/2624414566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f28040ceef0 0x7f28041f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.121+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 >> 172.21.15.62:0/2624414566 conn(0x7f280406b000 msgr2=0x7f280415e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.122+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 shutdown_connections 2024-04-25T09:08:43.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.122+0000 7f2809e00640 1 -- 172.21.15.62:0/2624414566 wait complete. 2024-04-25T09:08:43.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:43.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.353+0000 7f6f06c00640 1 Processor -- start 2024-04-25T09:08:43.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.353+0000 7f6f06c00640 1 -- start start 2024-04-25T09:08:43.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.355+0000 7f6f06c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f00072340 0x7f6f000702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:43.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.355+0000 7f6f06c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0006c310 con 0x7f6f000cfed0 2024-04-25T09:08:43.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.355+0000 7f6f06c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0006c5b0 con 0x7f6f000707f0 2024-04-25T09:08:43.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.355+0000 7f6f06c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0006c890 con 0x7f6f00072340 2024-04-25T09:08:43.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.356+0000 7f6effe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6f000cfed0 0x7f6f0015f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44332/0 (socket says 172.21.15.62:44332) 2024-04-25T09:08:43.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.356+0000 7f6effe00640 1 -- 172.21.15.62:0/2454858566 learned_addr learned my addr 172.21.15.62:0/2454858566 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:43.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.356+0000 7f6eff400640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f00072340 0x7f6f000702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:43.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.356+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707973379 0 0) 0x7f6f0006c310 con 0x7f6f000cfed0 2024-04-25T09:08:43.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.356+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee4003520 con 0x7f6f000cfed0 2024-04-25T09:08:43.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.356+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3129210602 0 0) 0x7f6ee4003520 con 0x7f6f000cfed0 2024-04-25T09:08:43.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f00072340 msgr2=0x7f6f000702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:43.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f00072340 0x7f6f000702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 >> v1:172.21.15.121:6789/0 conn(0x7f6f000707f0 legacy=0x7f6f000b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0006cb70 con 0x7f6f000cfed0 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f6ef00019f0 con 0x7f6f000cfed0 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 >> v1:172.21.15.62:6789/0 conn(0x7f6f000cfed0 legacy=0x7f6f0015f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee40039e0 0x7f6ee4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee4004390 0x7f6ee4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4005060 0x7f6ee400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0006c890 con 0x7f6ee4004390 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee400bf00 con 0x7f6ee4005060 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee400c0d0 con 0x7f6ee40039e0 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.357+0000 7f6eff400640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee4004390 0x7f6ee4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6effe00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee40039e0 0x7f6ee4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6f04c00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4005060 0x7f6ee400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:43.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6eff400640 1 -- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee40039e0 msgr2=0x7f6ee4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:43.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6eff400640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee40039e0 0x7f6ee4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6eff400640 1 -- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4005060 msgr2=0x7f6ee400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:43.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6eff400640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4005060 0x7f6ee400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6eff400640 1 -- 172.21.15.62:0/2454858566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4003520 con 0x7f6ee4004390 2024-04-25T09:08:43.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6effe00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee40039e0 0x7f6ee4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:43.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.358+0000 7f6f04c00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4005060 0x7f6ee400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.359+0000 7f6eff400640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee4004390 0x7f6ee4004860 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009ac0 tx=0x7f6ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2951575493c3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.359+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec036070 con 0x7f6ee4004390 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.359+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eec031400 con 0x7f6ee4004390 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.359+0000 7f6efea00640 1 -- 172.21.15.62:0/2454858566 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec009690 con 0x7f6ee4004390 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.360+0000 7f6f06c00640 1 -- 172.21.15.62:0/2454858566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee4004390 msgr2=0x7f6ee4004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.360+0000 7f6f06c00640 1 --2- 172.21.15.62:0/2454858566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ee4004390 0x7f6ee4004860 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009ac0 tx=0x7f6ee4005830 comp rx=0 tx=0).stop 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.360+0000 7f6f04c00640 1 -- 172.21.15.62:0/2454858566 reap_dead start 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.360+0000 7f6f06c00640 1 -- 172.21.15.62:0/2454858566 shutdown_connections 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.360+0000 7f6f06c00640 1 -- 172.21.15.62:0/2454858566 >> 172.21.15.62:0/2454858566 conn(0x7f6f000668f0 msgr2=0x7f6f00066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:43.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.360+0000 7f6f06c00640 1 -- 172.21.15.62:0/2454858566 shutdown_connections 2024-04-25T09:08:43.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.360+0000 7f6f06c00640 1 -- 172.21.15.62:0/2454858566 wait complete. 2024-04-25T09:08:43.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.362+0000 7f6f06c00640 1 Processor -- start 2024-04-25T09:08:43.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.362+0000 7f6f06c00640 1 -- start start 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6f06c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee4004390 0x7f6f000b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6f06c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f001f5000 0x7f6f001f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6f06c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f000c4680 0x7f6f000c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6f06c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f000702b0 con 0x7f6f001f5000 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6f06c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0006d130 con 0x7f6f000c4680 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6f06c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f00070430 con 0x7f6ee4004390 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6eff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f001f5000 0x7f6f001f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6eff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f001f5000 0x7f6f001f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35278/0 (socket says 172.21.15.62:35278) 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6eff400640 1 -- 172.21.15.62:0/1370960821 learned_addr learned my addr 172.21.15.62:0/1370960821 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6effe00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee4004390 0x7f6f000b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:43.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.363+0000 7f6f04c00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f000c4680 0x7f6f000c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:43.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6eff400640 1 -- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee4004390 msgr2=0x7f6f000b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6eff400640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee4004390 0x7f6f000b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6eff400640 1 -- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f000c4680 msgr2=0x7f6f000c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6eff400640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f000c4680 0x7f6f000c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6eff400640 1 -- 172.21.15.62:0/1370960821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f001f2740 con 0x7f6f001f5000 2024-04-25T09:08:43.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6eff400640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f001f5000 0x7f6f001f5460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6eec034040 tx=0x7f6eec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:43.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6ef7e00640 1 -- 172.21.15.62:0/1370960821 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec040020 con 0x7f6f001f5000 2024-04-25T09:08:43.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6ef7e00640 1 -- 172.21.15.62:0/1370960821 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eec003c60 con 0x7f6f001f5000 2024-04-25T09:08:43.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f001f1980 con 0x7f6f001f5000 2024-04-25T09:08:43.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.364+0000 7f6ef7e00640 1 -- 172.21.15.62:0/1370960821 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec036040 con 0x7f6f001f5000 2024-04-25T09:08:43.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.365+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f001f1f40 con 0x7f6f001f5000 2024-04-25T09:08:43.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.366+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f00072c70 con 0x7f6f001f5000 2024-04-25T09:08:43.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.367+0000 7f6ef7e00640 1 -- 172.21.15.62:0/1370960821 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6eec043020 con 0x7f6f001f5000 2024-04-25T09:08:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.368+0000 7f6ef7e00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ee006b4d0 0x7f6ee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:43.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.368+0000 7f6ef7e00640 1 -- 172.21.15.62:0/1370960821 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6eec0aa430 con 0x7f6f001f5000 2024-04-25T09:08:43.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.369+0000 7f6effe00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ee006b4d0 0x7f6ee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:43.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.370+0000 7f6effe00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ee006b4d0 0x7f6ee006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00020a0 tx=0x7f6ef0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:43.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.373+0000 7f6ef7e00640 1 -- 172.21.15.62:0/1370960821 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6eec04a070 con 0x7f6f001f5000 2024-04-25T09:08:43.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.619+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f0004f000 con 0x7f6f001f5000 2024-04-25T09:08:43.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.620+0000 7f6ef7e00640 1 -- 172.21.15.62:0/1370960821 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f6eec0044e0 con 0x7f6f001f5000 2024-04-25T09:08:43.608 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:43.608 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3737740461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3737740461},{"type":"v1","addr":"172.21.15.62:6835","nonce":3737740461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":5099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/4172870608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4172870608},{"type":"v1","addr":"172.21.15.121:6836","nonce":4172870608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2112486228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":2112486228},{"type":"v1","addr":"172.21.15.121:6837","nonce":2112486228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:40.023899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/3491368038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3491368038},{"type":"v1","addr":"172.21.15.62:6834","nonce":3491368038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:08:43.608 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ee006b4d0 msgr2=0x7f6ee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ee006b4d0 0x7f6ee006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00020a0 tx=0x7f6ef0009040 comp rx=0 tx=0).stop 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f001f5000 msgr2=0x7f6f001f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f001f5000 0x7f6f001f5460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6eec034040 tx=0x7f6eec004730 comp rx=0 tx=0).stop 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 shutdown_connections 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ee006b4d0 0x7f6ee006d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f001f5000 0x7f6f001f5460 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f000c4680 0x7f6f000c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 --2- 172.21.15.62:0/1370960821 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ee4004390 0x7f6f000b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 >> 172.21.15.62:0/1370960821 conn(0x7f6f000668f0 msgr2=0x7f6f000cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.623+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 shutdown_connections 2024-04-25T09:08:43.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:43.624+0000 7f6f06c00640 1 -- 172.21.15.62:0/1370960821 wait complete. 2024-04-25T09:08:44.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:08:36.958423+0000', 'modified': '2024-04-25T09:08:40.023899+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6390}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6390': {'gid': 6390, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6834/3491368038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 3491368038}, {'type': 'v1', 'addr': '172.21.15.62:6834', 'nonce': 3491368038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:08:44.032 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:08:44.032 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:08:44.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.273+0000 7f50e9a00640 1 Processor -- start 2024-04-25T09:08:44.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.273+0000 7f50e9a00640 1 -- start start 2024-04-25T09:08:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.275+0000 7f50e9a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f50e415c4e0 0x7f50e415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.275+0000 7f50e9a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e40cf5f0 0x7f50e40b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50e9a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e40b3720 con 0x7f50e40cfa70 2024-04-25T09:08:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50e9a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e40cff60 con 0x7f50e415c4e0 2024-04-25T09:08:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50e9a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4070360 con 0x7f50e40cf5f0 2024-04-25T09:08:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50e9000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f50e40cfa70 0x7f50e415f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44340/0 (socket says 172.21.15.62:44340) 2024-04-25T09:08:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50e9000640 1 -- 172.21.15.62:0/3200082282 learned_addr learned my addr 172.21.15.62:0/3200082282 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:44.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50dbe00640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e40cf5f0 0x7f50e40b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50e3e00640 1 --2- 172.21.15.62:0/3200082282 >> v2:172.21.15.121:3300/0 conn(0x7f50e415c4e0 0x7f50e415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872707247 0 0) 0x7f50e40b3720 con 0x7f50e40cfa70 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.276+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c8003520 con 0x7f50e40cfa70 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319971991 0 0) 0x7f50c8003520 con 0x7f50e40cfa70 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e40cf5f0 msgr2=0x7f50e40b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e40cf5f0 0x7f50e40b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 >> v2:172.21.15.121:3300/0 conn(0x7f50e415c4e0 msgr2=0x7f50e415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 --2- 172.21.15.62:0/3200082282 >> v2:172.21.15.121:3300/0 conn(0x7f50e415c4e0 0x7f50e415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e4070640 con 0x7f50e40cfa70 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f50dc0019f0 con 0x7f50e40cfa70 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 >> v1:172.21.15.62:6789/0 conn(0x7f50e40cfa70 legacy=0x7f50e415f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e40cf5f0 0x7f50c80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c8004d40 0x7f50c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40cff60 con 0x7f50c8003f30 2024-04-25T09:08:44.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4070360 con 0x7f50c8004d40 2024-04-25T09:08:44.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c800fdd0 con 0x7f50e40cf5f0 2024-04-25T09:08:44.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.277+0000 7f50dbe00640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.278+0000 7f50e3e00640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e40cf5f0 0x7f50c80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.278+0000 7f50dbe00640 1 -- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e40cf5f0 msgr2=0x7f50c80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.278+0000 7f50dbe00640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e40cf5f0 0x7f50c80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.278+0000 7f50e9000640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c8004d40 0x7f50c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.278+0000 7f50dbe00640 1 -- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c8004d40 msgr2=0x7f50c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.278+0000 7f50dbe00640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50c8004d40 0x7f50c80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.278+0000 7f50dbe00640 1 -- 172.21.15.62:0/3200082282 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8003520 con 0x7f50c8003f30 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.279+0000 7f50dbe00640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50c8004800 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009ac0 tx=0x7f50c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb33f93855b476a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.279+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc036070 con 0x7f50c8003f30 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.279+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc031400 con 0x7f50c8003f30 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.279+0000 7f50e3400640 1 -- 172.21.15.62:0/3200082282 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc009690 con 0x7f50c8003f30 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.280+0000 7f50e9a00640 1 -- 172.21.15.62:0/3200082282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 msgr2=0x7f50c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.280+0000 7f50e9a00640 1 --2- 172.21.15.62:0/3200082282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50c8004800 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009ac0 tx=0x7f50c8005830 comp rx=0 tx=0).stop 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.280+0000 7f50e9000640 1 -- 172.21.15.62:0/3200082282 reap_dead start 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.280+0000 7f50e9a00640 1 -- 172.21.15.62:0/3200082282 shutdown_connections 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.280+0000 7f50e9a00640 1 -- 172.21.15.62:0/3200082282 >> 172.21.15.62:0/3200082282 conn(0x7f50e40668f0 msgr2=0x7f50e4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.280+0000 7f50e9a00640 1 -- 172.21.15.62:0/3200082282 shutdown_connections 2024-04-25T09:08:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.280+0000 7f50e9a00640 1 -- 172.21.15.62:0/3200082282 wait complete. 2024-04-25T09:08:44.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.282+0000 7f50e9a00640 1 Processor -- start 2024-04-25T09:08:44.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.282+0000 7f50e9a00640 1 -- start start 2024-04-25T09:08:44.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50e40a9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e9a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e4149ea0 0x7f50e40ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e9a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e4158aa0 0x7f50e4149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e415ce00 con 0x7f50c8003f30 2024-04-25T09:08:44.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e9a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4070c00 con 0x7f50e4149ea0 2024-04-25T09:08:44.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e9a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4071120 con 0x7f50e4158aa0 2024-04-25T09:08:44.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50e40a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50e40a9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35282/0 (socket says 172.21.15.62:35282) 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.283+0000 7f50e3e00640 1 -- 172.21.15.62:0/1495474558 learned_addr learned my addr 172.21.15.62:0/1495474558 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50dbe00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e4149ea0 0x7f50e40ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50e3e00640 1 -- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e4158aa0 msgr2=0x7f50e4149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50e3e00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e4158aa0 0x7f50e4149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50e3e00640 1 -- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e4149ea0 msgr2=0x7f50e40ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50e3e00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e4149ea0 0x7f50e40ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50e3e00640 1 -- 172.21.15.62:0/1495474558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e40ae540 con 0x7f50c8003f30 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50dbe00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e4149ea0 0x7f50e40ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:44.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50e3e00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50e40a9770 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f50d40116d0 tx=0x7f50d4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.284+0000 7f50e0c00640 1 -- 172.21.15.62:0/1495474558 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d400f9c0 con 0x7f50c8003f30 2024-04-25T09:08:44.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.285+0000 7f50e0c00640 1 -- 172.21.15.62:0/1495474558 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d400fb60 con 0x7f50c8003f30 2024-04-25T09:08:44.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.285+0000 7f50e0c00640 1 -- 172.21.15.62:0/1495474558 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4015460 con 0x7f50c8003f30 2024-04-25T09:08:44.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.285+0000 7f50e9a00640 1 -- 172.21.15.62:0/1495474558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e40a9e50 con 0x7f50c8003f30 2024-04-25T09:08:44.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.285+0000 7f50e9a00640 1 -- 172.21.15.62:0/1495474558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e40aa410 con 0x7f50c8003f30 2024-04-25T09:08:44.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.287+0000 7f50e0c00640 1 -- 172.21.15.62:0/1495474558 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50d4015600 con 0x7f50c8003f30 2024-04-25T09:08:44.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.288+0000 7f50e0c00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50c006b500 0x7f50c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.288+0000 7f50e0c00640 1 -- 172.21.15.62:0/1495474558 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50d408ac50 con 0x7f50c8003f30 2024-04-25T09:08:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.289+0000 7f50dbe00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50c006b500 0x7f50c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.289+0000 7f50eddc6740 1 Processor -- start 2024-04-25T09:08:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.289+0000 7f50eddc6740 1 -- start start 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd2feb30 0x55f2cd326970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd2fef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd3477a0 0x55f2cd2fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50eddc6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd262620 con 0x55f2cd3477a0 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50eddc6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd20bca0 con 0x55f2cd326eb0 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50eddc6740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd25e6c0 con 0x55f2cd2feb30 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50dbe00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50c006b500 0x7f50c006d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f50cc033000 tx=0x7f50cc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50e9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd3477a0 0x55f2cd2fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50e9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd3477a0 0x55f2cd2fd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35296/0 (socket says 172.21.15.62:35296) 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50e9000640 1 -- 172.21.15.62:0/1115445728 learned_addr learned my addr 172.21.15.62:0/1115445728 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.290+0000 7f50dbe00640 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd2fef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50e3e00640 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd2feb30 0x55f2cd326970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50e9000640 1 -- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd2feb30 msgr2=0x55f2cd326970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50e9000640 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd2feb30 0x55f2cd326970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50e9000640 1 -- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 msgr2=0x55f2cd2fef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50e9000640 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd2fef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50e9000640 1 -- 172.21.15.62:0/1115445728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2cd212f40 con 0x55f2cd3477a0 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50e3e00640 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd2feb30 0x55f2cd326970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:44.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50e9000640 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd3477a0 0x55f2cd2fd2f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f50dc001370 tx=0x7f50dc001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dc826dfc7d3dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50d8c00640 1 -- 172.21.15.62:0/1115445728 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc004d60 con 0x55f2cd3477a0 2024-04-25T09:08:44.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50d8c00640 1 -- 172.21.15.62:0/1115445728 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc007110 con 0x55f2cd3477a0 2024-04-25T09:08:44.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.291+0000 7f50d8c00640 1 -- 172.21.15.62:0/1115445728 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc007500 con 0x55f2cd3477a0 2024-04-25T09:08:44.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 -- 172.21.15.62:0/1115445728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd3477a0 msgr2=0x55f2cd2fd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd3477a0 0x55f2cd2fd2f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f50dc001370 tx=0x7f50dc001b20 comp rx=0 tx=0).stop 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 -- 172.21.15.62:0/1115445728 shutdown_connections 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd3477a0 0x55f2cd2fd2f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd2fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 --2- 172.21.15.62:0/1115445728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd2feb30 0x55f2cd326970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 -- 172.21.15.62:0/1115445728 >> 172.21.15.62:0/1115445728 conn(0x55f2cd2fbd50 msgr2=0x55f2cd344730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 -- 172.21.15.62:0/1115445728 shutdown_connections 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.292+0000 7f50eddc6740 1 -- 172.21.15.62:0/1115445728 wait complete. 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.293+0000 7f50eddc6740 1 -- start start 2024-04-25T09:08:44.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.293+0000 7f50eddc6740 1 Processor -- start 2024-04-25T09:08:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd326eb0 0x55f2cd31e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd348110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd31ae70 0x55f2cd348650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50eddc6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd3264e0 con 0x55f2cd326eb0 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50eddc6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd2011f0 con 0x55f2cd31ae70 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50eddc6740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd325ed0 con 0x55f2cd3477a0 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50eddc6740 10 client.0 _ll_register_callbacks cb 0x55f2cd21ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd326eb0 0x55f2cd31e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd326eb0 0x55f2cd31e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35304/0 (socket says 172.21.15.62:35304) 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.294+0000 7f50e3e00640 1 -- 172.21.15.62:0/2497268939 learned_addr learned my addr 172.21.15.62:0/2497268939 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50e3e00640 1 -- 172.21.15.62:0/2497268939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 msgr2=0x55f2cd348110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50e3e00640 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd348110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50e3e00640 1 -- 172.21.15.62:0/2497268939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd31ae70 msgr2=0x55f2cd348650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50e3e00640 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd31ae70 0x55f2cd348650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50e3e00640 1 -- 172.21.15.62:0/2497268939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2cd220c50 con 0x55f2cd326eb0 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50e3e00640 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd326eb0 0x55f2cd31e240 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f50d4061510 tx=0x7f50d4059330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa92ac585cb33a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50d1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50d1600640 1 -- 172.21.15.62:0/2497268939 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4059590 con 0x55f2cd326eb0 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50d1600640 1 -- 172.21.15.62:0/2497268939 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50d4059730 con 0x55f2cd326eb0 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.295+0000 7f50eddc6740 1 -- 172.21.15.62:0/2497268939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50dc007110 con 0x55f2cd326eb0 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50d1600640 1 -- 172.21.15.62:0/2497268939 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50d40615b0 con 0x55f2cd326eb0 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50eddc6740 10 client.6537 fetch_fsmap learned FSMap version 38 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50eddc6740 1 -- 172.21.15.62:0/2497268939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55f2ccf772d0 con 0x55f2cd326eb0 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50d1600640 1 -- 172.21.15.62:0/2497268939 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc007110 con 0x55f2cd326eb0 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50d1600640 1 -- 172.21.15.62:0/2497268939 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f50d406e030 con 0x55f2cd326eb0 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50eddc6740 10 client.6537 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50eddc6740 10 client.6537 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50eddc6740 20 client.6537 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] conn(0x55f2cd31bb80 0x55f2cd31bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50eddc6740 4 client.6537 mds_command: new command op to 6390 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50eddc6740 1 -- 172.21.15.62:0/2497268939 --> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2cd26fb60 con 0x55f2cd31bb80 2024-04-25T09:08:44.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.296+0000 7f50dbe00640 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] conn(0x55f2cd31bb80 0x55f2cd31bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.297+0000 7f50dbe00640 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] conn(0x55f2cd31bb80 0x55f2cd31bf80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e02a68de80025aa server_cookie=d6510f1f6e2ee9ee in_seq=0 out_seq=0 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.297+0000 7f50d1600640 10 client.6537 ms_handle_connect on v2:172.21.15.62:6832/3491368038 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50d1600640 1 -- 172.21.15.62:0/2497268939 <== mds.0 v2:172.21.15.62:6832/3491368038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2cd26fb60 con 0x55f2cd31bb80 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50d1600640 10 client.6537 handle_command_reply: tid=0 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 client.6537 shutdown 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 -- 172.21.15.62:0/2497268939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd326eb0 msgr2=0x55f2cd31e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd326eb0 0x55f2cd31e240 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f50d4061510 tx=0x7f50d4059330 comp rx=0 tx=0).stop 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 -- 172.21.15.62:0/2497268939 shutdown_connections 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] conn(0x55f2cd31bb80 0x55f2cd31bf80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd31ae70 0x55f2cd348650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd348110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2497268939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd326eb0 0x55f2cd31e240 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50eddc6740 1 -- 172.21.15.62:0/2497268939 >> 172.21.15.62:0/2497268939 conn(0x55f2cd2feb30 msgr2=0x55f2cd344530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:44.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.298+0000 7f50d1600640 0 client.6537 ms_handle_reset on v2:172.21.15.62:6832/3491368038 2024-04-25T09:08:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.299+0000 7f50eddc6740 1 -- 172.21.15.62:0/2497268939 shutdown_connections 2024-04-25T09:08:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.299+0000 7f50eddc6740 1 -- 172.21.15.62:0/2497268939 wait complete. 2024-04-25T09:08:44.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.299+0000 7f50eddc6740 20 client.6537 trim_cache size 0 max 16384 2024-04-25T09:08:44.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.316+0000 7f50eddc6740 1 Processor -- start 2024-04-25T09:08:44.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.316+0000 7f50eddc6740 1 -- start start 2024-04-25T09:08:44.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.317+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd320bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.317+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd354240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.317+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd352e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.317+0000 7f50eddc6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e41fb2e0 con 0x55f2cd31ae70 2024-04-25T09:08:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.317+0000 7f50eddc6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e40ab880 con 0x55f2cd326eb0 2024-04-25T09:08:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.317+0000 7f50eddc6740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e41fadb0 con 0x55f2cd3477a0 2024-04-25T09:08:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.317+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd320bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd320bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35314/0 (socket says 172.21.15.62:35314) 2024-04-25T09:08:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50e3e00640 1 -- 172.21.15.62:0/2679002197 learned_addr learned my addr 172.21.15.62:0/2679002197 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50e3e00640 1 -- 172.21.15.62:0/2679002197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 msgr2=0x55f2cd352e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50e3e00640 1 --2- 172.21.15.62:0/2679002197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd352e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50e3e00640 1 -- 172.21.15.62:0/2679002197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 msgr2=0x55f2cd354240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50e3e00640 1 --2- 172.21.15.62:0/2679002197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd354240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50e3e00640 1 -- 172.21.15.62:0/2679002197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2cd0416b0 con 0x55f2cd31ae70 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50e3e00640 1 --2- 172.21.15.62:0/2679002197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd320bc0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f50d4059270 tx=0x7f50d401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e234f30c7bb57bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.318+0000 7f50d3e00640 1 -- 172.21.15.62:0/2679002197 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4059590 con 0x55f2cd31ae70 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50d3e00640 1 -- 172.21.15.62:0/2679002197 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d4059770 con 0x55f2cd31ae70 2024-04-25T09:08:44.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50d3e00640 1 -- 172.21.15.62:0/2679002197 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d405f650 con 0x55f2cd31ae70 2024-04-25T09:08:44.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 -- 172.21.15.62:0/2679002197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 msgr2=0x55f2cd320bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2679002197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd320bc0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f50d4059270 tx=0x7f50d401e000 comp rx=0 tx=0).stop 2024-04-25T09:08:44.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 -- 172.21.15.62:0/2679002197 shutdown_connections 2024-04-25T09:08:44.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2679002197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd352e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2679002197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd354240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 --2- 172.21.15.62:0/2679002197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd320bc0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 -- 172.21.15.62:0/2679002197 >> 172.21.15.62:0/2679002197 conn(0x55f2cd2feb30 msgr2=0x55f2cd31bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:44.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 -- 172.21.15.62:0/2679002197 shutdown_connections 2024-04-25T09:08:44.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.319+0000 7f50eddc6740 1 -- 172.21.15.62:0/2679002197 wait complete. 2024-04-25T09:08:44.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.320+0000 7f50eddc6740 1 -- start start 2024-04-25T09:08:44.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.320+0000 7f50eddc6740 1 Processor -- start 2024-04-25T09:08:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.321+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd3a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.321+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd3afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.321+0000 7f50eddc6740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd2ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.321+0000 7f50eddc6740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd31e780 con 0x55f2cd31ae70 2024-04-25T09:08:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.321+0000 7f50eddc6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd323720 con 0x55f2cd326eb0 2024-04-25T09:08:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.321+0000 7f50eddc6740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2cd324230 con 0x55f2cd3477a0 2024-04-25T09:08:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.321+0000 7f50eddc6740 10 client.0 _ll_register_callbacks cb 0x7f50d400ff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.321+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd3a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd3a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35316/0 (socket says 172.21.15.62:35316) 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50e3e00640 1 -- 172.21.15.62:0/969132453 learned_addr learned my addr 172.21.15.62:0/969132453 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50e3e00640 1 -- 172.21.15.62:0/969132453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 msgr2=0x55f2cd2ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50e3e00640 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd2ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50e3e00640 1 -- 172.21.15.62:0/969132453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 msgr2=0x55f2cd3afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50e3e00640 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd3afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50e3e00640 1 -- 172.21.15.62:0/969132453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2cd319740 con 0x55f2cd31ae70 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50e3e00640 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd3a6b30 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f50d4055cb0 tx=0x7f50d401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c85c22d1140a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50d1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50d1600640 1 -- 172.21.15.62:0/969132453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4059590 con 0x55f2cd31ae70 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50d1600640 1 -- 172.21.15.62:0/969132453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50d405f620 con 0x55f2cd31ae70 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.322+0000 7f50eddc6740 1 -- 172.21.15.62:0/969132453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50d4059770 con 0x55f2cd31ae70 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50d1600640 1 -- 172.21.15.62:0/969132453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50d4069050 con 0x55f2cd31ae70 2024-04-25T09:08:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50eddc6740 10 client.6543 fetch_fsmap learned FSMap version 38 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50eddc6740 1 -- 172.21.15.62:0/969132453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55f2cd3b0490 con 0x55f2cd31ae70 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50d1600640 1 -- 172.21.15.62:0/969132453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d4059770 con 0x55f2cd31ae70 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50d1600640 1 -- 172.21.15.62:0/969132453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 38) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f50d406c020 con 0x55f2cd31ae70 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50eddc6740 10 client.6543 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50eddc6740 10 client.6543 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50eddc6740 20 client.6543 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50eddc6740 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] conn(0x55f2cd2fd410 0x55f2cd2fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50eddc6740 4 client.6543 mds_command: new command op to 6390 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:08:44.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50eddc6740 1 -- 172.21.15.62:0/969132453 --> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f2cd269eb0 con 0x55f2cd2fd410 2024-04-25T09:08:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.323+0000 7f50dbe00640 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] conn(0x55f2cd2fd410 0x55f2cd2fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50dbe00640 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] conn(0x55f2cd2fd410 0x55f2cd2fd7f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7409de1b1783c413 server_cookie=c5ea30928c905d59 in_seq=0 out_seq=0 2024-04-25T09:08:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50d1600640 10 client.6543 ms_handle_connect on v2:172.21.15.62:6832/3491368038 2024-04-25T09:08:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50d1600640 1 -- 172.21.15.62:0/969132453 <== mds.0 v2:172.21.15.62:6832/3491368038 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55f2cd269eb0 con 0x55f2cd2fd410 2024-04-25T09:08:44.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50d1600640 10 client.6543 handle_command_reply: tid=0 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50eddc6740 1 client.6543 shutdown 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50eddc6740 1 -- 172.21.15.62:0/969132453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 msgr2=0x55f2cd3a6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50eddc6740 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd3a6b30 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f50d4055cb0 tx=0x7f50d401f040 comp rx=0 tx=0).stop 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50eddc6740 1 -- 172.21.15.62:0/969132453 shutdown_connections 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50eddc6740 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.62:6832/3491368038,v1:172.21.15.62:6834/3491368038] conn(0x55f2cd2fd410 0x55f2cd2fd7f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50eddc6740 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2cd3477a0 0x55f2cd2ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.324+0000 7f50eddc6740 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2cd326eb0 0x55f2cd3afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.325+0000 7f50eddc6740 1 --2- 172.21.15.62:0/969132453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2cd31ae70 0x55f2cd3a6b30 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.325+0000 7f50d1600640 0 client.6543 ms_handle_reset on v2:172.21.15.62:6832/3491368038 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.325+0000 7f50eddc6740 1 -- 172.21.15.62:0/969132453 >> 172.21.15.62:0/969132453 conn(0x55f2cd2feb30 msgr2=0x55f2cd325ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.325+0000 7f50eddc6740 1 -- 172.21.15.62:0/969132453 shutdown_connections 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.325+0000 7f50eddc6740 1 -- 172.21.15.62:0/969132453 wait complete. 2024-04-25T09:08:44.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.325+0000 7f50eddc6740 20 client.6543 trim_cache size 0 max 16384 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 6390, 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 6.827894583, 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 38, 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 62, 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 62, 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 13.275407041999999 2024-04-25T09:08:44.315 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.325+0000 7f50e9a00640 1 -- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50c006b500 msgr2=0x7f50c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.325+0000 7f50e9a00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50c006b500 0x7f50c006d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f50cc033000 tx=0x7f50cc008030 comp rx=0 tx=0).stop 2024-04-25T09:08:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 -- 172.21.15.62:0/1495474558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 msgr2=0x7f50e40a9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50e40a9770 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f50d40116d0 tx=0x7f50d4011b90 comp rx=0 tx=0).stop 2024-04-25T09:08:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 -- 172.21.15.62:0/1495474558 shutdown_connections 2024-04-25T09:08:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f50c006b500 0x7f50c006d9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f50e4158aa0 0x7f50e4149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50e4149ea0 0x7f50e40ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 --2- 172.21.15.62:0/1495474558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f50c8003f30 0x7f50e40a9770 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 -- 172.21.15.62:0/1495474558 >> 172.21.15.62:0/1495474558 conn(0x7f50e40668f0 msgr2=0x7f50e415d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 -- 172.21.15.62:0/1495474558 shutdown_connections 2024-04-25T09:08:44.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.326+0000 7f50e9a00640 1 -- 172.21.15.62:0/1495474558 wait complete. 2024-04-25T09:08:44.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:08:44.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.559+0000 7f7138e00640 1 Processor -- start 2024-04-25T09:08:44.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.559+0000 7f7138e00640 1 -- start start 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.560+0000 7f7138e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71340b56f0 0x7f7134160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.560+0000 7f7138e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 0x7f7134161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.560+0000 7f7138e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71340b27a0 0x7f7134161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.560+0000 7f7138e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7134161780 con 0x7f71340b27a0 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.560+0000 7f7138e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71340cedc0 con 0x7f71340b56f0 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.560+0000 7f7138e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340d3110 con 0x7f71340b6a90 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71340b27a0 0x7f7134161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71340b27a0 0x7f7134161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35330/0 (socket says 172.21.15.62:35330) 2024-04-25T09:08:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133e00640 1 -- 172.21.15.62:0/1037158237 learned_addr learned my addr 172.21.15.62:0/1037158237 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:44.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f712aa00640 1 --2- 172.21.15.62:0/1037158237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 0x7f7134161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133400640 1 --2- 172.21.15.62:0/1037158237 >> v2:172.21.15.121:3300/0 conn(0x7f71340b56f0 0x7f7134160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133e00640 1 -- 172.21.15.62:0/1037158237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 msgr2=0x7f7134161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133e00640 1 --2- 172.21.15.62:0/1037158237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 0x7f7134161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133e00640 1 -- 172.21.15.62:0/1037158237 >> v2:172.21.15.121:3300/0 conn(0x7f71340b56f0 msgr2=0x7f7134160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:44.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133e00640 1 --2- 172.21.15.62:0/1037158237 >> v2:172.21.15.121:3300/0 conn(0x7f71340b56f0 0x7f7134160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.561+0000 7f7133e00640 1 -- 172.21.15.62:0/1037158237 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340d33f0 con 0x7f71340b27a0 2024-04-25T09:08:44.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.563+0000 7f7133e00640 1 --2- 172.21.15.62:0/1037158237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b27a0 0x7f7134161980 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f713408c310 tx=0x7f712c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752989e79f9513d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7132a00640 1 -- 172.21.15.62:0/1037158237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c036070 con 0x7f71340b27a0 2024-04-25T09:08:44.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7132a00640 1 -- 172.21.15.62:0/1037158237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c031600 con 0x7f71340b27a0 2024-04-25T09:08:44.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7132a00640 1 -- 172.21.15.62:0/1037158237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0319d0 con 0x7f71340b27a0 2024-04-25T09:08:44.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7138e00640 1 -- 172.21.15.62:0/1037158237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b27a0 msgr2=0x7f7134161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7138e00640 1 --2- 172.21.15.62:0/1037158237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b27a0 0x7f7134161980 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f713408c310 tx=0x7f712c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7138e00640 1 -- 172.21.15.62:0/1037158237 shutdown_connections 2024-04-25T09:08:44.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7138e00640 1 --2- 172.21.15.62:0/1037158237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b27a0 0x7f7134161980 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7138e00640 1 --2- 172.21.15.62:0/1037158237 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 0x7f7134161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7138e00640 1 --2- 172.21.15.62:0/1037158237 >> v2:172.21.15.121:3300/0 conn(0x7f71340b56f0 0x7f7134160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.565+0000 7f7138e00640 1 -- 172.21.15.62:0/1037158237 >> 172.21.15.62:0/1037158237 conn(0x7f7134066d10 msgr2=0x7f7134067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:44.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.566+0000 7f7138e00640 1 -- 172.21.15.62:0/1037158237 shutdown_connections 2024-04-25T09:08:44.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.566+0000 7f7138e00640 1 -- 172.21.15.62:0/1037158237 wait complete. 2024-04-25T09:08:44.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.567+0000 7f7138e00640 1 Processor -- start 2024-04-25T09:08:44.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.568+0000 7f7138e00640 1 -- start start 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.568+0000 7f7138e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71340b27a0 0x7f713415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.568+0000 7f7138e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b56f0 0x7f713415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f7138e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 0x7f713415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f7138e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b2d60 con 0x7f71340b56f0 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f7138e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340cea50 con 0x7f71340b27a0 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f7138e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b2ee0 con 0x7f71340b6a90 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f712aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b56f0 0x7f713415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f712aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b56f0 0x7f713415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35332/0 (socket says 172.21.15.62:35332) 2024-04-25T09:08:44.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f712aa00640 1 -- 172.21.15.62:0/3272677230 learned_addr learned my addr 172.21.15.62:0/3272677230 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:44.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f7133400640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71340b27a0 0x7f713415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f712aa00640 1 -- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 msgr2=0x7f713415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f712aa00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 0x7f713415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f712aa00640 1 -- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71340b27a0 msgr2=0x7f713415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f712aa00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71340b27a0 0x7f713415a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.569+0000 7f712aa00640 1 -- 172.21.15.62:0/3272677230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7134157c30 con 0x7f71340b56f0 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.570+0000 7f712aa00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b56f0 0x7f713415adc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f711c00d5f0 tx=0x7f711c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.570+0000 7f712be00640 1 -- 172.21.15.62:0/3272677230 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c0125f0 con 0x7f71340b56f0 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.570+0000 7f712be00640 1 -- 172.21.15.62:0/3272677230 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c019070 con 0x7f71340b56f0 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.570+0000 7f712be00640 1 -- 172.21.15.62:0/3272677230 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c0129d0 con 0x7f71340b56f0 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.570+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7134157f10 con 0x7f71340b56f0 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.570+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71341584d0 con 0x7f71340b56f0 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.571+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713404ef20 con 0x7f71340b56f0 2024-04-25T09:08:44.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.572+0000 7f712be00640 1 -- 172.21.15.62:0/3272677230 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f711c012b70 con 0x7f71340b56f0 2024-04-25T09:08:44.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.573+0000 7f712be00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711006b4d0 0x7f711006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:44.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.573+0000 7f712be00640 1 -- 172.21.15.62:0/3272677230 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f711c08a660 con 0x7f71340b56f0 2024-04-25T09:08:44.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.574+0000 7f7133400640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711006b4d0 0x7f711006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:44.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.575+0000 7f7133400640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711006b4d0 0x7f711006d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:44.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.577+0000 7f712be00640 1 -- 172.21.15.62:0/3272677230 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f711c059440 con 0x7f71340b56f0 2024-04-25T09:08:44.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.816+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f71340b9270 con 0x7f71340b56f0 2024-04-25T09:08:44.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.817+0000 7f712be00640 1 -- 172.21.15.62:0/3272677230 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f711c058d40 con 0x7f71340b56f0 2024-04-25T09:08:44.804 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:08:44.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.819+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711006b4d0 msgr2=0x7f711006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.819+0000 7f7138e00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711006b4d0 0x7f711006d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124008040 comp rx=0 tx=0).stop 2024-04-25T09:08:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.819+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b56f0 msgr2=0x7f713415adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.819+0000 7f7138e00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b56f0 0x7f713415adc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f711c00d5f0 tx=0x7f711c00dab0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.820+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 shutdown_connections 2024-04-25T09:08:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.820+0000 7f7138e00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f711006b4d0 0x7f711006d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.820+0000 7f7138e00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71340b56f0 0x7f713415adc0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.820+0000 7f7138e00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71340b6a90 0x7f713415cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.820+0000 7f7138e00640 1 --2- 172.21.15.62:0/3272677230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71340b27a0 0x7f713415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.820+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 >> 172.21.15.62:0/3272677230 conn(0x7f7134066d10 msgr2=0x7f71340b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:44.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.820+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 shutdown_connections 2024-04-25T09:08:44.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:44.820+0000 7f7138e00640 1 -- 172.21.15.62:0/3272677230 wait complete. 2024-04-25T09:08:44.826 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:08:44.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:08:44.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:08:44.826 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:08:44.826 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:44.826 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:08:44.826 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:08:44.827 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:08:44.827 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:44.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:44.827 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:08:44.880 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:08:44.880 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:08:44.880 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:08:44.880 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42334sec preferred_lft 42334sec 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:08:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:08:44.882 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:44.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:44.883 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:08:44.969 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:08:44.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:44.973 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:08:45.029 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:08:45.029 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:08:45.029 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:08:45.029 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:08:45.029 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:08:45.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:45.029 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:08:45.030 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:08:45.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:08:45.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:08:45.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:08:45.030 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:08:45.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:45.191 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:08:45.238 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:08:45.238 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:08:45.238 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:45.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:45.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:08:45.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:08:45.366 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:45.420 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:45.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:08:45.421 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:45.472 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:08:45.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:45.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:08:45.601 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:08:45.601 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:08:45.635 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:08:45.635 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:08:45.635 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:08:45.635 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:08:45.635 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:08:45.639 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:08:45.640 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:08:45.640 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:08:45.640 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:08:45.640 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:08:45.644 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:08:45.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:45.644 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:45.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:45.714 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:08:45.789 INFO:teuthology.orchestra.run.smithi062.stdout:client5171 2024-04-25T09:08:45.789 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:45.789 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:45.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:45.844 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T09:08:45.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:45.907 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T09:08:45.967 DEBUG:teuthology.orchestra.run.smithi062:> stdin-killer -- python3 -c ' 2024-04-25T09:08:45.967 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-04-25T09:08:45.967 DEBUG:teuthology.orchestra.run.smithi062:> import stat 2024-04-25T09:08:45.967 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:08:45.967 DEBUG:teuthology.orchestra.run.smithi062:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T09:08:45.967 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-04-25T09:08:46.120 INFO:teuthology.orchestra.run.smithi062.stdout:1099511627776 2024-04-25T09:08:46.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:08:46.134 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:46.135 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:46.231 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:46.231 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:46.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:08:46.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.511+0000 7f87aac00640 1 Processor -- start 2024-04-25T09:08:46.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.512+0000 7f87aac00640 1 -- start start 2024-04-25T09:08:46.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.513+0000 7f87aac00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f87a40d1e80 0x7f87a40d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.513+0000 7f87aac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87a40d3450 0x7f87a40d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.513+0000 7f87aac00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a40d23b0 con 0x7f87a415dd70 2024-04-25T09:08:46.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.513+0000 7f87aac00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a40d0cc0 con 0x7f87a40d1e80 2024-04-25T09:08:46.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.513+0000 7f87aac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40d2690 con 0x7f87a40d3450 2024-04-25T09:08:46.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.513+0000 7f87a9800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f87a415dd70 0x7f87a40cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35604/0 (socket says 172.21.15.62:35604) 2024-04-25T09:08:46.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.513+0000 7f87a9800640 1 -- 172.21.15.62:0/1045107913 learned_addr learned my addr 172.21.15.62:0/1045107913 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a8e00640 1 --2- 172.21.15.62:0/1045107913 >> v2:172.21.15.121:3300/0 conn(0x7f87a40d1e80 0x7f87a40d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87aa200640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87a40d3450 0x7f87a40d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791743465 0 0) 0x7f87a40d23b0 con 0x7f87a415dd70 2024-04-25T09:08:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f878c003520 con 0x7f87a415dd70 2024-04-25T09:08:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1776141866 0 0) 0x7f878c003520 con 0x7f87a415dd70 2024-04-25T09:08:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87a40d3450 msgr2=0x7f87a40d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87a40d3450 0x7f87a40d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 >> v2:172.21.15.121:3300/0 conn(0x7f87a40d1e80 msgr2=0x7f87a40d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 --2- 172.21.15.62:0/1045107913 >> v2:172.21.15.121:3300/0 conn(0x7f87a40d1e80 0x7f87a40d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40d2970 con 0x7f87a415dd70 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f8798001670 con 0x7f87a415dd70 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 >> v1:172.21.15.62:6789/0 conn(0x7f87a415dd70 legacy=0x7f87a40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87a40d3450 0x7f878c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f878c003f30 0x7f878c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f878c004d40 0x7f878c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40d0cc0 con 0x7f878c003f30 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40d2690 con 0x7f878c004d40 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.514+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c00ba10 con 0x7f87a40d3450 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.515+0000 7f87a8e00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f878c003f30 0x7f878c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.515+0000 7f87a9800640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87a40d3450 0x7f878c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.515+0000 7f87a8e00640 1 -- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87a40d3450 msgr2=0x7f878c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.515+0000 7f87a8e00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f87a40d3450 0x7f878c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.515+0000 7f87aa200640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f878c004d40 0x7f878c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.515+0000 7f87a8e00640 1 -- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f878c004d40 msgr2=0x7f878c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.515+0000 7f87a8e00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f878c004d40 0x7f878c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.515+0000 7f87a8e00640 1 -- 172.21.15.62:0/1045107913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c003520 con 0x7f878c003f30 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.516+0000 7f87a8e00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f878c003f30 0x7f878c004800 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8790009ac0 tx=0x7f878c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a636412b4f0a8f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.516+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790036070 con 0x7f878c003f30 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.516+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8790031400 con 0x7f878c003f30 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.516+0000 7f87a3e00640 1 -- 172.21.15.62:0/1045107913 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790009690 con 0x7f878c003f30 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.517+0000 7f87aac00640 1 -- 172.21.15.62:0/1045107913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f878c003f30 msgr2=0x7f878c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.517+0000 7f87aac00640 1 --2- 172.21.15.62:0/1045107913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f878c003f30 0x7f878c004800 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8790009ac0 tx=0x7f878c005830 comp rx=0 tx=0).stop 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.517+0000 7f87aa200640 1 -- 172.21.15.62:0/1045107913 reap_dead start 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.517+0000 7f87aac00640 1 -- 172.21.15.62:0/1045107913 shutdown_connections 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.517+0000 7f87aac00640 1 -- 172.21.15.62:0/1045107913 >> 172.21.15.62:0/1045107913 conn(0x7f87a406b210 msgr2=0x7f87a40668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.517+0000 7f87aac00640 1 -- 172.21.15.62:0/1045107913 shutdown_connections 2024-04-25T09:08:46.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.517+0000 7f87aac00640 1 -- 172.21.15.62:0/1045107913 wait complete. 2024-04-25T09:08:46.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.519+0000 7f87aac00640 1 Processor -- start 2024-04-25T09:08:46.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.519+0000 7f87aac00640 1 -- start start 2024-04-25T09:08:46.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f878c003f30 0x7f87a40ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a40cd2f0 0x7f87a4158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87a4158790 0x7f87a4158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a415e150 con 0x7f87a4158790 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40d2f30 con 0x7f87a40cd2f0 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40d0780 con 0x7f878c003f30 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aa200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87a4158790 0x7f87a4158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aa200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87a4158790 0x7f87a4158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46210/0 (socket says 172.21.15.62:46210) 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.520+0000 7f87aa200640 1 -- 172.21.15.62:0/1602383768 learned_addr learned my addr 172.21.15.62:0/1602383768 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87a8e00640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a40cd2f0 0x7f87a4158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87a9800640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f878c003f30 0x7f87a40ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87aa200640 1 -- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f878c003f30 msgr2=0x7f87a40ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87aa200640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f878c003f30 0x7f87a40ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87aa200640 1 -- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a40cd2f0 msgr2=0x7f87a4158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87aa200640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a40cd2f0 0x7f87a4158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87aa200640 1 -- 172.21.15.62:0/1602383768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4146540 con 0x7f87a4158790 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87a9800640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f878c003f30 0x7f87a40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87aa200640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87a4158790 0x7f87a4158bf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f87a40d4a60 tx=0x7f879c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87a1600640 1 -- 172.21.15.62:0/1602383768 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c019070 con 0x7f87a4158790 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87a1600640 1 -- 172.21.15.62:0/1602383768 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c007d10 con 0x7f87a4158790 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87a1600640 1 -- 172.21.15.62:0/1602383768 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c014640 con 0x7f87a4158790 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.521+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a4146820 con 0x7f87a4158790 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.522+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a415fd00 con 0x7f87a4158790 2024-04-25T09:08:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.523+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a404ef20 con 0x7f87a4158790 2024-04-25T09:08:46.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.524+0000 7f87a1600640 1 -- 172.21.15.62:0/1602383768 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f879c01c070 con 0x7f87a4158790 2024-04-25T09:08:46.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.525+0000 7f87a1600640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f878406b4d0 0x7f878406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.525+0000 7f87a1600640 1 -- 172.21.15.62:0/1602383768 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f879c089bf0 con 0x7f87a4158790 2024-04-25T09:08:46.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.526+0000 7f87a9800640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f878406b4d0 0x7f878406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.527+0000 7f87a9800640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f878406b4d0 0x7f878406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8798001860 tx=0x7f8798009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.529+0000 7f87a1600640 1 -- 172.21.15.62:0/1602383768 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f879c058a70 con 0x7f87a4158790 2024-04-25T09:08:46.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.733+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87a40cd6d0 con 0x7f87a4158790 2024-04-25T09:08:46.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.735+0000 7f87a1600640 1 -- 172.21.15.62:0/1602383768 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f879c058370 con 0x7f87a4158790 2024-04-25T09:08:46.723 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:46.723 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":62,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:08:38.509961+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:34.817879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:35.997553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:08:46.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f878406b4d0 msgr2=0x7f878406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f878406b4d0 0x7f878406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8798001860 tx=0x7f8798009040 comp rx=0 tx=0).stop 2024-04-25T09:08:46.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87a4158790 msgr2=0x7f87a4158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87a4158790 0x7f87a4158bf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f87a40d4a60 tx=0x7f879c009cb0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 shutdown_connections 2024-04-25T09:08:46.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f878406b4d0 0x7f878406d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f87a4158790 0x7f87a4158bf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87a40cd2f0 0x7f87a4158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 --2- 172.21.15.62:0/1602383768 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f878c003f30 0x7f87a40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 >> 172.21.15.62:0/1602383768 conn(0x7f87a406b210 msgr2=0x7f87a40b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 shutdown_connections 2024-04-25T09:08:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.738+0000 7f87aac00640 1 -- 172.21.15.62:0/1602383768 wait complete. 2024-04-25T09:08:46.745 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:08:34.817879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:08:35.997553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:08:46.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:08:46.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.973+0000 7fa6af200640 1 Processor -- start 2024-04-25T09:08:46.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.974+0000 7fa6af200640 1 -- start start 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.975+0000 7fa6af200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa6b015c520 0x7fa6b00b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.975+0000 7fa6af200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa6b00b3360 0x7fa6b006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.975+0000 7fa6af200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b0166720 0x7fa6b006d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.975+0000 7fa6af200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b00b4640 con 0x7fa6b00b3360 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.975+0000 7fa6af200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b0165c40 con 0x7fa6b015c520 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.975+0000 7fa6af200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0165f20 con 0x7fa6b0166720 2024-04-25T09:08:46.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6a5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa6b00b3360 0x7fa6b006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6a5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa6b00b3360 0x7fa6b006d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46212/0 (socket says 172.21.15.62:46212) 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6a5400640 1 -- 172.21.15.62:0/3514039282 learned_addr learned my addr 172.21.15.62:0/3514039282 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6ade00640 1 --2- 172.21.15.62:0/3514039282 >> v2:172.21.15.121:3300/0 conn(0x7fa6b015c520 0x7fa6b00b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6ae800640 1 --2- 172.21.15.62:0/3514039282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b0166720 0x7fa6b006d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6a5400640 1 -- 172.21.15.62:0/3514039282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b0166720 msgr2=0x7fa6b006d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6a5400640 1 --2- 172.21.15.62:0/3514039282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b0166720 0x7fa6b006d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6a5400640 1 -- 172.21.15.62:0/3514039282 >> v2:172.21.15.121:3300/0 conn(0x7fa6b015c520 msgr2=0x7fa6b00b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:46.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.976+0000 7fa6a5400640 1 --2- 172.21.15.62:0/3514039282 >> v2:172.21.15.121:3300/0 conn(0x7fa6b015c520 0x7fa6b00b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.977+0000 7fa6a5400640 1 -- 172.21.15.62:0/3514039282 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b0166200 con 0x7fa6b00b3360 2024-04-25T09:08:46.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.978+0000 7fa6a5400640 1 --2- 172.21.15.62:0/3514039282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00b3360 0x7fa6b006d2c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00b1c60 tx=0x7fa69802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610d44fbd8244efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.978+0000 7fa6ad400640 1 -- 172.21.15.62:0/3514039282 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698037070 con 0x7fa6b00b3360 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6ad400640 1 -- 172.21.15.62:0/3514039282 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa698032440 con 0x7fa6b00b3360 2024-04-25T09:08:46.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6ad400640 1 -- 172.21.15.62:0/3514039282 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698032810 con 0x7fa6b00b3360 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6af200640 1 -- 172.21.15.62:0/3514039282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00b3360 msgr2=0x7fa6b006d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6af200640 1 --2- 172.21.15.62:0/3514039282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00b3360 0x7fa6b006d2c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00b1c60 tx=0x7fa69802a140 comp rx=0 tx=0).stop 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6af200640 1 -- 172.21.15.62:0/3514039282 shutdown_connections 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6af200640 1 --2- 172.21.15.62:0/3514039282 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b0166720 0x7fa6b006d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6af200640 1 --2- 172.21.15.62:0/3514039282 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00b3360 0x7fa6b006d2c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6af200640 1 --2- 172.21.15.62:0/3514039282 >> v2:172.21.15.121:3300/0 conn(0x7fa6b015c520 0x7fa6b00b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.979+0000 7fa6af200640 1 -- 172.21.15.62:0/3514039282 >> 172.21.15.62:0/3514039282 conn(0x7fa6b0066f20 msgr2=0x7fa6b00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.980+0000 7fa6af200640 1 -- 172.21.15.62:0/3514039282 shutdown_connections 2024-04-25T09:08:46.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.980+0000 7fa6af200640 1 -- 172.21.15.62:0/3514039282 wait complete. 2024-04-25T09:08:46.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.982+0000 7fa6af200640 1 Processor -- start 2024-04-25T09:08:46.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.982+0000 7fa6af200640 1 -- start start 2024-04-25T09:08:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6af200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b00b3360 0x7fa6b00cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6af200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b015c520 0x7fa6b00ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6af200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b0166720 0x7fa6b0149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6af200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00b4250 con 0x7fa6b015c520 2024-04-25T09:08:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6af200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b015c900 con 0x7fa6b0166720 2024-04-25T09:08:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6af200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00b43d0 con 0x7fa6b00b3360 2024-04-25T09:08:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b015c520 0x7fa6b00ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b015c520 0x7fa6b00ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46224/0 (socket says 172.21.15.62:46224) 2024-04-25T09:08:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6a5400640 1 -- 172.21.15.62:0/1223968360 learned_addr learned my addr 172.21.15.62:0/1223968360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6ade00640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b00b3360 0x7fa6b00cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6ae800640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b0166720 0x7fa6b0149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6a5400640 1 -- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b00b3360 msgr2=0x7fa6b00cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6a5400640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b00b3360 0x7fa6b00cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6a5400640 1 -- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b0166720 msgr2=0x7fa6b0149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6a5400640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b0166720 0x7fa6b0149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6a5400640 1 -- 172.21.15.62:0/1223968360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b00cd4b0 con 0x7fa6b015c520 2024-04-25T09:08:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.984+0000 7fa6ae800640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b0166720 0x7fa6b0149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.985+0000 7fa6a5400640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b015c520 0x7fa6b00ccf70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa69802a180 tx=0x7fa6980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.985+0000 7fa6a6a00640 1 -- 172.21.15.62:0/1223968360 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698041020 con 0x7fa6b015c520 2024-04-25T09:08:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.985+0000 7fa6a6a00640 1 -- 172.21.15.62:0/1223968360 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6980306c0 con 0x7fa6b015c520 2024-04-25T09:08:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.985+0000 7fa6a6a00640 1 -- 172.21.15.62:0/1223968360 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698037040 con 0x7fa6b015c520 2024-04-25T09:08:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.985+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b00cdd90 con 0x7fa6b015c520 2024-04-25T09:08:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.985+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b00ce350 con 0x7fa6b015c520 2024-04-25T09:08:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.987+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b004ef20 con 0x7fa6b015c520 2024-04-25T09:08:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.988+0000 7fa6a6a00640 1 -- 172.21.15.62:0/1223968360 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa698044020 con 0x7fa6b015c520 2024-04-25T09:08:46.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.989+0000 7fa6a6a00640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b4d0 0x7fa68c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.989+0000 7fa6a6a00640 1 -- 172.21.15.62:0/1223968360 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6980aa9c0 con 0x7fa6b015c520 2024-04-25T09:08:46.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.990+0000 7fa6ade00640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b4d0 0x7fa68c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:46.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.991+0000 7fa6ade00640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b4d0 0x7fa68c06d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009eb0 tx=0x7fa6a000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:46.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:46.994+0000 7fa6a6a00640 1 -- 172.21.15.62:0/1223968360 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa69804b030 con 0x7fa6b015c520 2024-04-25T09:08:47.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.190+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6b00ceac0 con 0x7fa6b015c520 2024-04-25T09:08:47.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.191+0000 7fa6a6a00640 1 -- 172.21.15.62:0/1223968360 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa698079830 con 0x7fa6b015c520 2024-04-25T09:08:47.179 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:08:47.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b4d0 msgr2=0x7fa68c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b4d0 0x7fa68c06d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009eb0 tx=0x7fa6a000b040 comp rx=0 tx=0).stop 2024-04-25T09:08:47.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b015c520 msgr2=0x7fa6b00ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b015c520 0x7fa6b00ccf70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa69802a180 tx=0x7fa6980046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 shutdown_connections 2024-04-25T09:08:47.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b4d0 0x7fa68c06d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b015c520 0x7fa6b00ccf70 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b0166720 0x7fa6b0149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 --2- 172.21.15.62:0/1223968360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b00b3360 0x7fa6b00cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 >> 172.21.15.62:0/1223968360 conn(0x7fa6b0066f20 msgr2=0x7fa6b006ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:47.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 shutdown_connections 2024-04-25T09:08:47.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.194+0000 7fa6af200640 1 -- 172.21.15.62:0/1223968360 wait complete. 2024-04-25T09:08:47.202 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:08:47.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.435+0000 7f0a73e00640 1 Processor -- start 2024-04-25T09:08:47.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.435+0000 7f0a73e00640 1 -- start start 2024-04-25T09:08:47.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.437+0000 7f0a73e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0a7406d9d0 0x7f0a7415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.437+0000 7f0a73e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7415ee30 con 0x7f0a7415e5f0 2024-04-25T09:08:47.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.437+0000 7f0a73e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a741618b0 con 0x7f0a740ceb50 2024-04-25T09:08:47.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.437+0000 7f0a73e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a74161b90 con 0x7f0a7406d9d0 2024-04-25T09:08:47.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.438+0000 7f0a73400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0a7415e5f0 0x7f0a740b4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35612/0 (socket says 172.21.15.62:35612) 2024-04-25T09:08:47.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.438+0000 7f0a73400640 1 -- 172.21.15.62:0/3398442335 learned_addr learned my addr 172.21.15.62:0/3398442335 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:47.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.438+0000 7f0a6a000640 1 --2- 172.21.15.62:0/3398442335 >> v2:172.21.15.121:3300/0 conn(0x7f0a7406d9d0 0x7f0a7415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.438+0000 7f0a72000640 1 -- 172.21.15.62:0/3398442335 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126106501 0 0) 0x7f0a7415ee30 con 0x7f0a7415e5f0 2024-04-25T09:08:47.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.438+0000 7f0a72000640 1 -- 172.21.15.62:0/3398442335 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a58003520 con 0x7f0a7415e5f0 2024-04-25T09:08:47.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.439+0000 7f0a6a000640 1 -- 172.21.15.62:0/3398442335 >> v1:172.21.15.121:6789/0 conn(0x7f0a740ceb50 legacy=0x7f0a7406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.439+0000 7f0a6a000640 1 -- 172.21.15.62:0/3398442335 >> v1:172.21.15.62:6789/0 conn(0x7f0a7415e5f0 legacy=0x7f0a740b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.439+0000 7f0a72000640 1 -- 172.21.15.62:0/3398442335 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3872536351 0 0) 0x7f0a58003520 con 0x7f0a7415e5f0 2024-04-25T09:08:47.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.439+0000 7f0a6a000640 1 -- 172.21.15.62:0/3398442335 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7415ea40 con 0x7f0a7406d9d0 2024-04-25T09:08:47.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.440+0000 7f0a6a000640 1 --2- 172.21.15.62:0/3398442335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7406d9d0 0x7f0a7415e0b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0097a0 tx=0x7f0a5c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18e84fb993f35352 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:47.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.441+0000 7f0a72000640 1 -- 172.21.15.62:0/3398442335 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c037070 con 0x7f0a7406d9d0 2024-04-25T09:08:47.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.441+0000 7f0a72000640 1 -- 172.21.15.62:0/3398442335 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c02abc0 con 0x7f0a7406d9d0 2024-04-25T09:08:47.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.441+0000 7f0a72000640 1 -- 172.21.15.62:0/3398442335 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c001670 con 0x7f0a7406d9d0 2024-04-25T09:08:47.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.441+0000 7f0a73e00640 1 -- 172.21.15.62:0/3398442335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7406d9d0 msgr2=0x7f0a7415e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.441+0000 7f0a73e00640 1 --2- 172.21.15.62:0/3398442335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7406d9d0 0x7f0a7415e0b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0097a0 tx=0x7f0a5c02a130 comp rx=0 tx=0).stop 2024-04-25T09:08:47.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.442+0000 7f0a73e00640 1 -- 172.21.15.62:0/3398442335 shutdown_connections 2024-04-25T09:08:47.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.442+0000 7f0a73e00640 1 --2- 172.21.15.62:0/3398442335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7406d9d0 0x7f0a7415e0b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.442+0000 7f0a73e00640 1 -- 172.21.15.62:0/3398442335 >> 172.21.15.62:0/3398442335 conn(0x7f0a74066f20 msgr2=0x7f0a740b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:47.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.442+0000 7f0a73e00640 1 -- 172.21.15.62:0/3398442335 shutdown_connections 2024-04-25T09:08:47.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.442+0000 7f0a73e00640 1 -- 172.21.15.62:0/3398442335 wait complete. 2024-04-25T09:08:47.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.443+0000 7f0a73e00640 1 Processor -- start 2024-04-25T09:08:47.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.444+0000 7f0a73e00640 1 -- start start 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.444+0000 7f0a73e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7406d9d0 0x7f0a7413ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.444+0000 7f0a73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a740ceb50 0x7f0a74156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a73e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7415e5f0 0x7f0a7415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a73e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74162510 con 0x7f0a740ceb50 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a73e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74162390 con 0x7f0a7415e5f0 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a73e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a740b4c60 con 0x7f0a7406d9d0 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a6a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a740ceb50 0x7f0a74156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a6a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a740ceb50 0x7f0a74156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46234/0 (socket says 172.21.15.62:46234) 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a6a000640 1 -- 172.21.15.62:0/710823883 learned_addr learned my addr 172.21.15.62:0/710823883 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a72a00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7406d9d0 0x7f0a7413ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a73400640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7415e5f0 0x7f0a7415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a6a000640 1 -- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7406d9d0 msgr2=0x7f0a7413ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a6a000640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7406d9d0 0x7f0a7413ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a6a000640 1 -- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7415e5f0 msgr2=0x7f0a7415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a6a000640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7415e5f0 0x7f0a7415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a6a000640 1 -- 172.21.15.62:0/710823883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74157510 con 0x7f0a740ceb50 2024-04-25T09:08:47.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.445+0000 7f0a73400640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7415e5f0 0x7f0a7415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:47.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.446+0000 7f0a6a000640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a740ceb50 0x7f0a74156ba0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c035040 tx=0x7f0a5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:47.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.446+0000 7f0a6be00640 1 -- 172.21.15.62:0/710823883 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c041070 con 0x7f0a740ceb50 2024-04-25T09:08:47.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.446+0000 7f0a6be00640 1 -- 172.21.15.62:0/710823883 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c03c660 con 0x7f0a740ceb50 2024-04-25T09:08:47.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.446+0000 7f0a6be00640 1 -- 172.21.15.62:0/710823883 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c037040 con 0x7f0a740ceb50 2024-04-25T09:08:47.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.446+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a741577f0 con 0x7f0a740ceb50 2024-04-25T09:08:47.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.446+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7413b2c0 con 0x7f0a740ceb50 2024-04-25T09:08:47.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.447+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7406abe0 con 0x7f0a740ceb50 2024-04-25T09:08:47.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.448+0000 7f0a6be00640 1 -- 172.21.15.62:0/710823883 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a5c044070 con 0x7f0a740ceb50 2024-04-25T09:08:47.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.449+0000 7f0a6be00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0a5006b4d0 0x7f0a5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.449+0000 7f0a6be00640 1 -- 172.21.15.62:0/710823883 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a5c0aaa50 con 0x7f0a740ceb50 2024-04-25T09:08:47.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.450+0000 7f0a72a00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0a5006b4d0 0x7f0a5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.451+0000 7f0a72a00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0a5006b4d0 0x7f0a5006d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0a64001d40 tx=0x7f0a64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:47.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.453+0000 7f0a6be00640 1 -- 172.21.15.62:0/710823883 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a5c0798c0 con 0x7f0a740ceb50 2024-04-25T09:08:47.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.654+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a7404ef20 con 0x7f0a740ceb50 2024-04-25T09:08:47.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.655+0000 7f0a6be00640 1 -- 172.21.15.62:0/710823883 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a5c0791c0 con 0x7f0a740ceb50 2024-04-25T09:08:47.643 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:08:47.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0a5006b4d0 msgr2=0x7f0a5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0a5006b4d0 0x7f0a5006d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0a64001d40 tx=0x7f0a64009040 comp rx=0 tx=0).stop 2024-04-25T09:08:47.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a740ceb50 msgr2=0x7f0a74156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a740ceb50 0x7f0a74156ba0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c035040 tx=0x7f0a5c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 shutdown_connections 2024-04-25T09:08:47.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0a5006b4d0 0x7f0a5006d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a740ceb50 0x7f0a74156ba0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0a7415e5f0 0x7f0a7415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 --2- 172.21.15.62:0/710823883 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0a7406d9d0 0x7f0a7413ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.658+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 >> 172.21.15.62:0/710823883 conn(0x7f0a74066f20 msgr2=0x7f0a740b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:47.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.659+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 shutdown_connections 2024-04-25T09:08:47.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.659+0000 7f0a73e00640 1 -- 172.21.15.62:0/710823883 wait complete. 2024-04-25T09:08:47.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:08:47.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.895+0000 7f67e5800640 1 Processor -- start 2024-04-25T09:08:47.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.895+0000 7f67e5800640 1 -- start start 2024-04-25T09:08:47.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.896+0000 7f67e5800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f67e015dd50 0x7f67e00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.896+0000 7f67e5800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f67e00d2fb0 0x7f67e00ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.896+0000 7f67e5800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00ce780 con 0x7f67e00d3960 2024-04-25T09:08:47.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.896+0000 7f67e5800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e00d3440 con 0x7f67e00d2fb0 2024-04-25T09:08:47.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.896+0000 7f67e5800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e00ce1c0 con 0x7f67e015dd50 2024-04-25T09:08:47.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.897+0000 7f67d7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f67e00d2fb0 0x7f67e00ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.897+0000 7f67d7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f67e00d2fb0 0x7f67e00ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46242/0 (socket says 172.21.15.62:46242) 2024-04-25T09:08:47.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.897+0000 7f67d7e00640 1 -- 172.21.15.62:0/1627164876 learned_addr learned my addr 172.21.15.62:0/1627164876 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:47.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.897+0000 7f67dfe00640 1 --2- 172.21.15.62:0/1627164876 >> v2:172.21.15.121:3300/0 conn(0x7f67e015dd50 0x7f67e00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.897+0000 7f67d7e00640 1 -- 172.21.15.62:0/1627164876 >> v2:172.21.15.121:3300/0 conn(0x7f67e015dd50 msgr2=0x7f67e00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:47.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.897+0000 7f67d7e00640 1 --2- 172.21.15.62:0/1627164876 >> v2:172.21.15.121:3300/0 conn(0x7f67e015dd50 0x7f67e00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.897+0000 7f67d7e00640 1 -- 172.21.15.62:0/1627164876 >> v1:172.21.15.121:6789/0 conn(0x7f67e00d3960 legacy=0x7f67e00d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:47.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.897+0000 7f67d7e00640 1 -- 172.21.15.62:0/1627164876 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00ce4a0 con 0x7f67e00d2fb0 2024-04-25T09:08:47.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.899+0000 7f67d7e00640 1 --2- 172.21.15.62:0/1627164876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e00d2fb0 0x7f67e00ce9c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f67e00d1320 tx=0x7f67c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88240fbbceb2b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:47.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.899+0000 7f67df400640 1 -- 172.21.15.62:0/1627164876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8037070 con 0x7f67e00d2fb0 2024-04-25T09:08:47.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.899+0000 7f67df400640 1 -- 172.21.15.62:0/1627164876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c8032440 con 0x7f67e00d2fb0 2024-04-25T09:08:47.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.899+0000 7f67df400640 1 -- 172.21.15.62:0/1627164876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8032810 con 0x7f67e00d2fb0 2024-04-25T09:08:47.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.900+0000 7f67e5800640 1 -- 172.21.15.62:0/1627164876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e00d2fb0 msgr2=0x7f67e00ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.900+0000 7f67e5800640 1 --2- 172.21.15.62:0/1627164876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e00d2fb0 0x7f67e00ce9c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f67e00d1320 tx=0x7f67c802a230 comp rx=0 tx=0).stop 2024-04-25T09:08:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.900+0000 7f67e5800640 1 -- 172.21.15.62:0/1627164876 shutdown_connections 2024-04-25T09:08:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.900+0000 7f67e5800640 1 --2- 172.21.15.62:0/1627164876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e00d2fb0 0x7f67e00ce9c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.900+0000 7f67e5800640 1 --2- 172.21.15.62:0/1627164876 >> v2:172.21.15.121:3300/0 conn(0x7f67e015dd50 0x7f67e00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.900+0000 7f67e5800640 1 -- 172.21.15.62:0/1627164876 >> 172.21.15.62:0/1627164876 conn(0x7f67e006b000 msgr2=0x7f67e006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.900+0000 7f67e5800640 1 -- 172.21.15.62:0/1627164876 shutdown_connections 2024-04-25T09:08:47.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.900+0000 7f67e5800640 1 -- 172.21.15.62:0/1627164876 wait complete. 2024-04-25T09:08:47.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.903+0000 7f67e5800640 1 Processor -- start 2024-04-25T09:08:47.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.903+0000 7f67e5800640 1 -- start start 2024-04-25T09:08:47.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e5800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67e00d3960 0x7f67e01e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e5800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67e015dd50 0x7f67e01602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e01f36e0 0x7f67e01607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e5800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00b6b00 con 0x7f67e01f36e0 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e5800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00cef00 con 0x7f67e015dd50 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e5800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00b6c80 con 0x7f67e00d3960 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e01f36e0 0x7f67e01607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e01f36e0 0x7f67e01607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46258/0 (socket says 172.21.15.62:46258) 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.905+0000 7f67e4e00640 1 -- 172.21.15.62:0/2300522415 learned_addr learned my addr 172.21.15.62:0/2300522415 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.906+0000 7f67dfe00640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67e00d3960 0x7f67e01e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.906+0000 7f67d7e00640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67e015dd50 0x7f67e01602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.906+0000 7f67e4e00640 1 -- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67e00d3960 msgr2=0x7f67e01e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.906+0000 7f67e4e00640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67e00d3960 0x7f67e01e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.906+0000 7f67e4e00640 1 -- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67e015dd50 msgr2=0x7f67e01602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.906+0000 7f67e4e00640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67e015dd50 0x7f67e01602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.906+0000 7f67e4e00640 1 -- 172.21.15.62:0/2300522415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e015f200 con 0x7f67e01f36e0 2024-04-25T09:08:47.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.906+0000 7f67e4e00640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e01f36e0 0x7f67e01607e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c780 tx=0x7f67d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:47.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.907+0000 7f67dcc00640 1 -- 172.21.15.62:0/2300522415 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004b60 con 0x7f67e01f36e0 2024-04-25T09:08:47.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.907+0000 7f67dcc00640 1 -- 172.21.15.62:0/2300522415 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8004d00 con 0x7f67e01f36e0 2024-04-25T09:08:47.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.907+0000 7f67dcc00640 1 -- 172.21.15.62:0/2300522415 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d800aa90 con 0x7f67e01f36e0 2024-04-25T09:08:47.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.907+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e015f4e0 con 0x7f67e01f36e0 2024-04-25T09:08:47.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.907+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e015faa0 con 0x7f67e01f36e0 2024-04-25T09:08:47.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.908+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e004ef20 con 0x7f67e01f36e0 2024-04-25T09:08:47.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.909+0000 7f67dcc00640 1 -- 172.21.15.62:0/2300522415 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67d800ac30 con 0x7f67e01f36e0 2024-04-25T09:08:47.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.910+0000 7f67dcc00640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67bc06b4d0 0x7f67bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:47.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.910+0000 7f67dcc00640 1 -- 172.21.15.62:0/2300522415 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67d808a470 con 0x7f67e01f36e0 2024-04-25T09:08:47.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.911+0000 7f67dfe00640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67bc06b4d0 0x7f67bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:47.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.912+0000 7f67dfe00640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67bc06b4d0 0x7f67bc06d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f67d0006fd0 tx=0x7f67d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:47.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:47.915+0000 7f67dcc00640 1 -- 172.21.15.62:0/2300522415 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67d8020070 con 0x7f67e01f36e0 2024-04-25T09:08:48.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.115+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67e00b9950 con 0x7f67e01f36e0 2024-04-25T09:08:48.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.116+0000 7f67dcc00640 1 -- 172.21.15.62:0/2300522415 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f67d80592e0 con 0x7f67e01f36e0 2024-04-25T09:08:48.103 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67bc06b4d0 msgr2=0x7f67bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67bc06b4d0 0x7f67bc06d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f67d0006fd0 tx=0x7f67d0008040 comp rx=0 tx=0).stop 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e01f36e0 msgr2=0x7f67e01607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e01f36e0 0x7f67e01607e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c780 tx=0x7f67d800cc40 comp rx=0 tx=0).stop 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 shutdown_connections 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67bc06b4d0 0x7f67bc06d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67e01f36e0 0x7f67e01607e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67e015dd50 0x7f67e01602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 --2- 172.21.15.62:0/2300522415 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67e00d3960 0x7f67e01e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 >> 172.21.15.62:0/2300522415 conn(0x7f67e006b000 msgr2=0x7f67e00cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.118+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 shutdown_connections 2024-04-25T09:08:48.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.119+0000 7f67e5800640 1 -- 172.21.15.62:0/2300522415 wait complete. 2024-04-25T09:08:48.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:08:48.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.341+0000 7f5203000640 1 Processor -- start 2024-04-25T09:08:48.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.342+0000 7f5203000640 1 -- start start 2024-04-25T09:08:48.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.343+0000 7f5203000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0cfed0 0x7f51fc15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.343+0000 7f5203000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f51fc072340 0x7f51fc0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.343+0000 7f5203000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51fc0707f0 0x7f51fc15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.343+0000 7f5203000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc06c310 con 0x7f51fc0707f0 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.343+0000 7f5203000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc15f5f0 con 0x7f51fc072340 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.343+0000 7f5203000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc06c5b0 con 0x7f51fc0cfed0 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f5201000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51fc0707f0 0x7f51fc15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f5201000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51fc0707f0 0x7f51fc15b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46262/0 (socket says 172.21.15.62:46262) 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f5201000640 1 -- 172.21.15.62:0/2388066401 learned_addr learned my addr 172.21.15.62:0/2388066401 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f51fb400640 1 --2- 172.21.15.62:0/2388066401 >> v2:172.21.15.121:3300/0 conn(0x7f51fc072340 0x7f51fc0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f51fbe00640 1 --2- 172.21.15.62:0/2388066401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0cfed0 0x7f51fc15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f5201000640 1 -- 172.21.15.62:0/2388066401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0cfed0 msgr2=0x7f51fc15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f5201000640 1 --2- 172.21.15.62:0/2388066401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0cfed0 0x7f51fc15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f5201000640 1 -- 172.21.15.62:0/2388066401 >> v2:172.21.15.121:3300/0 conn(0x7f51fc072340 msgr2=0x7f51fc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f5201000640 1 --2- 172.21.15.62:0/2388066401 >> v2:172.21.15.121:3300/0 conn(0x7f51fc072340 0x7f51fc0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.344+0000 7f5201000640 1 -- 172.21.15.62:0/2388066401 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc06c890 con 0x7f51fc0707f0 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.346+0000 7f5201000640 1 --2- 172.21.15.62:0/2388066401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0707f0 0x7f51fc15b0f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0cd3d0 tx=0x7f51f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c5ba00ad57b855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.346+0000 7f51faa00640 1 -- 172.21.15.62:0/2388066401 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4037070 con 0x7f51fc0707f0 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.346+0000 7f51faa00640 1 -- 172.21.15.62:0/2388066401 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f4032440 con 0x7f51fc0707f0 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.346+0000 7f51faa00640 1 -- 172.21.15.62:0/2388066401 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4032810 con 0x7f51fc0707f0 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 -- 172.21.15.62:0/2388066401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0707f0 msgr2=0x7f51fc15b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 --2- 172.21.15.62:0/2388066401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0707f0 0x7f51fc15b0f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0cd3d0 tx=0x7f51f402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 -- 172.21.15.62:0/2388066401 shutdown_connections 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 --2- 172.21.15.62:0/2388066401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0707f0 0x7f51fc15b0f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 --2- 172.21.15.62:0/2388066401 >> v2:172.21.15.121:3300/0 conn(0x7f51fc072340 0x7f51fc0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 --2- 172.21.15.62:0/2388066401 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0cfed0 0x7f51fc15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 -- 172.21.15.62:0/2388066401 >> 172.21.15.62:0/2388066401 conn(0x7f51fc0668f0 msgr2=0x7f51fc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:48.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 -- 172.21.15.62:0/2388066401 shutdown_connections 2024-04-25T09:08:48.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.347+0000 7f5203000640 1 -- 172.21.15.62:0/2388066401 wait complete. 2024-04-25T09:08:48.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.349+0000 7f5203000640 1 Processor -- start 2024-04-25T09:08:48.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.349+0000 7f5203000640 1 -- start start 2024-04-25T09:08:48.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f5203000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0707f0 0x7f51fc0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f5203000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0cfed0 0x7f51fc142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f5203000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc13dda0 0x7f51fc13e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f5203000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc06d130 con 0x7f51fc0cfed0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f5203000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc06ce50 con 0x7f51fc13dda0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f5203000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc15b630 con 0x7f51fc0707f0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f51fb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0cfed0 0x7f51fc142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f51fb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0cfed0 0x7f51fc142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46272/0 (socket says 172.21.15.62:46272) 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f51fb400640 1 -- 172.21.15.62:0/1754722827 learned_addr learned my addr 172.21.15.62:0/1754722827 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f51fbe00640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0707f0 0x7f51fc0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.350+0000 7f5201000640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc13dda0 0x7f51fc13e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51fb400640 1 -- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0707f0 msgr2=0x7f51fc0cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51fb400640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0707f0 0x7f51fc0cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51fb400640 1 -- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc13dda0 msgr2=0x7f51fc13e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51fb400640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc13dda0 0x7f51fc13e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51fb400640 1 -- 172.21.15.62:0/1754722827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0cf7b0 con 0x7f51fc0cfed0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51fb400640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0cfed0 0x7f51fc142450 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f51e400d830 tx=0x7f51e400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51f3e00640 1 -- 172.21.15.62:0/1754722827 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4018020 con 0x7f51fc0cfed0 2024-04-25T09:08:48.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51f3e00640 1 -- 172.21.15.62:0/1754722827 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e40096a0 con 0x7f51fc0cfed0 2024-04-25T09:08:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f51f3e00640 1 -- 172.21.15.62:0/1754722827 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4009a70 con 0x7f51fc0cfed0 2024-04-25T09:08:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc13dae0 con 0x7f51fc0cfed0 2024-04-25T09:08:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.351+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc13d800 con 0x7f51fc0cfed0 2024-04-25T09:08:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.353+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc072c70 con 0x7f51fc0cfed0 2024-04-25T09:08:48.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.354+0000 7f51f3e00640 1 -- 172.21.15.62:0/1754722827 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51e401b020 con 0x7f51fc0cfed0 2024-04-25T09:08:48.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.354+0000 7f51f3e00640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51d806b4d0 0x7f51d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.355+0000 7f51f3e00640 1 -- 172.21.15.62:0/1754722827 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51e408a640 con 0x7f51fc0cfed0 2024-04-25T09:08:48.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.355+0000 7f51fbe00640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51d806b4d0 0x7f51d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.356+0000 7f51fbe00640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51d806b4d0 0x7f51d806d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009e90 tx=0x7f51ec00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.359+0000 7f51f3e00640 1 -- 172.21.15.62:0/1754722827 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51e4022030 con 0x7f51fc0cfed0 2024-04-25T09:08:48.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.568+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51fc04f000 con 0x7f51fc0cfed0 2024-04-25T09:08:48.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.569+0000 7f51f3e00640 1 -- 172.21.15.62:0/1754722827 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f51e40102e0 con 0x7f51fc0cfed0 2024-04-25T09:08:48.556 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:08:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.571+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51d806b4d0 msgr2=0x7f51d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.571+0000 7f5203000640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51d806b4d0 0x7f51d806d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009e90 tx=0x7f51ec00b040 comp rx=0 tx=0).stop 2024-04-25T09:08:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.571+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0cfed0 msgr2=0x7f51fc142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0cfed0 0x7f51fc142450 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f51e400d830 tx=0x7f51e400dcf0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 shutdown_connections 2024-04-25T09:08:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51d806b4d0 0x7f51d806d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51fc0cfed0 0x7f51fc142450 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc13dda0 0x7f51fc13e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 --2- 172.21.15.62:0/1754722827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51fc0707f0 0x7f51fc0cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 >> 172.21.15.62:0/1754722827 conn(0x7f51fc0668f0 msgr2=0x7f51fc0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 shutdown_connections 2024-04-25T09:08:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.572+0000 7f5203000640 1 -- 172.21.15.62:0/1754722827 wait complete. 2024-04-25T09:08:48.581 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:08:48.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:48.581 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:48.700 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:08:48.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:48.700 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:08:48.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:48.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.942+0000 7f64d3400640 1 Processor -- start 2024-04-25T09:08:48.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.942+0000 7f64d3400640 1 -- start start 2024-04-25T09:08:48.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.943+0000 7f64d3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64cc0cccf0 0x7f64cc0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.944+0000 7f64d3400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc0aed00 0x7f64cc0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.944+0000 7f64d3400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc06ca70 con 0x7f64cc15bd10 2024-04-25T09:08:48.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.944+0000 7f64d3400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc06d660 con 0x7f64cc0cccf0 2024-04-25T09:08:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.944+0000 7f64d3400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc15c0f0 con 0x7f64cc0aed00 2024-04-25T09:08:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.944+0000 7f64cbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64cc0cccf0 0x7f64cc0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.944+0000 7f64cbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f64cc0cccf0 0x7f64cc0ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46286/0 (socket says 172.21.15.62:46286) 2024-04-25T09:08:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.944+0000 7f64cbe00640 1 -- 172.21.15.62:0/721279655 learned_addr learned my addr 172.21.15.62:0/721279655 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.944+0000 7f64d1400640 1 --2- 172.21.15.62:0/721279655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc0aed00 0x7f64cc0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.945+0000 7f64cbe00640 1 -- 172.21.15.62:0/721279655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc0aed00 msgr2=0x7f64cc0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:48.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.945+0000 7f64cbe00640 1 --2- 172.21.15.62:0/721279655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc0aed00 0x7f64cc0cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.945+0000 7f64cbe00640 1 -- 172.21.15.62:0/721279655 >> v1:172.21.15.121:6789/0 conn(0x7f64cc15bd10 legacy=0x7f64cc0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.945+0000 7f64cbe00640 1 -- 172.21.15.62:0/721279655 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc06cd30 con 0x7f64cc0cccf0 2024-04-25T09:08:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.946+0000 7f64cbe00640 1 --2- 172.21.15.62:0/721279655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc0ae7c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009870 tx=0x7f64b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd7715a5f4b87b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.946+0000 7f64cb400640 1 -- 172.21.15.62:0/721279655 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8037070 con 0x7f64cc0cccf0 2024-04-25T09:08:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.946+0000 7f64cb400640 1 -- 172.21.15.62:0/721279655 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b8032440 con 0x7f64cc0cccf0 2024-04-25T09:08:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.946+0000 7f64cb400640 1 -- 172.21.15.62:0/721279655 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8032810 con 0x7f64cc0cccf0 2024-04-25T09:08:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.946+0000 7f64d3400640 1 -- 172.21.15.62:0/721279655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 msgr2=0x7f64cc0ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.946+0000 7f64d3400640 1 --2- 172.21.15.62:0/721279655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc0ae7c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009870 tx=0x7f64b802a270 comp rx=0 tx=0).stop 2024-04-25T09:08:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.947+0000 7f64d3400640 1 -- 172.21.15.62:0/721279655 shutdown_connections 2024-04-25T09:08:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.947+0000 7f64d3400640 1 --2- 172.21.15.62:0/721279655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc0aed00 0x7f64cc0cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.947+0000 7f64d3400640 1 --2- 172.21.15.62:0/721279655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc0ae7c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.947+0000 7f64d3400640 1 -- 172.21.15.62:0/721279655 >> 172.21.15.62:0/721279655 conn(0x7f64cc0668f0 msgr2=0x7f64cc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.947+0000 7f64d3400640 1 -- 172.21.15.62:0/721279655 shutdown_connections 2024-04-25T09:08:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.947+0000 7f64d3400640 1 -- 172.21.15.62:0/721279655 wait complete. 2024-04-25T09:08:48.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.948+0000 7f64d3400640 1 Processor -- start 2024-04-25T09:08:48.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.949+0000 7f64d3400640 1 -- start start 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64cc0aed00 0x7f64cc1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d3400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc15bd10 0x7f64cc15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d3400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc06daf0 con 0x7f64cc0cccf0 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d3400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc06d2f0 con 0x7f64cc0aed00 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d3400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1f5080 con 0x7f64cc15bd10 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46288/0 (socket says 172.21.15.62:46288) 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64cbe00640 1 -- 172.21.15.62:0/1283737308 learned_addr learned my addr 172.21.15.62:0/1283737308 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d0a00640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64cc0aed00 0x7f64cc1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64cbe00640 1 -- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc15bd10 msgr2=0x7f64cc15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d1400640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc15bd10 0x7f64cc15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64cbe00640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc15bd10 0x7f64cc15a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64cbe00640 1 -- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64cc0aed00 msgr2=0x7f64cc1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64cbe00640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64cc0aed00 0x7f64cc1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64cbe00640 1 -- 172.21.15.62:0/1283737308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc1529a0 con 0x7f64cc0cccf0 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.950+0000 7f64d0a00640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64cc0aed00 0x7f64cc1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.951+0000 7f64cbe00640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc159eb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f64b8000c00 tx=0x7f64b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.951+0000 7f64c8c00640 1 -- 172.21.15.62:0/1283737308 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8041020 con 0x7f64cc0cccf0 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.951+0000 7f64c8c00640 1 -- 172.21.15.62:0/1283737308 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b80306c0 con 0x7f64cc0cccf0 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.951+0000 7f64c8c00640 1 -- 172.21.15.62:0/1283737308 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8037040 con 0x7f64cc0cccf0 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.951+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc152c80 con 0x7f64cc0cccf0 2024-04-25T09:08:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.951+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc153240 con 0x7f64cc0cccf0 2024-04-25T09:08:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.952+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc1f1dd0 con 0x7f64cc0cccf0 2024-04-25T09:08:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.953+0000 7f64c8c00640 1 -- 172.21.15.62:0/1283737308 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64b8044020 con 0x7f64cc0cccf0 2024-04-25T09:08:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.954+0000 7f64c8c00640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f64ac06b4d0 0x7f64ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.954+0000 7f64c8c00640 1 -- 172.21.15.62:0/1283737308 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64b80aaac0 con 0x7f64cc0cccf0 2024-04-25T09:08:48.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.955+0000 7f64d0a00640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f64ac06b4d0 0x7f64ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:48.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.956+0000 7f64d0a00640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f64ac06b4d0 0x7f64ac06d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64bc000f60 tx=0x7f64bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:48.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:48.959+0000 7f64c8c00640 1 -- 172.21.15.62:0/1283737308 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64b804b030 con 0x7f64cc0cccf0 2024-04-25T09:08:49.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.186+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64cc0b43d0 con 0x7f64cc0cccf0 2024-04-25T09:08:49.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.187+0000 7f64c8c00640 1 -- 172.21.15.62:0/1283737308 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f64cc0b43d0 con 0x7f64cc0cccf0 2024-04-25T09:08:49.175 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:49.175 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3737740461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3737740461},{"type":"v1","addr":"172.21.15.62:6835","nonce":3737740461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":5099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/4172870608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4172870608},{"type":"v1","addr":"172.21.15.121:6836","nonce":4172870608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":5102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2112486228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":2112486228},{"type":"v1","addr":"172.21.15.121:6837","nonce":2112486228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:40.023899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/3491368038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3491368038},{"type":"v1","addr":"172.21.15.62:6834","nonce":3491368038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:08:49.175 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 38 2024-04-25T09:08:49.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f64ac06b4d0 msgr2=0x7f64ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f64ac06b4d0 0x7f64ac06d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64bc000f60 tx=0x7f64bc009040 comp rx=0 tx=0).stop 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 msgr2=0x7f64cc159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc159eb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f64b8000c00 tx=0x7f64b80046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 shutdown_connections 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f64ac06b4d0 0x7f64ac06d990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f64cc0cccf0 0x7f64cc159eb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f64cc15bd10 0x7f64cc15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 --2- 172.21.15.62:0/1283737308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64cc0aed00 0x7f64cc1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 >> 172.21.15.62:0/1283737308 conn(0x7f64cc0668f0 msgr2=0x7f64cc0cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 shutdown_connections 2024-04-25T09:08:49.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.190+0000 7f64d3400640 1 -- 172.21.15.62:0/1283737308 wait complete. 2024-04-25T09:08:49.196 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T09:08:49.427 INFO:teuthology.orchestra.run.smithi062.stdout:{"message":"","return_code":0} 2024-04-25T09:08:49.428 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:08:49.429 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:08:49.429 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:49.429 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:08:49.429 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:49.429 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:08:49.430 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:49.430 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:08:49.430 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:08:49.448 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:08:49.448 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:49.449 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:08:49.449 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:49.451 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:08:49.451 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:49.453 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:08:49.453 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:49.454 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T09:08:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.612+0000 7f7fdaf08f40 1 Processor -- start 2024-04-25T09:08:49.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.612+0000 7f7fdaf08f40 1 -- start start 2024-04-25T09:08:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.612+0000 7f7fdaf08f40 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fbc0022d0 0x55f2d28add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.612+0000 7f7fdaf08f40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2d28c8770 0x55f2d28c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fdaf08f40 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fdaf08f40 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d2798dd0 con 0x55f2d2820950 2024-04-25T09:08:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fdaf08f40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d279cd30 con 0x55f2d28c8770 2024-04-25T09:08:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fdaf08f40 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2d28ca4f0 con 0x7f7fbc0022d0 2024-04-25T09:08:49.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46314/0 (socket says 172.21.15.62:46314) 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd8c00640 1 -- 172.21.15.62:0/3524858849 learned_addr learned my addr 172.21.15.62:0/3524858849 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd8c00640 1 -- 172.21.15.62:0/3524858849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fbc0022d0 msgr2=0x55f2d28add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd8c00640 1 --2- 172.21.15.62:0/3524858849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fbc0022d0 0x55f2d28add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd7800640 1 --2- 172.21.15.62:0/3524858849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2d28c8770 0x55f2d28c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd8c00640 1 -- 172.21.15.62:0/3524858849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2d28c8770 msgr2=0x55f2d28c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd8c00640 1 --2- 172.21.15.62:0/3524858849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2d28c8770 0x55f2d28c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.613+0000 7f7fd8c00640 1 -- 172.21.15.62:0/3524858849 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2d28ae250 con 0x55f2d2820950 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fd8c00640 1 --2- 172.21.15.62:0/3524858849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28c7a20 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x55f2d283d610 tx=0x7f7fd0004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9053eb6529b919c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fd6400640 1 -- 172.21.15.62:0/3524858849 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000f510 con 0x55f2d2820950 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fd6400640 1 -- 172.21.15.62:0/3524858849 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd000f6b0 con 0x55f2d2820950 2024-04-25T09:08:49.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fd6400640 1 -- 172.21.15.62:0/3524858849 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000faa0 con 0x55f2d2820950 2024-04-25T09:08:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/3524858849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 msgr2=0x55f2d28c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fdaf08f40 1 --2- 172.21.15.62:0/3524858849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28c7a20 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x55f2d283d610 tx=0x7f7fd0004830 comp rx=0 tx=0).stop 2024-04-25T09:08:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/3524858849 shutdown_connections 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fdaf08f40 1 --2- 172.21.15.62:0/3524858849 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28c7a20 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fdaf08f40 1 --2- 172.21.15.62:0/3524858849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f2d28c8770 0x55f2d28c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fdaf08f40 1 --2- 172.21.15.62:0/3524858849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fbc0022d0 0x55f2d28add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.614+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/3524858849 >> 172.21.15.62:0/3524858849 conn(0x55f2d28212b0 msgr2=0x55f2d28c6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.615+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/3524858849 shutdown_connections 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.615+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/3524858849 wait complete. 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.615+0000 7f7fdaf08f40 1 Processor -- start 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.615+0000 7f7fdaf08f40 1 -- start start 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.616+0000 7f7fdaf08f40 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.616+0000 7f7fdaf08f40 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2d28c8770 0x55f2d28ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.616+0000 7f7fdaf08f40 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fbc0022d0 0x55f2d2822750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.616+0000 7f7fdaf08f40 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d2822330 con 0x55f2d2820950 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.616+0000 7f7fdaf08f40 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2d28204c0 con 0x7f7fbc0022d0 2024-04-25T09:08:49.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.616+0000 7f7fdaf08f40 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2d281f3b0 con 0x55f2d28c8770 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46326/0 (socket says 172.21.15.62:46326) 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8200640 1 -- 172.21.15.62:0/1673852550 learned_addr learned my addr 172.21.15.62:0/1673852550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd7800640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2d28c8770 0x55f2d28ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8c00640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fbc0022d0 0x55f2d2822750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8200640 1 -- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2d28c8770 msgr2=0x55f2d28ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8200640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f2d28c8770 0x55f2d28ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8200640 1 -- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fbc0022d0 msgr2=0x55f2d2822750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8200640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fbc0022d0 0x55f2d2822750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.617+0000 7f7fd8200640 1 -- 172.21.15.62:0/1673852550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2d28ae800 con 0x55f2d2820950 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.618+0000 7f7fd8200640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28b0c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8025880 tx=0x7f7fc8045950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.618+0000 7f7fd5000640 1 -- 172.21.15.62:0/1673852550 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8001c20 con 0x55f2d2820950 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.618+0000 7f7fd5000640 1 -- 172.21.15.62:0/1673852550 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc80543f0 con 0x55f2d2820950 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.618+0000 7f7fd5000640 1 -- 172.21.15.62:0/1673852550 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80014e0 con 0x55f2d2820950 2024-04-25T09:08:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.618+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f2d28aeae0 con 0x55f2d2820950 2024-04-25T09:08:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.618+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2d28af0a0 con 0x55f2d2820950 2024-04-25T09:08:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.620+0000 7f7fd5000640 1 -- 172.21.15.62:0/1673852550 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fc80633f0 con 0x55f2d2820950 2024-04-25T09:08:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.621+0000 7f7fd5000640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fb8069eb0 0x7f7fb806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.621+0000 7f7fd5000640 1 -- 172.21.15.62:0/1673852550 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fc80cbfe0 con 0x55f2d2820950 2024-04-25T09:08:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.621+0000 7f7fdaf08f40 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x55f2d2963300 0x55f2d2963710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.621+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 --> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] -- osd_op(unknown.0.0:1 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62) v9 -- 0x55f2d295f7d0 con 0x55f2d2963300 2024-04-25T09:08:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.622+0000 7f7fd8c00640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x55f2d2963300 0x55f2d2963710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.622+0000 7f7fd7800640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fb8069eb0 0x7f7fb806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.623+0000 7f7fd8c00640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x55f2d2963300 0x55f2d2963710 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.623+0000 7f7fd7800640 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fb8069eb0 0x7f7fb806c370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005630 tx=0x7f7fc40054e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.624+0000 7f7fd8c00640 1 -- 172.21.15.62:0/1673852550 <== osd.3 v2:172.21.15.62:6817/3104548384 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v62'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7fd001c040 con 0x55f2d2963300 2024-04-25T09:08:49.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.624+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x55f2d2963300 msgr2=0x55f2d2963710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.624+0000 7f7fdaf08f40 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x55f2d2963300 0x55f2d2963710 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.624+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fb8069eb0 msgr2=0x7f7fb806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.624+0000 7f7fdaf08f40 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fb8069eb0 0x7f7fb806c370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005630 tx=0x7f7fc40054e0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.624+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 msgr2=0x55f2d28b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.624+0000 7f7fdaf08f40 1 --2- 172.21.15.62:0/1673852550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f2d2820950 0x55f2d28b0c80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8025880 tx=0x7f7fc8045950 comp rx=0 tx=0).stop 2024-04-25T09:08:49.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.625+0000 7f7fd8c00640 1 -- 172.21.15.62:0/1673852550 reap_dead start 2024-04-25T09:08:49.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.625+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 shutdown_connections 2024-04-25T09:08:49.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.625+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 >> 172.21.15.62:0/1673852550 conn(0x55f2d28212b0 msgr2=0x55f2d28238f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:49.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.625+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 shutdown_connections 2024-04-25T09:08:49.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.625+0000 7f7fdaf08f40 1 -- 172.21.15.62:0/1673852550 wait complete. 2024-04-25T09:08:49.617 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T09:08:49.617 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T09:08:49.617 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T09:08:49.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T09:08:49.618 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T09:08:49.618 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T09:08:49.618 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T09:08:49.619 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T09:08:49.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.810+0000 7f97a2a00640 1 Processor -- start 2024-04-25T09:08:49.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.810+0000 7f97a2a00640 1 -- start start 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.811+0000 7f97a2a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f979c161230 0x7f979c0b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.811+0000 7f97a2a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f979c0d0fb0 0x7f979c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.811+0000 7f97a2a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c0cea70 con 0x7f979c0b5a60 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.811+0000 7f97a2a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c0d2040 con 0x7f979c0d0fb0 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.811+0000 7f97a2a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c0ced50 con 0x7f979c161230 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.811+0000 7f97a0c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f979c0d0fb0 0x7f979c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.811+0000 7f97a0c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f979c0d0fb0 0x7f979c0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46328/0 (socket says 172.21.15.62:46328) 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.811+0000 7f97a0c00640 1 -- 172.21.15.62:0/2713085600 learned_addr learned my addr 172.21.15.62:0/2713085600 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.812+0000 7f97a1600640 1 --2- 172.21.15.62:0/2713085600 >> v2:172.21.15.121:3300/0 conn(0x7f979c161230 0x7f979c0b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.812+0000 7f97a0c00640 1 -- 172.21.15.62:0/2713085600 >> v2:172.21.15.121:3300/0 conn(0x7f979c161230 msgr2=0x7f979c0b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:49.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.812+0000 7f97a0c00640 1 --2- 172.21.15.62:0/2713085600 >> v2:172.21.15.121:3300/0 conn(0x7f979c161230 0x7f979c0b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.812+0000 7f97a0c00640 1 -- 172.21.15.62:0/2713085600 >> v1:172.21.15.121:6789/0 conn(0x7f979c0b5a60 legacy=0x7f979c0d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:49.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.812+0000 7f97a0c00640 1 -- 172.21.15.62:0/2713085600 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0cf030 con 0x7f979c0d0fb0 2024-04-25T09:08:49.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.813+0000 7f97a0c00640 1 --2- 172.21.15.62:0/2713085600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c0d0fb0 0x7f979c0b5520 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f979c08cf60 tx=0x7f978802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3fb3bdef64edf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.814+0000 7f979be00640 1 -- 172.21.15.62:0/2713085600 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788037070 con 0x7f979c0d0fb0 2024-04-25T09:08:49.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.814+0000 7f979be00640 1 -- 172.21.15.62:0/2713085600 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9788032440 con 0x7f979c0d0fb0 2024-04-25T09:08:49.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.814+0000 7f979be00640 1 -- 172.21.15.62:0/2713085600 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788032810 con 0x7f979c0d0fb0 2024-04-25T09:08:49.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.814+0000 7fd0a2400640 1 Processor -- start 2024-04-25T09:08:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7f97a2a00640 1 -- 172.21.15.62:0/2713085600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c0d0fb0 msgr2=0x7f979c0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7f97a2a00640 1 --2- 172.21.15.62:0/2713085600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c0d0fb0 0x7f979c0b5520 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f979c08cf60 tx=0x7f978802a230 comp rx=0 tx=0).stop 2024-04-25T09:08:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7fd0a2400640 1 -- start start 2024-04-25T09:08:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7f97a2a00640 1 -- 172.21.15.62:0/2713085600 shutdown_connections 2024-04-25T09:08:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7f97a2a00640 1 --2- 172.21.15.62:0/2713085600 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c0d0fb0 0x7f979c0b5520 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7f97a2a00640 1 --2- 172.21.15.62:0/2713085600 >> v2:172.21.15.121:3300/0 conn(0x7f979c161230 0x7f979c0b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7f97a2a00640 1 -- 172.21.15.62:0/2713085600 >> 172.21.15.62:0/2713085600 conn(0x7f979c066d10 msgr2=0x7f979c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7f97a2a00640 1 -- 172.21.15.62:0/2713085600 shutdown_connections 2024-04-25T09:08:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.815+0000 7f97a2a00640 1 -- 172.21.15.62:0/2713085600 wait complete. 2024-04-25T09:08:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.816+0000 7f59e7e00640 1 Processor -- start 2024-04-25T09:08:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.816+0000 7f59e7e00640 1 -- start start 2024-04-25T09:08:49.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7fd0a2400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c0cfed0 0x7fd09c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7fd0a2400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd09c0709c0 0x7fd09c0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7fd0a2400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09c06c310 con 0x7fd09c072340 2024-04-25T09:08:49.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7fd0a2400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09c15f5f0 con 0x7fd09c0709c0 2024-04-25T09:08:49.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7fd0a2400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c06c5b0 con 0x7fd09c0cfed0 2024-04-25T09:08:49.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7f59e7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f59e8161230 0x7f59e80b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7f59e7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59e80b5c30 0x7f59e8161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7f59e7e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e80cea70 con 0x7f59e80d0fb0 2024-04-25T09:08:49.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7f59e7e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e80d2040 con 0x7f59e8161230 2024-04-25T09:08:49.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7f59e7e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e80ced50 con 0x7f59e80b5c30 2024-04-25T09:08:49.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7f97a2a00640 1 Processor -- start 2024-04-25T09:08:49.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7f59e6a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f59e8161230 0x7f59e80b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7fd09aa00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd09c072340 0x7fd09c0702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:38018/0 (socket says 172.21.15.62:38018) 2024-04-25T09:08:49.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7fd09aa00640 1 -- 172.21.15.62:0/2742321491 learned_addr learned my addr 172.21.15.62:0/2742321491 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:49.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.817+0000 7f59e6a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f59e8161230 0x7f59e80b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46342/0 (socket says 172.21.15.62:46342) 2024-04-25T09:08:49.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.818+0000 7f97a2a00640 1 -- start start 2024-04-25T09:08:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.818+0000 7fd09b400640 1 --2- 172.21.15.62:0/2742321491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c0cfed0 0x7fd09c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.818+0000 7f59e6a00640 1 -- 172.21.15.62:0/1846538508 learned_addr learned my addr 172.21.15.62:0/1846538508 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.818+0000 7fd09be00640 1 --2- 172.21.15.62:0/2742321491 >> v2:172.21.15.121:3300/0 conn(0x7fd09c0709c0 0x7fd09c0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.818+0000 7f59e6a00640 1 -- 172.21.15.62:0/1846538508 >> v2:172.21.15.121:3300/0 conn(0x7f59e80b5c30 msgr2=0x7f59e8161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.818+0000 7fac14a00640 1 Processor -- start 2024-04-25T09:08:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.818+0000 7f59e6a00640 1 --2- 172.21.15.62:0/1846538508 >> v2:172.21.15.121:3300/0 conn(0x7f59e80b5c30 0x7f59e8161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.818+0000 7fac14a00640 1 -- start start 2024-04-25T09:08:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f59e6a00640 1 -- 172.21.15.62:0/1846538508 >> v1:172.21.15.62:6789/0 conn(0x7f59e80d0fb0 legacy=0x7f59e80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f59e6a00640 1 -- 172.21.15.62:0/1846538508 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e80cf030 con 0x7f59e8161230 2024-04-25T09:08:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f59e6000640 1 -- 172.21.15.62:0/1846538508 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214397090 0 0) 0x7f59e80cea70 con 0x7f59e80d0fb0 2024-04-25T09:08:49.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7fd09b400640 1 -- 172.21.15.62:0/2742321491 >> v2:172.21.15.121:3300/0 conn(0x7fd09c0709c0 msgr2=0x7fd09c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7fd09b400640 1 --2- 172.21.15.62:0/2742321491 >> v2:172.21.15.121:3300/0 conn(0x7fd09c0709c0 0x7fd09c0cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7fd09b400640 1 -- 172.21.15.62:0/2742321491 >> v1:172.21.15.121:6789/0 conn(0x7fd09c072340 legacy=0x7fd09c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7fd09b400640 1 -- 172.21.15.62:0/2742321491 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c06c890 con 0x7fd09c0cfed0 2024-04-25T09:08:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f97a2a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c0b5a60 0x7f979c1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f97a2a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f979c0d0fb0 0x7f979c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f97a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c161230 0x7f979c14b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f97a2a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c161610 con 0x7f979c161230 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f97a2a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0ca8d0 con 0x7f979c0b5a60 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f97a2a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c161790 con 0x7f979c0d0fb0 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.819+0000 7f97a2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c161230 0x7f979c14b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c161230 0x7f979c14b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46350/0 (socket says 172.21.15.62:46350) 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a2000640 1 -- 172.21.15.62:0/1425690939 learned_addr learned my addr 172.21.15.62:0/1425690939 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a0c00640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f979c0d0fb0 0x7f979c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a1600640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c0b5a60 0x7f979c1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f59e6a00640 1 --2- 172.21.15.62:0/1846538508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e8161230 0x7f59e80b6c40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f59e80d2570 tx=0x7f59d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8208a9d8856d3302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7fd09b400640 1 --2- 172.21.15.62:0/2742321491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c0cfed0 0x7fd09c15f0b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0097a0 tx=0x7fd08c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cfc73fcb055f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a2000640 1 -- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f979c0d0fb0 msgr2=0x7f979c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a2000640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f979c0d0fb0 0x7f979c0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a2000640 1 -- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c0b5a60 msgr2=0x7f979c1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7fac14a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fac100b0cc0 0x7fac100c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7fac14a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fac10169bb0 0x7fac1016bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7fac14a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac100c5170 con 0x7fac100c4850 2024-04-25T09:08:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7fac14a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1006bad0 con 0x7fac100b0cc0 2024-04-25T09:08:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a2000640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c0b5a60 0x7f979c1594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a2000640 1 -- 172.21.15.62:0/1425690939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0c8d50 con 0x7f979c161230 2024-04-25T09:08:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7f97a1600640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c0b5a60 0x7f979c1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7f59e6000640 1 -- 172.21.15.62:0/1846538508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8037070 con 0x7f59e8161230 2024-04-25T09:08:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7f59e6000640 1 -- 172.21.15.62:0/1846538508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d8032440 con 0x7f59e8161230 2024-04-25T09:08:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7f59e6000640 1 -- 172.21.15.62:0/1846538508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8032810 con 0x7f59e8161230 2024-04-25T09:08:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.820+0000 7fac14a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1006ad10 con 0x7fac10169bb0 2024-04-25T09:08:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fac0ea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fac100b0cc0 0x7fac100c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fac0ea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fac100b0cc0 0x7fac100c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46364/0 (socket says 172.21.15.62:46364) 2024-04-25T09:08:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fac0ea00640 1 -- 172.21.15.62:0/3121678096 learned_addr learned my addr 172.21.15.62:0/3121678096 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7f97a2000640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c161230 0x7f979c14b990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f979400c780 tx=0x7f979400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fac0ea00640 1 -- 172.21.15.62:0/3121678096 >> v2:172.21.15.121:3300/0 conn(0x7fac10169bb0 msgr2=0x7fac1016bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fac0ea00640 1 --2- 172.21.15.62:0/3121678096 >> v2:172.21.15.121:3300/0 conn(0x7fac10169bb0 0x7fac1016bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fac0ea00640 1 -- 172.21.15.62:0/3121678096 >> v1:172.21.15.121:6789/0 conn(0x7fac100c4850 legacy=0x7fac1006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:49.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fac0ea00640 1 -- 172.21.15.62:0/3121678096 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1006aff0 con 0x7fac100b0cc0 2024-04-25T09:08:49.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fd09a000640 1 -- 172.21.15.62:0/2742321491 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c037070 con 0x7fd09c0cfed0 2024-04-25T09:08:49.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fd09a000640 1 -- 172.21.15.62:0/2742321491 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c032440 con 0x7fd09c0cfed0 2024-04-25T09:08:49.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.821+0000 7fd09a000640 1 -- 172.21.15.62:0/2742321491 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c032810 con 0x7fd09c0cfed0 2024-04-25T09:08:49.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f9799600640 1 -- 172.21.15.62:0/1425690939 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794017070 con 0x7f979c161230 2024-04-25T09:08:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f9799600640 1 -- 172.21.15.62:0/1425690939 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794003da0 con 0x7f979c161230 2024-04-25T09:08:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f9799600640 1 -- 172.21.15.62:0/1425690939 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97940052b0 con 0x7f979c161230 2024-04-25T09:08:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c0c81d0 con 0x7f979c161230 2024-04-25T09:08:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c0c8790 con 0x7f979c161230 2024-04-25T09:08:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f59e7e00640 1 -- 172.21.15.62:0/1846538508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e8161230 msgr2=0x7f59e80b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1846538508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e8161230 0x7f59e80b6c40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f59e80d2570 tx=0x7f59d802a270 comp rx=0 tx=0).stop 2024-04-25T09:08:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f59e7e00640 1 -- 172.21.15.62:0/1846538508 shutdown_connections 2024-04-25T09:08:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1846538508 >> v2:172.21.15.121:3300/0 conn(0x7f59e80b5c30 0x7f59e8161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1846538508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e8161230 0x7f59e80b6c40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f59e7e00640 1 -- 172.21.15.62:0/1846538508 >> 172.21.15.62:0/1846538508 conn(0x7f59e8066d10 msgr2=0x7f59e8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7fd0a2400640 1 -- 172.21.15.62:0/2742321491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c0cfed0 msgr2=0x7fd09c15f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7fd0a2400640 1 --2- 172.21.15.62:0/2742321491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c0cfed0 0x7fd09c15f0b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0097a0 tx=0x7fd08c02a180 comp rx=0 tx=0).stop 2024-04-25T09:08:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7fd0a2400640 1 -- 172.21.15.62:0/2742321491 shutdown_connections 2024-04-25T09:08:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7fd0a2400640 1 --2- 172.21.15.62:0/2742321491 >> v2:172.21.15.121:3300/0 conn(0x7fd09c0709c0 0x7fd09c0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7fd0a2400640 1 --2- 172.21.15.62:0/2742321491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c0cfed0 0x7fd09c15f0b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7fd0a2400640 1 -- 172.21.15.62:0/2742321491 >> 172.21.15.62:0/2742321491 conn(0x7fd09c0668f0 msgr2=0x7fd09c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:49.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.822+0000 7f59e7e00640 1 -- 172.21.15.62:0/1846538508 shutdown_connections 2024-04-25T09:08:49.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.823+0000 7fd0a2400640 1 -- 172.21.15.62:0/2742321491 shutdown_connections 2024-04-25T09:08:49.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.823+0000 7fac0ea00640 1 --2- 172.21.15.62:0/3121678096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac100b0cc0 0x7fac100c4c30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fac100b1310 tx=0x7fabfc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a439d4c6109b88e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.823+0000 7f59e7e00640 1 -- 172.21.15.62:0/1846538508 wait complete. 2024-04-25T09:08:49.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.823+0000 7fac0e000640 1 -- 172.21.15.62:0/3121678096 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc037070 con 0x7fac100b0cc0 2024-04-25T09:08:49.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.823+0000 7fd0a2400640 1 -- 172.21.15.62:0/2742321491 wait complete. 2024-04-25T09:08:49.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.823+0000 7fac0e000640 1 -- 172.21.15.62:0/3121678096 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc032440 con 0x7fac100b0cc0 2024-04-25T09:08:49.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.823+0000 7fac0e000640 1 -- 172.21.15.62:0/3121678096 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc032810 con 0x7fac100b0cc0 2024-04-25T09:08:49.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.823+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c04ef20 con 0x7f979c161230 2024-04-25T09:08:49.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.824+0000 7fac14a00640 1 -- 172.21.15.62:0/3121678096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac100b0cc0 msgr2=0x7fac100c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.824+0000 7fac14a00640 1 --2- 172.21.15.62:0/3121678096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac100b0cc0 0x7fac100c4c30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fac100b1310 tx=0x7fabfc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.824+0000 7fac14a00640 1 -- 172.21.15.62:0/3121678096 shutdown_connections 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.824+0000 7fac14a00640 1 --2- 172.21.15.62:0/3121678096 >> v2:172.21.15.121:3300/0 conn(0x7fac10169bb0 0x7fac1016bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.824+0000 7fac14a00640 1 --2- 172.21.15.62:0/3121678096 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac100b0cc0 0x7fac100c4c30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.824+0000 7fac14a00640 1 -- 172.21.15.62:0/3121678096 >> 172.21.15.62:0/3121678096 conn(0x7fac100b33d0 msgr2=0x7fac100b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.824+0000 7fac14a00640 1 -- 172.21.15.62:0/3121678096 shutdown_connections 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.825+0000 7fac14a00640 1 -- 172.21.15.62:0/3121678096 wait complete. 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.825+0000 7f9799600640 1 -- 172.21.15.62:0/1425690939 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f979401a070 con 0x7f979c161230 2024-04-25T09:08:49.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.825+0000 7fd0a2400640 1 Processor -- start 2024-04-25T09:08:49.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.825+0000 7f59e7e00640 1 Processor -- start 2024-04-25T09:08:49.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.825+0000 7f59e7e00640 1 -- start start 2024-04-25T09:08:49.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.825+0000 7fd0a2400640 1 -- start start 2024-04-25T09:08:49.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.826+0000 7fac14a00640 1 Processor -- start 2024-04-25T09:08:49.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.826+0000 7f9799600640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f977c06b5d0 0x7f977c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.826+0000 7f9799600640 1 -- 172.21.15.62:0/1425690939 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9794089810 con 0x7f979c161230 2024-04-25T09:08:49.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.826+0000 7fac14a00640 1 -- start start 2024-04-25T09:08:49.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e80b5c30 0x7f59e8158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59e80d0fb0 0x7f59e8156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7fd0a2400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd09c0709c0 0x7fd09c1391f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7fd0a2400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd09c0cfed0 0x7fd09c13be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e80c9820 0x7f59e814a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7f59e7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e8161610 con 0x7f59e80b5c30 2024-04-25T09:08:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7f59e7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e80ca8d0 con 0x7f59e80c9820 2024-04-25T09:08:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7f59e7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e8161790 con 0x7f59e80d0fb0 2024-04-25T09:08:49.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7fac14a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac100b0cc0 0x7fac100cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7fd0a2400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c13c370 0x7fd09c0aa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7f97a1600640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f977c06b5d0 0x7f977c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fd0a2400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c06d130 con 0x7fd09c0709c0 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fd0a2400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c06ce50 con 0x7fd09c0cfed0 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fd0a2400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0cc740 con 0x7fd09c13c370 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7fac14a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac100c4850 0x7fac100ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7fac14a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac10169bb0 0x7fac1015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fac0fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac10169bb0 0x7fac1015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fac0fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac10169bb0 0x7fac1015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46400/0 (socket says 172.21.15.62:46400) 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fac0fe00640 1 -- 172.21.15.62:0/441521190 learned_addr learned my addr 172.21.15.62:0/441521190 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1006ed20 con 0x7fac10169bb0 2024-04-25T09:08:49.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1006b5b0 con 0x7fac100c4850 2024-04-25T09:08:49.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac100af6c0 con 0x7fac100b0cc0 2024-04-25T09:08:49.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.827+0000 7fd09b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd09c0709c0 0x7fd09c1391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fd09b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd09c0709c0 0x7fd09c1391f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46390/0 (socket says 172.21.15.62:46390) 2024-04-25T09:08:49.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fd09b400640 1 -- 172.21.15.62:0/3130516567 learned_addr learned my addr 172.21.15.62:0/3130516567 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7f59e7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e80c9820 0x7f59e814a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fac0ea00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac100c4850 0x7fac100ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fac0f400640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac100b0cc0 0x7fac100cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7f59e6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e80b5c30 0x7f59e8158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7f59e6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e80b5c30 0x7f59e8158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46378/0 (socket says 172.21.15.62:46378) 2024-04-25T09:08:49.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7f59e7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e80c9820 0x7f59e814a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:36798/0 (socket says 172.21.15.62:36798) 2024-04-25T09:08:49.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7f59e6a00640 1 -- 172.21.15.62:0/1657138615 learned_addr learned my addr 172.21.15.62:0/1657138615 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:49.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7f59dfe00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59e80d0fb0 0x7f59e8156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fac0fe00640 1 -- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac100b0cc0 msgr2=0x7fac100cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fd09be00640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c13c370 0x7fd09c0aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.828+0000 7fd09aa00640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd09c0cfed0 0x7fd09c13be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fac0fe00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac100b0cc0 0x7fac100cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fac0fe00640 1 -- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac100c4850 msgr2=0x7fac100ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fac0fe00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac100c4850 0x7fac100ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fd09b400640 1 -- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c13c370 msgr2=0x7fd09c0aa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fd09b400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c13c370 0x7fd09c0aa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fd09b400640 1 -- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd09c0cfed0 msgr2=0x7fd09c13be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fd09b400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd09c0cfed0 0x7fd09c13be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fac0fe00640 1 -- 172.21.15.62:0/441521190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1015d500 con 0x7fac10169bb0 2024-04-25T09:08:49.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fd09b400640 1 -- 172.21.15.62:0/3130516567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c139730 con 0x7fd09c0709c0 2024-04-25T09:08:49.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fd09be00640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c13c370 0x7fd09c0aa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:49.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fac0f400640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac100b0cc0 0x7fac100cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:49.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7f59e6a00640 1 -- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59e80d0fb0 msgr2=0x7f59e8156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7f59e6a00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59e80d0fb0 0x7f59e8156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7f59e6a00640 1 -- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e80c9820 msgr2=0x7f59e814a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:49.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7f59e6a00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e80c9820 0x7f59e814a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:49.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7f59e6a00640 1 -- 172.21.15.62:0/1657138615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e81571b0 con 0x7f59e80b5c30 2024-04-25T09:08:49.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fd09b400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd09c0709c0 0x7fd09c1391f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0098c0 tx=0x7fd08c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7f97a1600640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f977c06b5d0 0x7f977c06da90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9790006fd0 tx=0x7f9790008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7f59e6a00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e80b5c30 0x7f59e8158fd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009960 tx=0x7f59d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.829+0000 7fac0fe00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac10169bb0 0x7fac1015cfc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fac0800ba10 tx=0x7fac0800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.831+0000 7fd093400640 1 -- 172.21.15.62:0/3130516567 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c037040 con 0x7fd09c0709c0 2024-04-25T09:08:49.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.831+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09c15e420 con 0x7fd09c0709c0 2024-04-25T09:08:49.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.831+0000 7fac07400640 1 -- 172.21.15.62:0/441521190 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08019070 con 0x7fac10169bb0 2024-04-25T09:08:49.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c15e9e0 con 0x7fd09c0709c0 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7fd093400640 1 -- 172.21.15.62:0/3130516567 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c030500 con 0x7fd09c0709c0 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7fd093400640 1 -- 172.21.15.62:0/3130516567 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0308d0 con 0x7fd09c0709c0 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7f59dea00640 1 -- 172.21.15.62:0/1657138615 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8041020 con 0x7f59e80b5c30 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7f59dea00640 1 -- 172.21.15.62:0/1657138615 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d8030760 con 0x7f59e80b5c30 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7f59dea00640 1 -- 172.21.15.62:0/1657138615 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8037040 con 0x7f59e80b5c30 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac100cc200 con 0x7fac10169bb0 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac100cc7c0 con 0x7fac10169bb0 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7fac07400640 1 -- 172.21.15.62:0/441521190 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0800cb60 con 0x7fac10169bb0 2024-04-25T09:08:49.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7fac07400640 1 -- 172.21.15.62:0/441521190 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08014440 con 0x7fac10169bb0 2024-04-25T09:08:49.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e8157490 con 0x7f59e80b5c30 2024-04-25T09:08:49.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e815ae90 con 0x7f59e80b5c30 2024-04-25T09:08:49.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.832+0000 7f9799600640 1 -- 172.21.15.62:0/1425690939 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9794058020 con 0x7f979c161230 2024-04-25T09:08:49.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.833+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd074005290 con 0x7fd09c0709c0 2024-04-25T09:08:49.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.833+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe8005290 con 0x7fac10169bb0 2024-04-25T09:08:49.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.834+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e804ef20 con 0x7f59e80b5c30 2024-04-25T09:08:49.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.835+0000 7fd093400640 1 -- 172.21.15.62:0/3130516567 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd08c044020 con 0x7fd09c0709c0 2024-04-25T09:08:49.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.837+0000 7fd093400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd07c06b4d0 0x7fd07c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.837+0000 7fd093400640 1 -- 172.21.15.62:0/3130516567 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd08c0aa520 con 0x7fd09c0709c0 2024-04-25T09:08:49.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.836+0000 7fac07400640 1 -- 172.21.15.62:0/441521190 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac0801c070 con 0x7fac10169bb0 2024-04-25T09:08:49.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.837+0000 7f59dea00640 1 -- 172.21.15.62:0/1657138615 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59d8044020 con 0x7f59e80b5c30 2024-04-25T09:08:49.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.837+0000 7fac07400640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabf006b500 0x7fabf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.837+0000 7fd09aa00640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd07c06b4d0 0x7fd07c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.837+0000 7fac07400640 1 -- 172.21.15.62:0/441521190 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac0808a300 con 0x7fac10169bb0 2024-04-25T09:08:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.838+0000 7f59dea00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59c406b550 0x7f59c406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.838+0000 7f59dea00640 1 -- 172.21.15.62:0/1657138615 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59d80aaf00 con 0x7f59e80b5c30 2024-04-25T09:08:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.838+0000 7f59dfe00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59c406b550 0x7f59c406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.839+0000 7fac0f400640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabf006b500 0x7fabf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:49.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.839+0000 7fd09aa00640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd07c06b4d0 0x7fd07c06d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0b1d80 tx=0x7fd088009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.839+0000 7f59dfe00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59c406b550 0x7f59c406da10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f59e80cddc0 tx=0x7f59d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.840+0000 7fac0f400640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabf006b500 0x7fabf006d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac00003eb0 tx=0x7fac00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:49.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.840+0000 7fac07400640 1 -- 172.21.15.62:0/441521190 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac08023020 con 0x7fac10169bb0 2024-04-25T09:08:49.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.845+0000 7f59dea00640 1 -- 172.21.15.62:0/1657138615 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59d804b030 con 0x7f59e80b5c30 2024-04-25T09:08:49.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:49.852+0000 7fd093400640 1 -- 172.21.15.62:0/3130516567 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd08c079300 con 0x7fd09c0709c0 2024-04-25T09:08:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.112+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fabe8005850 con 0x7fac10169bb0 2024-04-25T09:08:50.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.114+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f59e80cea60 con 0x7f59e80b5c30 2024-04-25T09:08:50.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.248+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f979c0ce640 con 0x7f979c161230 2024-04-25T09:08:50.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.269+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7fd074005580 con 0x7fd09c0709c0 2024-04-25T09:08:50.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.601+0000 7fac07400640 1 -- 172.21.15.62:0/441521190 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5102 v39) v1 ==== 95+0+0 (secure 0 0 0) 0x7fac08059160 con 0x7fac10169bb0 2024-04-25T09:08:50.588 INFO:teuthology.orchestra.run.smithi062.stderr:failed mds gid 5102 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabf006b500 msgr2=0x7fabf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabf006b500 0x7fabf006d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac00003eb0 tx=0x7fac00009040 comp rx=0 tx=0).stop 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac10169bb0 msgr2=0x7fac1015cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac10169bb0 0x7fac1015cfc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fac0800ba10 tx=0x7fac0800bed0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 shutdown_connections 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fabf006b500 0x7fabf006d9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac10169bb0 0x7fac1015cfc0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac100c4850 0x7fac100ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 --2- 172.21.15.62:0/441521190 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac100b0cc0 0x7fac100cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 >> 172.21.15.62:0/441521190 conn(0x7fac100b33d0 msgr2=0x7fac1006f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 shutdown_connections 2024-04-25T09:08:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.604+0000 7fac14a00640 1 -- 172.21.15.62:0/441521190 wait complete. 2024-04-25T09:08:50.600 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:08:50.613+0000 7f47bda00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:08:50.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.618+0000 7f59dea00640 1 -- 172.21.15.62:0/1657138615 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5099 v40) v1 ==== 95+0+0 (secure 0 0 0) 0x7f59d80797a0 con 0x7f59e80b5c30 2024-04-25T09:08:50.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.618+0000 7f9799600640 1 -- 172.21.15.62:0/1425690939 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6390 v40) v1 ==== 95+0+0 (secure 0 0 0) 0x7f979405c0c0 con 0x7f979c161230 2024-04-25T09:08:50.606 INFO:teuthology.orchestra.run.smithi062.stderr:failed mds gid 5099 2024-04-25T09:08:50.606 INFO:teuthology.orchestra.run.smithi062.stderr:failed mds gid 6390 2024-04-25T09:08:50.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f977c06b5d0 msgr2=0x7f977c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f977c06b5d0 0x7f977c06da90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9790006fd0 tx=0x7f9790008040 comp rx=0 tx=0).stop 2024-04-25T09:08:50.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c161230 msgr2=0x7f979c14b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c161230 0x7f979c14b990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f979400c780 tx=0x7f979400cc40 comp rx=0 tx=0).stop 2024-04-25T09:08:50.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59c406b550 msgr2=0x7f59c406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59c406b550 0x7f59c406da10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f59e80cddc0 tx=0x7f59d0009040 comp rx=0 tx=0).stop 2024-04-25T09:08:50.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e80b5c30 msgr2=0x7f59e8158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:50.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e80b5c30 0x7f59e8158fd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009960 tx=0x7f59d80046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 shutdown_connections 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f977c06b5d0 0x7f977c06da90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f979c161230 0x7f979c14b990 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f979c0d0fb0 0x7f979c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 --2- 172.21.15.62:0/1425690939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f979c0b5a60 0x7f979c1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 >> 172.21.15.62:0/1425690939 conn(0x7f979c066d10 msgr2=0x7f979c0cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 shutdown_connections 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59c406b550 0x7f59c406da10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e80c9820 0x7f59e814a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f59e80d0fb0 0x7f59e8156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 --2- 172.21.15.62:0/1657138615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59e80b5c30 0x7f59e8158fd0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:50.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.621+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 >> 172.21.15.62:0/1657138615 conn(0x7f59e8066d10 msgr2=0x7f59e80b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:50.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.622+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 shutdown_connections 2024-04-25T09:08:50.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.622+0000 7f97a2a00640 1 -- 172.21.15.62:0/1425690939 wait complete. 2024-04-25T09:08:50.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.622+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 shutdown_connections 2024-04-25T09:08:50.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:50.622+0000 7f59e7e00640 1 -- 172.21.15.62:0/1657138615 wait complete. 2024-04-25T09:08:50.615 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:08:50.615 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:08:50.625 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:08:50.625 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:50.630 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:08:50.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:08:50.630 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:08:50.631 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:08:50.637 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:08:50.637 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:50.677 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:08:50.678 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:50.776 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:08:50.783 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:08:50.844 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:08:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.621+0000 7fd093400640 1 -- 172.21.15.62:0/3130516567 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 5093 v41) v1 ==== 95+0+0 (secure 0 0 0) 0x7fd08c078c00 con 0x7fd09c0709c0 2024-04-25T09:08:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:failed mds gid 5093 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd07c06b4d0 msgr2=0x7fd07c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd07c06b4d0 0x7fd07c06d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0b1d80 tx=0x7fd088009040 comp rx=0 tx=0).stop 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd09c0709c0 msgr2=0x7fd09c1391f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd09c0709c0 0x7fd09c1391f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0098c0 tx=0x7fd08c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 shutdown_connections 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd07c06b4d0 0x7fd07c06d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd09c13c370 0x7fd09c0aa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd09c0cfed0 0x7fd09c13be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 --2- 172.21.15.62:0/3130516567 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd09c0709c0 0x7fd09c1391f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.625+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 >> 172.21.15.62:0/3130516567 conn(0x7fd09c0668f0 msgr2=0x7fd09c0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.626+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 shutdown_connections 2024-04-25T09:08:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.626+0000 7fd0a2400640 1 -- 172.21.15.62:0/3130516567 wait complete. 2024-04-25T09:08:51.635 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:08:51.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:08:51.682 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:08:51.682 DEBUG:teuthology.parallel:result is None 2024-04-25T09:08:51.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:51.834 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:08:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.867+0000 7f899cc00640 1 Processor -- start 2024-04-25T09:08:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.867+0000 7f899cc00640 1 -- start start 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f899cc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89980b56f0 0x7f8998160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f899cc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f89980b27a0 0x7f8998161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f899cc00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89980d3110 con 0x7f89980b6a90 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f899cc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8998161780 con 0x7f89980b56f0 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f899cc00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89980cedc0 con 0x7f89980b27a0 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f8997400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89980b56f0 0x7f8998160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f8997400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89980b56f0 0x7f8998160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46436/0 (socket says 172.21.15.62:46436) 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f8997400640 1 -- 172.21.15.62:0/3744831486 learned_addr learned my addr 172.21.15.62:0/3744831486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.868+0000 7f8996a00640 1 --1- 172.21.15.62:0/3744831486 >> v1:172.21.15.62:6789/0 conn(0x7f89980b6a90 0x7f8998161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35638/0 (socket says 172.21.15.62:35638) 2024-04-25T09:08:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.869+0000 7f8997400640 1 -- 172.21.15.62:0/3744831486 >> v2:172.21.15.121:3300/0 conn(0x7f89980b27a0 msgr2=0x7f8998161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.869+0000 7f8997e00640 1 --2- 172.21.15.62:0/3744831486 >> v2:172.21.15.121:3300/0 conn(0x7f89980b27a0 0x7f8998161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.869+0000 7f8997400640 1 --2- 172.21.15.62:0/3744831486 >> v2:172.21.15.121:3300/0 conn(0x7f89980b27a0 0x7f8998161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.869+0000 7f8997400640 1 -- 172.21.15.62:0/3744831486 >> v1:172.21.15.62:6789/0 conn(0x7f89980b6a90 legacy=0x7f8998161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.869+0000 7f8997400640 1 -- 172.21.15.62:0/3744831486 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980d33f0 con 0x7f89980b56f0 2024-04-25T09:08:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.869+0000 7f8997e00640 1 --2- 172.21.15.62:0/3744831486 >> v2:172.21.15.121:3300/0 conn(0x7f89980b27a0 0x7f8998161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.870+0000 7f8997400640 1 --2- 172.21.15.62:0/3744831486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b56f0 0x7f8998160d00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f89980b2720 tx=0x7f898802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a20091971ed1782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.870+0000 7f8996000640 1 -- 172.21.15.62:0/3744831486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988037070 con 0x7f89980b56f0 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.870+0000 7f8996000640 1 -- 172.21.15.62:0/3744831486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8988032440 con 0x7f89980b56f0 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.870+0000 7f8996000640 1 -- 172.21.15.62:0/3744831486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988032810 con 0x7f89980b56f0 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.871+0000 7f899cc00640 1 -- 172.21.15.62:0/3744831486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b56f0 msgr2=0x7f8998160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.871+0000 7f899cc00640 1 --2- 172.21.15.62:0/3744831486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b56f0 0x7f8998160d00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f89980b2720 tx=0x7f898802a270 comp rx=0 tx=0).stop 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.871+0000 7f899cc00640 1 -- 172.21.15.62:0/3744831486 shutdown_connections 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.871+0000 7f899cc00640 1 --2- 172.21.15.62:0/3744831486 >> v2:172.21.15.121:3300/0 conn(0x7f89980b27a0 0x7f8998161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.871+0000 7f899cc00640 1 --2- 172.21.15.62:0/3744831486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b56f0 0x7f8998160d00 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.871+0000 7f899cc00640 1 -- 172.21.15.62:0/3744831486 >> 172.21.15.62:0/3744831486 conn(0x7f8998066d10 msgr2=0x7f8998067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.871+0000 7f899cc00640 1 -- 172.21.15.62:0/3744831486 shutdown_connections 2024-04-25T09:08:51.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.871+0000 7f899cc00640 1 -- 172.21.15.62:0/3744831486 wait complete. 2024-04-25T09:08:51.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.873+0000 7f899cc00640 1 Processor -- start 2024-04-25T09:08:51.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.873+0000 7f899cc00640 1 -- start start 2024-04-25T09:08:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f899cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89980b27a0 0x7f89980ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f899cc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89980b56f0 0x7f89980aa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f899cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b6a90 0x7f89980b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f899cc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980b2d60 con 0x7f89980b6a90 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f899cc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980cea50 con 0x7f89980b27a0 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f899cc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89980b2ee0 con 0x7f89980b56f0 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f8997e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b6a90 0x7f89980b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f8997e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b6a90 0x7f89980b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46444/0 (socket says 172.21.15.62:46444) 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f8997e00640 1 -- 172.21.15.62:0/2745169360 learned_addr learned my addr 172.21.15.62:0/2745169360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f8997400640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89980b27a0 0x7f89980ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.874+0000 7f8996a00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89980b56f0 0x7f89980aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f8997e00640 1 -- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89980b56f0 msgr2=0x7f89980aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f8997e00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89980b56f0 0x7f89980aa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f8997e00640 1 -- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89980b27a0 msgr2=0x7f89980ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f8997e00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89980b27a0 0x7f89980ab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f8997e00640 1 -- 172.21.15.62:0/2745169360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89981f4810 con 0x7f89980b6a90 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f8996a00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89980b56f0 0x7f89980aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:51.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f8997e00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b6a90 0x7f89980b31d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f899000cea0 tx=0x7f899000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f898f400640 1 -- 172.21.15.62:0/2745169360 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990003b90 con 0x7f89980b6a90 2024-04-25T09:08:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f898f400640 1 -- 172.21.15.62:0/2745169360 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8990003d50 con 0x7f89980b6a90 2024-04-25T09:08:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f898f400640 1 -- 172.21.15.62:0/2745169360 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990004260 con 0x7f89980b6a90 2024-04-25T09:08:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89981f4af0 con 0x7f89980b6a90 2024-04-25T09:08:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.875+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89981f50b0 con 0x7f89980b6a90 2024-04-25T09:08:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.876+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899804ef20 con 0x7f89980b6a90 2024-04-25T09:08:51.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.877+0000 7f898f400640 1 -- 172.21.15.62:0/2745169360 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8990004400 con 0x7f89980b6a90 2024-04-25T09:08:51.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.878+0000 7f898f400640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f897806b500 0x7f897806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:51.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.878+0000 7f898f400640 1 -- 172.21.15.62:0/2745169360 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f899008aca0 con 0x7f89980b6a90 2024-04-25T09:08:51.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.879+0000 7f8997400640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f897806b500 0x7f897806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:51.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.880+0000 7f8997400640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f897806b500 0x7f897806d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8988034000 tx=0x7f898802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:51.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:51.882+0000 7f898f400640 1 -- 172.21.15.62:0/2745169360 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8990059a20 con 0x7f89980b6a90 2024-04-25T09:08:52.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.122+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89980b8240 con 0x7f89980b6a90 2024-04-25T09:08:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.123+0000 7f898f400640 1 -- 172.21.15.62:0/2745169360 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4151 (secure 0 0 0) 0x7f8990059320 con 0x7f89980b6a90 2024-04-25T09:08:52.111 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:52.111 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1396005512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1396005512},{"type":"v1","addr":"172.21.15.121:6835","nonce":1396005512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3589939339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3589939339},{"type":"v1","addr":"172.21.15.121:6837","nonce":3589939339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:51.625634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":42,"state":"up:replay","state_seq":1,"addr":"172.21.15.62:6833/836333339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":836333339},{"type":"v1","addr":"172.21.15.62:6833","nonce":836333339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:08:52.111 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 42 2024-04-25T09:08:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f897806b500 msgr2=0x7f897806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f897806b500 0x7f897806d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8988034000 tx=0x7f898802e000 comp rx=0 tx=0).stop 2024-04-25T09:08:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b6a90 msgr2=0x7f89980b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:52.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b6a90 0x7f89980b31d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f899000cea0 tx=0x7f899000cf90 comp rx=0 tx=0).stop 2024-04-25T09:08:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 shutdown_connections 2024-04-25T09:08:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f897806b500 0x7f897806d9c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89980b6a90 0x7f89980b31d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89980b56f0 0x7f89980aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 --2- 172.21.15.62:0/2745169360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89980b27a0 0x7f89980ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 >> 172.21.15.62:0/2745169360 conn(0x7f8998066d10 msgr2=0x7f89980b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 shutdown_connections 2024-04-25T09:08:52.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:52.126+0000 7f899cc00640 1 -- 172.21.15.62:0/2745169360 wait complete. 2024-04-25T09:08:52.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:08:36.958423+0000', 'modified': '2024-04-25T09:08:51.625634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6633}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6633': {'gid': 6633, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.62:6833/836333339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 836333339}, {'type': 'v1', 'addr': '172.21.15.62:6833', 'nonce': 836333339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:08:52.535 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6633:up:replay 2024-04-25T09:08:53.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.768+0000 7fb14be00640 1 Processor -- start 2024-04-25T09:08:53.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.768+0000 7fb14be00640 1 -- start start 2024-04-25T09:08:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.770+0000 7fb14be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb14c0b27a0 0x7fb14c0d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.770+0000 7fb14be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0d3110 con 0x7fb14c0b6a90 2024-04-25T09:08:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.770+0000 7fb14be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0d33f0 con 0x7fb14c0b56f0 2024-04-25T09:08:53.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.770+0000 7fb14be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14c0cedc0 con 0x7fb14c0b27a0 2024-04-25T09:08:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.770+0000 7fb142000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb14c0b6a90 0x7fb14c0d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35640/0 (socket says 172.21.15.62:35640) 2024-04-25T09:08:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.770+0000 7fb142000640 1 -- 172.21.15.62:0/1305696528 learned_addr learned my addr 172.21.15.62:0/1305696528 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.770+0000 7fb14b400640 1 --2- 172.21.15.62:0/1305696528 >> v2:172.21.15.62:3300/0 conn(0x7fb14c0b27a0 0x7fb14c0d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.771+0000 7fb14b400640 1 -- 172.21.15.62:0/1305696528 >> v1:172.21.15.121:6789/0 conn(0x7fb14c0b56f0 legacy=0x7fb14c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.771+0000 7fb14b400640 1 -- 172.21.15.62:0/1305696528 >> v1:172.21.15.62:6789/0 conn(0x7fb14c0b6a90 legacy=0x7fb14c0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.771+0000 7fb14b400640 1 -- 172.21.15.62:0/1305696528 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c0d36d0 con 0x7fb14c0b27a0 2024-04-25T09:08:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.771+0000 7fb14a000640 1 -- 172.21.15.62:0/1305696528 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837204003 0 0) 0x7fb14c0d3110 con 0x7fb14c0b6a90 2024-04-25T09:08:53.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.772+0000 7fb14b400640 1 --2- 172.21.15.62:0/1305696528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b27a0 0x7fb14c0d26f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb144009d90 tx=0x7fb14402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e636345848dfc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.772+0000 7fb14a000640 1 -- 172.21.15.62:0/1305696528 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144007740 con 0x7fb14c0b27a0 2024-04-25T09:08:53.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.772+0000 7fb14a000640 1 -- 172.21.15.62:0/1305696528 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1440078e0 con 0x7fb14c0b27a0 2024-04-25T09:08:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.772+0000 7fb14a000640 1 -- 172.21.15.62:0/1305696528 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144007cb0 con 0x7fb14c0b27a0 2024-04-25T09:08:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.773+0000 7fb14be00640 1 -- 172.21.15.62:0/1305696528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b27a0 msgr2=0x7fb14c0d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.773+0000 7fb14be00640 1 --2- 172.21.15.62:0/1305696528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b27a0 0x7fb14c0d26f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb144009d90 tx=0x7fb14402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.773+0000 7fb14be00640 1 -- 172.21.15.62:0/1305696528 shutdown_connections 2024-04-25T09:08:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.773+0000 7fb14be00640 1 --2- 172.21.15.62:0/1305696528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b27a0 0x7fb14c0d26f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.773+0000 7fb14be00640 1 -- 172.21.15.62:0/1305696528 >> 172.21.15.62:0/1305696528 conn(0x7fb14c066d10 msgr2=0x7fb14c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.773+0000 7fb14be00640 1 -- 172.21.15.62:0/1305696528 shutdown_connections 2024-04-25T09:08:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.773+0000 7fb14be00640 1 -- 172.21.15.62:0/1305696528 wait complete. 2024-04-25T09:08:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.775+0000 7fb14be00640 1 Processor -- start 2024-04-25T09:08:53.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.775+0000 7fb14be00640 1 -- start start 2024-04-25T09:08:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb14be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb14c0b27a0 0x7fb14c156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb14be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b56f0 0x7fb14c15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb14be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb14c0b6a90 0x7fb14c15a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb14be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0b2ee0 con 0x7fb14c0b56f0 2024-04-25T09:08:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb14be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0b2d60 con 0x7fb14c0b27a0 2024-04-25T09:08:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb14be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0d2c30 con 0x7fb14c0b6a90 2024-04-25T09:08:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb142000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b56f0 0x7fb14c15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb142000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b56f0 0x7fb14c15b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:46466/0 (socket says 172.21.15.62:46466) 2024-04-25T09:08:53.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.776+0000 7fb142000640 1 -- 172.21.15.62:0/991193117 learned_addr learned my addr 172.21.15.62:0/991193117 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb142000640 1 -- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb14c0b6a90 msgr2=0x7fb14c15a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb142000640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb14c0b6a90 0x7fb14c15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb142000640 1 -- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb14c0b27a0 msgr2=0x7fb14c156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb142000640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb14c0b27a0 0x7fb14c156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb142000640 1 -- 172.21.15.62:0/991193117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c15aac0 con 0x7fb14c0b56f0 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb142000640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b56f0 0x7fb14c15b0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c3c0 tx=0x7fb13400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb143400640 1 -- 172.21.15.62:0/991193117 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1340074e0 con 0x7fb14c0b56f0 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb143400640 1 -- 172.21.15.62:0/991193117 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1340047e0 con 0x7fb14c0b56f0 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb143400640 1 -- 172.21.15.62:0/991193117 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134004bd0 con 0x7fb14c0b56f0 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.777+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c157be0 con 0x7fb14c0b56f0 2024-04-25T09:08:53.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.778+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c1581a0 con 0x7fb14c0b56f0 2024-04-25T09:08:53.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.779+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14c04ef20 con 0x7fb14c0b56f0 2024-04-25T09:08:53.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.780+0000 7fb143400640 1 -- 172.21.15.62:0/991193117 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb134011070 con 0x7fb14c0b56f0 2024-04-25T09:08:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.781+0000 7fb143400640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb12806b550 0x7fb12806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.781+0000 7fb143400640 1 -- 172.21.15.62:0/991193117 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb134089d60 con 0x7fb14c0b56f0 2024-04-25T09:08:53.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.781+0000 7fb14aa00640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb12806b550 0x7fb12806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:53.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.783+0000 7fb14aa00640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb12806b550 0x7fb12806da10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb13c000f60 tx=0x7fb13c0079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:53.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:53.785+0000 7fb143400640 1 -- 172.21.15.62:0/991193117 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb13400e280 con 0x7fb14c0b56f0 2024-04-25T09:08:54.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.035+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb14c0ce580 con 0x7fb14c0b56f0 2024-04-25T09:08:54.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.035+0000 7fb143400640 1 -- 172.21.15.62:0/991193117 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fb134058c60 con 0x7fb14c0b56f0 2024-04-25T09:08:54.023 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:54.023 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":44,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1396005512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1396005512},{"type":"v1","addr":"172.21.15.121:6835","nonce":1396005512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3589939339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3589939339},{"type":"v1","addr":"172.21.15.121:6837","nonce":3589939339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2551168226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2551168226},{"type":"v1","addr":"172.21.15.62:6835","nonce":2551168226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:52.648458+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":42,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.62:6833/836333339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":836333339},{"type":"v1","addr":"172.21.15.62:6833","nonce":836333339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:08:54.023 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 44 2024-04-25T09:08:54.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb12806b550 msgr2=0x7fb12806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb12806b550 0x7fb12806da10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb13c000f60 tx=0x7fb13c0079b0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b56f0 msgr2=0x7fb14c15b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:54.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b56f0 0x7fb14c15b0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c3c0 tx=0x7fb13400c880 comp rx=0 tx=0).stop 2024-04-25T09:08:54.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 shutdown_connections 2024-04-25T09:08:54.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb12806b550 0x7fb12806da10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb14c0b56f0 0x7fb14c15b0c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb14c0b6a90 0x7fb14c15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 --2- 172.21.15.62:0/991193117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb14c0b27a0 0x7fb14c156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:54.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 >> 172.21.15.62:0/991193117 conn(0x7fb14c066d10 msgr2=0x7fb14c0d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:54.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.038+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 shutdown_connections 2024-04-25T09:08:54.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:54.039+0000 7fb14be00640 1 -- 172.21.15.62:0/991193117 wait complete. 2024-04-25T09:08:54.447 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:08:36.958423+0000', 'modified': '2024-04-25T09:08:52.648458+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6633}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6633': {'gid': 6633, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.62:6833/836333339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 836333339}, {'type': 'v1', 'addr': '172.21.15.62:6833', 'nonce': 836333339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:08:54.447 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6633:up:rejoin 2024-04-25T09:08:55.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:08:55.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.683+0000 7fdcfc000640 1 Processor -- start 2024-04-25T09:08:55.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.683+0000 7fdcfc000640 1 -- start start 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcfc000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdcf415bd10 0x7fdcf40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcfc000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdcf40aeed0 0x7fdcf40cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcfc000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf406ca70 con 0x7fdcf40cccf0 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcfc000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf406d660 con 0x7fdcf40aeed0 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcfc000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf415c0f0 con 0x7fdcf415bd10 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcfa000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdcf40aeed0 0x7fdcf40cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcf8c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdcf40cccf0 0x7fdcf40ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58452/0 (socket says 172.21.15.62:58452) 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcfa000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdcf40aeed0 0x7fdcf40cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52186/0 (socket says 172.21.15.62:52186) 2024-04-25T09:08:55.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.685+0000 7fdcf8c00640 1 -- 172.21.15.62:0/2968727353 learned_addr learned my addr 172.21.15.62:0/2968727353 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:55.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.686+0000 7fdcf9600640 1 --2- 172.21.15.62:0/2968727353 >> v2:172.21.15.121:3300/0 conn(0x7fdcf415bd10 0x7fdcf40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.686+0000 7fdcfa000640 1 -- 172.21.15.62:0/2968727353 >> v2:172.21.15.121:3300/0 conn(0x7fdcf415bd10 msgr2=0x7fdcf40cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:55.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.686+0000 7fdcfa000640 1 --2- 172.21.15.62:0/2968727353 >> v2:172.21.15.121:3300/0 conn(0x7fdcf415bd10 0x7fdcf40cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.686+0000 7fdcfa000640 1 -- 172.21.15.62:0/2968727353 >> v1:172.21.15.62:6789/0 conn(0x7fdcf40cccf0 legacy=0x7fdcf40ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.686+0000 7fdcfa000640 1 -- 172.21.15.62:0/2968727353 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf406cd30 con 0x7fdcf40aeed0 2024-04-25T09:08:55.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.687+0000 7fdcfa000640 1 --2- 172.21.15.62:0/2968727353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40aeed0 0x7fdcf40cd6c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4072720 tx=0x7fdcec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c591129c5878903c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.687+0000 7fdcf3e00640 1 -- 172.21.15.62:0/2968727353 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec007740 con 0x7fdcf40aeed0 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.687+0000 7fdcf3e00640 1 -- 172.21.15.62:0/2968727353 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec0078e0 con 0x7fdcf40aeed0 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.687+0000 7fdcf3e00640 1 -- 172.21.15.62:0/2968727353 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec007cb0 con 0x7fdcf40aeed0 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.688+0000 7fdcfc000640 1 -- 172.21.15.62:0/2968727353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40aeed0 msgr2=0x7fdcf40cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.688+0000 7fdcfc000640 1 --2- 172.21.15.62:0/2968727353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40aeed0 0x7fdcf40cd6c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4072720 tx=0x7fdcec02a680 comp rx=0 tx=0).stop 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.688+0000 7fdcfc000640 1 -- 172.21.15.62:0/2968727353 shutdown_connections 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.688+0000 7fdcfc000640 1 --2- 172.21.15.62:0/2968727353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40aeed0 0x7fdcf40cd6c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.688+0000 7fdcfc000640 1 --2- 172.21.15.62:0/2968727353 >> v2:172.21.15.121:3300/0 conn(0x7fdcf415bd10 0x7fdcf40cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.688+0000 7fdcfc000640 1 -- 172.21.15.62:0/2968727353 >> 172.21.15.62:0/2968727353 conn(0x7fdcf40668f0 msgr2=0x7fdcf4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.688+0000 7fdcfc000640 1 -- 172.21.15.62:0/2968727353 shutdown_connections 2024-04-25T09:08:55.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.688+0000 7fdcfc000640 1 -- 172.21.15.62:0/2968727353 wait complete. 2024-04-25T09:08:55.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.690+0000 7fdcfc000640 1 Processor -- start 2024-04-25T09:08:55.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.690+0000 7fdcfc000640 1 -- start start 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.691+0000 7fdcfc000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf40aeed0 0x7fdcf40adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.691+0000 7fdcfc000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40cccf0 0x7fdcf414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.691+0000 7fdcfc000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf415bd10 0x7fdcf414d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.691+0000 7fdcfc000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40d01e0 con 0x7fdcf40cccf0 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.691+0000 7fdcfc000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf406d2f0 con 0x7fdcf40aeed0 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.691+0000 7fdcfc000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf406daf0 con 0x7fdcf415bd10 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.691+0000 7fdcf8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40cccf0 0x7fdcf414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40cccf0 0x7fdcf414d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52200/0 (socket says 172.21.15.62:52200) 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf8c00640 1 -- 172.21.15.62:0/3830114165 learned_addr learned my addr 172.21.15.62:0/3830114165 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf9600640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf40aeed0 0x7fdcf40adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcfa000640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf415bd10 0x7fdcf414d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf8c00640 1 -- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf415bd10 msgr2=0x7fdcf414d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf8c00640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf415bd10 0x7fdcf414d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf8c00640 1 -- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf40aeed0 msgr2=0x7fdcf40adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf8c00640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf40aeed0 0x7fdcf40adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf8c00640 1 -- 172.21.15.62:0/3830114165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf415b000 con 0x7fdcf40cccf0 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf8c00640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40cccf0 0x7fdcf414d150 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdce000c400 tx=0x7fdce000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf1600640 1 -- 172.21.15.62:0/3830114165 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0017070 con 0x7fdcf40cccf0 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.692+0000 7fdcf1600640 1 -- 172.21.15.62:0/3830114165 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce0005ce0 con 0x7fdcf40cccf0 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.693+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf40703e0 con 0x7fdcf40cccf0 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.693+0000 7fdcf1600640 1 -- 172.21.15.62:0/3830114165 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce00072f0 con 0x7fdcf40cccf0 2024-04-25T09:08:55.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.693+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf40709a0 con 0x7fdcf40cccf0 2024-04-25T09:08:55.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.694+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf4072c70 con 0x7fdcf40cccf0 2024-04-25T09:08:55.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.695+0000 7fdcf1600640 1 -- 172.21.15.62:0/3830114165 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdce001a070 con 0x7fdcf40cccf0 2024-04-25T09:08:55.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.696+0000 7fdcf1600640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdcd406b4d0 0x7fdcd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:55.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.696+0000 7fdcf1600640 1 -- 172.21.15.62:0/3830114165 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdce00896b0 con 0x7fdcf40cccf0 2024-04-25T09:08:55.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.696+0000 7fdcf9600640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdcd406b4d0 0x7fdcd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:55.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.698+0000 7fdcf9600640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdcd406b4d0 0x7fdcd406d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdce8006fd0 tx=0x7fdce8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.700+0000 7fdcf1600640 1 -- 172.21.15.62:0/3830114165 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdce00584c0 con 0x7fdcf40cccf0 2024-04-25T09:08:55.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.945+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcf404f000 con 0x7fdcf40cccf0 2024-04-25T09:08:55.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.946+0000 7fdcf1600640 1 -- 172.21.15.62:0/3830114165 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fdce0057dc0 con 0x7fdcf40cccf0 2024-04-25T09:08:55.934 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:08:55.934 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1396005512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1396005512},{"type":"v1","addr":"172.21.15.121:6835","nonce":1396005512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3589939339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3589939339},{"type":"v1","addr":"172.21.15.121:6837","nonce":3589939339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2551168226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2551168226},{"type":"v1","addr":"172.21.15.62:6835","nonce":2551168226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:53.657317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6833/836333339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":836333339},{"type":"v1","addr":"172.21.15.62:6833","nonce":836333339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:08:55.934 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 45 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdcd406b4d0 msgr2=0x7fdcd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdcd406b4d0 0x7fdcd406d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdce8006fd0 tx=0x7fdce8008040 comp rx=0 tx=0).stop 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40cccf0 msgr2=0x7fdcf414d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40cccf0 0x7fdcf414d150 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdce000c400 tx=0x7fdce000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 shutdown_connections 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdcd406b4d0 0x7fdcd406d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcf40cccf0 0x7fdcf414d150 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdcf415bd10 0x7fdcf414d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 --2- 172.21.15.62:0/3830114165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcf40aeed0 0x7fdcf40adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 >> 172.21.15.62:0/3830114165 conn(0x7fdcf40668f0 msgr2=0x7fdcf4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:55.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 shutdown_connections 2024-04-25T09:08:55.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:55.949+0000 7fdcfc000640 1 -- 172.21.15.62:0/3830114165 wait complete. 2024-04-25T09:08:56.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:08:36.958423+0000', 'modified': '2024-04-25T09:08:53.657317+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6633}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6633': {'gid': 6633, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.62:6833/836333339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 836333339}, {'type': 'v1', 'addr': '172.21.15.62:6833', 'nonce': 836333339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:08:56.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:08:56.363 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:08:56.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.595+0000 7fe199200640 1 Processor -- start 2024-04-25T09:08:56.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.595+0000 7fe199200640 1 -- start start 2024-04-25T09:08:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.597+0000 7fe199200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe194067000 0x7fe19415f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.597+0000 7fe199200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19406ad80 con 0x7fe194071c90 2024-04-25T09:08:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.597+0000 7fe199200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19406b060 con 0x7fe1940afeb0 2024-04-25T09:08:56.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.597+0000 7fe199200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19406b990 con 0x7fe194067000 2024-04-25T09:08:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.597+0000 7fe193e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe194071c90 0x7fe194072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58454/0 (socket says 172.21.15.62:58454) 2024-04-25T09:08:56.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.597+0000 7fe193e00640 1 -- 172.21.15.62:0/3135548162 learned_addr learned my addr 172.21.15.62:0/3135548162 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe18be00640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe194067000 0x7fe19415f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567040979 0 0) 0x7fe19406ad80 con 0x7fe194071c90 2024-04-25T09:08:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe178003520 con 0x7fe194071c90 2024-04-25T09:08:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148875075 0 0) 0x7fe178003520 con 0x7fe194071c90 2024-04-25T09:08:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe194067000 msgr2=0x7fe19415f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe192a00640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe194067000 0x7fe19415f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 >> v1:172.21.15.121:6789/0 conn(0x7fe1940afeb0 legacy=0x7fe1940668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19406b340 con 0x7fe194071c90 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.598+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fe18c0019f0 con 0x7fe194071c90 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 >> v1:172.21.15.62:6789/0 conn(0x7fe194071c90 legacy=0x7fe194072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe192a00640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1780039e0 0x7fe178003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe192a00640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178004800 0x7fe178004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe192a00640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe194067000 0x7fe1780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19406b990 con 0x7fe1780039e0 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17800ba70 con 0x7fe178004800 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17800bc70 con 0x7fe194067000 2024-04-25T09:08:56.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe193400640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1780039e0 0x7fe178003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe18be00640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178004800 0x7fe178004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.599+0000 7fe193e00640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe194067000 0x7fe1780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.600+0000 7fe193400640 1 -- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe194067000 msgr2=0x7fe1780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.600+0000 7fe193400640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe194067000 0x7fe1780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.600+0000 7fe193400640 1 -- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178004800 msgr2=0x7fe178004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.600+0000 7fe193400640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178004800 0x7fe178004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.600+0000 7fe193400640 1 -- 172.21.15.62:0/3135548162 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe178003520 con 0x7fe1780039e0 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.600+0000 7fe18be00640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe178004800 0x7fe178004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.601+0000 7fe193400640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1780039e0 0x7fe178003e00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe184001720 tx=0x7fe178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb741dab1aff2be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.601+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184001370 con 0x7fe1780039e0 2024-04-25T09:08:56.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1840052d0 con 0x7fe1780039e0 2024-04-25T09:08:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe192a00640 1 -- 172.21.15.62:0/3135548162 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18402ab90 con 0x7fe1780039e0 2024-04-25T09:08:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe199200640 1 -- 172.21.15.62:0/3135548162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1780039e0 msgr2=0x7fe178003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe199200640 1 --2- 172.21.15.62:0/3135548162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1780039e0 0x7fe178003e00 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe184001720 tx=0x7fe178005830 comp rx=0 tx=0).stop 2024-04-25T09:08:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe193e00640 1 -- 172.21.15.62:0/3135548162 reap_dead start 2024-04-25T09:08:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe199200640 1 -- 172.21.15.62:0/3135548162 shutdown_connections 2024-04-25T09:08:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe199200640 1 -- 172.21.15.62:0/3135548162 >> 172.21.15.62:0/3135548162 conn(0x7fe1940b2590 msgr2=0x7fe1940b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe199200640 1 -- 172.21.15.62:0/3135548162 shutdown_connections 2024-04-25T09:08:56.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.602+0000 7fe199200640 1 -- 172.21.15.62:0/3135548162 wait complete. 2024-04-25T09:08:56.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.605+0000 7fe199200640 1 Processor -- start 2024-04-25T09:08:56.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.605+0000 7fe199200640 1 -- start start 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe199200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1780039e0 0x7fe19414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe199200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1940ad8e0 0x7fe1940add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe199200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1941581c0 0x7fe194158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe199200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19415f550 con 0x7fe1941581c0 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe199200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940b0290 con 0x7fe1780039e0 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe199200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19415f6d0 con 0x7fe1940ad8e0 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe193e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1941581c0 0x7fe194158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe193e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1941581c0 0x7fe194158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52208/0 (socket says 172.21.15.62:52208) 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.607+0000 7fe193e00640 1 -- 172.21.15.62:0/670388935 learned_addr learned my addr 172.21.15.62:0/670388935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe193400640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1780039e0 0x7fe19414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe18be00640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1940ad8e0 0x7fe1940add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe193e00640 1 -- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1940ad8e0 msgr2=0x7fe1940add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe193e00640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1940ad8e0 0x7fe1940add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe193e00640 1 -- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1780039e0 msgr2=0x7fe19414a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe193e00640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1780039e0 0x7fe19414a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe193e00640 1 -- 172.21.15.62:0/670388935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940ae360 con 0x7fe1941581c0 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe18be00640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1940ad8e0 0x7fe1940add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.608+0000 7fe193e00640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1941581c0 0x7fe194158640 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe194074c10 tx=0x7fe18c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.609+0000 7fe18b400640 1 -- 172.21.15.62:0/670388935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00ebc0 con 0x7fe1941581c0 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.609+0000 7fe18b400640 1 -- 172.21.15.62:0/670388935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c018070 con 0x7fe1941581c0 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.609+0000 7fe18b400640 1 -- 172.21.15.62:0/670388935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c013420 con 0x7fe1941581c0 2024-04-25T09:08:56.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.609+0000 7fe199200640 1 -- 172.21.15.62:0/670388935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1940ae640 con 0x7fe1941581c0 2024-04-25T09:08:56.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.609+0000 7fe199200640 1 -- 172.21.15.62:0/670388935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19415e320 con 0x7fe1941581c0 2024-04-25T09:08:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.612+0000 7fe18b400640 1 -- 172.21.15.62:0/670388935 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe18c0135c0 con 0x7fe1941581c0 2024-04-25T09:08:56.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.613+0000 7fe19d57b740 1 Processor -- start 2024-04-25T09:08:56.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.613+0000 7fe18b400640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe17006b4d0 0x7fe17006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.613+0000 7fe19d57b740 1 -- start start 2024-04-25T09:08:56.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.613+0000 7fe18b400640 1 -- 172.21.15.62:0/670388935 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe18c08a4a0 con 0x7fe1941581c0 2024-04-25T09:08:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe18be00640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe17006b4d0 0x7fe17006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56adb30 0x562ca56d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe19d57b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca5611620 con 0x562ca56d5eb0 2024-04-25T09:08:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe19d57b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca55baca0 con 0x562ca56adb30 2024-04-25T09:08:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe19d57b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ca560d6c0 con 0x562ca56f67a0 2024-04-25T09:08:56.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe18be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe18be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52218/0 (socket says 172.21.15.62:52218) 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.614+0000 7fe18be00640 1 -- 172.21.15.62:0/3489913852 learned_addr learned my addr 172.21.15.62:0/3489913852 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe193400640 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56adb30 0x562ca56d5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe193e00640 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe18be00640 1 -- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 msgr2=0x562ca56ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe18be00640 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe18be00640 1 -- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56adb30 msgr2=0x562ca56d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe18be00640 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56adb30 0x562ca56d5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe18be00640 1 -- 172.21.15.62:0/3489913852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ca55c1f40 con 0x562ca56d5eb0 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe193e00640 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.615+0000 7fe18be00640 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56adf10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe17c01a9b0 tx=0x7fe17c01ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303a747c5a821642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe183e00640 1 -- 172.21.15.62:0/3489913852 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c028070 con 0x562ca56d5eb0 2024-04-25T09:08:56.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe183e00640 1 -- 172.21.15.62:0/3489913852 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c01a0e0 con 0x562ca56d5eb0 2024-04-25T09:08:56.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe183e00640 1 -- 172.21.15.62:0/3489913852 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c005bb0 con 0x562ca56d5eb0 2024-04-25T09:08:56.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe18be00640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe17006b4d0 0x7fe17006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe17c017750 tx=0x7fe17c013000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe19d57b740 1 -- 172.21.15.62:0/3489913852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 msgr2=0x562ca56adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56adf10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe17c01a9b0 tx=0x7fe17c01ae70 comp rx=0 tx=0).stop 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe19d57b740 1 -- 172.21.15.62:0/3489913852 shutdown_connections 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56adf10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3489913852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56adb30 0x562ca56d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.616+0000 7fe19d57b740 1 -- 172.21.15.62:0/3489913852 >> 172.21.15.62:0/3489913852 conn(0x562ca56aad50 msgr2=0x562ca56f3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.617+0000 7fe19d57b740 1 -- 172.21.15.62:0/3489913852 shutdown_connections 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.617+0000 7fe19d57b740 1 -- 172.21.15.62:0/3489913852 wait complete. 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.617+0000 7fe19d57b740 1 -- start start 2024-04-25T09:08:56.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.618+0000 7fe19d57b740 1 Processor -- start 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56d5eb0 0x562ca56ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56f67a0 0x562ca56f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56cdaa0 0x562ca56f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe19d57b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca56d54e0 con 0x562ca56cdaa0 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe19d57b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca55b01f0 con 0x562ca56f67a0 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe19d57b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ca56d4ed0 con 0x562ca56d5eb0 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe19d57b740 10 client.0 _ll_register_callbacks cb 0x562ca55cdad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe193e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56cdaa0 0x562ca56f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe193e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56cdaa0 0x562ca56f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52230/0 (socket says 172.21.15.62:52230) 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe193e00640 1 -- 172.21.15.62:0/513870671 learned_addr learned my addr 172.21.15.62:0/513870671 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe18be00640 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56f67a0 0x562ca56f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe193e00640 1 -- 172.21.15.62:0/513870671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56d5eb0 msgr2=0x562ca56ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe193e00640 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56d5eb0 0x562ca56ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe193e00640 1 -- 172.21.15.62:0/513870671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56f67a0 msgr2=0x562ca56f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe193e00640 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56f67a0 0x562ca56f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.619+0000 7fe193e00640 1 -- 172.21.15.62:0/513870671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ca55cfc50 con 0x562ca56cdaa0 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe193e00640 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56cdaa0 0x562ca56f5880 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe18c012040 tx=0x7fe18c058600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976dd3c9f921fd43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe180c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe180c00640 1 -- 172.21.15.62:0/513870671 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0612c0 con 0x562ca56cdaa0 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe180c00640 1 -- 172.21.15.62:0/513870671 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe18c060e30 con 0x562ca56cdaa0 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe19d57b740 1 -- 172.21.15.62:0/513870671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe17c01a0e0 con 0x562ca56cdaa0 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe180c00640 1 -- 172.21.15.62:0/513870671 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe18c061030 con 0x562ca56cdaa0 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe19d57b740 10 client.6675 fetch_fsmap learned FSMap version 45 2024-04-25T09:08:56.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe19d57b740 1 -- 172.21.15.62:0/513870671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x562ca53262d0 con 0x562ca56cdaa0 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe180c00640 1 -- 172.21.15.62:0/513870671 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c01a0e0 con 0x562ca56cdaa0 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe180c00640 1 -- 172.21.15.62:0/513870671 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 45) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe18c06cb70 con 0x562ca56cdaa0 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe19d57b740 10 client.6675 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe19d57b740 10 client.6675 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.620+0000 7fe19d57b740 20 client.6675 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.621+0000 7fe19d57b740 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x562ca57071a0 0x562ca57112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.621+0000 7fe19d57b740 4 client.6675 mds_command: new command op to 6633 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.621+0000 7fe19d57b740 1 -- 172.21.15.62:0/513870671 --> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ca561eb60 con 0x562ca57071a0 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.621+0000 7fe193400640 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x562ca57071a0 0x562ca57112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.622+0000 7fe193400640 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x562ca57071a0 0x562ca57112e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8f7354277c065d server_cookie=3594d832a7a9105b in_seq=0 out_seq=0 2024-04-25T09:08:56.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.622+0000 7fe180c00640 10 client.6675 ms_handle_connect on v2:172.21.15.62:6832/836333339 2024-04-25T09:08:56.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.622+0000 7fe180c00640 1 -- 172.21.15.62:0/513870671 <== mds.0 v2:172.21.15.62:6832/836333339 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562ca561eb60 con 0x562ca57071a0 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.622+0000 7fe180c00640 10 client.6675 handle_command_reply: tid=0 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.622+0000 7fe19d57b740 1 client.6675 shutdown 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 -- 172.21.15.62:0/513870671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56cdaa0 msgr2=0x562ca56f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56cdaa0 0x562ca56f5880 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe18c012040 tx=0x7fe18c058600 comp rx=0 tx=0).stop 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 -- 172.21.15.62:0/513870671 shutdown_connections 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x562ca57071a0 0x562ca57112e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56cdaa0 0x562ca56f5880 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56f67a0 0x562ca56f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 --2- 172.21.15.62:0/513870671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56d5eb0 0x562ca56ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe180c00640 0 client.6675 ms_handle_reset on v2:172.21.15.62:6832/836333339 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 -- 172.21.15.62:0/513870671 >> 172.21.15.62:0/513870671 conn(0x562ca56adb30 msgr2=0x562ca56f6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 -- 172.21.15.62:0/513870671 shutdown_connections 2024-04-25T09:08:56.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 1 -- 172.21.15.62:0/513870671 wait complete. 2024-04-25T09:08:56.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.623+0000 7fe19d57b740 20 client.6675 trim_cache size 0 max 16384 2024-04-25T09:08:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.641+0000 7fe19d57b740 1 Processor -- start 2024-04-25T09:08:56.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.641+0000 7fe19d57b740 1 -- start start 2024-04-25T09:08:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56cdaa0 0x562ca56d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe19d57b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1941ff860 con 0x562ca56d5eb0 2024-04-25T09:08:56.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe19d57b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940b5310 con 0x562ca56cdaa0 2024-04-25T09:08:56.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe19d57b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1941ff260 con 0x562ca56f67a0 2024-04-25T09:08:56.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe18be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe18be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52246/0 (socket says 172.21.15.62:52246) 2024-04-25T09:08:56.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.642+0000 7fe18be00640 1 -- 172.21.15.62:0/1825003797 learned_addr learned my addr 172.21.15.62:0/1825003797 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.643+0000 7fe193e00640 1 --2- 172.21.15.62:0/1825003797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.643+0000 7fe18be00640 1 -- 172.21.15.62:0/1825003797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 msgr2=0x562ca56ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.643+0000 7fe18be00640 1 --2- 172.21.15.62:0/1825003797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.643+0000 7fe18be00640 1 -- 172.21.15.62:0/1825003797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56cdaa0 msgr2=0x562ca56d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.643+0000 7fe18be00640 1 --2- 172.21.15.62:0/1825003797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56cdaa0 0x562ca56d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.643+0000 7fe18be00640 1 -- 172.21.15.62:0/1825003797 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ca53f06b0 con 0x562ca56d5eb0 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.643+0000 7fe18be00640 1 --2- 172.21.15.62:0/1825003797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56caf20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe17c006000 tx=0x7fe17c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345e3fb25dd77a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.643+0000 7fe183400640 1 -- 172.21.15.62:0/1825003797 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c028040 con 0x562ca56d5eb0 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe183400640 1 -- 172.21.15.62:0/1825003797 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c005e10 con 0x562ca56d5eb0 2024-04-25T09:08:56.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe183400640 1 -- 172.21.15.62:0/1825003797 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c009c60 con 0x562ca56d5eb0 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 -- 172.21.15.62:0/1825003797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 msgr2=0x562ca56caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 --2- 172.21.15.62:0/1825003797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56caf20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe17c006000 tx=0x7fe17c01f000 comp rx=0 tx=0).stop 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 -- 172.21.15.62:0/1825003797 shutdown_connections 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 --2- 172.21.15.62:0/1825003797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56d5eb0 0x562ca56caf20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 --2- 172.21.15.62:0/1825003797 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56f67a0 0x562ca56ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 --2- 172.21.15.62:0/1825003797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56cdaa0 0x562ca56d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 -- 172.21.15.62:0/1825003797 >> 172.21.15.62:0/1825003797 conn(0x562ca56adb30 msgr2=0x562ca56a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 -- 172.21.15.62:0/1825003797 shutdown_connections 2024-04-25T09:08:56.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.644+0000 7fe19d57b740 1 -- 172.21.15.62:0/1825003797 wait complete. 2024-04-25T09:08:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.645+0000 7fe19d57b740 1 -- start start 2024-04-25T09:08:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.645+0000 7fe19d57b740 1 Processor -- start 2024-04-25T09:08:56.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.646+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56cdaa0 0x562ca5709ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.646+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56d5eb0 0x562ca57582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.646+0000 7fe19d57b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56f67a0 0x562ca56ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.646+0000 7fe19d57b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca5707020 con 0x562ca56f67a0 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.646+0000 7fe19d57b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ca56d1b60 con 0x562ca56d5eb0 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.646+0000 7fe19d57b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ca5707c20 con 0x562ca56cdaa0 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.646+0000 7fe19d57b740 10 client.0 _ll_register_callbacks cb 0x7fe18c058760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56f67a0 0x562ca56ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56f67a0 0x562ca56ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52258/0 (socket says 172.21.15.62:52258) 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 -- 172.21.15.62:0/3758325653 learned_addr learned my addr 172.21.15.62:0/3758325653 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193400640 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56cdaa0 0x562ca5709ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe18be00640 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56d5eb0 0x562ca57582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 -- 172.21.15.62:0/3758325653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56cdaa0 msgr2=0x562ca5709ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56cdaa0 0x562ca5709ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 -- 172.21.15.62:0/3758325653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56d5eb0 msgr2=0x562ca57582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56d5eb0 0x562ca57582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 -- 172.21.15.62:0/3758325653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ca570a410 con 0x562ca56f67a0 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe193e00640 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56f67a0 0x562ca56ae130 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe18c001820 tx=0x7fe18c088830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662399d7be056fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.647+0000 7fe180c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe180c00640 1 -- 172.21.15.62:0/3758325653 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c07b070 con 0x562ca56f67a0 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe180c00640 1 -- 172.21.15.62:0/3758325653 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe18c058210 con 0x562ca56f67a0 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 1 -- 172.21.15.62:0/3758325653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe17c005e10 con 0x562ca56f67a0 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe180c00640 1 -- 172.21.15.62:0/3758325653 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe18c0612c0 con 0x562ca56f67a0 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 10 client.6681 fetch_fsmap learned FSMap version 45 2024-04-25T09:08:56.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 1 -- 172.21.15.62:0/3758325653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x562ca570a410 con 0x562ca56f67a0 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe180c00640 1 -- 172.21.15.62:0/3758325653 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c005e10 con 0x562ca56f67a0 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe180c00640 1 -- 172.21.15.62:0/3758325653 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 45) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe18c054900 con 0x562ca56f67a0 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 10 client.6681 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 10 client.6681 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 20 client.6681 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x562ca575ece0 0x562ca575f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 4 client.6681 mds_command: new command op to 6633 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.648+0000 7fe19d57b740 1 -- 172.21.15.62:0/3758325653 --> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ca5618eb0 con 0x562ca575ece0 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.649+0000 7fe193400640 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x562ca575ece0 0x562ca575f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.649+0000 7fe193400640 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x562ca575ece0 0x562ca575f0c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c8b4dc349187b3 server_cookie=b1bae2f10229b029 in_seq=0 out_seq=0 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.649+0000 7fe180c00640 10 client.6681 ms_handle_connect on v2:172.21.15.62:6832/836333339 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.649+0000 7fe180c00640 1 -- 172.21.15.62:0/3758325653 <== mds.0 v2:172.21.15.62:6832/836333339 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562ca5618eb0 con 0x562ca575ece0 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.649+0000 7fe180c00640 10 client.6681 handle_command_reply: tid=0 2024-04-25T09:08:56.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.649+0000 7fe19d57b740 1 client.6681 shutdown 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 -- 172.21.15.62:0/3758325653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56f67a0 msgr2=0x562ca56ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56f67a0 0x562ca56ae130 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe18c001820 tx=0x7fe18c088830 comp rx=0 tx=0).stop 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 -- 172.21.15.62:0/3758325653 shutdown_connections 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x562ca575ece0 0x562ca575f0c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562ca56f67a0 0x562ca56ae130 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ca56d5eb0 0x562ca57582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 --2- 172.21.15.62:0/3758325653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562ca56cdaa0 0x562ca5709ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 -- 172.21.15.62:0/3758325653 >> 172.21.15.62:0/3758325653 conn(0x562ca56adb30 msgr2=0x562ca5705fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe180c00640 0 client.6681 ms_handle_reset on v2:172.21.15.62:6832/836333339 2024-04-25T09:08:56.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 -- 172.21.15.62:0/3758325653 shutdown_connections 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 1 -- 172.21.15.62:0/3758325653 wait complete. 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.650+0000 7fe19d57b740 20 client.6681 trim_cache size 0 max 16384 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 6633, 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:08:56.642 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:08:56.643 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 5.0174146259999999, 2024-04-25T09:08:56.643 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 45, 2024-04-25T09:08:56.643 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 66, 2024-04-25T09:08:56.643 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 66, 2024-04-25T09:08:56.643 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 5.788733229 2024-04-25T09:08:56.643 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 -- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe17006b4d0 msgr2=0x7fe17006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe17006b4d0 0x7fe17006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe17c017750 tx=0x7fe17c013000 comp rx=0 tx=0).stop 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 -- 172.21.15.62:0/670388935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1941581c0 msgr2=0x7fe194158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1941581c0 0x7fe194158640 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe194074c10 tx=0x7fe18c007bc0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 -- 172.21.15.62:0/670388935 shutdown_connections 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe17006b4d0 0x7fe17006d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1941581c0 0x7fe194158640 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1940ad8e0 0x7fe1940add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 --2- 172.21.15.62:0/670388935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1780039e0 0x7fe19414a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 -- 172.21.15.62:0/670388935 >> 172.21.15.62:0/670388935 conn(0x7fe1940b2590 msgr2=0x7fe1940cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 -- 172.21.15.62:0/670388935 shutdown_connections 2024-04-25T09:08:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.651+0000 7fe199200640 1 -- 172.21.15.62:0/670388935 wait complete. 2024-04-25T09:08:56.658 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:08:56.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.881+0000 7f9929400640 1 Processor -- start 2024-04-25T09:08:56.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.881+0000 7f9929400640 1 -- start start 2024-04-25T09:08:56.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.883+0000 7f9929400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 0x7f992406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.883+0000 7f9929400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99240ca4e0 0x7f992406d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.883+0000 7f9929400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99240cf030 con 0x7f992415d810 2024-04-25T09:08:56.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.883+0000 7f9929400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99240cf310 con 0x7f99240ca4e0 2024-04-25T09:08:56.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.883+0000 7f9929400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99240c8cd0 con 0x7f992415dbf0 2024-04-25T09:08:56.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.883+0000 7f9923e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f992415d810 0x7f99240ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58462/0 (socket says 172.21.15.62:58462) 2024-04-25T09:08:56.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.883+0000 7f9923e00640 1 -- 172.21.15.62:0/2888785539 learned_addr learned my addr 172.21.15.62:0/2888785539 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9923400640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 0x7f992406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722390318 0 0) 0x7f99240cf030 con 0x7f992415d810 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9928a00640 1 --2- 172.21.15.62:0/2888785539 >> v2:172.21.15.121:3300/0 conn(0x7f99240ca4e0 0x7f992406d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9908003520 con 0x7f992415d810 2024-04-25T09:08:56.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1491137395 0 0) 0x7f9908003520 con 0x7f992415d810 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 msgr2=0x7f992406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 0x7f992406d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 >> v2:172.21.15.121:3300/0 conn(0x7f99240ca4e0 msgr2=0x7f992406d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 --2- 172.21.15.62:0/2888785539 >> v2:172.21.15.121:3300/0 conn(0x7f99240ca4e0 0x7f992406d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240c8fb0 con 0x7f992415d810 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f9914001670 con 0x7f992415d810 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 >> v1:172.21.15.62:6789/0 conn(0x7f992415d810 legacy=0x7f99240ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99080039b0 0x7f9908003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9908004800 0x7f9908004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 0x7f99080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240cf310 con 0x7f99080039b0 2024-04-25T09:08:56.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240c8cd0 con 0x7f9908004800 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.884+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990800bac0 con 0x7f992415dbf0 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9923e00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99080039b0 0x7f9908003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9923e00640 1 -- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 msgr2=0x7f99080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9928a00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 0x7f99080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9923400640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9908004800 0x7f9908004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9923e00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 0x7f99080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9923e00640 1 -- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9908004800 msgr2=0x7f9908004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9923e00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9908004800 0x7f9908004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9923e00640 1 -- 172.21.15.62:0/2888785539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908003520 con 0x7f99080039b0 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9928a00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f992415dbf0 0x7f99080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.885+0000 7f9923400640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9908004800 0x7f9908004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.886+0000 7f9923e00640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99080039b0 0x7f9908003dd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9914001eb0 tx=0x7f9908005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7081d92a77ab416f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914007c80 con 0x7f99080039b0 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914005ad0 con 0x7f99080039b0 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9922a00640 1 -- 172.21.15.62:0/2888785539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914007070 con 0x7f99080039b0 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9929400640 1 -- 172.21.15.62:0/2888785539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99080039b0 msgr2=0x7f9908003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9929400640 1 --2- 172.21.15.62:0/2888785539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99080039b0 0x7f9908003dd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9914001eb0 tx=0x7f9908005830 comp rx=0 tx=0).stop 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9928a00640 1 -- 172.21.15.62:0/2888785539 reap_dead start 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9929400640 1 -- 172.21.15.62:0/2888785539 shutdown_connections 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9929400640 1 -- 172.21.15.62:0/2888785539 >> 172.21.15.62:0/2888785539 conn(0x7f99240b14d0 msgr2=0x7f99240b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:56.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9929400640 1 -- 172.21.15.62:0/2888785539 shutdown_connections 2024-04-25T09:08:56.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.887+0000 7f9929400640 1 -- 172.21.15.62:0/2888785539 wait complete. 2024-04-25T09:08:56.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.889+0000 7f9929400640 1 Processor -- start 2024-04-25T09:08:56.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.889+0000 7f9929400640 1 -- start start 2024-04-25T09:08:56.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.890+0000 7f9929400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99080039b0 0x7f99240b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.890+0000 7f9929400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9924147cd0 0x7f9924157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.890+0000 7f9929400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99241545e0 0x7f9924154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.890+0000 7f9929400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992406d1f0 con 0x7f99241545e0 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.890+0000 7f9929400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240b10d0 con 0x7f99080039b0 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.890+0000 7f9929400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992406d370 con 0x7f9924147cd0 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9928a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99241545e0 0x7f9924154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9928a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99241545e0 0x7f9924154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52278/0 (socket says 172.21.15.62:52278) 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9928a00640 1 -- 172.21.15.62:0/3038293786 learned_addr learned my addr 172.21.15.62:0/3038293786 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9923400640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9924147cd0 0x7f9924157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9923e00640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99080039b0 0x7f99240b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9928a00640 1 -- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9924147cd0 msgr2=0x7f9924157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9928a00640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9924147cd0 0x7f9924157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9928a00640 1 -- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99080039b0 msgr2=0x7f99240b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9928a00640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99080039b0 0x7f99240b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9928a00640 1 -- 172.21.15.62:0/3038293786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240b2b80 con 0x7f99241545e0 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9923400640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9924147cd0 0x7f9924157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.891+0000 7f9923e00640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99080039b0 0x7f99240b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.892+0000 7f9928a00640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99241545e0 0x7f9924154a60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f99240cc6d0 tx=0x7f991c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.892+0000 7f991be00640 1 -- 172.21.15.62:0/3038293786 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c019070 con 0x7f99241545e0 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.892+0000 7f991be00640 1 -- 172.21.15.62:0/3038293786 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c007d10 con 0x7f99241545e0 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.892+0000 7f991be00640 1 -- 172.21.15.62:0/3038293786 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c014640 con 0x7f99241545e0 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.892+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99240b2e60 con 0x7f99241545e0 2024-04-25T09:08:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.892+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99240b3420 con 0x7f99241545e0 2024-04-25T09:08:56.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.893+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992404ef20 con 0x7f99241545e0 2024-04-25T09:08:56.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.894+0000 7f991be00640 1 -- 172.21.15.62:0/3038293786 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f991c01c070 con 0x7f99241545e0 2024-04-25T09:08:56.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.896+0000 7f991be00640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f990406b4d0 0x7f990406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:56.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.896+0000 7f991be00640 1 -- 172.21.15.62:0/3038293786 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f991c089cc0 con 0x7f99241545e0 2024-04-25T09:08:56.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.896+0000 7f9923e00640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f990406b4d0 0x7f990406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:56.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.897+0000 7f9923e00640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f990406b4d0 0x7f990406d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9914032000 tx=0x7f9914009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:56.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:56.900+0000 7f991be00640 1 -- 172.21.15.62:0/3038293786 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f991c090080 con 0x7f99241545e0 2024-04-25T09:08:57.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.144+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9924070280 con 0x7f99241545e0 2024-04-25T09:08:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.145+0000 7f991be00640 1 -- 172.21.15.62:0/3038293786 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7f991c058a40 con 0x7f99241545e0 2024-04-25T09:08:57.132 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:08:57.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f990406b4d0 msgr2=0x7f990406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f990406b4d0 0x7f990406d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9914032000 tx=0x7f9914009030 comp rx=0 tx=0).stop 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99241545e0 msgr2=0x7f9924154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99241545e0 0x7f9924154a60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f99240cc6d0 tx=0x7f991c009cb0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 shutdown_connections 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f990406b4d0 0x7f990406d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99241545e0 0x7f9924154a60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9924147cd0 0x7f9924157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 --2- 172.21.15.62:0/3038293786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99080039b0 0x7f99240b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 >> 172.21.15.62:0/3038293786 conn(0x7f99240b14d0 msgr2=0x7f99240cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 shutdown_connections 2024-04-25T09:08:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:57.148+0000 7f9929400640 1 -- 172.21.15.62:0/3038293786 wait complete. 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:08:57.155 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:57.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:57.156 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:57.210 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42322sec preferred_lft 42322sec 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:57.211 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:08:57.212 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:08:57.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:57.213 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:08:57.298 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:08:57.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:57.303 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:08:57.358 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:08:57.359 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:08:57.359 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:08:57.359 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:08:57.359 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:08:57.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:57.360 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:08:57.360 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:08:57.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:08:57.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:08:57.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:08:57.360 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:08:57.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:57.515 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:08:57.567 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:08:57.568 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:08:57.568 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:57.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:57.569 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:08:57.624 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-04-25T09:08:57.637+0000 7f2548c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:08:57.657 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:08:57.658 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:57.716 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:08:57.717 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:08:57.717 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:57.769 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:08:57.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:57.769 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:08:57.901 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:08:57.901 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:08:57.932 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:08:57.932 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:08:57.933 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:08:57.933 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:08:57.933 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:08:57.935 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:08:57.935 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:08:57.935 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:08:57.935 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:08:57.935 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:08:57.940 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:08:57.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:57.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:08:58.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:58.014 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:08:58.069 INFO:teuthology.orchestra.run.smithi062.stdout:client6690 2024-04-25T09:08:58.069 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:08:58.069 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:08:58.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:08:58.119 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T09:08:58.218 INFO:teuthology.orchestra.run.smithi062.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T09:08:58.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:08:58.220 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T09:08:58.220 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:08:58.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T09:08:58.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.453+0000 7f4aa3e00640 1 Processor -- start 2024-04-25T09:08:58.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.453+0000 7f4aa3e00640 1 -- start start 2024-04-25T09:08:58.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.454+0000 7f4aa3e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4aa415b520 0x7f4aa40ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa406c440 0x7f4aa40adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4aa415eb90 0x7f4aa415f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa415f560 con 0x7f4aa415eb90 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa406e270 con 0x7f4aa415b520 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40b3fb0 con 0x7f4aa406c440 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4aa415eb90 0x7f4aa415f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4aa415eb90 0x7f4aa415f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52288/0 (socket says 172.21.15.62:52288) 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3400640 1 -- 172.21.15.62:0/1276102726 learned_addr learned my addr 172.21.15.62:0/1276102726 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3400640 1 -- 172.21.15.62:0/1276102726 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa406c440 msgr2=0x7f4aa40adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3400640 1 --2- 172.21.15.62:0/1276102726 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa406c440 0x7f4aa40adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3400640 1 -- 172.21.15.62:0/1276102726 >> v2:172.21.15.121:3300/0 conn(0x7f4aa415b520 msgr2=0x7f4aa40ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3400640 1 --2- 172.21.15.62:0/1276102726 >> v2:172.21.15.121:3300/0 conn(0x7f4aa415b520 0x7f4aa40ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.455+0000 7f4aa3400640 1 -- 172.21.15.62:0/1276102726 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa406c920 con 0x7f4aa415eb90 2024-04-25T09:08:58.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.457+0000 7f4aa3400640 1 --2- 172.21.15.62:0/1276102726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa415eb90 0x7f4aa415f020 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40af410 tx=0x7f4a9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dac8879ddba1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.457+0000 7f4aa2000640 1 -- 172.21.15.62:0/1276102726 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c036070 con 0x7f4aa415eb90 2024-04-25T09:08:58.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.457+0000 7f4aa2000640 1 -- 172.21.15.62:0/1276102726 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c031600 con 0x7f4aa415eb90 2024-04-25T09:08:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.457+0000 7f4aa2000640 1 -- 172.21.15.62:0/1276102726 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c0319d0 con 0x7f4aa415eb90 2024-04-25T09:08:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 -- 172.21.15.62:0/1276102726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa415eb90 msgr2=0x7f4aa415f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/1276102726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa415eb90 0x7f4aa415f020 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40af410 tx=0x7f4a9c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 -- 172.21.15.62:0/1276102726 shutdown_connections 2024-04-25T09:08:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/1276102726 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa415eb90 0x7f4aa415f020 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/1276102726 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa406c440 0x7f4aa40adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/1276102726 >> v2:172.21.15.121:3300/0 conn(0x7f4aa415b520 0x7f4aa40ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 -- 172.21.15.62:0/1276102726 >> 172.21.15.62:0/1276102726 conn(0x7f4aa4066ad0 msgr2=0x7f4aa4066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:58.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 -- 172.21.15.62:0/1276102726 shutdown_connections 2024-04-25T09:08:58.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.458+0000 7f4aa3e00640 1 -- 172.21.15.62:0/1276102726 wait complete. 2024-04-25T09:08:58.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.460+0000 7f4aa3e00640 1 Processor -- start 2024-04-25T09:08:58.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.460+0000 7f4aa3e00640 1 -- start start 2024-04-25T09:08:58.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa406c440 0x7f4aa40cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa415b520 0x7f4aa40711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aa415eb90 0x7f4aa4071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa406d1c0 con 0x7f4aa406c440 2024-04-25T09:08:58.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa406cee0 con 0x7f4aa415eb90 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa415b900 con 0x7f4aa415b520 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa406c440 0x7f4aa40cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa406c440 0x7f4aa40cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52304/0 (socket says 172.21.15.62:52304) 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa2a00640 1 -- 172.21.15.62:0/3982971631 learned_addr learned my addr 172.21.15.62:0/3982971631 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4a9be00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa415b520 0x7f4aa40711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa3400640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aa415eb90 0x7f4aa4071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa2a00640 1 -- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa415b520 msgr2=0x7f4aa40711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa415b520 0x7f4aa40711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa2a00640 1 -- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aa415eb90 msgr2=0x7f4aa4071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aa415eb90 0x7f4aa4071700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.461+0000 7f4aa2a00640 1 -- 172.21.15.62:0/3982971631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa41f49f0 con 0x7f4aa406c440 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.462+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa406c440 0x7f4aa40cb190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400b4a0 tx=0x7f4a9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.462+0000 7f4a9aa00640 1 -- 172.21.15.62:0/3982971631 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94014070 con 0x7f4aa406c440 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.462+0000 7f4a9aa00640 1 -- 172.21.15.62:0/3982971631 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a94010610 con 0x7f4aa406c440 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.462+0000 7f4a9aa00640 1 -- 172.21.15.62:0/3982971631 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94010a00 con 0x7f4aa406c440 2024-04-25T09:08:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.462+0000 7f4aa3e00640 1 -- 172.21.15.62:0/3982971631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa41f3c50 con 0x7f4aa406c440 2024-04-25T09:08:58.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.462+0000 7f4aa3e00640 1 -- 172.21.15.62:0/3982971631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa41f41f0 con 0x7f4aa406c440 2024-04-25T09:08:58.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.465+0000 7f4a9aa00640 1 -- 172.21.15.62:0/3982971631 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a94017070 con 0x7f4aa406c440 2024-04-25T09:08:58.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.465+0000 7f4a9aa00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4a8006b4d0 0x7f4a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.466+0000 7f4a9aa00640 1 -- 172.21.15.62:0/3982971631 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4a94089c90 con 0x7f4aa406c440 2024-04-25T09:08:58.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.466+0000 7f4aac64b740 1 Processor -- start 2024-04-25T09:08:58.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.466+0000 7f4aac64b740 1 -- start start 2024-04-25T09:08:58.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.466+0000 7f4a9be00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4a8006b4d0 0x7f4a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.467+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f2f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.467+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2ebca0 0x55569f2efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.467+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f2f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.467+0000 7f4aac64b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f2b8a00 con 0x55569f2f0e10 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.467+0000 7f4aac64b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f254dd0 con 0x55569f2eff50 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.467+0000 7f4aac64b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569f1fda90 con 0x55569f2ebca0 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.467+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f2f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f2f11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52308/0 (socket says 172.21.15.62:52308) 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4aa2a00640 1 -- 172.21.15.62:0/3961896129 learned_addr learned my addr 172.21.15.62:0/3961896129 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4a9be00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4a8006b4d0 0x7f4a8006d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40af310 tx=0x7f4a8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4aa2a00640 1 -- 172.21.15.62:0/3961896129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2ebca0 msgr2=0x55569f2efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/3961896129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2ebca0 0x55569f2efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4aa2a00640 1 -- 172.21.15.62:0/3961896129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 msgr2=0x55569f2f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/3961896129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f2f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4aa2a00640 1 -- 172.21.15.62:0/3961896129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55569f035af0 con 0x55569f2f0e10 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/3961896129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f2f11f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4a94060580 tx=0x7f4a9400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7e4a4652865964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4a93400640 1 -- 172.21.15.62:0/3961896129 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9401e040 con 0x55569f2f0e10 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4a93400640 1 -- 172.21.15.62:0/3961896129 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a94054a10 con 0x55569f2f0e10 2024-04-25T09:08:58.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.468+0000 7f4a93400640 1 -- 172.21.15.62:0/3961896129 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94010ba0 con 0x55569f2f0e10 2024-04-25T09:08:58.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 -- 172.21.15.62:0/3961896129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 msgr2=0x55569f2f11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 --2- 172.21.15.62:0/3961896129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f2f11f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4a94060580 tx=0x7f4a9400bf90 comp rx=0 tx=0).stop 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 -- 172.21.15.62:0/3961896129 shutdown_connections 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 --2- 172.21.15.62:0/3961896129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f2f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 --2- 172.21.15.62:0/3961896129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2ebca0 0x55569f2efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 --2- 172.21.15.62:0/3961896129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f2f11f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 -- 172.21.15.62:0/3961896129 >> 172.21.15.62:0/3961896129 conn(0x55569f2edf10 msgr2=0x55569f3369d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 -- 172.21.15.62:0/3961896129 shutdown_connections 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.469+0000 7f4aac64b740 1 -- 172.21.15.62:0/3961896129 wait complete. 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.470+0000 7f4aac64b740 1 -- start start 2024-04-25T09:08:58.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.470+0000 7f4aac64b740 1 Processor -- start 2024-04-25T09:08:58.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.471+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2eff50 0x55569f30bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.471+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2f0e10 0x55569f31ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.471+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 0x55569f30cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.471+0000 7f4aac64b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f1f3080 con 0x55569f2eff50 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.471+0000 7f4aac64b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f250e70 con 0x55569f2f0e10 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.471+0000 7f4aac64b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569f31c9a0 con 0x55569f31e310 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.471+0000 7f4aac64b740 10 client.0 _ll_register_callbacks cb 0x55569f24b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2eff50 0x55569f30bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2eff50 0x55569f30bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52310/0 (socket says 172.21.15.62:52310) 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 -- 172.21.15.62:0/1505080247 learned_addr learned my addr 172.21.15.62:0/1505080247 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4a9be00640 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2f0e10 0x55569f31ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa3400640 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 0x55569f30cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 -- 172.21.15.62:0/1505080247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 msgr2=0x55569f30cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 0x55569f30cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 -- 172.21.15.62:0/1505080247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2f0e10 msgr2=0x55569f31ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2f0e10 0x55569f31ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 -- 172.21.15.62:0/1505080247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55569ef293e0 con 0x55569f2eff50 2024-04-25T09:08:58.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2eff50 0x55569f30bc40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400b470 tx=0x7f4a94088020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f2b96dd50e572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.472+0000 7f4a87e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4a87e00640 1 -- 172.21.15.62:0/1505080247 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9401e070 con 0x55569f2eff50 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4a87e00640 1 -- 172.21.15.62:0/1505080247 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4a94010da0 con 0x55569f2eff50 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 1 -- 172.21.15.62:0/1505080247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a94054a10 con 0x55569f2eff50 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4a87e00640 1 -- 172.21.15.62:0/1505080247 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9407f050 con 0x55569f2eff50 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 10 client.6702 fetch_fsmap learned FSMap version 45 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 1 -- 172.21.15.62:0/1505080247 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x55569f096280 con 0x55569f2eff50 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4a87e00640 1 -- 172.21.15.62:0/1505080247 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a94054a10 con 0x55569f2eff50 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4a87e00640 1 -- 172.21.15.62:0/1505080247 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 45) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a940577f0 con 0x55569f2eff50 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 10 client.6702 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 10 client.6702 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 20 client.6702 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x55569f30d5b0 0x55569f3385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 4 client.6702 mds_command: new command op to 6633 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.473+0000 7f4aac64b740 1 -- 172.21.15.62:0/1505080247 --> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55569f262310 con 0x55569f30d5b0 2024-04-25T09:08:58.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.474+0000 7f4a9be00640 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x55569f30d5b0 0x55569f3385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.474+0000 7f4a9be00640 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x55569f30d5b0 0x55569f3385e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa04a40aa54cdec6 server_cookie=49383eee72778e25 in_seq=0 out_seq=0 2024-04-25T09:08:58.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.474+0000 7f4a87e00640 10 client.6702 ms_handle_connect on v2:172.21.15.62:6832/836333339 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4a87e00640 1 -- 172.21.15.62:0/1505080247 <== mds.0 v2:172.21.15.62:6832/836333339 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55569f262310 con 0x55569f30d5b0 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4a87e00640 10 client.6702 handle_command_reply: tid=0 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 client.6702 shutdown 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 -- 172.21.15.62:0/1505080247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2eff50 msgr2=0x55569f30bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2eff50 0x55569f30bc40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400b470 tx=0x7f4a94088020 comp rx=0 tx=0).stop 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 -- 172.21.15.62:0/1505080247 shutdown_connections 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x55569f30d5b0 0x55569f3385e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 0x55569f30cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2f0e10 0x55569f31ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 --2- 172.21.15.62:0/1505080247 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2eff50 0x55569f30bc40 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4aac64b740 1 -- 172.21.15.62:0/1505080247 >> 172.21.15.62:0/1505080247 conn(0x55569f2ebca0 msgr2=0x55569f2eb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.475+0000 7f4a87e00640 0 client.6702 ms_handle_reset on v2:172.21.15.62:6832/836333339 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.476+0000 7f4aac64b740 1 -- 172.21.15.62:0/1505080247 shutdown_connections 2024-04-25T09:08:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.476+0000 7f4aac64b740 1 -- 172.21.15.62:0/1505080247 wait complete. 2024-04-25T09:08:58.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.476+0000 7f4aac64b740 20 client.6702 trim_cache size 0 max 16384 2024-04-25T09:08:58.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.493+0000 7f4aac64b740 1 Processor -- start 2024-04-25T09:08:58.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.493+0000 7f4aac64b740 1 -- start start 2024-04-25T09:08:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.493+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f30f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.493+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f339920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 0x55569f2f1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4aac64b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f350d60 con 0x55569f2f0e10 2024-04-25T09:08:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4aac64b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41ffdf0 con 0x55569f2eff50 2024-04-25T09:08:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4aac64b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41f5bb0 con 0x55569f31e310 2024-04-25T09:08:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4a9be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f339920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4a9be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f339920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52312/0 (socket says 172.21.15.62:52312) 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4a9be00640 1 -- 172.21.15.62:0/2040416387 learned_addr learned my addr 172.21.15.62:0/2040416387 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/2040416387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f30f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4a9be00640 1 -- 172.21.15.62:0/2040416387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 msgr2=0x55569f2f1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4a9be00640 1 --2- 172.21.15.62:0/2040416387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 0x55569f2f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4a9be00640 1 -- 172.21.15.62:0/2040416387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 msgr2=0x55569f30f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4a9be00640 1 --2- 172.21.15.62:0/2040416387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f30f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.494+0000 7f4a9be00640 1 -- 172.21.15.62:0/2040416387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55569ef02560 con 0x55569f2f0e10 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.495+0000 7f4a9be00640 1 --2- 172.21.15.62:0/2040416387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f339920 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4aa41625f0 tx=0x7f4a8c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dc5ae62c39c2b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.495+0000 7f4a92a00640 1 -- 172.21.15.62:0/2040416387 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c014830 con 0x55569f2f0e10 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.495+0000 7f4a92a00640 1 -- 172.21.15.62:0/2040416387 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c028070 con 0x55569f2f0e10 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.495+0000 7f4a92a00640 1 -- 172.21.15.62:0/2040416387 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0149f0 con 0x55569f2f0e10 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 -- 172.21.15.62:0/2040416387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 msgr2=0x55569f339920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 --2- 172.21.15.62:0/2040416387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f339920 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4aa41625f0 tx=0x7f4a8c010000 comp rx=0 tx=0).stop 2024-04-25T09:08:58.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 -- 172.21.15.62:0/2040416387 shutdown_connections 2024-04-25T09:08:58.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 --2- 172.21.15.62:0/2040416387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f2f0e10 0x55569f339920 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 --2- 172.21.15.62:0/2040416387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f31e310 0x55569f2f1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 --2- 172.21.15.62:0/2040416387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f30f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 -- 172.21.15.62:0/2040416387 >> 172.21.15.62:0/2040416387 conn(0x55569f2ebca0 msgr2=0x55569f2ef7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:58.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 -- 172.21.15.62:0/2040416387 shutdown_connections 2024-04-25T09:08:58.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.496+0000 7f4aac64b740 1 -- 172.21.15.62:0/2040416387 wait complete. 2024-04-25T09:08:58.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.497+0000 7f4aac64b740 1 -- start start 2024-04-25T09:08:58.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.497+0000 7f4aac64b740 1 Processor -- start 2024-04-25T09:08:58.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f39aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2f0e10 0x55569f30b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aac64b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f31e310 0x55569f30dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aac64b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41f55b0 con 0x55569f31e310 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aac64b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41364a0 con 0x55569f2eff50 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aac64b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569f2ef5d0 con 0x55569f2f0e10 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aac64b740 10 client.0 _ll_register_callbacks cb 0x7f4a940581f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f31e310 0x55569f30dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f31e310 0x55569f30dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52318/0 (socket says 172.21.15.62:52318) 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4aa3400640 1 -- 172.21.15.62:0/4198561145 learned_addr learned my addr 172.21.15.62:0/4198561145 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.498+0000 7f4a9be00640 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2f0e10 0x55569f30b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f39aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4aa3400640 1 -- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2f0e10 msgr2=0x55569f30b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4aa3400640 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2f0e10 0x55569f30b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4aa3400640 1 -- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 msgr2=0x55569f39aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4aa3400640 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f39aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4aa3400640 1 -- 172.21.15.62:0/4198561145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55569f31ce20 con 0x55569f31e310 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f39aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4aa3400640 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f31e310 0x55569f30dbc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009f40 tx=0x7f4a9c0318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c94ae2ab8229b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.499+0000 7f4a87e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4a87e00640 1 -- 172.21.15.62:0/4198561145 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c036040 con 0x55569f31e310 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4a87e00640 1 -- 172.21.15.62:0/4198561145 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4a9c046b50 con 0x55569f31e310 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 1 -- 172.21.15.62:0/4198561145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a8c028070 con 0x55569f31e310 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4a87e00640 1 -- 172.21.15.62:0/4198561145 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9c009510 con 0x55569f31e310 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 10 client.6708 fetch_fsmap learned FSMap version 45 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 1 -- 172.21.15.62:0/4198561145 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x55569f345410 con 0x55569f31e310 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4a87e00640 1 -- 172.21.15.62:0/4198561145 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c028070 con 0x55569f31e310 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4a87e00640 1 -- 172.21.15.62:0/4198561145 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 45) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a9c046020 con 0x55569f31e310 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 10 client.6708 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T09:08:58.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 10 client.6708 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:08:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 20 client.6708 populate_metadata read hostname 'smithi062' 2024-04-25T09:08:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x55569f30f070 0x55569f30f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:08:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 4 client.6708 mds_command: new command op to 6633 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T09:08:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.500+0000 7f4aac64b740 1 -- 172.21.15.62:0/4198561145 --> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x55569f25c660 con 0x55569f30f070 2024-04-25T09:08:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.501+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x55569f30f070 0x55569f30f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:08:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.501+0000 7f4aa2a00640 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x55569f30f070 0x55569f30f480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc27482abd4c4fbe server_cookie=8035b05688c38b8f in_seq=0 out_seq=0 2024-04-25T09:08:58.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.501+0000 7f4a87e00640 10 client.6708 ms_handle_connect on v2:172.21.15.62:6832/836333339 2024-04-25T09:08:58.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4a87e00640 1 -- 172.21.15.62:0/4198561145 <== mds.0 v2:172.21.15.62:6832/836333339 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55569f25c660 con 0x55569f30f070 2024-04-25T09:08:58.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4a87e00640 10 client.6708 handle_command_reply: tid=0 2024-04-25T09:08:58.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 client.6708 shutdown 2024-04-25T09:08:58.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 -- 172.21.15.62:0/4198561145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f31e310 msgr2=0x55569f30dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f31e310 0x55569f30dbc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009f40 tx=0x7f4a9c0318d0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 -- 172.21.15.62:0/4198561145 shutdown_connections 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.62:6832/836333339,v1:172.21.15.62:6833/836333339] conn(0x55569f30f070 0x55569f30f480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55569f31e310 0x55569f30dbc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55569f2f0e10 0x55569f30b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 --2- 172.21.15.62:0/4198561145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55569f2eff50 0x55569f39aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4a87e00640 0 client.6708 ms_handle_reset on v2:172.21.15.62:6832/836333339 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 -- 172.21.15.62:0/4198561145 >> 172.21.15.62:0/4198561145 conn(0x55569f2ebca0 msgr2=0x55569f30aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.502+0000 7f4aac64b740 1 -- 172.21.15.62:0/4198561145 shutdown_connections 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.503+0000 7f4aac64b740 1 -- 172.21.15.62:0/4198561145 wait complete. 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.503+0000 7f4aac64b740 20 client.6708 trim_cache size 0 max 16384 2024-04-25T09:08:58.493 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:08:58.494 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:08:58.494 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "ada1c476-44cd-4f82-a8e1-3ee1fcda45de", 2024-04-25T09:08:58.494 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:08:58.494 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:08:58.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.503+0000 7f4aa3e00640 1 -- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4a8006b4d0 msgr2=0x7f4a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.503+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4a8006b4d0 0x7f4a8006d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40af310 tx=0x7f4a8c008040 comp rx=0 tx=0).stop 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.503+0000 7f4aa3e00640 1 -- 172.21.15.62:0/3982971631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa406c440 msgr2=0x7f4aa40cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.503+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa406c440 0x7f4aa40cb190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400b4a0 tx=0x7f4a9400b960 comp rx=0 tx=0).stop 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.504+0000 7f4aa3e00640 1 -- 172.21.15.62:0/3982971631 shutdown_connections 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.504+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4a8006b4d0 0x7f4a8006d990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.504+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4aa415eb90 0x7f4aa4071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.504+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4aa415b520 0x7f4aa40711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.504+0000 7f4aa3e00640 1 --2- 172.21.15.62:0/3982971631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aa406c440 0x7f4aa40cb190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.504+0000 7f4aa3e00640 1 -- 172.21.15.62:0/3982971631 >> 172.21.15.62:0/3982971631 conn(0x7f4aa4066ad0 msgr2=0x7f4aa4066eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.504+0000 7f4aa3e00640 1 -- 172.21.15.62:0/3982971631 shutdown_connections 2024-04-25T09:08:58.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:08:58.504+0000 7f4aa3e00640 1 -- 172.21.15.62:0/3982971631 wait complete. 2024-04-25T09:08:58.508 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': 'ada1c476-44cd-4f82-a8e1-3ee1fcda45de', 'mode': 'asynchronous'}' 2024-04-25T09:09:08.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:08.510 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T09:09:08.575 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:09:08.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:08.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:08.795 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:09:08.795 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:09:08.795 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:08.855 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:09:08.855 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:08.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:09.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.139+0000 7f1b23000640 1 Processor -- start 2024-04-25T09:09:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.139+0000 7f1b23000640 1 -- start start 2024-04-25T09:09:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.141+0000 7f1b23000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c144c10 0x7f1b1c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.141+0000 7f1b23000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c15faa0 con 0x7f1b1c06d900 2024-04-25T09:09:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.141+0000 7f1b23000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c15fd80 con 0x7f1b1c0cee40 2024-04-25T09:09:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.141+0000 7f1b23000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c144ff0 con 0x7f1b1c144c10 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.142+0000 7f1b22600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1b1c06d900 0x7f1b1c0b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41186/0 (socket says 172.21.15.62:41186) 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.142+0000 7f1b22600640 1 -- 172.21.15.62:0/3286309737 learned_addr learned my addr 172.21.15.62:0/3286309737 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.142+0000 7f1b21c00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c144c10 0x7f1b1c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.142+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503711193 0 0) 0x7f1b1c15faa0 con 0x7f1b1c06d900 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.142+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b00003520 con 0x7f1b1c06d900 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392515372 0 0) 0x7f1b1c15fd80 con 0x7f1b1c0cee40 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b1c15faa0 con 0x7f1b1c0cee40 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1525841472 0 0) 0x7f1b00003520 con 0x7f1b1c06d900 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c144c10 msgr2=0x7f1b1c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c144c10 0x7f1b1c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 >> v1:172.21.15.121:6789/0 conn(0x7f1b1c0cee40 legacy=0x7f1b1c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c160060 con 0x7f1b1c06d900 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1b14001670 con 0x7f1b1c06d900 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 >> v1:172.21.15.62:6789/0 conn(0x7f1b1c06d900 legacy=0x7f1b1c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b00003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c144c10 0x7f1b00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b00004d40 0x7f1b000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c144ff0 con 0x7f1b00003b70 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0000bb50 con 0x7f1b00004d40 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0000bd20 con 0x7f1b1c144c10 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b21c00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b00003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.143+0000 7f1b21200640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c144c10 0x7f1b00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.144+0000 7f1b22600640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b00004d40 0x7f1b000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.144+0000 7f1b21c00640 1 -- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c144c10 msgr2=0x7f1b00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.144+0000 7f1b21c00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c144c10 0x7f1b00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.144+0000 7f1b21c00640 1 -- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b00004d40 msgr2=0x7f1b000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.144+0000 7f1b21c00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b00004d40 0x7f1b000051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.144+0000 7f1b21c00640 1 -- 172.21.15.62:0/3286309737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b00003520 con 0x7f1b00003b70 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.145+0000 7f1b21c00640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b00003f70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e170 tx=0x7f1b00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3afba04bd23b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.145+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c034020 con 0x7f1b00003b70 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.145+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c0305f0 con 0x7f1b00003b70 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.145+0000 7f1b1be00640 1 -- 172.21.15.62:0/3286309737 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c009c10 con 0x7f1b00003b70 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.146+0000 7f1b23000640 1 -- 172.21.15.62:0/3286309737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 msgr2=0x7f1b00003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.146+0000 7f1b23000640 1 --2- 172.21.15.62:0/3286309737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b00003f70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e170 tx=0x7f1b00005830 comp rx=0 tx=0).stop 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.146+0000 7f1b22600640 1 -- 172.21.15.62:0/3286309737 reap_dead start 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.146+0000 7f1b23000640 1 -- 172.21.15.62:0/3286309737 shutdown_connections 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.146+0000 7f1b23000640 1 -- 172.21.15.62:0/3286309737 >> 172.21.15.62:0/3286309737 conn(0x7f1b1c0b14d0 msgr2=0x7f1b1c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.146+0000 7f1b23000640 1 -- 172.21.15.62:0/3286309737 shutdown_connections 2024-04-25T09:09:09.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.146+0000 7f1b23000640 1 -- 172.21.15.62:0/3286309737 wait complete. 2024-04-25T09:09:09.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.148+0000 7f1b23000640 1 Processor -- start 2024-04-25T09:09:09.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.148+0000 7f1b23000640 1 -- start start 2024-04-25T09:09:09.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b23000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b1c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b23000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c140de0 0x7f1b1c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b23000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c15ae80 0x7f1b1c13f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b23000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0b41c0 con 0x7f1b00003b70 2024-04-25T09:09:09.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b23000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c160340 con 0x7f1b1c140de0 2024-04-25T09:09:09.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b23000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c15f450 con 0x7f1b1c15ae80 2024-04-25T09:09:09.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b21c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b1c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b21c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b1c15c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48214/0 (socket says 172.21.15.62:48214) 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b21c00640 1 -- 172.21.15.62:0/2415764431 learned_addr learned my addr 172.21.15.62:0/2415764431 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.149+0000 7f1b21200640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c140de0 0x7f1b1c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b22600640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c15ae80 0x7f1b1c13f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b21c00640 1 -- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c15ae80 msgr2=0x7f1b1c13f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b21c00640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c15ae80 0x7f1b1c13f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b21c00640 1 -- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c140de0 msgr2=0x7f1b1c15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b21c00640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c140de0 0x7f1b1c15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b21c00640 1 -- 172.21.15.62:0/2415764431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c15cde0 con 0x7f1b00003b70 2024-04-25T09:09:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b21c00640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b1c15c8a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e170 tx=0x7f1b0c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b19600640 1 -- 172.21.15.62:0/2415764431 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c040070 con 0x7f1b00003b70 2024-04-25T09:09:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b19600640 1 -- 172.21.15.62:0/2415764431 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c003990 con 0x7f1b00003b70 2024-04-25T09:09:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b19600640 1 -- 172.21.15.62:0/2415764431 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c034040 con 0x7f1b00003b70 2024-04-25T09:09:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.150+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c141270 con 0x7f1b00003b70 2024-04-25T09:09:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.151+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c141830 con 0x7f1b00003b70 2024-04-25T09:09:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.152+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c04ef20 con 0x7f1b00003b70 2024-04-25T09:09:09.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.153+0000 7f1b19600640 1 -- 172.21.15.62:0/2415764431 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b0c037020 con 0x7f1b00003b70 2024-04-25T09:09:09.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.154+0000 7f1b19600640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1afc06b4d0 0x7f1afc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.155+0000 7f1b19600640 1 -- 172.21.15.62:0/2415764431 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1b0c0aaee0 con 0x7f1b00003b70 2024-04-25T09:09:09.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.155+0000 7f1b21200640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1afc06b4d0 0x7f1afc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.156+0000 7f1b21200640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1afc06b4d0 0x7f1afc06d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1b08007fd0 tx=0x7f1b08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.159+0000 7f1b19600640 1 -- 172.21.15.62:0/2415764431 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b0c079c30 con 0x7f1b00003b70 2024-04-25T09:09:09.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.359+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b1c0b22b0 con 0x7f1b00003b70 2024-04-25T09:09:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.362+0000 7f1b19600640 1 -- 172.21.15.62:0/2415764431 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f1b0c079530 con 0x7f1b00003b70 2024-04-25T09:09:09.351 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:09.351 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":68,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:01.691769+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:34.817879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:35.997553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":6,"to":1}]},{"pgid":"9.37","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/3491368038":"2024-04-26T09:08:50.613426+0000","172.21.15.62:6832/3491368038":"2024-04-26T09:08:50.613426+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:09.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1afc06b4d0 msgr2=0x7f1afc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1afc06b4d0 0x7f1afc06d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1b08007fd0 tx=0x7f1b08009040 comp rx=0 tx=0).stop 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 msgr2=0x7f1b1c15c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b1c15c8a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e170 tx=0x7f1b0c009750 comp rx=0 tx=0).stop 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 shutdown_connections 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1afc06b4d0 0x7f1afc06d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1b1c15ae80 0x7f1b1c13f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b1c140de0 0x7f1b1c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 --2- 172.21.15.62:0/2415764431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b00003b70 0x7f1b1c15c8a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 >> 172.21.15.62:0/2415764431 conn(0x7f1b1c0b14d0 msgr2=0x7f1b1c0b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 shutdown_connections 2024-04-25T09:09:09.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.365+0000 7f1b23000640 1 -- 172.21.15.62:0/2415764431 wait complete. 2024-04-25T09:09:09.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:08:34.817879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:08:35.997553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:09.372 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:09.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.605+0000 7f274ae00640 1 Processor -- start 2024-04-25T09:09:09.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.605+0000 7f274ae00640 1 -- start start 2024-04-25T09:09:09.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.607+0000 7f274ae00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f27440d2fb0 0x7f27440ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.607+0000 7f274ae00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27440ce780 con 0x7f27440d3960 2024-04-25T09:09:09.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.607+0000 7f274ae00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27440d3440 con 0x7f274415dd50 2024-04-25T09:09:09.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.607+0000 7f274ae00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27440ce1c0 con 0x7f27440d2fb0 2024-04-25T09:09:09.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.608+0000 7f274a400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f27440d3960 0x7f27440d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41188/0 (socket says 172.21.15.62:41188) 2024-04-25T09:09:09.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.608+0000 7f274a400640 1 -- 172.21.15.62:0/2603628116 learned_addr learned my addr 172.21.15.62:0/2603628116 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.608+0000 7f2749000640 1 --2- 172.21.15.62:0/2603628116 >> v2:172.21.15.121:3300/0 conn(0x7f27440d2fb0 0x7f27440ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.608+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701465748 0 0) 0x7f27440ce780 con 0x7f27440d3960 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.608+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f272c003520 con 0x7f27440d3960 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1388026595 0 0) 0x7f272c003520 con 0x7f27440d3960 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 >> v2:172.21.15.121:3300/0 conn(0x7f27440d2fb0 msgr2=0x7f27440ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 --2- 172.21.15.62:0/2603628116 >> v2:172.21.15.121:3300/0 conn(0x7f27440d2fb0 0x7f27440ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 >> v1:172.21.15.121:6789/0 conn(0x7f274415dd50 legacy=0x7f27440d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27440ce4a0 con 0x7f27440d3960 2024-04-25T09:09:09.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f273c0019f0 con 0x7f27440d3960 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 >> v1:172.21.15.62:6789/0 conn(0x7f27440d3960 legacy=0x7f27440d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f272c0039b0 0x7f272c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f272c004800 0x7f272c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f272c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440ce1c0 con 0x7f272c0051c0 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c00c010 con 0x7f272c0039b0 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c00c210 con 0x7f272c004800 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f274a400640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f272c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.609+0000 7f2749000640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f272c004800 0x7f272c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.610+0000 7f274a400640 1 -- 172.21.15.62:0/2603628116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f272c004800 msgr2=0x7f272c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.610+0000 7f274a400640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f272c004800 0x7f272c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.610+0000 7f274a400640 1 -- 172.21.15.62:0/2603628116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f272c0039b0 msgr2=0x7f272c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.610+0000 7f274a400640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f272c0039b0 0x7f272c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.610+0000 7f274a400640 1 -- 172.21.15.62:0/2603628116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c003520 con 0x7f272c0051c0 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.610+0000 7f274a400640 1 -- 172.21.15.62:0/2603628116 reap_dead start 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.611+0000 7f274a400640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f272c00b9c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f273c00a7b0 tx=0x7f272c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fa71da8ca8745a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.611+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c00a3d0 con 0x7f272c0051c0 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.611+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c02f070 con 0x7f272c0051c0 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.611+0000 7f2743e00640 1 -- 172.21.15.62:0/2603628116 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c02bc20 con 0x7f272c0051c0 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.611+0000 7f274ae00640 1 -- 172.21.15.62:0/2603628116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 msgr2=0x7f272c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.611+0000 7f274ae00640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f272c00b9c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f273c00a7b0 tx=0x7f272c005830 comp rx=0 tx=0).stop 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.612+0000 7f274ae00640 1 -- 172.21.15.62:0/2603628116 shutdown_connections 2024-04-25T09:09:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.612+0000 7f274ae00640 1 --2- 172.21.15.62:0/2603628116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f272c00b9c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.612+0000 7f274ae00640 1 -- 172.21.15.62:0/2603628116 >> 172.21.15.62:0/2603628116 conn(0x7f274406b000 msgr2=0x7f274406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.612+0000 7f274ae00640 1 -- 172.21.15.62:0/2603628116 shutdown_connections 2024-04-25T09:09:09.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.612+0000 7f274ae00640 1 -- 172.21.15.62:0/2603628116 wait complete. 2024-04-25T09:09:09.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.613+0000 7f274ae00640 1 Processor -- start 2024-04-25T09:09:09.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.614+0000 7f274ae00640 1 -- start start 2024-04-25T09:09:09.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f274ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f2744157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f274ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274406d4c0 0x7f27440b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f274ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27440b5580 0x7f274406fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f274ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440d4140 con 0x7f272c0051c0 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f274ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27440d3fc0 con 0x7f27440b5580 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f274ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27440d42c0 con 0x7f274406d4c0 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f2744157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f2744157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48232/0 (socket says 172.21.15.62:48232) 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749a00640 1 -- 172.21.15.62:0/2527566975 learned_addr learned my addr 172.21.15.62:0/2527566975 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749000640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274406d4c0 0x7f27440b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749a00640 1 -- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274406d4c0 msgr2=0x7f27440b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749a00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274406d4c0 0x7f27440b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f274a400640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27440b5580 0x7f274406fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749a00640 1 -- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27440b5580 msgr2=0x7f274406fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749a00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27440b5580 0x7f274406fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749a00640 1 -- 172.21.15.62:0/2527566975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744152010 con 0x7f272c0051c0 2024-04-25T09:09:09.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.615+0000 7f2749000640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274406d4c0 0x7f27440b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:09.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.616+0000 7f2749a00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f2744157a70 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f273800c570 tx=0x7f273800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.616+0000 7f2741600640 1 -- 172.21.15.62:0/2527566975 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738005480 con 0x7f272c0051c0 2024-04-25T09:09:09.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.616+0000 7f2741600640 1 -- 172.21.15.62:0/2527566975 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27380035f0 con 0x7f272c0051c0 2024-04-25T09:09:09.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.616+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27441522f0 con 0x7f272c0051c0 2024-04-25T09:09:09.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.616+0000 7f2741600640 1 -- 172.21.15.62:0/2527566975 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738003a00 con 0x7f272c0051c0 2024-04-25T09:09:09.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.616+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27441528b0 con 0x7f272c0051c0 2024-04-25T09:09:09.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.617+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27440cef90 con 0x7f272c0051c0 2024-04-25T09:09:09.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.618+0000 7f2741600640 1 -- 172.21.15.62:0/2527566975 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2738003ba0 con 0x7f272c0051c0 2024-04-25T09:09:09.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.619+0000 7f2741600640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f272406b6a0 0x7f272406db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:09.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.619+0000 7f2741600640 1 -- 172.21.15.62:0/2527566975 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f273808a080 con 0x7f272c0051c0 2024-04-25T09:09:09.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.620+0000 7f2749000640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f272406b6a0 0x7f272406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:09.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.621+0000 7f2749000640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f272406b6a0 0x7f272406db60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f27440d01e0 tx=0x7f2730008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:09.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.624+0000 7f2741600640 1 -- 172.21.15.62:0/2527566975 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27440cef90 con 0x7f272c0051c0 2024-04-25T09:09:09.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.826+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f274404ef20 con 0x7f272c0051c0 2024-04-25T09:09:09.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.827+0000 7f2741600640 1 -- 172.21.15.62:0/2527566975 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2738058840 con 0x7f272c0051c0 2024-04-25T09:09:09.815 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f272406b6a0 msgr2=0x7f272406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f272406b6a0 0x7f272406db60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f27440d01e0 tx=0x7f2730008040 comp rx=0 tx=0).stop 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 msgr2=0x7f2744157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f2744157a70 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f273800c570 tx=0x7f273800ca30 comp rx=0 tx=0).stop 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 shutdown_connections 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f272406b6a0 0x7f272406db60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27440b5580 0x7f274406fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f274406d4c0 0x7f27440b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 --2- 172.21.15.62:0/2527566975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f272c0051c0 0x7f2744157a70 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.830+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 >> 172.21.15.62:0/2527566975 conn(0x7f274406b000 msgr2=0x7f27440cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.831+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 shutdown_connections 2024-04-25T09:09:09.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:09.831+0000 7f274ae00640 1 -- 172.21.15.62:0/2527566975 wait complete. 2024-04-25T09:09:09.838 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:09:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.067+0000 7f2da6200640 1 Processor -- start 2024-04-25T09:09:10.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.068+0000 7f2da6200640 1 -- start start 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2da6200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da00ccde0 0x7f2da00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2da6200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2da00b31a0 0x7f2da00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2da6200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da00ced40 con 0x7f2da006d2c0 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2da6200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da00cd260 con 0x7f2da00b31a0 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2da6200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00cf000 con 0x7f2da00ccde0 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2da5800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2da00b31a0 0x7f2da00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2da5800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2da00b31a0 0x7f2da00b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48248/0 (socket says 172.21.15.62:48248) 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2da5800640 1 -- 172.21.15.62:0/3811548863 learned_addr learned my addr 172.21.15.62:0/3811548863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.070+0000 7f2d9fe00640 1 --2- 172.21.15.62:0/3811548863 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da00ccde0 0x7f2da00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.071+0000 7f2da5800640 1 -- 172.21.15.62:0/3811548863 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da00ccde0 msgr2=0x7f2da00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.071+0000 7f2da5800640 1 --2- 172.21.15.62:0/3811548863 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da00ccde0 0x7f2da00ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.071+0000 7f2da5800640 1 -- 172.21.15.62:0/3811548863 >> v1:172.21.15.121:6789/0 conn(0x7f2da006d2c0 legacy=0x7f2da006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.071+0000 7f2da5800640 1 -- 172.21.15.62:0/3811548863 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da00cf2e0 con 0x7f2da00b31a0 2024-04-25T09:09:10.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.072+0000 7f2da5800640 1 --2- 172.21.15.62:0/3811548863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00b31a0 0x7f2da00b3f40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009870 tx=0x7f2d9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773035421f31ed94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.072+0000 7f2d9f400640 1 -- 172.21.15.62:0/3811548863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98037070 con 0x7f2da00b31a0 2024-04-25T09:09:10.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.072+0000 7f2d9f400640 1 -- 172.21.15.62:0/3811548863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98032440 con 0x7f2da00b31a0 2024-04-25T09:09:10.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.072+0000 7f2d9f400640 1 -- 172.21.15.62:0/3811548863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98032810 con 0x7f2da00b31a0 2024-04-25T09:09:10.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.073+0000 7f2da6200640 1 -- 172.21.15.62:0/3811548863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00b31a0 msgr2=0x7f2da00b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.073+0000 7f2da6200640 1 --2- 172.21.15.62:0/3811548863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00b31a0 0x7f2da00b3f40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009870 tx=0x7f2d9802a270 comp rx=0 tx=0).stop 2024-04-25T09:09:10.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.073+0000 7f2da6200640 1 -- 172.21.15.62:0/3811548863 shutdown_connections 2024-04-25T09:09:10.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.073+0000 7f2da6200640 1 --2- 172.21.15.62:0/3811548863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00b31a0 0x7f2da00b3f40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.073+0000 7f2da6200640 1 --2- 172.21.15.62:0/3811548863 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da00ccde0 0x7f2da00ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.073+0000 7f2da6200640 1 -- 172.21.15.62:0/3811548863 >> 172.21.15.62:0/3811548863 conn(0x7f2da0066f20 msgr2=0x7f2da00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:10.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.073+0000 7f2da6200640 1 -- 172.21.15.62:0/3811548863 shutdown_connections 2024-04-25T09:09:10.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.073+0000 7f2da6200640 1 -- 172.21.15.62:0/3811548863 wait complete. 2024-04-25T09:09:10.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.075+0000 7f2da6200640 1 Processor -- start 2024-04-25T09:09:10.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.075+0000 7f2da6200640 1 -- start start 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da6200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da006d2c0 0x7f2da01e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da6200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2da00b31a0 0x7f2da01e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00ccde0 0x7f2da00cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00b2c80 con 0x7f2da00ccde0 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da6200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00b29a0 con 0x7f2da00b31a0 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da6200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01448d0 con 0x7f2da006d2c0 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00ccde0 0x7f2da00cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00ccde0 0x7f2da00cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48258/0 (socket says 172.21.15.62:48258) 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.076+0000 7f2da5800640 1 -- 172.21.15.62:0/22540199 learned_addr learned my addr 172.21.15.62:0/22540199 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2da4e00640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da006d2c0 0x7f2da01e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2d9fe00640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2da00b31a0 0x7f2da01e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2da5800640 1 -- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da006d2c0 msgr2=0x7f2da01e4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2da5800640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da006d2c0 0x7f2da01e4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2da5800640 1 -- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2da00b31a0 msgr2=0x7f2da01e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2da5800640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2da00b31a0 0x7f2da01e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2da5800640 1 -- 172.21.15.62:0/22540199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da01e6a10 con 0x7f2da00ccde0 2024-04-25T09:09:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2da5800640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00ccde0 0x7f2da00cb6f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2d9802a2b0 tx=0x7f2d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.077+0000 7f2d9cc00640 1 -- 172.21.15.62:0/22540199 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98037040 con 0x7f2da00ccde0 2024-04-25T09:09:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.078+0000 7f2d9cc00640 1 -- 172.21.15.62:0/22540199 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98042070 con 0x7f2da00ccde0 2024-04-25T09:09:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.078+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da01e50b0 con 0x7f2da00ccde0 2024-04-25T09:09:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.078+0000 7f2d9cc00640 1 -- 172.21.15.62:0/22540199 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98030920 con 0x7f2da00ccde0 2024-04-25T09:09:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.078+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da01e5670 con 0x7f2da00ccde0 2024-04-25T09:09:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.079+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da006abe0 con 0x7f2da00ccde0 2024-04-25T09:09:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.080+0000 7f2d9cc00640 1 -- 172.21.15.62:0/22540199 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d98030ac0 con 0x7f2da00ccde0 2024-04-25T09:09:10.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.082+0000 7f2d9cc00640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.082+0000 7f2d9cc00640 1 -- 172.21.15.62:0/22540199 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2d980aabb0 con 0x7f2da00ccde0 2024-04-25T09:09:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.082+0000 7f2da4e00640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.083+0000 7f2da4e00640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2d90001ca0 tx=0x7f2d90009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.086+0000 7f2d9cc00640 1 -- 172.21.15.62:0/22540199 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d98079990 con 0x7f2da00ccde0 2024-04-25T09:09:10.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.288+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2da006ce50 con 0x7f2da00ccde0 2024-04-25T09:09:10.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.289+0000 7f2d9cc00640 1 -- 172.21.15.62:0/22540199 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d98079290 con 0x7f2da00ccde0 2024-04-25T09:09:10.277 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:09:10.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d7c06b4d0 msgr2=0x7f2d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2d90001ca0 tx=0x7f2d90009040 comp rx=0 tx=0).stop 2024-04-25T09:09:10.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00ccde0 msgr2=0x7f2da00cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00ccde0 0x7f2da00cb6f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2d9802a2b0 tx=0x7f2d980046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 shutdown_connections 2024-04-25T09:09:10.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2da00ccde0 0x7f2da00cb6f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2da00b31a0 0x7f2da01e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 --2- 172.21.15.62:0/22540199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2da006d2c0 0x7f2da01e4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 >> 172.21.15.62:0/22540199 conn(0x7f2da0066f20 msgr2=0x7f2da00cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:10.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 shutdown_connections 2024-04-25T09:09:10.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.292+0000 7f2da6200640 1 -- 172.21.15.62:0/22540199 wait complete. 2024-04-25T09:09:10.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:09:10.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.524+0000 7fec39000640 1 Processor -- start 2024-04-25T09:09:10.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.525+0000 7fec39000640 1 -- start start 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.526+0000 7fec39000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec3406d2c0 0x7fec340ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.526+0000 7fec39000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fec3406d6a0 0x7fec340ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.526+0000 7fec39000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec340b31a0 0x7fec340b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.526+0000 7fec39000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340ced40 con 0x7fec3406d2c0 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.527+0000 7fec39000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340cf000 con 0x7fec3406d6a0 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.527+0000 7fec39000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec340cf2e0 con 0x7fec340b31a0 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.527+0000 7fec33400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec3406d2c0 0x7fec340ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.527+0000 7fec33400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec3406d2c0 0x7fec340ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48270/0 (socket says 172.21.15.62:48270) 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.527+0000 7fec33400640 1 -- 172.21.15.62:0/4244140085 learned_addr learned my addr 172.21.15.62:0/4244140085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.527+0000 7fec33e00640 1 --2- 172.21.15.62:0/4244140085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec340b31a0 0x7fec340b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.527+0000 7fec2be00640 1 --2- 172.21.15.62:0/4244140085 >> v2:172.21.15.121:3300/0 conn(0x7fec3406d6a0 0x7fec340ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.528+0000 7fec33400640 1 -- 172.21.15.62:0/4244140085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec340b31a0 msgr2=0x7fec340b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:10.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.528+0000 7fec33400640 1 --2- 172.21.15.62:0/4244140085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec340b31a0 0x7fec340b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.528+0000 7fec33400640 1 -- 172.21.15.62:0/4244140085 >> v2:172.21.15.121:3300/0 conn(0x7fec3406d6a0 msgr2=0x7fec340ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.528+0000 7fec33400640 1 --2- 172.21.15.62:0/4244140085 >> v2:172.21.15.121:3300/0 conn(0x7fec3406d6a0 0x7fec340ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.528+0000 7fec33400640 1 -- 172.21.15.62:0/4244140085 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340b26c0 con 0x7fec3406d2c0 2024-04-25T09:09:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.529+0000 7fec33400640 1 --2- 172.21.15.62:0/4244140085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec340ccde0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fec340722f0 tx=0x7fec2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7228284b8c3d0c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.529+0000 7fec32a00640 1 -- 172.21.15.62:0/4244140085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24037070 con 0x7fec3406d2c0 2024-04-25T09:09:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.529+0000 7fec32a00640 1 -- 172.21.15.62:0/4244140085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24032440 con 0x7fec3406d2c0 2024-04-25T09:09:10.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.529+0000 7fec32a00640 1 -- 172.21.15.62:0/4244140085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24032810 con 0x7fec3406d2c0 2024-04-25T09:09:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 -- 172.21.15.62:0/4244140085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 msgr2=0x7fec340ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 --2- 172.21.15.62:0/4244140085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec340ccde0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fec340722f0 tx=0x7fec2402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 -- 172.21.15.62:0/4244140085 shutdown_connections 2024-04-25T09:09:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 --2- 172.21.15.62:0/4244140085 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec340b31a0 0x7fec340b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 --2- 172.21.15.62:0/4244140085 >> v2:172.21.15.121:3300/0 conn(0x7fec3406d6a0 0x7fec340ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 --2- 172.21.15.62:0/4244140085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec340ccde0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 -- 172.21.15.62:0/4244140085 >> 172.21.15.62:0/4244140085 conn(0x7fec34066f20 msgr2=0x7fec340b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 -- 172.21.15.62:0/4244140085 shutdown_connections 2024-04-25T09:09:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.530+0000 7fec39000640 1 -- 172.21.15.62:0/4244140085 wait complete. 2024-04-25T09:09:10.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.532+0000 7fec39000640 1 Processor -- start 2024-04-25T09:09:10.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.532+0000 7fec39000640 1 -- start start 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec39000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec341e7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec39000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec3406d6a0 0x7fec340ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec39000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec340b31a0 0x7fec340adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec39000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3406db20 con 0x7fec3406d2c0 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec39000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec340b2c80 con 0x7fec340b31a0 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec39000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec341448d0 con 0x7fec3406d6a0 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec33400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec341e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec33400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec341e7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48284/0 (socket says 172.21.15.62:48284) 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec33400640 1 -- 172.21.15.62:0/3039246520 learned_addr learned my addr 172.21.15.62:0/3039246520 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec2be00640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec3406d6a0 0x7fec340ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.533+0000 7fec33e00640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec340b31a0 0x7fec340adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec33400640 1 -- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec3406d6a0 msgr2=0x7fec340ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec33400640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec3406d6a0 0x7fec340ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec33400640 1 -- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec340b31a0 msgr2=0x7fec340adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec33400640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec340b31a0 0x7fec340adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec33400640 1 -- 172.21.15.62:0/3039246520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec34132ed0 con 0x7fec3406d2c0 2024-04-25T09:09:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec33400640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec341e7240 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fec2402a220 tx=0x7fec240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec2b400640 1 -- 172.21.15.62:0/3039246520 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24041020 con 0x7fec3406d2c0 2024-04-25T09:09:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec2b400640 1 -- 172.21.15.62:0/3039246520 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240306c0 con 0x7fec3406d2c0 2024-04-25T09:09:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec2b400640 1 -- 172.21.15.62:0/3039246520 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24037040 con 0x7fec3406d2c0 2024-04-25T09:09:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3412fff0 con 0x7fec3406d2c0 2024-04-25T09:09:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.534+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec341305b0 con 0x7fec3406d2c0 2024-04-25T09:09:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.536+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3406abe0 con 0x7fec3406d2c0 2024-04-25T09:09:10.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.537+0000 7fec2b400640 1 -- 172.21.15.62:0/3039246520 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec24044020 con 0x7fec3406d2c0 2024-04-25T09:09:10.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.537+0000 7fec2b400640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.538+0000 7fec2b400640 1 -- 172.21.15.62:0/3039246520 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fec240aab50 con 0x7fec3406d2c0 2024-04-25T09:09:10.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.538+0000 7fec2be00640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.539+0000 7fec2be00640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fec1006b4d0 0x7fec1006d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fec1c006fd0 tx=0x7fec1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.542+0000 7fec2b400640 1 -- 172.21.15.62:0/3039246520 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec2404b030 con 0x7fec3406d2c0 2024-04-25T09:09:10.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.741+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec3406b210 con 0x7fec3406d2c0 2024-04-25T09:09:10.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.742+0000 7fec2b400640 1 -- 172.21.15.62:0/3039246520 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec240798a0 con 0x7fec3406d2c0 2024-04-25T09:09:10.730 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:09:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.744+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fec1006b4d0 msgr2=0x7fec1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.744+0000 7fec39000640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fec1006b4d0 0x7fec1006d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fec1c006fd0 tx=0x7fec1c008040 comp rx=0 tx=0).stop 2024-04-25T09:09:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 msgr2=0x7fec341e7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec341e7240 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fec2402a220 tx=0x7fec240046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 shutdown_connections 2024-04-25T09:09:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fec1006b4d0 0x7fec1006d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec340b31a0 0x7fec340adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fec3406d6a0 0x7fec340ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 --2- 172.21.15.62:0/3039246520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec3406d2c0 0x7fec341e7240 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 >> 172.21.15.62:0/3039246520 conn(0x7fec34066f20 msgr2=0x7fec340af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:10.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 shutdown_connections 2024-04-25T09:09:10.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.745+0000 7fec39000640 1 -- 172.21.15.62:0/3039246520 wait complete. 2024-04-25T09:09:10.752 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:09:10.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.962+0000 7f2905c00640 1 Processor -- start 2024-04-25T09:09:10.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.963+0000 7f2905c00640 1 -- start start 2024-04-25T09:09:10.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.964+0000 7f2905c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f290006cc60 0x7f29000b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.964+0000 7f2905c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29000b2e00 0x7f29000adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.964+0000 7f2905c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29000ada60 con 0x7f29000cfaf0 2024-04-25T09:09:10.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.964+0000 7f2905c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290015e8e0 con 0x7f290006cc60 2024-04-25T09:09:10.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.964+0000 7f2905c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290015ebc0 con 0x7f29000b2e00 2024-04-25T09:09:10.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.965+0000 7f28f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f290006cc60 0x7f29000b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.965+0000 7f28f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f290006cc60 0x7f29000b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48298/0 (socket says 172.21.15.62:48298) 2024-04-25T09:09:10.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.965+0000 7f28f7e00640 1 -- 172.21.15.62:0/1328259792 learned_addr learned my addr 172.21.15.62:0/1328259792 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:10.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.965+0000 7f2905200640 1 --2- 172.21.15.62:0/1328259792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29000b2e00 0x7f29000adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.966+0000 7f28f7e00640 1 -- 172.21.15.62:0/1328259792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29000b2e00 msgr2=0x7f29000adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:10.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.966+0000 7f28f7e00640 1 --2- 172.21.15.62:0/1328259792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29000b2e00 0x7f29000adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.966+0000 7f28f7e00640 1 -- 172.21.15.62:0/1328259792 >> v1:172.21.15.62:6789/0 conn(0x7f29000cfaf0 legacy=0x7f290015e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.966+0000 7f28ff400640 1 -- 172.21.15.62:0/1328259792 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862222275 0 0) 0x7f29000ada60 con 0x7f29000cfaf0 2024-04-25T09:09:10.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.966+0000 7f28f7e00640 1 -- 172.21.15.62:0/1328259792 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290015eea0 con 0x7f290006cc60 2024-04-25T09:09:10.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.967+0000 7f28f7e00640 1 --2- 172.21.15.62:0/1328259792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f290006cc60 0x7f29000b28c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009870 tx=0x7f28e802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a6bddb035b3b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.967+0000 7f28ff400640 1 -- 172.21.15.62:0/1328259792 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8037070 con 0x7f290006cc60 2024-04-25T09:09:10.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.967+0000 7f28ff400640 1 -- 172.21.15.62:0/1328259792 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e802ae50 con 0x7f290006cc60 2024-04-25T09:09:10.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.967+0000 7f28ff400640 1 -- 172.21.15.62:0/1328259792 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8032860 con 0x7f290006cc60 2024-04-25T09:09:10.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.968+0000 7f2905c00640 1 -- 172.21.15.62:0/1328259792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f290006cc60 msgr2=0x7f29000b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.968+0000 7f2905c00640 1 --2- 172.21.15.62:0/1328259792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f290006cc60 0x7f29000b28c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009870 tx=0x7f28e802a300 comp rx=0 tx=0).stop 2024-04-25T09:09:10.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.968+0000 7f2905c00640 1 -- 172.21.15.62:0/1328259792 shutdown_connections 2024-04-25T09:09:10.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.968+0000 7f2905c00640 1 --2- 172.21.15.62:0/1328259792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f29000b2e00 0x7f29000adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.968+0000 7f2905c00640 1 --2- 172.21.15.62:0/1328259792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f290006cc60 0x7f29000b28c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.968+0000 7f2905c00640 1 -- 172.21.15.62:0/1328259792 >> 172.21.15.62:0/1328259792 conn(0x7f29000668f0 msgr2=0x7f2900066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:10.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.968+0000 7f2905c00640 1 -- 172.21.15.62:0/1328259792 shutdown_connections 2024-04-25T09:09:10.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.968+0000 7f2905c00640 1 -- 172.21.15.62:0/1328259792 wait complete. 2024-04-25T09:09:10.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.970+0000 7f2905c00640 1 Processor -- start 2024-04-25T09:09:10.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.970+0000 7f2905c00640 1 -- start start 2024-04-25T09:09:10.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f290006cc60 0x7f290014aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29000b2e00 0x7f29000cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29000cfaf0 0x7f29001f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29000ae620 con 0x7f29000cfaf0 2024-04-25T09:09:10.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29000ae4a0 con 0x7f29000b2e00 2024-04-25T09:09:10.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29000ae7a0 con 0x7f290006cc60 2024-04-25T09:09:10.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29000cfaf0 0x7f29001f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29000cfaf0 0x7f29001f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48312/0 (socket says 172.21.15.62:48312) 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.971+0000 7f2905200640 1 -- 172.21.15.62:0/3776968447 learned_addr learned my addr 172.21.15.62:0/3776968447 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f28ffe00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f290006cc60 0x7f290014aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f28f7e00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29000b2e00 0x7f29000cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f2905200640 1 -- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f290006cc60 msgr2=0x7f290014aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f2905200640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f290006cc60 0x7f290014aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f2905200640 1 -- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29000b2e00 msgr2=0x7f29000cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f2905200640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29000b2e00 0x7f29000cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f2905200640 1 -- 172.21.15.62:0/3776968447 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2900149f70 con 0x7f29000cfaf0 2024-04-25T09:09:10.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f2905200640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29000cfaf0 0x7f29001f34d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f28f8007e40 tx=0x7f28f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.972+0000 7f28fcc00640 1 -- 172.21.15.62:0/3776968447 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8014620 con 0x7f29000cfaf0 2024-04-25T09:09:10.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.973+0000 7f28fcc00640 1 -- 172.21.15.62:0/3776968447 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f8019070 con 0x7f29000cfaf0 2024-04-25T09:09:10.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.973+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29001f4000 con 0x7f29000cfaf0 2024-04-25T09:09:10.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.973+0000 7f28fcc00640 1 -- 172.21.15.62:0/3776968447 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8014a20 con 0x7f29000cfaf0 2024-04-25T09:09:10.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.973+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290014a230 con 0x7f29000cfaf0 2024-04-25T09:09:10.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.974+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290004ef20 con 0x7f29000cfaf0 2024-04-25T09:09:10.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.975+0000 7f28fcc00640 1 -- 172.21.15.62:0/3776968447 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28f8014bc0 con 0x7f29000cfaf0 2024-04-25T09:09:10.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.976+0000 7f28fcc00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f28dc06b4d0 0x7f28dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:10.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.976+0000 7f28fcc00640 1 -- 172.21.15.62:0/3776968447 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f28f808ac70 con 0x7f29000cfaf0 2024-04-25T09:09:10.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.977+0000 7f28ffe00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f28dc06b4d0 0x7f28dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:10.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.978+0000 7f28ffe00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f28dc06b4d0 0x7f28dc06d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f28f0000f60 tx=0x7f28f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:10.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:10.980+0000 7f28fcc00640 1 -- 172.21.15.62:0/3776968447 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28f80599c0 con 0x7f29000cfaf0 2024-04-25T09:09:11.078 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:09:11.086 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:09:11.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.195+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f290006da40 con 0x7f29000cfaf0 2024-04-25T09:09:11.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.196+0000 7f28fcc00640 1 -- 172.21.15.62:0/3776968447 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v68) v1 ==== 80+0+122 (secure 0 0 0) 0x7f28f80592c0 con 0x7f29000cfaf0 2024-04-25T09:09:11.185 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6834/3491368038 2024-04-26T09:08:50.613426+0000 2024-04-25T09:09:11.185 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6832/3491368038 2024-04-26T09:08:50.613426+0000 2024-04-25T09:09:11.185 INFO:teuthology.orchestra.run.smithi062.stderr:listed 2 entries 2024-04-25T09:09:11.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f28dc06b4d0 msgr2=0x7f28dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f28dc06b4d0 0x7f28dc06d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f28f0000f60 tx=0x7f28f0009040 comp rx=0 tx=0).stop 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29000cfaf0 msgr2=0x7f29001f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29000cfaf0 0x7f29001f34d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f28f8007e40 tx=0x7f28f800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 shutdown_connections 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f28dc06b4d0 0x7f28dc06d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f29000cfaf0 0x7f29001f34d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29000b2e00 0x7f29000cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 --2- 172.21.15.62:0/3776968447 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f290006cc60 0x7f290014aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 >> 172.21.15.62:0/3776968447 conn(0x7f29000668f0 msgr2=0x7f290015f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 shutdown_connections 2024-04-25T09:09:11.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.199+0000 7f2905c00640 1 -- 172.21.15.62:0/3776968447 wait complete. 2024-04-25T09:09:11.207 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:09:11.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:11.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:11.349 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:09:11.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:11.349 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:09:11.399 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:09:11.399 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:11.456 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:11.456 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:09:11.456 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:11.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:09:11.515 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:09:11.515 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:11.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:11.570 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.805+0000 7f778de00640 1 Processor -- start 2024-04-25T09:09:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.805+0000 7f778de00640 1 -- start start 2024-04-25T09:09:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.807+0000 7f778de00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f778815dd50 0x7f778806d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.807+0000 7f778de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.807+0000 7f778de00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77880aec20 con 0x7f77880ae7e0 2024-04-25T09:09:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.807+0000 7f778de00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7788162980 con 0x7f778815dd50 2024-04-25T09:09:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.807+0000 7f778de00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778806e120 con 0x7f77880ceac0 2024-04-25T09:09:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.808+0000 7f7787400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.808+0000 7f7787400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:60204/0 (socket says 172.21.15.62:60204) 2024-04-25T09:09:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.808+0000 7f7787400640 1 -- 172.21.15.62:0/922822390 learned_addr learned my addr 172.21.15.62:0/922822390 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.808+0000 7f7786a00640 1 --2- 172.21.15.62:0/922822390 >> v2:172.21.15.121:3300/0 conn(0x7f778815dd50 0x7f778806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.808+0000 7f7787400640 1 -- 172.21.15.62:0/922822390 >> v2:172.21.15.121:3300/0 conn(0x7f778815dd50 msgr2=0x7f778806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.808+0000 7f7787400640 1 --2- 172.21.15.62:0/922822390 >> v2:172.21.15.121:3300/0 conn(0x7f778815dd50 0x7f778806d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.808+0000 7f7787400640 1 -- 172.21.15.62:0/922822390 >> v1:172.21.15.121:6789/0 conn(0x7f77880ae7e0 legacy=0x7f7788162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.808+0000 7f7787400640 1 -- 172.21.15.62:0/922822390 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778806db60 con 0x7f77880ceac0 2024-04-25T09:09:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.809+0000 7f7787400640 1 --2- 172.21.15.62:0/922822390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880cef50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7780009e50 tx=0x7f778002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2a045f5862d4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.810+0000 7f7786000640 1 -- 172.21.15.62:0/922822390 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780036070 con 0x7f77880ceac0 2024-04-25T09:09:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.810+0000 7f7786000640 1 -- 172.21.15.62:0/922822390 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7780031600 con 0x7f77880ceac0 2024-04-25T09:09:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.810+0000 7f7786000640 1 -- 172.21.15.62:0/922822390 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77800319d0 con 0x7f77880ceac0 2024-04-25T09:09:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.811+0000 7f778de00640 1 -- 172.21.15.62:0/922822390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 msgr2=0x7f77880cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.811+0000 7f778de00640 1 --2- 172.21.15.62:0/922822390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880cef50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7780009e50 tx=0x7f778002a710 comp rx=0 tx=0).stop 2024-04-25T09:09:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.811+0000 7f778de00640 1 -- 172.21.15.62:0/922822390 shutdown_connections 2024-04-25T09:09:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.811+0000 7f778de00640 1 --2- 172.21.15.62:0/922822390 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880cef50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.811+0000 7f778de00640 1 --2- 172.21.15.62:0/922822390 >> v2:172.21.15.121:3300/0 conn(0x7f778815dd50 0x7f778806d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.811+0000 7f778de00640 1 -- 172.21.15.62:0/922822390 >> 172.21.15.62:0/922822390 conn(0x7f77880668f0 msgr2=0x7f7788066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.811+0000 7f778de00640 1 -- 172.21.15.62:0/922822390 shutdown_connections 2024-04-25T09:09:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.811+0000 7f778de00640 1 -- 172.21.15.62:0/922822390 wait complete. 2024-04-25T09:09:11.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.813+0000 7f778de00640 1 Processor -- start 2024-04-25T09:09:11.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.813+0000 7f778de00640 1 -- start start 2024-04-25T09:09:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.814+0000 7f778de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77880ae7e0 0x7f77881e8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.814+0000 7f778de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.814+0000 7f778de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778815dd50 0x7f77880b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.814+0000 7f778de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7788161d50 con 0x7f77880ae7e0 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.814+0000 7f778de00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7788161a70 con 0x7f778815dd50 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.814+0000 7f778de00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77880cf490 con 0x7f77880ceac0 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7786a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77880ae7e0 0x7f77881e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7786a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77880ae7e0 0x7f77881e8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48328/0 (socket says 172.21.15.62:48328) 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7786a00640 1 -- 172.21.15.62:0/1125910025 learned_addr learned my addr 172.21.15.62:0/1125910025 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f777e000640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7787400640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778815dd50 0x7f77880b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7786a00640 1 -- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 msgr2=0x7f77880b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7786a00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7786a00640 1 -- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778815dd50 msgr2=0x7f77880b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7786a00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778815dd50 0x7f77880b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.815+0000 7f7786a00640 1 -- 172.21.15.62:0/1125910025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77881e8870 con 0x7f77880ae7e0 2024-04-25T09:09:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.816+0000 7f7786a00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77880ae7e0 0x7f77881e8330 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f777800d5f0 tx=0x7f777800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.816+0000 7f777fe00640 1 -- 172.21.15.62:0/1125910025 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77780125f0 con 0x7f77880ae7e0 2024-04-25T09:09:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.816+0000 7f777fe00640 1 -- 172.21.15.62:0/1125910025 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7778019070 con 0x7f77880ae7e0 2024-04-25T09:09:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.816+0000 7f777fe00640 1 -- 172.21.15.62:0/1125910025 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77780129d0 con 0x7f77880ae7e0 2024-04-25T09:09:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.816+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77881f7590 con 0x7f77880ae7e0 2024-04-25T09:09:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.816+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77881f7b50 con 0x7f77880ae7e0 2024-04-25T09:09:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.817+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7788072c70 con 0x7f77880ae7e0 2024-04-25T09:09:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.818+0000 7f777fe00640 1 -- 172.21.15.62:0/1125910025 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7778012b70 con 0x7f77880ae7e0 2024-04-25T09:09:11.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.819+0000 7f777fe00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f776406b4d0 0x7f776406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:11.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.819+0000 7f777fe00640 1 -- 172.21.15.62:0/1125910025 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f777808a6d0 con 0x7f77880ae7e0 2024-04-25T09:09:11.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.820+0000 7f777e000640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f776406b4d0 0x7f776406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:11.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.821+0000 7f777e000640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f776406b4d0 0x7f776406d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7788159140 tx=0x7f7770009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:11.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:11.824+0000 7f777fe00640 1 -- 172.21.15.62:0/1125910025 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7778059420 con 0x7f77880ae7e0 2024-04-25T09:09:12.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.070+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f778804f000 con 0x7f77880ae7e0 2024-04-25T09:09:12.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.070+0000 7f777fe00640 1 -- 172.21.15.62:0/1125910025 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f7778058d20 con 0x7f77880ae7e0 2024-04-25T09:09:12.059 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:12.059 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1396005512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1396005512},{"type":"v1","addr":"172.21.15.121:6835","nonce":1396005512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3589939339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3589939339},{"type":"v1","addr":"172.21.15.121:6837","nonce":3589939339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2551168226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2551168226},{"type":"v1","addr":"172.21.15.62:6835","nonce":2551168226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:53.657317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6833/836333339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":836333339},{"type":"v1","addr":"172.21.15.62:6833","nonce":836333339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:09:12.059 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 45 2024-04-25T09:09:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f776406b4d0 msgr2=0x7f776406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f776406b4d0 0x7f776406d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7788159140 tx=0x7f7770009040 comp rx=0 tx=0).stop 2024-04-25T09:09:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77880ae7e0 msgr2=0x7f77881e8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77880ae7e0 0x7f77881e8330 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f777800d5f0 tx=0x7f777800dab0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 shutdown_connections 2024-04-25T09:09:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f776406b4d0 0x7f776406d990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f778815dd50 0x7f77880b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f77880ceac0 0x7f77880b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 --2- 172.21.15.62:0/1125910025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77880ae7e0 0x7f77881e8330 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.073+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 >> 172.21.15.62:0/1125910025 conn(0x7f77880668f0 msgr2=0x7f77880b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.074+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 shutdown_connections 2024-04-25T09:09:12.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.074+0000 7f778de00640 1 -- 172.21.15.62:0/1125910025 wait complete. 2024-04-25T09:09:12.081 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.300+0000 7f247f200640 1 Processor -- start 2024-04-25T09:09:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.300+0000 7f247f200640 1 -- start start 2024-04-25T09:09:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.302+0000 7f247f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2480161f00 0x7f24801622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.302+0000 7f247f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24800b2a20 con 0x7f24800cc9e0 2024-04-25T09:09:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.302+0000 7f247f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24800b2d00 con 0x7f248006d900 2024-04-25T09:09:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.302+0000 7f247f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24800b27d0 con 0x7f2480161f00 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.303+0000 7f2477e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f24800cc9e0 0x7f248006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41210/0 (socket says 172.21.15.62:41210) 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.303+0000 7f2477e00640 1 -- 172.21.15.62:0/1250222720 learned_addr learned my addr 172.21.15.62:0/1250222720 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.303+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866973787 0 0) 0x7f24800b2a20 con 0x7f24800cc9e0 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.303+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2464003520 con 0x7f24800cc9e0 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.303+0000 7f247de00640 1 --2- 172.21.15.62:0/1250222720 >> v2:172.21.15.121:3300/0 conn(0x7f2480161f00 0x7f24801622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482462499 0 0) 0x7f2464003520 con 0x7f24800cc9e0 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 >> v2:172.21.15.121:3300/0 conn(0x7f2480161f00 msgr2=0x7f24801622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 --2- 172.21.15.62:0/1250222720 >> v2:172.21.15.121:3300/0 conn(0x7f2480161f00 0x7f24801622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 >> v1:172.21.15.121:6789/0 conn(0x7f248006d900 legacy=0x7f24800b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24800b2fe0 con 0x7f24800cc9e0 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f2468001670 con 0x7f24800cc9e0 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 >> v1:172.21.15.62:6789/0 conn(0x7f24800cc9e0 legacy=0x7f248006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2464003a00 0x7f2464003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2464004800 0x7f2464004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24640052e0 0x7f246400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800b27d0 con 0x7f24640052e0 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246400c230 con 0x7f2464003a00 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246400c430 con 0x7f2464004800 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247de00640 1 --2- 172.21.15.62:0/1250222720 >> v2:172.21.15.121:3300/0 conn(0x7f2480161f00 0x7f24801622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247e800640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24640052e0 0x7f246400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f247de00640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2464003a00 0x7f2464003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.304+0000 7f2477e00640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2464004800 0x7f2464004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.305+0000 7f247e800640 1 -- 172.21.15.62:0/1250222720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2464004800 msgr2=0x7f2464004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:12.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.305+0000 7f247e800640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2464004800 0x7f2464004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.305+0000 7f247e800640 1 -- 172.21.15.62:0/1250222720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2464003a00 msgr2=0x7f2464003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.305+0000 7f247e800640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2464003a00 0x7f2464003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.305+0000 7f247e800640 1 -- 172.21.15.62:0/1250222720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2464003520 con 0x7f24640052e0 2024-04-25T09:09:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.306+0000 7f247e800640 1 -- 172.21.15.62:0/1250222720 reap_dead start 2024-04-25T09:09:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.306+0000 7f247e800640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24640052e0 0x7f246400b9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f248006e1e0 tx=0x7f2464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e624f7c3f17ca8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.306+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478004080 con 0x7f24640052e0 2024-04-25T09:09:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.306+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2478006d70 con 0x7f24640052e0 2024-04-25T09:09:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.306+0000 7f247d400640 1 -- 172.21.15.62:0/1250222720 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247802ad70 con 0x7f24640052e0 2024-04-25T09:09:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.307+0000 7f247f200640 1 -- 172.21.15.62:0/1250222720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24640052e0 msgr2=0x7f246400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.307+0000 7f247f200640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24640052e0 0x7f246400b9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f248006e1e0 tx=0x7f2464005830 comp rx=0 tx=0).stop 2024-04-25T09:09:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.307+0000 7f247f200640 1 -- 172.21.15.62:0/1250222720 shutdown_connections 2024-04-25T09:09:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.307+0000 7f247f200640 1 --2- 172.21.15.62:0/1250222720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f24640052e0 0x7f246400b9c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.307+0000 7f247f200640 1 -- 172.21.15.62:0/1250222720 >> 172.21.15.62:0/1250222720 conn(0x7f24800b14d0 msgr2=0x7f24800b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.307+0000 7f247f200640 1 -- 172.21.15.62:0/1250222720 shutdown_connections 2024-04-25T09:09:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.307+0000 7f247f200640 1 -- 172.21.15.62:0/1250222720 wait complete. 2024-04-25T09:09:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.309+0000 7f247f200640 1 Processor -- start 2024-04-25T09:09:12.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.309+0000 7f247f200640 1 -- start start 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24640052e0 0x7f248015d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24801fad60 0x7f2480152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2480152fb0 0x7f248015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800cce70 con 0x7f2480152fb0 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24800b35a0 con 0x7f24801fad60 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24800ccff0 con 0x7f24640052e0 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2480152fb0 0x7f248015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2480152fb0 0x7f248015c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48344/0 (socket says 172.21.15.62:48344) 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.310+0000 7f247e800640 1 -- 172.21.15.62:0/728773720 learned_addr learned my addr 172.21.15.62:0/728773720 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.311+0000 7f247de00640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24640052e0 0x7f248015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.311+0000 7f2477e00640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24801fad60 0x7f2480152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.311+0000 7f247e800640 1 -- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24640052e0 msgr2=0x7f248015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.311+0000 7f247e800640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24640052e0 0x7f248015d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.311+0000 7f247e800640 1 -- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24801fad60 msgr2=0x7f2480152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.311+0000 7f247e800640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24801fad60 0x7f2480152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.311+0000 7f247e800640 1 -- 172.21.15.62:0/728773720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24801fb520 con 0x7f2480152fb0 2024-04-25T09:09:12.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.311+0000 7f247e800640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2480152fb0 0x7f248015c4c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2478000f80 tx=0x7f2478003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.312+0000 7f2476000640 1 -- 172.21.15.62:0/728773720 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478007df0 con 0x7f2480152fb0 2024-04-25T09:09:12.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.312+0000 7f2476000640 1 -- 172.21.15.62:0/728773720 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2478042070 con 0x7f2480152fb0 2024-04-25T09:09:12.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.312+0000 7f2476000640 1 -- 172.21.15.62:0/728773720 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478004080 con 0x7f2480152fb0 2024-04-25T09:09:12.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.312+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24801fb800 con 0x7f2480152fb0 2024-04-25T09:09:12.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.312+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248015bfa0 con 0x7f2480152fb0 2024-04-25T09:09:12.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.313+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248015bd90 con 0x7f2480152fb0 2024-04-25T09:09:12.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.314+0000 7f2476000640 1 -- 172.21.15.62:0/728773720 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2478047460 con 0x7f2480152fb0 2024-04-25T09:09:12.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.315+0000 7f2476000640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f245c06b550 0x7f245c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.315+0000 7f2476000640 1 -- 172.21.15.62:0/728773720 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f24780aa760 con 0x7f2480152fb0 2024-04-25T09:09:12.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.316+0000 7f247de00640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f245c06b550 0x7f245c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.317+0000 7f247de00640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f245c06b550 0x7f245c06da10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f247000e950 tx=0x7f2470008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.319+0000 7f2476000640 1 -- 172.21.15.62:0/728773720 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2478079530 con 0x7f2480152fb0 2024-04-25T09:09:12.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.519+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f248004ef20 con 0x7f2480152fb0 2024-04-25T09:09:12.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.523+0000 7f2476000640 1 -- 172.21.15.62:0/728773720 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f2478078e30 con 0x7f2480152fb0 2024-04-25T09:09:12.511 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:12.511 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":68,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:01.691769+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:34.817879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:35.997553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":6,"to":1}]},{"pgid":"9.37","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/3491368038":"2024-04-26T09:08:50.613426+0000","172.21.15.62:6832/3491368038":"2024-04-26T09:08:50.613426+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.525+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f245c06b550 msgr2=0x7f245c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.525+0000 7f247f200640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f245c06b550 0x7f245c06da10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f247000e950 tx=0x7f2470008040 comp rx=0 tx=0).stop 2024-04-25T09:09:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.525+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2480152fb0 msgr2=0x7f248015c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.525+0000 7f247f200640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2480152fb0 0x7f248015c4c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2478000f80 tx=0x7f2478003a70 comp rx=0 tx=0).stop 2024-04-25T09:09:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.526+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 shutdown_connections 2024-04-25T09:09:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.526+0000 7f247f200640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f245c06b550 0x7f245c06da10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.526+0000 7f247f200640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2480152fb0 0x7f248015c4c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.526+0000 7f247f200640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24801fad60 0x7f2480152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.526+0000 7f247f200640 1 --2- 172.21.15.62:0/728773720 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24640052e0 0x7f248015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.526+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 >> 172.21.15.62:0/728773720 conn(0x7f24800b14d0 msgr2=0x7f2480071a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.526+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 shutdown_connections 2024-04-25T09:09:12.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.526+0000 7f247f200640 1 -- 172.21.15.62:0/728773720 wait complete. 2024-04-25T09:09:12.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:08:34.817879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:08:35.997553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:12.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:12.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.763+0000 7f0297200640 1 Processor -- start 2024-04-25T09:09:12.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.764+0000 7f0297200640 1 -- start start 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.765+0000 7f0297200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f029815dd50 0x7f02980d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.765+0000 7f0297200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02980ce780 con 0x7f02980d2fb0 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.765+0000 7f0297200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02980d3440 con 0x7f02980d3960 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.765+0000 7f0297200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02980ce1c0 con 0x7f029815dd50 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.765+0000 7f0295400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f02980d2fb0 0x7f02980ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41218/0 (socket says 172.21.15.62:41218) 2024-04-25T09:09:12.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.765+0000 7f0295400640 1 -- 172.21.15.62:0/2267129495 learned_addr learned my addr 172.21.15.62:0/2267129495 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:12.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.766+0000 7f0295e00640 1 --2- 172.21.15.62:0/2267129495 >> v2:172.21.15.121:3300/0 conn(0x7f029815dd50 0x7f02980d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.767+0000 7f0294a00640 1 -- 172.21.15.62:0/2267129495 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223871666 0 0) 0x7f02980ce780 con 0x7f02980d2fb0 2024-04-25T09:09:12.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.767+0000 7f0294a00640 1 -- 172.21.15.62:0/2267129495 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027c003520 con 0x7f02980d2fb0 2024-04-25T09:09:12.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.767+0000 7f0295e00640 1 -- 172.21.15.62:0/2267129495 >> v1:172.21.15.121:6789/0 conn(0x7f02980d3960 legacy=0x7f02980d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.767+0000 7f0295e00640 1 -- 172.21.15.62:0/2267129495 >> v1:172.21.15.62:6789/0 conn(0x7f02980d2fb0 legacy=0x7f02980ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.767+0000 7f0295e00640 1 -- 172.21.15.62:0/2267129495 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02980ce4a0 con 0x7f029815dd50 2024-04-25T09:09:12.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.768+0000 7f0295e00640 1 --2- 172.21.15.62:0/2267129495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029815dd50 0x7f02980d2300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0288009d90 tx=0x7f028802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ff9e3801dab84d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.769+0000 7f0294a00640 1 -- 172.21.15.62:0/2267129495 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288007740 con 0x7f029815dd50 2024-04-25T09:09:12.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.769+0000 7f0294a00640 1 -- 172.21.15.62:0/2267129495 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880078e0 con 0x7f029815dd50 2024-04-25T09:09:12.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.769+0000 7f0294a00640 1 -- 172.21.15.62:0/2267129495 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288007c90 con 0x7f029815dd50 2024-04-25T09:09:12.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.770+0000 7f0297200640 1 -- 172.21.15.62:0/2267129495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029815dd50 msgr2=0x7f02980d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.770+0000 7f0297200640 1 --2- 172.21.15.62:0/2267129495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029815dd50 0x7f02980d2300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0288009d90 tx=0x7f028802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.770+0000 7f0297200640 1 -- 172.21.15.62:0/2267129495 shutdown_connections 2024-04-25T09:09:12.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.770+0000 7f0297200640 1 --2- 172.21.15.62:0/2267129495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f029815dd50 0x7f02980d2300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.770+0000 7f0297200640 1 -- 172.21.15.62:0/2267129495 >> 172.21.15.62:0/2267129495 conn(0x7f029806b000 msgr2=0x7f029806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:12.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.770+0000 7f0297200640 1 -- 172.21.15.62:0/2267129495 shutdown_connections 2024-04-25T09:09:12.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.770+0000 7f0297200640 1 -- 172.21.15.62:0/2267129495 wait complete. 2024-04-25T09:09:12.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.772+0000 7f0297200640 1 Processor -- start 2024-04-25T09:09:12.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.772+0000 7f0297200640 1 -- start start 2024-04-25T09:09:12.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0297200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02980d3960 0x7f02980b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0297200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f029815dd50 0x7f02981f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0297200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02980b3f80 0x7f02980b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0297200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980b6c80 con 0x7f02980d3960 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0297200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980b6b00 con 0x7f02980b3f80 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0297200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980b6e00 con 0x7f029815dd50 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0295e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02980d3960 0x7f02980b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0295e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02980d3960 0x7f02980b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48352/0 (socket says 172.21.15.62:48352) 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0295e00640 1 -- 172.21.15.62:0/34633178 learned_addr learned my addr 172.21.15.62:0/34633178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0295400640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f029815dd50 0x7f02981f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.773+0000 7f0296800640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02980b3f80 0x7f02980b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f0295e00640 1 -- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f029815dd50 msgr2=0x7f02981f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f0295e00640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f029815dd50 0x7f02981f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f0295e00640 1 -- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02980b3f80 msgr2=0x7f02980b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f0295e00640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02980b3f80 0x7f02980b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f0295e00640 1 -- 172.21.15.62:0/34633178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02981f6e40 con 0x7f02980d3960 2024-04-25T09:09:12.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f0295e00640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02980d3960 0x7f02980b5570 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0288009eb0 tx=0x7f0288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f028e000640 1 -- 172.21.15.62:0/34633178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288009450 con 0x7f02980d3960 2024-04-25T09:09:12.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f028e000640 1 -- 172.21.15.62:0/34633178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0288004240 con 0x7f02980d3960 2024-04-25T09:09:12.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f028e000640 1 -- 172.21.15.62:0/34633178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880045f0 con 0x7f02980d3960 2024-04-25T09:09:12.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02980c9250 con 0x7f02980d3960 2024-04-25T09:09:12.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.774+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02980c8f70 con 0x7f02980d3960 2024-04-25T09:09:12.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.776+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029804ef20 con 0x7f02980d3960 2024-04-25T09:09:12.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.777+0000 7f028e000640 1 -- 172.21.15.62:0/34633178 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f028803b410 con 0x7f02980d3960 2024-04-25T09:09:12.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.778+0000 7f028e000640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f027406b500 0x7f027406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:12.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.778+0000 7f028e000640 1 -- 172.21.15.62:0/34633178 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f02880ab4e0 con 0x7f02980d3960 2024-04-25T09:09:12.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.779+0000 7f0295400640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f027406b500 0x7f027406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:12.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.780+0000 7f0295400640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f027406b500 0x7f027406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f02980ca460 tx=0x7f0280007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:12.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.783+0000 7f028e000640 1 -- 172.21.15.62:0/34633178 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f028803f030 con 0x7f02980d3960 2024-04-25T09:09:12.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.985+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02980b9950 con 0x7f02980d3960 2024-04-25T09:09:12.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.986+0000 7f028e000640 1 -- 172.21.15.62:0/34633178 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0288079c70 con 0x7f02980d3960 2024-04-25T09:09:12.974 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f027406b500 msgr2=0x7f027406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f027406b500 0x7f027406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f02980ca460 tx=0x7f0280007d30 comp rx=0 tx=0).stop 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02980d3960 msgr2=0x7f02980b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02980d3960 0x7f02980b5570 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0288009eb0 tx=0x7f0288004a40 comp rx=0 tx=0).stop 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 shutdown_connections 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f027406b500 0x7f027406d9c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02980b3f80 0x7f02980b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f029815dd50 0x7f02981f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 --2- 172.21.15.62:0/34633178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02980d3960 0x7f02980b5570 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.988+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 >> 172.21.15.62:0/34633178 conn(0x7f029806b000 msgr2=0x7f02980ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.989+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 shutdown_connections 2024-04-25T09:09:12.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:12.989+0000 7f0297200640 1 -- 172.21.15.62:0/34633178 wait complete. 2024-04-25T09:09:12.995 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:09:13.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.222+0000 7ff77b200640 1 Processor -- start 2024-04-25T09:09:13.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.222+0000 7ff77b200640 1 -- start start 2024-04-25T09:09:13.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.224+0000 7ff77b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c06d9d0 0x7ff77c15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.224+0000 7ff77b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff77c15e5f0 0x7ff77c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.224+0000 7ff77b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c15ee30 con 0x7ff77c0ceb50 2024-04-25T09:09:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.224+0000 7ff77b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c1618b0 con 0x7ff77c15e5f0 2024-04-25T09:09:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.224+0000 7ff77b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c161b90 con 0x7ff77c06d9d0 2024-04-25T09:09:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.225+0000 7ff77a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff77c15e5f0 0x7ff77c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.225+0000 7ff77a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff77c15e5f0 0x7ff77c0b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48360/0 (socket says 172.21.15.62:48360) 2024-04-25T09:09:13.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.225+0000 7ff77a800640 1 -- 172.21.15.62:0/3105569279 learned_addr learned my addr 172.21.15.62:0/3105569279 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.225+0000 7ff773e00640 1 --2- 172.21.15.62:0/3105569279 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c06d9d0 0x7ff77c15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.225+0000 7ff77a800640 1 -- 172.21.15.62:0/3105569279 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c06d9d0 msgr2=0x7ff77c15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.225+0000 7ff77a800640 1 --2- 172.21.15.62:0/3105569279 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c06d9d0 0x7ff77c15e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.225+0000 7ff77a800640 1 -- 172.21.15.62:0/3105569279 >> v1:172.21.15.121:6789/0 conn(0x7ff77c0ceb50 legacy=0x7ff77c06d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.225+0000 7ff77a800640 1 -- 172.21.15.62:0/3105569279 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0cef30 con 0x7ff77c15e5f0 2024-04-25T09:09:13.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.227+0000 7ff77a800640 1 --2- 172.21.15.62:0/3105569279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c0b4550 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff774009e50 tx=0x7ff77402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6069c81c7ffc18e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.227+0000 7ff779400640 1 -- 172.21.15.62:0/3105569279 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774036070 con 0x7ff77c15e5f0 2024-04-25T09:09:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.227+0000 7ff779400640 1 -- 172.21.15.62:0/3105569279 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774031600 con 0x7ff77c15e5f0 2024-04-25T09:09:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.227+0000 7ff779400640 1 -- 172.21.15.62:0/3105569279 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7740319d0 con 0x7ff77c15e5f0 2024-04-25T09:09:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.228+0000 7ff77b200640 1 -- 172.21.15.62:0/3105569279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 msgr2=0x7ff77c0b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.228+0000 7ff77b200640 1 --2- 172.21.15.62:0/3105569279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c0b4550 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff774009e50 tx=0x7ff77402a680 comp rx=0 tx=0).stop 2024-04-25T09:09:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.228+0000 7ff77b200640 1 -- 172.21.15.62:0/3105569279 shutdown_connections 2024-04-25T09:09:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.228+0000 7ff77b200640 1 --2- 172.21.15.62:0/3105569279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c0b4550 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.228+0000 7ff77b200640 1 --2- 172.21.15.62:0/3105569279 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c06d9d0 0x7ff77c15e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.228+0000 7ff77b200640 1 -- 172.21.15.62:0/3105569279 >> 172.21.15.62:0/3105569279 conn(0x7ff77c066f20 msgr2=0x7ff77c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.228+0000 7ff77b200640 1 -- 172.21.15.62:0/3105569279 shutdown_connections 2024-04-25T09:09:13.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.228+0000 7ff77b200640 1 -- 172.21.15.62:0/3105569279 wait complete. 2024-04-25T09:09:13.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.230+0000 7ff77b200640 1 Processor -- start 2024-04-25T09:09:13.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.230+0000 7ff77b200640 1 -- start start 2024-04-25T09:09:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff77c06d9d0 0x7ff77c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c0ceb50 0x7ff77c138610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c1366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c162510 con 0x7ff77c15e5f0 2024-04-25T09:09:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c162390 con 0x7ff77c06d9d0 2024-04-25T09:09:13.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0b4a90 con 0x7ff77c0ceb50 2024-04-25T09:09:13.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c1366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c1366c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48376/0 (socket says 172.21.15.62:48376) 2024-04-25T09:09:13.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.231+0000 7ff77a800640 1 -- 172.21.15.62:0/3512341655 learned_addr learned my addr 172.21.15.62:0/3512341655 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff779e00640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff77c06d9d0 0x7ff77c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff773e00640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c0ceb50 0x7ff77c138610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff77a800640 1 -- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c0ceb50 msgr2=0x7ff77c138610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff77a800640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c0ceb50 0x7ff77c138610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff77a800640 1 -- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff77c06d9d0 msgr2=0x7ff77c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff77a800640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff77c06d9d0 0x7ff77c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff77a800640 1 -- 172.21.15.62:0/3512341655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c135620 con 0x7ff77c15e5f0 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff77a800640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c1366c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff774009e50 tx=0x7ff774004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.232+0000 7ff772000640 1 -- 172.21.15.62:0/3512341655 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774036050 con 0x7ff77c15e5f0 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.233+0000 7ff772000640 1 -- 172.21.15.62:0/3512341655 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774042070 con 0x7ff77c15e5f0 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.233+0000 7ff772000640 1 -- 172.21.15.62:0/3512341655 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774009510 con 0x7ff77c15e5f0 2024-04-25T09:09:13.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.233+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c135900 con 0x7ff77c15e5f0 2024-04-25T09:09:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.233+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c135ec0 con 0x7ff77c15e5f0 2024-04-25T09:09:13.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.234+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c1613a0 con 0x7ff77c15e5f0 2024-04-25T09:09:13.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.235+0000 7ff772000640 1 -- 172.21.15.62:0/3512341655 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff774007bd0 con 0x7ff77c15e5f0 2024-04-25T09:09:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.236+0000 7ff772000640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff75806b4d0 0x7ff75806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.236+0000 7ff772000640 1 -- 172.21.15.62:0/3512341655 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff7740ab190 con 0x7ff77c15e5f0 2024-04-25T09:09:13.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.237+0000 7ff779e00640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff75806b4d0 0x7ff75806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.238+0000 7ff779e00640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff75806b4d0 0x7ff75806d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff76c007fd0 tx=0x7ff76c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.241+0000 7ff772000640 1 -- 172.21.15.62:0/3512341655 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff77c1613a0 con 0x7ff77c15e5f0 2024-04-25T09:09:13.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.436+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff77c0b47e0 con 0x7ff77c15e5f0 2024-04-25T09:09:13.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.437+0000 7ff772000640 1 -- 172.21.15.62:0/3512341655 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff774079ee0 con 0x7ff77c15e5f0 2024-04-25T09:09:13.425 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff75806b4d0 msgr2=0x7ff75806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff75806b4d0 0x7ff75806d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff76c007fd0 tx=0x7ff76c009040 comp rx=0 tx=0).stop 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 msgr2=0x7ff77c1366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c1366c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff774009e50 tx=0x7ff774004a40 comp rx=0 tx=0).stop 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 shutdown_connections 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff75806b4d0 0x7ff75806d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff77c15e5f0 0x7ff77c1366c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff77c0ceb50 0x7ff77c138610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 --2- 172.21.15.62:0/3512341655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff77c06d9d0 0x7ff77c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.439+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 >> 172.21.15.62:0/3512341655 conn(0x7ff77c066f20 msgr2=0x7ff77c071440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.440+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 shutdown_connections 2024-04-25T09:09:13.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.440+0000 7ff77b200640 1 -- 172.21.15.62:0/3512341655 wait complete. 2024-04-25T09:09:13.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:09:13.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.672+0000 7f06e9a00640 1 Processor -- start 2024-04-25T09:09:13.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.672+0000 7f06e9a00640 1 -- start start 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.674+0000 7f06e9a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f06e415e050 0x7f06e406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.674+0000 7f06e9a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06e406e330 0x7f06e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.674+0000 7f06e9a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e406d7f0 con 0x7f06e415dc70 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.674+0000 7f06e9a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e406d230 con 0x7f06e406e330 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.674+0000 7f06e9a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e406d510 con 0x7f06e415e050 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.675+0000 7f06e2a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f06e415dc70 0x7f06e406dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41228/0 (socket says 172.21.15.62:41228) 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.675+0000 7f06e2a00640 1 -- 172.21.15.62:0/999758532 learned_addr learned my addr 172.21.15.62:0/999758532 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.675+0000 7f06e3400640 1 --2- 172.21.15.62:0/999758532 >> v2:172.21.15.62:3300/0 conn(0x7f06e406e330 0x7f06e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.675+0000 7f06da000640 1 --2- 172.21.15.62:0/999758532 >> v2:172.21.15.121:3300/0 conn(0x7f06e415e050 0x7f06e406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.675+0000 7f06e2000640 1 -- 172.21.15.62:0/999758532 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530222449 0 0) 0x7f06e406d7f0 con 0x7f06e415dc70 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.675+0000 7f06e2000640 1 -- 172.21.15.62:0/999758532 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06c8003520 con 0x7f06e415dc70 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.676+0000 7f06e2000640 1 -- 172.21.15.62:0/999758532 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 727006360 0 0) 0x7f06c8003520 con 0x7f06e415dc70 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.676+0000 7f06e3400640 1 -- 172.21.15.62:0/999758532 >> v2:172.21.15.121:3300/0 conn(0x7f06e415e050 msgr2=0x7f06e406c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:13.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.676+0000 7f06e3400640 1 --2- 172.21.15.62:0/999758532 >> v2:172.21.15.121:3300/0 conn(0x7f06e415e050 0x7f06e406c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.676+0000 7f06e3400640 1 -- 172.21.15.62:0/999758532 >> v1:172.21.15.62:6789/0 conn(0x7f06e415dc70 legacy=0x7f06e406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.676+0000 7f06e3400640 1 -- 172.21.15.62:0/999758532 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e40b27b0 con 0x7f06e406e330 2024-04-25T09:09:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.677+0000 7f06e3400640 1 --2- 172.21.15.62:0/999758532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e406e330 0x7f06e406c8f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009d90 tx=0x7f06dc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26046bfd40b77a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.677+0000 7f06e2000640 1 -- 172.21.15.62:0/999758532 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4001670 con 0x7f06e406e330 2024-04-25T09:09:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.677+0000 7f06e2000640 1 -- 172.21.15.62:0/999758532 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc007540 con 0x7f06e406e330 2024-04-25T09:09:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.677+0000 7f06e2000640 1 -- 172.21.15.62:0/999758532 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0078f0 con 0x7f06e406e330 2024-04-25T09:09:13.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.678+0000 7f06e9a00640 1 -- 172.21.15.62:0/999758532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e406e330 msgr2=0x7f06e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.678+0000 7f06e9a00640 1 --2- 172.21.15.62:0/999758532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e406e330 0x7f06e406c8f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009d90 tx=0x7f06dc02a630 comp rx=0 tx=0).stop 2024-04-25T09:09:13.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.678+0000 7f06e9a00640 1 -- 172.21.15.62:0/999758532 shutdown_connections 2024-04-25T09:09:13.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.678+0000 7f06e9a00640 1 --2- 172.21.15.62:0/999758532 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e406e330 0x7f06e406c8f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.678+0000 7f06e9a00640 1 --2- 172.21.15.62:0/999758532 >> v2:172.21.15.121:3300/0 conn(0x7f06e415e050 0x7f06e406c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.678+0000 7f06e9a00640 1 -- 172.21.15.62:0/999758532 >> 172.21.15.62:0/999758532 conn(0x7f06e40b1410 msgr2=0x7f06e40b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.678+0000 7f06e9a00640 1 -- 172.21.15.62:0/999758532 shutdown_connections 2024-04-25T09:09:13.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.678+0000 7f06e9a00640 1 -- 172.21.15.62:0/999758532 wait complete. 2024-04-25T09:09:13.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.680+0000 7f06e9a00640 1 Processor -- start 2024-04-25T09:09:13.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.680+0000 7f06e9a00640 1 -- start start 2024-04-25T09:09:13.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.681+0000 7f06e9a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06e406e330 0x7f06e4155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.681+0000 7f06e9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e415dc70 0x7f06e40c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.681+0000 7f06e9a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06e415e050 0x7f06e4160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.681+0000 7f06e9a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e406cfb0 con 0x7f06e415dc70 2024-04-25T09:09:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.681+0000 7f06e9a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e406ce30 con 0x7f06e406e330 2024-04-25T09:09:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.681+0000 7f06e9a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40b2fb0 con 0x7f06e415e050 2024-04-25T09:09:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06da000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e415dc70 0x7f06e40c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06da000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e415dc70 0x7f06e40c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48392/0 (socket says 172.21.15.62:48392) 2024-04-25T09:09:13.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06da000640 1 -- 172.21.15.62:0/2433185347 learned_addr learned my addr 172.21.15.62:0/2433185347 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06e2a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06e406e330 0x7f06e4155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06e3400640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06e415e050 0x7f06e4160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06da000640 1 -- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06e415e050 msgr2=0x7f06e4160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06da000640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06e415e050 0x7f06e4160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06da000640 1 -- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06e406e330 msgr2=0x7f06e4155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06da000640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06e406e330 0x7f06e4155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.682+0000 7f06da000640 1 -- 172.21.15.62:0/2433185347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e41558a0 con 0x7f06e415dc70 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.683+0000 7f06da000640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e415dc70 0x7f06e40c4680 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00e8a0 tx=0x7f06cc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.683+0000 7f06db400640 1 -- 172.21.15.62:0/2433185347 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc017070 con 0x7f06e415dc70 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.683+0000 7f06db400640 1 -- 172.21.15.62:0/2433185347 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc00c940 con 0x7f06e415dc70 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.683+0000 7f06db400640 1 -- 172.21.15.62:0/2433185347 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00cd30 con 0x7f06e415dc70 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.683+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e415f5b0 con 0x7f06e415dc70 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.683+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e415fb70 con 0x7f06e415dc70 2024-04-25T09:09:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.684+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e404ef20 con 0x7f06e415dc70 2024-04-25T09:09:13.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.686+0000 7f06db400640 1 -- 172.21.15.62:0/2433185347 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06cc01a070 con 0x7f06e415dc70 2024-04-25T09:09:13.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.686+0000 7f06db400640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06c006b500 0x7f06c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:13.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.687+0000 7f06db400640 1 -- 172.21.15.62:0/2433185347 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f06cc08a4d0 con 0x7f06e415dc70 2024-04-25T09:09:13.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.687+0000 7f06e2a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06c006b500 0x7f06c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:13.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.688+0000 7f06e2a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06c006b500 0x7f06c006d9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f06d40011e0 tx=0x7f06d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:13.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.693+0000 7f06db400640 1 -- 172.21.15.62:0/2433185347 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06cc0592b0 con 0x7f06e415dc70 2024-04-25T09:09:13.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.903+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06e406b2c0 con 0x7f06e415dc70 2024-04-25T09:09:13.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.904+0000 7f06db400640 1 -- 172.21.15.62:0/2433185347 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06cc058bb0 con 0x7f06e415dc70 2024-04-25T09:09:13.892 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:09:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.907+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06c006b500 msgr2=0x7f06c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.907+0000 7f06e9a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06c006b500 0x7f06c006d9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f06d40011e0 tx=0x7f06d4009040 comp rx=0 tx=0).stop 2024-04-25T09:09:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.907+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e415dc70 msgr2=0x7f06e40c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.907+0000 7f06e9a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e415dc70 0x7f06e40c4680 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00e8a0 tx=0x7f06cc00ed60 comp rx=0 tx=0).stop 2024-04-25T09:09:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.907+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 shutdown_connections 2024-04-25T09:09:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.907+0000 7f06e9a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f06c006b500 0x7f06c006d9c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.908+0000 7f06e9a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06e415dc70 0x7f06e40c4680 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.908+0000 7f06e9a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f06e415e050 0x7f06e4160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.908+0000 7f06e9a00640 1 --2- 172.21.15.62:0/2433185347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06e406e330 0x7f06e4155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.908+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 >> 172.21.15.62:0/2433185347 conn(0x7f06e40b1410 msgr2=0x7f06e406c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.908+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 shutdown_connections 2024-04-25T09:09:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:13.908+0000 7f06e9a00640 1 -- 172.21.15.62:0/2433185347 wait complete. 2024-04-25T09:09:13.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:14.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.141+0000 7f0f93e00640 1 Processor -- start 2024-04-25T09:09:14.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.141+0000 7f0f93e00640 1 -- start start 2024-04-25T09:09:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.142+0000 7f0f93e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b56f0 0x7f0f94160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.142+0000 7f0f93e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0f940b6a90 0x7f0f94161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.142+0000 7f0f93e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0f940b27a0 0x7f0f94161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.142+0000 7f0f93e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f94161780 con 0x7f0f940b27a0 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.142+0000 7f0f93e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f940cedc0 con 0x7f0f940b6a90 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.142+0000 7f0f93e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940d3110 con 0x7f0f940b56f0 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f93400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0f940b27a0 0x7f0f94161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f93400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0f940b27a0 0x7f0f94161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48402/0 (socket says 172.21.15.62:48402) 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f93400640 1 -- 172.21.15.62:0/4229519539 learned_addr learned my addr 172.21.15.62:0/4229519539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f92a00640 1 --2- 172.21.15.62:0/4229519539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b56f0 0x7f0f94160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f92000640 1 --2- 172.21.15.62:0/4229519539 >> v2:172.21.15.121:3300/0 conn(0x7f0f940b6a90 0x7f0f94161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f93400640 1 -- 172.21.15.62:0/4229519539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b56f0 msgr2=0x7f0f94160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:14.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f93400640 1 --2- 172.21.15.62:0/4229519539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b56f0 0x7f0f94160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f93400640 1 -- 172.21.15.62:0/4229519539 >> v2:172.21.15.121:3300/0 conn(0x7f0f940b6a90 msgr2=0x7f0f94161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f93400640 1 --2- 172.21.15.62:0/4229519539 >> v2:172.21.15.121:3300/0 conn(0x7f0f940b6a90 0x7f0f94161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f93400640 1 -- 172.21.15.62:0/4229519539 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940d33f0 con 0x7f0f940b27a0 2024-04-25T09:09:14.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.143+0000 7f0f92000640 1 --2- 172.21.15.62:0/4229519539 >> v2:172.21.15.121:3300/0 conn(0x7f0f940b6a90 0x7f0f94161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.144+0000 7f0f93400640 1 --2- 172.21.15.62:0/4229519539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b27a0 0x7f0f94161980 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0f940728e0 tx=0x7f0f8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f0a43e879d2453 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f91600640 1 -- 172.21.15.62:0/4229519539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c036070 con 0x7f0f940b27a0 2024-04-25T09:09:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f91600640 1 -- 172.21.15.62:0/4229519539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c031600 con 0x7f0f940b27a0 2024-04-25T09:09:14.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f91600640 1 -- 172.21.15.62:0/4229519539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0319d0 con 0x7f0f940b27a0 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f93e00640 1 -- 172.21.15.62:0/4229519539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b27a0 msgr2=0x7f0f94161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f93e00640 1 --2- 172.21.15.62:0/4229519539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b27a0 0x7f0f94161980 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0f940728e0 tx=0x7f0f8c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f93e00640 1 -- 172.21.15.62:0/4229519539 shutdown_connections 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f93e00640 1 --2- 172.21.15.62:0/4229519539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b27a0 0x7f0f94161980 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f93e00640 1 --2- 172.21.15.62:0/4229519539 >> v2:172.21.15.121:3300/0 conn(0x7f0f940b6a90 0x7f0f94161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f93e00640 1 --2- 172.21.15.62:0/4229519539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b56f0 0x7f0f94160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f93e00640 1 -- 172.21.15.62:0/4229519539 >> 172.21.15.62:0/4229519539 conn(0x7f0f94066d10 msgr2=0x7f0f94067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.145+0000 7f0f93e00640 1 -- 172.21.15.62:0/4229519539 shutdown_connections 2024-04-25T09:09:14.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.146+0000 7f0f93e00640 1 -- 172.21.15.62:0/4229519539 wait complete. 2024-04-25T09:09:14.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.147+0000 7f0f93e00640 1 Processor -- start 2024-04-25T09:09:14.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.147+0000 7f0f93e00640 1 -- start start 2024-04-25T09:09:14.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.148+0000 7f0f93e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b27a0 0x7f0f940b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.148+0000 7f0f93e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f940b56f0 0x7f0f941e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.148+0000 7f0f93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b6a90 0x7f0f941e9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.148+0000 7f0f93e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940b2d60 con 0x7f0f940b6a90 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.148+0000 7f0f93e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940cea50 con 0x7f0f940b56f0 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.148+0000 7f0f93e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940b2ee0 con 0x7f0f940b27a0 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f93400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b6a90 0x7f0f941e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f93400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b6a90 0x7f0f941e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48414/0 (socket says 172.21.15.62:48414) 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f93400640 1 -- 172.21.15.62:0/3617704352 learned_addr learned my addr 172.21.15.62:0/3617704352 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f92000640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f940b56f0 0x7f0f941e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f92a00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b27a0 0x7f0f940b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f93400640 1 -- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b27a0 msgr2=0x7f0f940b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f93400640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b27a0 0x7f0f940b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f93400640 1 -- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f940b56f0 msgr2=0x7f0f941e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f93400640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f940b56f0 0x7f0f941e91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f93400640 1 -- 172.21.15.62:0/3617704352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f941e7de0 con 0x7f0f940b6a90 2024-04-25T09:09:14.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.149+0000 7f0f92a00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b27a0 0x7f0f940b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.150+0000 7f0f93400640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b6a90 0x7f0f941e9710 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009f40 tx=0x7f0f8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.150+0000 7f0f8aa00640 1 -- 172.21.15.62:0/3617704352 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c036050 con 0x7f0f940b6a90 2024-04-25T09:09:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.150+0000 7f0f8aa00640 1 -- 172.21.15.62:0/3617704352 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c042070 con 0x7f0f940b6a90 2024-04-25T09:09:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.150+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f941e80c0 con 0x7f0f940b6a90 2024-04-25T09:09:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.150+0000 7f0f8aa00640 1 -- 172.21.15.62:0/3617704352 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c009510 con 0x7f0f940b6a90 2024-04-25T09:09:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.150+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941e8680 con 0x7f0f940b6a90 2024-04-25T09:09:14.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.151+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9404ef20 con 0x7f0f940b6a90 2024-04-25T09:09:14.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.152+0000 7f0f8aa00640 1 -- 172.21.15.62:0/3617704352 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f8c007bd0 con 0x7f0f940b6a90 2024-04-25T09:09:14.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.153+0000 7f0f8aa00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f7406b500 0x7f0f7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.153+0000 7f0f8aa00640 1 -- 172.21.15.62:0/3617704352 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0f8c0ab230 con 0x7f0f940b6a90 2024-04-25T09:09:14.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.154+0000 7f0f92a00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f7406b500 0x7f0f7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.155+0000 7f0f92a00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f7406b500 0x7f0f7406d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f84006fd0 tx=0x7f0f84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.158+0000 7f0f8aa00640 1 -- 172.21.15.62:0/3617704352 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f8c045360 con 0x7f0f940b6a90 2024-04-25T09:09:14.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.396+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f940b6180 con 0x7f0f940b6a90 2024-04-25T09:09:14.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.397+0000 7f0f8aa00640 1 -- 172.21.15.62:0/3617704352 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f0f8c07a010 con 0x7f0f940b6a90 2024-04-25T09:09:14.385 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:14.385 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1396005512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1396005512},{"type":"v1","addr":"172.21.15.121:6835","nonce":1396005512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3589939339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3589939339},{"type":"v1","addr":"172.21.15.121:6837","nonce":3589939339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2551168226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2551168226},{"type":"v1","addr":"172.21.15.62:6835","nonce":2551168226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:53.657317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6833/836333339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":836333339},{"type":"v1","addr":"172.21.15.62:6833","nonce":836333339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:09:14.386 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 45 2024-04-25T09:09:14.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.399+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f7406b500 msgr2=0x7f0f7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.399+0000 7f0f93e00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f7406b500 0x7f0f7406d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0f84006fd0 tx=0x7f0f84008040 comp rx=0 tx=0).stop 2024-04-25T09:09:14.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b6a90 msgr2=0x7f0f941e9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b6a90 0x7f0f941e9710 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009f40 tx=0x7f0f8c004a40 comp rx=0 tx=0).stop 2024-04-25T09:09:14.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 shutdown_connections 2024-04-25T09:09:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f7406b500 0x7f0f7406d9c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f940b6a90 0x7f0f941e9710 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f940b56f0 0x7f0f941e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 --2- 172.21.15.62:0/3617704352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0f940b27a0 0x7f0f940b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 >> 172.21.15.62:0/3617704352 conn(0x7f0f94066d10 msgr2=0x7f0f940b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 shutdown_connections 2024-04-25T09:09:14.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.400+0000 7f0f93e00640 1 -- 172.21.15.62:0/3617704352 wait complete. 2024-04-25T09:09:14.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:14.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.622+0000 7f0360c00640 1 Processor -- start 2024-04-25T09:09:14.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.623+0000 7f0360c00640 1 -- start start 2024-04-25T09:09:14.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.624+0000 7f0360c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f035c0b6a90 0x7f035c0d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.624+0000 7f0360c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c0d3110 con 0x7f035c0b56f0 2024-04-25T09:09:14.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.624+0000 7f0360c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c0d33f0 con 0x7f035c0b27a0 2024-04-25T09:09:14.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.624+0000 7f0360c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035c0cedc0 con 0x7f035c0b6a90 2024-04-25T09:09:14.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.624+0000 7f0352a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f035c0b6a90 0x7f035c0d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.625+0000 7f0352a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f035c0b6a90 0x7f035c0d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48428/0 (socket says 172.21.15.62:48428) 2024-04-25T09:09:14.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.625+0000 7f0352a00640 1 -- 172.21.15.62:0/2992698767 learned_addr learned my addr 172.21.15.62:0/2992698767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:14.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.625+0000 7f035b400640 1 --1- 172.21.15.62:0/2992698767 >> v1:172.21.15.62:6789/0 conn(0x7f035c0b56f0 0x7f035c160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:41238/0 (socket says 172.21.15.62:41238) 2024-04-25T09:09:14.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.625+0000 7f0352a00640 1 -- 172.21.15.62:0/2992698767 >> v1:172.21.15.121:6789/0 conn(0x7f035c0b27a0 legacy=0x7f035c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.625+0000 7f0352a00640 1 -- 172.21.15.62:0/2992698767 >> v1:172.21.15.62:6789/0 conn(0x7f035c0b56f0 legacy=0x7f035c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.625+0000 7f0352a00640 1 -- 172.21.15.62:0/2992698767 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0d36d0 con 0x7f035c0b6a90 2024-04-25T09:09:14.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.625+0000 7f035aa00640 1 -- 172.21.15.62:0/2992698767 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780163230 0 0) 0x7f035c0d3110 con 0x7f035c0b56f0 2024-04-25T09:09:14.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.626+0000 7f0352a00640 1 --2- 172.21.15.62:0/2992698767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b6a90 0x7f035c0d1fe0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f0344009870 tx=0x7f034402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c41b29c2981c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f035aa00640 1 -- 172.21.15.62:0/2992698767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344037070 con 0x7f035c0b6a90 2024-04-25T09:09:14.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f035aa00640 1 -- 172.21.15.62:0/2992698767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034402ae50 con 0x7f035c0b6a90 2024-04-25T09:09:14.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f035aa00640 1 -- 172.21.15.62:0/2992698767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344032860 con 0x7f035c0b6a90 2024-04-25T09:09:14.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f0360c00640 1 -- 172.21.15.62:0/2992698767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b6a90 msgr2=0x7f035c0d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f0360c00640 1 --2- 172.21.15.62:0/2992698767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b6a90 0x7f035c0d1fe0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f0344009870 tx=0x7f034402a300 comp rx=0 tx=0).stop 2024-04-25T09:09:14.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f0360c00640 1 -- 172.21.15.62:0/2992698767 shutdown_connections 2024-04-25T09:09:14.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f0360c00640 1 --2- 172.21.15.62:0/2992698767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b6a90 0x7f035c0d1fe0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f0360c00640 1 -- 172.21.15.62:0/2992698767 >> 172.21.15.62:0/2992698767 conn(0x7f035c066d10 msgr2=0x7f035c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f0360c00640 1 -- 172.21.15.62:0/2992698767 shutdown_connections 2024-04-25T09:09:14.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.627+0000 7f0360c00640 1 -- 172.21.15.62:0/2992698767 wait complete. 2024-04-25T09:09:14.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.629+0000 7f0360c00640 1 Processor -- start 2024-04-25T09:09:14.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.629+0000 7f0360c00640 1 -- start start 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.630+0000 7f0360c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f035c0b27a0 0x7f035c14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.630+0000 7f0360c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b56f0 0x7f035c0b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.630+0000 7f0360c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f035c0b6a90 0x7f035c159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.630+0000 7f0360c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0b2ee0 con 0x7f035c0b56f0 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.630+0000 7f0360c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0b2d60 con 0x7f035c0b6a90 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.630+0000 7f0360c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0d2c30 con 0x7f035c0b27a0 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f0352a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b56f0 0x7f035c0b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f0352a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b56f0 0x7f035c0b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48440/0 (socket says 172.21.15.62:48440) 2024-04-25T09:09:14.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f0352a00640 1 -- 172.21.15.62:0/1632205966 learned_addr learned my addr 172.21.15.62:0/1632205966 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:14.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f0352a00640 1 -- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f035c0b27a0 msgr2=0x7f035c14bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f0352a00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f035c0b27a0 0x7f035c14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f0352a00640 1 -- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f035c0b6a90 msgr2=0x7f035c159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f035be00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f035c0b6a90 0x7f035c159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f0352a00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f035c0b6a90 0x7f035c159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f0352a00640 1 -- 172.21.15.62:0/1632205966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c158ea0 con 0x7f035c0b56f0 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.631+0000 7f035be00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f035c0b6a90 0x7f035c159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.632+0000 7f0352a00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b56f0 0x7f035c0b82b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f03440090b0 tx=0x7f03440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.632+0000 7f0353e00640 1 -- 172.21.15.62:0/1632205966 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344041070 con 0x7f035c0b56f0 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.632+0000 7f0353e00640 1 -- 172.21.15.62:0/1632205966 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034403c890 con 0x7f035c0b56f0 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.632+0000 7f0353e00640 1 -- 172.21.15.62:0/1632205966 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344037040 con 0x7f035c0b56f0 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.632+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c159180 con 0x7f035c0b56f0 2024-04-25T09:09:14.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.632+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c159740 con 0x7f035c0b56f0 2024-04-25T09:09:14.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.633+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c1601d0 con 0x7f035c0b56f0 2024-04-25T09:09:14.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.634+0000 7f0353e00640 1 -- 172.21.15.62:0/1632205966 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0344044070 con 0x7f035c0b56f0 2024-04-25T09:09:14.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.635+0000 7f0353e00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f033806b4d0 0x7f033806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:14.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.635+0000 7f0353e00640 1 -- 172.21.15.62:0/1632205966 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f03440aae10 con 0x7f035c0b56f0 2024-04-25T09:09:14.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.636+0000 7f035b400640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f033806b4d0 0x7f033806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.637+0000 7f035b400640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f033806b4d0 0x7f033806d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f034c007fd0 tx=0x7f034c007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:14.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.640+0000 7f0353e00640 1 -- 172.21.15.62:0/1632205966 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f035c1601d0 con 0x7f035c0b56f0 2024-04-25T09:09:14.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.841+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f035c0d2a10 con 0x7f035c0b56f0 2024-04-25T09:09:14.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.844+0000 7f0353e00640 1 -- 172.21.15.62:0/1632205966 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f0344079d20 con 0x7f035c0b56f0 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:14.833 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":68,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:01.691769+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:34.817879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:35.997553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":6,"to":1}]},{"pgid":"9.37","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/3491368038":"2024-04-26T09:08:50.613426+0000","172.21.15.62:6832/3491368038":"2024-04-26T09:08:50.613426+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:14.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f033806b4d0 msgr2=0x7f033806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f033806b4d0 0x7f033806d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f034c007fd0 tx=0x7f034c007900 comp rx=0 tx=0).stop 2024-04-25T09:09:14.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b56f0 msgr2=0x7f035c0b82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b56f0 0x7f035c0b82b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f03440090b0 tx=0x7f03440046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 shutdown_connections 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f033806b4d0 0x7f033806d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f035c0b56f0 0x7f035c0b82b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f035c0b6a90 0x7f035c159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 --2- 172.21.15.62:0/1632205966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f035c0b27a0 0x7f035c14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 >> 172.21.15.62:0/1632205966 conn(0x7f035c066d10 msgr2=0x7f035c0d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 shutdown_connections 2024-04-25T09:09:14.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:14.847+0000 7f0360c00640 1 -- 172.21.15.62:0/1632205966 wait complete. 2024-04-25T09:09:14.853 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:09:14.854 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:15.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.068+0000 7ff2a3800640 1 Processor -- start 2024-04-25T09:09:15.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.069+0000 7ff2a3800640 1 -- start start 2024-04-25T09:09:15.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.070+0000 7ff2a3800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff29c15de70 0x7ff29c0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.070+0000 7ff2a3800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff29c0b2d40 0x7ff29c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.070+0000 7ff2a3800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c0b7780 con 0x7ff29c0b9900 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.070+0000 7ff2a3800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c0b8320 con 0x7ff29c0b2d40 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.070+0000 7ff2a3800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c0b79f0 con 0x7ff29c15de70 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.070+0000 7ff2a1800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff29c0b2d40 0x7ff29c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.070+0000 7ff2a1800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff29c0b2d40 0x7ff29c0d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51082/0 (socket says 172.21.15.62:51082) 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.070+0000 7ff2a1800640 1 -- 172.21.15.62:0/1159933324 learned_addr learned my addr 172.21.15.62:0/1159933324 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.071+0000 7ff2a1800640 1 -- 172.21.15.62:0/1159933324 >> v2:172.21.15.121:3300/0 conn(0x7ff29c15de70 msgr2=0x7ff29c0b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.071+0000 7ff2a1800640 1 --2- 172.21.15.62:0/1159933324 >> v2:172.21.15.121:3300/0 conn(0x7ff29c15de70 0x7ff29c0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.071+0000 7ff2a1800640 1 -- 172.21.15.62:0/1159933324 >> v1:172.21.15.121:6789/0 conn(0x7ff29c0b9900 legacy=0x7ff29c0b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:15.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.071+0000 7ff2a1800640 1 -- 172.21.15.62:0/1159933324 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c0b7cd0 con 0x7ff29c0b2d40 2024-04-25T09:09:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.072+0000 7ff2a1800640 1 --2- 172.21.15.62:0/1159933324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c0b2d40 0x7ff29c0d1e80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0b66f0 tx=0x7ff29402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683d3bccf74ee36d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.072+0000 7ff29b400640 1 -- 172.21.15.62:0/1159933324 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294036070 con 0x7ff29c0b2d40 2024-04-25T09:09:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.072+0000 7ff29b400640 1 -- 172.21.15.62:0/1159933324 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff294031600 con 0x7ff29c0b2d40 2024-04-25T09:09:15.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.072+0000 7ff29b400640 1 -- 172.21.15.62:0/1159933324 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940319d0 con 0x7ff29c0b2d40 2024-04-25T09:09:15.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.072+0000 7ff2a3800640 1 -- 172.21.15.62:0/1159933324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c0b2d40 msgr2=0x7ff29c0d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.072+0000 7ff2a3800640 1 --2- 172.21.15.62:0/1159933324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c0b2d40 0x7ff29c0d1e80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0b66f0 tx=0x7ff29402a780 comp rx=0 tx=0).stop 2024-04-25T09:09:15.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.073+0000 7ff2a3800640 1 -- 172.21.15.62:0/1159933324 shutdown_connections 2024-04-25T09:09:15.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.073+0000 7ff2a3800640 1 --2- 172.21.15.62:0/1159933324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c0b2d40 0x7ff29c0d1e80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.073+0000 7ff2a3800640 1 --2- 172.21.15.62:0/1159933324 >> v2:172.21.15.121:3300/0 conn(0x7ff29c15de70 0x7ff29c0b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.073+0000 7ff2a3800640 1 -- 172.21.15.62:0/1159933324 >> 172.21.15.62:0/1159933324 conn(0x7ff29c06b000 msgr2=0x7ff29c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.073+0000 7ff2a3800640 1 -- 172.21.15.62:0/1159933324 shutdown_connections 2024-04-25T09:09:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.073+0000 7ff2a3800640 1 -- 172.21.15.62:0/1159933324 wait complete. 2024-04-25T09:09:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.074+0000 7ff2a3800640 1 Processor -- start 2024-04-25T09:09:15.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.075+0000 7ff2a3800640 1 -- start start 2024-04-25T09:09:15.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.075+0000 7ff2a3800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff29c0b2d40 0x7ff29c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.075+0000 7ff2a3800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff29c0b9900 0x7ff29c1f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.075+0000 7ff2a3800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c15de70 0x7ff29c0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a3800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0d23c0 con 0x7ff29c15de70 2024-04-25T09:09:15.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a3800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0b8ec0 con 0x7ff29c0b9900 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a3800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0d2540 con 0x7ff29c0b2d40 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c15de70 0x7ff29c0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c15de70 0x7ff29c0cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51092/0 (socket says 172.21.15.62:51092) 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a1800640 1 -- 172.21.15.62:0/537237157 learned_addr learned my addr 172.21.15.62:0/537237157 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff29be00640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff29c0b9900 0x7ff29c1f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a0e00640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff29c0b2d40 0x7ff29c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a1800640 1 -- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff29c0b2d40 msgr2=0x7ff29c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a1800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff29c0b2d40 0x7ff29c15fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a1800640 1 -- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff29c0b9900 msgr2=0x7ff29c1f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a1800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff29c0b9900 0x7ff29c1f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a1800640 1 -- 172.21.15.62:0/537237157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c0cd9d0 con 0x7ff29c15de70 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.076+0000 7ff2a0e00640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff29c0b2d40 0x7ff29c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:15.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.077+0000 7ff2a1800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c15de70 0x7ff29c0cd1a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff294034040 tx=0x7ff294004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.077+0000 7ff298c00640 1 -- 172.21.15.62:0/537237157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294041020 con 0x7ff29c15de70 2024-04-25T09:09:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.077+0000 7ff298c00640 1 -- 172.21.15.62:0/537237157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2940077a0 con 0x7ff29c15de70 2024-04-25T09:09:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.077+0000 7ff298c00640 1 -- 172.21.15.62:0/537237157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294036040 con 0x7ff29c15de70 2024-04-25T09:09:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.077+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c0cbed0 con 0x7ff29c15de70 2024-04-25T09:09:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.077+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c0cd6e0 con 0x7ff29c15de70 2024-04-25T09:09:15.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.078+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29c0729a0 con 0x7ff29c15de70 2024-04-25T09:09:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.079+0000 7ff298c00640 1 -- 172.21.15.62:0/537237157 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff294044020 con 0x7ff29c15de70 2024-04-25T09:09:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.081+0000 7ff298c00640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff27806b4d0 0x7ff27806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.081+0000 7ff298c00640 1 -- 172.21.15.62:0/537237157 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff2940ab160 con 0x7ff29c15de70 2024-04-25T09:09:15.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.081+0000 7ff2a0e00640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff27806b4d0 0x7ff27806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.082+0000 7ff2a0e00640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff27806b4d0 0x7ff27806d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff29c1f7840 tx=0x7ff28c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:15.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.085+0000 7ff298c00640 1 -- 172.21.15.62:0/537237157 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff29404b030 con 0x7ff29c15de70 2024-04-25T09:09:15.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.315+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff29c0c72a0 con 0x7ff29c15de70 2024-04-25T09:09:15.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.316+0000 7ff298c00640 1 -- 172.21.15.62:0/537237157 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4942 (secure 0 0 0) 0x7ff294079f40 con 0x7ff29c15de70 2024-04-25T09:09:15.304 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:15.305 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1396005512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1396005512},{"type":"v1","addr":"172.21.15.121:6835","nonce":1396005512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3589939339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3589939339},{"type":"v1","addr":"172.21.15.121:6837","nonce":3589939339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2551168226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2551168226},{"type":"v1","addr":"172.21.15.62:6835","nonce":2551168226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:53.657317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6833/836333339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":836333339},{"type":"v1","addr":"172.21.15.62:6833","nonce":836333339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:09:15.305 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 45 2024-04-25T09:09:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff27806b4d0 msgr2=0x7ff27806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff27806b4d0 0x7ff27806d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff29c1f7840 tx=0x7ff28c008040 comp rx=0 tx=0).stop 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c15de70 msgr2=0x7ff29c0cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c15de70 0x7ff29c0cd1a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff294034040 tx=0x7ff294004a40 comp rx=0 tx=0).stop 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 shutdown_connections 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff27806b4d0 0x7ff27806d990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff29c15de70 0x7ff29c0cd1a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff29c0b9900 0x7ff29c1f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 --2- 172.21.15.62:0/537237157 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff29c0b2d40 0x7ff29c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 >> 172.21.15.62:0/537237157 conn(0x7ff29c06b000 msgr2=0x7ff29c0c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.319+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 shutdown_connections 2024-04-25T09:09:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.320+0000 7ff2a3800640 1 -- 172.21.15.62:0/537237157 wait complete. 2024-04-25T09:09:15.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:15.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.537+0000 7f4861200640 1 Processor -- start 2024-04-25T09:09:15.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.537+0000 7f4861200640 1 -- start start 2024-04-25T09:09:15.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.538+0000 7f4861200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f485c0b27a0 0x7f485c0d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.538+0000 7f4861200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c0d3110 con 0x7f485c0b56f0 2024-04-25T09:09:15.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.539+0000 7f4861200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c0d33f0 con 0x7f485c0b6a90 2024-04-25T09:09:15.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.539+0000 7f4861200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c0cedc0 con 0x7f485c0b27a0 2024-04-25T09:09:15.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.539+0000 7f485b400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f485c0b56f0 0x7f485c160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44462/0 (socket says 172.21.15.62:44462) 2024-04-25T09:09:15.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.539+0000 7f485b400640 1 -- 172.21.15.62:0/2774591969 learned_addr learned my addr 172.21.15.62:0/2774591969 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.539+0000 7f485be00640 1 --2- 172.21.15.62:0/2774591969 >> v2:172.21.15.121:3300/0 conn(0x7f485c0b27a0 0x7f485c0d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.539+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101793438 0 0) 0x7f485c0d3110 con 0x7f485c0b56f0 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.539+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4840003520 con 0x7f485c0b56f0 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 379406043 0 0) 0x7f4840003520 con 0x7f485c0b56f0 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 >> v2:172.21.15.121:3300/0 conn(0x7f485c0b27a0 msgr2=0x7f485c0d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 --2- 172.21.15.62:0/2774591969 >> v2:172.21.15.121:3300/0 conn(0x7f485c0b27a0 0x7f485c0d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 >> v1:172.21.15.121:6789/0 conn(0x7f485c0b6a90 legacy=0x7f485c0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c0d36d0 con 0x7f485c0b56f0 2024-04-25T09:09:15.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f484c001670 con 0x7f485c0b56f0 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 >> v1:172.21.15.62:6789/0 conn(0x7f485c0b56f0 legacy=0x7f485c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4840003a00 0x7f4840003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f4840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48400051c0 0x7f484000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0cedc0 con 0x7f4840004800 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484000c0c0 con 0x7f48400051c0 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484000c2c0 con 0x7f4840003a00 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.540+0000 7f485aa00640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f4840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.541+0000 7f485b400640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4840003a00 0x7f4840003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.541+0000 7f485be00640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48400051c0 0x7f484000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.541+0000 7f485aa00640 1 -- 172.21.15.62:0/2774591969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4840003a00 msgr2=0x7f4840003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.541+0000 7f485aa00640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4840003a00 0x7f4840003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.541+0000 7f485aa00640 1 -- 172.21.15.62:0/2774591969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48400051c0 msgr2=0x7f484000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:15.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.541+0000 7f485aa00640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48400051c0 0x7f484000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.541+0000 7f485aa00640 1 -- 172.21.15.62:0/2774591969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4840003520 con 0x7f4840004800 2024-04-25T09:09:15.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.541+0000 7f485be00640 1 -- 172.21.15.62:0/2774591969 reap_dead start 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.542+0000 7f485aa00640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f4840004c80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4848001720 tx=0x7f4840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1663a61177f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.542+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848001370 con 0x7f4840004800 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.542+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48480052d0 con 0x7f4840004800 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.542+0000 7f485a000640 1 -- 172.21.15.62:0/2774591969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484802ab90 con 0x7f4840004800 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.543+0000 7f4861200640 1 -- 172.21.15.62:0/2774591969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 msgr2=0x7f4840004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.543+0000 7f4861200640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f4840004c80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4848001720 tx=0x7f4840005830 comp rx=0 tx=0).stop 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.543+0000 7f4861200640 1 -- 172.21.15.62:0/2774591969 shutdown_connections 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.543+0000 7f4861200640 1 --2- 172.21.15.62:0/2774591969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f4840004c80 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.543+0000 7f4861200640 1 -- 172.21.15.62:0/2774591969 >> 172.21.15.62:0/2774591969 conn(0x7f485c066d10 msgr2=0x7f485c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.543+0000 7f4861200640 1 -- 172.21.15.62:0/2774591969 shutdown_connections 2024-04-25T09:09:15.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.543+0000 7f4861200640 1 -- 172.21.15.62:0/2774591969 wait complete. 2024-04-25T09:09:15.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.545+0000 7f4861200640 1 Processor -- start 2024-04-25T09:09:15.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.545+0000 7f4861200640 1 -- start start 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f4861200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f485c1f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f4861200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f485c159e10 0x7f485c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f4861200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f485c15c8a0 0x7f485c15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f4861200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0b2ee0 con 0x7f4840004800 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f4861200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0b2d60 con 0x7f485c159e10 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f4861200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0b5ad0 con 0x7f485c15c8a0 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f485b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f485c1f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f485b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f485c1f3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51098/0 (socket says 172.21.15.62:51098) 2024-04-25T09:09:15.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f485b400640 1 -- 172.21.15.62:0/879784219 learned_addr learned my addr 172.21.15.62:0/879784219 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f485aa00640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f485c159e10 0x7f485c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.546+0000 7f485be00640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f485c15c8a0 0x7f485c15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f485b400640 1 -- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f485c15c8a0 msgr2=0x7f485c15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f485b400640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f485c15c8a0 0x7f485c15cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f485b400640 1 -- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f485c159e10 msgr2=0x7f485c15a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f485b400640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f485c159e10 0x7f485c15a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f485b400640 1 -- 172.21.15.62:0/879784219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c1f3fc0 con 0x7f4840004800 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f485b400640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f485c1f3a80 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f484c001bd0 tx=0x7f484c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f4853400640 1 -- 172.21.15.62:0/879784219 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00c660 con 0x7f4840004800 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f4853400640 1 -- 172.21.15.62:0/879784219 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c018070 con 0x7f4840004800 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f4853400640 1 -- 172.21.15.62:0/879784219 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00ca60 con 0x7f4840004800 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c15d5d0 con 0x7f4840004800 2024-04-25T09:09:15.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.547+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c15db90 con 0x7f4840004800 2024-04-25T09:09:15.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.548+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c04ef20 con 0x7f4840004800 2024-04-25T09:09:15.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.549+0000 7f4853400640 1 -- 172.21.15.62:0/879784219 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f484c00cc00 con 0x7f4840004800 2024-04-25T09:09:15.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.551+0000 7f4853400640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f483c06b500 0x7f483c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:15.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.551+0000 7f4853400640 1 -- 172.21.15.62:0/879784219 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f484c089f40 con 0x7f4840004800 2024-04-25T09:09:15.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.551+0000 7f485aa00640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f483c06b500 0x7f483c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:15.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.553+0000 7f485aa00640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f483c06b500 0x7f483c06d9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4848032000 tx=0x7f484802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:15.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.555+0000 7f4853400640 1 -- 172.21.15.62:0/879784219 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f484c058d20 con 0x7f4840004800 2024-04-25T09:09:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.794+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f485c0b6180 con 0x7f4840004800 2024-04-25T09:09:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.795+0000 7f4853400640 1 -- 172.21.15.62:0/879784219 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f484c058620 con 0x7f4840004800 2024-04-25T09:09:15.789 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:15.789 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":45,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1396005512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1396005512},{"type":"v1","addr":"172.21.15.121:6835","nonce":1396005512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3589939339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3589939339},{"type":"v1","addr":"172.21.15.121:6837","nonce":3589939339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2551168226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2551168226},{"type":"v1","addr":"172.21.15.62:6835","nonce":2551168226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:08:36.958423+0000","modified":"2024-04-25T09:08:53.657317+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6633},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6633":{"gid":6633,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.62:6833/836333339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":836333339},{"type":"v1","addr":"172.21.15.62:6833","nonce":836333339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 45 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.798+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f483c06b500 msgr2=0x7f483c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.798+0000 7f4861200640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f483c06b500 0x7f483c06d9c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4848032000 tx=0x7f484802e000 comp rx=0 tx=0).stop 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.798+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 msgr2=0x7f485c1f3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.798+0000 7f4861200640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f485c1f3a80 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f484c001bd0 tx=0x7f484c00ec10 comp rx=0 tx=0).stop 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.798+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 shutdown_connections 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.799+0000 7f4861200640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f483c06b500 0x7f483c06d9c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.799+0000 7f4861200640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f485c15c8a0 0x7f485c15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.799+0000 7f4861200640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f485c159e10 0x7f485c15a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.799+0000 7f4861200640 1 --2- 172.21.15.62:0/879784219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4840004800 0x7f485c1f3a80 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.799+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 >> 172.21.15.62:0/879784219 conn(0x7f485c066d10 msgr2=0x7f485c15e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.799+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 shutdown_connections 2024-04-25T09:09:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:15.799+0000 7f4861200640 1 -- 172.21.15.62:0/879784219 wait complete. 2024-04-25T09:09:15.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:16.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.020+0000 7f4b07e00640 1 Processor -- start 2024-04-25T09:09:16.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.021+0000 7f4b07e00640 1 -- start start 2024-04-25T09:09:16.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.022+0000 7f4b07e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4b080b0750 0x7f4b080d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.022+0000 7f4b07e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b080b4b40 con 0x7f4b080b0c30 2024-04-25T09:09:16.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.022+0000 7f4b07e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0806c750 con 0x7f4b0815dba0 2024-04-25T09:09:16.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.022+0000 7f4b07e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0806fb70 con 0x7f4b080b0750 2024-04-25T09:09:16.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.022+0000 7f4b06000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4b080b0750 0x7f4b080d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.022+0000 7f4b07400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4b080b0c30 0x7f4b080702e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44478/0 (socket says 172.21.15.62:44478) 2024-04-25T09:09:16.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.023+0000 7f4b07400640 1 -- 172.21.15.62:0/3334951090 learned_addr learned my addr 172.21.15.62:0/3334951090 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:16.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.023+0000 7f4b06000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4b080b0750 0x7f4b080d1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51106/0 (socket says 172.21.15.62:51106) 2024-04-25T09:09:16.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.023+0000 7f4b06000640 1 -- 172.21.15.62:0/3334951090 >> v1:172.21.15.121:6789/0 conn(0x7f4b0815dba0 legacy=0x7f4b0806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:16.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.023+0000 7f4b06000640 1 -- 172.21.15.62:0/3334951090 >> v1:172.21.15.62:6789/0 conn(0x7f4b080b0c30 legacy=0x7f4b080702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.023+0000 7f4b06000640 1 -- 172.21.15.62:0/3334951090 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0806ca30 con 0x7f4b080b0750 2024-04-25T09:09:16.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.024+0000 7f4b06000640 1 --2- 172.21.15.62:0/3334951090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0750 0x7f4b080d1690 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097a0 tx=0x7f4af002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ab115583542741 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.024+0000 7f4b05600640 1 -- 172.21.15.62:0/3334951090 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0037070 con 0x7f4b080b0750 2024-04-25T09:09:16.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.024+0000 7f4b05600640 1 -- 172.21.15.62:0/3334951090 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af002adc0 con 0x7f4b080b0750 2024-04-25T09:09:16.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.024+0000 7f4b05600640 1 -- 172.21.15.62:0/3334951090 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0032660 con 0x7f4b080b0750 2024-04-25T09:09:16.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.025+0000 7f4b07e00640 1 -- 172.21.15.62:0/3334951090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0750 msgr2=0x7f4b080d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.025+0000 7f4b07e00640 1 --2- 172.21.15.62:0/3334951090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0750 0x7f4b080d1690 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097a0 tx=0x7f4af002a0b0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.025+0000 7f4b07e00640 1 -- 172.21.15.62:0/3334951090 shutdown_connections 2024-04-25T09:09:16.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.025+0000 7f4b07e00640 1 --2- 172.21.15.62:0/3334951090 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0750 0x7f4b080d1690 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.025+0000 7f4b07e00640 1 -- 172.21.15.62:0/3334951090 >> 172.21.15.62:0/3334951090 conn(0x7f4b080b3480 msgr2=0x7f4b080b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:16.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.025+0000 7f4b07e00640 1 -- 172.21.15.62:0/3334951090 shutdown_connections 2024-04-25T09:09:16.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.025+0000 7f4b07e00640 1 -- 172.21.15.62:0/3334951090 wait complete. 2024-04-25T09:09:16.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.027+0000 7f4b07e00640 1 Processor -- start 2024-04-25T09:09:16.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.027+0000 7f4b07e00640 1 -- start start 2024-04-25T09:09:16.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b07e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b080b0750 0x7f4b08153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b07e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0c30 0x7f4b0815cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b07e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b0815dba0 0x7f4b08158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b07e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0815df80 con 0x7f4b080b0c30 2024-04-25T09:09:16.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b07e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0806fe50 con 0x7f4b080b0750 2024-04-25T09:09:16.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b07e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080709f0 con 0x7f4b0815dba0 2024-04-25T09:09:16.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b06000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0c30 0x7f4b0815cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b06000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0c30 0x7f4b0815cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51122/0 (socket says 172.21.15.62:51122) 2024-04-25T09:09:16.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.028+0000 7f4b06000640 1 -- 172.21.15.62:0/1282716223 learned_addr learned my addr 172.21.15.62:0/1282716223 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:16.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4b07400640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b0815dba0 0x7f4b08158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4b06a00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b080b0750 0x7f4b08153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4b06000640 1 -- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b0815dba0 msgr2=0x7f4b08158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4b06000640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b0815dba0 0x7f4b08158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4b06000640 1 -- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b080b0750 msgr2=0x7f4b08153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4b06000640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b080b0750 0x7f4b08153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4b06000640 1 -- 172.21.15.62:0/1282716223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08153980 con 0x7f4b080b0c30 2024-04-25T09:09:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4b06000640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0c30 0x7f4b0815cc90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4af002a6a0 tx=0x7f4af00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.029+0000 7f4afea00640 1 -- 172.21.15.62:0/1282716223 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0037040 con 0x7f4b080b0c30 2024-04-25T09:09:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.030+0000 7f4afea00640 1 -- 172.21.15.62:0/1282716223 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af0042070 con 0x7f4b080b0c30 2024-04-25T09:09:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.030+0000 7f4afea00640 1 -- 172.21.15.62:0/1282716223 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af003b870 con 0x7f4b080b0c30 2024-04-25T09:09:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.030+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b08150820 con 0x7f4b080b0c30 2024-04-25T09:09:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.030+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b08153c60 con 0x7f4b080b0c30 2024-04-25T09:09:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.031+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0804ef20 con 0x7f4b080b0c30 2024-04-25T09:09:16.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.032+0000 7f4afea00640 1 -- 172.21.15.62:0/1282716223 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4af003ba10 con 0x7f4b080b0c30 2024-04-25T09:09:16.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.033+0000 7f4afea00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ae406b4d0 0x7f4ae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.033+0000 7f4afea00640 1 -- 172.21.15.62:0/1282716223 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4af00aa9f0 con 0x7f4b080b0c30 2024-04-25T09:09:16.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.033+0000 7f4b06a00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ae406b4d0 0x7f4ae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.035+0000 7f4b06a00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ae406b4d0 0x7f4ae406d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4af8007bf0 tx=0x7f4af8007240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.037+0000 7f4afea00640 1 -- 172.21.15.62:0/1282716223 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4af0030e20 con 0x7f4b080b0c30 2024-04-25T09:09:16.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.237+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b08157d00 con 0x7f4b080b0c30 2024-04-25T09:09:16.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.240+0000 7f4afea00640 1 -- 172.21.15.62:0/1282716223 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f4af00797d0 con 0x7f4b080b0c30 2024-04-25T09:09:16.228 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:16.229 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":68,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:01.691769+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:08:34.817879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T09:08:35.997553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.d","mappings":[{"from":6,"to":1}]},{"pgid":"9.37","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/3491368038":"2024-04-26T09:08:50.613426+0000","172.21.15.62:6832/3491368038":"2024-04-26T09:08:50.613426+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.242+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ae406b4d0 msgr2=0x7f4ae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.242+0000 7f4b07e00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ae406b4d0 0x7f4ae406d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4af8007bf0 tx=0x7f4af8007240 comp rx=0 tx=0).stop 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.242+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0c30 msgr2=0x7f4b0815cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.242+0000 7f4b07e00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0c30 0x7f4b0815cc90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4af002a6a0 tx=0x7f4af00046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.242+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 shutdown_connections 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.242+0000 7f4b07e00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ae406b4d0 0x7f4ae406d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.242+0000 7f4b07e00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4b080b0c30 0x7f4b0815cc90 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.243+0000 7f4b07e00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4b0815dba0 0x7f4b08158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.243+0000 7f4b07e00640 1 --2- 172.21.15.62:0/1282716223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b080b0750 0x7f4b08153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.243+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 >> 172.21.15.62:0/1282716223 conn(0x7f4b080b3480 msgr2=0x7f4b0806b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:16.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.243+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 shutdown_connections 2024-04-25T09:09:16.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.243+0000 7f4b07e00640 1 -- 172.21.15.62:0/1282716223 wait complete. 2024-04-25T09:09:16.251 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:09:16.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.478+0000 7fe613200640 1 Processor -- start 2024-04-25T09:09:16.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.478+0000 7fe613200640 1 -- start start 2024-04-25T09:09:16.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.479+0000 7fe613200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe614161230 0x7fe6140b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.479+0000 7fe613200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe6140d0fb0 0x7fe6140b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.479+0000 7fe613200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140b5a60 0x7fe6140d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.479+0000 7fe613200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6140d2040 con 0x7fe6140d0fb0 2024-04-25T09:09:16.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.480+0000 7fe613200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6140cea70 con 0x7fe614161230 2024-04-25T09:09:16.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.480+0000 7fe613200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140ced50 con 0x7fe6140b5a60 2024-04-25T09:09:16.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.480+0000 7fe60be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe6140d0fb0 0x7fe6140b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.480+0000 7fe60be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe6140d0fb0 0x7fe6140b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51136/0 (socket says 172.21.15.62:51136) 2024-04-25T09:09:16.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.480+0000 7fe60be00640 1 -- 172.21.15.62:0/1735940784 learned_addr learned my addr 172.21.15.62:0/1735940784 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:16.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.480+0000 7fe611e00640 1 --2- 172.21.15.62:0/1735940784 >> v2:172.21.15.121:3300/0 conn(0x7fe614161230 0x7fe6140b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.480+0000 7fe612800640 1 --2- 172.21.15.62:0/1735940784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140b5a60 0x7fe6140d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.481+0000 7fe60be00640 1 -- 172.21.15.62:0/1735940784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140b5a60 msgr2=0x7fe6140d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:16.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.481+0000 7fe60be00640 1 --2- 172.21.15.62:0/1735940784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140b5a60 0x7fe6140d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.481+0000 7fe60be00640 1 -- 172.21.15.62:0/1735940784 >> v2:172.21.15.121:3300/0 conn(0x7fe614161230 msgr2=0x7fe6140b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:16.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.481+0000 7fe60be00640 1 --2- 172.21.15.62:0/1735940784 >> v2:172.21.15.121:3300/0 conn(0x7fe614161230 0x7fe6140b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.481+0000 7fe60be00640 1 -- 172.21.15.62:0/1735940784 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140cf030 con 0x7fe6140d0fb0 2024-04-25T09:09:16.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.482+0000 7fe60be00640 1 --2- 172.21.15.62:0/1735940784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6140d0fb0 0x7fe6140b5520 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe61408cf60 tx=0x7fe5fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98437a7f0b32cd70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.482+0000 7fe611400640 1 -- 172.21.15.62:0/1735940784 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc037070 con 0x7fe6140d0fb0 2024-04-25T09:09:16.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.482+0000 7fe611400640 1 -- 172.21.15.62:0/1735940784 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc032440 con 0x7fe6140d0fb0 2024-04-25T09:09:16.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.482+0000 7fe611400640 1 -- 172.21.15.62:0/1735940784 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc032810 con 0x7fe6140d0fb0 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 -- 172.21.15.62:0/1735940784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6140d0fb0 msgr2=0x7fe6140b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 --2- 172.21.15.62:0/1735940784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6140d0fb0 0x7fe6140b5520 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe61408cf60 tx=0x7fe5fc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 -- 172.21.15.62:0/1735940784 shutdown_connections 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 --2- 172.21.15.62:0/1735940784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140b5a60 0x7fe6140d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 --2- 172.21.15.62:0/1735940784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6140d0fb0 0x7fe6140b5520 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 --2- 172.21.15.62:0/1735940784 >> v2:172.21.15.121:3300/0 conn(0x7fe614161230 0x7fe6140b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 -- 172.21.15.62:0/1735940784 >> 172.21.15.62:0/1735940784 conn(0x7fe614066d10 msgr2=0x7fe614067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 -- 172.21.15.62:0/1735940784 shutdown_connections 2024-04-25T09:09:16.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.483+0000 7fe613200640 1 -- 172.21.15.62:0/1735940784 wait complete. 2024-04-25T09:09:16.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.485+0000 7fe613200640 1 Processor -- start 2024-04-25T09:09:16.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.485+0000 7fe613200640 1 -- start start 2024-04-25T09:09:16.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.486+0000 7fe613200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6140b5a60 0x7fe6141f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.486+0000 7fe613200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140d0fb0 0x7fe6141f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.486+0000 7fe613200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe614161230 0x7fe6141f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.486+0000 7fe613200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe614161610 con 0x7fe614161230 2024-04-25T09:09:16.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.486+0000 7fe613200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140ca8d0 con 0x7fe6140b5a60 2024-04-25T09:09:16.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.486+0000 7fe613200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe614161790 con 0x7fe6140d0fb0 2024-04-25T09:09:16.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.486+0000 7fe612800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe614161230 0x7fe6141f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe612800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe614161230 0x7fe6141f7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51152/0 (socket says 172.21.15.62:51152) 2024-04-25T09:09:16.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe612800640 1 -- 172.21.15.62:0/1301786870 learned_addr learned my addr 172.21.15.62:0/1301786870 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:16.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe611e00640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6140b5a60 0x7fe6141f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe60be00640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140d0fb0 0x7fe6141f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe612800640 1 -- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140d0fb0 msgr2=0x7fe6141f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe612800640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140d0fb0 0x7fe6141f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe612800640 1 -- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6140b5a60 msgr2=0x7fe6141f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe612800640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6140b5a60 0x7fe6141f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe612800640 1 -- 172.21.15.62:0/1301786870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140cdc10 con 0x7fe614161230 2024-04-25T09:09:16.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.487+0000 7fe612800640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe614161230 0x7fe6141f7900 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00ef00 tx=0x7fe60c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.488+0000 7fe60a000640 1 -- 172.21.15.62:0/1301786870 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c018020 con 0x7fe614161230 2024-04-25T09:09:16.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.488+0000 7fe60a000640 1 -- 172.21.15.62:0/1301786870 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c014400 con 0x7fe614161230 2024-04-25T09:09:16.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.488+0000 7fe60a000640 1 -- 172.21.15.62:0/1301786870 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0147f0 con 0x7fe614161230 2024-04-25T09:09:16.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.488+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6141e82d0 con 0x7fe614161230 2024-04-25T09:09:16.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.488+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6141e8890 con 0x7fe614161230 2024-04-25T09:09:16.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.489+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61404ef20 con 0x7fe614161230 2024-04-25T09:09:16.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.490+0000 7fe60a000640 1 -- 172.21.15.62:0/1301786870 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe60c005d10 con 0x7fe614161230 2024-04-25T09:09:16.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.491+0000 7fe60a000640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5f006b520 0x7fe5f006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.491+0000 7fe60a000640 1 -- 172.21.15.62:0/1301786870 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe60c08a720 con 0x7fe614161230 2024-04-25T09:09:16.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.492+0000 7fe611e00640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5f006b520 0x7fe5f006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.493+0000 7fe611e00640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5f006b520 0x7fe5f006d9e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe604009aa0 tx=0x7fe604009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.495+0000 7fe60a000640 1 -- 172.21.15.62:0/1301786870 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe60c022070 con 0x7fe614161230 2024-04-25T09:09:16.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.740+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe6140b98c0 con 0x7fe614161230 2024-04-25T09:09:16.729 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:09:16.741+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:09:16.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.745+0000 7fe60a000640 1 -- 172.21.15.62:0/1301786870 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe60c059680 con 0x7fe614161230 2024-04-25T09:09:16.733 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:09:16.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.748+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5f006b520 msgr2=0x7fe5f006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.748+0000 7fe613200640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5f006b520 0x7fe5f006d9e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe604009aa0 tx=0x7fe604009190 comp rx=0 tx=0).stop 2024-04-25T09:09:16.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.748+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe614161230 msgr2=0x7fe6141f7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.748+0000 7fe613200640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe614161230 0x7fe6141f7900 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00ef00 tx=0x7fe60c00b480 comp rx=0 tx=0).stop 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.749+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 shutdown_connections 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.749+0000 7fe613200640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe5f006b520 0x7fe5f006d9e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.749+0000 7fe613200640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe614161230 0x7fe6141f7900 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.749+0000 7fe613200640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6140d0fb0 0x7fe6141f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.749+0000 7fe613200640 1 --2- 172.21.15.62:0/1301786870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6140b5a60 0x7fe6141f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.749+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 >> 172.21.15.62:0/1301786870 conn(0x7fe614066d10 msgr2=0x7fe6140d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.749+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 shutdown_connections 2024-04-25T09:09:16.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.749+0000 7fe613200640 1 -- 172.21.15.62:0/1301786870 wait complete. 2024-04-25T09:09:16.749 INFO:tasks.ceph.mds.a.smithi062.stderr: -2537> 2024-04-25T09:08:57.637+0000 7f2548c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:09:16.758 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:09:16.806 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:09:16.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.959+0000 7ffbbb000640 1 Processor -- start 2024-04-25T09:09:16.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.959+0000 7ffbbb000640 1 -- start start 2024-04-25T09:09:16.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.961+0000 7ffbbb000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40ccde0 0x7ffbb40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.961+0000 7ffbbb000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b31a0 0x7ffbb40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.961+0000 7ffbbb000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb40ced40 con 0x7ffbb406d2c0 2024-04-25T09:09:16.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.961+0000 7ffbbb000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb40cd260 con 0x7ffbb40b31a0 2024-04-25T09:09:16.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.961+0000 7ffbbb000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40cf000 con 0x7ffbb40ccde0 2024-04-25T09:09:16.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.961+0000 7ffbb9c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ffbb406d2c0 0x7ffbb406d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44488/0 (socket says 172.21.15.62:44488) 2024-04-25T09:09:16.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.961+0000 7ffbb9c00640 1 -- 172.21.15.62:0/3541279818 learned_addr learned my addr 172.21.15.62:0/3541279818 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb9200640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40ccde0 0x7ffbb40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbba600640 1 --2- 172.21.15.62:0/3541279818 >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b31a0 0x7ffbb40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866212794 0 0) 0x7ffbb40ced40 con 0x7ffbb406d2c0 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb9c003520 con 0x7ffbb406d2c0 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3037805791 0 0) 0x7ffb9c003520 con 0x7ffbb406d2c0 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40ccde0 msgr2=0x7ffbb40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40ccde0 0x7ffbb40ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b31a0 msgr2=0x7ffbb40b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 --2- 172.21.15.62:0/3541279818 >> v2:172.21.15.121:3300/0 conn(0x7ffbb40b31a0 0x7ffbb40b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40cf2e0 con 0x7ffbb406d2c0 2024-04-25T09:09:16.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ffba8001670 con 0x7ffbb406d2c0 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 >> v1:172.21.15.62:6789/0 conn(0x7ffbb406d2c0 legacy=0x7ffbb406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40ccde0 0x7ffb9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb9c003f80 0x7ffb9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c004d40 0x7ffb9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40cd260 con 0x7ffb9c004d40 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40cf000 con 0x7ffb9c003f80 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.962+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c00b9c0 con 0x7ffbb40ccde0 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbba600640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c004d40 0x7ffb9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbb9c00640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40ccde0 0x7ffb9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbb9200640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb9c003f80 0x7ffb9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbba600640 1 -- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40ccde0 msgr2=0x7ffb9c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbba600640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40ccde0 0x7ffb9c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbba600640 1 -- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb9c003f80 msgr2=0x7ffb9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:16.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbba600640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb9c003f80 0x7ffb9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbba600640 1 -- 172.21.15.62:0/3541279818 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c003520 con 0x7ffb9c004d40 2024-04-25T09:09:16.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.963+0000 7ffbb9200640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb9c003f80 0x7ffb9c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:16.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.964+0000 7ffbba600640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c004d40 0x7ffb9c0051c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00e170 tx=0x7ffb9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7905dd04fbcd36d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac034020 con 0x7ffb9c004d40 2024-04-25T09:09:16.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac030430 con 0x7ffb9c004d40 2024-04-25T09:09:16.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbb3e00640 1 -- 172.21.15.62:0/3541279818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac009c10 con 0x7ffb9c004d40 2024-04-25T09:09:16.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbbb000640 1 -- 172.21.15.62:0/3541279818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c004d40 msgr2=0x7ffb9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbbb000640 1 --2- 172.21.15.62:0/3541279818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffb9c004d40 0x7ffb9c0051c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00e170 tx=0x7ffb9c005830 comp rx=0 tx=0).stop 2024-04-25T09:09:16.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbba600640 1 -- 172.21.15.62:0/3541279818 reap_dead start 2024-04-25T09:09:16.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbbb000640 1 -- 172.21.15.62:0/3541279818 shutdown_connections 2024-04-25T09:09:16.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbbb000640 1 -- 172.21.15.62:0/3541279818 >> 172.21.15.62:0/3541279818 conn(0x7ffbb4066f20 msgr2=0x7ffbb40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:16.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.965+0000 7ffbbb000640 1 -- 172.21.15.62:0/3541279818 shutdown_connections 2024-04-25T09:09:16.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.966+0000 7ffbbb000640 1 -- 172.21.15.62:0/3541279818 wait complete. 2024-04-25T09:09:16.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.967+0000 7ffbbb000640 1 Processor -- start 2024-04-25T09:09:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.967+0000 7ffbbb000640 1 -- start start 2024-04-25T09:09:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.968+0000 7ffbbb000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9c004d40 0x7ffbb4141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.968+0000 7ffbbb000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb41eee50 0x7ffbb41ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.968+0000 7ffbbb000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40c9df0 0x7ffbb40ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.968+0000 7ffbbb000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40b2c80 con 0x7ffbb41eee50 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.968+0000 7ffbbb000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40b29a0 con 0x7ffbb40c9df0 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.968+0000 7ffbbb000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb41448d0 con 0x7ffb9c004d40 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb41eee50 0x7ffbb41ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb41eee50 0x7ffbb41ef2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51192/0 (socket says 172.21.15.62:51192) 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 -- 172.21.15.62:0/3330747327 learned_addr learned my addr 172.21.15.62:0/3330747327 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbba600640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40c9df0 0x7ffbb40ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9c00640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9c004d40 0x7ffbb4141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 -- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9c004d40 msgr2=0x7ffbb4141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9c004d40 0x7ffbb4141230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 -- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40c9df0 msgr2=0x7ffbb40ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40c9df0 0x7ffbb40ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:16.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 -- 172.21.15.62:0/3330747327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40c77c0 con 0x7ffbb41eee50 2024-04-25T09:09:16.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbba600640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40c9df0 0x7ffbb40ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:16.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.969+0000 7ffbb9200640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb41eee50 0x7ffbb41ef2d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ffba00108b0 tx=0x7ffba0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.970+0000 7ffbb1600640 1 -- 172.21.15.62:0/3330747327 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000f040 con 0x7ffbb41eee50 2024-04-25T09:09:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.970+0000 7ffbb1600640 1 -- 172.21.15.62:0/3330747327 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000eb60 con 0x7ffbb41eee50 2024-04-25T09:09:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.970+0000 7ffbb1600640 1 -- 172.21.15.62:0/3330747327 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba001c400 con 0x7ffbb41eee50 2024-04-25T09:09:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.970+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb40c7aa0 con 0x7ffbb41eee50 2024-04-25T09:09:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.970+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb40c8060 con 0x7ffbb41eee50 2024-04-25T09:09:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.971+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb406abe0 con 0x7ffbb41eee50 2024-04-25T09:09:16.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.972+0000 7ffbb1600640 1 -- 172.21.15.62:0/3330747327 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffba001c5a0 con 0x7ffbb41eee50 2024-04-25T09:09:16.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.973+0000 7ffbb1600640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b500 0x7ffb9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:16.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.973+0000 7ffbb1600640 1 -- 172.21.15.62:0/3330747327 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ffba008a9d0 con 0x7ffbb41eee50 2024-04-25T09:09:16.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.973+0000 7ffbb9c00640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b500 0x7ffb9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:16.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.975+0000 7ffbb9c00640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b500 0x7ffb9406d9c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffba8001050 tx=0x7ffba8001e70 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:16.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:16.977+0000 7ffbb1600640 1 -- 172.21.15.62:0/3330747327 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffba0018030 con 0x7ffbb41eee50 2024-04-25T09:09:17.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.213+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffbb404ef20 con 0x7ffbb41eee50 2024-04-25T09:09:17.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.793+0000 7ffbb1600640 1 -- 172.21.15.62:0/3330747327 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffba0059750 con 0x7ffbb41eee50 2024-04-25T09:09:17.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b500 msgr2=0x7ffb9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:17.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b500 0x7ffb9406d9c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffba8001050 tx=0x7ffba8001e70 comp rx=0 tx=0).stop 2024-04-25T09:09:17.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb41eee50 msgr2=0x7ffbb41ef2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:17.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb41eee50 0x7ffbb41ef2d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ffba00108b0 tx=0x7ffba0010d70 comp rx=0 tx=0).stop 2024-04-25T09:09:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 shutdown_connections 2024-04-25T09:09:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9406b500 0x7ffb9406d9c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb41eee50 0x7ffbb41ef2d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb40c9df0 0x7ffbb40ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 --2- 172.21.15.62:0/3330747327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffb9c004d40 0x7ffbb4141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 >> 172.21.15.62:0/3330747327 conn(0x7ffbb4066f20 msgr2=0x7ffbb4068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 shutdown_connections 2024-04-25T09:09:17.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:17.797+0000 7ffbbb000640 1 -- 172.21.15.62:0/3330747327 wait complete. 2024-04-25T09:09:17.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:09:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.026+0000 7f90f1000640 1 Processor -- start 2024-04-25T09:09:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.026+0000 7f90f1000640 1 -- start start 2024-04-25T09:09:18.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.028+0000 7f90f1000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 0x7f90ec0b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.028+0000 7f90f1000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ec15c9a0 con 0x7f90ec0b4730 2024-04-25T09:09:18.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.028+0000 7f90f1000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ec15cc80 con 0x7f90ec06c440 2024-04-25T09:09:18.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.028+0000 7f90f1000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec0b3d60 con 0x7f90ec145140 2024-04-25T09:09:18.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.028+0000 7f90eaa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f90ec0b4730 0x7f90ec15beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44496/0 (socket says 172.21.15.62:44496) 2024-04-25T09:09:18.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.028+0000 7f90eaa00640 1 -- 172.21.15.62:0/2016265614 learned_addr learned my addr 172.21.15.62:0/2016265614 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:18.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90eb400640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 0x7f90ec0b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423478842 0 0) 0x7f90ec15c9a0 con 0x7f90ec0b4730 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90d0003520 con 0x7f90ec0b4730 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1900023848 0 0) 0x7f90d0003520 con 0x7f90ec0b4730 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 msgr2=0x7f90ec0b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 0x7f90ec0b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 >> v1:172.21.15.121:6789/0 conn(0x7f90ec06c440 legacy=0x7f90ec06c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec0b39f0 con 0x7f90ec0b4730 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f90d8001670 con 0x7f90ec0b4730 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 >> v1:172.21.15.62:6789/0 conn(0x7f90ec0b4730 legacy=0x7f90ec15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 0x7f90d00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0003fb0 0x7f90d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d0004d40 0x7f90d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec0b3d60 con 0x7f90d0004d40 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d000bb00 con 0x7f90d0003fb0 2024-04-25T09:09:18.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.029+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d000bd00 con 0x7f90ec145140 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90ebe00640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d0004d40 0x7f90d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90eaa00640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0003fb0 0x7f90d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90eb400640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 0x7f90d00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90ebe00640 1 -- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 msgr2=0x7f90d00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90ebe00640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 0x7f90d00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90ebe00640 1 -- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0003fb0 msgr2=0x7f90d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90ebe00640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0003fb0 0x7f90d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90ebe00640 1 -- 172.21.15.62:0/2016265614 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0003520 con 0x7f90d0004d40 2024-04-25T09:09:18.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.030+0000 7f90eb400640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec145140 0x7f90d00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:18.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.031+0000 7f90ebe00640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d0004d40 0x7f90d00051c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f90e4001e50 tx=0x7f90d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bbb062e984b112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.031+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4001a70 con 0x7f90d0004d40 2024-04-25T09:09:18.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.031+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4003970 con 0x7f90d0004d40 2024-04-25T09:09:18.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.031+0000 7f90ea000640 1 -- 172.21.15.62:0/2016265614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4003d60 con 0x7f90d0004d40 2024-04-25T09:09:18.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.032+0000 7f90f1000640 1 -- 172.21.15.62:0/2016265614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d0004d40 msgr2=0x7f90d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.032+0000 7f90f1000640 1 --2- 172.21.15.62:0/2016265614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90d0004d40 0x7f90d00051c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f90e4001e50 tx=0x7f90d0005830 comp rx=0 tx=0).stop 2024-04-25T09:09:18.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.032+0000 7f90ebe00640 1 -- 172.21.15.62:0/2016265614 reap_dead start 2024-04-25T09:09:18.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.032+0000 7f90f1000640 1 -- 172.21.15.62:0/2016265614 shutdown_connections 2024-04-25T09:09:18.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.032+0000 7f90f1000640 1 -- 172.21.15.62:0/2016265614 >> 172.21.15.62:0/2016265614 conn(0x7f90ec066ad0 msgr2=0x7f90ec066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:18.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.032+0000 7f90f1000640 1 -- 172.21.15.62:0/2016265614 shutdown_connections 2024-04-25T09:09:18.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.032+0000 7f90f1000640 1 -- 172.21.15.62:0/2016265614 wait complete. 2024-04-25T09:09:18.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.034+0000 7f90f1000640 1 Processor -- start 2024-04-25T09:09:18.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.034+0000 7f90f1000640 1 -- start start 2024-04-25T09:09:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90f1000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0004d40 0x7f90ec0cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90f1000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec13ad80 0x7f90ec13b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90f1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ec13e770 0x7f90ec13ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90f1000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec15c740 con 0x7f90ec13e770 2024-04-25T09:09:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90f1000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec15c5c0 con 0x7f90d0004d40 2024-04-25T09:09:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90f1000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec145520 con 0x7f90ec13ad80 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90ebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ec13e770 0x7f90ec13ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90ebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ec13e770 0x7f90ec13ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51204/0 (socket says 172.21.15.62:51204) 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.035+0000 7f90ebe00640 1 -- 172.21.15.62:0/3501301430 learned_addr learned my addr 172.21.15.62:0/3501301430 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90eb400640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0004d40 0x7f90ec0cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90eaa00640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec13ad80 0x7f90ec13b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90ebe00640 1 -- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec13ad80 msgr2=0x7f90ec13b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90ebe00640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec13ad80 0x7f90ec13b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90ebe00640 1 -- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0004d40 msgr2=0x7f90ec0cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90ebe00640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0004d40 0x7f90ec0cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90ebe00640 1 -- 172.21.15.62:0/3501301430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec1f75d0 con 0x7f90ec13e770 2024-04-25T09:09:18.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90ebe00640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ec13e770 0x7f90ec13ebf0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f90e4001f70 tx=0x7f90e40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90e3400640 1 -- 172.21.15.62:0/3501301430 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4001a70 con 0x7f90ec13e770 2024-04-25T09:09:18.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90e3400640 1 -- 172.21.15.62:0/3501301430 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4040070 con 0x7f90ec13e770 2024-04-25T09:09:18.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90e3400640 1 -- 172.21.15.62:0/3501301430 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4007db0 con 0x7f90ec13e770 2024-04-25T09:09:18.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.036+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ec1f78b0 con 0x7f90ec13e770 2024-04-25T09:09:18.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.037+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ec0c9a00 con 0x7f90ec13e770 2024-04-25T09:09:18.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.038+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ec072b30 con 0x7f90ec13e770 2024-04-25T09:09:18.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.039+0000 7f90e3400640 1 -- 172.21.15.62:0/3501301430 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90e403a070 con 0x7f90ec13e770 2024-04-25T09:09:18.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.040+0000 7f90e3400640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90cc06b4d0 0x7f90cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:18.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.040+0000 7f90e3400640 1 -- 172.21.15.62:0/3501301430 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f90e40a9c10 con 0x7f90ec13e770 2024-04-25T09:09:18.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.040+0000 7f90eb400640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90cc06b4d0 0x7f90cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:18.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.041+0000 7f90eb400640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90cc06b4d0 0x7f90cc06d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009aa0 tx=0x7f90dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:18.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.044+0000 7f90e3400640 1 -- 172.21.15.62:0/3501301430 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90e4078990 con 0x7f90ec13e770 2024-04-25T09:09:18.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.234+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f90ec0750e0 con 0x7f90ec13e770 2024-04-25T09:09:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.815+0000 7f90e3400640 1 -- 172.21.15.62:0/3501301430 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) v1 ==== 181+0+0 (secure 0 0 0) 0x7f90e4078290 con 0x7f90ec13e770 2024-04-25T09:09:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:09:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90cc06b4d0 msgr2=0x7f90cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90cc06b4d0 0x7f90cc06d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009aa0 tx=0x7f90dc008040 comp rx=0 tx=0).stop 2024-04-25T09:09:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ec13e770 msgr2=0x7f90ec13ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ec13e770 0x7f90ec13ebf0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f90e4001f70 tx=0x7f90e40014a0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 shutdown_connections 2024-04-25T09:09:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f90cc06b4d0 0x7f90cc06d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f90ec13e770 0x7f90ec13ebf0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90ec13ad80 0x7f90ec13b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 --2- 172.21.15.62:0/3501301430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90d0004d40 0x7f90ec0cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 >> 172.21.15.62:0/3501301430 conn(0x7f90ec066ad0 msgr2=0x7f90ec066eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 shutdown_connections 2024-04-25T09:09:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:18.819+0000 7f90f1000640 1 -- 172.21.15.62:0/3501301430 wait complete. 2024-04-25T09:09:18.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:09:19.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.044+0000 7f3d89800640 1 Processor -- start 2024-04-25T09:09:19.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.044+0000 7f3d89800640 1 -- start start 2024-04-25T09:09:19.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.045+0000 7f3d89800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3d840ceb10 0x7f3d840d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.045+0000 7f3d89800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d840d1c50 0x7f3d840b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.045+0000 7f3d89800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d840d3770 con 0x7f3d8415dd90 2024-04-25T09:09:19.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.045+0000 7f3d89800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8415e170 con 0x7f3d840ceb10 2024-04-25T09:09:19.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.046+0000 7f3d89800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840d3a50 con 0x7f3d840d1c50 2024-04-25T09:09:19.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.046+0000 7f3d82a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3d8415dd90 0x7f3d840ce400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44506/0 (socket says 172.21.15.62:44506) 2024-04-25T09:09:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.046+0000 7f3d82a00640 1 -- 172.21.15.62:0/2884502548 learned_addr learned my addr 172.21.15.62:0/2884502548 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:19.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.046+0000 7f3d7be00640 1 --2- 172.21.15.62:0/2884502548 >> v2:172.21.15.121:3300/0 conn(0x7f3d840ceb10 0x7f3d840d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.046+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163195820 0 0) 0x7f3d840d3770 con 0x7f3d8415dd90 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.046+0000 7f3d83400640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d840d1c50 0x7f3d840b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d68003520 con 0x7f3d8415dd90 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389889429 0 0) 0x7f3d68003520 con 0x7f3d8415dd90 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d840d1c50 msgr2=0x7f3d840b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d840d1c50 0x7f3d840b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 >> v2:172.21.15.121:3300/0 conn(0x7f3d840ceb10 msgr2=0x7f3d840d3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 --2- 172.21.15.62:0/2884502548 >> v2:172.21.15.121:3300/0 conn(0x7f3d840ceb10 0x7f3d840d3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840d3d30 con 0x7f3d8415dd90 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f3d74001690 con 0x7f3d8415dd90 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 >> v1:172.21.15.62:6789/0 conn(0x7f3d8415dd90 legacy=0x7f3d840ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d840d1c50 0x7f3d680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d68003f80 0x7f3d68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d68004d40 0x7f3d680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840d3a50 con 0x7f3d68004d40 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6800b9f0 con 0x7f3d68003f80 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.047+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6800bbf0 con 0x7f3d840d1c50 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.048+0000 7f3d83400640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d68004d40 0x7f3d680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.048+0000 7f3d82a00640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d840d1c50 0x7f3d680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.048+0000 7f3d7be00640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d68003f80 0x7f3d68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.048+0000 7f3d83400640 1 -- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d840d1c50 msgr2=0x7f3d680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.048+0000 7f3d83400640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d840d1c50 0x7f3d680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.048+0000 7f3d83400640 1 -- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d68003f80 msgr2=0x7f3d68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.048+0000 7f3d83400640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d68003f80 0x7f3d68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.048+0000 7f3d83400640 1 -- 172.21.15.62:0/2884502548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68003520 con 0x7f3d68004d40 2024-04-25T09:09:19.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.049+0000 7f3d83400640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d68004d40 0x7f3d680051c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c007360 tx=0x7f3d68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef69131179253f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.049+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c009be0 con 0x7f3d68004d40 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.049+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c036600 con 0x7f3d68004d40 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.049+0000 7f3d82000640 1 -- 172.21.15.62:0/2884502548 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0369d0 con 0x7f3d68004d40 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.050+0000 7f3d89800640 1 -- 172.21.15.62:0/2884502548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d68004d40 msgr2=0x7f3d680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.050+0000 7f3d89800640 1 --2- 172.21.15.62:0/2884502548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d68004d40 0x7f3d680051c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c007360 tx=0x7f3d68005830 comp rx=0 tx=0).stop 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.050+0000 7f3d83400640 1 -- 172.21.15.62:0/2884502548 reap_dead start 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.050+0000 7f3d89800640 1 -- 172.21.15.62:0/2884502548 shutdown_connections 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.050+0000 7f3d89800640 1 -- 172.21.15.62:0/2884502548 >> 172.21.15.62:0/2884502548 conn(0x7f3d8406b000 msgr2=0x7f3d8406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.050+0000 7f3d89800640 1 -- 172.21.15.62:0/2884502548 shutdown_connections 2024-04-25T09:09:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.050+0000 7f3d89800640 1 -- 172.21.15.62:0/2884502548 wait complete. 2024-04-25T09:09:19.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.052+0000 7f3d89800640 1 Processor -- start 2024-04-25T09:09:19.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.052+0000 7f3d89800640 1 -- start start 2024-04-25T09:09:19.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d89800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d68004d40 0x7f3d840b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d89800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d840ca370 0x7f3d841f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d89800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d840b4fe0 0x7f3d8415f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d89800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840cf1c0 con 0x7f3d840ca370 2024-04-25T09:09:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d89800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840d42f0 con 0x7f3d840b4fe0 2024-04-25T09:09:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d89800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840b6e30 con 0x7f3d68004d40 2024-04-25T09:09:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d840ca370 0x7f3d841f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d7be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d840ca370 0x7f3d841f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51218/0 (socket says 172.21.15.62:51218) 2024-04-25T09:09:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.053+0000 7f3d7be00640 1 -- 172.21.15.62:0/3383274130 learned_addr learned my addr 172.21.15.62:0/3383274130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d82a00640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d68004d40 0x7f3d840b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7be00640 1 -- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d68004d40 msgr2=0x7f3d840b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7be00640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d68004d40 0x7f3d840b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7be00640 1 -- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d840b4fe0 msgr2=0x7f3d8415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7be00640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d840b4fe0 0x7f3d8415f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7be00640 1 -- 172.21.15.62:0/3383274130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d841f4240 con 0x7f3d840ca370 2024-04-25T09:09:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7be00640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d840ca370 0x7f3d841f3d00 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c011700 tx=0x7f3d6c011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7aa00640 1 -- 172.21.15.62:0/3383274130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00f940 con 0x7f3d840ca370 2024-04-25T09:09:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7aa00640 1 -- 172.21.15.62:0/3383274130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c00fae0 con 0x7f3d840ca370 2024-04-25T09:09:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d7aa00640 1 -- 172.21.15.62:0/3383274130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c00fe90 con 0x7f3d840ca370 2024-04-25T09:09:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.054+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d840b3b50 con 0x7f3d840ca370 2024-04-25T09:09:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.055+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d840b4110 con 0x7f3d840ca370 2024-04-25T09:09:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.055+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d84067c40 con 0x7f3d840ca370 2024-04-25T09:09:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.057+0000 7f3d7aa00640 1 -- 172.21.15.62:0/3383274130 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d6c015460 con 0x7f3d840ca370 2024-04-25T09:09:19.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.058+0000 7f3d7aa00640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d6006b6f0 0x7f3d6006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:19.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.058+0000 7f3d7aa00640 1 -- 172.21.15.62:0/3383274130 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5501+0+0 (secure 0 0 0) 0x7f3d6c08aab0 con 0x7f3d840ca370 2024-04-25T09:09:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.060+0000 7f3d82a00640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d6006b6f0 0x7f3d6006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:19.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.066+0000 7f3d82a00640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d6006b6f0 0x7f3d6006dbb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3d74001050 tx=0x7f3d74001170 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:19.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.069+0000 7f3d7aa00640 1 -- 172.21.15.62:0/3383274130 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d841f4240 con 0x7f3d840ca370 2024-04-25T09:09:19.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.268+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d840ca7f0 con 0x7f3d840ca370 2024-04-25T09:09:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.830+0000 7f3d7aa00640 1 -- 172.21.15.62:0/3383274130 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3d6c019030 con 0x7f3d840ca370 2024-04-25T09:09:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:09:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.833+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d6006b6f0 msgr2=0x7f3d6006dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.833+0000 7f3d89800640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d6006b6f0 0x7f3d6006dbb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3d74001050 tx=0x7f3d74001170 comp rx=0 tx=0).stop 2024-04-25T09:09:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.834+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d840ca370 msgr2=0x7f3d841f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.834+0000 7f3d89800640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d840ca370 0x7f3d841f3d00 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c011700 tx=0x7f3d6c011bc0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.834+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 shutdown_connections 2024-04-25T09:09:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.834+0000 7f3d89800640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d6006b6f0 0x7f3d6006dbb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.834+0000 7f3d89800640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d840ca370 0x7f3d841f3d00 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.834+0000 7f3d89800640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d840b4fe0 0x7f3d8415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.834+0000 7f3d89800640 1 --2- 172.21.15.62:0/3383274130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d68004d40 0x7f3d840b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.834+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 >> 172.21.15.62:0/3383274130 conn(0x7f3d8406b000 msgr2=0x7f3d840cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.835+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 shutdown_connections 2024-04-25T09:09:19.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:19.836+0000 7f3d89800640 1 -- 172.21.15.62:0/3383274130 wait complete. 2024-04-25T09:09:19.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:09:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.059+0000 7fc180600640 1 Processor -- start 2024-04-25T09:09:20.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.059+0000 7fc180600640 1 -- start start 2024-04-25T09:09:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.061+0000 7fc180600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 0x7fc1780668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.061+0000 7fc180600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17806f130 con 0x7fc17815de20 2024-04-25T09:09:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.061+0000 7fc180600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc178072560 con 0x7fc178071f10 2024-04-25T09:09:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.061+0000 7fc180600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17806e510 con 0x7fc17806eca0 2024-04-25T09:09:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.061+0000 7fc17dc00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc17815de20 0x7fc17806d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44516/0 (socket says 172.21.15.62:44516) 2024-04-25T09:09:20.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.061+0000 7fc17dc00640 1 -- 172.21.15.62:0/2432475194 learned_addr learned my addr 172.21.15.62:0/2432475194 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568838628 0 0) 0x7fc17806f130 con 0x7fc17815de20 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc160003520 con 0x7fc17815de20 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc17e600640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 0x7fc1780668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 531970905 0 0) 0x7fc160003520 con 0x7fc17815de20 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 msgr2=0x7fc1780668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 0x7fc1780668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 >> v1:172.21.15.121:6789/0 conn(0x7fc178071f10 legacy=0x7fc1780b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780b5730 con 0x7fc17815de20 2024-04-25T09:09:20.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc16c0019f0 con 0x7fc17815de20 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc17e600640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 0x7fc1780668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 >> v1:172.21.15.62:6789/0 conn(0x7fc17815de20 legacy=0x7fc17806d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1600039e0 0x7fc160003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc160004800 0x7fc160004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 0x7fc160005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17806e510 con 0x7fc1600039e0 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000bb50 con 0x7fc160004800 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16000bd00 con 0x7fc17806eca0 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.062+0000 7fc17dc00640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1600039e0 0x7fc160003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.063+0000 7fc17e600640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 0x7fc160005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.063+0000 7fc17d200640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc160004800 0x7fc160004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.063+0000 7fc17dc00640 1 -- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 msgr2=0x7fc160005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.063+0000 7fc17dc00640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc17806eca0 0x7fc160005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.063+0000 7fc17dc00640 1 -- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc160004800 msgr2=0x7fc160004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.063+0000 7fc17dc00640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc160004800 0x7fc160004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.063+0000 7fc17dc00640 1 -- 172.21.15.62:0/2432475194 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160003520 con 0x7fc1600039e0 2024-04-25T09:09:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.063+0000 7fc17d200640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc160004800 0x7fc160004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:20.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.064+0000 7fc17dc00640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1600039e0 0x7fc160003e00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00a7b0 tx=0x7fc160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c07d1e87ac178a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.064+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00a3d0 con 0x7fc1600039e0 2024-04-25T09:09:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.064+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c02f070 con 0x7fc1600039e0 2024-04-25T09:09:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.064+0000 7fc177e00640 1 -- 172.21.15.62:0/2432475194 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c02bc40 con 0x7fc1600039e0 2024-04-25T09:09:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.065+0000 7fc180600640 1 -- 172.21.15.62:0/2432475194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1600039e0 msgr2=0x7fc160003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.065+0000 7fc180600640 1 --2- 172.21.15.62:0/2432475194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1600039e0 0x7fc160003e00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00a7b0 tx=0x7fc160005830 comp rx=0 tx=0).stop 2024-04-25T09:09:20.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.065+0000 7fc17e600640 1 -- 172.21.15.62:0/2432475194 reap_dead start 2024-04-25T09:09:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.065+0000 7fc180600640 1 -- 172.21.15.62:0/2432475194 shutdown_connections 2024-04-25T09:09:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.065+0000 7fc180600640 1 -- 172.21.15.62:0/2432475194 >> 172.21.15.62:0/2432475194 conn(0x7fc17806abe0 msgr2=0x7fc17806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.065+0000 7fc180600640 1 -- 172.21.15.62:0/2432475194 shutdown_connections 2024-04-25T09:09:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.065+0000 7fc180600640 1 -- 172.21.15.62:0/2432475194 wait complete. 2024-04-25T09:09:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.067+0000 7fc180600640 1 Processor -- start 2024-04-25T09:09:20.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.067+0000 7fc180600640 1 -- start start 2024-04-25T09:09:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.068+0000 7fc180600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1600039e0 0x7fc17813ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.068+0000 7fc180600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1781617a0 0x7fc178161c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.068+0000 7fc180600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc17813f9e0 0x7fc17813fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.068+0000 7fc180600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780b3fa0 con 0x7fc17813f9e0 2024-04-25T09:09:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.068+0000 7fc180600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d1010 con 0x7fc1781617a0 2024-04-25T09:09:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.068+0000 7fc180600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780b5cf0 con 0x7fc1600039e0 2024-04-25T09:09:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc17813f9e0 0x7fc17813fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc17813f9e0 0x7fc17813fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51248/0 (socket says 172.21.15.62:51248) 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 -- 172.21.15.62:0/820633846 learned_addr learned my addr 172.21.15.62:0/820633846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17dc00640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1600039e0 0x7fc17813ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 -- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1600039e0 msgr2=0x7fc17813ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1600039e0 0x7fc17813ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 -- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1781617a0 msgr2=0x7fc178161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1781617a0 0x7fc178161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 -- 172.21.15.62:0/820633846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17813d8c0 con 0x7fc17813f9e0 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.069+0000 7fc17e600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc17813f9e0 0x7fc17813fe60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc170006fd0 tx=0x7fc170010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.070+0000 7fc175600640 1 -- 172.21.15.62:0/820633846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17000f040 con 0x7fc17813f9e0 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.070+0000 7fc175600640 1 -- 172.21.15.62:0/820633846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17001c070 con 0x7fc17813f9e0 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.070+0000 7fc175600640 1 -- 172.21.15.62:0/820633846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170003610 con 0x7fc17813f9e0 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.070+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17813dba0 con 0x7fc17813f9e0 2024-04-25T09:09:20.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.070+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17813e160 con 0x7fc17813f9e0 2024-04-25T09:09:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.071+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17813e360 con 0x7fc17813f9e0 2024-04-25T09:09:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.072+0000 7fc175600640 1 -- 172.21.15.62:0/820633846 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1700037b0 con 0x7fc17813f9e0 2024-04-25T09:09:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.073+0000 7fc175600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc15806b4d0 0x7fc15806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.073+0000 7fc175600640 1 -- 172.21.15.62:0/820633846 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc170089b10 con 0x7fc17813f9e0 2024-04-25T09:09:20.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.074+0000 7fc17dc00640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc15806b4d0 0x7fc15806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.075+0000 7fc17dc00640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc15806b4d0 0x7fc15806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00a780 tx=0x7fc16c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.077+0000 7fc175600640 1 -- 172.21.15.62:0/820633846 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc17813e360 con 0x7fc17813f9e0 2024-04-25T09:09:20.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.279+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc17804ef20 con 0x7fc17813f9e0 2024-04-25T09:09:20.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.284+0000 7fc175600640 1 -- 172.21.15.62:0/820633846 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc170058cc0 con 0x7fc17813f9e0 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc15806b4d0 msgr2=0x7fc15806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc15806b4d0 0x7fc15806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00a780 tx=0x7fc16c009030 comp rx=0 tx=0).stop 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc17813f9e0 msgr2=0x7fc17813fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc17813f9e0 0x7fc17813fe60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc170006fd0 tx=0x7fc170010930 comp rx=0 tx=0).stop 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 shutdown_connections 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc15806b4d0 0x7fc15806d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc17813f9e0 0x7fc17813fe60 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1781617a0 0x7fc178161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 --2- 172.21.15.62:0/820633846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1600039e0 0x7fc17813ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.287+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 >> 172.21.15.62:0/820633846 conn(0x7fc17806abe0 msgr2=0x7fc1780adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:20.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.288+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 shutdown_connections 2024-04-25T09:09:20.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.288+0000 7fc180600640 1 -- 172.21.15.62:0/820633846 wait complete. 2024-04-25T09:09:20.295 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T09:09:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.515+0000 7fd6e0a00640 1 Processor -- start 2024-04-25T09:09:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.515+0000 7fd6e0a00640 1 -- start start 2024-04-25T09:09:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.516+0000 7fd6e0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6dc15d810 0x7fd6dc0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.517+0000 7fd6e0a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6dc06d900 0x7fd6dc1675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.517+0000 7fd6e0a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc0ced70 con 0x7fd6dc15dbf0 2024-04-25T09:09:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.517+0000 7fd6e0a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6dc0cf030 con 0x7fd6dc06d900 2024-04-25T09:09:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.517+0000 7fd6e0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0cf310 con 0x7fd6dc15d810 2024-04-25T09:09:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.517+0000 7fd6d3e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd6dc15dbf0 0x7fd6dc06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44518/0 (socket says 172.21.15.62:44518) 2024-04-25T09:09:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.517+0000 7fd6d3e00640 1 -- 172.21.15.62:0/1784795046 learned_addr learned my addr 172.21.15.62:0/1784795046 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.517+0000 7fd6da800640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6dc15d810 0x7fd6dc0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.517+0000 7fd6db200640 1 --2- 172.21.15.62:0/1784795046 >> v2:172.21.15.121:3300/0 conn(0x7fd6dc06d900 0x7fd6dc1675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048197458 0 0) 0x7fd6dc0ced70 con 0x7fd6dc15dbf0 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c0003520 con 0x7fd6dc15dbf0 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3698119303 0 0) 0x7fd6c0003520 con 0x7fd6dc15dbf0 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6dc15d810 msgr2=0x7fd6dc0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6dc15d810 0x7fd6dc0ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 >> v2:172.21.15.121:3300/0 conn(0x7fd6dc06d900 msgr2=0x7fd6dc1675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 --2- 172.21.15.62:0/1784795046 >> v2:172.21.15.121:3300/0 conn(0x7fd6dc06d900 0x7fd6dc1675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0c8cd0 con 0x7fd6dc15dbf0 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fd6c4001670 con 0x7fd6dc15dbf0 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 >> v1:172.21.15.62:6789/0 conn(0x7fd6dc15dbf0 legacy=0x7fd6dc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6c00039b0 0x7fd6c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c0004800 0x7fd6c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6dc15d810 0x7fd6c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0cf030 con 0x7fd6c00039b0 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0cf310 con 0x7fd6c0004800 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.518+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c000b9c0 con 0x7fd6dc15d810 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6da800640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6c00039b0 0x7fd6c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6db200640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6dc15d810 0x7fd6c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6d3e00640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c0004800 0x7fd6c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6da800640 1 -- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6dc15d810 msgr2=0x7fd6c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6da800640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6dc15d810 0x7fd6c00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6da800640 1 -- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c0004800 msgr2=0x7fd6c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6da800640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c0004800 0x7fd6c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6da800640 1 -- 172.21.15.62:0/1784795046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c0003520 con 0x7fd6c00039b0 2024-04-25T09:09:20.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.519+0000 7fd6d3e00640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6c0004800 0x7fd6c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.520+0000 7fd6da800640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6c00039b0 0x7fd6c0003dd0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009ac0 tx=0x7fd6c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfe5692d1d5b3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.520+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc036070 con 0x7fd6c00039b0 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.520+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc031400 con 0x7fd6c00039b0 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.521+0000 7fd6d9e00640 1 -- 172.21.15.62:0/1784795046 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc009690 con 0x7fd6c00039b0 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.521+0000 7fd6e0a00640 1 -- 172.21.15.62:0/1784795046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6c00039b0 msgr2=0x7fd6c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.521+0000 7fd6e0a00640 1 --2- 172.21.15.62:0/1784795046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6c00039b0 0x7fd6c0003dd0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009ac0 tx=0x7fd6c0005830 comp rx=0 tx=0).stop 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.521+0000 7fd6db200640 1 -- 172.21.15.62:0/1784795046 reap_dead start 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.521+0000 7fd6e0a00640 1 -- 172.21.15.62:0/1784795046 shutdown_connections 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.521+0000 7fd6e0a00640 1 -- 172.21.15.62:0/1784795046 >> 172.21.15.62:0/1784795046 conn(0x7fd6dc0b14d0 msgr2=0x7fd6dc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.521+0000 7fd6e0a00640 1 -- 172.21.15.62:0/1784795046 shutdown_connections 2024-04-25T09:09:20.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.521+0000 7fd6e0a00640 1 -- 172.21.15.62:0/1784795046 wait complete. 2024-04-25T09:09:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.523+0000 7fd6e0a00640 1 Processor -- start 2024-04-25T09:09:20.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.524+0000 7fd6e0a00640 1 -- start start 2024-04-25T09:09:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6e0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c00039b0 0x7fd6dc15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6e0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6dc148ae0 0x7fd6dc148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6e0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6dc15a900 0x7fd6dc0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6e0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0ca660 con 0x7fd6dc148ae0 2024-04-25T09:09:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6e0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0ca4e0 con 0x7fd6dc15a900 2024-04-25T09:09:20.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6e0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0ca7e0 con 0x7fd6c00039b0 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6d3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6dc148ae0 0x7fd6dc148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6d3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6dc148ae0 0x7fd6dc148190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51256/0 (socket says 172.21.15.62:51256) 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6d3e00640 1 -- 172.21.15.62:0/968191548 learned_addr learned my addr 172.21.15.62:0/968191548 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.525+0000 7fd6da800640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c00039b0 0x7fd6dc15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d3e00640 1 -- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c00039b0 msgr2=0x7fd6dc15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d3e00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c00039b0 0x7fd6dc15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d3e00640 1 -- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6dc15a900 msgr2=0x7fd6dc0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6db200640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6dc15a900 0x7fd6dc0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d3e00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6dc15a900 0x7fd6dc0b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d3e00640 1 -- 172.21.15.62:0/968191548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc15b0e0 con 0x7fd6dc148ae0 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d3e00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6dc148ae0 0x7fd6dc148190 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400d770 tx=0x7fd6c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d2a00640 1 -- 172.21.15.62:0/968191548 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4019070 con 0x7fd6dc148ae0 2024-04-25T09:09:20.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d2a00640 1 -- 172.21.15.62:0/968191548 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4007500 con 0x7fd6dc148ae0 2024-04-25T09:09:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6d2a00640 1 -- 172.21.15.62:0/968191548 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4004a20 con 0x7fd6dc148ae0 2024-04-25T09:09:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc15b3c0 con 0x7fd6dc148ae0 2024-04-25T09:09:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.526+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc154d60 con 0x7fd6dc148ae0 2024-04-25T09:09:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.528+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc04ef20 con 0x7fd6dc148ae0 2024-04-25T09:09:20.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.528+0000 7fd6d2a00640 1 -- 172.21.15.62:0/968191548 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6c401c070 con 0x7fd6dc148ae0 2024-04-25T09:09:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.530+0000 7fd6d2a00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6b806b4d0 0x7fd6b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.530+0000 7fd6d2a00640 1 -- 172.21.15.62:0/968191548 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd6c4089a40 con 0x7fd6dc148ae0 2024-04-25T09:09:20.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.531+0000 7fd6da800640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6b806b4d0 0x7fd6b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:20.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.532+0000 7fd6da800640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6b806b4d0 0x7fd6b806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc033000 tx=0x7fd6cc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:20.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.534+0000 7fd6d2a00640 1 -- 172.21.15.62:0/968191548 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6c400de20 con 0x7fd6dc148ae0 2024-04-25T09:09:20.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.809+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fd6dc06d070 con 0x7fd6dc148ae0 2024-04-25T09:09:20.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.833+0000 7fd6d2a00640 1 -- 172.21.15.62:0/968191548 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v294) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd6c4058ac0 con 0x7fd6dc148ae0 2024-04-25T09:09:20.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6b806b4d0 msgr2=0x7fd6b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6b806b4d0 0x7fd6b806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc033000 tx=0x7fd6cc008030 comp rx=0 tx=0).stop 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6dc148ae0 msgr2=0x7fd6dc148190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6dc148ae0 0x7fd6dc148190 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400d770 tx=0x7fd6c400dc30 comp rx=0 tx=0).stop 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 shutdown_connections 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6b806b4d0 0x7fd6b806d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6dc148ae0 0x7fd6dc148190 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6dc15a900 0x7fd6dc0b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 --2- 172.21.15.62:0/968191548 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6c00039b0 0x7fd6dc15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.836+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 >> 172.21.15.62:0/968191548 conn(0x7fd6dc0b14d0 msgr2=0x7fd6dc0cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.837+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 shutdown_connections 2024-04-25T09:09:20.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:20.837+0000 7fd6e0a00640 1 -- 172.21.15.62:0/968191548 wait complete. 2024-04-25T09:09:20.846 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T09:09:20.846 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T09:09:20.846 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:09:20.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:21.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.076+0000 7f0883e00640 1 Processor -- start 2024-04-25T09:09:21.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.076+0000 7f0883e00640 1 -- start start 2024-04-25T09:09:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.078+0000 7f0883e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f088415b900 0x7f08840d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.078+0000 7f0883e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08840b6900 0x7f08840b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.078+0000 7f0883e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840cf890 con 0x7f08840d1cc0 2024-04-25T09:09:21.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.078+0000 7f0883e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0884070a00 con 0x7f088415b900 2024-04-25T09:09:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.078+0000 7f0883e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088406fdf0 con 0x7f08840b6900 2024-04-25T09:09:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.078+0000 7f0882000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f08840d1cc0 0x7f08840b61f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44530/0 (socket says 172.21.15.62:44530) 2024-04-25T09:09:21.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.078+0000 7f0882000640 1 -- 172.21.15.62:0/63574091 learned_addr learned my addr 172.21.15.62:0/63574091 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:21.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.079+0000 7f0882a00640 1 --2- 172.21.15.62:0/63574091 >> v2:172.21.15.121:3300/0 conn(0x7f088415b900 0x7f08840d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.079+0000 7f0883400640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08840b6900 0x7f08840b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.079+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184881095 0 0) 0x7f086c001110 con 0x7f08840d1cc0 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.079+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0868003520 con 0x7f08840d1cc0 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.079+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 105596858 0 0) 0x7f0868003520 con 0x7f08840d1cc0 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.079+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08840b6900 msgr2=0x7f08840b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.079+0000 7f0881600640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08840b6900 0x7f08840b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 >> v2:172.21.15.121:3300/0 conn(0x7f088415b900 msgr2=0x7f08840d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 --2- 172.21.15.62:0/63574091 >> v2:172.21.15.121:3300/0 conn(0x7f088415b900 0x7f08840d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840700d0 con 0x7f08840d1cc0 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f086c001840 con 0x7f08840d1cc0 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 >> v1:172.21.15.62:6789/0 conn(0x7f08840d1cc0 legacy=0x7f08840b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08840b6900 0x7f08680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0868003f30 0x7f0868004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0868004d40 0x7f08680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0884070a00 con 0x7f0868003f30 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088406fdf0 con 0x7f0868004d40 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086800b9c0 con 0x7f08840b6900 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0882000640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0868003f30 0x7f0868004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0882a00640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08840b6900 0x7f08680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0882000640 1 -- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08840b6900 msgr2=0x7f08680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0882000640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08840b6900 0x7f08680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0883400640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0868004d40 0x7f08680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0882000640 1 -- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0868004d40 msgr2=0x7f08680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.080+0000 7f0882000640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0868004d40 0x7f08680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.081+0000 7f0882000640 1 -- 172.21.15.62:0/63574091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868003520 con 0x7f0868003f30 2024-04-25T09:09:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.082+0000 7f0882000640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0868003f30 0x7f0868004800 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c4c0 tx=0x7f0868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8b02434c59f137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.082+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00c0e0 con 0x7f0868003f30 2024-04-25T09:09:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.082+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c034070 con 0x7f0868003f30 2024-04-25T09:09:21.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.082+0000 7f0881600640 1 -- 172.21.15.62:0/63574091 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c005d70 con 0x7f0868003f30 2024-04-25T09:09:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.082+0000 7f0883e00640 1 -- 172.21.15.62:0/63574091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0868003f30 msgr2=0x7f0868004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.082+0000 7f0883e00640 1 --2- 172.21.15.62:0/63574091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0868003f30 0x7f0868004800 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c4c0 tx=0x7f0868005830 comp rx=0 tx=0).stop 2024-04-25T09:09:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.083+0000 7f0883400640 1 -- 172.21.15.62:0/63574091 reap_dead start 2024-04-25T09:09:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.083+0000 7f0883e00640 1 -- 172.21.15.62:0/63574091 shutdown_connections 2024-04-25T09:09:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.083+0000 7f0883e00640 1 -- 172.21.15.62:0/63574091 >> 172.21.15.62:0/63574091 conn(0x7f0884072340 msgr2=0x7f0884072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.083+0000 7f0883e00640 1 -- 172.21.15.62:0/63574091 shutdown_connections 2024-04-25T09:09:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.083+0000 7f0883e00640 1 -- 172.21.15.62:0/63574091 wait complete. 2024-04-25T09:09:21.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.085+0000 7f0883e00640 1 Processor -- start 2024-04-25T09:09:21.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.085+0000 7f0883e00640 1 -- start start 2024-04-25T09:09:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0868003f30 0x7f08840cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08840c71e0 0x7f0884154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0884152f90 0x7f0884153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088415bce0 con 0x7f0884152f90 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0884070690 con 0x7f08840c71e0 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088415be60 con 0x7f0868003f30 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0884152f90 0x7f0884153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0884152f90 0x7f0884153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51278/0 (socket says 172.21.15.62:51278) 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0883400640 1 -- 172.21.15.62:0/2055696213 learned_addr learned my addr 172.21.15.62:0/2055696213 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0882a00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0868003f30 0x7f08840cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.086+0000 7f0882000640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08840c71e0 0x7f0884154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0883400640 1 -- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0868003f30 msgr2=0x7f08840cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0883400640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0868003f30 0x7f08840cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0883400640 1 -- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08840c71e0 msgr2=0x7f0884154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0883400640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08840c71e0 0x7f0884154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0883400640 1 -- 172.21.15.62:0/2055696213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884157800 con 0x7f0884152f90 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0882000640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08840c71e0 0x7f0884154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:21.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0883400640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0884152f90 0x7f0884153410 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f08840b6d30 tx=0x7f0874010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f087b400640 1 -- 172.21.15.62:0/2055696213 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087400f040 con 0x7f0884152f90 2024-04-25T09:09:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f087b400640 1 -- 172.21.15.62:0/2055696213 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087401c070 con 0x7f0884152f90 2024-04-25T09:09:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f087b400640 1 -- 172.21.15.62:0/2055696213 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087400eb20 con 0x7f0884152f90 2024-04-25T09:09:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0884157ae0 con 0x7f0884152f90 2024-04-25T09:09:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.087+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08841580a0 con 0x7f0884152f90 2024-04-25T09:09:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.089+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088404ef20 con 0x7f0884152f90 2024-04-25T09:09:21.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.089+0000 7f087b400640 1 -- 172.21.15.62:0/2055696213 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f087400ecc0 con 0x7f0884152f90 2024-04-25T09:09:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.090+0000 7f087b400640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f086006b500 0x7f086006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.090+0000 7f087b400640 1 -- 172.21.15.62:0/2055696213 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0874089ad0 con 0x7f0884152f90 2024-04-25T09:09:21.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.091+0000 7f0882a00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f086006b500 0x7f086006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.092+0000 7f0882a00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f086006b500 0x7f086006d9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08840b0740 tx=0x7f087c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.095+0000 7f087b400640 1 -- 172.21.15.62:0/2055696213 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0874026020 con 0x7f0884152f90 2024-04-25T09:09:21.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.298+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0884156ae0 con 0x7f0884152f90 2024-04-25T09:09:21.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.300+0000 7f087b400640 1 -- 172.21.15.62:0/2055696213 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+13136 (secure 0 0 0) 0x7f0874058b50 con 0x7f0884152f90 2024-04-25T09:09:21.289 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:21.289 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:19.817037+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6832/836333339":"2024-04-26T09:09:16.742536+0000","172.21.15.62:6833/836333339":"2024-04-26T09:09:16.742536+0000","172.21.15.62:6834/3491368038":"2024-04-26T09:08:50.613426+0000","172.21.15.62:6832/3491368038":"2024-04-26T09:08:50.613426+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:21.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f086006b500 msgr2=0x7f086006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f086006b500 0x7f086006d9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f08840b0740 tx=0x7f087c00b040 comp rx=0 tx=0).stop 2024-04-25T09:09:21.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0884152f90 msgr2=0x7f0884153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0884152f90 0x7f0884153410 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f08840b6d30 tx=0x7f0874010c20 comp rx=0 tx=0).stop 2024-04-25T09:09:21.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 shutdown_connections 2024-04-25T09:09:21.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f086006b500 0x7f086006d9c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0884152f90 0x7f0884153410 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08840c71e0 0x7f0884154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 --2- 172.21.15.62:0/2055696213 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0868003f30 0x7f08840cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 >> 172.21.15.62:0/2055696213 conn(0x7f0884072340 msgr2=0x7f08840b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:21.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 shutdown_connections 2024-04-25T09:09:21.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.303+0000 7f0883e00640 1 -- 172.21.15.62:0/2055696213 wait complete. 2024-04-25T09:09:21.310 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:21.310 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.528+0000 7f134ec00640 1 Processor -- start 2024-04-25T09:09:21.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.529+0000 7f134ec00640 1 -- start start 2024-04-25T09:09:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.530+0000 7f134ec00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13480ceb50 0x7f134806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.530+0000 7f134ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134806d800 0x7f134815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.530+0000 7f134ec00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f134815e5f0 0x7f13480ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.530+0000 7f134ec00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134806dc60 con 0x7f134815e5f0 2024-04-25T09:09:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.530+0000 7f134ec00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134815ee30 con 0x7f13480ceb50 2024-04-25T09:09:21.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.530+0000 7f134ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13481618b0 con 0x7f134806d800 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.531+0000 7f134e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f134815e5f0 0x7f13480ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.531+0000 7f134ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134806d800 0x7f134815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.531+0000 7f134e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f134815e5f0 0x7f13480ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51286/0 (socket says 172.21.15.62:51286) 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.531+0000 7f134e200640 1 -- 172.21.15.62:0/2181204145 learned_addr learned my addr 172.21.15.62:0/2181204145 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.531+0000 7f134d800640 1 --2- 172.21.15.62:0/2181204145 >> v2:172.21.15.121:3300/0 conn(0x7f13480ceb50 0x7f134806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.532+0000 7f134e200640 1 -- 172.21.15.62:0/2181204145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134806d800 msgr2=0x7f134815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.532+0000 7f134e200640 1 --2- 172.21.15.62:0/2181204145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134806d800 0x7f134815e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.532+0000 7f134e200640 1 -- 172.21.15.62:0/2181204145 >> v2:172.21.15.121:3300/0 conn(0x7f13480ceb50 msgr2=0x7f134806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.532+0000 7f134e200640 1 --2- 172.21.15.62:0/2181204145 >> v2:172.21.15.121:3300/0 conn(0x7f13480ceb50 0x7f134806d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.532+0000 7f134e200640 1 -- 172.21.15.62:0/2181204145 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1348161b90 con 0x7f134815e5f0 2024-04-25T09:09:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.533+0000 7f134e200640 1 --2- 172.21.15.62:0/2181204145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134815e5f0 0x7f13480ad8e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f13480af200 tx=0x7f134002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b569aa3f02ac3a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.533+0000 7f1347e00640 1 -- 172.21.15.62:0/2181204145 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340036070 con 0x7f134815e5f0 2024-04-25T09:09:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.533+0000 7f1347e00640 1 -- 172.21.15.62:0/2181204145 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1340031600 con 0x7f134815e5f0 2024-04-25T09:09:21.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.533+0000 7f1347e00640 1 -- 172.21.15.62:0/2181204145 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400319d0 con 0x7f134815e5f0 2024-04-25T09:09:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 -- 172.21.15.62:0/2181204145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134815e5f0 msgr2=0x7f13480ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 --2- 172.21.15.62:0/2181204145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134815e5f0 0x7f13480ad8e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f13480af200 tx=0x7f134002a610 comp rx=0 tx=0).stop 2024-04-25T09:09:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 -- 172.21.15.62:0/2181204145 shutdown_connections 2024-04-25T09:09:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 --2- 172.21.15.62:0/2181204145 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134815e5f0 0x7f13480ad8e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 --2- 172.21.15.62:0/2181204145 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134806d800 0x7f134815e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 --2- 172.21.15.62:0/2181204145 >> v2:172.21.15.121:3300/0 conn(0x7f13480ceb50 0x7f134806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 -- 172.21.15.62:0/2181204145 >> 172.21.15.62:0/2181204145 conn(0x7f1348066f20 msgr2=0x7f13480b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:21.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 -- 172.21.15.62:0/2181204145 shutdown_connections 2024-04-25T09:09:21.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.534+0000 7f134ec00640 1 -- 172.21.15.62:0/2181204145 wait complete. 2024-04-25T09:09:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.536+0000 7f134ec00640 1 Processor -- start 2024-04-25T09:09:21.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.536+0000 7f134ec00640 1 -- start start 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134806d800 0x7f134813bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134ec00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13480ceb50 0x7f134813f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134815e5f0 0x7f134813b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480cf0b0 con 0x7f134806d800 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134ec00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480cef30 con 0x7f13480ceb50 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1348162390 con 0x7f134815e5f0 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134806d800 0x7f134813bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134806d800 0x7f134813bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51302/0 (socket says 172.21.15.62:51302) 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.537+0000 7f134d800640 1 -- 172.21.15.62:0/477246116 learned_addr learned my addr 172.21.15.62:0/477246116 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.538+0000 7f134d800640 1 -- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134815e5f0 msgr2=0x7f134813b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.538+0000 7f134d800640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134815e5f0 0x7f134813b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.538+0000 7f134d800640 1 -- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13480ceb50 msgr2=0x7f134813f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.538+0000 7f134d800640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13480ceb50 0x7f134813f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.538+0000 7f134d800640 1 -- 172.21.15.62:0/477246116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134813fb70 con 0x7f134806d800 2024-04-25T09:09:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.538+0000 7f134d800640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134806d800 0x7f134813bca0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f133c00e9b0 tx=0x7f133c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.538+0000 7f1345600640 1 -- 172.21.15.62:0/477246116 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c017070 con 0x7f134806d800 2024-04-25T09:09:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.538+0000 7f1345600640 1 -- 172.21.15.62:0/477246116 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c00c9a0 con 0x7f134806d800 2024-04-25T09:09:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.539+0000 7f1345600640 1 -- 172.21.15.62:0/477246116 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c00cd90 con 0x7f134806d800 2024-04-25T09:09:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.539+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1348156810 con 0x7f134806d800 2024-04-25T09:09:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.539+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1348156dd0 con 0x7f134806d800 2024-04-25T09:09:21.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.540+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134806abe0 con 0x7f134806d800 2024-04-25T09:09:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.541+0000 7f1345600640 1 -- 172.21.15.62:0/477246116 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f133c01a070 con 0x7f134806d800 2024-04-25T09:09:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.542+0000 7f1345600640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.542+0000 7f1345600640 1 -- 172.21.15.62:0/477246116 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f133c089fb0 con 0x7f134806d800 2024-04-25T09:09:21.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.543+0000 7f134ce00640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.544+0000 7f134ce00640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f132806b4d0 0x7f132806d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f13481550b0 tx=0x7f1334009500 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.546+0000 7f1345600640 1 -- 172.21.15.62:0/477246116 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f133c0590c0 con 0x7f134806d800 2024-04-25T09:09:21.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.747+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f134804ef20 con 0x7f134806d800 2024-04-25T09:09:21.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.748+0000 7f1345600640 1 -- 172.21.15.62:0/477246116 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f133c0589c0 con 0x7f134806d800 2024-04-25T09:09:21.736 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f132806b4d0 msgr2=0x7f132806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f132806b4d0 0x7f132806d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f13481550b0 tx=0x7f1334009500 comp rx=0 tx=0).stop 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134806d800 msgr2=0x7f134813bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134806d800 0x7f134813bca0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f133c00e9b0 tx=0x7f133c00ee70 comp rx=0 tx=0).stop 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 shutdown_connections 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f134815e5f0 0x7f134813b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13480ceb50 0x7f134813f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 --2- 172.21.15.62:0/477246116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f134806d800 0x7f134813bca0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.751+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 >> 172.21.15.62:0/477246116 conn(0x7f1348066f20 msgr2=0x7f134815f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.752+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 shutdown_connections 2024-04-25T09:09:21.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.752+0000 7f134ec00640 1 -- 172.21.15.62:0/477246116 wait complete. 2024-04-25T09:09:21.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T09:09:21.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.980+0000 7fb059600640 1 Processor -- start 2024-04-25T09:09:21.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.980+0000 7fb059600640 1 -- start start 2024-04-25T09:09:21.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.982+0000 7fb059600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb05415dc40 0x7fb0540d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.982+0000 7fb059600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb0540d0f70 0x7fb0540d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.982+0000 7fb059600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540d1400 con 0x7fb0540d0ae0 2024-04-25T09:09:21.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.982+0000 7fb059600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540d2620 con 0x7fb05415dc40 2024-04-25T09:09:21.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.982+0000 7fb059600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540d2900 con 0x7fb0540d0f70 2024-04-25T09:09:21.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.982+0000 7fb052a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb05415dc40 0x7fb0540d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.982+0000 7fb052a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb05415dc40 0x7fb0540d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51314/0 (socket says 172.21.15.62:51314) 2024-04-25T09:09:21.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.982+0000 7fb052a00640 1 -- 172.21.15.62:0/2692195722 learned_addr learned my addr 172.21.15.62:0/2692195722 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:21.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.983+0000 7fb052a00640 1 -- 172.21.15.62:0/2692195722 >> v2:172.21.15.121:3300/0 conn(0x7fb0540d0f70 msgr2=0x7fb0540d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:21.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.983+0000 7fb052a00640 1 --2- 172.21.15.62:0/2692195722 >> v2:172.21.15.121:3300/0 conn(0x7fb0540d0f70 0x7fb0540d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.983+0000 7fb052a00640 1 -- 172.21.15.62:0/2692195722 >> v1:172.21.15.121:6789/0 conn(0x7fb0540d0ae0 legacy=0x7fb0540d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.983+0000 7fb052a00640 1 -- 172.21.15.62:0/2692195722 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540d2be0 con 0x7fb05415dc40 2024-04-25T09:09:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.984+0000 7fb052a00640 1 --2- 172.21.15.62:0/2692195722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb05415dc40 0x7fb0540d1cb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb044009870 tx=0x7fb04402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a31cc4c8df196ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.984+0000 7fb051600640 1 -- 172.21.15.62:0/2692195722 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044037070 con 0x7fb05415dc40 2024-04-25T09:09:21.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.984+0000 7fb051600640 1 -- 172.21.15.62:0/2692195722 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb044032440 con 0x7fb05415dc40 2024-04-25T09:09:21.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.984+0000 7fb051600640 1 -- 172.21.15.62:0/2692195722 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044032810 con 0x7fb05415dc40 2024-04-25T09:09:21.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.985+0000 7fb059600640 1 -- 172.21.15.62:0/2692195722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb05415dc40 msgr2=0x7fb0540d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.985+0000 7fb059600640 1 --2- 172.21.15.62:0/2692195722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb05415dc40 0x7fb0540d1cb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb044009870 tx=0x7fb04402a270 comp rx=0 tx=0).stop 2024-04-25T09:09:21.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.985+0000 7fb059600640 1 -- 172.21.15.62:0/2692195722 shutdown_connections 2024-04-25T09:09:21.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.985+0000 7fb059600640 1 --2- 172.21.15.62:0/2692195722 >> v2:172.21.15.121:3300/0 conn(0x7fb0540d0f70 0x7fb0540d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.985+0000 7fb059600640 1 --2- 172.21.15.62:0/2692195722 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb05415dc40 0x7fb0540d1cb0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.985+0000 7fb059600640 1 -- 172.21.15.62:0/2692195722 >> 172.21.15.62:0/2692195722 conn(0x7fb05406b000 msgr2=0x7fb05406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:21.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.985+0000 7fb059600640 1 -- 172.21.15.62:0/2692195722 shutdown_connections 2024-04-25T09:09:21.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.985+0000 7fb059600640 1 -- 172.21.15.62:0/2692195722 wait complete. 2024-04-25T09:09:21.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.987+0000 7fb059600640 1 Processor -- start 2024-04-25T09:09:21.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.987+0000 7fb059600640 1 -- start start 2024-04-25T09:09:21.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.988+0000 7fb059600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0540d0ae0 0x7fb054160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.988+0000 7fb059600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540d0f70 0x7fb054156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.988+0000 7fb059600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb05415dc40 0x7fb0540c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.988+0000 7fb059600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b7d00 con 0x7fb0540d0ae0 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.988+0000 7fb059600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540d31a0 con 0x7fb05415dc40 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.988+0000 7fb059600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540d36c0 con 0x7fb0540d0f70 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0540d0ae0 0x7fb054160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0540d0ae0 0x7fb054160d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51324/0 (socket says 172.21.15.62:51324) 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052a00640 1 -- 172.21.15.62:0/1288104226 learned_addr learned my addr 172.21.15.62:0/1288104226 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb053400640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb05415dc40 0x7fb0540c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052000640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540d0f70 0x7fb054156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052a00640 1 -- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540d0f70 msgr2=0x7fb054156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052a00640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540d0f70 0x7fb054156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052a00640 1 -- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb05415dc40 msgr2=0x7fb0540c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:21.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052a00640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb05415dc40 0x7fb0540c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:21.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052a00640 1 -- 172.21.15.62:0/1288104226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0541f89e0 con 0x7fb0540d0ae0 2024-04-25T09:09:21.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb052000640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540d0f70 0x7fb054156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:21.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.989+0000 7fb053400640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb05415dc40 0x7fb0540c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:21.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.990+0000 7fb052a00640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0540d0ae0 0x7fb054160d20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb044009870 tx=0x7fb0440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.990+0000 7fb04aa00640 1 -- 172.21.15.62:0/1288104226 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044037040 con 0x7fb0540d0ae0 2024-04-25T09:09:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.990+0000 7fb04aa00640 1 -- 172.21.15.62:0/1288104226 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb044004230 con 0x7fb0540d0ae0 2024-04-25T09:09:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.990+0000 7fb04aa00640 1 -- 172.21.15.62:0/1288104226 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04403c670 con 0x7fb0540d0ae0 2024-04-25T09:09:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.990+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05415ff60 con 0x7fb0540d0ae0 2024-04-25T09:09:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.990+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb054160520 con 0x7fb0540d0ae0 2024-04-25T09:09:21.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.991+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0540729c0 con 0x7fb0540d0ae0 2024-04-25T09:09:21.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.992+0000 7fb04aa00640 1 -- 172.21.15.62:0/1288104226 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb04403a070 con 0x7fb0540d0ae0 2024-04-25T09:09:21.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.993+0000 7fb04aa00640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb03006b4d0 0x7fb03006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:21.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.993+0000 7fb04aa00640 1 -- 172.21.15.62:0/1288104226 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb0440aa8d0 con 0x7fb0540d0ae0 2024-04-25T09:09:21.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.994+0000 7fb052000640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb03006b4d0 0x7fb03006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:21.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.995+0000 7fb052000640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb03006b4d0 0x7fb03006d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb0541e7520 tx=0x7fb03c0073a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:21.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:21.998+0000 7fb04aa00640 1 -- 172.21.15.62:0/1288104226 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb044041020 con 0x7fb0540d0ae0 2024-04-25T09:09:22.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.275+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7fb0540d3b90 con 0x7fb0540d0ae0 2024-04-25T09:09:22.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.849+0000 7fb04aa00640 1 -- 172.21.15.62:0/1288104226 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v296) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb044079950 con 0x7fb0540d0ae0 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb03006b4d0 msgr2=0x7fb03006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb03006b4d0 0x7fb03006d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb0541e7520 tx=0x7fb03c0073a0 comp rx=0 tx=0).stop 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0540d0ae0 msgr2=0x7fb054160d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0540d0ae0 0x7fb054160d20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb044009870 tx=0x7fb0440046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 shutdown_connections 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb03006b4d0 0x7fb03006d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb05415dc40 0x7fb0540c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0540d0f70 0x7fb054156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.852+0000 7fb059600640 1 --2- 172.21.15.62:0/1288104226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0540d0ae0 0x7fb054160d20 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.853+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 >> 172.21.15.62:0/1288104226 conn(0x7fb05406b000 msgr2=0x7fb0540668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.853+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 shutdown_connections 2024-04-25T09:09:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:22.853+0000 7fb059600640 1 -- 172.21.15.62:0/1288104226 wait complete. 2024-04-25T09:09:22.862 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:09:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.086+0000 7f0d7f200640 1 Processor -- start 2024-04-25T09:09:23.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.086+0000 7f0d7f200640 1 -- start start 2024-04-25T09:09:23.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.088+0000 7f0d7f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d800ccde0 0x7f0d800ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.088+0000 7f0d7f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0d800b31a0 0x7f0d800b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.088+0000 7f0d7f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800ced40 con 0x7f0d8006d2c0 2024-04-25T09:09:23.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.088+0000 7f0d7f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800cd260 con 0x7f0d800b31a0 2024-04-25T09:09:23.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.088+0000 7f0d7f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800cf000 con 0x7f0d800ccde0 2024-04-25T09:09:23.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.088+0000 7f0d7de00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0d8006d2c0 0x7f0d8006d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44546/0 (socket says 172.21.15.62:44546) 2024-04-25T09:09:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.088+0000 7f0d7de00640 1 -- 172.21.15.62:0/1852260013 learned_addr learned my addr 172.21.15.62:0/1852260013 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.088+0000 7f0d7e800640 1 --2- 172.21.15.62:0/1852260013 >> v2:172.21.15.62:3300/0 conn(0x7f0d800b31a0 0x7f0d800b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.089+0000 7f0d7d400640 1 --2- 172.21.15.62:0/1852260013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d800ccde0 0x7f0d800ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.089+0000 7f0d7ca00640 1 -- 172.21.15.62:0/1852260013 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988971359 0 0) 0x7f0d800ced40 con 0x7f0d8006d2c0 2024-04-25T09:09:23.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.089+0000 7f0d7ca00640 1 -- 172.21.15.62:0/1852260013 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d64003520 con 0x7f0d8006d2c0 2024-04-25T09:09:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.089+0000 7f0d7e800640 1 -- 172.21.15.62:0/1852260013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d800ccde0 msgr2=0x7f0d800ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.089+0000 7f0d7e800640 1 --2- 172.21.15.62:0/1852260013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d800ccde0 0x7f0d800ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.089+0000 7f0d7e800640 1 -- 172.21.15.62:0/1852260013 >> v1:172.21.15.62:6789/0 conn(0x7f0d8006d2c0 legacy=0x7f0d8006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.089+0000 7f0d7e800640 1 -- 172.21.15.62:0/1852260013 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800cf2e0 con 0x7f0d800b31a0 2024-04-25T09:09:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.090+0000 7f0d7e800640 1 --2- 172.21.15.62:0/1852260013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d800b3f40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0d780097a0 tx=0x7f0d7802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae5a2100852dc1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.090+0000 7f0d7ca00640 1 -- 172.21.15.62:0/1852260013 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78037070 con 0x7f0d800b31a0 2024-04-25T09:09:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.090+0000 7f0d7ca00640 1 -- 172.21.15.62:0/1852260013 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d78032440 con 0x7f0d800b31a0 2024-04-25T09:09:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.090+0000 7f0d7ca00640 1 -- 172.21.15.62:0/1852260013 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78032810 con 0x7f0d800b31a0 2024-04-25T09:09:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.091+0000 7f0d7f200640 1 -- 172.21.15.62:0/1852260013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 msgr2=0x7f0d800b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.091+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1852260013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d800b3f40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0d780097a0 tx=0x7f0d7802a180 comp rx=0 tx=0).stop 2024-04-25T09:09:23.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.091+0000 7f0d7f200640 1 -- 172.21.15.62:0/1852260013 shutdown_connections 2024-04-25T09:09:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.091+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1852260013 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d800b3f40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.091+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1852260013 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d800ccde0 0x7f0d800ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.091+0000 7f0d7f200640 1 -- 172.21.15.62:0/1852260013 >> 172.21.15.62:0/1852260013 conn(0x7f0d80066f20 msgr2=0x7f0d800b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.091+0000 7f0d7f200640 1 -- 172.21.15.62:0/1852260013 shutdown_connections 2024-04-25T09:09:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.091+0000 7f0d7f200640 1 -- 172.21.15.62:0/1852260013 wait complete. 2024-04-25T09:09:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.092+0000 7f0d7f200640 1 Processor -- start 2024-04-25T09:09:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.093+0000 7f0d7f200640 1 -- start start 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.093+0000 7f0d7f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d8006d2c0 0x7f0d800c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.093+0000 7f0d7f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d8015b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.093+0000 7f0d7f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d800ccde0 0x7f0d800c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.093+0000 7f0d7f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b2c80 con 0x7f0d800b31a0 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.093+0000 7f0d7f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b29a0 con 0x7f0d800ccde0 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.093+0000 7f0d7f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d801448d0 con 0x7f0d8006d2c0 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d8015b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d8015b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51342/0 (socket says 172.21.15.62:51342) 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7d400640 1 -- 172.21.15.62:0/1337032301 learned_addr learned my addr 172.21.15.62:0/1337032301 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7de00640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d8006d2c0 0x7f0d800c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7d400640 1 -- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d8006d2c0 msgr2=0x7f0d800c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7d400640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d8006d2c0 0x7f0d800c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7d400640 1 -- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d800ccde0 msgr2=0x7f0d800c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7d400640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d800ccde0 0x7f0d800c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.094+0000 7f0d7d400640 1 -- 172.21.15.62:0/1337032301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800c4cf0 con 0x7f0d800b31a0 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.095+0000 7f0d7d400640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d8015b260 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800d800 tx=0x7f0d6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.095+0000 7f0d76000640 1 -- 172.21.15.62:0/1337032301 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800b6c0 con 0x7f0d800b31a0 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.095+0000 7f0d76000640 1 -- 172.21.15.62:0/1337032301 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d68019070 con 0x7f0d800b31a0 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.095+0000 7f0d76000640 1 -- 172.21.15.62:0/1337032301 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800ba80 con 0x7f0d800b31a0 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.095+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d800c4fd0 con 0x7f0d800b31a0 2024-04-25T09:09:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.095+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d800c5590 con 0x7f0d800b31a0 2024-04-25T09:09:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.096+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8006abe0 con 0x7f0d800b31a0 2024-04-25T09:09:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.097+0000 7f0d76000640 1 -- 172.21.15.62:0/1337032301 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d6800bc20 con 0x7f0d800b31a0 2024-04-25T09:09:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.098+0000 7f0d76000640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0d5c06b4d0 0x7f0d5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.099+0000 7f0d76000640 1 -- 172.21.15.62:0/1337032301 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0d6808a4e0 con 0x7f0d800b31a0 2024-04-25T09:09:23.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.099+0000 7f0d7de00640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0d5c06b4d0 0x7f0d5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.100+0000 7f0d7de00640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0d5c06b4d0 0x7f0d5c06d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0d700012e0 tx=0x7f0d70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:23.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.103+0000 7f0d76000640 1 -- 172.21.15.62:0/1337032301 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d68059560 con 0x7f0d800b31a0 2024-04-25T09:09:23.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.297+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0d8004ef20 con 0x7f0d800b31a0 2024-04-25T09:09:23.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.304+0000 7f0d76000640 1 -- 172.21.15.62:0/1337032301 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0d68058e60 con 0x7f0d800b31a0 2024-04-25T09:09:23.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0d5c06b4d0 msgr2=0x7f0d5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0d5c06b4d0 0x7f0d5c06d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0d700012e0 tx=0x7f0d70009040 comp rx=0 tx=0).stop 2024-04-25T09:09:23.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 msgr2=0x7f0d8015b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d8015b260 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800d800 tx=0x7f0d6800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 shutdown_connections 2024-04-25T09:09:23.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0d5c06b4d0 0x7f0d5c06d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d800b31a0 0x7f0d8015b260 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d800ccde0 0x7f0d800c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 --2- 172.21.15.62:0/1337032301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0d8006d2c0 0x7f0d800c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 >> 172.21.15.62:0/1337032301 conn(0x7f0d80066f20 msgr2=0x7f0d800b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:23.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 shutdown_connections 2024-04-25T09:09:23.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.307+0000 7f0d7f200640 1 -- 172.21.15.62:0/1337032301 wait complete. 2024-04-25T09:09:23.313 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:09:23.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:23.416 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:09:23.531 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:09:23.532 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:09:23.532 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:23.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:09:23.592 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:09:23.592 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:23.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:23.649 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:09:23.649 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:23.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:09:23.708 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:09:23.708 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:23.764 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:09:23.764 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:09:23.764 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.995+0000 7f6239800640 1 Processor -- start 2024-04-25T09:09:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.996+0000 7f6239800640 1 -- start start 2024-04-25T09:09:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6239800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b56f0 0x7f6234160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6239800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f62340b6a90 0x7f6234161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6239800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62340b27a0 0x7f6234161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6239800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6234161780 con 0x7f62340b27a0 2024-04-25T09:09:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6239800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340cedc0 con 0x7f62340b6a90 2024-04-25T09:09:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6239800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62340d3110 con 0x7f62340b56f0 2024-04-25T09:09:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6238e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62340b27a0 0x7f6234161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6238e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62340b27a0 0x7f6234161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51352/0 (socket says 172.21.15.62:51352) 2024-04-25T09:09:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.997+0000 7f6238e00640 1 -- 172.21.15.62:0/4187126579 learned_addr learned my addr 172.21.15.62:0/4187126579 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.998+0000 7f6233400640 1 --2- 172.21.15.62:0/4187126579 >> v2:172.21.15.121:3300/0 conn(0x7f62340b6a90 0x7f6234161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.998+0000 7f6233e00640 1 --2- 172.21.15.62:0/4187126579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b56f0 0x7f6234160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.998+0000 7f6238e00640 1 -- 172.21.15.62:0/4187126579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b56f0 msgr2=0x7f6234160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.998+0000 7f6238e00640 1 --2- 172.21.15.62:0/4187126579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b56f0 0x7f6234160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.998+0000 7f6238e00640 1 -- 172.21.15.62:0/4187126579 >> v2:172.21.15.121:3300/0 conn(0x7f62340b6a90 msgr2=0x7f6234161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.998+0000 7f6238e00640 1 --2- 172.21.15.62:0/4187126579 >> v2:172.21.15.121:3300/0 conn(0x7f62340b6a90 0x7f6234161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.998+0000 7f6238e00640 1 -- 172.21.15.62:0/4187126579 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340d33f0 con 0x7f62340b27a0 2024-04-25T09:09:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.999+0000 7f6238e00640 1 --2- 172.21.15.62:0/4187126579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b27a0 0x7f6234161980 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f62340728e0 tx=0x7f622c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0695d6349415959 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.999+0000 7f6232a00640 1 -- 172.21.15.62:0/4187126579 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c036070 con 0x7f62340b27a0 2024-04-25T09:09:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.999+0000 7f6232a00640 1 -- 172.21.15.62:0/4187126579 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c031600 con 0x7f62340b27a0 2024-04-25T09:09:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:23.999+0000 7f6232a00640 1 -- 172.21.15.62:0/4187126579 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0319d0 con 0x7f62340b27a0 2024-04-25T09:09:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 -- 172.21.15.62:0/4187126579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b27a0 msgr2=0x7f6234161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 --2- 172.21.15.62:0/4187126579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b27a0 0x7f6234161980 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f62340728e0 tx=0x7f622c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 -- 172.21.15.62:0/4187126579 shutdown_connections 2024-04-25T09:09:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 --2- 172.21.15.62:0/4187126579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b27a0 0x7f6234161980 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 --2- 172.21.15.62:0/4187126579 >> v2:172.21.15.121:3300/0 conn(0x7f62340b6a90 0x7f6234161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 --2- 172.21.15.62:0/4187126579 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b56f0 0x7f6234160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 -- 172.21.15.62:0/4187126579 >> 172.21.15.62:0/4187126579 conn(0x7f6234066d10 msgr2=0x7f6234067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 -- 172.21.15.62:0/4187126579 shutdown_connections 2024-04-25T09:09:23.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.000+0000 7f6239800640 1 -- 172.21.15.62:0/4187126579 wait complete. 2024-04-25T09:09:23.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.002+0000 7f6239800640 1 Processor -- start 2024-04-25T09:09:23.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.002+0000 7f6239800640 1 -- start start 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6239800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62340b27a0 0x7f6234152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6239800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b56f0 0x7f6234150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6239800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b6a90 0x7f62341527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6239800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340b2d60 con 0x7f62340b56f0 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6239800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340cea50 con 0x7f62340b27a0 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6239800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62340b2ee0 con 0x7f62340b6a90 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6233400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b56f0 0x7f6234150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6233400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b56f0 0x7f6234150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51364/0 (socket says 172.21.15.62:51364) 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.003+0000 7f6233400640 1 -- 172.21.15.62:0/1089729828 learned_addr learned my addr 172.21.15.62:0/1089729828 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6233e00640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62340b27a0 0x7f6234152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6238e00640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b6a90 0x7f62341527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6233400640 1 -- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b6a90 msgr2=0x7f62341527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6233400640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b6a90 0x7f62341527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6233400640 1 -- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62340b27a0 msgr2=0x7f6234152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6233400640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62340b27a0 0x7f6234152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6233400640 1 -- 172.21.15.62:0/1089729828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234152590 con 0x7f62340b56f0 2024-04-25T09:09:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6233e00640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62340b27a0 0x7f6234152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:23.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6233400640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b56f0 0x7f6234150160 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f622000d5f0 tx=0x7f622000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f622be00640 1 -- 172.21.15.62:0/1089729828 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200125f0 con 0x7f62340b56f0 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f622be00640 1 -- 172.21.15.62:0/1089729828 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6220019070 con 0x7f62340b56f0 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f622be00640 1 -- 172.21.15.62:0/1089729828 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200129d0 con 0x7f62340b56f0 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.004+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623414cec0 con 0x7f62340b56f0 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.005+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623414d480 con 0x7f62340b56f0 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.006+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623404ef20 con 0x7f62340b56f0 2024-04-25T09:09:23.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.007+0000 7f622be00640 1 -- 172.21.15.62:0/1089729828 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6220012b70 con 0x7f62340b56f0 2024-04-25T09:09:23.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.008+0000 7f622be00640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621406b520 0x7f621406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:23.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.008+0000 7f622be00640 1 -- 172.21.15.62:0/1089729828 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f622008a1b0 con 0x7f62340b56f0 2024-04-25T09:09:23.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.008+0000 7f6233e00640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621406b520 0x7f621406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:23.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.010+0000 7f6233e00640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621406b520 0x7f621406d9e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6224006fd0 tx=0x7f6224008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:24.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.012+0000 7f622be00640 1 -- 172.21.15.62:0/1089729828 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6220059440 con 0x7f62340b56f0 2024-04-25T09:09:24.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.248+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62340ce580 con 0x7f62340b56f0 2024-04-25T09:09:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.249+0000 7f622be00640 1 -- 172.21.15.62:0/1089729828 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f6220058d40 con 0x7f62340b56f0 2024-04-25T09:09:24.237 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:24.237 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1396005512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1396005512},{"type":"v1","addr":"172.21.15.121:6835","nonce":1396005512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5617,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3589939339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":3589939339},{"type":"v1","addr":"172.21.15.121:6837","nonce":3589939339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2551168226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2551168226},{"type":"v1","addr":"172.21.15.62:6835","nonce":2551168226}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/2672601805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2672601805},{"type":"v1","addr":"172.21.15.62:6833","nonce":2672601805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[]} 2024-04-25T09:09:24.237 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 47 2024-04-25T09:09:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621406b520 msgr2=0x7f621406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621406b520 0x7f621406d9e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6224006fd0 tx=0x7f6224008040 comp rx=0 tx=0).stop 2024-04-25T09:09:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b56f0 msgr2=0x7f6234150160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:24.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b56f0 0x7f6234150160 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f622000d5f0 tx=0x7f622000dab0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 shutdown_connections 2024-04-25T09:09:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f621406b520 0x7f621406d9e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62340b56f0 0x7f6234150160 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62340b6a90 0x7f62341527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 --2- 172.21.15.62:0/1089729828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62340b27a0 0x7f6234152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.251+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 >> 172.21.15.62:0/1089729828 conn(0x7f6234066d10 msgr2=0x7f62340b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.252+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 shutdown_connections 2024-04-25T09:09:24.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.252+0000 7f6239800640 1 -- 172.21.15.62:0/1089729828 wait complete. 2024-04-25T09:09:24.260 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:09:24.260 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:09:24.260 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:09:24.260 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:09:24.260 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:09:24.260 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:09:24.261 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:09:24.261 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:09:24.261 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:09:24.261 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:09:24.261 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:09:24.261 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:09:24.261 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:09:24.262 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:09:24.262 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:09:24.262 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:09:24.279 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:09:24.280 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:09:24.281 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:09:24.281 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:09:24.282 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:09:24.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:09:24.284 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:09:24.284 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:09:24.327 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:09:24.327 DEBUG:teuthology.parallel:result is None 2024-04-25T09:09:24.331 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:09:24.331 DEBUG:teuthology.parallel:result is None 2024-04-25T09:09:24.339 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:09:24.339 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:09:24.340 DEBUG:teuthology.parallel:result is None 2024-04-25T09:09:24.340 DEBUG:teuthology.parallel:result is None 2024-04-25T09:09:24.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:09:24.482 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:09:24.482 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:09:24.484 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:09:24.485 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:09:24.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.540+0000 7f656be00640 1 Processor -- start 2024-04-25T09:09:24.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.541+0000 7f656be00640 1 -- start start 2024-04-25T09:09:24.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.542+0000 7f656be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b2d40 0x7f65640b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:24.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.542+0000 7f656be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f65640b8cf0 0x7f65640d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:24.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.542+0000 7f656be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65640b7780 con 0x7f656415de70 2024-04-25T09:09:24.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.542+0000 7f656be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65640b8320 con 0x7f65640b8cf0 2024-04-25T09:09:24.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.542+0000 7f656be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65640b9ad0 con 0x7f65640b2d40 2024-04-25T09:09:24.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.542+0000 7f6569400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f656415de70 0x7f65640b93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:44560/0 (socket says 172.21.15.62:44560) 2024-04-25T09:09:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.542+0000 7f6569400640 1 -- 172.21.15.62:0/3316270216 learned_addr learned my addr 172.21.15.62:0/3316270216 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6568a00640 1 --2- 172.21.15.62:0/3316270216 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b2d40 0x7f65640b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6569e00640 1 --2- 172.21.15.62:0/3316270216 >> v2:172.21.15.121:3300/0 conn(0x7f65640b8cf0 0x7f65640d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6563e00640 1 -- 172.21.15.62:0/3316270216 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103382470 0 0) 0x7f65640b7780 con 0x7f656415de70 2024-04-25T09:09:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6563e00640 1 -- 172.21.15.62:0/3316270216 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f654c003520 con 0x7f656415de70 2024-04-25T09:09:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6568a00640 1 -- 172.21.15.62:0/3316270216 >> v2:172.21.15.121:3300/0 conn(0x7f65640b8cf0 msgr2=0x7f65640d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6568a00640 1 --2- 172.21.15.62:0/3316270216 >> v2:172.21.15.121:3300/0 conn(0x7f65640b8cf0 0x7f65640d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6568a00640 1 -- 172.21.15.62:0/3316270216 >> v1:172.21.15.62:6789/0 conn(0x7f656415de70 legacy=0x7f65640b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6563e00640 1 -- 172.21.15.62:0/3316270216 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2291105172 0 0) 0x7f654c003520 con 0x7f656415de70 2024-04-25T09:09:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.543+0000 7f6568a00640 1 -- 172.21.15.62:0/3316270216 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65640b79f0 con 0x7f65640b2d40 2024-04-25T09:09:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.544+0000 7f6568a00640 1 --2- 172.21.15.62:0/3316270216 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b2d40 0x7f65640b87b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65500097a0 tx=0x7f655002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aec0e6fd3a03c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f6563e00640 1 -- 172.21.15.62:0/3316270216 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550037070 con 0x7f65640b2d40 2024-04-25T09:09:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f6563e00640 1 -- 172.21.15.62:0/3316270216 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655002ac30 con 0x7f65640b2d40 2024-04-25T09:09:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f6563e00640 1 -- 172.21.15.62:0/3316270216 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65580019f0 con 0x7f65640b2d40 2024-04-25T09:09:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f656be00640 1 -- 172.21.15.62:0/3316270216 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b2d40 msgr2=0x7f65640b87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f656be00640 1 --2- 172.21.15.62:0/3316270216 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b2d40 0x7f65640b87b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65500097a0 tx=0x7f655002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f656be00640 1 -- 172.21.15.62:0/3316270216 shutdown_connections 2024-04-25T09:09:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f656be00640 1 --2- 172.21.15.62:0/3316270216 >> v2:172.21.15.121:3300/0 conn(0x7f65640b8cf0 0x7f65640d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f656be00640 1 --2- 172.21.15.62:0/3316270216 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b2d40 0x7f65640b87b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f656be00640 1 -- 172.21.15.62:0/3316270216 >> 172.21.15.62:0/3316270216 conn(0x7f656406b000 msgr2=0x7f656406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.545+0000 7f656be00640 1 -- 172.21.15.62:0/3316270216 shutdown_connections 2024-04-25T09:09:24.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.546+0000 7f656be00640 1 -- 172.21.15.62:0/3316270216 wait complete. 2024-04-25T09:09:24.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.547+0000 7f656be00640 1 Processor -- start 2024-04-25T09:09:24.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.547+0000 7f656be00640 1 -- start start 2024-04-25T09:09:24.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.548+0000 7f656be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b2d40 0x7f656415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:24.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.548+0000 7f656be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b8cf0 0x7f6564157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:24.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.548+0000 7f656be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f656415de70 0x7f6564157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.548+0000 7f656be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640d23c0 con 0x7f656415de70 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.548+0000 7f656be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640b7fb0 con 0x7f65640b2d40 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.548+0000 7f656be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65640d2540 con 0x7f65640b8cf0 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f656415de70 0x7f6564157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f656415de70 0x7f6564157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51412/0 (socket says 172.21.15.62:51412) 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569e00640 1 -- 172.21.15.62:0/3773633827 learned_addr learned my addr 172.21.15.62:0/3773633827 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569400640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b2d40 0x7f656415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6568a00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b8cf0 0x7f6564157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569e00640 1 -- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b8cf0 msgr2=0x7f6564157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569e00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b8cf0 0x7f6564157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569e00640 1 -- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b2d40 msgr2=0x7f656415f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569e00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b2d40 0x7f656415f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.549+0000 7f6569e00640 1 -- 172.21.15.62:0/3773633827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6564161270 con 0x7f656415de70 2024-04-25T09:09:24.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.550+0000 7f6569e00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f656415de70 0x7f6564157f50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f655c00e930 tx=0x7f655c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:24.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.550+0000 7f6561600640 1 -- 172.21.15.62:0/3773633827 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c017070 con 0x7f656415de70 2024-04-25T09:09:24.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.550+0000 7f6561600640 1 -- 172.21.15.62:0/3773633827 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c00c980 con 0x7f656415de70 2024-04-25T09:09:24.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.550+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65640cb940 con 0x7f656415de70 2024-04-25T09:09:24.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.550+0000 7f6561600640 1 -- 172.21.15.62:0/3773633827 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c00cd70 con 0x7f656415de70 2024-04-25T09:09:24.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.550+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65640cbf00 con 0x7f656415de70 2024-04-25T09:09:24.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.551+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65640729a0 con 0x7f656415de70 2024-04-25T09:09:24.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.553+0000 7f6561600640 1 -- 172.21.15.62:0/3773633827 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f655c01a070 con 0x7f656415de70 2024-04-25T09:09:24.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.553+0000 7f6561600640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f654406b500 0x7f654406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:24.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.554+0000 7f6561600640 1 -- 172.21.15.62:0/3773633827 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f655c08a460 con 0x7f656415de70 2024-04-25T09:09:24.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.554+0000 7f6569400640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f654406b500 0x7f654406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:24.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.555+0000 7f6569400640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f654406b500 0x7f654406d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6558001560 tx=0x7f6558009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:24.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.558+0000 7f6561600640 1 -- 172.21.15.62:0/3773633827 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f655c059510 con 0x7f656415de70 2024-04-25T09:09:24.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.774+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f656404f000 con 0x7f656415de70 2024-04-25T09:09:24.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.878+0000 7f6561600640 1 -- 172.21.15.62:0/3773633827 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) v1 ==== 97+0+0 (secure 0 0 0) 0x7f655c058e10 con 0x7f656415de70 2024-04-25T09:09:24.867 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.881+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f654406b500 msgr2=0x7f654406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.881+0000 7f656be00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f654406b500 0x7f654406d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6558001560 tx=0x7f6558009040 comp rx=0 tx=0).stop 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.881+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f656415de70 msgr2=0x7f6564157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.881+0000 7f656be00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f656415de70 0x7f6564157f50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f655c00e930 tx=0x7f655c00edf0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.882+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 shutdown_connections 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.882+0000 7f656be00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f654406b500 0x7f654406d9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.882+0000 7f656be00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f656415de70 0x7f6564157f50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.882+0000 7f656be00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65640b8cf0 0x7f6564157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.882+0000 7f656be00640 1 --2- 172.21.15.62:0/3773633827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65640b2d40 0x7f656415f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:24.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.882+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 >> 172.21.15.62:0/3773633827 conn(0x7f656406b000 msgr2=0x7f65640668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:24.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.882+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 shutdown_connections 2024-04-25T09:09:24.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:24.882+0000 7f656be00640 1 -- 172.21.15.62:0/3773633827 wait complete. 2024-04-25T09:09:24.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:09:25.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.110+0000 7f2856800640 1 Processor -- start 2024-04-25T09:09:25.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.111+0000 7f2856800640 1 -- start start 2024-04-25T09:09:25.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.112+0000 7f2856800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f28500b2880 0x7f28500b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.112+0000 7f2856800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2850070bb0 0x7f2850071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.112+0000 7f2856800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28500b31a0 con 0x7f285015ac70 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.112+0000 7f2856800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28500cefa0 con 0x7f28500b2880 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.112+0000 7f2856800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500cf280 con 0x7f2850070bb0 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f28500b2880 0x7f28500b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f28500b2880 0x7f28500b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45224/0 (socket says 172.21.15.62:45224) 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284f400640 1 -- 172.21.15.62:0/228376485 learned_addr learned my addr 172.21.15.62:0/228376485 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284fe00640 1 --2- 172.21.15.62:0/228376485 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2850070bb0 0x7f2850071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284f400640 1 -- 172.21.15.62:0/228376485 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2850070bb0 msgr2=0x7f2850071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284f400640 1 --2- 172.21.15.62:0/228376485 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2850070bb0 0x7f2850071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284f400640 1 -- 172.21.15.62:0/228376485 >> v1:172.21.15.121:6789/0 conn(0x7f285015ac70 legacy=0x7f28500b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284f400640 1 -- 172.21.15.62:0/228376485 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500703b0 con 0x7f28500b2880 2024-04-25T09:09:25.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.113+0000 7f284fe00640 1 --2- 172.21.15.62:0/228376485 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2850070bb0 0x7f2850071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.114+0000 7f284f400640 1 --2- 172.21.15.62:0/228376485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28500b2880 0x7f28500b2c60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2840009870 tx=0x7f284002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f41d8cd7b2192b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f284ea00640 1 -- 172.21.15.62:0/228376485 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840037070 con 0x7f28500b2880 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f284ea00640 1 -- 172.21.15.62:0/228376485 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840032440 con 0x7f28500b2880 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f284ea00640 1 -- 172.21.15.62:0/228376485 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840032810 con 0x7f28500b2880 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f2856800640 1 -- 172.21.15.62:0/228376485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28500b2880 msgr2=0x7f28500b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f2856800640 1 --2- 172.21.15.62:0/228376485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28500b2880 0x7f28500b2c60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2840009870 tx=0x7f284002a200 comp rx=0 tx=0).stop 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f2856800640 1 -- 172.21.15.62:0/228376485 shutdown_connections 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f2856800640 1 --2- 172.21.15.62:0/228376485 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2850070bb0 0x7f2850071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f2856800640 1 --2- 172.21.15.62:0/228376485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28500b2880 0x7f28500b2c60 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.115+0000 7f2856800640 1 -- 172.21.15.62:0/228376485 >> 172.21.15.62:0/228376485 conn(0x7f28500668f0 msgr2=0x7f2850066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.116+0000 7f2856800640 1 -- 172.21.15.62:0/228376485 shutdown_connections 2024-04-25T09:09:25.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.116+0000 7f2856800640 1 -- 172.21.15.62:0/228376485 wait complete. 2024-04-25T09:09:25.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.117+0000 7f2856800640 1 Processor -- start 2024-04-25T09:09:25.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.117+0000 7f2856800640 1 -- start start 2024-04-25T09:09:25.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.118+0000 7f2856800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2850070bb0 0x7f28501f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.118+0000 7f2856800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28500b2880 0x7f28501e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.118+0000 7f2856800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f285015ac70 0x7f2850071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.118+0000 7f2856800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500ceb30 con 0x7f2850070bb0 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.118+0000 7f2856800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500cf560 con 0x7f28500b2880 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.118+0000 7f2856800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500cecb0 con 0x7f285015ac70 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f284f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2850070bb0 0x7f28501f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f284f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2850070bb0 0x7f28501f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45228/0 (socket says 172.21.15.62:45228) 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f284f400640 1 -- 172.21.15.62:0/2091951765 learned_addr learned my addr 172.21.15.62:0/2091951765 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f284f400640 1 -- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f285015ac70 msgr2=0x7f2850071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f284f400640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f285015ac70 0x7f2850071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f284f400640 1 -- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28500b2880 msgr2=0x7f28501e3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f2847e00640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28500b2880 0x7f28501e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f284f400640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28500b2880 0x7f28501e3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.119+0000 7f284f400640 1 -- 172.21.15.62:0/2091951765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28501e4ce0 con 0x7f2850070bb0 2024-04-25T09:09:25.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.120+0000 7f284f400640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2850070bb0 0x7f28501f3bc0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2840000c00 tx=0x7f28400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:25.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.120+0000 7f2847400640 1 -- 172.21.15.62:0/2091951765 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840037040 con 0x7f2850070bb0 2024-04-25T09:09:25.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.120+0000 7f2847400640 1 -- 172.21.15.62:0/2091951765 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840042070 con 0x7f2850070bb0 2024-04-25T09:09:25.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.120+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28501e4fc0 con 0x7f2850070bb0 2024-04-25T09:09:25.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.120+0000 7f2847400640 1 -- 172.21.15.62:0/2091951765 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840030bb0 con 0x7f2850070bb0 2024-04-25T09:09:25.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.120+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28501e5580 con 0x7f2850070bb0 2024-04-25T09:09:25.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.121+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285004ef20 con 0x7f2850070bb0 2024-04-25T09:09:25.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.122+0000 7f2847400640 1 -- 172.21.15.62:0/2091951765 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f284002ad10 con 0x7f2850070bb0 2024-04-25T09:09:25.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.123+0000 7f2847400640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f282c06b4d0 0x7f282c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.123+0000 7f2847400640 1 -- 172.21.15.62:0/2091951765 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f28400aa820 con 0x7f2850070bb0 2024-04-25T09:09:25.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.124+0000 7f2847e00640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f282c06b4d0 0x7f282c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.125+0000 7f2847e00640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f282c06b4d0 0x7f282c06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28501e6ee0 tx=0x7f2838008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:25.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.127+0000 7f2847400640 1 -- 172.21.15.62:0/2091951765 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2840045360 con 0x7f2850070bb0 2024-04-25T09:09:25.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.397+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2850067050 con 0x7f2850070bb0 2024-04-25T09:09:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.399+0000 7f2847400640 1 -- 172.21.15.62:0/2091951765 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f28400799e0 con 0x7f2850070bb0 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:09:25.387 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.388 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:09:25.389 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.390 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.391 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.392 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.393 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:09:25.394 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:09:25.395 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:09:25.396 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.397 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:09:25.398 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:09:25.399 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:09:25.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f282c06b4d0 msgr2=0x7f282c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f282c06b4d0 0x7f282c06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28501e6ee0 tx=0x7f2838008040 comp rx=0 tx=0).stop 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2850070bb0 msgr2=0x7f28501f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2850070bb0 0x7f28501f3bc0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2840000c00 tx=0x7f28400046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 shutdown_connections 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f282c06b4d0 0x7f282c06d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f285015ac70 0x7f2850071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28500b2880 0x7f28501e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 --2- 172.21.15.62:0/2091951765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2850070bb0 0x7f28501f3bc0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 >> 172.21.15.62:0/2091951765 conn(0x7f28500668f0 msgr2=0x7f285006d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.401+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 shutdown_connections 2024-04-25T09:09:25.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.402+0000 7f2856800640 1 -- 172.21.15.62:0/2091951765 wait complete. 2024-04-25T09:09:25.408 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:25.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.627+0000 7f9e90e00640 1 Processor -- start 2024-04-25T09:09:25.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.627+0000 7f9e90e00640 1 -- start start 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.628+0000 7f9e90e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c06d900 0x7f9e8c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.628+0000 7f9e90e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9e8c0cee40 0x7f9e8c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.628+0000 7f9e90e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8c0cf2d0 con 0x7f9e8c144c10 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.628+0000 7f9e90e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8c144ff0 con 0x7f9e8c0cee40 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.628+0000 7f9e90e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c160340 con 0x7f9e8c06d900 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.629+0000 7f9e8be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9e8c0cee40 0x7f9e8c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.629+0000 7f9e8be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9e8c0cee40 0x7f9e8c0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45230/0 (socket says 172.21.15.62:45230) 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.629+0000 7f9e8be00640 1 -- 172.21.15.62:0/2727291196 learned_addr learned my addr 172.21.15.62:0/2727291196 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:25.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.629+0000 7f9e83e00640 1 --2- 172.21.15.62:0/2727291196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c06d900 0x7f9e8c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.629+0000 7f9e8be00640 1 -- 172.21.15.62:0/2727291196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c06d900 msgr2=0x7f9e8c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.629+0000 7f9e8be00640 1 --2- 172.21.15.62:0/2727291196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c06d900 0x7f9e8c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.629+0000 7f9e8be00640 1 -- 172.21.15.62:0/2727291196 >> v1:172.21.15.121:6789/0 conn(0x7f9e8c144c10 legacy=0x7f9e8c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.629+0000 7f9e8be00640 1 -- 172.21.15.62:0/2727291196 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c15faa0 con 0x7f9e8c0cee40 2024-04-25T09:09:25.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.630+0000 7f9e8be00640 1 --2- 172.21.15.62:0/2727291196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c0cee40 0x7f9e8c0b4650 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009e50 tx=0x7f9e8402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac408af9d1c8d825 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:25.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.630+0000 7f9e8aa00640 1 -- 172.21.15.62:0/2727291196 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84036070 con 0x7f9e8c0cee40 2024-04-25T09:09:25.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e8aa00640 1 -- 172.21.15.62:0/2727291196 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e84031600 con 0x7f9e8c0cee40 2024-04-25T09:09:25.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e8aa00640 1 -- 172.21.15.62:0/2727291196 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e840319d0 con 0x7f9e8c0cee40 2024-04-25T09:09:25.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e90e00640 1 -- 172.21.15.62:0/2727291196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c0cee40 msgr2=0x7f9e8c0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e90e00640 1 --2- 172.21.15.62:0/2727291196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c0cee40 0x7f9e8c0b4650 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009e50 tx=0x7f9e8402a680 comp rx=0 tx=0).stop 2024-04-25T09:09:25.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e90e00640 1 -- 172.21.15.62:0/2727291196 shutdown_connections 2024-04-25T09:09:25.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e90e00640 1 --2- 172.21.15.62:0/2727291196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c0cee40 0x7f9e8c0b4650 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e90e00640 1 --2- 172.21.15.62:0/2727291196 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c06d900 0x7f9e8c15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e90e00640 1 -- 172.21.15.62:0/2727291196 >> 172.21.15.62:0/2727291196 conn(0x7f9e8c0b14d0 msgr2=0x7f9e8c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:25.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e90e00640 1 -- 172.21.15.62:0/2727291196 shutdown_connections 2024-04-25T09:09:25.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.631+0000 7f9e90e00640 1 -- 172.21.15.62:0/2727291196 wait complete. 2024-04-25T09:09:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.633+0000 7f9e90e00640 1 Processor -- start 2024-04-25T09:09:25.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.633+0000 7f9e90e00640 1 -- start start 2024-04-25T09:09:25.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e90e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e8c06d900 0x7f9e8c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e90e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c0cee40 0x7f9e8c1430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e90e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c144c10 0x7f9e8c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e90e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c160060 con 0x7f9e8c144c10 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e90e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0b41c0 con 0x7f9e8c06d900 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e90e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0b4b90 con 0x7f9e8c0cee40 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c144c10 0x7f9e8c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c144c10 0x7f9e8c15d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45238/0 (socket says 172.21.15.62:45238) 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.634+0000 7f9e8be00640 1 -- 172.21.15.62:0/777489749 learned_addr learned my addr 172.21.15.62:0/777489749 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e8b400640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e8c06d900 0x7f9e8c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e83e00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c0cee40 0x7f9e8c1430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e8be00640 1 -- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c0cee40 msgr2=0x7f9e8c1430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e8be00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c0cee40 0x7f9e8c1430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e8be00640 1 -- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e8c06d900 msgr2=0x7f9e8c15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e8be00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e8c06d900 0x7f9e8c15b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e8be00640 1 -- 172.21.15.62:0/777489749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c15b8f0 con 0x7f9e8c144c10 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e8b400640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e8c06d900 0x7f9e8c15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e8be00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c144c10 0x7f9e8c15d8b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009f70 tx=0x7f9e84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e83400640 1 -- 172.21.15.62:0/777489749 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8403f020 con 0x7f9e8c144c10 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e83400640 1 -- 172.21.15.62:0/777489749 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e84037040 con 0x7f9e8c144c10 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e83400640 1 -- 172.21.15.62:0/777489749 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84036070 con 0x7f9e8c144c10 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c15c850 con 0x7f9e8c144c10 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.635+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c15cdd0 con 0x7f9e8c144c10 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.637+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c04ef20 con 0x7f9e8c144c10 2024-04-25T09:09:25.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.638+0000 7f9e83400640 1 -- 172.21.15.62:0/777489749 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e84042020 con 0x7f9e8c144c10 2024-04-25T09:09:25.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.639+0000 7f9e83400640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9e6806b4d0 0x7f9e6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:25.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.639+0000 7f9e83400640 1 -- 172.21.15.62:0/777489749 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9e840aa380 con 0x7f9e8c144c10 2024-04-25T09:09:25.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.639+0000 7f9e8b400640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9e6806b4d0 0x7f9e6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:25.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.640+0000 7f9e8b400640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9e6806b4d0 0x7f9e6806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0015e0 tx=0x7f9e7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:25.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.643+0000 7f9e83400640 1 -- 172.21.15.62:0/777489749 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e84049030 con 0x7f9e8c144c10 2024-04-25T09:09:25.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.843+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e8c06e470 con 0x7f9e8c144c10 2024-04-25T09:09:25.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.845+0000 7f9e83400640 1 -- 172.21.15.62:0/777489749 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+12879 (secure 0 0 0) 0x7f9e84079550 con 0x7f9e8c144c10 2024-04-25T09:09:25.834 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:25.834 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":75,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:24.895403+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9e6806b4d0 msgr2=0x7f9e6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9e6806b4d0 0x7f9e6806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0015e0 tx=0x7f9e7c009040 comp rx=0 tx=0).stop 2024-04-25T09:09:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c144c10 msgr2=0x7f9e8c15d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c144c10 0x7f9e8c15d8b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009f70 tx=0x7f9e84004a40 comp rx=0 tx=0).stop 2024-04-25T09:09:25.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 shutdown_connections 2024-04-25T09:09:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9e6806b4d0 0x7f9e6806d990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9e8c144c10 0x7f9e8c15d8b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9e8c0cee40 0x7f9e8c1430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 --2- 172.21.15.62:0/777489749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e8c06d900 0x7f9e8c15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 >> 172.21.15.62:0/777489749 conn(0x7f9e8c0b14d0 msgr2=0x7f9e8c0cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 shutdown_connections 2024-04-25T09:09:25.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:25.849+0000 7f9e90e00640 1 -- 172.21.15.62:0/777489749 wait complete. 2024-04-25T09:09:25.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:09:25.859 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:09:26.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.075+0000 7fc3e0a00640 1 Processor -- start 2024-04-25T09:09:26.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.076+0000 7fc3e0a00640 1 -- start start 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.077+0000 7fc3e0a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc3dc0ccdb0 0x7fc3dc0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.077+0000 7fc3e0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc0d0490 0x7fc3dc0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.077+0000 7fc3e0a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3dc0cd650 con 0x7fc3dc15b8f0 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.077+0000 7fc3e0a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3dc0cd370 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.077+0000 7fc3e0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc0cf470 con 0x7fc3dc0d0490 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3db400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc3dc15b8f0 0x7fc3dc0b3d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40752/0 (socket says 172.21.15.62:40752) 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3db400640 1 -- 172.21.15.62:0/1868575414 learned_addr learned my addr 172.21.15.62:0/1868575414 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3daa00640 1 --2- 172.21.15.62:0/1868575414 >> v2:172.21.15.62:3300/0 conn(0x7fc3dc0ccdb0 0x7fc3dc0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3dbe00640 1 --2- 172.21.15.62:0/1868575414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc0d0490 0x7fc3dc0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3da000640 1 -- 172.21.15.62:0/1868575414 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996344156 0 0) 0x7fc3dc0cd650 con 0x7fc3dc15b8f0 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3da000640 1 -- 172.21.15.62:0/1868575414 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c0003520 con 0x7fc3dc15b8f0 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3daa00640 1 -- 172.21.15.62:0/1868575414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc0d0490 msgr2=0x7fc3dc0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3daa00640 1 --2- 172.21.15.62:0/1868575414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc0d0490 0x7fc3dc0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3daa00640 1 -- 172.21.15.62:0/1868575414 >> v1:172.21.15.62:6789/0 conn(0x7fc3dc15b8f0 legacy=0x7fc3dc0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3daa00640 1 -- 172.21.15.62:0/1868575414 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0cf750 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.078+0000 7fc3dbe00640 1 --2- 172.21.15.62:0/1868575414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc0d0490 0x7fc3dc0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.079+0000 7fc3daa00640 1 --2- 172.21.15.62:0/1868575414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc0cff50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009d90 tx=0x7fc3c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f30e26487fd8f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.080+0000 7fc3da000640 1 -- 172.21.15.62:0/1868575414 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8007740 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.080+0000 7fc3da000640 1 -- 172.21.15.62:0/1868575414 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80078e0 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.080+0000 7fc3da000640 1 -- 172.21.15.62:0/1868575414 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8007c90 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.080+0000 7fc3e0a00640 1 -- 172.21.15.62:0/1868575414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 msgr2=0x7fc3dc0cff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.080+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/1868575414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc0cff50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009d90 tx=0x7fc3c802a660 comp rx=0 tx=0).stop 2024-04-25T09:09:26.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.081+0000 7fc3e0a00640 1 -- 172.21.15.62:0/1868575414 shutdown_connections 2024-04-25T09:09:26.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.081+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/1868575414 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc0d0490 0x7fc3dc0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.081+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/1868575414 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc0cff50 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.081+0000 7fc3e0a00640 1 -- 172.21.15.62:0/1868575414 >> 172.21.15.62:0/1868575414 conn(0x7fc3dc09aa70 msgr2=0x7fc3dc0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:26.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.081+0000 7fc3e0a00640 1 -- 172.21.15.62:0/1868575414 shutdown_connections 2024-04-25T09:09:26.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.081+0000 7fc3e0a00640 1 -- 172.21.15.62:0/1868575414 wait complete. 2024-04-25T09:09:26.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.082+0000 7fc3e0a00640 1 Processor -- start 2024-04-25T09:09:26.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.082+0000 7fc3e0a00640 1 -- start start 2024-04-25T09:09:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.083+0000 7fc3e0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.083+0000 7fc3e0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc0d0490 0x7fc3dc0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.083+0000 7fc3e0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc15b8f0 0x7fc3dc157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.083+0000 7fc3e0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc0ce380 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.083+0000 7fc3e0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc15bcd0 con 0x7fc3dc0d0490 2024-04-25T09:09:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.083+0000 7fc3e0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc0ce500 con 0x7fc3dc15b8f0 2024-04-25T09:09:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3db400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3db400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45256/0 (socket says 172.21.15.62:45256) 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3db400640 1 -- 172.21.15.62:0/3021740934 learned_addr learned my addr 172.21.15.62:0/3021740934 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3daa00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc0d0490 0x7fc3dc0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3db400640 1 -- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc15b8f0 msgr2=0x7fc3dc157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3dbe00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc15b8f0 0x7fc3dc157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3db400640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc15b8f0 0x7fc3dc157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3db400640 1 -- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc0d0490 msgr2=0x7fc3dc0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3db400640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc0d0490 0x7fc3dc0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3db400640 1 -- 172.21.15.62:0/3021740934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc15c1c0 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.084+0000 7fc3daa00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc0d0490 0x7fc3dc0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.085+0000 7fc3db400640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc159f10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00c390 tx=0x7fc3cc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.085+0000 7fc3d3400640 1 -- 172.21.15.62:0/3021740934 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc017070 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.085+0000 7fc3d3400640 1 -- 172.21.15.62:0/3021740934 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3cc005ce0 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.085+0000 7fc3d3400640 1 -- 172.21.15.62:0/3021740934 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0072f0 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.085+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3dc15c4a0 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.085+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3dc15ca60 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.086+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3dc0750e0 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.087+0000 7fc3d3400640 1 -- 172.21.15.62:0/3021740934 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3cc01a070 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.088+0000 7fc3d3400640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3bc06b4d0 0x7fc3bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.088+0000 7fc3d3400640 1 -- 172.21.15.62:0/3021740934 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc3cc089360 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.089+0000 7fc3daa00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3bc06b4d0 0x7fc3bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.090+0000 7fc3daa00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3bc06b4d0 0x7fc3bc06d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8034000 tx=0x7fc3c802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.093+0000 7fc3d3400640 1 -- 172.21.15.62:0/3021740934 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3cc058490 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.282+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3dc06c730 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.283+0000 7fc3d3400640 1 -- 172.21.15.62:0/3021740934 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3cc057d90 con 0x7fc3dc0ccdb0 2024-04-25T09:09:26.271 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3bc06b4d0 msgr2=0x7fc3bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3bc06b4d0 0x7fc3bc06d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8034000 tx=0x7fc3c802f000 comp rx=0 tx=0).stop 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 msgr2=0x7fc3dc159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc159f10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00c390 tx=0x7fc3cc00c850 comp rx=0 tx=0).stop 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 shutdown_connections 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3bc06b4d0 0x7fc3bc06d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3dc15b8f0 0x7fc3dc157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc0d0490 0x7fc3dc0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 --2- 172.21.15.62:0/3021740934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3dc0ccdb0 0x7fc3dc159f10 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 >> 172.21.15.62:0/3021740934 conn(0x7fc3dc09aa70 msgr2=0x7fc3dc0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:26.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.285+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 shutdown_connections 2024-04-25T09:09:26.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.286+0000 7fc3e0a00640 1 -- 172.21.15.62:0/3021740934 wait complete. 2024-04-25T09:09:26.291 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:09:26.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.510+0000 7f3c67200640 1 Processor -- start 2024-04-25T09:09:26.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.510+0000 7f3c67200640 1 -- start start 2024-04-25T09:09:26.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.512+0000 7f3c67200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c6015a1b0 0x7f3c600cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.512+0000 7f3c67200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c600b3db0 0x7f3c600cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.512+0000 7f3c67200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6015a590 con 0x7f3c600cd7b0 2024-04-25T09:09:26.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.512+0000 7f3c67200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c600b4320 con 0x7f3c6015a1b0 2024-04-25T09:09:26.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.512+0000 7f3c67200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c600b4600 con 0x7f3c600b3db0 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.512+0000 7f3c5fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c6015a1b0 0x7f3c600cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.513+0000 7f3c5fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c6015a1b0 0x7f3c600cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45272/0 (socket says 172.21.15.62:45272) 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.513+0000 7f3c5f400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c600b3db0 0x7f3c600cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.513+0000 7f3c5fe00640 1 -- 172.21.15.62:0/676102082 learned_addr learned my addr 172.21.15.62:0/676102082 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.513+0000 7f3c5fe00640 1 -- 172.21.15.62:0/676102082 >> v2:172.21.15.121:3300/0 conn(0x7f3c600b3db0 msgr2=0x7f3c600cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:26.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.513+0000 7f3c5fe00640 1 --2- 172.21.15.62:0/676102082 >> v2:172.21.15.121:3300/0 conn(0x7f3c600b3db0 0x7f3c600cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.513+0000 7f3c5fe00640 1 -- 172.21.15.62:0/676102082 >> v1:172.21.15.121:6789/0 conn(0x7f3c600cd7b0 legacy=0x7f3c6006cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.513+0000 7f3c5fe00640 1 -- 172.21.15.62:0/676102082 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600b48e0 con 0x7f3c6015a1b0 2024-04-25T09:09:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.515+0000 7f3c5fe00640 1 --2- 172.21.15.62:0/676102082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6015a1b0 0x7f3c600cef20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3c6006e820 tx=0x7f3c5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48277f3e9189142d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.515+0000 7f3c5ea00640 1 -- 172.21.15.62:0/676102082 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50037070 con 0x7f3c6015a1b0 2024-04-25T09:09:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.515+0000 7f3c5ea00640 1 -- 172.21.15.62:0/676102082 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50032440 con 0x7f3c6015a1b0 2024-04-25T09:09:26.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.515+0000 7f3c5ea00640 1 -- 172.21.15.62:0/676102082 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50032810 con 0x7f3c6015a1b0 2024-04-25T09:09:26.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.516+0000 7f3c67200640 1 -- 172.21.15.62:0/676102082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6015a1b0 msgr2=0x7f3c600cef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.516+0000 7f3c67200640 1 --2- 172.21.15.62:0/676102082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6015a1b0 0x7f3c600cef20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3c6006e820 tx=0x7f3c5002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.516+0000 7f3c67200640 1 -- 172.21.15.62:0/676102082 shutdown_connections 2024-04-25T09:09:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.516+0000 7f3c67200640 1 --2- 172.21.15.62:0/676102082 >> v2:172.21.15.121:3300/0 conn(0x7f3c600b3db0 0x7f3c600cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.516+0000 7f3c67200640 1 --2- 172.21.15.62:0/676102082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c6015a1b0 0x7f3c600cef20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.516+0000 7f3c67200640 1 -- 172.21.15.62:0/676102082 >> 172.21.15.62:0/676102082 conn(0x7f3c600668f0 msgr2=0x7f3c60066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.516+0000 7f3c67200640 1 -- 172.21.15.62:0/676102082 shutdown_connections 2024-04-25T09:09:26.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.517+0000 7f3c67200640 1 -- 172.21.15.62:0/676102082 wait complete. 2024-04-25T09:09:26.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.519+0000 7f3c67200640 1 Processor -- start 2024-04-25T09:09:26.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.519+0000 7f3c67200640 1 -- start start 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c67200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c600b3db0 0x7f3c60141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c67200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c600cd7b0 0x7f3c60151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c67200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6015a1b0 0x7f3c60141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c67200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600cf460 con 0x7f3c600b3db0 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c67200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600705f0 con 0x7f3c600cd7b0 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c67200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600cf5e0 con 0x7f3c6015a1b0 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c5fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c600b3db0 0x7f3c60141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c5fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c600b3db0 0x7f3c60141c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45276/0 (socket says 172.21.15.62:45276) 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.520+0000 7f3c5fe00640 1 -- 172.21.15.62:0/4284513639 learned_addr learned my addr 172.21.15.62:0/4284513639 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.521+0000 7f3c5fe00640 1 -- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6015a1b0 msgr2=0x7f3c60141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.521+0000 7f3c5fe00640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6015a1b0 0x7f3c60141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.521+0000 7f3c5fe00640 1 -- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c600cd7b0 msgr2=0x7f3c60151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.521+0000 7f3c5fe00640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c600cd7b0 0x7f3c60151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.521+0000 7f3c5fe00640 1 -- 172.21.15.62:0/4284513639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c601505e0 con 0x7f3c600b3db0 2024-04-25T09:09:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.521+0000 7f3c5fe00640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c600b3db0 0x7f3c60141c90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3c5002a220 tx=0x7f3c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.522+0000 7f3c57e00640 1 -- 172.21.15.62:0/4284513639 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50037040 con 0x7f3c600b3db0 2024-04-25T09:09:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.522+0000 7f3c57e00640 1 -- 172.21.15.62:0/4284513639 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50042070 con 0x7f3c600b3db0 2024-04-25T09:09:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.522+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c601508c0 con 0x7f3c600b3db0 2024-04-25T09:09:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.522+0000 7f3c57e00640 1 -- 172.21.15.62:0/4284513639 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50030880 con 0x7f3c600b3db0 2024-04-25T09:09:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.522+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c60150e80 con 0x7f3c600b3db0 2024-04-25T09:09:26.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.523+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6006d3a0 con 0x7f3c600b3db0 2024-04-25T09:09:26.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.524+0000 7f3c57e00640 1 -- 172.21.15.62:0/4284513639 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c50030a20 con 0x7f3c600b3db0 2024-04-25T09:09:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.525+0000 7f3c57e00640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3c4006b4d0 0x7f3c4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.525+0000 7f3c57e00640 1 -- 172.21.15.62:0/4284513639 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c500aa7b0 con 0x7f3c600b3db0 2024-04-25T09:09:26.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.526+0000 7f3c5f400640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3c4006b4d0 0x7f3c4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.527+0000 7f3c5f400640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3c4006b4d0 0x7f3c4006d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006a90 tx=0x7f3c4c006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.529+0000 7f3c57e00640 1 -- 172.21.15.62:0/4284513639 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c6006d3a0 con 0x7f3c600b3db0 2024-04-25T09:09:26.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.732+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3c600ade10 con 0x7f3c600b3db0 2024-04-25T09:09:26.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.733+0000 7f3c57e00640 1 -- 172.21.15.62:0/4284513639 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3c500798e0 con 0x7f3c600b3db0 2024-04-25T09:09:26.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:26.722 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:09:26.722 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:09:26.722 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:09:26.722 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:09:26.722 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:09:26.722 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3c4006b4d0 msgr2=0x7f3c4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3c4006b4d0 0x7f3c4006d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006a90 tx=0x7f3c4c006a20 comp rx=0 tx=0).stop 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c600b3db0 msgr2=0x7f3c60141c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c600b3db0 0x7f3c60141c90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3c5002a220 tx=0x7f3c500046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 shutdown_connections 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3c4006b4d0 0x7f3c4006d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3c6015a1b0 0x7f3c60141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c600cd7b0 0x7f3c60151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 --2- 172.21.15.62:0/4284513639 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c600b3db0 0x7f3c60141c90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 >> 172.21.15.62:0/4284513639 conn(0x7f3c600668f0 msgr2=0x7f3c600ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:26.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 shutdown_connections 2024-04-25T09:09:26.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.737+0000 7f3c67200640 1 -- 172.21.15.62:0/4284513639 wait complete. 2024-04-25T09:09:26.744 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:09:26.744 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:09:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.962+0000 7fb3fa600640 1 Processor -- start 2024-04-25T09:09:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.962+0000 7fb3fa600640 1 -- start start 2024-04-25T09:09:26.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3fa600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3f415beb0 0x7fb3f415c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3fa600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb3f406c440 0x7fb3f406c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3fa600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f406cdd0 con 0x7fb3f4145140 2024-04-25T09:09:26.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3fa600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f40b3d60 con 0x7fb3f415beb0 2024-04-25T09:09:26.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3fa600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f40b4900 con 0x7fb3f406c440 2024-04-25T09:09:26.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3f3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3f415beb0 0x7fb3f415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3f9200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb3f4145140 0x7fb3f40b41f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40754/0 (socket says 172.21.15.62:40754) 2024-04-25T09:09:26.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3f9200640 1 -- 172.21.15.62:0/525635183 learned_addr learned my addr 172.21.15.62:0/525635183 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:26.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.964+0000 7fb3f9c00640 1 --2- 172.21.15.62:0/525635183 >> v2:172.21.15.121:3300/0 conn(0x7fb3f406c440 0x7fb3f406c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.965+0000 7fb3f3e00640 1 -- 172.21.15.62:0/525635183 >> v2:172.21.15.121:3300/0 conn(0x7fb3f406c440 msgr2=0x7fb3f406c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:26.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.965+0000 7fb3f3e00640 1 --2- 172.21.15.62:0/525635183 >> v2:172.21.15.121:3300/0 conn(0x7fb3f406c440 0x7fb3f406c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.965+0000 7fb3f3e00640 1 -- 172.21.15.62:0/525635183 >> v1:172.21.15.62:6789/0 conn(0x7fb3f4145140 legacy=0x7fb3f40b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.965+0000 7fb3f3e00640 1 -- 172.21.15.62:0/525635183 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f415c9a0 con 0x7fb3f415beb0 2024-04-25T09:09:26.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.966+0000 7fb3f3e00640 1 --2- 172.21.15.62:0/525635183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f415beb0 0x7fb3f415c340 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40b3cd0 tx=0x7fb3e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faddd0b605f8c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.966+0000 7fb3f3400640 1 -- 172.21.15.62:0/525635183 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0037070 con 0x7fb3f415beb0 2024-04-25T09:09:26.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.966+0000 7fb3f3400640 1 -- 172.21.15.62:0/525635183 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e0032440 con 0x7fb3f415beb0 2024-04-25T09:09:26.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.966+0000 7fb3f3400640 1 -- 172.21.15.62:0/525635183 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0032810 con 0x7fb3f415beb0 2024-04-25T09:09:26.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.967+0000 7fb3fa600640 1 -- 172.21.15.62:0/525635183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f415beb0 msgr2=0x7fb3f415c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.967+0000 7fb3fa600640 1 --2- 172.21.15.62:0/525635183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f415beb0 0x7fb3f415c340 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40b3cd0 tx=0x7fb3e002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.967+0000 7fb3fa600640 1 -- 172.21.15.62:0/525635183 shutdown_connections 2024-04-25T09:09:26.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.967+0000 7fb3fa600640 1 --2- 172.21.15.62:0/525635183 >> v2:172.21.15.121:3300/0 conn(0x7fb3f406c440 0x7fb3f406c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.967+0000 7fb3fa600640 1 --2- 172.21.15.62:0/525635183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f415beb0 0x7fb3f415c340 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.967+0000 7fb3fa600640 1 -- 172.21.15.62:0/525635183 >> 172.21.15.62:0/525635183 conn(0x7fb3f4066ad0 msgr2=0x7fb3f4066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:26.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.967+0000 7fb3fa600640 1 -- 172.21.15.62:0/525635183 shutdown_connections 2024-04-25T09:09:26.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.967+0000 7fb3fa600640 1 -- 172.21.15.62:0/525635183 wait complete. 2024-04-25T09:09:26.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.968+0000 7fb3fa600640 1 Processor -- start 2024-04-25T09:09:26.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.969+0000 7fb3fa600640 1 -- start start 2024-04-25T09:09:26.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3fa600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3f406c440 0x7fb3f4140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3fa600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f4145140 0x7fb3f41e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3fa600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f415beb0 0x7fb3f41e5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3fa600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f40b5020 con 0x7fb3f4145140 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3fa600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f40b39f0 con 0x7fb3f415beb0 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3fa600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f40b51a0 con 0x7fb3f406c440 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3f3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f4145140 0x7fb3f41e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3f3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f4145140 0x7fb3f41e4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45296/0 (socket says 172.21.15.62:45296) 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3f3e00640 1 -- 172.21.15.62:0/2211336169 learned_addr learned my addr 172.21.15.62:0/2211336169 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3f3e00640 1 -- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3f406c440 msgr2=0x7fb3f4140240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3f3e00640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3f406c440 0x7fb3f4140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3f3e00640 1 -- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f415beb0 msgr2=0x7fb3f41e5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3f3e00640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f415beb0 0x7fb3f41e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.970+0000 7fb3f3e00640 1 -- 172.21.15.62:0/2211336169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f413c100 con 0x7fb3f4145140 2024-04-25T09:09:26.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.971+0000 7fb3f3e00640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f4145140 0x7fb3f41e4ee0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0000c00 tx=0x7fb3e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.971+0000 7fb3f0c00640 1 -- 172.21.15.62:0/2211336169 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0037040 con 0x7fb3f4145140 2024-04-25T09:09:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.971+0000 7fb3f0c00640 1 -- 172.21.15.62:0/2211336169 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e0042070 con 0x7fb3f4145140 2024-04-25T09:09:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.971+0000 7fb3f0c00640 1 -- 172.21.15.62:0/2211336169 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0030bd0 con 0x7fb3f4145140 2024-04-25T09:09:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.971+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f413c3e0 con 0x7fb3f4145140 2024-04-25T09:09:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.971+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f413c9a0 con 0x7fb3f4145140 2024-04-25T09:09:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.972+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f4072b30 con 0x7fb3f4145140 2024-04-25T09:09:26.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.973+0000 7fb3f0c00640 1 -- 172.21.15.62:0/2211336169 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3e0047460 con 0x7fb3f4145140 2024-04-25T09:09:26.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.974+0000 7fb3f0c00640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3d406b4d0 0x7fb3d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:26.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.974+0000 7fb3f0c00640 1 -- 172.21.15.62:0/2211336169 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb3e00aab80 con 0x7fb3f4145140 2024-04-25T09:09:26.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.975+0000 7fb3f9200640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3d406b4d0 0x7fb3d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.976+0000 7fb3f9200640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3d406b4d0 0x7fb3d406d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8000f60 tx=0x7fb3e8007a80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:26.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:26.979+0000 7fb3f0c00640 1 -- 172.21.15.62:0/2211336169 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3e0032e20 con 0x7fb3f4145140 2024-04-25T09:09:27.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.176+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb3f41f1a30 con 0x7fb3f4145140 2024-04-25T09:09:27.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.910+0000 7fb3f0c00640 1 -- 172.21.15.62:0/2211336169 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb3e0079cb0 con 0x7fb3f4145140 2024-04-25T09:09:27.898 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:09:27.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.913+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3d406b4d0 msgr2=0x7fb3d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:27.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.913+0000 7fb3fa600640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3d406b4d0 0x7fb3d406d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8000f60 tx=0x7fb3e8007a80 comp rx=0 tx=0).stop 2024-04-25T09:09:27.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.913+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f4145140 msgr2=0x7fb3f41e4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.913+0000 7fb3fa600640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f4145140 0x7fb3f41e4ee0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0000c00 tx=0x7fb3e00046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.914+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 shutdown_connections 2024-04-25T09:09:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.914+0000 7fb3fa600640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3d406b4d0 0x7fb3d406d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.914+0000 7fb3fa600640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3f4145140 0x7fb3f41e4ee0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.914+0000 7fb3fa600640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3f415beb0 0x7fb3f41e5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.914+0000 7fb3fa600640 1 --2- 172.21.15.62:0/2211336169 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3f406c440 0x7fb3f4140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.914+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 >> 172.21.15.62:0/2211336169 conn(0x7fb3f4066ad0 msgr2=0x7fb3f40ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.914+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 shutdown_connections 2024-04-25T09:09:27.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:27.914+0000 7fb3fa600640 1 -- 172.21.15.62:0/2211336169 wait complete. 2024-04-25T09:09:27.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:09:28.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.138+0000 7fb83e000640 1 Processor -- start 2024-04-25T09:09:28.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.139+0000 7fb83e000640 1 -- start start 2024-04-25T09:09:28.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.140+0000 7fb83e000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb838159680 0x7fb83815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:28.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.140+0000 7fb83e000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380cc960 0x7fb8380b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:28.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.140+0000 7fb83e000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83815d090 con 0x7fb8380b0eb0 2024-04-25T09:09:28.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.140+0000 7fb83e000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8380b1300 con 0x7fb838159680 2024-04-25T09:09:28.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.140+0000 7fb83e000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83806d000 con 0x7fb8380cc960 2024-04-25T09:09:28.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.141+0000 7fb83cc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb838159680 0x7fb83815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:28.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.141+0000 7fb83cc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb838159680 0x7fb83815cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:38616/0 (socket says 172.21.15.62:38616) 2024-04-25T09:09:28.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.141+0000 7fb83cc00640 1 -- 172.21.15.62:0/3420092429 learned_addr learned my addr 172.21.15.62:0/3420092429 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:28.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.141+0000 7fb837e00640 1 --2- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380cc960 0x7fb8380b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:28.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.142+0000 7fb83cc00640 1 -- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380cc960 msgr2=0x7fb8380b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:28.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.142+0000 7fb83cc00640 1 --2- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380cc960 0x7fb8380b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.142+0000 7fb83cc00640 1 -- 172.21.15.62:0/3420092429 >> v1:172.21.15.121:6789/0 conn(0x7fb8380b0eb0 legacy=0x7fb83815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:28.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.142+0000 7fb83cc00640 1 -- 172.21.15.62:0/3420092429 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83806d490 con 0x7fb838159680 2024-04-25T09:09:28.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.142+0000 7fb837e00640 1 --2- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380cc960 0x7fb8380b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:09:28.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.143+0000 7fb83cc00640 1 --2- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 0x7fb83815cb50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009870 tx=0x7fb82c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6786de0b709dc443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:28.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.144+0000 7fb837400640 1 -- 172.21.15.62:0/3420092429 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c037070 con 0x7fb838159680 2024-04-25T09:09:28.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.144+0000 7fb837400640 1 -- 172.21.15.62:0/3420092429 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c032440 con 0x7fb838159680 2024-04-25T09:09:28.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.144+0000 7fb837400640 1 -- 172.21.15.62:0/3420092429 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c032810 con 0x7fb838159680 2024-04-25T09:09:28.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.145+0000 7fb83e000640 1 -- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 msgr2=0x7fb83815cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.145+0000 7fb83e000640 1 --2- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 0x7fb83815cb50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009870 tx=0x7fb82c02a250 comp rx=0 tx=0).stop 2024-04-25T09:09:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.145+0000 7fb83e000640 1 -- 172.21.15.62:0/3420092429 shutdown_connections 2024-04-25T09:09:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.145+0000 7fb83e000640 1 --2- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380cc960 0x7fb8380b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.146+0000 7fb83e000640 1 --2- 172.21.15.62:0/3420092429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 0x7fb83815cb50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.146+0000 7fb83e000640 1 -- 172.21.15.62:0/3420092429 >> 172.21.15.62:0/3420092429 conn(0x7fb8380668f0 msgr2=0x7fb838066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.146+0000 7fb83e000640 1 -- 172.21.15.62:0/3420092429 shutdown_connections 2024-04-25T09:09:28.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.146+0000 7fb83e000640 1 -- 172.21.15.62:0/3420092429 wait complete. 2024-04-25T09:09:28.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.148+0000 7fb83e000640 1 Processor -- start 2024-04-25T09:09:28.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.148+0000 7fb83e000640 1 -- start start 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.149+0000 7fb83e000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380b0eb0 0x7fb838155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.149+0000 7fb83e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8380cc960 0x7fb8380a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.149+0000 7fb83e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 0x7fb8380ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.149+0000 7fb83e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83806c9b0 con 0x7fb8380cc960 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.149+0000 7fb83e000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83806c6d0 con 0x7fb838159680 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.149+0000 7fb83e000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83806cc90 con 0x7fb8380b0eb0 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb837e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8380cc960 0x7fb8380a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb837e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8380cc960 0x7fb8380a9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45298/0 (socket says 172.21.15.62:45298) 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb837e00640 1 -- 172.21.15.62:0/3139839172 learned_addr learned my addr 172.21.15.62:0/3139839172 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:28.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb83d600640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 0x7fb8380ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb83cc00640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380b0eb0 0x7fb838155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:28.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb837e00640 1 -- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380b0eb0 msgr2=0x7fb838155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb837e00640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380b0eb0 0x7fb838155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb837e00640 1 -- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 msgr2=0x7fb8380ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.150+0000 7fb837e00640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 0x7fb8380ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.151+0000 7fb837e00640 1 -- 172.21.15.62:0/3139839172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8380a9eb0 con 0x7fb8380cc960 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.151+0000 7fb837e00640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8380cc960 0x7fb8380a9970 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e9b0 tx=0x7fb82400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.151+0000 7fb834c00640 1 -- 172.21.15.62:0/3139839172 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824017070 con 0x7fb8380cc960 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.151+0000 7fb834c00640 1 -- 172.21.15.62:0/3139839172 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82400ca00 con 0x7fb8380cc960 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.151+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8380ab260 con 0x7fb8380cc960 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.151+0000 7fb834c00640 1 -- 172.21.15.62:0/3139839172 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82400cdf0 con 0x7fb8380cc960 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.152+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8380ab820 con 0x7fb8380cc960 2024-04-25T09:09:28.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.153+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb838158ca0 con 0x7fb8380cc960 2024-04-25T09:09:28.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.154+0000 7fb834c00640 1 -- 172.21.15.62:0/3139839172 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb82401a070 con 0x7fb8380cc960 2024-04-25T09:09:28.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.155+0000 7fb834c00640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb81806b4d0 0x7fb81806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:28.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.155+0000 7fb834c00640 1 -- 172.21.15.62:0/3139839172 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb824089e40 con 0x7fb8380cc960 2024-04-25T09:09:28.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.156+0000 7fb83cc00640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb81806b4d0 0x7fb81806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:28.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.157+0000 7fb83cc00640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb81806b4d0 0x7fb81806d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb82c034000 tx=0x7fb82c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:28.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.161+0000 7fb834c00640 1 -- 172.21.15.62:0/3139839172 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb838158ca0 con 0x7fb8380cc960 2024-04-25T09:09:28.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.365+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb83806e020 con 0x7fb8380cc960 2024-04-25T09:09:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.929+0000 7fb834c00640 1 -- 172.21.15.62:0/3139839172 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb824058930 con 0x7fb8380cc960 2024-04-25T09:09:28.917 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.932+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb81806b4d0 msgr2=0x7fb81806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.932+0000 7fb83e000640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb81806b4d0 0x7fb81806d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb82c034000 tx=0x7fb82c02e000 comp rx=0 tx=0).stop 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.932+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8380cc960 msgr2=0x7fb8380a9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.932+0000 7fb83e000640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8380cc960 0x7fb8380a9970 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e9b0 tx=0x7fb82400ee70 comp rx=0 tx=0).stop 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.933+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 shutdown_connections 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.933+0000 7fb83e000640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb81806b4d0 0x7fb81806d990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.933+0000 7fb83e000640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8380cc960 0x7fb8380a9970 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.933+0000 7fb83e000640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb838159680 0x7fb8380ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.933+0000 7fb83e000640 1 --2- 172.21.15.62:0/3139839172 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb8380b0eb0 0x7fb838155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.933+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 >> 172.21.15.62:0/3139839172 conn(0x7fb8380668f0 msgr2=0x7fb8380ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:28.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.933+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 shutdown_connections 2024-04-25T09:09:28.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:28.934+0000 7fb83e000640 1 -- 172.21.15.62:0/3139839172 wait complete. 2024-04-25T09:09:28.958 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:09:29.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.149+0000 7fc3e6800640 1 Processor -- start 2024-04-25T09:09:29.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.149+0000 7fc3e6800640 1 -- start start 2024-04-25T09:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.151+0000 7fc3e6800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e006dc40 0x7fc3e006e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.151+0000 7fc3e6800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc3e015af50 0x7fc3e015c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.151+0000 7fc3e6800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e006e560 con 0x7fc3e0155150 2024-04-25T09:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.151+0000 7fc3e6800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e015cbb0 con 0x7fc3e015af50 2024-04-25T09:09:29.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.151+0000 7fc3e6800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e015ce90 con 0x7fc3e006dc40 2024-04-25T09:09:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.151+0000 7fc3dea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc3e0155150 0x7fc3e015dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40766/0 (socket says 172.21.15.62:40766) 2024-04-25T09:09:29.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.151+0000 7fc3dea00640 1 -- 172.21.15.62:0/2530389328 learned_addr learned my addr 172.21.15.62:0/2530389328 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3df400640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e006dc40 0x7fc3e006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3dfe00640 1 --2- 172.21.15.62:0/2530389328 >> v2:172.21.15.121:3300/0 conn(0x7fc3e015af50 0x7fc3e015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431085002 0 0) 0x7fc3e006e560 con 0x7fc3e0155150 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c4005730 con 0x7fc3e0155150 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1850910222 0 0) 0x7fc3c4005730 con 0x7fc3e0155150 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e006dc40 msgr2=0x7fc3e006e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e006dc40 0x7fc3e006e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 >> v2:172.21.15.121:3300/0 conn(0x7fc3e015af50 msgr2=0x7fc3e015c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 --2- 172.21.15.62:0/2530389328 >> v2:172.21.15.121:3300/0 conn(0x7fc3e015af50 0x7fc3e015c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e015d170 con 0x7fc3e0155150 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc3cc001670 con 0x7fc3e0155150 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 >> v1:172.21.15.62:6789/0 conn(0x7fc3e0155150 legacy=0x7fc3e015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3c4005bc0 0x7fc3c4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.152+0000 7fc3de000640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e006dc40 0x7fc3c4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3de000640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4006f50 0x7fc3c40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e015cbb0 con 0x7fc3c4005bc0 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e015ce90 con 0x7fc3c4006f50 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c400dc00 con 0x7fc3e006dc40 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3df400640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3c4005bc0 0x7fc3c4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3dea00640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e006dc40 0x7fc3c4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3dfe00640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4006f50 0x7fc3c40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3df400640 1 -- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e006dc40 msgr2=0x7fc3c4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3df400640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e006dc40 0x7fc3c4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3df400640 1 -- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4006f50 msgr2=0x7fc3c40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3df400640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4006f50 0x7fc3c40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.153+0000 7fc3df400640 1 -- 172.21.15.62:0/2530389328 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c4005730 con 0x7fc3c4005bc0 2024-04-25T09:09:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.155+0000 7fc3df400640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3c4005bc0 0x7fc3c4005fe0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009ac0 tx=0x7fc3c4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c0e2ec3be2b409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.155+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0036070 con 0x7fc3c4005bc0 2024-04-25T09:09:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.155+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0031400 con 0x7fc3c4005bc0 2024-04-25T09:09:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.155+0000 7fc3de000640 1 -- 172.21.15.62:0/2530389328 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0009690 con 0x7fc3c4005bc0 2024-04-25T09:09:29.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.156+0000 7fc3e6800640 1 -- 172.21.15.62:0/2530389328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3c4005bc0 msgr2=0x7fc3c4005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:29.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.156+0000 7fc3e6800640 1 --2- 172.21.15.62:0/2530389328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3c4005bc0 0x7fc3c4005fe0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009ac0 tx=0x7fc3c4007a40 comp rx=0 tx=0).stop 2024-04-25T09:09:29.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.156+0000 7fc3dfe00640 1 -- 172.21.15.62:0/2530389328 reap_dead start 2024-04-25T09:09:29.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.156+0000 7fc3e6800640 1 -- 172.21.15.62:0/2530389328 shutdown_connections 2024-04-25T09:09:29.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.156+0000 7fc3e6800640 1 -- 172.21.15.62:0/2530389328 >> 172.21.15.62:0/2530389328 conn(0x7fc3e00b1410 msgr2=0x7fc3e00b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:29.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.156+0000 7fc3e6800640 1 -- 172.21.15.62:0/2530389328 shutdown_connections 2024-04-25T09:09:29.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.156+0000 7fc3e6800640 1 -- 172.21.15.62:0/2530389328 wait complete. 2024-04-25T09:09:29.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.158+0000 7fc3e6800640 1 Processor -- start 2024-04-25T09:09:29.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.158+0000 7fc3e6800640 1 -- start start 2024-04-25T09:09:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3e6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4005bc0 0x7fc3e014d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3e6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0071770 0x7fc3e0071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3e6800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e00aa750 0x7fc3e00aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3e6800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e015c8a0 con 0x7fc3e0071770 2024-04-25T09:09:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3e6800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e015d730 con 0x7fc3c4005bc0 2024-04-25T09:09:29.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3e6800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e006e020 con 0x7fc3e00aa750 2024-04-25T09:09:29.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3dea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0071770 0x7fc3e0071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3dea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0071770 0x7fc3e0071bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45320/0 (socket says 172.21.15.62:45320) 2024-04-25T09:09:29.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.159+0000 7fc3dea00640 1 -- 172.21.15.62:0/4045513681 learned_addr learned my addr 172.21.15.62:0/4045513681 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3df400640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4005bc0 0x7fc3e014d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3dfe00640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e00aa750 0x7fc3e00aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3dea00640 1 -- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e00aa750 msgr2=0x7fc3e00aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3dea00640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e00aa750 0x7fc3e00aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3dea00640 1 -- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4005bc0 msgr2=0x7fc3e014d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3dea00640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4005bc0 0x7fc3e014d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3dea00640 1 -- 172.21.15.62:0/4045513681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e00ab480 con 0x7fc3e0071770 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3dfe00640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e00aa750 0x7fc3e00aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3df400640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4005bc0 0x7fc3e014d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3dea00640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0071770 0x7fc3e0071bd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00e780 tx=0x7fc3cc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.160+0000 7fc3d7400640 1 -- 172.21.15.62:0/4045513681 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc00c6c0 con 0x7fc3e0071770 2024-04-25T09:09:29.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.161+0000 7fc3d7400640 1 -- 172.21.15.62:0/4045513681 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3cc018070 con 0x7fc3e0071770 2024-04-25T09:09:29.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.161+0000 7fc3d7400640 1 -- 172.21.15.62:0/4045513681 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc00cac0 con 0x7fc3e0071770 2024-04-25T09:09:29.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.161+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e00abbe0 con 0x7fc3e0071770 2024-04-25T09:09:29.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.161+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e00ac1a0 con 0x7fc3e0071770 2024-04-25T09:09:29.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.162+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e0072110 con 0x7fc3e0071770 2024-04-25T09:09:29.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.163+0000 7fc3d7400640 1 -- 172.21.15.62:0/4045513681 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3cc00cc60 con 0x7fc3e0071770 2024-04-25T09:09:29.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.164+0000 7fc3d7400640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3c006b4d0 0x7fc3c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:29.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.164+0000 7fc3d7400640 1 -- 172.21.15.62:0/4045513681 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc3cc089ff0 con 0x7fc3e0071770 2024-04-25T09:09:29.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.165+0000 7fc3df400640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3c006b4d0 0x7fc3c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:29.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.166+0000 7fc3df400640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3c006b4d0 0x7fc3c006d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0033000 tx=0x7fc3d0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:29.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.169+0000 7fc3d7400640 1 -- 172.21.15.62:0/4045513681 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3cc058ee0 con 0x7fc3e0071770 2024-04-25T09:09:29.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.407+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc3e01539f0 con 0x7fc3e0071770 2024-04-25T09:09:29.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.941+0000 7fc3d7400640 1 -- 172.21.15.62:0/4045513681 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v50) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc3cc0587e0 con 0x7fc3e0071770 2024-04-25T09:09:29.931 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T09:09:29.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.948+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3c006b4d0 msgr2=0x7fc3c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:29.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.948+0000 7fc3e6800640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3c006b4d0 0x7fc3c006d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0033000 tx=0x7fc3d0008030 comp rx=0 tx=0).stop 2024-04-25T09:09:29.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.948+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0071770 msgr2=0x7fc3e0071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:29.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.948+0000 7fc3e6800640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0071770 0x7fc3e0071bd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00e780 tx=0x7fc3cc00ec40 comp rx=0 tx=0).stop 2024-04-25T09:09:29.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.949+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 shutdown_connections 2024-04-25T09:09:29.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.949+0000 7fc3e6800640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc3c006b4d0 0x7fc3c006d990 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0033000 tx=0x7fc3d0008030 comp rx=0 tx=0).stop 2024-04-25T09:09:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.949+0000 7fc3e6800640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc3e0071770 0x7fc3e0071bd0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.949+0000 7fc3e6800640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc3e00aa750 0x7fc3e00aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.949+0000 7fc3e6800640 1 --2- 172.21.15.62:0/4045513681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3c4005bc0 0x7fc3e014d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.949+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 >> 172.21.15.62:0/4045513681 conn(0x7fc3e00b1410 msgr2=0x7fc3e006cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.952+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 shutdown_connections 2024-04-25T09:09:29.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:29.952+0000 7fc3e6800640 1 -- 172.21.15.62:0/4045513681 wait complete. 2024-04-25T09:09:29.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.158+0000 7f47bfe00640 1 Processor -- start 2024-04-25T09:09:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.158+0000 7f47bfe00640 1 -- start start 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.160+0000 7f47bfe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f47c015f450 0x7f47c00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.160+0000 7f47bfe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c015faa0 con 0x7f47c006d900 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.160+0000 7f47bfe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c015fd80 con 0x7f47c0144c10 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.160+0000 7f47bfe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c0144ff0 con 0x7f47c015f450 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.160+0000 7f47bf400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f47c015f450 0x7f47c00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.160+0000 7f47bf400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f47c015f450 0x7f47c00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45330/0 (socket says 172.21.15.62:45330) 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.160+0000 7f47b7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f47c006d900 0x7f47c00b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40770/0 (socket says 172.21.15.62:40770) 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.160+0000 7f47bf400640 1 -- 172.21.15.62:0/1790599714 learned_addr learned my addr 172.21.15.62:0/1790599714 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.161+0000 7f47bf400640 1 -- 172.21.15.62:0/1790599714 >> v1:172.21.15.121:6789/0 conn(0x7f47c0144c10 legacy=0x7f47c006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.161+0000 7f47bf400640 1 -- 172.21.15.62:0/1790599714 >> v1:172.21.15.62:6789/0 conn(0x7f47c006d900 legacy=0x7f47c00b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.161+0000 7f47bf400640 1 -- 172.21.15.62:0/1790599714 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0160060 con 0x7f47c015f450 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.162+0000 7f47bf400640 1 --2- 172.21.15.62:0/1790599714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c015f450 0x7f47c00cee40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009e50 tx=0x7f47b802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35937d29f5af6665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.162+0000 7f47be000640 1 -- 172.21.15.62:0/1790599714 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8036070 con 0x7f47c015f450 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.162+0000 7f47be000640 1 -- 172.21.15.62:0/1790599714 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b8031600 con 0x7f47c015f450 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.162+0000 7f47be000640 1 -- 172.21.15.62:0/1790599714 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80319d0 con 0x7f47c015f450 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.162+0000 7f47bfe00640 1 -- 172.21.15.62:0/1790599714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c015f450 msgr2=0x7f47c00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.162+0000 7f47bfe00640 1 --2- 172.21.15.62:0/1790599714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c015f450 0x7f47c00cee40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009e50 tx=0x7f47b802a6d0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.163+0000 7f47bfe00640 1 -- 172.21.15.62:0/1790599714 shutdown_connections 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.163+0000 7f47bfe00640 1 --2- 172.21.15.62:0/1790599714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c015f450 0x7f47c00cee40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.163+0000 7f47bfe00640 1 -- 172.21.15.62:0/1790599714 >> 172.21.15.62:0/1790599714 conn(0x7f47c00b14d0 msgr2=0x7f47c00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.163+0000 7f47bfe00640 1 -- 172.21.15.62:0/1790599714 shutdown_connections 2024-04-25T09:09:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.163+0000 7f47bfe00640 1 -- 172.21.15.62:0/1790599714 wait complete. 2024-04-25T09:09:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.164+0000 7f47bfe00640 1 Processor -- start 2024-04-25T09:09:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.164+0000 7f47bfe00640 1 -- start start 2024-04-25T09:09:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.165+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c006d900 0x7f47c00cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.165+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47c0144c10 0x7f47c00c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47c015f450 0x7f47c00c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bfe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c00b5030 con 0x7f47c006d900 2024-04-25T09:09:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bfe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c015f880 con 0x7f47c0144c10 2024-04-25T09:09:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bfe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c00b51b0 con 0x7f47c015f450 2024-04-25T09:09:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c006d900 0x7f47c00cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c006d900 0x7f47c00cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45342/0 (socket says 172.21.15.62:45342) 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 -- 172.21.15.62:0/2712811148 learned_addr learned my addr 172.21.15.62:0/2712811148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 -- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47c015f450 msgr2=0x7f47c00c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47c015f450 0x7f47c00c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 -- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47c0144c10 msgr2=0x7f47c00c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47c0144c10 0x7f47c00c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 -- 172.21.15.62:0/2712811148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c01e7d60 con 0x7f47c006d900 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.166+0000 7f47bea00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c006d900 0x7f47c00cb2d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c460 tx=0x7f47b000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.167+0000 7f47b6a00640 1 -- 172.21.15.62:0/2712811148 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0016020 con 0x7f47c006d900 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.167+0000 7f47b6a00640 1 -- 172.21.15.62:0/2712811148 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0012870 con 0x7f47c006d900 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.167+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c012c000 con 0x7f47c006d900 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.167+0000 7f47b6a00640 1 -- 172.21.15.62:0/2712811148 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0012c60 con 0x7f47c006d900 2024-04-25T09:09:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.167+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c00cb810 con 0x7f47c006d900 2024-04-25T09:09:30.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.168+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c012dc40 con 0x7f47c006d900 2024-04-25T09:09:30.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.169+0000 7f47b6a00640 1 -- 172.21.15.62:0/2712811148 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47b0019020 con 0x7f47c006d900 2024-04-25T09:09:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.170+0000 7f47b6a00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f479c06b4d0 0x7f479c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.170+0000 7f47b6a00640 1 -- 172.21.15.62:0/2712811148 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47b0089fc0 con 0x7f47c006d900 2024-04-25T09:09:30.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.170+0000 7f47b7e00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f479c06b4d0 0x7f479c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.172+0000 7f47b7e00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f479c06b4d0 0x7f479c06d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000f60 tx=0x7f47a8007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:30.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.174+0000 7f47b6a00640 1 -- 172.21.15.62:0/2712811148 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47b0020030 con 0x7f47c006d900 2024-04-25T09:09:30.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.369+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47c012bdf0 con 0x7f47c006d900 2024-04-25T09:09:30.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.372+0000 7f47b6a00640 1 -- 172.21.15.62:0/2712811148 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f47b0058da0 con 0x7f47c006d900 2024-04-25T09:09:30.361 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:30.361 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:29.409594+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:27.235650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:28.419955+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.374+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f479c06b4d0 msgr2=0x7f479c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.374+0000 7f47bfe00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f479c06b4d0 0x7f479c06d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000f60 tx=0x7f47a8007a00 comp rx=0 tx=0).stop 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.374+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c006d900 msgr2=0x7f47c00cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.374+0000 7f47bfe00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c006d900 0x7f47c00cb2d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c460 tx=0x7f47b000c920 comp rx=0 tx=0).stop 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.374+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 shutdown_connections 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.374+0000 7f47bfe00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f479c06b4d0 0x7f479c06d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.375+0000 7f47bfe00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f47c015f450 0x7f47c00c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.375+0000 7f47bfe00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47c0144c10 0x7f47c00c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.375+0000 7f47bfe00640 1 --2- 172.21.15.62:0/2712811148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f47c006d900 0x7f47c00cb2d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.375+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 >> 172.21.15.62:0/2712811148 conn(0x7f47c00b14d0 msgr2=0x7f47c013f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:30.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.375+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 shutdown_connections 2024-04-25T09:09:30.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.375+0000 7f47bfe00640 1 -- 172.21.15.62:0/2712811148 wait complete. 2024-04-25T09:09:30.381 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:30.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.600+0000 7f4e34a00640 1 Processor -- start 2024-04-25T09:09:30.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.601+0000 7f4e34a00640 1 -- start start 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.602+0000 7f4e34a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e30144c10 0x7f4e3015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.602+0000 7f4e34a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4e300cee40 0x7f4e3006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.602+0000 7f4e34a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4e3006d730 0x7f4e300b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.602+0000 7f4e34a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e30144ff0 con 0x7f4e3006d730 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.603+0000 7f4e34a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e30160340 con 0x7f4e300cee40 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.603+0000 7f4e34a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300b41c0 con 0x7f4e30144c10 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.603+0000 7f4e2f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4e3006d730 0x7f4e300b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.603+0000 7f4e2f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4e3006d730 0x7f4e300b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45358/0 (socket says 172.21.15.62:45358) 2024-04-25T09:09:30.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.603+0000 7f4e2f200640 1 -- 172.21.15.62:0/3664958575 learned_addr learned my addr 172.21.15.62:0/3664958575 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:30.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.603+0000 7f4e2e800640 1 --2- 172.21.15.62:0/3664958575 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e30144c10 0x7f4e3015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.603+0000 7f4e25e00640 1 --2- 172.21.15.62:0/3664958575 >> v2:172.21.15.121:3300/0 conn(0x7f4e300cee40 0x7f4e3006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.603+0000 7f4e2f200640 1 -- 172.21.15.62:0/3664958575 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e30144c10 msgr2=0x7f4e3015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:30.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.604+0000 7f4e2f200640 1 --2- 172.21.15.62:0/3664958575 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e30144c10 0x7f4e3015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.604+0000 7f4e2f200640 1 -- 172.21.15.62:0/3664958575 >> v2:172.21.15.121:3300/0 conn(0x7f4e300cee40 msgr2=0x7f4e3006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.604+0000 7f4e2f200640 1 --2- 172.21.15.62:0/3664958575 >> v2:172.21.15.121:3300/0 conn(0x7f4e300cee40 0x7f4e3006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.604+0000 7f4e2f200640 1 -- 172.21.15.62:0/3664958575 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3015faa0 con 0x7f4e3006d730 2024-04-25T09:09:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.605+0000 7f4e2f200640 1 --2- 172.21.15.62:0/3664958575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e3006d730 0x7f4e300b2da0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4e3006bae0 tx=0x7f4e2802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a21aa2447cd3f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.605+0000 7f4e2de00640 1 -- 172.21.15.62:0/3664958575 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28036070 con 0x7f4e3006d730 2024-04-25T09:09:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.605+0000 7f4e2de00640 1 -- 172.21.15.62:0/3664958575 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e28031600 con 0x7f4e3006d730 2024-04-25T09:09:30.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.605+0000 7f4e2de00640 1 -- 172.21.15.62:0/3664958575 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e280319d0 con 0x7f4e3006d730 2024-04-25T09:09:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 -- 172.21.15.62:0/3664958575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e3006d730 msgr2=0x7f4e300b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 --2- 172.21.15.62:0/3664958575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e3006d730 0x7f4e300b2da0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4e3006bae0 tx=0x7f4e2802a610 comp rx=0 tx=0).stop 2024-04-25T09:09:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 -- 172.21.15.62:0/3664958575 shutdown_connections 2024-04-25T09:09:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 --2- 172.21.15.62:0/3664958575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e3006d730 0x7f4e300b2da0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 --2- 172.21.15.62:0/3664958575 >> v2:172.21.15.121:3300/0 conn(0x7f4e300cee40 0x7f4e3006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 --2- 172.21.15.62:0/3664958575 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e30144c10 0x7f4e3015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 -- 172.21.15.62:0/3664958575 >> 172.21.15.62:0/3664958575 conn(0x7f4e300b14d0 msgr2=0x7f4e300b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 -- 172.21.15.62:0/3664958575 shutdown_connections 2024-04-25T09:09:30.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.606+0000 7f4e34a00640 1 -- 172.21.15.62:0/3664958575 wait complete. 2024-04-25T09:09:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.608+0000 7f4e34a00640 1 Processor -- start 2024-04-25T09:09:30.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.608+0000 7f4e34a00640 1 -- start start 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e34a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e3006d730 0x7f4e3013ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e34a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e300cee40 0x7f4e3013d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e34a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e30144c10 0x7f4e30149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e34a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300cf2a0 con 0x7f4e30144c10 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e34a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e30160060 con 0x7f4e300cee40 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e34a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300b32e0 con 0x7f4e3006d730 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e30144c10 0x7f4e30149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e30144c10 0x7f4e30149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45366/0 (socket says 172.21.15.62:45366) 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2f200640 1 -- 172.21.15.62:0/726432788 learned_addr learned my addr 172.21.15.62:0/726432788 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2e800640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e3006d730 0x7f4e3013ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e25e00640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e300cee40 0x7f4e3013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2f200640 1 -- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e3006d730 msgr2=0x7f4e3013ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2f200640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e3006d730 0x7f4e3013ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2f200640 1 -- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e300cee40 msgr2=0x7f4e3013d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2f200640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e300cee40 0x7f4e3013d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.610+0000 7f4e2f200640 1 -- 172.21.15.62:0/726432788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e301e67f0 con 0x7f4e30144c10 2024-04-25T09:09:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.611+0000 7f4e25e00640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e300cee40 0x7f4e3013d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:30.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.611+0000 7f4e2f200640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e30144c10 0x7f4e30149db0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009f40 tx=0x7f4e28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.611+0000 7f4e27400640 1 -- 172.21.15.62:0/726432788 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28041020 con 0x7f4e30144c10 2024-04-25T09:09:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.611+0000 7f4e27400640 1 -- 172.21.15.62:0/726432788 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e280077a0 con 0x7f4e30144c10 2024-04-25T09:09:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.611+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e301e6ad0 con 0x7f4e30144c10 2024-04-25T09:09:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.611+0000 7f4e27400640 1 -- 172.21.15.62:0/726432788 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28036040 con 0x7f4e30144c10 2024-04-25T09:09:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.611+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e301480e0 con 0x7f4e30144c10 2024-04-25T09:09:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.612+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3015b6f0 con 0x7f4e30144c10 2024-04-25T09:09:30.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.613+0000 7f4e27400640 1 -- 172.21.15.62:0/726432788 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e28044020 con 0x7f4e30144c10 2024-04-25T09:09:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.614+0000 7f4e27400640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4e0c06b4d0 0x7f4e0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:30.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.614+0000 7f4e27400640 1 -- 172.21.15.62:0/726432788 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e280aaee0 con 0x7f4e30144c10 2024-04-25T09:09:30.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.615+0000 7f4e2e800640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4e0c06b4d0 0x7f4e0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:30.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.616+0000 7f4e2e800640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4e0c06b4d0 0x7f4e0c06d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009a30 tx=0x7f4e20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:30.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.619+0000 7f4e27400640 1 -- 172.21.15.62:0/726432788 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e2804b030 con 0x7f4e30144c10 2024-04-25T09:09:30.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.820+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e3004ef20 con 0x7f4e30144c10 2024-04-25T09:09:30.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.823+0000 7f4e27400640 1 -- 172.21.15.62:0/726432788 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16609 (secure 0 0 0) 0x7f4e28079d80 con 0x7f4e30144c10 2024-04-25T09:09:30.811 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:30.811 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:29.409594+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:27.235650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:28.419955+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:30.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4e0c06b4d0 msgr2=0x7f4e0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4e0c06b4d0 0x7f4e0c06d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009a30 tx=0x7f4e20008040 comp rx=0 tx=0).stop 2024-04-25T09:09:30.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e30144c10 msgr2=0x7f4e30149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:30.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e30144c10 0x7f4e30149db0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009f40 tx=0x7f4e28004a40 comp rx=0 tx=0).stop 2024-04-25T09:09:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 shutdown_connections 2024-04-25T09:09:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4e0c06b4d0 0x7f4e0c06d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e30144c10 0x7f4e30149db0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4e300cee40 0x7f4e3013d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 --2- 172.21.15.62:0/726432788 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4e3006d730 0x7f4e3013ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 >> 172.21.15.62:0/726432788 conn(0x7f4e300b14d0 msgr2=0x7f4e3015caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 shutdown_connections 2024-04-25T09:09:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:30.826+0000 7f4e34a00640 1 -- 172.21.15.62:0/726432788 wait complete. 2024-04-25T09:09:30.835 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:09:31.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.064+0000 7f2bd7e00640 1 Processor -- start 2024-04-25T09:09:31.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.064+0000 7f2bd7e00640 1 -- start start 2024-04-25T09:09:31.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.066+0000 7f2bd7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2bd80ccde0 0x7f2bd80ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:31.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.066+0000 7f2bd7e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd80ced40 con 0x7f2bd806d2c0 2024-04-25T09:09:31.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.066+0000 7f2bd7e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd80cd260 con 0x7f2bd80b31a0 2024-04-25T09:09:31.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.066+0000 7f2bd7e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd80cf000 con 0x7f2bd80ccde0 2024-04-25T09:09:31.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.066+0000 7f2bd6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2bd80ccde0 0x7f2bd80ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:31.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.066+0000 7f2bd6a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2bd806d2c0 0x7f2bd806d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40776/0 (socket says 172.21.15.62:40776) 2024-04-25T09:09:31.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.066+0000 7f2bd6a00640 1 -- 172.21.15.62:0/2553869465 learned_addr learned my addr 172.21.15.62:0/2553869465 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:31.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.067+0000 7f2bd6000640 1 -- 172.21.15.62:0/2553869465 >> v1:172.21.15.121:6789/0 conn(0x7f2bd80b31a0 legacy=0x7f2bd80b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:31.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.067+0000 7f2bd6000640 1 -- 172.21.15.62:0/2553869465 >> v1:172.21.15.62:6789/0 conn(0x7f2bd806d2c0 legacy=0x7f2bd806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:31.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.067+0000 7f2bd6000640 1 -- 172.21.15.62:0/2553869465 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80cf2e0 con 0x7f2bd80ccde0 2024-04-25T09:09:31.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.068+0000 7f2bd6000640 1 --2- 172.21.15.62:0/2553869465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80ccde0 0x7f2bd80ce800 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40097a0 tx=0x7f2bc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fbc890606d6289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:31.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.068+0000 7f2bd5600640 1 -- 172.21.15.62:0/2553869465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4037070 con 0x7f2bd80ccde0 2024-04-25T09:09:31.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.068+0000 7f2bd5600640 1 -- 172.21.15.62:0/2553869465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc4032440 con 0x7f2bd80ccde0 2024-04-25T09:09:31.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.068+0000 7f2bd5600640 1 -- 172.21.15.62:0/2553869465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4032810 con 0x7f2bd80ccde0 2024-04-25T09:09:31.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.069+0000 7f2bd7e00640 1 -- 172.21.15.62:0/2553869465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80ccde0 msgr2=0x7f2bd80ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:31.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.069+0000 7f2bd7e00640 1 --2- 172.21.15.62:0/2553869465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80ccde0 0x7f2bd80ce800 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40097a0 tx=0x7f2bc402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.069+0000 7f2bd7e00640 1 -- 172.21.15.62:0/2553869465 shutdown_connections 2024-04-25T09:09:31.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.069+0000 7f2bd7e00640 1 --2- 172.21.15.62:0/2553869465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80ccde0 0x7f2bd80ce800 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.069+0000 7f2bd7e00640 1 -- 172.21.15.62:0/2553869465 >> 172.21.15.62:0/2553869465 conn(0x7f2bd8066f20 msgr2=0x7f2bd80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:31.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.069+0000 7f2bd7e00640 1 -- 172.21.15.62:0/2553869465 shutdown_connections 2024-04-25T09:09:31.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.069+0000 7f2bd7e00640 1 -- 172.21.15.62:0/2553869465 wait complete. 2024-04-25T09:09:31.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.071+0000 7f2bd7e00640 1 Processor -- start 2024-04-25T09:09:31.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.071+0000 7f2bd7e00640 1 -- start start 2024-04-25T09:09:31.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd806d2c0 0x7f2bd8143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:31.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80b31a0 0x7f2bd80c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:31.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bd80ccde0 0x7f2bd80c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:31.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80b2c80 con 0x7f2bd80b31a0 2024-04-25T09:09:31.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80b29a0 con 0x7f2bd806d2c0 2024-04-25T09:09:31.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81448d0 con 0x7f2bd80ccde0 2024-04-25T09:09:31.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80b31a0 0x7f2bd80c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:31.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80b31a0 0x7f2bd80c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45390/0 (socket says 172.21.15.62:45390) 2024-04-25T09:09:31.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.072+0000 7f2bd6000640 1 -- 172.21.15.62:0/3604693402 learned_addr learned my addr 172.21.15.62:0/3604693402 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:31.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.073+0000 7f2bd6a00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd806d2c0 0x7f2bd8143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:31.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.073+0000 7f2bd6000640 1 -- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bd80ccde0 msgr2=0x7f2bd80c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.073+0000 7f2bd6000640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bd80ccde0 0x7f2bd80c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.073+0000 7f2bd6000640 1 -- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd806d2c0 msgr2=0x7f2bd8143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.073+0000 7f2bd6000640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd806d2c0 0x7f2bd8143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.073+0000 7f2bd6000640 1 -- 172.21.15.62:0/3604693402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80c99f0 con 0x7f2bd80b31a0 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.073+0000 7f2bd6a00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd806d2c0 0x7f2bd8143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.073+0000 7f2bd6000640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80b31a0 0x7f2bd80c7100 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4035040 tx=0x7f2bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.074+0000 7f2bcea00640 1 -- 172.21.15.62:0/3604693402 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4042020 con 0x7f2bd80b31a0 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.074+0000 7f2bcea00640 1 -- 172.21.15.62:0/3604693402 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc40309b0 con 0x7f2bd80b31a0 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.074+0000 7f2bcea00640 1 -- 172.21.15.62:0/3604693402 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4037040 con 0x7f2bd80b31a0 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.074+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd80ad8e0 con 0x7f2bd80b31a0 2024-04-25T09:09:31.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.074+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80ae180 con 0x7f2bd80b31a0 2024-04-25T09:09:31.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.075+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd80adcd0 con 0x7f2bd80b31a0 2024-04-25T09:09:31.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.076+0000 7f2bcea00640 1 -- 172.21.15.62:0/3604693402 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bc4045020 con 0x7f2bd80b31a0 2024-04-25T09:09:31.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.077+0000 7f2bcea00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2bb806b4d0 0x7f2bb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:31.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.077+0000 7f2bcea00640 1 -- 172.21.15.62:0/3604693402 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bc40aafd0 con 0x7f2bd80b31a0 2024-04-25T09:09:31.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.078+0000 7f2bd6a00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2bb806b4d0 0x7f2bb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:31.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.079+0000 7f2bd6a00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2bb806b4d0 0x7f2bb806d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8000f60 tx=0x7f2bc8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:31.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.081+0000 7f2bcea00640 1 -- 172.21.15.62:0/3604693402 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bc404c030 con 0x7f2bd80b31a0 2024-04-25T09:09:31.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.325+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2bd806abe0 con 0x7f2bd80b31a0 2024-04-25T09:09:31.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.954+0000 7f2bcea00640 1 -- 172.21.15.62:0/3604693402 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2bc4079ed0 con 0x7f2bd80b31a0 2024-04-25T09:09:31.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.957+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2bb806b4d0 msgr2=0x7f2bb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:31.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.957+0000 7f2bd7e00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2bb806b4d0 0x7f2bb806d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8000f60 tx=0x7f2bc8009040 comp rx=0 tx=0).stop 2024-04-25T09:09:31.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.957+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80b31a0 msgr2=0x7f2bd80c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:31.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.957+0000 7f2bd7e00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80b31a0 0x7f2bd80c7100 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4035040 tx=0x7f2bc40046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.958+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 shutdown_connections 2024-04-25T09:09:31.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.958+0000 7f2bd7e00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2bb806b4d0 0x7f2bb806d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.958+0000 7f2bd7e00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bd80b31a0 0x7f2bd80c7100 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.958+0000 7f2bd7e00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2bd80ccde0 0x7f2bd80c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.958+0000 7f2bd7e00640 1 --2- 172.21.15.62:0/3604693402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bd806d2c0 0x7f2bd8143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:31.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.958+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 >> 172.21.15.62:0/3604693402 conn(0x7f2bd8066f20 msgr2=0x7f2bd815bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:31.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.958+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 shutdown_connections 2024-04-25T09:09:31.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:31.958+0000 7f2bd7e00640 1 -- 172.21.15.62:0/3604693402 wait complete. 2024-04-25T09:09:31.968 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:09:31.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:09:32.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.193+0000 7f15b8c00640 1 Processor -- start 2024-04-25T09:09:32.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.194+0000 7f15b8c00640 1 -- start start 2024-04-25T09:09:32.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.195+0000 7f15b8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 0x7f15b40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:32.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.195+0000 7f15b8c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f15b40b31a0 0x7f15b40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:32.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.196+0000 7f15b8c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b40ced40 con 0x7f15b406d2c0 2024-04-25T09:09:32.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.196+0000 7f15b8c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b40cd260 con 0x7f15b40b31a0 2024-04-25T09:09:32.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.196+0000 7f15b8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40cf000 con 0x7f15b40ccde0 2024-04-25T09:09:32.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.196+0000 7f15b3400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f15b406d2c0 0x7f15b406d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40786/0 (socket says 172.21.15.62:40786) 2024-04-25T09:09:32.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.196+0000 7f15b3400640 1 -- 172.21.15.62:0/1429930288 learned_addr learned my addr 172.21.15.62:0/1429930288 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:32.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.196+0000 7f15b3e00640 1 --2- 172.21.15.62:0/1429930288 >> v2:172.21.15.62:3300/0 conn(0x7f15b40b31a0 0x7f15b40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:32.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.196+0000 7f15b2a00640 1 --2- 172.21.15.62:0/1429930288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 0x7f15b40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:32.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.196+0000 7f15b2000640 1 -- 172.21.15.62:0/1429930288 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638202292 0 0) 0x7f15b40ced40 con 0x7f15b406d2c0 2024-04-25T09:09:32.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.197+0000 7f15b2000640 1 -- 172.21.15.62:0/1429930288 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1594003520 con 0x7f15b406d2c0 2024-04-25T09:09:32.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.197+0000 7f15b3e00640 1 -- 172.21.15.62:0/1429930288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 msgr2=0x7f15b40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:32.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.197+0000 7f15b3e00640 1 --2- 172.21.15.62:0/1429930288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 0x7f15b40ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.197+0000 7f15b3e00640 1 -- 172.21.15.62:0/1429930288 >> v1:172.21.15.62:6789/0 conn(0x7f15b406d2c0 legacy=0x7f15b406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:32.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.197+0000 7f15b3e00640 1 -- 172.21.15.62:0/1429930288 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40cf2e0 con 0x7f15b40b31a0 2024-04-25T09:09:32.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.198+0000 7f15b3e00640 1 --2- 172.21.15.62:0/1429930288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b40b31a0 0x7f15b40b3f40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0097a0 tx=0x7f15ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f7e784cd25c5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:32.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.198+0000 7f15b2000640 1 -- 172.21.15.62:0/1429930288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac037070 con 0x7f15b40b31a0 2024-04-25T09:09:32.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.198+0000 7f15b2000640 1 -- 172.21.15.62:0/1429930288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac032440 con 0x7f15b40b31a0 2024-04-25T09:09:32.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.198+0000 7f15b2000640 1 -- 172.21.15.62:0/1429930288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0327f0 con 0x7f15b40b31a0 2024-04-25T09:09:32.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.199+0000 7f15b8c00640 1 -- 172.21.15.62:0/1429930288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b40b31a0 msgr2=0x7f15b40b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:32.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.199+0000 7f15b8c00640 1 --2- 172.21.15.62:0/1429930288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b40b31a0 0x7f15b40b3f40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0097a0 tx=0x7f15ac02a180 comp rx=0 tx=0).stop 2024-04-25T09:09:32.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.199+0000 7f15b8c00640 1 -- 172.21.15.62:0/1429930288 shutdown_connections 2024-04-25T09:09:32.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.199+0000 7f15b8c00640 1 --2- 172.21.15.62:0/1429930288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b40b31a0 0x7f15b40b3f40 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.199+0000 7f15b8c00640 1 --2- 172.21.15.62:0/1429930288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 0x7f15b40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.199+0000 7f15b8c00640 1 -- 172.21.15.62:0/1429930288 >> 172.21.15.62:0/1429930288 conn(0x7f15b4066f20 msgr2=0x7f15b40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:32.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.199+0000 7f15b8c00640 1 -- 172.21.15.62:0/1429930288 shutdown_connections 2024-04-25T09:09:32.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.199+0000 7f15b8c00640 1 -- 172.21.15.62:0/1429930288 wait complete. 2024-04-25T09:09:32.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.201+0000 7f15b8c00640 1 Processor -- start 2024-04-25T09:09:32.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.201+0000 7f15b8c00640 1 -- start start 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.202+0000 7f15b8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b406d2c0 0x7f15b415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.202+0000 7f15b8c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40b31a0 0x7f15b4070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.202+0000 7f15b8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 0x7f15b4070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.202+0000 7f15b8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40b2c80 con 0x7f15b406d2c0 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.202+0000 7f15b8c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40b29a0 con 0x7f15b40b31a0 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.202+0000 7f15b8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b41448d0 con 0x7f15b40ccde0 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.202+0000 7f15b3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b406d2c0 0x7f15b415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15b3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b406d2c0 0x7f15b415e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45412/0 (socket says 172.21.15.62:45412) 2024-04-25T09:09:32.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15b3400640 1 -- 172.21.15.62:0/3536039756 learned_addr learned my addr 172.21.15.62:0/3536039756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:32.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15b3400640 1 -- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 msgr2=0x7f15b4070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:32.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15b3400640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 0x7f15b4070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15b3400640 1 -- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40b31a0 msgr2=0x7f15b4070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:32.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15b3400640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40b31a0 0x7f15b4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15b3400640 1 -- 172.21.15.62:0/3536039756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40711f0 con 0x7f15b406d2c0 2024-04-25T09:09:32.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15b3400640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b406d2c0 0x7f15b415e4c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c390 tx=0x7f15a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:32.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15ab400640 1 -- 172.21.15.62:0/3536039756 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4017070 con 0x7f15b406d2c0 2024-04-25T09:09:32.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15ab400640 1 -- 172.21.15.62:0/3536039756 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4005ce0 con 0x7f15b406d2c0 2024-04-25T09:09:32.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.203+0000 7f15ab400640 1 -- 172.21.15.62:0/3536039756 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40072f0 con 0x7f15b406d2c0 2024-04-25T09:09:32.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.204+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b40714c0 con 0x7f15b406d2c0 2024-04-25T09:09:32.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.204+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b4071a80 con 0x7f15b406d2c0 2024-04-25T09:09:32.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.205+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b406abe0 con 0x7f15b406d2c0 2024-04-25T09:09:32.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.206+0000 7f15ab400640 1 -- 172.21.15.62:0/3536039756 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15a401a070 con 0x7f15b406d2c0 2024-04-25T09:09:32.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.207+0000 7f15ab400640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:32.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.207+0000 7f15ab400640 1 -- 172.21.15.62:0/3536039756 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15a4089510 con 0x7f15b406d2c0 2024-04-25T09:09:32.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.207+0000 7f15b2a00640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:32.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.209+0000 7f15b2a00640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f159006b4d0 0x7f159006d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f15b40cb800 tx=0x7f159c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:32.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.211+0000 7f15ab400640 1 -- 172.21.15.62:0/3536039756 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15a4058420 con 0x7f15b406d2c0 2024-04-25T09:09:32.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.450+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f15b413e970 con 0x7f15b406d2c0 2024-04-25T09:09:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.964+0000 7f15ab400640 1 -- 172.21.15.62:0/3536039756 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7f15a4057d20 con 0x7f15b406d2c0 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.968+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f159006b4d0 msgr2=0x7f159006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f159006b4d0 0x7f159006d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f15b40cb800 tx=0x7f159c00b040 comp rx=0 tx=0).stop 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b406d2c0 msgr2=0x7f15b415e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b406d2c0 0x7f15b415e4c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f15a400c390 tx=0x7f15a400c850 comp rx=0 tx=0).stop 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 shutdown_connections 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f15b40ccde0 0x7f15b4070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15b40b31a0 0x7f15b4070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 --2- 172.21.15.62:0/3536039756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f15b406d2c0 0x7f15b415e4c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:32.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 >> 172.21.15.62:0/3536039756 conn(0x7f15b4066f20 msgr2=0x7f15b406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:32.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 shutdown_connections 2024-04-25T09:09:32.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:32.969+0000 7f15b8c00640 1 -- 172.21.15.62:0/3536039756 wait complete. 2024-04-25T09:09:32.981 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:09:32.981 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:33.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.207+0000 7fc157e00640 1 Processor -- start 2024-04-25T09:09:33.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.207+0000 7fc157e00640 1 -- start start 2024-04-25T09:09:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc1580b6a90 0x7fc1580d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc1580b27a0 0x7fc1580d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1580d3110 con 0x7fc1580b56f0 2024-04-25T09:09:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580cedc0 con 0x7fc1580b27a0 2024-04-25T09:09:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580d33f0 con 0x7fc1580b6a90 2024-04-25T09:09:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc1580b27a0 0x7fc1580d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc1580b27a0 0x7fc1580d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45414/0 (socket says 172.21.15.62:45414) 2024-04-25T09:09:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157400640 1 -- 172.21.15.62:0/67783783 learned_addr learned my addr 172.21.15.62:0/67783783 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc14fe00640 1 --2- 172.21.15.62:0/67783783 >> v2:172.21.15.121:3300/0 conn(0x7fc1580b6a90 0x7fc1580d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157400640 1 -- 172.21.15.62:0/67783783 >> v2:172.21.15.121:3300/0 conn(0x7fc1580b6a90 msgr2=0x7fc1580d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:33.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157400640 1 --2- 172.21.15.62:0/67783783 >> v2:172.21.15.121:3300/0 conn(0x7fc1580b6a90 0x7fc1580d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157400640 1 -- 172.21.15.62:0/67783783 >> v1:172.21.15.121:6789/0 conn(0x7fc1580b56f0 legacy=0x7fc158160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:33.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.209+0000 7fc157400640 1 -- 172.21.15.62:0/67783783 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580d36d0 con 0x7fc1580b27a0 2024-04-25T09:09:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.210+0000 7fc157400640 1 --2- 172.21.15.62:0/67783783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b27a0 0x7fc1580d2520 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc15808c570 tx=0x7fc15002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd85cdbce127c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.210+0000 7fc156000640 1 -- 172.21.15.62:0/67783783 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150036070 con 0x7fc1580b27a0 2024-04-25T09:09:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.210+0000 7fc156000640 1 -- 172.21.15.62:0/67783783 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc150031600 con 0x7fc1580b27a0 2024-04-25T09:09:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.210+0000 7fc156000640 1 -- 172.21.15.62:0/67783783 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1500319d0 con 0x7fc1580b27a0 2024-04-25T09:09:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.211+0000 7fc157e00640 1 -- 172.21.15.62:0/67783783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b27a0 msgr2=0x7fc1580d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.211+0000 7fc157e00640 1 --2- 172.21.15.62:0/67783783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b27a0 0x7fc1580d2520 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc15808c570 tx=0x7fc15002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.211+0000 7fc157e00640 1 -- 172.21.15.62:0/67783783 shutdown_connections 2024-04-25T09:09:33.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.211+0000 7fc157e00640 1 --2- 172.21.15.62:0/67783783 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b27a0 0x7fc1580d2520 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.211+0000 7fc157e00640 1 --2- 172.21.15.62:0/67783783 >> v2:172.21.15.121:3300/0 conn(0x7fc1580b6a90 0x7fc1580d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.211+0000 7fc157e00640 1 -- 172.21.15.62:0/67783783 >> 172.21.15.62:0/67783783 conn(0x7fc158066d10 msgr2=0x7fc158067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:33.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.211+0000 7fc157e00640 1 -- 172.21.15.62:0/67783783 shutdown_connections 2024-04-25T09:09:33.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.211+0000 7fc157e00640 1 -- 172.21.15.62:0/67783783 wait complete. 2024-04-25T09:09:33.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.213+0000 7fc157e00640 1 Processor -- start 2024-04-25T09:09:33.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.213+0000 7fc157e00640 1 -- start start 2024-04-25T09:09:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.214+0000 7fc157e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1580b27a0 0x7fc158156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.214+0000 7fc157e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1580b56f0 0x7fc1580c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.214+0000 7fc157e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b6a90 0x7fc1580c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.214+0000 7fc157e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580b2ee0 con 0x7fc1580b6a90 2024-04-25T09:09:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.214+0000 7fc157e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580b2d60 con 0x7fc1580b27a0 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.214+0000 7fc157e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580b5ad0 con 0x7fc1580b56f0 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc157400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b6a90 0x7fc1580c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc157400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b6a90 0x7fc1580c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45418/0 (socket says 172.21.15.62:45418) 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc157400640 1 -- 172.21.15.62:0/3746760873 learned_addr learned my addr 172.21.15.62:0/3746760873 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc14fe00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1580b56f0 0x7fc1580c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc156a00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1580b27a0 0x7fc158156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc157400640 1 -- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1580b56f0 msgr2=0x7fc1580c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc157400640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1580b56f0 0x7fc1580c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc157400640 1 -- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1580b27a0 msgr2=0x7fc158156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc157400640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1580b27a0 0x7fc158156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc157400640 1 -- 172.21.15.62:0/3746760873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580c8180 con 0x7fc1580b6a90 2024-04-25T09:09:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.215+0000 7fc156a00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1580b27a0 0x7fc158156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.216+0000 7fc157400640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b6a90 0x7fc1580c9420 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc150009f40 tx=0x7fc150004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.216+0000 7fc14ea00640 1 -- 172.21.15.62:0/3746760873 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150041020 con 0x7fc1580b6a90 2024-04-25T09:09:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.216+0000 7fc14ea00640 1 -- 172.21.15.62:0/3746760873 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc150007780 con 0x7fc1580b6a90 2024-04-25T09:09:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.216+0000 7fc14ea00640 1 -- 172.21.15.62:0/3746760873 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150036040 con 0x7fc1580b6a90 2024-04-25T09:09:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.216+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1580c8460 con 0x7fc1580b6a90 2024-04-25T09:09:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.216+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1580c8a20 con 0x7fc1580b6a90 2024-04-25T09:09:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.217+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15804ef20 con 0x7fc1580b6a90 2024-04-25T09:09:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.218+0000 7fc14ea00640 1 -- 172.21.15.62:0/3746760873 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc150044020 con 0x7fc1580b6a90 2024-04-25T09:09:33.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.219+0000 7fc14ea00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc13406b4d0 0x7fc13406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.220+0000 7fc14ea00640 1 -- 172.21.15.62:0/3746760873 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1500ab240 con 0x7fc1580b6a90 2024-04-25T09:09:33.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.220+0000 7fc156a00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc13406b4d0 0x7fc13406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.221+0000 7fc156a00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc13406b4d0 0x7fc13406d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc148003eb0 tx=0x7fc148009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:33.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.224+0000 7fc14ea00640 1 -- 172.21.15.62:0/3746760873 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc15004b020 con 0x7fc1580b6a90 2024-04-25T09:09:33.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.466+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1580ce580 con 0x7fc1580b6a90 2024-04-25T09:09:33.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.467+0000 7fc14ea00640 1 -- 172.21.15.62:0/3746760873 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fc15007a0b0 con 0x7fc1580b6a90 2024-04-25T09:09:33.455 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:33.455 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5336,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2992229593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2992229593},{"type":"v1","addr":"172.21.15.62:6835","nonce":2992229593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5342,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/518754003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":518754003},{"type":"v1","addr":"172.21.15.121:6837","nonce":518754003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/6918584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":6918584},{"type":"v1","addr":"172.21.15.121:6836","nonce":6918584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:29.410378+0000","modified":"2024-04-25T09:09:32.452815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2258257596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2258257596},{"type":"v1","addr":"172.21.15.62:6834","nonce":2258257596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:09:33.455 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 54 2024-04-25T09:09:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.469+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc13406b4d0 msgr2=0x7fc13406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.469+0000 7fc157e00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc13406b4d0 0x7fc13406d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc148003eb0 tx=0x7fc148009040 comp rx=0 tx=0).stop 2024-04-25T09:09:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.469+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b6a90 msgr2=0x7fc1580c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.469+0000 7fc157e00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b6a90 0x7fc1580c9420 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc150009f40 tx=0x7fc150004a40 comp rx=0 tx=0).stop 2024-04-25T09:09:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.470+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 shutdown_connections 2024-04-25T09:09:33.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.470+0000 7fc157e00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc13406b4d0 0x7fc13406d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.470+0000 7fc157e00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1580b6a90 0x7fc1580c9420 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.470+0000 7fc157e00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1580b56f0 0x7fc1580c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.470+0000 7fc157e00640 1 --2- 172.21.15.62:0/3746760873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1580b27a0 0x7fc158156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.470+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 >> 172.21.15.62:0/3746760873 conn(0x7fc158066d10 msgr2=0x7fc1580b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.470+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 shutdown_connections 2024-04-25T09:09:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.470+0000 7fc157e00640 1 -- 172.21.15.62:0/3746760873 wait complete. 2024-04-25T09:09:33.477 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:09:33.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.702+0000 7fc61e600640 1 Processor -- start 2024-04-25T09:09:33.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.702+0000 7fc61e600640 1 -- start start 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc61e600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc61815bd10 0x7fc6180cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc61e600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc6180cccf0 0x7fc6180ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc61e600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc6180aed00 0x7fc61815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc61e600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61806d660 con 0x7fc6180aed00 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc61e600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61806ca70 con 0x7fc6180cccf0 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc61e600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61815c0f0 con 0x7fc61815bd10 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc6180aed00 0x7fc61815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc6180aed00 0x7fc61815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45434/0 (socket says 172.21.15.62:45434) 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617e00640 1 -- 172.21.15.62:0/806649647 learned_addr learned my addr 172.21.15.62:0/806649647 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:33.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc60fe00640 1 --2- 172.21.15.62:0/806649647 >> v2:172.21.15.121:3300/0 conn(0x7fc6180cccf0 0x7fc6180ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617400640 1 --2- 172.21.15.62:0/806649647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc61815bd10 0x7fc6180cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617e00640 1 -- 172.21.15.62:0/806649647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc61815bd10 msgr2=0x7fc6180cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617e00640 1 --2- 172.21.15.62:0/806649647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc61815bd10 0x7fc6180cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617e00640 1 -- 172.21.15.62:0/806649647 >> v2:172.21.15.121:3300/0 conn(0x7fc6180cccf0 msgr2=0x7fc6180ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617e00640 1 --2- 172.21.15.62:0/806649647 >> v2:172.21.15.121:3300/0 conn(0x7fc6180cccf0 0x7fc6180ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.704+0000 7fc617e00640 1 -- 172.21.15.62:0/806649647 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61806cd30 con 0x7fc6180aed00 2024-04-25T09:09:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.705+0000 7fc617e00640 1 --2- 172.21.15.62:0/806649647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc61815f830 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc618072540 tx=0x7fc61002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51b5ab039540404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:33.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.705+0000 7fc616a00640 1 -- 172.21.15.62:0/806649647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc610036070 con 0x7fc6180aed00 2024-04-25T09:09:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc616a00640 1 -- 172.21.15.62:0/806649647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc610031600 con 0x7fc6180aed00 2024-04-25T09:09:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc616a00640 1 -- 172.21.15.62:0/806649647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6100319d0 con 0x7fc6180aed00 2024-04-25T09:09:33.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc61e600640 1 -- 172.21.15.62:0/806649647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 msgr2=0x7fc61815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc61e600640 1 --2- 172.21.15.62:0/806649647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc61815f830 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc618072540 tx=0x7fc61002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc61e600640 1 -- 172.21.15.62:0/806649647 shutdown_connections 2024-04-25T09:09:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc61e600640 1 --2- 172.21.15.62:0/806649647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc61815f830 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc61e600640 1 --2- 172.21.15.62:0/806649647 >> v2:172.21.15.121:3300/0 conn(0x7fc6180cccf0 0x7fc6180ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc61e600640 1 --2- 172.21.15.62:0/806649647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc61815bd10 0x7fc6180cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.706+0000 7fc61e600640 1 -- 172.21.15.62:0/806649647 >> 172.21.15.62:0/806649647 conn(0x7fc6180668f0 msgr2=0x7fc618066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.707+0000 7fc61e600640 1 -- 172.21.15.62:0/806649647 shutdown_connections 2024-04-25T09:09:33.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.707+0000 7fc61e600640 1 -- 172.21.15.62:0/806649647 wait complete. 2024-04-25T09:09:33.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.708+0000 7fc61e600640 1 Processor -- start 2024-04-25T09:09:33.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.709+0000 7fc61e600640 1 -- start start 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.709+0000 7fc61e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc6180716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc61e600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6180cccf0 0x7fc6180afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc61e600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc61815bd10 0x7fc61814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc61e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180d01e0 con 0x7fc6180aed00 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc61e600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61806d2f0 con 0x7fc61815bd10 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc61e600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61806daf0 con 0x7fc6180cccf0 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc6180716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc6180716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45444/0 (socket says 172.21.15.62:45444) 2024-04-25T09:09:33.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 -- 172.21.15.62:0/4136390718 learned_addr learned my addr 172.21.15.62:0/4136390718 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc60fe00640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6180cccf0 0x7fc6180afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 -- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6180cccf0 msgr2=0x7fc6180afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6180cccf0 0x7fc6180afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 -- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc61815bd10 msgr2=0x7fc61814d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc61815bd10 0x7fc61814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 -- 172.21.15.62:0/4136390718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180b0030 con 0x7fc6180aed00 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc617400640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc6180716f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc60800c820 tx=0x7fc60800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc60f400640 1 -- 172.21.15.62:0/4136390718 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608015070 con 0x7fc6180aed00 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc60f400640 1 -- 172.21.15.62:0/4136390718 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc608004020 con 0x7fc6180aed00 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc60f400640 1 -- 172.21.15.62:0/4136390718 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6080043f0 con 0x7fc6180aed00 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6180b1370 con 0x7fc6180aed00 2024-04-25T09:09:33.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.711+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6180b05b0 con 0x7fc6180aed00 2024-04-25T09:09:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.712+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc618072c70 con 0x7fc6180aed00 2024-04-25T09:09:33.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.713+0000 7fc60f400640 1 -- 172.21.15.62:0/4136390718 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc608007830 con 0x7fc6180aed00 2024-04-25T09:09:33.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.714+0000 7fc60f400640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc5f406b500 0x7fc5f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:33.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.714+0000 7fc60f400640 1 -- 172.21.15.62:0/4136390718 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc608089f60 con 0x7fc6180aed00 2024-04-25T09:09:33.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.715+0000 7fc60fe00640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc5f406b500 0x7fc5f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:33.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.716+0000 7fc60fe00640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc5f406b500 0x7fc5f406d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc618151ad0 tx=0x7fc600008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:33.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.718+0000 7fc60f400640 1 -- 172.21.15.62:0/4136390718 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc608058d40 con 0x7fc6180aed00 2024-04-25T09:09:33.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.920+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6180680d0 con 0x7fc6180aed00 2024-04-25T09:09:33.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.924+0000 7fc60f400640 1 -- 172.21.15.62:0/4136390718 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16596 (secure 0 0 0) 0x7fc608058640 con 0x7fc6180aed00 2024-04-25T09:09:33.911 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:33.911 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":79,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:09:30.942683+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:27.235650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:28.419955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc5f406b500 msgr2=0x7fc5f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc5f406b500 0x7fc5f406d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc618151ad0 tx=0x7fc600008040 comp rx=0 tx=0).stop 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 msgr2=0x7fc6180716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc6180716f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc60800c820 tx=0x7fc60800cce0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 shutdown_connections 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc5f406b500 0x7fc5f406d9c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc61815bd10 0x7fc61814d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6180cccf0 0x7fc6180afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 --2- 172.21.15.62:0/4136390718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6180aed00 0x7fc6180716f0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:33.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 >> 172.21.15.62:0/4136390718 conn(0x7fc6180668f0 msgr2=0x7fc6180cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:33.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 shutdown_connections 2024-04-25T09:09:33.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:33.926+0000 7fc61e600640 1 -- 172.21.15.62:0/4136390718 wait complete. 2024-04-25T09:09:33.933 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:09:33.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:09:34.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.148+0000 7ff46fe00640 1 Processor -- start 2024-04-25T09:09:34.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.148+0000 7ff46fe00640 1 -- start start 2024-04-25T09:09:34.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.150+0000 7ff46fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff47013d970 0x7ff4700b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.150+0000 7ff46fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff4700b1050 0x7ff47006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.150+0000 7ff46fe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47015b220 con 0x7ff47015acf0 2024-04-25T09:09:34.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.150+0000 7ff46fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47015b500 con 0x7ff47013d970 2024-04-25T09:09:34.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.150+0000 7ff46fe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47015b7e0 con 0x7ff4700b1050 2024-04-25T09:09:34.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.150+0000 7ff46ea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff47013d970 0x7ff4700b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:34.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.150+0000 7ff46f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff47015acf0 0x7ff47013ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:40790/0 (socket says 172.21.15.62:40790) 2024-04-25T09:09:34.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.150+0000 7ff46f400640 1 -- 172.21.15.62:0/1098337917 learned_addr learned my addr 172.21.15.62:0/1098337917 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:34.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.151+0000 7ff46e000640 1 --2- 172.21.15.62:0/1098337917 >> v2:172.21.15.121:3300/0 conn(0x7ff4700b1050 0x7ff47006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:34.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.151+0000 7ff46ea00640 1 -- 172.21.15.62:0/1098337917 >> v2:172.21.15.121:3300/0 conn(0x7ff4700b1050 msgr2=0x7ff47006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:34.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.151+0000 7ff46ea00640 1 --2- 172.21.15.62:0/1098337917 >> v2:172.21.15.121:3300/0 conn(0x7ff4700b1050 0x7ff47006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.151+0000 7ff46ea00640 1 -- 172.21.15.62:0/1098337917 >> v1:172.21.15.62:6789/0 conn(0x7ff47015acf0 legacy=0x7ff47013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.151+0000 7ff46ea00640 1 -- 172.21.15.62:0/1098337917 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47015bac0 con 0x7ff47013d970 2024-04-25T09:09:34.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.152+0000 7ff46ea00640 1 --2- 172.21.15.62:0/1098337917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47013d970 0x7ff4700b0b10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff4700cd200 tx=0x7ff46002c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64a9f256106001c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:34.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.152+0000 7ff46d600640 1 -- 172.21.15.62:0/1098337917 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460039070 con 0x7ff47013d970 2024-04-25T09:09:34.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.152+0000 7ff46d600640 1 -- 172.21.15.62:0/1098337917 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff460034640 con 0x7ff47013d970 2024-04-25T09:09:34.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.152+0000 7ff46d600640 1 -- 172.21.15.62:0/1098337917 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460034a10 con 0x7ff47013d970 2024-04-25T09:09:34.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.153+0000 7ff46fe00640 1 -- 172.21.15.62:0/1098337917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47013d970 msgr2=0x7ff4700b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.153+0000 7ff46fe00640 1 --2- 172.21.15.62:0/1098337917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47013d970 0x7ff4700b0b10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff4700cd200 tx=0x7ff46002c3b0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.153+0000 7ff46fe00640 1 -- 172.21.15.62:0/1098337917 shutdown_connections 2024-04-25T09:09:34.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.153+0000 7ff46fe00640 1 --2- 172.21.15.62:0/1098337917 >> v2:172.21.15.121:3300/0 conn(0x7ff4700b1050 0x7ff47006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.153+0000 7ff46fe00640 1 --2- 172.21.15.62:0/1098337917 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47013d970 0x7ff4700b0b10 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.153+0000 7ff46fe00640 1 -- 172.21.15.62:0/1098337917 >> 172.21.15.62:0/1098337917 conn(0x7ff4700b14d0 msgr2=0x7ff4700b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:34.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.153+0000 7ff46fe00640 1 -- 172.21.15.62:0/1098337917 shutdown_connections 2024-04-25T09:09:34.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.153+0000 7ff46fe00640 1 -- 172.21.15.62:0/1098337917 wait complete. 2024-04-25T09:09:34.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.155+0000 7ff46fe00640 1 Processor -- start 2024-04-25T09:09:34.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.155+0000 7ff46fe00640 1 -- start start 2024-04-25T09:09:34.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.156+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4700b1050 0x7ff47013bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.156+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff47013d970 0x7ff470159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.156+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47015acf0 0x7ff470159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.156+0000 7ff46fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47006d730 con 0x7ff47015acf0 2024-04-25T09:09:34.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.156+0000 7ff46fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47013f4b0 con 0x7ff47013d970 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.156+0000 7ff46fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47006d8b0 con 0x7ff4700b1050 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47015acf0 0x7ff470159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47015acf0 0x7ff470159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45452/0 (socket says 172.21.15.62:45452) 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46f400640 1 -- 172.21.15.62:0/734757490 learned_addr learned my addr 172.21.15.62:0/734757490 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46e000640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff47013d970 0x7ff470159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46f400640 1 -- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4700b1050 msgr2=0x7ff47013bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46f400640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4700b1050 0x7ff47013bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46f400640 1 -- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff47013d970 msgr2=0x7ff470159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46f400640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff47013d970 0x7ff470159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46f400640 1 -- 172.21.15.62:0/734757490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4701582a0 con 0x7ff47015acf0 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.157+0000 7ff46e000640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff47013d970 0x7ff470159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:34.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.158+0000 7ff46f400640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47015acf0 0x7ff470159e70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff46800ca80 tx=0x7ff46800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:34.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.158+0000 7ff466a00640 1 -- 172.21.15.62:0/734757490 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468017070 con 0x7ff47015acf0 2024-04-25T09:09:34.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.158+0000 7ff466a00640 1 -- 172.21.15.62:0/734757490 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46800ac30 con 0x7ff47015acf0 2024-04-25T09:09:34.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.158+0000 7ff466a00640 1 -- 172.21.15.62:0/734757490 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468012600 con 0x7ff47015acf0 2024-04-25T09:09:34.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.158+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff470158580 con 0x7ff47015acf0 2024-04-25T09:09:34.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.158+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff470158e20 con 0x7ff47015acf0 2024-04-25T09:09:34.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.159+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff470158970 con 0x7ff47015acf0 2024-04-25T09:09:34.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.160+0000 7ff466a00640 1 -- 172.21.15.62:0/734757490 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff46801a070 con 0x7ff47015acf0 2024-04-25T09:09:34.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.161+0000 7ff466a00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff45006b4d0 0x7ff45006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.161+0000 7ff466a00640 1 -- 172.21.15.62:0/734757490 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff468089ee0 con 0x7ff47015acf0 2024-04-25T09:09:34.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.162+0000 7ff46ea00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff45006b4d0 0x7ff45006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:34.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.163+0000 7ff46ea00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff45006b4d0 0x7ff45006d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff460036000 tx=0x7ff46002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.165+0000 7ff466a00640 1 -- 172.21.15.62:0/734757490 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff46800ae20 con 0x7ff47015acf0 2024-04-25T09:09:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.361+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff470159060 con 0x7ff45006b4d0 2024-04-25T09:09:34.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.380+0000 7ff466a00640 1 -- 172.21.15.62:0/734757490 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252027 (secure 0 0 0) 0x7ff460034590 con 0x7ff45006b4d0 2024-04-25T09:09:34.370 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:34.372 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff45006b4d0 msgr2=0x7ff45006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff45006b4d0 0x7ff45006d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff460036000 tx=0x7ff46002e000 comp rx=0 tx=0).stop 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47015acf0 msgr2=0x7ff470159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47015acf0 0x7ff470159e70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff46800ca80 tx=0x7ff46800cf40 comp rx=0 tx=0).stop 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 shutdown_connections 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff45006b4d0 0x7ff45006d990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff47015acf0 0x7ff470159e70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff47013d970 0x7ff470159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 --2- 172.21.15.62:0/734757490 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4700b1050 0x7ff47013bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 >> 172.21.15.62:0/734757490 conn(0x7ff4700b14d0 msgr2=0x7ff470068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 shutdown_connections 2024-04-25T09:09:34.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.385+0000 7ff46fe00640 1 -- 172.21.15.62:0/734757490 wait complete. 2024-04-25T09:09:34.391 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-04-25T09:09:34.029271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352464,"kb_used_data":8648,"kb_used_omap":12,"kb_used_meta":343667,"kb_avail":754622256,"statfs":{"total":773094113280,"available":772733190144,"internally_reserved":0,"allocated":8855552,"data_stored":4458212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":351915568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002209"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975307+0000","last_change":"2024-04-25T09:09:29.962291+0000","last_active":"2024-04-25T09:09:30.975307+0000","last_peered":"2024-04-25T09:09:30.975307+0000","last_clean":"2024-04-25T09:09:30.975307+0000","last_became_active":"2024-04-25T09:09:29.961940+0000","last_became_peered":"2024-04-25T09:09:29.961940+0000","last_unstale":"2024-04-25T09:09:30.975307+0000","last_undegraded":"2024-04-25T09:09:30.975307+0000","last_fullsized":"2024-04-25T09:09:30.975307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.470703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.975077+0000","last_change":"2024-04-25T09:09:29.971694+0000","last_active":"2024-04-25T09:09:29.975077+0000","last_peered":"2024-04-25T09:09:29.975077+0000","last_clean":"2024-04-25T09:09:29.975077+0000","last_became_active":"2024-04-25T09:09:29.971434+0000","last_became_peered":"2024-04-25T09:09:29.971434+0000","last_unstale":"2024-04-25T09:09:29.975077+0000","last_undegraded":"2024-04-25T09:09:29.975077+0000","last_fullsized":"2024-04-25T09:09:29.975077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:48.283315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965464+0000","last_change":"2024-04-25T09:09:29.959739+0000","last_active":"2024-04-25T09:09:29.965464+0000","last_peered":"2024-04-25T09:09:29.965464+0000","last_clean":"2024-04-25T09:09:29.965464+0000","last_became_active":"2024-04-25T09:09:29.959461+0000","last_became_peered":"2024-04-25T09:09:29.959461+0000","last_unstale":"2024-04-25T09:09:29.965464+0000","last_undegraded":"2024-04-25T09:09:29.965464+0000","last_fullsized":"2024-04-25T09:09:29.965464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:08.101675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965491+0000","last_change":"2024-04-25T09:09:29.959710+0000","last_active":"2024-04-25T09:09:29.965491+0000","last_peered":"2024-04-25T09:09:29.965491+0000","last_clean":"2024-04-25T09:09:29.965491+0000","last_became_active":"2024-04-25T09:09:29.959422+0000","last_became_peered":"2024-04-25T09:09:29.959422+0000","last_unstale":"2024-04-25T09:09:29.965491+0000","last_undegraded":"2024-04-25T09:09:29.965491+0000","last_fullsized":"2024-04-25T09:09:29.965491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:35.671401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.957725+0000","last_change":"2024-04-25T09:09:29.948074+0000","last_active":"2024-04-25T09:09:29.957725+0000","last_peered":"2024-04-25T09:09:29.957725+0000","last_clean":"2024-04-25T09:09:29.957725+0000","last_became_active":"2024-04-25T09:09:29.947689+0000","last_became_peered":"2024-04-25T09:09:29.947689+0000","last_unstale":"2024-04-25T09:09:29.957725+0000","last_undegraded":"2024-04-25T09:09:29.957725+0000","last_fullsized":"2024-04-25T09:09:29.957725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:06.590510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961786+0000","last_change":"2024-04-25T09:09:29.958279+0000","last_active":"2024-04-25T09:09:29.961786+0000","last_peered":"2024-04-25T09:09:29.961786+0000","last_clean":"2024-04-25T09:09:29.961786+0000","last_became_active":"2024-04-25T09:09:29.957976+0000","last_became_peered":"2024-04-25T09:09:29.957976+0000","last_unstale":"2024-04-25T09:09:29.961786+0000","last_undegraded":"2024-04-25T09:09:29.961786+0000","last_fullsized":"2024-04-25T09:09:29.961786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956125+0000","last_change":"2024-04-25T09:09:29.950410+0000","last_active":"2024-04-25T09:09:30.956125+0000","last_peered":"2024-04-25T09:09:30.956125+0000","last_clean":"2024-04-25T09:09:30.956125+0000","last_became_active":"2024-04-25T09:09:29.950134+0000","last_became_peered":"2024-04-25T09:09:29.950134+0000","last_unstale":"2024-04-25T09:09:30.956125+0000","last_undegraded":"2024-04-25T09:09:30.956125+0000","last_fullsized":"2024-04-25T09:09:30.956125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:07.318360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974537+0000","last_change":"2024-04-25T09:09:29.952586+0000","last_active":"2024-04-25T09:09:30.974537+0000","last_peered":"2024-04-25T09:09:30.974537+0000","last_clean":"2024-04-25T09:09:30.974537+0000","last_became_active":"2024-04-25T09:09:29.952291+0000","last_became_peered":"2024-04-25T09:09:29.952291+0000","last_unstale":"2024-04-25T09:09:30.974537+0000","last_undegraded":"2024-04-25T09:09:30.974537+0000","last_fullsized":"2024-04-25T09:09:30.974537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:34.493863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.966593+0000","last_change":"2024-04-25T09:09:29.964243+0000","last_active":"2024-04-25T09:09:29.966593+0000","last_peered":"2024-04-25T09:09:29.966593+0000","last_clean":"2024-04-25T09:09:29.966593+0000","last_became_active":"2024-04-25T09:09:29.963687+0000","last_became_peered":"2024-04-25T09:09:29.963687+0000","last_unstale":"2024-04-25T09:09:29.966593+0000","last_undegraded":"2024-04-25T09:09:29.966593+0000","last_fullsized":"2024-04-25T09:09:29.966593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:42.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956814+0000","last_change":"2024-04-25T09:09:29.961673+0000","last_active":"2024-04-25T09:09:30.956814+0000","last_peered":"2024-04-25T09:09:30.956814+0000","last_clean":"2024-04-25T09:09:30.956814+0000","last_became_active":"2024-04-25T09:09:29.961405+0000","last_became_peered":"2024-04-25T09:09:29.961405+0000","last_unstale":"2024-04-25T09:09:30.956814+0000","last_undegraded":"2024-04-25T09:09:30.956814+0000","last_fullsized":"2024-04-25T09:09:30.956814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:01.587456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975143+0000","last_change":"2024-04-25T09:09:29.962370+0000","last_active":"2024-04-25T09:09:30.975143+0000","last_peered":"2024-04-25T09:09:30.975143+0000","last_clean":"2024-04-25T09:09:30.975143+0000","last_became_active":"2024-04-25T09:09:29.962139+0000","last_became_peered":"2024-04-25T09:09:29.962139+0000","last_unstale":"2024-04-25T09:09:30.975143+0000","last_undegraded":"2024-04-25T09:09:30.975143+0000","last_fullsized":"2024-04-25T09:09:30.975143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:13.010727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962329+0000","last_change":"2024-04-25T09:09:29.960094+0000","last_active":"2024-04-25T09:09:30.962329+0000","last_peered":"2024-04-25T09:09:30.962329+0000","last_clean":"2024-04-25T09:09:30.962329+0000","last_became_active":"2024-04-25T09:09:29.959711+0000","last_became_peered":"2024-04-25T09:09:29.959711+0000","last_unstale":"2024-04-25T09:09:30.962329+0000","last_undegraded":"2024-04-25T09:09:30.962329+0000","last_fullsized":"2024-04-25T09:09:30.962329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:39.951616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963420+0000","last_change":"2024-04-25T09:09:29.958204+0000","last_active":"2024-04-25T09:09:29.963420+0000","last_peered":"2024-04-25T09:09:29.963420+0000","last_clean":"2024-04-25T09:09:29.963420+0000","last_became_active":"2024-04-25T09:09:29.957909+0000","last_became_peered":"2024-04-25T09:09:29.957909+0000","last_unstale":"2024-04-25T09:09:29.963420+0000","last_undegraded":"2024-04-25T09:09:29.963420+0000","last_fullsized":"2024-04-25T09:09:29.963420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:57.182042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961995+0000","last_change":"2024-04-25T09:09:29.961247+0000","last_active":"2024-04-25T09:09:30.961995+0000","last_peered":"2024-04-25T09:09:30.961995+0000","last_clean":"2024-04-25T09:09:30.961995+0000","last_became_active":"2024-04-25T09:09:29.960259+0000","last_became_peered":"2024-04-25T09:09:29.960259+0000","last_unstale":"2024-04-25T09:09:30.961995+0000","last_undegraded":"2024-04-25T09:09:30.961995+0000","last_fullsized":"2024-04-25T09:09:30.961995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:38.680490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961731+0000","last_change":"2024-04-25T09:09:29.961263+0000","last_active":"2024-04-25T09:09:30.961731+0000","last_peered":"2024-04-25T09:09:30.961731+0000","last_clean":"2024-04-25T09:09:30.961731+0000","last_became_active":"2024-04-25T09:09:29.960133+0000","last_became_peered":"2024-04-25T09:09:29.960133+0000","last_unstale":"2024-04-25T09:09:30.961731+0000","last_undegraded":"2024-04-25T09:09:30.961731+0000","last_fullsized":"2024-04-25T09:09:30.961731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:48.503660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974362+0000","last_change":"2024-04-25T09:09:29.961206+0000","last_active":"2024-04-25T09:09:30.974362+0000","last_peered":"2024-04-25T09:09:30.974362+0000","last_clean":"2024-04-25T09:09:30.974362+0000","last_became_active":"2024-04-25T09:09:29.957313+0000","last_became_peered":"2024-04-25T09:09:29.957313+0000","last_unstale":"2024-04-25T09:09:30.974362+0000","last_undegraded":"2024-04-25T09:09:30.974362+0000","last_fullsized":"2024-04-25T09:09:30.974362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.851729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961459+0000","last_change":"2024-04-25T09:09:29.957432+0000","last_active":"2024-04-25T09:09:29.961459+0000","last_peered":"2024-04-25T09:09:29.961459+0000","last_clean":"2024-04-25T09:09:29.961459+0000","last_became_active":"2024-04-25T09:09:29.957127+0000","last_became_peered":"2024-04-25T09:09:29.957127+0000","last_unstale":"2024-04-25T09:09:29.961459+0000","last_undegraded":"2024-04-25T09:09:29.961459+0000","last_fullsized":"2024-04-25T09:09:29.961459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:13.914484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.969125+0000","last_change":"2024-04-25T09:09:29.964899+0000","last_active":"2024-04-25T09:09:29.969125+0000","last_peered":"2024-04-25T09:09:29.969125+0000","last_clean":"2024-04-25T09:09:29.969125+0000","last_became_active":"2024-04-25T09:09:29.964567+0000","last_became_peered":"2024-04-25T09:09:29.964567+0000","last_unstale":"2024-04-25T09:09:29.969125+0000","last_undegraded":"2024-04-25T09:09:29.969125+0000","last_fullsized":"2024-04-25T09:09:29.969125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:02.458267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974972+0000","last_change":"2024-04-25T09:09:29.967065+0000","last_active":"2024-04-25T09:09:30.974972+0000","last_peered":"2024-04-25T09:09:30.974972+0000","last_clean":"2024-04-25T09:09:30.974972+0000","last_became_active":"2024-04-25T09:09:29.966846+0000","last_became_peered":"2024-04-25T09:09:29.966846+0000","last_unstale":"2024-04-25T09:09:30.974972+0000","last_undegraded":"2024-04-25T09:09:30.974972+0000","last_fullsized":"2024-04-25T09:09:30.974972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:49.341010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.958052+0000","last_change":"2024-04-25T09:09:29.951957+0000","last_active":"2024-04-25T09:09:29.958052+0000","last_peered":"2024-04-25T09:09:29.958052+0000","last_clean":"2024-04-25T09:09:29.958052+0000","last_became_active":"2024-04-25T09:09:29.951519+0000","last_became_peered":"2024-04-25T09:09:29.951519+0000","last_unstale":"2024-04-25T09:09:29.958052+0000","last_undegraded":"2024-04-25T09:09:29.958052+0000","last_fullsized":"2024-04-25T09:09:29.958052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:35.186664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.966066+0000","last_change":"2024-04-25T09:09:29.961508+0000","last_active":"2024-04-25T09:09:29.966066+0000","last_peered":"2024-04-25T09:09:29.966066+0000","last_clean":"2024-04-25T09:09:29.966066+0000","last_became_active":"2024-04-25T09:09:29.961208+0000","last_became_peered":"2024-04-25T09:09:29.961208+0000","last_unstale":"2024-04-25T09:09:29.966066+0000","last_undegraded":"2024-04-25T09:09:29.966066+0000","last_fullsized":"2024-04-25T09:09:29.966066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:47.169527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.956081+0000","last_change":"2024-04-25T09:09:29.953361+0000","last_active":"2024-04-25T09:09:29.956081+0000","last_peered":"2024-04-25T09:09:29.956081+0000","last_clean":"2024-04-25T09:09:29.956081+0000","last_became_active":"2024-04-25T09:09:29.951186+0000","last_became_peered":"2024-04-25T09:09:29.951186+0000","last_unstale":"2024-04-25T09:09:29.956081+0000","last_undegraded":"2024-04-25T09:09:29.956081+0000","last_fullsized":"2024-04-25T09:09:29.956081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:06.126657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956099+0000","last_change":"2024-04-25T09:09:29.958350+0000","last_active":"2024-04-25T09:09:30.956099+0000","last_peered":"2024-04-25T09:09:30.956099+0000","last_clean":"2024-04-25T09:09:30.956099+0000","last_became_active":"2024-04-25T09:09:29.958065+0000","last_became_peered":"2024-04-25T09:09:29.958065+0000","last_unstale":"2024-04-25T09:09:30.956099+0000","last_undegraded":"2024-04-25T09:09:30.956099+0000","last_fullsized":"2024-04-25T09:09:30.956099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:26.238732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975446+0000","last_change":"2024-04-25T09:09:29.959930+0000","last_active":"2024-04-25T09:09:30.975446+0000","last_peered":"2024-04-25T09:09:30.975446+0000","last_clean":"2024-04-25T09:09:30.975446+0000","last_became_active":"2024-04-25T09:09:29.959341+0000","last_became_peered":"2024-04-25T09:09:29.959341+0000","last_unstale":"2024-04-25T09:09:30.975446+0000","last_undegraded":"2024-04-25T09:09:30.975446+0000","last_fullsized":"2024-04-25T09:09:30.975446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:40.320391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956579+0000","last_change":"2024-04-25T09:09:29.958760+0000","last_active":"2024-04-25T09:09:30.956579+0000","last_peered":"2024-04-25T09:09:30.956579+0000","last_clean":"2024-04-25T09:09:30.956579+0000","last_became_active":"2024-04-25T09:09:29.958172+0000","last_became_peered":"2024-04-25T09:09:29.958172+0000","last_unstale":"2024-04-25T09:09:30.956579+0000","last_undegraded":"2024-04-25T09:09:30.956579+0000","last_fullsized":"2024-04-25T09:09:30.956579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:06.081153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975264+0000","last_change":"2024-04-25T09:09:29.970156+0000","last_active":"2024-04-25T09:09:30.975264+0000","last_peered":"2024-04-25T09:09:30.975264+0000","last_clean":"2024-04-25T09:09:30.975264+0000","last_became_active":"2024-04-25T09:09:29.969578+0000","last_became_peered":"2024-04-25T09:09:29.969578+0000","last_unstale":"2024-04-25T09:09:30.975264+0000","last_undegraded":"2024-04-25T09:09:30.975264+0000","last_fullsized":"2024-04-25T09:09:30.975264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:16.077887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961498+0000","last_change":"2024-04-25T09:09:29.955733+0000","last_active":"2024-04-25T09:09:29.961498+0000","last_peered":"2024-04-25T09:09:29.961498+0000","last_clean":"2024-04-25T09:09:29.961498+0000","last_became_active":"2024-04-25T09:09:29.955130+0000","last_became_peered":"2024-04-25T09:09:29.955130+0000","last_unstale":"2024-04-25T09:09:29.961498+0000","last_undegraded":"2024-04-25T09:09:29.961498+0000","last_fullsized":"2024-04-25T09:09:29.961498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:55.957705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.972308+0000","last_change":"2024-04-25T09:09:29.966749+0000","last_active":"2024-04-25T09:09:29.972308+0000","last_peered":"2024-04-25T09:09:29.972308+0000","last_clean":"2024-04-25T09:09:29.972308+0000","last_became_active":"2024-04-25T09:09:29.966456+0000","last_became_peered":"2024-04-25T09:09:29.966456+0000","last_unstale":"2024-04-25T09:09:29.972308+0000","last_undegraded":"2024-04-25T09:09:29.972308+0000","last_fullsized":"2024-04-25T09:09:29.972308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:37.258879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.969209+0000","last_change":"2024-04-25T09:09:29.965273+0000","last_active":"2024-04-25T09:09:29.969209+0000","last_peered":"2024-04-25T09:09:29.969209+0000","last_clean":"2024-04-25T09:09:29.969209+0000","last_became_active":"2024-04-25T09:09:29.964129+0000","last_became_peered":"2024-04-25T09:09:29.964129+0000","last_unstale":"2024-04-25T09:09:29.969209+0000","last_undegraded":"2024-04-25T09:09:29.969209+0000","last_fullsized":"2024-04-25T09:09:29.969209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:50.877336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975057+0000","last_change":"2024-04-25T09:09:29.965545+0000","last_active":"2024-04-25T09:09:30.975057+0000","last_peered":"2024-04-25T09:09:30.975057+0000","last_clean":"2024-04-25T09:09:30.975057+0000","last_became_active":"2024-04-25T09:09:29.964606+0000","last_became_peered":"2024-04-25T09:09:29.964606+0000","last_unstale":"2024-04-25T09:09:30.975057+0000","last_undegraded":"2024-04-25T09:09:30.975057+0000","last_fullsized":"2024-04-25T09:09:30.975057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:45.965290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.966447+0000","last_change":"2024-04-25T09:09:29.962536+0000","last_active":"2024-04-25T09:09:29.966447+0000","last_peered":"2024-04-25T09:09:29.966447+0000","last_clean":"2024-04-25T09:09:29.966447+0000","last_became_active":"2024-04-25T09:09:29.962132+0000","last_became_peered":"2024-04-25T09:09:29.962132+0000","last_unstale":"2024-04-25T09:09:29.966447+0000","last_undegraded":"2024-04-25T09:09:29.966447+0000","last_fullsized":"2024-04-25T09:09:29.966447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:40.507663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.972174+0000","last_change":"2024-04-25T09:09:29.968554+0000","last_active":"2024-04-25T09:09:29.972174+0000","last_peered":"2024-04-25T09:09:29.972174+0000","last_clean":"2024-04-25T09:09:29.972174+0000","last_became_active":"2024-04-25T09:09:29.968190+0000","last_became_peered":"2024-04-25T09:09:29.968190+0000","last_unstale":"2024-04-25T09:09:29.972174+0000","last_undegraded":"2024-04-25T09:09:29.972174+0000","last_fullsized":"2024-04-25T09:09:29.972174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:00.872742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974764+0000","last_change":"2024-04-25T09:09:29.970365+0000","last_active":"2024-04-25T09:09:30.974764+0000","last_peered":"2024-04-25T09:09:30.974764+0000","last_clean":"2024-04-25T09:09:30.974764+0000","last_became_active":"2024-04-25T09:09:29.969908+0000","last_became_peered":"2024-04-25T09:09:29.969908+0000","last_unstale":"2024-04-25T09:09:30.974764+0000","last_undegraded":"2024-04-25T09:09:30.974764+0000","last_fullsized":"2024-04-25T09:09:30.974764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:52.126126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.983995+0000","last_change":"2024-04-25T09:09:28.963922+0000","last_active":"2024-04-25T09:09:29.983995+0000","last_peered":"2024-04-25T09:09:29.983995+0000","last_clean":"2024-04-25T09:09:29.983995+0000","last_became_active":"2024-04-25T09:09:28.963563+0000","last_became_peered":"2024-04-25T09:09:28.963563+0000","last_unstale":"2024-04-25T09:09:29.983995+0000","last_undegraded":"2024-04-25T09:09:29.983995+0000","last_fullsized":"2024-04-25T09:09:29.983995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:23.286506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965460+0000","last_change":"2024-04-25T09:09:29.960876+0000","last_active":"2024-04-25T09:09:29.965460+0000","last_peered":"2024-04-25T09:09:29.965460+0000","last_clean":"2024-04-25T09:09:29.965460+0000","last_became_active":"2024-04-25T09:09:29.960357+0000","last_became_peered":"2024-04-25T09:09:29.960357+0000","last_unstale":"2024-04-25T09:09:29.965460+0000","last_undegraded":"2024-04-25T09:09:29.965460+0000","last_fullsized":"2024-04-25T09:09:29.965460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:17.093842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.980988+0000","last_change":"2024-04-25T09:09:28.964540+0000","last_active":"2024-04-25T09:09:29.980988+0000","last_peered":"2024-04-25T09:09:29.980988+0000","last_clean":"2024-04-25T09:09:29.980988+0000","last_became_active":"2024-04-25T09:09:28.963930+0000","last_became_peered":"2024-04-25T09:09:28.963930+0000","last_unstale":"2024-04-25T09:09:29.980988+0000","last_undegraded":"2024-04-25T09:09:29.980988+0000","last_fullsized":"2024-04-25T09:09:29.980988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.960112+0000","last_change":"2024-04-25T09:09:29.956250+0000","last_active":"2024-04-25T09:09:29.960112+0000","last_peered":"2024-04-25T09:09:29.960112+0000","last_clean":"2024-04-25T09:09:29.960112+0000","last_became_active":"2024-04-25T09:09:29.955976+0000","last_became_peered":"2024-04-25T09:09:29.955976+0000","last_unstale":"2024-04-25T09:09:29.960112+0000","last_undegraded":"2024-04-25T09:09:29.960112+0000","last_fullsized":"2024-04-25T09:09:29.960112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:39.951616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975132+0000","last_change":"2024-04-25T09:09:28.958191+0000","last_active":"2024-04-25T09:09:30.975132+0000","last_peered":"2024-04-25T09:09:30.975132+0000","last_clean":"2024-04-25T09:09:30.975132+0000","last_became_active":"2024-04-25T09:09:28.957873+0000","last_became_peered":"2024-04-25T09:09:28.957873+0000","last_unstale":"2024-04-25T09:09:30.975132+0000","last_undegraded":"2024-04-25T09:09:30.975132+0000","last_fullsized":"2024-04-25T09:09:30.975132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:51.354823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956369+0000","last_change":"2024-04-25T09:09:29.968427+0000","last_active":"2024-04-25T09:09:30.956369+0000","last_peered":"2024-04-25T09:09:30.956369+0000","last_clean":"2024-04-25T09:09:30.956369+0000","last_became_active":"2024-04-25T09:09:29.960209+0000","last_became_peered":"2024-04-25T09:09:29.960209+0000","last_unstale":"2024-04-25T09:09:30.956369+0000","last_undegraded":"2024-04-25T09:09:30.956369+0000","last_fullsized":"2024-04-25T09:09:30.956369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:55.096541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.980227+0000","last_change":"2024-04-25T09:09:28.964799+0000","last_active":"2024-04-25T09:09:29.980227+0000","last_peered":"2024-04-25T09:09:29.980227+0000","last_clean":"2024-04-25T09:09:29.980227+0000","last_became_active":"2024-04-25T09:09:28.964535+0000","last_became_peered":"2024-04-25T09:09:28.964535+0000","last_unstale":"2024-04-25T09:09:29.980227+0000","last_undegraded":"2024-04-25T09:09:29.980227+0000","last_fullsized":"2024-04-25T09:09:29.980227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:47.486345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974718+0000","last_change":"2024-04-25T09:09:29.961835+0000","last_active":"2024-04-25T09:09:30.974718+0000","last_peered":"2024-04-25T09:09:30.974718+0000","last_clean":"2024-04-25T09:09:30.974718+0000","last_became_active":"2024-04-25T09:09:29.960702+0000","last_became_peered":"2024-04-25T09:09:29.960702+0000","last_unstale":"2024-04-25T09:09:30.974718+0000","last_undegraded":"2024-04-25T09:09:30.974718+0000","last_fullsized":"2024-04-25T09:09:30.974718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.507627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961956+0000","last_change":"2024-04-25T09:09:28.968881+0000","last_active":"2024-04-25T09:09:30.961956+0000","last_peered":"2024-04-25T09:09:30.961956+0000","last_clean":"2024-04-25T09:09:30.961956+0000","last_became_active":"2024-04-25T09:09:28.968246+0000","last_became_peered":"2024-04-25T09:09:28.968246+0000","last_unstale":"2024-04-25T09:09:30.961956+0000","last_undegraded":"2024-04-25T09:09:30.961956+0000","last_fullsized":"2024-04-25T09:09:30.961956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:55.947582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.968757+0000","last_change":"2024-04-25T09:09:29.965006+0000","last_active":"2024-04-25T09:09:29.968757+0000","last_peered":"2024-04-25T09:09:29.968757+0000","last_clean":"2024-04-25T09:09:29.968757+0000","last_became_active":"2024-04-25T09:09:29.964332+0000","last_became_peered":"2024-04-25T09:09:29.964332+0000","last_unstale":"2024-04-25T09:09:29.968757+0000","last_undegraded":"2024-04-25T09:09:29.968757+0000","last_fullsized":"2024-04-25T09:09:29.968757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:06.590510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956031+0000","last_change":"2024-04-25T09:09:28.962365+0000","last_active":"2024-04-25T09:09:30.956031+0000","last_peered":"2024-04-25T09:09:30.956031+0000","last_clean":"2024-04-25T09:09:30.956031+0000","last_became_active":"2024-04-25T09:09:28.962047+0000","last_became_peered":"2024-04-25T09:09:28.962047+0000","last_unstale":"2024-04-25T09:09:30.956031+0000","last_undegraded":"2024-04-25T09:09:30.956031+0000","last_fullsized":"2024-04-25T09:09:30.956031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:37.754042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965958+0000","last_change":"2024-04-25T09:09:29.963625+0000","last_active":"2024-04-25T09:09:29.965958+0000","last_peered":"2024-04-25T09:09:29.965958+0000","last_clean":"2024-04-25T09:09:29.965958+0000","last_became_active":"2024-04-25T09:09:29.962966+0000","last_became_peered":"2024-04-25T09:09:29.962966+0000","last_unstale":"2024-04-25T09:09:29.965958+0000","last_undegraded":"2024-04-25T09:09:29.965958+0000","last_fullsized":"2024-04-25T09:09:29.965958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:49.077325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.949003+0000","last_change":"2024-04-25T09:09:28.969333+0000","last_active":"2024-04-25T09:09:29.949003+0000","last_peered":"2024-04-25T09:09:29.949003+0000","last_clean":"2024-04-25T09:09:29.949003+0000","last_became_active":"2024-04-25T09:09:28.969041+0000","last_became_peered":"2024-04-25T09:09:28.969041+0000","last_unstale":"2024-04-25T09:09:29.949003+0000","last_undegraded":"2024-04-25T09:09:29.949003+0000","last_fullsized":"2024-04-25T09:09:29.949003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:55.947582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961707+0000","last_change":"2024-04-25T09:09:29.961992+0000","last_active":"2024-04-25T09:09:30.961707+0000","last_peered":"2024-04-25T09:09:30.961707+0000","last_clean":"2024-04-25T09:09:30.961707+0000","last_became_active":"2024-04-25T09:09:29.960713+0000","last_became_peered":"2024-04-25T09:09:29.960713+0000","last_unstale":"2024-04-25T09:09:30.961707+0000","last_undegraded":"2024-04-25T09:09:30.961707+0000","last_fullsized":"2024-04-25T09:09:30.961707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:24.303821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961983+0000","last_change":"2024-04-25T09:09:28.967156+0000","last_active":"2024-04-25T09:09:30.961983+0000","last_peered":"2024-04-25T09:09:30.961983+0000","last_clean":"2024-04-25T09:09:30.961983+0000","last_became_active":"2024-04-25T09:09:28.964457+0000","last_became_peered":"2024-04-25T09:09:28.964457+0000","last_unstale":"2024-04-25T09:09:30.961983+0000","last_undegraded":"2024-04-25T09:09:30.961983+0000","last_fullsized":"2024-04-25T09:09:30.961983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:39.490348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.957174+0000","last_change":"2024-04-25T09:09:29.961617+0000","last_active":"2024-04-25T09:09:30.957174+0000","last_peered":"2024-04-25T09:09:30.957174+0000","last_clean":"2024-04-25T09:09:30.957174+0000","last_became_active":"2024-04-25T09:09:29.961118+0000","last_became_peered":"2024-04-25T09:09:29.961118+0000","last_unstale":"2024-04-25T09:09:30.957174+0000","last_undegraded":"2024-04-25T09:09:30.957174+0000","last_fullsized":"2024-04-25T09:09:30.957174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.648854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974860+0000","last_change":"2024-04-25T09:09:28.968252+0000","last_active":"2024-04-25T09:09:30.974860+0000","last_peered":"2024-04-25T09:09:30.974860+0000","last_clean":"2024-04-25T09:09:30.974860+0000","last_became_active":"2024-04-25T09:09:28.967946+0000","last_became_peered":"2024-04-25T09:09:28.967946+0000","last_unstale":"2024-04-25T09:09:30.974860+0000","last_undegraded":"2024-04-25T09:09:30.974860+0000","last_fullsized":"2024-04-25T09:09:30.974860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:28.096362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.970699+0000","last_change":"2024-04-25T09:09:29.966068+0000","last_active":"2024-04-25T09:09:29.970699+0000","last_peered":"2024-04-25T09:09:29.970699+0000","last_clean":"2024-04-25T09:09:29.970699+0000","last_became_active":"2024-04-25T09:09:29.965714+0000","last_became_peered":"2024-04-25T09:09:29.965714+0000","last_unstale":"2024-04-25T09:09:29.970699+0000","last_undegraded":"2024-04-25T09:09:29.970699+0000","last_fullsized":"2024-04-25T09:09:29.970699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:49.077325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.954085+0000","last_change":"2024-04-25T09:09:28.954777+0000","last_active":"2024-04-25T09:09:29.954085+0000","last_peered":"2024-04-25T09:09:29.954085+0000","last_clean":"2024-04-25T09:09:29.954085+0000","last_became_active":"2024-04-25T09:09:28.954456+0000","last_became_peered":"2024-04-25T09:09:28.954456+0000","last_unstale":"2024-04-25T09:09:29.954085+0000","last_undegraded":"2024-04-25T09:09:29.954085+0000","last_fullsized":"2024-04-25T09:09:29.954085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:36.293123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975066+0000","last_change":"2024-04-25T09:09:29.962015+0000","last_active":"2024-04-25T09:09:30.975066+0000","last_peered":"2024-04-25T09:09:30.975066+0000","last_clean":"2024-04-25T09:09:30.975066+0000","last_became_active":"2024-04-25T09:09:29.961739+0000","last_became_peered":"2024-04-25T09:09:29.961739+0000","last_unstale":"2024-04-25T09:09:30.975066+0000","last_undegraded":"2024-04-25T09:09:30.975066+0000","last_fullsized":"2024-04-25T09:09:30.975066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:38.924704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962508+0000","last_change":"2024-04-25T09:09:28.967736+0000","last_active":"2024-04-25T09:09:30.962508+0000","last_peered":"2024-04-25T09:09:30.962508+0000","last_clean":"2024-04-25T09:09:30.962508+0000","last_became_active":"2024-04-25T09:09:28.966985+0000","last_became_peered":"2024-04-25T09:09:28.966985+0000","last_unstale":"2024-04-25T09:09:30.962508+0000","last_undegraded":"2024-04-25T09:09:30.962508+0000","last_fullsized":"2024-04-25T09:09:30.962508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:46.111525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961477+0000","last_change":"2024-04-25T09:09:29.955413+0000","last_active":"2024-04-25T09:09:29.961477+0000","last_peered":"2024-04-25T09:09:29.961477+0000","last_clean":"2024-04-25T09:09:29.961477+0000","last_became_active":"2024-04-25T09:09:29.954652+0000","last_became_peered":"2024-04-25T09:09:29.954652+0000","last_unstale":"2024-04-25T09:09:29.961477+0000","last_undegraded":"2024-04-25T09:09:29.961477+0000","last_fullsized":"2024-04-25T09:09:29.961477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:07.861832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"78'6","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.987693+0000","last_change":"2024-04-25T09:09:28.964335+0000","last_active":"2024-04-25T09:09:29.987693+0000","last_peered":"2024-04-25T09:09:29.987693+0000","last_clean":"2024-04-25T09:09:29.987693+0000","last_became_active":"2024-04-25T09:09:28.963713+0000","last_became_peered":"2024-04-25T09:09:28.963713+0000","last_unstale":"2024-04-25T09:09:29.987693+0000","last_undegraded":"2024-04-25T09:09:29.987693+0000","last_fullsized":"2024-04-25T09:09:29.987693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:17.841750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963590+0000","last_change":"2024-04-25T09:09:29.962597+0000","last_active":"2024-04-25T09:09:29.963590+0000","last_peered":"2024-04-25T09:09:29.963590+0000","last_clean":"2024-04-25T09:09:29.963590+0000","last_became_active":"2024-04-25T09:09:29.960200+0000","last_became_peered":"2024-04-25T09:09:29.960200+0000","last_unstale":"2024-04-25T09:09:29.963590+0000","last_undegraded":"2024-04-25T09:09:29.963590+0000","last_fullsized":"2024-04-25T09:09:29.963590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:44.532729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.983126+0000","last_change":"2024-04-25T09:09:28.962537+0000","last_active":"2024-04-25T09:09:29.983126+0000","last_peered":"2024-04-25T09:09:29.983126+0000","last_clean":"2024-04-25T09:09:29.983126+0000","last_became_active":"2024-04-25T09:09:28.962236+0000","last_became_peered":"2024-04-25T09:09:28.962236+0000","last_unstale":"2024-04-25T09:09:29.983126+0000","last_undegraded":"2024-04-25T09:09:29.983126+0000","last_fullsized":"2024-04-25T09:09:29.983126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:36.293123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956315+0000","last_change":"2024-04-25T09:09:29.971364+0000","last_active":"2024-04-25T09:09:30.956315+0000","last_peered":"2024-04-25T09:09:30.956315+0000","last_clean":"2024-04-25T09:09:30.956315+0000","last_became_active":"2024-04-25T09:09:29.971042+0000","last_became_peered":"2024-04-25T09:09:29.971042+0000","last_unstale":"2024-04-25T09:09:30.956315+0000","last_undegraded":"2024-04-25T09:09:30.956315+0000","last_fullsized":"2024-04-25T09:09:30.956315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:55.551161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956034+0000","last_change":"2024-04-25T09:09:28.946867+0000","last_active":"2024-04-25T09:09:30.956034+0000","last_peered":"2024-04-25T09:09:30.956034+0000","last_clean":"2024-04-25T09:09:30.956034+0000","last_became_active":"2024-04-25T09:09:28.946542+0000","last_became_peered":"2024-04-25T09:09:28.946542+0000","last_unstale":"2024-04-25T09:09:30.956034+0000","last_undegraded":"2024-04-25T09:09:30.956034+0000","last_fullsized":"2024-04-25T09:09:30.956034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:11.437718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.958426+0000","last_change":"2024-04-25T09:09:29.952029+0000","last_active":"2024-04-25T09:09:29.958426+0000","last_peered":"2024-04-25T09:09:29.958426+0000","last_clean":"2024-04-25T09:09:29.958426+0000","last_became_active":"2024-04-25T09:09:29.951449+0000","last_became_peered":"2024-04-25T09:09:29.951449+0000","last_unstale":"2024-04-25T09:09:29.958426+0000","last_undegraded":"2024-04-25T09:09:29.958426+0000","last_fullsized":"2024-04-25T09:09:29.958426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:02.191228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956194+0000","last_change":"2024-04-25T09:09:28.965099+0000","last_active":"2024-04-25T09:09:30.956194+0000","last_peered":"2024-04-25T09:09:30.956194+0000","last_clean":"2024-04-25T09:09:30.956194+0000","last_became_active":"2024-04-25T09:09:28.964801+0000","last_became_peered":"2024-04-25T09:09:28.964801+0000","last_unstale":"2024-04-25T09:09:30.956194+0000","last_undegraded":"2024-04-25T09:09:30.956194+0000","last_fullsized":"2024-04-25T09:09:30.956194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:34.654086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.959211+0000","last_change":"2024-04-25T09:09:29.956202+0000","last_active":"2024-04-25T09:09:29.959211+0000","last_peered":"2024-04-25T09:09:29.959211+0000","last_clean":"2024-04-25T09:09:29.959211+0000","last_became_active":"2024-04-25T09:09:29.955852+0000","last_became_peered":"2024-04-25T09:09:29.955852+0000","last_unstale":"2024-04-25T09:09:29.959211+0000","last_undegraded":"2024-04-25T09:09:29.959211+0000","last_fullsized":"2024-04-25T09:09:29.959211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.771357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.955817+0000","last_change":"2024-04-25T09:09:28.959555+0000","last_active":"2024-04-25T09:09:30.955817+0000","last_peered":"2024-04-25T09:09:30.955817+0000","last_clean":"2024-04-25T09:09:30.955817+0000","last_became_active":"2024-04-25T09:09:28.958622+0000","last_became_peered":"2024-04-25T09:09:28.958622+0000","last_unstale":"2024-04-25T09:09:30.955817+0000","last_undegraded":"2024-04-25T09:09:30.955817+0000","last_fullsized":"2024-04-25T09:09:30.955817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:29.802770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965240+0000","last_change":"2024-04-25T09:09:29.954349+0000","last_active":"2024-04-25T09:09:29.965240+0000","last_peered":"2024-04-25T09:09:29.965240+0000","last_clean":"2024-04-25T09:09:29.965240+0000","last_became_active":"2024-04-25T09:09:29.953733+0000","last_became_peered":"2024-04-25T09:09:29.953733+0000","last_unstale":"2024-04-25T09:09:29.965240+0000","last_undegraded":"2024-04-25T09:09:29.965240+0000","last_fullsized":"2024-04-25T09:09:29.965240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:18.859065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974864+0000","last_change":"2024-04-25T09:09:28.965522+0000","last_active":"2024-04-25T09:09:30.974864+0000","last_peered":"2024-04-25T09:09:30.974864+0000","last_clean":"2024-04-25T09:09:30.974864+0000","last_became_active":"2024-04-25T09:09:28.965245+0000","last_became_peered":"2024-04-25T09:09:28.965245+0000","last_unstale":"2024-04-25T09:09:30.974864+0000","last_undegraded":"2024-04-25T09:09:30.974864+0000","last_fullsized":"2024-04-25T09:09:30.974864+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:09.571415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.967659+0000","last_change":"2024-04-25T09:09:29.964385+0000","last_active":"2024-04-25T09:09:29.967659+0000","last_peered":"2024-04-25T09:09:29.967659+0000","last_clean":"2024-04-25T09:09:29.967659+0000","last_became_active":"2024-04-25T09:09:29.964110+0000","last_became_peered":"2024-04-25T09:09:29.964110+0000","last_unstale":"2024-04-25T09:09:29.967659+0000","last_undegraded":"2024-04-25T09:09:29.967659+0000","last_fullsized":"2024-04-25T09:09:29.967659+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:47.128840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.980141+0000","last_change":"2024-04-25T09:09:28.952739+0000","last_active":"2024-04-25T09:09:29.980141+0000","last_peered":"2024-04-25T09:09:29.980141+0000","last_clean":"2024-04-25T09:09:29.980141+0000","last_became_active":"2024-04-25T09:09:28.952271+0000","last_became_peered":"2024-04-25T09:09:28.952271+0000","last_unstale":"2024-04-25T09:09:29.980141+0000","last_undegraded":"2024-04-25T09:09:29.980141+0000","last_fullsized":"2024-04-25T09:09:29.980141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:55.947582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963614+0000","last_change":"2024-04-25T09:09:29.958492+0000","last_active":"2024-04-25T09:09:29.963614+0000","last_peered":"2024-04-25T09:09:29.963614+0000","last_clean":"2024-04-25T09:09:29.963614+0000","last_became_active":"2024-04-25T09:09:29.958168+0000","last_became_peered":"2024-04-25T09:09:29.958168+0000","last_unstale":"2024-04-25T09:09:29.963614+0000","last_undegraded":"2024-04-25T09:09:29.963614+0000","last_fullsized":"2024-04-25T09:09:29.963614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:30.820085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.977708+0000","last_change":"2024-04-25T09:09:28.965611+0000","last_active":"2024-04-25T09:09:29.977708+0000","last_peered":"2024-04-25T09:09:29.977708+0000","last_clean":"2024-04-25T09:09:29.977708+0000","last_became_active":"2024-04-25T09:09:28.965307+0000","last_became_peered":"2024-04-25T09:09:28.965307+0000","last_unstale":"2024-04-25T09:09:29.977708+0000","last_undegraded":"2024-04-25T09:09:29.977708+0000","last_fullsized":"2024-04-25T09:09:29.977708+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:16.405829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.962501+0000","last_change":"2024-04-25T09:09:29.957555+0000","last_active":"2024-04-25T09:09:29.962501+0000","last_peered":"2024-04-25T09:09:29.962501+0000","last_clean":"2024-04-25T09:09:29.962501+0000","last_became_active":"2024-04-25T09:09:29.957294+0000","last_became_peered":"2024-04-25T09:09:29.957294+0000","last_unstale":"2024-04-25T09:09:29.962501+0000","last_undegraded":"2024-04-25T09:09:29.962501+0000","last_fullsized":"2024-04-25T09:09:29.962501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:12.455033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.950078+0000","last_change":"2024-04-25T09:09:28.960807+0000","last_active":"2024-04-25T09:09:29.950078+0000","last_peered":"2024-04-25T09:09:29.950078+0000","last_clean":"2024-04-25T09:09:29.950078+0000","last_became_active":"2024-04-25T09:09:28.960445+0000","last_became_peered":"2024-04-25T09:09:28.960445+0000","last_unstale":"2024-04-25T09:09:29.950078+0000","last_undegraded":"2024-04-25T09:09:29.950078+0000","last_fullsized":"2024-04-25T09:09:29.950078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.663175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963181+0000","last_change":"2024-04-25T09:09:29.957285+0000","last_active":"2024-04-25T09:09:29.963181+0000","last_peered":"2024-04-25T09:09:29.963181+0000","last_clean":"2024-04-25T09:09:29.963181+0000","last_became_active":"2024-04-25T09:09:29.957009+0000","last_became_peered":"2024-04-25T09:09:29.957009+0000","last_unstale":"2024-04-25T09:09:29.963181+0000","last_undegraded":"2024-04-25T09:09:29.963181+0000","last_fullsized":"2024-04-25T09:09:29.963181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:40.507663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.948355+0000","last_change":"2024-04-25T09:09:28.963736+0000","last_active":"2024-04-25T09:09:29.948355+0000","last_peered":"2024-04-25T09:09:29.948355+0000","last_clean":"2024-04-25T09:09:29.948355+0000","last_became_active":"2024-04-25T09:09:28.963389+0000","last_became_peered":"2024-04-25T09:09:28.963389+0000","last_unstale":"2024-04-25T09:09:29.948355+0000","last_undegraded":"2024-04-25T09:09:29.948355+0000","last_fullsized":"2024-04-25T09:09:29.948355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.266000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965938+0000","last_change":"2024-04-25T09:09:29.962572+0000","last_active":"2024-04-25T09:09:29.965938+0000","last_peered":"2024-04-25T09:09:29.965938+0000","last_clean":"2024-04-25T09:09:29.965938+0000","last_became_active":"2024-04-25T09:09:29.961683+0000","last_became_peered":"2024-04-25T09:09:29.961683+0000","last_unstale":"2024-04-25T09:09:29.965938+0000","last_undegraded":"2024-04-25T09:09:29.965938+0000","last_fullsized":"2024-04-25T09:09:29.965938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:47.128840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.942232+0000","last_change":"2024-04-25T09:09:28.964595+0000","last_active":"2024-04-25T09:09:29.942232+0000","last_peered":"2024-04-25T09:09:29.942232+0000","last_clean":"2024-04-25T09:09:29.942232+0000","last_became_active":"2024-04-25T09:09:28.964275+0000","last_became_peered":"2024-04-25T09:09:28.964275+0000","last_unstale":"2024-04-25T09:09:29.942232+0000","last_undegraded":"2024-04-25T09:09:29.942232+0000","last_fullsized":"2024-04-25T09:09:29.942232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:11.857238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962261+0000","last_change":"2024-04-25T09:09:29.955991+0000","last_active":"2024-04-25T09:09:30.962261+0000","last_peered":"2024-04-25T09:09:30.962261+0000","last_clean":"2024-04-25T09:09:30.962261+0000","last_became_active":"2024-04-25T09:09:29.955331+0000","last_became_peered":"2024-04-25T09:09:29.955331+0000","last_unstale":"2024-04-25T09:09:30.962261+0000","last_undegraded":"2024-04-25T09:09:30.962261+0000","last_fullsized":"2024-04-25T09:09:30.962261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:06.126657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.932836+0000","last_change":"2024-04-25T09:09:28.954742+0000","last_active":"2024-04-25T09:09:29.932836+0000","last_peered":"2024-04-25T09:09:29.932836+0000","last_clean":"2024-04-25T09:09:29.932836+0000","last_became_active":"2024-04-25T09:09:28.954147+0000","last_became_peered":"2024-04-25T09:09:28.954147+0000","last_unstale":"2024-04-25T09:09:29.932836+0000","last_undegraded":"2024-04-25T09:09:29.932836+0000","last_fullsized":"2024-04-25T09:09:29.932836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:38.934301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961293+0000","last_change":"2024-04-25T09:09:29.955617+0000","last_active":"2024-04-25T09:09:29.961293+0000","last_peered":"2024-04-25T09:09:29.961293+0000","last_clean":"2024-04-25T09:09:29.961293+0000","last_became_active":"2024-04-25T09:09:29.955312+0000","last_became_peered":"2024-04-25T09:09:29.955312+0000","last_unstale":"2024-04-25T09:09:29.961293+0000","last_undegraded":"2024-04-25T09:09:29.961293+0000","last_fullsized":"2024-04-25T09:09:29.961293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:18.495961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975293+0000","last_change":"2024-04-25T09:09:28.963697+0000","last_active":"2024-04-25T09:09:30.975293+0000","last_peered":"2024-04-25T09:09:30.975293+0000","last_clean":"2024-04-25T09:09:30.975293+0000","last_became_active":"2024-04-25T09:09:28.963420+0000","last_became_peered":"2024-04-25T09:09:28.963420+0000","last_unstale":"2024-04-25T09:09:30.975293+0000","last_undegraded":"2024-04-25T09:09:30.975293+0000","last_fullsized":"2024-04-25T09:09:30.975293+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:48.261419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956761+0000","last_change":"2024-04-25T09:09:29.958448+0000","last_active":"2024-04-25T09:09:30.956761+0000","last_peered":"2024-04-25T09:09:30.956761+0000","last_clean":"2024-04-25T09:09:30.956761+0000","last_became_active":"2024-04-25T09:09:29.957840+0000","last_became_peered":"2024-04-25T09:09:29.957840+0000","last_unstale":"2024-04-25T09:09:30.956761+0000","last_undegraded":"2024-04-25T09:09:30.956761+0000","last_fullsized":"2024-04-25T09:09:30.956761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:24.090990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.978060+0000","last_change":"2024-04-25T09:09:28.961166+0000","last_active":"2024-04-25T09:09:30.978060+0000","last_peered":"2024-04-25T09:09:30.978060+0000","last_clean":"2024-04-25T09:09:30.978060+0000","last_became_active":"2024-04-25T09:09:28.960876+0000","last_became_peered":"2024-04-25T09:09:28.960876+0000","last_unstale":"2024-04-25T09:09:30.978060+0000","last_undegraded":"2024-04-25T09:09:30.978060+0000","last_fullsized":"2024-04-25T09:09:30.978060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:34.852622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.969939+0000","last_change":"2024-04-25T09:09:29.966079+0000","last_active":"2024-04-25T09:09:29.969939+0000","last_peered":"2024-04-25T09:09:29.969939+0000","last_clean":"2024-04-25T09:09:29.969939+0000","last_became_active":"2024-04-25T09:09:29.965819+0000","last_became_peered":"2024-04-25T09:09:29.965819+0000","last_unstale":"2024-04-25T09:09:29.969939+0000","last_undegraded":"2024-04-25T09:09:29.969939+0000","last_fullsized":"2024-04-25T09:09:29.969939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:47.266776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962674+0000","last_change":"2024-04-25T09:09:28.961674+0000","last_active":"2024-04-25T09:09:30.962674+0000","last_peered":"2024-04-25T09:09:30.962674+0000","last_clean":"2024-04-25T09:09:30.962674+0000","last_became_active":"2024-04-25T09:09:28.961315+0000","last_became_peered":"2024-04-25T09:09:28.961315+0000","last_unstale":"2024-04-25T09:09:30.962674+0000","last_undegraded":"2024-04-25T09:09:30.962674+0000","last_fullsized":"2024-04-25T09:09:30.962674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:48.060010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956554+0000","last_change":"2024-04-25T09:09:29.955260+0000","last_active":"2024-04-25T09:09:30.956554+0000","last_peered":"2024-04-25T09:09:30.956554+0000","last_clean":"2024-04-25T09:09:30.956554+0000","last_became_active":"2024-04-25T09:09:29.954958+0000","last_became_peered":"2024-04-25T09:09:29.954958+0000","last_unstale":"2024-04-25T09:09:30.956554+0000","last_undegraded":"2024-04-25T09:09:30.956554+0000","last_fullsized":"2024-04-25T09:09:30.956554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:19.566183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962397+0000","last_change":"2024-04-25T09:09:28.966553+0000","last_active":"2024-04-25T09:09:30.962397+0000","last_peered":"2024-04-25T09:09:30.962397+0000","last_clean":"2024-04-25T09:09:30.962397+0000","last_became_active":"2024-04-25T09:09:28.966079+0000","last_became_peered":"2024-04-25T09:09:28.966079+0000","last_unstale":"2024-04-25T09:09:30.962397+0000","last_undegraded":"2024-04-25T09:09:30.962397+0000","last_fullsized":"2024-04-25T09:09:30.962397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:46.152212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961441+0000","last_change":"2024-04-25T09:09:29.955384+0000","last_active":"2024-04-25T09:09:29.961441+0000","last_peered":"2024-04-25T09:09:29.961441+0000","last_clean":"2024-04-25T09:09:29.961441+0000","last_became_active":"2024-04-25T09:09:29.954792+0000","last_became_peered":"2024-04-25T09:09:29.954792+0000","last_unstale":"2024-04-25T09:09:29.961441+0000","last_undegraded":"2024-04-25T09:09:29.961441+0000","last_fullsized":"2024-04-25T09:09:29.961441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:57.369951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962096+0000","last_change":"2024-04-25T09:09:28.968371+0000","last_active":"2024-04-25T09:09:30.962096+0000","last_peered":"2024-04-25T09:09:30.962096+0000","last_clean":"2024-04-25T09:09:30.962096+0000","last_became_active":"2024-04-25T09:09:28.968100+0000","last_became_peered":"2024-04-25T09:09:28.968100+0000","last_unstale":"2024-04-25T09:09:30.962096+0000","last_undegraded":"2024-04-25T09:09:30.962096+0000","last_fullsized":"2024-04-25T09:09:30.962096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:17.841750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975045+0000","last_change":"2024-04-25T09:09:29.957492+0000","last_active":"2024-04-25T09:09:30.975045+0000","last_peered":"2024-04-25T09:09:30.975045+0000","last_clean":"2024-04-25T09:09:30.975045+0000","last_became_active":"2024-04-25T09:09:29.957197+0000","last_became_peered":"2024-04-25T09:09:29.957197+0000","last_unstale":"2024-04-25T09:09:30.975045+0000","last_undegraded":"2024-04-25T09:09:30.975045+0000","last_fullsized":"2024-04-25T09:09:30.975045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:31.075691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.935906+0000","last_change":"2024-04-25T09:09:28.962797+0000","last_active":"2024-04-25T09:09:29.935906+0000","last_peered":"2024-04-25T09:09:29.935906+0000","last_clean":"2024-04-25T09:09:29.935906+0000","last_became_active":"2024-04-25T09:09:28.961552+0000","last_became_peered":"2024-04-25T09:09:28.961552+0000","last_unstale":"2024-04-25T09:09:29.935906+0000","last_undegraded":"2024-04-25T09:09:29.935906+0000","last_fullsized":"2024-04-25T09:09:29.935906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:34.989639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963762+0000","last_change":"2024-04-25T09:09:29.956755+0000","last_active":"2024-04-25T09:09:29.963762+0000","last_peered":"2024-04-25T09:09:29.963762+0000","last_clean":"2024-04-25T09:09:29.963762+0000","last_became_active":"2024-04-25T09:09:29.956468+0000","last_became_peered":"2024-04-25T09:09:29.956468+0000","last_unstale":"2024-04-25T09:09:29.963762+0000","last_undegraded":"2024-04-25T09:09:29.963762+0000","last_fullsized":"2024-04-25T09:09:29.963762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:47.169527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.942466+0000","last_change":"2024-04-25T09:09:28.956325+0000","last_active":"2024-04-25T09:09:29.942466+0000","last_peered":"2024-04-25T09:09:29.942466+0000","last_clean":"2024-04-25T09:09:29.942466+0000","last_became_active":"2024-04-25T09:09:28.955665+0000","last_became_peered":"2024-04-25T09:09:28.955665+0000","last_unstale":"2024-04-25T09:09:29.942466+0000","last_undegraded":"2024-04-25T09:09:29.942466+0000","last_fullsized":"2024-04-25T09:09:29.942466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:28.159180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.970528+0000","last_change":"2024-04-25T09:09:29.966312+0000","last_active":"2024-04-25T09:09:29.970528+0000","last_peered":"2024-04-25T09:09:29.970528+0000","last_clean":"2024-04-25T09:09:29.970528+0000","last_became_active":"2024-04-25T09:09:29.965785+0000","last_became_peered":"2024-04-25T09:09:29.965785+0000","last_unstale":"2024-04-25T09:09:29.970528+0000","last_undegraded":"2024-04-25T09:09:29.970528+0000","last_fullsized":"2024-04-25T09:09:29.970528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:26.318267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.942900+0000","last_change":"2024-04-25T09:09:28.963696+0000","last_active":"2024-04-25T09:09:29.942900+0000","last_peered":"2024-04-25T09:09:29.942900+0000","last_clean":"2024-04-25T09:09:29.942900+0000","last_became_active":"2024-04-25T09:09:28.963373+0000","last_became_peered":"2024-04-25T09:09:28.963373+0000","last_unstale":"2024-04-25T09:09:29.942900+0000","last_undegraded":"2024-04-25T09:09:29.942900+0000","last_fullsized":"2024-04-25T09:09:29.942900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.855428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.959176+0000","last_change":"2024-04-25T09:09:29.955116+0000","last_active":"2024-04-25T09:09:29.959176+0000","last_peered":"2024-04-25T09:09:29.959176+0000","last_clean":"2024-04-25T09:09:29.959176+0000","last_became_active":"2024-04-25T09:09:29.954280+0000","last_became_peered":"2024-04-25T09:09:29.954280+0000","last_unstale":"2024-04-25T09:09:29.959176+0000","last_undegraded":"2024-04-25T09:09:29.959176+0000","last_fullsized":"2024-04-25T09:09:29.959176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:43.759395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956467+0000","last_change":"2024-04-25T09:09:28.963883+0000","last_active":"2024-04-25T09:09:30.956467+0000","last_peered":"2024-04-25T09:09:30.956467+0000","last_clean":"2024-04-25T09:09:30.956467+0000","last_became_active":"2024-04-25T09:09:28.963611+0000","last_became_peered":"2024-04-25T09:09:28.963611+0000","last_unstale":"2024-04-25T09:09:30.956467+0000","last_undegraded":"2024-04-25T09:09:30.956467+0000","last_fullsized":"2024-04-25T09:09:30.956467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:56.164727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.955753+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:09:30.955753+0000","last_peered":"2024-04-25T09:09:30.955753+0000","last_clean":"2024-04-25T09:09:30.955753+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:09:30.955753+0000","last_undegraded":"2024-04-25T09:09:30.955753+0000","last_fullsized":"2024-04-25T09:09:30.955753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335552,"statfs":{"total":96636764160,"available":96599605248,"internally_reserved":0,"allocated":962560,"data_stored":410694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40444,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331396,"statfs":{"total":96636764160,"available":96595349504,"internally_reserved":0,"allocated":966656,"data_stored":410728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41020,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330820,"statfs":{"total":96636764160,"available":96594759680,"internally_reserved":0,"allocated":1548288,"data_stored":999540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40448,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":962560,"data_stored":409776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58496,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313344,"statfs":{"total":96636764160,"available":96576864256,"internally_reserved":0,"allocated":962560,"data_stored":409208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58480,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313360,"statfs":{"total":96636764160,"available":96576880640,"internally_reserved":0,"allocated":954368,"data_stored":409664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41008,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330832,"statfs":{"total":96636764160,"available":96594771968,"internally_reserved":0,"allocated":1544192,"data_stored":999450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36280,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335560,"statfs":{"total":96636764160,"available":96599613440,"internally_reserved":0,"allocated":954368,"data_stored":409152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:09:34.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:09:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.655+0000 7fbad7200640 1 Processor -- start 2024-04-25T09:09:34.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.656+0000 7fbad7200640 1 -- start start 2024-04-25T09:09:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.657+0000 7fbad7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbad8144c10 0x7fbad815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad80cf2d0 con 0x7fbad806d900 2024-04-25T09:09:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad815faa0 con 0x7fbad80cee40 2024-04-25T09:09:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad8144ff0 con 0x7fbad8144c10 2024-04-25T09:09:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbad8144c10 0x7fbad815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbad8144c10 0x7fbad815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45456/0 (socket says 172.21.15.62:45456) 2024-04-25T09:09:34.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad5e00640 1 -- 172.21.15.62:0/1796457548 learned_addr learned my addr 172.21.15.62:0/1796457548 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:34.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad5e00640 1 -- 172.21.15.62:0/1796457548 >> v1:172.21.15.121:6789/0 conn(0x7fbad80cee40 legacy=0x7fbad806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:34.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad5e00640 1 -- 172.21.15.62:0/1796457548 >> v1:172.21.15.62:6789/0 conn(0x7fbad806d900 legacy=0x7fbad80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.658+0000 7fbad5e00640 1 -- 172.21.15.62:0/1796457548 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad815fd80 con 0x7fbad8144c10 2024-04-25T09:09:34.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.660+0000 7fbad5e00640 1 --2- 172.21.15.62:0/1796457548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad8144c10 0x7fbad815f450 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009870 tx=0x7fbac002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9060c6edc1c7aa70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:34.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.660+0000 7fbad4a00640 1 -- 172.21.15.62:0/1796457548 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0037070 con 0x7fbad8144c10 2024-04-25T09:09:34.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.660+0000 7fbad4a00640 1 -- 172.21.15.62:0/1796457548 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac0032440 con 0x7fbad8144c10 2024-04-25T09:09:34.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.660+0000 7fbad4a00640 1 -- 172.21.15.62:0/1796457548 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0032810 con 0x7fbad8144c10 2024-04-25T09:09:34.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.660+0000 7fbad7200640 1 -- 172.21.15.62:0/1796457548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad8144c10 msgr2=0x7fbad815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.660+0000 7fbad7200640 1 --2- 172.21.15.62:0/1796457548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad8144c10 0x7fbad815f450 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009870 tx=0x7fbac002a210 comp rx=0 tx=0).stop 2024-04-25T09:09:34.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.661+0000 7fbad7200640 1 -- 172.21.15.62:0/1796457548 shutdown_connections 2024-04-25T09:09:34.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.661+0000 7fbad7200640 1 --2- 172.21.15.62:0/1796457548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad8144c10 0x7fbad815f450 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.661+0000 7fbad7200640 1 -- 172.21.15.62:0/1796457548 >> 172.21.15.62:0/1796457548 conn(0x7fbad80b14d0 msgr2=0x7fbad80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:34.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.661+0000 7fbad7200640 1 -- 172.21.15.62:0/1796457548 shutdown_connections 2024-04-25T09:09:34.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.661+0000 7fbad7200640 1 -- 172.21.15.62:0/1796457548 wait complete. 2024-04-25T09:09:34.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.662+0000 7fbad7200640 1 Processor -- start 2024-04-25T09:09:34.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.663+0000 7fbad7200640 1 -- start start 2024-04-25T09:09:34.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.663+0000 7fbad7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad806d900 0x7fbad80c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.663+0000 7fbad7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbad80cee40 0x7fbad81329a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbad8144c10 0x7fbad8132ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b41c0 con 0x7fbad806d900 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8160340 con 0x7fbad8144c10 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80b4d60 con 0x7fbad80cee40 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad806d900 0x7fbad80c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad806d900 0x7fbad80c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45458/0 (socket says 172.21.15.62:45458) 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 -- 172.21.15.62:0/3354293169 learned_addr learned my addr 172.21.15.62:0/3354293169 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 -- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbad80cee40 msgr2=0x7fbad81329a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbad80cee40 0x7fbad81329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 -- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbad8144c10 msgr2=0x7fbad8132ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbad8144c10 0x7fbad8132ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 -- 172.21.15.62:0/3354293169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad8141850 con 0x7fbad806d900 2024-04-25T09:09:34.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.664+0000 7fbad5e00640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad806d900 0x7fbad80c7010 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009870 tx=0x7fbac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:34.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.665+0000 7fbace000640 1 -- 172.21.15.62:0/3354293169 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0041070 con 0x7fbad806d900 2024-04-25T09:09:34.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.665+0000 7fbace000640 1 -- 172.21.15.62:0/3354293169 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac003c870 con 0x7fbad806d900 2024-04-25T09:09:34.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.665+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad8141b30 con 0x7fbad806d900 2024-04-25T09:09:34.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.665+0000 7fbace000640 1 -- 172.21.15.62:0/3354293169 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0037040 con 0x7fbad806d900 2024-04-25T09:09:34.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.665+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81420f0 con 0x7fbad806d900 2024-04-25T09:09:34.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.666+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad804ef20 con 0x7fbad806d900 2024-04-25T09:09:34.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.667+0000 7fbace000640 1 -- 172.21.15.62:0/3354293169 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbac0044070 con 0x7fbad806d900 2024-04-25T09:09:34.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.668+0000 7fbace000640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:34.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.668+0000 7fbace000640 1 -- 172.21.15.62:0/3354293169 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbac00aaf50 con 0x7fbad806d900 2024-04-25T09:09:34.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.669+0000 7fbad5400640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:34.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.670+0000 7fbad5400640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbab406b4d0 0x7fbab406d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbad806ce10 tx=0x7fbac8007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:34.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.673+0000 7fbace000640 1 -- 172.21.15.62:0/3354293169 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbac0079f80 con 0x7fbad806d900 2024-04-25T09:09:34.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.871+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbad8141600 con 0x7fbab406b4d0 2024-04-25T09:09:34.870 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-04-25T09:09:34.883+0000 7f8daf000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:09:34.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.890+0000 7fbace000640 1 -- 172.21.15.62:0/3354293169 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252027 (secure 0 0 0) 0x7fbad8141600 con 0x7fbab406b4d0 2024-04-25T09:09:34.881 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:34.882 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbab406b4d0 msgr2=0x7fbab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbab406b4d0 0x7fbab406d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbad806ce10 tx=0x7fbac8007d30 comp rx=0 tx=0).stop 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad806d900 msgr2=0x7fbad80c7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad806d900 0x7fbad80c7010 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fbac0009870 tx=0x7fbac00046c0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 shutdown_connections 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbad8144c10 0x7fbad8132ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbad80cee40 0x7fbad81329a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 --2- 172.21.15.62:0/3354293169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbad806d900 0x7fbad80c7010 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.896+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 >> 172.21.15.62:0/3354293169 conn(0x7fbad80b14d0 msgr2=0x7fbad815bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.897+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 shutdown_connections 2024-04-25T09:09:34.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:34.897+0000 7fbad7200640 1 -- 172.21.15.62:0/3354293169 wait complete. 2024-04-25T09:09:34.904 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-04-25T09:09:34.029271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352464,"kb_used_data":8648,"kb_used_omap":12,"kb_used_meta":343667,"kb_avail":754622256,"statfs":{"total":773094113280,"available":772733190144,"internally_reserved":0,"allocated":8855552,"data_stored":4458212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":351915568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002209"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975307+0000","last_change":"2024-04-25T09:09:29.962291+0000","last_active":"2024-04-25T09:09:30.975307+0000","last_peered":"2024-04-25T09:09:30.975307+0000","last_clean":"2024-04-25T09:09:30.975307+0000","last_became_active":"2024-04-25T09:09:29.961940+0000","last_became_peered":"2024-04-25T09:09:29.961940+0000","last_unstale":"2024-04-25T09:09:30.975307+0000","last_undegraded":"2024-04-25T09:09:30.975307+0000","last_fullsized":"2024-04-25T09:09:30.975307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.470703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.975077+0000","last_change":"2024-04-25T09:09:29.971694+0000","last_active":"2024-04-25T09:09:29.975077+0000","last_peered":"2024-04-25T09:09:29.975077+0000","last_clean":"2024-04-25T09:09:29.975077+0000","last_became_active":"2024-04-25T09:09:29.971434+0000","last_became_peered":"2024-04-25T09:09:29.971434+0000","last_unstale":"2024-04-25T09:09:29.975077+0000","last_undegraded":"2024-04-25T09:09:29.975077+0000","last_fullsized":"2024-04-25T09:09:29.975077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:48.283315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965464+0000","last_change":"2024-04-25T09:09:29.959739+0000","last_active":"2024-04-25T09:09:29.965464+0000","last_peered":"2024-04-25T09:09:29.965464+0000","last_clean":"2024-04-25T09:09:29.965464+0000","last_became_active":"2024-04-25T09:09:29.959461+0000","last_became_peered":"2024-04-25T09:09:29.959461+0000","last_unstale":"2024-04-25T09:09:29.965464+0000","last_undegraded":"2024-04-25T09:09:29.965464+0000","last_fullsized":"2024-04-25T09:09:29.965464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:08.101675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965491+0000","last_change":"2024-04-25T09:09:29.959710+0000","last_active":"2024-04-25T09:09:29.965491+0000","last_peered":"2024-04-25T09:09:29.965491+0000","last_clean":"2024-04-25T09:09:29.965491+0000","last_became_active":"2024-04-25T09:09:29.959422+0000","last_became_peered":"2024-04-25T09:09:29.959422+0000","last_unstale":"2024-04-25T09:09:29.965491+0000","last_undegraded":"2024-04-25T09:09:29.965491+0000","last_fullsized":"2024-04-25T09:09:29.965491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:35.671401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.957725+0000","last_change":"2024-04-25T09:09:29.948074+0000","last_active":"2024-04-25T09:09:29.957725+0000","last_peered":"2024-04-25T09:09:29.957725+0000","last_clean":"2024-04-25T09:09:29.957725+0000","last_became_active":"2024-04-25T09:09:29.947689+0000","last_became_peered":"2024-04-25T09:09:29.947689+0000","last_unstale":"2024-04-25T09:09:29.957725+0000","last_undegraded":"2024-04-25T09:09:29.957725+0000","last_fullsized":"2024-04-25T09:09:29.957725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:06.590510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961786+0000","last_change":"2024-04-25T09:09:29.958279+0000","last_active":"2024-04-25T09:09:29.961786+0000","last_peered":"2024-04-25T09:09:29.961786+0000","last_clean":"2024-04-25T09:09:29.961786+0000","last_became_active":"2024-04-25T09:09:29.957976+0000","last_became_peered":"2024-04-25T09:09:29.957976+0000","last_unstale":"2024-04-25T09:09:29.961786+0000","last_undegraded":"2024-04-25T09:09:29.961786+0000","last_fullsized":"2024-04-25T09:09:29.961786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:36.006954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956125+0000","last_change":"2024-04-25T09:09:29.950410+0000","last_active":"2024-04-25T09:09:30.956125+0000","last_peered":"2024-04-25T09:09:30.956125+0000","last_clean":"2024-04-25T09:09:30.956125+0000","last_became_active":"2024-04-25T09:09:29.950134+0000","last_became_peered":"2024-04-25T09:09:29.950134+0000","last_unstale":"2024-04-25T09:09:30.956125+0000","last_undegraded":"2024-04-25T09:09:30.956125+0000","last_fullsized":"2024-04-25T09:09:30.956125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:07.318360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974537+0000","last_change":"2024-04-25T09:09:29.952586+0000","last_active":"2024-04-25T09:09:30.974537+0000","last_peered":"2024-04-25T09:09:30.974537+0000","last_clean":"2024-04-25T09:09:30.974537+0000","last_became_active":"2024-04-25T09:09:29.952291+0000","last_became_peered":"2024-04-25T09:09:29.952291+0000","last_unstale":"2024-04-25T09:09:30.974537+0000","last_undegraded":"2024-04-25T09:09:30.974537+0000","last_fullsized":"2024-04-25T09:09:30.974537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:34.493863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.966593+0000","last_change":"2024-04-25T09:09:29.964243+0000","last_active":"2024-04-25T09:09:29.966593+0000","last_peered":"2024-04-25T09:09:29.966593+0000","last_clean":"2024-04-25T09:09:29.966593+0000","last_became_active":"2024-04-25T09:09:29.963687+0000","last_became_peered":"2024-04-25T09:09:29.963687+0000","last_unstale":"2024-04-25T09:09:29.966593+0000","last_undegraded":"2024-04-25T09:09:29.966593+0000","last_fullsized":"2024-04-25T09:09:29.966593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:42.518214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956814+0000","last_change":"2024-04-25T09:09:29.961673+0000","last_active":"2024-04-25T09:09:30.956814+0000","last_peered":"2024-04-25T09:09:30.956814+0000","last_clean":"2024-04-25T09:09:30.956814+0000","last_became_active":"2024-04-25T09:09:29.961405+0000","last_became_peered":"2024-04-25T09:09:29.961405+0000","last_unstale":"2024-04-25T09:09:30.956814+0000","last_undegraded":"2024-04-25T09:09:30.956814+0000","last_fullsized":"2024-04-25T09:09:30.956814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:01.587456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975143+0000","last_change":"2024-04-25T09:09:29.962370+0000","last_active":"2024-04-25T09:09:30.975143+0000","last_peered":"2024-04-25T09:09:30.975143+0000","last_clean":"2024-04-25T09:09:30.975143+0000","last_became_active":"2024-04-25T09:09:29.962139+0000","last_became_peered":"2024-04-25T09:09:29.962139+0000","last_unstale":"2024-04-25T09:09:30.975143+0000","last_undegraded":"2024-04-25T09:09:30.975143+0000","last_fullsized":"2024-04-25T09:09:30.975143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:13.010727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962329+0000","last_change":"2024-04-25T09:09:29.960094+0000","last_active":"2024-04-25T09:09:30.962329+0000","last_peered":"2024-04-25T09:09:30.962329+0000","last_clean":"2024-04-25T09:09:30.962329+0000","last_became_active":"2024-04-25T09:09:29.959711+0000","last_became_peered":"2024-04-25T09:09:29.959711+0000","last_unstale":"2024-04-25T09:09:30.962329+0000","last_undegraded":"2024-04-25T09:09:30.962329+0000","last_fullsized":"2024-04-25T09:09:30.962329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:39.951616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963420+0000","last_change":"2024-04-25T09:09:29.958204+0000","last_active":"2024-04-25T09:09:29.963420+0000","last_peered":"2024-04-25T09:09:29.963420+0000","last_clean":"2024-04-25T09:09:29.963420+0000","last_became_active":"2024-04-25T09:09:29.957909+0000","last_became_peered":"2024-04-25T09:09:29.957909+0000","last_unstale":"2024-04-25T09:09:29.963420+0000","last_undegraded":"2024-04-25T09:09:29.963420+0000","last_fullsized":"2024-04-25T09:09:29.963420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:57.182042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961995+0000","last_change":"2024-04-25T09:09:29.961247+0000","last_active":"2024-04-25T09:09:30.961995+0000","last_peered":"2024-04-25T09:09:30.961995+0000","last_clean":"2024-04-25T09:09:30.961995+0000","last_became_active":"2024-04-25T09:09:29.960259+0000","last_became_peered":"2024-04-25T09:09:29.960259+0000","last_unstale":"2024-04-25T09:09:30.961995+0000","last_undegraded":"2024-04-25T09:09:30.961995+0000","last_fullsized":"2024-04-25T09:09:30.961995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:38.680490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961731+0000","last_change":"2024-04-25T09:09:29.961263+0000","last_active":"2024-04-25T09:09:30.961731+0000","last_peered":"2024-04-25T09:09:30.961731+0000","last_clean":"2024-04-25T09:09:30.961731+0000","last_became_active":"2024-04-25T09:09:29.960133+0000","last_became_peered":"2024-04-25T09:09:29.960133+0000","last_unstale":"2024-04-25T09:09:30.961731+0000","last_undegraded":"2024-04-25T09:09:30.961731+0000","last_fullsized":"2024-04-25T09:09:30.961731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:48.503660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974362+0000","last_change":"2024-04-25T09:09:29.961206+0000","last_active":"2024-04-25T09:09:30.974362+0000","last_peered":"2024-04-25T09:09:30.974362+0000","last_clean":"2024-04-25T09:09:30.974362+0000","last_became_active":"2024-04-25T09:09:29.957313+0000","last_became_peered":"2024-04-25T09:09:29.957313+0000","last_unstale":"2024-04-25T09:09:30.974362+0000","last_undegraded":"2024-04-25T09:09:30.974362+0000","last_fullsized":"2024-04-25T09:09:30.974362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.851729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961459+0000","last_change":"2024-04-25T09:09:29.957432+0000","last_active":"2024-04-25T09:09:29.961459+0000","last_peered":"2024-04-25T09:09:29.961459+0000","last_clean":"2024-04-25T09:09:29.961459+0000","last_became_active":"2024-04-25T09:09:29.957127+0000","last_became_peered":"2024-04-25T09:09:29.957127+0000","last_unstale":"2024-04-25T09:09:29.961459+0000","last_undegraded":"2024-04-25T09:09:29.961459+0000","last_fullsized":"2024-04-25T09:09:29.961459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:13.914484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.969125+0000","last_change":"2024-04-25T09:09:29.964899+0000","last_active":"2024-04-25T09:09:29.969125+0000","last_peered":"2024-04-25T09:09:29.969125+0000","last_clean":"2024-04-25T09:09:29.969125+0000","last_became_active":"2024-04-25T09:09:29.964567+0000","last_became_peered":"2024-04-25T09:09:29.964567+0000","last_unstale":"2024-04-25T09:09:29.969125+0000","last_undegraded":"2024-04-25T09:09:29.969125+0000","last_fullsized":"2024-04-25T09:09:29.969125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:02.458267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974972+0000","last_change":"2024-04-25T09:09:29.967065+0000","last_active":"2024-04-25T09:09:30.974972+0000","last_peered":"2024-04-25T09:09:30.974972+0000","last_clean":"2024-04-25T09:09:30.974972+0000","last_became_active":"2024-04-25T09:09:29.966846+0000","last_became_peered":"2024-04-25T09:09:29.966846+0000","last_unstale":"2024-04-25T09:09:30.974972+0000","last_undegraded":"2024-04-25T09:09:30.974972+0000","last_fullsized":"2024-04-25T09:09:30.974972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:49.341010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.958052+0000","last_change":"2024-04-25T09:09:29.951957+0000","last_active":"2024-04-25T09:09:29.958052+0000","last_peered":"2024-04-25T09:09:29.958052+0000","last_clean":"2024-04-25T09:09:29.958052+0000","last_became_active":"2024-04-25T09:09:29.951519+0000","last_became_peered":"2024-04-25T09:09:29.951519+0000","last_unstale":"2024-04-25T09:09:29.958052+0000","last_undegraded":"2024-04-25T09:09:29.958052+0000","last_fullsized":"2024-04-25T09:09:29.958052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:35.186664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.966066+0000","last_change":"2024-04-25T09:09:29.961508+0000","last_active":"2024-04-25T09:09:29.966066+0000","last_peered":"2024-04-25T09:09:29.966066+0000","last_clean":"2024-04-25T09:09:29.966066+0000","last_became_active":"2024-04-25T09:09:29.961208+0000","last_became_peered":"2024-04-25T09:09:29.961208+0000","last_unstale":"2024-04-25T09:09:29.966066+0000","last_undegraded":"2024-04-25T09:09:29.966066+0000","last_fullsized":"2024-04-25T09:09:29.966066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:47.169527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.956081+0000","last_change":"2024-04-25T09:09:29.953361+0000","last_active":"2024-04-25T09:09:29.956081+0000","last_peered":"2024-04-25T09:09:29.956081+0000","last_clean":"2024-04-25T09:09:29.956081+0000","last_became_active":"2024-04-25T09:09:29.951186+0000","last_became_peered":"2024-04-25T09:09:29.951186+0000","last_unstale":"2024-04-25T09:09:29.956081+0000","last_undegraded":"2024-04-25T09:09:29.956081+0000","last_fullsized":"2024-04-25T09:09:29.956081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:06.126657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956099+0000","last_change":"2024-04-25T09:09:29.958350+0000","last_active":"2024-04-25T09:09:30.956099+0000","last_peered":"2024-04-25T09:09:30.956099+0000","last_clean":"2024-04-25T09:09:30.956099+0000","last_became_active":"2024-04-25T09:09:29.958065+0000","last_became_peered":"2024-04-25T09:09:29.958065+0000","last_unstale":"2024-04-25T09:09:30.956099+0000","last_undegraded":"2024-04-25T09:09:30.956099+0000","last_fullsized":"2024-04-25T09:09:30.956099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:26.238732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975446+0000","last_change":"2024-04-25T09:09:29.959930+0000","last_active":"2024-04-25T09:09:30.975446+0000","last_peered":"2024-04-25T09:09:30.975446+0000","last_clean":"2024-04-25T09:09:30.975446+0000","last_became_active":"2024-04-25T09:09:29.959341+0000","last_became_peered":"2024-04-25T09:09:29.959341+0000","last_unstale":"2024-04-25T09:09:30.975446+0000","last_undegraded":"2024-04-25T09:09:30.975446+0000","last_fullsized":"2024-04-25T09:09:30.975446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:40.320391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956579+0000","last_change":"2024-04-25T09:09:29.958760+0000","last_active":"2024-04-25T09:09:30.956579+0000","last_peered":"2024-04-25T09:09:30.956579+0000","last_clean":"2024-04-25T09:09:30.956579+0000","last_became_active":"2024-04-25T09:09:29.958172+0000","last_became_peered":"2024-04-25T09:09:29.958172+0000","last_unstale":"2024-04-25T09:09:30.956579+0000","last_undegraded":"2024-04-25T09:09:30.956579+0000","last_fullsized":"2024-04-25T09:09:30.956579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:06.081153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975264+0000","last_change":"2024-04-25T09:09:29.970156+0000","last_active":"2024-04-25T09:09:30.975264+0000","last_peered":"2024-04-25T09:09:30.975264+0000","last_clean":"2024-04-25T09:09:30.975264+0000","last_became_active":"2024-04-25T09:09:29.969578+0000","last_became_peered":"2024-04-25T09:09:29.969578+0000","last_unstale":"2024-04-25T09:09:30.975264+0000","last_undegraded":"2024-04-25T09:09:30.975264+0000","last_fullsized":"2024-04-25T09:09:30.975264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:16.077887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961498+0000","last_change":"2024-04-25T09:09:29.955733+0000","last_active":"2024-04-25T09:09:29.961498+0000","last_peered":"2024-04-25T09:09:29.961498+0000","last_clean":"2024-04-25T09:09:29.961498+0000","last_became_active":"2024-04-25T09:09:29.955130+0000","last_became_peered":"2024-04-25T09:09:29.955130+0000","last_unstale":"2024-04-25T09:09:29.961498+0000","last_undegraded":"2024-04-25T09:09:29.961498+0000","last_fullsized":"2024-04-25T09:09:29.961498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:55.957705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.972308+0000","last_change":"2024-04-25T09:09:29.966749+0000","last_active":"2024-04-25T09:09:29.972308+0000","last_peered":"2024-04-25T09:09:29.972308+0000","last_clean":"2024-04-25T09:09:29.972308+0000","last_became_active":"2024-04-25T09:09:29.966456+0000","last_became_peered":"2024-04-25T09:09:29.966456+0000","last_unstale":"2024-04-25T09:09:29.972308+0000","last_undegraded":"2024-04-25T09:09:29.972308+0000","last_fullsized":"2024-04-25T09:09:29.972308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:37.258879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.969209+0000","last_change":"2024-04-25T09:09:29.965273+0000","last_active":"2024-04-25T09:09:29.969209+0000","last_peered":"2024-04-25T09:09:29.969209+0000","last_clean":"2024-04-25T09:09:29.969209+0000","last_became_active":"2024-04-25T09:09:29.964129+0000","last_became_peered":"2024-04-25T09:09:29.964129+0000","last_unstale":"2024-04-25T09:09:29.969209+0000","last_undegraded":"2024-04-25T09:09:29.969209+0000","last_fullsized":"2024-04-25T09:09:29.969209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:50.877336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975057+0000","last_change":"2024-04-25T09:09:29.965545+0000","last_active":"2024-04-25T09:09:30.975057+0000","last_peered":"2024-04-25T09:09:30.975057+0000","last_clean":"2024-04-25T09:09:30.975057+0000","last_became_active":"2024-04-25T09:09:29.964606+0000","last_became_peered":"2024-04-25T09:09:29.964606+0000","last_unstale":"2024-04-25T09:09:30.975057+0000","last_undegraded":"2024-04-25T09:09:30.975057+0000","last_fullsized":"2024-04-25T09:09:30.975057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:45.965290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.966447+0000","last_change":"2024-04-25T09:09:29.962536+0000","last_active":"2024-04-25T09:09:29.966447+0000","last_peered":"2024-04-25T09:09:29.966447+0000","last_clean":"2024-04-25T09:09:29.966447+0000","last_became_active":"2024-04-25T09:09:29.962132+0000","last_became_peered":"2024-04-25T09:09:29.962132+0000","last_unstale":"2024-04-25T09:09:29.966447+0000","last_undegraded":"2024-04-25T09:09:29.966447+0000","last_fullsized":"2024-04-25T09:09:29.966447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:40.507663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.972174+0000","last_change":"2024-04-25T09:09:29.968554+0000","last_active":"2024-04-25T09:09:29.972174+0000","last_peered":"2024-04-25T09:09:29.972174+0000","last_clean":"2024-04-25T09:09:29.972174+0000","last_became_active":"2024-04-25T09:09:29.968190+0000","last_became_peered":"2024-04-25T09:09:29.968190+0000","last_unstale":"2024-04-25T09:09:29.972174+0000","last_undegraded":"2024-04-25T09:09:29.972174+0000","last_fullsized":"2024-04-25T09:09:29.972174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:00.872742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974764+0000","last_change":"2024-04-25T09:09:29.970365+0000","last_active":"2024-04-25T09:09:30.974764+0000","last_peered":"2024-04-25T09:09:30.974764+0000","last_clean":"2024-04-25T09:09:30.974764+0000","last_became_active":"2024-04-25T09:09:29.969908+0000","last_became_peered":"2024-04-25T09:09:29.969908+0000","last_unstale":"2024-04-25T09:09:30.974764+0000","last_undegraded":"2024-04-25T09:09:30.974764+0000","last_fullsized":"2024-04-25T09:09:30.974764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:52.126126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.983995+0000","last_change":"2024-04-25T09:09:28.963922+0000","last_active":"2024-04-25T09:09:29.983995+0000","last_peered":"2024-04-25T09:09:29.983995+0000","last_clean":"2024-04-25T09:09:29.983995+0000","last_became_active":"2024-04-25T09:09:28.963563+0000","last_became_peered":"2024-04-25T09:09:28.963563+0000","last_unstale":"2024-04-25T09:09:29.983995+0000","last_undegraded":"2024-04-25T09:09:29.983995+0000","last_fullsized":"2024-04-25T09:09:29.983995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:23.286506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965460+0000","last_change":"2024-04-25T09:09:29.960876+0000","last_active":"2024-04-25T09:09:29.965460+0000","last_peered":"2024-04-25T09:09:29.965460+0000","last_clean":"2024-04-25T09:09:29.965460+0000","last_became_active":"2024-04-25T09:09:29.960357+0000","last_became_peered":"2024-04-25T09:09:29.960357+0000","last_unstale":"2024-04-25T09:09:29.965460+0000","last_undegraded":"2024-04-25T09:09:29.965460+0000","last_fullsized":"2024-04-25T09:09:29.965460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:17.093842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.980988+0000","last_change":"2024-04-25T09:09:28.964540+0000","last_active":"2024-04-25T09:09:29.980988+0000","last_peered":"2024-04-25T09:09:29.980988+0000","last_clean":"2024-04-25T09:09:29.980988+0000","last_became_active":"2024-04-25T09:09:28.963930+0000","last_became_peered":"2024-04-25T09:09:28.963930+0000","last_unstale":"2024-04-25T09:09:29.980988+0000","last_undegraded":"2024-04-25T09:09:29.980988+0000","last_fullsized":"2024-04-25T09:09:29.980988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.960112+0000","last_change":"2024-04-25T09:09:29.956250+0000","last_active":"2024-04-25T09:09:29.960112+0000","last_peered":"2024-04-25T09:09:29.960112+0000","last_clean":"2024-04-25T09:09:29.960112+0000","last_became_active":"2024-04-25T09:09:29.955976+0000","last_became_peered":"2024-04-25T09:09:29.955976+0000","last_unstale":"2024-04-25T09:09:29.960112+0000","last_undegraded":"2024-04-25T09:09:29.960112+0000","last_fullsized":"2024-04-25T09:09:29.960112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:39.951616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975132+0000","last_change":"2024-04-25T09:09:28.958191+0000","last_active":"2024-04-25T09:09:30.975132+0000","last_peered":"2024-04-25T09:09:30.975132+0000","last_clean":"2024-04-25T09:09:30.975132+0000","last_became_active":"2024-04-25T09:09:28.957873+0000","last_became_peered":"2024-04-25T09:09:28.957873+0000","last_unstale":"2024-04-25T09:09:30.975132+0000","last_undegraded":"2024-04-25T09:09:30.975132+0000","last_fullsized":"2024-04-25T09:09:30.975132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:51.354823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956369+0000","last_change":"2024-04-25T09:09:29.968427+0000","last_active":"2024-04-25T09:09:30.956369+0000","last_peered":"2024-04-25T09:09:30.956369+0000","last_clean":"2024-04-25T09:09:30.956369+0000","last_became_active":"2024-04-25T09:09:29.960209+0000","last_became_peered":"2024-04-25T09:09:29.960209+0000","last_unstale":"2024-04-25T09:09:30.956369+0000","last_undegraded":"2024-04-25T09:09:30.956369+0000","last_fullsized":"2024-04-25T09:09:30.956369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:55.096541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.980227+0000","last_change":"2024-04-25T09:09:28.964799+0000","last_active":"2024-04-25T09:09:29.980227+0000","last_peered":"2024-04-25T09:09:29.980227+0000","last_clean":"2024-04-25T09:09:29.980227+0000","last_became_active":"2024-04-25T09:09:28.964535+0000","last_became_peered":"2024-04-25T09:09:28.964535+0000","last_unstale":"2024-04-25T09:09:29.980227+0000","last_undegraded":"2024-04-25T09:09:29.980227+0000","last_fullsized":"2024-04-25T09:09:29.980227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:47.486345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974718+0000","last_change":"2024-04-25T09:09:29.961835+0000","last_active":"2024-04-25T09:09:30.974718+0000","last_peered":"2024-04-25T09:09:30.974718+0000","last_clean":"2024-04-25T09:09:30.974718+0000","last_became_active":"2024-04-25T09:09:29.960702+0000","last_became_peered":"2024-04-25T09:09:29.960702+0000","last_unstale":"2024-04-25T09:09:30.974718+0000","last_undegraded":"2024-04-25T09:09:30.974718+0000","last_fullsized":"2024-04-25T09:09:30.974718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:17.507627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961956+0000","last_change":"2024-04-25T09:09:28.968881+0000","last_active":"2024-04-25T09:09:30.961956+0000","last_peered":"2024-04-25T09:09:30.961956+0000","last_clean":"2024-04-25T09:09:30.961956+0000","last_became_active":"2024-04-25T09:09:28.968246+0000","last_became_peered":"2024-04-25T09:09:28.968246+0000","last_unstale":"2024-04-25T09:09:30.961956+0000","last_undegraded":"2024-04-25T09:09:30.961956+0000","last_fullsized":"2024-04-25T09:09:30.961956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:55.947582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.968757+0000","last_change":"2024-04-25T09:09:29.965006+0000","last_active":"2024-04-25T09:09:29.968757+0000","last_peered":"2024-04-25T09:09:29.968757+0000","last_clean":"2024-04-25T09:09:29.968757+0000","last_became_active":"2024-04-25T09:09:29.964332+0000","last_became_peered":"2024-04-25T09:09:29.964332+0000","last_unstale":"2024-04-25T09:09:29.968757+0000","last_undegraded":"2024-04-25T09:09:29.968757+0000","last_fullsized":"2024-04-25T09:09:29.968757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:06.590510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956031+0000","last_change":"2024-04-25T09:09:28.962365+0000","last_active":"2024-04-25T09:09:30.956031+0000","last_peered":"2024-04-25T09:09:30.956031+0000","last_clean":"2024-04-25T09:09:30.956031+0000","last_became_active":"2024-04-25T09:09:28.962047+0000","last_became_peered":"2024-04-25T09:09:28.962047+0000","last_unstale":"2024-04-25T09:09:30.956031+0000","last_undegraded":"2024-04-25T09:09:30.956031+0000","last_fullsized":"2024-04-25T09:09:30.956031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:37.754042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965958+0000","last_change":"2024-04-25T09:09:29.963625+0000","last_active":"2024-04-25T09:09:29.965958+0000","last_peered":"2024-04-25T09:09:29.965958+0000","last_clean":"2024-04-25T09:09:29.965958+0000","last_became_active":"2024-04-25T09:09:29.962966+0000","last_became_peered":"2024-04-25T09:09:29.962966+0000","last_unstale":"2024-04-25T09:09:29.965958+0000","last_undegraded":"2024-04-25T09:09:29.965958+0000","last_fullsized":"2024-04-25T09:09:29.965958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:49.077325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.949003+0000","last_change":"2024-04-25T09:09:28.969333+0000","last_active":"2024-04-25T09:09:29.949003+0000","last_peered":"2024-04-25T09:09:29.949003+0000","last_clean":"2024-04-25T09:09:29.949003+0000","last_became_active":"2024-04-25T09:09:28.969041+0000","last_became_peered":"2024-04-25T09:09:28.969041+0000","last_unstale":"2024-04-25T09:09:29.949003+0000","last_undegraded":"2024-04-25T09:09:29.949003+0000","last_fullsized":"2024-04-25T09:09:29.949003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:55.947582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961707+0000","last_change":"2024-04-25T09:09:29.961992+0000","last_active":"2024-04-25T09:09:30.961707+0000","last_peered":"2024-04-25T09:09:30.961707+0000","last_clean":"2024-04-25T09:09:30.961707+0000","last_became_active":"2024-04-25T09:09:29.960713+0000","last_became_peered":"2024-04-25T09:09:29.960713+0000","last_unstale":"2024-04-25T09:09:30.961707+0000","last_undegraded":"2024-04-25T09:09:30.961707+0000","last_fullsized":"2024-04-25T09:09:30.961707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:24.303821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.961983+0000","last_change":"2024-04-25T09:09:28.967156+0000","last_active":"2024-04-25T09:09:30.961983+0000","last_peered":"2024-04-25T09:09:30.961983+0000","last_clean":"2024-04-25T09:09:30.961983+0000","last_became_active":"2024-04-25T09:09:28.964457+0000","last_became_peered":"2024-04-25T09:09:28.964457+0000","last_unstale":"2024-04-25T09:09:30.961983+0000","last_undegraded":"2024-04-25T09:09:30.961983+0000","last_fullsized":"2024-04-25T09:09:30.961983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:39.490348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.957174+0000","last_change":"2024-04-25T09:09:29.961617+0000","last_active":"2024-04-25T09:09:30.957174+0000","last_peered":"2024-04-25T09:09:30.957174+0000","last_clean":"2024-04-25T09:09:30.957174+0000","last_became_active":"2024-04-25T09:09:29.961118+0000","last_became_peered":"2024-04-25T09:09:29.961118+0000","last_unstale":"2024-04-25T09:09:30.957174+0000","last_undegraded":"2024-04-25T09:09:30.957174+0000","last_fullsized":"2024-04-25T09:09:30.957174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.648854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974860+0000","last_change":"2024-04-25T09:09:28.968252+0000","last_active":"2024-04-25T09:09:30.974860+0000","last_peered":"2024-04-25T09:09:30.974860+0000","last_clean":"2024-04-25T09:09:30.974860+0000","last_became_active":"2024-04-25T09:09:28.967946+0000","last_became_peered":"2024-04-25T09:09:28.967946+0000","last_unstale":"2024-04-25T09:09:30.974860+0000","last_undegraded":"2024-04-25T09:09:30.974860+0000","last_fullsized":"2024-04-25T09:09:30.974860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:28.096362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.970699+0000","last_change":"2024-04-25T09:09:29.966068+0000","last_active":"2024-04-25T09:09:29.970699+0000","last_peered":"2024-04-25T09:09:29.970699+0000","last_clean":"2024-04-25T09:09:29.970699+0000","last_became_active":"2024-04-25T09:09:29.965714+0000","last_became_peered":"2024-04-25T09:09:29.965714+0000","last_unstale":"2024-04-25T09:09:29.970699+0000","last_undegraded":"2024-04-25T09:09:29.970699+0000","last_fullsized":"2024-04-25T09:09:29.970699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:49.077325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.954085+0000","last_change":"2024-04-25T09:09:28.954777+0000","last_active":"2024-04-25T09:09:29.954085+0000","last_peered":"2024-04-25T09:09:29.954085+0000","last_clean":"2024-04-25T09:09:29.954085+0000","last_became_active":"2024-04-25T09:09:28.954456+0000","last_became_peered":"2024-04-25T09:09:28.954456+0000","last_unstale":"2024-04-25T09:09:29.954085+0000","last_undegraded":"2024-04-25T09:09:29.954085+0000","last_fullsized":"2024-04-25T09:09:29.954085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:36.293123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975066+0000","last_change":"2024-04-25T09:09:29.962015+0000","last_active":"2024-04-25T09:09:30.975066+0000","last_peered":"2024-04-25T09:09:30.975066+0000","last_clean":"2024-04-25T09:09:30.975066+0000","last_became_active":"2024-04-25T09:09:29.961739+0000","last_became_peered":"2024-04-25T09:09:29.961739+0000","last_unstale":"2024-04-25T09:09:30.975066+0000","last_undegraded":"2024-04-25T09:09:30.975066+0000","last_fullsized":"2024-04-25T09:09:30.975066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:38.924704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962508+0000","last_change":"2024-04-25T09:09:28.967736+0000","last_active":"2024-04-25T09:09:30.962508+0000","last_peered":"2024-04-25T09:09:30.962508+0000","last_clean":"2024-04-25T09:09:30.962508+0000","last_became_active":"2024-04-25T09:09:28.966985+0000","last_became_peered":"2024-04-25T09:09:28.966985+0000","last_unstale":"2024-04-25T09:09:30.962508+0000","last_undegraded":"2024-04-25T09:09:30.962508+0000","last_fullsized":"2024-04-25T09:09:30.962508+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:46.111525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961477+0000","last_change":"2024-04-25T09:09:29.955413+0000","last_active":"2024-04-25T09:09:29.961477+0000","last_peered":"2024-04-25T09:09:29.961477+0000","last_clean":"2024-04-25T09:09:29.961477+0000","last_became_active":"2024-04-25T09:09:29.954652+0000","last_became_peered":"2024-04-25T09:09:29.954652+0000","last_unstale":"2024-04-25T09:09:29.961477+0000","last_undegraded":"2024-04-25T09:09:29.961477+0000","last_fullsized":"2024-04-25T09:09:29.961477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:07.861832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"78'6","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.987693+0000","last_change":"2024-04-25T09:09:28.964335+0000","last_active":"2024-04-25T09:09:29.987693+0000","last_peered":"2024-04-25T09:09:29.987693+0000","last_clean":"2024-04-25T09:09:29.987693+0000","last_became_active":"2024-04-25T09:09:28.963713+0000","last_became_peered":"2024-04-25T09:09:28.963713+0000","last_unstale":"2024-04-25T09:09:29.987693+0000","last_undegraded":"2024-04-25T09:09:29.987693+0000","last_fullsized":"2024-04-25T09:09:29.987693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:17.841750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963590+0000","last_change":"2024-04-25T09:09:29.962597+0000","last_active":"2024-04-25T09:09:29.963590+0000","last_peered":"2024-04-25T09:09:29.963590+0000","last_clean":"2024-04-25T09:09:29.963590+0000","last_became_active":"2024-04-25T09:09:29.960200+0000","last_became_peered":"2024-04-25T09:09:29.960200+0000","last_unstale":"2024-04-25T09:09:29.963590+0000","last_undegraded":"2024-04-25T09:09:29.963590+0000","last_fullsized":"2024-04-25T09:09:29.963590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:44.532729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.983126+0000","last_change":"2024-04-25T09:09:28.962537+0000","last_active":"2024-04-25T09:09:29.983126+0000","last_peered":"2024-04-25T09:09:29.983126+0000","last_clean":"2024-04-25T09:09:29.983126+0000","last_became_active":"2024-04-25T09:09:28.962236+0000","last_became_peered":"2024-04-25T09:09:28.962236+0000","last_unstale":"2024-04-25T09:09:29.983126+0000","last_undegraded":"2024-04-25T09:09:29.983126+0000","last_fullsized":"2024-04-25T09:09:29.983126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:36.293123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956315+0000","last_change":"2024-04-25T09:09:29.971364+0000","last_active":"2024-04-25T09:09:30.956315+0000","last_peered":"2024-04-25T09:09:30.956315+0000","last_clean":"2024-04-25T09:09:30.956315+0000","last_became_active":"2024-04-25T09:09:29.971042+0000","last_became_peered":"2024-04-25T09:09:29.971042+0000","last_unstale":"2024-04-25T09:09:30.956315+0000","last_undegraded":"2024-04-25T09:09:30.956315+0000","last_fullsized":"2024-04-25T09:09:30.956315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:55.551161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956034+0000","last_change":"2024-04-25T09:09:28.946867+0000","last_active":"2024-04-25T09:09:30.956034+0000","last_peered":"2024-04-25T09:09:30.956034+0000","last_clean":"2024-04-25T09:09:30.956034+0000","last_became_active":"2024-04-25T09:09:28.946542+0000","last_became_peered":"2024-04-25T09:09:28.946542+0000","last_unstale":"2024-04-25T09:09:30.956034+0000","last_undegraded":"2024-04-25T09:09:30.956034+0000","last_fullsized":"2024-04-25T09:09:30.956034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:11.437718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.958426+0000","last_change":"2024-04-25T09:09:29.952029+0000","last_active":"2024-04-25T09:09:29.958426+0000","last_peered":"2024-04-25T09:09:29.958426+0000","last_clean":"2024-04-25T09:09:29.958426+0000","last_became_active":"2024-04-25T09:09:29.951449+0000","last_became_peered":"2024-04-25T09:09:29.951449+0000","last_unstale":"2024-04-25T09:09:29.958426+0000","last_undegraded":"2024-04-25T09:09:29.958426+0000","last_fullsized":"2024-04-25T09:09:29.958426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:02.191228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956194+0000","last_change":"2024-04-25T09:09:28.965099+0000","last_active":"2024-04-25T09:09:30.956194+0000","last_peered":"2024-04-25T09:09:30.956194+0000","last_clean":"2024-04-25T09:09:30.956194+0000","last_became_active":"2024-04-25T09:09:28.964801+0000","last_became_peered":"2024-04-25T09:09:28.964801+0000","last_unstale":"2024-04-25T09:09:30.956194+0000","last_undegraded":"2024-04-25T09:09:30.956194+0000","last_fullsized":"2024-04-25T09:09:30.956194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:34.654086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.959211+0000","last_change":"2024-04-25T09:09:29.956202+0000","last_active":"2024-04-25T09:09:29.959211+0000","last_peered":"2024-04-25T09:09:29.959211+0000","last_clean":"2024-04-25T09:09:29.959211+0000","last_became_active":"2024-04-25T09:09:29.955852+0000","last_became_peered":"2024-04-25T09:09:29.955852+0000","last_unstale":"2024-04-25T09:09:29.959211+0000","last_undegraded":"2024-04-25T09:09:29.959211+0000","last_fullsized":"2024-04-25T09:09:29.959211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:38.771357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.955817+0000","last_change":"2024-04-25T09:09:28.959555+0000","last_active":"2024-04-25T09:09:30.955817+0000","last_peered":"2024-04-25T09:09:30.955817+0000","last_clean":"2024-04-25T09:09:30.955817+0000","last_became_active":"2024-04-25T09:09:28.958622+0000","last_became_peered":"2024-04-25T09:09:28.958622+0000","last_unstale":"2024-04-25T09:09:30.955817+0000","last_undegraded":"2024-04-25T09:09:30.955817+0000","last_fullsized":"2024-04-25T09:09:30.955817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:29.802770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965240+0000","last_change":"2024-04-25T09:09:29.954349+0000","last_active":"2024-04-25T09:09:29.965240+0000","last_peered":"2024-04-25T09:09:29.965240+0000","last_clean":"2024-04-25T09:09:29.965240+0000","last_became_active":"2024-04-25T09:09:29.953733+0000","last_became_peered":"2024-04-25T09:09:29.953733+0000","last_unstale":"2024-04-25T09:09:29.965240+0000","last_undegraded":"2024-04-25T09:09:29.965240+0000","last_fullsized":"2024-04-25T09:09:29.965240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:18.859065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.974864+0000","last_change":"2024-04-25T09:09:28.965522+0000","last_active":"2024-04-25T09:09:30.974864+0000","last_peered":"2024-04-25T09:09:30.974864+0000","last_clean":"2024-04-25T09:09:30.974864+0000","last_became_active":"2024-04-25T09:09:28.965245+0000","last_became_peered":"2024-04-25T09:09:28.965245+0000","last_unstale":"2024-04-25T09:09:30.974864+0000","last_undegraded":"2024-04-25T09:09:30.974864+0000","last_fullsized":"2024-04-25T09:09:30.974864+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:09.571415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.967659+0000","last_change":"2024-04-25T09:09:29.964385+0000","last_active":"2024-04-25T09:09:29.967659+0000","last_peered":"2024-04-25T09:09:29.967659+0000","last_clean":"2024-04-25T09:09:29.967659+0000","last_became_active":"2024-04-25T09:09:29.964110+0000","last_became_peered":"2024-04-25T09:09:29.964110+0000","last_unstale":"2024-04-25T09:09:29.967659+0000","last_undegraded":"2024-04-25T09:09:29.967659+0000","last_fullsized":"2024-04-25T09:09:29.967659+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:47.128840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.980141+0000","last_change":"2024-04-25T09:09:28.952739+0000","last_active":"2024-04-25T09:09:29.980141+0000","last_peered":"2024-04-25T09:09:29.980141+0000","last_clean":"2024-04-25T09:09:29.980141+0000","last_became_active":"2024-04-25T09:09:28.952271+0000","last_became_peered":"2024-04-25T09:09:28.952271+0000","last_unstale":"2024-04-25T09:09:29.980141+0000","last_undegraded":"2024-04-25T09:09:29.980141+0000","last_fullsized":"2024-04-25T09:09:29.980141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:55.947582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963614+0000","last_change":"2024-04-25T09:09:29.958492+0000","last_active":"2024-04-25T09:09:29.963614+0000","last_peered":"2024-04-25T09:09:29.963614+0000","last_clean":"2024-04-25T09:09:29.963614+0000","last_became_active":"2024-04-25T09:09:29.958168+0000","last_became_peered":"2024-04-25T09:09:29.958168+0000","last_unstale":"2024-04-25T09:09:29.963614+0000","last_undegraded":"2024-04-25T09:09:29.963614+0000","last_fullsized":"2024-04-25T09:09:29.963614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:30.820085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.977708+0000","last_change":"2024-04-25T09:09:28.965611+0000","last_active":"2024-04-25T09:09:29.977708+0000","last_peered":"2024-04-25T09:09:29.977708+0000","last_clean":"2024-04-25T09:09:29.977708+0000","last_became_active":"2024-04-25T09:09:28.965307+0000","last_became_peered":"2024-04-25T09:09:28.965307+0000","last_unstale":"2024-04-25T09:09:29.977708+0000","last_undegraded":"2024-04-25T09:09:29.977708+0000","last_fullsized":"2024-04-25T09:09:29.977708+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:16.405829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.962501+0000","last_change":"2024-04-25T09:09:29.957555+0000","last_active":"2024-04-25T09:09:29.962501+0000","last_peered":"2024-04-25T09:09:29.962501+0000","last_clean":"2024-04-25T09:09:29.962501+0000","last_became_active":"2024-04-25T09:09:29.957294+0000","last_became_peered":"2024-04-25T09:09:29.957294+0000","last_unstale":"2024-04-25T09:09:29.962501+0000","last_undegraded":"2024-04-25T09:09:29.962501+0000","last_fullsized":"2024-04-25T09:09:29.962501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:12.455033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.950078+0000","last_change":"2024-04-25T09:09:28.960807+0000","last_active":"2024-04-25T09:09:29.950078+0000","last_peered":"2024-04-25T09:09:29.950078+0000","last_clean":"2024-04-25T09:09:29.950078+0000","last_became_active":"2024-04-25T09:09:28.960445+0000","last_became_peered":"2024-04-25T09:09:28.960445+0000","last_unstale":"2024-04-25T09:09:29.950078+0000","last_undegraded":"2024-04-25T09:09:29.950078+0000","last_fullsized":"2024-04-25T09:09:29.950078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.663175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963181+0000","last_change":"2024-04-25T09:09:29.957285+0000","last_active":"2024-04-25T09:09:29.963181+0000","last_peered":"2024-04-25T09:09:29.963181+0000","last_clean":"2024-04-25T09:09:29.963181+0000","last_became_active":"2024-04-25T09:09:29.957009+0000","last_became_peered":"2024-04-25T09:09:29.957009+0000","last_unstale":"2024-04-25T09:09:29.963181+0000","last_undegraded":"2024-04-25T09:09:29.963181+0000","last_fullsized":"2024-04-25T09:09:29.963181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:40.507663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.948355+0000","last_change":"2024-04-25T09:09:28.963736+0000","last_active":"2024-04-25T09:09:29.948355+0000","last_peered":"2024-04-25T09:09:29.948355+0000","last_clean":"2024-04-25T09:09:29.948355+0000","last_became_active":"2024-04-25T09:09:28.963389+0000","last_became_peered":"2024-04-25T09:09:28.963389+0000","last_unstale":"2024-04-25T09:09:29.948355+0000","last_undegraded":"2024-04-25T09:09:29.948355+0000","last_fullsized":"2024-04-25T09:09:29.948355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.266000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.965938+0000","last_change":"2024-04-25T09:09:29.962572+0000","last_active":"2024-04-25T09:09:29.965938+0000","last_peered":"2024-04-25T09:09:29.965938+0000","last_clean":"2024-04-25T09:09:29.965938+0000","last_became_active":"2024-04-25T09:09:29.961683+0000","last_became_peered":"2024-04-25T09:09:29.961683+0000","last_unstale":"2024-04-25T09:09:29.965938+0000","last_undegraded":"2024-04-25T09:09:29.965938+0000","last_fullsized":"2024-04-25T09:09:29.965938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:47.128840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.942232+0000","last_change":"2024-04-25T09:09:28.964595+0000","last_active":"2024-04-25T09:09:29.942232+0000","last_peered":"2024-04-25T09:09:29.942232+0000","last_clean":"2024-04-25T09:09:29.942232+0000","last_became_active":"2024-04-25T09:09:28.964275+0000","last_became_peered":"2024-04-25T09:09:28.964275+0000","last_unstale":"2024-04-25T09:09:29.942232+0000","last_undegraded":"2024-04-25T09:09:29.942232+0000","last_fullsized":"2024-04-25T09:09:29.942232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:11.857238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962261+0000","last_change":"2024-04-25T09:09:29.955991+0000","last_active":"2024-04-25T09:09:30.962261+0000","last_peered":"2024-04-25T09:09:30.962261+0000","last_clean":"2024-04-25T09:09:30.962261+0000","last_became_active":"2024-04-25T09:09:29.955331+0000","last_became_peered":"2024-04-25T09:09:29.955331+0000","last_unstale":"2024-04-25T09:09:30.962261+0000","last_undegraded":"2024-04-25T09:09:30.962261+0000","last_fullsized":"2024-04-25T09:09:30.962261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:06.126657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.932836+0000","last_change":"2024-04-25T09:09:28.954742+0000","last_active":"2024-04-25T09:09:29.932836+0000","last_peered":"2024-04-25T09:09:29.932836+0000","last_clean":"2024-04-25T09:09:29.932836+0000","last_became_active":"2024-04-25T09:09:28.954147+0000","last_became_peered":"2024-04-25T09:09:28.954147+0000","last_unstale":"2024-04-25T09:09:29.932836+0000","last_undegraded":"2024-04-25T09:09:29.932836+0000","last_fullsized":"2024-04-25T09:09:29.932836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:38.934301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961293+0000","last_change":"2024-04-25T09:09:29.955617+0000","last_active":"2024-04-25T09:09:29.961293+0000","last_peered":"2024-04-25T09:09:29.961293+0000","last_clean":"2024-04-25T09:09:29.961293+0000","last_became_active":"2024-04-25T09:09:29.955312+0000","last_became_peered":"2024-04-25T09:09:29.955312+0000","last_unstale":"2024-04-25T09:09:29.961293+0000","last_undegraded":"2024-04-25T09:09:29.961293+0000","last_fullsized":"2024-04-25T09:09:29.961293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:18.495961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975293+0000","last_change":"2024-04-25T09:09:28.963697+0000","last_active":"2024-04-25T09:09:30.975293+0000","last_peered":"2024-04-25T09:09:30.975293+0000","last_clean":"2024-04-25T09:09:30.975293+0000","last_became_active":"2024-04-25T09:09:28.963420+0000","last_became_peered":"2024-04-25T09:09:28.963420+0000","last_unstale":"2024-04-25T09:09:30.975293+0000","last_undegraded":"2024-04-25T09:09:30.975293+0000","last_fullsized":"2024-04-25T09:09:30.975293+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:48.261419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956761+0000","last_change":"2024-04-25T09:09:29.958448+0000","last_active":"2024-04-25T09:09:30.956761+0000","last_peered":"2024-04-25T09:09:30.956761+0000","last_clean":"2024-04-25T09:09:30.956761+0000","last_became_active":"2024-04-25T09:09:29.957840+0000","last_became_peered":"2024-04-25T09:09:29.957840+0000","last_unstale":"2024-04-25T09:09:30.956761+0000","last_undegraded":"2024-04-25T09:09:30.956761+0000","last_fullsized":"2024-04-25T09:09:30.956761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:24.090990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.978060+0000","last_change":"2024-04-25T09:09:28.961166+0000","last_active":"2024-04-25T09:09:30.978060+0000","last_peered":"2024-04-25T09:09:30.978060+0000","last_clean":"2024-04-25T09:09:30.978060+0000","last_became_active":"2024-04-25T09:09:28.960876+0000","last_became_peered":"2024-04-25T09:09:28.960876+0000","last_unstale":"2024-04-25T09:09:30.978060+0000","last_undegraded":"2024-04-25T09:09:30.978060+0000","last_fullsized":"2024-04-25T09:09:30.978060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:34.852622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.969939+0000","last_change":"2024-04-25T09:09:29.966079+0000","last_active":"2024-04-25T09:09:29.969939+0000","last_peered":"2024-04-25T09:09:29.969939+0000","last_clean":"2024-04-25T09:09:29.969939+0000","last_became_active":"2024-04-25T09:09:29.965819+0000","last_became_peered":"2024-04-25T09:09:29.965819+0000","last_unstale":"2024-04-25T09:09:29.969939+0000","last_undegraded":"2024-04-25T09:09:29.969939+0000","last_fullsized":"2024-04-25T09:09:29.969939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:47.266776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962674+0000","last_change":"2024-04-25T09:09:28.961674+0000","last_active":"2024-04-25T09:09:30.962674+0000","last_peered":"2024-04-25T09:09:30.962674+0000","last_clean":"2024-04-25T09:09:30.962674+0000","last_became_active":"2024-04-25T09:09:28.961315+0000","last_became_peered":"2024-04-25T09:09:28.961315+0000","last_unstale":"2024-04-25T09:09:30.962674+0000","last_undegraded":"2024-04-25T09:09:30.962674+0000","last_fullsized":"2024-04-25T09:09:30.962674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:48.060010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956554+0000","last_change":"2024-04-25T09:09:29.955260+0000","last_active":"2024-04-25T09:09:30.956554+0000","last_peered":"2024-04-25T09:09:30.956554+0000","last_clean":"2024-04-25T09:09:30.956554+0000","last_became_active":"2024-04-25T09:09:29.954958+0000","last_became_peered":"2024-04-25T09:09:29.954958+0000","last_unstale":"2024-04-25T09:09:30.956554+0000","last_undegraded":"2024-04-25T09:09:30.956554+0000","last_fullsized":"2024-04-25T09:09:30.956554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:19.566183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962397+0000","last_change":"2024-04-25T09:09:28.966553+0000","last_active":"2024-04-25T09:09:30.962397+0000","last_peered":"2024-04-25T09:09:30.962397+0000","last_clean":"2024-04-25T09:09:30.962397+0000","last_became_active":"2024-04-25T09:09:28.966079+0000","last_became_peered":"2024-04-25T09:09:28.966079+0000","last_unstale":"2024-04-25T09:09:30.962397+0000","last_undegraded":"2024-04-25T09:09:30.962397+0000","last_fullsized":"2024-04-25T09:09:30.962397+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:46.152212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.961441+0000","last_change":"2024-04-25T09:09:29.955384+0000","last_active":"2024-04-25T09:09:29.961441+0000","last_peered":"2024-04-25T09:09:29.961441+0000","last_clean":"2024-04-25T09:09:29.961441+0000","last_became_active":"2024-04-25T09:09:29.954792+0000","last_became_peered":"2024-04-25T09:09:29.954792+0000","last_unstale":"2024-04-25T09:09:29.961441+0000","last_undegraded":"2024-04-25T09:09:29.961441+0000","last_fullsized":"2024-04-25T09:09:29.961441+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:57.369951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.962096+0000","last_change":"2024-04-25T09:09:28.968371+0000","last_active":"2024-04-25T09:09:30.962096+0000","last_peered":"2024-04-25T09:09:30.962096+0000","last_clean":"2024-04-25T09:09:30.962096+0000","last_became_active":"2024-04-25T09:09:28.968100+0000","last_became_peered":"2024-04-25T09:09:28.968100+0000","last_unstale":"2024-04-25T09:09:30.962096+0000","last_undegraded":"2024-04-25T09:09:30.962096+0000","last_fullsized":"2024-04-25T09:09:30.962096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:17.841750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.975045+0000","last_change":"2024-04-25T09:09:29.957492+0000","last_active":"2024-04-25T09:09:30.975045+0000","last_peered":"2024-04-25T09:09:30.975045+0000","last_clean":"2024-04-25T09:09:30.975045+0000","last_became_active":"2024-04-25T09:09:29.957197+0000","last_became_peered":"2024-04-25T09:09:29.957197+0000","last_unstale":"2024-04-25T09:09:30.975045+0000","last_undegraded":"2024-04-25T09:09:30.975045+0000","last_fullsized":"2024-04-25T09:09:30.975045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:31.075691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.935906+0000","last_change":"2024-04-25T09:09:28.962797+0000","last_active":"2024-04-25T09:09:29.935906+0000","last_peered":"2024-04-25T09:09:29.935906+0000","last_clean":"2024-04-25T09:09:29.935906+0000","last_became_active":"2024-04-25T09:09:28.961552+0000","last_became_peered":"2024-04-25T09:09:28.961552+0000","last_unstale":"2024-04-25T09:09:29.935906+0000","last_undegraded":"2024-04-25T09:09:29.935906+0000","last_fullsized":"2024-04-25T09:09:29.935906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:34.989639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.963762+0000","last_change":"2024-04-25T09:09:29.956755+0000","last_active":"2024-04-25T09:09:29.963762+0000","last_peered":"2024-04-25T09:09:29.963762+0000","last_clean":"2024-04-25T09:09:29.963762+0000","last_became_active":"2024-04-25T09:09:29.956468+0000","last_became_peered":"2024-04-25T09:09:29.956468+0000","last_unstale":"2024-04-25T09:09:29.963762+0000","last_undegraded":"2024-04-25T09:09:29.963762+0000","last_fullsized":"2024-04-25T09:09:29.963762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:47.169527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.942466+0000","last_change":"2024-04-25T09:09:28.956325+0000","last_active":"2024-04-25T09:09:29.942466+0000","last_peered":"2024-04-25T09:09:29.942466+0000","last_clean":"2024-04-25T09:09:29.942466+0000","last_became_active":"2024-04-25T09:09:28.955665+0000","last_became_peered":"2024-04-25T09:09:28.955665+0000","last_unstale":"2024-04-25T09:09:29.942466+0000","last_undegraded":"2024-04-25T09:09:29.942466+0000","last_fullsized":"2024-04-25T09:09:29.942466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:28.159180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.970528+0000","last_change":"2024-04-25T09:09:29.966312+0000","last_active":"2024-04-25T09:09:29.970528+0000","last_peered":"2024-04-25T09:09:29.970528+0000","last_clean":"2024-04-25T09:09:29.970528+0000","last_became_active":"2024-04-25T09:09:29.965785+0000","last_became_peered":"2024-04-25T09:09:29.965785+0000","last_unstale":"2024-04-25T09:09:29.970528+0000","last_undegraded":"2024-04-25T09:09:29.970528+0000","last_fullsized":"2024-04-25T09:09:29.970528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:26.318267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.942900+0000","last_change":"2024-04-25T09:09:28.963696+0000","last_active":"2024-04-25T09:09:29.942900+0000","last_peered":"2024-04-25T09:09:29.942900+0000","last_clean":"2024-04-25T09:09:29.942900+0000","last_became_active":"2024-04-25T09:09:28.963373+0000","last_became_peered":"2024-04-25T09:09:28.963373+0000","last_unstale":"2024-04-25T09:09:29.942900+0000","last_undegraded":"2024-04-25T09:09:29.942900+0000","last_fullsized":"2024-04-25T09:09:29.942900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:59.855428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T09:09:29.959176+0000","last_change":"2024-04-25T09:09:29.955116+0000","last_active":"2024-04-25T09:09:29.959176+0000","last_peered":"2024-04-25T09:09:29.959176+0000","last_clean":"2024-04-25T09:09:29.959176+0000","last_became_active":"2024-04-25T09:09:29.954280+0000","last_became_peered":"2024-04-25T09:09:29.954280+0000","last_unstale":"2024-04-25T09:09:29.959176+0000","last_undegraded":"2024-04-25T09:09:29.959176+0000","last_fullsized":"2024-04-25T09:09:29.959176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:28.912268+0000","last_clean_scrub_stamp":"2024-04-25T09:09:28.912268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:43.759395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.956467+0000","last_change":"2024-04-25T09:09:28.963883+0000","last_active":"2024-04-25T09:09:30.956467+0000","last_peered":"2024-04-25T09:09:30.956467+0000","last_clean":"2024-04-25T09:09:30.956467+0000","last_became_active":"2024-04-25T09:09:28.963611+0000","last_became_peered":"2024-04-25T09:09:28.963611+0000","last_unstale":"2024-04-25T09:09:30.956467+0000","last_undegraded":"2024-04-25T09:09:30.956467+0000","last_fullsized":"2024-04-25T09:09:30.956467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:09:27.894953+0000","last_clean_scrub_stamp":"2024-04-25T09:09:27.894953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:56.164727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":276,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T09:09:30.955753+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:09:30.955753+0000","last_peered":"2024-04-25T09:09:30.955753+0000","last_clean":"2024-04-25T09:09:30.955753+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:09:30.955753+0000","last_undegraded":"2024-04-25T09:09:30.955753+0000","last_fullsized":"2024-04-25T09:09:30.955753+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335552,"statfs":{"total":96636764160,"available":96599605248,"internally_reserved":0,"allocated":962560,"data_stored":410694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40444,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331396,"statfs":{"total":96636764160,"available":96595349504,"internally_reserved":0,"allocated":966656,"data_stored":410728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41020,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330820,"statfs":{"total":96636764160,"available":96594759680,"internally_reserved":0,"allocated":1548288,"data_stored":999540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40448,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":962560,"data_stored":409776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58496,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313344,"statfs":{"total":96636764160,"available":96576864256,"internally_reserved":0,"allocated":962560,"data_stored":409208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58480,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313360,"statfs":{"total":96636764160,"available":96576880640,"internally_reserved":0,"allocated":954368,"data_stored":409664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41008,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330832,"statfs":{"total":96636764160,"available":96594771968,"internally_reserved":0,"allocated":1544192,"data_stored":999450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36280,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335560,"statfs":{"total":96636764160,"available":96599613440,"internally_reserved":0,"allocated":954368,"data_stored":409152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:09:34.907 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:09:34.907 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:09:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.174+0000 7f2e19c00640 1 Processor -- start 2024-04-25T09:09:35.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.174+0000 7f2e19c00640 1 -- start start 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.175+0000 7f2e19c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e1415bdf0 0x7f2e140720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.175+0000 7f2e19c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2e140cebc0 0x7f2e1406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.175+0000 7f2e19c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e1406c7e0 0x7f2e140cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.175+0000 7f2e19c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1415c1d0 con 0x7f2e1415bdf0 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.175+0000 7f2e19c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e140ae8c0 con 0x7f2e140cebc0 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.175+0000 7f2e19c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1406d070 con 0x7f2e1406c7e0 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.176+0000 7f2e12a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e1415bdf0 0x7f2e140720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.176+0000 7f2e12a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e1415bdf0 0x7f2e140720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36894/0 (socket says 172.21.15.62:36894) 2024-04-25T09:09:35.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.176+0000 7f2e12a00640 1 -- 172.21.15.62:0/702266446 learned_addr learned my addr 172.21.15.62:0/702266446 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:35.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.176+0000 7f2e12a00640 1 -- 172.21.15.62:0/702266446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e1406c7e0 msgr2=0x7f2e140cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:35.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.176+0000 7f2e12a00640 1 --2- 172.21.15.62:0/702266446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e1406c7e0 0x7f2e140cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.176+0000 7f2e12a00640 1 -- 172.21.15.62:0/702266446 >> v2:172.21.15.121:3300/0 conn(0x7f2e140cebc0 msgr2=0x7f2e1406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:35.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.176+0000 7f2e12a00640 1 --2- 172.21.15.62:0/702266446 >> v2:172.21.15.121:3300/0 conn(0x7f2e140cebc0 0x7f2e1406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.176+0000 7f2e12a00640 1 -- 172.21.15.62:0/702266446 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1406d350 con 0x7f2e1415bdf0 2024-04-25T09:09:35.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.177+0000 7f2e12a00640 1 --2- 172.21.15.62:0/702266446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140720c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2e14068870 tx=0x7f2e0c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9201d15222a5d903 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:35.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.178+0000 7f2e11600640 1 -- 172.21.15.62:0/702266446 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c036070 con 0x7f2e1415bdf0 2024-04-25T09:09:35.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.178+0000 7f2e11600640 1 -- 172.21.15.62:0/702266446 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c031600 con 0x7f2e1415bdf0 2024-04-25T09:09:35.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.178+0000 7f2e11600640 1 -- 172.21.15.62:0/702266446 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0319d0 con 0x7f2e1415bdf0 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 -- 172.21.15.62:0/702266446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 msgr2=0x7f2e140720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 --2- 172.21.15.62:0/702266446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140720c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2e14068870 tx=0x7f2e0c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 -- 172.21.15.62:0/702266446 shutdown_connections 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 --2- 172.21.15.62:0/702266446 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e1406c7e0 0x7f2e140cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 --2- 172.21.15.62:0/702266446 >> v2:172.21.15.121:3300/0 conn(0x7f2e140cebc0 0x7f2e1406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 --2- 172.21.15.62:0/702266446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140720c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 -- 172.21.15.62:0/702266446 >> 172.21.15.62:0/702266446 conn(0x7f2e140b1410 msgr2=0x7f2e140b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 -- 172.21.15.62:0/702266446 shutdown_connections 2024-04-25T09:09:35.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.179+0000 7f2e19c00640 1 -- 172.21.15.62:0/702266446 wait complete. 2024-04-25T09:09:35.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.181+0000 7f2e19c00640 1 Processor -- start 2024-04-25T09:09:35.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.181+0000 7f2e19c00640 1 -- start start 2024-04-25T09:09:35.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e19c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e1406c7e0 0x7f2e141534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e19c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e140cebc0 0x7f2e140afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e19c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e19c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1406cc70 con 0x7f2e1415bdf0 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e19c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140aedf0 con 0x7f2e1406c7e0 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e19c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1406cdf0 con 0x7f2e140cebc0 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e13400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e13400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36902/0 (socket says 172.21.15.62:36902) 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e13400640 1 -- 172.21.15.62:0/1471842909 learned_addr learned my addr 172.21.15.62:0/1471842909 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e12a00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e1406c7e0 0x7f2e141534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.182+0000 7f2e12000640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e140cebc0 0x7f2e140afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e13400640 1 -- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e140cebc0 msgr2=0x7f2e140afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e13400640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e140cebc0 0x7f2e140afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e13400640 1 -- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e1406c7e0 msgr2=0x7f2e141534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e13400640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e1406c7e0 0x7f2e141534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e13400640 1 -- 172.21.15.62:0/1471842909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140a94d0 con 0x7f2e1415bdf0 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e13400640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140b0010 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009fd0 tx=0x7f2e0400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e0aa00640 1 -- 172.21.15.62:0/1471842909 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04011830 con 0x7f2e1415bdf0 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e0aa00640 1 -- 172.21.15.62:0/1471842909 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e04016070 con 0x7f2e1415bdf0 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e0aa00640 1 -- 172.21.15.62:0/1471842909 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04011bf0 con 0x7f2e1415bdf0 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e140a97b0 con 0x7f2e1415bdf0 2024-04-25T09:09:35.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.183+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e140a9d70 con 0x7f2e1415bdf0 2024-04-25T09:09:35.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.184+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1404ef20 con 0x7f2e1415bdf0 2024-04-25T09:09:35.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.186+0000 7f2e0aa00640 1 -- 172.21.15.62:0/1471842909 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e0401b460 con 0x7f2e1415bdf0 2024-04-25T09:09:35.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.186+0000 7f2e0aa00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2df006b4d0 0x7f2df006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.187+0000 7f2e0aa00640 1 -- 172.21.15.62:0/1471842909 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e04089fb0 con 0x7f2e1415bdf0 2024-04-25T09:09:35.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.187+0000 7f2e12a00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2df006b4d0 0x7f2df006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.188+0000 7f2e12a00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2df006b4d0 0x7f2df006d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c033000 tx=0x7f2e0c009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:35.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.191+0000 7f2e0aa00640 1 -- 172.21.15.62:0/1471842909 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e04058e20 con 0x7f2e1415bdf0 2024-04-25T09:09:35.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.460+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2e14071c90 con 0x7f2e1415bdf0 2024-04-25T09:09:35.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.465+0000 7f2e0aa00640 1 -- 172.21.15.62:0/1471842909 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2e04058720 con 0x7f2e1415bdf0 2024-04-25T09:09:35.453 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:09:35.453 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:09:35.453 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:09:35.453 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:09:35.453 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:09:35.453 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:09:35.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.468+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2df006b4d0 msgr2=0x7f2df006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.468+0000 7f2e19c00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2df006b4d0 0x7f2df006d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c033000 tx=0x7f2e0c009000 comp rx=0 tx=0).stop 2024-04-25T09:09:35.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.468+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 msgr2=0x7f2e140b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.468+0000 7f2e19c00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140b0010 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009fd0 tx=0x7f2e0400b8e0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.469+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 shutdown_connections 2024-04-25T09:09:35.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.469+0000 7f2e19c00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2df006b4d0 0x7f2df006d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.469+0000 7f2e19c00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e1415bdf0 0x7f2e140b0010 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.469+0000 7f2e19c00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e140cebc0 0x7f2e140afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.469+0000 7f2e19c00640 1 --2- 172.21.15.62:0/1471842909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e1406c7e0 0x7f2e141534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.469+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 >> 172.21.15.62:0/1471842909 conn(0x7f2e140b1410 msgr2=0x7f2e1415a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:35.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.469+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 shutdown_connections 2024-04-25T09:09:35.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.469+0000 7f2e19c00640 1 -- 172.21.15.62:0/1471842909 wait complete. 2024-04-25T09:09:35.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:09:35.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.685+0000 7fa91a800640 1 Processor -- start 2024-04-25T09:09:35.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.685+0000 7fa91a800640 1 -- start start 2024-04-25T09:09:35.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.687+0000 7fa91a800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 0x7fa9140b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.687+0000 7fa91a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa9140b1050 0x7fa91406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.687+0000 7fa91a800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91415b220 con 0x7fa91415acf0 2024-04-25T09:09:35.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.687+0000 7fa91a800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91415b500 con 0x7fa9140b1050 2024-04-25T09:09:35.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.687+0000 7fa91a800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91415b7e0 con 0x7fa91413d970 2024-04-25T09:09:35.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.688+0000 7fa918a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa9140b1050 0x7fa91406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.688+0000 7fa918a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa9140b1050 0x7fa91406d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36904/0 (socket says 172.21.15.62:36904) 2024-04-25T09:09:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.688+0000 7fa918a00640 1 -- 172.21.15.62:0/1559183373 learned_addr learned my addr 172.21.15.62:0/1559183373 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.688+0000 7fa919400640 1 --2- 172.21.15.62:0/1559183373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 0x7fa9140b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.688+0000 7fa918a00640 1 -- 172.21.15.62:0/1559183373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 msgr2=0x7fa9140b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.688+0000 7fa918a00640 1 --2- 172.21.15.62:0/1559183373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 0x7fa9140b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.688+0000 7fa918a00640 1 -- 172.21.15.62:0/1559183373 >> v1:172.21.15.121:6789/0 conn(0x7fa91415acf0 legacy=0x7fa91413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:35.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.688+0000 7fa918a00640 1 -- 172.21.15.62:0/1559183373 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91415bac0 con 0x7fa9140b1050 2024-04-25T09:09:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.689+0000 7fa918a00640 1 --2- 172.21.15.62:0/1559183373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa91406d1f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa900009870 tx=0x7fa90002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00c83b92b086fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.690+0000 7fa913e00640 1 -- 172.21.15.62:0/1559183373 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900037070 con 0x7fa9140b1050 2024-04-25T09:09:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.690+0000 7fa913e00640 1 -- 172.21.15.62:0/1559183373 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa900032440 con 0x7fa9140b1050 2024-04-25T09:09:35.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.690+0000 7fa913e00640 1 -- 172.21.15.62:0/1559183373 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900032810 con 0x7fa9140b1050 2024-04-25T09:09:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.690+0000 7fa91a800640 1 -- 172.21.15.62:0/1559183373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 msgr2=0x7fa91406d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.690+0000 7fa91a800640 1 --2- 172.21.15.62:0/1559183373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa91406d1f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa900009870 tx=0x7fa90002a210 comp rx=0 tx=0).stop 2024-04-25T09:09:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.691+0000 7fa91a800640 1 -- 172.21.15.62:0/1559183373 shutdown_connections 2024-04-25T09:09:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.691+0000 7fa91a800640 1 --2- 172.21.15.62:0/1559183373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa91406d1f0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.691+0000 7fa91a800640 1 --2- 172.21.15.62:0/1559183373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 0x7fa9140b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.691+0000 7fa91a800640 1 -- 172.21.15.62:0/1559183373 >> 172.21.15.62:0/1559183373 conn(0x7fa9140b14d0 msgr2=0x7fa9140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.691+0000 7fa91a800640 1 -- 172.21.15.62:0/1559183373 shutdown_connections 2024-04-25T09:09:35.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.691+0000 7fa91a800640 1 -- 172.21.15.62:0/1559183373 wait complete. 2024-04-25T09:09:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.692+0000 7fa91a800640 1 Processor -- start 2024-04-25T09:09:35.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.692+0000 7fa91a800640 1 -- start start 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.693+0000 7fa91a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa914153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.693+0000 7fa91a800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 0x7fa914159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.693+0000 7fa91a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91415acf0 0x7fa914159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.693+0000 7fa91a800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91406d730 con 0x7fa9140b1050 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.693+0000 7fa91a800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91413f470 con 0x7fa91415acf0 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.693+0000 7fa91a800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91406d8b0 con 0x7fa91413d970 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.693+0000 7fa919400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa914153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.694+0000 7fa919400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa914153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36906/0 (socket says 172.21.15.62:36906) 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.694+0000 7fa919400640 1 -- 172.21.15.62:0/2380539904 learned_addr learned my addr 172.21.15.62:0/2380539904 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.694+0000 7fa919400640 1 -- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 msgr2=0x7fa914159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.694+0000 7fa919400640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 0x7fa914159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.694+0000 7fa919400640 1 -- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91415acf0 msgr2=0x7fa914159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.694+0000 7fa919400640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91415acf0 0x7fa914159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.694+0000 7fa919400640 1 -- 172.21.15.62:0/2380539904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140b3900 con 0x7fa9140b1050 2024-04-25T09:09:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.694+0000 7fa919400640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa914153b10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa90800d800 tx=0x7fa90800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.695+0000 7fa911600640 1 -- 172.21.15.62:0/2380539904 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800b6c0 con 0x7fa9140b1050 2024-04-25T09:09:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.695+0000 7fa911600640 1 -- 172.21.15.62:0/2380539904 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908019070 con 0x7fa9140b1050 2024-04-25T09:09:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.695+0000 7fa911600640 1 -- 172.21.15.62:0/2380539904 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800ba80 con 0x7fa9140b1050 2024-04-25T09:09:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.695+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9140b3be0 con 0x7fa9140b1050 2024-04-25T09:09:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.695+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9140b41a0 con 0x7fa9140b1050 2024-04-25T09:09:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.696+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91404ef20 con 0x7fa9140b1050 2024-04-25T09:09:35.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.697+0000 7fa911600640 1 -- 172.21.15.62:0/2380539904 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa90800bc20 con 0x7fa9140b1050 2024-04-25T09:09:35.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.698+0000 7fa911600640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa8f406b4d0 0x7fa8f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:35.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.698+0000 7fa911600640 1 -- 172.21.15.62:0/2380539904 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa90808a780 con 0x7fa9140b1050 2024-04-25T09:09:35.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.698+0000 7fa918a00640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa8f406b4d0 0x7fa8f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:35.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.699+0000 7fa918a00640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa8f406b4d0 0x7fa8f406d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa900034000 tx=0x7fa90002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:35.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.702+0000 7fa911600640 1 -- 172.21.15.62:0/2380539904 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa908059560 con 0x7fa9140b1050 2024-04-25T09:09:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.932+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa914154ee0 con 0x7fa9140b1050 2024-04-25T09:09:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.933+0000 7fa911600640 1 -- 172.21.15.62:0/2380539904 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fa908058e60 con 0x7fa9140b1050 2024-04-25T09:09:35.920 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:09:35.920 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":54,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5336,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2992229593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2992229593},{"type":"v1","addr":"172.21.15.62:6835","nonce":2992229593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5342,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/518754003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":518754003},{"type":"v1","addr":"172.21.15.121:6837","nonce":518754003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/6918584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":6918584},{"type":"v1","addr":"172.21.15.121:6836","nonce":6918584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:09:29.410378+0000","modified":"2024-04-25T09:09:32.452815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2258257596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2258257596},{"type":"v1","addr":"172.21.15.62:6834","nonce":2258257596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:09:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 54 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa8f406b4d0 msgr2=0x7fa8f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa8f406b4d0 0x7fa8f406d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa900034000 tx=0x7fa90002c000 comp rx=0 tx=0).stop 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 msgr2=0x7fa914153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa914153b10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa90800d800 tx=0x7fa90800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 shutdown_connections 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa8f406b4d0 0x7fa8f406d990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa91415acf0 0x7fa914159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa91413d970 0x7fa914159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 --2- 172.21.15.62:0/2380539904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9140b1050 0x7fa914153b10 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.935+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 >> 172.21.15.62:0/2380539904 conn(0x7fa9140b14d0 msgr2=0x7fa9140b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.936+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 shutdown_connections 2024-04-25T09:09:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:35.936+0000 7fa91a800640 1 -- 172.21.15.62:0/2380539904 wait complete. 2024-04-25T09:09:36.344 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:09:29.410378+0000', 'modified': '2024-04-25T09:09:32.452815+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6894': {'gid': 6894, 'name': 'a', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6834/2258257596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 2258257596}, {'type': 'v1', 'addr': '172.21.15.62:6834', 'nonce': 2258257596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:09:36.344 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:09:36.344 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:09:36.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.579+0000 7efeaf200640 1 Processor -- start 2024-04-25T09:09:36.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.579+0000 7efeaf200640 1 -- start start 2024-04-25T09:09:36.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.581+0000 7efeaf200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efeb00b5c30 0x7efeb00b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.581+0000 7efeaf200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efeb00d0fb0 0x7efeb0161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.581+0000 7efeaf200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb00cea70 con 0x7efeb0161230 2024-04-25T09:09:36.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.581+0000 7efeaf200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb00d2040 con 0x7efeb00b5c30 2024-04-25T09:09:36.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.581+0000 7efeaf200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb00ced50 con 0x7efeb00d0fb0 2024-04-25T09:09:36.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.581+0000 7efea7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efeb00b5c30 0x7efeb00b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.582+0000 7efea7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efeb00b5c30 0x7efeb00b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36912/0 (socket says 172.21.15.62:36912) 2024-04-25T09:09:36.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.582+0000 7efea7e00640 1 -- 172.21.15.62:0/1010774744 learned_addr learned my addr 172.21.15.62:0/1010774744 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:36.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.582+0000 7efeae800640 1 --2- 172.21.15.62:0/1010774744 >> v2:172.21.15.121:3300/0 conn(0x7efeb00d0fb0 0x7efeb0161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.582+0000 7efea7e00640 1 -- 172.21.15.62:0/1010774744 >> v2:172.21.15.121:3300/0 conn(0x7efeb00d0fb0 msgr2=0x7efeb0161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.582+0000 7efea7e00640 1 --2- 172.21.15.62:0/1010774744 >> v2:172.21.15.121:3300/0 conn(0x7efeb00d0fb0 0x7efeb0161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.582+0000 7efea7e00640 1 -- 172.21.15.62:0/1010774744 >> v1:172.21.15.121:6789/0 conn(0x7efeb0161230 legacy=0x7efeb00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.582+0000 7efea7e00640 1 -- 172.21.15.62:0/1010774744 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00cf030 con 0x7efeb00b5c30 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.583+0000 7efea7e00640 1 --2- 172.21.15.62:0/1010774744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb00b6c40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7efeb008c590 tx=0x7efe9802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f8f9c51fc89edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efead400640 1 -- 172.21.15.62:0/1010774744 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98037070 con 0x7efeb00b5c30 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efead400640 1 -- 172.21.15.62:0/1010774744 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe98032440 con 0x7efeb00b5c30 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efead400640 1 -- 172.21.15.62:0/1010774744 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98032810 con 0x7efeb00b5c30 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efeaf200640 1 -- 172.21.15.62:0/1010774744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 msgr2=0x7efeb00b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efeaf200640 1 --2- 172.21.15.62:0/1010774744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb00b6c40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7efeb008c590 tx=0x7efe9802a230 comp rx=0 tx=0).stop 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efeaf200640 1 -- 172.21.15.62:0/1010774744 shutdown_connections 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efeaf200640 1 --2- 172.21.15.62:0/1010774744 >> v2:172.21.15.121:3300/0 conn(0x7efeb00d0fb0 0x7efeb0161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efeaf200640 1 --2- 172.21.15.62:0/1010774744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb00b6c40 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.584+0000 7efeaf200640 1 -- 172.21.15.62:0/1010774744 >> 172.21.15.62:0/1010774744 conn(0x7efeb0066d10 msgr2=0x7efeb0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:36.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.585+0000 7efeaf200640 1 -- 172.21.15.62:0/1010774744 shutdown_connections 2024-04-25T09:09:36.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.585+0000 7efeaf200640 1 -- 172.21.15.62:0/1010774744 wait complete. 2024-04-25T09:09:36.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.586+0000 7efeaf200640 1 Processor -- start 2024-04-25T09:09:36.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.586+0000 7efeaf200640 1 -- start start 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.587+0000 7efeaf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb01f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.587+0000 7efeaf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d0fb0 0x7efeb00c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.587+0000 7efeaf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb0161230 0x7efeb01492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.587+0000 7efeaf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0161610 con 0x7efeb00b5c30 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.587+0000 7efeaf200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00ca8d0 con 0x7efeb00d0fb0 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.587+0000 7efeaf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0161790 con 0x7efeb0161230 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb01f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb01f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36916/0 (socket says 172.21.15.62:36916) 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 -- 172.21.15.62:0/657719125 learned_addr learned my addr 172.21.15.62:0/657719125 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:36.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efea7e00640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d0fb0 0x7efeb00c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 -- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb0161230 msgr2=0x7efeb01492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb0161230 0x7efeb01492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 -- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d0fb0 msgr2=0x7efeb00c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d0fb0 0x7efeb00c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 -- 172.21.15.62:0/657719125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb015aad0 con 0x7efeb00b5c30 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.588+0000 7efeade00640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb01f6c20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7efea000b420 tx=0x7efea000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.589+0000 7efea6000640 1 -- 172.21.15.62:0/657719125 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0011830 con 0x7efeb00b5c30 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.589+0000 7efea6000640 1 -- 172.21.15.62:0/657719125 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0016070 con 0x7efeb00b5c30 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.589+0000 7efea6000640 1 -- 172.21.15.62:0/657719125 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0011c10 con 0x7efeb00b5c30 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.589+0000 7efeaf200640 1 -- 172.21.15.62:0/657719125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb015adb0 con 0x7efeb00b5c30 2024-04-25T09:09:36.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.589+0000 7efeaf200640 1 -- 172.21.15.62:0/657719125 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb00c91c0 con 0x7efeb00b5c30 2024-04-25T09:09:36.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.591+0000 7efea6000640 1 -- 172.21.15.62:0/657719125 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efea001b460 con 0x7efeb00b5c30 2024-04-25T09:09:36.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.592+0000 7efea6000640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe8c06b4d0 0x7efe8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.592+0000 7efea6000640 1 -- 172.21.15.62:0/657719125 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efea0089ca0 con 0x7efeb00b5c30 2024-04-25T09:09:36.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.593+0000 7efeb7c2e740 1 Processor -- start 2024-04-25T09:09:36.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.593+0000 7efea7e00640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe8c06b4d0 0x7efe8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.593+0000 7efeb7c2e740 1 -- start start 2024-04-25T09:09:36.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddb4b30 0x55e4ddddc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4ddddceb0 0x55e4dddb4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddfd7a0 0x55e4dddb32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ddd18620 con 0x55e4dddb4b30 2024-04-25T09:09:36.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ddcc1ca0 con 0x55e4ddddceb0 2024-04-25T09:09:36.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4ddd146c0 con 0x55e4dddfd7a0 2024-04-25T09:09:36.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efea7e00640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe8c06b4d0 0x7efe8c06d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efe98034000 tx=0x7efe9802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddb4b30 0x55e4ddddc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddb4b30 0x55e4ddddc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36922/0 (socket says 172.21.15.62:36922) 2024-04-25T09:09:36.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.594+0000 7efeade00640 1 -- 172.21.15.62:0/1664143114 learned_addr learned my addr 172.21.15.62:0/1664143114 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efeae800640 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddfd7a0 0x55e4dddb32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efea7e00640 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4ddddceb0 0x55e4dddb4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efeade00640 1 -- 172.21.15.62:0/1664143114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddfd7a0 msgr2=0x55e4dddb32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efeade00640 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddfd7a0 0x55e4dddb32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efeade00640 1 -- 172.21.15.62:0/1664143114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4ddddceb0 msgr2=0x55e4dddb4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efeade00640 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4ddddceb0 0x55e4dddb4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efeade00640 1 -- 172.21.15.62:0/1664143114 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ddcc8f40 con 0x55e4dddb4b30 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efeade00640 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddb4b30 0x55e4ddddc970 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7efea0060690 tx=0x7efea0057f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26155013ea5cfa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efe9ea00640 1 -- 172.21.15.62:0/1664143114 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00567b0 con 0x55e4dddb4b30 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efe9ea00640 1 -- 172.21.15.62:0/1664143114 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0054b20 con 0x55e4dddb4b30 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.595+0000 7efe9ea00640 1 -- 172.21.15.62:0/1664143114 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea005c370 con 0x55e4dddb4b30 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1664143114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddb4b30 msgr2=0x55e4ddddc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddb4b30 0x55e4ddddc970 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7efea0060690 tx=0x7efea0057f40 comp rx=0 tx=0).stop 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1664143114 shutdown_connections 2024-04-25T09:09:36.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddfd7a0 0x55e4dddb32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4ddddceb0 0x55e4dddb4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1664143114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddb4b30 0x55e4ddddc970 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1664143114 >> 172.21.15.62:0/1664143114 conn(0x55e4dddb1d50 msgr2=0x55e4dddfa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:36.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1664143114 shutdown_connections 2024-04-25T09:09:36.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.596+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1664143114 wait complete. 2024-04-25T09:09:36.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.597+0000 7efeb7c2e740 1 -- start start 2024-04-25T09:09:36.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.597+0000 7efeb7c2e740 1 Processor -- start 2024-04-25T09:09:36.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.598+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4ddddceb0 0x55e4dddc9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.598+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4ddde1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.598+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddd9c20 0x55e4dddc6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.598+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ddddc4e0 con 0x55e4dddd9c20 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.598+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ddcb71f0 con 0x55e4dddfd7a0 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.598+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4ddddbed0 con 0x55e4ddddceb0 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.598+0000 7efeb7c2e740 10 client.0 _ll_register_callbacks cb 0x55e4ddcd4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddd9c20 0x55e4dddc6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddd9c20 0x55e4dddc6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36932/0 (socket says 172.21.15.62:36932) 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 -- 172.21.15.62:0/1166896780 learned_addr learned my addr 172.21.15.62:0/1166896780 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efea7e00640 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4ddde1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeade00640 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4ddddceb0 0x55e4dddc9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 -- 172.21.15.62:0/1166896780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4ddddceb0 msgr2=0x55e4dddc9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4ddddceb0 0x55e4dddc9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 -- 172.21.15.62:0/1166896780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 msgr2=0x55e4ddde1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4ddde1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 -- 172.21.15.62:0/1166896780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4ddcd6c50 con 0x55e4dddd9c20 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.599+0000 7efeae800640 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddd9c20 0x55e4dddc6800 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7efea8007a30 tx=0x7efea8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb874db0f1e1dc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.600+0000 7efe93400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.600+0000 7efe93400640 1 -- 172.21.15.62:0/1166896780 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8018020 con 0x55e4dddd9c20 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.600+0000 7efe93400640 1 -- 172.21.15.62:0/1166896780 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efea800fcc0 con 0x55e4dddd9c20 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.600+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1166896780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea0054b20 con 0x55e4dddd9c20 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.600+0000 7efe93400640 1 -- 172.21.15.62:0/1166896780 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea8022050 con 0x55e4dddd9c20 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.600+0000 7efeb7c2e740 10 client.7038 fetch_fsmap learned FSMap version 54 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.600+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1166896780 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55e4dda2d2d0 con 0x55e4dddd9c20 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.600+0000 7efe93400640 1 -- 172.21.15.62:0/1166896780 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0054b20 con 0x55e4dddd9c20 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efe93400640 1 -- 172.21.15.62:0/1166896780 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efea8025020 con 0x55e4dddd9c20 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efeb7c2e740 10 client.7038 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efeb7c2e740 10 client.7038 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:09:36.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efeb7c2e740 20 client.7038 populate_metadata read hostname 'smithi062' 2024-04-25T09:09:36.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55e4dde0e120 0x55e4dde0e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efeb7c2e740 4 client.7038 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:09:36.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1166896780 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4ddd25b60 con 0x55e4dde0e120 2024-04-25T09:09:36.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efeade00640 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55e4dde0e120 0x55e4dde0e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efeade00640 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55e4dde0e120 0x55e4dde0e520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c54c921b36ef8f4 server_cookie=9fb25aef6e194715 in_seq=0 out_seq=0 2024-04-25T09:09:36.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.601+0000 7efe93400640 10 client.7038 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:09:36.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.602+0000 7efe93400640 1 -- 172.21.15.62:0/1166896780 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e4ddd25b60 con 0x55e4dde0e120 2024-04-25T09:09:36.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.602+0000 7efe93400640 10 client.7038 handle_command_reply: tid=0 2024-04-25T09:09:36.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.602+0000 7efeb7c2e740 1 client.7038 shutdown 2024-04-25T09:09:36.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.602+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1166896780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddd9c20 msgr2=0x55e4dddc6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.602+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddd9c20 0x55e4dddc6800 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7efea8007a30 tx=0x7efea8007ef0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1166896780 shutdown_connections 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55e4dde0e120 0x55e4dde0e520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4dddd9c20 0x55e4dddc6800 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4ddde1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efe93400640 0 client.7038 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1166896780 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4ddddceb0 0x55e4dddc9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1166896780 >> 172.21.15.62:0/1166896780 conn(0x55e4dddb4b30 msgr2=0x55e4dddfdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1166896780 shutdown_connections 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1166896780 wait complete. 2024-04-25T09:09:36.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.603+0000 7efeb7c2e740 20 client.7038 trim_cache size 0 max 16384 2024-04-25T09:09:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.620+0000 7efeb7c2e740 1 Processor -- start 2024-04-25T09:09:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.621+0000 7efeb7c2e740 1 -- start start 2024-04-25T09:09:36.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.621+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 0x55e4dde14720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.621+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4ddde0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.621+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddd76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb015d8b0 con 0x55e4ddddceb0 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb015ccb0 con 0x55e4dddfd7a0 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb015d2b0 con 0x55e4dddd9c20 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efea7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4ddde0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efea7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4ddde0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36942/0 (socket says 172.21.15.62:36942) 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efea7e00640 1 -- 172.21.15.62:0/1379157367 learned_addr learned my addr 172.21.15.62:0/1379157367 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efeae800640 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddd76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efeade00640 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 0x55e4dde14720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efea7e00640 1 -- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 msgr2=0x55e4dde14720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efea7e00640 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 0x55e4dde14720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efea7e00640 1 -- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 msgr2=0x55e4dddd76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efea7e00640 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddd76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efea7e00640 1 -- 172.21.15.62:0/1379157367 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ddaf76b0 con 0x55e4ddddceb0 2024-04-25T09:09:36.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.622+0000 7efeae800640 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddd76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:36.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.623+0000 7efea7e00640 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4ddde0150 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7efe98009960 tx=0x7efe9802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f518ba9a80c7a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.623+0000 7efe9e000640 1 -- 172.21.15.62:0/1379157367 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98037040 con 0x55e4ddddceb0 2024-04-25T09:09:36.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.623+0000 7efe9e000640 1 -- 172.21.15.62:0/1379157367 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9804c070 con 0x55e4ddddceb0 2024-04-25T09:09:36.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.623+0000 7efe9e000640 1 -- 172.21.15.62:0/1379157367 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9803c920 con 0x55e4ddddceb0 2024-04-25T09:09:36.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1379157367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 msgr2=0x55e4ddde0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4ddde0150 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7efe98009960 tx=0x7efe9802c000 comp rx=0 tx=0).stop 2024-04-25T09:09:36.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1379157367 shutdown_connections 2024-04-25T09:09:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4ddde0150 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddd76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/1379157367 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 0x55e4dde14720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1379157367 >> 172.21.15.62:0/1379157367 conn(0x55e4dddb4b30 msgr2=0x55e4ddde1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1379157367 shutdown_connections 2024-04-25T09:09:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.624+0000 7efeb7c2e740 1 -- 172.21.15.62:0/1379157367 wait complete. 2024-04-25T09:09:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.625+0000 7efeb7c2e740 1 -- start start 2024-04-25T09:09:36.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.625+0000 7efeb7c2e740 1 Processor -- start 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 0x55e4dde09100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4dde09640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efeb7c2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddb3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ddddb410 con 0x55e4ddddceb0 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4ddddac10 con 0x55e4dddfd7a0 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efeb7c2e740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4dddc7750 con 0x55e4dddd9c20 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efeb7c2e740 10 client.0 _ll_register_callbacks cb 0x7efea80097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efea7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4dde09640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efea7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4dde09640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36954/0 (socket says 172.21.15.62:36954) 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.626+0000 7efea7e00640 1 -- 172.21.15.62:0/822212918 learned_addr learned my addr 172.21.15.62:0/822212918 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efea7e00640 1 -- 172.21.15.62:0/822212918 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 msgr2=0x55e4dde09100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efea7e00640 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 0x55e4dde09100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efeae800640 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddb3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efea7e00640 1 -- 172.21.15.62:0/822212918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 msgr2=0x55e4dddb3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efea7e00640 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddb3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efea7e00640 1 -- 172.21.15.62:0/822212918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4dddfd5d0 con 0x55e4ddddceb0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efea7e00640 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4dde09640 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efe9802a820 tx=0x7efe98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a15f47c0675e551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efe93400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efe93400640 1 -- 172.21.15.62:0/822212918 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98037040 con 0x55e4ddddceb0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efe93400640 1 -- 172.21.15.62:0/822212918 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe98003aa0 con 0x55e4ddddceb0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.627+0000 7efeb7c2e740 1 -- 172.21.15.62:0/822212918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe9804c070 con 0x55e4ddddceb0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efe93400640 1 -- 172.21.15.62:0/822212918 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9805f050 con 0x55e4ddddceb0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeb7c2e740 10 client.7044 fetch_fsmap learned FSMap version 54 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeb7c2e740 1 -- 172.21.15.62:0/822212918 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55e4dddc8c60 con 0x55e4ddddceb0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efe93400640 1 -- 172.21.15.62:0/822212918 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9804c070 con 0x55e4ddddceb0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efe93400640 1 -- 172.21.15.62:0/822212918 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe98062020 con 0x55e4ddddceb0 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeb7c2e740 10 client.7044 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeb7c2e740 10 client.7044 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:09:36.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeb7c2e740 20 client.7044 populate_metadata read hostname 'smithi062' 2024-04-25T09:09:36.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55e4dddb3ab0 0x55e4dddc5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeb7c2e740 4 client.7044 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:09:36.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeb7c2e740 1 -- 172.21.15.62:0/822212918 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e4dddb11a0 con 0x55e4dddb3ab0 2024-04-25T09:09:36.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.628+0000 7efeade00640 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55e4dddb3ab0 0x55e4dddc5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeade00640 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55e4dddb3ab0 0x55e4dddc5570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242a602c2376f91d server_cookie=faa48d62a7ee194 in_seq=0 out_seq=0 2024-04-25T09:09:36.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efe93400640 10 client.7044 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:09:36.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efe93400640 1 -- 172.21.15.62:0/822212918 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e4dddb11a0 con 0x55e4dddb3ab0 2024-04-25T09:09:36.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efe93400640 10 client.7044 handle_command_reply: tid=0 2024-04-25T09:09:36.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 client.7044 shutdown 2024-04-25T09:09:36.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 -- 172.21.15.62:0/822212918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 msgr2=0x55e4dde09640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4dde09640 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efe9802a820 tx=0x7efe98004730 comp rx=0 tx=0).stop 2024-04-25T09:09:36.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 -- 172.21.15.62:0/822212918 shutdown_connections 2024-04-25T09:09:36.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55e4dddb3ab0 0x55e4dddc5570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e4ddddceb0 0x55e4dde09640 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4dddfd7a0 0x55e4dddb3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 --2- 172.21.15.62:0/822212918 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e4dddd9c20 0x55e4dde09100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efe93400640 0 client.7044 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.629+0000 7efeb7c2e740 1 -- 172.21.15.62:0/822212918 >> 172.21.15.62:0/822212918 conn(0x55e4dddb4b30 msgr2=0x55e4ddde1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeb7c2e740 1 -- 172.21.15.62:0/822212918 shutdown_connections 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeb7c2e740 1 -- 172.21.15.62:0/822212918 wait complete. 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeb7c2e740 20 client.7044 trim_cache size 0 max 16384 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 6894, 2024-04-25T09:09:36.620 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:09:36.621 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:09:36.621 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:09:36.621 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 6.6803807820000003, 2024-04-25T09:09:36.621 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 54, 2024-04-25T09:09:36.621 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 79, 2024-04-25T09:09:36.621 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T09:09:36.621 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 12.130606351000001 2024-04-25T09:09:36.621 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:09:36.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 -- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe8c06b4d0 msgr2=0x7efe8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe8c06b4d0 0x7efe8c06d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efe98034000 tx=0x7efe9802e000 comp rx=0 tx=0).stop 2024-04-25T09:09:36.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 -- 172.21.15.62:0/657719125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 msgr2=0x7efeb01f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb01f6c20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7efea000b420 tx=0x7efea000b8e0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 -- 172.21.15.62:0/657719125 shutdown_connections 2024-04-25T09:09:36.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe8c06b4d0 0x7efe8c06d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeb0161230 0x7efeb01492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d0fb0 0x7efeb00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 --2- 172.21.15.62:0/657719125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeb00b5c30 0x7efeb01f6c20 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.630+0000 7efeaf200640 1 -- 172.21.15.62:0/657719125 >> 172.21.15.62:0/657719125 conn(0x7efeb0066d10 msgr2=0x7efeb00cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:36.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.631+0000 7efeaf200640 1 -- 172.21.15.62:0/657719125 shutdown_connections 2024-04-25T09:09:36.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.631+0000 7efeaf200640 1 -- 172.21.15.62:0/657719125 wait complete. 2024-04-25T09:09:36.637 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:09:36.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.852+0000 7f00c8c00640 1 Processor -- start 2024-04-25T09:09:36.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.852+0000 7f00c8c00640 1 -- start start 2024-04-25T09:09:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00c8c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f00c40d0ae0 0x7f00c40d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00c8c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c40d2620 con 0x7f00c415dc40 2024-04-25T09:09:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00c8c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c40d2900 con 0x7f00c40b6cf0 2024-04-25T09:09:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00c8c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c40d23c0 con 0x7f00c40d0ae0 2024-04-25T09:09:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00c1e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f00c415dc40 0x7f00c40d1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60072/0 (socket says 172.21.15.62:60072) 2024-04-25T09:09:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00c1e00640 1 -- 172.21.15.62:0/230658097 learned_addr learned my addr 172.21.15.62:0/230658097 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00bbe00640 1 --2- 172.21.15.62:0/230658097 >> v2:172.21.15.121:3300/0 conn(0x7f00c40d0ae0 0x7f00c40d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686284679 0 0) 0x7f00c40d2620 con 0x7f00c415dc40 2024-04-25T09:09:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.854+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00a8003520 con 0x7f00c415dc40 2024-04-25T09:09:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 481977584 0 0) 0x7f00a8003520 con 0x7f00c415dc40 2024-04-25T09:09:36.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 >> v2:172.21.15.121:3300/0 conn(0x7f00c40d0ae0 msgr2=0x7f00c40d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 --2- 172.21.15.62:0/230658097 >> v2:172.21.15.121:3300/0 conn(0x7f00c40d0ae0 0x7f00c40d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 >> v1:172.21.15.121:6789/0 conn(0x7f00c40b6cf0 legacy=0x7f00c40b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40d2be0 con 0x7f00c415dc40 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f00b4001670 con 0x7f00c415dc40 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 >> v1:172.21.15.62:6789/0 conn(0x7f00c415dc40 legacy=0x7f00c40d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a80039e0 0x7f00a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00a8004800 0x7f00a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a8005210 0x7f00a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40d23c0 con 0x7f00a8005210 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a800c140 con 0x7f00a80039e0 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a800c340 con 0x7f00a8004800 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.855+0000 7f00c2800640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a8005210 0x7f00a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.856+0000 7f00c1e00640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a80039e0 0x7f00a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.856+0000 7f00bbe00640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00a8004800 0x7f00a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.856+0000 7f00c2800640 1 -- 172.21.15.62:0/230658097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00a8004800 msgr2=0x7f00a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.856+0000 7f00c2800640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00a8004800 0x7f00a8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.856+0000 7f00c2800640 1 -- 172.21.15.62:0/230658097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a80039e0 msgr2=0x7f00a8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:36.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.856+0000 7f00c2800640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a80039e0 0x7f00a8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.856+0000 7f00c2800640 1 -- 172.21.15.62:0/230658097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a8003520 con 0x7f00a8005210 2024-04-25T09:09:36.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.857+0000 7f00c2800640 1 -- 172.21.15.62:0/230658097 reap_dead start 2024-04-25T09:09:36.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.857+0000 7f00c2800640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a8005210 0x7f00a800b9c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0044b0 tx=0x7f00a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7336b7b34dd3b2d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.857+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc004080 con 0x7f00a8005210 2024-04-25T09:09:36.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.857+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc02ada0 con 0x7f00a8005210 2024-04-25T09:09:36.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.857+0000 7f00c1400640 1 -- 172.21.15.62:0/230658097 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc006d90 con 0x7f00a8005210 2024-04-25T09:09:36.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.858+0000 7f00c8c00640 1 -- 172.21.15.62:0/230658097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a8005210 msgr2=0x7f00a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.858+0000 7f00c8c00640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a8005210 0x7f00a800b9c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0044b0 tx=0x7f00a8005830 comp rx=0 tx=0).stop 2024-04-25T09:09:36.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.858+0000 7f00c8c00640 1 -- 172.21.15.62:0/230658097 shutdown_connections 2024-04-25T09:09:36.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.858+0000 7f00c8c00640 1 --2- 172.21.15.62:0/230658097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00a8005210 0x7f00a800b9c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.858+0000 7f00c8c00640 1 -- 172.21.15.62:0/230658097 >> 172.21.15.62:0/230658097 conn(0x7f00c406b000 msgr2=0x7f00c406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:36.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.858+0000 7f00c8c00640 1 -- 172.21.15.62:0/230658097 shutdown_connections 2024-04-25T09:09:36.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.858+0000 7f00c8c00640 1 -- 172.21.15.62:0/230658097 wait complete. 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.860+0000 7f00c8c00640 1 Processor -- start 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.860+0000 7f00c8c00640 1 -- start start 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00c8c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a8005210 0x7f00c4156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00c8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00c4153e70 0x7f00c40c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00c8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c41f6df0 0x7f00c41f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00c8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40d36c0 con 0x7f00c4153e70 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00c8c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40d31a0 con 0x7f00a8005210 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00c8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40ca110 con 0x7f00c41f6df0 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00c4153e70 0x7f00c40c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00c4153e70 0x7f00c40c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36960/0 (socket says 172.21.15.62:36960) 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00bbe00640 1 -- 172.21.15.62:0/4080498809 learned_addr learned my addr 172.21.15.62:0/4080498809 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.861+0000 7f00c1e00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a8005210 0x7f00c4156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00bbe00640 1 -- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c41f6df0 msgr2=0x7f00c41f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00bbe00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c41f6df0 0x7f00c41f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00bbe00640 1 -- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a8005210 msgr2=0x7f00c4156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:36.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00bbe00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a8005210 0x7f00c4156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:36.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00bbe00640 1 -- 172.21.15.62:0/4080498809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40ca3f0 con 0x7f00c4153e70 2024-04-25T09:09:36.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00bbe00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00c4153e70 0x7f00c40c6b60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e7e0 tx=0x7f00ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00ba000640 1 -- 172.21.15.62:0/4080498809 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac017070 con 0x7f00c4153e70 2024-04-25T09:09:36.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00ba000640 1 -- 172.21.15.62:0/4080498809 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac00c6d0 con 0x7f00c4153e70 2024-04-25T09:09:36.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00ba000640 1 -- 172.21.15.62:0/4080498809 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00f040 con 0x7f00c4153e70 2024-04-25T09:09:36.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c4156610 con 0x7f00c4153e70 2024-04-25T09:09:36.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.862+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c4156300 con 0x7f00c4153e70 2024-04-25T09:09:36.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.864+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c40729c0 con 0x7f00c4153e70 2024-04-25T09:09:36.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.864+0000 7f00ba000640 1 -- 172.21.15.62:0/4080498809 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00ac01a070 con 0x7f00c4153e70 2024-04-25T09:09:36.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.866+0000 7f00ba000640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f00a006b500 0x7f00a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:36.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.866+0000 7f00ba000640 1 -- 172.21.15.62:0/4080498809 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00ac089e00 con 0x7f00c4153e70 2024-04-25T09:09:36.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.867+0000 7f00c1e00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f00a006b500 0x7f00a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:36.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.868+0000 7f00c1e00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f00a006b500 0x7f00a006d9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f00b4001eb0 tx=0x7f00b4001190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:36.871+0000 7f00ba000640 1 -- 172.21.15.62:0/4080498809 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00ac058c70 con 0x7f00c4153e70 2024-04-25T09:09:37.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.123+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f00c404f000 con 0x7f00c4153e70 2024-04-25T09:09:37.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.124+0000 7f00ba000640 1 -- 172.21.15.62:0/4080498809 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7f00ac058570 con 0x7f00c4153e70 2024-04-25T09:09:37.111 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.126+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f00a006b500 msgr2=0x7f00a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.126+0000 7f00c8c00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f00a006b500 0x7f00a006d9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f00b4001eb0 tx=0x7f00b4001190 comp rx=0 tx=0).stop 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.126+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00c4153e70 msgr2=0x7f00c40c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.126+0000 7f00c8c00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00c4153e70 0x7f00c40c6b60 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e7e0 tx=0x7f00ac00eca0 comp rx=0 tx=0).stop 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.127+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 shutdown_connections 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.127+0000 7f00c8c00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f00a006b500 0x7f00a006d9c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.127+0000 7f00c8c00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f00c4153e70 0x7f00c40c6b60 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.127+0000 7f00c8c00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f00c41f6df0 0x7f00c41f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:37.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.127+0000 7f00c8c00640 1 --2- 172.21.15.62:0/4080498809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00a8005210 0x7f00c4156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:37.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.127+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 >> 172.21.15.62:0/4080498809 conn(0x7f00c406b000 msgr2=0x7f00c40d1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:37.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.127+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 shutdown_connections 2024-04-25T09:09:37.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:37.127+0000 7f00c8c00640 1 -- 172.21.15.62:0/4080498809 wait complete. 2024-04-25T09:09:37.132 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:09:37.132 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:09:37.132 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:09:37.132 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:09:37.132 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:37.133 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:09:37.133 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:09:37.133 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:09:37.133 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:09:37.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.133 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:09:37.182 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:09:37.182 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:09:37.183 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42282sec preferred_lft 42282sec 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:37.184 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:09:37.185 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:09:37.185 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:09:37.185 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:09:37.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.186 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:09:37.315 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:09:37.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.320 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:09:37.377 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:09:37.377 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:09:37.377 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:09:37.377 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:09:37.377 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:09:37.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.378 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:09:37.378 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:09:37.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:09:37.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:09:37.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:09:37.378 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:09:37.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.532 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:09:37.584 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:09:37.584 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:09:37.586 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:09:37.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.586 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:09:37.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:09:37.666 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:37.719 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:09:37.719 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:09:37.719 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:37.773 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:09:37.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.773 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:09:37.909 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:09:37.910 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:09:37.950 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:09:37.950 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:09:37.951 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:09:37.951 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:09:37.951 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:09:37.954 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:09:37.954 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:09:37.954 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:09:37.954 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:09:37.954 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:09:37.960 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:09:37.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:37.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:09:38.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:38.037 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:09:38.105 INFO:teuthology.orchestra.run.smithi062.stdout:client5384 2024-04-25T09:09:38.105 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:09:38.105 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:38.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T09:09:38.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.389+0000 7f6ea5c00640 1 Processor -- start 2024-04-25T09:09:38.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.390+0000 7f6ea5c00640 1 -- start start 2024-04-25T09:09:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.391+0000 7f6ea5c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6ea015b520 0x7f6ea00ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.391+0000 7f6ea5c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ea006c440 0x7f6ea00adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.391+0000 7f6ea5c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea006e270 con 0x7f6ea015eb90 2024-04-25T09:09:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.391+0000 7f6ea5c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea00b3fb0 con 0x7f6ea006c440 2024-04-25T09:09:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.391+0000 7f6ea5c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea006c920 con 0x7f6ea015b520 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6e9f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ea006c440 0x7f6ea00adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6ea5200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6ea015eb90 0x7f6ea015f040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60084/0 (socket says 172.21.15.62:60084) 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6ea5200640 1 -- 172.21.15.62:0/365714599 learned_addr learned my addr 172.21.15.62:0/365714599 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6e9fe00640 1 --2- 172.21.15.62:0/365714599 >> v2:172.21.15.121:3300/0 conn(0x7f6ea015b520 0x7f6ea00ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6e9f400640 1 -- 172.21.15.62:0/365714599 >> v2:172.21.15.121:3300/0 conn(0x7f6ea015b520 msgr2=0x7f6ea00ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6e9f400640 1 --2- 172.21.15.62:0/365714599 >> v2:172.21.15.121:3300/0 conn(0x7f6ea015b520 0x7f6ea00ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6e9f400640 1 -- 172.21.15.62:0/365714599 >> v1:172.21.15.62:6789/0 conn(0x7f6ea015eb90 legacy=0x7f6ea015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6e9f400640 1 -- 172.21.15.62:0/365714599 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea006cc00 con 0x7f6ea006c440 2024-04-25T09:09:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.392+0000 7f6e9fe00640 1 --2- 172.21.15.62:0/365714599 >> v2:172.21.15.121:3300/0 conn(0x7f6ea015b520 0x7f6ea00ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:09:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.393+0000 7f6e9f400640 1 --2- 172.21.15.62:0/365714599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea00adea0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6e880097a0 tx=0x7f6e8802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119a6bb81bc29ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6e9ea00640 1 -- 172.21.15.62:0/365714599 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88037070 con 0x7f6ea006c440 2024-04-25T09:09:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6e9ea00640 1 -- 172.21.15.62:0/365714599 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e88032440 con 0x7f6ea006c440 2024-04-25T09:09:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6e9ea00640 1 -- 172.21.15.62:0/365714599 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88032810 con 0x7f6ea006c440 2024-04-25T09:09:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6ea5c00640 1 -- 172.21.15.62:0/365714599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 msgr2=0x7f6ea00adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/365714599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea00adea0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6e880097a0 tx=0x7f6e8802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6ea5c00640 1 -- 172.21.15.62:0/365714599 shutdown_connections 2024-04-25T09:09:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/365714599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea00adea0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/365714599 >> v2:172.21.15.121:3300/0 conn(0x7f6ea015b520 0x7f6ea00ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6ea5c00640 1 -- 172.21.15.62:0/365714599 >> 172.21.15.62:0/365714599 conn(0x7f6ea0066ad0 msgr2=0x7f6ea0066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6ea5c00640 1 -- 172.21.15.62:0/365714599 shutdown_connections 2024-04-25T09:09:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.394+0000 7f6ea5c00640 1 -- 172.21.15.62:0/365714599 wait complete. 2024-04-25T09:09:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.396+0000 7f6ea5c00640 1 Processor -- start 2024-04-25T09:09:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.396+0000 7f6ea5c00640 1 -- start start 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.397+0000 7f6ea5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.397+0000 7f6ea5c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea015b520 0x7f6ea01f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.397+0000 7f6ea5c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ea015eb90 0x7f6ea0143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.397+0000 7f6ea5c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea015b900 con 0x7f6ea006c440 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.397+0000 7f6ea5c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea006d1c0 con 0x7f6ea015b520 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.397+0000 7f6ea5c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea015ba80 con 0x7f6ea015eb90 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea015a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36980/0 (socket says 172.21.15.62:36980) 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 -- 172.21.15.62:0/4075090174 learned_addr learned my addr 172.21.15.62:0/4075090174 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:09:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9f400640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea015b520 0x7f6ea01f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 -- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ea015eb90 msgr2=0x7f6ea0143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ea015eb90 0x7f6ea0143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 -- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea015b520 msgr2=0x7f6ea01f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea015b520 0x7f6ea01f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 -- 172.21.15.62:0/4075090174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea01f39b0 con 0x7f6ea006c440 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.398+0000 7f6e9fe00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea015a590 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000cf90 tx=0x7f6e9000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.399+0000 7f6e97e00640 1 -- 172.21.15.62:0/4075090174 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9001a070 con 0x7f6ea006c440 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.399+0000 7f6e97e00640 1 -- 172.21.15.62:0/4075090174 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9000d840 con 0x7f6ea006c440 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.399+0000 7f6e97e00640 1 -- 172.21.15.62:0/4075090174 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000dc10 con 0x7f6ea006c440 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.399+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea01f3c90 con 0x7f6ea006c440 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.399+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea01f4250 con 0x7f6ea006c440 2024-04-25T09:09:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.400+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea0072b30 con 0x7f6ea006c440 2024-04-25T09:09:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.401+0000 7f6e97e00640 1 -- 172.21.15.62:0/4075090174 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e9001d070 con 0x7f6ea006c440 2024-04-25T09:09:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.402+0000 7f6e97e00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e7c06b4d0 0x7f6e7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:09:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.402+0000 7f6e97e00640 1 -- 172.21.15.62:0/4075090174 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e9008a9c0 con 0x7f6ea006c440 2024-04-25T09:09:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.403+0000 7f6e9f400640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e7c06b4d0 0x7f6e7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:09:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.404+0000 7f6e9f400640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e7c06b4d0 0x7f6e7c06d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6e88034000 tx=0x7f6e8802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:09:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.406+0000 7f6e97e00640 1 -- 172.21.15.62:0/4075090174 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e900591e0 con 0x7f6ea006c440 2024-04-25T09:09:38.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:38.646+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6ea00750e0 con 0x7f6ea006c440 2024-04-25T09:09:38.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.010+0000 7f6e97e00640 1 -- 172.21.15.62:0/4075090174 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v55) v1 ==== 167+0+0 (secure 0 0 0) 0x7f6e9005d280 con 0x7f6ea006c440 2024-04-25T09:09:38.997 INFO:teuthology.orchestra.run.smithi062.stderr:enabled new snapshots 2024-04-25T09:09:39.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e7c06b4d0 msgr2=0x7f6e7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e7c06b4d0 0x7f6e7c06d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6e88034000 tx=0x7f6e8802e000 comp rx=0 tx=0).stop 2024-04-25T09:09:39.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 msgr2=0x7f6ea015a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea015a590 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000cf90 tx=0x7f6e9000a7d0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 shutdown_connections 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e7c06b4d0 0x7f6e7c06d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ea015eb90 0x7f6ea0143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ea015b520 0x7f6ea01f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 --2- 172.21.15.62:0/4075090174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea006c440 0x7f6ea015a590 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 >> 172.21.15.62:0/4075090174 conn(0x7f6ea0066ad0 msgr2=0x7f6ea00ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 shutdown_connections 2024-04-25T09:09:39.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:09:39.014+0000 7f6ea5c00640 1 -- 172.21.15.62:0/4075090174 wait complete. 2024-04-25T09:09:39.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:09:39.023 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T09:09:39.090 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:09:39.090 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:09:39.146 DEBUG:teuthology.orchestra.run.smithi062:> stdin-killer -- python3 -c ' 2024-04-25T09:09:39.146 DEBUG:teuthology.orchestra.run.smithi062:> import os 2024-04-25T09:09:39.146 DEBUG:teuthology.orchestra.run.smithi062:> import uuid 2024-04-25T09:09:39.146 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:09:39.146 DEBUG:teuthology.orchestra.run.smithi062:> n = 2000 2024-04-25T09:09:39.146 DEBUG:teuthology.orchestra.run.smithi062:> create_hard_links = False 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> if 3 > 0: 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> create_hard_links = True 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> dpath = os.path.dirname(path) 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> fnameprefix = os.path.basename(path) 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> os.makedirs(dpath, exist_ok=True) 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> try: 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> 2024-04-25T09:09:39.147 DEBUG:teuthology.orchestra.run.smithi062:> for i in range(n): 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> f.write(f"{i}") 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> if False: 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> f.flush() 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> os.fsync(f.fileno()) 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> if False: 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> os.unlink(fpath) 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> if False: 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> os.fsync(dirfd) 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> if create_hard_links: 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> for j in range(3): 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T09:09:39.148 DEBUG:teuthology.orchestra.run.smithi062:> if False: 2024-04-25T09:09:39.149 DEBUG:teuthology.orchestra.run.smithi062:> os.fsync(dirfd) 2024-04-25T09:09:39.149 DEBUG:teuthology.orchestra.run.smithi062:> finally: 2024-04-25T09:09:39.149 DEBUG:teuthology.orchestra.run.smithi062:> os.close(dirfd) 2024-04-25T09:09:39.149 DEBUG:teuthology.orchestra.run.smithi062:> ' 2024-04-25T09:09:41.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:09:41.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:10:11.344 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:10:11.349 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:10:28.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:10:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:10:28.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:28.254 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:10:28.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:10:28.335 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T09:10:41.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:10:41.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:11:11.600 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:11:11.609 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:11:29.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:29.529 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:11:29.602 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:11:29.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T09:11:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.808+0000 7f4fb2200640 1 Processor -- start 2024-04-25T09:11:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.808+0000 7f4fb2200640 1 -- start start 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.809+0000 7f4fb2200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4fac06d2c0 0x7f4fac0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.809+0000 7f4fb2200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fac06d6a0 0x7f4fac0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.809+0000 7f4fb2200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac0b31a0 0x7f4fac0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.809+0000 7f4fb2200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac0ced40 con 0x7f4fac06d6a0 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fb2200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac0cf000 con 0x7f4fac06d2c0 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fb2200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0cf2e0 con 0x7f4fac0b31a0 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fabe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fac06d6a0 0x7f4fac0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fabe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fac06d6a0 0x7f4fac0ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47742/0 (socket says 172.21.15.62:47742) 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fabe00640 1 -- 172.21.15.62:0/1791428113 learned_addr learned my addr 172.21.15.62:0/1791428113 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/1791428113 >> v2:172.21.15.121:3300/0 conn(0x7f4fac06d2c0 0x7f4fac0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fb1800640 1 --2- 172.21.15.62:0/1791428113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac0b31a0 0x7f4fac0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fabe00640 1 -- 172.21.15.62:0/1791428113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac0b31a0 msgr2=0x7f4fac0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.810+0000 7f4fabe00640 1 --2- 172.21.15.62:0/1791428113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac0b31a0 0x7f4fac0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.811+0000 7f4fabe00640 1 -- 172.21.15.62:0/1791428113 >> v2:172.21.15.121:3300/0 conn(0x7f4fac06d2c0 msgr2=0x7f4fac0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.811+0000 7f4fabe00640 1 --2- 172.21.15.62:0/1791428113 >> v2:172.21.15.121:3300/0 conn(0x7f4fac06d2c0 0x7f4fac0ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.811+0000 7f4fabe00640 1 -- 172.21.15.62:0/1791428113 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0b26c0 con 0x7f4fac06d6a0 2024-04-25T09:11:29.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.812+0000 7f4fabe00640 1 --2- 172.21.15.62:0/1791428113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0ce800 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0b20a0 tx=0x7f4f9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fed8eeb6ace8709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:29.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.812+0000 7f4fab400640 1 -- 172.21.15.62:0/1791428113 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98037070 con 0x7f4fac06d6a0 2024-04-25T09:11:29.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.812+0000 7f4fab400640 1 -- 172.21.15.62:0/1791428113 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f98032440 con 0x7f4fac06d6a0 2024-04-25T09:11:29.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.812+0000 7f4fab400640 1 -- 172.21.15.62:0/1791428113 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98032810 con 0x7f4fac06d6a0 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 -- 172.21.15.62:0/1791428113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 msgr2=0x7f4fac0ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 --2- 172.21.15.62:0/1791428113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0ce800 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0b20a0 tx=0x7f4f9802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 -- 172.21.15.62:0/1791428113 shutdown_connections 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 --2- 172.21.15.62:0/1791428113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac0b31a0 0x7f4fac0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 --2- 172.21.15.62:0/1791428113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0ce800 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 --2- 172.21.15.62:0/1791428113 >> v2:172.21.15.121:3300/0 conn(0x7f4fac06d2c0 0x7f4fac0ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 -- 172.21.15.62:0/1791428113 >> 172.21.15.62:0/1791428113 conn(0x7f4fac066f20 msgr2=0x7f4fac0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 -- 172.21.15.62:0/1791428113 shutdown_connections 2024-04-25T09:11:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.813+0000 7f4fb2200640 1 -- 172.21.15.62:0/1791428113 wait complete. 2024-04-25T09:11:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.815+0000 7f4fb2200640 1 Processor -- start 2024-04-25T09:11:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.815+0000 7f4fb2200640 1 -- start start 2024-04-25T09:11:29.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fb2200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac06d2c0 0x7f4fac131530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fb2200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fb2200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac0b31a0 0x7f4fac1418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fb2200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac06db20 con 0x7f4fac06d6a0 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fb2200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0b2c80 con 0x7f4fac0b31a0 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fb2200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1448d0 con 0x7f4fac06d2c0 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47744/0 (socket says 172.21.15.62:47744) 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fabe00640 1 -- 172.21.15.62:0/4201663793 learned_addr learned my addr 172.21.15.62:0/4201663793 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac06d2c0 0x7f4fac131530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.816+0000 7f4fb1800640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac0b31a0 0x7f4fac1418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fabe00640 1 -- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac06d2c0 msgr2=0x7f4fac131530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fabe00640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac06d2c0 0x7f4fac131530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fabe00640 1 -- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac0b31a0 msgr2=0x7f4fac1418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fabe00640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac0b31a0 0x7f4fac1418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fabe00640 1 -- 172.21.15.62:0/4201663793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac12e290 con 0x7f4fac06d6a0 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac06d2c0 0x7f4fac131530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fb1800640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac0b31a0 0x7f4fac1418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fabe00640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0a9800 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4f98000c00 tx=0x7f4f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:29.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fa8c00640 1 -- 172.21.15.62:0/4201663793 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98037040 con 0x7f4fac06d6a0 2024-04-25T09:11:29.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fa8c00640 1 -- 172.21.15.62:0/4201663793 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f98042070 con 0x7f4fac06d6a0 2024-04-25T09:11:29.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fa8c00640 1 -- 172.21.15.62:0/4201663793 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98030880 con 0x7f4fac06d6a0 2024-04-25T09:11:29.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fb2200640 1 -- 172.21.15.62:0/4201663793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac12e570 con 0x7f4fac06d6a0 2024-04-25T09:11:29.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.817+0000 7f4fb2200640 1 -- 172.21.15.62:0/4201663793 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac12eb30 con 0x7f4fac06d6a0 2024-04-25T09:11:29.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.820+0000 7f4fa8c00640 1 -- 172.21.15.62:0/4201663793 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f98030a20 con 0x7f4fac06d6a0 2024-04-25T09:11:29.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.821+0000 7f4fa8c00640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.821+0000 7f4fa8c00640 1 -- 172.21.15.62:0/4201663793 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4f980aabc0 con 0x7f4fac06d6a0 2024-04-25T09:11:29.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.822+0000 7f4fb65f1740 1 Processor -- start 2024-04-25T09:11:29.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.822+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.822+0000 7f4fb65f1740 1 -- start start 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6ec80 0x562af0a6f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6f6a0 0x562af0a3ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a3f3c0 0x562af0a67100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af09a3870 con 0x562af0a6ec80 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af094cd60 con 0x562af0a6f6a0 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562af099f910 con 0x562af0a3f3c0 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6ec80 0x562af0a6f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6ec80 0x562af0a6f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47748/0 (socket says 172.21.15.62:47748) 2024-04-25T09:11:29.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.823+0000 7f4fb0e00640 1 -- 172.21.15.62:0/2239950248 learned_addr learned my addr 172.21.15.62:0/2239950248 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00099a0 tx=0x7f4fa0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fabe00640 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6f6a0 0x562af0a3ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb1800640 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a3f3c0 0x562af0a67100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb0e00640 1 -- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a3f3c0 msgr2=0x562af0a67100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a3f3c0 0x562af0a67100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb0e00640 1 -- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6f6a0 msgr2=0x562af0a3ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6f6a0 0x562af0a3ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb0e00640 1 -- 172.21.15.62:0/2239950248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af07827d0 con 0x562af0a6ec80 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fabe00640 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6f6a0 0x562af0a3ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb1800640 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a3f3c0 0x562af0a67100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6ec80 0x562af0a6f060 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0010b20 tx=0x7f4fa0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177c97be8baab24f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4f9d600640 1 -- 172.21.15.62:0/2239950248 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa001c050 con 0x562af0a6ec80 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4f9d600640 1 -- 172.21.15.62:0/2239950248 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa0026070 con 0x562af0a6ec80 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.824+0000 7f4f9d600640 1 -- 172.21.15.62:0/2239950248 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000f040 con 0x562af0a6ec80 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 -- 172.21.15.62:0/2239950248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6ec80 msgr2=0x562af0a6f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6ec80 0x562af0a6f060 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0010b20 tx=0x7f4fa0013000 comp rx=0 tx=0).stop 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 -- 172.21.15.62:0/2239950248 shutdown_connections 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a3f3c0 0x562af0a67100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6f6a0 0x562af0a3ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/2239950248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6ec80 0x562af0a6f060 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 -- 172.21.15.62:0/2239950248 >> 172.21.15.62:0/2239950248 conn(0x562af0a41300 msgr2=0x562af0a89a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 -- 172.21.15.62:0/2239950248 shutdown_connections 2024-04-25T09:11:29.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.825+0000 7f4fb65f1740 1 -- 172.21.15.62:0/2239950248 wait complete. 2024-04-25T09:11:29.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.826+0000 7f4fb65f1740 1 -- start start 2024-04-25T09:11:29.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.826+0000 7f4fb65f1740 1 Processor -- start 2024-04-25T09:11:29.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.827+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a5f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.827+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.827+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 0x562af0a63f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.827+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af0a6b3d0 con 0x562af0a6f6a0 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.827+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af09601c0 con 0x562af0a6ec80 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.827+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562af0a6b670 con 0x562af0a5f080 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.827+0000 7f4fb65f1740 10 client.0 _ll_register_callbacks cb 0x562af099e8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47758/0 (socket says 172.21.15.62:47758) 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 -- 172.21.15.62:0/1088354912 learned_addr learned my addr 172.21.15.62:0/1088354912 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a5f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 -- 172.21.15.62:0/1088354912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 msgr2=0x562af0a63f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 0x562af0a63f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 -- 172.21.15.62:0/1088354912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 msgr2=0x562af0a5f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a5f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 -- 172.21.15.62:0/1088354912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562af06780f0 con 0x562af0a6f6a0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a5f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4fabe00640 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a639e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4f98078f60 tx=0x7f4f98032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a63cba3f215338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.828+0000 7f4f92000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4f92000640 1 -- 172.21.15.62:0/1088354912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9807d3b0 con 0x562af0a6f6a0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4f92000640 1 -- 172.21.15.62:0/1088354912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4f9807d0d0 con 0x562af0a6f6a0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 1 -- 172.21.15.62:0/1088354912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fa0026070 con 0x562af0a6f6a0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4f92000640 1 -- 172.21.15.62:0/1088354912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f98081970 con 0x562af0a6f6a0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 10 client.7071 fetch_fsmap learned FSMap version 55 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 1 -- 172.21.15.62:0/1088354912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x562af07e2f60 con 0x562af0a6f6a0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4f92000640 1 -- 172.21.15.62:0/1088354912 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa0026070 con 0x562af0a6f6a0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4f92000640 1 -- 172.21.15.62:0/1088354912 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4f9803a070 con 0x562af0a6f6a0 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 10 client.7071 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 10 client.7071 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:11:29.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 20 client.7071 populate_metadata read hostname 'smithi062' 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x562af0a6d320 0x562af0a6d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 4 client.7071 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.829+0000 7f4fb65f1740 1 -- 172.21.15.62:0/1088354912 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562af0979690 con 0x562af0a6d320 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.830+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x562af0a6d320 0x562af0a6d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.830+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x562af0a6d320 0x562af0a6d720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d91a89f158a2d5 server_cookie=58066aefba951a44 in_seq=0 out_seq=0 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.830+0000 7f4f92000640 10 client.7071 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4f92000640 1 -- 172.21.15.62:0/1088354912 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562af0979690 con 0x562af0a6d320 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4f92000640 10 client.7071 handle_command_reply: tid=0 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 client.7071 shutdown 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 -- 172.21.15.62:0/1088354912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 msgr2=0x562af0a639e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a639e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4f98078f60 tx=0x7f4f98032440 comp rx=0 tx=0).stop 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 -- 172.21.15.62:0/1088354912 shutdown_connections 2024-04-25T09:11:29.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x562af0a6d320 0x562af0a6d720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a639e0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 0x562af0a63f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/1088354912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a5f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4f92000640 0 client.7071 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.831+0000 7f4fb65f1740 1 -- 172.21.15.62:0/1088354912 >> 172.21.15.62:0/1088354912 conn(0x562af0a3f3c0 msgr2=0x562af0a42840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.832+0000 7f4fb65f1740 1 -- 172.21.15.62:0/1088354912 shutdown_connections 2024-04-25T09:11:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.832+0000 7f4fb65f1740 1 -- 172.21.15.62:0/1088354912 wait complete. 2024-04-25T09:11:29.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.832+0000 7f4fb65f1740 20 client.7071 trim_cache size 0 max 16384 2024-04-25T09:11:29.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.847+0000 7f4fb65f1740 1 Processor -- start 2024-04-25T09:11:29.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.848+0000 7f4fb65f1740 1 -- start start 2024-04-25T09:11:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.848+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 0x562af0a6d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.848+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a61b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.848+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a6bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.848+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af0a40100 con 0x562af0a6f6a0 2024-04-25T09:11:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.848+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af0a624a0 con 0x562af0a6ec80 2024-04-25T09:11:29.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.848+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1fba40 con 0x562af0a5f080 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a6bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a6bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47766/0 (socket says 172.21.15.62:47766) 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb1800640 1 -- 172.21.15.62:0/3077551079 learned_addr learned my addr 172.21.15.62:0/3077551079 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fabe00640 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a61b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 0x562af0a6d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb1800640 1 -- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 msgr2=0x562af0a6d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb1800640 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 0x562af0a6d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb1800640 1 -- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 msgr2=0x562af0a61b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb1800640 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a61b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fb1800640 1 -- 172.21.15.62:0/3077551079 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af0a6f3d0 con 0x562af0a6f6a0 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.849+0000 7f4fabe00640 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a61b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:29.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.850+0000 7f4fb1800640 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a6bb00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009d20 tx=0x7f4fa4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0642206da87c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.850+0000 7f4f9cc00640 1 -- 172.21.15.62:0/3077551079 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4007e10 con 0x562af0a6f6a0 2024-04-25T09:11:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.850+0000 7f4f9cc00640 1 -- 172.21.15.62:0/3077551079 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa400e070 con 0x562af0a6f6a0 2024-04-25T09:11:29.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.850+0000 7f4f9cc00640 1 -- 172.21.15.62:0/3077551079 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4003bd0 con 0x562af0a6f6a0 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3077551079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 msgr2=0x562af0a6bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a6bb00 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009d20 tx=0x7f4fa4007700 comp rx=0 tx=0).stop 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3077551079 shutdown_connections 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a6bb00 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a6ec80 0x562af0a61b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3077551079 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a5f080 0x562af0a6d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3077551079 >> 172.21.15.62:0/3077551079 conn(0x562af0a3f3c0 msgr2=0x562af0a6c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3077551079 shutdown_connections 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.851+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3077551079 wait complete. 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.852+0000 7f4fb65f1740 1 -- start start 2024-04-25T09:11:29.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.852+0000 7f4fb65f1740 1 Processor -- start 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a5f080 0x562af0a62f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a6ec80 0x562af0a43b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb65f1740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a8d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac12fc40 con 0x562af0a6f6a0 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac071420 con 0x562af0a5f080 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb65f1740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1e4ac0 con 0x562af0a6ec80 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb65f1740 10 client.0 _ll_register_callbacks cb 0x7f4fa0009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a8d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a8d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47774/0 (socket says 172.21.15.62:47774) 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb1800640 1 -- 172.21.15.62:0/3285181839 learned_addr learned my addr 172.21.15.62:0/3285181839 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a5f080 0x562af0a62f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.853+0000 7f4fb1800640 1 -- 172.21.15.62:0/3285181839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a6ec80 msgr2=0x562af0a43b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4fb1800640 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a6ec80 0x562af0a43b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4fb1800640 1 -- 172.21.15.62:0/3285181839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a5f080 msgr2=0x562af0a62f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4fb1800640 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a5f080 0x562af0a62f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4fb1800640 1 -- 172.21.15.62:0/3285181839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562af06511b0 con 0x562af0a6f6a0 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4fb1800640 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a8d490 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4007740 tx=0x7f4fa400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bc564d709b4903 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4f92000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4f92000640 1 -- 172.21.15.62:0/3285181839 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40147f0 con 0x562af0a6f6a0 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4f92000640 1 -- 172.21.15.62:0/3285181839 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4fa4014990 con 0x562af0a6f6a0 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3285181839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fa400e070 con 0x562af0a6f6a0 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4f92000640 1 -- 172.21.15.62:0/3285181839 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fa4031050 con 0x562af0a6f6a0 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.854+0000 7f4fb65f1740 10 client.7077 fetch_fsmap learned FSMap version 55 2024-04-25T09:11:29.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3285181839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x562af0a65420 con 0x562af0a6f6a0 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4f92000640 1 -- 172.21.15.62:0/3285181839 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa400e070 con 0x562af0a6f6a0 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4f92000640 1 -- 172.21.15.62:0/3285181839 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4fa4034020 con 0x562af0a6f6a0 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4fb65f1740 10 client.7077 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4fb65f1740 10 client.7077 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4fb65f1740 20 client.7077 populate_metadata read hostname 'smithi062' 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x562af0a42460 0x562af0a8d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4fb65f1740 4 client.7077 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3285181839 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x562af09b0db0 con 0x562af0a42460 2024-04-25T09:11:29.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.855+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x562af0a42460 0x562af0a8d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.856+0000 7f4fb0e00640 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x562af0a42460 0x562af0a8d9d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2db662958cd521 server_cookie=a2cc5f5da3e31db5 in_seq=0 out_seq=0 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.856+0000 7f4f92000640 10 client.7077 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.856+0000 7f4f92000640 1 -- 172.21.15.62:0/3285181839 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17926 (crc 0 0 0) 0x562af09b0db0 con 0x562af0a42460 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.856+0000 7f4f92000640 10 client.7077 handle_command_reply: tid=0 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.856+0000 7f4fb65f1740 1 client.7077 shutdown 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3285181839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 msgr2=0x562af0a8d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a8d490 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4007740 tx=0x7f4fa400b040 comp rx=0 tx=0).stop 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3285181839 shutdown_connections 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x562af0a42460 0x562af0a8d9d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562af0a6f6a0 0x562af0a8d490 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562af0a6ec80 0x562af0a43b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 --2- 172.21.15.62:0/3285181839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562af0a5f080 0x562af0a62f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4f92000640 0 client.7077 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3285181839 >> 172.21.15.62:0/3285181839 conn(0x562af0a3f3c0 msgr2=0x562af0a36080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3285181839 shutdown_connections 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 1 -- 172.21.15.62:0/3285181839 wait complete. 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.857+0000 7f4fb65f1740 20 client.7077 trim_cache size 0 max 16384 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2651, 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2652, 2024-04-25T09:11:29.848 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 657645, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 35591940, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 9, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.631402353, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.317534291, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.117353692, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.147308861, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2651, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.601780050, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000227001 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:11:29.849 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000149, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000074 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 29573, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 50576, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 10717907, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 56628523, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 12.405735512, 2024-04-25T09:11:29.850 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 6.915363900, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 5.939297779, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.757649974, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 50575, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.670657685, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000072578 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 69499, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.041661313, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000599 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 121520, 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 27280 2024-04-25T09:11:29.851 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2161, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2186, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 735948, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 26191262, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.599651580, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.322096542, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.111835482, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.127647594, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2185, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.470112366, 2024-04-25T09:11:29.852 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000215154 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 128720 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-25T09:11:29.853 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4963, 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 29.107633815, 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.005864927 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1985, 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.663749313, 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000838160 2024-04-25T09:11:29.854 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 25900, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 25900, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 25900, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 571.911498103, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.022081525 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 12, 2024-04-25T09:11:29.855 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 19872, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 19872, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 19872, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 21, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 21, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 2, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 48226, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 47724, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-25T09:11:29.856 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 157, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-04-25T09:11:29.857 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 913, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 4029, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 3766, 2024-04-25T09:11:29.858 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 61, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 14030 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 2000, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 165, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 4124, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 1817, 2024-04-25T09:11:29.859 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 2222, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-25T09:11:29.860 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 24786, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 24786, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 35, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 35, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 4, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 94926762, 2024-04-25T09:11:29.861 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4414, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 147.483639994, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.033412695 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 2014, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 2014, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 0, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 13, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 13, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-04-25T09:11:29.862 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 19872, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 19873, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 1, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 21, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 8131, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 8110, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 108428, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215396 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 29415, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 25900, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 7, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-25T09:11:29.863 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2000, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.762960091, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001381480 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 29, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.009297966, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000320619 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-25T09:11:29.864 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6000, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 13.645006699, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002274167 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 7636, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 14.989634055, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001963021 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-25T09:11:29.865 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.866 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000083678, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000083678 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.006282833, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006282833 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-25T09:11:29.867 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.007704157, 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.007704157 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.868 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.152198260, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.019024782 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2222, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 505.153400569, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.227341764 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.869 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.020501202, 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.020501202 2024-04-25T09:11:29.870 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000341124, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000341124 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.871 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.872 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8000, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 35.164087469, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004395510 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.873 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 14338, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 14338, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 111, 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-25T09:11:29.874 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:11:29.875 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:11:29.876 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 16505790, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 6218, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 528, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 6, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:11:29.877 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 5504, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 79, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 22084337, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 902510, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.878 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 6633, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 91124948, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 27, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 6633, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6633, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 46.662263052, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.007034865 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-25T09:11:29.879 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6633, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 6793 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 6633, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 92, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 6541, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 11, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 92, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 4506, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 54, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-25T09:11:29.880 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 1817, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 33, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 22, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:11:29.881 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-25T09:11:29.882 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 84, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 6, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 8, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 8, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 210, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 48 2024-04-25T09:11:29.883 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 32, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 1817, 2024-04-25T09:11:29.884 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 1817, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 34381, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 9011772, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 34381, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 34381, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 9011772, 2024-04-25T09:11:29.885 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T09:11:29.886 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 6633, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 91323001, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 6633, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 91323001, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:11:29.887 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 6633, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 6633, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 6633, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 6633, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:11:29.888 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 1817, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 198053, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 1817, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 92, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 198053, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.889 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x555f15b746e0": { 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 60, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 24786, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 90732458, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 24786, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4414, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 90732398, 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:11:29.890 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:11:29.891 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:11:29.891 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:11:29.891 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.891 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:29.891 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:11:29.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 -- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8c06b4d0 msgr2=0x7f4f8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00099a0 tx=0x7f4fa0008040 comp rx=0 tx=0).stop 2024-04-25T09:11:29.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 -- 172.21.15.62:0/4201663793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 msgr2=0x7f4fac0a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:29.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0a9800 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4f98000c00 tx=0x7f4f980046c0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 -- 172.21.15.62:0/4201663793 shutdown_connections 2024-04-25T09:11:29.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fac06d6a0 0x7f4fac0a9800 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fac0b31a0 0x7f4fac1418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 --2- 172.21.15.62:0/4201663793 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fac06d2c0 0x7f4fac131530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:29.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 -- 172.21.15.62:0/4201663793 >> 172.21.15.62:0/4201663793 conn(0x7f4fac066f20 msgr2=0x7f4fac15cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:29.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 -- 172.21.15.62:0/4201663793 shutdown_connections 2024-04-25T09:11:29.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:29.858+0000 7f4fb2200640 1 -- 172.21.15.62:0/4201663793 wait complete. 2024-04-25T09:11:29.894 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T09:11:29.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:29.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T09:11:30.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.135+0000 7fe0ebe00640 1 Processor -- start 2024-04-25T09:11:30.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.135+0000 7fe0ebe00640 1 -- start start 2024-04-25T09:11:30.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.137+0000 7fe0ebe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe0ec15b520 0x7fe0ec0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.137+0000 7fe0ebe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0ec06c440 0x7fe0ec0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.137+0000 7fe0ebe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec15eb90 0x7fe0ec15f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.137+0000 7fe0ebe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ec15f580 con 0x7fe0ec06c440 2024-04-25T09:11:30.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.137+0000 7fe0ebe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ec06e270 con 0x7fe0ec15b520 2024-04-25T09:11:30.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.137+0000 7fe0ebe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec0b3fb0 con 0x7fe0ec15eb90 2024-04-25T09:11:30.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0e2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0ec06c440 0x7fe0ec0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0e2000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0ec06c440 0x7fe0ec0adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47786/0 (socket says 172.21.15.62:47786) 2024-04-25T09:11:30.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0e2000640 1 -- 172.21.15.62:0/3155260593 learned_addr learned my addr 172.21.15.62:0/3155260593 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0eb400640 1 --2- 172.21.15.62:0/3155260593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec15eb90 0x7fe0ec15f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3155260593 >> v2:172.21.15.121:3300/0 conn(0x7fe0ec15b520 0x7fe0ec0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0e2000640 1 -- 172.21.15.62:0/3155260593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec15eb90 msgr2=0x7fe0ec15f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3155260593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec15eb90 0x7fe0ec15f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0e2000640 1 -- 172.21.15.62:0/3155260593 >> v2:172.21.15.121:3300/0 conn(0x7fe0ec15b520 msgr2=0x7fe0ec0ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3155260593 >> v2:172.21.15.121:3300/0 conn(0x7fe0ec15b520 0x7fe0ec0ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.138+0000 7fe0e2000640 1 -- 172.21.15.62:0/3155260593 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec06c920 con 0x7fe0ec06c440 2024-04-25T09:11:30.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.140+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3155260593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec06c440 0x7fe0ec0adea0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009870 tx=0x7fe0d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4490f5cc029871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.140+0000 7fe0ea000640 1 -- 172.21.15.62:0/3155260593 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4037070 con 0x7fe0ec06c440 2024-04-25T09:11:30.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.140+0000 7fe0ea000640 1 -- 172.21.15.62:0/3155260593 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d4032440 con 0x7fe0ec06c440 2024-04-25T09:11:30.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.140+0000 7fe0ea000640 1 -- 172.21.15.62:0/3155260593 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4032810 con 0x7fe0ec06c440 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 -- 172.21.15.62:0/3155260593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec06c440 msgr2=0x7fe0ec0adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/3155260593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec06c440 0x7fe0ec0adea0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009870 tx=0x7fe0d402a210 comp rx=0 tx=0).stop 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 -- 172.21.15.62:0/3155260593 shutdown_connections 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/3155260593 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec15eb90 0x7fe0ec15f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/3155260593 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec06c440 0x7fe0ec0adea0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/3155260593 >> v2:172.21.15.121:3300/0 conn(0x7fe0ec15b520 0x7fe0ec0ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 -- 172.21.15.62:0/3155260593 >> 172.21.15.62:0/3155260593 conn(0x7fe0ec066ad0 msgr2=0x7fe0ec066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 -- 172.21.15.62:0/3155260593 shutdown_connections 2024-04-25T09:11:30.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.141+0000 7fe0ebe00640 1 -- 172.21.15.62:0/3155260593 wait complete. 2024-04-25T09:11:30.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.143+0000 7fe0ebe00640 1 Processor -- start 2024-04-25T09:11:30.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.143+0000 7fe0ebe00640 1 -- start start 2024-04-25T09:11:30.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0ebe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec06c440 0x7fe0ec15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0ebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec15b520 0x7fe0ec1362d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0ebe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0ec15eb90 0x7fe0ec0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0ebe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec06d1c0 con 0x7fe0ec15b520 2024-04-25T09:11:30.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0ebe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec06cee0 con 0x7fe0ec15eb90 2024-04-25T09:11:30.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0ebe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec15b900 con 0x7fe0ec06c440 2024-04-25T09:11:30.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0e2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec15b520 0x7fe0ec1362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0e2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec15b520 0x7fe0ec1362d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47798/0 (socket says 172.21.15.62:47798) 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.144+0000 7fe0e2000640 1 -- 172.21.15.62:0/1966649931 learned_addr learned my addr 172.21.15.62:0/1966649931 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0eb400640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0ec15eb90 0x7fe0ec0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec06c440 0x7fe0ec15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e2000640 1 -- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec06c440 msgr2=0x7fe0ec15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e2000640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec06c440 0x7fe0ec15a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e2000640 1 -- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0ec15eb90 msgr2=0x7fe0ec0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e2000640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0ec15eb90 0x7fe0ec0711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e2000640 1 -- 172.21.15.62:0/1966649931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec136810 con 0x7fe0ec15b520 2024-04-25T09:11:30.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e2000640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec15b520 0x7fe0ec1362d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4000c00 tx=0x7fe0d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e3400640 1 -- 172.21.15.62:0/1966649931 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4041020 con 0x7fe0ec15b520 2024-04-25T09:11:30.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e3400640 1 -- 172.21.15.62:0/1966649931 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d40306c0 con 0x7fe0ec15b520 2024-04-25T09:11:30.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0e3400640 1 -- 172.21.15.62:0/1966649931 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4037040 con 0x7fe0ec15b520 2024-04-25T09:11:30.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.145+0000 7fe0ebe00640 1 -- 172.21.15.62:0/1966649931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec0cc7e0 con 0x7fe0ec15b520 2024-04-25T09:11:30.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.146+0000 7fe0ebe00640 1 -- 172.21.15.62:0/1966649931 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec0ccda0 con 0x7fe0ec15b520 2024-04-25T09:11:30.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.148+0000 7fe0e3400640 1 -- 172.21.15.62:0/1966649931 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0d4044020 con 0x7fe0ec15b520 2024-04-25T09:11:30.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.149+0000 7fe0e3400640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0c806b4d0 0x7fe0c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.149+0000 7fe0e3400640 1 -- 172.21.15.62:0/1966649931 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe0d40aab00 con 0x7fe0ec15b520 2024-04-25T09:11:30.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.150+0000 7fe0f4a4a740 1 Processor -- start 2024-04-25T09:11:30.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.150+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0c806b4d0 0x7fe0c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.150+0000 7fe0f4a4a740 1 -- start start 2024-04-25T09:11:30.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146b540 0x56469146ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x56469146b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x56469148af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646913a7ee0 con 0x56469146b540 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564691350cf0 con 0x56469146f010 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646913a3f80 con 0x56469146be60 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0c806b4d0 0x7fe0c806d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc006fd0 tx=0x7fe0dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146b540 0x56469146ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146b540 0x56469146ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47802/0 (socket says 172.21.15.62:47802) 2024-04-25T09:11:30.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0eaa00640 1 -- 172.21.15.62:0/1104189484 learned_addr learned my addr 172.21.15.62:0/1104189484 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0e2000640 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x56469146b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.151+0000 7fe0eb400640 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x56469148af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0eaa00640 1 -- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 msgr2=0x56469148af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x56469148af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0eaa00640 1 -- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 msgr2=0x56469146b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x56469146b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0eaa00640 1 -- 172.21.15.62:0/1104189484 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564691188a20 con 0x56469146b540 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0e2000640 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x56469146b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146b540 0x56469146ead0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00e910 tx=0x7fe0dc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f022d75b3e4f7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0db400640 1 -- 172.21.15.62:0/1104189484 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc025070 con 0x56469146b540 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0db400640 1 -- 172.21.15.62:0/1104189484 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc015640 con 0x56469146b540 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.152+0000 7fe0db400640 1 -- 172.21.15.62:0/1104189484 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00f040 con 0x56469146b540 2024-04-25T09:11:30.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/1104189484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146b540 msgr2=0x56469146ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146b540 0x56469146ead0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00e910 tx=0x7fe0dc011000 comp rx=0 tx=0).stop 2024-04-25T09:11:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/1104189484 shutdown_connections 2024-04-25T09:11:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x56469148af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x56469146b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/1104189484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146b540 0x56469146ead0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/1104189484 >> 172.21.15.62:0/1104189484 conn(0x564691441330 msgr2=0x564691442330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/1104189484 shutdown_connections 2024-04-25T09:11:30.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.153+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/1104189484 wait complete. 2024-04-25T09:11:30.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.154+0000 7fe0f4a4a740 1 -- start start 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.154+0000 7fe0f4a4a740 1 Processor -- start 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146be60 0x564691467790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x564691467dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564691468310 0x564691469ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56469143fd70 con 0x56469146be60 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564691346240 con 0x56469146f010 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56469143f160 con 0x564691468310 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0f4a4a740 10 client.0 _ll_register_callbacks cb 0x5646910bbe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146be60 0x564691467790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0eaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146be60 0x564691467790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47806/0 (socket says 172.21.15.62:47806) 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.155+0000 7fe0eaa00640 1 -- 172.21.15.62:0/3195686867 learned_addr learned my addr 172.21.15.62:0/3195686867 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x564691467dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0eb400640 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564691468310 0x564691469ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0eaa00640 1 -- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564691468310 msgr2=0x564691469ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564691468310 0x564691469ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0eaa00640 1 -- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 msgr2=0x564691467dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x564691467dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0eaa00640 1 -- 172.21.15.62:0/3195686867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56469107c310 con 0x56469146be60 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x564691467dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146be60 0x564691467790 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc023040 tx=0x7fe0dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ac5a2ecdf8725e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.156+0000 7fe0cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3195686867 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc02d020 con 0x56469146be60 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3195686867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0dc015640 con 0x56469146be60 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3195686867 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe0dc029410 con 0x56469146be60 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3195686867 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0dc037050 con 0x56469146be60 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 10 client.7089 fetch_fsmap learned FSMap version 55 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3195686867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5646911e91b0 con 0x56469146be60 2024-04-25T09:11:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3195686867 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc015640 con 0x56469146be60 2024-04-25T09:11:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3195686867 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0dc03a020 con 0x56469146be60 2024-04-25T09:11:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 10 client.7089 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:11:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 10 client.7089 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:11:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 20 client.7089 populate_metadata read hostname 'smithi062' 2024-04-25T09:11:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x564691484b60 0x564691484f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 4 client.7089 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.157+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3195686867 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646914060c0 con 0x564691484b60 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.158+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x564691484b60 0x564691484f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.158+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x564691484b60 0x564691484f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85349b746cfb457 server_cookie=825c704cfc322e85 in_seq=0 out_seq=0 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.158+0000 7fe0cfe00640 10 client.7089 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3195686867 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646914060c0 con 0x564691484b60 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0cfe00640 10 client.7089 handle_command_reply: tid=0 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 client.7089 shutdown 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3195686867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146be60 msgr2=0x564691467790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146be60 0x564691467790 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc023040 tx=0x7fe0dc004730 comp rx=0 tx=0).stop 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3195686867 shutdown_connections 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x564691484b60 0x564691484f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564691468310 0x564691469ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56469146f010 0x564691467dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3195686867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146be60 0x564691467790 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0cfe00640 0 client.7089 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:30.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.159+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3195686867 >> 172.21.15.62:0/3195686867 conn(0x56469146b540 msgr2=0x56469148cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.160+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3195686867 shutdown_connections 2024-04-25T09:11:30.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.160+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3195686867 wait complete. 2024-04-25T09:11:30.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.160+0000 7fe0f4a4a740 20 client.7089 trim_cache size 0 max 16384 2024-04-25T09:11:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.176+0000 7fe0f4a4a740 1 Processor -- start 2024-04-25T09:11:30.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.177+0000 7fe0f4a4a740 1 -- start start 2024-04-25T09:11:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.177+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x564691483740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.177+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x564691485c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.177+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x564691471cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.177+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1ffe70 con 0x56469146f010 2024-04-25T09:11:30.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.177+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1ff0a0 con 0x564691468310 2024-04-25T09:11:30.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.177+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec134290 con 0x56469146be60 2024-04-25T09:11:30.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x564691471cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x564691471cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47810/0 (socket says 172.21.15.62:47810) 2024-04-25T09:11:30.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eb400640 1 -- 172.21.15.62:0/3616976949 learned_addr learned my addr 172.21.15.62:0/3616976949 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x564691485c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x564691483740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eb400640 1 -- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 msgr2=0x564691485c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eb400640 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x564691485c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eb400640 1 -- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 msgr2=0x564691483740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eb400640 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x564691483740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eb400640 1 -- 172.21.15.62:0/3616976949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56469146bc90 con 0x56469146f010 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.178+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x564691483740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.179+0000 7fe0eb400640 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x564691471cb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009c70 tx=0x7fe0e4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fbb9b0fec9fce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.179+0000 7fe0daa00640 1 -- 172.21.15.62:0/3616976949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4019030 con 0x56469146f010 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.179+0000 7fe0daa00640 1 -- 172.21.15.62:0/3616976949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e4003990 con 0x56469146f010 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.179+0000 7fe0daa00640 1 -- 172.21.15.62:0/3616976949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4003d40 con 0x56469146f010 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.179+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3616976949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 msgr2=0x564691471cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.179+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x564691471cb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009c70 tx=0x7fe0e4007700 comp rx=0 tx=0).stop 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.180+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3616976949 shutdown_connections 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.180+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x564691471cb0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.180+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x564691485c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.180+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3616976949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x564691483740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.180+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3616976949 >> 172.21.15.62:0/3616976949 conn(0x56469146b540 msgr2=0x5646914675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.180+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3616976949 shutdown_connections 2024-04-25T09:11:30.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.180+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3616976949 wait complete. 2024-04-25T09:11:30.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.180+0000 7fe0f4a4a740 1 -- start start 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.181+0000 7fe0f4a4a740 1 Processor -- start 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x5646914f4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x564691471c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0f4a4a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x5646914721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec134890 con 0x56469146f010 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646914443d0 con 0x564691468310 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0f4a4a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x56469143a880 con 0x56469146be60 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0f4a4a740 10 client.0 _ll_register_callbacks cb 0x7fe0dc0094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x5646914721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x5646914721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47824/0 (socket says 172.21.15.62:47824) 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eb400640 1 -- 172.21.15.62:0/3552662298 learned_addr learned my addr 172.21.15.62:0/3552662298 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0e2000640 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x564691471c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x5646914f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eb400640 1 -- 172.21.15.62:0/3552662298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 msgr2=0x564691471c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eb400640 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x564691471c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eb400640 1 -- 172.21.15.62:0/3552662298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 msgr2=0x5646914f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eb400640 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x5646914f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.182+0000 7fe0eb400640 1 -- 172.21.15.62:0/3552662298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564691055490 con 0x56469146f010 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.183+0000 7fe0eb400640 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x5646914721a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4007c00 tx=0x7fe0e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7738b7287c206dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.183+0000 7fe0cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.183+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3552662298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4004810 con 0x56469146f010 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.183+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3552662298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe0e4028470 con 0x56469146f010 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.183+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3552662298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0e4003990 con 0x56469146f010 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.183+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3552662298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e4030050 con 0x56469146f010 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.183+0000 7fe0f4a4a740 10 client.7095 fetch_fsmap learned FSMap version 55 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.183+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3552662298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x564691444110 con 0x56469146f010 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3552662298 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e4003990 con 0x56469146f010 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3552662298 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0e4033020 con 0x56469146f010 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0f4a4a740 10 client.7095 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0f4a4a740 10 client.7095 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:11:30.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0f4a4a740 20 client.7095 populate_metadata read hostname 'smithi062' 2024-04-25T09:11:30.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x564691440050 0x564691470560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0f4a4a740 4 client.7095 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T09:11:30.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3552662298 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x564691469620 con 0x564691440050 2024-04-25T09:11:30.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.184+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x564691440050 0x564691470560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.185+0000 7fe0cfe00640 10 client.7095 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:30.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.185+0000 7fe0eaa00640 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x564691440050 0x564691470560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cff7c2a9e654b6c server_cookie=f7dd2ea3d4d137ca in_seq=0 out_seq=0 2024-04-25T09:11:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0cfe00640 1 -- 172.21.15.62:0/3552662298 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564691469620 con 0x564691440050 2024-04-25T09:11:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0cfe00640 10 client.7095 handle_command_reply: tid=0 2024-04-25T09:11:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 client.7095 shutdown 2024-04-25T09:11:30.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3552662298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 msgr2=0x5646914721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x5646914721a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4007c00 tx=0x7fe0e4004ab0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3552662298 shutdown_connections 2024-04-25T09:11:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x564691440050 0x564691470560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56469146f010 0x5646914721a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56469146be60 0x564691471c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 --2- 172.21.15.62:0/3552662298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564691468310 0x5646914f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0cfe00640 0 client.7095 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3552662298 >> 172.21.15.62:0/3552662298 conn(0x56469146b540 msgr2=0x564691485c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3552662298 shutdown_connections 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.193+0000 7fe0f4a4a740 1 -- 172.21.15.62:0/3552662298 wait complete. 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0f4a4a740 20 client.7095 trim_cache size 0 max 16384 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "c2ca1b76-2f2c-40e1-884b-2f9937544962", 2024-04-25T09:11:30.182 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:11:30.183 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:11:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 -- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0c806b4d0 msgr2=0x7fe0c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0c806b4d0 0x7fe0c806d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc006fd0 tx=0x7fe0dc008040 comp rx=0 tx=0).stop 2024-04-25T09:11:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 -- 172.21.15.62:0/1966649931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec15b520 msgr2=0x7fe0ec1362d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec15b520 0x7fe0ec1362d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4000c00 tx=0x7fe0d40046c0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 -- 172.21.15.62:0/1966649931 shutdown_connections 2024-04-25T09:11:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0c806b4d0 0x7fe0c806d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0ec15b520 0x7fe0ec1362d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0ec15eb90 0x7fe0ec0711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 --2- 172.21.15.62:0/1966649931 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0ec06c440 0x7fe0ec15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.194+0000 7fe0ebe00640 1 -- 172.21.15.62:0/1966649931 >> 172.21.15.62:0/1966649931 conn(0x7fe0ec066ad0 msgr2=0x7fe0ec145c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.195+0000 7fe0ebe00640 1 -- 172.21.15.62:0/1966649931 shutdown_connections 2024-04-25T09:11:30.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.195+0000 7fe0ebe00640 1 -- 172.21.15.62:0/1966649931 wait complete. 2024-04-25T09:11:30.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:11:30.200 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:11:30.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.471+0000 7f3342600640 1 Processor -- start 2024-04-25T09:11:30.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.471+0000 7f3342600640 1 -- start start 2024-04-25T09:11:30.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f3342600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f333c15dc10 0x7f333c0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f3342600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f333c0d20a0 0x7f333c0b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f3342600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 0x7f333c0b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f3342600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c0b81a0 con 0x7f333c0d20a0 2024-04-25T09:11:30.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f3342600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333c0b7590 con 0x7f333c15dc10 2024-04-25T09:11:30.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f3342600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0b7870 con 0x7f333c0b70d0 2024-04-25T09:11:30.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f333aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f333c0d20a0 0x7f333c0b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f333b400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f333c15dc10 0x7f333c0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f333aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f333c0d20a0 0x7f333c0b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47834/0 (socket says 172.21.15.62:47834) 2024-04-25T09:11:30.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.473+0000 7f333aa00640 1 -- 172.21.15.62:0/3852136494 learned_addr learned my addr 172.21.15.62:0/3852136494 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.474+0000 7f333aa00640 1 -- 172.21.15.62:0/3852136494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 msgr2=0x7f333c0b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.474+0000 7f333aa00640 1 --2- 172.21.15.62:0/3852136494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 0x7f333c0b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.474+0000 7f333aa00640 1 -- 172.21.15.62:0/3852136494 >> v2:172.21.15.121:3300/0 conn(0x7f333c15dc10 msgr2=0x7f333c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.474+0000 7f333aa00640 1 --2- 172.21.15.62:0/3852136494 >> v2:172.21.15.121:3300/0 conn(0x7f333c15dc10 0x7f333c0d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.474+0000 7f333aa00640 1 -- 172.21.15.62:0/3852136494 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c0b7b50 con 0x7f333c0d20a0 2024-04-25T09:11:30.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.475+0000 7f333aa00640 1 --2- 172.21.15.62:0/3852136494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c0d20a0 0x7f333c0b6b90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f333c08cd20 tx=0x7f332802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5115e7d8a5fa27da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.476+0000 7f333a000640 1 -- 172.21.15.62:0/3852136494 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328037070 con 0x7f333c0d20a0 2024-04-25T09:11:30.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.476+0000 7f333a000640 1 -- 172.21.15.62:0/3852136494 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3328032440 con 0x7f333c0d20a0 2024-04-25T09:11:30.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.476+0000 7f333a000640 1 -- 172.21.15.62:0/3852136494 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328032810 con 0x7f333c0d20a0 2024-04-25T09:11:30.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 -- 172.21.15.62:0/3852136494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c0d20a0 msgr2=0x7f333c0b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 --2- 172.21.15.62:0/3852136494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c0d20a0 0x7f333c0b6b90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f333c08cd20 tx=0x7f332802a270 comp rx=0 tx=0).stop 2024-04-25T09:11:30.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 -- 172.21.15.62:0/3852136494 shutdown_connections 2024-04-25T09:11:30.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 --2- 172.21.15.62:0/3852136494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 0x7f333c0b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 --2- 172.21.15.62:0/3852136494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c0d20a0 0x7f333c0b6b90 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 --2- 172.21.15.62:0/3852136494 >> v2:172.21.15.121:3300/0 conn(0x7f333c15dc10 0x7f333c0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 -- 172.21.15.62:0/3852136494 >> 172.21.15.62:0/3852136494 conn(0x7f333c06b000 msgr2=0x7f333c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 -- 172.21.15.62:0/3852136494 shutdown_connections 2024-04-25T09:11:30.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.477+0000 7f3342600640 1 -- 172.21.15.62:0/3852136494 wait complete. 2024-04-25T09:11:30.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.480+0000 7f3342600640 1 Processor -- start 2024-04-25T09:11:30.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.480+0000 7f3342600640 1 -- start start 2024-04-25T09:11:30.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.481+0000 7f3342600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 0x7f333c1eceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.481+0000 7f3342600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c15dc10 0x7f333c070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.481+0000 7f3342600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333c1faf30 0x7f333c06cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.481+0000 7f3342600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0b97b0 con 0x7f333c15dc10 2024-04-25T09:11:30.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.481+0000 7f3342600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c15dff0 con 0x7f333c1faf30 2024-04-25T09:11:30.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.481+0000 7f3342600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0b9930 con 0x7f333c0b70d0 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c15dc10 0x7f333c070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c15dc10 0x7f333c070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47844/0 (socket says 172.21.15.62:47844) 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333aa00640 1 -- 172.21.15.62:0/760286206 learned_addr learned my addr 172.21.15.62:0/760286206 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333b400640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 0x7f333c1eceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333be00640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333c1faf30 0x7f333c06cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333aa00640 1 -- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 msgr2=0x7f333c1eceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333aa00640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 0x7f333c1eceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333aa00640 1 -- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333c1faf30 msgr2=0x7f333c06cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333aa00640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333c1faf30 0x7f333c06cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333aa00640 1 -- 172.21.15.62:0/760286206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c1f9e90 con 0x7f333c15dc10 2024-04-25T09:11:30.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.482+0000 7f333be00640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333c1faf30 0x7f333c06cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:11:30.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.483+0000 7f333aa00640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c15dc10 0x7f333c070740 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3328000c00 tx=0x7f33280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.484+0000 7f3333400640 1 -- 172.21.15.62:0/760286206 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328041020 con 0x7f333c15dc10 2024-04-25T09:11:30.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.484+0000 7f3333400640 1 -- 172.21.15.62:0/760286206 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33280306c0 con 0x7f333c15dc10 2024-04-25T09:11:30.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.484+0000 7f3333400640 1 -- 172.21.15.62:0/760286206 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328037040 con 0x7f333c15dc10 2024-04-25T09:11:30.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.484+0000 7f3342600640 1 -- 172.21.15.62:0/760286206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c1fa170 con 0x7f333c15dc10 2024-04-25T09:11:30.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.484+0000 7f3342600640 1 -- 172.21.15.62:0/760286206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c1fa730 con 0x7f333c15dc10 2024-04-25T09:11:30.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.489+0000 7f3333400640 1 -- 172.21.15.62:0/760286206 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3328044020 con 0x7f333c15dc10 2024-04-25T09:11:30.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.490+0000 7f3345378740 1 Processor -- start 2024-04-25T09:11:30.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.490+0000 7f3345378740 1 -- start start 2024-04-25T09:11:30.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.491+0000 7f3333400640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f331c06b5a0 0x7f331c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.491+0000 7f3333400640 1 -- 172.21.15.62:0/760286206 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f33280aa940 con 0x7f333c15dc10 2024-04-25T09:11:30.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.492+0000 7f3345378740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 0x556aae418d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.492+0000 7f3345378740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae416f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.492+0000 7f3345378740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae441670 0x556aae441af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.492+0000 7f3345378740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aae37ac70 con 0x556aae453280 2024-04-25T09:11:30.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.492+0000 7f3345378740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aae323cf0 con 0x556aae441670 2024-04-25T09:11:30.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.492+0000 7f3345378740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aae376d10 con 0x556aae416bb0 2024-04-25T09:11:30.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae416f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae416f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47848/0 (socket says 172.21.15.62:47848) 2024-04-25T09:11:30.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333aa00640 1 -- 172.21.15.62:0/898689283 learned_addr learned my addr 172.21.15.62:0/898689283 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333b400640 1 --2- 172.21.15.62:0/898689283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 0x556aae418d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333aa00640 1 -- 172.21.15.62:0/898689283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 msgr2=0x556aae418d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333aa00640 1 --2- 172.21.15.62:0/898689283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 0x556aae418d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333aa00640 1 -- 172.21.15.62:0/898689283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae441670 msgr2=0x556aae441af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333aa00640 1 --2- 172.21.15.62:0/898689283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae441670 0x556aae441af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333be00640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f331c06b5a0 0x7f331c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.493+0000 7f333aa00640 1 -- 172.21.15.62:0/898689283 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aae1597d0 con 0x556aae453280 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.494+0000 7f333aa00640 1 --2- 172.21.15.62:0/898689283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae416f90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f33280a8de0 tx=0x7f3328078b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850b18e7a2f5fd6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.494+0000 7f3327e00640 1 -- 172.21.15.62:0/898689283 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332807d240 con 0x556aae453280 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.494+0000 7f3327e00640 1 -- 172.21.15.62:0/898689283 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3328081800 con 0x556aae453280 2024-04-25T09:11:30.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.494+0000 7f3327e00640 1 -- 172.21.15.62:0/898689283 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328030a90 con 0x556aae453280 2024-04-25T09:11:30.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.495+0000 7f333be00640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f331c06b5a0 0x7f331c06da60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x556aae45d770 tx=0x7f33340073e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.495+0000 7f3345378740 1 -- 172.21.15.62:0/898689283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 msgr2=0x556aae416f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.495+0000 7f3345378740 1 --2- 172.21.15.62:0/898689283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae416f90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f33280a8de0 tx=0x7f3328078b10 comp rx=0 tx=0).stop 2024-04-25T09:11:30.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.496+0000 7f3345378740 1 -- 172.21.15.62:0/898689283 shutdown_connections 2024-04-25T09:11:30.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.496+0000 7f3345378740 1 --2- 172.21.15.62:0/898689283 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae416f90 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.496+0000 7f3345378740 1 --2- 172.21.15.62:0/898689283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae441670 0x556aae441af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.496+0000 7f3345378740 1 --2- 172.21.15.62:0/898689283 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 0x556aae418d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.496+0000 7f3345378740 1 -- 172.21.15.62:0/898689283 >> 172.21.15.62:0/898689283 conn(0x556aae3dea30 msgr2=0x556aae45c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.496+0000 7f3345378740 1 -- 172.21.15.62:0/898689283 shutdown_connections 2024-04-25T09:11:30.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.496+0000 7f3345378740 1 -- 172.21.15.62:0/898689283 wait complete. 2024-04-25T09:11:30.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.497+0000 7f3345378740 1 -- start start 2024-04-25T09:11:30.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.497+0000 7f3345378740 1 Processor -- start 2024-04-25T09:11:30.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.498+0000 7f3345378740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 0x556aae433f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.498+0000 7f3345378740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae441670 0x556aae434480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.498+0000 7f3345378740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae4349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.498+0000 7f3345378740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aae323cf0 con 0x556aae441670 2024-04-25T09:11:30.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.498+0000 7f3345378740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aae3193a0 con 0x556aae453280 2024-04-25T09:11:30.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.498+0000 7f3345378740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aae37ac70 con 0x556aae416bb0 2024-04-25T09:11:30.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.498+0000 7f3345378740 10 client.0 _ll_register_callbacks cb 0x556aae336b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.499+0000 7f333be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae4349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.499+0000 7f333be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae4349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:40358/0 (socket says 172.21.15.62:40358) 2024-04-25T09:11:30.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.499+0000 7f333be00640 1 -- 172.21.15.62:0/2029803998 learned_addr learned my addr 172.21.15.62:0/2029803998 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.499+0000 7f333b400640 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 0x556aae433f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.499+0000 7f333aa00640 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae441670 0x556aae434480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.500+0000 7f333be00640 1 -- 172.21.15.62:0/2029803998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 msgr2=0x556aae433f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.500+0000 7f333be00640 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 0x556aae433f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.500+0000 7f333be00640 1 -- 172.21.15.62:0/2029803998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae441670 msgr2=0x556aae434480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.500+0000 7f333be00640 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae441670 0x556aae434480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.500+0000 7f333be00640 1 -- 172.21.15.62:0/2029803998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556aae08f3f0 con 0x556aae453280 2024-04-25T09:11:30.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.500+0000 7f333be00640 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae4349c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f333400fe20 tx=0x7f3334004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d63692ab58d7944b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.500+0000 7f3324c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-04-25T09:11:30.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.501+0000 7f3324c00640 1 -- 172.21.15.62:0/2029803998 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334020020 con 0x556aae453280 2024-04-25T09:11:30.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.501+0000 7f3345378740 1 -- 172.21.15.62:0/2029803998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3328081800 con 0x556aae453280 2024-04-25T09:11:30.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.501+0000 7f3324c00640 1 -- 172.21.15.62:0/2029803998 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f333400e920 con 0x556aae453280 2024-04-25T09:11:30.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.502+0000 7f3324c00640 1 -- 172.21.15.62:0/2029803998 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f333402a050 con 0x556aae453280 2024-04-25T09:11:30.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.502+0000 7f3345378740 10 client.5860 fetch_fsmap learned FSMap version 55 2024-04-25T09:11:30.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.502+0000 7f3345378740 1 -- 172.21.15.62:0/2029803998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x556aae04f0f0 con 0x556aae453280 2024-04-25T09:11:30.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.503+0000 7f3324c00640 1 -- 172.21.15.62:0/2029803998 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3328081800 con 0x556aae453280 2024-04-25T09:11:30.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.503+0000 7f3324c00640 1 -- 172.21.15.62:0/2029803998 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f333402d020 con 0x556aae453280 2024-04-25T09:11:30.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.503+0000 7f3345378740 10 client.5860 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:11:30.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.503+0000 7f3345378740 10 client.5860 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:11:30.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.503+0000 7f3345378740 20 client.5860 populate_metadata read hostname 'smithi062' 2024-04-25T09:11:30.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.503+0000 7f3345378740 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x556aae433260 0x556aae437de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.503+0000 7f3345378740 4 client.5860 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:11:30.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.503+0000 7f3345378740 1 -- 172.21.15.62:0/2029803998 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556aae3881b0 con 0x556aae433260 2024-04-25T09:11:30.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.504+0000 7f333b400640 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x556aae433260 0x556aae437de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.505+0000 7f333b400640 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x556aae433260 0x556aae437de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3cae6976ce7d9 server_cookie=dc4a7f954cd5d42b in_seq=0 out_seq=0 2024-04-25T09:11:30.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.505+0000 7f3324c00640 10 client.5860 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:30.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.506+0000 7f3324c00640 1 -- 172.21.15.62:0/2029803998 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556aae3881b0 con 0x556aae433260 2024-04-25T09:11:30.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.506+0000 7f3324c00640 10 client.5860 handle_command_reply: tid=0 2024-04-25T09:11:30.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.506+0000 7f3345378740 1 client.5860 shutdown 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 -- 172.21.15.62:0/2029803998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 msgr2=0x556aae4349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae4349c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f333400fe20 tx=0x7f3334004b50 comp rx=0 tx=0).stop 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 -- 172.21.15.62:0/2029803998 shutdown_connections 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x556aae433260 0x556aae437de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae441670 0x556aae434480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae4349c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 --2- 172.21.15.62:0/2029803998 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae416bb0 0x556aae433f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3324c00640 0 client.5860 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 -- 172.21.15.62:0/2029803998 >> 172.21.15.62:0/2029803998 conn(0x556aae3dea30 msgr2=0x556aae45d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 -- 172.21.15.62:0/2029803998 shutdown_connections 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 1 -- 172.21.15.62:0/2029803998 wait complete. 2024-04-25T09:11:30.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.507+0000 7f3345378740 20 client.5860 trim_cache size 0 max 16384 2024-04-25T09:11:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.532+0000 7f3345378740 1 Processor -- start 2024-04-25T09:11:30.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.533+0000 7f3345378740 1 -- start start 2024-04-25T09:11:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.534+0000 7f3345378740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae416bb0 0x556aae4437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.534+0000 7f3345378740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae45fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.534+0000 7f3345378740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae45fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.534+0000 7f3345378740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c204530 con 0x556aae416bb0 2024-04-25T09:11:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.534+0000 7f3345378740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c203740 con 0x556aae453280 2024-04-25T09:11:30.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.534+0000 7f3345378740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333c1f8ce0 con 0x556aae441670 2024-04-25T09:11:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333aa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae45fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333aa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae45fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:60602/0 (socket says 172.21.15.62:60602) 2024-04-25T09:11:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333aa00640 1 -- 172.21.15.62:0/3510578742 learned_addr learned my addr 172.21.15.62:0/3510578742 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333b400640 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae416bb0 0x556aae4437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333be00640 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae45fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333aa00640 1 -- 172.21.15.62:0/3510578742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 msgr2=0x556aae45fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333aa00640 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae45fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333aa00640 1 -- 172.21.15.62:0/3510578742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae416bb0 msgr2=0x556aae4437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333aa00640 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae416bb0 0x556aae4437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333aa00640 1 -- 172.21.15.62:0/3510578742 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aae442200 con 0x556aae441670 2024-04-25T09:11:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.535+0000 7f333b400640 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae416bb0 0x556aae4437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:11:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.536+0000 7f333aa00640 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae45fab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f332807d1b0 tx=0x7f3328078b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a79a0f90c8822a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.537+0000 7f3327400640 1 -- 172.21.15.62:0/3510578742 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332807d240 con 0x556aae441670 2024-04-25T09:11:30.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.537+0000 7f3327400640 1 -- 172.21.15.62:0/3510578742 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3328081800 con 0x556aae441670 2024-04-25T09:11:30.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.537+0000 7f3327400640 1 -- 172.21.15.62:0/3510578742 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328075760 con 0x556aae441670 2024-04-25T09:11:30.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 -- 172.21.15.62:0/3510578742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 msgr2=0x556aae45fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae45fab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f332807d1b0 tx=0x7f3328078b80 comp rx=0 tx=0).stop 2024-04-25T09:11:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 -- 172.21.15.62:0/3510578742 shutdown_connections 2024-04-25T09:11:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae453280 0x556aae45fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae45fab0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 --2- 172.21.15.62:0/3510578742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae416bb0 0x556aae4437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 -- 172.21.15.62:0/3510578742 >> 172.21.15.62:0/3510578742 conn(0x556aae3dea30 msgr2=0x556aae45d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 -- 172.21.15.62:0/3510578742 shutdown_connections 2024-04-25T09:11:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.538+0000 7f3345378740 1 -- 172.21.15.62:0/3510578742 wait complete. 2024-04-25T09:11:30.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.539+0000 7f3345378740 1 -- start start 2024-04-25T09:11:30.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.539+0000 7f3345378740 1 Processor -- start 2024-04-25T09:11:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f3345378740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae416bb0 0x556aae415370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f3345378740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae4158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f3345378740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae45fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f3345378740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c14d4f0 con 0x556aae453280 2024-04-25T09:11:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f3345378740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aae4381c0 con 0x556aae416bb0 2024-04-25T09:11:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f3345378740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aae416530 con 0x556aae441670 2024-04-25T09:11:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f3345378740 10 client.0 _ll_register_callbacks cb 0x7f333400fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:11:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f333be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae45fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f333be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae45fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47874/0 (socket says 172.21.15.62:47874) 2024-04-25T09:11:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f333be00640 1 -- 172.21.15.62:0/2471124844 learned_addr learned my addr 172.21.15.62:0/2471124844 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:11:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.541+0000 7f333b400640 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae416bb0 0x556aae415370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.542+0000 7f333aa00640 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae4158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.542+0000 7f333be00640 1 -- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 msgr2=0x556aae4158b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.542+0000 7f333be00640 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae4158b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.542+0000 7f333be00640 1 -- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae416bb0 msgr2=0x556aae415370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:11:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.542+0000 7f333be00640 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae416bb0 0x556aae415370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.542+0000 7f333be00640 1 -- 172.21.15.62:0/2471124844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556aae43f910 con 0x556aae453280 2024-04-25T09:11:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.543+0000 7f333aa00640 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae4158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:11:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.543+0000 7f333be00640 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae45fa20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3334004b90 tx=0x7f3334011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0974fa71982d150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:11:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.543+0000 7f3324c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:11:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.543+0000 7f3324c00640 1 -- 172.21.15.62:0/2471124844 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333403c060 con 0x556aae453280 2024-04-25T09:11:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.543+0000 7f3324c00640 1 -- 172.21.15.62:0/2471124844 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f333400cdd0 con 0x556aae453280 2024-04-25T09:11:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.543+0000 7f3345378740 1 -- 172.21.15.62:0/2471124844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3328081800 con 0x556aae453280 2024-04-25T09:11:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.544+0000 7f3324c00640 1 -- 172.21.15.62:0/2471124844 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3334042050 con 0x556aae453280 2024-04-25T09:11:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.544+0000 7f3345378740 10 client.7113 fetch_fsmap learned FSMap version 55 2024-04-25T09:11:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.544+0000 7f3345378740 1 -- 172.21.15.62:0/2471124844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x556aae416300 con 0x556aae453280 2024-04-25T09:11:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f3324c00640 1 -- 172.21.15.62:0/2471124844 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3328081800 con 0x556aae453280 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f3324c00640 1 -- 172.21.15.62:0/2471124844 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3334045020 con 0x556aae453280 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f3345378740 10 client.7113 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f3345378740 10 client.7113 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f3345378740 20 client.7113 populate_metadata read hostname 'smithi062' 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f3345378740 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x556aae432730 0x556aae45ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f3345378740 4 client.7113 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f3345378740 1 -- 172.21.15.62:0/2471124844 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556aae382500 con 0x556aae432730 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.545+0000 7f333b400640 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x556aae432730 0x556aae45ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.546+0000 7f333b400640 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x556aae432730 0x556aae45ff60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f10509b1bf6eacd1 server_cookie=d6c773a0a5d63c57 in_seq=0 out_seq=0 2024-04-25T09:11:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.546+0000 7f3324c00640 10 client.7113 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.546+0000 7f3324c00640 1 -- 172.21.15.62:0/2471124844 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+264 (crc 0 0 0) 0x556aae382500 con 0x556aae432730 2024-04-25T09:11:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.546+0000 7f3324c00640 10 client.7113 handle_command_reply: tid=0 2024-04-25T09:11:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 client.7113 shutdown 2024-04-25T09:11:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 -- 172.21.15.62:0/2471124844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 msgr2=0x556aae45fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae45fa20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3334004b90 tx=0x7f3334011000 comp rx=0 tx=0).stop 2024-04-25T09:11:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 -- 172.21.15.62:0/2471124844 shutdown_connections 2024-04-25T09:11:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x556aae432730 0x556aae45ff60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x556aae453280 0x556aae45fa20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x556aae441670 0x556aae4158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 --2- 172.21.15.62:0/2471124844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556aae416bb0 0x556aae415370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3324c00640 0 client.7113 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 -- 172.21.15.62:0/2471124844 >> 172.21.15.62:0/2471124844 conn(0x556aae3dea30 msgr2=0x556aae438440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 -- 172.21.15.62:0/2471124844 shutdown_connections 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.547+0000 7f3345378740 1 -- 172.21.15.62:0/2471124844 wait complete. 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.548+0000 7f3345378740 20 client.7113 trim_cache size 0 max 16384 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "scrub active (10 inodes in the stack)", 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stdout: "c2ca1b76-2f2c-40e1-884b-2f9937544962": { 2024-04-25T09:11:30.536 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "~mds0", 2024-04-25T09:11:30.537 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "c2ca1b76-2f2c-40e1-884b-2f9937544962", 2024-04-25T09:11:30.537 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive" 2024-04-25T09:11:30.537 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:30.537 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:11:30.537 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 -- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f331c06b5a0 msgr2=0x7f331c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f331c06b5a0 0x7f331c06da60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x556aae45d770 tx=0x7f33340073e0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 -- 172.21.15.62:0/760286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c15dc10 msgr2=0x7f333c070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c15dc10 0x7f333c070740 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3328000c00 tx=0x7f33280046c0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 -- 172.21.15.62:0/760286206 shutdown_connections 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f331c06b5a0 0x7f331c06da60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f333c15dc10 0x7f333c070740 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f333c1faf30 0x7f333c06cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 --2- 172.21.15.62:0/760286206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f333c0b70d0 0x7f333c1eceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 -- 172.21.15.62:0/760286206 >> 172.21.15.62:0/760286206 conn(0x7f333c06b000 msgr2=0x7f333c0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 -- 172.21.15.62:0/760286206 shutdown_connections 2024-04-25T09:11:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:11:30.549+0000 7f3342600640 1 -- 172.21.15.62:0/760286206 wait complete. 2024-04-25T09:11:30.553 INFO:tasks.cephfs.filesystem:scrub status for tag:c2ca1b76-2f2c-40e1-884b-2f9937544962 - {'path': '~mds0', 'tag': 'c2ca1b76-2f2c-40e1-884b-2f9937544962', 'options': 'recursive'} 2024-04-25T09:11:44.036 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:11:44.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:00.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:00.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:12:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.792+0000 7fc7bee00640 1 Processor -- start 2024-04-25T09:12:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.792+0000 7fc7bee00640 1 -- start start 2024-04-25T09:12:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.794+0000 7fc7bee00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc7b815dc10 0x7fc7b80d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.794+0000 7fc7bee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b80b97b0 0x7fc7b80b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.794+0000 7fc7bee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b80b7590 con 0x7fc7b80d20a0 2024-04-25T09:12:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.794+0000 7fc7bee00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b80b81a0 con 0x7fc7b815dc10 2024-04-25T09:12:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.794+0000 7fc7bee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b80b7870 con 0x7fc7b80b97b0 2024-04-25T09:12:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.794+0000 7fc7b7400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc7b80d20a0 0x7fc7b80b6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38562/0 (socket says 172.21.15.62:38562) 2024-04-25T09:12:00.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.794+0000 7fc7b7400640 1 -- 172.21.15.62:0/721916192 learned_addr learned my addr 172.21.15.62:0/721916192 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.795+0000 7fc7bce00640 1 --2- 172.21.15.62:0/721916192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b80b97b0 0x7fc7b80b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.795+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/721916192 >> v2:172.21.15.121:3300/0 conn(0x7fc7b815dc10 0x7fc7b80d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.795+0000 7fc7b7e00640 1 -- 172.21.15.62:0/721916192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b80b97b0 msgr2=0x7fc7b80b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.795+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/721916192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b80b97b0 0x7fc7b80b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.795+0000 7fc7b7e00640 1 -- 172.21.15.62:0/721916192 >> v1:172.21.15.62:6789/0 conn(0x7fc7b80d20a0 legacy=0x7fc7b80b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.795+0000 7fc7b7e00640 1 -- 172.21.15.62:0/721916192 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80b7b50 con 0x7fc7b815dc10 2024-04-25T09:12:00.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.797+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/721916192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b815dc10 0x7fc7b80d1b60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80097a0 tx=0x7fc7a802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3fec9294898d642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.800+0000 7fc7b6a00640 1 -- 172.21.15.62:0/721916192 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8037070 con 0x7fc7b815dc10 2024-04-25T09:12:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.800+0000 7fc7b6a00640 1 -- 172.21.15.62:0/721916192 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8032440 con 0x7fc7b815dc10 2024-04-25T09:12:00.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.800+0000 7fc7b6a00640 1 -- 172.21.15.62:0/721916192 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8032810 con 0x7fc7b815dc10 2024-04-25T09:12:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.801+0000 7fc7bee00640 1 -- 172.21.15.62:0/721916192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b815dc10 msgr2=0x7fc7b80d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.801+0000 7fc7bee00640 1 --2- 172.21.15.62:0/721916192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b815dc10 0x7fc7b80d1b60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80097a0 tx=0x7fc7a802a230 comp rx=0 tx=0).stop 2024-04-25T09:12:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.801+0000 7fc7bee00640 1 -- 172.21.15.62:0/721916192 shutdown_connections 2024-04-25T09:12:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.801+0000 7fc7bee00640 1 --2- 172.21.15.62:0/721916192 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b80b97b0 0x7fc7b80b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.801+0000 7fc7bee00640 1 --2- 172.21.15.62:0/721916192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b815dc10 0x7fc7b80d1b60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.801+0000 7fc7bee00640 1 -- 172.21.15.62:0/721916192 >> 172.21.15.62:0/721916192 conn(0x7fc7b806b000 msgr2=0x7fc7b806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.801+0000 7fc7bee00640 1 -- 172.21.15.62:0/721916192 shutdown_connections 2024-04-25T09:12:00.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.801+0000 7fc7bee00640 1 -- 172.21.15.62:0/721916192 wait complete. 2024-04-25T09:12:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.804+0000 7fc7bee00640 1 Processor -- start 2024-04-25T09:12:00.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.804+0000 7fc7bee00640 1 -- start start 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b80b97b0 0x7fc7b80cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b815dc10 0x7fc7b814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8148620 0x7fc7b8148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b815dff0 con 0x7fc7b8148620 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b80b72a0 con 0x7fc7b80b97b0 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b80b8b70 con 0x7fc7b815dc10 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8148620 0x7fc7b8148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8148620 0x7fc7b8148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37196/0 (socket says 172.21.15.62:37196) 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.805+0000 7fc7bce00640 1 -- 172.21.15.62:0/2835407576 learned_addr learned my addr 172.21.15.62:0/2835407576 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b80b97b0 0x7fc7b80cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7b7400640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b815dc10 0x7fc7b814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7bce00640 1 -- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b815dc10 msgr2=0x7fc7b814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7bce00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b815dc10 0x7fc7b814ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7bce00640 1 -- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b80b97b0 msgr2=0x7fc7b80cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7bce00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b80b97b0 0x7fc7b80cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7bce00640 1 -- 172.21.15.62:0/2835407576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b815bb20 con 0x7fc7b8148620 2024-04-25T09:12:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b80b97b0 0x7fc7b80cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.806+0000 7fc7bce00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8148620 0x7fc7b8148ab0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000efc0 tx=0x7fc7b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.807+0000 7fc7afe00640 1 -- 172.21.15.62:0/2835407576 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0018020 con 0x7fc7b8148620 2024-04-25T09:12:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.807+0000 7fc7afe00640 1 -- 172.21.15.62:0/2835407576 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b0014480 con 0x7fc7b8148620 2024-04-25T09:12:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.807+0000 7fc7bee00640 1 -- 172.21.15.62:0/2835407576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b815bdc0 con 0x7fc7b8148620 2024-04-25T09:12:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.807+0000 7fc7afe00640 1 -- 172.21.15.62:0/2835407576 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0014870 con 0x7fc7b8148620 2024-04-25T09:12:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.807+0000 7fc7bee00640 1 -- 172.21.15.62:0/2835407576 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b8149880 con 0x7fc7b8148620 2024-04-25T09:12:00.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.809+0000 7fc7afe00640 1 -- 172.21.15.62:0/2835407576 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7b0005d10 con 0x7fc7b8148620 2024-04-25T09:12:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.810+0000 7fc7afe00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc79806b4d0 0x7fc79806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.810+0000 7fc7afe00640 1 -- 172.21.15.62:0/2835407576 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc7b008ad50 con 0x7fc7b8148620 2024-04-25T09:12:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.811+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc79806b4d0 0x7fc79806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.811+0000 7fc7c1daa740 1 Processor -- start 2024-04-25T09:12:00.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.811+0000 7fc7c1daa740 1 -- start start 2024-04-25T09:12:00.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.812+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc79806b4d0 0x7fc79806d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8034000 tx=0x7fc7a802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.812+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe4f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.812+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe4f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.812+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 0x561ebe51eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.812+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ebe457c70 con 0x561ebe4f3bb0 2024-04-25T09:12:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.812+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ebe400cf0 con 0x561ebe530280 2024-04-25T09:12:00.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.812+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ebe453d10 con 0x561ebe51e670 2024-04-25T09:12:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe4f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe4f5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37202/0 (socket says 172.21.15.62:37202) 2024-04-25T09:12:00.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7e00640 1 -- 172.21.15.62:0/4177714443 learned_addr learned my addr 172.21.15.62:0/4177714443 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7400640 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe4f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7e00640 1 -- 172.21.15.62:0/4177714443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 msgr2=0x561ebe51eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 0x561ebe51eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7e00640 1 -- 172.21.15.62:0/4177714443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 msgr2=0x561ebe4f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe4f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7e00640 1 -- 172.21.15.62:0/4177714443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ebe2367d0 con 0x561ebe4f3bb0 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.813+0000 7fc7b7400640 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe4f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.814+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe4f5d10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009890 tx=0x7fc7a802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9e3cb834d4901 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.814+0000 7fc7acc00640 1 -- 172.21.15.62:0/4177714443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a804e070 con 0x561ebe4f3bb0 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.814+0000 7fc7acc00640 1 -- 172.21.15.62:0/4177714443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8049470 con 0x561ebe4f3bb0 2024-04-25T09:12:00.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.814+0000 7fc7acc00640 1 -- 172.21.15.62:0/4177714443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8037040 con 0x561ebe4f3bb0 2024-04-25T09:12:00.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 -- 172.21.15.62:0/4177714443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 msgr2=0x561ebe4f5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe4f5d10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009890 tx=0x7fc7a802c000 comp rx=0 tx=0).stop 2024-04-25T09:12:00.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 -- 172.21.15.62:0/4177714443 shutdown_connections 2024-04-25T09:12:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 0x561ebe51eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe4f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/4177714443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe4f5d10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 -- 172.21.15.62:0/4177714443 >> 172.21.15.62:0/4177714443 conn(0x561ebe4bba30 msgr2=0x561ebe539670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 -- 172.21.15.62:0/4177714443 shutdown_connections 2024-04-25T09:12:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.815+0000 7fc7c1daa740 1 -- 172.21.15.62:0/4177714443 wait complete. 2024-04-25T09:12:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.816+0000 7fc7c1daa740 1 -- start start 2024-04-25T09:12:00.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.816+0000 7fc7c1daa740 1 Processor -- start 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 0x561ebe51fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe51e670 0x561ebe5332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe530280 0x561ebe5095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ebe3f63a0 con 0x561ebe530280 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ebe51c1c0 con 0x561ebe51e670 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ebe520140 con 0x561ebe4f3bb0 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7c1daa740 10 client.0 _ll_register_callbacks cb 0x561ebe413b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7bce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe530280 0x561ebe5095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7bce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe530280 0x561ebe5095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37204/0 (socket says 172.21.15.62:37204) 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7bce00640 1 -- 172.21.15.62:0/3226638725 learned_addr learned my addr 172.21.15.62:0/3226638725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:00.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 0x561ebe51fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7b7400640 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe51e670 0x561ebe5332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7bce00640 1 -- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 msgr2=0x561ebe51fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7bce00640 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 0x561ebe51fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.817+0000 7fc7bce00640 1 -- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe51e670 msgr2=0x561ebe5332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7bce00640 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe51e670 0x561ebe5332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7bce00640 1 -- 172.21.15.62:0/3226638725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ebe16c3f0 con 0x561ebe530280 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7b7400640 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe51e670 0x561ebe5332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7bce00640 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe530280 0x561ebe5095e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c990 tx=0x7fc7b0059cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60944899cc39848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7a1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7a1600640 1 -- 172.21.15.62:0/3226638725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0087060 con 0x561ebe530280 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7a1600640 1 -- 172.21.15.62:0/3226638725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc7b0082b80 con 0x561ebe530280 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7c1daa740 1 -- 172.21.15.62:0/3226638725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7a8049470 con 0x561ebe530280 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7a1600640 1 -- 172.21.15.62:0/3226638725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7b005d370 con 0x561ebe530280 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7c1daa740 10 client.7122 fetch_fsmap learned FSMap version 55 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.818+0000 7fc7c1daa740 1 -- 172.21.15.62:0/3226638725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x561ebe12c0f0 con 0x561ebe530280 2024-04-25T09:12:00.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7a1600640 1 -- 172.21.15.62:0/3226638725 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a8049470 con 0x561ebe530280 2024-04-25T09:12:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7a1600640 1 -- 172.21.15.62:0/3226638725 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc7b00588c0 con 0x561ebe530280 2024-04-25T09:12:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7c1daa740 10 client.7122 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:12:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7c1daa740 10 client.7122 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:12:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7c1daa740 20 client.7122 populate_metadata read hostname 'smithi062' 2024-04-25T09:12:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x561ebe50a360 0x561ebe50a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7c1daa740 4 client.7122 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:12:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7c1daa740 1 -- 172.21.15.62:0/3226638725 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ebe4651b0 con 0x561ebe50a360 2024-04-25T09:12:00.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.819+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x561ebe50a360 0x561ebe50a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.820+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x561ebe50a360 0x561ebe50a770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=754f665b0a55e7e1 server_cookie=1eb859a2372871a3 in_seq=0 out_seq=0 2024-04-25T09:12:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.820+0000 7fc7a1600640 10 client.7122 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:12:00.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.820+0000 7fc7a1600640 1 -- 172.21.15.62:0/3226638725 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ebe4651b0 con 0x561ebe50a360 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.820+0000 7fc7a1600640 10 client.7122 handle_command_reply: tid=0 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.820+0000 7fc7c1daa740 1 client.7122 shutdown 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 -- 172.21.15.62:0/3226638725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe530280 msgr2=0x561ebe5095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe530280 0x561ebe5095e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c990 tx=0x7fc7b0059cb0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 -- 172.21.15.62:0/3226638725 shutdown_connections 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x561ebe50a360 0x561ebe50a770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe530280 0x561ebe5095e0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe51e670 0x561ebe5332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/3226638725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 0x561ebe51fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 -- 172.21.15.62:0/3226638725 >> 172.21.15.62:0/3226638725 conn(0x561ebe4bba30 msgr2=0x561ebe51b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7a1600640 0 client.7122 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 -- 172.21.15.62:0/3226638725 shutdown_connections 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 1 -- 172.21.15.62:0/3226638725 wait complete. 2024-04-25T09:12:00.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.821+0000 7fc7c1daa740 20 client.7122 trim_cache size 0 max 16384 2024-04-25T09:12:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.838+0000 7fc7c1daa740 1 Processor -- start 2024-04-25T09:12:00.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.838+0000 7fc7c1daa740 1 -- start start 2024-04-25T09:12:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 0x561ebe50c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe51e670 0x561ebe50cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe54b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b8200180 con 0x561ebe51e670 2024-04-25T09:12:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b81ff340 con 0x561ebe530280 2024-04-25T09:12:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b814bef0 con 0x561ebe4f3bb0 2024-04-25T09:12:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7b7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe51e670 0x561ebe50cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7b7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe51e670 0x561ebe50cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37216/0 (socket says 172.21.15.62:37216) 2024-04-25T09:12:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7b7400640 1 -- 172.21.15.62:0/2836634909 learned_addr learned my addr 172.21.15.62:0/2836634909 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.839+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 0x561ebe50c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.840+0000 7fc7bce00640 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe54b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.840+0000 7fc7b7400640 1 -- 172.21.15.62:0/2836634909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 msgr2=0x561ebe50c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.840+0000 7fc7b7400640 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 0x561ebe50c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.840+0000 7fc7b7400640 1 -- 172.21.15.62:0/2836634909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 msgr2=0x561ebe54b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.840+0000 7fc7b7400640 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe54b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.840+0000 7fc7b7400640 1 -- 172.21.15.62:0/2836634909 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ebe50adb0 con 0x561ebe51e670 2024-04-25T09:12:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.840+0000 7fc7b7400640 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe51e670 0x561ebe50cac0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc7b81f6f20 tx=0x7fc7a4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851cbe41afacd784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7a3e00640 1 -- 172.21.15.62:0/2836634909 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4005cc0 con 0x561ebe51e670 2024-04-25T09:12:00.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7a3e00640 1 -- 172.21.15.62:0/2836634909 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a4019070 con 0x561ebe51e670 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7a3e00640 1 -- 172.21.15.62:0/2836634909 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4007310 con 0x561ebe51e670 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2836634909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe51e670 msgr2=0x561ebe50cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe51e670 0x561ebe50cac0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc7b81f6f20 tx=0x7fc7a4001b10 comp rx=0 tx=0).stop 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2836634909 shutdown_connections 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe51e670 0x561ebe50cac0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe54b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2836634909 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe4f3bb0 0x561ebe50c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2836634909 >> 172.21.15.62:0/2836634909 conn(0x561ebe4bba30 msgr2=0x561ebe4f4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.841+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2836634909 shutdown_connections 2024-04-25T09:12:00.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.842+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2836634909 wait complete. 2024-04-25T09:12:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.842+0000 7fc7c1daa740 1 -- start start 2024-04-25T09:12:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.842+0000 7fc7c1daa740 1 Processor -- start 2024-04-25T09:12:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.843+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe50c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.843+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 0x561ebe51eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.843+0000 7fc7c1daa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe51eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.843+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b814c490 con 0x561ebe4f3bb0 2024-04-25T09:12:00.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.843+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ebe539ac0 con 0x561ebe530280 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.843+0000 7fc7c1daa740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ebe55ae90 con 0x561ebe51e670 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.843+0000 7fc7c1daa740 10 client.0 _ll_register_callbacks cb 0x7fc7b000cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe50c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe50c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37230/0 (socket says 172.21.15.62:37230) 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 -- 172.21.15.62:0/2786059156 learned_addr learned my addr 172.21.15.62:0/2786059156 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7bce00640 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe51eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7400640 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 0x561ebe51eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 -- 172.21.15.62:0/2786059156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 msgr2=0x561ebe51eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 0x561ebe51eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 -- 172.21.15.62:0/2786059156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 msgr2=0x561ebe51eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe51eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 -- 172.21.15.62:0/2786059156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ebe50cec0 con 0x561ebe4f3bb0 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7b7e00640 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe50c980 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8035040 tx=0x7fc7a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d143eaa83ec1ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.844+0000 7fc7a1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7a1600640 1 -- 172.21.15.62:0/2786059156 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8058060 con 0x561ebe4f3bb0 2024-04-25T09:12:00.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7a1600640 1 -- 172.21.15.62:0/2786059156 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc7a80533a0 con 0x561ebe4f3bb0 2024-04-25T09:12:00.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2786059156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7a4019070 con 0x561ebe4f3bb0 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7a1600640 1 -- 172.21.15.62:0/2786059156 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a805e050 con 0x561ebe4f3bb0 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7c1daa740 10 client.7128 fetch_fsmap learned FSMap version 55 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2786059156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x561ebe4f5750 con 0x561ebe4f3bb0 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7a1600640 1 -- 172.21.15.62:0/2786059156 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a4019070 con 0x561ebe4f3bb0 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7a1600640 1 -- 172.21.15.62:0/2786059156 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc7a8061020 con 0x561ebe4f3bb0 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7c1daa740 10 client.7128 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7c1daa740 10 client.7128 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7c1daa740 20 client.7128 populate_metadata read hostname 'smithi062' 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.845+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x561ebe51f530 0x561ebe5a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.846+0000 7fc7c1daa740 4 client.7128 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.846+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2786059156 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561ebe45f500 con 0x561ebe51f530 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.846+0000 7fc7b7400640 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x561ebe51f530 0x561ebe5a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.846+0000 7fc7a1600640 10 client.7128 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:12:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.846+0000 7fc7b7400640 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x561ebe51f530 0x561ebe5a6790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d1b36087684a986 server_cookie=7138d76dd0ad4c2e in_seq=0 out_seq=0 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7a1600640 1 -- 172.21.15.62:0/2786059156 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561ebe45f500 con 0x561ebe51f530 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7a1600640 10 client.7128 handle_command_reply: tid=0 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 client.7128 shutdown 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2786059156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 msgr2=0x561ebe50c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe50c980 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8035040 tx=0x7fc7a8004730 comp rx=0 tx=0).stop 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2786059156 shutdown_connections 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x561ebe51f530 0x561ebe5a6790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ebe530280 0x561ebe51eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x561ebe51e670 0x561ebe51eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7a1600640 0 client.7128 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 --2- 172.21.15.62:0/2786059156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561ebe4f3bb0 0x561ebe50c980 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2786059156 >> 172.21.15.62:0/2786059156 conn(0x561ebe4bba30 msgr2=0x561ebe51b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2786059156 shutdown_connections 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 1 -- 172.21.15.62:0/2786059156 wait complete. 2024-04-25T09:12:00.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.847+0000 7fc7c1daa740 20 client.7128 trim_cache size 0 max 16384 2024-04-25T09:12:00.837 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:12:00.837 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-25T09:12:00.837 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:12:00.837 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 -- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc79806b4d0 msgr2=0x7fc79806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc79806b4d0 0x7fc79806d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8034000 tx=0x7fc7a802e000 comp rx=0 tx=0).stop 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 -- 172.21.15.62:0/2835407576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8148620 msgr2=0x7fc7b8148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8148620 0x7fc7b8148ab0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000efc0 tx=0x7fc7b000c490 comp rx=0 tx=0).stop 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 -- 172.21.15.62:0/2835407576 shutdown_connections 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc79806b4d0 0x7fc79806d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7b8148620 0x7fc7b8148ab0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7b815dc10 0x7fc7b814ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 --2- 172.21.15.62:0/2835407576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7b80b97b0 0x7fc7b80cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:00.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 -- 172.21.15.62:0/2835407576 >> 172.21.15.62:0/2835407576 conn(0x7fc7b806b000 msgr2=0x7fc7b806b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:00.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 -- 172.21.15.62:0/2835407576 shutdown_connections 2024-04-25T09:12:00.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:00.848+0000 7fc7bee00640 1 -- 172.21.15.62:0/2835407576 wait complete. 2024-04-25T09:12:00.852 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:12:00.853 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:12:00.853 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T09:12:01.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.071+0000 7f9c09800640 1 Processor -- start 2024-04-25T09:12:01.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.072+0000 7f9c09800640 1 -- start start 2024-04-25T09:12:01.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.073+0000 7f9c09800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c0415f330 0x7f9c040b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.073+0000 7f9c09800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9c040b2400 0x7f9c040b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.073+0000 7f9c09800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 0x7f9c040ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.073+0000 7f9c09800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c040d2630 con 0x7f9c0415f330 2024-04-25T09:12:01.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.073+0000 7f9c09800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c040b18b0 con 0x7f9c040b2400 2024-04-25T09:12:01.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.073+0000 7f9c09800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040b1b90 con 0x7f9c040b1220 2024-04-25T09:12:01.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c0415f330 0x7f9c040b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c0415f330 0x7f9c040b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37238/0 (socket says 172.21.15.62:37238) 2024-04-25T09:12:01.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03e00640 1 -- 172.21.15.62:0/1953811108 learned_addr learned my addr 172.21.15.62:0/1953811108 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:01.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c08e00640 1 --2- 172.21.15.62:0/1953811108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 0x7f9c040ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03400640 1 --2- 172.21.15.62:0/1953811108 >> v2:172.21.15.121:3300/0 conn(0x7f9c040b2400 0x7f9c040b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03e00640 1 -- 172.21.15.62:0/1953811108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 msgr2=0x7f9c040ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03e00640 1 --2- 172.21.15.62:0/1953811108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 0x7f9c040ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03e00640 1 -- 172.21.15.62:0/1953811108 >> v2:172.21.15.121:3300/0 conn(0x7f9c040b2400 msgr2=0x7f9c040b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03e00640 1 --2- 172.21.15.62:0/1953811108 >> v2:172.21.15.121:3300/0 conn(0x7f9c040b2400 0x7f9c040b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03e00640 1 -- 172.21.15.62:0/1953811108 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040b1e70 con 0x7f9c0415f330 2024-04-25T09:12:01.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.074+0000 7f9c03400640 1 --2- 172.21.15.62:0/1953811108 >> v2:172.21.15.121:3300/0 conn(0x7f9c040b2400 0x7f9c040b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:01.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.075+0000 7f9c03e00640 1 --2- 172.21.15.62:0/1953811108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b2c80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9c040b42b0 tx=0x7f9bf402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ad417e449c11a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.076+0000 7f9c02a00640 1 -- 172.21.15.62:0/1953811108 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4037070 con 0x7f9c0415f330 2024-04-25T09:12:01.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.076+0000 7f9c02a00640 1 -- 172.21.15.62:0/1953811108 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf4032440 con 0x7f9c0415f330 2024-04-25T09:12:01.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.076+0000 7f9c02a00640 1 -- 172.21.15.62:0/1953811108 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4032810 con 0x7f9c0415f330 2024-04-25T09:12:01.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.076+0000 7f9c09800640 1 -- 172.21.15.62:0/1953811108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 msgr2=0x7f9c040b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.076+0000 7f9c09800640 1 --2- 172.21.15.62:0/1953811108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b2c80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9c040b42b0 tx=0x7f9bf402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.077+0000 7f9c09800640 1 -- 172.21.15.62:0/1953811108 shutdown_connections 2024-04-25T09:12:01.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.077+0000 7f9c09800640 1 --2- 172.21.15.62:0/1953811108 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 0x7f9c040ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.077+0000 7f9c09800640 1 --2- 172.21.15.62:0/1953811108 >> v2:172.21.15.121:3300/0 conn(0x7f9c040b2400 0x7f9c040b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.077+0000 7f9c09800640 1 --2- 172.21.15.62:0/1953811108 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b2c80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.077+0000 7f9c09800640 1 -- 172.21.15.62:0/1953811108 >> 172.21.15.62:0/1953811108 conn(0x7f9c040d1750 msgr2=0x7f9c040d1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.077+0000 7f9c09800640 1 -- 172.21.15.62:0/1953811108 shutdown_connections 2024-04-25T09:12:01.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.077+0000 7f9c09800640 1 -- 172.21.15.62:0/1953811108 wait complete. 2024-04-25T09:12:01.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.078+0000 7f9c09800640 1 Processor -- start 2024-04-25T09:12:01.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.079+0000 7f9c09800640 1 -- start start 2024-04-25T09:12:01.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.079+0000 7f9c09800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 0x7f9c041f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c09800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c040b2400 0x7f9c04151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c09800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c09800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040aeb80 con 0x7f9c0415f330 2024-04-25T09:12:01.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c09800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040aea00 con 0x7f9c040b2400 2024-04-25T09:12:01.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c09800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040aed00 con 0x7f9c040b1220 2024-04-25T09:12:01.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37240/0 (socket says 172.21.15.62:37240) 2024-04-25T09:12:01.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c08e00640 1 -- 172.21.15.62:0/2376613766 learned_addr learned my addr 172.21.15.62:0/2376613766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:01.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c03400640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c040b2400 0x7f9c04151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c03e00640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 0x7f9c041f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c08e00640 1 -- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 msgr2=0x7f9c041f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c08e00640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 0x7f9c041f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c08e00640 1 -- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c040b2400 msgr2=0x7f9c04151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c08e00640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c040b2400 0x7f9c04151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.080+0000 7f9c08e00640 1 -- 172.21.15.62:0/2376613766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c041f2a50 con 0x7f9c0415f330 2024-04-25T09:12:01.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.081+0000 7f9c08e00640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b58a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c5a0 tx=0x7f9bfc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.081+0000 7f9bfbe00640 1 -- 172.21.15.62:0/2376613766 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc015070 con 0x7f9c0415f330 2024-04-25T09:12:01.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.081+0000 7f9bfbe00640 1 -- 172.21.15.62:0/2376613766 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc004220 con 0x7f9c0415f330 2024-04-25T09:12:01.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.081+0000 7f9bfbe00640 1 -- 172.21.15.62:0/2376613766 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc003610 con 0x7f9c0415f330 2024-04-25T09:12:01.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.081+0000 7f9c09800640 1 -- 172.21.15.62:0/2376613766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c040b6a20 con 0x7f9c0415f330 2024-04-25T09:12:01.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.081+0000 7f9c09800640 1 -- 172.21.15.62:0/2376613766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c040b6cc0 con 0x7f9c0415f330 2024-04-25T09:12:01.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.083+0000 7f9bfbe00640 1 -- 172.21.15.62:0/2376613766 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bfc0037b0 con 0x7f9c0415f330 2024-04-25T09:12:01.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.084+0000 7f9bfbe00640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9be006b4d0 0x7f9be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.085+0000 7f9bfbe00640 1 -- 172.21.15.62:0/2376613766 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9bfc08a0e0 con 0x7f9c0415f330 2024-04-25T09:12:01.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.085+0000 7f9c03e00640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9be006b4d0 0x7f9be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.085+0000 7f9c0dc36740 1 Processor -- start 2024-04-25T09:12:01.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.085+0000 7f9c0dc36740 1 -- start start 2024-04-25T09:12:01.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.086+0000 7f9c03e00640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9be006b4d0 0x7f9be006d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4034000 tx=0x7f9bf402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.086+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 0x55ff0001cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.086+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001d290 0x55fefffeca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.086+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffecfb0 0x55ff00014cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.086+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefff55870 con 0x55fefffecfb0 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.086+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55feffefed60 con 0x55ff0001d290 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.086+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefff51910 con 0x55ff0001c870 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffecfb0 0x55ff00014cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffecfb0 0x55ff00014cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37242/0 (socket says 172.21.15.62:37242) 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 -- 172.21.15.62:0/2315751193 learned_addr learned my addr 172.21.15.62:0/2315751193 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c03e00640 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 0x55ff0001cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c03400640 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001d290 0x55fefffeca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 -- 172.21.15.62:0/2315751193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 msgr2=0x55ff0001cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 0x55ff0001cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 -- 172.21.15.62:0/2315751193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001d290 msgr2=0x55fefffeca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001d290 0x55fefffeca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 -- 172.21.15.62:0/2315751193 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feffd347d0 con 0x55fefffecfb0 2024-04-25T09:12:01.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.087+0000 7f9c08e00640 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffecfb0 0x55ff00014cf0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc088480 tx=0x7f9bfc0581f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8101fc3ccea82edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.088+0000 7f9bf8c00640 1 -- 172.21.15.62:0/2315751193 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc01e030 con 0x55fefffecfb0 2024-04-25T09:12:01.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.088+0000 7f9bf8c00640 1 -- 172.21.15.62:0/2315751193 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc0701d0 con 0x55fefffecfb0 2024-04-25T09:12:01.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.088+0000 7f9bf8c00640 1 -- 172.21.15.62:0/2315751193 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc057bb0 con 0x55fefffecfb0 2024-04-25T09:12:01.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.088+0000 7f9c0dc36740 1 -- 172.21.15.62:0/2315751193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffecfb0 msgr2=0x55ff00014cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.088+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffecfb0 0x55ff00014cf0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc088480 tx=0x7f9bfc0581f0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.089+0000 7f9c0dc36740 1 -- 172.21.15.62:0/2315751193 shutdown_connections 2024-04-25T09:12:01.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.089+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffecfb0 0x55ff00014cf0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.089+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001d290 0x55fefffeca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.089+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/2315751193 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 0x55ff0001cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.089+0000 7f9c0dc36740 1 -- 172.21.15.62:0/2315751193 >> 172.21.15.62:0/2315751193 conn(0x55fefffeeef0 msgr2=0x55ff00037680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.089+0000 7f9c0dc36740 1 -- 172.21.15.62:0/2315751193 shutdown_connections 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.089+0000 7f9c0dc36740 1 -- 172.21.15.62:0/2315751193 wait complete. 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.089+0000 7f9c0dc36740 1 -- start start 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.090+0000 7f9c0dc36740 1 Processor -- start 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001c870 0x55ff0003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 0x55ff0001a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffff880 0x55fefffffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff00018fc0 con 0x55fefffff880 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefff121c0 con 0x55ff0001c870 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff00019260 con 0x55ff0001d290 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c0dc36740 10 client.0 _ll_register_callbacks cb 0x55fefff508f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:01.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffff880 0x55fefffffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffff880 0x55fefffffd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37246/0 (socket says 172.21.15.62:37246) 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c08e00640 1 -- 172.21.15.62:0/45226696 learned_addr learned my addr 172.21.15.62:0/45226696 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c03e00640 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001c870 0x55ff0003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.091+0000 7f9c03400640 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 0x55ff0001a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c08e00640 1 -- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 msgr2=0x55ff0001a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c08e00640 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 0x55ff0001a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c08e00640 1 -- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001c870 msgr2=0x55ff0003bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c08e00640 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001c870 0x55ff0003bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c08e00640 1 -- 172.21.15.62:0/45226696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55feffc2a0f0 con 0x55fefffff880 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c03e00640 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001c870 0x55ff0003bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c08e00640 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffff880 0x55fefffffd00 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc056a10 tx=0x7f9bfc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaab49497c394e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9bf1600640 1 -- 172.21.15.62:0/45226696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc085020 con 0x55fefffff880 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9bf1600640 1 -- 172.21.15.62:0/45226696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9bfc057db0 con 0x55fefffff880 2024-04-25T09:12:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c0dc36740 1 -- 172.21.15.62:0/45226696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bfc0701d0 con 0x55fefffff880 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9bf1600640 1 -- 172.21.15.62:0/45226696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bfc01e070 con 0x55fefffff880 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c0dc36740 10 client.7140 fetch_fsmap learned FSMap version 55 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.092+0000 7f9c0dc36740 1 -- 172.21.15.62:0/45226696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55feffd94f60 con 0x55fefffff880 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9bf1600640 1 -- 172.21.15.62:0/45226696 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc0701d0 con 0x55fefffff880 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9bf1600640 1 -- 172.21.15.62:0/45226696 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9bfc0770d0 con 0x55fefffff880 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9c0dc36740 10 client.7140 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9c0dc36740 10 client.7140 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9c0dc36740 20 client.7140 populate_metadata read hostname 'smithi062' 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55feffffdeb0 0x55feffffe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9c0dc36740 4 client.7140 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9c0dc36740 1 -- 172.21.15.62:0/45226696 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fefff2b690 con 0x55feffffdeb0 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.093+0000 7f9c03e00640 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55feffffdeb0 0x55feffffe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.094+0000 7f9c03e00640 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55feffffdeb0 0x55feffffe2c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb14101f581911b server_cookie=b0e6ce143e2ed213 in_seq=0 out_seq=0 2024-04-25T09:12:01.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.094+0000 7f9bf1600640 10 client.7140 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.094+0000 7f9bf1600640 1 -- 172.21.15.62:0/45226696 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fefff2b690 con 0x55feffffdeb0 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.094+0000 7f9bf1600640 10 client.7140 handle_command_reply: tid=0 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 client.7140 shutdown 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 -- 172.21.15.62:0/45226696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffff880 msgr2=0x55fefffffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffff880 0x55fefffffd00 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc056a10 tx=0x7f9bfc00cf60 comp rx=0 tx=0).stop 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 -- 172.21.15.62:0/45226696 shutdown_connections 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55feffffdeb0 0x55feffffe2c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fefffff880 0x55fefffffd00 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 0x55ff0001a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/45226696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff0001c870 0x55ff0003bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9bf1600640 0 client.7140 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 -- 172.21.15.62:0/45226696 >> 172.21.15.62:0/45226696 conn(0x55fefffecfb0 msgr2=0x55fefffec1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 -- 172.21.15.62:0/45226696 shutdown_connections 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 1 -- 172.21.15.62:0/45226696 wait complete. 2024-04-25T09:12:01.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.095+0000 7f9c0dc36740 20 client.7140 trim_cache size 0 max 16384 2024-04-25T09:12:01.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.113+0000 7f9c0dc36740 1 Processor -- start 2024-04-25T09:12:01.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.113+0000 7f9c0dc36740 1 -- start start 2024-04-25T09:12:01.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.113+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 0x55ff00019980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.113+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 0x55ff00059ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.113+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001d290 0x55ff00037480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.113+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041fdc20 con 0x55ff0001d290 2024-04-25T09:12:01.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.113+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041fce00 con 0x55fefffff880 2024-04-25T09:12:01.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.113+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041fc230 con 0x55ff0001c870 2024-04-25T09:12:01.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001d290 0x55ff00037480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c08e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001d290 0x55ff00037480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37248/0 (socket says 172.21.15.62:37248) 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c08e00640 1 -- 172.21.15.62:0/3059885489 learned_addr learned my addr 172.21.15.62:0/3059885489 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c08e00640 1 -- 172.21.15.62:0/3059885489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 msgr2=0x55ff00059ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c03e00640 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 0x55ff00019980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c03400640 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 0x55ff00059ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c08e00640 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 0x55ff00059ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c08e00640 1 -- 172.21.15.62:0/3059885489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 msgr2=0x55ff00019980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c08e00640 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 0x55ff00019980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.114+0000 7f9c08e00640 1 -- 172.21.15.62:0/3059885489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55feffc031b0 con 0x55ff0001d290 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.115+0000 7f9c08e00640 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001d290 0x55ff00037480 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0774e0 tx=0x7f9bfc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b48cded247317b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.115+0000 7f9bf3e00640 1 -- 172.21.15.62:0/3059885489 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc085070 con 0x55ff0001d290 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.115+0000 7f9bf3e00640 1 -- 172.21.15.62:0/3059885489 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc0770d0 con 0x55ff0001d290 2024-04-25T09:12:01.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.115+0000 7f9bf3e00640 1 -- 172.21.15.62:0/3059885489 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc057df0 con 0x55ff0001d290 2024-04-25T09:12:01.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3059885489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001d290 msgr2=0x55ff00037480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001d290 0x55ff00037480 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0774e0 tx=0x7f9bfc01e000 comp rx=0 tx=0).stop 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3059885489 shutdown_connections 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001d290 0x55ff00037480 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001c870 0x55ff00059ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3059885489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 0x55ff00019980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3059885489 >> 172.21.15.62:0/3059885489 conn(0x55fefffecfb0 msgr2=0x55fefffffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3059885489 shutdown_connections 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3059885489 wait complete. 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.116+0000 7f9c0dc36740 1 -- start start 2024-04-25T09:12:01.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.117+0000 7f9c0dc36740 1 Processor -- start 2024-04-25T09:12:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 0x55ff000a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001c870 0x55ff0001f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c0dc36740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 0x55ff0003b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefffee790 con 0x55ff0001c870 2024-04-25T09:12:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041fc800 con 0x55fefffff880 2024-04-25T09:12:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c0dc36740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefffee2c0 con 0x55ff0001d290 2024-04-25T09:12:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c0dc36740 10 client.0 _ll_register_callbacks cb 0x7f9bfc058530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001c870 0x55ff0001f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001c870 0x55ff0001f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37256/0 (socket says 172.21.15.62:37256) 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03400640 1 -- 172.21.15.62:0/3138916653 learned_addr learned my addr 172.21.15.62:0/3138916653 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03e00640 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 0x55ff000a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c08e00640 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 0x55ff0003b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03400640 1 -- 172.21.15.62:0/3138916653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 msgr2=0x55ff0003b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03400640 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 0x55ff0003b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03400640 1 -- 172.21.15.62:0/3138916653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 msgr2=0x55ff000a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03400640 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 0x55ff000a2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.118+0000 7f9c03400640 1 -- 172.21.15.62:0/3138916653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff00037f80 con 0x55ff0001c870 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.119+0000 7f9c03400640 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001c870 0x55ff0001f680 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00daa0 tx=0x7f9bec00df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fac8ef7f50ba46c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.119+0000 7f9bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.119+0000 7f9bf1600640 1 -- 172.21.15.62:0/3138916653 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec01e060 con 0x55ff0001c870 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.119+0000 7f9bf1600640 1 -- 172.21.15.62:0/3138916653 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9bec019400 con 0x55ff0001c870 2024-04-25T09:12:01.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.119+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3138916653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bfc0770d0 con 0x55ff0001c870 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.119+0000 7f9bf1600640 1 -- 172.21.15.62:0/3138916653 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bec019600 con 0x55ff0001c870 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.119+0000 7f9c0dc36740 10 client.7146 fetch_fsmap learned FSMap version 55 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.119+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3138916653 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55ff0001cea0 con 0x55ff0001c870 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9bf1600640 1 -- 172.21.15.62:0/3138916653 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc0770d0 con 0x55ff0001c870 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9bf1600640 1 -- 172.21.15.62:0/3138916653 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9bec027020 con 0x55ff0001c870 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9c0dc36740 10 client.7146 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9c0dc36740 10 client.7146 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9c0dc36740 20 client.7146 populate_metadata read hostname 'smithi062' 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55ff00037800 0x55feffff0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9c0dc36740 4 client.7146 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3138916653 --> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55fefff62db0 con 0x55ff00037800 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.120+0000 7f9c03e00640 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55ff00037800 0x55feffff0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c03e00640 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55ff00037800 0x55feffff0430 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72db84c588a3b4b server_cookie=e577ff603c61af43 in_seq=0 out_seq=0 2024-04-25T09:12:01.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9bf1600640 10 client.7146 ms_handle_connect on v2:172.21.15.62:6832/2258257596 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9bf1600640 1 -- 172.21.15.62:0/3138916653 <== mds.0 v2:172.21.15.62:6832/2258257596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17952 (crc 0 0 0) 0x55fefff62db0 con 0x55ff00037800 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9bf1600640 10 client.7146 handle_command_reply: tid=0 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 client.7146 shutdown 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3138916653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001c870 msgr2=0x55ff0001f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001c870 0x55ff0001f680 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00daa0 tx=0x7f9bec00df60 comp rx=0 tx=0).stop 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3138916653 shutdown_connections 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.62:6832/2258257596,v1:172.21.15.62:6834/2258257596] conn(0x55ff00037800 0x55feffff0430 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0001c870 0x55ff0001f680 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ff0001d290 0x55ff0003b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 --2- 172.21.15.62:0/3138916653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fefffff880 0x55ff000a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9bf1600640 0 client.7146 ms_handle_reset on v2:172.21.15.62:6832/2258257596 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.121+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3138916653 >> 172.21.15.62:0/3138916653 conn(0x55fefffecfb0 msgr2=0x55ff00019f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.122+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3138916653 shutdown_connections 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.122+0000 7f9c0dc36740 1 -- 172.21.15.62:0/3138916653 wait complete. 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.122+0000 7f9c0dc36740 20 client.7146 trim_cache size 0 max 16384 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:12:01.112 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 4055, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 4056, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1013685, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 36146610, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 17, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.783591084, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.371791667, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.157167749, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.191451456, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4055, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.817382299, 2024-04-25T09:12:01.113 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000201573 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 10, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000001080, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000108 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 30637, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 51608, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 10994299, 2024-04-25T09:12:01.114 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 58183575, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 7, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 12.547452036, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 6.980698134, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 5.968290399, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.795252381, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 51607, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.833857646, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000074289 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 69535, 2024-04-25T09:12:01.115 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.041670545, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000599 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 125840, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 34320 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 3680, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 3712, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1324334, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 26905252, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.774026253, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.397806080, 2024-04-25T09:12:01.116 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.149539962, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.174773446, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3711, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.715457117, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000192793 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 141312 2024-04-25T09:12:01.117 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6434, 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 29.619652159, 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004603613 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.118 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3910, 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.048910865, 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000524018 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 25900, 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 25900, 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 25900, 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 571.911498103, 2024-04-25T09:12:01.119 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.022081525 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 12, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 17872, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 17872, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 17872, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 2, 2024-04-25T09:12:01.120 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 2, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 2, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 48227, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 47725, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 157, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-25T09:12:01.121 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 1900, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 1900, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 11, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T09:12:01.122 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 1889, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 913, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 4029, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 3766, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 62, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 14030 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-25T09:12:01.123 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 4124, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 3817, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 2222, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T09:12:01.124 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 26793, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 26793, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-25T09:12:01.125 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 37, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 37, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 4, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 96139978, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4418, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 147.555447904, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.033398698 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.126 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 14, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 2015, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 2001, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 13, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 13, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 0, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 17872, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 19873, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 2001, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 2, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 8131, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 8129, 2024-04-25T09:12:01.127 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 97336, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215396 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 29415, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 25900, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 9, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2000, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.762960091, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001381480 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.128 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 29, 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.009297966, 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000320619 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 6000, 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 13.645006699, 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002274167 2024-04-25T09:12:01.129 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 7636, 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 14.989634055, 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001963021 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.130 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000083678, 2024-04-25T09:12:01.131 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000083678 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.006282833, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006282833 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.132 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.007704157, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.007704157 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.152198260, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.019024782 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.133 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2222, 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 505.153400569, 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.227341764 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.134 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.020501202, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.020501202 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-25T09:12:01.135 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000341124, 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000341124 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.136 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.137 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8000, 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 35.164087469, 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004395510 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.138 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 1, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 0, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 1, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 14338, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 14338, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 141, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-25T09:12:01.139 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:12:01.140 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:12:01.141 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 250643, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 92, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 440, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 5, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-25T09:12:01.142 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 5608, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 81, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 13457253, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 661242, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-25T09:12:01.143 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 10563, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 92552369, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 27, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 10563, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 10563, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 61.947017846, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.005864528 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.144 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 10563, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 10726 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 10563, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 2002, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 8561, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 0, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 11, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 101, 2024-04-25T09:12:01.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 4518, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 59, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 1817, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 2033, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1901, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 22, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-25T09:12:01.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-25T09:12:01.147 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 86, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 8, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 8, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 8, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-25T09:12:01.148 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 214, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 50 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-25T09:12:01.149 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 32, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 3817, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 3817, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 38368, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 9873047, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 38368, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:12:01.150 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 38368, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 9873047, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T09:12:01.151 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 10563, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 92952422, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 10563, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 92952422, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-25T09:12:01.152 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 10563, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 10563, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 10563, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 10563, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:12:01.153 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 3817, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 400053, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 3817, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 100, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 400053, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.154 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x555f15b746e0": { 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 26793, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 91945674, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 26793, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 4418, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 91945674, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T09:12:01.155 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.156 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:01.156 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.122+0000 7f9c09800640 1 -- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9be006b4d0 msgr2=0x7f9be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.122+0000 7f9c09800640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9be006b4d0 0x7f9be006d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4034000 tx=0x7f9bf402e000 comp rx=0 tx=0).stop 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.122+0000 7f9c09800640 1 -- 172.21.15.62:0/2376613766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 msgr2=0x7f9c040b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.122+0000 7f9c09800640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b58a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c5a0 tx=0x7f9bfc00ca60 comp rx=0 tx=0).stop 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.123+0000 7f9c09800640 1 -- 172.21.15.62:0/2376613766 shutdown_connections 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.123+0000 7f9c09800640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9be006b4d0 0x7f9be006d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.123+0000 7f9c09800640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c0415f330 0x7f9c040b58a0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.123+0000 7f9c09800640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c040b2400 0x7f9c04151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.123+0000 7f9c09800640 1 --2- 172.21.15.62:0/2376613766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c040b1220 0x7f9c041f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.123+0000 7f9c09800640 1 -- 172.21.15.62:0/2376613766 >> 172.21.15.62:0/2376613766 conn(0x7f9c040d1750 msgr2=0x7f9c0415d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.123+0000 7f9c09800640 1 -- 172.21.15.62:0/2376613766 shutdown_connections 2024-04-25T09:12:01.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.123+0000 7f9c09800640 1 -- 172.21.15.62:0/2376613766 wait complete. 2024-04-25T09:12:01.158 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:12:01.159 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:12:01.266 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:12:01.373 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:12:01.373 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:12:01.373 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:01.433 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:12:01.433 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:01.488 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:01.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.717+0000 7f58ff200640 1 Processor -- start 2024-04-25T09:12:01.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.718+0000 7f58ff200640 1 -- start start 2024-04-25T09:12:01.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.719+0000 7f58ff200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58f80afcf0 0x7f58f80d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.719+0000 7f58ff200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58f815b630 0x7f58f80d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.719+0000 7f58ff200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f815bc80 con 0x7f58f815f1c0 2024-04-25T09:12:01.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.719+0000 7f58ff200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f815e6e0 con 0x7f58f80afcf0 2024-04-25T09:12:01.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.719+0000 7f58ff200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f815e9c0 con 0x7f58f815b630 2024-04-25T09:12:01.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.719+0000 7f58f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58f80afcf0 0x7f58f80d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.720+0000 7f58f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58f80afcf0 0x7f58f80d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37272/0 (socket says 172.21.15.62:37272) 2024-04-25T09:12:01.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.720+0000 7f58f7e00640 1 -- 172.21.15.62:0/2160537015 learned_addr learned my addr 172.21.15.62:0/2160537015 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:01.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.720+0000 7f58fd200640 1 --2- 172.21.15.62:0/2160537015 >> v2:172.21.15.121:3300/0 conn(0x7f58f815b630 0x7f58f80d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.720+0000 7f58f7e00640 1 -- 172.21.15.62:0/2160537015 >> v2:172.21.15.121:3300/0 conn(0x7f58f815b630 msgr2=0x7f58f80d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.720+0000 7f58f7e00640 1 --2- 172.21.15.62:0/2160537015 >> v2:172.21.15.121:3300/0 conn(0x7f58f815b630 0x7f58f80d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.720+0000 7f58f7e00640 1 -- 172.21.15.62:0/2160537015 >> v1:172.21.15.121:6789/0 conn(0x7f58f815f1c0 legacy=0x7f58f815af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:01.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.720+0000 7f58f7e00640 1 -- 172.21.15.62:0/2160537015 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f815eca0 con 0x7f58f80afcf0 2024-04-25T09:12:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.721+0000 7f58f7e00640 1 --2- 172.21.15.62:0/2160537015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f80d0650 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f58f80cdad0 tx=0x7f58e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3525a65fa105afeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.721+0000 7f58f6a00640 1 -- 172.21.15.62:0/2160537015 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8036070 con 0x7f58f80afcf0 2024-04-25T09:12:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.721+0000 7f58f6a00640 1 -- 172.21.15.62:0/2160537015 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e8031600 con 0x7f58f80afcf0 2024-04-25T09:12:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.721+0000 7f58f6a00640 1 -- 172.21.15.62:0/2160537015 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e80319d0 con 0x7f58f80afcf0 2024-04-25T09:12:01.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.722+0000 7f58ff200640 1 -- 172.21.15.62:0/2160537015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 msgr2=0x7f58f80d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.722+0000 7f58ff200640 1 --2- 172.21.15.62:0/2160537015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f80d0650 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f58f80cdad0 tx=0x7f58e802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.722+0000 7f58ff200640 1 -- 172.21.15.62:0/2160537015 shutdown_connections 2024-04-25T09:12:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.722+0000 7f58ff200640 1 --2- 172.21.15.62:0/2160537015 >> v2:172.21.15.121:3300/0 conn(0x7f58f815b630 0x7f58f80d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.722+0000 7f58ff200640 1 --2- 172.21.15.62:0/2160537015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f80d0650 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.722+0000 7f58ff200640 1 -- 172.21.15.62:0/2160537015 >> 172.21.15.62:0/2160537015 conn(0x7f58f80b2490 msgr2=0x7f58f80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.722+0000 7f58ff200640 1 -- 172.21.15.62:0/2160537015 shutdown_connections 2024-04-25T09:12:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.722+0000 7f58ff200640 1 -- 172.21.15.62:0/2160537015 wait complete. 2024-04-25T09:12:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.724+0000 7f58ff200640 1 Processor -- start 2024-04-25T09:12:01.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.724+0000 7f58ff200640 1 -- start start 2024-04-25T09:12:01.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58ff200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f8145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58ff200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f815b630 0x7f58f80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58ff200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f815f1c0 0x7f58f80af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58ff200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80b3910 con 0x7f58f80afcf0 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58ff200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80b3790 con 0x7f58f815f1c0 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58ff200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80b3a90 con 0x7f58f815b630 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f8145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f8145940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37284/0 (socket says 172.21.15.62:37284) 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.725+0000 7f58f7e00640 1 -- 172.21.15.62:0/1765574508 learned_addr learned my addr 172.21.15.62:0/1765574508 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58fd200640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f815f1c0 0x7f58f80af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58f7400640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f815b630 0x7f58f80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58f7e00640 1 -- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f815b630 msgr2=0x7f58f80aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58f7e00640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f815b630 0x7f58f80aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58f7e00640 1 -- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f815f1c0 msgr2=0x7f58f80af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58f7e00640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f815f1c0 0x7f58f80af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58f7e00640 1 -- 172.21.15.62:0/1765574508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f8145e80 con 0x7f58f80afcf0 2024-04-25T09:12:01.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58fd200640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f815f1c0 0x7f58f80af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:01.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58f7400640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f815b630 0x7f58f80aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:01.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58f7e00640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f8145940 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f58e8000f80 tx=0x7f58e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58efe00640 1 -- 172.21.15.62:0/1765574508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8036050 con 0x7f58f80afcf0 2024-04-25T09:12:01.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58efe00640 1 -- 172.21.15.62:0/1765574508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e8042070 con 0x7f58f80afcf0 2024-04-25T09:12:01.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.726+0000 7f58efe00640 1 -- 172.21.15.62:0/1765574508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8009510 con 0x7f58f80afcf0 2024-04-25T09:12:01.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.727+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f80cf460 con 0x7f58f80afcf0 2024-04-25T09:12:01.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.727+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f80cfa20 con 0x7f58f80afcf0 2024-04-25T09:12:01.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.728+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f8072cb0 con 0x7f58f80afcf0 2024-04-25T09:12:01.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.729+0000 7f58efe00640 1 -- 172.21.15.62:0/1765574508 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58e8007c70 con 0x7f58f80afcf0 2024-04-25T09:12:01.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.730+0000 7f58efe00640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58d406b4d0 0x7f58d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:01.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.730+0000 7f58efe00640 1 -- 172.21.15.62:0/1765574508 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f58e80ab240 con 0x7f58f80afcf0 2024-04-25T09:12:01.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.730+0000 7f58f7400640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58d406b4d0 0x7f58d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:01.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.731+0000 7f58f7400640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58d406b4d0 0x7f58d406d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f58f8157fc0 tx=0x7f58e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:01.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.734+0000 7f58efe00640 1 -- 172.21.15.62:0/1765574508 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58e8045360 con 0x7f58f80afcf0 2024-04-25T09:12:01.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.934+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58f804f000 con 0x7f58f80afcf0 2024-04-25T09:12:01.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.936+0000 7f58efe00640 1 -- 172.21.15.62:0/1765574508 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16760 (secure 0 0 0) 0x7f58e8079fc0 con 0x7f58f80afcf0 2024-04-25T09:12:01.923 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:01.924 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:11:37.590166+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:27.235650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:28.419955+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":85,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1b","mappings":[{"from":2,"to":7}]},{"pgid":"11.36","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:01.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58d406b4d0 msgr2=0x7f58d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58d406b4d0 0x7f58d406d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f58f8157fc0 tx=0x7f58e0008040 comp rx=0 tx=0).stop 2024-04-25T09:12:01.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 msgr2=0x7f58f8145940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f8145940 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f58e8000f80 tx=0x7f58e8004a40 comp rx=0 tx=0).stop 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 shutdown_connections 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58d406b4d0 0x7f58d406d990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58f815f1c0 0x7f58f80af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58f815b630 0x7f58f80aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 --2- 172.21.15.62:0/1765574508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58f80afcf0 0x7f58f8145940 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 >> 172.21.15.62:0/1765574508 conn(0x7f58f80b2490 msgr2=0x7f58f80b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 shutdown_connections 2024-04-25T09:12:01.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:01.939+0000 7f58ff200640 1 -- 172.21.15.62:0/1765574508 wait complete. 2024-04-25T09:12:01.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:09:27.235650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:09:28.419955+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 85, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:01.947 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:02.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.171+0000 7f9386c00640 1 Processor -- start 2024-04-25T09:12:02.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.172+0000 7f9386c00640 1 -- start start 2024-04-25T09:12:02.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9386c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f938015dd50 0x7f93800d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9386c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f93800d2fb0 0x7f93800ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9386c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93800d3960 0x7f93800d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9386c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93800ce780 con 0x7f938015dd50 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9386c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93800d42f0 con 0x7f93800d2fb0 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9386c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93800d3440 con 0x7f93800d3960 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9385800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f938015dd50 0x7f93800d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9385800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f938015dd50 0x7f93800d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37300/0 (socket says 172.21.15.62:37300) 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.173+0000 7f9385800640 1 -- 172.21.15.62:0/1397197748 learned_addr learned my addr 172.21.15.62:0/1397197748 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.174+0000 7f9385800640 1 -- 172.21.15.62:0/1397197748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93800d3960 msgr2=0x7f93800d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.174+0000 7f9385800640 1 --2- 172.21.15.62:0/1397197748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93800d3960 0x7f93800d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.174+0000 7f9385800640 1 -- 172.21.15.62:0/1397197748 >> v2:172.21.15.121:3300/0 conn(0x7f93800d2fb0 msgr2=0x7f93800ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.174+0000 7f9385800640 1 --2- 172.21.15.62:0/1397197748 >> v2:172.21.15.121:3300/0 conn(0x7f93800d2fb0 0x7f93800ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.174+0000 7f9385800640 1 -- 172.21.15.62:0/1397197748 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93800ce1c0 con 0x7f938015dd50 2024-04-25T09:12:02.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.175+0000 7f9385800640 1 --2- 172.21.15.62:0/1397197748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f938015dd50 0x7f93800d2300 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f93800d3880 tx=0x7f937402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9659aac55850d35b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.175+0000 7f937fe00640 1 -- 172.21.15.62:0/1397197748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374037070 con 0x7f938015dd50 2024-04-25T09:12:02.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.175+0000 7f937fe00640 1 -- 172.21.15.62:0/1397197748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9374032440 con 0x7f938015dd50 2024-04-25T09:12:02.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.175+0000 7f937fe00640 1 -- 172.21.15.62:0/1397197748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374032810 con 0x7f938015dd50 2024-04-25T09:12:02.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 -- 172.21.15.62:0/1397197748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f938015dd50 msgr2=0x7f93800d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 --2- 172.21.15.62:0/1397197748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f938015dd50 0x7f93800d2300 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f93800d3880 tx=0x7f937402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 -- 172.21.15.62:0/1397197748 shutdown_connections 2024-04-25T09:12:02.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 --2- 172.21.15.62:0/1397197748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93800d3960 0x7f93800d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 --2- 172.21.15.62:0/1397197748 >> v2:172.21.15.121:3300/0 conn(0x7f93800d2fb0 0x7f93800ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 --2- 172.21.15.62:0/1397197748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f938015dd50 0x7f93800d2300 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 -- 172.21.15.62:0/1397197748 >> 172.21.15.62:0/1397197748 conn(0x7f938006b000 msgr2=0x7f938006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 -- 172.21.15.62:0/1397197748 shutdown_connections 2024-04-25T09:12:02.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.176+0000 7f9386c00640 1 -- 172.21.15.62:0/1397197748 wait complete. 2024-04-25T09:12:02.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.178+0000 7f9386c00640 1 Processor -- start 2024-04-25T09:12:02.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.178+0000 7f9386c00640 1 -- start start 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9386c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93800d3960 0x7f9380152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9386c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f938015dd50 0x7f93800b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9386c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93800b8550 0x7f93800b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9386c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93800cef00 con 0x7f93800d3960 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9386c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93800d2840 con 0x7f93800b8550 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9386c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93800b6b00 con 0x7f938015dd50 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9385800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93800d3960 0x7f9380152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9385800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93800d3960 0x7f9380152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37310/0 (socket says 172.21.15.62:37310) 2024-04-25T09:12:02.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.179+0000 7f9385800640 1 -- 172.21.15.62:0/1269363672 learned_addr learned my addr 172.21.15.62:0/1269363672 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9386200640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93800b8550 0x7f93800b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9384e00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f938015dd50 0x7f93800b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9385800640 1 -- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f938015dd50 msgr2=0x7f93800b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9385800640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f938015dd50 0x7f93800b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9385800640 1 -- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93800b8550 msgr2=0x7f93800b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9385800640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93800b8550 0x7f93800b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9385800640 1 -- 172.21.15.62:0/1269363672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380152dc0 con 0x7f93800d3960 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9386200640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93800b8550 0x7f93800b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.180+0000 7f9385800640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93800d3960 0x7f9380152880 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9374000c00 tx=0x7f93740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.181+0000 7f937d600640 1 -- 172.21.15.62:0/1269363672 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374041020 con 0x7f93800d3960 2024-04-25T09:12:02.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.181+0000 7f937d600640 1 -- 172.21.15.62:0/1269363672 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9374030970 con 0x7f93800d3960 2024-04-25T09:12:02.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.181+0000 7f937d600640 1 -- 172.21.15.62:0/1269363672 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374037040 con 0x7f93800d3960 2024-04-25T09:12:02.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.181+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938015e1a0 con 0x7f93800d3960 2024-04-25T09:12:02.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.181+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938015e760 con 0x7f93800d3960 2024-04-25T09:12:02.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.182+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938004ef20 con 0x7f93800d3960 2024-04-25T09:12:02.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.184+0000 7f937d600640 1 -- 172.21.15.62:0/1269363672 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9374044020 con 0x7f93800d3960 2024-04-25T09:12:02.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.185+0000 7f937d600640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f936406b4d0 0x7f936406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.185+0000 7f937d600640 1 -- 172.21.15.62:0/1269363672 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f93740aac70 con 0x7f93800d3960 2024-04-25T09:12:02.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.185+0000 7f9384e00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f936406b4d0 0x7f936406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.186+0000 7f9384e00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f936406b4d0 0x7f936406d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f93800cd370 tx=0x7f93700072f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.189+0000 7f937d600640 1 -- 172.21.15.62:0/1269363672 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f937404b030 con 0x7f93800d3960 2024-04-25T09:12:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.395+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93800ca720 con 0x7f93800d3960 2024-04-25T09:12:02.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.396+0000 7f937d600640 1 -- 172.21.15.62:0/1269363672 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9374079a80 con 0x7f93800d3960 2024-04-25T09:12:02.383 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f936406b4d0 msgr2=0x7f936406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f936406b4d0 0x7f936406d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f93800cd370 tx=0x7f93700072f0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93800d3960 msgr2=0x7f9380152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93800d3960 0x7f9380152880 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9374000c00 tx=0x7f93740046c0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 shutdown_connections 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f936406b4d0 0x7f936406d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93800b8550 0x7f93800b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f938015dd50 0x7f93800b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 --2- 172.21.15.62:0/1269363672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93800d3960 0x7f9380152880 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 >> 172.21.15.62:0/1269363672 conn(0x7f938006b000 msgr2=0x7f938015c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 shutdown_connections 2024-04-25T09:12:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.399+0000 7f9386c00640 1 -- 172.21.15.62:0/1269363672 wait complete. 2024-04-25T09:12:02.405 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:12:02.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.635+0000 7efc88e00640 1 Processor -- start 2024-04-25T09:12:02.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.635+0000 7efc88e00640 1 -- start start 2024-04-25T09:12:02.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.637+0000 7efc88e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc840b3360 0x7efc8406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.637+0000 7efc88e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efc84166720 0x7efc8406d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.637+0000 7efc88e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc840b4640 con 0x7efc8415c520 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.637+0000 7efc88e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc840b4420 con 0x7efc84166720 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.637+0000 7efc88e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc84165c40 con 0x7efc840b3360 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.637+0000 7efc83e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efc84166720 0x7efc8406d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.637+0000 7efc83e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efc84166720 0x7efc8406d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37324/0 (socket says 172.21.15.62:37324) 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.637+0000 7efc83e00640 1 -- 172.21.15.62:0/2712921051 learned_addr learned my addr 172.21.15.62:0/2712921051 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.638+0000 7efc82a00640 1 --2- 172.21.15.62:0/2712921051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc840b3360 0x7efc8406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.638+0000 7efc83e00640 1 -- 172.21.15.62:0/2712921051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc840b3360 msgr2=0x7efc8406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.638+0000 7efc83e00640 1 --2- 172.21.15.62:0/2712921051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc840b3360 0x7efc8406d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.638+0000 7efc83e00640 1 -- 172.21.15.62:0/2712921051 >> v1:172.21.15.121:6789/0 conn(0x7efc8415c520 legacy=0x7efc840b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:02.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.638+0000 7efc83e00640 1 -- 172.21.15.62:0/2712921051 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84165f20 con 0x7efc84166720 2024-04-25T09:12:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.639+0000 7efc83e00640 1 --2- 172.21.15.62:0/2712921051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84166720 0x7efc8406d800 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009e50 tx=0x7efc7c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d401eae0442906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.639+0000 7efc82000640 1 -- 172.21.15.62:0/2712921051 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c036070 con 0x7efc84166720 2024-04-25T09:12:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.639+0000 7efc82000640 1 -- 172.21.15.62:0/2712921051 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c031600 con 0x7efc84166720 2024-04-25T09:12:02.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.639+0000 7efc82000640 1 -- 172.21.15.62:0/2712921051 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0319d0 con 0x7efc84166720 2024-04-25T09:12:02.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.640+0000 7efc88e00640 1 -- 172.21.15.62:0/2712921051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84166720 msgr2=0x7efc8406d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.640+0000 7efc88e00640 1 --2- 172.21.15.62:0/2712921051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84166720 0x7efc8406d800 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009e50 tx=0x7efc7c02a630 comp rx=0 tx=0).stop 2024-04-25T09:12:02.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.640+0000 7efc88e00640 1 -- 172.21.15.62:0/2712921051 shutdown_connections 2024-04-25T09:12:02.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.640+0000 7efc88e00640 1 --2- 172.21.15.62:0/2712921051 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc84166720 0x7efc8406d800 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.640+0000 7efc88e00640 1 --2- 172.21.15.62:0/2712921051 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc840b3360 0x7efc8406d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.640+0000 7efc88e00640 1 -- 172.21.15.62:0/2712921051 >> 172.21.15.62:0/2712921051 conn(0x7efc84066f20 msgr2=0x7efc840b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.640+0000 7efc88e00640 1 -- 172.21.15.62:0/2712921051 shutdown_connections 2024-04-25T09:12:02.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.640+0000 7efc88e00640 1 -- 172.21.15.62:0/2712921051 wait complete. 2024-04-25T09:12:02.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.642+0000 7efc88e00640 1 Processor -- start 2024-04-25T09:12:02.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.642+0000 7efc88e00640 1 -- start start 2024-04-25T09:12:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc88e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc840b3360 0x7efc841f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc88e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc8415c520 0x7efc8415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc88e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc84166720 0x7efc841ec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc88e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8415c900 con 0x7efc8415c520 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc88e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840b4920 con 0x7efc840b3360 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc88e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840ce6f0 con 0x7efc84166720 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc82a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc8415c520 0x7efc8415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc82a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc8415c520 0x7efc8415abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37330/0 (socket says 172.21.15.62:37330) 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.643+0000 7efc82a00640 1 -- 172.21.15.62:0/2833245030 learned_addr learned my addr 172.21.15.62:0/2833245030 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc83400640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc840b3360 0x7efc841f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc83e00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc84166720 0x7efc841ec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc82a00640 1 -- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc84166720 msgr2=0x7efc841ec9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc82a00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc84166720 0x7efc841ec9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc82a00640 1 -- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc840b3360 msgr2=0x7efc841f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc82a00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc840b3360 0x7efc841f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc82a00640 1 -- 172.21.15.62:0/2833245030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc841ed380 con 0x7efc8415c520 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc83400640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc840b3360 0x7efc841f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:02.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc82a00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc8415c520 0x7efc8415abc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7efc7000c820 tx=0x7efc7000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc7b400640 1 -- 172.21.15.62:0/2833245030 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70015070 con 0x7efc8415c520 2024-04-25T09:12:02.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc7b400640 1 -- 172.21.15.62:0/2833245030 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc70004240 con 0x7efc8415c520 2024-04-25T09:12:02.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc7b400640 1 -- 172.21.15.62:0/2833245030 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7000ce10 con 0x7efc8415c520 2024-04-25T09:12:02.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.644+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc841ed660 con 0x7efc8415c520 2024-04-25T09:12:02.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.645+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc841f7b40 con 0x7efc8415c520 2024-04-25T09:12:02.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.646+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8406abe0 con 0x7efc8415c520 2024-04-25T09:12:02.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.647+0000 7efc7b400640 1 -- 172.21.15.62:0/2833245030 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc70007830 con 0x7efc8415c520 2024-04-25T09:12:02.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.648+0000 7efc7b400640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc6406b4d0 0x7efc6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:02.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.648+0000 7efc7b400640 1 -- 172.21.15.62:0/2833245030 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efc7008a1e0 con 0x7efc8415c520 2024-04-25T09:12:02.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.649+0000 7efc83400640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc6406b4d0 0x7efc6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:02.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.650+0000 7efc83400640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc6406b4d0 0x7efc6406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc74003eb0 tx=0x7efc74009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:02.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.653+0000 7efc7b400640 1 -- 172.21.15.62:0/2833245030 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc70058f60 con 0x7efc8415c520 2024-04-25T09:12:02.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.852+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc8404ef20 con 0x7efc8415c520 2024-04-25T09:12:02.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.853+0000 7efc7b400640 1 -- 172.21.15.62:0/2833245030 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc70058860 con 0x7efc8415c520 2024-04-25T09:12:02.840 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc6406b4d0 msgr2=0x7efc6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc6406b4d0 0x7efc6406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc74003eb0 tx=0x7efc74009040 comp rx=0 tx=0).stop 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc8415c520 msgr2=0x7efc8415abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc8415c520 0x7efc8415abc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7efc7000c820 tx=0x7efc7000cce0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 shutdown_connections 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc6406b4d0 0x7efc6406d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc8415c520 0x7efc8415abc0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc84166720 0x7efc841ec9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 --2- 172.21.15.62:0/2833245030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc840b3360 0x7efc841f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 >> 172.21.15.62:0/2833245030 conn(0x7efc84066f20 msgr2=0x7efc84068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 shutdown_connections 2024-04-25T09:12:02.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:02.856+0000 7efc88e00640 1 -- 172.21.15.62:0/2833245030 wait complete. 2024-04-25T09:12:02.864 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:12:03.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.098+0000 7f68b3000640 1 Processor -- start 2024-04-25T09:12:03.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.099+0000 7f68b3000640 1 -- start start 2024-04-25T09:12:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.101+0000 7f68b3000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f68ac0cda90 0x7f68ac0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.101+0000 7f68b3000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 0x7f68ac15d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.101+0000 7f68b3000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac0d1050 con 0x7f68ac06e7a0 2024-04-25T09:12:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.101+0000 7f68b3000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac0cffb0 con 0x7f68ac0cda90 2024-04-25T09:12:03.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.101+0000 7f68b3000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0d0290 con 0x7f68ac06f6b0 2024-04-25T09:12:03.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.101+0000 7f68abe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f68ac0cda90 0x7f68ac0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.101+0000 7f68a3e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f68ac06e7a0 0x7f68ac06efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38574/0 (socket says 172.21.15.62:38574) 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.101+0000 7f68a3e00640 1 -- 172.21.15.62:0/1504772897 learned_addr learned my addr 172.21.15.62:0/1504772897 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.102+0000 7f68b1000640 1 --2- 172.21.15.62:0/1504772897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 0x7f68ac15d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.102+0000 7f68abe00640 1 -- 172.21.15.62:0/1504772897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 msgr2=0x7f68ac15d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.102+0000 7f68abe00640 1 --2- 172.21.15.62:0/1504772897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 0x7f68ac15d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.102+0000 7f68abe00640 1 -- 172.21.15.62:0/1504772897 >> v1:172.21.15.62:6789/0 conn(0x7f68ac06e7a0 legacy=0x7f68ac06efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.102+0000 7f68abe00640 1 -- 172.21.15.62:0/1504772897 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac0d0570 con 0x7f68ac0cda90 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.102+0000 7f68b1000640 1 --2- 172.21.15.62:0/1504772897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 0x7f68ac15d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:03.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.102+0000 7f68ab400640 1 -- 172.21.15.62:0/1504772897 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654710369 0 0) 0x7f6894001110 con 0x7f68ac06e7a0 2024-04-25T09:12:03.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.103+0000 7f68abe00640 1 --2- 172.21.15.62:0/1504772897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac0cda90 0x7f68ac0ceb90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f689c0097a0 tx=0x7f689c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dcc49a966e36f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.103+0000 7f68ab400640 1 -- 172.21.15.62:0/1504772897 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c037070 con 0x7f68ac0cda90 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.103+0000 7f68ab400640 1 -- 172.21.15.62:0/1504772897 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c02ae50 con 0x7f68ac0cda90 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.103+0000 7f68ab400640 1 -- 172.21.15.62:0/1504772897 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c032660 con 0x7f68ac0cda90 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.104+0000 7f68b3000640 1 -- 172.21.15.62:0/1504772897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac0cda90 msgr2=0x7f68ac0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.104+0000 7f68b3000640 1 --2- 172.21.15.62:0/1504772897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac0cda90 0x7f68ac0ceb90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f689c0097a0 tx=0x7f689c02a100 comp rx=0 tx=0).stop 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.104+0000 7f68b3000640 1 -- 172.21.15.62:0/1504772897 shutdown_connections 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.104+0000 7f68b3000640 1 --2- 172.21.15.62:0/1504772897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 0x7f68ac15d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.104+0000 7f68b3000640 1 --2- 172.21.15.62:0/1504772897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac0cda90 0x7f68ac0ceb90 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.104+0000 7f68b3000640 1 -- 172.21.15.62:0/1504772897 >> 172.21.15.62:0/1504772897 conn(0x7f68ac0b3440 msgr2=0x7f68ac0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.104+0000 7f68b3000640 1 -- 172.21.15.62:0/1504772897 shutdown_connections 2024-04-25T09:12:03.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.104+0000 7f68b3000640 1 -- 172.21.15.62:0/1504772897 wait complete. 2024-04-25T09:12:03.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.106+0000 7f68b3000640 1 Processor -- start 2024-04-25T09:12:03.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.106+0000 7f68b3000640 1 -- start start 2024-04-25T09:12:03.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68b3000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac06e7a0 0x7f68ac0c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68b3000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 0x7f68ac15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68b3000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68ac0cda90 0x7f68ac0b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68b3000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0cfc60 con 0x7f68ac06e7a0 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68b3000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0d0b30 con 0x7f68ac0cda90 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68b3000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac15dae0 con 0x7f68ac06f6b0 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac06e7a0 0x7f68ac0c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68abe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac06e7a0 0x7f68ac0c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37340/0 (socket says 172.21.15.62:37340) 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68abe00640 1 -- 172.21.15.62:0/2960953232 learned_addr learned my addr 172.21.15.62:0/2960953232 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68abe00640 1 -- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 msgr2=0x7f68ac15cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68abe00640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 0x7f68ac15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.107+0000 7f68b1000640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68ac0cda90 0x7f68ac0b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.108+0000 7f68abe00640 1 -- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68ac0cda90 msgr2=0x7f68ac0b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.108+0000 7f68abe00640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68ac0cda90 0x7f68ac0b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.108+0000 7f68abe00640 1 -- 172.21.15.62:0/2960953232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac0c9a60 con 0x7f68ac06e7a0 2024-04-25T09:12:03.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.108+0000 7f68abe00640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac06e7a0 0x7f68ac0c9480 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f689c035040 tx=0x7f689c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.108+0000 7f68a8c00640 1 -- 172.21.15.62:0/2960953232 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c041070 con 0x7f68ac06e7a0 2024-04-25T09:12:03.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.108+0000 7f68a8c00640 1 -- 172.21.15.62:0/2960953232 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c03c670 con 0x7f68ac06e7a0 2024-04-25T09:12:03.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.108+0000 7f68a8c00640 1 -- 172.21.15.62:0/2960953232 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c037040 con 0x7f68ac06e7a0 2024-04-25T09:12:03.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.108+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac0c9d40 con 0x7f68ac06e7a0 2024-04-25T09:12:03.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.109+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac0ca300 con 0x7f68ac06e7a0 2024-04-25T09:12:03.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.110+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac0b3260 con 0x7f68ac06e7a0 2024-04-25T09:12:03.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.111+0000 7f68a8c00640 1 -- 172.21.15.62:0/2960953232 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f689c03a070 con 0x7f68ac06e7a0 2024-04-25T09:12:03.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.112+0000 7f68a8c00640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f688806b4d0 0x7f688806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.112+0000 7f68a8c00640 1 -- 172.21.15.62:0/2960953232 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f689c0aa710 con 0x7f68ac06e7a0 2024-04-25T09:12:03.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.112+0000 7f68a3e00640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f688806b4d0 0x7f688806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.114+0000 7f68a3e00640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f688806b4d0 0x7f688806d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0c7ad0 tx=0x7f6894001310 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.116+0000 7f68a8c00640 1 -- 172.21.15.62:0/2960953232 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68ac0b3260 con 0x7f68ac06e7a0 2024-04-25T09:12:03.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.317+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68ac04ef20 con 0x7f68ac06e7a0 2024-04-25T09:12:03.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.318+0000 7f68a8c00640 1 -- 172.21.15.62:0/2960953232 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f689c079490 con 0x7f68ac06e7a0 2024-04-25T09:12:03.305 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:12:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f688806b4d0 msgr2=0x7f688806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f688806b4d0 0x7f688806d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0c7ad0 tx=0x7f6894001310 comp rx=0 tx=0).stop 2024-04-25T09:12:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac06e7a0 msgr2=0x7f68ac0c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac06e7a0 0x7f68ac0c9480 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f689c035040 tx=0x7f689c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 shutdown_connections 2024-04-25T09:12:03.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f688806b4d0 0x7f688806d990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68ac0cda90 0x7f68ac0b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68ac06f6b0 0x7f68ac15cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 --2- 172.21.15.62:0/2960953232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68ac06e7a0 0x7f68ac0c9480 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 >> 172.21.15.62:0/2960953232 conn(0x7f68ac0b3440 msgr2=0x7f68ac06efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:03.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 shutdown_connections 2024-04-25T09:12:03.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.321+0000 7f68b3000640 1 -- 172.21.15.62:0/2960953232 wait complete. 2024-04-25T09:12:03.328 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:12:03.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.558+0000 7f044f200640 1 Processor -- start 2024-04-25T09:12:03.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.559+0000 7f044f200640 1 -- start start 2024-04-25T09:12:03.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.560+0000 7f044f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f045015fa10 0x7f045006e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.560+0000 7f044f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0450070460 con 0x7f04500b2710 2024-04-25T09:12:03.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.560+0000 7f044f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0450070740 con 0x7f0450166200 2024-04-25T09:12:03.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.560+0000 7f044f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04500b2e60 con 0x7f045015fa10 2024-04-25T09:12:03.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.561+0000 7f0445400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f04500b2710 0x7f0450165af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38576/0 (socket says 172.21.15.62:38576) 2024-04-25T09:12:03.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.561+0000 7f044de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f045015fa10 0x7f045006e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.561+0000 7f0445400640 1 -- 172.21.15.62:0/3796550432 learned_addr learned my addr 172.21.15.62:0/3796550432 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:03.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.561+0000 7f044d400640 1 -- 172.21.15.62:0/3796550432 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774925550 0 0) 0x7f0450070460 con 0x7f04500b2710 2024-04-25T09:12:03.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.561+0000 7f044d400640 1 -- 172.21.15.62:0/3796550432 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0434003520 con 0x7f04500b2710 2024-04-25T09:12:03.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.561+0000 7f044de00640 1 -- 172.21.15.62:0/3796550432 >> v1:172.21.15.121:6789/0 conn(0x7f0450166200 legacy=0x7f045006c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:03.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.561+0000 7f044de00640 1 -- 172.21.15.62:0/3796550432 >> v1:172.21.15.62:6789/0 conn(0x7f04500b2710 legacy=0x7f0450165af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.561+0000 7f044de00640 1 -- 172.21.15.62:0/3796550432 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450070a20 con 0x7f045015fa10 2024-04-25T09:12:03.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.563+0000 7f044de00640 1 --2- 172.21.15.62:0/3796550432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045006e1b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f04500b2fe0 tx=0x7f044002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd188e2b59dcbdf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.563+0000 7f044d400640 1 -- 172.21.15.62:0/3796550432 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440037070 con 0x7f045015fa10 2024-04-25T09:12:03.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.563+0000 7f044d400640 1 -- 172.21.15.62:0/3796550432 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0440032440 con 0x7f045015fa10 2024-04-25T09:12:03.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.563+0000 7f044d400640 1 -- 172.21.15.62:0/3796550432 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440032810 con 0x7f045015fa10 2024-04-25T09:12:03.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.564+0000 7f044f200640 1 -- 172.21.15.62:0/3796550432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 msgr2=0x7f045006e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.564+0000 7f044f200640 1 --2- 172.21.15.62:0/3796550432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045006e1b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f04500b2fe0 tx=0x7f044002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.564+0000 7f044f200640 1 -- 172.21.15.62:0/3796550432 shutdown_connections 2024-04-25T09:12:03.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.564+0000 7f044f200640 1 --2- 172.21.15.62:0/3796550432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045006e1b0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.564+0000 7f044f200640 1 -- 172.21.15.62:0/3796550432 >> 172.21.15.62:0/3796550432 conn(0x7f04500668f0 msgr2=0x7f0450066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:03.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.564+0000 7f044f200640 1 -- 172.21.15.62:0/3796550432 shutdown_connections 2024-04-25T09:12:03.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.564+0000 7f044f200640 1 -- 172.21.15.62:0/3796550432 wait complete. 2024-04-25T09:12:03.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.566+0000 7f044f200640 1 Processor -- start 2024-04-25T09:12:03.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.566+0000 7f044f200640 1 -- start start 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f044f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04500b2710 0x7f0450154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f044f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045015dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f044f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0450166200 0x7f0450158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f044f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0450071220 con 0x7f045015fa10 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f044f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04500b2b80 con 0x7f0450166200 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f044f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045015fdf0 con 0x7f04500b2710 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f0445400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045015dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f0445400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045015dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37346/0 (socket says 172.21.15.62:37346) 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f0445400640 1 -- 172.21.15.62:0/2291306544 learned_addr learned my addr 172.21.15.62:0/2291306544 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f044de00640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04500b2710 0x7f0450154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f0445400640 1 -- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04500b2710 msgr2=0x7f0450154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f0445400640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04500b2710 0x7f0450154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f0445400640 1 -- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0450166200 msgr2=0x7f0450158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f0445400640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0450166200 0x7f0450158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.567+0000 7f0445400640 1 -- 172.21.15.62:0/2291306544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450155130 con 0x7f045015fa10 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.568+0000 7f0445400640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045015dbb0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f043800c430 tx=0x7f043800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.568+0000 7f0446a00640 1 -- 172.21.15.62:0/2291306544 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438017070 con 0x7f045015fa10 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.568+0000 7f0446a00640 1 -- 172.21.15.62:0/2291306544 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0438005ce0 con 0x7f045015fa10 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.568+0000 7f0446a00640 1 -- 172.21.15.62:0/2291306544 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04380072f0 con 0x7f045015fa10 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.568+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045015cb10 con 0x7f045015fa10 2024-04-25T09:12:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.568+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045015d0d0 con 0x7f045015fa10 2024-04-25T09:12:03.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.569+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0450072560 con 0x7f045015fa10 2024-04-25T09:12:03.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.570+0000 7f0446a00640 1 -- 172.21.15.62:0/2291306544 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f043801a070 con 0x7f045015fa10 2024-04-25T09:12:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.571+0000 7f0446a00640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f042c06b4d0 0x7f042c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.571+0000 7f0446a00640 1 -- 172.21.15.62:0/2291306544 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0438089750 con 0x7f045015fa10 2024-04-25T09:12:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.572+0000 7f044de00640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f042c06b4d0 0x7f042c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.573+0000 7f044de00640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f042c06b4d0 0x7f042c06d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0440034000 tx=0x7f044002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:03.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.575+0000 7f0446a00640 1 -- 172.21.15.62:0/2291306544 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0450072560 con 0x7f045015fa10 2024-04-25T09:12:03.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.789+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0450071de0 con 0x7f045015fa10 2024-04-25T09:12:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.790+0000 7f0446a00640 1 -- 172.21.15.62:0/2291306544 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0450155130 con 0x7f045015fa10 2024-04-25T09:12:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:12:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f042c06b4d0 msgr2=0x7f042c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f042c06b4d0 0x7f042c06d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0440034000 tx=0x7f044002e000 comp rx=0 tx=0).stop 2024-04-25T09:12:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 msgr2=0x7f045015dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045015dbb0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f043800c430 tx=0x7f043800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 shutdown_connections 2024-04-25T09:12:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f042c06b4d0 0x7f042c06d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f045015fa10 0x7f045015dbb0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0450166200 0x7f0450158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 --2- 172.21.15.62:0/2291306544 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04500b2710 0x7f0450154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 >> 172.21.15.62:0/2291306544 conn(0x7f04500668f0 msgr2=0x7f04501637c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 shutdown_connections 2024-04-25T09:12:03.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:03.793+0000 7f044f200640 1 -- 172.21.15.62:0/2291306544 wait complete. 2024-04-25T09:12:03.799 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:12:03.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:03.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:03.913 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:12:03.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:03.913 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:12:03.963 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:12:03.964 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:04.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:04.008 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:12:04.008 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:04.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:12:04.053 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:12:04.054 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:04.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:04.107 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:04.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.335+0000 7f6952000640 1 Processor -- start 2024-04-25T09:12:04.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.336+0000 7f6952000640 1 -- start start 2024-04-25T09:12:04.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.337+0000 7f6952000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f694c0d0ae0 0x7f694c0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.337+0000 7f6952000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.337+0000 7f6952000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694c0d2620 con 0x7f694c15dc40 2024-04-25T09:12:04.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.337+0000 7f6952000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694c0d23c0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.337+0000 7f6952000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0d2900 con 0x7f694c0b6cf0 2024-04-25T09:12:04.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.338+0000 7f694be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.338+0000 7f694be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c0b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:56478/0 (socket says 172.21.15.62:56478) 2024-04-25T09:12:04.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.338+0000 7f694be00640 1 -- 172.21.15.62:0/1707611573 learned_addr learned my addr 172.21.15.62:0/1707611573 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:04.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.338+0000 7f6943e00640 1 --2- 172.21.15.62:0/1707611573 >> v2:172.21.15.121:3300/0 conn(0x7f694c0d0ae0 0x7f694c0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.339+0000 7f694be00640 1 -- 172.21.15.62:0/1707611573 >> v2:172.21.15.121:3300/0 conn(0x7f694c0d0ae0 msgr2=0x7f694c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:04.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.339+0000 7f694be00640 1 --2- 172.21.15.62:0/1707611573 >> v2:172.21.15.121:3300/0 conn(0x7f694c0d0ae0 0x7f694c0d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.339+0000 7f694be00640 1 -- 172.21.15.62:0/1707611573 >> v1:172.21.15.121:6789/0 conn(0x7f694c15dc40 legacy=0x7f694c0d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.339+0000 7f694be00640 1 -- 172.21.15.62:0/1707611573 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0d2be0 con 0x7f694c0b6cf0 2024-04-25T09:12:04.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.340+0000 7f694be00640 1 --2- 172.21.15.62:0/1707611573 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c0b7180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6944009e80 tx=0x7f694402a740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06f9721fa9c0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.341+0000 7f694aa00640 1 -- 172.21.15.62:0/1707611573 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6944036070 con 0x7f694c0b6cf0 2024-04-25T09:12:04.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.341+0000 7f694aa00640 1 -- 172.21.15.62:0/1707611573 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6944031600 con 0x7f694c0b6cf0 2024-04-25T09:12:04.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.341+0000 7f694aa00640 1 -- 172.21.15.62:0/1707611573 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69440319d0 con 0x7f694c0b6cf0 2024-04-25T09:12:04.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.341+0000 7f6952000640 1 -- 172.21.15.62:0/1707611573 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 msgr2=0x7f694c0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.341+0000 7f6952000640 1 --2- 172.21.15.62:0/1707611573 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c0b7180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6944009e80 tx=0x7f694402a740 comp rx=0 tx=0).stop 2024-04-25T09:12:04.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.342+0000 7f6952000640 1 -- 172.21.15.62:0/1707611573 shutdown_connections 2024-04-25T09:12:04.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.342+0000 7f6952000640 1 --2- 172.21.15.62:0/1707611573 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c0b7180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.342+0000 7f6952000640 1 --2- 172.21.15.62:0/1707611573 >> v2:172.21.15.121:3300/0 conn(0x7f694c0d0ae0 0x7f694c0d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.342+0000 7f6952000640 1 -- 172.21.15.62:0/1707611573 >> 172.21.15.62:0/1707611573 conn(0x7f694c06b000 msgr2=0x7f694c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:04.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.342+0000 7f6952000640 1 -- 172.21.15.62:0/1707611573 shutdown_connections 2024-04-25T09:12:04.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.342+0000 7f6952000640 1 -- 172.21.15.62:0/1707611573 wait complete. 2024-04-25T09:12:04.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.344+0000 7f6952000640 1 Processor -- start 2024-04-25T09:12:04.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.344+0000 7f6952000640 1 -- start start 2024-04-25T09:12:04.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6952000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6952000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694c0d0ae0 0x7f694c156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6952000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f694c15dc40 0x7f694c0cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6952000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0d36c0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6952000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c0d31a0 con 0x7f694c15dc40 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6952000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694c15e020 con 0x7f694c0b6cf0 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6943e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694c0d0ae0 0x7f694c156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6943e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694c0d0ae0 0x7f694c156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37354/0 (socket says 172.21.15.62:37354) 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6943e00640 1 -- 172.21.15.62:0/4111266417 learned_addr learned my addr 172.21.15.62:0/4111266417 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f694b400640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f694be00640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f694c15dc40 0x7f694c0cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6943e00640 1 -- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 msgr2=0x7f694c153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.345+0000 7f6943e00640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6943e00640 1 -- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f694c15dc40 msgr2=0x7f694c0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6943e00640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f694c15dc40 0x7f694c0cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6943e00640 1 -- 172.21.15.62:0/4111266417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c1573d0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f694be00640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f694c15dc40 0x7f694c0cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:04.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6943e00640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694c0d0ae0 0x7f694c156960 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f693400e9b0 tx=0x7f693400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6943400640 1 -- 172.21.15.62:0/4111266417 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934017070 con 0x7f694c0d0ae0 2024-04-25T09:12:04.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6943400640 1 -- 172.21.15.62:0/4111266417 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693400c9a0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6943400640 1 -- 172.21.15.62:0/4111266417 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693400cd90 con 0x7f694c0d0ae0 2024-04-25T09:12:04.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c15e270 con 0x7f694c0d0ae0 2024-04-25T09:12:04.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.346+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c1542c0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.347+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c068da0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.349+0000 7f6943400640 1 -- 172.21.15.62:0/4111266417 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f693401a070 con 0x7f694c0d0ae0 2024-04-25T09:12:04.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.350+0000 7f6943400640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692806b4d0 0x7f692806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.350+0000 7f6943400640 1 -- 172.21.15.62:0/4111266417 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f693408a5b0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.351+0000 7f694b400640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692806b4d0 0x7f692806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.352+0000 7f694b400640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692806b4d0 0x7f692806d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f693c007fd0 tx=0x7f693c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.354+0000 7f6943400640 1 -- 172.21.15.62:0/4111266417 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f694c068da0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.596+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f694c0729c0 con 0x7f694c0d0ae0 2024-04-25T09:12:04.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.598+0000 7f6943400640 1 -- 172.21.15.62:0/4111266417 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f6934059460 con 0x7f694c0d0ae0 2024-04-25T09:12:04.584 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:04.585 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5336,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2992229593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2992229593},{"type":"v1","addr":"172.21.15.62:6835","nonce":2992229593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5342,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/518754003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":518754003},{"type":"v1","addr":"172.21.15.121:6837","nonce":518754003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/6918584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":6918584},{"type":"v1","addr":"172.21.15.121:6836","nonce":6918584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:09:29.410378+0000","modified":"2024-04-25T09:09:38.647936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2258257596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2258257596},{"type":"v1","addr":"172.21.15.62:6834","nonce":2258257596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:12:04.585 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 55 2024-04-25T09:12:04.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.600+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692806b4d0 msgr2=0x7f692806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.600+0000 7f6952000640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692806b4d0 0x7f692806d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f693c007fd0 tx=0x7f693c009040 comp rx=0 tx=0).stop 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694c0d0ae0 msgr2=0x7f694c156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694c0d0ae0 0x7f694c156960 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f693400e9b0 tx=0x7f693400ee70 comp rx=0 tx=0).stop 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 shutdown_connections 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f692806b4d0 0x7f692806d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694c0d0ae0 0x7f694c156960 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f694c15dc40 0x7f694c0cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 --2- 172.21.15.62:0/4111266417 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f694c0b6cf0 0x7f694c153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 >> 172.21.15.62:0/4111266417 conn(0x7f694c06b000 msgr2=0x7f694c153a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 shutdown_connections 2024-04-25T09:12:04.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.601+0000 7f6952000640 1 -- 172.21.15.62:0/4111266417 wait complete. 2024-04-25T09:12:04.607 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:04.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.832+0000 7f688ea00640 1 Processor -- start 2024-04-25T09:12:04.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.832+0000 7f688ea00640 1 -- start start 2024-04-25T09:12:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.835+0000 7f688ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880afed0 0x7f68880cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.835+0000 7f688ea00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6888161b40 con 0x7f68880d00d0 2024-04-25T09:12:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.835+0000 7f688ea00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6888161e20 con 0x7f688815dcd0 2024-04-25T09:12:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.835+0000 7f688ea00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68880d0560 con 0x7f68880afed0 2024-04-25T09:12:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.835+0000 7f688ca00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f68880d00d0 0x7f68880ad8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59890/0 (socket says 172.21.15.62:59890) 2024-04-25T09:12:04.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.835+0000 7f688ca00640 1 -- 172.21.15.62:0/605405360 learned_addr learned my addr 172.21.15.62:0/605405360 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.835+0000 7f687fe00640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880afed0 0x7f68880cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138617107 0 0) 0x7f6888161b40 con 0x7f68880d00d0 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f686c003520 con 0x7f68880d00d0 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102955846 0 0) 0x7f686c003520 con 0x7f68880d00d0 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880afed0 msgr2=0x7f68880cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880afed0 0x7f68880cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 >> v1:172.21.15.121:6789/0 conn(0x7f688815dcd0 legacy=0x7f6888071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888162100 con 0x7f68880d00d0 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f68800019f0 con 0x7f68880d00d0 2024-04-25T09:12:04.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 >> v1:172.21.15.62:6789/0 conn(0x7f68880d00d0 legacy=0x7f68880ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f686c0039e0 0x7f686c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880afed0 0x7f686c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f686c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68880d0560 con 0x7f686c004d40 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c00baa0 con 0x7f686c0039e0 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.836+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c00bca0 con 0x7f68880afed0 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.837+0000 7f688ca00640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f686c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.837+0000 7f687fe00640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880afed0 0x7f686c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.837+0000 7f6887e00640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f686c0039e0 0x7f686c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.837+0000 7f688ca00640 1 -- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880afed0 msgr2=0x7f686c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.837+0000 7f688ca00640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880afed0 0x7f686c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.837+0000 7f688ca00640 1 -- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f686c0039e0 msgr2=0x7f686c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.837+0000 7f688ca00640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f686c0039e0 0x7f686c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.837+0000 7f688ca00640 1 -- 172.21.15.62:0/605405360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c003520 con 0x7f686c004d40 2024-04-25T09:12:04.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.838+0000 7f688ca00640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f686c0051c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f688000a780 tx=0x7f686c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4c683b916d350d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.838+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000a3d0 con 0x7f686c004d40 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688002f070 con 0x7f686c004d40 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f6887400640 1 -- 172.21.15.62:0/605405360 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688002bc20 con 0x7f686c004d40 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f688ea00640 1 -- 172.21.15.62:0/605405360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 msgr2=0x7f686c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f688ea00640 1 --2- 172.21.15.62:0/605405360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f686c0051c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f688000a780 tx=0x7f686c005830 comp rx=0 tx=0).stop 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f688ca00640 1 -- 172.21.15.62:0/605405360 reap_dead start 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f688ea00640 1 -- 172.21.15.62:0/605405360 shutdown_connections 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f688ea00640 1 -- 172.21.15.62:0/605405360 >> 172.21.15.62:0/605405360 conn(0x7f68880b2420 msgr2=0x7f68880b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f688ea00640 1 -- 172.21.15.62:0/605405360 shutdown_connections 2024-04-25T09:12:04.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.839+0000 7f688ea00640 1 -- 172.21.15.62:0/605405360 wait complete. 2024-04-25T09:12:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.841+0000 7f688ea00640 1 Processor -- start 2024-04-25T09:12:04.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.842+0000 7f688ea00640 1 -- start start 2024-04-25T09:12:04.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f688ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f6888156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f688ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880c8f40 0x7f688815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f688ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68880aeee0 0x7f68880af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f688ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6888162470 con 0x7f686c004d40 2024-04-25T09:12:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f688ea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68881618f0 con 0x7f68880aeee0 2024-04-25T09:12:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f688ea00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68880cfb90 con 0x7f68880c8f40 2024-04-25T09:12:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f6887e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f6888156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f6887e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f6888156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43636/0 (socket says 172.21.15.62:43636) 2024-04-25T09:12:04.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f6887e00640 1 -- 172.21.15.62:0/2890072241 learned_addr learned my addr 172.21.15.62:0/2890072241 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f6887e00640 1 -- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880c8f40 msgr2=0x7f688815ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f6887e00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880c8f40 0x7f688815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f6887e00640 1 -- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68880aeee0 msgr2=0x7f68880af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.843+0000 7f6887e00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68880aeee0 0x7f68880af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:04.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.844+0000 7f6887e00640 1 -- 172.21.15.62:0/2890072241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688815ef80 con 0x7f686c004d40 2024-04-25T09:12:04.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.844+0000 7f6887e00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f6888156e00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f687800e580 tx=0x7f687800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.844+0000 7f6884c00640 1 -- 172.21.15.62:0/2890072241 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68780052e0 con 0x7f686c004d40 2024-04-25T09:12:04.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.844+0000 7f6884c00640 1 -- 172.21.15.62:0/2890072241 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878018070 con 0x7f686c004d40 2024-04-25T09:12:04.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.844+0000 7f6884c00640 1 -- 172.21.15.62:0/2890072241 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878003810 con 0x7f686c004d40 2024-04-25T09:12:04.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.844+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68880c95b0 con 0x7f686c004d40 2024-04-25T09:12:04.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.844+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68880c9b70 con 0x7f686c004d40 2024-04-25T09:12:04.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.846+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6888068120 con 0x7f686c004d40 2024-04-25T09:12:04.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.847+0000 7f6884c00640 1 -- 172.21.15.62:0/2890072241 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68780039b0 con 0x7f686c004d40 2024-04-25T09:12:04.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.848+0000 7f6884c00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686406b4d0 0x7f686406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:04.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.848+0000 7f6884c00640 1 -- 172.21.15.62:0/2890072241 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6878089e40 con 0x7f686c004d40 2024-04-25T09:12:04.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.848+0000 7f687fe00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686406b4d0 0x7f686406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.849+0000 7f687fe00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686406b4d0 0x7f686406d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f68880c8ed0 tx=0x7f6870008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:04.852+0000 7f6884c00640 1 -- 172.21.15.62:0/2890072241 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6878058c70 con 0x7f686c004d40 2024-04-25T09:12:05.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.057+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6888072c80 con 0x7f686c004d40 2024-04-25T09:12:05.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.060+0000 7f6884c00640 1 -- 172.21.15.62:0/2890072241 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16760 (secure 0 0 0) 0x7f6878058570 con 0x7f686c004d40 2024-04-25T09:12:05.047 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:05.047 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:11:37.590166+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:27.235650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:28.419955+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":85,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1b","mappings":[{"from":2,"to":7}]},{"pgid":"11.36","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.062+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686406b4d0 msgr2=0x7f686406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.062+0000 7f688ea00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686406b4d0 0x7f686406d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f68880c8ed0 tx=0x7f6870008040 comp rx=0 tx=0).stop 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.062+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 msgr2=0x7f6888156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.062+0000 7f688ea00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f6888156e00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f687800e580 tx=0x7f687800ea40 comp rx=0 tx=0).stop 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.062+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 shutdown_connections 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.063+0000 7f688ea00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686406b4d0 0x7f686406d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.063+0000 7f688ea00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68880aeee0 0x7f68880af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.063+0000 7f688ea00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68880c8f40 0x7f688815ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.063+0000 7f688ea00640 1 --2- 172.21.15.62:0/2890072241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f686c004d40 0x7f6888156e00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.063+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 >> 172.21.15.62:0/2890072241 conn(0x7f68880b2420 msgr2=0x7f688806e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.063+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 shutdown_connections 2024-04-25T09:12:05.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.063+0000 7f688ea00640 1 -- 172.21.15.62:0/2890072241 wait complete. 2024-04-25T09:12:05.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:09:27.235650+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:09:28.419955+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 85, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:05.070 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.296+0000 7ffa96000640 1 Processor -- start 2024-04-25T09:12:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.296+0000 7ffa96000640 1 -- start start 2024-04-25T09:12:05.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.298+0000 7ffa96000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa900ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.298+0000 7ffa96000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa900b31a0 0x7ffa900b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.298+0000 7ffa96000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa900cf000 con 0x7ffa9006d6a0 2024-04-25T09:12:05.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.298+0000 7ffa96000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa900cf2e0 con 0x7ffa900b31a0 2024-04-25T09:12:05.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.298+0000 7ffa96000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900b26c0 con 0x7ffa9006d2c0 2024-04-25T09:12:05.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.299+0000 7ffa94c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa900ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.299+0000 7ffa95600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffa900b31a0 0x7ffa900b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.299+0000 7ffa94c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa900ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:52998/0 (socket says 172.21.15.62:52998) 2024-04-25T09:12:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.299+0000 7ffa94c00640 1 -- 172.21.15.62:0/1580973243 learned_addr learned my addr 172.21.15.62:0/1580973243 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.300+0000 7ffa95600640 1 -- 172.21.15.62:0/1580973243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 msgr2=0x7ffa900ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.300+0000 7ffa95600640 1 --2- 172.21.15.62:0/1580973243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa900ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.300+0000 7ffa95600640 1 -- 172.21.15.62:0/1580973243 >> v1:172.21.15.121:6789/0 conn(0x7ffa9006d6a0 legacy=0x7ffa900ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.300+0000 7ffa95600640 1 -- 172.21.15.62:0/1580973243 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa900b29a0 con 0x7ffa900b31a0 2024-04-25T09:12:05.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.301+0000 7ffa95600640 1 --2- 172.21.15.62:0/1580973243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa900b31a0 0x7ffa900b3f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffa880097a0 tx=0x7ffa8802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71726eeed9da1ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.301+0000 7ffa8fe00640 1 -- 172.21.15.62:0/1580973243 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88037070 con 0x7ffa900b31a0 2024-04-25T09:12:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa8fe00640 1 -- 172.21.15.62:0/1580973243 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa88032440 con 0x7ffa900b31a0 2024-04-25T09:12:05.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa8fe00640 1 -- 172.21.15.62:0/1580973243 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88032810 con 0x7ffa900b31a0 2024-04-25T09:12:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa96000640 1 -- 172.21.15.62:0/1580973243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa900b31a0 msgr2=0x7ffa900b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa96000640 1 --2- 172.21.15.62:0/1580973243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa900b31a0 0x7ffa900b3f40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffa880097a0 tx=0x7ffa8802a180 comp rx=0 tx=0).stop 2024-04-25T09:12:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa96000640 1 -- 172.21.15.62:0/1580973243 shutdown_connections 2024-04-25T09:12:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa96000640 1 --2- 172.21.15.62:0/1580973243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa900b31a0 0x7ffa900b3f40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa96000640 1 --2- 172.21.15.62:0/1580973243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa900ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa96000640 1 -- 172.21.15.62:0/1580973243 >> 172.21.15.62:0/1580973243 conn(0x7ffa90066f20 msgr2=0x7ffa900b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:05.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa96000640 1 -- 172.21.15.62:0/1580973243 shutdown_connections 2024-04-25T09:12:05.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.302+0000 7ffa96000640 1 -- 172.21.15.62:0/1580973243 wait complete. 2024-04-25T09:12:05.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.304+0000 7ffa96000640 1 Processor -- start 2024-04-25T09:12:05.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.304+0000 7ffa96000640 1 -- start start 2024-04-25T09:12:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.305+0000 7ffa96000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa9013f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.305+0000 7ffa96000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9006d6a0 0x7ffa9015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.305+0000 7ffa96000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa900b31a0 0x7ffa90153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.305+0000 7ffa96000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa901448d0 con 0x7ffa900b31a0 2024-04-25T09:12:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.305+0000 7ffa96000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9006db20 con 0x7ffa9006d6a0 2024-04-25T09:12:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.305+0000 7ffa96000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa90144a50 con 0x7ffa9006d2c0 2024-04-25T09:12:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa95600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa900b31a0 0x7ffa90153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa95600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa900b31a0 0x7ffa90153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43650/0 (socket says 172.21.15.62:43650) 2024-04-25T09:12:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa95600640 1 -- 172.21.15.62:0/2844689412 learned_addr learned my addr 172.21.15.62:0/2844689412 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:05.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa87e00640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9006d6a0 0x7ffa9015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa94c00640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa9013f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa95600640 1 -- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 msgr2=0x7ffa9013f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa95600640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa9013f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa95600640 1 -- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9006d6a0 msgr2=0x7ffa9015b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa95600640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9006d6a0 0x7ffa9015b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa95600640 1 -- 172.21.15.62:0/2844689412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa90142e10 con 0x7ffa900b31a0 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.306+0000 7ffa87e00640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9006d6a0 0x7ffa9015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.307+0000 7ffa95600640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa900b31a0 0x7ffa90153ed0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ffa880097a0 tx=0x7ffa880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.307+0000 7ffa8d600640 1 -- 172.21.15.62:0/2844689412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88041070 con 0x7ffa900b31a0 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.307+0000 7ffa8d600640 1 -- 172.21.15.62:0/2844689412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8803c870 con 0x7ffa900b31a0 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.307+0000 7ffa8d600640 1 -- 172.21.15.62:0/2844689412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88037040 con 0x7ffa900b31a0 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.307+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa90143080 con 0x7ffa900b31a0 2024-04-25T09:12:05.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.307+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa90071250 con 0x7ffa900b31a0 2024-04-25T09:12:05.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.308+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9006abe0 con 0x7ffa900b31a0 2024-04-25T09:12:05.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.310+0000 7ffa8d600640 1 -- 172.21.15.62:0/2844689412 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa88044070 con 0x7ffa900b31a0 2024-04-25T09:12:05.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.311+0000 7ffa8d600640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa6c06b4d0 0x7ffa6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.311+0000 7ffa8d600640 1 -- 172.21.15.62:0/2844689412 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffa880aae60 con 0x7ffa900b31a0 2024-04-25T09:12:05.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.311+0000 7ffa94c00640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa6c06b4d0 0x7ffa6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.312+0000 7ffa94c00640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa6c06b4d0 0x7ffa6c06d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009d80 tx=0x7ffa8000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.315+0000 7ffa8d600640 1 -- 172.21.15.62:0/2844689412 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa88079da0 con 0x7ffa900b31a0 2024-04-25T09:12:05.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.517+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa9004ef20 con 0x7ffa900b31a0 2024-04-25T09:12:05.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.518+0000 7ffa8d600640 1 -- 172.21.15.62:0/2844689412 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa880796a0 con 0x7ffa900b31a0 2024-04-25T09:12:05.504 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:12:05.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa6c06b4d0 msgr2=0x7ffa6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa6c06b4d0 0x7ffa6c06d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009d80 tx=0x7ffa8000b040 comp rx=0 tx=0).stop 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa900b31a0 msgr2=0x7ffa90153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa900b31a0 0x7ffa90153ed0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ffa880097a0 tx=0x7ffa880046c0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 shutdown_connections 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa6c06b4d0 0x7ffa6c06d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffa900b31a0 0x7ffa90153ed0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa9006d6a0 0x7ffa9015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 --2- 172.21.15.62:0/2844689412 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffa9006d2c0 0x7ffa9013f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.520+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 >> 172.21.15.62:0/2844689412 conn(0x7ffa90066f20 msgr2=0x7ffa900b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.521+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 shutdown_connections 2024-04-25T09:12:05.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.521+0000 7ffa96000640 1 -- 172.21.15.62:0/2844689412 wait complete. 2024-04-25T09:12:05.527 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:12:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.756+0000 7f56d9600640 1 Processor -- start 2024-04-25T09:12:05.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.757+0000 7f56d9600640 1 -- start start 2024-04-25T09:12:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.758+0000 7f56d9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d40b54d0 0x7f56d40ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.758+0000 7f56d9600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d40b4430 con 0x7f56d40ce7f0 2024-04-25T09:12:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.758+0000 7f56d9600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d40b4710 con 0x7f56d40b2d80 2024-04-25T09:12:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.758+0000 7f56d9600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40b49f0 con 0x7f56d40b54d0 2024-04-25T09:12:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.759+0000 7f56d8c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f56d40ce7f0 0x7f56d4164310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59892/0 (socket says 172.21.15.62:59892) 2024-04-25T09:12:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.759+0000 7f56d8c00640 1 -- 172.21.15.62:0/203020551 learned_addr learned my addr 172.21.15.62:0/203020551 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.759+0000 7f56cbe00640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d40b54d0 0x7f56d40ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.759+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800888923 0 0) 0x7f56d40b4430 con 0x7f56d40ce7f0 2024-04-25T09:12:05.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.759+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b8005730 con 0x7f56d40ce7f0 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2070325074 0 0) 0x7f56b8005730 con 0x7f56d40ce7f0 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d40b54d0 msgr2=0x7f56d40ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d40b54d0 0x7f56d40ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 >> v1:172.21.15.121:6789/0 conn(0x7f56d40b2d80 legacy=0x7f56d40b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40b4cd0 con 0x7f56d40ce7f0 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f56cc0019f0 con 0x7f56d40ce7f0 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 >> v1:172.21.15.62:6789/0 conn(0x7f56d40ce7f0 legacy=0x7f56d4164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8005bf0 0x7f56b8006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b8006550 0x7f56b8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d40b54d0 0x7f56b80071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40b49f0 con 0x7f56b8006550 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b800ffd0 con 0x7f56b8005bf0 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8010150 con 0x7f56d40b54d0 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56cbe00640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b8006550 0x7f56b8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d3e00640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8005bf0 0x7f56b8006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.760+0000 7f56d8c00640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d40b54d0 0x7f56b80071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.761+0000 7f56cbe00640 1 -- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d40b54d0 msgr2=0x7f56b80071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.761+0000 7f56cbe00640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d40b54d0 0x7f56b80071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.761+0000 7f56cbe00640 1 -- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8005bf0 msgr2=0x7f56b8006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.761+0000 7f56cbe00640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8005bf0 0x7f56b8006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.761+0000 7f56cbe00640 1 -- 172.21.15.62:0/203020551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b8005730 con 0x7f56b8006550 2024-04-25T09:12:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.762+0000 7f56cbe00640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b8006550 0x7f56b8006a10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009a90 tx=0x7f56b8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6179d9af879f6776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.762+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc036070 con 0x7f56b8006550 2024-04-25T09:12:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.762+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc031400 con 0x7f56b8006550 2024-04-25T09:12:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.762+0000 7f56d3400640 1 -- 172.21.15.62:0/203020551 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc009690 con 0x7f56b8006550 2024-04-25T09:12:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.763+0000 7f56d9600640 1 -- 172.21.15.62:0/203020551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b8006550 msgr2=0x7f56b8006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.763+0000 7f56d9600640 1 --2- 172.21.15.62:0/203020551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b8006550 0x7f56b8006a10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009a90 tx=0x7f56b8007a40 comp rx=0 tx=0).stop 2024-04-25T09:12:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.763+0000 7f56d8c00640 1 -- 172.21.15.62:0/203020551 reap_dead start 2024-04-25T09:12:05.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.763+0000 7f56d9600640 1 -- 172.21.15.62:0/203020551 shutdown_connections 2024-04-25T09:12:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.763+0000 7f56d9600640 1 -- 172.21.15.62:0/203020551 >> 172.21.15.62:0/203020551 conn(0x7f56d4066f20 msgr2=0x7f56d40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.763+0000 7f56d9600640 1 -- 172.21.15.62:0/203020551 shutdown_connections 2024-04-25T09:12:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.763+0000 7f56d9600640 1 -- 172.21.15.62:0/203020551 wait complete. 2024-04-25T09:12:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.765+0000 7f56d9600640 1 Processor -- start 2024-04-25T09:12:05.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.765+0000 7f56d9600640 1 -- start start 2024-04-25T09:12:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56d9600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8006550 0x7f56d41e3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56d9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d40b0820 0x7f56d40b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56d9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d41eeb00 0x7f56d41eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56d9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40b3870 con 0x7f56d40b0820 2024-04-25T09:12:05.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56d9600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40b40e0 con 0x7f56b8006550 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56d9600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40ce2b0 con 0x7f56d41eeb00 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d40b0820 0x7f56d40b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d40b0820 0x7f56d40b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43658/0 (socket says 172.21.15.62:43658) 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56cbe00640 1 -- 172.21.15.62:0/2255116130 learned_addr learned my addr 172.21.15.62:0/2255116130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.766+0000 7f56d3e00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8006550 0x7f56d41e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56d8c00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d41eeb00 0x7f56d41eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56cbe00640 1 -- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d41eeb00 msgr2=0x7f56d41eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56cbe00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d41eeb00 0x7f56d41eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56cbe00640 1 -- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8006550 msgr2=0x7f56d41e3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56cbe00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8006550 0x7f56d41e3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56cbe00640 1 -- 172.21.15.62:0/2255116130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d41ef4f0 con 0x7f56d40b0820 2024-04-25T09:12:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56d8c00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d41eeb00 0x7f56d41eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56cbe00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d40b0820 0x7f56d40b0ca0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f56bc034040 tx=0x7f56bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56d0c00640 1 -- 172.21.15.62:0/2255116130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc040070 con 0x7f56d40b0820 2024-04-25T09:12:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56d0c00640 1 -- 172.21.15.62:0/2255116130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc03b460 con 0x7f56d40b0820 2024-04-25T09:12:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56d0c00640 1 -- 172.21.15.62:0/2255116130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc036040 con 0x7f56d40b0820 2024-04-25T09:12:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.767+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d41e3ed0 con 0x7f56d40b0820 2024-04-25T09:12:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.768+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d415b110 con 0x7f56d40b0820 2024-04-25T09:12:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.769+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d406abe0 con 0x7f56d40b0820 2024-04-25T09:12:05.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.770+0000 7f56d0c00640 1 -- 172.21.15.62:0/2255116130 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56bc043070 con 0x7f56d40b0820 2024-04-25T09:12:05.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.771+0000 7f56d0c00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56b006b520 0x7f56b006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:05.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.771+0000 7f56d0c00640 1 -- 172.21.15.62:0/2255116130 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f56bc0aaa20 con 0x7f56d40b0820 2024-04-25T09:12:05.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.771+0000 7f56d3e00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56b006b520 0x7f56b006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:05.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.772+0000 7f56d3e00640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56b006b520 0x7f56b006d9e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f56d40b1560 tx=0x7f56c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:05.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.775+0000 7f56d0c00640 1 -- 172.21.15.62:0/2255116130 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56bc0791d0 con 0x7f56d40b0820 2024-04-25T09:12:05.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.977+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56d404ef20 con 0x7f56d40b0820 2024-04-25T09:12:05.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.978+0000 7f56d0c00640 1 -- 172.21.15.62:0/2255116130 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56bc07d270 con 0x7f56d40b0820 2024-04-25T09:12:05.965 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56b006b520 msgr2=0x7f56b006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56b006b520 0x7f56b006d9e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f56d40b1560 tx=0x7f56c4009040 comp rx=0 tx=0).stop 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d40b0820 msgr2=0x7f56d40b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d40b0820 0x7f56d40b0ca0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f56bc034040 tx=0x7f56bc004730 comp rx=0 tx=0).stop 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 shutdown_connections 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f56b006b520 0x7f56b006d9e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d40b0820 0x7f56d40b0ca0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56d41eeb00 0x7f56d41eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 --2- 172.21.15.62:0/2255116130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b8006550 0x7f56d41e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 >> 172.21.15.62:0/2255116130 conn(0x7f56d4066f20 msgr2=0x7f56d4070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:05.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 shutdown_connections 2024-04-25T09:12:05.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:05.981+0000 7f56d9600640 1 -- 172.21.15.62:0/2255116130 wait complete. 2024-04-25T09:12:05.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:12:06.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.205+0000 7f3299000640 1 Processor -- start 2024-04-25T09:12:06.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.206+0000 7f3299000640 1 -- start start 2024-04-25T09:12:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.207+0000 7f3299000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f329415bdf0 0x7f32940720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.207+0000 7f3299000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f32940cebc0 0x7f329406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.207+0000 7f3299000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329406d070 con 0x7f329406c7e0 2024-04-25T09:12:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.207+0000 7f3299000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329415c1d0 con 0x7f329415bdf0 2024-04-25T09:12:06.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.207+0000 7f3299000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32940ae8c0 con 0x7f32940cebc0 2024-04-25T09:12:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.208+0000 7f3291e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f329415bdf0 0x7f32940720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.208+0000 7f3291e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f329415bdf0 0x7f32940720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43662/0 (socket says 172.21.15.62:43662) 2024-04-25T09:12:06.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.208+0000 7f3291e00640 1 -- 172.21.15.62:0/3602566966 learned_addr learned my addr 172.21.15.62:0/3602566966 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:06.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.208+0000 7f3289400640 1 --2- 172.21.15.62:0/3602566966 >> v2:172.21.15.121:3300/0 conn(0x7f32940cebc0 0x7f329406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.208+0000 7f3291e00640 1 -- 172.21.15.62:0/3602566966 >> v2:172.21.15.121:3300/0 conn(0x7f32940cebc0 msgr2=0x7f329406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:06.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.208+0000 7f3291e00640 1 --2- 172.21.15.62:0/3602566966 >> v2:172.21.15.121:3300/0 conn(0x7f32940cebc0 0x7f329406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.208+0000 7f3291e00640 1 -- 172.21.15.62:0/3602566966 >> v1:172.21.15.121:6789/0 conn(0x7f329406c7e0 legacy=0x7f32940cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:06.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.208+0000 7f3291e00640 1 -- 172.21.15.62:0/3602566966 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329406d350 con 0x7f329415bdf0 2024-04-25T09:12:06.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.210+0000 7f3291e00640 1 --2- 172.21.15.62:0/3602566966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329415bdf0 0x7f32940720c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3294068870 tx=0x7f328402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f81d527cba69f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.210+0000 7f3291400640 1 -- 172.21.15.62:0/3602566966 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284036070 con 0x7f329415bdf0 2024-04-25T09:12:06.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.210+0000 7f3291400640 1 -- 172.21.15.62:0/3602566966 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3284031600 con 0x7f329415bdf0 2024-04-25T09:12:06.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.210+0000 7f3291400640 1 -- 172.21.15.62:0/3602566966 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32840319d0 con 0x7f329415bdf0 2024-04-25T09:12:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.211+0000 7f3299000640 1 -- 172.21.15.62:0/3602566966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329415bdf0 msgr2=0x7f32940720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.211+0000 7f3299000640 1 --2- 172.21.15.62:0/3602566966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329415bdf0 0x7f32940720c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3294068870 tx=0x7f328402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.211+0000 7f3299000640 1 -- 172.21.15.62:0/3602566966 shutdown_connections 2024-04-25T09:12:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.211+0000 7f3299000640 1 --2- 172.21.15.62:0/3602566966 >> v2:172.21.15.121:3300/0 conn(0x7f32940cebc0 0x7f329406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.211+0000 7f3299000640 1 --2- 172.21.15.62:0/3602566966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329415bdf0 0x7f32940720c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.211+0000 7f3299000640 1 -- 172.21.15.62:0/3602566966 >> 172.21.15.62:0/3602566966 conn(0x7f32940b1410 msgr2=0x7f32940b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.211+0000 7f3299000640 1 -- 172.21.15.62:0/3602566966 shutdown_connections 2024-04-25T09:12:06.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.211+0000 7f3299000640 1 -- 172.21.15.62:0/3602566966 wait complete. 2024-04-25T09:12:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.213+0000 7f3299000640 1 Processor -- start 2024-04-25T09:12:06.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.213+0000 7f3299000640 1 -- start start 2024-04-25T09:12:06.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3299000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329406c7e0 0x7f3294151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3299000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32940cebc0 0x7f3294156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3299000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f329415bdf0 0x7f3294156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3299000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329406cc30 con 0x7f329406c7e0 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3299000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32940aedf0 con 0x7f32940cebc0 2024-04-25T09:12:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3299000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329406cdb0 con 0x7f329415bdf0 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3291e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329406c7e0 0x7f3294151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3291e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329406c7e0 0x7f3294151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43668/0 (socket says 172.21.15.62:43668) 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3291e00640 1 -- 172.21.15.62:0/2257417689 learned_addr learned my addr 172.21.15.62:0/2257417689 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3289400640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32940cebc0 0x7f3294156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3291e00640 1 -- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f329415bdf0 msgr2=0x7f3294156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.214+0000 7f3292800640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f329415bdf0 0x7f3294156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3291e00640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f329415bdf0 0x7f3294156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3291e00640 1 -- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32940cebc0 msgr2=0x7f3294156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3291e00640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32940cebc0 0x7f3294156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3291e00640 1 -- 172.21.15.62:0/2257417689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294152860 con 0x7f329406c7e0 2024-04-25T09:12:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3292800640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f329415bdf0 0x7f3294156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:06.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3289400640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32940cebc0 0x7f3294156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:06.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3291e00640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329406c7e0 0x7f3294151790 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3284000f80 tx=0x7f3284004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f328aa00640 1 -- 172.21.15.62:0/2257417689 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284036050 con 0x7f329406c7e0 2024-04-25T09:12:06.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f328aa00640 1 -- 172.21.15.62:0/2257417689 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3284042070 con 0x7f329406c7e0 2024-04-25T09:12:06.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f328aa00640 1 -- 172.21.15.62:0/2257417689 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284009510 con 0x7f329406c7e0 2024-04-25T09:12:06.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3294152e30 con 0x7f329406c7e0 2024-04-25T09:12:06.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.215+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32941530f0 con 0x7f329406c7e0 2024-04-25T09:12:06.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.217+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329404ef20 con 0x7f329406c7e0 2024-04-25T09:12:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.218+0000 7f328aa00640 1 -- 172.21.15.62:0/2257417689 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3284007c70 con 0x7f329406c7e0 2024-04-25T09:12:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.219+0000 7f328aa00640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f327006b4d0 0x7f327006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.219+0000 7f328aa00640 1 -- 172.21.15.62:0/2257417689 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f32840ab1b0 con 0x7f329406c7e0 2024-04-25T09:12:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.219+0000 7f3289400640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f327006b4d0 0x7f327006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.220+0000 7f3289400640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f327006b4d0 0x7f327006d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32940afb70 tx=0x7f327c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.223+0000 7f328aa00640 1 -- 172.21.15.62:0/2257417689 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3294152860 con 0x7f329406c7e0 2024-04-25T09:12:06.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.422+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3294159fe0 con 0x7f329406c7e0 2024-04-25T09:12:06.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.424+0000 7f328aa00640 1 -- 172.21.15.62:0/2257417689 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3284045360 con 0x7f329406c7e0 2024-04-25T09:12:06.410 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:12:06.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f327006b4d0 msgr2=0x7f327006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f327006b4d0 0x7f327006d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32940afb70 tx=0x7f327c008040 comp rx=0 tx=0).stop 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329406c7e0 msgr2=0x7f3294151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329406c7e0 0x7f3294151790 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3284000f80 tx=0x7f3284004a40 comp rx=0 tx=0).stop 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 shutdown_connections 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f327006b4d0 0x7f327006d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f329415bdf0 0x7f3294156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32940cebc0 0x7f3294156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 --2- 172.21.15.62:0/2257417689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f329406c7e0 0x7f3294151790 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.426+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 >> 172.21.15.62:0/2257417689 conn(0x7f32940b1410 msgr2=0x7f32940cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.427+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 shutdown_connections 2024-04-25T09:12:06.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.427+0000 7f3299000640 1 -- 172.21.15.62:0/2257417689 wait complete. 2024-04-25T09:12:06.431 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:06.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.643+0000 7fc8bce00640 1 Processor -- start 2024-04-25T09:12:06.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.644+0000 7fc8bce00640 1 -- start start 2024-04-25T09:12:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.645+0000 7fc8bce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc8b80b56f0 0x7fc8b8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.645+0000 7fc8bce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc8b80b6a90 0x7fc8b8161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.645+0000 7fc8bce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.645+0000 7fc8bce00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b8161780 con 0x7fc8b80b6a90 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.645+0000 7fc8bce00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b80cedc0 con 0x7fc8b80b56f0 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.645+0000 7fc8bce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80d3110 con 0x7fc8b80b27a0 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b6a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc8b80b6a90 0x7fc8b8161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b6a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc8b80b6a90 0x7fc8b8161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43678/0 (socket says 172.21.15.62:43678) 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b6a00640 1 -- 172.21.15.62:0/1452567808 learned_addr learned my addr 172.21.15.62:0/1452567808 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b7400640 1 --2- 172.21.15.62:0/1452567808 >> v2:172.21.15.121:3300/0 conn(0x7fc8b80b56f0 0x7fc8b8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b7e00640 1 --2- 172.21.15.62:0/1452567808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b6a00640 1 -- 172.21.15.62:0/1452567808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 msgr2=0x7fc8b8161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b6a00640 1 --2- 172.21.15.62:0/1452567808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b6a00640 1 -- 172.21.15.62:0/1452567808 >> v2:172.21.15.121:3300/0 conn(0x7fc8b80b56f0 msgr2=0x7fc8b8160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b6a00640 1 --2- 172.21.15.62:0/1452567808 >> v2:172.21.15.121:3300/0 conn(0x7fc8b80b56f0 0x7fc8b8160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b6a00640 1 -- 172.21.15.62:0/1452567808 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b80d33f0 con 0x7fc8b80b6a90 2024-04-25T09:12:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.646+0000 7fc8b7e00640 1 --2- 172.21.15.62:0/1452567808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.647+0000 7fc8b6a00640 1 --2- 172.21.15.62:0/1452567808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b8161240 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc8b808c310 tx=0x7fc8a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40584b64c81b8126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.647+0000 7fc8b6000640 1 -- 172.21.15.62:0/1452567808 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4037070 con 0x7fc8b80b6a90 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8b6000640 1 -- 172.21.15.62:0/1452567808 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a4032440 con 0x7fc8b80b6a90 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8b6000640 1 -- 172.21.15.62:0/1452567808 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4032810 con 0x7fc8b80b6a90 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 -- 172.21.15.62:0/1452567808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 msgr2=0x7fc8b8161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 --2- 172.21.15.62:0/1452567808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b8161240 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc8b808c310 tx=0x7fc8a402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 -- 172.21.15.62:0/1452567808 shutdown_connections 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 --2- 172.21.15.62:0/1452567808 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 --2- 172.21.15.62:0/1452567808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b8161240 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 --2- 172.21.15.62:0/1452567808 >> v2:172.21.15.121:3300/0 conn(0x7fc8b80b56f0 0x7fc8b8160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 -- 172.21.15.62:0/1452567808 >> 172.21.15.62:0/1452567808 conn(0x7fc8b8066d10 msgr2=0x7fc8b8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 -- 172.21.15.62:0/1452567808 shutdown_connections 2024-04-25T09:12:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.648+0000 7fc8bce00640 1 -- 172.21.15.62:0/1452567808 wait complete. 2024-04-25T09:12:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.650+0000 7fc8bce00640 1 Processor -- start 2024-04-25T09:12:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.650+0000 7fc8bce00640 1 -- start start 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8bce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8bce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b80b56f0 0x7fc8b814b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8bce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b814b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8bce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80b2d60 con 0x7fc8b80b6a90 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8bce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80cea50 con 0x7fc8b80b56f0 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8bce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80b2ee0 con 0x7fc8b80b27a0 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b814b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b814b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43682/0 (socket says 172.21.15.62:43682) 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8b7e00640 1 -- 172.21.15.62:0/3064684877 learned_addr learned my addr 172.21.15.62:0/3064684877 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.651+0000 7fc8b6a00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b80b56f0 0x7fc8b814b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8b7400640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8b7e00640 1 -- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 msgr2=0x7fc8b8149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8b7e00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8b7e00640 1 -- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b80b56f0 msgr2=0x7fc8b814b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8b7e00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b80b56f0 0x7fc8b814b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8b7e00640 1 -- 172.21.15.62:0/3064684877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b814bc50 con 0x7fc8b80b6a90 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8b7e00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b814b620 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ce50 tx=0x7fc8b000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8af400640 1 -- 172.21.15.62:0/3064684877 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0003b90 con 0x7fc8b80b6a90 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8af400640 1 -- 172.21.15.62:0/3064684877 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0003d50 con 0x7fc8b80b6a90 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8af400640 1 -- 172.21.15.62:0/3064684877 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0004260 con 0x7fc8b80b6a90 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b814bf30 con 0x7fc8b80b6a90 2024-04-25T09:12:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.652+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b814dcf0 con 0x7fc8b80b6a90 2024-04-25T09:12:06.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.654+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b804ef20 con 0x7fc8b80b6a90 2024-04-25T09:12:06.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.655+0000 7fc8af400640 1 -- 172.21.15.62:0/3064684877 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8b0004400 con 0x7fc8b80b6a90 2024-04-25T09:12:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.655+0000 7fc8af400640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89806b4d0 0x7fc89806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.656+0000 7fc8af400640 1 -- 172.21.15.62:0/3064684877 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc8b008ac20 con 0x7fc8b80b6a90 2024-04-25T09:12:06.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.656+0000 7fc8b7400640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89806b4d0 0x7fc89806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:06.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.657+0000 7fc8b7400640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89806b4d0 0x7fc89806d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8006fd0 tx=0x7fc8a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:06.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.660+0000 7fc8af400640 1 -- 172.21.15.62:0/3064684877 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8b0059a30 con 0x7fc8b80b6a90 2024-04-25T09:12:06.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.892+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8b814dec0 con 0x7fc8b80b6a90 2024-04-25T09:12:06.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.893+0000 7fc8af400640 1 -- 172.21.15.62:0/3064684877 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fc8b0059330 con 0x7fc8b80b6a90 2024-04-25T09:12:06.879 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:06.879 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5336,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2992229593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2992229593},{"type":"v1","addr":"172.21.15.62:6835","nonce":2992229593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5342,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/518754003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":518754003},{"type":"v1","addr":"172.21.15.121:6837","nonce":518754003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/6918584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":6918584},{"type":"v1","addr":"172.21.15.121:6836","nonce":6918584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:09:29.410378+0000","modified":"2024-04-25T09:09:38.647936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2258257596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2258257596},{"type":"v1","addr":"172.21.15.62:6834","nonce":2258257596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:12:06.880 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 55 2024-04-25T09:12:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89806b4d0 msgr2=0x7fc89806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89806b4d0 0x7fc89806d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8006fd0 tx=0x7fc8a8008040 comp rx=0 tx=0).stop 2024-04-25T09:12:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 msgr2=0x7fc8b814b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b814b620 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ce50 tx=0x7fc8b000a710 comp rx=0 tx=0).stop 2024-04-25T09:12:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 shutdown_connections 2024-04-25T09:12:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89806b4d0 0x7fc89806d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8b80b6a90 0x7fc8b814b620 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8b80b56f0 0x7fc8b814b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 --2- 172.21.15.62:0/3064684877 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8b80b27a0 0x7fc8b8149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:06.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 >> 172.21.15.62:0/3064684877 conn(0x7fc8b8066d10 msgr2=0x7fc8b80b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:06.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 shutdown_connections 2024-04-25T09:12:06.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:06.895+0000 7fc8bce00640 1 -- 172.21.15.62:0/3064684877 wait complete. 2024-04-25T09:12:06.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.110+0000 7efcd3200640 1 Processor -- start 2024-04-25T09:12:07.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.111+0000 7efcd3200640 1 -- start start 2024-04-25T09:12:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.112+0000 7efcd3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efcd40b6a90 0x7efcd40d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.112+0000 7efcd3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd40d3110 con 0x7efcd40b27a0 2024-04-25T09:12:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.112+0000 7efcd3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd40d33f0 con 0x7efcd40b56f0 2024-04-25T09:12:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.112+0000 7efcd3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd40cedc0 con 0x7efcd40b6a90 2024-04-25T09:12:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd2800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7efcd40b27a0 0x7efcd40d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59908/0 (socket says 172.21.15.62:59908) 2024-04-25T09:12:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd2800640 1 -- 172.21.15.62:0/814738231 learned_addr learned my addr 172.21.15.62:0/814738231 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:07.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd1400640 1 --2- 172.21.15.62:0/814738231 >> v2:172.21.15.121:3300/0 conn(0x7efcd40b6a90 0x7efcd40d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154876159 0 0) 0x7efcd40d3110 con 0x7efcd40b27a0 2024-04-25T09:12:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcb8003520 con 0x7efcd40b27a0 2024-04-25T09:12:07.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3817705096 0 0) 0x7efcb8003520 con 0x7efcd40b27a0 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 >> v2:172.21.15.121:3300/0 conn(0x7efcd40b6a90 msgr2=0x7efcd40d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 --2- 172.21.15.62:0/814738231 >> v2:172.21.15.121:3300/0 conn(0x7efcd40b6a90 0x7efcd40d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 >> v1:172.21.15.121:6789/0 conn(0x7efcd40b56f0 legacy=0x7efcd4160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd40d36d0 con 0x7efcd40b27a0 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7efccc0019f0 con 0x7efcd40b27a0 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.113+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 >> v1:172.21.15.62:6789/0 conn(0x7efcd40b27a0 legacy=0x7efcd40d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd0a00640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcb8003a00 0x7efcb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd0a00640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb8004800 0x7efcb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd0a00640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb80051c0 0x7efcb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40cedc0 con 0x7efcb8004800 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb800c0c0 con 0x7efcb80051c0 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb800c2c0 con 0x7efcb8003a00 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd1400640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb8004800 0x7efcb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd1400640 1 -- 172.21.15.62:0/814738231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcb8003a00 msgr2=0x7efcb8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd1400640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcb8003a00 0x7efcb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd1400640 1 -- 172.21.15.62:0/814738231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb80051c0 msgr2=0x7efcb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd2800640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb80051c0 0x7efcb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd1400640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb80051c0 0x7efcb800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd2800640 1 -- 172.21.15.62:0/814738231 reap_dead start 2024-04-25T09:12:07.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.114+0000 7efcd1400640 1 -- 172.21.15.62:0/814738231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb8003520 con 0x7efcb8004800 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.115+0000 7efcd1400640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb8004800 0x7efcb8004c80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009a90 tx=0x7efcb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e28f7576e951dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc036070 con 0x7efcb8004800 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc031400 con 0x7efcb8004800 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd0a00640 1 -- 172.21.15.62:0/814738231 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc009690 con 0x7efcb8004800 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd3200640 1 -- 172.21.15.62:0/814738231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb8004800 msgr2=0x7efcb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd3200640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb8004800 0x7efcb8004c80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009a90 tx=0x7efcb8005830 comp rx=0 tx=0).stop 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd3200640 1 -- 172.21.15.62:0/814738231 shutdown_connections 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd3200640 1 --2- 172.21.15.62:0/814738231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcb8004800 0x7efcb8004c80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd3200640 1 -- 172.21.15.62:0/814738231 >> 172.21.15.62:0/814738231 conn(0x7efcd4066d10 msgr2=0x7efcd4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd3200640 1 -- 172.21.15.62:0/814738231 shutdown_connections 2024-04-25T09:12:07.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.116+0000 7efcd3200640 1 -- 172.21.15.62:0/814738231 wait complete. 2024-04-25T09:12:07.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.118+0000 7efcd3200640 1 Processor -- start 2024-04-25T09:12:07.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.118+0000 7efcd3200640 1 -- start start 2024-04-25T09:12:07.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb8004800 0x7efcd414bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd4157e40 0x7efcd415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd4159450 0x7efcd415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b2ee0 con 0x7efcd4159450 2024-04-25T09:12:07.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b2d60 con 0x7efcb8004800 2024-04-25T09:12:07.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40b5ad0 con 0x7efcd4157e40 2024-04-25T09:12:07.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd4159450 0x7efcd415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd4159450 0x7efcd415d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43704/0 (socket says 172.21.15.62:43704) 2024-04-25T09:12:07.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.119+0000 7efcd2800640 1 -- 172.21.15.62:0/488998929 learned_addr learned my addr 172.21.15.62:0/488998929 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd1e00640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb8004800 0x7efcd414bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd1400640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd4157e40 0x7efcd415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd2800640 1 -- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd4157e40 msgr2=0x7efcd415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd2800640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd4157e40 0x7efcd415c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd2800640 1 -- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb8004800 msgr2=0x7efcd414bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd2800640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb8004800 0x7efcd414bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd2800640 1 -- 172.21.15.62:0/488998929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd41583b0 con 0x7efcd4159450 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd2800640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd4159450 0x7efcd415d310 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7efcd40ced30 tx=0x7efccc00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcca000640 1 -- 172.21.15.62:0/488998929 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc00b860 con 0x7efcd4159450 2024-04-25T09:12:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcca000640 1 -- 172.21.15.62:0/488998929 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc00ba00 con 0x7efcd4159450 2024-04-25T09:12:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcca000640 1 -- 172.21.15.62:0/488998929 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc00bdb0 con 0x7efcd4159450 2024-04-25T09:12:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.120+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd4158690 con 0x7efcd4159450 2024-04-25T09:12:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.121+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd4158c50 con 0x7efcd4159450 2024-04-25T09:12:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.122+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd404ef20 con 0x7efcd4159450 2024-04-25T09:12:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.123+0000 7efcca000640 1 -- 172.21.15.62:0/488998929 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efccc01b020 con 0x7efcd4159450 2024-04-25T09:12:07.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.124+0000 7efcca000640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcb006b500 0x7efcb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.124+0000 7efcca000640 1 -- 172.21.15.62:0/488998929 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efccc08a2b0 con 0x7efcd4159450 2024-04-25T09:12:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.125+0000 7efcd1e00640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcb006b500 0x7efcb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.126+0000 7efcd1e00640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcb006b500 0x7efcb006d9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efcd4159f00 tx=0x7efcc4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.128+0000 7efcca000640 1 -- 172.21.15.62:0/488998929 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efccc0590c0 con 0x7efcd4159450 2024-04-25T09:12:07.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.321+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcd40ce580 con 0x7efcd4159450 2024-04-25T09:12:07.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.323+0000 7efcca000640 1 -- 172.21.15.62:0/488998929 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16760 (secure 0 0 0) 0x7efccc0589c0 con 0x7efcd4159450 2024-04-25T09:12:07.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:07.310 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:11:37.590166+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:27.235650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:28.419955+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":85,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1b","mappings":[{"from":2,"to":7}]},{"pgid":"11.36","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.325+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcb006b500 msgr2=0x7efcb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.325+0000 7efcd3200640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcb006b500 0x7efcb006d9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efcd4159f00 tx=0x7efcc4009040 comp rx=0 tx=0).stop 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.325+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd4159450 msgr2=0x7efcd415d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.325+0000 7efcd3200640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd4159450 0x7efcd415d310 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7efcd40ced30 tx=0x7efccc00df50 comp rx=0 tx=0).stop 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.325+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 shutdown_connections 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.325+0000 7efcd3200640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcb006b500 0x7efcb006d9c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.325+0000 7efcd3200640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd4159450 0x7efcd415d310 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.325+0000 7efcd3200640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd4157e40 0x7efcd415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.326+0000 7efcd3200640 1 --2- 172.21.15.62:0/488998929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb8004800 0x7efcd414bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.326+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 >> 172.21.15.62:0/488998929 conn(0x7efcd4066d10 msgr2=0x7efcd40b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.326+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 shutdown_connections 2024-04-25T09:12:07.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.326+0000 7efcd3200640 1 -- 172.21.15.62:0/488998929 wait complete. 2024-04-25T09:12:07.331 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:12:07.331 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.548+0000 7faeebe00640 1 Processor -- start 2024-04-25T09:12:07.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.549+0000 7faeebe00640 1 -- start start 2024-04-25T09:12:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.550+0000 7faeebe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faeec0b56f0 0x7faeec160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.550+0000 7faeebe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faeec0b6a90 0x7faeec161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.550+0000 7faeebe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec0d3110 con 0x7faeec0b27a0 2024-04-25T09:12:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.550+0000 7faeebe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeec161780 con 0x7faeec0b56f0 2024-04-25T09:12:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.550+0000 7faeebe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeec0cedc0 con 0x7faeec0b6a90 2024-04-25T09:12:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.550+0000 7faeeaa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faeec0b56f0 0x7faeec160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.551+0000 7faeeaa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7faeec0b56f0 0x7faeec160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43706/0 (socket says 172.21.15.62:43706) 2024-04-25T09:12:07.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.551+0000 7faeeaa00640 1 -- 172.21.15.62:0/1393560812 learned_addr learned my addr 172.21.15.62:0/1393560812 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.551+0000 7faeea000640 1 --2- 172.21.15.62:0/1393560812 >> v2:172.21.15.121:3300/0 conn(0x7faeec0b6a90 0x7faeec161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.551+0000 7faeeaa00640 1 -- 172.21.15.62:0/1393560812 >> v2:172.21.15.121:3300/0 conn(0x7faeec0b6a90 msgr2=0x7faeec161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.551+0000 7faeeaa00640 1 --2- 172.21.15.62:0/1393560812 >> v2:172.21.15.121:3300/0 conn(0x7faeec0b6a90 0x7faeec161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.551+0000 7faeeaa00640 1 -- 172.21.15.62:0/1393560812 >> v1:172.21.15.62:6789/0 conn(0x7faeec0b27a0 legacy=0x7faeec161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.551+0000 7faeeaa00640 1 -- 172.21.15.62:0/1393560812 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec0d33f0 con 0x7faeec0b56f0 2024-04-25T09:12:07.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.551+0000 7faeea000640 1 --2- 172.21.15.62:0/1393560812 >> v2:172.21.15.121:3300/0 conn(0x7faeec0b6a90 0x7faeec161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:07.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.552+0000 7faeeaa00640 1 --2- 172.21.15.62:0/1393560812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec160d00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7faeec08c570 tx=0x7faedc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72d219c2b41e7e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faee9600640 1 -- 172.21.15.62:0/1393560812 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc037070 con 0x7faeec0b56f0 2024-04-25T09:12:07.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faee9600640 1 -- 172.21.15.62:0/1393560812 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc032440 con 0x7faeec0b56f0 2024-04-25T09:12:07.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faee9600640 1 -- 172.21.15.62:0/1393560812 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc032810 con 0x7faeec0b56f0 2024-04-25T09:12:07.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faeebe00640 1 -- 172.21.15.62:0/1393560812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 msgr2=0x7faeec160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faeebe00640 1 --2- 172.21.15.62:0/1393560812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec160d00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7faeec08c570 tx=0x7faedc02a270 comp rx=0 tx=0).stop 2024-04-25T09:12:07.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faeebe00640 1 -- 172.21.15.62:0/1393560812 shutdown_connections 2024-04-25T09:12:07.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faeebe00640 1 --2- 172.21.15.62:0/1393560812 >> v2:172.21.15.121:3300/0 conn(0x7faeec0b6a90 0x7faeec161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faeebe00640 1 --2- 172.21.15.62:0/1393560812 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec160d00 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faeebe00640 1 -- 172.21.15.62:0/1393560812 >> 172.21.15.62:0/1393560812 conn(0x7faeec066d10 msgr2=0x7faeec067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.553+0000 7faeebe00640 1 -- 172.21.15.62:0/1393560812 shutdown_connections 2024-04-25T09:12:07.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.554+0000 7faeebe00640 1 -- 172.21.15.62:0/1393560812 wait complete. 2024-04-25T09:12:07.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.555+0000 7faeebe00640 1 Processor -- start 2024-04-25T09:12:07.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.555+0000 7faeebe00640 1 -- start start 2024-04-25T09:12:07.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.556+0000 7faeebe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faeec0b27a0 0x7faeec15f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.556+0000 7faeebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.556+0000 7faeebe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faeec0b6a90 0x7faeec0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.556+0000 7faeebe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0b2d60 con 0x7faeec0b56f0 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.556+0000 7faeebe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0cea50 con 0x7faeec0b27a0 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.556+0000 7faeebe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0b2ee0 con 0x7faeec0b6a90 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.556+0000 7faeea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43720/0 (socket says 172.21.15.62:43720) 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeea000640 1 -- 172.21.15.62:0/2675181924 learned_addr learned my addr 172.21.15.62:0/2675181924 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeeaa00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faeec0b27a0 0x7faeec15f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeea000640 1 -- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faeec0b6a90 msgr2=0x7faeec0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeea000640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faeec0b6a90 0x7faeec0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeea000640 1 -- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faeec0b27a0 msgr2=0x7faeec15f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeea000640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faeec0b27a0 0x7faeec15f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeea000640 1 -- 172.21.15.62:0/2675181924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec1f4090 con 0x7faeec0b56f0 2024-04-25T09:12:07.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faeea000640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec155f50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7faed400ce30 tx=0x7faed400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.557+0000 7faee3400640 1 -- 172.21.15.62:0/2675181924 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400f680 con 0x7faeec0b56f0 2024-04-25T09:12:07.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.558+0000 7faee3400640 1 -- 172.21.15.62:0/2675181924 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed400f820 con 0x7faeec0b56f0 2024-04-25T09:12:07.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.558+0000 7faee3400640 1 -- 172.21.15.62:0/2675181924 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400fc30 con 0x7faeec0b56f0 2024-04-25T09:12:07.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.558+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec1f4370 con 0x7faeec0b56f0 2024-04-25T09:12:07.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.558+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec1f4930 con 0x7faeec0b56f0 2024-04-25T09:12:07.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.559+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec04ef20 con 0x7faeec0b56f0 2024-04-25T09:12:07.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.560+0000 7faee3400640 1 -- 172.21.15.62:0/2675181924 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faed4018400 con 0x7faeec0b56f0 2024-04-25T09:12:07.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.561+0000 7faee3400640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:07.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.561+0000 7faee3400640 1 -- 172.21.15.62:0/2675181924 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faed408afe0 con 0x7faeec0b56f0 2024-04-25T09:12:07.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.562+0000 7faeeaa00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:07.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.563+0000 7faeeaa00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faec806b4d0 0x7faec806d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faedc034000 tx=0x7faedc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:07.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.566+0000 7faee3400640 1 -- 172.21.15.62:0/2675181924 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faed4014030 con 0x7faeec0b56f0 2024-04-25T09:12:07.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.797+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faeec0b79c0 con 0x7faeec0b56f0 2024-04-25T09:12:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.798+0000 7faee3400640 1 -- 172.21.15.62:0/2675181924 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4934 (secure 0 0 0) 0x7faed4059e50 con 0x7faeec0b56f0 2024-04-25T09:12:07.785 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:07.785 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5336,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2992229593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2992229593},{"type":"v1","addr":"172.21.15.62:6835","nonce":2992229593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5342,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/518754003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":518754003},{"type":"v1","addr":"172.21.15.121:6837","nonce":518754003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/6918584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":6918584},{"type":"v1","addr":"172.21.15.121:6836","nonce":6918584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:09:29.410378+0000","modified":"2024-04-25T09:09:38.647936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2258257596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2258257596},{"type":"v1","addr":"172.21.15.62:6834","nonce":2258257596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:12:07.785 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 55 2024-04-25T09:12:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faec806b4d0 msgr2=0x7faec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faec806b4d0 0x7faec806d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faedc034000 tx=0x7faedc02e000 comp rx=0 tx=0).stop 2024-04-25T09:12:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 msgr2=0x7faeec155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec155f50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7faed400ce30 tx=0x7faed400a650 comp rx=0 tx=0).stop 2024-04-25T09:12:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 shutdown_connections 2024-04-25T09:12:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faeec0b56f0 0x7faeec155f50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faeec0b6a90 0x7faeec0b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 --2- 172.21.15.62:0/2675181924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faeec0b27a0 0x7faeec15f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 >> 172.21.15.62:0/2675181924 conn(0x7faeec066d10 msgr2=0x7faeec0c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 shutdown_connections 2024-04-25T09:12:07.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:07.800+0000 7faeebe00640 1 -- 172.21.15.62:0/2675181924 wait complete. 2024-04-25T09:12:07.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:08.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.019+0000 7f5e1d000640 1 Processor -- start 2024-04-25T09:12:08.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.020+0000 7f5e1d000640 1 -- start start 2024-04-25T09:12:08.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.021+0000 7f5e1d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 0x7f5e180d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.021+0000 7f5e1d000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5e180b27a0 0x7f5e180d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.021+0000 7f5e1d000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e180d3110 con 0x7f5e180b56f0 2024-04-25T09:12:08.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.021+0000 7f5e1d000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e180cedc0 con 0x7f5e180b27a0 2024-04-25T09:12:08.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.021+0000 7f5e1d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180d33f0 con 0x7f5e180b6a90 2024-04-25T09:12:08.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.021+0000 7f5e17400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5e180b56f0 0x7f5e18160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59936/0 (socket says 172.21.15.62:59936) 2024-04-25T09:12:08.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.021+0000 7f5e17400640 1 -- 172.21.15.62:0/2164483565 learned_addr learned my addr 172.21.15.62:0/2164483565 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:08.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.022+0000 7f5e17e00640 1 --2- 172.21.15.62:0/2164483565 >> v2:172.21.15.62:3300/0 conn(0x7f5e180b27a0 0x7f5e180d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.022+0000 7f5e16a00640 1 --2- 172.21.15.62:0/2164483565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 0x7f5e180d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.022+0000 7f5e16000640 1 -- 172.21.15.62:0/2164483565 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604560978 0 0) 0x7f5e180d3110 con 0x7f5e180b56f0 2024-04-25T09:12:08.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.022+0000 7f5e16000640 1 -- 172.21.15.62:0/2164483565 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dfc003520 con 0x7f5e180b56f0 2024-04-25T09:12:08.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.022+0000 7f5e17e00640 1 -- 172.21.15.62:0/2164483565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 msgr2=0x7f5e180d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:08.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.022+0000 7f5e17e00640 1 --2- 172.21.15.62:0/2164483565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 0x7f5e180d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.022+0000 7f5e17e00640 1 -- 172.21.15.62:0/2164483565 >> v1:172.21.15.62:6789/0 conn(0x7f5e180b56f0 legacy=0x7f5e18160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.022+0000 7f5e17e00640 1 -- 172.21.15.62:0/2164483565 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180d36d0 con 0x7f5e180b27a0 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.023+0000 7f5e17e00640 1 --2- 172.21.15.62:0/2164483565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b27a0 0x7f5e180d2520 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5e100097a0 tx=0x7f5e1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d1301664173950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e16000640 1 -- 172.21.15.62:0/2164483565 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10037070 con 0x7f5e180b27a0 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e16000640 1 -- 172.21.15.62:0/2164483565 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10032440 con 0x7f5e180b27a0 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e16000640 1 -- 172.21.15.62:0/2164483565 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10032810 con 0x7f5e180b27a0 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e1d000640 1 -- 172.21.15.62:0/2164483565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b27a0 msgr2=0x7f5e180d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e1d000640 1 --2- 172.21.15.62:0/2164483565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b27a0 0x7f5e180d2520 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5e100097a0 tx=0x7f5e1002a180 comp rx=0 tx=0).stop 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e1d000640 1 -- 172.21.15.62:0/2164483565 shutdown_connections 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e1d000640 1 --2- 172.21.15.62:0/2164483565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b27a0 0x7f5e180d2520 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e1d000640 1 --2- 172.21.15.62:0/2164483565 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 0x7f5e180d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e1d000640 1 -- 172.21.15.62:0/2164483565 >> 172.21.15.62:0/2164483565 conn(0x7f5e18066d10 msgr2=0x7f5e18067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e1d000640 1 -- 172.21.15.62:0/2164483565 shutdown_connections 2024-04-25T09:12:08.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.024+0000 7f5e1d000640 1 -- 172.21.15.62:0/2164483565 wait complete. 2024-04-25T09:12:08.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.026+0000 7f5e1d000640 1 Processor -- start 2024-04-25T09:12:08.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.026+0000 7f5e1d000640 1 -- start start 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.027+0000 7f5e1d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e180b27a0 0x7f5e180c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.027+0000 7f5e1d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b56f0 0x7f5e18158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.027+0000 7f5e1d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 0x7f5e18158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.027+0000 7f5e1d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180b2ee0 con 0x7f5e180b56f0 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.027+0000 7f5e1d000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180b2d60 con 0x7f5e180b27a0 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.027+0000 7f5e1d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180b5ad0 con 0x7f5e180b6a90 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b56f0 0x7f5e18158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b56f0 0x7f5e18158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43734/0 (socket says 172.21.15.62:43734) 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 -- 172.21.15.62:0/394276525 learned_addr learned my addr 172.21.15.62:0/394276525 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e17e00640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 0x7f5e18158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 -- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 msgr2=0x7f5e18158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 0x7f5e18158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 -- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e180b27a0 msgr2=0x7f5e180c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e180b27a0 0x7f5e180c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 -- 172.21.15.62:0/394276525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180c9c40 con 0x7f5e180b56f0 2024-04-25T09:12:08.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.028+0000 7f5e16a00640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b56f0 0x7f5e18158050 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000d5f0 tx=0x7f5e0000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.029+0000 7f5e0f400640 1 -- 172.21.15.62:0/394276525 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000125f0 con 0x7f5e180b56f0 2024-04-25T09:12:08.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.029+0000 7f5e0f400640 1 -- 172.21.15.62:0/394276525 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00019070 con 0x7f5e180b56f0 2024-04-25T09:12:08.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.029+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e180c9f20 con 0x7f5e180b56f0 2024-04-25T09:12:08.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.029+0000 7f5e0f400640 1 -- 172.21.15.62:0/394276525 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000129d0 con 0x7f5e180b56f0 2024-04-25T09:12:08.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.029+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1815b220 con 0x7f5e180b56f0 2024-04-25T09:12:08.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.030+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1804ef20 con 0x7f5e180b56f0 2024-04-25T09:12:08.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.031+0000 7f5e0f400640 1 -- 172.21.15.62:0/394276525 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e00012b70 con 0x7f5e180b56f0 2024-04-25T09:12:08.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.032+0000 7f5e0f400640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5df406b4d0 0x7f5df406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.032+0000 7f5e0f400640 1 -- 172.21.15.62:0/394276525 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5e0008a660 con 0x7f5e180b56f0 2024-04-25T09:12:08.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.032+0000 7f5e17400640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5df406b4d0 0x7f5df406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.033+0000 7f5e17400640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5df406b4d0 0x7f5df406d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5e080012e0 tx=0x7f5e08001650 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.036+0000 7f5e0f400640 1 -- 172.21.15.62:0/394276525 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e00059470 con 0x7f5e180b56f0 2024-04-25T09:12:08.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.275+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e180ce580 con 0x7f5e180b56f0 2024-04-25T09:12:08.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.275+0000 7f5e0f400640 1 -- 172.21.15.62:0/394276525 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f5e00058d70 con 0x7f5e180b56f0 2024-04-25T09:12:08.262 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:08.262 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":55,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5336,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2992229593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2992229593},{"type":"v1","addr":"172.21.15.62:6835","nonce":2992229593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5342,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/518754003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":518754003},{"type":"v1","addr":"172.21.15.121:6837","nonce":518754003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/6918584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":6918584},{"type":"v1","addr":"172.21.15.121:6836","nonce":6918584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:09:29.410378+0000","modified":"2024-04-25T09:09:38.647936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2258257596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2258257596},{"type":"v1","addr":"172.21.15.62:6834","nonce":2258257596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:12:08.263 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 55 2024-04-25T09:12:08.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5df406b4d0 msgr2=0x7f5df406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5df406b4d0 0x7f5df406d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5e080012e0 tx=0x7f5e08001650 comp rx=0 tx=0).stop 2024-04-25T09:12:08.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b56f0 msgr2=0x7f5e18158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b56f0 0x7f5e18158050 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000d5f0 tx=0x7f5e0000dab0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 shutdown_connections 2024-04-25T09:12:08.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5df406b4d0 0x7f5df406d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e180b56f0 0x7f5e18158050 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5e180b6a90 0x7f5e18158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 --2- 172.21.15.62:0/394276525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e180b27a0 0x7f5e180c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 >> 172.21.15.62:0/394276525 conn(0x7f5e18066d10 msgr2=0x7f5e180b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 shutdown_connections 2024-04-25T09:12:08.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.278+0000 7f5e1d000640 1 -- 172.21.15.62:0/394276525 wait complete. 2024-04-25T09:12:08.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:08.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.496+0000 7f9893e00640 1 Processor -- start 2024-04-25T09:12:08.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.497+0000 7f9893e00640 1 -- start start 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.498+0000 7f9893e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f989406d900 0x7f989415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.498+0000 7f9893e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98940cee40 0x7f98940b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.498+0000 7f9893e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98940cf2b0 con 0x7f9894144c10 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.498+0000 7f9893e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9894144ff0 con 0x7f989406d900 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.498+0000 7f9893e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9894160340 con 0x7f98940cee40 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.499+0000 7f988a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f989406d900 0x7f989415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.499+0000 7f988a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f989406d900 0x7f989415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43740/0 (socket says 172.21.15.62:43740) 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.499+0000 7f988a000640 1 -- 172.21.15.62:0/3604270898 learned_addr learned my addr 172.21.15.62:0/3604270898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:08.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.499+0000 7f9893400640 1 --2- 172.21.15.62:0/3604270898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98940cee40 0x7f98940b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.499+0000 7f988a000640 1 -- 172.21.15.62:0/3604270898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98940cee40 msgr2=0x7f98940b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:08.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.499+0000 7f988a000640 1 --2- 172.21.15.62:0/3604270898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98940cee40 0x7f98940b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.499+0000 7f988a000640 1 -- 172.21.15.62:0/3604270898 >> v1:172.21.15.121:6789/0 conn(0x7f9894144c10 legacy=0x7f989406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:08.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.499+0000 7f988a000640 1 -- 172.21.15.62:0/3604270898 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989415faa0 con 0x7f989406d900 2024-04-25T09:12:08.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.500+0000 7f988a000640 1 --2- 172.21.15.62:0/3604270898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f989415f450 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f987c009870 tx=0x7f987c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8c74a4b64c3fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.501+0000 7f9892000640 1 -- 172.21.15.62:0/3604270898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c037070 con 0x7f989406d900 2024-04-25T09:12:08.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.501+0000 7f9892000640 1 -- 172.21.15.62:0/3604270898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c032440 con 0x7f989406d900 2024-04-25T09:12:08.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.501+0000 7f9892000640 1 -- 172.21.15.62:0/3604270898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c032810 con 0x7f989406d900 2024-04-25T09:12:08.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.501+0000 7f9893e00640 1 -- 172.21.15.62:0/3604270898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 msgr2=0x7f989415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.501+0000 7f9893e00640 1 --2- 172.21.15.62:0/3604270898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f989415f450 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f987c009870 tx=0x7f987c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.502+0000 7f9893e00640 1 -- 172.21.15.62:0/3604270898 shutdown_connections 2024-04-25T09:12:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.502+0000 7f9893e00640 1 --2- 172.21.15.62:0/3604270898 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f98940cee40 0x7f98940b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.502+0000 7f9893e00640 1 --2- 172.21.15.62:0/3604270898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f989415f450 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.502+0000 7f9893e00640 1 -- 172.21.15.62:0/3604270898 >> 172.21.15.62:0/3604270898 conn(0x7f98940b14d0 msgr2=0x7f98940b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.502+0000 7f9893e00640 1 -- 172.21.15.62:0/3604270898 shutdown_connections 2024-04-25T09:12:08.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.502+0000 7f9893e00640 1 -- 172.21.15.62:0/3604270898 wait complete. 2024-04-25T09:12:08.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.503+0000 7f9893e00640 1 Processor -- start 2024-04-25T09:12:08.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.503+0000 7f9893e00640 1 -- start start 2024-04-25T09:12:08.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.504+0000 7f9893e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f9894132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.504+0000 7f9893e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98940cee40 0x7f98941e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.504+0000 7f9893e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9894144c10 0x7f98941e9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.504+0000 7f9893e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9894160060 con 0x7f989406d900 2024-04-25T09:12:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.504+0000 7f9893e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98940b41c0 con 0x7f98940cee40 2024-04-25T09:12:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.504+0000 7f9893e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98940b4b90 con 0x7f9894144c10 2024-04-25T09:12:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9892a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f9894132eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9892a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f9894132eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43744/0 (socket says 172.21.15.62:43744) 2024-04-25T09:12:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9892a00640 1 -- 172.21.15.62:0/3447610024 learned_addr learned my addr 172.21.15.62:0/3447610024 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9893400640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9894144c10 0x7f98941e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f988a000640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98940cee40 0x7f98941e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9892a00640 1 -- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9894144c10 msgr2=0x7f98941e9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9892a00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9894144c10 0x7f98941e9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9892a00640 1 -- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98940cee40 msgr2=0x7f98941e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9892a00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98940cee40 0x7f98941e7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.505+0000 7f9892a00640 1 -- 172.21.15.62:0/3447610024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98941333f0 con 0x7f989406d900 2024-04-25T09:12:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.506+0000 7f9892a00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f9894132eb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f988400c430 tx=0x7f988400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.506+0000 7f988b400640 1 -- 172.21.15.62:0/3447610024 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884017070 con 0x7f989406d900 2024-04-25T09:12:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.506+0000 7f988b400640 1 -- 172.21.15.62:0/3447610024 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98840052d0 con 0x7f989406d900 2024-04-25T09:12:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.506+0000 7f988b400640 1 -- 172.21.15.62:0/3447610024 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884003830 con 0x7f989406d900 2024-04-25T09:12:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.506+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98941e8c30 con 0x7f989406d900 2024-04-25T09:12:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.506+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98941e8950 con 0x7f989406d900 2024-04-25T09:12:08.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.507+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989404ef20 con 0x7f989406d900 2024-04-25T09:12:08.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.508+0000 7f988b400640 1 -- 172.21.15.62:0/3447610024 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f988401a070 con 0x7f989406d900 2024-04-25T09:12:08.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.509+0000 7f988b400640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f987006b4d0 0x7f987006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.509+0000 7f988b400640 1 -- 172.21.15.62:0/3447610024 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f98840895e0 con 0x7f989406d900 2024-04-25T09:12:08.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.510+0000 7f988a000640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f987006b4d0 0x7f987006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.511+0000 7f988a000640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f987006b4d0 0x7f987006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f987c034000 tx=0x7f987c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.514+0000 7f988b400640 1 -- 172.21.15.62:0/3447610024 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9884058400 con 0x7f989406d900 2024-04-25T09:12:08.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.715+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98940b19e0 con 0x7f989406d900 2024-04-25T09:12:08.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.718+0000 7f988b400640 1 -- 172.21.15.62:0/3447610024 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16760 (secure 0 0 0) 0x7f9884057d00 con 0x7f989406d900 2024-04-25T09:12:08.705 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:08.705 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:11:37.590166+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:09:27.235650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T09:09:28.419955+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":85,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1b","mappings":[{"from":2,"to":7}]},{"pgid":"11.36","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:08.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.720+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f987006b4d0 msgr2=0x7f987006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.720+0000 7f9893e00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f987006b4d0 0x7f987006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f987c034000 tx=0x7f987c02e000 comp rx=0 tx=0).stop 2024-04-25T09:12:08.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.720+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 msgr2=0x7f9894132eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.720+0000 7f9893e00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f9894132eb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f988400c430 tx=0x7f988400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.721+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 shutdown_connections 2024-04-25T09:12:08.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.721+0000 7f9893e00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f987006b4d0 0x7f987006d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.721+0000 7f9893e00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9894144c10 0x7f98941e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.721+0000 7f9893e00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98940cee40 0x7f98941e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.721+0000 7f9893e00640 1 --2- 172.21.15.62:0/3447610024 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f989406d900 0x7f9894132eb0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.721+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 >> 172.21.15.62:0/3447610024 conn(0x7f98940b14d0 msgr2=0x7f98940ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.721+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 shutdown_connections 2024-04-25T09:12:08.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.721+0000 7f9893e00640 1 -- 172.21.15.62:0/3447610024 wait complete. 2024-04-25T09:12:08.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:12:08.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.955+0000 7f35b3e00640 1 Processor -- start 2024-04-25T09:12:08.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.955+0000 7f35b3e00640 1 -- start start 2024-04-25T09:12:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b3e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35b40cfaf0 0x7f35b415e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f35b40adf60 0x7f35b40ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b3e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b415e640 con 0x7f35b406cc60 2024-04-25T09:12:08.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b3e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b40ada60 con 0x7f35b40adf60 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b3e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b415e8e0 con 0x7f35b40cfaf0 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f35b40adf60 0x7f35b40ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f35b40adf60 0x7f35b40ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43746/0 (socket says 172.21.15.62:43746) 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b3400640 1 -- 172.21.15.62:0/3889214164 learned_addr learned my addr 172.21.15.62:0/3889214164 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.957+0000 7f35b2a00640 1 --2- 172.21.15.62:0/3889214164 >> v2:172.21.15.121:3300/0 conn(0x7f35b40cfaf0 0x7f35b415e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.958+0000 7f35b3400640 1 -- 172.21.15.62:0/3889214164 >> v2:172.21.15.121:3300/0 conn(0x7f35b40cfaf0 msgr2=0x7f35b415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.958+0000 7f35b3400640 1 --2- 172.21.15.62:0/3889214164 >> v2:172.21.15.121:3300/0 conn(0x7f35b40cfaf0 0x7f35b415e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.958+0000 7f35b3400640 1 -- 172.21.15.62:0/3889214164 >> v1:172.21.15.121:6789/0 conn(0x7f35b406cc60 legacy=0x7f35b40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:08.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.958+0000 7f35b3400640 1 -- 172.21.15.62:0/3889214164 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b415ebc0 con 0x7f35b40adf60 2024-04-25T09:12:08.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.959+0000 7f35b3400640 1 --2- 172.21.15.62:0/3889214164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b40adf60 0x7f35b40ae3f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f35b40677a0 tx=0x7f35ac02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c55ca86978b2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.959+0000 7f35b1600640 1 -- 172.21.15.62:0/3889214164 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac036070 con 0x7f35b40adf60 2024-04-25T09:12:08.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.959+0000 7f35b1600640 1 -- 172.21.15.62:0/3889214164 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac031600 con 0x7f35b40adf60 2024-04-25T09:12:08.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.959+0000 7f35b1600640 1 -- 172.21.15.62:0/3889214164 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0319d0 con 0x7f35b40adf60 2024-04-25T09:12:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.960+0000 7f35b3e00640 1 -- 172.21.15.62:0/3889214164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b40adf60 msgr2=0x7f35b40ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.960+0000 7f35b3e00640 1 --2- 172.21.15.62:0/3889214164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b40adf60 0x7f35b40ae3f0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f35b40677a0 tx=0x7f35ac02a780 comp rx=0 tx=0).stop 2024-04-25T09:12:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.960+0000 7f35b3e00640 1 -- 172.21.15.62:0/3889214164 shutdown_connections 2024-04-25T09:12:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.960+0000 7f35b3e00640 1 --2- 172.21.15.62:0/3889214164 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b40adf60 0x7f35b40ae3f0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.960+0000 7f35b3e00640 1 --2- 172.21.15.62:0/3889214164 >> v2:172.21.15.121:3300/0 conn(0x7f35b40cfaf0 0x7f35b415e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.960+0000 7f35b3e00640 1 -- 172.21.15.62:0/3889214164 >> 172.21.15.62:0/3889214164 conn(0x7f35b40668f0 msgr2=0x7f35b4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.960+0000 7f35b3e00640 1 -- 172.21.15.62:0/3889214164 shutdown_connections 2024-04-25T09:12:08.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.960+0000 7f35b3e00640 1 -- 172.21.15.62:0/3889214164 wait complete. 2024-04-25T09:12:08.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.962+0000 7f35b3e00640 1 Processor -- start 2024-04-25T09:12:08.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.962+0000 7f35b3e00640 1 -- start start 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.963+0000 7f35b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b406cc60 0x7f35b41f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.963+0000 7f35b3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35b40adf60 0x7f35b40ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.963+0000 7f35b3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35b40cfaf0 0x7f35b4157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.963+0000 7f35b3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40b3150 con 0x7f35b406cc60 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.963+0000 7f35b3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40b2fd0 con 0x7f35b40adf60 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.963+0000 7f35b3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b40cfed0 con 0x7f35b40cfaf0 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.963+0000 7f35b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b406cc60 0x7f35b41f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b406cc60 0x7f35b41f09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43760/0 (socket says 172.21.15.62:43760) 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2a00640 1 -- 172.21.15.62:0/1972793538 learned_addr learned my addr 172.21.15.62:0/1972793538 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b3400640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35b40cfaf0 0x7f35b4157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2a00640 1 -- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35b40cfaf0 msgr2=0x7f35b4157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2a00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35b40cfaf0 0x7f35b4157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2000640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35b40adf60 0x7f35b40ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2a00640 1 -- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35b40adf60 msgr2=0x7f35b40ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:08.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2a00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35b40adf60 0x7f35b40ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:08.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2a00640 1 -- 172.21.15.62:0/1972793538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b41f3a10 con 0x7f35b406cc60 2024-04-25T09:12:08.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2000640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35b40adf60 0x7f35b40ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:08.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.964+0000 7f35b2a00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b406cc60 0x7f35b41f09d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f35a400d5f0 tx=0x7f35a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.965+0000 7f35aaa00640 1 -- 172.21.15.62:0/1972793538 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40125f0 con 0x7f35b406cc60 2024-04-25T09:12:08.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.965+0000 7f35aaa00640 1 -- 172.21.15.62:0/1972793538 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a4019070 con 0x7f35b406cc60 2024-04-25T09:12:08.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.965+0000 7f35aaa00640 1 -- 172.21.15.62:0/1972793538 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40129d0 con 0x7f35b406cc60 2024-04-25T09:12:08.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.965+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b41f3cf0 con 0x7f35b406cc60 2024-04-25T09:12:08.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.965+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b41f36c0 con 0x7f35b406cc60 2024-04-25T09:12:08.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.966+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b404ef20 con 0x7f35b406cc60 2024-04-25T09:12:08.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.967+0000 7f35aaa00640 1 -- 172.21.15.62:0/1972793538 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35a4012b70 con 0x7f35b406cc60 2024-04-25T09:12:08.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.968+0000 7f35aaa00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f359406b4d0 0x7f359406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:08.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.969+0000 7f35aaa00640 1 -- 172.21.15.62:0/1972793538 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f35a408a640 con 0x7f35b406cc60 2024-04-25T09:12:08.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.969+0000 7f35b2000640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f359406b4d0 0x7f359406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:08.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.971+0000 7f35b2000640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f359406b4d0 0x7f359406d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35b4158aa0 tx=0x7f35a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:08.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:08.974+0000 7f35aaa00640 1 -- 172.21.15.62:0/1972793538 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35a4059450 con 0x7f35b406cc60 2024-04-25T09:12:09.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.214+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f35b4159e30 con 0x7f35b406cc60 2024-04-25T09:12:09.202 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:12:09.215+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:12:09.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.219+0000 7f35aaa00640 1 -- 172.21.15.62:0/1972793538 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f35a4058d50 con 0x7f35b406cc60 2024-04-25T09:12:09.205 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:12:09.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.222+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f359406b4d0 msgr2=0x7f359406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.222+0000 7f35b3e00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f359406b4d0 0x7f359406d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35b4158aa0 tx=0x7f35a0009040 comp rx=0 tx=0).stop 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.222+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b406cc60 msgr2=0x7f35b41f09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.222+0000 7f35b3e00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b406cc60 0x7f35b41f09d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f35a400d5f0 tx=0x7f35a400dab0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.223+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 shutdown_connections 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.223+0000 7f35b3e00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f359406b4d0 0x7f359406d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.223+0000 7f35b3e00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f35b40cfaf0 0x7f35b4157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.223+0000 7f35b3e00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35b40adf60 0x7f35b40ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.223+0000 7f35b3e00640 1 --2- 172.21.15.62:0/1972793538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35b406cc60 0x7f35b41f09d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.223+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 >> 172.21.15.62:0/1972793538 conn(0x7f35b40668f0 msgr2=0x7f35b4071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.223+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 shutdown_connections 2024-04-25T09:12:09.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.223+0000 7f35b3e00640 1 -- 172.21.15.62:0/1972793538 wait complete. 2024-04-25T09:12:09.237 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:12:09.295 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:12:09.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.438+0000 7f0238c00640 1 Processor -- start 2024-04-25T09:12:09.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.438+0000 7f0238c00640 1 -- start start 2024-04-25T09:12:09.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.440+0000 7f0238c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0234162cc0 0x7f023406c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.440+0000 7f0238c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023406e560 con 0x7f02340b40e0 2024-04-25T09:12:09.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.440+0000 7f0238c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02340cf340 con 0x7f02340ceeb0 2024-04-25T09:12:09.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.440+0000 7f0238c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023406dd00 con 0x7f0234162cc0 2024-04-25T09:12:09.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.440+0000 7f0231e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f02340b40e0 0x7f02341625b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59948/0 (socket says 172.21.15.62:59948) 2024-04-25T09:12:09.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.440+0000 7f0231e00640 1 -- 172.21.15.62:0/2699686823 learned_addr learned my addr 172.21.15.62:0/2699686823 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:09.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.440+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196724954 0 0) 0x7f023406e560 con 0x7f02340b40e0 2024-04-25T09:12:09.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0218003520 con 0x7f02340b40e0 2024-04-25T09:12:09.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0229400640 1 --2- 172.21.15.62:0/2699686823 >> v2:172.21.15.121:3300/0 conn(0x7f0234162cc0 0x7f023406c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 364739310 0 0) 0x7f0218003520 con 0x7f02340b40e0 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 >> v2:172.21.15.121:3300/0 conn(0x7f0234162cc0 msgr2=0x7f023406c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 --2- 172.21.15.62:0/2699686823 >> v2:172.21.15.121:3300/0 conn(0x7f0234162cc0 0x7f023406c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 >> v1:172.21.15.121:6789/0 conn(0x7f02340ceeb0 legacy=0x7f023416bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023406dfa0 con 0x7f02340b40e0 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f0224001670 con 0x7f02340b40e0 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 >> v1:172.21.15.62:6789/0 conn(0x7f02340b40e0 legacy=0x7f02341625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180039b0 0x7f0218003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0218004800 0x7f0218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180052e0 0x7f021800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023406dd00 con 0x7f0218004800 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021800c230 con 0x7f02180039b0 2024-04-25T09:12:09.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021800c430 con 0x7f02180052e0 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.441+0000 7f0229400640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0218004800 0x7f0218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.442+0000 7f0231e00640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180039b0 0x7f0218003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.442+0000 7f0232800640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180052e0 0x7f021800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.442+0000 7f0229400640 1 -- 172.21.15.62:0/2699686823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180052e0 msgr2=0x7f021800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.442+0000 7f0229400640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02180052e0 0x7f021800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.442+0000 7f0229400640 1 -- 172.21.15.62:0/2699686823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180039b0 msgr2=0x7f0218003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.442+0000 7f0229400640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02180039b0 0x7f0218003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.442+0000 7f0229400640 1 -- 172.21.15.62:0/2699686823 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0218003520 con 0x7f0218004800 2024-04-25T09:12:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.442+0000 7f0232800640 1 -- 172.21.15.62:0/2699686823 reap_dead start 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.443+0000 7f0229400640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0218004800 0x7f0218004c80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f021c00ba00 tx=0x7f0218005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ca9f9daa4374be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.443+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c030070 con 0x7f0218004800 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.443+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c02ada0 con 0x7f0218004800 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.443+0000 7f0231400640 1 -- 172.21.15.62:0/2699686823 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c00b600 con 0x7f0218004800 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.444+0000 7f0238c00640 1 -- 172.21.15.62:0/2699686823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0218004800 msgr2=0x7f0218004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.444+0000 7f0238c00640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0218004800 0x7f0218004c80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f021c00ba00 tx=0x7f0218005830 comp rx=0 tx=0).stop 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.444+0000 7f0238c00640 1 -- 172.21.15.62:0/2699686823 shutdown_connections 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.444+0000 7f0238c00640 1 --2- 172.21.15.62:0/2699686823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0218004800 0x7f0218004c80 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.444+0000 7f0238c00640 1 -- 172.21.15.62:0/2699686823 >> 172.21.15.62:0/2699686823 conn(0x7f02340b1410 msgr2=0x7f02340b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.444+0000 7f0238c00640 1 -- 172.21.15.62:0/2699686823 shutdown_connections 2024-04-25T09:12:09.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.444+0000 7f0238c00640 1 -- 172.21.15.62:0/2699686823 wait complete. 2024-04-25T09:12:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.446+0000 7f0238c00640 1 Processor -- start 2024-04-25T09:12:09.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.446+0000 7f0238c00640 1 -- start start 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.447+0000 7f0238c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0218004800 0x7f0234146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.447+0000 7f0238c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02340af5d0 0x7f02340cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.447+0000 7f0238c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02340cc590 0x7f02340cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.447+0000 7f0238c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0234072460 con 0x7f02340cc590 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.447+0000 7f0238c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0234071f40 con 0x7f02340af5d0 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.447+0000 7f0238c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340725e0 con 0x7f0218004800 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.447+0000 7f0232800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02340cc590 0x7f02340cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0232800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02340cc590 0x7f02340cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43780/0 (socket says 172.21.15.62:43780) 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0232800640 1 -- 172.21.15.62:0/36279968 learned_addr learned my addr 172.21.15.62:0/36279968 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:09.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0229400640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02340af5d0 0x7f02340cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0231e00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0218004800 0x7f0234146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0232800640 1 -- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0218004800 msgr2=0x7f0234146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0232800640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0218004800 0x7f0234146240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0232800640 1 -- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02340af5d0 msgr2=0x7f02340cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0232800640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02340af5d0 0x7f02340cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0232800640 1 -- 172.21.15.62:0/36279968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234146780 con 0x7f02340cc590 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.448+0000 7f0232800640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02340cc590 0x7f02340cca10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f02341637e0 tx=0x7f022c00d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.449+0000 7f022aa00640 1 -- 172.21.15.62:0/36279968 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c012800 con 0x7f02340cc590 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.449+0000 7f022aa00640 1 -- 172.21.15.62:0/36279968 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c019070 con 0x7f02340cc590 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.449+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02340cdfe0 con 0x7f02340cc590 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.449+0000 7f022aa00640 1 -- 172.21.15.62:0/36279968 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c012c00 con 0x7f02340cc590 2024-04-25T09:12:09.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.449+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02340afe20 con 0x7f02340cc590 2024-04-25T09:12:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.450+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023404ef20 con 0x7f02340cc590 2024-04-25T09:12:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.451+0000 7f022aa00640 1 -- 172.21.15.62:0/36279968 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f022c01e460 con 0x7f02340cc590 2024-04-25T09:12:09.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.452+0000 7f022aa00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f021006b4d0 0x7f021006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:09.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.452+0000 7f022aa00640 1 -- 172.21.15.62:0/36279968 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f022c089fe0 con 0x7f02340cc590 2024-04-25T09:12:09.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.453+0000 7f0231e00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f021006b4d0 0x7f021006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:09.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.454+0000 7f0231e00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f021006b4d0 0x7f021006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0224001eb0 tx=0x7f0224009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:09.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.456+0000 7f022aa00640 1 -- 172.21.15.62:0/36279968 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0234146780 con 0x7f02340cc590 2024-04-25T09:12:09.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:09.698+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0234162a70 con 0x7f02340cc590 2024-04-25T09:12:10.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.222+0000 7f022aa00640 1 -- 172.21.15.62:0/36279968 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f022c058d90 con 0x7f02340cc590 2024-04-25T09:12:10.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f021006b4d0 msgr2=0x7f021006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f021006b4d0 0x7f021006d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0224001eb0 tx=0x7f0224009040 comp rx=0 tx=0).stop 2024-04-25T09:12:10.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02340cc590 msgr2=0x7f02340cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02340cc590 0x7f02340cca10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f02341637e0 tx=0x7f022c00d9b0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 shutdown_connections 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f021006b4d0 0x7f021006d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02340cc590 0x7f02340cca10 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02340af5d0 0x7f02340cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.226+0000 7f0238c00640 1 --2- 172.21.15.62:0/36279968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0218004800 0x7f0234146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.227+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 >> 172.21.15.62:0/36279968 conn(0x7f02340b1410 msgr2=0x7f0234164220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.227+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 shutdown_connections 2024-04-25T09:12:10.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.227+0000 7f0238c00640 1 -- 172.21.15.62:0/36279968 wait complete. 2024-04-25T09:12:10.235 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:12:10.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.445+0000 7fe04ec00640 1 Processor -- start 2024-04-25T09:12:10.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.446+0000 7fe04ec00640 1 -- start start 2024-04-25T09:12:10.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.447+0000 7fe04ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 0x7fe0480c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.447+0000 7fe04ec00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0480b43d0 con 0x7fe0480ce470 2024-04-25T09:12:10.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.448+0000 7fe04ec00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0480c7630 con 0x7fe0480cc0b0 2024-04-25T09:12:10.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.448+0000 7fe04ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04815e9d0 con 0x7fe04815e570 2024-04-25T09:12:10.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.448+0000 7fe047e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe0480ce470 0x7fe0480ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59958/0 (socket says 172.21.15.62:59958) 2024-04-25T09:12:10.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.448+0000 7fe047e00640 1 -- 172.21.15.62:0/2254711440 learned_addr learned my addr 172.21.15.62:0/2254711440 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:10.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.448+0000 7fe03f400640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 0x7fe0480c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.448+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137549278 0 0) 0x7fe0480b43d0 con 0x7fe0480ce470 2024-04-25T09:12:10.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.448+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe02c003520 con 0x7fe0480ce470 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1480089980 0 0) 0x7fe02c003520 con 0x7fe0480ce470 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 msgr2=0x7fe0480c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 0x7fe0480c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 >> v1:172.21.15.121:6789/0 conn(0x7fe0480cc0b0 legacy=0x7fe0480cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0480c7910 con 0x7fe0480ce470 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fe038001670 con 0x7fe0480ce470 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 >> v1:172.21.15.62:6789/0 conn(0x7fe0480ce470 legacy=0x7fe0480ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe02c0039e0 0x7fe02c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 0x7fe02c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe02c004d40 0x7fe02c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04815e9d0 con 0x7fe02c0039e0 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c00ba70 con 0x7fe02c004d40 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c00bc70 con 0x7fe04815e570 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.449+0000 7fe047e00640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe02c0039e0 0x7fe02c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.450+0000 7fe04cc00640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe02c004d40 0x7fe02c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.450+0000 7fe03f400640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 0x7fe02c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.450+0000 7fe047e00640 1 -- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 msgr2=0x7fe02c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.450+0000 7fe047e00640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 0x7fe02c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.450+0000 7fe047e00640 1 -- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe02c004d40 msgr2=0x7fe02c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.450+0000 7fe047e00640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe02c004d40 0x7fe02c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.450+0000 7fe047e00640 1 -- 172.21.15.62:0/2254711440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c003520 con 0x7fe02c0039e0 2024-04-25T09:12:10.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.450+0000 7fe03f400640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe04815e570 0x7fe02c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:10.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.451+0000 7fe047e00640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe02c0039e0 0x7fe02c003e00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe038001eb0 tx=0x7fe02c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63ed8e6d72bafe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038007d30 con 0x7fe02c0039e0 2024-04-25T09:12:10.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0380058d0 con 0x7fe02c0039e0 2024-04-25T09:12:10.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe047400640 1 -- 172.21.15.62:0/2254711440 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038005ce0 con 0x7fe02c0039e0 2024-04-25T09:12:10.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe04ec00640 1 -- 172.21.15.62:0/2254711440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe02c0039e0 msgr2=0x7fe02c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe04ec00640 1 --2- 172.21.15.62:0/2254711440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe02c0039e0 0x7fe02c003e00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe038001eb0 tx=0x7fe02c005830 comp rx=0 tx=0).stop 2024-04-25T09:12:10.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe04cc00640 1 -- 172.21.15.62:0/2254711440 reap_dead start 2024-04-25T09:12:10.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe04ec00640 1 -- 172.21.15.62:0/2254711440 shutdown_connections 2024-04-25T09:12:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe04ec00640 1 -- 172.21.15.62:0/2254711440 >> 172.21.15.62:0/2254711440 conn(0x7fe0480b3690 msgr2=0x7fe0480b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe04ec00640 1 -- 172.21.15.62:0/2254711440 shutdown_connections 2024-04-25T09:12:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.452+0000 7fe04ec00640 1 -- 172.21.15.62:0/2254711440 wait complete. 2024-04-25T09:12:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.454+0000 7fe04ec00640 1 Processor -- start 2024-04-25T09:12:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.454+0000 7fe04ec00640 1 -- start start 2024-04-25T09:12:10.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.455+0000 7fe04ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02c0039e0 0x7fe0480c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.455+0000 7fe04ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe04814fa20 0x7fe0480b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.455+0000 7fe04ec00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0480b6c60 0x7fe0480b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.455+0000 7fe04ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480c81b0 con 0x7fe04814fa20 2024-04-25T09:12:10.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.455+0000 7fe04ec00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480c7ed0 con 0x7fe0480b6c60 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.455+0000 7fe04ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480c86d0 con 0x7fe02c0039e0 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe04814fa20 0x7fe0480b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe04814fa20 0x7fe0480b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43804/0 (socket says 172.21.15.62:43804) 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 -- 172.21.15.62:0/4262496197 learned_addr learned my addr 172.21.15.62:0/4262496197 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe04cc00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0480b6c60 0x7fe0480b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe047e00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02c0039e0 0x7fe0480c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 -- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02c0039e0 msgr2=0x7fe0480c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02c0039e0 0x7fe0480c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 -- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0480b6c60 msgr2=0x7fe0480b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0480b6c60 0x7fe0480b70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 -- 172.21.15.62:0/4262496197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0480b4c00 con 0x7fe04814fa20 2024-04-25T09:12:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe04cc00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0480b6c60 0x7fe0480b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.456+0000 7fe03f400640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe04814fa20 0x7fe0480b46c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe0300108b0 tx=0x7fe030010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.457+0000 7fe044c00640 1 -- 172.21.15.62:0/4262496197 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03000f040 con 0x7fe04814fa20 2024-04-25T09:12:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.457+0000 7fe044c00640 1 -- 172.21.15.62:0/4262496197 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03000eb00 con 0x7fe04814fa20 2024-04-25T09:12:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.457+0000 7fe044c00640 1 -- 172.21.15.62:0/4262496197 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03001c400 con 0x7fe04814fa20 2024-04-25T09:12:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.457+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0480b4e80 con 0x7fe04814fa20 2024-04-25T09:12:10.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.457+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0480b5440 con 0x7fe04814fa20 2024-04-25T09:12:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.458+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe048067c40 con 0x7fe04814fa20 2024-04-25T09:12:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.459+0000 7fe044c00640 1 -- 172.21.15.62:0/4262496197 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe03001c5a0 con 0x7fe04814fa20 2024-04-25T09:12:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.460+0000 7fe044c00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe02406b550 0x7fe02406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.460+0000 7fe044c00640 1 -- 172.21.15.62:0/4262496197 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe03008a6d0 con 0x7fe04814fa20 2024-04-25T09:12:10.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.461+0000 7fe047e00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe02406b550 0x7fe02406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:10.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.462+0000 7fe047e00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe02406b550 0x7fe02406da10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe038033000 tx=0x7fe03802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:10.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.464+0000 7fe044c00640 1 -- 172.21.15.62:0/4262496197 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0480b4c00 con 0x7fe04814fa20 2024-04-25T09:12:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:10.658+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe04815dda0 con 0x7fe04814fa20 2024-04-25T09:12:11.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.226+0000 7fe044c00640 1 -- 172.21.15.62:0/4262496197 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe030018030 con 0x7fe04814fa20 2024-04-25T09:12:11.213 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:12:11.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.230+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe02406b550 msgr2=0x7fe02406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.230+0000 7fe04ec00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe02406b550 0x7fe02406da10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe038033000 tx=0x7fe03802e000 comp rx=0 tx=0).stop 2024-04-25T09:12:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.230+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe04814fa20 msgr2=0x7fe0480b46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.230+0000 7fe04ec00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe04814fa20 0x7fe0480b46c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe0300108b0 tx=0x7fe030010d70 comp rx=0 tx=0).stop 2024-04-25T09:12:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.231+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 shutdown_connections 2024-04-25T09:12:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.231+0000 7fe04ec00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe02406b550 0x7fe02406da10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.231+0000 7fe04ec00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe04814fa20 0x7fe0480b46c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.231+0000 7fe04ec00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0480b6c60 0x7fe0480b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.231+0000 7fe04ec00640 1 --2- 172.21.15.62:0/4262496197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe02c0039e0 0x7fe0480c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.231+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 >> 172.21.15.62:0/4262496197 conn(0x7fe0480b3690 msgr2=0x7fe04806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.231+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 shutdown_connections 2024-04-25T09:12:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.232+0000 7fe04ec00640 1 -- 172.21.15.62:0/4262496197 wait complete. 2024-04-25T09:12:11.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:12:11.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.445+0000 7f8aef200640 1 Processor -- start 2024-04-25T09:12:11.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.446+0000 7f8aef200640 1 -- start start 2024-04-25T09:12:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.447+0000 7f8aef200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161270 0x7f8af00b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.447+0000 7f8aef200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8af0161650 0x7f8af00b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.447+0000 7f8aef200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8af00b2980 0x7f8af0161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.447+0000 7f8aef200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af00d3900 con 0x7f8af0161650 2024-04-25T09:12:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.447+0000 7f8aef200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af00d1480 con 0x7f8af00b2980 2024-04-25T09:12:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.447+0000 7f8aef200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00d11a0 con 0x7f8af0161270 2024-04-25T09:12:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aed400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8af0161650 0x7f8af00b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aed400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8af0161650 0x7f8af00b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43816/0 (socket says 172.21.15.62:43816) 2024-04-25T09:12:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aed400640 1 -- 172.21.15.62:0/1336450303 learned_addr learned my addr 172.21.15.62:0/1336450303 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:11.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aee800640 1 --2- 172.21.15.62:0/1336450303 >> v2:172.21.15.121:3300/0 conn(0x7f8af00b2980 0x7f8af0161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aede00640 1 --2- 172.21.15.62:0/1336450303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161270 0x7f8af00b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aed400640 1 -- 172.21.15.62:0/1336450303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161270 msgr2=0x7f8af00b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aed400640 1 --2- 172.21.15.62:0/1336450303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161270 0x7f8af00b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aed400640 1 -- 172.21.15.62:0/1336450303 >> v2:172.21.15.121:3300/0 conn(0x7f8af00b2980 msgr2=0x7f8af0161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aed400640 1 --2- 172.21.15.62:0/1336450303 >> v2:172.21.15.121:3300/0 conn(0x7f8af00b2980 0x7f8af0161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.448+0000 7f8aed400640 1 -- 172.21.15.62:0/1336450303 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00cee80 con 0x7f8af0161650 2024-04-25T09:12:11.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.450+0000 7f8aed400640 1 --2- 172.21.15.62:0/1336450303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161650 0x7f8af00b5aa0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8af00d4af0 tx=0x7f8adc02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120f846ba93f9dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.450+0000 7f8aeca00640 1 -- 172.21.15.62:0/1336450303 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc036070 con 0x7f8af0161650 2024-04-25T09:12:11.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.450+0000 7f8aeca00640 1 -- 172.21.15.62:0/1336450303 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc031600 con 0x7f8af0161650 2024-04-25T09:12:11.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.450+0000 7f8aeca00640 1 -- 172.21.15.62:0/1336450303 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0319d0 con 0x7f8af0161650 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 -- 172.21.15.62:0/1336450303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161650 msgr2=0x7f8af00b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 --2- 172.21.15.62:0/1336450303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161650 0x7f8af00b5aa0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8af00d4af0 tx=0x7f8adc02a570 comp rx=0 tx=0).stop 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 -- 172.21.15.62:0/1336450303 shutdown_connections 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 --2- 172.21.15.62:0/1336450303 >> v2:172.21.15.121:3300/0 conn(0x7f8af00b2980 0x7f8af0161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 --2- 172.21.15.62:0/1336450303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161650 0x7f8af00b5aa0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 --2- 172.21.15.62:0/1336450303 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161270 0x7f8af00b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 -- 172.21.15.62:0/1336450303 >> 172.21.15.62:0/1336450303 conn(0x7f8af0066d10 msgr2=0x7f8af0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 -- 172.21.15.62:0/1336450303 shutdown_connections 2024-04-25T09:12:11.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.451+0000 7f8aef200640 1 -- 172.21.15.62:0/1336450303 wait complete. 2024-04-25T09:12:11.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.453+0000 7f8aef200640 1 Processor -- start 2024-04-25T09:12:11.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.453+0000 7f8aef200640 1 -- start start 2024-04-25T09:12:11.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aef200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8af00b2980 0x7f8af00ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aef200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161270 0x7f8af01f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aef200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161650 0x7f8af00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aef200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b2ed0 con 0x7f8af0161270 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aef200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00d3590 con 0x7f8af00b2980 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aef200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00b3050 con 0x7f8af0161650 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aed400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161270 0x7f8af01f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aed400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161270 0x7f8af01f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43830/0 (socket says 172.21.15.62:43830) 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.454+0000 7f8aed400640 1 -- 172.21.15.62:0/2614540449 learned_addr learned my addr 172.21.15.62:0/2614540449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.455+0000 7f8aede00640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8af00b2980 0x7f8af00ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.455+0000 7f8aed400640 1 -- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161650 msgr2=0x7f8af00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.455+0000 7f8aed400640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161650 0x7f8af00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.455+0000 7f8aed400640 1 -- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8af00b2980 msgr2=0x7f8af00ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.455+0000 7f8aed400640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8af00b2980 0x7f8af00ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.455+0000 7f8aed400640 1 -- 172.21.15.62:0/2614540449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00af9f0 con 0x7f8af0161270 2024-04-25T09:12:11.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.455+0000 7f8aede00640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8af00b2980 0x7f8af00ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:11.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.455+0000 7f8aed400640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161270 0x7f8af01f7410 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8adc034040 tx=0x7f8adc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.456+0000 7f8ae6000640 1 -- 172.21.15.62:0/2614540449 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc036050 con 0x7f8af0161270 2024-04-25T09:12:11.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.456+0000 7f8ae6000640 1 -- 172.21.15.62:0/2614540449 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc042070 con 0x7f8af0161270 2024-04-25T09:12:11.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.456+0000 7f8ae6000640 1 -- 172.21.15.62:0/2614540449 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc009510 con 0x7f8af0161270 2024-04-25T09:12:11.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.456+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af01f7bb0 con 0x7f8af0161270 2024-04-25T09:12:11.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.456+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af01f8170 con 0x7f8af0161270 2024-04-25T09:12:11.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.457+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af0072910 con 0x7f8af0161270 2024-04-25T09:12:11.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.458+0000 7f8ae6000640 1 -- 172.21.15.62:0/2614540449 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8adc007c30 con 0x7f8af0161270 2024-04-25T09:12:11.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.459+0000 7f8ae6000640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ad006b4d0 0x7f8ad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:11.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.459+0000 7f8ae6000640 1 -- 172.21.15.62:0/2614540449 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f8adc0ab080 con 0x7f8af0161270 2024-04-25T09:12:11.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.460+0000 7f8aede00640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ad006b4d0 0x7f8ad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:11.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.461+0000 7f8aede00640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ad006b4d0 0x7f8ad006d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0006fd0 tx=0x7f8ae0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:11.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.465+0000 7f8ae6000640 1 -- 172.21.15.62:0/2614540449 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8af00af9f0 con 0x7f8af0161270 2024-04-25T09:12:11.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:11.662+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8af00750e0 con 0x7f8af0161270 2024-04-25T09:12:12.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.229+0000 7f8ae6000640 1 -- 172.21.15.62:0/2614540449 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8adc045360 con 0x7f8af0161270 2024-04-25T09:12:12.217 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:12:12.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ad006b4d0 msgr2=0x7f8ad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ad006b4d0 0x7f8ad006d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0006fd0 tx=0x7f8ae0008040 comp rx=0 tx=0).stop 2024-04-25T09:12:12.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161270 msgr2=0x7f8af01f7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161270 0x7f8af01f7410 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8adc034040 tx=0x7f8adc004a40 comp rx=0 tx=0).stop 2024-04-25T09:12:12.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 shutdown_connections 2024-04-25T09:12:12.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ad006b4d0 0x7f8ad006d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af0161270 0x7f8af01f7410 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8af0161650 0x7f8af00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 --2- 172.21.15.62:0/2614540449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8af00b2980 0x7f8af00ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.233+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 >> 172.21.15.62:0/2614540449 conn(0x7f8af0066d10 msgr2=0x7f8af00b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:12.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.234+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 shutdown_connections 2024-04-25T09:12:12.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.234+0000 7f8aef200640 1 -- 172.21.15.62:0/2614540449 wait complete. 2024-04-25T09:12:12.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:12:12.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.466+0000 7f8da2000640 1 Processor -- start 2024-04-25T09:12:12.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.467+0000 7f8da2000640 1 -- start start 2024-04-25T09:12:12.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.469+0000 7f8da2000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8d9c0b2d80 0x7f8d9c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.469+0000 7f8da2000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c0b4430 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.469+0000 7f8da2000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c0b4710 con 0x7f8d9c0b36a0 2024-04-25T09:12:12.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.469+0000 7f8da2000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b49f0 con 0x7f8d9c0b2d80 2024-04-25T09:12:12.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.469+0000 7f8da1600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8d9c0b54d0 0x7f8d9c164310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59968/0 (socket says 172.21.15.62:59968) 2024-04-25T09:12:12.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.469+0000 7f8da1600640 1 -- 172.21.15.62:0/3576046659 learned_addr learned my addr 172.21.15.62:0/3576046659 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:12.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.470+0000 7f8da0c00640 1 --2- 172.21.15.62:0/3576046659 >> v2:172.21.15.121:3300/0 conn(0x7f8d9c0b2d80 0x7f8d9c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.471+0000 7f8da0c00640 1 -- 172.21.15.62:0/3576046659 >> v1:172.21.15.121:6789/0 conn(0x7f8d9c0b36a0 legacy=0x7f8d9c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:12.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.471+0000 7f8da0c00640 1 -- 172.21.15.62:0/3576046659 >> v1:172.21.15.62:6789/0 conn(0x7f8d9c0b54d0 legacy=0x7f8d9c164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.471+0000 7f8da0c00640 1 -- 172.21.15.62:0/3576046659 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0b4cd0 con 0x7f8d9c0b2d80 2024-04-25T09:12:12.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.472+0000 7f8da0c00640 1 --2- 172.21.15.62:0/3576046659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b2d80 0x7f8d9c0b3160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8d900097a0 tx=0x7f8d9002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a5124b62154ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.473+0000 7f8d9b400640 1 -- 172.21.15.62:0/3576046659 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90037070 con 0x7f8d9c0b2d80 2024-04-25T09:12:12.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.473+0000 7f8d9b400640 1 -- 172.21.15.62:0/3576046659 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d90032440 con 0x7f8d9c0b2d80 2024-04-25T09:12:12.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.473+0000 7f8d9b400640 1 -- 172.21.15.62:0/3576046659 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90032810 con 0x7f8d9c0b2d80 2024-04-25T09:12:12.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.474+0000 7f8da2000640 1 -- 172.21.15.62:0/3576046659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b2d80 msgr2=0x7f8d9c0b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.474+0000 7f8da2000640 1 --2- 172.21.15.62:0/3576046659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b2d80 0x7f8d9c0b3160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8d900097a0 tx=0x7f8d9002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.474+0000 7f8da2000640 1 -- 172.21.15.62:0/3576046659 shutdown_connections 2024-04-25T09:12:12.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.474+0000 7f8da2000640 1 --2- 172.21.15.62:0/3576046659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b2d80 0x7f8d9c0b3160 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.474+0000 7f8da2000640 1 -- 172.21.15.62:0/3576046659 >> 172.21.15.62:0/3576046659 conn(0x7f8d9c066f20 msgr2=0x7f8d9c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:12.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.475+0000 7f8da2000640 1 -- 172.21.15.62:0/3576046659 shutdown_connections 2024-04-25T09:12:12.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.475+0000 7f8da2000640 1 -- 172.21.15.62:0/3576046659 wait complete. 2024-04-25T09:12:12.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.477+0000 7f8da2000640 1 Processor -- start 2024-04-25T09:12:12.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.477+0000 7f8da2000640 1 -- start start 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da2000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d9c0b2d80 0x7f8d9c15aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da2000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b36a0 0x7f8d9c1e6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d9c0b54d0 0x7f8d9c149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da2000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0ce9c0 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da2000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0b40e0 con 0x7f8d9c0b36a0 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da2000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0ceb40 con 0x7f8d9c0b2d80 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d9c0b54d0 0x7f8d9c149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d9c0b54d0 0x7f8d9c149020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43844/0 (socket says 172.21.15.62:43844) 2024-04-25T09:12:12.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.478+0000 7f8da1600640 1 -- 172.21.15.62:0/3968918611 learned_addr learned my addr 172.21.15.62:0/3968918611 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:12.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.479+0000 7f8da0c00640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d9c0b2d80 0x7f8d9c15aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.479+0000 7f8d9be00640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b36a0 0x7f8d9c1e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.479+0000 7f8da1600640 1 -- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d9c0b2d80 msgr2=0x7f8d9c15aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.479+0000 7f8da1600640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d9c0b2d80 0x7f8d9c15aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.479+0000 7f8da1600640 1 -- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b36a0 msgr2=0x7f8d9c1e6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.479+0000 7f8da1600640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b36a0 0x7f8d9c1e6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.479+0000 7f8da1600640 1 -- 172.21.15.62:0/3968918611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c1e75f0 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.479+0000 7f8da1600640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d9c0b54d0 0x7f8d9c149020 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400ca80 tx=0x7f8d9400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.480+0000 7f8d98c00640 1 -- 172.21.15.62:0/3968918611 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94017070 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.480+0000 7f8d98c00640 1 -- 172.21.15.62:0/3968918611 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9400ac30 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.480+0000 7f8d98c00640 1 -- 172.21.15.62:0/3968918611 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94012600 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.480+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c148830 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.480+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c15b430 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.481+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c06abe0 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.482+0000 7f8d98c00640 1 -- 172.21.15.62:0/3968918611 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d9401a070 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.483+0000 7f8d98c00640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.483+0000 7f8d98c00640 1 -- 172.21.15.62:0/3968918611 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d94089b50 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.484+0000 7f8da0c00640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.485+0000 7f8da0c00640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8d90034000 tx=0x7f8d9002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.488+0000 7f8d98c00640 1 -- 172.21.15.62:0/3968918611 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d9400ae20 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.691+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8d9c158e60 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.699+0000 7f8d98c00640 1 -- 172.21.15.62:0/3968918611 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8d940585e0 con 0x7f8d9c0b54d0 2024-04-25T09:12:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d7c06b4d0 msgr2=0x7f8d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8d90034000 tx=0x7f8d9002e000 comp rx=0 tx=0).stop 2024-04-25T09:12:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d9c0b54d0 msgr2=0x7f8d9c149020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d9c0b54d0 0x7f8d9c149020 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400ca80 tx=0x7f8d9400cf40 comp rx=0 tx=0).stop 2024-04-25T09:12:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 shutdown_connections 2024-04-25T09:12:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8d7c06b4d0 0x7f8d7c06d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d9c0b54d0 0x7f8d9c149020 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c0b36a0 0x7f8d9c1e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 --2- 172.21.15.62:0/3968918611 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8d9c0b2d80 0x7f8d9c15aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 >> 172.21.15.62:0/3968918611 conn(0x7f8d9c066f20 msgr2=0x7f8d9c15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 shutdown_connections 2024-04-25T09:12:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.702+0000 7f8da2000640 1 -- 172.21.15.62:0/3968918611 wait complete. 2024-04-25T09:12:12.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T09:12:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.933+0000 7f4743000640 1 Processor -- start 2024-04-25T09:12:12.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.933+0000 7f4743000640 1 -- start start 2024-04-25T09:12:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.935+0000 7f4743000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c0cfed0 0x7f473c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.935+0000 7f4743000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f473c072340 0x7f473c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.935+0000 7f4743000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473c06c310 con 0x7f473c0707f0 2024-04-25T09:12:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.935+0000 7f4743000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c15f5f0 con 0x7f473c072340 2024-04-25T09:12:12.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.935+0000 7f4743000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c06c5b0 con 0x7f473c0cfed0 2024-04-25T09:12:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.936+0000 7f4741000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f473c0707f0 0x7f473c15b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59976/0 (socket says 172.21.15.62:59976) 2024-04-25T09:12:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.936+0000 7f4741000640 1 -- 172.21.15.62:0/3720404082 learned_addr learned my addr 172.21.15.62:0/3720404082 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.936+0000 7f473be00640 1 --2- 172.21.15.62:0/3720404082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c0cfed0 0x7f473c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.936+0000 7f4733e00640 1 --2- 172.21.15.62:0/3720404082 >> v2:172.21.15.62:3300/0 conn(0x7f473c072340 0x7f473c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.937+0000 7f473be00640 1 -- 172.21.15.62:0/3720404082 >> v2:172.21.15.62:3300/0 conn(0x7f473c072340 msgr2=0x7f473c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.937+0000 7f473b400640 1 -- 172.21.15.62:0/3720404082 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960192931 0 0) 0x7f473c06c310 con 0x7f473c0707f0 2024-04-25T09:12:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.937+0000 7f473be00640 1 --2- 172.21.15.62:0/3720404082 >> v2:172.21.15.62:3300/0 conn(0x7f473c072340 0x7f473c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.937+0000 7f473be00640 1 -- 172.21.15.62:0/3720404082 >> v1:172.21.15.62:6789/0 conn(0x7f473c0707f0 legacy=0x7f473c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.937+0000 7f473be00640 1 -- 172.21.15.62:0/3720404082 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c06c890 con 0x7f473c0cfed0 2024-04-25T09:12:12.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.938+0000 7f473be00640 1 --2- 172.21.15.62:0/3720404082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c0cfed0 0x7f473c15f0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f473c06c4e0 tx=0x7f472c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a439fe8b262de22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.939+0000 7f473b400640 1 -- 172.21.15.62:0/3720404082 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c037070 con 0x7f473c0cfed0 2024-04-25T09:12:12.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.939+0000 7f473b400640 1 -- 172.21.15.62:0/3720404082 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c032440 con 0x7f473c0cfed0 2024-04-25T09:12:12.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.939+0000 7f473b400640 1 -- 172.21.15.62:0/3720404082 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c032810 con 0x7f473c0cfed0 2024-04-25T09:12:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.939+0000 7f4743000640 1 -- 172.21.15.62:0/3720404082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c0cfed0 msgr2=0x7f473c15f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.939+0000 7f4743000640 1 --2- 172.21.15.62:0/3720404082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c0cfed0 0x7f473c15f0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f473c06c4e0 tx=0x7f472c02a180 comp rx=0 tx=0).stop 2024-04-25T09:12:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.940+0000 7f4743000640 1 -- 172.21.15.62:0/3720404082 shutdown_connections 2024-04-25T09:12:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.940+0000 7f4743000640 1 --2- 172.21.15.62:0/3720404082 >> v2:172.21.15.62:3300/0 conn(0x7f473c072340 0x7f473c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.940+0000 7f4743000640 1 --2- 172.21.15.62:0/3720404082 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c0cfed0 0x7f473c15f0b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.940+0000 7f4743000640 1 -- 172.21.15.62:0/3720404082 >> 172.21.15.62:0/3720404082 conn(0x7f473c0668f0 msgr2=0x7f473c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.940+0000 7f4743000640 1 -- 172.21.15.62:0/3720404082 shutdown_connections 2024-04-25T09:12:12.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.940+0000 7f4743000640 1 -- 172.21.15.62:0/3720404082 wait complete. 2024-04-25T09:12:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.941+0000 7f4743000640 1 Processor -- start 2024-04-25T09:12:12.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.942+0000 7f4743000640 1 -- start start 2024-04-25T09:12:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4743000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f473c0707f0 0x7f473c13e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4743000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473c0cfed0 0x7f473c140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4743000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c1435b0 0x7f473c14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4743000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c06d130 con 0x7f473c0cfed0 2024-04-25T09:12:12.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4743000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c06ce50 con 0x7f473c0707f0 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4743000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0b3890 con 0x7f473c1435b0 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4733e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473c0cfed0 0x7f473c140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4733e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473c0cfed0 0x7f473c140810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43862/0 (socket says 172.21.15.62:43862) 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4733e00640 1 -- 172.21.15.62:0/1488267453 learned_addr learned my addr 172.21.15.62:0/1488267453 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f473be00640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f473c0707f0 0x7f473c13e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.943+0000 7f4741000640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c1435b0 0x7f473c14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4733e00640 1 -- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c1435b0 msgr2=0x7f473c14d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4733e00640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c1435b0 0x7f473c14d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4733e00640 1 -- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f473c0707f0 msgr2=0x7f473c13e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4733e00640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f473c0707f0 0x7f473c13e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4733e00640 1 -- 172.21.15.62:0/1488267453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c13e8c0 con 0x7f473c0cfed0 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4741000640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c1435b0 0x7f473c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:12.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4733e00640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473c0cfed0 0x7f473c140810 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f472400d800 tx=0x7f472400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4738c00640 1 -- 172.21.15.62:0/1488267453 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472400b6c0 con 0x7f473c0cfed0 2024-04-25T09:12:12.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4738c00640 1 -- 172.21.15.62:0/1488267453 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4724019070 con 0x7f473c0cfed0 2024-04-25T09:12:12.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4738c00640 1 -- 172.21.15.62:0/1488267453 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472400ba80 con 0x7f473c0cfed0 2024-04-25T09:12:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.944+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c13eba0 con 0x7f473c0cfed0 2024-04-25T09:12:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.945+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c13f160 con 0x7f473c0cfed0 2024-04-25T09:12:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.946+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c072c70 con 0x7f473c0cfed0 2024-04-25T09:12:12.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.947+0000 7f4738c00640 1 -- 172.21.15.62:0/1488267453 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f472400bc20 con 0x7f473c0cfed0 2024-04-25T09:12:12.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.948+0000 7f4738c00640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f471806b4d0 0x7f471806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:12.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.948+0000 7f4738c00640 1 -- 172.21.15.62:0/1488267453 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f472408a450 con 0x7f473c0cfed0 2024-04-25T09:12:12.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.948+0000 7f473be00640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f471806b4d0 0x7f471806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:12.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.950+0000 7f473be00640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f471806b4d0 0x7f471806d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f472c034000 tx=0x7f472c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:12.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:12.952+0000 7f4738c00640 1 -- 172.21.15.62:0/1488267453 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4724059530 con 0x7f473c0cfed0 2024-04-25T09:12:13.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:13.228+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f473c06d950 con 0x7f473c0cfed0 2024-04-25T09:12:14.181 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:14.190 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:14.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.234+0000 7f4738c00640 1 -- 172.21.15.62:0/1488267453 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v400) v1 ==== 158+0+0 (secure 0 0 0) 0x7f4724058e30 con 0x7f473c0cfed0 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f471806b4d0 msgr2=0x7f471806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f471806b4d0 0x7f471806d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f472c034000 tx=0x7f472c02e000 comp rx=0 tx=0).stop 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473c0cfed0 msgr2=0x7f473c140810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473c0cfed0 0x7f473c140810 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f472400d800 tx=0x7f472400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 shutdown_connections 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f471806b4d0 0x7f471806d990 secure :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f472c034000 tx=0x7f472c02e000 comp rx=0 tx=0).stop 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f473c0cfed0 0x7f473c140810 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f473c1435b0 0x7f473c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 --2- 172.21.15.62:0/1488267453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f473c0707f0 0x7f473c13e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 >> 172.21.15.62:0/1488267453 conn(0x7f473c0668f0 msgr2=0x7f473c0cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 shutdown_connections 2024-04-25T09:12:14.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.237+0000 7f4743000640 1 -- 172.21.15.62:0/1488267453 wait complete. 2024-04-25T09:12:14.249 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T09:12:14.249 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:12:14.250 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.510+0000 7f0763200640 1 Processor -- start 2024-04-25T09:12:14.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.510+0000 7f0763200640 1 -- start start 2024-04-25T09:12:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.512+0000 7f0763200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076413d970 0x7f07640b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.512+0000 7f0763200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f076406d900 0x7f076413e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.512+0000 7f0763200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076415b220 con 0x7f07640b1050 2024-04-25T09:12:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.512+0000 7f0763200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076415b500 con 0x7f076406d900 2024-04-25T09:12:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.512+0000 7f0763200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076415b7e0 con 0x7f076413d970 2024-04-25T09:12:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.512+0000 7f0761400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f07640b1050 0x7f076406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59990/0 (socket says 172.21.15.62:59990) 2024-04-25T09:12:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.512+0000 7f0761400640 1 -- 172.21.15.62:0/1382549083 learned_addr learned my addr 172.21.15.62:0/1382549083 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0762800640 1 --2- 172.21.15.62:0/1382549083 >> v2:172.21.15.62:3300/0 conn(0x7f076406d900 0x7f076413e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0761e00640 1 --2- 172.21.15.62:0/1382549083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076413d970 0x7f07640b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0760a00640 1 -- 172.21.15.62:0/1382549083 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854068677 0 0) 0x7f076415b220 con 0x7f07640b1050 2024-04-25T09:12:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0760a00640 1 -- 172.21.15.62:0/1382549083 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0744005730 con 0x7f07640b1050 2024-04-25T09:12:14.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0762800640 1 -- 172.21.15.62:0/1382549083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076413d970 msgr2=0x7f07640b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0762800640 1 --2- 172.21.15.62:0/1382549083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076413d970 0x7f07640b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0762800640 1 -- 172.21.15.62:0/1382549083 >> v1:172.21.15.62:6789/0 conn(0x7f07640b1050 legacy=0x7f076406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0762800640 1 -- 172.21.15.62:0/1382549083 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076415bac0 con 0x7f076406d900 2024-04-25T09:12:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.513+0000 7f0761e00640 1 --2- 172.21.15.62:0/1382549083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076413d970 0x7f07640b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:14.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.515+0000 7f0762800640 1 --2- 172.21.15.62:0/1382549083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076406d900 0x7f076413e250 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f075c0097a0 tx=0x7f075c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4dff866c4f7a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.515+0000 7f0760a00640 1 -- 172.21.15.62:0/1382549083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c037070 con 0x7f076406d900 2024-04-25T09:12:14.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.515+0000 7f0760a00640 1 -- 172.21.15.62:0/1382549083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c032440 con 0x7f076406d900 2024-04-25T09:12:14.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.515+0000 7f0760a00640 1 -- 172.21.15.62:0/1382549083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c0327f0 con 0x7f076406d900 2024-04-25T09:12:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.516+0000 7f0763200640 1 -- 172.21.15.62:0/1382549083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076406d900 msgr2=0x7f076413e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.516+0000 7f0763200640 1 --2- 172.21.15.62:0/1382549083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076406d900 0x7f076413e250 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f075c0097a0 tx=0x7f075c02a180 comp rx=0 tx=0).stop 2024-04-25T09:12:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.516+0000 7f0763200640 1 -- 172.21.15.62:0/1382549083 shutdown_connections 2024-04-25T09:12:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.516+0000 7f0763200640 1 --2- 172.21.15.62:0/1382549083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076406d900 0x7f076413e250 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.516+0000 7f0763200640 1 --2- 172.21.15.62:0/1382549083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f076413d970 0x7f07640b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.516+0000 7f0763200640 1 -- 172.21.15.62:0/1382549083 >> 172.21.15.62:0/1382549083 conn(0x7f07640b14d0 msgr2=0x7f07640b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.516+0000 7f0763200640 1 -- 172.21.15.62:0/1382549083 shutdown_connections 2024-04-25T09:12:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.516+0000 7f0763200640 1 -- 172.21.15.62:0/1382549083 wait complete. 2024-04-25T09:12:14.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.518+0000 7f0763200640 1 Processor -- start 2024-04-25T09:12:14.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.518+0000 7f0763200640 1 -- start start 2024-04-25T09:12:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0763200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f076406d900 0x7f07641f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0763200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07640b1050 0x7f076413c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0763200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076413d970 0x7f07640b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0763200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076413e790 con 0x7f076413d970 2024-04-25T09:12:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0763200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076415af00 con 0x7f076406d900 2024-04-25T09:12:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0763200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076413e910 con 0x7f07640b1050 2024-04-25T09:12:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0762800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076413d970 0x7f07640b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0762800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076413d970 0x7f07640b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43872/0 (socket says 172.21.15.62:43872) 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.519+0000 7f0762800640 1 -- 172.21.15.62:0/3243161666 learned_addr learned my addr 172.21.15.62:0/3243161666 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0761400640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07640b1050 0x7f076413c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0761e00640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f076406d900 0x7f07641f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0762800640 1 -- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07640b1050 msgr2=0x7f076413c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0762800640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07640b1050 0x7f076413c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0762800640 1 -- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f076406d900 msgr2=0x7f07641f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0762800640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f076406d900 0x7f07641f7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0762800640 1 -- 172.21.15.62:0/3243161666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764153bb0 con 0x7f076413d970 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0761400640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07640b1050 0x7f076413c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0762800640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076413d970 0x7f07640b41d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f075c035040 tx=0x7f075c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f075a000640 1 -- 172.21.15.62:0/3243161666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c041030 con 0x7f076413d970 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f075a000640 1 -- 172.21.15.62:0/3243161666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c030440 con 0x7f076413d970 2024-04-25T09:12:14.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f075a000640 1 -- 172.21.15.62:0/3243161666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c037040 con 0x7f076413d970 2024-04-25T09:12:14.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.520+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0764153e90 con 0x7f076413d970 2024-04-25T09:12:14.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.521+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07640aff50 con 0x7f076413d970 2024-04-25T09:12:14.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.522+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076404ef20 con 0x7f076413d970 2024-04-25T09:12:14.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.523+0000 7f075a000640 1 -- 172.21.15.62:0/3243161666 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f075c044070 con 0x7f076413d970 2024-04-25T09:12:14.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.524+0000 7f075a000640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f074006b4d0 0x7f074006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.524+0000 7f075a000640 1 -- 172.21.15.62:0/3243161666 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f075c0aa6d0 con 0x7f076413d970 2024-04-25T09:12:14.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.524+0000 7f0761e00640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f074006b4d0 0x7f074006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.526+0000 7f0761e00640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f074006b4d0 0x7f074006d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f07540096d0 tx=0x7f0754008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.529+0000 7f075a000640 1 -- 172.21.15.62:0/3243161666 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f075c0798d0 con 0x7f076413d970 2024-04-25T09:12:14.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.732+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f076415d6a0 con 0x7f076413d970 2024-04-25T09:12:14.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.734+0000 7f075a000640 1 -- 172.21.15.62:0/3243161666 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+13009 (secure 0 0 0) 0x7f075c0791d0 con 0x7f076413d970 2024-04-25T09:12:14.722 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:14.722 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":89,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:12.226038+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/2258257596":"2024-04-26T09:12:09.215718+0000","172.21.15.62:6832/2258257596":"2024-04-26T09:12:09.215718+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:14.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f074006b4d0 msgr2=0x7f074006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f074006b4d0 0x7f074006d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f07540096d0 tx=0x7f0754008040 comp rx=0 tx=0).stop 2024-04-25T09:12:14.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076413d970 msgr2=0x7f07640b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076413d970 0x7f07640b41d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f075c035040 tx=0x7f075c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 shutdown_connections 2024-04-25T09:12:14.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f074006b4d0 0x7f074006d990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f076413d970 0x7f07640b41d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07640b1050 0x7f076413c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 --2- 172.21.15.62:0/3243161666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f076406d900 0x7f07641f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 >> 172.21.15.62:0/3243161666 conn(0x7f07640b14d0 msgr2=0x7f07640b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:14.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.737+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 shutdown_connections 2024-04-25T09:12:14.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.738+0000 7f0763200640 1 -- 172.21.15.62:0/3243161666 wait complete. 2024-04-25T09:12:14.745 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:14.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.978+0000 7f63a8000640 1 Processor -- start 2024-04-25T09:12:14.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.978+0000 7f63a8000640 1 -- start start 2024-04-25T09:12:14.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a8000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f63a006dc40 0x7f63a006e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a8000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 0x7f63a015dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a8000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63a015af50 0x7f63a015c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a8000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a006e560 con 0x7f63a015af50 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a8000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a015cbb0 con 0x7f63a006dc40 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a8000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a015ce90 con 0x7f63a0155150 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63a015af50 0x7f63a015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63a015af50 0x7f63a015c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41132/0 (socket says 172.21.15.62:41132) 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.980+0000 7f63a6000640 1 -- 172.21.15.62:0/887641590 learned_addr learned my addr 172.21.15.62:0/887641590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.981+0000 7f63a5600640 1 --2- 172.21.15.62:0/887641590 >> v2:172.21.15.121:3300/0 conn(0x7f63a006dc40 0x7f63a006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.981+0000 7f63a4c00640 1 --2- 172.21.15.62:0/887641590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 0x7f63a015dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.981+0000 7f63a6000640 1 -- 172.21.15.62:0/887641590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 msgr2=0x7f63a015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.981+0000 7f63a6000640 1 --2- 172.21.15.62:0/887641590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 0x7f63a015dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.981+0000 7f63a6000640 1 -- 172.21.15.62:0/887641590 >> v2:172.21.15.121:3300/0 conn(0x7f63a006dc40 msgr2=0x7f63a006e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.981+0000 7f63a6000640 1 --2- 172.21.15.62:0/887641590 >> v2:172.21.15.121:3300/0 conn(0x7f63a006dc40 0x7f63a006e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.981+0000 7f63a6000640 1 -- 172.21.15.62:0/887641590 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a015d170 con 0x7f63a015af50 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.982+0000 7f63a6000640 1 --2- 172.21.15.62:0/887641590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a015af50 0x7f63a015c360 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f63a00678d0 tx=0x7f639802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969f0f396ec7392a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.982+0000 7f639fe00640 1 -- 172.21.15.62:0/887641590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398036070 con 0x7f63a015af50 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.982+0000 7f639fe00640 1 -- 172.21.15.62:0/887641590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6398031600 con 0x7f63a015af50 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.982+0000 7f639fe00640 1 -- 172.21.15.62:0/887641590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63980319d0 con 0x7f63a015af50 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 -- 172.21.15.62:0/887641590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a015af50 msgr2=0x7f63a015c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 --2- 172.21.15.62:0/887641590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a015af50 0x7f63a015c360 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f63a00678d0 tx=0x7f639802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 -- 172.21.15.62:0/887641590 shutdown_connections 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 --2- 172.21.15.62:0/887641590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a015af50 0x7f63a015c360 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 --2- 172.21.15.62:0/887641590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 0x7f63a015dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 --2- 172.21.15.62:0/887641590 >> v2:172.21.15.121:3300/0 conn(0x7f63a006dc40 0x7f63a006e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 -- 172.21.15.62:0/887641590 >> 172.21.15.62:0/887641590 conn(0x7f63a00b1410 msgr2=0x7f63a00b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 -- 172.21.15.62:0/887641590 shutdown_connections 2024-04-25T09:12:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.983+0000 7f63a8000640 1 -- 172.21.15.62:0/887641590 wait complete. 2024-04-25T09:12:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.985+0000 7f63a8000640 1 Processor -- start 2024-04-25T09:12:14.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.985+0000 7f63a8000640 1 -- start start 2024-04-25T09:12:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a8000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a006dc40 0x7f63a00b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a8000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 0x7f63a0151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a8000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63a015af50 0x7f63a0151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a8000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a015e190 con 0x7f63a006dc40 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a8000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a015d730 con 0x7f63a015af50 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a8000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a015c8a0 con 0x7f63a0155150 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a006dc40 0x7f63a00b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a006dc40 0x7f63a00b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41142/0 (socket says 172.21.15.62:41142) 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.986+0000 7f63a5600640 1 -- 172.21.15.62:0/2541929440 learned_addr learned my addr 172.21.15.62:0/2541929440 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.987+0000 7f63a5600640 1 -- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 msgr2=0x7f63a0151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.987+0000 7f63a5600640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 0x7f63a0151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.987+0000 7f63a5600640 1 -- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63a015af50 msgr2=0x7f63a0151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.987+0000 7f63a5600640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63a015af50 0x7f63a0151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.987+0000 7f63a5600640 1 -- 172.21.15.62:0/2541929440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a0152570 con 0x7f63a006dc40 2024-04-25T09:12:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.987+0000 7f63a5600640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a006dc40 0x7f63a00b09c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f639400d5f0 tx=0x7f639400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.987+0000 7f639d600640 1 -- 172.21.15.62:0/2541929440 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940125f0 con 0x7f63a006dc40 2024-04-25T09:12:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.988+0000 7f639d600640 1 -- 172.21.15.62:0/2541929440 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6394019070 con 0x7f63a006dc40 2024-04-25T09:12:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.988+0000 7f639d600640 1 -- 172.21.15.62:0/2541929440 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940129d0 con 0x7f63a006dc40 2024-04-25T09:12:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.988+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a0159a20 con 0x7f63a006dc40 2024-04-25T09:12:14.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.988+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a0159fe0 con 0x7f63a006dc40 2024-04-25T09:12:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.989+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a006fd80 con 0x7f63a006dc40 2024-04-25T09:12:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.990+0000 7f639d600640 1 -- 172.21.15.62:0/2541929440 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6394012b70 con 0x7f63a006dc40 2024-04-25T09:12:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.991+0000 7f639d600640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f638006b4d0 0x7f638006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.991+0000 7f639d600640 1 -- 172.21.15.62:0/2541929440 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f639408a2f0 con 0x7f63a006dc40 2024-04-25T09:12:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.992+0000 7f63a4c00640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f638006b4d0 0x7f638006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.993+0000 7f63a4c00640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f638006b4d0 0x7f638006d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f63a0152cf0 tx=0x7f638c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:14.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:14.996+0000 7f639d600640 1 -- 172.21.15.62:0/2541929440 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63a006fd80 con 0x7f63a006dc40 2024-04-25T09:12:15.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.195+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63a004ef20 con 0x7f63a006dc40 2024-04-25T09:12:15.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.196+0000 7f639d600640 1 -- 172.21.15.62:0/2541929440 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6394059470 con 0x7f63a006dc40 2024-04-25T09:12:15.184 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f638006b4d0 msgr2=0x7f638006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f638006b4d0 0x7f638006d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f63a0152cf0 tx=0x7f638c008040 comp rx=0 tx=0).stop 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a006dc40 msgr2=0x7f63a00b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a006dc40 0x7f63a00b09c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f639400d5f0 tx=0x7f639400dab0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 shutdown_connections 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f638006b4d0 0x7f638006d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63a015af50 0x7f63a0151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63a0155150 0x7f63a0151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 --2- 172.21.15.62:0/2541929440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63a006dc40 0x7f63a00b09c0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 >> 172.21.15.62:0/2541929440 conn(0x7f63a00b1410 msgr2=0x7f63a00b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:15.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 shutdown_connections 2024-04-25T09:12:15.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.199+0000 7f63a8000640 1 -- 172.21.15.62:0/2541929440 wait complete. 2024-04-25T09:12:15.208 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T09:12:15.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.434+0000 7f7d73200640 1 Processor -- start 2024-04-25T09:12:15.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.434+0000 7f7d73200640 1 -- start start 2024-04-25T09:12:15.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d73200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7d6c15bd10 0x7f7d6c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d73200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0aeed0 0x7f7d6c0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d73200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6c06ca70 con 0x7f7d6c0cccf0 2024-04-25T09:12:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d73200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d6c06d660 con 0x7f7d6c15bd10 2024-04-25T09:12:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d73200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c15c0f0 con 0x7f7d6c0aeed0 2024-04-25T09:12:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d63e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7d6c0cccf0 0x7f7d6c0ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38502/0 (socket says 172.21.15.62:38502) 2024-04-25T09:12:15.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d63e00640 1 -- 172.21.15.62:0/3151569707 learned_addr learned my addr 172.21.15.62:0/3151569707 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d6be00640 1 --2- 172.21.15.62:0/3151569707 >> v2:172.21.15.121:3300/0 conn(0x7f7d6c15bd10 0x7f7d6c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.436+0000 7f7d71200640 1 --2- 172.21.15.62:0/3151569707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0aeed0 0x7f7d6c0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.437+0000 7f7d6b400640 1 -- 172.21.15.62:0/3151569707 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550642608 0 0) 0x7f7d6c06ca70 con 0x7f7d6c0cccf0 2024-04-25T09:12:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.437+0000 7f7d6b400640 1 -- 172.21.15.62:0/3151569707 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d4c003520 con 0x7f7d6c0cccf0 2024-04-25T09:12:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.437+0000 7f7d71200640 1 -- 172.21.15.62:0/3151569707 >> v2:172.21.15.121:3300/0 conn(0x7f7d6c15bd10 msgr2=0x7f7d6c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.437+0000 7f7d71200640 1 --2- 172.21.15.62:0/3151569707 >> v2:172.21.15.121:3300/0 conn(0x7f7d6c15bd10 0x7f7d6c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.437+0000 7f7d71200640 1 -- 172.21.15.62:0/3151569707 >> v1:172.21.15.62:6789/0 conn(0x7f7d6c0cccf0 legacy=0x7f7d6c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.437+0000 7f7d6b400640 1 -- 172.21.15.62:0/3151569707 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 350651808 0 0) 0x7f7d4c003520 con 0x7f7d6c0cccf0 2024-04-25T09:12:15.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.437+0000 7f7d71200640 1 -- 172.21.15.62:0/3151569707 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c06cd30 con 0x7f7d6c0aeed0 2024-04-25T09:12:15.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.437+0000 7f7d6be00640 1 --2- 172.21.15.62:0/3151569707 >> v2:172.21.15.121:3300/0 conn(0x7f7d6c15bd10 0x7f7d6c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:12:15.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.439+0000 7f7d71200640 1 --2- 172.21.15.62:0/3151569707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0aeed0 0x7f7d6c0cd6c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c06d7e0 tx=0x7f7d6402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12cb41ab715e204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:15.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.439+0000 7f7d6b400640 1 -- 172.21.15.62:0/3151569707 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6402ae60 con 0x7f7d6c0aeed0 2024-04-25T09:12:15.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.439+0000 7f7d6b400640 1 -- 172.21.15.62:0/3151569707 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d64007740 con 0x7f7d6c0aeed0 2024-04-25T09:12:15.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.439+0000 7f7d6b400640 1 -- 172.21.15.62:0/3151569707 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64007b10 con 0x7f7d6c0aeed0 2024-04-25T09:12:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.440+0000 7f7d73200640 1 -- 172.21.15.62:0/3151569707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0aeed0 msgr2=0x7f7d6c0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.440+0000 7f7d73200640 1 --2- 172.21.15.62:0/3151569707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0aeed0 0x7f7d6c0cd6c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c06d7e0 tx=0x7f7d6402a640 comp rx=0 tx=0).stop 2024-04-25T09:12:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.440+0000 7f7d73200640 1 -- 172.21.15.62:0/3151569707 shutdown_connections 2024-04-25T09:12:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.440+0000 7f7d73200640 1 --2- 172.21.15.62:0/3151569707 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0aeed0 0x7f7d6c0cd6c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.440+0000 7f7d73200640 1 --2- 172.21.15.62:0/3151569707 >> v2:172.21.15.121:3300/0 conn(0x7f7d6c15bd10 0x7f7d6c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.440+0000 7f7d73200640 1 -- 172.21.15.62:0/3151569707 >> 172.21.15.62:0/3151569707 conn(0x7f7d6c0668f0 msgr2=0x7f7d6c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.440+0000 7f7d73200640 1 -- 172.21.15.62:0/3151569707 shutdown_connections 2024-04-25T09:12:15.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.440+0000 7f7d73200640 1 -- 172.21.15.62:0/3151569707 wait complete. 2024-04-25T09:12:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.443+0000 7f7d73200640 1 Processor -- start 2024-04-25T09:12:15.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.443+0000 7f7d73200640 1 -- start start 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.444+0000 7f7d73200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c0aeed0 0x7f7d6c0aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.444+0000 7f7d73200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0cccf0 0x7f7d6c15d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.444+0000 7f7d73200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6c15bd10 0x7f7d6c146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.444+0000 7f7d73200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c0d01e0 con 0x7f7d6c15bd10 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.444+0000 7f7d73200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c06d2f0 con 0x7f7d6c0aeed0 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.444+0000 7f7d73200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c06daf0 con 0x7f7d6c0cccf0 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.444+0000 7f7d71200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6c15bd10 0x7f7d6c146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d71200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6c15bd10 0x7f7d6c146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41146/0 (socket says 172.21.15.62:41146) 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d71200640 1 -- 172.21.15.62:0/739184345 learned_addr learned my addr 172.21.15.62:0/739184345 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d6be00640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c0aeed0 0x7f7d6c0aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d63e00640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0cccf0 0x7f7d6c15d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d71200640 1 -- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0cccf0 msgr2=0x7f7d6c15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d71200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0cccf0 0x7f7d6c15d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d71200640 1 -- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c0aeed0 msgr2=0x7f7d6c0aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d71200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c0aeed0 0x7f7d6c0aa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d71200640 1 -- 172.21.15.62:0/739184345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c0ab140 con 0x7f7d6c15bd10 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.445+0000 7f7d6be00640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c0aeed0 0x7f7d6c0aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.446+0000 7f7d71200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6c15bd10 0x7f7d6c146ff0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7d6402ac30 tx=0x7f7d64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.446+0000 7f7d68c00640 1 -- 172.21.15.62:0/739184345 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6403f070 con 0x7f7d6c15bd10 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.446+0000 7f7d68c00640 1 -- 172.21.15.62:0/739184345 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d640319c0 con 0x7f7d6c15bd10 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.446+0000 7f7d68c00640 1 -- 172.21.15.62:0/739184345 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64031b90 con 0x7f7d6c15bd10 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.446+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c0ab420 con 0x7f7d6c15bd10 2024-04-25T09:12:15.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.446+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c0ab9e0 con 0x7f7d6c15bd10 2024-04-25T09:12:15.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.447+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c04f000 con 0x7f7d6c15bd10 2024-04-25T09:12:15.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.448+0000 7f7d68c00640 1 -- 172.21.15.62:0/739184345 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d6403b410 con 0x7f7d6c15bd10 2024-04-25T09:12:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.449+0000 7f7d68c00640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d4806b500 0x7f7d4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.449+0000 7f7d68c00640 1 -- 172.21.15.62:0/739184345 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7d640aadf0 con 0x7f7d6c15bd10 2024-04-25T09:12:15.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.450+0000 7f7d6be00640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d4806b500 0x7f7d4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:15.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.451+0000 7f7d6be00640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d4806b500 0x7f7d4806d9c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ba0 tx=0x7f7d5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:15.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.454+0000 7f7d68c00640 1 -- 172.21.15.62:0/739184345 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d64079ee0 con 0x7f7d6c15bd10 2024-04-25T09:12:15.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:15.730+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f7d6c0ce440 con 0x7f7d6c15bd10 2024-04-25T09:12:16.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.247+0000 7f7d68c00640 1 -- 172.21.15.62:0/739184345 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v402) v1 ==== 147+0+0 (secure 0 0 0) 0x7f7d640797e0 con 0x7f7d6c15bd10 2024-04-25T09:12:16.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d4806b500 msgr2=0x7f7d4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d4806b500 0x7f7d4806d9c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ba0 tx=0x7f7d5c008040 comp rx=0 tx=0).stop 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6c15bd10 msgr2=0x7f7d6c146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6c15bd10 0x7f7d6c146ff0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7d6402ac30 tx=0x7f7d64004a40 comp rx=0 tx=0).stop 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 shutdown_connections 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7d4806b500 0x7f7d4806d9c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7d6c15bd10 0x7f7d6c146ff0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7d6c0cccf0 0x7f7d6c15d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 --2- 172.21.15.62:0/739184345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d6c0aeed0 0x7f7d6c0aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.250+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 >> 172.21.15.62:0/739184345 conn(0x7f7d6c0668f0 msgr2=0x7f7d6c15c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.251+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 shutdown_connections 2024-04-25T09:12:16.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.251+0000 7f7d73200640 1 -- 172.21.15.62:0/739184345 wait complete. 2024-04-25T09:12:16.261 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:12:16.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.491+0000 7f073e000640 1 Processor -- start 2024-04-25T09:12:16.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.492+0000 7f073e000640 1 -- start start 2024-04-25T09:12:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.493+0000 7f073e000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f073815dd50 0x7f07380d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.493+0000 7f073e000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f07380d2fb0 0x7f07380ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.493+0000 7f073e000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07380ce780 con 0x7f07380d3960 2024-04-25T09:12:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.493+0000 7f073e000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07380d3440 con 0x7f07380d2fb0 2024-04-25T09:12:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.493+0000 7f073e000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07380ce1c0 con 0x7f073815dd50 2024-04-25T09:12:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.493+0000 7f073d600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f07380d3960 0x7f07380d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38504/0 (socket says 172.21.15.62:38504) 2024-04-25T09:12:16.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f073d600640 1 -- 172.21.15.62:0/518339264 learned_addr learned my addr 172.21.15.62:0/518339264 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f072fe00640 1 --2- 172.21.15.62:0/518339264 >> v2:172.21.15.121:3300/0 conn(0x7f07380d2fb0 0x7f07380ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f073cc00640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f073815dd50 0x7f07380d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997517883 0 0) 0x7f07380ce780 con 0x7f07380d3960 2024-04-25T09:12:16.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f071c003520 con 0x7f07380d3960 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207197501 0 0) 0x7f071c003520 con 0x7f07380d3960 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f073815dd50 msgr2=0x7f07380d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f073815dd50 0x7f07380d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 >> v2:172.21.15.121:3300/0 conn(0x7f07380d2fb0 msgr2=0x7f07380ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 --2- 172.21.15.62:0/518339264 >> v2:172.21.15.121:3300/0 conn(0x7f07380d2fb0 0x7f07380ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380ce4a0 con 0x7f07380d3960 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f07300019f0 con 0x7f07380d3960 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.494+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 >> v1:172.21.15.62:6789/0 conn(0x7f07380d3960 legacy=0x7f07380d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f0737e00640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c0039b0 0x7f071c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f0737e00640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f073815dd50 0x7f071c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f0737e00640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f071c004d40 0x7f071c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380d3440 con 0x7f071c004d40 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380ce1c0 con 0x7f071c0039b0 2024-04-25T09:12:16.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c00b9c0 con 0x7f073815dd50 2024-04-25T09:12:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f073d600640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f071c004d40 0x7f071c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f072fe00640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f073815dd50 0x7f071c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f073cc00640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c0039b0 0x7f071c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f073d600640 1 -- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f073815dd50 msgr2=0x7f071c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f073d600640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f073815dd50 0x7f071c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f073d600640 1 -- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c0039b0 msgr2=0x7f071c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f073d600640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c0039b0 0x7f071c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.495+0000 7f073d600640 1 -- 172.21.15.62:0/518339264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c003520 con 0x7f071c004d40 2024-04-25T09:12:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.497+0000 7f073d600640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f071c004d40 0x7f071c0051c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f073000a870 tx=0x7f071c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15360921c84b613d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:16.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.497+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073000a4c0 con 0x7f071c004d40 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.497+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073002f070 con 0x7f071c004d40 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.497+0000 7f0737e00640 1 -- 172.21.15.62:0/518339264 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073002bcc0 con 0x7f071c004d40 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.497+0000 7f073e000640 1 -- 172.21.15.62:0/518339264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f071c004d40 msgr2=0x7f071c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.497+0000 7f073e000640 1 --2- 172.21.15.62:0/518339264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f071c004d40 0x7f071c0051c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f073000a870 tx=0x7f071c005830 comp rx=0 tx=0).stop 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.497+0000 7f073d600640 1 -- 172.21.15.62:0/518339264 reap_dead start 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.498+0000 7f073e000640 1 -- 172.21.15.62:0/518339264 shutdown_connections 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.498+0000 7f073e000640 1 -- 172.21.15.62:0/518339264 >> 172.21.15.62:0/518339264 conn(0x7f073806b000 msgr2=0x7f073806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.498+0000 7f073e000640 1 -- 172.21.15.62:0/518339264 shutdown_connections 2024-04-25T09:12:16.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.498+0000 7f073e000640 1 -- 172.21.15.62:0/518339264 wait complete. 2024-04-25T09:12:16.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.499+0000 7f073e000640 1 Processor -- start 2024-04-25T09:12:16.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.500+0000 7f073e000640 1 -- start start 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f073e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c004d40 0x7f07380c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f073e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0738160820 0x7f0738160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f073e000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07380b4570 0x7f07380b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f073e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380d2480 con 0x7f0738160820 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f073e000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380d2300 con 0x7f071c004d40 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f073e000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07380d2600 con 0x7f07380b4570 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f072fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0738160820 0x7f0738160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f072fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0738160820 0x7f0738160c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41162/0 (socket says 172.21.15.62:41162) 2024-04-25T09:12:16.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f072fe00640 1 -- 172.21.15.62:0/1676343964 learned_addr learned my addr 172.21.15.62:0/1676343964 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:16.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f072fe00640 1 -- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07380b4570 msgr2=0x7f07380b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f072fe00640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07380b4570 0x7f07380b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f073cc00640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c004d40 0x7f07380c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f072fe00640 1 -- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c004d40 msgr2=0x7f07380c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f072fe00640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c004d40 0x7f07380c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.501+0000 7f072fe00640 1 -- 172.21.15.62:0/1676343964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073815ef50 con 0x7f0738160820 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.502+0000 7f072fe00640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0738160820 0x7f0738160c80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0720011650 tx=0x7f0720011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.502+0000 7f0735600640 1 -- 172.21.15.62:0/1676343964 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072000f9c0 con 0x7f0738160820 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.502+0000 7f0735600640 1 -- 172.21.15.62:0/1676343964 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072000fb60 con 0x7f0738160820 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.502+0000 7f0735600640 1 -- 172.21.15.62:0/1676343964 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720015460 con 0x7f0738160820 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.502+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07380b3a90 con 0x7f0738160820 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.502+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07380b4050 con 0x7f0738160820 2024-04-25T09:12:16.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.503+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073804ef20 con 0x7f0738160820 2024-04-25T09:12:16.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.504+0000 7f0735600640 1 -- 172.21.15.62:0/1676343964 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0720015600 con 0x7f0738160820 2024-04-25T09:12:16.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.505+0000 7f0735600640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b4d0 0x7f071406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:16.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.505+0000 7f0735600640 1 -- 172.21.15.62:0/1676343964 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f072008a950 con 0x7f0738160820 2024-04-25T09:12:16.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.506+0000 7f073cc00640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b4d0 0x7f071406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:16.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.507+0000 7f073cc00640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b4d0 0x7f071406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f07280098d0 tx=0x7f0728008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:16.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.510+0000 7f0735600640 1 -- 172.21.15.62:0/1676343964 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0720019030 con 0x7f0738160820 2024-04-25T09:12:16.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.711+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f07380cd5f0 con 0x7f0738160820 2024-04-25T09:12:16.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.717+0000 7f0735600640 1 -- 172.21.15.62:0/1676343964 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0720059a30 con 0x7f0738160820 2024-04-25T09:12:16.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.719+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b4d0 msgr2=0x7f071406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.719+0000 7f073e000640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b4d0 0x7f071406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f07280098d0 tx=0x7f0728008040 comp rx=0 tx=0).stop 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.719+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0738160820 msgr2=0x7f0738160c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.719+0000 7f073e000640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0738160820 0x7f0738160c80 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0720011650 tx=0x7f0720011b10 comp rx=0 tx=0).stop 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.720+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 shutdown_connections 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.720+0000 7f073e000640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b4d0 0x7f071406d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.720+0000 7f073e000640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0738160820 0x7f0738160c80 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.720+0000 7f073e000640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07380b4570 0x7f07380b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.720+0000 7f073e000640 1 --2- 172.21.15.62:0/1676343964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f071c004d40 0x7f07380c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.720+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 >> 172.21.15.62:0/1676343964 conn(0x7f073806b000 msgr2=0x7f07380cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.720+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 shutdown_connections 2024-04-25T09:12:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:16.720+0000 7f073e000640 1 -- 172.21.15.62:0/1676343964 wait complete. 2024-04-25T09:12:16.727 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:12:16.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:12:16.845 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:12:16.960 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:12:16.961 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:12:16.961 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:17.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:12:17.022 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:12:17.022 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:17.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:17.078 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:12:17.078 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:17.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:12:17.140 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:12:17.140 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:17.198 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:12:17.198 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:12:17.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:17.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.428+0000 7f7105000640 1 Processor -- start 2024-04-25T09:12:17.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.428+0000 7f7105000640 1 -- start start 2024-04-25T09:12:17.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.429+0000 7f7105000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71000b56f0 0x7f7100160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.429+0000 7f7105000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000b6a90 0x7f7100161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.429+0000 7f7105000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71000d3110 con 0x7f71000b27a0 2024-04-25T09:12:17.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.429+0000 7f7105000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7100161780 con 0x7f71000b56f0 2024-04-25T09:12:17.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.429+0000 7f7105000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71000cedc0 con 0x7f71000b6a90 2024-04-25T09:12:17.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.430+0000 7f70ffe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f71000b27a0 0x7f7100161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38520/0 (socket says 172.21.15.62:38520) 2024-04-25T09:12:17.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.430+0000 7f70ffe00640 1 -- 172.21.15.62:0/3269784385 learned_addr learned my addr 172.21.15.62:0/3269784385 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:17.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.430+0000 7f70ff400640 1 --2- 172.21.15.62:0/3269784385 >> v2:172.21.15.121:3300/0 conn(0x7f71000b56f0 0x7f7100160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.430+0000 7f70fea00640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000b6a90 0x7f7100161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.430+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082201389 0 0) 0x7f71000d3110 con 0x7f71000b27a0 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.430+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e4003520 con 0x7f71000b27a0 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4251423755 0 0) 0x7f70e4003520 con 0x7f71000b27a0 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000b6a90 msgr2=0x7f7100161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000b6a90 0x7f7100161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 >> v2:172.21.15.121:3300/0 conn(0x7f71000b56f0 msgr2=0x7f7100160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 --2- 172.21.15.62:0/3269784385 >> v2:172.21.15.121:3300/0 conn(0x7f71000b56f0 0x7f7100160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000d33f0 con 0x7f71000b27a0 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f70f8001670 con 0x7f71000b27a0 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 >> v1:172.21.15.62:6789/0 conn(0x7f71000b27a0 legacy=0x7f7100161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e40039b0 0x7f70e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000b6a90 0x7f70e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f70e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000cedc0 con 0x7f70e4004d40 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e400b9c0 con 0x7f70e40039b0 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e400bb70 con 0x7f71000b6a90 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70ffe00640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f70e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70ff400640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e40039b0 0x7f70e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.431+0000 7f70fea00640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000b6a90 0x7f70e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.432+0000 7f70ffe00640 1 -- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000b6a90 msgr2=0x7f70e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.432+0000 7f70ffe00640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71000b6a90 0x7f70e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.432+0000 7f70ffe00640 1 -- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e40039b0 msgr2=0x7f70e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.432+0000 7f70ffe00640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e40039b0 0x7f70e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.432+0000 7f70ffe00640 1 -- 172.21.15.62:0/3269784385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4003520 con 0x7f70e4004d40 2024-04-25T09:12:17.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.433+0000 7f70ffe00640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f70e40051c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c240 tx=0x7f70e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f74187aa01e286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.433+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f802e070 con 0x7f70e4004d40 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.433+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8005af0 con 0x7f70e4004d40 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.433+0000 7f70fe000640 1 -- 172.21.15.62:0/3269784385 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8007d30 con 0x7f70e4004d40 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.434+0000 7f7105000640 1 -- 172.21.15.62:0/3269784385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 msgr2=0x7f70e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.434+0000 7f7105000640 1 --2- 172.21.15.62:0/3269784385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f70e40051c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c240 tx=0x7f70e4005830 comp rx=0 tx=0).stop 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.434+0000 7f70ffe00640 1 -- 172.21.15.62:0/3269784385 reap_dead start 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.434+0000 7f7105000640 1 -- 172.21.15.62:0/3269784385 shutdown_connections 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.434+0000 7f7105000640 1 -- 172.21.15.62:0/3269784385 >> 172.21.15.62:0/3269784385 conn(0x7f7100066d10 msgr2=0x7f7100067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.434+0000 7f7105000640 1 -- 172.21.15.62:0/3269784385 shutdown_connections 2024-04-25T09:12:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.434+0000 7f7105000640 1 -- 172.21.15.62:0/3269784385 wait complete. 2024-04-25T09:12:17.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.436+0000 7f7105000640 1 Processor -- start 2024-04-25T09:12:17.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.436+0000 7f7105000640 1 -- start start 2024-04-25T09:12:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f7105000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f7100156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f7105000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71001e5dd0 0x7f710015f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f7105000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71001e6250 0x7f710015f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f7105000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000b2d60 con 0x7f70e4004d40 2024-04-25T09:12:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f7105000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000cea50 con 0x7f71001e5dd0 2024-04-25T09:12:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f7105000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71000b2ee0 con 0x7f71001e6250 2024-04-25T09:12:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f70ff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f7100156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f70ff400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f7100156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41178/0 (socket says 172.21.15.62:41178) 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f70ff400640 1 -- 172.21.15.62:0/863575632 learned_addr learned my addr 172.21.15.62:0/863575632 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.437+0000 7f70ffe00640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71001e6250 0x7f710015f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70fea00640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71001e5dd0 0x7f710015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70ff400640 1 -- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71001e6250 msgr2=0x7f710015f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70ff400640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71001e6250 0x7f710015f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70ff400640 1 -- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71001e5dd0 msgr2=0x7f710015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70ff400640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71001e5dd0 0x7f710015f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70ff400640 1 -- 172.21.15.62:0/863575632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71001e6820 con 0x7f70e4004d40 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70ff400640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f7100156fd0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f70f0007f90 tx=0x7f70f0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70f7400640 1 -- 172.21.15.62:0/863575632 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f001a020 con 0x7f70e4004d40 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70f7400640 1 -- 172.21.15.62:0/863575632 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0016470 con 0x7f70e4004d40 2024-04-25T09:12:17.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f70f7400640 1 -- 172.21.15.62:0/863575632 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0016860 con 0x7f70e4004d40 2024-04-25T09:12:17.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.438+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71001e6b00 con 0x7f70e4004d40 2024-04-25T09:12:17.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.439+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7100158dd0 con 0x7f70e4004d40 2024-04-25T09:12:17.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.440+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710004ef20 con 0x7f70e4004d40 2024-04-25T09:12:17.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.441+0000 7f70f7400640 1 -- 172.21.15.62:0/863575632 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70f0005d10 con 0x7f70e4004d40 2024-04-25T09:12:17.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.442+0000 7f70f7400640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.442+0000 7f70f7400640 1 -- 172.21.15.62:0/863575632 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f70f008af50 con 0x7f70e4004d40 2024-04-25T09:12:17.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.442+0000 7f70fea00640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.443+0000 7f70fea00640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f70e006b4d0 0x7f70e006d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f71001f6ff0 tx=0x7f70ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.446+0000 7f70f7400640 1 -- 172.21.15.62:0/863575632 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70f0024030 con 0x7f70e4004d40 2024-04-25T09:12:17.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.683+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71000ce580 con 0x7f70e4004d40 2024-04-25T09:12:17.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.685+0000 7f70f7400640 1 -- 172.21.15.62:0/863575632 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+3690 (secure 0 0 0) 0x7f70f005a030 con 0x7f70e4004d40 2024-04-25T09:12:17.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:17.673 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":57,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5336,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2992229593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2992229593},{"type":"v1","addr":"172.21.15.62:6835","nonce":2992229593}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5342,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/518754003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":518754003},{"type":"v1","addr":"172.21.15.121:6837","nonce":518754003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5785,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/6918584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":6918584},{"type":"v1","addr":"172.21.15.121:6836","nonce":6918584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/369439481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":369439481},{"type":"v1","addr":"172.21.15.62:6834","nonce":369439481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[]} 2024-04-25T09:12:17.673 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 57 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f70e006b4d0 msgr2=0x7f70e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f70e006b4d0 0x7f70e006d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f71001f6ff0 tx=0x7f70ec008040 comp rx=0 tx=0).stop 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 msgr2=0x7f7100156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f7100156fd0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f70f0007f90 tx=0x7f70f0011660 comp rx=0 tx=0).stop 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 shutdown_connections 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f70e006b4d0 0x7f70e006d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71001e6250 0x7f710015f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71001e5dd0 0x7f710015f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 --2- 172.21.15.62:0/863575632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70e4004d40 0x7f7100156fd0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 >> 172.21.15.62:0/863575632 conn(0x7f7100066d10 msgr2=0x7f71000b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 shutdown_connections 2024-04-25T09:12:17.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.688+0000 7f7105000640 1 -- 172.21.15.62:0/863575632 wait complete. 2024-04-25T09:12:17.695 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:12:17.695 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:12:17.696 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:12:17.696 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:12:17.696 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:12:17.696 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:12:17.696 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:12:17.696 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:12:17.696 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:12:17.696 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:12:17.697 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:12:17.697 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:12:17.697 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:12:17.697 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:12:17.697 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:12:17.697 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:12:17.715 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:12:17.715 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:12:17.718 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:12:17.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:12:17.723 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:12:17.723 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:12:17.726 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:12:17.726 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:12:17.765 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:12:17.765 DEBUG:teuthology.parallel:result is None 2024-04-25T09:12:17.771 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:12:17.771 DEBUG:teuthology.parallel:result is None 2024-04-25T09:12:17.772 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:12:17.773 DEBUG:teuthology.parallel:result is None 2024-04-25T09:12:17.779 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:12:17.779 DEBUG:teuthology.parallel:result is None 2024-04-25T09:12:17.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:12:17.915 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:12:17.916 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:12:17.929 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:12:17.932 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:12:17.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.976+0000 7f7957200640 1 Processor -- start 2024-04-25T09:12:17.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.976+0000 7f7957200640 1 -- start start 2024-04-25T09:12:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.977+0000 7f7957200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79580b5c30 0x7f79580b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.977+0000 7f7957200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580cea70 con 0x7f79580d0fb0 2024-04-25T09:12:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.977+0000 7f7957200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79580ced50 con 0x7f7958161230 2024-04-25T09:12:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.977+0000 7f7957200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79580d2040 con 0x7f79580b5c30 2024-04-25T09:12:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.977+0000 7f7956800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f79580d0fb0 0x7f7958161a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38530/0 (socket says 172.21.15.62:38530) 2024-04-25T09:12:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.977+0000 7f7956800640 1 -- 172.21.15.62:0/3701151657 learned_addr learned my addr 172.21.15.62:0/3701151657 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7955400640 1 --2- 172.21.15.62:0/3701151657 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79580b5c30 0x7f79580b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7954a00640 1 -- 172.21.15.62:0/3701151657 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024650753 0 0) 0x7f79580cea70 con 0x7f79580d0fb0 2024-04-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7954a00640 1 -- 172.21.15.62:0/3701151657 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f793c003520 con 0x7f79580d0fb0 2024-04-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7954a00640 1 -- 172.21.15.62:0/3701151657 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240855751 0 0) 0x7f79580ced50 con 0x7f7958161230 2024-04-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7954a00640 1 -- 172.21.15.62:0/3701151657 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79580cea70 con 0x7f7958161230 2024-04-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7955400640 1 -- 172.21.15.62:0/3701151657 >> v1:172.21.15.121:6789/0 conn(0x7f7958161230 legacy=0x7f79580b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7954a00640 1 -- 172.21.15.62:0/3701151657 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1896612381 0 0) 0x7f793c003520 con 0x7f79580d0fb0 2024-04-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7955400640 1 -- 172.21.15.62:0/3701151657 >> v1:172.21.15.62:6789/0 conn(0x7f79580d0fb0 legacy=0x7f7958161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.978+0000 7f7955400640 1 -- 172.21.15.62:0/3701151657 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79580cf030 con 0x7f79580b5c30 2024-04-25T09:12:17.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.979+0000 7f7955400640 1 --2- 172.21.15.62:0/3701151657 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79580b5c30 0x7f79580b6c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79440097d0 tx=0x7f794402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5c5f913f9b29723 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.980+0000 7f7954a00640 1 -- 172.21.15.62:0/3701151657 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944037070 con 0x7f79580b5c30 2024-04-25T09:12:17.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.980+0000 7f7954a00640 1 -- 172.21.15.62:0/3701151657 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794402ac30 con 0x7f79580b5c30 2024-04-25T09:12:17.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.980+0000 7f7954a00640 1 -- 172.21.15.62:0/3701151657 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500019f0 con 0x7f79580b5c30 2024-04-25T09:12:17.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.980+0000 7f7957200640 1 -- 172.21.15.62:0/3701151657 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79580b5c30 msgr2=0x7f79580b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.980+0000 7f7957200640 1 --2- 172.21.15.62:0/3701151657 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79580b5c30 0x7f79580b6c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79440097d0 tx=0x7f794402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.981+0000 7f7957200640 1 -- 172.21.15.62:0/3701151657 shutdown_connections 2024-04-25T09:12:17.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.981+0000 7f7957200640 1 --2- 172.21.15.62:0/3701151657 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f79580b5c30 0x7f79580b6c40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.981+0000 7f7957200640 1 -- 172.21.15.62:0/3701151657 >> 172.21.15.62:0/3701151657 conn(0x7f7958066d10 msgr2=0x7f7958067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:17.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.981+0000 7f7957200640 1 -- 172.21.15.62:0/3701151657 shutdown_connections 2024-04-25T09:12:17.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.981+0000 7f7957200640 1 -- 172.21.15.62:0/3701151657 wait complete. 2024-04-25T09:12:17.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.982+0000 7f7957200640 1 Processor -- start 2024-04-25T09:12:17.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.983+0000 7f7957200640 1 -- start start 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7957200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79580b5c30 0x7f79580cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7957200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79580d0fb0 0x7f79581f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7957200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7958161230 0x7f79581e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7957200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7958161610 con 0x7f79580d0fb0 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7957200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79580ca8d0 con 0x7f79580b5c30 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7957200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7958161790 con 0x7f7958161230 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79580d0fb0 0x7f79581f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79580d0fb0 0x7f79581f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41218/0 (socket says 172.21.15.62:41218) 2024-04-25T09:12:17.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955400640 1 -- 172.21.15.62:0/4227337801 learned_addr learned my addr 172.21.15.62:0/4227337801 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955e00640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79580b5c30 0x7f79580cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7956800640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7958161230 0x7f79581e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955400640 1 -- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7958161230 msgr2=0x7f79581e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955400640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7958161230 0x7f79581e81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955400640 1 -- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79580b5c30 msgr2=0x7f79580cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955400640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79580b5c30 0x7f79580cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.984+0000 7f7955400640 1 -- 172.21.15.62:0/4227337801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79581e8a80 con 0x7f79580d0fb0 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.985+0000 7f7955400640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79580d0fb0 0x7f79581f6b30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f7944035040 tx=0x7f79440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.985+0000 7f794e000640 1 -- 172.21.15.62:0/4227337801 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944041020 con 0x7f79580d0fb0 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.985+0000 7f794e000640 1 -- 172.21.15.62:0/4227337801 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794403d640 con 0x7f79580d0fb0 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.985+0000 7f794e000640 1 -- 172.21.15.62:0/4227337801 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944037040 con 0x7f79580d0fb0 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.985+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79581e8d60 con 0x7f79580d0fb0 2024-04-25T09:12:17.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.985+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79581e9320 con 0x7f79580d0fb0 2024-04-25T09:12:17.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.987+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795804ef20 con 0x7f79580d0fb0 2024-04-25T09:12:17.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.988+0000 7f794e000640 1 -- 172.21.15.62:0/4227337801 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7944044020 con 0x7f79580d0fb0 2024-04-25T09:12:17.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.989+0000 7f794e000640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f793806b4d0 0x7f793806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:17.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.989+0000 7f794e000640 1 -- 172.21.15.62:0/4227337801 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f79440aa4b0 con 0x7f79580d0fb0 2024-04-25T09:12:17.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.990+0000 7f7955e00640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f793806b4d0 0x7f793806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:17.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.990+0000 7f7955e00640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f793806b4d0 0x7f793806d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f79480015b0 tx=0x7f7948009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:17.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:17.994+0000 7f794e000640 1 -- 172.21.15.62:0/4227337801 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f794404b030 con 0x7f79580d0fb0 2024-04-25T09:12:18.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.206+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f79580ce640 con 0x7f79580d0fb0 2024-04-25T09:12:18.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.274+0000 7f794e000640 1 -- 172.21.15.62:0/4227337801 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7944079590 con 0x7f79580d0fb0 2024-04-25T09:12:18.262 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:12:18.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.277+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f793806b4d0 msgr2=0x7f793806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.277+0000 7f7957200640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f793806b4d0 0x7f793806d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f79480015b0 tx=0x7f7948009040 comp rx=0 tx=0).stop 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.277+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79580d0fb0 msgr2=0x7f79581f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.277+0000 7f7957200640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79580d0fb0 0x7f79581f6b30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f7944035040 tx=0x7f79440046c0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.278+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 shutdown_connections 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.278+0000 7f7957200640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f793806b4d0 0x7f793806d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.278+0000 7f7957200640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79580d0fb0 0x7f79581f6b30 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.278+0000 7f7957200640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7958161230 0x7f79581e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.278+0000 7f7957200640 1 --2- 172.21.15.62:0/4227337801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79580b5c30 0x7f79580cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.278+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 >> 172.21.15.62:0/4227337801 conn(0x7f7958066d10 msgr2=0x7f79580b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.278+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 shutdown_connections 2024-04-25T09:12:18.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.278+0000 7f7957200640 1 -- 172.21.15.62:0/4227337801 wait complete. 2024-04-25T09:12:18.287 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:12:18.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.505+0000 7f39b2600640 1 Processor -- start 2024-04-25T09:12:18.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.505+0000 7f39b2600640 1 -- start start 2024-04-25T09:12:18.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.506+0000 7f39b2600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39ac1454c0 0x7f39ac06c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39b2600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f39ac0ccbe0 0x7f39ac0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39b2600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac0b34b0 con 0x7f39ac06c8b0 2024-04-25T09:12:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39b2600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac160530 con 0x7f39ac1454c0 2024-04-25T09:12:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39b2600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac0cd530 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39ab400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39ac1454c0 0x7f39ac06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39a2a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f39ac06c8b0 0x7f39ac0b44e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38536/0 (socket says 172.21.15.62:38536) 2024-04-25T09:12:18.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39a2a00640 1 -- 172.21.15.62:0/4164202078 learned_addr learned my addr 172.21.15.62:0/4164202078 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:18.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39abe00640 1 --2- 172.21.15.62:0/4164202078 >> v2:172.21.15.121:3300/0 conn(0x7f39ac0ccbe0 0x7f39ac0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39ab400640 1 -- 172.21.15.62:0/4164202078 >> v2:172.21.15.121:3300/0 conn(0x7f39ac0ccbe0 msgr2=0x7f39ac0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39ab400640 1 --2- 172.21.15.62:0/4164202078 >> v2:172.21.15.121:3300/0 conn(0x7f39ac0ccbe0 0x7f39ac0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.507+0000 7f39ab400640 1 -- 172.21.15.62:0/4164202078 >> v1:172.21.15.62:6789/0 conn(0x7f39ac06c8b0 legacy=0x7f39ac0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.508+0000 7f39ab400640 1 -- 172.21.15.62:0/4164202078 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac0b3720 con 0x7f39ac1454c0 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.508+0000 7f39aaa00640 1 -- 172.21.15.62:0/4164202078 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860049055 0 0) 0x7f39ac0b34b0 con 0x7f39ac06c8b0 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.508+0000 7f39abe00640 1 --2- 172.21.15.62:0/4164202078 >> v2:172.21.15.121:3300/0 conn(0x7f39ac0ccbe0 0x7f39ac0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.509+0000 7f39ab400640 1 --2- 172.21.15.62:0/4164202078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac1454c0 0x7f39ac06c370 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0702a0 tx=0x7f399c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f9bdb1e84664c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.509+0000 7f39aaa00640 1 -- 172.21.15.62:0/4164202078 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c037070 con 0x7f39ac1454c0 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.509+0000 7f39aaa00640 1 -- 172.21.15.62:0/4164202078 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c032440 con 0x7f39ac1454c0 2024-04-25T09:12:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.509+0000 7f39aaa00640 1 -- 172.21.15.62:0/4164202078 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c032810 con 0x7f39ac1454c0 2024-04-25T09:12:18.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.510+0000 7f39b2600640 1 -- 172.21.15.62:0/4164202078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac1454c0 msgr2=0x7f39ac06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.510+0000 7f39b2600640 1 --2- 172.21.15.62:0/4164202078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac1454c0 0x7f39ac06c370 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0702a0 tx=0x7f399c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.510+0000 7f39b2600640 1 -- 172.21.15.62:0/4164202078 shutdown_connections 2024-04-25T09:12:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.510+0000 7f39b2600640 1 --2- 172.21.15.62:0/4164202078 >> v2:172.21.15.121:3300/0 conn(0x7f39ac0ccbe0 0x7f39ac0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.510+0000 7f39b2600640 1 --2- 172.21.15.62:0/4164202078 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac1454c0 0x7f39ac06c370 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.510+0000 7f39b2600640 1 -- 172.21.15.62:0/4164202078 >> 172.21.15.62:0/4164202078 conn(0x7f39ac0668f0 msgr2=0x7f39ac066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.510+0000 7f39b2600640 1 -- 172.21.15.62:0/4164202078 shutdown_connections 2024-04-25T09:12:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.510+0000 7f39b2600640 1 -- 172.21.15.62:0/4164202078 wait complete. 2024-04-25T09:12:18.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.512+0000 7f39b2600640 1 Processor -- start 2024-04-25T09:12:18.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.512+0000 7f39b2600640 1 -- start start 2024-04-25T09:12:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.513+0000 7f39b2600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39ac06c8b0 0x7f39ac0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.513+0000 7f39b2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac0ccbe0 0x7f39ac140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.513+0000 7f39b2600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39ac1454c0 0x7f39ac13d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.513+0000 7f39b2600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b3fc0 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.513+0000 7f39b2600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b3ce0 con 0x7f39ac1454c0 2024-04-25T09:12:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.513+0000 7f39b2600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b4bf0 con 0x7f39ac06c8b0 2024-04-25T09:12:18.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.513+0000 7f39a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac0ccbe0 0x7f39ac140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.513+0000 7f39a2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac0ccbe0 0x7f39ac140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41240/0 (socket says 172.21.15.62:41240) 2024-04-25T09:12:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39a2a00640 1 -- 172.21.15.62:0/561856467 learned_addr learned my addr 172.21.15.62:0/561856467 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39abe00640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39ac1454c0 0x7f39ac13d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39ab400640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39ac06c8b0 0x7f39ac0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39a2a00640 1 -- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39ac06c8b0 msgr2=0x7f39ac0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39a2a00640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39ac06c8b0 0x7f39ac0b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39a2a00640 1 -- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39ac1454c0 msgr2=0x7f39ac13d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39a2a00640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39ac1454c0 0x7f39ac13d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39a2a00640 1 -- 172.21.15.62:0/561856467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac140a70 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39ab400640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39ac06c8b0 0x7f39ac0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.514+0000 7f39a2a00640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac0ccbe0 0x7f39ac140530 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f399400c460 tx=0x7f399400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.515+0000 7f39a3e00640 1 -- 172.21.15.62:0/561856467 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994018070 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.515+0000 7f39a3e00640 1 -- 172.21.15.62:0/561856467 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994007500 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.515+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac0b1500 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.515+0000 7f39a3e00640 1 -- 172.21.15.62:0/561856467 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994004a20 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.515+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac0b1ac0 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.516+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac04ef20 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.517+0000 7f39a3e00640 1 -- 172.21.15.62:0/561856467 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f399401b070 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.519+0000 7f39a3e00640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f398806b4d0 0x7f398806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:18.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.519+0000 7f39a3e00640 1 -- 172.21.15.62:0/561856467 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3994089260 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.519+0000 7f39ab400640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f398806b4d0 0x7f398806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:18.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.521+0000 7f39ab400640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f398806b4d0 0x7f398806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f399c034000 tx=0x7f399c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:18.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.524+0000 7f39a3e00640 1 -- 172.21.15.62:0/561856467 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3994058300 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.800+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f39ac06b2f0 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.801+0000 7f39a3e00640 1 -- 172.21.15.62:0/561856467 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3994057c00 con 0x7f39ac0ccbe0 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:12:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.790 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.791 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.792 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.793 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:12:18.794 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:18.795 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.797 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.798 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.799 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:12:18.800 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.801 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:12:18.802 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.803 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:12:18.804 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:12:18.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.803+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f398806b4d0 msgr2=0x7f398806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.803+0000 7f39b2600640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f398806b4d0 0x7f398806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f399c034000 tx=0x7f399c02e000 comp rx=0 tx=0).stop 2024-04-25T09:12:18.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.803+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac0ccbe0 msgr2=0x7f39ac140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.803+0000 7f39b2600640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac0ccbe0 0x7f39ac140530 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f399400c460 tx=0x7f399400c920 comp rx=0 tx=0).stop 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.803+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 shutdown_connections 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.803+0000 7f39b2600640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f398806b4d0 0x7f398806d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.804+0000 7f39b2600640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39ac0ccbe0 0x7f39ac140530 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.804+0000 7f39b2600640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39ac1454c0 0x7f39ac13d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.804+0000 7f39b2600640 1 --2- 172.21.15.62:0/561856467 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39ac06c8b0 0x7f39ac0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.804+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 >> 172.21.15.62:0/561856467 conn(0x7f39ac0668f0 msgr2=0x7f39ac15fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.804+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 shutdown_connections 2024-04-25T09:12:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:18.804+0000 7f39b2600640 1 -- 172.21.15.62:0/561856467 wait complete. 2024-04-25T09:12:18.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.023+0000 7f88c5000640 1 Processor -- start 2024-04-25T09:12:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.023+0000 7f88c5000640 1 -- start start 2024-04-25T09:12:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.025+0000 7f88c5000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f88c015acf0 0x7f88c006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.025+0000 7f88c5000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 0x7f88c013e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.025+0000 7f88c5000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c00b1220 con 0x7f88c013d970 2024-04-25T09:12:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.025+0000 7f88c5000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c015b220 con 0x7f88c015acf0 2024-04-25T09:12:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.025+0000 7f88c5000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c015b500 con 0x7f88c006d730 2024-04-25T09:12:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.025+0000 7f88bf400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f88c013d970 0x7f88c00b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38552/0 (socket says 172.21.15.62:38552) 2024-04-25T09:12:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.025+0000 7f88bf400640 1 -- 172.21.15.62:0/403035857 learned_addr learned my addr 172.21.15.62:0/403035857 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.026+0000 7f88b7e00640 1 --2- 172.21.15.62:0/403035857 >> v2:172.21.15.62:3300/0 conn(0x7f88c015acf0 0x7f88c006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.026+0000 7f88bfe00640 1 --2- 172.21.15.62:0/403035857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 0x7f88c013e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.026+0000 7f88bea00640 1 -- 172.21.15.62:0/403035857 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728499464 0 0) 0x7f88c00b1220 con 0x7f88c013d970 2024-04-25T09:12:19.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.026+0000 7f88bea00640 1 -- 172.21.15.62:0/403035857 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a4005730 con 0x7f88c013d970 2024-04-25T09:12:19.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.026+0000 7f88b7e00640 1 -- 172.21.15.62:0/403035857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 msgr2=0x7f88c013e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.026+0000 7f88b7e00640 1 --2- 172.21.15.62:0/403035857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 0x7f88c013e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.026+0000 7f88b7e00640 1 -- 172.21.15.62:0/403035857 >> v1:172.21.15.62:6789/0 conn(0x7f88c013d970 legacy=0x7f88c00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.026+0000 7f88b7e00640 1 -- 172.21.15.62:0/403035857 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c015b7e0 con 0x7f88c015acf0 2024-04-25T09:12:19.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.027+0000 7f88b7e00640 1 --2- 172.21.15.62:0/403035857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c006d1f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f88a80097a0 tx=0x7f88a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa488ca9f8f77a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88bea00640 1 -- 172.21.15.62:0/403035857 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8037070 con 0x7f88c015acf0 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88bea00640 1 -- 172.21.15.62:0/403035857 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a8032440 con 0x7f88c015acf0 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88bea00640 1 -- 172.21.15.62:0/403035857 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8032810 con 0x7f88c015acf0 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88c5000640 1 -- 172.21.15.62:0/403035857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 msgr2=0x7f88c006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88c5000640 1 --2- 172.21.15.62:0/403035857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c006d1f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f88a80097a0 tx=0x7f88a802a180 comp rx=0 tx=0).stop 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88c5000640 1 -- 172.21.15.62:0/403035857 shutdown_connections 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88c5000640 1 --2- 172.21.15.62:0/403035857 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 0x7f88c013e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88c5000640 1 --2- 172.21.15.62:0/403035857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c006d1f0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88c5000640 1 -- 172.21.15.62:0/403035857 >> 172.21.15.62:0/403035857 conn(0x7f88c00b14d0 msgr2=0x7f88c00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.028+0000 7f88c5000640 1 -- 172.21.15.62:0/403035857 shutdown_connections 2024-04-25T09:12:19.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.029+0000 7f88c5000640 1 -- 172.21.15.62:0/403035857 wait complete. 2024-04-25T09:12:19.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.030+0000 7f88c5000640 1 Processor -- start 2024-04-25T09:12:19.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.030+0000 7f88c5000640 1 -- start start 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.031+0000 7f88c5000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 0x7f88c0141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.031+0000 7f88c5000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88c013d970 0x7f88c013c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.031+0000 7f88c5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c01597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.031+0000 7f88c5000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c013e910 con 0x7f88c015acf0 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.031+0000 7f88c5000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c013e790 con 0x7f88c013d970 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.031+0000 7f88c5000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c013ea90 con 0x7f88c006d730 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c01597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c01597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41256/0 (socket says 172.21.15.62:41256) 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bfe00640 1 -- 172.21.15.62:0/3879664856 learned_addr learned my addr 172.21.15.62:0/3879664856 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bf400640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 0x7f88c0141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88b7e00640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88c013d970 0x7f88c013c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bfe00640 1 -- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 msgr2=0x7f88c0141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bfe00640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 0x7f88c0141a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bfe00640 1 -- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88c013d970 msgr2=0x7f88c013c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bfe00640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88c013d970 0x7f88c013c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.032+0000 7f88bfe00640 1 -- 172.21.15.62:0/3879664856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c01422c0 con 0x7f88c015acf0 2024-04-25T09:12:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.033+0000 7f88bfe00640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c01597d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f88b8007e40 tx=0x7f88b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.033+0000 7f88b7400640 1 -- 172.21.15.62:0/3879664856 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8014620 con 0x7f88c015acf0 2024-04-25T09:12:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.033+0000 7f88b7400640 1 -- 172.21.15.62:0/3879664856 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b8019070 con 0x7f88c015acf0 2024-04-25T09:12:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.033+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c01425a0 con 0x7f88c015acf0 2024-04-25T09:12:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.033+0000 7f88b7400640 1 -- 172.21.15.62:0/3879664856 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8014a20 con 0x7f88c015acf0 2024-04-25T09:12:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.033+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c0142b60 con 0x7f88c015acf0 2024-04-25T09:12:19.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.034+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c01f3990 con 0x7f88c015acf0 2024-04-25T09:12:19.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.035+0000 7f88b7400640 1 -- 172.21.15.62:0/3879664856 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88b8014bc0 con 0x7f88c015acf0 2024-04-25T09:12:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.037+0000 7f88b7400640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.037+0000 7f88b7400640 1 -- 172.21.15.62:0/3879664856 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f88b808a8c0 con 0x7f88c015acf0 2024-04-25T09:12:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.037+0000 7f88bf400640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.038+0000 7f88bf400640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f889c06b4d0 0x7f889c06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f88b0001200 tx=0x7f88b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.041+0000 7f88b7400640 1 -- 172.21.15.62:0/3879664856 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88b80599f0 con 0x7f88c015acf0 2024-04-25T09:12:19.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.243+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88c004ef20 con 0x7f88c015acf0 2024-04-25T09:12:19.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.245+0000 7f88b7400640 1 -- 172.21.15.62:0/3879664856 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+12880 (secure 0 0 0) 0x7f88b80592f0 con 0x7f88c015acf0 2024-04-25T09:12:19.235 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:19.235 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":93,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:18.290431+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f889c06b4d0 msgr2=0x7f889c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f889c06b4d0 0x7f889c06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f88b0001200 tx=0x7f88b0009040 comp rx=0 tx=0).stop 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 msgr2=0x7f88c01597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c01597d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f88b8007e40 tx=0x7f88b800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 shutdown_connections 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88c015acf0 0x7f88c01597d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88c013d970 0x7f88c013c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 --2- 172.21.15.62:0/3879664856 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88c006d730 0x7f88c0141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 >> 172.21.15.62:0/3879664856 conn(0x7f88c00b14d0 msgr2=0x7f88c015bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 shutdown_connections 2024-04-25T09:12:19.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.248+0000 7f88c5000640 1 -- 172.21.15.62:0/3879664856 wait complete. 2024-04-25T09:12:19.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:12:19.256 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:12:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.474+0000 7fa20a400640 1 Processor -- start 2024-04-25T09:12:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.474+0000 7fa20a400640 1 -- start start 2024-04-25T09:12:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.476+0000 7fa20a400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2040ceb50 0x7fa20406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.476+0000 7fa20a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa20415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.476+0000 7fa20a400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2040b5560 con 0x7fa20415e5f0 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.476+0000 7fa20a400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20406dc60 con 0x7fa2040ceb50 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.476+0000 7fa20a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20415ee30 con 0x7fa20406d800 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.476+0000 7fa209000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2040ceb50 0x7fa20406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.477+0000 7fa209000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2040ceb50 0x7fa20406d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41262/0 (socket says 172.21.15.62:41262) 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.477+0000 7fa209000640 1 -- 172.21.15.62:0/142209701 learned_addr learned my addr 172.21.15.62:0/142209701 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.477+0000 7fa203e00640 1 --2- 172.21.15.62:0/142209701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa20415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.477+0000 7fa209000640 1 -- 172.21.15.62:0/142209701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 msgr2=0x7fa20415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.477+0000 7fa209000640 1 --2- 172.21.15.62:0/142209701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa20415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.477+0000 7fa209000640 1 -- 172.21.15.62:0/142209701 >> v1:172.21.15.121:6789/0 conn(0x7fa20415e5f0 legacy=0x7fa2040ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.477+0000 7fa209000640 1 -- 172.21.15.62:0/142209701 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2041618b0 con 0x7fa2040ceb50 2024-04-25T09:12:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.477+0000 7fa203e00640 1 --2- 172.21.15.62:0/142209701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa20415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.478+0000 7fa209000640 1 --2- 172.21.15.62:0/142209701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa20406d2c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009870 tx=0x7fa1f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fc5a8af34806d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.478+0000 7fa203400640 1 -- 172.21.15.62:0/142209701 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8037070 con 0x7fa2040ceb50 2024-04-25T09:12:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.478+0000 7fa203400640 1 -- 172.21.15.62:0/142209701 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8032440 con 0x7fa2040ceb50 2024-04-25T09:12:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.478+0000 7fa203400640 1 -- 172.21.15.62:0/142209701 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8032810 con 0x7fa2040ceb50 2024-04-25T09:12:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.479+0000 7fa20a400640 1 -- 172.21.15.62:0/142209701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 msgr2=0x7fa20406d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.479+0000 7fa20a400640 1 --2- 172.21.15.62:0/142209701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa20406d2c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009870 tx=0x7fa1f802a180 comp rx=0 tx=0).stop 2024-04-25T09:12:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.479+0000 7fa20a400640 1 -- 172.21.15.62:0/142209701 shutdown_connections 2024-04-25T09:12:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.479+0000 7fa20a400640 1 --2- 172.21.15.62:0/142209701 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa20415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.479+0000 7fa20a400640 1 --2- 172.21.15.62:0/142209701 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa20406d2c0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.479+0000 7fa20a400640 1 -- 172.21.15.62:0/142209701 >> 172.21.15.62:0/142209701 conn(0x7fa204066f20 msgr2=0x7fa2040b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.480+0000 7fa20a400640 1 -- 172.21.15.62:0/142209701 shutdown_connections 2024-04-25T09:12:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.480+0000 7fa20a400640 1 -- 172.21.15.62:0/142209701 wait complete. 2024-04-25T09:12:19.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.482+0000 7fa20a400640 1 Processor -- start 2024-04-25T09:12:19.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.482+0000 7fa20a400640 1 -- start start 2024-04-25T09:12:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.483+0000 7fa20a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa2041e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.483+0000 7fa20a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa204143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.483+0000 7fa20a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20415e5f0 0x7fa2041f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa20a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20415ebe0 con 0x7fa2040ceb50 2024-04-25T09:12:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa20a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20415ea60 con 0x7fa20415e5f0 2024-04-25T09:12:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa20a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040cef30 con 0x7fa20406d800 2024-04-25T09:12:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa203e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa204143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa203e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa204143030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41274/0 (socket says 172.21.15.62:41274) 2024-04-25T09:12:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa203e00640 1 -- 172.21.15.62:0/2311206249 learned_addr learned my addr 172.21.15.62:0/2311206249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa209000640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa2041e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa203e00640 1 -- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 msgr2=0x7fa2041e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa203e00640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa2041e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa203e00640 1 -- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20415e5f0 msgr2=0x7fa2041f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa209a00640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20415e5f0 0x7fa2041f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa203e00640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20415e5f0 0x7fa2041f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.484+0000 7fa203e00640 1 -- 172.21.15.62:0/2311206249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204143570 con 0x7fa2040ceb50 2024-04-25T09:12:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.485+0000 7fa203e00640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa204143030 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c7f0 tx=0x7fa1f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.485+0000 7fa200c00640 1 -- 172.21.15.62:0/2311206249 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0015070 con 0x7fa2040ceb50 2024-04-25T09:12:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.485+0000 7fa200c00640 1 -- 172.21.15.62:0/2311206249 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0007a30 con 0x7fa2040ceb50 2024-04-25T09:12:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.485+0000 7fa200c00640 1 -- 172.21.15.62:0/2311206249 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0007e00 con 0x7fa2040ceb50 2024-04-25T09:12:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.485+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2041f4390 con 0x7fa2040ceb50 2024-04-25T09:12:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.485+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2041f4950 con 0x7fa2040ceb50 2024-04-25T09:12:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.486+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20404ef20 con 0x7fa2040ceb50 2024-04-25T09:12:19.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.487+0000 7fa200c00640 1 -- 172.21.15.62:0/2311206249 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1f0018070 con 0x7fa2040ceb50 2024-04-25T09:12:19.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.488+0000 7fa200c00640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e406b500 0x7fa1e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.488+0000 7fa200c00640 1 -- 172.21.15.62:0/2311206249 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa1f0089e10 con 0x7fa2040ceb50 2024-04-25T09:12:19.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.489+0000 7fa209000640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e406b500 0x7fa1e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.490+0000 7fa209000640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e406b500 0x7fa1e406d9c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8034000 tx=0x7fa1f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.492+0000 7fa200c00640 1 -- 172.21.15.62:0/2311206249 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1f008d030 con 0x7fa2040ceb50 2024-04-25T09:12:19.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.692+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2041467d0 con 0x7fa2040ceb50 2024-04-25T09:12:19.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.693+0000 7fa200c00640 1 -- 172.21.15.62:0/2311206249 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa204143570 con 0x7fa2040ceb50 2024-04-25T09:12:19.681 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:12:19.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e406b500 msgr2=0x7fa1e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e406b500 0x7fa1e406d9c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8034000 tx=0x7fa1f802e000 comp rx=0 tx=0).stop 2024-04-25T09:12:19.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 msgr2=0x7fa204143030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa204143030 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c7f0 tx=0x7fa1f000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 shutdown_connections 2024-04-25T09:12:19.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e406b500 0x7fa1e406d9c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2040ceb50 0x7fa204143030 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20415e5f0 0x7fa2041f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 --2- 172.21.15.62:0/2311206249 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20406d800 0x7fa2041e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 >> 172.21.15.62:0/2311206249 conn(0x7fa204066f20 msgr2=0x7fa20406cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 shutdown_connections 2024-04-25T09:12:19.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.696+0000 7fa20a400640 1 -- 172.21.15.62:0/2311206249 wait complete. 2024-04-25T09:12:19.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:12:19.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.919+0000 7f3b4a400640 1 Processor -- start 2024-04-25T09:12:19.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.919+0000 7f3b4a400640 1 -- start start 2024-04-25T09:12:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.921+0000 7f3b4a400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b4415b960 0x7f3b4406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.921+0000 7f3b4a400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b440b47e0 0x7f3b440cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.921+0000 7f3b4a400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4406d4f0 con 0x7f3b440ae9d0 2024-04-25T09:12:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.921+0000 7f3b4a400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4415bd40 con 0x7f3b4415b960 2024-04-25T09:12:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.921+0000 7f3b4a400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b440b3110 con 0x7f3b440b47e0 2024-04-25T09:12:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.921+0000 7f3b49000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b4415b960 0x7f3b4406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.921+0000 7f3b49000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b4415b960 0x7f3b4406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41286/0 (socket says 172.21.15.62:41286) 2024-04-25T09:12:19.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.921+0000 7f3b49000640 1 -- 172.21.15.62:0/1438671850 learned_addr learned my addr 172.21.15.62:0/1438671850 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:19.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.922+0000 7f3b49a00640 1 --2- 172.21.15.62:0/1438671850 >> v2:172.21.15.121:3300/0 conn(0x7f3b440b47e0 0x7f3b440cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.922+0000 7f3b49000640 1 -- 172.21.15.62:0/1438671850 >> v2:172.21.15.121:3300/0 conn(0x7f3b440b47e0 msgr2=0x7f3b440cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:19.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.922+0000 7f3b49000640 1 --2- 172.21.15.62:0/1438671850 >> v2:172.21.15.121:3300/0 conn(0x7f3b440b47e0 0x7f3b440cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.922+0000 7f3b49000640 1 -- 172.21.15.62:0/1438671850 >> v1:172.21.15.121:6789/0 conn(0x7f3b440ae9d0 legacy=0x7f3b440b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.922+0000 7f3b49000640 1 -- 172.21.15.62:0/1438671850 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b440b2500 con 0x7f3b4415b960 2024-04-25T09:12:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.924+0000 7f3b49000640 1 --2- 172.21.15.62:0/1438671850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4415b960 0x7f3b4406ca80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3b440ae840 tx=0x7f3b3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cd94372e45a1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.924+0000 7f3b43e00640 1 -- 172.21.15.62:0/1438671850 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34037070 con 0x7f3b4415b960 2024-04-25T09:12:19.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.924+0000 7f3b43e00640 1 -- 172.21.15.62:0/1438671850 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b34032440 con 0x7f3b4415b960 2024-04-25T09:12:19.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.924+0000 7f3b43e00640 1 -- 172.21.15.62:0/1438671850 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34032810 con 0x7f3b4415b960 2024-04-25T09:12:19.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.925+0000 7f3b4a400640 1 -- 172.21.15.62:0/1438671850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4415b960 msgr2=0x7f3b4406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.925+0000 7f3b4a400640 1 --2- 172.21.15.62:0/1438671850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4415b960 0x7f3b4406ca80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3b440ae840 tx=0x7f3b3402a270 comp rx=0 tx=0).stop 2024-04-25T09:12:19.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.925+0000 7f3b4a400640 1 -- 172.21.15.62:0/1438671850 shutdown_connections 2024-04-25T09:12:19.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.925+0000 7f3b4a400640 1 --2- 172.21.15.62:0/1438671850 >> v2:172.21.15.121:3300/0 conn(0x7f3b440b47e0 0x7f3b440cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.925+0000 7f3b4a400640 1 --2- 172.21.15.62:0/1438671850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b4415b960 0x7f3b4406ca80 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.925+0000 7f3b4a400640 1 -- 172.21.15.62:0/1438671850 >> 172.21.15.62:0/1438671850 conn(0x7f3b44074380 msgr2=0x7f3b440668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:19.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.925+0000 7f3b4a400640 1 -- 172.21.15.62:0/1438671850 shutdown_connections 2024-04-25T09:12:19.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.925+0000 7f3b4a400640 1 -- 172.21.15.62:0/1438671850 wait complete. 2024-04-25T09:12:19.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.927+0000 7f3b4a400640 1 Processor -- start 2024-04-25T09:12:19.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.927+0000 7f3b4a400640 1 -- start start 2024-04-25T09:12:19.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.928+0000 7f3b4a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b440ae9d0 0x7f3b44153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.928+0000 7f3b4a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b440b47e0 0x7f3b44151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.928+0000 7f3b4a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4415b960 0x7f3b44156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.928+0000 7f3b4a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b440b2da0 con 0x7f3b440ae9d0 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.928+0000 7f3b4a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b440b2ac0 con 0x7f3b4415b960 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.928+0000 7f3b4a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b440cdab0 con 0x7f3b440b47e0 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b440ae9d0 0x7f3b44153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b440ae9d0 0x7f3b44153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41300/0 (socket says 172.21.15.62:41300) 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 -- 172.21.15.62:0/469316624 learned_addr learned my addr 172.21.15.62:0/469316624 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 -- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b440b47e0 msgr2=0x7f3b44151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b440b47e0 0x7f3b44151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 -- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4415b960 msgr2=0x7f3b44156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4415b960 0x7f3b44156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:19.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 -- 172.21.15.62:0/469316624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b44153e60 con 0x7f3b440ae9d0 2024-04-25T09:12:19.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.929+0000 7f3b49000640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b440ae9d0 0x7f3b44153920 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3b3402a2b0 tx=0x7f3b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.930+0000 7f3b41600640 1 -- 172.21.15.62:0/469316624 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34041020 con 0x7f3b440ae9d0 2024-04-25T09:12:19.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.930+0000 7f3b41600640 1 -- 172.21.15.62:0/469316624 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b340306c0 con 0x7f3b440ae9d0 2024-04-25T09:12:19.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.930+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b44154700 con 0x7f3b440ae9d0 2024-04-25T09:12:19.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.930+0000 7f3b41600640 1 -- 172.21.15.62:0/469316624 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34037040 con 0x7f3b440ae9d0 2024-04-25T09:12:19.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.930+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b44154cc0 con 0x7f3b440ae9d0 2024-04-25T09:12:19.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.931+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b44075070 con 0x7f3b440ae9d0 2024-04-25T09:12:19.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.932+0000 7f3b41600640 1 -- 172.21.15.62:0/469316624 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b34044020 con 0x7f3b440ae9d0 2024-04-25T09:12:19.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.933+0000 7f3b41600640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2006b4d0 0x7f3b2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:19.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.933+0000 7f3b41600640 1 -- 172.21.15.62:0/469316624 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3b340aa780 con 0x7f3b440ae9d0 2024-04-25T09:12:19.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.934+0000 7f3b3be00640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2006b4d0 0x7f3b2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:19.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.935+0000 7f3b3be00640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2006b4d0 0x7f3b2006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000f60 tx=0x7f3b2c0079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:19.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:19.937+0000 7f3b41600640 1 -- 172.21.15.62:0/469316624 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b3404b030 con 0x7f3b440ae9d0 2024-04-25T09:12:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.139+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3b440b3eb0 con 0x7f3b440ae9d0 2024-04-25T09:12:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.140+0000 7f3b41600640 1 -- 172.21.15.62:0/469316624 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3b340798b0 con 0x7f3b440ae9d0 2024-04-25T09:12:20.128 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:20.128 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:12:20.128 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:12:20.128 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:12:20.128 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:12:20.128 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:12:20.128 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.142+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2006b4d0 msgr2=0x7f3b2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2006b4d0 0x7f3b2006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000f60 tx=0x7f3b2c0079d0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b440ae9d0 msgr2=0x7f3b44153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b440ae9d0 0x7f3b44153920 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3b3402a2b0 tx=0x7f3b340046c0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 shutdown_connections 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2006b4d0 0x7f3b2006d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4415b960 0x7f3b44156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b440b47e0 0x7f3b44151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 --2- 172.21.15.62:0/469316624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b440ae9d0 0x7f3b44153920 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 >> 172.21.15.62:0/469316624 conn(0x7f3b44074380 msgr2=0x7f3b440ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 shutdown_connections 2024-04-25T09:12:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.143+0000 7f3b4a400640 1 -- 172.21.15.62:0/469316624 wait complete. 2024-04-25T09:12:20.150 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:12:20.150 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:12:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.378+0000 7fc79e600640 1 Processor -- start 2024-04-25T09:12:20.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.378+0000 7fc79e600640 1 -- start start 2024-04-25T09:12:20.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.380+0000 7fc79e600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc798145140 0x7fc7980b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:20.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.380+0000 7fc79e600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7980b4730 0x7fc79815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:20.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.380+0000 7fc79e600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79815c9a0 con 0x7fc79815c3f0 2024-04-25T09:12:20.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.380+0000 7fc79e600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980b3d60 con 0x7fc798145140 2024-04-25T09:12:20.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.380+0000 7fc79e600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79815cc80 con 0x7fc7980b4730 2024-04-25T09:12:20.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.380+0000 7fc797e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7980b4730 0x7fc79815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:20.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.380+0000 7fc79d200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc798145140 0x7fc7980b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:20.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.381+0000 7fc797e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7980b4730 0x7fc79815beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:36144/0 (socket says 172.21.15.62:36144) 2024-04-25T09:12:20.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.381+0000 7fc797e00640 1 -- 172.21.15.62:0/2392409264 learned_addr learned my addr 172.21.15.62:0/2392409264 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:20.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.381+0000 7fc797e00640 1 -- 172.21.15.62:0/2392409264 >> v2:172.21.15.121:3300/0 conn(0x7fc798145140 msgr2=0x7fc7980b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:20.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.381+0000 7fc797e00640 1 --2- 172.21.15.62:0/2392409264 >> v2:172.21.15.121:3300/0 conn(0x7fc798145140 0x7fc7980b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.381+0000 7fc797e00640 1 -- 172.21.15.62:0/2392409264 >> v1:172.21.15.121:6789/0 conn(0x7fc79815c3f0 legacy=0x7fc79806c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:20.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.381+0000 7fc797e00640 1 -- 172.21.15.62:0/2392409264 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980b39f0 con 0x7fc7980b4730 2024-04-25T09:12:20.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.383+0000 7fc797e00640 1 --2- 172.21.15.62:0/2392409264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7980b4730 0x7fc79815beb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc780009870 tx=0x7fc78002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc603fa5b4672667 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:20.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.383+0000 7fc797400640 1 -- 172.21.15.62:0/2392409264 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780037070 con 0x7fc7980b4730 2024-04-25T09:12:20.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.383+0000 7fc797400640 1 -- 172.21.15.62:0/2392409264 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780032440 con 0x7fc7980b4730 2024-04-25T09:12:20.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.384+0000 7fc797400640 1 -- 172.21.15.62:0/2392409264 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780032810 con 0x7fc7980b4730 2024-04-25T09:12:20.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.384+0000 7fc79e600640 1 -- 172.21.15.62:0/2392409264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7980b4730 msgr2=0x7fc79815beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:20.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.384+0000 7fc79e600640 1 --2- 172.21.15.62:0/2392409264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7980b4730 0x7fc79815beb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc780009870 tx=0x7fc78002a250 comp rx=0 tx=0).stop 2024-04-25T09:12:20.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.384+0000 7fc79e600640 1 -- 172.21.15.62:0/2392409264 shutdown_connections 2024-04-25T09:12:20.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.384+0000 7fc79e600640 1 --2- 172.21.15.62:0/2392409264 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc7980b4730 0x7fc79815beb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.384+0000 7fc79e600640 1 --2- 172.21.15.62:0/2392409264 >> v2:172.21.15.121:3300/0 conn(0x7fc798145140 0x7fc7980b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.384+0000 7fc79e600640 1 -- 172.21.15.62:0/2392409264 >> 172.21.15.62:0/2392409264 conn(0x7fc798066ad0 msgr2=0x7fc798066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:20.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.385+0000 7fc79e600640 1 -- 172.21.15.62:0/2392409264 shutdown_connections 2024-04-25T09:12:20.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.385+0000 7fc79e600640 1 -- 172.21.15.62:0/2392409264 wait complete. 2024-04-25T09:12:20.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.386+0000 7fc79e600640 1 Processor -- start 2024-04-25T09:12:20.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.386+0000 7fc79e600640 1 -- start start 2024-04-25T09:12:20.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.387+0000 7fc79e600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc798145140 0x7fc7981412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:20.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.387+0000 7fc79e600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79815c3f0 0x7fc79815f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:20.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.387+0000 7fc79e600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7981f3930 0x7fc79813bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:20.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.387+0000 7fc79e600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7981456a0 con 0x7fc79815c3f0 2024-04-25T09:12:20.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.387+0000 7fc79e600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798145520 con 0x7fc7981f3930 2024-04-25T09:12:20.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.387+0000 7fc79e600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79806cb50 con 0x7fc798145140 2024-04-25T09:12:20.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc797e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79815c3f0 0x7fc79815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc797e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79815c3f0 0x7fc79815f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41310/0 (socket says 172.21.15.62:41310) 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc797e00640 1 -- 172.21.15.62:0/1031407073 learned_addr learned my addr 172.21.15.62:0/1031407073 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc79d200640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc798145140 0x7fc7981412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc79dc00640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7981f3930 0x7fc79813bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc797e00640 1 -- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc798145140 msgr2=0x7fc7981412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc797e00640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc798145140 0x7fc7981412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc797e00640 1 -- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7981f3930 msgr2=0x7fc79813bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc797e00640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7981f3930 0x7fc79813bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc797e00640 1 -- 172.21.15.62:0/1031407073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc798142410 con 0x7fc79815c3f0 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.388+0000 7fc79dc00640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7981f3930 0x7fc79813bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:20.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.389+0000 7fc797e00640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79815c3f0 0x7fc79815f140 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fc78002a290 tx=0x7fc7800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:20.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.389+0000 7fc794c00640 1 -- 172.21.15.62:0/1031407073 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780037040 con 0x7fc79815c3f0 2024-04-25T09:12:20.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.389+0000 7fc794c00640 1 -- 172.21.15.62:0/1031407073 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780042070 con 0x7fc79815c3f0 2024-04-25T09:12:20.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.389+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7981426f0 con 0x7fc79815c3f0 2024-04-25T09:12:20.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.389+0000 7fc794c00640 1 -- 172.21.15.62:0/1031407073 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780030880 con 0x7fc79815c3f0 2024-04-25T09:12:20.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.389+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79813e790 con 0x7fc79815c3f0 2024-04-25T09:12:20.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.390+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc798072b30 con 0x7fc79815c3f0 2024-04-25T09:12:20.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.391+0000 7fc794c00640 1 -- 172.21.15.62:0/1031407073 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc780030a20 con 0x7fc79815c3f0 2024-04-25T09:12:20.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.392+0000 7fc794c00640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77406b4d0 0x7fc77406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:20.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.392+0000 7fc794c00640 1 -- 172.21.15.62:0/1031407073 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7800aa760 con 0x7fc79815c3f0 2024-04-25T09:12:20.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.393+0000 7fc79d200640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77406b4d0 0x7fc77406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:20.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.394+0000 7fc79d200640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77406b4d0 0x7fc77406d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc7880074a0 tx=0x7fc78800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:20.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.397+0000 7fc794c00640 1 -- 172.21.15.62:0/1031407073 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc780079920 con 0x7fc79815c3f0 2024-04-25T09:12:20.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:20.597+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc7980750e0 con 0x7fc79815c3f0 2024-04-25T09:12:21.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.303+0000 7fc794c00640 1 -- 172.21.15.62:0/1031407073 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc780079220 con 0x7fc79815c3f0 2024-04-25T09:12:21.292 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:12:21.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77406b4d0 msgr2=0x7fc77406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:21.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77406b4d0 0x7fc77406d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc7880074a0 tx=0x7fc78800b040 comp rx=0 tx=0).stop 2024-04-25T09:12:21.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79815c3f0 msgr2=0x7fc79815f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:21.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79815c3f0 0x7fc79815f140 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fc78002a290 tx=0x7fc7800046c0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 shutdown_connections 2024-04-25T09:12:21.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77406b4d0 0x7fc77406d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79815c3f0 0x7fc79815f140 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7981f3930 0x7fc79813bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 --2- 172.21.15.62:0/1031407073 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc798145140 0x7fc7981412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 >> 172.21.15.62:0/1031407073 conn(0x7fc798066ad0 msgr2=0x7fc7980c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:21.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.307+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 shutdown_connections 2024-04-25T09:12:21.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.308+0000 7fc79e600640 1 -- 172.21.15.62:0/1031407073 wait complete. 2024-04-25T09:12:21.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:12:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.552+0000 7f5fcf200640 1 Processor -- start 2024-04-25T09:12:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.553+0000 7f5fcf200640 1 -- start start 2024-04-25T09:12:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.554+0000 7f5fcf200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5fd015dd20 0x7f5fd00d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.554+0000 7f5fcf200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5fd00ceef0 0x7f5fd00b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.554+0000 7f5fcf200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd015e100 con 0x7f5fd00d34f0 2024-04-25T09:12:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.554+0000 7f5fcf200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd00b9a20 con 0x7f5fd00ceef0 2024-04-25T09:12:21.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.554+0000 7f5fcf200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd00b7b50 con 0x7f5fd015dd20 2024-04-25T09:12:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.555+0000 7f5fc7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5fd00d34f0 0x7f5fd00ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38564/0 (socket says 172.21.15.62:38564) 2024-04-25T09:12:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.555+0000 7f5fc7e00640 1 -- 172.21.15.62:0/3098729079 learned_addr learned my addr 172.21.15.62:0/3098729079 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.555+0000 7f5fce800640 1 --2- 172.21.15.62:0/3098729079 >> v2:172.21.15.62:3300/0 conn(0x7f5fd00ceef0 0x7f5fd00b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.555+0000 7f5fcde00640 1 --2- 172.21.15.62:0/3098729079 >> v2:172.21.15.121:3300/0 conn(0x7f5fd015dd20 0x7f5fd00d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:21.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.555+0000 7f5fcd400640 1 -- 172.21.15.62:0/3098729079 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780815032 0 0) 0x7f5fd015e100 con 0x7f5fd00d34f0 2024-04-25T09:12:21.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.556+0000 7f5fcd400640 1 -- 172.21.15.62:0/3098729079 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb4003520 con 0x7f5fd00d34f0 2024-04-25T09:12:21.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.556+0000 7f5fce800640 1 -- 172.21.15.62:0/3098729079 >> v2:172.21.15.121:3300/0 conn(0x7f5fd015dd20 msgr2=0x7f5fd00d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:21.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.556+0000 7f5fce800640 1 --2- 172.21.15.62:0/3098729079 >> v2:172.21.15.121:3300/0 conn(0x7f5fd015dd20 0x7f5fd00d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.556+0000 7f5fce800640 1 -- 172.21.15.62:0/3098729079 >> v1:172.21.15.62:6789/0 conn(0x7f5fd00d34f0 legacy=0x7f5fd00ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:21.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.556+0000 7f5fce800640 1 -- 172.21.15.62:0/3098729079 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00d2450 con 0x7f5fd00ceef0 2024-04-25T09:12:21.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.557+0000 7f5fce800640 1 --2- 172.21.15.62:0/3098729079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd00ceef0 0x7f5fd00b94e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00b5f50 tx=0x7f5fc802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d8ea3846916487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:21.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.557+0000 7f5fcd400640 1 -- 172.21.15.62:0/3098729079 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8007740 con 0x7f5fd00ceef0 2024-04-25T09:12:21.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.557+0000 7f5fcd400640 1 -- 172.21.15.62:0/3098729079 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc80078e0 con 0x7f5fd00ceef0 2024-04-25T09:12:21.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.557+0000 7f5fcd400640 1 -- 172.21.15.62:0/3098729079 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8007c90 con 0x7f5fd00ceef0 2024-04-25T09:12:21.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.558+0000 7f5fcf200640 1 -- 172.21.15.62:0/3098729079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd00ceef0 msgr2=0x7f5fd00b94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:21.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.558+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3098729079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd00ceef0 0x7f5fd00b94e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00b5f50 tx=0x7f5fc802a680 comp rx=0 tx=0).stop 2024-04-25T09:12:21.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.558+0000 7f5fcf200640 1 -- 172.21.15.62:0/3098729079 shutdown_connections 2024-04-25T09:12:21.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.558+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3098729079 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd00ceef0 0x7f5fd00b94e0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.558+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3098729079 >> v2:172.21.15.121:3300/0 conn(0x7f5fd015dd20 0x7f5fd00d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.558+0000 7f5fcf200640 1 -- 172.21.15.62:0/3098729079 >> 172.21.15.62:0/3098729079 conn(0x7f5fd006b000 msgr2=0x7f5fd006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:21.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.558+0000 7f5fcf200640 1 -- 172.21.15.62:0/3098729079 shutdown_connections 2024-04-25T09:12:21.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.558+0000 7f5fcf200640 1 -- 172.21.15.62:0/3098729079 wait complete. 2024-04-25T09:12:21.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.559+0000 7f5fcf200640 1 Processor -- start 2024-04-25T09:12:21.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.560+0000 7f5fcf200640 1 -- start start 2024-04-25T09:12:21.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00ceef0 0x7f5fd00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:21.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd00d34f0 0x7f5fd00c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:21.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd015dd20 0x7f5fd01f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:21.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fcf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd00d2cf0 con 0x7f5fd015dd20 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fcf200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd00d2a10 con 0x7f5fd00ceef0 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fcf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd00b6b40 con 0x7f5fd00d34f0 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fce800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd015dd20 0x7f5fd01f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fce800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd015dd20 0x7f5fd01f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41322/0 (socket says 172.21.15.62:41322) 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fce800640 1 -- 172.21.15.62:0/3482580529 learned_addr learned my addr 172.21.15.62:0/3482580529 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fcde00640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00ceef0 0x7f5fd00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fce800640 1 -- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd00d34f0 msgr2=0x7f5fd00c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.561+0000 7f5fc7e00640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd00d34f0 0x7f5fd00c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fce800640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd00d34f0 0x7f5fd00c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fce800640 1 -- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00ceef0 msgr2=0x7f5fd00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fce800640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00ceef0 0x7f5fd00c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fce800640 1 -- 172.21.15.62:0/3482580529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00c9500 con 0x7f5fd015dd20 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fcde00640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00ceef0 0x7f5fd00c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fce800640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd015dd20 0x7f5fd01f6400 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5fc802ac70 tx=0x7f5fc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:21.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fc6000640 1 -- 172.21.15.62:0/3482580529 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8040070 con 0x7f5fd015dd20 2024-04-25T09:12:21.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fc6000640 1 -- 172.21.15.62:0/3482580529 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc8031c60 con 0x7f5fd015dd20 2024-04-25T09:12:21.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fc6000640 1 -- 172.21.15.62:0/3482580529 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8009450 con 0x7f5fd015dd20 2024-04-25T09:12:21.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd0154c50 con 0x7f5fd015dd20 2024-04-25T09:12:21.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.562+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd0155210 con 0x7f5fd015dd20 2024-04-25T09:12:21.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.563+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd0075070 con 0x7f5fd015dd20 2024-04-25T09:12:21.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.565+0000 7f5fc6000640 1 -- 172.21.15.62:0/3482580529 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fc8039040 con 0x7f5fd015dd20 2024-04-25T09:12:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.566+0000 7f5fc6000640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fac06b500 0x7f5fac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:21.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.566+0000 7f5fc6000640 1 -- 172.21.15.62:0/3482580529 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5fc80ab670 con 0x7f5fd015dd20 2024-04-25T09:12:21.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.567+0000 7f5fcde00640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fac06b500 0x7f5fac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:21.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.568+0000 7f5fcde00640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fac06b500 0x7f5fac06d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0149940 tx=0x7f5fc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:21.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.570+0000 7f5fc6000640 1 -- 172.21.15.62:0/3482580529 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fc8038350 con 0x7f5fd015dd20 2024-04-25T09:12:21.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:21.776+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5fd00b9240 con 0x7f5fd015dd20 2024-04-25T09:12:22.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.319+0000 7f5fc6000640 1 -- 172.21.15.62:0/3482580529 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5fc807a860 con 0x7f5fd015dd20 2024-04-25T09:12:22.307 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:12:22.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fac06b500 msgr2=0x7f5fac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fac06b500 0x7f5fac06d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0149940 tx=0x7f5fc0008040 comp rx=0 tx=0).stop 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd015dd20 msgr2=0x7f5fd01f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd015dd20 0x7f5fd01f6400 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5fc802ac70 tx=0x7f5fc8004a40 comp rx=0 tx=0).stop 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 shutdown_connections 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fac06b500 0x7f5fac06d9c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5fd015dd20 0x7f5fd01f6400 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5fd00d34f0 0x7f5fd00c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 --2- 172.21.15.62:0/3482580529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fd00ceef0 0x7f5fd00c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.324+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 >> 172.21.15.62:0/3482580529 conn(0x7f5fd006b000 msgr2=0x7f5fd00b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.325+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 shutdown_connections 2024-04-25T09:12:22.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.325+0000 7f5fcf200640 1 -- 172.21.15.62:0/3482580529 wait complete. 2024-04-25T09:12:22.352 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:12:22.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.545+0000 7f6e49400640 1 Processor -- start 2024-04-25T09:12:22.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.546+0000 7f6e49400640 1 -- start start 2024-04-25T09:12:22.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.547+0000 7f6e49400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 0x7f6e440ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:22.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.547+0000 7f6e49400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e440b31a0 0x7f6e440b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:22.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.547+0000 7f6e49400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440ced40 con 0x7f6e4406d2c0 2024-04-25T09:12:22.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.548+0000 7f6e49400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e440cd260 con 0x7f6e440b31a0 2024-04-25T09:12:22.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.548+0000 7f6e49400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440cf000 con 0x7f6e440ccde0 2024-04-25T09:12:22.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.548+0000 7f6e3be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 0x7f6e440ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:22.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.548+0000 7f6e48a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e440b31a0 0x7f6e440b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:22.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.548+0000 7f6e48a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6e440b31a0 0x7f6e440b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:44920/0 (socket says 172.21.15.62:44920) 2024-04-25T09:12:22.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.548+0000 7f6e48a00640 1 -- 172.21.15.62:0/130857240 learned_addr learned my addr 172.21.15.62:0/130857240 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:22.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.548+0000 7f6e43e00640 1 --1- 172.21.15.62:0/130857240 >> v1:172.21.15.121:6789/0 conn(0x7f6e4406d2c0 0x7f6e4406d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35762/0 (socket says 172.21.15.62:35762) 2024-04-25T09:12:22.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.549+0000 7f6e48a00640 1 -- 172.21.15.62:0/130857240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 msgr2=0x7f6e440ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:22.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.549+0000 7f6e48a00640 1 --2- 172.21.15.62:0/130857240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 0x7f6e440ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.549+0000 7f6e48a00640 1 -- 172.21.15.62:0/130857240 >> v1:172.21.15.121:6789/0 conn(0x7f6e4406d2c0 legacy=0x7f6e4406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:22.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.549+0000 7f6e48a00640 1 -- 172.21.15.62:0/130857240 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440cf2e0 con 0x7f6e440b31a0 2024-04-25T09:12:22.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.550+0000 7f6e48a00640 1 --2- 172.21.15.62:0/130857240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 0x7f6e440b3f40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009e50 tx=0x7f6e3c02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ae438ae8f2dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:22.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.551+0000 7f6e43400640 1 -- 172.21.15.62:0/130857240 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c036070 con 0x7f6e440b31a0 2024-04-25T09:12:22.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.551+0000 7f6e43400640 1 -- 172.21.15.62:0/130857240 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c031600 con 0x7f6e440b31a0 2024-04-25T09:12:22.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.551+0000 7f6e43400640 1 -- 172.21.15.62:0/130857240 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0319d0 con 0x7f6e440b31a0 2024-04-25T09:12:22.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.551+0000 7f6e49400640 1 -- 172.21.15.62:0/130857240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 msgr2=0x7f6e440b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:22.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.551+0000 7f6e49400640 1 --2- 172.21.15.62:0/130857240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 0x7f6e440b3f40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009e50 tx=0x7f6e3c02a730 comp rx=0 tx=0).stop 2024-04-25T09:12:22.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.551+0000 7f6e49400640 1 -- 172.21.15.62:0/130857240 shutdown_connections 2024-04-25T09:12:22.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.551+0000 7f6e49400640 1 --2- 172.21.15.62:0/130857240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 0x7f6e440b3f40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.552+0000 7f6e49400640 1 --2- 172.21.15.62:0/130857240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 0x7f6e440ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.552+0000 7f6e49400640 1 -- 172.21.15.62:0/130857240 >> 172.21.15.62:0/130857240 conn(0x7f6e44066f20 msgr2=0x7f6e440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:22.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.552+0000 7f6e49400640 1 -- 172.21.15.62:0/130857240 shutdown_connections 2024-04-25T09:12:22.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.552+0000 7f6e49400640 1 -- 172.21.15.62:0/130857240 wait complete. 2024-04-25T09:12:22.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.553+0000 7f6e49400640 1 Processor -- start 2024-04-25T09:12:22.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.554+0000 7f6e49400640 1 -- start start 2024-04-25T09:12:22.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e49400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e4406d2c0 0x7f6e440c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:22.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e49400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 0x7f6e440adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:22.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e49400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 0x7f6e44142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e49400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b2c80 con 0x7f6e4406d2c0 2024-04-25T09:12:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e49400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b29a0 con 0x7f6e440b31a0 2024-04-25T09:12:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e49400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e441448d0 con 0x7f6e440ccde0 2024-04-25T09:12:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e43e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e4406d2c0 0x7f6e440c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e43e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e4406d2c0 0x7f6e440c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41336/0 (socket says 172.21.15.62:41336) 2024-04-25T09:12:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e43e00640 1 -- 172.21.15.62:0/887508352 learned_addr learned my addr 172.21.15.62:0/887508352 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:22.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e48a00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 0x7f6e44142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:22.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e3be00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 0x7f6e440adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:22.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e43e00640 1 -- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 msgr2=0x7f6e44142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:22.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.555+0000 7f6e43e00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 0x7f6e44142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e43e00640 1 -- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 msgr2=0x7f6e440adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e43e00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 0x7f6e440adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e43e00640 1 -- 172.21.15.62:0/887508352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440ab740 con 0x7f6e4406d2c0 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e3be00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 0x7f6e440adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e43e00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e4406d2c0 0x7f6e440c4630 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c3e0 tx=0x7f6e3400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e40c00640 1 -- 172.21.15.62:0/887508352 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340074e0 con 0x7f6e4406d2c0 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e40c00640 1 -- 172.21.15.62:0/887508352 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e34007680 con 0x7f6e4406d2c0 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e440ab9b0 con 0x7f6e4406d2c0 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e40c00640 1 -- 172.21.15.62:0/887508352 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340049e0 con 0x7f6e4406d2c0 2024-04-25T09:12:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.556+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e440abf70 con 0x7f6e4406d2c0 2024-04-25T09:12:22.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.558+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4406abe0 con 0x7f6e4406d2c0 2024-04-25T09:12:22.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.559+0000 7f6e40c00640 1 -- 172.21.15.62:0/887508352 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e3401a020 con 0x7f6e4406d2c0 2024-04-25T09:12:22.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.560+0000 7f6e40c00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e2006b4d0 0x7f6e2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:22.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.560+0000 7f6e40c00640 1 -- 172.21.15.62:0/887508352 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6e34089a40 con 0x7f6e4406d2c0 2024-04-25T09:12:22.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.561+0000 7f6e3be00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e2006b4d0 0x7f6e2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.562+0000 7f6e3be00640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e2006b4d0 0x7f6e2006d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e44143670 tx=0x7f6e2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:22.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.565+0000 7f6e40c00640 1 -- 172.21.15.62:0/887508352 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e34058880 con 0x7f6e4406d2c0 2024-04-25T09:12:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:22.807+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6e4404ef20 con 0x7f6e4406d2c0 2024-04-25T09:12:23.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.323+0000 7f6e40c00640 1 -- 172.21.15.62:0/887508352 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6e34058180 con 0x7f6e4406d2c0 2024-04-25T09:12:23.311 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e2006b4d0 msgr2=0x7f6e2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e2006b4d0 0x7f6e2006d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6e44143670 tx=0x7f6e2c008040 comp rx=0 tx=0).stop 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e4406d2c0 msgr2=0x7f6e440c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e4406d2c0 0x7f6e440c4630 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c3e0 tx=0x7f6e3400c8a0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 shutdown_connections 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6e2006b4d0 0x7f6e2006d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6e440ccde0 0x7f6e44142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6e440b31a0 0x7f6e440adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 --2- 172.21.15.62:0/887508352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e4406d2c0 0x7f6e440c4630 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 >> 172.21.15.62:0/887508352 conn(0x7f6e44066f20 msgr2=0x7f6e4406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:23.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 shutdown_connections 2024-04-25T09:12:23.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.327+0000 7f6e49400640 1 -- 172.21.15.62:0/887508352 wait complete. 2024-04-25T09:12:23.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:23.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.538+0000 7f4ade400640 1 Processor -- start 2024-04-25T09:12:23.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.539+0000 7f4ade400640 1 -- start start 2024-04-25T09:12:23.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.540+0000 7f4ade400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 0x7f4ad80ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.540+0000 7f4ade400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad806ca70 con 0x7f4ad80aed00 2024-04-25T09:12:23.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.540+0000 7f4ade400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad815c0f0 con 0x7f4ad815bd10 2024-04-25T09:12:23.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.540+0000 7f4ade400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad806d660 con 0x7f4ad80cccf0 2024-04-25T09:12:23.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.540+0000 7f4ad7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4ad80aed00 0x7f4ad80cd6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38566/0 (socket says 172.21.15.62:38566) 2024-04-25T09:12:23.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.540+0000 7f4ad7e00640 1 -- 172.21.15.62:0/3047525441 learned_addr learned my addr 172.21.15.62:0/3047525441 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6a00640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 0x7f4ad80ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232371947 0 0) 0x7f4ad806ca70 con 0x7f4ad80aed00 2024-04-25T09:12:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4abc003520 con 0x7f4ad80aed00 2024-04-25T09:12:23.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 585841712 0 0) 0x7f4abc003520 con 0x7f4ad80aed00 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 msgr2=0x7f4ad80ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 0x7f4ad80ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 >> v1:172.21.15.121:6789/0 conn(0x7f4ad815bd10 legacy=0x7f4ad80cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad806cd30 con 0x7f4ad80aed00 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f4ad0001670 con 0x7f4ad80aed00 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 >> v1:172.21.15.62:6789/0 conn(0x7f4ad80aed00 legacy=0x7f4ad80cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc0039e0 0x7f4abc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4abc004800 0x7f4abc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 0x7f4abc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.541+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad806d660 con 0x7f4abc004800 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc00ba70 con 0x7f4abc0039e0 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc00bc70 con 0x7f4ad80cccf0 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad6a00640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4abc004800 0x7f4abc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad7400640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc0039e0 0x7f4abc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad7e00640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 0x7f4abc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad6a00640 1 -- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 msgr2=0x7f4abc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad6a00640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 0x7f4abc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad6a00640 1 -- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc0039e0 msgr2=0x7f4abc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad6a00640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc0039e0 0x7f4abc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad6a00640 1 -- 172.21.15.62:0/3047525441 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc003520 con 0x7f4abc004800 2024-04-25T09:12:23.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.542+0000 7f4ad7e00640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80cccf0 0x7f4abc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:23.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.543+0000 7f4ad6a00640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4abc004800 0x7f4abc004c80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009f90 tx=0x7f4abc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96397c8657593804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:23.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.543+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0037070 con 0x7f4abc004800 2024-04-25T09:12:23.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.543+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac002e050 con 0x7f4abc004800 2024-04-25T09:12:23.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.543+0000 7f4ad6000640 1 -- 172.21.15.62:0/3047525441 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0009c10 con 0x7f4abc004800 2024-04-25T09:12:23.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.544+0000 7f4ade400640 1 -- 172.21.15.62:0/3047525441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4abc004800 msgr2=0x7f4abc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.544+0000 7f4ade400640 1 --2- 172.21.15.62:0/3047525441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4abc004800 0x7f4abc004c80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009f90 tx=0x7f4abc005830 comp rx=0 tx=0).stop 2024-04-25T09:12:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.544+0000 7f4ad7e00640 1 -- 172.21.15.62:0/3047525441 reap_dead start 2024-04-25T09:12:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.544+0000 7f4ade400640 1 -- 172.21.15.62:0/3047525441 shutdown_connections 2024-04-25T09:12:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.544+0000 7f4ade400640 1 -- 172.21.15.62:0/3047525441 >> 172.21.15.62:0/3047525441 conn(0x7f4ad80668f0 msgr2=0x7f4ad8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.544+0000 7f4ade400640 1 -- 172.21.15.62:0/3047525441 shutdown_connections 2024-04-25T09:12:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.544+0000 7f4ade400640 1 -- 172.21.15.62:0/3047525441 wait complete. 2024-04-25T09:12:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.546+0000 7f4ade400640 1 Processor -- start 2024-04-25T09:12:23.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.546+0000 7f4ade400640 1 -- start start 2024-04-25T09:12:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ade400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc004800 0x7f4ad8151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ade400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad8150860 0x7f4ad814f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ade400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80b40d0 0x7f4ad80b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ade400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad806daf0 con 0x7f4ad8150860 2024-04-25T09:12:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ade400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad806d2f0 con 0x7f4abc004800 2024-04-25T09:12:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ade400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad814fd50 con 0x7f4ad80b40d0 2024-04-25T09:12:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad8150860 0x7f4ad814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ad6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad8150860 0x7f4ad814f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41352/0 (socket says 172.21.15.62:41352) 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.547+0000 7f4ad6a00640 1 -- 172.21.15.62:0/3553399473 learned_addr learned my addr 172.21.15.62:0/3553399473 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ad7e00640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80b40d0 0x7f4ad80b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ad7400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc004800 0x7f4ad8151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ad6a00640 1 -- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80b40d0 msgr2=0x7f4ad80b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ad6a00640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80b40d0 0x7f4ad80b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ad6a00640 1 -- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc004800 msgr2=0x7f4ad8151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ad6a00640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc004800 0x7f4ad8151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ad6a00640 1 -- 172.21.15.62:0/3553399473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8156e60 con 0x7f4ad8150860 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ad6a00640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad8150860 0x7f4ad814f810 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4ac002ae80 tx=0x7f4ac0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4acf400640 1 -- 172.21.15.62:0/3553399473 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0041030 con 0x7f4ad8150860 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4acf400640 1 -- 172.21.15.62:0/3553399473 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0003990 con 0x7f4ad8150860 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad8151890 con 0x7f4ad8150860 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.548+0000 7f4acf400640 1 -- 172.21.15.62:0/3553399473 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0037040 con 0x7f4ad8150860 2024-04-25T09:12:23.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.549+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad8159150 con 0x7f4ad8150860 2024-04-25T09:12:23.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.550+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad8072c70 con 0x7f4ad8150860 2024-04-25T09:12:23.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.551+0000 7f4acf400640 1 -- 172.21.15.62:0/3553399473 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ac0044070 con 0x7f4ad8150860 2024-04-25T09:12:23.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.552+0000 7f4acf400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ab406b500 0x7f4ab406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.552+0000 7f4acf400640 1 -- 172.21.15.62:0/3553399473 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ac00aaf30 con 0x7f4ad8150860 2024-04-25T09:12:23.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.552+0000 7f4ad7400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ab406b500 0x7f4ab406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.554+0000 7f4ad7400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ab406b500 0x7f4ab406d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8007fd0 tx=0x7f4ac8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:23.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.557+0000 7f4acf400640 1 -- 172.21.15.62:0/3553399473 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ac0079d10 con 0x7f4ad8150860 2024-04-25T09:12:23.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.758+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ad804f000 con 0x7f4ad8150860 2024-04-25T09:12:23.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.762+0000 7f4acf400640 1 -- 172.21.15.62:0/3553399473 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16673 (secure 0 0 0) 0x7f4ac0079610 con 0x7f4ad8150860 2024-04-25T09:12:23.750 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:23.750 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":96,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:22.809392+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:20.654480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:21.833814+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ab406b500 msgr2=0x7f4ab406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ab406b500 0x7f4ab406d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8007fd0 tx=0x7f4ac8009040 comp rx=0 tx=0).stop 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad8150860 msgr2=0x7f4ad814f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad8150860 0x7f4ad814f810 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4ac002ae80 tx=0x7f4ac0009750 comp rx=0 tx=0).stop 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 shutdown_connections 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4ab406b500 0x7f4ab406d9c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad8150860 0x7f4ad814f810 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4ad80b40d0 0x7f4ad80b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 --2- 172.21.15.62:0/3553399473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4abc004800 0x7f4ad8151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 >> 172.21.15.62:0/3553399473 conn(0x7f4ad80668f0 msgr2=0x7f4ad80ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:23.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 shutdown_connections 2024-04-25T09:12:23.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.765+0000 7f4ade400640 1 -- 172.21.15.62:0/3553399473 wait complete. 2024-04-25T09:12:23.770 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:23.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.985+0000 7f89e2200640 1 Processor -- start 2024-04-25T09:12:23.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.986+0000 7f89e2200640 1 -- start start 2024-04-25T09:12:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.987+0000 7f89e2200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89dc0b27a0 0x7f89dc0d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.987+0000 7f89e2200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc0d3110 con 0x7f89dc0b56f0 2024-04-25T09:12:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.987+0000 7f89e2200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc0d33f0 con 0x7f89dc0b6a90 2024-04-25T09:12:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.987+0000 7f89e2200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc0cedc0 con 0x7f89dc0b27a0 2024-04-25T09:12:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.988+0000 7f89e0e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f89dc0b56f0 0x7f89dc160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38570/0 (socket says 172.21.15.62:38570) 2024-04-25T09:12:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.988+0000 7f89e0e00640 1 -- 172.21.15.62:0/1970941854 learned_addr learned my addr 172.21.15.62:0/1970941854 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.988+0000 7f89e1800640 1 --2- 172.21.15.62:0/1970941854 >> v2:172.21.15.62:3300/0 conn(0x7f89dc0b27a0 0x7f89dc0d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.988+0000 7f89e1800640 1 -- 172.21.15.62:0/1970941854 >> v1:172.21.15.121:6789/0 conn(0x7f89dc0b6a90 legacy=0x7f89dc0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:23.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.988+0000 7f89e1800640 1 -- 172.21.15.62:0/1970941854 >> v1:172.21.15.62:6789/0 conn(0x7f89dc0b56f0 legacy=0x7f89dc160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.988+0000 7f89db400640 1 -- 172.21.15.62:0/1970941854 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107400884 0 0) 0x7f89dc0d3110 con 0x7f89dc0b56f0 2024-04-25T09:12:23.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.988+0000 7f89e1800640 1 -- 172.21.15.62:0/1970941854 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc0d36d0 con 0x7f89dc0b27a0 2024-04-25T09:12:23.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.989+0000 7f89e1800640 1 --2- 172.21.15.62:0/1970941854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc0d26f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f89d40097a0 tx=0x7f89d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298a9914ee66acfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:23.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.990+0000 7f89db400640 1 -- 172.21.15.62:0/1970941854 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4037070 con 0x7f89dc0b27a0 2024-04-25T09:12:23.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.990+0000 7f89db400640 1 -- 172.21.15.62:0/1970941854 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4032440 con 0x7f89dc0b27a0 2024-04-25T09:12:23.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.990+0000 7f89db400640 1 -- 172.21.15.62:0/1970941854 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4032810 con 0x7f89dc0b27a0 2024-04-25T09:12:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.990+0000 7f89e2200640 1 -- 172.21.15.62:0/1970941854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 msgr2=0x7f89dc0d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.990+0000 7f89e2200640 1 --2- 172.21.15.62:0/1970941854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc0d26f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f89d40097a0 tx=0x7f89d402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.990+0000 7f89e2200640 1 -- 172.21.15.62:0/1970941854 shutdown_connections 2024-04-25T09:12:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.990+0000 7f89e2200640 1 --2- 172.21.15.62:0/1970941854 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc0d26f0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.990+0000 7f89e2200640 1 -- 172.21.15.62:0/1970941854 >> 172.21.15.62:0/1970941854 conn(0x7f89dc066d10 msgr2=0x7f89dc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.991+0000 7f89e2200640 1 -- 172.21.15.62:0/1970941854 shutdown_connections 2024-04-25T09:12:23.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.991+0000 7f89e2200640 1 -- 172.21.15.62:0/1970941854 wait complete. 2024-04-25T09:12:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.992+0000 7f89e2200640 1 Processor -- start 2024-04-25T09:12:23.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.993+0000 7f89e2200640 1 -- start start 2024-04-25T09:12:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e2200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e2200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89dc0b56f0 0x7f89dc1f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e2200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89dc0b6a90 0x7f89dc157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e2200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0b2ee0 con 0x7f89dc0b27a0 2024-04-25T09:12:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e2200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0b2d60 con 0x7f89dc0b56f0 2024-04-25T09:12:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e2200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0d2c30 con 0x7f89dc0b6a90 2024-04-25T09:12:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc15b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41370/0 (socket says 172.21.15.62:41370) 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e0e00640 1 -- 172.21.15.62:0/184069855 learned_addr learned my addr 172.21.15.62:0/184069855 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89e1800640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89dc0b6a90 0x7f89dc157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.994+0000 7f89dbe00640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89dc0b56f0 0x7f89dc1f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89e0e00640 1 -- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89dc0b6a90 msgr2=0x7f89dc157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89e0e00640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89dc0b6a90 0x7f89dc157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89e0e00640 1 -- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89dc0b56f0 msgr2=0x7f89dc1f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89e0e00640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89dc0b56f0 0x7f89dc1f4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89e0e00640 1 -- 172.21.15.62:0/184069855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc15b7c0 con 0x7f89dc0b27a0 2024-04-25T09:12:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89dbe00640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89dc0b56f0 0x7f89dc1f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89e0e00640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc15b1e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00c4d0 tx=0x7f89cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89d8c00640 1 -- 172.21.15.62:0/184069855 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc018070 con 0x7f89dc0b27a0 2024-04-25T09:12:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89d8c00640 1 -- 172.21.15.62:0/184069855 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc007500 con 0x7f89dc0b27a0 2024-04-25T09:12:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89d8c00640 1 -- 172.21.15.62:0/184069855 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc004a20 con 0x7f89dc0b27a0 2024-04-25T09:12:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.995+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc15baa0 con 0x7f89dc0b27a0 2024-04-25T09:12:23.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.996+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc15c060 con 0x7f89dc0b27a0 2024-04-25T09:12:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.997+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc04ef20 con 0x7f89dc0b27a0 2024-04-25T09:12:23.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.998+0000 7f89d8c00640 1 -- 172.21.15.62:0/184069855 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89cc004bc0 con 0x7f89dc0b27a0 2024-04-25T09:12:23.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.999+0000 7f89d8c00640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89b806b4d0 0x7f89b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:23.999+0000 7f89d8c00640 1 -- 172.21.15.62:0/184069855 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89cc089560 con 0x7f89dc0b27a0 2024-04-25T09:12:23.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.000+0000 7f89dbe00640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89b806b4d0 0x7f89b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:23.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.001+0000 7f89dbe00640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89b806b4d0 0x7f89b806d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d2620 tx=0x7f89c40073a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:23.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.003+0000 7f89d8c00640 1 -- 172.21.15.62:0/184069855 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89cc058340 con 0x7f89dc0b27a0 2024-04-25T09:12:24.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.204+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89dc0ce580 con 0x7f89dc0b27a0 2024-04-25T09:12:24.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.206+0000 7f89d8c00640 1 -- 172.21.15.62:0/184069855 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16673 (secure 0 0 0) 0x7f89cc057c40 con 0x7f89dc0b27a0 2024-04-25T09:12:24.194 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:24.194 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":96,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:22.809392+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:20.654480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:21.833814+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:24.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89b806b4d0 msgr2=0x7f89b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:24.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89b806b4d0 0x7f89b806d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d2620 tx=0x7f89c40073a0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 msgr2=0x7f89dc15b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc15b1e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00c4d0 tx=0x7f89cc00c990 comp rx=0 tx=0).stop 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 shutdown_connections 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f89b806b4d0 0x7f89b806d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89dc0b6a90 0x7f89dc157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89dc0b56f0 0x7f89dc1f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 --2- 172.21.15.62:0/184069855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89dc0b27a0 0x7f89dc15b1e0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 >> 172.21.15.62:0/184069855 conn(0x7f89dc066d10 msgr2=0x7f89dc0d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 shutdown_connections 2024-04-25T09:12:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.208+0000 7f89e2200640 1 -- 172.21.15.62:0/184069855 wait complete. 2024-04-25T09:12:24.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:12:24.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.432+0000 7f0327e00640 1 Processor -- start 2024-04-25T09:12:24.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.432+0000 7f0327e00640 1 -- start start 2024-04-25T09:12:24.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0327e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f032806d9d0 0x7f032815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:24.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0327e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f032815e5f0 0x7f03280b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:24.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0327e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032815ee30 con 0x7f03280ceb50 2024-04-25T09:12:24.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0327e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03281618b0 con 0x7f032815e5f0 2024-04-25T09:12:24.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0327e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0328161b90 con 0x7f032806d9d0 2024-04-25T09:12:24.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0326a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f03280ceb50 0x7f032806d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38572/0 (socket says 172.21.15.62:38572) 2024-04-25T09:12:24.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0326a00640 1 -- 172.21.15.62:0/3765804379 learned_addr learned my addr 172.21.15.62:0/3765804379 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:24.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0327400640 1 --2- 172.21.15.62:0/3765804379 >> v2:172.21.15.62:3300/0 conn(0x7f032815e5f0 0x7f03280b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:24.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.434+0000 7f0326000640 1 --2- 172.21.15.62:0/3765804379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f032806d9d0 0x7f032815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:24.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.435+0000 7f0327400640 1 -- 172.21.15.62:0/3765804379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f032806d9d0 msgr2=0x7f032815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:24.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.435+0000 7f0327400640 1 --2- 172.21.15.62:0/3765804379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f032806d9d0 0x7f032815e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.435+0000 7f0327400640 1 -- 172.21.15.62:0/3765804379 >> v1:172.21.15.62:6789/0 conn(0x7f03280ceb50 legacy=0x7f032806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:24.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.435+0000 7f0327400640 1 -- 172.21.15.62:0/3765804379 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03280cef30 con 0x7f032815e5f0 2024-04-25T09:12:24.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.435+0000 7f0325600640 1 -- 172.21.15.62:0/3765804379 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805063233 0 0) 0x7f032815ee30 con 0x7f03280ceb50 2024-04-25T09:12:24.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.436+0000 7f0327400640 1 --2- 172.21.15.62:0/3765804379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032815e5f0 0x7f03280b4550 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f03200097a0 tx=0x7f032002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8157647d9bb194fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:24.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.436+0000 7f0325600640 1 -- 172.21.15.62:0/3765804379 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320037070 con 0x7f032815e5f0 2024-04-25T09:12:24.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.436+0000 7f0325600640 1 -- 172.21.15.62:0/3765804379 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032002ae50 con 0x7f032815e5f0 2024-04-25T09:12:24.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.436+0000 7f0325600640 1 -- 172.21.15.62:0/3765804379 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320032660 con 0x7f032815e5f0 2024-04-25T09:12:24.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.437+0000 7f0327e00640 1 -- 172.21.15.62:0/3765804379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032815e5f0 msgr2=0x7f03280b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:24.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.437+0000 7f0327e00640 1 --2- 172.21.15.62:0/3765804379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032815e5f0 0x7f03280b4550 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f03200097a0 tx=0x7f032002a100 comp rx=0 tx=0).stop 2024-04-25T09:12:24.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.437+0000 7f0327e00640 1 -- 172.21.15.62:0/3765804379 shutdown_connections 2024-04-25T09:12:24.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.437+0000 7f0327e00640 1 --2- 172.21.15.62:0/3765804379 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032815e5f0 0x7f03280b4550 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.437+0000 7f0327e00640 1 --2- 172.21.15.62:0/3765804379 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f032806d9d0 0x7f032815e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.437+0000 7f0327e00640 1 -- 172.21.15.62:0/3765804379 >> 172.21.15.62:0/3765804379 conn(0x7f0328066f20 msgr2=0x7f03280b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:24.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.437+0000 7f0327e00640 1 -- 172.21.15.62:0/3765804379 shutdown_connections 2024-04-25T09:12:24.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.437+0000 7f0327e00640 1 -- 172.21.15.62:0/3765804379 wait complete. 2024-04-25T09:12:24.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.439+0000 7f0327e00640 1 Processor -- start 2024-04-25T09:12:24.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.439+0000 7f0327e00640 1 -- start start 2024-04-25T09:12:24.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032806d9d0 0x7f0328143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:24.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0327e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03280ceb50 0x7f0328140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:24.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0327e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e5f0 0x7f032806d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:24.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0327e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328162510 con 0x7f032806d9d0 2024-04-25T09:12:24.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0327e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328162390 con 0x7f032815e5f0 2024-04-25T09:12:24.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0327e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03280b4a90 con 0x7f03280ceb50 2024-04-25T09:12:24.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032806d9d0 0x7f0328143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:24.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032806d9d0 0x7f0328143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41386/0 (socket says 172.21.15.62:41386) 2024-04-25T09:12:24.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0326a00640 1 -- 172.21.15.62:0/1827392734 learned_addr learned my addr 172.21.15.62:0/1827392734 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0326a00640 1 -- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03280ceb50 msgr2=0x7f0328140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0326a00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03280ceb50 0x7f0328140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0326a00640 1 -- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e5f0 msgr2=0x7f032806d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0326a00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e5f0 0x7f032806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.440+0000 7f0326a00640 1 -- 172.21.15.62:0/1827392734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03281559c0 con 0x7f032806d9d0 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.441+0000 7f0326a00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032806d9d0 0x7f0328143640 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f031800c8c0 tx=0x7f031800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.441+0000 7f031ea00640 1 -- 172.21.15.62:0/1827392734 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318018070 con 0x7f032806d9d0 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.441+0000 7f031ea00640 1 -- 172.21.15.62:0/1827392734 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031800a640 con 0x7f032806d9d0 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.441+0000 7f031ea00640 1 -- 172.21.15.62:0/1827392734 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031800aa30 con 0x7f032806d9d0 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.441+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0328155f40 con 0x7f032806d9d0 2024-04-25T09:12:24.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.441+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0328142200 con 0x7f032806d9d0 2024-04-25T09:12:24.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.442+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032806abe0 con 0x7f032806d9d0 2024-04-25T09:12:24.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.443+0000 7f031ea00640 1 -- 172.21.15.62:0/1827392734 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f031800abd0 con 0x7f032806d9d0 2024-04-25T09:12:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.445+0000 7f031ea00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f030406b4d0 0x7f030406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.445+0000 7f031ea00640 1 -- 172.21.15.62:0/1827392734 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03180899c0 con 0x7f032806d9d0 2024-04-25T09:12:24.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.445+0000 7f0326000640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f030406b4d0 0x7f030406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:24.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.447+0000 7f0326000640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f030406b4d0 0x7f030406d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f03280aecb0 tx=0x7f0310008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:24.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.449+0000 7f031ea00640 1 -- 172.21.15.62:0/1827392734 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03180587a0 con 0x7f032806d9d0 2024-04-25T09:12:24.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:24.689+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f03281433b0 con 0x7f032806d9d0 2024-04-25T09:12:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.372+0000 7f031ea00640 1 -- 172.21.15.62:0/1827392734 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7f03180580a0 con 0x7f032806d9d0 2024-04-25T09:12:25.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.375+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f030406b4d0 msgr2=0x7f030406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:25.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.375+0000 7f0327e00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f030406b4d0 0x7f030406d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f03280aecb0 tx=0x7f0310008040 comp rx=0 tx=0).stop 2024-04-25T09:12:25.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.375+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032806d9d0 msgr2=0x7f0328143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:25.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.375+0000 7f0327e00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032806d9d0 0x7f0328143640 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f031800c8c0 tx=0x7f031800cd80 comp rx=0 tx=0).stop 2024-04-25T09:12:25.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.376+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 shutdown_connections 2024-04-25T09:12:25.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.376+0000 7f0327e00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f030406b4d0 0x7f030406d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.376+0000 7f0327e00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f032815e5f0 0x7f032806d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.376+0000 7f0327e00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03280ceb50 0x7f0328140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.376+0000 7f0327e00640 1 --2- 172.21.15.62:0/1827392734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f032806d9d0 0x7f0328143640 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.376+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 >> 172.21.15.62:0/1827392734 conn(0x7f0328066f20 msgr2=0x7f0328160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:25.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.376+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 shutdown_connections 2024-04-25T09:12:25.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.376+0000 7f0327e00640 1 -- 172.21.15.62:0/1827392734 wait complete. 2024-04-25T09:12:25.385 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:12:25.385 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:12:25.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.609+0000 7f435f200640 1 Processor -- start 2024-04-25T09:12:25.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.609+0000 7f435f200640 1 -- start start 2024-04-25T09:12:25.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.610+0000 7f435f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4360163310 0x7f4360165700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:25.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.610+0000 7f435f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43600b4430 con 0x7f43600b2d80 2024-04-25T09:12:25.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.611+0000 7f435f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43600b4710 con 0x7f43600b54d0 2024-04-25T09:12:25.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.611+0000 7f435f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43600b49f0 con 0x7f4360163310 2024-04-25T09:12:25.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.611+0000 7f435de00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f43600b2d80 0x7f43600b3160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45622/0 (socket says 172.21.15.62:45622) 2024-04-25T09:12:25.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.611+0000 7f435de00640 1 -- 172.21.15.62:0/749531801 learned_addr learned my addr 172.21.15.62:0/749531801 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.611+0000 7f435e800640 1 --2- 172.21.15.62:0/749531801 >> v2:172.21.15.121:3300/0 conn(0x7f4360163310 0x7f4360165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.611+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280711095 0 0) 0x7f43600b4430 con 0x7f43600b2d80 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.611+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4344003520 con 0x7f43600b2d80 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2428879568 0 0) 0x7f4344003520 con 0x7f43600b2d80 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 >> v2:172.21.15.121:3300/0 conn(0x7f4360163310 msgr2=0x7f4360165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 --2- 172.21.15.62:0/749531801 >> v2:172.21.15.121:3300/0 conn(0x7f4360163310 0x7f4360165700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 >> v1:172.21.15.121:6789/0 conn(0x7f43600b54d0 legacy=0x7f43600ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43600b4cd0 con 0x7f43600b2d80 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f4350001670 con 0x7f43600b2d80 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 >> v1:172.21.15.62:6789/0 conn(0x7f43600b2d80 legacy=0x7f43600b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f4344003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:25.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344004800 0x7f4344004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43440051c0 0x7f434400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600b49f0 con 0x7f4344003a00 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434400e2d0 con 0x7f4344004800 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434400e4d0 con 0x7f43440051c0 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.612+0000 7f435de00640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f4344003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.613+0000 7f435d400640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344004800 0x7f4344004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.613+0000 7f435de00640 1 -- 172.21.15.62:0/749531801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43440051c0 msgr2=0x7f434400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.613+0000 7f435de00640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f43440051c0 0x7f434400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.613+0000 7f435de00640 1 -- 172.21.15.62:0/749531801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344004800 msgr2=0x7f4344004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.613+0000 7f435de00640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344004800 0x7f4344004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.613+0000 7f435e800640 1 -- 172.21.15.62:0/749531801 reap_dead start 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.613+0000 7f435de00640 1 -- 172.21.15.62:0/749531801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344003520 con 0x7f4344003a00 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.613+0000 7f435d400640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4344004800 0x7f4344004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:25.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.614+0000 7f435de00640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f4344003e20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f435000c0e0 tx=0x7f4344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387d702e84aaf62f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:25.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.614+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350007c20 con 0x7f4344003a00 2024-04-25T09:12:25.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.614+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4350005af0 con 0x7f4344003a00 2024-04-25T09:12:25.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.614+0000 7f435ca00640 1 -- 172.21.15.62:0/749531801 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350007070 con 0x7f4344003a00 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.615+0000 7f435f200640 1 -- 172.21.15.62:0/749531801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 msgr2=0x7f4344003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.615+0000 7f435f200640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f4344003e20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f435000c0e0 tx=0x7f4344005830 comp rx=0 tx=0).stop 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.615+0000 7f435f200640 1 -- 172.21.15.62:0/749531801 shutdown_connections 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.615+0000 7f435f200640 1 --2- 172.21.15.62:0/749531801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f4344003e20 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.615+0000 7f435f200640 1 -- 172.21.15.62:0/749531801 >> 172.21.15.62:0/749531801 conn(0x7f4360066f20 msgr2=0x7f43600b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.615+0000 7f435f200640 1 -- 172.21.15.62:0/749531801 shutdown_connections 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.615+0000 7f435f200640 1 -- 172.21.15.62:0/749531801 wait complete. 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.617+0000 7f435f200640 1 Processor -- start 2024-04-25T09:12:25.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.617+0000 7f435f200640 1 -- start start 2024-04-25T09:12:25.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f436013c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:25.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4360156660 0x7f4360139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:25.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f436014c2b0 0x7f436014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:25.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600b3160 con 0x7f4344003a00 2024-04-25T09:12:25.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600b40e0 con 0x7f4360156660 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43600b32e0 con 0x7f436014c2b0 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f436013c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f436013c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44890/0 (socket says 172.21.15.62:44890) 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.618+0000 7f435de00640 1 -- 172.21.15.62:0/3079438534 learned_addr learned my addr 172.21.15.62:0/3079438534 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f435de00640 1 -- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f436014c2b0 msgr2=0x7f436014c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f435de00640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f436014c2b0 0x7f436014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f435de00640 1 -- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4360156660 msgr2=0x7f4360139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f435de00640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4360156660 0x7f4360139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f435de00640 1 -- 172.21.15.62:0/3079438534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360139f90 con 0x7f4344003a00 2024-04-25T09:12:25.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f435de00640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f436013c320 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f435000c0e0 tx=0x7f43600b4ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f4356000640 1 -- 172.21.15.62:0/3079438534 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350037040 con 0x7f4344003a00 2024-04-25T09:12:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f4356000640 1 -- 172.21.15.62:0/3079438534 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4350041070 con 0x7f4344003a00 2024-04-25T09:12:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f4356000640 1 -- 172.21.15.62:0/3079438534 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350001670 con 0x7f4344003a00 2024-04-25T09:12:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436013c900 con 0x7f4344003a00 2024-04-25T09:12:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.619+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436013cec0 con 0x7f4344003a00 2024-04-25T09:12:25.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.621+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436006abe0 con 0x7f4344003a00 2024-04-25T09:12:25.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.621+0000 7f4356000640 1 -- 172.21.15.62:0/3079438534 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43500049e0 con 0x7f4344003a00 2024-04-25T09:12:25.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.622+0000 7f4356000640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f434006b4d0 0x7f434006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:25.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.623+0000 7f4356000640 1 -- 172.21.15.62:0/3079438534 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43500aa060 con 0x7f4344003a00 2024-04-25T09:12:25.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.623+0000 7f435d400640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f434006b4d0 0x7f434006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:25.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.624+0000 7f435d400640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f434006b4d0 0x7f434006d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f434c007fd0 tx=0x7f434c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:25.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.627+0000 7f4356000640 1 -- 172.21.15.62:0/3079438534 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4350078e40 con 0x7f4344003a00 2024-04-25T09:12:25.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:25.866+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f43601630f0 con 0x7f4344003a00 2024-04-25T09:12:26.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.385+0000 7f4356000640 1 -- 172.21.15.62:0/3079438534 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4350078740 con 0x7f4344003a00 2024-04-25T09:12:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f434006b4d0 msgr2=0x7f434006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f434006b4d0 0x7f434006d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f434c007fd0 tx=0x7f434c009040 comp rx=0 tx=0).stop 2024-04-25T09:12:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 msgr2=0x7f436013c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f436013c320 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f435000c0e0 tx=0x7f43600b4ea0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 shutdown_connections 2024-04-25T09:12:26.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f434006b4d0 0x7f434006d990 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f436014c2b0 0x7f436014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4360156660 0x7f4360139a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 --2- 172.21.15.62:0/3079438534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4344003a00 0x7f436013c320 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.389+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 >> 172.21.15.62:0/3079438534 conn(0x7f4360066f20 msgr2=0x7f43600cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:26.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.390+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 shutdown_connections 2024-04-25T09:12:26.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.390+0000 7f435f200640 1 -- 172.21.15.62:0/3079438534 wait complete. 2024-04-25T09:12:26.398 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:12:26.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:26.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.620+0000 7ff1f5000640 1 Processor -- start 2024-04-25T09:12:26.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.620+0000 7ff1f5000640 1 -- start start 2024-04-25T09:12:26.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.622+0000 7ff1f5000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1f00cccf0 0x7ff1f00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:26.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.622+0000 7ff1f5000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f00aed00 0x7ff1f00cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:26.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.622+0000 7ff1f5000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f006ca70 con 0x7ff1f015bd10 2024-04-25T09:12:26.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.622+0000 7ff1f5000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f006d660 con 0x7ff1f00cccf0 2024-04-25T09:12:26.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.622+0000 7ff1f5000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f015c0f0 con 0x7ff1f00aed00 2024-04-25T09:12:26.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.622+0000 7ff1ed400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1f00cccf0 0x7ff1f00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:26.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.622+0000 7ff1ed400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1f00cccf0 0x7ff1f00ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44894/0 (socket says 172.21.15.62:44894) 2024-04-25T09:12:26.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.622+0000 7ff1ed400640 1 -- 172.21.15.62:0/1770751274 learned_addr learned my addr 172.21.15.62:0/1770751274 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:26.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.623+0000 7ff1ee800640 1 --2- 172.21.15.62:0/1770751274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f00aed00 0x7ff1f00cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:26.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.623+0000 7ff1ed400640 1 -- 172.21.15.62:0/1770751274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f00aed00 msgr2=0x7ff1f00cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:26.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.623+0000 7ff1ed400640 1 --2- 172.21.15.62:0/1770751274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f00aed00 0x7ff1f00cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.623+0000 7ff1ed400640 1 -- 172.21.15.62:0/1770751274 >> v1:172.21.15.121:6789/0 conn(0x7ff1f015bd10 legacy=0x7ff1f00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:26.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.623+0000 7ff1ed400640 1 -- 172.21.15.62:0/1770751274 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f006cd30 con 0x7ff1f00cccf0 2024-04-25T09:12:26.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.624+0000 7ff1ed400640 1 --2- 172.21.15.62:0/1770751274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f00ae7c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00cf720 tx=0x7ff1dc02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49e33782a140f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:26.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.625+0000 7ff1eca00640 1 -- 172.21.15.62:0/1770751274 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc037070 con 0x7ff1f00cccf0 2024-04-25T09:12:26.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.625+0000 7ff1eca00640 1 -- 172.21.15.62:0/1770751274 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc032440 con 0x7ff1f00cccf0 2024-04-25T09:12:26.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.625+0000 7ff1eca00640 1 -- 172.21.15.62:0/1770751274 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc032810 con 0x7ff1f00cccf0 2024-04-25T09:12:26.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.626+0000 7ff1f5000640 1 -- 172.21.15.62:0/1770751274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 msgr2=0x7ff1f00ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:26.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.626+0000 7ff1f5000640 1 --2- 172.21.15.62:0/1770751274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f00ae7c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00cf720 tx=0x7ff1dc02a1d0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.626+0000 7ff1f5000640 1 -- 172.21.15.62:0/1770751274 shutdown_connections 2024-04-25T09:12:26.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.626+0000 7ff1f5000640 1 --2- 172.21.15.62:0/1770751274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f00aed00 0x7ff1f00cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.626+0000 7ff1f5000640 1 --2- 172.21.15.62:0/1770751274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f00ae7c0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.626+0000 7ff1f5000640 1 -- 172.21.15.62:0/1770751274 >> 172.21.15.62:0/1770751274 conn(0x7ff1f00668f0 msgr2=0x7ff1f0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:26.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.626+0000 7ff1f5000640 1 -- 172.21.15.62:0/1770751274 shutdown_connections 2024-04-25T09:12:26.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.626+0000 7ff1f5000640 1 -- 172.21.15.62:0/1770751274 wait complete. 2024-04-25T09:12:26.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.628+0000 7ff1f5000640 1 Processor -- start 2024-04-25T09:12:26.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.628+0000 7ff1f5000640 1 -- start start 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1f5000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1f00aed00 0x7ff1f015a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1f5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1f5000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f015bd10 0x7ff1f0152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1f5000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f006daf0 con 0x7ff1f00cccf0 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1f5000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f006d2f0 con 0x7ff1f00aed00 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1f5000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0152bf0 con 0x7ff1f015bd10 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1ed400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1ed400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f0151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44896/0 (socket says 172.21.15.62:44896) 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1ed400640 1 -- 172.21.15.62:0/2850837457 learned_addr learned my addr 172.21.15.62:0/2850837457 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1ed400640 1 -- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f015bd10 msgr2=0x7ff1f0152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:26.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1ed400640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f015bd10 0x7ff1f0152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1ed400640 1 -- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1f00aed00 msgr2=0x7ff1f015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1ed400640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1f00aed00 0x7ff1f015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.629+0000 7ff1ed400640 1 -- 172.21.15.62:0/2850837457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0152ed0 con 0x7ff1f00cccf0 2024-04-25T09:12:26.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.630+0000 7ff1ed400640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f0151fe0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc000c00 tx=0x7ff1dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:26.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.630+0000 7ff1e6000640 1 -- 172.21.15.62:0/2850837457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc041020 con 0x7ff1f00cccf0 2024-04-25T09:12:26.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.630+0000 7ff1e6000640 1 -- 172.21.15.62:0/2850837457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc0306c0 con 0x7ff1f00cccf0 2024-04-25T09:12:26.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.630+0000 7ff1e6000640 1 -- 172.21.15.62:0/2850837457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc037040 con 0x7ff1f00cccf0 2024-04-25T09:12:26.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.630+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f01531b0 con 0x7ff1f00cccf0 2024-04-25T09:12:26.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.630+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f015aca0 con 0x7ff1f00cccf0 2024-04-25T09:12:26.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.632+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f0072c70 con 0x7ff1f00cccf0 2024-04-25T09:12:26.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.633+0000 7ff1e6000640 1 -- 172.21.15.62:0/2850837457 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1dc044020 con 0x7ff1f00cccf0 2024-04-25T09:12:26.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.634+0000 7ff1e6000640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff1d006b4d0 0x7ff1d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.634+0000 7ff1e6000640 1 -- 172.21.15.62:0/2850837457 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1dc0aaa40 con 0x7ff1f00cccf0 2024-04-25T09:12:26.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.635+0000 7ff1ede00640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff1d006b4d0 0x7ff1d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:26.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.636+0000 7ff1ede00640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff1d006b4d0 0x7ff1d006d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0006fd0 tx=0x7ff1e0006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:26.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.639+0000 7ff1e6000640 1 -- 172.21.15.62:0/2850837457 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1dc04b030 con 0x7ff1f00cccf0 2024-04-25T09:12:26.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.877+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1f004f000 con 0x7ff1f00cccf0 2024-04-25T09:12:26.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.878+0000 7ff1e6000640 1 -- 172.21.15.62:0/2850837457 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff1dc0798b0 con 0x7ff1f00cccf0 2024-04-25T09:12:26.866 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:26.866 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1798148826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1798148826},{"type":"v1","addr":"172.21.15.121:6835","nonce":1798148826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2045746846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2045746846},{"type":"v1","addr":"172.21.15.121:6837","nonce":2045746846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1182284820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1182284820},{"type":"v1","addr":"172.21.15.62:6834","nonce":1182284820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:22.810299+0000","modified":"2024-04-25T09:12:25.868942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3997398243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3997398243},{"type":"v1","addr":"172.21.15.62:6835","nonce":3997398243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:12:26.866 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 64 2024-04-25T09:12:26.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff1d006b4d0 msgr2=0x7ff1d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff1d006b4d0 0x7ff1d006d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0006fd0 tx=0x7ff1e0006bc0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 msgr2=0x7ff1f0151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f0151fe0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc000c00 tx=0x7ff1dc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 shutdown_connections 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff1d006b4d0 0x7ff1d006d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f00cccf0 0x7ff1f0151fe0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff1f015bd10 0x7ff1f0152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 --2- 172.21.15.62:0/2850837457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff1f00aed00 0x7ff1f015a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 >> 172.21.15.62:0/2850837457 conn(0x7ff1f00668f0 msgr2=0x7ff1f0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 shutdown_connections 2024-04-25T09:12:26.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:26.881+0000 7ff1f5000640 1 -- 172.21.15.62:0/2850837457 wait complete. 2024-04-25T09:12:26.889 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:12:27.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.112+0000 7fc040c00640 1 Processor -- start 2024-04-25T09:12:27.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.113+0000 7fc040c00640 1 -- start start 2024-04-25T09:12:27.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.114+0000 7fc040c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc03c0b56f0 0x7fc03c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.114+0000 7fc040c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b6a90 0x7fc03c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.114+0000 7fc040c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03c0d3110 con 0x7fc03c0b27a0 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.114+0000 7fc040c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c161780 con 0x7fc03c0b56f0 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.114+0000 7fc040c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0cedc0 con 0x7fc03c0b6a90 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.114+0000 7fc03b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc03c0b56f0 0x7fc03c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.115+0000 7fc03be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc03c0b27a0 0x7fc03c161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45634/0 (socket says 172.21.15.62:45634) 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.115+0000 7fc03b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc03c0b56f0 0x7fc03c160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44906/0 (socket says 172.21.15.62:44906) 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.115+0000 7fc03be00640 1 -- 172.21.15.62:0/718679799 learned_addr learned my addr 172.21.15.62:0/718679799 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.115+0000 7fc03b400640 1 -- 172.21.15.62:0/718679799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b6a90 msgr2=0x7fc03c161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.115+0000 7fc03b400640 1 --2- 172.21.15.62:0/718679799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b6a90 0x7fc03c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.115+0000 7fc03b400640 1 -- 172.21.15.62:0/718679799 >> v1:172.21.15.62:6789/0 conn(0x7fc03c0b27a0 legacy=0x7fc03c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.115+0000 7fc03b400640 1 -- 172.21.15.62:0/718679799 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0d33f0 con 0x7fc03c0b56f0 2024-04-25T09:12:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.116+0000 7fc03b400640 1 --2- 172.21.15.62:0/718679799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b56f0 0x7fc03c160d00 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0097a0 tx=0x7fc02c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28cb6f5de502d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.116+0000 7fc03aa00640 1 -- 172.21.15.62:0/718679799 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c037070 con 0x7fc03c0b56f0 2024-04-25T09:12:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.116+0000 7fc03aa00640 1 -- 172.21.15.62:0/718679799 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c032440 con 0x7fc03c0b56f0 2024-04-25T09:12:27.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc03aa00640 1 -- 172.21.15.62:0/718679799 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c032810 con 0x7fc03c0b56f0 2024-04-25T09:12:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc040c00640 1 -- 172.21.15.62:0/718679799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b56f0 msgr2=0x7fc03c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc040c00640 1 --2- 172.21.15.62:0/718679799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b56f0 0x7fc03c160d00 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0097a0 tx=0x7fc02c02a180 comp rx=0 tx=0).stop 2024-04-25T09:12:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc040c00640 1 -- 172.21.15.62:0/718679799 shutdown_connections 2024-04-25T09:12:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc040c00640 1 --2- 172.21.15.62:0/718679799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b6a90 0x7fc03c161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc040c00640 1 --2- 172.21.15.62:0/718679799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b56f0 0x7fc03c160d00 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc040c00640 1 -- 172.21.15.62:0/718679799 >> 172.21.15.62:0/718679799 conn(0x7fc03c066d10 msgr2=0x7fc03c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc040c00640 1 -- 172.21.15.62:0/718679799 shutdown_connections 2024-04-25T09:12:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.117+0000 7fc040c00640 1 -- 172.21.15.62:0/718679799 wait complete. 2024-04-25T09:12:27.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.119+0000 7fc040c00640 1 Processor -- start 2024-04-25T09:12:27.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.119+0000 7fc040c00640 1 -- start start 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.120+0000 7fc040c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b27a0 0x7fc03c1f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.120+0000 7fc040c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c0b56f0 0x7fc03c0c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.120+0000 7fc040c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b6a90 0x7fc03c0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.120+0000 7fc040c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b2d60 con 0x7fc03c0b6a90 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.120+0000 7fc040c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0cea50 con 0x7fc03c0b56f0 2024-04-25T09:12:27.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.120+0000 7fc040c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b2ee0 con 0x7fc03c0b27a0 2024-04-25T09:12:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b6a90 0x7fc03c0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b6a90 0x7fc03c0c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44914/0 (socket says 172.21.15.62:44914) 2024-04-25T09:12:27.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03be00640 1 -- 172.21.15.62:0/3365635327 learned_addr learned my addr 172.21.15.62:0/3365635327 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03b400640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b27a0 0x7fc03c1f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc033e00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c0b56f0 0x7fc03c0c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03be00640 1 -- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b27a0 msgr2=0x7fc03c1f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03be00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b27a0 0x7fc03c1f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03be00640 1 -- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c0b56f0 msgr2=0x7fc03c0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03be00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c0b56f0 0x7fc03c0c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.121+0000 7fc03be00640 1 -- 172.21.15.62:0/3365635327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c1f4cf0 con 0x7fc03c0b6a90 2024-04-25T09:12:27.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.122+0000 7fc03be00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b6a90 0x7fc03c0c9b00 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc03400cbc0 tx=0x7fc03400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.122+0000 7fc033400640 1 -- 172.21.15.62:0/3365635327 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034017070 con 0x7fc03c0b6a90 2024-04-25T09:12:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.122+0000 7fc033400640 1 -- 172.21.15.62:0/3365635327 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03400ad40 con 0x7fc03c0b6a90 2024-04-25T09:12:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.122+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c1f40e0 con 0x7fc03c0b6a90 2024-04-25T09:12:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.122+0000 7fc033400640 1 -- 172.21.15.62:0/3365635327 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034012600 con 0x7fc03c0b6a90 2024-04-25T09:12:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.123+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c1f46a0 con 0x7fc03c0b6a90 2024-04-25T09:12:27.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.124+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c04ef20 con 0x7fc03c0b6a90 2024-04-25T09:12:27.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.124+0000 7fc033400640 1 -- 172.21.15.62:0/3365635327 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc03401a070 con 0x7fc03c0b6a90 2024-04-25T09:12:27.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.125+0000 7fc033400640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc01806b4d0 0x7fc01806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.125+0000 7fc033400640 1 -- 172.21.15.62:0/3365635327 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc034089e00 con 0x7fc03c0b6a90 2024-04-25T09:12:27.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.126+0000 7fc03b400640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc01806b4d0 0x7fc01806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.127+0000 7fc03b400640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc01806b4d0 0x7fc01806d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc02c034000 tx=0x7fc02c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:27.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.130+0000 7fc033400640 1 -- 172.21.15.62:0/3365635327 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc034058630 con 0x7fc03c0b6a90 2024-04-25T09:12:27.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.330+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc03c0ce580 con 0x7fc03c0b6a90 2024-04-25T09:12:27.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.333+0000 7fc033400640 1 -- 172.21.15.62:0/3365635327 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16660 (secure 0 0 0) 0x7fc03405c6d0 con 0x7fc03c0b6a90 2024-04-25T09:12:27.321 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:27.321 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":97,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:24.344250+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:20.654480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:21.833814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:12:27.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc01806b4d0 msgr2=0x7fc01806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc01806b4d0 0x7fc01806d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc02c034000 tx=0x7fc02c02e000 comp rx=0 tx=0).stop 2024-04-25T09:12:27.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b6a90 msgr2=0x7fc03c0c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b6a90 0x7fc03c0c9b00 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc03400cbc0 tx=0x7fc03400a430 comp rx=0 tx=0).stop 2024-04-25T09:12:27.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 shutdown_connections 2024-04-25T09:12:27.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc01806b4d0 0x7fc01806d990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc03c0b6a90 0x7fc03c0c9b00 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c0b56f0 0x7fc03c0c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 --2- 172.21.15.62:0/3365635327 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc03c0b27a0 0x7fc03c1f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.336+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 >> 172.21.15.62:0/3365635327 conn(0x7fc03c066d10 msgr2=0x7fc03c0b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:27.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.337+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 shutdown_connections 2024-04-25T09:12:27.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.337+0000 7fc040c00640 1 -- 172.21.15.62:0/3365635327 wait complete. 2024-04-25T09:12:27.343 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:12:27.343 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:12:27.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.564+0000 7f7623e00640 1 Processor -- start 2024-04-25T09:12:27.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.564+0000 7f7623e00640 1 -- start start 2024-04-25T09:12:27.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.566+0000 7f7623e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f762415d810 0x7f76240ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.566+0000 7f7623e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f762406d900 0x7f76241675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.566+0000 7f7623e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76240ced70 con 0x7f762415dbf0 2024-04-25T09:12:27.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.566+0000 7f7623e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76240cf030 con 0x7f762415d810 2024-04-25T09:12:27.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.566+0000 7f7623e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76240cf310 con 0x7f762406d900 2024-04-25T09:12:27.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.566+0000 7f7622a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f762415d810 0x7f76240ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.566+0000 7f7622a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f762415d810 0x7f76240ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44926/0 (socket says 172.21.15.62:44926) 2024-04-25T09:12:27.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.566+0000 7f7622a00640 1 -- 172.21.15.62:0/348885957 learned_addr learned my addr 172.21.15.62:0/348885957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:27.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.567+0000 7f7623400640 1 --2- 172.21.15.62:0/348885957 >> v2:172.21.15.121:3300/0 conn(0x7f762406d900 0x7f76241675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.567+0000 7f7622a00640 1 -- 172.21.15.62:0/348885957 >> v2:172.21.15.121:3300/0 conn(0x7f762406d900 msgr2=0x7f76241675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:27.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.567+0000 7f7622a00640 1 --2- 172.21.15.62:0/348885957 >> v2:172.21.15.121:3300/0 conn(0x7f762406d900 0x7f76241675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.567+0000 7f7622a00640 1 -- 172.21.15.62:0/348885957 >> v1:172.21.15.121:6789/0 conn(0x7f762415dbf0 legacy=0x7f762406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:27.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.567+0000 7f7622a00640 1 -- 172.21.15.62:0/348885957 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240c8cd0 con 0x7f762415d810 2024-04-25T09:12:27.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.568+0000 7f7622a00640 1 --2- 172.21.15.62:0/348885957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76240ce830 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f76240cd540 tx=0x7f761402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754e5433fdbcb39a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:27.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7621600640 1 -- 172.21.15.62:0/348885957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614037070 con 0x7f762415d810 2024-04-25T09:12:27.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7621600640 1 -- 172.21.15.62:0/348885957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7614032440 con 0x7f762415d810 2024-04-25T09:12:27.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7621600640 1 -- 172.21.15.62:0/348885957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7614032810 con 0x7f762415d810 2024-04-25T09:12:27.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7623e00640 1 -- 172.21.15.62:0/348885957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 msgr2=0x7f76240ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7623e00640 1 --2- 172.21.15.62:0/348885957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76240ce830 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f76240cd540 tx=0x7f761402a230 comp rx=0 tx=0).stop 2024-04-25T09:12:27.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7623e00640 1 -- 172.21.15.62:0/348885957 shutdown_connections 2024-04-25T09:12:27.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7623e00640 1 --2- 172.21.15.62:0/348885957 >> v2:172.21.15.121:3300/0 conn(0x7f762406d900 0x7f76241675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7623e00640 1 --2- 172.21.15.62:0/348885957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76240ce830 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.569+0000 7f7623e00640 1 -- 172.21.15.62:0/348885957 >> 172.21.15.62:0/348885957 conn(0x7f76240b14d0 msgr2=0x7f76240b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:27.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.570+0000 7f7623e00640 1 -- 172.21.15.62:0/348885957 shutdown_connections 2024-04-25T09:12:27.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.570+0000 7f7623e00640 1 -- 172.21.15.62:0/348885957 wait complete. 2024-04-25T09:12:27.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.571+0000 7f7623e00640 1 Processor -- start 2024-04-25T09:12:27.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.571+0000 7f7623e00640 1 -- start start 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.572+0000 7f7623e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f762406d900 0x7f762414e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.572+0000 7f7623e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76241e9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.572+0000 7f7623e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f762415dbf0 0x7f76240b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.572+0000 7f7623e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76240ca660 con 0x7f762415d810 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.572+0000 7f7623e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76240ca4e0 con 0x7f762415dbf0 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.572+0000 7f7623e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76240ca7e0 con 0x7f762406d900 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76241e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76241e9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44940/0 (socket says 172.21.15.62:44940) 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622000640 1 -- 172.21.15.62:0/938658380 learned_addr learned my addr 172.21.15.62:0/938658380 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622a00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f762406d900 0x7f762414e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7623400640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f762415dbf0 0x7f76240b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622000640 1 -- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f762406d900 msgr2=0x7f762414e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622000640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f762406d900 0x7f762414e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622000640 1 -- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f762415dbf0 msgr2=0x7f76240b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622000640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f762415dbf0 0x7f76240b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622000640 1 -- 172.21.15.62:0/938658380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240b5630 con 0x7f762415d810 2024-04-25T09:12:27.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7622a00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f762406d900 0x7f762414e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:27.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.573+0000 7f7623400640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f762415dbf0 0x7f76240b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:27.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.574+0000 7f7622000640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76241e9610 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f761000b3c0 tx=0x7f761000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:27.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.574+0000 7f761aa00640 1 -- 172.21.15.62:0/938658380 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011830 con 0x7f762415d810 2024-04-25T09:12:27.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.574+0000 7f761aa00640 1 -- 172.21.15.62:0/938658380 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7610016070 con 0x7f762415d810 2024-04-25T09:12:27.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.574+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762414efb0 con 0x7f762415d810 2024-04-25T09:12:27.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.574+0000 7f761aa00640 1 -- 172.21.15.62:0/938658380 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011c50 con 0x7f762415d810 2024-04-25T09:12:27.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.574+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762414f570 con 0x7f762415d810 2024-04-25T09:12:27.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.575+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76240cd8f0 con 0x7f762415d810 2024-04-25T09:12:27.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.576+0000 7f761aa00640 1 -- 172.21.15.62:0/938658380 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f761001b460 con 0x7f762415d810 2024-04-25T09:12:27.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.577+0000 7f761aa00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f760406b4d0 0x7f760406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:27.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.577+0000 7f761aa00640 1 -- 172.21.15.62:0/938658380 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7610089cd0 con 0x7f762415d810 2024-04-25T09:12:27.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.578+0000 7f7622a00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f760406b4d0 0x7f760406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:27.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.579+0000 7f7622a00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f760406b4d0 0x7f760406d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7614034000 tx=0x7f761402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:27.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.582+0000 7f761aa00640 1 -- 172.21.15.62:0/938658380 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76240cd8f0 con 0x7f762415d810 2024-04-25T09:12:27.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.776+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f762404ef20 con 0x7f760406b4d0 2024-04-25T09:12:27.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.796+0000 7f761aa00640 1 -- 172.21.15.62:0/938658380 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252086 (secure 0 0 0) 0x7f762404ef20 con 0x7f760406b4d0 2024-04-25T09:12:27.787 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:27.789 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:12:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.801+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f760406b4d0 msgr2=0x7f760406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.801+0000 7f7623e00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f760406b4d0 0x7f760406d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7614034000 tx=0x7f761402e000 comp rx=0 tx=0).stop 2024-04-25T09:12:27.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.801+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 msgr2=0x7f76241e9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.801+0000 7f7623e00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76241e9610 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f761000b3c0 tx=0x7f761000b880 comp rx=0 tx=0).stop 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.802+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 shutdown_connections 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.802+0000 7f7623e00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f760406b4d0 0x7f760406d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.802+0000 7f7623e00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f762415d810 0x7f76241e9610 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.802+0000 7f7623e00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f762415dbf0 0x7f76240b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.802+0000 7f7623e00640 1 --2- 172.21.15.62:0/938658380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f762406d900 0x7f762414e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.802+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 >> 172.21.15.62:0/938658380 conn(0x7f76240b14d0 msgr2=0x7f762415cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.802+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 shutdown_connections 2024-04-25T09:12:27.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:27.802+0000 7f7623e00640 1 -- 172.21.15.62:0/938658380 wait complete. 2024-04-25T09:12:27.808 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-04-25T09:12:26.095867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429340,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754545380,"statfs":{"total":773094113280,"available":772654469120,"internally_reserved":0,"allocated":10788864,"data_stored":5355132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":428723760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.788381"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719887+0000","last_change":"2024-04-25T09:12:23.363561+0000","last_active":"2024-04-25T09:12:24.719887+0000","last_peered":"2024-04-25T09:12:24.719887+0000","last_clean":"2024-04-25T09:12:24.719887+0000","last_became_active":"2024-04-25T09:12:23.363068+0000","last_became_peered":"2024-04-25T09:12:23.363068+0000","last_unstale":"2024-04-25T09:12:24.719887+0000","last_undegraded":"2024-04-25T09:12:24.719887+0000","last_fullsized":"2024-04-25T09:12:24.719887+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.452687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.368156+0000","last_change":"2024-04-25T09:12:23.357428+0000","last_active":"2024-04-25T09:12:24.368156+0000","last_peered":"2024-04-25T09:12:24.368156+0000","last_clean":"2024-04-25T09:12:24.368156+0000","last_became_active":"2024-04-25T09:12:23.356681+0000","last_became_peered":"2024-04-25T09:12:23.356681+0000","last_unstale":"2024-04-25T09:12:24.368156+0000","last_undegraded":"2024-04-25T09:12:24.368156+0000","last_fullsized":"2024-04-25T09:12:24.368156+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:31.995588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720935+0000","last_change":"2024-04-25T09:12:23.352185+0000","last_active":"2024-04-25T09:12:24.720935+0000","last_peered":"2024-04-25T09:12:24.720935+0000","last_clean":"2024-04-25T09:12:24.720935+0000","last_became_active":"2024-04-25T09:12:23.351897+0000","last_became_peered":"2024-04-25T09:12:23.351897+0000","last_unstale":"2024-04-25T09:12:24.720935+0000","last_undegraded":"2024-04-25T09:12:24.720935+0000","last_fullsized":"2024-04-25T09:12:24.720935+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:49.338135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719605+0000","last_change":"2024-04-25T09:12:23.369219+0000","last_active":"2024-04-25T09:12:24.719605+0000","last_peered":"2024-04-25T09:12:24.719605+0000","last_clean":"2024-04-25T09:12:24.719605+0000","last_became_active":"2024-04-25T09:12:23.368759+0000","last_became_peered":"2024-04-25T09:12:23.368759+0000","last_unstale":"2024-04-25T09:12:24.719605+0000","last_undegraded":"2024-04-25T09:12:24.719605+0000","last_fullsized":"2024-04-25T09:12:24.719605+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:37.359994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718962+0000","last_change":"2024-04-25T09:12:23.351335+0000","last_active":"2024-04-25T09:12:24.718962+0000","last_peered":"2024-04-25T09:12:24.718962+0000","last_clean":"2024-04-25T09:12:24.718962+0000","last_became_active":"2024-04-25T09:12:23.351021+0000","last_became_peered":"2024-04-25T09:12:23.351021+0000","last_unstale":"2024-04-25T09:12:24.718962+0000","last_undegraded":"2024-04-25T09:12:24.718962+0000","last_fullsized":"2024-04-25T09:12:24.718962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:16.915019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717112+0000","last_change":"2024-04-25T09:12:23.367317+0000","last_active":"2024-04-25T09:12:24.717112+0000","last_peered":"2024-04-25T09:12:24.717112+0000","last_clean":"2024-04-25T09:12:24.717112+0000","last_became_active":"2024-04-25T09:12:23.367036+0000","last_became_peered":"2024-04-25T09:12:23.367036+0000","last_unstale":"2024-04-25T09:12:24.717112+0000","last_undegraded":"2024-04-25T09:12:24.717112+0000","last_fullsized":"2024-04-25T09:12:24.717112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:14.123371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367014+0000","last_change":"2024-04-25T09:12:23.354746+0000","last_active":"2024-04-25T09:12:24.367014+0000","last_peered":"2024-04-25T09:12:24.367014+0000","last_clean":"2024-04-25T09:12:24.367014+0000","last_became_active":"2024-04-25T09:12:23.354429+0000","last_became_peered":"2024-04-25T09:12:23.354429+0000","last_unstale":"2024-04-25T09:12:24.367014+0000","last_undegraded":"2024-04-25T09:12:24.367014+0000","last_fullsized":"2024-04-25T09:12:24.367014+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.829948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363016+0000","last_change":"2024-04-25T09:12:23.359967+0000","last_active":"2024-04-25T09:12:24.363016+0000","last_peered":"2024-04-25T09:12:24.363016+0000","last_clean":"2024-04-25T09:12:24.363016+0000","last_became_active":"2024-04-25T09:12:23.358160+0000","last_became_peered":"2024-04-25T09:12:23.358160+0000","last_unstale":"2024-04-25T09:12:24.363016+0000","last_undegraded":"2024-04-25T09:12:24.363016+0000","last_fullsized":"2024-04-25T09:12:24.363016+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:10.876375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720176+0000","last_change":"2024-04-25T09:12:23.362905+0000","last_active":"2024-04-25T09:12:24.720176+0000","last_peered":"2024-04-25T09:12:24.720176+0000","last_clean":"2024-04-25T09:12:24.720176+0000","last_became_active":"2024-04-25T09:12:23.362395+0000","last_became_peered":"2024-04-25T09:12:23.362395+0000","last_unstale":"2024-04-25T09:12:24.720176+0000","last_undegraded":"2024-04-25T09:12:24.720176+0000","last_fullsized":"2024-04-25T09:12:24.720176+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:48.967945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721062+0000","last_change":"2024-04-25T09:12:23.362156+0000","last_active":"2024-04-25T09:12:24.721062+0000","last_peered":"2024-04-25T09:12:24.721062+0000","last_clean":"2024-04-25T09:12:24.721062+0000","last_became_active":"2024-04-25T09:12:23.361860+0000","last_became_peered":"2024-04-25T09:12:23.361860+0000","last_unstale":"2024-04-25T09:12:24.721062+0000","last_undegraded":"2024-04-25T09:12:24.721062+0000","last_fullsized":"2024-04-25T09:12:24.721062+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:07.120712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657560+0000","last_change":"2024-04-25T09:12:23.357120+0000","last_active":"2024-04-25T09:12:24.657560+0000","last_peered":"2024-04-25T09:12:24.657560+0000","last_clean":"2024-04-25T09:12:24.657560+0000","last_became_active":"2024-04-25T09:12:23.356839+0000","last_became_peered":"2024-04-25T09:12:23.356839+0000","last_unstale":"2024-04-25T09:12:24.657560+0000","last_undegraded":"2024-04-25T09:12:24.657560+0000","last_fullsized":"2024-04-25T09:12:24.657560+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:54.174119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363260+0000","last_change":"2024-04-25T09:12:23.352975+0000","last_active":"2024-04-25T09:12:24.363260+0000","last_peered":"2024-04-25T09:12:24.363260+0000","last_clean":"2024-04-25T09:12:24.363260+0000","last_became_active":"2024-04-25T09:12:23.352685+0000","last_became_peered":"2024-04-25T09:12:23.352685+0000","last_unstale":"2024-04-25T09:12:24.363260+0000","last_undegraded":"2024-04-25T09:12:24.363260+0000","last_fullsized":"2024-04-25T09:12:24.363260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:33.916224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719323+0000","last_change":"2024-04-25T09:12:23.364560+0000","last_active":"2024-04-25T09:12:24.719323+0000","last_peered":"2024-04-25T09:12:24.719323+0000","last_clean":"2024-04-25T09:12:24.719323+0000","last_became_active":"2024-04-25T09:12:23.354942+0000","last_became_peered":"2024-04-25T09:12:23.354942+0000","last_unstale":"2024-04-25T09:12:24.719323+0000","last_undegraded":"2024-04-25T09:12:24.719323+0000","last_fullsized":"2024-04-25T09:12:24.719323+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:02.384046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719063+0000","last_change":"2024-04-25T09:12:23.369263+0000","last_active":"2024-04-25T09:12:24.719063+0000","last_peered":"2024-04-25T09:12:24.719063+0000","last_clean":"2024-04-25T09:12:24.719063+0000","last_became_active":"2024-04-25T09:12:23.365974+0000","last_became_peered":"2024-04-25T09:12:23.365974+0000","last_unstale":"2024-04-25T09:12:24.719063+0000","last_undegraded":"2024-04-25T09:12:24.719063+0000","last_fullsized":"2024-04-25T09:12:24.719063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:48.967945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367003+0000","last_change":"2024-04-25T09:12:23.364833+0000","last_active":"2024-04-25T09:12:24.367003+0000","last_peered":"2024-04-25T09:12:24.367003+0000","last_clean":"2024-04-25T09:12:24.367003+0000","last_became_active":"2024-04-25T09:12:23.364499+0000","last_became_peered":"2024-04-25T09:12:23.364499+0000","last_unstale":"2024-04-25T09:12:24.367003+0000","last_undegraded":"2024-04-25T09:12:24.367003+0000","last_fullsized":"2024-04-25T09:12:24.367003+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:19.409124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.716616+0000","last_change":"2024-04-25T09:12:23.355632+0000","last_active":"2024-04-25T09:12:24.716616+0000","last_peered":"2024-04-25T09:12:24.716616+0000","last_clean":"2024-04-25T09:12:24.716616+0000","last_became_active":"2024-04-25T09:12:23.355315+0000","last_became_peered":"2024-04-25T09:12:23.355315+0000","last_unstale":"2024-04-25T09:12:24.716616+0000","last_undegraded":"2024-04-25T09:12:24.716616+0000","last_fullsized":"2024-04-25T09:12:24.716616+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:33.916224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363812+0000","last_change":"2024-04-25T09:12:23.356055+0000","last_active":"2024-04-25T09:12:24.363812+0000","last_peered":"2024-04-25T09:12:24.363812+0000","last_clean":"2024-04-25T09:12:24.363812+0000","last_became_active":"2024-04-25T09:12:23.355778+0000","last_became_peered":"2024-04-25T09:12:23.355778+0000","last_unstale":"2024-04-25T09:12:24.363812+0000","last_undegraded":"2024-04-25T09:12:24.363812+0000","last_fullsized":"2024-04-25T09:12:24.363812+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:40.458845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657017+0000","last_change":"2024-04-25T09:12:23.355611+0000","last_active":"2024-04-25T09:12:24.657017+0000","last_peered":"2024-04-25T09:12:24.657017+0000","last_clean":"2024-04-25T09:12:24.657017+0000","last_became_active":"2024-04-25T09:12:23.355312+0000","last_became_peered":"2024-04-25T09:12:23.355312+0000","last_unstale":"2024-04-25T09:12:24.657017+0000","last_undegraded":"2024-04-25T09:12:24.657017+0000","last_fullsized":"2024-04-25T09:12:24.657017+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:28.470124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720063+0000","last_change":"2024-04-25T09:12:23.362736+0000","last_active":"2024-04-25T09:12:24.720063+0000","last_peered":"2024-04-25T09:12:24.720063+0000","last_clean":"2024-04-25T09:12:24.720063+0000","last_became_active":"2024-04-25T09:12:23.362472+0000","last_became_peered":"2024-04-25T09:12:23.362472+0000","last_unstale":"2024-04-25T09:12:24.720063+0000","last_undegraded":"2024-04-25T09:12:24.720063+0000","last_fullsized":"2024-04-25T09:12:24.720063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:37.790506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718818+0000","last_change":"2024-04-25T09:12:23.365680+0000","last_active":"2024-04-25T09:12:24.718818+0000","last_peered":"2024-04-25T09:12:24.718818+0000","last_clean":"2024-04-25T09:12:24.718818+0000","last_became_active":"2024-04-25T09:12:23.363779+0000","last_became_peered":"2024-04-25T09:12:23.363779+0000","last_unstale":"2024-04-25T09:12:24.718818+0000","last_undegraded":"2024-04-25T09:12:24.718818+0000","last_fullsized":"2024-04-25T09:12:24.718818+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.195167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719412+0000","last_change":"2024-04-25T09:12:23.364578+0000","last_active":"2024-04-25T09:12:24.719412+0000","last_peered":"2024-04-25T09:12:24.719412+0000","last_clean":"2024-04-25T09:12:24.719412+0000","last_became_active":"2024-04-25T09:12:23.364011+0000","last_became_peered":"2024-04-25T09:12:23.364011+0000","last_unstale":"2024-04-25T09:12:24.719412+0000","last_undegraded":"2024-04-25T09:12:24.719412+0000","last_fullsized":"2024-04-25T09:12:24.719412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:54.211322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718546+0000","last_change":"2024-04-25T09:12:23.360423+0000","last_active":"2024-04-25T09:12:24.718546+0000","last_peered":"2024-04-25T09:12:24.718546+0000","last_clean":"2024-04-25T09:12:24.718546+0000","last_became_active":"2024-04-25T09:12:23.360102+0000","last_became_peered":"2024-04-25T09:12:23.360102+0000","last_unstale":"2024-04-25T09:12:24.718546+0000","last_undegraded":"2024-04-25T09:12:24.718546+0000","last_fullsized":"2024-04-25T09:12:24.718546+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:02.513225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.362928+0000","last_change":"2024-04-25T09:12:23.352452+0000","last_active":"2024-04-25T09:12:24.362928+0000","last_peered":"2024-04-25T09:12:24.362928+0000","last_clean":"2024-04-25T09:12:24.362928+0000","last_became_active":"2024-04-25T09:12:23.352136+0000","last_became_peered":"2024-04-25T09:12:23.352136+0000","last_unstale":"2024-04-25T09:12:24.362928+0000","last_undegraded":"2024-04-25T09:12:24.362928+0000","last_fullsized":"2024-04-25T09:12:24.362928+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.201674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656571+0000","last_change":"2024-04-25T09:12:23.367532+0000","last_active":"2024-04-25T09:12:24.656571+0000","last_peered":"2024-04-25T09:12:24.656571+0000","last_clean":"2024-04-25T09:12:24.656571+0000","last_became_active":"2024-04-25T09:12:23.366970+0000","last_became_peered":"2024-04-25T09:12:23.366970+0000","last_unstale":"2024-04-25T09:12:24.656571+0000","last_undegraded":"2024-04-25T09:12:24.656571+0000","last_fullsized":"2024-04-25T09:12:24.656571+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:02.967559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657444+0000","last_change":"2024-04-25T09:12:23.371369+0000","last_active":"2024-04-25T09:12:24.657444+0000","last_peered":"2024-04-25T09:12:24.657444+0000","last_clean":"2024-04-25T09:12:24.657444+0000","last_became_active":"2024-04-25T09:12:23.370785+0000","last_became_peered":"2024-04-25T09:12:23.370785+0000","last_unstale":"2024-04-25T09:12:24.657444+0000","last_undegraded":"2024-04-25T09:12:24.657444+0000","last_fullsized":"2024-04-25T09:12:24.657444+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:36.812863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719683+0000","last_change":"2024-04-25T09:12:23.364411+0000","last_active":"2024-04-25T09:12:24.719683+0000","last_peered":"2024-04-25T09:12:24.719683+0000","last_clean":"2024-04-25T09:12:24.719683+0000","last_became_active":"2024-04-25T09:12:23.364141+0000","last_became_peered":"2024-04-25T09:12:23.364141+0000","last_unstale":"2024-04-25T09:12:24.719683+0000","last_undegraded":"2024-04-25T09:12:24.719683+0000","last_fullsized":"2024-04-25T09:12:24.719683+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:27.575588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720028+0000","last_change":"2024-04-25T09:12:23.370480+0000","last_active":"2024-04-25T09:12:24.720028+0000","last_peered":"2024-04-25T09:12:24.720028+0000","last_clean":"2024-04-25T09:12:24.720028+0000","last_became_active":"2024-04-25T09:12:23.369413+0000","last_became_peered":"2024-04-25T09:12:23.369413+0000","last_unstale":"2024-04-25T09:12:24.720028+0000","last_undegraded":"2024-04-25T09:12:24.720028+0000","last_fullsized":"2024-04-25T09:12:24.720028+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:59.265413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717963+0000","last_change":"2024-04-25T09:12:23.356312+0000","last_active":"2024-04-25T09:12:24.717963+0000","last_peered":"2024-04-25T09:12:24.717963+0000","last_clean":"2024-04-25T09:12:24.717963+0000","last_became_active":"2024-04-25T09:12:23.356042+0000","last_became_peered":"2024-04-25T09:12:23.356042+0000","last_unstale":"2024-04-25T09:12:24.717963+0000","last_undegraded":"2024-04-25T09:12:24.717963+0000","last_fullsized":"2024-04-25T09:12:24.717963+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:33.878768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719063+0000","last_change":"2024-04-25T09:12:23.363469+0000","last_active":"2024-04-25T09:12:24.719063+0000","last_peered":"2024-04-25T09:12:24.719063+0000","last_clean":"2024-04-25T09:12:24.719063+0000","last_became_active":"2024-04-25T09:12:23.363100+0000","last_became_peered":"2024-04-25T09:12:23.363100+0000","last_unstale":"2024-04-25T09:12:24.719063+0000","last_undegraded":"2024-04-25T09:12:24.719063+0000","last_fullsized":"2024-04-25T09:12:24.719063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:31.630427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719016+0000","last_change":"2024-04-25T09:12:23.365167+0000","last_active":"2024-04-25T09:12:24.719016+0000","last_peered":"2024-04-25T09:12:24.719016+0000","last_clean":"2024-04-25T09:12:24.719016+0000","last_became_active":"2024-04-25T09:12:23.364592+0000","last_became_peered":"2024-04-25T09:12:23.364592+0000","last_unstale":"2024-04-25T09:12:24.719016+0000","last_undegraded":"2024-04-25T09:12:24.719016+0000","last_fullsized":"2024-04-25T09:12:24.719016+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.452687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.716857+0000","last_change":"2024-04-25T09:12:23.364754+0000","last_active":"2024-04-25T09:12:24.716857+0000","last_peered":"2024-04-25T09:12:24.716857+0000","last_clean":"2024-04-25T09:12:24.716857+0000","last_became_active":"2024-04-25T09:12:23.364467+0000","last_became_peered":"2024-04-25T09:12:23.364467+0000","last_unstale":"2024-04-25T09:12:24.716857+0000","last_undegraded":"2024-04-25T09:12:24.716857+0000","last_fullsized":"2024-04-25T09:12:24.716857+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:19.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.716951+0000","last_change":"2024-04-25T09:12:23.360379+0000","last_active":"2024-04-25T09:12:24.716951+0000","last_peered":"2024-04-25T09:12:24.716951+0000","last_clean":"2024-04-25T09:12:24.716951+0000","last_became_active":"2024-04-25T09:12:23.359709+0000","last_became_peered":"2024-04-25T09:12:23.359709+0000","last_unstale":"2024-04-25T09:12:24.716951+0000","last_undegraded":"2024-04-25T09:12:24.716951+0000","last_fullsized":"2024-04-25T09:12:24.716951+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:22.449045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363721+0000","last_change":"2024-04-25T09:12:23.350569+0000","last_active":"2024-04-25T09:12:24.363721+0000","last_peered":"2024-04-25T09:12:24.363721+0000","last_clean":"2024-04-25T09:12:24.363721+0000","last_became_active":"2024-04-25T09:12:23.350218+0000","last_became_peered":"2024-04-25T09:12:23.350218+0000","last_unstale":"2024-04-25T09:12:24.363721+0000","last_undegraded":"2024-04-25T09:12:24.363721+0000","last_fullsized":"2024-04-25T09:12:24.363721+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:14.123371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719208+0000","last_change":"2024-04-25T09:12:22.359912+0000","last_active":"2024-04-25T09:12:24.719208+0000","last_peered":"2024-04-25T09:12:24.719208+0000","last_clean":"2024-04-25T09:12:24.719208+0000","last_became_active":"2024-04-25T09:12:22.359523+0000","last_became_peered":"2024-04-25T09:12:22.359523+0000","last_unstale":"2024-04-25T09:12:24.719208+0000","last_undegraded":"2024-04-25T09:12:24.719208+0000","last_fullsized":"2024-04-25T09:12:24.719208+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:59.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720535+0000","last_change":"2024-04-25T09:12:23.355083+0000","last_active":"2024-04-25T09:12:24.720535+0000","last_peered":"2024-04-25T09:12:24.720535+0000","last_clean":"2024-04-25T09:12:24.720535+0000","last_became_active":"2024-04-25T09:12:23.354751+0000","last_became_peered":"2024-04-25T09:12:23.354751+0000","last_unstale":"2024-04-25T09:12:24.720535+0000","last_undegraded":"2024-04-25T09:12:24.720535+0000","last_fullsized":"2024-04-25T09:12:24.720535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:42.666545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721236+0000","last_change":"2024-04-25T09:12:22.352115+0000","last_active":"2024-04-25T09:12:24.721236+0000","last_peered":"2024-04-25T09:12:24.721236+0000","last_clean":"2024-04-25T09:12:24.721236+0000","last_became_active":"2024-04-25T09:12:22.350540+0000","last_became_peered":"2024-04-25T09:12:22.350540+0000","last_unstale":"2024-04-25T09:12:24.721236+0000","last_undegraded":"2024-04-25T09:12:24.721236+0000","last_fullsized":"2024-04-25T09:12:24.721236+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:51.657793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363260+0000","last_change":"2024-04-25T09:12:23.366788+0000","last_active":"2024-04-25T09:12:24.363260+0000","last_peered":"2024-04-25T09:12:24.363260+0000","last_clean":"2024-04-25T09:12:24.363260+0000","last_became_active":"2024-04-25T09:12:23.365851+0000","last_became_peered":"2024-04-25T09:12:23.365851+0000","last_unstale":"2024-04-25T09:12:24.363260+0000","last_undegraded":"2024-04-25T09:12:24.363260+0000","last_fullsized":"2024-04-25T09:12:24.363260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:19.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657487+0000","last_change":"2024-04-25T09:12:22.361074+0000","last_active":"2024-04-25T09:12:24.657487+0000","last_peered":"2024-04-25T09:12:24.657487+0000","last_clean":"2024-04-25T09:12:24.657487+0000","last_became_active":"2024-04-25T09:12:22.360567+0000","last_became_peered":"2024-04-25T09:12:22.360567+0000","last_unstale":"2024-04-25T09:12:24.657487+0000","last_undegraded":"2024-04-25T09:12:24.657487+0000","last_fullsized":"2024-04-25T09:12:24.657487+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:01.496718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717613+0000","last_change":"2024-04-25T09:12:23.366562+0000","last_active":"2024-04-25T09:12:24.717613+0000","last_peered":"2024-04-25T09:12:24.717613+0000","last_clean":"2024-04-25T09:12:24.717613+0000","last_became_active":"2024-04-25T09:12:23.366174+0000","last_became_peered":"2024-04-25T09:12:23.366174+0000","last_unstale":"2024-04-25T09:12:24.717613+0000","last_undegraded":"2024-04-25T09:12:24.717613+0000","last_fullsized":"2024-04-25T09:12:24.717613+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.194225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718667+0000","last_change":"2024-04-25T09:12:22.359789+0000","last_active":"2024-04-25T09:12:24.718667+0000","last_peered":"2024-04-25T09:12:24.718667+0000","last_clean":"2024-04-25T09:12:24.718667+0000","last_became_active":"2024-04-25T09:12:22.358566+0000","last_became_peered":"2024-04-25T09:12:22.358566+0000","last_unstale":"2024-04-25T09:12:24.718667+0000","last_undegraded":"2024-04-25T09:12:24.718667+0000","last_fullsized":"2024-04-25T09:12:24.718667+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:09.859868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717744+0000","last_change":"2024-04-25T09:12:23.362086+0000","last_active":"2024-04-25T09:12:24.717744+0000","last_peered":"2024-04-25T09:12:24.717744+0000","last_clean":"2024-04-25T09:12:24.717744+0000","last_became_active":"2024-04-25T09:12:23.359743+0000","last_became_peered":"2024-04-25T09:12:23.359743+0000","last_unstale":"2024-04-25T09:12:24.717744+0000","last_undegraded":"2024-04-25T09:12:24.717744+0000","last_fullsized":"2024-04-25T09:12:24.717744+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:04.822543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363088+0000","last_change":"2024-04-25T09:12:22.346968+0000","last_active":"2024-04-25T09:12:24.363088+0000","last_peered":"2024-04-25T09:12:24.363088+0000","last_clean":"2024-04-25T09:12:24.363088+0000","last_became_active":"2024-04-25T09:12:22.346347+0000","last_became_peered":"2024-04-25T09:12:22.346347+0000","last_unstale":"2024-04-25T09:12:24.363088+0000","last_undegraded":"2024-04-25T09:12:24.363088+0000","last_fullsized":"2024-04-25T09:12:24.363088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:30.979081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718048+0000","last_change":"2024-04-25T09:12:23.356680+0000","last_active":"2024-04-25T09:12:24.718048+0000","last_peered":"2024-04-25T09:12:24.718048+0000","last_clean":"2024-04-25T09:12:24.718048+0000","last_became_active":"2024-04-25T09:12:23.356359+0000","last_became_peered":"2024-04-25T09:12:23.356359+0000","last_unstale":"2024-04-25T09:12:24.718048+0000","last_undegraded":"2024-04-25T09:12:24.718048+0000","last_fullsized":"2024-04-25T09:12:24.718048+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:40.458845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717133+0000","last_change":"2024-04-25T09:12:22.354477+0000","last_active":"2024-04-25T09:12:24.717133+0000","last_peered":"2024-04-25T09:12:24.717133+0000","last_clean":"2024-04-25T09:12:24.717133+0000","last_became_active":"2024-04-25T09:12:22.354117+0000","last_became_peered":"2024-04-25T09:12:22.354117+0000","last_unstale":"2024-04-25T09:12:24.717133+0000","last_undegraded":"2024-04-25T09:12:24.717133+0000","last_fullsized":"2024-04-25T09:12:24.717133+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:01.951052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717754+0000","last_change":"2024-04-25T09:12:23.358579+0000","last_active":"2024-04-25T09:12:24.717754+0000","last_peered":"2024-04-25T09:12:24.717754+0000","last_clean":"2024-04-25T09:12:24.717754+0000","last_became_active":"2024-04-25T09:12:23.357627+0000","last_became_peered":"2024-04-25T09:12:23.357627+0000","last_unstale":"2024-04-25T09:12:24.717754+0000","last_undegraded":"2024-04-25T09:12:24.717754+0000","last_fullsized":"2024-04-25T09:12:24.717754+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:15.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719192+0000","last_change":"2024-04-25T09:12:22.355980+0000","last_active":"2024-04-25T09:12:24.719192+0000","last_peered":"2024-04-25T09:12:24.719192+0000","last_clean":"2024-04-25T09:12:24.719192+0000","last_became_active":"2024-04-25T09:12:22.355680+0000","last_became_peered":"2024-04-25T09:12:22.355680+0000","last_unstale":"2024-04-25T09:12:24.719192+0000","last_undegraded":"2024-04-25T09:12:24.719192+0000","last_fullsized":"2024-04-25T09:12:24.719192+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:46.792156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718685+0000","last_change":"2024-04-25T09:12:23.351336+0000","last_active":"2024-04-25T09:12:24.718685+0000","last_peered":"2024-04-25T09:12:24.718685+0000","last_clean":"2024-04-25T09:12:24.718685+0000","last_became_active":"2024-04-25T09:12:23.350504+0000","last_became_peered":"2024-04-25T09:12:23.350504+0000","last_unstale":"2024-04-25T09:12:24.718685+0000","last_undegraded":"2024-04-25T09:12:24.718685+0000","last_fullsized":"2024-04-25T09:12:24.718685+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:43.386330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718941+0000","last_change":"2024-04-25T09:12:22.354648+0000","last_active":"2024-04-25T09:12:24.718941+0000","last_peered":"2024-04-25T09:12:24.718941+0000","last_clean":"2024-04-25T09:12:24.718941+0000","last_became_active":"2024-04-25T09:12:22.354149+0000","last_became_peered":"2024-04-25T09:12:22.354149+0000","last_unstale":"2024-04-25T09:12:24.718941+0000","last_undegraded":"2024-04-25T09:12:24.718941+0000","last_fullsized":"2024-04-25T09:12:24.718941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:41.968283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720075+0000","last_change":"2024-04-25T09:12:23.351435+0000","last_active":"2024-04-25T09:12:24.720075+0000","last_peered":"2024-04-25T09:12:24.720075+0000","last_clean":"2024-04-25T09:12:24.720075+0000","last_became_active":"2024-04-25T09:12:23.349248+0000","last_became_peered":"2024-04-25T09:12:23.349248+0000","last_unstale":"2024-04-25T09:12:24.720075+0000","last_undegraded":"2024-04-25T09:12:24.720075+0000","last_fullsized":"2024-04-25T09:12:24.720075+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:42.666545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720461+0000","last_change":"2024-04-25T09:12:22.358623+0000","last_active":"2024-04-25T09:12:24.720461+0000","last_peered":"2024-04-25T09:12:24.720461+0000","last_clean":"2024-04-25T09:12:24.720461+0000","last_became_active":"2024-04-25T09:12:22.357743+0000","last_became_peered":"2024-04-25T09:12:22.357743+0000","last_unstale":"2024-04-25T09:12:24.720461+0000","last_undegraded":"2024-04-25T09:12:24.720461+0000","last_fullsized":"2024-04-25T09:12:24.720461+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:32.697694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720347+0000","last_change":"2024-04-25T09:12:23.359572+0000","last_active":"2024-04-25T09:12:24.720347+0000","last_peered":"2024-04-25T09:12:24.720347+0000","last_clean":"2024-04-25T09:12:24.720347+0000","last_became_active":"2024-04-25T09:12:23.357744+0000","last_became_peered":"2024-04-25T09:12:23.357744+0000","last_unstale":"2024-04-25T09:12:24.720347+0000","last_undegraded":"2024-04-25T09:12:24.720347+0000","last_fullsized":"2024-04-25T09:12:24.720347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:37.790506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721181+0000","last_change":"2024-04-25T09:12:22.345277+0000","last_active":"2024-04-25T09:12:24.721181+0000","last_peered":"2024-04-25T09:12:24.721181+0000","last_clean":"2024-04-25T09:12:24.721181+0000","last_became_active":"2024-04-25T09:12:22.344740+0000","last_became_peered":"2024-04-25T09:12:22.344740+0000","last_unstale":"2024-04-25T09:12:24.721181+0000","last_undegraded":"2024-04-25T09:12:24.721181+0000","last_fullsized":"2024-04-25T09:12:24.721181+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:35.796356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656843+0000","last_change":"2024-04-25T09:12:23.367136+0000","last_active":"2024-04-25T09:12:24.656843+0000","last_peered":"2024-04-25T09:12:24.656843+0000","last_clean":"2024-04-25T09:12:24.656843+0000","last_became_active":"2024-04-25T09:12:23.366314+0000","last_became_peered":"2024-04-25T09:12:23.366314+0000","last_unstale":"2024-04-25T09:12:24.656843+0000","last_undegraded":"2024-04-25T09:12:24.656843+0000","last_fullsized":"2024-04-25T09:12:24.656843+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:50.350111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717737+0000","last_change":"2024-04-25T09:12:22.363200+0000","last_active":"2024-04-25T09:12:24.717737+0000","last_peered":"2024-04-25T09:12:24.717737+0000","last_clean":"2024-04-25T09:12:24.717737+0000","last_became_active":"2024-04-25T09:12:22.362680+0000","last_became_peered":"2024-04-25T09:12:22.362680+0000","last_unstale":"2024-04-25T09:12:24.717737+0000","last_undegraded":"2024-04-25T09:12:24.717737+0000","last_fullsized":"2024-04-25T09:12:24.717737+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:04.848006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719606+0000","last_change":"2024-04-25T09:12:23.360790+0000","last_active":"2024-04-25T09:12:24.719606+0000","last_peered":"2024-04-25T09:12:24.719606+0000","last_clean":"2024-04-25T09:12:24.719606+0000","last_became_active":"2024-04-25T09:12:23.358806+0000","last_became_peered":"2024-04-25T09:12:23.358806+0000","last_unstale":"2024-04-25T09:12:24.719606+0000","last_undegraded":"2024-04-25T09:12:24.719606+0000","last_fullsized":"2024-04-25T09:12:24.719606+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:48.820243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363422+0000","last_change":"2024-04-25T09:12:22.351891+0000","last_active":"2024-04-25T09:12:24.363422+0000","last_peered":"2024-04-25T09:12:24.363422+0000","last_clean":"2024-04-25T09:12:24.363422+0000","last_became_active":"2024-04-25T09:12:22.350910+0000","last_became_peered":"2024-04-25T09:12:22.350910+0000","last_unstale":"2024-04-25T09:12:24.363422+0000","last_undegraded":"2024-04-25T09:12:24.363422+0000","last_fullsized":"2024-04-25T09:12:24.363422+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:49.333604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656550+0000","last_change":"2024-04-25T09:12:23.360853+0000","last_active":"2024-04-25T09:12:24.656550+0000","last_peered":"2024-04-25T09:12:24.656550+0000","last_clean":"2024-04-25T09:12:24.656550+0000","last_became_active":"2024-04-25T09:12:23.360489+0000","last_became_peered":"2024-04-25T09:12:23.360489+0000","last_unstale":"2024-04-25T09:12:24.656550+0000","last_undegraded":"2024-04-25T09:12:24.656550+0000","last_fullsized":"2024-04-25T09:12:24.656550+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:31.995588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719358+0000","last_change":"2024-04-25T09:12:22.353510+0000","last_active":"2024-04-25T09:12:24.719358+0000","last_peered":"2024-04-25T09:12:24.719358+0000","last_clean":"2024-04-25T09:12:24.719358+0000","last_became_active":"2024-04-25T09:12:22.353191+0000","last_became_peered":"2024-04-25T09:12:22.353191+0000","last_unstale":"2024-04-25T09:12:24.719358+0000","last_undegraded":"2024-04-25T09:12:24.719358+0000","last_fullsized":"2024-04-25T09:12:24.719358+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:18.362066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717425+0000","last_change":"2024-04-25T09:12:23.357555+0000","last_active":"2024-04-25T09:12:24.717425+0000","last_peered":"2024-04-25T09:12:24.717425+0000","last_clean":"2024-04-25T09:12:24.717425+0000","last_became_active":"2024-04-25T09:12:23.357250+0000","last_became_peered":"2024-04-25T09:12:23.357250+0000","last_unstale":"2024-04-25T09:12:24.717425+0000","last_undegraded":"2024-04-25T09:12:24.717425+0000","last_fullsized":"2024-04-25T09:12:24.717425+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:25.657056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.366978+0000","last_change":"2024-04-25T09:12:22.346390+0000","last_active":"2024-04-25T09:12:24.366978+0000","last_peered":"2024-04-25T09:12:24.366978+0000","last_clean":"2024-04-25T09:12:24.366978+0000","last_became_active":"2024-04-25T09:12:22.341575+0000","last_became_peered":"2024-04-25T09:12:22.341575+0000","last_unstale":"2024-04-25T09:12:24.366978+0000","last_undegraded":"2024-04-25T09:12:24.366978+0000","last_fullsized":"2024-04-25T09:12:24.366978+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:48.178660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718515+0000","last_change":"2024-04-25T09:12:23.354007+0000","last_active":"2024-04-25T09:12:24.718515+0000","last_peered":"2024-04-25T09:12:24.718515+0000","last_clean":"2024-04-25T09:12:24.718515+0000","last_became_active":"2024-04-25T09:12:23.353620+0000","last_became_peered":"2024-04-25T09:12:23.353620+0000","last_unstale":"2024-04-25T09:12:24.718515+0000","last_undegraded":"2024-04-25T09:12:24.718515+0000","last_fullsized":"2024-04-25T09:12:24.718515+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:33.878768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.716809+0000","last_change":"2024-04-25T09:12:22.341391+0000","last_active":"2024-04-25T09:12:24.716809+0000","last_peered":"2024-04-25T09:12:24.716809+0000","last_clean":"2024-04-25T09:12:24.716809+0000","last_became_active":"2024-04-25T09:12:22.341052+0000","last_became_peered":"2024-04-25T09:12:22.341052+0000","last_unstale":"2024-04-25T09:12:24.716809+0000","last_undegraded":"2024-04-25T09:12:24.716809+0000","last_fullsized":"2024-04-25T09:12:24.716809+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.453617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656317+0000","last_change":"2024-04-25T09:12:23.361775+0000","last_active":"2024-04-25T09:12:24.656317+0000","last_peered":"2024-04-25T09:12:24.656317+0000","last_clean":"2024-04-25T09:12:24.656317+0000","last_became_active":"2024-04-25T09:12:23.361472+0000","last_became_peered":"2024-04-25T09:12:23.361472+0000","last_unstale":"2024-04-25T09:12:24.656317+0000","last_undegraded":"2024-04-25T09:12:24.656317+0000","last_fullsized":"2024-04-25T09:12:24.656317+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.941987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656438+0000","last_change":"2024-04-25T09:12:22.361753+0000","last_active":"2024-04-25T09:12:24.656438+0000","last_peered":"2024-04-25T09:12:24.656438+0000","last_clean":"2024-04-25T09:12:24.656438+0000","last_became_active":"2024-04-25T09:12:22.360449+0000","last_became_peered":"2024-04-25T09:12:22.360449+0000","last_unstale":"2024-04-25T09:12:24.656438+0000","last_undegraded":"2024-04-25T09:12:24.656438+0000","last_fullsized":"2024-04-25T09:12:24.656438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:00.177718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363828+0000","last_change":"2024-04-25T09:12:23.349327+0000","last_active":"2024-04-25T09:12:24.363828+0000","last_peered":"2024-04-25T09:12:24.363828+0000","last_clean":"2024-04-25T09:12:24.363828+0000","last_became_active":"2024-04-25T09:12:23.348943+0000","last_became_peered":"2024-04-25T09:12:23.348943+0000","last_unstale":"2024-04-25T09:12:24.363828+0000","last_undegraded":"2024-04-25T09:12:24.363828+0000","last_fullsized":"2024-04-25T09:12:24.363828+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:49.338135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719348+0000","last_change":"2024-04-25T09:12:22.346547+0000","last_active":"2024-04-25T09:12:24.719348+0000","last_peered":"2024-04-25T09:12:24.719348+0000","last_clean":"2024-04-25T09:12:24.719348+0000","last_became_active":"2024-04-25T09:12:22.346148+0000","last_became_peered":"2024-04-25T09:12:22.346148+0000","last_unstale":"2024-04-25T09:12:24.719348+0000","last_undegraded":"2024-04-25T09:12:24.719348+0000","last_fullsized":"2024-04-25T09:12:24.719348+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:15.858635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721014+0000","last_change":"2024-04-25T09:12:23.352679+0000","last_active":"2024-04-25T09:12:24.721014+0000","last_peered":"2024-04-25T09:12:24.721014+0000","last_clean":"2024-04-25T09:12:24.721014+0000","last_became_active":"2024-04-25T09:12:23.352350+0000","last_became_peered":"2024-04-25T09:12:23.352350+0000","last_unstale":"2024-04-25T09:12:24.721014+0000","last_undegraded":"2024-04-25T09:12:24.721014+0000","last_fullsized":"2024-04-25T09:12:24.721014+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:11.269960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720197+0000","last_change":"2024-04-25T09:12:22.357472+0000","last_active":"2024-04-25T09:12:24.720197+0000","last_peered":"2024-04-25T09:12:24.720197+0000","last_clean":"2024-04-25T09:12:24.720197+0000","last_became_active":"2024-04-25T09:12:22.356885+0000","last_became_peered":"2024-04-25T09:12:22.356885+0000","last_unstale":"2024-04-25T09:12:24.720197+0000","last_undegraded":"2024-04-25T09:12:24.720197+0000","last_fullsized":"2024-04-25T09:12:24.720197+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:35.017637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363529+0000","last_change":"2024-04-25T09:12:23.349283+0000","last_active":"2024-04-25T09:12:24.363529+0000","last_peered":"2024-04-25T09:12:24.363529+0000","last_clean":"2024-04-25T09:12:24.363529+0000","last_became_active":"2024-04-25T09:12:23.348975+0000","last_became_peered":"2024-04-25T09:12:23.348975+0000","last_unstale":"2024-04-25T09:12:24.363529+0000","last_undegraded":"2024-04-25T09:12:24.363529+0000","last_fullsized":"2024-04-25T09:12:24.363529+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:22.449045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719255+0000","last_change":"2024-04-25T09:12:22.355340+0000","last_active":"2024-04-25T09:12:24.719255+0000","last_peered":"2024-04-25T09:12:24.719255+0000","last_clean":"2024-04-25T09:12:24.719255+0000","last_became_active":"2024-04-25T09:12:22.354061+0000","last_became_peered":"2024-04-25T09:12:22.354061+0000","last_unstale":"2024-04-25T09:12:24.719255+0000","last_undegraded":"2024-04-25T09:12:24.719255+0000","last_fullsized":"2024-04-25T09:12:24.719255+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:28.059988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367117+0000","last_change":"2024-04-25T09:12:23.356118+0000","last_active":"2024-04-25T09:12:24.367117+0000","last_peered":"2024-04-25T09:12:24.367117+0000","last_clean":"2024-04-25T09:12:24.367117+0000","last_became_active":"2024-04-25T09:12:23.355822+0000","last_became_peered":"2024-04-25T09:12:23.355822+0000","last_unstale":"2024-04-25T09:12:24.367117+0000","last_undegraded":"2024-04-25T09:12:24.367117+0000","last_fullsized":"2024-04-25T09:12:24.367117+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.941987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718901+0000","last_change":"2024-04-25T09:12:22.356298+0000","last_active":"2024-04-25T09:12:24.718901+0000","last_peered":"2024-04-25T09:12:24.718901+0000","last_clean":"2024-04-25T09:12:24.718901+0000","last_became_active":"2024-04-25T09:12:22.354454+0000","last_became_peered":"2024-04-25T09:12:22.354454+0000","last_unstale":"2024-04-25T09:12:24.718901+0000","last_undegraded":"2024-04-25T09:12:24.718901+0000","last_fullsized":"2024-04-25T09:12:24.718901+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:46.993545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719677+0000","last_change":"2024-04-25T09:12:23.359084+0000","last_active":"2024-04-25T09:12:24.719677+0000","last_peered":"2024-04-25T09:12:24.719677+0000","last_clean":"2024-04-25T09:12:24.719677+0000","last_became_active":"2024-04-25T09:12:23.354574+0000","last_became_peered":"2024-04-25T09:12:23.354574+0000","last_unstale":"2024-04-25T09:12:24.719677+0000","last_undegraded":"2024-04-25T09:12:24.719677+0000","last_fullsized":"2024-04-25T09:12:24.719677+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:34.232266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367270+0000","last_change":"2024-04-25T09:12:22.353236+0000","last_active":"2024-04-25T09:12:24.367270+0000","last_peered":"2024-04-25T09:12:24.367270+0000","last_clean":"2024-04-25T09:12:24.367270+0000","last_became_active":"2024-04-25T09:12:22.352085+0000","last_became_peered":"2024-04-25T09:12:22.352085+0000","last_unstale":"2024-04-25T09:12:24.367270+0000","last_undegraded":"2024-04-25T09:12:24.367270+0000","last_fullsized":"2024-04-25T09:12:24.367270+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.436180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717085+0000","last_change":"2024-04-25T09:12:23.359685+0000","last_active":"2024-04-25T09:12:24.717085+0000","last_peered":"2024-04-25T09:12:24.717085+0000","last_clean":"2024-04-25T09:12:24.717085+0000","last_became_active":"2024-04-25T09:12:23.359371+0000","last_became_peered":"2024-04-25T09:12:23.359371+0000","last_unstale":"2024-04-25T09:12:24.717085+0000","last_undegraded":"2024-04-25T09:12:24.717085+0000","last_fullsized":"2024-04-25T09:12:24.717085+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:07.120712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656260+0000","last_change":"2024-04-25T09:12:22.358659+0000","last_active":"2024-04-25T09:12:24.656260+0000","last_peered":"2024-04-25T09:12:24.656260+0000","last_clean":"2024-04-25T09:12:24.656260+0000","last_became_active":"2024-04-25T09:12:22.358302+0000","last_became_peered":"2024-04-25T09:12:22.358302+0000","last_unstale":"2024-04-25T09:12:24.656260+0000","last_undegraded":"2024-04-25T09:12:24.656260+0000","last_fullsized":"2024-04-25T09:12:24.656260+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:24.640549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.362896+0000","last_change":"2024-04-25T09:12:23.365712+0000","last_active":"2024-04-25T09:12:24.362896+0000","last_peered":"2024-04-25T09:12:24.362896+0000","last_clean":"2024-04-25T09:12:24.362896+0000","last_became_active":"2024-04-25T09:12:23.365395+0000","last_became_peered":"2024-04-25T09:12:23.365395+0000","last_unstale":"2024-04-25T09:12:24.362896+0000","last_undegraded":"2024-04-25T09:12:24.362896+0000","last_fullsized":"2024-04-25T09:12:24.362896+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:55.028096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718830+0000","last_change":"2024-04-25T09:12:22.360555+0000","last_active":"2024-04-25T09:12:24.718830+0000","last_peered":"2024-04-25T09:12:24.718830+0000","last_clean":"2024-04-25T09:12:24.718830+0000","last_became_active":"2024-04-25T09:12:22.360072+0000","last_became_peered":"2024-04-25T09:12:22.360072+0000","last_unstale":"2024-04-25T09:12:24.718830+0000","last_undegraded":"2024-04-25T09:12:24.718830+0000","last_fullsized":"2024-04-25T09:12:24.718830+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:05.428154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720046+0000","last_change":"2024-04-25T09:12:23.365229+0000","last_active":"2024-04-25T09:12:24.720046+0000","last_peered":"2024-04-25T09:12:24.720046+0000","last_clean":"2024-04-25T09:12:24.720046+0000","last_became_active":"2024-04-25T09:12:23.364963+0000","last_became_peered":"2024-04-25T09:12:23.364963+0000","last_unstale":"2024-04-25T09:12:24.720046+0000","last_undegraded":"2024-04-25T09:12:24.720046+0000","last_fullsized":"2024-04-25T09:12:24.720046+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:15.769996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720073+0000","last_change":"2024-04-25T09:12:22.357840+0000","last_active":"2024-04-25T09:12:24.720073+0000","last_peered":"2024-04-25T09:12:24.720073+0000","last_clean":"2024-04-25T09:12:24.720073+0000","last_became_active":"2024-04-25T09:12:22.357526+0000","last_became_peered":"2024-04-25T09:12:22.357526+0000","last_unstale":"2024-04-25T09:12:24.720073+0000","last_undegraded":"2024-04-25T09:12:24.720073+0000","last_fullsized":"2024-04-25T09:12:24.720073+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:47.104616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719134+0000","last_change":"2024-04-25T09:12:23.348472+0000","last_active":"2024-04-25T09:12:24.719134+0000","last_peered":"2024-04-25T09:12:24.719134+0000","last_clean":"2024-04-25T09:12:24.719134+0000","last_became_active":"2024-04-25T09:12:23.348113+0000","last_became_peered":"2024-04-25T09:12:23.348113+0000","last_unstale":"2024-04-25T09:12:24.719134+0000","last_undegraded":"2024-04-25T09:12:24.719134+0000","last_fullsized":"2024-04-25T09:12:24.719134+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:12.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:25.344536+0000","last_change":"2024-04-25T09:12:22.361998+0000","last_active":"2024-04-25T09:12:25.344536+0000","last_peered":"2024-04-25T09:12:25.344536+0000","last_clean":"2024-04-25T09:12:25.344536+0000","last_became_active":"2024-04-25T09:12:22.360414+0000","last_became_peered":"2024-04-25T09:12:22.360414+0000","last_unstale":"2024-04-25T09:12:25.344536+0000","last_undegraded":"2024-04-25T09:12:25.344536+0000","last_fullsized":"2024-04-25T09:12:25.344536+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:14.466470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719932+0000","last_change":"2024-04-25T09:12:23.366025+0000","last_active":"2024-04-25T09:12:24.719932+0000","last_peered":"2024-04-25T09:12:24.719932+0000","last_clean":"2024-04-25T09:12:24.719932+0000","last_became_active":"2024-04-25T09:12:23.365683+0000","last_became_peered":"2024-04-25T09:12:23.365683+0000","last_unstale":"2024-04-25T09:12:24.719932+0000","last_undegraded":"2024-04-25T09:12:24.719932+0000","last_fullsized":"2024-04-25T09:12:24.719932+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:17.828493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363670+0000","last_change":"2024-04-25T09:12:22.361307+0000","last_active":"2024-04-25T09:12:24.363670+0000","last_peered":"2024-04-25T09:12:24.363670+0000","last_clean":"2024-04-25T09:12:24.363670+0000","last_became_active":"2024-04-25T09:12:22.360910+0000","last_became_peered":"2024-04-25T09:12:22.360910+0000","last_unstale":"2024-04-25T09:12:24.363670+0000","last_undegraded":"2024-04-25T09:12:24.363670+0000","last_fullsized":"2024-04-25T09:12:24.363670+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:01.951052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721383+0000","last_change":"2024-04-25T09:12:23.357366+0000","last_active":"2024-04-25T09:12:24.721383+0000","last_peered":"2024-04-25T09:12:24.721383+0000","last_clean":"2024-04-25T09:12:24.721383+0000","last_became_active":"2024-04-25T09:12:23.357096+0000","last_became_peered":"2024-04-25T09:12:23.357096+0000","last_unstale":"2024-04-25T09:12:24.721383+0000","last_undegraded":"2024-04-25T09:12:24.721383+0000","last_fullsized":"2024-04-25T09:12:24.721383+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:04.822543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718719+0000","last_change":"2024-04-25T09:12:22.345094+0000","last_active":"2024-04-25T09:12:24.718719+0000","last_peered":"2024-04-25T09:12:24.718719+0000","last_clean":"2024-04-25T09:12:24.718719+0000","last_became_active":"2024-04-25T09:12:22.344681+0000","last_became_peered":"2024-04-25T09:12:22.344681+0000","last_unstale":"2024-04-25T09:12:24.718719+0000","last_undegraded":"2024-04-25T09:12:24.718719+0000","last_fullsized":"2024-04-25T09:12:24.718719+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:56.104002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718241+0000","last_change":"2024-04-25T09:12:23.360265+0000","last_active":"2024-04-25T09:12:24.718241+0000","last_peered":"2024-04-25T09:12:24.718241+0000","last_clean":"2024-04-25T09:12:24.718241+0000","last_became_active":"2024-04-25T09:12:23.354735+0000","last_became_peered":"2024-04-25T09:12:23.354735+0000","last_unstale":"2024-04-25T09:12:24.718241+0000","last_undegraded":"2024-04-25T09:12:24.718241+0000","last_fullsized":"2024-04-25T09:12:24.718241+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:43.386330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367905+0000","last_change":"2024-04-25T09:12:22.344916+0000","last_active":"2024-04-25T09:12:24.367905+0000","last_peered":"2024-04-25T09:12:24.367905+0000","last_clean":"2024-04-25T09:12:24.367905+0000","last_became_active":"2024-04-25T09:12:22.344411+0000","last_became_peered":"2024-04-25T09:12:22.344411+0000","last_unstale":"2024-04-25T09:12:24.367905+0000","last_undegraded":"2024-04-25T09:12:24.367905+0000","last_fullsized":"2024-04-25T09:12:24.367905+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:42.369822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657004+0000","last_change":"2024-04-25T09:12:23.369912+0000","last_active":"2024-04-25T09:12:24.657004+0000","last_peered":"2024-04-25T09:12:24.657004+0000","last_clean":"2024-04-25T09:12:24.657004+0000","last_became_active":"2024-04-25T09:12:23.368162+0000","last_became_peered":"2024-04-25T09:12:23.368162+0000","last_unstale":"2024-04-25T09:12:24.657004+0000","last_undegraded":"2024-04-25T09:12:24.657004+0000","last_fullsized":"2024-04-25T09:12:24.657004+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.674300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717342+0000","last_change":"2024-04-25T09:12:22.355253+0000","last_active":"2024-04-25T09:12:24.717342+0000","last_peered":"2024-04-25T09:12:24.717342+0000","last_clean":"2024-04-25T09:12:24.717342+0000","last_became_active":"2024-04-25T09:12:22.354938+0000","last_became_peered":"2024-04-25T09:12:22.354938+0000","last_unstale":"2024-04-25T09:12:24.717342+0000","last_undegraded":"2024-04-25T09:12:24.717342+0000","last_fullsized":"2024-04-25T09:12:24.717342+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:49.333604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718149+0000","last_change":"2024-04-25T09:12:23.357648+0000","last_active":"2024-04-25T09:12:24.718149+0000","last_peered":"2024-04-25T09:12:24.718149+0000","last_clean":"2024-04-25T09:12:24.718149+0000","last_became_active":"2024-04-25T09:12:23.354035+0000","last_became_peered":"2024-04-25T09:12:23.354035+0000","last_unstale":"2024-04-25T09:12:24.718149+0000","last_undegraded":"2024-04-25T09:12:24.718149+0000","last_fullsized":"2024-04-25T09:12:24.718149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:02.513225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718386+0000","last_change":"2024-04-25T09:12:22.360133+0000","last_active":"2024-04-25T09:12:24.718386+0000","last_peered":"2024-04-25T09:12:24.718386+0000","last_clean":"2024-04-25T09:12:24.718386+0000","last_became_active":"2024-04-25T09:12:22.359774+0000","last_became_peered":"2024-04-25T09:12:22.359774+0000","last_unstale":"2024-04-25T09:12:24.718386+0000","last_undegraded":"2024-04-25T09:12:24.718386+0000","last_fullsized":"2024-04-25T09:12:24.718386+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:08.455190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718361+0000","last_change":"2024-04-25T09:12:23.364886+0000","last_active":"2024-04-25T09:12:24.718361+0000","last_peered":"2024-04-25T09:12:24.718361+0000","last_clean":"2024-04-25T09:12:24.718361+0000","last_became_active":"2024-04-25T09:12:23.364568+0000","last_became_peered":"2024-04-25T09:12:23.364568+0000","last_unstale":"2024-04-25T09:12:24.718361+0000","last_undegraded":"2024-04-25T09:12:24.718361+0000","last_fullsized":"2024-04-25T09:12:24.718361+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.195167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.362814+0000","last_change":"2024-04-25T09:12:22.347385+0000","last_active":"2024-04-25T09:12:24.362814+0000","last_peered":"2024-04-25T09:12:24.362814+0000","last_clean":"2024-04-25T09:12:24.362814+0000","last_became_active":"2024-04-25T09:12:22.346669+0000","last_became_peered":"2024-04-25T09:12:22.346669+0000","last_unstale":"2024-04-25T09:12:24.362814+0000","last_undegraded":"2024-04-25T09:12:24.362814+0000","last_fullsized":"2024-04-25T09:12:24.362814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:37.925480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657071+0000","last_change":"2024-04-25T09:12:23.368579+0000","last_active":"2024-04-25T09:12:24.657071+0000","last_peered":"2024-04-25T09:12:24.657071+0000","last_clean":"2024-04-25T09:12:24.657071+0000","last_became_active":"2024-04-25T09:12:23.365882+0000","last_became_peered":"2024-04-25T09:12:23.365882+0000","last_unstale":"2024-04-25T09:12:24.657071+0000","last_undegraded":"2024-04-25T09:12:24.657071+0000","last_fullsized":"2024-04-25T09:12:24.657071+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:57.120509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720301+0000","last_change":"2024-04-25T09:12:22.352239+0000","last_active":"2024-04-25T09:12:24.720301+0000","last_peered":"2024-04-25T09:12:24.720301+0000","last_clean":"2024-04-25T09:12:24.720301+0000","last_became_active":"2024-04-25T09:12:22.351941+0000","last_became_peered":"2024-04-25T09:12:22.351941+0000","last_unstale":"2024-04-25T09:12:24.720301+0000","last_undegraded":"2024-04-25T09:12:24.720301+0000","last_fullsized":"2024-04-25T09:12:24.720301+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:11.963038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":311,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717892+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:12:24.717892+0000","last_peered":"2024-04-25T09:12:24.717892+0000","last_clean":"2024-04-25T09:12:24.717892+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:12:24.717892+0000","last_undegraded":"2024-04-25T09:12:24.717892+0000","last_fullsized":"2024-04-25T09:12:24.717892+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46244,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325596,"statfs":{"total":96636764160,"available":96589410304,"internally_reserved":0,"allocated":1196032,"data_stored":521267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50400,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321440,"statfs":{"total":96636764160,"available":96585154560,"internally_reserved":0,"allocated":1200128,"data_stored":521869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50992,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320848,"statfs":{"total":96636764160,"available":96584548352,"internally_reserved":0,"allocated":1802240,"data_stored":1113211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54564,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317276,"statfs":{"total":96636764160,"available":96580890624,"internally_reserved":0,"allocated":1196032,"data_stored":521231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54564,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317276,"statfs":{"total":96636764160,"available":96580890624,"internally_reserved":0,"allocated":1204224,"data_stored":521871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76760,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295080,"statfs":{"total":96636764160,"available":96558161920,"internally_reserved":0,"allocated":1191936,"data_stored":521197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50976,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320864,"statfs":{"total":96636764160,"available":96584564736,"internally_reserved":0,"allocated":1789952,"data_stored":1111655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44840,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327000,"statfs":{"total":96636764160,"available":96590848000,"internally_reserved":0,"allocated":1208320,"data_stored":522831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:12:27.813 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:12:28.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.076+0000 7f3a95e00640 1 Processor -- start 2024-04-25T09:12:28.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.076+0000 7f3a95e00640 1 -- start start 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a95e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a900b6a90 0x7f3a900d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a95e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 0x7f3a900d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a95e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a900d3110 con 0x7f3a900b56f0 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a95e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a900cedc0 con 0x7f3a900b6a90 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a95e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900d33f0 con 0x7f3a900b27a0 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a8fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a900b6a90 0x7f3a900d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a8fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a900b6a90 0x7f3a900d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44944/0 (socket says 172.21.15.62:44944) 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a8fe00640 1 -- 172.21.15.62:0/365918034 learned_addr learned my addr 172.21.15.62:0/365918034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:28.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.078+0000 7f3a95400640 1 --2- 172.21.15.62:0/365918034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 0x7f3a900d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.079+0000 7f3a8fe00640 1 -- 172.21.15.62:0/365918034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 msgr2=0x7f3a900d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:28.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.079+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/365918034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 0x7f3a900d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.079+0000 7f3a8fe00640 1 -- 172.21.15.62:0/365918034 >> v1:172.21.15.121:6789/0 conn(0x7f3a900b56f0 legacy=0x7f3a90160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.079+0000 7f3a8fe00640 1 -- 172.21.15.62:0/365918034 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900d36d0 con 0x7f3a900b6a90 2024-04-25T09:12:28.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.080+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/365918034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a900d1fe0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009870 tx=0x7f3a7c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de0d5a0d050dc7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:28.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.080+0000 7f3a8f400640 1 -- 172.21.15.62:0/365918034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c037070 con 0x7f3a900b6a90 2024-04-25T09:12:28.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.080+0000 7f3a8f400640 1 -- 172.21.15.62:0/365918034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c032440 con 0x7f3a900b6a90 2024-04-25T09:12:28.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.080+0000 7f3a8f400640 1 -- 172.21.15.62:0/365918034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c032810 con 0x7f3a900b6a90 2024-04-25T09:12:28.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.081+0000 7f3a95e00640 1 -- 172.21.15.62:0/365918034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 msgr2=0x7f3a900d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.081+0000 7f3a95e00640 1 --2- 172.21.15.62:0/365918034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a900d1fe0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009870 tx=0x7f3a7c02a250 comp rx=0 tx=0).stop 2024-04-25T09:12:28.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.081+0000 7f3a95e00640 1 -- 172.21.15.62:0/365918034 shutdown_connections 2024-04-25T09:12:28.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.081+0000 7f3a95e00640 1 --2- 172.21.15.62:0/365918034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 0x7f3a900d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.081+0000 7f3a95e00640 1 --2- 172.21.15.62:0/365918034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a900d1fe0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.081+0000 7f3a95e00640 1 -- 172.21.15.62:0/365918034 >> 172.21.15.62:0/365918034 conn(0x7f3a90066d10 msgr2=0x7f3a90067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:28.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.081+0000 7f3a95e00640 1 -- 172.21.15.62:0/365918034 shutdown_connections 2024-04-25T09:12:28.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.081+0000 7f3a95e00640 1 -- 172.21.15.62:0/365918034 wait complete. 2024-04-25T09:12:28.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.083+0000 7f3a95e00640 1 Processor -- start 2024-04-25T09:12:28.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.083+0000 7f3a95e00640 1 -- start start 2024-04-25T09:12:28.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 0x7f3a900cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a900b56f0 0x7f3a9014ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a90159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900b2ee0 con 0x7f3a900b6a90 2024-04-25T09:12:28.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900b2d60 con 0x7f3a900b56f0 2024-04-25T09:12:28.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a900b5ad0 con 0x7f3a900b27a0 2024-04-25T09:12:28.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a90159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a90159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44950/0 (socket says 172.21.15.62:44950) 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.084+0000 7f3a95400640 1 -- 172.21.15.62:0/2624091451 learned_addr learned my addr 172.21.15.62:0/2624091451 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a94a00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 0x7f3a900cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a900b56f0 0x7f3a9014ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a95400640 1 -- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 msgr2=0x7f3a900cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a95400640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 0x7f3a900cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a95400640 1 -- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a900b56f0 msgr2=0x7f3a9014ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a95400640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a900b56f0 0x7f3a9014ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a95400640 1 -- 172.21.15.62:0/2624091451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900c8440 con 0x7f3a900b6a90 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a900b56f0 0x7f3a9014ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:28.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a95400640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a90159c00 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007e30 tx=0x7f3a8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a8cc00640 1 -- 172.21.15.62:0/2624091451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88014620 con 0x7f3a900b6a90 2024-04-25T09:12:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a8cc00640 1 -- 172.21.15.62:0/2624091451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88019070 con 0x7f3a900b6a90 2024-04-25T09:12:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a8cc00640 1 -- 172.21.15.62:0/2624091451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88014a00 con 0x7f3a900b6a90 2024-04-25T09:12:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.085+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a900cd8c0 con 0x7f3a900b6a90 2024-04-25T09:12:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.086+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a900cbcc0 con 0x7f3a900b6a90 2024-04-25T09:12:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.087+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9004ef20 con 0x7f3a900b6a90 2024-04-25T09:12:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.088+0000 7f3a8cc00640 1 -- 172.21.15.62:0/2624091451 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a88014ba0 con 0x7f3a900b6a90 2024-04-25T09:12:28.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.089+0000 7f3a8cc00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7006b4d0 0x7f3a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.089+0000 7f3a8cc00640 1 -- 172.21.15.62:0/2624091451 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a8808aa50 con 0x7f3a900b6a90 2024-04-25T09:12:28.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.089+0000 7f3a94a00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7006b4d0 0x7f3a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.091+0000 7f3a94a00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7006b4d0 0x7f3a7006d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3a84000f60 tx=0x7f3a84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:28.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.094+0000 7f3a8cc00640 1 -- 172.21.15.62:0/2624091451 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a900c8440 con 0x7f3a900b6a90 2024-04-25T09:12:28.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.294+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a900ce580 con 0x7f3a7006b4d0 2024-04-25T09:12:28.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.312+0000 7f3a8cc00640 1 -- 172.21.15.62:0/2624091451 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252086 (secure 0 0 0) 0x7f3a900ce580 con 0x7f3a7006b4d0 2024-04-25T09:12:28.303 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:28.305 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:12:28.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7006b4d0 msgr2=0x7f3a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7006b4d0 0x7f3a7006d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3a84000f60 tx=0x7f3a84009040 comp rx=0 tx=0).stop 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 msgr2=0x7f3a90159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a90159c00 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007e30 tx=0x7f3a8800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 shutdown_connections 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7006b4d0 0x7f3a7006d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a900b6a90 0x7f3a90159c00 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a900b56f0 0x7f3a9014ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 --2- 172.21.15.62:0/2624091451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a900b27a0 0x7f3a900cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.317+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 >> 172.21.15.62:0/2624091451 conn(0x7f3a90066d10 msgr2=0x7f3a900b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.318+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 shutdown_connections 2024-04-25T09:12:28.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.318+0000 7f3a95e00640 1 -- 172.21.15.62:0/2624091451 wait complete. 2024-04-25T09:12:28.326 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-04-25T09:12:28.096701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429340,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754545380,"statfs":{"total":773094113280,"available":772654469120,"internally_reserved":0,"allocated":10788864,"data_stored":5355132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":428723760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002186"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719887+0000","last_change":"2024-04-25T09:12:23.363561+0000","last_active":"2024-04-25T09:12:24.719887+0000","last_peered":"2024-04-25T09:12:24.719887+0000","last_clean":"2024-04-25T09:12:24.719887+0000","last_became_active":"2024-04-25T09:12:23.363068+0000","last_became_peered":"2024-04-25T09:12:23.363068+0000","last_unstale":"2024-04-25T09:12:24.719887+0000","last_undegraded":"2024-04-25T09:12:24.719887+0000","last_fullsized":"2024-04-25T09:12:24.719887+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.452687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.368156+0000","last_change":"2024-04-25T09:12:23.357428+0000","last_active":"2024-04-25T09:12:24.368156+0000","last_peered":"2024-04-25T09:12:24.368156+0000","last_clean":"2024-04-25T09:12:24.368156+0000","last_became_active":"2024-04-25T09:12:23.356681+0000","last_became_peered":"2024-04-25T09:12:23.356681+0000","last_unstale":"2024-04-25T09:12:24.368156+0000","last_undegraded":"2024-04-25T09:12:24.368156+0000","last_fullsized":"2024-04-25T09:12:24.368156+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:31.995588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720935+0000","last_change":"2024-04-25T09:12:23.352185+0000","last_active":"2024-04-25T09:12:24.720935+0000","last_peered":"2024-04-25T09:12:24.720935+0000","last_clean":"2024-04-25T09:12:24.720935+0000","last_became_active":"2024-04-25T09:12:23.351897+0000","last_became_peered":"2024-04-25T09:12:23.351897+0000","last_unstale":"2024-04-25T09:12:24.720935+0000","last_undegraded":"2024-04-25T09:12:24.720935+0000","last_fullsized":"2024-04-25T09:12:24.720935+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:49.338135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719605+0000","last_change":"2024-04-25T09:12:23.369219+0000","last_active":"2024-04-25T09:12:24.719605+0000","last_peered":"2024-04-25T09:12:24.719605+0000","last_clean":"2024-04-25T09:12:24.719605+0000","last_became_active":"2024-04-25T09:12:23.368759+0000","last_became_peered":"2024-04-25T09:12:23.368759+0000","last_unstale":"2024-04-25T09:12:24.719605+0000","last_undegraded":"2024-04-25T09:12:24.719605+0000","last_fullsized":"2024-04-25T09:12:24.719605+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:37.359994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718962+0000","last_change":"2024-04-25T09:12:23.351335+0000","last_active":"2024-04-25T09:12:24.718962+0000","last_peered":"2024-04-25T09:12:24.718962+0000","last_clean":"2024-04-25T09:12:24.718962+0000","last_became_active":"2024-04-25T09:12:23.351021+0000","last_became_peered":"2024-04-25T09:12:23.351021+0000","last_unstale":"2024-04-25T09:12:24.718962+0000","last_undegraded":"2024-04-25T09:12:24.718962+0000","last_fullsized":"2024-04-25T09:12:24.718962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:16.915019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717112+0000","last_change":"2024-04-25T09:12:23.367317+0000","last_active":"2024-04-25T09:12:24.717112+0000","last_peered":"2024-04-25T09:12:24.717112+0000","last_clean":"2024-04-25T09:12:24.717112+0000","last_became_active":"2024-04-25T09:12:23.367036+0000","last_became_peered":"2024-04-25T09:12:23.367036+0000","last_unstale":"2024-04-25T09:12:24.717112+0000","last_undegraded":"2024-04-25T09:12:24.717112+0000","last_fullsized":"2024-04-25T09:12:24.717112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:14.123371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367014+0000","last_change":"2024-04-25T09:12:23.354746+0000","last_active":"2024-04-25T09:12:24.367014+0000","last_peered":"2024-04-25T09:12:24.367014+0000","last_clean":"2024-04-25T09:12:24.367014+0000","last_became_active":"2024-04-25T09:12:23.354429+0000","last_became_peered":"2024-04-25T09:12:23.354429+0000","last_unstale":"2024-04-25T09:12:24.367014+0000","last_undegraded":"2024-04-25T09:12:24.367014+0000","last_fullsized":"2024-04-25T09:12:24.367014+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.829948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363016+0000","last_change":"2024-04-25T09:12:23.359967+0000","last_active":"2024-04-25T09:12:24.363016+0000","last_peered":"2024-04-25T09:12:24.363016+0000","last_clean":"2024-04-25T09:12:24.363016+0000","last_became_active":"2024-04-25T09:12:23.358160+0000","last_became_peered":"2024-04-25T09:12:23.358160+0000","last_unstale":"2024-04-25T09:12:24.363016+0000","last_undegraded":"2024-04-25T09:12:24.363016+0000","last_fullsized":"2024-04-25T09:12:24.363016+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:10.876375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720176+0000","last_change":"2024-04-25T09:12:23.362905+0000","last_active":"2024-04-25T09:12:24.720176+0000","last_peered":"2024-04-25T09:12:24.720176+0000","last_clean":"2024-04-25T09:12:24.720176+0000","last_became_active":"2024-04-25T09:12:23.362395+0000","last_became_peered":"2024-04-25T09:12:23.362395+0000","last_unstale":"2024-04-25T09:12:24.720176+0000","last_undegraded":"2024-04-25T09:12:24.720176+0000","last_fullsized":"2024-04-25T09:12:24.720176+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:48.967945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721062+0000","last_change":"2024-04-25T09:12:23.362156+0000","last_active":"2024-04-25T09:12:24.721062+0000","last_peered":"2024-04-25T09:12:24.721062+0000","last_clean":"2024-04-25T09:12:24.721062+0000","last_became_active":"2024-04-25T09:12:23.361860+0000","last_became_peered":"2024-04-25T09:12:23.361860+0000","last_unstale":"2024-04-25T09:12:24.721062+0000","last_undegraded":"2024-04-25T09:12:24.721062+0000","last_fullsized":"2024-04-25T09:12:24.721062+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:07.120712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657560+0000","last_change":"2024-04-25T09:12:23.357120+0000","last_active":"2024-04-25T09:12:24.657560+0000","last_peered":"2024-04-25T09:12:24.657560+0000","last_clean":"2024-04-25T09:12:24.657560+0000","last_became_active":"2024-04-25T09:12:23.356839+0000","last_became_peered":"2024-04-25T09:12:23.356839+0000","last_unstale":"2024-04-25T09:12:24.657560+0000","last_undegraded":"2024-04-25T09:12:24.657560+0000","last_fullsized":"2024-04-25T09:12:24.657560+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:54.174119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363260+0000","last_change":"2024-04-25T09:12:23.352975+0000","last_active":"2024-04-25T09:12:24.363260+0000","last_peered":"2024-04-25T09:12:24.363260+0000","last_clean":"2024-04-25T09:12:24.363260+0000","last_became_active":"2024-04-25T09:12:23.352685+0000","last_became_peered":"2024-04-25T09:12:23.352685+0000","last_unstale":"2024-04-25T09:12:24.363260+0000","last_undegraded":"2024-04-25T09:12:24.363260+0000","last_fullsized":"2024-04-25T09:12:24.363260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:33.916224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719323+0000","last_change":"2024-04-25T09:12:23.364560+0000","last_active":"2024-04-25T09:12:24.719323+0000","last_peered":"2024-04-25T09:12:24.719323+0000","last_clean":"2024-04-25T09:12:24.719323+0000","last_became_active":"2024-04-25T09:12:23.354942+0000","last_became_peered":"2024-04-25T09:12:23.354942+0000","last_unstale":"2024-04-25T09:12:24.719323+0000","last_undegraded":"2024-04-25T09:12:24.719323+0000","last_fullsized":"2024-04-25T09:12:24.719323+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:02.384046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719063+0000","last_change":"2024-04-25T09:12:23.369263+0000","last_active":"2024-04-25T09:12:24.719063+0000","last_peered":"2024-04-25T09:12:24.719063+0000","last_clean":"2024-04-25T09:12:24.719063+0000","last_became_active":"2024-04-25T09:12:23.365974+0000","last_became_peered":"2024-04-25T09:12:23.365974+0000","last_unstale":"2024-04-25T09:12:24.719063+0000","last_undegraded":"2024-04-25T09:12:24.719063+0000","last_fullsized":"2024-04-25T09:12:24.719063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:48.967945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367003+0000","last_change":"2024-04-25T09:12:23.364833+0000","last_active":"2024-04-25T09:12:24.367003+0000","last_peered":"2024-04-25T09:12:24.367003+0000","last_clean":"2024-04-25T09:12:24.367003+0000","last_became_active":"2024-04-25T09:12:23.364499+0000","last_became_peered":"2024-04-25T09:12:23.364499+0000","last_unstale":"2024-04-25T09:12:24.367003+0000","last_undegraded":"2024-04-25T09:12:24.367003+0000","last_fullsized":"2024-04-25T09:12:24.367003+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:19.409124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.716616+0000","last_change":"2024-04-25T09:12:23.355632+0000","last_active":"2024-04-25T09:12:24.716616+0000","last_peered":"2024-04-25T09:12:24.716616+0000","last_clean":"2024-04-25T09:12:24.716616+0000","last_became_active":"2024-04-25T09:12:23.355315+0000","last_became_peered":"2024-04-25T09:12:23.355315+0000","last_unstale":"2024-04-25T09:12:24.716616+0000","last_undegraded":"2024-04-25T09:12:24.716616+0000","last_fullsized":"2024-04-25T09:12:24.716616+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:33.916224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363812+0000","last_change":"2024-04-25T09:12:23.356055+0000","last_active":"2024-04-25T09:12:24.363812+0000","last_peered":"2024-04-25T09:12:24.363812+0000","last_clean":"2024-04-25T09:12:24.363812+0000","last_became_active":"2024-04-25T09:12:23.355778+0000","last_became_peered":"2024-04-25T09:12:23.355778+0000","last_unstale":"2024-04-25T09:12:24.363812+0000","last_undegraded":"2024-04-25T09:12:24.363812+0000","last_fullsized":"2024-04-25T09:12:24.363812+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:40.458845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657017+0000","last_change":"2024-04-25T09:12:23.355611+0000","last_active":"2024-04-25T09:12:24.657017+0000","last_peered":"2024-04-25T09:12:24.657017+0000","last_clean":"2024-04-25T09:12:24.657017+0000","last_became_active":"2024-04-25T09:12:23.355312+0000","last_became_peered":"2024-04-25T09:12:23.355312+0000","last_unstale":"2024-04-25T09:12:24.657017+0000","last_undegraded":"2024-04-25T09:12:24.657017+0000","last_fullsized":"2024-04-25T09:12:24.657017+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:28.470124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720063+0000","last_change":"2024-04-25T09:12:23.362736+0000","last_active":"2024-04-25T09:12:24.720063+0000","last_peered":"2024-04-25T09:12:24.720063+0000","last_clean":"2024-04-25T09:12:24.720063+0000","last_became_active":"2024-04-25T09:12:23.362472+0000","last_became_peered":"2024-04-25T09:12:23.362472+0000","last_unstale":"2024-04-25T09:12:24.720063+0000","last_undegraded":"2024-04-25T09:12:24.720063+0000","last_fullsized":"2024-04-25T09:12:24.720063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:37.790506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718818+0000","last_change":"2024-04-25T09:12:23.365680+0000","last_active":"2024-04-25T09:12:24.718818+0000","last_peered":"2024-04-25T09:12:24.718818+0000","last_clean":"2024-04-25T09:12:24.718818+0000","last_became_active":"2024-04-25T09:12:23.363779+0000","last_became_peered":"2024-04-25T09:12:23.363779+0000","last_unstale":"2024-04-25T09:12:24.718818+0000","last_undegraded":"2024-04-25T09:12:24.718818+0000","last_fullsized":"2024-04-25T09:12:24.718818+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.195167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719412+0000","last_change":"2024-04-25T09:12:23.364578+0000","last_active":"2024-04-25T09:12:24.719412+0000","last_peered":"2024-04-25T09:12:24.719412+0000","last_clean":"2024-04-25T09:12:24.719412+0000","last_became_active":"2024-04-25T09:12:23.364011+0000","last_became_peered":"2024-04-25T09:12:23.364011+0000","last_unstale":"2024-04-25T09:12:24.719412+0000","last_undegraded":"2024-04-25T09:12:24.719412+0000","last_fullsized":"2024-04-25T09:12:24.719412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:54.211322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718546+0000","last_change":"2024-04-25T09:12:23.360423+0000","last_active":"2024-04-25T09:12:24.718546+0000","last_peered":"2024-04-25T09:12:24.718546+0000","last_clean":"2024-04-25T09:12:24.718546+0000","last_became_active":"2024-04-25T09:12:23.360102+0000","last_became_peered":"2024-04-25T09:12:23.360102+0000","last_unstale":"2024-04-25T09:12:24.718546+0000","last_undegraded":"2024-04-25T09:12:24.718546+0000","last_fullsized":"2024-04-25T09:12:24.718546+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:02.513225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.362928+0000","last_change":"2024-04-25T09:12:23.352452+0000","last_active":"2024-04-25T09:12:24.362928+0000","last_peered":"2024-04-25T09:12:24.362928+0000","last_clean":"2024-04-25T09:12:24.362928+0000","last_became_active":"2024-04-25T09:12:23.352136+0000","last_became_peered":"2024-04-25T09:12:23.352136+0000","last_unstale":"2024-04-25T09:12:24.362928+0000","last_undegraded":"2024-04-25T09:12:24.362928+0000","last_fullsized":"2024-04-25T09:12:24.362928+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:26.201674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656571+0000","last_change":"2024-04-25T09:12:23.367532+0000","last_active":"2024-04-25T09:12:24.656571+0000","last_peered":"2024-04-25T09:12:24.656571+0000","last_clean":"2024-04-25T09:12:24.656571+0000","last_became_active":"2024-04-25T09:12:23.366970+0000","last_became_peered":"2024-04-25T09:12:23.366970+0000","last_unstale":"2024-04-25T09:12:24.656571+0000","last_undegraded":"2024-04-25T09:12:24.656571+0000","last_fullsized":"2024-04-25T09:12:24.656571+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:02.967559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657444+0000","last_change":"2024-04-25T09:12:23.371369+0000","last_active":"2024-04-25T09:12:24.657444+0000","last_peered":"2024-04-25T09:12:24.657444+0000","last_clean":"2024-04-25T09:12:24.657444+0000","last_became_active":"2024-04-25T09:12:23.370785+0000","last_became_peered":"2024-04-25T09:12:23.370785+0000","last_unstale":"2024-04-25T09:12:24.657444+0000","last_undegraded":"2024-04-25T09:12:24.657444+0000","last_fullsized":"2024-04-25T09:12:24.657444+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:36.812863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719683+0000","last_change":"2024-04-25T09:12:23.364411+0000","last_active":"2024-04-25T09:12:24.719683+0000","last_peered":"2024-04-25T09:12:24.719683+0000","last_clean":"2024-04-25T09:12:24.719683+0000","last_became_active":"2024-04-25T09:12:23.364141+0000","last_became_peered":"2024-04-25T09:12:23.364141+0000","last_unstale":"2024-04-25T09:12:24.719683+0000","last_undegraded":"2024-04-25T09:12:24.719683+0000","last_fullsized":"2024-04-25T09:12:24.719683+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:27.575588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720028+0000","last_change":"2024-04-25T09:12:23.370480+0000","last_active":"2024-04-25T09:12:24.720028+0000","last_peered":"2024-04-25T09:12:24.720028+0000","last_clean":"2024-04-25T09:12:24.720028+0000","last_became_active":"2024-04-25T09:12:23.369413+0000","last_became_peered":"2024-04-25T09:12:23.369413+0000","last_unstale":"2024-04-25T09:12:24.720028+0000","last_undegraded":"2024-04-25T09:12:24.720028+0000","last_fullsized":"2024-04-25T09:12:24.720028+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:59.265413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717963+0000","last_change":"2024-04-25T09:12:23.356312+0000","last_active":"2024-04-25T09:12:24.717963+0000","last_peered":"2024-04-25T09:12:24.717963+0000","last_clean":"2024-04-25T09:12:24.717963+0000","last_became_active":"2024-04-25T09:12:23.356042+0000","last_became_peered":"2024-04-25T09:12:23.356042+0000","last_unstale":"2024-04-25T09:12:24.717963+0000","last_undegraded":"2024-04-25T09:12:24.717963+0000","last_fullsized":"2024-04-25T09:12:24.717963+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:33.878768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719063+0000","last_change":"2024-04-25T09:12:23.363469+0000","last_active":"2024-04-25T09:12:24.719063+0000","last_peered":"2024-04-25T09:12:24.719063+0000","last_clean":"2024-04-25T09:12:24.719063+0000","last_became_active":"2024-04-25T09:12:23.363100+0000","last_became_peered":"2024-04-25T09:12:23.363100+0000","last_unstale":"2024-04-25T09:12:24.719063+0000","last_undegraded":"2024-04-25T09:12:24.719063+0000","last_fullsized":"2024-04-25T09:12:24.719063+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:31.630427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719016+0000","last_change":"2024-04-25T09:12:23.365167+0000","last_active":"2024-04-25T09:12:24.719016+0000","last_peered":"2024-04-25T09:12:24.719016+0000","last_clean":"2024-04-25T09:12:24.719016+0000","last_became_active":"2024-04-25T09:12:23.364592+0000","last_became_peered":"2024-04-25T09:12:23.364592+0000","last_unstale":"2024-04-25T09:12:24.719016+0000","last_undegraded":"2024-04-25T09:12:24.719016+0000","last_fullsized":"2024-04-25T09:12:24.719016+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.452687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.716857+0000","last_change":"2024-04-25T09:12:23.364754+0000","last_active":"2024-04-25T09:12:24.716857+0000","last_peered":"2024-04-25T09:12:24.716857+0000","last_clean":"2024-04-25T09:12:24.716857+0000","last_became_active":"2024-04-25T09:12:23.364467+0000","last_became_peered":"2024-04-25T09:12:23.364467+0000","last_unstale":"2024-04-25T09:12:24.716857+0000","last_undegraded":"2024-04-25T09:12:24.716857+0000","last_fullsized":"2024-04-25T09:12:24.716857+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:19.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.716951+0000","last_change":"2024-04-25T09:12:23.360379+0000","last_active":"2024-04-25T09:12:24.716951+0000","last_peered":"2024-04-25T09:12:24.716951+0000","last_clean":"2024-04-25T09:12:24.716951+0000","last_became_active":"2024-04-25T09:12:23.359709+0000","last_became_peered":"2024-04-25T09:12:23.359709+0000","last_unstale":"2024-04-25T09:12:24.716951+0000","last_undegraded":"2024-04-25T09:12:24.716951+0000","last_fullsized":"2024-04-25T09:12:24.716951+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:22.449045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363721+0000","last_change":"2024-04-25T09:12:23.350569+0000","last_active":"2024-04-25T09:12:24.363721+0000","last_peered":"2024-04-25T09:12:24.363721+0000","last_clean":"2024-04-25T09:12:24.363721+0000","last_became_active":"2024-04-25T09:12:23.350218+0000","last_became_peered":"2024-04-25T09:12:23.350218+0000","last_unstale":"2024-04-25T09:12:24.363721+0000","last_undegraded":"2024-04-25T09:12:24.363721+0000","last_fullsized":"2024-04-25T09:12:24.363721+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:14.123371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719208+0000","last_change":"2024-04-25T09:12:22.359912+0000","last_active":"2024-04-25T09:12:24.719208+0000","last_peered":"2024-04-25T09:12:24.719208+0000","last_clean":"2024-04-25T09:12:24.719208+0000","last_became_active":"2024-04-25T09:12:22.359523+0000","last_became_peered":"2024-04-25T09:12:22.359523+0000","last_unstale":"2024-04-25T09:12:24.719208+0000","last_undegraded":"2024-04-25T09:12:24.719208+0000","last_fullsized":"2024-04-25T09:12:24.719208+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:59.651888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720535+0000","last_change":"2024-04-25T09:12:23.355083+0000","last_active":"2024-04-25T09:12:24.720535+0000","last_peered":"2024-04-25T09:12:24.720535+0000","last_clean":"2024-04-25T09:12:24.720535+0000","last_became_active":"2024-04-25T09:12:23.354751+0000","last_became_peered":"2024-04-25T09:12:23.354751+0000","last_unstale":"2024-04-25T09:12:24.720535+0000","last_undegraded":"2024-04-25T09:12:24.720535+0000","last_fullsized":"2024-04-25T09:12:24.720535+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:42.666545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721236+0000","last_change":"2024-04-25T09:12:22.352115+0000","last_active":"2024-04-25T09:12:24.721236+0000","last_peered":"2024-04-25T09:12:24.721236+0000","last_clean":"2024-04-25T09:12:24.721236+0000","last_became_active":"2024-04-25T09:12:22.350540+0000","last_became_peered":"2024-04-25T09:12:22.350540+0000","last_unstale":"2024-04-25T09:12:24.721236+0000","last_undegraded":"2024-04-25T09:12:24.721236+0000","last_fullsized":"2024-04-25T09:12:24.721236+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:51.657793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363260+0000","last_change":"2024-04-25T09:12:23.366788+0000","last_active":"2024-04-25T09:12:24.363260+0000","last_peered":"2024-04-25T09:12:24.363260+0000","last_clean":"2024-04-25T09:12:24.363260+0000","last_became_active":"2024-04-25T09:12:23.365851+0000","last_became_peered":"2024-04-25T09:12:23.365851+0000","last_unstale":"2024-04-25T09:12:24.363260+0000","last_undegraded":"2024-04-25T09:12:24.363260+0000","last_fullsized":"2024-04-25T09:12:24.363260+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:19.378573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657487+0000","last_change":"2024-04-25T09:12:22.361074+0000","last_active":"2024-04-25T09:12:24.657487+0000","last_peered":"2024-04-25T09:12:24.657487+0000","last_clean":"2024-04-25T09:12:24.657487+0000","last_became_active":"2024-04-25T09:12:22.360567+0000","last_became_peered":"2024-04-25T09:12:22.360567+0000","last_unstale":"2024-04-25T09:12:24.657487+0000","last_undegraded":"2024-04-25T09:12:24.657487+0000","last_fullsized":"2024-04-25T09:12:24.657487+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:01.496718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717613+0000","last_change":"2024-04-25T09:12:23.366562+0000","last_active":"2024-04-25T09:12:24.717613+0000","last_peered":"2024-04-25T09:12:24.717613+0000","last_clean":"2024-04-25T09:12:24.717613+0000","last_became_active":"2024-04-25T09:12:23.366174+0000","last_became_peered":"2024-04-25T09:12:23.366174+0000","last_unstale":"2024-04-25T09:12:24.717613+0000","last_undegraded":"2024-04-25T09:12:24.717613+0000","last_fullsized":"2024-04-25T09:12:24.717613+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:01.194225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718667+0000","last_change":"2024-04-25T09:12:22.359789+0000","last_active":"2024-04-25T09:12:24.718667+0000","last_peered":"2024-04-25T09:12:24.718667+0000","last_clean":"2024-04-25T09:12:24.718667+0000","last_became_active":"2024-04-25T09:12:22.358566+0000","last_became_peered":"2024-04-25T09:12:22.358566+0000","last_unstale":"2024-04-25T09:12:24.718667+0000","last_undegraded":"2024-04-25T09:12:24.718667+0000","last_fullsized":"2024-04-25T09:12:24.718667+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:09.859868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717744+0000","last_change":"2024-04-25T09:12:23.362086+0000","last_active":"2024-04-25T09:12:24.717744+0000","last_peered":"2024-04-25T09:12:24.717744+0000","last_clean":"2024-04-25T09:12:24.717744+0000","last_became_active":"2024-04-25T09:12:23.359743+0000","last_became_peered":"2024-04-25T09:12:23.359743+0000","last_unstale":"2024-04-25T09:12:24.717744+0000","last_undegraded":"2024-04-25T09:12:24.717744+0000","last_fullsized":"2024-04-25T09:12:24.717744+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:04.822543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363088+0000","last_change":"2024-04-25T09:12:22.346968+0000","last_active":"2024-04-25T09:12:24.363088+0000","last_peered":"2024-04-25T09:12:24.363088+0000","last_clean":"2024-04-25T09:12:24.363088+0000","last_became_active":"2024-04-25T09:12:22.346347+0000","last_became_peered":"2024-04-25T09:12:22.346347+0000","last_unstale":"2024-04-25T09:12:24.363088+0000","last_undegraded":"2024-04-25T09:12:24.363088+0000","last_fullsized":"2024-04-25T09:12:24.363088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:30.979081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718048+0000","last_change":"2024-04-25T09:12:23.356680+0000","last_active":"2024-04-25T09:12:24.718048+0000","last_peered":"2024-04-25T09:12:24.718048+0000","last_clean":"2024-04-25T09:12:24.718048+0000","last_became_active":"2024-04-25T09:12:23.356359+0000","last_became_peered":"2024-04-25T09:12:23.356359+0000","last_unstale":"2024-04-25T09:12:24.718048+0000","last_undegraded":"2024-04-25T09:12:24.718048+0000","last_fullsized":"2024-04-25T09:12:24.718048+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:40.458845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717133+0000","last_change":"2024-04-25T09:12:22.354477+0000","last_active":"2024-04-25T09:12:24.717133+0000","last_peered":"2024-04-25T09:12:24.717133+0000","last_clean":"2024-04-25T09:12:24.717133+0000","last_became_active":"2024-04-25T09:12:22.354117+0000","last_became_peered":"2024-04-25T09:12:22.354117+0000","last_unstale":"2024-04-25T09:12:24.717133+0000","last_undegraded":"2024-04-25T09:12:24.717133+0000","last_fullsized":"2024-04-25T09:12:24.717133+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:01.951052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717754+0000","last_change":"2024-04-25T09:12:23.358579+0000","last_active":"2024-04-25T09:12:24.717754+0000","last_peered":"2024-04-25T09:12:24.717754+0000","last_clean":"2024-04-25T09:12:24.717754+0000","last_became_active":"2024-04-25T09:12:23.357627+0000","last_became_peered":"2024-04-25T09:12:23.357627+0000","last_unstale":"2024-04-25T09:12:24.717754+0000","last_undegraded":"2024-04-25T09:12:24.717754+0000","last_fullsized":"2024-04-25T09:12:24.717754+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:15.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719192+0000","last_change":"2024-04-25T09:12:22.355980+0000","last_active":"2024-04-25T09:12:24.719192+0000","last_peered":"2024-04-25T09:12:24.719192+0000","last_clean":"2024-04-25T09:12:24.719192+0000","last_became_active":"2024-04-25T09:12:22.355680+0000","last_became_peered":"2024-04-25T09:12:22.355680+0000","last_unstale":"2024-04-25T09:12:24.719192+0000","last_undegraded":"2024-04-25T09:12:24.719192+0000","last_fullsized":"2024-04-25T09:12:24.719192+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:46.792156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718685+0000","last_change":"2024-04-25T09:12:23.351336+0000","last_active":"2024-04-25T09:12:24.718685+0000","last_peered":"2024-04-25T09:12:24.718685+0000","last_clean":"2024-04-25T09:12:24.718685+0000","last_became_active":"2024-04-25T09:12:23.350504+0000","last_became_peered":"2024-04-25T09:12:23.350504+0000","last_unstale":"2024-04-25T09:12:24.718685+0000","last_undegraded":"2024-04-25T09:12:24.718685+0000","last_fullsized":"2024-04-25T09:12:24.718685+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:43.386330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718941+0000","last_change":"2024-04-25T09:12:22.354648+0000","last_active":"2024-04-25T09:12:24.718941+0000","last_peered":"2024-04-25T09:12:24.718941+0000","last_clean":"2024-04-25T09:12:24.718941+0000","last_became_active":"2024-04-25T09:12:22.354149+0000","last_became_peered":"2024-04-25T09:12:22.354149+0000","last_unstale":"2024-04-25T09:12:24.718941+0000","last_undegraded":"2024-04-25T09:12:24.718941+0000","last_fullsized":"2024-04-25T09:12:24.718941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:41.968283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720075+0000","last_change":"2024-04-25T09:12:23.351435+0000","last_active":"2024-04-25T09:12:24.720075+0000","last_peered":"2024-04-25T09:12:24.720075+0000","last_clean":"2024-04-25T09:12:24.720075+0000","last_became_active":"2024-04-25T09:12:23.349248+0000","last_became_peered":"2024-04-25T09:12:23.349248+0000","last_unstale":"2024-04-25T09:12:24.720075+0000","last_undegraded":"2024-04-25T09:12:24.720075+0000","last_fullsized":"2024-04-25T09:12:24.720075+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:42.666545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720461+0000","last_change":"2024-04-25T09:12:22.358623+0000","last_active":"2024-04-25T09:12:24.720461+0000","last_peered":"2024-04-25T09:12:24.720461+0000","last_clean":"2024-04-25T09:12:24.720461+0000","last_became_active":"2024-04-25T09:12:22.357743+0000","last_became_peered":"2024-04-25T09:12:22.357743+0000","last_unstale":"2024-04-25T09:12:24.720461+0000","last_undegraded":"2024-04-25T09:12:24.720461+0000","last_fullsized":"2024-04-25T09:12:24.720461+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:32.697694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720347+0000","last_change":"2024-04-25T09:12:23.359572+0000","last_active":"2024-04-25T09:12:24.720347+0000","last_peered":"2024-04-25T09:12:24.720347+0000","last_clean":"2024-04-25T09:12:24.720347+0000","last_became_active":"2024-04-25T09:12:23.357744+0000","last_became_peered":"2024-04-25T09:12:23.357744+0000","last_unstale":"2024-04-25T09:12:24.720347+0000","last_undegraded":"2024-04-25T09:12:24.720347+0000","last_fullsized":"2024-04-25T09:12:24.720347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:37.790506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721181+0000","last_change":"2024-04-25T09:12:22.345277+0000","last_active":"2024-04-25T09:12:24.721181+0000","last_peered":"2024-04-25T09:12:24.721181+0000","last_clean":"2024-04-25T09:12:24.721181+0000","last_became_active":"2024-04-25T09:12:22.344740+0000","last_became_peered":"2024-04-25T09:12:22.344740+0000","last_unstale":"2024-04-25T09:12:24.721181+0000","last_undegraded":"2024-04-25T09:12:24.721181+0000","last_fullsized":"2024-04-25T09:12:24.721181+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:35.796356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656843+0000","last_change":"2024-04-25T09:12:23.367136+0000","last_active":"2024-04-25T09:12:24.656843+0000","last_peered":"2024-04-25T09:12:24.656843+0000","last_clean":"2024-04-25T09:12:24.656843+0000","last_became_active":"2024-04-25T09:12:23.366314+0000","last_became_peered":"2024-04-25T09:12:23.366314+0000","last_unstale":"2024-04-25T09:12:24.656843+0000","last_undegraded":"2024-04-25T09:12:24.656843+0000","last_fullsized":"2024-04-25T09:12:24.656843+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:50.350111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717737+0000","last_change":"2024-04-25T09:12:22.363200+0000","last_active":"2024-04-25T09:12:24.717737+0000","last_peered":"2024-04-25T09:12:24.717737+0000","last_clean":"2024-04-25T09:12:24.717737+0000","last_became_active":"2024-04-25T09:12:22.362680+0000","last_became_peered":"2024-04-25T09:12:22.362680+0000","last_unstale":"2024-04-25T09:12:24.717737+0000","last_undegraded":"2024-04-25T09:12:24.717737+0000","last_fullsized":"2024-04-25T09:12:24.717737+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:04.848006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719606+0000","last_change":"2024-04-25T09:12:23.360790+0000","last_active":"2024-04-25T09:12:24.719606+0000","last_peered":"2024-04-25T09:12:24.719606+0000","last_clean":"2024-04-25T09:12:24.719606+0000","last_became_active":"2024-04-25T09:12:23.358806+0000","last_became_peered":"2024-04-25T09:12:23.358806+0000","last_unstale":"2024-04-25T09:12:24.719606+0000","last_undegraded":"2024-04-25T09:12:24.719606+0000","last_fullsized":"2024-04-25T09:12:24.719606+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:48.820243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363422+0000","last_change":"2024-04-25T09:12:22.351891+0000","last_active":"2024-04-25T09:12:24.363422+0000","last_peered":"2024-04-25T09:12:24.363422+0000","last_clean":"2024-04-25T09:12:24.363422+0000","last_became_active":"2024-04-25T09:12:22.350910+0000","last_became_peered":"2024-04-25T09:12:22.350910+0000","last_unstale":"2024-04-25T09:12:24.363422+0000","last_undegraded":"2024-04-25T09:12:24.363422+0000","last_fullsized":"2024-04-25T09:12:24.363422+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:49.333604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656550+0000","last_change":"2024-04-25T09:12:23.360853+0000","last_active":"2024-04-25T09:12:24.656550+0000","last_peered":"2024-04-25T09:12:24.656550+0000","last_clean":"2024-04-25T09:12:24.656550+0000","last_became_active":"2024-04-25T09:12:23.360489+0000","last_became_peered":"2024-04-25T09:12:23.360489+0000","last_unstale":"2024-04-25T09:12:24.656550+0000","last_undegraded":"2024-04-25T09:12:24.656550+0000","last_fullsized":"2024-04-25T09:12:24.656550+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:31.995588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719358+0000","last_change":"2024-04-25T09:12:22.353510+0000","last_active":"2024-04-25T09:12:24.719358+0000","last_peered":"2024-04-25T09:12:24.719358+0000","last_clean":"2024-04-25T09:12:24.719358+0000","last_became_active":"2024-04-25T09:12:22.353191+0000","last_became_peered":"2024-04-25T09:12:22.353191+0000","last_unstale":"2024-04-25T09:12:24.719358+0000","last_undegraded":"2024-04-25T09:12:24.719358+0000","last_fullsized":"2024-04-25T09:12:24.719358+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:18.362066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717425+0000","last_change":"2024-04-25T09:12:23.357555+0000","last_active":"2024-04-25T09:12:24.717425+0000","last_peered":"2024-04-25T09:12:24.717425+0000","last_clean":"2024-04-25T09:12:24.717425+0000","last_became_active":"2024-04-25T09:12:23.357250+0000","last_became_peered":"2024-04-25T09:12:23.357250+0000","last_unstale":"2024-04-25T09:12:24.717425+0000","last_undegraded":"2024-04-25T09:12:24.717425+0000","last_fullsized":"2024-04-25T09:12:24.717425+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:25.657056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.366978+0000","last_change":"2024-04-25T09:12:22.346390+0000","last_active":"2024-04-25T09:12:24.366978+0000","last_peered":"2024-04-25T09:12:24.366978+0000","last_clean":"2024-04-25T09:12:24.366978+0000","last_became_active":"2024-04-25T09:12:22.341575+0000","last_became_peered":"2024-04-25T09:12:22.341575+0000","last_unstale":"2024-04-25T09:12:24.366978+0000","last_undegraded":"2024-04-25T09:12:24.366978+0000","last_fullsized":"2024-04-25T09:12:24.366978+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:48.178660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718515+0000","last_change":"2024-04-25T09:12:23.354007+0000","last_active":"2024-04-25T09:12:24.718515+0000","last_peered":"2024-04-25T09:12:24.718515+0000","last_clean":"2024-04-25T09:12:24.718515+0000","last_became_active":"2024-04-25T09:12:23.353620+0000","last_became_peered":"2024-04-25T09:12:23.353620+0000","last_unstale":"2024-04-25T09:12:24.718515+0000","last_undegraded":"2024-04-25T09:12:24.718515+0000","last_fullsized":"2024-04-25T09:12:24.718515+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:33.878768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.716809+0000","last_change":"2024-04-25T09:12:22.341391+0000","last_active":"2024-04-25T09:12:24.716809+0000","last_peered":"2024-04-25T09:12:24.716809+0000","last_clean":"2024-04-25T09:12:24.716809+0000","last_became_active":"2024-04-25T09:12:22.341052+0000","last_became_peered":"2024-04-25T09:12:22.341052+0000","last_unstale":"2024-04-25T09:12:24.716809+0000","last_undegraded":"2024-04-25T09:12:24.716809+0000","last_fullsized":"2024-04-25T09:12:24.716809+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.453617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656317+0000","last_change":"2024-04-25T09:12:23.361775+0000","last_active":"2024-04-25T09:12:24.656317+0000","last_peered":"2024-04-25T09:12:24.656317+0000","last_clean":"2024-04-25T09:12:24.656317+0000","last_became_active":"2024-04-25T09:12:23.361472+0000","last_became_peered":"2024-04-25T09:12:23.361472+0000","last_unstale":"2024-04-25T09:12:24.656317+0000","last_undegraded":"2024-04-25T09:12:24.656317+0000","last_fullsized":"2024-04-25T09:12:24.656317+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.941987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656438+0000","last_change":"2024-04-25T09:12:22.361753+0000","last_active":"2024-04-25T09:12:24.656438+0000","last_peered":"2024-04-25T09:12:24.656438+0000","last_clean":"2024-04-25T09:12:24.656438+0000","last_became_active":"2024-04-25T09:12:22.360449+0000","last_became_peered":"2024-04-25T09:12:22.360449+0000","last_unstale":"2024-04-25T09:12:24.656438+0000","last_undegraded":"2024-04-25T09:12:24.656438+0000","last_fullsized":"2024-04-25T09:12:24.656438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:00.177718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363828+0000","last_change":"2024-04-25T09:12:23.349327+0000","last_active":"2024-04-25T09:12:24.363828+0000","last_peered":"2024-04-25T09:12:24.363828+0000","last_clean":"2024-04-25T09:12:24.363828+0000","last_became_active":"2024-04-25T09:12:23.348943+0000","last_became_peered":"2024-04-25T09:12:23.348943+0000","last_unstale":"2024-04-25T09:12:24.363828+0000","last_undegraded":"2024-04-25T09:12:24.363828+0000","last_fullsized":"2024-04-25T09:12:24.363828+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:49.338135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719348+0000","last_change":"2024-04-25T09:12:22.346547+0000","last_active":"2024-04-25T09:12:24.719348+0000","last_peered":"2024-04-25T09:12:24.719348+0000","last_clean":"2024-04-25T09:12:24.719348+0000","last_became_active":"2024-04-25T09:12:22.346148+0000","last_became_peered":"2024-04-25T09:12:22.346148+0000","last_unstale":"2024-04-25T09:12:24.719348+0000","last_undegraded":"2024-04-25T09:12:24.719348+0000","last_fullsized":"2024-04-25T09:12:24.719348+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:15.858635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721014+0000","last_change":"2024-04-25T09:12:23.352679+0000","last_active":"2024-04-25T09:12:24.721014+0000","last_peered":"2024-04-25T09:12:24.721014+0000","last_clean":"2024-04-25T09:12:24.721014+0000","last_became_active":"2024-04-25T09:12:23.352350+0000","last_became_peered":"2024-04-25T09:12:23.352350+0000","last_unstale":"2024-04-25T09:12:24.721014+0000","last_undegraded":"2024-04-25T09:12:24.721014+0000","last_fullsized":"2024-04-25T09:12:24.721014+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:11.269960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720197+0000","last_change":"2024-04-25T09:12:22.357472+0000","last_active":"2024-04-25T09:12:24.720197+0000","last_peered":"2024-04-25T09:12:24.720197+0000","last_clean":"2024-04-25T09:12:24.720197+0000","last_became_active":"2024-04-25T09:12:22.356885+0000","last_became_peered":"2024-04-25T09:12:22.356885+0000","last_unstale":"2024-04-25T09:12:24.720197+0000","last_undegraded":"2024-04-25T09:12:24.720197+0000","last_fullsized":"2024-04-25T09:12:24.720197+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:35.017637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363529+0000","last_change":"2024-04-25T09:12:23.349283+0000","last_active":"2024-04-25T09:12:24.363529+0000","last_peered":"2024-04-25T09:12:24.363529+0000","last_clean":"2024-04-25T09:12:24.363529+0000","last_became_active":"2024-04-25T09:12:23.348975+0000","last_became_peered":"2024-04-25T09:12:23.348975+0000","last_unstale":"2024-04-25T09:12:24.363529+0000","last_undegraded":"2024-04-25T09:12:24.363529+0000","last_fullsized":"2024-04-25T09:12:24.363529+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:22.449045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719255+0000","last_change":"2024-04-25T09:12:22.355340+0000","last_active":"2024-04-25T09:12:24.719255+0000","last_peered":"2024-04-25T09:12:24.719255+0000","last_clean":"2024-04-25T09:12:24.719255+0000","last_became_active":"2024-04-25T09:12:22.354061+0000","last_became_peered":"2024-04-25T09:12:22.354061+0000","last_unstale":"2024-04-25T09:12:24.719255+0000","last_undegraded":"2024-04-25T09:12:24.719255+0000","last_fullsized":"2024-04-25T09:12:24.719255+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:28.059988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367117+0000","last_change":"2024-04-25T09:12:23.356118+0000","last_active":"2024-04-25T09:12:24.367117+0000","last_peered":"2024-04-25T09:12:24.367117+0000","last_clean":"2024-04-25T09:12:24.367117+0000","last_became_active":"2024-04-25T09:12:23.355822+0000","last_became_peered":"2024-04-25T09:12:23.355822+0000","last_unstale":"2024-04-25T09:12:24.367117+0000","last_undegraded":"2024-04-25T09:12:24.367117+0000","last_fullsized":"2024-04-25T09:12:24.367117+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.941987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718901+0000","last_change":"2024-04-25T09:12:22.356298+0000","last_active":"2024-04-25T09:12:24.718901+0000","last_peered":"2024-04-25T09:12:24.718901+0000","last_clean":"2024-04-25T09:12:24.718901+0000","last_became_active":"2024-04-25T09:12:22.354454+0000","last_became_peered":"2024-04-25T09:12:22.354454+0000","last_unstale":"2024-04-25T09:12:24.718901+0000","last_undegraded":"2024-04-25T09:12:24.718901+0000","last_fullsized":"2024-04-25T09:12:24.718901+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:46.993545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719677+0000","last_change":"2024-04-25T09:12:23.359084+0000","last_active":"2024-04-25T09:12:24.719677+0000","last_peered":"2024-04-25T09:12:24.719677+0000","last_clean":"2024-04-25T09:12:24.719677+0000","last_became_active":"2024-04-25T09:12:23.354574+0000","last_became_peered":"2024-04-25T09:12:23.354574+0000","last_unstale":"2024-04-25T09:12:24.719677+0000","last_undegraded":"2024-04-25T09:12:24.719677+0000","last_fullsized":"2024-04-25T09:12:24.719677+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:34.232266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367270+0000","last_change":"2024-04-25T09:12:22.353236+0000","last_active":"2024-04-25T09:12:24.367270+0000","last_peered":"2024-04-25T09:12:24.367270+0000","last_clean":"2024-04-25T09:12:24.367270+0000","last_became_active":"2024-04-25T09:12:22.352085+0000","last_became_peered":"2024-04-25T09:12:22.352085+0000","last_unstale":"2024-04-25T09:12:24.367270+0000","last_undegraded":"2024-04-25T09:12:24.367270+0000","last_fullsized":"2024-04-25T09:12:24.367270+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.436180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717085+0000","last_change":"2024-04-25T09:12:23.359685+0000","last_active":"2024-04-25T09:12:24.717085+0000","last_peered":"2024-04-25T09:12:24.717085+0000","last_clean":"2024-04-25T09:12:24.717085+0000","last_became_active":"2024-04-25T09:12:23.359371+0000","last_became_peered":"2024-04-25T09:12:23.359371+0000","last_unstale":"2024-04-25T09:12:24.717085+0000","last_undegraded":"2024-04-25T09:12:24.717085+0000","last_fullsized":"2024-04-25T09:12:24.717085+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:07.120712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.656260+0000","last_change":"2024-04-25T09:12:22.358659+0000","last_active":"2024-04-25T09:12:24.656260+0000","last_peered":"2024-04-25T09:12:24.656260+0000","last_clean":"2024-04-25T09:12:24.656260+0000","last_became_active":"2024-04-25T09:12:22.358302+0000","last_became_peered":"2024-04-25T09:12:22.358302+0000","last_unstale":"2024-04-25T09:12:24.656260+0000","last_undegraded":"2024-04-25T09:12:24.656260+0000","last_fullsized":"2024-04-25T09:12:24.656260+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:24.640549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.362896+0000","last_change":"2024-04-25T09:12:23.365712+0000","last_active":"2024-04-25T09:12:24.362896+0000","last_peered":"2024-04-25T09:12:24.362896+0000","last_clean":"2024-04-25T09:12:24.362896+0000","last_became_active":"2024-04-25T09:12:23.365395+0000","last_became_peered":"2024-04-25T09:12:23.365395+0000","last_unstale":"2024-04-25T09:12:24.362896+0000","last_undegraded":"2024-04-25T09:12:24.362896+0000","last_fullsized":"2024-04-25T09:12:24.362896+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:55.028096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718830+0000","last_change":"2024-04-25T09:12:22.360555+0000","last_active":"2024-04-25T09:12:24.718830+0000","last_peered":"2024-04-25T09:12:24.718830+0000","last_clean":"2024-04-25T09:12:24.718830+0000","last_became_active":"2024-04-25T09:12:22.360072+0000","last_became_peered":"2024-04-25T09:12:22.360072+0000","last_unstale":"2024-04-25T09:12:24.718830+0000","last_undegraded":"2024-04-25T09:12:24.718830+0000","last_fullsized":"2024-04-25T09:12:24.718830+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:05.428154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720046+0000","last_change":"2024-04-25T09:12:23.365229+0000","last_active":"2024-04-25T09:12:24.720046+0000","last_peered":"2024-04-25T09:12:24.720046+0000","last_clean":"2024-04-25T09:12:24.720046+0000","last_became_active":"2024-04-25T09:12:23.364963+0000","last_became_peered":"2024-04-25T09:12:23.364963+0000","last_unstale":"2024-04-25T09:12:24.720046+0000","last_undegraded":"2024-04-25T09:12:24.720046+0000","last_fullsized":"2024-04-25T09:12:24.720046+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:15.769996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720073+0000","last_change":"2024-04-25T09:12:22.357840+0000","last_active":"2024-04-25T09:12:24.720073+0000","last_peered":"2024-04-25T09:12:24.720073+0000","last_clean":"2024-04-25T09:12:24.720073+0000","last_became_active":"2024-04-25T09:12:22.357526+0000","last_became_peered":"2024-04-25T09:12:22.357526+0000","last_unstale":"2024-04-25T09:12:24.720073+0000","last_undegraded":"2024-04-25T09:12:24.720073+0000","last_fullsized":"2024-04-25T09:12:24.720073+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:47.104616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719134+0000","last_change":"2024-04-25T09:12:23.348472+0000","last_active":"2024-04-25T09:12:24.719134+0000","last_peered":"2024-04-25T09:12:24.719134+0000","last_clean":"2024-04-25T09:12:24.719134+0000","last_became_active":"2024-04-25T09:12:23.348113+0000","last_became_peered":"2024-04-25T09:12:23.348113+0000","last_unstale":"2024-04-25T09:12:24.719134+0000","last_undegraded":"2024-04-25T09:12:24.719134+0000","last_fullsized":"2024-04-25T09:12:24.719134+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:12.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:25.344536+0000","last_change":"2024-04-25T09:12:22.361998+0000","last_active":"2024-04-25T09:12:25.344536+0000","last_peered":"2024-04-25T09:12:25.344536+0000","last_clean":"2024-04-25T09:12:25.344536+0000","last_became_active":"2024-04-25T09:12:22.360414+0000","last_became_peered":"2024-04-25T09:12:22.360414+0000","last_unstale":"2024-04-25T09:12:25.344536+0000","last_undegraded":"2024-04-25T09:12:25.344536+0000","last_fullsized":"2024-04-25T09:12:25.344536+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:14.466470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.719932+0000","last_change":"2024-04-25T09:12:23.366025+0000","last_active":"2024-04-25T09:12:24.719932+0000","last_peered":"2024-04-25T09:12:24.719932+0000","last_clean":"2024-04-25T09:12:24.719932+0000","last_became_active":"2024-04-25T09:12:23.365683+0000","last_became_peered":"2024-04-25T09:12:23.365683+0000","last_unstale":"2024-04-25T09:12:24.719932+0000","last_undegraded":"2024-04-25T09:12:24.719932+0000","last_fullsized":"2024-04-25T09:12:24.719932+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:17.828493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.363670+0000","last_change":"2024-04-25T09:12:22.361307+0000","last_active":"2024-04-25T09:12:24.363670+0000","last_peered":"2024-04-25T09:12:24.363670+0000","last_clean":"2024-04-25T09:12:24.363670+0000","last_became_active":"2024-04-25T09:12:22.360910+0000","last_became_peered":"2024-04-25T09:12:22.360910+0000","last_unstale":"2024-04-25T09:12:24.363670+0000","last_undegraded":"2024-04-25T09:12:24.363670+0000","last_fullsized":"2024-04-25T09:12:24.363670+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:01.951052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.721383+0000","last_change":"2024-04-25T09:12:23.357366+0000","last_active":"2024-04-25T09:12:24.721383+0000","last_peered":"2024-04-25T09:12:24.721383+0000","last_clean":"2024-04-25T09:12:24.721383+0000","last_became_active":"2024-04-25T09:12:23.357096+0000","last_became_peered":"2024-04-25T09:12:23.357096+0000","last_unstale":"2024-04-25T09:12:24.721383+0000","last_undegraded":"2024-04-25T09:12:24.721383+0000","last_fullsized":"2024-04-25T09:12:24.721383+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:04.822543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718719+0000","last_change":"2024-04-25T09:12:22.345094+0000","last_active":"2024-04-25T09:12:24.718719+0000","last_peered":"2024-04-25T09:12:24.718719+0000","last_clean":"2024-04-25T09:12:24.718719+0000","last_became_active":"2024-04-25T09:12:22.344681+0000","last_became_peered":"2024-04-25T09:12:22.344681+0000","last_unstale":"2024-04-25T09:12:24.718719+0000","last_undegraded":"2024-04-25T09:12:24.718719+0000","last_fullsized":"2024-04-25T09:12:24.718719+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:56.104002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718241+0000","last_change":"2024-04-25T09:12:23.360265+0000","last_active":"2024-04-25T09:12:24.718241+0000","last_peered":"2024-04-25T09:12:24.718241+0000","last_clean":"2024-04-25T09:12:24.718241+0000","last_became_active":"2024-04-25T09:12:23.354735+0000","last_became_peered":"2024-04-25T09:12:23.354735+0000","last_unstale":"2024-04-25T09:12:24.718241+0000","last_undegraded":"2024-04-25T09:12:24.718241+0000","last_fullsized":"2024-04-25T09:12:24.718241+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:43.386330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.367905+0000","last_change":"2024-04-25T09:12:22.344916+0000","last_active":"2024-04-25T09:12:24.367905+0000","last_peered":"2024-04-25T09:12:24.367905+0000","last_clean":"2024-04-25T09:12:24.367905+0000","last_became_active":"2024-04-25T09:12:22.344411+0000","last_became_peered":"2024-04-25T09:12:22.344411+0000","last_unstale":"2024-04-25T09:12:24.367905+0000","last_undegraded":"2024-04-25T09:12:24.367905+0000","last_fullsized":"2024-04-25T09:12:24.367905+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:42.369822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657004+0000","last_change":"2024-04-25T09:12:23.369912+0000","last_active":"2024-04-25T09:12:24.657004+0000","last_peered":"2024-04-25T09:12:24.657004+0000","last_clean":"2024-04-25T09:12:24.657004+0000","last_became_active":"2024-04-25T09:12:23.368162+0000","last_became_peered":"2024-04-25T09:12:23.368162+0000","last_unstale":"2024-04-25T09:12:24.657004+0000","last_undegraded":"2024-04-25T09:12:24.657004+0000","last_fullsized":"2024-04-25T09:12:24.657004+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.674300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717342+0000","last_change":"2024-04-25T09:12:22.355253+0000","last_active":"2024-04-25T09:12:24.717342+0000","last_peered":"2024-04-25T09:12:24.717342+0000","last_clean":"2024-04-25T09:12:24.717342+0000","last_became_active":"2024-04-25T09:12:22.354938+0000","last_became_peered":"2024-04-25T09:12:22.354938+0000","last_unstale":"2024-04-25T09:12:24.717342+0000","last_undegraded":"2024-04-25T09:12:24.717342+0000","last_fullsized":"2024-04-25T09:12:24.717342+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:49.333604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718149+0000","last_change":"2024-04-25T09:12:23.357648+0000","last_active":"2024-04-25T09:12:24.718149+0000","last_peered":"2024-04-25T09:12:24.718149+0000","last_clean":"2024-04-25T09:12:24.718149+0000","last_became_active":"2024-04-25T09:12:23.354035+0000","last_became_peered":"2024-04-25T09:12:23.354035+0000","last_unstale":"2024-04-25T09:12:24.718149+0000","last_undegraded":"2024-04-25T09:12:24.718149+0000","last_fullsized":"2024-04-25T09:12:24.718149+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:02.513225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718386+0000","last_change":"2024-04-25T09:12:22.360133+0000","last_active":"2024-04-25T09:12:24.718386+0000","last_peered":"2024-04-25T09:12:24.718386+0000","last_clean":"2024-04-25T09:12:24.718386+0000","last_became_active":"2024-04-25T09:12:22.359774+0000","last_became_peered":"2024-04-25T09:12:22.359774+0000","last_unstale":"2024-04-25T09:12:24.718386+0000","last_undegraded":"2024-04-25T09:12:24.718386+0000","last_fullsized":"2024-04-25T09:12:24.718386+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:08.455190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.718361+0000","last_change":"2024-04-25T09:12:23.364886+0000","last_active":"2024-04-25T09:12:24.718361+0000","last_peered":"2024-04-25T09:12:24.718361+0000","last_clean":"2024-04-25T09:12:24.718361+0000","last_became_active":"2024-04-25T09:12:23.364568+0000","last_became_peered":"2024-04-25T09:12:23.364568+0000","last_unstale":"2024-04-25T09:12:24.718361+0000","last_undegraded":"2024-04-25T09:12:24.718361+0000","last_fullsized":"2024-04-25T09:12:24.718361+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:49.195167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.362814+0000","last_change":"2024-04-25T09:12:22.347385+0000","last_active":"2024-04-25T09:12:24.362814+0000","last_peered":"2024-04-25T09:12:24.362814+0000","last_clean":"2024-04-25T09:12:24.362814+0000","last_became_active":"2024-04-25T09:12:22.346669+0000","last_became_peered":"2024-04-25T09:12:22.346669+0000","last_unstale":"2024-04-25T09:12:24.362814+0000","last_undegraded":"2024-04-25T09:12:24.362814+0000","last_fullsized":"2024-04-25T09:12:24.362814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:37.925480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.657071+0000","last_change":"2024-04-25T09:12:23.368579+0000","last_active":"2024-04-25T09:12:24.657071+0000","last_peered":"2024-04-25T09:12:24.657071+0000","last_clean":"2024-04-25T09:12:24.657071+0000","last_became_active":"2024-04-25T09:12:23.365882+0000","last_became_peered":"2024-04-25T09:12:23.365882+0000","last_unstale":"2024-04-25T09:12:24.657071+0000","last_undegraded":"2024-04-25T09:12:24.657071+0000","last_fullsized":"2024-04-25T09:12:24.657071+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:22.306078+0000","last_clean_scrub_stamp":"2024-04-25T09:12:22.306078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:57.120509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.720301+0000","last_change":"2024-04-25T09:12:22.352239+0000","last_active":"2024-04-25T09:12:24.720301+0000","last_peered":"2024-04-25T09:12:24.720301+0000","last_clean":"2024-04-25T09:12:24.720301+0000","last_became_active":"2024-04-25T09:12:22.351941+0000","last_became_peered":"2024-04-25T09:12:22.351941+0000","last_unstale":"2024-04-25T09:12:24.720301+0000","last_undegraded":"2024-04-25T09:12:24.720301+0000","last_fullsized":"2024-04-25T09:12:24.720301+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:12:21.289571+0000","last_clean_scrub_stamp":"2024-04-25T09:12:21.289571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:11.963038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":311,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T09:12:24.717892+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:12:24.717892+0000","last_peered":"2024-04-25T09:12:24.717892+0000","last_clean":"2024-04-25T09:12:24.717892+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:12:24.717892+0000","last_undegraded":"2024-04-25T09:12:24.717892+0000","last_fullsized":"2024-04-25T09:12:24.717892+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46244,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325596,"statfs":{"total":96636764160,"available":96589410304,"internally_reserved":0,"allocated":1196032,"data_stored":521267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50400,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321440,"statfs":{"total":96636764160,"available":96585154560,"internally_reserved":0,"allocated":1200128,"data_stored":521869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50992,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320848,"statfs":{"total":96636764160,"available":96584548352,"internally_reserved":0,"allocated":1802240,"data_stored":1113211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54564,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317276,"statfs":{"total":96636764160,"available":96580890624,"internally_reserved":0,"allocated":1196032,"data_stored":521231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54564,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317276,"statfs":{"total":96636764160,"available":96580890624,"internally_reserved":0,"allocated":1204224,"data_stored":521871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76760,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295080,"statfs":{"total":96636764160,"available":96558161920,"internally_reserved":0,"allocated":1191936,"data_stored":521197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50976,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320864,"statfs":{"total":96636764160,"available":96584564736,"internally_reserved":0,"allocated":1789952,"data_stored":1111655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44840,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327000,"statfs":{"total":96636764160,"available":96590848000,"internally_reserved":0,"allocated":1208320,"data_stored":522831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:12:28.329 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:12:28.329 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:12:28.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.606+0000 7f535b200640 1 Processor -- start 2024-04-25T09:12:28.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.607+0000 7f535b200640 1 -- start start 2024-04-25T09:12:28.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.608+0000 7f535b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f535415dd20 0x7f53540d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.608+0000 7f535b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f53540ceef0 0x7f53540b94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.608+0000 7f535b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535415e100 con 0x7f53540d34f0 2024-04-25T09:12:28.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.608+0000 7f535b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540b9a20 con 0x7f535415dd20 2024-04-25T09:12:28.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.608+0000 7f535b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540b7b50 con 0x7f53540ceef0 2024-04-25T09:12:28.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f5359e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f535415dd20 0x7f53540d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f5359e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f535415dd20 0x7f53540d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44962/0 (socket says 172.21.15.62:44962) 2024-04-25T09:12:28.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f5359e00640 1 -- 172.21.15.62:0/152726846 learned_addr learned my addr 172.21.15.62:0/152726846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:28.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f5359400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f53540d34f0 0x7f53540ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45640/0 (socket says 172.21.15.62:45640) 2024-04-25T09:12:28.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f535a800640 1 --2- 172.21.15.62:0/152726846 >> v2:172.21.15.121:3300/0 conn(0x7f53540ceef0 0x7f53540b94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f5359e00640 1 -- 172.21.15.62:0/152726846 >> v2:172.21.15.121:3300/0 conn(0x7f53540ceef0 msgr2=0x7f53540b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:28.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f5359e00640 1 --2- 172.21.15.62:0/152726846 >> v2:172.21.15.121:3300/0 conn(0x7f53540ceef0 0x7f53540b94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f5359e00640 1 -- 172.21.15.62:0/152726846 >> v1:172.21.15.62:6789/0 conn(0x7f53540d34f0 legacy=0x7f53540ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.609+0000 7f5359e00640 1 -- 172.21.15.62:0/152726846 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540d2450 con 0x7f535415dd20 2024-04-25T09:12:28.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.610+0000 7f5359e00640 1 --2- 172.21.15.62:0/152726846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540d1d70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f53540d1510 tx=0x7f534802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81873323a07f4b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:28.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f5358a00640 1 -- 172.21.15.62:0/152726846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348037070 con 0x7f535415dd20 2024-04-25T09:12:28.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f5358a00640 1 -- 172.21.15.62:0/152726846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5348032440 con 0x7f535415dd20 2024-04-25T09:12:28.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f5358a00640 1 -- 172.21.15.62:0/152726846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348032810 con 0x7f535415dd20 2024-04-25T09:12:28.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f535b200640 1 -- 172.21.15.62:0/152726846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 msgr2=0x7f53540d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f535b200640 1 --2- 172.21.15.62:0/152726846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540d1d70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f53540d1510 tx=0x7f534802a270 comp rx=0 tx=0).stop 2024-04-25T09:12:28.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f535b200640 1 -- 172.21.15.62:0/152726846 shutdown_connections 2024-04-25T09:12:28.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f535b200640 1 --2- 172.21.15.62:0/152726846 >> v2:172.21.15.121:3300/0 conn(0x7f53540ceef0 0x7f53540b94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f535b200640 1 --2- 172.21.15.62:0/152726846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540d1d70 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.611+0000 7f535b200640 1 -- 172.21.15.62:0/152726846 >> 172.21.15.62:0/152726846 conn(0x7f535406b000 msgr2=0x7f535406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:28.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.612+0000 7f535b200640 1 -- 172.21.15.62:0/152726846 shutdown_connections 2024-04-25T09:12:28.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.612+0000 7f535b200640 1 -- 172.21.15.62:0/152726846 wait complete. 2024-04-25T09:12:28.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.613+0000 7f535b200640 1 Processor -- start 2024-04-25T09:12:28.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.613+0000 7f535b200640 1 -- start start 2024-04-25T09:12:28.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.614+0000 7f535b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540ceef0 0x7f53540c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.614+0000 7f535b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53540d34f0 0x7f535414aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.614+0000 7f535b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.614+0000 7f535b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53540d2cf0 con 0x7f535415dd20 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.614+0000 7f535b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53540d2a10 con 0x7f53540d34f0 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.614+0000 7f535b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53540b6b40 con 0x7f53540ceef0 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f535a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f535a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44968/0 (socket says 172.21.15.62:44968) 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f535a800640 1 -- 172.21.15.62:0/1936245588 learned_addr learned my addr 172.21.15.62:0/1936245588 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f5359e00640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540ceef0 0x7f53540c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f5359400640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53540d34f0 0x7f535414aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f535a800640 1 -- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540ceef0 msgr2=0x7f53540c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f535a800640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540ceef0 0x7f53540c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f535a800640 1 -- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53540d34f0 msgr2=0x7f535414aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f535a800640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53540d34f0 0x7f535414aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.615+0000 7f535a800640 1 -- 172.21.15.62:0/1936245588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540c76c0 con 0x7f535415dd20 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.616+0000 7f535a800640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540caae0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5350007e70 tx=0x7f535000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.616+0000 7f534e000640 1 -- 172.21.15.62:0/1936245588 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350014620 con 0x7f535415dd20 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.616+0000 7f534e000640 1 -- 172.21.15.62:0/1936245588 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350019070 con 0x7f535415dd20 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.616+0000 7f534e000640 1 -- 172.21.15.62:0/1936245588 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350014a20 con 0x7f535415dd20 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.616+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53540c7960 con 0x7f535415dd20 2024-04-25T09:12:28.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.616+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53540c7f20 con 0x7f535415dd20 2024-04-25T09:12:28.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.617+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5354075070 con 0x7f535415dd20 2024-04-25T09:12:28.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.618+0000 7f534e000640 1 -- 172.21.15.62:0/1936245588 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5350014bc0 con 0x7f535415dd20 2024-04-25T09:12:28.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.619+0000 7f534e000640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f533806b500 0x7f533806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:28.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.619+0000 7f534e000640 1 -- 172.21.15.62:0/1936245588 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f535008ab50 con 0x7f535415dd20 2024-04-25T09:12:28.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.620+0000 7f5359e00640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f533806b500 0x7f533806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:28.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.621+0000 7f5359e00640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f533806b500 0x7f533806d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5348034000 tx=0x7f534802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:28.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.623+0000 7f534e000640 1 -- 172.21.15.62:0/1936245588 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5350059930 con 0x7f535415dd20 2024-04-25T09:12:28.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.884+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f53540c6c80 con 0x7f535415dd20 2024-04-25T09:12:28.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.890+0000 7f534e000640 1 -- 172.21.15.62:0/1936245588 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5350059230 con 0x7f535415dd20 2024-04-25T09:12:28.878 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:12:28.878 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:12:28.878 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:12:28.878 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:12:28.878 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:12:28.879 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:12:28.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f533806b500 msgr2=0x7f533806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f533806b500 0x7f533806d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5348034000 tx=0x7f534802e000 comp rx=0 tx=0).stop 2024-04-25T09:12:28.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 msgr2=0x7f53540caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540caae0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5350007e70 tx=0x7f535000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 shutdown_connections 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f533806b500 0x7f533806d9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f535415dd20 0x7f53540caae0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53540d34f0 0x7f535414aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 --2- 172.21.15.62:0/1936245588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f53540ceef0 0x7f53540c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.892+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 >> 172.21.15.62:0/1936245588 conn(0x7f535406b000 msgr2=0x7f53540cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.893+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 shutdown_connections 2024-04-25T09:12:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:28.893+0000 7f535b200640 1 -- 172.21.15.62:0/1936245588 wait complete. 2024-04-25T09:12:28.898 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:12:29.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.110+0000 7f0e5d400640 1 Processor -- start 2024-04-25T09:12:29.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.110+0000 7f0e5d400640 1 -- start start 2024-04-25T09:12:29.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.111+0000 7f0e5d400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0e580b56f0 0x7f0e58160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:29.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.111+0000 7f0e5d400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0e580b6a90 0x7f0e58161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:29.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.111+0000 7f0e5d400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e580d3110 con 0x7f0e580b27a0 2024-04-25T09:12:29.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.111+0000 7f0e5d400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e58161780 con 0x7f0e580b56f0 2024-04-25T09:12:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.111+0000 7f0e5d400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e580cedc0 con 0x7f0e580b6a90 2024-04-25T09:12:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e57e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0e580b56f0 0x7f0e58160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e57e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0e580b56f0 0x7f0e58160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44972/0 (socket says 172.21.15.62:44972) 2024-04-25T09:12:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e57e00640 1 -- 172.21.15.62:0/29134088 learned_addr learned my addr 172.21.15.62:0/29134088 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:29.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e5ca00640 1 --1- 172.21.15.62:0/29134088 >> v1:172.21.15.62:6789/0 conn(0x7f0e580b27a0 0x7f0e58161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45648/0 (socket says 172.21.15.62:45648) 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e57e00640 1 -- 172.21.15.62:0/29134088 >> v2:172.21.15.121:3300/0 conn(0x7f0e580b6a90 msgr2=0x7f0e58161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e57e00640 1 --2- 172.21.15.62:0/29134088 >> v2:172.21.15.121:3300/0 conn(0x7f0e580b6a90 0x7f0e58161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e57e00640 1 -- 172.21.15.62:0/29134088 >> v1:172.21.15.62:6789/0 conn(0x7f0e580b27a0 legacy=0x7f0e58161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e57e00640 1 -- 172.21.15.62:0/29134088 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580d33f0 con 0x7f0e580b56f0 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.112+0000 7f0e56a00640 1 -- 172.21.15.62:0/29134088 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857124841 0 0) 0x7f0e580d3110 con 0x7f0e580b27a0 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.113+0000 7f0e57e00640 1 --2- 172.21.15.62:0/29134088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e58160d00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0e5808c570 tx=0x7f0e4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baceaaf4413b54fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e56a00640 1 -- 172.21.15.62:0/29134088 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48037070 con 0x7f0e580b56f0 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e56a00640 1 -- 172.21.15.62:0/29134088 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48032440 con 0x7f0e580b56f0 2024-04-25T09:12:29.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e56a00640 1 -- 172.21.15.62:0/29134088 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48032810 con 0x7f0e580b56f0 2024-04-25T09:12:29.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e5d400640 1 -- 172.21.15.62:0/29134088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 msgr2=0x7f0e58160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:29.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e5d400640 1 --2- 172.21.15.62:0/29134088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e58160d00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0e5808c570 tx=0x7f0e4802a270 comp rx=0 tx=0).stop 2024-04-25T09:12:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e5d400640 1 -- 172.21.15.62:0/29134088 shutdown_connections 2024-04-25T09:12:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e5d400640 1 --2- 172.21.15.62:0/29134088 >> v2:172.21.15.121:3300/0 conn(0x7f0e580b6a90 0x7f0e58161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e5d400640 1 --2- 172.21.15.62:0/29134088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e58160d00 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e5d400640 1 -- 172.21.15.62:0/29134088 >> 172.21.15.62:0/29134088 conn(0x7f0e58066d10 msgr2=0x7f0e58067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e5d400640 1 -- 172.21.15.62:0/29134088 shutdown_connections 2024-04-25T09:12:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.114+0000 7f0e5d400640 1 -- 172.21.15.62:0/29134088 wait complete. 2024-04-25T09:12:29.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.116+0000 7f0e5d400640 1 Processor -- start 2024-04-25T09:12:29.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.116+0000 7f0e5d400640 1 -- start start 2024-04-25T09:12:29.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.117+0000 7f0e5d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e580b27a0 0x7f0e58155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:29.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.117+0000 7f0e5d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e5815a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:29.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.117+0000 7f0e5d400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e580b6a90 0x7f0e580c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.117+0000 7f0e5d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b2d60 con 0x7f0e580b56f0 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.117+0000 7f0e5d400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580cea50 con 0x7f0e580b27a0 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.117+0000 7f0e5d400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580b2ee0 con 0x7f0e580b6a90 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e5815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e5815a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44976/0 (socket says 172.21.15.62:44976) 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57400640 1 -- 172.21.15.62:0/2993689348 learned_addr learned my addr 172.21.15.62:0/2993689348 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57e00640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e580b27a0 0x7f0e58155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e5ca00640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e580b6a90 0x7f0e580c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57400640 1 -- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e580b6a90 msgr2=0x7f0e580c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e580b6a90 0x7f0e580c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57400640 1 -- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e580b27a0 msgr2=0x7f0e58155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e580b27a0 0x7f0e58155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57400640 1 -- 172.21.15.62:0/2993689348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5815cfa0 con 0x7f0e580b56f0 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.118+0000 7f0e57e00640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e580b27a0 0x7f0e58155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:29.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.119+0000 7f0e57400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e5815a440 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000b410 tx=0x7f0e4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.119+0000 7f0e4fe00640 1 -- 172.21.15.62:0/2993689348 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40015020 con 0x7f0e580b56f0 2024-04-25T09:12:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.119+0000 7f0e4fe00640 1 -- 172.21.15.62:0/2993689348 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e40011850 con 0x7f0e580b56f0 2024-04-25T09:12:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.119+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5815d280 con 0x7f0e580b56f0 2024-04-25T09:12:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.119+0000 7f0e4fe00640 1 -- 172.21.15.62:0/2993689348 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40011c40 con 0x7f0e580b56f0 2024-04-25T09:12:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.119+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5815cc00 con 0x7f0e580b56f0 2024-04-25T09:12:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.120+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5804ef20 con 0x7f0e580b56f0 2024-04-25T09:12:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.121+0000 7f0e4fe00640 1 -- 172.21.15.62:0/2993689348 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e40018020 con 0x7f0e580b56f0 2024-04-25T09:12:29.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.122+0000 7f0e4fe00640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e3406b4d0 0x7f0e3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:29.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.122+0000 7f0e4fe00640 1 -- 172.21.15.62:0/2993689348 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e4008a0f0 con 0x7f0e580b56f0 2024-04-25T09:12:29.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.123+0000 7f0e57e00640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e3406b4d0 0x7f0e3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:29.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.124+0000 7f0e57e00640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e3406b4d0 0x7f0e3406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0e48034000 tx=0x7f0e4802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:29.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.126+0000 7f0e4fe00640 1 -- 172.21.15.62:0/2993689348 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e4001f020 con 0x7f0e580b56f0 2024-04-25T09:12:29.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.365+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e5815fb10 con 0x7f0e580b56f0 2024-04-25T09:12:29.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.367+0000 7f0e4fe00640 1 -- 172.21.15.62:0/2993689348 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0e400172c0 con 0x7f0e580b56f0 2024-04-25T09:12:29.355 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:12:29.355 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1798148826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1798148826},{"type":"v1","addr":"172.21.15.121:6835","nonce":1798148826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2045746846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2045746846},{"type":"v1","addr":"172.21.15.121:6837","nonce":2045746846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1182284820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1182284820},{"type":"v1","addr":"172.21.15.62:6834","nonce":1182284820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:22.810299+0000","modified":"2024-04-25T09:12:25.868942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3997398243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3997398243},{"type":"v1","addr":"172.21.15.62:6835","nonce":3997398243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:12:29.355 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 64 2024-04-25T09:12:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e3406b4d0 msgr2=0x7f0e3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:29.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e3406b4d0 0x7f0e3406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0e48034000 tx=0x7f0e4802e000 comp rx=0 tx=0).stop 2024-04-25T09:12:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 msgr2=0x7f0e5815a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e5815a440 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000b410 tx=0x7f0e4000b8d0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 shutdown_connections 2024-04-25T09:12:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e3406b4d0 0x7f0e3406d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e580b56f0 0x7f0e5815a440 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0e580b6a90 0x7f0e580c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 --2- 172.21.15.62:0/2993689348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e580b27a0 0x7f0e58155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:29.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.369+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 >> 172.21.15.62:0/2993689348 conn(0x7f0e58066d10 msgr2=0x7f0e580b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:29.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.370+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 shutdown_connections 2024-04-25T09:12:29.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:29.370+0000 7f0e5d400640 1 -- 172.21.15.62:0/2993689348 wait complete. 2024-04-25T09:12:29.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:12:22.810299+0000', 'modified': '2024-04-25T09:12:25.868942+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7332}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7332': {'gid': 7332, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/3997398243', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 3997398243}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3997398243}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:12:29.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:12:29.779 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:12:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.013+0000 7fa6b6200640 1 Processor -- start 2024-04-25T09:12:30.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.013+0000 7fa6b6200640 1 -- start start 2024-04-25T09:12:30.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6b6200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 0x7fa6b015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6b6200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa6b00cf5f0 0x7fa6b00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6b6200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b00b3720 con 0x7fa6b00cfa70 2024-04-25T09:12:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6b6200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b00cff60 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6b6200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0070360 con 0x7fa6b015c4e0 2024-04-25T09:12:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6afe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa6b00cf5f0 0x7fa6b00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6afe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa6b00cf5f0 0x7fa6b00b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44986/0 (socket says 172.21.15.62:44986) 2024-04-25T09:12:30.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6afe00640 1 -- 172.21.15.62:0/1134739002 learned_addr learned my addr 172.21.15.62:0/1134739002 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/1134739002 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 0x7fa6b015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6afe00640 1 -- 172.21.15.62:0/1134739002 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 msgr2=0x7fa6b015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.015+0000 7fa6afe00640 1 --2- 172.21.15.62:0/1134739002 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 0x7fa6b015c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.016+0000 7fa6afe00640 1 -- 172.21.15.62:0/1134739002 >> v1:172.21.15.121:6789/0 conn(0x7fa6b00cfa70 legacy=0x7fa6b015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.016+0000 7fa6afe00640 1 -- 172.21.15.62:0/1134739002 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b0070640 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.017+0000 7fa6afe00640 1 --2- 172.21.15.62:0/1134739002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b42d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa698009870 tx=0x7fa69802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c5c2949bc00dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.017+0000 7fa6af400640 1 -- 172.21.15.62:0/1134739002 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698037070 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.017+0000 7fa6af400640 1 -- 172.21.15.62:0/1134739002 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa698032440 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.017+0000 7fa6af400640 1 -- 172.21.15.62:0/1134739002 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698032810 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.018+0000 7fa6b6200640 1 -- 172.21.15.62:0/1134739002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 msgr2=0x7fa6b00b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.018+0000 7fa6b6200640 1 --2- 172.21.15.62:0/1134739002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b42d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa698009870 tx=0x7fa69802a210 comp rx=0 tx=0).stop 2024-04-25T09:12:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.018+0000 7fa6b6200640 1 -- 172.21.15.62:0/1134739002 shutdown_connections 2024-04-25T09:12:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.018+0000 7fa6b6200640 1 --2- 172.21.15.62:0/1134739002 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b42d0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.018+0000 7fa6b6200640 1 --2- 172.21.15.62:0/1134739002 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 0x7fa6b015c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.018+0000 7fa6b6200640 1 -- 172.21.15.62:0/1134739002 >> 172.21.15.62:0/1134739002 conn(0x7fa6b00668f0 msgr2=0x7fa6b0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.018+0000 7fa6b6200640 1 -- 172.21.15.62:0/1134739002 shutdown_connections 2024-04-25T09:12:30.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.018+0000 7fa6b6200640 1 -- 172.21.15.62:0/1134739002 wait complete. 2024-04-25T09:12:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.020+0000 7fa6b6200640 1 Processor -- start 2024-04-25T09:12:30.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.020+0000 7fa6b6200640 1 -- start start 2024-04-25T09:12:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b6200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b00cfa70 0x7fa6b015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b6200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 0x7fa6b014b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b6200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b015ce00 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b6200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0070c00 con 0x7fa6b00cfa70 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b6200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0071120 con 0x7fa6b015c4e0 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44992/0 (socket says 172.21.15.62:44992) 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.021+0000 7fa6b4e00640 1 -- 172.21.15.62:0/3511148529 learned_addr learned my addr 172.21.15.62:0/3511148529 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6afe00640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b00cfa70 0x7fa6b015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6b5800640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 0x7fa6b014b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6b4e00640 1 -- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 msgr2=0x7fa6b014b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 0x7fa6b014b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6b4e00640 1 -- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b00cfa70 msgr2=0x7fa6b015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b00cfa70 0x7fa6b015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6b4e00640 1 -- 172.21.15.62:0/3511148529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b015a6f0 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6afe00640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b00cfa70 0x7fa6b015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:30.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b1f00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c7f0 tx=0x7fa6a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6acc00640 1 -- 172.21.15.62:0/3511148529 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0015070 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6acc00640 1 -- 172.21.15.62:0/3511148529 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a0004220 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6acc00640 1 -- 172.21.15.62:0/3511148529 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0007620 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.022+0000 7fa6b6200640 1 -- 172.21.15.62:0/3511148529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b015a9d0 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.023+0000 7fa6b6200640 1 -- 172.21.15.62:0/3511148529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b015af90 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.025+0000 7fa6acc00640 1 -- 172.21.15.62:0/3511148529 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6a00077c0 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.026+0000 7fa6acc00640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b500 0x7fa68c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.026+0000 7fa6acc00640 1 -- 172.21.15.62:0/3511148529 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6a0089fd0 con 0x7fa6b00cf5f0 2024-04-25T09:12:30.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.027+0000 7fa6ba5b8740 1 Processor -- start 2024-04-25T09:12:30.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.027+0000 7fa6ba5b8740 1 -- start start 2024-04-25T09:12:30.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.027+0000 7fa6afe00640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b500 0x7fa68c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e143524b30 0x55e14354c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e143524f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 0x55e1435232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e143488620 con 0x55e14354ceb0 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e143431ca0 con 0x55e143524b30 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1434846c0 con 0x55e14356d7a0 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e143524f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e143524f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44994/0 (socket says 172.21.15.62:44994) 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 -- 172.21.15.62:0/2004520501 learned_addr learned my addr 172.21.15.62:0/2004520501 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b500 0x7fa68c06d9c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa698034000 tx=0x7fa69802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6b5800640 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 0x55e1435232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e143524b30 0x55e14354c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 -- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 msgr2=0x55e1435232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 0x55e1435232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 -- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e143524b30 msgr2=0x55e14354c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e143524b30 0x55e14354c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6afe00640 1 -- 172.21.15.62:0/2004520501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e143438f40 con 0x55e14354ceb0 2024-04-25T09:12:30.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.028+0000 7fa6b5800640 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 0x55e1435232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:30.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.029+0000 7fa6afe00640 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e143524f10 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa6980090b0 tx=0x7fa698009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04bc15799d85b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.029+0000 7fa6a5600640 1 -- 172.21.15.62:0/2004520501 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69804d070 con 0x55e14354ceb0 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.029+0000 7fa6a5600640 1 -- 172.21.15.62:0/2004520501 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69803c660 con 0x55e14354ceb0 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.029+0000 7fa6a5600640 1 -- 172.21.15.62:0/2004520501 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698037040 con 0x55e14354ceb0 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/2004520501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 msgr2=0x55e143524f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e143524f10 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa6980090b0 tx=0x7fa698009240 comp rx=0 tx=0).stop 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/2004520501 shutdown_connections 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e143524f10 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 0x55e1435232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/2004520501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e143524b30 0x55e14354c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/2004520501 >> 172.21.15.62:0/2004520501 conn(0x55e143521d50 msgr2=0x55e14356a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/2004520501 shutdown_connections 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.030+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/2004520501 wait complete. 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.031+0000 7fa6ba5b8740 1 -- start start 2024-04-25T09:12:30.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.031+0000 7fa6ba5b8740 1 Processor -- start 2024-04-25T09:12:30.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e1435413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14356d7a0 0x55e143544010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 0x55e14354ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14354c4e0 con 0x55e14354ceb0 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1434271f0 con 0x55e14356d7a0 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e14354bed0 con 0x55e143545000 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6ba5b8740 10 client.0 _ll_register_callbacks cb 0x55e143444ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e1435413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e1435413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45004/0 (socket says 172.21.15.62:45004) 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.032+0000 7fa6b4e00640 1 -- 172.21.15.62:0/963363751 learned_addr learned my addr 172.21.15.62:0/963363751 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6b4e00640 1 -- 172.21.15.62:0/963363751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 msgr2=0x55e14354ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 0x55e14354ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6b4e00640 1 -- 172.21.15.62:0/963363751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14356d7a0 msgr2=0x55e143544010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14356d7a0 0x55e143544010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6b4e00640 1 -- 172.21.15.62:0/963363751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e143446c50 con 0x55e14354ceb0 2024-04-25T09:12:30.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e1435413c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0060920 tx=0x7fa6a0058f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e76333b0a757bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa69e000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa69e000640 1 -- 172.21.15.62:0/963363751 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0070020 con 0x55e14354ceb0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa69e000640 1 -- 172.21.15.62:0/963363751 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6a0057da0 con 0x55e14354ceb0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/963363751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa69803c660 con 0x55e14354ceb0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa69e000640 1 -- 172.21.15.62:0/963363751 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6a001e050 con 0x55e14354ceb0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6ba5b8740 10 client.7485 fetch_fsmap learned FSMap version 64 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.033+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/963363751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55e14319d2d0 con 0x55e14354ceb0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa69e000640 1 -- 172.21.15.62:0/963363751 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69803c660 con 0x55e14354ceb0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa69e000640 1 -- 172.21.15.62:0/963363751 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa6a0054490 con 0x55e14354ceb0 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa6ba5b8740 10 client.7485 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa6ba5b8740 10 client.7485 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa6ba5b8740 20 client.7485 populate_metadata read hostname 'smithi062' 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55e14354b440 0x55e14357e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa6ba5b8740 4 client.7485 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:12:30.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/963363751 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e143495b60 con 0x55e14354b440 2024-04-25T09:12:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.034+0000 7fa6afe00640 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55e14354b440 0x55e14357e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.035+0000 7fa6afe00640 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55e14354b440 0x55e14357e120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7b65888ca4896 server_cookie=71e5dd498cb25e1 in_seq=0 out_seq=0 2024-04-25T09:12:30.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.035+0000 7fa69e000640 10 client.7485 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa69e000640 1 -- 172.21.15.62:0/963363751 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e143495b60 con 0x55e14354b440 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa69e000640 10 client.7485 handle_command_reply: tid=0 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 client.7485 shutdown 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/963363751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 msgr2=0x55e1435413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e1435413c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0060920 tx=0x7fa6a0058f80 comp rx=0 tx=0).stop 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/963363751 shutdown_connections 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55e14354b440 0x55e14357e120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 0x55e14354ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14356d7a0 0x55e143544010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/963363751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14354ceb0 0x55e1435413c0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa69e000640 0 client.7485 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:12:30.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/963363751 >> 172.21.15.62:0/963363751 conn(0x55e143524b30 msgr2=0x55e14356dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:30.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/963363751 shutdown_connections 2024-04-25T09:12:30.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/963363751 wait complete. 2024-04-25T09:12:30.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.036+0000 7fa6ba5b8740 20 client.7485 trim_cache size 0 max 16384 2024-04-25T09:12:30.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.054+0000 7fa6ba5b8740 1 Processor -- start 2024-04-25T09:12:30.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.054+0000 7fa6ba5b8740 1 -- start start 2024-04-25T09:12:30.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 0x55e143525bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 0x55e143547b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14356d7a0 0x55e14354a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b01ff020 con 0x55e14356d7a0 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b006dfb0 con 0x55e14354ceb0 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b01f4180 con 0x55e143545000 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6b5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14356d7a0 0x55e14354a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6b5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14356d7a0 0x55e14354a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45008/0 (socket says 172.21.15.62:45008) 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6b5800640 1 -- 172.21.15.62:0/3623543985 learned_addr learned my addr 172.21.15.62:0/3623543985 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 0x55e143525bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6b5800640 1 -- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 msgr2=0x55e143525bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6b5800640 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 0x55e143525bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6afe00640 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 0x55e143547b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.055+0000 7fa6b5800640 1 -- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 msgr2=0x55e143547b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.056+0000 7fa6b5800640 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 0x55e143547b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.056+0000 7fa6b5800640 1 -- 172.21.15.62:0/3623543985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1432676b0 con 0x55e14356d7a0 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.056+0000 7fa6afe00640 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 0x55e143547b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.056+0000 7fa6b5800640 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14356d7a0 0x55e14354a6a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80052d0 tx=0x7fa6a8005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4223586335d5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.056+0000 7fa6a4c00640 1 -- 172.21.15.62:0/3623543985 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8017020 con 0x55e14356d7a0 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.056+0000 7fa6a4c00640 1 -- 172.21.15.62:0/3623543985 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a800c070 con 0x55e14356d7a0 2024-04-25T09:12:30.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.056+0000 7fa6a4c00640 1 -- 172.21.15.62:0/3623543985 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800ee70 con 0x55e14356d7a0 2024-04-25T09:12:30.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/3623543985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14356d7a0 msgr2=0x55e14354a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14356d7a0 0x55e14354a6a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80052d0 tx=0x7fa6a8005790 comp rx=0 tx=0).stop 2024-04-25T09:12:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/3623543985 shutdown_connections 2024-04-25T09:12:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e14356d7a0 0x55e14354a6a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 0x55e143547b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/3623543985 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e143545000 0x55e143525bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/3623543985 >> 172.21.15.62:0/3623543985 conn(0x55e143524b30 msgr2=0x55e1435420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/3623543985 shutdown_connections 2024-04-25T09:12:30.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.057+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/3623543985 wait complete. 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.058+0000 7fa6ba5b8740 1 -- start start 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.058+0000 7fa6ba5b8740 1 Processor -- start 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e143545000 0x55e14354ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 0x55e1435d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6ba5b8740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 0x55e143525410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e14354b2c0 con 0x55e143545000 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1435493e0 con 0x55e14354ceb0 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6ba5b8740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e143549930 con 0x55e14356d7a0 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6ba5b8740 10 client.0 _ll_register_callbacks cb 0x7fa6a000a3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e143545000 0x55e14354ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e143545000 0x55e14354ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45010/0 (socket says 172.21.15.62:45010) 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6b4e00640 1 -- 172.21.15.62:0/4213665948 learned_addr learned my addr 172.21.15.62:0/4213665948 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:30.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6b4e00640 1 -- 172.21.15.62:0/4213665948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 msgr2=0x55e143525410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 0x55e143525410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6b4e00640 1 -- 172.21.15.62:0/4213665948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 msgr2=0x55e1435d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 0x55e1435d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.059+0000 7fa6b4e00640 1 -- 172.21.15.62:0/4213665948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1435424c0 con 0x55e143545000 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa6b4e00640 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e143545000 0x55e14354ac70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000a480 tx=0x7fa6a001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3440883da05f932c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa69e000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa69e000640 1 -- 172.21.15.62:0/4213665948 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0074070 con 0x55e143545000 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa69e000640 1 -- 172.21.15.62:0/4213665948 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6a0007ab0 con 0x55e143545000 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/4213665948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6a800c070 con 0x55e143545000 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa69e000640 1 -- 172.21.15.62:0/4213665948 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6a007e050 con 0x55e143545000 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa6ba5b8740 10 client.7491 fetch_fsmap learned FSMap version 64 2024-04-25T09:12:30.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/4213665948 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55e143524140 con 0x55e143545000 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa69e000640 1 -- 172.21.15.62:0/4213665948 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a800c070 con 0x55e143545000 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa69e000640 1 -- 172.21.15.62:0/4213665948 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa6a0081020 con 0x55e143545000 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.060+0000 7fa6ba5b8740 10 client.7491 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.061+0000 7fa6ba5b8740 10 client.7491 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.061+0000 7fa6ba5b8740 20 client.7491 populate_metadata read hostname 'smithi062' 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.061+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55e14351e8b0 0x55e143525950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.061+0000 7fa6ba5b8740 4 client.7491 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.061+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/4213665948 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e14348feb0 con 0x55e14351e8b0 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.061+0000 7fa6afe00640 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55e14351e8b0 0x55e143525950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.061+0000 7fa69e000640 10 client.7491 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:12:30.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.061+0000 7fa6afe00640 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55e14351e8b0 0x55e143525950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c06ab9023d3204 server_cookie=686ca2f0c6c9de60 in_seq=0 out_seq=0 2024-04-25T09:12:30.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa69e000640 1 -- 172.21.15.62:0/4213665948 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e14348feb0 con 0x55e14351e8b0 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa69e000640 10 client.7491 handle_command_reply: tid=0 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 client.7491 shutdown 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/4213665948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e143545000 msgr2=0x55e14354ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e143545000 0x55e14354ac70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000a480 tx=0x7fa6a001b040 comp rx=0 tx=0).stop 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/4213665948 shutdown_connections 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55e14351e8b0 0x55e143525950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55e14356d7a0 0x55e143525410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e14354ceb0 0x55e1435d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 --2- 172.21.15.62:0/4213665948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e143545000 0x55e14354ac70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/4213665948 >> 172.21.15.62:0/4213665948 conn(0x55e143524b30 msgr2=0x55e143547d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:30.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa69e000640 0 client.7491 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/4213665948 shutdown_connections 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 1 -- 172.21.15.62:0/4213665948 wait complete. 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.062+0000 7fa6ba5b8740 20 client.7491 trim_cache size 0 max 16384 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 7332, 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:12:30.055 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:12:30.056 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:12:30.056 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 6.7364068169999998, 2024-04-25T09:12:30.056 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 64, 2024-04-25T09:12:30.056 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 97, 2024-04-25T09:12:30.056 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 97, 2024-04-25T09:12:30.056 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 12.128853819 2024-04-25T09:12:30.056 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 -- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b500 msgr2=0x7fa68c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b500 0x7fa68c06d9c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa698034000 tx=0x7fa69802e000 comp rx=0 tx=0).stop 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 -- 172.21.15.62:0/3511148529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 msgr2=0x7fa6b00b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b1f00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c7f0 tx=0x7fa6a000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 -- 172.21.15.62:0/3511148529 shutdown_connections 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa68c06b500 0x7fa68c06d9c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6b015c4e0 0x7fa6b014b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6b00cfa70 0x7fa6b015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 --2- 172.21.15.62:0/3511148529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6b00cf5f0 0x7fa6b00b1f00 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 -- 172.21.15.62:0/3511148529 >> 172.21.15.62:0/3511148529 conn(0x7fa6b00668f0 msgr2=0x7fa6b00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:30.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 -- 172.21.15.62:0/3511148529 shutdown_connections 2024-04-25T09:12:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.063+0000 7fa6b6200640 1 -- 172.21.15.62:0/3511148529 wait complete. 2024-04-25T09:12:30.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:12:30.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.286+0000 7f7683000640 1 Processor -- start 2024-04-25T09:12:30.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.287+0000 7f7683000640 1 -- start start 2024-04-25T09:12:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.288+0000 7f7683000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f767c0b56f0 0x7f767c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.288+0000 7f7683000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b6a90 0x7f767c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.288+0000 7f7683000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c0d3110 con 0x7f767c0b27a0 2024-04-25T09:12:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.288+0000 7f7683000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767c161780 con 0x7f767c0b56f0 2024-04-25T09:12:30.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.288+0000 7f7683000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0cedc0 con 0x7f767c0b6a90 2024-04-25T09:12:30.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.289+0000 7f7681200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b6a90 0x7f767c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.289+0000 7f7681c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f767c0b56f0 0x7f767c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.289+0000 7f7681200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b6a90 0x7f767c161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:36984/0 (socket says 172.21.15.62:36984) 2024-04-25T09:12:30.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.289+0000 7f7681200640 1 -- 172.21.15.62:0/4138022547 learned_addr learned my addr 172.21.15.62:0/4138022547 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:30.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.289+0000 7f7682600640 1 --1- 172.21.15.62:0/4138022547 >> v1:172.21.15.121:6789/0 conn(0x7f767c0b27a0 0x7f767c161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35184/0 (socket says 172.21.15.62:35184) 2024-04-25T09:12:30.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.290+0000 7f7681c00640 1 -- 172.21.15.62:0/4138022547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b6a90 msgr2=0x7f767c161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:12:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.290+0000 7f7681c00640 1 --2- 172.21.15.62:0/4138022547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b6a90 0x7f767c161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.290+0000 7f7681c00640 1 -- 172.21.15.62:0/4138022547 >> v1:172.21.15.121:6789/0 conn(0x7f767c0b27a0 legacy=0x7f767c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.290+0000 7f7681c00640 1 -- 172.21.15.62:0/4138022547 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0d33f0 con 0x7f767c0b56f0 2024-04-25T09:12:30.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.290+0000 7f7681200640 1 --2- 172.21.15.62:0/4138022547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b6a90 0x7f767c161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:12:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.291+0000 7f7681c00640 1 --2- 172.21.15.62:0/4138022547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b56f0 0x7f767c160d00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f76700097a0 tx=0x7f767002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f87d85f813a826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.292+0000 7f767be00640 1 -- 172.21.15.62:0/4138022547 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670037070 con 0x7f767c0b56f0 2024-04-25T09:12:30.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.292+0000 7f767be00640 1 -- 172.21.15.62:0/4138022547 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7670032440 con 0x7f767c0b56f0 2024-04-25T09:12:30.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.292+0000 7f767be00640 1 -- 172.21.15.62:0/4138022547 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670032810 con 0x7f767c0b56f0 2024-04-25T09:12:30.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.293+0000 7f7683000640 1 -- 172.21.15.62:0/4138022547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b56f0 msgr2=0x7f767c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.293+0000 7f7683000640 1 --2- 172.21.15.62:0/4138022547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b56f0 0x7f767c160d00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f76700097a0 tx=0x7f767002a180 comp rx=0 tx=0).stop 2024-04-25T09:12:30.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.293+0000 7f7683000640 1 -- 172.21.15.62:0/4138022547 shutdown_connections 2024-04-25T09:12:30.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.293+0000 7f7683000640 1 --2- 172.21.15.62:0/4138022547 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b6a90 0x7f767c161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.293+0000 7f7683000640 1 --2- 172.21.15.62:0/4138022547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b56f0 0x7f767c160d00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.293+0000 7f7683000640 1 -- 172.21.15.62:0/4138022547 >> 172.21.15.62:0/4138022547 conn(0x7f767c066d10 msgr2=0x7f767c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:30.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.293+0000 7f7683000640 1 -- 172.21.15.62:0/4138022547 shutdown_connections 2024-04-25T09:12:30.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.293+0000 7f7683000640 1 -- 172.21.15.62:0/4138022547 wait complete. 2024-04-25T09:12:30.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.295+0000 7f7683000640 1 Processor -- start 2024-04-25T09:12:30.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.295+0000 7f7683000640 1 -- start start 2024-04-25T09:12:30.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7683000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b27a0 0x7f767c0aa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7683000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f767c0b56f0 0x7f767c0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7683000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b6a90 0x7f767c150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7683000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0b2d60 con 0x7f767c0b56f0 2024-04-25T09:12:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7683000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0cea50 con 0x7f767c0b6a90 2024-04-25T09:12:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7683000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0b2ee0 con 0x7f767c0b27a0 2024-04-25T09:12:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7681200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f767c0b56f0 0x7f767c0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7681200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f767c0b56f0 0x7f767c0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45012/0 (socket says 172.21.15.62:45012) 2024-04-25T09:12:30.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7681200640 1 -- 172.21.15.62:0/783909668 learned_addr learned my addr 172.21.15.62:0/783909668 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:12:30.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7681c00640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b27a0 0x7f767c0aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.296+0000 7f7682600640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b6a90 0x7f767c150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.297+0000 7f7681200640 1 -- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b27a0 msgr2=0x7f767c0aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.297+0000 7f7681200640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b27a0 0x7f767c0aa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.297+0000 7f7681200640 1 -- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b6a90 msgr2=0x7f767c150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.297+0000 7f7681200640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b6a90 0x7f767c150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.297+0000 7f7681200640 1 -- 172.21.15.62:0/783909668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0cbb40 con 0x7f767c0b56f0 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.297+0000 7f7681200640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f767c0b56f0 0x7f767c0cb600 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f766800e9b0 tx=0x7f766800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.297+0000 7f7679600640 1 -- 172.21.15.62:0/783909668 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668017070 con 0x7f767c0b56f0 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.298+0000 7f7679600640 1 -- 172.21.15.62:0/783909668 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766800c9a0 con 0x7f767c0b56f0 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.298+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c0aaa50 con 0x7f767c0b56f0 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.298+0000 7f7679600640 1 -- 172.21.15.62:0/783909668 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766800cd50 con 0x7f767c0b56f0 2024-04-25T09:12:30.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.298+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c0ab010 con 0x7f767c0b56f0 2024-04-25T09:12:30.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.299+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c04ef20 con 0x7f767c0b56f0 2024-04-25T09:12:30.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.300+0000 7f7679600640 1 -- 172.21.15.62:0/783909668 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f766801a070 con 0x7f767c0b56f0 2024-04-25T09:12:30.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.301+0000 7f7679600640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f765c06b4d0 0x7f765c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:12:30.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.301+0000 7f7679600640 1 -- 172.21.15.62:0/783909668 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f766808a150 con 0x7f767c0b56f0 2024-04-25T09:12:30.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.302+0000 7f7681c00640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f765c06b4d0 0x7f765c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:12:30.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.303+0000 7f7681c00640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f765c06b4d0 0x7f765c06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7670034000 tx=0x7f767002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:12:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.305+0000 7f7679600640 1 -- 172.21.15.62:0/783909668 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7668058f30 con 0x7f767c0b56f0 2024-04-25T09:12:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.549+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f767c0ce580 con 0x7f767c0b56f0 2024-04-25T09:12:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.550+0000 7f7679600640 1 -- 172.21.15.62:0/783909668 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7668058830 con 0x7f767c0b56f0 2024-04-25T09:12:30.537 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f765c06b4d0 msgr2=0x7f765c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f765c06b4d0 0x7f765c06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7670034000 tx=0x7f767002e000 comp rx=0 tx=0).stop 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f767c0b56f0 msgr2=0x7f767c0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f767c0b56f0 0x7f767c0cb600 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f766800e9b0 tx=0x7f766800ee70 comp rx=0 tx=0).stop 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 shutdown_connections 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f765c06b4d0 0x7f765c06d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f767c0b56f0 0x7f767c0cb600 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f767c0b6a90 0x7f767c150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 --2- 172.21.15.62:0/783909668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f767c0b27a0 0x7f767c0aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:12:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 >> 172.21.15.62:0/783909668 conn(0x7f767c066d10 msgr2=0x7f767c0b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:12:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 shutdown_connections 2024-04-25T09:12:30.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:12:30.552+0000 7f7683000640 1 -- 172.21.15.62:0/783909668 wait complete. 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:12:30.560 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:12:30.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:30.560 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:12:30.612 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42108sec preferred_lft 42108sec 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:12:30.613 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:30.614 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:12:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:12:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:12:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:30.614 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:12:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:12:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:12:30.614 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:12:30.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:30.615 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:12:30.696 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:12:30.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:30.700 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:12:30.756 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:12:30.756 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:12:30.756 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:12:30.756 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:12:30.756 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:12:30.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:30.757 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:12:30.757 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:30.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:12:30.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:12:30.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:12:30.757 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:30.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:30.913 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:12:30.965 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:12:30.965 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:12:30.966 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:12:30.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:30.966 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:12:31.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:12:31.053 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:31.109 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:12:31.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:12:31.109 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:31.164 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:12:31.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:31.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:12:31.304 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:12:31.304 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:12:31.338 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:12:31.338 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:12:31.338 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:12:31.338 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:12:31.339 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:12:31.341 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:12:31.341 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:12:31.341 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:12:31.341 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:12:31.341 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:12:31.346 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:12:31.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:31.346 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:12:31.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:31.418 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:12:31.476 INFO:teuthology.orchestra.run.smithi062.stdout:client5567 2024-04-25T09:12:31.476 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:12:31.477 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:12:31.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:31.523 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:31.523 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:31.523 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:12:31.523 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:31.523 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:31.523 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:31.523 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0414409 s, 25.3 MB/s 2024-04-25T09:12:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:31.748 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0469906 s, 22.3 MB/s 2024-04-25T09:12:31.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:31.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:31.789 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036527 s, 28.7 MB/s 2024-04-25T09:12:31.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371532 s, 28.2 MB/s 2024-04-25T09:12:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:31.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476663 s, 22.0 MB/s 2024-04-25T09:12:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:31.923 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371283 s, 28.2 MB/s 2024-04-25T09:12:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299236 s, 35.0 MB/s 2024-04-25T09:12:31.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:31.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:31.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288749 s, 36.3 MB/s 2024-04-25T09:12:32.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.023 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286556 s, 36.6 MB/s 2024-04-25T09:12:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.055 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270982 s, 38.7 MB/s 2024-04-25T09:12:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298746 s, 35.1 MB/s 2024-04-25T09:12:32.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209002 s, 50.2 MB/s 2024-04-25T09:12:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.149 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300253 s, 34.9 MB/s 2024-04-25T09:12:32.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.181 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028944 s, 36.2 MB/s 2024-04-25T09:12:32.210 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.210 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.210 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243596 s, 43.0 MB/s 2024-04-25T09:12:32.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.242 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274753 s, 38.2 MB/s 2024-04-25T09:12:32.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.276 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294026 s, 35.7 MB/s 2024-04-25T09:12:32.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.303 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239358 s, 43.8 MB/s 2024-04-25T09:12:32.333 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.333 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.333 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256769 s, 40.8 MB/s 2024-04-25T09:12:32.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.367 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02842 s, 36.9 MB/s 2024-04-25T09:12:32.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023485 s, 44.6 MB/s 2024-04-25T09:12:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226486 s, 46.3 MB/s 2024-04-25T09:12:32.467 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.468 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410419 s, 25.5 MB/s 2024-04-25T09:12:32.495 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.495 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.496 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237614 s, 44.1 MB/s 2024-04-25T09:12:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.524 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248167 s, 42.3 MB/s 2024-04-25T09:12:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030991 s, 33.8 MB/s 2024-04-25T09:12:32.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.593 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292624 s, 35.8 MB/s 2024-04-25T09:12:32.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.618 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199294 s, 52.6 MB/s 2024-04-25T09:12:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224254 s, 46.8 MB/s 2024-04-25T09:12:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.669 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213549 s, 49.1 MB/s 2024-04-25T09:12:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237474 s, 44.2 MB/s 2024-04-25T09:12:32.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226668 s, 46.3 MB/s 2024-04-25T09:12:32.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:32.728 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:32.728 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:32.728 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:12:32.728 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:32.728 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:32.728 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:32.728 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:32.812 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.812 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.812 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207224 s, 50.6 MB/s 2024-04-25T09:12:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.839 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235519 s, 44.5 MB/s 2024-04-25T09:12:32.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.868 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240645 s, 43.6 MB/s 2024-04-25T09:12:32.893 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.893 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.893 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212151 s, 49.4 MB/s 2024-04-25T09:12:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385456 s, 27.2 MB/s 2024-04-25T09:12:32.967 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.967 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.967 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270364 s, 38.8 MB/s 2024-04-25T09:12:32.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:32.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:32.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199391 s, 52.6 MB/s 2024-04-25T09:12:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.021 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246307 s, 42.6 MB/s 2024-04-25T09:12:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192872 s, 54.4 MB/s 2024-04-25T09:12:33.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.076 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276836 s, 37.9 MB/s 2024-04-25T09:12:33.104 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.104 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.104 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242944 s, 43.2 MB/s 2024-04-25T09:12:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223893 s, 46.8 MB/s 2024-04-25T09:12:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.157 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215245 s, 48.7 MB/s 2024-04-25T09:12:33.193 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.193 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.193 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031428 s, 33.4 MB/s 2024-04-25T09:12:33.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.224 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268166 s, 39.1 MB/s 2024-04-25T09:12:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247035 s, 42.4 MB/s 2024-04-25T09:12:33.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.279 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224937 s, 46.6 MB/s 2024-04-25T09:12:33.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.313 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305559 s, 34.3 MB/s 2024-04-25T09:12:33.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.340 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236904 s, 44.3 MB/s 2024-04-25T09:12:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.377 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325018 s, 32.3 MB/s 2024-04-25T09:12:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265017 s, 39.6 MB/s 2024-04-25T09:12:33.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.438 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247469 s, 42.4 MB/s 2024-04-25T09:12:33.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.464 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223595 s, 46.9 MB/s 2024-04-25T09:12:33.488 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.488 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.488 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206257 s, 50.8 MB/s 2024-04-25T09:12:33.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.515 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230086 s, 45.6 MB/s 2024-04-25T09:12:33.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.543 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024256 s, 43.2 MB/s 2024-04-25T09:12:33.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211663 s, 49.5 MB/s 2024-04-25T09:12:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250313 s, 41.9 MB/s 2024-04-25T09:12:33.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.621 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189388 s, 55.4 MB/s 2024-04-25T09:12:33.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.648 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223541 s, 46.9 MB/s 2024-04-25T09:12:33.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205058 s, 51.1 MB/s 2024-04-25T09:12:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229252 s, 45.7 MB/s 2024-04-25T09:12:33.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:33.701 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:33.702 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:33.702 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:12:33.702 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:33.702 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:33.702 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:33.702 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:33.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.790 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231958 s, 45.2 MB/s 2024-04-25T09:12:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.816 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216253 s, 48.5 MB/s 2024-04-25T09:12:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203876 s, 51.4 MB/s 2024-04-25T09:12:33.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.868 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234296 s, 44.8 MB/s 2024-04-25T09:12:33.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246933 s, 42.5 MB/s 2024-04-25T09:12:33.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.925 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232964 s, 45.0 MB/s 2024-04-25T09:12:33.954 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.954 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.954 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247886 s, 42.3 MB/s 2024-04-25T09:12:33.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:33.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:33.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219122 s, 47.9 MB/s 2024-04-25T09:12:34.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.011 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266725 s, 39.3 MB/s 2024-04-25T09:12:34.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021833 s, 48.0 MB/s 2024-04-25T09:12:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.069 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274296 s, 38.2 MB/s 2024-04-25T09:12:34.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.095 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224861 s, 46.6 MB/s 2024-04-25T09:12:34.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.121 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022162 s, 47.3 MB/s 2024-04-25T09:12:34.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.147 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213798 s, 49.0 MB/s 2024-04-25T09:12:34.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.176 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243472 s, 43.1 MB/s 2024-04-25T09:12:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247564 s, 42.4 MB/s 2024-04-25T09:12:34.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.233 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241589 s, 43.4 MB/s 2024-04-25T09:12:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274749 s, 38.2 MB/s 2024-04-25T09:12:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278773 s, 37.6 MB/s 2024-04-25T09:12:34.325 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.325 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.325 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236012 s, 44.4 MB/s 2024-04-25T09:12:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.355 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248702 s, 42.2 MB/s 2024-04-25T09:12:34.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.385 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256803 s, 40.8 MB/s 2024-04-25T09:12:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.417 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279646 s, 37.5 MB/s 2024-04-25T09:12:34.451 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.451 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.452 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296593 s, 35.4 MB/s 2024-04-25T09:12:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.478 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220011 s, 47.7 MB/s 2024-04-25T09:12:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.507 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250194 s, 41.9 MB/s 2024-04-25T09:12:34.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.538 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270844 s, 38.7 MB/s 2024-04-25T09:12:34.565 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.565 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.565 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230213 s, 45.5 MB/s 2024-04-25T09:12:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.592 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227895 s, 46.0 MB/s 2024-04-25T09:12:34.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.622 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254262 s, 41.2 MB/s 2024-04-25T09:12:34.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.647 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.647 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208279 s, 50.3 MB/s 2024-04-25T09:12:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201339 s, 52.1 MB/s 2024-04-25T09:12:34.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:34.674 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:34.675 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:34.675 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:12:34.675 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:34.675 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:34.675 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:34.675 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.761 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021056 s, 49.8 MB/s 2024-04-25T09:12:34.791 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.791 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.791 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255711 s, 41.0 MB/s 2024-04-25T09:12:34.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.816 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204724 s, 51.2 MB/s 2024-04-25T09:12:34.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.836 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170359 s, 61.6 MB/s 2024-04-25T09:12:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.861 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021783 s, 48.1 MB/s 2024-04-25T09:12:34.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212378 s, 49.4 MB/s 2024-04-25T09:12:34.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.911 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019567 s, 53.6 MB/s 2024-04-25T09:12:34.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.936 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213166 s, 49.2 MB/s 2024-04-25T09:12:34.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254164 s, 41.3 MB/s 2024-04-25T09:12:34.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:34.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:34.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206551 s, 50.8 MB/s 2024-04-25T09:12:35.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192346 s, 54.5 MB/s 2024-04-25T09:12:35.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.041 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220485 s, 47.6 MB/s 2024-04-25T09:12:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.067 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223125 s, 47.0 MB/s 2024-04-25T09:12:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227726 s, 46.0 MB/s 2024-04-25T09:12:35.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.118 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206114 s, 50.9 MB/s 2024-04-25T09:12:35.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234766 s, 44.7 MB/s 2024-04-25T09:12:35.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.173 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232943 s, 45.0 MB/s 2024-04-25T09:12:35.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.203 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258358 s, 40.6 MB/s 2024-04-25T09:12:35.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.230 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229016 s, 45.8 MB/s 2024-04-25T09:12:35.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.256 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228975 s, 45.8 MB/s 2024-04-25T09:12:35.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.282 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212064 s, 49.4 MB/s 2024-04-25T09:12:35.311 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.311 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.311 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258137 s, 40.6 MB/s 2024-04-25T09:12:35.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.341 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258021 s, 40.6 MB/s 2024-04-25T09:12:35.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.369 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236473 s, 44.3 MB/s 2024-04-25T09:12:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.394 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207357 s, 50.6 MB/s 2024-04-25T09:12:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254013 s, 41.3 MB/s 2024-04-25T09:12:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.455 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264314 s, 39.7 MB/s 2024-04-25T09:12:35.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.484 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025358 s, 41.4 MB/s 2024-04-25T09:12:35.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.513 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246724 s, 42.5 MB/s 2024-04-25T09:12:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.539 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215061 s, 48.8 MB/s 2024-04-25T09:12:35.565 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.565 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.565 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021761 s, 48.2 MB/s 2024-04-25T09:12:35.590 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.590 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.590 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202354 s, 51.8 MB/s 2024-04-25T09:12:35.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:35.593 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:35.593 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:35.594 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:12:35.594 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:35.594 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:35.594 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:35.594 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:35.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.684 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.684 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244046 s, 43.0 MB/s 2024-04-25T09:12:35.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.706 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182127 s, 57.6 MB/s 2024-04-25T09:12:35.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.726 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167097 s, 62.8 MB/s 2024-04-25T09:12:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230663 s, 45.5 MB/s 2024-04-25T09:12:35.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.781 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232327 s, 45.1 MB/s 2024-04-25T09:12:35.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.807 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213852 s, 49.0 MB/s 2024-04-25T09:12:35.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205818 s, 50.9 MB/s 2024-04-25T09:12:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022232 s, 47.2 MB/s 2024-04-25T09:12:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.890 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275768 s, 38.0 MB/s 2024-04-25T09:12:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228313 s, 45.9 MB/s 2024-04-25T09:12:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.944 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.945 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227262 s, 46.1 MB/s 2024-04-25T09:12:35.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.970 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222186 s, 47.2 MB/s 2024-04-25T09:12:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:35.996 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219279 s, 47.8 MB/s 2024-04-25T09:12:36.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.022 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226819 s, 46.2 MB/s 2024-04-25T09:12:36.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.048 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219595 s, 47.8 MB/s 2024-04-25T09:12:36.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.074 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022026 s, 47.6 MB/s 2024-04-25T09:12:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.101 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223769 s, 46.9 MB/s 2024-04-25T09:12:36.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.131 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245917 s, 42.6 MB/s 2024-04-25T09:12:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214006 s, 49.0 MB/s 2024-04-25T09:12:36.185 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.185 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.185 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247577 s, 42.4 MB/s 2024-04-25T09:12:36.212 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.212 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.212 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222071 s, 47.2 MB/s 2024-04-25T09:12:36.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.237 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212188 s, 49.4 MB/s 2024-04-25T09:12:36.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.263 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206029 s, 50.9 MB/s 2024-04-25T09:12:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.290 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233973 s, 44.8 MB/s 2024-04-25T09:12:36.315 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.316 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021612 s, 48.5 MB/s 2024-04-25T09:12:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020852 s, 50.3 MB/s 2024-04-25T09:12:36.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.370 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024616 s, 42.6 MB/s 2024-04-25T09:12:36.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.398 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023868 s, 43.9 MB/s 2024-04-25T09:12:36.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219948 s, 47.7 MB/s 2024-04-25T09:12:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244017 s, 43.0 MB/s 2024-04-25T09:12:36.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.480 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232007 s, 45.2 MB/s 2024-04-25T09:12:36.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.505 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208393 s, 50.3 MB/s 2024-04-25T09:12:36.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:36.508 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:36.508 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:36.508 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:12:36.508 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:36.509 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:36.509 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:36.509 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:36.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.602 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258337 s, 40.6 MB/s 2024-04-25T09:12:36.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.630 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244369 s, 42.9 MB/s 2024-04-25T09:12:36.655 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.655 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.655 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02016 s, 52.0 MB/s 2024-04-25T09:12:36.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.682 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.682 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224337 s, 46.7 MB/s 2024-04-25T09:12:36.708 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.708 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.708 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216 s, 48.5 MB/s 2024-04-25T09:12:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.735 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223444 s, 46.9 MB/s 2024-04-25T09:12:36.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.760 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217524 s, 48.2 MB/s 2024-04-25T09:12:36.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.787 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228851 s, 45.8 MB/s 2024-04-25T09:12:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.815 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236366 s, 44.4 MB/s 2024-04-25T09:12:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213724 s, 49.1 MB/s 2024-04-25T09:12:36.867 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.867 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.867 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223871 s, 46.8 MB/s 2024-04-25T09:12:36.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.893 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.893 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215708 s, 48.6 MB/s 2024-04-25T09:12:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.921 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243572 s, 43.0 MB/s 2024-04-25T09:12:36.947 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.947 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.947 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218553 s, 48.0 MB/s 2024-04-25T09:12:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228274 s, 45.9 MB/s 2024-04-25T09:12:36.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:36.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:36.999 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210921 s, 49.7 MB/s 2024-04-25T09:12:37.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.023 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020444 s, 51.3 MB/s 2024-04-25T09:12:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.053 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261156 s, 40.2 MB/s 2024-04-25T09:12:37.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.079 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217299 s, 48.3 MB/s 2024-04-25T09:12:37.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.108 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.108 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245077 s, 42.8 MB/s 2024-04-25T09:12:37.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.137 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248428 s, 42.2 MB/s 2024-04-25T09:12:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022824 s, 45.9 MB/s 2024-04-25T09:12:37.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.191 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230435 s, 45.5 MB/s 2024-04-25T09:12:37.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.216 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209447 s, 50.1 MB/s 2024-04-25T09:12:37.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.240 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205011 s, 51.1 MB/s 2024-04-25T09:12:37.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.269 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237284 s, 44.2 MB/s 2024-04-25T09:12:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209794 s, 50.0 MB/s 2024-04-25T09:12:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213101 s, 49.2 MB/s 2024-04-25T09:12:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.343 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221169 s, 47.4 MB/s 2024-04-25T09:12:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.376 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282155 s, 37.2 MB/s 2024-04-25T09:12:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.406 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254614 s, 41.2 MB/s 2024-04-25T09:12:37.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224904 s, 46.6 MB/s 2024-04-25T09:12:37.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:37.436 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:37.436 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:37.436 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:12:37.436 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:37.436 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:37.436 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:37.436 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:37.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.522 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213773 s, 49.1 MB/s 2024-04-25T09:12:37.551 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.552 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.552 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264409 s, 39.7 MB/s 2024-04-25T09:12:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.576 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208187 s, 50.4 MB/s 2024-04-25T09:12:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225299 s, 46.5 MB/s 2024-04-25T09:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218436 s, 48.0 MB/s 2024-04-25T09:12:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189068 s, 55.5 MB/s 2024-04-25T09:12:37.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.677 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219876 s, 47.7 MB/s 2024-04-25T09:12:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018464 s, 56.8 MB/s 2024-04-25T09:12:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.729 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254115 s, 41.3 MB/s 2024-04-25T09:12:37.759 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.759 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.759 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259162 s, 40.5 MB/s 2024-04-25T09:12:37.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.785 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209266 s, 50.1 MB/s 2024-04-25T09:12:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021608 s, 48.5 MB/s 2024-04-25T09:12:37.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.837 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221931 s, 47.2 MB/s 2024-04-25T09:12:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211263 s, 49.6 MB/s 2024-04-25T09:12:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210544 s, 49.8 MB/s 2024-04-25T09:12:37.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230718 s, 45.4 MB/s 2024-04-25T09:12:37.944 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.944 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.944 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260052 s, 40.3 MB/s 2024-04-25T09:12:37.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:37.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:37.973 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224941 s, 46.6 MB/s 2024-04-25T09:12:38.001 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.001 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.001 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023416 s, 44.8 MB/s 2024-04-25T09:12:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256538 s, 40.9 MB/s 2024-04-25T09:12:38.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.054 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180732 s, 58.0 MB/s 2024-04-25T09:12:38.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.078 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205002 s, 51.1 MB/s 2024-04-25T09:12:38.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204035 s, 51.4 MB/s 2024-04-25T09:12:38.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.126 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191672 s, 54.7 MB/s 2024-04-25T09:12:38.155 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.155 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.155 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237629 s, 44.1 MB/s 2024-04-25T09:12:38.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210754 s, 49.8 MB/s 2024-04-25T09:12:38.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.204 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195828 s, 53.5 MB/s 2024-04-25T09:12:38.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.228 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208912 s, 50.2 MB/s 2024-04-25T09:12:38.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.255 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218992 s, 47.9 MB/s 2024-04-25T09:12:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208049 s, 50.4 MB/s 2024-04-25T09:12:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.307 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229815 s, 45.6 MB/s 2024-04-25T09:12:38.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.341 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029297 s, 35.8 MB/s 2024-04-25T09:12:38.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:38.345 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:38.345 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:38.345 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:12:38.345 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:38.345 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:38.345 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:38.345 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:38.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024843 s, 42.2 MB/s 2024-04-25T09:12:38.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.460 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213288 s, 49.2 MB/s 2024-04-25T09:12:38.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.488 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236115 s, 44.4 MB/s 2024-04-25T09:12:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.513 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219214 s, 47.8 MB/s 2024-04-25T09:12:38.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.536 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198638 s, 52.8 MB/s 2024-04-25T09:12:38.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.568 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274408 s, 38.2 MB/s 2024-04-25T09:12:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252623 s, 41.5 MB/s 2024-04-25T09:12:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.626 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245052 s, 42.8 MB/s 2024-04-25T09:12:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208356 s, 50.3 MB/s 2024-04-25T09:12:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.681 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025988 s, 40.3 MB/s 2024-04-25T09:12:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.717 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321957 s, 32.6 MB/s 2024-04-25T09:12:38.744 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.744 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.744 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022744 s, 46.1 MB/s 2024-04-25T09:12:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246834 s, 42.5 MB/s 2024-04-25T09:12:38.799 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.799 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.799 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022362 s, 46.9 MB/s 2024-04-25T09:12:38.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.825 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021771 s, 48.2 MB/s 2024-04-25T09:12:38.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.850 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223494 s, 46.9 MB/s 2024-04-25T09:12:38.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.877 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222744 s, 47.1 MB/s 2024-04-25T09:12:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.901 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204428 s, 51.3 MB/s 2024-04-25T09:12:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213201 s, 49.2 MB/s 2024-04-25T09:12:38.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.952 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213247 s, 49.2 MB/s 2024-04-25T09:12:38.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:38.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:38.976 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204053 s, 51.4 MB/s 2024-04-25T09:12:39.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.003 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228389 s, 45.9 MB/s 2024-04-25T09:12:39.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.027 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197176 s, 53.2 MB/s 2024-04-25T09:12:39.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.051 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187455 s, 55.9 MB/s 2024-04-25T09:12:39.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.074 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198162 s, 52.9 MB/s 2024-04-25T09:12:39.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.096 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180403 s, 58.1 MB/s 2024-04-25T09:12:39.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230016 s, 45.6 MB/s 2024-04-25T09:12:39.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189165 s, 55.4 MB/s 2024-04-25T09:12:39.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222795 s, 47.1 MB/s 2024-04-25T09:12:39.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242021 s, 43.3 MB/s 2024-04-25T09:12:39.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.229 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228816 s, 45.8 MB/s 2024-04-25T09:12:39.257 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.257 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.257 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238478 s, 44.0 MB/s 2024-04-25T09:12:39.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:39.262 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:39.262 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:39.262 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:12:39.262 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:39.262 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:39.262 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:39.262 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:39.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245434 s, 42.7 MB/s 2024-04-25T09:12:39.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.376 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199293 s, 52.6 MB/s 2024-04-25T09:12:39.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.404 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223329 s, 47.0 MB/s 2024-04-25T09:12:39.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273431 s, 38.3 MB/s 2024-04-25T09:12:39.461 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.461 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.461 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214958 s, 48.8 MB/s 2024-04-25T09:12:39.488 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.488 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.488 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232074 s, 45.2 MB/s 2024-04-25T09:12:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.516 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235882 s, 44.5 MB/s 2024-04-25T09:12:39.542 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.543 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225953 s, 46.4 MB/s 2024-04-25T09:12:39.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.569 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218073 s, 48.1 MB/s 2024-04-25T09:12:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209751 s, 50.0 MB/s 2024-04-25T09:12:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206368 s, 50.8 MB/s 2024-04-25T09:12:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211436 s, 49.6 MB/s 2024-04-25T09:12:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264404 s, 39.7 MB/s 2024-04-25T09:12:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.701 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209866 s, 50.0 MB/s 2024-04-25T09:12:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207248 s, 50.6 MB/s 2024-04-25T09:12:39.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.748 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185367 s, 56.6 MB/s 2024-04-25T09:12:39.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226315 s, 46.3 MB/s 2024-04-25T09:12:39.797 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193819 s, 54.1 MB/s 2024-04-25T09:12:39.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245023 s, 42.8 MB/s 2024-04-25T09:12:39.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254948 s, 41.1 MB/s 2024-04-25T09:12:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228051 s, 46.0 MB/s 2024-04-25T09:12:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203016 s, 51.6 MB/s 2024-04-25T09:12:39.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.935 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226009 s, 46.4 MB/s 2024-04-25T09:12:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221121 s, 47.4 MB/s 2024-04-25T09:12:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188685 s, 55.6 MB/s 2024-04-25T09:12:40.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.009 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204354 s, 51.3 MB/s 2024-04-25T09:12:40.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.034 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209152 s, 50.1 MB/s 2024-04-25T09:12:40.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.061 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217517 s, 48.2 MB/s 2024-04-25T09:12:40.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.086 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211255 s, 49.6 MB/s 2024-04-25T09:12:40.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.113 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.113 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222801 s, 47.1 MB/s 2024-04-25T09:12:40.140 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.140 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.140 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229506 s, 45.7 MB/s 2024-04-25T09:12:40.168 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.168 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.168 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243045 s, 43.1 MB/s 2024-04-25T09:12:40.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:40.171 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:40.172 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:40.172 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:12:40.172 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:40.172 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:40.172 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:40.172 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.256 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200849 s, 52.2 MB/s 2024-04-25T09:12:40.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231433 s, 45.3 MB/s 2024-04-25T09:12:40.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.308 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215553 s, 48.6 MB/s 2024-04-25T09:12:40.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.338 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262089 s, 40.0 MB/s 2024-04-25T09:12:40.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.370 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263138 s, 39.8 MB/s 2024-04-25T09:12:40.397 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.397 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.397 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227508 s, 46.1 MB/s 2024-04-25T09:12:40.428 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.428 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.428 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272278 s, 38.5 MB/s 2024-04-25T09:12:40.451 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.451 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.451 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192736 s, 54.4 MB/s 2024-04-25T09:12:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.481 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259555 s, 40.4 MB/s 2024-04-25T09:12:40.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.507 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022104 s, 47.4 MB/s 2024-04-25T09:12:40.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.536 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235549 s, 44.5 MB/s 2024-04-25T09:12:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210058 s, 49.9 MB/s 2024-04-25T09:12:40.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.586 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214955 s, 48.8 MB/s 2024-04-25T09:12:40.608 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.608 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.608 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194025 s, 54.0 MB/s 2024-04-25T09:12:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255806 s, 41.0 MB/s 2024-04-25T09:12:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177881 s, 58.9 MB/s 2024-04-25T09:12:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.686 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216333 s, 48.5 MB/s 2024-04-25T09:12:40.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.711 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216409 s, 48.5 MB/s 2024-04-25T09:12:40.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.738 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.738 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213342 s, 49.1 MB/s 2024-04-25T09:12:40.764 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.764 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.764 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227639 s, 46.1 MB/s 2024-04-25T09:12:40.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.789 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207569 s, 50.5 MB/s 2024-04-25T09:12:40.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.814 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196636 s, 53.3 MB/s 2024-04-25T09:12:40.840 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.840 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.840 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221079 s, 47.4 MB/s 2024-04-25T09:12:40.867 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.867 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.867 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224618 s, 46.7 MB/s 2024-04-25T09:12:40.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.890 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188397 s, 55.7 MB/s 2024-04-25T09:12:40.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.921 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261457 s, 40.1 MB/s 2024-04-25T09:12:40.948 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.948 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.948 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230795 s, 45.4 MB/s 2024-04-25T09:12:40.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.971 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184706 s, 56.8 MB/s 2024-04-25T09:12:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:40.994 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191847 s, 54.7 MB/s 2024-04-25T09:12:41.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.019 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204395 s, 51.3 MB/s 2024-04-25T09:12:41.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.044 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206102 s, 50.9 MB/s 2024-04-25T09:12:41.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.070 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220965 s, 47.5 MB/s 2024-04-25T09:12:41.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:41.073 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:41.073 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:41.073 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:12:41.073 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:41.073 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:41.073 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:41.073 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.166 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262853 s, 39.9 MB/s 2024-04-25T09:12:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026956 s, 38.9 MB/s 2024-04-25T09:12:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225121 s, 46.6 MB/s 2024-04-25T09:12:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.250 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022363 s, 46.9 MB/s 2024-04-25T09:12:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230582 s, 45.5 MB/s 2024-04-25T09:12:41.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.306 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252162 s, 41.6 MB/s 2024-04-25T09:12:41.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.336 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248784 s, 42.1 MB/s 2024-04-25T09:12:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.363 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230291 s, 45.5 MB/s 2024-04-25T09:12:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251517 s, 41.7 MB/s 2024-04-25T09:12:41.417 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.417 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.417 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212413 s, 49.4 MB/s 2024-04-25T09:12:41.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.443 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021303 s, 49.2 MB/s 2024-04-25T09:12:41.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.473 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257363 s, 40.7 MB/s 2024-04-25T09:12:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.498 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210747 s, 49.8 MB/s 2024-04-25T09:12:41.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.524 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218428 s, 48.0 MB/s 2024-04-25T09:12:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.554 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259367 s, 40.4 MB/s 2024-04-25T09:12:41.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.581 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230416 s, 45.5 MB/s 2024-04-25T09:12:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.609 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221328 s, 47.4 MB/s 2024-04-25T09:12:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236163 s, 44.4 MB/s 2024-04-25T09:12:41.662 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.662 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.662 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213305 s, 49.2 MB/s 2024-04-25T09:12:41.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.688 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212461 s, 49.4 MB/s 2024-04-25T09:12:41.718 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.718 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.719 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263323 s, 39.8 MB/s 2024-04-25T09:12:41.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.746 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234878 s, 44.6 MB/s 2024-04-25T09:12:41.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.772 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217838 s, 48.1 MB/s 2024-04-25T09:12:41.797 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208049 s, 50.4 MB/s 2024-04-25T09:12:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250272 s, 41.9 MB/s 2024-04-25T09:12:41.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.852 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213238 s, 49.2 MB/s 2024-04-25T09:12:41.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.878 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231099 s, 45.4 MB/s 2024-04-25T09:12:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.905 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225838 s, 46.4 MB/s 2024-04-25T09:12:41.931 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.931 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.931 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214231 s, 48.9 MB/s 2024-04-25T09:12:41.957 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.957 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.957 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222085 s, 47.2 MB/s 2024-04-25T09:12:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247096 s, 42.4 MB/s 2024-04-25T09:12:42.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.012 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221294 s, 47.4 MB/s 2024-04-25T09:12:42.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:42.015 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:42.015 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:42.016 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:12:42.016 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:42.016 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:42.016 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:42.016 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:42.108 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.108 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.108 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266096 s, 39.4 MB/s 2024-04-25T09:12:42.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.129 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.129 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017805 s, 58.9 MB/s 2024-04-25T09:12:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.157 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239689 s, 43.7 MB/s 2024-04-25T09:12:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.182 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217582 s, 48.2 MB/s 2024-04-25T09:12:42.208 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.208 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.208 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222208 s, 47.2 MB/s 2024-04-25T09:12:42.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.234 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207235 s, 50.6 MB/s 2024-04-25T09:12:42.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242082 s, 43.3 MB/s 2024-04-25T09:12:42.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219552 s, 47.8 MB/s 2024-04-25T09:12:42.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.314 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219057 s, 47.9 MB/s 2024-04-25T09:12:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204317 s, 51.3 MB/s 2024-04-25T09:12:42.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.365 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.365 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216641 s, 48.4 MB/s 2024-04-25T09:12:42.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.386 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185779 s, 56.4 MB/s 2024-04-25T09:12:42.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.410 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207351 s, 50.6 MB/s 2024-04-25T09:12:42.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.440 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252426 s, 41.5 MB/s 2024-04-25T09:12:42.470 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.471 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264508 s, 39.6 MB/s 2024-04-25T09:12:42.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.494 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198857 s, 52.7 MB/s 2024-04-25T09:12:42.519 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.519 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.519 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208997 s, 50.2 MB/s 2024-04-25T09:12:42.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.545 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214416 s, 48.9 MB/s 2024-04-25T09:12:42.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.571 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217334 s, 48.2 MB/s 2024-04-25T09:12:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221758 s, 47.3 MB/s 2024-04-25T09:12:42.620 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.620 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.620 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196483 s, 53.4 MB/s 2024-04-25T09:12:42.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.649 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241052 s, 43.5 MB/s 2024-04-25T09:12:42.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.675 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226649 s, 46.3 MB/s 2024-04-25T09:12:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.700 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193899 s, 54.1 MB/s 2024-04-25T09:12:42.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204932 s, 51.2 MB/s 2024-04-25T09:12:42.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.753 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024677 s, 42.5 MB/s 2024-04-25T09:12:42.780 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.780 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.780 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232354 s, 45.1 MB/s 2024-04-25T09:12:42.803 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.803 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.803 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198237 s, 52.9 MB/s 2024-04-25T09:12:42.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.823 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172432 s, 60.8 MB/s 2024-04-25T09:12:42.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.848 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.848 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204564 s, 51.3 MB/s 2024-04-25T09:12:42.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.872 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207598 s, 50.5 MB/s 2024-04-25T09:12:42.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215692 s, 48.6 MB/s 2024-04-25T09:12:42.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:42.900 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:42.901 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:42.901 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:12:42.901 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:42.901 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:42.901 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:42.901 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:42.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:42.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:42.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169448 s, 61.9 MB/s 2024-04-25T09:12:43.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.008 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249806 s, 42.0 MB/s 2024-04-25T09:12:43.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.030 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197115 s, 53.2 MB/s 2024-04-25T09:12:43.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.054 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021794 s, 48.1 MB/s 2024-04-25T09:12:43.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.079 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211084 s, 49.7 MB/s 2024-04-25T09:12:43.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204366 s, 51.3 MB/s 2024-04-25T09:12:43.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.128 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209844 s, 50.0 MB/s 2024-04-25T09:12:43.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.154 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215104 s, 48.7 MB/s 2024-04-25T09:12:43.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.181 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235139 s, 44.6 MB/s 2024-04-25T09:12:43.206 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.206 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.207 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215063 s, 48.8 MB/s 2024-04-25T09:12:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194668 s, 53.9 MB/s 2024-04-25T09:12:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238312 s, 44.0 MB/s 2024-04-25T09:12:43.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.283 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212331 s, 49.4 MB/s 2024-04-25T09:12:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.314 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254368 s, 41.2 MB/s 2024-04-25T09:12:43.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.341 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230553 s, 45.5 MB/s 2024-04-25T09:12:43.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.367 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222017 s, 47.2 MB/s 2024-04-25T09:12:43.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.395 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244261 s, 42.9 MB/s 2024-04-25T09:12:43.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250088 s, 41.9 MB/s 2024-04-25T09:12:43.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.450 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221583 s, 47.3 MB/s 2024-04-25T09:12:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232353 s, 45.1 MB/s 2024-04-25T09:12:43.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.503 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211742 s, 49.5 MB/s 2024-04-25T09:12:43.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.532 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025604 s, 41.0 MB/s 2024-04-25T09:12:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.554 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183599 s, 57.1 MB/s 2024-04-25T09:12:43.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209034 s, 50.2 MB/s 2024-04-25T09:12:43.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.607 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242987 s, 43.2 MB/s 2024-04-25T09:12:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.632 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208597 s, 50.3 MB/s 2024-04-25T09:12:43.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.656 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198094 s, 52.9 MB/s 2024-04-25T09:12:43.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.683 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227671 s, 46.1 MB/s 2024-04-25T09:12:43.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.709 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213486 s, 49.1 MB/s 2024-04-25T09:12:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200299 s, 52.4 MB/s 2024-04-25T09:12:43.762 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.763 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253032 s, 41.4 MB/s 2024-04-25T09:12:43.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.788 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213566 s, 49.1 MB/s 2024-04-25T09:12:43.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:43.791 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:43.791 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:43.791 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:12:43.791 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:43.791 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:43.791 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:43.791 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:43.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.883 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253129 s, 41.4 MB/s 2024-04-25T09:12:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226063 s, 46.4 MB/s 2024-04-25T09:12:43.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.936 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240276 s, 43.6 MB/s 2024-04-25T09:12:43.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197449 s, 53.1 MB/s 2024-04-25T09:12:43.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:43.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:43.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227599 s, 46.1 MB/s 2024-04-25T09:12:44.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.011 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202989 s, 51.7 MB/s 2024-04-25T09:12:44.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.035 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197301 s, 53.1 MB/s 2024-04-25T09:12:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.060 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205746 s, 51.0 MB/s 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.082 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186398 s, 56.3 MB/s 2024-04-25T09:12:44.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.109 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217987 s, 48.1 MB/s 2024-04-25T09:12:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.137 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247412 s, 42.4 MB/s 2024-04-25T09:12:44.162 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.163 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216026 s, 48.5 MB/s 2024-04-25T09:12:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.188 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211024 s, 49.7 MB/s 2024-04-25T09:12:44.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.215 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241139 s, 43.5 MB/s 2024-04-25T09:12:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.242 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224512 s, 46.7 MB/s 2024-04-25T09:12:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.268 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218612 s, 48.0 MB/s 2024-04-25T09:12:44.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.294 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221575 s, 47.3 MB/s 2024-04-25T09:12:44.310 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:44.317 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:12:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020224 s, 51.8 MB/s 2024-04-25T09:12:44.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.343 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200374 s, 52.3 MB/s 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.363 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176777 s, 59.3 MB/s 2024-04-25T09:12:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.379 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138958 s, 75.5 MB/s 2024-04-25T09:12:44.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.400 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179041 s, 58.6 MB/s 2024-04-25T09:12:44.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202227 s, 51.9 MB/s 2024-04-25T09:12:44.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.446 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196408 s, 53.4 MB/s 2024-04-25T09:12:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.472 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215315 s, 48.7 MB/s 2024-04-25T09:12:44.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.495 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200688 s, 52.2 MB/s 2024-04-25T09:12:44.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.523 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025123 s, 41.7 MB/s 2024-04-25T09:12:44.551 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.551 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.551 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023023 s, 45.5 MB/s 2024-04-25T09:12:44.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.586 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02941 s, 35.7 MB/s 2024-04-25T09:12:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.621 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311002 s, 33.7 MB/s 2024-04-25T09:12:44.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.648 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224829 s, 46.6 MB/s 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017385 s, 60.3 MB/s 2024-04-25T09:12:44.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:44.671 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:44.671 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:44.672 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:12:44.672 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:44.672 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:44.672 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:44.672 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:44.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239353 s, 43.8 MB/s 2024-04-25T09:12:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.832 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214958 s, 48.8 MB/s 2024-04-25T09:12:44.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.858 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216856 s, 48.4 MB/s 2024-04-25T09:12:44.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.886 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233305 s, 44.9 MB/s 2024-04-25T09:12:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.909 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194455 s, 53.9 MB/s 2024-04-25T09:12:44.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236598 s, 44.3 MB/s 2024-04-25T09:12:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230155 s, 45.6 MB/s 2024-04-25T09:12:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:44.995 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255464 s, 41.0 MB/s 2024-04-25T09:12:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.022 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022958 s, 45.7 MB/s 2024-04-25T09:12:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224328 s, 46.7 MB/s 2024-04-25T09:12:45.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.074 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221446 s, 47.4 MB/s 2024-04-25T09:12:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.101 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223849 s, 46.8 MB/s 2024-04-25T09:12:45.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.126 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214669 s, 48.8 MB/s 2024-04-25T09:12:45.154 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.154 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.154 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023048 s, 45.5 MB/s 2024-04-25T09:12:45.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.176 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189059 s, 55.5 MB/s 2024-04-25T09:12:45.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.199 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019813 s, 52.9 MB/s 2024-04-25T09:12:45.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.230 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02579 s, 40.7 MB/s 2024-04-25T09:12:45.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.256 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217734 s, 48.2 MB/s 2024-04-25T09:12:45.281 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.281 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.281 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205347 s, 51.1 MB/s 2024-04-25T09:12:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.308 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223705 s, 46.9 MB/s 2024-04-25T09:12:45.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.338 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253966 s, 41.3 MB/s 2024-04-25T09:12:45.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.363 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209258 s, 50.1 MB/s 2024-04-25T09:12:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.390 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227284 s, 46.1 MB/s 2024-04-25T09:12:45.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.419 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245942 s, 42.6 MB/s 2024-04-25T09:12:45.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.454 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309692 s, 33.9 MB/s 2024-04-25T09:12:45.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.481 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231583 s, 45.3 MB/s 2024-04-25T09:12:45.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.509 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235698 s, 44.5 MB/s 2024-04-25T09:12:45.533 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.534 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216566 s, 48.4 MB/s 2024-04-25T09:12:45.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.559 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020947 s, 50.1 MB/s 2024-04-25T09:12:45.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.585 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220151 s, 47.6 MB/s 2024-04-25T09:12:45.608 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.608 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.608 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203087 s, 51.6 MB/s 2024-04-25T09:12:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.637 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248772 s, 42.2 MB/s 2024-04-25T09:12:45.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:45.640 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:45.640 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:45.641 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:12:45.641 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:45.641 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:45.641 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:45.641 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:45.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191619 s, 54.7 MB/s 2024-04-25T09:12:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199802 s, 52.5 MB/s 2024-04-25T09:12:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.773 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227358 s, 46.1 MB/s 2024-04-25T09:12:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211847 s, 49.5 MB/s 2024-04-25T09:12:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232331 s, 45.1 MB/s 2024-04-25T09:12:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.857 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270552 s, 38.8 MB/s 2024-04-25T09:12:45.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.884 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222333 s, 47.2 MB/s 2024-04-25T09:12:45.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.913 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246138 s, 42.6 MB/s 2024-04-25T09:12:45.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.938 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206347 s, 50.8 MB/s 2024-04-25T09:12:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019312 s, 54.3 MB/s 2024-04-25T09:12:45.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:45.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:45.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222964 s, 47.0 MB/s 2024-04-25T09:12:46.017 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.017 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.017 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025249 s, 41.5 MB/s 2024-04-25T09:12:46.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.044 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217945 s, 48.1 MB/s 2024-04-25T09:12:46.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.071 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228473 s, 45.9 MB/s 2024-04-25T09:12:46.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.099 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234218 s, 44.8 MB/s 2024-04-25T09:12:46.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212513 s, 49.3 MB/s 2024-04-25T09:12:46.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.152 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.152 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236871 s, 44.3 MB/s 2024-04-25T09:12:46.177 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.177 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.177 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208582 s, 50.3 MB/s 2024-04-25T09:12:46.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.199 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018214 s, 57.6 MB/s 2024-04-25T09:12:46.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.226 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231463 s, 45.3 MB/s 2024-04-25T09:12:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204825 s, 51.2 MB/s 2024-04-25T09:12:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218882 s, 47.9 MB/s 2024-04-25T09:12:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.309 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273511 s, 38.3 MB/s 2024-04-25T09:12:46.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226014 s, 46.4 MB/s 2024-04-25T09:12:46.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.357 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187586 s, 55.9 MB/s 2024-04-25T09:12:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248267 s, 42.2 MB/s 2024-04-25T09:12:46.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.413 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229094 s, 45.8 MB/s 2024-04-25T09:12:46.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.442 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246018 s, 42.6 MB/s 2024-04-25T09:12:46.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.468 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220761 s, 47.5 MB/s 2024-04-25T09:12:46.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.495 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022131 s, 47.4 MB/s 2024-04-25T09:12:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.520 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216197 s, 48.5 MB/s 2024-04-25T09:12:46.547 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.547 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.547 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230683 s, 45.5 MB/s 2024-04-25T09:12:46.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:46.550 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:46.550 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:46.550 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:12:46.550 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:46.550 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:46.550 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:46.550 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229652 s, 45.7 MB/s 2024-04-25T09:12:46.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.665 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022842 s, 45.9 MB/s 2024-04-25T09:12:46.692 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.692 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.692 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228559 s, 45.9 MB/s 2024-04-25T09:12:46.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.720 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235954 s, 44.4 MB/s 2024-04-25T09:12:46.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226589 s, 46.3 MB/s 2024-04-25T09:12:46.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.771 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020669 s, 50.7 MB/s 2024-04-25T09:12:46.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228698 s, 45.8 MB/s 2024-04-25T09:12:46.824 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.825 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224447 s, 46.7 MB/s 2024-04-25T09:12:46.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262706 s, 39.9 MB/s 2024-04-25T09:12:46.880 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.973 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218466 s, 48.0 MB/s 2024-04-25T09:12:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211395 s, 49.6 MB/s 2024-04-25T09:12:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221496 s, 47.3 MB/s 2024-04-25T09:12:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246792 s, 42.5 MB/s 2024-04-25T09:12:46.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:46.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:46.986 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221019 s, 47.4 MB/s 2024-04-25T09:12:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.011 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201935 s, 51.9 MB/s 2024-04-25T09:12:47.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228544 s, 45.9 MB/s 2024-04-25T09:12:47.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.065 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226975 s, 46.2 MB/s 2024-04-25T09:12:47.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.093 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232547 s, 45.1 MB/s 2024-04-25T09:12:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.119 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219631 s, 47.7 MB/s 2024-04-25T09:12:47.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.143 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208997 s, 50.2 MB/s 2024-04-25T09:12:47.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.164 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017723 s, 59.2 MB/s 2024-04-25T09:12:47.189 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.189 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.189 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210093 s, 49.9 MB/s 2024-04-25T09:12:47.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.220 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026657 s, 39.3 MB/s 2024-04-25T09:12:47.244 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.245 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205384 s, 51.1 MB/s 2024-04-25T09:12:47.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.278 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293629 s, 35.7 MB/s 2024-04-25T09:12:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239538 s, 43.8 MB/s 2024-04-25T09:12:47.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.339 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280989 s, 37.3 MB/s 2024-04-25T09:12:47.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.371 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259975 s, 40.3 MB/s 2024-04-25T09:12:47.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.393 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192228 s, 54.5 MB/s 2024-04-25T09:12:47.417 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.418 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.418 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207491 s, 50.5 MB/s 2024-04-25T09:12:47.444 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233132 s, 45.0 MB/s 2024-04-25T09:12:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.469 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195023 s, 53.8 MB/s 2024-04-25T09:12:47.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:47.472 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:47.472 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:47.473 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:12:47.473 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:47.473 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:47.473 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:47.473 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.563 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238213 s, 44.0 MB/s 2024-04-25T09:12:47.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.593 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270938 s, 38.7 MB/s 2024-04-25T09:12:47.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216425 s, 48.4 MB/s 2024-04-25T09:12:47.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.648 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252302 s, 41.6 MB/s 2024-04-25T09:12:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.674 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218359 s, 48.0 MB/s 2024-04-25T09:12:47.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.701 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222861 s, 47.1 MB/s 2024-04-25T09:12:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.726 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223083 s, 47.0 MB/s 2024-04-25T09:12:47.752 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.752 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.752 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209882 s, 50.0 MB/s 2024-04-25T09:12:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211196 s, 49.6 MB/s 2024-04-25T09:12:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.804 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220411 s, 47.6 MB/s 2024-04-25T09:12:47.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.830 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222216 s, 47.2 MB/s 2024-04-25T09:12:47.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.858 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230114 s, 45.6 MB/s 2024-04-25T09:12:47.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.883 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212324 s, 49.4 MB/s 2024-04-25T09:12:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.907 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203448 s, 51.5 MB/s 2024-04-25T09:12:47.929 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.929 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.929 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193294 s, 54.2 MB/s 2024-04-25T09:12:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219028 s, 47.9 MB/s 2024-04-25T09:12:47.979 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:47.979 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:47.979 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194888 s, 53.8 MB/s 2024-04-25T09:12:48.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.008 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225667 s, 46.5 MB/s 2024-04-25T09:12:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170958 s, 61.3 MB/s 2024-04-25T09:12:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169912 s, 61.7 MB/s 2024-04-25T09:12:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215844 s, 48.6 MB/s 2024-04-25T09:12:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.100 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213649 s, 49.1 MB/s 2024-04-25T09:12:48.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.126 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215019 s, 48.8 MB/s 2024-04-25T09:12:48.152 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.152 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.152 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228827 s, 45.8 MB/s 2024-04-25T09:12:48.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.179 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213247 s, 49.2 MB/s 2024-04-25T09:12:48.206 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.207 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243024 s, 43.1 MB/s 2024-04-25T09:12:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.230 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193743 s, 54.1 MB/s 2024-04-25T09:12:48.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.251 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186385 s, 56.3 MB/s 2024-04-25T09:12:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.278 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231905 s, 45.2 MB/s 2024-04-25T09:12:48.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.305 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230114 s, 45.6 MB/s 2024-04-25T09:12:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.333 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228183 s, 46.0 MB/s 2024-04-25T09:12:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.361 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236072 s, 44.4 MB/s 2024-04-25T09:12:48.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:48.364 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:48.364 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:48.364 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:12:48.364 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:48.364 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:48.364 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:48.364 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249139 s, 42.1 MB/s 2024-04-25T09:12:48.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.478 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217353 s, 48.2 MB/s 2024-04-25T09:12:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.505 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023425 s, 44.8 MB/s 2024-04-25T09:12:48.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.532 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238562 s, 44.0 MB/s 2024-04-25T09:12:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241901 s, 43.3 MB/s 2024-04-25T09:12:48.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.591 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027018 s, 38.8 MB/s 2024-04-25T09:12:48.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.617 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022122 s, 47.4 MB/s 2024-04-25T09:12:48.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.646 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248983 s, 42.1 MB/s 2024-04-25T09:12:48.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.671 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207746 s, 50.5 MB/s 2024-04-25T09:12:48.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.696 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217361 s, 48.2 MB/s 2024-04-25T09:12:48.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.720 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201049 s, 52.2 MB/s 2024-04-25T09:12:48.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.746 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206316 s, 50.8 MB/s 2024-04-25T09:12:48.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.772 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211765 s, 49.5 MB/s 2024-04-25T09:12:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177724 s, 59.0 MB/s 2024-04-25T09:12:48.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.819 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227628 s, 46.1 MB/s 2024-04-25T09:12:48.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.847 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223768 s, 46.9 MB/s 2024-04-25T09:12:48.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.871 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204894 s, 51.2 MB/s 2024-04-25T09:12:48.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215008 s, 48.8 MB/s 2024-04-25T09:12:48.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.924 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219076 s, 47.9 MB/s 2024-04-25T09:12:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.952 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247281 s, 42.4 MB/s 2024-04-25T09:12:48.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:48.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:48.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023462 s, 44.7 MB/s 2024-04-25T09:12:49.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.004 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198836 s, 52.7 MB/s 2024-04-25T09:12:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.029 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214018 s, 49.0 MB/s 2024-04-25T09:12:49.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.057 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236658 s, 44.3 MB/s 2024-04-25T09:12:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218568 s, 48.0 MB/s 2024-04-25T09:12:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219984 s, 47.7 MB/s 2024-04-25T09:12:49.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.134 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209282 s, 50.1 MB/s 2024-04-25T09:12:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.157 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185956 s, 56.4 MB/s 2024-04-25T09:12:49.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.187 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259577 s, 40.4 MB/s 2024-04-25T09:12:49.212 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.212 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.212 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020762 s, 50.5 MB/s 2024-04-25T09:12:49.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.240 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233718 s, 44.9 MB/s 2024-04-25T09:12:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.270 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.271 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256569 s, 40.9 MB/s 2024-04-25T09:12:49.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:49.273 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:49.274 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:49.274 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:12:49.274 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:49.274 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:49.274 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:49.274 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.367 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289906 s, 36.2 MB/s 2024-04-25T09:12:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.393 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022258 s, 47.1 MB/s 2024-04-25T09:12:49.421 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.421 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.421 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024487 s, 42.8 MB/s 2024-04-25T09:12:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.446 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207998 s, 50.4 MB/s 2024-04-25T09:12:49.475 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.476 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.476 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023784 s, 44.1 MB/s 2024-04-25T09:12:49.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.502 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227882 s, 46.0 MB/s 2024-04-25T09:12:49.525 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.525 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.525 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192723 s, 54.4 MB/s 2024-04-25T09:12:49.552 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.552 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.552 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237184 s, 44.2 MB/s 2024-04-25T09:12:49.580 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.581 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237222 s, 44.2 MB/s 2024-04-25T09:12:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.605 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198774 s, 52.8 MB/s 2024-04-25T09:12:49.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.630 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207945 s, 50.4 MB/s 2024-04-25T09:12:49.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.652 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187681 s, 55.9 MB/s 2024-04-25T09:12:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.680 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240424 s, 43.6 MB/s 2024-04-25T09:12:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.702 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188385 s, 55.7 MB/s 2024-04-25T09:12:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.730 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.731 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237279 s, 44.2 MB/s 2024-04-25T09:12:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.755 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020664 s, 50.7 MB/s 2024-04-25T09:12:49.777 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.777 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.778 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196026 s, 53.5 MB/s 2024-04-25T09:12:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210838 s, 49.7 MB/s 2024-04-25T09:12:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.825 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175053 s, 59.9 MB/s 2024-04-25T09:12:49.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.852 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219141 s, 47.8 MB/s 2024-04-25T09:12:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.878 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225419 s, 46.5 MB/s 2024-04-25T09:12:49.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.905 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225787 s, 46.4 MB/s 2024-04-25T09:12:49.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.935 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026101 s, 40.2 MB/s 2024-04-25T09:12:49.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.963 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237184 s, 44.2 MB/s 2024-04-25T09:12:49.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:49.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:49.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236645 s, 44.3 MB/s 2024-04-25T09:12:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.024 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273299 s, 38.4 MB/s 2024-04-25T09:12:50.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.050 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228629 s, 45.9 MB/s 2024-04-25T09:12:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.076 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208992 s, 50.2 MB/s 2024-04-25T09:12:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.105 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258846 s, 40.5 MB/s 2024-04-25T09:12:50.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.132 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022655 s, 46.3 MB/s 2024-04-25T09:12:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220887 s, 47.5 MB/s 2024-04-25T09:12:50.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.184 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212523 s, 49.3 MB/s 2024-04-25T09:12:50.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:50.187 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:50.187 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:50.187 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:12:50.187 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:50.187 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:50.187 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:50.187 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020021 s, 52.4 MB/s 2024-04-25T09:12:50.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.302 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250679 s, 41.8 MB/s 2024-04-25T09:12:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.326 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194524 s, 53.9 MB/s 2024-04-25T09:12:50.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223328 s, 47.0 MB/s 2024-04-25T09:12:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224039 s, 46.8 MB/s 2024-04-25T09:12:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215908 s, 48.6 MB/s 2024-04-25T09:12:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249459 s, 42.0 MB/s 2024-04-25T09:12:50.467 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.467 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.467 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279296 s, 37.5 MB/s 2024-04-25T09:12:50.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.492 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020465 s, 51.2 MB/s 2024-04-25T09:12:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025612 s, 40.9 MB/s 2024-04-25T09:12:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227169 s, 46.2 MB/s 2024-04-25T09:12:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252792 s, 41.5 MB/s 2024-04-25T09:12:50.608 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.608 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.608 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259557 s, 40.4 MB/s 2024-04-25T09:12:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248519 s, 42.2 MB/s 2024-04-25T09:12:50.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.667 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256889 s, 40.8 MB/s 2024-04-25T09:12:50.698 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275519 s, 38.1 MB/s 2024-04-25T09:12:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024046 s, 43.6 MB/s 2024-04-25T09:12:50.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025307 s, 41.4 MB/s 2024-04-25T09:12:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.784 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223806 s, 46.9 MB/s 2024-04-25T09:12:50.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226323 s, 46.3 MB/s 2024-04-25T09:12:50.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.838 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229081 s, 45.8 MB/s 2024-04-25T09:12:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.865 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237568 s, 44.1 MB/s 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264934 s, 39.6 MB/s 2024-04-25T09:12:50.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.923 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224747 s, 46.7 MB/s 2024-04-25T09:12:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214157 s, 49.0 MB/s 2024-04-25T09:12:50.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.974 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212666 s, 49.3 MB/s 2024-04-25T09:12:50.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:50.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:50.999 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214712 s, 48.8 MB/s 2024-04-25T09:12:51.026 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.026 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.026 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244309 s, 42.9 MB/s 2024-04-25T09:12:51.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.054 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237296 s, 44.2 MB/s 2024-04-25T09:12:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206127 s, 50.9 MB/s 2024-04-25T09:12:51.102 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.102 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.102 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019862 s, 52.8 MB/s 2024-04-25T09:12:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02144 s, 48.9 MB/s 2024-04-25T09:12:51.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:51.129 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:51.129 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:51.129 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:12:51.129 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:51.129 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:51.129 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:51.129 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250575 s, 41.8 MB/s 2024-04-25T09:12:51.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.246 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201156 s, 52.1 MB/s 2024-04-25T09:12:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.268 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193694 s, 54.1 MB/s 2024-04-25T09:12:51.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.295 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223271 s, 47.0 MB/s 2024-04-25T09:12:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225778 s, 46.4 MB/s 2024-04-25T09:12:51.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.350 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239789 s, 43.7 MB/s 2024-04-25T09:12:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259153 s, 40.5 MB/s 2024-04-25T09:12:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250473 s, 41.9 MB/s 2024-04-25T09:12:51.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221701 s, 47.3 MB/s 2024-04-25T09:12:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209108 s, 50.1 MB/s 2024-04-25T09:12:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232837 s, 45.0 MB/s 2024-04-25T09:12:51.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290478 s, 36.1 MB/s 2024-04-25T09:12:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243008 s, 43.1 MB/s 2024-04-25T09:12:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.581 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02741 s, 38.3 MB/s 2024-04-25T09:12:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.609 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239847 s, 43.7 MB/s 2024-04-25T09:12:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227417 s, 46.1 MB/s 2024-04-25T09:12:51.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.663 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227186 s, 46.2 MB/s 2024-04-25T09:12:51.692 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.692 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.692 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259967 s, 40.3 MB/s 2024-04-25T09:12:51.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.720 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231659 s, 45.3 MB/s 2024-04-25T09:12:51.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.746 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214178 s, 49.0 MB/s 2024-04-25T09:12:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205771 s, 51.0 MB/s 2024-04-25T09:12:51.797 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.797 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.797 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219996 s, 47.7 MB/s 2024-04-25T09:12:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.825 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240912 s, 43.5 MB/s 2024-04-25T09:12:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208382 s, 50.3 MB/s 2024-04-25T09:12:51.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.880 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.880 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253738 s, 41.3 MB/s 2024-04-25T09:12:51.904 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.905 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209822 s, 50.0 MB/s 2024-04-25T09:12:51.931 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.931 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.932 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210696 s, 49.8 MB/s 2024-04-25T09:12:51.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222771 s, 47.1 MB/s 2024-04-25T09:12:51.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:51.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:51.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284119 s, 36.9 MB/s 2024-04-25T09:12:52.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.021 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241488 s, 43.4 MB/s 2024-04-25T09:12:52.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.050 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245593 s, 42.7 MB/s 2024-04-25T09:12:52.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206152 s, 50.9 MB/s 2024-04-25T09:12:52.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:52.078 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:52.078 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:52.078 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:12:52.078 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:52.078 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:52.078 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:52.078 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:52.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.165 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022678 s, 46.2 MB/s 2024-04-25T09:12:52.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.191 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224304 s, 46.7 MB/s 2024-04-25T09:12:52.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.215 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205978 s, 50.9 MB/s 2024-04-25T09:12:52.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.242 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230317 s, 45.5 MB/s 2024-04-25T09:12:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.267 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207445 s, 50.5 MB/s 2024-04-25T09:12:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.290 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019987 s, 52.5 MB/s 2024-04-25T09:12:52.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.315 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.315 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207828 s, 50.5 MB/s 2024-04-25T09:12:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.347 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263219 s, 39.8 MB/s 2024-04-25T09:12:52.375 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.375 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.376 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242761 s, 43.2 MB/s 2024-04-25T09:12:52.401 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.401 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.401 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212333 s, 49.4 MB/s 2024-04-25T09:12:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.429 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227527 s, 46.1 MB/s 2024-04-25T09:12:52.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.458 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245994 s, 42.6 MB/s 2024-04-25T09:12:52.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.483 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214602 s, 48.9 MB/s 2024-04-25T09:12:52.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.514 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261924 s, 40.0 MB/s 2024-04-25T09:12:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.545 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273274 s, 38.4 MB/s 2024-04-25T09:12:52.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.574 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241362 s, 43.4 MB/s 2024-04-25T09:12:52.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.600 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228211 s, 45.9 MB/s 2024-04-25T09:12:52.627 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.627 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.627 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225814 s, 46.4 MB/s 2024-04-25T09:12:52.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.653 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214496 s, 48.9 MB/s 2024-04-25T09:12:52.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.683 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255594 s, 41.0 MB/s 2024-04-25T09:12:52.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.709 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220602 s, 47.5 MB/s 2024-04-25T09:12:52.734 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.735 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020588 s, 50.9 MB/s 2024-04-25T09:12:52.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.760 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213956 s, 49.0 MB/s 2024-04-25T09:12:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.788 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235683 s, 44.5 MB/s 2024-04-25T09:12:52.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.816 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241487 s, 43.4 MB/s 2024-04-25T09:12:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219751 s, 47.7 MB/s 2024-04-25T09:12:52.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216858 s, 48.4 MB/s 2024-04-25T09:12:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.894 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211083 s, 49.7 MB/s 2024-04-25T09:12:52.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.920 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215893 s, 48.6 MB/s 2024-04-25T09:12:52.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276452 s, 37.9 MB/s 2024-04-25T09:12:52.982 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:52.982 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:52.982 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265569 s, 39.5 MB/s 2024-04-25T09:12:53.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.010 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250333 s, 41.9 MB/s 2024-04-25T09:12:53.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:53.013 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:53.013 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:53.013 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:12:53.013 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:53.013 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:53.013 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:53.013 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:53.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.095 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212024 s, 49.5 MB/s 2024-04-25T09:12:53.113 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165472 s, 63.4 MB/s 2024-04-25T09:12:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.139 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210996 s, 49.7 MB/s 2024-04-25T09:12:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.164 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207533 s, 50.5 MB/s 2024-04-25T09:12:53.186 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.186 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.186 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179918 s, 58.3 MB/s 2024-04-25T09:12:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225933 s, 46.4 MB/s 2024-04-25T09:12:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.238 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205636 s, 51.0 MB/s 2024-04-25T09:12:53.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.263 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214067 s, 49.0 MB/s 2024-04-25T09:12:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227801 s, 46.0 MB/s 2024-04-25T09:12:53.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.316 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211779 s, 49.5 MB/s 2024-04-25T09:12:53.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.343 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225906 s, 46.4 MB/s 2024-04-25T09:12:53.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.372 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249698 s, 42.0 MB/s 2024-04-25T09:12:53.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.396 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208416 s, 50.3 MB/s 2024-04-25T09:12:53.427 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.427 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.427 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270123 s, 38.8 MB/s 2024-04-25T09:12:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.449 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176754 s, 59.3 MB/s 2024-04-25T09:12:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217341 s, 48.2 MB/s 2024-04-25T09:12:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224014 s, 46.8 MB/s 2024-04-25T09:12:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228015 s, 46.0 MB/s 2024-04-25T09:12:53.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.558 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022476 s, 46.7 MB/s 2024-04-25T09:12:53.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.585 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234031 s, 44.8 MB/s 2024-04-25T09:12:53.613 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.613 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.613 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229615 s, 45.7 MB/s 2024-04-25T09:12:53.635 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.636 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186528 s, 56.2 MB/s 2024-04-25T09:12:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189569 s, 55.3 MB/s 2024-04-25T09:12:53.684 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.684 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.684 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213879 s, 49.0 MB/s 2024-04-25T09:12:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.707 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198667 s, 52.8 MB/s 2024-04-25T09:12:53.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.733 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213673 s, 49.1 MB/s 2024-04-25T09:12:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.761 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233618 s, 44.9 MB/s 2024-04-25T09:12:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.790 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242762 s, 43.2 MB/s 2024-04-25T09:12:53.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.815 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218993 s, 47.9 MB/s 2024-04-25T09:12:53.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.842 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230215 s, 45.5 MB/s 2024-04-25T09:12:53.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227387 s, 46.1 MB/s 2024-04-25T09:12:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225316 s, 46.5 MB/s 2024-04-25T09:12:53.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:53.900 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:53.901 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:53.901 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:12:53.901 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:53.901 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:53.901 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:53.901 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:53.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:53.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:53.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230624 s, 45.5 MB/s 2024-04-25T09:12:54.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.023 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273768 s, 38.3 MB/s 2024-04-25T09:12:54.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.051 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244622 s, 42.9 MB/s 2024-04-25T09:12:54.080 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.080 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.080 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246202 s, 42.6 MB/s 2024-04-25T09:12:54.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.106 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225551 s, 46.5 MB/s 2024-04-25T09:12:54.129 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.129 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.129 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179502 s, 58.4 MB/s 2024-04-25T09:12:54.160 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.160 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.160 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249369 s, 42.0 MB/s 2024-04-25T09:12:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.187 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235095 s, 44.6 MB/s 2024-04-25T09:12:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.213 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215214 s, 48.7 MB/s 2024-04-25T09:12:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.240 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227939 s, 46.0 MB/s 2024-04-25T09:12:54.266 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.266 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.266 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224796 s, 46.6 MB/s 2024-04-25T09:12:54.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.291 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208496 s, 50.3 MB/s 2024-04-25T09:12:54.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226639 s, 46.3 MB/s 2024-04-25T09:12:54.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.342 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205379 s, 51.1 MB/s 2024-04-25T09:12:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.366 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207976 s, 50.4 MB/s 2024-04-25T09:12:54.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.394 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023214 s, 45.2 MB/s 2024-04-25T09:12:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248632 s, 42.2 MB/s 2024-04-25T09:12:54.451 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.451 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.451 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248919 s, 42.1 MB/s 2024-04-25T09:12:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.481 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254423 s, 41.2 MB/s 2024-04-25T09:12:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.504 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185254 s, 56.6 MB/s 2024-04-25T09:12:54.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.536 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028224 s, 37.2 MB/s 2024-04-25T09:12:54.561 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.561 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.561 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208219 s, 50.4 MB/s 2024-04-25T09:12:54.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.587 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212418 s, 49.4 MB/s 2024-04-25T09:12:54.613 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.613 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.613 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228493 s, 45.9 MB/s 2024-04-25T09:12:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.642 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250228 s, 41.9 MB/s 2024-04-25T09:12:54.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212776 s, 49.3 MB/s 2024-04-25T09:12:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.696 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231695 s, 45.3 MB/s 2024-04-25T09:12:54.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.722 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218612 s, 48.0 MB/s 2024-04-25T09:12:54.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215283 s, 48.7 MB/s 2024-04-25T09:12:54.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225579 s, 46.5 MB/s 2024-04-25T09:12:54.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.802 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239169 s, 43.8 MB/s 2024-04-25T09:12:54.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.828 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210097 s, 49.9 MB/s 2024-04-25T09:12:54.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:54.831 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:54.831 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:54.831 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:12:54.831 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:54.831 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:54.831 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:54.831 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209536 s, 50.0 MB/s 2024-04-25T09:12:54.943 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.943 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.943 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257034 s, 40.8 MB/s 2024-04-25T09:12:54.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.970 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227028 s, 46.2 MB/s 2024-04-25T09:12:54.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:54.996 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:54.996 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212856 s, 49.3 MB/s 2024-04-25T09:12:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.021 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216387 s, 48.5 MB/s 2024-04-25T09:12:55.047 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.047 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.047 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022312 s, 47.0 MB/s 2024-04-25T09:12:55.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.074 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224616 s, 46.7 MB/s 2024-04-25T09:12:55.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.099 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205249 s, 51.1 MB/s 2024-04-25T09:12:55.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.127 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235063 s, 44.6 MB/s 2024-04-25T09:12:55.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225919 s, 46.4 MB/s 2024-04-25T09:12:55.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230364 s, 45.5 MB/s 2024-04-25T09:12:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198867 s, 52.7 MB/s 2024-04-25T09:12:55.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.232 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229324 s, 45.7 MB/s 2024-04-25T09:12:55.257 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.257 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.257 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203444 s, 51.5 MB/s 2024-04-25T09:12:55.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.282 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206342 s, 50.8 MB/s 2024-04-25T09:12:55.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.313 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266563 s, 39.3 MB/s 2024-04-25T09:12:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.342 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255425 s, 41.1 MB/s 2024-04-25T09:12:55.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.368 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209555 s, 50.0 MB/s 2024-04-25T09:12:55.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.392 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212595 s, 49.3 MB/s 2024-04-25T09:12:55.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.415 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177397 s, 59.1 MB/s 2024-04-25T09:12:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168266 s, 62.3 MB/s 2024-04-25T09:12:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240231 s, 43.6 MB/s 2024-04-25T09:12:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023651 s, 44.3 MB/s 2024-04-25T09:12:55.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.518 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232274 s, 45.1 MB/s 2024-04-25T09:12:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.544 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022044 s, 47.6 MB/s 2024-04-25T09:12:55.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.569 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215184 s, 48.7 MB/s 2024-04-25T09:12:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.595 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230663 s, 45.5 MB/s 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.622 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209329 s, 50.1 MB/s 2024-04-25T09:12:55.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246198 s, 42.6 MB/s 2024-04-25T09:12:55.678 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.678 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.678 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235474 s, 44.5 MB/s 2024-04-25T09:12:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.700 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190909 s, 54.9 MB/s 2024-04-25T09:12:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.727 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227068 s, 46.2 MB/s 2024-04-25T09:12:55.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:55.730 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:55.730 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:55.730 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:12:55.730 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:55.730 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:55.730 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:55.730 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:55.824 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.824 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.824 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267705 s, 39.2 MB/s 2024-04-25T09:12:55.848 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.848 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.848 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212425 s, 49.4 MB/s 2024-04-25T09:12:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.879 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268612 s, 39.0 MB/s 2024-04-25T09:12:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.906 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230819 s, 45.4 MB/s 2024-04-25T09:12:55.930 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.930 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.930 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212795 s, 49.3 MB/s 2024-04-25T09:12:55.957 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.957 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.957 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022631 s, 46.3 MB/s 2024-04-25T09:12:55.983 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:55.983 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:55.983 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220666 s, 47.5 MB/s 2024-04-25T09:12:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.009 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221101 s, 47.4 MB/s 2024-04-25T09:12:56.039 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.039 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.039 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243121 s, 43.1 MB/s 2024-04-25T09:12:56.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.065 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218572 s, 48.0 MB/s 2024-04-25T09:12:56.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193739 s, 54.1 MB/s 2024-04-25T09:12:56.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.117 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248006 s, 42.3 MB/s 2024-04-25T09:12:56.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255301 s, 41.1 MB/s 2024-04-25T09:12:56.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.174 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.174 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216007 s, 48.5 MB/s 2024-04-25T09:12:56.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.203 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249513 s, 42.0 MB/s 2024-04-25T09:12:56.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.241 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324176 s, 32.3 MB/s 2024-04-25T09:12:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.263 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200025 s, 52.4 MB/s 2024-04-25T09:12:56.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.292 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025004 s, 41.9 MB/s 2024-04-25T09:12:56.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208124 s, 50.4 MB/s 2024-04-25T09:12:56.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.342 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199476 s, 52.6 MB/s 2024-04-25T09:12:56.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.367 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021512 s, 48.7 MB/s 2024-04-25T09:12:56.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.392 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209042 s, 50.2 MB/s 2024-04-25T09:12:56.421 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248072 s, 42.3 MB/s 2024-04-25T09:12:56.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.451 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.451 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256904 s, 40.8 MB/s 2024-04-25T09:12:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.479 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243671 s, 43.0 MB/s 2024-04-25T09:12:56.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.506 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222956 s, 47.0 MB/s 2024-04-25T09:12:56.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.531 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210407 s, 49.8 MB/s 2024-04-25T09:12:56.562 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.563 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.563 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272458 s, 38.5 MB/s 2024-04-25T09:12:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.589 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231295 s, 45.3 MB/s 2024-04-25T09:12:56.620 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.620 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.620 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250087 s, 41.9 MB/s 2024-04-25T09:12:56.653 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.653 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.653 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295604 s, 35.5 MB/s 2024-04-25T09:12:56.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.681 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233504 s, 44.9 MB/s 2024-04-25T09:12:56.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:56.683 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:56.684 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:56.684 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:12:56.684 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:56.684 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:56.684 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:56.684 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:56.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.773 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226797 s, 46.2 MB/s 2024-04-25T09:12:56.800 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.800 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.800 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227565 s, 46.1 MB/s 2024-04-25T09:12:56.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.827 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233635 s, 44.9 MB/s 2024-04-25T09:12:56.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.854 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225008 s, 46.6 MB/s 2024-04-25T09:12:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.883 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242834 s, 43.2 MB/s 2024-04-25T09:12:56.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219747 s, 47.7 MB/s 2024-04-25T09:12:56.932 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.932 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.932 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202954 s, 51.7 MB/s 2024-04-25T09:12:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.962 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260655 s, 40.2 MB/s 2024-04-25T09:12:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:56.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208235 s, 50.4 MB/s 2024-04-25T09:12:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214929 s, 48.8 MB/s 2024-04-25T09:12:57.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213932 s, 49.0 MB/s 2024-04-25T09:12:57.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197644 s, 53.1 MB/s 2024-04-25T09:12:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220746 s, 47.5 MB/s 2024-04-25T09:12:57.113 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.113 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.113 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220028 s, 47.7 MB/s 2024-04-25T09:12:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201209 s, 52.1 MB/s 2024-04-25T09:12:57.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.159 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179598 s, 58.4 MB/s 2024-04-25T09:12:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.188 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254907 s, 41.1 MB/s 2024-04-25T09:12:57.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.216 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235889 s, 44.5 MB/s 2024-04-25T09:12:57.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.243 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224174 s, 46.8 MB/s 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.268 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021397 s, 49.0 MB/s 2024-04-25T09:12:57.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.293 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209488 s, 50.1 MB/s 2024-04-25T09:12:57.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205401 s, 51.1 MB/s 2024-04-25T09:12:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.343 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207361 s, 50.6 MB/s 2024-04-25T09:12:57.368 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.368 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.368 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217762 s, 48.2 MB/s 2024-04-25T09:12:57.397 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.397 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.397 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238713 s, 43.9 MB/s 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.425 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022961 s, 45.7 MB/s 2024-04-25T09:12:57.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.459 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028614 s, 36.6 MB/s 2024-04-25T09:12:57.488 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.488 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.488 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245427 s, 42.7 MB/s 2024-04-25T09:12:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.514 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228722 s, 45.8 MB/s 2024-04-25T09:12:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.545 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265919 s, 39.4 MB/s 2024-04-25T09:12:57.575 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.575 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.575 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260374 s, 40.3 MB/s 2024-04-25T09:12:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025622 s, 40.9 MB/s 2024-04-25T09:12:57.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:57.608 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:57.608 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:57.609 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:12:57.609 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:57.609 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:57.609 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:57.609 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.686 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158399 s, 66.2 MB/s 2024-04-25T09:12:57.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.712 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230346 s, 45.5 MB/s 2024-04-25T09:12:57.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221056 s, 47.4 MB/s 2024-04-25T09:12:57.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.763 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224283 s, 46.8 MB/s 2024-04-25T09:12:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.797 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279786 s, 37.5 MB/s 2024-04-25T09:12:57.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.822 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210008 s, 49.9 MB/s 2024-04-25T09:12:57.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.846 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189094 s, 55.5 MB/s 2024-04-25T09:12:57.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185324 s, 56.6 MB/s 2024-04-25T09:12:57.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.892 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193736 s, 54.1 MB/s 2024-04-25T09:12:57.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.917 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198259 s, 52.9 MB/s 2024-04-25T09:12:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241133 s, 43.5 MB/s 2024-04-25T09:12:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.972 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220778 s, 47.5 MB/s 2024-04-25T09:12:57.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:57.996 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208244 s, 50.4 MB/s 2024-04-25T09:12:58.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.020 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213045 s, 49.2 MB/s 2024-04-25T09:12:58.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242535 s, 43.2 MB/s 2024-04-25T09:12:58.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.077 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229278 s, 45.7 MB/s 2024-04-25T09:12:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.101 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197968 s, 53.0 MB/s 2024-04-25T09:12:58.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.125 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201758 s, 52.0 MB/s 2024-04-25T09:12:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186742 s, 56.2 MB/s 2024-04-25T09:12:58.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.174 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.174 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234361 s, 44.7 MB/s 2024-04-25T09:12:58.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233787 s, 44.9 MB/s 2024-04-25T09:12:58.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216377 s, 48.5 MB/s 2024-04-25T09:12:58.257 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.257 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.257 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254896 s, 41.1 MB/s 2024-04-25T09:12:58.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.282 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211441 s, 49.6 MB/s 2024-04-25T09:12:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251268 s, 41.7 MB/s 2024-04-25T09:12:58.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.338 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212394 s, 49.4 MB/s 2024-04-25T09:12:58.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.364 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223472 s, 46.9 MB/s 2024-04-25T09:12:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.390 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022003 s, 47.7 MB/s 2024-04-25T09:12:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207671 s, 50.5 MB/s 2024-04-25T09:12:58.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.440 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020822 s, 50.4 MB/s 2024-04-25T09:12:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.469 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245814 s, 42.7 MB/s 2024-04-25T09:12:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.496 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240898 s, 43.5 MB/s 2024-04-25T09:12:58.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:58.499 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:58.499 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:58.500 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:12:58.500 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:58.500 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:58.500 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:58.500 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:58.582 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.583 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226469 s, 46.3 MB/s 2024-04-25T09:12:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231437 s, 45.3 MB/s 2024-04-25T09:12:58.635 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.635 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.635 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221205 s, 47.4 MB/s 2024-04-25T09:12:58.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.663 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242353 s, 43.3 MB/s 2024-04-25T09:12:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.689 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215395 s, 48.7 MB/s 2024-04-25T09:12:58.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.714 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219176 s, 47.8 MB/s 2024-04-25T09:12:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.742 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233522 s, 44.9 MB/s 2024-04-25T09:12:58.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.766 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201746 s, 52.0 MB/s 2024-04-25T09:12:58.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.792 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221866 s, 47.3 MB/s 2024-04-25T09:12:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238785 s, 43.9 MB/s 2024-04-25T09:12:58.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.846 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211284 s, 49.6 MB/s 2024-04-25T09:12:58.874 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.874 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.874 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238345 s, 44.0 MB/s 2024-04-25T09:12:58.903 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.903 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.903 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246404 s, 42.6 MB/s 2024-04-25T09:12:58.928 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.928 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.928 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218329 s, 48.0 MB/s 2024-04-25T09:12:58.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264833 s, 39.6 MB/s 2024-04-25T09:12:58.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:58.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:58.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271154 s, 38.7 MB/s 2024-04-25T09:12:59.017 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.018 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023158 s, 45.3 MB/s 2024-04-25T09:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252514 s, 41.5 MB/s 2024-04-25T09:12:59.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.077 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256101 s, 40.9 MB/s 2024-04-25T09:12:59.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215841 s, 48.6 MB/s 2024-04-25T09:12:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.131 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236625 s, 44.3 MB/s 2024-04-25T09:12:59.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.156 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194374 s, 53.9 MB/s 2024-04-25T09:12:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020043 s, 52.3 MB/s 2024-04-25T09:12:59.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.204 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205861 s, 50.9 MB/s 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.233 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250963 s, 41.8 MB/s 2024-04-25T09:12:59.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.259 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217916 s, 48.1 MB/s 2024-04-25T09:12:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.286 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198266 s, 52.9 MB/s 2024-04-25T09:12:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.307 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198509 s, 52.8 MB/s 2024-04-25T09:12:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.332 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209408 s, 50.1 MB/s 2024-04-25T09:12:59.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.355 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199903 s, 52.5 MB/s 2024-04-25T09:12:59.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213621 s, 49.1 MB/s 2024-04-25T09:12:59.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.407 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215818 s, 48.6 MB/s 2024-04-25T09:12:59.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:12:59.410 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:12:59.410 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:12:59.410 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:12:59.410 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:12:59.410 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:12:59.410 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:12:59.410 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:12:59.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.509 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313184 s, 33.5 MB/s 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.537 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245303 s, 42.7 MB/s 2024-04-25T09:12:59.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.567 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265347 s, 39.5 MB/s 2024-04-25T09:12:59.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.596 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239526 s, 43.8 MB/s 2024-04-25T09:12:59.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.625 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260571 s, 40.2 MB/s 2024-04-25T09:12:59.653 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.653 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.653 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229442 s, 45.7 MB/s 2024-04-25T09:12:59.678 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.679 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212804 s, 49.3 MB/s 2024-04-25T09:12:59.705 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.706 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234091 s, 44.8 MB/s 2024-04-25T09:12:59.730 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.730 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.730 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204082 s, 51.4 MB/s 2024-04-25T09:12:59.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.755 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204515 s, 51.3 MB/s 2024-04-25T09:12:59.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.780 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.780 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203609 s, 51.5 MB/s 2024-04-25T09:12:59.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02216 s, 47.3 MB/s 2024-04-25T09:12:59.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213595 s, 49.1 MB/s 2024-04-25T09:12:59.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.856 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210858 s, 49.7 MB/s 2024-04-25T09:12:59.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.884 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216432 s, 48.4 MB/s 2024-04-25T09:12:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.911 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240506 s, 43.6 MB/s 2024-04-25T09:12:59.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.939 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242828 s, 43.2 MB/s 2024-04-25T09:12:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192793 s, 54.4 MB/s 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:12:59.989 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229099 s, 45.8 MB/s 2024-04-25T09:13:00.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192455 s, 54.5 MB/s 2024-04-25T09:13:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.038 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219958 s, 47.7 MB/s 2024-04-25T09:13:00.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.066 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236261 s, 44.4 MB/s 2024-04-25T09:13:00.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.094 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231282 s, 45.3 MB/s 2024-04-25T09:13:00.122 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.123 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242966 s, 43.2 MB/s 2024-04-25T09:13:00.149 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.149 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.149 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227642 s, 46.1 MB/s 2024-04-25T09:13:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220396 s, 47.6 MB/s 2024-04-25T09:13:00.200 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210758 s, 49.8 MB/s 2024-04-25T09:13:00.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.228 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229409 s, 45.7 MB/s 2024-04-25T09:13:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.254 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207014 s, 50.7 MB/s 2024-04-25T09:13:00.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.283 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251431 s, 41.7 MB/s 2024-04-25T09:13:00.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.309 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021696 s, 48.3 MB/s 2024-04-25T09:13:00.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.337 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239584 s, 43.8 MB/s 2024-04-25T09:13:00.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:00.339 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:00.340 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:00.340 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:13:00.340 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:00.340 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:00.340 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:00.340 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:00.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218396 s, 48.0 MB/s 2024-04-25T09:13:00.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.454 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251816 s, 41.6 MB/s 2024-04-25T09:13:00.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.482 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246891 s, 42.5 MB/s 2024-04-25T09:13:00.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.515 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250267 s, 41.9 MB/s 2024-04-25T09:13:00.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.541 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024247 s, 43.2 MB/s 2024-04-25T09:13:00.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.569 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233074 s, 45.0 MB/s 2024-04-25T09:13:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.594 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210455 s, 49.8 MB/s 2024-04-25T09:13:00.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.623 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253196 s, 41.4 MB/s 2024-04-25T09:13:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.649 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220225 s, 47.6 MB/s 2024-04-25T09:13:00.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.675 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216431 s, 48.4 MB/s 2024-04-25T09:13:00.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204763 s, 51.2 MB/s 2024-04-25T09:13:00.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.723 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196275 s, 53.4 MB/s 2024-04-25T09:13:00.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.746 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194076 s, 54.0 MB/s 2024-04-25T09:13:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225386 s, 46.5 MB/s 2024-04-25T09:13:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.793 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158073 s, 66.3 MB/s 2024-04-25T09:13:00.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.822 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248313 s, 42.2 MB/s 2024-04-25T09:13:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.850 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023723 s, 44.2 MB/s 2024-04-25T09:13:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.876 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218402 s, 48.0 MB/s 2024-04-25T09:13:00.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.900 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200591 s, 52.3 MB/s 2024-04-25T09:13:00.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212327 s, 49.4 MB/s 2024-04-25T09:13:00.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213684 s, 49.1 MB/s 2024-04-25T09:13:00.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:00.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:00.976 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210392 s, 49.8 MB/s 2024-04-25T09:13:01.002 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.002 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.002 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220049 s, 47.7 MB/s 2024-04-25T09:13:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.028 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203523 s, 51.5 MB/s 2024-04-25T09:13:01.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.054 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226753 s, 46.2 MB/s 2024-04-25T09:13:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.082 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024183 s, 43.4 MB/s 2024-04-25T09:13:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.107 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208178 s, 50.4 MB/s 2024-04-25T09:13:01.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.132 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211618 s, 49.6 MB/s 2024-04-25T09:13:01.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.156 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209329 s, 50.1 MB/s 2024-04-25T09:13:01.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.184 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242058 s, 43.3 MB/s 2024-04-25T09:13:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.215 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265641 s, 39.5 MB/s 2024-04-25T09:13:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:01.238 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180161 s, 58.2 MB/s 2024-04-25T09:13:01.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:01.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:13:01.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.474+0000 7fd187e00640 1 Processor -- start 2024-04-25T09:13:01.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.474+0000 7fd187e00640 1 -- start start 2024-04-25T09:13:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd187e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd188161270 0x7fd1880b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd187e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd188161650 0x7fd1880b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd187e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880d1480 con 0x7fd1880b2980 2024-04-25T09:13:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd187e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1880d3900 con 0x7fd188161650 2024-04-25T09:13:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd187e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1880d11a0 con 0x7fd188161270 2024-04-25T09:13:01.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd17e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd188161650 0x7fd1880b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd17e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd188161650 0x7fd1880b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50200/0 (socket says 172.21.15.62:50200) 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd17e000640 1 -- 172.21.15.62:0/1039262900 learned_addr learned my addr 172.21.15.62:0/1039262900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.476+0000 7fd187400640 1 --1- 172.21.15.62:0/1039262900 >> v1:172.21.15.62:6789/0 conn(0x7fd1880b2980 0x7fd188161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58348/0 (socket says 172.21.15.62:58348) 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.477+0000 7fd186a00640 1 --2- 172.21.15.62:0/1039262900 >> v2:172.21.15.121:3300/0 conn(0x7fd188161270 0x7fd1880b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.477+0000 7fd17e000640 1 -- 172.21.15.62:0/1039262900 >> v2:172.21.15.121:3300/0 conn(0x7fd188161270 msgr2=0x7fd1880b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.477+0000 7fd17e000640 1 --2- 172.21.15.62:0/1039262900 >> v2:172.21.15.121:3300/0 conn(0x7fd188161270 0x7fd1880b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.477+0000 7fd17e000640 1 -- 172.21.15.62:0/1039262900 >> v1:172.21.15.62:6789/0 conn(0x7fd1880b2980 legacy=0x7fd188161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.477+0000 7fd17e000640 1 -- 172.21.15.62:0/1039262900 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880cee80 con 0x7fd188161650 2024-04-25T09:13:01.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.477+0000 7fd186000640 1 -- 172.21.15.62:0/1039262900 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294241150 0 0) 0x7fd1880d1480 con 0x7fd1880b2980 2024-04-25T09:13:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.478+0000 7fd17e000640 1 --2- 172.21.15.62:0/1039262900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161650 0x7fd1880b5aa0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd1880d4b40 tx=0x7fd17002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b32fb02d374a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.478+0000 7fd186000640 1 -- 172.21.15.62:0/1039262900 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170037070 con 0x7fd188161650 2024-04-25T09:13:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.478+0000 7fd186000640 1 -- 172.21.15.62:0/1039262900 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170032440 con 0x7fd188161650 2024-04-25T09:13:01.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.478+0000 7fd186000640 1 -- 172.21.15.62:0/1039262900 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170032810 con 0x7fd188161650 2024-04-25T09:13:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.479+0000 7fd187e00640 1 -- 172.21.15.62:0/1039262900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161650 msgr2=0x7fd1880b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.479+0000 7fd187e00640 1 --2- 172.21.15.62:0/1039262900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161650 0x7fd1880b5aa0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd1880d4b40 tx=0x7fd17002a150 comp rx=0 tx=0).stop 2024-04-25T09:13:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.479+0000 7fd187e00640 1 -- 172.21.15.62:0/1039262900 shutdown_connections 2024-04-25T09:13:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.479+0000 7fd187e00640 1 --2- 172.21.15.62:0/1039262900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161650 0x7fd1880b5aa0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.479+0000 7fd187e00640 1 --2- 172.21.15.62:0/1039262900 >> v2:172.21.15.121:3300/0 conn(0x7fd188161270 0x7fd1880b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.479+0000 7fd187e00640 1 -- 172.21.15.62:0/1039262900 >> 172.21.15.62:0/1039262900 conn(0x7fd188066d10 msgr2=0x7fd188067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.479+0000 7fd187e00640 1 -- 172.21.15.62:0/1039262900 shutdown_connections 2024-04-25T09:13:01.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.479+0000 7fd187e00640 1 -- 172.21.15.62:0/1039262900 wait complete. 2024-04-25T09:13:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.481+0000 7fd187e00640 1 Processor -- start 2024-04-25T09:13:01.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.481+0000 7fd187e00640 1 -- start start 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd187e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1880b2980 0x7fd1880b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd187e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161270 0x7fd1880ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd187e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd188161650 0x7fd18814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd187e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b2ed0 con 0x7fd188161270 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd187e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880d3590 con 0x7fd1880b2980 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd187e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880b3050 con 0x7fd188161650 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd17e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161270 0x7fd1880ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd17e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161270 0x7fd1880ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50204/0 (socket says 172.21.15.62:50204) 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.482+0000 7fd17e000640 1 -- 172.21.15.62:0/3434678260 learned_addr learned my addr 172.21.15.62:0/3434678260 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.483+0000 7fd186a00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1880b2980 0x7fd1880b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.483+0000 7fd17e000640 1 -- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd188161650 msgr2=0x7fd18814eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.483+0000 7fd17e000640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd188161650 0x7fd18814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.483+0000 7fd17e000640 1 -- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1880b2980 msgr2=0x7fd1880b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.483+0000 7fd17e000640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1880b2980 0x7fd1880b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.483+0000 7fd17e000640 1 -- 172.21.15.62:0/3434678260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18815abe0 con 0x7fd188161270 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.483+0000 7fd17e000640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161270 0x7fd1880ce270 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd17002a190 tx=0x7fd1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.484+0000 7fd17f400640 1 -- 172.21.15.62:0/3434678260 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170041070 con 0x7fd188161270 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.484+0000 7fd17f400640 1 -- 172.21.15.62:0/3434678260 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17003c870 con 0x7fd188161270 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.484+0000 7fd17f400640 1 -- 172.21.15.62:0/3434678260 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170037040 con 0x7fd188161270 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.484+0000 7fd187e00640 1 -- 172.21.15.62:0/3434678260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18815aec0 con 0x7fd188161270 2024-04-25T09:13:01.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.484+0000 7fd187e00640 1 -- 172.21.15.62:0/3434678260 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18815b480 con 0x7fd188161270 2024-04-25T09:13:01.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.486+0000 7fd17f400640 1 -- 172.21.15.62:0/3434678260 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd170044070 con 0x7fd188161270 2024-04-25T09:13:01.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.487+0000 7fd17f400640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd16406b4d0 0x7fd16406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.487+0000 7fd17f400640 1 -- 172.21.15.62:0/3434678260 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1700aaff0 con 0x7fd188161270 2024-04-25T09:13:01.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.488+0000 7fd190879740 1 Processor -- start 2024-04-25T09:13:01.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.488+0000 7fd186a00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd16406b4d0 0x7fd16406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.488+0000 7fd190879740 1 -- start start 2024-04-25T09:13:01.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd190879740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91ace0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd190879740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 0x55f91acfdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd190879740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acb5c60 0x55f91acfbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd190879740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ac13f00 con 0x55f91acdff80 2024-04-25T09:13:01.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd190879740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ac18f50 con 0x55f91acb5c60 2024-04-25T09:13:01.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd190879740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f91abc20b0 con 0x55f91acfd820 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd186a00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd16406b4d0 0x7fd16406d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd178006fd0 tx=0x7fd178008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd186a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91ace0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd186a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91ace0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50220/0 (socket says 172.21.15.62:50220) 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.489+0000 7fd186a00640 1 -- 172.21.15.62:0/3462407241 learned_addr learned my addr 172.21.15.62:0/3462407241 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd17e000640 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 0x55f91acfdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd187400640 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acb5c60 0x55f91acfbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd186a00640 1 -- 172.21.15.62:0/3462407241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 msgr2=0x55f91acfdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd186a00640 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 0x55f91acfdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd186a00640 1 -- 172.21.15.62:0/3462407241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acb5c60 msgr2=0x55f91acfbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd186a00640 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acb5c60 0x55f91acfbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd186a00640 1 -- 172.21.15.62:0/3462407241 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91a9f9ba0 con 0x55f91acdff80 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd186a00640 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91ace0360 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd17800fa20 tx=0x7fd178012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03c497e8b0c6027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd177400640 1 -- 172.21.15.62:0/3462407241 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17801b050 con 0x55f91acdff80 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.490+0000 7fd177400640 1 -- 172.21.15.62:0/3462407241 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178025070 con 0x55f91acdff80 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd177400640 1 -- 172.21.15.62:0/3462407241 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17800bea0 con 0x55f91acdff80 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 -- 172.21.15.62:0/3462407241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 msgr2=0x55f91ace0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91ace0360 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd17800fa20 tx=0x7fd178012000 comp rx=0 tx=0).stop 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 -- 172.21.15.62:0/3462407241 shutdown_connections 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acb5c60 0x55f91acfbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 0x55f91acfdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 --2- 172.21.15.62:0/3462407241 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91ace0360 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 -- 172.21.15.62:0/3462407241 >> 172.21.15.62:0/3462407241 conn(0x55f91acb2820 msgr2=0x55f91acfad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 -- 172.21.15.62:0/3462407241 shutdown_connections 2024-04-25T09:13:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.491+0000 7fd190879740 1 -- 172.21.15.62:0/3462407241 wait complete. 2024-04-25T09:13:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.492+0000 7fd190879740 1 -- start start 2024-04-25T09:13:01.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.492+0000 7fd190879740 1 Processor -- start 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.493+0000 7fd190879740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acdff80 0x55f91acf8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.493+0000 7fd190879740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acfd820 0x55f91acf4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.493+0000 7fd190879740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 0x55f91acf2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.493+0000 7fd190879740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91abb7620 con 0x55f91acfd820 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.493+0000 7fd190879740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ac14ff0 con 0x55f91acdff80 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.493+0000 7fd190879740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f91acb6a00 con 0x55f91acf5090 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.493+0000 7fd190879740 10 client.0 _ll_register_callbacks cb 0x55f91a92cf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acfd820 0x55f91acf4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acfd820 0x55f91acf4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50236/0 (socket says 172.21.15.62:50236) 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 -- 172.21.15.62:0/3424296934 learned_addr learned my addr 172.21.15.62:0/3424296934 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd187400640 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 0x55f91acf2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 -- 172.21.15.62:0/3424296934 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 msgr2=0x55f91acf2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 0x55f91acf2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 -- 172.21.15.62:0/3424296934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acdff80 msgr2=0x55f91acf8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd186a00640 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acdff80 0x55f91acf8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acdff80 0x55f91acf8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 -- 172.21.15.62:0/3424296934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91a8ed490 con 0x55f91acfd820 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd17e000640 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acfd820 0x55f91acf4b50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd17002a740 tx=0x7fd170079830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165d9949dc7f85fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.494+0000 7fd16be00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd16be00640 1 -- 172.21.15.62:0/3424296934 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17004b070 con 0x55f91acfd820 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd16be00640 1 -- 172.21.15.62:0/3424296934 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd170079a90 con 0x55f91acfd820 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 1 -- 172.21.15.62:0/3424296934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd178025070 con 0x55f91acfd820 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd16be00640 1 -- 172.21.15.62:0/3424296934 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd170081f40 con 0x55f91acfd820 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 10 client.7509 fetch_fsmap learned FSMap version 64 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 1 -- 172.21.15.62:0/3424296934 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55f91aa5a330 con 0x55f91acfd820 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd16be00640 1 -- 172.21.15.62:0/3424296934 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178025070 con 0x55f91acfd820 2024-04-25T09:13:01.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd16be00640 1 -- 172.21.15.62:0/3424296934 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd170081ab0 con 0x55f91acfd820 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 10 client.7509 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 10 client.7509 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 20 client.7509 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55f91acf6100 0x55f91acf6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 4 client.7509 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.495+0000 7fd190879740 1 -- 172.21.15.62:0/3424296934 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f91ac26490 con 0x55f91acf6100 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.496+0000 7fd186a00640 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55f91acf6100 0x55f91acf6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.496+0000 7fd186a00640 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55f91acf6100 0x55f91acf6570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8646bdef03a6895d server_cookie=efbf1c8018d7850e in_seq=0 out_seq=0 2024-04-25T09:13:01.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.496+0000 7fd16be00640 10 client.7509 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:01.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.497+0000 7fd16be00640 1 -- 172.21.15.62:0/3424296934 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f91ac26490 con 0x55f91acf6100 2024-04-25T09:13:01.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.497+0000 7fd16be00640 10 client.7509 handle_command_reply: tid=0 2024-04-25T09:13:01.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.497+0000 7fd190879740 1 client.7509 shutdown 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.497+0000 7fd190879740 1 -- 172.21.15.62:0/3424296934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acfd820 msgr2=0x55f91acf4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.497+0000 7fd190879740 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acfd820 0x55f91acf4b50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd17002a740 tx=0x7fd170079830 comp rx=0 tx=0).stop 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.497+0000 7fd190879740 1 -- 172.21.15.62:0/3424296934 shutdown_connections 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.497+0000 7fd190879740 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55f91acf6100 0x55f91acf6570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.498+0000 7fd190879740 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acfd820 0x55f91acf4b50 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.498+0000 7fd190879740 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 0x55f91acf2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.498+0000 7fd190879740 1 --2- 172.21.15.62:0/3424296934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acdff80 0x55f91acf8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.498+0000 7fd190879740 1 -- 172.21.15.62:0/3424296934 >> 172.21.15.62:0/3424296934 conn(0x55f91acb5c60 msgr2=0x55f91acfeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.498+0000 7fd16be00640 0 client.7509 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.498+0000 7fd190879740 1 -- 172.21.15.62:0/3424296934 shutdown_connections 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.498+0000 7fd190879740 1 -- 172.21.15.62:0/3424296934 wait complete. 2024-04-25T09:13:01.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.498+0000 7fd190879740 20 client.7509 trim_cache size 0 max 16384 2024-04-25T09:13:01.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.515+0000 7fd190879740 1 Processor -- start 2024-04-25T09:13:01.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.515+0000 7fd190879740 1 -- start start 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd190879740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91acb7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd190879740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 0x55f91acf75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd190879740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acfd820 0x55f91acf8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd190879740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1881ffe50 con 0x55f91acdff80 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd190879740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815f1d0 con 0x55f91acfd820 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd190879740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880cd760 con 0x55f91acf5090 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd186a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91acb7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd186a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91acb7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50248/0 (socket says 172.21.15.62:50248) 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.516+0000 7fd186a00640 1 -- 172.21.15.62:0/3902615968 learned_addr learned my addr 172.21.15.62:0/3902615968 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd17e000640 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 0x55f91acf75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd187400640 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acfd820 0x55f91acf8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd186a00640 1 -- 172.21.15.62:0/3902615968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 msgr2=0x55f91acf75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd186a00640 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 0x55f91acf75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd186a00640 1 -- 172.21.15.62:0/3902615968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acfd820 msgr2=0x55f91acf8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd186a00640 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acfd820 0x55f91acf8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd186a00640 1 -- 172.21.15.62:0/3902615968 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91a8c6610 con 0x55f91acdff80 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd186a00640 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91acb7030 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd17800f9f0 tx=0x7fd178014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1a90d0f7ca96f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd176a00640 1 -- 172.21.15.62:0/3902615968 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178039070 con 0x55f91acdff80 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd176a00640 1 -- 172.21.15.62:0/3902615968 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178025040 con 0x55f91acdff80 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.517+0000 7fd176a00640 1 -- 172.21.15.62:0/3902615968 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17801b050 con 0x55f91acdff80 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 -- 172.21.15.62:0/3902615968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 msgr2=0x55f91acb7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91acb7030 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd17800f9f0 tx=0x7fd178014000 comp rx=0 tx=0).stop 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 -- 172.21.15.62:0/3902615968 shutdown_connections 2024-04-25T09:13:01.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acfd820 0x55f91acf8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acf5090 0x55f91acf75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 --2- 172.21.15.62:0/3902615968 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91acdff80 0x55f91acb7030 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 -- 172.21.15.62:0/3902615968 >> 172.21.15.62:0/3902615968 conn(0x55f91acb5c60 msgr2=0x55f91acf4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 -- 172.21.15.62:0/3902615968 shutdown_connections 2024-04-25T09:13:01.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.518+0000 7fd190879740 1 -- 172.21.15.62:0/3902615968 wait complete. 2024-04-25T09:13:01.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.519+0000 7fd190879740 1 -- start start 2024-04-25T09:13:01.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.519+0000 7fd190879740 1 Processor -- start 2024-04-25T09:13:01.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.520+0000 7fd190879740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acf5090 0x55f91acdcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.520+0000 7fd190879740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 0x55f91ad0cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.520+0000 7fd190879740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91ad53fb0 0x55f91ad0d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.520+0000 7fd190879740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18815ebd0 con 0x55f91ad53fb0 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.520+0000 7fd190879740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ace41d0 con 0x55f91acf5090 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.520+0000 7fd190879740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f91ace3a30 con 0x55f91acfd820 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.520+0000 7fd190879740 10 client.0 _ll_register_callbacks cb 0x55f91acb0c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.520+0000 7fd187400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91ad53fb0 0x55f91ad0d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd187400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91ad53fb0 0x55f91ad0d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50250/0 (socket says 172.21.15.62:50250) 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd187400640 1 -- 172.21.15.62:0/3910373695 learned_addr learned my addr 172.21.15.62:0/3910373695 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd186a00640 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acf5090 0x55f91acdcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd187400640 1 -- 172.21.15.62:0/3910373695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 msgr2=0x55f91ad0cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd17e000640 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 0x55f91ad0cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd187400640 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 0x55f91ad0cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd187400640 1 -- 172.21.15.62:0/3910373695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acf5090 msgr2=0x55f91acdcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd187400640 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acf5090 0x55f91acdcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd187400640 1 -- 172.21.15.62:0/3910373695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91acf16e0 con 0x55f91ad53fb0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd187400640 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91ad53fb0 0x55f91ad0d0a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd180007b30 tx=0x7fd1800018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c216aee9b634fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.521+0000 7fd16be00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd16be00640 1 -- 172.21.15.62:0/3910373695 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180014420 con 0x55f91ad53fb0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd16be00640 1 -- 172.21.15.62:0/3910373695 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd180016720 con 0x55f91ad53fb0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 1 -- 172.21.15.62:0/3910373695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd178025040 con 0x55f91ad53fb0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd16be00640 1 -- 172.21.15.62:0/3910373695 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd180016920 con 0x55f91ad53fb0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 10 client.7515 fetch_fsmap learned FSMap version 64 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 1 -- 172.21.15.62:0/3910373695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55f91acf1090 con 0x55f91ad53fb0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd16be00640 1 -- 172.21.15.62:0/3910373695 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd178025040 con 0x55f91ad53fb0 2024-04-25T09:13:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd16be00640 1 -- 172.21.15.62:0/3910373695 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd180016ac0 con 0x55f91ad53fb0 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 10 client.7515 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 10 client.7515 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 20 client.7515 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55f91ace2c50 0x55f91ace3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 4 client.7515 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.522+0000 7fd190879740 1 -- 172.21.15.62:0/3910373695 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55f91ac207e0 con 0x55f91ace2c50 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.523+0000 7fd186a00640 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55f91ace2c50 0x55f91ace3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.523+0000 7fd186a00640 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55f91ace2c50 0x55f91ace3060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c95f71c610018c server_cookie=48bebb86a1195998 in_seq=0 out_seq=0 2024-04-25T09:13:01.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.523+0000 7fd16be00640 10 client.7515 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:01.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd16be00640 1 -- 172.21.15.62:0/3910373695 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f91ac207e0 con 0x55f91ace2c50 2024-04-25T09:13:01.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd16be00640 10 client.7515 handle_command_reply: tid=0 2024-04-25T09:13:01.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 client.7515 shutdown 2024-04-25T09:13:01.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 -- 172.21.15.62:0/3910373695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91ad53fb0 msgr2=0x55f91ad0d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91ad53fb0 0x55f91ad0d0a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fd180007b30 tx=0x7fd1800018a0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 -- 172.21.15.62:0/3910373695 shutdown_connections 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55f91ace2c50 0x55f91ace3060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f91ad53fb0 0x55f91ad0d0a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55f91acfd820 0x55f91ad0cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 --2- 172.21.15.62:0/3910373695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f91acf5090 0x55f91acdcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd16be00640 0 client.7515 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.526+0000 7fd190879740 1 -- 172.21.15.62:0/3910373695 >> 172.21.15.62:0/3910373695 conn(0x55f91acb5c60 msgr2=0x55f91acf48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.527+0000 7fd190879740 1 -- 172.21.15.62:0/3910373695 shutdown_connections 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.527+0000 7fd190879740 1 -- 172.21.15.62:0/3910373695 wait complete. 2024-04-25T09:13:01.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.527+0000 7fd190879740 20 client.7515 trim_cache size 0 max 16384 2024-04-25T09:13:01.515 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:13:01.515 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:13:01.515 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "c1c6f054-ee80-4d61-b25d-6fc42e2886d1", 2024-04-25T09:13:01.515 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:13:01.515 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:13:01.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.527+0000 7fd187e00640 1 -- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd16406b4d0 msgr2=0x7fd16406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd16406b4d0 0x7fd16406d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd178006fd0 tx=0x7fd178008040 comp rx=0 tx=0).stop 2024-04-25T09:13:01.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 -- 172.21.15.62:0/3434678260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161270 msgr2=0x7fd1880ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161270 0x7fd1880ce270 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd17002a190 tx=0x7fd1700046c0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 -- 172.21.15.62:0/3434678260 shutdown_connections 2024-04-25T09:13:01.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd16406b4d0 0x7fd16406d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd188161270 0x7fd1880ce270 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd188161650 0x7fd18814eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 --2- 172.21.15.62:0/3434678260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1880b2980 0x7fd1880b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 -- 172.21.15.62:0/3434678260 >> 172.21.15.62:0/3434678260 conn(0x7fd188066d10 msgr2=0x7fd1880d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 -- 172.21.15.62:0/3434678260 shutdown_connections 2024-04-25T09:13:01.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.528+0000 7fd187e00640 1 -- 172.21.15.62:0/3434678260 wait complete. 2024-04-25T09:13:01.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:01.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T09:13:01.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.769+0000 7fe4a1000640 1 Processor -- start 2024-04-25T09:13:01.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.769+0000 7fe4a1000640 1 -- start start 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.770+0000 7fe4a1000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe49c0cebc0 0x7fe49c06c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.770+0000 7fe4a1000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe49c0720c0 0x7fe49c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.770+0000 7fe4a1000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c072510 con 0x7fe49c15bdf0 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.770+0000 7fe4a1000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c15c1d0 con 0x7fe49c0720c0 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.770+0000 7fe4a1000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c0ae8c0 con 0x7fe49c0cebc0 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.770+0000 7fe49a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe49c0720c0 0x7fe49c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.771+0000 7fe49a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe49c0720c0 0x7fe49c0703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50256/0 (socket says 172.21.15.62:50256) 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.771+0000 7fe49a800640 1 -- 172.21.15.62:0/1598718690 learned_addr learned my addr 172.21.15.62:0/1598718690 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.771+0000 7fe493e00640 1 --2- 172.21.15.62:0/1598718690 >> v2:172.21.15.121:3300/0 conn(0x7fe49c0cebc0 0x7fe49c06c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.771+0000 7fe49a800640 1 -- 172.21.15.62:0/1598718690 >> v2:172.21.15.121:3300/0 conn(0x7fe49c0cebc0 msgr2=0x7fe49c06c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.771+0000 7fe49a800640 1 --2- 172.21.15.62:0/1598718690 >> v2:172.21.15.121:3300/0 conn(0x7fe49c0cebc0 0x7fe49c06c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.771+0000 7fe49a800640 1 -- 172.21.15.62:0/1598718690 >> v1:172.21.15.121:6789/0 conn(0x7fe49c15bdf0 legacy=0x7fe49c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.771+0000 7fe49a800640 1 -- 172.21.15.62:0/1598718690 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c06d070 con 0x7fe49c0720c0 2024-04-25T09:13:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.772+0000 7fe49a800640 1 --2- 172.21.15.62:0/1598718690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0720c0 0x7fe49c0703a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe49c068870 tx=0x7fe49402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c142dc39140f586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe499400640 1 -- 172.21.15.62:0/1598718690 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494036070 con 0x7fe49c0720c0 2024-04-25T09:13:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe499400640 1 -- 172.21.15.62:0/1598718690 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe494031600 con 0x7fe49c0720c0 2024-04-25T09:13:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe499400640 1 -- 172.21.15.62:0/1598718690 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940319d0 con 0x7fe49c0720c0 2024-04-25T09:13:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe4a1000640 1 -- 172.21.15.62:0/1598718690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0720c0 msgr2=0x7fe49c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe4a1000640 1 --2- 172.21.15.62:0/1598718690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0720c0 0x7fe49c0703a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe49c068870 tx=0x7fe49402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe4a1000640 1 -- 172.21.15.62:0/1598718690 shutdown_connections 2024-04-25T09:13:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe4a1000640 1 --2- 172.21.15.62:0/1598718690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0720c0 0x7fe49c0703a0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe4a1000640 1 --2- 172.21.15.62:0/1598718690 >> v2:172.21.15.121:3300/0 conn(0x7fe49c0cebc0 0x7fe49c06c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.773+0000 7fe4a1000640 1 -- 172.21.15.62:0/1598718690 >> 172.21.15.62:0/1598718690 conn(0x7fe49c0b1410 msgr2=0x7fe49c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.774+0000 7fe4a1000640 1 -- 172.21.15.62:0/1598718690 shutdown_connections 2024-04-25T09:13:01.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.774+0000 7fe4a1000640 1 -- 172.21.15.62:0/1598718690 wait complete. 2024-04-25T09:13:01.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.775+0000 7fe4a1000640 1 Processor -- start 2024-04-25T09:13:01.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.775+0000 7fe4a1000640 1 -- start start 2024-04-25T09:13:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.776+0000 7fe4a1000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe49c0720c0 0x7fe49c1f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.776+0000 7fe4a1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0cebc0 0x7fe49c0adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.776+0000 7fe4a1000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe49c15bdf0 0x7fe49c150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe4a1000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0aedf0 con 0x7fe49c0cebc0 2024-04-25T09:13:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe4a1000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0aeb10 con 0x7fe49c0720c0 2024-04-25T09:13:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe4a1000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0708e0 con 0x7fe49c15bdf0 2024-04-25T09:13:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe493e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0cebc0 0x7fe49c0adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe493e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0cebc0 0x7fe49c0adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50268/0 (socket says 172.21.15.62:50268) 2024-04-25T09:13:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe493e00640 1 -- 172.21.15.62:0/2889355080 learned_addr learned my addr 172.21.15.62:0/2889355080 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe493e00640 1 -- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe49c15bdf0 msgr2=0x7fe49c150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe493e00640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe49c15bdf0 0x7fe49c150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe493e00640 1 -- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe49c0720c0 msgr2=0x7fe49c1f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe499e00640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe49c0720c0 0x7fe49c1f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe493e00640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe49c0720c0 0x7fe49c1f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.777+0000 7fe493e00640 1 -- 172.21.15.62:0/2889355080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c143a90 con 0x7fe49c0cebc0 2024-04-25T09:13:01.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.778+0000 7fe493e00640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0cebc0 0x7fe49c0adb50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe48400d800 tx=0x7fe48400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.778+0000 7fe492000640 1 -- 172.21.15.62:0/2889355080 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48400b6c0 con 0x7fe49c0cebc0 2024-04-25T09:13:01.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.778+0000 7fe492000640 1 -- 172.21.15.62:0/2889355080 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe484019070 con 0x7fe49c0cebc0 2024-04-25T09:13:01.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.778+0000 7fe492000640 1 -- 172.21.15.62:0/2889355080 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48400ba80 con 0x7fe49c0cebc0 2024-04-25T09:13:01.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.778+0000 7fe4a1000640 1 -- 172.21.15.62:0/2889355080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c143d70 con 0x7fe49c0cebc0 2024-04-25T09:13:01.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.778+0000 7fe4a1000640 1 -- 172.21.15.62:0/2889355080 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c144330 con 0x7fe49c0cebc0 2024-04-25T09:13:01.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.780+0000 7fe492000640 1 -- 172.21.15.62:0/2889355080 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe48400bc20 con 0x7fe49c0cebc0 2024-04-25T09:13:01.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.781+0000 7fe492000640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe47806b4d0 0x7fe47806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.781+0000 7fe492000640 1 -- 172.21.15.62:0/2889355080 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe48408a750 con 0x7fe49c0cebc0 2024-04-25T09:13:01.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.782+0000 7fe4a3d67740 1 Processor -- start 2024-04-25T09:13:01.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.782+0000 7fe499e00640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe47806b4d0 0x7fe47806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.782+0000 7fe4a3d67740 1 -- start start 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.783+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.783+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 0x55c718579890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.783+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c718579dd0 0x55c7185a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.783+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7184e2870 con 0x55c718579dd0 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.783+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71848bd60 con 0x55c7185a9690 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.783+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7184de910 con 0x55c7185aa0b0 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.783+0000 7fe499e00640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe47806b4d0 0x7fe47806d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00a750 tx=0x7fe48c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe49a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c718579dd0 0x55c7185a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe49a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c718579dd0 0x55c7185a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50272/0 (socket says 172.21.15.62:50272) 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe49a800640 1 -- 172.21.15.62:0/2986037841 learned_addr learned my addr 172.21.15.62:0/2986037841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe493e00640 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 0x55c718579890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe499e00640 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe49a800640 1 -- 172.21.15.62:0/2986037841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 msgr2=0x55c718579890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe49a800640 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 0x55c718579890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe49a800640 1 -- 172.21.15.62:0/2986037841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 msgr2=0x55c7185a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe49a800640 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.784+0000 7fe49a800640 1 -- 172.21.15.62:0/2986037841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7182c17d0 con 0x55c718579dd0 2024-04-25T09:13:01.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.785+0000 7fe49a800640 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c718579dd0 0x55c7185a1b10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe494009f40 tx=0x7fe49402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fedccae4aa7ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.785+0000 7fe48aa00640 1 -- 172.21.15.62:0/2986037841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494036050 con 0x55c718579dd0 2024-04-25T09:13:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.785+0000 7fe48aa00640 1 -- 172.21.15.62:0/2986037841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe494042070 con 0x55c718579dd0 2024-04-25T09:13:01.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.785+0000 7fe48aa00640 1 -- 172.21.15.62:0/2986037841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494009510 con 0x55c718579dd0 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 -- 172.21.15.62:0/2986037841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c718579dd0 msgr2=0x55c7185a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c718579dd0 0x55c7185a1b10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe494009f40 tx=0x7fe49402c000 comp rx=0 tx=0).stop 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 -- 172.21.15.62:0/2986037841 shutdown_connections 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c718579dd0 0x55c7185a1b10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 0x55c718579890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/2986037841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 -- 172.21.15.62:0/2986037841 >> 172.21.15.62:0/2986037841 conn(0x55c71857bd10 msgr2=0x55c7185c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 -- 172.21.15.62:0/2986037841 shutdown_connections 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.786+0000 7fe4a3d67740 1 -- 172.21.15.62:0/2986037841 wait complete. 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.787+0000 7fe4a3d67740 1 -- start start 2024-04-25T09:13:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.787+0000 7fe4a3d67740 1 Processor -- start 2024-04-25T09:13:01.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c7185a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a07c0 0x55c7185a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7185a5de0 con 0x55c7185aa0b0 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c71849f1c0 con 0x55c7185a9690 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7185a6080 con 0x55c7185a07c0 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe4a3d67740 10 client.0 _ll_register_callbacks cb 0x55c7184dd8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe493e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c7185a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe493e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c7185a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50278/0 (socket says 172.21.15.62:50278) 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe493e00640 1 -- 172.21.15.62:0/1339402428 learned_addr learned my addr 172.21.15.62:0/1339402428 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe499e00640 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe49a800640 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a07c0 0x55c7185a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe493e00640 1 -- 172.21.15.62:0/1339402428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a07c0 msgr2=0x55c7185a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.788+0000 7fe493e00640 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a07c0 0x55c7185a85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe493e00640 1 -- 172.21.15.62:0/1339402428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 msgr2=0x55c7185a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe493e00640 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe493e00640 1 -- 172.21.15.62:0/1339402428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7181b70f0 con 0x55c7185aa0b0 2024-04-25T09:13:01.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe493e00640 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c7185a8080 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe48400bfd0 tx=0x7fe484059530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790996e4e4fe9198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe47f400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe47f400640 1 -- 172.21.15.62:0/1339402428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484087060 con 0x55c7185aa0b0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe47f400640 1 -- 172.21.15.62:0/1339402428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4840888b0 con 0x55c7185aa0b0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1339402428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe494042070 con 0x55c7185aa0b0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe47f400640 1 -- 172.21.15.62:0/1339402428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe484070070 con 0x55c7185aa0b0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe4a3d67740 10 client.7527 fetch_fsmap learned FSMap version 64 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.789+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1339402428 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55c718321f60 con 0x55c7185aa0b0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe47f400640 1 -- 172.21.15.62:0/1339402428 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe494042070 con 0x55c7185aa0b0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe47f400640 1 -- 172.21.15.62:0/1339402428 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe484013070 con 0x55c7185aa0b0 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe4a3d67740 10 client.7527 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe4a3d67740 10 client.7527 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe4a3d67740 20 client.7527 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55c7185ac4b0 0x55c7185ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe4a3d67740 4 client.7527 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:13:01.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1339402428 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7184b8690 con 0x55c7185ac4b0 2024-04-25T09:13:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.790+0000 7fe499e00640 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55c7185ac4b0 0x55c7185ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.791+0000 7fe499e00640 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55c7185ac4b0 0x55c7185ac8c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8b5e44e7884581 server_cookie=30ea345afc7e68c7 in_seq=0 out_seq=0 2024-04-25T09:13:01.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.791+0000 7fe47f400640 10 client.7527 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe47f400640 1 -- 172.21.15.62:0/1339402428 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7184b8690 con 0x55c7185ac4b0 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe47f400640 10 client.7527 handle_command_reply: tid=0 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 client.7527 shutdown 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1339402428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 msgr2=0x55c7185a8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c7185a8080 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe48400bfd0 tx=0x7fe484059530 comp rx=0 tx=0).stop 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1339402428 shutdown_connections 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55c7185ac4b0 0x55c7185ac8c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c7185a8080 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a07c0 0x55c7185a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1339402428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe47f400640 0 client.7527 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1339402428 >> 172.21.15.62:0/1339402428 conn(0x55c718579dd0 msgr2=0x55c7185c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1339402428 shutdown_connections 2024-04-25T09:13:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1339402428 wait complete. 2024-04-25T09:13:01.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.792+0000 7fe4a3d67740 20 client.7527 trim_cache size 0 max 16384 2024-04-25T09:13:01.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.809+0000 7fe4a3d67740 1 Processor -- start 2024-04-25T09:13:01.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.809+0000 7fe4a3d67740 1 -- start start 2024-04-25T09:13:01.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185a07c0 0x55c7185dd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 0x55c7185a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c145f40 con 0x55c7185a07c0 2024-04-25T09:13:01.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c142f90 con 0x55c7185a9690 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c149250 con 0x55c7185aa0b0 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe499e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185a07c0 0x55c7185dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe499e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185a07c0 0x55c7185dd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50284/0 (socket says 172.21.15.62:50284) 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.810+0000 7fe499e00640 1 -- 172.21.15.62:0/1292133559 learned_addr learned my addr 172.21.15.62:0/1292133559 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe499e00640 1 -- 172.21.15.62:0/1292133559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 msgr2=0x55c7185a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe499e00640 1 --2- 172.21.15.62:0/1292133559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 0x55c7185a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe499e00640 1 -- 172.21.15.62:0/1292133559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 msgr2=0x55c7185a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe499e00640 1 --2- 172.21.15.62:0/1292133559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe499e00640 1 -- 172.21.15.62:0/1292133559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7181901b0 con 0x55c7185a07c0 2024-04-25T09:13:01.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe499e00640 1 --2- 172.21.15.62:0/1292133559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185a07c0 0x55c7185dd550 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x55c7185a5250 tx=0x7fe48c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1263c9d4e120ab16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe48a000640 1 -- 172.21.15.62:0/1292133559 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c019040 con 0x55c7185a07c0 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe48a000640 1 -- 172.21.15.62:0/1292133559 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c02b070 con 0x55c7185a07c0 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.811+0000 7fe48a000640 1 -- 172.21.15.62:0/1292133559 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c021e10 con 0x55c7185a07c0 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1292133559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185a07c0 msgr2=0x55c7185dd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1292133559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185a07c0 0x55c7185dd550 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x55c7185a5250 tx=0x7fe48c013000 comp rx=0 tx=0).stop 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1292133559 shutdown_connections 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1292133559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185aa0b0 0x55c7185a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1292133559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a9690 0x55c7185a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1292133559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185a07c0 0x55c7185dd550 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1292133559 >> 172.21.15.62:0/1292133559 conn(0x55c718579dd0 msgr2=0x55c7185ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1292133559 shutdown_connections 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.812+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1292133559 wait complete. 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.813+0000 7fe4a3d67740 1 -- start start 2024-04-25T09:13:01.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.813+0000 7fe4a3d67740 1 Processor -- start 2024-04-25T09:13:01.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.814+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a07c0 0x55c7185a6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.814+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a9690 0x55c718636dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.814+0000 7fe4a3d67740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c71861fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.814+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7185ad630 con 0x55c7185aa0b0 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.814+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c142990 con 0x55c7185a07c0 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.814+0000 7fe4a3d67740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7184b0450 con 0x55c7185a9690 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.814+0000 7fe4a3d67740 10 client.0 _ll_register_callbacks cb 0x7fe494034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.814+0000 7fe49a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c71861fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe49a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c71861fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50300/0 (socket says 172.21.15.62:50300) 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe49a800640 1 -- 172.21.15.62:0/1312449144 learned_addr learned my addr 172.21.15.62:0/1312449144 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe493e00640 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a9690 0x55c718636dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe499e00640 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a07c0 0x55c7185a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe49a800640 1 -- 172.21.15.62:0/1312449144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a9690 msgr2=0x55c718636dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe49a800640 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a9690 0x55c718636dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe49a800640 1 -- 172.21.15.62:0/1312449144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a07c0 msgr2=0x55c7185a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe49a800640 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a07c0 0x55c7185a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe49a800640 1 -- 172.21.15.62:0/1312449144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7185a6120 con 0x55c7185aa0b0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe49a800640 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c71861fd40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fe49402a730 tx=0x7fe494004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880702aae867628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.815+0000 7fe47f400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe47f400640 1 -- 172.21.15.62:0/1312449144 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49404e070 con 0x55c7185aa0b0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe47f400640 1 -- 172.21.15.62:0/1312449144 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe494004780 con 0x55c7185aa0b0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1312449144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe48c02b070 con 0x55c7185aa0b0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe47f400640 1 -- 172.21.15.62:0/1312449144 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe494036020 con 0x55c7185aa0b0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 10 client.7533 fetch_fsmap learned FSMap version 64 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1312449144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55c7185d5da0 con 0x55c7185aa0b0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe47f400640 1 -- 172.21.15.62:0/1312449144 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c02b070 con 0x55c7185aa0b0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe47f400640 1 -- 172.21.15.62:0/1312449144 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe494008040 con 0x55c7185aa0b0 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 10 client.7533 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 10 client.7533 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:13:01.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 20 client.7533 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:01.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55c71859d6c0 0x55c71859dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:01.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 4 client.7533 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T09:13:01.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.816+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1312449144 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55c7184efdb0 con 0x55c71859d6c0 2024-04-25T09:13:01.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.817+0000 7fe499e00640 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55c71859d6c0 0x55c71859dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:01.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.817+0000 7fe499e00640 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55c71859d6c0 0x55c71859dad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9dc4b2ab43894f server_cookie=e46e0a1ed70044c3 in_seq=0 out_seq=0 2024-04-25T09:13:01.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.817+0000 7fe47f400640 10 client.7533 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe47f400640 1 -- 172.21.15.62:0/1312449144 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55c7184efdb0 con 0x55c71859d6c0 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe47f400640 10 client.7533 handle_command_reply: tid=0 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 client.7533 shutdown 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1312449144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 msgr2=0x55c71861fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c71861fd40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fe49402a730 tx=0x7fe494004a40 comp rx=0 tx=0).stop 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1312449144 shutdown_connections 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55c71859d6c0 0x55c71859dad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7185aa0b0 0x55c71861fd40 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7185a9690 0x55c718636dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 --2- 172.21.15.62:0/1312449144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7185a07c0 0x55c7185a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe47f400640 0 client.7533 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1312449144 >> 172.21.15.62:0/1312449144 conn(0x55c718579dd0 msgr2=0x55c7185c3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1312449144 shutdown_connections 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 1 -- 172.21.15.62:0/1312449144 wait complete. 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.818+0000 7fe4a3d67740 20 client.7533 trim_cache size 0 max 16384 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:13:01.808 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:13:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 -- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe47806b4d0 msgr2=0x7fe47806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe47806b4d0 0x7fe47806d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00a750 tx=0x7fe48c008040 comp rx=0 tx=0).stop 2024-04-25T09:13:01.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 -- 172.21.15.62:0/2889355080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0cebc0 msgr2=0x7fe49c0adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0cebc0 0x7fe49c0adb50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe48400d800 tx=0x7fe48400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 -- 172.21.15.62:0/2889355080 shutdown_connections 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe47806b4d0 0x7fe47806d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe49c0cebc0 0x7fe49c0adb50 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe49c15bdf0 0x7fe49c150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 --2- 172.21.15.62:0/2889355080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe49c0720c0 0x7fe49c1f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 -- 172.21.15.62:0/2889355080 >> 172.21.15.62:0/2889355080 conn(0x7fe49c0b1410 msgr2=0x7fe49c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 -- 172.21.15.62:0/2889355080 shutdown_connections 2024-04-25T09:13:01.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:01.819+0000 7fe4a1000640 1 -- 172.21.15.62:0/2889355080 wait complete. 2024-04-25T09:13:01.823 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:13:01.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:13:02.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.051+0000 7eff37e00640 1 Processor -- start 2024-04-25T09:13:02.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.051+0000 7eff37e00640 1 -- start start 2024-04-25T09:13:02.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.053+0000 7eff37e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7eff380b31a0 0x7eff380b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.053+0000 7eff37e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff380cd260 con 0x7eff3806d2c0 2024-04-25T09:13:02.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.053+0000 7eff37e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff380cf000 con 0x7eff380ccde0 2024-04-25T09:13:02.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.053+0000 7eff37e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff380cf2e0 con 0x7eff380b31a0 2024-04-25T09:13:02.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.053+0000 7eff36a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7eff3806d2c0 0x7eff3806d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:58362/0 (socket says 172.21.15.62:58362) 2024-04-25T09:13:02.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.053+0000 7eff36a00640 1 -- 172.21.15.62:0/2637315163 learned_addr learned my addr 172.21.15.62:0/2637315163 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:02.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.053+0000 7eff37400640 1 --2- 172.21.15.62:0/2637315163 >> v2:172.21.15.121:3300/0 conn(0x7eff380b31a0 0x7eff380b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015171484 0 0) 0x7eff380cd260 con 0x7eff3806d2c0 2024-04-25T09:13:02.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff1c003520 con 0x7eff3806d2c0 2024-04-25T09:13:02.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3902399541 0 0) 0x7eff1c003520 con 0x7eff3806d2c0 2024-04-25T09:13:02.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 >> v2:172.21.15.121:3300/0 conn(0x7eff380b31a0 msgr2=0x7eff380b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 --2- 172.21.15.62:0/2637315163 >> v2:172.21.15.121:3300/0 conn(0x7eff380b31a0 0x7eff380b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 >> v1:172.21.15.121:6789/0 conn(0x7eff380ccde0 legacy=0x7eff380ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff380b26c0 con 0x7eff3806d2c0 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7eff28001670 con 0x7eff3806d2c0 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 >> v1:172.21.15.62:6789/0 conn(0x7eff3806d2c0 legacy=0x7eff3806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff1c003a00 0x7eff1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff1c004800 0x7eff1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c0051c0 0x7eff1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff380cf2e0 con 0x7eff1c004800 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c00c0c0 con 0x7eff1c0051c0 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.054+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c00c2c0 con 0x7eff1c003a00 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff36000640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff1c004800 0x7eff1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff37400640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c0051c0 0x7eff1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff36a00640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff1c003a00 0x7eff1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff36000640 1 -- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff1c003a00 msgr2=0x7eff1c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff36000640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff1c003a00 0x7eff1c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff36000640 1 -- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c0051c0 msgr2=0x7eff1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff36000640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c0051c0 0x7eff1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff36000640 1 -- 172.21.15.62:0/2637315163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c003520 con 0x7eff1c004800 2024-04-25T09:13:02.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff37400640 1 -- 172.21.15.62:0/2637315163 reap_dead start 2024-04-25T09:13:02.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.055+0000 7eff36a00640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff1c003a00 0x7eff1c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff36000640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff1c004800 0x7eff1c004c80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7eff24001750 tx=0x7eff1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a176668c42d1ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24001370 con 0x7eff1c004800 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff240052d0 con 0x7eff1c004800 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff35600640 1 -- 172.21.15.62:0/2637315163 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2402abc0 con 0x7eff1c004800 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff37e00640 1 -- 172.21.15.62:0/2637315163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff1c004800 msgr2=0x7eff1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff37e00640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff1c004800 0x7eff1c004c80 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7eff24001750 tx=0x7eff1c005830 comp rx=0 tx=0).stop 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff37e00640 1 -- 172.21.15.62:0/2637315163 shutdown_connections 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff37e00640 1 --2- 172.21.15.62:0/2637315163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff1c004800 0x7eff1c004c80 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.057+0000 7eff37e00640 1 -- 172.21.15.62:0/2637315163 >> 172.21.15.62:0/2637315163 conn(0x7eff38066f20 msgr2=0x7eff380b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.058+0000 7eff37e00640 1 -- 172.21.15.62:0/2637315163 shutdown_connections 2024-04-25T09:13:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.058+0000 7eff37e00640 1 -- 172.21.15.62:0/2637315163 wait complete. 2024-04-25T09:13:02.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.059+0000 7eff37e00640 1 Processor -- start 2024-04-25T09:13:02.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.059+0000 7eff37e00640 1 -- start start 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.060+0000 7eff37e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c004800 0x7eff381e8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.060+0000 7eff37e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff381f4c20 0x7eff381e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.060+0000 7eff37e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff381e58b0 0x7eff3815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.060+0000 7eff37e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381448d0 con 0x7eff381f4c20 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.060+0000 7eff37e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff380b2c80 con 0x7eff1c004800 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.060+0000 7eff37e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff38144a50 con 0x7eff381e58b0 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff381f4c20 0x7eff381e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff381f4c20 0x7eff381e5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50324/0 (socket says 172.21.15.62:50324) 2024-04-25T09:13:02.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 -- 172.21.15.62:0/2484215594 learned_addr learned my addr 172.21.15.62:0/2484215594 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:02.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 -- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff381e58b0 msgr2=0x7eff3815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff381e58b0 0x7eff3815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 -- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c004800 msgr2=0x7eff381e8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36a00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c004800 0x7eff381e8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c004800 0x7eff381e8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 -- 172.21.15.62:0/2484215594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff381e66f0 con 0x7eff381f4c20 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.061+0000 7eff36000640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff381f4c20 0x7eff381e5370 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7eff24001870 tx=0x7eff24001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.062+0000 7eff2ea00640 1 -- 172.21.15.62:0/2484215594 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2403f020 con 0x7eff381f4c20 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.062+0000 7eff2ea00640 1 -- 172.21.15.62:0/2484215594 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24005480 con 0x7eff381f4c20 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.062+0000 7eff2ea00640 1 -- 172.21.15.62:0/2484215594 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24037040 con 0x7eff381f4c20 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.062+0000 7eff37e00640 1 -- 172.21.15.62:0/2484215594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff381e69d0 con 0x7eff381f4c20 2024-04-25T09:13:02.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.062+0000 7eff37e00640 1 -- 172.21.15.62:0/2484215594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff381e7270 con 0x7eff381f4c20 2024-04-25T09:13:02.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.064+0000 7eff2ea00640 1 -- 172.21.15.62:0/2484215594 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff24042020 con 0x7eff381f4c20 2024-04-25T09:13:02.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.065+0000 7eff2ea00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7eff1806b520 0x7eff1806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.065+0000 7eff2ea00640 1 -- 172.21.15.62:0/2484215594 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff240a9c90 con 0x7eff381f4c20 2024-04-25T09:13:02.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.066+0000 7eff40322740 1 Processor -- start 2024-04-25T09:13:02.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.066+0000 7eff36a00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7eff1806b520 0x7eff1806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.066+0000 7eff40322740 1 -- start start 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff40322740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 0x55d050ec2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff40322740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050efd440 0x55d050ec1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff40322740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050eebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff40322740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d050e24c70 con 0x55d050efd440 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff40322740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d050dcdcf0 con 0x55d050ec0d70 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff40322740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d050e20d10 con 0x55d050eeb830 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff36a00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7eff1806b520 0x7eff1806d9e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7eff28001c00 tx=0x7eff28001110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff36000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050efd440 0x55d050ec1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff36000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050efd440 0x55d050ec1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50340/0 (socket says 172.21.15.62:50340) 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.067+0000 7eff36000640 1 -- 172.21.15.62:0/138711739 learned_addr learned my addr 172.21.15.62:0/138711739 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:02.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff36a00640 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 0x55d050ec2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff37400640 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050eebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff36000640 1 -- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 msgr2=0x55d050eebcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff36000640 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050eebcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff36000640 1 -- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 msgr2=0x55d050ec2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff36000640 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 0x55d050ec2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff36000640 1 -- 172.21.15.62:0/138711739 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d050c037d0 con 0x55d050efd440 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff37400640 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050eebcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff36000640 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050efd440 0x55d050ec1150 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7eff240a80e0 tx=0x7eff240782f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e7b8f31ca07b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff23400640 1 -- 172.21.15.62:0/138711739 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2407c540 con 0x55d050efd440 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff23400640 1 -- 172.21.15.62:0/138711739 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24080b00 con 0x55d050efd440 2024-04-25T09:13:02.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.068+0000 7eff23400640 1 -- 172.21.15.62:0/138711739 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2402be10 con 0x55d050efd440 2024-04-25T09:13:02.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 -- 172.21.15.62:0/138711739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050efd440 msgr2=0x55d050ec1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050efd440 0x55d050ec1150 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7eff240a80e0 tx=0x7eff240782f0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 -- 172.21.15.62:0/138711739 shutdown_connections 2024-04-25T09:13:02.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050efd440 0x55d050ec1150 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050eebcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 --2- 172.21.15.62:0/138711739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 0x55d050ec2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 -- 172.21.15.62:0/138711739 >> 172.21.15.62:0/138711739 conn(0x55d050e88a30 msgr2=0x55d050f06830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 -- 172.21.15.62:0/138711739 shutdown_connections 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.069+0000 7eff40322740 1 -- 172.21.15.62:0/138711739 wait complete. 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.070+0000 7eff40322740 1 -- start start 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.070+0000 7eff40322740 1 Processor -- start 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff40322740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050eed4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff40322740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050f19510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff40322740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050efd440 0x55d050f19a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff40322740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d050dcdcf0 con 0x55d050ec0d70 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff40322740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d050dc33a0 con 0x55d050efd440 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff40322740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d050e24c70 con 0x55d050eeb830 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff40322740 10 client.0 _ll_register_callbacks cb 0x55d050de0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:02.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050eed4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050eed4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50346/0 (socket says 172.21.15.62:50346) 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.071+0000 7eff36a00640 1 -- 172.21.15.62:0/1038787319 learned_addr learned my addr 172.21.15.62:0/1038787319 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff37400640 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050efd440 0x55d050f19a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff36000640 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050f19510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff36a00640 1 -- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 msgr2=0x55d050f19510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff36a00640 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050f19510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff36a00640 1 -- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050efd440 msgr2=0x55d050f19a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff36a00640 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050efd440 0x55d050f19a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff36a00640 1 -- 172.21.15.62:0/1038787319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d050b393f0 con 0x55d050ec0d70 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff36000640 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050f19510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff36a00640 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050eed4b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7eff380cef10 tx=0x7eff28012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f04520b2d3d01e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.072+0000 7eff17e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff17e00640 1 -- 172.21.15.62:0/1038787319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28028060 con 0x55d050ec0d70 2024-04-25T09:13:02.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff17e00640 1 -- 172.21.15.62:0/1038787319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff28016110 con 0x55d050ec0d70 2024-04-25T09:13:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 1 -- 172.21.15.62:0/1038787319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff24080b00 con 0x55d050ec0d70 2024-04-25T09:13:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff17e00640 1 -- 172.21.15.62:0/1038787319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff28010040 con 0x55d050ec0d70 2024-04-25T09:13:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 10 client.7545 fetch_fsmap learned FSMap version 64 2024-04-25T09:13:02.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 1 -- 172.21.15.62:0/1038787319 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55d050af90f0 con 0x55d050ec0d70 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff17e00640 1 -- 172.21.15.62:0/1038787319 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24080b00 con 0x55d050ec0d70 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff17e00640 1 -- 172.21.15.62:0/1038787319 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7eff28013070 con 0x55d050ec0d70 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 10 client.7545 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 10 client.7545 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 20 client.7545 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55d050eea870 0x55d050eeace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 4 client.7545 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.073+0000 7eff40322740 1 -- 172.21.15.62:0/1038787319 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d050e321b0 con 0x55d050eea870 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.074+0000 7eff36000640 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55d050eea870 0x55d050eeace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.074+0000 7eff36000640 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55d050eea870 0x55d050eeace0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d842c4b1adf75fc6 server_cookie=7c6d19e64bcc3801 in_seq=0 out_seq=0 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.074+0000 7eff17e00640 10 client.7545 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff17e00640 1 -- 172.21.15.62:0/1038787319 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7eff24095a90 con 0x55d050eea870 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff17e00640 10 client.7545 handle_command_reply: tid=0 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 client.7545 shutdown 2024-04-25T09:13:02.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 -- 172.21.15.62:0/1038787319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 msgr2=0x55d050eed4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050eed4b0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7eff380cef10 tx=0x7eff28012000 comp rx=0 tx=0).stop 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 -- 172.21.15.62:0/1038787319 shutdown_connections 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55d050eea870 0x55d050eeace0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050efd440 0x55d050f19a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050f19510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 --2- 172.21.15.62:0/1038787319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050eed4b0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff40322740 1 -- 172.21.15.62:0/1038787319 >> 172.21.15.62:0/1038787319 conn(0x55d050e88a30 msgr2=0x55d050f0a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.075+0000 7eff17e00640 0 client.7545 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.076+0000 7eff40322740 1 -- 172.21.15.62:0/1038787319 shutdown_connections 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.076+0000 7eff40322740 1 -- 172.21.15.62:0/1038787319 wait complete. 2024-04-25T09:13:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.076+0000 7eff40322740 20 client.7545 trim_cache size 0 max 16384 2024-04-25T09:13:02.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.093+0000 7eff40322740 1 Processor -- start 2024-04-25T09:13:02.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.093+0000 7eff40322740 1 -- start start 2024-04-25T09:13:02.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff40322740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050e88a30 0x55d050f021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff40322740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 0x55d050f019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff40322740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050f04570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff40322740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff38200670 con 0x55d050e88a30 2024-04-25T09:13:02.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff40322740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381ff8a0 con 0x55d050ec0d70 2024-04-25T09:13:02.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff40322740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff381f4550 con 0x55d050eeb830 2024-04-25T09:13:02.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050e88a30 0x55d050f021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050e88a30 0x55d050f021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50348/0 (socket says 172.21.15.62:50348) 2024-04-25T09:13:02.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36a00640 1 -- 172.21.15.62:0/2763028163 learned_addr learned my addr 172.21.15.62:0/2763028163 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:02.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36a00640 1 -- 172.21.15.62:0/2763028163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 msgr2=0x55d050f04570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36a00640 1 --2- 172.21.15.62:0/2763028163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050f04570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36a00640 1 -- 172.21.15.62:0/2763028163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 msgr2=0x55d050f019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36000640 1 --2- 172.21.15.62:0/2763028163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 0x55d050f019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36a00640 1 --2- 172.21.15.62:0/2763028163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 0x55d050f019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.094+0000 7eff36a00640 1 -- 172.21.15.62:0/2763028163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d050f01760 con 0x55d050e88a30 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.095+0000 7eff36a00640 1 --2- 172.21.15.62:0/2763028163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050e88a30 0x55d050f021a0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7eff28000fd0 tx=0x7eff28007030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645ea217704e907e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.095+0000 7eff22a00640 1 -- 172.21.15.62:0/2763028163 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280160f0 con 0x55d050e88a30 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.095+0000 7eff22a00640 1 -- 172.21.15.62:0/2763028163 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff28016290 con 0x55d050e88a30 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.095+0000 7eff22a00640 1 -- 172.21.15.62:0/2763028163 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28014700 con 0x55d050e88a30 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 -- 172.21.15.62:0/2763028163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050e88a30 msgr2=0x55d050f021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 --2- 172.21.15.62:0/2763028163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050e88a30 0x55d050f021a0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7eff28000fd0 tx=0x7eff28007030 comp rx=0 tx=0).stop 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 -- 172.21.15.62:0/2763028163 shutdown_connections 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 --2- 172.21.15.62:0/2763028163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050eeb830 0x55d050f04570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 --2- 172.21.15.62:0/2763028163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050ec0d70 0x55d050f019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 --2- 172.21.15.62:0/2763028163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050e88a30 0x55d050f021a0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 -- 172.21.15.62:0/2763028163 >> 172.21.15.62:0/2763028163 conn(0x55d050eeafc0 msgr2=0x55d050ec2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:02.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 -- 172.21.15.62:0/2763028163 shutdown_connections 2024-04-25T09:13:02.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 -- 172.21.15.62:0/2763028163 wait complete. 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.096+0000 7eff40322740 1 -- start start 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.097+0000 7eff40322740 1 Processor -- start 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff40322740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050e88a30 0x55d050f20050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff40322740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050f28cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff40322740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050eeb830 0x55d050f01d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff40322740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d050f13890 con 0x55d050ec0d70 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff40322740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381ff2a0 con 0x55d050eeb830 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff40322740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d050f13640 con 0x55d050e88a30 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff40322740 10 client.0 _ll_register_callbacks cb 0x7eff28001430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:02.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050f28cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050f28cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50356/0 (socket says 172.21.15.62:50356) 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36000640 1 -- 172.21.15.62:0/1492993089 learned_addr learned my addr 172.21.15.62:0/1492993089 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36a00640 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050e88a30 0x55d050f20050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff37400640 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050eeb830 0x55d050f01d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36000640 1 -- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050e88a30 msgr2=0x55d050f20050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36000640 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050e88a30 0x55d050f20050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36000640 1 -- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050eeb830 msgr2=0x55d050f01d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36000640 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050eeb830 0x55d050f01d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff36000640 1 -- 172.21.15.62:0/1492993089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d050f25390 con 0x55d050ec0d70 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.098+0000 7eff37400640 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050eeb830 0x55d050f01d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff36000640 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050f28cd0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7eff240050a0 tx=0x7eff24000f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efac710c85b5db18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff17e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:02.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff17e00640 1 -- 172.21.15.62:0/1492993089 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2407c540 con 0x55d050ec0d70 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff17e00640 1 -- 172.21.15.62:0/1492993089 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff24095a10 con 0x55d050ec0d70 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff40322740 1 -- 172.21.15.62:0/1492993089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff28016290 con 0x55d050ec0d70 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff17e00640 1 -- 172.21.15.62:0/1492993089 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff24080b00 con 0x55d050ec0d70 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff40322740 10 client.7551 fetch_fsmap learned FSMap version 64 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff40322740 1 -- 172.21.15.62:0/1492993089 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55d050f06e00 con 0x55d050ec0d70 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.099+0000 7eff17e00640 1 -- 172.21.15.62:0/1492993089 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff28016290 con 0x55d050ec0d70 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff17e00640 1 -- 172.21.15.62:0/1492993089 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7eff2403a030 con 0x55d050ec0d70 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff40322740 10 client.7551 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff40322740 10 client.7551 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff40322740 20 client.7551 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff40322740 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55d050efd440 0x55d050eec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff40322740 4 client.7551 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff40322740 1 -- 172.21.15.62:0/1492993089 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d050e2c500 con 0x55d050efd440 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff36a00640 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55d050efd440 0x55d050eec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff36a00640 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55d050efd440 0x55d050eec7a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=958de06e54ddd064 server_cookie=8976e1ec91fbac69 in_seq=0 out_seq=0 2024-04-25T09:13:02.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.100+0000 7eff17e00640 10 client.7551 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:02.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff17e00640 1 -- 172.21.15.62:0/1492993089 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55d050e2c500 con 0x55d050efd440 2024-04-25T09:13:02.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff17e00640 10 client.7551 handle_command_reply: tid=0 2024-04-25T09:13:02.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 client.7551 shutdown 2024-04-25T09:13:02.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 -- 172.21.15.62:0/1492993089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 msgr2=0x55d050f28cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050f28cd0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7eff240050a0 tx=0x7eff24000f90 comp rx=0 tx=0).stop 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 -- 172.21.15.62:0/1492993089 shutdown_connections 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x55d050efd440 0x55d050eec7a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d050ec0d70 0x55d050f28cd0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff17e00640 0 client.7551 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d050eeb830 0x55d050f01d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 --2- 172.21.15.62:0/1492993089 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55d050e88a30 0x55d050f20050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 -- 172.21.15.62:0/1492993089 >> 172.21.15.62:0/1492993089 conn(0x55d050eeafc0 msgr2=0x55d050f05610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 -- 172.21.15.62:0/1492993089 shutdown_connections 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 1 -- 172.21.15.62:0/1492993089 wait complete. 2024-04-25T09:13:02.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.101+0000 7eff40322740 20 client.7551 trim_cache size 0 max 16384 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout: "c1c6f054-ee80-4d61-b25d-6fc42e2886d1": { 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "c1c6f054-ee80-4d61-b25d-6fc42e2886d1", 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive" 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:02.092 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:13:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 -- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7eff1806b520 msgr2=0x7eff1806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7eff1806b520 0x7eff1806d9e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7eff28001c00 tx=0x7eff28001110 comp rx=0 tx=0).stop 2024-04-25T09:13:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 -- 172.21.15.62:0/2484215594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff381f4c20 msgr2=0x7eff381e5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff381f4c20 0x7eff381e5370 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7eff24001870 tx=0x7eff24001120 comp rx=0 tx=0).stop 2024-04-25T09:13:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 -- 172.21.15.62:0/2484215594 shutdown_connections 2024-04-25T09:13:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7eff1806b520 0x7eff1806d9e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7eff381f4c20 0x7eff381e5370 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7eff381e58b0 0x7eff3815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 --2- 172.21.15.62:0/2484215594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff1c004800 0x7eff381e8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:02.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 -- 172.21.15.62:0/2484215594 >> 172.21.15.62:0/2484215594 conn(0x7eff38066f20 msgr2=0x7eff380af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:02.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 -- 172.21.15.62:0/2484215594 shutdown_connections 2024-04-25T09:13:02.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:02.102+0000 7eff37e00640 1 -- 172.21.15.62:0/2484215594 wait complete. 2024-04-25T09:13:07.107 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:13:07.107 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:13:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.338+0000 7fc1f7200640 1 Processor -- start 2024-04-25T09:13:07.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.338+0000 7fc1f7200640 1 -- start start 2024-04-25T09:13:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.340+0000 7fc1f7200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc1f806d2c0 0x7fc1f80ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.340+0000 7fc1f7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc1f80b31a0 0x7fc1f80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.340+0000 7fc1f7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f80cf000 con 0x7fc1f806d6a0 2024-04-25T09:13:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.340+0000 7fc1f7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f80cf2e0 con 0x7fc1f80b31a0 2024-04-25T09:13:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.340+0000 7fc1f7200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f80b26c0 con 0x7fc1f806d2c0 2024-04-25T09:13:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.340+0000 7fc1f6800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc1f80b31a0 0x7fc1f80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.340+0000 7fc1f5400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc1f806d6a0 0x7fc1f80ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36974/0 (socket says 172.21.15.62:36974) 2024-04-25T09:13:07.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.340+0000 7fc1f5400640 1 -- 172.21.15.62:0/1171736006 learned_addr learned my addr 172.21.15.62:0/1171736006 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.341+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/1171736006 >> v2:172.21.15.121:3300/0 conn(0x7fc1f806d2c0 0x7fc1f80ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.341+0000 7fc1f6800640 1 -- 172.21.15.62:0/1171736006 >> v2:172.21.15.121:3300/0 conn(0x7fc1f806d2c0 msgr2=0x7fc1f80ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.341+0000 7fc1f6800640 1 --2- 172.21.15.62:0/1171736006 >> v2:172.21.15.121:3300/0 conn(0x7fc1f806d2c0 0x7fc1f80ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.341+0000 7fc1f6800640 1 -- 172.21.15.62:0/1171736006 >> v1:172.21.15.62:6789/0 conn(0x7fc1f806d6a0 legacy=0x7fc1f80ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.341+0000 7fc1f4a00640 1 -- 172.21.15.62:0/1171736006 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798707475 0 0) 0x7fc1f80cf000 con 0x7fc1f806d6a0 2024-04-25T09:13:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.341+0000 7fc1f6800640 1 -- 172.21.15.62:0/1171736006 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f80b29a0 con 0x7fc1f80b31a0 2024-04-25T09:13:07.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.341+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/1171736006 >> v2:172.21.15.121:3300/0 conn(0x7fc1f806d2c0 0x7fc1f80ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.342+0000 7fc1f6800640 1 --2- 172.21.15.62:0/1171736006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f80b31a0 0x7fc1f80b3f40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80b20a0 tx=0x7fc1f002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdc84cc05b68b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.342+0000 7fc1f4a00640 1 -- 172.21.15.62:0/1171736006 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0007740 con 0x7fc1f80b31a0 2024-04-25T09:13:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f4a00640 1 -- 172.21.15.62:0/1171736006 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f00078e0 con 0x7fc1f80b31a0 2024-04-25T09:13:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f4a00640 1 -- 172.21.15.62:0/1171736006 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0007cb0 con 0x7fc1f80b31a0 2024-04-25T09:13:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f7200640 1 -- 172.21.15.62:0/1171736006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f80b31a0 msgr2=0x7fc1f80b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f7200640 1 --2- 172.21.15.62:0/1171736006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f80b31a0 0x7fc1f80b3f40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80b20a0 tx=0x7fc1f002a680 comp rx=0 tx=0).stop 2024-04-25T09:13:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f7200640 1 -- 172.21.15.62:0/1171736006 shutdown_connections 2024-04-25T09:13:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f7200640 1 --2- 172.21.15.62:0/1171736006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f80b31a0 0x7fc1f80b3f40 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f7200640 1 --2- 172.21.15.62:0/1171736006 >> v2:172.21.15.121:3300/0 conn(0x7fc1f806d2c0 0x7fc1f80ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f7200640 1 -- 172.21.15.62:0/1171736006 >> 172.21.15.62:0/1171736006 conn(0x7fc1f8066f20 msgr2=0x7fc1f80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f7200640 1 -- 172.21.15.62:0/1171736006 shutdown_connections 2024-04-25T09:13:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.343+0000 7fc1f7200640 1 -- 172.21.15.62:0/1171736006 wait complete. 2024-04-25T09:13:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.345+0000 7fc1f7200640 1 Processor -- start 2024-04-25T09:13:07.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.345+0000 7fc1f7200640 1 -- start start 2024-04-25T09:13:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.346+0000 7fc1f7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1f806d2c0 0x7fc1f815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.346+0000 7fc1f7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f806d6a0 0x7fc1f815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.346+0000 7fc1f7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1f80b31a0 0x7fc1f815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.346+0000 7fc1f7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81448d0 con 0x7fc1f806d6a0 2024-04-25T09:13:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.346+0000 7fc1f7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f806db20 con 0x7fc1f806d2c0 2024-04-25T09:13:07.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.346+0000 7fc1f7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f8144a50 con 0x7fc1f80b31a0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.346+0000 7fc1f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f806d6a0 0x7fc1f815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f806d6a0 0x7fc1f815add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56246/0 (socket says 172.21.15.62:56246) 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5400640 1 -- 172.21.15.62:0/159634113 learned_addr learned my addr 172.21.15.62:0/159634113 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1f806d2c0 0x7fc1f815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f6800640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1f80b31a0 0x7fc1f815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5400640 1 -- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1f80b31a0 msgr2=0x7fc1f815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5400640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1f80b31a0 0x7fc1f815b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5400640 1 -- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1f806d2c0 msgr2=0x7fc1f815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5400640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1f806d2c0 0x7fc1f815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5400640 1 -- 172.21.15.62:0/159634113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f81e44c0 con 0x7fc1f806d6a0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.347+0000 7fc1f5400640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f806d6a0 0x7fc1f815add0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c500 tx=0x7fc1e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.348+0000 7fc1ee000640 1 -- 172.21.15.62:0/159634113 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4018070 con 0x7fc1f806d6a0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.348+0000 7fc1ee000640 1 -- 172.21.15.62:0/159634113 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4007580 con 0x7fc1f806d6a0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.348+0000 7fc1ee000640 1 -- 172.21.15.62:0/159634113 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4004a20 con 0x7fc1f806d6a0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.348+0000 7fc1f7200640 1 -- 172.21.15.62:0/159634113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f81e47a0 con 0x7fc1f806d6a0 2024-04-25T09:13:07.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.348+0000 7fc1f7200640 1 -- 172.21.15.62:0/159634113 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f81e4d60 con 0x7fc1f806d6a0 2024-04-25T09:13:07.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.351+0000 7fc1ee000640 1 -- 172.21.15.62:0/159634113 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1e4004bc0 con 0x7fc1f806d6a0 2024-04-25T09:13:07.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.352+0000 7fc1ee000640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1d806b4d0 0x7fc1d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.352+0000 7fc1ee000640 1 -- 172.21.15.62:0/159634113 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1e4089530 con 0x7fc1f806d6a0 2024-04-25T09:13:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.353+0000 7fc1ffbdf740 1 Processor -- start 2024-04-25T09:13:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.353+0000 7fc1ffbdf740 1 -- start start 2024-04-25T09:13:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.353+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1d806b4d0 0x7fc1d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88783440 0x564e88747150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88771830 0x564e88771cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e886aac70 con 0x564e88783440 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88653cf0 con 0x564e88771830 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e886a6d10 con 0x564e88746d70 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88783440 0x564e88747150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88783440 0x564e88747150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56248/0 (socket says 172.21.15.62:56248) 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5400640 1 -- 172.21.15.62:0/354866027 learned_addr learned my addr 172.21.15.62:0/354866027 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1d806b4d0 0x7fc1d806d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800a990 tx=0x7fc1e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f6800640 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88771830 0x564e88771cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5400640 1 -- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 msgr2=0x564e88748ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5400640 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5400640 1 -- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88771830 msgr2=0x564e88771cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.354+0000 7fc1f5400640 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88771830 0x564e88771cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.355+0000 7fc1f5400640 1 -- 172.21.15.62:0/354866027 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e884897d0 con 0x564e88783440 2024-04-25T09:13:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.355+0000 7fc1f6800640 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88771830 0x564e88771cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:07.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.355+0000 7fc1f5400640 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88783440 0x564e88747150 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc1e405fe80 tx=0x7fc1e4057ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3358ad48384ee86d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.355+0000 7fc1e2a00640 1 -- 172.21.15.62:0/354866027 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e405e0c0 con 0x564e88783440 2024-04-25T09:13:07.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.355+0000 7fc1e2a00640 1 -- 172.21.15.62:0/354866027 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e40603b0 con 0x564e88783440 2024-04-25T09:13:07.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.355+0000 7fc1e2a00640 1 -- 172.21.15.62:0/354866027 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4054310 con 0x564e88783440 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/354866027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88783440 msgr2=0x564e88747150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88783440 0x564e88747150 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc1e405fe80 tx=0x7fc1e4057ba0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/354866027 shutdown_connections 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88783440 0x564e88747150 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88771830 0x564e88771cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/354866027 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/354866027 >> 172.21.15.62:0/354866027 conn(0x564e8870ea30 msgr2=0x564e8878c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/354866027 shutdown_connections 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.356+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/354866027 wait complete. 2024-04-25T09:13:07.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.357+0000 7fc1ffbdf740 1 -- start start 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.357+0000 7fc1ffbdf740 1 Processor -- start 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88766620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88773460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e886493a0 con 0x564e88771830 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8876f380 con 0x564e88783440 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e887845c0 con 0x564e88746d70 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1ffbdf740 10 client.0 _ll_register_callbacks cb 0x564e88666b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56260/0 (socket says 172.21.15.62:56260) 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.358+0000 7fc1f5400640 1 -- 172.21.15.62:0/2482911400 learned_addr learned my addr 172.21.15.62:0/2482911400 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88766620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1f6800640 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88773460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1f5400640 1 -- 172.21.15.62:0/2482911400 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 msgr2=0x564e88766620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1f5400640 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88766620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1f5400640 1 -- 172.21.15.62:0/2482911400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 msgr2=0x564e88773460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1f5400640 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88773460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1f5400640 1 -- 172.21.15.62:0/2482911400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e883bf3f0 con 0x564e88771830 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1f5400640 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772c10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4014040 tx=0x7fc1e405a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf1e3f2a5942d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1d7400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1d7400640 1 -- 172.21.15.62:0/2482911400 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40572f0 con 0x564e88771830 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1d7400640 1 -- 172.21.15.62:0/2482911400 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1e4089030 con 0x564e88771830 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.359+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2482911400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1e40603b0 con 0x564e88771830 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1d7400640 1 -- 172.21.15.62:0/2482911400 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e4089230 con 0x564e88771830 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1ffbdf740 10 client.7566 fetch_fsmap learned FSMap version 64 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2482911400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x564e8837f0f0 con 0x564e88771830 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1d7400640 1 -- 172.21.15.62:0/2482911400 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e40603b0 con 0x564e88771830 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1d7400640 1 -- 172.21.15.62:0/2482911400 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1e4076b70 con 0x564e88771830 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1ffbdf740 10 client.7566 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:13:07.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1ffbdf740 10 client.7566 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:13:07.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1ffbdf740 20 client.7566 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:07.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x564e8876d9d0 0x564e8876ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1ffbdf740 4 client.7566 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:13:07.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2482911400 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e886b81b0 con 0x564e8876d9d0 2024-04-25T09:13:07.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.360+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x564e8876d9d0 0x564e8876ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.361+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x564e8876d9d0 0x564e8876ddb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b24a0baf6deea9 server_cookie=10012f7c3ba47fe9 in_seq=0 out_seq=0 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.361+0000 7fc1d7400640 10 client.7566 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1d7400640 1 -- 172.21.15.62:0/2482911400 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e886b81b0 con 0x564e8876d9d0 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1d7400640 10 client.7566 handle_command_reply: tid=0 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 client.7566 shutdown 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2482911400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 msgr2=0x564e88772c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772c10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4014040 tx=0x7fc1e405a040 comp rx=0 tx=0).stop 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2482911400 shutdown_connections 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x564e8876d9d0 0x564e8876ddb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772c10 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88773460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2482911400 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88766620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1d7400640 0 client.7566 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:07.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2482911400 >> 172.21.15.62:0/2482911400 conn(0x564e8870ea30 msgr2=0x564e8876f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:07.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2482911400 shutdown_connections 2024-04-25T09:13:07.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.362+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2482911400 wait complete. 2024-04-25T09:13:07.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.363+0000 7fc1ffbdf740 20 client.7566 trim_cache size 0 max 16384 2024-04-25T09:13:07.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.379+0000 7fc1ffbdf740 1 Processor -- start 2024-04-25T09:13:07.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.379+0000 7fc1ffbdf740 1 -- start start 2024-04-25T09:13:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.380+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.380+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.380+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88766dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.380+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81ffa60 con 0x564e88771830 2024-04-25T09:13:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.380+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81fec20 con 0x564e88783440 2024-04-25T09:13:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.380+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f815cc20 con 0x564e88746d70 2024-04-25T09:13:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.380+0000 7fc1f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56268/0 (socket says 172.21.15.62:56268) 2024-04-25T09:13:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5400640 1 -- 172.21.15.62:0/2339648581 learned_addr learned my addr 172.21.15.62:0/2339648581 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5400640 1 -- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 msgr2=0x564e88748f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f6800640 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88766dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5400640 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5400640 1 -- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 msgr2=0x564e88766dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5400640 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88766dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5400640 1 -- 172.21.15.62:0/2339648581 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e88748200 con 0x564e88771830 2024-04-25T09:13:07.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f6800640 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88766dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:07.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:07.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.381+0000 7fc1f5400640 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772de0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c4d0 tx=0x7fc1e4088000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25452d672c71dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1e2000640 1 -- 172.21.15.62:0/2339648581 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4080020 con 0x564e88771830 2024-04-25T09:13:07.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1e2000640 1 -- 172.21.15.62:0/2339648581 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4074070 con 0x564e88771830 2024-04-25T09:13:07.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1e2000640 1 -- 172.21.15.62:0/2339648581 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4053f40 con 0x564e88771830 2024-04-25T09:13:07.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2339648581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 msgr2=0x564e88772de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772de0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c4d0 tx=0x7fc1e4088000 comp rx=0 tx=0).stop 2024-04-25T09:13:07.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2339648581 shutdown_connections 2024-04-25T09:13:07.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88771830 0x564e88772de0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88783440 0x564e88766dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/2339648581 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88746d70 0x564e88748f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.382+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2339648581 >> 172.21.15.62:0/2339648581 conn(0x564e8870ea30 msgr2=0x564e8876f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:07.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.383+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2339648581 shutdown_connections 2024-04-25T09:13:07.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.383+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/2339648581 wait complete. 2024-04-25T09:13:07.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.383+0000 7fc1ffbdf740 1 -- start start 2024-04-25T09:13:07.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.383+0000 7fc1ffbdf740 1 Processor -- start 2024-04-25T09:13:07.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.384+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88746d70 0x564e887fe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.384+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88783440 0x564e887821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.384+0000 7fc1ffbdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88782700 0x564e887ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.384+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e88743a40 con 0x564e88746d70 2024-04-25T09:13:07.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.384+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81e6180 con 0x564e88782700 2024-04-25T09:13:07.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.384+0000 7fc1ffbdf740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e88747820 con 0x564e88783440 2024-04-25T09:13:07.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.384+0000 7fc1ffbdf740 10 client.0 _ll_register_callbacks cb 0x564e88741b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:07.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88746d70 0x564e887fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88746d70 0x564e887fe550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56278/0 (socket says 172.21.15.62:56278) 2024-04-25T09:13:07.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5e00640 1 -- 172.21.15.62:0/1756788524 learned_addr learned my addr 172.21.15.62:0/1756788524 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:07.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5400640 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88783440 0x564e887821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5e00640 1 -- 172.21.15.62:0/1756788524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88783440 msgr2=0x564e887821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f6800640 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88782700 0x564e887ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88783440 0x564e887821c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5e00640 1 -- 172.21.15.62:0/1756788524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88782700 msgr2=0x564e887ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88782700 0x564e887ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.385+0000 7fc1f5e00640 1 -- 172.21.15.62:0/1756788524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e887afc10 con 0x564e88746d70 2024-04-25T09:13:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.386+0000 7fc1f5e00640 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88746d70 0x564e887fe550 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800fb50 tx=0x7fc1e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcc3f2f75626428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.386+0000 7fc1d7400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.386+0000 7fc1d7400640 1 -- 172.21.15.62:0/1756788524 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e801b040 con 0x564e88746d70 2024-04-25T09:13:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.386+0000 7fc1d7400640 1 -- 172.21.15.62:0/1756788524 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1e8005b90 con 0x564e88746d70 2024-04-25T09:13:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.386+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/1756788524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1e4074070 con 0x564e88746d70 2024-04-25T09:13:07.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.386+0000 7fc1d7400640 1 -- 172.21.15.62:0/1756788524 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e800fe90 con 0x564e88746d70 2024-04-25T09:13:07.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.386+0000 7fc1ffbdf740 10 client.7572 fetch_fsmap learned FSMap version 64 2024-04-25T09:13:07.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.386+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/1756788524 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x564e88773f00 con 0x564e88746d70 2024-04-25T09:13:07.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1d7400640 1 -- 172.21.15.62:0/1756788524 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4074070 con 0x564e88746d70 2024-04-25T09:13:07.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1d7400640 1 -- 172.21.15.62:0/1756788524 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1e8038020 con 0x564e88746d70 2024-04-25T09:13:07.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1ffbdf740 10 client.7572 fetch_fsmap finished waiting for FSMap version 64 2024-04-25T09:13:07.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1ffbdf740 10 client.7572 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:13:07.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1ffbdf740 20 client.7572 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:07.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x564e887ace20 0x564e8876c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1ffbdf740 4 client.7572 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:13:07.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/1756788524 --> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e886b2500 con 0x564e887ace20 2024-04-25T09:13:07.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1f5400640 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x564e887ace20 0x564e8876c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1f5400640 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x564e887ace20 0x564e8876c6d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b01ef9c8c2c8d0f3 server_cookie=4b0041724f91947b in_seq=0 out_seq=0 2024-04-25T09:13:07.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.387+0000 7fc1d7400640 10 client.7572 ms_handle_connect on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:07.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1d7400640 1 -- 172.21.15.62:0/1756788524 <== mds.0 v2:172.21.15.62:6833/3997398243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564e886b2500 con 0x564e887ace20 2024-04-25T09:13:07.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1d7400640 10 client.7572 handle_command_reply: tid=0 2024-04-25T09:13:07.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 client.7572 shutdown 2024-04-25T09:13:07.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/1756788524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88746d70 msgr2=0x564e887fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88746d70 0x564e887fe550 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800fb50 tx=0x7fc1e8004730 comp rx=0 tx=0).stop 2024-04-25T09:13:07.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/1756788524 shutdown_connections 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.62:6833/3997398243,v1:172.21.15.62:6835/3997398243] conn(0x564e887ace20 0x564e8876c6d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e88782700 0x564e887ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564e88783440 0x564e887821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 --2- 172.21.15.62:0/1756788524 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564e88746d70 0x564e887fe550 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1d7400640 0 client.7572 ms_handle_reset on v2:172.21.15.62:6833/3997398243 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/1756788524 >> 172.21.15.62:0/1756788524 conn(0x564e8870ea30 msgr2=0x564e8875d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/1756788524 shutdown_connections 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 1 -- 172.21.15.62:0/1756788524 wait complete. 2024-04-25T09:13:07.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.388+0000 7fc1ffbdf740 20 client.7572 trim_cache size 0 max 16384 2024-04-25T09:13:07.437 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:13:07.437 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-25T09:13:07.437 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:13:07.437 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:13:07.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 -- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1d806b4d0 msgr2=0x7fc1d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1d806b4d0 0x7fc1d806d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800a990 tx=0x7fc1e8008040 comp rx=0 tx=0).stop 2024-04-25T09:13:07.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 -- 172.21.15.62:0/159634113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f806d6a0 msgr2=0x7fc1f815add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f806d6a0 0x7fc1f815add0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c500 tx=0x7fc1e400c9c0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 -- 172.21.15.62:0/159634113 shutdown_connections 2024-04-25T09:13:07.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc1d806b4d0 0x7fc1d806d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc1f806d6a0 0x7fc1f815add0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc1f80b31a0 0x7fc1f815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 --2- 172.21.15.62:0/159634113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1f806d2c0 0x7fc1f815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 -- 172.21.15.62:0/159634113 >> 172.21.15.62:0/159634113 conn(0x7fc1f8066f20 msgr2=0x7fc1f80ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:07.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 -- 172.21.15.62:0/159634113 shutdown_connections 2024-04-25T09:13:07.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.389+0000 7fc1f7200640 1 -- 172.21.15.62:0/159634113 wait complete. 2024-04-25T09:13:07.441 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:13:07.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.673+0000 7f9ad0c00640 1 Processor -- start 2024-04-25T09:13:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.674+0000 7f9ad0c00640 1 -- start start 2024-04-25T09:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.675+0000 7f9ad0c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9acc0b6a90 0x7f9acc0d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.675+0000 7f9ad0c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0d3110 con 0x7f9acc0b27a0 2024-04-25T09:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.675+0000 7f9ad0c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc0d33f0 con 0x7f9acc0b56f0 2024-04-25T09:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.675+0000 7f9ad0c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0cedc0 con 0x7f9acc0b6a90 2024-04-25T09:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.675+0000 7f9acbe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9acc0b27a0 0x7f9acc0d2520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36980/0 (socket says 172.21.15.62:36980) 2024-04-25T09:13:07.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.675+0000 7f9acbe00640 1 -- 172.21.15.62:0/2448497840 learned_addr learned my addr 172.21.15.62:0/2448497840 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:07.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9ac3e00640 1 --2- 172.21.15.62:0/2448497840 >> v2:172.21.15.121:3300/0 conn(0x7f9acc0b6a90 0x7f9acc0d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344966324 0 0) 0x7f9acc0d3110 con 0x7f9acc0b27a0 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab0003520 con 0x7f9acc0b27a0 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 796086699 0 0) 0x7f9ab0003520 con 0x7f9acc0b27a0 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 >> v2:172.21.15.121:3300/0 conn(0x7f9acc0b6a90 msgr2=0x7f9acc0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 --2- 172.21.15.62:0/2448497840 >> v2:172.21.15.121:3300/0 conn(0x7f9acc0b6a90 0x7f9acc0d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 >> v1:172.21.15.121:6789/0 conn(0x7f9acc0b56f0 legacy=0x7f9acc160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0d36d0 con 0x7f9acc0b27a0 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f9ac40019f0 con 0x7f9acc0b27a0 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.676+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 >> v1:172.21.15.62:6789/0 conn(0x7f9acc0b27a0 legacy=0x7f9acc0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acaa00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ab0003a00 0x7f9ab0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acaa00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ab0004800 0x7f9ab0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acaa00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9ab000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0cedc0 con 0x7f9ab00051c0 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab000c0c0 con 0x7f9ab0003a00 2024-04-25T09:13:07.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab000c2c0 con 0x7f9ab0004800 2024-04-25T09:13:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acbe00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9ab000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acb400640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ab0003a00 0x7f9ab0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acbe00640 1 -- 172.21.15.62:0/2448497840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ab0004800 msgr2=0x7f9ab0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acbe00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ab0004800 0x7f9ab0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acbe00640 1 -- 172.21.15.62:0/2448497840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ab0003a00 msgr2=0x7f9ab0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acbe00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ab0003a00 0x7f9ab0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.677+0000 7f9acbe00640 1 -- 172.21.15.62:0/2448497840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab0003520 con 0x7f9ab00051c0 2024-04-25T09:13:07.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.678+0000 7f9acbe00640 1 -- 172.21.15.62:0/2448497840 reap_dead start 2024-04-25T09:13:07.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.678+0000 7f9acbe00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9ab000b9c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400a740 tx=0x7f9ab0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7777cedcd8f448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400a390 con 0x7f9ab00051c0 2024-04-25T09:13:07.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac402f070 con 0x7f9ab00051c0 2024-04-25T09:13:07.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9acaa00640 1 -- 172.21.15.62:0/2448497840 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac402bbc0 con 0x7f9ab00051c0 2024-04-25T09:13:07.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2448497840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 msgr2=0x7f9ab000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9ad0c00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9ab000b9c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400a740 tx=0x7f9ab0005830 comp rx=0 tx=0).stop 2024-04-25T09:13:07.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2448497840 shutdown_connections 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9ad0c00640 1 --2- 172.21.15.62:0/2448497840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9ab000b9c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2448497840 >> 172.21.15.62:0/2448497840 conn(0x7f9acc066d10 msgr2=0x7f9acc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2448497840 shutdown_connections 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.679+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2448497840 wait complete. 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.681+0000 7f9ad0c00640 1 Processor -- start 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.681+0000 7f9ad0c00640 1 -- start start 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.682+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9acc14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.682+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9acc14f4b0 0x7f9acc14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.682+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9acc14d570 0x7f9acc1527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.682+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b2ee0 con 0x7f9ab00051c0 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.682+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b2d60 con 0x7f9acc14d570 2024-04-25T09:13:07.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.682+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b5ad0 con 0x7f9acc14f4b0 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9acc14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9acc14ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56292/0 (socket says 172.21.15.62:56292) 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 -- 172.21.15.62:0/2142771340 learned_addr learned my addr 172.21.15.62:0/2142771340 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9ac3e00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9acc14f4b0 0x7f9acc14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acbe00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9acc14d570 0x7f9acc1527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 -- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9acc14f4b0 msgr2=0x7f9acc14d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9acc14f4b0 0x7f9acc14d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 -- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9acc14d570 msgr2=0x7f9acc1527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9acc14d570 0x7f9acc1527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 -- 172.21.15.62:0/2142771340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc14dda0 con 0x7f9ab00051c0 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.683+0000 7f9acb400640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9acc14ef70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e500 tx=0x7f9abc00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.684+0000 7f9ac3400640 1 -- 172.21.15.62:0/2142771340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0052e0 con 0x7f9ab00051c0 2024-04-25T09:13:07.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.684+0000 7f9ac3400640 1 -- 172.21.15.62:0/2142771340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc018070 con 0x7f9ab00051c0 2024-04-25T09:13:07.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.684+0000 7f9ac3400640 1 -- 172.21.15.62:0/2142771340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc003810 con 0x7f9ab00051c0 2024-04-25T09:13:07.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.684+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc14e080 con 0x7f9ab00051c0 2024-04-25T09:13:07.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.684+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc14e640 con 0x7f9ab00051c0 2024-04-25T09:13:07.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.685+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc04ef20 con 0x7f9ab00051c0 2024-04-25T09:13:07.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.686+0000 7f9ac3400640 1 -- 172.21.15.62:0/2142771340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9abc0039b0 con 0x7f9ab00051c0 2024-04-25T09:13:07.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.687+0000 7f9ac3400640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9aa806b4d0 0x7f9aa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:07.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.687+0000 7f9ac3400640 1 -- 172.21.15.62:0/2142771340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9abc089ea0 con 0x7f9ab00051c0 2024-04-25T09:13:07.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.687+0000 7f9ac3e00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9aa806b4d0 0x7f9aa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:07.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.689+0000 7f9ac3e00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9aa806b4d0 0x7f9aa806d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0cdbb0 tx=0x7f9ab4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:07.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.692+0000 7f9ac3400640 1 -- 172.21.15.62:0/2142771340 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9abc058c90 con 0x7f9ab00051c0 2024-04-25T09:13:07.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.933+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9acc0ce580 con 0x7f9ab00051c0 2024-04-25T09:13:07.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.934+0000 7f9ac3400640 1 -- 172.21.15.62:0/2142771340 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9abc058590 con 0x7f9ab00051c0 2024-04-25T09:13:07.921 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:07.921 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1798148826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1798148826},{"type":"v1","addr":"172.21.15.121:6835","nonce":1798148826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2045746846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2045746846},{"type":"v1","addr":"172.21.15.121:6837","nonce":2045746846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1182284820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1182284820},{"type":"v1","addr":"172.21.15.62:6834","nonce":1182284820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:22.810299+0000","modified":"2024-04-25T09:12:25.868942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3997398243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3997398243},{"type":"v1","addr":"172.21.15.62:6835","nonce":3997398243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:13:07.922 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 64 2024-04-25T09:13:07.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9aa806b4d0 msgr2=0x7f9aa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9aa806b4d0 0x7f9aa806d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0cdbb0 tx=0x7f9ab4008040 comp rx=0 tx=0).stop 2024-04-25T09:13:07.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 msgr2=0x7f9acc14ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:07.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9acc14ef70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e500 tx=0x7f9abc00e9c0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 shutdown_connections 2024-04-25T09:13:07.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9aa806b4d0 0x7f9aa806d990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9acc14d570 0x7f9acc1527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9acc14f4b0 0x7f9acc14d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 --2- 172.21.15.62:0/2142771340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ab00051c0 0x7f9acc14ef70 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.937+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 >> 172.21.15.62:0/2142771340 conn(0x7f9acc066d10 msgr2=0x7f9acc0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.938+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 shutdown_connections 2024-04-25T09:13:07.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:07.938+0000 7f9ad0c00640 1 -- 172.21.15.62:0/2142771340 wait complete. 2024-04-25T09:13:07.945 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:13:07.945 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:13:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.157+0000 7efe93600640 1 Processor -- start 2024-04-25T09:13:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.157+0000 7efe93600640 1 -- start start 2024-04-25T09:13:08.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.159+0000 7efe93600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efe8c15ee40 0x7efe8c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:08.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.159+0000 7efe93600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15f600 0x7efe8c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:08.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.159+0000 7efe93600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe8c071320 con 0x7efe8c15d660 2024-04-25T09:13:08.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.159+0000 7efe93600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8c15ffb0 con 0x7efe8c15ee40 2024-04-25T09:13:08.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.159+0000 7efe93600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c070d60 con 0x7efe8c15f600 2024-04-25T09:13:08.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.160+0000 7efe90c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efe8c15ee40 0x7efe8c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:08.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.160+0000 7efe91600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15f600 0x7efe8c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.160+0000 7efe90c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efe8c15ee40 0x7efe8c15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:42430/0 (socket says 172.21.15.62:42430) 2024-04-25T09:13:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.160+0000 7efe90c00640 1 -- 172.21.15.62:0/3576345294 learned_addr learned my addr 172.21.15.62:0/3576345294 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.160+0000 7efe90c00640 1 -- 172.21.15.62:0/3576345294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15f600 msgr2=0x7efe8c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.160+0000 7efe90c00640 1 --2- 172.21.15.62:0/3576345294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15f600 0x7efe8c15fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.160+0000 7efe90c00640 1 -- 172.21.15.62:0/3576345294 >> v1:172.21.15.121:6789/0 conn(0x7efe8c15d660 legacy=0x7efe8c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.161+0000 7efe90c00640 1 -- 172.21.15.62:0/3576345294 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c071040 con 0x7efe8c15ee40 2024-04-25T09:13:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.161+0000 7efe91600640 1 --2- 172.21.15.62:0/3576345294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15f600 0x7efe8c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:13:08.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.162+0000 7efe90c00640 1 --2- 172.21.15.62:0/3576345294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15ee40 0x7efe8c15d120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efe80009870 tx=0x7efe8002a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c55d3af9c229f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:08.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.162+0000 7efe8b400640 1 -- 172.21.15.62:0/3576345294 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80037070 con 0x7efe8c15ee40 2024-04-25T09:13:08.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.163+0000 7efe8b400640 1 -- 172.21.15.62:0/3576345294 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe80032440 con 0x7efe8c15ee40 2024-04-25T09:13:08.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.163+0000 7efe8b400640 1 -- 172.21.15.62:0/3576345294 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80032810 con 0x7efe8c15ee40 2024-04-25T09:13:08.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.163+0000 7efe93600640 1 -- 172.21.15.62:0/3576345294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15ee40 msgr2=0x7efe8c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:08.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.163+0000 7efe93600640 1 --2- 172.21.15.62:0/3576345294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15ee40 0x7efe8c15d120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efe80009870 tx=0x7efe8002a1d0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.163+0000 7efe93600640 1 -- 172.21.15.62:0/3576345294 shutdown_connections 2024-04-25T09:13:08.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.163+0000 7efe93600640 1 --2- 172.21.15.62:0/3576345294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15f600 0x7efe8c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.163+0000 7efe93600640 1 --2- 172.21.15.62:0/3576345294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15ee40 0x7efe8c15d120 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.163+0000 7efe93600640 1 -- 172.21.15.62:0/3576345294 >> 172.21.15.62:0/3576345294 conn(0x7efe8c0668f0 msgr2=0x7efe8c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:08.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.164+0000 7efe93600640 1 -- 172.21.15.62:0/3576345294 shutdown_connections 2024-04-25T09:13:08.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.164+0000 7efe93600640 1 -- 172.21.15.62:0/3576345294 wait complete. 2024-04-25T09:13:08.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.165+0000 7efe93600640 1 Processor -- start 2024-04-25T09:13:08.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.166+0000 7efe93600640 1 -- start start 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.166+0000 7efe93600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15d660 0x7efe8c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.166+0000 7efe93600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe8c15ee40 0x7efe8c1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.166+0000 7efe93600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15f600 0x7efe8c143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.166+0000 7efe93600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c1603d0 con 0x7efe8c15ee40 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.166+0000 7efe93600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0b2b00 con 0x7efe8c15f600 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.166+0000 7efe93600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0709d0 con 0x7efe8c15d660 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe8c15ee40 0x7efe8c1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe8c15ee40 0x7efe8c1530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56296/0 (socket says 172.21.15.62:56296) 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe8be00640 1 -- 172.21.15.62:0/1379436538 learned_addr learned my addr 172.21.15.62:0/1379436538 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe90c00640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15d660 0x7efe8c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe91600640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15f600 0x7efe8c143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:08.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe8be00640 1 -- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15d660 msgr2=0x7efe8c155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:08.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe8be00640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15d660 0x7efe8c155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe8be00640 1 -- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15f600 msgr2=0x7efe8c143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:08.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe8be00640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15f600 0x7efe8c143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.167+0000 7efe8be00640 1 -- 172.21.15.62:0/1379436538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c153810 con 0x7efe8c15ee40 2024-04-25T09:13:08.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.168+0000 7efe8be00640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe8c15ee40 0x7efe8c1530a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7efe7800c3c0 tx=0x7efe7800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:08.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.168+0000 7efe88c00640 1 -- 172.21.15.62:0/1379436538 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe780052d0 con 0x7efe8c15ee40 2024-04-25T09:13:08.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.168+0000 7efe88c00640 1 -- 172.21.15.62:0/1379436538 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe78005490 con 0x7efe8c15ee40 2024-04-25T09:13:08.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.168+0000 7efe88c00640 1 -- 172.21.15.62:0/1379436538 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78007260 con 0x7efe8c15ee40 2024-04-25T09:13:08.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.168+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c153af0 con 0x7efe8c15ee40 2024-04-25T09:13:08.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.168+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c1540b0 con 0x7efe8c15ee40 2024-04-25T09:13:08.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.169+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8c04ef20 con 0x7efe8c15ee40 2024-04-25T09:13:08.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.170+0000 7efe88c00640 1 -- 172.21.15.62:0/1379436538 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe78007400 con 0x7efe8c15ee40 2024-04-25T09:13:08.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.171+0000 7efe88c00640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe6c06b500 0x7efe6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:08.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.171+0000 7efe88c00640 1 -- 172.21.15.62:0/1379436538 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe780896d0 con 0x7efe8c15ee40 2024-04-25T09:13:08.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.172+0000 7efe90c00640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe6c06b500 0x7efe6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:08.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.173+0000 7efe90c00640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe6c06b500 0x7efe6c06d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efe80034000 tx=0x7efe8002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:08.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.176+0000 7efe88c00640 1 -- 172.21.15.62:0/1379436538 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe78058540 con 0x7efe8c15ee40 2024-04-25T09:13:08.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.456+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efe8c0704d0 con 0x7efe8c15ee40 2024-04-25T09:13:08.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.456+0000 7efe88c00640 1 -- 172.21.15.62:0/1379436538 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1026 (secure 0 0 0) 0x7efe780894d0 con 0x7efe8c15ee40 2024-04-25T09:13:08.443 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:08.443 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":615,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580778,"bytes_used":2578784256,"bytes_avail":770515329024,"bytes_total":773094113280,"write_bytes_sec":31271480,"read_op_per_sec":0,"write_op_per_sec":32},"fsmap":{"epoch":64,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7332}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T09:12:10.090849+0000","services":{}},"progress_events":{}} 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.459+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe6c06b500 msgr2=0x7efe6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.459+0000 7efe93600640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe6c06b500 0x7efe6c06d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efe80034000 tx=0x7efe8002e000 comp rx=0 tx=0).stop 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.459+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe8c15ee40 msgr2=0x7efe8c1530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.459+0000 7efe93600640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe8c15ee40 0x7efe8c1530a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7efe7800c3c0 tx=0x7efe7800c7c0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.460+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 shutdown_connections 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.460+0000 7efe93600640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe6c06b500 0x7efe6c06d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.460+0000 7efe93600640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe8c15ee40 0x7efe8c1530a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.460+0000 7efe93600640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe8c15f600 0x7efe8c143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.460+0000 7efe93600640 1 --2- 172.21.15.62:0/1379436538 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe8c15d660 0x7efe8c155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:08.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.460+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 >> 172.21.15.62:0/1379436538 conn(0x7efe8c0668f0 msgr2=0x7efe8c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:08.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.460+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 shutdown_connections 2024-04-25T09:13:08.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:08.460+0000 7efe93600640 1 -- 172.21.15.62:0/1379436538 wait complete. 2024-04-25T09:13:08.466 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:13:08.466 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:13:08.581 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:13:08.682 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:13:08.682 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:13:08.682 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:08.744 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:13:08.744 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:08.801 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:09.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.030+0000 7ffbb8c00640 1 Processor -- start 2024-04-25T09:13:09.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.031+0000 7ffbb8c00640 1 -- start start 2024-04-25T09:13:09.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.032+0000 7ffbb8c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffbb406d900 0x7ffbb415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.032+0000 7ffbb8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 0x7ffbb40b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.032+0000 7ffbb8c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb40cf2b0 con 0x7ffbb4144c10 2024-04-25T09:13:09.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.032+0000 7ffbb8c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb4144ff0 con 0x7ffbb406d900 2024-04-25T09:13:09.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.032+0000 7ffbb8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4160340 con 0x7ffbb40cee40 2024-04-25T09:13:09.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.033+0000 7ffbabe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffbb406d900 0x7ffbb415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.033+0000 7ffbabe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffbb406d900 0x7ffbb415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56306/0 (socket says 172.21.15.62:56306) 2024-04-25T09:13:09.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.033+0000 7ffbabe00640 1 -- 172.21.15.62:0/1020862297 learned_addr learned my addr 172.21.15.62:0/1020862297 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:09.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.033+0000 7ffbabe00640 1 -- 172.21.15.62:0/1020862297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 msgr2=0x7ffbb40b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:09.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.033+0000 7ffbabe00640 1 --2- 172.21.15.62:0/1020862297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 0x7ffbb40b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.033+0000 7ffbabe00640 1 -- 172.21.15.62:0/1020862297 >> v1:172.21.15.62:6789/0 conn(0x7ffbb4144c10 legacy=0x7ffbb406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.033+0000 7ffbabe00640 1 -- 172.21.15.62:0/1020862297 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb415faa0 con 0x7ffbb406d900 2024-04-25T09:13:09.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.034+0000 7ffbabe00640 1 --2- 172.21.15.62:0/1020862297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb406d900 0x7ffbb415f450 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009870 tx=0x7ffb9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e3695214a325cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.035+0000 7ffbb2a00640 1 -- 172.21.15.62:0/1020862297 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c037070 con 0x7ffbb406d900 2024-04-25T09:13:09.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.035+0000 7ffbb2a00640 1 -- 172.21.15.62:0/1020862297 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9c032440 con 0x7ffbb406d900 2024-04-25T09:13:09.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.035+0000 7ffbb2a00640 1 -- 172.21.15.62:0/1020862297 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c032810 con 0x7ffbb406d900 2024-04-25T09:13:09.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.035+0000 7ffbb8c00640 1 -- 172.21.15.62:0/1020862297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb406d900 msgr2=0x7ffbb415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.035+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/1020862297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb406d900 0x7ffbb415f450 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009870 tx=0x7ffb9c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.036+0000 7ffbb8c00640 1 -- 172.21.15.62:0/1020862297 shutdown_connections 2024-04-25T09:13:09.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.036+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/1020862297 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 0x7ffbb40b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.036+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/1020862297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb406d900 0x7ffbb415f450 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.036+0000 7ffbb8c00640 1 -- 172.21.15.62:0/1020862297 >> 172.21.15.62:0/1020862297 conn(0x7ffbb40b14d0 msgr2=0x7ffbb40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:09.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.036+0000 7ffbb8c00640 1 -- 172.21.15.62:0/1020862297 shutdown_connections 2024-04-25T09:13:09.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.036+0000 7ffbb8c00640 1 -- 172.21.15.62:0/1020862297 wait complete. 2024-04-25T09:13:09.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.037+0000 7ffbb8c00640 1 Processor -- start 2024-04-25T09:13:09.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.038+0000 7ffbb8c00640 1 -- start start 2024-04-25T09:13:09.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.038+0000 7ffbb8c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb406d900 0x7ffbb40c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.038+0000 7ffbb8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 0x7ffbb4070880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb4144c10 0x7ffbb415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4160060 con 0x7ffbb4144c10 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb8c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40b41c0 con 0x7ffbb406d900 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40b4b90 con 0x7ffbb40cee40 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb4144c10 0x7ffbb415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb4144c10 0x7ffbb415acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56322/0 (socket says 172.21.15.62:56322) 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3e00640 1 -- 172.21.15.62:0/2712212144 learned_addr learned my addr 172.21.15.62:0/2712212144 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbabe00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 0x7ffbb4070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3400640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb406d900 0x7ffbb40c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3e00640 1 -- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 msgr2=0x7ffbb4070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3e00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 0x7ffbb4070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3e00640 1 -- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb406d900 msgr2=0x7ffbb40c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3e00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb406d900 0x7ffbb40c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.039+0000 7ffbb3e00640 1 -- 172.21.15.62:0/2712212144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb41f5300 con 0x7ffbb4144c10 2024-04-25T09:13:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.040+0000 7ffbb3e00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb4144c10 0x7ffbb415acd0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00b970 tx=0x7ffbac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.040+0000 7ffbab400640 1 -- 172.21.15.62:0/2712212144 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac019070 con 0x7ffbb4144c10 2024-04-25T09:13:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.040+0000 7ffbab400640 1 -- 172.21.15.62:0/2712212144 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac00cb60 con 0x7ffbb4144c10 2024-04-25T09:13:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.040+0000 7ffbab400640 1 -- 172.21.15.62:0/2712212144 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac014440 con 0x7ffbb4144c10 2024-04-25T09:13:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.040+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb41f55e0 con 0x7ffbb4144c10 2024-04-25T09:13:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.040+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb41f5ba0 con 0x7ffbb4144c10 2024-04-25T09:13:09.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.041+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb404ef20 con 0x7ffbb4144c10 2024-04-25T09:13:09.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.042+0000 7ffbab400640 1 -- 172.21.15.62:0/2712212144 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbac01c070 con 0x7ffbb4144c10 2024-04-25T09:13:09.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.043+0000 7ffbab400640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9006b4d0 0x7ffb9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.043+0000 7ffbab400640 1 -- 172.21.15.62:0/2712212144 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbac089ff0 con 0x7ffbb4144c10 2024-04-25T09:13:09.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.044+0000 7ffbb3400640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9006b4d0 0x7ffb9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.045+0000 7ffbb3400640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9006b4d0 0x7ffb9006d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ffba4001500 tx=0x7ffba4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.048+0000 7ffbab400640 1 -- 172.21.15.62:0/2712212144 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbac058e30 con 0x7ffbb4144c10 2024-04-25T09:13:09.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.246+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbb40b1c30 con 0x7ffbb4144c10 2024-04-25T09:13:09.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.249+0000 7ffbab400640 1 -- 172.21.15.62:0/2712212144 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16660 (secure 0 0 0) 0x7ffbac058730 con 0x7ffbb4144c10 2024-04-25T09:13:09.236 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:09.236 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":97,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:24.344250+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:20.654480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:21.833814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:09.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9006b4d0 msgr2=0x7ffb9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9006b4d0 0x7ffb9006d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ffba4001500 tx=0x7ffba4009040 comp rx=0 tx=0).stop 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb4144c10 msgr2=0x7ffbb415acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb4144c10 0x7ffbb415acd0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00b970 tx=0x7ffbac00be30 comp rx=0 tx=0).stop 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 shutdown_connections 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffb9006b4d0 0x7ffb9006d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffbb4144c10 0x7ffbb415acd0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffbb40cee40 0x7ffbb4070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 --2- 172.21.15.62:0/2712212144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffbb406d900 0x7ffbb40c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 >> 172.21.15.62:0/2712212144 conn(0x7ffbb40b14d0 msgr2=0x7ffbb40b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 shutdown_connections 2024-04-25T09:13:09.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.252+0000 7ffbb8c00640 1 -- 172.21.15.62:0/2712212144 wait complete. 2024-04-25T09:13:09.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:12:20.654480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:12:21.833814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:13:09.258 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:13:09.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.480+0000 7fa7c6c00640 1 Processor -- start 2024-04-25T09:13:09.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.480+0000 7fa7c6c00640 1 -- start start 2024-04-25T09:13:09.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.481+0000 7fa7c6c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.481+0000 7fa7c6c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa7c00d3960 0x7fa7c00d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.481+0000 7fa7c6c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c00ce780 con 0x7fa7c00d2fb0 2024-04-25T09:13:09.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.481+0000 7fa7c6c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c00d3440 con 0x7fa7c00d3960 2024-04-25T09:13:09.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.481+0000 7fa7c6c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00ce1c0 con 0x7fa7c015dd50 2024-04-25T09:13:09.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7c6200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa7c00d3960 0x7fa7c00d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7c4e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa7c00d2fb0 0x7fa7c00ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36992/0 (socket says 172.21.15.62:36992) 2024-04-25T09:13:09.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7c4e00640 1 -- 172.21.15.62:0/2926983473 learned_addr learned my addr 172.21.15.62:0/2926983473 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:09.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7c5800640 1 --2- 172.21.15.62:0/2926983473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7bfe00640 1 -- 172.21.15.62:0/2926983473 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091816355 0 0) 0x7fa7c00ce780 con 0x7fa7c00d2fb0 2024-04-25T09:13:09.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7c6200640 1 -- 172.21.15.62:0/2926983473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 msgr2=0x7fa7c00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:09.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7c6200640 1 --2- 172.21.15.62:0/2926983473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c00d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7c6200640 1 -- 172.21.15.62:0/2926983473 >> v1:172.21.15.62:6789/0 conn(0x7fa7c00d2fb0 legacy=0x7fa7c00ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.482+0000 7fa7c6200640 1 -- 172.21.15.62:0/2926983473 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c00d4310 con 0x7fa7c00d3960 2024-04-25T09:13:09.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.484+0000 7fa7c6200640 1 --2- 172.21.15.62:0/2926983473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c00d3dd0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009d90 tx=0x7fa7b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f164f351efb51573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.484+0000 7fa7bfe00640 1 -- 172.21.15.62:0/2926983473 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8007740 con 0x7fa7c00d3960 2024-04-25T09:13:09.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.484+0000 7fa7bfe00640 1 -- 172.21.15.62:0/2926983473 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b80078e0 con 0x7fa7c00d3960 2024-04-25T09:13:09.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.484+0000 7fa7bfe00640 1 -- 172.21.15.62:0/2926983473 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8007cb0 con 0x7fa7c00d3960 2024-04-25T09:13:09.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.485+0000 7fa7c6c00640 1 -- 172.21.15.62:0/2926983473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 msgr2=0x7fa7c00d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.485+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/2926983473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c00d3dd0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009d90 tx=0x7fa7b802a660 comp rx=0 tx=0).stop 2024-04-25T09:13:09.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.485+0000 7fa7c6c00640 1 -- 172.21.15.62:0/2926983473 shutdown_connections 2024-04-25T09:13:09.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.485+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/2926983473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c00d3dd0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.485+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/2926983473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c00d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.485+0000 7fa7c6c00640 1 -- 172.21.15.62:0/2926983473 >> 172.21.15.62:0/2926983473 conn(0x7fa7c006b000 msgr2=0x7fa7c006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:09.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.485+0000 7fa7c6c00640 1 -- 172.21.15.62:0/2926983473 shutdown_connections 2024-04-25T09:13:09.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.485+0000 7fa7c6c00640 1 -- 172.21.15.62:0/2926983473 wait complete. 2024-04-25T09:13:09.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.487+0000 7fa7c6c00640 1 Processor -- start 2024-04-25T09:13:09.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.487+0000 7fa7c6c00640 1 -- start start 2024-04-25T09:13:09.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.488+0000 7fa7c6c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c0153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.488+0000 7fa7c6c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c015f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.488+0000 7fa7c6c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c01f4550 0x7fa7c015f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.488+0000 7fa7c6c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00b6b00 con 0x7fa7c00d3960 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.488+0000 7fa7c6c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00d2840 con 0x7fa7c01f4550 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.488+0000 7fa7c6c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c00b6c80 con 0x7fa7c015dd50 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c0153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c0153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56336/0 (socket says 172.21.15.62:56336) 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c5800640 1 -- 172.21.15.62:0/1378835711 learned_addr learned my addr 172.21.15.62:0/1378835711 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c4e00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c015f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c6200640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c01f4550 0x7fa7c015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c5800640 1 -- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 msgr2=0x7fa7c015f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c5800640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c015f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c5800640 1 -- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c01f4550 msgr2=0x7fa7c015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c5800640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c01f4550 0x7fa7c015f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c5800640 1 -- 172.21.15.62:0/1378835711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c015d180 con 0x7fa7c00d3960 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c4e00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c015f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.489+0000 7fa7c6200640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c01f4550 0x7fa7c015f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.490+0000 7fa7c5800640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c0153e70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d5f0 tx=0x7fa7b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.490+0000 7fa7bd600640 1 -- 172.21.15.62:0/1378835711 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40125f0 con 0x7fa7c00d3960 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.490+0000 7fa7bd600640 1 -- 172.21.15.62:0/1378835711 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b4019070 con 0x7fa7c00d3960 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.490+0000 7fa7bd600640 1 -- 172.21.15.62:0/1378835711 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40129d0 con 0x7fa7c00d3960 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.490+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c00b3d50 con 0x7fa7c00d3960 2024-04-25T09:13:09.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.490+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c01f3d50 con 0x7fa7c00d3960 2024-04-25T09:13:09.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.491+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c004ef20 con 0x7fa7c00d3960 2024-04-25T09:13:09.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.492+0000 7fa7bd600640 1 -- 172.21.15.62:0/1378835711 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7b4012b70 con 0x7fa7c00d3960 2024-04-25T09:13:09.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.494+0000 7fa7bd600640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa7a006b4d0 0x7fa7a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.494+0000 7fa7bd600640 1 -- 172.21.15.62:0/1378835711 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7b408a690 con 0x7fa7c00d3960 2024-04-25T09:13:09.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.494+0000 7fa7c4e00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa7a006b4d0 0x7fa7a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.496+0000 7fa7c4e00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa7a006b4d0 0x7fa7a006d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00b5b20 tx=0x7fa7ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.498+0000 7fa7bd600640 1 -- 172.21.15.62:0/1378835711 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7b4059470 con 0x7fa7c00d3960 2024-04-25T09:13:09.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.699+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7c00b9950 con 0x7fa7c00d3960 2024-04-25T09:13:09.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.700+0000 7fa7bd600640 1 -- 172.21.15.62:0/1378835711 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7b4058d70 con 0x7fa7c00d3960 2024-04-25T09:13:09.687 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa7a006b4d0 msgr2=0x7fa7a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa7a006b4d0 0x7fa7a006d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00b5b20 tx=0x7fa7ac009040 comp rx=0 tx=0).stop 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 msgr2=0x7fa7c0153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c0153e70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d5f0 tx=0x7fa7b400dab0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 shutdown_connections 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa7a006b4d0 0x7fa7a006d990 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c01f4550 0x7fa7c015f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7c015dd50 0x7fa7c015f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 --2- 172.21.15.62:0/1378835711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7c00d3960 0x7fa7c0153e70 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 >> 172.21.15.62:0/1378835711 conn(0x7fa7c006b000 msgr2=0x7fa7c006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:09.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 shutdown_connections 2024-04-25T09:13:09.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.703+0000 7fa7c6c00640 1 -- 172.21.15.62:0/1378835711 wait complete. 2024-04-25T09:13:09.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:13:09.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.939+0000 7f14f5a00640 1 Processor -- start 2024-04-25T09:13:09.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.939+0000 7f14f5a00640 1 -- start start 2024-04-25T09:13:09.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.942+0000 7f14f5a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14f006d2c0 0x7f14f00ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.942+0000 7f14f5a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14f00b31a0 0x7f14f00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.942+0000 7f14f5a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f00cf000 con 0x7f14f006d6a0 2024-04-25T09:13:09.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.942+0000 7f14f5a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f00cf2e0 con 0x7f14f006d2c0 2024-04-25T09:13:09.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.942+0000 7f14f5a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b26c0 con 0x7f14f00b31a0 2024-04-25T09:13:09.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.942+0000 7f14ef400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f14f006d6a0 0x7f14f00ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36996/0 (socket says 172.21.15.62:36996) 2024-04-25T09:13:09.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.942+0000 7f14ef400640 1 -- 172.21.15.62:0/2364422816 learned_addr learned my addr 172.21.15.62:0/2364422816 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.943+0000 7f14f5000640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14f00b31a0 0x7f14f00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.943+0000 7f14efe00640 1 --2- 172.21.15.62:0/2364422816 >> v2:172.21.15.121:3300/0 conn(0x7f14f006d2c0 0x7f14f00ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.943+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780785832 0 0) 0x7f14f00cf000 con 0x7f14f006d6a0 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.943+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d4003520 con 0x7f14f006d6a0 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.943+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2085644193 0 0) 0x7f14d4003520 con 0x7f14f006d6a0 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14f00b31a0 msgr2=0x7f14f00b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14f00b31a0 0x7f14f00b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 >> v2:172.21.15.121:3300/0 conn(0x7f14f006d2c0 msgr2=0x7f14f00ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 --2- 172.21.15.62:0/2364422816 >> v2:172.21.15.121:3300/0 conn(0x7f14f006d2c0 0x7f14f00ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f00b29a0 con 0x7f14f006d6a0 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f14dc001670 con 0x7f14f006d6a0 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 >> v1:172.21.15.62:6789/0 conn(0x7f14f006d6a0 legacy=0x7f14f00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d40039b0 0x7f14d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14f00b31a0 0x7f14d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d4004d40 0x7f14d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00cf2e0 con 0x7f14d40039b0 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b26c0 con 0x7f14d4004d40 2024-04-25T09:13:09.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d400b9c0 con 0x7f14f00b31a0 2024-04-25T09:13:09.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14efe00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d40039b0 0x7f14d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14ef400640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14f00b31a0 0x7f14d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.944+0000 7f14f5000640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d4004d40 0x7f14d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.945+0000 7f14efe00640 1 -- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14f00b31a0 msgr2=0x7f14d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.945+0000 7f14efe00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14f00b31a0 0x7f14d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.945+0000 7f14efe00640 1 -- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d4004d40 msgr2=0x7f14d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.945+0000 7f14efe00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d4004d40 0x7f14d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.945+0000 7f14efe00640 1 -- 172.21.15.62:0/2364422816 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d4003520 con 0x7f14d40039b0 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.946+0000 7f14efe00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d40039b0 0x7f14d4003dd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ac0 tx=0x7f14d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbce444756ac429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.946+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0036070 con 0x7f14d40039b0 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.946+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e0031400 con 0x7f14d40039b0 2024-04-25T09:13:09.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.946+0000 7f14eea00640 1 -- 172.21.15.62:0/2364422816 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0009690 con 0x7f14d40039b0 2024-04-25T09:13:09.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.947+0000 7f14f5a00640 1 -- 172.21.15.62:0/2364422816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d40039b0 msgr2=0x7f14d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.947+0000 7f14f5a00640 1 --2- 172.21.15.62:0/2364422816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14d40039b0 0x7f14d4003dd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009ac0 tx=0x7f14d4005830 comp rx=0 tx=0).stop 2024-04-25T09:13:09.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.947+0000 7f14f5000640 1 -- 172.21.15.62:0/2364422816 reap_dead start 2024-04-25T09:13:09.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.947+0000 7f14f5a00640 1 -- 172.21.15.62:0/2364422816 shutdown_connections 2024-04-25T09:13:09.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.947+0000 7f14f5a00640 1 -- 172.21.15.62:0/2364422816 >> 172.21.15.62:0/2364422816 conn(0x7f14f0066f20 msgr2=0x7f14f00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:09.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.947+0000 7f14f5a00640 1 -- 172.21.15.62:0/2364422816 shutdown_connections 2024-04-25T09:13:09.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.947+0000 7f14f5a00640 1 -- 172.21.15.62:0/2364422816 wait complete. 2024-04-25T09:13:09.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.949+0000 7f14f5a00640 1 Processor -- start 2024-04-25T09:13:09.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.949+0000 7f14f5a00640 1 -- start start 2024-04-25T09:13:09.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14d40039b0 0x7f14f015b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14f0137750 0x7f14f015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f0137bd0 0x7f14f0157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f01448d0 con 0x7f14f0137bd0 2024-04-25T09:13:09.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f006db20 con 0x7f14f0137750 2024-04-25T09:13:09.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f0144a50 con 0x7f14d40039b0 2024-04-25T09:13:09.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f0137bd0 0x7f14f0157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f0137bd0 0x7f14f0157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56350/0 (socket says 172.21.15.62:56350) 2024-04-25T09:13:09.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.950+0000 7f14f5000640 1 -- 172.21.15.62:0/3005898384 learned_addr learned my addr 172.21.15.62:0/3005898384 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:09.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.951+0000 7f14efe00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14d40039b0 0x7f14f015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.951+0000 7f14ef400640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14f0137750 0x7f14f015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.951+0000 7f14f5000640 1 -- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14d40039b0 msgr2=0x7f14f015b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.951+0000 7f14f5000640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14d40039b0 0x7f14f015b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.951+0000 7f14f5000640 1 -- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14f0137750 msgr2=0x7f14f015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.951+0000 7f14f5000640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14f0137750 0x7f14f015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.951+0000 7f14f5000640 1 -- 172.21.15.62:0/3005898384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0156ed0 con 0x7f14f0137bd0 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.952+0000 7f14f5000640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f0137bd0 0x7f14f0157c90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f14f006e560 tx=0x7f14e80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.952+0000 7f14e7e00640 1 -- 172.21.15.62:0/3005898384 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011e10 con 0x7f14f0137bd0 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.952+0000 7f14e7e00640 1 -- 172.21.15.62:0/3005898384 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e801b070 con 0x7f14f0137bd0 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.952+0000 7f14e7e00640 1 -- 172.21.15.62:0/3005898384 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8003990 con 0x7f14f0137bd0 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.953+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f01571b0 con 0x7f14f0137bd0 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.953+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f0157770 con 0x7f14f0137bd0 2024-04-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.954+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f006abe0 con 0x7f14f0137bd0 2024-04-25T09:13:09.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.955+0000 7f14e7e00640 1 -- 172.21.15.62:0/3005898384 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14e8005d10 con 0x7f14f0137bd0 2024-04-25T09:13:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.956+0000 7f14e7e00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14d006b4d0 0x7f14d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.956+0000 7f14e7e00640 1 -- 172.21.15.62:0/3005898384 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14e808a6b0 con 0x7f14f0137bd0 2024-04-25T09:13:09.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.957+0000 7f14efe00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14d006b4d0 0x7f14d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:09.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.958+0000 7f14efe00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14d006b4d0 0x7f14d006d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f14e0033000 tx=0x7f14e0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:09.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:09.961+0000 7f14e7e00640 1 -- 172.21.15.62:0/3005898384 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14e8025020 con 0x7f14f0137bd0 2024-04-25T09:13:10.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.159+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14f0157970 con 0x7f14f0137bd0 2024-04-25T09:13:10.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.160+0000 7f14e7e00640 1 -- 172.21.15.62:0/3005898384 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14f0157970 con 0x7f14f0137bd0 2024-04-25T09:13:10.147 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14d006b4d0 msgr2=0x7f14d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14d006b4d0 0x7f14d006d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f14e0033000 tx=0x7f14e0008030 comp rx=0 tx=0).stop 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f0137bd0 msgr2=0x7f14f0157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f0137bd0 0x7f14f0157c90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f14f006e560 tx=0x7f14e80115a0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 shutdown_connections 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14d006b4d0 0x7f14d006d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14f0137bd0 0x7f14f0157c90 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14f0137750 0x7f14f015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 --2- 172.21.15.62:0/3005898384 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14d40039b0 0x7f14f015b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.163+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 >> 172.21.15.62:0/3005898384 conn(0x7f14f0066f20 msgr2=0x7f14f00702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:10.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.164+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 shutdown_connections 2024-04-25T09:13:10.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.164+0000 7f14f5a00640 1 -- 172.21.15.62:0/3005898384 wait complete. 2024-04-25T09:13:10.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:13:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.399+0000 7f1972000640 1 Processor -- start 2024-04-25T09:13:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.399+0000 7f1972000640 1 -- start start 2024-04-25T09:13:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.400+0000 7f1972000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f196c0d2fb0 0x7f196c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.400+0000 7f1972000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f196c0d3960 0x7f196c0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.400+0000 7f1972000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0ce780 con 0x7f196c15dd50 2024-04-25T09:13:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.400+0000 7f1972000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0d42f0 con 0x7f196c0d2fb0 2024-04-25T09:13:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.400+0000 7f1972000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0d3440 con 0x7f196c0d3960 2024-04-25T09:13:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.400+0000 7f196be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f196c0d2fb0 0x7f196c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.400+0000 7f196be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f196c0d2fb0 0x7f196c0ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56362/0 (socket says 172.21.15.62:56362) 2024-04-25T09:13:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.400+0000 7f196be00640 1 -- 172.21.15.62:0/1872766844 learned_addr learned my addr 172.21.15.62:0/1872766844 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.401+0000 7f1971600640 1 --2- 172.21.15.62:0/1872766844 >> v2:172.21.15.121:3300/0 conn(0x7f196c0d3960 0x7f196c0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.401+0000 7f196be00640 1 -- 172.21.15.62:0/1872766844 >> v2:172.21.15.121:3300/0 conn(0x7f196c0d3960 msgr2=0x7f196c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.401+0000 7f196be00640 1 --2- 172.21.15.62:0/1872766844 >> v2:172.21.15.121:3300/0 conn(0x7f196c0d3960 0x7f196c0d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.401+0000 7f196be00640 1 -- 172.21.15.62:0/1872766844 >> v1:172.21.15.121:6789/0 conn(0x7f196c15dd50 legacy=0x7f196c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:10.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.401+0000 7f196be00640 1 -- 172.21.15.62:0/1872766844 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0ce1c0 con 0x7f196c0d2fb0 2024-04-25T09:13:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.402+0000 7f196be00640 1 --2- 172.21.15.62:0/1872766844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0d2fb0 0x7f196c0ce9c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f196c0caa50 tx=0x7f195c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7613a3f2ae947e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.402+0000 7f196b400640 1 -- 172.21.15.62:0/1872766844 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c036070 con 0x7f196c0d2fb0 2024-04-25T09:13:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.402+0000 7f196b400640 1 -- 172.21.15.62:0/1872766844 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c031600 con 0x7f196c0d2fb0 2024-04-25T09:13:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.402+0000 7f196b400640 1 -- 172.21.15.62:0/1872766844 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0319d0 con 0x7f196c0d2fb0 2024-04-25T09:13:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.403+0000 7f1972000640 1 -- 172.21.15.62:0/1872766844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0d2fb0 msgr2=0x7f196c0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.403+0000 7f1972000640 1 --2- 172.21.15.62:0/1872766844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0d2fb0 0x7f196c0ce9c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f196c0caa50 tx=0x7f195c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.403+0000 7f1972000640 1 -- 172.21.15.62:0/1872766844 shutdown_connections 2024-04-25T09:13:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.403+0000 7f1972000640 1 --2- 172.21.15.62:0/1872766844 >> v2:172.21.15.121:3300/0 conn(0x7f196c0d3960 0x7f196c0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.403+0000 7f1972000640 1 --2- 172.21.15.62:0/1872766844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c0d2fb0 0x7f196c0ce9c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.403+0000 7f1972000640 1 -- 172.21.15.62:0/1872766844 >> 172.21.15.62:0/1872766844 conn(0x7f196c06b000 msgr2=0x7f196c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.403+0000 7f1972000640 1 -- 172.21.15.62:0/1872766844 shutdown_connections 2024-04-25T09:13:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.403+0000 7f1972000640 1 -- 172.21.15.62:0/1872766844 wait complete. 2024-04-25T09:13:10.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.405+0000 7f1972000640 1 Processor -- start 2024-04-25T09:13:10.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.405+0000 7f1972000640 1 -- start start 2024-04-25T09:13:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f1972000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f196c0d3960 0x7f196c0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f1972000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c15dd50 0x7f196c1e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f1972000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f196c1e9a10 0x7f196c15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f1972000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0b6b00 con 0x7f196c15dd50 2024-04-25T09:13:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f1972000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0cef00 con 0x7f196c0d3960 2024-04-25T09:13:10.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f1972000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0b6c80 con 0x7f196c1e9a10 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f196be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c15dd50 0x7f196c1e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f196be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c15dd50 0x7f196c1e94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56378/0 (socket says 172.21.15.62:56378) 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f196be00640 1 -- 172.21.15.62:0/1285018692 learned_addr learned my addr 172.21.15.62:0/1285018692 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.406+0000 7f1970c00640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f196c0d3960 0x7f196c0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f1971600640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f196c1e9a10 0x7f196c15fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f196be00640 1 -- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f196c1e9a10 msgr2=0x7f196c15fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f196be00640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f196c1e9a10 0x7f196c15fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f196be00640 1 -- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f196c0d3960 msgr2=0x7f196c0c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f196be00640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f196c0d3960 0x7f196c0c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f196be00640 1 -- 172.21.15.62:0/1285018692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c15e320 con 0x7f196c15dd50 2024-04-25T09:13:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f196be00640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c15dd50 0x7f196c1e94d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f195c000f80 tx=0x7f195c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f1968c00640 1 -- 172.21.15.62:0/1285018692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c036050 con 0x7f196c15dd50 2024-04-25T09:13:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f1968c00640 1 -- 172.21.15.62:0/1285018692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c042070 con 0x7f196c15dd50 2024-04-25T09:13:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f1968c00640 1 -- 172.21.15.62:0/1285018692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c009510 con 0x7f196c15dd50 2024-04-25T09:13:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.407+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c15e600 con 0x7f196c15dd50 2024-04-25T09:13:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.408+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c15ebc0 con 0x7f196c15dd50 2024-04-25T09:13:10.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.409+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c15eef0 con 0x7f196c15dd50 2024-04-25T09:13:10.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.410+0000 7f1968c00640 1 -- 172.21.15.62:0/1285018692 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f195c007c70 con 0x7f196c15dd50 2024-04-25T09:13:10.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.410+0000 7f1968c00640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f194806b500 0x7f194806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.411+0000 7f1968c00640 1 -- 172.21.15.62:0/1285018692 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f195c0ab0f0 con 0x7f196c15dd50 2024-04-25T09:13:10.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.411+0000 7f1970c00640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f194806b500 0x7f194806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.412+0000 7f1970c00640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f194806b500 0x7f194806d9c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1954006fd0 tx=0x7f1954008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:10.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.417+0000 7f1968c00640 1 -- 172.21.15.62:0/1285018692 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f195c045360 con 0x7f196c15dd50 2024-04-25T09:13:10.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.618+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f196c04ef20 con 0x7f196c15dd50 2024-04-25T09:13:10.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.619+0000 7f1968c00640 1 -- 172.21.15.62:0/1285018692 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f195c079f70 con 0x7f196c15dd50 2024-04-25T09:13:10.606 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f194806b500 msgr2=0x7f194806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f194806b500 0x7f194806d9c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1954006fd0 tx=0x7f1954008040 comp rx=0 tx=0).stop 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c15dd50 msgr2=0x7f196c1e94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c15dd50 0x7f196c1e94d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f195c000f80 tx=0x7f195c004a40 comp rx=0 tx=0).stop 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 shutdown_connections 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f194806b500 0x7f194806d9c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f196c15dd50 0x7f196c1e94d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f196c1e9a10 0x7f196c15fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 --2- 172.21.15.62:0/1285018692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f196c0d3960 0x7f196c0c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 >> 172.21.15.62:0/1285018692 conn(0x7f196c06b000 msgr2=0x7f196c0c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:10.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 shutdown_connections 2024-04-25T09:13:10.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.622+0000 7f1972000640 1 -- 172.21.15.62:0/1285018692 wait complete. 2024-04-25T09:13:10.628 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:13:10.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.853+0000 7fe53d600640 1 Processor -- start 2024-04-25T09:13:10.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.853+0000 7fe53d600640 1 -- start start 2024-04-25T09:13:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.855+0000 7fe53d600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe5380cfed0 0x7fe53815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.855+0000 7fe53d600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5380709c0 0x7fe5380b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.855+0000 7fe53d600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53806c310 con 0x7fe538072340 2024-04-25T09:13:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.855+0000 7fe53d600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53815f5f0 con 0x7fe5380709c0 2024-04-25T09:13:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.855+0000 7fe53d600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53806c5b0 con 0x7fe5380cfed0 2024-04-25T09:13:10.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.855+0000 7fe537400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5380709c0 0x7fe5380b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.856+0000 7fe537400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe5380709c0 0x7fe5380b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56386/0 (socket says 172.21.15.62:56386) 2024-04-25T09:13:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.856+0000 7fe537400640 1 -- 172.21.15.62:0/1638701157 learned_addr learned my addr 172.21.15.62:0/1638701157 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.856+0000 7fe536a00640 1 --2- 172.21.15.62:0/1638701157 >> v2:172.21.15.121:3300/0 conn(0x7fe5380cfed0 0x7fe53815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.856+0000 7fe537400640 1 -- 172.21.15.62:0/1638701157 >> v2:172.21.15.121:3300/0 conn(0x7fe5380cfed0 msgr2=0x7fe53815f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.856+0000 7fe537400640 1 --2- 172.21.15.62:0/1638701157 >> v2:172.21.15.121:3300/0 conn(0x7fe5380cfed0 0x7fe53815f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.856+0000 7fe537400640 1 -- 172.21.15.62:0/1638701157 >> v1:172.21.15.121:6789/0 conn(0x7fe538072340 legacy=0x7fe5380702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:10.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.856+0000 7fe537400640 1 -- 172.21.15.62:0/1638701157 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53806c890 con 0x7fe5380709c0 2024-04-25T09:13:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.857+0000 7fe537400640 1 --2- 172.21.15.62:0/1638701157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5380709c0 0x7fe5380b2880 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe5380cd560 tx=0x7fe53002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354d92229a349f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe535600640 1 -- 172.21.15.62:0/1638701157 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530009600 con 0x7fe5380709c0 2024-04-25T09:13:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe535600640 1 -- 172.21.15.62:0/1638701157 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5300097a0 con 0x7fe5380709c0 2024-04-25T09:13:10.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe535600640 1 -- 172.21.15.62:0/1638701157 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530009b70 con 0x7fe5380709c0 2024-04-25T09:13:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe53d600640 1 -- 172.21.15.62:0/1638701157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5380709c0 msgr2=0x7fe5380b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe53d600640 1 --2- 172.21.15.62:0/1638701157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5380709c0 0x7fe5380b2880 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe5380cd560 tx=0x7fe53002c900 comp rx=0 tx=0).stop 2024-04-25T09:13:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe53d600640 1 -- 172.21.15.62:0/1638701157 shutdown_connections 2024-04-25T09:13:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe53d600640 1 --2- 172.21.15.62:0/1638701157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe5380709c0 0x7fe5380b2880 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe53d600640 1 --2- 172.21.15.62:0/1638701157 >> v2:172.21.15.121:3300/0 conn(0x7fe5380cfed0 0x7fe53815f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.858+0000 7fe53d600640 1 -- 172.21.15.62:0/1638701157 >> 172.21.15.62:0/1638701157 conn(0x7fe5380668f0 msgr2=0x7fe538066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.859+0000 7fe53d600640 1 -- 172.21.15.62:0/1638701157 shutdown_connections 2024-04-25T09:13:10.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.859+0000 7fe53d600640 1 -- 172.21.15.62:0/1638701157 wait complete. 2024-04-25T09:13:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.860+0000 7fe53d600640 1 Processor -- start 2024-04-25T09:13:10.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.861+0000 7fe53d600640 1 -- start start 2024-04-25T09:13:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.861+0000 7fe53d600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5380709c0 0x7fe53815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.861+0000 7fe53d600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5380cfed0 0x7fe5381e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.861+0000 7fe53d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe538134550 0x7fe538159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.861+0000 7fe53d600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53806d130 con 0x7fe538134550 2024-04-25T09:13:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.861+0000 7fe53d600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53806ce50 con 0x7fe5380cfed0 2024-04-25T09:13:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.861+0000 7fe53d600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380b2dc0 con 0x7fe5380709c0 2024-04-25T09:13:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe537400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe538134550 0x7fe538159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe537400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe538134550 0x7fe538159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56402/0 (socket says 172.21.15.62:56402) 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe537400640 1 -- 172.21.15.62:0/344253469 learned_addr learned my addr 172.21.15.62:0/344253469 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe536a00640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5380709c0 0x7fe53815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe536000640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5380cfed0 0x7fe5381e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe537400640 1 -- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5380709c0 msgr2=0x7fe53815d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe537400640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5380709c0 0x7fe53815d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe537400640 1 -- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5380cfed0 msgr2=0x7fe5381e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe537400640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5380cfed0 0x7fe5381e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe537400640 1 -- 172.21.15.62:0/344253469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe538134af0 con 0x7fe538134550 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.862+0000 7fe536000640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5380cfed0 0x7fe5381e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:10.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.863+0000 7fe537400640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe538134550 0x7fe538159c00 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe53002cef0 tx=0x7fe530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.863+0000 7fe52ea00640 1 -- 172.21.15.62:0/344253469 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530042070 con 0x7fe538134550 2024-04-25T09:13:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.863+0000 7fe52ea00640 1 -- 172.21.15.62:0/344253469 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe530034c60 con 0x7fe538134550 2024-04-25T09:13:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.863+0000 7fe52ea00640 1 -- 172.21.15.62:0/344253469 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530009600 con 0x7fe538134550 2024-04-25T09:13:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.863+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe538134dd0 con 0x7fe538134550 2024-04-25T09:13:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.863+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe538135390 con 0x7fe538134550 2024-04-25T09:13:10.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.864+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5380b1080 con 0x7fe538134550 2024-04-25T09:13:10.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.865+0000 7fe52ea00640 1 -- 172.21.15.62:0/344253469 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe530045070 con 0x7fe538134550 2024-04-25T09:13:10.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.866+0000 7fe52ea00640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe51806b4d0 0x7fe51806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:10.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.866+0000 7fe52ea00640 1 -- 172.21.15.62:0/344253469 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5300ab390 con 0x7fe538134550 2024-04-25T09:13:10.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.867+0000 7fe536a00640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe51806b4d0 0x7fe51806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:10.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.868+0000 7fe536a00640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe51806b4d0 0x7fe51806d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe528006fd0 tx=0x7fe528008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:10.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:10.871+0000 7fe52ea00640 1 -- 172.21.15.62:0/344253469 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5380b1080 con 0x7fe538134550 2024-04-25T09:13:11.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.083+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe538072c70 con 0x7fe538134550 2024-04-25T09:13:11.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.085+0000 7fe52ea00640 1 -- 172.21.15.62:0/344253469 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe53007a210 con 0x7fe538134550 2024-04-25T09:13:11.071 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:13:11.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.087+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe51806b4d0 msgr2=0x7fe51806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.087+0000 7fe53d600640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe51806b4d0 0x7fe51806d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe528006fd0 tx=0x7fe528008040 comp rx=0 tx=0).stop 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe538134550 msgr2=0x7fe538159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe538134550 0x7fe538159c00 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe53002cef0 tx=0x7fe530004a40 comp rx=0 tx=0).stop 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 shutdown_connections 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe51806b4d0 0x7fe51806d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe538134550 0x7fe538159c00 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5380cfed0 0x7fe5381e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 --2- 172.21.15.62:0/344253469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe5380709c0 0x7fe53815d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 >> 172.21.15.62:0/344253469 conn(0x7fe5380668f0 msgr2=0x7fe5380ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 shutdown_connections 2024-04-25T09:13:11.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:11.088+0000 7fe53d600640 1 -- 172.21.15.62:0/344253469 wait complete. 2024-04-25T09:13:11.094 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:13:11.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:11.095 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:13:12.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:13:12.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:12.204 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:13:12.245 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:13:12.245 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:12.288 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:13:12.288 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:13:12.288 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:12.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:13:12.331 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:13:12.331 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:12.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:13:12.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.611+0000 7f33cf200640 1 Processor -- start 2024-04-25T09:13:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.611+0000 7f33cf200640 1 -- start start 2024-04-25T09:13:12.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.613+0000 7f33cf200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f33c815bd10 0x7f33c80cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:12.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.613+0000 7f33cf200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f33c80cccf0 0x7f33c80ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:12.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.613+0000 7f33cf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80aed00 0x7f33c815f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.613+0000 7f33cf200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c806d660 con 0x7f33c815bd10 2024-04-25T09:13:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.613+0000 7f33cf200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c806ca70 con 0x7f33c80cccf0 2024-04-25T09:13:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.613+0000 7f33cf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c815c0f0 con 0x7f33c80aed00 2024-04-25T09:13:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f33c815bd10 0x7f33c80cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f33c815bd10 0x7f33c80cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56414/0 (socket says 172.21.15.62:56414) 2024-04-25T09:13:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7e00640 1 -- 172.21.15.62:0/320963357 learned_addr learned my addr 172.21.15.62:0/320963357 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7400640 1 --2- 172.21.15.62:0/320963357 >> v2:172.21.15.121:3300/0 conn(0x7f33c80cccf0 0x7f33c80ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33cd200640 1 --2- 172.21.15.62:0/320963357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80aed00 0x7f33c815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7e00640 1 -- 172.21.15.62:0/320963357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80aed00 msgr2=0x7f33c815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7e00640 1 --2- 172.21.15.62:0/320963357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80aed00 0x7f33c815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7e00640 1 -- 172.21.15.62:0/320963357 >> v2:172.21.15.121:3300/0 conn(0x7f33c80cccf0 msgr2=0x7f33c80ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7e00640 1 --2- 172.21.15.62:0/320963357 >> v2:172.21.15.121:3300/0 conn(0x7f33c80cccf0 0x7f33c80ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7e00640 1 -- 172.21.15.62:0/320963357 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c806cd30 con 0x7f33c815bd10 2024-04-25T09:13:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.614+0000 7f33c7400640 1 --2- 172.21.15.62:0/320963357 >> v2:172.21.15.121:3300/0 conn(0x7f33c80cccf0 0x7f33c80ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:12.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.616+0000 7f33c7e00640 1 --2- 172.21.15.62:0/320963357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80cfca0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f33c80cf720 tx=0x7f33b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d8c2075bf35bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:12.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.616+0000 7f33c6a00640 1 -- 172.21.15.62:0/320963357 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8037070 con 0x7f33c815bd10 2024-04-25T09:13:12.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.616+0000 7f33c6a00640 1 -- 172.21.15.62:0/320963357 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b8032440 con 0x7f33c815bd10 2024-04-25T09:13:12.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.616+0000 7f33c6a00640 1 -- 172.21.15.62:0/320963357 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8032810 con 0x7f33c815bd10 2024-04-25T09:13:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 -- 172.21.15.62:0/320963357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 msgr2=0x7f33c80cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 --2- 172.21.15.62:0/320963357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80cfca0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f33c80cf720 tx=0x7f33b802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 -- 172.21.15.62:0/320963357 shutdown_connections 2024-04-25T09:13:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 --2- 172.21.15.62:0/320963357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80aed00 0x7f33c815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 --2- 172.21.15.62:0/320963357 >> v2:172.21.15.121:3300/0 conn(0x7f33c80cccf0 0x7f33c80ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 --2- 172.21.15.62:0/320963357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80cfca0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 -- 172.21.15.62:0/320963357 >> 172.21.15.62:0/320963357 conn(0x7f33c80668f0 msgr2=0x7f33c8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 -- 172.21.15.62:0/320963357 shutdown_connections 2024-04-25T09:13:12.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.617+0000 7f33cf200640 1 -- 172.21.15.62:0/320963357 wait complete. 2024-04-25T09:13:12.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.620+0000 7f33cf200640 1 Processor -- start 2024-04-25T09:13:12.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.620+0000 7f33cf200640 1 -- start start 2024-04-25T09:13:12.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.621+0000 7f33cf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33c80aed00 0x7f33c8152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:12.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.621+0000 7f33cf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80cccf0 0x7f33c81441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:12.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.621+0000 7f33cf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:12.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.621+0000 7f33cf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80d01e0 con 0x7f33c815bd10 2024-04-25T09:13:12.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.621+0000 7f33cf200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c806d2f0 con 0x7f33c80aed00 2024-04-25T09:13:12.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.621+0000 7f33cf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c806daf0 con 0x7f33c80cccf0 2024-04-25T09:13:12.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33cd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:12.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33cd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56422/0 (socket says 172.21.15.62:56422) 2024-04-25T09:13:12.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33cd200640 1 -- 172.21.15.62:0/482154518 learned_addr learned my addr 172.21.15.62:0/482154518 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:12.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33c7e00640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33c80aed00 0x7f33c8152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33c7400640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80cccf0 0x7f33c81441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33cd200640 1 -- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80cccf0 msgr2=0x7f33c81441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33cd200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80cccf0 0x7f33c81441c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33cd200640 1 -- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33c80aed00 msgr2=0x7f33c8152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33cd200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33c80aed00 0x7f33c8152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.622+0000 7f33cd200640 1 -- 172.21.15.62:0/482154518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c8153480 con 0x7f33c815bd10 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.623+0000 7f33cd200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80afba0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f33c0007e60 tx=0x7f33c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.623+0000 7f33bfe00640 1 -- 172.21.15.62:0/482154518 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c0014620 con 0x7f33c815bd10 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.623+0000 7f33bfe00640 1 -- 172.21.15.62:0/482154518 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c0019070 con 0x7f33c815bd10 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.623+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c8144700 con 0x7f33c815bd10 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.623+0000 7f33bfe00640 1 -- 172.21.15.62:0/482154518 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c0014a20 con 0x7f33c815bd10 2024-04-25T09:13:12.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.623+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c8152180 con 0x7f33c815bd10 2024-04-25T09:13:12.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.624+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c8072c70 con 0x7f33c815bd10 2024-04-25T09:13:12.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.625+0000 7f33bfe00640 1 -- 172.21.15.62:0/482154518 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33c0014bc0 con 0x7f33c815bd10 2024-04-25T09:13:12.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.626+0000 7f33bfe00640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f33a806b550 0x7f33a806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:12.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.626+0000 7f33bfe00640 1 -- 172.21.15.62:0/482154518 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33c008a9f0 con 0x7f33c815bd10 2024-04-25T09:13:12.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.627+0000 7f33c7e00640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f33a806b550 0x7f33a806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:12.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.628+0000 7f33c7e00640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f33a806b550 0x7f33a806da10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f33b8034000 tx=0x7f33b802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:12.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.631+0000 7f33bfe00640 1 -- 172.21.15.62:0/482154518 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33c00598e0 con 0x7f33c815bd10 2024-04-25T09:13:12.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.875+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33c8155000 con 0x7f33c815bd10 2024-04-25T09:13:12.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.876+0000 7f33bfe00640 1 -- 172.21.15.62:0/482154518 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f33c00591e0 con 0x7f33c815bd10 2024-04-25T09:13:12.863 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:12.863 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1798148826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1798148826},{"type":"v1","addr":"172.21.15.121:6835","nonce":1798148826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2045746846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2045746846},{"type":"v1","addr":"172.21.15.121:6837","nonce":2045746846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1182284820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1182284820},{"type":"v1","addr":"172.21.15.62:6834","nonce":1182284820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:22.810299+0000","modified":"2024-04-25T09:12:25.868942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3997398243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3997398243},{"type":"v1","addr":"172.21.15.62:6835","nonce":3997398243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:13:12.863 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 64 2024-04-25T09:13:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f33a806b550 msgr2=0x7f33a806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f33a806b550 0x7f33a806da10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f33b8034000 tx=0x7f33b802e000 comp rx=0 tx=0).stop 2024-04-25T09:13:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 msgr2=0x7f33c80afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80afba0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f33c0007e60 tx=0x7f33c000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 shutdown_connections 2024-04-25T09:13:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f33a806b550 0x7f33a806da10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33c815bd10 0x7f33c80afba0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f33c80cccf0 0x7f33c81441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 --2- 172.21.15.62:0/482154518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33c80aed00 0x7f33c8152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.879+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 >> 172.21.15.62:0/482154518 conn(0x7f33c80668f0 msgr2=0x7f33c815d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.880+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 shutdown_connections 2024-04-25T09:13:12.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:12.880+0000 7f33cf200640 1 -- 172.21.15.62:0/482154518 wait complete. 2024-04-25T09:13:12.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.105+0000 7fb2ed400640 1 Processor -- start 2024-04-25T09:13:13.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.105+0000 7fb2ed400640 1 -- start start 2024-04-25T09:13:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.107+0000 7fb2ed400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2e813d970 0x7fb2e80b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.107+0000 7fb2ed400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb2e806d900 0x7fb2e813e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.107+0000 7fb2ed400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e815b220 con 0x7fb2e80b1050 2024-04-25T09:13:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.107+0000 7fb2ed400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e815b500 con 0x7fb2e806d900 2024-04-25T09:13:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.107+0000 7fb2ed400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e815b7e0 con 0x7fb2e813d970 2024-04-25T09:13:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.107+0000 7fb2df400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb2e80b1050 0x7fb2e806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37006/0 (socket says 172.21.15.62:37006) 2024-04-25T09:13:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.107+0000 7fb2df400640 1 -- 172.21.15.62:0/4236379098 learned_addr learned my addr 172.21.15.62:0/4236379098 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:13.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2eca00640 1 --2- 172.21.15.62:0/4236379098 >> v2:172.21.15.121:3300/0 conn(0x7fb2e806d900 0x7fb2e813e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7e00640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2e813d970 0x7fb2e80b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78104138 0 0) 0x7fb2e815b220 con 0x7fb2e80b1050 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2cc005730 con 0x7fb2e80b1050 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 730555016 0 0) 0x7fb2cc005730 con 0x7fb2e80b1050 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2e813d970 msgr2=0x7fb2e80b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2e813d970 0x7fb2e80b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 >> v2:172.21.15.121:3300/0 conn(0x7fb2e806d900 msgr2=0x7fb2e813e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 --2- 172.21.15.62:0/4236379098 >> v2:172.21.15.121:3300/0 conn(0x7fb2e806d900 0x7fb2e813e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e815bac0 con 0x7fb2e80b1050 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb2d0001670 con 0x7fb2e80b1050 2024-04-25T09:13:13.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 >> v1:172.21.15.62:6789/0 conn(0x7fb2e80b1050 legacy=0x7fb2e806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.108+0000 7fb2e7400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2e813d970 0x7fb2cc005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2e7400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2cc006140 0x7fb2cc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2e7400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2cc006f50 0x7fb2cc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e815b500 con 0x7fb2cc006140 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e815b7e0 con 0x7fb2cc006f50 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc00dbd0 con 0x7fb2e813d970 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2df400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2cc006140 0x7fb2cc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2e7e00640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2e813d970 0x7fb2cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2eca00640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2cc006f50 0x7fb2cc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2df400640 1 -- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2e813d970 msgr2=0x7fb2cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2df400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2e813d970 0x7fb2cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2df400640 1 -- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2cc006f50 msgr2=0x7fb2cc0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2df400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2cc006f50 0x7fb2cc0073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.109+0000 7fb2df400640 1 -- 172.21.15.62:0/4236379098 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc005730 con 0x7fb2cc006140 2024-04-25T09:13:13.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.110+0000 7fb2df400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2cc006140 0x7fb2cc006a10 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c240 tx=0x7fb2cc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299aa158a7439b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:13.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.111+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0007d30 con 0x7fb2cc006140 2024-04-25T09:13:13.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.111+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0005ad0 con 0x7fb2cc006140 2024-04-25T09:13:13.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.111+0000 7fb2e7400640 1 -- 172.21.15.62:0/4236379098 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0007070 con 0x7fb2cc006140 2024-04-25T09:13:13.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.111+0000 7fb2ed400640 1 -- 172.21.15.62:0/4236379098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2cc006140 msgr2=0x7fb2cc006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.111+0000 7fb2ed400640 1 --2- 172.21.15.62:0/4236379098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2cc006140 0x7fb2cc006a10 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c240 tx=0x7fb2cc007a40 comp rx=0 tx=0).stop 2024-04-25T09:13:13.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.111+0000 7fb2eca00640 1 -- 172.21.15.62:0/4236379098 reap_dead start 2024-04-25T09:13:13.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.111+0000 7fb2ed400640 1 -- 172.21.15.62:0/4236379098 shutdown_connections 2024-04-25T09:13:13.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.111+0000 7fb2ed400640 1 -- 172.21.15.62:0/4236379098 >> 172.21.15.62:0/4236379098 conn(0x7fb2e80b14d0 msgr2=0x7fb2e80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:13.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.112+0000 7fb2ed400640 1 -- 172.21.15.62:0/4236379098 shutdown_connections 2024-04-25T09:13:13.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.112+0000 7fb2ed400640 1 -- 172.21.15.62:0/4236379098 wait complete. 2024-04-25T09:13:13.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.113+0000 7fb2ed400640 1 Processor -- start 2024-04-25T09:13:13.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.113+0000 7fb2ed400640 1 -- start start 2024-04-25T09:13:13.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.114+0000 7fb2ed400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2cc006140 0x7fb2e80b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.114+0000 7fb2ed400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e8142220 0x7fb2e81f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.114+0000 7fb2ed400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e80a94d0 0x7fb2e80a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.114+0000 7fb2ed400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80b0b10 con 0x7fb2e80a94d0 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.114+0000 7fb2ed400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e815af00 con 0x7fb2e8142220 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.114+0000 7fb2ed400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80b0c90 con 0x7fb2cc006140 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2eca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e80a94d0 0x7fb2e80a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2eca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e80a94d0 0x7fb2e80a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56448/0 (socket says 172.21.15.62:56448) 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2eca00640 1 -- 172.21.15.62:0/4172016632 learned_addr learned my addr 172.21.15.62:0/4172016632 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2e7e00640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2cc006140 0x7fb2e80b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2df400640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e8142220 0x7fb2e81f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2eca00640 1 -- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2cc006140 msgr2=0x7fb2e80b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2eca00640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2cc006140 0x7fb2e80b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2eca00640 1 -- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e8142220 msgr2=0x7fb2e81f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2eca00640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e8142220 0x7fb2e81f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2eca00640 1 -- 172.21.15.62:0/4172016632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e81429d0 con 0x7fb2e80a94d0 2024-04-25T09:13:13.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.115+0000 7fb2df400640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e8142220 0x7fb2e81f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:13.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.116+0000 7fb2eca00640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e80a94d0 0x7fb2e80a9950 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb2e806e100 tx=0x7fb2e000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:13.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.116+0000 7fb2e4c00640 1 -- 172.21.15.62:0/4172016632 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e001c070 con 0x7fb2e80a94d0 2024-04-25T09:13:13.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.116+0000 7fb2e4c00640 1 -- 172.21.15.62:0/4172016632 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e0017400 con 0x7fb2e80a94d0 2024-04-25T09:13:13.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.116+0000 7fb2e4c00640 1 -- 172.21.15.62:0/4172016632 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00177f0 con 0x7fb2e80a94d0 2024-04-25T09:13:13.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.116+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e8142cb0 con 0x7fb2e80a94d0 2024-04-25T09:13:13.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.116+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e8143270 con 0x7fb2e80a94d0 2024-04-25T09:13:13.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.117+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e804ef20 con 0x7fb2e80a94d0 2024-04-25T09:13:13.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.118+0000 7fb2e4c00640 1 -- 172.21.15.62:0/4172016632 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2e001f070 con 0x7fb2e80a94d0 2024-04-25T09:13:13.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.119+0000 7fb2e4c00640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2c406b4d0 0x7fb2c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.119+0000 7fb2e4c00640 1 -- 172.21.15.62:0/4172016632 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2e008a240 con 0x7fb2e80a94d0 2024-04-25T09:13:13.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.120+0000 7fb2e7e00640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2c406b4d0 0x7fb2c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.121+0000 7fb2e7e00640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2c406b4d0 0x7fb2c406d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009930 tx=0x7fb2d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:13.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.124+0000 7fb2e4c00640 1 -- 172.21.15.62:0/4172016632 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2e0012070 con 0x7fb2e80a94d0 2024-04-25T09:13:13.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.324+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2e80ceb50 con 0x7fb2e80a94d0 2024-04-25T09:13:13.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.327+0000 7fb2e4c00640 1 -- 172.21.15.62:0/4172016632 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16660 (secure 0 0 0) 0x7fb2e0058f90 con 0x7fb2e80a94d0 2024-04-25T09:13:13.314 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:13.314 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":97,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:24.344250+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:20.654480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:21.833814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2c406b4d0 msgr2=0x7fb2c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2c406b4d0 0x7fb2c406d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009930 tx=0x7fb2d8008040 comp rx=0 tx=0).stop 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e80a94d0 msgr2=0x7fb2e80a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e80a94d0 0x7fb2e80a9950 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb2e806e100 tx=0x7fb2e000f4a0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 shutdown_connections 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb2c406b4d0 0x7fb2c406d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2e80a94d0 0x7fb2e80a9950 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2e8142220 0x7fb2e81f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 --2- 172.21.15.62:0/4172016632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2cc006140 0x7fb2e80b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 >> 172.21.15.62:0/4172016632 conn(0x7fb2e80b14d0 msgr2=0x7fb2e80b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 shutdown_connections 2024-04-25T09:13:13.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.330+0000 7fb2ed400640 1 -- 172.21.15.62:0/4172016632 wait complete. 2024-04-25T09:13:13.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:12:20.654480+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:12:21.833814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:13:13.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:13:13.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.567+0000 7faf6b200640 1 Processor -- start 2024-04-25T09:13:13.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.567+0000 7faf6b200640 1 -- start start 2024-04-25T09:13:13.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.569+0000 7faf6b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c0ccde0 0x7faf6c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.569+0000 7faf6b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7faf6c0b31a0 0x7faf6c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.569+0000 7faf6b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6c0ced40 con 0x7faf6c06d2c0 2024-04-25T09:13:13.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.569+0000 7faf6b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6c0cd260 con 0x7faf6c0b31a0 2024-04-25T09:13:13.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.569+0000 7faf6b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0cf000 con 0x7faf6c0ccde0 2024-04-25T09:13:13.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.569+0000 7faf69e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7faf6c06d2c0 0x7faf6c06d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37016/0 (socket says 172.21.15.62:37016) 2024-04-25T09:13:13.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.569+0000 7faf69e00640 1 -- 172.21.15.62:0/3167554826 learned_addr learned my addr 172.21.15.62:0/3167554826 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:13.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf6a800640 1 --2- 172.21.15.62:0/3167554826 >> v2:172.21.15.121:3300/0 conn(0x7faf6c0b31a0 0x7faf6c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf61400640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c0ccde0 0x7faf6c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140925038 0 0) 0x7faf6c0ced40 con 0x7faf6c06d2c0 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf50003520 con 0x7faf6c06d2c0 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1054456335 0 0) 0x7faf50003520 con 0x7faf6c06d2c0 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c0ccde0 msgr2=0x7faf6c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c0ccde0 0x7faf6c0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 >> v2:172.21.15.121:3300/0 conn(0x7faf6c0b31a0 msgr2=0x7faf6c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 --2- 172.21.15.62:0/3167554826 >> v2:172.21.15.121:3300/0 conn(0x7faf6c0b31a0 0x7faf6c0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c0cf2e0 con 0x7faf6c06d2c0 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7faf5c0019f0 con 0x7faf6c06d2c0 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 >> v1:172.21.15.62:6789/0 conn(0x7faf6c06d2c0 legacy=0x7faf6c06d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf500039b0 0x7faf50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c0ccde0 0x7faf50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0cd260 con 0x7faf50004d40 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0cf000 con 0x7faf500039b0 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.570+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5000b9c0 con 0x7faf6c0ccde0 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.571+0000 7faf6a800640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.571+0000 7faf69e00640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf500039b0 0x7faf50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.571+0000 7faf61400640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c0ccde0 0x7faf50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.571+0000 7faf6a800640 1 -- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c0ccde0 msgr2=0x7faf50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.571+0000 7faf6a800640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c0ccde0 0x7faf50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.571+0000 7faf6a800640 1 -- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf500039b0 msgr2=0x7faf50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.571+0000 7faf6a800640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf500039b0 0x7faf50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.572+0000 7faf6a800640 1 -- 172.21.15.62:0/3167554826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf50003520 con 0x7faf50004d40 2024-04-25T09:13:13.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.573+0000 7faf6a800640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf500051c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7faf64009ac0 tx=0x7faf50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710ee0f14b6a5686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:13.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.573+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64036070 con 0x7faf50004d40 2024-04-25T09:13:13.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.573+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf64031400 con 0x7faf50004d40 2024-04-25T09:13:13.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.573+0000 7faf69400640 1 -- 172.21.15.62:0/3167554826 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64009690 con 0x7faf50004d40 2024-04-25T09:13:13.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.574+0000 7faf6b200640 1 -- 172.21.15.62:0/3167554826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 msgr2=0x7faf500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.574+0000 7faf6b200640 1 --2- 172.21.15.62:0/3167554826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf500051c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7faf64009ac0 tx=0x7faf50005830 comp rx=0 tx=0).stop 2024-04-25T09:13:13.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.574+0000 7faf6a800640 1 -- 172.21.15.62:0/3167554826 reap_dead start 2024-04-25T09:13:13.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.574+0000 7faf6b200640 1 -- 172.21.15.62:0/3167554826 shutdown_connections 2024-04-25T09:13:13.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.574+0000 7faf6b200640 1 -- 172.21.15.62:0/3167554826 >> 172.21.15.62:0/3167554826 conn(0x7faf6c066f20 msgr2=0x7faf6c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:13.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.574+0000 7faf6b200640 1 -- 172.21.15.62:0/3167554826 shutdown_connections 2024-04-25T09:13:13.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.574+0000 7faf6b200640 1 -- 172.21.15.62:0/3167554826 wait complete. 2024-04-25T09:13:13.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.576+0000 7faf6b200640 1 Processor -- start 2024-04-25T09:13:13.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.576+0000 7faf6b200640 1 -- start start 2024-04-25T09:13:13.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf6b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf6c15e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf6b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c1f3330 0x7faf6c1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf6b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6c1e7c10 0x7faf6c1e80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf6b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0b2c80 con 0x7faf50004d40 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf6b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0b29a0 con 0x7faf6c1e7c10 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf6b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c1448d0 con 0x7faf6c1f3330 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf69e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf6c15e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf69e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf6c15e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56460/0 (socket says 172.21.15.62:56460) 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.577+0000 7faf69e00640 1 -- 172.21.15.62:0/1247527280 learned_addr learned my addr 172.21.15.62:0/1247527280 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf69e00640 1 -- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c1f3330 msgr2=0x7faf6c1f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf6a800640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6c1e7c10 0x7faf6c1e80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf61400640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c1f3330 0x7faf6c1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf69e00640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c1f3330 0x7faf6c1f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf69e00640 1 -- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6c1e7c10 msgr2=0x7faf6c1e80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf69e00640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6c1e7c10 0x7faf6c1e80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf69e00640 1 -- 172.21.15.62:0/1247527280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c15e770 con 0x7faf50004d40 2024-04-25T09:13:13.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf69e00640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf6c15e230 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7faf5c007b00 tx=0x7faf5c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:13.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf62a00640 1 -- 172.21.15.62:0/1247527280 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00ee30 con 0x7faf50004d40 2024-04-25T09:13:13.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf62a00640 1 -- 172.21.15.62:0/1247527280 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c018070 con 0x7faf50004d40 2024-04-25T09:13:13.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.578+0000 7faf62a00640 1 -- 172.21.15.62:0/1247527280 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c013660 con 0x7faf50004d40 2024-04-25T09:13:13.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.579+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c1e8c60 con 0x7faf50004d40 2024-04-25T09:13:13.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.579+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c0c47d0 con 0x7faf50004d40 2024-04-25T09:13:13.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.580+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c06abe0 con 0x7faf50004d40 2024-04-25T09:13:13.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.581+0000 7faf62a00640 1 -- 172.21.15.62:0/1247527280 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf5c013800 con 0x7faf50004d40 2024-04-25T09:13:13.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.582+0000 7faf62a00640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf4806b500 0x7faf4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:13.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.582+0000 7faf62a00640 1 -- 172.21.15.62:0/1247527280 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf5c08a900 con 0x7faf50004d40 2024-04-25T09:13:13.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.583+0000 7faf61400640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf4806b500 0x7faf4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:13.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.584+0000 7faf61400640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf4806b500 0x7faf4806d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0af880 tx=0x7faf54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:13.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.586+0000 7faf62a00640 1 -- 172.21.15.62:0/1247527280 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf6c15e770 con 0x7faf50004d40 2024-04-25T09:13:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.784+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf6c04ef20 con 0x7faf50004d40 2024-04-25T09:13:13.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.785+0000 7faf62a00640 1 -- 172.21.15.62:0/1247527280 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf5c08a030 con 0x7faf50004d40 2024-04-25T09:13:13.772 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:13:13.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf4806b500 msgr2=0x7faf4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf4806b500 0x7faf4806d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0af880 tx=0x7faf54008040 comp rx=0 tx=0).stop 2024-04-25T09:13:13.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 msgr2=0x7faf6c15e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:13.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf6c15e230 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7faf5c007b00 tx=0x7faf5c00e530 comp rx=0 tx=0).stop 2024-04-25T09:13:13.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 shutdown_connections 2024-04-25T09:13:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7faf4806b500 0x7faf4806d9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf6c1e7c10 0x7faf6c1e80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7faf6c1f3330 0x7faf6c1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 --2- 172.21.15.62:0/1247527280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faf50004d40 0x7faf6c15e230 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.788+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 >> 172.21.15.62:0/1247527280 conn(0x7faf6c066f20 msgr2=0x7faf6c06d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.789+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 shutdown_connections 2024-04-25T09:13:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:13.789+0000 7faf6b200640 1 -- 172.21.15.62:0/1247527280 wait complete. 2024-04-25T09:13:13.795 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:13:14.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.025+0000 7f462e400640 1 Processor -- start 2024-04-25T09:13:14.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.026+0000 7f462e400640 1 -- start start 2024-04-25T09:13:14.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.027+0000 7f462e400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462806d2c0 0x7f46280ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f462e400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f462806d6a0 0x7f46280ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f462e400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f46280b31a0 0x7f46280b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f462e400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46280ced40 con 0x7f462806d6a0 2024-04-25T09:13:14.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f462e400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46280cf000 con 0x7f46280b31a0 2024-04-25T09:13:14.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f462e400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46280cf2e0 con 0x7f462806d2c0 2024-04-25T09:13:14.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f4627e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f462806d6a0 0x7f46280ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f4627e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f462806d6a0 0x7f46280ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56464/0 (socket says 172.21.15.62:56464) 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f4627e00640 1 -- 172.21.15.62:0/3220623725 learned_addr learned my addr 172.21.15.62:0/3220623725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f462d000640 1 --2- 172.21.15.62:0/3220623725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462806d2c0 0x7f46280ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.028+0000 7f462da00640 1 --2- 172.21.15.62:0/3220623725 >> v2:172.21.15.121:3300/0 conn(0x7f46280b31a0 0x7f46280b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.029+0000 7f4627e00640 1 -- 172.21.15.62:0/3220623725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462806d2c0 msgr2=0x7f46280ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.029+0000 7f4627e00640 1 --2- 172.21.15.62:0/3220623725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462806d2c0 0x7f46280ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.029+0000 7f4627e00640 1 -- 172.21.15.62:0/3220623725 >> v2:172.21.15.121:3300/0 conn(0x7f46280b31a0 msgr2=0x7f46280b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.029+0000 7f4627e00640 1 --2- 172.21.15.62:0/3220623725 >> v2:172.21.15.121:3300/0 conn(0x7f46280b31a0 0x7f46280b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.029+0000 7f4627e00640 1 -- 172.21.15.62:0/3220623725 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46280b26c0 con 0x7f462806d6a0 2024-04-25T09:13:14.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.029+0000 7f462da00640 1 --2- 172.21.15.62:0/3220623725 >> v2:172.21.15.121:3300/0 conn(0x7f46280b31a0 0x7f46280b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:14.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.030+0000 7f4627e00640 1 --2- 172.21.15.62:0/3220623725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d6a0 0x7f46280ce800 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f46280cc6f0 tx=0x7f461402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2930900dc1c139f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.030+0000 7f4627400640 1 -- 172.21.15.62:0/3220623725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614037070 con 0x7f462806d6a0 2024-04-25T09:13:14.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f4627400640 1 -- 172.21.15.62:0/3220623725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4614032440 con 0x7f462806d6a0 2024-04-25T09:13:14.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f4627400640 1 -- 172.21.15.62:0/3220623725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614032810 con 0x7f462806d6a0 2024-04-25T09:13:14.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f462e400640 1 -- 172.21.15.62:0/3220623725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d6a0 msgr2=0x7f46280ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f462e400640 1 --2- 172.21.15.62:0/3220623725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d6a0 0x7f46280ce800 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f46280cc6f0 tx=0x7f461402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f462e400640 1 -- 172.21.15.62:0/3220623725 shutdown_connections 2024-04-25T09:13:14.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f462e400640 1 --2- 172.21.15.62:0/3220623725 >> v2:172.21.15.121:3300/0 conn(0x7f46280b31a0 0x7f46280b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f462e400640 1 --2- 172.21.15.62:0/3220623725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d6a0 0x7f46280ce800 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f462e400640 1 --2- 172.21.15.62:0/3220623725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f462806d2c0 0x7f46280ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.031+0000 7f462e400640 1 -- 172.21.15.62:0/3220623725 >> 172.21.15.62:0/3220623725 conn(0x7f4628066f20 msgr2=0x7f46280b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:14.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.032+0000 7f462e400640 1 -- 172.21.15.62:0/3220623725 shutdown_connections 2024-04-25T09:13:14.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.032+0000 7f462e400640 1 -- 172.21.15.62:0/3220623725 wait complete. 2024-04-25T09:13:14.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.033+0000 7f462e400640 1 Processor -- start 2024-04-25T09:13:14.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.034+0000 7f462e400640 1 -- start start 2024-04-25T09:13:14.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.034+0000 7f462e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d2c0 0x7f46281f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462e400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462806d6a0 0x7f462815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462e400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46280b31a0 0x7f462813ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462806db20 con 0x7f462806d2c0 2024-04-25T09:13:14.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462e400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46280b2c80 con 0x7f462806d6a0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462e400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46281448d0 con 0x7f46280b31a0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d2c0 0x7f46281f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d2c0 0x7f46281f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56478/0 (socket says 172.21.15.62:56478) 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462d000640 1 -- 172.21.15.62:0/3976925364 learned_addr learned my addr 172.21.15.62:0/3976925364 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f4627e00640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462806d6a0 0x7f462815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462d000640 1 -- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46280b31a0 msgr2=0x7f462813ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462d000640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46280b31a0 0x7f462813ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462d000640 1 -- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462806d6a0 msgr2=0x7f462815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462d000640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462806d6a0 0x7f462815e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.035+0000 7f462d000640 1 -- 172.21.15.62:0/3976925364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46281f48e0 con 0x7f462806d2c0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.036+0000 7f462d000640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d2c0 0x7f46281f3720 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f461c00e8a0 tx=0x7f461c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.036+0000 7f4624c00640 1 -- 172.21.15.62:0/3976925364 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c017070 con 0x7f462806d2c0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.036+0000 7f4624c00640 1 -- 172.21.15.62:0/3976925364 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c00ee90 con 0x7f462806d2c0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.036+0000 7f4624c00640 1 -- 172.21.15.62:0/3976925364 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c00cb80 con 0x7f462806d2c0 2024-04-25T09:13:14.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.036+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46281f46a0 con 0x7f462806d2c0 2024-04-25T09:13:14.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.036+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46281f40e0 con 0x7f462806d2c0 2024-04-25T09:13:14.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.037+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462804ef20 con 0x7f462806d2c0 2024-04-25T09:13:14.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.038+0000 7f4624c00640 1 -- 172.21.15.62:0/3976925364 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f461c01a070 con 0x7f462806d2c0 2024-04-25T09:13:14.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.039+0000 7f4624c00640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f460806b4d0 0x7f460806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.039+0000 7f4624c00640 1 -- 172.21.15.62:0/3976925364 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f461c08a120 con 0x7f462806d2c0 2024-04-25T09:13:14.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.040+0000 7f4627e00640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f460806b4d0 0x7f460806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.041+0000 7f4627e00640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f460806b4d0 0x7f460806d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4614034000 tx=0x7f461402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.043+0000 7f4624c00640 1 -- 172.21.15.62:0/3976925364 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46281f48e0 con 0x7f462806d2c0 2024-04-25T09:13:14.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.244+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46280b5700 con 0x7f462806d2c0 2024-04-25T09:13:14.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.245+0000 7f4624c00640 1 -- 172.21.15.62:0/3976925364 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f461c058f00 con 0x7f462806d2c0 2024-04-25T09:13:14.231 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:13:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f460806b4d0 msgr2=0x7f460806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f460806b4d0 0x7f460806d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4614034000 tx=0x7f461402e000 comp rx=0 tx=0).stop 2024-04-25T09:13:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d2c0 msgr2=0x7f46281f3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d2c0 0x7f46281f3720 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f461c00e8a0 tx=0x7f461c00ed60 comp rx=0 tx=0).stop 2024-04-25T09:13:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 shutdown_connections 2024-04-25T09:13:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f460806b4d0 0x7f460806d990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f46280b31a0 0x7f462813ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462806d6a0 0x7f462815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 --2- 172.21.15.62:0/3976925364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462806d2c0 0x7f46281f3720 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.247+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 >> 172.21.15.62:0/3976925364 conn(0x7f4628066f20 msgr2=0x7f46280cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.248+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 shutdown_connections 2024-04-25T09:13:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.248+0000 7f462e400640 1 -- 172.21.15.62:0/3976925364 wait complete. 2024-04-25T09:13:14.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:13:14.438 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:14.444 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.467+0000 7f9804c00640 1 Processor -- start 2024-04-25T09:13:14.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.467+0000 7f9804c00640 1 -- start start 2024-04-25T09:13:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.469+0000 7f9804c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9800147410 0x7f9800162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.469+0000 7f9804c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9800162540 con 0x7f980006d2f0 2024-04-25T09:13:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.469+0000 7f9804c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980015ea30 con 0x7f9800071580 2024-04-25T09:13:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.469+0000 7f9804c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98001477f0 con 0x7f9800147410 2024-04-25T09:13:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.469+0000 7f97ff400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9800147410 0x7f9800162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.469+0000 7f97ff400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9800147410 0x7f9800162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56480/0 (socket says 172.21.15.62:56480) 2024-04-25T09:13:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.469+0000 7f97ff400640 1 -- 172.21.15.62:0/1069620958 learned_addr learned my addr 172.21.15.62:0/1069620958 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.470+0000 7f97ff400640 1 -- 172.21.15.62:0/1069620958 >> v1:172.21.15.121:6789/0 conn(0x7f9800071580 legacy=0x7f98000b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:14.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.470+0000 7f97ff400640 1 -- 172.21.15.62:0/1069620958 >> v1:172.21.15.62:6789/0 conn(0x7f980006d2f0 legacy=0x7f980015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.470+0000 7f97ff400640 1 -- 172.21.15.62:0/1069620958 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980015ed10 con 0x7f9800147410 2024-04-25T09:13:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.471+0000 7f97ff400640 1 --2- 172.21.15.62:0/1069620958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f9800162000 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009e50 tx=0x7f97f002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6952a3abbba5c65a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.471+0000 7f97fea00640 1 -- 172.21.15.62:0/1069620958 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0036070 con 0x7f9800147410 2024-04-25T09:13:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.471+0000 7f97fea00640 1 -- 172.21.15.62:0/1069620958 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0031600 con 0x7f9800147410 2024-04-25T09:13:14.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.471+0000 7f97fea00640 1 -- 172.21.15.62:0/1069620958 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00319d0 con 0x7f9800147410 2024-04-25T09:13:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.471+0000 7f9804c00640 1 -- 172.21.15.62:0/1069620958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 msgr2=0x7f9800162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.471+0000 7f9804c00640 1 --2- 172.21.15.62:0/1069620958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f9800162000 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009e50 tx=0x7f97f002a760 comp rx=0 tx=0).stop 2024-04-25T09:13:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.472+0000 7f9804c00640 1 -- 172.21.15.62:0/1069620958 shutdown_connections 2024-04-25T09:13:14.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.472+0000 7f9804c00640 1 --2- 172.21.15.62:0/1069620958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f9800162000 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.472+0000 7f9804c00640 1 -- 172.21.15.62:0/1069620958 >> 172.21.15.62:0/1069620958 conn(0x7f980006b000 msgr2=0x7f980006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.472+0000 7f9804c00640 1 -- 172.21.15.62:0/1069620958 shutdown_connections 2024-04-25T09:13:14.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.472+0000 7f9804c00640 1 -- 172.21.15.62:0/1069620958 wait complete. 2024-04-25T09:13:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.473+0000 7f9804c00640 1 Processor -- start 2024-04-25T09:13:14.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.474+0000 7f9804c00640 1 -- start start 2024-04-25T09:13:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.474+0000 7f9804c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f980006d2f0 0x7f98001e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f9804c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9800071580 0x7f98000c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f9804c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f98000c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f9804c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980015e720 con 0x7f9800147410 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f9804c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800161b70 con 0x7f9800071580 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f9804c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98000719d0 con 0x7f980006d2f0 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ffe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f98000c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ffe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f98000c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56486/0 (socket says 172.21.15.62:56486) 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ffe00640 1 -- 172.21.15.62:0/2281740397 learned_addr learned my addr 172.21.15.62:0/2281740397 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ff400640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f980006d2f0 0x7f98001e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97f6a00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9800071580 0x7f98000c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ffe00640 1 -- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f980006d2f0 msgr2=0x7f98001e8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ffe00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f980006d2f0 0x7f98001e8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ffe00640 1 -- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9800071580 msgr2=0x7f98000c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ffe00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9800071580 0x7f98000c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.475+0000 7f97ffe00640 1 -- 172.21.15.62:0/2281740397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98000c5d90 con 0x7f9800147410 2024-04-25T09:13:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.476+0000 7f97ffe00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f98000c5850 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f97f800c440 tx=0x7f97f800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.476+0000 7f97f7e00640 1 -- 172.21.15.62:0/2281740397 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8011790 con 0x7f9800147410 2024-04-25T09:13:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.476+0000 7f97f7e00640 1 -- 172.21.15.62:0/2281740397 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f8017070 con 0x7f9800147410 2024-04-25T09:13:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.476+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98000c6060 con 0x7f9800147410 2024-04-25T09:13:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.476+0000 7f97f7e00640 1 -- 172.21.15.62:0/2281740397 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8011b70 con 0x7f9800147410 2024-04-25T09:13:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.476+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98000c6620 con 0x7f9800147410 2024-04-25T09:13:14.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.478+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980004ef20 con 0x7f9800147410 2024-04-25T09:13:14.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.478+0000 7f97f7e00640 1 -- 172.21.15.62:0/2281740397 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97f8011d10 con 0x7f9800147410 2024-04-25T09:13:14.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.480+0000 7f97f7e00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f97dc06b4d0 0x7f97dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.480+0000 7f97f7e00640 1 -- 172.21.15.62:0/2281740397 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97f8089f50 con 0x7f9800147410 2024-04-25T09:13:14.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.480+0000 7f97ff400640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f97dc06b4d0 0x7f97dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.481+0000 7f97ff400640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f97dc06b4d0 0x7f97dc06d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f97f0033000 tx=0x7f97f0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.484+0000 7f97f7e00640 1 -- 172.21.15.62:0/2281740397 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97f8058dd0 con 0x7f9800147410 2024-04-25T09:13:14.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.683+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98000b5870 con 0x7f9800147410 2024-04-25T09:13:14.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.684+0000 7f97f7e00640 1 -- 172.21.15.62:0/2281740397 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97f80586d0 con 0x7f9800147410 2024-04-25T09:13:14.671 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:13:14.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.686+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f97dc06b4d0 msgr2=0x7f97dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.686+0000 7f9804c00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f97dc06b4d0 0x7f97dc06d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f97f0033000 tx=0x7f97f0009000 comp rx=0 tx=0).stop 2024-04-25T09:13:14.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.686+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 msgr2=0x7f98000c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.686+0000 7f9804c00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f98000c5850 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f97f800c440 tx=0x7f97f800c900 comp rx=0 tx=0).stop 2024-04-25T09:13:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.687+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 shutdown_connections 2024-04-25T09:13:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.687+0000 7f9804c00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f97dc06b4d0 0x7f97dc06d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.687+0000 7f9804c00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9800147410 0x7f98000c5850 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.687+0000 7f9804c00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9800071580 0x7f98000c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.687+0000 7f9804c00640 1 --2- 172.21.15.62:0/2281740397 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f980006d2f0 0x7f98001e8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.687+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 >> 172.21.15.62:0/2281740397 conn(0x7f980006b000 msgr2=0x7f98000b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.687+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 shutdown_connections 2024-04-25T09:13:14.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.687+0000 7f9804c00640 1 -- 172.21.15.62:0/2281740397 wait complete. 2024-04-25T09:13:14.691 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:14.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.916+0000 7efc9cc00640 1 Processor -- start 2024-04-25T09:13:14.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.916+0000 7efc9cc00640 1 -- start start 2024-04-25T09:13:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.917+0000 7efc9cc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efc9813d970 0x7efc980b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.917+0000 7efc9cc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efc980b1050 0x7efc9806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.917+0000 7efc9cc00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9815b220 con 0x7efc9815acf0 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.917+0000 7efc9cc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9815b500 con 0x7efc9813d970 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.917+0000 7efc9cc00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9815b7e0 con 0x7efc980b1050 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.917+0000 7efc97400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efc9813d970 0x7efc980b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.918+0000 7efc97400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efc9813d970 0x7efc980b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39340/0 (socket says 172.21.15.62:39340) 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.918+0000 7efc97400640 1 -- 172.21.15.62:0/3168567746 learned_addr learned my addr 172.21.15.62:0/3168567746 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.918+0000 7efc97400640 1 -- 172.21.15.62:0/3168567746 >> v2:172.21.15.121:3300/0 conn(0x7efc980b1050 msgr2=0x7efc9806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.918+0000 7efc97400640 1 --2- 172.21.15.62:0/3168567746 >> v2:172.21.15.121:3300/0 conn(0x7efc980b1050 0x7efc9806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.918+0000 7efc97400640 1 -- 172.21.15.62:0/3168567746 >> v1:172.21.15.62:6789/0 conn(0x7efc9815acf0 legacy=0x7efc9813ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.918+0000 7efc97400640 1 -- 172.21.15.62:0/3168567746 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9815bac0 con 0x7efc9813d970 2024-04-25T09:13:14.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.919+0000 7efc97400640 1 --2- 172.21.15.62:0/3168567746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc980b0b10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7efc980cd200 tx=0x7efc8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d723548b447d3ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.919+0000 7efc96a00640 1 -- 172.21.15.62:0/3168567746 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88037070 con 0x7efc9813d970 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.919+0000 7efc96a00640 1 -- 172.21.15.62:0/3168567746 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc88032440 con 0x7efc9813d970 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.919+0000 7efc96a00640 1 -- 172.21.15.62:0/3168567746 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88032810 con 0x7efc9813d970 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.920+0000 7efc9cc00640 1 -- 172.21.15.62:0/3168567746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 msgr2=0x7efc980b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.920+0000 7efc9cc00640 1 --2- 172.21.15.62:0/3168567746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc980b0b10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7efc980cd200 tx=0x7efc8802a270 comp rx=0 tx=0).stop 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.920+0000 7efc9cc00640 1 -- 172.21.15.62:0/3168567746 shutdown_connections 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.920+0000 7efc9cc00640 1 --2- 172.21.15.62:0/3168567746 >> v2:172.21.15.121:3300/0 conn(0x7efc980b1050 0x7efc9806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.920+0000 7efc9cc00640 1 --2- 172.21.15.62:0/3168567746 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc980b0b10 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.920+0000 7efc9cc00640 1 -- 172.21.15.62:0/3168567746 >> 172.21.15.62:0/3168567746 conn(0x7efc980b14d0 msgr2=0x7efc980b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:14.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.920+0000 7efc9cc00640 1 -- 172.21.15.62:0/3168567746 shutdown_connections 2024-04-25T09:13:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.920+0000 7efc9cc00640 1 -- 172.21.15.62:0/3168567746 wait complete. 2024-04-25T09:13:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.922+0000 7efc9cc00640 1 Processor -- start 2024-04-25T09:13:14.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.922+0000 7efc9cc00640 1 -- start start 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc9cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc980b1050 0x7efc9813a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc9cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc98159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc9cc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc9815acf0 0x7efc98159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc9cc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9806d730 con 0x7efc9813d970 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc9cc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9813f4b0 con 0x7efc980b1050 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc9cc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9806d8b0 con 0x7efc9815acf0 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc8ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc98159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc8ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc98159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39344/0 (socket says 172.21.15.62:39344) 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.923+0000 7efc8ea00640 1 -- 172.21.15.62:0/743682702 learned_addr learned my addr 172.21.15.62:0/743682702 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:14.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc97400640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc980b1050 0x7efc9813a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8ea00640 1 -- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc9815acf0 msgr2=0x7efc98159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8ea00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc9815acf0 0x7efc98159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8ea00640 1 -- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc980b1050 msgr2=0x7efc9813a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8ea00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc980b1050 0x7efc9813a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8ea00640 1 -- 172.21.15.62:0/743682702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc981f3860 con 0x7efc9813d970 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8ea00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc98159810 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7efc8000b4a0 tx=0x7efc8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8fe00640 1 -- 172.21.15.62:0/743682702 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80014070 con 0x7efc9813d970 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8fe00640 1 -- 172.21.15.62:0/743682702 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80010610 con 0x7efc9813d970 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc8fe00640 1 -- 172.21.15.62:0/743682702 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80010a00 con 0x7efc9813d970 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.924+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc981f3b40 con 0x7efc9813d970 2024-04-25T09:13:14.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.925+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc981f4100 con 0x7efc9813d970 2024-04-25T09:13:14.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.926+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9804ef20 con 0x7efc9813d970 2024-04-25T09:13:14.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.927+0000 7efc8fe00640 1 -- 172.21.15.62:0/743682702 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc80017070 con 0x7efc9813d970 2024-04-25T09:13:14.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.928+0000 7efc8fe00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc7406b4d0 0x7efc7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:14.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.928+0000 7efc8fe00640 1 -- 172.21.15.62:0/743682702 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc80089ce0 con 0x7efc9813d970 2024-04-25T09:13:14.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.929+0000 7efc97400640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc7406b4d0 0x7efc7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.930+0000 7efc97400640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc7406b4d0 0x7efc7406d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efc88034000 tx=0x7efc8802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:14.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:14.932+0000 7efc8fe00640 1 -- 172.21.15.62:0/743682702 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc80058ac0 con 0x7efc9813d970 2024-04-25T09:13:15.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.174+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc9815bea0 con 0x7efc9813d970 2024-04-25T09:13:15.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.175+0000 7efc8fe00640 1 -- 172.21.15.62:0/743682702 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4946 (secure 0 0 0) 0x7efc800583c0 con 0x7efc9813d970 2024-04-25T09:13:15.163 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:15.163 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1798148826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1798148826},{"type":"v1","addr":"172.21.15.121:6835","nonce":1798148826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2045746846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2045746846},{"type":"v1","addr":"172.21.15.121:6837","nonce":2045746846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1182284820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1182284820},{"type":"v1","addr":"172.21.15.62:6834","nonce":1182284820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:22.810299+0000","modified":"2024-04-25T09:12:25.868942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3997398243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3997398243},{"type":"v1","addr":"172.21.15.62:6835","nonce":3997398243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:13:15.164 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 64 2024-04-25T09:13:15.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc7406b4d0 msgr2=0x7efc7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc7406b4d0 0x7efc7406d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efc88034000 tx=0x7efc8802e000 comp rx=0 tx=0).stop 2024-04-25T09:13:15.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 msgr2=0x7efc98159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc98159810 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7efc8000b4a0 tx=0x7efc8000b960 comp rx=0 tx=0).stop 2024-04-25T09:13:15.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 shutdown_connections 2024-04-25T09:13:15.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efc7406b4d0 0x7efc7406d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc9813d970 0x7efc98159810 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efc9815acf0 0x7efc98159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 --2- 172.21.15.62:0/743682702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc980b1050 0x7efc9813a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.178+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 >> 172.21.15.62:0/743682702 conn(0x7efc980b14d0 msgr2=0x7efc9813f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:15.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.179+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 shutdown_connections 2024-04-25T09:13:15.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.179+0000 7efc9cc00640 1 -- 172.21.15.62:0/743682702 wait complete. 2024-04-25T09:13:15.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:15.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.411+0000 7f7e96400640 1 Processor -- start 2024-04-25T09:13:15.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.412+0000 7f7e96400640 1 -- start start 2024-04-25T09:13:15.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.413+0000 7f7e96400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7e900b6a90 0x7f7e900d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.413+0000 7f7e96400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 0x7f7e900d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.413+0000 7f7e96400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e900d3110 con 0x7f7e900b56f0 2024-04-25T09:13:15.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.413+0000 7f7e96400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e900cedc0 con 0x7f7e900b6a90 2024-04-25T09:13:15.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.413+0000 7f7e96400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900d33f0 con 0x7f7e900b27a0 2024-04-25T09:13:15.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.413+0000 7f7e95000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7e900b56f0 0x7f7e90160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35744/0 (socket says 172.21.15.62:35744) 2024-04-25T09:13:15.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.413+0000 7f7e95000640 1 -- 172.21.15.62:0/1029579298 learned_addr learned my addr 172.21.15.62:0/1029579298 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:15.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e95a00640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 0x7f7e900d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1029579298 >> v2:172.21.15.121:3300/0 conn(0x7f7e900b6a90 0x7f7e900d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616603490 0 0) 0x7f7e900d3110 con 0x7f7e900b56f0 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e74003520 con 0x7f7e900b56f0 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2880973813 0 0) 0x7f7e74003520 con 0x7f7e900b56f0 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 msgr2=0x7f7e900d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 0x7f7e900d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 >> v2:172.21.15.121:3300/0 conn(0x7f7e900b6a90 msgr2=0x7f7e900d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 --2- 172.21.15.62:0/1029579298 >> v2:172.21.15.121:3300/0 conn(0x7f7e900b6a90 0x7f7e900d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900d36d0 con 0x7f7e900b56f0 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f7e78001670 con 0x7f7e900b56f0 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1029579298 >> v2:172.21.15.121:3300/0 conn(0x7f7e900b6a90 0x7f7e900d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e95a00640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 0x7f7e900d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.414+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 >> v1:172.21.15.62:6789/0 conn(0x7f7e900b56f0 legacy=0x7f7e90160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e8f400640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 0x7f7e740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e8f400640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e74003fa0 0x7f7e74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e8f400640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74004d40 0x7f7e740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900cedc0 con 0x7f7e74003fa0 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900d33f0 con 0x7f7e74004d40 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7400baa0 con 0x7f7e900b27a0 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e74003fa0 0x7f7e74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e95a00640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74004d40 0x7f7e740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.415+0000 7f7e95000640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 0x7f7e740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.416+0000 7f7e8fe00640 1 -- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 msgr2=0x7f7e740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.416+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900b27a0 0x7f7e740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.416+0000 7f7e8fe00640 1 -- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74004d40 msgr2=0x7f7e740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.416+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74004d40 0x7f7e740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.416+0000 7f7e8fe00640 1 -- 172.21.15.62:0/1029579298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e74003520 con 0x7f7e74003fa0 2024-04-25T09:13:15.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.417+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e74003fa0 0x7f7e74004800 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7e800073b0 tx=0x7f7e74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41173a31b381bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.417+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80009c80 con 0x7f7e74003fa0 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.417+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8002c650 con 0x7f7e74003fa0 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.417+0000 7f7e8f400640 1 -- 172.21.15.62:0/1029579298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8002ca00 con 0x7f7e74003fa0 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.417+0000 7f7e96400640 1 -- 172.21.15.62:0/1029579298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e74003fa0 msgr2=0x7f7e74004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.417+0000 7f7e96400640 1 --2- 172.21.15.62:0/1029579298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e74003fa0 0x7f7e74004800 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7e800073b0 tx=0x7f7e74005830 comp rx=0 tx=0).stop 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.417+0000 7f7e95a00640 1 -- 172.21.15.62:0/1029579298 reap_dead start 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.418+0000 7f7e96400640 1 -- 172.21.15.62:0/1029579298 shutdown_connections 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.418+0000 7f7e96400640 1 -- 172.21.15.62:0/1029579298 >> 172.21.15.62:0/1029579298 conn(0x7f7e90066d10 msgr2=0x7f7e90067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.418+0000 7f7e96400640 1 -- 172.21.15.62:0/1029579298 shutdown_connections 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.418+0000 7f7e96400640 1 -- 172.21.15.62:0/1029579298 wait complete. 2024-04-25T09:13:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.419+0000 7f7e96400640 1 Processor -- start 2024-04-25T09:13:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.420+0000 7f7e96400640 1 -- start start 2024-04-25T09:13:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e96400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74003fa0 0x7f7e900cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e96400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e900cd210 0x7f7e9014ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e96400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900c8590 0x7f7e900c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e96400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900b2ee0 con 0x7f7e900cd210 2024-04-25T09:13:15.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e96400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900b2d60 con 0x7f7e74003fa0 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e96400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900b5ad0 con 0x7f7e900c8590 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e900cd210 0x7f7e9014ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e900cd210 0x7f7e9014ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39366/0 (socket says 172.21.15.62:39366) 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e8fe00640 1 -- 172.21.15.62:0/1887568525 learned_addr learned my addr 172.21.15.62:0/1887568525 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e95000640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74003fa0 0x7f7e900cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e95a00640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900c8590 0x7f7e900c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e8fe00640 1 -- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900c8590 msgr2=0x7f7e900c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900c8590 0x7f7e900c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e8fe00640 1 -- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74003fa0 msgr2=0x7f7e900cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74003fa0 0x7f7e900cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.421+0000 7f7e8fe00640 1 -- 172.21.15.62:0/1887568525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900cb580 con 0x7f7e900cd210 2024-04-25T09:13:15.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.422+0000 7f7e8fe00640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e900cd210 0x7f7e9014ede0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7e800074d0 tx=0x7f7e80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:15.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.422+0000 7f7e8cc00640 1 -- 172.21.15.62:0/1887568525 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80044020 con 0x7f7e900cd210 2024-04-25T09:13:15.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.422+0000 7f7e8cc00640 1 -- 172.21.15.62:0/1887568525 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8002d9f0 con 0x7f7e900cd210 2024-04-25T09:13:15.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.422+0000 7f7e8cc00640 1 -- 172.21.15.62:0/1887568525 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80009c80 con 0x7f7e900cd210 2024-04-25T09:13:15.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.422+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e900cb820 con 0x7f7e900cd210 2024-04-25T09:13:15.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.422+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e900cbde0 con 0x7f7e900cd210 2024-04-25T09:13:15.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.424+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9004ef20 con 0x7f7e900cd210 2024-04-25T09:13:15.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.424+0000 7f7e8cc00640 1 -- 172.21.15.62:0/1887568525 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e80047070 con 0x7f7e900cd210 2024-04-25T09:13:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.425+0000 7f7e8cc00640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e6c06b500 0x7f7e6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.426+0000 7f7e8cc00640 1 -- 172.21.15.62:0/1887568525 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e800aaa50 con 0x7f7e900cd210 2024-04-25T09:13:15.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.426+0000 7f7e95000640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e6c06b500 0x7f7e6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.427+0000 7f7e95000640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e6c06b500 0x7f7e6c06d9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7e78001eb0 tx=0x7f7e78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:15.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.430+0000 7f7e8cc00640 1 -- 172.21.15.62:0/1887568525 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e8003d070 con 0x7f7e900cd210 2024-04-25T09:13:15.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.634+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e900ce580 con 0x7f7e900cd210 2024-04-25T09:13:15.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.635+0000 7f7e8cc00640 1 -- 172.21.15.62:0/1887568525 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16660 (secure 0 0 0) 0x7f7e80032030 con 0x7f7e900cd210 2024-04-25T09:13:15.623 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:15.623 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":97,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:24.344250+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:20.654480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:21.833814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:15.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e6c06b500 msgr2=0x7f7e6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e6c06b500 0x7f7e6c06d9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7e78001eb0 tx=0x7f7e78009040 comp rx=0 tx=0).stop 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e900cd210 msgr2=0x7f7e9014ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e900cd210 0x7f7e9014ede0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7e800074d0 tx=0x7f7e80004ab0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 shutdown_connections 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e6c06b500 0x7f7e6c06d9c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e900cd210 0x7f7e9014ede0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e900c8590 0x7f7e900c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 --2- 172.21.15.62:0/1887568525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e74003fa0 0x7f7e900cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 >> 172.21.15.62:0/1887568525 conn(0x7f7e90066d10 msgr2=0x7f7e9015e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 shutdown_connections 2024-04-25T09:13:15.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.638+0000 7f7e96400640 1 -- 172.21.15.62:0/1887568525 wait complete. 2024-04-25T09:13:15.646 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:13:15.646 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.870+0000 7fddfb200640 1 Processor -- start 2024-04-25T09:13:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.870+0000 7fddfb200640 1 -- start start 2024-04-25T09:13:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.872+0000 7fddfb200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fddfc15f450 0x7fddfc0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.872+0000 7fddfb200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc15faa0 con 0x7fddfc06d900 2024-04-25T09:13:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.872+0000 7fddfb200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc15fd80 con 0x7fddfc144c10 2024-04-25T09:13:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.872+0000 7fddfb200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddfc144ff0 con 0x7fddfc15f450 2024-04-25T09:13:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.872+0000 7fddfa800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fddfc15f450 0x7fddfc0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.872+0000 7fddfa800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fddfc15f450 0x7fddfc0cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39374/0 (socket says 172.21.15.62:39374) 2024-04-25T09:13:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.872+0000 7fddfa800640 1 -- 172.21.15.62:0/3864776521 learned_addr learned my addr 172.21.15.62:0/3864776521 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.872+0000 7fddf9400640 1 --1- 172.21.15.62:0/3864776521 >> v1:172.21.15.62:6789/0 conn(0x7fddfc06d900 0x7fddfc0b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35746/0 (socket says 172.21.15.62:35746) 2024-04-25T09:13:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.873+0000 7fddfa800640 1 -- 172.21.15.62:0/3864776521 >> v1:172.21.15.121:6789/0 conn(0x7fddfc144c10 legacy=0x7fddfc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.873+0000 7fddfa800640 1 -- 172.21.15.62:0/3864776521 >> v1:172.21.15.62:6789/0 conn(0x7fddfc06d900 legacy=0x7fddfc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.873+0000 7fddfa800640 1 -- 172.21.15.62:0/3864776521 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc160060 con 0x7fddfc15f450 2024-04-25T09:13:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.874+0000 7fddfa800640 1 --2- 172.21.15.62:0/3864776521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc0cee40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009e50 tx=0x7fddf402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf87cc7c7b23f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.874+0000 7fddf8a00640 1 -- 172.21.15.62:0/3864776521 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4036070 con 0x7fddfc15f450 2024-04-25T09:13:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.874+0000 7fddf8a00640 1 -- 172.21.15.62:0/3864776521 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf4031600 con 0x7fddfc15f450 2024-04-25T09:13:15.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.874+0000 7fddf8a00640 1 -- 172.21.15.62:0/3864776521 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40319d0 con 0x7fddfc15f450 2024-04-25T09:13:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.875+0000 7fddfb200640 1 -- 172.21.15.62:0/3864776521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 msgr2=0x7fddfc0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.875+0000 7fddfb200640 1 --2- 172.21.15.62:0/3864776521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc0cee40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009e50 tx=0x7fddf402a6d0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.875+0000 7fddfb200640 1 -- 172.21.15.62:0/3864776521 shutdown_connections 2024-04-25T09:13:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.875+0000 7fddfb200640 1 --2- 172.21.15.62:0/3864776521 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc0cee40 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.875+0000 7fddfb200640 1 -- 172.21.15.62:0/3864776521 >> 172.21.15.62:0/3864776521 conn(0x7fddfc0b14d0 msgr2=0x7fddfc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.875+0000 7fddfb200640 1 -- 172.21.15.62:0/3864776521 shutdown_connections 2024-04-25T09:13:15.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.875+0000 7fddfb200640 1 -- 172.21.15.62:0/3864776521 wait complete. 2024-04-25T09:13:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.877+0000 7fddfb200640 1 Processor -- start 2024-04-25T09:13:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.877+0000 7fddfb200640 1 -- start start 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.878+0000 7fddfb200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddfc06d900 0x7fddfc13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.878+0000 7fddfb200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddfc144c10 0x7fddfc13fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.878+0000 7fddfb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.878+0000 7fddfb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc0b5030 con 0x7fddfc15f450 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.878+0000 7fddfb200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc15f880 con 0x7fddfc06d900 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.878+0000 7fddfb200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc0b51b0 con 0x7fddfc144c10 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.878+0000 7fddfa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddfa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39386/0 (socket says 172.21.15.62:39386) 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddfa800640 1 -- 172.21.15.62:0/2732285215 learned_addr learned my addr 172.21.15.62:0/2732285215 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddf9e00640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddfc06d900 0x7fddfc13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddf9400640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddfc144c10 0x7fddfc13fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddfa800640 1 -- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddfc144c10 msgr2=0x7fddfc13fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddfa800640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddfc144c10 0x7fddfc13fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddfa800640 1 -- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddfc06d900 msgr2=0x7fddfc13cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddfa800640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddfc06d900 0x7fddfc13cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddfa800640 1 -- 172.21.15.62:0/2732285215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0cc3a0 con 0x7fddfc15f450 2024-04-25T09:13:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.879+0000 7fddfa800640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc140290 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009e50 tx=0x7fddf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:15.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.880+0000 7fddf2000640 1 -- 172.21.15.62:0/2732285215 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4036050 con 0x7fddfc15f450 2024-04-25T09:13:15.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.880+0000 7fddf2000640 1 -- 172.21.15.62:0/2732285215 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf4042070 con 0x7fddfc15f450 2024-04-25T09:13:15.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.880+0000 7fddf2000640 1 -- 172.21.15.62:0/2732285215 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4009510 con 0x7fddfc15f450 2024-04-25T09:13:15.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.880+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc0cc680 con 0x7fddfc15f450 2024-04-25T09:13:15.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.880+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc0ccc40 con 0x7fddfc15f450 2024-04-25T09:13:15.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.881+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc04ef20 con 0x7fddfc15f450 2024-04-25T09:13:15.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.882+0000 7fddf2000640 1 -- 172.21.15.62:0/2732285215 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddf4007bd0 con 0x7fddfc15f450 2024-04-25T09:13:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.883+0000 7fddf2000640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdddc06b4d0 0x7fdddc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:15.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.883+0000 7fddf2000640 1 -- 172.21.15.62:0/2732285215 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddf40ab2d0 con 0x7fddfc15f450 2024-04-25T09:13:15.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.884+0000 7fddf9e00640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdddc06b4d0 0x7fdddc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:15.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.885+0000 7fddf9e00640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdddc06b4d0 0x7fdddc06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fddec004ea0 tx=0x7fddec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:15.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:15.887+0000 7fddf2000640 1 -- 172.21.15.62:0/2732285215 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddf407a0b0 con 0x7fddfc15f450 2024-04-25T09:13:16.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.125+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddfc0cc0f0 con 0x7fddfc15f450 2024-04-25T09:13:16.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.127+0000 7fddf2000640 1 -- 172.21.15.62:0/2732285215 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fddf40799b0 con 0x7fddfc15f450 2024-04-25T09:13:16.114 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:16.114 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1798148826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1798148826},{"type":"v1","addr":"172.21.15.121:6835","nonce":1798148826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2045746846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2045746846},{"type":"v1","addr":"172.21.15.121:6837","nonce":2045746846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1182284820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1182284820},{"type":"v1","addr":"172.21.15.62:6834","nonce":1182284820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:22.810299+0000","modified":"2024-04-25T09:12:25.868942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3997398243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3997398243},{"type":"v1","addr":"172.21.15.62:6835","nonce":3997398243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:13:16.115 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 64 2024-04-25T09:13:16.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.129+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdddc06b4d0 msgr2=0x7fdddc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.129+0000 7fddfb200640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdddc06b4d0 0x7fdddc06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fddec004ea0 tx=0x7fddec009040 comp rx=0 tx=0).stop 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.129+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 msgr2=0x7fddfc140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.129+0000 7fddfb200640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc140290 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009e50 tx=0x7fddf4004a40 comp rx=0 tx=0).stop 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.130+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 shutdown_connections 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.130+0000 7fddfb200640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdddc06b4d0 0x7fdddc06d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.130+0000 7fddfb200640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fddfc15f450 0x7fddfc140290 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.130+0000 7fddfb200640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fddfc144c10 0x7fddfc13fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.130+0000 7fddfb200640 1 --2- 172.21.15.62:0/2732285215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fddfc06d900 0x7fddfc13cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.130+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 >> 172.21.15.62:0/2732285215 conn(0x7fddfc0b14d0 msgr2=0x7fddfc13edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.130+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 shutdown_connections 2024-04-25T09:13:16.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.130+0000 7fddfb200640 1 -- 172.21.15.62:0/2732285215 wait complete. 2024-04-25T09:13:16.137 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.354+0000 7f5792a00640 1 Processor -- start 2024-04-25T09:13:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.355+0000 7f5792a00640 1 -- start start 2024-04-25T09:13:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.356+0000 7f5792a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f578c0b56f0 0x7f578c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.356+0000 7f5792a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f578c0b6a90 0x7f578c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.356+0000 7f5792a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578c0d3110 con 0x7f578c0b27a0 2024-04-25T09:13:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.356+0000 7f5792a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578c161780 con 0x7f578c0b6a90 2024-04-25T09:13:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.356+0000 7f5792a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578c0cedc0 con 0x7f578c0b56f0 2024-04-25T09:13:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f5792000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f578c0b27a0 0x7f578c161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35752/0 (socket says 172.21.15.62:35752) 2024-04-25T09:13:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f5792000640 1 -- 172.21.15.62:0/1727305450 learned_addr learned my addr 172.21.15.62:0/1727305450 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f5790c00640 1 --2- 172.21.15.62:0/1727305450 >> v2:172.21.15.62:3300/0 conn(0x7f578c0b6a90 0x7f578c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f5791600640 1 --2- 172.21.15.62:0/1727305450 >> v2:172.21.15.121:3300/0 conn(0x7f578c0b56f0 0x7f578c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f5790c00640 1 -- 172.21.15.62:0/1727305450 >> v2:172.21.15.121:3300/0 conn(0x7f578c0b56f0 msgr2=0x7f578c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f5790c00640 1 --2- 172.21.15.62:0/1727305450 >> v2:172.21.15.121:3300/0 conn(0x7f578c0b56f0 0x7f578c160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f5790c00640 1 -- 172.21.15.62:0/1727305450 >> v1:172.21.15.62:6789/0 conn(0x7f578c0b27a0 legacy=0x7f578c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f578be00640 1 -- 172.21.15.62:0/1727305450 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984278795 0 0) 0x7f578c0d3110 con 0x7f578c0b27a0 2024-04-25T09:13:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.357+0000 7f5790c00640 1 -- 172.21.15.62:0/1727305450 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0d33f0 con 0x7f578c0b6a90 2024-04-25T09:13:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.358+0000 7f5790c00640 1 --2- 172.21.15.62:0/1727305450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b6a90 0x7f578c161240 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f578c08c310 tx=0x7f577802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df125b62e01588ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.359+0000 7f578be00640 1 -- 172.21.15.62:0/1727305450 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778037070 con 0x7f578c0b6a90 2024-04-25T09:13:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.359+0000 7f578be00640 1 -- 172.21.15.62:0/1727305450 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5778032440 con 0x7f578c0b6a90 2024-04-25T09:13:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.359+0000 7f578be00640 1 -- 172.21.15.62:0/1727305450 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778032810 con 0x7f578c0b6a90 2024-04-25T09:13:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.359+0000 7f5792a00640 1 -- 172.21.15.62:0/1727305450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b6a90 msgr2=0x7f578c161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.359+0000 7f5792a00640 1 --2- 172.21.15.62:0/1727305450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b6a90 0x7f578c161240 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f578c08c310 tx=0x7f577802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.360+0000 7f5792a00640 1 -- 172.21.15.62:0/1727305450 shutdown_connections 2024-04-25T09:13:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.360+0000 7f5792a00640 1 --2- 172.21.15.62:0/1727305450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b6a90 0x7f578c161240 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.360+0000 7f5792a00640 1 --2- 172.21.15.62:0/1727305450 >> v2:172.21.15.121:3300/0 conn(0x7f578c0b56f0 0x7f578c160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.360+0000 7f5792a00640 1 -- 172.21.15.62:0/1727305450 >> 172.21.15.62:0/1727305450 conn(0x7f578c066d10 msgr2=0x7f578c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.360+0000 7f5792a00640 1 -- 172.21.15.62:0/1727305450 shutdown_connections 2024-04-25T09:13:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.360+0000 7f5792a00640 1 -- 172.21.15.62:0/1727305450 wait complete. 2024-04-25T09:13:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.361+0000 7f5792a00640 1 Processor -- start 2024-04-25T09:13:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.362+0000 7f5792a00640 1 -- start start 2024-04-25T09:13:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.362+0000 7f5792a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578c0b27a0 0x7f578c14bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5792a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b56f0 0x7f578c1e9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5792a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578c0b6a90 0x7f578c1e9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5792a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0b2d60 con 0x7f578c0b56f0 2024-04-25T09:13:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5792a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0cea50 con 0x7f578c0b6a90 2024-04-25T09:13:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5792a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0b2ee0 con 0x7f578c0b27a0 2024-04-25T09:13:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5790c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b56f0 0x7f578c1e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5790c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b56f0 0x7f578c1e9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39392/0 (socket says 172.21.15.62:39392) 2024-04-25T09:13:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5790c00640 1 -- 172.21.15.62:0/1069473373 learned_addr learned my addr 172.21.15.62:0/1069473373 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5790c00640 1 -- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578c0b27a0 msgr2=0x7f578c14bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5790c00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578c0b27a0 0x7f578c14bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5792000640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578c0b6a90 0x7f578c1e9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5790c00640 1 -- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578c0b6a90 msgr2=0x7f578c1e9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5790c00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578c0b6a90 0x7f578c1e9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.363+0000 7f5790c00640 1 -- 172.21.15.62:0/1069473373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c14c4f0 con 0x7f578c0b56f0 2024-04-25T09:13:16.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.364+0000 7f5790c00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b56f0 0x7f578c1e9240 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5778009890 tx=0x7f57780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.364+0000 7f5789600640 1 -- 172.21.15.62:0/1069473373 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778040020 con 0x7f578c0b56f0 2024-04-25T09:13:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.364+0000 7f5789600640 1 -- 172.21.15.62:0/1069473373 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5778038040 con 0x7f578c0b56f0 2024-04-25T09:13:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.364+0000 7f5789600640 1 -- 172.21.15.62:0/1069473373 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778037070 con 0x7f578c0b56f0 2024-04-25T09:13:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.364+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c0cd900 con 0x7f578c0b56f0 2024-04-25T09:13:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.364+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c14cb80 con 0x7f578c0b56f0 2024-04-25T09:13:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.366+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578c04ef20 con 0x7f578c0b56f0 2024-04-25T09:13:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.366+0000 7f5789600640 1 -- 172.21.15.62:0/1069473373 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5778043020 con 0x7f578c0b56f0 2024-04-25T09:13:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.367+0000 7f5789600640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b4d0 0x7f576c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.367+0000 7f5789600640 1 -- 172.21.15.62:0/1069473373 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57780aa800 con 0x7f578c0b56f0 2024-04-25T09:13:16.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.368+0000 7f5791600640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b4d0 0x7f576c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.369+0000 7f5791600640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b4d0 0x7f576c06d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f57800098f0 tx=0x7f5780008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:16.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.372+0000 7f5789600640 1 -- 172.21.15.62:0/1069473373 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f577804a030 con 0x7f578c0b56f0 2024-04-25T09:13:16.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.619+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f578c0ce580 con 0x7f578c0b56f0 2024-04-25T09:13:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.620+0000 7f5789600640 1 -- 172.21.15.62:0/1069473373 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5778032e20 con 0x7f578c0b56f0 2024-04-25T09:13:16.608 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:16.608 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1798148826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1798148826},{"type":"v1","addr":"172.21.15.121:6835","nonce":1798148826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2045746846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2045746846},{"type":"v1","addr":"172.21.15.121:6837","nonce":2045746846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1182284820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1182284820},{"type":"v1","addr":"172.21.15.62:6834","nonce":1182284820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:12:22.810299+0000","modified":"2024-04-25T09:12:25.868942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3997398243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3997398243},{"type":"v1","addr":"172.21.15.62:6835","nonce":3997398243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:13:16.608 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 64 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b4d0 msgr2=0x7f576c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b4d0 0x7f576c06d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f57800098f0 tx=0x7f5780008040 comp rx=0 tx=0).stop 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b56f0 msgr2=0x7f578c1e9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b56f0 0x7f578c1e9240 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5778009890 tx=0x7f57780046c0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 shutdown_connections 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b4d0 0x7f576c06d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f578c0b56f0 0x7f578c1e9240 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f578c0b6a90 0x7f578c1e9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 --2- 172.21.15.62:0/1069473373 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f578c0b27a0 0x7f578c14bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 >> 172.21.15.62:0/1069473373 conn(0x7f578c066d10 msgr2=0x7f578c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 shutdown_connections 2024-04-25T09:13:16.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.623+0000 7f5792a00640 1 -- 172.21.15.62:0/1069473373 wait complete. 2024-04-25T09:13:16.629 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:16.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.851+0000 7f39bb200640 1 Processor -- start 2024-04-25T09:13:16.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.851+0000 7f39bb200640 1 -- start start 2024-04-25T09:13:16.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.852+0000 7f39bb200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f39b415d810 0x7f39b40ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.852+0000 7f39bb200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39b415dbf0 0x7f39b406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.852+0000 7f39bb200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b406d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39bb200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40ced70 con 0x7f39b415dbf0 2024-04-25T09:13:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39bb200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40cf030 con 0x7f39b415d810 2024-04-25T09:13:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39bb200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40cf310 con 0x7f39b40b0ec0 2024-04-25T09:13:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39b9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39b415dbf0 0x7f39b406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39b9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39b415dbf0 0x7f39b406d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39404/0 (socket says 172.21.15.62:39404) 2024-04-25T09:13:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39b9400640 1 -- 172.21.15.62:0/3214495668 learned_addr learned my addr 172.21.15.62:0/3214495668 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39ba800640 1 --2- 172.21.15.62:0/3214495668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b406d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39b9e00640 1 --2- 172.21.15.62:0/3214495668 >> v2:172.21.15.121:3300/0 conn(0x7f39b415d810 0x7f39b40ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39b9400640 1 -- 172.21.15.62:0/3214495668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 msgr2=0x7f39b406d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.853+0000 7f39b9400640 1 --2- 172.21.15.62:0/3214495668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b406d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.854+0000 7f39b9400640 1 -- 172.21.15.62:0/3214495668 >> v2:172.21.15.121:3300/0 conn(0x7f39b415d810 msgr2=0x7f39b40ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.854+0000 7f39b9400640 1 --2- 172.21.15.62:0/3214495668 >> v2:172.21.15.121:3300/0 conn(0x7f39b415d810 0x7f39b40ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.854+0000 7f39b9400640 1 -- 172.21.15.62:0/3214495668 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40c8cd0 con 0x7f39b415dbf0 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.854+0000 7f39ba800640 1 --2- 172.21.15.62:0/3214495668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b406d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.854+0000 7f39b9e00640 1 --2- 172.21.15.62:0/3214495668 >> v2:172.21.15.121:3300/0 conn(0x7f39b415d810 0x7f39b40ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.855+0000 7f39b9400640 1 --2- 172.21.15.62:0/3214495668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b406d1f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f39b40b5a90 tx=0x7f39a402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d420b0220ba8a6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.855+0000 7f39b8a00640 1 -- 172.21.15.62:0/3214495668 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4037070 con 0x7f39b415dbf0 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.855+0000 7f39b8a00640 1 -- 172.21.15.62:0/3214495668 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a4032440 con 0x7f39b415dbf0 2024-04-25T09:13:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.855+0000 7f39b8a00640 1 -- 172.21.15.62:0/3214495668 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4032810 con 0x7f39b415dbf0 2024-04-25T09:13:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 -- 172.21.15.62:0/3214495668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 msgr2=0x7f39b406d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 --2- 172.21.15.62:0/3214495668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b406d1f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f39b40b5a90 tx=0x7f39a402a1d0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 -- 172.21.15.62:0/3214495668 shutdown_connections 2024-04-25T09:13:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 --2- 172.21.15.62:0/3214495668 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b406d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 --2- 172.21.15.62:0/3214495668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b406d1f0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 --2- 172.21.15.62:0/3214495668 >> v2:172.21.15.121:3300/0 conn(0x7f39b415d810 0x7f39b40ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 -- 172.21.15.62:0/3214495668 >> 172.21.15.62:0/3214495668 conn(0x7f39b40b14d0 msgr2=0x7f39b40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:16.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 -- 172.21.15.62:0/3214495668 shutdown_connections 2024-04-25T09:13:16.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.856+0000 7f39bb200640 1 -- 172.21.15.62:0/3214495668 wait complete. 2024-04-25T09:13:16.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.858+0000 7f39bb200640 1 Processor -- start 2024-04-25T09:13:16.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.858+0000 7f39bb200640 1 -- start start 2024-04-25T09:13:16.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39bb200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39bb200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39b415d810 0x7f39b4154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39bb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b4156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39bb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40c9650 con 0x7f39b415dbf0 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39bb200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40c94d0 con 0x7f39b415d810 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39bb200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40c97d0 con 0x7f39b40b0ec0 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39ba800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b4156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39ba800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b4156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39412/0 (socket says 172.21.15.62:39412) 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39ba800640 1 -- 172.21.15.62:0/2867328876 learned_addr learned my addr 172.21.15.62:0/2867328876 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.859+0000 7f39b9400640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39b415d810 0x7f39b4154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39b9e00640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39ba800640 1 -- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 msgr2=0x7f39b4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39ba800640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b4152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39ba800640 1 -- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39b415d810 msgr2=0x7f39b4154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39ba800640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39b415d810 0x7f39b4154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39ba800640 1 -- 172.21.15.62:0/2867328876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4151e10 con 0x7f39b415dbf0 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39b9400640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39b415d810 0x7f39b4154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39ba800640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b4156d40 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f39b000cef0 tx=0x7f39b000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:16.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.860+0000 7f39ae000640 1 -- 172.21.15.62:0/2867328876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b001a070 con 0x7f39b415dbf0 2024-04-25T09:13:16.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.861+0000 7f39ae000640 1 -- 172.21.15.62:0/2867328876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b000d860 con 0x7f39b415dbf0 2024-04-25T09:13:16.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.861+0000 7f39ae000640 1 -- 172.21.15.62:0/2867328876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b000dc50 con 0x7f39b415dbf0 2024-04-25T09:13:16.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.861+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b41520f0 con 0x7f39b415dbf0 2024-04-25T09:13:16.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.861+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b41526b0 con 0x7f39b415dbf0 2024-04-25T09:13:16.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.862+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b404ef20 con 0x7f39b415dbf0 2024-04-25T09:13:16.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.863+0000 7f39ae000640 1 -- 172.21.15.62:0/2867328876 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39b001d070 con 0x7f39b415dbf0 2024-04-25T09:13:16.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.864+0000 7f39ae000640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f399806b4d0 0x7f399806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:16.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.864+0000 7f39ae000640 1 -- 172.21.15.62:0/2867328876 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39b008ab20 con 0x7f39b415dbf0 2024-04-25T09:13:16.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.864+0000 7f39b9e00640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f399806b4d0 0x7f399806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:16.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.866+0000 7f39b9e00640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f399806b4d0 0x7f399806d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f39a800a910 tx=0x7f39a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:16.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:16.868+0000 7f39ae000640 1 -- 172.21.15.62:0/2867328876 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39b00599c0 con 0x7f39b415dbf0 2024-04-25T09:13:17.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.061+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39b41536b0 con 0x7f39b415dbf0 2024-04-25T09:13:17.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.063+0000 7f39ae000640 1 -- 172.21.15.62:0/2867328876 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16660 (secure 0 0 0) 0x7f39b00592c0 con 0x7f39b415dbf0 2024-04-25T09:13:17.051 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:17.052 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":97,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:12:24.344250+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:12:20.654480+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:12:21.833814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:17.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f399806b4d0 msgr2=0x7f399806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f399806b4d0 0x7f399806d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f39a800a910 tx=0x7f39a8008040 comp rx=0 tx=0).stop 2024-04-25T09:13:17.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 msgr2=0x7f39b4156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b4156d40 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f39b000cef0 tx=0x7f39b000a710 comp rx=0 tx=0).stop 2024-04-25T09:13:17.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 shutdown_connections 2024-04-25T09:13:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f399806b4d0 0x7f399806d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39b415dbf0 0x7f39b4156d40 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39b415d810 0x7f39b4154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 --2- 172.21.15.62:0/2867328876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f39b40b0ec0 0x7f39b4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.066+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 >> 172.21.15.62:0/2867328876 conn(0x7f39b40b14d0 msgr2=0x7f39b40cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.067+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 shutdown_connections 2024-04-25T09:13:17.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.067+0000 7f39bb200640 1 -- 172.21.15.62:0/2867328876 wait complete. 2024-04-25T09:13:17.073 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:13:17.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.292+0000 7f1a4e400640 1 Processor -- start 2024-04-25T09:13:17.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.292+0000 7f1a4e400640 1 -- start start 2024-04-25T09:13:17.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a4e400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1a480aeb30 0x7f1a4806ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a4e400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a4806d3f0 0x7f1a480b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a4e400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aef10 0x7f1a480b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a4e400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4806d8a0 con 0x7f1a4806d3f0 2024-04-25T09:13:17.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a4e400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4806db60 con 0x7f1a480aeb30 2024-04-25T09:13:17.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a4e400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4806de40 con 0x7f1a480aef10 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a47e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a4806d3f0 0x7f1a480b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a47e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a4806d3f0 0x7f1a480b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39426/0 (socket says 172.21.15.62:39426) 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.294+0000 7f1a47e00640 1 -- 172.21.15.62:0/3606301709 learned_addr learned my addr 172.21.15.62:0/3606301709 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.295+0000 7f1a4d000640 1 --2- 172.21.15.62:0/3606301709 >> v2:172.21.15.121:3300/0 conn(0x7f1a480aeb30 0x7f1a4806ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.295+0000 7f1a4da00640 1 --2- 172.21.15.62:0/3606301709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aef10 0x7f1a480b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.295+0000 7f1a47e00640 1 -- 172.21.15.62:0/3606301709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aef10 msgr2=0x7f1a480b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.295+0000 7f1a47e00640 1 --2- 172.21.15.62:0/3606301709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aef10 0x7f1a480b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.295+0000 7f1a47e00640 1 -- 172.21.15.62:0/3606301709 >> v2:172.21.15.121:3300/0 conn(0x7f1a480aeb30 msgr2=0x7f1a4806ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.295+0000 7f1a47e00640 1 --2- 172.21.15.62:0/3606301709 >> v2:172.21.15.121:3300/0 conn(0x7f1a480aeb30 0x7f1a4806ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.295+0000 7f1a47e00640 1 -- 172.21.15.62:0/3606301709 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4815b150 con 0x7f1a4806d3f0 2024-04-25T09:13:17.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.296+0000 7f1a47e00640 1 --2- 172.21.15.62:0/3606301709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a4806d3f0 0x7f1a480b3190 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f1a4806dd70 tx=0x7f1a3402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7588dfe4042424b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:17.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a47400640 1 -- 172.21.15.62:0/3606301709 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34037070 con 0x7f1a4806d3f0 2024-04-25T09:13:17.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a47400640 1 -- 172.21.15.62:0/3606301709 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34032440 con 0x7f1a4806d3f0 2024-04-25T09:13:17.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a47400640 1 -- 172.21.15.62:0/3606301709 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34032810 con 0x7f1a4806d3f0 2024-04-25T09:13:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a4e400640 1 -- 172.21.15.62:0/3606301709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a4806d3f0 msgr2=0x7f1a480b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a4e400640 1 --2- 172.21.15.62:0/3606301709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a4806d3f0 0x7f1a480b3190 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f1a4806dd70 tx=0x7f1a3402a130 comp rx=0 tx=0).stop 2024-04-25T09:13:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a4e400640 1 -- 172.21.15.62:0/3606301709 shutdown_connections 2024-04-25T09:13:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a4e400640 1 --2- 172.21.15.62:0/3606301709 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aef10 0x7f1a480b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a4e400640 1 --2- 172.21.15.62:0/3606301709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a4806d3f0 0x7f1a480b3190 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a4e400640 1 --2- 172.21.15.62:0/3606301709 >> v2:172.21.15.121:3300/0 conn(0x7f1a480aeb30 0x7f1a4806ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.297+0000 7f1a4e400640 1 -- 172.21.15.62:0/3606301709 >> 172.21.15.62:0/3606301709 conn(0x7f1a480668f0 msgr2=0x7f1a48066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:17.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.298+0000 7f1a4e400640 1 -- 172.21.15.62:0/3606301709 shutdown_connections 2024-04-25T09:13:17.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.298+0000 7f1a4e400640 1 -- 172.21.15.62:0/3606301709 wait complete. 2024-04-25T09:13:17.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.300+0000 7f1a4e400640 1 Processor -- start 2024-04-25T09:13:17.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.300+0000 7f1a4e400640 1 -- start start 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.301+0000 7f1a4e400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a4806d3f0 0x7f1a48148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.301+0000 7f1a4e400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aeb30 0x7f1a481f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.301+0000 7f1a4e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a480aef10 0x7f1a480b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.301+0000 7f1a4e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4815ae90 con 0x7f1a480aef10 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.301+0000 7f1a4e400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4815ad10 con 0x7f1a4806d3f0 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.301+0000 7f1a4e400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4815b670 con 0x7f1a480aeb30 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a480aef10 0x7f1a480b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a480aef10 0x7f1a480b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39428/0 (socket says 172.21.15.62:39428) 2024-04-25T09:13:17.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4da00640 1 -- 172.21.15.62:0/1490740399 learned_addr learned my addr 172.21.15.62:0/1490740399 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:17.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4d000640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a4806d3f0 0x7f1a48148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a47e00640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aeb30 0x7f1a481f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4da00640 1 -- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aeb30 msgr2=0x7f1a481f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4da00640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aeb30 0x7f1a481f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4da00640 1 -- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a4806d3f0 msgr2=0x7f1a48148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4da00640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a4806d3f0 0x7f1a48148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a4da00640 1 -- 172.21.15.62:0/1490740399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a481f4820 con 0x7f1a480aef10 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.302+0000 7f1a47e00640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aeb30 0x7f1a481f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.303+0000 7f1a4da00640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a480aef10 0x7f1a480b1650 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000cc30 tx=0x7f1a4000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.303+0000 7f1a44c00640 1 -- 172.21.15.62:0/1490740399 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000ce40 con 0x7f1a480aef10 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.303+0000 7f1a44c00640 1 -- 172.21.15.62:0/1490740399 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a40003990 con 0x7f1a480aef10 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.303+0000 7f1a44c00640 1 -- 172.21.15.62:0/1490740399 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40003d40 con 0x7f1a480aef10 2024-04-25T09:13:17.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.303+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a481f4b00 con 0x7f1a480aef10 2024-04-25T09:13:17.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.303+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a481e44e0 con 0x7f1a480aef10 2024-04-25T09:13:17.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.304+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4804ef20 con 0x7f1a480aef10 2024-04-25T09:13:17.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.306+0000 7f1a44c00640 1 -- 172.21.15.62:0/1490740399 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a40003ee0 con 0x7f1a480aef10 2024-04-25T09:13:17.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.307+0000 7f1a44c00640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a2806b4d0 0x7f1a2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.307+0000 7f1a44c00640 1 -- 172.21.15.62:0/1490740399 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a4008a670 con 0x7f1a480aef10 2024-04-25T09:13:17.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.308+0000 7f1a4d000640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a2806b4d0 0x7f1a2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.309+0000 7f1a4d000640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a2806b4d0 0x7f1a2806d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c006fd0 tx=0x7f1a3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:17.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.312+0000 7f1a44c00640 1 -- 172.21.15.62:0/1490740399 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a40014030 con 0x7f1a480aef10 2024-04-25T09:13:17.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.553+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1a480cf4f0 con 0x7f1a480aef10 2024-04-25T09:13:17.541 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:13:17.554+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:13:17.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.557+0000 7f1a44c00640 1 -- 172.21.15.62:0/1490740399 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1a400594e0 con 0x7f1a480aef10 2024-04-25T09:13:17.545 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:13:17.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a2806b4d0 msgr2=0x7f1a2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a2806b4d0 0x7f1a2806d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c006fd0 tx=0x7f1a3c008040 comp rx=0 tx=0).stop 2024-04-25T09:13:17.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a480aef10 msgr2=0x7f1a480b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a480aef10 0x7f1a480b1650 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000cc30 tx=0x7f1a4000a5c0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 shutdown_connections 2024-04-25T09:13:17.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1a2806b4d0 0x7f1a2806d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a480aef10 0x7f1a480b1650 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a480aeb30 0x7f1a481f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 --2- 172.21.15.62:0/1490740399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a4806d3f0 0x7f1a48148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 >> 172.21.15.62:0/1490740399 conn(0x7f1a480668f0 msgr2=0x7f1a480ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:17.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.560+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 shutdown_connections 2024-04-25T09:13:17.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.561+0000 7f1a4e400640 1 -- 172.21.15.62:0/1490740399 wait complete. 2024-04-25T09:13:17.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:13:17.630 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:13:17.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.764+0000 7f0413200640 1 Processor -- start 2024-04-25T09:13:17.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.764+0000 7f0413200640 1 -- start start 2024-04-25T09:13:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.766+0000 7f0413200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04140ccde0 0x7f04140ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.766+0000 7f0413200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f04140b31a0 0x7f04140b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.766+0000 7f0413200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04140ced40 con 0x7f041406d2c0 2024-04-25T09:13:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.766+0000 7f0413200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140cd260 con 0x7f04140b31a0 2024-04-25T09:13:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.766+0000 7f0413200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140cf000 con 0x7f04140ccde0 2024-04-25T09:13:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.766+0000 7f0411e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f041406d2c0 0x7f041406d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35762/0 (socket says 172.21.15.62:35762) 2024-04-25T09:13:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.766+0000 7f0411e00640 1 -- 172.21.15.62:0/3553316391 learned_addr learned my addr 172.21.15.62:0/3553316391 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.766+0000 7f0412800640 1 --2- 172.21.15.62:0/3553316391 >> v2:172.21.15.62:3300/0 conn(0x7f04140b31a0 0x7f04140b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.767+0000 7f0411400640 1 --2- 172.21.15.62:0/3553316391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04140ccde0 0x7f04140ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.767+0000 7f0410a00640 1 -- 172.21.15.62:0/3553316391 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832290633 0 0) 0x7f04140ced40 con 0x7f041406d2c0 2024-04-25T09:13:17.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.767+0000 7f0410a00640 1 -- 172.21.15.62:0/3553316391 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03f8003520 con 0x7f041406d2c0 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.767+0000 7f0412800640 1 -- 172.21.15.62:0/3553316391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04140ccde0 msgr2=0x7f04140ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.767+0000 7f0412800640 1 --2- 172.21.15.62:0/3553316391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04140ccde0 0x7f04140ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.767+0000 7f0412800640 1 -- 172.21.15.62:0/3553316391 >> v1:172.21.15.62:6789/0 conn(0x7f041406d2c0 legacy=0x7f041406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.767+0000 7f0412800640 1 -- 172.21.15.62:0/3553316391 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140cf2e0 con 0x7f04140b31a0 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.768+0000 7f0412800640 1 --2- 172.21.15.62:0/3553316391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140b31a0 0x7f04140b3f40 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f040c0097a0 tx=0x7f040c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e5ba22f0aa174a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.768+0000 7f0410a00640 1 -- 172.21.15.62:0/3553316391 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c037070 con 0x7f04140b31a0 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.768+0000 7f0410a00640 1 -- 172.21.15.62:0/3553316391 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c032440 con 0x7f04140b31a0 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.768+0000 7f0410a00640 1 -- 172.21.15.62:0/3553316391 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c032810 con 0x7f04140b31a0 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.769+0000 7f0413200640 1 -- 172.21.15.62:0/3553316391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140b31a0 msgr2=0x7f04140b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.769+0000 7f0413200640 1 --2- 172.21.15.62:0/3553316391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140b31a0 0x7f04140b3f40 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f040c0097a0 tx=0x7f040c02a180 comp rx=0 tx=0).stop 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.769+0000 7f0413200640 1 -- 172.21.15.62:0/3553316391 shutdown_connections 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.769+0000 7f0413200640 1 --2- 172.21.15.62:0/3553316391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140b31a0 0x7f04140b3f40 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.769+0000 7f0413200640 1 --2- 172.21.15.62:0/3553316391 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f04140ccde0 0x7f04140ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.769+0000 7f0413200640 1 -- 172.21.15.62:0/3553316391 >> 172.21.15.62:0/3553316391 conn(0x7f0414066f20 msgr2=0x7f04140b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:17.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.769+0000 7f0413200640 1 -- 172.21.15.62:0/3553316391 shutdown_connections 2024-04-25T09:13:17.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.769+0000 7f0413200640 1 -- 172.21.15.62:0/3553316391 wait complete. 2024-04-25T09:13:17.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.771+0000 7f0413200640 1 Processor -- start 2024-04-25T09:13:17.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.771+0000 7f0413200640 1 -- start start 2024-04-25T09:13:17.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0413200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041406d2c0 0x7f0414139e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0413200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04140b31a0 0x7f04140ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0413200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140ccde0 0x7f04140ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0413200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140b2c80 con 0x7f04140ccde0 2024-04-25T09:13:17.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0413200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140b29a0 con 0x7f04140b31a0 2024-04-25T09:13:17.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0413200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04141448d0 con 0x7f041406d2c0 2024-04-25T09:13:17.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0412800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140ccde0 0x7f04140ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0412800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140ccde0 0x7f04140ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39472/0 (socket says 172.21.15.62:39472) 2024-04-25T09:13:17.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0412800640 1 -- 172.21.15.62:0/2514250494 learned_addr learned my addr 172.21.15.62:0/2514250494 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:17.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.772+0000 7f0411e00640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041406d2c0 0x7f0414139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0411400640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04140b31a0 0x7f04140ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0412800640 1 -- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041406d2c0 msgr2=0x7f0414139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0412800640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041406d2c0 0x7f0414139e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0412800640 1 -- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04140b31a0 msgr2=0x7f04140ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0412800640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04140b31a0 0x7f04140ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0412800640 1 -- 172.21.15.62:0/2514250494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041413aac0 con 0x7f04140ccde0 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0411e00640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041406d2c0 0x7f0414139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0412800640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140ccde0 0x7f04140ade20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f040c035040 tx=0x7f040c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f040a000640 1 -- 172.21.15.62:0/2514250494 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c037040 con 0x7f04140ccde0 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f040a000640 1 -- 172.21.15.62:0/2514250494 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c042070 con 0x7f04140ccde0 2024-04-25T09:13:17.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f040a000640 1 -- 172.21.15.62:0/2514250494 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c0308a0 con 0x7f04140ccde0 2024-04-25T09:13:17.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041413ada0 con 0x7f04140ccde0 2024-04-25T09:13:17.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.773+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041413b360 con 0x7f04140ccde0 2024-04-25T09:13:17.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.775+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041406abe0 con 0x7f04140ccde0 2024-04-25T09:13:17.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.776+0000 7f040a000640 1 -- 172.21.15.62:0/2514250494 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f040c030a40 con 0x7f04140ccde0 2024-04-25T09:13:17.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.777+0000 7f040a000640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03f006b4d0 0x7f03f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:17.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.777+0000 7f040a000640 1 -- 172.21.15.62:0/2514250494 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f040c0ab5e0 con 0x7f04140ccde0 2024-04-25T09:13:17.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.777+0000 7f0411e00640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03f006b4d0 0x7f03f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:17.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.778+0000 7f0411e00640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03f006b4d0 0x7f03f006d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0404005c00 tx=0x7f0404009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:17.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:17.781+0000 7f040a000640 1 -- 172.21.15.62:0/2514250494 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f040c07a450 con 0x7f04140ccde0 2024-04-25T09:13:17.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.010+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f04141451f0 con 0x7f04140ccde0 2024-04-25T09:13:18.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.591+0000 7f040a000640 1 -- 172.21.15.62:0/2514250494 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7f040c079d50 con 0x7f04140ccde0 2024-04-25T09:13:18.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03f006b4d0 msgr2=0x7f03f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:18.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03f006b4d0 0x7f03f006d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0404005c00 tx=0x7f0404009040 comp rx=0 tx=0).stop 2024-04-25T09:13:18.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140ccde0 msgr2=0x7f04140ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:18.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140ccde0 0x7f04140ade20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f040c035040 tx=0x7f040c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 shutdown_connections 2024-04-25T09:13:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03f006b4d0 0x7f03f006d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04140ccde0 0x7f04140ade20 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04140b31a0 0x7f04140ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.594+0000 7f0413200640 1 --2- 172.21.15.62:0/2514250494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f041406d2c0 0x7f0414139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.595+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 >> 172.21.15.62:0/2514250494 conn(0x7f0414066f20 msgr2=0x7f041406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.595+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 shutdown_connections 2024-04-25T09:13:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.595+0000 7f0413200640 1 -- 172.21.15.62:0/2514250494 wait complete. 2024-04-25T09:13:18.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:13:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.827+0000 7f0fafe00640 1 Processor -- start 2024-04-25T09:13:18.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.827+0000 7f0fafe00640 1 -- start start 2024-04-25T09:13:18.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.828+0000 7f0fafe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0fb0145140 0x7f0fb00b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.828+0000 7f0fafe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0fb006c440 0x7f0fb006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.828+0000 7f0fafe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb006ce10 con 0x7f0fb00b4730 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.828+0000 7f0fafe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb00b3d60 con 0x7f0fb006c440 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.828+0000 7f0fafe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb015c9a0 con 0x7f0fb0145140 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.828+0000 7f0faf400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0fb006c440 0x7f0fb006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0faf400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0fb006c440 0x7f0fb006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39480/0 (socket says 172.21.15.62:39480) 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0faf400640 1 -- 172.21.15.62:0/413283202 learned_addr learned my addr 172.21.15.62:0/413283202 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0fa6000640 1 --1- 172.21.15.62:0/413283202 >> v1:172.21.15.62:6789/0 conn(0x7f0fb00b4730 0x7f0fb015beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35778/0 (socket says 172.21.15.62:35778) 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0faea00640 1 --2- 172.21.15.62:0/413283202 >> v2:172.21.15.121:3300/0 conn(0x7f0fb0145140 0x7f0fb00b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0faf400640 1 -- 172.21.15.62:0/413283202 >> v2:172.21.15.121:3300/0 conn(0x7f0fb0145140 msgr2=0x7f0fb00b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:18.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0faf400640 1 --2- 172.21.15.62:0/413283202 >> v2:172.21.15.121:3300/0 conn(0x7f0fb0145140 0x7f0fb00b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0faf400640 1 -- 172.21.15.62:0/413283202 >> v1:172.21.15.62:6789/0 conn(0x7f0fb00b4730 legacy=0x7f0fb015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0faf400640 1 -- 172.21.15.62:0/413283202 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb015cc80 con 0x7f0fb006c440 2024-04-25T09:13:18.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.829+0000 7f0fae000640 1 -- 172.21.15.62:0/413283202 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988569340 0 0) 0x7f0fb006ce10 con 0x7f0fb00b4730 2024-04-25T09:13:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.830+0000 7f0faf400640 1 --2- 172.21.15.62:0/413283202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb006c440 0x7f0fb006c8d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0fb006e5b0 tx=0x7f0fa802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09095138616d470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.830+0000 7f0fae000640 1 -- 172.21.15.62:0/413283202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8036070 con 0x7f0fb006c440 2024-04-25T09:13:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.830+0000 7f0fae000640 1 -- 172.21.15.62:0/413283202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa8031600 con 0x7f0fb006c440 2024-04-25T09:13:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.830+0000 7f0fae000640 1 -- 172.21.15.62:0/413283202 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80319d0 con 0x7f0fb006c440 2024-04-25T09:13:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.831+0000 7f0fafe00640 1 -- 172.21.15.62:0/413283202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb006c440 msgr2=0x7f0fb006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.831+0000 7f0fafe00640 1 --2- 172.21.15.62:0/413283202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb006c440 0x7f0fb006c8d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0fb006e5b0 tx=0x7f0fa802a730 comp rx=0 tx=0).stop 2024-04-25T09:13:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.831+0000 7f0fafe00640 1 -- 172.21.15.62:0/413283202 shutdown_connections 2024-04-25T09:13:18.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.831+0000 7f0fafe00640 1 --2- 172.21.15.62:0/413283202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb006c440 0x7f0fb006c8d0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.831+0000 7f0fafe00640 1 --2- 172.21.15.62:0/413283202 >> v2:172.21.15.121:3300/0 conn(0x7f0fb0145140 0x7f0fb00b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.831+0000 7f0fafe00640 1 -- 172.21.15.62:0/413283202 >> 172.21.15.62:0/413283202 conn(0x7f0fb0066ad0 msgr2=0x7f0fb0066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.831+0000 7f0fafe00640 1 -- 172.21.15.62:0/413283202 shutdown_connections 2024-04-25T09:13:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.831+0000 7f0fafe00640 1 -- 172.21.15.62:0/413283202 wait complete. 2024-04-25T09:13:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.833+0000 7f0fafe00640 1 Processor -- start 2024-04-25T09:13:18.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.833+0000 7f0fafe00640 1 -- start start 2024-04-25T09:13:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fafe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb006c440 0x7f0fb00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fafe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb0145140 0x7f0fb01f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fafe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb00c6220 0x7f0fb01308c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:18.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fafe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b51a0 con 0x7f0fb0145140 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fafe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b5020 con 0x7f0fb00c6220 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fafe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb015c5c0 con 0x7f0fb006c440 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fa6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb0145140 0x7f0fb01f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fa6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb0145140 0x7f0fb01f0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39490/0 (socket says 172.21.15.62:39490) 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.834+0000 7f0fa6000640 1 -- 172.21.15.62:0/1585248893 learned_addr learned my addr 172.21.15.62:0/1585248893 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0faea00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb006c440 0x7f0fb00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0faf400640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb00c6220 0x7f0fb01308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa6000640 1 -- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb006c440 msgr2=0x7f0fb00cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa6000640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb006c440 0x7f0fb00cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa6000640 1 -- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb00c6220 msgr2=0x7f0fb01308c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa6000640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb00c6220 0x7f0fb01308c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa6000640 1 -- 172.21.15.62:0/1585248893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00cb2a0 con 0x7f0fb0145140 2024-04-25T09:13:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa6000640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb0145140 0x7f0fb01f0b80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800c4a0 tx=0x7f0f9800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa7e00640 1 -- 172.21.15.62:0/1585248893 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98018070 con 0x7f0fb0145140 2024-04-25T09:13:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa7e00640 1 -- 172.21.15.62:0/1585248893 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f980074e0 con 0x7f0fb0145140 2024-04-25T09:13:18.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.835+0000 7f0fa7e00640 1 -- 172.21.15.62:0/1585248893 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98004a20 con 0x7f0fb0145140 2024-04-25T09:13:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.836+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb00cb580 con 0x7f0fb0145140 2024-04-25T09:13:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.836+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb01300c0 con 0x7f0fb0145140 2024-04-25T09:13:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.837+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb0072b30 con 0x7f0fb0145140 2024-04-25T09:13:18.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.838+0000 7f0fa7e00640 1 -- 172.21.15.62:0/1585248893 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f98004bc0 con 0x7f0fb0145140 2024-04-25T09:13:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.839+0000 7f0fa7e00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f8c06b500 0x7f0f8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:18.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.839+0000 7f0fa7e00640 1 -- 172.21.15.62:0/1585248893 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0f980894b0 con 0x7f0fb0145140 2024-04-25T09:13:18.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.840+0000 7f0faea00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f8c06b500 0x7f0f8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:18.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.841+0000 7f0faea00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f8c06b500 0x7f0f8c06d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0006fd0 tx=0x7f0fa0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:18.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:18.844+0000 7f0fa7e00640 1 -- 172.21.15.62:0/1585248893 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f980582c0 con 0x7f0fb0145140 2024-04-25T09:13:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.044+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0fb00750e0 con 0x7f0fb0145140 2024-04-25T09:13:19.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.618+0000 7f0fa7e00640 1 -- 172.21.15.62:0/1585248893 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0f98057bc0 con 0x7f0fb0145140 2024-04-25T09:13:19.606 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:13:19.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f8c06b500 msgr2=0x7f0f8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:19.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f8c06b500 0x7f0f8c06d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0006fd0 tx=0x7f0fa0008040 comp rx=0 tx=0).stop 2024-04-25T09:13:19.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb0145140 msgr2=0x7f0fb01f0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:19.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb0145140 0x7f0fb01f0b80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800c4a0 tx=0x7f0f9800c960 comp rx=0 tx=0).stop 2024-04-25T09:13:19.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 shutdown_connections 2024-04-25T09:13:19.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f8c06b500 0x7f0f8c06d9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0fb0145140 0x7f0fb01f0b80 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb00c6220 0x7f0fb01308c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 --2- 172.21.15.62:0/1585248893 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb006c440 0x7f0fb00cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.621+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 >> 172.21.15.62:0/1585248893 conn(0x7f0fb0066ad0 msgr2=0x7f0fb00cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:19.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.622+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 shutdown_connections 2024-04-25T09:13:19.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.622+0000 7f0fafe00640 1 -- 172.21.15.62:0/1585248893 wait complete. 2024-04-25T09:13:19.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:13:19.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.856+0000 7f6896c00640 1 Processor -- start 2024-04-25T09:13:19.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.856+0000 7f6896c00640 1 -- start start 2024-04-25T09:13:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.857+0000 7f6896c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689015c6e0 0x7f6890163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.857+0000 7f6896c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6890163640 con 0x7f68900705f0 2024-04-25T09:13:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.857+0000 7f6896c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6890163840 con 0x7f689006c990 2024-04-25T09:13:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.857+0000 7f6896c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b44b0 con 0x7f689015c6e0 2024-04-25T09:13:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.858+0000 7f6887e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f68900705f0 0x7f689006c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35788/0 (socket says 172.21.15.62:35788) 2024-04-25T09:13:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.858+0000 7f6887e00640 1 -- 172.21.15.62:0/1003566049 learned_addr learned my addr 172.21.15.62:0/1003566049 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:19.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.858+0000 7f688fe00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689015c6e0 0x7f6890163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:19.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.858+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473905184 0 0) 0x7f6878001110 con 0x7f68900705f0 2024-04-25T09:13:19.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.858+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6874003520 con 0x7f68900705f0 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2158047895 0 0) 0x7f6874003520 con 0x7f68900705f0 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689015c6e0 msgr2=0x7f6890163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689015c6e0 0x7f6890163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 >> v1:172.21.15.121:6789/0 conn(0x7f689006c990 legacy=0x7f68900b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900b4700 con 0x7f68900705f0 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f6878001840 con 0x7f68900705f0 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 >> v1:172.21.15.62:6789/0 conn(0x7f68900705f0 legacy=0x7f689006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68740039e0 0x7f6874003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689015c6e0 0x7f6874004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6874004d40 0x7f68740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b44b0 con 0x7f68740039e0 2024-04-25T09:13:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687400bb50 con 0x7f6874004d40 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687400bd50 con 0x7f689015c6e0 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.859+0000 7f688fe00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68740039e0 0x7f6874003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.860+0000 7f6887e00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689015c6e0 0x7f6874004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.860+0000 7f6894c00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6874004d40 0x7f68740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.860+0000 7f688fe00640 1 -- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689015c6e0 msgr2=0x7f6874004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.860+0000 7f688fe00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689015c6e0 0x7f6874004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.860+0000 7f688fe00640 1 -- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6874004d40 msgr2=0x7f68740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.860+0000 7f688fe00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6874004d40 0x7f68740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.860+0000 7f688fe00640 1 -- 172.21.15.62:0/1003566049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6874003520 con 0x7f68740039e0 2024-04-25T09:13:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.860+0000 7f6894c00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6874004d40 0x7f68740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.861+0000 7f688fe00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68740039e0 0x7f6874003e20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f6880009ac0 tx=0x7f6874005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67162ad5a4555433 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.861+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880036070 con 0x7f68740039e0 2024-04-25T09:13:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.861+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880031400 con 0x7f68740039e0 2024-04-25T09:13:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.861+0000 7f688f400640 1 -- 172.21.15.62:0/1003566049 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880009690 con 0x7f68740039e0 2024-04-25T09:13:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.862+0000 7f6896c00640 1 -- 172.21.15.62:0/1003566049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68740039e0 msgr2=0x7f6874003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.862+0000 7f6896c00640 1 --2- 172.21.15.62:0/1003566049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68740039e0 0x7f6874003e20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f6880009ac0 tx=0x7f6874005830 comp rx=0 tx=0).stop 2024-04-25T09:13:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.862+0000 7f6894c00640 1 -- 172.21.15.62:0/1003566049 reap_dead start 2024-04-25T09:13:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.862+0000 7f6896c00640 1 -- 172.21.15.62:0/1003566049 shutdown_connections 2024-04-25T09:13:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.862+0000 7f6896c00640 1 -- 172.21.15.62:0/1003566049 >> 172.21.15.62:0/1003566049 conn(0x7f68900668f0 msgr2=0x7f6890066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.862+0000 7f6896c00640 1 -- 172.21.15.62:0/1003566049 shutdown_connections 2024-04-25T09:13:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.862+0000 7f6896c00640 1 -- 172.21.15.62:0/1003566049 wait complete. 2024-04-25T09:13:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.864+0000 7f6896c00640 1 Processor -- start 2024-04-25T09:13:19.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.864+0000 7f6896c00640 1 -- start start 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6896c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68740039e0 0x7f6890160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6896c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890162c20 0x7f68901fad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6896c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689014d150 0x7f689014d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6896c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890072370 con 0x7f689014d150 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6896c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890072090 con 0x7f6890162c20 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6896c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689015cac0 con 0x7f68740039e0 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6894c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689014d150 0x7f689014d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6894c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689014d150 0x7f689014d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39500/0 (socket says 172.21.15.62:39500) 2024-04-25T09:13:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.865+0000 7f6894c00640 1 -- 172.21.15.62:0/2668438026 learned_addr learned my addr 172.21.15.62:0/2668438026 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:19.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f688fe00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68740039e0 0x7f6890160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:19.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f6887e00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890162c20 0x7f68901fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:19.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f6894c00640 1 -- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68740039e0 msgr2=0x7f6890160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:19.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f6894c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68740039e0 0x7f6890160e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f6894c00640 1 -- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890162c20 msgr2=0x7f68901fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f6894c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890162c20 0x7f68901fad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f6894c00640 1 -- 172.21.15.62:0/2668438026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68901613f0 con 0x7f689014d150 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f6894c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689014d150 0x7f689014d5d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f689006e860 tx=0x7f688800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f688cc00640 1 -- 172.21.15.62:0/2668438026 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888006d50 con 0x7f689014d150 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f688cc00640 1 -- 172.21.15.62:0/2668438026 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888003970 con 0x7f689014d150 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f688cc00640 1 -- 172.21.15.62:0/2668438026 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888003da0 con 0x7f689014d150 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.866+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68901616d0 con 0x7f689014d150 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.867+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6890161c90 con 0x7f689014d150 2024-04-25T09:13:19.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.868+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6890067c40 con 0x7f689014d150 2024-04-25T09:13:19.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.869+0000 7f688cc00640 1 -- 172.21.15.62:0/2668438026 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f688801b020 con 0x7f689014d150 2024-04-25T09:13:19.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.870+0000 7f688cc00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b4d0 0x7f686c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:19.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.870+0000 7f688cc00640 1 -- 172.21.15.62:0/2668438026 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f6888089ca0 con 0x7f689014d150 2024-04-25T09:13:19.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.871+0000 7f688fe00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b4d0 0x7f686c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:19.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.872+0000 7f688fe00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b4d0 0x7f686c06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6880033000 tx=0x7f6880008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:19.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:19.875+0000 7f688cc00640 1 -- 172.21.15.62:0/2668438026 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6888058c70 con 0x7f689014d150 2024-04-25T09:13:20.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.076+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f689015e570 con 0x7f689014d150 2024-04-25T09:13:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.629+0000 7f688cc00640 1 -- 172.21.15.62:0/2668438026 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6888058570 con 0x7f689014d150 2024-04-25T09:13:20.617 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:13:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.633+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b4d0 msgr2=0x7f686c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.633+0000 7f6896c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b4d0 0x7f686c06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6880033000 tx=0x7f6880008030 comp rx=0 tx=0).stop 2024-04-25T09:13:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.633+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689014d150 msgr2=0x7f689014d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:20.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.633+0000 7f6896c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689014d150 0x7f689014d5d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f689006e860 tx=0x7f688800dc00 comp rx=0 tx=0).stop 2024-04-25T09:13:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.635+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 shutdown_connections 2024-04-25T09:13:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.635+0000 7f6896c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b4d0 0x7f686c06d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.635+0000 7f6896c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689014d150 0x7f689014d5d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.635+0000 7f6896c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890162c20 0x7f68901fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.635+0000 7f6896c00640 1 --2- 172.21.15.62:0/2668438026 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f68740039e0 0x7f6890160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.635+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 >> 172.21.15.62:0/2668438026 conn(0x7f68900668f0 msgr2=0x7f68900b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:20.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.636+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 shutdown_connections 2024-04-25T09:13:20.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.636+0000 7f6896c00640 1 -- 172.21.15.62:0/2668438026 wait complete. 2024-04-25T09:13:20.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:13:20.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.853+0000 7f548f200640 1 Processor -- start 2024-04-25T09:13:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.853+0000 7f548f200640 1 -- start start 2024-04-25T09:13:20.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f548f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f549006d2c0 0x7f54900ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f548f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54900b31a0 0x7f54900b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f548f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900cf000 con 0x7f549006d6a0 2024-04-25T09:13:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f548f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54900cf2e0 con 0x7f549006d2c0 2024-04-25T09:13:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f548f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900b26c0 con 0x7f54900b31a0 2024-04-25T09:13:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f548de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f549006d2c0 0x7f54900ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f548de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f549006d2c0 0x7f54900ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39516/0 (socket says 172.21.15.62:39516) 2024-04-25T09:13:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f5487e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f549006d6a0 0x7f54900ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35804/0 (socket says 172.21.15.62:35804) 2024-04-25T09:13:20.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.855+0000 7f548de00640 1 -- 172.21.15.62:0/3621056106 learned_addr learned my addr 172.21.15.62:0/3621056106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.856+0000 7f548e800640 1 --2- 172.21.15.62:0/3621056106 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54900b31a0 0x7f54900b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.856+0000 7f548de00640 1 -- 172.21.15.62:0/3621056106 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54900b31a0 msgr2=0x7f54900b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.856+0000 7f548de00640 1 --2- 172.21.15.62:0/3621056106 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54900b31a0 0x7f54900b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.856+0000 7f548de00640 1 -- 172.21.15.62:0/3621056106 >> v1:172.21.15.62:6789/0 conn(0x7f549006d6a0 legacy=0x7f54900ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.856+0000 7f548de00640 1 -- 172.21.15.62:0/3621056106 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900b29a0 con 0x7f549006d2c0 2024-04-25T09:13:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.856+0000 7f548e800640 1 --2- 172.21.15.62:0/3621056106 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54900b31a0 0x7f54900b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:20.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.856+0000 7f548d400640 1 -- 172.21.15.62:0/3621056106 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349134874 0 0) 0x7f54900cf000 con 0x7f549006d6a0 2024-04-25T09:13:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.857+0000 7f548de00640 1 --2- 172.21.15.62:0/3621056106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f549006d2c0 0x7f54900ccde0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5480009870 tx=0x7f548002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a40f0e81231844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:20.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.857+0000 7f548d400640 1 -- 172.21.15.62:0/3621056106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480037070 con 0x7f549006d2c0 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.857+0000 7f548d400640 1 -- 172.21.15.62:0/3621056106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480032440 con 0x7f549006d2c0 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.857+0000 7f548d400640 1 -- 172.21.15.62:0/3621056106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480032810 con 0x7f549006d2c0 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.858+0000 7f548f200640 1 -- 172.21.15.62:0/3621056106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f549006d2c0 msgr2=0x7f54900ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.858+0000 7f548f200640 1 --2- 172.21.15.62:0/3621056106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f549006d2c0 0x7f54900ccde0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5480009870 tx=0x7f548002a250 comp rx=0 tx=0).stop 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.858+0000 7f548f200640 1 -- 172.21.15.62:0/3621056106 shutdown_connections 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.858+0000 7f548f200640 1 --2- 172.21.15.62:0/3621056106 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f54900b31a0 0x7f54900b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.858+0000 7f548f200640 1 --2- 172.21.15.62:0/3621056106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f549006d2c0 0x7f54900ccde0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.858+0000 7f548f200640 1 -- 172.21.15.62:0/3621056106 >> 172.21.15.62:0/3621056106 conn(0x7f5490066f20 msgr2=0x7f54900b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.858+0000 7f548f200640 1 -- 172.21.15.62:0/3621056106 shutdown_connections 2024-04-25T09:13:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.858+0000 7f548f200640 1 -- 172.21.15.62:0/3621056106 wait complete. 2024-04-25T09:13:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.860+0000 7f548f200640 1 Processor -- start 2024-04-25T09:13:20.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.860+0000 7f548f200640 1 -- start start 2024-04-25T09:13:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f549006d2c0 0x7f54900cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f549006d6a0 0x7f54900c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54900b31a0 0x7f54900c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901448d0 con 0x7f54900b31a0 2024-04-25T09:13:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549006db20 con 0x7f549006d2c0 2024-04-25T09:13:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5490144a50 con 0x7f549006d6a0 2024-04-25T09:13:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54900b31a0 0x7f54900c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:20.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54900b31a0 0x7f54900c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39532/0 (socket says 172.21.15.62:39532) 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548e800640 1 -- 172.21.15.62:0/2960770782 learned_addr learned my addr 172.21.15.62:0/2960770782 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.861+0000 7f548de00640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f549006d2c0 0x7f54900cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f5487e00640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f549006d6a0 0x7f54900c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f548e800640 1 -- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f549006d6a0 msgr2=0x7f54900c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f548e800640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f549006d6a0 0x7f54900c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f548e800640 1 -- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f549006d2c0 msgr2=0x7f54900cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f548e800640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f549006d2c0 0x7f54900cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f548e800640 1 -- 172.21.15.62:0/2960770782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549013c820 con 0x7f54900b31a0 2024-04-25T09:13:20.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f5487e00640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f549006d6a0 0x7f54900c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:20.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f548e800640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54900b31a0 0x7f54900c8a00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f548800cd80 tx=0x7f548800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:20.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f5486000640 1 -- 172.21.15.62:0/2960770782 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800f680 con 0x7f54900b31a0 2024-04-25T09:13:20.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f5486000640 1 -- 172.21.15.62:0/2960770782 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548800f840 con 0x7f54900b31a0 2024-04-25T09:13:20.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.862+0000 7f5486000640 1 -- 172.21.15.62:0/2960770782 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800fc30 con 0x7f54900b31a0 2024-04-25T09:13:20.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.863+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54900c5420 con 0x7f54900b31a0 2024-04-25T09:13:20.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.863+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54900c59e0 con 0x7f54900b31a0 2024-04-25T09:13:20.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.864+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549006abe0 con 0x7f54900b31a0 2024-04-25T09:13:20.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.865+0000 7f5486000640 1 -- 172.21.15.62:0/2960770782 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f548800e3e0 con 0x7f54900b31a0 2024-04-25T09:13:20.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.866+0000 7f5486000640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f546c06b4d0 0x7f546c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:20.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.866+0000 7f5486000640 1 -- 172.21.15.62:0/2960770782 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f548808acf0 con 0x7f54900b31a0 2024-04-25T09:13:20.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.866+0000 7f548de00640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f546c06b4d0 0x7f546c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:20.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.868+0000 7f548de00640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f546c06b4d0 0x7f546c06d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5480034000 tx=0x7f548002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:20.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:20.870+0000 7f5486000640 1 -- 172.21.15.62:0/2960770782 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5488014030 con 0x7f54900b31a0 2024-04-25T09:13:21.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.093+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f54900b5980 con 0x7f54900b31a0 2024-04-25T09:13:21.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.101+0000 7f5486000640 1 -- 172.21.15.62:0/2960770782 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5488059dd0 con 0x7f54900b31a0 2024-04-25T09:13:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.103+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f546c06b4d0 msgr2=0x7f546c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.103+0000 7f548f200640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f546c06b4d0 0x7f546c06d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5480034000 tx=0x7f548002e000 comp rx=0 tx=0).stop 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.103+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54900b31a0 msgr2=0x7f54900c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.103+0000 7f548f200640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54900b31a0 0x7f54900c8a00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f548800cd80 tx=0x7f548800a650 comp rx=0 tx=0).stop 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.104+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 shutdown_connections 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.104+0000 7f548f200640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f546c06b4d0 0x7f546c06d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.104+0000 7f548f200640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f54900b31a0 0x7f54900c8a00 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.104+0000 7f548f200640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f549006d6a0 0x7f54900c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.104+0000 7f548f200640 1 --2- 172.21.15.62:0/2960770782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f549006d2c0 0x7f54900cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.104+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 >> 172.21.15.62:0/2960770782 conn(0x7f5490066f20 msgr2=0x7f54900ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.104+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 shutdown_connections 2024-04-25T09:13:21.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.104+0000 7f548f200640 1 -- 172.21.15.62:0/2960770782 wait complete. 2024-04-25T09:13:21.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T09:13:21.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.322+0000 7f7422a00640 1 Processor -- start 2024-04-25T09:13:21.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.322+0000 7f7422a00640 1 -- start start 2024-04-25T09:13:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.324+0000 7f7422a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f741c15d270 0x7f741c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.324+0000 7f7422a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c0cccb0 0x7f741c06e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.324+0000 7f7422a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741c15bfa0 con 0x7f741c15bbc0 2024-04-25T09:13:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.324+0000 7f7422a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f741c0cead0 con 0x7f741c15d270 2024-04-25T09:13:21.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.324+0000 7f7422a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c06f1a0 con 0x7f741c0cccb0 2024-04-25T09:13:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.325+0000 7f741b400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f741c15d270 0x7f741c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.325+0000 7f7420a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c0cccb0 0x7f741c06e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.325+0000 7f741b400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f741c15d270 0x7f741c06e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:53846/0 (socket says 172.21.15.62:53846) 2024-04-25T09:13:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.325+0000 7f741b400640 1 -- 172.21.15.62:0/3320109572 learned_addr learned my addr 172.21.15.62:0/3320109572 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:21.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.325+0000 7f741be00640 1 --1- 172.21.15.62:0/3320109572 >> v1:172.21.15.121:6789/0 conn(0x7f741c15bbc0 0x7f741c15cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:46408/0 (socket says 172.21.15.62:46408) 2024-04-25T09:13:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.326+0000 7f741b400640 1 -- 172.21.15.62:0/3320109572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c0cccb0 msgr2=0x7f741c06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.326+0000 7f741b400640 1 --2- 172.21.15.62:0/3320109572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c0cccb0 0x7f741c06e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.326+0000 7f741b400640 1 -- 172.21.15.62:0/3320109572 >> v1:172.21.15.121:6789/0 conn(0x7f741c15bbc0 legacy=0x7f741c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.326+0000 7f741b400640 1 -- 172.21.15.62:0/3320109572 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c06f480 con 0x7f741c15d270 2024-04-25T09:13:21.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.326+0000 7f7420a00640 1 --2- 172.21.15.62:0/3320109572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c0cccb0 0x7f741c06e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:13:21.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.327+0000 7f741b400640 1 --2- 172.21.15.62:0/3320109572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741c15d270 0x7f741c06e080 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f741c0cec50 tx=0x7f740c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7394d374113b3a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.327+0000 7f741aa00640 1 -- 172.21.15.62:0/3320109572 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c036070 con 0x7f741c15d270 2024-04-25T09:13:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.328+0000 7f741aa00640 1 -- 172.21.15.62:0/3320109572 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c031600 con 0x7f741c15d270 2024-04-25T09:13:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.328+0000 7f741aa00640 1 -- 172.21.15.62:0/3320109572 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c0319d0 con 0x7f741c15d270 2024-04-25T09:13:21.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.328+0000 7f7422a00640 1 -- 172.21.15.62:0/3320109572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741c15d270 msgr2=0x7f741c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.328+0000 7f7422a00640 1 --2- 172.21.15.62:0/3320109572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741c15d270 0x7f741c06e080 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f741c0cec50 tx=0x7f740c02a710 comp rx=0 tx=0).stop 2024-04-25T09:13:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.328+0000 7f7422a00640 1 -- 172.21.15.62:0/3320109572 shutdown_connections 2024-04-25T09:13:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.328+0000 7f7422a00640 1 --2- 172.21.15.62:0/3320109572 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c0cccb0 0x7f741c06e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.328+0000 7f7422a00640 1 --2- 172.21.15.62:0/3320109572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741c15d270 0x7f741c06e080 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.328+0000 7f7422a00640 1 -- 172.21.15.62:0/3320109572 >> 172.21.15.62:0/3320109572 conn(0x7f741c0b1e40 msgr2=0x7f741c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.329+0000 7f7422a00640 1 -- 172.21.15.62:0/3320109572 shutdown_connections 2024-04-25T09:13:21.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.329+0000 7f7422a00640 1 -- 172.21.15.62:0/3320109572 wait complete. 2024-04-25T09:13:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.330+0000 7f7422a00640 1 Processor -- start 2024-04-25T09:13:21.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.331+0000 7f7422a00640 1 -- start start 2024-04-25T09:13:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f7422a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741c0cccb0 0x7f741c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f7422a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741c15bbc0 0x7f741c158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f7422a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c15d270 0x7f741c072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f7422a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0ce3a0 con 0x7f741c15bbc0 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f7422a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c06ee50 con 0x7f741c0cccb0 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f7422a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f741c0ce520 con 0x7f741c15d270 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f741b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741c15bbc0 0x7f741c158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f741b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741c15bbc0 0x7f741c158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39538/0 (socket says 172.21.15.62:39538) 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f741b400640 1 -- 172.21.15.62:0/1930829509 learned_addr learned my addr 172.21.15.62:0/1930829509 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f7420a00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c15d270 0x7f741c072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f741b400640 1 -- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c15d270 msgr2=0x7f741c072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f741b400640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c15d270 0x7f741c072920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f741b400640 1 -- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741c0cccb0 msgr2=0x7f741c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f741b400640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741c0cccb0 0x7f741c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.332+0000 7f741b400640 1 -- 172.21.15.62:0/1930829509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c0b4040 con 0x7f741c15bbc0 2024-04-25T09:13:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.333+0000 7f741b400640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741c15bbc0 0x7f741c158720 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f740c034040 tx=0x7f740c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.333+0000 7f7413e00640 1 -- 172.21.15.62:0/1930829509 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c041020 con 0x7f741c15bbc0 2024-04-25T09:13:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.333+0000 7f7413e00640 1 -- 172.21.15.62:0/1930829509 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c007820 con 0x7f741c15bbc0 2024-04-25T09:13:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.333+0000 7f7413e00640 1 -- 172.21.15.62:0/1930829509 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c036040 con 0x7f741c15bbc0 2024-04-25T09:13:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.333+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c0b4320 con 0x7f741c15bbc0 2024-04-25T09:13:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.333+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c1f4140 con 0x7f741c15bbc0 2024-04-25T09:13:21.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.334+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c1f3f70 con 0x7f741c15bbc0 2024-04-25T09:13:21.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.335+0000 7f7413e00640 1 -- 172.21.15.62:0/1930829509 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f740c044020 con 0x7f741c15bbc0 2024-04-25T09:13:21.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.336+0000 7f7413e00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73f806b4d0 0x7f73f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.337+0000 7f7413e00640 1 -- 172.21.15.62:0/1930829509 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f740c0aab80 con 0x7f741c15bbc0 2024-04-25T09:13:21.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.337+0000 7f741be00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73f806b4d0 0x7f73f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.338+0000 7f741be00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73f806b4d0 0x7f73f806d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7404000f60 tx=0x7f7404007a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.341+0000 7f7413e00640 1 -- 172.21.15.62:0/1930829509 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f740c04b030 con 0x7f741c15bbc0 2024-04-25T09:13:21.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.615+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f741c0b5660 con 0x7f741c15bbc0 2024-04-25T09:13:21.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.632+0000 7f7413e00640 1 -- 172.21.15.62:0/1930829509 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v448) v1 ==== 144+0+0 (secure 0 0 0) 0x7f741c0b5660 con 0x7f741c15bbc0 2024-04-25T09:13:21.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.635+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73f806b4d0 msgr2=0x7f73f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.635+0000 7f7422a00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73f806b4d0 0x7f73f806d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7404000f60 tx=0x7f7404007a00 comp rx=0 tx=0).stop 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.635+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741c15bbc0 msgr2=0x7f741c158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.635+0000 7f7422a00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741c15bbc0 0x7f741c158720 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f740c034040 tx=0x7f740c004a40 comp rx=0 tx=0).stop 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.636+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 shutdown_connections 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.636+0000 7f7422a00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f73f806b4d0 0x7f73f806d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.636+0000 7f7422a00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741c15bbc0 0x7f741c158720 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.636+0000 7f7422a00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f741c15d270 0x7f741c072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.636+0000 7f7422a00640 1 --2- 172.21.15.62:0/1930829509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f741c0cccb0 0x7f741c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.636+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 >> 172.21.15.62:0/1930829509 conn(0x7f741c0b1e40 msgr2=0x7f741c0b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.636+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 shutdown_connections 2024-04-25T09:13:21.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.636+0000 7f7422a00640 1 -- 172.21.15.62:0/1930829509 wait complete. 2024-04-25T09:13:21.643 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:13:21.643 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:13:21.643 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:21.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.868+0000 7f65ce600640 1 Processor -- start 2024-04-25T09:13:21.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.868+0000 7f65ce600640 1 -- start start 2024-04-25T09:13:21.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.870+0000 7f65ce600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65c806d900 0x7f65c8165370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.870+0000 7f65ce600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80b1220 con 0x7f65c813d970 2024-04-25T09:13:21.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.870+0000 7f65ce600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c815b220 con 0x7f65c815acf0 2024-04-25T09:13:21.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.870+0000 7f65ce600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815b500 con 0x7f65c806d900 2024-04-25T09:13:21.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.871+0000 7f65cd200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f65c813d970 0x7f65c80b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35808/0 (socket says 172.21.15.62:35808) 2024-04-25T09:13:21.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.871+0000 7f65cd200640 1 -- 172.21.15.62:0/871449129 learned_addr learned my addr 172.21.15.62:0/871449129 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.871+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889738850 0 0) 0x7f65c80b1220 con 0x7f65c813d970 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.871+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b0003520 con 0x7f65c813d970 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.871+0000 7f65cdc00640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65c806d900 0x7f65c8165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.871+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1193508542 0 0) 0x7f65b0003520 con 0x7f65c813d970 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65c806d900 msgr2=0x7f65c8165370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65c806d900 0x7f65c8165370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 >> v1:172.21.15.121:6789/0 conn(0x7f65c815acf0 legacy=0x7f65c806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c815b7e0 con 0x7f65c813d970 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f65bc001670 con 0x7f65c813d970 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 >> v1:172.21.15.62:6789/0 conn(0x7f65c813d970 legacy=0x7f65c80b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b00039e0 0x7f65b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65b0004800 0x7f65b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65c806d900 0x7f65b0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815b500 con 0x7f65b00039e0 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b000bb80 con 0x7f65b0004800 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b000df90 con 0x7f65c806d900 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65cd200640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b00039e0 0x7f65b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65c7e00640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65b0004800 0x7f65b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65cd200640 1 -- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65c806d900 msgr2=0x7f65b0005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65cd200640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65c806d900 0x7f65b0005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.872+0000 7f65cd200640 1 -- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65b0004800 msgr2=0x7f65b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:21.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.873+0000 7f65cd200640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65b0004800 0x7f65b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.873+0000 7f65cd200640 1 -- 172.21.15.62:0/871449129 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b0003520 con 0x7f65b00039e0 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.874+0000 7f65cd200640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b00039e0 0x7f65b0003e00 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c0e0 tx=0x7f65b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd06b212d112a5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.874+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc007c20 con 0x7f65b00039e0 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.874+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc005af0 con 0x7f65b00039e0 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.874+0000 7f65c7400640 1 -- 172.21.15.62:0/871449129 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc007070 con 0x7f65b00039e0 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.874+0000 7f65ce600640 1 -- 172.21.15.62:0/871449129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b00039e0 msgr2=0x7f65b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.874+0000 7f65ce600640 1 --2- 172.21.15.62:0/871449129 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65b00039e0 0x7f65b0003e00 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c0e0 tx=0x7f65b0005830 comp rx=0 tx=0).stop 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.874+0000 7f65cdc00640 1 -- 172.21.15.62:0/871449129 reap_dead start 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.875+0000 7f65ce600640 1 -- 172.21.15.62:0/871449129 shutdown_connections 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.875+0000 7f65ce600640 1 -- 172.21.15.62:0/871449129 >> 172.21.15.62:0/871449129 conn(0x7f65c80b14d0 msgr2=0x7f65c80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.875+0000 7f65ce600640 1 -- 172.21.15.62:0/871449129 shutdown_connections 2024-04-25T09:13:21.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.875+0000 7f65ce600640 1 -- 172.21.15.62:0/871449129 wait complete. 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.876+0000 7f65ce600640 1 Processor -- start 2024-04-25T09:13:21.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.877+0000 7f65ce600640 1 -- start start 2024-04-25T09:13:21.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65ce600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65b00039e0 0x7f65c813d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65ce600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65c8143920 0x7f65c813c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65ce600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c80a99a0 0x7f65c80a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65ce600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c813f3e0 con 0x7f65c80a99a0 2024-04-25T09:13:21.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65ce600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c813f260 con 0x7f65c8143920 2024-04-25T09:13:21.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65ce600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c813f560 con 0x7f65b00039e0 2024-04-25T09:13:21.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65cdc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c80a99a0 0x7f65c80a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65cdc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c80a99a0 0x7f65c80a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39566/0 (socket says 172.21.15.62:39566) 2024-04-25T09:13:21.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65cdc00640 1 -- 172.21.15.62:0/3399396887 learned_addr learned my addr 172.21.15.62:0/3399396887 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:21.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65c7e00640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65c8143920 0x7f65c813c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.878+0000 7f65cd200640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65b00039e0 0x7f65c813d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65cdc00640 1 -- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65b00039e0 msgr2=0x7f65c813d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65cdc00640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65b00039e0 0x7f65c813d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65cdc00640 1 -- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65c8143920 msgr2=0x7f65c813c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65cdc00640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65c8143920 0x7f65c813c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65cdc00640 1 -- 172.21.15.62:0/3399396887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80b3d00 con 0x7f65c80a99a0 2024-04-25T09:13:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65cdc00640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c80a99a0 0x7f65c80a9e10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f65c8140fc0 tx=0x7f65c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65c4c00640 1 -- 172.21.15.62:0/3399396887 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000ebc0 con 0x7f65c80a99a0 2024-04-25T09:13:21.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65c4c00640 1 -- 172.21.15.62:0/3399396887 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000ed80 con 0x7f65c80a99a0 2024-04-25T09:13:21.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c80b3fe0 con 0x7f65c80a99a0 2024-04-25T09:13:21.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65c4c00640 1 -- 172.21.15.62:0/3399396887 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0003bd0 con 0x7f65c80a99a0 2024-04-25T09:13:21.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.879+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c813cea0 con 0x7f65c80a99a0 2024-04-25T09:13:21.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.880+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c804ef20 con 0x7f65c80a99a0 2024-04-25T09:13:21.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.882+0000 7f65c4c00640 1 -- 172.21.15.62:0/3399396887 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65c001b020 con 0x7f65c80a99a0 2024-04-25T09:13:21.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.883+0000 7f65c4c00640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f65a806b4d0 0x7f65a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:21.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.883+0000 7f65c4c00640 1 -- 172.21.15.62:0/3399396887 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65c0089ca0 con 0x7f65c80a99a0 2024-04-25T09:13:21.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.883+0000 7f65cd200640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f65a806b4d0 0x7f65a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:21.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.884+0000 7f65cd200640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f65a806b4d0 0x7f65a806d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f65bc032000 tx=0x7f65bc009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:21.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:21.887+0000 7f65c4c00640 1 -- 172.21.15.62:0/3399396887 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65c0058e10 con 0x7f65c80a99a0 2024-04-25T09:13:22.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.099+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65c8070bd0 con 0x7f65c80a99a0 2024-04-25T09:13:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.101+0000 7f65c4c00640 1 -- 172.21.15.62:0/3399396887 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+13010 (secure 0 0 0) 0x7f65c0081e20 con 0x7f65c80a99a0 2024-04-25T09:13:22.089 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:22.089 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":100,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:13:20.620029+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/3997398243":"2024-04-26T09:13:17.555762+0000","172.21.15.62:6833/3997398243":"2024-04-26T09:13:17.555762+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:22.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f65a806b4d0 msgr2=0x7f65a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f65a806b4d0 0x7f65a806d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f65bc032000 tx=0x7f65bc009030 comp rx=0 tx=0).stop 2024-04-25T09:13:22.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c80a99a0 msgr2=0x7f65c80a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c80a99a0 0x7f65c80a9e10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f65c8140fc0 tx=0x7f65c000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 shutdown_connections 2024-04-25T09:13:22.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f65a806b4d0 0x7f65a806d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65c80a99a0 0x7f65c80a9e10 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65c8143920 0x7f65c813c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 --2- 172.21.15.62:0/3399396887 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f65b00039e0 0x7f65c813d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 >> 172.21.15.62:0/3399396887 conn(0x7f65c80b14d0 msgr2=0x7f65c80710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:22.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 shutdown_connections 2024-04-25T09:13:22.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.104+0000 7f65ce600640 1 -- 172.21.15.62:0/3399396887 wait complete. 2024-04-25T09:13:22.112 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:13:22.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:13:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.335+0000 7f6ae9600640 1 Processor -- start 2024-04-25T09:13:22.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.335+0000 7f6ae9600640 1 -- start start 2024-04-25T09:13:22.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.337+0000 7f6ae9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 0x7f6ae406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.337+0000 7f6ae9600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae415ee30 con 0x7f6ae406d800 2024-04-25T09:13:22.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.337+0000 7f6ae9600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae41618b0 con 0x7f6ae415e7c0 2024-04-25T09:13:22.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.337+0000 7f6ae9600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae406dc60 con 0x7f6ae40ceb50 2024-04-25T09:13:22.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.338+0000 7f6adb400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6ae406d800 0x7f6ae415e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35814/0 (socket says 172.21.15.62:35814) 2024-04-25T09:13:22.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.338+0000 7f6adb400640 1 -- 172.21.15.62:0/2632509874 learned_addr learned my addr 172.21.15.62:0/2632509874 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:22.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.338+0000 7f6ae3e00640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 0x7f6ae406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.338+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045055148 0 0) 0x7f6ae415ee30 con 0x7f6ae406d800 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.338+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac8003520 con 0x7f6ae406d800 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1819961885 0 0) 0x7f6ac8003520 con 0x7f6ae406d800 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 msgr2=0x7f6ae406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 0x7f6ae406d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 >> v1:172.21.15.121:6789/0 conn(0x7f6ae415e7c0 legacy=0x7f6ae40b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae4161b90 con 0x7f6ae406d800 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f6acc001670 con 0x7f6ae406d800 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 >> v1:172.21.15.62:6789/0 conn(0x7f6ae406d800 legacy=0x7f6ae415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ac8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ac8004800 0x7f6ac8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 0x7f6ac8005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac800bb50 con 0x7f6ac80039e0 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac800bcd0 con 0x7f6ac8004800 2024-04-25T09:13:22.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac800bed0 con 0x7f6ae40ceb50 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6ae3e00640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ac8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.339+0000 7f6adb400640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ac8004800 0x7f6ac8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.340+0000 7f6ae8c00640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 0x7f6ac8005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.340+0000 7f6ae3e00640 1 -- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 msgr2=0x7f6ac8005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.340+0000 7f6ae3e00640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 0x7f6ac8005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.340+0000 7f6ae3e00640 1 -- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ac8004800 msgr2=0x7f6ac8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.340+0000 7f6ae3e00640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ac8004800 0x7f6ac8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.340+0000 7f6ae3e00640 1 -- 172.21.15.62:0/2632509874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac8003520 con 0x7f6ac80039e0 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.340+0000 7f6ae8c00640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae40ceb50 0x7f6ac8005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.341+0000 7f6ae3e00640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ac8003e00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009ac0 tx=0x7f6ac8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873ef3d9cefd1fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:22.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.341+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4036070 con 0x7f6ac80039e0 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.341+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad4031400 con 0x7f6ac80039e0 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.341+0000 7f6ae3400640 1 -- 172.21.15.62:0/2632509874 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4009690 con 0x7f6ac80039e0 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.342+0000 7f6ae9600640 1 -- 172.21.15.62:0/2632509874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 msgr2=0x7f6ac8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.342+0000 7f6ae9600640 1 --2- 172.21.15.62:0/2632509874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ac8003e00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009ac0 tx=0x7f6ac8005830 comp rx=0 tx=0).stop 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.342+0000 7f6ae8c00640 1 -- 172.21.15.62:0/2632509874 reap_dead start 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.342+0000 7f6ae9600640 1 -- 172.21.15.62:0/2632509874 shutdown_connections 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.342+0000 7f6ae9600640 1 -- 172.21.15.62:0/2632509874 >> 172.21.15.62:0/2632509874 conn(0x7f6ae4066f20 msgr2=0x7f6ae40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.342+0000 7f6ae9600640 1 -- 172.21.15.62:0/2632509874 shutdown_connections 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.342+0000 7f6ae9600640 1 -- 172.21.15.62:0/2632509874 wait complete. 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.344+0000 7f6ae9600640 1 Processor -- start 2024-04-25T09:13:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.344+0000 7f6ae9600640 1 -- start start 2024-04-25T09:13:22.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae9600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ae4157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae4136110 0x7f6ae4143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae9600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ae4145380 0x7f6ae4145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae9600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4162510 con 0x7f6ac80039e0 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae9600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4162390 con 0x7f6ae4145380 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae9600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae406d2c0 con 0x7f6ae4136110 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ae4157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ae4157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39590/0 (socket says 172.21.15.62:39590) 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.345+0000 7f6ae3e00640 1 -- 172.21.15.62:0/410806088 learned_addr learned my addr 172.21.15.62:0/410806088 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae8c00640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ae4145380 0x7f6ae4145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6adb400640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae4136110 0x7f6ae4143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae3e00640 1 -- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae4136110 msgr2=0x7f6ae4143ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae3e00640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae4136110 0x7f6ae4143ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae3e00640 1 -- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ae4145380 msgr2=0x7f6ae4145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae3e00640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ae4145380 0x7f6ae4145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae3e00640 1 -- 172.21.15.62:0/410806088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae4144640 con 0x7f6ac80039e0 2024-04-25T09:13:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae3e00640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ae4157090 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6ad402abb0 tx=0x7f6ad4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae0c00640 1 -- 172.21.15.62:0/410806088 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4040020 con 0x7f6ac80039e0 2024-04-25T09:13:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae0c00640 1 -- 172.21.15.62:0/410806088 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad4003bc0 con 0x7f6ac80039e0 2024-04-25T09:13:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae0c00640 1 -- 172.21.15.62:0/410806088 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4036040 con 0x7f6ac80039e0 2024-04-25T09:13:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.346+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae4144920 con 0x7f6ac80039e0 2024-04-25T09:13:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.347+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae4144ee0 con 0x7f6ac80039e0 2024-04-25T09:13:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.348+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae406abe0 con 0x7f6ac80039e0 2024-04-25T09:13:22.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.349+0000 7f6ae0c00640 1 -- 172.21.15.62:0/410806088 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ad4043020 con 0x7f6ac80039e0 2024-04-25T09:13:22.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.350+0000 7f6ae0c00640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ac006b4d0 0x7f6ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.350+0000 7f6ae0c00640 1 -- 172.21.15.62:0/410806088 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6ad40aa350 con 0x7f6ac80039e0 2024-04-25T09:13:22.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.350+0000 7f6adb400640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ac006b4d0 0x7f6ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.351+0000 7f6adb400640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ac006b4d0 0x7f6ac006d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6acc001050 tx=0x7f6acc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:22.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.354+0000 7f6ae0c00640 1 -- 172.21.15.62:0/410806088 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ad404a030 con 0x7f6ac80039e0 2024-04-25T09:13:22.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.558+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ae41364f0 con 0x7f6ac80039e0 2024-04-25T09:13:22.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.558+0000 7f6ae0c00640 1 -- 172.21.15.62:0/410806088 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ad40044e0 con 0x7f6ac80039e0 2024-04-25T09:13:22.546 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ac006b4d0 msgr2=0x7f6ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ac006b4d0 0x7f6ac006d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6acc001050 tx=0x7f6acc009040 comp rx=0 tx=0).stop 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 msgr2=0x7f6ae4157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ae4157090 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f6ad402abb0 tx=0x7f6ad4004730 comp rx=0 tx=0).stop 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 shutdown_connections 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6ac006b4d0 0x7f6ac006d990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ae4145380 0x7f6ae4145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6ae4136110 0x7f6ae4143ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 --2- 172.21.15.62:0/410806088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ac80039e0 0x7f6ae4157090 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 >> 172.21.15.62:0/410806088 conn(0x7f6ae4066f20 msgr2=0x7f6ae415f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:22.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.561+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 shutdown_connections 2024-04-25T09:13:22.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.562+0000 7f6ae9600640 1 -- 172.21.15.62:0/410806088 wait complete. 2024-04-25T09:13:22.569 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T09:13:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.785+0000 7fafc6000640 1 Processor -- start 2024-04-25T09:13:22.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.786+0000 7fafc6000640 1 -- start start 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.787+0000 7fafc6000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fafc006d900 0x7fafc015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.787+0000 7fafc6000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fafc00cee40 0x7fafc00b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.787+0000 7fafc6000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc015faa0 con 0x7fafc0144c10 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.787+0000 7fafc6000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc0144ff0 con 0x7fafc006d900 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.787+0000 7fafc6000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc0160340 con 0x7fafc00cee40 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafc4c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fafc0144c10 0x7fafc006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35830/0 (socket says 172.21.15.62:35830) 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafc4c00640 1 -- 172.21.15.62:0/3223838139 learned_addr learned my addr 172.21.15.62:0/3223838139 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafbfe00640 1 --2- 172.21.15.62:0/3223838139 >> v2:172.21.15.62:3300/0 conn(0x7fafc006d900 0x7fafc015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafc5600640 1 --2- 172.21.15.62:0/3223838139 >> v2:172.21.15.121:3300/0 conn(0x7fafc00cee40 0x7fafc00b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafbfe00640 1 -- 172.21.15.62:0/3223838139 >> v2:172.21.15.121:3300/0 conn(0x7fafc00cee40 msgr2=0x7fafc00b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:22.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafbfe00640 1 --2- 172.21.15.62:0/3223838139 >> v2:172.21.15.121:3300/0 conn(0x7fafc00cee40 0x7fafc00b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafbfe00640 1 -- 172.21.15.62:0/3223838139 >> v1:172.21.15.62:6789/0 conn(0x7fafc0144c10 legacy=0x7fafc006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafbf400640 1 -- 172.21.15.62:0/3223838139 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412789589 0 0) 0x7fafc015faa0 con 0x7fafc0144c10 2024-04-25T09:13:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafbfe00640 1 -- 172.21.15.62:0/3223838139 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc015fd80 con 0x7fafc006d900 2024-04-25T09:13:22.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.788+0000 7fafc5600640 1 --2- 172.21.15.62:0/3223838139 >> v2:172.21.15.121:3300/0 conn(0x7fafc00cee40 0x7fafc00b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.789+0000 7fafbfe00640 1 --2- 172.21.15.62:0/3223838139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc015f450 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fafc015f390 tx=0x7fafac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b3c17e61fa1264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.790+0000 7fafbf400640 1 -- 172.21.15.62:0/3223838139 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac037070 con 0x7fafc006d900 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.790+0000 7fafbf400640 1 -- 172.21.15.62:0/3223838139 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafac032440 con 0x7fafc006d900 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.790+0000 7fafbf400640 1 -- 172.21.15.62:0/3223838139 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac02ae60 con 0x7fafc006d900 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.790+0000 7fafc6000640 1 -- 172.21.15.62:0/3223838139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 msgr2=0x7fafc015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.790+0000 7fafc6000640 1 --2- 172.21.15.62:0/3223838139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc015f450 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fafc015f390 tx=0x7fafac02a110 comp rx=0 tx=0).stop 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.790+0000 7fafc6000640 1 -- 172.21.15.62:0/3223838139 shutdown_connections 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.790+0000 7fafc6000640 1 --2- 172.21.15.62:0/3223838139 >> v2:172.21.15.121:3300/0 conn(0x7fafc00cee40 0x7fafc00b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.790+0000 7fafc6000640 1 --2- 172.21.15.62:0/3223838139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc015f450 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.791+0000 7fafc6000640 1 -- 172.21.15.62:0/3223838139 >> 172.21.15.62:0/3223838139 conn(0x7fafc00b14d0 msgr2=0x7fafc00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:22.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.791+0000 7fafc6000640 1 -- 172.21.15.62:0/3223838139 shutdown_connections 2024-04-25T09:13:22.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.791+0000 7fafc6000640 1 -- 172.21.15.62:0/3223838139 wait complete. 2024-04-25T09:13:22.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.792+0000 7fafc6000640 1 Processor -- start 2024-04-25T09:13:22.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.793+0000 7fafc6000640 1 -- start start 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.793+0000 7fafc6000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc013fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.793+0000 7fafc6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00cee40 0x7fafc0140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.793+0000 7fafc6000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fafc0144c10 0x7fafc0140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc6000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00cf270 con 0x7fafc006d900 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc6000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00b41c0 con 0x7fafc00cee40 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc6000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00b4b90 con 0x7fafc0144c10 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc013fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc013fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39606/0 (socket says 172.21.15.62:39606) 2024-04-25T09:13:22.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc4c00640 1 -- 172.21.15.62:0/3553533534 learned_addr learned my addr 172.21.15.62:0/3553533534 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafbfe00640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00cee40 0x7fafc0140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc5600640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fafc0144c10 0x7fafc0140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc4c00640 1 -- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fafc0144c10 msgr2=0x7fafc0140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc4c00640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fafc0144c10 0x7fafc0140b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc4c00640 1 -- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00cee40 msgr2=0x7fafc0140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc4c00640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00cee40 0x7fafc0140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.794+0000 7fafc4c00640 1 -- 172.21.15.62:0/3553533534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc00cd6f0 con 0x7fafc006d900 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.795+0000 7fafc4c00640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc013fcf0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fafb400c430 tx=0x7fafb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.795+0000 7fafbcc00640 1 -- 172.21.15.62:0/3553533534 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb40074e0 con 0x7fafc006d900 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.795+0000 7fafbcc00640 1 -- 172.21.15.62:0/3553533534 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb40047e0 con 0x7fafc006d900 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.795+0000 7fafbcc00640 1 -- 172.21.15.62:0/3553533534 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004bd0 con 0x7fafc006d900 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.795+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc00cd9d0 con 0x7fafc006d900 2024-04-25T09:13:22.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.795+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc00cdf90 con 0x7fafc006d900 2024-04-25T09:13:22.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.796+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc004ef20 con 0x7fafc006d900 2024-04-25T09:13:22.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.798+0000 7fafbcc00640 1 -- 172.21.15.62:0/3553533534 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafb4011070 con 0x7fafc006d900 2024-04-25T09:13:22.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.799+0000 7fafbcc00640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafa006b4d0 0x7fafa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:22.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.799+0000 7fafbcc00640 1 -- 172.21.15.62:0/3553533534 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fafb4089b90 con 0x7fafc006d900 2024-04-25T09:13:22.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.800+0000 7fafbfe00640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafa006b4d0 0x7fafa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:22.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.801+0000 7fafbfe00640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafa006b4d0 0x7fafa006d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fafac034000 tx=0x7fafac02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:22.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:22.804+0000 7fafbcc00640 1 -- 172.21.15.62:0/3553533534 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fafb4058d00 con 0x7fafc006d900 2024-04-25T09:13:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.076+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fafc00ce1d0 con 0x7fafc006d900 2024-04-25T09:13:23.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.645+0000 7fafbcc00640 1 -- 172.21.15.62:0/3553533534 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v450) v1 ==== 158+0+0 (secure 0 0 0) 0x7fafb4058600 con 0x7fafc006d900 2024-04-25T09:13:23.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafa006b4d0 msgr2=0x7fafa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafa006b4d0 0x7fafa006d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fafac034000 tx=0x7fafac02e000 comp rx=0 tx=0).stop 2024-04-25T09:13:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 msgr2=0x7fafc013fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc013fcf0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fafb400c430 tx=0x7fafb400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 shutdown_connections 2024-04-25T09:13:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafa006b4d0 0x7fafa006d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fafc0144c10 0x7fafc0140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafc00cee40 0x7fafc0140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 --2- 172.21.15.62:0/3553533534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fafc006d900 0x7fafc013fcf0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.648+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 >> 172.21.15.62:0/3553533534 conn(0x7fafc00b14d0 msgr2=0x7fafc015c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.649+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 shutdown_connections 2024-04-25T09:13:23.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.649+0000 7fafc6000640 1 -- 172.21.15.62:0/3553533534 wait complete. 2024-04-25T09:13:23.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:13:23.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.882+0000 7f7cf3200640 1 Processor -- start 2024-04-25T09:13:23.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.882+0000 7f7cf3200640 1 -- start start 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.883+0000 7f7cf3200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cf40ceb50 0x7f7cf406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.883+0000 7f7cf3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf406d800 0x7f7cf415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.883+0000 7f7cf3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf40b5560 con 0x7f7cf415e5f0 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf3200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf406dc60 con 0x7f7cf40ceb50 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf415ee30 con 0x7f7cf406d800 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cf40ceb50 0x7f7cf406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf2800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7cf415e5f0 0x7f7cf40ad8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35840/0 (socket says 172.21.15.62:35840) 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf2800640 1 -- 172.21.15.62:0/678068470 learned_addr learned my addr 172.21.15.62:0/678068470 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cebe00640 1 --2- 172.21.15.62:0/678068470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf406d800 0x7f7cf415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf1e00640 1 -- 172.21.15.62:0/678068470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf406d800 msgr2=0x7f7cf415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:23.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf1e00640 1 --2- 172.21.15.62:0/678068470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf406d800 0x7f7cf415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf1e00640 1 -- 172.21.15.62:0/678068470 >> v1:172.21.15.62:6789/0 conn(0x7f7cf415e5f0 legacy=0x7f7cf40ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:23.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf1e00640 1 -- 172.21.15.62:0/678068470 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf41618b0 con 0x7f7cf40ceb50 2024-04-25T09:13:23.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.884+0000 7f7cf1400640 1 -- 172.21.15.62:0/678068470 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434863563 0 0) 0x7f7cf40b5560 con 0x7f7cf415e5f0 2024-04-25T09:13:23.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.886+0000 7f7cf1e00640 1 --2- 172.21.15.62:0/678068470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf40ceb50 0x7f7cf406d2c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246c587178944df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:23.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.886+0000 7f7cf1400640 1 -- 172.21.15.62:0/678068470 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4037070 con 0x7f7cf40ceb50 2024-04-25T09:13:23.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.886+0000 7f7cf1400640 1 -- 172.21.15.62:0/678068470 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce402ae40 con 0x7f7cf40ceb50 2024-04-25T09:13:23.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.886+0000 7f7cf1400640 1 -- 172.21.15.62:0/678068470 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4032660 con 0x7f7cf40ceb50 2024-04-25T09:13:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.887+0000 7f7cf3200640 1 -- 172.21.15.62:0/678068470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf40ceb50 msgr2=0x7f7cf406d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.887+0000 7f7cf3200640 1 --2- 172.21.15.62:0/678068470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf40ceb50 0x7f7cf406d2c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce402a0f0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.887+0000 7f7cf3200640 1 -- 172.21.15.62:0/678068470 shutdown_connections 2024-04-25T09:13:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.887+0000 7f7cf3200640 1 --2- 172.21.15.62:0/678068470 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf406d800 0x7f7cf415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.887+0000 7f7cf3200640 1 --2- 172.21.15.62:0/678068470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf40ceb50 0x7f7cf406d2c0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.887+0000 7f7cf3200640 1 -- 172.21.15.62:0/678068470 >> 172.21.15.62:0/678068470 conn(0x7f7cf4066f20 msgr2=0x7f7cf40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.887+0000 7f7cf3200640 1 -- 172.21.15.62:0/678068470 shutdown_connections 2024-04-25T09:13:23.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.887+0000 7f7cf3200640 1 -- 172.21.15.62:0/678068470 wait complete. 2024-04-25T09:13:23.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.889+0000 7f7cf3200640 1 Processor -- start 2024-04-25T09:13:23.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.889+0000 7f7cf3200640 1 -- start start 2024-04-25T09:13:23.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf406d800 0x7f7cf4158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf40ceb50 0x7f7cf4159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cf415e5f0 0x7f7cf41f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf415ebe0 con 0x7f7cf406d800 2024-04-25T09:13:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf415ea60 con 0x7f7cf415e5f0 2024-04-25T09:13:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf40cef30 con 0x7f7cf40ceb50 2024-04-25T09:13:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf406d800 0x7f7cf4158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf406d800 0x7f7cf4158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39622/0 (socket says 172.21.15.62:39622) 2024-04-25T09:13:23.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.890+0000 7f7cf1e00640 1 -- 172.21.15.62:0/1314737294 learned_addr learned my addr 172.21.15.62:0/1314737294 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cf2800640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cf415e5f0 0x7f7cf41f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cf1e00640 1 -- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf40ceb50 msgr2=0x7f7cf4159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cf1e00640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf40ceb50 0x7f7cf4159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cf1e00640 1 -- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cf415e5f0 msgr2=0x7f7cf41f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cf1e00640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cf415e5f0 0x7f7cf41f0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cf1e00640 1 -- 172.21.15.62:0/1314737294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4159230 con 0x7f7cf406d800 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cf1e00640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf406d800 0x7f7cf4158800 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cea000640 1 -- 172.21.15.62:0/1314737294 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4037040 con 0x7f7cf406d800 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cea000640 1 -- 172.21.15.62:0/1314737294 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce4004030 con 0x7f7cf406d800 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cea000640 1 -- 172.21.15.62:0/1314737294 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4004420 con 0x7f7cf406d800 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.891+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf4159510 con 0x7f7cf406d800 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.892+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf4159ad0 con 0x7f7cf406d800 2024-04-25T09:13:23.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.893+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf406abe0 con 0x7f7cf406d800 2024-04-25T09:13:23.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.894+0000 7f7cea000640 1 -- 172.21.15.62:0/1314737294 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ce4003610 con 0x7f7cf406d800 2024-04-25T09:13:23.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.895+0000 7f7cea000640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd006b4d0 0x7f7cd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:23.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.895+0000 7f7cea000640 1 -- 172.21.15.62:0/1314737294 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7ce40aacd0 con 0x7f7cf406d800 2024-04-25T09:13:23.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.896+0000 7f7cebe00640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd006b4d0 0x7f7cd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:23.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.897+0000 7f7cebe00640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd006b4d0 0x7f7cd006d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7cf415a900 tx=0x7f7cdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:23.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:23.899+0000 7f7cea000640 1 -- 172.21.15.62:0/1314737294 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ce4079e50 con 0x7f7cf406d800 2024-04-25T09:13:24.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.105+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7cf4071770 con 0x7f7cf406d800 2024-04-25T09:13:24.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.111+0000 7f7cea000640 1 -- 172.21.15.62:0/1314737294 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ce4079750 con 0x7f7cf406d800 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd006b4d0 msgr2=0x7f7cd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd006b4d0 0x7f7cd006d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7cf415a900 tx=0x7f7cdc008040 comp rx=0 tx=0).stop 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf406d800 msgr2=0x7f7cf4158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf406d800 0x7f7cf4158800 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40097a0 tx=0x7f7ce40046c0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 shutdown_connections 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cd006b4d0 0x7f7cd006d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cf415e5f0 0x7f7cf41f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cf40ceb50 0x7f7cf4159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 --2- 172.21.15.62:0/1314737294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cf406d800 0x7f7cf4158800 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 >> 172.21.15.62:0/1314737294 conn(0x7f7cf4066f20 msgr2=0x7f7cf415f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:24.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 shutdown_connections 2024-04-25T09:13:24.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.114+0000 7f7cf3200640 1 -- 172.21.15.62:0/1314737294 wait complete. 2024-04-25T09:13:24.121 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:13:24.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:13:24.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:13:24.331 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:13:24.332 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:13:24.332 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:24.393 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:13:24.394 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:13:24.394 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:24.452 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:13:24.452 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:13:24.452 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:24.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:13:24.514 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:13:24.514 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:24.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:13:24.572 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:13:24.573 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:24.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.806+0000 7f9c63e00640 1 Processor -- start 2024-04-25T09:13:24.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.807+0000 7f9c63e00640 1 -- start start 2024-04-25T09:13:24.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.808+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6415b900 0x7f9c640d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:24.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.808+0000 7f9c63e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c640d1cc0 0x7f9c640b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:24.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.808+0000 7f9c63e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c640cf890 con 0x7f9c640b6730 2024-04-25T09:13:24.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.808+0000 7f9c63e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c640d2120 con 0x7f9c640d1cc0 2024-04-25T09:13:24.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.808+0000 7f9c63e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64070a00 con 0x7f9c6415b900 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.808+0000 7f9c62000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c640d1cc0 0x7f9c640b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.808+0000 7f9c62000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9c640d1cc0 0x7f9c640b61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47454/0 (socket says 172.21.15.62:47454) 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.808+0000 7f9c62000640 1 -- 172.21.15.62:0/1345675041 learned_addr learned my addr 172.21.15.62:0/1345675041 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.809+0000 7f9c62a00640 1 --2- 172.21.15.62:0/1345675041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6415b900 0x7f9c640d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.809+0000 7f9c62000640 1 -- 172.21.15.62:0/1345675041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6415b900 msgr2=0x7f9c640d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.809+0000 7f9c62000640 1 --2- 172.21.15.62:0/1345675041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6415b900 0x7f9c640d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.809+0000 7f9c62000640 1 -- 172.21.15.62:0/1345675041 >> v1:172.21.15.121:6789/0 conn(0x7f9c640b6730 legacy=0x7f9c6415f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.809+0000 7f9c62000640 1 -- 172.21.15.62:0/1345675041 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6406fdf0 con 0x7f9c640d1cc0 2024-04-25T09:13:24.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.809+0000 7f9c62a00640 1 --2- 172.21.15.62:0/1345675041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6415b900 0x7f9c640d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.810+0000 7f9c62000640 1 --2- 172.21.15.62:0/1345675041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b61f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009870 tx=0x7f9c4c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1720c5fe7501b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.810+0000 7f9c61600640 1 -- 172.21.15.62:0/1345675041 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c037070 con 0x7f9c640d1cc0 2024-04-25T09:13:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c61600640 1 -- 172.21.15.62:0/1345675041 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c032440 con 0x7f9c640d1cc0 2024-04-25T09:13:24.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c61600640 1 -- 172.21.15.62:0/1345675041 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c032810 con 0x7f9c640d1cc0 2024-04-25T09:13:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c63e00640 1 -- 172.21.15.62:0/1345675041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 msgr2=0x7f9c640b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c63e00640 1 --2- 172.21.15.62:0/1345675041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b61f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009870 tx=0x7f9c4c02a250 comp rx=0 tx=0).stop 2024-04-25T09:13:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c63e00640 1 -- 172.21.15.62:0/1345675041 shutdown_connections 2024-04-25T09:13:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c63e00640 1 --2- 172.21.15.62:0/1345675041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b61f0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c63e00640 1 --2- 172.21.15.62:0/1345675041 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c6415b900 0x7f9c640d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c63e00640 1 -- 172.21.15.62:0/1345675041 >> 172.21.15.62:0/1345675041 conn(0x7f9c64072340 msgr2=0x7f9c64072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:24.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c63e00640 1 -- 172.21.15.62:0/1345675041 shutdown_connections 2024-04-25T09:13:24.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.811+0000 7f9c63e00640 1 -- 172.21.15.62:0/1345675041 wait complete. 2024-04-25T09:13:24.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.813+0000 7f9c63e00640 1 Processor -- start 2024-04-25T09:13:24.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.813+0000 7f9c63e00640 1 -- start start 2024-04-25T09:13:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c640b6730 0x7f9c640b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:24.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6415b900 0x7f9c640b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c63e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640703b0 con 0x7f9c640d1cc0 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c63e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b6ba0 con 0x7f9c6415b900 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c63e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64070690 con 0x7f9c640b6730 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c62000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c62000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47462/0 (socket says 172.21.15.62:47462) 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.814+0000 7f9c62000640 1 -- 172.21.15.62:0/3694541992 learned_addr learned my addr 172.21.15.62:0/3694541992 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c63400640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6415b900 0x7f9c640b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c62a00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c640b6730 0x7f9c640b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c62000640 1 -- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c640b6730 msgr2=0x7f9c640b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c62000640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c640b6730 0x7f9c640b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c62000640 1 -- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6415b900 msgr2=0x7f9c640b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c62000640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6415b900 0x7f9c640b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c62000640 1 -- 172.21.15.62:0/3694541992 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c640b4990 con 0x7f9c640d1cc0 2024-04-25T09:13:24.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c63400640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6415b900 0x7f9c640b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:24.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.815+0000 7f9c62000640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b5a30 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000c00 tx=0x7f9c4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:24.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.816+0000 7f9c5aa00640 1 -- 172.21.15.62:0/3694541992 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c041020 con 0x7f9c640d1cc0 2024-04-25T09:13:24.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.816+0000 7f9c5aa00640 1 -- 172.21.15.62:0/3694541992 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0306c0 con 0x7f9c640d1cc0 2024-04-25T09:13:24.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.816+0000 7f9c5aa00640 1 -- 172.21.15.62:0/3694541992 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c037040 con 0x7f9c640d1cc0 2024-04-25T09:13:24.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.816+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c640b4cd0 con 0x7f9c640d1cc0 2024-04-25T09:13:24.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.816+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c640b5510 con 0x7f9c640d1cc0 2024-04-25T09:13:24.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.817+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6404ef20 con 0x7f9c640d1cc0 2024-04-25T09:13:24.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.818+0000 7f9c5aa00640 1 -- 172.21.15.62:0/3694541992 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c4c044020 con 0x7f9c640d1cc0 2024-04-25T09:13:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.819+0000 7f9c5aa00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c4006b4d0 0x7f9c4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.819+0000 7f9c5aa00640 1 -- 172.21.15.62:0/3694541992 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9c4c0aa7c0 con 0x7f9c640d1cc0 2024-04-25T09:13:24.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.819+0000 7f9c62a00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c4006b4d0 0x7f9c4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:24.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.821+0000 7f9c62a00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c4006b4d0 0x7f9c4006d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9c540096d0 tx=0x7f9c54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:24.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:24.823+0000 7f9c5aa00640 1 -- 172.21.15.62:0/3694541992 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c4c04b030 con 0x7f9c640d1cc0 2024-04-25T09:13:25.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.060+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c64157190 con 0x7f9c640d1cc0 2024-04-25T09:13:25.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.061+0000 7f9c5aa00640 1 -- 172.21.15.62:0/3694541992 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f9c640b4cd0 con 0x7f9c640d1cc0 2024-04-25T09:13:25.049 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:25.049 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6001,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/1798148826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1798148826},{"type":"v1","addr":"172.21.15.121:6835","nonce":1798148826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2045746846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2045746846},{"type":"v1","addr":"172.21.15.121:6837","nonce":2045746846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1182284820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1182284820},{"type":"v1","addr":"172.21.15.62:6834","nonce":1182284820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/644335367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":644335367},{"type":"v1","addr":"172.21.15.62:6835","nonce":644335367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[]} 2024-04-25T09:13:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 66 2024-04-25T09:13:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c4006b4d0 msgr2=0x7f9c4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:25.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c4006b4d0 0x7f9c4006d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9c540096d0 tx=0x7f9c54008040 comp rx=0 tx=0).stop 2024-04-25T09:13:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 msgr2=0x7f9c640b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b5a30 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000c00 tx=0x7f9c4c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 shutdown_connections 2024-04-25T09:13:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c4006b4d0 0x7f9c4006d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c640d1cc0 0x7f9c640b5a30 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c6415b900 0x7f9c640b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 --2- 172.21.15.62:0/3694541992 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c640b6730 0x7f9c640b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 >> 172.21.15.62:0/3694541992 conn(0x7f9c64072340 msgr2=0x7f9c640cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:25.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 shutdown_connections 2024-04-25T09:13:25.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.064+0000 7f9c63e00640 1 -- 172.21.15.62:0/3694541992 wait complete. 2024-04-25T09:13:25.071 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:13:25.071 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:13:25.071 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:13:25.072 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:25.072 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:13:25.072 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:13:25.072 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:13:25.072 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:25.072 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:13:25.072 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:13:25.073 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:13:25.073 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:25.073 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:13:25.073 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:13:25.073 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:13:25.073 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:13:25.088 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:13:25.089 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:13:25.094 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:13:25.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:13:25.114 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:13:25.114 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:13:25.115 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:13:25.115 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:13:25.141 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:13:25.141 DEBUG:teuthology.parallel:result is None 2024-04-25T09:13:25.147 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:13:25.148 DEBUG:teuthology.parallel:result is None 2024-04-25T09:13:25.162 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:13:25.162 DEBUG:teuthology.parallel:result is None 2024-04-25T09:13:25.168 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:13:25.169 DEBUG:teuthology.parallel:result is None 2024-04-25T09:13:25.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:13:25.295 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:13:25.296 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:13:25.316 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:13:25.316 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:13:25.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.363+0000 7f073fe00640 1 Processor -- start 2024-04-25T09:13:25.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.363+0000 7f073fe00640 1 -- start start 2024-04-25T09:13:25.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0740161230 0x7f07400b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:25.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f07400d0fb0 0x7f07400b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:25.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07400b5a60 0x7f07400d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:25.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07400b5ec0 con 0x7f07400b5a60 2024-04-25T09:13:25.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073fe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07400d2040 con 0x7f07400d0fb0 2024-04-25T09:13:25.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07400cea70 con 0x7f0740161230 2024-04-25T09:13:25.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07400b5a60 0x7f07400d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:25.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07400b5a60 0x7f07400d2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47516/0 (socket says 172.21.15.62:47516) 2024-04-25T09:13:25.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.364+0000 7f073f400640 1 -- 172.21.15.62:0/3953892846 learned_addr learned my addr 172.21.15.62:0/3953892846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:25.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.365+0000 7f073ea00640 1 --2- 172.21.15.62:0/3953892846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0740161230 0x7f07400b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:25.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.365+0000 7f0737e00640 1 --2- 172.21.15.62:0/3953892846 >> v2:172.21.15.121:3300/0 conn(0x7f07400d0fb0 0x7f07400b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:25.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.365+0000 7f073f400640 1 -- 172.21.15.62:0/3953892846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0740161230 msgr2=0x7f07400b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:25.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.365+0000 7f073f400640 1 --2- 172.21.15.62:0/3953892846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0740161230 0x7f07400b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.365+0000 7f073f400640 1 -- 172.21.15.62:0/3953892846 >> v2:172.21.15.121:3300/0 conn(0x7f07400d0fb0 msgr2=0x7f07400b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:25.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.365+0000 7f073f400640 1 --2- 172.21.15.62:0/3953892846 >> v2:172.21.15.121:3300/0 conn(0x7f07400d0fb0 0x7f07400b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.365+0000 7f073f400640 1 -- 172.21.15.62:0/3953892846 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400ced50 con 0x7f07400b5a60 2024-04-25T09:13:25.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.366+0000 7f073f400640 1 --2- 172.21.15.62:0/3953892846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400b5a60 0x7f07400d2a90 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f07400747d0 tx=0x7f073802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84f431d7644dcfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:25.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.366+0000 7f073e000640 1 -- 172.21.15.62:0/3953892846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738036070 con 0x7f07400b5a60 2024-04-25T09:13:25.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.366+0000 7f073e000640 1 -- 172.21.15.62:0/3953892846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0738031600 con 0x7f07400b5a60 2024-04-25T09:13:25.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.366+0000 7f073e000640 1 -- 172.21.15.62:0/3953892846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07380319d0 con 0x7f07400b5a60 2024-04-25T09:13:25.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 -- 172.21.15.62:0/3953892846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400b5a60 msgr2=0x7f07400d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:25.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 --2- 172.21.15.62:0/3953892846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400b5a60 0x7f07400d2a90 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f07400747d0 tx=0x7f073802a690 comp rx=0 tx=0).stop 2024-04-25T09:13:25.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 -- 172.21.15.62:0/3953892846 shutdown_connections 2024-04-25T09:13:25.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 --2- 172.21.15.62:0/3953892846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400b5a60 0x7f07400d2a90 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 --2- 172.21.15.62:0/3953892846 >> v2:172.21.15.121:3300/0 conn(0x7f07400d0fb0 0x7f07400b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 --2- 172.21.15.62:0/3953892846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0740161230 0x7f07400b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 -- 172.21.15.62:0/3953892846 >> 172.21.15.62:0/3953892846 conn(0x7f0740066d10 msgr2=0x7f0740067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:25.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 -- 172.21.15.62:0/3953892846 shutdown_connections 2024-04-25T09:13:25.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.367+0000 7f073fe00640 1 -- 172.21.15.62:0/3953892846 wait complete. 2024-04-25T09:13:25.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.369+0000 7f073fe00640 1 Processor -- start 2024-04-25T09:13:25.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.369+0000 7f073fe00640 1 -- start start 2024-04-25T09:13:25.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f073fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07400b5a60 0x7f07400c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:25.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f073fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400d0fb0 0x7f074015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:25.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f073fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0740161230 0x7f07401f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:25.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f073fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400ca8d0 con 0x7f07400d0fb0 2024-04-25T09:13:25.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f073fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400d2290 con 0x7f0740161230 2024-04-25T09:13:25.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f073fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0740161610 con 0x7f07400b5a60 2024-04-25T09:13:25.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f0737e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400d0fb0 0x7f074015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:25.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f0737e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400d0fb0 0x7f074015ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47532/0 (socket says 172.21.15.62:47532) 2024-04-25T09:13:25.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f0737e00640 1 -- 172.21.15.62:0/1057393176 learned_addr learned my addr 172.21.15.62:0/1057393176 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:25.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f073f400640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0740161230 0x7f07401f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:25.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.370+0000 7f073ea00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07400b5a60 0x7f07400c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:25.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737e00640 1 -- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07400b5a60 msgr2=0x7f07400c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737e00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07400b5a60 0x7f07400c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737e00640 1 -- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0740161230 msgr2=0x7f07401f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737e00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0740161230 0x7f07401f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737e00640 1 -- 172.21.15.62:0/1057393176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074015a720 con 0x7f07400d0fb0 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737e00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400d0fb0 0x7f074015ce00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f072800d800 tx=0x7f072800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737400640 1 -- 172.21.15.62:0/1057393176 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728018020 con 0x7f07400d0fb0 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737400640 1 -- 172.21.15.62:0/1057393176 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07280096a0 con 0x7f07400d0fb0 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f0737400640 1 -- 172.21.15.62:0/1057393176 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728009a70 con 0x7f07400d0fb0 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.371+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07400c8480 con 0x7f07400d0fb0 2024-04-25T09:13:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.372+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07400c8a40 con 0x7f07400d0fb0 2024-04-25T09:13:25.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.373+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074004ef20 con 0x7f07400d0fb0 2024-04-25T09:13:25.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.374+0000 7f0737400640 1 -- 172.21.15.62:0/1057393176 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f072801b020 con 0x7f07400d0fb0 2024-04-25T09:13:25.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.375+0000 7f0737400640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071c06b4d0 0x7f071c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:25.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.375+0000 7f0737400640 1 -- 172.21.15.62:0/1057393176 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0728089fd0 con 0x7f07400d0fb0 2024-04-25T09:13:25.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.375+0000 7f073ea00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071c06b4d0 0x7f071c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:25.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.376+0000 7f073ea00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071c06b4d0 0x7f071c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0730006fd0 tx=0x7f0730008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:25.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.379+0000 7f0737400640 1 -- 172.21.15.62:0/1057393176 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0728022030 con 0x7f07400d0fb0 2024-04-25T09:13:25.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:25.591+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f07400ce640 con 0x7f07400d0fb0 2024-04-25T09:13:26.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.683+0000 7f0737400640 1 -- 172.21.15.62:0/1057393176 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) v1 ==== 97+0+0 (secure 0 0 0) 0x7f07280591e0 con 0x7f07400d0fb0 2024-04-25T09:13:26.671 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071c06b4d0 msgr2=0x7f071c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071c06b4d0 0x7f071c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0730006fd0 tx=0x7f0730008040 comp rx=0 tx=0).stop 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400d0fb0 msgr2=0x7f074015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400d0fb0 0x7f074015ce00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f072800d800 tx=0x7f072800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 shutdown_connections 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071c06b4d0 0x7f071c06d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07400d0fb0 0x7f074015ce00 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0740161230 0x7f07401f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 --2- 172.21.15.62:0/1057393176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07400b5a60 0x7f07400c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 >> 172.21.15.62:0/1057393176 conn(0x7f0740066d10 msgr2=0x7f07400b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 shutdown_connections 2024-04-25T09:13:26.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.687+0000 7f073fe00640 1 -- 172.21.15.62:0/1057393176 wait complete. 2024-04-25T09:13:26.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:13:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.924+0000 7f059da00640 1 Processor -- start 2024-04-25T09:13:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.924+0000 7f059da00640 1 -- start start 2024-04-25T09:13:26.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.925+0000 7f059da00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f059815dca0 0x7f05980b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:26.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.925+0000 7f059da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f05980b5ee0 0x7f05980d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:26.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f059da00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05980b85a0 con 0x7f05980d25a0 2024-04-25T09:13:26.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f059da00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980d1bd0 con 0x7f05980b5ee0 2024-04-25T09:13:26.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f059da00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980b6320 con 0x7f059815dca0 2024-04-25T09:13:26.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f058f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f05980b5ee0 0x7f05980d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:26.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f058f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f05980b5ee0 0x7f05980d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47542/0 (socket says 172.21.15.62:47542) 2024-04-25T09:13:26.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f058f400640 1 -- 172.21.15.62:0/162187767 learned_addr learned my addr 172.21.15.62:0/162187767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:26.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f0597e00640 1 --2- 172.21.15.62:0/162187767 >> v2:172.21.15.121:3300/0 conn(0x7f059815dca0 0x7f05980b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:26.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f058f400640 1 -- 172.21.15.62:0/162187767 >> v2:172.21.15.121:3300/0 conn(0x7f059815dca0 msgr2=0x7f05980b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:26.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f058f400640 1 --2- 172.21.15.62:0/162187767 >> v2:172.21.15.121:3300/0 conn(0x7f059815dca0 0x7f05980b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f058f400640 1 -- 172.21.15.62:0/162187767 >> v1:172.21.15.121:6789/0 conn(0x7f05980d25a0 legacy=0x7f05980d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:26.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f058f400640 1 -- 172.21.15.62:0/162187767 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980b8880 con 0x7f05980b5ee0 2024-04-25T09:13:26.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.926+0000 7f0597e00640 1 --2- 172.21.15.62:0/162187767 >> v2:172.21.15.121:3300/0 conn(0x7f059815dca0 0x7f05980b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:26.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.927+0000 7f058f400640 1 --2- 172.21.15.62:0/162187767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980b5ee0 0x7f05980d2060 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f05980d05c0 tx=0x7f058002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8223dce37868954c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:26.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.928+0000 7f0597400640 1 -- 172.21.15.62:0/162187767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580037070 con 0x7f05980b5ee0 2024-04-25T09:13:26.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.928+0000 7f0597400640 1 -- 172.21.15.62:0/162187767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580032440 con 0x7f05980b5ee0 2024-04-25T09:13:26.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.928+0000 7f0597400640 1 -- 172.21.15.62:0/162187767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580032810 con 0x7f05980b5ee0 2024-04-25T09:13:26.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.928+0000 7f059da00640 1 -- 172.21.15.62:0/162187767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980b5ee0 msgr2=0x7f05980d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:26.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.928+0000 7f059da00640 1 --2- 172.21.15.62:0/162187767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980b5ee0 0x7f05980d2060 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f05980d05c0 tx=0x7f058002a2c0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.929+0000 7f059da00640 1 -- 172.21.15.62:0/162187767 shutdown_connections 2024-04-25T09:13:26.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.929+0000 7f059da00640 1 --2- 172.21.15.62:0/162187767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980b5ee0 0x7f05980d2060 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.929+0000 7f059da00640 1 --2- 172.21.15.62:0/162187767 >> v2:172.21.15.121:3300/0 conn(0x7f059815dca0 0x7f05980b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.929+0000 7f059da00640 1 -- 172.21.15.62:0/162187767 >> 172.21.15.62:0/162187767 conn(0x7f059806b000 msgr2=0x7f059806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:26.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.929+0000 7f059da00640 1 -- 172.21.15.62:0/162187767 shutdown_connections 2024-04-25T09:13:26.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.929+0000 7f059da00640 1 -- 172.21.15.62:0/162187767 wait complete. 2024-04-25T09:13:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.930+0000 7f059da00640 1 Processor -- start 2024-04-25T09:13:26.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.931+0000 7f059da00640 1 -- start start 2024-04-25T09:13:26.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.931+0000 7f059da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05980b5ee0 0x7f059815e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.931+0000 7f059da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980d25a0 0x7f059815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.931+0000 7f059da00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f059815dca0 0x7f059815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f059da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980b9900 con 0x7f05980d25a0 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f059da00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980b9780 con 0x7f05980b5ee0 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f059da00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05980d3110 con 0x7f059815dca0 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f058f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980d25a0 0x7f059815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f058f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980d25a0 0x7f059815ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47558/0 (socket says 172.21.15.62:47558) 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f058f400640 1 -- 172.21.15.62:0/2924128563 learned_addr learned my addr 172.21.15.62:0/2924128563 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f058f400640 1 -- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f059815dca0 msgr2=0x7f059815efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f058f400640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f059815dca0 0x7f059815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f058f400640 1 -- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05980b5ee0 msgr2=0x7f059815e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f058f400640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05980b5ee0 0x7f059815e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:26.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.932+0000 7f058f400640 1 -- 172.21.15.62:0/2924128563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598160f50 con 0x7f05980d25a0 2024-04-25T09:13:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.933+0000 7f058f400640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980d25a0 0x7f059815ea90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0580000c00 tx=0x7f05800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.933+0000 7f0594c00640 1 -- 172.21.15.62:0/2924128563 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580041020 con 0x7f05980d25a0 2024-04-25T09:13:26.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.933+0000 7f0594c00640 1 -- 172.21.15.62:0/2924128563 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580030970 con 0x7f05980d25a0 2024-04-25T09:13:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.933+0000 7f0594c00640 1 -- 172.21.15.62:0/2924128563 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580037040 con 0x7f05980d25a0 2024-04-25T09:13:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.933+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059814e430 con 0x7f05980d25a0 2024-04-25T09:13:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.933+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059814e9f0 con 0x7f05980d25a0 2024-04-25T09:13:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.934+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0598075070 con 0x7f05980d25a0 2024-04-25T09:13:26.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.935+0000 7f0594c00640 1 -- 172.21.15.62:0/2924128563 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0580044020 con 0x7f05980d25a0 2024-04-25T09:13:26.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.936+0000 7f0594c00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f057406b4d0 0x7f057406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:26.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.936+0000 7f0594c00640 1 -- 172.21.15.62:0/2924128563 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05800aa950 con 0x7f05980d25a0 2024-04-25T09:13:26.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.937+0000 7f0597e00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f057406b4d0 0x7f057406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:26.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.938+0000 7f0597e00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f057406b4d0 0x7f057406d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0588006fd0 tx=0x7f0588008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:26.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:26.941+0000 7f0594c00640 1 -- 172.21.15.62:0/2924128563 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f058004b030 con 0x7f05980d25a0 2024-04-25T09:13:27.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.216+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f059815e330 con 0x7f05980d25a0 2024-04-25T09:13:27.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.217+0000 7f0594c00640 1 -- 172.21.15.62:0/2924128563 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0580079a80 con 0x7f05980d25a0 2024-04-25T09:13:27.205 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:27.205 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:13:27.205 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:13:27.205 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.205 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.206 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.207 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.208 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:13:27.209 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.210 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:13:27.211 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.212 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.213 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.214 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.215 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:13:27.216 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:13:27.217 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:13:27.218 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.219 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:27.219 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:13:27.219 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:13:27.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.220+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f057406b4d0 msgr2=0x7f057406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.220+0000 7f059da00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f057406b4d0 0x7f057406d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0588006fd0 tx=0x7f0588008040 comp rx=0 tx=0).stop 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.220+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980d25a0 msgr2=0x7f059815ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.220+0000 7f059da00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980d25a0 0x7f059815ea90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0580000c00 tx=0x7f05800046c0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.221+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 shutdown_connections 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.221+0000 7f059da00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f057406b4d0 0x7f057406d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.221+0000 7f059da00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05980d25a0 0x7f059815ea90 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.221+0000 7f059da00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f059815dca0 0x7f059815efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.221+0000 7f059da00640 1 --2- 172.21.15.62:0/2924128563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05980b5ee0 0x7f059815e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.221+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 >> 172.21.15.62:0/2924128563 conn(0x7f059806b000 msgr2=0x7f05980b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.221+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 shutdown_connections 2024-04-25T09:13:27.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.221+0000 7f059da00640 1 -- 172.21.15.62:0/2924128563 wait complete. 2024-04-25T09:13:27.228 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:27.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.443+0000 7f08bd000640 1 Processor -- start 2024-04-25T09:13:27.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.444+0000 7f08bd000640 1 -- start start 2024-04-25T09:13:27.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.445+0000 7f08bd000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f08b815dbf0 0x7f08b806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.445+0000 7f08bd000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08b80ca4e0 0x7f08b806d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.445+0000 7f08bd000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b80cf030 con 0x7f08b815d810 2024-04-25T09:13:27.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.445+0000 7f08bd000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80cf310 con 0x7f08b80ca4e0 2024-04-25T09:13:27.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.445+0000 7f08bd000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80c8cd0 con 0x7f08b815dbf0 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08b80ca4e0 0x7f08b806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08b80ca4e0 0x7f08b806d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47572/0 (socket says 172.21.15.62:47572) 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b7e00640 1 -- 172.21.15.62:0/1301339407 learned_addr learned my addr 172.21.15.62:0/1301339407 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b6a00640 1 --2- 172.21.15.62:0/1301339407 >> v2:172.21.15.121:3300/0 conn(0x7f08b815dbf0 0x7f08b806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b7e00640 1 -- 172.21.15.62:0/1301339407 >> v2:172.21.15.121:3300/0 conn(0x7f08b815dbf0 msgr2=0x7f08b806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b7e00640 1 --2- 172.21.15.62:0/1301339407 >> v2:172.21.15.121:3300/0 conn(0x7f08b815dbf0 0x7f08b806d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b7e00640 1 -- 172.21.15.62:0/1301339407 >> v1:172.21.15.121:6789/0 conn(0x7f08b815d810 legacy=0x7f08b80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b7e00640 1 -- 172.21.15.62:0/1301339407 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80c8fb0 con 0x7f08b80ca4e0 2024-04-25T09:13:27.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.446+0000 7f08b6a00640 1 --2- 172.21.15.62:0/1301339407 >> v2:172.21.15.121:3300/0 conn(0x7f08b815dbf0 0x7f08b806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:27.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.448+0000 7f08b7e00640 1 --2- 172.21.15.62:0/1301339407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b806d730 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f08b806e100 tx=0x7f08b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f939d02aa2e1aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:27.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.448+0000 7f08b6000640 1 -- 172.21.15.62:0/1301339407 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0037070 con 0x7f08b80ca4e0 2024-04-25T09:13:27.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.448+0000 7f08b6000640 1 -- 172.21.15.62:0/1301339407 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b0032440 con 0x7f08b80ca4e0 2024-04-25T09:13:27.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08b6000640 1 -- 172.21.15.62:0/1301339407 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0032810 con 0x7f08b80ca4e0 2024-04-25T09:13:27.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08bd000640 1 -- 172.21.15.62:0/1301339407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 msgr2=0x7f08b806d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08bd000640 1 --2- 172.21.15.62:0/1301339407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b806d730 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f08b806e100 tx=0x7f08b002a230 comp rx=0 tx=0).stop 2024-04-25T09:13:27.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08bd000640 1 -- 172.21.15.62:0/1301339407 shutdown_connections 2024-04-25T09:13:27.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08bd000640 1 --2- 172.21.15.62:0/1301339407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b806d730 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08bd000640 1 --2- 172.21.15.62:0/1301339407 >> v2:172.21.15.121:3300/0 conn(0x7f08b815dbf0 0x7f08b806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08bd000640 1 -- 172.21.15.62:0/1301339407 >> 172.21.15.62:0/1301339407 conn(0x7f08b80b14d0 msgr2=0x7f08b80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:27.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08bd000640 1 -- 172.21.15.62:0/1301339407 shutdown_connections 2024-04-25T09:13:27.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.449+0000 7f08bd000640 1 -- 172.21.15.62:0/1301339407 wait complete. 2024-04-25T09:13:27.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.451+0000 7f08bd000640 1 Processor -- start 2024-04-25T09:13:27.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.451+0000 7f08bd000640 1 -- start start 2024-04-25T09:13:27.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.452+0000 7f08bd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b81494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.452+0000 7f08bd000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08b815d810 0x7f08b813b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.452+0000 7f08bd000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08b815dbf0 0x7f08b815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.452+0000 7f08bd000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b81675e0 con 0x7f08b80ca4e0 2024-04-25T09:13:27.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.452+0000 7f08bd000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80b10d0 con 0x7f08b815d810 2024-04-25T09:13:27.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.452+0000 7f08bd000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8167760 con 0x7f08b815dbf0 2024-04-25T09:13:27.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.452+0000 7f08b7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b81494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.453+0000 7f08b7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b81494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47586/0 (socket says 172.21.15.62:47586) 2024-04-25T09:13:27.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.453+0000 7f08b7400640 1 -- 172.21.15.62:0/3006730167 learned_addr learned my addr 172.21.15.62:0/3006730167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.453+0000 7f08b7400640 1 -- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08b815dbf0 msgr2=0x7f08b815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.453+0000 7f08b7400640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08b815dbf0 0x7f08b815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.453+0000 7f08b7400640 1 -- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08b815d810 msgr2=0x7f08b813b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.453+0000 7f08b7400640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08b815d810 0x7f08b813b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.453+0000 7f08b7400640 1 -- 172.21.15.62:0/3006730167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b813be90 con 0x7f08b80ca4e0 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.453+0000 7f08b7400640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b81494d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c7c0 tx=0x7f08a800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.454+0000 7f08af400640 1 -- 172.21.15.62:0/3006730167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8016020 con 0x7f08b80ca4e0 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.454+0000 7f08af400640 1 -- 172.21.15.62:0/3006730167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a800aa50 con 0x7f08b80ca4e0 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.454+0000 7f08af400640 1 -- 172.21.15.62:0/3006730167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8012420 con 0x7f08b80ca4e0 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.454+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b815b340 con 0x7f08b80ca4e0 2024-04-25T09:13:27.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.454+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b815b8c0 con 0x7f08b80ca4e0 2024-04-25T09:13:27.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.455+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b804ef20 con 0x7f08b80ca4e0 2024-04-25T09:13:27.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.456+0000 7f08af400640 1 -- 172.21.15.62:0/3006730167 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08a8019020 con 0x7f08b80ca4e0 2024-04-25T09:13:27.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.457+0000 7f08af400640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f089806b4d0 0x7f089806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.457+0000 7f08af400640 1 -- 172.21.15.62:0/3006730167 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f08a8089bd0 con 0x7f08b80ca4e0 2024-04-25T09:13:27.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.457+0000 7f08b6a00640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f089806b4d0 0x7f089806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.458+0000 7f08b6a00640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f089806b4d0 0x7f089806d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f08a4006fd0 tx=0x7f08a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:27.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.462+0000 7f08af400640 1 -- 172.21.15.62:0/3006730167 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08a8020070 con 0x7f08b80ca4e0 2024-04-25T09:13:27.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.669+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08b80ced20 con 0x7f08b80ca4e0 2024-04-25T09:13:27.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.670+0000 7f08af400640 1 -- 172.21.15.62:0/3006730167 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+12881 (secure 0 0 0) 0x7f08a8058d90 con 0x7f08b80ca4e0 2024-04-25T09:13:27.658 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:27.658 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":105,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:13:26.681042+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:27.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f089806b4d0 msgr2=0x7f089806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f089806b4d0 0x7f089806d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f08a4006fd0 tx=0x7f08a4008040 comp rx=0 tx=0).stop 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 msgr2=0x7f08b81494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b81494d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c7c0 tx=0x7f08a800cc80 comp rx=0 tx=0).stop 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 shutdown_connections 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f089806b4d0 0x7f089806d990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f08b815dbf0 0x7f08b815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08b815d810 0x7f08b813b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 --2- 172.21.15.62:0/3006730167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08b80ca4e0 0x7f08b81494d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 >> 172.21.15.62:0/3006730167 conn(0x7f08b80b14d0 msgr2=0x7f08b806d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 shutdown_connections 2024-04-25T09:13:27.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.673+0000 7f08bd000640 1 -- 172.21.15.62:0/3006730167 wait complete. 2024-04-25T09:13:27.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:13:27.680 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:13:27.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.909+0000 7efef0e00640 1 Processor -- start 2024-04-25T09:13:27.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.909+0000 7efef0e00640 1 -- start start 2024-04-25T09:13:27.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.910+0000 7efef0e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efeec15dd50 0x7efeec0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.910+0000 7efef0e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efeec0d3960 0x7efeec0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.910+0000 7efef0e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec0ce780 con 0x7efeec0d2fb0 2024-04-25T09:13:27.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.910+0000 7efef0e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec0d42f0 con 0x7efeec0d3960 2024-04-25T09:13:27.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.910+0000 7efef0e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec0d3440 con 0x7efeec15dd50 2024-04-25T09:13:27.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.911+0000 7efeebe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efeec0d3960 0x7efeec0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.911+0000 7efeebe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efeec0d3960 0x7efeec0d3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47588/0 (socket says 172.21.15.62:47588) 2024-04-25T09:13:27.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.911+0000 7efeebe00640 1 -- 172.21.15.62:0/1747217465 learned_addr learned my addr 172.21.15.62:0/1747217465 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:27.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.911+0000 7efeeb400640 1 --2- 172.21.15.62:0/1747217465 >> v2:172.21.15.121:3300/0 conn(0x7efeec15dd50 0x7efeec0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.911+0000 7efeebe00640 1 -- 172.21.15.62:0/1747217465 >> v2:172.21.15.121:3300/0 conn(0x7efeec15dd50 msgr2=0x7efeec0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:27.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.911+0000 7efeebe00640 1 --2- 172.21.15.62:0/1747217465 >> v2:172.21.15.121:3300/0 conn(0x7efeec15dd50 0x7efeec0d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.911+0000 7efeebe00640 1 -- 172.21.15.62:0/1747217465 >> v1:172.21.15.121:6789/0 conn(0x7efeec0d2fb0 legacy=0x7efeec0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:27.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.912+0000 7efeebe00640 1 -- 172.21.15.62:0/1747217465 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0ce1c0 con 0x7efeec0d3960 2024-04-25T09:13:27.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.913+0000 7efeebe00640 1 --2- 172.21.15.62:0/1747217465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0d3960 0x7efeec0d3db0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efeec0d1320 tx=0x7efee402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690132a677458b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:27.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.913+0000 7efeea000640 1 -- 172.21.15.62:0/1747217465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4036070 con 0x7efeec0d3960 2024-04-25T09:13:27.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.913+0000 7efeea000640 1 -- 172.21.15.62:0/1747217465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee4031600 con 0x7efeec0d3960 2024-04-25T09:13:27.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.913+0000 7efeea000640 1 -- 172.21.15.62:0/1747217465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee40319d0 con 0x7efeec0d3960 2024-04-25T09:13:27.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.914+0000 7efef0e00640 1 -- 172.21.15.62:0/1747217465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0d3960 msgr2=0x7efeec0d3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.914+0000 7efef0e00640 1 --2- 172.21.15.62:0/1747217465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0d3960 0x7efeec0d3db0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efeec0d1320 tx=0x7efee402a730 comp rx=0 tx=0).stop 2024-04-25T09:13:27.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.914+0000 7efef0e00640 1 -- 172.21.15.62:0/1747217465 shutdown_connections 2024-04-25T09:13:27.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.914+0000 7efef0e00640 1 --2- 172.21.15.62:0/1747217465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec0d3960 0x7efeec0d3db0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.914+0000 7efef0e00640 1 --2- 172.21.15.62:0/1747217465 >> v2:172.21.15.121:3300/0 conn(0x7efeec15dd50 0x7efeec0d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.914+0000 7efef0e00640 1 -- 172.21.15.62:0/1747217465 >> 172.21.15.62:0/1747217465 conn(0x7efeec06b000 msgr2=0x7efeec06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.914+0000 7efef0e00640 1 -- 172.21.15.62:0/1747217465 shutdown_connections 2024-04-25T09:13:27.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.914+0000 7efef0e00640 1 -- 172.21.15.62:0/1747217465 wait complete. 2024-04-25T09:13:27.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.916+0000 7efef0e00640 1 Processor -- start 2024-04-25T09:13:27.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.916+0000 7efef0e00640 1 -- start start 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efef0e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeec0d3960 0x7efeec0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efef0e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec15dd50 0x7efeec15a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efef0e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeec0b3170 0x7efeec0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efef0e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0b6b00 con 0x7efeec15dd50 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efef0e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0d2840 con 0x7efeec0d3960 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efef0e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0b6c80 con 0x7efeec0b3170 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efeeaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec15dd50 0x7efeec15a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efeeaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec15dd50 0x7efeec15a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47602/0 (socket says 172.21.15.62:47602) 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efeeaa00640 1 -- 172.21.15.62:0/2736872663 learned_addr learned my addr 172.21.15.62:0/2736872663 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.917+0000 7efeeb400640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeec0d3960 0x7efeec0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efeeaa00640 1 -- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeec0b3170 msgr2=0x7efeec0c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efeeaa00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeec0b3170 0x7efeec0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efeeaa00640 1 -- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeec0d3960 msgr2=0x7efeec0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efeeaa00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeec0d3960 0x7efeec0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efeeaa00640 1 -- 172.21.15.62:0/2736872663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0b3730 con 0x7efeec15dd50 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efeeb400640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeec0d3960 0x7efeec0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efeeaa00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec15dd50 0x7efeec15a060 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7efed800c430 tx=0x7efed800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efee3400640 1 -- 172.21.15.62:0/2736872663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed80052d0 con 0x7efeec15dd50 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efee3400640 1 -- 172.21.15.62:0/2736872663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed8005490 con 0x7efeec15dd50 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec0b3a10 con 0x7efeec15dd50 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efee3400640 1 -- 172.21.15.62:0/2736872663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed8007260 con 0x7efeec15dd50 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.918+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec0b3fd0 con 0x7efeec15dd50 2024-04-25T09:13:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.919+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec0cf620 con 0x7efeec15dd50 2024-04-25T09:13:27.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.921+0000 7efee3400640 1 -- 172.21.15.62:0/2736872663 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efed8007400 con 0x7efeec15dd50 2024-04-25T09:13:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.922+0000 7efee3400640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efecc06b4d0 0x7efecc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.922+0000 7efee3400640 1 -- 172.21.15.62:0/2736872663 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efed8089440 con 0x7efeec15dd50 2024-04-25T09:13:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.922+0000 7efeeb400640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efecc06b4d0 0x7efecc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:27.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.924+0000 7efeeb400640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efecc06b4d0 0x7efecc06d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efedc006fd0 tx=0x7efedc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:27.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:27.927+0000 7efee3400640 1 -- 172.21.15.62:0/2736872663 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efeec0cf620 con 0x7efeec15dd50 2024-04-25T09:13:28.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.130+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efeec0b9950 con 0x7efeec15dd50 2024-04-25T09:13:28.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.131+0000 7efee3400640 1 -- 172.21.15.62:0/2736872663 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7efed8058570 con 0x7efeec15dd50 2024-04-25T09:13:28.118 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:13:28.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.133+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efecc06b4d0 msgr2=0x7efecc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efecc06b4d0 0x7efecc06d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efedc006fd0 tx=0x7efedc008040 comp rx=0 tx=0).stop 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec15dd50 msgr2=0x7efeec15a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec15dd50 0x7efeec15a060 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7efed800c430 tx=0x7efed800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 shutdown_connections 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efecc06b4d0 0x7efecc06d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efeec15dd50 0x7efeec15a060 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efeec0b3170 0x7efeec0c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 --2- 172.21.15.62:0/2736872663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeec0d3960 0x7efeec0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 >> 172.21.15.62:0/2736872663 conn(0x7efeec06b000 msgr2=0x7efeec1601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:28.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 shutdown_connections 2024-04-25T09:13:28.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.134+0000 7efef0e00640 1 -- 172.21.15.62:0/2736872663 wait complete. 2024-04-25T09:13:28.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:13:28.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.365+0000 7f7812400640 1 Processor -- start 2024-04-25T09:13:28.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.365+0000 7f7812400640 1 -- start start 2024-04-25T09:13:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.366+0000 7f7812400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f780c0b60b0 0x7f780c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.366+0000 7f7812400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0d3070 0x7f780c0b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.366+0000 7f7812400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c0b85a0 con 0x7f780c15dca0 2024-04-25T09:13:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.366+0000 7f7812400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c0d1bd0 con 0x7f780c0b60b0 2024-04-25T09:13:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.366+0000 7f7812400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0b8880 con 0x7f780c0d3070 2024-04-25T09:13:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.366+0000 7f7811000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f780c15dca0 0x7f780c0b59a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60672/0 (socket says 172.21.15.62:60672) 2024-04-25T09:13:28.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.367+0000 7f7811000640 1 -- 172.21.15.62:0/3832072496 learned_addr learned my addr 172.21.15.62:0/3832072496 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.367+0000 7f780be00640 1 --2- 172.21.15.62:0/3832072496 >> v2:172.21.15.121:3300/0 conn(0x7f780c0b60b0 0x7f780c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.367+0000 7f7811a00640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0d3070 0x7f780c0b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.367+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989722599 0 0) 0x7f780c0b85a0 con 0x7f780c15dca0 2024-04-25T09:13:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.367+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f0003520 con 0x7f780c15dca0 2024-04-25T09:13:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.367+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 233059113 0 0) 0x7f77f0003520 con 0x7f780c15dca0 2024-04-25T09:13:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0d3070 msgr2=0x7f780c0b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0d3070 0x7f780c0b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 >> v2:172.21.15.121:3300/0 conn(0x7f780c0b60b0 msgr2=0x7f780c0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 --2- 172.21.15.62:0/3832072496 >> v2:172.21.15.121:3300/0 conn(0x7f780c0b60b0 0x7f780c0b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c0b8b60 con 0x7f780c15dca0 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f77fc001670 con 0x7f780c15dca0 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 >> v1:172.21.15.62:6789/0 conn(0x7f780c15dca0 legacy=0x7f780c0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f0003a20 0x7f77f0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0d3070 0x7f77f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f77f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0d1bd0 con 0x7f77f0004d40 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0b8880 con 0x7f77f0003a20 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f000ba40 con 0x7f780c0d3070 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f7811a00640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f77f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f780be00640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0d3070 0x7f77f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.368+0000 7f7811000640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f0003a20 0x7f77f0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.369+0000 7f7811a00640 1 -- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0d3070 msgr2=0x7f77f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:28.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.369+0000 7f7811a00640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c0d3070 0x7f77f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.369+0000 7f7811a00640 1 -- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f0003a20 msgr2=0x7f77f0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:28.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.369+0000 7f7811a00640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f0003a20 0x7f77f0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.369+0000 7f7811a00640 1 -- 172.21.15.62:0/3832072496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0003520 con 0x7f77f0004d40 2024-04-25T09:13:28.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.369+0000 7f7811000640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77f0003a20 0x7f77f0003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.370+0000 7f7811a00640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f77f00051c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7804009f90 tx=0x7f77f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52226266564c3ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.370+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804009c10 con 0x7f77f0004d40 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.370+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7804036070 con 0x7f77f0004d40 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.370+0000 7f780b400640 1 -- 172.21.15.62:0/3832072496 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804031850 con 0x7f77f0004d40 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.371+0000 7f7812400640 1 -- 172.21.15.62:0/3832072496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 msgr2=0x7f77f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.371+0000 7f7812400640 1 --2- 172.21.15.62:0/3832072496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f77f00051c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f7804009f90 tx=0x7f77f0005830 comp rx=0 tx=0).stop 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.371+0000 7f7811a00640 1 -- 172.21.15.62:0/3832072496 reap_dead start 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.371+0000 7f7812400640 1 -- 172.21.15.62:0/3832072496 shutdown_connections 2024-04-25T09:13:28.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.371+0000 7f7812400640 1 -- 172.21.15.62:0/3832072496 >> 172.21.15.62:0/3832072496 conn(0x7f780c06b000 msgr2=0x7f780c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:28.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.371+0000 7f7812400640 1 -- 172.21.15.62:0/3832072496 shutdown_connections 2024-04-25T09:13:28.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.371+0000 7f7812400640 1 -- 172.21.15.62:0/3832072496 wait complete. 2024-04-25T09:13:28.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.373+0000 7f7812400640 1 Processor -- start 2024-04-25T09:13:28.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.373+0000 7f7812400640 1 -- start start 2024-04-25T09:13:28.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.374+0000 7f7812400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f780c0c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.374+0000 7f7812400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c153bf0 0x7f780c15fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.374+0000 7f7812400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c15ff90 0x7f780c151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.374+0000 7f7812400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0b7090 con 0x7f77f0004d40 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.374+0000 7f7812400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0b6f10 con 0x7f780c153bf0 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.374+0000 7f7812400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0b7210 con 0x7f780c15ff90 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f7811000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f780c0c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f7811000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f780c0c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47620/0 (socket says 172.21.15.62:47620) 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f7811000640 1 -- 172.21.15.62:0/2131128176 learned_addr learned my addr 172.21.15.62:0/2131128176 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f780be00640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c153bf0 0x7f780c15fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f7811000640 1 -- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c15ff90 msgr2=0x7f780c151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f7811000640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c15ff90 0x7f780c151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f7811000640 1 -- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c153bf0 msgr2=0x7f780c15fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f7811000640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c153bf0 0x7f780c15fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f7811000640 1 -- 172.21.15.62:0/2131128176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c151280 con 0x7f77f0004d40 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.375+0000 7f780be00640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c153bf0 0x7f780c15fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.376+0000 7f7811000640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f780c0c9720 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e750 tx=0x7f77fc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.376+0000 7f7808c00640 1 -- 172.21.15.62:0/2131128176 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00ee60 con 0x7f77f0004d40 2024-04-25T09:13:28.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.376+0000 7f7808c00640 1 -- 172.21.15.62:0/2131128176 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc018070 con 0x7f77f0004d40 2024-04-25T09:13:28.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.376+0000 7f7808c00640 1 -- 172.21.15.62:0/2131128176 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00cad0 con 0x7f77f0004d40 2024-04-25T09:13:28.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.376+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c151560 con 0x7f77f0004d40 2024-04-25T09:13:28.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.376+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c160a00 con 0x7f77f0004d40 2024-04-25T09:13:28.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.378+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c075070 con 0x7f77f0004d40 2024-04-25T09:13:28.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.378+0000 7f7808c00640 1 -- 172.21.15.62:0/2131128176 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f77fc00cc70 con 0x7f77f0004d40 2024-04-25T09:13:28.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.379+0000 7f7808c00640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.379+0000 7f7808c00640 1 -- 172.21.15.62:0/2131128176 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f77fc089d00 con 0x7f77f0004d40 2024-04-25T09:13:28.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.380+0000 7f780be00640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.381+0000 7f780be00640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f780c0ccac0 tx=0x7f77f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:28.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.384+0000 7f7808c00640 1 -- 172.21.15.62:0/2131128176 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77fc058ed0 con 0x7f77f0004d40 2024-04-25T09:13:28.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.583+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f780c066a90 con 0x7f77f0004d40 2024-04-25T09:13:28.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.584+0000 7f7808c00640 1 -- 172.21.15.62:0/2131128176 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) v1 ==== 84+0+65 (secure 0 0 0) 0x7f77fc0587d0 con 0x7f77f0004d40 2024-04-25T09:13:28.571 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:28.571 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:13:28.571 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:13:28.572 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:13:28.572 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:13:28.572 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:13:28.572 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:13:28.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 msgr2=0x7f77ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f780c0ccac0 tx=0x7f77f8008040 comp rx=0 tx=0).stop 2024-04-25T09:13:28.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 msgr2=0x7f780c0c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f780c0c9720 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00e750 tx=0x7f77fc00ec10 comp rx=0 tx=0).stop 2024-04-25T09:13:28.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 shutdown_connections 2024-04-25T09:13:28.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f77ec06b4d0 0x7f77ec06d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f780c15ff90 0x7f780c151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f780c153bf0 0x7f780c15fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 --2- 172.21.15.62:0/2131128176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f77f0004d40 0x7f780c0c9720 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 >> 172.21.15.62:0/2131128176 conn(0x7f780c06b000 msgr2=0x7f780c0b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:28.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 shutdown_connections 2024-04-25T09:13:28.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.586+0000 7f7812400640 1 -- 172.21.15.62:0/2131128176 wait complete. 2024-04-25T09:13:28.592 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:13:28.593 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:13:28.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.820+0000 7f9b3e000640 1 Processor -- start 2024-04-25T09:13:28.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.821+0000 7f9b3e000640 1 -- start start 2024-04-25T09:13:28.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.822+0000 7f9b3e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9b38161270 0x7f9b380b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.822+0000 7f9b3e000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b38161650 0x7f9b380b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.822+0000 7f9b3e000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9b380b2980 0x7f9b38161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.822+0000 7f9b3e000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b380d3900 con 0x7f9b38161270 2024-04-25T09:13:28.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.822+0000 7f9b3e000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b380d1480 con 0x7f9b380b2980 2024-04-25T09:13:28.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.822+0000 7f9b3e000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380d11a0 con 0x7f9b38161650 2024-04-25T09:13:28.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.822+0000 7f9b3cc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9b38161270 0x7f9b380b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.823+0000 7f9b3cc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9b38161270 0x7f9b380b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47622/0 (socket says 172.21.15.62:47622) 2024-04-25T09:13:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.823+0000 7f9b3cc00640 1 -- 172.21.15.62:0/31741795 learned_addr learned my addr 172.21.15.62:0/31741795 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.823+0000 7f9b37e00640 1 --2- 172.21.15.62:0/31741795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b38161650 0x7f9b380b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.823+0000 7f9b3cc00640 1 -- 172.21.15.62:0/31741795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b38161650 msgr2=0x7f9b380b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.823+0000 7f9b3cc00640 1 --2- 172.21.15.62:0/31741795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b38161650 0x7f9b380b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.823+0000 7f9b3cc00640 1 -- 172.21.15.62:0/31741795 >> v2:172.21.15.121:3300/0 conn(0x7f9b380b2980 msgr2=0x7f9b38161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.823+0000 7f9b3cc00640 1 --2- 172.21.15.62:0/31741795 >> v2:172.21.15.121:3300/0 conn(0x7f9b380b2980 0x7f9b38161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.823+0000 7f9b3cc00640 1 -- 172.21.15.62:0/31741795 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b380cee80 con 0x7f9b38161270 2024-04-25T09:13:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.824+0000 7f9b3cc00640 1 --2- 172.21.15.62:0/31741795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b380b5560 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9b380d4af0 tx=0x7f9b2802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58325f2445408f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.825+0000 7f9b37400640 1 -- 172.21.15.62:0/31741795 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28037070 con 0x7f9b38161270 2024-04-25T09:13:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.825+0000 7f9b37400640 1 -- 172.21.15.62:0/31741795 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2802ae00 con 0x7f9b38161270 2024-04-25T09:13:28.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.825+0000 7f9b37400640 1 -- 172.21.15.62:0/31741795 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28032660 con 0x7f9b38161270 2024-04-25T09:13:28.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.825+0000 7f9b3e000640 1 -- 172.21.15.62:0/31741795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 msgr2=0x7f9b380b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.825+0000 7f9b3e000640 1 --2- 172.21.15.62:0/31741795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b380b5560 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9b380d4af0 tx=0x7f9b2802a0b0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.826+0000 7f9b3e000640 1 -- 172.21.15.62:0/31741795 shutdown_connections 2024-04-25T09:13:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.826+0000 7f9b3e000640 1 --2- 172.21.15.62:0/31741795 >> v2:172.21.15.121:3300/0 conn(0x7f9b380b2980 0x7f9b38161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.826+0000 7f9b3e000640 1 --2- 172.21.15.62:0/31741795 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b38161650 0x7f9b380b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.826+0000 7f9b3e000640 1 --2- 172.21.15.62:0/31741795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b380b5560 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.826+0000 7f9b3e000640 1 -- 172.21.15.62:0/31741795 >> 172.21.15.62:0/31741795 conn(0x7f9b38066d10 msgr2=0x7f9b38067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.826+0000 7f9b3e000640 1 -- 172.21.15.62:0/31741795 shutdown_connections 2024-04-25T09:13:28.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.826+0000 7f9b3e000640 1 -- 172.21.15.62:0/31741795 wait complete. 2024-04-25T09:13:28.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.828+0000 7f9b3e000640 1 Processor -- start 2024-04-25T09:13:28.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.828+0000 7f9b3e000640 1 -- start start 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.829+0000 7f9b3e000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b380b2980 0x7f9b380c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.829+0000 7f9b3e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b3814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.829+0000 7f9b3e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b38161650 0x7f9b3815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.829+0000 7f9b3e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380b2ed0 con 0x7f9b38161270 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.829+0000 7f9b3e000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380d3590 con 0x7f9b38161650 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.829+0000 7f9b3e000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380b3050 con 0x7f9b380b2980 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b37e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b3814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b37e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b3814cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47638/0 (socket says 172.21.15.62:47638) 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b37e00640 1 -- 172.21.15.62:0/3612004088 learned_addr learned my addr 172.21.15.62:0/3612004088 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:28.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b3cc00640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b380b2980 0x7f9b380c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b3d600640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b38161650 0x7f9b3815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b37e00640 1 -- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b380b2980 msgr2=0x7f9b380c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b37e00640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b380b2980 0x7f9b380c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b37e00640 1 -- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b38161650 msgr2=0x7f9b3815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:28.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b37e00640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b38161650 0x7f9b3815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:28.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b37e00640 1 -- 172.21.15.62:0/3612004088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3814d450 con 0x7f9b38161270 2024-04-25T09:13:28.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.830+0000 7f9b3d600640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b38161650 0x7f9b3815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:28.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.831+0000 7f9b37e00640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b3814cf10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000ccd0 tx=0x7f9b2000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:28.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.831+0000 7f9b34c00640 1 -- 172.21.15.62:0/3612004088 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20003990 con 0x7f9b38161270 2024-04-25T09:13:28.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.831+0000 7f9b34c00640 1 -- 172.21.15.62:0/3612004088 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b20003b30 con 0x7f9b38161270 2024-04-25T09:13:28.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.831+0000 7f9b34c00640 1 -- 172.21.15.62:0/3612004088 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20004020 con 0x7f9b38161270 2024-04-25T09:13:28.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.831+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b381585f0 con 0x7f9b38161270 2024-04-25T09:13:28.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.831+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b38158bb0 con 0x7f9b38161270 2024-04-25T09:13:28.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.832+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b38072910 con 0x7f9b38161270 2024-04-25T09:13:28.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.833+0000 7f9b34c00640 1 -- 172.21.15.62:0/3612004088 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b200041c0 con 0x7f9b38161270 2024-04-25T09:13:28.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.834+0000 7f9b34c00640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b1406b4d0 0x7f9b1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:28.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.834+0000 7f9b34c00640 1 -- 172.21.15.62:0/3612004088 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b2008a380 con 0x7f9b38161270 2024-04-25T09:13:28.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.835+0000 7f9b3cc00640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b1406b4d0 0x7f9b1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:28.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.836+0000 7f9b3cc00640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b1406b4d0 0x7f9b1406d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9b28034000 tx=0x7f9b2802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:28.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:28.838+0000 7f9b34c00640 1 -- 172.21.15.62:0/3612004088 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b20014030 con 0x7f9b38161270 2024-04-25T09:13:29.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.037+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9b380750e0 con 0x7f9b38161270 2024-04-25T09:13:29.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.715+0000 7f9b34c00640 1 -- 172.21.15.62:0/3612004088 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v106) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9b20059540 con 0x7f9b38161270 2024-04-25T09:13:29.703 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:13:29.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b1406b4d0 msgr2=0x7f9b1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:29.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b1406b4d0 0x7f9b1406d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9b28034000 tx=0x7f9b2802e000 comp rx=0 tx=0).stop 2024-04-25T09:13:29.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 msgr2=0x7f9b3814cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:29.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b3814cf10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000ccd0 tx=0x7f9b2000a5c0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 shutdown_connections 2024-04-25T09:13:29.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b1406b4d0 0x7f9b1406d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b38161270 0x7f9b3814cf10 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b38161650 0x7f9b3815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 --2- 172.21.15.62:0/3612004088 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b380b2980 0x7f9b380c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.719+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 >> 172.21.15.62:0/3612004088 conn(0x7f9b38066d10 msgr2=0x7f9b380670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:29.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.720+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 shutdown_connections 2024-04-25T09:13:29.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.720+0000 7f9b3e000640 1 -- 172.21.15.62:0/3612004088 wait complete. 2024-04-25T09:13:29.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:13:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.961+0000 7f639da00640 1 Processor -- start 2024-04-25T09:13:29.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.962+0000 7f639da00640 1 -- start start 2024-04-25T09:13:29.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.963+0000 7f639da00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f639815dbf0 0x7f63980b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:29.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.963+0000 7f639da00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63980b7e20 con 0x7f63980ceee0 2024-04-25T09:13:29.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f639da00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63980b8100 con 0x7f63980b76a0 2024-04-25T09:13:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f639da00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63980b83e0 con 0x7f639815dbf0 2024-04-25T09:13:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f6396a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f639815dbf0 0x7f63980b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f6396a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f639815dbf0 0x7f63980b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47644/0 (socket says 172.21.15.62:47644) 2024-04-25T09:13:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f6396a00640 1 -- 172.21.15.62:0/1168021380 learned_addr learned my addr 172.21.15.62:0/1168021380 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f6397400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f63980ceee0 0x7f63980d1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60674/0 (socket says 172.21.15.62:60674) 2024-04-25T09:13:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f6396a00640 1 -- 172.21.15.62:0/1168021380 >> v1:172.21.15.121:6789/0 conn(0x7f63980b76a0 legacy=0x7f63980ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f6396a00640 1 -- 172.21.15.62:0/1168021380 >> v1:172.21.15.62:6789/0 conn(0x7f63980ceee0 legacy=0x7f63980d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:29.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f6396a00640 1 -- 172.21.15.62:0/1168021380 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63980b86c0 con 0x7f639815dbf0 2024-04-25T09:13:29.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.964+0000 7f6395600640 1 -- 172.21.15.62:0/1168021380 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164436711 0 0) 0x7f63980b7e20 con 0x7f63980ceee0 2024-04-25T09:13:29.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.966+0000 7f6396a00640 1 --2- 172.21.15.62:0/1168021380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980b96d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6388009870 tx=0x7f638802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2fbe06fe0c3b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:29.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.966+0000 7f6395600640 1 -- 172.21.15.62:0/1168021380 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388037070 con 0x7f639815dbf0 2024-04-25T09:13:29.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.966+0000 7f6395600640 1 -- 172.21.15.62:0/1168021380 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6388032440 con 0x7f639815dbf0 2024-04-25T09:13:29.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.966+0000 7f6395600640 1 -- 172.21.15.62:0/1168021380 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388032810 con 0x7f639815dbf0 2024-04-25T09:13:29.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.967+0000 7f639da00640 1 -- 172.21.15.62:0/1168021380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 msgr2=0x7f63980b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:29.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.967+0000 7f639da00640 1 --2- 172.21.15.62:0/1168021380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980b96d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6388009870 tx=0x7f638802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.967+0000 7f639da00640 1 -- 172.21.15.62:0/1168021380 shutdown_connections 2024-04-25T09:13:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.967+0000 7f639da00640 1 --2- 172.21.15.62:0/1168021380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980b96d0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.967+0000 7f639da00640 1 -- 172.21.15.62:0/1168021380 >> 172.21.15.62:0/1168021380 conn(0x7f639806b000 msgr2=0x7f639806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.967+0000 7f639da00640 1 -- 172.21.15.62:0/1168021380 shutdown_connections 2024-04-25T09:13:29.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.967+0000 7f639da00640 1 -- 172.21.15.62:0/1168021380 wait complete. 2024-04-25T09:13:29.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.969+0000 7f639da00640 1 Processor -- start 2024-04-25T09:13:29.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.969+0000 7f639da00640 1 -- start start 2024-04-25T09:13:29.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f639da00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63980b76a0 0x7f639814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:29.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f639da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63980ceee0 0x7f63980c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:29.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f639da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:29.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f639da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980b7ae0 con 0x7f639815dbf0 2024-04-25T09:13:29.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f639da00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639815dfd0 con 0x7f63980ceee0 2024-04-25T09:13:29.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f639da00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63980b8ec0 con 0x7f63980b76a0 2024-04-25T09:13:29.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6397400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:29.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6397400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47650/0 (socket says 172.21.15.62:47650) 2024-04-25T09:13:29.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6397400640 1 -- 172.21.15.62:0/1308964552 learned_addr learned my addr 172.21.15.62:0/1308964552 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:29.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6396a00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63980b76a0 0x7f639814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:29.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6396000640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63980ceee0 0x7f63980c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:29.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6397400640 1 -- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63980b76a0 msgr2=0x7f639814cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6397400640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63980b76a0 0x7f639814cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6397400640 1 -- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63980ceee0 msgr2=0x7f63980c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6397400640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63980ceee0 0x7f63980c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.970+0000 7f6397400640 1 -- 172.21.15.62:0/1308964552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63981e7030 con 0x7f639815dbf0 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.971+0000 7f6397400640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980c98c0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f639000c7e0 tx=0x7f639000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.971+0000 7f638ea00640 1 -- 172.21.15.62:0/1308964552 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390018070 con 0x7f639815dbf0 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.971+0000 7f638ea00640 1 -- 172.21.15.62:0/1308964552 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639000a5e0 con 0x7f639815dbf0 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.971+0000 7f638ea00640 1 -- 172.21.15.62:0/1308964552 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639000a9d0 con 0x7f639815dbf0 2024-04-25T09:13:29.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.971+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63981e7310 con 0x7f639815dbf0 2024-04-25T09:13:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.971+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63981e78d0 con 0x7f639815dbf0 2024-04-25T09:13:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.972+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639804ef20 con 0x7f639815dbf0 2024-04-25T09:13:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.974+0000 7f638ea00640 1 -- 172.21.15.62:0/1308964552 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f639001b070 con 0x7f639815dbf0 2024-04-25T09:13:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.975+0000 7f638ea00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.975+0000 7f638ea00640 1 -- 172.21.15.62:0/1308964552 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6390089670 con 0x7f639815dbf0 2024-04-25T09:13:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.976+0000 7f6396a00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.977+0000 7f6396a00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f637406b4d0 0x7f637406d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6388034000 tx=0x7f638802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:29.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:29.980+0000 7f638ea00640 1 -- 172.21.15.62:0/1308964552 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6390058630 con 0x7f639815dbf0 2024-04-25T09:13:30.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.182+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f63980d4310 con 0x7f639815dbf0 2024-04-25T09:13:30.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.732+0000 7f638ea00640 1 -- 172.21.15.62:0/1308964552 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v107) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6390057f30 con 0x7f639815dbf0 2024-04-25T09:13:30.720 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:13:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.738+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f637406b4d0 msgr2=0x7f637406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.738+0000 7f639da00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f637406b4d0 0x7f637406d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6388034000 tx=0x7f638802e000 comp rx=0 tx=0).stop 2024-04-25T09:13:30.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.738+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 msgr2=0x7f63980c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.738+0000 7f639da00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980c98c0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f639000c7e0 tx=0x7f639000cca0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.739+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 shutdown_connections 2024-04-25T09:13:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.739+0000 7f639da00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f637406b4d0 0x7f637406d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.739+0000 7f639da00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f639815dbf0 0x7f63980c98c0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.739+0000 7f639da00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63980ceee0 0x7f63980c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.739+0000 7f639da00640 1 --2- 172.21.15.62:0/1308964552 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63980b76a0 0x7f639814cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.739+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 >> 172.21.15.62:0/1308964552 conn(0x7f639806b000 msgr2=0x7f639815c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:30.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.739+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 shutdown_connections 2024-04-25T09:13:30.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.739+0000 7f639da00640 1 -- 172.21.15.62:0/1308964552 wait complete. 2024-04-25T09:13:30.763 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:13:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.957+0000 7fc983e00640 1 Processor -- start 2024-04-25T09:13:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.957+0000 7fc983e00640 1 -- start start 2024-04-25T09:13:30.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.959+0000 7fc983e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9840b2d80 0x7fc9840b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:30.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.959+0000 7fc983e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9840ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:30.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.959+0000 7fc983e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9840b4430 con 0x7fc9840b54d0 2024-04-25T09:13:30.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.959+0000 7fc983e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9840b4710 con 0x7fc9840b2d80 2024-04-25T09:13:30.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.959+0000 7fc983e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840b49f0 con 0x7fc9840b36a0 2024-04-25T09:13:30.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.960+0000 7fc982000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9840ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:30.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.960+0000 7fc982a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9840b2d80 0x7fc9840b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:30.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.960+0000 7fc982000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9840ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41950/0 (socket says 172.21.15.62:41950) 2024-04-25T09:13:30.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.960+0000 7fc982000640 1 -- 172.21.15.62:0/2164002695 learned_addr learned my addr 172.21.15.62:0/2164002695 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:30.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.960+0000 7fc983400640 1 --1- 172.21.15.62:0/2164002695 >> v1:172.21.15.121:6789/0 conn(0x7fc9840b54d0 0x7fc9840700c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:45302/0 (socket says 172.21.15.62:45302) 2024-04-25T09:13:30.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.961+0000 7fc982a00640 1 -- 172.21.15.62:0/2164002695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 msgr2=0x7fc9840ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:30.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.961+0000 7fc982a00640 1 --2- 172.21.15.62:0/2164002695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9840ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.961+0000 7fc982a00640 1 -- 172.21.15.62:0/2164002695 >> v1:172.21.15.121:6789/0 conn(0x7fc9840b54d0 legacy=0x7fc9840700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:30.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.961+0000 7fc982a00640 1 -- 172.21.15.62:0/2164002695 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840b4cd0 con 0x7fc9840b2d80 2024-04-25T09:13:30.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.962+0000 7fc982a00640 1 --2- 172.21.15.62:0/2164002695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 0x7fc9840b3160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc974009d90 tx=0x7fc97402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a60e94db76de57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:30.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc981600640 1 -- 172.21.15.62:0/2164002695 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974007740 con 0x7fc9840b2d80 2024-04-25T09:13:30.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc981600640 1 -- 172.21.15.62:0/2164002695 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9740078e0 con 0x7fc9840b2d80 2024-04-25T09:13:30.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc981600640 1 -- 172.21.15.62:0/2164002695 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974007cb0 con 0x7fc9840b2d80 2024-04-25T09:13:30.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc983e00640 1 -- 172.21.15.62:0/2164002695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 msgr2=0x7fc9840b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:30.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc983e00640 1 --2- 172.21.15.62:0/2164002695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 0x7fc9840b3160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc974009d90 tx=0x7fc97402a660 comp rx=0 tx=0).stop 2024-04-25T09:13:30.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc983e00640 1 -- 172.21.15.62:0/2164002695 shutdown_connections 2024-04-25T09:13:30.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc983e00640 1 --2- 172.21.15.62:0/2164002695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9840ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc983e00640 1 --2- 172.21.15.62:0/2164002695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 0x7fc9840b3160 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc983e00640 1 -- 172.21.15.62:0/2164002695 >> 172.21.15.62:0/2164002695 conn(0x7fc984066f20 msgr2=0x7fc9840b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:30.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.963+0000 7fc983e00640 1 -- 172.21.15.62:0/2164002695 shutdown_connections 2024-04-25T09:13:30.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.964+0000 7fc983e00640 1 -- 172.21.15.62:0/2164002695 wait complete. 2024-04-25T09:13:30.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.965+0000 7fc983e00640 1 Processor -- start 2024-04-25T09:13:30.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.965+0000 7fc983e00640 1 -- start start 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.966+0000 7fc983e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 0x7fc98415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.966+0000 7fc983e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9841575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.966+0000 7fc983e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9840b54d0 0x7fc9841ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.966+0000 7fc983e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840ce7f0 con 0x7fc9840b54d0 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.966+0000 7fc983e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840b40e0 con 0x7fc9840b2d80 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.966+0000 7fc983e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840ce970 con 0x7fc9840b36a0 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc983400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9840b54d0 0x7fc9841ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc983400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9840b54d0 0x7fc9841ef780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47656/0 (socket says 172.21.15.62:47656) 2024-04-25T09:13:30.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc983400640 1 -- 172.21.15.62:0/2308941306 learned_addr learned my addr 172.21.15.62:0/2308941306 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc982a00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 0x7fc98415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc982000640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9841575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc983400640 1 -- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 msgr2=0x7fc9841575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc983400640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9841575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc983400640 1 -- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 msgr2=0x7fc98415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc983400640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 0x7fc98415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc983400640 1 -- 172.21.15.62:0/2308941306 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98415aa50 con 0x7fc9840b54d0 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc982a00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 0x7fc98415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.967+0000 7fc982000640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9841575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.968+0000 7fc983400640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9840b54d0 0x7fc9841ef780 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001eb0 tx=0x7fc97c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.968+0000 7fc97aa00640 1 -- 172.21.15.62:0/2308941306 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0074e0 con 0x7fc9840b54d0 2024-04-25T09:13:30.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.968+0000 7fc97aa00640 1 -- 172.21.15.62:0/2308941306 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c007680 con 0x7fc9840b54d0 2024-04-25T09:13:30.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.968+0000 7fc97aa00640 1 -- 172.21.15.62:0/2308941306 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c0049e0 con 0x7fc9840b54d0 2024-04-25T09:13:30.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.968+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9841f0340 con 0x7fc9840b54d0 2024-04-25T09:13:30.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.968+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98415ad10 con 0x7fc9840b54d0 2024-04-25T09:13:30.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.969+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98404ef20 con 0x7fc9840b54d0 2024-04-25T09:13:30.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.971+0000 7fc97aa00640 1 -- 172.21.15.62:0/2308941306 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc97c01a020 con 0x7fc9840b54d0 2024-04-25T09:13:30.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.972+0000 7fc97aa00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96006b4d0 0x7fc96006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:30.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.972+0000 7fc97aa00640 1 -- 172.21.15.62:0/2308941306 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc97c089a00 con 0x7fc9840b54d0 2024-04-25T09:13:30.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.972+0000 7fc982a00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96006b4d0 0x7fc96006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:30.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.973+0000 7fc982a00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96006b4d0 0x7fc96006d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc974034000 tx=0x7fc97402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:30.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:30.977+0000 7fc97aa00640 1 -- 172.21.15.62:0/2308941306 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc97c0587c0 con 0x7fc9840b54d0 2024-04-25T09:13:31.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.217+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc9840af210 con 0x7fc9840b54d0 2024-04-25T09:13:31.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.752+0000 7fc97aa00640 1 -- 172.21.15.62:0/2308941306 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc97c0580c0 con 0x7fc9840b54d0 2024-04-25T09:13:31.743 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T09:13:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.761+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96006b4d0 msgr2=0x7fc96006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.761+0000 7fc983e00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96006b4d0 0x7fc96006d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc974034000 tx=0x7fc97402f000 comp rx=0 tx=0).stop 2024-04-25T09:13:31.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.761+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9840b54d0 msgr2=0x7fc9841ef780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.761+0000 7fc983e00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9840b54d0 0x7fc9841ef780 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fc97c001eb0 tx=0x7fc97c00c820 comp rx=0 tx=0).stop 2024-04-25T09:13:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.762+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 shutdown_connections 2024-04-25T09:13:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.762+0000 7fc983e00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc96006b4d0 0x7fc96006d990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.762+0000 7fc983e00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9840b54d0 0x7fc9841ef780 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.762+0000 7fc983e00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9840b36a0 0x7fc9841575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.762+0000 7fc983e00640 1 --2- 172.21.15.62:0/2308941306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9840b2d80 0x7fc98415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.762+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 >> 172.21.15.62:0/2308941306 conn(0x7fc984066f20 msgr2=0x7fc984159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:31.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.766+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 shutdown_connections 2024-04-25T09:13:31.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.766+0000 7fc983e00640 1 -- 172.21.15.62:0/2308941306 wait complete. 2024-04-25T09:13:31.779 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:31.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.969+0000 7f9fade00640 1 Processor -- start 2024-04-25T09:13:31.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.969+0000 7f9fade00640 1 -- start start 2024-04-25T09:13:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.970+0000 7f9fade00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9fa80b56f0 0x7f9fa8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.971+0000 7f9fade00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9fa80b27a0 0x7f9fa8161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.971+0000 7f9fade00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa80d3110 con 0x7f9fa80b6a90 2024-04-25T09:13:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.971+0000 7f9fade00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa8161780 con 0x7f9fa80b56f0 2024-04-25T09:13:31.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.971+0000 7f9fade00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa80cedc0 con 0x7f9fa80b27a0 2024-04-25T09:13:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.971+0000 7f9faca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9fa80b56f0 0x7f9fa8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.971+0000 7f9fa7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9fa80b6a90 0x7f9fa8161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60676/0 (socket says 172.21.15.62:60676) 2024-04-25T09:13:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.971+0000 7f9fa7e00640 1 -- 172.21.15.62:0/1047886052 learned_addr learned my addr 172.21.15.62:0/1047886052 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.971+0000 7f9fad400640 1 --2- 172.21.15.62:0/1047886052 >> v2:172.21.15.121:3300/0 conn(0x7f9fa80b27a0 0x7f9fa8161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.972+0000 7f9faca00640 1 -- 172.21.15.62:0/1047886052 >> v2:172.21.15.121:3300/0 conn(0x7f9fa80b27a0 msgr2=0x7f9fa8161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.972+0000 7f9faca00640 1 --2- 172.21.15.62:0/1047886052 >> v2:172.21.15.121:3300/0 conn(0x7f9fa80b27a0 0x7f9fa8161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.972+0000 7f9faca00640 1 -- 172.21.15.62:0/1047886052 >> v1:172.21.15.62:6789/0 conn(0x7f9fa80b6a90 legacy=0x7f9fa8161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.972+0000 7f9faca00640 1 -- 172.21.15.62:0/1047886052 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80d33f0 con 0x7f9fa80b56f0 2024-04-25T09:13:31.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.972+0000 7f9fa7400640 1 -- 172.21.15.62:0/1047886052 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478035925 0 0) 0x7f9fa80d3110 con 0x7f9fa80b6a90 2024-04-25T09:13:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.973+0000 7f9faca00640 1 --2- 172.21.15.62:0/1047886052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b56f0 0x7f9fa8160d00 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80b2720 tx=0x7f9f9802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881f9bd185e1963b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.973+0000 7f9fa7400640 1 -- 172.21.15.62:0/1047886052 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98007740 con 0x7f9fa80b56f0 2024-04-25T09:13:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.973+0000 7f9fa7400640 1 -- 172.21.15.62:0/1047886052 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f980078e0 con 0x7f9fa80b56f0 2024-04-25T09:13:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.973+0000 7f9fa7400640 1 -- 172.21.15.62:0/1047886052 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98007cb0 con 0x7f9fa80b56f0 2024-04-25T09:13:31.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.974+0000 7f9fade00640 1 -- 172.21.15.62:0/1047886052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b56f0 msgr2=0x7f9fa8160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.974+0000 7f9fade00640 1 --2- 172.21.15.62:0/1047886052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b56f0 0x7f9fa8160d00 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80b2720 tx=0x7f9f9802a680 comp rx=0 tx=0).stop 2024-04-25T09:13:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.974+0000 7f9fade00640 1 -- 172.21.15.62:0/1047886052 shutdown_connections 2024-04-25T09:13:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.974+0000 7f9fade00640 1 --2- 172.21.15.62:0/1047886052 >> v2:172.21.15.121:3300/0 conn(0x7f9fa80b27a0 0x7f9fa8161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.974+0000 7f9fade00640 1 --2- 172.21.15.62:0/1047886052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b56f0 0x7f9fa8160d00 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.974+0000 7f9fade00640 1 -- 172.21.15.62:0/1047886052 >> 172.21.15.62:0/1047886052 conn(0x7f9fa8066d10 msgr2=0x7f9fa8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.974+0000 7f9fade00640 1 -- 172.21.15.62:0/1047886052 shutdown_connections 2024-04-25T09:13:31.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.974+0000 7f9fade00640 1 -- 172.21.15.62:0/1047886052 wait complete. 2024-04-25T09:13:31.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.976+0000 7f9fade00640 1 Processor -- start 2024-04-25T09:13:31.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.976+0000 7f9fade00640 1 -- start start 2024-04-25T09:13:31.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9fade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b27a0 0x7f9fa81f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9fade00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fa80b56f0 0x7f9fa81f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9fade00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fa80b6a90 0x7f9fa815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9fade00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80b2d60 con 0x7f9fa80b27a0 2024-04-25T09:13:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9fade00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80cea50 con 0x7f9fa80b6a90 2024-04-25T09:13:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9fade00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80b2ee0 con 0x7f9fa80b56f0 2024-04-25T09:13:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9faca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b27a0 0x7f9fa81f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9faca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b27a0 0x7f9fa81f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47662/0 (socket says 172.21.15.62:47662) 2024-04-25T09:13:31.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.977+0000 7f9faca00640 1 -- 172.21.15.62:0/3031193388 learned_addr learned my addr 172.21.15.62:0/3031193388 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.978+0000 7f9faca00640 1 -- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fa80b56f0 msgr2=0x7f9fa81f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.978+0000 7f9faca00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fa80b56f0 0x7f9fa81f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.978+0000 7f9faca00640 1 -- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fa80b6a90 msgr2=0x7f9fa815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.978+0000 7f9fad400640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fa80b6a90 0x7f9fa815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.978+0000 7f9faca00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fa80b6a90 0x7f9fa815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.978+0000 7f9faca00640 1 -- 172.21.15.62:0/3031193388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80c8900 con 0x7f9fa80b27a0 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.978+0000 7f9fad400640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fa80b6a90 0x7f9fa815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.978+0000 7f9faca00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b27a0 0x7f9fa81f3b50 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f9f9802ac70 tx=0x7f9f98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.979+0000 7f9fa4c00640 1 -- 172.21.15.62:0/3031193388 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98040070 con 0x7f9fa80b27a0 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.979+0000 7f9fa4c00640 1 -- 172.21.15.62:0/3031193388 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f98031c40 con 0x7f9fa80b27a0 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.979+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa80c8be0 con 0x7f9fa80b27a0 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.979+0000 7f9fa4c00640 1 -- 172.21.15.62:0/3031193388 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98045460 con 0x7f9fa80b27a0 2024-04-25T09:13:31.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.979+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa80c91a0 con 0x7f9fa80b27a0 2024-04-25T09:13:31.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.980+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa80d26c0 con 0x7f9fa80b27a0 2024-04-25T09:13:31.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.981+0000 7f9fa4c00640 1 -- 172.21.15.62:0/3031193388 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f98039040 con 0x7f9fa80b27a0 2024-04-25T09:13:31.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.982+0000 7f9fa4c00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:31.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.982+0000 7f9fa4c00640 1 -- 172.21.15.62:0/3031193388 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f980abb90 con 0x7f9fa80b27a0 2024-04-25T09:13:31.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.983+0000 7f9fa7e00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:31.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.984+0000 7f9fa7e00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f8806b4d0 0x7f9f8806d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9f94007fd0 tx=0x7f9f94007960 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:31.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:31.986+0000 7f9fa4c00640 1 -- 172.21.15.62:0/3031193388 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fa80d26c0 con 0x7f9fa80b27a0 2024-04-25T09:13:32.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.187+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fa804ef20 con 0x7f9fa80b27a0 2024-04-25T09:13:32.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.189+0000 7f9fa4c00640 1 -- 172.21.15.62:0/3031193388 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f9f9807a970 con 0x7f9fa80b27a0 2024-04-25T09:13:32.177 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:32.177 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":108,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:13:31.219468+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:13:29.095930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:13:30.242813+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f8806b4d0 msgr2=0x7f9f8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f8806b4d0 0x7f9f8806d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9f94007fd0 tx=0x7f9f94007960 comp rx=0 tx=0).stop 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b27a0 msgr2=0x7f9fa81f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b27a0 0x7f9fa81f3b50 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f9f9802ac70 tx=0x7f9f98004a40 comp rx=0 tx=0).stop 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 shutdown_connections 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fa80b6a90 0x7f9fa815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fa80b56f0 0x7f9fa81f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 --2- 172.21.15.62:0/3031193388 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fa80b27a0 0x7f9fa81f3b50 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 >> 172.21.15.62:0/3031193388 conn(0x7f9fa8066d10 msgr2=0x7f9fa80b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:32.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 shutdown_connections 2024-04-25T09:13:32.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.192+0000 7f9fade00640 1 -- 172.21.15.62:0/3031193388 wait complete. 2024-04-25T09:13:32.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:32.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.421+0000 7f2c91800640 1 Processor -- start 2024-04-25T09:13:32.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.421+0000 7f2c91800640 1 -- start start 2024-04-25T09:13:32.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.423+0000 7f2c91800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c06d900 0x7f2c8c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.423+0000 7f2c91800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c8c0cee40 0x7f2c8c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.423+0000 7f2c91800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8c0cf2d0 con 0x7f2c8c144c10 2024-04-25T09:13:32.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.423+0000 7f2c91800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8c144ff0 con 0x7f2c8c0cee40 2024-04-25T09:13:32.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.423+0000 7f2c91800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c160340 con 0x7f2c8c06d900 2024-04-25T09:13:32.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.423+0000 7f2c8be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2c8c144c10 0x7f2c8c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60686/0 (socket says 172.21.15.62:60686) 2024-04-25T09:13:32.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.423+0000 7f2c8be00640 1 -- 172.21.15.62:0/2142682272 learned_addr learned my addr 172.21.15.62:0/2142682272 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8b400640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c06d900 0x7f2c8c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c90e00640 1 --2- 172.21.15.62:0/2142682272 >> v2:172.21.15.121:3300/0 conn(0x7f2c8c0cee40 0x7f2c8c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658129275 0 0) 0x7f2c8c0cf2d0 con 0x7f2c8c144c10 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c70003520 con 0x7f2c8c144c10 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754688285 0 0) 0x7f2c70003520 con 0x7f2c8c144c10 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c06d900 msgr2=0x7f2c8c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c06d900 0x7f2c8c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 >> v2:172.21.15.121:3300/0 conn(0x7f2c8c0cee40 msgr2=0x7f2c8c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 --2- 172.21.15.62:0/2142682272 >> v2:172.21.15.121:3300/0 conn(0x7f2c8c0cee40 0x7f2c8c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c15faa0 con 0x7f2c8c144c10 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f2c7c001670 con 0x7f2c8c144c10 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 >> v1:172.21.15.62:6789/0 conn(0x7f2c8c144c10 legacy=0x7f2c8c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70004800 0x7f2c70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c06d900 0x7f2c700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c144ff0 con 0x7f2c700039b0 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c160340 con 0x7f2c70004800 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.424+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7000b9f0 con 0x7f2c8c06d900 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.425+0000 7f2c8be00640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.425+0000 7f2c8be00640 1 -- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c06d900 msgr2=0x7f2c700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.425+0000 7f2c8be00640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c06d900 0x7f2c700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.425+0000 7f2c8be00640 1 -- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70004800 msgr2=0x7f2c70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.425+0000 7f2c8be00640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70004800 0x7f2c70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.425+0000 7f2c8be00640 1 -- 172.21.15.62:0/2142682272 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c70003520 con 0x7f2c700039b0 2024-04-25T09:13:32.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.426+0000 7f2c8be00640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c70003dd0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c240 tx=0x7f2c70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad2cfc461446920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:32.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.426+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c007d30 con 0x7f2c700039b0 2024-04-25T09:13:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c005ad0 con 0x7f2c700039b0 2024-04-25T09:13:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c8aa00640 1 -- 172.21.15.62:0/2142682272 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c007070 con 0x7f2c700039b0 2024-04-25T09:13:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c91800640 1 -- 172.21.15.62:0/2142682272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 msgr2=0x7f2c70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c91800640 1 --2- 172.21.15.62:0/2142682272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c70003dd0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c240 tx=0x7f2c70005830 comp rx=0 tx=0).stop 2024-04-25T09:13:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c90e00640 1 -- 172.21.15.62:0/2142682272 reap_dead start 2024-04-25T09:13:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c91800640 1 -- 172.21.15.62:0/2142682272 shutdown_connections 2024-04-25T09:13:32.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c91800640 1 -- 172.21.15.62:0/2142682272 >> 172.21.15.62:0/2142682272 conn(0x7f2c8c0b14d0 msgr2=0x7f2c8c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:32.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c91800640 1 -- 172.21.15.62:0/2142682272 shutdown_connections 2024-04-25T09:13:32.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.427+0000 7f2c91800640 1 -- 172.21.15.62:0/2142682272 wait complete. 2024-04-25T09:13:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.429+0000 7f2c91800640 1 Processor -- start 2024-04-25T09:13:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.429+0000 7f2c91800640 1 -- start start 2024-04-25T09:13:32.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.430+0000 7f2c91800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c8c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.430+0000 7f2c91800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c1430b0 0x7f2c8c0c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.430+0000 7f2c91800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8c0c6eb0 0x7f2c8c0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.430+0000 7f2c91800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c160060 con 0x7f2c700039b0 2024-04-25T09:13:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.430+0000 7f2c91800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0b41c0 con 0x7f2c8c0c6eb0 2024-04-25T09:13:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.430+0000 7f2c91800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c15f450 con 0x7f2c8c1430b0 2024-04-25T09:13:32.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.430+0000 7f2c8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c8c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c8c0cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47686/0 (socket says 172.21.15.62:47686) 2024-04-25T09:13:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8be00640 1 -- 172.21.15.62:0/2591816425 learned_addr learned my addr 172.21.15.62:0/2591816425 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8b400640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c1430b0 0x7f2c8c0c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c90e00640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8c0c6eb0 0x7f2c8c0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8be00640 1 -- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c1430b0 msgr2=0x7f2c8c0c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8be00640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c1430b0 0x7f2c8c0c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8be00640 1 -- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8c0c6eb0 msgr2=0x7f2c8c0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8be00640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8c0c6eb0 0x7f2c8c0c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8be00640 1 -- 172.21.15.62:0/2591816425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0c6a70 con 0x7f2c700039b0 2024-04-25T09:13:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.431+0000 7f2c8be00640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c8c0cb7f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c240 tx=0x7f2c8c15fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.432+0000 7f2c83e00640 1 -- 172.21.15.62:0/2591816425 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c042070 con 0x7f2c700039b0 2024-04-25T09:13:32.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.432+0000 7f2c83e00640 1 -- 172.21.15.62:0/2591816425 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c007480 con 0x7f2c700039b0 2024-04-25T09:13:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.432+0000 7f2c83e00640 1 -- 172.21.15.62:0/2591816425 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c001c70 con 0x7f2c700039b0 2024-04-25T09:13:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.432+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c143650 con 0x7f2c700039b0 2024-04-25T09:13:32.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.432+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c143c10 con 0x7f2c700039b0 2024-04-25T09:13:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.433+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8c04ef20 con 0x7f2c700039b0 2024-04-25T09:13:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.434+0000 7f2c83e00640 1 -- 172.21.15.62:0/2591816425 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c7c045070 con 0x7f2c700039b0 2024-04-25T09:13:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.435+0000 7f2c83e00640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c6c06b4d0 0x7f2c6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.435+0000 7f2c83e00640 1 -- 172.21.15.62:0/2591816425 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c7c0aafe0 con 0x7f2c700039b0 2024-04-25T09:13:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.436+0000 7f2c8b400640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c6c06b4d0 0x7f2c6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.437+0000 7f2c8b400640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c6c06b4d0 0x7f2c6c06d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0b4c60 tx=0x7f2c78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:32.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.440+0000 7f2c83e00640 1 -- 172.21.15.62:0/2591816425 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c7c04c030 con 0x7f2c700039b0 2024-04-25T09:13:32.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.643+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c8c0b22b0 con 0x7f2c700039b0 2024-04-25T09:13:32.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.646+0000 7f2c83e00640 1 -- 172.21.15.62:0/2591816425 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f2c7c03b340 con 0x7f2c700039b0 2024-04-25T09:13:32.635 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:32.635 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":108,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:13:31.219468+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:13:29.095930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:13:30.242813+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:32.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c6c06b4d0 msgr2=0x7f2c6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c6c06b4d0 0x7f2c6c06d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0b4c60 tx=0x7f2c78008040 comp rx=0 tx=0).stop 2024-04-25T09:13:32.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 msgr2=0x7f2c8c0cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c8c0cb7f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c240 tx=0x7f2c8c15fc70 comp rx=0 tx=0).stop 2024-04-25T09:13:32.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 shutdown_connections 2024-04-25T09:13:32.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2c6c06b4d0 0x7f2c6c06d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c8c0c6eb0 0x7f2c8c0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2c8c1430b0 0x7f2c8c0c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 --2- 172.21.15.62:0/2591816425 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c700039b0 0x7f2c8c0cb7f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 >> 172.21.15.62:0/2591816425 conn(0x7f2c8c0b14d0 msgr2=0x7f2c8c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:32.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.649+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 shutdown_connections 2024-04-25T09:13:32.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.650+0000 7f2c91800640 1 -- 172.21.15.62:0/2591816425 wait complete. 2024-04-25T09:13:32.658 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:13:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.892+0000 7fc52be00640 1 Processor -- start 2024-04-25T09:13:32.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.893+0000 7fc52be00640 1 -- start start 2024-04-25T09:13:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.894+0000 7fc52be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc52c0d2fb0 0x7fc52c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.894+0000 7fc52be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c0d3960 0x7fc52c0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.895+0000 7fc52be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c0ce780 con 0x7fc52c15dd50 2024-04-25T09:13:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.895+0000 7fc52be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc52c0d42f0 con 0x7fc52c0d2fb0 2024-04-25T09:13:32.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.895+0000 7fc52be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0d3440 con 0x7fc52c0d3960 2024-04-25T09:13:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.895+0000 7fc52b400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c0d3960 0x7fc52c0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.895+0000 7fc52aa00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc52c15dd50 0x7fc52c0d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:45310/0 (socket says 172.21.15.62:45310) 2024-04-25T09:13:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.895+0000 7fc52aa00640 1 -- 172.21.15.62:0/3377344156 learned_addr learned my addr 172.21.15.62:0/3377344156 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.895+0000 7fc52a000640 1 --2- 172.21.15.62:0/3377344156 >> v2:172.21.15.121:3300/0 conn(0x7fc52c0d2fb0 0x7fc52c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.896+0000 7fc52a000640 1 -- 172.21.15.62:0/3377344156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c0d3960 msgr2=0x7fc52c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.896+0000 7fc52a000640 1 --2- 172.21.15.62:0/3377344156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c0d3960 0x7fc52c0d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.896+0000 7fc52a000640 1 -- 172.21.15.62:0/3377344156 >> v1:172.21.15.121:6789/0 conn(0x7fc52c15dd50 legacy=0x7fc52c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.896+0000 7fc52a000640 1 -- 172.21.15.62:0/3377344156 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0ce1c0 con 0x7fc52c0d2fb0 2024-04-25T09:13:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.896+0000 7fc52b400640 1 --2- 172.21.15.62:0/3377344156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c0d3960 0x7fc52c0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:13:32.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.898+0000 7fc52a000640 1 --2- 172.21.15.62:0/3377344156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c0d2fb0 0x7fc52c0ce9c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc5140097a0 tx=0x7fc51402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a80bb4b491092e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:32.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.898+0000 7fc529600640 1 -- 172.21.15.62:0/3377344156 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514037070 con 0x7fc52c0d2fb0 2024-04-25T09:13:32.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.898+0000 7fc529600640 1 -- 172.21.15.62:0/3377344156 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc514032440 con 0x7fc52c0d2fb0 2024-04-25T09:13:32.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.898+0000 7fc529600640 1 -- 172.21.15.62:0/3377344156 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514032810 con 0x7fc52c0d2fb0 2024-04-25T09:13:32.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.899+0000 7fc52be00640 1 -- 172.21.15.62:0/3377344156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c0d2fb0 msgr2=0x7fc52c0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.899+0000 7fc52be00640 1 --2- 172.21.15.62:0/3377344156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c0d2fb0 0x7fc52c0ce9c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc5140097a0 tx=0x7fc51402a180 comp rx=0 tx=0).stop 2024-04-25T09:13:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.899+0000 7fc52be00640 1 -- 172.21.15.62:0/3377344156 shutdown_connections 2024-04-25T09:13:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.899+0000 7fc52be00640 1 --2- 172.21.15.62:0/3377344156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c0d3960 0x7fc52c0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.899+0000 7fc52be00640 1 --2- 172.21.15.62:0/3377344156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c0d2fb0 0x7fc52c0ce9c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.899+0000 7fc52be00640 1 -- 172.21.15.62:0/3377344156 >> 172.21.15.62:0/3377344156 conn(0x7fc52c06b000 msgr2=0x7fc52c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.899+0000 7fc52be00640 1 -- 172.21.15.62:0/3377344156 shutdown_connections 2024-04-25T09:13:32.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.899+0000 7fc52be00640 1 -- 172.21.15.62:0/3377344156 wait complete. 2024-04-25T09:13:32.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.901+0000 7fc52be00640 1 Processor -- start 2024-04-25T09:13:32.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.902+0000 7fc52be00640 1 -- start start 2024-04-25T09:13:32.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc52c0d3960 0x7fc52c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c15dd50 0x7fc52c1509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c151f00 0x7fc52c150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0b6b00 con 0x7fc52c0d3960 2024-04-25T09:13:32.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0cef00 con 0x7fc52c151f00 2024-04-25T09:13:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0b6c80 con 0x7fc52c15dd50 2024-04-25T09:13:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc52c0d3960 0x7fc52c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc52c0d3960 0x7fc52c15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47696/0 (socket says 172.21.15.62:47696) 2024-04-25T09:13:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.903+0000 7fc52aa00640 1 -- 172.21.15.62:0/1913572418 learned_addr learned my addr 172.21.15.62:0/1913572418 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.904+0000 7fc52a000640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c15dd50 0x7fc52c1509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.904+0000 7fc52b400640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c151f00 0x7fc52c150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.904+0000 7fc52aa00640 1 -- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c15dd50 msgr2=0x7fc52c1509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.904+0000 7fc52aa00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c15dd50 0x7fc52c1509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.904+0000 7fc52aa00640 1 -- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c151f00 msgr2=0x7fc52c150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.904+0000 7fc52aa00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c151f00 0x7fc52c150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.904+0000 7fc52aa00640 1 -- 172.21.15.62:0/1913572418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c151d00 con 0x7fc52c0d3960 2024-04-25T09:13:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.904+0000 7fc52aa00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc52c0d3960 0x7fc52c15a180 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00c3c0 tx=0x7fc51c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.905+0000 7fc522a00640 1 -- 172.21.15.62:0/1913572418 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c017070 con 0x7fc52c0d3960 2024-04-25T09:13:32.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.905+0000 7fc522a00640 1 -- 172.21.15.62:0/1913572418 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c005ce0 con 0x7fc52c0d3960 2024-04-25T09:13:32.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.905+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c0b3a90 con 0x7fc52c0d3960 2024-04-25T09:13:32.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.905+0000 7fc522a00640 1 -- 172.21.15.62:0/1913572418 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c0072f0 con 0x7fc52c0d3960 2024-04-25T09:13:32.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.905+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c0b4050 con 0x7fc52c0d3960 2024-04-25T09:13:32.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.906+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c0d0fa0 con 0x7fc52c0d3960 2024-04-25T09:13:32.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.907+0000 7fc522a00640 1 -- 172.21.15.62:0/1913572418 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc51c01a070 con 0x7fc52c0d3960 2024-04-25T09:13:32.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.908+0000 7fc522a00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc50806b500 0x7fc50806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:32.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.908+0000 7fc522a00640 1 -- 172.21.15.62:0/1913572418 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc51c089550 con 0x7fc52c0d3960 2024-04-25T09:13:32.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.909+0000 7fc52a000640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc50806b500 0x7fc50806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:32.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.910+0000 7fc52a000640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc50806b500 0x7fc50806d9c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc514034000 tx=0x7fc51402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:32.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:32.913+0000 7fc522a00640 1 -- 172.21.15.62:0/1913572418 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc52c0d0fa0 con 0x7fc52c0d3960 2024-04-25T09:13:33.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.154+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc52c0b9950 con 0x7fc52c0d3960 2024-04-25T09:13:33.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.757+0000 7fc522a00640 1 -- 172.21.15.62:0/1913572418 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc52c151d00 con 0x7fc52c0d3960 2024-04-25T09:13:33.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc50806b500 msgr2=0x7fc50806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:33.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc50806b500 0x7fc50806d9c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc514034000 tx=0x7fc51402e000 comp rx=0 tx=0).stop 2024-04-25T09:13:33.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc52c0d3960 msgr2=0x7fc52c15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc52c0d3960 0x7fc52c15a180 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00c3c0 tx=0x7fc51c00c880 comp rx=0 tx=0).stop 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 shutdown_connections 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc50806b500 0x7fc50806d9c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc52c151f00 0x7fc52c150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc52c15dd50 0x7fc52c1509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 --2- 172.21.15.62:0/1913572418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc52c0d3960 0x7fc52c15a180 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 >> 172.21.15.62:0/1913572418 conn(0x7fc52c06b000 msgr2=0x7fc52c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 shutdown_connections 2024-04-25T09:13:33.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:33.762+0000 7fc52be00640 1 -- 172.21.15.62:0/1913572418 wait complete. 2024-04-25T09:13:33.770 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:13:33.771 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:13:33.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.006+0000 7ff416800640 1 Processor -- start 2024-04-25T09:13:33.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.006+0000 7ff416800640 1 -- start start 2024-04-25T09:13:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.008+0000 7ff416800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff4100b54d0 0x7ff4100ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.008+0000 7ff416800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff4100ce7f0 0x7ff410164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:33.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.008+0000 7ff416800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4100b4430 con 0x7ff4100b2d80 2024-04-25T09:13:33.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.008+0000 7ff416800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4100b4710 con 0x7ff4100ce7f0 2024-04-25T09:13:33.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.008+0000 7ff416800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4100b49f0 con 0x7ff4100b54d0 2024-04-25T09:13:33.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.008+0000 7ff415e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff4100ce7f0 0x7ff410164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:33.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.008+0000 7ff415e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff4100ce7f0 0x7ff410164310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47708/0 (socket says 172.21.15.62:47708) 2024-04-25T09:13:33.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.008+0000 7ff415e00640 1 -- 172.21.15.62:0/3965730745 learned_addr learned my addr 172.21.15.62:0/3965730745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:33.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.009+0000 7ff415e00640 1 -- 172.21.15.62:0/3965730745 >> v2:172.21.15.121:3300/0 conn(0x7ff4100b54d0 msgr2=0x7ff4100ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:33.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.009+0000 7ff414a00640 1 --2- 172.21.15.62:0/3965730745 >> v2:172.21.15.121:3300/0 conn(0x7ff4100b54d0 0x7ff4100ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:33.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.009+0000 7ff415e00640 1 --2- 172.21.15.62:0/3965730745 >> v2:172.21.15.121:3300/0 conn(0x7ff4100b54d0 0x7ff4100ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:33.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.009+0000 7ff415e00640 1 -- 172.21.15.62:0/3965730745 >> v1:172.21.15.121:6789/0 conn(0x7ff4100b2d80 legacy=0x7ff4100b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:33.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.009+0000 7ff415e00640 1 -- 172.21.15.62:0/3965730745 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4100b4cd0 con 0x7ff4100ce7f0 2024-04-25T09:13:33.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.009+0000 7ff414a00640 1 --2- 172.21.15.62:0/3965730745 >> v2:172.21.15.121:3300/0 conn(0x7ff4100b54d0 0x7ff4100ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:33.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.010+0000 7ff415e00640 1 --2- 172.21.15.62:0/3965730745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ce7f0 0x7ff410164310 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff4100cd4c0 tx=0x7ff40802c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6417ade58cd663c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:33.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.010+0000 7ff40fe00640 1 -- 172.21.15.62:0/3965730745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40800b2d0 con 0x7ff4100ce7f0 2024-04-25T09:13:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.010+0000 7ff40fe00640 1 -- 172.21.15.62:0/3965730745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40800b470 con 0x7ff4100ce7f0 2024-04-25T09:13:33.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.010+0000 7ff40fe00640 1 -- 172.21.15.62:0/3965730745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408009820 con 0x7ff4100ce7f0 2024-04-25T09:13:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.011+0000 7ff416800640 1 -- 172.21.15.62:0/3965730745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ce7f0 msgr2=0x7ff410164310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.011+0000 7ff416800640 1 --2- 172.21.15.62:0/3965730745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ce7f0 0x7ff410164310 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff4100cd4c0 tx=0x7ff40802c8f0 comp rx=0 tx=0).stop 2024-04-25T09:13:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.011+0000 7ff416800640 1 -- 172.21.15.62:0/3965730745 shutdown_connections 2024-04-25T09:13:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.011+0000 7ff416800640 1 --2- 172.21.15.62:0/3965730745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100ce7f0 0x7ff410164310 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.011+0000 7ff416800640 1 --2- 172.21.15.62:0/3965730745 >> v2:172.21.15.121:3300/0 conn(0x7ff4100b54d0 0x7ff4100ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.011+0000 7ff416800640 1 -- 172.21.15.62:0/3965730745 >> 172.21.15.62:0/3965730745 conn(0x7ff410066f20 msgr2=0x7ff4100b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.011+0000 7ff416800640 1 -- 172.21.15.62:0/3965730745 shutdown_connections 2024-04-25T09:13:33.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.011+0000 7ff416800640 1 -- 172.21.15.62:0/3965730745 wait complete. 2024-04-25T09:13:34.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.013+0000 7ff416800640 1 Processor -- start 2024-04-25T09:13:34.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.013+0000 7ff416800640 1 -- start start 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff416800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4100b2d80 0x7ff4101513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff416800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100b54d0 0x7ff410146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff416800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4100ce7f0 0x7ff410150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff416800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100b3870 con 0x7ff4100b54d0 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff416800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4100b40e0 con 0x7ff4100b2d80 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff416800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41006d2c0 con 0x7ff4100ce7f0 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff414a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100b54d0 0x7ff410146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff414a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100b54d0 0x7ff410146060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47710/0 (socket says 172.21.15.62:47710) 2024-04-25T09:13:34.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff414a00640 1 -- 172.21.15.62:0/3727713792 learned_addr learned my addr 172.21.15.62:0/3727713792 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:34.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.014+0000 7ff415400640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4100b2d80 0x7ff4101513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:34.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff414a00640 1 -- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4100ce7f0 msgr2=0x7ff410150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:34.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff414a00640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4100ce7f0 0x7ff410150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff414a00640 1 -- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4100b2d80 msgr2=0x7ff4101513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:34.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff414a00640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4100b2d80 0x7ff4101513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff414a00640 1 -- 172.21.15.62:0/3727713792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410151040 con 0x7ff4100b54d0 2024-04-25T09:13:34.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff415400640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4100b2d80 0x7ff4101513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:34.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff414a00640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100b54d0 0x7ff410146060 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff40000c820 tx=0x7ff40000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:34.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff40d600640 1 -- 172.21.15.62:0/3727713792 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400015070 con 0x7ff4100b54d0 2024-04-25T09:13:34.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff40d600640 1 -- 172.21.15.62:0/3727713792 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff400007a30 con 0x7ff4100b54d0 2024-04-25T09:13:34.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff40d600640 1 -- 172.21.15.62:0/3727713792 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400007e00 con 0x7ff4100b54d0 2024-04-25T09:13:34.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4101519c0 con 0x7ff4100b54d0 2024-04-25T09:13:34.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.015+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4101f2b20 con 0x7ff4100b54d0 2024-04-25T09:13:34.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.017+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41004ef20 con 0x7ff4100b54d0 2024-04-25T09:13:34.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.018+0000 7ff40d600640 1 -- 172.21.15.62:0/3727713792 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff400018070 con 0x7ff4100b54d0 2024-04-25T09:13:34.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.018+0000 7ff40d600640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff3f406b4d0 0x7ff3f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.019+0000 7ff40d600640 1 -- 172.21.15.62:0/3727713792 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff40008a0b0 con 0x7ff4100b54d0 2024-04-25T09:13:34.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.019+0000 7ff415400640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff3f406b4d0 0x7ff3f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:34.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.020+0000 7ff415400640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff3f406b4d0 0x7ff3f406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff404003eb0 tx=0x7ff404009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:34.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.023+0000 7ff40d600640 1 -- 172.21.15.62:0/3727713792 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff400058fc0 con 0x7ff4100b54d0 2024-04-25T09:13:34.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.271+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff410158230 con 0x7ff4100b54d0 2024-04-25T09:13:34.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.771+0000 7ff40d600640 1 -- 172.21.15.62:0/3727713792 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff4000588c0 con 0x7ff4100b54d0 2024-04-25T09:13:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff3f406b4d0 msgr2=0x7ff3f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff3f406b4d0 0x7ff3f406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff404003eb0 tx=0x7ff404009040 comp rx=0 tx=0).stop 2024-04-25T09:13:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100b54d0 msgr2=0x7ff410146060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100b54d0 0x7ff410146060 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff40000c820 tx=0x7ff40000cc20 comp rx=0 tx=0).stop 2024-04-25T09:13:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 shutdown_connections 2024-04-25T09:13:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff3f406b4d0 0x7ff3f406d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4100b54d0 0x7ff410146060 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4100ce7f0 0x7ff410150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 --2- 172.21.15.62:0/3727713792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4100b2d80 0x7ff4101513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 >> 172.21.15.62:0/3727713792 conn(0x7ff410066f20 msgr2=0x7ff410070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 shutdown_connections 2024-04-25T09:13:34.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:34.775+0000 7ff416800640 1 -- 172.21.15.62:0/3727713792 wait complete. 2024-04-25T09:13:34.784 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:13:34.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:34.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.005+0000 7f5801800640 1 Processor -- start 2024-04-25T09:13:34.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.005+0000 7f5801800640 1 -- start start 2024-04-25T09:13:34.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.007+0000 7f5801800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57fc06d900 0x7f57fc15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.007+0000 7f5801800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f57fc0cee40 0x7f57fc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.007+0000 7f5801800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc0cf2d0 con 0x7f57fc144c10 2024-04-25T09:13:34.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.007+0000 7f5801800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc144ff0 con 0x7f57fc0cee40 2024-04-25T09:13:34.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.007+0000 7f5801800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc160340 con 0x7f57fc06d900 2024-04-25T09:13:34.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.007+0000 7f57fbe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f57fc144c10 0x7f57fc06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:55576/0 (socket says 172.21.15.62:55576) 2024-04-25T09:13:34.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.007+0000 7f57fbe00640 1 -- 172.21.15.62:0/2727939156 learned_addr learned my addr 172.21.15.62:0/2727939156 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57f3400640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57fc06d900 0x7f57fc15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f5800e00640 1 --2- 172.21.15.62:0/2727939156 >> v2:172.21.15.121:3300/0 conn(0x7f57fc0cee40 0x7f57fc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172745890 0 0) 0x7f57fc0cf2d0 con 0x7f57fc144c10 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e0003520 con 0x7f57fc144c10 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3222509863 0 0) 0x7f57e0003520 con 0x7f57fc144c10 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57fc06d900 msgr2=0x7f57fc15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57fc06d900 0x7f57fc15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 >> v2:172.21.15.121:3300/0 conn(0x7f57fc0cee40 msgr2=0x7f57fc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 --2- 172.21.15.62:0/2727939156 >> v2:172.21.15.121:3300/0 conn(0x7f57fc0cee40 0x7f57fc0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc15faa0 con 0x7f57fc144c10 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f57ec001670 con 0x7f57fc144c10 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 >> v1:172.21.15.62:6789/0 conn(0x7f57fc144c10 legacy=0x7f57fc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:34.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00039b0 0x7f57e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57e0004800 0x7f57e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57fc06d900 0x7f57e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc144ff0 con 0x7f57e00039b0 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc160340 con 0x7f57e0004800 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.008+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e000baa0 con 0x7f57fc06d900 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.009+0000 7f5800e00640 1 --2- 172.21.15.62:0/2727939156 >> v2:172.21.15.121:3300/0 conn(0x7f57fc0cee40 0x7f57fc0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.009+0000 7f57fbe00640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00039b0 0x7f57e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.009+0000 7f5800e00640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57fc06d900 0x7f57e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.009+0000 7f57fbe00640 1 -- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57fc06d900 msgr2=0x7f57e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.009+0000 7f57fbe00640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57fc06d900 0x7f57e00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.009+0000 7f57fbe00640 1 -- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57e0004800 msgr2=0x7f57e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.009+0000 7f57fbe00640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57e0004800 0x7f57e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:34.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.009+0000 7f57fbe00640 1 -- 172.21.15.62:0/2727939156 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0003520 con 0x7f57e00039b0 2024-04-25T09:13:34.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.011+0000 7f57fbe00640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00039b0 0x7f57e0003dd0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c210 tx=0x7f57e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87c464acbe831b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.011+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec007d30 con 0x7f57e00039b0 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.011+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec005af0 con 0x7f57e00039b0 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.011+0000 7f57fb400640 1 -- 172.21.15.62:0/2727939156 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec007070 con 0x7f57e00039b0 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.011+0000 7f5801800640 1 -- 172.21.15.62:0/2727939156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00039b0 msgr2=0x7f57e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.011+0000 7f5801800640 1 --2- 172.21.15.62:0/2727939156 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57e00039b0 0x7f57e0003dd0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c210 tx=0x7f57e0005830 comp rx=0 tx=0).stop 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.012+0000 7f5800e00640 1 -- 172.21.15.62:0/2727939156 reap_dead start 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.012+0000 7f5801800640 1 -- 172.21.15.62:0/2727939156 shutdown_connections 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.012+0000 7f5801800640 1 -- 172.21.15.62:0/2727939156 >> 172.21.15.62:0/2727939156 conn(0x7f57fc0b14d0 msgr2=0x7f57fc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.012+0000 7f5801800640 1 -- 172.21.15.62:0/2727939156 shutdown_connections 2024-04-25T09:13:35.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.012+0000 7f5801800640 1 -- 172.21.15.62:0/2727939156 wait complete. 2024-04-25T09:13:35.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.014+0000 7f5801800640 1 Processor -- start 2024-04-25T09:13:35.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.015+0000 7f5801800640 1 -- start start 2024-04-25T09:13:35.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5801800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57e00039b0 0x7f57fc141140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5801800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc15b2a0 0x7f57fc132860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5801800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57fc143b90 0x7f57fc141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5801800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc160060 con 0x7f57fc143b90 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5801800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0b41c0 con 0x7f57fc15b2a0 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5801800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc15f450 con 0x7f57e00039b0 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5800e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57fc143b90 0x7f57fc141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5800e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57fc143b90 0x7f57fc141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55764/0 (socket says 172.21.15.62:55764) 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.016+0000 7f5800e00640 1 -- 172.21.15.62:0/2929586632 learned_addr learned my addr 172.21.15.62:0/2929586632 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f57f3400640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc15b2a0 0x7f57fc132860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f57fbe00640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57e00039b0 0x7f57fc141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f5800e00640 1 -- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57e00039b0 msgr2=0x7f57fc141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f5800e00640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57e00039b0 0x7f57fc141140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f5800e00640 1 -- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc15b2a0 msgr2=0x7f57fc132860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f5800e00640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc15b2a0 0x7f57fc132860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f5800e00640 1 -- 172.21.15.62:0/2929586632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc15b720 con 0x7f57fc143b90 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f57fbe00640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57e00039b0 0x7f57fc141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:35.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f5800e00640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57fc143b90 0x7f57fc141ba0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f57f40043b0 tx=0x7f57f4004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f57f8c00640 1 -- 172.21.15.62:0/2929586632 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4019070 con 0x7f57fc143b90 2024-04-25T09:13:35.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f57f8c00640 1 -- 172.21.15.62:0/2929586632 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f4007d80 con 0x7f57fc143b90 2024-04-25T09:13:35.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.017+0000 7f57f8c00640 1 -- 172.21.15.62:0/2929586632 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4014640 con 0x7f57fc143b90 2024-04-25T09:13:35.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.018+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc0cc8d0 con 0x7f57fc143b90 2024-04-25T09:13:35.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.018+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc0cce90 con 0x7f57fc143b90 2024-04-25T09:13:35.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.019+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc04ef20 con 0x7f57fc143b90 2024-04-25T09:13:35.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.020+0000 7f57f8c00640 1 -- 172.21.15.62:0/2929586632 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57f401c070 con 0x7f57fc143b90 2024-04-25T09:13:35.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.021+0000 7f57f8c00640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d806b4d0 0x7f57d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.021+0000 7f57f8c00640 1 -- 172.21.15.62:0/2929586632 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f4089df0 con 0x7f57fc143b90 2024-04-25T09:13:35.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.021+0000 7f57fbe00640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d806b4d0 0x7f57d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.023+0000 7f57fbe00640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d806b4d0 0x7f57d806d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57ec032000 tx=0x7f57ec009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.025+0000 7f57f8c00640 1 -- 172.21.15.62:0/2929586632 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57f4058b50 con 0x7f57fc143b90 2024-04-25T09:13:35.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.261+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57fc0b22b0 con 0x7f57fc143b90 2024-04-25T09:13:35.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.263+0000 7f57f8c00640 1 -- 172.21.15.62:0/2929586632 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f57f4058450 con 0x7f57fc143b90 2024-04-25T09:13:35.250 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:35.251 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:13:35.251 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d806b4d0 msgr2=0x7f57d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d806b4d0 0x7f57d806d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57ec032000 tx=0x7f57ec009030 comp rx=0 tx=0).stop 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57fc143b90 msgr2=0x7f57fc141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57fc143b90 0x7f57fc141ba0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f57f40043b0 tx=0x7f57f4004870 comp rx=0 tx=0).stop 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 shutdown_connections 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f57d806b4d0 0x7f57d806d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57fc143b90 0x7f57fc141ba0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57fc15b2a0 0x7f57fc132860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 --2- 172.21.15.62:0/2929586632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57e00039b0 0x7f57fc141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.265+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 >> 172.21.15.62:0/2929586632 conn(0x7f57fc0b14d0 msgr2=0x7f57fc15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.266+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 shutdown_connections 2024-04-25T09:13:35.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.266+0000 7f5801800640 1 -- 172.21.15.62:0/2929586632 wait complete. 2024-04-25T09:13:35.272 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:13:35.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.485+0000 7fbbe3200640 1 Processor -- start 2024-04-25T09:13:35.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.485+0000 7fbbe3200640 1 -- start start 2024-04-25T09:13:35.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.486+0000 7fbbe3200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbbe413d970 0x7fbbe40b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.486+0000 7fbbe3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe415b220 con 0x7fbbe406d900 2024-04-25T09:13:35.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.486+0000 7fbbe3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe415b500 con 0x7fbbe40b1050 2024-04-25T09:13:35.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.486+0000 7fbbe3200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe415b7e0 con 0x7fbbe413d970 2024-04-25T09:13:35.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.487+0000 7fbbe2800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbbe406d900 0x7fbbe413e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:55588/0 (socket says 172.21.15.62:55588) 2024-04-25T09:13:35.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.487+0000 7fbbe2800640 1 -- 172.21.15.62:0/1987496846 learned_addr learned my addr 172.21.15.62:0/1987496846 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:35.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.487+0000 7fbbe1e00640 1 --2- 172.21.15.62:0/1987496846 >> v2:172.21.15.62:3300/0 conn(0x7fbbe413d970 0x7fbbe40b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.487+0000 7fbbe1400640 1 -- 172.21.15.62:0/1987496846 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981463914 0 0) 0x7fbbe415b220 con 0x7fbbe406d900 2024-04-25T09:13:35.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.487+0000 7fbbe1400640 1 -- 172.21.15.62:0/1987496846 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc8005730 con 0x7fbbe406d900 2024-04-25T09:13:35.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.487+0000 7fbbe1e00640 1 -- 172.21.15.62:0/1987496846 >> v1:172.21.15.121:6789/0 conn(0x7fbbe40b1050 legacy=0x7fbbe406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:35.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.487+0000 7fbbe1e00640 1 -- 172.21.15.62:0/1987496846 >> v1:172.21.15.62:6789/0 conn(0x7fbbe406d900 legacy=0x7fbbe413e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.487+0000 7fbbe1e00640 1 -- 172.21.15.62:0/1987496846 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe415bac0 con 0x7fbbe413d970 2024-04-25T09:13:35.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.488+0000 7fbbe1e00640 1 --2- 172.21.15.62:0/1987496846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe413d970 0x7fbbe40b0b10 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40097a0 tx=0x7fbbd402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adaf78ecffc26b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.488+0000 7fbbe1400640 1 -- 172.21.15.62:0/1987496846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4037070 con 0x7fbbe413d970 2024-04-25T09:13:35.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe1400640 1 -- 172.21.15.62:0/1987496846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4032440 con 0x7fbbe413d970 2024-04-25T09:13:35.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe1400640 1 -- 172.21.15.62:0/1987496846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4032810 con 0x7fbbe413d970 2024-04-25T09:13:35.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe3200640 1 -- 172.21.15.62:0/1987496846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe413d970 msgr2=0x7fbbe40b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe3200640 1 --2- 172.21.15.62:0/1987496846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe413d970 0x7fbbe40b0b10 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40097a0 tx=0x7fbbd402a230 comp rx=0 tx=0).stop 2024-04-25T09:13:35.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe3200640 1 -- 172.21.15.62:0/1987496846 shutdown_connections 2024-04-25T09:13:35.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe3200640 1 --2- 172.21.15.62:0/1987496846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe413d970 0x7fbbe40b0b10 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe3200640 1 -- 172.21.15.62:0/1987496846 >> 172.21.15.62:0/1987496846 conn(0x7fbbe40b14d0 msgr2=0x7fbbe40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:35.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe3200640 1 -- 172.21.15.62:0/1987496846 shutdown_connections 2024-04-25T09:13:35.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.489+0000 7fbbe3200640 1 -- 172.21.15.62:0/1987496846 wait complete. 2024-04-25T09:13:35.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.491+0000 7fbbe3200640 1 Processor -- start 2024-04-25T09:13:35.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.491+0000 7fbbe3200640 1 -- start start 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe406d900 0x7fbbe41f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbbe40b1050 0x7fbbe41f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe413d970 0x7fbbe40b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe413e960 con 0x7fbbe406d900 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe415af80 con 0x7fbbe413d970 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe413eae0 con 0x7fbbe40b1050 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe406d900 0x7fbbe41f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe406d900 0x7fbbe41f7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55780/0 (socket says 172.21.15.62:55780) 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe1e00640 1 -- 172.21.15.62:0/750250271 learned_addr learned my addr 172.21.15.62:0/750250271 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe1e00640 1 -- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbbe40b1050 msgr2=0x7fbbe41f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe1e00640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbbe40b1050 0x7fbbe41f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe1e00640 1 -- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe413d970 msgr2=0x7fbbe40b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe1e00640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe413d970 0x7fbbe40b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.492+0000 7fbbe1e00640 1 -- 172.21.15.62:0/750250271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe41f4110 con 0x7fbbe406d900 2024-04-25T09:13:35.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.493+0000 7fbbe1e00640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe406d900 0x7fbbe41f7a00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4035040 tx=0x7fbbd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.493+0000 7fbbdaa00640 1 -- 172.21.15.62:0/750250271 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4037040 con 0x7fbbe406d900 2024-04-25T09:13:35.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.493+0000 7fbbdaa00640 1 -- 172.21.15.62:0/750250271 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4030500 con 0x7fbbe406d900 2024-04-25T09:13:35.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.493+0000 7fbbdaa00640 1 -- 172.21.15.62:0/750250271 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4030910 con 0x7fbbe406d900 2024-04-25T09:13:35.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.493+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe41f43f0 con 0x7fbbe406d900 2024-04-25T09:13:35.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.493+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe40b4450 con 0x7fbbe406d900 2024-04-25T09:13:35.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.494+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe404ef20 con 0x7fbbe406d900 2024-04-25T09:13:35.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.495+0000 7fbbdaa00640 1 -- 172.21.15.62:0/750250271 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbd4044020 con 0x7fbbe406d900 2024-04-25T09:13:35.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.496+0000 7fbbdaa00640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.496+0000 7fbbdaa00640 1 -- 172.21.15.62:0/750250271 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbd40aa510 con 0x7fbbe406d900 2024-04-25T09:13:35.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.497+0000 7fbbd9400640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.498+0000 7fbbd9400640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbbc006b4d0 0x7fbbc006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0012e0 tx=0x7fbbcc007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.501+0000 7fbbdaa00640 1 -- 172.21.15.62:0/750250271 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbd4079380 con 0x7fbbe406d900 2024-04-25T09:13:35.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.698+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbe40cec60 con 0x7fbbe406d900 2024-04-25T09:13:35.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.700+0000 7fbbdaa00640 1 -- 172.21.15.62:0/750250271 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16607 (secure 0 0 0) 0x7fbbd4078c80 con 0x7fbbe406d900 2024-04-25T09:13:35.688 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:35.688 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":109,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:13:32.725071+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:13:29.095930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:13:30.242813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:13:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.702+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbbc006b4d0 msgr2=0x7fbbc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.702+0000 7fbbe3200640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbbc006b4d0 0x7fbbc006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0012e0 tx=0x7fbbcc007d50 comp rx=0 tx=0).stop 2024-04-25T09:13:35.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.702+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe406d900 msgr2=0x7fbbe41f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.702+0000 7fbbe3200640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe406d900 0x7fbbe41f7a00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4035040 tx=0x7fbbd40046c0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.703+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 shutdown_connections 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.703+0000 7fbbe3200640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbbc006b4d0 0x7fbbc006d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.703+0000 7fbbe3200640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbbe413d970 0x7fbbe40b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.703+0000 7fbbe3200640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbbe40b1050 0x7fbbe41f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.703+0000 7fbbe3200640 1 --2- 172.21.15.62:0/750250271 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbbe406d900 0x7fbbe41f7a00 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.703+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 >> 172.21.15.62:0/750250271 conn(0x7fbbe40b14d0 msgr2=0x7fbbe40708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.703+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 shutdown_connections 2024-04-25T09:13:35.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.703+0000 7fbbe3200640 1 -- 172.21.15.62:0/750250271 wait complete. 2024-04-25T09:13:35.708 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:13:35.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:13:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.927+0000 7fef40a00640 1 Processor -- start 2024-04-25T09:13:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.927+0000 7fef40a00640 1 -- start start 2024-04-25T09:13:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.929+0000 7fef40a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef3c0cfed0 0x7fef3c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.929+0000 7fef40a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3c06c310 con 0x7fef3c0709c0 2024-04-25T09:13:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.929+0000 7fef40a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3c15f5f0 con 0x7fef3c072340 2024-04-25T09:13:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.929+0000 7fef40a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c06c5b0 con 0x7fef3c0cfed0 2024-04-25T09:13:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.929+0000 7fef3a800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fef3c0709c0 0x7fef3c0b2880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:55592/0 (socket says 172.21.15.62:55592) 2024-04-25T09:13:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.929+0000 7fef3a800640 1 -- 172.21.15.62:0/2826442063 learned_addr learned my addr 172.21.15.62:0/2826442063 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef39e00640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef3c0cfed0 0x7fef3c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210818295 0 0) 0x7fef3c06c310 con 0x7fef3c0709c0 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef20005730 con 0x7fef3c0709c0 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1436962276 0 0) 0x7fef20005730 con 0x7fef3c0709c0 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef3c0cfed0 msgr2=0x7fef3c15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef3c0cfed0 0x7fef3c15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 >> v1:172.21.15.121:6789/0 conn(0x7fef3c072340 legacy=0x7fef3c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c06c890 con 0x7fef3c0709c0 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fef34001a10 con 0x7fef3c0709c0 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 >> v1:172.21.15.62:6789/0 conn(0x7fef3c0709c0 legacy=0x7fef3c0b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef3c0cfed0 0x7fef20005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef200061c0 0x7fef20006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.930+0000 7fef38a00640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef20006f50 0x7fef200073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c06c5b0 con 0x7fef200061c0 2024-04-25T09:13:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2000dd10 con 0x7fef20006f50 2024-04-25T09:13:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2000df10 con 0x7fef3c0cfed0 2024-04-25T09:13:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef39400640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef200061c0 0x7fef20006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef39e00640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef3c0cfed0 0x7fef20005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef39400640 1 -- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef3c0cfed0 msgr2=0x7fef20005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef39400640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef3c0cfed0 0x7fef20005bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef39400640 1 -- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef20006f50 msgr2=0x7fef200073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef39400640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef20006f50 0x7fef200073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.931+0000 7fef39400640 1 -- 172.21.15.62:0/2826442063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20005730 con 0x7fef200061c0 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.932+0000 7fef39400640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef200061c0 0x7fef20006a10 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fef28001750 tx=0x7fef20007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41150d369605d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.932+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28001370 con 0x7fef200061c0 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef280052d0 con 0x7fef200061c0 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef38a00640 1 -- 172.21.15.62:0/2826442063 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2802ab20 con 0x7fef200061c0 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef40a00640 1 -- 172.21.15.62:0/2826442063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef200061c0 msgr2=0x7fef20006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef40a00640 1 --2- 172.21.15.62:0/2826442063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef200061c0 0x7fef20006a10 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fef28001750 tx=0x7fef20007a40 comp rx=0 tx=0).stop 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef3a800640 1 -- 172.21.15.62:0/2826442063 reap_dead start 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef40a00640 1 -- 172.21.15.62:0/2826442063 shutdown_connections 2024-04-25T09:13:35.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef40a00640 1 -- 172.21.15.62:0/2826442063 >> 172.21.15.62:0/2826442063 conn(0x7fef3c0668f0 msgr2=0x7fef3c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef40a00640 1 -- 172.21.15.62:0/2826442063 shutdown_connections 2024-04-25T09:13:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.933+0000 7fef40a00640 1 -- 172.21.15.62:0/2826442063 wait complete. 2024-04-25T09:13:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.935+0000 7fef40a00640 1 Processor -- start 2024-04-25T09:13:35.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.935+0000 7fef40a00640 1 -- start start 2024-04-25T09:13:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.936+0000 7fef40a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef200061c0 0x7fef3c0c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.936+0000 7fef40a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef3c1349c0 0x7fef3c134e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.936+0000 7fef40a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef3c0ad8e0 0x7fef3c0add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.936+0000 7fef40a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c06d130 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.936+0000 7fef40a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c06ce50 con 0x7fef3c1349c0 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.936+0000 7fef40a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c15f0b0 con 0x7fef200061c0 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef3c0ad8e0 0x7fef3c0add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef3c0ad8e0 0x7fef3c0add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55790/0 (socket says 172.21.15.62:55790) 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 -- 172.21.15.62:0/2887570016 learned_addr learned my addr 172.21.15.62:0/2887570016 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef39e00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef200061c0 0x7fef3c0c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef39400640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef3c1349c0 0x7fef3c134e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 -- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef200061c0 msgr2=0x7fef3c0c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef200061c0 0x7fef3c0c4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 -- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef3c1349c0 msgr2=0x7fef3c134e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef3c1349c0 0x7fef3c134e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 -- 172.21.15.62:0/2887570016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c135530 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.937+0000 7fef3a800640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef3c0ad8e0 0x7fef3c0add60 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0cd470 tx=0x7fef3400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.938+0000 7fef32000640 1 -- 172.21.15.62:0/2887570016 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3400a5b0 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.938+0000 7fef32000640 1 -- 172.21.15.62:0/2887570016 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3400a770 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.938+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c135810 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.938+0000 7fef32000640 1 -- 172.21.15.62:0/2887570016 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3400ab60 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.938+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c132c10 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.939+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c0ae2a0 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.940+0000 7fef32000640 1 -- 172.21.15.62:0/2887570016 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef3400ad00 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.941+0000 7fef32000640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fef1c06b4d0 0x7fef1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:35.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.941+0000 7fef32000640 1 -- 172.21.15.62:0/2887570016 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef34089b20 con 0x7fef3c0ad8e0 2024-04-25T09:13:35.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.942+0000 7fef39e00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fef1c06b4d0 0x7fef1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:35.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.943+0000 7fef39e00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fef1c06b4d0 0x7fef1c06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009910 tx=0x7fef2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:35.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:35.945+0000 7fef32000640 1 -- 172.21.15.62:0/2887570016 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef34058900 con 0x7fef3c0ad8e0 2024-04-25T09:13:36.116 INFO:tasks.ceph.mds.d.smithi121.stderr:2024-04-25T09:13:36.129+0000 7f394e800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:13:36.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.145+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef3c072c70 con 0x7fef1c06b4d0 2024-04-25T09:13:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.165+0000 7fef32000640 1 -- 172.21.15.62:0/2887570016 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7fef3c072c70 con 0x7fef1c06b4d0 2024-04-25T09:13:36.156 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fef1c06b4d0 msgr2=0x7fef1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fef1c06b4d0 0x7fef1c06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009910 tx=0x7fef2c008040 comp rx=0 tx=0).stop 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef3c0ad8e0 msgr2=0x7fef3c0add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef3c0ad8e0 0x7fef3c0add60 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0cd470 tx=0x7fef3400ccd0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 shutdown_connections 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fef1c06b4d0 0x7fef1c06d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fef3c0ad8e0 0x7fef3c0add60 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef3c1349c0 0x7fef3c134e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 --2- 172.21.15.62:0/2887570016 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fef200061c0 0x7fef3c0c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.170+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 >> 172.21.15.62:0/2887570016 conn(0x7fef3c0668f0 msgr2=0x7fef3c143010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.171+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 shutdown_connections 2024-04-25T09:13:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.171+0000 7fef40a00640 1 -- 172.21.15.62:0/2887570016 wait complete. 2024-04-25T09:13:36.178 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-04-25T09:13:36.122286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422240,"kb_used_data":11752,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754552480,"statfs":{"total":773094113280,"available":772661739520,"internally_reserved":0,"allocated":12034048,"data_stored":5903252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001886"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737972+0000","last_change":"2024-04-25T09:13:31.766948+0000","last_active":"2024-04-25T09:13:32.737972+0000","last_peered":"2024-04-25T09:13:32.737972+0000","last_clean":"2024-04-25T09:13:32.737972+0000","last_became_active":"2024-04-25T09:13:31.766197+0000","last_became_peered":"2024-04-25T09:13:31.766197+0000","last_unstale":"2024-04-25T09:13:32.737972+0000","last_undegraded":"2024-04-25T09:13:32.737972+0000","last_fullsized":"2024-04-25T09:13:32.737972+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:36.087719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737522+0000","last_change":"2024-04-25T09:13:31.768097+0000","last_active":"2024-04-25T09:13:32.737522+0000","last_peered":"2024-04-25T09:13:32.737522+0000","last_clean":"2024-04-25T09:13:32.737522+0000","last_became_active":"2024-04-25T09:13:31.767760+0000","last_became_peered":"2024-04-25T09:13:31.767760+0000","last_unstale":"2024-04-25T09:13:32.737522+0000","last_undegraded":"2024-04-25T09:13:32.737522+0000","last_fullsized":"2024-04-25T09:13:32.737522+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:35.910056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777397+0000","last_change":"2024-04-25T09:13:31.751466+0000","last_active":"2024-04-25T09:13:32.777397+0000","last_peered":"2024-04-25T09:13:32.777397+0000","last_clean":"2024-04-25T09:13:32.777397+0000","last_became_active":"2024-04-25T09:13:31.750906+0000","last_became_peered":"2024-04-25T09:13:31.750906+0000","last_unstale":"2024-04-25T09:13:32.777397+0000","last_undegraded":"2024-04-25T09:13:32.777397+0000","last_fullsized":"2024-04-25T09:13:32.777397+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:40.512196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736566+0000","last_change":"2024-04-25T09:13:31.772818+0000","last_active":"2024-04-25T09:13:32.736566+0000","last_peered":"2024-04-25T09:13:32.736566+0000","last_clean":"2024-04-25T09:13:32.736566+0000","last_became_active":"2024-04-25T09:13:31.769664+0000","last_became_peered":"2024-04-25T09:13:31.769664+0000","last_unstale":"2024-04-25T09:13:32.736566+0000","last_undegraded":"2024-04-25T09:13:32.736566+0000","last_fullsized":"2024-04-25T09:13:32.736566+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:26.240304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680846+0000","last_change":"2024-04-25T09:13:31.758568+0000","last_active":"2024-04-25T09:13:33.680846+0000","last_peered":"2024-04-25T09:13:33.680846+0000","last_clean":"2024-04-25T09:13:33.680846+0000","last_became_active":"2024-04-25T09:13:31.758270+0000","last_became_peered":"2024-04-25T09:13:31.758270+0000","last_unstale":"2024-04-25T09:13:33.680846+0000","last_undegraded":"2024-04-25T09:13:33.680846+0000","last_fullsized":"2024-04-25T09:13:33.680846+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:43.319815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681446+0000","last_change":"2024-04-25T09:13:31.766705+0000","last_active":"2024-04-25T09:13:33.681446+0000","last_peered":"2024-04-25T09:13:33.681446+0000","last_clean":"2024-04-25T09:13:33.681446+0000","last_became_active":"2024-04-25T09:13:31.765788+0000","last_became_peered":"2024-04-25T09:13:31.765788+0000","last_unstale":"2024-04-25T09:13:33.681446+0000","last_undegraded":"2024-04-25T09:13:33.681446+0000","last_fullsized":"2024-04-25T09:13:33.681446+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:57.687898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680682+0000","last_change":"2024-04-25T09:13:31.766949+0000","last_active":"2024-04-25T09:13:33.680682+0000","last_peered":"2024-04-25T09:13:33.680682+0000","last_clean":"2024-04-25T09:13:33.680682+0000","last_became_active":"2024-04-25T09:13:31.766588+0000","last_became_peered":"2024-04-25T09:13:31.766588+0000","last_unstale":"2024-04-25T09:13:33.680682+0000","last_undegraded":"2024-04-25T09:13:33.680682+0000","last_fullsized":"2024-04-25T09:13:33.680682+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:48.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776376+0000","last_change":"2024-04-25T09:13:31.753196+0000","last_active":"2024-04-25T09:13:32.776376+0000","last_peered":"2024-04-25T09:13:32.776376+0000","last_clean":"2024-04-25T09:13:32.776376+0000","last_became_active":"2024-04-25T09:13:31.752817+0000","last_became_peered":"2024-04-25T09:13:31.752817+0000","last_unstale":"2024-04-25T09:13:32.776376+0000","last_undegraded":"2024-04-25T09:13:32.776376+0000","last_fullsized":"2024-04-25T09:13:32.776376+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:25.286953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682752+0000","last_change":"2024-04-25T09:13:31.763832+0000","last_active":"2024-04-25T09:13:33.682752+0000","last_peered":"2024-04-25T09:13:33.682752+0000","last_clean":"2024-04-25T09:13:33.682752+0000","last_became_active":"2024-04-25T09:13:31.763375+0000","last_became_peered":"2024-04-25T09:13:31.763375+0000","last_unstale":"2024-04-25T09:13:33.682752+0000","last_undegraded":"2024-04-25T09:13:33.682752+0000","last_fullsized":"2024-04-25T09:13:33.682752+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:33.121123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680441+0000","last_change":"2024-04-25T09:13:31.771696+0000","last_active":"2024-04-25T09:13:33.680441+0000","last_peered":"2024-04-25T09:13:33.680441+0000","last_clean":"2024-04-25T09:13:33.680441+0000","last_became_active":"2024-04-25T09:13:31.771358+0000","last_became_peered":"2024-04-25T09:13:31.771358+0000","last_unstale":"2024-04-25T09:13:33.680441+0000","last_undegraded":"2024-04-25T09:13:33.680441+0000","last_fullsized":"2024-04-25T09:13:33.680441+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:11.384981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681955+0000","last_change":"2024-04-25T09:13:31.767014+0000","last_active":"2024-04-25T09:13:33.681955+0000","last_peered":"2024-04-25T09:13:33.681955+0000","last_clean":"2024-04-25T09:13:33.681955+0000","last_became_active":"2024-04-25T09:13:31.766316+0000","last_became_peered":"2024-04-25T09:13:31.766316+0000","last_unstale":"2024-04-25T09:13:33.681955+0000","last_undegraded":"2024-04-25T09:13:33.681955+0000","last_fullsized":"2024-04-25T09:13:33.681955+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:50.418299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.679359+0000","last_change":"2024-04-25T09:13:31.768680+0000","last_active":"2024-04-25T09:13:33.679359+0000","last_peered":"2024-04-25T09:13:33.679359+0000","last_clean":"2024-04-25T09:13:33.679359+0000","last_became_active":"2024-04-25T09:13:31.768381+0000","last_became_peered":"2024-04-25T09:13:31.768381+0000","last_unstale":"2024-04-25T09:13:33.679359+0000","last_undegraded":"2024-04-25T09:13:33.679359+0000","last_fullsized":"2024-04-25T09:13:33.679359+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:53.808615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.677964+0000","last_change":"2024-04-25T09:13:31.766561+0000","last_active":"2024-04-25T09:13:33.677964+0000","last_peered":"2024-04-25T09:13:33.677964+0000","last_clean":"2024-04-25T09:13:33.677964+0000","last_became_active":"2024-04-25T09:13:31.766213+0000","last_became_peered":"2024-04-25T09:13:31.766213+0000","last_unstale":"2024-04-25T09:13:33.677964+0000","last_undegraded":"2024-04-25T09:13:33.677964+0000","last_fullsized":"2024-04-25T09:13:33.677964+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:40.387620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737209+0000","last_change":"2024-04-25T09:13:31.768089+0000","last_active":"2024-04-25T09:13:32.737209+0000","last_peered":"2024-04-25T09:13:32.737209+0000","last_clean":"2024-04-25T09:13:32.737209+0000","last_became_active":"2024-04-25T09:13:31.767781+0000","last_became_peered":"2024-04-25T09:13:31.767781+0000","last_unstale":"2024-04-25T09:13:32.737209+0000","last_undegraded":"2024-04-25T09:13:32.737209+0000","last_fullsized":"2024-04-25T09:13:32.737209+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:24.181807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735530+0000","last_change":"2024-04-25T09:13:31.762995+0000","last_active":"2024-04-25T09:13:32.735530+0000","last_peered":"2024-04-25T09:13:32.735530+0000","last_clean":"2024-04-25T09:13:32.735530+0000","last_became_active":"2024-04-25T09:13:31.762621+0000","last_became_peered":"2024-04-25T09:13:31.762621+0000","last_unstale":"2024-04-25T09:13:32.735530+0000","last_undegraded":"2024-04-25T09:13:32.735530+0000","last_fullsized":"2024-04-25T09:13:32.735530+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:10.795857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681623+0000","last_change":"2024-04-25T09:13:31.767738+0000","last_active":"2024-04-25T09:13:33.681623+0000","last_peered":"2024-04-25T09:13:33.681623+0000","last_clean":"2024-04-25T09:13:33.681623+0000","last_became_active":"2024-04-25T09:13:31.767369+0000","last_became_peered":"2024-04-25T09:13:31.767369+0000","last_unstale":"2024-04-25T09:13:33.681623+0000","last_undegraded":"2024-04-25T09:13:33.681623+0000","last_fullsized":"2024-04-25T09:13:33.681623+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:49.795922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771534+0000","last_change":"2024-04-25T09:13:31.766905+0000","last_active":"2024-04-25T09:13:32.771534+0000","last_peered":"2024-04-25T09:13:32.771534+0000","last_clean":"2024-04-25T09:13:32.771534+0000","last_became_active":"2024-04-25T09:13:31.766629+0000","last_became_peered":"2024-04-25T09:13:31.766629+0000","last_unstale":"2024-04-25T09:13:32.771534+0000","last_undegraded":"2024-04-25T09:13:32.771534+0000","last_fullsized":"2024-04-25T09:13:32.771534+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:05.050039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737667+0000","last_change":"2024-04-25T09:13:31.766511+0000","last_active":"2024-04-25T09:13:32.737667+0000","last_peered":"2024-04-25T09:13:32.737667+0000","last_clean":"2024-04-25T09:13:32.737667+0000","last_became_active":"2024-04-25T09:13:31.764825+0000","last_became_peered":"2024-04-25T09:13:31.764825+0000","last_unstale":"2024-04-25T09:13:32.737667+0000","last_undegraded":"2024-04-25T09:13:32.737667+0000","last_fullsized":"2024-04-25T09:13:32.737667+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:07.411063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771808+0000","last_change":"2024-04-25T09:13:31.769715+0000","last_active":"2024-04-25T09:13:32.771808+0000","last_peered":"2024-04-25T09:13:32.771808+0000","last_clean":"2024-04-25T09:13:32.771808+0000","last_became_active":"2024-04-25T09:13:31.769435+0000","last_became_peered":"2024-04-25T09:13:31.769435+0000","last_unstale":"2024-04-25T09:13:32.771808+0000","last_undegraded":"2024-04-25T09:13:32.771808+0000","last_fullsized":"2024-04-25T09:13:32.771808+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:45.085992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.679413+0000","last_change":"2024-04-25T09:13:31.759575+0000","last_active":"2024-04-25T09:13:33.679413+0000","last_peered":"2024-04-25T09:13:33.679413+0000","last_clean":"2024-04-25T09:13:33.679413+0000","last_became_active":"2024-04-25T09:13:31.759253+0000","last_became_peered":"2024-04-25T09:13:31.759253+0000","last_unstale":"2024-04-25T09:13:33.679413+0000","last_undegraded":"2024-04-25T09:13:33.679413+0000","last_fullsized":"2024-04-25T09:13:33.679413+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:37.452417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777125+0000","last_change":"2024-04-25T09:13:31.767268+0000","last_active":"2024-04-25T09:13:32.777125+0000","last_peered":"2024-04-25T09:13:32.777125+0000","last_clean":"2024-04-25T09:13:32.777125+0000","last_became_active":"2024-04-25T09:13:31.764073+0000","last_became_peered":"2024-04-25T09:13:31.764073+0000","last_unstale":"2024-04-25T09:13:32.777125+0000","last_undegraded":"2024-04-25T09:13:32.777125+0000","last_fullsized":"2024-04-25T09:13:32.777125+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:11.384981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776455+0000","last_change":"2024-04-25T09:13:31.767758+0000","last_active":"2024-04-25T09:13:32.776455+0000","last_peered":"2024-04-25T09:13:32.776455+0000","last_clean":"2024-04-25T09:13:32.776455+0000","last_became_active":"2024-04-25T09:13:31.767479+0000","last_became_peered":"2024-04-25T09:13:31.767479+0000","last_unstale":"2024-04-25T09:13:32.776455+0000","last_undegraded":"2024-04-25T09:13:32.776455+0000","last_fullsized":"2024-04-25T09:13:32.776455+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:11.242013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735532+0000","last_change":"2024-04-25T09:13:31.763797+0000","last_active":"2024-04-25T09:13:32.735532+0000","last_peered":"2024-04-25T09:13:32.735532+0000","last_clean":"2024-04-25T09:13:32.735532+0000","last_became_active":"2024-04-25T09:13:31.763451+0000","last_became_peered":"2024-04-25T09:13:31.763451+0000","last_unstale":"2024-04-25T09:13:32.735532+0000","last_undegraded":"2024-04-25T09:13:32.735532+0000","last_fullsized":"2024-04-25T09:13:32.735532+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:20.933503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731309+0000","last_change":"2024-04-25T09:13:31.772907+0000","last_active":"2024-04-25T09:13:32.731309+0000","last_peered":"2024-04-25T09:13:32.731309+0000","last_clean":"2024-04-25T09:13:32.731309+0000","last_became_active":"2024-04-25T09:13:31.772572+0000","last_became_peered":"2024-04-25T09:13:31.772572+0000","last_unstale":"2024-04-25T09:13:32.731309+0000","last_undegraded":"2024-04-25T09:13:32.731309+0000","last_fullsized":"2024-04-25T09:13:32.731309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:26.240304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680648+0000","last_change":"2024-04-25T09:13:31.774461+0000","last_active":"2024-04-25T09:13:33.680648+0000","last_peered":"2024-04-25T09:13:33.680648+0000","last_clean":"2024-04-25T09:13:33.680648+0000","last_became_active":"2024-04-25T09:13:31.774022+0000","last_became_peered":"2024-04-25T09:13:31.774022+0000","last_unstale":"2024-04-25T09:13:33.680648+0000","last_undegraded":"2024-04-25T09:13:33.680648+0000","last_fullsized":"2024-04-25T09:13:33.680648+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.507625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678408+0000","last_change":"2024-04-25T09:13:31.762091+0000","last_active":"2024-04-25T09:13:33.678408+0000","last_peered":"2024-04-25T09:13:33.678408+0000","last_clean":"2024-04-25T09:13:33.678408+0000","last_became_active":"2024-04-25T09:13:31.761782+0000","last_became_peered":"2024-04-25T09:13:31.761782+0000","last_unstale":"2024-04-25T09:13:33.678408+0000","last_undegraded":"2024-04-25T09:13:33.678408+0000","last_fullsized":"2024-04-25T09:13:33.678408+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:46.551687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731709+0000","last_change":"2024-04-25T09:13:31.761119+0000","last_active":"2024-04-25T09:13:32.731709+0000","last_peered":"2024-04-25T09:13:32.731709+0000","last_clean":"2024-04-25T09:13:32.731709+0000","last_became_active":"2024-04-25T09:13:31.760820+0000","last_became_peered":"2024-04-25T09:13:31.760820+0000","last_unstale":"2024-04-25T09:13:32.731709+0000","last_undegraded":"2024-04-25T09:13:32.731709+0000","last_fullsized":"2024-04-25T09:13:32.731709+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:35.910056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.732265+0000","last_change":"2024-04-25T09:13:31.756872+0000","last_active":"2024-04-25T09:13:32.732265+0000","last_peered":"2024-04-25T09:13:32.732265+0000","last_clean":"2024-04-25T09:13:32.732265+0000","last_became_active":"2024-04-25T09:13:31.756574+0000","last_became_peered":"2024-04-25T09:13:31.756574+0000","last_unstale":"2024-04-25T09:13:32.732265+0000","last_undegraded":"2024-04-25T09:13:32.732265+0000","last_fullsized":"2024-04-25T09:13:32.732265+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:41.959312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678083+0000","last_change":"2024-04-25T09:13:31.760937+0000","last_active":"2024-04-25T09:13:33.678083+0000","last_peered":"2024-04-25T09:13:33.678083+0000","last_clean":"2024-04-25T09:13:33.678083+0000","last_became_active":"2024-04-25T09:13:31.760250+0000","last_became_peered":"2024-04-25T09:13:31.760250+0000","last_unstale":"2024-04-25T09:13:33.678083+0000","last_undegraded":"2024-04-25T09:13:33.678083+0000","last_fullsized":"2024-04-25T09:13:33.678083+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:30.451727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678120+0000","last_change":"2024-04-25T09:13:31.768344+0000","last_active":"2024-04-25T09:13:33.678120+0000","last_peered":"2024-04-25T09:13:33.678120+0000","last_clean":"2024-04-25T09:13:33.678120+0000","last_became_active":"2024-04-25T09:13:31.768062+0000","last_became_peered":"2024-04-25T09:13:31.768062+0000","last_unstale":"2024-04-25T09:13:33.678120+0000","last_undegraded":"2024-04-25T09:13:33.678120+0000","last_fullsized":"2024-04-25T09:13:33.678120+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:42.623821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776951+0000","last_change":"2024-04-25T09:13:31.765940+0000","last_active":"2024-04-25T09:13:32.776951+0000","last_peered":"2024-04-25T09:13:32.776951+0000","last_clean":"2024-04-25T09:13:32.776951+0000","last_became_active":"2024-04-25T09:13:31.765656+0000","last_became_peered":"2024-04-25T09:13:31.765656+0000","last_unstale":"2024-04-25T09:13:32.776951+0000","last_undegraded":"2024-04-25T09:13:32.776951+0000","last_fullsized":"2024-04-25T09:13:32.776951+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:22.359448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735964+0000","last_change":"2024-04-25T09:13:31.781458+0000","last_active":"2024-04-25T09:13:32.735964+0000","last_peered":"2024-04-25T09:13:32.735964+0000","last_clean":"2024-04-25T09:13:32.735964+0000","last_became_active":"2024-04-25T09:13:31.780478+0000","last_became_peered":"2024-04-25T09:13:31.780478+0000","last_unstale":"2024-04-25T09:13:32.735964+0000","last_undegraded":"2024-04-25T09:13:32.735964+0000","last_fullsized":"2024-04-25T09:13:32.735964+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:57.232055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771202+0000","last_change":"2024-04-25T09:13:31.771544+0000","last_active":"2024-04-25T09:13:32.771202+0000","last_peered":"2024-04-25T09:13:32.771202+0000","last_clean":"2024-04-25T09:13:32.771202+0000","last_became_active":"2024-04-25T09:13:31.761287+0000","last_became_peered":"2024-04-25T09:13:31.761287+0000","last_unstale":"2024-04-25T09:13:32.771202+0000","last_undegraded":"2024-04-25T09:13:32.771202+0000","last_fullsized":"2024-04-25T09:13:32.771202+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:02.500416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678486+0000","last_change":"2024-04-25T09:13:30.753476+0000","last_active":"2024-04-25T09:13:33.678486+0000","last_peered":"2024-04-25T09:13:33.678486+0000","last_clean":"2024-04-25T09:13:33.678486+0000","last_became_active":"2024-04-25T09:13:30.751179+0000","last_became_peered":"2024-04-25T09:13:30.751179+0000","last_unstale":"2024-04-25T09:13:33.678486+0000","last_undegraded":"2024-04-25T09:13:33.678486+0000","last_fullsized":"2024-04-25T09:13:33.678486+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:12.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678578+0000","last_change":"2024-04-25T09:13:31.760469+0000","last_active":"2024-04-25T09:13:33.678578+0000","last_peered":"2024-04-25T09:13:33.678578+0000","last_clean":"2024-04-25T09:13:33.678578+0000","last_became_active":"2024-04-25T09:13:31.759945+0000","last_became_peered":"2024-04-25T09:13:31.759945+0000","last_unstale":"2024-04-25T09:13:33.678578+0000","last_undegraded":"2024-04-25T09:13:33.678578+0000","last_fullsized":"2024-04-25T09:13:33.678578+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.266138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681646+0000","last_change":"2024-04-25T09:13:30.769991+0000","last_active":"2024-04-25T09:13:33.681646+0000","last_peered":"2024-04-25T09:13:33.681646+0000","last_clean":"2024-04-25T09:13:33.681646+0000","last_became_active":"2024-04-25T09:13:30.768898+0000","last_became_peered":"2024-04-25T09:13:30.768898+0000","last_unstale":"2024-04-25T09:13:33.681646+0000","last_undegraded":"2024-04-25T09:13:33.681646+0000","last_fullsized":"2024-04-25T09:13:33.681646+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:24.945887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737038+0000","last_change":"2024-04-25T09:13:31.760603+0000","last_active":"2024-04-25T09:13:32.737038+0000","last_peered":"2024-04-25T09:13:32.737038+0000","last_clean":"2024-04-25T09:13:32.737038+0000","last_became_active":"2024-04-25T09:13:31.760288+0000","last_became_peered":"2024-04-25T09:13:31.760288+0000","last_unstale":"2024-04-25T09:13:32.737038+0000","last_undegraded":"2024-04-25T09:13:32.737038+0000","last_fullsized":"2024-04-25T09:13:32.737038+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.677224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777475+0000","last_change":"2024-04-25T09:13:30.767362+0000","last_active":"2024-04-25T09:13:32.777475+0000","last_peered":"2024-04-25T09:13:32.777475+0000","last_clean":"2024-04-25T09:13:32.777475+0000","last_became_active":"2024-04-25T09:13:30.767088+0000","last_became_peered":"2024-04-25T09:13:30.767088+0000","last_unstale":"2024-04-25T09:13:32.777475+0000","last_undegraded":"2024-04-25T09:13:32.777475+0000","last_fullsized":"2024-04-25T09:13:32.777475+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:39.369419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680337+0000","last_change":"2024-04-25T09:13:31.768124+0000","last_active":"2024-04-25T09:13:33.680337+0000","last_peered":"2024-04-25T09:13:33.680337+0000","last_clean":"2024-04-25T09:13:33.680337+0000","last_became_active":"2024-04-25T09:13:31.767797+0000","last_became_peered":"2024-04-25T09:13:31.767797+0000","last_unstale":"2024-04-25T09:13:33.680337+0000","last_undegraded":"2024-04-25T09:13:33.680337+0000","last_fullsized":"2024-04-25T09:13:33.680337+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:22.359448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736919+0000","last_change":"2024-04-25T09:13:30.766810+0000","last_active":"2024-04-25T09:13:32.736919+0000","last_peered":"2024-04-25T09:13:32.736919+0000","last_clean":"2024-04-25T09:13:32.736919+0000","last_became_active":"2024-04-25T09:13:30.766347+0000","last_became_peered":"2024-04-25T09:13:30.766347+0000","last_unstale":"2024-04-25T09:13:32.736919+0000","last_undegraded":"2024-04-25T09:13:32.736919+0000","last_fullsized":"2024-04-25T09:13:32.736919+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:56.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.679098+0000","last_change":"2024-04-25T09:13:31.765365+0000","last_active":"2024-04-25T09:13:33.679098+0000","last_peered":"2024-04-25T09:13:33.679098+0000","last_clean":"2024-04-25T09:13:33.679098+0000","last_became_active":"2024-04-25T09:13:31.765023+0000","last_became_peered":"2024-04-25T09:13:31.765023+0000","last_unstale":"2024-04-25T09:13:33.679098+0000","last_undegraded":"2024-04-25T09:13:33.679098+0000","last_fullsized":"2024-04-25T09:13:33.679098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:18.419728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735441+0000","last_change":"2024-04-25T09:13:30.763213+0000","last_active":"2024-04-25T09:13:32.735441+0000","last_peered":"2024-04-25T09:13:32.735441+0000","last_clean":"2024-04-25T09:13:32.735441+0000","last_became_active":"2024-04-25T09:13:30.762921+0000","last_became_peered":"2024-04-25T09:13:30.762921+0000","last_unstale":"2024-04-25T09:13:32.735441+0000","last_undegraded":"2024-04-25T09:13:32.735441+0000","last_fullsized":"2024-04-25T09:13:32.735441+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:17.401527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681033+0000","last_change":"2024-04-25T09:13:31.766073+0000","last_active":"2024-04-25T09:13:33.681033+0000","last_peered":"2024-04-25T09:13:33.681033+0000","last_clean":"2024-04-25T09:13:33.681033+0000","last_became_active":"2024-04-25T09:13:31.764318+0000","last_became_peered":"2024-04-25T09:13:31.764318+0000","last_unstale":"2024-04-25T09:13:33.681033+0000","last_undegraded":"2024-04-25T09:13:33.681033+0000","last_fullsized":"2024-04-25T09:13:33.681033+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:24.790304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776452+0000","last_change":"2024-04-25T09:13:30.761921+0000","last_active":"2024-04-25T09:13:32.776452+0000","last_peered":"2024-04-25T09:13:32.776452+0000","last_clean":"2024-04-25T09:13:32.776452+0000","last_became_active":"2024-04-25T09:13:30.761545+0000","last_became_peered":"2024-04-25T09:13:30.761545+0000","last_unstale":"2024-04-25T09:13:32.776452+0000","last_undegraded":"2024-04-25T09:13:32.776452+0000","last_fullsized":"2024-04-25T09:13:32.776452+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:11.237418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.677630+0000","last_change":"2024-04-25T09:13:31.752402+0000","last_active":"2024-04-25T09:13:33.677630+0000","last_peered":"2024-04-25T09:13:33.677630+0000","last_clean":"2024-04-25T09:13:33.677630+0000","last_became_active":"2024-04-25T09:13:31.751890+0000","last_became_peered":"2024-04-25T09:13:31.751890+0000","last_unstale":"2024-04-25T09:13:33.677630+0000","last_undegraded":"2024-04-25T09:13:33.677630+0000","last_fullsized":"2024-04-25T09:13:33.677630+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:12.255619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680990+0000","last_change":"2024-04-25T09:13:30.757062+0000","last_active":"2024-04-25T09:13:33.680990+0000","last_peered":"2024-04-25T09:13:33.680990+0000","last_clean":"2024-04-25T09:13:33.680990+0000","last_became_active":"2024-04-25T09:13:30.756455+0000","last_became_peered":"2024-04-25T09:13:30.756455+0000","last_unstale":"2024-04-25T09:13:33.680990+0000","last_undegraded":"2024-04-25T09:13:33.680990+0000","last_fullsized":"2024-04-25T09:13:33.680990+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:11.833303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678022+0000","last_change":"2024-04-25T09:13:31.769059+0000","last_active":"2024-04-25T09:13:33.678022+0000","last_peered":"2024-04-25T09:13:33.678022+0000","last_clean":"2024-04-25T09:13:33.678022+0000","last_became_active":"2024-04-25T09:13:31.768762+0000","last_became_peered":"2024-04-25T09:13:31.768762+0000","last_unstale":"2024-04-25T09:13:33.678022+0000","last_undegraded":"2024-04-25T09:13:33.678022+0000","last_fullsized":"2024-04-25T09:13:33.678022+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:46.551687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770923+0000","last_change":"2024-04-25T09:13:30.772848+0000","last_active":"2024-04-25T09:13:32.770923+0000","last_peered":"2024-04-25T09:13:32.770923+0000","last_clean":"2024-04-25T09:13:32.770923+0000","last_became_active":"2024-04-25T09:13:30.772209+0000","last_became_peered":"2024-04-25T09:13:30.772209+0000","last_unstale":"2024-04-25T09:13:32.770923+0000","last_undegraded":"2024-04-25T09:13:32.770923+0000","last_fullsized":"2024-04-25T09:13:32.770923+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:24.308630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737970+0000","last_change":"2024-04-25T09:13:31.779971+0000","last_active":"2024-04-25T09:13:32.737970+0000","last_peered":"2024-04-25T09:13:32.737970+0000","last_clean":"2024-04-25T09:13:32.737970+0000","last_became_active":"2024-04-25T09:13:31.779690+0000","last_became_peered":"2024-04-25T09:13:31.779690+0000","last_unstale":"2024-04-25T09:13:32.737970+0000","last_undegraded":"2024-04-25T09:13:32.737970+0000","last_fullsized":"2024-04-25T09:13:32.737970+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:51.743854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777822+0000","last_change":"2024-04-25T09:13:30.758834+0000","last_active":"2024-04-25T09:13:32.777822+0000","last_peered":"2024-04-25T09:13:32.777822+0000","last_clean":"2024-04-25T09:13:32.777822+0000","last_became_active":"2024-04-25T09:13:30.758446+0000","last_became_peered":"2024-04-25T09:13:30.758446+0000","last_unstale":"2024-04-25T09:13:32.777822+0000","last_undegraded":"2024-04-25T09:13:32.777822+0000","last_fullsized":"2024-04-25T09:13:32.777822+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:41.605620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731972+0000","last_change":"2024-04-25T09:13:31.760295+0000","last_active":"2024-04-25T09:13:32.731972+0000","last_peered":"2024-04-25T09:13:32.731972+0000","last_clean":"2024-04-25T09:13:32.731972+0000","last_became_active":"2024-04-25T09:13:31.759984+0000","last_became_peered":"2024-04-25T09:13:31.759984+0000","last_unstale":"2024-04-25T09:13:32.731972+0000","last_undegraded":"2024-04-25T09:13:32.731972+0000","last_fullsized":"2024-04-25T09:13:32.731972+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:36.087719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737920+0000","last_change":"2024-04-25T09:13:30.768482+0000","last_active":"2024-04-25T09:13:32.737920+0000","last_peered":"2024-04-25T09:13:32.737920+0000","last_clean":"2024-04-25T09:13:32.737920+0000","last_became_active":"2024-04-25T09:13:30.766941+0000","last_became_peered":"2024-04-25T09:13:30.766941+0000","last_unstale":"2024-04-25T09:13:32.737920+0000","last_undegraded":"2024-04-25T09:13:32.737920+0000","last_fullsized":"2024-04-25T09:13:32.737920+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:20.373156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681794+0000","last_change":"2024-04-25T09:13:31.766013+0000","last_active":"2024-04-25T09:13:33.681794+0000","last_peered":"2024-04-25T09:13:33.681794+0000","last_clean":"2024-04-25T09:13:33.681794+0000","last_became_active":"2024-04-25T09:13:31.765713+0000","last_became_peered":"2024-04-25T09:13:31.765713+0000","last_unstale":"2024-04-25T09:13:33.681794+0000","last_undegraded":"2024-04-25T09:13:33.681794+0000","last_fullsized":"2024-04-25T09:13:33.681794+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.572396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682225+0000","last_change":"2024-04-25T09:13:30.751075+0000","last_active":"2024-04-25T09:13:33.682225+0000","last_peered":"2024-04-25T09:13:33.682225+0000","last_clean":"2024-04-25T09:13:33.682225+0000","last_became_active":"2024-04-25T09:13:30.750670+0000","last_became_peered":"2024-04-25T09:13:30.750670+0000","last_unstale":"2024-04-25T09:13:33.682225+0000","last_undegraded":"2024-04-25T09:13:33.682225+0000","last_fullsized":"2024-04-25T09:13:33.682225+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:45.319158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731629+0000","last_change":"2024-04-25T09:13:31.755644+0000","last_active":"2024-04-25T09:13:32.731629+0000","last_peered":"2024-04-25T09:13:32.731629+0000","last_clean":"2024-04-25T09:13:32.731629+0000","last_became_active":"2024-04-25T09:13:31.754027+0000","last_became_peered":"2024-04-25T09:13:31.754027+0000","last_unstale":"2024-04-25T09:13:32.731629+0000","last_undegraded":"2024-04-25T09:13:32.731629+0000","last_fullsized":"2024-04-25T09:13:32.731629+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:24.181807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.679632+0000","last_change":"2024-04-25T09:13:30.772429+0000","last_active":"2024-04-25T09:13:33.679632+0000","last_peered":"2024-04-25T09:13:33.679632+0000","last_clean":"2024-04-25T09:13:33.679632+0000","last_became_active":"2024-04-25T09:13:30.772072+0000","last_became_peered":"2024-04-25T09:13:30.772072+0000","last_unstale":"2024-04-25T09:13:33.679632+0000","last_undegraded":"2024-04-25T09:13:33.679632+0000","last_fullsized":"2024-04-25T09:13:33.679632+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:01.604932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736206+0000","last_change":"2024-04-25T09:13:31.768130+0000","last_active":"2024-04-25T09:13:32.736206+0000","last_peered":"2024-04-25T09:13:32.736206+0000","last_clean":"2024-04-25T09:13:32.736206+0000","last_became_active":"2024-04-25T09:13:31.767400+0000","last_became_peered":"2024-04-25T09:13:31.767400+0000","last_unstale":"2024-04-25T09:13:32.736206+0000","last_undegraded":"2024-04-25T09:13:32.736206+0000","last_fullsized":"2024-04-25T09:13:32.736206+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:35.987399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771470+0000","last_change":"2024-04-25T09:13:30.768374+0000","last_active":"2024-04-25T09:13:32.771470+0000","last_peered":"2024-04-25T09:13:32.771470+0000","last_clean":"2024-04-25T09:13:32.771470+0000","last_became_active":"2024-04-25T09:13:30.768014+0000","last_became_peered":"2024-04-25T09:13:30.768014+0000","last_unstale":"2024-04-25T09:13:32.771470+0000","last_undegraded":"2024-04-25T09:13:32.771470+0000","last_fullsized":"2024-04-25T09:13:32.771470+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:00.453307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777256+0000","last_change":"2024-04-25T09:13:31.757781+0000","last_active":"2024-04-25T09:13:32.777256+0000","last_peered":"2024-04-25T09:13:32.777256+0000","last_clean":"2024-04-25T09:13:32.777256+0000","last_became_active":"2024-04-25T09:13:31.755628+0000","last_became_peered":"2024-04-25T09:13:31.755628+0000","last_unstale":"2024-04-25T09:13:32.777256+0000","last_undegraded":"2024-04-25T09:13:32.777256+0000","last_fullsized":"2024-04-25T09:13:32.777256+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.507625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736467+0000","last_change":"2024-04-25T09:13:30.764781+0000","last_active":"2024-04-25T09:13:32.736467+0000","last_peered":"2024-04-25T09:13:32.736467+0000","last_clean":"2024-04-25T09:13:32.736467+0000","last_became_active":"2024-04-25T09:13:30.764484+0000","last_became_peered":"2024-04-25T09:13:30.764484+0000","last_unstale":"2024-04-25T09:13:32.736467+0000","last_undegraded":"2024-04-25T09:13:32.736467+0000","last_fullsized":"2024-04-25T09:13:32.736467+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:35.340007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682107+0000","last_change":"2024-04-25T09:13:31.760410+0000","last_active":"2024-04-25T09:13:33.682107+0000","last_peered":"2024-04-25T09:13:33.682107+0000","last_clean":"2024-04-25T09:13:33.682107+0000","last_became_active":"2024-04-25T09:13:31.760054+0000","last_became_peered":"2024-04-25T09:13:31.760054+0000","last_unstale":"2024-04-25T09:13:33.682107+0000","last_undegraded":"2024-04-25T09:13:33.682107+0000","last_fullsized":"2024-04-25T09:13:33.682107+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:09.370583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678458+0000","last_change":"2024-04-25T09:13:30.752404+0000","last_active":"2024-04-25T09:13:33.678458+0000","last_peered":"2024-04-25T09:13:33.678458+0000","last_clean":"2024-04-25T09:13:33.678458+0000","last_became_active":"2024-04-25T09:13:30.752093+0000","last_became_peered":"2024-04-25T09:13:30.752093+0000","last_unstale":"2024-04-25T09:13:33.678458+0000","last_undegraded":"2024-04-25T09:13:33.678458+0000","last_fullsized":"2024-04-25T09:13:33.678458+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:39.024037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776947+0000","last_change":"2024-04-25T09:13:31.748109+0000","last_active":"2024-04-25T09:13:32.776947+0000","last_peered":"2024-04-25T09:13:32.776947+0000","last_clean":"2024-04-25T09:13:32.776947+0000","last_became_active":"2024-04-25T09:13:31.747602+0000","last_became_peered":"2024-04-25T09:13:31.747602+0000","last_unstale":"2024-04-25T09:13:32.776947+0000","last_undegraded":"2024-04-25T09:13:32.776947+0000","last_fullsized":"2024-04-25T09:13:32.776947+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:20.690352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736543+0000","last_change":"2024-04-25T09:13:30.766653+0000","last_active":"2024-04-25T09:13:32.736543+0000","last_peered":"2024-04-25T09:13:32.736543+0000","last_clean":"2024-04-25T09:13:32.736543+0000","last_became_active":"2024-04-25T09:13:30.763878+0000","last_became_peered":"2024-04-25T09:13:30.763878+0000","last_unstale":"2024-04-25T09:13:32.736543+0000","last_undegraded":"2024-04-25T09:13:32.736543+0000","last_fullsized":"2024-04-25T09:13:32.736543+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:36.434216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.732299+0000","last_change":"2024-04-25T09:13:31.754988+0000","last_active":"2024-04-25T09:13:32.732299+0000","last_peered":"2024-04-25T09:13:32.732299+0000","last_clean":"2024-04-25T09:13:32.732299+0000","last_became_active":"2024-04-25T09:13:31.754671+0000","last_became_peered":"2024-04-25T09:13:31.754671+0000","last_unstale":"2024-04-25T09:13:32.732299+0000","last_undegraded":"2024-04-25T09:13:32.732299+0000","last_fullsized":"2024-04-25T09:13:32.732299+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:51.743854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678812+0000","last_change":"2024-04-25T09:13:30.769669+0000","last_active":"2024-04-25T09:13:33.678812+0000","last_peered":"2024-04-25T09:13:33.678812+0000","last_clean":"2024-04-25T09:13:33.678812+0000","last_became_active":"2024-04-25T09:13:30.769201+0000","last_became_peered":"2024-04-25T09:13:30.769201+0000","last_unstale":"2024-04-25T09:13:33.678812+0000","last_undegraded":"2024-04-25T09:13:33.678812+0000","last_fullsized":"2024-04-25T09:13:33.678812+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:12.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731940+0000","last_change":"2024-04-25T09:13:31.751894+0000","last_active":"2024-04-25T09:13:32.731940+0000","last_peered":"2024-04-25T09:13:32.731940+0000","last_clean":"2024-04-25T09:13:32.731940+0000","last_became_active":"2024-04-25T09:13:31.751329+0000","last_became_peered":"2024-04-25T09:13:31.751329+0000","last_unstale":"2024-04-25T09:13:32.731940+0000","last_undegraded":"2024-04-25T09:13:32.731940+0000","last_fullsized":"2024-04-25T09:13:32.731940+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:07.411063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682330+0000","last_change":"2024-04-25T09:13:30.750947+0000","last_active":"2024-04-25T09:13:33.682330+0000","last_peered":"2024-04-25T09:13:33.682330+0000","last_clean":"2024-04-25T09:13:33.682330+0000","last_became_active":"2024-04-25T09:13:30.749398+0000","last_became_peered":"2024-04-25T09:13:30.749398+0000","last_unstale":"2024-04-25T09:13:33.682330+0000","last_undegraded":"2024-04-25T09:13:33.682330+0000","last_fullsized":"2024-04-25T09:13:33.682330+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:19.276510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678838+0000","last_change":"2024-04-25T09:13:31.758363+0000","last_active":"2024-04-25T09:13:33.678838+0000","last_peered":"2024-04-25T09:13:33.678838+0000","last_clean":"2024-04-25T09:13:33.678838+0000","last_became_active":"2024-04-25T09:13:31.758036+0000","last_became_peered":"2024-04-25T09:13:31.758036+0000","last_unstale":"2024-04-25T09:13:33.678838+0000","last_undegraded":"2024-04-25T09:13:33.678838+0000","last_fullsized":"2024-04-25T09:13:33.678838+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:57.130586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770899+0000","last_change":"2024-04-25T09:13:30.768388+0000","last_active":"2024-04-25T09:13:32.770899+0000","last_peered":"2024-04-25T09:13:32.770899+0000","last_clean":"2024-04-25T09:13:32.770899+0000","last_became_active":"2024-04-25T09:13:30.768074+0000","last_became_peered":"2024-04-25T09:13:30.768074+0000","last_unstale":"2024-04-25T09:13:32.770899+0000","last_undegraded":"2024-04-25T09:13:32.770899+0000","last_fullsized":"2024-04-25T09:13:32.770899+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.398817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770569+0000","last_change":"2024-04-25T09:13:31.768360+0000","last_active":"2024-04-25T09:13:32.770569+0000","last_peered":"2024-04-25T09:13:32.770569+0000","last_clean":"2024-04-25T09:13:32.770569+0000","last_became_active":"2024-04-25T09:13:31.757296+0000","last_became_peered":"2024-04-25T09:13:31.757296+0000","last_unstale":"2024-04-25T09:13:32.770569+0000","last_undegraded":"2024-04-25T09:13:32.770569+0000","last_fullsized":"2024-04-25T09:13:32.770569+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:36.087719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735941+0000","last_change":"2024-04-25T09:13:30.760335+0000","last_active":"2024-04-25T09:13:32.735941+0000","last_peered":"2024-04-25T09:13:32.735941+0000","last_clean":"2024-04-25T09:13:32.735941+0000","last_became_active":"2024-04-25T09:13:30.759965+0000","last_became_peered":"2024-04-25T09:13:30.759965+0000","last_unstale":"2024-04-25T09:13:32.735941+0000","last_undegraded":"2024-04-25T09:13:32.735941+0000","last_fullsized":"2024-04-25T09:13:32.735941+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:52.790415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770915+0000","last_change":"2024-04-25T09:13:31.769243+0000","last_active":"2024-04-25T09:13:32.770915+0000","last_peered":"2024-04-25T09:13:32.770915+0000","last_clean":"2024-04-25T09:13:32.770915+0000","last_became_active":"2024-04-25T09:13:31.757711+0000","last_became_peered":"2024-04-25T09:13:31.757711+0000","last_unstale":"2024-04-25T09:13:32.770915+0000","last_undegraded":"2024-04-25T09:13:32.770915+0000","last_fullsized":"2024-04-25T09:13:32.770915+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:41.959312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731330+0000","last_change":"2024-04-25T09:13:30.758510+0000","last_active":"2024-04-25T09:13:32.731330+0000","last_peered":"2024-04-25T09:13:32.731330+0000","last_clean":"2024-04-25T09:13:32.731330+0000","last_became_active":"2024-04-25T09:13:30.757982+0000","last_became_peered":"2024-04-25T09:13:30.757982+0000","last_unstale":"2024-04-25T09:13:32.731330+0000","last_undegraded":"2024-04-25T09:13:32.731330+0000","last_fullsized":"2024-04-25T09:13:32.731330+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:21.341248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736430+0000","last_change":"2024-04-25T09:13:31.771139+0000","last_active":"2024-04-25T09:13:32.736430+0000","last_peered":"2024-04-25T09:13:32.736430+0000","last_clean":"2024-04-25T09:13:32.736430+0000","last_became_active":"2024-04-25T09:13:31.769796+0000","last_became_peered":"2024-04-25T09:13:31.769796+0000","last_unstale":"2024-04-25T09:13:32.736430+0000","last_undegraded":"2024-04-25T09:13:32.736430+0000","last_fullsized":"2024-04-25T09:13:32.736430+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:42.644077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681277+0000","last_change":"2024-04-25T09:13:30.763069+0000","last_active":"2024-04-25T09:13:33.681277+0000","last_peered":"2024-04-25T09:13:33.681277+0000","last_clean":"2024-04-25T09:13:33.681277+0000","last_became_active":"2024-04-25T09:13:30.762715+0000","last_became_peered":"2024-04-25T09:13:30.762715+0000","last_unstale":"2024-04-25T09:13:33.681277+0000","last_undegraded":"2024-04-25T09:13:33.681277+0000","last_fullsized":"2024-04-25T09:13:33.681277+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:08.062006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731556+0000","last_change":"2024-04-25T09:13:31.742003+0000","last_active":"2024-04-25T09:13:32.731556+0000","last_peered":"2024-04-25T09:13:32.731556+0000","last_clean":"2024-04-25T09:13:32.731556+0000","last_became_active":"2024-04-25T09:13:31.738182+0000","last_became_peered":"2024-04-25T09:13:31.738182+0000","last_unstale":"2024-04-25T09:13:32.731556+0000","last_undegraded":"2024-04-25T09:13:32.731556+0000","last_fullsized":"2024-04-25T09:13:32.731556+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.677224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776420+0000","last_change":"2024-04-25T09:13:30.772054+0000","last_active":"2024-04-25T09:13:32.776420+0000","last_peered":"2024-04-25T09:13:32.776420+0000","last_clean":"2024-04-25T09:13:32.776420+0000","last_became_active":"2024-04-25T09:13:30.771547+0000","last_became_peered":"2024-04-25T09:13:30.771547+0000","last_unstale":"2024-04-25T09:13:32.776420+0000","last_undegraded":"2024-04-25T09:13:32.776420+0000","last_fullsized":"2024-04-25T09:13:32.776420+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.707958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771510+0000","last_change":"2024-04-25T09:13:31.771862+0000","last_active":"2024-04-25T09:13:32.771510+0000","last_peered":"2024-04-25T09:13:32.771510+0000","last_clean":"2024-04-25T09:13:32.771510+0000","last_became_active":"2024-04-25T09:13:31.770395+0000","last_became_peered":"2024-04-25T09:13:31.770395+0000","last_unstale":"2024-04-25T09:13:32.771510+0000","last_undegraded":"2024-04-25T09:13:32.771510+0000","last_fullsized":"2024-04-25T09:13:32.771510+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:09.234944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678878+0000","last_change":"2024-04-25T09:13:30.759153+0000","last_active":"2024-04-25T09:13:33.678878+0000","last_peered":"2024-04-25T09:13:33.678878+0000","last_clean":"2024-04-25T09:13:33.678878+0000","last_became_active":"2024-04-25T09:13:30.758830+0000","last_became_peered":"2024-04-25T09:13:30.758830+0000","last_unstale":"2024-04-25T09:13:33.678878+0000","last_undegraded":"2024-04-25T09:13:33.678878+0000","last_fullsized":"2024-04-25T09:13:33.678878+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:44.753604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771896+0000","last_change":"2024-04-25T09:13:31.773186+0000","last_active":"2024-04-25T09:13:32.771896+0000","last_peered":"2024-04-25T09:13:32.771896+0000","last_clean":"2024-04-25T09:13:32.771896+0000","last_became_active":"2024-04-25T09:13:31.770150+0000","last_became_peered":"2024-04-25T09:13:31.770150+0000","last_unstale":"2024-04-25T09:13:32.771896+0000","last_undegraded":"2024-04-25T09:13:32.771896+0000","last_fullsized":"2024-04-25T09:13:32.771896+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:55.723506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"109'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771897+0000","last_change":"2024-04-25T09:13:30.767521+0000","last_active":"2024-04-25T09:13:32.771897+0000","last_peered":"2024-04-25T09:13:32.771897+0000","last_clean":"2024-04-25T09:13:32.771897+0000","last_became_active":"2024-04-25T09:13:30.766649+0000","last_became_peered":"2024-04-25T09:13:30.766649+0000","last_unstale":"2024-04-25T09:13:32.771897+0000","last_undegraded":"2024-04-25T09:13:32.771897+0000","last_fullsized":"2024-04-25T09:13:32.771897+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:19.915302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777552+0000","last_change":"2024-04-25T09:13:31.755552+0000","last_active":"2024-04-25T09:13:32.777552+0000","last_peered":"2024-04-25T09:13:32.777552+0000","last_clean":"2024-04-25T09:13:32.777552+0000","last_became_active":"2024-04-25T09:13:31.755236+0000","last_became_peered":"2024-04-25T09:13:31.755236+0000","last_unstale":"2024-04-25T09:13:32.777552+0000","last_undegraded":"2024-04-25T09:13:32.777552+0000","last_fullsized":"2024-04-25T09:13:32.777552+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:38.417018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771265+0000","last_change":"2024-04-25T09:13:30.765673+0000","last_active":"2024-04-25T09:13:32.771265+0000","last_peered":"2024-04-25T09:13:32.771265+0000","last_clean":"2024-04-25T09:13:32.771265+0000","last_became_active":"2024-04-25T09:13:30.765362+0000","last_became_peered":"2024-04-25T09:13:30.765362+0000","last_unstale":"2024-04-25T09:13:32.771265+0000","last_undegraded":"2024-04-25T09:13:32.771265+0000","last_fullsized":"2024-04-25T09:13:32.771265+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:39.493995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682179+0000","last_change":"2024-04-25T09:13:31.763659+0000","last_active":"2024-04-25T09:13:33.682179+0000","last_peered":"2024-04-25T09:13:33.682179+0000","last_clean":"2024-04-25T09:13:33.682179+0000","last_became_active":"2024-04-25T09:13:31.763182+0000","last_became_peered":"2024-04-25T09:13:31.763182+0000","last_unstale":"2024-04-25T09:13:33.682179+0000","last_undegraded":"2024-04-25T09:13:33.682179+0000","last_fullsized":"2024-04-25T09:13:33.682179+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:08.567402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.732085+0000","last_change":"2024-04-25T09:13:30.755732+0000","last_active":"2024-04-25T09:13:32.732085+0000","last_peered":"2024-04-25T09:13:32.732085+0000","last_clean":"2024-04-25T09:13:32.732085+0000","last_became_active":"2024-04-25T09:13:30.755311+0000","last_became_peered":"2024-04-25T09:13:30.755311+0000","last_unstale":"2024-04-25T09:13:32.732085+0000","last_undegraded":"2024-04-25T09:13:32.732085+0000","last_fullsized":"2024-04-25T09:13:32.732085+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:11.489424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771490+0000","last_change":"2024-04-25T09:13:31.768786+0000","last_active":"2024-04-25T09:13:32.771490+0000","last_peered":"2024-04-25T09:13:32.771490+0000","last_clean":"2024-04-25T09:13:32.771490+0000","last_became_active":"2024-04-25T09:13:31.768427+0000","last_became_peered":"2024-04-25T09:13:31.768427+0000","last_unstale":"2024-04-25T09:13:32.771490+0000","last_undegraded":"2024-04-25T09:13:32.771490+0000","last_fullsized":"2024-04-25T09:13:32.771490+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:39.445673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777588+0000","last_change":"2024-04-25T09:13:30.762156+0000","last_active":"2024-04-25T09:13:32.777588+0000","last_peered":"2024-04-25T09:13:32.777588+0000","last_clean":"2024-04-25T09:13:32.777588+0000","last_became_active":"2024-04-25T09:13:30.761176+0000","last_became_peered":"2024-04-25T09:13:30.761176+0000","last_unstale":"2024-04-25T09:13:32.777588+0000","last_undegraded":"2024-04-25T09:13:32.777588+0000","last_fullsized":"2024-04-25T09:13:32.777588+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:29.433527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681164+0000","last_change":"2024-04-25T09:13:31.756281+0000","last_active":"2024-04-25T09:13:33.681164+0000","last_peered":"2024-04-25T09:13:33.681164+0000","last_clean":"2024-04-25T09:13:33.681164+0000","last_became_active":"2024-04-25T09:13:31.755824+0000","last_became_peered":"2024-04-25T09:13:31.755824+0000","last_unstale":"2024-04-25T09:13:33.681164+0000","last_undegraded":"2024-04-25T09:13:33.681164+0000","last_fullsized":"2024-04-25T09:13:33.681164+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:27.101533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731507+0000","last_change":"2024-04-25T09:13:30.768177+0000","last_active":"2024-04-25T09:13:32.731507+0000","last_peered":"2024-04-25T09:13:32.731507+0000","last_clean":"2024-04-25T09:13:32.731507+0000","last_became_active":"2024-04-25T09:13:30.767744+0000","last_became_peered":"2024-04-25T09:13:30.767744+0000","last_unstale":"2024-04-25T09:13:32.731507+0000","last_undegraded":"2024-04-25T09:13:32.731507+0000","last_fullsized":"2024-04-25T09:13:32.731507+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:10.366780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770870+0000","last_change":"2024-04-25T09:13:31.765477+0000","last_active":"2024-04-25T09:13:32.770870+0000","last_peered":"2024-04-25T09:13:32.770870+0000","last_clean":"2024-04-25T09:13:32.770870+0000","last_became_active":"2024-04-25T09:13:31.765181+0000","last_became_peered":"2024-04-25T09:13:31.765181+0000","last_unstale":"2024-04-25T09:13:32.770870+0000","last_undegraded":"2024-04-25T09:13:32.770870+0000","last_fullsized":"2024-04-25T09:13:32.770870+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:51.743854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.677744+0000","last_change":"2024-04-25T09:13:30.758745+0000","last_active":"2024-04-25T09:13:33.677744+0000","last_peered":"2024-04-25T09:13:33.677744+0000","last_clean":"2024-04-25T09:13:33.677744+0000","last_became_active":"2024-04-25T09:13:30.758378+0000","last_became_peered":"2024-04-25T09:13:30.758378+0000","last_unstale":"2024-04-25T09:13:33.677744+0000","last_undegraded":"2024-04-25T09:13:33.677744+0000","last_fullsized":"2024-04-25T09:13:33.677744+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:14.514322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678132+0000","last_change":"2024-04-25T09:13:31.765240+0000","last_active":"2024-04-25T09:13:33.678132+0000","last_peered":"2024-04-25T09:13:33.678132+0000","last_clean":"2024-04-25T09:13:33.678132+0000","last_became_active":"2024-04-25T09:13:31.764923+0000","last_became_peered":"2024-04-25T09:13:31.764923+0000","last_unstale":"2024-04-25T09:13:33.678132+0000","last_undegraded":"2024-04-25T09:13:33.678132+0000","last_fullsized":"2024-04-25T09:13:33.678132+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.266138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776688+0000","last_change":"2024-04-25T09:13:30.764575+0000","last_active":"2024-04-25T09:13:32.776688+0000","last_peered":"2024-04-25T09:13:32.776688+0000","last_clean":"2024-04-25T09:13:32.776688+0000","last_became_active":"2024-04-25T09:13:30.764286+0000","last_became_peered":"2024-04-25T09:13:30.764286+0000","last_unstale":"2024-04-25T09:13:32.776688+0000","last_undegraded":"2024-04-25T09:13:32.776688+0000","last_fullsized":"2024-04-25T09:13:32.776688+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:01.389513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777835+0000","last_change":"2024-04-25T09:13:31.770097+0000","last_active":"2024-04-25T09:13:32.777835+0000","last_peered":"2024-04-25T09:13:32.777835+0000","last_clean":"2024-04-25T09:13:32.777835+0000","last_became_active":"2024-04-25T09:13:31.769811+0000","last_became_peered":"2024-04-25T09:13:31.769811+0000","last_unstale":"2024-04-25T09:13:32.777835+0000","last_undegraded":"2024-04-25T09:13:32.777835+0000","last_fullsized":"2024-04-25T09:13:32.777835+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:01.471507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678545+0000","last_change":"2024-04-25T09:13:30.766905+0000","last_active":"2024-04-25T09:13:33.678545+0000","last_peered":"2024-04-25T09:13:33.678545+0000","last_clean":"2024-04-25T09:13:33.678545+0000","last_became_active":"2024-04-25T09:13:30.766629+0000","last_became_peered":"2024-04-25T09:13:30.766629+0000","last_unstale":"2024-04-25T09:13:33.678545+0000","last_undegraded":"2024-04-25T09:13:33.678545+0000","last_fullsized":"2024-04-25T09:13:33.678545+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:01.547908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":332,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678026+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:13:33.678026+0000","last_peered":"2024-04-25T09:13:33.678026+0000","last_clean":"2024-04-25T09:13:33.678026+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:13:33.678026+0000","last_undegraded":"2024-04-25T09:13:33.678026+0000","last_fullsized":"2024-04-25T09:13:33.678026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50552,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321288,"statfs":{"total":96636764160,"available":96584998912,"internally_reserved":0,"allocated":1351680,"data_stored":590294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50552,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321288,"statfs":{"total":96636764160,"available":96584998912,"internally_reserved":0,"allocated":1347584,"data_stored":589712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574978,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":1181748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574978,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54712,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317128,"statfs":{"total":96636764160,"available":96580739072,"internally_reserved":0,"allocated":1351680,"data_stored":590294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54716,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317124,"statfs":{"total":96636764160,"available":96580734976,"internally_reserved":0,"allocated":1359872,"data_stored":589906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76924,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294916,"statfs":{"total":96636764160,"available":96557993984,"internally_reserved":0,"allocated":1351680,"data_stored":589782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574978,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51144,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320696,"statfs":{"total":96636764160,"available":96584392704,"internally_reserved":0,"allocated":1961984,"data_stored":1181804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574977,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1347584,"data_stored":589712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:13:36.183 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:13:36.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.450+0000 7fa243e00640 1 Processor -- start 2024-04-25T09:13:36.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.450+0000 7fa243e00640 1 -- start start 2024-04-25T09:13:36.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.452+0000 7fa243e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2440d1e90 0x7fa2440b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.452+0000 7fa243e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2440b6730 0x7fa2440b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.452+0000 7fa243e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2440cf890 con 0x7fa24415b900 2024-04-25T09:13:36.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.452+0000 7fa243e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa244070a00 con 0x7fa2440b6730 2024-04-25T09:13:36.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.452+0000 7fa243e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440b6b80 con 0x7fa2440d1e90 2024-04-25T09:13:36.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.452+0000 7fa243400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2440b6730 0x7fa2440b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.452+0000 7fa242a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa24415b900 0x7fa2440d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:55448/0 (socket says 172.21.15.62:55448) 2024-04-25T09:13:36.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.452+0000 7fa242a00640 1 -- 172.21.15.62:0/620136845 learned_addr learned my addr 172.21.15.62:0/620136845 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:36.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.453+0000 7fa242000640 1 --2- 172.21.15.62:0/620136845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2440d1e90 0x7fa2440b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.453+0000 7fa243400640 1 -- 172.21.15.62:0/620136845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2440d1e90 msgr2=0x7fa2440b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:36.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.453+0000 7fa243400640 1 --2- 172.21.15.62:0/620136845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2440d1e90 0x7fa2440b61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.453+0000 7fa243400640 1 -- 172.21.15.62:0/620136845 >> v1:172.21.15.121:6789/0 conn(0x7fa24415b900 legacy=0x7fa2440d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.453+0000 7fa243400640 1 -- 172.21.15.62:0/620136845 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24406fdf0 con 0x7fa2440b6730 2024-04-25T09:13:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.455+0000 7fa243400640 1 --2- 172.21.15.62:0/620136845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 0x7fa2440b4860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009d90 tx=0x7fa23c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c45dbfc2bc59435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.455+0000 7fa241600640 1 -- 172.21.15.62:0/620136845 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c007740 con 0x7fa2440b6730 2024-04-25T09:13:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.455+0000 7fa241600640 1 -- 172.21.15.62:0/620136845 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23c0078e0 con 0x7fa2440b6730 2024-04-25T09:13:36.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.455+0000 7fa241600640 1 -- 172.21.15.62:0/620136845 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c007cb0 con 0x7fa2440b6730 2024-04-25T09:13:36.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.456+0000 7fa243e00640 1 -- 172.21.15.62:0/620136845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 msgr2=0x7fa2440b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.456+0000 7fa243e00640 1 --2- 172.21.15.62:0/620136845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 0x7fa2440b4860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009d90 tx=0x7fa23c02a660 comp rx=0 tx=0).stop 2024-04-25T09:13:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.456+0000 7fa243e00640 1 -- 172.21.15.62:0/620136845 shutdown_connections 2024-04-25T09:13:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.456+0000 7fa243e00640 1 --2- 172.21.15.62:0/620136845 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 0x7fa2440b4860 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.456+0000 7fa243e00640 1 --2- 172.21.15.62:0/620136845 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2440d1e90 0x7fa2440b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.456+0000 7fa243e00640 1 -- 172.21.15.62:0/620136845 >> 172.21.15.62:0/620136845 conn(0x7fa244072340 msgr2=0x7fa244072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.456+0000 7fa243e00640 1 -- 172.21.15.62:0/620136845 shutdown_connections 2024-04-25T09:13:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.456+0000 7fa243e00640 1 -- 172.21.15.62:0/620136845 wait complete. 2024-04-25T09:13:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.458+0000 7fa243e00640 1 Processor -- start 2024-04-25T09:13:36.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.458+0000 7fa243e00640 1 -- start start 2024-04-25T09:13:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa243e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 0x7fa244143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa243e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2440d1e90 0x7fa24415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa243e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa24415b900 0x7fa24415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa243e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa244070690 con 0x7fa2440d1e90 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa243e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440703b0 con 0x7fa2440b6730 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa243e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24415bce0 con 0x7fa24415b900 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa242000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2440d1e90 0x7fa24415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa242000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2440d1e90 0x7fa24415ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55792/0 (socket says 172.21.15.62:55792) 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.459+0000 7fa242000640 1 -- 172.21.15.62:0/1607106313 learned_addr learned my addr 172.21.15.62:0/1607106313 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.460+0000 7fa242a00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 0x7fa244143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.460+0000 7fa243400640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa24415b900 0x7fa24415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.460+0000 7fa242000640 1 -- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa24415b900 msgr2=0x7fa24415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.460+0000 7fa242000640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa24415b900 0x7fa24415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.460+0000 7fa242000640 1 -- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 msgr2=0x7fa244143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.460+0000 7fa242000640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 0x7fa244143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.460+0000 7fa242000640 1 -- 172.21.15.62:0/1607106313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24414e940 con 0x7fa2440d1e90 2024-04-25T09:13:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.460+0000 7fa242000640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2440d1e90 0x7fa24415ea00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa23000e9b0 tx=0x7fa23000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.461+0000 7fa23aa00640 1 -- 172.21.15.62:0/1607106313 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230017070 con 0x7fa2440d1e90 2024-04-25T09:13:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.461+0000 7fa23aa00640 1 -- 172.21.15.62:0/1607106313 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23000c9a0 con 0x7fa2440d1e90 2024-04-25T09:13:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.461+0000 7fa23aa00640 1 -- 172.21.15.62:0/1607106313 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23000cd90 con 0x7fa2440d1e90 2024-04-25T09:13:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.461+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa244143770 con 0x7fa2440d1e90 2024-04-25T09:13:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.461+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa244151c20 con 0x7fa2440d1e90 2024-04-25T09:13:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.462+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2441525c0 con 0x7fa2440d1e90 2024-04-25T09:13:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.463+0000 7fa23aa00640 1 -- 172.21.15.62:0/1607106313 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa23001a070 con 0x7fa2440d1e90 2024-04-25T09:13:36.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.464+0000 7fa23aa00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa22406b4d0 0x7fa22406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.464+0000 7fa23aa00640 1 -- 172.21.15.62:0/1607106313 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa23008a2e0 con 0x7fa2440d1e90 2024-04-25T09:13:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.465+0000 7fa242a00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa22406b4d0 0x7fa22406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.466+0000 7fa242a00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa22406b4d0 0x7fa22406d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa234005880 tx=0x7fa234009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:36.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.469+0000 7fa23aa00640 1 -- 172.21.15.62:0/1607106313 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2441525c0 con 0x7fa2440d1e90 2024-04-25T09:13:36.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.665+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa244067cf0 con 0x7fa22406b4d0 2024-04-25T09:13:36.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.684+0000 7fa23aa00640 1 -- 172.21.15.62:0/1607106313 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7fa2340058b0 con 0x7fa22406b4d0 2024-04-25T09:13:36.675 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:36.677 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:13:36.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa22406b4d0 msgr2=0x7fa22406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa22406b4d0 0x7fa22406d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa234005880 tx=0x7fa234009040 comp rx=0 tx=0).stop 2024-04-25T09:13:36.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2440d1e90 msgr2=0x7fa24415ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2440d1e90 0x7fa24415ea00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa23000e9b0 tx=0x7fa23000ee70 comp rx=0 tx=0).stop 2024-04-25T09:13:36.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 shutdown_connections 2024-04-25T09:13:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa22406b4d0 0x7fa22406d990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2440d1e90 0x7fa24415ea00 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa24415b900 0x7fa24415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 --2- 172.21.15.62:0/1607106313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2440b6730 0x7fa244143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.689+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 >> 172.21.15.62:0/1607106313 conn(0x7fa244072340 msgr2=0x7fa2440cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.690+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 shutdown_connections 2024-04-25T09:13:36.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.690+0000 7fa243e00640 1 -- 172.21.15.62:0/1607106313 wait complete. 2024-04-25T09:13:36.697 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-04-25T09:13:36.122286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422240,"kb_used_data":11752,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754552480,"statfs":{"total":773094113280,"available":772661739520,"internally_reserved":0,"allocated":12034048,"data_stored":5903252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001886"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737972+0000","last_change":"2024-04-25T09:13:31.766948+0000","last_active":"2024-04-25T09:13:32.737972+0000","last_peered":"2024-04-25T09:13:32.737972+0000","last_clean":"2024-04-25T09:13:32.737972+0000","last_became_active":"2024-04-25T09:13:31.766197+0000","last_became_peered":"2024-04-25T09:13:31.766197+0000","last_unstale":"2024-04-25T09:13:32.737972+0000","last_undegraded":"2024-04-25T09:13:32.737972+0000","last_fullsized":"2024-04-25T09:13:32.737972+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:36.087719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737522+0000","last_change":"2024-04-25T09:13:31.768097+0000","last_active":"2024-04-25T09:13:32.737522+0000","last_peered":"2024-04-25T09:13:32.737522+0000","last_clean":"2024-04-25T09:13:32.737522+0000","last_became_active":"2024-04-25T09:13:31.767760+0000","last_became_peered":"2024-04-25T09:13:31.767760+0000","last_unstale":"2024-04-25T09:13:32.737522+0000","last_undegraded":"2024-04-25T09:13:32.737522+0000","last_fullsized":"2024-04-25T09:13:32.737522+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:35.910056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777397+0000","last_change":"2024-04-25T09:13:31.751466+0000","last_active":"2024-04-25T09:13:32.777397+0000","last_peered":"2024-04-25T09:13:32.777397+0000","last_clean":"2024-04-25T09:13:32.777397+0000","last_became_active":"2024-04-25T09:13:31.750906+0000","last_became_peered":"2024-04-25T09:13:31.750906+0000","last_unstale":"2024-04-25T09:13:32.777397+0000","last_undegraded":"2024-04-25T09:13:32.777397+0000","last_fullsized":"2024-04-25T09:13:32.777397+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:40.512196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736566+0000","last_change":"2024-04-25T09:13:31.772818+0000","last_active":"2024-04-25T09:13:32.736566+0000","last_peered":"2024-04-25T09:13:32.736566+0000","last_clean":"2024-04-25T09:13:32.736566+0000","last_became_active":"2024-04-25T09:13:31.769664+0000","last_became_peered":"2024-04-25T09:13:31.769664+0000","last_unstale":"2024-04-25T09:13:32.736566+0000","last_undegraded":"2024-04-25T09:13:32.736566+0000","last_fullsized":"2024-04-25T09:13:32.736566+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:26.240304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680846+0000","last_change":"2024-04-25T09:13:31.758568+0000","last_active":"2024-04-25T09:13:33.680846+0000","last_peered":"2024-04-25T09:13:33.680846+0000","last_clean":"2024-04-25T09:13:33.680846+0000","last_became_active":"2024-04-25T09:13:31.758270+0000","last_became_peered":"2024-04-25T09:13:31.758270+0000","last_unstale":"2024-04-25T09:13:33.680846+0000","last_undegraded":"2024-04-25T09:13:33.680846+0000","last_fullsized":"2024-04-25T09:13:33.680846+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:43.319815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681446+0000","last_change":"2024-04-25T09:13:31.766705+0000","last_active":"2024-04-25T09:13:33.681446+0000","last_peered":"2024-04-25T09:13:33.681446+0000","last_clean":"2024-04-25T09:13:33.681446+0000","last_became_active":"2024-04-25T09:13:31.765788+0000","last_became_peered":"2024-04-25T09:13:31.765788+0000","last_unstale":"2024-04-25T09:13:33.681446+0000","last_undegraded":"2024-04-25T09:13:33.681446+0000","last_fullsized":"2024-04-25T09:13:33.681446+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:57.687898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680682+0000","last_change":"2024-04-25T09:13:31.766949+0000","last_active":"2024-04-25T09:13:33.680682+0000","last_peered":"2024-04-25T09:13:33.680682+0000","last_clean":"2024-04-25T09:13:33.680682+0000","last_became_active":"2024-04-25T09:13:31.766588+0000","last_became_peered":"2024-04-25T09:13:31.766588+0000","last_unstale":"2024-04-25T09:13:33.680682+0000","last_undegraded":"2024-04-25T09:13:33.680682+0000","last_fullsized":"2024-04-25T09:13:33.680682+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:48.584478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776376+0000","last_change":"2024-04-25T09:13:31.753196+0000","last_active":"2024-04-25T09:13:32.776376+0000","last_peered":"2024-04-25T09:13:32.776376+0000","last_clean":"2024-04-25T09:13:32.776376+0000","last_became_active":"2024-04-25T09:13:31.752817+0000","last_became_peered":"2024-04-25T09:13:31.752817+0000","last_unstale":"2024-04-25T09:13:32.776376+0000","last_undegraded":"2024-04-25T09:13:32.776376+0000","last_fullsized":"2024-04-25T09:13:32.776376+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:25.286953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682752+0000","last_change":"2024-04-25T09:13:31.763832+0000","last_active":"2024-04-25T09:13:33.682752+0000","last_peered":"2024-04-25T09:13:33.682752+0000","last_clean":"2024-04-25T09:13:33.682752+0000","last_became_active":"2024-04-25T09:13:31.763375+0000","last_became_peered":"2024-04-25T09:13:31.763375+0000","last_unstale":"2024-04-25T09:13:33.682752+0000","last_undegraded":"2024-04-25T09:13:33.682752+0000","last_fullsized":"2024-04-25T09:13:33.682752+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:33.121123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680441+0000","last_change":"2024-04-25T09:13:31.771696+0000","last_active":"2024-04-25T09:13:33.680441+0000","last_peered":"2024-04-25T09:13:33.680441+0000","last_clean":"2024-04-25T09:13:33.680441+0000","last_became_active":"2024-04-25T09:13:31.771358+0000","last_became_peered":"2024-04-25T09:13:31.771358+0000","last_unstale":"2024-04-25T09:13:33.680441+0000","last_undegraded":"2024-04-25T09:13:33.680441+0000","last_fullsized":"2024-04-25T09:13:33.680441+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:11.384981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681955+0000","last_change":"2024-04-25T09:13:31.767014+0000","last_active":"2024-04-25T09:13:33.681955+0000","last_peered":"2024-04-25T09:13:33.681955+0000","last_clean":"2024-04-25T09:13:33.681955+0000","last_became_active":"2024-04-25T09:13:31.766316+0000","last_became_peered":"2024-04-25T09:13:31.766316+0000","last_unstale":"2024-04-25T09:13:33.681955+0000","last_undegraded":"2024-04-25T09:13:33.681955+0000","last_fullsized":"2024-04-25T09:13:33.681955+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:50.418299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.679359+0000","last_change":"2024-04-25T09:13:31.768680+0000","last_active":"2024-04-25T09:13:33.679359+0000","last_peered":"2024-04-25T09:13:33.679359+0000","last_clean":"2024-04-25T09:13:33.679359+0000","last_became_active":"2024-04-25T09:13:31.768381+0000","last_became_peered":"2024-04-25T09:13:31.768381+0000","last_unstale":"2024-04-25T09:13:33.679359+0000","last_undegraded":"2024-04-25T09:13:33.679359+0000","last_fullsized":"2024-04-25T09:13:33.679359+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:53.808615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.677964+0000","last_change":"2024-04-25T09:13:31.766561+0000","last_active":"2024-04-25T09:13:33.677964+0000","last_peered":"2024-04-25T09:13:33.677964+0000","last_clean":"2024-04-25T09:13:33.677964+0000","last_became_active":"2024-04-25T09:13:31.766213+0000","last_became_peered":"2024-04-25T09:13:31.766213+0000","last_unstale":"2024-04-25T09:13:33.677964+0000","last_undegraded":"2024-04-25T09:13:33.677964+0000","last_fullsized":"2024-04-25T09:13:33.677964+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:40.387620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737209+0000","last_change":"2024-04-25T09:13:31.768089+0000","last_active":"2024-04-25T09:13:32.737209+0000","last_peered":"2024-04-25T09:13:32.737209+0000","last_clean":"2024-04-25T09:13:32.737209+0000","last_became_active":"2024-04-25T09:13:31.767781+0000","last_became_peered":"2024-04-25T09:13:31.767781+0000","last_unstale":"2024-04-25T09:13:32.737209+0000","last_undegraded":"2024-04-25T09:13:32.737209+0000","last_fullsized":"2024-04-25T09:13:32.737209+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:24.181807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735530+0000","last_change":"2024-04-25T09:13:31.762995+0000","last_active":"2024-04-25T09:13:32.735530+0000","last_peered":"2024-04-25T09:13:32.735530+0000","last_clean":"2024-04-25T09:13:32.735530+0000","last_became_active":"2024-04-25T09:13:31.762621+0000","last_became_peered":"2024-04-25T09:13:31.762621+0000","last_unstale":"2024-04-25T09:13:32.735530+0000","last_undegraded":"2024-04-25T09:13:32.735530+0000","last_fullsized":"2024-04-25T09:13:32.735530+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:10.795857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681623+0000","last_change":"2024-04-25T09:13:31.767738+0000","last_active":"2024-04-25T09:13:33.681623+0000","last_peered":"2024-04-25T09:13:33.681623+0000","last_clean":"2024-04-25T09:13:33.681623+0000","last_became_active":"2024-04-25T09:13:31.767369+0000","last_became_peered":"2024-04-25T09:13:31.767369+0000","last_unstale":"2024-04-25T09:13:33.681623+0000","last_undegraded":"2024-04-25T09:13:33.681623+0000","last_fullsized":"2024-04-25T09:13:33.681623+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:49.795922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771534+0000","last_change":"2024-04-25T09:13:31.766905+0000","last_active":"2024-04-25T09:13:32.771534+0000","last_peered":"2024-04-25T09:13:32.771534+0000","last_clean":"2024-04-25T09:13:32.771534+0000","last_became_active":"2024-04-25T09:13:31.766629+0000","last_became_peered":"2024-04-25T09:13:31.766629+0000","last_unstale":"2024-04-25T09:13:32.771534+0000","last_undegraded":"2024-04-25T09:13:32.771534+0000","last_fullsized":"2024-04-25T09:13:32.771534+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:05.050039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737667+0000","last_change":"2024-04-25T09:13:31.766511+0000","last_active":"2024-04-25T09:13:32.737667+0000","last_peered":"2024-04-25T09:13:32.737667+0000","last_clean":"2024-04-25T09:13:32.737667+0000","last_became_active":"2024-04-25T09:13:31.764825+0000","last_became_peered":"2024-04-25T09:13:31.764825+0000","last_unstale":"2024-04-25T09:13:32.737667+0000","last_undegraded":"2024-04-25T09:13:32.737667+0000","last_fullsized":"2024-04-25T09:13:32.737667+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:07.411063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771808+0000","last_change":"2024-04-25T09:13:31.769715+0000","last_active":"2024-04-25T09:13:32.771808+0000","last_peered":"2024-04-25T09:13:32.771808+0000","last_clean":"2024-04-25T09:13:32.771808+0000","last_became_active":"2024-04-25T09:13:31.769435+0000","last_became_peered":"2024-04-25T09:13:31.769435+0000","last_unstale":"2024-04-25T09:13:32.771808+0000","last_undegraded":"2024-04-25T09:13:32.771808+0000","last_fullsized":"2024-04-25T09:13:32.771808+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:45.085992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.679413+0000","last_change":"2024-04-25T09:13:31.759575+0000","last_active":"2024-04-25T09:13:33.679413+0000","last_peered":"2024-04-25T09:13:33.679413+0000","last_clean":"2024-04-25T09:13:33.679413+0000","last_became_active":"2024-04-25T09:13:31.759253+0000","last_became_peered":"2024-04-25T09:13:31.759253+0000","last_unstale":"2024-04-25T09:13:33.679413+0000","last_undegraded":"2024-04-25T09:13:33.679413+0000","last_fullsized":"2024-04-25T09:13:33.679413+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:37.452417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777125+0000","last_change":"2024-04-25T09:13:31.767268+0000","last_active":"2024-04-25T09:13:32.777125+0000","last_peered":"2024-04-25T09:13:32.777125+0000","last_clean":"2024-04-25T09:13:32.777125+0000","last_became_active":"2024-04-25T09:13:31.764073+0000","last_became_peered":"2024-04-25T09:13:31.764073+0000","last_unstale":"2024-04-25T09:13:32.777125+0000","last_undegraded":"2024-04-25T09:13:32.777125+0000","last_fullsized":"2024-04-25T09:13:32.777125+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:11.384981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776455+0000","last_change":"2024-04-25T09:13:31.767758+0000","last_active":"2024-04-25T09:13:32.776455+0000","last_peered":"2024-04-25T09:13:32.776455+0000","last_clean":"2024-04-25T09:13:32.776455+0000","last_became_active":"2024-04-25T09:13:31.767479+0000","last_became_peered":"2024-04-25T09:13:31.767479+0000","last_unstale":"2024-04-25T09:13:32.776455+0000","last_undegraded":"2024-04-25T09:13:32.776455+0000","last_fullsized":"2024-04-25T09:13:32.776455+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:11.242013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735532+0000","last_change":"2024-04-25T09:13:31.763797+0000","last_active":"2024-04-25T09:13:32.735532+0000","last_peered":"2024-04-25T09:13:32.735532+0000","last_clean":"2024-04-25T09:13:32.735532+0000","last_became_active":"2024-04-25T09:13:31.763451+0000","last_became_peered":"2024-04-25T09:13:31.763451+0000","last_unstale":"2024-04-25T09:13:32.735532+0000","last_undegraded":"2024-04-25T09:13:32.735532+0000","last_fullsized":"2024-04-25T09:13:32.735532+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:20.933503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731309+0000","last_change":"2024-04-25T09:13:31.772907+0000","last_active":"2024-04-25T09:13:32.731309+0000","last_peered":"2024-04-25T09:13:32.731309+0000","last_clean":"2024-04-25T09:13:32.731309+0000","last_became_active":"2024-04-25T09:13:31.772572+0000","last_became_peered":"2024-04-25T09:13:31.772572+0000","last_unstale":"2024-04-25T09:13:32.731309+0000","last_undegraded":"2024-04-25T09:13:32.731309+0000","last_fullsized":"2024-04-25T09:13:32.731309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:26.240304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680648+0000","last_change":"2024-04-25T09:13:31.774461+0000","last_active":"2024-04-25T09:13:33.680648+0000","last_peered":"2024-04-25T09:13:33.680648+0000","last_clean":"2024-04-25T09:13:33.680648+0000","last_became_active":"2024-04-25T09:13:31.774022+0000","last_became_peered":"2024-04-25T09:13:31.774022+0000","last_unstale":"2024-04-25T09:13:33.680648+0000","last_undegraded":"2024-04-25T09:13:33.680648+0000","last_fullsized":"2024-04-25T09:13:33.680648+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.507625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678408+0000","last_change":"2024-04-25T09:13:31.762091+0000","last_active":"2024-04-25T09:13:33.678408+0000","last_peered":"2024-04-25T09:13:33.678408+0000","last_clean":"2024-04-25T09:13:33.678408+0000","last_became_active":"2024-04-25T09:13:31.761782+0000","last_became_peered":"2024-04-25T09:13:31.761782+0000","last_unstale":"2024-04-25T09:13:33.678408+0000","last_undegraded":"2024-04-25T09:13:33.678408+0000","last_fullsized":"2024-04-25T09:13:33.678408+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:46.551687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731709+0000","last_change":"2024-04-25T09:13:31.761119+0000","last_active":"2024-04-25T09:13:32.731709+0000","last_peered":"2024-04-25T09:13:32.731709+0000","last_clean":"2024-04-25T09:13:32.731709+0000","last_became_active":"2024-04-25T09:13:31.760820+0000","last_became_peered":"2024-04-25T09:13:31.760820+0000","last_unstale":"2024-04-25T09:13:32.731709+0000","last_undegraded":"2024-04-25T09:13:32.731709+0000","last_fullsized":"2024-04-25T09:13:32.731709+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:35.910056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.732265+0000","last_change":"2024-04-25T09:13:31.756872+0000","last_active":"2024-04-25T09:13:32.732265+0000","last_peered":"2024-04-25T09:13:32.732265+0000","last_clean":"2024-04-25T09:13:32.732265+0000","last_became_active":"2024-04-25T09:13:31.756574+0000","last_became_peered":"2024-04-25T09:13:31.756574+0000","last_unstale":"2024-04-25T09:13:32.732265+0000","last_undegraded":"2024-04-25T09:13:32.732265+0000","last_fullsized":"2024-04-25T09:13:32.732265+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:41.959312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678083+0000","last_change":"2024-04-25T09:13:31.760937+0000","last_active":"2024-04-25T09:13:33.678083+0000","last_peered":"2024-04-25T09:13:33.678083+0000","last_clean":"2024-04-25T09:13:33.678083+0000","last_became_active":"2024-04-25T09:13:31.760250+0000","last_became_peered":"2024-04-25T09:13:31.760250+0000","last_unstale":"2024-04-25T09:13:33.678083+0000","last_undegraded":"2024-04-25T09:13:33.678083+0000","last_fullsized":"2024-04-25T09:13:33.678083+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:30.451727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678120+0000","last_change":"2024-04-25T09:13:31.768344+0000","last_active":"2024-04-25T09:13:33.678120+0000","last_peered":"2024-04-25T09:13:33.678120+0000","last_clean":"2024-04-25T09:13:33.678120+0000","last_became_active":"2024-04-25T09:13:31.768062+0000","last_became_peered":"2024-04-25T09:13:31.768062+0000","last_unstale":"2024-04-25T09:13:33.678120+0000","last_undegraded":"2024-04-25T09:13:33.678120+0000","last_fullsized":"2024-04-25T09:13:33.678120+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:42.623821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776951+0000","last_change":"2024-04-25T09:13:31.765940+0000","last_active":"2024-04-25T09:13:32.776951+0000","last_peered":"2024-04-25T09:13:32.776951+0000","last_clean":"2024-04-25T09:13:32.776951+0000","last_became_active":"2024-04-25T09:13:31.765656+0000","last_became_peered":"2024-04-25T09:13:31.765656+0000","last_unstale":"2024-04-25T09:13:32.776951+0000","last_undegraded":"2024-04-25T09:13:32.776951+0000","last_fullsized":"2024-04-25T09:13:32.776951+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:22.359448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735964+0000","last_change":"2024-04-25T09:13:31.781458+0000","last_active":"2024-04-25T09:13:32.735964+0000","last_peered":"2024-04-25T09:13:32.735964+0000","last_clean":"2024-04-25T09:13:32.735964+0000","last_became_active":"2024-04-25T09:13:31.780478+0000","last_became_peered":"2024-04-25T09:13:31.780478+0000","last_unstale":"2024-04-25T09:13:32.735964+0000","last_undegraded":"2024-04-25T09:13:32.735964+0000","last_fullsized":"2024-04-25T09:13:32.735964+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:57.232055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771202+0000","last_change":"2024-04-25T09:13:31.771544+0000","last_active":"2024-04-25T09:13:32.771202+0000","last_peered":"2024-04-25T09:13:32.771202+0000","last_clean":"2024-04-25T09:13:32.771202+0000","last_became_active":"2024-04-25T09:13:31.761287+0000","last_became_peered":"2024-04-25T09:13:31.761287+0000","last_unstale":"2024-04-25T09:13:32.771202+0000","last_undegraded":"2024-04-25T09:13:32.771202+0000","last_fullsized":"2024-04-25T09:13:32.771202+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:02.500416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678486+0000","last_change":"2024-04-25T09:13:30.753476+0000","last_active":"2024-04-25T09:13:33.678486+0000","last_peered":"2024-04-25T09:13:33.678486+0000","last_clean":"2024-04-25T09:13:33.678486+0000","last_became_active":"2024-04-25T09:13:30.751179+0000","last_became_peered":"2024-04-25T09:13:30.751179+0000","last_unstale":"2024-04-25T09:13:33.678486+0000","last_undegraded":"2024-04-25T09:13:33.678486+0000","last_fullsized":"2024-04-25T09:13:33.678486+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:12.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678578+0000","last_change":"2024-04-25T09:13:31.760469+0000","last_active":"2024-04-25T09:13:33.678578+0000","last_peered":"2024-04-25T09:13:33.678578+0000","last_clean":"2024-04-25T09:13:33.678578+0000","last_became_active":"2024-04-25T09:13:31.759945+0000","last_became_peered":"2024-04-25T09:13:31.759945+0000","last_unstale":"2024-04-25T09:13:33.678578+0000","last_undegraded":"2024-04-25T09:13:33.678578+0000","last_fullsized":"2024-04-25T09:13:33.678578+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.266138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681646+0000","last_change":"2024-04-25T09:13:30.769991+0000","last_active":"2024-04-25T09:13:33.681646+0000","last_peered":"2024-04-25T09:13:33.681646+0000","last_clean":"2024-04-25T09:13:33.681646+0000","last_became_active":"2024-04-25T09:13:30.768898+0000","last_became_peered":"2024-04-25T09:13:30.768898+0000","last_unstale":"2024-04-25T09:13:33.681646+0000","last_undegraded":"2024-04-25T09:13:33.681646+0000","last_fullsized":"2024-04-25T09:13:33.681646+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:24.945887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737038+0000","last_change":"2024-04-25T09:13:31.760603+0000","last_active":"2024-04-25T09:13:32.737038+0000","last_peered":"2024-04-25T09:13:32.737038+0000","last_clean":"2024-04-25T09:13:32.737038+0000","last_became_active":"2024-04-25T09:13:31.760288+0000","last_became_peered":"2024-04-25T09:13:31.760288+0000","last_unstale":"2024-04-25T09:13:32.737038+0000","last_undegraded":"2024-04-25T09:13:32.737038+0000","last_fullsized":"2024-04-25T09:13:32.737038+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.677224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777475+0000","last_change":"2024-04-25T09:13:30.767362+0000","last_active":"2024-04-25T09:13:32.777475+0000","last_peered":"2024-04-25T09:13:32.777475+0000","last_clean":"2024-04-25T09:13:32.777475+0000","last_became_active":"2024-04-25T09:13:30.767088+0000","last_became_peered":"2024-04-25T09:13:30.767088+0000","last_unstale":"2024-04-25T09:13:32.777475+0000","last_undegraded":"2024-04-25T09:13:32.777475+0000","last_fullsized":"2024-04-25T09:13:32.777475+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:39.369419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680337+0000","last_change":"2024-04-25T09:13:31.768124+0000","last_active":"2024-04-25T09:13:33.680337+0000","last_peered":"2024-04-25T09:13:33.680337+0000","last_clean":"2024-04-25T09:13:33.680337+0000","last_became_active":"2024-04-25T09:13:31.767797+0000","last_became_peered":"2024-04-25T09:13:31.767797+0000","last_unstale":"2024-04-25T09:13:33.680337+0000","last_undegraded":"2024-04-25T09:13:33.680337+0000","last_fullsized":"2024-04-25T09:13:33.680337+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:22.359448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736919+0000","last_change":"2024-04-25T09:13:30.766810+0000","last_active":"2024-04-25T09:13:32.736919+0000","last_peered":"2024-04-25T09:13:32.736919+0000","last_clean":"2024-04-25T09:13:32.736919+0000","last_became_active":"2024-04-25T09:13:30.766347+0000","last_became_peered":"2024-04-25T09:13:30.766347+0000","last_unstale":"2024-04-25T09:13:32.736919+0000","last_undegraded":"2024-04-25T09:13:32.736919+0000","last_fullsized":"2024-04-25T09:13:32.736919+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:56.112385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.679098+0000","last_change":"2024-04-25T09:13:31.765365+0000","last_active":"2024-04-25T09:13:33.679098+0000","last_peered":"2024-04-25T09:13:33.679098+0000","last_clean":"2024-04-25T09:13:33.679098+0000","last_became_active":"2024-04-25T09:13:31.765023+0000","last_became_peered":"2024-04-25T09:13:31.765023+0000","last_unstale":"2024-04-25T09:13:33.679098+0000","last_undegraded":"2024-04-25T09:13:33.679098+0000","last_fullsized":"2024-04-25T09:13:33.679098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:18.419728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735441+0000","last_change":"2024-04-25T09:13:30.763213+0000","last_active":"2024-04-25T09:13:32.735441+0000","last_peered":"2024-04-25T09:13:32.735441+0000","last_clean":"2024-04-25T09:13:32.735441+0000","last_became_active":"2024-04-25T09:13:30.762921+0000","last_became_peered":"2024-04-25T09:13:30.762921+0000","last_unstale":"2024-04-25T09:13:32.735441+0000","last_undegraded":"2024-04-25T09:13:32.735441+0000","last_fullsized":"2024-04-25T09:13:32.735441+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:17.401527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681033+0000","last_change":"2024-04-25T09:13:31.766073+0000","last_active":"2024-04-25T09:13:33.681033+0000","last_peered":"2024-04-25T09:13:33.681033+0000","last_clean":"2024-04-25T09:13:33.681033+0000","last_became_active":"2024-04-25T09:13:31.764318+0000","last_became_peered":"2024-04-25T09:13:31.764318+0000","last_unstale":"2024-04-25T09:13:33.681033+0000","last_undegraded":"2024-04-25T09:13:33.681033+0000","last_fullsized":"2024-04-25T09:13:33.681033+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:24.790304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776452+0000","last_change":"2024-04-25T09:13:30.761921+0000","last_active":"2024-04-25T09:13:32.776452+0000","last_peered":"2024-04-25T09:13:32.776452+0000","last_clean":"2024-04-25T09:13:32.776452+0000","last_became_active":"2024-04-25T09:13:30.761545+0000","last_became_peered":"2024-04-25T09:13:30.761545+0000","last_unstale":"2024-04-25T09:13:32.776452+0000","last_undegraded":"2024-04-25T09:13:32.776452+0000","last_fullsized":"2024-04-25T09:13:32.776452+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:11.237418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.677630+0000","last_change":"2024-04-25T09:13:31.752402+0000","last_active":"2024-04-25T09:13:33.677630+0000","last_peered":"2024-04-25T09:13:33.677630+0000","last_clean":"2024-04-25T09:13:33.677630+0000","last_became_active":"2024-04-25T09:13:31.751890+0000","last_became_peered":"2024-04-25T09:13:31.751890+0000","last_unstale":"2024-04-25T09:13:33.677630+0000","last_undegraded":"2024-04-25T09:13:33.677630+0000","last_fullsized":"2024-04-25T09:13:33.677630+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:12.255619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.680990+0000","last_change":"2024-04-25T09:13:30.757062+0000","last_active":"2024-04-25T09:13:33.680990+0000","last_peered":"2024-04-25T09:13:33.680990+0000","last_clean":"2024-04-25T09:13:33.680990+0000","last_became_active":"2024-04-25T09:13:30.756455+0000","last_became_peered":"2024-04-25T09:13:30.756455+0000","last_unstale":"2024-04-25T09:13:33.680990+0000","last_undegraded":"2024-04-25T09:13:33.680990+0000","last_fullsized":"2024-04-25T09:13:33.680990+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:11.833303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678022+0000","last_change":"2024-04-25T09:13:31.769059+0000","last_active":"2024-04-25T09:13:33.678022+0000","last_peered":"2024-04-25T09:13:33.678022+0000","last_clean":"2024-04-25T09:13:33.678022+0000","last_became_active":"2024-04-25T09:13:31.768762+0000","last_became_peered":"2024-04-25T09:13:31.768762+0000","last_unstale":"2024-04-25T09:13:33.678022+0000","last_undegraded":"2024-04-25T09:13:33.678022+0000","last_fullsized":"2024-04-25T09:13:33.678022+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:46.551687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770923+0000","last_change":"2024-04-25T09:13:30.772848+0000","last_active":"2024-04-25T09:13:32.770923+0000","last_peered":"2024-04-25T09:13:32.770923+0000","last_clean":"2024-04-25T09:13:32.770923+0000","last_became_active":"2024-04-25T09:13:30.772209+0000","last_became_peered":"2024-04-25T09:13:30.772209+0000","last_unstale":"2024-04-25T09:13:32.770923+0000","last_undegraded":"2024-04-25T09:13:32.770923+0000","last_fullsized":"2024-04-25T09:13:32.770923+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:24.308630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737970+0000","last_change":"2024-04-25T09:13:31.779971+0000","last_active":"2024-04-25T09:13:32.737970+0000","last_peered":"2024-04-25T09:13:32.737970+0000","last_clean":"2024-04-25T09:13:32.737970+0000","last_became_active":"2024-04-25T09:13:31.779690+0000","last_became_peered":"2024-04-25T09:13:31.779690+0000","last_unstale":"2024-04-25T09:13:32.737970+0000","last_undegraded":"2024-04-25T09:13:32.737970+0000","last_fullsized":"2024-04-25T09:13:32.737970+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:51.743854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777822+0000","last_change":"2024-04-25T09:13:30.758834+0000","last_active":"2024-04-25T09:13:32.777822+0000","last_peered":"2024-04-25T09:13:32.777822+0000","last_clean":"2024-04-25T09:13:32.777822+0000","last_became_active":"2024-04-25T09:13:30.758446+0000","last_became_peered":"2024-04-25T09:13:30.758446+0000","last_unstale":"2024-04-25T09:13:32.777822+0000","last_undegraded":"2024-04-25T09:13:32.777822+0000","last_fullsized":"2024-04-25T09:13:32.777822+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:41.605620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731972+0000","last_change":"2024-04-25T09:13:31.760295+0000","last_active":"2024-04-25T09:13:32.731972+0000","last_peered":"2024-04-25T09:13:32.731972+0000","last_clean":"2024-04-25T09:13:32.731972+0000","last_became_active":"2024-04-25T09:13:31.759984+0000","last_became_peered":"2024-04-25T09:13:31.759984+0000","last_unstale":"2024-04-25T09:13:32.731972+0000","last_undegraded":"2024-04-25T09:13:32.731972+0000","last_fullsized":"2024-04-25T09:13:32.731972+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:36.087719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.737920+0000","last_change":"2024-04-25T09:13:30.768482+0000","last_active":"2024-04-25T09:13:32.737920+0000","last_peered":"2024-04-25T09:13:32.737920+0000","last_clean":"2024-04-25T09:13:32.737920+0000","last_became_active":"2024-04-25T09:13:30.766941+0000","last_became_peered":"2024-04-25T09:13:30.766941+0000","last_unstale":"2024-04-25T09:13:32.737920+0000","last_undegraded":"2024-04-25T09:13:32.737920+0000","last_fullsized":"2024-04-25T09:13:32.737920+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:20.373156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681794+0000","last_change":"2024-04-25T09:13:31.766013+0000","last_active":"2024-04-25T09:13:33.681794+0000","last_peered":"2024-04-25T09:13:33.681794+0000","last_clean":"2024-04-25T09:13:33.681794+0000","last_became_active":"2024-04-25T09:13:31.765713+0000","last_became_peered":"2024-04-25T09:13:31.765713+0000","last_unstale":"2024-04-25T09:13:33.681794+0000","last_undegraded":"2024-04-25T09:13:33.681794+0000","last_fullsized":"2024-04-25T09:13:33.681794+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:57.572396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682225+0000","last_change":"2024-04-25T09:13:30.751075+0000","last_active":"2024-04-25T09:13:33.682225+0000","last_peered":"2024-04-25T09:13:33.682225+0000","last_clean":"2024-04-25T09:13:33.682225+0000","last_became_active":"2024-04-25T09:13:30.750670+0000","last_became_peered":"2024-04-25T09:13:30.750670+0000","last_unstale":"2024-04-25T09:13:33.682225+0000","last_undegraded":"2024-04-25T09:13:33.682225+0000","last_fullsized":"2024-04-25T09:13:33.682225+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:45.319158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731629+0000","last_change":"2024-04-25T09:13:31.755644+0000","last_active":"2024-04-25T09:13:32.731629+0000","last_peered":"2024-04-25T09:13:32.731629+0000","last_clean":"2024-04-25T09:13:32.731629+0000","last_became_active":"2024-04-25T09:13:31.754027+0000","last_became_peered":"2024-04-25T09:13:31.754027+0000","last_unstale":"2024-04-25T09:13:32.731629+0000","last_undegraded":"2024-04-25T09:13:32.731629+0000","last_fullsized":"2024-04-25T09:13:32.731629+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:24.181807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.679632+0000","last_change":"2024-04-25T09:13:30.772429+0000","last_active":"2024-04-25T09:13:33.679632+0000","last_peered":"2024-04-25T09:13:33.679632+0000","last_clean":"2024-04-25T09:13:33.679632+0000","last_became_active":"2024-04-25T09:13:30.772072+0000","last_became_peered":"2024-04-25T09:13:30.772072+0000","last_unstale":"2024-04-25T09:13:33.679632+0000","last_undegraded":"2024-04-25T09:13:33.679632+0000","last_fullsized":"2024-04-25T09:13:33.679632+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:01.604932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736206+0000","last_change":"2024-04-25T09:13:31.768130+0000","last_active":"2024-04-25T09:13:32.736206+0000","last_peered":"2024-04-25T09:13:32.736206+0000","last_clean":"2024-04-25T09:13:32.736206+0000","last_became_active":"2024-04-25T09:13:31.767400+0000","last_became_peered":"2024-04-25T09:13:31.767400+0000","last_unstale":"2024-04-25T09:13:32.736206+0000","last_undegraded":"2024-04-25T09:13:32.736206+0000","last_fullsized":"2024-04-25T09:13:32.736206+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:35.987399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771470+0000","last_change":"2024-04-25T09:13:30.768374+0000","last_active":"2024-04-25T09:13:32.771470+0000","last_peered":"2024-04-25T09:13:32.771470+0000","last_clean":"2024-04-25T09:13:32.771470+0000","last_became_active":"2024-04-25T09:13:30.768014+0000","last_became_peered":"2024-04-25T09:13:30.768014+0000","last_unstale":"2024-04-25T09:13:32.771470+0000","last_undegraded":"2024-04-25T09:13:32.771470+0000","last_fullsized":"2024-04-25T09:13:32.771470+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:00.453307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777256+0000","last_change":"2024-04-25T09:13:31.757781+0000","last_active":"2024-04-25T09:13:32.777256+0000","last_peered":"2024-04-25T09:13:32.777256+0000","last_clean":"2024-04-25T09:13:32.777256+0000","last_became_active":"2024-04-25T09:13:31.755628+0000","last_became_peered":"2024-04-25T09:13:31.755628+0000","last_unstale":"2024-04-25T09:13:32.777256+0000","last_undegraded":"2024-04-25T09:13:32.777256+0000","last_fullsized":"2024-04-25T09:13:32.777256+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:12.507625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736467+0000","last_change":"2024-04-25T09:13:30.764781+0000","last_active":"2024-04-25T09:13:32.736467+0000","last_peered":"2024-04-25T09:13:32.736467+0000","last_clean":"2024-04-25T09:13:32.736467+0000","last_became_active":"2024-04-25T09:13:30.764484+0000","last_became_peered":"2024-04-25T09:13:30.764484+0000","last_unstale":"2024-04-25T09:13:32.736467+0000","last_undegraded":"2024-04-25T09:13:32.736467+0000","last_fullsized":"2024-04-25T09:13:32.736467+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:35.340007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682107+0000","last_change":"2024-04-25T09:13:31.760410+0000","last_active":"2024-04-25T09:13:33.682107+0000","last_peered":"2024-04-25T09:13:33.682107+0000","last_clean":"2024-04-25T09:13:33.682107+0000","last_became_active":"2024-04-25T09:13:31.760054+0000","last_became_peered":"2024-04-25T09:13:31.760054+0000","last_unstale":"2024-04-25T09:13:33.682107+0000","last_undegraded":"2024-04-25T09:13:33.682107+0000","last_fullsized":"2024-04-25T09:13:33.682107+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:09.370583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678458+0000","last_change":"2024-04-25T09:13:30.752404+0000","last_active":"2024-04-25T09:13:33.678458+0000","last_peered":"2024-04-25T09:13:33.678458+0000","last_clean":"2024-04-25T09:13:33.678458+0000","last_became_active":"2024-04-25T09:13:30.752093+0000","last_became_peered":"2024-04-25T09:13:30.752093+0000","last_unstale":"2024-04-25T09:13:33.678458+0000","last_undegraded":"2024-04-25T09:13:33.678458+0000","last_fullsized":"2024-04-25T09:13:33.678458+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:39.024037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776947+0000","last_change":"2024-04-25T09:13:31.748109+0000","last_active":"2024-04-25T09:13:32.776947+0000","last_peered":"2024-04-25T09:13:32.776947+0000","last_clean":"2024-04-25T09:13:32.776947+0000","last_became_active":"2024-04-25T09:13:31.747602+0000","last_became_peered":"2024-04-25T09:13:31.747602+0000","last_unstale":"2024-04-25T09:13:32.776947+0000","last_undegraded":"2024-04-25T09:13:32.776947+0000","last_fullsized":"2024-04-25T09:13:32.776947+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:20.690352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736543+0000","last_change":"2024-04-25T09:13:30.766653+0000","last_active":"2024-04-25T09:13:32.736543+0000","last_peered":"2024-04-25T09:13:32.736543+0000","last_clean":"2024-04-25T09:13:32.736543+0000","last_became_active":"2024-04-25T09:13:30.763878+0000","last_became_peered":"2024-04-25T09:13:30.763878+0000","last_unstale":"2024-04-25T09:13:32.736543+0000","last_undegraded":"2024-04-25T09:13:32.736543+0000","last_fullsized":"2024-04-25T09:13:32.736543+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:36.434216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.732299+0000","last_change":"2024-04-25T09:13:31.754988+0000","last_active":"2024-04-25T09:13:32.732299+0000","last_peered":"2024-04-25T09:13:32.732299+0000","last_clean":"2024-04-25T09:13:32.732299+0000","last_became_active":"2024-04-25T09:13:31.754671+0000","last_became_peered":"2024-04-25T09:13:31.754671+0000","last_unstale":"2024-04-25T09:13:32.732299+0000","last_undegraded":"2024-04-25T09:13:32.732299+0000","last_fullsized":"2024-04-25T09:13:32.732299+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:51.743854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678812+0000","last_change":"2024-04-25T09:13:30.769669+0000","last_active":"2024-04-25T09:13:33.678812+0000","last_peered":"2024-04-25T09:13:33.678812+0000","last_clean":"2024-04-25T09:13:33.678812+0000","last_became_active":"2024-04-25T09:13:30.769201+0000","last_became_peered":"2024-04-25T09:13:30.769201+0000","last_unstale":"2024-04-25T09:13:33.678812+0000","last_undegraded":"2024-04-25T09:13:33.678812+0000","last_fullsized":"2024-04-25T09:13:33.678812+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:12.216153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731940+0000","last_change":"2024-04-25T09:13:31.751894+0000","last_active":"2024-04-25T09:13:32.731940+0000","last_peered":"2024-04-25T09:13:32.731940+0000","last_clean":"2024-04-25T09:13:32.731940+0000","last_became_active":"2024-04-25T09:13:31.751329+0000","last_became_peered":"2024-04-25T09:13:31.751329+0000","last_unstale":"2024-04-25T09:13:32.731940+0000","last_undegraded":"2024-04-25T09:13:32.731940+0000","last_fullsized":"2024-04-25T09:13:32.731940+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:07.411063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682330+0000","last_change":"2024-04-25T09:13:30.750947+0000","last_active":"2024-04-25T09:13:33.682330+0000","last_peered":"2024-04-25T09:13:33.682330+0000","last_clean":"2024-04-25T09:13:33.682330+0000","last_became_active":"2024-04-25T09:13:30.749398+0000","last_became_peered":"2024-04-25T09:13:30.749398+0000","last_unstale":"2024-04-25T09:13:33.682330+0000","last_undegraded":"2024-04-25T09:13:33.682330+0000","last_fullsized":"2024-04-25T09:13:33.682330+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:19.276510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678838+0000","last_change":"2024-04-25T09:13:31.758363+0000","last_active":"2024-04-25T09:13:33.678838+0000","last_peered":"2024-04-25T09:13:33.678838+0000","last_clean":"2024-04-25T09:13:33.678838+0000","last_became_active":"2024-04-25T09:13:31.758036+0000","last_became_peered":"2024-04-25T09:13:31.758036+0000","last_unstale":"2024-04-25T09:13:33.678838+0000","last_undegraded":"2024-04-25T09:13:33.678838+0000","last_fullsized":"2024-04-25T09:13:33.678838+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:57.130586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770899+0000","last_change":"2024-04-25T09:13:30.768388+0000","last_active":"2024-04-25T09:13:32.770899+0000","last_peered":"2024-04-25T09:13:32.770899+0000","last_clean":"2024-04-25T09:13:32.770899+0000","last_became_active":"2024-04-25T09:13:30.768074+0000","last_became_peered":"2024-04-25T09:13:30.768074+0000","last_unstale":"2024-04-25T09:13:32.770899+0000","last_undegraded":"2024-04-25T09:13:32.770899+0000","last_fullsized":"2024-04-25T09:13:32.770899+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:37.398817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770569+0000","last_change":"2024-04-25T09:13:31.768360+0000","last_active":"2024-04-25T09:13:32.770569+0000","last_peered":"2024-04-25T09:13:32.770569+0000","last_clean":"2024-04-25T09:13:32.770569+0000","last_became_active":"2024-04-25T09:13:31.757296+0000","last_became_peered":"2024-04-25T09:13:31.757296+0000","last_unstale":"2024-04-25T09:13:32.770569+0000","last_undegraded":"2024-04-25T09:13:32.770569+0000","last_fullsized":"2024-04-25T09:13:32.770569+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:36.087719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.735941+0000","last_change":"2024-04-25T09:13:30.760335+0000","last_active":"2024-04-25T09:13:32.735941+0000","last_peered":"2024-04-25T09:13:32.735941+0000","last_clean":"2024-04-25T09:13:32.735941+0000","last_became_active":"2024-04-25T09:13:30.759965+0000","last_became_peered":"2024-04-25T09:13:30.759965+0000","last_unstale":"2024-04-25T09:13:32.735941+0000","last_undegraded":"2024-04-25T09:13:32.735941+0000","last_fullsized":"2024-04-25T09:13:32.735941+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:52.790415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770915+0000","last_change":"2024-04-25T09:13:31.769243+0000","last_active":"2024-04-25T09:13:32.770915+0000","last_peered":"2024-04-25T09:13:32.770915+0000","last_clean":"2024-04-25T09:13:32.770915+0000","last_became_active":"2024-04-25T09:13:31.757711+0000","last_became_peered":"2024-04-25T09:13:31.757711+0000","last_unstale":"2024-04-25T09:13:32.770915+0000","last_undegraded":"2024-04-25T09:13:32.770915+0000","last_fullsized":"2024-04-25T09:13:32.770915+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:41.959312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731330+0000","last_change":"2024-04-25T09:13:30.758510+0000","last_active":"2024-04-25T09:13:32.731330+0000","last_peered":"2024-04-25T09:13:32.731330+0000","last_clean":"2024-04-25T09:13:32.731330+0000","last_became_active":"2024-04-25T09:13:30.757982+0000","last_became_peered":"2024-04-25T09:13:30.757982+0000","last_unstale":"2024-04-25T09:13:32.731330+0000","last_undegraded":"2024-04-25T09:13:32.731330+0000","last_fullsized":"2024-04-25T09:13:32.731330+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:21.341248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.736430+0000","last_change":"2024-04-25T09:13:31.771139+0000","last_active":"2024-04-25T09:13:32.736430+0000","last_peered":"2024-04-25T09:13:32.736430+0000","last_clean":"2024-04-25T09:13:32.736430+0000","last_became_active":"2024-04-25T09:13:31.769796+0000","last_became_peered":"2024-04-25T09:13:31.769796+0000","last_unstale":"2024-04-25T09:13:32.736430+0000","last_undegraded":"2024-04-25T09:13:32.736430+0000","last_fullsized":"2024-04-25T09:13:32.736430+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:42.644077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681277+0000","last_change":"2024-04-25T09:13:30.763069+0000","last_active":"2024-04-25T09:13:33.681277+0000","last_peered":"2024-04-25T09:13:33.681277+0000","last_clean":"2024-04-25T09:13:33.681277+0000","last_became_active":"2024-04-25T09:13:30.762715+0000","last_became_peered":"2024-04-25T09:13:30.762715+0000","last_unstale":"2024-04-25T09:13:33.681277+0000","last_undegraded":"2024-04-25T09:13:33.681277+0000","last_fullsized":"2024-04-25T09:13:33.681277+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:08.062006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731556+0000","last_change":"2024-04-25T09:13:31.742003+0000","last_active":"2024-04-25T09:13:32.731556+0000","last_peered":"2024-04-25T09:13:32.731556+0000","last_clean":"2024-04-25T09:13:32.731556+0000","last_became_active":"2024-04-25T09:13:31.738182+0000","last_became_peered":"2024-04-25T09:13:31.738182+0000","last_unstale":"2024-04-25T09:13:32.731556+0000","last_undegraded":"2024-04-25T09:13:32.731556+0000","last_fullsized":"2024-04-25T09:13:32.731556+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.677224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776420+0000","last_change":"2024-04-25T09:13:30.772054+0000","last_active":"2024-04-25T09:13:32.776420+0000","last_peered":"2024-04-25T09:13:32.776420+0000","last_clean":"2024-04-25T09:13:32.776420+0000","last_became_active":"2024-04-25T09:13:30.771547+0000","last_became_peered":"2024-04-25T09:13:30.771547+0000","last_unstale":"2024-04-25T09:13:32.776420+0000","last_undegraded":"2024-04-25T09:13:32.776420+0000","last_fullsized":"2024-04-25T09:13:32.776420+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:58.707958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771510+0000","last_change":"2024-04-25T09:13:31.771862+0000","last_active":"2024-04-25T09:13:32.771510+0000","last_peered":"2024-04-25T09:13:32.771510+0000","last_clean":"2024-04-25T09:13:32.771510+0000","last_became_active":"2024-04-25T09:13:31.770395+0000","last_became_peered":"2024-04-25T09:13:31.770395+0000","last_unstale":"2024-04-25T09:13:32.771510+0000","last_undegraded":"2024-04-25T09:13:32.771510+0000","last_fullsized":"2024-04-25T09:13:32.771510+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:09.234944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678878+0000","last_change":"2024-04-25T09:13:30.759153+0000","last_active":"2024-04-25T09:13:33.678878+0000","last_peered":"2024-04-25T09:13:33.678878+0000","last_clean":"2024-04-25T09:13:33.678878+0000","last_became_active":"2024-04-25T09:13:30.758830+0000","last_became_peered":"2024-04-25T09:13:30.758830+0000","last_unstale":"2024-04-25T09:13:33.678878+0000","last_undegraded":"2024-04-25T09:13:33.678878+0000","last_fullsized":"2024-04-25T09:13:33.678878+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:44.753604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771896+0000","last_change":"2024-04-25T09:13:31.773186+0000","last_active":"2024-04-25T09:13:32.771896+0000","last_peered":"2024-04-25T09:13:32.771896+0000","last_clean":"2024-04-25T09:13:32.771896+0000","last_became_active":"2024-04-25T09:13:31.770150+0000","last_became_peered":"2024-04-25T09:13:31.770150+0000","last_unstale":"2024-04-25T09:13:32.771896+0000","last_undegraded":"2024-04-25T09:13:32.771896+0000","last_fullsized":"2024-04-25T09:13:32.771896+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:55.723506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"109'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771897+0000","last_change":"2024-04-25T09:13:30.767521+0000","last_active":"2024-04-25T09:13:32.771897+0000","last_peered":"2024-04-25T09:13:32.771897+0000","last_clean":"2024-04-25T09:13:32.771897+0000","last_became_active":"2024-04-25T09:13:30.766649+0000","last_became_peered":"2024-04-25T09:13:30.766649+0000","last_unstale":"2024-04-25T09:13:32.771897+0000","last_undegraded":"2024-04-25T09:13:32.771897+0000","last_fullsized":"2024-04-25T09:13:32.771897+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:19.915302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777552+0000","last_change":"2024-04-25T09:13:31.755552+0000","last_active":"2024-04-25T09:13:32.777552+0000","last_peered":"2024-04-25T09:13:32.777552+0000","last_clean":"2024-04-25T09:13:32.777552+0000","last_became_active":"2024-04-25T09:13:31.755236+0000","last_became_peered":"2024-04-25T09:13:31.755236+0000","last_unstale":"2024-04-25T09:13:32.777552+0000","last_undegraded":"2024-04-25T09:13:32.777552+0000","last_fullsized":"2024-04-25T09:13:32.777552+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:38.417018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771265+0000","last_change":"2024-04-25T09:13:30.765673+0000","last_active":"2024-04-25T09:13:32.771265+0000","last_peered":"2024-04-25T09:13:32.771265+0000","last_clean":"2024-04-25T09:13:32.771265+0000","last_became_active":"2024-04-25T09:13:30.765362+0000","last_became_peered":"2024-04-25T09:13:30.765362+0000","last_unstale":"2024-04-25T09:13:32.771265+0000","last_undegraded":"2024-04-25T09:13:32.771265+0000","last_fullsized":"2024-04-25T09:13:32.771265+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:39.493995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.682179+0000","last_change":"2024-04-25T09:13:31.763659+0000","last_active":"2024-04-25T09:13:33.682179+0000","last_peered":"2024-04-25T09:13:33.682179+0000","last_clean":"2024-04-25T09:13:33.682179+0000","last_became_active":"2024-04-25T09:13:31.763182+0000","last_became_peered":"2024-04-25T09:13:31.763182+0000","last_unstale":"2024-04-25T09:13:33.682179+0000","last_undegraded":"2024-04-25T09:13:33.682179+0000","last_fullsized":"2024-04-25T09:13:33.682179+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:08.567402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.732085+0000","last_change":"2024-04-25T09:13:30.755732+0000","last_active":"2024-04-25T09:13:32.732085+0000","last_peered":"2024-04-25T09:13:32.732085+0000","last_clean":"2024-04-25T09:13:32.732085+0000","last_became_active":"2024-04-25T09:13:30.755311+0000","last_became_peered":"2024-04-25T09:13:30.755311+0000","last_unstale":"2024-04-25T09:13:32.732085+0000","last_undegraded":"2024-04-25T09:13:32.732085+0000","last_fullsized":"2024-04-25T09:13:32.732085+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:11.489424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.771490+0000","last_change":"2024-04-25T09:13:31.768786+0000","last_active":"2024-04-25T09:13:32.771490+0000","last_peered":"2024-04-25T09:13:32.771490+0000","last_clean":"2024-04-25T09:13:32.771490+0000","last_became_active":"2024-04-25T09:13:31.768427+0000","last_became_peered":"2024-04-25T09:13:31.768427+0000","last_unstale":"2024-04-25T09:13:32.771490+0000","last_undegraded":"2024-04-25T09:13:32.771490+0000","last_fullsized":"2024-04-25T09:13:32.771490+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:39.445673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777588+0000","last_change":"2024-04-25T09:13:30.762156+0000","last_active":"2024-04-25T09:13:32.777588+0000","last_peered":"2024-04-25T09:13:32.777588+0000","last_clean":"2024-04-25T09:13:32.777588+0000","last_became_active":"2024-04-25T09:13:30.761176+0000","last_became_peered":"2024-04-25T09:13:30.761176+0000","last_unstale":"2024-04-25T09:13:32.777588+0000","last_undegraded":"2024-04-25T09:13:32.777588+0000","last_fullsized":"2024-04-25T09:13:32.777588+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:29.433527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.681164+0000","last_change":"2024-04-25T09:13:31.756281+0000","last_active":"2024-04-25T09:13:33.681164+0000","last_peered":"2024-04-25T09:13:33.681164+0000","last_clean":"2024-04-25T09:13:33.681164+0000","last_became_active":"2024-04-25T09:13:31.755824+0000","last_became_peered":"2024-04-25T09:13:31.755824+0000","last_unstale":"2024-04-25T09:13:33.681164+0000","last_undegraded":"2024-04-25T09:13:33.681164+0000","last_fullsized":"2024-04-25T09:13:33.681164+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:27.101533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.731507+0000","last_change":"2024-04-25T09:13:30.768177+0000","last_active":"2024-04-25T09:13:32.731507+0000","last_peered":"2024-04-25T09:13:32.731507+0000","last_clean":"2024-04-25T09:13:32.731507+0000","last_became_active":"2024-04-25T09:13:30.767744+0000","last_became_peered":"2024-04-25T09:13:30.767744+0000","last_unstale":"2024-04-25T09:13:32.731507+0000","last_undegraded":"2024-04-25T09:13:32.731507+0000","last_fullsized":"2024-04-25T09:13:32.731507+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:10.366780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.770870+0000","last_change":"2024-04-25T09:13:31.765477+0000","last_active":"2024-04-25T09:13:32.770870+0000","last_peered":"2024-04-25T09:13:32.770870+0000","last_clean":"2024-04-25T09:13:32.770870+0000","last_became_active":"2024-04-25T09:13:31.765181+0000","last_became_peered":"2024-04-25T09:13:31.765181+0000","last_unstale":"2024-04-25T09:13:32.770870+0000","last_undegraded":"2024-04-25T09:13:32.770870+0000","last_fullsized":"2024-04-25T09:13:32.770870+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:51.743854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.677744+0000","last_change":"2024-04-25T09:13:30.758745+0000","last_active":"2024-04-25T09:13:33.677744+0000","last_peered":"2024-04-25T09:13:33.677744+0000","last_clean":"2024-04-25T09:13:33.677744+0000","last_became_active":"2024-04-25T09:13:30.758378+0000","last_became_peered":"2024-04-25T09:13:30.758378+0000","last_unstale":"2024-04-25T09:13:33.677744+0000","last_undegraded":"2024-04-25T09:13:33.677744+0000","last_fullsized":"2024-04-25T09:13:33.677744+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:14.514322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678132+0000","last_change":"2024-04-25T09:13:31.765240+0000","last_active":"2024-04-25T09:13:33.678132+0000","last_peered":"2024-04-25T09:13:33.678132+0000","last_clean":"2024-04-25T09:13:33.678132+0000","last_became_active":"2024-04-25T09:13:31.764923+0000","last_became_peered":"2024-04-25T09:13:31.764923+0000","last_unstale":"2024-04-25T09:13:33.678132+0000","last_undegraded":"2024-04-25T09:13:33.678132+0000","last_fullsized":"2024-04-25T09:13:33.678132+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.266138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.776688+0000","last_change":"2024-04-25T09:13:30.764575+0000","last_active":"2024-04-25T09:13:32.776688+0000","last_peered":"2024-04-25T09:13:32.776688+0000","last_clean":"2024-04-25T09:13:32.776688+0000","last_became_active":"2024-04-25T09:13:30.764286+0000","last_became_peered":"2024-04-25T09:13:30.764286+0000","last_unstale":"2024-04-25T09:13:32.776688+0000","last_undegraded":"2024-04-25T09:13:32.776688+0000","last_fullsized":"2024-04-25T09:13:32.776688+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:01.389513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:32.777835+0000","last_change":"2024-04-25T09:13:31.770097+0000","last_active":"2024-04-25T09:13:32.777835+0000","last_peered":"2024-04-25T09:13:32.777835+0000","last_clean":"2024-04-25T09:13:32.777835+0000","last_became_active":"2024-04-25T09:13:31.769811+0000","last_became_peered":"2024-04-25T09:13:31.769811+0000","last_unstale":"2024-04-25T09:13:32.777835+0000","last_undegraded":"2024-04-25T09:13:32.777835+0000","last_fullsized":"2024-04-25T09:13:32.777835+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:30.717889+0000","last_clean_scrub_stamp":"2024-04-25T09:13:30.717889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:01.471507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678545+0000","last_change":"2024-04-25T09:13:30.766905+0000","last_active":"2024-04-25T09:13:33.678545+0000","last_peered":"2024-04-25T09:13:33.678545+0000","last_clean":"2024-04-25T09:13:33.678545+0000","last_became_active":"2024-04-25T09:13:30.766629+0000","last_became_peered":"2024-04-25T09:13:30.766629+0000","last_unstale":"2024-04-25T09:13:33.678545+0000","last_undegraded":"2024-04-25T09:13:33.678545+0000","last_fullsized":"2024-04-25T09:13:33.678545+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:13:29.699688+0000","last_clean_scrub_stamp":"2024-04-25T09:13:29.699688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:01.547908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":332,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T09:13:33.678026+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:13:33.678026+0000","last_peered":"2024-04-25T09:13:33.678026+0000","last_clean":"2024-04-25T09:13:33.678026+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:13:33.678026+0000","last_undegraded":"2024-04-25T09:13:33.678026+0000","last_fullsized":"2024-04-25T09:13:33.678026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50552,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321288,"statfs":{"total":96636764160,"available":96584998912,"internally_reserved":0,"allocated":1351680,"data_stored":590294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50552,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321288,"statfs":{"total":96636764160,"available":96584998912,"internally_reserved":0,"allocated":1347584,"data_stored":589712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574978,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":1181748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574978,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54712,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317128,"statfs":{"total":96636764160,"available":96580739072,"internally_reserved":0,"allocated":1351680,"data_stored":590294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54716,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317124,"statfs":{"total":96636764160,"available":96580734976,"internally_reserved":0,"allocated":1359872,"data_stored":589906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76924,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294916,"statfs":{"total":96636764160,"available":96557993984,"internally_reserved":0,"allocated":1351680,"data_stored":589782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574978,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51144,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320696,"statfs":{"total":96636764160,"available":96584392704,"internally_reserved":0,"allocated":1961984,"data_stored":1181804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574977,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28336,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343504,"statfs":{"total":96636764160,"available":96607748096,"internally_reserved":0,"allocated":1347584,"data_stored":589712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:13:36.701 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:13:36.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:13:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.980+0000 7f01f2600640 1 Processor -- start 2024-04-25T09:13:36.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.980+0000 7f01f2600640 1 -- start start 2024-04-25T09:13:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.981+0000 7f01f2600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec15dc60 0x7f01ec1618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.981+0000 7f01f2600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01ec161e00 0x7f01ec0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.981+0000 7f01f2600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0b6bf0 con 0x7f01ec0ced60 2024-04-25T09:13:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.981+0000 7f01f2600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0cf1b0 con 0x7f01ec161e00 2024-04-25T09:13:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.981+0000 7f01f2600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0d1b40 con 0x7f01ec15dc60 2024-04-25T09:13:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01ebe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01ec161e00 0x7f01ec0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01ebe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01ec161e00 0x7f01ec0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55806/0 (socket says 172.21.15.62:55806) 2024-04-25T09:13:36.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01ebe00640 1 -- 172.21.15.62:0/2652982036 learned_addr learned my addr 172.21.15.62:0/2652982036 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01f1c00640 1 --1- 172.21.15.62:0/2652982036 >> v1:172.21.15.62:6789/0 conn(0x7f01ec0ced60 0x7f01ec0b54d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:55608/0 (socket says 172.21.15.62:55608) 2024-04-25T09:13:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01f1200640 1 --2- 172.21.15.62:0/2652982036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec15dc60 0x7f01ec1618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01ebe00640 1 -- 172.21.15.62:0/2652982036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec15dc60 msgr2=0x7f01ec1618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01ebe00640 1 --2- 172.21.15.62:0/2652982036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec15dc60 0x7f01ec1618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01ebe00640 1 -- 172.21.15.62:0/2652982036 >> v1:172.21.15.62:6789/0 conn(0x7f01ec0ced60 legacy=0x7f01ec0b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01ebe00640 1 -- 172.21.15.62:0/2652982036 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec162380 con 0x7f01ec161e00 2024-04-25T09:13:36.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.982+0000 7f01eb400640 1 -- 172.21.15.62:0/2652982036 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305169042 0 0) 0x7f01e4001490 con 0x7f01ec0ced60 2024-04-25T09:13:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.984+0000 7f01ebe00640 1 --2- 172.21.15.62:0/2652982036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0ce820 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009870 tx=0x7f01d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feba1568dadf39c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.984+0000 7f01eb400640 1 -- 172.21.15.62:0/2652982036 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8037070 con 0x7f01ec161e00 2024-04-25T09:13:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.984+0000 7f01eb400640 1 -- 172.21.15.62:0/2652982036 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8032440 con 0x7f01ec161e00 2024-04-25T09:13:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.984+0000 7f01eb400640 1 -- 172.21.15.62:0/2652982036 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8032810 con 0x7f01ec161e00 2024-04-25T09:13:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.984+0000 7f01f2600640 1 -- 172.21.15.62:0/2652982036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 msgr2=0x7f01ec0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.984+0000 7f01f2600640 1 --2- 172.21.15.62:0/2652982036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0ce820 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009870 tx=0x7f01d802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.985+0000 7f01f2600640 1 -- 172.21.15.62:0/2652982036 shutdown_connections 2024-04-25T09:13:36.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.985+0000 7f01f2600640 1 --2- 172.21.15.62:0/2652982036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0ce820 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.985+0000 7f01f2600640 1 --2- 172.21.15.62:0/2652982036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec15dc60 0x7f01ec1618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.985+0000 7f01f2600640 1 -- 172.21.15.62:0/2652982036 >> 172.21.15.62:0/2652982036 conn(0x7f01ec0668f0 msgr2=0x7f01ec066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.985+0000 7f01f2600640 1 -- 172.21.15.62:0/2652982036 shutdown_connections 2024-04-25T09:13:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.985+0000 7f01f2600640 1 -- 172.21.15.62:0/2652982036 wait complete. 2024-04-25T09:13:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.986+0000 7f01f2600640 1 Processor -- start 2024-04-25T09:13:36.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.987+0000 7f01f2600640 1 -- start start 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.987+0000 7f01f2600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec0ced60 0x7f01ec1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.987+0000 7f01f2600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec15dc60 0x7f01ec160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.987+0000 7f01f2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.987+0000 7f01f2600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0b9670 con 0x7f01ec161e00 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.987+0000 7f01f2600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0d1d90 con 0x7f01ec15dc60 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.987+0000 7f01f2600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec15e040 con 0x7f01ec0ced60 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55810/0 (socket says 172.21.15.62:55810) 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 -- 172.21.15.62:0/2195053370 learned_addr learned my addr 172.21.15.62:0/2195053370 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1200640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec0ced60 0x7f01ec1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 -- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec0ced60 msgr2=0x7f01ec1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec0ced60 0x7f01ec1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 -- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec15dc60 msgr2=0x7f01ec160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:36.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec15dc60 0x7f01ec160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:36.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 -- 172.21.15.62:0/2195053370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0cc670 con 0x7f01ec161e00 2024-04-25T09:13:36.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.988+0000 7f01f1c00640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0b3d50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c700 tx=0x7f01e400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.989+0000 7f01e8c00640 1 -- 172.21.15.62:0/2195053370 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4018070 con 0x7f01ec161e00 2024-04-25T09:13:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.989+0000 7f01e8c00640 1 -- 172.21.15.62:0/2195053370 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e400ce10 con 0x7f01ec161e00 2024-04-25T09:13:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.989+0000 7f01e8c00640 1 -- 172.21.15.62:0/2195053370 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e400a860 con 0x7f01ec161e00 2024-04-25T09:13:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.989+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec0cc950 con 0x7f01ec161e00 2024-04-25T09:13:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.989+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec0ccf10 con 0x7f01ec161e00 2024-04-25T09:13:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.990+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec075070 con 0x7f01ec161e00 2024-04-25T09:13:36.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.991+0000 7f01e8c00640 1 -- 172.21.15.62:0/2195053370 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01e400aa00 con 0x7f01ec161e00 2024-04-25T09:13:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.992+0000 7f01e8c00640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f01cc06b4d0 0x7f01cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.993+0000 7f01e8c00640 1 -- 172.21.15.62:0/2195053370 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01e4089800 con 0x7f01ec161e00 2024-04-25T09:13:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.993+0000 7f01f1200640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f01cc06b4d0 0x7f01cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.994+0000 7f01f1200640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f01cc06b4d0 0x7f01cc06d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f01dc006fd0 tx=0x7f01dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:36.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:36.997+0000 7f01e8c00640 1 -- 172.21.15.62:0/2195053370 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01ec0b6bf0 con 0x7f01ec161e00 2024-04-25T09:13:37.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.263+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f01ec0cfc40 con 0x7f01ec161e00 2024-04-25T09:13:37.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.268+0000 7f01e8c00640 1 -- 172.21.15.62:0/2195053370 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f01e4058670 con 0x7f01ec161e00 2024-04-25T09:13:37.255 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:13:37.256 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:13:37.256 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:13:37.256 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:13:37.256 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:13:37.256 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f01cc06b4d0 msgr2=0x7f01cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f01cc06b4d0 0x7f01cc06d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f01dc006fd0 tx=0x7f01dc008040 comp rx=0 tx=0).stop 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 msgr2=0x7f01ec0b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0b3d50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c700 tx=0x7f01e400cbc0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 shutdown_connections 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f01cc06b4d0 0x7f01cc06d990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01ec161e00 0x7f01ec0b3d50 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01ec15dc60 0x7f01ec160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 --2- 172.21.15.62:0/2195053370 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01ec0ced60 0x7f01ec1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 >> 172.21.15.62:0/2195053370 conn(0x7f01ec0668f0 msgr2=0x7f01ec15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:37.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 shutdown_connections 2024-04-25T09:13:37.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.271+0000 7f01f2600640 1 -- 172.21.15.62:0/2195053370 wait complete. 2024-04-25T09:13:37.278 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:13:37.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.496+0000 7f0c0b200640 1 Processor -- start 2024-04-25T09:13:37.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.496+0000 7f0c0b200640 1 -- start start 2024-04-25T09:13:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.498+0000 7f0c0b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 0x7f0c0c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.498+0000 7f0c0b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0c0c0cee40 0x7f0c0c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.498+0000 7f0c0b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c15faa0 con 0x7f0c0c06d730 2024-04-25T09:13:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.498+0000 7f0c0b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c144ff0 con 0x7f0c0c0cee40 2024-04-25T09:13:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.498+0000 7f0c0b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c160340 con 0x7f0c0c144c10 2024-04-25T09:13:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.498+0000 7f0c0a800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0c0c06d730 0x7f0c0c0b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:55614/0 (socket says 172.21.15.62:55614) 2024-04-25T09:13:37.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.498+0000 7f0c0a800640 1 -- 172.21.15.62:0/4274969135 learned_addr learned my addr 172.21.15.62:0/4274969135 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.498+0000 7f0c09e00640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 0x7f0c0c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c03e00640 1 --2- 172.21.15.62:0/4274969135 >> v2:172.21.15.121:3300/0 conn(0x7f0c0c0cee40 0x7f0c0c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005666269 0 0) 0x7f0c0c15faa0 con 0x7f0c0c06d730 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf0003520 con 0x7f0c0c06d730 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1619860178 0 0) 0x7f0bf0003520 con 0x7f0c0c06d730 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 msgr2=0x7f0c0c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 0x7f0c0c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 >> v2:172.21.15.121:3300/0 conn(0x7f0c0c0cee40 msgr2=0x7f0c0c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 --2- 172.21.15.62:0/4274969135 >> v2:172.21.15.121:3300/0 conn(0x7f0c0c0cee40 0x7f0c0c06d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c15fd80 con 0x7f0c0c06d730 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f0c040019f0 con 0x7f0c0c06d730 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09e00640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 0x7f0c0c15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 >> v1:172.21.15.62:6789/0 conn(0x7f0c0c06d730 legacy=0x7f0c0c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 0x7f0bf00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0bf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf0004d40 0x7f0bf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c144ff0 con 0x7f0bf0003ef0 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c160340 con 0x7f0bf0004d40 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.499+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf000b9c0 con 0x7f0c0c144c10 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.500+0000 7f0c03e00640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0bf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.500+0000 7f0c09e00640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 0x7f0bf00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.500+0000 7f0c03e00640 1 -- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 msgr2=0x7f0bf00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.500+0000 7f0c03e00640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 0x7f0bf00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.500+0000 7f0c03e00640 1 -- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf0004d40 msgr2=0x7f0bf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:37.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.500+0000 7f0c03e00640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf0004d40 0x7f0bf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.500+0000 7f0c03e00640 1 -- 172.21.15.62:0/4274969135 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0003520 con 0x7f0bf0003ef0 2024-04-25T09:13:37.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.500+0000 7f0c09e00640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c144c10 0x7f0bf00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.501+0000 7f0c03e00640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0bf0004800 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009ac0 tx=0x7f0bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e486f3309d77da11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.501+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4036070 con 0x7f0bf0003ef0 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.501+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf4031400 con 0x7f0bf0003ef0 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.501+0000 7f0c09400640 1 -- 172.21.15.62:0/4274969135 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4009690 con 0x7f0bf0003ef0 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.502+0000 7f0c0b200640 1 -- 172.21.15.62:0/4274969135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 msgr2=0x7f0bf0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.502+0000 7f0c0b200640 1 --2- 172.21.15.62:0/4274969135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0bf0004800 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009ac0 tx=0x7f0bf0005830 comp rx=0 tx=0).stop 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.502+0000 7f0c0a800640 1 -- 172.21.15.62:0/4274969135 reap_dead start 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.502+0000 7f0c0b200640 1 -- 172.21.15.62:0/4274969135 shutdown_connections 2024-04-25T09:13:37.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.502+0000 7f0c0b200640 1 -- 172.21.15.62:0/4274969135 >> 172.21.15.62:0/4274969135 conn(0x7f0c0c0b14d0 msgr2=0x7f0c0c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:37.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.502+0000 7f0c0b200640 1 -- 172.21.15.62:0/4274969135 shutdown_connections 2024-04-25T09:13:37.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.502+0000 7f0c0b200640 1 -- 172.21.15.62:0/4274969135 wait complete. 2024-04-25T09:13:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.504+0000 7f0c0b200640 1 Processor -- start 2024-04-25T09:13:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.504+0000 7f0c0b200640 1 -- start start 2024-04-25T09:13:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c0b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0c0c1e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c0b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c0c9f00 0x7f0c0c142b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c0b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c0c143050 0x7f0c0c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c0b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0cf2a0 con 0x7f0bf0003ef0 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c0b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0b41c0 con 0x7f0c0c143050 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c0b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0b5660 con 0x7f0c0c0c9f00 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c09e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0c0c1e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c09e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0c0c1e6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55834/0 (socket says 172.21.15.62:55834) 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.505+0000 7f0c09e00640 1 -- 172.21.15.62:0/1295096305 learned_addr learned my addr 172.21.15.62:0/1295096305 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c09e00640 1 -- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c0c9f00 msgr2=0x7f0c0c142b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c09e00640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c0c9f00 0x7f0c0c142b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c09e00640 1 -- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c0c143050 msgr2=0x7f0c0c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c09e00640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c0c143050 0x7f0c0c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c09e00640 1 -- 172.21.15.62:0/1295096305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0ca8e0 con 0x7f0bf0003ef0 2024-04-25T09:13:37.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c09e00640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0c0c1e6c10 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc010810 tx=0x7f0bfc010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:37.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c02000640 1 -- 172.21.15.62:0/1295096305 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00f040 con 0x7f0bf0003ef0 2024-04-25T09:13:37.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c02000640 1 -- 172.21.15.62:0/1295096305 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc00eb80 con 0x7f0bf0003ef0 2024-04-25T09:13:37.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c02000640 1 -- 172.21.15.62:0/1295096305 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc01c400 con 0x7f0bf0003ef0 2024-04-25T09:13:37.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.506+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c0cabc0 con 0x7f0bf0003ef0 2024-04-25T09:13:37.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.507+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c0caea0 con 0x7f0bf0003ef0 2024-04-25T09:13:37.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.508+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c06d1f0 con 0x7f0bf0003ef0 2024-04-25T09:13:37.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.509+0000 7f0c02000640 1 -- 172.21.15.62:0/1295096305 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bfc01c5a0 con 0x7f0bf0003ef0 2024-04-25T09:13:37.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.509+0000 7f0c02000640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0be806b500 0x7f0be806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:37.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.510+0000 7f0c02000640 1 -- 172.21.15.62:0/1295096305 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bfc08a760 con 0x7f0bf0003ef0 2024-04-25T09:13:37.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.510+0000 7f0c03e00640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0be806b500 0x7f0be806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:37.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.512+0000 7f0c03e00640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0be806b500 0x7f0be806d9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4033000 tx=0x7f0bf402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:37.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.514+0000 7f0c02000640 1 -- 172.21.15.62:0/1295096305 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c0c06d1f0 con 0x7f0bf0003ef0 2024-04-25T09:13:37.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.746+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c0c04ef20 con 0x7f0bf0003ef0 2024-04-25T09:13:37.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.748+0000 7f0c02000640 1 -- 172.21.15.62:0/1295096305 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0bfc018030 con 0x7f0bf0003ef0 2024-04-25T09:13:37.735 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:13:37.735 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:13:37.736 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:13:37.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0be806b500 msgr2=0x7f0be806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0be806b500 0x7f0be806d9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4033000 tx=0x7f0bf402e000 comp rx=0 tx=0).stop 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 msgr2=0x7f0c0c1e6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0c0c1e6c10 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc010810 tx=0x7f0bfc010cd0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 shutdown_connections 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0be806b500 0x7f0be806d9c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c0c143050 0x7f0c0c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c0c0c9f00 0x7f0c0c142b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 --2- 172.21.15.62:0/1295096305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bf0003ef0 0x7f0c0c1e6c10 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 >> 172.21.15.62:0/1295096305 conn(0x7f0c0c0b14d0 msgr2=0x7f0c0c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:37.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 shutdown_connections 2024-04-25T09:13:37.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:37.750+0000 7f0c0b200640 1 -- 172.21.15.62:0/1295096305 wait complete. 2024-04-25T09:13:38.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:13:31.220283+0000', 'modified': '2024-04-25T09:13:34.273693+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7800}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7800': {'gid': 7800, 'name': 'd', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.121:6836/2489379768', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 2489379768}, {'type': 'v1', 'addr': '172.21.15.121:6836', 'nonce': 2489379768}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:13:38.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:13:38.159 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:13:38.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.389+0000 7f3e1a400640 1 Processor -- start 2024-04-25T09:13:38.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.389+0000 7f3e1a400640 1 -- start start 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.390+0000 7f3e1a400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3e1415c4e0 0x7f3e1415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.390+0000 7f3e1a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cf5f0 0x7f3e140b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.390+0000 7f3e1a400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3e140cfa70 0x7f3e140b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.390+0000 7f3e1a400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e140cff60 con 0x7f3e140cfa70 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.390+0000 7f3e1a400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e14070360 con 0x7f3e1415c4e0 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.390+0000 7f3e1a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14070640 con 0x7f3e140cf5f0 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3e140cfa70 0x7f3e140b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3e140cfa70 0x7f3e140b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55838/0 (socket says 172.21.15.62:55838) 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19a00640 1 -- 172.21.15.62:0/1163190582 learned_addr learned my addr 172.21.15.62:0/1163190582 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19000640 1 --2- 172.21.15.62:0/1163190582 >> v2:172.21.15.121:3300/0 conn(0x7f3e1415c4e0 0x7f3e1415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e0be00640 1 --2- 172.21.15.62:0/1163190582 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cf5f0 0x7f3e140b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19a00640 1 -- 172.21.15.62:0/1163190582 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cf5f0 msgr2=0x7f3e140b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19a00640 1 --2- 172.21.15.62:0/1163190582 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cf5f0 0x7f3e140b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19a00640 1 -- 172.21.15.62:0/1163190582 >> v2:172.21.15.121:3300/0 conn(0x7f3e1415c4e0 msgr2=0x7f3e1415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19a00640 1 --2- 172.21.15.62:0/1163190582 >> v2:172.21.15.121:3300/0 conn(0x7f3e1415c4e0 0x7f3e1415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.391+0000 7f3e19a00640 1 -- 172.21.15.62:0/1163190582 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14070920 con 0x7f3e140cfa70 2024-04-25T09:13:38.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.393+0000 7f3e19a00640 1 --2- 172.21.15.62:0/1163190582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cfa70 0x7f3e140b4810 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3e140677a0 tx=0x7f3e0c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce37e55eb68b0cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.393+0000 7f3e13e00640 1 -- 172.21.15.62:0/1163190582 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c036070 con 0x7f3e140cfa70 2024-04-25T09:13:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.393+0000 7f3e13e00640 1 -- 172.21.15.62:0/1163190582 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c031600 con 0x7f3e140cfa70 2024-04-25T09:13:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.393+0000 7f3e13e00640 1 -- 172.21.15.62:0/1163190582 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0319d0 con 0x7f3e140cfa70 2024-04-25T09:13:38.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 -- 172.21.15.62:0/1163190582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cfa70 msgr2=0x7f3e140b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 --2- 172.21.15.62:0/1163190582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cfa70 0x7f3e140b4810 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f3e140677a0 tx=0x7f3e0c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 -- 172.21.15.62:0/1163190582 shutdown_connections 2024-04-25T09:13:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 --2- 172.21.15.62:0/1163190582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cfa70 0x7f3e140b4810 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 --2- 172.21.15.62:0/1163190582 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cf5f0 0x7f3e140b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 --2- 172.21.15.62:0/1163190582 >> v2:172.21.15.121:3300/0 conn(0x7f3e1415c4e0 0x7f3e1415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 -- 172.21.15.62:0/1163190582 >> 172.21.15.62:0/1163190582 conn(0x7f3e140668f0 msgr2=0x7f3e14066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 -- 172.21.15.62:0/1163190582 shutdown_connections 2024-04-25T09:13:38.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.394+0000 7f3e1a400640 1 -- 172.21.15.62:0/1163190582 wait complete. 2024-04-25T09:13:38.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.396+0000 7f3e1a400640 1 Processor -- start 2024-04-25T09:13:38.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.396+0000 7f3e1a400640 1 -- start start 2024-04-25T09:13:38.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.397+0000 7f3e1a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cf5f0 0x7f3e141f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.397+0000 7f3e1a400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cfa70 0x7f3e141550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.397+0000 7f3e1a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e1415c4e0 0x7f3e14158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.397+0000 7f3e1a400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e14071120 con 0x7f3e140cf5f0 2024-04-25T09:13:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.397+0000 7f3e1a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1415ce00 con 0x7f3e1415c4e0 2024-04-25T09:13:38.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.397+0000 7f3e1a400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1415f520 con 0x7f3e140cfa70 2024-04-25T09:13:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cf5f0 0x7f3e141f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cf5f0 0x7f3e141f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55842/0 (socket says 172.21.15.62:55842) 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 -- 172.21.15.62:0/4145255468 learned_addr learned my addr 172.21.15.62:0/4145255468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e1415c4e0 0x7f3e14158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e0be00640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cfa70 0x7f3e141550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 -- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cfa70 msgr2=0x7f3e141550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cfa70 0x7f3e141550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 -- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e1415c4e0 msgr2=0x7f3e14158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e1415c4e0 0x7f3e14158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 -- 172.21.15.62:0/4145255468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140b14b0 con 0x7f3e140cf5f0 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.398+0000 7f3e19000640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cf5f0 0x7f3e141f5390 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400d5f0 tx=0x7f3e0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.399+0000 7f3e11600640 1 -- 172.21.15.62:0/4145255468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e040125f0 con 0x7f3e140cf5f0 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.399+0000 7f3e11600640 1 -- 172.21.15.62:0/4145255468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e04019070 con 0x7f3e140cf5f0 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.399+0000 7f3e11600640 1 -- 172.21.15.62:0/4145255468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e040129d0 con 0x7f3e140cf5f0 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.399+0000 7f3e1a400640 1 -- 172.21.15.62:0/4145255468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e140b1790 con 0x7f3e140cf5f0 2024-04-25T09:13:38.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.399+0000 7f3e1a400640 1 -- 172.21.15.62:0/4145255468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e140b1d50 con 0x7f3e140cf5f0 2024-04-25T09:13:38.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.401+0000 7f3e11600640 1 -- 172.21.15.62:0/4145255468 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e04012b70 con 0x7f3e140cf5f0 2024-04-25T09:13:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.402+0000 7f3e11600640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3df006f8e0 0x7f3df0071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.402+0000 7f3e11600640 1 -- 172.21.15.62:0/4145255468 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e0408a5d0 con 0x7f3e140cf5f0 2024-04-25T09:13:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.403+0000 7f3e1e970740 1 Processor -- start 2024-04-25T09:13:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.403+0000 7f3e0be00640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3df006f8e0 0x7f3df0071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.403+0000 7f3e1e970740 1 -- start start 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.403+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46ba9b30 0x55bd46bd1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.403+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46ba9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bf27a0 0x55bd46ba82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e1e970740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd46b0d620 con 0x55bd46bd1eb0 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e1e970740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd46ab6ca0 con 0x55bd46bf27a0 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e1e970740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd46b096c0 con 0x55bd46ba9b30 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46ba9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46ba9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55856/0 (socket says 172.21.15.62:55856) 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 -- 172.21.15.62:0/2270164595 learned_addr learned my addr 172.21.15.62:0/2270164595 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3df006f8e0 0x7f3df0071da0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3e141f48e0 tx=0x7f3dfc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e19a00640 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bf27a0 0x55bd46ba82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e19000640 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46ba9b30 0x55bd46bd1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 -- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46ba9b30 msgr2=0x55bd46bd1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46ba9b30 0x55bd46bd1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 -- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bf27a0 msgr2=0x55bd46ba82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bf27a0 0x55bd46ba82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e0be00640 1 -- 172.21.15.62:0/2270164595 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd46abdf40 con 0x55bd46bd1eb0 2024-04-25T09:13:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.404+0000 7f3e19000640 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46ba9b30 0x55bd46bd1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:38.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.405+0000 7f3e0be00640 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46ba9f10 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00fa70 tx=0x7f3dfc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143580a6384cba03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.405+0000 7f3e09600640 1 -- 172.21.15.62:0/2270164595 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc01b050 con 0x55bd46bd1eb0 2024-04-25T09:13:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.405+0000 7f3e09600640 1 -- 172.21.15.62:0/2270164595 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc025070 con 0x55bd46bd1eb0 2024-04-25T09:13:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.405+0000 7f3e09600640 1 -- 172.21.15.62:0/2270164595 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc020440 con 0x55bd46bd1eb0 2024-04-25T09:13:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 -- 172.21.15.62:0/2270164595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 msgr2=0x55bd46ba9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46ba9f10 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00fa70 tx=0x7f3dfc012000 comp rx=0 tx=0).stop 2024-04-25T09:13:38.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 -- 172.21.15.62:0/2270164595 shutdown_connections 2024-04-25T09:13:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46ba9f10 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bf27a0 0x55bd46ba82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2270164595 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46ba9b30 0x55bd46bd1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 -- 172.21.15.62:0/2270164595 >> 172.21.15.62:0/2270164595 conn(0x55bd46ba6d50 msgr2=0x55bd46bef730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 -- 172.21.15.62:0/2270164595 shutdown_connections 2024-04-25T09:13:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 -- 172.21.15.62:0/2270164595 wait complete. 2024-04-25T09:13:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.406+0000 7f3e1e970740 1 -- start start 2024-04-25T09:13:38.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.407+0000 7f3e1e970740 1 Processor -- start 2024-04-25T09:13:38.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46beeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bf27a0 0x55bd46bc6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 0x55bd46bcfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e1e970740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd46bd14e0 con 0x55bd46bd1eb0 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e1e970740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd46aac1f0 con 0x55bd46bcd950 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e1e970740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd46bd0ed0 con 0x55bd46bf27a0 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e1e970740 10 client.0 _ll_register_callbacks cb 0x55bd46ac9ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46beeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46beeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55858/0 (socket says 172.21.15.62:55858) 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19000640 1 -- 172.21.15.62:0/2091469014 learned_addr learned my addr 172.21.15.62:0/2091469014 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e0be00640 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bf27a0 0x55bd46bc6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19000640 1 -- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bf27a0 msgr2=0x55bd46bc6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19a00640 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 0x55bd46bcfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19000640 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bf27a0 0x55bd46bc6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19000640 1 -- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 msgr2=0x55bd46bcfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19000640 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 0x55bd46bcfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.408+0000 7f3e19000640 1 -- 172.21.15.62:0/2091469014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd46acbc50 con 0x55bd46bd1eb0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e19000640 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46beeba0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3e04088a00 tx=0x7f3e04059510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279044af42da1377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e02000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e02000640 1 -- 172.21.15.62:0/2091469014 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e040613c0 con 0x55bd46bd1eb0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e02000640 1 -- 172.21.15.62:0/2091469014 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e04055320 con 0x55bd46bd1eb0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e1e970740 1 -- 172.21.15.62:0/2091469014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dfc025070 con 0x55bd46bd1eb0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e02000640 1 -- 172.21.15.62:0/2091469014 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e04058100 con 0x55bd46bd1eb0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e1e970740 10 client.7926 fetch_fsmap learned FSMap version 71 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e1e970740 1 -- 172.21.15.62:0/2091469014 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55bd468222d0 con 0x55bd46bd1eb0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.409+0000 7f3e02000640 1 -- 172.21.15.62:0/2091469014 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc025070 con 0x55bd46bd1eb0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.410+0000 7f3e02000640 1 -- 172.21.15.62:0/2091469014 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e04013070 con 0x55bd46bd1eb0 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.410+0000 7f3e1e970740 10 client.7926 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.410+0000 7f3e1e970740 10 client.7926 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:13:38.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.410+0000 7f3e1e970740 20 client.7926 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.410+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bd46c02db0 0x55bd46c03190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.410+0000 7f3e1e970740 4 client.7926 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:13:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.410+0000 7f3e1e970740 1 -- 172.21.15.62:0/2091469014 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd46b1ab60 con 0x55bd46c02db0 2024-04-25T09:13:38.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.410+0000 7f3e0be00640 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bd46c02db0 0x55bd46c03190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.411+0000 7f3e0be00640 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bd46c02db0 0x55bd46c03190 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7061cee68bff44 server_cookie=1df26cacb5b05161 in_seq=0 out_seq=0 2024-04-25T09:13:38.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.411+0000 7f3e02000640 10 client.7926 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:13:38.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e02000640 1 -- 172.21.15.62:0/2091469014 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd46b1ab60 con 0x55bd46c02db0 2024-04-25T09:13:38.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e02000640 10 client.7926 handle_command_reply: tid=0 2024-04-25T09:13:38.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 client.7926 shutdown 2024-04-25T09:13:38.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 -- 172.21.15.62:0/2091469014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 msgr2=0x55bd46beeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46beeba0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f3e04088a00 tx=0x7f3e04059510 comp rx=0 tx=0).stop 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 -- 172.21.15.62:0/2091469014 shutdown_connections 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bd46c02db0 0x55bd46c03190 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 0x55bd46bcfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bf27a0 0x55bd46bc6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2091469014 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bd1eb0 0x55bd46beeba0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e02000640 0 client.7926 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 -- 172.21.15.62:0/2091469014 >> 172.21.15.62:0/2091469014 conn(0x55bd46ba9b30 msgr2=0x55bd46ba38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.413+0000 7f3e1e970740 1 -- 172.21.15.62:0/2091469014 shutdown_connections 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.414+0000 7f3e1e970740 1 -- 172.21.15.62:0/2091469014 wait complete. 2024-04-25T09:13:38.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.414+0000 7f3e1e970740 20 client.7926 trim_cache size 0 max 16384 2024-04-25T09:13:38.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.431+0000 7f3e1e970740 1 Processor -- start 2024-04-25T09:13:38.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.431+0000 7f3e1e970740 1 -- start start 2024-04-25T09:13:38.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 0x55bd46beba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bd1eb0 0x55bd46bc8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46ba3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e1e970740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1406d520 con 0x55bd46bf27a0 2024-04-25T09:13:38.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e1e970740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140afc60 con 0x55bd46bcd950 2024-04-25T09:13:38.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e1e970740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140b0260 con 0x55bd46bd1eb0 2024-04-25T09:13:38.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46ba3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46ba3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55866/0 (socket says 172.21.15.62:55866) 2024-04-25T09:13:38.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e19a00640 1 -- 172.21.15.62:0/1922344062 learned_addr learned my addr 172.21.15.62:0/1922344062 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e19000640 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 0x55bd46beba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e0be00640 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bd1eb0 0x55bd46bc8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e19a00640 1 -- 172.21.15.62:0/1922344062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bd1eb0 msgr2=0x55bd46bc8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.432+0000 7f3e19a00640 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bd1eb0 0x55bd46bc8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.433+0000 7f3e19a00640 1 -- 172.21.15.62:0/1922344062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 msgr2=0x55bd46beba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.433+0000 7f3e19a00640 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 0x55bd46beba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.433+0000 7f3e19a00640 1 -- 172.21.15.62:0/1922344062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd468ec6b0 con 0x55bd46bf27a0 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.433+0000 7f3e19a00640 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46ba3b50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c02a6e0 tx=0x7f3e0c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2861451b72a9a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.433+0000 7f3e08c00640 1 -- 172.21.15.62:0/1922344062 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c036040 con 0x55bd46bf27a0 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.433+0000 7f3e08c00640 1 -- 172.21.15.62:0/1922344062 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c037040 con 0x55bd46bf27a0 2024-04-25T09:13:38.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.433+0000 7f3e08c00640 1 -- 172.21.15.62:0/1922344062 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c03d870 con 0x55bd46bf27a0 2024-04-25T09:13:38.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 -- 172.21.15.62:0/1922344062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 msgr2=0x55bd46ba3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46ba3b50 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c02a6e0 tx=0x7f3e0c02e000 comp rx=0 tx=0).stop 2024-04-25T09:13:38.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 -- 172.21.15.62:0/1922344062 shutdown_connections 2024-04-25T09:13:38.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46ba3b50 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bd1eb0 0x55bd46bc8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 --2- 172.21.15.62:0/1922344062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bcd950 0x55bd46beba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 -- 172.21.15.62:0/1922344062 >> 172.21.15.62:0/1922344062 conn(0x55bd46ba9b30 msgr2=0x55bd46b9ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:38.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 -- 172.21.15.62:0/1922344062 shutdown_connections 2024-04-25T09:13:38.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.434+0000 7f3e1e970740 1 -- 172.21.15.62:0/1922344062 wait complete. 2024-04-25T09:13:38.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.435+0000 7f3e1e970740 1 -- start start 2024-04-25T09:13:38.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.435+0000 7f3e1e970740 1 Processor -- start 2024-04-25T09:13:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bcd950 0x55bd46c575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bd1eb0 0x55bd46bcc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e1e970740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46bcc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e1e970740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd46bc6500 con 0x55bd46bf27a0 2024-04-25T09:13:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e1e970740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd46bc6d90 con 0x55bd46bd1eb0 2024-04-25T09:13:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e1e970740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd46bcddd0 con 0x55bd46bcd950 2024-04-25T09:13:38.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e1e970740 10 client.0 _ll_register_callbacks cb 0x7f3e04058680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46bcc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46bcc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55868/0 (socket says 172.21.15.62:55868) 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19a00640 1 -- 172.21.15.62:0/2154224817 learned_addr learned my addr 172.21.15.62:0/2154224817 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19000640 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bcd950 0x55bd46c575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e0be00640 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bd1eb0 0x55bd46bcc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19a00640 1 -- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bcd950 msgr2=0x55bd46c575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19a00640 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bcd950 0x55bd46c575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19a00640 1 -- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bd1eb0 msgr2=0x55bd46bcc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19a00640 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bd1eb0 0x55bd46bcc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e19a00640 1 -- 172.21.15.62:0/2154224817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd46bf19c0 con 0x55bd46bf27a0 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.436+0000 7f3e0be00640 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bd1eb0 0x55bd46bcc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.437+0000 7f3e19a00640 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46bcc930 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009f40 tx=0x7f3e0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2c030a2849bfc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.437+0000 7f3e02000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.437+0000 7f3e02000640 1 -- 172.21.15.62:0/2154224817 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c04e070 con 0x55bd46bf27a0 2024-04-25T09:13:38.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.437+0000 7f3e02000640 1 -- 172.21.15.62:0/2154224817 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e0c004450 con 0x55bd46bf27a0 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.437+0000 7f3e1e970740 1 -- 172.21.15.62:0/2154224817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e0c037040 con 0x55bd46bf27a0 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.437+0000 7f3e02000640 1 -- 172.21.15.62:0/2154224817 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e0c059050 con 0x55bd46bf27a0 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.437+0000 7f3e1e970740 10 client.7932 fetch_fsmap learned FSMap version 71 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.437+0000 7f3e1e970740 1 -- 172.21.15.62:0/2154224817 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55bd46ba4260 con 0x55bd46bf27a0 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e02000640 1 -- 172.21.15.62:0/2154224817 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c037040 con 0x55bd46bf27a0 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e02000640 1 -- 172.21.15.62:0/2154224817 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e0c05c020 con 0x55bd46bf27a0 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e1e970740 10 client.7932 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e1e970740 10 client.7932 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e1e970740 20 client.7932 populate_metadata read hostname 'smithi062' 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bd46bcfd20 0x55bd46ba82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e1e970740 4 client.7932 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:13:38.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e1e970740 1 -- 172.21.15.62:0/2154224817 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bd46bc9800 con 0x55bd46bcfd20 2024-04-25T09:13:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.438+0000 7f3e19000640 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bd46bcfd20 0x55bd46ba82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.439+0000 7f3e19000640 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bd46bcfd20 0x55bd46ba82f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af25725dca5bea05 server_cookie=5a29fb990be3684f in_seq=0 out_seq=0 2024-04-25T09:13:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.439+0000 7f3e02000640 10 client.7932 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:13:38.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e02000640 1 -- 172.21.15.62:0/2154224817 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55bd46bc9800 con 0x55bd46bcfd20 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e02000640 10 client.7932 handle_command_reply: tid=0 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 client.7932 shutdown 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 -- 172.21.15.62:0/2154224817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 msgr2=0x55bd46bcc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46bcc930 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009f40 tx=0x7f3e0c004ab0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 -- 172.21.15.62:0/2154224817 shutdown_connections 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bd46bcfd20 0x55bd46ba82f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bd46bf27a0 0x55bd46bcc930 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bd46bd1eb0 0x55bd46bcc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 --2- 172.21.15.62:0/2154224817 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bd46bcd950 0x55bd46c575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e02000640 0 client.7932 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 -- 172.21.15.62:0/2154224817 >> 172.21.15.62:0/2154224817 conn(0x55bd46ba9b30 msgr2=0x55bd46ba01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 -- 172.21.15.62:0/2154224817 shutdown_connections 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.440+0000 7f3e1e970740 1 -- 172.21.15.62:0/2154224817 wait complete. 2024-04-25T09:13:38.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1e970740 20 client.7932 trim_cache size 0 max 16384 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 7800, 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 6.686012893, 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 71, 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 109, 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 109, 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 13.107199162000001 2024-04-25T09:13:38.430 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:13:38.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 -- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3df006f8e0 msgr2=0x7f3df0071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3df006f8e0 0x7f3df0071da0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3e141f48e0 tx=0x7f3dfc008040 comp rx=0 tx=0).stop 2024-04-25T09:13:38.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 -- 172.21.15.62:0/4145255468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cf5f0 msgr2=0x7f3e141f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cf5f0 0x7f3e141f5390 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400d5f0 tx=0x7f3e0400dab0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 -- 172.21.15.62:0/4145255468 shutdown_connections 2024-04-25T09:13:38.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3df006f8e0 0x7f3df0071da0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e1415c4e0 0x7f3e14158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e140cfa70 0x7f3e141550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 --2- 172.21.15.62:0/4145255468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e140cf5f0 0x7f3e141f5390 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 -- 172.21.15.62:0/4145255468 >> 172.21.15.62:0/4145255468 conn(0x7f3e140668f0 msgr2=0x7f3e140ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:38.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.441+0000 7f3e1a400640 1 -- 172.21.15.62:0/4145255468 shutdown_connections 2024-04-25T09:13:38.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.442+0000 7f3e1a400640 1 -- 172.21.15.62:0/4145255468 wait complete. 2024-04-25T09:13:38.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:13:38.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.652+0000 7fcff1e00640 1 Processor -- start 2024-04-25T09:13:38.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.653+0000 7fcff1e00640 1 -- start start 2024-04-25T09:13:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.654+0000 7fcff1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec158180 0x7fcfec15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcff1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcfec06e5e0 0x7fcfec0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcff1e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec15f0f0 con 0x7fcfec06ed90 2024-04-25T09:13:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcff1e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec0b3b40 con 0x7fcfec06e5e0 2024-04-25T09:13:38.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcff1e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec071400 con 0x7fcfec158180 2024-04-25T09:13:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcff1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcfec06e5e0 0x7fcfec0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcff1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcfec06e5e0 0x7fcfec0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55882/0 (socket says 172.21.15.62:55882) 2024-04-25T09:13:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcff1400640 1 -- 172.21.15.62:0/421289745 learned_addr learned my addr 172.21.15.62:0/421289745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:38.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcfebe00640 1 --1- 172.21.15.62:0/421289745 >> v1:172.21.15.62:6789/0 conn(0x7fcfec06ed90 0x7fcfec06ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:55624/0 (socket says 172.21.15.62:55624) 2024-04-25T09:13:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.655+0000 7fcff0a00640 1 --2- 172.21.15.62:0/421289745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec158180 0x7fcfec15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.656+0000 7fcff1400640 1 -- 172.21.15.62:0/421289745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec158180 msgr2=0x7fcfec15ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:13:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.656+0000 7fcff1400640 1 --2- 172.21.15.62:0/421289745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec158180 0x7fcfec15ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.656+0000 7fcff1400640 1 -- 172.21.15.62:0/421289745 >> v1:172.21.15.62:6789/0 conn(0x7fcfec06ed90 legacy=0x7fcfec06ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.656+0000 7fcff1400640 1 -- 172.21.15.62:0/421289745 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0716a0 con 0x7fcfec06e5e0 2024-04-25T09:13:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.657+0000 7fcff1400640 1 --2- 172.21.15.62:0/421289745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec06e5e0 0x7fcfec0b3fd0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009870 tx=0x7fcfe402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92825fd110dba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.657+0000 7fcfeb400640 1 -- 172.21.15.62:0/421289745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4037070 con 0x7fcfec06e5e0 2024-04-25T09:13:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.657+0000 7fcfeb400640 1 -- 172.21.15.62:0/421289745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe4032440 con 0x7fcfec06e5e0 2024-04-25T09:13:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.657+0000 7fcfeb400640 1 -- 172.21.15.62:0/421289745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4032810 con 0x7fcfec06e5e0 2024-04-25T09:13:38.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.658+0000 7fcff1e00640 1 -- 172.21.15.62:0/421289745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec06e5e0 msgr2=0x7fcfec0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.658+0000 7fcff1e00640 1 --2- 172.21.15.62:0/421289745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec06e5e0 0x7fcfec0b3fd0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009870 tx=0x7fcfe402a250 comp rx=0 tx=0).stop 2024-04-25T09:13:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.658+0000 7fcff1e00640 1 -- 172.21.15.62:0/421289745 shutdown_connections 2024-04-25T09:13:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.658+0000 7fcff1e00640 1 --2- 172.21.15.62:0/421289745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec06e5e0 0x7fcfec0b3fd0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.658+0000 7fcff1e00640 1 --2- 172.21.15.62:0/421289745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec158180 0x7fcfec15ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.658+0000 7fcff1e00640 1 -- 172.21.15.62:0/421289745 >> 172.21.15.62:0/421289745 conn(0x7fcfec06b350 msgr2=0x7fcfec0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.658+0000 7fcff1e00640 1 -- 172.21.15.62:0/421289745 shutdown_connections 2024-04-25T09:13:38.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.658+0000 7fcff1e00640 1 -- 172.21.15.62:0/421289745 wait complete. 2024-04-25T09:13:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.660+0000 7fcff1e00640 1 Processor -- start 2024-04-25T09:13:38.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.660+0000 7fcff1e00640 1 -- start start 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcff1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec06e5e0 0x7fcfec1465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcff1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec158180 0x7fcfec152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcff1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfec0ae930 0x7fcfec0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcff1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec071f40 con 0x7fcfec158180 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcff1e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec071c60 con 0x7fcfec0ae930 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcff1e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec158560 con 0x7fcfec06e5e0 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcfebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec158180 0x7fcfec152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcfebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec158180 0x7fcfec152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55896/0 (socket says 172.21.15.62:55896) 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcfebe00640 1 -- 172.21.15.62:0/2092844054 learned_addr learned my addr 172.21.15.62:0/2092844054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcfebe00640 1 -- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec06e5e0 msgr2=0x7fcfec1465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcff0a00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec06e5e0 0x7fcfec1465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcff1400640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfec0ae930 0x7fcfec0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcfebe00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec06e5e0 0x7fcfec1465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcfebe00640 1 -- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfec0ae930 msgr2=0x7fcfec0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcfebe00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfec0ae930 0x7fcfec0aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.661+0000 7fcfebe00640 1 -- 172.21.15.62:0/2092844054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec152aa0 con 0x7fcfec158180 2024-04-25T09:13:38.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.662+0000 7fcfebe00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec158180 0x7fcfec152e60 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400c7f0 tx=0x7fcfd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.662+0000 7fcfe8c00640 1 -- 172.21.15.62:0/2092844054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4017070 con 0x7fcfec158180 2024-04-25T09:13:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.662+0000 7fcfe8c00640 1 -- 172.21.15.62:0/2092844054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd400a640 con 0x7fcfec158180 2024-04-25T09:13:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.662+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec154110 con 0x7fcfec158180 2024-04-25T09:13:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.662+0000 7fcfe8c00640 1 -- 172.21.15.62:0/2092844054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd400a9f0 con 0x7fcfec158180 2024-04-25T09:13:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.662+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec154690 con 0x7fcfec158180 2024-04-25T09:13:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.663+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec04ef20 con 0x7fcfec158180 2024-04-25T09:13:38.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.664+0000 7fcfe8c00640 1 -- 172.21.15.62:0/2092844054 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfd401a070 con 0x7fcfec158180 2024-04-25T09:13:38.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.665+0000 7fcfe8c00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcfc806b500 0x7fcfc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:13:38.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.665+0000 7fcfe8c00640 1 -- 172.21.15.62:0/2092844054 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfd4089800 con 0x7fcfec158180 2024-04-25T09:13:38.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.666+0000 7fcff0a00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcfc806b500 0x7fcfc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:13:38.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.667+0000 7fcff0a00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcfc806b500 0x7fcfc806d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009a10 tx=0x7fcfdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:13:38.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.670+0000 7fcfe8c00640 1 -- 172.21.15.62:0/2092844054 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfec15f0f0 con 0x7fcfec158180 2024-04-25T09:13:38.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.911+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcfec06ea90 con 0x7fcfec158180 2024-04-25T09:13:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.912+0000 7fcfe8c00640 1 -- 172.21.15.62:0/2092844054 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcfd4057f00 con 0x7fcfec158180 2024-04-25T09:13:38.900 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:13:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcfc806b500 msgr2=0x7fcfc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcfc806b500 0x7fcfc806d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009a10 tx=0x7fcfdc008040 comp rx=0 tx=0).stop 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec158180 msgr2=0x7fcfec152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec158180 0x7fcfec152e60 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400c7f0 tx=0x7fcfd400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 shutdown_connections 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcfc806b500 0x7fcfc806d9c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfec158180 0x7fcfec152e60 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfec0ae930 0x7fcfec0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 --2- 172.21.15.62:0/2092844054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfec06e5e0 0x7fcfec1465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 >> 172.21.15.62:0/2092844054 conn(0x7fcfec06b350 msgr2=0x7fcfec15f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:13:38.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.915+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 shutdown_connections 2024-04-25T09:13:38.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:13:38.916+0000 7fcff1e00640 1 -- 172.21.15.62:0/2092844054 wait complete. 2024-04-25T09:13:38.922 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:13:38.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:13:38.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:13:38.922 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:13:38.922 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:13:38.922 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:13:38.922 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:13:38.923 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:13:38.923 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:13:38.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:38.923 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:13:38.975 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42040sec preferred_lft 42040sec 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:13:38.976 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:13:38.977 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:13:38.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:38.978 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:13:39.054 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:13:39.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:39.059 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:13:39.111 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:13:39.111 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:13:39.111 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:13:39.112 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:13:39.112 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:13:39.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:39.113 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:13:39.113 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:39.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:13:39.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:13:39.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:13:39.113 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:39.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:39.273 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:13:39.323 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:13:39.323 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:13:39.324 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:13:39.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:39.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:13:39.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:13:39.413 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:13:39.466 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:13:39.466 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:13:39.466 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:13:39.520 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:13:39.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:39.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:13:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:13:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:13:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:13:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:13:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:13:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:13:39.693 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:13:39.696 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:13:39.696 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:13:39.696 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:13:39.696 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:13:39.696 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:13:39.701 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:13:39.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:39.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:13:39.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:39.784 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:13:39.856 INFO:teuthology.orchestra.run.smithi062.stdout:client7944 2024-04-25T09:13:39.856 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:13:39.856 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:13:39.914 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:13:39.914 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T09:13:39.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:39.914 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:39.915 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:39.915 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:13:39.915 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:39.915 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:39.915 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:39.915 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:40.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.038 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340101 s, 30.8 MB/s 2024-04-25T09:13:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.068 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255257 s, 41.1 MB/s 2024-04-25T09:13:40.108 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.108 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.108 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360223 s, 29.1 MB/s 2024-04-25T09:13:40.142 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.142 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.142 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292028 s, 35.9 MB/s 2024-04-25T09:13:40.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.171 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236249 s, 44.4 MB/s 2024-04-25T09:13:40.206 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.206 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.206 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307437 s, 34.1 MB/s 2024-04-25T09:13:40.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.253 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440486 s, 23.8 MB/s 2024-04-25T09:13:40.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.291 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337929 s, 31.0 MB/s 2024-04-25T09:13:40.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.320 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242605 s, 43.2 MB/s 2024-04-25T09:13:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.352 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284762 s, 36.8 MB/s 2024-04-25T09:13:40.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.377 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213254 s, 49.2 MB/s 2024-04-25T09:13:40.403 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.404 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221588 s, 47.3 MB/s 2024-04-25T09:13:40.430 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.430 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.430 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021735 s, 48.2 MB/s 2024-04-25T09:13:40.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.464 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029882 s, 35.1 MB/s 2024-04-25T09:13:40.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.493 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253574 s, 41.4 MB/s 2024-04-25T09:13:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.525 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.526 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282546 s, 37.1 MB/s 2024-04-25T09:13:40.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242871 s, 43.2 MB/s 2024-04-25T09:13:40.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245799 s, 42.7 MB/s 2024-04-25T09:13:40.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.611 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230881 s, 45.4 MB/s 2024-04-25T09:13:40.641 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.641 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.641 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244495 s, 42.9 MB/s 2024-04-25T09:13:40.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231372 s, 45.3 MB/s 2024-04-25T09:13:40.716 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.716 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.716 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0444376 s, 23.6 MB/s 2024-04-25T09:13:40.744 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.744 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.744 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237228 s, 44.2 MB/s 2024-04-25T09:13:40.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031273 s, 33.5 MB/s 2024-04-25T09:13:40.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277479 s, 37.8 MB/s 2024-04-25T09:13:40.839 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.839 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.839 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234272 s, 44.8 MB/s 2024-04-25T09:13:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.865 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220474 s, 47.6 MB/s 2024-04-25T09:13:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282735 s, 37.1 MB/s 2024-04-25T09:13:40.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252879 s, 41.5 MB/s 2024-04-25T09:13:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207177 s, 50.6 MB/s 2024-04-25T09:13:40.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:40.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:40.981 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255476 s, 41.0 MB/s 2024-04-25T09:13:41.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.011 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262489 s, 39.9 MB/s 2024-04-25T09:13:41.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:41.014 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:41.014 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:41.014 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:13:41.014 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:41.014 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:41.014 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:41.014 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:41.102 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023406 s, 44.8 MB/s 2024-04-25T09:13:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.132 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253264 s, 41.4 MB/s 2024-04-25T09:13:41.158 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.158 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.158 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230057 s, 45.6 MB/s 2024-04-25T09:13:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218154 s, 48.1 MB/s 2024-04-25T09:13:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.210 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220684 s, 47.5 MB/s 2024-04-25T09:13:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.238 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238418 s, 44.0 MB/s 2024-04-25T09:13:41.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.263 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209667 s, 50.0 MB/s 2024-04-25T09:13:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.287 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207061 s, 50.6 MB/s 2024-04-25T09:13:41.315 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.315 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.315 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238468 s, 44.0 MB/s 2024-04-25T09:13:41.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.340 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204813 s, 51.2 MB/s 2024-04-25T09:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.369 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025153 s, 41.7 MB/s 2024-04-25T09:13:41.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.400 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275101 s, 38.1 MB/s 2024-04-25T09:13:41.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.433 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028349 s, 37.0 MB/s 2024-04-25T09:13:41.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.462 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250375 s, 41.9 MB/s 2024-04-25T09:13:41.491 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.492 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256527 s, 40.9 MB/s 2024-04-25T09:13:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.523 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265451 s, 39.5 MB/s 2024-04-25T09:13:41.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.546 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020004 s, 52.4 MB/s 2024-04-25T09:13:41.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.573 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228857 s, 45.8 MB/s 2024-04-25T09:13:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.601 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241691 s, 43.4 MB/s 2024-04-25T09:13:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224728 s, 46.7 MB/s 2024-04-25T09:13:41.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250089 s, 41.9 MB/s 2024-04-25T09:13:41.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.689 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276756 s, 37.9 MB/s 2024-04-25T09:13:41.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.723 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295617 s, 35.5 MB/s 2024-04-25T09:13:41.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.751 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241372 s, 43.4 MB/s 2024-04-25T09:13:41.783 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.783 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.783 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276237 s, 38.0 MB/s 2024-04-25T09:13:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232719 s, 45.1 MB/s 2024-04-25T09:13:41.838 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.838 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.838 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231473 s, 45.3 MB/s 2024-04-25T09:13:41.867 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.867 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.867 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253791 s, 41.3 MB/s 2024-04-25T09:13:41.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.894 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223136 s, 47.0 MB/s 2024-04-25T09:13:41.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.920 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223081 s, 47.0 MB/s 2024-04-25T09:13:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260236 s, 40.3 MB/s 2024-04-25T09:13:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271413 s, 38.6 MB/s 2024-04-25T09:13:41.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:41.984 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:41.984 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:41.984 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:13:41.984 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:41.985 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:41.985 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:41.985 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:42.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.074 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218086 s, 48.1 MB/s 2024-04-25T09:13:42.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.106 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279941 s, 37.5 MB/s 2024-04-25T09:13:42.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.131 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208118 s, 50.4 MB/s 2024-04-25T09:13:42.160 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.160 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.160 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249835 s, 42.0 MB/s 2024-04-25T09:13:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.189 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025226 s, 41.6 MB/s 2024-04-25T09:13:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.217 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235523 s, 44.5 MB/s 2024-04-25T09:13:42.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.240 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197138 s, 53.2 MB/s 2024-04-25T09:13:42.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.269 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255851 s, 41.0 MB/s 2024-04-25T09:13:42.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.293 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198681 s, 52.8 MB/s 2024-04-25T09:13:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.323 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254832 s, 41.1 MB/s 2024-04-25T09:13:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227937 s, 46.0 MB/s 2024-04-25T09:13:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217379 s, 48.2 MB/s 2024-04-25T09:13:42.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.401 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.401 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217463 s, 48.2 MB/s 2024-04-25T09:13:42.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.426 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212549 s, 49.3 MB/s 2024-04-25T09:13:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212317 s, 49.4 MB/s 2024-04-25T09:13:42.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.480 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241247 s, 43.5 MB/s 2024-04-25T09:13:42.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.510 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259371 s, 40.4 MB/s 2024-04-25T09:13:42.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.535 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212974 s, 49.2 MB/s 2024-04-25T09:13:42.563 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.563 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.563 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229481 s, 45.7 MB/s 2024-04-25T09:13:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.591 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023858 s, 44.0 MB/s 2024-04-25T09:13:42.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240903 s, 43.5 MB/s 2024-04-25T09:13:42.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.652 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028526 s, 36.8 MB/s 2024-04-25T09:13:42.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.677 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208338 s, 50.3 MB/s 2024-04-25T09:13:42.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.709 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264025 s, 39.7 MB/s 2024-04-25T09:13:42.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237533 s, 44.1 MB/s 2024-04-25T09:13:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.764 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235092 s, 44.6 MB/s 2024-04-25T09:13:42.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.795 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255323 s, 41.1 MB/s 2024-04-25T09:13:42.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.825 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257848 s, 40.7 MB/s 2024-04-25T09:13:42.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.851 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224506 s, 46.7 MB/s 2024-04-25T09:13:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.878 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218966 s, 47.9 MB/s 2024-04-25T09:13:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268074 s, 39.1 MB/s 2024-04-25T09:13:42.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:42.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228518 s, 45.9 MB/s 2024-04-25T09:13:42.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:42.940 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:42.940 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:42.940 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:13:42.940 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:42.940 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:42.940 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:42.940 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:43.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.027 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201175 s, 52.1 MB/s 2024-04-25T09:13:43.053 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.053 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.053 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020973 s, 50.0 MB/s 2024-04-25T09:13:43.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.078 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204436 s, 51.3 MB/s 2024-04-25T09:13:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.102 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020444 s, 51.3 MB/s 2024-04-25T09:13:43.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.127 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214274 s, 48.9 MB/s 2024-04-25T09:13:43.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017519 s, 59.9 MB/s 2024-04-25T09:13:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023117 s, 45.4 MB/s 2024-04-25T09:13:43.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.204 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242237 s, 43.3 MB/s 2024-04-25T09:13:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.231 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210409 s, 49.8 MB/s 2024-04-25T09:13:43.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.262 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02733 s, 38.4 MB/s 2024-04-25T09:13:43.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.287 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211111 s, 49.7 MB/s 2024-04-25T09:13:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.313 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021435 s, 48.9 MB/s 2024-04-25T09:13:43.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.338 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211808 s, 49.5 MB/s 2024-04-25T09:13:43.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.364 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021349 s, 49.1 MB/s 2024-04-25T09:13:43.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.390 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224578 s, 46.7 MB/s 2024-04-25T09:13:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.416 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216184 s, 48.5 MB/s 2024-04-25T09:13:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231959 s, 45.2 MB/s 2024-04-25T09:13:43.470 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.471 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230014 s, 45.6 MB/s 2024-04-25T09:13:43.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.498 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224341 s, 46.7 MB/s 2024-04-25T09:13:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.528 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257699 s, 40.7 MB/s 2024-04-25T09:13:43.556 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.557 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247252 s, 42.4 MB/s 2024-04-25T09:13:43.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.586 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250961 s, 41.8 MB/s 2024-04-25T09:13:43.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.611 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212195 s, 49.4 MB/s 2024-04-25T09:13:43.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.645 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030802 s, 34.0 MB/s 2024-04-25T09:13:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.669 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191054 s, 54.9 MB/s 2024-04-25T09:13:43.698 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.698 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.698 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230021 s, 45.6 MB/s 2024-04-25T09:13:43.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215271 s, 48.7 MB/s 2024-04-25T09:13:43.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.750 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217807 s, 48.1 MB/s 2024-04-25T09:13:43.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.776 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218242 s, 48.0 MB/s 2024-04-25T09:13:43.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265317 s, 39.5 MB/s 2024-04-25T09:13:43.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.832 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212704 s, 49.3 MB/s 2024-04-25T09:13:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.857 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213479 s, 49.1 MB/s 2024-04-25T09:13:43.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:43.860 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:43.860 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:43.860 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:13:43.860 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:43.860 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:43.860 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:43.860 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:43.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228176 s, 46.0 MB/s 2024-04-25T09:13:43.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:43.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:43.981 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258234 s, 40.6 MB/s 2024-04-25T09:13:44.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.007 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212087 s, 49.4 MB/s 2024-04-25T09:13:44.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.034 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240466 s, 43.6 MB/s 2024-04-25T09:13:44.058 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.058 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203482 s, 51.5 MB/s 2024-04-25T09:13:44.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.085 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.085 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222519 s, 47.1 MB/s 2024-04-25T09:13:44.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.115 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265522 s, 39.5 MB/s 2024-04-25T09:13:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.143 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237672 s, 44.1 MB/s 2024-04-25T09:13:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.169 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213118 s, 49.2 MB/s 2024-04-25T09:13:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236732 s, 44.3 MB/s 2024-04-25T09:13:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257922 s, 40.7 MB/s 2024-04-25T09:13:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021266 s, 49.3 MB/s 2024-04-25T09:13:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.276 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194791 s, 53.8 MB/s 2024-04-25T09:13:44.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.303 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220094 s, 47.6 MB/s 2024-04-25T09:13:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.332 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245488 s, 42.7 MB/s 2024-04-25T09:13:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.357 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212679 s, 49.3 MB/s 2024-04-25T09:13:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.382 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203047 s, 51.6 MB/s 2024-04-25T09:13:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.404 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189436 s, 55.4 MB/s 2024-04-25T09:13:44.431 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.432 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236767 s, 44.3 MB/s 2024-04-25T09:13:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.457 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021654 s, 48.4 MB/s 2024-04-25T09:13:44.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.482 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021245 s, 49.4 MB/s 2024-04-25T09:13:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.505 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187605 s, 55.9 MB/s 2024-04-25T09:13:44.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.534 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247993 s, 42.3 MB/s 2024-04-25T09:13:44.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.567 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291168 s, 36.0 MB/s 2024-04-25T09:13:44.576 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:44.584 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:13:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.598 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266713 s, 39.3 MB/s 2024-04-25T09:13:44.616 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.617 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160645 s, 65.3 MB/s 2024-04-25T09:13:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209009 s, 50.2 MB/s 2024-04-25T09:13:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.665 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214207 s, 49.0 MB/s 2024-04-25T09:13:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.691 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228159 s, 46.0 MB/s 2024-04-25T09:13:44.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.712 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175255 s, 59.8 MB/s 2024-04-25T09:13:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246161 s, 42.6 MB/s 2024-04-25T09:13:44.762 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.762 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.762 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180325 s, 58.1 MB/s 2024-04-25T09:13:44.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:44.764 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:44.764 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:44.764 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:13:44.764 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:44.764 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:44.765 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:44.765 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:44.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195521 s, 53.6 MB/s 2024-04-25T09:13:44.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.923 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230098 s, 45.6 MB/s 2024-04-25T09:13:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256467 s, 40.9 MB/s 2024-04-25T09:13:44.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:44.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:44.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222257 s, 47.2 MB/s 2024-04-25T09:13:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.007 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233821 s, 44.8 MB/s 2024-04-25T09:13:45.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.035 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225531 s, 46.5 MB/s 2024-04-25T09:13:45.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228434 s, 45.9 MB/s 2024-04-25T09:13:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186913 s, 56.1 MB/s 2024-04-25T09:13:45.110 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.110 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.111 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021266 s, 49.3 MB/s 2024-04-25T09:13:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.135 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206505 s, 50.8 MB/s 2024-04-25T09:13:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.159 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200139 s, 52.4 MB/s 2024-04-25T09:13:45.188 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.188 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.188 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244206 s, 42.9 MB/s 2024-04-25T09:13:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232904 s, 45.0 MB/s 2024-04-25T09:13:45.241 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.241 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.241 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213628 s, 49.1 MB/s 2024-04-25T09:13:45.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.268 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223301 s, 47.0 MB/s 2024-04-25T09:13:45.294 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.295 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225517 s, 46.5 MB/s 2024-04-25T09:13:45.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.323 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247905 s, 42.3 MB/s 2024-04-25T09:13:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025397 s, 41.3 MB/s 2024-04-25T09:13:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.385 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271837 s, 38.6 MB/s 2024-04-25T09:13:45.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.411 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.411 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021759 s, 48.2 MB/s 2024-04-25T09:13:45.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.439 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238023 s, 44.1 MB/s 2024-04-25T09:13:45.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198064 s, 52.9 MB/s 2024-04-25T09:13:45.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.490 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234244 s, 44.8 MB/s 2024-04-25T09:13:45.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.515 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208468 s, 50.3 MB/s 2024-04-25T09:13:45.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.542 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220575 s, 47.5 MB/s 2024-04-25T09:13:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.569 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233271 s, 45.0 MB/s 2024-04-25T09:13:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.598 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260174 s, 40.3 MB/s 2024-04-25T09:13:45.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.626 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236019 s, 44.4 MB/s 2024-04-25T09:13:45.654 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.654 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.654 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230652 s, 45.5 MB/s 2024-04-25T09:13:45.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.681 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229613 s, 45.7 MB/s 2024-04-25T09:13:45.710 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.710 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.710 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252039 s, 41.6 MB/s 2024-04-25T09:13:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232333 s, 45.1 MB/s 2024-04-25T09:13:45.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:45.740 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:45.740 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:45.740 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:13:45.740 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:45.740 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:45.741 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:45.741 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.829 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020609 s, 50.9 MB/s 2024-04-25T09:13:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.856 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238892 s, 43.9 MB/s 2024-04-25T09:13:45.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.885 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024873 s, 42.2 MB/s 2024-04-25T09:13:45.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.910 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222505 s, 47.1 MB/s 2024-04-25T09:13:45.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.938 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223801 s, 46.9 MB/s 2024-04-25T09:13:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215061 s, 48.8 MB/s 2024-04-25T09:13:45.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:45.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:45.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228371 s, 45.9 MB/s 2024-04-25T09:13:46.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.018 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213232 s, 49.2 MB/s 2024-04-25T09:13:46.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214854 s, 48.8 MB/s 2024-04-25T09:13:46.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.067 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214739 s, 48.8 MB/s 2024-04-25T09:13:46.092 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.092 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.092 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208564 s, 50.3 MB/s 2024-04-25T09:13:46.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203515 s, 51.5 MB/s 2024-04-25T09:13:46.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.139 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189123 s, 55.4 MB/s 2024-04-25T09:13:46.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.165 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211865 s, 49.5 MB/s 2024-04-25T09:13:46.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.194 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250361 s, 41.9 MB/s 2024-04-25T09:13:46.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.220 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222532 s, 47.1 MB/s 2024-04-25T09:13:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235315 s, 44.6 MB/s 2024-04-25T09:13:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198907 s, 52.7 MB/s 2024-04-25T09:13:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.294 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182769 s, 57.4 MB/s 2024-04-25T09:13:46.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.320 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213874 s, 49.0 MB/s 2024-04-25T09:13:46.344 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.344 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.344 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198762 s, 52.8 MB/s 2024-04-25T09:13:46.368 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.368 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.368 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201597 s, 52.0 MB/s 2024-04-25T09:13:46.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.399 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263412 s, 39.8 MB/s 2024-04-25T09:13:46.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210246 s, 49.9 MB/s 2024-04-25T09:13:46.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246745 s, 42.5 MB/s 2024-04-25T09:13:46.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.480 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022995 s, 45.6 MB/s 2024-04-25T09:13:46.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.510 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257066 s, 40.8 MB/s 2024-04-25T09:13:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229234 s, 45.7 MB/s 2024-04-25T09:13:46.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.568 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267401 s, 39.2 MB/s 2024-04-25T09:13:46.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250872 s, 41.8 MB/s 2024-04-25T09:13:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.621 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199817 s, 52.5 MB/s 2024-04-25T09:13:46.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251525 s, 41.7 MB/s 2024-04-25T09:13:46.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:46.653 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:46.653 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:46.653 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:13:46.654 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:46.654 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:46.654 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:46.654 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:46.745 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.745 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.745 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233373 s, 44.9 MB/s 2024-04-25T09:13:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268951 s, 39.0 MB/s 2024-04-25T09:13:46.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267099 s, 39.3 MB/s 2024-04-25T09:13:46.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.834 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229058 s, 45.8 MB/s 2024-04-25T09:13:46.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.864 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241633 s, 43.4 MB/s 2024-04-25T09:13:46.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.890 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225684 s, 46.5 MB/s 2024-04-25T09:13:46.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.915 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205398 s, 51.1 MB/s 2024-04-25T09:13:46.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.943 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227628 s, 46.1 MB/s 2024-04-25T09:13:46.969 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.969 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.969 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227367 s, 46.1 MB/s 2024-04-25T09:13:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201251 s, 52.1 MB/s 2024-04-25T09:13:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.016 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194653 s, 53.9 MB/s 2024-04-25T09:13:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.040 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193971 s, 54.1 MB/s 2024-04-25T09:13:47.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.065 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020744 s, 50.5 MB/s 2024-04-25T09:13:47.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019935 s, 52.6 MB/s 2024-04-25T09:13:47.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.115 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225383 s, 46.5 MB/s 2024-04-25T09:13:47.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255998 s, 41.0 MB/s 2024-04-25T09:13:47.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.171 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201215 s, 52.1 MB/s 2024-04-25T09:13:47.200 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.200 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251861 s, 41.6 MB/s 2024-04-25T09:13:47.224 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.225 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201377 s, 52.1 MB/s 2024-04-25T09:13:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.252 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226528 s, 46.3 MB/s 2024-04-25T09:13:47.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.279 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221699 s, 47.3 MB/s 2024-04-25T09:13:47.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.305 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214162 s, 49.0 MB/s 2024-04-25T09:13:47.330 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.331 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216302 s, 48.5 MB/s 2024-04-25T09:13:47.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.354 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197326 s, 53.1 MB/s 2024-04-25T09:13:47.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.383 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234399 s, 44.7 MB/s 2024-04-25T09:13:47.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218446 s, 48.0 MB/s 2024-04-25T09:13:47.436 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.436 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.436 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234921 s, 44.6 MB/s 2024-04-25T09:13:47.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.461 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198528 s, 52.8 MB/s 2024-04-25T09:13:47.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.494 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294495 s, 35.6 MB/s 2024-04-25T09:13:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.522 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226313 s, 46.3 MB/s 2024-04-25T09:13:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.550 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240243 s, 43.6 MB/s 2024-04-25T09:13:47.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.580 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024581 s, 42.7 MB/s 2024-04-25T09:13:47.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:47.583 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:47.583 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:47.583 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:13:47.583 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:47.583 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:47.583 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:47.583 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:47.676 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.676 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.676 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248701 s, 42.2 MB/s 2024-04-25T09:13:47.710 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.710 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.710 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262578 s, 39.9 MB/s 2024-04-25T09:13:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.736 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024706 s, 42.4 MB/s 2024-04-25T09:13:47.768 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.768 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.768 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277319 s, 37.8 MB/s 2024-04-25T09:13:47.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.796 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240055 s, 43.7 MB/s 2024-04-25T09:13:47.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.821 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217376 s, 48.2 MB/s 2024-04-25T09:13:47.848 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.848 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.848 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227699 s, 46.1 MB/s 2024-04-25T09:13:47.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181074 s, 57.9 MB/s 2024-04-25T09:13:47.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.896 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225539 s, 46.5 MB/s 2024-04-25T09:13:47.924 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.924 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.924 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236948 s, 44.3 MB/s 2024-04-25T09:13:47.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.949 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214483 s, 48.9 MB/s 2024-04-25T09:13:47.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:47.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:47.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217941 s, 48.1 MB/s 2024-04-25T09:13:48.002 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.002 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.002 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220919 s, 47.5 MB/s 2024-04-25T09:13:48.028 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.029 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213674 s, 49.1 MB/s 2024-04-25T09:13:48.055 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.055 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.055 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217376 s, 48.2 MB/s 2024-04-25T09:13:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.082 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227637 s, 46.1 MB/s 2024-04-25T09:13:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.107 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215147 s, 48.7 MB/s 2024-04-25T09:13:48.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.130 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193923 s, 54.1 MB/s 2024-04-25T09:13:48.160 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.160 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.160 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254202 s, 41.2 MB/s 2024-04-25T09:13:48.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260243 s, 40.3 MB/s 2024-04-25T09:13:48.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.221 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260981 s, 40.2 MB/s 2024-04-25T09:13:48.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.246 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208461 s, 50.3 MB/s 2024-04-25T09:13:48.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.272 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214877 s, 48.8 MB/s 2024-04-25T09:13:48.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.302 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244617 s, 42.9 MB/s 2024-04-25T09:13:48.327 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.327 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.327 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199748 s, 52.5 MB/s 2024-04-25T09:13:48.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.354 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229303 s, 45.7 MB/s 2024-04-25T09:13:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.379 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211561 s, 49.6 MB/s 2024-04-25T09:13:48.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.406 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228603 s, 45.9 MB/s 2024-04-25T09:13:48.428 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.429 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186134 s, 56.3 MB/s 2024-04-25T09:13:48.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208941 s, 50.2 MB/s 2024-04-25T09:13:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260982 s, 40.2 MB/s 2024-04-25T09:13:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.511 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224219 s, 46.8 MB/s 2024-04-25T09:13:48.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:48.514 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:48.514 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:48.514 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:13:48.514 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:48.514 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:48.514 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:48.514 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.600 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225577 s, 46.5 MB/s 2024-04-25T09:13:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239605 s, 43.8 MB/s 2024-04-25T09:13:48.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.659 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027599 s, 38.0 MB/s 2024-04-25T09:13:48.692 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.692 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.692 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289742 s, 36.2 MB/s 2024-04-25T09:13:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.722 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267301 s, 39.2 MB/s 2024-04-25T09:13:48.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205074 s, 51.1 MB/s 2024-04-25T09:13:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.776 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247555 s, 42.4 MB/s 2024-04-25T09:13:48.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.804 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238894 s, 43.9 MB/s 2024-04-25T09:13:48.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227924 s, 46.0 MB/s 2024-04-25T09:13:48.857 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.857 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.857 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225521 s, 46.5 MB/s 2024-04-25T09:13:48.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.883 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219974 s, 47.7 MB/s 2024-04-25T09:13:48.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211579 s, 49.6 MB/s 2024-04-25T09:13:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.934 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216997 s, 48.3 MB/s 2024-04-25T09:13:48.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.961 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229485 s, 45.7 MB/s 2024-04-25T09:13:48.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:48.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:48.988 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219378 s, 47.8 MB/s 2024-04-25T09:13:49.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.019 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266886 s, 39.3 MB/s 2024-04-25T09:13:49.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.045 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213049 s, 49.2 MB/s 2024-04-25T09:13:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.076 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263738 s, 39.8 MB/s 2024-04-25T09:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224813 s, 46.6 MB/s 2024-04-25T09:13:49.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.128 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206147 s, 50.9 MB/s 2024-04-25T09:13:49.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214883 s, 48.8 MB/s 2024-04-25T09:13:49.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022249 s, 47.1 MB/s 2024-04-25T09:13:49.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.207 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228704 s, 45.8 MB/s 2024-04-25T09:13:49.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.234 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232457 s, 45.1 MB/s 2024-04-25T09:13:49.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.259 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020996 s, 49.9 MB/s 2024-04-25T09:13:49.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.284 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205538 s, 51.0 MB/s 2024-04-25T09:13:49.310 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.310 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.310 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216261 s, 48.5 MB/s 2024-04-25T09:13:49.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209288 s, 50.1 MB/s 2024-04-25T09:13:49.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.362 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219676 s, 47.7 MB/s 2024-04-25T09:13:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.385 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194297 s, 54.0 MB/s 2024-04-25T09:13:49.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196968 s, 53.2 MB/s 2024-04-25T09:13:49.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.438 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025057 s, 41.8 MB/s 2024-04-25T09:13:49.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:49.441 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:49.441 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:49.441 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:13:49.441 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:49.441 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:49.441 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:49.441 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:49.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.534 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254373 s, 41.2 MB/s 2024-04-25T09:13:49.561 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.561 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.561 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223442 s, 46.9 MB/s 2024-04-25T09:13:49.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192472 s, 54.5 MB/s 2024-04-25T09:13:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.614 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025898 s, 40.5 MB/s 2024-04-25T09:13:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208945 s, 50.2 MB/s 2024-04-25T09:13:49.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.666 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225777 s, 46.4 MB/s 2024-04-25T09:13:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.689 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191202 s, 54.8 MB/s 2024-04-25T09:13:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.721 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276598 s, 37.9 MB/s 2024-04-25T09:13:49.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.750 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021337 s, 49.1 MB/s 2024-04-25T09:13:49.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241234 s, 43.5 MB/s 2024-04-25T09:13:49.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194047 s, 54.0 MB/s 2024-04-25T09:13:49.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.823 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203681 s, 51.5 MB/s 2024-04-25T09:13:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203076 s, 51.6 MB/s 2024-04-25T09:13:49.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.871 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193549 s, 54.2 MB/s 2024-04-25T09:13:49.895 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.895 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.896 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210675 s, 49.8 MB/s 2024-04-25T09:13:49.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.921 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219559 s, 47.8 MB/s 2024-04-25T09:13:49.948 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.948 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.948 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218679 s, 48.0 MB/s 2024-04-25T09:13:49.978 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:49.978 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:49.978 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261817 s, 40.0 MB/s 2024-04-25T09:13:50.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.004 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210151 s, 49.9 MB/s 2024-04-25T09:13:50.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.033 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253399 s, 41.4 MB/s 2024-04-25T09:13:50.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.064 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263594 s, 39.8 MB/s 2024-04-25T09:13:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198752 s, 52.8 MB/s 2024-04-25T09:13:50.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020884 s, 50.2 MB/s 2024-04-25T09:13:50.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.139 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223195 s, 47.0 MB/s 2024-04-25T09:13:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.165 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214827 s, 48.8 MB/s 2024-04-25T09:13:50.189 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215097 s, 48.7 MB/s 2024-04-25T09:13:50.217 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.217 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.217 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239118 s, 43.9 MB/s 2024-04-25T09:13:50.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.247 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.247 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023932 s, 43.8 MB/s 2024-04-25T09:13:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.273 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229138 s, 45.8 MB/s 2024-04-25T09:13:50.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.303 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254773 s, 41.2 MB/s 2024-04-25T09:13:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.331 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230476 s, 45.5 MB/s 2024-04-25T09:13:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.354 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194037 s, 54.0 MB/s 2024-04-25T09:13:50.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:50.356 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:50.357 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:50.357 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:13:50.357 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:50.357 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:50.357 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:50.357 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234832 s, 44.7 MB/s 2024-04-25T09:13:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222033 s, 47.2 MB/s 2024-04-25T09:13:50.500 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.501 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.501 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228255 s, 45.9 MB/s 2024-04-25T09:13:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227108 s, 46.2 MB/s 2024-04-25T09:13:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211967 s, 49.5 MB/s 2024-04-25T09:13:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224693 s, 46.7 MB/s 2024-04-25T09:13:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265875 s, 39.4 MB/s 2024-04-25T09:13:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224258 s, 46.8 MB/s 2024-04-25T09:13:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201019 s, 52.2 MB/s 2024-04-25T09:13:50.690 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.691 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.691 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260697 s, 40.2 MB/s 2024-04-25T09:13:50.717 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.718 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.718 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229715 s, 45.6 MB/s 2024-04-25T09:13:50.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.751 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274725 s, 38.2 MB/s 2024-04-25T09:13:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205214 s, 51.1 MB/s 2024-04-25T09:13:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.800 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216761 s, 48.4 MB/s 2024-04-25T09:13:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245706 s, 42.7 MB/s 2024-04-25T09:13:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223733 s, 46.9 MB/s 2024-04-25T09:13:50.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.876 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180382 s, 58.1 MB/s 2024-04-25T09:13:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248412 s, 42.2 MB/s 2024-04-25T09:13:50.932 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.933 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234152 s, 44.8 MB/s 2024-04-25T09:13:50.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.960 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237343 s, 44.2 MB/s 2024-04-25T09:13:50.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:50.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:50.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259324 s, 40.4 MB/s 2024-04-25T09:13:51.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.020 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257617 s, 40.7 MB/s 2024-04-25T09:13:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200557 s, 52.3 MB/s 2024-04-25T09:13:51.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.067 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193174 s, 54.3 MB/s 2024-04-25T09:13:51.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.093 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220241 s, 47.6 MB/s 2024-04-25T09:13:51.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.121 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236888 s, 44.3 MB/s 2024-04-25T09:13:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020213 s, 51.9 MB/s 2024-04-25T09:13:51.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.170 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203678 s, 51.5 MB/s 2024-04-25T09:13:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230801 s, 45.4 MB/s 2024-04-25T09:13:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189633 s, 55.3 MB/s 2024-04-25T09:13:51.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.246 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195806 s, 53.6 MB/s 2024-04-25T09:13:51.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.273 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227516 s, 46.1 MB/s 2024-04-25T09:13:51.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:51.275 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:51.275 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:51.275 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:13:51.275 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:51.276 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:51.276 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:51.276 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:51.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.363 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198024 s, 53.0 MB/s 2024-04-25T09:13:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218865 s, 47.9 MB/s 2024-04-25T09:13:51.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.413 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203661 s, 51.5 MB/s 2024-04-25T09:13:51.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.446 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284032 s, 36.9 MB/s 2024-04-25T09:13:51.470 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.470 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.470 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200624 s, 52.3 MB/s 2024-04-25T09:13:51.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.498 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248954 s, 42.1 MB/s 2024-04-25T09:13:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.526 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238384 s, 44.0 MB/s 2024-04-25T09:13:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220082 s, 47.6 MB/s 2024-04-25T09:13:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021017 s, 49.9 MB/s 2024-04-25T09:13:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227272 s, 46.1 MB/s 2024-04-25T09:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.633 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233732 s, 44.9 MB/s 2024-04-25T09:13:51.658 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.659 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207481 s, 50.5 MB/s 2024-04-25T09:13:51.687 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.688 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250305 s, 41.9 MB/s 2024-04-25T09:13:51.716 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.716 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.716 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243423 s, 43.1 MB/s 2024-04-25T09:13:51.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.744 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236439 s, 44.3 MB/s 2024-04-25T09:13:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281731 s, 37.2 MB/s 2024-04-25T09:13:51.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.802 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217667 s, 48.2 MB/s 2024-04-25T09:13:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025028 s, 41.9 MB/s 2024-04-25T09:13:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.857 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212146 s, 49.4 MB/s 2024-04-25T09:13:51.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.885 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242009 s, 43.3 MB/s 2024-04-25T09:13:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.913 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244683 s, 42.9 MB/s 2024-04-25T09:13:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020615 s, 50.9 MB/s 2024-04-25T09:13:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224855 s, 46.6 MB/s 2024-04-25T09:13:51.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:51.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:51.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213669 s, 49.1 MB/s 2024-04-25T09:13:52.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.018 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229121 s, 45.8 MB/s 2024-04-25T09:13:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.048 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260912 s, 40.2 MB/s 2024-04-25T09:13:52.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.074 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212603 s, 49.3 MB/s 2024-04-25T09:13:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.096 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183972 s, 57.0 MB/s 2024-04-25T09:13:52.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.125 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024895 s, 42.1 MB/s 2024-04-25T09:13:52.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.150 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208993 s, 50.2 MB/s 2024-04-25T09:13:52.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204589 s, 51.3 MB/s 2024-04-25T09:13:52.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.199 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196014 s, 53.5 MB/s 2024-04-25T09:13:52.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:52.202 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:52.202 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:52.202 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:13:52.202 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:52.202 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:52.202 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:52.202 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.291 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215552 s, 48.6 MB/s 2024-04-25T09:13:52.321 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.321 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.321 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270824 s, 38.7 MB/s 2024-04-25T09:13:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.348 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229014 s, 45.8 MB/s 2024-04-25T09:13:52.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.373 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213636 s, 49.1 MB/s 2024-04-25T09:13:52.402 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.402 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.402 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214374 s, 48.9 MB/s 2024-04-25T09:13:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.427 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243322 s, 43.1 MB/s 2024-04-25T09:13:52.457 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.457 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.457 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266599 s, 39.3 MB/s 2024-04-25T09:13:52.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.483 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02141 s, 49.0 MB/s 2024-04-25T09:13:52.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.508 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217943 s, 48.1 MB/s 2024-04-25T09:13:52.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.534 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218718 s, 47.9 MB/s 2024-04-25T09:13:52.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022503 s, 46.6 MB/s 2024-04-25T09:13:52.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.588 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023661 s, 44.3 MB/s 2024-04-25T09:13:52.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.612 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201426 s, 52.1 MB/s 2024-04-25T09:13:52.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.640 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234014 s, 44.8 MB/s 2024-04-25T09:13:52.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236798 s, 44.3 MB/s 2024-04-25T09:13:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.693 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207077 s, 50.6 MB/s 2024-04-25T09:13:52.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279183 s, 37.6 MB/s 2024-04-25T09:13:52.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.755 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258889 s, 40.5 MB/s 2024-04-25T09:13:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.787 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028213 s, 37.2 MB/s 2024-04-25T09:13:52.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.817 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258443 s, 40.6 MB/s 2024-04-25T09:13:52.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.846 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241948 s, 43.3 MB/s 2024-04-25T09:13:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291688 s, 35.9 MB/s 2024-04-25T09:13:52.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.910 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249976 s, 41.9 MB/s 2024-04-25T09:13:52.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237324 s, 44.2 MB/s 2024-04-25T09:13:52.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228286 s, 45.9 MB/s 2024-04-25T09:13:52.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:52.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:52.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253451 s, 41.4 MB/s 2024-04-25T09:13:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.022 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239431 s, 43.8 MB/s 2024-04-25T09:13:53.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.050 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234375 s, 44.7 MB/s 2024-04-25T09:13:53.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.076 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224841 s, 46.6 MB/s 2024-04-25T09:13:53.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221086 s, 47.4 MB/s 2024-04-25T09:13:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244762 s, 42.8 MB/s 2024-04-25T09:13:53.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.159 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226259 s, 46.3 MB/s 2024-04-25T09:13:53.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:53.162 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:53.162 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:53.162 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:13:53.162 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:53.162 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:53.162 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:53.162 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:53.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.256 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230544 s, 45.5 MB/s 2024-04-25T09:13:53.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.286 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263384 s, 39.8 MB/s 2024-04-25T09:13:53.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.313 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.313 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227486 s, 46.1 MB/s 2024-04-25T09:13:53.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.337 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200021 s, 52.4 MB/s 2024-04-25T09:13:53.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.363 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215157 s, 48.7 MB/s 2024-04-25T09:13:53.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.385 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192217 s, 54.6 MB/s 2024-04-25T09:13:53.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.412 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232192 s, 45.2 MB/s 2024-04-25T09:13:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.441 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241947 s, 43.3 MB/s 2024-04-25T09:13:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.465 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204433 s, 51.3 MB/s 2024-04-25T09:13:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228895 s, 45.8 MB/s 2024-04-25T09:13:53.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.520 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023744 s, 44.2 MB/s 2024-04-25T09:13:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.547 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.548 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02329 s, 45.0 MB/s 2024-04-25T09:13:53.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.573 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219043 s, 47.9 MB/s 2024-04-25T09:13:53.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.595 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183532 s, 57.1 MB/s 2024-04-25T09:13:53.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.623 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023043 s, 45.5 MB/s 2024-04-25T09:13:53.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.646 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020022 s, 52.4 MB/s 2024-04-25T09:13:53.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192685 s, 54.4 MB/s 2024-04-25T09:13:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252777 s, 41.5 MB/s 2024-04-25T09:13:53.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239984 s, 43.7 MB/s 2024-04-25T09:13:53.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.751 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214992 s, 48.8 MB/s 2024-04-25T09:13:53.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.777 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216061 s, 48.5 MB/s 2024-04-25T09:13:53.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.805 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238752 s, 43.9 MB/s 2024-04-25T09:13:53.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.832 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235654 s, 44.5 MB/s 2024-04-25T09:13:53.861 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.861 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.861 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239992 s, 43.7 MB/s 2024-04-25T09:13:53.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.890 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248654 s, 42.2 MB/s 2024-04-25T09:13:53.918 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.918 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.918 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235934 s, 44.4 MB/s 2024-04-25T09:13:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.945 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224649 s, 46.7 MB/s 2024-04-25T09:13:53.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.972 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230048 s, 45.6 MB/s 2024-04-25T09:13:53.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:53.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:53.999 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229552 s, 45.7 MB/s 2024-04-25T09:13:54.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.027 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237831 s, 44.1 MB/s 2024-04-25T09:13:54.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.052 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.052 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209537 s, 50.0 MB/s 2024-04-25T09:13:54.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.077 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217439 s, 48.2 MB/s 2024-04-25T09:13:54.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:54.080 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:54.080 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:54.080 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:13:54.080 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:54.080 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:54.080 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:54.080 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224706 s, 46.7 MB/s 2024-04-25T09:13:54.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.200 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025566 s, 41.0 MB/s 2024-04-25T09:13:54.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.222 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185122 s, 56.6 MB/s 2024-04-25T09:13:54.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.249 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224643 s, 46.7 MB/s 2024-04-25T09:13:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.270 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.271 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189477 s, 55.3 MB/s 2024-04-25T09:13:54.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.297 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225633 s, 46.5 MB/s 2024-04-25T09:13:54.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.324 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232023 s, 45.2 MB/s 2024-04-25T09:13:54.351 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.351 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.351 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219135 s, 47.9 MB/s 2024-04-25T09:13:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.378 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.379 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232587 s, 45.1 MB/s 2024-04-25T09:13:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232498 s, 45.1 MB/s 2024-04-25T09:13:54.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.433 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216018 s, 48.5 MB/s 2024-04-25T09:13:54.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.458 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215218 s, 48.7 MB/s 2024-04-25T09:13:54.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250421 s, 41.9 MB/s 2024-04-25T09:13:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.515 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023598 s, 44.4 MB/s 2024-04-25T09:13:54.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.540 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210817 s, 49.7 MB/s 2024-04-25T09:13:54.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.567 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225716 s, 46.5 MB/s 2024-04-25T09:13:54.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.594 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230849 s, 45.4 MB/s 2024-04-25T09:13:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.624 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243906 s, 43.0 MB/s 2024-04-25T09:13:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.650 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209261 s, 50.1 MB/s 2024-04-25T09:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.677 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245302 s, 42.7 MB/s 2024-04-25T09:13:54.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.706 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246908 s, 42.5 MB/s 2024-04-25T09:13:54.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271092 s, 38.7 MB/s 2024-04-25T09:13:54.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.767 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259859 s, 40.4 MB/s 2024-04-25T09:13:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.793 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208216 s, 50.4 MB/s 2024-04-25T09:13:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025004 s, 41.9 MB/s 2024-04-25T09:13:54.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.850 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230713 s, 45.4 MB/s 2024-04-25T09:13:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.875 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210663 s, 49.8 MB/s 2024-04-25T09:13:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.898 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193234 s, 54.3 MB/s 2024-04-25T09:13:54.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.927 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024174 s, 43.4 MB/s 2024-04-25T09:13:54.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.954 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.954 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022348 s, 46.9 MB/s 2024-04-25T09:13:54.982 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:54.982 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:54.982 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231059 s, 45.4 MB/s 2024-04-25T09:13:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.009 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225912 s, 46.4 MB/s 2024-04-25T09:13:55.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:55.012 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:55.012 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:55.012 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:13:55.012 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:55.012 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:55.012 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:55.012 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:55.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.095 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174187 s, 60.2 MB/s 2024-04-25T09:13:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.123 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242032 s, 43.3 MB/s 2024-04-25T09:13:55.149 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.149 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.149 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227423 s, 46.1 MB/s 2024-04-25T09:13:55.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210628 s, 49.8 MB/s 2024-04-25T09:13:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.205 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247662 s, 42.3 MB/s 2024-04-25T09:13:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.233 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228414 s, 45.9 MB/s 2024-04-25T09:13:55.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.262 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242576 s, 43.2 MB/s 2024-04-25T09:13:55.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.287 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217536 s, 48.2 MB/s 2024-04-25T09:13:55.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.315 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236227 s, 44.4 MB/s 2024-04-25T09:13:55.344 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.344 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.344 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244003 s, 43.0 MB/s 2024-04-25T09:13:55.375 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.375 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.375 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259837 s, 40.4 MB/s 2024-04-25T09:13:55.403 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.404 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247178 s, 42.4 MB/s 2024-04-25T09:13:55.430 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.431 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.431 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023182 s, 45.2 MB/s 2024-04-25T09:13:55.457 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.458 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223377 s, 46.9 MB/s 2024-04-25T09:13:55.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.484 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225942 s, 46.4 MB/s 2024-04-25T09:13:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.512 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228398 s, 45.9 MB/s 2024-04-25T09:13:55.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.539 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241258 s, 43.5 MB/s 2024-04-25T09:13:55.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.564 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209228 s, 50.1 MB/s 2024-04-25T09:13:55.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.592 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240568 s, 43.6 MB/s 2024-04-25T09:13:55.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.617 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216231 s, 48.5 MB/s 2024-04-25T09:13:55.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.646 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245669 s, 42.7 MB/s 2024-04-25T09:13:55.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214056 s, 49.0 MB/s 2024-04-25T09:13:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.695 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202012 s, 51.9 MB/s 2024-04-25T09:13:55.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245706 s, 42.7 MB/s 2024-04-25T09:13:55.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.753 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244491 s, 42.9 MB/s 2024-04-25T09:13:55.777 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.777 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.778 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201887 s, 51.9 MB/s 2024-04-25T09:13:55.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.804 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219174 s, 47.8 MB/s 2024-04-25T09:13:55.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234139 s, 44.8 MB/s 2024-04-25T09:13:55.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.858 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228912 s, 45.8 MB/s 2024-04-25T09:13:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.885 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225735 s, 46.5 MB/s 2024-04-25T09:13:55.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.915 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265852 s, 39.4 MB/s 2024-04-25T09:13:55.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:55.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:55.942 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022346 s, 46.9 MB/s 2024-04-25T09:13:55.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:55.944 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:55.945 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:55.945 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:13:55.945 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:55.945 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:55.945 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:55.945 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231475 s, 45.3 MB/s 2024-04-25T09:13:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227259 s, 46.1 MB/s 2024-04-25T09:13:56.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255476 s, 41.0 MB/s 2024-04-25T09:13:56.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230257 s, 45.5 MB/s 2024-04-25T09:13:56.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.147 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026573 s, 39.5 MB/s 2024-04-25T09:13:56.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.170 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195382 s, 53.7 MB/s 2024-04-25T09:13:56.196 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214944 s, 48.8 MB/s 2024-04-25T09:13:56.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.228 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262347 s, 40.0 MB/s 2024-04-25T09:13:56.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.253 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212396 s, 49.4 MB/s 2024-04-25T09:13:56.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.278 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205848 s, 50.9 MB/s 2024-04-25T09:13:56.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.304 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204965 s, 51.2 MB/s 2024-04-25T09:13:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.331 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229364 s, 45.7 MB/s 2024-04-25T09:13:56.358 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.358 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.359 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231158 s, 45.4 MB/s 2024-04-25T09:13:56.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.383 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205301 s, 51.1 MB/s 2024-04-25T09:13:56.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022172 s, 47.3 MB/s 2024-04-25T09:13:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.437 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239465 s, 43.8 MB/s 2024-04-25T09:13:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.462 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207941 s, 50.4 MB/s 2024-04-25T09:13:56.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.492 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249247 s, 42.1 MB/s 2024-04-25T09:13:56.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.519 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.519 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223726 s, 46.9 MB/s 2024-04-25T09:13:56.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.547 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244452 s, 42.9 MB/s 2024-04-25T09:13:56.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.575 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236124 s, 44.4 MB/s 2024-04-25T09:13:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.599 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213723 s, 49.1 MB/s 2024-04-25T09:13:56.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.627 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231999 s, 45.2 MB/s 2024-04-25T09:13:56.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261116 s, 40.2 MB/s 2024-04-25T09:13:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238477 s, 44.0 MB/s 2024-04-25T09:13:56.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.712 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225691 s, 46.5 MB/s 2024-04-25T09:13:56.741 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.741 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.741 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251346 s, 41.7 MB/s 2024-04-25T09:13:56.768 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.768 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.768 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230712 s, 45.4 MB/s 2024-04-25T09:13:56.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.796 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236899 s, 44.3 MB/s 2024-04-25T09:13:56.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.823 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229461 s, 45.7 MB/s 2024-04-25T09:13:56.849 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.849 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.849 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222354 s, 47.2 MB/s 2024-04-25T09:13:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289202 s, 36.3 MB/s 2024-04-25T09:13:56.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:56.885 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:56.885 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:56.885 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:13:56.885 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:56.885 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:56.885 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:56.885 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:56.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:56.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:56.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02263 s, 46.3 MB/s 2024-04-25T09:13:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.003 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240645 s, 43.6 MB/s 2024-04-25T09:13:57.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.032 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242058 s, 43.3 MB/s 2024-04-25T09:13:57.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.061 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247128 s, 42.4 MB/s 2024-04-25T09:13:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.087 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224911 s, 46.6 MB/s 2024-04-25T09:13:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018737 s, 56.0 MB/s 2024-04-25T09:13:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239996 s, 43.7 MB/s 2024-04-25T09:13:57.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.165 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240433 s, 43.6 MB/s 2024-04-25T09:13:57.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.194 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249082 s, 42.1 MB/s 2024-04-25T09:13:57.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.220 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216525 s, 48.4 MB/s 2024-04-25T09:13:57.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.246 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221016 s, 47.4 MB/s 2024-04-25T09:13:57.275 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.275 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.275 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247662 s, 42.3 MB/s 2024-04-25T09:13:57.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.304 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248434 s, 42.2 MB/s 2024-04-25T09:13:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.331 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218935 s, 47.9 MB/s 2024-04-25T09:13:57.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.355 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198827 s, 52.7 MB/s 2024-04-25T09:13:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228954 s, 45.8 MB/s 2024-04-25T09:13:57.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209762 s, 50.0 MB/s 2024-04-25T09:13:57.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.438 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251652 s, 41.7 MB/s 2024-04-25T09:13:57.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.465 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224283 s, 46.8 MB/s 2024-04-25T09:13:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.490 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214675 s, 48.8 MB/s 2024-04-25T09:13:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231783 s, 45.2 MB/s 2024-04-25T09:13:57.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.544 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218478 s, 48.0 MB/s 2024-04-25T09:13:57.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.574 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251174 s, 41.7 MB/s 2024-04-25T09:13:57.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.598 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200725 s, 52.2 MB/s 2024-04-25T09:13:57.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.625 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226854 s, 46.2 MB/s 2024-04-25T09:13:57.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.656 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253687 s, 41.3 MB/s 2024-04-25T09:13:57.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.681 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214127 s, 49.0 MB/s 2024-04-25T09:13:57.710 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.710 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.710 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248578 s, 42.2 MB/s 2024-04-25T09:13:57.740 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.741 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.741 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263713 s, 39.8 MB/s 2024-04-25T09:13:57.768 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.768 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.768 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230618 s, 45.5 MB/s 2024-04-25T09:13:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.795 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225468 s, 46.5 MB/s 2024-04-25T09:13:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.821 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208903 s, 50.2 MB/s 2024-04-25T09:13:57.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:57.824 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:57.824 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:57.824 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:13:57.824 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:57.824 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:57.824 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:57.824 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:57.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249703 s, 42.0 MB/s 2024-04-25T09:13:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.945 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272384 s, 38.5 MB/s 2024-04-25T09:13:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:57.974 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023647 s, 44.3 MB/s 2024-04-25T09:13:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.001 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227154 s, 46.2 MB/s 2024-04-25T09:13:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217919 s, 48.1 MB/s 2024-04-25T09:13:58.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.051 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212991 s, 49.2 MB/s 2024-04-25T09:13:58.080 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.080 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.080 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253696 s, 41.3 MB/s 2024-04-25T09:13:58.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.107 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222644 s, 47.1 MB/s 2024-04-25T09:13:58.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.133 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.133 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214292 s, 48.9 MB/s 2024-04-25T09:13:58.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.159 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022742 s, 46.1 MB/s 2024-04-25T09:13:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.187 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023039 s, 45.5 MB/s 2024-04-25T09:13:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.211 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201008 s, 52.2 MB/s 2024-04-25T09:13:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.235 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.235 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194934 s, 53.8 MB/s 2024-04-25T09:13:58.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.261 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230128 s, 45.6 MB/s 2024-04-25T09:13:58.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.288 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227636 s, 46.1 MB/s 2024-04-25T09:13:58.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.316 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239327 s, 43.8 MB/s 2024-04-25T09:13:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.341 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212164 s, 49.4 MB/s 2024-04-25T09:13:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221665 s, 47.3 MB/s 2024-04-25T09:13:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240232 s, 43.6 MB/s 2024-04-25T09:13:58.420 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.420 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.420 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209429 s, 50.1 MB/s 2024-04-25T09:13:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.445 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208359 s, 50.3 MB/s 2024-04-25T09:13:58.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.472 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021752 s, 48.2 MB/s 2024-04-25T09:13:58.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.502 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254685 s, 41.2 MB/s 2024-04-25T09:13:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206174 s, 50.9 MB/s 2024-04-25T09:13:58.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239715 s, 43.7 MB/s 2024-04-25T09:13:58.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.656 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023901 s, 43.9 MB/s 2024-04-25T09:13:58.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243373 s, 43.1 MB/s 2024-04-25T09:13:58.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231436 s, 45.3 MB/s 2024-04-25T09:13:58.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284378 s, 36.9 MB/s 2024-04-25T09:13:58.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.700 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231778 s, 45.2 MB/s 2024-04-25T09:13:58.730 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.730 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.730 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235042 s, 44.6 MB/s 2024-04-25T09:13:58.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.757 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.757 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245419 s, 42.7 MB/s 2024-04-25T09:13:58.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:58.759 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:58.759 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:58.759 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:13:58.759 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:58.760 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:58.760 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:58.760 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:58.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.854 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252103 s, 41.6 MB/s 2024-04-25T09:13:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.880 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221417 s, 47.4 MB/s 2024-04-25T09:13:58.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246895 s, 42.5 MB/s 2024-04-25T09:13:58.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.934 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022294 s, 47.0 MB/s 2024-04-25T09:13:58.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207144 s, 50.6 MB/s 2024-04-25T09:13:58.983 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:58.983 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:58.983 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020217 s, 51.9 MB/s 2024-04-25T09:13:59.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.011 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238117 s, 44.0 MB/s 2024-04-25T09:13:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.041 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260468 s, 40.3 MB/s 2024-04-25T09:13:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222005 s, 47.2 MB/s 2024-04-25T09:13:59.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.096 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236482 s, 44.3 MB/s 2024-04-25T09:13:59.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235515 s, 44.5 MB/s 2024-04-25T09:13:59.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.151 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223149 s, 47.0 MB/s 2024-04-25T09:13:59.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.176 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212845 s, 49.3 MB/s 2024-04-25T09:13:59.206 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.206 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.206 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269897 s, 38.9 MB/s 2024-04-25T09:13:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.231 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.232 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211874 s, 49.5 MB/s 2024-04-25T09:13:59.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.260 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238429 s, 44.0 MB/s 2024-04-25T09:13:59.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242062 s, 43.3 MB/s 2024-04-25T09:13:59.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.314 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215229 s, 48.7 MB/s 2024-04-25T09:13:59.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.340 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217474 s, 48.2 MB/s 2024-04-25T09:13:59.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.364 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209752 s, 50.0 MB/s 2024-04-25T09:13:59.391 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.391 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.392 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230442 s, 45.5 MB/s 2024-04-25T09:13:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.418 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230064 s, 45.6 MB/s 2024-04-25T09:13:59.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.446 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233597 s, 44.9 MB/s 2024-04-25T09:13:59.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.469 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192758 s, 54.4 MB/s 2024-04-25T09:13:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239571 s, 43.8 MB/s 2024-04-25T09:13:59.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.527 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258065 s, 40.6 MB/s 2024-04-25T09:13:59.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.557 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258052 s, 40.6 MB/s 2024-04-25T09:13:59.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.585 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023018 s, 45.6 MB/s 2024-04-25T09:13:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209499 s, 50.1 MB/s 2024-04-25T09:13:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.634 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01981 s, 52.9 MB/s 2024-04-25T09:13:59.664 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.664 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.665 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258118 s, 40.6 MB/s 2024-04-25T09:13:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260115 s, 40.3 MB/s 2024-04-25T09:13:59.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:13:59.697 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:13:59.698 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:13:59.698 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:13:59.698 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:13:59.698 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:13:59.698 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:13:59.698 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:13:59.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.792 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258068 s, 40.6 MB/s 2024-04-25T09:13:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209537 s, 50.0 MB/s 2024-04-25T09:13:59.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.846 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252584 s, 41.5 MB/s 2024-04-25T09:13:59.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.876 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258317 s, 40.6 MB/s 2024-04-25T09:13:59.901 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.901 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.901 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021135 s, 49.6 MB/s 2024-04-25T09:13:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.929 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242123 s, 43.3 MB/s 2024-04-25T09:13:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251457 s, 41.7 MB/s 2024-04-25T09:13:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:13:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:13:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238882 s, 43.9 MB/s 2024-04-25T09:14:00.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.009 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190143 s, 55.1 MB/s 2024-04-25T09:14:00.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.034 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220477 s, 47.6 MB/s 2024-04-25T09:14:00.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.061 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225769 s, 46.4 MB/s 2024-04-25T09:14:00.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.087 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213619 s, 49.1 MB/s 2024-04-25T09:14:00.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.112 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021545 s, 48.7 MB/s 2024-04-25T09:14:00.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.139 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225192 s, 46.6 MB/s 2024-04-25T09:14:00.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.165 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021268 s, 49.3 MB/s 2024-04-25T09:14:00.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200379 s, 52.3 MB/s 2024-04-25T09:14:00.217 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.218 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.218 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229546 s, 45.7 MB/s 2024-04-25T09:14:00.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.245 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227984 s, 46.0 MB/s 2024-04-25T09:14:00.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.272 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229069 s, 45.8 MB/s 2024-04-25T09:14:00.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.297 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207943 s, 50.4 MB/s 2024-04-25T09:14:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.326 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254911 s, 41.1 MB/s 2024-04-25T09:14:00.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.352 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213553 s, 49.1 MB/s 2024-04-25T09:14:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229732 s, 45.6 MB/s 2024-04-25T09:14:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.406 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219665 s, 47.7 MB/s 2024-04-25T09:14:00.430 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.430 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.430 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203008 s, 51.7 MB/s 2024-04-25T09:14:00.456 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.457 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.457 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022979 s, 45.6 MB/s 2024-04-25T09:14:00.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.482 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221097 s, 47.4 MB/s 2024-04-25T09:14:00.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.507 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204818 s, 51.2 MB/s 2024-04-25T09:14:00.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.536 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247477 s, 42.4 MB/s 2024-04-25T09:14:00.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201426 s, 52.1 MB/s 2024-04-25T09:14:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203273 s, 51.6 MB/s 2024-04-25T09:14:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214723 s, 48.8 MB/s 2024-04-25T09:14:00.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:00.612 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:00.612 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:00.613 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:14:00.613 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:00.613 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:00.613 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:00.613 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:00.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200196 s, 52.4 MB/s 2024-04-25T09:14:00.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.722 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213441 s, 49.1 MB/s 2024-04-25T09:14:00.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196869 s, 53.3 MB/s 2024-04-25T09:14:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.773 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224989 s, 46.6 MB/s 2024-04-25T09:14:00.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272886 s, 38.4 MB/s 2024-04-25T09:14:00.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.837 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269076 s, 39.0 MB/s 2024-04-25T09:14:00.866 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.866 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.866 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247737 s, 42.3 MB/s 2024-04-25T09:14:00.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.894 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238835 s, 43.9 MB/s 2024-04-25T09:14:00.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.921 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223946 s, 46.8 MB/s 2024-04-25T09:14:00.946 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.946 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.946 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207324 s, 50.6 MB/s 2024-04-25T09:14:00.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.973 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225564 s, 46.5 MB/s 2024-04-25T09:14:00.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:00.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:00.999 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218909 s, 47.9 MB/s 2024-04-25T09:14:01.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.030 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249374 s, 42.0 MB/s 2024-04-25T09:14:01.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249479 s, 42.0 MB/s 2024-04-25T09:14:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250398 s, 41.9 MB/s 2024-04-25T09:14:01.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234394 s, 44.7 MB/s 2024-04-25T09:14:01.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.144 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022757 s, 46.1 MB/s 2024-04-25T09:14:01.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023505 s, 44.6 MB/s 2024-04-25T09:14:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215469 s, 48.7 MB/s 2024-04-25T09:14:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.223 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220244 s, 47.6 MB/s 2024-04-25T09:14:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.250 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228859 s, 45.8 MB/s 2024-04-25T09:14:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.277 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229958 s, 45.6 MB/s 2024-04-25T09:14:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.301 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203783 s, 51.5 MB/s 2024-04-25T09:14:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.327 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.328 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224339 s, 46.7 MB/s 2024-04-25T09:14:01.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214082 s, 49.0 MB/s 2024-04-25T09:14:01.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.382 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244934 s, 42.8 MB/s 2024-04-25T09:14:01.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.406 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197813 s, 53.0 MB/s 2024-04-25T09:14:01.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.429 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194698 s, 53.9 MB/s 2024-04-25T09:14:01.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.455 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218217 s, 48.1 MB/s 2024-04-25T09:14:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.483 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245995 s, 42.6 MB/s 2024-04-25T09:14:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.510 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229328 s, 45.7 MB/s 2024-04-25T09:14:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024464 s, 42.9 MB/s 2024-04-25T09:14:01.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:01.542 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:01.542 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:01.542 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:14:01.542 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:01.542 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:01.542 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:01.542 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028839 s, 36.4 MB/s 2024-04-25T09:14:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.667 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242275 s, 43.3 MB/s 2024-04-25T09:14:01.694 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.694 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.695 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232071 s, 45.2 MB/s 2024-04-25T09:14:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295963 s, 35.4 MB/s 2024-04-25T09:14:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.759 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263516 s, 39.8 MB/s 2024-04-25T09:14:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.785 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219486 s, 47.8 MB/s 2024-04-25T09:14:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.815 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255682 s, 41.0 MB/s 2024-04-25T09:14:01.844 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.845 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243922 s, 43.0 MB/s 2024-04-25T09:14:01.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.879 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300248 s, 34.9 MB/s 2024-04-25T09:14:01.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235624 s, 44.5 MB/s 2024-04-25T09:14:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.934 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229901 s, 45.6 MB/s 2024-04-25T09:14:01.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.965 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026081 s, 40.2 MB/s 2024-04-25T09:14:01.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:01.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:01.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232075 s, 45.2 MB/s 2024-04-25T09:14:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.019 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217644 s, 48.2 MB/s 2024-04-25T09:14:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.045 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218402 s, 48.0 MB/s 2024-04-25T09:14:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.065 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172963 s, 60.6 MB/s 2024-04-25T09:14:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.094 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234485 s, 44.7 MB/s 2024-04-25T09:14:02.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.118 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191025 s, 54.9 MB/s 2024-04-25T09:14:02.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239116 s, 43.9 MB/s 2024-04-25T09:14:02.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216376 s, 48.5 MB/s 2024-04-25T09:14:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.198 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223054 s, 47.0 MB/s 2024-04-25T09:14:02.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.225 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216394 s, 48.5 MB/s 2024-04-25T09:14:02.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.250 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208085 s, 50.4 MB/s 2024-04-25T09:14:02.275 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.275 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.275 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207294 s, 50.6 MB/s 2024-04-25T09:14:02.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.300 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207962 s, 50.4 MB/s 2024-04-25T09:14:02.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.331 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264888 s, 39.6 MB/s 2024-04-25T09:14:02.360 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.361 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249417 s, 42.0 MB/s 2024-04-25T09:14:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.387 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221245 s, 47.4 MB/s 2024-04-25T09:14:02.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.415 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226458 s, 46.3 MB/s 2024-04-25T09:14:02.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.437 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178831 s, 58.6 MB/s 2024-04-25T09:14:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222912 s, 47.0 MB/s 2024-04-25T09:14:02.486 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194996 s, 53.8 MB/s 2024-04-25T09:14:02.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:02.489 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:02.490 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:02.490 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:14:02.490 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:02.490 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:02.490 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:02.490 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:02.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.581 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224481 s, 46.7 MB/s 2024-04-25T09:14:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022561 s, 46.5 MB/s 2024-04-25T09:14:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.629 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183921 s, 57.0 MB/s 2024-04-25T09:14:02.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239529 s, 43.8 MB/s 2024-04-25T09:14:02.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.688 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264031 s, 39.7 MB/s 2024-04-25T09:14:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.712 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197862 s, 53.0 MB/s 2024-04-25T09:14:02.741 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.742 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025183 s, 41.6 MB/s 2024-04-25T09:14:02.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.771 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256058 s, 41.0 MB/s 2024-04-25T09:14:02.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218084 s, 48.1 MB/s 2024-04-25T09:14:02.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235371 s, 44.5 MB/s 2024-04-25T09:14:02.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.853 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237134 s, 44.2 MB/s 2024-04-25T09:14:02.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251682 s, 41.7 MB/s 2024-04-25T09:14:02.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211565 s, 49.6 MB/s 2024-04-25T09:14:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242966 s, 43.2 MB/s 2024-04-25T09:14:02.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.962 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212216 s, 49.4 MB/s 2024-04-25T09:14:02.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:02.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:02.992 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256448 s, 40.9 MB/s 2024-04-25T09:14:03.017 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.018 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216248 s, 48.5 MB/s 2024-04-25T09:14:03.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.047 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254628 s, 41.2 MB/s 2024-04-25T09:14:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.080 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287633 s, 36.5 MB/s 2024-04-25T09:14:03.104 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.104 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.104 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020577 s, 51.0 MB/s 2024-04-25T09:14:03.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.135 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256635 s, 40.9 MB/s 2024-04-25T09:14:03.161 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.161 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.161 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218531 s, 48.0 MB/s 2024-04-25T09:14:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.187 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220334 s, 47.6 MB/s 2024-04-25T09:14:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.216 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250652 s, 41.8 MB/s 2024-04-25T09:14:03.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.243 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232521 s, 45.1 MB/s 2024-04-25T09:14:03.271 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.271 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.271 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237288 s, 44.2 MB/s 2024-04-25T09:14:03.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.302 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268511 s, 39.1 MB/s 2024-04-25T09:14:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232714 s, 45.1 MB/s 2024-04-25T09:14:03.356 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.357 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224502 s, 46.7 MB/s 2024-04-25T09:14:03.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209054 s, 50.2 MB/s 2024-04-25T09:14:03.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.408 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220503 s, 47.6 MB/s 2024-04-25T09:14:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217208 s, 48.3 MB/s 2024-04-25T09:14:03.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:03.436 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:03.436 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:03.436 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:14:03.436 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:03.436 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:03.436 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:03.436 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:03.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.531 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253327 s, 41.4 MB/s 2024-04-25T09:14:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241256 s, 43.5 MB/s 2024-04-25T09:14:03.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.585 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221183 s, 47.4 MB/s 2024-04-25T09:14:03.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.612 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226731 s, 46.2 MB/s 2024-04-25T09:14:03.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.638 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206015 s, 50.9 MB/s 2024-04-25T09:14:03.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.665 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230061 s, 45.6 MB/s 2024-04-25T09:14:03.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.686 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171639 s, 61.1 MB/s 2024-04-25T09:14:03.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.714 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248391 s, 42.2 MB/s 2024-04-25T09:14:03.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.743 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251108 s, 41.8 MB/s 2024-04-25T09:14:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.777 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276305 s, 37.9 MB/s 2024-04-25T09:14:03.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247384 s, 42.4 MB/s 2024-04-25T09:14:03.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197039 s, 53.2 MB/s 2024-04-25T09:14:03.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237501 s, 44.2 MB/s 2024-04-25T09:14:03.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.885 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216037 s, 48.5 MB/s 2024-04-25T09:14:03.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192634 s, 54.4 MB/s 2024-04-25T09:14:03.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.938 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256002 s, 41.0 MB/s 2024-04-25T09:14:03.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220935 s, 47.5 MB/s 2024-04-25T09:14:03.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:03.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:03.988 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194981 s, 53.8 MB/s 2024-04-25T09:14:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.016 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232644 s, 45.1 MB/s 2024-04-25T09:14:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.045 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253253 s, 41.4 MB/s 2024-04-25T09:14:04.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.071 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219481 s, 47.8 MB/s 2024-04-25T09:14:04.101 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.102 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.102 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252374 s, 41.5 MB/s 2024-04-25T09:14:04.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.130 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247724 s, 42.3 MB/s 2024-04-25T09:14:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.156 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222058 s, 47.2 MB/s 2024-04-25T09:14:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.183 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213326 s, 49.2 MB/s 2024-04-25T09:14:04.212 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.212 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.212 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246978 s, 42.5 MB/s 2024-04-25T09:14:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.238 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216659 s, 48.4 MB/s 2024-04-25T09:14:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.264 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225816 s, 46.4 MB/s 2024-04-25T09:14:04.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.292 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232765 s, 45.0 MB/s 2024-04-25T09:14:04.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228216 s, 45.9 MB/s 2024-04-25T09:14:04.345 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.345 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.345 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223901 s, 46.8 MB/s 2024-04-25T09:14:04.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.373 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239543 s, 43.8 MB/s 2024-04-25T09:14:04.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:04.376 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:04.376 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:04.376 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:14:04.376 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:04.376 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:04.376 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:04.376 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:04.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.472 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242802 s, 43.2 MB/s 2024-04-25T09:14:04.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.497 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021611 s, 48.5 MB/s 2024-04-25T09:14:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.522 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205891 s, 50.9 MB/s 2024-04-25T09:14:04.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.544 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183844 s, 57.0 MB/s 2024-04-25T09:14:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235586 s, 44.5 MB/s 2024-04-25T09:14:04.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.596 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209609 s, 50.0 MB/s 2024-04-25T09:14:04.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.621 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208457 s, 50.3 MB/s 2024-04-25T09:14:04.654 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.654 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.654 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288776 s, 36.3 MB/s 2024-04-25T09:14:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.679 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205607 s, 51.0 MB/s 2024-04-25T09:14:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206946 s, 50.7 MB/s 2024-04-25T09:14:04.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.731 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236535 s, 44.3 MB/s 2024-04-25T09:14:04.762 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.762 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.762 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265332 s, 39.5 MB/s 2024-04-25T09:14:04.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.785 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020238 s, 51.8 MB/s 2024-04-25T09:14:04.812 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.812 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.812 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228694 s, 45.9 MB/s 2024-04-25T09:14:04.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.838 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214106 s, 49.0 MB/s 2024-04-25T09:14:04.865 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.866 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.866 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241964 s, 43.3 MB/s 2024-04-25T09:14:04.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.893 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.893 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225127 s, 46.6 MB/s 2024-04-25T09:14:04.924 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.924 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.924 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278173 s, 37.7 MB/s 2024-04-25T09:14:04.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222212 s, 47.2 MB/s 2024-04-25T09:14:04.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:04.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:04.976 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219515 s, 47.8 MB/s 2024-04-25T09:14:05.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.003 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227473 s, 46.1 MB/s 2024-04-25T09:14:05.028 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.028 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.028 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208117 s, 50.4 MB/s 2024-04-25T09:14:05.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.057 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253121 s, 41.4 MB/s 2024-04-25T09:14:05.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.087 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252944 s, 41.5 MB/s 2024-04-25T09:14:05.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237899 s, 44.1 MB/s 2024-04-25T09:14:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.139 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205075 s, 51.1 MB/s 2024-04-25T09:14:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.165 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219725 s, 47.7 MB/s 2024-04-25T09:14:05.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.192 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.192 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213218 s, 49.2 MB/s 2024-04-25T09:14:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.214 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183538 s, 57.1 MB/s 2024-04-25T09:14:05.244 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.244 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.244 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261111 s, 40.2 MB/s 2024-04-25T09:14:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.272 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217862 s, 48.1 MB/s 2024-04-25T09:14:05.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.298 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220486 s, 47.6 MB/s 2024-04-25T09:14:05.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:05.300 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:05.300 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:05.300 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:14:05.300 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:05.300 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:05.300 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:05.300 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.484 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268762 s, 39.0 MB/s 2024-04-25T09:14:05.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.514 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250747 s, 41.8 MB/s 2024-04-25T09:14:05.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.541 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229893 s, 45.6 MB/s 2024-04-25T09:14:05.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252313 s, 41.6 MB/s 2024-04-25T09:14:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224565 s, 46.7 MB/s 2024-04-25T09:14:05.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.624 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231313 s, 45.3 MB/s 2024-04-25T09:14:05.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229653 s, 45.7 MB/s 2024-04-25T09:14:05.680 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.680 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.680 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241454 s, 43.4 MB/s 2024-04-25T09:14:05.708 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.708 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.708 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231258 s, 45.3 MB/s 2024-04-25T09:14:05.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.738 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.738 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250129 s, 41.9 MB/s 2024-04-25T09:14:05.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.766 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243657 s, 43.0 MB/s 2024-04-25T09:14:05.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274559 s, 38.2 MB/s 2024-04-25T09:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.828 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258351 s, 40.6 MB/s 2024-04-25T09:14:05.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228995 s, 45.8 MB/s 2024-04-25T09:14:05.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231039 s, 45.4 MB/s 2024-04-25T09:14:05.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.909 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223228 s, 47.0 MB/s 2024-04-25T09:14:05.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.936 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023169 s, 45.3 MB/s 2024-04-25T09:14:05.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194918 s, 53.8 MB/s 2024-04-25T09:14:05.982 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:05.983 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:05.983 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201412 s, 52.1 MB/s 2024-04-25T09:14:06.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.009 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222657 s, 47.1 MB/s 2024-04-25T09:14:06.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.032 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195904 s, 53.5 MB/s 2024-04-25T09:14:06.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.058 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.058 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215434 s, 48.7 MB/s 2024-04-25T09:14:06.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.081 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202172 s, 51.9 MB/s 2024-04-25T09:14:06.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.106 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208452 s, 50.3 MB/s 2024-04-25T09:14:06.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.131 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208382 s, 50.3 MB/s 2024-04-25T09:14:06.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.157 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206108 s, 50.9 MB/s 2024-04-25T09:14:06.183 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.183 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.183 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213579 s, 49.1 MB/s 2024-04-25T09:14:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.209 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219104 s, 47.9 MB/s 2024-04-25T09:14:06.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.237 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238294 s, 44.0 MB/s 2024-04-25T09:14:06.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.263 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021576 s, 48.6 MB/s 2024-04-25T09:14:06.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219919 s, 47.7 MB/s 2024-04-25T09:14:06.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.314 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204074 s, 51.4 MB/s 2024-04-25T09:14:06.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:06.317 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:06.317 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:06.317 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:14:06.317 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:06.317 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:06.317 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:06.317 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:06.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.400 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212624 s, 49.3 MB/s 2024-04-25T09:14:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.425 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217681 s, 48.2 MB/s 2024-04-25T09:14:06.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.450 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214651 s, 48.9 MB/s 2024-04-25T09:14:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.476 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021939 s, 47.8 MB/s 2024-04-25T09:14:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.503 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231526 s, 45.3 MB/s 2024-04-25T09:14:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.532 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243745 s, 43.0 MB/s 2024-04-25T09:14:06.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.557 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212577 s, 49.3 MB/s 2024-04-25T09:14:06.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.587 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.587 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243784 s, 43.0 MB/s 2024-04-25T09:14:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.611 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201037 s, 52.2 MB/s 2024-04-25T09:14:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.637 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207276 s, 50.6 MB/s 2024-04-25T09:14:06.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.665 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242561 s, 43.2 MB/s 2024-04-25T09:14:06.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.695 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260519 s, 40.2 MB/s 2024-04-25T09:14:06.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257007 s, 40.8 MB/s 2024-04-25T09:14:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.756 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275632 s, 38.0 MB/s 2024-04-25T09:14:06.786 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.786 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.786 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248804 s, 42.1 MB/s 2024-04-25T09:14:06.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.815 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237734 s, 44.1 MB/s 2024-04-25T09:14:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224867 s, 46.6 MB/s 2024-04-25T09:14:06.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235701 s, 44.5 MB/s 2024-04-25T09:14:06.893 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.894 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209032 s, 50.2 MB/s 2024-04-25T09:14:06.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.921 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231248 s, 45.3 MB/s 2024-04-25T09:14:06.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.952 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270385 s, 38.8 MB/s 2024-04-25T09:14:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:06.977 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209061 s, 50.2 MB/s 2024-04-25T09:14:07.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.005 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224879 s, 46.6 MB/s 2024-04-25T09:14:07.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.034 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255398 s, 41.1 MB/s 2024-04-25T09:14:07.058 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192521 s, 54.5 MB/s 2024-04-25T09:14:07.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.087 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023808 s, 44.0 MB/s 2024-04-25T09:14:07.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253843 s, 41.3 MB/s 2024-04-25T09:14:07.142 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.143 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224869 s, 46.6 MB/s 2024-04-25T09:14:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259239 s, 40.4 MB/s 2024-04-25T09:14:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022375 s, 46.9 MB/s 2024-04-25T09:14:07.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.223 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207652 s, 50.5 MB/s 2024-04-25T09:14:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.248 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203421 s, 51.5 MB/s 2024-04-25T09:14:07.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:07.250 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:07.250 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:07.251 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:14:07.251 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:07.251 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:07.251 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:07.251 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:07.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.338 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.339 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210961 s, 49.7 MB/s 2024-04-25T09:14:07.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.364 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221306 s, 47.4 MB/s 2024-04-25T09:14:07.389 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.389 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.389 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215 s, 48.8 MB/s 2024-04-25T09:14:07.417 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.417 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.417 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236512 s, 44.3 MB/s 2024-04-25T09:14:07.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.447 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260467 s, 40.3 MB/s 2024-04-25T09:14:07.476 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.476 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.476 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248951 s, 42.1 MB/s 2024-04-25T09:14:07.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.507 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256386 s, 40.9 MB/s 2024-04-25T09:14:07.533 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.533 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.533 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220211 s, 47.6 MB/s 2024-04-25T09:14:07.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023251 s, 45.1 MB/s 2024-04-25T09:14:07.587 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.587 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.587 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223 s, 47.0 MB/s 2024-04-25T09:14:07.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.612 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206021 s, 50.9 MB/s 2024-04-25T09:14:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244625 s, 42.9 MB/s 2024-04-25T09:14:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.670 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260517 s, 40.2 MB/s 2024-04-25T09:14:07.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.696 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212874 s, 49.3 MB/s 2024-04-25T09:14:07.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.722 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213261 s, 49.2 MB/s 2024-04-25T09:14:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.750 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241599 s, 43.4 MB/s 2024-04-25T09:14:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.784 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288822 s, 36.3 MB/s 2024-04-25T09:14:07.813 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.813 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.813 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245802 s, 42.7 MB/s 2024-04-25T09:14:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025108 s, 41.8 MB/s 2024-04-25T09:14:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023699 s, 44.2 MB/s 2024-04-25T09:14:07.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214635 s, 48.9 MB/s 2024-04-25T09:14:07.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.922 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206137 s, 50.9 MB/s 2024-04-25T09:14:07.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.949 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227894 s, 46.0 MB/s 2024-04-25T09:14:07.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:07.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:07.977 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235574 s, 44.5 MB/s 2024-04-25T09:14:08.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.005 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242637 s, 43.2 MB/s 2024-04-25T09:14:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246979 s, 42.5 MB/s 2024-04-25T09:14:08.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208114 s, 50.4 MB/s 2024-04-25T09:14:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256755 s, 40.8 MB/s 2024-04-25T09:14:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.117 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244148 s, 42.9 MB/s 2024-04-25T09:14:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262313 s, 40.0 MB/s 2024-04-25T09:14:08.177 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.177 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.177 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243331 s, 43.1 MB/s 2024-04-25T09:14:08.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.202 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217366 s, 48.2 MB/s 2024-04-25T09:14:08.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:08.205 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:08.205 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:08.205 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:14:08.205 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:08.205 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:08.205 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:08.205 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:08.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.293 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021179 s, 49.5 MB/s 2024-04-25T09:14:08.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.320 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022062 s, 47.5 MB/s 2024-04-25T09:14:08.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.352 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285793 s, 36.7 MB/s 2024-04-25T09:14:08.384 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.384 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.384 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266454 s, 39.4 MB/s 2024-04-25T09:14:08.411 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.411 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.412 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232384 s, 45.1 MB/s 2024-04-25T09:14:08.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018851 s, 55.6 MB/s 2024-04-25T09:14:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.464 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249878 s, 42.0 MB/s 2024-04-25T09:14:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.492 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238876 s, 43.9 MB/s 2024-04-25T09:14:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.523 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262158 s, 40.0 MB/s 2024-04-25T09:14:08.551 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.552 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.552 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241613 s, 43.4 MB/s 2024-04-25T09:14:08.580 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.580 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.580 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229644 s, 45.7 MB/s 2024-04-25T09:14:08.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.611 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270913 s, 38.7 MB/s 2024-04-25T09:14:08.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.636 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217302 s, 48.3 MB/s 2024-04-25T09:14:08.664 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.664 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.664 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236952 s, 44.3 MB/s 2024-04-25T09:14:08.694 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.694 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.694 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257967 s, 40.6 MB/s 2024-04-25T09:14:08.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.726 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275866 s, 38.0 MB/s 2024-04-25T09:14:08.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.756 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257914 s, 40.7 MB/s 2024-04-25T09:14:08.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.790 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295893 s, 35.4 MB/s 2024-04-25T09:14:08.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.819 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252343 s, 41.6 MB/s 2024-04-25T09:14:08.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.847 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233131 s, 45.0 MB/s 2024-04-25T09:14:08.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.877 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249023 s, 42.1 MB/s 2024-04-25T09:14:08.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.898 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178053 s, 58.9 MB/s 2024-04-25T09:14:08.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023727 s, 44.2 MB/s 2024-04-25T09:14:08.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.955 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025496 s, 41.1 MB/s 2024-04-25T09:14:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:08.985 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02595 s, 40.4 MB/s 2024-04-25T09:14:09.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233634 s, 44.9 MB/s 2024-04-25T09:14:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.040 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232207 s, 45.2 MB/s 2024-04-25T09:14:09.068 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.068 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.069 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244468 s, 42.9 MB/s 2024-04-25T09:14:09.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.094 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212868 s, 49.3 MB/s 2024-04-25T09:14:09.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.118 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215354 s, 48.7 MB/s 2024-04-25T09:14:09.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244247 s, 42.9 MB/s 2024-04-25T09:14:09.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.171 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204403 s, 51.3 MB/s 2024-04-25T09:14:09.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:09.174 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:09.174 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:09.174 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:14:09.174 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:09.175 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:09.175 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:09.175 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:09.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.266 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.266 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218076 s, 48.1 MB/s 2024-04-25T09:14:09.294 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.294 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.295 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250891 s, 41.8 MB/s 2024-04-25T09:14:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.323 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244292 s, 42.9 MB/s 2024-04-25T09:14:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.350 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222427 s, 47.1 MB/s 2024-04-25T09:14:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025576 s, 41.0 MB/s 2024-04-25T09:14:09.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.410 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263337 s, 39.8 MB/s 2024-04-25T09:14:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.438 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241139 s, 43.5 MB/s 2024-04-25T09:14:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214576 s, 48.9 MB/s 2024-04-25T09:14:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.489 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216345 s, 48.5 MB/s 2024-04-25T09:14:09.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.516 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228408 s, 45.9 MB/s 2024-04-25T09:14:09.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.546 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260785 s, 40.2 MB/s 2024-04-25T09:14:09.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.571 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204688 s, 51.2 MB/s 2024-04-25T09:14:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.601 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250034 s, 41.9 MB/s 2024-04-25T09:14:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230547 s, 45.5 MB/s 2024-04-25T09:14:09.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.656 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240239 s, 43.6 MB/s 2024-04-25T09:14:09.686 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.686 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.686 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257317 s, 40.8 MB/s 2024-04-25T09:14:09.716 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.717 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.717 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265653 s, 39.5 MB/s 2024-04-25T09:14:09.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.743 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224559 s, 46.7 MB/s 2024-04-25T09:14:09.768 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.768 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.768 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210497 s, 49.8 MB/s 2024-04-25T09:14:09.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.796 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235879 s, 44.5 MB/s 2024-04-25T09:14:09.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.825 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253197 s, 41.4 MB/s 2024-04-25T09:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.851 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218913 s, 47.9 MB/s 2024-04-25T09:14:09.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.878 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223804 s, 46.9 MB/s 2024-04-25T09:14:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.906 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234977 s, 44.6 MB/s 2024-04-25T09:14:09.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.936 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256117 s, 40.9 MB/s 2024-04-25T09:14:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.960 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206694 s, 50.7 MB/s 2024-04-25T09:14:09.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:09.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:09.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227843 s, 46.0 MB/s 2024-04-25T09:14:10.015 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:10.015 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:10.015 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222897 s, 47.0 MB/s 2024-04-25T09:14:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021232 s, 49.4 MB/s 2024-04-25T09:14:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:10.066 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022766 s, 46.1 MB/s 2024-04-25T09:14:10.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:10.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:10.090 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190971 s, 54.9 MB/s 2024-04-25T09:14:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:10.120 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026366 s, 39.8 MB/s 2024-04-25T09:14:10.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:10.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:14:10.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.356+0000 7fed08a00640 1 Processor -- start 2024-04-25T09:14:10.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.356+0000 7fed08a00640 1 -- start start 2024-04-25T09:14:10.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.357+0000 7fed08a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 0x7fed040b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.357+0000 7fed08a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fed04161650 0x7fed040b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.357+0000 7fed08a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed040d1480 con 0x7fed040b2980 2024-04-25T09:14:10.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.357+0000 7fed08a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed040d3900 con 0x7fed04161650 2024-04-25T09:14:10.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.357+0000 7fed08a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed040d11a0 con 0x7fed04161270 2024-04-25T09:14:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.358+0000 7fed02a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fed04161650 0x7fed040b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.358+0000 7fed02a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fed04161650 0x7fed040b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36130/0 (socket says 172.21.15.62:36130) 2024-04-25T09:14:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.358+0000 7fed02a00640 1 -- 172.21.15.62:0/4116986756 learned_addr learned my addr 172.21.15.62:0/4116986756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.358+0000 7fed03400640 1 --2- 172.21.15.62:0/4116986756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 0x7fed040b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.358+0000 7fed02a00640 1 -- 172.21.15.62:0/4116986756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 msgr2=0x7fed040b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.358+0000 7fed02a00640 1 --2- 172.21.15.62:0/4116986756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 0x7fed040b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.358+0000 7fed02a00640 1 -- 172.21.15.62:0/4116986756 >> v1:172.21.15.121:6789/0 conn(0x7fed040b2980 legacy=0x7fed04161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.358+0000 7fed02a00640 1 -- 172.21.15.62:0/4116986756 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed040cee80 con 0x7fed04161650 2024-04-25T09:14:10.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.359+0000 7fed02a00640 1 --2- 172.21.15.62:0/4116986756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed04161650 0x7fed040b5aa0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009870 tx=0x7fecf002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d0bb08520ee83b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.360+0000 7fed02000640 1 -- 172.21.15.62:0/4116986756 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0037070 con 0x7fed04161650 2024-04-25T09:14:10.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.360+0000 7fed02000640 1 -- 172.21.15.62:0/4116986756 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf0032440 con 0x7fed04161650 2024-04-25T09:14:10.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.360+0000 7fed02000640 1 -- 172.21.15.62:0/4116986756 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0032810 con 0x7fed04161650 2024-04-25T09:14:10.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.360+0000 7fed08a00640 1 -- 172.21.15.62:0/4116986756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed04161650 msgr2=0x7fed040b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.360+0000 7fed08a00640 1 --2- 172.21.15.62:0/4116986756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed04161650 0x7fed040b5aa0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009870 tx=0x7fecf002a130 comp rx=0 tx=0).stop 2024-04-25T09:14:10.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.361+0000 7fed08a00640 1 -- 172.21.15.62:0/4116986756 shutdown_connections 2024-04-25T09:14:10.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.361+0000 7fed08a00640 1 --2- 172.21.15.62:0/4116986756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed04161650 0x7fed040b5aa0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.361+0000 7fed08a00640 1 --2- 172.21.15.62:0/4116986756 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 0x7fed040b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.361+0000 7fed08a00640 1 -- 172.21.15.62:0/4116986756 >> 172.21.15.62:0/4116986756 conn(0x7fed04066d10 msgr2=0x7fed04067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.361+0000 7fed08a00640 1 -- 172.21.15.62:0/4116986756 shutdown_connections 2024-04-25T09:14:10.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.361+0000 7fed08a00640 1 -- 172.21.15.62:0/4116986756 wait complete. 2024-04-25T09:14:10.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.363+0000 7fed08a00640 1 Processor -- start 2024-04-25T09:14:10.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.363+0000 7fed08a00640 1 -- start start 2024-04-25T09:14:10.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed08a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed040b2980 0x7fed040ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed08a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 0x7fed040c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed08a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed04161650 0x7fed041538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed08a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed040b2ed0 con 0x7fed040b2980 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed08a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed040d3590 con 0x7fed04161650 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed08a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed040b3050 con 0x7fed04161270 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed03400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed040b2980 0x7fed040ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed03400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed040b2980 0x7fed040ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36132/0 (socket says 172.21.15.62:36132) 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.364+0000 7fed03400640 1 -- 172.21.15.62:0/1280262718 learned_addr learned my addr 172.21.15.62:0/1280262718 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.365+0000 7fed02a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 0x7fed040c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.365+0000 7fed03400640 1 -- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 msgr2=0x7fed040c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.365+0000 7fed03400640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 0x7fed040c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.365+0000 7fed03400640 1 -- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed04161650 msgr2=0x7fed041538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.365+0000 7fed03400640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed04161650 0x7fed041538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.365+0000 7fed03400640 1 -- 172.21.15.62:0/1280262718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed040c9320 con 0x7fed040b2980 2024-04-25T09:14:10.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.365+0000 7fed03400640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed040b2980 0x7fed040ca0b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fecf4007e70 tx=0x7fecf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.366+0000 7fecfb400640 1 -- 172.21.15.62:0/1280262718 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4014620 con 0x7fed040b2980 2024-04-25T09:14:10.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.366+0000 7fecfb400640 1 -- 172.21.15.62:0/1280262718 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf4019070 con 0x7fed040b2980 2024-04-25T09:14:10.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.366+0000 7fecfb400640 1 -- 172.21.15.62:0/1280262718 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4014a00 con 0x7fed040b2980 2024-04-25T09:14:10.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.366+0000 7fed08a00640 1 -- 172.21.15.62:0/1280262718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0415bc00 con 0x7fed040b2980 2024-04-25T09:14:10.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.366+0000 7fed08a00640 1 -- 172.21.15.62:0/1280262718 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed04153050 con 0x7fed040b2980 2024-04-25T09:14:10.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.368+0000 7fecfb400640 1 -- 172.21.15.62:0/1280262718 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecf4014ba0 con 0x7fed040b2980 2024-04-25T09:14:10.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.370+0000 7fecfb400640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fece406b4d0 0x7fece406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.370+0000 7fecfb400640 1 -- 172.21.15.62:0/1280262718 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fecf408aa20 con 0x7fed040b2980 2024-04-25T09:14:10.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.370+0000 7fed02a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fece406b4d0 0x7fece406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.371+0000 7fed0cd7b740 1 Processor -- start 2024-04-25T09:14:10.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.371+0000 7fed0cd7b740 1 -- start start 2024-04-25T09:14:10.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed02a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fece406b4d0 0x7fece406d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fecf0034000 tx=0x7fecf002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22cca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22ce7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22c9fc60 0x555b22ce5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b22bfdf00 con 0x555b22cc9f80 2024-04-25T09:14:10.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b22c02f50 con 0x555b22c9fc60 2024-04-25T09:14:10.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b22bac0b0 con 0x555b22ce7820 2024-04-25T09:14:10.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed03400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22cca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed03400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22cca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36138/0 (socket says 172.21.15.62:36138) 2024-04-25T09:14:10.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.372+0000 7fed03400640 1 -- 172.21.15.62:0/3488506189 learned_addr learned my addr 172.21.15.62:0/3488506189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fed03400640 1 -- 172.21.15.62:0/3488506189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 msgr2=0x555b22ce7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fed03400640 1 --2- 172.21.15.62:0/3488506189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22ce7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fed03400640 1 -- 172.21.15.62:0/3488506189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22c9fc60 msgr2=0x555b22ce5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fed03e00640 1 --2- 172.21.15.62:0/3488506189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22c9fc60 0x555b22ce5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fed03400640 1 --2- 172.21.15.62:0/3488506189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22c9fc60 0x555b22ce5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fed03400640 1 -- 172.21.15.62:0/3488506189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b229e3ba0 con 0x555b22cc9f80 2024-04-25T09:14:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fed03400640 1 --2- 172.21.15.62:0/3488506189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22cca360 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fecf40613d0 tx=0x7fecf4059a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4526795ead3bbe58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fecef400640 1 -- 172.21.15.62:0/3488506189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4061900 con 0x555b22cc9f80 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fecef400640 1 -- 172.21.15.62:0/3488506189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40574f0 con 0x555b22cc9f80 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.373+0000 7fecef400640 1 -- 172.21.15.62:0/3488506189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4061470 con 0x555b22cc9f80 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3488506189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 msgr2=0x555b22cca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3488506189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22cca360 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fecf40613d0 tx=0x7fecf4059a30 comp rx=0 tx=0).stop 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3488506189 shutdown_connections 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3488506189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22c9fc60 0x555b22ce5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3488506189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22ce7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3488506189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22cca360 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3488506189 >> 172.21.15.62:0/3488506189 conn(0x555b22c9c820 msgr2=0x555b22ce4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3488506189 shutdown_connections 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.374+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3488506189 wait complete. 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.375+0000 7fed0cd7b740 1 -- start start 2024-04-25T09:14:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.375+0000 7fed0cd7b740 1 Processor -- start 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 0x555b22cdd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22ccac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b22ba1620 con 0x555b22cdcca0 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b22bfeff0 con 0x555b22cc9f80 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b22ca0a00 con 0x555b22ce7820 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed0cd7b740 10 client.0 _ll_register_callbacks cb 0x555b22916f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed03e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22ccac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed03e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22ccac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36152/0 (socket says 172.21.15.62:36152) 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.376+0000 7fed03e00640 1 -- 172.21.15.62:0/1255489054 learned_addr learned my addr 172.21.15.62:0/1255489054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed03400640 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 0x555b22cdd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed02a00640 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed03e00640 1 -- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 msgr2=0x555b22cddf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed03e00640 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cddf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed03e00640 1 -- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 msgr2=0x555b22cdd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed03e00640 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 0x555b22cdd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed03e00640 1 -- 172.21.15.62:0/1255489054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b228d7490 con 0x555b22cdcca0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed02a00640 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed03e00640 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22ccac40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fecfc001cb0 tx=0x7fecfc00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d168e74d36c627e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fece3e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fece3e00640 1 -- 172.21.15.62:0/1255489054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0052d0 con 0x555b22cdcca0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fece3e00640 1 -- 172.21.15.62:0/1255489054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fecfc003610 con 0x555b22cdcca0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.377+0000 7fed0cd7b740 1 -- 172.21.15.62:0/1255489054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecf40574f0 con 0x555b22cdcca0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fece3e00640 1 -- 172.21.15.62:0/1255489054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecfc01e050 con 0x555b22cdcca0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fed0cd7b740 10 client.7956 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fed0cd7b740 1 -- 172.21.15.62:0/1255489054 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x555b22a44330 con 0x555b22cdcca0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fece3e00640 1 -- 172.21.15.62:0/1255489054 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40574f0 con 0x555b22cdcca0 2024-04-25T09:14:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fece3e00640 1 -- 172.21.15.62:0/1255489054 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fecfc021020 con 0x555b22cdcca0 2024-04-25T09:14:10.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fed0cd7b740 10 client.7956 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:10.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fed0cd7b740 10 client.7956 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:10.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fed0cd7b740 20 client.7956 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:10.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x555b22cc5be0 0x555b22ccb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fed0cd7b740 4 client.7956 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:14:10.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.378+0000 7fed0cd7b740 1 -- 172.21.15.62:0/1255489054 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b22c10490 con 0x555b22cc5be0 2024-04-25T09:14:10.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.379+0000 7fed03400640 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x555b22cc5be0 0x555b22ccb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.380+0000 7fed03400640 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x555b22cc5be0 0x555b22ccb180 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f654e34ea4f22c5 server_cookie=b07984bfb2112010 in_seq=0 out_seq=0 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.380+0000 7fece3e00640 10 client.7956 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.383+0000 7fece3e00640 1 -- 172.21.15.62:0/1255489054 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b22c10490 con 0x555b22cc5be0 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.383+0000 7fece3e00640 10 client.7956 handle_command_reply: tid=0 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 client.7956 shutdown 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 -- 172.21.15.62:0/1255489054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 msgr2=0x555b22ccac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22ccac40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fecfc001cb0 tx=0x7fecfc00d520 comp rx=0 tx=0).stop 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 -- 172.21.15.62:0/1255489054 shutdown_connections 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x555b22cc5be0 0x555b22ccb180 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22ccac40 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/1255489054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 0x555b22cdd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fece3e00640 0 client.7956 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 -- 172.21.15.62:0/1255489054 >> 172.21.15.62:0/1255489054 conn(0x555b22c9fc60 msgr2=0x555b22c9ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 -- 172.21.15.62:0/1255489054 shutdown_connections 2024-04-25T09:14:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 1 -- 172.21.15.62:0/1255489054 wait complete. 2024-04-25T09:14:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.384+0000 7fed0cd7b740 20 client.7956 trim_cache size 0 max 16384 2024-04-25T09:14:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.401+0000 7fed0cd7b740 1 Processor -- start 2024-04-25T09:14:10.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.401+0000 7fed0cd7b740 1 -- start start 2024-04-25T09:14:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22ccc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22cdcca0 0x555b22c99630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22ce7820 0x555b22cda720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed042000b0 con 0x555b22cc9f80 2024-04-25T09:14:10.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed040cb930 con 0x555b22ce7820 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed040cad30 con 0x555b22cdcca0 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed03400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22ccc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed03400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22ccc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36162/0 (socket says 172.21.15.62:36162) 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.402+0000 7fed03400640 1 -- 172.21.15.62:0/772148311 learned_addr learned my addr 172.21.15.62:0/772148311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.403+0000 7fed02a00640 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22cdcca0 0x555b22c99630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.403+0000 7fed03e00640 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22ce7820 0x555b22cda720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.403+0000 7fed03400640 1 -- 172.21.15.62:0/772148311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22cdcca0 msgr2=0x555b22c99630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.403+0000 7fed03400640 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22cdcca0 0x555b22c99630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.403+0000 7fed03400640 1 -- 172.21.15.62:0/772148311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22ce7820 msgr2=0x555b22cda720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.403+0000 7fed03400640 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22ce7820 0x555b22cda720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.403+0000 7fed03400640 1 -- 172.21.15.62:0/772148311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b22cdbc70 con 0x555b22cc9f80 2024-04-25T09:14:10.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.403+0000 7fed03400640 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22ccc4c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fecf4058ea0 tx=0x7fecf405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ed35dba123f180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7feceea00640 1 -- 172.21.15.62:0/772148311 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4077070 con 0x555b22cc9f80 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7feceea00640 1 -- 172.21.15.62:0/772148311 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40574f0 con 0x555b22cc9f80 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7feceea00640 1 -- 172.21.15.62:0/772148311 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4087040 con 0x555b22cc9f80 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 -- 172.21.15.62:0/772148311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 msgr2=0x555b22ccc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22ccc4c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fecf4058ea0 tx=0x7fecf405b000 comp rx=0 tx=0).stop 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 -- 172.21.15.62:0/772148311 shutdown_connections 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22ce7820 0x555b22cda720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22cdcca0 0x555b22c99630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/772148311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cc9f80 0x555b22ccc4c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 -- 172.21.15.62:0/772148311 >> 172.21.15.62:0/772148311 conn(0x555b22c9fc60 msgr2=0x555b22cf5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 -- 172.21.15.62:0/772148311 shutdown_connections 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.404+0000 7fed0cd7b740 1 -- 172.21.15.62:0/772148311 wait complete. 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.405+0000 7fed0cd7b740 1 -- start start 2024-04-25T09:14:10.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.405+0000 7fed0cd7b740 1 Processor -- start 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.406+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 0x555b22d39400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.406+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22d53fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.406+0000 7fed0cd7b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cfec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.406+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b22cdef50 con 0x555b22cdcca0 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.406+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed040cb330 con 0x555b22cc9f80 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.406+0000 7fed0cd7b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b22cdca60 con 0x555b22ce7820 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.406+0000 7fed0cd7b740 10 client.0 _ll_register_callbacks cb 0x555b22c9ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed02a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22d53fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed02a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22d53fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36176/0 (socket says 172.21.15.62:36176) 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed02a00640 1 -- 172.21.15.62:0/3730246122 learned_addr learned my addr 172.21.15.62:0/3730246122 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed03400640 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 0x555b22d39400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed03e00640 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cfec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed02a00640 1 -- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 msgr2=0x555b22cfec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed02a00640 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cfec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed02a00640 1 -- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 msgr2=0x555b22d39400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed02a00640 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 0x555b22d39400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.407+0000 7fed02a00640 1 -- 172.21.15.62:0/3730246122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b228b0610 con 0x555b22cdcca0 2024-04-25T09:14:10.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.408+0000 7fed02a00640 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22d53fb0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fecf0035040 tx=0x7fecf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296eb478a05a8221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.408+0000 7fece3e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:10.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.408+0000 7fece3e00640 1 -- 172.21.15.62:0/3730246122 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0037040 con 0x555b22cdcca0 2024-04-25T09:14:10.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.408+0000 7fece3e00640 1 -- 172.21.15.62:0/3730246122 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fecf003c9f0 con 0x555b22cdcca0 2024-04-25T09:14:10.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.408+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3730246122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecf40574f0 con 0x555b22cdcca0 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.408+0000 7fece3e00640 1 -- 172.21.15.62:0/3730246122 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecf003cbf0 con 0x555b22cdcca0 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.408+0000 7fed0cd7b740 10 client.7962 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.408+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3730246122 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x555b22d53810 con 0x555b22cdcca0 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.409+0000 7fece3e00640 1 -- 172.21.15.62:0/3730246122 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40574f0 con 0x555b22cdcca0 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.409+0000 7fece3e00640 1 -- 172.21.15.62:0/3730246122 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fecf0059020 con 0x555b22cdcca0 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.409+0000 7fed0cd7b740 10 client.7962 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.409+0000 7fed0cd7b740 10 client.7962 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.409+0000 7fed0cd7b740 20 client.7962 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.409+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x555b22cdf290 0x555b22ce5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.409+0000 7fed0cd7b740 4 client.7962 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.409+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3730246122 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x555b22c0a7e0 con 0x555b22cdf290 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.410+0000 7fed03400640 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x555b22cdf290 0x555b22ce5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.410+0000 7fed03400640 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x555b22cdf290 0x555b22ce5de0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e1df7770a81e9d server_cookie=b10dc5d03a3760ed in_seq=0 out_seq=0 2024-04-25T09:14:10.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.410+0000 7fece3e00640 10 client.7962 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.454+0000 7fece3e00640 1 -- 172.21.15.62:0/3730246122 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555b22c0a7e0 con 0x555b22cdf290 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.454+0000 7fece3e00640 10 client.7962 handle_command_reply: tid=0 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 client.7962 shutdown 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3730246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 msgr2=0x555b22d53fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22d53fb0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fecf0035040 tx=0x7fecf0004730 comp rx=0 tx=0).stop 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3730246122 shutdown_connections 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x555b22cdf290 0x555b22ce5de0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555b22cdcca0 0x555b22d53fb0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x555b22ce7820 0x555b22cfec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fece3e00640 0 client.7962 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:10.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 --2- 172.21.15.62:0/3730246122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b22cc9f80 0x555b22d39400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3730246122 >> 172.21.15.62:0/3730246122 conn(0x555b22c9fc60 msgr2=0x555b22ccac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3730246122 shutdown_connections 2024-04-25T09:14:10.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.455+0000 7fed0cd7b740 1 -- 172.21.15.62:0/3730246122 wait complete. 2024-04-25T09:14:10.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.456+0000 7fed0cd7b740 20 client.7962 trim_cache size 0 max 16384 2024-04-25T09:14:10.443 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:14:10.444 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:14:10.444 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "0ae8b2d2-cab3-400f-b6de-a6ece47182b4", 2024-04-25T09:14:10.444 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:14:10.444 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 -- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fece406b4d0 msgr2=0x7fece406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fece406b4d0 0x7fece406d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fecf0034000 tx=0x7fecf002e000 comp rx=0 tx=0).stop 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 -- 172.21.15.62:0/1280262718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed040b2980 msgr2=0x7fed040ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed040b2980 0x7fed040ca0b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fecf4007e70 tx=0x7fecf400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 -- 172.21.15.62:0/1280262718 shutdown_connections 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fece406b4d0 0x7fece406d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed04161650 0x7fed041538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed04161270 0x7fed040c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 --2- 172.21.15.62:0/1280262718 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed040b2980 0x7fed040ca0b0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 -- 172.21.15.62:0/1280262718 >> 172.21.15.62:0/1280262718 conn(0x7fed04066d10 msgr2=0x7fed0415f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 -- 172.21.15.62:0/1280262718 shutdown_connections 2024-04-25T09:14:10.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.457+0000 7fed08a00640 1 -- 172.21.15.62:0/1280262718 wait complete. 2024-04-25T09:14:10.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:10.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T09:14:10.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.705+0000 7f96f5200640 1 Processor -- start 2024-04-25T09:14:10.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.705+0000 7f96f5200640 1 -- start start 2024-04-25T09:14:10.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.707+0000 7f96f5200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 0x7f96f006b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.707+0000 7f96f5200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f006c950 con 0x7f96f00b0b70 2024-04-25T09:14:10.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.707+0000 7f96f5200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f006ad80 con 0x7f96f00d1780 2024-04-25T09:14:10.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.707+0000 7f96f5200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f006b060 con 0x7f96f00b4860 2024-04-25T09:14:10.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.707+0000 7f96eea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f96f00b0b70 0x7f96f015dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:47552/0 (socket says 172.21.15.62:47552) 2024-04-25T09:14:10.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.707+0000 7f96eea00640 1 -- 172.21.15.62:0/1199287465 learned_addr learned my addr 172.21.15.62:0/1199287465 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:10.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.707+0000 7f96efe00640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 0x7f96f006b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.707+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417189856 0 0) 0x7f96f006c950 con 0x7f96f00b0b70 2024-04-25T09:14:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d4003520 con 0x7f96f00b0b70 2024-04-25T09:14:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 69814493 0 0) 0x7f96d4003520 con 0x7f96f00b0b70 2024-04-25T09:14:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 msgr2=0x7f96f006b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 0x7f96f006b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 >> v1:172.21.15.121:6789/0 conn(0x7f96f00d1780 legacy=0x7f96f00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f006b340 con 0x7f96f00b0b70 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f96dc001670 con 0x7f96f00b0b70 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 >> v1:172.21.15.62:6789/0 conn(0x7f96f00b0b70 legacy=0x7f96f015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96d40039e0 0x7f96d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 0x7f96d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d4004d40 0x7f96d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f006b060 con 0x7f96d40039e0 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d400baf0 con 0x7f96d4004d40 2024-04-25T09:14:10.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d400bcf0 con 0x7f96f00b4860 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.708+0000 7f96ef400640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96d40039e0 0x7f96d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.709+0000 7f96efe00640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d4004d40 0x7f96d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.709+0000 7f96eea00640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 0x7f96d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.709+0000 7f96ef400640 1 -- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 msgr2=0x7f96d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.709+0000 7f96ef400640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 0x7f96d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.709+0000 7f96ef400640 1 -- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d4004d40 msgr2=0x7f96d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.709+0000 7f96ef400640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96d4004d40 0x7f96d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.709+0000 7f96ef400640 1 -- 172.21.15.62:0/1199287465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4003520 con 0x7f96d40039e0 2024-04-25T09:14:10.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.709+0000 7f96eea00640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96f00b4860 0x7f96d4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:10.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.710+0000 7f96ef400640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96d40039e0 0x7f96d4003e00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f96e0001750 tx=0x7f96d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d9b00e5605bec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.710+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0001370 con 0x7f96d40039e0 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.710+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e00052d0 con 0x7f96d40039e0 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.710+0000 7f96ee000640 1 -- 172.21.15.62:0/1199287465 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e002ab20 con 0x7f96d40039e0 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.711+0000 7f96f5200640 1 -- 172.21.15.62:0/1199287465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96d40039e0 msgr2=0x7f96d4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.711+0000 7f96f5200640 1 --2- 172.21.15.62:0/1199287465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96d40039e0 0x7f96d4003e00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f96e0001750 tx=0x7f96d4005830 comp rx=0 tx=0).stop 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.711+0000 7f96efe00640 1 -- 172.21.15.62:0/1199287465 reap_dead start 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.711+0000 7f96f5200640 1 -- 172.21.15.62:0/1199287465 shutdown_connections 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.711+0000 7f96f5200640 1 -- 172.21.15.62:0/1199287465 >> 172.21.15.62:0/1199287465 conn(0x7f96f0072340 msgr2=0x7f96f0072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.711+0000 7f96f5200640 1 -- 172.21.15.62:0/1199287465 shutdown_connections 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.711+0000 7f96f5200640 1 -- 172.21.15.62:0/1199287465 wait complete. 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.713+0000 7f96f5200640 1 Processor -- start 2024-04-25T09:14:10.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.713+0000 7f96f5200640 1 -- start start 2024-04-25T09:14:10.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96f5200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96d40039e0 0x7f96f013fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96f5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96f0153db0 0x7f96f0154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96f5200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f01571c0 0x7f96f0157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96f5200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f006bc30 con 0x7f96f0153db0 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96f5200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f006bab0 con 0x7f96f01571c0 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96f5200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f015e450 con 0x7f96d40039e0 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96eea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96f0153db0 0x7f96f0154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96eea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96f0153db0 0x7f96f0154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36194/0 (socket says 172.21.15.62:36194) 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.714+0000 7f96eea00640 1 -- 172.21.15.62:0/1278486647 learned_addr learned my addr 172.21.15.62:0/1278486647 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.715+0000 7f96ef400640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96d40039e0 0x7f96f013fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.715+0000 7f96efe00640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f01571c0 0x7f96f0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.715+0000 7f96eea00640 1 -- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96d40039e0 msgr2=0x7f96f013fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.715+0000 7f96eea00640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96d40039e0 0x7f96f013fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.715+0000 7f96eea00640 1 -- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f01571c0 msgr2=0x7f96f0157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.715+0000 7f96eea00640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f01571c0 0x7f96f0157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.715+0000 7f96eea00640 1 -- 172.21.15.62:0/1278486647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0151960 con 0x7f96f0153db0 2024-04-25T09:14:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.715+0000 7f96eea00640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96f0153db0 0x7f96f0154210 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00d8b0 tx=0x7f96dc00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.716+0000 7f96e7400640 1 -- 172.21.15.62:0/1278486647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc019070 con 0x7f96f0153db0 2024-04-25T09:14:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.716+0000 7f96e7400640 1 -- 172.21.15.62:0/1278486647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc0074e0 con 0x7f96f0153db0 2024-04-25T09:14:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.716+0000 7f96e7400640 1 -- 172.21.15.62:0/1278486647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0049e0 con 0x7f96f0153db0 2024-04-25T09:14:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.716+0000 7f96f5200640 1 -- 172.21.15.62:0/1278486647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f0140600 con 0x7f96f0153db0 2024-04-25T09:14:10.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.716+0000 7f96f5200640 1 -- 172.21.15.62:0/1278486647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f0158180 con 0x7f96f0153db0 2024-04-25T09:14:10.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.719+0000 7f96e7400640 1 -- 172.21.15.62:0/1278486647 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96dc004b80 con 0x7f96f0153db0 2024-04-25T09:14:10.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.720+0000 7f96e7400640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f96d006b500 0x7f96d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.720+0000 7f96e7400640 1 -- 172.21.15.62:0/1278486647 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f96dc089cb0 con 0x7f96f0153db0 2024-04-25T09:14:10.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.720+0000 7f96f956a740 1 Processor -- start 2024-04-25T09:14:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.721+0000 7f96f956a740 1 -- start start 2024-04-25T09:14:10.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.721+0000 7f96ef400640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f96d006b500 0x7f96d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.721+0000 7f96f956a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.721+0000 7f96f956a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 0x55ba67b04a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.721+0000 7f96f956a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b04fb0 0x55ba67b2ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.721+0000 7f96f956a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67a6d870 con 0x55ba67b34870 2024-04-25T09:14:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.721+0000 7f96f956a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67a16d60 con 0x55ba67b35290 2024-04-25T09:14:10.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.721+0000 7f96f956a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba67a69910 con 0x55ba67b04fb0 2024-04-25T09:14:10.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b34c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36208/0 (socket says 172.21.15.62:36208) 2024-04-25T09:14:10.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96ef400640 1 -- 172.21.15.62:0/3119545421 learned_addr learned my addr 172.21.15.62:0/3119545421 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96efe00640 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b04fb0 0x55ba67b2ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96eea00640 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 0x55ba67b04a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96ef400640 1 -- 172.21.15.62:0/3119545421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b04fb0 msgr2=0x55ba67b2ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96ef400640 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b04fb0 0x55ba67b2ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96ef400640 1 -- 172.21.15.62:0/3119545421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 msgr2=0x55ba67b04a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96ef400640 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 0x55ba67b04a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.722+0000 7f96ef400640 1 -- 172.21.15.62:0/3119545421 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba6784c7d0 con 0x55ba67b34870 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.723+0000 7f96ef400640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f96d006b500 0x7f96d006d9c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f96e0032000 tx=0x7f96e002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.723+0000 7f96ef400640 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b34c50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f96e0001750 tx=0x7f96e003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41446928df9a3af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.723+0000 7f96dbe00640 1 -- 172.21.15.62:0/3119545421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0037040 con 0x55ba67b34870 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.723+0000 7f96dbe00640 1 -- 172.21.15.62:0/3119545421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0007040 con 0x55ba67b34870 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.723+0000 7f96dbe00640 1 -- 172.21.15.62:0/3119545421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00073f0 con 0x55ba67b34870 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 -- 172.21.15.62:0/3119545421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 msgr2=0x55ba67b34c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b34c50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f96e0001750 tx=0x7f96e003d000 comp rx=0 tx=0).stop 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 -- 172.21.15.62:0/3119545421 shutdown_connections 2024-04-25T09:14:10.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b04fb0 0x55ba67b2ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 0x55ba67b04a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 --2- 172.21.15.62:0/3119545421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b34c50 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 -- 172.21.15.62:0/3119545421 >> 172.21.15.62:0/3119545421 conn(0x55ba67b06ef0 msgr2=0x55ba67b4f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 -- 172.21.15.62:0/3119545421 shutdown_connections 2024-04-25T09:14:10.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.724+0000 7f96f956a740 1 -- 172.21.15.62:0/3119545421 wait complete. 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.725+0000 7f96f956a740 1 -- start start 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.725+0000 7f96f956a740 1 Processor -- start 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96f956a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b4ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96f956a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 0x55ba67b36320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96f956a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96f956a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67b30fc0 con 0x55ba67b34870 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96f956a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67a2a1c0 con 0x55ba67b35290 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96f956a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba67b31260 con 0x55ba67b28070 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96f956a740 10 client.0 _ll_register_callbacks cb 0x55ba67a688f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b4ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96ef400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b4ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36216/0 (socket says 172.21.15.62:36216) 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96ef400640 1 -- 172.21.15.62:0/1342364179 learned_addr learned my addr 172.21.15.62:0/1342364179 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.726+0000 7f96efe00640 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96ef400640 1 -- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 msgr2=0x55ba67b36860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96ef400640 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96ef400640 1 -- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 msgr2=0x55ba67b36320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96ef400640 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 0x55ba67b36320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96ef400640 1 -- 172.21.15.62:0/1342364179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba677420f0 con 0x55ba67b34870 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96ef400640 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b4ebf0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f96e0042040 tx=0x7f96e0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b046f82ec5868c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96d8c00640 1 -- 172.21.15.62:0/1342364179 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0006be0 con 0x55ba67b34870 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96d8c00640 1 -- 172.21.15.62:0/1342364179 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f96e0006d80 con 0x55ba67b34870 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.727+0000 7f96f956a740 1 -- 172.21.15.62:0/1342364179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96e0007040 con 0x55ba67b34870 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96d8c00640 1 -- 172.21.15.62:0/1342364179 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e005f050 con 0x55ba67b34870 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96f956a740 10 client.7974 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96f956a740 1 -- 172.21.15.62:0/1342364179 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ba678acf60 con 0x55ba67b34870 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96d8c00640 1 -- 172.21.15.62:0/1342364179 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0007040 con 0x55ba67b34870 2024-04-25T09:14:10.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96d8c00640 1 -- 172.21.15.62:0/1342364179 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f96e0062020 con 0x55ba67b34870 2024-04-25T09:14:10.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96f956a740 10 client.7974 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:10.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96f956a740 10 client.7974 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:10.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96f956a740 20 client.7974 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:10.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96f956a740 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55ba67b33d50 0x55ba67b289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96f956a740 4 client.7974 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:14:10.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.728+0000 7f96f956a740 1 -- 172.21.15.62:0/1342364179 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba67a43690 con 0x55ba67b33d50 2024-04-25T09:14:10.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.729+0000 7f96eea00640 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55ba67b33d50 0x55ba67b289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.729+0000 7f96eea00640 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55ba67b33d50 0x55ba67b289d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29644b16c96ae5c2 server_cookie=d05ca1f621ab1953 in_seq=0 out_seq=0 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.729+0000 7f96d8c00640 10 client.7974 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96d8c00640 1 -- 172.21.15.62:0/1342364179 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba67a43690 con 0x55ba67b33d50 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96d8c00640 10 client.7974 handle_command_reply: tid=0 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 client.7974 shutdown 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 -- 172.21.15.62:0/1342364179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 msgr2=0x55ba67b4ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b4ebf0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f96e0042040 tx=0x7f96e0001190 comp rx=0 tx=0).stop 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 -- 172.21.15.62:0/1342364179 shutdown_connections 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55ba67b33d50 0x55ba67b289d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b35290 0x55ba67b36320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 --2- 172.21.15.62:0/1342364179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b34870 0x55ba67b4ebf0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96d8c00640 0 client.7974 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 -- 172.21.15.62:0/1342364179 >> 172.21.15.62:0/1342364179 conn(0x55ba67b04fb0 msgr2=0x55ba67b4f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 -- 172.21.15.62:0/1342364179 shutdown_connections 2024-04-25T09:14:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.731+0000 7f96f956a740 1 -- 172.21.15.62:0/1342364179 wait complete. 2024-04-25T09:14:10.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.732+0000 7f96f956a740 20 client.7974 trim_cache size 0 max 16384 2024-04-25T09:14:10.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.748+0000 7f96f956a740 1 Processor -- start 2024-04-25T09:14:10.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.748+0000 7f96f956a740 1 -- start start 2024-04-25T09:14:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96f956a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96f956a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b00160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96f956a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b4ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96f956a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f02004e0 con 0x55ba67b35290 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96f956a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f01ff710 con 0x55ba67b34870 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96f956a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0141750 con 0x55ba67b28070 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96efe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b4ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96efe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b4ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36228/0 (socket says 172.21.15.62:36228) 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.749+0000 7f96efe00640 1 -- 172.21.15.62:0/1475598298 learned_addr learned my addr 172.21.15.62:0/1475598298 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96ef400640 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96eea00640 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b00160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96efe00640 1 -- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 msgr2=0x55ba67b36500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96efe00640 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96efe00640 1 -- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 msgr2=0x55ba67b00160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96efe00640 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b00160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96efe00640 1 -- 172.21.15.62:0/1475598298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba6771b1b0 con 0x55ba67b35290 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96ef400640 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96eea00640 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b00160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:10.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96efe00640 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b4ea00 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f96f0074a30 tx=0x7f96e8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37cec1a8ef13895 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96db400640 1 -- 172.21.15.62:0/1475598298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e801b070 con 0x55ba67b35290 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96db400640 1 -- 172.21.15.62:0/1475598298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e8017640 con 0x55ba67b35290 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.750+0000 7f96db400640 1 -- 172.21.15.62:0/1475598298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8017a30 con 0x55ba67b35290 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 -- 172.21.15.62:0/1475598298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 msgr2=0x55ba67b4ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b4ea00 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f96f0074a30 tx=0x7f96e8009750 comp rx=0 tx=0).stop 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 -- 172.21.15.62:0/1475598298 shutdown_connections 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b4ea00 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b00160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 --2- 172.21.15.62:0/1475598298 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b36500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 -- 172.21.15.62:0/1475598298 >> 172.21.15.62:0/1475598298 conn(0x55ba67b04fb0 msgr2=0x55ba67b37a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 -- 172.21.15.62:0/1475598298 shutdown_connections 2024-04-25T09:14:10.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.751+0000 7f96f956a740 1 -- 172.21.15.62:0/1475598298 wait complete. 2024-04-25T09:14:10.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.752+0000 7f96f956a740 1 -- start start 2024-04-25T09:14:10.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.752+0000 7f96f956a740 1 Processor -- start 2024-04-25T09:14:10.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.753+0000 7f96f956a740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b4d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.753+0000 7f96f956a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b51500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.753+0000 7f96f956a740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b51a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.753+0000 7f96f956a740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba67b325d0 con 0x55ba67b35290 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.753+0000 7f96f956a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0141d50 con 0x55ba67b34870 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.753+0000 7f96f956a740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba67b38580 con 0x55ba67b28070 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.753+0000 7f96f956a740 10 client.0 _ll_register_callbacks cb 0x7f96d40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96efe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b51a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96efe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b51a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36238/0 (socket says 172.21.15.62:36238) 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96efe00640 1 -- 172.21.15.62:0/3988093432 learned_addr learned my addr 172.21.15.62:0/3988093432 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96ef400640 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b4d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96eea00640 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b51500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96efe00640 1 -- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 msgr2=0x55ba67b4d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96efe00640 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b4d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96efe00640 1 -- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 msgr2=0x55ba67b51500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96efe00640 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b51500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96efe00640 1 -- 172.21.15.62:0/3988093432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba67b29010 con 0x55ba67b35290 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.754+0000 7f96eea00640 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b51500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96efe00640 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b51a40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f96e80048a0 tx=0x7f96e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b6f2933106b0bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96d8c00640 1 -- 172.21.15.62:0/3988093432 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e802e060 con 0x55ba67b35290 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96f956a740 1 -- 172.21.15.62:0/3988093432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96e8017640 con 0x55ba67b35290 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96d8c00640 1 -- 172.21.15.62:0/3988093432 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f96e8017440 con 0x55ba67b35290 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96d8c00640 1 -- 172.21.15.62:0/3988093432 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e8034050 con 0x55ba67b35290 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96f956a740 10 client.7980 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96f956a740 1 -- 172.21.15.62:0/3988093432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ba67b0bc30 con 0x55ba67b35290 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.755+0000 7f96d8c00640 1 -- 172.21.15.62:0/3988093432 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e8017640 con 0x55ba67b35290 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.756+0000 7f96d8c00640 1 -- 172.21.15.62:0/3988093432 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f96e8037020 con 0x55ba67b35290 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.756+0000 7f96f956a740 10 client.7980 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.756+0000 7f96f956a740 10 client.7980 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.756+0000 7f96f956a740 20 client.7980 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:10.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.756+0000 7f96f956a740 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55ba67b53810 0x55ba67b61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.756+0000 7f96f956a740 4 client.7980 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:14:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.756+0000 7f96f956a740 1 -- 172.21.15.62:0/3988093432 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55ba67a7adb0 con 0x55ba67b53810 2024-04-25T09:14:10.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.756+0000 7f96ef400640 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55ba67b53810 0x55ba67b61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.757+0000 7f96ef400640 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55ba67b53810 0x55ba67b61020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c91d4e20217e058 server_cookie=6337690a88c66a8d in_seq=0 out_seq=0 2024-04-25T09:14:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.757+0000 7f96d8c00640 10 client.7980 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96d8c00640 1 -- 172.21.15.62:0/3988093432 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55ba67a7adb0 con 0x55ba67b53810 2024-04-25T09:14:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96d8c00640 10 client.7980 handle_command_reply: tid=0 2024-04-25T09:14:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 client.7980 shutdown 2024-04-25T09:14:10.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 -- 172.21.15.62:0/3988093432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 msgr2=0x55ba67b51a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b51a40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f96e80048a0 tx=0x7f96e8004a40 comp rx=0 tx=0).stop 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 -- 172.21.15.62:0/3988093432 shutdown_connections 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55ba67b53810 0x55ba67b61020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba67b35290 0x55ba67b51a40 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba67b34870 0x55ba67b51500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 --2- 172.21.15.62:0/3988093432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55ba67b28070 0x55ba67b4d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96d8c00640 0 client.7980 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.758+0000 7f96f956a740 1 -- 172.21.15.62:0/3988093432 >> 172.21.15.62:0/3988093432 conn(0x55ba67b04fb0 msgr2=0x55ba67b4ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.759+0000 7f96f956a740 1 -- 172.21.15.62:0/3988093432 shutdown_connections 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.759+0000 7f96f956a740 1 -- 172.21.15.62:0/3988093432 wait complete. 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.759+0000 7f96f956a740 20 client.7980 trim_cache size 0 max 16384 2024-04-25T09:14:10.747 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:14:10.748 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:14:10.748 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:14:10.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.759+0000 7f96f5200640 1 -- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f96d006b500 msgr2=0x7f96d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.759+0000 7f96f5200640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f96d006b500 0x7f96d006d9c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f96e0032000 tx=0x7f96e002c000 comp rx=0 tx=0).stop 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.759+0000 7f96f5200640 1 -- 172.21.15.62:0/1278486647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96f0153db0 msgr2=0x7f96f0154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.759+0000 7f96f5200640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96f0153db0 0x7f96f0154210 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00d8b0 tx=0x7f96dc00dd70 comp rx=0 tx=0).stop 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.760+0000 7f96f5200640 1 -- 172.21.15.62:0/1278486647 shutdown_connections 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.760+0000 7f96f5200640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f96d006b500 0x7f96d006d9c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.760+0000 7f96f5200640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f96f0153db0 0x7f96f0154210 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.760+0000 7f96f5200640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f01571c0 0x7f96f0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.760+0000 7f96f5200640 1 --2- 172.21.15.62:0/1278486647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f96d40039e0 0x7f96f013fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.760+0000 7f96f5200640 1 -- 172.21.15.62:0/1278486647 >> 172.21.15.62:0/1278486647 conn(0x7f96f0072340 msgr2=0x7f96f015a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.760+0000 7f96f5200640 1 -- 172.21.15.62:0/1278486647 shutdown_connections 2024-04-25T09:14:10.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.760+0000 7f96f5200640 1 -- 172.21.15.62:0/1278486647 wait complete. 2024-04-25T09:14:10.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:10.765 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:14:10.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.986+0000 7f7b3b200640 1 Processor -- start 2024-04-25T09:14:10.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.986+0000 7f7b3b200640 1 -- start start 2024-04-25T09:14:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.988+0000 7f7b3b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d2c0 0x7f7b340ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.988+0000 7f7b3b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7b3406d6a0 0x7f7b340ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.988+0000 7f7b3b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b340ced40 con 0x7f7b340b31a0 2024-04-25T09:14:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.988+0000 7f7b3b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b340cf000 con 0x7f7b3406d6a0 2024-04-25T09:14:10.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.988+0000 7f7b3b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340cf2e0 con 0x7f7b3406d2c0 2024-04-25T09:14:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.988+0000 7f7b39e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d2c0 0x7f7b340ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.988+0000 7f7b39400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7b3406d6a0 0x7f7b340ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.989+0000 7f7b39e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d2c0 0x7f7b340ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:33034/0 (socket says 172.21.15.62:33034) 2024-04-25T09:14:10.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.989+0000 7f7b39e00640 1 -- 172.21.15.62:0/2427897200 learned_addr learned my addr 172.21.15.62:0/2427897200 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.989+0000 7f7b39e00640 1 -- 172.21.15.62:0/2427897200 >> v2:172.21.15.121:3300/0 conn(0x7f7b3406d6a0 msgr2=0x7f7b340ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.989+0000 7f7b39e00640 1 --2- 172.21.15.62:0/2427897200 >> v2:172.21.15.121:3300/0 conn(0x7f7b3406d6a0 0x7f7b340ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.989+0000 7f7b39e00640 1 -- 172.21.15.62:0/2427897200 >> v1:172.21.15.121:6789/0 conn(0x7f7b340b31a0 legacy=0x7f7b340b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.989+0000 7f7b39e00640 1 -- 172.21.15.62:0/2427897200 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b340b26c0 con 0x7f7b3406d2c0 2024-04-25T09:14:10.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.989+0000 7f7b39400640 1 --2- 172.21.15.62:0/2427897200 >> v2:172.21.15.121:3300/0 conn(0x7f7b3406d6a0 0x7f7b340ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:14:10.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.991+0000 7f7b39e00640 1 --2- 172.21.15.62:0/2427897200 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d2c0 0x7f7b340ccde0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009870 tx=0x7f7b2802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b520d6ee29a19f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.991+0000 7f7b38a00640 1 -- 172.21.15.62:0/2427897200 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28037070 con 0x7f7b3406d2c0 2024-04-25T09:14:10.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.991+0000 7f7b38a00640 1 -- 172.21.15.62:0/2427897200 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b28032440 con 0x7f7b3406d2c0 2024-04-25T09:14:10.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.991+0000 7f7b38a00640 1 -- 172.21.15.62:0/2427897200 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28032810 con 0x7f7b3406d2c0 2024-04-25T09:14:10.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.992+0000 7f7b3b200640 1 -- 172.21.15.62:0/2427897200 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d2c0 msgr2=0x7f7b340ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.992+0000 7f7b3b200640 1 --2- 172.21.15.62:0/2427897200 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d2c0 0x7f7b340ccde0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009870 tx=0x7f7b2802a250 comp rx=0 tx=0).stop 2024-04-25T09:14:10.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.992+0000 7f7b3b200640 1 -- 172.21.15.62:0/2427897200 shutdown_connections 2024-04-25T09:14:10.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.992+0000 7f7b3b200640 1 --2- 172.21.15.62:0/2427897200 >> v2:172.21.15.121:3300/0 conn(0x7f7b3406d6a0 0x7f7b340ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.992+0000 7f7b3b200640 1 --2- 172.21.15.62:0/2427897200 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d2c0 0x7f7b340ccde0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.992+0000 7f7b3b200640 1 -- 172.21.15.62:0/2427897200 >> 172.21.15.62:0/2427897200 conn(0x7f7b34066f20 msgr2=0x7f7b340b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.992+0000 7f7b3b200640 1 -- 172.21.15.62:0/2427897200 shutdown_connections 2024-04-25T09:14:10.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.992+0000 7f7b3b200640 1 -- 172.21.15.62:0/2427897200 wait complete. 2024-04-25T09:14:10.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.994+0000 7f7b3b200640 1 Processor -- start 2024-04-25T09:14:10.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.994+0000 7f7b3b200640 1 -- start start 2024-04-25T09:14:10.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b3b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3406d2c0 0x7f7b341e6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b3b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d6a0 0x7f7b34142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b3b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b340b31a0 0x7f7b34070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b3b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3406db20 con 0x7f7b3406d2c0 2024-04-25T09:14:10.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b3b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340b2c80 con 0x7f7b340b31a0 2024-04-25T09:14:10.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b3b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b341448d0 con 0x7f7b3406d6a0 2024-04-25T09:14:10.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b39e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3406d2c0 0x7f7b341e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b39e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3406d2c0 0x7f7b341e6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36248/0 (socket says 172.21.15.62:36248) 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.995+0000 7f7b39e00640 1 -- 172.21.15.62:0/1039262943 learned_addr learned my addr 172.21.15.62:0/1039262943 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.996+0000 7f7b39400640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d6a0 0x7f7b34142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.996+0000 7f7b3a800640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b340b31a0 0x7f7b34070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.996+0000 7f7b39e00640 1 -- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d6a0 msgr2=0x7f7b34142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.996+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d6a0 0x7f7b34142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.996+0000 7f7b39e00640 1 -- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b340b31a0 msgr2=0x7f7b34070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.996+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b340b31a0 0x7f7b34070c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.996+0000 7f7b39e00640 1 -- 172.21.15.62:0/1039262943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34143540 con 0x7f7b3406d2c0 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.996+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3406d2c0 0x7f7b341e6560 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009870 tx=0x7f7b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.997+0000 7f7b2e000640 1 -- 172.21.15.62:0/1039262943 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28041070 con 0x7f7b3406d2c0 2024-04-25T09:14:10.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.997+0000 7f7b2e000640 1 -- 172.21.15.62:0/1039262943 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2803c890 con 0x7f7b3406d2c0 2024-04-25T09:14:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.997+0000 7f7b2e000640 1 -- 172.21.15.62:0/1039262943 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28037040 con 0x7f7b3406d2c0 2024-04-25T09:14:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.997+0000 7f7b3b200640 1 -- 172.21.15.62:0/1039262943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b341e6aa0 con 0x7f7b3406d2c0 2024-04-25T09:14:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.997+0000 7f7b3b200640 1 -- 172.21.15.62:0/1039262943 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b341e57f0 con 0x7f7b3406d2c0 2024-04-25T09:14:10.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:10.999+0000 7f7b2e000640 1 -- 172.21.15.62:0/1039262943 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b28044070 con 0x7f7b3406d2c0 2024-04-25T09:14:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.000+0000 7f7b2e000640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7b1806b4d0 0x7f7b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.000+0000 7f7b2e000640 1 -- 172.21.15.62:0/1039262943 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7b280aac90 con 0x7f7b3406d2c0 2024-04-25T09:14:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.001+0000 7f7b39400640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7b1806b4d0 0x7f7b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.001+0000 7f7b3f59c740 1 Processor -- start 2024-04-25T09:14:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.001+0000 7f7b3f59c740 1 -- start start 2024-04-25T09:14:10.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b39400640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7b1806b4d0 0x7f7b1806d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009560 tx=0x7f7b24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 0x560b255cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b25607440 0x560b255cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b2552ec70 con 0x560b25607440 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b254d7cf0 con 0x560b255f5830 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b2552ad10 con 0x560b255cad70 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b39400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b25607440 0x560b255cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b39400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b25607440 0x560b255cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36262/0 (socket says 172.21.15.62:36262) 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.002+0000 7f7b39400640 1 -- 172.21.15.62:0/751229468 learned_addr learned my addr 172.21.15.62:0/751229468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b39e00640 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 0x560b255cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b3a800640 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b39400640 1 -- 172.21.15.62:0/751229468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 msgr2=0x560b255cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b39400640 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 0x560b255cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b39400640 1 -- 172.21.15.62:0/751229468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 msgr2=0x560b255f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b39400640 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b39400640 1 -- 172.21.15.62:0/751229468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b2530d7d0 con 0x560b25607440 2024-04-25T09:14:10.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b39400640 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b25607440 0x560b255cb150 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7b24011d30 tx=0x7f7b24014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb80e6c2a95b4ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.003+0000 7f7b22600640 1 -- 172.21.15.62:0/751229468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24011d60 con 0x560b25607440 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b22600640 1 -- 172.21.15.62:0/751229468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24026070 con 0x560b25607440 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b22600640 1 -- 172.21.15.62:0/751229468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24021640 con 0x560b25607440 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b3f59c740 1 -- 172.21.15.62:0/751229468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b25607440 msgr2=0x560b255cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b25607440 0x560b255cb150 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7b24011d30 tx=0x7f7b24014000 comp rx=0 tx=0).stop 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b3f59c740 1 -- 172.21.15.62:0/751229468 shutdown_connections 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b25607440 0x560b255cb150 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/751229468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 0x560b255cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.004+0000 7f7b3f59c740 1 -- 172.21.15.62:0/751229468 >> 172.21.15.62:0/751229468 conn(0x560b25592a30 msgr2=0x560b25610830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:10.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.005+0000 7f7b3f59c740 1 -- 172.21.15.62:0/751229468 shutdown_connections 2024-04-25T09:14:10.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.005+0000 7f7b3f59c740 1 -- 172.21.15.62:0/751229468 wait complete. 2024-04-25T09:14:10.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.005+0000 7f7b3f59c740 1 -- start start 2024-04-25T09:14:10.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.005+0000 7f7b3f59c740 1 Processor -- start 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.006+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 0x560b255ea660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.006+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255f5830 0x560b25613dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.006+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b25607440 0x560b25614310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.006+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b254cd3a0 con 0x560b255f5830 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.006+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b255f3380 con 0x560b25607440 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.006+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b255f8880 con 0x560b255cad70 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.006+0000 7f7b3f59c740 10 client.0 _ll_register_callbacks cb 0x560b254eab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255f5830 0x560b25613dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255f5830 0x560b25613dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36266/0 (socket says 172.21.15.62:36266) 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39400640 1 -- 172.21.15.62:0/1405506969 learned_addr learned my addr 172.21.15.62:0/1405506969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:10.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39400640 1 -- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 msgr2=0x560b255ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 0x560b255ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b3a800640 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b25607440 0x560b25614310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39400640 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 0x560b255ea660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39400640 1 -- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b25607440 msgr2=0x560b25614310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39400640 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b25607440 0x560b25614310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.007+0000 7f7b39400640 1 -- 172.21.15.62:0/1405506969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b252433f0 con 0x560b255f5830 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b39400640 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255f5830 0x560b25613dd0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400a010 tx=0x7f7b240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f1fe7a772c9819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b16a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b16a00640 1 -- 172.21.15.62:0/1405506969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24011970 con 0x560b255f5830 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1405506969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b24026070 con 0x560b255f5830 2024-04-25T09:14:10.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b16a00640 1 -- 172.21.15.62:0/1405506969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7b24021b10 con 0x560b255f5830 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b16a00640 1 -- 172.21.15.62:0/1405506969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b2403e050 con 0x560b255f5830 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b3f59c740 10 client.7989 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1405506969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x560b252030f0 con 0x560b255f5830 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.008+0000 7f7b16a00640 1 -- 172.21.15.62:0/1405506969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24026070 con 0x560b255f5830 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.009+0000 7f7b16a00640 1 -- 172.21.15.62:0/1405506969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7b24041020 con 0x560b255f5830 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.009+0000 7f7b3f59c740 10 client.7989 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.009+0000 7f7b3f59c740 10 client.7989 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.009+0000 7f7b3f59c740 20 client.7989 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.009+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x560b25612b80 0x560b255f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.009+0000 7f7b3f59c740 4 client.7989 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.009+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1405506969 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b2553c1b0 con 0x560b25612b80 2024-04-25T09:14:10.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.009+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x560b25612b80 0x560b255f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.011+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x560b25612b80 0x560b255f7a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703f46b4d37acab4 server_cookie=97cfb5fd2c2dbb0b in_seq=0 out_seq=0 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.011+0000 7f7b16a00640 10 client.7989 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.012+0000 7f7b16a00640 1 -- 172.21.15.62:0/1405506969 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b2553c1b0 con 0x560b25612b80 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.012+0000 7f7b16a00640 10 client.7989 handle_command_reply: tid=0 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.012+0000 7f7b3f59c740 1 client.7989 shutdown 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1405506969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255f5830 msgr2=0x560b25613dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255f5830 0x560b25613dd0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400a010 tx=0x7f7b240046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1405506969 shutdown_connections 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x560b25612b80 0x560b255f7a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255f5830 0x560b25613dd0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b25607440 0x560b25614310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1405506969 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b255cad70 0x560b255ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1405506969 >> 172.21.15.62:0/1405506969 conn(0x560b25592a30 msgr2=0x560b255f3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b16a00640 0 client.7989 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1405506969 shutdown_connections 2024-04-25T09:14:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1405506969 wait complete. 2024-04-25T09:14:11.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.013+0000 7f7b3f59c740 20 client.7989 trim_cache size 0 max 16384 2024-04-25T09:14:11.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.030+0000 7f7b3f59c740 1 Processor -- start 2024-04-25T09:14:11.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.030+0000 7f7b3f59c740 1 -- start start 2024-04-25T09:14:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b255f9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b341ffd70 con 0x560b255cad70 2024-04-25T09:14:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b341fefa0 con 0x560b255f5830 2024-04-25T09:14:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b341f79d0 con 0x560b25607440 2024-04-25T09:14:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b39e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b39e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36280/0 (socket says 172.21.15.62:36280) 2024-04-25T09:14:11.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b39e00640 1 -- 172.21.15.62:0/1866962853 learned_addr learned my addr 172.21.15.62:0/1866962853 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.031+0000 7f7b3a800640 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b255f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b39400640 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b39e00640 1 -- 172.21.15.62:0/1866962853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 msgr2=0x560b255f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b255f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b39e00640 1 -- 172.21.15.62:0/1866962853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 msgr2=0x560b255f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b39e00640 1 -- 172.21.15.62:0/1866962853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b255f5ee0 con 0x560b255cad70 2024-04-25T09:14:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f2cf0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7b2802a840 tx=0x7f7b28006ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d7b0bf17d0309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b21c00640 1 -- 172.21.15.62:0/1866962853 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2804b040 con 0x560b255cad70 2024-04-25T09:14:11.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b21c00640 1 -- 172.21.15.62:0/1866962853 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b28079330 con 0x560b255cad70 2024-04-25T09:14:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.032+0000 7f7b21c00640 1 -- 172.21.15.62:0/1866962853 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2807d480 con 0x560b255cad70 2024-04-25T09:14:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1866962853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 msgr2=0x560b255f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f2cf0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f7b2802a840 tx=0x7f7b28006ad0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1866962853 shutdown_connections 2024-04-25T09:14:11.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b255f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1866962853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f2cf0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1866962853 >> 172.21.15.62:0/1866962853 conn(0x560b25592a30 msgr2=0x560b255ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:11.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1866962853 shutdown_connections 2024-04-25T09:14:11.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.033+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1866962853 wait complete. 2024-04-25T09:14:11.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.034+0000 7f7b3f59c740 1 -- start start 2024-04-25T09:14:11.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.034+0000 7f7b3f59c740 1 Processor -- start 2024-04-25T09:14:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b3f59c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b25613670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b255c6060 con 0x560b255cad70 2024-04-25T09:14:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b341e7e90 con 0x560b255f5830 2024-04-25T09:14:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b3f59c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b25561ca0 con 0x560b25607440 2024-04-25T09:14:11.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b3f59c740 10 client.0 _ll_register_callbacks cb 0x7f7b24011bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b39e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b39e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36288/0 (socket says 172.21.15.62:36288) 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.035+0000 7f7b39e00640 1 -- 172.21.15.62:0/1670712647 learned_addr learned my addr 172.21.15.62:0/1670712647 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b3a800640 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b25613670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b39400640 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b39e00640 1 -- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 msgr2=0x560b25613670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b25613670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b39e00640 1 -- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 msgr2=0x560b255f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b39e00640 1 -- 172.21.15.62:0/1670712647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b255c4bd0 con 0x560b255cad70 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b3a800640 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b25613670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b39e00640 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f7a10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009080 tx=0x7f7b28048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a6d8e5713ae53b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b16a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:11.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b16a00640 1 -- 172.21.15.62:0/1670712647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2807d480 con 0x560b255cad70 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b16a00640 1 -- 172.21.15.62:0/1670712647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7b280a9160 con 0x560b255cad70 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1670712647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b28079330 con 0x560b255cad70 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.036+0000 7f7b16a00640 1 -- 172.21.15.62:0/1670712647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b280759a0 con 0x560b255cad70 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b3f59c740 10 client.7995 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1670712647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x560b255e7b10 con 0x560b255cad70 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b16a00640 1 -- 172.21.15.62:0/1670712647 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b28079330 con 0x560b255cad70 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b16a00640 1 -- 172.21.15.62:0/1670712647 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7b2808e020 con 0x560b255cad70 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b3f59c740 10 client.7995 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b3f59c740 10 client.7995 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b3f59c740 20 client.7995 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x560b255e76d0 0x560b25613bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b3f59c740 4 client.7995 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.037+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1670712647 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b25536500 con 0x560b255e76d0 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.038+0000 7f7b39400640 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x560b255e76d0 0x560b25613bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.039+0000 7f7b39400640 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x560b255e76d0 0x560b25613bb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55ec6104cc22771 server_cookie=2ac6934ffac9344d in_seq=0 out_seq=0 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.039+0000 7f7b16a00640 10 client.7995 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b16a00640 1 -- 172.21.15.62:0/1670712647 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x560b25536500 con 0x560b255e76d0 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b16a00640 10 client.7995 handle_command_reply: tid=0 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 client.7995 shutdown 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1670712647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 msgr2=0x560b255f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f7a10 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009080 tx=0x7f7b28048040 comp rx=0 tx=0).stop 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1670712647 shutdown_connections 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x560b255e76d0 0x560b25613bb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x560b25607440 0x560b25613670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b255f5830 0x560b255f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 --2- 172.21.15.62:0/1670712647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b255cad70 0x560b255f7a10 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b16a00640 0 client.7995 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1670712647 >> 172.21.15.62:0/1670712647 conn(0x560b25592a30 msgr2=0x560b255c13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:11.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1670712647 shutdown_connections 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 1 -- 172.21.15.62:0/1670712647 wait complete. 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.040+0000 7f7b3f59c740 20 client.7995 trim_cache size 0 max 16384 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "0ae8b2d2-cab3-400f-b6de-a6ece47182b4": { 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "0ae8b2d2-cab3-400f-b6de-a6ece47182b4", 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive" 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:11.035 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:11.036 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:14:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 -- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7b1806b4d0 msgr2=0x7f7b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7b1806b4d0 0x7f7b1806d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009560 tx=0x7f7b24008040 comp rx=0 tx=0).stop 2024-04-25T09:14:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 -- 172.21.15.62:0/1039262943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3406d2c0 msgr2=0x7f7b341e6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3406d2c0 0x7f7b341e6560 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009870 tx=0x7f7b280046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 -- 172.21.15.62:0/1039262943 shutdown_connections 2024-04-25T09:14:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7b1806b4d0 0x7f7b1806d990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b340b31a0 0x7f7b34070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7b3406d6a0 0x7f7b34142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 --2- 172.21.15.62:0/1039262943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3406d2c0 0x7f7b341e6560 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 -- 172.21.15.62:0/1039262943 >> 172.21.15.62:0/1039262943 conn(0x7f7b34066f20 msgr2=0x7f7b340af170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 -- 172.21.15.62:0/1039262943 shutdown_connections 2024-04-25T09:14:11.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.041+0000 7f7b3b200640 1 -- 172.21.15.62:0/1039262943 wait complete. 2024-04-25T09:14:11.046 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:11.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.272+0000 7f5c57200640 1 Processor -- start 2024-04-25T09:14:11.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.272+0000 7f5c57200640 1 -- start start 2024-04-25T09:14:11.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.273+0000 7f5c57200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 0x7f5c580d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.273+0000 7f5c57200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580d3110 con 0x7f5c580b56f0 2024-04-25T09:14:11.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.273+0000 7f5c57200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580d33f0 con 0x7f5c580b6a90 2024-04-25T09:14:11.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.273+0000 7f5c57200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580cedc0 con 0x7f5c580b27a0 2024-04-25T09:14:11.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.274+0000 7f5c55e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5c580b56f0 0x7f5c58160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:47560/0 (socket says 172.21.15.62:47560) 2024-04-25T09:14:11.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.274+0000 7f5c55e00640 1 -- 172.21.15.62:0/399804562 learned_addr learned my addr 172.21.15.62:0/399804562 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:11.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.274+0000 7f5c56800640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 0x7f5c580d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.274+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437700761 0 0) 0x7f5c580d3110 con 0x7f5c580b56f0 2024-04-25T09:14:11.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.274+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c3c003520 con 0x7f5c580b56f0 2024-04-25T09:14:11.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212074955 0 0) 0x7f5c3c003520 con 0x7f5c580b56f0 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 msgr2=0x7f5c580d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 0x7f5c580d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 >> v1:172.21.15.121:6789/0 conn(0x7f5c580b6a90 legacy=0x7f5c580d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580d36d0 con 0x7f5c580b56f0 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f5c48001670 con 0x7f5c580b56f0 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 >> v1:172.21.15.62:6789/0 conn(0x7f5c580b56f0 legacy=0x7f5c58160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0039e0 0x7f5c3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c3c004800 0x7f5c3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 0x7f5c3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580cedc0 con 0x7f5c3c004800 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c00bac0 con 0x7f5c3c0039e0 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c00bcc0 con 0x7f5c580b27a0 2024-04-25T09:14:11.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.275+0000 7f5c55400640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c3c004800 0x7f5c3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c55e00640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0039e0 0x7f5c3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c56800640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 0x7f5c3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c55400640 1 -- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 msgr2=0x7f5c3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c55400640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 0x7f5c3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c55400640 1 -- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0039e0 msgr2=0x7f5c3c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c55400640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0039e0 0x7f5c3c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c55400640 1 -- 172.21.15.62:0/399804562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c003520 con 0x7f5c3c004800 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c56800640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580b27a0 0x7f5c3c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:11.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.276+0000 7f5c55e00640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c0039e0 0x7f5c3c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:11.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.277+0000 7f5c55400640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c3c004800 0x7f5c3c004c80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5c40001720 tx=0x7f5c3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694c48a179395770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:11.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.277+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40001370 con 0x7f5c3c004800 2024-04-25T09:14:11.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.277+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c400052d0 con 0x7f5c3c004800 2024-04-25T09:14:11.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.277+0000 7f5c54a00640 1 -- 172.21.15.62:0/399804562 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4002abc0 con 0x7f5c3c004800 2024-04-25T09:14:11.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.278+0000 7f5c57200640 1 -- 172.21.15.62:0/399804562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c3c004800 msgr2=0x7f5c3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.278+0000 7f5c57200640 1 --2- 172.21.15.62:0/399804562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c3c004800 0x7f5c3c004c80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5c40001720 tx=0x7f5c3c005830 comp rx=0 tx=0).stop 2024-04-25T09:14:11.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.278+0000 7f5c56800640 1 -- 172.21.15.62:0/399804562 reap_dead start 2024-04-25T09:14:11.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.278+0000 7f5c57200640 1 -- 172.21.15.62:0/399804562 shutdown_connections 2024-04-25T09:14:11.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.278+0000 7f5c57200640 1 -- 172.21.15.62:0/399804562 >> 172.21.15.62:0/399804562 conn(0x7f5c58066d10 msgr2=0x7f5c58067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:11.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.278+0000 7f5c57200640 1 -- 172.21.15.62:0/399804562 shutdown_connections 2024-04-25T09:14:11.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.278+0000 7f5c57200640 1 -- 172.21.15.62:0/399804562 wait complete. 2024-04-25T09:14:11.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.280+0000 7f5c57200640 1 Processor -- start 2024-04-25T09:14:11.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.280+0000 7f5c57200640 1 -- start start 2024-04-25T09:14:11.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.281+0000 7f5c57200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c004800 0x7f5c581f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.281+0000 7f5c57200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580c99d0 0x7f5c5815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.281+0000 7f5c57200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c580b8090 0x7f5c580b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.281+0000 7f5c57200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b2ee0 con 0x7f5c580b8090 2024-04-25T09:14:11.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.281+0000 7f5c57200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b2d60 con 0x7f5c3c004800 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.281+0000 7f5c57200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b5ad0 con 0x7f5c580c99d0 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.281+0000 7f5c56800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c580b8090 0x7f5c580b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c56800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c580b8090 0x7f5c580b8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36312/0 (socket says 172.21.15.62:36312) 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c56800640 1 -- 172.21.15.62:0/2902157092 learned_addr learned my addr 172.21.15.62:0/2902157092 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c55400640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580c99d0 0x7f5c5815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c56800640 1 -- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580c99d0 msgr2=0x7f5c5815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c55e00640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c004800 0x7f5c581f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c56800640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580c99d0 0x7f5c5815df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c56800640 1 -- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c004800 msgr2=0x7f5c581f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c56800640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c004800 0x7f5c581f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c56800640 1 -- 172.21.15.62:0/2902157092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c581e5880 con 0x7f5c580b8090 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c55e00640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c004800 0x7f5c581f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.282+0000 7f5c56800640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c580b8090 0x7f5c580b8510 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5c580d4ad0 tx=0x7f5c5000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:11.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.283+0000 7f5c4e000640 1 -- 172.21.15.62:0/2902157092 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5001c070 con 0x7f5c580b8090 2024-04-25T09:14:11.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.283+0000 7f5c4e000640 1 -- 172.21.15.62:0/2902157092 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c50017400 con 0x7f5c580b8090 2024-04-25T09:14:11.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.283+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c581f7630 con 0x7f5c580b8090 2024-04-25T09:14:11.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.283+0000 7f5c4e000640 1 -- 172.21.15.62:0/2902157092 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500177f0 con 0x7f5c580b8090 2024-04-25T09:14:11.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.283+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c581f7bf0 con 0x7f5c580b8090 2024-04-25T09:14:11.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.284+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5804ef20 con 0x7f5c580b8090 2024-04-25T09:14:11.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.285+0000 7f5c4e000640 1 -- 172.21.15.62:0/2902157092 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c50005d10 con 0x7f5c580b8090 2024-04-25T09:14:11.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.286+0000 7f5c4e000640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c3406b500 0x7f5c3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.286+0000 7f5c4e000640 1 -- 172.21.15.62:0/2902157092 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5c5008a440 con 0x7f5c580b8090 2024-04-25T09:14:11.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.287+0000 7f5c55e00640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c3406b500 0x7f5c3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.288+0000 7f5c55e00640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c3406b500 0x7f5c3406d9c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5c48001eb0 tx=0x7f5c48009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:11.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.291+0000 7f5c4e000640 1 -- 172.21.15.62:0/2902157092 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c50012070 con 0x7f5c580b8090 2024-04-25T09:14:11.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.533+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c580ce580 con 0x7f5c580b8090 2024-04-25T09:14:11.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.535+0000 7f5c4e000640 1 -- 172.21.15.62:0/2902157092 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5c500591e0 con 0x7f5c580b8090 2024-04-25T09:14:11.522 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:11.522 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:11.522 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c3406b500 msgr2=0x7f5c3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c3406b500 0x7f5c3406d9c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5c48001eb0 tx=0x7f5c48009040 comp rx=0 tx=0).stop 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c580b8090 msgr2=0x7f5c580b8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c580b8090 0x7f5c580b8510 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5c580d4ad0 tx=0x7f5c5000f4a0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 shutdown_connections 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c3406b500 0x7f5c3406d9c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c580b8090 0x7f5c580b8510 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c580c99d0 0x7f5c5815df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 --2- 172.21.15.62:0/2902157092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c3c004800 0x7f5c581f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 >> 172.21.15.62:0/2902157092 conn(0x7f5c58066d10 msgr2=0x7f5c580b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 shutdown_connections 2024-04-25T09:14:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.538+0000 7f5c57200640 1 -- 172.21.15.62:0/2902157092 wait complete. 2024-04-25T09:14:11.544 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:14:11.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:14:11.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.765+0000 7fb549000640 1 Processor -- start 2024-04-25T09:14:11.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.765+0000 7fb549000640 1 -- start start 2024-04-25T09:14:11.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.766+0000 7fb549000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415ee40 0x7fb54415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.766+0000 7fb549000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb54415d660 0x7fb5440702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.766+0000 7fb549000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb544070800 0x7fb54415f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.766+0000 7fb549000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb544071320 con 0x7fb54415d660 2024-04-25T09:14:11.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.767+0000 7fb549000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb544070d60 con 0x7fb544070800 2024-04-25T09:14:11.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.767+0000 7fb549000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb544071040 con 0x7fb54415ee40 2024-04-25T09:14:11.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.767+0000 7fb541400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb54415d660 0x7fb5440702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.767+0000 7fb541400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb54415d660 0x7fb5440702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36320/0 (socket says 172.21.15.62:36320) 2024-04-25T09:14:11.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.767+0000 7fb541400640 1 -- 172.21.15.62:0/2947948818 learned_addr learned my addr 172.21.15.62:0/2947948818 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:11.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.767+0000 7fb542800640 1 --2- 172.21.15.62:0/2947948818 >> v2:172.21.15.121:3300/0 conn(0x7fb544070800 0x7fb54415f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.768+0000 7fb541e00640 1 --2- 172.21.15.62:0/2947948818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415ee40 0x7fb54415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.768+0000 7fb541400640 1 -- 172.21.15.62:0/2947948818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415ee40 msgr2=0x7fb54415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.768+0000 7fb541400640 1 --2- 172.21.15.62:0/2947948818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415ee40 0x7fb54415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.768+0000 7fb541400640 1 -- 172.21.15.62:0/2947948818 >> v2:172.21.15.121:3300/0 conn(0x7fb544070800 msgr2=0x7fb54415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:11.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.768+0000 7fb541400640 1 --2- 172.21.15.62:0/2947948818 >> v2:172.21.15.121:3300/0 conn(0x7fb544070800 0x7fb54415f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.768+0000 7fb541400640 1 -- 172.21.15.62:0/2947948818 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440b2820 con 0x7fb54415d660 2024-04-25T09:14:11.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.769+0000 7fb541400640 1 --2- 172.21.15.62:0/2947948818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb54415d660 0x7fb5440702c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fb5440678a0 tx=0x7fb52c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2366bcdb4e574fea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:11.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.769+0000 7fb540a00640 1 -- 172.21.15.62:0/2947948818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c037070 con 0x7fb54415d660 2024-04-25T09:14:11.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb540a00640 1 -- 172.21.15.62:0/2947948818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c032440 con 0x7fb54415d660 2024-04-25T09:14:11.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb540a00640 1 -- 172.21.15.62:0/2947948818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c032810 con 0x7fb54415d660 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb549000640 1 -- 172.21.15.62:0/2947948818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb54415d660 msgr2=0x7fb5440702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb549000640 1 --2- 172.21.15.62:0/2947948818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb54415d660 0x7fb5440702c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fb5440678a0 tx=0x7fb52c02a140 comp rx=0 tx=0).stop 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb549000640 1 -- 172.21.15.62:0/2947948818 shutdown_connections 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb549000640 1 --2- 172.21.15.62:0/2947948818 >> v2:172.21.15.121:3300/0 conn(0x7fb544070800 0x7fb54415f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb549000640 1 --2- 172.21.15.62:0/2947948818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb54415d660 0x7fb5440702c0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb549000640 1 --2- 172.21.15.62:0/2947948818 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415ee40 0x7fb54415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.770+0000 7fb549000640 1 -- 172.21.15.62:0/2947948818 >> 172.21.15.62:0/2947948818 conn(0x7fb5440668f0 msgr2=0x7fb544066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.771+0000 7fb549000640 1 -- 172.21.15.62:0/2947948818 shutdown_connections 2024-04-25T09:14:11.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.771+0000 7fb549000640 1 -- 172.21.15.62:0/2947948818 wait complete. 2024-04-25T09:14:11.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.772+0000 7fb549000640 1 Processor -- start 2024-04-25T09:14:11.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.772+0000 7fb549000640 1 -- start start 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.773+0000 7fb549000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb544070800 0x7fb54414ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.773+0000 7fb549000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415d660 0x7fb544151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.773+0000 7fb549000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54415ee40 0x7fb544151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.773+0000 7fb549000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54415f220 con 0x7fb544070800 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.773+0000 7fb549000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441603d0 con 0x7fb54415ee40 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.773+0000 7fb549000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54415f3a0 con 0x7fb54415d660 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb544070800 0x7fb54414ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb544070800 0x7fb54414ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36328/0 (socket says 172.21.15.62:36328) 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541e00640 1 -- 172.21.15.62:0/370529850 learned_addr learned my addr 172.21.15.62:0/370529850 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb542800640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54415ee40 0x7fb544151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541400640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415d660 0x7fb544151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541e00640 1 -- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415d660 msgr2=0x7fb544151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541e00640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415d660 0x7fb544151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541e00640 1 -- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54415ee40 msgr2=0x7fb544151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:11.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541e00640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54415ee40 0x7fb544151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:11.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541e00640 1 -- 172.21.15.62:0/370529850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440cf2b0 con 0x7fb544070800 2024-04-25T09:14:11.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb541400640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415d660 0x7fb544151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:11.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.774+0000 7fb542800640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54415ee40 0x7fb544151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:11.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.775+0000 7fb541e00640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb544070800 0x7fb54414ce40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb53400d800 tx=0x7fb53400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:11.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.775+0000 7fb53a000640 1 -- 172.21.15.62:0/370529850 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53400b6c0 con 0x7fb544070800 2024-04-25T09:14:11.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.775+0000 7fb53a000640 1 -- 172.21.15.62:0/370529850 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb534019070 con 0x7fb544070800 2024-04-25T09:14:11.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.775+0000 7fb53a000640 1 -- 172.21.15.62:0/370529850 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53400ba80 con 0x7fb544070800 2024-04-25T09:14:11.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.775+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5440cf590 con 0x7fb544070800 2024-04-25T09:14:11.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.775+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5440cfb50 con 0x7fb544070800 2024-04-25T09:14:11.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.776+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54404ef20 con 0x7fb544070800 2024-04-25T09:14:11.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.777+0000 7fb53a000640 1 -- 172.21.15.62:0/370529850 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb53400bc20 con 0x7fb544070800 2024-04-25T09:14:11.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.778+0000 7fb53a000640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb52006b4d0 0x7fb52006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:11.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.778+0000 7fb53a000640 1 -- 172.21.15.62:0/370529850 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb53408a790 con 0x7fb544070800 2024-04-25T09:14:11.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.779+0000 7fb541400640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb52006b4d0 0x7fb52006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:11.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.780+0000 7fb541400640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb52006b4d0 0x7fb52006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb52c034000 tx=0x7fb52c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:11.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:11.783+0000 7fb53a000640 1 -- 172.21.15.62:0/370529850 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb534059530 con 0x7fb544070800 2024-04-25T09:14:12.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.059+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb5440b0670 con 0x7fb544070800 2024-04-25T09:14:12.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.060+0000 7fb53a000640 1 -- 172.21.15.62:0/370529850 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1350 (secure 0 0 0) 0x7fb534058e30 con 0x7fb544070800 2024-04-25T09:14:12.047 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:12.047 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":908,"data_bytes":929429479,"bytes_used":2257072128,"bytes_avail":770837041152,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":92,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":768,"write_bytes_sec":21358870,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7800}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T09:13:20.117690+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:14:12.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb52006b4d0 msgr2=0x7fb52006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:12.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb52006b4d0 0x7fb52006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb52c034000 tx=0x7fb52c02e000 comp rx=0 tx=0).stop 2024-04-25T09:14:12.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb544070800 msgr2=0x7fb54414ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb544070800 0x7fb54414ce40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb53400d800 tx=0x7fb53400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 shutdown_connections 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb52006b4d0 0x7fb52006d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54415ee40 0x7fb544151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb54415d660 0x7fb544151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 --2- 172.21.15.62:0/370529850 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb544070800 0x7fb54414ce40 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 >> 172.21.15.62:0/370529850 conn(0x7fb5440668f0 msgr2=0x7fb54415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 shutdown_connections 2024-04-25T09:14:12.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:12.063+0000 7fb549000640 1 -- 172.21.15.62:0/370529850 wait complete. 2024-04-25T09:14:12.067 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:14:13.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.303+0000 7f7159c00640 1 Processor -- start 2024-04-25T09:14:13.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.303+0000 7f7159c00640 1 -- start start 2024-04-25T09:14:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.304+0000 7f7159c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f71540d1b40 0x7f71540b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.304+0000 7f7159c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f71540b6640 0x7f715406fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.304+0000 7f7159c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715415bb90 con 0x7f715415b7b0 2024-04-25T09:14:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.304+0000 7f7159c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715415e250 con 0x7f71540d1b40 2024-04-25T09:14:13.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.304+0000 7f7159c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715415e530 con 0x7f71540b6640 2024-04-25T09:14:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7153e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f715415b7b0 0x7f71540d1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:47570/0 (socket says 172.21.15.62:47570) 2024-04-25T09:14:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7153e00640 1 -- 172.21.15.62:0/2961179806 learned_addr learned my addr 172.21.15.62:0/2961179806 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:13.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7153400640 1 --2- 172.21.15.62:0/2961179806 >> v2:172.21.15.62:3300/0 conn(0x7f71540d1b40 0x7f71540b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7159200640 1 --2- 172.21.15.62:0/2961179806 >> v2:172.21.15.121:3300/0 conn(0x7f71540b6640 0x7f715406fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7153400640 1 -- 172.21.15.62:0/2961179806 >> v2:172.21.15.121:3300/0 conn(0x7f71540b6640 msgr2=0x7f715406fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7153400640 1 --2- 172.21.15.62:0/2961179806 >> v2:172.21.15.121:3300/0 conn(0x7f71540b6640 0x7f715406fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7153400640 1 -- 172.21.15.62:0/2961179806 >> v1:172.21.15.62:6789/0 conn(0x7f715415b7b0 legacy=0x7f71540d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7152a00640 1 -- 172.21.15.62:0/2961179806 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270015584 0 0) 0x7f715415bb90 con 0x7f715415b7b0 2024-04-25T09:14:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.305+0000 7f7153400640 1 -- 172.21.15.62:0/2961179806 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715415e810 con 0x7f71540d1b40 2024-04-25T09:14:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.306+0000 7f7153400640 1 --2- 172.21.15.62:0/2961179806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71540d1b40 0x7f71540b6100 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f71540b1320 tx=0x7f714402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98976a481729f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7152a00640 1 -- 172.21.15.62:0/2961179806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144037070 con 0x7f71540d1b40 2024-04-25T09:14:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7152a00640 1 -- 172.21.15.62:0/2961179806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7144032440 con 0x7f71540d1b40 2024-04-25T09:14:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7152a00640 1 -- 172.21.15.62:0/2961179806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144032810 con 0x7f71540d1b40 2024-04-25T09:14:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7159c00640 1 -- 172.21.15.62:0/2961179806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71540d1b40 msgr2=0x7f71540b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7159c00640 1 --2- 172.21.15.62:0/2961179806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71540d1b40 0x7f71540b6100 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f71540b1320 tx=0x7f714402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7159c00640 1 -- 172.21.15.62:0/2961179806 shutdown_connections 2024-04-25T09:14:13.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7159c00640 1 --2- 172.21.15.62:0/2961179806 >> v2:172.21.15.121:3300/0 conn(0x7f71540b6640 0x7f715406fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7159c00640 1 --2- 172.21.15.62:0/2961179806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71540d1b40 0x7f71540b6100 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7159c00640 1 -- 172.21.15.62:0/2961179806 >> 172.21.15.62:0/2961179806 conn(0x7f7154072340 msgr2=0x7f7154072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:13.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7159c00640 1 -- 172.21.15.62:0/2961179806 shutdown_connections 2024-04-25T09:14:13.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.307+0000 7f7159c00640 1 -- 172.21.15.62:0/2961179806 wait complete. 2024-04-25T09:14:13.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.309+0000 7f7159c00640 1 Processor -- start 2024-04-25T09:14:13.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.309+0000 7f7159c00640 1 -- start start 2024-04-25T09:14:13.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.310+0000 7f7159c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71540b6640 0x7f715414b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.310+0000 7f7159c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71540d1b40 0x7f71541f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.310+0000 7f7159c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715415b7b0 0x7f715415cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.310+0000 7f7159c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71540d2080 con 0x7f715415b7b0 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.310+0000 7f7159c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71540b6ad0 con 0x7f71540b6640 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.310+0000 7f7159c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715415f010 con 0x7f71540d1b40 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7159200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715415b7b0 0x7f715415cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7159200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715415b7b0 0x7f715415cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36350/0 (socket says 172.21.15.62:36350) 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7159200640 1 -- 172.21.15.62:0/1887306005 learned_addr learned my addr 172.21.15.62:0/1887306005 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7153400640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71540d1b40 0x7f71541f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7159200640 1 -- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71540d1b40 msgr2=0x7f71541f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7153e00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71540b6640 0x7f715414b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7159200640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71540d1b40 0x7f71541f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7159200640 1 -- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71540b6640 msgr2=0x7f715414b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7159200640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71540b6640 0x7f715414b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7159200640 1 -- 172.21.15.62:0/1887306005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71541f3530 con 0x7f715415b7b0 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.311+0000 7f7153e00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71540b6640 0x7f715414b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.312+0000 7f7159200640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715415b7b0 0x7f715415cbe0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f714c007e40 tx=0x7f714c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.312+0000 7f714be00640 1 -- 172.21.15.62:0/1887306005 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c014620 con 0x7f715415b7b0 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.312+0000 7f714be00640 1 -- 172.21.15.62:0/1887306005 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c019070 con 0x7f715415b7b0 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.312+0000 7f714be00640 1 -- 172.21.15.62:0/1887306005 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c014a20 con 0x7f715415b7b0 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.312+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7154153f80 con 0x7f715415b7b0 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.312+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7154154820 con 0x7f715415b7b0 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.313+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715404ef20 con 0x7f715415b7b0 2024-04-25T09:14:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.314+0000 7f714be00640 1 -- 172.21.15.62:0/1887306005 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f714c014bc0 con 0x7f715415b7b0 2024-04-25T09:14:13.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.315+0000 7f714be00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f713006b520 0x7f713006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.315+0000 7f714be00640 1 -- 172.21.15.62:0/1887306005 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f714c08a980 con 0x7f715415b7b0 2024-04-25T09:14:13.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.316+0000 7f7153e00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f713006b520 0x7f713006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.317+0000 7f7153e00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f713006b520 0x7f713006d9e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f713c000f60 tx=0x7f713c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:13.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.320+0000 7f714be00640 1 -- 172.21.15.62:0/1887306005 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f714c059930 con 0x7f715415b7b0 2024-04-25T09:14:13.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.558+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f715414d8f0 con 0x7f715415b7b0 2024-04-25T09:14:13.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.559+0000 7f714be00640 1 -- 172.21.15.62:0/1887306005 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f714c059230 con 0x7f715415b7b0 2024-04-25T09:14:13.545 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:13.546 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:13.546 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:13.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f713006b520 msgr2=0x7f713006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f713006b520 0x7f713006d9e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f713c000f60 tx=0x7f713c009040 comp rx=0 tx=0).stop 2024-04-25T09:14:13.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715415b7b0 msgr2=0x7f715415cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715415b7b0 0x7f715415cbe0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f714c007e40 tx=0x7f714c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 shutdown_connections 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f713006b520 0x7f713006d9e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f715415b7b0 0x7f715415cbe0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f71540d1b40 0x7f71541f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 --2- 172.21.15.62:0/1887306005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71540b6640 0x7f715414b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 >> 172.21.15.62:0/1887306005 conn(0x7f7154072340 msgr2=0x7f715406c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 shutdown_connections 2024-04-25T09:14:13.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.562+0000 7f7159c00640 1 -- 172.21.15.62:0/1887306005 wait complete. 2024-04-25T09:14:13.568 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:14:13.568 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:14:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.789+0000 7f7e76200640 1 Processor -- start 2024-04-25T09:14:13.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.790+0000 7f7e76200640 1 -- start start 2024-04-25T09:14:13.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.791+0000 7f7e76200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e7015dd70 0x7f7e700d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.791+0000 7f7e76200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700b6560 con 0x7f7e700d2e60 2024-04-25T09:14:13.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.791+0000 7f7e76200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700b57a0 con 0x7f7e700cf0d0 2024-04-25T09:14:13.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.791+0000 7f7e76200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700b5a80 con 0x7f7e7015dd70 2024-04-25T09:14:13.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.791+0000 7f7e6fe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7e700d2e60 0x7f7e700ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:47584/0 (socket says 172.21.15.62:47584) 2024-04-25T09:14:13.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.791+0000 7f7e6fe00640 1 -- 172.21.15.62:0/2533642734 learned_addr learned my addr 172.21.15.62:0/2533642734 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:13.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e74e00640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e7015dd70 0x7f7e700d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055623028 0 0) 0x7f7e700b6560 con 0x7f7e700d2e60 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e58003520 con 0x7f7e700d2e60 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401444918 0 0) 0x7f7e58003520 con 0x7f7e700d2e60 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e7015dd70 msgr2=0x7f7e700d33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e7015dd70 0x7f7e700d33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 >> v1:172.21.15.121:6789/0 conn(0x7f7e700cf0d0 legacy=0x7f7e70161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e700b5d60 con 0x7f7e700d2e60 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f7e5c001670 con 0x7f7e700d2e60 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.792+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 >> v1:172.21.15.62:6789/0 conn(0x7f7e700d2e60 legacy=0x7f7e700ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e6f400640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580039e0 0x7f7e58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e6f400640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e7015dd70 0x7f7e58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e6f400640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e58004d40 0x7f7e580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700b5a80 con 0x7f7e58004d40 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5800baf0 con 0x7f7e580039e0 2024-04-25T09:14:13.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5800bcf0 con 0x7f7e7015dd70 2024-04-25T09:14:13.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e75800640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e58004d40 0x7f7e580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e6fe00640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e7015dd70 0x7f7e58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e74e00640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580039e0 0x7f7e58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e75800640 1 -- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e7015dd70 msgr2=0x7f7e58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e75800640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e7015dd70 0x7f7e58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e75800640 1 -- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580039e0 msgr2=0x7f7e58003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.793+0000 7f7e75800640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e580039e0 0x7f7e58003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.794+0000 7f7e75800640 1 -- 172.21.15.62:0/2533642734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e58003520 con 0x7f7e58004d40 2024-04-25T09:14:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e75800640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e58004d40 0x7f7e580051c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7e68001e50 tx=0x7f7e58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1e179c92cbab05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68001a70 con 0x7f7e58004d40 2024-04-25T09:14:13.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68003970 con 0x7f7e58004d40 2024-04-25T09:14:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e6f400640 1 -- 172.21.15.62:0/2533642734 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68003d60 con 0x7f7e58004d40 2024-04-25T09:14:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e76200640 1 -- 172.21.15.62:0/2533642734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e58004d40 msgr2=0x7f7e580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e76200640 1 --2- 172.21.15.62:0/2533642734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e58004d40 0x7f7e580051c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f7e68001e50 tx=0x7f7e58005830 comp rx=0 tx=0).stop 2024-04-25T09:14:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e75800640 1 -- 172.21.15.62:0/2533642734 reap_dead start 2024-04-25T09:14:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e76200640 1 -- 172.21.15.62:0/2533642734 shutdown_connections 2024-04-25T09:14:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.795+0000 7f7e76200640 1 -- 172.21.15.62:0/2533642734 >> 172.21.15.62:0/2533642734 conn(0x7f7e700668f0 msgr2=0x7f7e70066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.796+0000 7f7e76200640 1 -- 172.21.15.62:0/2533642734 shutdown_connections 2024-04-25T09:14:13.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.796+0000 7f7e76200640 1 -- 172.21.15.62:0/2533642734 wait complete. 2024-04-25T09:14:13.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.797+0000 7f7e76200640 1 Processor -- start 2024-04-25T09:14:13.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.798+0000 7f7e76200640 1 -- start start 2024-04-25T09:14:13.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e76200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e58004d40 0x7f7e700cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e76200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e70152520 0x7f7e70160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e76200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e70160fa0 0x7f7e70161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e76200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7015e150 con 0x7f7e70160fa0 2024-04-25T09:14:13.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e76200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700b6eb0 con 0x7f7e70152520 2024-04-25T09:14:13.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e76200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7015e2d0 con 0x7f7e58004d40 2024-04-25T09:14:13.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e75800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e70160fa0 0x7f7e70161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e75800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e70160fa0 0x7f7e70161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36374/0 (socket says 172.21.15.62:36374) 2024-04-25T09:14:13.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e75800640 1 -- 172.21.15.62:0/2232344311 learned_addr learned my addr 172.21.15.62:0/2232344311 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:13.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e6fe00640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e70152520 0x7f7e70160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.799+0000 7f7e74e00640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e58004d40 0x7f7e700cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e75800640 1 -- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e58004d40 msgr2=0x7f7e700cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e75800640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e58004d40 0x7f7e700cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e75800640 1 -- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e70152520 msgr2=0x7f7e70160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e75800640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e70152520 0x7f7e70160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e75800640 1 -- 172.21.15.62:0/2232344311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e700cd5a0 con 0x7f7e70160fa0 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e75800640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e70160fa0 0x7f7e70161690 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7e6802a920 tx=0x7f7e680014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e6cc00640 1 -- 172.21.15.62:0/2232344311 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68007db0 con 0x7f7e70160fa0 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e6cc00640 1 -- 172.21.15.62:0/2232344311 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68041070 con 0x7f7e70160fa0 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e6cc00640 1 -- 172.21.15.62:0/2232344311 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680050d0 con 0x7f7e70160fa0 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e700ccd00 con 0x7f7e70160fa0 2024-04-25T09:14:13.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.800+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e700cd2c0 con 0x7f7e70160fa0 2024-04-25T09:14:13.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.802+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e70072a20 con 0x7f7e70160fa0 2024-04-25T09:14:13.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.803+0000 7f7e6cc00640 1 -- 172.21.15.62:0/2232344311 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e6803a070 con 0x7f7e70160fa0 2024-04-25T09:14:13.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.803+0000 7f7e6cc00640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e5006b500 0x7f7e5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:13.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.804+0000 7f7e6cc00640 1 -- 172.21.15.62:0/2232344311 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7e680a9a70 con 0x7f7e70160fa0 2024-04-25T09:14:13.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.804+0000 7f7e74e00640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e5006b500 0x7f7e5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:13.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.805+0000 7f7e74e00640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e5006b500 0x7f7e5006d9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e70155f90 tx=0x7f7e64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:13.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:13.808+0000 7f7e6cc00640 1 -- 172.21.15.62:0/2232344311 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e68078820 con 0x7f7e70160fa0 2024-04-25T09:14:14.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.075+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7e700d1a00 con 0x7f7e70160fa0 2024-04-25T09:14:14.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.076+0000 7f7e6cc00640 1 -- 172.21.15.62:0/2232344311 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1352 (secure 0 0 0) 0x7f7e68078120 con 0x7f7e70160fa0 2024-04-25T09:14:14.062 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:14.062 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":681,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076577212,"bytes_used":2596569088,"bytes_avail":770497544192,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":76,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":768,"write_bytes_sec":32360997,"read_op_per_sec":0,"write_op_per_sec":33},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7800}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T09:13:20.117690+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e5006b500 msgr2=0x7f7e5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e5006b500 0x7f7e5006d9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e70155f90 tx=0x7f7e64008040 comp rx=0 tx=0).stop 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e70160fa0 msgr2=0x7f7e70161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e70160fa0 0x7f7e70161690 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f7e6802a920 tx=0x7f7e680014a0 comp rx=0 tx=0).stop 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 shutdown_connections 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7e5006b500 0x7f7e5006d9c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e70160fa0 0x7f7e70161690 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e70152520 0x7f7e70160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:14.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 --2- 172.21.15.62:0/2232344311 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7e58004d40 0x7f7e700cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.078+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 >> 172.21.15.62:0/2232344311 conn(0x7f7e700668f0 msgr2=0x7f7e700b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.079+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 shutdown_connections 2024-04-25T09:14:14.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:14.079+0000 7f7e76200640 1 -- 172.21.15.62:0/2232344311 wait complete. 2024-04-25T09:14:14.083 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:14:14.725 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:14:14.733 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:14:15.084 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:15.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.320+0000 7f3d73e00640 1 Processor -- start 2024-04-25T09:14:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.320+0000 7f3d73e00640 1 -- start start 2024-04-25T09:14:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d73e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3d74144c10 0x7f3d7415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 0x7f3d7406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d73e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3d7406d730 0x7f3d740b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d73e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d74144ff0 con 0x7f3d74144c10 2024-04-25T09:14:15.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d73e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d74160340 con 0x7f3d7406d730 2024-04-25T09:14:15.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d73e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740b41c0 con 0x7f3d740cee40 2024-04-25T09:14:15.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d72a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3d74144c10 0x7f3d7415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d72a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3d74144c10 0x7f3d7415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56500/0 (socket says 172.21.15.62:56500) 2024-04-25T09:14:15.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.322+0000 7f3d72a00640 1 -- 172.21.15.62:0/353808766 learned_addr learned my addr 172.21.15.62:0/353808766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:15.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.323+0000 7f3d72000640 1 --2- 172.21.15.62:0/353808766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 0x7f3d7406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.323+0000 7f3d72a00640 1 -- 172.21.15.62:0/353808766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 msgr2=0x7f3d7406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:15.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.323+0000 7f3d72a00640 1 --2- 172.21.15.62:0/353808766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 0x7f3d7406d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.323+0000 7f3d72a00640 1 -- 172.21.15.62:0/353808766 >> v2:172.21.15.121:3300/0 conn(0x7f3d7406d730 msgr2=0x7f3d740b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:15.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.323+0000 7f3d73400640 1 --2- 172.21.15.62:0/353808766 >> v2:172.21.15.121:3300/0 conn(0x7f3d7406d730 0x7f3d740b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.323+0000 7f3d72a00640 1 --2- 172.21.15.62:0/353808766 >> v2:172.21.15.121:3300/0 conn(0x7f3d7406d730 0x7f3d740b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.323+0000 7f3d72a00640 1 -- 172.21.15.62:0/353808766 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7415faa0 con 0x7f3d74144c10 2024-04-25T09:14:15.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.324+0000 7f3d72a00640 1 --2- 172.21.15.62:0/353808766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d7415f450 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3d740b4130 tx=0x7f3d6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9bf3e3e6e2ef6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:15.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.324+0000 7f3d71600640 1 -- 172.21.15.62:0/353808766 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64037070 con 0x7f3d74144c10 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.324+0000 7f3d71600640 1 -- 172.21.15.62:0/353808766 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d64032440 con 0x7f3d74144c10 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.324+0000 7f3d71600640 1 -- 172.21.15.62:0/353808766 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64032810 con 0x7f3d74144c10 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 -- 172.21.15.62:0/353808766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 msgr2=0x7f3d7415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 --2- 172.21.15.62:0/353808766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d7415f450 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f3d740b4130 tx=0x7f3d6402a140 comp rx=0 tx=0).stop 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 -- 172.21.15.62:0/353808766 shutdown_connections 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 --2- 172.21.15.62:0/353808766 >> v2:172.21.15.121:3300/0 conn(0x7f3d7406d730 0x7f3d740b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 --2- 172.21.15.62:0/353808766 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 0x7f3d7406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 --2- 172.21.15.62:0/353808766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d7415f450 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 -- 172.21.15.62:0/353808766 >> 172.21.15.62:0/353808766 conn(0x7f3d740b14d0 msgr2=0x7f3d740b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 -- 172.21.15.62:0/353808766 shutdown_connections 2024-04-25T09:14:15.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.325+0000 7f3d73e00640 1 -- 172.21.15.62:0/353808766 wait complete. 2024-04-25T09:14:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.327+0000 7f3d73e00640 1 Processor -- start 2024-04-25T09:14:15.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.327+0000 7f3d73e00640 1 -- start start 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d7406d730 0x7f3d7413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 0x7f3d740c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d740ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740cf2a0 con 0x7f3d74144c10 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74160060 con 0x7f3d7406d730 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740b32e0 con 0x7f3d740cee40 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d740ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d740ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56512/0 (socket says 172.21.15.62:56512) 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.328+0000 7f3d73400640 1 -- 172.21.15.62:0/204494539 learned_addr learned my addr 172.21.15.62:0/204494539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d72a00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d7406d730 0x7f3d7413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d72000640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 0x7f3d740c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d73400640 1 -- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 msgr2=0x7f3d740c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d73400640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 0x7f3d740c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d73400640 1 -- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d7406d730 msgr2=0x7f3d7413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d73400640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d7406d730 0x7f3d7413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d73400640 1 -- 172.21.15.62:0/204494539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d741f4810 con 0x7f3d74144c10 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d72a00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d7406d730 0x7f3d7413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d73400640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d740ca430 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00cd20 tx=0x7f3d6c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.329+0000 7f3d6aa00640 1 -- 172.21.15.62:0/204494539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c003b90 con 0x7f3d74144c10 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.330+0000 7f3d6aa00640 1 -- 172.21.15.62:0/204494539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c003d30 con 0x7f3d74144c10 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.330+0000 7f3d6aa00640 1 -- 172.21.15.62:0/204494539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c004240 con 0x7f3d74144c10 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.330+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d74140b80 con 0x7f3d74144c10 2024-04-25T09:14:15.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.330+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d74141140 con 0x7f3d74144c10 2024-04-25T09:14:15.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.331+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7404ef20 con 0x7f3d74144c10 2024-04-25T09:14:15.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.332+0000 7f3d6aa00640 1 -- 172.21.15.62:0/204494539 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d6c0043e0 con 0x7f3d74144c10 2024-04-25T09:14:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.334+0000 7f3d6aa00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d5406b4d0 0x7f3d5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.334+0000 7f3d6aa00640 1 -- 172.21.15.62:0/204494539 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3d6c08ac40 con 0x7f3d74144c10 2024-04-25T09:14:15.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.334+0000 7f3d72a00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d5406b4d0 0x7f3d5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.336+0000 7f3d72a00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d5406b4d0 0x7f3d5406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3d64034000 tx=0x7f3d6402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:15.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.338+0000 7f3d6aa00640 1 -- 172.21.15.62:0/204494539 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d6c059b10 con 0x7f3d74144c10 2024-04-25T09:14:15.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.580+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d7415ea10 con 0x7f3d74144c10 2024-04-25T09:14:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.581+0000 7f3d6aa00640 1 -- 172.21.15.62:0/204494539 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3d6c059410 con 0x7f3d74144c10 2024-04-25T09:14:15.568 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:15.568 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d5406b4d0 msgr2=0x7f3d5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d5406b4d0 0x7f3d5406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3d64034000 tx=0x7f3d6402e000 comp rx=0 tx=0).stop 2024-04-25T09:14:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 msgr2=0x7f3d740ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d740ca430 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00cd20 tx=0x7f3d6c00a660 comp rx=0 tx=0).stop 2024-04-25T09:14:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 shutdown_connections 2024-04-25T09:14:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3d5406b4d0 0x7f3d5406d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d74144c10 0x7f3d740ca430 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3d740cee40 0x7f3d740c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 --2- 172.21.15.62:0/204494539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d7406d730 0x7f3d7413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 >> 172.21.15.62:0/204494539 conn(0x7f3d740b14d0 msgr2=0x7f3d740ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:15.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 shutdown_connections 2024-04-25T09:14:15.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.584+0000 7f3d73e00640 1 -- 172.21.15.62:0/204494539 wait complete. 2024-04-25T09:14:15.589 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:14:15.589 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:14:15.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.799+0000 7f832b200640 1 Processor -- start 2024-04-25T09:14:15.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.799+0000 7f832b200640 1 -- start start 2024-04-25T09:14:15.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.801+0000 7f832b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 0x7f832c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.801+0000 7f832b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f832c0b9750 0x7f832c0d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.801+0000 7f832b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c0d2690 con 0x7f832c15db60 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.801+0000 7f832b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0b7c20 con 0x7f832c0b9750 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.801+0000 7f832b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0d2970 con 0x7f832c15df40 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.801+0000 7f832a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f832c0b9750 0x7f832c0d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.801+0000 7f832a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f832c0b9750 0x7f832c0d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56520/0 (socket says 172.21.15.62:56520) 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.801+0000 7f832a800640 1 -- 172.21.15.62:0/3802281308 learned_addr learned my addr 172.21.15.62:0/3802281308 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.802+0000 7f832a800640 1 -- 172.21.15.62:0/3802281308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 msgr2=0x7f832c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:15.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.802+0000 7f832a800640 1 --2- 172.21.15.62:0/3802281308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 0x7f832c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.802+0000 7f832a800640 1 -- 172.21.15.62:0/3802281308 >> v1:172.21.15.121:6789/0 conn(0x7f832c15db60 legacy=0x7f832c0d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:15.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.802+0000 7f832a800640 1 -- 172.21.15.62:0/3802281308 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0d2c50 con 0x7f832c0b9750 2024-04-25T09:14:15.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.803+0000 7f832a800640 1 --2- 172.21.15.62:0/3802281308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c0d0830 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f8324009870 tx=0x7f832402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5544c77dd7f8827c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.803+0000 7f8328a00640 1 -- 172.21.15.62:0/3802281308 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324037070 con 0x7f832c0b9750 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.803+0000 7f8328a00640 1 -- 172.21.15.62:0/3802281308 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8324032440 con 0x7f832c0b9750 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.803+0000 7f8328a00640 1 -- 172.21.15.62:0/3802281308 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324032810 con 0x7f832c0b9750 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.804+0000 7f832b200640 1 -- 172.21.15.62:0/3802281308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 msgr2=0x7f832c0d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.804+0000 7f832b200640 1 --2- 172.21.15.62:0/3802281308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c0d0830 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f8324009870 tx=0x7f832402a180 comp rx=0 tx=0).stop 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.804+0000 7f832b200640 1 -- 172.21.15.62:0/3802281308 shutdown_connections 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.804+0000 7f832b200640 1 --2- 172.21.15.62:0/3802281308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c0d0830 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.804+0000 7f832b200640 1 --2- 172.21.15.62:0/3802281308 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 0x7f832c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.804+0000 7f832b200640 1 -- 172.21.15.62:0/3802281308 >> 172.21.15.62:0/3802281308 conn(0x7f832c06b210 msgr2=0x7f832c0668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.804+0000 7f832b200640 1 -- 172.21.15.62:0/3802281308 shutdown_connections 2024-04-25T09:14:15.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.804+0000 7f832b200640 1 -- 172.21.15.62:0/3802281308 wait complete. 2024-04-25T09:14:15.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.806+0000 7f832b200640 1 Processor -- start 2024-04-25T09:14:15.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.806+0000 7f832b200640 1 -- start start 2024-04-25T09:14:15.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f832b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c1e3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f832b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832c15db60 0x7f832c14bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f832b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 0x7f832c158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f832b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0d3890 con 0x7f832c0b9750 2024-04-25T09:14:15.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f832b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0d3710 con 0x7f832c15db60 2024-04-25T09:14:15.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f832b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b6c10 con 0x7f832c15df40 2024-04-25T09:14:15.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c1e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c1e3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56530/0 (socket says 172.21.15.62:56530) 2024-04-25T09:14:15.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329e00640 1 -- 172.21.15.62:0/1161093745 learned_addr learned my addr 172.21.15.62:0/1161093745 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:15.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f832a800640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 0x7f832c158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329e00640 1 -- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 msgr2=0x7f832c158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:15.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329400640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832c15db60 0x7f832c14bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329e00640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 0x7f832c158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329e00640 1 -- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832c15db60 msgr2=0x7f832c14bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329e00640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832c15db60 0x7f832c14bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.807+0000 7f8329e00640 1 -- 172.21.15.62:0/1161093745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c159780 con 0x7f832c0b9750 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.808+0000 7f8329400640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832c15db60 0x7f832c14bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.808+0000 7f8329e00640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c1e3060 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f831c00c7c0 tx=0x7f831c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.808+0000 7f8322000640 1 -- 172.21.15.62:0/1161093745 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c016020 con 0x7f832c0b9750 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.808+0000 7f8322000640 1 -- 172.21.15.62:0/1161093745 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00aa50 con 0x7f832c0b9750 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.808+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c1f7190 con 0x7f832c0b9750 2024-04-25T09:14:15.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.808+0000 7f8322000640 1 -- 172.21.15.62:0/1161093745 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c012420 con 0x7f832c0b9750 2024-04-25T09:14:15.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.808+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c1f7750 con 0x7f832c0b9750 2024-04-25T09:14:15.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.809+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c072910 con 0x7f832c0b9750 2024-04-25T09:14:15.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.810+0000 7f8322000640 1 -- 172.21.15.62:0/1161093745 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f831c019020 con 0x7f832c0b9750 2024-04-25T09:14:15.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.812+0000 7f8322000640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f830c06b4d0 0x7f830c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:15.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.812+0000 7f8322000640 1 -- 172.21.15.62:0/1161093745 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f831c089fb0 con 0x7f832c0b9750 2024-04-25T09:14:15.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.812+0000 7f8329400640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f830c06b4d0 0x7f830c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:15.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.813+0000 7f8329400640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f830c06b4d0 0x7f830c06d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f832c159cf0 tx=0x7f8318008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:15.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:15.816+0000 7f8322000640 1 -- 172.21.15.62:0/1161093745 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f831c020070 con 0x7f832c0b9750 2024-04-25T09:14:16.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.077+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f832c0ccff0 con 0x7f832c0b9750 2024-04-25T09:14:16.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.077+0000 7f8322000640 1 -- 172.21.15.62:0/1161093745 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1352 (secure 0 0 0) 0x7f831c058de0 con 0x7f832c0b9750 2024-04-25T09:14:16.065 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:16.065 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":683,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076577212,"bytes_used":2596569088,"bytes_avail":770497544192,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":68,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":768,"write_bytes_sec":28766037,"read_op_per_sec":0,"write_op_per_sec":29},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7800}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T09:13:20.117690+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f830c06b4d0 msgr2=0x7f830c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f830c06b4d0 0x7f830c06d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f832c159cf0 tx=0x7f8318008040 comp rx=0 tx=0).stop 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 msgr2=0x7f832c1e3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c1e3060 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f831c00c7c0 tx=0x7f831c00cc80 comp rx=0 tx=0).stop 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 shutdown_connections 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f830c06b4d0 0x7f830c06d990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f832c15df40 0x7f832c158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832c15db60 0x7f832c14bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 --2- 172.21.15.62:0/1161093745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f832c0b9750 0x7f832c1e3060 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 >> 172.21.15.62:0/1161093745 conn(0x7f832c06b210 msgr2=0x7f832c0b7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 shutdown_connections 2024-04-25T09:14:16.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:16.080+0000 7f832b200640 1 -- 172.21.15.62:0/1161093745 wait complete. 2024-04-25T09:14:16.085 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:14:17.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:17.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.319+0000 7fb3e1000640 1 Processor -- start 2024-04-25T09:14:17.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.320+0000 7fb3e1000640 1 -- start start 2024-04-25T09:14:17.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.321+0000 7fb3e1000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3dc15b9f0 0x7fb3dc0d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.321+0000 7fb3e1000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3dc15f2c0 con 0x7fb3dc0b9470 2024-04-25T09:14:17.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.321+0000 7fb3e1000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3dc15e6b0 con 0x7fb3dc0d11d0 2024-04-25T09:14:17.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.321+0000 7fb3e1000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3dc0b3420 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.322+0000 7fb3d9e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3dc15b9f0 0x7fb3dc0d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.322+0000 7fb3d9e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3dc15b9f0 0x7fb3dc0d1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56534/0 (socket says 172.21.15.62:56534) 2024-04-25T09:14:17.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.322+0000 7fb3d9e00640 1 -- 172.21.15.62:0/1126073922 learned_addr learned my addr 172.21.15.62:0/1126073922 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:17.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.322+0000 7fb3d9e00640 1 -- 172.21.15.62:0/1126073922 >> v1:172.21.15.121:6789/0 conn(0x7fb3dc0d11d0 legacy=0x7fb3dc161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:17.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.322+0000 7fb3d9e00640 1 -- 172.21.15.62:0/1126073922 >> v1:172.21.15.62:6789/0 conn(0x7fb3dc0b9470 legacy=0x7fb3dc0d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.322+0000 7fb3d9e00640 1 -- 172.21.15.62:0/1126073922 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc15e990 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.323+0000 7fb3d9e00640 1 --2- 172.21.15.62:0/1126073922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc0d1f20 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009870 tx=0x7fb3cc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dc3da3d9378731 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:17.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.324+0000 7fb3d8a00640 1 -- 172.21.15.62:0/1126073922 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc037070 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.324+0000 7fb3d8a00640 1 -- 172.21.15.62:0/1126073922 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc032440 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.324+0000 7fb3d8a00640 1 -- 172.21.15.62:0/1126073922 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc032810 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.324+0000 7fb3e1000640 1 -- 172.21.15.62:0/1126073922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 msgr2=0x7fb3dc0d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.324+0000 7fb3e1000640 1 --2- 172.21.15.62:0/1126073922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc0d1f20 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009870 tx=0x7fb3cc02a2a0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.325+0000 7fb3e1000640 1 -- 172.21.15.62:0/1126073922 shutdown_connections 2024-04-25T09:14:17.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.325+0000 7fb3e1000640 1 --2- 172.21.15.62:0/1126073922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc0d1f20 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.325+0000 7fb3e1000640 1 -- 172.21.15.62:0/1126073922 >> 172.21.15.62:0/1126073922 conn(0x7fb3dc06b000 msgr2=0x7fb3dc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:17.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.325+0000 7fb3e1000640 1 -- 172.21.15.62:0/1126073922 shutdown_connections 2024-04-25T09:14:17.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.325+0000 7fb3e1000640 1 -- 172.21.15.62:0/1126073922 wait complete. 2024-04-25T09:14:17.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.326+0000 7fb3e1000640 1 Processor -- start 2024-04-25T09:14:17.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.327+0000 7fb3e1000640 1 -- start start 2024-04-25T09:14:17.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.327+0000 7fb3e1000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3dc0b9470 0x7fb3dc15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3e1000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3dc0d11d0 0x7fb3dc0b8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3e1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc14f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3e1000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc15e310 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3e1000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc15ef50 con 0x7fb3dc0d11d0 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3e1000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc15e490 con 0x7fb3dc0b9470 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3da800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc14f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3da800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc14f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56542/0 (socket says 172.21.15.62:56542) 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3da800640 1 -- 172.21.15.62:0/2345441912 learned_addr learned my addr 172.21.15.62:0/2345441912 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3d9e00640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3dc0b9470 0x7fb3dc15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3d9400640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3dc0d11d0 0x7fb3dc0b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3da800640 1 -- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3dc0b9470 msgr2=0x7fb3dc15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3da800640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3dc0b9470 0x7fb3dc15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3da800640 1 -- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3dc0d11d0 msgr2=0x7fb3dc0b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3da800640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3dc0d11d0 0x7fb3dc0b8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3da800640 1 -- 172.21.15.62:0/2345441912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc15a780 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3d9400640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3dc0d11d0 0x7fb3dc0b8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:17.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.328+0000 7fb3d9e00640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3dc0b9470 0x7fb3dc15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:17.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.329+0000 7fb3da800640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc14f930 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400c3e0 tx=0x7fb3d400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:17.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.329+0000 7fb3d2000640 1 -- 172.21.15.62:0/2345441912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4011790 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.329+0000 7fb3d2000640 1 -- 172.21.15.62:0/2345441912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4017070 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.329+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3dc15aa60 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.329+0000 7fb3d2000640 1 -- 172.21.15.62:0/2345441912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4011bb0 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.329+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3dc0c9410 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.330+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3dc0729c0 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.331+0000 7fb3d2000640 1 -- 172.21.15.62:0/2345441912 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3d401c460 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.332+0000 7fb3d2000640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3bc06b500 0x7fb3bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.332+0000 7fb3d2000640 1 -- 172.21.15.62:0/2345441912 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb3d408a0a0 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.333+0000 7fb3d9e00640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3bc06b500 0x7fb3bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.334+0000 7fb3d9e00640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3bc06b500 0x7fb3bc06d9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc034000 tx=0x7fb3cc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:17.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.337+0000 7fb3d2000640 1 -- 172.21.15.62:0/2345441912 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3d4058e40 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.577+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3dc04f000 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.578+0000 7fb3d2000640 1 -- 172.21.15.62:0/2345441912 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb3d4058740 con 0x7fb3dc15b9f0 2024-04-25T09:14:17.566 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:17.566 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:17.566 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3bc06b500 msgr2=0x7fb3bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3bc06b500 0x7fb3bc06d9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc034000 tx=0x7fb3cc02e000 comp rx=0 tx=0).stop 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 msgr2=0x7fb3dc14f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc14f930 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400c3e0 tx=0x7fb3d400c8a0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 shutdown_connections 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb3bc06b500 0x7fb3bc06d9c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3dc15b9f0 0x7fb3dc14f930 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3dc0d11d0 0x7fb3dc0b8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 --2- 172.21.15.62:0/2345441912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3dc0b9470 0x7fb3dc15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.581+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 >> 172.21.15.62:0/2345441912 conn(0x7fb3dc06b000 msgr2=0x7fb3dc0d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:17.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.582+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 shutdown_connections 2024-04-25T09:14:17.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.582+0000 7fb3e1000640 1 -- 172.21.15.62:0/2345441912 wait complete. 2024-04-25T09:14:17.587 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:14:17.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:14:17.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.811+0000 7f7c27e00640 1 Processor -- start 2024-04-25T09:14:17.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.811+0000 7f7c27e00640 1 -- start start 2024-04-25T09:14:17.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c27e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c28144000 0x7f7c2806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c27e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c2815ed50 0x7f7c280b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c27e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c280ada60 con 0x7f7c2806d800 2024-04-25T09:14:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c27e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2815f160 con 0x7f7c2815ed50 2024-04-25T09:14:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c27e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c280add20 con 0x7f7c28144000 2024-04-25T09:14:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c27400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c2815ed50 0x7f7c280b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c27400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7c2815ed50 0x7f7c280b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56550/0 (socket says 172.21.15.62:56550) 2024-04-25T09:14:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c27400640 1 -- 172.21.15.62:0/646727899 learned_addr learned my addr 172.21.15.62:0/646727899 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:17.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.813+0000 7f7c26000640 1 --1- 172.21.15.62:0/646727899 >> v1:172.21.15.62:6789/0 conn(0x7f7c2806d800 0x7f7c2815e640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:43384/0 (socket says 172.21.15.62:43384) 2024-04-25T09:14:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.814+0000 7f7c26a00640 1 --2- 172.21.15.62:0/646727899 >> v2:172.21.15.121:3300/0 conn(0x7f7c28144000 0x7f7c2806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.814+0000 7f7c27400640 1 -- 172.21.15.62:0/646727899 >> v2:172.21.15.121:3300/0 conn(0x7f7c28144000 msgr2=0x7f7c2806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.814+0000 7f7c27400640 1 --2- 172.21.15.62:0/646727899 >> v2:172.21.15.121:3300/0 conn(0x7f7c28144000 0x7f7c2806d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.814+0000 7f7c27400640 1 -- 172.21.15.62:0/646727899 >> v1:172.21.15.62:6789/0 conn(0x7f7c2806d800 legacy=0x7f7c2815e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.814+0000 7f7c27400640 1 -- 172.21.15.62:0/646727899 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c280ae000 con 0x7f7c2815ed50 2024-04-25T09:14:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.815+0000 7f7c27400640 1 --2- 172.21.15.62:0/646727899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c2815ed50 0x7f7c280b12b0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7c2815f2e0 tx=0x7f7c2002a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b0442ecfd91cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.815+0000 7f7c25600640 1 -- 172.21.15.62:0/646727899 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20037070 con 0x7f7c2815ed50 2024-04-25T09:14:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.815+0000 7f7c25600640 1 -- 172.21.15.62:0/646727899 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c20032440 con 0x7f7c2815ed50 2024-04-25T09:14:17.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.815+0000 7f7c25600640 1 -- 172.21.15.62:0/646727899 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20032810 con 0x7f7c2815ed50 2024-04-25T09:14:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.816+0000 7f7c27e00640 1 -- 172.21.15.62:0/646727899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c2815ed50 msgr2=0x7f7c280b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.816+0000 7f7c27e00640 1 --2- 172.21.15.62:0/646727899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c2815ed50 0x7f7c280b12b0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7c2815f2e0 tx=0x7f7c2002a280 comp rx=0 tx=0).stop 2024-04-25T09:14:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.816+0000 7f7c27e00640 1 -- 172.21.15.62:0/646727899 shutdown_connections 2024-04-25T09:14:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.816+0000 7f7c27e00640 1 --2- 172.21.15.62:0/646727899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c2815ed50 0x7f7c280b12b0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.816+0000 7f7c27e00640 1 --2- 172.21.15.62:0/646727899 >> v2:172.21.15.121:3300/0 conn(0x7f7c28144000 0x7f7c2806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.816+0000 7f7c27e00640 1 -- 172.21.15.62:0/646727899 >> 172.21.15.62:0/646727899 conn(0x7f7c280668f0 msgr2=0x7f7c28066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.816+0000 7f7c27e00640 1 -- 172.21.15.62:0/646727899 shutdown_connections 2024-04-25T09:14:17.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.816+0000 7f7c27e00640 1 -- 172.21.15.62:0/646727899 wait complete. 2024-04-25T09:14:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.818+0000 7f7c27e00640 1 Processor -- start 2024-04-25T09:14:17.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.818+0000 7f7c27e00640 1 -- start start 2024-04-25T09:14:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.819+0000 7f7c27e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c2806d800 0x7f7c2815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.819+0000 7f7c27e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c28144000 0x7f7c281e7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.819+0000 7f7c27e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c2815ed50 0x7f7c280709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.819+0000 7f7c27e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c281443e0 con 0x7f7c28144000 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.819+0000 7f7c27e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280ae800 con 0x7f7c2806d800 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.819+0000 7f7c27e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28144560 con 0x7f7c2815ed50 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c28144000 0x7f7c281e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c28144000 0x7f7c281e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56566/0 (socket says 172.21.15.62:56566) 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26000640 1 -- 172.21.15.62:0/4118739199 learned_addr learned my addr 172.21.15.62:0/4118739199 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26a00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c2806d800 0x7f7c2815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c27400640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c2815ed50 0x7f7c280709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26000640 1 -- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c2815ed50 msgr2=0x7f7c280709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26000640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c2815ed50 0x7f7c280709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26000640 1 -- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c2806d800 msgr2=0x7f7c2815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26000640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c2806d800 0x7f7c2815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.820+0000 7f7c26000640 1 -- 172.21.15.62:0/4118739199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c281e8220 con 0x7f7c28144000 2024-04-25T09:14:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.821+0000 7f7c26000640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c28144000 0x7f7c281e7760 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000c430 tx=0x7f7c1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:17.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.821+0000 7f7c1ea00640 1 -- 172.21.15.62:0/4118739199 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c100074e0 con 0x7f7c28144000 2024-04-25T09:14:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.821+0000 7f7c1ea00640 1 -- 172.21.15.62:0/4118739199 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c10007680 con 0x7f7c28144000 2024-04-25T09:14:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.821+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c281e8500 con 0x7f7c28144000 2024-04-25T09:14:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.821+0000 7f7c1ea00640 1 -- 172.21.15.62:0/4118739199 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10004a00 con 0x7f7c28144000 2024-04-25T09:14:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.821+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c28147770 con 0x7f7c28144000 2024-04-25T09:14:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.822+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c28072b10 con 0x7f7c28144000 2024-04-25T09:14:17.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.823+0000 7f7c1ea00640 1 -- 172.21.15.62:0/4118739199 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c1001a020 con 0x7f7c28144000 2024-04-25T09:14:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.824+0000 7f7c1ea00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7c0406b4d0 0x7f7c0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:17.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.824+0000 7f7c1ea00640 1 -- 172.21.15.62:0/4118739199 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7c10089af0 con 0x7f7c28144000 2024-04-25T09:14:17.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.825+0000 7f7c26a00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7c0406b4d0 0x7f7c0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:17.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.826+0000 7f7c26a00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7c0406b4d0 0x7f7c0406d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7c280afbe0 tx=0x7f7c180039b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:17.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:17.829+0000 7f7c1ea00640 1 -- 172.21.15.62:0/4118739199 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c10058890 con 0x7f7c28144000 2024-04-25T09:14:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.106+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7c280aeed0 con 0x7f7c28144000 2024-04-25T09:14:18.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.107+0000 7f7c1ea00640 1 -- 172.21.15.62:0/4118739199 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1762 (secure 0 0 0) 0x7f7c10058190 con 0x7f7c28144000 2024-04-25T09:14:18.094 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:18.094 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":685,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076638708,"bytes_used":2597257216,"bytes_avail":770496856064,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":63,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":768,"write_bytes_sec":26972238,"read_op_per_sec":0,"write_op_per_sec":28},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7800}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T09:14:16.138918+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T09:14:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7c0406b4d0 msgr2=0x7f7c0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7c0406b4d0 0x7f7c0406d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7c280afbe0 tx=0x7f7c180039b0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c28144000 msgr2=0x7f7c281e7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c28144000 0x7f7c281e7760 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000c430 tx=0x7f7c1000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 shutdown_connections 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7c0406b4d0 0x7f7c0406d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c28144000 0x7f7c281e7760 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7c2815ed50 0x7f7c280709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 --2- 172.21.15.62:0/4118739199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c2806d800 0x7f7c2815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 >> 172.21.15.62:0/4118739199 conn(0x7f7c280668f0 msgr2=0x7f7c280b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 shutdown_connections 2024-04-25T09:14:18.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.110+0000 7f7c27e00640 1 -- 172.21.15.62:0/4118739199 wait complete. 2024-04-25T09:14:18.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T09:14:18.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:18.115 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T09:14:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.335+0000 7fe298a00640 1 Processor -- start 2024-04-25T09:14:18.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.335+0000 7fe298a00640 1 -- start start 2024-04-25T09:14:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.337+0000 7fe298a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2940d2fb0 0x7fe2940ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.337+0000 7fe298a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe2940d3960 0x7fe2940d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.337+0000 7fe298a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2940ce780 con 0x7fe29415dd50 2024-04-25T09:14:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.337+0000 7fe298a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2940d42f0 con 0x7fe2940d2fb0 2024-04-25T09:14:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.337+0000 7fe298a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2940d3440 con 0x7fe2940d3960 2024-04-25T09:14:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.337+0000 7fe289e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2940d2fb0 0x7fe2940ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.337+0000 7fe289e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2940d2fb0 0x7fe2940ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56572/0 (socket says 172.21.15.62:56572) 2024-04-25T09:14:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.337+0000 7fe289e00640 1 -- 172.21.15.62:0/1240155685 learned_addr learned my addr 172.21.15.62:0/1240155685 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.338+0000 7fe293200640 1 --2- 172.21.15.62:0/1240155685 >> v2:172.21.15.121:3300/0 conn(0x7fe2940d3960 0x7fe2940d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.338+0000 7fe289e00640 1 -- 172.21.15.62:0/1240155685 >> v2:172.21.15.121:3300/0 conn(0x7fe2940d3960 msgr2=0x7fe2940d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.338+0000 7fe289e00640 1 --2- 172.21.15.62:0/1240155685 >> v2:172.21.15.121:3300/0 conn(0x7fe2940d3960 0x7fe2940d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.338+0000 7fe289e00640 1 -- 172.21.15.62:0/1240155685 >> v1:172.21.15.121:6789/0 conn(0x7fe29415dd50 legacy=0x7fe2940d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.338+0000 7fe289e00640 1 -- 172.21.15.62:0/1240155685 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2940ce1c0 con 0x7fe2940d2fb0 2024-04-25T09:14:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.339+0000 7fe289e00640 1 --2- 172.21.15.62:0/1240155685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2940d2fb0 0x7fe2940ce9c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fe2940caa50 tx=0x7fe27c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b87b222657aa4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe291e00640 1 -- 172.21.15.62:0/1240155685 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c037070 con 0x7fe2940d2fb0 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe291e00640 1 -- 172.21.15.62:0/1240155685 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c032440 con 0x7fe2940d2fb0 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe291e00640 1 -- 172.21.15.62:0/1240155685 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c032810 con 0x7fe2940d2fb0 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe298a00640 1 -- 172.21.15.62:0/1240155685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2940d2fb0 msgr2=0x7fe2940ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe298a00640 1 --2- 172.21.15.62:0/1240155685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2940d2fb0 0x7fe2940ce9c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fe2940caa50 tx=0x7fe27c02a230 comp rx=0 tx=0).stop 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe298a00640 1 -- 172.21.15.62:0/1240155685 shutdown_connections 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe298a00640 1 --2- 172.21.15.62:0/1240155685 >> v2:172.21.15.121:3300/0 conn(0x7fe2940d3960 0x7fe2940d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe298a00640 1 --2- 172.21.15.62:0/1240155685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2940d2fb0 0x7fe2940ce9c0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe298a00640 1 -- 172.21.15.62:0/1240155685 >> 172.21.15.62:0/1240155685 conn(0x7fe29406b000 msgr2=0x7fe29406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe298a00640 1 -- 172.21.15.62:0/1240155685 shutdown_connections 2024-04-25T09:14:18.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.340+0000 7fe298a00640 1 -- 172.21.15.62:0/1240155685 wait complete. 2024-04-25T09:14:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.342+0000 7fe298a00640 1 Processor -- start 2024-04-25T09:14:18.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.342+0000 7fe298a00640 1 -- start start 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe298a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2940d3960 0x7fe29415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe298a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe29415dd50 0x7fe29415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe298a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe294160db0 0x7fe2941f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe298a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2940b6b00 con 0x7fe294160db0 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe298a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2940cef00 con 0x7fe29415dd50 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe298a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2940b6c80 con 0x7fe2940d3960 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe293200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe294160db0 0x7fe2941f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe293200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe294160db0 0x7fe2941f6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56584/0 (socket says 172.21.15.62:56584) 2024-04-25T09:14:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.343+0000 7fe293200640 1 -- 172.21.15.62:0/816846769 learned_addr learned my addr 172.21.15.62:0/816846769 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe292800640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2940d3960 0x7fe29415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe289e00640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe29415dd50 0x7fe29415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe293200640 1 -- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2940d3960 msgr2=0x7fe29415ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe293200640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2940d3960 0x7fe29415ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe293200640 1 -- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe29415dd50 msgr2=0x7fe29415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe293200640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe29415dd50 0x7fe29415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe293200640 1 -- 172.21.15.62:0/816846769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29415fd10 con 0x7fe294160db0 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe293200640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe294160db0 0x7fe2941f6d30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00d830 tx=0x7fe28c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe28b400640 1 -- 172.21.15.62:0/816846769 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00b6c0 con 0x7fe294160db0 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe28b400640 1 -- 172.21.15.62:0/816846769 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c019070 con 0x7fe294160db0 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.344+0000 7fe28b400640 1 -- 172.21.15.62:0/816846769 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c00bac0 con 0x7fe294160db0 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.345+0000 7fe298a00640 1 -- 172.21.15.62:0/816846769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29415fff0 con 0x7fe294160db0 2024-04-25T09:14:18.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.345+0000 7fe298a00640 1 -- 172.21.15.62:0/816846769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2941605b0 con 0x7fe294160db0 2024-04-25T09:14:18.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.347+0000 7fe28b400640 1 -- 172.21.15.62:0/816846769 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe28c01e460 con 0x7fe294160db0 2024-04-25T09:14:18.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.348+0000 7fe28b400640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe27006b500 0x7fe27006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.348+0000 7fe28b400640 1 -- 172.21.15.62:0/816846769 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe28c08a910 con 0x7fe294160db0 2024-04-25T09:14:18.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.348+0000 7fe29c20b740 1 Processor -- start 2024-04-25T09:14:18.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.348+0000 7fe292800640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe27006b500 0x7fe27006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.348+0000 7fe29c20b740 1 -- start start 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.349+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 0x55bdfb990ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.349+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb98f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.349+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.349+0000 7fe29c20b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb8f2c70 con 0x55bdfb9cb440 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.349+0000 7fe29c20b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb89bcf0 con 0x55bdfb9b9830 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.349+0000 7fe29c20b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb8eed10 con 0x55bdfb98ed70 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.349+0000 7fe292800640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe27006b500 0x7fe27006d9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe284009f20 tx=0x7fe2840094d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb98f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb98f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56588/0 (socket says 172.21.15.62:56588) 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 -- 172.21.15.62:0/3087396609 learned_addr learned my addr 172.21.15.62:0/3087396609 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe292800640 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 0x55bdfb990ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 -- 172.21.15.62:0/3087396609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 msgr2=0x55bdfb990ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 0x55bdfb990ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe293200640 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 -- 172.21.15.62:0/3087396609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 msgr2=0x55bdfb9b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9b9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 -- 172.21.15.62:0/3087396609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfb6d17d0 con 0x55bdfb9cb440 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.350+0000 7fe289e00640 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb98f150 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000c00 tx=0x7fe27c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5bc51e3c66fdb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe283000640 1 -- 172.21.15.62:0/3087396609 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c042070 con 0x55bdfb9cb440 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe283000640 1 -- 172.21.15.62:0/3087396609 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c032740 con 0x55bdfb9cb440 2024-04-25T09:14:18.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe283000640 1 -- 172.21.15.62:0/3087396609 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c037040 con 0x55bdfb9cb440 2024-04-25T09:14:18.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe29c20b740 1 -- 172.21.15.62:0/3087396609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 msgr2=0x55bdfb98f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb98f150 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000c00 tx=0x7fe27c02c000 comp rx=0 tx=0).stop 2024-04-25T09:14:18.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe29c20b740 1 -- 172.21.15.62:0/3087396609 shutdown_connections 2024-04-25T09:14:18.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb98f150 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3087396609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 0x55bdfb990ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe29c20b740 1 -- 172.21.15.62:0/3087396609 >> 172.21.15.62:0/3087396609 conn(0x55bdfb956a30 msgr2=0x55bdfb9d4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.351+0000 7fe29c20b740 1 -- 172.21.15.62:0/3087396609 shutdown_connections 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.352+0000 7fe29c20b740 1 -- 172.21.15.62:0/3087396609 wait complete. 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.352+0000 7fe29c20b740 1 -- start start 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.352+0000 7fe29c20b740 1 Processor -- start 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.353+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 0x55bdfb9bb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.353+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 0x55bdfb9b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.353+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.353+0000 7fe29c20b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb8913a0 con 0x55bdfb9cb440 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.353+0000 7fe29c20b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb9b7380 con 0x55bdfb98ed70 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.353+0000 7fe29c20b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb9b0540 con 0x55bdfb9b9830 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.353+0000 7fe29c20b740 10 client.0 _ll_register_callbacks cb 0x55bdfb8aeb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56598/0 (socket says 172.21.15.62:56598) 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 -- 172.21.15.62:0/3821280386 learned_addr learned my addr 172.21.15.62:0/3821280386 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 -- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 msgr2=0x55bdfb9b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe292800640 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 0x55bdfb9bb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe289e00640 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 0x55bdfb9b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 0x55bdfb9b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 -- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 msgr2=0x55bdfb9bb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 0x55bdfb9bb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 -- 172.21.15.62:0/3821280386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdfb6073f0 con 0x55bdfb9cb440 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe293200640 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b5dc0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00b4b0 tx=0x7fe28c0597f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5359f3de9fccb5e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.354+0000 7fe277400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe277400640 1 -- 172.21.15.62:0/3821280386 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c070030 con 0x55bdfb9cb440 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe277400640 1 -- 172.21.15.62:0/3821280386 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe28c058400 con 0x55bdfb9cb440 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 1 -- 172.21.15.62:0/3821280386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe27c032740 con 0x55bdfb9cb440 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe277400640 1 -- 172.21.15.62:0/3821280386 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe28c077050 con 0x55bdfb9cb440 2024-04-25T09:14:18.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 10 client.8064 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 1 -- 172.21.15.62:0/3821280386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55bdfb5c70f0 con 0x55bdfb9cb440 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe277400640 1 -- 172.21.15.62:0/3821280386 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c032740 con 0x55bdfb9cb440 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe277400640 1 -- 172.21.15.62:0/3821280386 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe28c013070 con 0x55bdfb9cb440 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 10 client.8064 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 10 client.8064 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 20 client.8064 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bdfb9ae800 0x55bdfb9b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 4 client.8064 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.355+0000 7fe29c20b740 1 -- 172.21.15.62:0/3821280386 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdfb9001b0 con 0x55bdfb9ae800 2024-04-25T09:14:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.356+0000 7fe292800640 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bdfb9ae800 0x55bdfb9b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.357+0000 7fe292800640 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bdfb9ae800 0x55bdfb9b2730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740ea0590849a404 server_cookie=f970d1b555f050f7 in_seq=0 out_seq=0 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.357+0000 7fe277400640 10 client.8064 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.358+0000 7fe277400640 1 -- 172.21.15.62:0/3821280386 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bdfb9001b0 con 0x55bdfb9ae800 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.358+0000 7fe277400640 10 client.8064 handle_command_reply: tid=0 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.358+0000 7fe29c20b740 1 client.8064 shutdown 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 -- 172.21.15.62:0/3821280386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 msgr2=0x55bdfb9b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b5dc0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00b4b0 tx=0x7fe28c0597f0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 -- 172.21.15.62:0/3821280386 shutdown_connections 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bdfb9ae800 0x55bdfb9b2730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b5dc0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 0x55bdfb9b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 --2- 172.21.15.62:0/3821280386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 0x55bdfb9bb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe277400640 0 client.8064 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:18.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 -- 172.21.15.62:0/3821280386 >> 172.21.15.62:0/3821280386 conn(0x55bdfb956a30 msgr2=0x55bdfb9b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 -- 172.21.15.62:0/3821280386 shutdown_connections 2024-04-25T09:14:18.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 1 -- 172.21.15.62:0/3821280386 wait complete. 2024-04-25T09:14:18.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.359+0000 7fe29c20b740 20 client.8064 trim_cache size 0 max 16384 2024-04-25T09:14:18.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.375+0000 7fe29c20b740 1 Processor -- start 2024-04-25T09:14:18.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.375+0000 7fe29c20b740 1 -- start start 2024-04-25T09:14:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.376+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 0x55bdfb989140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.376+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 0x55bdfb9b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.376+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.376+0000 7fe29c20b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2941ffe70 con 0x55bdfb9cb440 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.376+0000 7fe29c20b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe294157f50 con 0x55bdfb98ed70 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.376+0000 7fe29c20b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe294155740 con 0x55bdfb9b9830 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.376+0000 7fe293200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe293200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56602/0 (socket says 172.21.15.62:56602) 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe293200640 1 -- 172.21.15.62:0/1465907623 learned_addr learned my addr 172.21.15.62:0/1465907623 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe289e00640 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 0x55bdfb9b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe293200640 1 -- 172.21.15.62:0/1465907623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 msgr2=0x55bdfb9b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe293200640 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 0x55bdfb9b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe292800640 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 0x55bdfb989140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe293200640 1 -- 172.21.15.62:0/1465907623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 msgr2=0x55bdfb989140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe293200640 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 0x55bdfb989140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe293200640 1 -- 172.21.15.62:0/1465907623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfb9ae610 con 0x55bdfb9cb440 2024-04-25T09:14:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe293200640 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b70f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe28c057260 tx=0x7fe28c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f76d8674919e31f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe282600640 1 -- 172.21.15.62:0/1465907623 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c07b020 con 0x55bdfb9cb440 2024-04-25T09:14:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe282600640 1 -- 172.21.15.62:0/1465907623 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c058400 con 0x55bdfb9cb440 2024-04-25T09:14:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.377+0000 7fe282600640 1 -- 172.21.15.62:0/1465907623 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c070070 con 0x55bdfb9cb440 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 -- 172.21.15.62:0/1465907623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 msgr2=0x55bdfb9b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b70f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe28c057260 tx=0x7fe28c01f000 comp rx=0 tx=0).stop 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 -- 172.21.15.62:0/1465907623 shutdown_connections 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfb9b70f0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb9b9830 0x55bdfb9b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 --2- 172.21.15.62:0/1465907623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb98ed70 0x55bdfb989140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 -- 172.21.15.62:0/1465907623 >> 172.21.15.62:0/1465907623 conn(0x55bdfb956a30 msgr2=0x55bdfb98d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 -- 172.21.15.62:0/1465907623 shutdown_connections 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.378+0000 7fe29c20b740 1 -- 172.21.15.62:0/1465907623 wait complete. 2024-04-25T09:14:18.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.379+0000 7fe29c20b740 1 -- start start 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.379+0000 7fe29c20b740 1 Processor -- start 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 0x55bdfb9b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe29c20b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfba381d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe29c20b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe294157950 con 0x55bdfb9cb440 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe29c20b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb9cb0c0 con 0x55bdfb9b9830 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe29c20b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdfb98da30 con 0x55bdfb98ed70 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe29c20b740 10 client.0 _ll_register_callbacks cb 0x7fe28c058d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe293200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfba381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe293200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfba381d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56614/0 (socket says 172.21.15.62:56614) 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe293200640 1 -- 172.21.15.62:0/2114460511 learned_addr learned my addr 172.21.15.62:0/2114460511 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe289e00640 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe293200640 1 -- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 msgr2=0x55bdfb9b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.380+0000 7fe292800640 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 0x55bdfb9b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe293200640 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 0x55bdfb9b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe293200640 1 -- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 msgr2=0x55bdfb9adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe293200640 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe293200640 1 -- 172.21.15.62:0/2114460511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdfba3e660 con 0x55bdfb9cb440 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe289e00640 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe293200640 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfba381d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe28c058f40 tx=0x7fe28c088c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c516756d7959cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe277400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe277400640 1 -- 172.21.15.62:0/2114460511 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c07b050 con 0x55bdfb9cb440 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe277400640 1 -- 172.21.15.62:0/2114460511 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe28c054f00 con 0x55bdfb9cb440 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe29c20b740 1 -- 172.21.15.62:0/2114460511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe28c058400 con 0x55bdfb9cb440 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe277400640 1 -- 172.21.15.62:0/2114460511 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe28c069050 con 0x55bdfb9cb440 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe29c20b740 10 client.8070 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:18.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe29c20b740 1 -- 172.21.15.62:0/2114460511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55bdfba3f420 con 0x55bdfb9cb440 2024-04-25T09:14:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe277400640 1 -- 172.21.15.62:0/2114460511 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c058400 con 0x55bdfb9cb440 2024-04-25T09:14:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.381+0000 7fe277400640 1 -- 172.21.15.62:0/2114460511 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe28c06c020 con 0x55bdfb9cb440 2024-04-25T09:14:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.382+0000 7fe29c20b740 10 client.8070 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.382+0000 7fe29c20b740 10 client.8070 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.382+0000 7fe29c20b740 20 client.8070 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.382+0000 7fe29c20b740 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bdfba38760 0x55bdfb9ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.382+0000 7fe29c20b740 4 client.8070 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T09:14:18.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.382+0000 7fe29c20b740 1 -- 172.21.15.62:0/2114460511 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55bdfb98b6a0 con 0x55bdfba38760 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.382+0000 7fe292800640 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bdfba38760 0x55bdfb9ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.383+0000 7fe292800640 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bdfba38760 0x55bdfb9ca1c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3d6970a0b81cf8 server_cookie=6b2f8f28520561f in_seq=0 out_seq=0 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.383+0000 7fe277400640 10 client.8070 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe277400640 1 -- 172.21.15.62:0/2114460511 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55bdfb98b6a0 con 0x55bdfba38760 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe277400640 10 client.8070 handle_command_reply: tid=0 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 client.8070 shutdown 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 -- 172.21.15.62:0/2114460511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 msgr2=0x55bdfba381d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfba381d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe28c058f40 tx=0x7fe28c088c30 comp rx=0 tx=0).stop 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 -- 172.21.15.62:0/2114460511 shutdown_connections 2024-04-25T09:14:18.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x55bdfba38760 0x55bdfb9ca1c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bdfb9cb440 0x55bdfba381d0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bdfb9b9830 0x55bdfb9adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 --2- 172.21.15.62:0/2114460511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55bdfb98ed70 0x55bdfb9b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe277400640 0 client.8070 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 -- 172.21.15.62:0/2114460511 >> 172.21.15.62:0/2114460511 conn(0x55bdfb956a30 msgr2=0x55bdfb9b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 -- 172.21.15.62:0/2114460511 shutdown_connections 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 1 -- 172.21.15.62:0/2114460511 wait complete. 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.384+0000 7fe29c20b740 20 client.8070 trim_cache size 0 max 16384 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:14:18.375 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:14:18.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 -- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe27006b500 msgr2=0x7fe27006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe27006b500 0x7fe27006d9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe284009f20 tx=0x7fe2840094d0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 -- 172.21.15.62:0/816846769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe294160db0 msgr2=0x7fe2941f6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe294160db0 0x7fe2941f6d30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00d830 tx=0x7fe28c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 -- 172.21.15.62:0/816846769 shutdown_connections 2024-04-25T09:14:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe27006b500 0x7fe27006d9c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe294160db0 0x7fe2941f6d30 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe29415dd50 0x7fe29415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 --2- 172.21.15.62:0/816846769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe2940d3960 0x7fe29415ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 -- 172.21.15.62:0/816846769 >> 172.21.15.62:0/816846769 conn(0x7fe29406b000 msgr2=0x7fe2940cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 -- 172.21.15.62:0/816846769 shutdown_connections 2024-04-25T09:14:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.385+0000 7fe298a00640 1 -- 172.21.15.62:0/816846769 wait complete. 2024-04-25T09:14:18.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:18.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:14:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.590+0000 7f8b49000640 1 Processor -- start 2024-04-25T09:14:18.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.590+0000 7f8b49000640 1 -- start start 2024-04-25T09:14:18.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.591+0000 7f8b49000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b2d80 0x7f8b440b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.591+0000 7f8b49000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b440b36a0 0x7f8b440ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.592+0000 7f8b49000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b440b4430 con 0x7f8b440b54d0 2024-04-25T09:14:18.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.592+0000 7f8b49000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b440b4710 con 0x7f8b440b36a0 2024-04-25T09:14:18.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.592+0000 7f8b49000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440b49f0 con 0x7f8b440b2d80 2024-04-25T09:14:18.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.592+0000 7f8b43e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8b440b54d0 0x7f8b440700c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:36948/0 (socket says 172.21.15.62:36948) 2024-04-25T09:14:18.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.592+0000 7f8b43e00640 1 -- 172.21.15.62:0/3964963849 learned_addr learned my addr 172.21.15.62:0/3964963849 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:18.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.592+0000 7f8b43400640 1 --2- 172.21.15.62:0/3964963849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b2d80 0x7f8b440b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.592+0000 7f8b42a00640 1 --2- 172.21.15.62:0/3964963849 >> v2:172.21.15.121:3300/0 conn(0x7f8b440b36a0 0x7f8b440ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.593+0000 7f8b42000640 1 -- 172.21.15.62:0/3964963849 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029567020 0 0) 0x7f8b440b4430 con 0x7f8b440b54d0 2024-04-25T09:14:18.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.593+0000 7f8b42000640 1 -- 172.21.15.62:0/3964963849 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b28005730 con 0x7f8b440b54d0 2024-04-25T09:14:18.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.593+0000 7f8b43400640 1 -- 172.21.15.62:0/3964963849 >> v2:172.21.15.121:3300/0 conn(0x7f8b440b36a0 msgr2=0x7f8b440ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.593+0000 7f8b43400640 1 --2- 172.21.15.62:0/3964963849 >> v2:172.21.15.121:3300/0 conn(0x7f8b440b36a0 0x7f8b440ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.593+0000 7f8b43400640 1 -- 172.21.15.62:0/3964963849 >> v1:172.21.15.121:6789/0 conn(0x7f8b440b54d0 legacy=0x7f8b440700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.593+0000 7f8b43400640 1 -- 172.21.15.62:0/3964963849 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440b4cd0 con 0x7f8b440b2d80 2024-04-25T09:14:18.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.594+0000 7f8b43400640 1 --2- 172.21.15.62:0/3964963849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b2d80 0x7f8b440b3160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b340097a0 tx=0x7f8b3402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad69afe135b31ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.595+0000 7f8b42000640 1 -- 172.21.15.62:0/3964963849 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34037070 con 0x7f8b440b2d80 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.595+0000 7f8b42000640 1 -- 172.21.15.62:0/3964963849 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34032440 con 0x7f8b440b2d80 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.595+0000 7f8b42000640 1 -- 172.21.15.62:0/3964963849 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340327f0 con 0x7f8b440b2d80 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.595+0000 7f8b49000640 1 -- 172.21.15.62:0/3964963849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b2d80 msgr2=0x7f8b440b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.595+0000 7f8b49000640 1 --2- 172.21.15.62:0/3964963849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b2d80 0x7f8b440b3160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b340097a0 tx=0x7f8b3402a180 comp rx=0 tx=0).stop 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.596+0000 7f8b49000640 1 -- 172.21.15.62:0/3964963849 shutdown_connections 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.596+0000 7f8b49000640 1 --2- 172.21.15.62:0/3964963849 >> v2:172.21.15.121:3300/0 conn(0x7f8b440b36a0 0x7f8b440ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.596+0000 7f8b49000640 1 --2- 172.21.15.62:0/3964963849 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b2d80 0x7f8b440b3160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.596+0000 7f8b49000640 1 -- 172.21.15.62:0/3964963849 >> 172.21.15.62:0/3964963849 conn(0x7f8b44066f20 msgr2=0x7f8b440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.596+0000 7f8b49000640 1 -- 172.21.15.62:0/3964963849 shutdown_connections 2024-04-25T09:14:18.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.596+0000 7f8b49000640 1 -- 172.21.15.62:0/3964963849 wait complete. 2024-04-25T09:14:18.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.597+0000 7f8b49000640 1 Processor -- start 2024-04-25T09:14:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.597+0000 7f8b49000640 1 -- start start 2024-04-25T09:14:18.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.598+0000 7f8b49000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b440b2d80 0x7f8b44149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.598+0000 7f8b49000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b36a0 0x7f8b4414bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.598+0000 7f8b49000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b440b54d0 0x7f8b4414a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.598+0000 7f8b49000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440ce7f0 con 0x7f8b440b54d0 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.598+0000 7f8b49000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440b40e0 con 0x7f8b440b2d80 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.598+0000 7f8b49000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440ce970 con 0x7f8b440b36a0 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b440b54d0 0x7f8b4414a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b440b54d0 0x7f8b4414a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56626/0 (socket says 172.21.15.62:56626) 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43e00640 1 -- 172.21.15.62:0/603924471 learned_addr learned my addr 172.21.15.62:0/603924471 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b42a00640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b36a0 0x7f8b4414bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43400640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b440b2d80 0x7f8b44149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43e00640 1 -- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b36a0 msgr2=0x7f8b4414bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43e00640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b36a0 0x7f8b4414bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43e00640 1 -- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b440b2d80 msgr2=0x7f8b44149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43e00640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b440b2d80 0x7f8b44149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.599+0000 7f8b43e00640 1 -- 172.21.15.62:0/603924471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4414ad50 con 0x7f8b440b54d0 2024-04-25T09:14:18.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.600+0000 7f8b43e00640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b440b54d0 0x7f8b4414a1e0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00cb70 tx=0x7f8b3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.600+0000 7f8b3b400640 1 -- 172.21.15.62:0/603924471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c017070 con 0x7f8b440b54d0 2024-04-25T09:14:18.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.600+0000 7f8b3b400640 1 -- 172.21.15.62:0/603924471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c00ad40 con 0x7f8b440b54d0 2024-04-25T09:14:18.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.600+0000 7f8b3b400640 1 -- 172.21.15.62:0/603924471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c012620 con 0x7f8b440b54d0 2024-04-25T09:14:18.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.600+0000 7f8b49000640 1 -- 172.21.15.62:0/603924471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4414b030 con 0x7f8b440b54d0 2024-04-25T09:14:18.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.600+0000 7f8b49000640 1 -- 172.21.15.62:0/603924471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4414b5f0 con 0x7f8b440b54d0 2024-04-25T09:14:18.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.603+0000 7f8b3b400640 1 -- 172.21.15.62:0/603924471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b3c01a070 con 0x7f8b440b54d0 2024-04-25T09:14:18.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.604+0000 7f8b3b400640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b2406b4d0 0x7f8b2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.604+0000 7f8b3b400640 1 -- 172.21.15.62:0/603924471 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8b3c08a030 con 0x7f8b440b54d0 2024-04-25T09:14:18.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.604+0000 7f8b4d5c2740 1 Processor -- start 2024-04-25T09:14:18.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.605+0000 7f8b4d5c2740 1 -- start start 2024-04-25T09:14:18.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.605+0000 7f8b43400640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b2406b4d0 0x7f8b2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.605+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a356fd70 0x5605a3571ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.605+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a35ac440 0x5605a3570150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.605+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a359a830 0x5605a359acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a34d3c70 con 0x5605a35ac440 2024-04-25T09:14:18.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a347ccf0 con 0x5605a356fd70 2024-04-25T09:14:18.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605a34cfd10 con 0x5605a359a830 2024-04-25T09:14:18.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b42a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a35ac440 0x5605a3570150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b43400640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b2406b4d0 0x7f8b2406d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8b34034000 tx=0x7f8b3402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b42a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a35ac440 0x5605a3570150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56640/0 (socket says 172.21.15.62:56640) 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b42a00640 1 -- 172.21.15.62:0/3435341533 learned_addr learned my addr 172.21.15.62:0/3435341533 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b43400640 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a356fd70 0x5605a3571ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b43e00640 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a359a830 0x5605a359acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b42a00640 1 -- 172.21.15.62:0/3435341533 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a359a830 msgr2=0x5605a359acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b42a00640 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a359a830 0x5605a359acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b42a00640 1 -- 172.21.15.62:0/3435341533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a356fd70 msgr2=0x5605a3571ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b42a00640 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a356fd70 0x5605a3571ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.606+0000 7f8b42a00640 1 -- 172.21.15.62:0/3435341533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605a32b27d0 con 0x5605a35ac440 2024-04-25T09:14:18.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.607+0000 7f8b42a00640 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a35ac440 0x5605a3570150 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009ae0 tx=0x7f8b30009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15939b72c56bb20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.607+0000 7f8b2fe00640 1 -- 172.21.15.62:0/3435341533 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30019020 con 0x5605a35ac440 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.607+0000 7f8b2fe00640 1 -- 172.21.15.62:0/3435341533 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30004230 con 0x5605a35ac440 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.607+0000 7f8b2fe00640 1 -- 172.21.15.62:0/3435341533 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30011960 con 0x5605a35ac440 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.607+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3435341533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a35ac440 msgr2=0x5605a3570150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.607+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a35ac440 0x5605a3570150 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009ae0 tx=0x7f8b30009190 comp rx=0 tx=0).stop 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.608+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3435341533 shutdown_connections 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.608+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a35ac440 0x5605a3570150 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.608+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a359a830 0x5605a359acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.608+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3435341533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a356fd70 0x5605a3571ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.608+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3435341533 >> 172.21.15.62:0/3435341533 conn(0x5605a3537a30 msgr2=0x5605a35b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.608+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3435341533 shutdown_connections 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.608+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3435341533 wait complete. 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.608+0000 7f8b4d5c2740 1 -- start start 2024-04-25T09:14:18.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.609+0000 7f8b4d5c2740 1 Processor -- start 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 0x5605a358c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3591640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 0x5605a3599120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a34723a0 con 0x5605a359a830 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a3598380 con 0x5605a35ac440 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605a3593990 con 0x5605a356fd70 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b4d5c2740 10 client.0 _ll_register_callbacks cb 0x5605a348fb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b42a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3591640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b42a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3591640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56648/0 (socket says 172.21.15.62:56648) 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b42a00640 1 -- 172.21.15.62:0/1247228074 learned_addr learned my addr 172.21.15.62:0/1247228074 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b43e00640 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 0x5605a3599120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b42a00640 1 -- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 msgr2=0x5605a358c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b43400640 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 0x5605a358c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b42a00640 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 0x5605a358c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b42a00640 1 -- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 msgr2=0x5605a3599120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b42a00640 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 0x5605a3599120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b42a00640 1 -- 172.21.15.62:0/1247228074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605a31e83f0 con 0x5605a359a830 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.610+0000 7f8b43e00640 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 0x5605a3599120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.611+0000 7f8b42a00640 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3591640 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8b30008010 tx=0x7f8b30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211ded826330d2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.611+0000 7f8b2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.611+0000 7f8b2cc00640 1 -- 172.21.15.62:0/1247228074 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3002c060 con 0x5605a359a830 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.611+0000 7f8b2cc00640 1 -- 172.21.15.62:0/1247228074 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8b30003610 con 0x5605a359a830 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.611+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1247228074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b30004230 con 0x5605a359a830 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.611+0000 7f8b2cc00640 1 -- 172.21.15.62:0/1247228074 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b30019040 con 0x5605a359a830 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.611+0000 7f8b4d5c2740 10 client.8079 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.611+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1247228074 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x5605a31a80f0 con 0x5605a359a830 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b2cc00640 1 -- 172.21.15.62:0/1247228074 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30004230 con 0x5605a359a830 2024-04-25T09:14:18.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b2cc00640 1 -- 172.21.15.62:0/1247228074 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b30003810 con 0x5605a359a830 2024-04-25T09:14:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b4d5c2740 10 client.8079 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b4d5c2740 10 client.8079 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b4d5c2740 20 client.8079 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x5605a35997a0 0x5605a358fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b4d5c2740 4 client.8079 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:14:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1247228074 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605a34e11b0 con 0x5605a35997a0 2024-04-25T09:14:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.612+0000 7f8b43400640 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x5605a35997a0 0x5605a358fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.613+0000 7f8b43400640 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x5605a35997a0 0x5605a358fed0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83200a6bcc9a579e server_cookie=c7bd219684df1660 in_seq=0 out_seq=0 2024-04-25T09:14:18.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.613+0000 7f8b2cc00640 10 client.8079 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:18.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b2cc00640 1 -- 172.21.15.62:0/1247228074 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605a34e11b0 con 0x5605a35997a0 2024-04-25T09:14:18.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b2cc00640 10 client.8079 handle_command_reply: tid=0 2024-04-25T09:14:18.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 client.8079 shutdown 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1247228074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 msgr2=0x5605a3591640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3591640 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f8b30008010 tx=0x7f8b30004730 comp rx=0 tx=0).stop 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1247228074 shutdown_connections 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x5605a35997a0 0x5605a358fed0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3591640 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 0x5605a3599120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b2cc00640 0 client.8079 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1247228074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 0x5605a358c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1247228074 >> 172.21.15.62:0/1247228074 conn(0x5605a3537a30 msgr2=0x5605a35958a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1247228074 shutdown_connections 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1247228074 wait complete. 2024-04-25T09:14:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.615+0000 7f8b4d5c2740 20 client.8079 trim_cache size 0 max 16384 2024-04-25T09:14:18.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.631+0000 7f8b4d5c2740 1 Processor -- start 2024-04-25T09:14:18.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.631+0000 7f8b4d5c2740 1 -- start start 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 0x5605a3592640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3619990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 0x5605a3598b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b441e4a60 con 0x5605a359a830 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b441e3c20 con 0x5605a35ac440 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b441f7910 con 0x5605a356fd70 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b42a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3619990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b42a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3619990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56658/0 (socket says 172.21.15.62:56658) 2024-04-25T09:14:18.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.632+0000 7f8b42a00640 1 -- 172.21.15.62:0/1597621294 learned_addr learned my addr 172.21.15.62:0/1597621294 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.633+0000 7f8b42a00640 1 -- 172.21.15.62:0/1597621294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 msgr2=0x5605a3592640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.633+0000 7f8b43400640 1 --2- 172.21.15.62:0/1597621294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 0x5605a3592640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.633+0000 7f8b42a00640 1 --2- 172.21.15.62:0/1597621294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 0x5605a3592640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.633+0000 7f8b42a00640 1 -- 172.21.15.62:0/1597621294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 msgr2=0x5605a3598b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.633+0000 7f8b42a00640 1 --2- 172.21.15.62:0/1597621294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 0x5605a3598b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.633+0000 7f8b42a00640 1 -- 172.21.15.62:0/1597621294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605a359dfe0 con 0x5605a359a830 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.633+0000 7f8b42a00640 1 --2- 172.21.15.62:0/1597621294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3619990 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006b10 tx=0x7f8b30021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b5a0ad8bb9f718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.633+0000 7f8b2f400640 1 -- 172.21.15.62:0/1597621294 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3003b070 con 0x5605a359a830 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b2f400640 1 -- 172.21.15.62:0/1597621294 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b300040f0 con 0x5605a359a830 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b2f400640 1 -- 172.21.15.62:0/1597621294 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30003810 con 0x5605a359a830 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1597621294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 msgr2=0x5605a3619990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1597621294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3619990 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8b30006b10 tx=0x7f8b30021040 comp rx=0 tx=0).stop 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1597621294 shutdown_connections 2024-04-25T09:14:18.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1597621294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a359a830 0x5605a3619990 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1597621294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a35ac440 0x5605a3598b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/1597621294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a356fd70 0x5605a3592640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1597621294 >> 172.21.15.62:0/1597621294 conn(0x5605a3537a30 msgr2=0x5605a3594be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1597621294 shutdown_connections 2024-04-25T09:14:18.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.634+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/1597621294 wait complete. 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.635+0000 7f8b4d5c2740 1 -- start start 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.635+0000 7f8b4d5c2740 1 Processor -- start 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a356fd70 0x5605a35f6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a359a830 0x5605a35ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b4d5c2740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a35ac440 0x5605a35ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4415bd60 con 0x5605a356fd70 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605a356aab0 con 0x5605a359a830 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b4d5c2740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605a360a270 con 0x5605a35ac440 2024-04-25T09:14:18.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b4d5c2740 10 client.0 _ll_register_callbacks cb 0x5605a356c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b43400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a356fd70 0x5605a35f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b43400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a356fd70 0x5605a35f6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56670/0 (socket says 172.21.15.62:56670) 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.636+0000 7f8b43400640 1 -- 172.21.15.62:0/3957620350 learned_addr learned my addr 172.21.15.62:0/3957620350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b43400640 1 -- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a35ac440 msgr2=0x5605a35ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b43400640 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a35ac440 0x5605a35ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b43400640 1 -- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a359a830 msgr2=0x5605a35ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b43400640 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a359a830 0x5605a35ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b43400640 1 -- 172.21.15.62:0/3957620350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605a360a050 con 0x5605a356fd70 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b43400640 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a356fd70 0x5605a35f6600 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8b340098c0 tx=0x7f8b3402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9292af95ee42e540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b2cc00640 1 -- 172.21.15.62:0/3957620350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34037040 con 0x5605a356fd70 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b2cc00640 1 -- 172.21.15.62:0/3957620350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8b3404b050 con 0x5605a356fd70 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3957620350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b300040f0 con 0x5605a356fd70 2024-04-25T09:14:18.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.637+0000 7f8b2cc00640 1 -- 172.21.15.62:0/3957620350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b3405a050 con 0x5605a356fd70 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b4d5c2740 10 client.8085 fetch_fsmap learned FSMap version 71 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3957620350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x5605a35f6ec0 con 0x5605a356fd70 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b2cc00640 1 -- 172.21.15.62:0/3957620350 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b300040f0 con 0x5605a356fd70 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b2cc00640 1 -- 172.21.15.62:0/3957620350 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8b3405d020 con 0x5605a356fd70 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b4d5c2740 10 client.8085 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b4d5c2740 10 client.8085 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b4d5c2740 20 client.8085 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x5605a35c9260 0x5605a3593780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b4d5c2740 4 client.8085 mds_command: new command op to 7800 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.638+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3957620350 --> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5605a34db500 con 0x5605a35c9260 2024-04-25T09:14:18.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.639+0000 7f8b42a00640 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x5605a35c9260 0x5605a3593780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b42a00640 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x5605a35c9260 0x5605a3593780 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c01b506b83d9d9 server_cookie=4efaee21bc6551cb in_seq=0 out_seq=0 2024-04-25T09:14:18.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b2cc00640 10 client.8085 ms_handle_connect on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b2cc00640 1 -- 172.21.15.62:0/3957620350 <== mds.0 v2:172.21.15.121:6834/2489379768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5605a34db500 con 0x5605a35c9260 2024-04-25T09:14:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b2cc00640 10 client.8085 handle_command_reply: tid=0 2024-04-25T09:14:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 client.8085 shutdown 2024-04-25T09:14:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3957620350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a356fd70 msgr2=0x5605a35f6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a356fd70 0x5605a35f6600 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8b340098c0 tx=0x7f8b3402c070 comp rx=0 tx=0).stop 2024-04-25T09:14:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3957620350 shutdown_connections 2024-04-25T09:14:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:6834/2489379768,v1:172.21.15.121:6836/2489379768] conn(0x5605a35c9260 0x5605a3593780 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5605a35ac440 0x5605a35ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5605a359a830 0x5605a35ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b2cc00640 0 client.8085 ms_handle_reset on v2:172.21.15.121:6834/2489379768 2024-04-25T09:14:18.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 --2- 172.21.15.62:0/3957620350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5605a356fd70 0x5605a35f6600 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.640+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3957620350 >> 172.21.15.62:0/3957620350 conn(0x5605a3537a30 msgr2=0x5605a3598b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3957620350 shutdown_connections 2024-04-25T09:14:18.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b4d5c2740 1 -- 172.21.15.62:0/3957620350 wait complete. 2024-04-25T09:14:18.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b4d5c2740 20 client.8085 trim_cache size 0 max 16384 2024-04-25T09:14:18.632 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:14:18.632 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-25T09:14:18.632 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:14:18.632 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 -- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b2406b4d0 msgr2=0x7f8b2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b2406b4d0 0x7f8b2406d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8b34034000 tx=0x7f8b3402e000 comp rx=0 tx=0).stop 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 -- 172.21.15.62:0/603924471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b440b54d0 msgr2=0x7f8b4414a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b440b54d0 0x7f8b4414a1e0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00cb70 tx=0x7f8b3c00a430 comp rx=0 tx=0).stop 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 -- 172.21.15.62:0/603924471 shutdown_connections 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b2406b4d0 0x7f8b2406d990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b440b54d0 0x7f8b4414a1e0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8b440b36a0 0x7f8b4414bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 --2- 172.21.15.62:0/603924471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b440b2d80 0x7f8b44149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.641+0000 7f8b49000640 1 -- 172.21.15.62:0/603924471 >> 172.21.15.62:0/603924471 conn(0x7f8b44066f20 msgr2=0x7f8b441599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.642+0000 7f8b49000640 1 -- 172.21.15.62:0/603924471 shutdown_connections 2024-04-25T09:14:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.642+0000 7f8b49000640 1 -- 172.21.15.62:0/603924471 wait complete. 2024-04-25T09:14:18.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:18.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.860+0000 7fb695600640 1 Processor -- start 2024-04-25T09:14:18.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.861+0000 7fb695600640 1 -- start start 2024-04-25T09:14:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb695600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb6900b56f0 0x7fb690160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb695600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb6900b27a0 0x7fb690161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb695600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900d3110 con 0x7fb6900b6a90 2024-04-25T09:14:18.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb695600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb690161780 con 0x7fb6900b27a0 2024-04-25T09:14:18.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb695600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900cedc0 con 0x7fb6900b56f0 2024-04-25T09:14:18.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb694c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb6900b27a0 0x7fb690161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb68f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb6900b6a90 0x7fb690161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:43394/0 (socket says 172.21.15.62:43394) 2024-04-25T09:14:18.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb694c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb6900b27a0 0x7fb690161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56674/0 (socket says 172.21.15.62:56674) 2024-04-25T09:14:18.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.862+0000 7fb68f400640 1 -- 172.21.15.62:0/3884079161 learned_addr learned my addr 172.21.15.62:0/3884079161 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:18.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.863+0000 7fb68fe00640 1 --2- 172.21.15.62:0/3884079161 >> v2:172.21.15.121:3300/0 conn(0x7fb6900b56f0 0x7fb690160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.863+0000 7fb694c00640 1 -- 172.21.15.62:0/3884079161 >> v2:172.21.15.121:3300/0 conn(0x7fb6900b56f0 msgr2=0x7fb690160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:18.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.863+0000 7fb694c00640 1 --2- 172.21.15.62:0/3884079161 >> v2:172.21.15.121:3300/0 conn(0x7fb6900b56f0 0x7fb690160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.863+0000 7fb694c00640 1 -- 172.21.15.62:0/3884079161 >> v1:172.21.15.62:6789/0 conn(0x7fb6900b6a90 legacy=0x7fb690161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.863+0000 7fb694c00640 1 -- 172.21.15.62:0/3884079161 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6900d33f0 con 0x7fb6900b27a0 2024-04-25T09:14:18.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.863+0000 7fb68ea00640 1 -- 172.21.15.62:0/3884079161 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812808005 0 0) 0x7fb6900d3110 con 0x7fb6900b6a90 2024-04-25T09:14:18.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.864+0000 7fb694c00640 1 --2- 172.21.15.62:0/3884079161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b27a0 0x7fb690161240 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb69008c310 tx=0x7fb68802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666ac11ebbb01f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.864+0000 7fb68ea00640 1 -- 172.21.15.62:0/3884079161 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688007740 con 0x7fb6900b27a0 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.864+0000 7fb68ea00640 1 -- 172.21.15.62:0/3884079161 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6880078e0 con 0x7fb6900b27a0 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.864+0000 7fb68ea00640 1 -- 172.21.15.62:0/3884079161 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688007cb0 con 0x7fb6900b27a0 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.865+0000 7fb695600640 1 -- 172.21.15.62:0/3884079161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b27a0 msgr2=0x7fb690161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.865+0000 7fb695600640 1 --2- 172.21.15.62:0/3884079161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b27a0 0x7fb690161240 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb69008c310 tx=0x7fb68802a680 comp rx=0 tx=0).stop 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.865+0000 7fb695600640 1 -- 172.21.15.62:0/3884079161 shutdown_connections 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.865+0000 7fb695600640 1 --2- 172.21.15.62:0/3884079161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b27a0 0x7fb690161240 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.865+0000 7fb695600640 1 --2- 172.21.15.62:0/3884079161 >> v2:172.21.15.121:3300/0 conn(0x7fb6900b56f0 0x7fb690160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.865+0000 7fb695600640 1 -- 172.21.15.62:0/3884079161 >> 172.21.15.62:0/3884079161 conn(0x7fb690066d10 msgr2=0x7fb690067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.865+0000 7fb695600640 1 -- 172.21.15.62:0/3884079161 shutdown_connections 2024-04-25T09:14:18.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.865+0000 7fb695600640 1 -- 172.21.15.62:0/3884079161 wait complete. 2024-04-25T09:14:18.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.867+0000 7fb695600640 1 Processor -- start 2024-04-25T09:14:18.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.867+0000 7fb695600640 1 -- start start 2024-04-25T09:14:18.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb695600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6900b27a0 0x7fb6900c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb695600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6900b56f0 0x7fb6900c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb695600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b6a90 0x7fb6900cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb695600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900b2d60 con 0x7fb6900b6a90 2024-04-25T09:14:18.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb695600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900cea50 con 0x7fb6900b27a0 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb695600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900b2ee0 con 0x7fb6900b56f0 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb694c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b6a90 0x7fb6900cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb694c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b6a90 0x7fb6900cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56682/0 (socket says 172.21.15.62:56682) 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.868+0000 7fb694c00640 1 -- 172.21.15.62:0/1168960189 learned_addr learned my addr 172.21.15.62:0/1168960189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.869+0000 7fb68fe00640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6900b27a0 0x7fb6900c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.869+0000 7fb68f400640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6900b56f0 0x7fb6900c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.869+0000 7fb694c00640 1 -- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6900b56f0 msgr2=0x7fb6900c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.869+0000 7fb694c00640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6900b56f0 0x7fb6900c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.869+0000 7fb694c00640 1 -- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6900b27a0 msgr2=0x7fb6900c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.869+0000 7fb694c00640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6900b27a0 0x7fb6900c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:18.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.869+0000 7fb694c00640 1 -- 172.21.15.62:0/1168960189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6900c9920 con 0x7fb6900b6a90 2024-04-25T09:14:18.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.869+0000 7fb694c00640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b6a90 0x7fb6900cb3b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb68802a6c0 tx=0x7fb688004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.870+0000 7fb687e00640 1 -- 172.21.15.62:0/1168960189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688040070 con 0x7fb6900b6a90 2024-04-25T09:14:18.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.870+0000 7fb687e00640 1 -- 172.21.15.62:0/1168960189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb688031bc0 con 0x7fb6900b6a90 2024-04-25T09:14:18.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.870+0000 7fb687e00640 1 -- 172.21.15.62:0/1168960189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688009450 con 0x7fb6900b6a90 2024-04-25T09:14:18.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.870+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6900cd980 con 0x7fb6900b6a90 2024-04-25T09:14:18.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.870+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb690159690 con 0x7fb6900b6a90 2024-04-25T09:14:18.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.871+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6901591c0 con 0x7fb6900b6a90 2024-04-25T09:14:18.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.872+0000 7fb687e00640 1 -- 172.21.15.62:0/1168960189 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb688039040 con 0x7fb6900b6a90 2024-04-25T09:14:18.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.873+0000 7fb687e00640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb67006b4d0 0x7fb67006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:18.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.873+0000 7fb687e00640 1 -- 172.21.15.62:0/1168960189 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb6880ab260 con 0x7fb6900b6a90 2024-04-25T09:14:18.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.874+0000 7fb68fe00640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb67006b4d0 0x7fb67006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:18.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.875+0000 7fb68fe00640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb67006b4d0 0x7fb67006d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb680006fd0 tx=0x7fb680008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:18.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:18.878+0000 7fb687e00640 1 -- 172.21.15.62:0/1168960189 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb68807a000 con 0x7fb6900b6a90 2024-04-25T09:14:19.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.125+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb69004ef20 con 0x7fb6900b6a90 2024-04-25T09:14:19.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.127+0000 7fb687e00640 1 -- 172.21.15.62:0/1168960189 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb688079900 con 0x7fb6900b6a90 2024-04-25T09:14:19.114 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:19.114 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:19.114 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb67006b4d0 msgr2=0x7fb67006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:19.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb67006b4d0 0x7fb67006d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb680006fd0 tx=0x7fb680008040 comp rx=0 tx=0).stop 2024-04-25T09:14:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b6a90 msgr2=0x7fb6900cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b6a90 0x7fb6900cb3b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb68802a6c0 tx=0x7fb688004a40 comp rx=0 tx=0).stop 2024-04-25T09:14:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 shutdown_connections 2024-04-25T09:14:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb67006b4d0 0x7fb67006d990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6900b6a90 0x7fb6900cb3b0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6900b56f0 0x7fb6900c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 --2- 172.21.15.62:0/1168960189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6900b27a0 0x7fb6900c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.129+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 >> 172.21.15.62:0/1168960189 conn(0x7fb690066d10 msgr2=0x7fb6900d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.130+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 shutdown_connections 2024-04-25T09:14:19.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.130+0000 7fb695600640 1 -- 172.21.15.62:0/1168960189 wait complete. 2024-04-25T09:14:19.137 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:14:19.138 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:14:19.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.348+0000 7f602ec00640 1 Processor -- start 2024-04-25T09:14:19.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.348+0000 7f602ec00640 1 -- start start 2024-04-25T09:14:19.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.349+0000 7f602ec00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f602815df40 0x7f60280d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:19.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.349+0000 7f602ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60280b9750 0x7f60280d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:19.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.349+0000 7f602ec00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60280d2690 con 0x7f602815db60 2024-04-25T09:14:19.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.349+0000 7f602ec00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60280b7c20 con 0x7f602815df40 2024-04-25T09:14:19.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.349+0000 7f602ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60280d2970 con 0x7f60280b9750 2024-04-25T09:14:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.350+0000 7f602e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60280b9750 0x7f60280d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.350+0000 7f602d800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f602815db60 0x7f60280d1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:36964/0 (socket says 172.21.15.62:36964) 2024-04-25T09:14:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.350+0000 7f602e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60280b9750 0x7f60280d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:47006/0 (socket says 172.21.15.62:47006) 2024-04-25T09:14:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.350+0000 7f602d800640 1 -- 172.21.15.62:0/1067900716 learned_addr learned my addr 172.21.15.62:0/1067900716 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:19.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.350+0000 7f602ce00640 1 --2- 172.21.15.62:0/1067900716 >> v2:172.21.15.121:3300/0 conn(0x7f602815df40 0x7f60280d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:19.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.351+0000 7f602ce00640 1 -- 172.21.15.62:0/1067900716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60280b9750 msgr2=0x7f60280d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:19.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.351+0000 7f602ce00640 1 --2- 172.21.15.62:0/1067900716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60280b9750 0x7f60280d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.351+0000 7f602ce00640 1 -- 172.21.15.62:0/1067900716 >> v1:172.21.15.121:6789/0 conn(0x7f602815db60 legacy=0x7f60280d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:19.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.351+0000 7f602ce00640 1 -- 172.21.15.62:0/1067900716 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280d2c50 con 0x7f602815df40 2024-04-25T09:14:19.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.351+0000 7f602e200640 1 --2- 172.21.15.62:0/1067900716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60280b9750 0x7f60280d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:14:19.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.352+0000 7f602ce00640 1 --2- 172.21.15.62:0/1067900716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f602815df40 0x7f60280d02f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f601c009d90 tx=0x7f601c02a540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612d844fe1de9967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:19.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.353+0000 7f6027e00640 1 -- 172.21.15.62:0/1067900716 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c007740 con 0x7f602815df40 2024-04-25T09:14:19.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.353+0000 7f6027e00640 1 -- 172.21.15.62:0/1067900716 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c0078e0 con 0x7f602815df40 2024-04-25T09:14:19.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.353+0000 7f6027e00640 1 -- 172.21.15.62:0/1067900716 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c007cb0 con 0x7f602815df40 2024-04-25T09:14:19.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.354+0000 7f602ec00640 1 -- 172.21.15.62:0/1067900716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f602815df40 msgr2=0x7f60280d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:19.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.354+0000 7f602ec00640 1 --2- 172.21.15.62:0/1067900716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f602815df40 0x7f60280d02f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f601c009d90 tx=0x7f601c02a540 comp rx=0 tx=0).stop 2024-04-25T09:14:19.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.354+0000 7f602ec00640 1 -- 172.21.15.62:0/1067900716 shutdown_connections 2024-04-25T09:14:19.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.354+0000 7f602ec00640 1 --2- 172.21.15.62:0/1067900716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60280b9750 0x7f60280d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.354+0000 7f602ec00640 1 --2- 172.21.15.62:0/1067900716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f602815df40 0x7f60280d02f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.354+0000 7f602ec00640 1 -- 172.21.15.62:0/1067900716 >> 172.21.15.62:0/1067900716 conn(0x7f602806b210 msgr2=0x7f60280668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:19.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.354+0000 7f602ec00640 1 -- 172.21.15.62:0/1067900716 shutdown_connections 2024-04-25T09:14:19.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.354+0000 7f602ec00640 1 -- 172.21.15.62:0/1067900716 wait complete. 2024-04-25T09:14:19.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.355+0000 7f602ec00640 1 Processor -- start 2024-04-25T09:14:19.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.356+0000 7f602ec00640 1 -- start start 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ec00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60280b9750 0x7f60280cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602815db60 0x7f60281f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f602815df40 0x7f60281f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60280d3890 con 0x7f602815db60 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ec00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60280d3710 con 0x7f60280b9750 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60280b6c10 con 0x7f602815df40 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602815db60 0x7f60281f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602815db60 0x7f60281f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56698/0 (socket says 172.21.15.62:56698) 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ce00640 1 -- 172.21.15.62:0/63207864 learned_addr learned my addr 172.21.15.62:0/63207864 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602d800640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60280b9750 0x7f60280cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602e200640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f602815df40 0x7f60281f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:19.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ce00640 1 -- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f602815df40 msgr2=0x7f60281f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:19.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ce00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f602815df40 0x7f60281f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ce00640 1 -- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60280b9750 msgr2=0x7f60280cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:19.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ce00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60280b9750 0x7f60280cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.357+0000 7f602ce00640 1 -- 172.21.15.62:0/63207864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028158f00 con 0x7f602815db60 2024-04-25T09:14:19.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.358+0000 7f602ce00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602815db60 0x7f60281f64d0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f601c009d90 tx=0x7f601c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:19.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.358+0000 7f6025600640 1 -- 172.21.15.62:0/63207864 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c03f070 con 0x7f602815db60 2024-04-25T09:14:19.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.358+0000 7f6025600640 1 -- 172.21.15.62:0/63207864 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c031c40 con 0x7f602815db60 2024-04-25T09:14:19.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.358+0000 7f6025600640 1 -- 172.21.15.62:0/63207864 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c009450 con 0x7f602815db60 2024-04-25T09:14:19.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.358+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60281591e0 con 0x7f602815db60 2024-04-25T09:14:19.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.358+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60281597a0 con 0x7f602815db60 2024-04-25T09:14:19.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.359+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028072910 con 0x7f602815db60 2024-04-25T09:14:19.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.361+0000 7f6025600640 1 -- 172.21.15.62:0/63207864 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f601c044460 con 0x7f602815db60 2024-04-25T09:14:19.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.362+0000 7f6025600640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f600c06b4d0 0x7f600c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:19.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.362+0000 7f6025600640 1 -- 172.21.15.62:0/63207864 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f601c0aba90 con 0x7f602815db60 2024-04-25T09:14:19.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.362+0000 7f602d800640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f600c06b4d0 0x7f600c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:19.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.363+0000 7f602d800640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f600c06b4d0 0x7f600c06d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6018005880 tx=0x7f6018009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.366+0000 7f6025600640 1 -- 172.21.15.62:0/63207864 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f601c07aa80 con 0x7f602815db60 2024-04-25T09:14:19.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.628+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f60280750e0 con 0x7f602815db60 2024-04-25T09:14:19.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.629+0000 7f6025600640 1 -- 172.21.15.62:0/63207864 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f601c07a380 con 0x7f602815db60 2024-04-25T09:14:19.616 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:19.616 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":686,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076638708,"bytes_used":2597257216,"bytes_avail":770496856064,"bytes_total":773094113280,"write_bytes_sec":12264159,"read_op_per_sec":0,"write_op_per_sec":13},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7800}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T09:14:16.138918+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T09:14:19.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f600c06b4d0 msgr2=0x7f600c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:19.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f600c06b4d0 0x7f600c06d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6018005880 tx=0x7f6018009040 comp rx=0 tx=0).stop 2024-04-25T09:14:19.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602815db60 msgr2=0x7f60281f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602815db60 0x7f60281f64d0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f601c009d90 tx=0x7f601c004a40 comp rx=0 tx=0).stop 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 shutdown_connections 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f600c06b4d0 0x7f600c06d990 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f602815db60 0x7f60281f64d0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f602815df40 0x7f60281f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 --2- 172.21.15.62:0/63207864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60280b9750 0x7f60280cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 >> 172.21.15.62:0/63207864 conn(0x7f602806b210 msgr2=0x7f60280cbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 shutdown_connections 2024-04-25T09:14:19.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:19.632+0000 7f602ec00640 1 -- 172.21.15.62:0/63207864 wait complete. 2024-04-25T09:14:19.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T09:14:20.637 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:20.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.871+0000 7fc7a1e00640 1 Processor -- start 2024-04-25T09:14:20.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.871+0000 7fc7a1e00640 1 -- start start 2024-04-25T09:14:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.872+0000 7fc7a1e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc79c06d900 0x7fc79c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.872+0000 7fc7a1e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c15faa0 con 0x7fc79c0cee40 2024-04-25T09:14:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.872+0000 7fc7a1e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c15fd80 con 0x7fc79c144c10 2024-04-25T09:14:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.872+0000 7fc7a1e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79c144ff0 con 0x7fc79c06d900 2024-04-25T09:14:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.873+0000 7fc7a1400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc79c0cee40 0x7fc79c0b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:43400/0 (socket says 172.21.15.62:43400) 2024-04-25T09:14:20.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.873+0000 7fc7a1400640 1 -- 172.21.15.62:0/2376458165 learned_addr learned my addr 172.21.15.62:0/2376458165 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.873+0000 7fc79be00640 1 --2- 172.21.15.62:0/2376458165 >> v2:172.21.15.121:3300/0 conn(0x7fc79c06d900 0x7fc79c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:20.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.873+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873929163 0 0) 0x7fc79c15faa0 con 0x7fc79c0cee40 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.873+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc784003520 con 0x7fc79c0cee40 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3785504106 0 0) 0x7fc784003520 con 0x7fc79c0cee40 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 >> v2:172.21.15.121:3300/0 conn(0x7fc79c06d900 msgr2=0x7fc79c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 --2- 172.21.15.62:0/2376458165 >> v2:172.21.15.121:3300/0 conn(0x7fc79c06d900 0x7fc79c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 >> v1:172.21.15.121:6789/0 conn(0x7fc79c144c10 legacy=0x7fc79c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c160060 con 0x7fc79c0cee40 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc7940019f0 con 0x7fc79c0cee40 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 >> v1:172.21.15.62:6789/0 conn(0x7fc79c0cee40 legacy=0x7fc79c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784003a00 0x7fc784003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc784004800 0x7fc784004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc784005210 0x7fc78400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c144ff0 con 0x7fc784004800 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78400c160 con 0x7fc784003a00 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78400c360 con 0x7fc784005210 2024-04-25T09:14:20.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.874+0000 7fc79be00640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc784004800 0x7fc784004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.875+0000 7fc7a0a00640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784003a00 0x7fc784003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.875+0000 7fc79be00640 1 -- 172.21.15.62:0/2376458165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc784005210 msgr2=0x7fc78400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.875+0000 7fc79be00640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc784005210 0x7fc78400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.875+0000 7fc79be00640 1 -- 172.21.15.62:0/2376458165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784003a00 msgr2=0x7fc784003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.875+0000 7fc79be00640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784003a00 0x7fc784003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.875+0000 7fc79be00640 1 -- 172.21.15.62:0/2376458165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784003520 con 0x7fc784004800 2024-04-25T09:14:20.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.875+0000 7fc7a1400640 1 -- 172.21.15.62:0/2376458165 reap_dead start 2024-04-25T09:14:20.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.876+0000 7fc79be00640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc784004800 0x7fc784004c80 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fc788009a90 tx=0x7fc784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f7f4d5e7f90da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.876+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788036070 con 0x7fc784004800 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.876+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788031400 con 0x7fc784004800 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.876+0000 7fc79b400640 1 -- 172.21.15.62:0/2376458165 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788009690 con 0x7fc784004800 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.877+0000 7fc7a1e00640 1 -- 172.21.15.62:0/2376458165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc784004800 msgr2=0x7fc784004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.877+0000 7fc7a1e00640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc784004800 0x7fc784004c80 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fc788009a90 tx=0x7fc784005830 comp rx=0 tx=0).stop 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.877+0000 7fc7a1e00640 1 -- 172.21.15.62:0/2376458165 shutdown_connections 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.877+0000 7fc7a1e00640 1 --2- 172.21.15.62:0/2376458165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc784004800 0x7fc784004c80 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.877+0000 7fc7a1e00640 1 -- 172.21.15.62:0/2376458165 >> 172.21.15.62:0/2376458165 conn(0x7fc79c0b14d0 msgr2=0x7fc79c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.877+0000 7fc7a1e00640 1 -- 172.21.15.62:0/2376458165 shutdown_connections 2024-04-25T09:14:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.877+0000 7fc7a1e00640 1 -- 172.21.15.62:0/2376458165 wait complete. 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.879+0000 7fc7a1e00640 1 Processor -- start 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.879+0000 7fc7a1e00640 1 -- start start 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc7a1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784004800 0x7fc79c1e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc7a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c140f10 0x7fc79c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc7a1e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc79c15d8f0 0x7fc79c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc7a1e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0b41c0 con 0x7fc79c140f10 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc7a1e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c160340 con 0x7fc784004800 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc7a1e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c15f450 con 0x7fc79c15d8f0 2024-04-25T09:14:20.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc79be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c140f10 0x7fc79c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc79be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c140f10 0x7fc79c14dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56720/0 (socket says 172.21.15.62:56720) 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.880+0000 7fc79be00640 1 -- 172.21.15.62:0/3061068609 learned_addr learned my addr 172.21.15.62:0/3061068609 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc7a1400640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc79c15d8f0 0x7fc79c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc79be00640 1 -- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc79c15d8f0 msgr2=0x7fc79c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc79be00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc79c15d8f0 0x7fc79c15dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc7a0a00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784004800 0x7fc79c1e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc79be00640 1 -- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784004800 msgr2=0x7fc79c1e67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc79be00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784004800 0x7fc79c1e67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc79be00640 1 -- 172.21.15.62:0/3061068609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c141390 con 0x7fc79c140f10 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc7a1400640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc79c15d8f0 0x7fc79c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc79be00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c140f10 0x7fc79c14dfc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc788000c00 tx=0x7fc788004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc798c00640 1 -- 172.21.15.62:0/3061068609 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78803f030 con 0x7fc79c140f10 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.881+0000 7fc798c00640 1 -- 172.21.15.62:0/3061068609 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788037040 con 0x7fc79c140f10 2024-04-25T09:14:20.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.882+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c141610 con 0x7fc79c140f10 2024-04-25T09:14:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.882+0000 7fc798c00640 1 -- 172.21.15.62:0/3061068609 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788036070 con 0x7fc79c140f10 2024-04-25T09:14:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.882+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c141bd0 con 0x7fc79c140f10 2024-04-25T09:14:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.883+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79c15b580 con 0x7fc79c140f10 2024-04-25T09:14:20.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.884+0000 7fc798c00640 1 -- 172.21.15.62:0/3061068609 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc788042070 con 0x7fc79c140f10 2024-04-25T09:14:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.885+0000 7fc798c00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77c06b4d0 0x7fc77c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.885+0000 7fc798c00640 1 -- 172.21.15.62:0/3061068609 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc7880a9880 con 0x7fc79c140f10 2024-04-25T09:14:20.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.885+0000 7fc7a0a00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77c06b4d0 0x7fc77c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:20.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.886+0000 7fc7a0a00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77c06b4d0 0x7fc77c06d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc790001550 tx=0x7fc790009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:20.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:20.889+0000 7fc798c00640 1 -- 172.21.15.62:0/3061068609 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc79c141390 con 0x7fc79c140f10 2024-04-25T09:14:21.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.119+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc79c04ef20 con 0x7fc79c140f10 2024-04-25T09:14:21.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.121+0000 7fc798c00640 1 -- 172.21.15.62:0/3061068609 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc7880044e0 con 0x7fc79c140f10 2024-04-25T09:14:21.108 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:21.108 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:21.109 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77c06b4d0 msgr2=0x7fc77c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77c06b4d0 0x7fc77c06d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc790001550 tx=0x7fc790009040 comp rx=0 tx=0).stop 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c140f10 msgr2=0x7fc79c14dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c140f10 0x7fc79c14dfc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc788000c00 tx=0x7fc788004730 comp rx=0 tx=0).stop 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 shutdown_connections 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc77c06b4d0 0x7fc77c06d990 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc79c140f10 0x7fc79c14dfc0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc79c15d8f0 0x7fc79c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 --2- 172.21.15.62:0/3061068609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc784004800 0x7fc79c1e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 >> 172.21.15.62:0/3061068609 conn(0x7fc79c0b14d0 msgr2=0x7fc79c13f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:21.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 shutdown_connections 2024-04-25T09:14:21.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.123+0000 7fc7a1e00640 1 -- 172.21.15.62:0/3061068609 wait complete. 2024-04-25T09:14:21.129 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:14:21.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:14:21.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.347+0000 7f1366800640 1 Processor -- start 2024-04-25T09:14:21.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.347+0000 7f1366800640 1 -- start start 2024-04-25T09:14:21.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.348+0000 7f1366800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f136015e210 0x7f13600d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:21.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.348+0000 7f1366800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f13600ce5a0 0x7f13600d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:21.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.348+0000 7f1366800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13600cee20 con 0x7f136015de30 2024-04-25T09:14:21.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.348+0000 7f1366800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13600cac60 con 0x7f13600ce5a0 2024-04-25T09:14:21.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.348+0000 7f1366800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13600d2180 con 0x7f136015e210 2024-04-25T09:14:21.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f135fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f13600ce5a0 0x7f13600d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f135fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f13600ce5a0 0x7f13600d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56728/0 (socket says 172.21.15.62:56728) 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f135fe00640 1 -- 172.21.15.62:0/3976163287 learned_addr learned my addr 172.21.15.62:0/3976163287 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f135fe00640 1 -- 172.21.15.62:0/3976163287 >> v2:172.21.15.121:3300/0 conn(0x7f136015e210 msgr2=0x7f13600d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f1356a00640 1 --2- 172.21.15.62:0/3976163287 >> v2:172.21.15.121:3300/0 conn(0x7f136015e210 0x7f13600d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f135fe00640 1 --2- 172.21.15.62:0/3976163287 >> v2:172.21.15.121:3300/0 conn(0x7f136015e210 0x7f13600d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f135fe00640 1 -- 172.21.15.62:0/3976163287 >> v1:172.21.15.121:6789/0 conn(0x7f136015de30 legacy=0x7f13601624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f135fe00640 1 -- 172.21.15.62:0/3976163287 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13600cf100 con 0x7f13600ce5a0 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.349+0000 7f1356a00640 1 --2- 172.21.15.62:0/3976163287 >> v2:172.21.15.121:3300/0 conn(0x7f136015e210 0x7f13600d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:21.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.350+0000 7f135fe00640 1 --2- 172.21.15.62:0/3976163287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600d0c90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f136008c360 tx=0x7f135802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569c09d5622477a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f135ea00640 1 -- 172.21.15.62:0/3976163287 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358036070 con 0x7f13600ce5a0 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f135ea00640 1 -- 172.21.15.62:0/3976163287 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1358031600 con 0x7f13600ce5a0 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f135ea00640 1 -- 172.21.15.62:0/3976163287 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13580319d0 con 0x7f13600ce5a0 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f1366800640 1 -- 172.21.15.62:0/3976163287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 msgr2=0x7f13600d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f1366800640 1 --2- 172.21.15.62:0/3976163287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600d0c90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f136008c360 tx=0x7f135802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f1366800640 1 -- 172.21.15.62:0/3976163287 shutdown_connections 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f1366800640 1 --2- 172.21.15.62:0/3976163287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600d0c90 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f1366800640 1 --2- 172.21.15.62:0/3976163287 >> v2:172.21.15.121:3300/0 conn(0x7f136015e210 0x7f13600d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f1366800640 1 -- 172.21.15.62:0/3976163287 >> 172.21.15.62:0/3976163287 conn(0x7f13600668f0 msgr2=0x7f1360066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.351+0000 7f1366800640 1 -- 172.21.15.62:0/3976163287 shutdown_connections 2024-04-25T09:14:21.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.352+0000 7f1366800640 1 -- 172.21.15.62:0/3976163287 wait complete. 2024-04-25T09:14:21.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.353+0000 7f1366800640 1 Processor -- start 2024-04-25T09:14:21.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.353+0000 7f1366800640 1 -- start start 2024-04-25T09:14:21.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.354+0000 7f1366800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:21.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.354+0000 7f1366800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f136015de30 0x7f13600b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:21.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.354+0000 7f1366800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f136015e210 0x7f13600b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:21.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.354+0000 7f1366800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13600b6f70 con 0x7f13600ce5a0 2024-04-25T09:14:21.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.354+0000 7f1366800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13600d23d0 con 0x7f136015e210 2024-04-25T09:14:21.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.354+0000 7f1366800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13600d28f0 con 0x7f136015de30 2024-04-25T09:14:21.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:21.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56742/0 (socket says 172.21.15.62:56742) 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 -- 172.21.15.62:0/2233523536 learned_addr learned my addr 172.21.15.62:0/2233523536 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 -- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f136015de30 msgr2=0x7f13600b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f136015de30 0x7f13600b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 -- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f136015e210 msgr2=0x7f13600b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f136015e210 0x7f13600b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 -- 172.21.15.62:0/2233523536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13600cd130 con 0x7f13600ce5a0 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.355+0000 7f135f400640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600cc830 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f135000c430 tx=0x7f135000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.356+0000 7f1357e00640 1 -- 172.21.15.62:0/2233523536 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1350011790 con 0x7f13600ce5a0 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.356+0000 7f1357e00640 1 -- 172.21.15.62:0/2233523536 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1350017070 con 0x7f13600ce5a0 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.356+0000 7f1357e00640 1 -- 172.21.15.62:0/2233523536 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1350011bb0 con 0x7f13600ce5a0 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.356+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13600cd410 con 0x7f13600ce5a0 2024-04-25T09:14:21.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.356+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13600cdcb0 con 0x7f13600ce5a0 2024-04-25T09:14:21.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.357+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136004ef20 con 0x7f13600ce5a0 2024-04-25T09:14:21.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.358+0000 7f1357e00640 1 -- 172.21.15.62:0/2233523536 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f135001c460 con 0x7f13600ce5a0 2024-04-25T09:14:21.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.359+0000 7f1357e00640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f133c06b4d0 0x7f133c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:21.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.359+0000 7f1357e00640 1 -- 172.21.15.62:0/2233523536 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f135008a1a0 con 0x7f13600ce5a0 2024-04-25T09:14:21.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.360+0000 7f1356a00640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f133c06b4d0 0x7f133c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:21.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.361+0000 7f1356a00640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f133c06b4d0 0x7f133c06d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1348006fd0 tx=0x7f1348008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:21.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.363+0000 7f1357e00640 1 -- 172.21.15.62:0/2233523536 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1350058f40 con 0x7f13600ce5a0 2024-04-25T09:14:21.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.641+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f13600d1a50 con 0x7f13600ce5a0 2024-04-25T09:14:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.642+0000 7f1357e00640 1 -- 172.21.15.62:0/2233523536 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f1350058840 con 0x7f13600ce5a0 2024-04-25T09:14:21.630 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:21.630 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":688,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076638708,"bytes_used":2597257216,"bytes_avail":770496856064,"bytes_total":773094113280,"write_bytes_sec":12264531,"read_op_per_sec":0,"write_op_per_sec":13},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7800}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T09:14:16.138918+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T09:14:21.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f133c06b4d0 msgr2=0x7f133c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:21.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f133c06b4d0 0x7f133c06d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1348006fd0 tx=0x7f1348008040 comp rx=0 tx=0).stop 2024-04-25T09:14:21.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 msgr2=0x7f13600cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:21.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600cc830 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f135000c430 tx=0x7f135000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 shutdown_connections 2024-04-25T09:14:21.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f133c06b4d0 0x7f133c06d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f136015e210 0x7f13600b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f136015de30 0x7f13600b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.645+0000 7f1366800640 1 --2- 172.21.15.62:0/2233523536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13600ce5a0 0x7f13600cc830 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:21.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.646+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 >> 172.21.15.62:0/2233523536 conn(0x7f13600668f0 msgr2=0x7f1360160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:21.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.646+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 shutdown_connections 2024-04-25T09:14:21.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:21.646+0000 7f1366800640 1 -- 172.21.15.62:0/2233523536 wait complete. 2024-04-25T09:14:21.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T09:14:22.654 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:22.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.889+0000 7ffabde00640 1 Processor -- start 2024-04-25T09:14:22.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.889+0000 7ffabde00640 1 -- start start 2024-04-25T09:14:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffabde00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffab8144c10 0x7ffab815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffabde00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffab80cee40 0x7ffab806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffabde00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab80cf2d0 con 0x7ffab806d730 2024-04-25T09:14:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffabde00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab8144ff0 con 0x7ffab80cee40 2024-04-25T09:14:22.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffabde00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab806dba0 con 0x7ffab8144c10 2024-04-25T09:14:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffaafe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffab80cee40 0x7ffab806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffaafe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ffab80cee40 0x7ffab806d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56746/0 (socket says 172.21.15.62:56746) 2024-04-25T09:14:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffaafe00640 1 -- 172.21.15.62:0/2892035708 learned_addr learned my addr 172.21.15.62:0/2892035708 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffaafe00640 1 -- 172.21.15.62:0/2892035708 >> v2:172.21.15.121:3300/0 conn(0x7ffab8144c10 msgr2=0x7ffab815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.891+0000 7ffaafe00640 1 --2- 172.21.15.62:0/2892035708 >> v2:172.21.15.121:3300/0 conn(0x7ffab8144c10 0x7ffab815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.892+0000 7ffaafe00640 1 -- 172.21.15.62:0/2892035708 >> v1:172.21.15.121:6789/0 conn(0x7ffab806d730 legacy=0x7ffab80b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:22.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.892+0000 7ffaafe00640 1 -- 172.21.15.62:0/2892035708 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab815faa0 con 0x7ffab80cee40 2024-04-25T09:14:22.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.893+0000 7ffaafe00640 1 --2- 172.21.15.62:0/2892035708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab806d1f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ffab80724a0 tx=0x7ffaa002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b29a9dd0bf98b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:22.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.893+0000 7ffab7e00640 1 -- 172.21.15.62:0/2892035708 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0037070 con 0x7ffab80cee40 2024-04-25T09:14:22.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.893+0000 7ffab7e00640 1 -- 172.21.15.62:0/2892035708 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa0032440 con 0x7ffab80cee40 2024-04-25T09:14:22.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.893+0000 7ffab7e00640 1 -- 172.21.15.62:0/2892035708 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0032810 con 0x7ffab80cee40 2024-04-25T09:14:22.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.894+0000 7ffabde00640 1 -- 172.21.15.62:0/2892035708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 msgr2=0x7ffab806d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:22.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.894+0000 7ffabde00640 1 --2- 172.21.15.62:0/2892035708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab806d1f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ffab80724a0 tx=0x7ffaa002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:14:22.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.894+0000 7ffabde00640 1 -- 172.21.15.62:0/2892035708 shutdown_connections 2024-04-25T09:14:22.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.894+0000 7ffabde00640 1 --2- 172.21.15.62:0/2892035708 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab806d1f0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.894+0000 7ffabde00640 1 --2- 172.21.15.62:0/2892035708 >> v2:172.21.15.121:3300/0 conn(0x7ffab8144c10 0x7ffab815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.894+0000 7ffabde00640 1 -- 172.21.15.62:0/2892035708 >> 172.21.15.62:0/2892035708 conn(0x7ffab80b14d0 msgr2=0x7ffab80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.894+0000 7ffabde00640 1 -- 172.21.15.62:0/2892035708 shutdown_connections 2024-04-25T09:14:22.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.894+0000 7ffabde00640 1 -- 172.21.15.62:0/2892035708 wait complete. 2024-04-25T09:14:22.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.896+0000 7ffabde00640 1 Processor -- start 2024-04-25T09:14:22.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.896+0000 7ffabde00640 1 -- start start 2024-04-25T09:14:22.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.897+0000 7ffabde00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffab806d730 0x7ffab8142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.897+0000 7ffabde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab8140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.897+0000 7ffabde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab8144c10 0x7ffab8141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.897+0000 7ffabde00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab80b41c0 con 0x7ffab80cee40 2024-04-25T09:14:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.897+0000 7ffabde00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8160340 con 0x7ffab8144c10 2024-04-25T09:14:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.897+0000 7ffabde00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab8160060 con 0x7ffab806d730 2024-04-25T09:14:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.897+0000 7ffaafe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab8140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffaafe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab8140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56758/0 (socket says 172.21.15.62:56758) 2024-04-25T09:14:22.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffaafe00640 1 -- 172.21.15.62:0/1973830046 learned_addr learned my addr 172.21.15.62:0/1973830046 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:22.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffabd400640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab8144c10 0x7ffab8141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:22.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffabca00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffab806d730 0x7ffab8142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:22.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffaafe00640 1 -- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffab806d730 msgr2=0x7ffab8142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:22.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffaafe00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffab806d730 0x7ffab8142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffaafe00640 1 -- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab8144c10 msgr2=0x7ffab8141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffaafe00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab8144c10 0x7ffab8141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffaafe00640 1 -- 172.21.15.62:0/1973830046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab80b1110 con 0x7ffab80cee40 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.898+0000 7ffaafe00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab8140e10 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0000c00 tx=0x7ffaa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.899+0000 7ffab5600640 1 -- 172.21.15.62:0/1973830046 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0041070 con 0x7ffab80cee40 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.899+0000 7ffab5600640 1 -- 172.21.15.62:0/1973830046 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa003c870 con 0x7ffab80cee40 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.899+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab8143040 con 0x7ffab80cee40 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.899+0000 7ffab5600640 1 -- 172.21.15.62:0/1973830046 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0037040 con 0x7ffab80cee40 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.899+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab8141d40 con 0x7ffab80cee40 2024-04-25T09:14:22.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.900+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab804ef20 con 0x7ffab80cee40 2024-04-25T09:14:22.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.901+0000 7ffab5600640 1 -- 172.21.15.62:0/1973830046 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaa0044070 con 0x7ffab80cee40 2024-04-25T09:14:22.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.902+0000 7ffab5600640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa9406b4d0 0x7ffa9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:22.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.902+0000 7ffab5600640 1 -- 172.21.15.62:0/1973830046 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ffaa00aad50 con 0x7ffab80cee40 2024-04-25T09:14:22.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.903+0000 7ffabca00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa9406b4d0 0x7ffa9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:22.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.904+0000 7ffabca00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa9406b4d0 0x7ffa9406d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800b810 tx=0x7ffaa8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:22.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:22.907+0000 7ffab5600640 1 -- 172.21.15.62:0/1973830046 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffaa00ae030 con 0x7ffab80cee40 2024-04-25T09:14:23.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.150+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffab80b22b0 con 0x7ffab80cee40 2024-04-25T09:14:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.150+0000 7ffab5600640 1 -- 172.21.15.62:0/1973830046 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ffaa0079d40 con 0x7ffab80cee40 2024-04-25T09:14:23.138 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:23.138 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:23.138 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa9406b4d0 msgr2=0x7ffa9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa9406b4d0 0x7ffa9406d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800b810 tx=0x7ffaa8008040 comp rx=0 tx=0).stop 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 msgr2=0x7ffab8140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab8140e10 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0000c00 tx=0x7ffaa00046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 shutdown_connections 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ffa9406b4d0 0x7ffa9406d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ffab80cee40 0x7ffab8140e10 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab8144c10 0x7ffab8141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 --2- 172.21.15.62:0/1973830046 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ffab806d730 0x7ffab8142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 >> 172.21.15.62:0/1973830046 conn(0x7ffab80b14d0 msgr2=0x7ffab80b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:23.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 shutdown_connections 2024-04-25T09:14:23.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.153+0000 7ffabde00640 1 -- 172.21.15.62:0/1973830046 wait complete. 2024-04-25T09:14:23.160 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T09:14:23.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:14:23.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.387+0000 7fd6a8200640 1 Processor -- start 2024-04-25T09:14:23.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.387+0000 7fd6a8200640 1 -- start start 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a8200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6a00d0680 0x7fd6a0072880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a8200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a0070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a8200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a01436b0 con 0x7fd6a0146f50 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a8200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a0147810 con 0x7fd6a00d0680 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a8200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0143990 con 0x7fd6a00ad8e0 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a5800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6a00d0680 0x7fd6a0072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a5800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6a00d0680 0x7fd6a0072880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56762/0 (socket says 172.21.15.62:56762) 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a5800640 1 -- 172.21.15.62:0/677140011 learned_addr learned my addr 172.21.15.62:0/677140011 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a4e00640 1 --2- 172.21.15.62:0/677140011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a0070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.389+0000 7fd6a5800640 1 -- 172.21.15.62:0/677140011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 msgr2=0x7fd6a0070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:23.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.390+0000 7fd6a5800640 1 --2- 172.21.15.62:0/677140011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a0070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.390+0000 7fd6a5800640 1 -- 172.21.15.62:0/677140011 >> v1:172.21.15.121:6789/0 conn(0x7fd6a0146f50 legacy=0x7fd6a006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:23.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.390+0000 7fd6a5800640 1 -- 172.21.15.62:0/677140011 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a0143c70 con 0x7fd6a00d0680 2024-04-25T09:14:23.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.390+0000 7fd6a4e00640 1 --2- 172.21.15.62:0/677140011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a0070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:23.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.391+0000 7fd6a5800640 1 --2- 172.21.15.62:0/677140011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a00d0680 0x7fd6a0072880 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd694009870 tx=0x7fd69402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a13594f3056805 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:23.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.391+0000 7fd69fe00640 1 -- 172.21.15.62:0/677140011 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694037070 con 0x7fd6a00d0680 2024-04-25T09:14:23.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.391+0000 7fd69fe00640 1 -- 172.21.15.62:0/677140011 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd694032440 con 0x7fd6a00d0680 2024-04-25T09:14:23.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.391+0000 7fd69fe00640 1 -- 172.21.15.62:0/677140011 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694032810 con 0x7fd6a00d0680 2024-04-25T09:14:23.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.392+0000 7fd6a8200640 1 -- 172.21.15.62:0/677140011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a00d0680 msgr2=0x7fd6a0072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:23.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.392+0000 7fd6a8200640 1 --2- 172.21.15.62:0/677140011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a00d0680 0x7fd6a0072880 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd694009870 tx=0x7fd69402a180 comp rx=0 tx=0).stop 2024-04-25T09:14:23.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.392+0000 7fd6a8200640 1 -- 172.21.15.62:0/677140011 shutdown_connections 2024-04-25T09:14:23.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.392+0000 7fd6a8200640 1 --2- 172.21.15.62:0/677140011 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a0070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.392+0000 7fd6a8200640 1 --2- 172.21.15.62:0/677140011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a00d0680 0x7fd6a0072880 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.392+0000 7fd6a8200640 1 -- 172.21.15.62:0/677140011 >> 172.21.15.62:0/677140011 conn(0x7fd6a006b520 msgr2=0x7fd6a00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:23.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.392+0000 7fd6a8200640 1 -- 172.21.15.62:0/677140011 shutdown_connections 2024-04-25T09:14:23.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.392+0000 7fd6a8200640 1 -- 172.21.15.62:0/677140011 wait complete. 2024-04-25T09:14:23.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.394+0000 7fd6a8200640 1 Processor -- start 2024-04-25T09:14:23.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.394+0000 7fd6a8200640 1 -- start start 2024-04-25T09:14:23.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a8200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a013f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:23.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a8200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6a00d0680 0x7fd6a0160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:23.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a8200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a0146f50 0x7fd6a0144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:23.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a8200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a00710d0 con 0x7fd6a0146f50 2024-04-25T09:14:23.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a8200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0070f50 con 0x7fd6a00d0680 2024-04-25T09:14:23.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a8200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0071250 con 0x7fd6a00ad8e0 2024-04-25T09:14:23.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a0146f50 0x7fd6a0144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:23.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a0146f50 0x7fd6a0144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56776/0 (socket says 172.21.15.62:56776) 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a6200640 1 -- 172.21.15.62:0/2652783702 learned_addr learned my addr 172.21.15.62:0/2652783702 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.395+0000 7fd6a5800640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a013f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd6a4e00640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6a00d0680 0x7fd6a0160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd6a6200640 1 -- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 msgr2=0x7fd6a013f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd6a6200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a013f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd6a6200640 1 -- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6a00d0680 msgr2=0x7fd6a0160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd6a6200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6a00d0680 0x7fd6a0160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd6a6200640 1 -- 172.21.15.62:0/2652783702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a0161190 con 0x7fd6a0146f50 2024-04-25T09:14:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd6a6200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a0146f50 0x7fd6a0144860 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd69800b970 tx=0x7fd69800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:23.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd69d600640 1 -- 172.21.15.62:0/2652783702 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698019070 con 0x7fd6a0146f50 2024-04-25T09:14:23.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd69d600640 1 -- 172.21.15.62:0/2652783702 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69800cb40 con 0x7fd6a0146f50 2024-04-25T09:14:23.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd69d600640 1 -- 172.21.15.62:0/2652783702 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698014440 con 0x7fd6a0146f50 2024-04-25T09:14:23.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.396+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a01450b0 con 0x7fd6a0146f50 2024-04-25T09:14:23.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.397+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a0145670 con 0x7fd6a0146f50 2024-04-25T09:14:23.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.398+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a00cfd00 con 0x7fd6a0146f50 2024-04-25T09:14:23.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.399+0000 7fd69d600640 1 -- 172.21.15.62:0/2652783702 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd69801c070 con 0x7fd6a0146f50 2024-04-25T09:14:23.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.400+0000 7fd69d600640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd68006b4d0 0x7fd68006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:23.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.400+0000 7fd69d600640 1 -- 172.21.15.62:0/2652783702 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd698089fc0 con 0x7fd6a0146f50 2024-04-25T09:14:23.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.400+0000 7fd6a5800640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd68006b4d0 0x7fd68006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:23.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.402+0000 7fd6a5800640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd68006b4d0 0x7fd68006d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd694034000 tx=0x7fd69402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:23.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.404+0000 7fd69d600640 1 -- 172.21.15.62:0/2652783702 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6a00cfd00 con 0x7fd6a0146f50 2024-04-25T09:14:23.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.673+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd6a004ef20 con 0x7fd6a0146f50 2024-04-25T09:14:23.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.674+0000 7fd69d600640 1 -- 172.21.15.62:0/2652783702 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7fd698058cd0 con 0x7fd6a0146f50 2024-04-25T09:14:23.661 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:23.661 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":690,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076992595,"bytes_used":2597380096,"bytes_avail":770496733184,"bytes_total":773094113280,"write_bytes_sec":12294228,"read_op_per_sec":0,"write_op_per_sec":13},"fsmap":{"epoch":71,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":7800}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T09:14:22.141169+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:14:23.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.676+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd68006b4d0 msgr2=0x7fd68006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:23.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.676+0000 7fd6a8200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd68006b4d0 0x7fd68006d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd694034000 tx=0x7fd69402e000 comp rx=0 tx=0).stop 2024-04-25T09:14:23.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.676+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a0146f50 msgr2=0x7fd6a0144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:23.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.676+0000 7fd6a8200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a0146f50 0x7fd6a0144860 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd69800b970 tx=0x7fd69800be30 comp rx=0 tx=0).stop 2024-04-25T09:14:23.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.677+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 shutdown_connections 2024-04-25T09:14:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.677+0000 7fd6a8200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd68006b4d0 0x7fd68006d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.677+0000 7fd6a8200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6a0146f50 0x7fd6a0144860 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.677+0000 7fd6a8200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6a00d0680 0x7fd6a0160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.677+0000 7fd6a8200640 1 --2- 172.21.15.62:0/2652783702 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6a00ad8e0 0x7fd6a013f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.677+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 >> 172.21.15.62:0/2652783702 conn(0x7fd6a006b520 msgr2=0x7fd6a00b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.677+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 shutdown_connections 2024-04-25T09:14:23.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:23.677+0000 7fd6a8200640 1 -- 172.21.15.62:0/2652783702 wait complete. 2024-04-25T09:14:23.681 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:14:23.682 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:14:23.682 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:14:23.771 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:14:23.883 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:14:23.884 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:14:23.884 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:23.944 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:14:23.945 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:23.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.225+0000 7f0535400640 1 Processor -- start 2024-04-25T09:14:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.226+0000 7f0535400640 1 -- start start 2024-04-25T09:14:24.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.227+0000 7f0535400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f05300b6a90 0x7f05300d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.227+0000 7f0535400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05300d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.227+0000 7f0535400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300d3110 con 0x7f05300b56f0 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.227+0000 7f0535400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300cedc0 con 0x7f05300b6a90 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.227+0000 7f0535400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300d33f0 con 0x7f05300b27a0 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.227+0000 7f052fe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f05300b56f0 0x7f0530160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:43408/0 (socket says 172.21.15.62:43408) 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f052fe00640 1 -- 172.21.15.62:0/1323496943 learned_addr learned my addr 172.21.15.62:0/1323496943 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f052f400640 1 --2- 172.21.15.62:0/1323496943 >> v2:172.21.15.62:3300/0 conn(0x7f05300b6a90 0x7f05300d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f0534a00640 1 --2- 172.21.15.62:0/1323496943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05300d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f052f400640 1 -- 172.21.15.62:0/1323496943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 msgr2=0x7f05300d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f052ea00640 1 -- 172.21.15.62:0/1323496943 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359707489 0 0) 0x7f05300d3110 con 0x7f05300b56f0 2024-04-25T09:14:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f052f400640 1 --2- 172.21.15.62:0/1323496943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05300d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f052f400640 1 -- 172.21.15.62:0/1323496943 >> v1:172.21.15.62:6789/0 conn(0x7f05300b56f0 legacy=0x7f0530160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f052f400640 1 -- 172.21.15.62:0/1323496943 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300d36d0 con 0x7f05300b6a90 2024-04-25T09:14:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.228+0000 7f0534a00640 1 --2- 172.21.15.62:0/1323496943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05300d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.229+0000 7f052f400640 1 --2- 172.21.15.62:0/1323496943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b6a90 0x7f05300d1fe0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0518009d90 tx=0x7f051802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38bddf6790718b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f052ea00640 1 -- 172.21.15.62:0/1323496943 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518007740 con 0x7f05300b6a90 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f052ea00640 1 -- 172.21.15.62:0/1323496943 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05180078e0 con 0x7f05300b6a90 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f052ea00640 1 -- 172.21.15.62:0/1323496943 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518007cb0 con 0x7f05300b6a90 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f0535400640 1 -- 172.21.15.62:0/1323496943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b6a90 msgr2=0x7f05300d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f0535400640 1 --2- 172.21.15.62:0/1323496943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b6a90 0x7f05300d1fe0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0518009d90 tx=0x7f051802a660 comp rx=0 tx=0).stop 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f0535400640 1 -- 172.21.15.62:0/1323496943 shutdown_connections 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f0535400640 1 --2- 172.21.15.62:0/1323496943 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05300d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f0535400640 1 --2- 172.21.15.62:0/1323496943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b6a90 0x7f05300d1fe0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.230+0000 7f0535400640 1 -- 172.21.15.62:0/1323496943 >> 172.21.15.62:0/1323496943 conn(0x7f0530066d10 msgr2=0x7f0530067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.231+0000 7f0535400640 1 -- 172.21.15.62:0/1323496943 shutdown_connections 2024-04-25T09:14:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.231+0000 7f0535400640 1 -- 172.21.15.62:0/1323496943 wait complete. 2024-04-25T09:14:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.233+0000 7f0535400640 1 Processor -- start 2024-04-25T09:14:24.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.233+0000 7f0535400640 1 -- start start 2024-04-25T09:14:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f0535400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05301532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f0535400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b56f0 0x7f05300ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f0535400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05300b6a90 0x7f05300cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f0535400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b2ee0 con 0x7f05300b56f0 2024-04-25T09:14:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f0535400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b2d60 con 0x7f05300b6a90 2024-04-25T09:14:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f0535400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b5ad0 con 0x7f05300b27a0 2024-04-25T09:14:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f052f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b56f0 0x7f05300ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f052f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b56f0 0x7f05300ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56794/0 (socket says 172.21.15.62:56794) 2024-04-25T09:14:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f052f400640 1 -- 172.21.15.62:0/1086679539 learned_addr learned my addr 172.21.15.62:0/1086679539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.234+0000 7f052fe00640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05301532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f0534a00640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05300b6a90 0x7f05300cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f052f400640 1 -- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 msgr2=0x7f05301532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f052f400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05301532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f052f400640 1 -- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05300b6a90 msgr2=0x7f05300cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f052f400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05300b6a90 0x7f05300cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f052f400640 1 -- 172.21.15.62:0/1086679539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05301537f0 con 0x7f05300b56f0 2024-04-25T09:14:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f0534a00640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05300b6a90 0x7f05300cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f052f400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b56f0 0x7f05300ccca0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f0518009d90 tx=0x7f0518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f0527e00640 1 -- 172.21.15.62:0/1086679539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051803f070 con 0x7f05300b56f0 2024-04-25T09:14:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f0527e00640 1 -- 172.21.15.62:0/1086679539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0518031c40 con 0x7f05300b56f0 2024-04-25T09:14:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05300c8c70 con 0x7f05300b56f0 2024-04-25T09:14:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f0527e00640 1 -- 172.21.15.62:0/1086679539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518043410 con 0x7f05300b56f0 2024-04-25T09:14:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.235+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05300c9230 con 0x7f05300b56f0 2024-04-25T09:14:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.237+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053004ef20 con 0x7f05300b56f0 2024-04-25T09:14:24.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.237+0000 7f0527e00640 1 -- 172.21.15.62:0/1086679539 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05180435b0 con 0x7f05300b56f0 2024-04-25T09:14:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.238+0000 7f0527e00640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.239+0000 7f0527e00640 1 -- 172.21.15.62:0/1086679539 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f05180ab820 con 0x7f05300b56f0 2024-04-25T09:14:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.239+0000 7f052fe00640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.240+0000 7f052fe00640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f050c06b4d0 0x7f050c06d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0520007fd0 tx=0x7f0520009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:24.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.243+0000 7f0527e00640 1 -- 172.21.15.62:0/1086679539 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f051807a650 con 0x7f05300b56f0 2024-04-25T09:14:24.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.439+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05300ce580 con 0x7f05300b56f0 2024-04-25T09:14:24.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.441+0000 7f0527e00640 1 -- 172.21.15.62:0/1086679539 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16703 (secure 0 0 0) 0x7f0518079f50 con 0x7f05300b56f0 2024-04-25T09:14:24.430 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:24.430 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":111,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:02.867113+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:13:29.095930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:13:30.242813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.6","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f050c06b4d0 msgr2=0x7f050c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f050c06b4d0 0x7f050c06d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0520007fd0 tx=0x7f0520009040 comp rx=0 tx=0).stop 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b56f0 msgr2=0x7f05300ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b56f0 0x7f05300ccca0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f0518009d90 tx=0x7f0518004a40 comp rx=0 tx=0).stop 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 shutdown_connections 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f05300b56f0 0x7f05300ccca0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05300b6a90 0x7f05300cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 --2- 172.21.15.62:0/1086679539 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f05300b27a0 0x7f05301532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 >> 172.21.15.62:0/1086679539 conn(0x7f0530066d10 msgr2=0x7f05300b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 shutdown_connections 2024-04-25T09:14:24.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.444+0000 7f0535400640 1 -- 172.21.15.62:0/1086679539 wait complete. 2024-04-25T09:14:24.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:13:29.095930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:13:30.242813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:14:24.452 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:14:24.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.672+0000 7f8914c00640 1 Processor -- start 2024-04-25T09:14:24.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.673+0000 7f8914c00640 1 -- start start 2024-04-25T09:14:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.674+0000 7f8914c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89100ccde0 0x7f89100ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.674+0000 7f8914c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89100ced40 con 0x7f891006d2c0 2024-04-25T09:14:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.674+0000 7f8914c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89100cd260 con 0x7f89100b31a0 2024-04-25T09:14:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f8914c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89100cf000 con 0x7f89100ccde0 2024-04-25T09:14:24.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f890f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f891006d2c0 0x7f891006d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:43416/0 (socket says 172.21.15.62:43416) 2024-04-25T09:14:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f890f400640 1 -- 172.21.15.62:0/3158021383 learned_addr learned my addr 172.21.15.62:0/3158021383 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f8907e00640 1 --2- 172.21.15.62:0/3158021383 >> v2:172.21.15.62:3300/0 conn(0x7f89100ccde0 0x7f89100ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f890ea00640 1 -- 172.21.15.62:0/3158021383 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848978762 0 0) 0x7f89100ced40 con 0x7f891006d2c0 2024-04-25T09:14:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f890ea00640 1 -- 172.21.15.62:0/3158021383 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f4003520 con 0x7f891006d2c0 2024-04-25T09:14:24.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f8907e00640 1 -- 172.21.15.62:0/3158021383 >> v1:172.21.15.121:6789/0 conn(0x7f89100b31a0 legacy=0x7f89100b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f8907e00640 1 -- 172.21.15.62:0/3158021383 >> v1:172.21.15.62:6789/0 conn(0x7f891006d2c0 legacy=0x7f891006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.675+0000 7f8907e00640 1 -- 172.21.15.62:0/3158021383 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100cf2e0 con 0x7f89100ccde0 2024-04-25T09:14:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.677+0000 7f8907e00640 1 --2- 172.21.15.62:0/3158021383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89100ce800 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f88f80097a0 tx=0x7f88f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b03a8bcb0f68c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.677+0000 7f890ea00640 1 -- 172.21.15.62:0/3158021383 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f8037070 con 0x7f89100ccde0 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.677+0000 7f890ea00640 1 -- 172.21.15.62:0/3158021383 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f8032440 con 0x7f89100ccde0 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.677+0000 7f890ea00640 1 -- 172.21.15.62:0/3158021383 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f8032810 con 0x7f89100ccde0 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.677+0000 7f8914c00640 1 -- 172.21.15.62:0/3158021383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 msgr2=0x7f89100ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.677+0000 7f8914c00640 1 --2- 172.21.15.62:0/3158021383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89100ce800 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f88f80097a0 tx=0x7f88f802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.678+0000 7f8914c00640 1 -- 172.21.15.62:0/3158021383 shutdown_connections 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.678+0000 7f8914c00640 1 --2- 172.21.15.62:0/3158021383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89100ce800 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.678+0000 7f8914c00640 1 -- 172.21.15.62:0/3158021383 >> 172.21.15.62:0/3158021383 conn(0x7f8910066f20 msgr2=0x7f89100b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.678+0000 7f8914c00640 1 -- 172.21.15.62:0/3158021383 shutdown_connections 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.678+0000 7f8914c00640 1 -- 172.21.15.62:0/3158021383 wait complete. 2024-04-25T09:14:24.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.679+0000 7f8914c00640 1 Processor -- start 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.680+0000 7f8914c00640 1 -- start start 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f8914c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f891006d2c0 0x7f89101e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f8914c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89100b31a0 0x7f89101e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f8914c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89101e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f8914c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100b2c80 con 0x7f89100ccde0 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f8914c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100b29a0 con 0x7f89100b31a0 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f8914c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89101448d0 con 0x7f891006d2c0 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f890fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89101e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f890fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89101e9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:56808/0 (socket says 172.21.15.62:56808) 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f890fe00640 1 -- 172.21.15.62:0/2815957033 learned_addr learned my addr 172.21.15.62:0/2815957033 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f8907e00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89100b31a0 0x7f89101e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f890f400640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f891006d2c0 0x7f89101e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f890fe00640 1 -- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f891006d2c0 msgr2=0x7f89101e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f890fe00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f891006d2c0 0x7f89101e79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f890fe00640 1 -- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89100b31a0 msgr2=0x7f89101e8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.681+0000 7f890fe00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89100b31a0 0x7f89101e8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.682+0000 7f890fe00640 1 -- 172.21.15.62:0/2815957033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89101e8210 con 0x7f89100ccde0 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.682+0000 7f890f400640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f891006d2c0 0x7f89101e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.682+0000 7f890fe00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89101e9510 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f890800c780 tx=0x7f890800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.682+0000 7f8907400640 1 -- 172.21.15.62:0/2815957033 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908004b60 con 0x7f89100ccde0 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.682+0000 7f8907400640 1 -- 172.21.15.62:0/2815957033 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8908004d00 con 0x7f89100ccde0 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.682+0000 7f8907400640 1 -- 172.21.15.62:0/2815957033 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890800aa90 con 0x7f89100ccde0 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.682+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89101e4310 con 0x7f89100ccde0 2024-04-25T09:14:24.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.682+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89101e7f00 con 0x7f89100ccde0 2024-04-25T09:14:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.683+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891004ef20 con 0x7f89100ccde0 2024-04-25T09:14:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.684+0000 7f8907400640 1 -- 172.21.15.62:0/2815957033 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f890800ac30 con 0x7f89100ccde0 2024-04-25T09:14:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.685+0000 7f8907400640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88ec06b4d0 0x7f88ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.685+0000 7f8907400640 1 -- 172.21.15.62:0/2815957033 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f890808a480 con 0x7f89100ccde0 2024-04-25T09:14:24.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.686+0000 7f890f400640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88ec06b4d0 0x7f88ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:24.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.687+0000 7f890f400640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88ec06b4d0 0x7f88ec06d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8900005880 tx=0x7f8900009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:24.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.690+0000 7f8907400640 1 -- 172.21.15.62:0/2815957033 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8908020070 con 0x7f89100ccde0 2024-04-25T09:14:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.889+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f891006d9e0 con 0x7f89100ccde0 2024-04-25T09:14:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.890+0000 7f8907400640 1 -- 172.21.15.62:0/2815957033 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8908059350 con 0x7f89100ccde0 2024-04-25T09:14:24.878 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:14:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.892+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88ec06b4d0 msgr2=0x7f88ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.892+0000 7f8914c00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88ec06b4d0 0x7f88ec06d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8900005880 tx=0x7f8900009040 comp rx=0 tx=0).stop 2024-04-25T09:14:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 msgr2=0x7f89101e9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89101e9510 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f890800c780 tx=0x7f890800cc40 comp rx=0 tx=0).stop 2024-04-25T09:14:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 shutdown_connections 2024-04-25T09:14:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88ec06b4d0 0x7f88ec06d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89100ccde0 0x7f89101e9510 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89100b31a0 0x7f89101e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 --2- 172.21.15.62:0/2815957033 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f891006d2c0 0x7f89101e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 >> 172.21.15.62:0/2815957033 conn(0x7f8910066f20 msgr2=0x7f891006c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 shutdown_connections 2024-04-25T09:14:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:24.893+0000 7f8914c00640 1 -- 172.21.15.62:0/2815957033 wait complete. 2024-04-25T09:14:24.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:14:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.130+0000 7ff973200640 1 Processor -- start 2024-04-25T09:14:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.130+0000 7ff973200640 1 -- start start 2024-04-25T09:14:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.131+0000 7ff973200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9740d3960 0x7ff9740d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.131+0000 7ff973200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740ce780 con 0x7ff97415dd50 2024-04-25T09:14:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.131+0000 7ff973200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740d42f0 con 0x7ff9740d2fb0 2024-04-25T09:14:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.131+0000 7ff973200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740d3440 con 0x7ff9740d3960 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff971e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff97415dd50 0x7ff9740d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52140/0 (socket says 172.21.15.62:52140) 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff971e00640 1 -- 172.21.15.62:0/1702045350 learned_addr learned my addr 172.21.15.62:0/1702045350 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958086169 0 0) 0x7ff9740ce780 con 0x7ff97415dd50 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff954003520 con 0x7ff97415dd50 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff972800640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9740d3960 0x7ff9740d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1264022145 0 0) 0x7ff954003520 con 0x7ff97415dd50 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9740d3960 msgr2=0x7ff9740d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff970a00640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9740d3960 0x7ff9740d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 >> v1:172.21.15.121:6789/0 conn(0x7ff9740d2fb0 legacy=0x7ff9740ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.132+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740ce1c0 con 0x7ff97415dd50 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ff964001670 con 0x7ff97415dd50 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 >> v1:172.21.15.62:6789/0 conn(0x7ff97415dd50 legacy=0x7ff9740d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff970a00640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9740d3960 0x7ff9540039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff970a00640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff954004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff970a00640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff954004d40 0x7ff9540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740d3440 con 0x7ff954004010 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95400bb80 con 0x7ff954004d40 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95400bd80 con 0x7ff9740d3960 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff971400640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff954004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff971e00640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9740d3960 0x7ff9540039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.133+0000 7ff972800640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff954004d40 0x7ff9540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.134+0000 7ff971400640 1 -- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9740d3960 msgr2=0x7ff9540039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:25.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.134+0000 7ff971400640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff9740d3960 0x7ff9540039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.134+0000 7ff971400640 1 -- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff954004d40 msgr2=0x7ff9540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:25.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.134+0000 7ff971400640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff954004d40 0x7ff9540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.134+0000 7ff971400640 1 -- 172.21.15.62:0/1702045350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff954003520 con 0x7ff954004010 2024-04-25T09:14:25.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.134+0000 7ff972800640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff954004d40 0x7ff9540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.135+0000 7ff971400640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff954004800 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff95c001e50 tx=0x7ff954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834adc63c26cf17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.135+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c001a70 con 0x7ff954004010 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.135+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c003970 con 0x7ff954004010 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.135+0000 7ff970a00640 1 -- 172.21.15.62:0/1702045350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c003d60 con 0x7ff954004010 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.135+0000 7ff973200640 1 -- 172.21.15.62:0/1702045350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 msgr2=0x7ff954004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.135+0000 7ff973200640 1 --2- 172.21.15.62:0/1702045350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff954004800 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff95c001e50 tx=0x7ff954005830 comp rx=0 tx=0).stop 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.135+0000 7ff972800640 1 -- 172.21.15.62:0/1702045350 reap_dead start 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.136+0000 7ff973200640 1 -- 172.21.15.62:0/1702045350 shutdown_connections 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.136+0000 7ff973200640 1 -- 172.21.15.62:0/1702045350 >> 172.21.15.62:0/1702045350 conn(0x7ff97406b000 msgr2=0x7ff97406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.136+0000 7ff973200640 1 -- 172.21.15.62:0/1702045350 shutdown_connections 2024-04-25T09:14:25.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.136+0000 7ff973200640 1 -- 172.21.15.62:0/1702045350 wait complete. 2024-04-25T09:14:25.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.137+0000 7ff973200640 1 Processor -- start 2024-04-25T09:14:25.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.138+0000 7ff973200640 1 -- start start 2024-04-25T09:14:25.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff973200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff9740b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff973200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff974151dd0 0x7ff9740b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff973200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff97414f7f0 0x7ff97415f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff973200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740d3f30 con 0x7ff954004010 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff973200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740d3db0 con 0x7ff97414f7f0 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff973200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740d40b0 con 0x7ff974151dd0 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff9740b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff9740b8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53954/0 (socket says 172.21.15.62:53954) 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971e00640 1 -- 172.21.15.62:0/3315019334 learned_addr learned my addr 172.21.15.62:0/3315019334 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971400640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff974151dd0 0x7ff9740b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff972800640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff97414f7f0 0x7ff97415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971e00640 1 -- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff974151dd0 msgr2=0x7ff9740b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971e00640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff974151dd0 0x7ff9740b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971e00640 1 -- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff97414f7f0 msgr2=0x7ff97415f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971e00640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff97414f7f0 0x7ff97415f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.139+0000 7ff971e00640 1 -- 172.21.15.62:0/3315019334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9741e88f0 con 0x7ff954004010 2024-04-25T09:14:25.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.140+0000 7ff971e00640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff9740b8b50 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff96400e870 tx=0x7ff96400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:25.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.140+0000 7ff96a000640 1 -- 172.21.15.62:0/3315019334 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96400c870 con 0x7ff954004010 2024-04-25T09:14:25.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.140+0000 7ff96a000640 1 -- 172.21.15.62:0/3315019334 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff964018070 con 0x7ff954004010 2024-04-25T09:14:25.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.140+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9741e8bd0 con 0x7ff954004010 2024-04-25T09:14:25.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.140+0000 7ff96a000640 1 -- 172.21.15.62:0/3315019334 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96400cc70 con 0x7ff954004010 2024-04-25T09:14:25.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.140+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9741e9190 con 0x7ff954004010 2024-04-25T09:14:25.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.141+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff97404ef20 con 0x7ff954004010 2024-04-25T09:14:25.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.142+0000 7ff96a000640 1 -- 172.21.15.62:0/3315019334 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff964013420 con 0x7ff954004010 2024-04-25T09:14:25.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.144+0000 7ff96a000640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff95006b500 0x7ff95006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.144+0000 7ff96a000640 1 -- 172.21.15.62:0/3315019334 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff964089f40 con 0x7ff954004010 2024-04-25T09:14:25.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.144+0000 7ff971400640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff95006b500 0x7ff95006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.146+0000 7ff971400640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff95006b500 0x7ff95006d9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff95c032000 tx=0x7ff95c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:25.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.148+0000 7ff96a000640 1 -- 172.21.15.62:0/3315019334 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff964022250 con 0x7ff954004010 2024-04-25T09:14:25.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.343+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9740b9950 con 0x7ff954004010 2024-04-25T09:14:25.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.343+0000 7ff96a000640 1 -- 172.21.15.62:0/3315019334 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff964058d10 con 0x7ff954004010 2024-04-25T09:14:25.331 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:14:25.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff95006b500 msgr2=0x7ff95006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff95006b500 0x7ff95006d9c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff95c032000 tx=0x7ff95c02e000 comp rx=0 tx=0).stop 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 msgr2=0x7ff9740b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff9740b8b50 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff96400e870 tx=0x7ff96400ed30 comp rx=0 tx=0).stop 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 shutdown_connections 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff95006b500 0x7ff95006d9c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff97414f7f0 0x7ff97415f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff974151dd0 0x7ff9740b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 --2- 172.21.15.62:0/3315019334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff954004010 0x7ff9740b8b50 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 >> 172.21.15.62:0/3315019334 conn(0x7ff97406b000 msgr2=0x7ff9740c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 shutdown_connections 2024-04-25T09:14:25.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.346+0000 7ff973200640 1 -- 172.21.15.62:0/3315019334 wait complete. 2024-04-25T09:14:25.353 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:14:25.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.576+0000 7f0284e00640 1 Processor -- start 2024-04-25T09:14:25.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.576+0000 7f0284e00640 1 -- start start 2024-04-25T09:14:25.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.577+0000 7f0284e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02800ceb50 0x7f028006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.577+0000 7f0284e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f028006d800 0x7f028015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.577+0000 7f0284e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02800b5560 con 0x7f028015e5f0 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.577+0000 7f0284e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028006dc60 con 0x7f028006d800 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.578+0000 7f0284e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028015ee30 con 0x7f02800ceb50 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.578+0000 7f027fe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f028015e5f0 0x7f02800ad8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52148/0 (socket says 172.21.15.62:52148) 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.578+0000 7f027fe00640 1 -- 172.21.15.62:0/1777423981 learned_addr learned my addr 172.21.15.62:0/1777423981 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.578+0000 7f0277e00640 1 --2- 172.21.15.62:0/1777423981 >> v2:172.21.15.121:3300/0 conn(0x7f028006d800 0x7f028015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.578+0000 7f027f400640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02800ceb50 0x7f028006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.578+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159886188 0 0) 0x7f02800b5560 con 0x7f028015e5f0 2024-04-25T09:14:25.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.578+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0264003520 con 0x7f028015e5f0 2024-04-25T09:14:25.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990005766 0 0) 0x7f0264003520 con 0x7f028015e5f0 2024-04-25T09:14:25.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02800ceb50 msgr2=0x7f028006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:25.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02800ceb50 0x7f028006d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 >> v2:172.21.15.121:3300/0 conn(0x7f028006d800 msgr2=0x7f028015e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 --2- 172.21.15.62:0/1777423981 >> v2:172.21.15.121:3300/0 conn(0x7f028006d800 0x7f028015e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02801618b0 con 0x7f028015e5f0 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f02780019f0 con 0x7f028015e5f0 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 >> v1:172.21.15.62:6789/0 conn(0x7f028015e5f0 legacy=0x7f02800ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02800ceb50 0x7f02640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0264003f80 0x7f0264004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264004d40 0x7f02640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028015ee30 con 0x7f0264003f80 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026400b9f0 con 0x7f0264004d40 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026400bbf0 con 0x7f02800ceb50 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.579+0000 7f0277e00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0264003f80 0x7f0264004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.580+0000 7f027f400640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02800ceb50 0x7f02640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.580+0000 7f0277e00640 1 -- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02800ceb50 msgr2=0x7f02640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.580+0000 7f0277e00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f02800ceb50 0x7f02640039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.580+0000 7f0277e00640 1 -- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264004d40 msgr2=0x7f02640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.580+0000 7f0277e00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264004d40 0x7f02640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.580+0000 7f0277e00640 1 -- 172.21.15.62:0/1777423981 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0264003520 con 0x7f0264003f80 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.581+0000 7f0277e00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0264003f80 0x7f0264004800 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f0268009ac0 tx=0x7f0264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebd8a8a80b0a55f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.581+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268036070 con 0x7f0264003f80 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.581+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0268031400 con 0x7f0264003f80 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.581+0000 7f027ea00640 1 -- 172.21.15.62:0/1777423981 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268009690 con 0x7f0264003f80 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.581+0000 7f0284e00640 1 -- 172.21.15.62:0/1777423981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0264003f80 msgr2=0x7f0264004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.582+0000 7f0284e00640 1 --2- 172.21.15.62:0/1777423981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0264003f80 0x7f0264004800 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f0268009ac0 tx=0x7f0264005830 comp rx=0 tx=0).stop 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.582+0000 7f027fe00640 1 -- 172.21.15.62:0/1777423981 reap_dead start 2024-04-25T09:14:25.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.582+0000 7f0284e00640 1 -- 172.21.15.62:0/1777423981 shutdown_connections 2024-04-25T09:14:25.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.582+0000 7f0284e00640 1 -- 172.21.15.62:0/1777423981 >> 172.21.15.62:0/1777423981 conn(0x7f0280066f20 msgr2=0x7f02800b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:25.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.582+0000 7f0284e00640 1 -- 172.21.15.62:0/1777423981 shutdown_connections 2024-04-25T09:14:25.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.582+0000 7f0284e00640 1 -- 172.21.15.62:0/1777423981 wait complete. 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.584+0000 7f0284e00640 1 Processor -- start 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.584+0000 7f0284e00640 1 -- start start 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0284e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264003f80 0x7f028015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0284e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02801f2df0 0x7f028015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0284e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f028015f8e0 0x7f0280156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0284e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028015ebe0 con 0x7f02801f2df0 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0284e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028015ea60 con 0x7f0264003f80 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0284e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02800cef30 con 0x7f028015f8e0 2024-04-25T09:14:25.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0277e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02801f2df0 0x7f028015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0277e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02801f2df0 0x7f028015f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53966/0 (socket says 172.21.15.62:53966) 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0277e00640 1 -- 172.21.15.62:0/1588789105 learned_addr learned my addr 172.21.15.62:0/1588789105 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f027fe00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f028015f8e0 0x7f0280156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f027f400640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264003f80 0x7f028015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0277e00640 1 -- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f028015f8e0 msgr2=0x7f0280156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0277e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f028015f8e0 0x7f0280156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0277e00640 1 -- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264003f80 msgr2=0x7f028015cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0277e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264003f80 0x7f028015cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f0277e00640 1 -- 172.21.15.62:0/1588789105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028015d000 con 0x7f02801f2df0 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.585+0000 7f027f400640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264003f80 0x7f028015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.586+0000 7f0277e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02801f2df0 0x7f028015f3a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f0268004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:25.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.586+0000 7f0277400640 1 -- 172.21.15.62:0/1588789105 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268040020 con 0x7f02801f2df0 2024-04-25T09:14:25.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.586+0000 7f0277400640 1 -- 172.21.15.62:0/1588789105 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0268003c60 con 0x7f02801f2df0 2024-04-25T09:14:25.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.586+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02801f6860 con 0x7f02801f2df0 2024-04-25T09:14:25.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.586+0000 7f0277400640 1 -- 172.21.15.62:0/1588789105 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268036040 con 0x7f02801f2df0 2024-04-25T09:14:25.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.586+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02801f6e00 con 0x7f02801f2df0 2024-04-25T09:14:25.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.587+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028006abe0 con 0x7f02801f2df0 2024-04-25T09:14:25.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.588+0000 7f0277400640 1 -- 172.21.15.62:0/1588789105 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0268043020 con 0x7f02801f2df0 2024-04-25T09:14:25.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.590+0000 7f0277400640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f025c06b4d0 0x7f025c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:25.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.590+0000 7f0277400640 1 -- 172.21.15.62:0/1588789105 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f02680aa4a0 con 0x7f02801f2df0 2024-04-25T09:14:25.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.591+0000 7f027f400640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f025c06b4d0 0x7f025c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:25.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.592+0000 7f027f400640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f025c06b4d0 0x7f025c06d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0270009910 tx=0x7f0270008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:25.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.594+0000 7f0277400640 1 -- 172.21.15.62:0/1588789105 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f026804a070 con 0x7f02801f2df0 2024-04-25T09:14:25.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.796+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0280137700 con 0x7f02801f2df0 2024-04-25T09:14:25.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.797+0000 7f0277400640 1 -- 172.21.15.62:0/1588789105 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0280137700 con 0x7f02801f2df0 2024-04-25T09:14:25.784 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:14:25.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f025c06b4d0 msgr2=0x7f025c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f025c06b4d0 0x7f025c06d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0270009910 tx=0x7f0270008040 comp rx=0 tx=0).stop 2024-04-25T09:14:25.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02801f2df0 msgr2=0x7f028015f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:25.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02801f2df0 0x7f028015f3a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f0268004730 comp rx=0 tx=0).stop 2024-04-25T09:14:25.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 shutdown_connections 2024-04-25T09:14:25.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f025c06b4d0 0x7f025c06d990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02801f2df0 0x7f028015f3a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f028015f8e0 0x7f0280156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 --2- 172.21.15.62:0/1588789105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0264003f80 0x7f028015cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:25.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.799+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 >> 172.21.15.62:0/1588789105 conn(0x7f0280066f20 msgr2=0x7f02800b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:25.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.800+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 shutdown_connections 2024-04-25T09:14:25.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:25.800+0000 7f0284e00640 1 -- 172.21.15.62:0/1588789105 wait complete. 2024-04-25T09:14:25.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:14:26.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.034+0000 7fcfbb600640 1 Processor -- start 2024-04-25T09:14:26.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.034+0000 7fcfbb600640 1 -- start start 2024-04-25T09:14:26.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.035+0000 7fcfbb600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcfb415de40 0x7fcfb40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:26.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.035+0000 7fcfbb600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40d0b80 0x7fcfb40d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:26.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.035+0000 7fcfbb600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb4161c80 con 0x7fcfb40b2f50 2024-04-25T09:14:26.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.035+0000 7fcfbb600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb4161a30 con 0x7fcfb415de40 2024-04-25T09:14:26.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.035+0000 7fcfbb600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40b3400 con 0x7fcfb40d0b80 2024-04-25T09:14:26.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.036+0000 7fcfb9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40d0b80 0x7fcfb40d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:26.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.036+0000 7fcfb9600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40d0b80 0x7fcfb40d2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:46186/0 (socket says 172.21.15.62:46186) 2024-04-25T09:14:26.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.036+0000 7fcfb9600640 1 -- 172.21.15.62:0/1682882972 learned_addr learned my addr 172.21.15.62:0/1682882972 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:26.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.036+0000 7fcfb8c00640 1 --2- 172.21.15.62:0/1682882972 >> v2:172.21.15.121:3300/0 conn(0x7fcfb415de40 0x7fcfb40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:26.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.037+0000 7fcfb9600640 1 -- 172.21.15.62:0/1682882972 >> v2:172.21.15.121:3300/0 conn(0x7fcfb415de40 msgr2=0x7fcfb40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:26.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.037+0000 7fcfb9600640 1 --2- 172.21.15.62:0/1682882972 >> v2:172.21.15.121:3300/0 conn(0x7fcfb415de40 0x7fcfb40ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.037+0000 7fcfb9600640 1 -- 172.21.15.62:0/1682882972 >> v1:172.21.15.121:6789/0 conn(0x7fcfb40b2f50 legacy=0x7fcfb40d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:26.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.037+0000 7fcfb9600640 1 -- 172.21.15.62:0/1682882972 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb4161f60 con 0x7fcfb40d0b80 2024-04-25T09:14:26.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.037+0000 7fcfb3400640 1 -- 172.21.15.62:0/1682882972 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687561452 0 0) 0x7fcfb4161c80 con 0x7fcfb40b2f50 2024-04-25T09:14:26.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.038+0000 7fcfb9600640 1 --2- 172.21.15.62:0/1682882972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40d0b80 0x7fcfb40d2000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009870 tx=0x7fcfac02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc62915b64f9b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:26.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.039+0000 7fcfb3400640 1 -- 172.21.15.62:0/1682882972 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac037070 con 0x7fcfb40d0b80 2024-04-25T09:14:26.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.039+0000 7fcfb3400640 1 -- 172.21.15.62:0/1682882972 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac032440 con 0x7fcfb40d0b80 2024-04-25T09:14:26.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.039+0000 7fcfb3400640 1 -- 172.21.15.62:0/1682882972 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac032810 con 0x7fcfb40d0b80 2024-04-25T09:14:26.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.040+0000 7fcfbb600640 1 -- 172.21.15.62:0/1682882972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40d0b80 msgr2=0x7fcfb40d2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:26.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.040+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1682882972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40d0b80 0x7fcfb40d2000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009870 tx=0x7fcfac02a130 comp rx=0 tx=0).stop 2024-04-25T09:14:26.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.040+0000 7fcfbb600640 1 -- 172.21.15.62:0/1682882972 shutdown_connections 2024-04-25T09:14:26.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.040+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1682882972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40d0b80 0x7fcfb40d2000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.040+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1682882972 >> v2:172.21.15.121:3300/0 conn(0x7fcfb415de40 0x7fcfb40ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.040+0000 7fcfbb600640 1 -- 172.21.15.62:0/1682882972 >> 172.21.15.62:0/1682882972 conn(0x7fcfb40668f0 msgr2=0x7fcfb4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:26.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.040+0000 7fcfbb600640 1 -- 172.21.15.62:0/1682882972 shutdown_connections 2024-04-25T09:14:26.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.040+0000 7fcfbb600640 1 -- 172.21.15.62:0/1682882972 wait complete. 2024-04-25T09:14:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.042+0000 7fcfbb600640 1 Processor -- start 2024-04-25T09:14:26.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.042+0000 7fcfbb600640 1 -- start start 2024-04-25T09:14:26.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.043+0000 7fcfbb600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b2f50 0x7fcfb40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:26.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.043+0000 7fcfbb600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40d0b80 0x7fcfb40b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:26.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.043+0000 7fcfbb600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb415de40 0x7fcfb414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:26.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.043+0000 7fcfbb600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb415e220 con 0x7fcfb40d0b80 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.043+0000 7fcfbb600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40d2540 con 0x7fcfb415de40 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.043+0000 7fcfbb600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40cbe80 con 0x7fcfb40b2f50 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.043+0000 7fcfb3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40d0b80 0x7fcfb40b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.043+0000 7fcfb3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40d0b80 0x7fcfb40b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:53978/0 (socket says 172.21.15.62:53978) 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb3e00640 1 -- 172.21.15.62:0/1749955345 learned_addr learned my addr 172.21.15.62:0/1749955345 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb8c00640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b2f50 0x7fcfb40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb3e00640 1 -- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b2f50 msgr2=0x7fcfb40cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb3e00640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b2f50 0x7fcfb40cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb3e00640 1 -- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb415de40 msgr2=0x7fcfb414cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb3e00640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb415de40 0x7fcfb414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb3e00640 1 -- 172.21.15.62:0/1749955345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40cc160 con 0x7fcfb40d0b80 2024-04-25T09:14:26.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb3e00640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40d0b80 0x7fcfb40b1f80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c890 tx=0x7fcfa000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:26.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.044+0000 7fcfb0c00640 1 -- 172.21.15.62:0/1749955345 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0017070 con 0x7fcfb40d0b80 2024-04-25T09:14:26.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.045+0000 7fcfb0c00640 1 -- 172.21.15.62:0/1749955345 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0003dc0 con 0x7fcfb40d0b80 2024-04-25T09:14:26.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.045+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb40cc440 con 0x7fcfb40d0b80 2024-04-25T09:14:26.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.045+0000 7fcfb0c00640 1 -- 172.21.15.62:0/1749955345 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00052b0 con 0x7fcfb40d0b80 2024-04-25T09:14:26.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.045+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb40cbb70 con 0x7fcfb40d0b80 2024-04-25T09:14:26.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.046+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb414af80 con 0x7fcfb40d0b80 2024-04-25T09:14:26.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.047+0000 7fcfb0c00640 1 -- 172.21.15.62:0/1749955345 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfa001a070 con 0x7fcfb40d0b80 2024-04-25T09:14:26.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.048+0000 7fcfb0c00640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9406b4d0 0x7fcf9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:26.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.048+0000 7fcfb0c00640 1 -- 172.21.15.62:0/1749955345 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcfa0089b20 con 0x7fcfb40d0b80 2024-04-25T09:14:26.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.048+0000 7fcfb8c00640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9406b4d0 0x7fcf9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:26.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.050+0000 7fcfb8c00640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9406b4d0 0x7fcf9406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009a30 tx=0x7fcfa4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:26.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.052+0000 7fcfb0c00640 1 -- 172.21.15.62:0/1749955345 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfa0058840 con 0x7fcfb40d0b80 2024-04-25T09:14:26.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.267+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcfb4072960 con 0x7fcfb40d0b80 2024-04-25T09:14:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.268+0000 7fcfb0c00640 1 -- 172.21.15.62:0/1749955345 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v111) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcfa005e020 con 0x7fcfb40d0b80 2024-04-25T09:14:26.255 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9406b4d0 msgr2=0x7fcf9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9406b4d0 0x7fcf9406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009a30 tx=0x7fcfa4008040 comp rx=0 tx=0).stop 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40d0b80 msgr2=0x7fcfb40b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40d0b80 0x7fcfb40b1f80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c890 tx=0x7fcfa000cd50 comp rx=0 tx=0).stop 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 shutdown_connections 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf9406b4d0 0x7fcf9406d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcfb40d0b80 0x7fcfb40b1f80 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcfb415de40 0x7fcfb414cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 --2- 172.21.15.62:0/1749955345 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcfb40b2f50 0x7fcfb40cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.270+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 >> 172.21.15.62:0/1749955345 conn(0x7fcfb40668f0 msgr2=0x7fcfb40b8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.271+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 shutdown_connections 2024-04-25T09:14:26.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:26.271+0000 7fcfbb600640 1 -- 172.21.15.62:0/1749955345 wait complete. 2024-04-25T09:14:26.278 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:14:26.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:26.278 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:14:27.079 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:14:27.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:27.080 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:14:27.121 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:14:27.121 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:27.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:14:27.161 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:14:27.161 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:27.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:14:27.212 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:14:27.212 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:27.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:14:27.262 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:27.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.496+0000 7f83a6c00640 1 Processor -- start 2024-04-25T09:14:27.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.497+0000 7f83a6c00640 1 -- start start 2024-04-25T09:14:27.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.498+0000 7f83a6c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83a00aeed0 0x7f83a0166900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.498+0000 7f83a6c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a006ca70 con 0x7f83a00cccf0 2024-04-25T09:14:27.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.498+0000 7f83a6c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a015c0f0 con 0x7f83a015bd10 2024-04-25T09:14:27.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.499+0000 7f83a6c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a006d660 con 0x7f83a00aeed0 2024-04-25T09:14:27.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.499+0000 7f839f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f83a00cccf0 0x7f83a00ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52152/0 (socket says 172.21.15.62:52152) 2024-04-25T09:14:27.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.499+0000 7f839f400640 1 -- 172.21.15.62:0/2103937926 learned_addr learned my addr 172.21.15.62:0/2103937926 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:27.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.499+0000 7f83a4c00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83a00aeed0 0x7f83a0166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.499+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824436246 0 0) 0x7f83a006ca70 con 0x7f83a00cccf0 2024-04-25T09:14:27.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.499+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8384003520 con 0x7f83a00cccf0 2024-04-25T09:14:27.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 668140334 0 0) 0x7f8384003520 con 0x7f83a00cccf0 2024-04-25T09:14:27.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83a00aeed0 msgr2=0x7f83a0166900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:27.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83a00aeed0 0x7f83a0166900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 >> v1:172.21.15.121:6789/0 conn(0x7f83a015bd10 legacy=0x7f83a00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a006cd30 con 0x7f83a00cccf0 2024-04-25T09:14:27.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f8388001670 con 0x7f83a00cccf0 2024-04-25T09:14:27.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 >> v1:172.21.15.62:6789/0 conn(0x7f83a00cccf0 legacy=0x7f83a00ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83840039e0 0x7f8384003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83a00aeed0 0x7f8384004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8384004d40 0x7f83840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a006d660 con 0x7f8384004d40 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838400bad0 con 0x7f83840039e0 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838400bcd0 con 0x7f83a00aeed0 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.500+0000 7f83a4c00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8384004d40 0x7f83840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.501+0000 7f839f400640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83a00aeed0 0x7f8384004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.501+0000 7f83a4c00640 1 -- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83a00aeed0 msgr2=0x7f8384004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.501+0000 7f83a4c00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f83a00aeed0 0x7f8384004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.501+0000 7f83a4c00640 1 -- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83840039e0 msgr2=0x7f8384003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:27.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.501+0000 7f83a4c00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83840039e0 0x7f8384003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.501+0000 7f83a4c00640 1 -- 172.21.15.62:0/2103937926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384003520 con 0x7f8384004d40 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.502+0000 7f83a4c00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8384004d40 0x7f83840051c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f83a0070260 tx=0x7f8384005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81947805ee57d28c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.502+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398037070 con 0x7f8384004d40 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.502+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839802e050 con 0x7f8384004d40 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.502+0000 7f839ea00640 1 -- 172.21.15.62:0/2103937926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398009c10 con 0x7f8384004d40 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.503+0000 7f83a6c00640 1 -- 172.21.15.62:0/2103937926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8384004d40 msgr2=0x7f83840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.503+0000 7f83a6c00640 1 --2- 172.21.15.62:0/2103937926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8384004d40 0x7f83840051c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f83a0070260 tx=0x7f8384005830 comp rx=0 tx=0).stop 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.503+0000 7f83a4c00640 1 -- 172.21.15.62:0/2103937926 reap_dead start 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.503+0000 7f83a6c00640 1 -- 172.21.15.62:0/2103937926 shutdown_connections 2024-04-25T09:14:27.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.503+0000 7f83a6c00640 1 -- 172.21.15.62:0/2103937926 >> 172.21.15.62:0/2103937926 conn(0x7f83a00668f0 msgr2=0x7f83a0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.503+0000 7f83a6c00640 1 -- 172.21.15.62:0/2103937926 shutdown_connections 2024-04-25T09:14:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.503+0000 7f83a6c00640 1 -- 172.21.15.62:0/2103937926 wait complete. 2024-04-25T09:14:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.505+0000 7f83a6c00640 1 Processor -- start 2024-04-25T09:14:27.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.505+0000 7f83a6c00640 1 -- start start 2024-04-25T09:14:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a6c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8384004d40 0x7f83a01f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a6c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a01f1ee0 0x7f83a015a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a6c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a015a7b0 0x7f83a014f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a6c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a006d2f0 con 0x7f83a015a7b0 2024-04-25T09:14:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a6c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a00ce6d0 con 0x7f83a01f1ee0 2024-04-25T09:14:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a6c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a006daf0 con 0x7f8384004d40 2024-04-25T09:14:27.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a015a7b0 0x7f83a014f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a4c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a015a7b0 0x7f83a014f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54000/0 (socket says 172.21.15.62:54000) 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a4c00640 1 -- 172.21.15.62:0/256624608 learned_addr learned my addr 172.21.15.62:0/256624608 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f839f400640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a01f1ee0 0x7f83a015a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f83a4c00640 1 -- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8384004d40 msgr2=0x7f83a01f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.506+0000 7f839fe00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8384004d40 0x7f83a01f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f83a4c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8384004d40 0x7f83a01f19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f83a4c00640 1 -- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a01f1ee0 msgr2=0x7f83a015a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f83a4c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a01f1ee0 0x7f83a015a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f83a4c00640 1 -- 172.21.15.62:0/256624608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a015ad30 con 0x7f83a015a7b0 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f839f400640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a01f1ee0 0x7f83a015a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f83a4c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a015a7b0 0x7f83a014f1e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8398035040 tx=0x7f8398009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f8397e00640 1 -- 172.21.15.62:0/256624608 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398037070 con 0x7f83a015a7b0 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f8397e00640 1 -- 172.21.15.62:0/256624608 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8398038040 con 0x7f83a015a7b0 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f8397e00640 1 -- 172.21.15.62:0/256624608 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398009c10 con 0x7f83a015a7b0 2024-04-25T09:14:27.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a015b010 con 0x7f83a015a7b0 2024-04-25T09:14:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.507+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a014ecc0 con 0x7f83a015a7b0 2024-04-25T09:14:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.509+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a0072c70 con 0x7f83a015a7b0 2024-04-25T09:14:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.510+0000 7f8397e00640 1 -- 172.21.15.62:0/256624608 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8398044030 con 0x7f83a015a7b0 2024-04-25T09:14:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.510+0000 7f8397e00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.511+0000 7f8397e00640 1 -- 172.21.15.62:0/256624608 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f83980aa4b0 con 0x7f83a015a7b0 2024-04-25T09:14:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.511+0000 7f839fe00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.512+0000 7f839fe00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f837c06b4d0 0x7f837c06d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f83900015d0 tx=0x7f8390009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:27.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.515+0000 7f8397e00640 1 -- 172.21.15.62:0/256624608 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f839804b020 con 0x7f83a015a7b0 2024-04-25T09:14:27.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.753+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83a004f000 con 0x7f83a015a7b0 2024-04-25T09:14:27.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.754+0000 7f8397e00640 1 -- 172.21.15.62:0/256624608 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f83980792e0 con 0x7f83a015a7b0 2024-04-25T09:14:27.742 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:27.742 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:27.742 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:27.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.757+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f837c06b4d0 msgr2=0x7f837c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.757+0000 7f83a6c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f837c06b4d0 0x7f837c06d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f83900015d0 tx=0x7f8390009040 comp rx=0 tx=0).stop 2024-04-25T09:14:27.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.757+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a015a7b0 msgr2=0x7f83a014f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.757+0000 7f83a6c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a015a7b0 0x7f83a014f1e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f8398035040 tx=0x7f8398009750 comp rx=0 tx=0).stop 2024-04-25T09:14:27.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.758+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 shutdown_connections 2024-04-25T09:14:27.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.758+0000 7f83a6c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.758+0000 7f83a6c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f83a015a7b0 0x7f83a014f1e0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.758+0000 7f83a6c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a01f1ee0 0x7f83a015a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.758+0000 7f83a6c00640 1 --2- 172.21.15.62:0/256624608 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8384004d40 0x7f83a01f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.758+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 >> 172.21.15.62:0/256624608 conn(0x7f83a00668f0 msgr2=0x7f83a00b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:27.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.758+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 shutdown_connections 2024-04-25T09:14:27.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.758+0000 7f83a6c00640 1 -- 172.21.15.62:0/256624608 wait complete. 2024-04-25T09:14:27.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:27.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.980+0000 7f5b6b800640 1 Processor -- start 2024-04-25T09:14:27.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.980+0000 7f5b6b800640 1 -- start start 2024-04-25T09:14:27.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.982+0000 7f5b6b800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b640ce300 0x7f5b640ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.982+0000 7f5b6b800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b641666d0 con 0x7f5b64072020 2024-04-25T09:14:27.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.982+0000 7f5b6b800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b641669b0 con 0x7f5b640ccce0 2024-04-25T09:14:27.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.982+0000 7f5b6b800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b64166c90 con 0x7f5b640ce300 2024-04-25T09:14:27.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.982+0000 7f5b68e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5b64072020 0x7f5b64165e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52156/0 (socket says 172.21.15.62:52156) 2024-04-25T09:14:27.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.982+0000 7f5b68e00640 1 -- 172.21.15.62:0/413876432 learned_addr learned my addr 172.21.15.62:0/413876432 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:27.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b69800640 1 --2- 172.21.15.62:0/413876432 >> v2:172.21.15.121:3300/0 conn(0x7f5b640ce300 0x7f5b640ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338154955 0 0) 0x7f5b641666d0 con 0x7f5b64072020 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b4c003520 con 0x7f5b64072020 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544649295 0 0) 0x7f5b4c003520 con 0x7f5b64072020 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 >> v2:172.21.15.121:3300/0 conn(0x7f5b640ce300 msgr2=0x7f5b640ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 --2- 172.21.15.62:0/413876432 >> v2:172.21.15.121:3300/0 conn(0x7f5b640ce300 0x7f5b640ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 >> v1:172.21.15.121:6789/0 conn(0x7f5b640ccce0 legacy=0x7f5b64070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6406d000 con 0x7f5b64072020 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f5b58001670 con 0x7f5b64072020 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 >> v1:172.21.15.62:6789/0 conn(0x7f5b64072020 legacy=0x7f5b64165e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b4c0039b0 0x7f5b4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b4c004800 0x7f5b4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b4c0051c0 0x7f5b4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b64166c90 con 0x7f5b4c0051c0 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c00c060 con 0x7f5b4c004800 2024-04-25T09:14:27.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.983+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c00c260 con 0x7f5b4c0039b0 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.984+0000 7f5b69800640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b4c0051c0 0x7f5b4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.984+0000 7f5b63e00640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b4c004800 0x7f5b4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.984+0000 7f5b69800640 1 -- 172.21.15.62:0/413876432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b4c0039b0 msgr2=0x7f5b4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.984+0000 7f5b69800640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b4c0039b0 0x7f5b4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.984+0000 7f5b69800640 1 -- 172.21.15.62:0/413876432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b4c004800 msgr2=0x7f5b4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.984+0000 7f5b69800640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b4c004800 0x7f5b4c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.984+0000 7f5b69800640 1 -- 172.21.15.62:0/413876432 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c003520 con 0x7f5b4c0051c0 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.985+0000 7f5b69800640 1 -- 172.21.15.62:0/413876432 reap_dead start 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.985+0000 7f5b69800640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b4c0051c0 0x7f5b4c00b9c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009fc0 tx=0x7f5b4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ee97dba6ae822d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.985+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c035020 con 0x7f5b4c0051c0 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.985+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c003b90 con 0x7f5b4c0051c0 2024-04-25T09:14:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.985+0000 7f5b63400640 1 -- 172.21.15.62:0/413876432 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c009c10 con 0x7f5b4c0051c0 2024-04-25T09:14:27.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.986+0000 7f5b6b800640 1 -- 172.21.15.62:0/413876432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b4c0051c0 msgr2=0x7f5b4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.986+0000 7f5b6b800640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b4c0051c0 0x7f5b4c00b9c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009fc0 tx=0x7f5b4c005830 comp rx=0 tx=0).stop 2024-04-25T09:14:27.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.986+0000 7f5b6b800640 1 -- 172.21.15.62:0/413876432 shutdown_connections 2024-04-25T09:14:27.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.986+0000 7f5b6b800640 1 --2- 172.21.15.62:0/413876432 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b4c0051c0 0x7f5b4c00b9c0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.986+0000 7f5b6b800640 1 -- 172.21.15.62:0/413876432 >> 172.21.15.62:0/413876432 conn(0x7f5b640668f0 msgr2=0x7f5b64066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.986+0000 7f5b6b800640 1 -- 172.21.15.62:0/413876432 shutdown_connections 2024-04-25T09:14:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.986+0000 7f5b6b800640 1 -- 172.21.15.62:0/413876432 wait complete. 2024-04-25T09:14:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.988+0000 7f5b6b800640 1 Processor -- start 2024-04-25T09:14:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.988+0000 7f5b6b800640 1 -- start start 2024-04-25T09:14:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.989+0000 7f5b6b800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b4c0051c0 0x7f5b6415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.989+0000 7f5b6b800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b640ada80 0x7f5b640adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.989+0000 7f5b6b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b64164e80 0x7f5b64165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.989+0000 7f5b6b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b64072580 con 0x7f5b64164e80 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.989+0000 7f5b6b800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b64072400 con 0x7f5b4c0051c0 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.989+0000 7f5b6b800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640ce790 con 0x7f5b640ada80 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.989+0000 7f5b69800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b64164e80 0x7f5b64165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b69800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b64164e80 0x7f5b64165300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54012/0 (socket says 172.21.15.62:54012) 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b69800640 1 -- 172.21.15.62:0/2564605591 learned_addr learned my addr 172.21.15.62:0/2564605591 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b63e00640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b640ada80 0x7f5b640adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b69800640 1 -- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b640ada80 msgr2=0x7f5b640adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b69800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b640ada80 0x7f5b640adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b69800640 1 -- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b4c0051c0 msgr2=0x7f5b6415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b69800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b4c0051c0 0x7f5b6415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b69800640 1 -- 172.21.15.62:0/2564605591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b641ebf30 con 0x7f5b64164e80 2024-04-25T09:14:27.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.990+0000 7f5b69800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b64164e80 0x7f5b64165300 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e230 tx=0x7f5b5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.991+0000 7f5b60c00640 1 -- 172.21.15.62:0/2564605591 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c03f030 con 0x7f5b64164e80 2024-04-25T09:14:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.991+0000 7f5b60c00640 1 -- 172.21.15.62:0/2564605591 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c003990 con 0x7f5b64164e80 2024-04-25T09:14:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.991+0000 7f5b60c00640 1 -- 172.21.15.62:0/2564605591 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c035070 con 0x7f5b64164e80 2024-04-25T09:14:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.991+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b641ec210 con 0x7f5b64164e80 2024-04-25T09:14:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.991+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b641ec7d0 con 0x7f5b64164e80 2024-04-25T09:14:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.992+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b64072c70 con 0x7f5b64164e80 2024-04-25T09:14:27.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.993+0000 7f5b60c00640 1 -- 172.21.15.62:0/2564605591 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b5c042030 con 0x7f5b64164e80 2024-04-25T09:14:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.994+0000 7f5b60c00640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b4406b4d0 0x7f5b4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.994+0000 7f5b60c00640 1 -- 172.21.15.62:0/2564605591 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5b5c0aa980 con 0x7f5b64164e80 2024-04-25T09:14:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.994+0000 7f5b68e00640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b4406b4d0 0x7f5b4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.996+0000 7f5b68e00640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b4406b4d0 0x7f5b4406d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5b64164d40 tx=0x7f5b58001470 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:27.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:27.998+0000 7f5b60c00640 1 -- 172.21.15.62:0/2564605591 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b5c079160 con 0x7f5b64164e80 2024-04-25T09:14:28.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.205+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b64165bf0 con 0x7f5b64164e80 2024-04-25T09:14:28.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.208+0000 7f5b60c00640 1 -- 172.21.15.62:0/2564605591 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16703 (secure 0 0 0) 0x7f5b5c07d200 con 0x7f5b64164e80 2024-04-25T09:14:28.196 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:28.196 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":111,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:02.867113+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:13:29.095930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:13:30.242813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.6","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:28.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.211+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b4406b4d0 msgr2=0x7f5b4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.211+0000 7f5b6b800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b4406b4d0 0x7f5b4406d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5b64164d40 tx=0x7f5b58001470 comp rx=0 tx=0).stop 2024-04-25T09:14:28.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.211+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b64164e80 msgr2=0x7f5b64165300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.211+0000 7f5b6b800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b64164e80 0x7f5b64165300 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e230 tx=0x7f5b5c009750 comp rx=0 tx=0).stop 2024-04-25T09:14:28.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.212+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 shutdown_connections 2024-04-25T09:14:28.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.212+0000 7f5b6b800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5b4406b4d0 0x7f5b4406d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.212+0000 7f5b6b800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b64164e80 0x7f5b64165300 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.212+0000 7f5b6b800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5b640ada80 0x7f5b640adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.212+0000 7f5b6b800640 1 --2- 172.21.15.62:0/2564605591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b4c0051c0 0x7f5b6415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.212+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 >> 172.21.15.62:0/2564605591 conn(0x7f5b640668f0 msgr2=0x7f5b640b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:28.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.212+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 shutdown_connections 2024-04-25T09:14:28.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.212+0000 7f5b6b800640 1 -- 172.21.15.62:0/2564605591 wait complete. 2024-04-25T09:14:28.219 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:13:29.095930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:13:30.242813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:14:28.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:14:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.445+0000 7f3e6d400640 1 Processor -- start 2024-04-25T09:14:28.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.446+0000 7f3e6d400640 1 -- start start 2024-04-25T09:14:28.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.447+0000 7f3e6d400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3e680d3960 0x7f3e680d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.447+0000 7f3e6d400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e680ce780 con 0x7f3e6815dd50 2024-04-25T09:14:28.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.447+0000 7f3e6d400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e680d42f0 con 0x7f3e680d2fb0 2024-04-25T09:14:28.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.447+0000 7f3e6d400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e680d3440 con 0x7f3e680d3960 2024-04-25T09:14:28.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.448+0000 7f3e67e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3e6815dd50 0x7f3e680d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52170/0 (socket says 172.21.15.62:52170) 2024-04-25T09:14:28.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.448+0000 7f3e67e00640 1 -- 172.21.15.62:0/3858283252 learned_addr learned my addr 172.21.15.62:0/3858283252 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:28.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.448+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867845802 0 0) 0x7f3e680ce780 con 0x7f3e6815dd50 2024-04-25T09:14:28.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.448+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e48003520 con 0x7f3e6815dd50 2024-04-25T09:14:28.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.448+0000 7f3e6ca00640 1 --2- 172.21.15.62:0/3858283252 >> v2:172.21.15.121:3300/0 conn(0x7f3e680d3960 0x7f3e680d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456172582 0 0) 0x7f3e48003520 con 0x7f3e6815dd50 2024-04-25T09:14:28.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 >> v2:172.21.15.121:3300/0 conn(0x7f3e680d3960 msgr2=0x7f3e680d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:28.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 --2- 172.21.15.62:0/3858283252 >> v2:172.21.15.121:3300/0 conn(0x7f3e680d3960 0x7f3e680d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 >> v1:172.21.15.121:6789/0 conn(0x7f3e680d2fb0 legacy=0x7f3e680ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e680ce1c0 con 0x7f3e6815dd50 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f3e58001670 con 0x7f3e6815dd50 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 >> v1:172.21.15.62:6789/0 conn(0x7f3e6815dd50 legacy=0x7f3e680d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48003a00 0x7f3e48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e48004800 0x7f3e48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e48005210 0x7f3e4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680d3440 con 0x7f3e48005210 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4800c160 con 0x7f3e48003a00 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4800c360 con 0x7f3e48004800 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e6ca00640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e48005210 0x7f3e4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e67e00640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48003a00 0x7f3e48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.449+0000 7f3e67400640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e48004800 0x7f3e48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.450+0000 7f3e6ca00640 1 -- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e48004800 msgr2=0x7f3e48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.450+0000 7f3e6ca00640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e48004800 0x7f3e48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.450+0000 7f3e6ca00640 1 -- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48003a00 msgr2=0x7f3e48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:28.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.450+0000 7f3e6ca00640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48003a00 0x7f3e48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.450+0000 7f3e6ca00640 1 -- 172.21.15.62:0/3858283252 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48003520 con 0x7f3e48005210 2024-04-25T09:14:28.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.450+0000 7f3e67400640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e48004800 0x7f3e48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.450+0000 7f3e6ca00640 1 -- 172.21.15.62:0/3858283252 reap_dead start 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.451+0000 7f3e6ca00640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e48005210 0x7f3e4800b9c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000ba30 tx=0x7f3e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28675c70122eb51e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.451+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60030070 con 0x7f3e48005210 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.451+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6002ae40 con 0x7f3e48005210 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.451+0000 7f3e66a00640 1 -- 172.21.15.62:0/3858283252 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6000b600 con 0x7f3e48005210 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.452+0000 7f3e6d400640 1 -- 172.21.15.62:0/3858283252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e48005210 msgr2=0x7f3e4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.452+0000 7f3e6d400640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e48005210 0x7f3e4800b9c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000ba30 tx=0x7f3e48005830 comp rx=0 tx=0).stop 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.452+0000 7f3e6d400640 1 -- 172.21.15.62:0/3858283252 shutdown_connections 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.452+0000 7f3e6d400640 1 --2- 172.21.15.62:0/3858283252 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e48005210 0x7f3e4800b9c0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.452+0000 7f3e6d400640 1 -- 172.21.15.62:0/3858283252 >> 172.21.15.62:0/3858283252 conn(0x7f3e6806b000 msgr2=0x7f3e6806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.452+0000 7f3e6d400640 1 -- 172.21.15.62:0/3858283252 shutdown_connections 2024-04-25T09:14:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.452+0000 7f3e6d400640 1 -- 172.21.15.62:0/3858283252 wait complete. 2024-04-25T09:14:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.453+0000 7f3e6d400640 1 Processor -- start 2024-04-25T09:14:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.454+0000 7f3e6d400640 1 -- start start 2024-04-25T09:14:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e6d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48005210 0x7f3e6815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e6d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e68159020 0x7f3e680b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e6d400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e680b56d0 0x7f3e6814e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e6d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680d3f30 con 0x7f3e68159020 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e6d400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680d3db0 con 0x7f3e48005210 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e6d400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680d40b0 con 0x7f3e680b56d0 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e68159020 0x7f3e680b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e68159020 0x7f3e680b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54038/0 (socket says 172.21.15.62:54038) 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67400640 1 -- 172.21.15.62:0/3607550723 learned_addr learned my addr 172.21.15.62:0/3607550723 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67e00640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48005210 0x7f3e6815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e6ca00640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e680b56d0 0x7f3e6814e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67400640 1 -- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e680b56d0 msgr2=0x7f3e6814e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e680b56d0 0x7f3e6814e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67400640 1 -- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48005210 msgr2=0x7f3e6815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48005210 0x7f3e6815b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67400640 1 -- 172.21.15.62:0/3607550723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68159570 con 0x7f3e68159020 2024-04-25T09:14:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.455+0000 7f3e67e00640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48005210 0x7f3e6815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.456+0000 7f3e67400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e68159020 0x7f3e680b5190 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000db40 tx=0x7f3e500020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:28.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.456+0000 7f3e5fe00640 1 -- 172.21.15.62:0/3607550723 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50003da0 con 0x7f3e68159020 2024-04-25T09:14:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.456+0000 7f3e5fe00640 1 -- 172.21.15.62:0/3607550723 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e50017070 con 0x7f3e68159020 2024-04-25T09:14:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.456+0000 7f3e5fe00640 1 -- 172.21.15.62:0/3607550723 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000b950 con 0x7f3e68159020 2024-04-25T09:14:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.456+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e68159850 con 0x7f3e68159020 2024-04-25T09:14:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.456+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e68159e10 con 0x7f3e68159020 2024-04-25T09:14:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.457+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6804ef20 con 0x7f3e68159020 2024-04-25T09:14:28.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.458+0000 7f3e5fe00640 1 -- 172.21.15.62:0/3607550723 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e5000baf0 con 0x7f3e68159020 2024-04-25T09:14:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.460+0000 7f3e5fe00640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e4406b4d0 0x7f3e4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.460+0000 7f3e5fe00640 1 -- 172.21.15.62:0/3607550723 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3e5008a480 con 0x7f3e68159020 2024-04-25T09:14:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.461+0000 7f3e67e00640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e4406b4d0 0x7f3e4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.462+0000 7f3e67e00640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e4406b4d0 0x7f3e4406d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3e580016f0 tx=0x7f3e58000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:28.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.464+0000 7f3e5fe00640 1 -- 172.21.15.62:0/3607550723 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e50059220 con 0x7f3e68159020 2024-04-25T09:14:28.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.663+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e680b3170 con 0x7f3e68159020 2024-04-25T09:14:28.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.663+0000 7f3e5fe00640 1 -- 172.21.15.62:0/3607550723 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e50058b20 con 0x7f3e68159020 2024-04-25T09:14:28.651 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:14:28.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e4406b4d0 msgr2=0x7f3e4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e4406b4d0 0x7f3e4406d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3e580016f0 tx=0x7f3e58000ed0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e68159020 msgr2=0x7f3e680b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e68159020 0x7f3e680b5190 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000db40 tx=0x7f3e500020a0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 shutdown_connections 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3e4406b4d0 0x7f3e4406d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3e68159020 0x7f3e680b5190 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3e680b56d0 0x7f3e6814e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 --2- 172.21.15.62:0/3607550723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e48005210 0x7f3e6815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 >> 172.21.15.62:0/3607550723 conn(0x7f3e6806b000 msgr2=0x7f3e680b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 shutdown_connections 2024-04-25T09:14:28.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.666+0000 7f3e6d400640 1 -- 172.21.15.62:0/3607550723 wait complete. 2024-04-25T09:14:28.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:14:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.893+0000 7f394b000640 1 Processor -- start 2024-04-25T09:14:28.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.894+0000 7f394b000640 1 -- start start 2024-04-25T09:14:28.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.895+0000 7f394b000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394415dd50 0x7f39440d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.895+0000 7f394b000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440ce780 con 0x7f39440d3960 2024-04-25T09:14:28.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.895+0000 7f394b000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39440d3440 con 0x7f39440d2fb0 2024-04-25T09:14:28.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.895+0000 7f394b000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39440ce1c0 con 0x7f394415dd50 2024-04-25T09:14:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.895+0000 7f394a600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f39440d3960 0x7f39440d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52180/0 (socket says 172.21.15.62:52180) 2024-04-25T09:14:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.895+0000 7f394a600640 1 -- 172.21.15.62:0/3875958678 learned_addr learned my addr 172.21.15.62:0/3875958678 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:28.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.895+0000 7f3949c00640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394415dd50 0x7f39440d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470601023 0 0) 0x7f39440ce780 con 0x7f39440d3960 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f392c003520 con 0x7f39440d3960 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 34259765 0 0) 0x7f392c003520 con 0x7f39440d3960 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394415dd50 msgr2=0x7f39440d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394415dd50 0x7f39440d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 >> v1:172.21.15.121:6789/0 conn(0x7f39440d2fb0 legacy=0x7f39440ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440ce4a0 con 0x7f39440d3960 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f393c0019f0 con 0x7f39440d3960 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 >> v1:172.21.15.62:6789/0 conn(0x7f39440d3960 legacy=0x7f39440d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c0039e0 0x7f392c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394415dd50 0x7f392c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440ce1c0 con 0x7f392c004800 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c00ba70 con 0x7f392c0039e0 2024-04-25T09:14:28.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.896+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c00bc70 con 0x7f394415dd50 2024-04-25T09:14:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.897+0000 7f3949200640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.897+0000 7f394a600640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394415dd50 0x7f392c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.897+0000 7f3949c00640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c0039e0 0x7f392c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.897+0000 7f3949200640 1 -- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394415dd50 msgr2=0x7f392c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.897+0000 7f3949200640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394415dd50 0x7f392c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.897+0000 7f3949200640 1 -- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c0039e0 msgr2=0x7f392c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.897+0000 7f3949200640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c0039e0 0x7f392c003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.897+0000 7f3949200640 1 -- 172.21.15.62:0/3875958678 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c003520 con 0x7f392c004800 2024-04-25T09:14:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.898+0000 7f3949200640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392c004800 0x7f392c004c80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3930001720 tx=0x7f392c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9dd414b527d317 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.898+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930001370 con 0x7f392c004800 2024-04-25T09:14:28.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39300052d0 con 0x7f392c004800 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f3943e00640 1 -- 172.21.15.62:0/3875958678 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393002ab90 con 0x7f392c004800 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f394b000640 1 -- 172.21.15.62:0/3875958678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392c004800 msgr2=0x7f392c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f394b000640 1 --2- 172.21.15.62:0/3875958678 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f392c004800 0x7f392c004c80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3930001720 tx=0x7f392c005830 comp rx=0 tx=0).stop 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f394a600640 1 -- 172.21.15.62:0/3875958678 reap_dead start 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f394b000640 1 -- 172.21.15.62:0/3875958678 shutdown_connections 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f394b000640 1 -- 172.21.15.62:0/3875958678 >> 172.21.15.62:0/3875958678 conn(0x7f394406b000 msgr2=0x7f394406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f394b000640 1 -- 172.21.15.62:0/3875958678 shutdown_connections 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.899+0000 7f394b000640 1 -- 172.21.15.62:0/3875958678 wait complete. 2024-04-25T09:14:28.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.901+0000 7f394b000640 1 Processor -- start 2024-04-25T09:14:28.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.901+0000 7f394b000640 1 -- start start 2024-04-25T09:14:28.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.902+0000 7f394b000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c004800 0x7f394414ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.902+0000 7f394b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f394414bfc0 0x7f394414b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.902+0000 7f394b000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394414f4b0 0x7f394414f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.902+0000 7f394b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440d2710 con 0x7f394414bfc0 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.902+0000 7f394b000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440d2590 con 0x7f392c004800 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.902+0000 7f394b000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39440d2890 con 0x7f394414f4b0 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f394414bfc0 0x7f394414b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f394414bfc0 0x7f394414b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54048/0 (socket says 172.21.15.62:54048) 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949200640 1 -- 172.21.15.62:0/3564785212 learned_addr learned my addr 172.21.15.62:0/3564785212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949c00640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c004800 0x7f394414ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f394a600640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394414f4b0 0x7f394414f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949200640 1 -- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394414f4b0 msgr2=0x7f394414f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949200640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394414f4b0 0x7f394414f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949200640 1 -- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c004800 msgr2=0x7f394414ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949200640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c004800 0x7f394414ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:28.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f3949200640 1 -- 172.21.15.62:0/3564785212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394414efa0 con 0x7f394414bfc0 2024-04-25T09:14:28.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.903+0000 7f394a600640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394414f4b0 0x7f394414f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:28.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.904+0000 7f3949200640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f394414bfc0 0x7f394414b770 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f3930005040 tx=0x7f3930001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:28.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.904+0000 7f3941600640 1 -- 172.21.15.62:0/3564785212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930040020 con 0x7f394414bfc0 2024-04-25T09:14:28.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.904+0000 7f3941600640 1 -- 172.21.15.62:0/3564785212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393002be40 con 0x7f394414bfc0 2024-04-25T09:14:28.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.904+0000 7f3941600640 1 -- 172.21.15.62:0/3564785212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930001370 con 0x7f394414bfc0 2024-04-25T09:14:28.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.904+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394414bcb0 con 0x7f394414bfc0 2024-04-25T09:14:28.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.904+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394414c740 con 0x7f394414bfc0 2024-04-25T09:14:28.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.905+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39440cf5a0 con 0x7f394414bfc0 2024-04-25T09:14:28.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.906+0000 7f3941600640 1 -- 172.21.15.62:0/3564785212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3930043020 con 0x7f394414bfc0 2024-04-25T09:14:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.907+0000 7f3941600640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f392406b500 0x7f392406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.907+0000 7f3941600640 1 -- 172.21.15.62:0/3564785212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f39300aa130 con 0x7f394414bfc0 2024-04-25T09:14:28.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.908+0000 7f3949c00640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f392406b500 0x7f392406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.909+0000 7f3949c00640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f392406b500 0x7f392406d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3938009a10 tx=0x7f3938008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:28.912+0000 7f3941600640 1 -- 172.21.15.62:0/3564785212 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39440cf5a0 con 0x7f394414bfc0 2024-04-25T09:14:29.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.115+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f394404ef20 con 0x7f394414bfc0 2024-04-25T09:14:29.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.116+0000 7f3941600640 1 -- 172.21.15.62:0/3564785212 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7f393004a030 con 0x7f394414bfc0 2024-04-25T09:14:29.103 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f392406b500 msgr2=0x7f392406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f392406b500 0x7f392406d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3938009a10 tx=0x7f3938008040 comp rx=0 tx=0).stop 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f394414bfc0 msgr2=0x7f394414b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f394414bfc0 0x7f394414b770 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f3930005040 tx=0x7f3930001120 comp rx=0 tx=0).stop 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 shutdown_connections 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f392406b500 0x7f392406d9c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f394414bfc0 0x7f394414b770 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f394414f4b0 0x7f394414f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 --2- 172.21.15.62:0/3564785212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f392c004800 0x7f394414ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 >> 172.21.15.62:0/3564785212 conn(0x7f394406b000 msgr2=0x7f39440cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 shutdown_connections 2024-04-25T09:14:29.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.120+0000 7f394b000640 1 -- 172.21.15.62:0/3564785212 wait complete. 2024-04-25T09:14:29.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:14:29.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.349+0000 7f4127000640 1 Processor -- start 2024-04-25T09:14:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.349+0000 7f4127000640 1 -- start start 2024-04-25T09:14:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.350+0000 7f4127000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 0x7f41200d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4127000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f41200d2fb0 0x7f41200ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4127000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41200ce780 con 0x7f41200d3960 2024-04-25T09:14:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4127000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41200d3440 con 0x7f41200d2fb0 2024-04-25T09:14:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4127000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41200ce1c0 con 0x7f412015dd50 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4125200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f41200d2fb0 0x7f41200ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4125200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f41200d2fb0 0x7f41200ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54058/0 (socket says 172.21.15.62:54058) 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4125200640 1 -- 172.21.15.62:0/3885460999 learned_addr learned my addr 172.21.15.62:0/3885460999 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4125200640 1 -- 172.21.15.62:0/3885460999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 msgr2=0x7f41200d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4125c00640 1 --2- 172.21.15.62:0/3885460999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 0x7f41200d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4125200640 1 --2- 172.21.15.62:0/3885460999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 0x7f41200d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.351+0000 7f4125200640 1 -- 172.21.15.62:0/3885460999 >> v1:172.21.15.121:6789/0 conn(0x7f41200d3960 legacy=0x7f41200d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.352+0000 7f4125200640 1 -- 172.21.15.62:0/3885460999 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200ce4a0 con 0x7f41200d2fb0 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.353+0000 7f4125200640 1 --2- 172.21.15.62:0/3885460999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41200d2fb0 0x7f41200ce9c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f410c009870 tx=0x7f410c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbe64179a43a657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.353+0000 7f411fe00640 1 -- 172.21.15.62:0/3885460999 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c037070 con 0x7f41200d2fb0 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.353+0000 7f411fe00640 1 -- 172.21.15.62:0/3885460999 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c032440 con 0x7f41200d2fb0 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.353+0000 7f411fe00640 1 -- 172.21.15.62:0/3885460999 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c032810 con 0x7f41200d2fb0 2024-04-25T09:14:29.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.353+0000 7f4127000640 1 -- 172.21.15.62:0/3885460999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41200d2fb0 msgr2=0x7f41200ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.353+0000 7f4127000640 1 --2- 172.21.15.62:0/3885460999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41200d2fb0 0x7f41200ce9c0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f410c009870 tx=0x7f410c02a210 comp rx=0 tx=0).stop 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.354+0000 7f4127000640 1 -- 172.21.15.62:0/3885460999 shutdown_connections 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.354+0000 7f4127000640 1 --2- 172.21.15.62:0/3885460999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41200d2fb0 0x7f41200ce9c0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.354+0000 7f4127000640 1 --2- 172.21.15.62:0/3885460999 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 0x7f41200d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.354+0000 7f4127000640 1 -- 172.21.15.62:0/3885460999 >> 172.21.15.62:0/3885460999 conn(0x7f412006b000 msgr2=0x7f412006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.354+0000 7f4127000640 1 -- 172.21.15.62:0/3885460999 shutdown_connections 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.354+0000 7f4127000640 1 -- 172.21.15.62:0/3885460999 wait complete. 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.356+0000 7f4127000640 1 Processor -- start 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.356+0000 7f4127000640 1 -- start start 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4127000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41200d3960 0x7f412014fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4127000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 0x7f41201e6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4127000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41201e7d90 0x7f41201e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4127000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200b6b00 con 0x7f41201e7d90 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4127000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200cef00 con 0x7f41200d3960 2024-04-25T09:14:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4127000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41200b6c80 con 0x7f412015dd50 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4126600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41201e7d90 0x7f41201e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4126600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41201e7d90 0x7f41201e66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54068/0 (socket says 172.21.15.62:54068) 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4126600640 1 -- 172.21.15.62:0/6892115 learned_addr learned my addr 172.21.15.62:0/6892115 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4125c00640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41200d3960 0x7f412014fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.357+0000 7f4125200640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 0x7f41201e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f4126600640 1 -- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 msgr2=0x7f41201e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f4126600640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 0x7f41201e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f4126600640 1 -- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41200d3960 msgr2=0x7f412014fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f4126600640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41200d3960 0x7f412014fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f4126600640 1 -- 172.21.15.62:0/6892115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41201e6cf0 con 0x7f41201e7d90 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f4126600640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41201e7d90 0x7f41201e66c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f411800c7b0 tx=0x7f411800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f411d600640 1 -- 172.21.15.62:0/6892115 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118004b60 con 0x7f41201e7d90 2024-04-25T09:14:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f411d600640 1 -- 172.21.15.62:0/6892115 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118004d00 con 0x7f41201e7d90 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41201e6fd0 con 0x7f41201e7d90 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f411d600640 1 -- 172.21.15.62:0/6892115 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411800aa90 con 0x7f41201e7d90 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.358+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41201e7590 con 0x7f41201e7d90 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.359+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412004ef20 con 0x7f41201e7d90 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.360+0000 7f411d600640 1 -- 172.21.15.62:0/6892115 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f411800ac30 con 0x7f41201e7d90 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.361+0000 7f411d600640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f410006b500 0x7f410006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.361+0000 7f411d600640 1 -- 172.21.15.62:0/6892115 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f411808a5b0 con 0x7f41201e7d90 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.362+0000 7f4125c00640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f410006b500 0x7f410006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.363+0000 7f4125c00640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f410006b500 0x7f410006d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4110006fd0 tx=0x7f4110008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.366+0000 7f411d600640 1 -- 172.21.15.62:0/6892115 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4118020070 con 0x7f41201e7d90 2024-04-25T09:14:29.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.570+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41200b9950 con 0x7f41201e7d90 2024-04-25T09:14:29.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.570+0000 7f411d600640 1 -- 172.21.15.62:0/6892115 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4118088e20 con 0x7f41201e7d90 2024-04-25T09:14:29.558 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:14:29.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f410006b500 msgr2=0x7f410006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f410006b500 0x7f410006d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4110006fd0 tx=0x7f4110008040 comp rx=0 tx=0).stop 2024-04-25T09:14:29.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41201e7d90 msgr2=0x7f41201e66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41201e7d90 0x7f41201e66c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f411800c7b0 tx=0x7f411800cc70 comp rx=0 tx=0).stop 2024-04-25T09:14:29.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 shutdown_connections 2024-04-25T09:14:29.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f410006b500 0x7f410006d9c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f41201e7d90 0x7f41201e66c0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f412015dd50 0x7f41201e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 --2- 172.21.15.62:0/6892115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41200d3960 0x7f412014fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.573+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 >> 172.21.15.62:0/6892115 conn(0x7f412006b000 msgr2=0x7f41200cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:29.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.574+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 shutdown_connections 2024-04-25T09:14:29.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.574+0000 7f4127000640 1 -- 172.21.15.62:0/6892115 wait complete. 2024-04-25T09:14:29.581 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:29.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.805+0000 7f7a2fe00640 1 Processor -- start 2024-04-25T09:14:29.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.806+0000 7f7a2fe00640 1 -- start start 2024-04-25T09:14:29.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.807+0000 7f7a2fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a30144c10 0x7f7a3015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.807+0000 7f7a2fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7a300cee40 0x7f7a3006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.807+0000 7f7a2fe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3015faa0 con 0x7f7a3006d730 2024-04-25T09:14:29.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.807+0000 7f7a2fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a30144ff0 con 0x7f7a300cee40 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.807+0000 7f7a2fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30160340 con 0x7f7a30144c10 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.808+0000 7f7a27e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7a300cee40 0x7f7a3006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.808+0000 7f7a27e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7a300cee40 0x7f7a3006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54082/0 (socket says 172.21.15.62:54082) 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.808+0000 7f7a27e00640 1 -- 172.21.15.62:0/1086549222 learned_addr learned my addr 172.21.15.62:0/1086549222 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.808+0000 7f7a2ea00640 1 --2- 172.21.15.62:0/1086549222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a30144c10 0x7f7a3015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.808+0000 7f7a27e00640 1 -- 172.21.15.62:0/1086549222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a30144c10 msgr2=0x7f7a3015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.808+0000 7f7a27e00640 1 --2- 172.21.15.62:0/1086549222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a30144c10 0x7f7a3015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.808+0000 7f7a27e00640 1 -- 172.21.15.62:0/1086549222 >> v1:172.21.15.121:6789/0 conn(0x7f7a3006d730 legacy=0x7f7a300b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.808+0000 7f7a27e00640 1 -- 172.21.15.62:0/1086549222 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3015fd80 con 0x7f7a300cee40 2024-04-25T09:14:29.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.809+0000 7f7a27e00640 1 --2- 172.21.15.62:0/1086549222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a3006d1f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009870 tx=0x7f7a1802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe49603b0d8d21e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:29.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.809+0000 7f7a2e000640 1 -- 172.21.15.62:0/1086549222 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18037070 con 0x7f7a300cee40 2024-04-25T09:14:29.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.809+0000 7f7a2e000640 1 -- 172.21.15.62:0/1086549222 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18032440 con 0x7f7a300cee40 2024-04-25T09:14:29.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.809+0000 7f7a2e000640 1 -- 172.21.15.62:0/1086549222 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18032810 con 0x7f7a300cee40 2024-04-25T09:14:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.810+0000 7f7a2fe00640 1 -- 172.21.15.62:0/1086549222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 msgr2=0x7f7a3006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.810+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/1086549222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a3006d1f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009870 tx=0x7f7a1802a180 comp rx=0 tx=0).stop 2024-04-25T09:14:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.810+0000 7f7a2fe00640 1 -- 172.21.15.62:0/1086549222 shutdown_connections 2024-04-25T09:14:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.810+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/1086549222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a3006d1f0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.810+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/1086549222 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a30144c10 0x7f7a3015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.810+0000 7f7a2fe00640 1 -- 172.21.15.62:0/1086549222 >> 172.21.15.62:0/1086549222 conn(0x7f7a300b14d0 msgr2=0x7f7a300b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.810+0000 7f7a2fe00640 1 -- 172.21.15.62:0/1086549222 shutdown_connections 2024-04-25T09:14:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.810+0000 7f7a2fe00640 1 -- 172.21.15.62:0/1086549222 wait complete. 2024-04-25T09:14:29.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.812+0000 7f7a2fe00640 1 Processor -- start 2024-04-25T09:14:29.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.812+0000 7f7a2fe00640 1 -- start start 2024-04-25T09:14:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.813+0000 7f7a2fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a3006d730 0x7f7a30143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.813+0000 7f7a2fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a300c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.813+0000 7f7a2fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a30144c10 0x7f7a30140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.813+0000 7f7a2fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300cf2a0 con 0x7f7a300cee40 2024-04-25T09:14:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.813+0000 7f7a2fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300b41c0 con 0x7f7a30144c10 2024-04-25T09:14:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.813+0000 7f7a2fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300b5660 con 0x7f7a3006d730 2024-04-25T09:14:29.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.813+0000 7f7a27e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a300c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.814+0000 7f7a27e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a300c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54084/0 (socket says 172.21.15.62:54084) 2024-04-25T09:14:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.814+0000 7f7a27e00640 1 -- 172.21.15.62:0/2988434329 learned_addr learned my addr 172.21.15.62:0/2988434329 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.814+0000 7f7a27e00640 1 -- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a3006d730 msgr2=0x7f7a30143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.814+0000 7f7a27e00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a3006d730 0x7f7a30143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.814+0000 7f7a27e00640 1 -- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a30144c10 msgr2=0x7f7a30140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.814+0000 7f7a27e00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a30144c10 0x7f7a30140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:29.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.814+0000 7f7a27e00640 1 -- 172.21.15.62:0/2988434329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30144350 con 0x7f7a300cee40 2024-04-25T09:14:29.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.814+0000 7f7a27e00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a300c6570 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7a18000c00 tx=0x7f7a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:29.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.815+0000 7f7a26a00640 1 -- 172.21.15.62:0/2988434329 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18037040 con 0x7f7a300cee40 2024-04-25T09:14:29.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.815+0000 7f7a26a00640 1 -- 172.21.15.62:0/2988434329 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18042070 con 0x7f7a300cee40 2024-04-25T09:14:29.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.815+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a300c6e10 con 0x7f7a300cee40 2024-04-25T09:14:29.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.815+0000 7f7a26a00640 1 -- 172.21.15.62:0/2988434329 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18030bd0 con 0x7f7a300cee40 2024-04-25T09:14:29.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.815+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a300c73d0 con 0x7f7a300cee40 2024-04-25T09:14:29.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.816+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3004ef20 con 0x7f7a300cee40 2024-04-25T09:14:29.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.817+0000 7f7a26a00640 1 -- 172.21.15.62:0/2988434329 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a18047460 con 0x7f7a300cee40 2024-04-25T09:14:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.818+0000 7f7a26a00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.818+0000 7f7a26a00640 1 -- 172.21.15.62:0/2988434329 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7a180aaf30 con 0x7f7a300cee40 2024-04-25T09:14:29.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.819+0000 7f7a2ea00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:29.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.820+0000 7f7a2ea00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000a810 tx=0x7f7a20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:29.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:29.823+0000 7f7a26a00640 1 -- 172.21.15.62:0/2988434329 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a18045360 con 0x7f7a300cee40 2024-04-25T09:14:30.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.066+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a3006df50 con 0x7f7a300cee40 2024-04-25T09:14:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.067+0000 7f7a26a00640 1 -- 172.21.15.62:0/2988434329 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7a18079cd0 con 0x7f7a300cee40 2024-04-25T09:14:30.055 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:30.055 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:30.055 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a0c06b4d0 msgr2=0x7f7a0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000a810 tx=0x7f7a20008040 comp rx=0 tx=0).stop 2024-04-25T09:14:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 msgr2=0x7f7a300c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a300c6570 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7a18000c00 tx=0x7f7a180046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 shutdown_connections 2024-04-25T09:14:30.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a300cee40 0x7f7a300c6570 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a30144c10 0x7f7a30140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 --2- 172.21.15.62:0/2988434329 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7a3006d730 0x7f7a30143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.070+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 >> 172.21.15.62:0/2988434329 conn(0x7f7a300b14d0 msgr2=0x7f7a30071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.071+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 shutdown_connections 2024-04-25T09:14:30.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.071+0000 7f7a2fe00640 1 -- 172.21.15.62:0/2988434329 wait complete. 2024-04-25T09:14:30.078 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:30.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.303+0000 7fe3f8e00640 1 Processor -- start 2024-04-25T09:14:30.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.303+0000 7fe3f8e00640 1 -- start start 2024-04-25T09:14:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.304+0000 7fe3f8e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3f406d900 0x7fe3f415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.304+0000 7fe3f8e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f40cee40 0x7fe3f40b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.304+0000 7fe3f8e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f40cf2b0 con 0x7fe3f4144c10 2024-04-25T09:14:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.305+0000 7fe3f8e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f4144ff0 con 0x7fe3f406d900 2024-04-25T09:14:30.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.305+0000 7fe3f8e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4160340 con 0x7fe3f40cee40 2024-04-25T09:14:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.305+0000 7fe3f3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f40cee40 0x7fe3f40b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.305+0000 7fe3f3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f40cee40 0x7fe3f40b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:46338/0 (socket says 172.21.15.62:46338) 2024-04-25T09:14:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.305+0000 7fe3f3e00640 1 -- 172.21.15.62:0/1790188150 learned_addr learned my addr 172.21.15.62:0/1790188150 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:30.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.305+0000 7fe3f2a00640 1 --2- 172.21.15.62:0/1790188150 >> v2:172.21.15.121:3300/0 conn(0x7fe3f406d900 0x7fe3f415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.306+0000 7fe3f3e00640 1 -- 172.21.15.62:0/1790188150 >> v2:172.21.15.121:3300/0 conn(0x7fe3f406d900 msgr2=0x7fe3f415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.306+0000 7fe3f3e00640 1 --2- 172.21.15.62:0/1790188150 >> v2:172.21.15.121:3300/0 conn(0x7fe3f406d900 0x7fe3f415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.306+0000 7fe3f3e00640 1 -- 172.21.15.62:0/1790188150 >> v1:172.21.15.121:6789/0 conn(0x7fe3f4144c10 legacy=0x7fe3f406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.306+0000 7fe3f3e00640 1 -- 172.21.15.62:0/1790188150 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f415faa0 con 0x7fe3f40cee40 2024-04-25T09:14:30.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.306+0000 7fe3f2000640 1 -- 172.21.15.62:0/1790188150 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504347496 0 0) 0x7fe3f40cf2b0 con 0x7fe3f4144c10 2024-04-25T09:14:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.308+0000 7fe3f3e00640 1 --2- 172.21.15.62:0/1790188150 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f40cee40 0x7fe3f40b4650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcaa02d2c89a3cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.308+0000 7fe3f2000640 1 -- 172.21.15.62:0/1790188150 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec037070 con 0x7fe3f40cee40 2024-04-25T09:14:30.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.308+0000 7fe3f2000640 1 -- 172.21.15.62:0/1790188150 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec032440 con 0x7fe3f40cee40 2024-04-25T09:14:30.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.308+0000 7fe3f2000640 1 -- 172.21.15.62:0/1790188150 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec032810 con 0x7fe3f40cee40 2024-04-25T09:14:30.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.309+0000 7fe3f8e00640 1 -- 172.21.15.62:0/1790188150 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f40cee40 msgr2=0x7fe3f40b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.309+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/1790188150 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f40cee40 0x7fe3f40b4650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.309+0000 7fe3f8e00640 1 -- 172.21.15.62:0/1790188150 shutdown_connections 2024-04-25T09:14:30.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.309+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/1790188150 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f40cee40 0x7fe3f40b4650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.309+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/1790188150 >> v2:172.21.15.121:3300/0 conn(0x7fe3f406d900 0x7fe3f415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.309+0000 7fe3f8e00640 1 -- 172.21.15.62:0/1790188150 >> 172.21.15.62:0/1790188150 conn(0x7fe3f40b14d0 msgr2=0x7fe3f40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:30.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.309+0000 7fe3f8e00640 1 -- 172.21.15.62:0/1790188150 shutdown_connections 2024-04-25T09:14:30.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.309+0000 7fe3f8e00640 1 -- 172.21.15.62:0/1790188150 wait complete. 2024-04-25T09:14:30.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.311+0000 7fe3f8e00640 1 Processor -- start 2024-04-25T09:14:30.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.312+0000 7fe3f8e00640 1 -- start start 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f406d900 0x7fe3f415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f8e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f40cee40 0x7fe3f4142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f8e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f4144c10 0x7fe3f4142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f8e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4160060 con 0x7fe3f406d900 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f8e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40b41c0 con 0x7fe3f40cee40 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f8e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40b4b90 con 0x7fe3f4144c10 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f406d900 0x7fe3f415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f406d900 0x7fe3f415af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54098/0 (socket says 172.21.15.62:54098) 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.313+0000 7fe3f3400640 1 -- 172.21.15.62:0/124484331 learned_addr learned my addr 172.21.15.62:0/124484331 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f2a00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f40cee40 0x7fe3f4142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f3e00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f4144c10 0x7fe3f4142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f3400640 1 -- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f4144c10 msgr2=0x7fe3f4142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f3400640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f4144c10 0x7fe3f4142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f3400640 1 -- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f40cee40 msgr2=0x7fe3f4142a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f3400640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f40cee40 0x7fe3f4142a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f3400640 1 -- 172.21.15.62:0/124484331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f415b480 con 0x7fe3f406d900 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f2a00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f40cee40 0x7fe3f4142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.314+0000 7fe3f3400640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f406d900 0x7fe3f415af40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c780 tx=0x7fe3e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.315+0000 7fe3eb400640 1 -- 172.21.15.62:0/124484331 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4003da0 con 0x7fe3f406d900 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.315+0000 7fe3eb400640 1 -- 172.21.15.62:0/124484331 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4005070 con 0x7fe3f406d900 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.315+0000 7fe3eb400640 1 -- 172.21.15.62:0/124484331 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4005420 con 0x7fe3f406d900 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.315+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f4134b10 con 0x7fe3f406d900 2024-04-25T09:14:30.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.315+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f4143910 con 0x7fe3f406d900 2024-04-25T09:14:30.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.316+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f404ef20 con 0x7fe3f406d900 2024-04-25T09:14:30.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.317+0000 7fe3eb400640 1 -- 172.21.15.62:0/124484331 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3e40055c0 con 0x7fe3f406d900 2024-04-25T09:14:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.318+0000 7fe3eb400640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe3d006b4d0 0x7fe3d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.318+0000 7fe3eb400640 1 -- 172.21.15.62:0/124484331 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe3e408a220 con 0x7fe3f406d900 2024-04-25T09:14:30.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.319+0000 7fe3f2a00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe3d006b4d0 0x7fe3d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.320+0000 7fe3f2a00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe3d006b4d0 0x7fe3d006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe3f415ef90 tx=0x7fe3dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:30.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.322+0000 7fe3eb400640 1 -- 172.21.15.62:0/124484331 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3e4058fc0 con 0x7fe3f406d900 2024-04-25T09:14:30.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.523+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3f40b1a30 con 0x7fe3f406d900 2024-04-25T09:14:30.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.525+0000 7fe3eb400640 1 -- 172.21.15.62:0/124484331 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16703 (secure 0 0 0) 0x7fe3e40588c0 con 0x7fe3f406d900 2024-04-25T09:14:30.513 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:30.513 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":111,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:02.867113+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:13:29.095930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:13:30.242813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.6","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:30.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.527+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe3d006b4d0 msgr2=0x7fe3d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.527+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe3d006b4d0 0x7fe3d006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe3f415ef90 tx=0x7fe3dc008040 comp rx=0 tx=0).stop 2024-04-25T09:14:30.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.527+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f406d900 msgr2=0x7fe3f415af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.527+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f406d900 0x7fe3f415af40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c780 tx=0x7fe3e400cc40 comp rx=0 tx=0).stop 2024-04-25T09:14:30.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.528+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 shutdown_connections 2024-04-25T09:14:30.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.528+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe3d006b4d0 0x7fe3d006d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.528+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe3f4144c10 0x7fe3f4142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.528+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f40cee40 0x7fe3f4142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.528+0000 7fe3f8e00640 1 --2- 172.21.15.62:0/124484331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3f406d900 0x7fe3f415af40 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.528+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 >> 172.21.15.62:0/124484331 conn(0x7fe3f40b14d0 msgr2=0x7fe3f40cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.528+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 shutdown_connections 2024-04-25T09:14:30.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.528+0000 7fe3f8e00640 1 -- 172.21.15.62:0/124484331 wait complete. 2024-04-25T09:14:30.535 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:14:30.536 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:30.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.740+0000 7f9d45000640 1 Processor -- start 2024-04-25T09:14:30.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.740+0000 7f9d45000640 1 -- start start 2024-04-25T09:14:30.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d45000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d40160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d45000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9d400b27a0 0x7f9d40161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d45000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d400d3110 con 0x7f9d400b6a90 2024-04-25T09:14:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d45000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d40161780 con 0x7f9d400b27a0 2024-04-25T09:14:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d45000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400cedc0 con 0x7f9d400b56f0 2024-04-25T09:14:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d3fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9d400b27a0 0x7f9d40161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d3fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9d400b27a0 0x7f9d40161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54110/0 (socket says 172.21.15.62:54110) 2024-04-25T09:14:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d3fe00640 1 -- 172.21.15.62:0/1070933612 learned_addr learned my addr 172.21.15.62:0/1070933612 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:30.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.742+0000 7f9d3f400640 1 --2- 172.21.15.62:0/1070933612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d40160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.743+0000 7f9d3fe00640 1 -- 172.21.15.62:0/1070933612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 msgr2=0x7f9d40160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:30.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.743+0000 7f9d3fe00640 1 --2- 172.21.15.62:0/1070933612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d40160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.743+0000 7f9d3fe00640 1 -- 172.21.15.62:0/1070933612 >> v1:172.21.15.121:6789/0 conn(0x7f9d400b6a90 legacy=0x7f9d40161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.743+0000 7f9d3fe00640 1 -- 172.21.15.62:0/1070933612 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d400d33f0 con 0x7f9d400b27a0 2024-04-25T09:14:30.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.744+0000 7f9d3fe00640 1 --2- 172.21.15.62:0/1070933612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b27a0 0x7f9d40161240 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009870 tx=0x7f9d3802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdc13f86bd7b6a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:30.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.744+0000 7f9d3e000640 1 -- 172.21.15.62:0/1070933612 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38037070 con 0x7f9d400b27a0 2024-04-25T09:14:30.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.744+0000 7f9d3e000640 1 -- 172.21.15.62:0/1070933612 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d38032440 con 0x7f9d400b27a0 2024-04-25T09:14:30.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.744+0000 7f9d3e000640 1 -- 172.21.15.62:0/1070933612 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38032810 con 0x7f9d400b27a0 2024-04-25T09:14:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.745+0000 7f9d45000640 1 -- 172.21.15.62:0/1070933612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b27a0 msgr2=0x7f9d40161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.745+0000 7f9d45000640 1 --2- 172.21.15.62:0/1070933612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b27a0 0x7f9d40161240 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009870 tx=0x7f9d3802a250 comp rx=0 tx=0).stop 2024-04-25T09:14:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.745+0000 7f9d45000640 1 -- 172.21.15.62:0/1070933612 shutdown_connections 2024-04-25T09:14:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.745+0000 7f9d45000640 1 --2- 172.21.15.62:0/1070933612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b27a0 0x7f9d40161240 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.745+0000 7f9d45000640 1 --2- 172.21.15.62:0/1070933612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d40160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.745+0000 7f9d45000640 1 -- 172.21.15.62:0/1070933612 >> 172.21.15.62:0/1070933612 conn(0x7f9d40066d10 msgr2=0x7f9d40067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.745+0000 7f9d45000640 1 -- 172.21.15.62:0/1070933612 shutdown_connections 2024-04-25T09:14:30.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.745+0000 7f9d45000640 1 -- 172.21.15.62:0/1070933612 wait complete. 2024-04-25T09:14:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.747+0000 7f9d45000640 1 Processor -- start 2024-04-25T09:14:30.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.747+0000 7f9d45000640 1 -- start start 2024-04-25T09:14:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d45000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d400b27a0 0x7f9d4015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d45000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d401f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d45000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b6a90 0x7f9d401f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d45000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400b2d60 con 0x7f9d400b6a90 2024-04-25T09:14:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d45000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400cea50 con 0x7f9d400b27a0 2024-04-25T09:14:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d45000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400b2ee0 con 0x7f9d400b56f0 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d3fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b6a90 0x7f9d401f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d3fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b6a90 0x7f9d401f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54122/0 (socket says 172.21.15.62:54122) 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.748+0000 7f9d3fe00640 1 -- 172.21.15.62:0/1828671326 learned_addr learned my addr 172.21.15.62:0/1828671326 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3f400640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d400b27a0 0x7f9d4015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3ea00640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d401f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3fe00640 1 -- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 msgr2=0x7f9d401f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3fe00640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d401f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3fe00640 1 -- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d400b27a0 msgr2=0x7f9d4015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3fe00640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d400b27a0 0x7f9d4015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3fe00640 1 -- 172.21.15.62:0/1828671326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d401f6dd0 con 0x7f9d400b6a90 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3f400640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d400b27a0 0x7f9d4015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3ea00640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d401f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d3fe00640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b6a90 0x7f9d401f6890 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009870 tx=0x7f9d380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:30.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d37400640 1 -- 172.21.15.62:0/1828671326 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38037040 con 0x7f9d400b6a90 2024-04-25T09:14:30.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d37400640 1 -- 172.21.15.62:0/1828671326 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d38042070 con 0x7f9d400b6a90 2024-04-25T09:14:30.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d37400640 1 -- 172.21.15.62:0/1828671326 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38030920 con 0x7f9d400b6a90 2024-04-25T09:14:30.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.749+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d401f3b30 con 0x7f9d400b6a90 2024-04-25T09:14:30.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.750+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d401f40f0 con 0x7f9d400b6a90 2024-04-25T09:14:30.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.751+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4004ef20 con 0x7f9d400b6a90 2024-04-25T09:14:30.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.752+0000 7f9d37400640 1 -- 172.21.15.62:0/1828671326 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d38030ac0 con 0x7f9d400b6a90 2024-04-25T09:14:30.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.753+0000 7f9d37400640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d1c06b4d0 0x7f9d1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:30.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.753+0000 7f9d37400640 1 -- 172.21.15.62:0/1828671326 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9d380aab20 con 0x7f9d400b6a90 2024-04-25T09:14:30.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.754+0000 7f9d3f400640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d1c06b4d0 0x7f9d1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:30.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.755+0000 7f9d3f400640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d1c06b4d0 0x7f9d1c06d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9d30007330 tx=0x7f9d3000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:30.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.757+0000 7f9d37400640 1 -- 172.21.15.62:0/1828671326 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d380798c0 con 0x7f9d400b6a90 2024-04-25T09:14:30.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.989+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d400b6180 con 0x7f9d400b6a90 2024-04-25T09:14:30.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.989+0000 7f9d37400640 1 -- 172.21.15.62:0/1828671326 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9d380791c0 con 0x7f9d400b6a90 2024-04-25T09:14:30.977 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:30.977 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:30.977 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d1c06b4d0 msgr2=0x7f9d1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d1c06b4d0 0x7f9d1c06d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9d30007330 tx=0x7f9d3000b040 comp rx=0 tx=0).stop 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b6a90 msgr2=0x7f9d401f6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b6a90 0x7f9d401f6890 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009870 tx=0x7f9d380046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 shutdown_connections 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9d1c06b4d0 0x7f9d1c06d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9d400b6a90 0x7f9d401f6890 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9d400b56f0 0x7f9d401f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 --2- 172.21.15.62:0/1828671326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d400b27a0 0x7f9d4015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:30.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 >> 172.21.15.62:0/1828671326 conn(0x7f9d40066d10 msgr2=0x7f9d400b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:30.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 shutdown_connections 2024-04-25T09:14:30.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:30.992+0000 7f9d45000640 1 -- 172.21.15.62:0/1828671326 wait complete. 2024-04-25T09:14:30.998 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:31.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.211+0000 7fb7e6800640 1 Processor -- start 2024-04-25T09:14:31.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.211+0000 7fb7e6800640 1 -- start start 2024-04-25T09:14:31.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.212+0000 7fb7e6800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb7e006d900 0x7fb7e015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.212+0000 7fb7e6800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb7e00cee40 0x7fb7e00b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.212+0000 7fb7e6800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e015faa0 con 0x7fb7e0144c10 2024-04-25T09:14:31.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.212+0000 7fb7e6800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e0144ff0 con 0x7fb7e006d900 2024-04-25T09:14:31.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.212+0000 7fb7e6800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e0160340 con 0x7fb7e00cee40 2024-04-25T09:14:31.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.213+0000 7fb7e4a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb7e006d900 0x7fb7e015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.213+0000 7fb7e4a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb7e006d900 0x7fb7e015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54132/0 (socket says 172.21.15.62:54132) 2024-04-25T09:14:31.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.213+0000 7fb7e4a00640 1 -- 172.21.15.62:0/859536735 learned_addr learned my addr 172.21.15.62:0/859536735 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:31.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.213+0000 7fb7e4a00640 1 -- 172.21.15.62:0/859536735 >> v2:172.21.15.121:3300/0 conn(0x7fb7e00cee40 msgr2=0x7fb7e00b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:31.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.213+0000 7fb7e4a00640 1 --2- 172.21.15.62:0/859536735 >> v2:172.21.15.121:3300/0 conn(0x7fb7e00cee40 0x7fb7e00b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.213+0000 7fb7e4a00640 1 -- 172.21.15.62:0/859536735 >> v1:172.21.15.121:6789/0 conn(0x7fb7e0144c10 legacy=0x7fb7e006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:31.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.213+0000 7fb7e4a00640 1 -- 172.21.15.62:0/859536735 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e015fd80 con 0x7fb7e006d900 2024-04-25T09:14:31.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.215+0000 7fb7e4a00640 1 --2- 172.21.15.62:0/859536735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e006d900 0x7fb7e015f450 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb7e015f390 tx=0x7fb7cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b87a538fec3a405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:31.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.215+0000 7fb7dfe00640 1 -- 172.21.15.62:0/859536735 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc037070 con 0x7fb7e006d900 2024-04-25T09:14:31.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.215+0000 7fb7dfe00640 1 -- 172.21.15.62:0/859536735 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc032440 con 0x7fb7e006d900 2024-04-25T09:14:31.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.215+0000 7fb7dfe00640 1 -- 172.21.15.62:0/859536735 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc032810 con 0x7fb7e006d900 2024-04-25T09:14:31.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.216+0000 7fb7e6800640 1 -- 172.21.15.62:0/859536735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e006d900 msgr2=0x7fb7e015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.216+0000 7fb7e6800640 1 --2- 172.21.15.62:0/859536735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e006d900 0x7fb7e015f450 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb7e015f390 tx=0x7fb7cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.216+0000 7fb7e6800640 1 -- 172.21.15.62:0/859536735 shutdown_connections 2024-04-25T09:14:31.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.216+0000 7fb7e6800640 1 --2- 172.21.15.62:0/859536735 >> v2:172.21.15.121:3300/0 conn(0x7fb7e00cee40 0x7fb7e00b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.216+0000 7fb7e6800640 1 --2- 172.21.15.62:0/859536735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e006d900 0x7fb7e015f450 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.216+0000 7fb7e6800640 1 -- 172.21.15.62:0/859536735 >> 172.21.15.62:0/859536735 conn(0x7fb7e00b14d0 msgr2=0x7fb7e00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:31.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.216+0000 7fb7e6800640 1 -- 172.21.15.62:0/859536735 shutdown_connections 2024-04-25T09:14:31.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.216+0000 7fb7e6800640 1 -- 172.21.15.62:0/859536735 wait complete. 2024-04-25T09:14:31.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.218+0000 7fb7e6800640 1 Processor -- start 2024-04-25T09:14:31.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.218+0000 7fb7e6800640 1 -- start start 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e006d900 0x7fb7e013fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e00cee40 0x7fb7e0136f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e6800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e0144c10 0x7fb7e01374c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e6800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00cf270 con 0x7fb7e00cee40 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e6800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00b41c0 con 0x7fb7e006d900 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e6800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00b4b90 con 0x7fb7e0144c10 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e00cee40 0x7fb7e0136f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e00cee40 0x7fb7e0136f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54144/0 (socket says 172.21.15.62:54144) 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e4a00640 1 -- 172.21.15.62:0/3887192979 learned_addr learned my addr 172.21.15.62:0/3887192979 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e5400640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e006d900 0x7fb7e013fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e5e00640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e0144c10 0x7fb7e01374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.219+0000 7fb7e4a00640 1 -- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e0144c10 msgr2=0x7fb7e01374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7e4a00640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e0144c10 0x7fb7e01374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7e4a00640 1 -- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e006d900 msgr2=0x7fb7e013fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7e4a00640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e006d900 0x7fb7e013fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7e4a00640 1 -- 172.21.15.62:0/3887192979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0142d30 con 0x7fb7e00cee40 2024-04-25T09:14:31.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7e5400640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e006d900 0x7fb7e013fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:31.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7e5e00640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e0144c10 0x7fb7e01374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:31.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7e4a00640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e00cee40 0x7fb7e0136f80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc000c00 tx=0x7fb7cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:31.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7dd600640 1 -- 172.21.15.62:0/3887192979 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc041070 con 0x7fb7e00cee40 2024-04-25T09:14:31.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7dd600640 1 -- 172.21.15.62:0/3887192979 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc03c870 con 0x7fb7e00cee40 2024-04-25T09:14:31.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7dd600640 1 -- 172.21.15.62:0/3887192979 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc037040 con 0x7fb7e00cee40 2024-04-25T09:14:31.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.220+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e0143010 con 0x7fb7e00cee40 2024-04-25T09:14:31.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.221+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e01435d0 con 0x7fb7e00cee40 2024-04-25T09:14:31.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.222+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e004ef20 con 0x7fb7e00cee40 2024-04-25T09:14:31.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.224+0000 7fb7dd600640 1 -- 172.21.15.62:0/3887192979 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7cc044070 con 0x7fb7e00cee40 2024-04-25T09:14:31.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.224+0000 7fb7dd600640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7c006b4d0 0x7fb7c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.225+0000 7fb7dd600640 1 -- 172.21.15.62:0/3887192979 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb7cc0ab220 con 0x7fb7e00cee40 2024-04-25T09:14:31.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.225+0000 7fb7e5400640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7c006b4d0 0x7fb7c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.227+0000 7fb7e5400640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7c006b4d0 0x7fb7c006d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4007e90 tx=0x7fb7d40073a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:31.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.228+0000 7fb7dd600640 1 -- 172.21.15.62:0/3887192979 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7cc07a1f0 con 0x7fb7e00cee40 2024-04-25T09:14:31.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.465+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7e00b22b0 con 0x7fb7e00cee40 2024-04-25T09:14:31.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.467+0000 7fb7dd600640 1 -- 172.21.15.62:0/3887192979 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb7cc079af0 con 0x7fb7e00cee40 2024-04-25T09:14:31.455 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:31.455 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":71,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:13:31.220283+0000","modified":"2024-04-25T09:13:34.273693+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7800},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7800":{"gid":7800,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.121:6836/2489379768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2489379768},{"type":"v1","addr":"172.21.15.121:6836","nonce":2489379768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:14:31.455 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 71 2024-04-25T09:14:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7c006b4d0 msgr2=0x7fb7c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7c006b4d0 0x7fb7c006d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4007e90 tx=0x7fb7d40073a0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e00cee40 msgr2=0x7fb7e0136f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e00cee40 0x7fb7e0136f80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc000c00 tx=0x7fb7cc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 shutdown_connections 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb7c006b4d0 0x7fb7c006d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb7e00cee40 0x7fb7e0136f80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb7e0144c10 0x7fb7e01374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 --2- 172.21.15.62:0/3887192979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7e006d900 0x7fb7e013fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 >> 172.21.15.62:0/3887192979 conn(0x7fb7e00b14d0 msgr2=0x7fb7e00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 shutdown_connections 2024-04-25T09:14:31.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.470+0000 7fb7e6800640 1 -- 172.21.15.62:0/3887192979 wait complete. 2024-04-25T09:14:31.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:31.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.699+0000 7f8e5be00640 1 Processor -- start 2024-04-25T09:14:31.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.700+0000 7f8e5be00640 1 -- start start 2024-04-25T09:14:31.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.701+0000 7f8e5be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e5c0b6a90 0x7f8e5c0d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.701+0000 7f8e5be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5c0d3110 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.701+0000 7f8e5be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5c0d33f0 con 0x7f8e5c0b56f0 2024-04-25T09:14:31.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.701+0000 7f8e5be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0cedc0 con 0x7f8e5c0b6a90 2024-04-25T09:14:31.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.701+0000 7f8e52000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e5c0b6a90 0x7f8e5c0d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.702+0000 7f8e52000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8e5c0b6a90 0x7f8e5c0d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54160/0 (socket says 172.21.15.62:54160) 2024-04-25T09:14:31.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.702+0000 7f8e52000640 1 -- 172.21.15.62:0/815810959 learned_addr learned my addr 172.21.15.62:0/815810959 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.702+0000 7f8e52000640 1 -- 172.21.15.62:0/815810959 >> v1:172.21.15.121:6789/0 conn(0x7f8e5c0b56f0 legacy=0x7f8e5c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.702+0000 7f8e52000640 1 -- 172.21.15.62:0/815810959 >> v1:172.21.15.62:6789/0 conn(0x7f8e5c0b27a0 legacy=0x7f8e5c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.702+0000 7f8e52000640 1 -- 172.21.15.62:0/815810959 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0d36d0 con 0x7f8e5c0b6a90 2024-04-25T09:14:31.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.703+0000 7f8e52000640 1 --2- 172.21.15.62:0/815810959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b6a90 0x7f8e5c0d1fe0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009870 tx=0x7f8e4402a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f837167df072cb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.703+0000 7f8e5a000640 1 -- 172.21.15.62:0/815810959 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44037070 con 0x7f8e5c0b6a90 2024-04-25T09:14:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5a000640 1 -- 172.21.15.62:0/815810959 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44032640 con 0x7f8e5c0b6a90 2024-04-25T09:14:31.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5a000640 1 -- 172.21.15.62:0/815810959 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44032a10 con 0x7f8e5c0b6a90 2024-04-25T09:14:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5be00640 1 -- 172.21.15.62:0/815810959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b6a90 msgr2=0x7f8e5c0d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5be00640 1 --2- 172.21.15.62:0/815810959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b6a90 0x7f8e5c0d1fe0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009870 tx=0x7f8e4402a2f0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5be00640 1 -- 172.21.15.62:0/815810959 shutdown_connections 2024-04-25T09:14:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5be00640 1 --2- 172.21.15.62:0/815810959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b6a90 0x7f8e5c0d1fe0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5be00640 1 -- 172.21.15.62:0/815810959 >> 172.21.15.62:0/815810959 conn(0x7f8e5c066d10 msgr2=0x7f8e5c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5be00640 1 -- 172.21.15.62:0/815810959 shutdown_connections 2024-04-25T09:14:31.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.704+0000 7f8e5be00640 1 -- 172.21.15.62:0/815810959 wait complete. 2024-04-25T09:14:31.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.706+0000 7f8e5be00640 1 Processor -- start 2024-04-25T09:14:31.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.706+0000 7f8e5be00640 1 -- start start 2024-04-25T09:14:31.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.707+0000 7f8e5be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b27a0 0x7f8e5c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.707+0000 7f8e5be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e5c0b56f0 0x7f8e5c1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.707+0000 7f8e5be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e5c0b6a90 0x7f8e5c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.707+0000 7f8e5be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b2ee0 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.707+0000 7f8e5be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b2d60 con 0x7f8e5c0b6a90 2024-04-25T09:14:31.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.707+0000 7f8e5be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0d2c30 con 0x7f8e5c0b56f0 2024-04-25T09:14:31.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.707+0000 7f8e5aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b27a0 0x7f8e5c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.707+0000 7f8e5aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b27a0 0x7f8e5c1574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54166/0 (socket says 172.21.15.62:54166) 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e5aa00640 1 -- 172.21.15.62:0/1653194601 learned_addr learned my addr 172.21.15.62:0/1653194601 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e52000640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e5c0b56f0 0x7f8e5c1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e5b400640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e5c0b6a90 0x7f8e5c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e5aa00640 1 -- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e5c0b56f0 msgr2=0x7f8e5c1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e5aa00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e5c0b56f0 0x7f8e5c1593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e5aa00640 1 -- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e5c0b6a90 msgr2=0x7f8e5c15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e5aa00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e5c0b6a90 0x7f8e5c15a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e5aa00640 1 -- 172.21.15.62:0/1653194601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c158930 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e52000640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e5c0b56f0 0x7f8e5c1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e5aa00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b27a0 0x7f8e5c1574c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00c450 tx=0x7f8e4c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.708+0000 7f8e53e00640 1 -- 172.21.15.62:0/1653194601 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0052d0 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.709+0000 7f8e53e00640 1 -- 172.21.15.62:0/1653194601 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c005470 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.709+0000 7f8e53e00640 1 -- 172.21.15.62:0/1653194601 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c012870 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.709+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c0cbb80 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.709+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c158bf0 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.710+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c04ef20 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.711+0000 7f8e53e00640 1 -- 172.21.15.62:0/1653194601 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e4c012a10 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.712+0000 7f8e53e00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8e3806b4d0 0x7f8e3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.712+0000 7f8e53e00640 1 -- 172.21.15.62:0/1653194601 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8e4c089ef0 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.712+0000 7f8e52000640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8e3806b4d0 0x7f8e3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:31.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.714+0000 7f8e52000640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8e3806b4d0 0x7f8e3806d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8e44034000 tx=0x7f8e4402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:31.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.716+0000 7f8e53e00640 1 -- 172.21.15.62:0/1653194601 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e4c058c90 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.923+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e5c0ce580 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.925+0000 7f8e53e00640 1 -- 172.21.15.62:0/1653194601 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16703 (secure 0 0 0) 0x7f8e4c058590 con 0x7f8e5c0b27a0 2024-04-25T09:14:31.912 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:31.913 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":111,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:02.867113+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:13:29.095930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:13:30.242813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.6","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.927+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8e3806b4d0 msgr2=0x7f8e3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.927+0000 7f8e5be00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8e3806b4d0 0x7f8e3806d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8e44034000 tx=0x7f8e4402e000 comp rx=0 tx=0).stop 2024-04-25T09:14:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.927+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b27a0 msgr2=0x7f8e5c1574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.927+0000 7f8e5be00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b27a0 0x7f8e5c1574c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00c450 tx=0x7f8e4c00c910 comp rx=0 tx=0).stop 2024-04-25T09:14:31.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.928+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 shutdown_connections 2024-04-25T09:14:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.928+0000 7f8e5be00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8e3806b4d0 0x7f8e3806d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.928+0000 7f8e5be00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e5c0b6a90 0x7f8e5c15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.928+0000 7f8e5be00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8e5c0b56f0 0x7f8e5c1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.928+0000 7f8e5be00640 1 --2- 172.21.15.62:0/1653194601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8e5c0b27a0 0x7f8e5c1574c0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.928+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 >> 172.21.15.62:0/1653194601 conn(0x7f8e5c066d10 msgr2=0x7f8e5c0b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.928+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 shutdown_connections 2024-04-25T09:14:31.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:31.928+0000 7f8e5be00640 1 -- 172.21.15.62:0/1653194601 wait complete. 2024-04-25T09:14:31.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:14:32.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.158+0000 7f9bd3c00640 1 Processor -- start 2024-04-25T09:14:32.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.159+0000 7f9bd3c00640 1 -- start start 2024-04-25T09:14:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.160+0000 7f9bd3c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc15dc40 0x7f9bcc0d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.160+0000 7f9bd3c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9bcc0d0f40 0x7f9bcc0d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.160+0000 7f9bd3c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc0d2620 con 0x7f9bcc0d0ae0 2024-04-25T09:14:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.160+0000 7f9bd3c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc0d2900 con 0x7f9bcc0d0f40 2024-04-25T09:14:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.160+0000 7f9bd3c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0d2be0 con 0x7f9bcc15dc40 2024-04-25T09:14:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bcbe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9bcc0d0ae0 0x7f9bcc0d3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52206/0 (socket says 172.21.15.62:52206) 2024-04-25T09:14:32.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bcbe00640 1 -- 172.21.15.62:0/972873986 learned_addr learned my addr 172.21.15.62:0/972873986 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bd1200640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc15dc40 0x7f9bcc0d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bd1c00640 1 --2- 172.21.15.62:0/972873986 >> v2:172.21.15.121:3300/0 conn(0x7f9bcc0d0f40 0x7f9bcc0d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056615961 0 0) 0x7f9bcc0d2620 con 0x7f9bcc0d0ae0 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb4003520 con 0x7f9bcc0d0ae0 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1491186669 0 0) 0x7f9bb4003520 con 0x7f9bcc0d0ae0 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc15dc40 msgr2=0x7f9bcc0d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bcb400640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc15dc40 0x7f9bcc0d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.161+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 >> v2:172.21.15.121:3300/0 conn(0x7f9bcc0d0f40 msgr2=0x7f9bcc0d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 --2- 172.21.15.62:0/972873986 >> v2:172.21.15.121:3300/0 conn(0x7f9bcc0d0f40 0x7f9bcc0d41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0d2ec0 con 0x7f9bcc0d0ae0 2024-04-25T09:14:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f9bb8001670 con 0x7f9bcc0d0ae0 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 >> v1:172.21.15.62:6789/0 conn(0x7f9bcc0d0ae0 legacy=0x7f9bcc0d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb40039b0 0x7f9bb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc15dc40 0x7f9bb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0d2900 con 0x7f9bb4004800 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0d2be0 con 0x7f9bb40039b0 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb400b9c0 con 0x7f9bcc15dc40 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcbe00640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bd1200640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb40039b0 0x7f9bb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bd1c00640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc15dc40 0x7f9bb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcbe00640 1 -- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc15dc40 msgr2=0x7f9bb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcbe00640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc15dc40 0x7f9bb40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcbe00640 1 -- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb40039b0 msgr2=0x7f9bb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.162+0000 7f9bcbe00640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bb40039b0 0x7f9bb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.163+0000 7f9bcbe00640 1 -- 172.21.15.62:0/972873986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb4003520 con 0x7f9bb4004800 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.164+0000 7f9bcbe00640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bb4004c80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800c160 tx=0x7f9bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af3e234f741d5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.164+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8007c80 con 0x7f9bb4004800 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.164+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8005af0 con 0x7f9bb4004800 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.164+0000 7f9bcb400640 1 -- 172.21.15.62:0/972873986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8007070 con 0x7f9bb4004800 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.164+0000 7f9bd3c00640 1 -- 172.21.15.62:0/972873986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 msgr2=0x7f9bb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.164+0000 7f9bd3c00640 1 --2- 172.21.15.62:0/972873986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bb4004c80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800c160 tx=0x7f9bb4005830 comp rx=0 tx=0).stop 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.164+0000 7f9bd1c00640 1 -- 172.21.15.62:0/972873986 reap_dead start 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.165+0000 7f9bd3c00640 1 -- 172.21.15.62:0/972873986 shutdown_connections 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.165+0000 7f9bd3c00640 1 -- 172.21.15.62:0/972873986 >> 172.21.15.62:0/972873986 conn(0x7f9bcc06b000 msgr2=0x7f9bcc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.165+0000 7f9bd3c00640 1 -- 172.21.15.62:0/972873986 shutdown_connections 2024-04-25T09:14:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.165+0000 7f9bd3c00640 1 -- 172.21.15.62:0/972873986 wait complete. 2024-04-25T09:14:32.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.166+0000 7f9bd3c00640 1 Processor -- start 2024-04-25T09:14:32.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.167+0000 7f9bd3c00640 1 -- start start 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd3c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bcc0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd3c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc160830 0x7f9bcc151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd3c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bcc159a90 0x7f9bcc159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd3c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b7d00 con 0x7f9bb4004800 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd3c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0d13d0 con 0x7f9bcc159a90 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd3c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0d36c0 con 0x7f9bcc160830 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bcc0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bcc0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54184/0 (socket says 172.21.15.62:54184) 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1200640 1 -- 172.21.15.62:0/2963270340 learned_addr learned my addr 172.21.15.62:0/2963270340 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1200640 1 -- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc160830 msgr2=0x7f9bcc151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bcc159a90 0x7f9bcc159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1200640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc160830 0x7f9bcc151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1200640 1 -- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bcc159a90 msgr2=0x7f9bcc159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1200640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bcc159a90 0x7f9bcc159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.168+0000 7f9bd1200640 1 -- 172.21.15.62:0/2963270340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0cbda0 con 0x7f9bb4004800 2024-04-25T09:14:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.169+0000 7f9bd1c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bcc159a90 0x7f9bcc159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.169+0000 7f9bd1200640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bcc0cb630 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0011540 tx=0x7f9bc0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:32.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.169+0000 7f9bc8c00640 1 -- 172.21.15.62:0/2963270340 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0015930 con 0x7f9bb4004800 2024-04-25T09:14:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.169+0000 7f9bc8c00640 1 -- 172.21.15.62:0/2963270340 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0015ad0 con 0x7f9bb4004800 2024-04-25T09:14:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.169+0000 7f9bc8c00640 1 -- 172.21.15.62:0/2963270340 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000f460 con 0x7f9bb4004800 2024-04-25T09:14:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.169+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc0cc080 con 0x7f9bb4004800 2024-04-25T09:14:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.169+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc0cc640 con 0x7f9bb4004800 2024-04-25T09:14:32.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.170+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc0729c0 con 0x7f9bb4004800 2024-04-25T09:14:32.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.172+0000 7f9bc8c00640 1 -- 172.21.15.62:0/2963270340 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bc000f600 con 0x7f9bb4004800 2024-04-25T09:14:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.172+0000 7f9bc8c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bac06b500 0x7f9bac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.173+0000 7f9bc8c00640 1 -- 172.21.15.62:0/2963270340 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9bc008a910 con 0x7f9bb4004800 2024-04-25T09:14:32.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.173+0000 7f9bcbe00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bac06b500 0x7f9bac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.174+0000 7f9bcbe00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bac06b500 0x7f9bac06d9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8032000 tx=0x7f9bb8007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:32.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.177+0000 7f9bc8c00640 1 -- 172.21.15.62:0/2963270340 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bc0019030 con 0x7f9bb4004800 2024-04-25T09:14:32.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.416+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9bcc067180 con 0x7f9bb4004800 2024-04-25T09:14:32.405 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:14:32.417+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:14:32.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.421+0000 7f9bc8c00640 1 -- 172.21.15.62:0/2963270340 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9bc00597e0 con 0x7f9bb4004800 2024-04-25T09:14:32.408 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:14:32.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.424+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bac06b500 msgr2=0x7f9bac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.424+0000 7f9bd3c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bac06b500 0x7f9bac06d9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8032000 tx=0x7f9bb8007000 comp rx=0 tx=0).stop 2024-04-25T09:14:32.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.424+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 msgr2=0x7f9bcc0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.424+0000 7f9bd3c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bcc0cb630 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0011540 tx=0x7f9bc0011a00 comp rx=0 tx=0).stop 2024-04-25T09:14:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.425+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 shutdown_connections 2024-04-25T09:14:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.425+0000 7f9bd3c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bac06b500 0x7f9bac06d9c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.425+0000 7f9bd3c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bcc159a90 0x7f9bcc159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.425+0000 7f9bd3c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bcc160830 0x7f9bcc151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.425+0000 7f9bd3c00640 1 --2- 172.21.15.62:0/2963270340 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bb4004800 0x7f9bcc0cb630 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.425+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 >> 172.21.15.62:0/2963270340 conn(0x7f9bcc06b000 msgr2=0x7f9bcc0b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.426+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 shutdown_connections 2024-04-25T09:14:32.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.426+0000 7f9bd3c00640 1 -- 172.21.15.62:0/2963270340 wait complete. 2024-04-25T09:14:32.434 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:14:32.530 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:14:32.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.654+0000 7f49af200640 1 Processor -- start 2024-04-25T09:14:32.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.655+0000 7f49af200640 1 -- start start 2024-04-25T09:14:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.656+0000 7f49af200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b006d9d0 0x7f49b015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.656+0000 7f49af200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f49b015e5f0 0x7f49b00b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.656+0000 7f49af200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b015ee30 con 0x7f49b00ceb50 2024-04-25T09:14:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.656+0000 7f49af200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b01618b0 con 0x7f49b015e5f0 2024-04-25T09:14:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.656+0000 7f49af200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0161b90 con 0x7f49b006d9d0 2024-04-25T09:14:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.656+0000 7f49ae800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f49b015e5f0 0x7f49b00b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.656+0000 7f49ade00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f49b00ceb50 0x7f49b006d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52222/0 (socket says 172.21.15.62:52222) 2024-04-25T09:14:32.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.657+0000 7f49ade00640 1 -- 172.21.15.62:0/4150920527 learned_addr learned my addr 172.21.15.62:0/4150920527 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.657+0000 7f49a7e00640 1 --2- 172.21.15.62:0/4150920527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b006d9d0 0x7f49b015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.657+0000 7f49ae800640 1 -- 172.21.15.62:0/4150920527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b006d9d0 msgr2=0x7f49b015e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.657+0000 7f49ae800640 1 --2- 172.21.15.62:0/4150920527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b006d9d0 0x7f49b015e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.657+0000 7f49ae800640 1 -- 172.21.15.62:0/4150920527 >> v1:172.21.15.62:6789/0 conn(0x7f49b00ceb50 legacy=0x7f49b006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.657+0000 7f49ae800640 1 -- 172.21.15.62:0/4150920527 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00cef30 con 0x7f49b015e5f0 2024-04-25T09:14:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.657+0000 7f49ad400640 1 -- 172.21.15.62:0/4150920527 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537726281 0 0) 0x7f49b015ee30 con 0x7f49b00ceb50 2024-04-25T09:14:32.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.657+0000 7f49a7e00640 1 --2- 172.21.15.62:0/4150920527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b006d9d0 0x7f49b015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.658+0000 7f49ae800640 1 --2- 172.21.15.62:0/4150920527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b015e5f0 0x7f49b00b4550 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009d90 tx=0x7f49a802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110120affe0dd5f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.658+0000 7f49ad400640 1 -- 172.21.15.62:0/4150920527 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8031600 con 0x7f49b015e5f0 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.658+0000 7f49ad400640 1 -- 172.21.15.62:0/4150920527 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a80317a0 con 0x7f49b015e5f0 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49ad400640 1 -- 172.21.15.62:0/4150920527 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8031b70 con 0x7f49b015e5f0 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49af200640 1 -- 172.21.15.62:0/4150920527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b015e5f0 msgr2=0x7f49b00b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49af200640 1 --2- 172.21.15.62:0/4150920527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b015e5f0 0x7f49b00b4550 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009d90 tx=0x7f49a802a5d0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49af200640 1 -- 172.21.15.62:0/4150920527 shutdown_connections 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49af200640 1 --2- 172.21.15.62:0/4150920527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b015e5f0 0x7f49b00b4550 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49af200640 1 --2- 172.21.15.62:0/4150920527 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b006d9d0 0x7f49b015e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49af200640 1 -- 172.21.15.62:0/4150920527 >> 172.21.15.62:0/4150920527 conn(0x7f49b0066f20 msgr2=0x7f49b00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49af200640 1 -- 172.21.15.62:0/4150920527 shutdown_connections 2024-04-25T09:14:32.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.659+0000 7f49af200640 1 -- 172.21.15.62:0/4150920527 wait complete. 2024-04-25T09:14:32.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.661+0000 7f49af200640 1 Processor -- start 2024-04-25T09:14:32.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.661+0000 7f49af200640 1 -- start start 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.662+0000 7f49af200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b006d9d0 0x7f49b013f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.662+0000 7f49af200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b00ceb50 0x7f49b00b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.662+0000 7f49af200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49b015e5f0 0x7f49b0134a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.662+0000 7f49af200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0162510 con 0x7f49b006d9d0 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.662+0000 7f49af200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b0162390 con 0x7f49b015e5f0 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.662+0000 7f49af200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b00b4a90 con 0x7f49b00ceb50 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.662+0000 7f49ade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b006d9d0 0x7f49b013f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49ade00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b006d9d0 0x7f49b013f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54190/0 (socket says 172.21.15.62:54190) 2024-04-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49ade00640 1 -- 172.21.15.62:0/1928770120 learned_addr learned my addr 172.21.15.62:0/1928770120 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49ade00640 1 -- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b00ceb50 msgr2=0x7f49b00b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49ade00640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b00ceb50 0x7f49b00b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49ade00640 1 -- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49b015e5f0 msgr2=0x7f49b0134a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49ade00640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49b015e5f0 0x7f49b0134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49ade00640 1 -- 172.21.15.62:0/1928770120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b0157a10 con 0x7f49b006d9d0 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49ade00640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b006d9d0 0x7f49b013f730 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f49a000c500 tx=0x7f49a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.663+0000 7f49a6000640 1 -- 172.21.15.62:0/1928770120 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0018070 con 0x7f49b006d9d0 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.664+0000 7f49a6000640 1 -- 172.21.15.62:0/1928770120 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a0007560 con 0x7f49b006d9d0 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.664+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b0157cb0 con 0x7f49b006d9d0 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.664+0000 7f49a6000640 1 -- 172.21.15.62:0/1928770120 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0004a20 con 0x7f49b006d9d0 2024-04-25T09:14:32.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.664+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b0157f90 con 0x7f49b006d9d0 2024-04-25T09:14:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.665+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b004ef20 con 0x7f49b006d9d0 2024-04-25T09:14:32.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.666+0000 7f49a6000640 1 -- 172.21.15.62:0/1928770120 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49a001b070 con 0x7f49b006d9d0 2024-04-25T09:14:32.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.667+0000 7f49a6000640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f498c06b4d0 0x7f498c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:32.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.667+0000 7f49a6000640 1 -- 172.21.15.62:0/1928770120 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f49a0089580 con 0x7f49b006d9d0 2024-04-25T09:14:32.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.667+0000 7f49a7e00640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f498c06b4d0 0x7f498c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:32.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.669+0000 7f49a7e00640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f498c06b4d0 0x7f498c06d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4998006fd0 tx=0x7f4998008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:32.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.671+0000 7f49a6000640 1 -- 172.21.15.62:0/1928770120 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49a0058360 con 0x7f49b006d9d0 2024-04-25T09:14:32.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:32.911+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f49b00cf250 con 0x7f49b006d9d0 2024-04-25T09:14:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.425+0000 7f49a6000640 1 -- 172.21.15.62:0/1928770120 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f49a0057c60 con 0x7f49b006d9d0 2024-04-25T09:14:33.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.428+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f498c06b4d0 msgr2=0x7f498c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:33.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f498c06b4d0 0x7f498c06d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4998006fd0 tx=0x7f4998008040 comp rx=0 tx=0).stop 2024-04-25T09:14:33.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b006d9d0 msgr2=0x7f49b013f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:33.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b006d9d0 0x7f49b013f730 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f49a000c500 tx=0x7f49a000c9c0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 shutdown_connections 2024-04-25T09:14:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f498c06b4d0 0x7f498c06d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49b015e5f0 0x7f49b0134a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49b00ceb50 0x7f49b00b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 --2- 172.21.15.62:0/1928770120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49b006d9d0 0x7f49b013f730 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 >> 172.21.15.62:0/1928770120 conn(0x7f49b0066f20 msgr2=0x7f49b0146ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 shutdown_connections 2024-04-25T09:14:33.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.429+0000 7f49af200640 1 -- 172.21.15.62:0/1928770120 wait complete. 2024-04-25T09:14:33.437 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:14:33.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.670+0000 7f0eb2400640 1 Processor -- start 2024-04-25T09:14:33.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.671+0000 7f0eb2400640 1 -- start start 2024-04-25T09:14:33.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.672+0000 7f0eb2400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac0b5c70 0x7f0eac0d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:33.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.672+0000 7f0eb2400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0eac161650 0x7f0eac0d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:33.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.672+0000 7f0eb2400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eac0d1480 con 0x7f0eac161270 2024-04-25T09:14:33.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.672+0000 7f0eb2400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eac0d3900 con 0x7f0eac161650 2024-04-25T09:14:33.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.672+0000 7f0eb2400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0d11a0 con 0x7f0eac0b5c70 2024-04-25T09:14:33.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.673+0000 7f0eb1000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0eac161270 0x7f0eac0b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:52226/0 (socket says 172.21.15.62:52226) 2024-04-25T09:14:33.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.673+0000 7f0eb1000640 1 -- 172.21.15.62:0/1546196860 learned_addr learned my addr 172.21.15.62:0/1546196860 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:33.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.673+0000 7f0eb1a00640 1 --2- 172.21.15.62:0/1546196860 >> v2:172.21.15.62:3300/0 conn(0x7f0eac161650 0x7f0eac0d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:33.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.673+0000 7f0eabe00640 1 --2- 172.21.15.62:0/1546196860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac0b5c70 0x7f0eac0d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:33.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.673+0000 7f0eb1a00640 1 -- 172.21.15.62:0/1546196860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac0b5c70 msgr2=0x7f0eac0d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:33.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.673+0000 7f0eb1a00640 1 --2- 172.21.15.62:0/1546196860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac0b5c70 0x7f0eac0d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.673+0000 7f0eb1a00640 1 -- 172.21.15.62:0/1546196860 >> v1:172.21.15.62:6789/0 conn(0x7f0eac161270 legacy=0x7f0eac0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:33.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.673+0000 7f0eb1a00640 1 -- 172.21.15.62:0/1546196860 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0cee80 con 0x7f0eac161650 2024-04-25T09:14:33.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.674+0000 7f0eb1a00640 1 --2- 172.21.15.62:0/1546196860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac161650 0x7f0eac0d1da0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009d90 tx=0x7f0ea402a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6ff54d6ce64fff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:33.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eab400640 1 -- 172.21.15.62:0/1546196860 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4007740 con 0x7f0eac161650 2024-04-25T09:14:33.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eab400640 1 -- 172.21.15.62:0/1546196860 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea40078e0 con 0x7f0eac161650 2024-04-25T09:14:33.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eab400640 1 -- 172.21.15.62:0/1546196860 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4007cb0 con 0x7f0eac161650 2024-04-25T09:14:33.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eb2400640 1 -- 172.21.15.62:0/1546196860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac161650 msgr2=0x7f0eac0d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:33.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eb2400640 1 --2- 172.21.15.62:0/1546196860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac161650 0x7f0eac0d1da0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009d90 tx=0x7f0ea402a540 comp rx=0 tx=0).stop 2024-04-25T09:14:33.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eb2400640 1 -- 172.21.15.62:0/1546196860 shutdown_connections 2024-04-25T09:14:33.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eb2400640 1 --2- 172.21.15.62:0/1546196860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac161650 0x7f0eac0d1da0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eb2400640 1 --2- 172.21.15.62:0/1546196860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac0b5c70 0x7f0eac0d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eb2400640 1 -- 172.21.15.62:0/1546196860 >> 172.21.15.62:0/1546196860 conn(0x7f0eac066d10 msgr2=0x7f0eac067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:33.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eb2400640 1 -- 172.21.15.62:0/1546196860 shutdown_connections 2024-04-25T09:14:33.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.675+0000 7f0eb2400640 1 -- 172.21.15.62:0/1546196860 wait complete. 2024-04-25T09:14:33.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.677+0000 7f0eb2400640 1 Processor -- start 2024-04-25T09:14:33.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.677+0000 7f0eb2400640 1 -- start start 2024-04-25T09:14:33.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.678+0000 7f0eb2400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac0b5c70 0x7f0eac0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:33.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.678+0000 7f0eb2400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac161270 0x7f0eac157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:33.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.678+0000 7f0eb2400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eac161650 0x7f0eac0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:33.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.678+0000 7f0eb2400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0b2ed0 con 0x7f0eac0b5c70 2024-04-25T09:14:33.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.678+0000 7f0eb2400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0d3590 con 0x7f0eac161650 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.678+0000 7f0eb2400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac0b3050 con 0x7f0eac161270 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac0b5c70 0x7f0eac0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac0b5c70 0x7f0eac0cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54208/0 (socket says 172.21.15.62:54208) 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 -- 172.21.15.62:0/4293397195 learned_addr learned my addr 172.21.15.62:0/4293397195 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 -- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac161270 msgr2=0x7f0eac157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac161270 0x7f0eac157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 -- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eac161650 msgr2=0x7f0eac0c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eac161650 0x7f0eac0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 -- 172.21.15.62:0/4293397195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac15d0a0 con 0x7f0eac0b5c70 2024-04-25T09:14:33.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0eb1000640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac0b5c70 0x7f0eac0cd2a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c3f0 tx=0x7f0ea000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:33.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.679+0000 7f0ea8c00640 1 -- 172.21.15.62:0/4293397195 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0017070 con 0x7f0eac0b5c70 2024-04-25T09:14:33.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.680+0000 7f0ea8c00640 1 -- 172.21.15.62:0/4293397195 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea0005ce0 con 0x7f0eac0b5c70 2024-04-25T09:14:33.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.680+0000 7f0ea8c00640 1 -- 172.21.15.62:0/4293397195 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00072f0 con 0x7f0eac0b5c70 2024-04-25T09:14:33.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.680+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eac15c520 con 0x7f0eac0b5c70 2024-04-25T09:14:33.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.680+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eac15cae0 con 0x7f0eac0b5c70 2024-04-25T09:14:33.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.681+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eac072910 con 0x7f0eac0b5c70 2024-04-25T09:14:33.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.682+0000 7f0ea8c00640 1 -- 172.21.15.62:0/4293397195 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ea001a070 con 0x7f0eac0b5c70 2024-04-25T09:14:33.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.683+0000 7f0ea8c00640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:33.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.683+0000 7f0ea8c00640 1 -- 172.21.15.62:0/4293397195 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0ea00896d0 con 0x7f0eac0b5c70 2024-04-25T09:14:33.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.684+0000 7f0eabe00640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:33.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.685+0000 7f0eabe00640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006fd0 tx=0x7f0e98008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:33.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.688+0000 7f0ea8c00640 1 -- 172.21.15.62:0/4293397195 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ea0058420 con 0x7f0eac0b5c70 2024-04-25T09:14:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:33.885+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0eac0750e0 con 0x7f0eac0b5c70 2024-04-25T09:14:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.428+0000 7f0ea8c00640 1 -- 172.21.15.62:0/4293397195 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0ea0057d20 con 0x7f0eac0b5c70 2024-04-25T09:14:34.416 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:14:34.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e8c06b4d0 msgr2=0x7f0e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006fd0 tx=0x7f0e98008040 comp rx=0 tx=0).stop 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac0b5c70 msgr2=0x7f0eac0cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac0b5c70 0x7f0eac0cd2a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c3f0 tx=0x7f0ea000c8b0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 shutdown_connections 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0e8c06b4d0 0x7f0e8c06d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eac161650 0x7f0eac0c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0eac161270 0x7f0eac157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 --2- 172.21.15.62:0/4293397195 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0eac0b5c70 0x7f0eac0cd2a0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 >> 172.21.15.62:0/4293397195 conn(0x7f0eac066d10 msgr2=0x7f0eac0b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 shutdown_connections 2024-04-25T09:14:34.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.432+0000 7f0eb2400640 1 -- 172.21.15.62:0/4293397195 wait complete. 2024-04-25T09:14:34.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:14:34.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.677+0000 7fe965e00640 1 Processor -- start 2024-04-25T09:14:34.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.677+0000 7fe965e00640 1 -- start start 2024-04-25T09:14:34.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.679+0000 7fe965e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe960161270 0x7fe9600b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:34.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.679+0000 7fe965e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe960161650 0x7fe9600b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:34.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.679+0000 7fe965e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9600d1480 con 0x7fe9600b2980 2024-04-25T09:14:34.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.679+0000 7fe965e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600d3900 con 0x7fe960161270 2024-04-25T09:14:34.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.679+0000 7fe965e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600d11a0 con 0x7fe960161650 2024-04-25T09:14:34.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.680+0000 7fe957e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe960161650 0x7fe9600b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:34.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.680+0000 7fe957e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe960161650 0x7fe9600b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:46452/0 (socket says 172.21.15.62:46452) 2024-04-25T09:14:34.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.680+0000 7fe957e00640 1 -- 172.21.15.62:0/1695589214 learned_addr learned my addr 172.21.15.62:0/1695589214 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:34.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.680+0000 7fe964a00640 1 --2- 172.21.15.62:0/1695589214 >> v2:172.21.15.121:3300/0 conn(0x7fe960161270 0x7fe9600b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:34.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.681+0000 7fe964a00640 1 -- 172.21.15.62:0/1695589214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe960161650 msgr2=0x7fe9600b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:34.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.681+0000 7fe964a00640 1 --2- 172.21.15.62:0/1695589214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe960161650 0x7fe9600b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.681+0000 7fe964a00640 1 -- 172.21.15.62:0/1695589214 >> v1:172.21.15.121:6789/0 conn(0x7fe9600b2980 legacy=0x7fe960161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:34.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.681+0000 7fe964a00640 1 -- 172.21.15.62:0/1695589214 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600cee80 con 0x7fe960161270 2024-04-25T09:14:34.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.682+0000 7fe964a00640 1 --2- 172.21.15.62:0/1695589214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 0x7fe9600b5560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9500097a0 tx=0x7fe95002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98ea26cf858dcee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:34.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.682+0000 7fe95fe00640 1 -- 172.21.15.62:0/1695589214 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950037070 con 0x7fe960161270 2024-04-25T09:14:34.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.682+0000 7fe95fe00640 1 -- 172.21.15.62:0/1695589214 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95002adb0 con 0x7fe960161270 2024-04-25T09:14:34.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.682+0000 7fe95fe00640 1 -- 172.21.15.62:0/1695589214 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950032660 con 0x7fe960161270 2024-04-25T09:14:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.683+0000 7fe965e00640 1 -- 172.21.15.62:0/1695589214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 msgr2=0x7fe9600b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.683+0000 7fe965e00640 1 --2- 172.21.15.62:0/1695589214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 0x7fe9600b5560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9500097a0 tx=0x7fe95002a060 comp rx=0 tx=0).stop 2024-04-25T09:14:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.683+0000 7fe965e00640 1 -- 172.21.15.62:0/1695589214 shutdown_connections 2024-04-25T09:14:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.683+0000 7fe965e00640 1 --2- 172.21.15.62:0/1695589214 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe960161650 0x7fe9600b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.683+0000 7fe965e00640 1 --2- 172.21.15.62:0/1695589214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 0x7fe9600b5560 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.683+0000 7fe965e00640 1 -- 172.21.15.62:0/1695589214 >> 172.21.15.62:0/1695589214 conn(0x7fe960066d10 msgr2=0x7fe960067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.683+0000 7fe965e00640 1 -- 172.21.15.62:0/1695589214 shutdown_connections 2024-04-25T09:14:34.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.683+0000 7fe965e00640 1 -- 172.21.15.62:0/1695589214 wait complete. 2024-04-25T09:14:34.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.686+0000 7fe965e00640 1 Processor -- start 2024-04-25T09:14:34.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.686+0000 7fe965e00640 1 -- start start 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.687+0000 7fe965e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9600b2980 0x7fe960151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.687+0000 7fe965e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 0x7fe9600cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.687+0000 7fe965e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe960161650 0x7fe9600cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.687+0000 7fe965e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600b2ed0 con 0x7fe960161650 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.687+0000 7fe965e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600d3590 con 0x7fe960161270 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.687+0000 7fe965e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600b3050 con 0x7fe9600b2980 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.687+0000 7fe965400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe960161650 0x7fe9600cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe965400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe960161650 0x7fe9600cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:54224/0 (socket says 172.21.15.62:54224) 2024-04-25T09:14:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe965400640 1 -- 172.21.15.62:0/92845606 learned_addr learned my addr 172.21.15.62:0/92845606 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:34.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe964a00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9600b2980 0x7fe960151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:34.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe957e00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 0x7fe9600cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe965400640 1 -- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9600b2980 msgr2=0x7fe960151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe965400640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9600b2980 0x7fe960151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe965400640 1 -- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 msgr2=0x7fe9600cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe965400640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 0x7fe9600cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe965400640 1 -- 172.21.15.62:0/92845606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960159070 con 0x7fe960161650 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.688+0000 7fe965400640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe960161650 0x7fe9600cc820 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c6b0 tx=0x7fe95800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.689+0000 7fe95d600640 1 -- 172.21.15.62:0/92845606 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958003b70 con 0x7fe960161650 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.689+0000 7fe95d600640 1 -- 172.21.15.62:0/92845606 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe958003d30 con 0x7fe960161650 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.689+0000 7fe95d600640 1 -- 172.21.15.62:0/92845606 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958005310 con 0x7fe960161650 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.689+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe960159310 con 0x7fe960161650 2024-04-25T09:14:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.689+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9601598d0 con 0x7fe960161650 2024-04-25T09:14:34.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.690+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe960072910 con 0x7fe960161650 2024-04-25T09:14:34.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.691+0000 7fe95d600640 1 -- 172.21.15.62:0/92845606 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9580054b0 con 0x7fe960161650 2024-04-25T09:14:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.693+0000 7fe95d600640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe93c06b500 0x7fe93c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.693+0000 7fe95d600640 1 -- 172.21.15.62:0/92845606 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fe958089ed0 con 0x7fe960161650 2024-04-25T09:14:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.693+0000 7fe964a00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe93c06b500 0x7fe93c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:34.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.695+0000 7fe964a00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe93c06b500 0x7fe93c06d9c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe950034000 tx=0x7fe95002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:34.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.697+0000 7fe95d600640 1 -- 172.21.15.62:0/92845606 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe958058ea0 con 0x7fe960161650 2024-04-25T09:14:34.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:34.904+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9600b36c0 con 0x7fe960161650 2024-04-25T09:14:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.432+0000 7fe95d600640 1 -- 172.21.15.62:0/92845606 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe9580587a0 con 0x7fe960161650 2024-04-25T09:14:35.420 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:14:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.437+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe93c06b500 msgr2=0x7fe93c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.437+0000 7fe965e00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe93c06b500 0x7fe93c06d9c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe950034000 tx=0x7fe95002e000 comp rx=0 tx=0).stop 2024-04-25T09:14:35.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.437+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe960161650 msgr2=0x7fe9600cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:35.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.437+0000 7fe965e00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe960161650 0x7fe9600cc820 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe95800c6b0 tx=0x7fe95800cb70 comp rx=0 tx=0).stop 2024-04-25T09:14:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.438+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 shutdown_connections 2024-04-25T09:14:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.438+0000 7fe965e00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe93c06b500 0x7fe93c06d9c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.438+0000 7fe965e00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe960161650 0x7fe9600cc820 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.438+0000 7fe965e00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe960161270 0x7fe9600cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.438+0000 7fe965e00640 1 --2- 172.21.15.62:0/92845606 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe9600b2980 0x7fe960151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.438+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 >> 172.21.15.62:0/92845606 conn(0x7fe960066d10 msgr2=0x7fe9600b72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.440+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 shutdown_connections 2024-04-25T09:14:35.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.440+0000 7fe965e00640 1 -- 172.21.15.62:0/92845606 wait complete. 2024-04-25T09:14:35.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:14:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.666+0000 7f7f1b200640 1 Processor -- start 2024-04-25T09:14:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.666+0000 7f7f1b200640 1 -- start start 2024-04-25T09:14:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.668+0000 7f7f1b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7f1c06e310 0x7f7f1c0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.668+0000 7f7f1b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c0b5a40 con 0x7f7f1c143840 2024-04-25T09:14:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.668+0000 7f7f1b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c0718e0 con 0x7f7f1c0b5210 2024-04-25T09:14:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.668+0000 7f7f1b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c070b20 con 0x7f7f1c06e310 2024-04-25T09:14:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.668+0000 7f7f19e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7f1c143840 0x7f7f1c06dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35756/0 (socket says 172.21.15.62:35756) 2024-04-25T09:14:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.668+0000 7f7f19e00640 1 -- 172.21.15.62:0/943042943 learned_addr learned my addr 172.21.15.62:0/943042943 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.668+0000 7f7f19400640 1 --2- 172.21.15.62:0/943042943 >> v2:172.21.15.62:3300/0 conn(0x7f7f1c06e310 0x7f7f1c0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.669+0000 7f7f19400640 1 -- 172.21.15.62:0/943042943 >> v1:172.21.15.121:6789/0 conn(0x7f7f1c0b5210 legacy=0x7f7f1c0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.669+0000 7f7f19400640 1 -- 172.21.15.62:0/943042943 >> v1:172.21.15.62:6789/0 conn(0x7f7f1c143840 legacy=0x7f7f1c06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.669+0000 7f7f18a00640 1 -- 172.21.15.62:0/943042943 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737468678 0 0) 0x7f7f1c0b5a40 con 0x7f7f1c143840 2024-04-25T09:14:35.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.669+0000 7f7f19400640 1 -- 172.21.15.62:0/943042943 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c070e00 con 0x7f7f1c06e310 2024-04-25T09:14:35.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.670+0000 7f7f19400640 1 --2- 172.21.15.62:0/943042943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c0b4cd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800a940 tx=0x7f7f0802b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1143d041312bc926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:35.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.670+0000 7f7f18a00640 1 -- 172.21.15.62:0/943042943 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0800b040 con 0x7f7f1c06e310 2024-04-25T09:14:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.670+0000 7f7f18a00640 1 -- 172.21.15.62:0/943042943 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f08030710 con 0x7f7f1c06e310 2024-04-25T09:14:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.670+0000 7f7f18a00640 1 -- 172.21.15.62:0/943042943 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08030ae0 con 0x7f7f1c06e310 2024-04-25T09:14:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.671+0000 7f7f1b200640 1 -- 172.21.15.62:0/943042943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 msgr2=0x7f7f1c0b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.671+0000 7f7f1b200640 1 --2- 172.21.15.62:0/943042943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c0b4cd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800a940 tx=0x7f7f0802b5a0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.671+0000 7f7f1b200640 1 -- 172.21.15.62:0/943042943 shutdown_connections 2024-04-25T09:14:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.671+0000 7f7f1b200640 1 --2- 172.21.15.62:0/943042943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c0b4cd0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.671+0000 7f7f1b200640 1 -- 172.21.15.62:0/943042943 >> 172.21.15.62:0/943042943 conn(0x7f7f1c0b1bf0 msgr2=0x7f7f1c0b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.671+0000 7f7f1b200640 1 -- 172.21.15.62:0/943042943 shutdown_connections 2024-04-25T09:14:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.671+0000 7f7f1b200640 1 -- 172.21.15.62:0/943042943 wait complete. 2024-04-25T09:14:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.673+0000 7f7f1b200640 1 Processor -- start 2024-04-25T09:14:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.673+0000 7f7f1b200640 1 -- start start 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.674+0000 7f7f1b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c13d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.674+0000 7f7f1b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f1c0b5210 0x7f7f1c140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.674+0000 7f7f1b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f1c143840 0x7f7f1c1e4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.674+0000 7f7f1b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c143c20 con 0x7f7f1c06e310 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.674+0000 7f7f1b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0713c0 con 0x7f7f1c143840 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.674+0000 7f7f1b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b5630 con 0x7f7f1c0b5210 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.674+0000 7f7f19e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c13d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.675+0000 7f7f19e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c13d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47058/0 (socket says 172.21.15.62:47058) 2024-04-25T09:14:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.675+0000 7f7f19e00640 1 -- 172.21.15.62:0/3124991413 learned_addr learned my addr 172.21.15.62:0/3124991413 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:35.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.675+0000 7f7f19e00640 1 -- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f1c0b5210 msgr2=0x7f7f1c140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:35.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.675+0000 7f7f19e00640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f1c0b5210 0x7f7f1c140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.675+0000 7f7f19e00640 1 -- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f1c143840 msgr2=0x7f7f1c1e4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.675+0000 7f7f19e00640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f1c143840 0x7f7f1c1e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.675+0000 7f7f19e00640 1 -- 172.21.15.62:0/3124991413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c13ddf0 con 0x7f7f1c06e310 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.675+0000 7f7f19e00640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c13d8b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00c4a0 tx=0x7f7f0c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.676+0000 7f7f12000640 1 -- 172.21.15.62:0/3124991413 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c017070 con 0x7f7f1c06e310 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.676+0000 7f7f12000640 1 -- 172.21.15.62:0/3124991413 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c005ce0 con 0x7f7f1c06e310 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.676+0000 7f7f12000640 1 -- 172.21.15.62:0/3124991413 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0072b0 con 0x7f7f1c06e310 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.676+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c140f60 con 0x7f7f1c06e310 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.676+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c141520 con 0x7f7f1c06e310 2024-04-25T09:14:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.677+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c15b3f0 con 0x7f7f1c06e310 2024-04-25T09:14:35.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.678+0000 7f7f12000640 1 -- 172.21.15.62:0/3124991413 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f0c01a070 con 0x7f7f1c06e310 2024-04-25T09:14:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.679+0000 7f7f12000640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7efc06b4d0 0x7f7efc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.679+0000 7f7f12000640 1 -- 172.21.15.62:0/3124991413 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7f0c089290 con 0x7f7f1c06e310 2024-04-25T09:14:35.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.680+0000 7f7f19400640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7efc06b4d0 0x7f7efc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:35.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.681+0000 7f7f19400640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7efc06b4d0 0x7f7efc06d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7f08035000 tx=0x7f7f0802f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:35.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.683+0000 7f7f12000640 1 -- 172.21.15.62:0/3124991413 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f1c15b3f0 con 0x7f7f1c06e310 2024-04-25T09:14:35.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.930+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7f1c04ef20 con 0x7f7f1c06e310 2024-04-25T09:14:35.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.935+0000 7f7f12000640 1 -- 172.21.15.62:0/3124991413 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7f0c058400 con 0x7f7f1c06e310 2024-04-25T09:14:35.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7efc06b4d0 msgr2=0x7f7efc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7efc06b4d0 0x7f7efc06d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7f08035000 tx=0x7f7f0802f000 comp rx=0 tx=0).stop 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 msgr2=0x7f7f1c13d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c13d8b0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00c4a0 tx=0x7f7f0c00c960 comp rx=0 tx=0).stop 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 shutdown_connections 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7efc06b4d0 0x7f7efc06d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f1c143840 0x7f7f1c1e4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7f1c0b5210 0x7f7f1c140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 --2- 172.21.15.62:0/3124991413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f1c06e310 0x7f7f1c13d8b0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 >> 172.21.15.62:0/3124991413 conn(0x7f7f1c0b1bf0 msgr2=0x7f7f1c0aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 shutdown_connections 2024-04-25T09:14:35.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:35.938+0000 7f7f1b200640 1 -- 172.21.15.62:0/3124991413 wait complete. 2024-04-25T09:14:35.945 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T09:14:36.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.161+0000 7f58e3e00640 1 Processor -- start 2024-04-25T09:14:36.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.161+0000 7f58e3e00640 1 -- start start 2024-04-25T09:14:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.162+0000 7f58e3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e4144c10 0x7f58e415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.162+0000 7f58e3e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58e40cee40 0x7f58e406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.162+0000 7f58e3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58e406d730 0x7f58e40b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.162+0000 7f58e3e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e4144ff0 con 0x7f58e406d730 2024-04-25T09:14:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.162+0000 7f58e3e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e4160340 con 0x7f58e40cee40 2024-04-25T09:14:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.162+0000 7f58e3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40b41c0 con 0x7f58e4144c10 2024-04-25T09:14:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.162+0000 7f58e3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58e406d730 0x7f58e40b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.163+0000 7f58e3400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58e406d730 0x7f58e40b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47068/0 (socket says 172.21.15.62:47068) 2024-04-25T09:14:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.163+0000 7f58e3400640 1 -- 172.21.15.62:0/1932203346 learned_addr learned my addr 172.21.15.62:0/1932203346 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.163+0000 7f58e3400640 1 -- 172.21.15.62:0/1932203346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e4144c10 msgr2=0x7f58e415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.163+0000 7f58e3400640 1 --2- 172.21.15.62:0/1932203346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e4144c10 0x7f58e415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.163+0000 7f58e3400640 1 -- 172.21.15.62:0/1932203346 >> v2:172.21.15.121:3300/0 conn(0x7f58e40cee40 msgr2=0x7f58e406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.163+0000 7f58e3400640 1 --2- 172.21.15.62:0/1932203346 >> v2:172.21.15.121:3300/0 conn(0x7f58e40cee40 0x7f58e406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.163+0000 7f58e3400640 1 -- 172.21.15.62:0/1932203346 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e415faa0 con 0x7f58e406d730 2024-04-25T09:14:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.164+0000 7f58e3400640 1 --2- 172.21.15.62:0/1932203346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e406d730 0x7f58e40b2da0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f58e406bae0 tx=0x7f58dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d6e06dc4f895a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.164+0000 7f58e2000640 1 -- 172.21.15.62:0/1932203346 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc036070 con 0x7f58e406d730 2024-04-25T09:14:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.164+0000 7f58e2000640 1 -- 172.21.15.62:0/1932203346 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc031600 con 0x7f58e406d730 2024-04-25T09:14:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.164+0000 7f58e2000640 1 -- 172.21.15.62:0/1932203346 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0319d0 con 0x7f58e406d730 2024-04-25T09:14:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 -- 172.21.15.62:0/1932203346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e406d730 msgr2=0x7f58e40b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 --2- 172.21.15.62:0/1932203346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e406d730 0x7f58e40b2da0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f58e406bae0 tx=0x7f58dc02a610 comp rx=0 tx=0).stop 2024-04-25T09:14:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 -- 172.21.15.62:0/1932203346 shutdown_connections 2024-04-25T09:14:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 --2- 172.21.15.62:0/1932203346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e406d730 0x7f58e40b2da0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 --2- 172.21.15.62:0/1932203346 >> v2:172.21.15.121:3300/0 conn(0x7f58e40cee40 0x7f58e406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 --2- 172.21.15.62:0/1932203346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e4144c10 0x7f58e415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 -- 172.21.15.62:0/1932203346 >> 172.21.15.62:0/1932203346 conn(0x7f58e40b14d0 msgr2=0x7f58e40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 -- 172.21.15.62:0/1932203346 shutdown_connections 2024-04-25T09:14:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.165+0000 7f58e3e00640 1 -- 172.21.15.62:0/1932203346 wait complete. 2024-04-25T09:14:36.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.167+0000 7f58e3e00640 1 Processor -- start 2024-04-25T09:14:36.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.167+0000 7f58e3e00640 1 -- start start 2024-04-25T09:14:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e406d730 0x7f58e413f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58e40cee40 0x7f58e4143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e4144c10 0x7f58e4143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40cf2a0 con 0x7f58e4144c10 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e4160060 con 0x7f58e40cee40 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40b32e0 con 0x7f58e406d730 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e4144c10 0x7f58e4143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e4144c10 0x7f58e4143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47078/0 (socket says 172.21.15.62:47078) 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3400640 1 -- 172.21.15.62:0/3245736876 learned_addr learned my addr 172.21.15.62:0/3245736876 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e2a00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e406d730 0x7f58e413f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3400640 1 -- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e406d730 msgr2=0x7f58e413f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3400640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e406d730 0x7f58e413f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3400640 1 -- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58e40cee40 msgr2=0x7f58e4143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3400640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58e40cee40 0x7f58e4143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.168+0000 7f58e3400640 1 -- 172.21.15.62:0/3245736876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e4141f40 con 0x7f58e4144c10 2024-04-25T09:14:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.169+0000 7f58e3400640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e4144c10 0x7f58e4143fb0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f58dc034040 tx=0x7f58dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.169+0000 7f58daa00640 1 -- 172.21.15.62:0/3245736876 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc041020 con 0x7f58e4144c10 2024-04-25T09:14:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.169+0000 7f58daa00640 1 -- 172.21.15.62:0/3245736876 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc007780 con 0x7f58e4144c10 2024-04-25T09:14:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.169+0000 7f58daa00640 1 -- 172.21.15.62:0/3245736876 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc036040 con 0x7f58e4144c10 2024-04-25T09:14:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.169+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e4142220 con 0x7f58e4144c10 2024-04-25T09:14:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.169+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e41427e0 con 0x7f58e4144c10 2024-04-25T09:14:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.170+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e404ef20 con 0x7f58e4144c10 2024-04-25T09:14:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.171+0000 7f58daa00640 1 -- 172.21.15.62:0/3245736876 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58dc044020 con 0x7f58e4144c10 2024-04-25T09:14:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.172+0000 7f58daa00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58c006b4d0 0x7f58c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.172+0000 7f58daa00640 1 -- 172.21.15.62:0/3245736876 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f58dc0aad70 con 0x7f58e4144c10 2024-04-25T09:14:36.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.173+0000 7f58e2a00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58c006b4d0 0x7f58c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:36.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.174+0000 7f58e2a00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58c006b4d0 0x7f58c006d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f58d400a770 tx=0x7f58d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:36.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.176+0000 7f58daa00640 1 -- 172.21.15.62:0/3245736876 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58dc04b030 con 0x7f58e4144c10 2024-04-25T09:14:36.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:36.454+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f58e40b30c0 con 0x7f58e4144c10 2024-04-25T09:14:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.451+0000 7f58daa00640 1 -- 172.21.15.62:0/3245736876 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v506) v1 ==== 155+0+0 (secure 0 0 0) 0x7f58dc079f90 con 0x7f58e4144c10 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58c006b4d0 msgr2=0x7f58c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58c006b4d0 0x7f58c006d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f58d400a770 tx=0x7f58d4008040 comp rx=0 tx=0).stop 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e4144c10 msgr2=0x7f58e4143fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e4144c10 0x7f58e4143fb0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f58dc034040 tx=0x7f58dc004a40 comp rx=0 tx=0).stop 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 shutdown_connections 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58c006b4d0 0x7f58c006d990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58e4144c10 0x7f58e4143fb0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58e40cee40 0x7f58e4143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 --2- 172.21.15.62:0/3245736876 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58e406d730 0x7f58e413f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.455+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 >> 172.21.15.62:0/3245736876 conn(0x7f58e40b14d0 msgr2=0x7f58e40b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.456+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 shutdown_connections 2024-04-25T09:14:37.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.456+0000 7f58e3e00640 1 -- 172.21.15.62:0/3245736876 wait complete. 2024-04-25T09:14:37.464 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:14:37.465 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:14:37.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:37.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.691+0000 7fac69800640 1 Processor -- start 2024-04-25T09:14:37.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.692+0000 7fac69800640 1 -- start start 2024-04-25T09:14:37.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.693+0000 7fac69800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fac6415dc40 0x7fac640d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.693+0000 7fac69800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fac640d0ae0 0x7fac640d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.693+0000 7fac69800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac640d2620 con 0x7fac640d3ae0 2024-04-25T09:14:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.693+0000 7fac69800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac640d2900 con 0x7fac640d0ae0 2024-04-25T09:14:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.693+0000 7fac69800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac640d2be0 con 0x7fac6415dc40 2024-04-25T09:14:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac62000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fac640d0ae0 0x7fac640d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac63400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fac640d3ae0 0x7fac640d3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35768/0 (socket says 172.21.15.62:35768) 2024-04-25T09:14:37.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac63400640 1 -- 172.21.15.62:0/1472878504 learned_addr learned my addr 172.21.15.62:0/1472878504 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:37.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac62a00640 1 --2- 172.21.15.62:0/1472878504 >> v2:172.21.15.121:3300/0 conn(0x7fac6415dc40 0x7fac640d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:37.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac62000640 1 -- 172.21.15.62:0/1472878504 >> v2:172.21.15.121:3300/0 conn(0x7fac6415dc40 msgr2=0x7fac640d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:37.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac62000640 1 --2- 172.21.15.62:0/1472878504 >> v2:172.21.15.121:3300/0 conn(0x7fac6415dc40 0x7fac640d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac62000640 1 -- 172.21.15.62:0/1472878504 >> v1:172.21.15.62:6789/0 conn(0x7fac640d3ae0 legacy=0x7fac640d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:37.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac62000640 1 -- 172.21.15.62:0/1472878504 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac640d2ec0 con 0x7fac640d0ae0 2024-04-25T09:14:37.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.694+0000 7fac61600640 1 -- 172.21.15.62:0/1472878504 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797727776 0 0) 0x7fac640d2620 con 0x7fac640d3ae0 2024-04-25T09:14:37.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.695+0000 7fac62000640 1 --2- 172.21.15.62:0/1472878504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d0ae0 0x7fac640d0f70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fac500097a0 tx=0x7fac5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225af4bdf9889f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:37.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac61600640 1 -- 172.21.15.62:0/1472878504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50037070 con 0x7fac640d0ae0 2024-04-25T09:14:37.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac61600640 1 -- 172.21.15.62:0/1472878504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac50032440 con 0x7fac640d0ae0 2024-04-25T09:14:37.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac61600640 1 -- 172.21.15.62:0/1472878504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50032810 con 0x7fac640d0ae0 2024-04-25T09:14:37.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac69800640 1 -- 172.21.15.62:0/1472878504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d0ae0 msgr2=0x7fac640d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:37.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac69800640 1 --2- 172.21.15.62:0/1472878504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d0ae0 0x7fac640d0f70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fac500097a0 tx=0x7fac5002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac69800640 1 -- 172.21.15.62:0/1472878504 shutdown_connections 2024-04-25T09:14:37.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac69800640 1 --2- 172.21.15.62:0/1472878504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d0ae0 0x7fac640d0f70 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac69800640 1 --2- 172.21.15.62:0/1472878504 >> v2:172.21.15.121:3300/0 conn(0x7fac6415dc40 0x7fac640d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac69800640 1 -- 172.21.15.62:0/1472878504 >> 172.21.15.62:0/1472878504 conn(0x7fac6406b000 msgr2=0x7fac6406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:37.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.696+0000 7fac69800640 1 -- 172.21.15.62:0/1472878504 shutdown_connections 2024-04-25T09:14:37.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.697+0000 7fac69800640 1 -- 172.21.15.62:0/1472878504 wait complete. 2024-04-25T09:14:37.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.698+0000 7fac69800640 1 Processor -- start 2024-04-25T09:14:37.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.698+0000 7fac69800640 1 -- start start 2024-04-25T09:14:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.699+0000 7fac69800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac640d0ae0 0x7fac6414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.699+0000 7fac69800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d3ae0 0x7fac641e6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.699+0000 7fac69800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac6415dc40 0x7fac64160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.699+0000 7fac69800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640d36c0 con 0x7fac640d3ae0 2024-04-25T09:14:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.699+0000 7fac69800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640b7d00 con 0x7fac640d0ae0 2024-04-25T09:14:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.699+0000 7fac69800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac641e7c10 con 0x7fac6415dc40 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d3ae0 0x7fac641e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d3ae0 0x7fac641e6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47098/0 (socket says 172.21.15.62:47098) 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62000640 1 -- 172.21.15.62:0/2271711063 learned_addr learned my addr 172.21.15.62:0/2271711063 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62a00640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac640d0ae0 0x7fac6414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62000640 1 -- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac6415dc40 msgr2=0x7fac64160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62000640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac6415dc40 0x7fac64160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62000640 1 -- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac640d0ae0 msgr2=0x7fac6414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62000640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac640d0ae0 0x7fac6414f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62000640 1 -- 172.21.15.62:0/2271711063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac641e7000 con 0x7fac640d3ae0 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.700+0000 7fac62a00640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac640d0ae0 0x7fac6414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.701+0000 7fac62000640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d3ae0 0x7fac641e6560 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fac500097a0 tx=0x7fac500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.701+0000 7fac5aa00640 1 -- 172.21.15.62:0/2271711063 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50037050 con 0x7fac640d3ae0 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.701+0000 7fac5aa00640 1 -- 172.21.15.62:0/2271711063 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac50038040 con 0x7fac640d3ae0 2024-04-25T09:14:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.701+0000 7fac5aa00640 1 -- 172.21.15.62:0/2271711063 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50004470 con 0x7fac640d3ae0 2024-04-25T09:14:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.701+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac641e72e0 con 0x7fac640d3ae0 2024-04-25T09:14:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.701+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac641e78a0 con 0x7fac640d3ae0 2024-04-25T09:14:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.702+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac641e7d90 con 0x7fac640d3ae0 2024-04-25T09:14:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.703+0000 7fac5aa00640 1 -- 172.21.15.62:0/2271711063 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac50030900 con 0x7fac640d3ae0 2024-04-25T09:14:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.704+0000 7fac5aa00640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fac4406b4d0 0x7fac4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.705+0000 7fac5aa00640 1 -- 172.21.15.62:0/2271711063 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fac500aa9f0 con 0x7fac640d3ae0 2024-04-25T09:14:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.705+0000 7fac62a00640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fac4406b4d0 0x7fac4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.707+0000 7fac62a00640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fac4406b4d0 0x7fac4406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fac54006fd0 tx=0x7fac54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:37.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.710+0000 7fac5aa00640 1 -- 172.21.15.62:0/2271711063 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac50079b60 con 0x7fac640d3ae0 2024-04-25T09:14:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.912+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac640cd880 con 0x7fac640d3ae0 2024-04-25T09:14:37.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.914+0000 7fac5aa00640 1 -- 172.21.15.62:0/2271711063 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+13020 (secure 0 0 0) 0x7fac50079460 con 0x7fac640d3ae0 2024-04-25T09:14:37.902 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:37.902 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":114,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:35.429939+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6836/2489379768":"2024-04-26T09:14:32.419058+0000","172.21.15.121:6834/2489379768":"2024-04-26T09:14:32.419058+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fac4406b4d0 msgr2=0x7fac4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fac4406b4d0 0x7fac4406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fac54006fd0 tx=0x7fac54008040 comp rx=0 tx=0).stop 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d3ae0 msgr2=0x7fac641e6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d3ae0 0x7fac641e6560 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fac500097a0 tx=0x7fac500046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 shutdown_connections 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fac4406b4d0 0x7fac4406d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac640d3ae0 0x7fac641e6560 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fac6415dc40 0x7fac64160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 --2- 172.21.15.62:0/2271711063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac640d0ae0 0x7fac6414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 >> 172.21.15.62:0/2271711063 conn(0x7fac6406b000 msgr2=0x7fac640b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:37.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.917+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 shutdown_connections 2024-04-25T09:14:37.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:37.918+0000 7fac69800640 1 -- 172.21.15.62:0/2271711063 wait complete. 2024-04-25T09:14:37.925 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:14:37.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:14:38.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.153+0000 7f150b200640 1 Processor -- start 2024-04-25T09:14:38.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.153+0000 7f150b200640 1 -- start start 2024-04-25T09:14:38.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.154+0000 7f150b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150406d2c0 0x7f15040ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.155+0000 7f150b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15040cf000 con 0x7f150406d6a0 2024-04-25T09:14:38.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.155+0000 7f150b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15040cf2e0 con 0x7f15040b31a0 2024-04-25T09:14:38.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.155+0000 7f150b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15040b26c0 con 0x7f150406d2c0 2024-04-25T09:14:38.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.155+0000 7f1509400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f150406d6a0 0x7f15040ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35770/0 (socket says 172.21.15.62:35770) 2024-04-25T09:14:38.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.155+0000 7f1509400640 1 -- 172.21.15.62:0/2631897288 learned_addr learned my addr 172.21.15.62:0/2631897288 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.155+0000 7f1509e00640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150406d2c0 0x7f15040ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.155+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312750915 0 0) 0x7f15040cf000 con 0x7f150406d6a0 2024-04-25T09:14:38.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14ec003520 con 0x7f150406d6a0 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1124563231 0 0) 0x7f14ec003520 con 0x7f150406d6a0 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150406d2c0 msgr2=0x7f15040ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150406d2c0 0x7f15040ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 >> v1:172.21.15.121:6789/0 conn(0x7f15040b31a0 legacy=0x7f15040b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15040b29a0 con 0x7f150406d6a0 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f14f4001670 con 0x7f150406d6a0 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 >> v1:172.21.15.62:6789/0 conn(0x7f150406d6a0 legacy=0x7f15040ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150406d2c0 0x7f14ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec003f60 0x7f14ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec004d40 0x7f14ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15040b26c0 con 0x7f14ec003f60 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec00baa0 con 0x7f14ec004d40 2024-04-25T09:14:38.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec00bca0 con 0x7f150406d2c0 2024-04-25T09:14:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.156+0000 7f1509400640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec003f60 0x7f14ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.157+0000 7f1509e00640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150406d2c0 0x7f14ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.157+0000 7f150a800640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec004d40 0x7f14ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.157+0000 7f1509400640 1 -- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150406d2c0 msgr2=0x7f14ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.157+0000 7f1509400640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150406d2c0 0x7f14ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.157+0000 7f1509400640 1 -- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec004d40 msgr2=0x7f14ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.157+0000 7f1509400640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec004d40 0x7f14ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.157+0000 7f1509400640 1 -- 172.21.15.62:0/2631897288 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec003520 con 0x7f14ec003f60 2024-04-25T09:14:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.158+0000 7f1509400640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec003f60 0x7f14ec004800 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c240 tx=0x7f14ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac07456372d5561 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.158+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4007d10 con 0x7f14ec003f60 2024-04-25T09:14:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.159+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f4005ad0 con 0x7f14ec003f60 2024-04-25T09:14:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.159+0000 7f1508a00640 1 -- 172.21.15.62:0/2631897288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4007070 con 0x7f14ec003f60 2024-04-25T09:14:38.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.159+0000 7f150b200640 1 -- 172.21.15.62:0/2631897288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec003f60 msgr2=0x7f14ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.159+0000 7f150b200640 1 --2- 172.21.15.62:0/2631897288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14ec003f60 0x7f14ec004800 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c240 tx=0x7f14ec005830 comp rx=0 tx=0).stop 2024-04-25T09:14:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.159+0000 7f150a800640 1 -- 172.21.15.62:0/2631897288 reap_dead start 2024-04-25T09:14:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.159+0000 7f150b200640 1 -- 172.21.15.62:0/2631897288 shutdown_connections 2024-04-25T09:14:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.159+0000 7f150b200640 1 -- 172.21.15.62:0/2631897288 >> 172.21.15.62:0/2631897288 conn(0x7f1504066f20 msgr2=0x7f15040b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.160+0000 7f150b200640 1 -- 172.21.15.62:0/2631897288 shutdown_connections 2024-04-25T09:14:38.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.160+0000 7f150b200640 1 -- 172.21.15.62:0/2631897288 wait complete. 2024-04-25T09:14:38.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.162+0000 7f150b200640 1 Processor -- start 2024-04-25T09:14:38.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.162+0000 7f150b200640 1 -- start start 2024-04-25T09:14:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.163+0000 7f150b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec003f60 0x7f15040ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.163+0000 7f150b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150413c910 0x7f1504071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.163+0000 7f150b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150415b200 0x7f150415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.163+0000 7f150b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15041448d0 con 0x7f150413c910 2024-04-25T09:14:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.163+0000 7f150b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150406db20 con 0x7f14ec003f60 2024-04-25T09:14:38.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.163+0000 7f150b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1504144a50 con 0x7f150415b200 2024-04-25T09:14:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150413c910 0x7f1504071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150413c910 0x7f1504071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47104/0 (socket says 172.21.15.62:47104) 2024-04-25T09:14:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509400640 1 -- 172.21.15.62:0/35773202 learned_addr learned my addr 172.21.15.62:0/35773202 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509e00640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec003f60 0x7f15040ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f150a800640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150415b200 0x7f150415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509400640 1 -- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150415b200 msgr2=0x7f150415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509400640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150415b200 0x7f150415e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509400640 1 -- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec003f60 msgr2=0x7f15040ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509400640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec003f60 0x7f15040ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509400640 1 -- 172.21.15.62:0/35773202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15040cafd0 con 0x7f150413c910 2024-04-25T09:14:38.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.164+0000 7f1509e00640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec003f60 0x7f15040ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:14:38.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.165+0000 7f1509400640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150413c910 0x7f1504071c80 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c240 tx=0x7f15040cf1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:38.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.165+0000 7f14fe000640 1 -- 172.21.15.62:0/35773202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4042020 con 0x7f150413c910 2024-04-25T09:14:38.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.165+0000 7f14fe000640 1 -- 172.21.15.62:0/35773202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f4004bf0 con 0x7f150413c910 2024-04-25T09:14:38.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.165+0000 7f14fe000640 1 -- 172.21.15.62:0/35773202 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4007d10 con 0x7f150413c910 2024-04-25T09:14:38.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.165+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15040cb2b0 con 0x7f150413c910 2024-04-25T09:14:38.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.165+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15040acd60 con 0x7f150413c910 2024-04-25T09:14:38.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.167+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150406abe0 con 0x7f150413c910 2024-04-25T09:14:38.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.167+0000 7f14fe000640 1 -- 172.21.15.62:0/35773202 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14f4045020 con 0x7f150413c910 2024-04-25T09:14:38.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.168+0000 7f14fe000640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14e806b4d0 0x7f14e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.168+0000 7f14fe000640 1 -- 172.21.15.62:0/35773202 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f14f40aa1d0 con 0x7f150413c910 2024-04-25T09:14:38.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.169+0000 7f1509e00640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14e806b4d0 0x7f14e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.170+0000 7f1509e00640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14e806b4d0 0x7f14e806d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1504071b40 tx=0x7f14f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.173+0000 7f14fe000640 1 -- 172.21.15.62:0/35773202 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14f40794e0 con 0x7f150413c910 2024-04-25T09:14:38.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.372+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1504143390 con 0x7f150413c910 2024-04-25T09:14:38.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.373+0000 7f14fe000640 1 -- 172.21.15.62:0/35773202 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1504143390 con 0x7f150413c910 2024-04-25T09:14:38.360 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14e806b4d0 msgr2=0x7f14e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14e806b4d0 0x7f14e806d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1504071b40 tx=0x7f14f8008040 comp rx=0 tx=0).stop 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150413c910 msgr2=0x7f1504071c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150413c910 0x7f1504071c80 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c240 tx=0x7f15040cf1d0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 shutdown_connections 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f14e806b4d0 0x7f14e806d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150413c910 0x7f1504071c80 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f150415b200 0x7f150415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 --2- 172.21.15.62:0/35773202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14ec003f60 0x7f15040ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.375+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 >> 172.21.15.62:0/35773202 conn(0x7f1504066f20 msgr2=0x7f15040b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:38.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.376+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 shutdown_connections 2024-04-25T09:14:38.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.376+0000 7f150b200640 1 -- 172.21.15.62:0/35773202 wait complete. 2024-04-25T09:14:38.382 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T09:14:38.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.613+0000 7f6b69e00640 1 Processor -- start 2024-04-25T09:14:38.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.613+0000 7f6b69e00640 1 -- start start 2024-04-25T09:14:38.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.614+0000 7f6b69e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6b6415f450 0x7f6b640cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.614+0000 7f6b69e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6415faa0 con 0x7f6b64144c10 2024-04-25T09:14:38.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.615+0000 7f6b69e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6415fd80 con 0x7f6b6406d900 2024-04-25T09:14:38.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.615+0000 7f6b69e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b64144ff0 con 0x7f6b6415f450 2024-04-25T09:14:38.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.615+0000 7f6b68a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6b64144c10 0x7f6b6406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35786/0 (socket says 172.21.15.62:35786) 2024-04-25T09:14:38.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.615+0000 7f6b68a00640 1 -- 172.21.15.62:0/1445291522 learned_addr learned my addr 172.21.15.62:0/1445291522 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:38.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.615+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433612525 0 0) 0x7f6b6415faa0 con 0x7f6b64144c10 2024-04-25T09:14:38.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.615+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b48003520 con 0x7f6b64144c10 2024-04-25T09:14:38.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.615+0000 7f6b69400640 1 --2- 172.21.15.62:0/1445291522 >> v2:172.21.15.121:3300/0 conn(0x7f6b6415f450 0x7f6b640cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034490377 0 0) 0x7f6b48003520 con 0x7f6b64144c10 2024-04-25T09:14:38.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 >> v2:172.21.15.121:3300/0 conn(0x7f6b6415f450 msgr2=0x7f6b640cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 --2- 172.21.15.62:0/1445291522 >> v2:172.21.15.121:3300/0 conn(0x7f6b6415f450 0x7f6b640cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 >> v1:172.21.15.121:6789/0 conn(0x7f6b6406d900 legacy=0x7f6b640b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b64160060 con 0x7f6b64144c10 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f6b54001670 con 0x7f6b64144c10 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 >> v1:172.21.15.62:6789/0 conn(0x7f6b64144c10 legacy=0x7f6b6406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b48003a00 0x7f6b48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b48004800 0x7f6b48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b480052e0 0x7f6b4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b64144ff0 con 0x7f6b480052e0 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4800c200 con 0x7f6b48003a00 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4800c400 con 0x7f6b48004800 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.616+0000 7f6b69400640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b480052e0 0x7f6b4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.617+0000 7f6b68a00640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b48003a00 0x7f6b48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.617+0000 7f6b63e00640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b48004800 0x7f6b48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.617+0000 7f6b69400640 1 -- 172.21.15.62:0/1445291522 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b48004800 msgr2=0x7f6b48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:38.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.617+0000 7f6b69400640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b48004800 0x7f6b48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.617+0000 7f6b69400640 1 -- 172.21.15.62:0/1445291522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b48003a00 msgr2=0x7f6b48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:38.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.617+0000 7f6b69400640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b48003a00 0x7f6b48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.617+0000 7f6b69400640 1 -- 172.21.15.62:0/1445291522 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b48003520 con 0x7f6b480052e0 2024-04-25T09:14:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.618+0000 7f6b69400640 1 -- 172.21.15.62:0/1445291522 reap_dead start 2024-04-25T09:14:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.618+0000 7f6b69400640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b480052e0 0x7f6b4800b9c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c007360 tx=0x7f6b48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d4d2560d2a8bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.618+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c009be0 con 0x7f6b480052e0 2024-04-25T09:14:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.618+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c003b90 con 0x7f6b480052e0 2024-04-25T09:14:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.618+0000 7f6b63400640 1 -- 172.21.15.62:0/1445291522 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c004020 con 0x7f6b480052e0 2024-04-25T09:14:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.619+0000 7f6b69e00640 1 -- 172.21.15.62:0/1445291522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b480052e0 msgr2=0x7f6b4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.619+0000 7f6b69e00640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b480052e0 0x7f6b4800b9c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c007360 tx=0x7f6b48005830 comp rx=0 tx=0).stop 2024-04-25T09:14:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.619+0000 7f6b69e00640 1 -- 172.21.15.62:0/1445291522 shutdown_connections 2024-04-25T09:14:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.619+0000 7f6b69e00640 1 --2- 172.21.15.62:0/1445291522 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b480052e0 0x7f6b4800b9c0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.619+0000 7f6b69e00640 1 -- 172.21.15.62:0/1445291522 >> 172.21.15.62:0/1445291522 conn(0x7f6b640b14d0 msgr2=0x7f6b640b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.619+0000 7f6b69e00640 1 -- 172.21.15.62:0/1445291522 shutdown_connections 2024-04-25T09:14:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.619+0000 7f6b69e00640 1 -- 172.21.15.62:0/1445291522 wait complete. 2024-04-25T09:14:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.621+0000 7f6b69e00640 1 Processor -- start 2024-04-25T09:14:38.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.621+0000 7f6b69e00640 1 -- start start 2024-04-25T09:14:38.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.622+0000 7f6b69e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b480052e0 0x7f6b6413f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.622+0000 7f6b69e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b641e5050 0x7f6b64133c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b640cd0f0 0x7f6b6415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640cf050 con 0x7f6b640cd0f0 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6415f880 con 0x7f6b480052e0 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640cf1d0 con 0x7f6b641e5050 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b640cd0f0 0x7f6b6415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b640cd0f0 0x7f6b6415c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47118/0 (socket says 172.21.15.62:47118) 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69400640 1 -- 172.21.15.62:0/4076787590 learned_addr learned my addr 172.21.15.62:0/4076787590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b63e00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b641e5050 0x7f6b64133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b68a00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b480052e0 0x7f6b6413f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69400640 1 -- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b641e5050 msgr2=0x7f6b64133c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69400640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b641e5050 0x7f6b64133c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69400640 1 -- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b480052e0 msgr2=0x7f6b6413f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69400640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b480052e0 0x7f6b6413f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.623+0000 7f6b69400640 1 -- 172.21.15.62:0/4076787590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b64134570 con 0x7f6b640cd0f0 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.624+0000 7f6b69400640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b640cd0f0 0x7f6b6415c600 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c038d30 tx=0x7f6b5c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.624+0000 7f6b60c00640 1 -- 172.21.15.62:0/4076787590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c003990 con 0x7f6b640cd0f0 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.624+0000 7f6b60c00640 1 -- 172.21.15.62:0/4076787590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c042420 con 0x7f6b640cd0f0 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.624+0000 7f6b60c00640 1 -- 172.21.15.62:0/4076787590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c042810 con 0x7f6b640cd0f0 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.624+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b640cc330 con 0x7f6b640cd0f0 2024-04-25T09:14:38.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.624+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b640cc8f0 con 0x7f6b640cd0f0 2024-04-25T09:14:38.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.626+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b641e5430 con 0x7f6b640cd0f0 2024-04-25T09:14:38.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.627+0000 7f6b60c00640 1 -- 172.21.15.62:0/4076787590 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b5c02e420 con 0x7f6b640cd0f0 2024-04-25T09:14:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.627+0000 7f6b60c00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b4406b4d0 0x7f6b4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.628+0000 7f6b60c00640 1 -- 172.21.15.62:0/4076787590 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6b5c0aa2f0 con 0x7f6b640cd0f0 2024-04-25T09:14:38.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.628+0000 7f6b68a00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b4406b4d0 0x7f6b4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:38.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.629+0000 7f6b68a00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b4406b4d0 0x7f6b4406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b54001050 tx=0x7f6b54009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:38.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.632+0000 7f6b60c00640 1 -- 172.21.15.62:0/4076787590 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b5c032030 con 0x7f6b640cd0f0 2024-04-25T09:14:38.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:38.905+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f6b6404ef20 con 0x7f6b640cd0f0 2024-04-25T09:14:39.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.464+0000 7f6b60c00640 1 -- 172.21.15.62:0/4076787590 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v508) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6b5c078d80 con 0x7f6b640cd0f0 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b4406b4d0 msgr2=0x7f6b4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b4406b4d0 0x7f6b4406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b54001050 tx=0x7f6b54009040 comp rx=0 tx=0).stop 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b640cd0f0 msgr2=0x7f6b6415c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b640cd0f0 0x7f6b6415c600 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c038d30 tx=0x7f6b5c004ab0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 shutdown_connections 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b4406b4d0 0x7f6b4406d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6b640cd0f0 0x7f6b6415c600 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6b641e5050 0x7f6b64133c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 --2- 172.21.15.62:0/4076787590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b480052e0 0x7f6b6413f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 >> 172.21.15.62:0/4076787590 conn(0x7f6b640b14d0 msgr2=0x7f6b640717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 shutdown_connections 2024-04-25T09:14:39.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.468+0000 7f6b69e00640 1 -- 172.21.15.62:0/4076787590 wait complete. 2024-04-25T09:14:39.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:14:39.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.706+0000 7f4d41200640 1 Processor -- start 2024-04-25T09:14:39.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.706+0000 7f4d41200640 1 -- start start 2024-04-25T09:14:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.708+0000 7f4d41200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4d3c15bdf0 0x7f4d3c0720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.708+0000 7f4d41200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c06c9b0 0x7f4d3c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:39.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.708+0000 7f4d41200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c06d070 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.708+0000 7f4d41200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c06ce40 con 0x7f4d3c15bdf0 2024-04-25T09:14:39.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.708+0000 7f4d41200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c15c1d0 con 0x7f4d3c06c9b0 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.709+0000 7f4d3a800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c06c9b0 0x7f4d3c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.709+0000 7f4d3a800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c06c9b0 0x7f4d3c0703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:43006/0 (socket says 172.21.15.62:43006) 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.709+0000 7f4d3a800640 1 -- 172.21.15.62:0/78807585 learned_addr learned my addr 172.21.15.62:0/78807585 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.709+0000 7f4d39e00640 1 --2- 172.21.15.62:0/78807585 >> v2:172.21.15.121:3300/0 conn(0x7f4d3c15bdf0 0x7f4d3c0720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.709+0000 7f4d39400640 1 --1- 172.21.15.62:0/78807585 >> v1:172.21.15.121:6789/0 conn(0x7f4d3c0cebc0 0x7f4d3c06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:49944/0 (socket says 172.21.15.62:49944) 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.709+0000 7f4d38a00640 1 -- 172.21.15.62:0/78807585 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481347296 0 0) 0x7f4d3c06d070 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.710+0000 7f4d38a00640 1 -- 172.21.15.62:0/78807585 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d20003520 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.710+0000 7f4d3a800640 1 -- 172.21.15.62:0/78807585 >> v2:172.21.15.121:3300/0 conn(0x7f4d3c15bdf0 msgr2=0x7f4d3c0720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:39.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.710+0000 7f4d3a800640 1 --2- 172.21.15.62:0/78807585 >> v2:172.21.15.121:3300/0 conn(0x7f4d3c15bdf0 0x7f4d3c0720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.710+0000 7f4d3a800640 1 -- 172.21.15.62:0/78807585 >> v1:172.21.15.121:6789/0 conn(0x7f4d3c0cebc0 legacy=0x7f4d3c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:39.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.710+0000 7f4d3a800640 1 -- 172.21.15.62:0/78807585 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c06d350 con 0x7f4d3c06c9b0 2024-04-25T09:14:39.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.711+0000 7f4d3a800640 1 --2- 172.21.15.62:0/78807585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c06c9b0 0x7f4d3c0703a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009e50 tx=0x7f4d3402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89d2f4caca68bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:39.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d38a00640 1 -- 172.21.15.62:0/78807585 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34036070 con 0x7f4d3c06c9b0 2024-04-25T09:14:39.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d38a00640 1 -- 172.21.15.62:0/78807585 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d34031600 con 0x7f4d3c06c9b0 2024-04-25T09:14:39.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d38a00640 1 -- 172.21.15.62:0/78807585 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340319b0 con 0x7f4d3c06c9b0 2024-04-25T09:14:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d41200640 1 -- 172.21.15.62:0/78807585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c06c9b0 msgr2=0x7f4d3c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d41200640 1 --2- 172.21.15.62:0/78807585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c06c9b0 0x7f4d3c0703a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009e50 tx=0x7f4d3402a710 comp rx=0 tx=0).stop 2024-04-25T09:14:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d41200640 1 -- 172.21.15.62:0/78807585 shutdown_connections 2024-04-25T09:14:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d41200640 1 --2- 172.21.15.62:0/78807585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c06c9b0 0x7f4d3c0703a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d41200640 1 --2- 172.21.15.62:0/78807585 >> v2:172.21.15.121:3300/0 conn(0x7f4d3c15bdf0 0x7f4d3c0720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.712+0000 7f4d41200640 1 -- 172.21.15.62:0/78807585 >> 172.21.15.62:0/78807585 conn(0x7f4d3c0b1410 msgr2=0x7f4d3c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.713+0000 7f4d41200640 1 -- 172.21.15.62:0/78807585 shutdown_connections 2024-04-25T09:14:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.713+0000 7f4d41200640 1 -- 172.21.15.62:0/78807585 wait complete. 2024-04-25T09:14:39.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.714+0000 7f4d41200640 1 Processor -- start 2024-04-25T09:14:39.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.715+0000 7f4d41200640 1 -- start start 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d41200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c06c9b0 0x7f4d3c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d41200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d3c0cebc0 0x7f4d3c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d41200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c15bdf0 0x7f4d3c0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d41200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0aedf0 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d41200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0ae8c0 con 0x7f4d3c06c9b0 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d41200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0708e0 con 0x7f4d3c15bdf0 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d39400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d3c0cebc0 0x7f4d3c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d39400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d3c0cebc0 0x7f4d3c154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47126/0 (socket says 172.21.15.62:47126) 2024-04-25T09:14:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d39400640 1 -- 172.21.15.62:0/261360655 learned_addr learned my addr 172.21.15.62:0/261360655 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d39e00640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c06c9b0 0x7f4d3c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.716+0000 7f4d3a800640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c15bdf0 0x7f4d3c0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d39400640 1 -- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c15bdf0 msgr2=0x7f4d3c0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d39400640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c15bdf0 0x7f4d3c0b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d39400640 1 -- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c06c9b0 msgr2=0x7f4d3c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d39400640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c06c9b0 0x7f4d3c0b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d39400640 1 -- 172.21.15.62:0/261360655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0cbdf0 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d39e00640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c06c9b0 0x7f4d3c0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d39400640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d3c0cebc0 0x7f4d3c154750 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400c430 tx=0x7f4d2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d32000640 1 -- 172.21.15.62:0/261360655 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240074e0 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d32000640 1 -- 172.21.15.62:0/261360655 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d240047e0 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c0b2e60 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.717+0000 7f4d32000640 1 -- 172.21.15.62:0/261360655 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24004bd0 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.718+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c0b3420 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.719+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c04ef20 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.720+0000 7f4d32000640 1 -- 172.21.15.62:0/261360655 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d2401a020 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.721+0000 7f4d32000640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4d1806b500 0x7f4d1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:39.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.721+0000 7f4d32000640 1 -- 172.21.15.62:0/261360655 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4d24089740 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.722+0000 7f4d39e00640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4d1806b500 0x7f4d1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:39.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.723+0000 7f4d39e00640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4d1806b500 0x7f4d1806d9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009af0 tx=0x7f4d2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:39.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.726+0000 7f4d32000640 1 -- 172.21.15.62:0/261360655 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d24010300 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.924+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4d3c06c210 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.931+0000 7f4d32000640 1 -- 172.21.15.62:0/261360655 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4d24058970 con 0x7f4d3c0cebc0 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4d1806b500 msgr2=0x7f4d1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4d1806b500 0x7f4d1806d9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009af0 tx=0x7f4d2c008040 comp rx=0 tx=0).stop 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d3c0cebc0 msgr2=0x7f4d3c154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d3c0cebc0 0x7f4d3c154750 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400c430 tx=0x7f4d2400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 shutdown_connections 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4d1806b500 0x7f4d1806d9c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d3c0cebc0 0x7f4d3c154750 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4d3c15bdf0 0x7f4d3c0b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.933+0000 7f4d41200640 1 --2- 172.21.15.62:0/261360655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c06c9b0 0x7f4d3c0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.934+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 >> 172.21.15.62:0/261360655 conn(0x7f4d3c0b1410 msgr2=0x7f4d3c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:39.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.934+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 shutdown_connections 2024-04-25T09:14:39.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:39.934+0000 7f4d41200640 1 -- 172.21.15.62:0/261360655 wait complete. 2024-04-25T09:14:39.941 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:14:39.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:14:40.058 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:14:40.168 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:14:40.169 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:14:40.169 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:40.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:14:40.229 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:14:40.229 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:40.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:14:40.287 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:14:40.288 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:40.347 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:14:40.347 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:14:40.347 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:40.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:14:40.405 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:14:40.405 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:40.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.642+0000 7fe70d000640 1 Processor -- start 2024-04-25T09:14:40.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.642+0000 7fe70d000640 1 -- start start 2024-04-25T09:14:40.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.644+0000 7fe70d000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe70806d900 0x7fe708165370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:40.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.644+0000 7fe70d000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080b1220 con 0x7fe70815acf0 2024-04-25T09:14:40.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.644+0000 7fe70d000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70815b220 con 0x7fe70813d970 2024-04-25T09:14:40.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.644+0000 7fe70d000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70815b500 con 0x7fe70806d900 2024-04-25T09:14:40.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.644+0000 7fe706a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe70815acf0 0x7fe70806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35794/0 (socket says 172.21.15.62:35794) 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.644+0000 7fe706a00640 1 -- 172.21.15.62:0/3702692479 learned_addr learned my addr 172.21.15.62:0/3702692479 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe707e00640 1 --2- 172.21.15.62:0/3702692479 >> v2:172.21.15.121:3300/0 conn(0x7fe70806d900 0x7fe708165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037508075 0 0) 0x7fe7080b1220 con 0x7fe70815acf0 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6ec003520 con 0x7fe70815acf0 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1101370498 0 0) 0x7fe6ec003520 con 0x7fe70815acf0 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 >> v2:172.21.15.121:3300/0 conn(0x7fe70806d900 msgr2=0x7fe708165370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 --2- 172.21.15.62:0/3702692479 >> v2:172.21.15.121:3300/0 conn(0x7fe70806d900 0x7fe708165370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 >> v1:172.21.15.121:6789/0 conn(0x7fe70813d970 legacy=0x7fe7080b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70815b7e0 con 0x7fe70815acf0 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fe6f4001670 con 0x7fe70815acf0 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 >> v1:172.21.15.62:6789/0 conn(0x7fe70815acf0 legacy=0x7fe70806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6ec0039b0 0x7fe6ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:40.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe6ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6ec0051c0 0x7fe6ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70815b500 con 0x7fe6ec004800 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec00c060 con 0x7fe6ec0051c0 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.645+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec00c260 con 0x7fe6ec0039b0 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe706a00640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe6ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe707400640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6ec0039b0 0x7fe6ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe707e00640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6ec0051c0 0x7fe6ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe706a00640 1 -- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6ec0039b0 msgr2=0x7fe6ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe706a00640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6ec0039b0 0x7fe6ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe706a00640 1 -- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6ec0051c0 msgr2=0x7fe6ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe706a00640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6ec0051c0 0x7fe6ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe706a00640 1 -- 172.21.15.62:0/3702692479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec003520 con 0x7fe6ec004800 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe707400640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe6ec0039b0 0x7fe6ec003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe707e00640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe6ec0051c0 0x7fe6ec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.646+0000 7fe707e00640 1 -- 172.21.15.62:0/3702692479 reap_dead start 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.647+0000 7fe706a00640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe6ec004c80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c210 tx=0x7fe6ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31af4fff12d4b71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4007cf0 con 0x7fe6ec004800 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f4005af0 con 0x7fe6ec004800 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe706000640 1 -- 172.21.15.62:0/3702692479 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4007070 con 0x7fe6ec004800 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe70d000640 1 -- 172.21.15.62:0/3702692479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 msgr2=0x7fe6ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe70d000640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe6ec004c80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c210 tx=0x7fe6ec005830 comp rx=0 tx=0).stop 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe70d000640 1 -- 172.21.15.62:0/3702692479 shutdown_connections 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe70d000640 1 --2- 172.21.15.62:0/3702692479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe6ec004c80 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe70d000640 1 -- 172.21.15.62:0/3702692479 >> 172.21.15.62:0/3702692479 conn(0x7fe7080b14d0 msgr2=0x7fe7080b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe70d000640 1 -- 172.21.15.62:0/3702692479 shutdown_connections 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.648+0000 7fe70d000640 1 -- 172.21.15.62:0/3702692479 wait complete. 2024-04-25T09:14:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.650+0000 7fe70d000640 1 Processor -- start 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.650+0000 7fe70d000640 1 -- start start 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.651+0000 7fe70d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe708135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.651+0000 7fe70d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7081431c0 0x7fe708143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.651+0000 7fe70d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7080b4120 0x7fe7080b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.651+0000 7fe70d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080b0c90 con 0x7fe6ec004800 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.651+0000 7fe70d000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080b0b10 con 0x7fe7081431c0 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.651+0000 7fe70d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080b0e10 con 0x7fe7080b4120 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe708135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe708135640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47152/0 (socket says 172.21.15.62:47152) 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 -- 172.21.15.62:0/251960940 learned_addr learned my addr 172.21.15.62:0/251960940 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 -- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7080b4120 msgr2=0x7fe7080b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707e00640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7080b4120 0x7fe7080b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7080b4120 0x7fe7080b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe706a00640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7081431c0 0x7fe708143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 -- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7081431c0 msgr2=0x7fe708143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7081431c0 0x7fe708143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 -- 172.21.15.62:0/251960940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080b2c00 con 0x7fe6ec004800 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.652+0000 7fe707400640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe708135640 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800d4b0 tx=0x7fe6f800d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.653+0000 7fe6ff400640 1 -- 172.21.15.62:0/251960940 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8005480 con 0x7fe6ec004800 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.653+0000 7fe6ff400640 1 -- 172.21.15.62:0/251960940 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8017070 con 0x7fe6ec004800 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.653+0000 7fe6ff400640 1 -- 172.21.15.62:0/251960940 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8003830 con 0x7fe6ec004800 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.653+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7080b4d00 con 0x7fe6ec004800 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.653+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7080b52c0 con 0x7fe6ec004800 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.654+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe708143b80 con 0x7fe6ec004800 2024-04-25T09:14:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.655+0000 7fe6ff400640 1 -- 172.21.15.62:0/251960940 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6f80039d0 con 0x7fe6ec004800 2024-04-25T09:14:40.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.656+0000 7fe6ff400640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe6e806b4d0 0x7fe6e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:40.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.656+0000 7fe6ff400640 1 -- 172.21.15.62:0/251960940 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe6f8089c70 con 0x7fe6ec004800 2024-04-25T09:14:40.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.657+0000 7fe706a00640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe6e806b4d0 0x7fe6e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:40.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.658+0000 7fe706a00640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe6e806b4d0 0x7fe6e806d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4032000 tx=0x7fe6f4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:40.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.660+0000 7fe6ff400640 1 -- 172.21.15.62:0/251960940 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7080b2c00 con 0x7fe6ec004800 2024-04-25T09:14:40.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.902+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe70804ef20 con 0x7fe6ec004800 2024-04-25T09:14:40.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.903+0000 7fe6ff400640 1 -- 172.21.15.62:0/251960940 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fe6f8058ea0 con 0x7fe6ec004800 2024-04-25T09:14:40.891 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:40.891 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5855,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1217641676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1217641676},{"type":"v1","addr":"172.21.15.121:6836","nonce":1217641676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":6217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2614109972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2614109972},{"type":"v1","addr":"172.21.15.62:6835","nonce":2614109972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":6232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1632311193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1632311193},{"type":"v1","addr":"172.21.15.121:6837","nonce":1632311193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7788,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/385213484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":385213484},{"type":"v1","addr":"172.21.15.62:6834","nonce":385213484}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[]} 2024-04-25T09:14:40.891 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 73 2024-04-25T09:14:40.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe6e806b4d0 msgr2=0x7fe6e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:40.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe6e806b4d0 0x7fe6e806d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4032000 tx=0x7fe6f4009030 comp rx=0 tx=0).stop 2024-04-25T09:14:40.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 msgr2=0x7fe708135640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:40.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe708135640 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800d4b0 tx=0x7fe6f800d970 comp rx=0 tx=0).stop 2024-04-25T09:14:40.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 shutdown_connections 2024-04-25T09:14:40.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe6e806b4d0 0x7fe6e806d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe7080b4120 0x7fe7080b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7081431c0 0x7fe708143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 --2- 172.21.15.62:0/251960940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6ec004800 0x7fe708135640 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 >> 172.21.15.62:0/251960940 conn(0x7fe7080b14d0 msgr2=0x7fe70815c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 shutdown_connections 2024-04-25T09:14:40.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:40.906+0000 7fe70d000640 1 -- 172.21.15.62:0/251960940 wait complete. 2024-04-25T09:14:40.913 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:14:40.913 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:14:40.913 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:14:40.913 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:14:40.913 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:14:40.913 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:14:40.914 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:14:40.914 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:14:40.914 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:14:40.914 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:14:40.914 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:14:40.914 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:14:40.915 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:14:40.915 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:14:40.915 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:14:40.915 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:14:40.931 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:14:40.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:14:40.935 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:14:40.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:14:40.939 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:14:40.939 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:14:40.940 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:14:40.940 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:14:40.980 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:14:40.981 DEBUG:teuthology.parallel:result is None 2024-04-25T09:14:40.985 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:14:40.985 DEBUG:teuthology.parallel:result is None 2024-04-25T09:14:40.988 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:14:40.988 DEBUG:teuthology.parallel:result is None 2024-04-25T09:14:40.993 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:14:40.994 DEBUG:teuthology.parallel:result is None 2024-04-25T09:14:40.994 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:14:41.130 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:14:41.133 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:14:41.146 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:14:41.149 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:14:41.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.192+0000 7f19cc000640 1 Processor -- start 2024-04-25T09:14:41.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.192+0000 7f19cc000640 1 -- start start 2024-04-25T09:14:41.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.193+0000 7f19cc000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19c40cfed0 0x7f19c415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:41.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.193+0000 7f19cc000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19c4072340 0x7f19c40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:41.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.193+0000 7f19cc000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.193+0000 7f19cc000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c4070c60 con 0x7f19c40cfed0 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.193+0000 7f19cc000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c406c310 con 0x7f19c4072340 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.193+0000 7f19cc000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c415f5f0 con 0x7f19c40707f0 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c9600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19c40cfed0 0x7f19c415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c9600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f19c40cfed0 0x7f19c415f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47188/0 (socket says 172.21.15.62:47188) 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c9600640 1 -- 172.21.15.62:0/365144787 learned_addr learned my addr 172.21.15.62:0/365144787 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c8c00640 1 --2- 172.21.15.62:0/365144787 >> v2:172.21.15.121:3300/0 conn(0x7f19c4072340 0x7f19c40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19ca000640 1 --2- 172.21.15.62:0/365144787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c9600640 1 -- 172.21.15.62:0/365144787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 msgr2=0x7f19c415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c9600640 1 --2- 172.21.15.62:0/365144787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c9600640 1 -- 172.21.15.62:0/365144787 >> v2:172.21.15.121:3300/0 conn(0x7f19c4072340 msgr2=0x7f19c40702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c9600640 1 --2- 172.21.15.62:0/365144787 >> v2:172.21.15.121:3300/0 conn(0x7f19c4072340 0x7f19c40702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:41.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c9600640 1 -- 172.21.15.62:0/365144787 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c406c5b0 con 0x7f19c40cfed0 2024-04-25T09:14:41.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.194+0000 7f19c8c00640 1 --2- 172.21.15.62:0/365144787 >> v2:172.21.15.121:3300/0 conn(0x7f19c4072340 0x7f19c40702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.195+0000 7f19c9600640 1 --2- 172.21.15.62:0/365144787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c40cfed0 0x7f19c415f0b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f19c4072760 tx=0x7f19b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff0ed75a52a8620 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19c3e00640 1 -- 172.21.15.62:0/365144787 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b8037070 con 0x7f19c40cfed0 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19c3e00640 1 -- 172.21.15.62:0/365144787 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b8032440 con 0x7f19c40cfed0 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19c3e00640 1 -- 172.21.15.62:0/365144787 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b8032810 con 0x7f19c40cfed0 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19cc000640 1 -- 172.21.15.62:0/365144787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c40cfed0 msgr2=0x7f19c415f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19cc000640 1 --2- 172.21.15.62:0/365144787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c40cfed0 0x7f19c415f0b0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f19c4072760 tx=0x7f19b802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19cc000640 1 -- 172.21.15.62:0/365144787 shutdown_connections 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19cc000640 1 --2- 172.21.15.62:0/365144787 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19cc000640 1 --2- 172.21.15.62:0/365144787 >> v2:172.21.15.121:3300/0 conn(0x7f19c4072340 0x7f19c40702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:41.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19cc000640 1 --2- 172.21.15.62:0/365144787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c40cfed0 0x7f19c415f0b0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.196+0000 7f19cc000640 1 -- 172.21.15.62:0/365144787 >> 172.21.15.62:0/365144787 conn(0x7f19c40668f0 msgr2=0x7f19c4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.197+0000 7f19cc000640 1 -- 172.21.15.62:0/365144787 shutdown_connections 2024-04-25T09:14:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.197+0000 7f19cc000640 1 -- 172.21.15.62:0/365144787 wait complete. 2024-04-25T09:14:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.198+0000 7f19cc000640 1 Processor -- start 2024-04-25T09:14:41.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.198+0000 7f19cc000640 1 -- start start 2024-04-25T09:14:41.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.199+0000 7f19cc000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.199+0000 7f19cc000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19c40cfed0 0x7f19c40cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.199+0000 7f19cc000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c4155ce0 0x7f19c4156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.199+0000 7f19cc000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c406ce50 con 0x7f19c4155ce0 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.199+0000 7f19cc000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c406cb70 con 0x7f19c40cfed0 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.199+0000 7f19cc000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c406d130 con 0x7f19c40707f0 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c4155ce0 0x7f19c4156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c4155ce0 0x7f19c4156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47196/0 (socket says 172.21.15.62:47196) 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 -- 172.21.15.62:0/3593527110 learned_addr learned my addr 172.21.15.62:0/3593527110 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19c9600640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19c8c00640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19c40cfed0 0x7f19c40cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 -- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 msgr2=0x7f19c415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 -- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19c40cfed0 msgr2=0x7f19c40cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:41.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19c40cfed0 0x7f19c40cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:41.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 -- 172.21.15.62:0/3593527110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c413df40 con 0x7f19c4155ce0 2024-04-25T09:14:41.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19c9600640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:14:41.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.200+0000 7f19ca000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c4155ce0 0x7f19c4156170 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00eed0 tx=0x7f19bc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.201+0000 7f19c1600640 1 -- 172.21.15.62:0/3593527110 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc018020 con 0x7f19c4155ce0 2024-04-25T09:14:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.201+0000 7f19c1600640 1 -- 172.21.15.62:0/3593527110 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc00bd20 con 0x7f19c4155ce0 2024-04-25T09:14:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.201+0000 7f19c1600640 1 -- 172.21.15.62:0/3593527110 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc014640 con 0x7f19c4155ce0 2024-04-25T09:14:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.201+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c415e3a0 con 0x7f19c4155ce0 2024-04-25T09:14:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.201+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c413e4c0 con 0x7f19c4155ce0 2024-04-25T09:14:41.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.202+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c40719e0 con 0x7f19c4155ce0 2024-04-25T09:14:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.204+0000 7f19c1600640 1 -- 172.21.15.62:0/3593527110 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19bc005d10 con 0x7f19c4155ce0 2024-04-25T09:14:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.204+0000 7f19c1600640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19a406b4d0 0x7f19a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.205+0000 7f19c1600640 1 -- 172.21.15.62:0/3593527110 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f19bc08ab70 con 0x7f19c4155ce0 2024-04-25T09:14:41.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.205+0000 7f19c9600640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19a406b4d0 0x7f19a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:41.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.206+0000 7f19c9600640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19a406b4d0 0x7f19a406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f19b8034000 tx=0x7f19b802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:41.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.207+0000 7f19c1600640 1 -- 172.21.15.62:0/3593527110 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19c40719e0 con 0x7f19c4155ce0 2024-04-25T09:14:41.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:41.419+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f19c4072c70 con 0x7f19c4155ce0 2024-04-25T09:14:42.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.507+0000 7f19c1600640 1 -- 172.21.15.62:0/3593527110 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v119) v1 ==== 97+0+0 (secure 0 0 0) 0x7f19bc022070 con 0x7f19c4155ce0 2024-04-25T09:14:42.495 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19a406b4d0 msgr2=0x7f19a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19a406b4d0 0x7f19a406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f19b8034000 tx=0x7f19b802e000 comp rx=0 tx=0).stop 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c4155ce0 msgr2=0x7f19c4156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c4155ce0 0x7f19c4156170 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00eed0 tx=0x7f19bc00b480 comp rx=0 tx=0).stop 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 shutdown_connections 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19a406b4d0 0x7f19a406d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f19c4155ce0 0x7f19c4156170 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19c40cfed0 0x7f19c40cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 --2- 172.21.15.62:0/3593527110 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f19c40707f0 0x7f19c415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 >> 172.21.15.62:0/3593527110 conn(0x7f19c40668f0 msgr2=0x7f19c40b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.511+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 shutdown_connections 2024-04-25T09:14:42.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.512+0000 7f19cc000640 1 -- 172.21.15.62:0/3593527110 wait complete. 2024-04-25T09:14:42.521 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:14:42.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.738+0000 7f8288200640 1 Processor -- start 2024-04-25T09:14:42.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.738+0000 7f8288200640 1 -- start start 2024-04-25T09:14:42.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.740+0000 7f8288200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f828006caf0 0x7f82800b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:42.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.740+0000 7f8288200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82800b3b70 con 0x7f82800adff0 2024-04-25T09:14:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.740+0000 7f8288200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828006d110 con 0x7f8280144100 2024-04-25T09:14:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.740+0000 7f8288200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82800b4460 con 0x7f828006caf0 2024-04-25T09:14:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.740+0000 7f8284e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f82800adff0 0x7f828006c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35804/0 (socket says 172.21.15.62:35804) 2024-04-25T09:14:42.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.740+0000 7f8284e00640 1 -- 172.21.15.62:0/2280140530 learned_addr learned my addr 172.21.15.62:0/2280140530 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241796315 0 0) 0x7f82800b3b70 con 0x7f82800adff0 2024-04-25T09:14:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f8286200640 1 --2- 172.21.15.62:0/2280140530 >> v2:172.21.15.121:3300/0 conn(0x7f828006caf0 0x7f82800b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8268003520 con 0x7f82800adff0 2024-04-25T09:14:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603158095 0 0) 0x7f8268003520 con 0x7f82800adff0 2024-04-25T09:14:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 >> v2:172.21.15.121:3300/0 conn(0x7f828006caf0 msgr2=0x7f82800b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 --2- 172.21.15.62:0/2280140530 >> v2:172.21.15.121:3300/0 conn(0x7f828006caf0 0x7f82800b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 >> v1:172.21.15.121:6789/0 conn(0x7f8280144100 legacy=0x7f82800ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82800b3e10 con 0x7f82800adff0 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f826c001670 con 0x7f82800adff0 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 >> v1:172.21.15.62:6789/0 conn(0x7f82800adff0 legacy=0x7f828006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.741+0000 7f827fe00640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82680039b0 0x7f8268003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f827fe00640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268004800 0x7f8268004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f827fe00640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8268005210 0x7f826800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82800b4460 con 0x7f8268004800 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826800c140 con 0x7f82680039b0 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826800c340 con 0x7f8268005210 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8284e00640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268004800 0x7f8268004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8285800640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82680039b0 0x7f8268003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8286200640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8268005210 0x7f826800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8284e00640 1 -- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8268005210 msgr2=0x7f826800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:42.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8284e00640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8268005210 0x7f826800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8284e00640 1 -- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82680039b0 msgr2=0x7f8268003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8284e00640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82680039b0 0x7f8268003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8284e00640 1 -- 172.21.15.62:0/2280140530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268003520 con 0x7f8268004800 2024-04-25T09:14:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8286200640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8268005210 0x7f826800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:42.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.742+0000 7f8286200640 1 -- 172.21.15.62:0/2280140530 reap_dead start 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.743+0000 7f8284e00640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268004800 0x7f8268004c80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c240 tx=0x7f8268005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5a8f8267a0b8d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.744+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c007cf0 con 0x7f8268004800 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.744+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c005af0 con 0x7f8268004800 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.744+0000 7f827fe00640 1 -- 172.21.15.62:0/2280140530 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c007070 con 0x7f8268004800 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.744+0000 7f8288200640 1 -- 172.21.15.62:0/2280140530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268004800 msgr2=0x7f8268004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.744+0000 7f8288200640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268004800 0x7f8268004c80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c240 tx=0x7f8268005830 comp rx=0 tx=0).stop 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.744+0000 7f8288200640 1 -- 172.21.15.62:0/2280140530 shutdown_connections 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.744+0000 7f8288200640 1 --2- 172.21.15.62:0/2280140530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8268004800 0x7f8268004c80 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.744+0000 7f8288200640 1 -- 172.21.15.62:0/2280140530 >> 172.21.15.62:0/2280140530 conn(0x7f82800668f0 msgr2=0x7f8280066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.745+0000 7f8288200640 1 -- 172.21.15.62:0/2280140530 shutdown_connections 2024-04-25T09:14:42.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.745+0000 7f8288200640 1 -- 172.21.15.62:0/2280140530 wait complete. 2024-04-25T09:14:42.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.746+0000 7f8288200640 1 Processor -- start 2024-04-25T09:14:42.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.747+0000 7f8288200640 1 -- start start 2024-04-25T09:14:42.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8288200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8268004800 0x7f82801409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8288200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f828013f4c0 0x7f82801f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8288200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82801f3aa0 0x7f82801f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8288200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8280144660 con 0x7f82801f3aa0 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8288200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801444e0 con 0x7f8268004800 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8288200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82800b48f0 con 0x7f828013f4c0 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8286200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82801f3aa0 0x7f82801f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8286200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82801f3aa0 0x7f82801f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47206/0 (socket says 172.21.15.62:47206) 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8286200640 1 -- 172.21.15.62:0/664416801 learned_addr learned my addr 172.21.15.62:0/664416801 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8285800640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8268004800 0x7f82801409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8284e00640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f828013f4c0 0x7f82801f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8286200640 1 -- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f828013f4c0 msgr2=0x7f82801f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8286200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f828013f4c0 0x7f82801f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8286200640 1 -- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8268004800 msgr2=0x7f82801409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:42.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8286200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8268004800 0x7f82801409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:42.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.748+0000 7f8286200640 1 -- 172.21.15.62:0/664416801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828013f950 con 0x7f82801f3aa0 2024-04-25T09:14:42.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.749+0000 7f8286200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82801f3aa0 0x7f82801f3f30 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f82800af4a0 tx=0x7f8278009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:42.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.749+0000 7f827d600640 1 -- 172.21.15.62:0/664416801 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278019070 con 0x7f82801f3aa0 2024-04-25T09:14:42.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.749+0000 7f827d600640 1 -- 172.21.15.62:0/664416801 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8278007d40 con 0x7f82801f3aa0 2024-04-25T09:14:42.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.749+0000 7f827d600640 1 -- 172.21.15.62:0/664416801 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278014640 con 0x7f82801f3aa0 2024-04-25T09:14:42.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.749+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828013fc30 con 0x7f82801f3aa0 2024-04-25T09:14:42.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.749+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82801401f0 con 0x7f82801f3aa0 2024-04-25T09:14:42.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.751+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828004ef20 con 0x7f82801f3aa0 2024-04-25T09:14:42.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.752+0000 7f827d600640 1 -- 172.21.15.62:0/664416801 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f827801c070 con 0x7f82801f3aa0 2024-04-25T09:14:42.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.753+0000 7f827d600640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f826006b500 0x7f826006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:42.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.753+0000 7f827d600640 1 -- 172.21.15.62:0/664416801 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8278089a70 con 0x7f82801f3aa0 2024-04-25T09:14:42.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.754+0000 7f8285800640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f826006b500 0x7f826006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:42.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.755+0000 7f8285800640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f826006b500 0x7f826006d9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8274007fd0 tx=0x7f8274009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:42.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:42.758+0000 7f827d600640 1 -- 172.21.15.62:0/664416801 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8278058ba0 con 0x7f82801f3aa0 2024-04-25T09:14:43.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.029+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f828006c540 con 0x7f82801f3aa0 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.029+0000 7f827d600640 1 -- 172.21.15.62:0/664416801 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f82780584a0 con 0x7f82801f3aa0 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:14:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:14:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:14:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:14:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:14:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:14:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:14:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:14:43.029 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:14:43.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f826006b500 msgr2=0x7f826006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f826006b500 0x7f826006d9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8274007fd0 tx=0x7f8274009040 comp rx=0 tx=0).stop 2024-04-25T09:14:43.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82801f3aa0 msgr2=0x7f82801f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82801f3aa0 0x7f82801f3f30 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f82800af4a0 tx=0x7f8278009d40 comp rx=0 tx=0).stop 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 shutdown_connections 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f826006b500 0x7f826006d9c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82801f3aa0 0x7f82801f3f30 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f828013f4c0 0x7f82801f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 --2- 172.21.15.62:0/664416801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8268004800 0x7f82801409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 >> 172.21.15.62:0/664416801 conn(0x7f82800668f0 msgr2=0x7f82800ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 shutdown_connections 2024-04-25T09:14:43.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.032+0000 7f8288200640 1 -- 172.21.15.62:0/664416801 wait complete. 2024-04-25T09:14:43.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:43.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.261+0000 7f370b200640 1 Processor -- start 2024-04-25T09:14:43.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.262+0000 7f370b200640 1 -- start start 2024-04-25T09:14:43.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.263+0000 7f370b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f370c13d970 0x7f370c0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.263+0000 7f370b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c0b1050 0x7f370c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.263+0000 7f370b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c15b220 con 0x7f370c15acf0 2024-04-25T09:14:43.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.263+0000 7f370b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c15b500 con 0x7f370c13d970 2024-04-25T09:14:43.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.263+0000 7f370b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c15b7e0 con 0x7f370c0b1050 2024-04-25T09:14:43.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.264+0000 7f3703e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c0b1050 0x7f370c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.264+0000 7f3703e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c0b1050 0x7f370c06d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:43072/0 (socket says 172.21.15.62:43072) 2024-04-25T09:14:43.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.264+0000 7f3703e00640 1 -- 172.21.15.62:0/1889615074 learned_addr learned my addr 172.21.15.62:0/1889615074 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:43.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.264+0000 7f3709e00640 1 --2- 172.21.15.62:0/1889615074 >> v2:172.21.15.121:3300/0 conn(0x7f370c13d970 0x7f370c0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.265+0000 7f3703e00640 1 -- 172.21.15.62:0/1889615074 >> v2:172.21.15.121:3300/0 conn(0x7f370c13d970 msgr2=0x7f370c0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:43.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.265+0000 7f3703e00640 1 --2- 172.21.15.62:0/1889615074 >> v2:172.21.15.121:3300/0 conn(0x7f370c13d970 0x7f370c0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.265+0000 7f3703e00640 1 -- 172.21.15.62:0/1889615074 >> v1:172.21.15.121:6789/0 conn(0x7f370c15acf0 legacy=0x7f370c13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.265+0000 7f3703e00640 1 -- 172.21.15.62:0/1889615074 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c15bac0 con 0x7f370c0b1050 2024-04-25T09:14:43.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.265+0000 7f3709e00640 1 --2- 172.21.15.62:0/1889615074 >> v2:172.21.15.121:3300/0 conn(0x7f370c13d970 0x7f370c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:14:43.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.266+0000 7f3703e00640 1 --2- 172.21.15.62:0/1889615074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c0b1050 0x7f370c06d1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009870 tx=0x7f36f402c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5924374d4d02cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:43.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.266+0000 7f3709400640 1 -- 172.21.15.62:0/1889615074 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4038070 con 0x7f370c0b1050 2024-04-25T09:14:43.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f3709400640 1 -- 172.21.15.62:0/1889615074 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f4031710 con 0x7f370c0b1050 2024-04-25T09:14:43.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f3709400640 1 -- 172.21.15.62:0/1889615074 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4031ae0 con 0x7f370c0b1050 2024-04-25T09:14:43.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f370b200640 1 -- 172.21.15.62:0/1889615074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c0b1050 msgr2=0x7f370c06d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f370b200640 1 --2- 172.21.15.62:0/1889615074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c0b1050 0x7f370c06d1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009870 tx=0x7f36f402c460 comp rx=0 tx=0).stop 2024-04-25T09:14:43.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f370b200640 1 -- 172.21.15.62:0/1889615074 shutdown_connections 2024-04-25T09:14:43.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f370b200640 1 --2- 172.21.15.62:0/1889615074 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c0b1050 0x7f370c06d1f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f370b200640 1 --2- 172.21.15.62:0/1889615074 >> v2:172.21.15.121:3300/0 conn(0x7f370c13d970 0x7f370c0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f370b200640 1 -- 172.21.15.62:0/1889615074 >> 172.21.15.62:0/1889615074 conn(0x7f370c0b14d0 msgr2=0x7f370c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:43.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f370b200640 1 -- 172.21.15.62:0/1889615074 shutdown_connections 2024-04-25T09:14:43.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.267+0000 7f370b200640 1 -- 172.21.15.62:0/1889615074 wait complete. 2024-04-25T09:14:43.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.269+0000 7f370b200640 1 Processor -- start 2024-04-25T09:14:43.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.269+0000 7f370b200640 1 -- start start 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.270+0000 7f370b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370c0b1050 0x7f370c154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.270+0000 7f370b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c13d970 0x7f370c0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.270+0000 7f370b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c15acf0 0x7f370c1e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.270+0000 7f370b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c06d730 con 0x7f370c0b1050 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.270+0000 7f370b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c13f470 con 0x7f370c15acf0 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.270+0000 7f370b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c06d8b0 con 0x7f370c13d970 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370c0b1050 0x7f370c154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370c0b1050 0x7f370c154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47216/0 (socket says 172.21.15.62:47216) 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 -- 172.21.15.62:0/554712932 learned_addr learned my addr 172.21.15.62:0/554712932 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:43.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3703e00640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c13d970 0x7f370c0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 -- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c13d970 msgr2=0x7f370c0b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c13d970 0x7f370c0b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f370a800640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c15acf0 0x7f370c1e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 -- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c15acf0 msgr2=0x7f370c1e4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c15acf0 0x7f370c1e4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 -- 172.21.15.62:0/554712932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0b4df0 con 0x7f370c0b1050 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f370a800640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c15acf0 0x7f370c1e4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.271+0000 7f3709e00640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370c0b1050 0x7f370c154520 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00ec10 tx=0x7f36fc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.272+0000 7f3702000640 1 -- 172.21.15.62:0/554712932 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc017070 con 0x7f370c0b1050 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.272+0000 7f3702000640 1 -- 172.21.15.62:0/554712932 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc00cc30 con 0x7f370c0b1050 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.272+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c0b2d20 con 0x7f370c0b1050 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.272+0000 7f3702000640 1 -- 172.21.15.62:0/554712932 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc013470 con 0x7f370c0b1050 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.272+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c0b32e0 con 0x7f370c0b1050 2024-04-25T09:14:43.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.273+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c04ef20 con 0x7f370c0b1050 2024-04-25T09:14:43.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.274+0000 7f3702000640 1 -- 172.21.15.62:0/554712932 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36fc01a070 con 0x7f370c0b1050 2024-04-25T09:14:43.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.275+0000 7f3702000640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f36e806b500 0x7f36e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.275+0000 7f3702000640 1 -- 172.21.15.62:0/554712932 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f36fc089f40 con 0x7f370c0b1050 2024-04-25T09:14:43.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.276+0000 7f3703e00640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f36e806b500 0x7f36e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.277+0000 7f3703e00640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f36e806b500 0x7f36e806d9c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f36f4035000 tx=0x7f36f4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:43.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.280+0000 7f3702000640 1 -- 172.21.15.62:0/554712932 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36fc058fe0 con 0x7f370c0b1050 2024-04-25T09:14:43.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.484+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f370c0b5020 con 0x7f370c0b1050 2024-04-25T09:14:43.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.486+0000 7f3702000640 1 -- 172.21.15.62:0/554712932 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+12889 (secure 0 0 0) 0x7f36fc0588e0 con 0x7f370c0b1050 2024-04-25T09:14:43.473 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:43.474 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":119,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:42.505439+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:43.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f36e806b500 msgr2=0x7f36e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f36e806b500 0x7f36e806d9c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f36f4035000 tx=0x7f36f4008030 comp rx=0 tx=0).stop 2024-04-25T09:14:43.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370c0b1050 msgr2=0x7f370c154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370c0b1050 0x7f370c154520 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00ec10 tx=0x7f36fc00c520 comp rx=0 tx=0).stop 2024-04-25T09:14:43.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 shutdown_connections 2024-04-25T09:14:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f36e806b500 0x7f36e806d9c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c15acf0 0x7f370c1e4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f370c13d970 0x7f370c0b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 --2- 172.21.15.62:0/554712932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f370c0b1050 0x7f370c154520 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 >> 172.21.15.62:0/554712932 conn(0x7f370c0b14d0 msgr2=0x7f370c071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 shutdown_connections 2024-04-25T09:14:43.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.489+0000 7f370b200640 1 -- 172.21.15.62:0/554712932 wait complete. 2024-04-25T09:14:43.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:14:43.495 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:14:43.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.729+0000 7f0dee800640 1 Processor -- start 2024-04-25T09:14:43.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.729+0000 7f0dee800640 1 -- start start 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.730+0000 7f0dee800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 0x7f0de80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.730+0000 7f0dee800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0de80d2fb0 0x7f0de80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.730+0000 7f0dee800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0de80d3960 0x7f0de80d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.730+0000 7f0dee800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de80ce780 con 0x7f0de80d3960 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.730+0000 7f0dee800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de80d4310 con 0x7f0de80d2fb0 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.730+0000 7f0dee800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80d3440 con 0x7f0de815dd50 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.730+0000 7f0dede00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0de80d3960 0x7f0de80d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.730+0000 7f0dede00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0de80d3960 0x7f0de80d3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47218/0 (socket says 172.21.15.62:47218) 2024-04-25T09:14:43.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0dede00640 1 -- 172.21.15.62:0/1478037032 learned_addr learned my addr 172.21.15.62:0/1478037032 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0ded400640 1 --2- 172.21.15.62:0/1478037032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 0x7f0de80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0deca00640 1 --2- 172.21.15.62:0/1478037032 >> v2:172.21.15.121:3300/0 conn(0x7f0de80d2fb0 0x7f0de80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0dede00640 1 -- 172.21.15.62:0/1478037032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 msgr2=0x7f0de80d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0dede00640 1 --2- 172.21.15.62:0/1478037032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 0x7f0de80d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0dede00640 1 -- 172.21.15.62:0/1478037032 >> v2:172.21.15.121:3300/0 conn(0x7f0de80d2fb0 msgr2=0x7f0de80ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0dede00640 1 --2- 172.21.15.62:0/1478037032 >> v2:172.21.15.121:3300/0 conn(0x7f0de80d2fb0 0x7f0de80ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0dede00640 1 -- 172.21.15.62:0/1478037032 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de80ce1c0 con 0x7f0de80d3960 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.731+0000 7f0ded400640 1 --2- 172.21.15.62:0/1478037032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 0x7f0de80d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.732+0000 7f0dede00640 1 --2- 172.21.15.62:0/1478037032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80d3960 0x7f0de80d3dd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0de8072f60 tx=0x7f0de002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57f71474dd16e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:43.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0de7e00640 1 -- 172.21.15.62:0/1478037032 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0037070 con 0x7f0de80d3960 2024-04-25T09:14:43.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0de7e00640 1 -- 172.21.15.62:0/1478037032 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0032440 con 0x7f0de80d3960 2024-04-25T09:14:43.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0de7e00640 1 -- 172.21.15.62:0/1478037032 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0032810 con 0x7f0de80d3960 2024-04-25T09:14:43.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0dee800640 1 -- 172.21.15.62:0/1478037032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80d3960 msgr2=0x7f0de80d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0dee800640 1 --2- 172.21.15.62:0/1478037032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80d3960 0x7f0de80d3dd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0de8072f60 tx=0x7f0de002a200 comp rx=0 tx=0).stop 2024-04-25T09:14:43.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0dee800640 1 -- 172.21.15.62:0/1478037032 shutdown_connections 2024-04-25T09:14:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0dee800640 1 --2- 172.21.15.62:0/1478037032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80d3960 0x7f0de80d3dd0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0dee800640 1 --2- 172.21.15.62:0/1478037032 >> v2:172.21.15.121:3300/0 conn(0x7f0de80d2fb0 0x7f0de80ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0dee800640 1 --2- 172.21.15.62:0/1478037032 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 0x7f0de80d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.733+0000 7f0dee800640 1 -- 172.21.15.62:0/1478037032 >> 172.21.15.62:0/1478037032 conn(0x7f0de806b000 msgr2=0x7f0de806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.734+0000 7f0dee800640 1 -- 172.21.15.62:0/1478037032 shutdown_connections 2024-04-25T09:14:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.734+0000 7f0dee800640 1 -- 172.21.15.62:0/1478037032 wait complete. 2024-04-25T09:14:43.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.735+0000 7f0dee800640 1 Processor -- start 2024-04-25T09:14:43.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.735+0000 7f0dee800640 1 -- start start 2024-04-25T09:14:43.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.736+0000 7f0dee800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0de80d3960 0x7f0de814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.736+0000 7f0dee800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 0x7f0de80b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.736+0000 7f0dee800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80b3930 0x7f0de8158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dee800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80cef00 con 0x7f0de80b3930 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dee800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80d2840 con 0x7f0de80d3960 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dee800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80b6b00 con 0x7f0de815dd50 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dede00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80b3930 0x7f0de8158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dede00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80b3930 0x7f0de8158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47234/0 (socket says 172.21.15.62:47234) 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dede00640 1 -- 172.21.15.62:0/482104528 learned_addr learned my addr 172.21.15.62:0/482104528 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dede00640 1 -- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 msgr2=0x7f0de80b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dede00640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 0x7f0de80b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dede00640 1 -- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0de80d3960 msgr2=0x7f0de814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dede00640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0de80d3960 0x7f0de814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.737+0000 7f0dede00640 1 -- 172.21.15.62:0/482104528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de80b3de0 con 0x7f0de80b3930 2024-04-25T09:14:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.738+0000 7f0dede00640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80b3930 0x7f0de8158b60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009870 tx=0x7f0de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.738+0000 7f0de5600640 1 -- 172.21.15.62:0/482104528 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0037040 con 0x7f0de80b3930 2024-04-25T09:14:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.738+0000 7f0de5600640 1 -- 172.21.15.62:0/482104528 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0042070 con 0x7f0de80b3930 2024-04-25T09:14:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.738+0000 7f0de5600640 1 -- 172.21.15.62:0/482104528 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0030bd0 con 0x7f0de80b3930 2024-04-25T09:14:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.738+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de80b40b0 con 0x7f0de80b3930 2024-04-25T09:14:43.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.738+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de80b4670 con 0x7f0de80b3930 2024-04-25T09:14:43.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.740+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de80d0200 con 0x7f0de80b3930 2024-04-25T09:14:43.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.741+0000 7f0de5600640 1 -- 172.21.15.62:0/482104528 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0de002ad10 con 0x7f0de80b3930 2024-04-25T09:14:43.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.742+0000 7f0de5600640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0dc806b500 0x7f0dc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:43.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.742+0000 7f0de5600640 1 -- 172.21.15.62:0/482104528 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0de00aa890 con 0x7f0de80b3930 2024-04-25T09:14:43.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.742+0000 7f0ded400640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0dc806b500 0x7f0dc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:43.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.743+0000 7f0ded400640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0dc806b500 0x7f0dc806d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc007130 tx=0x7f0ddc007300 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:43.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.746+0000 7f0de5600640 1 -- 172.21.15.62:0/482104528 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0de80d0200 con 0x7f0de80b3930 2024-04-25T09:14:43.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.945+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0de804ef20 con 0x7f0de80b3930 2024-04-25T09:14:43.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.946+0000 7f0de5600640 1 -- 172.21.15.62:0/482104528 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0de00799d0 con 0x7f0de80b3930 2024-04-25T09:14:43.933 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:14:43.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0dc806b500 msgr2=0x7f0dc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0dc806b500 0x7f0dc806d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc007130 tx=0x7f0ddc007300 comp rx=0 tx=0).stop 2024-04-25T09:14:43.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80b3930 msgr2=0x7f0de8158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:43.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80b3930 0x7f0de8158b60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009870 tx=0x7f0de00046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 shutdown_connections 2024-04-25T09:14:43.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0dc806b500 0x7f0dc806d9c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0de80b3930 0x7f0de8158b60 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0de815dd50 0x7f0de80b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 --2- 172.21.15.62:0/482104528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0de80d3960 0x7f0de814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:43.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.949+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 >> 172.21.15.62:0/482104528 conn(0x7f0de806b000 msgr2=0x7f0de80cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:43.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.950+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 shutdown_connections 2024-04-25T09:14:43.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:43.950+0000 7f0dee800640 1 -- 172.21.15.62:0/482104528 wait complete. 2024-04-25T09:14:43.956 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:14:44.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.186+0000 7fea0fe00640 1 Processor -- start 2024-04-25T09:14:44.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.186+0000 7fea0fe00640 1 -- start start 2024-04-25T09:14:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.188+0000 7fea0fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fea100b47e0 0x7fea1006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.188+0000 7fea0fe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1006d4f0 con 0x7fea100ae9d0 2024-04-25T09:14:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.188+0000 7fea0fe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea100b2500 con 0x7fea1015b960 2024-04-25T09:14:44.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.188+0000 7fea0fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1015bd40 con 0x7fea100b47e0 2024-04-25T09:14:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.188+0000 7fea06000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fea100b47e0 0x7fea1006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.188+0000 7fea0f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fea100ae9d0 0x7fea100cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35816/0 (socket says 172.21.15.62:35816) 2024-04-25T09:14:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.188+0000 7fea06000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fea100b47e0 0x7fea1006ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47240/0 (socket says 172.21.15.62:47240) 2024-04-25T09:14:44.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.188+0000 7fea0f400640 1 -- 172.21.15.62:0/1760057879 learned_addr learned my addr 172.21.15.62:0/1760057879 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.189+0000 7fea06000640 1 -- 172.21.15.62:0/1760057879 >> v1:172.21.15.121:6789/0 conn(0x7fea1015b960 legacy=0x7fea100b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.189+0000 7fea06000640 1 -- 172.21.15.62:0/1760057879 >> v1:172.21.15.62:6789/0 conn(0x7fea100ae9d0 legacy=0x7fea100cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.189+0000 7fea0e000640 1 -- 172.21.15.62:0/1760057879 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178074755 0 0) 0x7fea1006d4f0 con 0x7fea100ae9d0 2024-04-25T09:14:44.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.189+0000 7fea06000640 1 -- 172.21.15.62:0/1760057879 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea100b27e0 con 0x7fea100b47e0 2024-04-25T09:14:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.190+0000 7fea06000640 1 --2- 172.21.15.62:0/1760057879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea100b47e0 0x7fea1006ca80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80097a0 tx=0x7fe9f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcaffbc318bd4aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.190+0000 7fea0e000640 1 -- 172.21.15.62:0/1760057879 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8037070 con 0x7fea100b47e0 2024-04-25T09:14:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.190+0000 7fea0e000640 1 -- 172.21.15.62:0/1760057879 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f8032440 con 0x7fea100b47e0 2024-04-25T09:14:44.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.190+0000 7fea0e000640 1 -- 172.21.15.62:0/1760057879 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8032810 con 0x7fea100b47e0 2024-04-25T09:14:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.191+0000 7fea0fe00640 1 -- 172.21.15.62:0/1760057879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea100b47e0 msgr2=0x7fea1006ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.191+0000 7fea0fe00640 1 --2- 172.21.15.62:0/1760057879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea100b47e0 0x7fea1006ca80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80097a0 tx=0x7fe9f802a230 comp rx=0 tx=0).stop 2024-04-25T09:14:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.191+0000 7fea0fe00640 1 -- 172.21.15.62:0/1760057879 shutdown_connections 2024-04-25T09:14:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.191+0000 7fea0fe00640 1 --2- 172.21.15.62:0/1760057879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea100b47e0 0x7fea1006ca80 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.191+0000 7fea0fe00640 1 -- 172.21.15.62:0/1760057879 >> 172.21.15.62:0/1760057879 conn(0x7fea10074380 msgr2=0x7fea100668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.191+0000 7fea0fe00640 1 -- 172.21.15.62:0/1760057879 shutdown_connections 2024-04-25T09:14:44.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.191+0000 7fea0fe00640 1 -- 172.21.15.62:0/1760057879 wait complete. 2024-04-25T09:14:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.193+0000 7fea0fe00640 1 Processor -- start 2024-04-25T09:14:44.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.193+0000 7fea0fe00640 1 -- start start 2024-04-25T09:14:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.194+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea100ae9d0 0x7fea10157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.194+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea100b47e0 0x7fea1015f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.194+0000 7fea0fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea1015b960 0x7fea1015f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.194+0000 7fea0fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea100b2da0 con 0x7fea1015b960 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.194+0000 7fea0fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea100b3110 con 0x7fea100ae9d0 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.194+0000 7fea0fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea100cdc80 con 0x7fea100b47e0 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.194+0000 7fea0f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea1015b960 0x7fea1015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea0f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea1015b960 0x7fea1015f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47246/0 (socket says 172.21.15.62:47246) 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea0f400640 1 -- 172.21.15.62:0/3075289497 learned_addr learned my addr 172.21.15.62:0/3075289497 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea06000640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea100b47e0 0x7fea1015f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea0f400640 1 -- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea100b47e0 msgr2=0x7fea1015f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea0f400640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea100b47e0 0x7fea1015f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea0f400640 1 -- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea100ae9d0 msgr2=0x7fea10157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea0f400640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea100ae9d0 0x7fea10157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea0f400640 1 -- 172.21.15.62:0/3075289497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1015a900 con 0x7fea1015b960 2024-04-25T09:14:44.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea0f400640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea1015b960 0x7fea1015f650 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c7a0 tx=0x7fea0800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:44.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea07e00640 1 -- 172.21.15.62:0/3075289497 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08018070 con 0x7fea1015b960 2024-04-25T09:14:44.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.195+0000 7fea07e00640 1 -- 172.21.15.62:0/3075289497 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0800a640 con 0x7fea1015b960 2024-04-25T09:14:44.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.196+0000 7fea07e00640 1 -- 172.21.15.62:0/3075289497 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800aab0 con 0x7fea1015b960 2024-04-25T09:14:44.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.196+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1015a060 con 0x7fea1015b960 2024-04-25T09:14:44.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.196+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1015a620 con 0x7fea1015b960 2024-04-25T09:14:44.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.197+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea10075070 con 0x7fea1015b960 2024-04-25T09:14:44.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.198+0000 7fea07e00640 1 -- 172.21.15.62:0/3075289497 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea0801b070 con 0x7fea1015b960 2024-04-25T09:14:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.199+0000 7fea07e00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.199+0000 7fea07e00640 1 -- 172.21.15.62:0/3075289497 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fea080894e0 con 0x7fea1015b960 2024-04-25T09:14:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.199+0000 7fea0ea00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.201+0000 7fea0ea00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fea00000f60 tx=0x7fea000079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:44.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.203+0000 7fea07e00640 1 -- 172.21.15.62:0/3075289497 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea080586a0 con 0x7fea1015b960 2024-04-25T09:14:44.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.404+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fea1013fb30 con 0x7fea1015b960 2024-04-25T09:14:44.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.405+0000 7fea07e00640 1 -- 172.21.15.62:0/3075289497 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v119) v1 ==== 84+0+65 (secure 0 0 0) 0x7fea08057fa0 con 0x7fea1015b960 2024-04-25T09:14:44.393 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:44.393 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:14:44.393 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:14:44.393 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:14:44.393 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:14:44.393 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:14:44.393 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:14:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.407+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe9ec06b4d0 msgr2=0x7fe9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.407+0000 7fea0fe00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fea00000f60 tx=0x7fea000079b0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.407+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea1015b960 msgr2=0x7fea1015f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.407+0000 7fea0fe00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea1015b960 0x7fea1015f650 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c7a0 tx=0x7fea0800cc60 comp rx=0 tx=0).stop 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.408+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 shutdown_connections 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.408+0000 7fea0fe00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.408+0000 7fea0fe00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea1015b960 0x7fea1015f650 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.408+0000 7fea0fe00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fea100b47e0 0x7fea1015f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.408+0000 7fea0fe00640 1 --2- 172.21.15.62:0/3075289497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea100ae9d0 0x7fea10157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.408+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 >> 172.21.15.62:0/3075289497 conn(0x7fea10074380 msgr2=0x7fea100668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.408+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 shutdown_connections 2024-04-25T09:14:44.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.408+0000 7fea0fe00640 1 -- 172.21.15.62:0/3075289497 wait complete. 2024-04-25T09:14:44.417 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:14:44.417 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:14:44.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.647+0000 7ff4ba600640 1 Processor -- start 2024-04-25T09:14:44.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.647+0000 7ff4ba600640 1 -- start start 2024-04-25T09:14:44.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.649+0000 7ff4ba600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff4b415bc00 0x7ff4b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.649+0000 7ff4ba600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b4071600 con 0x7ff4b406d8d0 2024-04-25T09:14:44.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.649+0000 7ff4ba600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40718e0 con 0x7ff4b40aede0 2024-04-25T09:14:44.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.649+0000 7ff4ba600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b406dd20 con 0x7ff4b415bc00 2024-04-25T09:14:44.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.649+0000 7ff4b3e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff4b406d8d0 0x7ff4b415cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35830/0 (socket says 172.21.15.62:35830) 2024-04-25T09:14:44.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.649+0000 7ff4b3e00640 1 -- 172.21.15.62:0/4123816131 learned_addr learned my addr 172.21.15.62:0/4123816131 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.649+0000 7ff4b2a00640 1 --2- 172.21.15.62:0/4123816131 >> v2:172.21.15.121:3300/0 conn(0x7ff4b415bc00 0x7ff4b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197229606 0 0) 0x7ff4b4071600 con 0x7ff4b406d8d0 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff498003520 con 0x7ff4b406d8d0 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2728494264 0 0) 0x7ff498003520 con 0x7ff4b406d8d0 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 >> v2:172.21.15.121:3300/0 conn(0x7ff4b415bc00 msgr2=0x7ff4b406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 --2- 172.21.15.62:0/4123816131 >> v2:172.21.15.121:3300/0 conn(0x7ff4b415bc00 0x7ff4b406d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 >> v1:172.21.15.121:6789/0 conn(0x7ff4b40aede0 legacy=0x7ff4b4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b415c170 con 0x7ff4b406d8d0 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ff4ac001670 con 0x7ff4b406d8d0 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 >> v1:172.21.15.62:6789/0 conn(0x7ff4b406d8d0 legacy=0x7ff4b415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4980039b0 0x7ff498003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498004800 0x7ff498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980051c0 0x7ff49800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b406dd20 con 0x7ff4980039b0 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49800c060 con 0x7ff4980051c0 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49800e470 con 0x7ff498004800 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.650+0000 7ff4b3400640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4980039b0 0x7ff498003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.651+0000 7ff4b3e00640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980051c0 0x7ff49800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.651+0000 7ff4b2a00640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498004800 0x7ff498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.651+0000 7ff4b3400640 1 -- 172.21.15.62:0/4123816131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498004800 msgr2=0x7ff498004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.651+0000 7ff4b3400640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff498004800 0x7ff498004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.651+0000 7ff4b3400640 1 -- 172.21.15.62:0/4123816131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980051c0 msgr2=0x7ff49800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.651+0000 7ff4b3400640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980051c0 0x7ff49800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.651+0000 7ff4b3400640 1 -- 172.21.15.62:0/4123816131 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498003520 con 0x7ff4980039b0 2024-04-25T09:14:44.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.651+0000 7ff4b3e00640 1 -- 172.21.15.62:0/4123816131 reap_dead start 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.652+0000 7ff4b3400640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4980039b0 0x7ff498003dd0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4001e50 tx=0x7ff498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405ae80f9f084b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.652+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4001a70 con 0x7ff4980039b0 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.652+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a402ae40 con 0x7ff4980039b0 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.652+0000 7ff4b2000640 1 -- 172.21.15.62:0/4123816131 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4003bb0 con 0x7ff4980039b0 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.653+0000 7ff4ba600640 1 -- 172.21.15.62:0/4123816131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4980039b0 msgr2=0x7ff498003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.653+0000 7ff4ba600640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4980039b0 0x7ff498003dd0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4001e50 tx=0x7ff498005830 comp rx=0 tx=0).stop 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.653+0000 7ff4ba600640 1 -- 172.21.15.62:0/4123816131 shutdown_connections 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.653+0000 7ff4ba600640 1 --2- 172.21.15.62:0/4123816131 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4980039b0 0x7ff498003dd0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.653+0000 7ff4ba600640 1 -- 172.21.15.62:0/4123816131 >> 172.21.15.62:0/4123816131 conn(0x7ff4b40668f0 msgr2=0x7ff4b4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:44.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.653+0000 7ff4ba600640 1 -- 172.21.15.62:0/4123816131 shutdown_connections 2024-04-25T09:14:44.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.653+0000 7ff4ba600640 1 -- 172.21.15.62:0/4123816131 wait complete. 2024-04-25T09:14:44.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.655+0000 7ff4ba600640 1 Processor -- start 2024-04-25T09:14:44.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.655+0000 7ff4ba600640 1 -- start start 2024-04-25T09:14:44.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4ba600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980039b0 0x7ff4b41548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4ba600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41451c0 0x7ff4b4150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4ba600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4b4151640 0x7ff4b4151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4ba600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40af1c0 con 0x7ff4b41451c0 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4ba600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b415c730 con 0x7ff4980039b0 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4ba600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40af340 con 0x7ff4b4151640 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41451c0 0x7ff4b4150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41451c0 0x7ff4b4150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47262/0 (socket says 172.21.15.62:47262) 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4b2a00640 1 -- 172.21.15.62:0/2775596364 learned_addr learned my addr 172.21.15.62:0/2775596364 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.656+0000 7ff4b3e00640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4b4151640 0x7ff4b4151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b3400640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980039b0 0x7ff4b41548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b2a00640 1 -- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4b4151640 msgr2=0x7ff4b4151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b2a00640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4b4151640 0x7ff4b4151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b2a00640 1 -- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980039b0 msgr2=0x7ff4b41548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:44.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b2a00640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980039b0 0x7ff4b41548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b2a00640 1 -- 172.21.15.62:0/2775596364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b414faf0 con 0x7ff4b41451c0 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b3e00640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4b4151640 0x7ff4b4151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b3400640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980039b0 0x7ff4b41548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4b2a00640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41451c0 0x7ff4b4150890 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff49c010790 tx=0x7ff49c010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4ab400640 1 -- 172.21.15.62:0/2775596364 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00f040 con 0x7ff4b41451c0 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4ab400640 1 -- 172.21.15.62:0/2775596364 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c00eb60 con 0x7ff4b41451c0 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b4154df0 con 0x7ff4b41451c0 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4ab400640 1 -- 172.21.15.62:0/2775596364 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c010ea0 con 0x7ff4b41451c0 2024-04-25T09:14:44.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.657+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b4153af0 con 0x7ff4b41451c0 2024-04-25T09:14:44.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.659+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b4067c40 con 0x7ff4b41451c0 2024-04-25T09:14:44.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.659+0000 7ff4ab400640 1 -- 172.21.15.62:0/2775596364 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff49c01c400 con 0x7ff4b41451c0 2024-04-25T09:14:44.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.661+0000 7ff4ab400640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff49006b4d0 0x7ff49006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:44.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.661+0000 7ff4ab400640 1 -- 172.21.15.62:0/2775596364 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff49c08a440 con 0x7ff4b41451c0 2024-04-25T09:14:44.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.662+0000 7ff4b3400640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff49006b4d0 0x7ff49006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:44.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.663+0000 7ff4b3400640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff49006b4d0 0x7ff49006d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4032000 tx=0x7ff4a402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:44.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.666+0000 7ff4ab400640 1 -- 172.21.15.62:0/2775596364 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4b414faf0 con 0x7ff4b41451c0 2024-04-25T09:14:44.850 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:14:44.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:44.864+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff4b4067050 con 0x7ff4b41451c0 2024-04-25T09:14:44.858 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:14:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.539+0000 7ff4ab400640 1 -- 172.21.15.62:0/2775596364 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v120) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff49c018030 con 0x7ff4b41451c0 2024-04-25T09:14:45.527 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:14:45.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.542+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff49006b4d0 msgr2=0x7ff49006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:45.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff49006b4d0 0x7ff49006d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4032000 tx=0x7ff4a402e000 comp rx=0 tx=0).stop 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41451c0 msgr2=0x7ff4b4150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41451c0 0x7ff4b4150890 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7ff49c010790 tx=0x7ff49c010c50 comp rx=0 tx=0).stop 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 shutdown_connections 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff49006b4d0 0x7ff49006d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff4b41451c0 0x7ff4b4150890 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff4b4151640 0x7ff4b4151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 --2- 172.21.15.62:0/2775596364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4980039b0 0x7ff4b41548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 >> 172.21.15.62:0/2775596364 conn(0x7ff4b40668f0 msgr2=0x7ff4b40b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 shutdown_connections 2024-04-25T09:14:45.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.543+0000 7ff4ba600640 1 -- 172.21.15.62:0/2775596364 wait complete. 2024-04-25T09:14:45.558 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:14:45.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.796+0000 7f5796200640 1 Processor -- start 2024-04-25T09:14:45.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.796+0000 7f5796200640 1 -- start start 2024-04-25T09:14:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.798+0000 7f5796200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5790159680 0x7f579015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.798+0000 7f5796200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579015d090 con 0x7f57900b1080 2024-04-25T09:14:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.798+0000 7f5796200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579006d490 con 0x7f57900cc960 2024-04-25T09:14:45.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.798+0000 7f5796200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579006d000 con 0x7f5790159680 2024-04-25T09:14:45.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.798+0000 7f5795800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f57900b1080 0x7f57901656f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60546/0 (socket says 172.21.15.62:60546) 2024-04-25T09:14:45.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.798+0000 7f5795800640 1 -- 172.21.15.62:0/3725686093 learned_addr learned my addr 172.21.15.62:0/3725686093 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:45.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618198593 0 0) 0x7f5788001490 con 0x7f57900b1080 2024-04-25T09:14:45.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f5794e00640 1 --2- 172.21.15.62:0/3725686093 >> v2:172.21.15.121:3300/0 conn(0x7f5790159680 0x7f579015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:45.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5774003520 con 0x7f57900b1080 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4050329235 0 0) 0x7f5774003520 con 0x7f57900b1080 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 >> v2:172.21.15.121:3300/0 conn(0x7f5790159680 msgr2=0x7f579015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 --2- 172.21.15.62:0/3725686093 >> v2:172.21.15.121:3300/0 conn(0x7f5790159680 0x7f579015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 >> v1:172.21.15.121:6789/0 conn(0x7f57900cc960 legacy=0x7f57900b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579006c3f0 con 0x7f57900b1080 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f5788001bc0 con 0x7f57900b1080 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.799+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 >> v1:172.21.15.62:6789/0 conn(0x7f57900b1080 legacy=0x7f57901656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f578fe00640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57740039b0 0x7f5774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:45.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f578fe00640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5774004800 0x7f5774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f578fe00640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57740051c0 0x7f577400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579006d000 con 0x7f57740051c0 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577400c110 con 0x7f57740039b0 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577400c310 con 0x7f5774004800 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5795800640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57740051c0 0x7f577400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5794e00640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57740039b0 0x7f5774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5795800640 1 -- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5774004800 msgr2=0x7f5774004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5787e00640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5774004800 0x7f5774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5795800640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5774004800 0x7f5774004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5795800640 1 -- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57740039b0 msgr2=0x7f5774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5795800640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57740039b0 0x7f5774003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5795800640 1 -- 172.21.15.62:0/3725686093 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5774003520 con 0x7f57740051c0 2024-04-25T09:14:45.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.800+0000 7f5787e00640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5774004800 0x7f5774004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.801+0000 7f5795800640 1 -- 172.21.15.62:0/3725686093 reap_dead start 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.801+0000 7f5795800640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57740051c0 0x7f577400b9c0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f578800a9f0 tx=0x7f5774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bf9f1421cea48c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.802+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800a640 con 0x7f57740051c0 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.802+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578802f070 con 0x7f57740051c0 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.802+0000 7f578fe00640 1 -- 172.21.15.62:0/3725686093 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788034470 con 0x7f57740051c0 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.802+0000 7f5796200640 1 -- 172.21.15.62:0/3725686093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57740051c0 msgr2=0x7f577400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.802+0000 7f5796200640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57740051c0 0x7f577400b9c0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f578800a9f0 tx=0x7f5774005830 comp rx=0 tx=0).stop 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.803+0000 7f5796200640 1 -- 172.21.15.62:0/3725686093 shutdown_connections 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.803+0000 7f5796200640 1 --2- 172.21.15.62:0/3725686093 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57740051c0 0x7f577400b9c0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.803+0000 7f5796200640 1 -- 172.21.15.62:0/3725686093 >> 172.21.15.62:0/3725686093 conn(0x7f57900668f0 msgr2=0x7f5790066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.803+0000 7f5796200640 1 -- 172.21.15.62:0/3725686093 shutdown_connections 2024-04-25T09:14:45.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.803+0000 7f5796200640 1 -- 172.21.15.62:0/3725686093 wait complete. 2024-04-25T09:14:45.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.805+0000 7f5796200640 1 Processor -- start 2024-04-25T09:14:45.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.805+0000 7f5796200640 1 -- start start 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5796200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57740051c0 0x7f579014f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5796200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57901536b0 0x7f579015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5796200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5790153b30 0x7f579015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5796200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579006cc90 con 0x7f57901536b0 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5796200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579006c9b0 con 0x7f5790153b30 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5796200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5790159a60 con 0x7f57740051c0 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5787e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57901536b0 0x7f579015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5787e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57901536b0 0x7f579015b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34378/0 (socket says 172.21.15.62:34378) 2024-04-25T09:14:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5787e00640 1 -- 172.21.15.62:0/927720594 learned_addr learned my addr 172.21.15.62:0/927720594 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5795800640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5790153b30 0x7f579015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.806+0000 7f5794e00640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57740051c0 0x7f579014f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5787e00640 1 -- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57740051c0 msgr2=0x7f579014f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5787e00640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57740051c0 0x7f579014f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5787e00640 1 -- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5790153b30 msgr2=0x7f579015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5787e00640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5790153b30 0x7f579015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5787e00640 1 -- 172.21.15.62:0/927720594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900b25a0 con 0x7f57901536b0 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5794e00640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57740051c0 0x7f579014f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:45.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5787e00640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57901536b0 0x7f579015b1a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f577800d540 tx=0x7f577800da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:45.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f578d600640 1 -- 172.21.15.62:0/927720594 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778005480 con 0x7f57901536b0 2024-04-25T09:14:45.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f578d600640 1 -- 172.21.15.62:0/927720594 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5778017070 con 0x7f57901536b0 2024-04-25T09:14:45.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579015e1f0 con 0x7f57901536b0 2024-04-25T09:14:45.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f578d600640 1 -- 172.21.15.62:0/927720594 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5778003850 con 0x7f57901536b0 2024-04-25T09:14:45.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.807+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579015e7b0 con 0x7f57901536b0 2024-04-25T09:14:45.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.809+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790075070 con 0x7f57901536b0 2024-04-25T09:14:45.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.810+0000 7f578d600640 1 -- 172.21.15.62:0/927720594 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57780039f0 con 0x7f57901536b0 2024-04-25T09:14:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.811+0000 7f578d600640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b500 0x7f576c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:45.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.811+0000 7f578d600640 1 -- 172.21.15.62:0/927720594 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5778089dd0 con 0x7f57901536b0 2024-04-25T09:14:45.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.812+0000 7f5794e00640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b500 0x7f576c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:45.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.813+0000 7f5794e00640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b500 0x7f576c06d9c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5780006fd0 tx=0x7f5780008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:45.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:45.816+0000 7f578d600640 1 -- 172.21.15.62:0/927720594 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5778058e20 con 0x7f57901536b0 2024-04-25T09:14:46.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.017+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f579006e020 con 0x7f57901536b0 2024-04-25T09:14:46.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.557+0000 7f578d600640 1 -- 172.21.15.62:0/927720594 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v121) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5778058720 con 0x7f57901536b0 2024-04-25T09:14:46.544 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:14:46.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b500 msgr2=0x7f576c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:46.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b500 0x7f576c06d9c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5780006fd0 tx=0x7f5780008040 comp rx=0 tx=0).stop 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57901536b0 msgr2=0x7f579015b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57901536b0 0x7f579015b1a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f577800d540 tx=0x7f577800da00 comp rx=0 tx=0).stop 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 shutdown_connections 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f576c06b500 0x7f576c06d9c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f57901536b0 0x7f579015b1a0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5790153b30 0x7f579015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 --2- 172.21.15.62:0/927720594 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f57740051c0 0x7f579014f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 >> 172.21.15.62:0/927720594 conn(0x7f57900668f0 msgr2=0x7f57900ca160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 shutdown_connections 2024-04-25T09:14:46.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.560+0000 7f5796200640 1 -- 172.21.15.62:0/927720594 wait complete. 2024-04-25T09:14:46.581 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:14:46.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.776+0000 7f91a5800640 1 Processor -- start 2024-04-25T09:14:46.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.776+0000 7f91a5800640 1 -- start start 2024-04-25T09:14:46.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.777+0000 7f91a5800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91a015dd50 0x7f91a00d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:46.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.777+0000 7f91a5800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 0x7f91a00d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:46.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.777+0000 7f91a5800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a00ce780 con 0x7f91a00d2fb0 2024-04-25T09:14:46.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.777+0000 7f91a5800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a00d42f0 con 0x7f91a015dd50 2024-04-25T09:14:46.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.777+0000 7f91a5800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00d3440 con 0x7f91a00d3960 2024-04-25T09:14:46.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.777+0000 7f919fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91a015dd50 0x7f91a00d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:46.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.777+0000 7f919fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91a015dd50 0x7f91a00d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34382/0 (socket says 172.21.15.62:34382) 2024-04-25T09:14:46.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.778+0000 7f919fe00640 1 -- 172.21.15.62:0/749912181 learned_addr learned my addr 172.21.15.62:0/749912181 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:46.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.778+0000 7f91a4e00640 1 --2- 172.21.15.62:0/749912181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 0x7f91a00d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:46.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.778+0000 7f919fe00640 1 -- 172.21.15.62:0/749912181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 msgr2=0x7f91a00d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:46.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.778+0000 7f919fe00640 1 --2- 172.21.15.62:0/749912181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 0x7f91a00d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.778+0000 7f919fe00640 1 -- 172.21.15.62:0/749912181 >> v1:172.21.15.121:6789/0 conn(0x7f91a00d2fb0 legacy=0x7f91a00ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:46.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.778+0000 7f919fe00640 1 -- 172.21.15.62:0/749912181 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00ce1c0 con 0x7f91a015dd50 2024-04-25T09:14:46.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.779+0000 7f919fe00640 1 --2- 172.21.15.62:0/749912181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a00d2300 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9190009870 tx=0x7f919002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9d2fa988d79c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:46.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.780+0000 7f919ea00640 1 -- 172.21.15.62:0/749912181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190037070 con 0x7f91a015dd50 2024-04-25T09:14:46.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.780+0000 7f919ea00640 1 -- 172.21.15.62:0/749912181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9190032440 con 0x7f91a015dd50 2024-04-25T09:14:46.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.780+0000 7f919ea00640 1 -- 172.21.15.62:0/749912181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190032810 con 0x7f91a015dd50 2024-04-25T09:14:46.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.780+0000 7f91a5800640 1 -- 172.21.15.62:0/749912181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 msgr2=0x7f91a00d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:46.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.780+0000 7f91a5800640 1 --2- 172.21.15.62:0/749912181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a00d2300 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9190009870 tx=0x7f919002a210 comp rx=0 tx=0).stop 2024-04-25T09:14:46.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.782+0000 7f91a5800640 1 -- 172.21.15.62:0/749912181 shutdown_connections 2024-04-25T09:14:46.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.782+0000 7f91a5800640 1 --2- 172.21.15.62:0/749912181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 0x7f91a00d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.782+0000 7f91a5800640 1 --2- 172.21.15.62:0/749912181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a00d2300 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.782+0000 7f91a5800640 1 -- 172.21.15.62:0/749912181 >> 172.21.15.62:0/749912181 conn(0x7f91a006b000 msgr2=0x7f91a006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:46.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.782+0000 7f91a5800640 1 -- 172.21.15.62:0/749912181 shutdown_connections 2024-04-25T09:14:46.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.782+0000 7f91a5800640 1 -- 172.21.15.62:0/749912181 wait complete. 2024-04-25T09:14:46.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.782+0000 7f91a5800640 1 Processor -- start 2024-04-25T09:14:46.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.783+0000 7f91a5800640 1 -- start start 2024-04-25T09:14:46.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.783+0000 7f91a5800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 0x7f91a0151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:46.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.783+0000 7f91a5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a015f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:46.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.783+0000 7f91a5800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a0151b00 0x7f91a00c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:46.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f91a5800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00b6b00 con 0x7f91a015dd50 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f91a5800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00d2840 con 0x7f91a0151b00 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f91a5800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00b6c80 con 0x7f91a00d3960 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a015f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a015f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34388/0 (socket says 172.21.15.62:34388) 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919f400640 1 -- 172.21.15.62:0/830817955 learned_addr learned my addr 172.21.15.62:0/830817955 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919fe00640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 0x7f91a0151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f91a4e00640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a0151b00 0x7f91a00c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919f400640 1 -- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 msgr2=0x7f91a0151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919f400640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 0x7f91a0151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919f400640 1 -- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a0151b00 msgr2=0x7f91a00c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919f400640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a0151b00 0x7f91a00c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:46.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.784+0000 7f919f400640 1 -- 172.21.15.62:0/830817955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a01517a0 con 0x7f91a015dd50 2024-04-25T09:14:46.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.785+0000 7f919f400640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a015f1f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f918800c7f0 tx=0x7f918800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:46.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.785+0000 7f9197e00640 1 -- 172.21.15.62:0/830817955 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188016020 con 0x7f91a015dd50 2024-04-25T09:14:46.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.785+0000 7f9197e00640 1 -- 172.21.15.62:0/830817955 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918800aa50 con 0x7f91a015dd50 2024-04-25T09:14:46.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.785+0000 7f9197e00640 1 -- 172.21.15.62:0/830817955 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918800ae40 con 0x7f91a015dd50 2024-04-25T09:14:46.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.785+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a0155da0 con 0x7f91a015dd50 2024-04-25T09:14:46.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.785+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a0158ab0 con 0x7f91a015dd50 2024-04-25T09:14:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.787+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a004ef20 con 0x7f91a015dd50 2024-04-25T09:14:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.788+0000 7f9197e00640 1 -- 172.21.15.62:0/830817955 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9188019020 con 0x7f91a015dd50 2024-04-25T09:14:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.788+0000 7f9197e00640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f917c06b500 0x7f917c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.789+0000 7f9197e00640 1 -- 172.21.15.62:0/830817955 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9188089eb0 con 0x7f91a015dd50 2024-04-25T09:14:46.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.789+0000 7f919fe00640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f917c06b500 0x7f917c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:46.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.791+0000 7f919fe00640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f917c06b500 0x7f917c06d9c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9190034000 tx=0x7f919002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:46.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:46.794+0000 7f9197e00640 1 -- 172.21.15.62:0/830817955 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9188020070 con 0x7f91a015dd50 2024-04-25T09:14:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.036+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f91a015f730 con 0x7f91a015dd50 2024-04-25T09:14:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.561+0000 7f9197e00640 1 -- 172.21.15.62:0/830817955 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9188058d90 con 0x7f91a015dd50 2024-04-25T09:14:47.548 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T09:14:47.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.566+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f917c06b500 msgr2=0x7f917c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.566+0000 7f91a5800640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f917c06b500 0x7f917c06d9c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9190034000 tx=0x7f919002e000 comp rx=0 tx=0).stop 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.566+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 msgr2=0x7f91a015f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.566+0000 7f91a5800640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a015f1f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f918800c7f0 tx=0x7f918800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.566+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 shutdown_connections 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.566+0000 7f91a5800640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f917c06b500 0x7f917c06d9c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.566+0000 7f91a5800640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91a015dd50 0x7f91a015f1f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.566+0000 7f91a5800640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a0151b00 0x7f91a00c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.567+0000 7f91a5800640 1 --2- 172.21.15.62:0/830817955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f91a00d3960 0x7f91a0151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.567+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 >> 172.21.15.62:0/830817955 conn(0x7f91a006b000 msgr2=0x7f91a00cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.568+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 shutdown_connections 2024-04-25T09:14:47.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.568+0000 7f91a5800640 1 -- 172.21.15.62:0/830817955 wait complete. 2024-04-25T09:14:47.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.791+0000 7f55bf200640 1 Processor -- start 2024-04-25T09:14:47.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.792+0000 7f55bf200640 1 -- start start 2024-04-25T09:14:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.793+0000 7f55bf200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55c013d970 0x7f55c00b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.793+0000 7f55bf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c00b1050 0x7f55c006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.793+0000 7f55bf200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c015b220 con 0x7f55c015acf0 2024-04-25T09:14:47.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.793+0000 7f55bf200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c015b500 con 0x7f55c013d970 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.793+0000 7f55bf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c015b7e0 con 0x7f55c00b1050 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55bde00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55c013d970 0x7f55c00b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55bde00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f55c013d970 0x7f55c00b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34400/0 (socket says 172.21.15.62:34400) 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55bde00640 1 -- 172.21.15.62:0/1804904276 learned_addr learned my addr 172.21.15.62:0/1804904276 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55bde00640 1 -- 172.21.15.62:0/1804904276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c00b1050 msgr2=0x7f55c006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55b5400640 1 --2- 172.21.15.62:0/1804904276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c00b1050 0x7f55c006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55bde00640 1 --2- 172.21.15.62:0/1804904276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c00b1050 0x7f55c006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55bde00640 1 -- 172.21.15.62:0/1804904276 >> v1:172.21.15.121:6789/0 conn(0x7f55c015acf0 legacy=0x7f55c013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55bde00640 1 -- 172.21.15.62:0/1804904276 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c015bac0 con 0x7f55c013d970 2024-04-25T09:14:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.794+0000 7f55b5400640 1 --2- 172.21.15.62:0/1804904276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c00b1050 0x7f55c006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.795+0000 7f55bde00640 1 --2- 172.21.15.62:0/1804904276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b0b10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009870 tx=0x7f55b002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48859861d39941ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bd400640 1 -- 172.21.15.62:0/1804904276 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0037070 con 0x7f55c013d970 2024-04-25T09:14:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bd400640 1 -- 172.21.15.62:0/1804904276 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b0032440 con 0x7f55c013d970 2024-04-25T09:14:47.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bd400640 1 -- 172.21.15.62:0/1804904276 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0032810 con 0x7f55c013d970 2024-04-25T09:14:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bf200640 1 -- 172.21.15.62:0/1804904276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 msgr2=0x7f55c00b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bf200640 1 --2- 172.21.15.62:0/1804904276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b0b10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009870 tx=0x7f55b002a200 comp rx=0 tx=0).stop 2024-04-25T09:14:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bf200640 1 -- 172.21.15.62:0/1804904276 shutdown_connections 2024-04-25T09:14:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bf200640 1 --2- 172.21.15.62:0/1804904276 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c00b1050 0x7f55c006d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bf200640 1 --2- 172.21.15.62:0/1804904276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b0b10 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bf200640 1 -- 172.21.15.62:0/1804904276 >> 172.21.15.62:0/1804904276 conn(0x7f55c00b14d0 msgr2=0x7f55c00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bf200640 1 -- 172.21.15.62:0/1804904276 shutdown_connections 2024-04-25T09:14:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.796+0000 7f55bf200640 1 -- 172.21.15.62:0/1804904276 wait complete. 2024-04-25T09:14:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.798+0000 7f55bf200640 1 Processor -- start 2024-04-25T09:14:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.798+0000 7f55bf200640 1 -- start start 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.799+0000 7f55bf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00b1050 0x7f55c013c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.799+0000 7f55bf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.799+0000 7f55bf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c015acf0 0x7f55c01546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.799+0000 7f55bf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c006d730 con 0x7f55c013d970 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.799+0000 7f55bf200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c013f470 con 0x7f55c00b1050 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.799+0000 7f55bf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c006d8b0 con 0x7f55c015acf0 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34406/0 (socket says 172.21.15.62:34406) 2024-04-25T09:14:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 -- 172.21.15.62:0/3048779686 learned_addr learned my addr 172.21.15.62:0/3048779686 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55bde00640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00b1050 0x7f55c013c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55be800640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c015acf0 0x7f55c01546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 -- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c015acf0 msgr2=0x7f55c01546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c015acf0 0x7f55c01546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 -- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00b1050 msgr2=0x7f55c013c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00b1050 0x7f55c013c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 -- 172.21.15.62:0/3048779686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00b3bb0 con 0x7f55c013d970 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.800+0000 7f55b5400640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b4c50 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f55a800c7f0 tx=0x7f55a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.801+0000 7f55b6a00640 1 -- 172.21.15.62:0/3048779686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a8015070 con 0x7f55c013d970 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.801+0000 7f55b6a00640 1 -- 172.21.15.62:0/3048779686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a8007820 con 0x7f55c013d970 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.801+0000 7f55b6a00640 1 -- 172.21.15.62:0/3048779686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a8007bd0 con 0x7f55c013d970 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.801+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c00b3e90 con 0x7f55c013d970 2024-04-25T09:14:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.801+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c00b4450 con 0x7f55c013d970 2024-04-25T09:14:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.802+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c004ef20 con 0x7f55c013d970 2024-04-25T09:14:47.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.803+0000 7f55b6a00640 1 -- 172.21.15.62:0/3048779686 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55a8018070 con 0x7f55c013d970 2024-04-25T09:14:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.804+0000 7f55b6a00640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f559c06b4d0 0x7f559c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.804+0000 7f55b6a00640 1 -- 172.21.15.62:0/3048779686 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55a808a410 con 0x7f55c013d970 2024-04-25T09:14:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.805+0000 7f55bde00640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f559c06b4d0 0x7f559c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.806+0000 7f55bde00640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f559c06b4d0 0x7f559c06d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f55b0034000 tx=0x7f55b002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:47.809+0000 7f55b6a00640 1 -- 172.21.15.62:0/3048779686 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55a80591f0 con 0x7f55c013d970 2024-04-25T09:14:47.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.009+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55c013c810 con 0x7f55c013d970 2024-04-25T09:14:47.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.011+0000 7f55b6a00640 1 -- 172.21.15.62:0/3048779686 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16684 (secure 0 0 0) 0x7f55a8058af0 con 0x7f55c013d970 2024-04-25T09:14:47.998 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:47.998 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":122,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:47.039020+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:14:44.925556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:14:46.076732+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:48.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f559c06b4d0 msgr2=0x7f559c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f559c06b4d0 0x7f559c06d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f55b0034000 tx=0x7f55b002e000 comp rx=0 tx=0).stop 2024-04-25T09:14:48.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 msgr2=0x7f55c00b4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b4c50 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f55a800c7f0 tx=0x7f55a800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 shutdown_connections 2024-04-25T09:14:48.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f559c06b4d0 0x7f559c06d990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c013d970 0x7f55c00b4c50 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f55c015acf0 0x7f55c01546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 --2- 172.21.15.62:0/3048779686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c00b1050 0x7f55c013c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.013+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 >> 172.21.15.62:0/3048779686 conn(0x7f55c00b14d0 msgr2=0x7f55c0159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:48.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.014+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 shutdown_connections 2024-04-25T09:14:48.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.014+0000 7f55bf200640 1 -- 172.21.15.62:0/3048779686 wait complete. 2024-04-25T09:14:48.019 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:48.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.237+0000 7f994b200640 1 Processor -- start 2024-04-25T09:14:48.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.237+0000 7f994b200640 1 -- start start 2024-04-25T09:14:48.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.238+0000 7f994b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f994c15f450 0x7f994c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.238+0000 7f994b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c15faa0 con 0x7f994c06d900 2024-04-25T09:14:48.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.238+0000 7f994b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c15fd80 con 0x7f994c144c10 2024-04-25T09:14:48.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.238+0000 7f994b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994c144ff0 con 0x7f994c15f450 2024-04-25T09:14:48.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.239+0000 7f994a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f994c15f450 0x7f994c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.239+0000 7f994a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f994c15f450 0x7f994c0cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34410/0 (socket says 172.21.15.62:34410) 2024-04-25T09:14:48.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.239+0000 7f994a800640 1 -- 172.21.15.62:0/3219110748 learned_addr learned my addr 172.21.15.62:0/3219110748 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:48.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.239+0000 7f9941400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f994c06d900 0x7f994c0b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60550/0 (socket says 172.21.15.62:60550) 2024-04-25T09:14:48.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.239+0000 7f994a800640 1 -- 172.21.15.62:0/3219110748 >> v1:172.21.15.121:6789/0 conn(0x7f994c144c10 legacy=0x7f994c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:48.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.239+0000 7f994a800640 1 -- 172.21.15.62:0/3219110748 >> v1:172.21.15.62:6789/0 conn(0x7f994c06d900 legacy=0x7f994c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.239+0000 7f994a800640 1 -- 172.21.15.62:0/3219110748 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c160060 con 0x7f994c15f450 2024-04-25T09:14:48.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.239+0000 7f9949400640 1 -- 172.21.15.62:0/3219110748 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773544459 0 0) 0x7f994c15faa0 con 0x7f994c06d900 2024-04-25T09:14:48.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.240+0000 7f994a800640 1 --2- 172.21.15.62:0/3219110748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c15f450 0x7f994c0cee40 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f9944009e50 tx=0x7f994402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607e5175883457b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:48.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.240+0000 7f9949400640 1 -- 172.21.15.62:0/3219110748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944036070 con 0x7f994c15f450 2024-04-25T09:14:48.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f9949400640 1 -- 172.21.15.62:0/3219110748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944031600 con 0x7f994c15f450 2024-04-25T09:14:48.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f9949400640 1 -- 172.21.15.62:0/3219110748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99440319d0 con 0x7f994c15f450 2024-04-25T09:14:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f994b200640 1 -- 172.21.15.62:0/3219110748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c15f450 msgr2=0x7f994c0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f994b200640 1 --2- 172.21.15.62:0/3219110748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c15f450 0x7f994c0cee40 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f9944009e50 tx=0x7f994402a6d0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f994b200640 1 -- 172.21.15.62:0/3219110748 shutdown_connections 2024-04-25T09:14:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f994b200640 1 --2- 172.21.15.62:0/3219110748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c15f450 0x7f994c0cee40 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f994b200640 1 -- 172.21.15.62:0/3219110748 >> 172.21.15.62:0/3219110748 conn(0x7f994c0b14d0 msgr2=0x7f994c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f994b200640 1 -- 172.21.15.62:0/3219110748 shutdown_connections 2024-04-25T09:14:48.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.241+0000 7f994b200640 1 -- 172.21.15.62:0/3219110748 wait complete. 2024-04-25T09:14:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.243+0000 7f994b200640 1 Processor -- start 2024-04-25T09:14:48.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.243+0000 7f994b200640 1 -- start start 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.244+0000 7f994b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c06d900 0x7f994c1310b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.244+0000 7f994b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994c144c10 0x7f994c15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.244+0000 7f994b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994c15f450 0x7f994c1e5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.244+0000 7f994b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c0b5030 con 0x7f994c06d900 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.244+0000 7f994b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c15f880 con 0x7f994c15f450 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f994b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994c0b51b0 con 0x7f994c144c10 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f9949e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c06d900 0x7f994c1310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f9949e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c06d900 0x7f994c1310b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34416/0 (socket says 172.21.15.62:34416) 2024-04-25T09:14:48.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f9949e00640 1 -- 172.21.15.62:0/3414401473 learned_addr learned my addr 172.21.15.62:0/3414401473 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:48.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f9949e00640 1 -- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994c144c10 msgr2=0x7f994c15b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f9949e00640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994c144c10 0x7f994c15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f9949e00640 1 -- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994c15f450 msgr2=0x7f994c1e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f9949e00640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994c15f450 0x7f994c1e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.245+0000 7f9949e00640 1 -- 172.21.15.62:0/3414401473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c1315f0 con 0x7f994c06d900 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.246+0000 7f9949e00640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c06d900 0x7f994c1310b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f993c00c430 tx=0x7f993c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.246+0000 7f9942a00640 1 -- 172.21.15.62:0/3414401473 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c011790 con 0x7f994c06d900 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.246+0000 7f9942a00640 1 -- 172.21.15.62:0/3414401473 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c017070 con 0x7f994c06d900 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.246+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c15b720 con 0x7f994c06d900 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.246+0000 7f9942a00640 1 -- 172.21.15.62:0/3414401473 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c011bb0 con 0x7f994c06d900 2024-04-25T09:14:48.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.246+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c1f3e10 con 0x7f994c06d900 2024-04-25T09:14:48.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.247+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c04ef20 con 0x7f994c06d900 2024-04-25T09:14:48.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.248+0000 7f9942a00640 1 -- 172.21.15.62:0/3414401473 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f993c01c460 con 0x7f994c06d900 2024-04-25T09:14:48.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.249+0000 7f9942a00640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f992806b4d0 0x7f992806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.249+0000 7f9942a00640 1 -- 172.21.15.62:0/3414401473 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f993c08a180 con 0x7f994c06d900 2024-04-25T09:14:48.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.250+0000 7f9941400640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f992806b4d0 0x7f992806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.251+0000 7f9941400640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f992806b4d0 0x7f992806d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9934007fd0 tx=0x7f9934007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:48.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.253+0000 7f9942a00640 1 -- 172.21.15.62:0/3414401473 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f994c1315f0 con 0x7f994c06d900 2024-04-25T09:14:48.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.448+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f994c0b22b0 con 0x7f994c06d900 2024-04-25T09:14:48.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.451+0000 7f9942a00640 1 -- 172.21.15.62:0/3414401473 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16684 (secure 0 0 0) 0x7f993c058f60 con 0x7f994c06d900 2024-04-25T09:14:48.438 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:48.439 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":122,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:47.039020+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:14:44.925556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:14:46.076732+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f992806b4d0 msgr2=0x7f992806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f992806b4d0 0x7f992806d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9934007fd0 tx=0x7f9934007920 comp rx=0 tx=0).stop 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c06d900 msgr2=0x7f994c1310b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c06d900 0x7f994c1310b0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f993c00c430 tx=0x7f993c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 shutdown_connections 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f992806b4d0 0x7f992806d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f994c15f450 0x7f994c1e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f994c144c10 0x7f994c15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 --2- 172.21.15.62:0/3414401473 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f994c06d900 0x7f994c1310b0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.454+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 >> 172.21.15.62:0/3414401473 conn(0x7f994c0b14d0 msgr2=0x7f994c0b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.455+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 shutdown_connections 2024-04-25T09:14:48.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.455+0000 7f994b200640 1 -- 172.21.15.62:0/3414401473 wait complete. 2024-04-25T09:14:48.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:14:48.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.684+0000 7f6f3be00640 1 Processor -- start 2024-04-25T09:14:48.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.685+0000 7f6f3be00640 1 -- start start 2024-04-25T09:14:48.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.686+0000 7f6f3be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f3c0ce990 0x7f6f3c0cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.686+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.686+0000 7f6f3be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3c0d1c90 con 0x7f6f3c15dd60 2024-04-25T09:14:48.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.686+0000 7f6f3be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3c0d1f70 con 0x7f6f3c0ce990 2024-04-25T09:14:48.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.686+0000 7f6f3be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0d2250 con 0x7f6f3c161bf0 2024-04-25T09:14:48.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.687+0000 7f6f3aa00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f6f3c15dd60 0x7f6f3c0d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35230/0 (socket says 172.21.15.62:35230) 2024-04-25T09:14:48.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.687+0000 7f6f3aa00640 1 -- 172.21.15.62:0/3850900528 learned_addr learned my addr 172.21.15.62:0/3850900528 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:48.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.687+0000 7f6f33e00640 1 --2- 172.21.15.62:0/3850900528 >> v2:172.21.15.121:3300/0 conn(0x7f6f3c0ce990 0x7f6f3c0cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.687+0000 7f6f3b400640 1 --2- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.688+0000 7f6f3a000640 1 -- 172.21.15.62:0/3850900528 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669176324 0 0) 0x7f6f3c0d1c90 con 0x7f6f3c15dd60 2024-04-25T09:14:48.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.688+0000 7f6f3a000640 1 -- 172.21.15.62:0/3850900528 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f20003520 con 0x7f6f3c15dd60 2024-04-25T09:14:48.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.688+0000 7f6f33e00640 1 -- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 msgr2=0x7f6f3c162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:48.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.688+0000 7f6f33e00640 1 --2- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c162080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.688+0000 7f6f33e00640 1 -- 172.21.15.62:0/3850900528 >> v1:172.21.15.121:6789/0 conn(0x7f6f3c15dd60 legacy=0x7f6f3c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.688+0000 7f6f33e00640 1 -- 172.21.15.62:0/3850900528 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0d2530 con 0x7f6f3c0ce990 2024-04-25T09:14:48.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.688+0000 7f6f3b400640 1 --2- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:14:48.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.689+0000 7f6f33e00640 1 --2- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c0cee20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6f240097a0 tx=0x7f6f2402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b62a4a8c4c61f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:48.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3a000640 1 -- 172.21.15.62:0/3850900528 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24037070 con 0x7f6f3c0ce990 2024-04-25T09:14:48.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3a000640 1 -- 172.21.15.62:0/3850900528 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2402ae50 con 0x7f6f3c0ce990 2024-04-25T09:14:48.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3a000640 1 -- 172.21.15.62:0/3850900528 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24032640 con 0x7f6f3c0ce990 2024-04-25T09:14:48.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3be00640 1 -- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c0ce990 msgr2=0x7f6f3c0cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3be00640 1 --2- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c0cee20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6f240097a0 tx=0x7f6f2402a100 comp rx=0 tx=0).stop 2024-04-25T09:14:48.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3be00640 1 -- 172.21.15.62:0/3850900528 shutdown_connections 2024-04-25T09:14:48.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3be00640 1 --2- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c162080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3be00640 1 --2- 172.21.15.62:0/3850900528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c0cee20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.690+0000 7f6f3be00640 1 -- 172.21.15.62:0/3850900528 >> 172.21.15.62:0/3850900528 conn(0x7f6f3c06b210 msgr2=0x7f6f3c0668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:48.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.691+0000 7f6f3be00640 1 -- 172.21.15.62:0/3850900528 shutdown_connections 2024-04-25T09:14:48.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.691+0000 7f6f3be00640 1 -- 172.21.15.62:0/3850900528 wait complete. 2024-04-25T09:14:48.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.692+0000 7f6f3be00640 1 Processor -- start 2024-04-25T09:14:48.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.692+0000 7f6f3be00640 1 -- start start 2024-04-25T09:14:48.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.693+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.693+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c15dd60 0x7f6f3c0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.693+0000 7f6f3be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.693+0000 7f6f3be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c162740 con 0x7f6f3c0ce990 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.693+0000 7f6f3be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c1625c0 con 0x7f6f3c15dd60 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.693+0000 7f6f3be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0b6dd0 con 0x7f6f3c161bf0 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.693+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c14fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34426/0 (socket says 172.21.15.62:34426) 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3aa00640 1 -- 172.21.15.62:0/1480159404 learned_addr learned my addr 172.21.15.62:0/1480159404 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3b400640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3aa00640 1 -- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 msgr2=0x7f6f3c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3aa00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c0cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3aa00640 1 -- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c15dd60 msgr2=0x7f6f3c0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f33e00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c15dd60 0x7f6f3c0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3aa00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c15dd60 0x7f6f3c0b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3aa00640 1 -- 172.21.15.62:0/1480159404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0b4cf0 con 0x7f6f3c0ce990 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f33e00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c15dd60 0x7f6f3c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:48.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.694+0000 7f6f3aa00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c14fab0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c430 tx=0x7f6f2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:48.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.695+0000 7f6f32a00640 1 -- 172.21.15.62:0/1480159404 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c017070 con 0x7f6f3c0ce990 2024-04-25T09:14:48.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.695+0000 7f6f32a00640 1 -- 172.21.15.62:0/1480159404 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c005ce0 con 0x7f6f3c0ce990 2024-04-25T09:14:48.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.695+0000 7f6f32a00640 1 -- 172.21.15.62:0/1480159404 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0072f0 con 0x7f6f3c0ce990 2024-04-25T09:14:48.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.695+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3c0b4fd0 con 0x7f6f3c0ce990 2024-04-25T09:14:48.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.695+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c0b5590 con 0x7f6f3c0ce990 2024-04-25T09:14:48.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.696+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3c0750e0 con 0x7f6f3c0ce990 2024-04-25T09:14:48.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.697+0000 7f6f32a00640 1 -- 172.21.15.62:0/1480159404 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f2c01a070 con 0x7f6f3c0ce990 2024-04-25T09:14:48.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.698+0000 7f6f32a00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f1806b4d0 0x7f6f1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:48.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.699+0000 7f6f32a00640 1 -- 172.21.15.62:0/1480159404 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f2c089750 con 0x7f6f3c0ce990 2024-04-25T09:14:48.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.699+0000 7f6f33e00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f1806b4d0 0x7f6f1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:48.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.700+0000 7f6f33e00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f1806b4d0 0x7f6f1806d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6f24034000 tx=0x7f6f2402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:48.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.703+0000 7f6f32a00640 1 -- 172.21.15.62:0/1480159404 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f2c058530 con 0x7f6f3c0ce990 2024-04-25T09:14:48.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:48.946+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6f3c0d18f0 con 0x7f6f3c0ce990 2024-04-25T09:14:49.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.653+0000 7f6f32a00640 1 -- 172.21.15.62:0/1480159404 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6f2c057e30 con 0x7f6f3c0ce990 2024-04-25T09:14:49.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f1806b4d0 msgr2=0x7f6f1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:49.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f1806b4d0 0x7f6f1806d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6f24034000 tx=0x7f6f2402e000 comp rx=0 tx=0).stop 2024-04-25T09:14:49.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c0ce990 msgr2=0x7f6f3c14fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:49.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c14fab0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c430 tx=0x7f6f2c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 shutdown_connections 2024-04-25T09:14:49.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f1806b4d0 0x7f6f1806d990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f3c161bf0 0x7f6f3c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f3c15dd60 0x7f6f3c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 --2- 172.21.15.62:0/1480159404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f3c0ce990 0x7f6f3c14fab0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 >> 172.21.15.62:0/1480159404 conn(0x7f6f3c06b210 msgr2=0x7f6f3c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:49.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.657+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 shutdown_connections 2024-04-25T09:14:49.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.658+0000 7f6f3be00640 1 -- 172.21.15.62:0/1480159404 wait complete. 2024-04-25T09:14:49.667 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:14:49.667 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:14:49.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.897+0000 7f8489400640 1 Processor -- start 2024-04-25T09:14:49.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.897+0000 7f8489400640 1 -- start start 2024-04-25T09:14:49.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f8489400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848406d9d0 0x7f848415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:49.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f8489400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f848415e5f0 0x7f84840b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:49.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f8489400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848415ee30 con 0x7f84840ceb50 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f8489400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84841618b0 con 0x7f848415e5f0 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f8489400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8484161b90 con 0x7f848406d9d0 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f8488a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f848415e5f0 0x7f84840b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f8483e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f84840ceb50 0x7f848406d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60560/0 (socket says 172.21.15.62:60560) 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f8483e00640 1 -- 172.21.15.62:0/917309504 learned_addr learned my addr 172.21.15.62:0/917309504 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.899+0000 7f847be00640 1 --2- 172.21.15.62:0/917309504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848406d9d0 0x7f848415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.900+0000 7f8488a00640 1 -- 172.21.15.62:0/917309504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848406d9d0 msgr2=0x7f848415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.900+0000 7f8488a00640 1 --2- 172.21.15.62:0/917309504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848406d9d0 0x7f848415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.900+0000 7f8488a00640 1 -- 172.21.15.62:0/917309504 >> v1:172.21.15.62:6789/0 conn(0x7f84840ceb50 legacy=0x7f848406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.900+0000 7f8488a00640 1 -- 172.21.15.62:0/917309504 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84840cef30 con 0x7f848415e5f0 2024-04-25T09:14:49.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.900+0000 7f8483400640 1 -- 172.21.15.62:0/917309504 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905614160 0 0) 0x7f848415ee30 con 0x7f84840ceb50 2024-04-25T09:14:49.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.901+0000 7f8488a00640 1 --2- 172.21.15.62:0/917309504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f848415e5f0 0x7f84840b4550 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f847c009d90 tx=0x7f847c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15813a09580dfaf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:49.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.901+0000 7f8483400640 1 -- 172.21.15.62:0/917309504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c031600 con 0x7f848415e5f0 2024-04-25T09:14:49.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.901+0000 7f8483400640 1 -- 172.21.15.62:0/917309504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c0317a0 con 0x7f848415e5f0 2024-04-25T09:14:49.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.901+0000 7f8483400640 1 -- 172.21.15.62:0/917309504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c031b70 con 0x7f848415e5f0 2024-04-25T09:14:49.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.902+0000 7f8489400640 1 -- 172.21.15.62:0/917309504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f848415e5f0 msgr2=0x7f84840b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:49.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.902+0000 7f8489400640 1 --2- 172.21.15.62:0/917309504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f848415e5f0 0x7f84840b4550 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f847c009d90 tx=0x7f847c02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.902+0000 7f8489400640 1 -- 172.21.15.62:0/917309504 shutdown_connections 2024-04-25T09:14:49.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.902+0000 7f8489400640 1 --2- 172.21.15.62:0/917309504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f848415e5f0 0x7f84840b4550 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.902+0000 7f8489400640 1 --2- 172.21.15.62:0/917309504 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848406d9d0 0x7f848415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.902+0000 7f8489400640 1 -- 172.21.15.62:0/917309504 >> 172.21.15.62:0/917309504 conn(0x7f8484066f20 msgr2=0x7f84840b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:49.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.902+0000 7f8489400640 1 -- 172.21.15.62:0/917309504 shutdown_connections 2024-04-25T09:14:49.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.902+0000 7f8489400640 1 -- 172.21.15.62:0/917309504 wait complete. 2024-04-25T09:14:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.904+0000 7f8489400640 1 Processor -- start 2024-04-25T09:14:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.904+0000 7f8489400640 1 -- start start 2024-04-25T09:14:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f8489400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f848406d9d0 0x7f84841586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f8489400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84840ceb50 0x7f8484158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f8489400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848415e5f0 0x7f84841e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:49.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f8489400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8484162510 con 0x7f84840ceb50 2024-04-25T09:14:49.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f8489400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8484162390 con 0x7f848406d9d0 2024-04-25T09:14:49.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f8489400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84840b4a90 con 0x7f848415e5f0 2024-04-25T09:14:49.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f847be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84840ceb50 0x7f8484158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:49.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f847be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84840ceb50 0x7f8484158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34440/0 (socket says 172.21.15.62:34440) 2024-04-25T09:14:49.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f847be00640 1 -- 172.21.15.62:0/119550752 learned_addr learned my addr 172.21.15.62:0/119550752 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:49.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f847be00640 1 -- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848415e5f0 msgr2=0x7f84841e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:49.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f847be00640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848415e5f0 0x7f84841e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f847be00640 1 -- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f848406d9d0 msgr2=0x7f84841586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:49.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.905+0000 7f8483e00640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f848406d9d0 0x7f84841586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f847be00640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f848406d9d0 0x7f84841586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f847be00640 1 -- 172.21.15.62:0/119550752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84841e9ea0 con 0x7f84840ceb50 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f8483e00640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f848406d9d0 0x7f84841586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f847be00640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84840ceb50 0x7f8484158c20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f846c00d830 tx=0x7f846c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f8480c00640 1 -- 172.21.15.62:0/119550752 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c00b6c0 con 0x7f84840ceb50 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f8480c00640 1 -- 172.21.15.62:0/119550752 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c019070 con 0x7f84840ceb50 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f8480c00640 1 -- 172.21.15.62:0/119550752 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c00ba80 con 0x7f84840ceb50 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84841e9bc0 con 0x7f84840ceb50 2024-04-25T09:14:49.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.906+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8484132a50 con 0x7f84840ceb50 2024-04-25T09:14:49.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.908+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848404ef20 con 0x7f84840ceb50 2024-04-25T09:14:49.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.908+0000 7f8480c00640 1 -- 172.21.15.62:0/119550752 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f846c01e460 con 0x7f84840ceb50 2024-04-25T09:14:49.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.909+0000 7f8480c00640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f846006b4d0 0x7f846006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:49.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.909+0000 7f8480c00640 1 -- 172.21.15.62:0/119550752 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f846c08a8c0 con 0x7f84840ceb50 2024-04-25T09:14:49.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.910+0000 7f8483e00640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f846006b4d0 0x7f846006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:49.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.911+0000 7f8483e00640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f846006b4d0 0x7f846006d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8474000f60 tx=0x7f8474009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:49.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:49.914+0000 7f8480c00640 1 -- 172.21.15.62:0/119550752 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f846c059730 con 0x7f84840ceb50 2024-04-25T09:14:50.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.153+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f84840b5700 con 0x7f84840ceb50 2024-04-25T09:14:50.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.657+0000 7f8480c00640 1 -- 172.21.15.62:0/119550752 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7f846c059030 con 0x7f84840ceb50 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f846006b4d0 msgr2=0x7f846006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f846006b4d0 0x7f846006d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8474000f60 tx=0x7f8474009040 comp rx=0 tx=0).stop 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84840ceb50 msgr2=0x7f8484158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84840ceb50 0x7f8484158c20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f846c00d830 tx=0x7f846c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 shutdown_connections 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f846006b4d0 0x7f846006d990 secure :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8474000f60 tx=0x7f8474009040 comp rx=0 tx=0).stop 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84840ceb50 0x7f8484158c20 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f848415e5f0 0x7f84841e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 --2- 172.21.15.62:0/119550752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f848406d9d0 0x7f84841586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.660+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 >> 172.21.15.62:0/119550752 conn(0x7f8484066f20 msgr2=0x7f8484146ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.661+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 shutdown_connections 2024-04-25T09:14:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.661+0000 7f8489400640 1 -- 172.21.15.62:0/119550752 wait complete. 2024-04-25T09:14:50.669 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:14:50.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:50.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.896+0000 7fd353200640 1 Processor -- start 2024-04-25T09:14:50.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.897+0000 7fd353200640 1 -- start start 2024-04-25T09:14:50.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.898+0000 7fd353200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd34c0d0950 0x7fd34c168960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:50.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.898+0000 7fd353200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34c0d23b0 con 0x7fd34c15dd70 2024-04-25T09:14:50.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.898+0000 7fd353200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34c0d2690 con 0x7fd34c0d1e80 2024-04-25T09:14:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.898+0000 7fd353200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c0d2970 con 0x7fd34c0d0950 2024-04-25T09:14:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.899+0000 7fd351e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd34c15dd70 0x7fd34c0cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60576/0 (socket says 172.21.15.62:60576) 2024-04-25T09:14:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.899+0000 7fd351e00640 1 -- 172.21.15.62:0/4077651594 learned_addr learned my addr 172.21.15.62:0/4077651594 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.899+0000 7fd352800640 1 --2- 172.21.15.62:0/4077651594 >> v2:172.21.15.62:3300/0 conn(0x7fd34c0d0950 0x7fd34c168960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.899+0000 7fd350a00640 1 -- 172.21.15.62:0/4077651594 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384998059 0 0) 0x7fd34c0d23b0 con 0x7fd34c15dd70 2024-04-25T09:14:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.899+0000 7fd350a00640 1 -- 172.21.15.62:0/4077651594 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd334003520 con 0x7fd34c15dd70 2024-04-25T09:14:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.899+0000 7fd352800640 1 -- 172.21.15.62:0/4077651594 >> v1:172.21.15.121:6789/0 conn(0x7fd34c0d1e80 legacy=0x7fd34c0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.899+0000 7fd352800640 1 -- 172.21.15.62:0/4077651594 >> v1:172.21.15.62:6789/0 conn(0x7fd34c15dd70 legacy=0x7fd34c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:50.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.899+0000 7fd352800640 1 -- 172.21.15.62:0/4077651594 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c0d2c50 con 0x7fd34c0d0950 2024-04-25T09:14:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.900+0000 7fd352800640 1 --2- 172.21.15.62:0/4077651594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c168960 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd348009d90 tx=0x7fd34802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a68ec4c5340d6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd350a00640 1 -- 172.21.15.62:0/4077651594 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348007740 con 0x7fd34c0d0950 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd350a00640 1 -- 172.21.15.62:0/4077651594 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3480078e0 con 0x7fd34c0d0950 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd350a00640 1 -- 172.21.15.62:0/4077651594 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348007cb0 con 0x7fd34c0d0950 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd353200640 1 -- 172.21.15.62:0/4077651594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 msgr2=0x7fd34c168960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd353200640 1 --2- 172.21.15.62:0/4077651594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c168960 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd348009d90 tx=0x7fd34802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd353200640 1 -- 172.21.15.62:0/4077651594 shutdown_connections 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd353200640 1 --2- 172.21.15.62:0/4077651594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c168960 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd353200640 1 -- 172.21.15.62:0/4077651594 >> 172.21.15.62:0/4077651594 conn(0x7fd34c06b210 msgr2=0x7fd34c0668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd353200640 1 -- 172.21.15.62:0/4077651594 shutdown_connections 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.901+0000 7fd353200640 1 -- 172.21.15.62:0/4077651594 wait complete. 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.903+0000 7fd353200640 1 Processor -- start 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.903+0000 7fd353200640 1 -- start start 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.904+0000 7fd353200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.904+0000 7fd353200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd34c0d1e80 0x7fd34c0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.904+0000 7fd353200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd34c15dd70 0x7fd34c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.904+0000 7fd353200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c0d3450 con 0x7fd34c0d0950 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.904+0000 7fd353200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c15e150 con 0x7fd34c15dd70 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.904+0000 7fd353200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c0d35d0 con 0x7fd34c0d1e80 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34450/0 (socket says 172.21.15.62:34450) 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 -- 172.21.15.62:0/2993110459 learned_addr learned my addr 172.21.15.62:0/2993110459 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:50.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 -- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd34c0d1e80 msgr2=0x7fd34c0b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd34c0d1e80 0x7fd34c0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 -- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd34c15dd70 msgr2=0x7fd34c0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd34c15dd70 0x7fd34c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 -- 172.21.15.62:0/2993110459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c0b5400 con 0x7fd34c0d0950 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.905+0000 7fd351e00640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c152f20 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd34000c410 tx=0x7fd34000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.906+0000 7fd346000640 1 -- 172.21.15.62:0/2993110459 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340017070 con 0x7fd34c0d0950 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.906+0000 7fd346000640 1 -- 172.21.15.62:0/2993110459 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd340005ce0 con 0x7fd34c0d0950 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.906+0000 7fd346000640 1 -- 172.21.15.62:0/2993110459 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3400072f0 con 0x7fd34c0d0950 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.906+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c0b56e0 con 0x7fd34c0d0950 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.906+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c15f1d0 con 0x7fd34c0d0950 2024-04-25T09:14:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.907+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34c04ef20 con 0x7fd34c0d0950 2024-04-25T09:14:50.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.908+0000 7fd346000640 1 -- 172.21.15.62:0/2993110459 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd34001a070 con 0x7fd34c0d0950 2024-04-25T09:14:50.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.909+0000 7fd346000640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd33006b500 0x7fd33006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:50.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.909+0000 7fd346000640 1 -- 172.21.15.62:0/2993110459 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd340089680 con 0x7fd34c0d0950 2024-04-25T09:14:50.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.910+0000 7fd351400640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd33006b500 0x7fd33006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:50.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.911+0000 7fd351400640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd33006b500 0x7fd33006d9c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd33c000f60 tx=0x7fd33c0079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:50.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:50.913+0000 7fd346000640 1 -- 172.21.15.62:0/2993110459 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3400585b0 con 0x7fd34c0d0950 2024-04-25T09:14:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.158+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd34c0ce2a0 con 0x7fd34c0d0950 2024-04-25T09:14:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.159+0000 7fd346000640 1 -- 172.21.15.62:0/2993110459 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd340057eb0 con 0x7fd34c0d0950 2024-04-25T09:14:51.146 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:51.146 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:14:51.147 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd33006b500 msgr2=0x7fd33006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd33006b500 0x7fd33006d9c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd33c000f60 tx=0x7fd33c0079d0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 msgr2=0x7fd34c152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c152f20 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd34000c410 tx=0x7fd34000c8d0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 shutdown_connections 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd33006b500 0x7fd33006d9c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd34c15dd70 0x7fd34c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd34c0d1e80 0x7fd34c0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 --2- 172.21.15.62:0/2993110459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd34c0d0950 0x7fd34c152f20 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 >> 172.21.15.62:0/2993110459 conn(0x7fd34c06b210 msgr2=0x7fd34c0cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.162+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 shutdown_connections 2024-04-25T09:14:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.163+0000 7fd353200640 1 -- 172.21.15.62:0/2993110459 wait complete. 2024-04-25T09:14:51.168 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:14:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.394+0000 7f1fdf200640 1 Processor -- start 2024-04-25T09:14:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.394+0000 7f1fdf200640 1 -- start start 2024-04-25T09:14:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.395+0000 7f1fdf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe013d970 0x7f1fe00b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.396+0000 7f1fdf200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1fe006d900 0x7f1fe013e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.396+0000 7f1fdf200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe015b220 con 0x7f1fe00b1050 2024-04-25T09:14:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.396+0000 7f1fdf200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe015b500 con 0x7f1fe006d900 2024-04-25T09:14:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.396+0000 7f1fdf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe015b7e0 con 0x7f1fe013d970 2024-04-25T09:14:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.396+0000 7f1fd5400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1fe00b1050 0x7f1fe006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60584/0 (socket says 172.21.15.62:60584) 2024-04-25T09:14:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.396+0000 7f1fd5400640 1 -- 172.21.15.62:0/1402719647 learned_addr learned my addr 172.21.15.62:0/1402719647 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdde00640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe013d970 0x7f1fe00b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fde800640 1 --2- 172.21.15.62:0/1402719647 >> v2:172.21.15.121:3300/0 conn(0x7f1fe006d900 0x7f1fe013e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123045871 0 0) 0x7f1fe015b220 con 0x7f1fe00b1050 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc4003520 con 0x7f1fe00b1050 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1431727068 0 0) 0x7f1fc4003520 con 0x7f1fe00b1050 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe013d970 msgr2=0x7f1fe00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe013d970 0x7f1fe00b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 >> v2:172.21.15.121:3300/0 conn(0x7f1fe006d900 msgr2=0x7f1fe013e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 --2- 172.21.15.62:0/1402719647 >> v2:172.21.15.121:3300/0 conn(0x7f1fe006d900 0x7f1fe013e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe015bac0 con 0x7f1fe00b1050 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1fd0001670 con 0x7f1fe00b1050 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 >> v1:172.21.15.62:6789/0 conn(0x7f1fe00b1050 legacy=0x7f1fe006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe013d970 0x7f1fc40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc4003f30 0x7f1fc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe015b500 con 0x7f1fc4004d40 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe015b7e0 con 0x7f1fc4003f30 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.397+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc400dbd0 con 0x7f1fe013d970 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fde800640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fdde00640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe013d970 0x7f1fc40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fd5400640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc4003f30 0x7f1fc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fde800640 1 -- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe013d970 msgr2=0x7f1fc40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fde800640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe013d970 0x7f1fc40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fde800640 1 -- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc4003f30 msgr2=0x7f1fc4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fde800640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc4003f30 0x7f1fc4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fde800640 1 -- 172.21.15.62:0/1402719647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4003520 con 0x7f1fc4004d40 2024-04-25T09:14:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.398+0000 7f1fd5400640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc4003f30 0x7f1fc4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.399+0000 7f1fde800640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fc40051c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009fc0 tx=0x7f1fc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612cfbee9bed8a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.399+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8009c10 con 0x7f1fc4004d40 2024-04-25T09:14:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.399+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8036070 con 0x7f1fc4004d40 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.400+0000 7f1fdd400640 1 -- 172.21.15.62:0/1402719647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd802d070 con 0x7f1fc4004d40 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.400+0000 7f1fdf200640 1 -- 172.21.15.62:0/1402719647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 msgr2=0x7f1fc40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.400+0000 7f1fdf200640 1 --2- 172.21.15.62:0/1402719647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fc40051c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009fc0 tx=0x7f1fc4005830 comp rx=0 tx=0).stop 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.400+0000 7f1fde800640 1 -- 172.21.15.62:0/1402719647 reap_dead start 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.400+0000 7f1fdf200640 1 -- 172.21.15.62:0/1402719647 shutdown_connections 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.400+0000 7f1fdf200640 1 -- 172.21.15.62:0/1402719647 >> 172.21.15.62:0/1402719647 conn(0x7f1fe00b14d0 msgr2=0x7f1fe00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.400+0000 7f1fdf200640 1 -- 172.21.15.62:0/1402719647 shutdown_connections 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.400+0000 7f1fdf200640 1 -- 172.21.15.62:0/1402719647 wait complete. 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.402+0000 7f1fdf200640 1 Processor -- start 2024-04-25T09:14:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.402+0000 7f1fdf200640 1 -- start start 2024-04-25T09:14:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.403+0000 7f1fdf200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fe0143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.403+0000 7f1fdf200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe0143500 0x7f1fe0141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.403+0000 7f1fdf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0153fd0 0x7f1fe012fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.403+0000 7f1fdf200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe00b0b10 con 0x7f1fc4004d40 2024-04-25T09:14:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.403+0000 7f1fdf200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe015af00 con 0x7f1fe0153fd0 2024-04-25T09:14:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.403+0000 7f1fdf200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe00b0c90 con 0x7f1fe0143500 2024-04-25T09:14:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fe0143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fe0143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34462/0 (socket says 172.21.15.62:34462) 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 -- 172.21.15.62:0/579797226 learned_addr learned my addr 172.21.15.62:0/579797226 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 -- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe0143500 msgr2=0x7f1fe0141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe0143500 0x7f1fe0141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 -- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0153fd0 msgr2=0x7f1fe012fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0153fd0 0x7f1fe012fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 -- 172.21.15.62:0/579797226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe01421e0 con 0x7f1fc4004d40 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.404+0000 7f1fdde00640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fe0143b90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8011600 tx=0x7f1fc8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.405+0000 7f1fd6a00640 1 -- 172.21.15.62:0/579797226 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8015870 con 0x7f1fc4004d40 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.405+0000 7f1fd6a00640 1 -- 172.21.15.62:0/579797226 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc8015a10 con 0x7f1fc4004d40 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.405+0000 7f1fd6a00640 1 -- 172.21.15.62:0/579797226 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8015dc0 con 0x7f1fc4004d40 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.405+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe0142460 con 0x7f1fc4004d40 2024-04-25T09:14:51.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.405+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe0142a20 con 0x7f1fc4004d40 2024-04-25T09:14:51.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.406+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe0143940 con 0x7f1fc4004d40 2024-04-25T09:14:51.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.407+0000 7f1fd6a00640 1 -- 172.21.15.62:0/579797226 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fc8013430 con 0x7f1fc4004d40 2024-04-25T09:14:51.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.408+0000 7f1fd6a00640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.408+0000 7f1fd6a00640 1 -- 172.21.15.62:0/579797226 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fc808ab00 con 0x7f1fc4004d40 2024-04-25T09:14:51.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.409+0000 7f1fd5400640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.410+0000 7f1fd5400640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0001eb0 tx=0x7f1fd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:51.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.412+0000 7f1fd6a00640 1 -- 172.21.15.62:0/579797226 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1fe0143940 con 0x7f1fc4004d40 2024-04-25T09:14:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.613+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fe004ef20 con 0x7f1fc4004d40 2024-04-25T09:14:51.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.615+0000 7f1fd6a00640 1 -- 172.21.15.62:0/579797226 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16671 (secure 0 0 0) 0x7f1fe01421e0 con 0x7f1fc4004d40 2024-04-25T09:14:51.602 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:51.602 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":123,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:14:48.577911+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:14:44.925556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:14:46.076732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:14:51.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 msgr2=0x7f1fbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0001eb0 tx=0x7f1fd0009040 comp rx=0 tx=0).stop 2024-04-25T09:14:51.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 msgr2=0x7f1fe0143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fe0143b90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8011600 tx=0x7f1fc8011ac0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 shutdown_connections 2024-04-25T09:14:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1fbc06b4d0 0x7f1fbc06d990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fe0153fd0 0x7f1fe012fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1fe0143500 0x7f1fe0141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 --2- 172.21.15.62:0/579797226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1fc4004d40 0x7f1fe0143b90 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 >> 172.21.15.62:0/579797226 conn(0x7f1fe00b14d0 msgr2=0x7f1fe015be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 shutdown_connections 2024-04-25T09:14:51.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.619+0000 7f1fdf200640 1 -- 172.21.15.62:0/579797226 wait complete. 2024-04-25T09:14:51.625 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:14:51.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:14:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.847+0000 7f7655c00640 1 Processor -- start 2024-04-25T09:14:51.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.847+0000 7f7655c00640 1 -- start start 2024-04-25T09:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f7655c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7650144c10 0x7f765015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f7655c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f76500cee40 0x7f765006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f7655c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765015faa0 con 0x7f765006d730 2024-04-25T09:14:51.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f7655c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7650144ff0 con 0x7f76500cee40 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f7655c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7650160340 con 0x7f7650144c10 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f764f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f76500cee40 0x7f765006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f7655200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f765006d730 0x7f76500b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60592/0 (socket says 172.21.15.62:60592) 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f7655200640 1 -- 172.21.15.62:0/1045309231 learned_addr learned my addr 172.21.15.62:0/1045309231 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.849+0000 7f764f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f76500cee40 0x7f765006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34476/0 (socket says 172.21.15.62:34476) 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.850+0000 7f764fe00640 1 --2- 172.21.15.62:0/1045309231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7650144c10 0x7f765015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.850+0000 7f764f400640 1 -- 172.21.15.62:0/1045309231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7650144c10 msgr2=0x7f765015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.850+0000 7f764f400640 1 --2- 172.21.15.62:0/1045309231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7650144c10 0x7f765015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.850+0000 7f764f400640 1 -- 172.21.15.62:0/1045309231 >> v1:172.21.15.62:6789/0 conn(0x7f765006d730 legacy=0x7f76500b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.850+0000 7f764f400640 1 -- 172.21.15.62:0/1045309231 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765015fd80 con 0x7f76500cee40 2024-04-25T09:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.851+0000 7f764f400640 1 --2- 172.21.15.62:0/1045309231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f765006d1f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f763c0097a0 tx=0x7f763c02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418cb0ec7135320d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:51.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.851+0000 7f764ea00640 1 -- 172.21.15.62:0/1045309231 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c037070 con 0x7f76500cee40 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.851+0000 7f764ea00640 1 -- 172.21.15.62:0/1045309231 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c02ae40 con 0x7f76500cee40 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.851+0000 7f764ea00640 1 -- 172.21.15.62:0/1045309231 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c032660 con 0x7f76500cee40 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.852+0000 7f7655c00640 1 -- 172.21.15.62:0/1045309231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 msgr2=0x7f765006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.852+0000 7f7655c00640 1 --2- 172.21.15.62:0/1045309231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f765006d1f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f763c0097a0 tx=0x7f763c02a0f0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.852+0000 7f7655c00640 1 -- 172.21.15.62:0/1045309231 shutdown_connections 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.852+0000 7f7655c00640 1 --2- 172.21.15.62:0/1045309231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f765006d1f0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.852+0000 7f7655c00640 1 --2- 172.21.15.62:0/1045309231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7650144c10 0x7f765015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.852+0000 7f7655c00640 1 -- 172.21.15.62:0/1045309231 >> 172.21.15.62:0/1045309231 conn(0x7f76500b14d0 msgr2=0x7f76500b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.852+0000 7f7655c00640 1 -- 172.21.15.62:0/1045309231 shutdown_connections 2024-04-25T09:14:51.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.852+0000 7f7655c00640 1 -- 172.21.15.62:0/1045309231 wait complete. 2024-04-25T09:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.854+0000 7f7655c00640 1 Processor -- start 2024-04-25T09:14:51.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.854+0000 7f7655c00640 1 -- start start 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f7655c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f765006d730 0x7f7650143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f7655c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f7650134c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f7655c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7650144c10 0x7f76500ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f7655c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500cf2a0 con 0x7f76500cee40 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f7655c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500b41c0 con 0x7f7650144c10 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f7655c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500b5660 con 0x7f765006d730 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f764f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f7650134c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f764f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f7650134c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34488/0 (socket says 172.21.15.62:34488) 2024-04-25T09:14:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.855+0000 7f764f400640 1 -- 172.21.15.62:0/3184180350 learned_addr learned my addr 172.21.15.62:0/3184180350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f764fe00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f765006d730 0x7f7650143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f764f400640 1 -- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f765006d730 msgr2=0x7f7650143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f764f400640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f765006d730 0x7f7650143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f7655200640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7650144c10 0x7f76500ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f764f400640 1 -- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7650144c10 msgr2=0x7f76500ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f764f400640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7650144c10 0x7f76500ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f764f400640 1 -- 172.21.15.62:0/3184180350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500cb870 con 0x7f76500cee40 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f7655200640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7650144c10 0x7f76500ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f764f400640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f7650134c10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f763c0098c0 tx=0x7f763c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f7647e00640 1 -- 172.21.15.62:0/3184180350 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c041030 con 0x7f76500cee40 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f7647e00640 1 -- 172.21.15.62:0/3184180350 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c02ac40 con 0x7f76500cee40 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76500caab0 con 0x7f76500cee40 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.856+0000 7f7647e00640 1 -- 172.21.15.62:0/3184180350 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c037040 con 0x7f76500cee40 2024-04-25T09:14:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.857+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76500cb070 con 0x7f76500cee40 2024-04-25T09:14:51.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.858+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765004ef20 con 0x7f76500cee40 2024-04-25T09:14:51.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.859+0000 7f7647e00640 1 -- 172.21.15.62:0/3184180350 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f763c044070 con 0x7f76500cee40 2024-04-25T09:14:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.860+0000 7f7647e00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:51.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.860+0000 7f7647e00640 1 -- 172.21.15.62:0/3184180350 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f763c0aa820 con 0x7f76500cee40 2024-04-25T09:14:51.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.860+0000 7f764fe00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:51.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.862+0000 7f764fe00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f763006b4d0 0x7f763006d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7640006fd0 tx=0x7f7640008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:51.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:51.864+0000 7f7647e00640 1 -- 172.21.15.62:0/3184180350 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f763c04b070 con 0x7f76500cee40 2024-04-25T09:14:51.939 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-04-25T09:14:51.953+0000 7f1fb8c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:14:52.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.060+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76500ce210 con 0x7f763006b4d0 2024-04-25T09:14:52.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.080+0000 7f7647e00640 1 -- 172.21.15.62:0/3184180350 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240298 (secure 0 0 0) 0x7f76500ce210 con 0x7f763006b4d0 2024-04-25T09:14:52.070 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:52.072 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:14:52.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.084+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f763006b4d0 msgr2=0x7f763006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.084+0000 7f7655c00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f763006b4d0 0x7f763006d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7640006fd0 tx=0x7f7640008040 comp rx=0 tx=0).stop 2024-04-25T09:14:52.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.084+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 msgr2=0x7f7650134c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.084+0000 7f7655c00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f7650134c10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f763c0098c0 tx=0x7f763c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.084+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 shutdown_connections 2024-04-25T09:14:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.085+0000 7f7655c00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.085+0000 7f7655c00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76500cee40 0x7f7650134c10 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.085+0000 7f7655c00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7650144c10 0x7f76500ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.085+0000 7f7655c00640 1 --2- 172.21.15.62:0/3184180350 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f765006d730 0x7f7650143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.085+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 >> 172.21.15.62:0/3184180350 conn(0x7f76500b14d0 msgr2=0x7f76500b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.085+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 shutdown_connections 2024-04-25T09:14:52.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.085+0000 7f7655c00640 1 -- 172.21.15.62:0/3184180350 wait complete. 2024-04-25T09:14:52.092 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-04-25T09:14:50.150478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":457916,"kb_used_data":12724,"kb_used_omap":12,"kb_used_meta":445043,"kb_avail":754516804,"statfs":{"total":773094113280,"available":772625207296,"internally_reserved":0,"allocated":13029376,"data_stored":6347769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":455724592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.607045"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.656680+0000","last_change":"2024-04-25T09:14:45.656680+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.656680+0000","last_undegraded":"2024-04-25T09:14:45.656680+0000","last_fullsized":"2024-04-25T09:14:45.656680+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.554735+0000","last_change":"2024-04-25T09:14:45.554735+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.554735+0000","last_undegraded":"2024-04-25T09:14:45.554735+0000","last_fullsized":"2024-04-25T09:14:45.554735+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.548485+0000","last_change":"2024-04-25T09:14:45.548485+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.548485+0000","last_undegraded":"2024-04-25T09:14:45.548485+0000","last_fullsized":"2024-04-25T09:14:45.548485+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.554214+0000","last_change":"2024-04-25T09:14:45.554214+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.554214+0000","last_undegraded":"2024-04-25T09:14:45.554214+0000","last_fullsized":"2024-04-25T09:14:45.554214+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.551630+0000","last_change":"2024-04-25T09:14:45.551630+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.551630+0000","last_undegraded":"2024-04-25T09:14:45.551630+0000","last_fullsized":"2024-04-25T09:14:45.551630+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.557118+0000","last_change":"2024-04-25T09:14:45.557118+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.557118+0000","last_undegraded":"2024-04-25T09:14:45.557118+0000","last_fullsized":"2024-04-25T09:14:45.557118+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.656997+0000","last_change":"2024-04-25T09:14:45.656997+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.656997+0000","last_undegraded":"2024-04-25T09:14:45.656997+0000","last_fullsized":"2024-04-25T09:14:45.656997+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.551800+0000","last_change":"2024-04-25T09:14:45.551800+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.551800+0000","last_undegraded":"2024-04-25T09:14:45.551800+0000","last_fullsized":"2024-04-25T09:14:45.551800+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.548759+0000","last_change":"2024-04-25T09:14:45.548759+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.548759+0000","last_undegraded":"2024-04-25T09:14:45.548759+0000","last_fullsized":"2024-04-25T09:14:45.548759+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.551196+0000","last_change":"2024-04-25T09:14:45.551196+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.551196+0000","last_undegraded":"2024-04-25T09:14:45.551196+0000","last_fullsized":"2024-04-25T09:14:45.551196+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.548756+0000","last_change":"2024-04-25T09:14:45.548756+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.548756+0000","last_undegraded":"2024-04-25T09:14:45.548756+0000","last_fullsized":"2024-04-25T09:14:45.548756+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.553328+0000","last_change":"2024-04-25T09:14:45.553328+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.553328+0000","last_undegraded":"2024-04-25T09:14:45.553328+0000","last_fullsized":"2024-04-25T09:14:45.553328+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.549660+0000","last_change":"2024-04-25T09:14:45.549660+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.549660+0000","last_undegraded":"2024-04-25T09:14:45.549660+0000","last_fullsized":"2024-04-25T09:14:45.549660+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.551810+0000","last_change":"2024-04-25T09:14:45.551810+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.551810+0000","last_undegraded":"2024-04-25T09:14:45.551810+0000","last_fullsized":"2024-04-25T09:14:45.551810+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.555671+0000","last_change":"2024-04-25T09:14:45.555671+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.555671+0000","last_undegraded":"2024-04-25T09:14:45.555671+0000","last_fullsized":"2024-04-25T09:14:45.555671+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.558586+0000","last_change":"2024-04-25T09:14:45.558586+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.558586+0000","last_undegraded":"2024-04-25T09:14:45.558586+0000","last_fullsized":"2024-04-25T09:14:45.558586+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.550182+0000","last_change":"2024-04-25T09:14:45.550182+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.550182+0000","last_undegraded":"2024-04-25T09:14:45.550182+0000","last_fullsized":"2024-04-25T09:14:45.550182+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.656691+0000","last_change":"2024-04-25T09:14:45.656691+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.656691+0000","last_undegraded":"2024-04-25T09:14:45.656691+0000","last_fullsized":"2024-04-25T09:14:45.656691+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.657045+0000","last_change":"2024-04-25T09:14:45.657045+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.657045+0000","last_undegraded":"2024-04-25T09:14:45.657045+0000","last_fullsized":"2024-04-25T09:14:45.657045+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.548344+0000","last_change":"2024-04-25T09:14:45.548344+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.548344+0000","last_undegraded":"2024-04-25T09:14:45.548344+0000","last_fullsized":"2024-04-25T09:14:45.548344+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.555952+0000","last_change":"2024-04-25T09:14:45.555952+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.555952+0000","last_undegraded":"2024-04-25T09:14:45.555952+0000","last_fullsized":"2024-04-25T09:14:45.555952+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.547628+0000","last_change":"2024-04-25T09:14:45.547628+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.547628+0000","last_undegraded":"2024-04-25T09:14:45.547628+0000","last_fullsized":"2024-04-25T09:14:45.547628+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.547137+0000","last_change":"2024-04-25T09:14:45.547137+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.547137+0000","last_undegraded":"2024-04-25T09:14:45.547137+0000","last_fullsized":"2024-04-25T09:14:45.547137+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.547477+0000","last_change":"2024-04-25T09:14:45.547477+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.547477+0000","last_undegraded":"2024-04-25T09:14:45.547477+0000","last_fullsized":"2024-04-25T09:14:45.547477+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.656724+0000","last_change":"2024-04-25T09:14:45.656724+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.656724+0000","last_undegraded":"2024-04-25T09:14:45.656724+0000","last_fullsized":"2024-04-25T09:14:45.656724+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.547331+0000","last_change":"2024-04-25T09:14:45.547331+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.547331+0000","last_undegraded":"2024-04-25T09:14:45.547331+0000","last_fullsized":"2024-04-25T09:14:45.547331+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.550542+0000","last_change":"2024-04-25T09:14:45.550542+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.550542+0000","last_undegraded":"2024-04-25T09:14:45.550542+0000","last_fullsized":"2024-04-25T09:14:45.550542+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.657106+0000","last_change":"2024-04-25T09:14:45.657106+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.657106+0000","last_undegraded":"2024-04-25T09:14:45.657106+0000","last_fullsized":"2024-04-25T09:14:45.657106+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.658349+0000","last_change":"2024-04-25T09:14:45.658349+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.658349+0000","last_undegraded":"2024-04-25T09:14:45.658349+0000","last_fullsized":"2024-04-25T09:14:45.658349+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.556674+0000","last_change":"2024-04-25T09:14:45.556674+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.556674+0000","last_undegraded":"2024-04-25T09:14:45.556674+0000","last_fullsized":"2024-04-25T09:14:45.556674+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.558743+0000","last_change":"2024-04-25T09:14:45.558743+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.558743+0000","last_undegraded":"2024-04-25T09:14:45.558743+0000","last_fullsized":"2024-04-25T09:14:45.558743+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:14:46.541370+0000","last_change":"2024-04-25T09:14:46.541370+0000","last_active":"2024-04-25T09:14:46.541370+0000","last_peered":"2024-04-25T09:14:46.541370+0000","last_clean":"2024-04-25T09:14:46.541370+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:46.541370+0000","last_undegraded":"2024-04-25T09:14:46.541370+0000","last_fullsized":"2024-04-25T09:14:46.541370+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":120,"state":"creating+peering","last_fresh":"2024-04-25T09:14:45.554745+0000","last_change":"2024-04-25T09:14:45.554745+0000","last_active":"2024-04-25T09:14:45.523667+0000","last_peered":"2024-04-25T09:14:45.523667+0000","last_clean":"2024-04-25T09:14:45.523667+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:14:45.554745+0000","last_undegraded":"2024-04-25T09:14:45.554745+0000","last_fullsized":"2024-04-25T09:14:45.554745+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":351,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T09:14:45.552470+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:14:45.552470+0000","last_peered":"2024-04-25T09:14:45.552470+0000","last_clean":"2024-04-25T09:14:45.552470+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:14:45.552470+0000","last_undegraded":"2024-04-25T09:14:45.552470+0000","last_fullsized":"2024-04-25T09:14:45.552470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50676,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321164,"statfs":{"total":96636764160,"available":96584871936,"internally_reserved":0,"allocated":1478656,"data_stored":645152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54844,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316996,"statfs":{"total":96636764160,"available":96580603904,"internally_reserved":0,"allocated":1490944,"data_stored":650969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55416,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316424,"statfs":{"total":96636764160,"available":96580018176,"internally_reserved":0,"allocated":2072576,"data_stored":1235520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59004,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312836,"statfs":{"total":96636764160,"available":96576344064,"internally_reserved":0,"allocated":1478656,"data_stored":645152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58992,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312848,"statfs":{"total":96636764160,"available":96576356352,"internally_reserved":0,"allocated":1478656,"data_stored":645152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77056,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294784,"statfs":{"total":96636764160,"available":96557858816,"internally_reserved":0,"allocated":1478656,"data_stored":645152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55412,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316428,"statfs":{"total":96636764160,"available":96580022272,"internally_reserved":0,"allocated":2072576,"data_stored":1235520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46516,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325324,"statfs":{"total":96636764160,"available":96589131776,"internally_reserved":0,"allocated":1478656,"data_stored":645152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:14:52.097 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:14:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.359+0000 7fa85ce00640 1 Processor -- start 2024-04-25T09:14:52.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.360+0000 7fa85ce00640 1 -- start start 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.361+0000 7fa85ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa858160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.361+0000 7fa85ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa8580b6a90 0x7fa858161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.361+0000 7fa85ce00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8580d3110 con 0x7fa8580b27a0 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.361+0000 7fa85ce00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa858161780 con 0x7fa8580b6a90 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.361+0000 7fa85ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580cedc0 con 0x7fa8580b56f0 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa84fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa8580b6a90 0x7fa858161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa857e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa8580b27a0 0x7fa858161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60608/0 (socket says 172.21.15.62:60608) 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa857e00640 1 -- 172.21.15.62:0/3895706715 learned_addr learned my addr 172.21.15.62:0/3895706715 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa857400640 1 --2- 172.21.15.62:0/3895706715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa858160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa84fe00640 1 -- 172.21.15.62:0/3895706715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 msgr2=0x7fa858160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa84fe00640 1 --2- 172.21.15.62:0/3895706715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa858160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa84fe00640 1 -- 172.21.15.62:0/3895706715 >> v1:172.21.15.62:6789/0 conn(0x7fa8580b27a0 legacy=0x7fa858161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa856a00640 1 -- 172.21.15.62:0/3895706715 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249925129 0 0) 0x7fa8580d3110 con 0x7fa8580b27a0 2024-04-25T09:14:52.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.362+0000 7fa84fe00640 1 -- 172.21.15.62:0/3895706715 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580d33f0 con 0x7fa8580b6a90 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.364+0000 7fa84fe00640 1 --2- 172.21.15.62:0/3895706715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b6a90 0x7fa858161240 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097a0 tx=0x7fa84002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e098a4a008612ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.364+0000 7fa856a00640 1 -- 172.21.15.62:0/3895706715 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840037070 con 0x7fa8580b6a90 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.364+0000 7fa856a00640 1 -- 172.21.15.62:0/3895706715 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa840032440 con 0x7fa8580b6a90 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.364+0000 7fa856a00640 1 -- 172.21.15.62:0/3895706715 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840032810 con 0x7fa8580b6a90 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.364+0000 7fa85ce00640 1 -- 172.21.15.62:0/3895706715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b6a90 msgr2=0x7fa858161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.364+0000 7fa85ce00640 1 --2- 172.21.15.62:0/3895706715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b6a90 0x7fa858161240 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097a0 tx=0x7fa84002a180 comp rx=0 tx=0).stop 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.365+0000 7fa85ce00640 1 -- 172.21.15.62:0/3895706715 shutdown_connections 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.365+0000 7fa85ce00640 1 --2- 172.21.15.62:0/3895706715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b6a90 0x7fa858161240 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.365+0000 7fa85ce00640 1 --2- 172.21.15.62:0/3895706715 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa858160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.365+0000 7fa85ce00640 1 -- 172.21.15.62:0/3895706715 >> 172.21.15.62:0/3895706715 conn(0x7fa858066d10 msgr2=0x7fa858067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:52.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.365+0000 7fa85ce00640 1 -- 172.21.15.62:0/3895706715 shutdown_connections 2024-04-25T09:14:52.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.365+0000 7fa85ce00640 1 -- 172.21.15.62:0/3895706715 wait complete. 2024-04-25T09:14:52.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.366+0000 7fa85ce00640 1 Processor -- start 2024-04-25T09:14:52.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.367+0000 7fa85ce00640 1 -- start start 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.367+0000 7fa85ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b27a0 0x7fa8580cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.367+0000 7fa85ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa8580cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.367+0000 7fa85ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580b6a90 0x7fa8580cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.367+0000 7fa85ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580b2d60 con 0x7fa8580b27a0 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.367+0000 7fa85ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580cea50 con 0x7fa8580b6a90 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.367+0000 7fa85ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580b2ee0 con 0x7fa8580b56f0 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b27a0 0x7fa8580cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b27a0 0x7fa8580cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34500/0 (socket says 172.21.15.62:34500) 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857400640 1 -- 172.21.15.62:0/1165027476 learned_addr learned my addr 172.21.15.62:0/1165027476 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857400640 1 -- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 msgr2=0x7fa8580cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa84fe00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa8580cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857e00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580b6a90 0x7fa8580cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857400640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa8580cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857400640 1 -- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580b6a90 msgr2=0x7fa8580cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857400640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580b6a90 0x7fa8580cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa857400640 1 -- 172.21.15.62:0/1165027476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85815d620 con 0x7fa8580b27a0 2024-04-25T09:14:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.368+0000 7fa84fe00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa8580cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:52.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.369+0000 7fa857400640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b27a0 0x7fa8580cadd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e9b0 tx=0x7fa84800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.369+0000 7fa84f400640 1 -- 172.21.15.62:0/1165027476 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848017070 con 0x7fa8580b27a0 2024-04-25T09:14:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.369+0000 7fa84f400640 1 -- 172.21.15.62:0/1165027476 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84800c9a0 con 0x7fa8580b27a0 2024-04-25T09:14:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.369+0000 7fa84f400640 1 -- 172.21.15.62:0/1165027476 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84800cd90 con 0x7fa8580b27a0 2024-04-25T09:14:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.369+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8580cb310 con 0x7fa8580b27a0 2024-04-25T09:14:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.369+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85815cb40 con 0x7fa8580b27a0 2024-04-25T09:14:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.370+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85804ef20 con 0x7fa8580b27a0 2024-04-25T09:14:52.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.371+0000 7fa84f400640 1 -- 172.21.15.62:0/1165027476 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa84801a070 con 0x7fa8580b27a0 2024-04-25T09:14:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.372+0000 7fa84f400640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa83406b4d0 0x7fa83406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.372+0000 7fa84f400640 1 -- 172.21.15.62:0/1165027476 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa84808a250 con 0x7fa8580b27a0 2024-04-25T09:14:52.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.373+0000 7fa84fe00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa83406b4d0 0x7fa83406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.374+0000 7fa84fe00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa83406b4d0 0x7fa83406d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa840034000 tx=0x7fa84002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:52.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.377+0000 7fa84f400640 1 -- 172.21.15.62:0/1165027476 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8480590c0 con 0x7fa8580b27a0 2024-04-25T09:14:52.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.572+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8580cb540 con 0x7fa83406b4d0 2024-04-25T09:14:52.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.591+0000 7fa84f400640 1 -- 172.21.15.62:0/1165027476 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252478 (secure 0 0 0) 0x7fa8580cb540 con 0x7fa83406b4d0 2024-04-25T09:14:52.579 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:52.583 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:14:52.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa83406b4d0 msgr2=0x7fa83406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa83406b4d0 0x7fa83406d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa840034000 tx=0x7fa84002e000 comp rx=0 tx=0).stop 2024-04-25T09:14:52.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b27a0 msgr2=0x7fa8580cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b27a0 0x7fa8580cadd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fa84800e9b0 tx=0x7fa84800ee70 comp rx=0 tx=0).stop 2024-04-25T09:14:52.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 shutdown_connections 2024-04-25T09:14:52.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa83406b4d0 0x7fa83406d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8580b6a90 0x7fa8580cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa8580b56f0 0x7fa8580cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 --2- 172.21.15.62:0/1165027476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8580b27a0 0x7fa8580cadd0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 >> 172.21.15.62:0/1165027476 conn(0x7fa858066d10 msgr2=0x7fa8580b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:52.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 shutdown_connections 2024-04-25T09:14:52.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.598+0000 7fa85ce00640 1 -- 172.21.15.62:0/1165027476 wait complete. 2024-04-25T09:14:52.604 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-04-25T09:14:52.151119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462488,"kb_used_data":13160,"kb_used_omap":12,"kb_used_meta":449203,"kb_avail":754512232,"statfs":{"total":773094113280,"available":772620525568,"internally_reserved":0,"allocated":13475840,"data_stored":6548108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":459984432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001982"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.641414+0000","last_change":"2024-04-25T09:14:47.589331+0000","last_active":"2024-04-25T09:14:49.641414+0000","last_peered":"2024-04-25T09:14:49.641414+0000","last_clean":"2024-04-25T09:14:49.641414+0000","last_became_active":"2024-04-25T09:14:47.588058+0000","last_became_peered":"2024-04-25T09:14:47.588058+0000","last_unstale":"2024-04-25T09:14:49.641414+0000","last_undegraded":"2024-04-25T09:14:49.641414+0000","last_fullsized":"2024-04-25T09:14:49.641414+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:57.420793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640800+0000","last_change":"2024-04-25T09:14:47.595638+0000","last_active":"2024-04-25T09:14:49.640800+0000","last_peered":"2024-04-25T09:14:49.640800+0000","last_clean":"2024-04-25T09:14:49.640800+0000","last_became_active":"2024-04-25T09:14:47.593611+0000","last_became_peered":"2024-04-25T09:14:47.593611+0000","last_unstale":"2024-04-25T09:14:49.640800+0000","last_undegraded":"2024-04-25T09:14:49.640800+0000","last_fullsized":"2024-04-25T09:14:49.640800+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.407959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639759+0000","last_change":"2024-04-25T09:14:47.583171+0000","last_active":"2024-04-25T09:14:49.639759+0000","last_peered":"2024-04-25T09:14:49.639759+0000","last_clean":"2024-04-25T09:14:49.639759+0000","last_became_active":"2024-04-25T09:14:47.579954+0000","last_became_peered":"2024-04-25T09:14:47.579954+0000","last_unstale":"2024-04-25T09:14:49.639759+0000","last_undegraded":"2024-04-25T09:14:49.639759+0000","last_fullsized":"2024-04-25T09:14:49.639759+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:06.116051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.597117+0000","last_change":"2024-04-25T09:14:47.602003+0000","last_active":"2024-04-25T09:14:48.597117+0000","last_peered":"2024-04-25T09:14:48.597117+0000","last_clean":"2024-04-25T09:14:48.597117+0000","last_became_active":"2024-04-25T09:14:47.600956+0000","last_became_peered":"2024-04-25T09:14:47.600956+0000","last_unstale":"2024-04-25T09:14:48.597117+0000","last_undegraded":"2024-04-25T09:14:48.597117+0000","last_fullsized":"2024-04-25T09:14:48.597117+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:05.619403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.594788+0000","last_change":"2024-04-25T09:14:47.605287+0000","last_active":"2024-04-25T09:14:48.594788+0000","last_peered":"2024-04-25T09:14:48.594788+0000","last_clean":"2024-04-25T09:14:48.594788+0000","last_became_active":"2024-04-25T09:14:47.605019+0000","last_became_peered":"2024-04-25T09:14:47.605019+0000","last_unstale":"2024-04-25T09:14:48.594788+0000","last_undegraded":"2024-04-25T09:14:48.594788+0000","last_fullsized":"2024-04-25T09:14:48.594788+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:07.551898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639257+0000","last_change":"2024-04-25T09:14:47.598366+0000","last_active":"2024-04-25T09:14:49.639257+0000","last_peered":"2024-04-25T09:14:49.639257+0000","last_clean":"2024-04-25T09:14:49.639257+0000","last_became_active":"2024-04-25T09:14:47.596201+0000","last_became_peered":"2024-04-25T09:14:47.596201+0000","last_unstale":"2024-04-25T09:14:49.639257+0000","last_undegraded":"2024-04-25T09:14:49.639257+0000","last_fullsized":"2024-04-25T09:14:49.639257+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:48.944604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656041+0000","last_change":"2024-04-25T09:14:47.584400+0000","last_active":"2024-04-25T09:14:48.656041+0000","last_peered":"2024-04-25T09:14:48.656041+0000","last_clean":"2024-04-25T09:14:48.656041+0000","last_became_active":"2024-04-25T09:14:47.584095+0000","last_became_peered":"2024-04-25T09:14:47.584095+0000","last_unstale":"2024-04-25T09:14:48.656041+0000","last_undegraded":"2024-04-25T09:14:48.656041+0000","last_fullsized":"2024-04-25T09:14:48.656041+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:13.853360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.655749+0000","last_change":"2024-04-25T09:14:47.586609+0000","last_active":"2024-04-25T09:14:48.655749+0000","last_peered":"2024-04-25T09:14:48.655749+0000","last_clean":"2024-04-25T09:14:48.655749+0000","last_became_active":"2024-04-25T09:14:47.586311+0000","last_became_peered":"2024-04-25T09:14:47.586311+0000","last_unstale":"2024-04-25T09:14:48.655749+0000","last_undegraded":"2024-04-25T09:14:48.655749+0000","last_fullsized":"2024-04-25T09:14:48.655749+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:52.448295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.641350+0000","last_change":"2024-04-25T09:14:47.600892+0000","last_active":"2024-04-25T09:14:49.641350+0000","last_peered":"2024-04-25T09:14:49.641350+0000","last_clean":"2024-04-25T09:14:49.641350+0000","last_became_active":"2024-04-25T09:14:47.600172+0000","last_became_peered":"2024-04-25T09:14:47.600172+0000","last_unstale":"2024-04-25T09:14:49.641350+0000","last_undegraded":"2024-04-25T09:14:49.641350+0000","last_fullsized":"2024-04-25T09:14:49.641350+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:13.437185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.595344+0000","last_change":"2024-04-25T09:14:47.600453+0000","last_active":"2024-04-25T09:14:48.595344+0000","last_peered":"2024-04-25T09:14:48.595344+0000","last_clean":"2024-04-25T09:14:48.595344+0000","last_became_active":"2024-04-25T09:14:47.592776+0000","last_became_peered":"2024-04-25T09:14:47.592776+0000","last_unstale":"2024-04-25T09:14:48.595344+0000","last_undegraded":"2024-04-25T09:14:48.595344+0000","last_fullsized":"2024-04-25T09:14:48.595344+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:41.787569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656845+0000","last_change":"2024-04-25T09:14:47.585046+0000","last_active":"2024-04-25T09:14:48.656845+0000","last_peered":"2024-04-25T09:14:48.656845+0000","last_clean":"2024-04-25T09:14:48.656845+0000","last_became_active":"2024-04-25T09:14:47.584719+0000","last_became_peered":"2024-04-25T09:14:47.584719+0000","last_unstale":"2024-04-25T09:14:48.656845+0000","last_undegraded":"2024-04-25T09:14:48.656845+0000","last_fullsized":"2024-04-25T09:14:48.656845+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:29.393856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.595749+0000","last_change":"2024-04-25T09:14:47.594517+0000","last_active":"2024-04-25T09:14:48.595749+0000","last_peered":"2024-04-25T09:14:48.595749+0000","last_clean":"2024-04-25T09:14:48.595749+0000","last_became_active":"2024-04-25T09:14:47.594041+0000","last_became_peered":"2024-04-25T09:14:47.594041+0000","last_unstale":"2024-04-25T09:14:48.595749+0000","last_undegraded":"2024-04-25T09:14:48.595749+0000","last_fullsized":"2024-04-25T09:14:48.595749+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:36.118192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.641018+0000","last_change":"2024-04-25T09:14:47.588879+0000","last_active":"2024-04-25T09:14:49.641018+0000","last_peered":"2024-04-25T09:14:49.641018+0000","last_clean":"2024-04-25T09:14:49.641018+0000","last_became_active":"2024-04-25T09:14:47.585358+0000","last_became_peered":"2024-04-25T09:14:47.585358+0000","last_unstale":"2024-04-25T09:14:49.641018+0000","last_undegraded":"2024-04-25T09:14:49.641018+0000","last_fullsized":"2024-04-25T09:14:49.641018+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:40.613785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633074+0000","last_change":"2024-04-25T09:14:47.590187+0000","last_active":"2024-04-25T09:14:48.633074+0000","last_peered":"2024-04-25T09:14:48.633074+0000","last_clean":"2024-04-25T09:14:48.633074+0000","last_became_active":"2024-04-25T09:14:47.589891+0000","last_became_peered":"2024-04-25T09:14:47.589891+0000","last_unstale":"2024-04-25T09:14:48.633074+0000","last_undegraded":"2024-04-25T09:14:48.633074+0000","last_fullsized":"2024-04-25T09:14:48.633074+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:29.389568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640236+0000","last_change":"2024-04-25T09:14:47.587599+0000","last_active":"2024-04-25T09:14:49.640236+0000","last_peered":"2024-04-25T09:14:49.640236+0000","last_clean":"2024-04-25T09:14:49.640236+0000","last_became_active":"2024-04-25T09:14:47.586856+0000","last_became_peered":"2024-04-25T09:14:47.586856+0000","last_unstale":"2024-04-25T09:14:49.640236+0000","last_undegraded":"2024-04-25T09:14:49.640236+0000","last_fullsized":"2024-04-25T09:14:49.640236+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:48.944604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.594912+0000","last_change":"2024-04-25T09:14:47.597880+0000","last_active":"2024-04-25T09:14:48.594912+0000","last_peered":"2024-04-25T09:14:48.594912+0000","last_clean":"2024-04-25T09:14:48.594912+0000","last_became_active":"2024-04-25T09:14:47.596749+0000","last_became_peered":"2024-04-25T09:14:47.596749+0000","last_unstale":"2024-04-25T09:14:48.594912+0000","last_undegraded":"2024-04-25T09:14:48.594912+0000","last_fullsized":"2024-04-25T09:14:48.594912+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:57.757591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640232+0000","last_change":"2024-04-25T09:14:47.586047+0000","last_active":"2024-04-25T09:14:49.640232+0000","last_peered":"2024-04-25T09:14:49.640232+0000","last_clean":"2024-04-25T09:14:49.640232+0000","last_became_active":"2024-04-25T09:14:47.585741+0000","last_became_peered":"2024-04-25T09:14:47.585741+0000","last_unstale":"2024-04-25T09:14:49.640232+0000","last_undegraded":"2024-04-25T09:14:49.640232+0000","last_fullsized":"2024-04-25T09:14:49.640232+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:29.244214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.657201+0000","last_change":"2024-04-25T09:14:47.590644+0000","last_active":"2024-04-25T09:14:48.657201+0000","last_peered":"2024-04-25T09:14:48.657201+0000","last_clean":"2024-04-25T09:14:48.657201+0000","last_became_active":"2024-04-25T09:14:47.590313+0000","last_became_peered":"2024-04-25T09:14:47.590313+0000","last_unstale":"2024-04-25T09:14:48.657201+0000","last_undegraded":"2024-04-25T09:14:48.657201+0000","last_fullsized":"2024-04-25T09:14:48.657201+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:01.613663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640233+0000","last_change":"2024-04-25T09:14:47.586554+0000","last_active":"2024-04-25T09:14:49.640233+0000","last_peered":"2024-04-25T09:14:49.640233+0000","last_clean":"2024-04-25T09:14:49.640233+0000","last_became_active":"2024-04-25T09:14:47.585428+0000","last_became_peered":"2024-04-25T09:14:47.585428+0000","last_unstale":"2024-04-25T09:14:49.640233+0000","last_undegraded":"2024-04-25T09:14:49.640233+0000","last_fullsized":"2024-04-25T09:14:49.640233+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:57.757591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639301+0000","last_change":"2024-04-25T09:14:47.599152+0000","last_active":"2024-04-25T09:14:49.639301+0000","last_peered":"2024-04-25T09:14:49.639301+0000","last_clean":"2024-04-25T09:14:49.639301+0000","last_became_active":"2024-04-25T09:14:47.596750+0000","last_became_peered":"2024-04-25T09:14:47.596750+0000","last_unstale":"2024-04-25T09:14:49.639301+0000","last_undegraded":"2024-04-25T09:14:49.639301+0000","last_fullsized":"2024-04-25T09:14:49.639301+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:39.271771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656467+0000","last_change":"2024-04-25T09:14:47.597580+0000","last_active":"2024-04-25T09:14:48.656467+0000","last_peered":"2024-04-25T09:14:48.656467+0000","last_clean":"2024-04-25T09:14:48.656467+0000","last_became_active":"2024-04-25T09:14:47.597279+0000","last_became_peered":"2024-04-25T09:14:47.597279+0000","last_unstale":"2024-04-25T09:14:48.656467+0000","last_undegraded":"2024-04-25T09:14:48.656467+0000","last_fullsized":"2024-04-25T09:14:48.656467+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:42.990057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639439+0000","last_change":"2024-04-25T09:14:47.602180+0000","last_active":"2024-04-25T09:14:49.639439+0000","last_peered":"2024-04-25T09:14:49.639439+0000","last_clean":"2024-04-25T09:14:49.639439+0000","last_became_active":"2024-04-25T09:14:47.595752+0000","last_became_peered":"2024-04-25T09:14:47.595752+0000","last_unstale":"2024-04-25T09:14:49.639439+0000","last_undegraded":"2024-04-25T09:14:49.639439+0000","last_fullsized":"2024-04-25T09:14:49.639439+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:36.410285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.632885+0000","last_change":"2024-04-25T09:14:47.578598+0000","last_active":"2024-04-25T09:14:48.632885+0000","last_peered":"2024-04-25T09:14:48.632885+0000","last_clean":"2024-04-25T09:14:48.632885+0000","last_became_active":"2024-04-25T09:14:47.578260+0000","last_became_peered":"2024-04-25T09:14:47.578260+0000","last_unstale":"2024-04-25T09:14:48.632885+0000","last_undegraded":"2024-04-25T09:14:48.632885+0000","last_fullsized":"2024-04-25T09:14:48.632885+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:27.255603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.596263+0000","last_change":"2024-04-25T09:14:47.605890+0000","last_active":"2024-04-25T09:14:48.596263+0000","last_peered":"2024-04-25T09:14:48.596263+0000","last_clean":"2024-04-25T09:14:48.596263+0000","last_became_active":"2024-04-25T09:14:47.603831+0000","last_became_peered":"2024-04-25T09:14:47.603831+0000","last_unstale":"2024-04-25T09:14:48.596263+0000","last_undegraded":"2024-04-25T09:14:48.596263+0000","last_fullsized":"2024-04-25T09:14:48.596263+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:59.143296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.657113+0000","last_change":"2024-04-25T09:14:47.592539+0000","last_active":"2024-04-25T09:14:48.657113+0000","last_peered":"2024-04-25T09:14:48.657113+0000","last_clean":"2024-04-25T09:14:48.657113+0000","last_became_active":"2024-04-25T09:14:47.591403+0000","last_became_peered":"2024-04-25T09:14:47.591403+0000","last_unstale":"2024-04-25T09:14:48.657113+0000","last_undegraded":"2024-04-25T09:14:48.657113+0000","last_fullsized":"2024-04-25T09:14:48.657113+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:39.183036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640644+0000","last_change":"2024-04-25T09:14:47.594218+0000","last_active":"2024-04-25T09:14:49.640644+0000","last_peered":"2024-04-25T09:14:49.640644+0000","last_clean":"2024-04-25T09:14:49.640644+0000","last_became_active":"2024-04-25T09:14:47.592502+0000","last_became_peered":"2024-04-25T09:14:47.592502+0000","last_unstale":"2024-04-25T09:14:49.640644+0000","last_undegraded":"2024-04-25T09:14:49.640644+0000","last_fullsized":"2024-04-25T09:14:49.640644+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:15.414150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.596737+0000","last_change":"2024-04-25T09:14:47.604031+0000","last_active":"2024-04-25T09:14:48.596737+0000","last_peered":"2024-04-25T09:14:48.596737+0000","last_clean":"2024-04-25T09:14:48.596737+0000","last_became_active":"2024-04-25T09:14:47.603732+0000","last_became_peered":"2024-04-25T09:14:47.603732+0000","last_unstale":"2024-04-25T09:14:48.596737+0000","last_undegraded":"2024-04-25T09:14:48.596737+0000","last_fullsized":"2024-04-25T09:14:48.596737+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:13.511379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640012+0000","last_change":"2024-04-25T09:14:47.595940+0000","last_active":"2024-04-25T09:14:49.640012+0000","last_peered":"2024-04-25T09:14:49.640012+0000","last_clean":"2024-04-25T09:14:49.640012+0000","last_became_active":"2024-04-25T09:14:47.595568+0000","last_became_peered":"2024-04-25T09:14:47.595568+0000","last_unstale":"2024-04-25T09:14:49.640012+0000","last_undegraded":"2024-04-25T09:14:49.640012+0000","last_fullsized":"2024-04-25T09:14:49.640012+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.052942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633404+0000","last_change":"2024-04-25T09:14:47.586226+0000","last_active":"2024-04-25T09:14:48.633404+0000","last_peered":"2024-04-25T09:14:48.633404+0000","last_clean":"2024-04-25T09:14:48.633404+0000","last_became_active":"2024-04-25T09:14:47.585936+0000","last_became_peered":"2024-04-25T09:14:47.585936+0000","last_unstale":"2024-04-25T09:14:48.633404+0000","last_undegraded":"2024-04-25T09:14:48.633404+0000","last_fullsized":"2024-04-25T09:14:48.633404+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:50.415513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656359+0000","last_change":"2024-04-25T09:14:47.592101+0000","last_active":"2024-04-25T09:14:48.656359+0000","last_peered":"2024-04-25T09:14:48.656359+0000","last_clean":"2024-04-25T09:14:48.656359+0000","last_became_active":"2024-04-25T09:14:47.591823+0000","last_became_peered":"2024-04-25T09:14:47.591823+0000","last_unstale":"2024-04-25T09:14:48.656359+0000","last_undegraded":"2024-04-25T09:14:48.656359+0000","last_fullsized":"2024-04-25T09:14:48.656359+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:17.879044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639407+0000","last_change":"2024-04-25T09:14:47.578967+0000","last_active":"2024-04-25T09:14:49.639407+0000","last_peered":"2024-04-25T09:14:49.639407+0000","last_clean":"2024-04-25T09:14:49.639407+0000","last_became_active":"2024-04-25T09:14:47.577443+0000","last_became_peered":"2024-04-25T09:14:47.577443+0000","last_unstale":"2024-04-25T09:14:49.639407+0000","last_undegraded":"2024-04-25T09:14:49.639407+0000","last_fullsized":"2024-04-25T09:14:49.639407+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.390883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.595300+0000","last_change":"2024-04-25T09:14:47.584065+0000","last_active":"2024-04-25T09:14:48.595300+0000","last_peered":"2024-04-25T09:14:48.595300+0000","last_clean":"2024-04-25T09:14:48.595300+0000","last_became_active":"2024-04-25T09:14:47.583648+0000","last_became_peered":"2024-04-25T09:14:47.583648+0000","last_unstale":"2024-04-25T09:14:48.595300+0000","last_undegraded":"2024-04-25T09:14:48.595300+0000","last_fullsized":"2024-04-25T09:14:48.595300+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:45.902598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.641648+0000","last_change":"2024-04-25T09:14:47.600860+0000","last_active":"2024-04-25T09:14:49.641648+0000","last_peered":"2024-04-25T09:14:49.641648+0000","last_clean":"2024-04-25T09:14:49.641648+0000","last_became_active":"2024-04-25T09:14:47.600553+0000","last_became_peered":"2024-04-25T09:14:47.600553+0000","last_unstale":"2024-04-25T09:14:49.641648+0000","last_undegraded":"2024-04-25T09:14:49.641648+0000","last_fullsized":"2024-04-25T09:14:49.641648+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:08.683384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"122'4","reported_seq":18,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633979+0000","last_change":"2024-04-25T09:14:46.592475+0000","last_active":"2024-04-25T09:14:48.633979+0000","last_peered":"2024-04-25T09:14:48.633979+0000","last_clean":"2024-04-25T09:14:48.633979+0000","last_became_active":"2024-04-25T09:14:46.591364+0000","last_became_peered":"2024-04-25T09:14:46.591364+0000","last_unstale":"2024-04-25T09:14:48.633979+0000","last_undegraded":"2024-04-25T09:14:48.633979+0000","last_fullsized":"2024-04-25T09:14:48.633979+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:40.769866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633920+0000","last_change":"2024-04-25T09:14:47.589748+0000","last_active":"2024-04-25T09:14:48.633920+0000","last_peered":"2024-04-25T09:14:48.633920+0000","last_clean":"2024-04-25T09:14:48.633920+0000","last_became_active":"2024-04-25T09:14:47.587239+0000","last_became_peered":"2024-04-25T09:14:47.587239+0000","last_unstale":"2024-04-25T09:14:48.633920+0000","last_undegraded":"2024-04-25T09:14:48.633920+0000","last_fullsized":"2024-04-25T09:14:48.633920+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:59.143296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.641024+0000","last_change":"2024-04-25T09:14:46.583783+0000","last_active":"2024-04-25T09:14:49.641024+0000","last_peered":"2024-04-25T09:14:49.641024+0000","last_clean":"2024-04-25T09:14:49.641024+0000","last_became_active":"2024-04-25T09:14:46.583467+0000","last_became_peered":"2024-04-25T09:14:46.583467+0000","last_unstale":"2024-04-25T09:14:49.641024+0000","last_undegraded":"2024-04-25T09:14:49.641024+0000","last_fullsized":"2024-04-25T09:14:49.641024+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:28.371865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640425+0000","last_change":"2024-04-25T09:14:47.601682+0000","last_active":"2024-04-25T09:14:49.640425+0000","last_peered":"2024-04-25T09:14:49.640425+0000","last_clean":"2024-04-25T09:14:49.640425+0000","last_became_active":"2024-04-25T09:14:47.599822+0000","last_became_peered":"2024-04-25T09:14:47.599822+0000","last_unstale":"2024-04-25T09:14:49.640425+0000","last_undegraded":"2024-04-25T09:14:49.640425+0000","last_fullsized":"2024-04-25T09:14:49.640425+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:57.420793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.596530+0000","last_change":"2024-04-25T09:14:46.591349+0000","last_active":"2024-04-25T09:14:48.596530+0000","last_peered":"2024-04-25T09:14:48.596530+0000","last_clean":"2024-04-25T09:14:48.596530+0000","last_became_active":"2024-04-25T09:14:46.587225+0000","last_became_peered":"2024-04-25T09:14:46.587225+0000","last_unstale":"2024-04-25T09:14:48.596530+0000","last_undegraded":"2024-04-25T09:14:48.596530+0000","last_fullsized":"2024-04-25T09:14:48.596530+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:27.657283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640420+0000","last_change":"2024-04-25T09:14:47.594412+0000","last_active":"2024-04-25T09:14:49.640420+0000","last_peered":"2024-04-25T09:14:49.640420+0000","last_clean":"2024-04-25T09:14:49.640420+0000","last_became_active":"2024-04-25T09:14:47.594101+0000","last_became_peered":"2024-04-25T09:14:47.594101+0000","last_unstale":"2024-04-25T09:14:49.640420+0000","last_undegraded":"2024-04-25T09:14:49.640420+0000","last_fullsized":"2024-04-25T09:14:49.640420+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:38.203449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.597263+0000","last_change":"2024-04-25T09:14:46.591099+0000","last_active":"2024-04-25T09:14:48.597263+0000","last_peered":"2024-04-25T09:14:48.597263+0000","last_clean":"2024-04-25T09:14:48.597263+0000","last_became_active":"2024-04-25T09:14:46.590733+0000","last_became_peered":"2024-04-25T09:14:46.590733+0000","last_unstale":"2024-04-25T09:14:48.597263+0000","last_undegraded":"2024-04-25T09:14:48.597263+0000","last_fullsized":"2024-04-25T09:14:48.597263+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:40.769866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.597744+0000","last_change":"2024-04-25T09:14:47.609840+0000","last_active":"2024-04-25T09:14:48.597744+0000","last_peered":"2024-04-25T09:14:48.597744+0000","last_clean":"2024-04-25T09:14:48.597744+0000","last_became_active":"2024-04-25T09:14:47.609429+0000","last_became_peered":"2024-04-25T09:14:47.609429+0000","last_unstale":"2024-04-25T09:14:48.597744+0000","last_undegraded":"2024-04-25T09:14:48.597744+0000","last_fullsized":"2024-04-25T09:14:48.597744+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:13.395877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.638941+0000","last_change":"2024-04-25T09:14:46.584321+0000","last_active":"2024-04-25T09:14:49.638941+0000","last_peered":"2024-04-25T09:14:49.638941+0000","last_clean":"2024-04-25T09:14:49.638941+0000","last_became_active":"2024-04-25T09:14:46.583994+0000","last_became_peered":"2024-04-25T09:14:46.583994+0000","last_unstale":"2024-04-25T09:14:49.638941+0000","last_undegraded":"2024-04-25T09:14:49.638941+0000","last_fullsized":"2024-04-25T09:14:49.638941+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:50.893497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656715+0000","last_change":"2024-04-25T09:14:47.588434+0000","last_active":"2024-04-25T09:14:48.656715+0000","last_peered":"2024-04-25T09:14:48.656715+0000","last_clean":"2024-04-25T09:14:48.656715+0000","last_became_active":"2024-04-25T09:14:47.588159+0000","last_became_peered":"2024-04-25T09:14:47.588159+0000","last_unstale":"2024-04-25T09:14:48.656715+0000","last_undegraded":"2024-04-25T09:14:48.656715+0000","last_fullsized":"2024-04-25T09:14:48.656715+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:25.842328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639873+0000","last_change":"2024-04-25T09:14:46.590802+0000","last_active":"2024-04-25T09:14:49.639873+0000","last_peered":"2024-04-25T09:14:49.639873+0000","last_clean":"2024-04-25T09:14:49.639873+0000","last_became_active":"2024-04-25T09:14:46.590494+0000","last_became_peered":"2024-04-25T09:14:46.590494+0000","last_unstale":"2024-04-25T09:14:49.639873+0000","last_undegraded":"2024-04-25T09:14:49.639873+0000","last_fullsized":"2024-04-25T09:14:49.639873+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:31.607730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.595341+0000","last_change":"2024-04-25T09:14:47.604600+0000","last_active":"2024-04-25T09:14:48.595341+0000","last_peered":"2024-04-25T09:14:48.595341+0000","last_clean":"2024-04-25T09:14:48.595341+0000","last_became_active":"2024-04-25T09:14:47.603310+0000","last_became_peered":"2024-04-25T09:14:47.603310+0000","last_unstale":"2024-04-25T09:14:48.595341+0000","last_undegraded":"2024-04-25T09:14:48.595341+0000","last_fullsized":"2024-04-25T09:14:48.595341+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:02.160840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656178+0000","last_change":"2024-04-25T09:14:46.581372+0000","last_active":"2024-04-25T09:14:48.656178+0000","last_peered":"2024-04-25T09:14:48.656178+0000","last_clean":"2024-04-25T09:14:48.656178+0000","last_became_active":"2024-04-25T09:14:46.581076+0000","last_became_peered":"2024-04-25T09:14:46.581076+0000","last_unstale":"2024-04-25T09:14:48.656178+0000","last_undegraded":"2024-04-25T09:14:48.656178+0000","last_fullsized":"2024-04-25T09:14:48.656178+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:50.667498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656171+0000","last_change":"2024-04-25T09:14:47.586219+0000","last_active":"2024-04-25T09:14:48.656171+0000","last_peered":"2024-04-25T09:14:48.656171+0000","last_clean":"2024-04-25T09:14:48.656171+0000","last_became_active":"2024-04-25T09:14:47.585918+0000","last_became_peered":"2024-04-25T09:14:47.585918+0000","last_unstale":"2024-04-25T09:14:48.656171+0000","last_undegraded":"2024-04-25T09:14:48.656171+0000","last_fullsized":"2024-04-25T09:14:48.656171+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:32.241884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.595426+0000","last_change":"2024-04-25T09:14:46.585543+0000","last_active":"2024-04-25T09:14:48.595426+0000","last_peered":"2024-04-25T09:14:48.595426+0000","last_clean":"2024-04-25T09:14:48.595426+0000","last_became_active":"2024-04-25T09:14:46.585248+0000","last_became_peered":"2024-04-25T09:14:46.585248+0000","last_unstale":"2024-04-25T09:14:48.595426+0000","last_undegraded":"2024-04-25T09:14:48.595426+0000","last_fullsized":"2024-04-25T09:14:48.595426+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:50.893497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640956+0000","last_change":"2024-04-25T09:14:47.582386+0000","last_active":"2024-04-25T09:14:49.640956+0000","last_peered":"2024-04-25T09:14:49.640956+0000","last_clean":"2024-04-25T09:14:49.640956+0000","last_became_active":"2024-04-25T09:14:47.582099+0000","last_became_peered":"2024-04-25T09:14:47.582099+0000","last_unstale":"2024-04-25T09:14:49.640956+0000","last_undegraded":"2024-04-25T09:14:49.640956+0000","last_fullsized":"2024-04-25T09:14:49.640956+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:59.143296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.597389+0000","last_change":"2024-04-25T09:14:46.589636+0000","last_active":"2024-04-25T09:14:48.597389+0000","last_peered":"2024-04-25T09:14:48.597389+0000","last_clean":"2024-04-25T09:14:48.597389+0000","last_became_active":"2024-04-25T09:14:46.587837+0000","last_became_peered":"2024-04-25T09:14:46.587837+0000","last_unstale":"2024-04-25T09:14:48.597389+0000","last_undegraded":"2024-04-25T09:14:48.597389+0000","last_fullsized":"2024-04-25T09:14:48.597389+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:23.179698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.641683+0000","last_change":"2024-04-25T09:14:47.586168+0000","last_active":"2024-04-25T09:14:49.641683+0000","last_peered":"2024-04-25T09:14:49.641683+0000","last_clean":"2024-04-25T09:14:49.641683+0000","last_became_active":"2024-04-25T09:14:47.585895+0000","last_became_peered":"2024-04-25T09:14:47.585895+0000","last_unstale":"2024-04-25T09:14:49.641683+0000","last_undegraded":"2024-04-25T09:14:49.641683+0000","last_fullsized":"2024-04-25T09:14:49.641683+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:42.925014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.595289+0000","last_change":"2024-04-25T09:14:46.584401+0000","last_active":"2024-04-25T09:14:48.595289+0000","last_peered":"2024-04-25T09:14:48.595289+0000","last_clean":"2024-04-25T09:14:48.595289+0000","last_became_active":"2024-04-25T09:14:46.584104+0000","last_became_peered":"2024-04-25T09:14:46.584104+0000","last_unstale":"2024-04-25T09:14:48.595289+0000","last_undegraded":"2024-04-25T09:14:48.595289+0000","last_fullsized":"2024-04-25T09:14:48.595289+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.765091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640893+0000","last_change":"2024-04-25T09:14:47.591677+0000","last_active":"2024-04-25T09:14:49.640893+0000","last_peered":"2024-04-25T09:14:49.640893+0000","last_clean":"2024-04-25T09:14:49.640893+0000","last_became_active":"2024-04-25T09:14:47.591382+0000","last_became_peered":"2024-04-25T09:14:47.591382+0000","last_unstale":"2024-04-25T09:14:49.640893+0000","last_undegraded":"2024-04-25T09:14:49.640893+0000","last_fullsized":"2024-04-25T09:14:49.640893+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:08.683384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"122'3","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.594861+0000","last_change":"2024-04-25T09:14:46.594407+0000","last_active":"2024-04-25T09:14:48.594861+0000","last_peered":"2024-04-25T09:14:48.594861+0000","last_clean":"2024-04-25T09:14:48.594861+0000","last_became_active":"2024-04-25T09:14:46.593915+0000","last_became_peered":"2024-04-25T09:14:46.593915+0000","last_unstale":"2024-04-25T09:14:48.594861+0000","last_undegraded":"2024-04-25T09:14:48.594861+0000","last_fullsized":"2024-04-25T09:14:48.594861+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:06.549632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.657124+0000","last_change":"2024-04-25T09:14:47.593557+0000","last_active":"2024-04-25T09:14:48.657124+0000","last_peered":"2024-04-25T09:14:48.657124+0000","last_clean":"2024-04-25T09:14:48.657124+0000","last_became_active":"2024-04-25T09:14:47.592156+0000","last_became_peered":"2024-04-25T09:14:47.592156+0000","last_unstale":"2024-04-25T09:14:48.657124+0000","last_undegraded":"2024-04-25T09:14:48.657124+0000","last_fullsized":"2024-04-25T09:14:48.657124+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:58.775236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"122'6","reported_seq":20,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.597588+0000","last_change":"2024-04-25T09:14:46.589114+0000","last_active":"2024-04-25T09:14:48.597588+0000","last_peered":"2024-04-25T09:14:48.597588+0000","last_clean":"2024-04-25T09:14:48.597588+0000","last_became_active":"2024-04-25T09:14:46.588827+0000","last_became_peered":"2024-04-25T09:14:46.588827+0000","last_unstale":"2024-04-25T09:14:48.597588+0000","last_undegraded":"2024-04-25T09:14:48.597588+0000","last_fullsized":"2024-04-25T09:14:48.597588+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:01.143137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.594075+0000","last_change":"2024-04-25T09:14:47.598682+0000","last_active":"2024-04-25T09:14:48.594075+0000","last_peered":"2024-04-25T09:14:48.594075+0000","last_clean":"2024-04-25T09:14:48.594075+0000","last_became_active":"2024-04-25T09:14:47.594622+0000","last_became_peered":"2024-04-25T09:14:47.594622+0000","last_unstale":"2024-04-25T09:14:48.594075+0000","last_undegraded":"2024-04-25T09:14:48.594075+0000","last_fullsized":"2024-04-25T09:14:48.594075+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:38.203449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.594602+0000","last_change":"2024-04-25T09:14:46.575598+0000","last_active":"2024-04-25T09:14:48.594602+0000","last_peered":"2024-04-25T09:14:48.594602+0000","last_clean":"2024-04-25T09:14:48.594602+0000","last_became_active":"2024-04-25T09:14:46.575147+0000","last_became_peered":"2024-04-25T09:14:46.575147+0000","last_unstale":"2024-04-25T09:14:48.594602+0000","last_undegraded":"2024-04-25T09:14:48.594602+0000","last_fullsized":"2024-04-25T09:14:48.594602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:54.251451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.596495+0000","last_change":"2024-04-25T09:14:47.601854+0000","last_active":"2024-04-25T09:14:48.596495+0000","last_peered":"2024-04-25T09:14:48.596495+0000","last_clean":"2024-04-25T09:14:48.596495+0000","last_became_active":"2024-04-25T09:14:47.601569+0000","last_became_peered":"2024-04-25T09:14:47.601569+0000","last_unstale":"2024-04-25T09:14:48.596495+0000","last_undegraded":"2024-04-25T09:14:48.596495+0000","last_fullsized":"2024-04-25T09:14:48.596495+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:40.613785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639052+0000","last_change":"2024-04-25T09:14:46.593027+0000","last_active":"2024-04-25T09:14:49.639052+0000","last_peered":"2024-04-25T09:14:49.639052+0000","last_clean":"2024-04-25T09:14:49.639052+0000","last_became_active":"2024-04-25T09:14:46.592086+0000","last_became_peered":"2024-04-25T09:14:46.592086+0000","last_unstale":"2024-04-25T09:14:49.639052+0000","last_undegraded":"2024-04-25T09:14:49.639052+0000","last_fullsized":"2024-04-25T09:14:49.639052+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:22.216841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.598067+0000","last_change":"2024-04-25T09:14:47.601818+0000","last_active":"2024-04-25T09:14:48.598067+0000","last_peered":"2024-04-25T09:14:48.598067+0000","last_clean":"2024-04-25T09:14:48.598067+0000","last_became_active":"2024-04-25T09:14:47.601498+0000","last_became_peered":"2024-04-25T09:14:47.601498+0000","last_unstale":"2024-04-25T09:14:48.598067+0000","last_undegraded":"2024-04-25T09:14:48.598067+0000","last_fullsized":"2024-04-25T09:14:48.598067+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:06.241780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"122'8","reported_seq":22,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.638832+0000","last_change":"2024-04-25T09:14:46.587580+0000","last_active":"2024-04-25T09:14:49.638832+0000","last_peered":"2024-04-25T09:14:49.638832+0000","last_clean":"2024-04-25T09:14:49.638832+0000","last_became_active":"2024-04-25T09:14:46.587260+0000","last_became_peered":"2024-04-25T09:14:46.587260+0000","last_unstale":"2024-04-25T09:14:49.638832+0000","last_undegraded":"2024-04-25T09:14:49.638832+0000","last_fullsized":"2024-04-25T09:14:49.638832+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:54.022509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.641868+0000","last_change":"2024-04-25T09:14:47.591406+0000","last_active":"2024-04-25T09:14:49.641868+0000","last_peered":"2024-04-25T09:14:49.641868+0000","last_clean":"2024-04-25T09:14:49.641868+0000","last_became_active":"2024-04-25T09:14:47.590218+0000","last_became_peered":"2024-04-25T09:14:47.590218+0000","last_unstale":"2024-04-25T09:14:49.641868+0000","last_undegraded":"2024-04-25T09:14:49.641868+0000","last_fullsized":"2024-04-25T09:14:49.641868+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.040243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640532+0000","last_change":"2024-04-25T09:14:46.587941+0000","last_active":"2024-04-25T09:14:49.640532+0000","last_peered":"2024-04-25T09:14:49.640532+0000","last_clean":"2024-04-25T09:14:49.640532+0000","last_became_active":"2024-04-25T09:14:46.587634+0000","last_became_peered":"2024-04-25T09:14:46.587634+0000","last_unstale":"2024-04-25T09:14:49.640532+0000","last_undegraded":"2024-04-25T09:14:49.640532+0000","last_fullsized":"2024-04-25T09:14:49.640532+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:05.263932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.641580+0000","last_change":"2024-04-25T09:14:47.600548+0000","last_active":"2024-04-25T09:14:49.641580+0000","last_peered":"2024-04-25T09:14:49.641580+0000","last_clean":"2024-04-25T09:14:49.641580+0000","last_became_active":"2024-04-25T09:14:47.599345+0000","last_became_peered":"2024-04-25T09:14:47.599345+0000","last_unstale":"2024-04-25T09:14:49.641580+0000","last_undegraded":"2024-04-25T09:14:49.641580+0000","last_fullsized":"2024-04-25T09:14:49.641580+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:15.414150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.595476+0000","last_change":"2024-04-25T09:14:46.585852+0000","last_active":"2024-04-25T09:14:48.595476+0000","last_peered":"2024-04-25T09:14:48.595476+0000","last_clean":"2024-04-25T09:14:48.595476+0000","last_became_active":"2024-04-25T09:14:46.585514+0000","last_became_peered":"2024-04-25T09:14:46.585514+0000","last_unstale":"2024-04-25T09:14:48.595476+0000","last_undegraded":"2024-04-25T09:14:48.595476+0000","last_fullsized":"2024-04-25T09:14:48.595476+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:24.040722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656845+0000","last_change":"2024-04-25T09:14:47.596183+0000","last_active":"2024-04-25T09:14:48.656845+0000","last_peered":"2024-04-25T09:14:48.656845+0000","last_clean":"2024-04-25T09:14:48.656845+0000","last_became_active":"2024-04-25T09:14:47.592481+0000","last_became_peered":"2024-04-25T09:14:47.592481+0000","last_unstale":"2024-04-25T09:14:48.656845+0000","last_undegraded":"2024-04-25T09:14:48.656845+0000","last_fullsized":"2024-04-25T09:14:48.656845+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:16.000897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633702+0000","last_change":"2024-04-25T09:14:46.591007+0000","last_active":"2024-04-25T09:14:48.633702+0000","last_peered":"2024-04-25T09:14:48.633702+0000","last_clean":"2024-04-25T09:14:48.633702+0000","last_became_active":"2024-04-25T09:14:46.590700+0000","last_became_peered":"2024-04-25T09:14:46.590700+0000","last_unstale":"2024-04-25T09:14:48.633702+0000","last_undegraded":"2024-04-25T09:14:48.633702+0000","last_fullsized":"2024-04-25T09:14:48.633702+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:44.884895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633461+0000","last_change":"2024-04-25T09:14:47.593703+0000","last_active":"2024-04-25T09:14:48.633461+0000","last_peered":"2024-04-25T09:14:48.633461+0000","last_clean":"2024-04-25T09:14:48.633461+0000","last_became_active":"2024-04-25T09:14:47.590929+0000","last_became_peered":"2024-04-25T09:14:47.590929+0000","last_unstale":"2024-04-25T09:14:48.633461+0000","last_undegraded":"2024-04-25T09:14:48.633461+0000","last_fullsized":"2024-04-25T09:14:48.633461+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:06.281635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633344+0000","last_change":"2024-04-25T09:14:46.594760+0000","last_active":"2024-04-25T09:14:48.633344+0000","last_peered":"2024-04-25T09:14:48.633344+0000","last_clean":"2024-04-25T09:14:48.633344+0000","last_became_active":"2024-04-25T09:14:46.594507+0000","last_became_peered":"2024-04-25T09:14:46.594507+0000","last_unstale":"2024-04-25T09:14:48.633344+0000","last_undegraded":"2024-04-25T09:14:48.633344+0000","last_fullsized":"2024-04-25T09:14:48.633344+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:10.016248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.638802+0000","last_change":"2024-04-25T09:14:47.578634+0000","last_active":"2024-04-25T09:14:49.638802+0000","last_peered":"2024-04-25T09:14:49.638802+0000","last_clean":"2024-04-25T09:14:49.638802+0000","last_became_active":"2024-04-25T09:14:47.578306+0000","last_became_peered":"2024-04-25T09:14:47.578306+0000","last_unstale":"2024-04-25T09:14:49.638802+0000","last_undegraded":"2024-04-25T09:14:49.638802+0000","last_fullsized":"2024-04-25T09:14:49.638802+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:17.012252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.595481+0000","last_change":"2024-04-25T09:14:46.581913+0000","last_active":"2024-04-25T09:14:48.595481+0000","last_peered":"2024-04-25T09:14:48.595481+0000","last_clean":"2024-04-25T09:14:48.595481+0000","last_became_active":"2024-04-25T09:14:46.580713+0000","last_became_peered":"2024-04-25T09:14:46.580713+0000","last_unstale":"2024-04-25T09:14:48.595481+0000","last_undegraded":"2024-04-25T09:14:48.595481+0000","last_fullsized":"2024-04-25T09:14:48.595481+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:19.855817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640380+0000","last_change":"2024-04-25T09:14:47.591398+0000","last_active":"2024-04-25T09:14:49.640380+0000","last_peered":"2024-04-25T09:14:49.640380+0000","last_clean":"2024-04-25T09:14:49.640380+0000","last_became_active":"2024-04-25T09:14:47.591113+0000","last_became_peered":"2024-04-25T09:14:47.591113+0000","last_unstale":"2024-04-25T09:14:49.640380+0000","last_undegraded":"2024-04-25T09:14:49.640380+0000","last_fullsized":"2024-04-25T09:14:49.640380+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:36.410285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640183+0000","last_change":"2024-04-25T09:14:46.596314+0000","last_active":"2024-04-25T09:14:49.640183+0000","last_peered":"2024-04-25T09:14:49.640183+0000","last_clean":"2024-04-25T09:14:49.640183+0000","last_became_active":"2024-04-25T09:14:46.595756+0000","last_became_peered":"2024-04-25T09:14:46.595756+0000","last_unstale":"2024-04-25T09:14:49.640183+0000","last_undegraded":"2024-04-25T09:14:49.640183+0000","last_fullsized":"2024-04-25T09:14:49.640183+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:49.397810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.632908+0000","last_change":"2024-04-25T09:14:47.582379+0000","last_active":"2024-04-25T09:14:48.632908+0000","last_peered":"2024-04-25T09:14:48.632908+0000","last_clean":"2024-04-25T09:14:48.632908+0000","last_became_active":"2024-04-25T09:14:47.582060+0000","last_became_peered":"2024-04-25T09:14:47.582060+0000","last_unstale":"2024-04-25T09:14:48.632908+0000","last_undegraded":"2024-04-25T09:14:48.632908+0000","last_fullsized":"2024-04-25T09:14:48.632908+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:18.347038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.597551+0000","last_change":"2024-04-25T09:14:46.594582+0000","last_active":"2024-04-25T09:14:48.597551+0000","last_peered":"2024-04-25T09:14:48.597551+0000","last_clean":"2024-04-25T09:14:48.597551+0000","last_became_active":"2024-04-25T09:14:46.594272+0000","last_became_peered":"2024-04-25T09:14:46.594272+0000","last_unstale":"2024-04-25T09:14:48.597551+0000","last_undegraded":"2024-04-25T09:14:48.597551+0000","last_fullsized":"2024-04-25T09:14:48.597551+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:20.847835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.594359+0000","last_change":"2024-04-25T09:14:47.601609+0000","last_active":"2024-04-25T09:14:48.594359+0000","last_peered":"2024-04-25T09:14:48.594359+0000","last_clean":"2024-04-25T09:14:48.594359+0000","last_became_active":"2024-04-25T09:14:47.601329+0000","last_became_peered":"2024-04-25T09:14:47.601329+0000","last_unstale":"2024-04-25T09:14:48.594359+0000","last_undegraded":"2024-04-25T09:14:48.594359+0000","last_fullsized":"2024-04-25T09:14:48.594359+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:11.033950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.596675+0000","last_change":"2024-04-25T09:14:46.585918+0000","last_active":"2024-04-25T09:14:48.596675+0000","last_peered":"2024-04-25T09:14:48.596675+0000","last_clean":"2024-04-25T09:14:48.596675+0000","last_became_active":"2024-04-25T09:14:46.585624+0000","last_became_peered":"2024-04-25T09:14:46.585624+0000","last_unstale":"2024-04-25T09:14:48.596675+0000","last_undegraded":"2024-04-25T09:14:48.596675+0000","last_fullsized":"2024-04-25T09:14:48.596675+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:32.774002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.634003+0000","last_change":"2024-04-25T09:14:47.601019+0000","last_active":"2024-04-25T09:14:48.634003+0000","last_peered":"2024-04-25T09:14:48.634003+0000","last_clean":"2024-04-25T09:14:48.634003+0000","last_became_active":"2024-04-25T09:14:47.598192+0000","last_became_peered":"2024-04-25T09:14:47.598192+0000","last_unstale":"2024-04-25T09:14:48.634003+0000","last_undegraded":"2024-04-25T09:14:48.634003+0000","last_fullsized":"2024-04-25T09:14:48.634003+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:40.613785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.597549+0000","last_change":"2024-04-25T09:14:46.592457+0000","last_active":"2024-04-25T09:14:48.597549+0000","last_peered":"2024-04-25T09:14:48.597549+0000","last_clean":"2024-04-25T09:14:48.597549+0000","last_became_active":"2024-04-25T09:14:46.590215+0000","last_became_peered":"2024-04-25T09:14:46.590215+0000","last_unstale":"2024-04-25T09:14:48.597549+0000","last_undegraded":"2024-04-25T09:14:48.597549+0000","last_fullsized":"2024-04-25T09:14:48.597549+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:17.329335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640405+0000","last_change":"2024-04-25T09:14:47.598602+0000","last_active":"2024-04-25T09:14:49.640405+0000","last_peered":"2024-04-25T09:14:49.640405+0000","last_clean":"2024-04-25T09:14:49.640405+0000","last_became_active":"2024-04-25T09:14:47.598350+0000","last_became_peered":"2024-04-25T09:14:47.598350+0000","last_unstale":"2024-04-25T09:14:49.640405+0000","last_undegraded":"2024-04-25T09:14:49.640405+0000","last_fullsized":"2024-04-25T09:14:49.640405+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:13.437185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.657031+0000","last_change":"2024-04-25T09:14:46.584660+0000","last_active":"2024-04-25T09:14:48.657031+0000","last_peered":"2024-04-25T09:14:48.657031+0000","last_clean":"2024-04-25T09:14:48.657031+0000","last_became_active":"2024-04-25T09:14:46.583729+0000","last_became_peered":"2024-04-25T09:14:46.583729+0000","last_unstale":"2024-04-25T09:14:48.657031+0000","last_undegraded":"2024-04-25T09:14:48.657031+0000","last_fullsized":"2024-04-25T09:14:48.657031+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:47.105902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640945+0000","last_change":"2024-04-25T09:14:47.599967+0000","last_active":"2024-04-25T09:14:49.640945+0000","last_peered":"2024-04-25T09:14:49.640945+0000","last_clean":"2024-04-25T09:14:49.640945+0000","last_became_active":"2024-04-25T09:14:47.599562+0000","last_became_peered":"2024-04-25T09:14:47.599562+0000","last_unstale":"2024-04-25T09:14:49.640945+0000","last_undegraded":"2024-04-25T09:14:49.640945+0000","last_fullsized":"2024-04-25T09:14:49.640945+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:16.052942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.596895+0000","last_change":"2024-04-25T09:14:46.586702+0000","last_active":"2024-04-25T09:14:48.596895+0000","last_peered":"2024-04-25T09:14:48.596895+0000","last_clean":"2024-04-25T09:14:48.596895+0000","last_became_active":"2024-04-25T09:14:46.585417+0000","last_became_peered":"2024-04-25T09:14:46.585417+0000","last_unstale":"2024-04-25T09:14:48.596895+0000","last_undegraded":"2024-04-25T09:14:48.596895+0000","last_fullsized":"2024-04-25T09:14:48.596895+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:40.369999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633889+0000","last_change":"2024-04-25T09:14:47.596995+0000","last_active":"2024-04-25T09:14:48.633889+0000","last_peered":"2024-04-25T09:14:48.633889+0000","last_clean":"2024-04-25T09:14:48.633889+0000","last_became_active":"2024-04-25T09:14:47.596704+0000","last_became_peered":"2024-04-25T09:14:47.596704+0000","last_unstale":"2024-04-25T09:14:48.633889+0000","last_undegraded":"2024-04-25T09:14:48.633889+0000","last_fullsized":"2024-04-25T09:14:48.633889+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:04.407959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.597843+0000","last_change":"2024-04-25T09:14:46.587955+0000","last_active":"2024-04-25T09:14:48.597843+0000","last_peered":"2024-04-25T09:14:48.597843+0000","last_clean":"2024-04-25T09:14:48.597843+0000","last_became_active":"2024-04-25T09:14:46.587640+0000","last_became_peered":"2024-04-25T09:14:46.587640+0000","last_unstale":"2024-04-25T09:14:48.597843+0000","last_undegraded":"2024-04-25T09:14:48.597843+0000","last_fullsized":"2024-04-25T09:14:48.597843+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.100489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.640096+0000","last_change":"2024-04-25T09:14:47.598883+0000","last_active":"2024-04-25T09:14:49.640096+0000","last_peered":"2024-04-25T09:14:49.640096+0000","last_clean":"2024-04-25T09:14:49.640096+0000","last_became_active":"2024-04-25T09:14:47.596134+0000","last_became_peered":"2024-04-25T09:14:47.596134+0000","last_unstale":"2024-04-25T09:14:49.640096+0000","last_undegraded":"2024-04-25T09:14:49.640096+0000","last_fullsized":"2024-04-25T09:14:49.640096+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:57.536377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.656537+0000","last_change":"2024-04-25T09:14:46.580246+0000","last_active":"2024-04-25T09:14:48.656537+0000","last_peered":"2024-04-25T09:14:48.656537+0000","last_clean":"2024-04-25T09:14:48.656537+0000","last_became_active":"2024-04-25T09:14:46.579901+0000","last_became_peered":"2024-04-25T09:14:46.579901+0000","last_unstale":"2024-04-25T09:14:48.656537+0000","last_undegraded":"2024-04-25T09:14:48.656537+0000","last_fullsized":"2024-04-25T09:14:48.656537+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:17.806853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633382+0000","last_change":"2024-04-25T09:14:47.597052+0000","last_active":"2024-04-25T09:14:48.633382+0000","last_peered":"2024-04-25T09:14:48.633382+0000","last_clean":"2024-04-25T09:14:48.633382+0000","last_became_active":"2024-04-25T09:14:47.596642+0000","last_became_peered":"2024-04-25T09:14:47.596642+0000","last_unstale":"2024-04-25T09:14:48.633382+0000","last_undegraded":"2024-04-25T09:14:48.633382+0000","last_fullsized":"2024-04-25T09:14:48.633382+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:32.625432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.632993+0000","last_change":"2024-04-25T09:14:46.591719+0000","last_active":"2024-04-25T09:14:48.632993+0000","last_peered":"2024-04-25T09:14:48.632993+0000","last_clean":"2024-04-25T09:14:48.632993+0000","last_became_active":"2024-04-25T09:14:46.591253+0000","last_became_peered":"2024-04-25T09:14:46.591253+0000","last_unstale":"2024-04-25T09:14:48.632993+0000","last_undegraded":"2024-04-25T09:14:48.632993+0000","last_fullsized":"2024-04-25T09:14:48.632993+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:06.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.633133+0000","last_change":"2024-04-25T09:14:47.581104+0000","last_active":"2024-04-25T09:14:48.633133+0000","last_peered":"2024-04-25T09:14:48.633133+0000","last_clean":"2024-04-25T09:14:48.633133+0000","last_became_active":"2024-04-25T09:14:47.579601+0000","last_became_peered":"2024-04-25T09:14:47.579601+0000","last_unstale":"2024-04-25T09:14:48.633133+0000","last_undegraded":"2024-04-25T09:14:48.633133+0000","last_fullsized":"2024-04-25T09:14:48.633133+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:55.040212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639742+0000","last_change":"2024-04-25T09:14:46.589960+0000","last_active":"2024-04-25T09:14:49.639742+0000","last_peered":"2024-04-25T09:14:49.639742+0000","last_clean":"2024-04-25T09:14:49.639742+0000","last_became_active":"2024-04-25T09:14:46.589342+0000","last_became_peered":"2024-04-25T09:14:46.589342+0000","last_unstale":"2024-04-25T09:14:49.639742+0000","last_undegraded":"2024-04-25T09:14:49.639742+0000","last_fullsized":"2024-04-25T09:14:49.639742+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:05.263932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.638905+0000","last_change":"2024-04-25T09:14:47.601429+0000","last_active":"2024-04-25T09:14:49.638905+0000","last_peered":"2024-04-25T09:14:49.638905+0000","last_clean":"2024-04-25T09:14:49.638905+0000","last_became_active":"2024-04-25T09:14:47.600120+0000","last_became_peered":"2024-04-25T09:14:47.600120+0000","last_unstale":"2024-04-25T09:14:49.638905+0000","last_undegraded":"2024-04-25T09:14:49.638905+0000","last_fullsized":"2024-04-25T09:14:49.638905+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.390883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.638466+0000","last_change":"2024-04-25T09:14:46.587572+0000","last_active":"2024-04-25T09:14:49.638466+0000","last_peered":"2024-04-25T09:14:49.638466+0000","last_clean":"2024-04-25T09:14:49.638466+0000","last_became_active":"2024-04-25T09:14:46.587291+0000","last_became_peered":"2024-04-25T09:14:46.587291+0000","last_unstale":"2024-04-25T09:14:49.638466+0000","last_undegraded":"2024-04-25T09:14:49.638466+0000","last_fullsized":"2024-04-25T09:14:49.638466+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:49.397810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:48.596708+0000","last_change":"2024-04-25T09:14:47.597789+0000","last_active":"2024-04-25T09:14:48.596708+0000","last_peered":"2024-04-25T09:14:48.596708+0000","last_clean":"2024-04-25T09:14:48.596708+0000","last_became_active":"2024-04-25T09:14:47.597478+0000","last_became_peered":"2024-04-25T09:14:47.597478+0000","last_unstale":"2024-04-25T09:14:48.596708+0000","last_undegraded":"2024-04-25T09:14:48.596708+0000","last_fullsized":"2024-04-25T09:14:48.596708+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:46.541370+0000","last_clean_scrub_stamp":"2024-04-25T09:14:46.541370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:42.925014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.639423+0000","last_change":"2024-04-25T09:14:46.588031+0000","last_active":"2024-04-25T09:14:49.639423+0000","last_peered":"2024-04-25T09:14:49.639423+0000","last_clean":"2024-04-25T09:14:49.639423+0000","last_became_active":"2024-04-25T09:14:46.587752+0000","last_became_peered":"2024-04-25T09:14:46.587752+0000","last_unstale":"2024-04-25T09:14:49.639423+0000","last_undegraded":"2024-04-25T09:14:49.639423+0000","last_fullsized":"2024-04-25T09:14:49.639423+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:14:45.523667+0000","last_clean_scrub_stamp":"2024-04-25T09:14:45.523667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:26.237900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":357,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T09:14:49.638616+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:14:49.638616+0000","last_peered":"2024-04-25T09:14:49.638616+0000","last_clean":"2024-04-25T09:14:49.638616+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:14:49.638616+0000","last_undegraded":"2024-04-25T09:14:49.638616+0000","last_fullsized":"2024-04-25T09:14:49.638616+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50744,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321096,"statfs":{"total":96636764160,"available":96584802304,"internally_reserved":0,"allocated":1548288,"data_stored":672055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54880,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316960,"statfs":{"total":96636764160,"available":96580567040,"internally_reserved":0,"allocated":1527808,"data_stored":670319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574993,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55468,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316372,"statfs":{"total":96636764160,"available":96579964928,"internally_reserved":0,"allocated":2129920,"data_stored":1260779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59060,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312780,"statfs":{"total":96636764160,"available":96576286720,"internally_reserved":0,"allocated":1544192,"data_stored":672021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59048,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312792,"statfs":{"total":96636764160,"available":96576299008,"internally_reserved":0,"allocated":1527808,"data_stored":670319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81260,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290580,"statfs":{"total":96636764160,"available":96553553920,"internally_reserved":0,"allocated":1540096,"data_stored":670957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574993,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55460,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316380,"statfs":{"total":96636764160,"available":96579973120,"internally_reserved":0,"allocated":2121728,"data_stored":1260687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46568,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325272,"statfs":{"total":96636764160,"available":96589078528,"internally_reserved":0,"allocated":1536000,"data_stored":670971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:14:52.607 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:14:52.607 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:14:52.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.882+0000 7fa211000640 1 Processor -- start 2024-04-25T09:14:52.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.883+0000 7fa211000640 1 -- start start 2024-04-25T09:14:52.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.884+0000 7fa211000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa20c0cc960 0x7fa20c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.884+0000 7fa211000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c06d490 con 0x7fa20c159680 2024-04-25T09:14:52.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.884+0000 7fa211000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c06c3f0 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.884+0000 7fa211000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c06d000 con 0x7fa20c0cc960 2024-04-25T09:14:52.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.885+0000 7fa20aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa20c0cc960 0x7fa20c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.885+0000 7fa20b400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa20c159680 0x7fa20c15cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60624/0 (socket says 172.21.15.62:60624) 2024-04-25T09:14:52.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.885+0000 7fa20b400640 1 -- 172.21.15.62:0/506435652 learned_addr learned my addr 172.21.15.62:0/506435652 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:52.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.885+0000 7fa20aa00640 1 -- 172.21.15.62:0/506435652 >> v1:172.21.15.121:6789/0 conn(0x7fa20c0b0eb0 legacy=0x7fa20c1656f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:52.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.885+0000 7fa20aa00640 1 -- 172.21.15.62:0/506435652 >> v1:172.21.15.62:6789/0 conn(0x7fa20c159680 legacy=0x7fa20c15cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.885+0000 7fa20aa00640 1 -- 172.21.15.62:0/506435652 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c06c6d0 con 0x7fa20c0cc960 2024-04-25T09:14:52.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.885+0000 7fa20a000640 1 -- 172.21.15.62:0/506435652 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519883166 0 0) 0x7fa20c06d490 con 0x7fa20c159680 2024-04-25T09:14:52.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.887+0000 7fa20aa00640 1 --2- 172.21.15.62:0/506435652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0cc960 0x7fa20c0b0970 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40097a0 tx=0x7fa1f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d6bbcc3e2f68cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:52.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.887+0000 7fa20a000640 1 -- 172.21.15.62:0/506435652 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4037070 con 0x7fa20c0cc960 2024-04-25T09:14:52.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.887+0000 7fa20a000640 1 -- 172.21.15.62:0/506435652 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f4032440 con 0x7fa20c0cc960 2024-04-25T09:14:52.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.887+0000 7fa20a000640 1 -- 172.21.15.62:0/506435652 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4032810 con 0x7fa20c0cc960 2024-04-25T09:14:52.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.888+0000 7fa211000640 1 -- 172.21.15.62:0/506435652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0cc960 msgr2=0x7fa20c0b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.888+0000 7fa211000640 1 --2- 172.21.15.62:0/506435652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0cc960 0x7fa20c0b0970 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40097a0 tx=0x7fa1f402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.888+0000 7fa211000640 1 -- 172.21.15.62:0/506435652 shutdown_connections 2024-04-25T09:14:52.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.888+0000 7fa211000640 1 --2- 172.21.15.62:0/506435652 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0cc960 0x7fa20c0b0970 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.888+0000 7fa211000640 1 -- 172.21.15.62:0/506435652 >> 172.21.15.62:0/506435652 conn(0x7fa20c0668f0 msgr2=0x7fa20c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:52.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.888+0000 7fa211000640 1 -- 172.21.15.62:0/506435652 shutdown_connections 2024-04-25T09:14:52.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.888+0000 7fa211000640 1 -- 172.21.15.62:0/506435652 wait complete. 2024-04-25T09:14:52.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.890+0000 7fa211000640 1 Processor -- start 2024-04-25T09:14:52.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.890+0000 7fa211000640 1 -- start start 2024-04-25T09:14:52.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa211000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0b0eb0 0x7fa20c143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa211000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20c0cc960 0x7fa20c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa211000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20c159680 0x7fa20c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa211000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c159a60 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa211000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c06cc90 con 0x7fa20c159680 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa211000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c15e2a0 con 0x7fa20c0cc960 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa20b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0b0eb0 0x7fa20c143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa20b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0b0eb0 0x7fa20c143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34516/0 (socket says 172.21.15.62:34516) 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.891+0000 7fa20b400640 1 -- 172.21.15.62:0/906318023 learned_addr learned my addr 172.21.15.62:0/906318023 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20aa00640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20c0cc960 0x7fa20c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20be00640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20c159680 0x7fa20c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20b400640 1 -- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20c0cc960 msgr2=0x7fa20c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20b400640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20c0cc960 0x7fa20c154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20b400640 1 -- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20c159680 msgr2=0x7fa20c151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20b400640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20c159680 0x7fa20c151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:52.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20b400640 1 -- 172.21.15.62:0/906318023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c153b50 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20aa00640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20c0cc960 0x7fa20c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:52.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20be00640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20c159680 0x7fa20c151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:52.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa20b400640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0b0eb0 0x7fa20c143fc0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c7c0 tx=0x7fa1fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:52.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa203400640 1 -- 172.21.15.62:0/906318023 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc018070 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa203400640 1 -- 172.21.15.62:0/906318023 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc00a640 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa203400640 1 -- 172.21.15.62:0/906318023 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00aa30 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.892+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c153e30 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.893+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c1543f0 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.894+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c075070 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.895+0000 7fa203400640 1 -- 172.21.15.62:0/906318023 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1fc00abd0 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.896+0000 7fa203400640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:52.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.896+0000 7fa203400640 1 -- 172.21.15.62:0/906318023 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1fc089890 con 0x7fa20c0b0eb0 2024-04-25T09:14:52.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.896+0000 7fa20aa00640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:52.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.898+0000 7fa20aa00640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e806b4d0 0x7fa1e806d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4034000 tx=0x7fa1f402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:52.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:52.901+0000 7fa203400640 1 -- 172.21.15.62:0/906318023 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1fc058670 con 0x7fa20c0b0eb0 2024-04-25T09:14:53.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.177+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa20c06e020 con 0x7fa20c0b0eb0 2024-04-25T09:14:53.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.183+0000 7fa203400640 1 -- 172.21.15.62:0/906318023 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa1fc057f70 con 0x7fa20c0b0eb0 2024-04-25T09:14:53.170 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:14:53.170 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:14:53.170 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:14:53.170 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:14:53.170 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:14:53.170 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e806b4d0 msgr2=0x7fa1e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e806b4d0 0x7fa1e806d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4034000 tx=0x7fa1f402e000 comp rx=0 tx=0).stop 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0b0eb0 msgr2=0x7fa20c143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0b0eb0 0x7fa20c143fc0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c7c0 tx=0x7fa1fc00cc80 comp rx=0 tx=0).stop 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 shutdown_connections 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20c159680 0x7fa20c151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa20c0cc960 0x7fa20c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 --2- 172.21.15.62:0/906318023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa20c0b0eb0 0x7fa20c143fc0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 >> 172.21.15.62:0/906318023 conn(0x7fa20c0668f0 msgr2=0x7fa20c0cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 shutdown_connections 2024-04-25T09:14:53.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.186+0000 7fa211000640 1 -- 172.21.15.62:0/906318023 wait complete. 2024-04-25T09:14:53.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:14:53.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.413+0000 7f624b800640 1 Processor -- start 2024-04-25T09:14:53.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.413+0000 7f624b800640 1 -- start start 2024-04-25T09:14:53.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f624b800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62440cfed0 0x7f624415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:53.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f624b800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6244072340 0x7f62440702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:53.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f624b800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62440707f0 0x7f624415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:53.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f624b800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6244070c60 con 0x7f62440cfed0 2024-04-25T09:14:53.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f624b800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624406c310 con 0x7f6244072340 2024-04-25T09:14:53.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f624b800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624415f5f0 con 0x7f62440707f0 2024-04-25T09:14:53.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f6248e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62440cfed0 0x7f624415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:53.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f6248e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62440cfed0 0x7f624415f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34520/0 (socket says 172.21.15.62:34520) 2024-04-25T09:14:53.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.415+0000 7f6248e00640 1 -- 172.21.15.62:0/4159000966 learned_addr learned my addr 172.21.15.62:0/4159000966 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:53.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.416+0000 7f6249800640 1 --2- 172.21.15.62:0/4159000966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62440707f0 0x7f624415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:53.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.416+0000 7f623be00640 1 --2- 172.21.15.62:0/4159000966 >> v2:172.21.15.121:3300/0 conn(0x7f6244072340 0x7f62440702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:53.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.416+0000 7f6248e00640 1 -- 172.21.15.62:0/4159000966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62440707f0 msgr2=0x7f624415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:53.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.416+0000 7f6248e00640 1 --2- 172.21.15.62:0/4159000966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62440707f0 0x7f624415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.416+0000 7f6248e00640 1 -- 172.21.15.62:0/4159000966 >> v2:172.21.15.121:3300/0 conn(0x7f6244072340 msgr2=0x7f62440702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:53.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.416+0000 7f6248e00640 1 --2- 172.21.15.62:0/4159000966 >> v2:172.21.15.121:3300/0 conn(0x7f6244072340 0x7f62440702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.416+0000 7f6248e00640 1 -- 172.21.15.62:0/4159000966 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624406c5b0 con 0x7f62440cfed0 2024-04-25T09:14:53.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.416+0000 7f6249800640 1 --2- 172.21.15.62:0/4159000966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62440707f0 0x7f624415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:53.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.417+0000 7f6248e00640 1 --2- 172.21.15.62:0/4159000966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624415f0b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f6244072760 tx=0x7f623402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c6d8c48e6987fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.417+0000 7f6243e00640 1 -- 172.21.15.62:0/4159000966 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234036070 con 0x7f62440cfed0 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f6243e00640 1 -- 172.21.15.62:0/4159000966 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6234031600 con 0x7f62440cfed0 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f6243e00640 1 -- 172.21.15.62:0/4159000966 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62340319d0 con 0x7f62440cfed0 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 -- 172.21.15.62:0/4159000966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 msgr2=0x7f624415f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 --2- 172.21.15.62:0/4159000966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624415f0b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f6244072760 tx=0x7f623402a6a0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 -- 172.21.15.62:0/4159000966 shutdown_connections 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 --2- 172.21.15.62:0/4159000966 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62440707f0 0x7f624415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 --2- 172.21.15.62:0/4159000966 >> v2:172.21.15.121:3300/0 conn(0x7f6244072340 0x7f62440702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 --2- 172.21.15.62:0/4159000966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624415f0b0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 -- 172.21.15.62:0/4159000966 >> 172.21.15.62:0/4159000966 conn(0x7f62440668f0 msgr2=0x7f6244066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 -- 172.21.15.62:0/4159000966 shutdown_connections 2024-04-25T09:14:53.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.418+0000 7f624b800640 1 -- 172.21.15.62:0/4159000966 wait complete. 2024-04-25T09:14:53.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.420+0000 7f624b800640 1 Processor -- start 2024-04-25T09:14:53.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.420+0000 7f624b800640 1 -- start start 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f624b800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62440707f0 0x7f62441525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f624b800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624413abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f624b800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f624413b120 0x7f6244155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f624b800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624406ce50 con 0x7f62440cfed0 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f624b800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624406cb70 con 0x7f62440707f0 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f624b800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624406d130 con 0x7f624413b120 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f623be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624413abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f623be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624413abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34528/0 (socket says 172.21.15.62:34528) 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.421+0000 7f623be00640 1 -- 172.21.15.62:0/1935877369 learned_addr learned my addr 172.21.15.62:0/1935877369 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f623be00640 1 -- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f624413b120 msgr2=0x7f6244155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:53.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f623be00640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f624413b120 0x7f6244155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f623be00640 1 -- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62440707f0 msgr2=0x7f62441525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:53.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f623be00640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62440707f0 0x7f62441525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f623be00640 1 -- 172.21.15.62:0/1935877369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244156400 con 0x7f62440cfed0 2024-04-25T09:14:53.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f623be00640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624413abe0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c7f0 tx=0x7f622c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:53.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f6241600640 1 -- 172.21.15.62:0/1935877369 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c015070 con 0x7f62440cfed0 2024-04-25T09:14:53.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f6241600640 1 -- 172.21.15.62:0/1935877369 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c004240 con 0x7f62440cfed0 2024-04-25T09:14:53.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.422+0000 7f6241600640 1 -- 172.21.15.62:0/1935877369 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c007620 con 0x7f62440cfed0 2024-04-25T09:14:53.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.423+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62441566e0 con 0x7f62440cfed0 2024-04-25T09:14:53.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.423+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624413c910 con 0x7f62440cfed0 2024-04-25T09:14:53.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.424+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624413c420 con 0x7f62440cfed0 2024-04-25T09:14:53.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.425+0000 7f6241600640 1 -- 172.21.15.62:0/1935877369 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f622c0079d0 con 0x7f62440cfed0 2024-04-25T09:14:53.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.426+0000 7f6241600640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f622006b4d0 0x7f622006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:53.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.426+0000 7f6241600640 1 -- 172.21.15.62:0/1935877369 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f622c08a3a0 con 0x7f62440cfed0 2024-04-25T09:14:53.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.426+0000 7f6248e00640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f622006b4d0 0x7f622006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:53.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.428+0000 7f6248e00640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f622006b4d0 0x7f622006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6234033000 tx=0x7f623402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:53.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.430+0000 7f6241600640 1 -- 172.21.15.62:0/1935877369 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f622c059180 con 0x7f62440cfed0 2024-04-25T09:14:53.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.667+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6244072c70 con 0x7f62440cfed0 2024-04-25T09:14:53.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.668+0000 7f6241600640 1 -- 172.21.15.62:0/1935877369 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f622c058a80 con 0x7f62440cfed0 2024-04-25T09:14:53.655 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:14:53.655 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:14:53.655 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:14:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f622006b4d0 msgr2=0x7f622006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f622006b4d0 0x7f622006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6234033000 tx=0x7f623402e000 comp rx=0 tx=0).stop 2024-04-25T09:14:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 msgr2=0x7f624413abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:53.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624413abe0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c7f0 tx=0x7f622c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 shutdown_connections 2024-04-25T09:14:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f622006b4d0 0x7f622006d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62440cfed0 0x7f624413abe0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f624413b120 0x7f6244155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 --2- 172.21.15.62:0/1935877369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62440707f0 0x7f62441525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 >> 172.21.15.62:0/1935877369 conn(0x7f62440668f0 msgr2=0x7f62440ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:53.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 shutdown_connections 2024-04-25T09:14:53.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:53.671+0000 7f624b800640 1 -- 172.21.15.62:0/1935877369 wait complete. 2024-04-25T09:14:54.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:14:47.039566+0000', 'modified': '2024-04-25T09:14:50.155038+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8304}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8304': {'gid': 8304, 'name': 'c', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/2051823365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2051823365}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2051823365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:14:54.079 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:14:54.080 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:14:54.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.316+0000 7fca4e200640 1 Processor -- start 2024-04-25T09:14:54.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.316+0000 7fca4e200640 1 -- start start 2024-04-25T09:14:54.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.318+0000 7fca4e200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca48161230 0x7fca480b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.318+0000 7fca4e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480d0fb0 0x7fca480b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.318+0000 7fca4e200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca480cea70 con 0x7fca480b5a60 2024-04-25T09:14:54.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.318+0000 7fca4e200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca480b5ec0 con 0x7fca48161230 2024-04-25T09:14:54.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.318+0000 7fca4e200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca480d2040 con 0x7fca480d0fb0 2024-04-25T09:14:54.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca47e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480d0fb0 0x7fca480b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca4ce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fca48161230 0x7fca480b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca4d800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fca480b5a60 0x7fca480d2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:35258/0 (socket says 172.21.15.62:35258) 2024-04-25T09:14:54.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca47e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480d0fb0 0x7fca480b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:38174/0 (socket says 172.21.15.62:38174) 2024-04-25T09:14:54.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca4d800640 1 -- 172.21.15.62:0/3925015972 learned_addr learned my addr 172.21.15.62:0/3925015972 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:14:54.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca4ce00640 1 -- 172.21.15.62:0/3925015972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480d0fb0 msgr2=0x7fca480b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca4ce00640 1 --2- 172.21.15.62:0/3925015972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480d0fb0 0x7fca480b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca4ce00640 1 -- 172.21.15.62:0/3925015972 >> v1:172.21.15.121:6789/0 conn(0x7fca480b5a60 legacy=0x7fca480d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.319+0000 7fca4ce00640 1 -- 172.21.15.62:0/3925015972 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480ced50 con 0x7fca48161230 2024-04-25T09:14:54.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.321+0000 7fca4ce00640 1 --2- 172.21.15.62:0/3925015972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 0x7fca480b6c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0097a0 tx=0x7fca3c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea4006f6fa162ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.321+0000 7fca47400640 1 -- 172.21.15.62:0/3925015972 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c037070 con 0x7fca48161230 2024-04-25T09:14:54.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca47400640 1 -- 172.21.15.62:0/3925015972 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c032440 con 0x7fca48161230 2024-04-25T09:14:54.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca47400640 1 -- 172.21.15.62:0/3925015972 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c032810 con 0x7fca48161230 2024-04-25T09:14:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca4e200640 1 -- 172.21.15.62:0/3925015972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 msgr2=0x7fca480b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca4e200640 1 --2- 172.21.15.62:0/3925015972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 0x7fca480b6c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0097a0 tx=0x7fca3c02a180 comp rx=0 tx=0).stop 2024-04-25T09:14:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca4e200640 1 -- 172.21.15.62:0/3925015972 shutdown_connections 2024-04-25T09:14:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca4e200640 1 --2- 172.21.15.62:0/3925015972 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480d0fb0 0x7fca480b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca4e200640 1 --2- 172.21.15.62:0/3925015972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 0x7fca480b6c40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca4e200640 1 -- 172.21.15.62:0/3925015972 >> 172.21.15.62:0/3925015972 conn(0x7fca48066d10 msgr2=0x7fca48067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca4e200640 1 -- 172.21.15.62:0/3925015972 shutdown_connections 2024-04-25T09:14:54.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.322+0000 7fca4e200640 1 -- 172.21.15.62:0/3925015972 wait complete. 2024-04-25T09:14:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.324+0000 7fca4e200640 1 Processor -- start 2024-04-25T09:14:54.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.324+0000 7fca4e200640 1 -- start start 2024-04-25T09:14:54.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca4e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480b5a60 0x7fca48154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca4e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca480d0fb0 0x7fca4814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca4e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 0x7fca480c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca4e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480ca8d0 con 0x7fca480d0fb0 2024-04-25T09:14:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca4e200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480d2290 con 0x7fca48161230 2024-04-25T09:14:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca4e200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca48161610 con 0x7fca480b5a60 2024-04-25T09:14:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca47e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca480d0fb0 0x7fca4814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca47e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca480d0fb0 0x7fca4814af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34532/0 (socket says 172.21.15.62:34532) 2024-04-25T09:14:54.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.325+0000 7fca47e00640 1 -- 172.21.15.62:0/2407952437 learned_addr learned my addr 172.21.15.62:0/2407952437 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:54.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.326+0000 7fca4d800640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 0x7fca480c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.326+0000 7fca4ce00640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480b5a60 0x7fca48154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.326+0000 7fca47e00640 1 -- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480b5a60 msgr2=0x7fca48154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.326+0000 7fca47e00640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480b5a60 0x7fca48154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.326+0000 7fca47e00640 1 -- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 msgr2=0x7fca480c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.326+0000 7fca47e00640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 0x7fca480c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.326+0000 7fca47e00640 1 -- 172.21.15.62:0/2407952437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480c9270 con 0x7fca480d0fb0 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.326+0000 7fca47e00640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca480d0fb0 0x7fca4814af40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fca3400e9e0 tx=0x7fca3400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.327+0000 7fca44c00640 1 -- 172.21.15.62:0/2407952437 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34017070 con 0x7fca480d0fb0 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.327+0000 7fca44c00640 1 -- 172.21.15.62:0/2407952437 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400c9d0 con 0x7fca480d0fb0 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.327+0000 7fca44c00640 1 -- 172.21.15.62:0/2407952437 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400cdc0 con 0x7fca480d0fb0 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.327+0000 7fca4e200640 1 -- 172.21.15.62:0/2407952437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca480c9510 con 0x7fca480d0fb0 2024-04-25T09:14:54.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.327+0000 7fca4e200640 1 -- 172.21.15.62:0/2407952437 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca48159060 con 0x7fca480d0fb0 2024-04-25T09:14:54.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.329+0000 7fca44c00640 1 -- 172.21.15.62:0/2407952437 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca3401a070 con 0x7fca480d0fb0 2024-04-25T09:14:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.330+0000 7fca44c00640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca2806b4d0 0x7fca2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.330+0000 7fca44c00640 1 -- 172.21.15.62:0/2407952437 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca3408a650 con 0x7fca480d0fb0 2024-04-25T09:14:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.331+0000 7fca4ce00640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca2806b4d0 0x7fca2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.331+0000 7fca52799740 1 Processor -- start 2024-04-25T09:14:54.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.331+0000 7fca52799740 1 -- start start 2024-04-25T09:14:54.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca4ce00640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca2806b4d0 0x7fca2806d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fca3c034000 tx=0x7fca3c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca52799740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e863eb30 0x5635e8666970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca52799740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca52799740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 0x5635e863d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca52799740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e85a2620 con 0x5635e8666eb0 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca52799740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e854bca0 con 0x5635e86877a0 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca52799740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635e859e6c0 con 0x5635e863eb30 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca47e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca47e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34534/0 (socket says 172.21.15.62:34534) 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca47e00640 1 -- 172.21.15.62:0/2250049841 learned_addr learned my addr 172.21.15.62:0/2250049841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca4ce00640 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e863eb30 0x5635e8666970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca4d800640 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 0x5635e863d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca47e00640 1 -- 172.21.15.62:0/2250049841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e863eb30 msgr2=0x5635e8666970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca47e00640 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e863eb30 0x5635e8666970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca47e00640 1 -- 172.21.15.62:0/2250049841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 msgr2=0x5635e863d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca47e00640 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 0x5635e863d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.332+0000 7fca47e00640 1 -- 172.21.15.62:0/2250049841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e8552f40 con 0x5635e8666eb0 2024-04-25T09:14:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.333+0000 7fca47e00640 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863ef10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fca34058ac0 tx=0x7fca34058850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52182392952f9aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.333+0000 7fca39600640 1 -- 172.21.15.62:0/2250049841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34021070 con 0x5635e8666eb0 2024-04-25T09:14:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.333+0000 7fca39600640 1 -- 172.21.15.62:0/2250049841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3405cf10 con 0x5635e8666eb0 2024-04-25T09:14:54.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.333+0000 7fca39600640 1 -- 172.21.15.62:0/2250049841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340614d0 con 0x5635e8666eb0 2024-04-25T09:14:54.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.333+0000 7fca52799740 1 -- 172.21.15.62:0/2250049841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 msgr2=0x5635e863ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.333+0000 7fca52799740 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863ef10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fca34058ac0 tx=0x7fca34058850 comp rx=0 tx=0).stop 2024-04-25T09:14:54.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.334+0000 7fca52799740 1 -- 172.21.15.62:0/2250049841 shutdown_connections 2024-04-25T09:14:54.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.334+0000 7fca52799740 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863ef10 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.334+0000 7fca52799740 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 0x5635e863d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.334+0000 7fca52799740 1 --2- 172.21.15.62:0/2250049841 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e863eb30 0x5635e8666970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.334+0000 7fca52799740 1 -- 172.21.15.62:0/2250049841 >> 172.21.15.62:0/2250049841 conn(0x5635e863bd50 msgr2=0x5635e8684730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.334+0000 7fca52799740 1 -- 172.21.15.62:0/2250049841 shutdown_connections 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.334+0000 7fca52799740 1 -- 172.21.15.62:0/2250049841 wait complete. 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.334+0000 7fca52799740 1 -- start start 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.335+0000 7fca52799740 1 Processor -- start 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.335+0000 7fca52799740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e8666eb0 0x5635e8683ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca52799740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 0x5635e866a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca52799740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e866acb0 0x5635e866cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca52799740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e86664e0 con 0x5635e866acb0 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca52799740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e85411f0 con 0x5635e86877a0 2024-04-25T09:14:54.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca52799740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635e8665ed0 con 0x5635e8666eb0 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca52799740 10 client.0 _ll_register_callbacks cb 0x5635e855ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e866acb0 0x5635e866cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e866acb0 0x5635e866cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34540/0 (socket says 172.21.15.62:34540) 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4d800640 1 -- 172.21.15.62:0/1566822227 learned_addr learned my addr 172.21.15.62:0/1566822227 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca47e00640 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 0x5635e866a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4ce00640 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e8666eb0 0x5635e8683ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4d800640 1 -- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e8666eb0 msgr2=0x5635e8683ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4d800640 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e8666eb0 0x5635e8683ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4d800640 1 -- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 msgr2=0x5635e866a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4d800640 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 0x5635e866a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4d800640 1 -- 172.21.15.62:0/1566822227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e8560c50 con 0x5635e866acb0 2024-04-25T09:14:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.336+0000 7fca4ce00640 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e8666eb0 0x5635e8683ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.337+0000 7fca4d800640 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e866acb0 0x5635e866cb70 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fca4000e870 tx=0x7fca4000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281d2f1d878c6379 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.337+0000 7fca2e000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.337+0000 7fca2e000640 1 -- 172.21.15.62:0/1566822227 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4000cb30 con 0x5635e866acb0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.337+0000 7fca52799740 1 -- 172.21.15.62:0/1566822227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca3405cf10 con 0x5635e866acb0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.337+0000 7fca2e000640 1 -- 172.21.15.62:0/1566822227 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca4000ccd0 con 0x5635e866acb0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.337+0000 7fca2e000640 1 -- 172.21.15.62:0/1566822227 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca40020050 con 0x5635e866acb0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.337+0000 7fca52799740 10 client.8451 fetch_fsmap learned FSMap version 78 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.337+0000 7fca52799740 1 -- 172.21.15.62:0/1566822227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x5635e82b72d0 con 0x5635e866acb0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca2e000640 1 -- 172.21.15.62:0/1566822227 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3405cf10 con 0x5635e866acb0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca2e000640 1 -- 172.21.15.62:0/1566822227 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca40023020 con 0x5635e866acb0 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca52799740 10 client.8451 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca52799740 10 client.8451 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca52799740 20 client.8451 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:54.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca52799740 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5635e8663bf0 0x5635e86690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca52799740 4 client.8451 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:14:54.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca52799740 1 -- 172.21.15.62:0/1566822227 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e85afb60 con 0x5635e8663bf0 2024-04-25T09:14:54.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.338+0000 7fca4ce00640 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5635e8663bf0 0x5635e86690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.339+0000 7fca4ce00640 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5635e8663bf0 0x5635e86690b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34773b232dbaba3b server_cookie=de6768a23472db18 in_seq=0 out_seq=0 2024-04-25T09:14:54.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.339+0000 7fca2e000640 10 client.8451 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:14:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.339+0000 7fca2e000640 1 -- 172.21.15.62:0/1566822227 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5635e85afb60 con 0x5635e8663bf0 2024-04-25T09:14:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.339+0000 7fca2e000640 10 client.8451 handle_command_reply: tid=0 2024-04-25T09:14:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 client.8451 shutdown 2024-04-25T09:14:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 -- 172.21.15.62:0/1566822227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e866acb0 msgr2=0x5635e866cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e866acb0 0x5635e866cb70 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fca4000e870 tx=0x7fca4000ed30 comp rx=0 tx=0).stop 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 -- 172.21.15.62:0/1566822227 shutdown_connections 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5635e8663bf0 0x5635e86690b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e866acb0 0x5635e866cb70 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e86877a0 0x5635e866a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 --2- 172.21.15.62:0/1566822227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e8666eb0 0x5635e8683ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca2e000640 0 client.8451 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 -- 172.21.15.62:0/1566822227 >> 172.21.15.62:0/1566822227 conn(0x5635e863eb30 msgr2=0x5635e8687c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 -- 172.21.15.62:0/1566822227 shutdown_connections 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 1 -- 172.21.15.62:0/1566822227 wait complete. 2024-04-25T09:14:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.340+0000 7fca52799740 20 client.8451 trim_cache size 0 max 16384 2024-04-25T09:14:54.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.357+0000 7fca52799740 1 Processor -- start 2024-04-25T09:14:54.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.357+0000 7fca52799740 1 -- start start 2024-04-25T09:14:54.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca52799740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca52799740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 0x5635e866a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca52799740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 0x5635e866a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca52799740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480c7fe0 con 0x5635e8666eb0 2024-04-25T09:14:54.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca52799740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480c73e0 con 0x5635e866acb0 2024-04-25T09:14:54.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca52799740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca480c79e0 con 0x5635e86877a0 2024-04-25T09:14:54.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca4ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca4ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34556/0 (socket says 172.21.15.62:34556) 2024-04-25T09:14:54.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.358+0000 7fca4ce00640 1 -- 172.21.15.62:0/211787562 learned_addr learned my addr 172.21.15.62:0/211787562 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:54.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca4ce00640 1 -- 172.21.15.62:0/211787562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 msgr2=0x5635e866a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca4ce00640 1 --2- 172.21.15.62:0/211787562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 0x5635e866a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca4ce00640 1 -- 172.21.15.62:0/211787562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 msgr2=0x5635e866a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca4ce00640 1 --2- 172.21.15.62:0/211787562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 0x5635e866a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca4ce00640 1 -- 172.21.15.62:0/211787562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e866cee0 con 0x5635e8666eb0 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca4ce00640 1 --2- 172.21.15.62:0/211787562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863d710 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fca3c035040 tx=0x7fca3c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d7c04ec3c87771 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca38c00640 1 -- 172.21.15.62:0/211787562 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c037040 con 0x5635e8666eb0 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca38c00640 1 -- 172.21.15.62:0/211787562 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c004240 con 0x5635e8666eb0 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.359+0000 7fca38c00640 1 -- 172.21.15.62:0/211787562 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c006e10 con 0x5635e8666eb0 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 -- 172.21.15.62:0/211787562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 msgr2=0x5635e863d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 --2- 172.21.15.62:0/211787562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863d710 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fca3c035040 tx=0x7fca3c008030 comp rx=0 tx=0).stop 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 -- 172.21.15.62:0/211787562 shutdown_connections 2024-04-25T09:14:54.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 --2- 172.21.15.62:0/211787562 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 0x5635e866a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 --2- 172.21.15.62:0/211787562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 0x5635e866a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 --2- 172.21.15.62:0/211787562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e863d710 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 -- 172.21.15.62:0/211787562 >> 172.21.15.62:0/211787562 conn(0x5635e863eb30 msgr2=0x5635e8686f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:54.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 -- 172.21.15.62:0/211787562 shutdown_connections 2024-04-25T09:14:54.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.360+0000 7fca52799740 1 -- 172.21.15.62:0/211787562 wait complete. 2024-04-25T09:14:54.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.361+0000 7fca52799740 1 -- start start 2024-04-25T09:14:54.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.361+0000 7fca52799740 1 Processor -- start 2024-04-25T09:14:54.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.362+0000 7fca52799740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e8687b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.362+0000 7fca52799740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 0x5635e86880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.362+0000 7fca52799740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 0x5635e86a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.362+0000 7fca52799740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e8663a70 con 0x5635e8666eb0 2024-04-25T09:14:54.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.362+0000 7fca52799740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e866b4a0 con 0x5635e866acb0 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.362+0000 7fca52799740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635e8664070 con 0x5635e86877a0 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.362+0000 7fca52799740 10 client.0 _ll_register_callbacks cb 0x5635e8640d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.362+0000 7fca4ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e8687b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e8687b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34558/0 (socket says 172.21.15.62:34558) 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4ce00640 1 -- 172.21.15.62:0/3769036987 learned_addr learned my addr 172.21.15.62:0/3769036987 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4d800640 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 0x5635e86a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4ce00640 1 -- 172.21.15.62:0/3769036987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 msgr2=0x5635e86a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4ce00640 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 0x5635e86a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca47e00640 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 0x5635e86880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4ce00640 1 -- 172.21.15.62:0/3769036987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 msgr2=0x5635e86880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4ce00640 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 0x5635e86880c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4ce00640 1 -- 172.21.15.62:0/3769036987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e83816b0 con 0x5635e8666eb0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca4ce00640 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e8687b80 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fca3c02a770 tx=0x7fca3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95da5a570d5b0767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.363+0000 7fca2e000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca2e000640 1 -- 172.21.15.62:0/3769036987 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c037040 con 0x5635e8666eb0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca2e000640 1 -- 172.21.15.62:0/3769036987 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca3c04b2f0 con 0x5635e8666eb0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca52799740 1 -- 172.21.15.62:0/3769036987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca3c004240 con 0x5635e8666eb0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca2e000640 1 -- 172.21.15.62:0/3769036987 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c04b530 con 0x5635e8666eb0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca52799740 10 client.8457 fetch_fsmap learned FSMap version 78 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca52799740 1 -- 172.21.15.62:0/3769036987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x5635e86a4390 con 0x5635e8666eb0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca2e000640 1 -- 172.21.15.62:0/3769036987 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c004240 con 0x5635e8666eb0 2024-04-25T09:14:54.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca2e000640 1 -- 172.21.15.62:0/3769036987 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca3c04b6d0 con 0x5635e8666eb0 2024-04-25T09:14:54.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca52799740 10 client.8457 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:14:54.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca52799740 10 client.8457 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.364+0000 7fca52799740 20 client.8457 populate_metadata read hostname 'smithi062' 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca52799740 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5635e86a6f00 0x5635e86a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca52799740 4 client.8457 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca52799740 1 -- 172.21.15.62:0/3769036987 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5635e85a9eb0 con 0x5635e86a6f00 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca47e00640 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5635e86a6f00 0x5635e86a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca47e00640 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5635e86a6f00 0x5635e86a3310 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8d56eb6a575924 server_cookie=87c172a1bb961dfa in_seq=0 out_seq=0 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca2e000640 10 client.8457 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca2e000640 1 -- 172.21.15.62:0/3769036987 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5635e85a9eb0 con 0x5635e86a6f00 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca2e000640 10 client.8457 handle_command_reply: tid=0 2024-04-25T09:14:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.365+0000 7fca52799740 1 client.8457 shutdown 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 -- 172.21.15.62:0/3769036987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 msgr2=0x5635e8687b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e8687b80 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fca3c02a770 tx=0x7fca3c004730 comp rx=0 tx=0).stop 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 -- 172.21.15.62:0/3769036987 shutdown_connections 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5635e86a6f00 0x5635e86a3310 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5635e86877a0 0x5635e86a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635e866acb0 0x5635e86880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 --2- 172.21.15.62:0/3769036987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5635e8666eb0 0x5635e8687b80 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 -- 172.21.15.62:0/3769036987 >> 172.21.15.62:0/3769036987 conn(0x5635e863eb30 msgr2=0x5635e8655880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca2e000640 0 client.8457 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 -- 172.21.15.62:0/3769036987 shutdown_connections 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 1 -- 172.21.15.62:0/3769036987 wait complete. 2024-04-25T09:14:54.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.366+0000 7fca52799740 20 client.8457 trim_cache size 0 max 16384 2024-04-25T09:14:54.360 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:14:54.360 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:14:54.360 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:14:54.360 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 8304, 2024-04-25T09:14:54.360 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:14:54.360 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:14:54.360 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:14:54.360 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 6.8025410050000001, 2024-04-25T09:14:54.361 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 78, 2024-04-25T09:14:54.361 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 123, 2024-04-25T09:14:54.362 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 123, 2024-04-25T09:14:54.362 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 13.212982766 2024-04-25T09:14:54.362 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 -- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca2806b4d0 msgr2=0x7fca2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca2806b4d0 0x7fca2806d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fca3c034000 tx=0x7fca3c02e000 comp rx=0 tx=0).stop 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 -- 172.21.15.62:0/2407952437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca480d0fb0 msgr2=0x7fca4814af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca480d0fb0 0x7fca4814af40 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fca3400e9e0 tx=0x7fca3400eea0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 -- 172.21.15.62:0/2407952437 shutdown_connections 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fca2806b4d0 0x7fca2806d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca480d0fb0 0x7fca4814af40 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca48161230 0x7fca480c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 --2- 172.21.15.62:0/2407952437 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fca480b5a60 0x7fca48154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 -- 172.21.15.62:0/2407952437 >> 172.21.15.62:0/2407952437 conn(0x7fca48066d10 msgr2=0x7fca480b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 -- 172.21.15.62:0/2407952437 shutdown_connections 2024-04-25T09:14:54.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.367+0000 7fca4e200640 1 -- 172.21.15.62:0/2407952437 wait complete. 2024-04-25T09:14:54.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:14:54.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.573+0000 7f6f76e00640 1 Processor -- start 2024-04-25T09:14:54.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.574+0000 7f6f76e00640 1 -- start start 2024-04-25T09:14:54.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.575+0000 7f6f76e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6f700b6a90 0x7f6f700d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.575+0000 7f6f76e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6f700b27a0 0x7f6f700d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.575+0000 7f6f76e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f700d3110 con 0x7f6f700b56f0 2024-04-25T09:14:54.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.575+0000 7f6f76e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f700cedc0 con 0x7f6f700b27a0 2024-04-25T09:14:54.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.575+0000 7f6f76e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f700d33f0 con 0x7f6f700b6a90 2024-04-25T09:14:54.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.576+0000 7f6f76400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6f700b27a0 0x7f6f700d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.576+0000 7f6f76400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6f700b27a0 0x7f6f700d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34570/0 (socket says 172.21.15.62:34570) 2024-04-25T09:14:54.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.576+0000 7f6f76400640 1 -- 172.21.15.62:0/188656517 learned_addr learned my addr 172.21.15.62:0/188656517 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:54.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.576+0000 7f6f75000640 1 --2- 172.21.15.62:0/188656517 >> v2:172.21.15.121:3300/0 conn(0x7f6f700b6a90 0x7f6f700d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.576+0000 7f6f76400640 1 -- 172.21.15.62:0/188656517 >> v2:172.21.15.121:3300/0 conn(0x7f6f700b6a90 msgr2=0x7f6f700d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.576+0000 7f6f76400640 1 --2- 172.21.15.62:0/188656517 >> v2:172.21.15.121:3300/0 conn(0x7f6f700b6a90 0x7f6f700d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.576+0000 7f6f76400640 1 -- 172.21.15.62:0/188656517 >> v1:172.21.15.121:6789/0 conn(0x7f6f700b56f0 legacy=0x7f6f70160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:14:54.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.576+0000 7f6f76400640 1 -- 172.21.15.62:0/188656517 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f700d36d0 con 0x7f6f700b27a0 2024-04-25T09:14:54.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.577+0000 7f6f76400640 1 --2- 172.21.15.62:0/188656517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b27a0 0x7f6f700d2520 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6f7008c570 tx=0x7f6f6802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6011951944628f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.578+0000 7f6f6fe00640 1 -- 172.21.15.62:0/188656517 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68036070 con 0x7f6f700b27a0 2024-04-25T09:14:54.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.578+0000 7f6f6fe00640 1 -- 172.21.15.62:0/188656517 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f68031600 con 0x7f6f700b27a0 2024-04-25T09:14:54.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.578+0000 7f6f6fe00640 1 -- 172.21.15.62:0/188656517 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f680319d0 con 0x7f6f700b27a0 2024-04-25T09:14:54.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.578+0000 7f6f76e00640 1 -- 172.21.15.62:0/188656517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b27a0 msgr2=0x7f6f700d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.578+0000 7f6f76e00640 1 --2- 172.21.15.62:0/188656517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b27a0 0x7f6f700d2520 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6f7008c570 tx=0x7f6f6802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.579+0000 7f6f76e00640 1 -- 172.21.15.62:0/188656517 shutdown_connections 2024-04-25T09:14:54.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.579+0000 7f6f76e00640 1 --2- 172.21.15.62:0/188656517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b27a0 0x7f6f700d2520 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.579+0000 7f6f76e00640 1 --2- 172.21.15.62:0/188656517 >> v2:172.21.15.121:3300/0 conn(0x7f6f700b6a90 0x7f6f700d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.579+0000 7f6f76e00640 1 -- 172.21.15.62:0/188656517 >> 172.21.15.62:0/188656517 conn(0x7f6f70066d10 msgr2=0x7f6f70067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:54.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.579+0000 7f6f76e00640 1 -- 172.21.15.62:0/188656517 shutdown_connections 2024-04-25T09:14:54.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.579+0000 7f6f76e00640 1 -- 172.21.15.62:0/188656517 wait complete. 2024-04-25T09:14:54.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.581+0000 7f6f76e00640 1 Processor -- start 2024-04-25T09:14:54.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.581+0000 7f6f76e00640 1 -- start start 2024-04-25T09:14:54.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f76e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f700b27a0 0x7f6f7014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f76e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b56f0 0x7f6f700cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f76e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f700b6a90 0x7f6f700cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f76e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700b2ee0 con 0x7f6f700b56f0 2024-04-25T09:14:54.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f76e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700b2d60 con 0x7f6f700b27a0 2024-04-25T09:14:54.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f76e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700b5ad0 con 0x7f6f700b6a90 2024-04-25T09:14:54.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f75000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b56f0 0x7f6f700cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f75000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b56f0 0x7f6f700cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34574/0 (socket says 172.21.15.62:34574) 2024-04-25T09:14:54.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.582+0000 7f6f75000640 1 -- 172.21.15.62:0/2483262015 learned_addr learned my addr 172.21.15.62:0/2483262015 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:14:54.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.583+0000 7f6f75a00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f700b27a0 0x7f6f7014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.583+0000 7f6f76400640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f700b6a90 0x7f6f700cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.583+0000 7f6f75000640 1 -- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f700b6a90 msgr2=0x7f6f700cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.583+0000 7f6f75000640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f700b6a90 0x7f6f700cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.583+0000 7f6f75000640 1 -- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f700b27a0 msgr2=0x7f6f7014edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.583+0000 7f6f75000640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f700b27a0 0x7f6f7014edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.583+0000 7f6f75000640 1 -- 172.21.15.62:0/2483262015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f700cb900 con 0x7f6f700b56f0 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.583+0000 7f6f75000640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b56f0 0x7f6f700cd050 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00d5f0 tx=0x7f6f5c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.584+0000 7f6f6d600640 1 -- 172.21.15.62:0/2483262015 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0125f0 con 0x7f6f700b56f0 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.584+0000 7f6f6d600640 1 -- 172.21.15.62:0/2483262015 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c019070 con 0x7f6f700b56f0 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.584+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f700cbbe0 con 0x7f6f700b56f0 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.584+0000 7f6f6d600640 1 -- 172.21.15.62:0/2483262015 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0129d0 con 0x7f6f700b56f0 2024-04-25T09:14:54.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.584+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f700cc1a0 con 0x7f6f700b56f0 2024-04-25T09:14:54.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.585+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7004ef20 con 0x7f6f700b56f0 2024-04-25T09:14:54.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.586+0000 7f6f6d600640 1 -- 172.21.15.62:0/2483262015 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f5c012b70 con 0x7f6f700b56f0 2024-04-25T09:14:54.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.588+0000 7f6f6d600640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f5006b4d0 0x7f6f5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:14:54.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.588+0000 7f6f6d600640 1 -- 172.21.15.62:0/2483262015 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f5c08a640 con 0x7f6f700b56f0 2024-04-25T09:14:54.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.588+0000 7f6f75a00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f5006b4d0 0x7f6f5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:14:54.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.590+0000 7f6f75a00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f5006b4d0 0x7f6f5006d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6f60004ea0 tx=0x7f6f60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:14:54.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.593+0000 7f6f6d600640 1 -- 172.21.15.62:0/2483262015 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f5c059420 con 0x7f6f700b56f0 2024-04-25T09:14:54.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.832+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6f700ce580 con 0x7f6f700b56f0 2024-04-25T09:14:54.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.832+0000 7f6f6d600640 1 -- 172.21.15.62:0/2483262015 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6f5c058d20 con 0x7f6f700b56f0 2024-04-25T09:14:54.819 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:14:54.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f5006b4d0 msgr2=0x7f6f5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f5006b4d0 0x7f6f5006d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6f60004ea0 tx=0x7f6f60009040 comp rx=0 tx=0).stop 2024-04-25T09:14:54.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b56f0 msgr2=0x7f6f700cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b56f0 0x7f6f700cd050 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00d5f0 tx=0x7f6f5c00dab0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 shutdown_connections 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6f5006b4d0 0x7f6f5006d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f700b56f0 0x7f6f700cd050 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6f700b6a90 0x7f6f700cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 --2- 172.21.15.62:0/2483262015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f700b27a0 0x7f6f7014edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 >> 172.21.15.62:0/2483262015 conn(0x7f6f70066d10 msgr2=0x7f6f7015f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 shutdown_connections 2024-04-25T09:14:54.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:14:54.835+0000 7f6f76e00640 1 -- 172.21.15.62:0/2483262015 wait complete. 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:14:54.839 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:14:54.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:54.840 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:14:54.891 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41964sec preferred_lft 41964sec 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:14:54.892 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:14:54.893 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:14:54.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:54.894 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:14:54.973 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:14:54.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:54.977 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:14:55.037 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:14:55.037 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:14:55.037 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:14:55.038 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:14:55.038 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:14:55.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:55.039 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:14:55.039 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:55.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:14:55.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:14:55.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:14:55.039 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:55.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:55.199 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:14:55.251 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:14:55.251 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:14:55.252 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:14:55.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:55.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:14:55.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:14:55.331 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:14:55.385 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:14:55.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:14:55.386 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:14:55.437 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:14:55.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:55.438 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:14:55.572 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:14:55.573 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:14:55.611 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:14:55.611 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:14:55.611 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:14:55.611 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:14:55.611 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:14:55.614 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:14:55.614 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:14:55.614 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:14:55.614 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:14:55.614 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:14:55.617 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:14:55.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:55.617 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:14:55.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:55.696 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:14:55.759 INFO:teuthology.orchestra.run.smithi062.stdout:client6601 2024-04-25T09:14:55.759 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:14:55.760 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:14:55.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:55.809 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:55.810 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:55.810 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:14:55.810 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:55.810 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:55.810 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:55.810 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:55.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:55.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:55.927 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344771 s, 30.4 MB/s 2024-04-25T09:14:55.969 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:55.969 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:55.969 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369936 s, 28.3 MB/s 2024-04-25T09:14:56.002 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.002 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.003 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290277 s, 36.1 MB/s 2024-04-25T09:14:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.033 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271952 s, 38.6 MB/s 2024-04-25T09:14:56.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.074 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366368 s, 28.6 MB/s 2024-04-25T09:14:56.111 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.111 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.111 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338275 s, 31.0 MB/s 2024-04-25T09:14:56.142 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.142 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.142 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026115 s, 40.2 MB/s 2024-04-25T09:14:56.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.179 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315597 s, 33.2 MB/s 2024-04-25T09:14:56.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.223 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388813 s, 27.0 MB/s 2024-04-25T09:14:56.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.251 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235132 s, 44.6 MB/s 2024-04-25T09:14:56.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.276 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208982 s, 50.2 MB/s 2024-04-25T09:14:56.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.309 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302088 s, 34.7 MB/s 2024-04-25T09:14:56.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400507 s, 26.2 MB/s 2024-04-25T09:14:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223184 s, 47.0 MB/s 2024-04-25T09:14:56.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237302 s, 44.2 MB/s 2024-04-25T09:14:56.441 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.442 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289677 s, 36.2 MB/s 2024-04-25T09:14:56.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.468 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021653 s, 48.4 MB/s 2024-04-25T09:14:56.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.493 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220012 s, 47.7 MB/s 2024-04-25T09:14:56.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.518 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210638 s, 49.8 MB/s 2024-04-25T09:14:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.543 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020877 s, 50.2 MB/s 2024-04-25T09:14:56.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238242 s, 44.0 MB/s 2024-04-25T09:14:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.596 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213328 s, 49.2 MB/s 2024-04-25T09:14:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204884 s, 51.2 MB/s 2024-04-25T09:14:56.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.646 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225525 s, 46.5 MB/s 2024-04-25T09:14:56.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.673 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231091 s, 45.4 MB/s 2024-04-25T09:14:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257273 s, 40.8 MB/s 2024-04-25T09:14:56.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290423 s, 36.1 MB/s 2024-04-25T09:14:56.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.766 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250249 s, 41.9 MB/s 2024-04-25T09:14:56.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.795 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255823 s, 41.0 MB/s 2024-04-25T09:14:56.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257892 s, 40.7 MB/s 2024-04-25T09:14:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185071 s, 56.7 MB/s 2024-04-25T09:14:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.875 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239406 s, 43.8 MB/s 2024-04-25T09:14:56.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:56.878 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:56.878 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:56.878 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:14:56.878 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:56.878 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:56.878 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:56.878 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:56.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201142 s, 52.1 MB/s 2024-04-25T09:14:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214907 s, 48.8 MB/s 2024-04-25T09:14:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.020 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273598 s, 38.3 MB/s 2024-04-25T09:14:57.052 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.052 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.053 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280533 s, 37.4 MB/s 2024-04-25T09:14:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.081 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247586 s, 42.4 MB/s 2024-04-25T09:14:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.109 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237914 s, 44.1 MB/s 2024-04-25T09:14:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.136 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220809 s, 47.5 MB/s 2024-04-25T09:14:57.166 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.166 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.166 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257676 s, 40.7 MB/s 2024-04-25T09:14:57.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.196 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255887 s, 41.0 MB/s 2024-04-25T09:14:57.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.222 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213201 s, 49.2 MB/s 2024-04-25T09:14:57.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204978 s, 51.2 MB/s 2024-04-25T09:14:57.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.272 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212 s, 49.5 MB/s 2024-04-25T09:14:57.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.298 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219991 s, 47.7 MB/s 2024-04-25T09:14:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.329 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269464 s, 38.9 MB/s 2024-04-25T09:14:57.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.355 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214452 s, 48.9 MB/s 2024-04-25T09:14:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233291 s, 44.9 MB/s 2024-04-25T09:14:57.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.412 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243255 s, 43.1 MB/s 2024-04-25T09:14:57.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.440 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229711 s, 45.6 MB/s 2024-04-25T09:14:57.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.470 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.470 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273338 s, 38.4 MB/s 2024-04-25T09:14:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.499 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024762 s, 42.3 MB/s 2024-04-25T09:14:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.530 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257936 s, 40.7 MB/s 2024-04-25T09:14:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212049 s, 49.4 MB/s 2024-04-25T09:14:57.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208149 s, 50.4 MB/s 2024-04-25T09:14:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.605 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223184 s, 47.0 MB/s 2024-04-25T09:14:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.635 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246236 s, 42.6 MB/s 2024-04-25T09:14:57.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.660 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220289 s, 47.6 MB/s 2024-04-25T09:14:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.686 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210053 s, 49.9 MB/s 2024-04-25T09:14:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244469 s, 42.9 MB/s 2024-04-25T09:14:57.738 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.738 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.738 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020816 s, 50.4 MB/s 2024-04-25T09:14:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.761 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195161 s, 53.7 MB/s 2024-04-25T09:14:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.789 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224534 s, 46.7 MB/s 2024-04-25T09:14:57.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.812 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185258 s, 56.6 MB/s 2024-04-25T09:14:57.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:57.814 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:57.815 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:57.815 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:14:57.815 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:57.815 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:57.815 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:57.815 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245049 s, 42.8 MB/s 2024-04-25T09:14:57.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.933 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244202 s, 42.9 MB/s 2024-04-25T09:14:57.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232254 s, 45.1 MB/s 2024-04-25T09:14:57.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:57.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:57.988 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247141 s, 42.4 MB/s 2024-04-25T09:14:58.015 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.015 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.015 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237928 s, 44.1 MB/s 2024-04-25T09:14:58.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.043 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024715 s, 42.4 MB/s 2024-04-25T09:14:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.072 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024575 s, 42.7 MB/s 2024-04-25T09:14:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.099 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228437 s, 45.9 MB/s 2024-04-25T09:14:58.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.125 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217257 s, 48.3 MB/s 2024-04-25T09:14:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239992 s, 43.7 MB/s 2024-04-25T09:14:58.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219387 s, 47.8 MB/s 2024-04-25T09:14:58.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.209 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257265 s, 40.8 MB/s 2024-04-25T09:14:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.234 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209971 s, 49.9 MB/s 2024-04-25T09:14:58.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.261 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022297 s, 47.0 MB/s 2024-04-25T09:14:58.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.286 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214557 s, 48.9 MB/s 2024-04-25T09:14:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.312 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214072 s, 49.0 MB/s 2024-04-25T09:14:58.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.342 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263266 s, 39.8 MB/s 2024-04-25T09:14:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.367 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021058 s, 49.8 MB/s 2024-04-25T09:14:58.394 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.395 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220864 s, 47.5 MB/s 2024-04-25T09:14:58.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.419 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205125 s, 51.1 MB/s 2024-04-25T09:14:58.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.442 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193216 s, 54.3 MB/s 2024-04-25T09:14:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.465 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187652 s, 55.9 MB/s 2024-04-25T09:14:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.489 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203702 s, 51.5 MB/s 2024-04-25T09:14:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.516 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233288 s, 44.9 MB/s 2024-04-25T09:14:58.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.544 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225846 s, 46.4 MB/s 2024-04-25T09:14:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021142 s, 49.6 MB/s 2024-04-25T09:14:58.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.596 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021049 s, 49.8 MB/s 2024-04-25T09:14:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204945 s, 51.2 MB/s 2024-04-25T09:14:58.647 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.647 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.647 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231949 s, 45.2 MB/s 2024-04-25T09:14:58.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.673 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223005 s, 47.0 MB/s 2024-04-25T09:14:58.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.702 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227924 s, 46.0 MB/s 2024-04-25T09:14:58.728 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.728 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.728 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223975 s, 46.8 MB/s 2024-04-25T09:14:58.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:58.731 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:58.731 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:58.732 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:14:58.732 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:58.732 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:58.732 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:58.732 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218296 s, 48.0 MB/s 2024-04-25T09:14:58.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.846 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226015 s, 46.4 MB/s 2024-04-25T09:14:58.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.871 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194115 s, 54.0 MB/s 2024-04-25T09:14:58.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.898 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232263 s, 45.1 MB/s 2024-04-25T09:14:58.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.920 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192615 s, 54.4 MB/s 2024-04-25T09:14:58.948 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.948 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.948 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249086 s, 42.1 MB/s 2024-04-25T09:14:58.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:58.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:58.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277385 s, 37.8 MB/s 2024-04-25T09:14:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.008 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238022 s, 44.1 MB/s 2024-04-25T09:14:59.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.035 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232611 s, 45.1 MB/s 2024-04-25T09:14:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.064 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024449 s, 42.9 MB/s 2024-04-25T09:14:59.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.091 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023335 s, 44.9 MB/s 2024-04-25T09:14:59.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.118 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217353 s, 48.2 MB/s 2024-04-25T09:14:59.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.147 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.147 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250777 s, 41.8 MB/s 2024-04-25T09:14:59.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206308 s, 50.8 MB/s 2024-04-25T09:14:59.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211072 s, 49.7 MB/s 2024-04-25T09:14:59.219 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.220 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189516 s, 55.3 MB/s 2024-04-25T09:14:59.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.249 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250345 s, 41.9 MB/s 2024-04-25T09:14:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.273 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199314 s, 52.6 MB/s 2024-04-25T09:14:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.299 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220258 s, 47.6 MB/s 2024-04-25T09:14:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.323 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204646 s, 51.2 MB/s 2024-04-25T09:14:59.349 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.350 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223696 s, 46.9 MB/s 2024-04-25T09:14:59.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.376 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220985 s, 47.5 MB/s 2024-04-25T09:14:59.403 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.403 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.403 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225975 s, 46.4 MB/s 2024-04-25T09:14:59.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.434 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265805 s, 39.4 MB/s 2024-04-25T09:14:59.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251403 s, 41.7 MB/s 2024-04-25T09:14:59.488 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.489 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210881 s, 49.7 MB/s 2024-04-25T09:14:59.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.513 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020169 s, 52.0 MB/s 2024-04-25T09:14:59.537 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.537 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.538 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206413 s, 50.8 MB/s 2024-04-25T09:14:59.566 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.566 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.566 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230862 s, 45.4 MB/s 2024-04-25T09:14:59.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.592 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214052 s, 49.0 MB/s 2024-04-25T09:14:59.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227025 s, 46.2 MB/s 2024-04-25T09:14:59.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.640 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179402 s, 58.4 MB/s 2024-04-25T09:14:59.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:14:59.643 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:14:59.643 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:14:59.643 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:14:59.643 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:14:59.643 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:14:59.643 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:14:59.643 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:14:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192144 s, 54.6 MB/s 2024-04-25T09:14:59.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.756 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272136 s, 38.5 MB/s 2024-04-25T09:14:59.783 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.784 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232936 s, 45.0 MB/s 2024-04-25T09:14:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214825 s, 48.8 MB/s 2024-04-25T09:14:59.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.833 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194187 s, 54.0 MB/s 2024-04-25T09:14:59.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.856 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193125 s, 54.3 MB/s 2024-04-25T09:14:59.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.886 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257003 s, 40.8 MB/s 2024-04-25T09:14:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.910 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198501 s, 52.8 MB/s 2024-04-25T09:14:59.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.935 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211364 s, 49.6 MB/s 2024-04-25T09:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205239 s, 51.1 MB/s 2024-04-25T09:14:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:14:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:14:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244021 s, 43.0 MB/s 2024-04-25T09:15:00.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.016 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240001 s, 43.7 MB/s 2024-04-25T09:15:00.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.043 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227581 s, 46.1 MB/s 2024-04-25T09:15:00.068 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.068 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.068 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207051 s, 50.6 MB/s 2024-04-25T09:15:00.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.095 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228768 s, 45.8 MB/s 2024-04-25T09:15:00.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.118 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196916 s, 53.2 MB/s 2024-04-25T09:15:00.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.145 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229045 s, 45.8 MB/s 2024-04-25T09:15:00.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.171 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216063 s, 48.5 MB/s 2024-04-25T09:15:00.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.198 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227226 s, 46.1 MB/s 2024-04-25T09:15:00.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.231 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273298 s, 38.4 MB/s 2024-04-25T09:15:00.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.379 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227483 s, 46.1 MB/s 2024-04-25T09:15:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210222 s, 49.9 MB/s 2024-04-25T09:15:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227192 s, 46.2 MB/s 2024-04-25T09:15:00.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017972 s, 58.3 MB/s 2024-04-25T09:15:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218571 s, 48.0 MB/s 2024-04-25T09:15:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212174 s, 49.4 MB/s 2024-04-25T09:15:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241389 s, 43.4 MB/s 2024-04-25T09:15:00.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020456 s, 51.3 MB/s 2024-04-25T09:15:00.461 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.461 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.461 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021144 s, 49.6 MB/s 2024-04-25T09:15:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211204 s, 49.6 MB/s 2024-04-25T09:15:00.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.514 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230112 s, 45.6 MB/s 2024-04-25T09:15:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219977 s, 47.7 MB/s 2024-04-25T09:15:00.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:00.543 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:00.543 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:00.543 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:15:00.543 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:00.543 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:00.543 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:00.544 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:00.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241417 s, 43.4 MB/s 2024-04-25T09:15:00.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.669 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261118 s, 40.2 MB/s 2024-04-25T09:15:00.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.693 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205881 s, 50.9 MB/s 2024-04-25T09:15:00.718 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.718 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.718 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214433 s, 48.9 MB/s 2024-04-25T09:15:00.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025253 s, 41.5 MB/s 2024-04-25T09:15:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274709 s, 38.2 MB/s 2024-04-25T09:15:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.805 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221288 s, 47.4 MB/s 2024-04-25T09:15:00.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.830 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200877 s, 52.2 MB/s 2024-04-25T09:15:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025043 s, 41.9 MB/s 2024-04-25T09:15:00.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.885 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221245 s, 47.4 MB/s 2024-04-25T09:15:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.915 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251981 s, 41.6 MB/s 2024-04-25T09:15:00.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.939 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206333 s, 50.8 MB/s 2024-04-25T09:15:00.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233151 s, 45.0 MB/s 2024-04-25T09:15:00.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:00.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:00.995 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245584 s, 42.7 MB/s 2024-04-25T09:15:01.025 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.025 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.026 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026546 s, 39.5 MB/s 2024-04-25T09:15:01.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290446 s, 36.1 MB/s 2024-04-25T09:15:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.088 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244903 s, 42.8 MB/s 2024-04-25T09:15:01.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.117 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244478 s, 42.9 MB/s 2024-04-25T09:15:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.143 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222842 s, 47.1 MB/s 2024-04-25T09:15:01.169 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.169 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.169 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212705 s, 49.3 MB/s 2024-04-25T09:15:01.196 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.196 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.196 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211856 s, 49.5 MB/s 2024-04-25T09:15:01.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.226 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243338 s, 43.1 MB/s 2024-04-25T09:15:01.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.253 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022586 s, 46.4 MB/s 2024-04-25T09:15:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.279 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232701 s, 45.1 MB/s 2024-04-25T09:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.303 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195744 s, 53.6 MB/s 2024-04-25T09:15:01.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.324 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176453 s, 59.4 MB/s 2024-04-25T09:15:01.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.348 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210165 s, 49.9 MB/s 2024-04-25T09:15:01.372 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.372 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.372 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198121 s, 52.9 MB/s 2024-04-25T09:15:01.397 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.397 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.397 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222377 s, 47.2 MB/s 2024-04-25T09:15:01.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217305 s, 48.3 MB/s 2024-04-25T09:15:01.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.449 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217288 s, 48.3 MB/s 2024-04-25T09:15:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.477 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233314 s, 44.9 MB/s 2024-04-25T09:15:01.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:01.480 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:01.480 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:01.480 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:15:01.480 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:01.480 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:01.480 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:01.480 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:01.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.571 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235279 s, 44.6 MB/s 2024-04-25T09:15:01.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231071 s, 45.4 MB/s 2024-04-25T09:15:01.624 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.625 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225082 s, 46.6 MB/s 2024-04-25T09:15:01.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.649 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203867 s, 51.4 MB/s 2024-04-25T09:15:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.675 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218408 s, 48.0 MB/s 2024-04-25T09:15:01.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.702 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023635 s, 44.4 MB/s 2024-04-25T09:15:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.728 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205706 s, 51.0 MB/s 2024-04-25T09:15:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.756 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235404 s, 44.5 MB/s 2024-04-25T09:15:01.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.784 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237406 s, 44.2 MB/s 2024-04-25T09:15:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228531 s, 45.9 MB/s 2024-04-25T09:15:01.836 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.836 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.836 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201446 s, 52.1 MB/s 2024-04-25T09:15:01.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226107 s, 46.4 MB/s 2024-04-25T09:15:01.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.890 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237436 s, 44.2 MB/s 2024-04-25T09:15:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203832 s, 51.4 MB/s 2024-04-25T09:15:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.949 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298267 s, 35.2 MB/s 2024-04-25T09:15:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:01.977 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238944 s, 43.9 MB/s 2024-04-25T09:15:02.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.006 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248365 s, 42.2 MB/s 2024-04-25T09:15:02.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272147 s, 38.5 MB/s 2024-04-25T09:15:02.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.066 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025152 s, 41.7 MB/s 2024-04-25T09:15:02.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.096 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257775 s, 40.7 MB/s 2024-04-25T09:15:02.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232859 s, 45.0 MB/s 2024-04-25T09:15:02.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.151 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231282 s, 45.3 MB/s 2024-04-25T09:15:02.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.173 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188381 s, 55.7 MB/s 2024-04-25T09:15:02.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234199 s, 44.8 MB/s 2024-04-25T09:15:02.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.229 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236488 s, 44.3 MB/s 2024-04-25T09:15:02.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.254 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021191 s, 49.5 MB/s 2024-04-25T09:15:02.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.279 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208776 s, 50.2 MB/s 2024-04-25T09:15:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.306 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233257 s, 45.0 MB/s 2024-04-25T09:15:02.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241536 s, 43.4 MB/s 2024-04-25T09:15:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.362 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229415 s, 45.7 MB/s 2024-04-25T09:15:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.390 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235745 s, 44.5 MB/s 2024-04-25T09:15:02.417 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.418 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023661 s, 44.3 MB/s 2024-04-25T09:15:02.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:02.420 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:02.420 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:02.420 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:15:02.420 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:02.420 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:02.420 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:02.420 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:02.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.504 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222718 s, 47.1 MB/s 2024-04-25T09:15:02.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.531 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023007 s, 45.6 MB/s 2024-04-25T09:15:02.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.558 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230708 s, 45.5 MB/s 2024-04-25T09:15:02.587 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.587 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.587 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025505 s, 41.1 MB/s 2024-04-25T09:15:02.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.611 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020711 s, 50.6 MB/s 2024-04-25T09:15:02.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.638 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224704 s, 46.7 MB/s 2024-04-25T09:15:02.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.660 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178874 s, 58.6 MB/s 2024-04-25T09:15:02.687 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.687 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.687 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02312 s, 45.4 MB/s 2024-04-25T09:15:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.715 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244575 s, 42.9 MB/s 2024-04-25T09:15:02.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.744 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245064 s, 42.8 MB/s 2024-04-25T09:15:02.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.771 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228513 s, 45.9 MB/s 2024-04-25T09:15:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.791 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174482 s, 60.1 MB/s 2024-04-25T09:15:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.820 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.821 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241035 s, 43.5 MB/s 2024-04-25T09:15:02.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.845 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211497 s, 49.6 MB/s 2024-04-25T09:15:02.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.872 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232991 s, 45.0 MB/s 2024-04-25T09:15:02.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.901 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.901 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244624 s, 42.9 MB/s 2024-04-25T09:15:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211355 s, 49.6 MB/s 2024-04-25T09:15:02.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.958 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267735 s, 39.2 MB/s 2024-04-25T09:15:02.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:02.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:02.986 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234941 s, 44.6 MB/s 2024-04-25T09:15:03.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.010 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210496 s, 49.8 MB/s 2024-04-25T09:15:03.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.038 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241261 s, 43.5 MB/s 2024-04-25T09:15:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.066 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242509 s, 43.2 MB/s 2024-04-25T09:15:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.094 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237327 s, 44.2 MB/s 2024-04-25T09:15:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.127 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290106 s, 36.1 MB/s 2024-04-25T09:15:03.154 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.155 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.155 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023246 s, 45.1 MB/s 2024-04-25T09:15:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216389 s, 48.5 MB/s 2024-04-25T09:15:03.210 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.210 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.210 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257189 s, 40.8 MB/s 2024-04-25T09:15:03.236 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.236 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.236 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220263 s, 47.6 MB/s 2024-04-25T09:15:03.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.262 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215448 s, 48.7 MB/s 2024-04-25T09:15:03.285 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.285 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.286 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196422 s, 53.4 MB/s 2024-04-25T09:15:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.314 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235064 s, 44.6 MB/s 2024-04-25T09:15:03.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.340 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021959 s, 47.8 MB/s 2024-04-25T09:15:03.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:03.342 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:03.342 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:03.342 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:15:03.343 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:03.343 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:03.343 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:03.343 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:03.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247931 s, 42.3 MB/s 2024-04-25T09:15:03.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241207 s, 43.5 MB/s 2024-04-25T09:15:03.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.490 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221208 s, 47.4 MB/s 2024-04-25T09:15:03.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.514 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205884 s, 50.9 MB/s 2024-04-25T09:15:03.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.537 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189492 s, 55.3 MB/s 2024-04-25T09:15:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020169 s, 52.0 MB/s 2024-04-25T09:15:03.589 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.589 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.590 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248625 s, 42.2 MB/s 2024-04-25T09:15:03.616 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.616 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.616 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216212 s, 48.5 MB/s 2024-04-25T09:15:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.642 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021764 s, 48.2 MB/s 2024-04-25T09:15:03.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.671 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252169 s, 41.6 MB/s 2024-04-25T09:15:03.698 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.698 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.698 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223566 s, 46.9 MB/s 2024-04-25T09:15:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.730 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277608 s, 37.8 MB/s 2024-04-25T09:15:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294226 s, 35.6 MB/s 2024-04-25T09:15:03.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.795 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255162 s, 41.1 MB/s 2024-04-25T09:15:03.818 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.818 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.818 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205421 s, 51.0 MB/s 2024-04-25T09:15:03.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.845 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222449 s, 47.1 MB/s 2024-04-25T09:15:03.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.871 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225586 s, 46.5 MB/s 2024-04-25T09:15:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.898 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229203 s, 45.7 MB/s 2024-04-25T09:15:03.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.923 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.923 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021281 s, 49.3 MB/s 2024-04-25T09:15:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.955 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268956 s, 39.0 MB/s 2024-04-25T09:15:03.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:03.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:03.981 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222739 s, 47.1 MB/s 2024-04-25T09:15:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025013 s, 41.9 MB/s 2024-04-25T09:15:04.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229492 s, 45.7 MB/s 2024-04-25T09:15:04.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.066 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247549 s, 42.4 MB/s 2024-04-25T09:15:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.094 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238981 s, 43.9 MB/s 2024-04-25T09:15:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.120 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212683 s, 49.3 MB/s 2024-04-25T09:15:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214534 s, 48.9 MB/s 2024-04-25T09:15:04.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021665 s, 48.4 MB/s 2024-04-25T09:15:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.199 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214799 s, 48.8 MB/s 2024-04-25T09:15:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.226 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221933 s, 47.2 MB/s 2024-04-25T09:15:04.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.256 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246881 s, 42.5 MB/s 2024-04-25T09:15:04.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.284 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242659 s, 43.2 MB/s 2024-04-25T09:15:04.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:04.287 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:04.287 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:04.287 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:15:04.287 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:04.287 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:04.287 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:04.287 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:04.376 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.377 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226393 s, 46.3 MB/s 2024-04-25T09:15:04.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.404 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229854 s, 45.6 MB/s 2024-04-25T09:15:04.431 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.431 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.431 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233318 s, 44.9 MB/s 2024-04-25T09:15:04.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.459 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235668 s, 44.5 MB/s 2024-04-25T09:15:04.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233611 s, 44.9 MB/s 2024-04-25T09:15:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229658 s, 45.7 MB/s 2024-04-25T09:15:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.541 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226163 s, 46.4 MB/s 2024-04-25T09:15:04.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.564 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195728 s, 53.6 MB/s 2024-04-25T09:15:04.590 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.590 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.590 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218981 s, 47.9 MB/s 2024-04-25T09:15:04.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.617 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230816 s, 45.4 MB/s 2024-04-25T09:15:04.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.645 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240432 s, 43.6 MB/s 2024-04-25T09:15:04.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.669 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204311 s, 51.3 MB/s 2024-04-25T09:15:04.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239453 s, 43.8 MB/s 2024-04-25T09:15:04.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225655 s, 46.5 MB/s 2024-04-25T09:15:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.749 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209193 s, 50.1 MB/s 2024-04-25T09:15:04.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.775 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204552 s, 51.3 MB/s 2024-04-25T09:15:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265876 s, 39.4 MB/s 2024-04-25T09:15:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.836 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263411 s, 39.8 MB/s 2024-04-25T09:15:04.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023154 s, 45.3 MB/s 2024-04-25T09:15:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.888 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210293 s, 49.9 MB/s 2024-04-25T09:15:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.915 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232662 s, 45.1 MB/s 2024-04-25T09:15:04.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185681 s, 56.5 MB/s 2024-04-25T09:15:04.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235624 s, 44.5 MB/s 2024-04-25T09:15:04.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:04.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:04.994 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250695 s, 41.8 MB/s 2024-04-25T09:15:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.023 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242268 s, 43.3 MB/s 2024-04-25T09:15:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.051 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249503 s, 42.0 MB/s 2024-04-25T09:15:05.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.082 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259952 s, 40.3 MB/s 2024-04-25T09:15:05.108 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.108 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.108 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217013 s, 48.3 MB/s 2024-04-25T09:15:05.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.136 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235483 s, 44.5 MB/s 2024-04-25T09:15:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.167 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269532 s, 38.9 MB/s 2024-04-25T09:15:05.192 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.193 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217143 s, 48.3 MB/s 2024-04-25T09:15:05.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.223 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263334 s, 39.8 MB/s 2024-04-25T09:15:05.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:05.226 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:05.226 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:05.227 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:15:05.227 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:05.227 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:05.227 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:05.227 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:05.315 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.315 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.315 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225799 s, 46.4 MB/s 2024-04-25T09:15:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.342 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234455 s, 44.7 MB/s 2024-04-25T09:15:05.372 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.372 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.372 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263684 s, 39.8 MB/s 2024-04-25T09:15:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.402 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260468 s, 40.3 MB/s 2024-04-25T09:15:05.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.430 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237324 s, 44.2 MB/s 2024-04-25T09:15:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.456 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220531 s, 47.5 MB/s 2024-04-25T09:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.482 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217418 s, 48.2 MB/s 2024-04-25T09:15:05.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.509 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230814 s, 45.4 MB/s 2024-04-25T09:15:05.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.538 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251031 s, 41.8 MB/s 2024-04-25T09:15:05.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.564 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215437 s, 48.7 MB/s 2024-04-25T09:15:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.592 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024191 s, 43.3 MB/s 2024-04-25T09:15:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207046 s, 50.6 MB/s 2024-04-25T09:15:05.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181927 s, 57.6 MB/s 2024-04-25T09:15:05.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252932 s, 41.5 MB/s 2024-04-25T09:15:05.694 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.694 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.694 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021693 s, 48.3 MB/s 2024-04-25T09:15:05.719 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.719 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.719 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218865 s, 47.9 MB/s 2024-04-25T09:15:05.745 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.746 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022364 s, 46.9 MB/s 2024-04-25T09:15:05.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.773 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227943 s, 46.0 MB/s 2024-04-25T09:15:05.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.796 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198231 s, 52.9 MB/s 2024-04-25T09:15:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.823 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223493 s, 46.9 MB/s 2024-04-25T09:15:05.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.851 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238946 s, 43.9 MB/s 2024-04-25T09:15:05.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.877 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220962 s, 47.5 MB/s 2024-04-25T09:15:05.902 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.902 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.902 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206964 s, 50.7 MB/s 2024-04-25T09:15:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204084 s, 51.4 MB/s 2024-04-25T09:15:05.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.955 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247451 s, 42.4 MB/s 2024-04-25T09:15:05.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:05.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:05.985 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246912 s, 42.5 MB/s 2024-04-25T09:15:06.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.014 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247544 s, 42.4 MB/s 2024-04-25T09:15:06.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.042 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234342 s, 44.7 MB/s 2024-04-25T09:15:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.070 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237873 s, 44.1 MB/s 2024-04-25T09:15:06.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.096 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215326 s, 48.7 MB/s 2024-04-25T09:15:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.122 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227259 s, 46.1 MB/s 2024-04-25T09:15:06.155 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.156 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288037 s, 36.4 MB/s 2024-04-25T09:15:06.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:06.159 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:06.159 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:06.159 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:15:06.159 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:06.159 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:06.159 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:06.159 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.244 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185837 s, 56.4 MB/s 2024-04-25T09:15:06.266 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.267 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.267 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192655 s, 54.4 MB/s 2024-04-25T09:15:06.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.288 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018093 s, 58.0 MB/s 2024-04-25T09:15:06.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.315 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229954 s, 45.6 MB/s 2024-04-25T09:15:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.336 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180301 s, 58.2 MB/s 2024-04-25T09:15:06.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.364 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239691 s, 43.7 MB/s 2024-04-25T09:15:06.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216976 s, 48.3 MB/s 2024-04-25T09:15:06.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.416 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.416 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209306 s, 50.1 MB/s 2024-04-25T09:15:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.442 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218361 s, 48.0 MB/s 2024-04-25T09:15:06.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.469 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220253 s, 47.6 MB/s 2024-04-25T09:15:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.495 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224556 s, 46.7 MB/s 2024-04-25T09:15:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.520 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210799 s, 49.7 MB/s 2024-04-25T09:15:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195617 s, 53.6 MB/s 2024-04-25T09:15:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221833 s, 47.3 MB/s 2024-04-25T09:15:06.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.595 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200177 s, 52.4 MB/s 2024-04-25T09:15:06.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.618 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191903 s, 54.6 MB/s 2024-04-25T09:15:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.644 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218955 s, 47.9 MB/s 2024-04-25T09:15:06.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.669 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209272 s, 50.1 MB/s 2024-04-25T09:15:06.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.695 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215974 s, 48.6 MB/s 2024-04-25T09:15:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.720 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218277 s, 48.0 MB/s 2024-04-25T09:15:06.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.751 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258657 s, 40.5 MB/s 2024-04-25T09:15:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236692 s, 44.3 MB/s 2024-04-25T09:15:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.801 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189874 s, 55.2 MB/s 2024-04-25T09:15:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214612 s, 48.9 MB/s 2024-04-25T09:15:06.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.851 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207031 s, 50.6 MB/s 2024-04-25T09:15:06.880 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.880 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.880 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235191 s, 44.6 MB/s 2024-04-25T09:15:06.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236777 s, 44.3 MB/s 2024-04-25T09:15:06.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.936 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227012 s, 46.2 MB/s 2024-04-25T09:15:06.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.961 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020257 s, 51.8 MB/s 2024-04-25T09:15:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:06.992 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267956 s, 39.1 MB/s 2024-04-25T09:15:07.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.023 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269839 s, 38.9 MB/s 2024-04-25T09:15:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.055 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263023 s, 39.9 MB/s 2024-04-25T09:15:07.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:07.058 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:07.058 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:07.058 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:15:07.058 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:07.058 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:07.058 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:07.058 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:07.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.144 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216086 s, 48.5 MB/s 2024-04-25T09:15:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.171 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230612 s, 45.5 MB/s 2024-04-25T09:15:07.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.195 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205085 s, 51.1 MB/s 2024-04-25T09:15:07.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.221 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212786 s, 49.3 MB/s 2024-04-25T09:15:07.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.249 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230778 s, 45.4 MB/s 2024-04-25T09:15:07.271 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.271 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.271 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182173 s, 57.6 MB/s 2024-04-25T09:15:07.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.295 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199379 s, 52.6 MB/s 2024-04-25T09:15:07.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.319 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.319 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192977 s, 54.3 MB/s 2024-04-25T09:15:07.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.343 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205412 s, 51.0 MB/s 2024-04-25T09:15:07.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.367 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.367 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205419 s, 51.0 MB/s 2024-04-25T09:15:07.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.393 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223005 s, 47.0 MB/s 2024-04-25T09:15:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213709 s, 49.1 MB/s 2024-04-25T09:15:07.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.447 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229319 s, 45.7 MB/s 2024-04-25T09:15:07.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.473 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222452 s, 47.1 MB/s 2024-04-25T09:15:07.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.498 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211404 s, 49.6 MB/s 2024-04-25T09:15:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.524 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219356 s, 47.8 MB/s 2024-04-25T09:15:07.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.550 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021755 s, 48.2 MB/s 2024-04-25T09:15:07.572 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.572 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.572 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192895 s, 54.4 MB/s 2024-04-25T09:15:07.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213022 s, 49.2 MB/s 2024-04-25T09:15:07.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.622 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210058 s, 49.9 MB/s 2024-04-25T09:15:07.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.652 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248385 s, 42.2 MB/s 2024-04-25T09:15:07.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.679 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227506 s, 46.1 MB/s 2024-04-25T09:15:07.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.712 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284465 s, 36.9 MB/s 2024-04-25T09:15:07.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211009 s, 49.7 MB/s 2024-04-25T09:15:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.765 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242652 s, 43.2 MB/s 2024-04-25T09:15:07.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.790 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207295 s, 50.6 MB/s 2024-04-25T09:15:07.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.817 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219645 s, 47.7 MB/s 2024-04-25T09:15:07.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212451 s, 49.4 MB/s 2024-04-25T09:15:07.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186332 s, 56.3 MB/s 2024-04-25T09:15:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208215 s, 50.4 MB/s 2024-04-25T09:15:07.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01683 s, 62.3 MB/s 2024-04-25T09:15:07.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:07.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:07.938 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256435 s, 40.9 MB/s 2024-04-25T09:15:07.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:07.941 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:07.941 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:07.941 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:15:07.941 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:07.942 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:07.942 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:07.942 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260703 s, 40.2 MB/s 2024-04-25T09:15:08.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.061 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229465 s, 45.7 MB/s 2024-04-25T09:15:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227739 s, 46.0 MB/s 2024-04-25T09:15:08.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022515 s, 46.6 MB/s 2024-04-25T09:15:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245398 s, 42.7 MB/s 2024-04-25T09:15:08.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.173 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244639 s, 42.9 MB/s 2024-04-25T09:15:08.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238391 s, 44.0 MB/s 2024-04-25T09:15:08.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226823 s, 46.2 MB/s 2024-04-25T09:15:08.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.252 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206129 s, 50.9 MB/s 2024-04-25T09:15:08.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.280 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02284 s, 45.9 MB/s 2024-04-25T09:15:08.311 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.311 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.311 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263934 s, 39.7 MB/s 2024-04-25T09:15:08.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.336 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210246 s, 49.9 MB/s 2024-04-25T09:15:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.362 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214972 s, 48.8 MB/s 2024-04-25T09:15:08.388 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.388 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.388 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214966 s, 48.8 MB/s 2024-04-25T09:15:08.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.413 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214886 s, 48.8 MB/s 2024-04-25T09:15:08.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.439 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220176 s, 47.6 MB/s 2024-04-25T09:15:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.465 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221774 s, 47.3 MB/s 2024-04-25T09:15:08.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.491 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202817 s, 51.7 MB/s 2024-04-25T09:15:08.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.515 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207295 s, 50.6 MB/s 2024-04-25T09:15:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.538 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194094 s, 54.0 MB/s 2024-04-25T09:15:08.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.564 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217514 s, 48.2 MB/s 2024-04-25T09:15:08.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.592 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233959 s, 44.8 MB/s 2024-04-25T09:15:08.616 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.616 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.616 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204312 s, 51.3 MB/s 2024-04-25T09:15:08.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.642 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217756 s, 48.2 MB/s 2024-04-25T09:15:08.674 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.675 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248953 s, 42.1 MB/s 2024-04-25T09:15:08.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.696 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211378 s, 49.6 MB/s 2024-04-25T09:15:08.728 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.729 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028509 s, 36.8 MB/s 2024-04-25T09:15:08.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.759 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.759 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256529 s, 40.9 MB/s 2024-04-25T09:15:08.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.784 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216625 s, 48.4 MB/s 2024-04-25T09:15:08.812 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.812 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.812 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236816 s, 44.3 MB/s 2024-04-25T09:15:08.840 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.840 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235884 s, 44.5 MB/s 2024-04-25T09:15:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.872 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276793 s, 37.9 MB/s 2024-04-25T09:15:08.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:08.875 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:08.876 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:08.876 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:15:08.876 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:08.876 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:08.876 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:08.876 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:08.967 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.967 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.967 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250189 s, 41.9 MB/s 2024-04-25T09:15:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:08.994 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236684 s, 44.3 MB/s 2024-04-25T09:15:09.026 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.027 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285063 s, 36.8 MB/s 2024-04-25T09:15:09.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.050 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019192 s, 54.6 MB/s 2024-04-25T09:15:09.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.078 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239696 s, 43.7 MB/s 2024-04-25T09:15:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.105 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023238 s, 45.1 MB/s 2024-04-25T09:15:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197269 s, 53.2 MB/s 2024-04-25T09:15:09.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215901 s, 48.6 MB/s 2024-04-25T09:15:09.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.178 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210431 s, 49.8 MB/s 2024-04-25T09:15:09.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197922 s, 53.0 MB/s 2024-04-25T09:15:09.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.230 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024379 s, 43.0 MB/s 2024-04-25T09:15:09.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.255 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205592 s, 51.0 MB/s 2024-04-25T09:15:09.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.277 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182711 s, 57.4 MB/s 2024-04-25T09:15:09.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.303 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203459 s, 51.5 MB/s 2024-04-25T09:15:09.327 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.327 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.327 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209124 s, 50.1 MB/s 2024-04-25T09:15:09.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207685 s, 50.5 MB/s 2024-04-25T09:15:09.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249629 s, 42.0 MB/s 2024-04-25T09:15:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.408 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216043 s, 48.5 MB/s 2024-04-25T09:15:09.431 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.432 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202937 s, 51.7 MB/s 2024-04-25T09:15:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239375 s, 43.8 MB/s 2024-04-25T09:15:09.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.491 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256907 s, 40.8 MB/s 2024-04-25T09:15:09.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.517 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224746 s, 46.7 MB/s 2024-04-25T09:15:09.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.544 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024089 s, 43.5 MB/s 2024-04-25T09:15:09.572 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.572 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.572 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233912 s, 44.8 MB/s 2024-04-25T09:15:09.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.595 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191046 s, 54.9 MB/s 2024-04-25T09:15:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.618 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192984 s, 54.3 MB/s 2024-04-25T09:15:09.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.646 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244695 s, 42.9 MB/s 2024-04-25T09:15:09.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.673 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217376 s, 48.2 MB/s 2024-04-25T09:15:09.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.702 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257534 s, 40.7 MB/s 2024-04-25T09:15:09.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.731 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236395 s, 44.4 MB/s 2024-04-25T09:15:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.755 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020616 s, 50.9 MB/s 2024-04-25T09:15:09.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.782 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.782 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224005 s, 46.8 MB/s 2024-04-25T09:15:09.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:09.784 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:09.784 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:09.785 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:15:09.785 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:09.785 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:09.785 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:09.785 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:09.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199265 s, 52.6 MB/s 2024-04-25T09:15:09.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.892 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198749 s, 52.8 MB/s 2024-04-25T09:15:09.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.922 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024667 s, 42.5 MB/s 2024-04-25T09:15:09.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244814 s, 42.8 MB/s 2024-04-25T09:15:09.979 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:09.979 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:09.979 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023846 s, 44.0 MB/s 2024-04-25T09:15:10.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.006 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223888 s, 46.8 MB/s 2024-04-25T09:15:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216402 s, 48.5 MB/s 2024-04-25T09:15:10.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.060 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243649 s, 43.0 MB/s 2024-04-25T09:15:10.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.088 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231637 s, 45.3 MB/s 2024-04-25T09:15:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.117 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237099 s, 44.2 MB/s 2024-04-25T09:15:10.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273642 s, 38.3 MB/s 2024-04-25T09:15:10.174 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.174 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.174 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215396 s, 48.7 MB/s 2024-04-25T09:15:10.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.204 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246695 s, 42.5 MB/s 2024-04-25T09:15:10.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.234 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259067 s, 40.5 MB/s 2024-04-25T09:15:10.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.261 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222814 s, 47.1 MB/s 2024-04-25T09:15:10.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248494 s, 42.2 MB/s 2024-04-25T09:15:10.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.314 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211991 s, 49.5 MB/s 2024-04-25T09:15:10.344 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.345 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262959 s, 39.9 MB/s 2024-04-25T09:15:10.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.374 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251133 s, 41.8 MB/s 2024-04-25T09:15:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.401 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229654 s, 45.7 MB/s 2024-04-25T09:15:10.428 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.428 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.428 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227191 s, 46.2 MB/s 2024-04-25T09:15:10.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.455 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233168 s, 45.0 MB/s 2024-04-25T09:15:10.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.484 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254843 s, 41.1 MB/s 2024-04-25T09:15:10.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.509 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208483 s, 50.3 MB/s 2024-04-25T09:15:10.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.535 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214915 s, 48.8 MB/s 2024-04-25T09:15:10.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.559 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201166 s, 52.1 MB/s 2024-04-25T09:15:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250427 s, 41.9 MB/s 2024-04-25T09:15:10.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.618 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248728 s, 42.2 MB/s 2024-04-25T09:15:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211741 s, 49.5 MB/s 2024-04-25T09:15:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.669 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216814 s, 48.4 MB/s 2024-04-25T09:15:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.696 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229089 s, 45.8 MB/s 2024-04-25T09:15:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231293 s, 45.3 MB/s 2024-04-25T09:15:10.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:10.727 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:10.727 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:10.727 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:15:10.727 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:10.727 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:10.727 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:10.727 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:10.813 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.814 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213215 s, 49.2 MB/s 2024-04-25T09:15:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195941 s, 53.5 MB/s 2024-04-25T09:15:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222148 s, 47.2 MB/s 2024-04-25T09:15:10.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200987 s, 52.2 MB/s 2024-04-25T09:15:10.912 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.912 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.912 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209564 s, 50.0 MB/s 2024-04-25T09:15:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.935 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189818 s, 55.2 MB/s 2024-04-25T09:15:10.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.963 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230049 s, 45.6 MB/s 2024-04-25T09:15:10.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:10.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:10.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242631 s, 43.2 MB/s 2024-04-25T09:15:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.017 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226381 s, 46.3 MB/s 2024-04-25T09:15:11.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.047 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.047 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246826 s, 42.5 MB/s 2024-04-25T09:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.073 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223542 s, 46.9 MB/s 2024-04-25T09:15:11.100 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.100 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.100 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225962 s, 46.4 MB/s 2024-04-25T09:15:11.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.138 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.138 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293174 s, 35.8 MB/s 2024-04-25T09:15:11.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.163 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246512 s, 42.5 MB/s 2024-04-25T09:15:11.185 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.186 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.186 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198182 s, 52.9 MB/s 2024-04-25T09:15:11.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.207 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186048 s, 56.4 MB/s 2024-04-25T09:15:11.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.233 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225094 s, 46.6 MB/s 2024-04-25T09:15:11.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.255 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018174 s, 57.7 MB/s 2024-04-25T09:15:11.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.277 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191937 s, 54.6 MB/s 2024-04-25T09:15:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.303 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224501 s, 46.7 MB/s 2024-04-25T09:15:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.329 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217408 s, 48.2 MB/s 2024-04-25T09:15:11.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.355 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225164 s, 46.6 MB/s 2024-04-25T09:15:11.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222714 s, 47.1 MB/s 2024-04-25T09:15:11.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.407 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212439 s, 49.4 MB/s 2024-04-25T09:15:11.430 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.430 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.430 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192424 s, 54.5 MB/s 2024-04-25T09:15:11.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.455 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205782 s, 51.0 MB/s 2024-04-25T09:15:11.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.482 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222656 s, 47.1 MB/s 2024-04-25T09:15:11.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.503 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179402 s, 58.4 MB/s 2024-04-25T09:15:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227063 s, 46.2 MB/s 2024-04-25T09:15:11.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.556 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.556 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222422 s, 47.1 MB/s 2024-04-25T09:15:11.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.585 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247721 s, 42.3 MB/s 2024-04-25T09:15:11.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.612 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237105 s, 44.2 MB/s 2024-04-25T09:15:11.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:11.615 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:11.615 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:11.615 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:15:11.615 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:11.615 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:11.615 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:11.615 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:11.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.702 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212142 s, 49.4 MB/s 2024-04-25T09:15:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.727 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214976 s, 48.8 MB/s 2024-04-25T09:15:11.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.756 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233876 s, 44.8 MB/s 2024-04-25T09:15:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.782 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215267 s, 48.7 MB/s 2024-04-25T09:15:11.808 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.808 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.808 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021639 s, 48.5 MB/s 2024-04-25T09:15:11.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.832 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198042 s, 52.9 MB/s 2024-04-25T09:15:11.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218479 s, 48.0 MB/s 2024-04-25T09:15:11.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232233 s, 45.2 MB/s 2024-04-25T09:15:11.912 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.913 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021733 s, 48.2 MB/s 2024-04-25T09:15:11.941 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.941 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.941 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235298 s, 44.6 MB/s 2024-04-25T09:15:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204612 s, 51.2 MB/s 2024-04-25T09:15:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:11.999 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268709 s, 39.0 MB/s 2024-04-25T09:15:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.030 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02802 s, 37.4 MB/s 2024-04-25T09:15:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.061 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269534 s, 38.9 MB/s 2024-04-25T09:15:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.090 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247089 s, 42.4 MB/s 2024-04-25T09:15:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266965 s, 39.3 MB/s 2024-04-25T09:15:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230384 s, 45.5 MB/s 2024-04-25T09:15:12.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.181 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289649 s, 36.2 MB/s 2024-04-25T09:15:12.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.209 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242594 s, 43.2 MB/s 2024-04-25T09:15:12.235 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.235 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.236 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224301 s, 46.7 MB/s 2024-04-25T09:15:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.262 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227303 s, 46.1 MB/s 2024-04-25T09:15:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216311 s, 48.5 MB/s 2024-04-25T09:15:12.317 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.317 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.317 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242831 s, 43.2 MB/s 2024-04-25T09:15:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223641 s, 46.9 MB/s 2024-04-25T09:15:12.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.371 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231504 s, 45.3 MB/s 2024-04-25T09:15:12.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.398 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211903 s, 49.5 MB/s 2024-04-25T09:15:12.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.426 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227642 s, 46.1 MB/s 2024-04-25T09:15:12.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226703 s, 46.3 MB/s 2024-04-25T09:15:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.479 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215824 s, 48.6 MB/s 2024-04-25T09:15:12.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.506 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221077 s, 47.4 MB/s 2024-04-25T09:15:12.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.530 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019651 s, 53.4 MB/s 2024-04-25T09:15:12.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.557 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227193 s, 46.2 MB/s 2024-04-25T09:15:12.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:12.560 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:12.560 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:12.560 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:15:12.560 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:12.560 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:12.560 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:12.560 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.645 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022101 s, 47.4 MB/s 2024-04-25T09:15:12.674 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025588 s, 41.0 MB/s 2024-04-25T09:15:12.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.707 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254097 s, 41.3 MB/s 2024-04-25T09:15:12.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.733 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247709 s, 42.3 MB/s 2024-04-25T09:15:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024014 s, 43.7 MB/s 2024-04-25T09:15:12.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.789 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233452 s, 44.9 MB/s 2024-04-25T09:15:12.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.814 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216076 s, 48.5 MB/s 2024-04-25T09:15:12.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.837 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190952 s, 54.9 MB/s 2024-04-25T09:15:12.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222462 s, 47.1 MB/s 2024-04-25T09:15:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.892 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237421 s, 44.2 MB/s 2024-04-25T09:15:12.916 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.916 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.916 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197563 s, 53.1 MB/s 2024-04-25T09:15:12.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.945 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251955 s, 41.6 MB/s 2024-04-25T09:15:12.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:12.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:12.972 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228361 s, 45.9 MB/s 2024-04-25T09:15:13.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.004 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273156 s, 38.4 MB/s 2024-04-25T09:15:13.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.031 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.031 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226765 s, 46.2 MB/s 2024-04-25T09:15:13.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.057 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223453 s, 46.9 MB/s 2024-04-25T09:15:13.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.084 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232057 s, 45.2 MB/s 2024-04-25T09:15:13.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.109 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212322 s, 49.4 MB/s 2024-04-25T09:15:13.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.134 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208456 s, 50.3 MB/s 2024-04-25T09:15:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227114 s, 46.2 MB/s 2024-04-25T09:15:13.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.187 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216563 s, 48.4 MB/s 2024-04-25T09:15:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.219 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277998 s, 37.7 MB/s 2024-04-25T09:15:13.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.242 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.242 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193436 s, 54.2 MB/s 2024-04-25T09:15:13.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.268 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217367 s, 48.2 MB/s 2024-04-25T09:15:13.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.296 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240424 s, 43.6 MB/s 2024-04-25T09:15:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.320 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205677 s, 51.0 MB/s 2024-04-25T09:15:13.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.347 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023395 s, 44.8 MB/s 2024-04-25T09:15:13.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.371 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193458 s, 54.2 MB/s 2024-04-25T09:15:13.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.398 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234583 s, 44.7 MB/s 2024-04-25T09:15:13.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.425 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217404 s, 48.2 MB/s 2024-04-25T09:15:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022709 s, 46.2 MB/s 2024-04-25T09:15:13.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.478 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218307 s, 48.0 MB/s 2024-04-25T09:15:13.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:13.481 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:13.481 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:13.481 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:15:13.481 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:13.481 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:13.481 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:13.481 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:13.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226283 s, 46.3 MB/s 2024-04-25T09:15:13.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.596 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021286 s, 49.3 MB/s 2024-04-25T09:15:13.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.621 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.622 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216489 s, 48.4 MB/s 2024-04-25T09:15:13.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.646 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.646 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219867 s, 47.7 MB/s 2024-04-25T09:15:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214127 s, 49.0 MB/s 2024-04-25T09:15:13.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.702 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253132 s, 41.4 MB/s 2024-04-25T09:15:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.733 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261992 s, 40.0 MB/s 2024-04-25T09:15:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.760 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232107 s, 45.2 MB/s 2024-04-25T09:15:13.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.790 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257425 s, 40.7 MB/s 2024-04-25T09:15:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243656 s, 43.0 MB/s 2024-04-25T09:15:13.844 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.844 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.844 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211907 s, 49.5 MB/s 2024-04-25T09:15:13.874 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.874 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.874 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260952 s, 40.2 MB/s 2024-04-25T09:15:13.901 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.901 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.901 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226744 s, 46.2 MB/s 2024-04-25T09:15:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217212 s, 48.3 MB/s 2024-04-25T09:15:13.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.953 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225217 s, 46.6 MB/s 2024-04-25T09:15:13.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:13.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:13.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233368 s, 44.9 MB/s 2024-04-25T09:15:14.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.008 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236558 s, 44.3 MB/s 2024-04-25T09:15:14.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.036 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.036 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224661 s, 46.7 MB/s 2024-04-25T09:15:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212587 s, 49.3 MB/s 2024-04-25T09:15:14.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219868 s, 47.7 MB/s 2024-04-25T09:15:14.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247219 s, 42.4 MB/s 2024-04-25T09:15:14.141 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.141 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.141 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209076 s, 50.2 MB/s 2024-04-25T09:15:14.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.165 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.165 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198687 s, 52.8 MB/s 2024-04-25T09:15:14.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.191 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218651 s, 48.0 MB/s 2024-04-25T09:15:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.218 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224944 s, 46.6 MB/s 2024-04-25T09:15:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.245 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226754 s, 46.2 MB/s 2024-04-25T09:15:14.274 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.274 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.274 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024395 s, 43.0 MB/s 2024-04-25T09:15:14.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.299 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213084 s, 49.2 MB/s 2024-04-25T09:15:14.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.326 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219687 s, 47.7 MB/s 2024-04-25T09:15:14.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.352 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226226 s, 46.4 MB/s 2024-04-25T09:15:14.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.383 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262734 s, 39.9 MB/s 2024-04-25T09:15:14.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.413 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250035 s, 41.9 MB/s 2024-04-25T09:15:14.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:14.415 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:14.415 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:14.415 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:15:14.416 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:14.416 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:14.416 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:14.416 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.500 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218902 s, 47.9 MB/s 2024-04-25T09:15:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223584 s, 46.9 MB/s 2024-04-25T09:15:14.552 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.552 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.552 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203381 s, 51.6 MB/s 2024-04-25T09:15:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229937 s, 45.6 MB/s 2024-04-25T09:15:14.603 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.604 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206964 s, 50.7 MB/s 2024-04-25T09:15:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204564 s, 51.3 MB/s 2024-04-25T09:15:14.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224961 s, 46.6 MB/s 2024-04-25T09:15:14.682 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.682 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.682 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210488 s, 49.8 MB/s 2024-04-25T09:15:14.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.707 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205015 s, 51.1 MB/s 2024-04-25T09:15:14.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.733 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207453 s, 50.5 MB/s 2024-04-25T09:15:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210499 s, 49.8 MB/s 2024-04-25T09:15:14.780 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.781 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187306 s, 56.0 MB/s 2024-04-25T09:15:14.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.808 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215073 s, 48.8 MB/s 2024-04-25T09:15:14.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.833 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211616 s, 49.6 MB/s 2024-04-25T09:15:14.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.858 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214545 s, 48.9 MB/s 2024-04-25T09:15:14.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288148 s, 36.4 MB/s 2024-04-25T09:15:14.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.918 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226358 s, 46.3 MB/s 2024-04-25T09:15:14.941 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.941 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.941 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188696 s, 55.6 MB/s 2024-04-25T09:15:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263593 s, 39.8 MB/s 2024-04-25T09:15:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:14.999 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240949 s, 43.5 MB/s 2024-04-25T09:15:15.012 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:15:15.021 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:15:15.025 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.025 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.025 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217021 s, 48.3 MB/s 2024-04-25T09:15:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205666 s, 51.0 MB/s 2024-04-25T09:15:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.068 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158353 s, 66.2 MB/s 2024-04-25T09:15:15.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023521 s, 44.6 MB/s 2024-04-25T09:15:15.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.121 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231262 s, 45.3 MB/s 2024-04-25T09:15:15.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.144 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189018 s, 55.5 MB/s 2024-04-25T09:15:15.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.171 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220888 s, 47.5 MB/s 2024-04-25T09:15:15.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211741 s, 49.5 MB/s 2024-04-25T09:15:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259953 s, 40.3 MB/s 2024-04-25T09:15:15.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.258 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268578 s, 39.0 MB/s 2024-04-25T09:15:15.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.285 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.285 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227114 s, 46.2 MB/s 2024-04-25T09:15:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.307 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195116 s, 53.7 MB/s 2024-04-25T09:15:15.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:15.310 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:15.310 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:15.310 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:15:15.311 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:15.311 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:15.311 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:15.311 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.451 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270853 s, 38.7 MB/s 2024-04-25T09:15:15.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.475 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202217 s, 51.9 MB/s 2024-04-25T09:15:15.500 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.500 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.500 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219065 s, 47.9 MB/s 2024-04-25T09:15:15.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.526 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217655 s, 48.2 MB/s 2024-04-25T09:15:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.550 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196919 s, 53.2 MB/s 2024-04-25T09:15:15.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.581 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.581 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261854 s, 40.0 MB/s 2024-04-25T09:15:15.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.607 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211656 s, 49.5 MB/s 2024-04-25T09:15:15.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.634 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209853 s, 50.0 MB/s 2024-04-25T09:15:15.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202819 s, 51.7 MB/s 2024-04-25T09:15:15.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.688 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261769 s, 40.1 MB/s 2024-04-25T09:15:15.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.714 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224274 s, 46.8 MB/s 2024-04-25T09:15:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.740 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219678 s, 47.7 MB/s 2024-04-25T09:15:15.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.772 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026174 s, 40.1 MB/s 2024-04-25T09:15:15.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.799 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.799 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022313 s, 47.0 MB/s 2024-04-25T09:15:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.827 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242119 s, 43.3 MB/s 2024-04-25T09:15:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210804 s, 49.7 MB/s 2024-04-25T09:15:15.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250968 s, 41.8 MB/s 2024-04-25T09:15:15.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.909 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229861 s, 45.6 MB/s 2024-04-25T09:15:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.934 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214068 s, 49.0 MB/s 2024-04-25T09:15:15.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259111 s, 40.5 MB/s 2024-04-25T09:15:15.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:15.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:15.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217608 s, 48.2 MB/s 2024-04-25T09:15:16.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.019 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245455 s, 42.7 MB/s 2024-04-25T09:15:16.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.048 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255415 s, 41.1 MB/s 2024-04-25T09:15:16.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.079 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025944 s, 40.4 MB/s 2024-04-25T09:15:16.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.106 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236475 s, 44.3 MB/s 2024-04-25T09:15:16.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.136 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259485 s, 40.4 MB/s 2024-04-25T09:15:16.169 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.169 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.169 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274338 s, 38.2 MB/s 2024-04-25T09:15:16.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217487 s, 48.2 MB/s 2024-04-25T09:15:16.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.225 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258596 s, 40.5 MB/s 2024-04-25T09:15:16.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.250 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211186 s, 49.7 MB/s 2024-04-25T09:15:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.272 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184464 s, 56.8 MB/s 2024-04-25T09:15:16.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.292 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01647 s, 63.7 MB/s 2024-04-25T09:15:16.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:16.294 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:16.294 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:16.294 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:15:16.294 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:16.294 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:16.294 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:16.295 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:16.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.369 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170607 s, 61.5 MB/s 2024-04-25T09:15:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.393 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020735 s, 50.6 MB/s 2024-04-25T09:15:16.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.419 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226052 s, 46.4 MB/s 2024-04-25T09:15:16.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.446 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226461 s, 46.3 MB/s 2024-04-25T09:15:16.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.473 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227822 s, 46.0 MB/s 2024-04-25T09:15:16.500 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.500 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.500 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225313 s, 46.5 MB/s 2024-04-25T09:15:16.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.524 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205425 s, 51.0 MB/s 2024-04-25T09:15:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.549 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219051 s, 47.9 MB/s 2024-04-25T09:15:16.577 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.578 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229636 s, 45.7 MB/s 2024-04-25T09:15:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.605 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218289 s, 48.0 MB/s 2024-04-25T09:15:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.634 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233633 s, 44.9 MB/s 2024-04-25T09:15:16.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.659 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213078 s, 49.2 MB/s 2024-04-25T09:15:16.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.683 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201291 s, 52.1 MB/s 2024-04-25T09:15:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.708 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197918 s, 53.0 MB/s 2024-04-25T09:15:16.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.733 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211713 s, 49.5 MB/s 2024-04-25T09:15:16.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.758 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216187 s, 48.5 MB/s 2024-04-25T09:15:16.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.788 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256517 s, 40.9 MB/s 2024-04-25T09:15:16.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.814 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215993 s, 48.5 MB/s 2024-04-25T09:15:16.839 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.839 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.839 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198543 s, 52.8 MB/s 2024-04-25T09:15:16.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249956 s, 42.0 MB/s 2024-04-25T09:15:16.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233891 s, 44.8 MB/s 2024-04-25T09:15:16.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.923 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219855 s, 47.7 MB/s 2024-04-25T09:15:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.953 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263589 s, 39.8 MB/s 2024-04-25T09:15:16.978 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:16.978 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:16.978 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207585 s, 50.5 MB/s 2024-04-25T09:15:17.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.006 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238221 s, 44.0 MB/s 2024-04-25T09:15:17.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.034 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233196 s, 45.0 MB/s 2024-04-25T09:15:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.060 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226962 s, 46.2 MB/s 2024-04-25T09:15:17.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.086 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021971 s, 47.7 MB/s 2024-04-25T09:15:17.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.115 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251765 s, 41.6 MB/s 2024-04-25T09:15:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.143 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223783 s, 46.9 MB/s 2024-04-25T09:15:17.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247936 s, 42.3 MB/s 2024-04-25T09:15:17.196 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204014 s, 51.4 MB/s 2024-04-25T09:15:17.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:17.199 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:17.199 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:17.199 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:15:17.199 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:17.200 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:17.200 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:17.200 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:17.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.293 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269506 s, 38.9 MB/s 2024-04-25T09:15:17.319 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.319 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.319 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02215 s, 47.3 MB/s 2024-04-25T09:15:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257402 s, 40.7 MB/s 2024-04-25T09:15:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.373 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197448 s, 53.1 MB/s 2024-04-25T09:15:17.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.399 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216964 s, 48.3 MB/s 2024-04-25T09:15:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205521 s, 51.0 MB/s 2024-04-25T09:15:17.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252736 s, 41.5 MB/s 2024-04-25T09:15:17.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.479 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218787 s, 47.9 MB/s 2024-04-25T09:15:17.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.504 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201671 s, 52.0 MB/s 2024-04-25T09:15:17.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.529 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208189 s, 50.4 MB/s 2024-04-25T09:15:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210416 s, 49.8 MB/s 2024-04-25T09:15:17.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246524 s, 42.5 MB/s 2024-04-25T09:15:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218238 s, 48.0 MB/s 2024-04-25T09:15:17.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.631 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178017 s, 58.9 MB/s 2024-04-25T09:15:17.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225156 s, 46.6 MB/s 2024-04-25T09:15:17.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.683 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214874 s, 48.8 MB/s 2024-04-25T09:15:17.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.708 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204946 s, 51.2 MB/s 2024-04-25T09:15:17.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.731 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019807 s, 52.9 MB/s 2024-04-25T09:15:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210535 s, 49.8 MB/s 2024-04-25T09:15:17.777 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.778 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.778 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191114 s, 54.9 MB/s 2024-04-25T09:15:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.802 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201905 s, 51.9 MB/s 2024-04-25T09:15:17.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.828 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208669 s, 50.3 MB/s 2024-04-25T09:15:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238104 s, 44.0 MB/s 2024-04-25T09:15:17.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.883 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211301 s, 49.6 MB/s 2024-04-25T09:15:17.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.906 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216653 s, 48.4 MB/s 2024-04-25T09:15:17.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.938 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275526 s, 38.1 MB/s 2024-04-25T09:15:17.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228833 s, 45.8 MB/s 2024-04-25T09:15:17.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:17.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:17.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240428 s, 43.6 MB/s 2024-04-25T09:15:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.024 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270302 s, 38.8 MB/s 2024-04-25T09:15:18.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.050 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222781 s, 47.1 MB/s 2024-04-25T09:15:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.077 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225432 s, 46.5 MB/s 2024-04-25T09:15:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244263 s, 42.9 MB/s 2024-04-25T09:15:18.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:18.109 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:18.109 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:18.109 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:15:18.110 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:18.110 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:18.110 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:18.110 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:18.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244541 s, 42.9 MB/s 2024-04-25T09:15:18.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.225 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234935 s, 44.6 MB/s 2024-04-25T09:15:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.245 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017185 s, 61.0 MB/s 2024-04-25T09:15:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.272 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224204 s, 46.8 MB/s 2024-04-25T09:15:18.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.301 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254002 s, 41.3 MB/s 2024-04-25T09:15:18.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.332 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251023 s, 41.8 MB/s 2024-04-25T09:15:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248141 s, 42.3 MB/s 2024-04-25T09:15:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.389 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.390 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250521 s, 41.9 MB/s 2024-04-25T09:15:18.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.415 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216586 s, 48.4 MB/s 2024-04-25T09:15:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.443 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024027 s, 43.6 MB/s 2024-04-25T09:15:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.470 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224905 s, 46.6 MB/s 2024-04-25T09:15:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.497 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229202 s, 45.7 MB/s 2024-04-25T09:15:18.525 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.526 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247815 s, 42.3 MB/s 2024-04-25T09:15:18.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.554 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251074 s, 41.8 MB/s 2024-04-25T09:15:18.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.585 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262847 s, 39.9 MB/s 2024-04-25T09:15:18.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.609 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204685 s, 51.2 MB/s 2024-04-25T09:15:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.634 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211427 s, 49.6 MB/s 2024-04-25T09:15:18.662 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.662 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.663 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244408 s, 42.9 MB/s 2024-04-25T09:15:18.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.688 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218348 s, 48.0 MB/s 2024-04-25T09:15:18.715 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.715 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.715 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223243 s, 47.0 MB/s 2024-04-25T09:15:18.744 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.744 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.744 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250393 s, 41.9 MB/s 2024-04-25T09:15:18.770 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.770 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.770 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220929 s, 47.5 MB/s 2024-04-25T09:15:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.795 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01967 s, 53.3 MB/s 2024-04-25T09:15:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.823 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239542 s, 43.8 MB/s 2024-04-25T09:15:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.849 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223216 s, 47.0 MB/s 2024-04-25T09:15:18.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.878 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251917 s, 41.6 MB/s 2024-04-25T09:15:18.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.907 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238092 s, 44.0 MB/s 2024-04-25T09:15:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.931 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191824 s, 54.7 MB/s 2024-04-25T09:15:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.955 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198601 s, 52.8 MB/s 2024-04-25T09:15:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:18.982 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228725 s, 45.8 MB/s 2024-04-25T09:15:19.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.007 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210503 s, 49.8 MB/s 2024-04-25T09:15:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260271 s, 40.3 MB/s 2024-04-25T09:15:19.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:19.040 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:19.040 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:19.040 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:15:19.041 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:19.041 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:19.041 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:19.041 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:19.133 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.133 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.133 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247067 s, 42.4 MB/s 2024-04-25T09:15:19.162 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.162 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.162 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025224 s, 41.6 MB/s 2024-04-25T09:15:19.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.191 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.191 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256989 s, 40.8 MB/s 2024-04-25T09:15:19.219 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243354 s, 43.1 MB/s 2024-04-25T09:15:19.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.245 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021514 s, 48.7 MB/s 2024-04-25T09:15:19.271 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.271 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230735 s, 45.4 MB/s 2024-04-25T09:15:19.296 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.297 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204946 s, 51.2 MB/s 2024-04-25T09:15:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.324 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023333 s, 44.9 MB/s 2024-04-25T09:15:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024866 s, 42.2 MB/s 2024-04-25T09:15:19.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.379 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214584 s, 48.9 MB/s 2024-04-25T09:15:19.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.409 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244682 s, 42.9 MB/s 2024-04-25T09:15:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238583 s, 44.0 MB/s 2024-04-25T09:15:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236742 s, 44.3 MB/s 2024-04-25T09:15:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232913 s, 45.0 MB/s 2024-04-25T09:15:19.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.520 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227892 s, 46.0 MB/s 2024-04-25T09:15:19.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220779 s, 47.5 MB/s 2024-04-25T09:15:19.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.574 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222266 s, 47.2 MB/s 2024-04-25T09:15:19.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.601 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.601 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225861 s, 46.4 MB/s 2024-04-25T09:15:19.627 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.627 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.627 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220391 s, 47.6 MB/s 2024-04-25T09:15:19.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.653 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.653 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214647 s, 48.9 MB/s 2024-04-25T09:15:19.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.679 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021975 s, 47.7 MB/s 2024-04-25T09:15:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.704 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210718 s, 49.8 MB/s 2024-04-25T09:15:19.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.732 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226865 s, 46.2 MB/s 2024-04-25T09:15:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.756 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198391 s, 52.9 MB/s 2024-04-25T09:15:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221412 s, 47.4 MB/s 2024-04-25T09:15:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209912 s, 50.0 MB/s 2024-04-25T09:15:19.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.835 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257219 s, 40.8 MB/s 2024-04-25T09:15:19.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.858 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190586 s, 55.0 MB/s 2024-04-25T09:15:19.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.885 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226499 s, 46.3 MB/s 2024-04-25T09:15:19.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.911 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217623 s, 48.2 MB/s 2024-04-25T09:15:19.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.940 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241378 s, 43.4 MB/s 2024-04-25T09:15:19.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:19.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215549 s, 48.6 MB/s 2024-04-25T09:15:19.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:19.968 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:19.968 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:19.968 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:15:19.968 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:19.968 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:19.968 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:19.968 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.060 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257133 s, 40.8 MB/s 2024-04-25T09:15:20.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.086 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229479 s, 45.7 MB/s 2024-04-25T09:15:20.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259236 s, 40.4 MB/s 2024-04-25T09:15:20.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.145 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245791 s, 42.7 MB/s 2024-04-25T09:15:20.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.176 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270089 s, 38.8 MB/s 2024-04-25T09:15:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269309 s, 38.9 MB/s 2024-04-25T09:15:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208418 s, 50.3 MB/s 2024-04-25T09:15:20.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.261 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240343 s, 43.6 MB/s 2024-04-25T09:15:20.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.292 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263163 s, 39.8 MB/s 2024-04-25T09:15:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249143 s, 42.1 MB/s 2024-04-25T09:15:20.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.346 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197467 s, 53.1 MB/s 2024-04-25T09:15:20.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.374 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240115 s, 43.7 MB/s 2024-04-25T09:15:20.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.399 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208216 s, 50.4 MB/s 2024-04-25T09:15:20.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.423 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200358 s, 52.3 MB/s 2024-04-25T09:15:20.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.450 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216669 s, 48.4 MB/s 2024-04-25T09:15:20.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.474 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198926 s, 52.7 MB/s 2024-04-25T09:15:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.499 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209133 s, 50.1 MB/s 2024-04-25T09:15:20.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.526 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.526 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226867 s, 46.2 MB/s 2024-04-25T09:15:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.554 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232955 s, 45.0 MB/s 2024-04-25T09:15:20.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216626 s, 48.4 MB/s 2024-04-25T09:15:20.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.608 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244692 s, 42.9 MB/s 2024-04-25T09:15:20.629 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.630 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189341 s, 55.4 MB/s 2024-04-25T09:15:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.656 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219263 s, 47.8 MB/s 2024-04-25T09:15:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226097 s, 46.4 MB/s 2024-04-25T09:15:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.708 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200074 s, 52.4 MB/s 2024-04-25T09:15:20.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.733 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210484 s, 49.8 MB/s 2024-04-25T09:15:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231332 s, 45.3 MB/s 2024-04-25T09:15:20.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.789 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249366 s, 42.0 MB/s 2024-04-25T09:15:20.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.817 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023537 s, 44.6 MB/s 2024-04-25T09:15:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198478 s, 52.8 MB/s 2024-04-25T09:15:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.866 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.867 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212741 s, 49.3 MB/s 2024-04-25T09:15:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.896 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254198 s, 41.3 MB/s 2024-04-25T09:15:20.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:20.899 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:20.899 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:20.899 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:15:20.899 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:20.899 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:20.899 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:20.899 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:20.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:20.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:20.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237579 s, 44.1 MB/s 2024-04-25T09:15:21.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.015 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.015 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206684 s, 50.7 MB/s 2024-04-25T09:15:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.038 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196502 s, 53.4 MB/s 2024-04-25T09:15:21.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.063 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020318 s, 51.6 MB/s 2024-04-25T09:15:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.095 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270036 s, 38.8 MB/s 2024-04-25T09:15:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271902 s, 38.6 MB/s 2024-04-25T09:15:21.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.156 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.156 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254657 s, 41.2 MB/s 2024-04-25T09:15:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211393 s, 49.6 MB/s 2024-04-25T09:15:21.211 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.211 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.211 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257225 s, 40.8 MB/s 2024-04-25T09:15:21.236 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.237 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208463 s, 50.3 MB/s 2024-04-25T09:15:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195835 s, 53.5 MB/s 2024-04-25T09:15:21.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.280 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158276 s, 66.2 MB/s 2024-04-25T09:15:21.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.304 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210972 s, 49.7 MB/s 2024-04-25T09:15:21.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.326 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188212 s, 55.7 MB/s 2024-04-25T09:15:21.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.348 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183793 s, 57.1 MB/s 2024-04-25T09:15:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.369 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175419 s, 59.8 MB/s 2024-04-25T09:15:21.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.398 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242406 s, 43.3 MB/s 2024-04-25T09:15:21.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.423 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207453 s, 50.5 MB/s 2024-04-25T09:15:21.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.452 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239033 s, 43.9 MB/s 2024-04-25T09:15:21.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.480 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240095 s, 43.7 MB/s 2024-04-25T09:15:21.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.503 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187098 s, 56.0 MB/s 2024-04-25T09:15:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.529 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222125 s, 47.2 MB/s 2024-04-25T09:15:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.550 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181571 s, 57.8 MB/s 2024-04-25T09:15:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235534 s, 44.5 MB/s 2024-04-25T09:15:21.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.605 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226883 s, 46.2 MB/s 2024-04-25T09:15:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.630 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213703 s, 49.1 MB/s 2024-04-25T09:15:21.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.656 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210773 s, 49.7 MB/s 2024-04-25T09:15:21.682 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.682 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.682 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219195 s, 47.8 MB/s 2024-04-25T09:15:21.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.705 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189857 s, 55.2 MB/s 2024-04-25T09:15:21.734 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.734 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.734 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248008 s, 42.3 MB/s 2024-04-25T09:15:21.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.761 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223885 s, 46.8 MB/s 2024-04-25T09:15:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.786 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202144 s, 51.9 MB/s 2024-04-25T09:15:21.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:21.788 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:21.789 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:21.789 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:15:21.789 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:21.789 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:21.789 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:21.789 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:21.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.864 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172592 s, 60.8 MB/s 2024-04-25T09:15:21.889 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.889 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.889 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208858 s, 50.2 MB/s 2024-04-25T09:15:21.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021456 s, 48.9 MB/s 2024-04-25T09:15:21.940 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.940 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.940 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022249 s, 47.1 MB/s 2024-04-25T09:15:21.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.962 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185123 s, 56.6 MB/s 2024-04-25T09:15:21.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:21.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:21.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253566 s, 41.4 MB/s 2024-04-25T09:15:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186548 s, 56.2 MB/s 2024-04-25T09:15:22.042 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.042 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.042 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249162 s, 42.1 MB/s 2024-04-25T09:15:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.070 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242867 s, 43.2 MB/s 2024-04-25T09:15:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.099 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236515 s, 44.3 MB/s 2024-04-25T09:15:22.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.130 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262228 s, 40.0 MB/s 2024-04-25T09:15:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.157 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224816 s, 46.6 MB/s 2024-04-25T09:15:22.182 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.182 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.182 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217671 s, 48.2 MB/s 2024-04-25T09:15:22.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.204 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186928 s, 56.1 MB/s 2024-04-25T09:15:22.231 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.231 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.231 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223106 s, 47.0 MB/s 2024-04-25T09:15:22.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.260 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246856 s, 42.5 MB/s 2024-04-25T09:15:22.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.284 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204684 s, 51.2 MB/s 2024-04-25T09:15:22.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.310 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.310 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215546 s, 48.6 MB/s 2024-04-25T09:15:22.334 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212674 s, 49.3 MB/s 2024-04-25T09:15:22.359 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.359 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.359 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197006 s, 53.2 MB/s 2024-04-25T09:15:22.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.386 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223554 s, 46.9 MB/s 2024-04-25T09:15:22.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.411 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.411 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206649 s, 50.7 MB/s 2024-04-25T09:15:22.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.435 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.435 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205371 s, 51.1 MB/s 2024-04-25T09:15:22.461 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.461 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.461 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214393 s, 48.9 MB/s 2024-04-25T09:15:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.486 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207414 s, 50.6 MB/s 2024-04-25T09:15:22.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.513 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226246 s, 46.3 MB/s 2024-04-25T09:15:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.543 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245056 s, 42.8 MB/s 2024-04-25T09:15:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189897 s, 55.2 MB/s 2024-04-25T09:15:22.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.594 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224576 s, 46.7 MB/s 2024-04-25T09:15:22.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.618 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.618 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208339 s, 50.3 MB/s 2024-04-25T09:15:22.641 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.641 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.641 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194463 s, 53.9 MB/s 2024-04-25T09:15:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.667 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215769 s, 48.6 MB/s 2024-04-25T09:15:22.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:22.670 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:22.670 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:22.670 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:15:22.670 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:22.670 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:22.670 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:22.670 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.763 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245512 s, 42.7 MB/s 2024-04-25T09:15:22.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.790 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198617 s, 52.8 MB/s 2024-04-25T09:15:22.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.817 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254126 s, 41.3 MB/s 2024-04-25T09:15:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205432 s, 51.0 MB/s 2024-04-25T09:15:22.867 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.867 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.867 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213012 s, 49.2 MB/s 2024-04-25T09:15:22.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.894 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222543 s, 47.1 MB/s 2024-04-25T09:15:22.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.917 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195415 s, 53.7 MB/s 2024-04-25T09:15:22.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.945 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229069 s, 45.8 MB/s 2024-04-25T09:15:22.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:22.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:22.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256446 s, 40.9 MB/s 2024-04-25T09:15:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.005 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254413 s, 41.2 MB/s 2024-04-25T09:15:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.035 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264617 s, 39.6 MB/s 2024-04-25T09:15:23.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233217 s, 45.0 MB/s 2024-04-25T09:15:23.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.088 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223887 s, 46.8 MB/s 2024-04-25T09:15:23.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.113 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202308 s, 51.8 MB/s 2024-04-25T09:15:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216905 s, 48.3 MB/s 2024-04-25T09:15:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.165 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215049 s, 48.8 MB/s 2024-04-25T09:15:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251442 s, 41.7 MB/s 2024-04-25T09:15:23.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.221 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229275 s, 45.7 MB/s 2024-04-25T09:15:23.247 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.247 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.247 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224153 s, 46.8 MB/s 2024-04-25T09:15:23.274 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.274 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.274 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224682 s, 46.7 MB/s 2024-04-25T09:15:23.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.300 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220099 s, 47.6 MB/s 2024-04-25T09:15:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194134 s, 54.0 MB/s 2024-04-25T09:15:23.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.346 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182244 s, 57.5 MB/s 2024-04-25T09:15:23.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.371 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020626 s, 50.8 MB/s 2024-04-25T09:15:23.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.396 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219896 s, 47.7 MB/s 2024-04-25T09:15:23.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220854 s, 47.5 MB/s 2024-04-25T09:15:23.445 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.445 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.445 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192065 s, 54.6 MB/s 2024-04-25T09:15:23.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.472 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219415 s, 47.8 MB/s 2024-04-25T09:15:23.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.499 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232684 s, 45.1 MB/s 2024-04-25T09:15:23.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.527 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237678 s, 44.1 MB/s 2024-04-25T09:15:23.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236422 s, 44.4 MB/s 2024-04-25T09:15:23.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.583 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02398 s, 43.7 MB/s 2024-04-25T09:15:23.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:23.586 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:23.586 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:23.586 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:15:23.586 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:23.586 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:23.586 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:23.586 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:23.676 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.677 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228281 s, 45.9 MB/s 2024-04-25T09:15:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197151 s, 53.2 MB/s 2024-04-25T09:15:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207937 s, 50.4 MB/s 2024-04-25T09:15:23.748 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.748 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.748 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191659 s, 54.7 MB/s 2024-04-25T09:15:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.776 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235207 s, 44.6 MB/s 2024-04-25T09:15:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.801 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213525 s, 49.1 MB/s 2024-04-25T09:15:23.829 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.829 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.829 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228871 s, 45.8 MB/s 2024-04-25T09:15:23.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218471 s, 48.0 MB/s 2024-04-25T09:15:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.879 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213921 s, 49.0 MB/s 2024-04-25T09:15:23.904 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.904 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.904 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200712 s, 52.2 MB/s 2024-04-25T09:15:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.933 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243982 s, 43.0 MB/s 2024-04-25T09:15:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218062 s, 48.1 MB/s 2024-04-25T09:15:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:23.983 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204651 s, 51.2 MB/s 2024-04-25T09:15:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.008 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207161 s, 50.6 MB/s 2024-04-25T09:15:24.036 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247221 s, 42.4 MB/s 2024-04-25T09:15:24.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.066 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243729 s, 43.0 MB/s 2024-04-25T09:15:24.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.091 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210778 s, 49.7 MB/s 2024-04-25T09:15:24.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.117 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222931 s, 47.0 MB/s 2024-04-25T09:15:24.141 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.141 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.141 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204676 s, 51.2 MB/s 2024-04-25T09:15:24.168 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.168 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.168 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223247 s, 47.0 MB/s 2024-04-25T09:15:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.196 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233127 s, 45.0 MB/s 2024-04-25T09:15:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239437 s, 43.8 MB/s 2024-04-25T09:15:24.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.256 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261238 s, 40.1 MB/s 2024-04-25T09:15:24.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.280 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198836 s, 52.7 MB/s 2024-04-25T09:15:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.308 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241366 s, 43.4 MB/s 2024-04-25T09:15:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.329 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179716 s, 58.3 MB/s 2024-04-25T09:15:24.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.355 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020379 s, 51.5 MB/s 2024-04-25T09:15:24.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.382 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227987 s, 46.0 MB/s 2024-04-25T09:15:24.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.407 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022071 s, 47.5 MB/s 2024-04-25T09:15:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.432 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205276 s, 51.1 MB/s 2024-04-25T09:15:24.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.458 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219172 s, 47.8 MB/s 2024-04-25T09:15:24.485 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.485 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.485 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222373 s, 47.2 MB/s 2024-04-25T09:15:24.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:24.488 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:15:24.488 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:15:24.488 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:15:24.488 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:15:24.488 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:15:24.488 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:15:24.488 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:15:24.580 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.580 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.580 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258362 s, 40.6 MB/s 2024-04-25T09:15:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.605 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217462 s, 48.2 MB/s 2024-04-25T09:15:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.631 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225233 s, 46.6 MB/s 2024-04-25T09:15:24.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221007 s, 47.4 MB/s 2024-04-25T09:15:24.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.683 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210362 s, 49.8 MB/s 2024-04-25T09:15:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.707 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197792 s, 53.0 MB/s 2024-04-25T09:15:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228619 s, 45.9 MB/s 2024-04-25T09:15:24.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.758 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209996 s, 49.9 MB/s 2024-04-25T09:15:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.783 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021632 s, 48.5 MB/s 2024-04-25T09:15:24.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231314 s, 45.3 MB/s 2024-04-25T09:15:24.838 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.838 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.838 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236723 s, 44.3 MB/s 2024-04-25T09:15:24.862 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201438 s, 52.1 MB/s 2024-04-25T09:15:24.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.890 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.890 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228556 s, 45.9 MB/s 2024-04-25T09:15:24.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.917 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229193 s, 45.8 MB/s 2024-04-25T09:15:24.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.945 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229772 s, 45.6 MB/s 2024-04-25T09:15:24.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:24.977 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262776 s, 39.9 MB/s 2024-04-25T09:15:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226549 s, 46.3 MB/s 2024-04-25T09:15:25.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.028 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199444 s, 52.6 MB/s 2024-04-25T09:15:25.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.058 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253978 s, 41.3 MB/s 2024-04-25T09:15:25.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.084 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216006 s, 48.5 MB/s 2024-04-25T09:15:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265624 s, 39.5 MB/s 2024-04-25T09:15:25.141 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.141 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.141 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219296 s, 47.8 MB/s 2024-04-25T09:15:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267106 s, 39.3 MB/s 2024-04-25T09:15:25.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.202 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255171 s, 41.1 MB/s 2024-04-25T09:15:25.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221708 s, 47.3 MB/s 2024-04-25T09:15:25.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.254 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229726 s, 45.6 MB/s 2024-04-25T09:15:25.281 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.282 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228766 s, 45.8 MB/s 2024-04-25T09:15:25.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.308 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218782 s, 47.9 MB/s 2024-04-25T09:15:25.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226816 s, 46.2 MB/s 2024-04-25T09:15:25.359 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.360 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202614 s, 51.8 MB/s 2024-04-25T09:15:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.386 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226035 s, 46.4 MB/s 2024-04-25T09:15:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:15:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:15:25.413 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229094 s, 45.8 MB/s 2024-04-25T09:15:25.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:25.416 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:15:25.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.652+0000 7f991e400640 1 Processor -- start 2024-04-25T09:15:25.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.652+0000 7f991e400640 1 -- start start 2024-04-25T09:15:25.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.653+0000 7f991e400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9918161270 0x7f99180b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.653+0000 7f991e400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f99180d2270 0x7f99180b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.653+0000 7f991e400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9918161650 con 0x7f99180b9390 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.653+0000 7f991e400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99180b84a0 con 0x7f9918161270 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.653+0000 7f991e400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99180b9820 con 0x7f99180d2270 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991d000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9918161270 0x7f99180b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991d000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9918161270 0x7f99180b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36582/0 (socket says 172.21.15.62:36582) 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991d000640 1 -- 172.21.15.62:0/3779307011 learned_addr learned my addr 172.21.15.62:0/3779307011 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991da00640 1 --2- 172.21.15.62:0/3779307011 >> v2:172.21.15.121:3300/0 conn(0x7f99180d2270 0x7f99180b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991d000640 1 -- 172.21.15.62:0/3779307011 >> v2:172.21.15.121:3300/0 conn(0x7f99180d2270 msgr2=0x7f99180b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991d000640 1 --2- 172.21.15.62:0/3779307011 >> v2:172.21.15.121:3300/0 conn(0x7f99180d2270 0x7f99180b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991d000640 1 -- 172.21.15.62:0/3779307011 >> v1:172.21.15.121:6789/0 conn(0x7f99180b9390 legacy=0x7f99180d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991d000640 1 -- 172.21.15.62:0/3779307011 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180b7e50 con 0x7f9918161270 2024-04-25T09:15:25.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.654+0000 7f991da00640 1 --2- 172.21.15.62:0/3779307011 >> v2:172.21.15.121:3300/0 conn(0x7f99180d2270 0x7f99180b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:25.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.655+0000 7f991d000640 1 --2- 172.21.15.62:0/3779307011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9918161270 0x7f99180b8930 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f99180b8620 tx=0x7f990802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3873dc3f9ed815 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.656+0000 7f9917400640 1 -- 172.21.15.62:0/3779307011 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908037070 con 0x7f9918161270 2024-04-25T09:15:25.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.656+0000 7f9917400640 1 -- 172.21.15.62:0/3779307011 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9908032440 con 0x7f9918161270 2024-04-25T09:15:25.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.656+0000 7f9917400640 1 -- 172.21.15.62:0/3779307011 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908032810 con 0x7f9918161270 2024-04-25T09:15:25.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.656+0000 7f991e400640 1 -- 172.21.15.62:0/3779307011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9918161270 msgr2=0x7f99180b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.656+0000 7f991e400640 1 --2- 172.21.15.62:0/3779307011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9918161270 0x7f99180b8930 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f99180b8620 tx=0x7f990802a180 comp rx=0 tx=0).stop 2024-04-25T09:15:25.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.657+0000 7f991e400640 1 -- 172.21.15.62:0/3779307011 shutdown_connections 2024-04-25T09:15:25.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.657+0000 7f991e400640 1 --2- 172.21.15.62:0/3779307011 >> v2:172.21.15.121:3300/0 conn(0x7f99180d2270 0x7f99180b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.657+0000 7f991e400640 1 --2- 172.21.15.62:0/3779307011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9918161270 0x7f99180b8930 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.657+0000 7f991e400640 1 -- 172.21.15.62:0/3779307011 >> 172.21.15.62:0/3779307011 conn(0x7f991806adf0 msgr2=0x7f991806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.657+0000 7f991e400640 1 -- 172.21.15.62:0/3779307011 shutdown_connections 2024-04-25T09:15:25.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.657+0000 7f991e400640 1 -- 172.21.15.62:0/3779307011 wait complete. 2024-04-25T09:15:25.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.659+0000 7f991e400640 1 Processor -- start 2024-04-25T09:15:25.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.659+0000 7f991e400640 1 -- start start 2024-04-25T09:15:25.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.660+0000 7f991e400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99180d2270 0x7f991814fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.660+0000 7f991e400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9918161270 0x7f99180c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.660+0000 7f991e400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99180c84c0 0x7f99180c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.660+0000 7f991e400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180b8e70 con 0x7f99180c84c0 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.660+0000 7f991e400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180b2fc0 con 0x7f9918161270 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991e400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99180ce690 con 0x7f99180d2270 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99180c84c0 0x7f99180c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99180c84c0 0x7f99180c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36588/0 (socket says 172.21.15.62:36588) 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991da00640 1 -- 172.21.15.62:0/548124295 learned_addr learned my addr 172.21.15.62:0/548124295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991d000640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99180d2270 0x7f991814fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f9917e00640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9918161270 0x7f99180c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991da00640 1 -- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99180d2270 msgr2=0x7f991814fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991da00640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99180d2270 0x7f991814fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991da00640 1 -- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9918161270 msgr2=0x7f99180c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991da00640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9918161270 0x7f99180c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.661+0000 7f991da00640 1 -- 172.21.15.62:0/548124295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180ce0b0 con 0x7f99180c84c0 2024-04-25T09:15:25.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.662+0000 7f991da00640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99180c84c0 0x7f99180c9190 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f991000cf20 tx=0x7f991000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.662+0000 7f9914c00640 1 -- 172.21.15.62:0/548124295 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991001a070 con 0x7f99180c84c0 2024-04-25T09:15:25.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.662+0000 7f9914c00640 1 -- 172.21.15.62:0/548124295 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991000d840 con 0x7f99180c84c0 2024-04-25T09:15:25.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.662+0000 7f9914c00640 1 -- 172.21.15.62:0/548124295 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991000dc30 con 0x7f99180c84c0 2024-04-25T09:15:25.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.662+0000 7f991e400640 1 -- 172.21.15.62:0/548124295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9918150530 con 0x7f99180c84c0 2024-04-25T09:15:25.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.662+0000 7f991e400640 1 -- 172.21.15.62:0/548124295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99180c8b20 con 0x7f99180c84c0 2024-04-25T09:15:25.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.665+0000 7f9914c00640 1 -- 172.21.15.62:0/548124295 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f991001d070 con 0x7f99180c84c0 2024-04-25T09:15:25.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.666+0000 7f9914c00640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f98f806b500 0x7f98f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.666+0000 7f9914c00640 1 -- 172.21.15.62:0/548124295 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f991008aaf0 con 0x7f99180c84c0 2024-04-25T09:15:25.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.666+0000 7f9922788740 1 Processor -- start 2024-04-25T09:15:25.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.666+0000 7f991d000640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f98f806b500 0x7f98f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.666+0000 7f9922788740 1 -- start start 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.667+0000 7f9922788740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb6d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.667+0000 7f9922788740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb8ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.667+0000 7f9922788740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb42c60 0x5653feb88ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.667+0000 7f9922788740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653feaa0f00 con 0x5653feb42c60 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.667+0000 7f9922788740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653feaa5f50 con 0x5653feb6cf80 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.667+0000 7f9922788740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653fea4f0b0 con 0x5653feb8a820 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991d000640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f98f806b500 0x7f98f806d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9908034000 tx=0x7f990802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb42c60 0x5653feb88ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb42c60 0x5653feb88ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36592/0 (socket says 172.21.15.62:36592) 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 -- 172.21.15.62:0/792008772 learned_addr learned my addr 172.21.15.62:0/792008772 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f9917e00640 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb8ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991d000640 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb6d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 -- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 msgr2=0x5653feb8ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb8ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 -- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 msgr2=0x5653feb6d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb6d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 -- 172.21.15.62:0/792008772 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653fe886ba0 con 0x5653feb42c60 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991d000640 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb6d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.668+0000 7f991da00640 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb42c60 0x5653feb88ee0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f99100556b0 tx=0x7f9910059a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8ba0ee3db752df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f990d600640 1 -- 172.21.15.62:0/792008772 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910087070 con 0x5653feb42c60 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f990d600640 1 -- 172.21.15.62:0/792008772 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991005d170 con 0x5653feb42c60 2024-04-25T09:15:25.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f990d600640 1 -- 172.21.15.62:0/792008772 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910061960 con 0x5653feb42c60 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f9922788740 1 -- 172.21.15.62:0/792008772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb42c60 msgr2=0x5653feb88ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f9922788740 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb42c60 0x5653feb88ee0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f99100556b0 tx=0x7f9910059a90 comp rx=0 tx=0).stop 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f9922788740 1 -- 172.21.15.62:0/792008772 shutdown_connections 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f9922788740 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb42c60 0x5653feb88ee0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f9922788740 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb8ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f9922788740 1 --2- 172.21.15.62:0/792008772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb6d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.669+0000 7f9922788740 1 -- 172.21.15.62:0/792008772 >> 172.21.15.62:0/792008772 conn(0x5653feb3f820 msgr2=0x5653feb87d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.670+0000 7f9922788740 1 -- 172.21.15.62:0/792008772 shutdown_connections 2024-04-25T09:15:25.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.670+0000 7f9922788740 1 -- 172.21.15.62:0/792008772 wait complete. 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.670+0000 7f9922788740 1 -- start start 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.670+0000 7f9922788740 1 Processor -- start 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.671+0000 7f9922788740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb6cf80 0x5653feb3c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.671+0000 7f9922788740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb7ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.671+0000 7f9922788740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb5bf20 0x5653feb80a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.671+0000 7f9922788740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653fea44620 con 0x5653feb6cf80 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.671+0000 7f9922788740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653feaa1ff0 con 0x5653feb5bf20 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.671+0000 7f9922788740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653feb43a00 con 0x5653feb8a820 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f9922788740 10 client.0 _ll_register_callbacks cb 0x5653fe7b9f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb6cf80 0x5653feb3c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb6cf80 0x5653feb3c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36604/0 (socket says 172.21.15.62:36604) 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991d000640 1 -- 172.21.15.62:0/3945232332 learned_addr learned my addr 172.21.15.62:0/3945232332 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f9917e00640 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb7ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991da00640 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb5bf20 0x5653feb80a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991d000640 1 -- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 msgr2=0x5653feb7ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991d000640 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb7ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991d000640 1 -- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb5bf20 msgr2=0x5653feb80a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991d000640 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb5bf20 0x5653feb80a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f991d000640 1 -- 172.21.15.62:0/3945232332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653fe77a490 con 0x5653feb6cf80 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.672+0000 7f9917e00640 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb7ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.673+0000 7f991d000640 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb6cf80 0x5653feb3c8a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f990802a770 tx=0x7f990802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43aaa513e20082f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.673+0000 7f9902000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.673+0000 7f9902000640 1 -- 172.21.15.62:0/3945232332 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990804d060 con 0x5653feb6cf80 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.673+0000 7f9902000640 1 -- 172.21.15.62:0/3945232332 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f990803edc0 con 0x5653feb6cf80 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.673+0000 7f9922788740 1 -- 172.21.15.62:0/3945232332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f991005d170 con 0x5653feb6cf80 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.673+0000 7f9902000640 1 -- 172.21.15.62:0/3945232332 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9908053050 con 0x5653feb6cf80 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.673+0000 7f9922788740 10 client.8475 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.673+0000 7f9922788740 1 -- 172.21.15.62:0/3945232332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x5653fe8e7330 con 0x5653feb6cf80 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9902000640 1 -- 172.21.15.62:0/3945232332 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991005d170 con 0x5653feb6cf80 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9902000640 1 -- 172.21.15.62:0/3945232332 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9908056020 con 0x5653feb6cf80 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9922788740 10 client.8475 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:25.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9922788740 10 client.8475 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:25.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9922788740 20 client.8475 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:25.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9922788740 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5653feb5b260 0x5653feb5b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9922788740 4 client.8475 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:15:25.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9922788740 1 -- 172.21.15.62:0/3945232332 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653feab3490 con 0x5653feb5b260 2024-04-25T09:15:25.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.674+0000 7f9917e00640 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5653feb5b260 0x5653feb5b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.675+0000 7f9902000640 10 client.8475 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:25.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.675+0000 7f9917e00640 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5653feb5b260 0x5653feb5b670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf19f2a024df343 server_cookie=7d796804655427c0 in_seq=0 out_seq=0 2024-04-25T09:15:25.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9902000640 1 -- 172.21.15.62:0/3945232332 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653feab3490 con 0x5653feb5b260 2024-04-25T09:15:25.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9902000640 10 client.8475 handle_command_reply: tid=0 2024-04-25T09:15:25.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 client.8475 shutdown 2024-04-25T09:15:25.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 -- 172.21.15.62:0/3945232332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb6cf80 msgr2=0x5653feb3c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb6cf80 0x5653feb3c8a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f990802a770 tx=0x7f990802c070 comp rx=0 tx=0).stop 2024-04-25T09:15:25.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 -- 172.21.15.62:0/3945232332 shutdown_connections 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5653feb5b260 0x5653feb5b670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb5bf20 0x5653feb80a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb8a820 0x5653feb7ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 --2- 172.21.15.62:0/3945232332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb6cf80 0x5653feb3c8a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9902000640 0 client.8475 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 -- 172.21.15.62:0/3945232332 >> 172.21.15.62:0/3945232332 conn(0x5653feb42c60 msgr2=0x5653feb8b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 -- 172.21.15.62:0/3945232332 shutdown_connections 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 1 -- 172.21.15.62:0/3945232332 wait complete. 2024-04-25T09:15:25.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.676+0000 7f9922788740 20 client.8475 trim_cache size 0 max 16384 2024-04-25T09:15:25.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.693+0000 7f9922788740 1 Processor -- start 2024-04-25T09:15:25.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.693+0000 7f9922788740 1 -- start start 2024-04-25T09:15:25.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.694+0000 7f9922788740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb5bf20 0x5653feb7f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.694+0000 7f9922788740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.694+0000 7f9922788740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb8a820 0x5653feb87f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.694+0000 7f9922788740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918200070 con 0x5653feb8a820 2024-04-25T09:15:25.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.694+0000 7f9922788740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99181ff270 con 0x5653feb6cf80 2024-04-25T09:15:25.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.694+0000 7f9922788740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99181f7bd0 con 0x5653feb5bf20 2024-04-25T09:15:25.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb8a820 0x5653feb87f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb8a820 0x5653feb87f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36614/0 (socket says 172.21.15.62:36614) 2024-04-25T09:15:25.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 -- 172.21.15.62:0/3955305991 learned_addr learned my addr 172.21.15.62:0/3955305991 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 -- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb5bf20 msgr2=0x5653feb7f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991d000640 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb5bf20 0x5653feb7f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb5bf20 0x5653feb7f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f9917e00640 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 -- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 msgr2=0x5653feb87a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb87a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 -- 172.21.15.62:0/3955305991 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653fe753610 con 0x5653feb8a820 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991d000640 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb5bf20 0x5653feb7f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.695+0000 7f991da00640 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb8a820 0x5653feb87f50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f991005d560 tx=0x7f991000f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81388509c66ca156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f990cc00640 1 -- 172.21.15.62:0/3955305991 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910087070 con 0x5653feb8a820 2024-04-25T09:15:25.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f990cc00640 1 -- 172.21.15.62:0/3955305991 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991005d170 con 0x5653feb8a820 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f990cc00640 1 -- 172.21.15.62:0/3955305991 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910061960 con 0x5653feb8a820 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 -- 172.21.15.62:0/3955305991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb8a820 msgr2=0x5653feb87f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb8a820 0x5653feb87f50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f991005d560 tx=0x7f991000f070 comp rx=0 tx=0).stop 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 -- 172.21.15.62:0/3955305991 shutdown_connections 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb8a820 0x5653feb87f50 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb6cf80 0x5653feb87a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 --2- 172.21.15.62:0/3955305991 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb5bf20 0x5653feb7f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 -- 172.21.15.62:0/3955305991 >> 172.21.15.62:0/3955305991 conn(0x5653feb42c60 msgr2=0x5653feb43060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 -- 172.21.15.62:0/3955305991 shutdown_connections 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.696+0000 7f9922788740 1 -- 172.21.15.62:0/3955305991 wait complete. 2024-04-25T09:15:25.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.697+0000 7f9922788740 1 -- start start 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.697+0000 7f9922788740 1 Processor -- start 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.698+0000 7f9922788740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb5bf20 0x5653febeffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.698+0000 7f9922788740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb6cf80 0x5653feba1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.698+0000 7f9922788740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb8a820 0x5653feba16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.698+0000 7f9922788740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653feb8a290 con 0x5653feb5bf20 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.698+0000 7f9922788740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180cf740 con 0x5653feb8a820 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.698+0000 7f9922788740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653feb44570 con 0x5653feb6cf80 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.698+0000 7f9922788740 10 client.0 _ll_register_callbacks cb 0x5653feb3dc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb5bf20 0x5653febeffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb5bf20 0x5653febeffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36626/0 (socket says 172.21.15.62:36626) 2024-04-25T09:15:25.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 -- 172.21.15.62:0/1475490719 learned_addr learned my addr 172.21.15.62:0/1475490719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 -- 172.21.15.62:0/1475490719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb6cf80 msgr2=0x5653feba1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f9917e00640 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb6cf80 0x5653feba1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991da00640 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb8a820 0x5653feba16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb6cf80 0x5653feba1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 -- 172.21.15.62:0/1475490719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb8a820 msgr2=0x5653feba16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb8a820 0x5653feba16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 -- 172.21.15.62:0/1475490719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653feb80aa0 con 0x5653feb5bf20 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f991d000640 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb5bf20 0x5653febeffb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f990803c6e0 tx=0x7f9908004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dda6932a52c94d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.699+0000 7f9902000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9902000640 1 -- 172.21.15.62:0/1475490719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990804d070 con 0x5653feb5bf20 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9902000640 1 -- 172.21.15.62:0/1475490719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f990803ccd0 con 0x5653feb5bf20 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9922788740 1 -- 172.21.15.62:0/1475490719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f991005d170 con 0x5653feb5bf20 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9902000640 1 -- 172.21.15.62:0/1475490719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9908037040 con 0x5653feb5bf20 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9922788740 10 client.8481 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:25.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9922788740 1 -- 172.21.15.62:0/1475490719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x5653feb9a7d0 con 0x5653feb5bf20 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9902000640 1 -- 172.21.15.62:0/1475490719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991005d170 con 0x5653feb5bf20 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9902000640 1 -- 172.21.15.62:0/1475490719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f990805f050 con 0x5653feb5bf20 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9922788740 10 client.8481 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9922788740 10 client.8481 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9922788740 20 client.8481 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.700+0000 7f9922788740 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5653feb6ba70 0x5653feb7d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.701+0000 7f9922788740 4 client.8481 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.701+0000 7f9922788740 1 -- 172.21.15.62:0/1475490719 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5653feaad7e0 con 0x5653feb6ba70 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.701+0000 7f9917e00640 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5653feb6ba70 0x5653feb7d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.701+0000 7f9917e00640 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5653feb6ba70 0x5653feb7d720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14ec603101b88ee server_cookie=499dfcf5201dcfee in_seq=0 out_seq=0 2024-04-25T09:15:25.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.701+0000 7f9902000640 10 client.8481 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9902000640 1 -- 172.21.15.62:0/1475490719 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5653feaad7e0 con 0x5653feb6ba70 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9902000640 10 client.8481 handle_command_reply: tid=0 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 client.8481 shutdown 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 -- 172.21.15.62:0/1475490719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb5bf20 msgr2=0x5653febeffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb5bf20 0x5653febeffb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f990803c6e0 tx=0x7f9908004730 comp rx=0 tx=0).stop 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 -- 172.21.15.62:0/1475490719 shutdown_connections 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5653feb6ba70 0x5653feb7d720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653feb8a820 0x5653feba16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5653feb6cf80 0x5653feba1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 --2- 172.21.15.62:0/1475490719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5653feb5bf20 0x5653febeffb0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 -- 172.21.15.62:0/1475490719 >> 172.21.15.62:0/1475490719 conn(0x5653feb42c60 msgr2=0x5653feb81980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9902000640 0 client.8481 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.704+0000 7f9922788740 1 -- 172.21.15.62:0/1475490719 shutdown_connections 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.705+0000 7f9922788740 1 -- 172.21.15.62:0/1475490719 wait complete. 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.705+0000 7f9922788740 20 client.8481 trim_cache size 0 max 16384 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "990d925a-2b50-4eff-bf9c-f1b763eaab80", 2024-04-25T09:15:25.693 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:15:25.694 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:15:25.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.705+0000 7f991e400640 1 -- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f98f806b500 msgr2=0x7f98f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.705+0000 7f991e400640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f98f806b500 0x7f98f806d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9908034000 tx=0x7f990802e000 comp rx=0 tx=0).stop 2024-04-25T09:15:25.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.705+0000 7f991e400640 1 -- 172.21.15.62:0/548124295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99180c84c0 msgr2=0x7f99180c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.705+0000 7f991e400640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99180c84c0 0x7f99180c9190 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f991000cf20 tx=0x7f991000a7d0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.706+0000 7f991e400640 1 -- 172.21.15.62:0/548124295 shutdown_connections 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.706+0000 7f991e400640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f98f806b500 0x7f98f806d9c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.706+0000 7f991e400640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99180c84c0 0x7f99180c9190 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.706+0000 7f991e400640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9918161270 0x7f99180c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.706+0000 7f991e400640 1 --2- 172.21.15.62:0/548124295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f99180d2270 0x7f991814fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.706+0000 7f991e400640 1 -- 172.21.15.62:0/548124295 >> 172.21.15.62:0/548124295 conn(0x7f991806adf0 msgr2=0x7f99180d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.706+0000 7f991e400640 1 -- 172.21.15.62:0/548124295 shutdown_connections 2024-04-25T09:15:25.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.706+0000 7f991e400640 1 -- 172.21.15.62:0/548124295 wait complete. 2024-04-25T09:15:25.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:25.712 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T09:15:25.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.946+0000 7f608b200640 1 Processor -- start 2024-04-25T09:15:25.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.946+0000 7f608b200640 1 -- start start 2024-04-25T09:15:25.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.948+0000 7f608b200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f608c0ad9e0 0x7f608c15e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.948+0000 7f608b200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c15e950 con 0x7f608c1411a0 2024-04-25T09:15:25.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.948+0000 7f608b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c15ec30 con 0x7f608c071a90 2024-04-25T09:15:25.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.948+0000 7f608b200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c0b4980 con 0x7f608c0ad9e0 2024-04-25T09:15:25.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.948+0000 7f6089e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f608c0ad9e0 0x7f608c15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.948+0000 7f6089e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f608c0ad9e0 0x7f608c15e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36630/0 (socket says 172.21.15.62:36630) 2024-04-25T09:15:25.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.948+0000 7f6089e00640 1 -- 172.21.15.62:0/1589643823 learned_addr learned my addr 172.21.15.62:0/1589643823 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.949+0000 7f6089e00640 1 -- 172.21.15.62:0/1589643823 >> v1:172.21.15.121:6789/0 conn(0x7f608c071a90 legacy=0x7f608c0b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.949+0000 7f6089e00640 1 -- 172.21.15.62:0/1589643823 >> v1:172.21.15.62:6789/0 conn(0x7f608c1411a0 legacy=0x7f608c071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.949+0000 7f6089e00640 1 -- 172.21.15.62:0/1589643823 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c15ef10 con 0x7f608c0ad9e0 2024-04-25T09:15:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.950+0000 7f6089e00640 1 --2- 172.21.15.62:0/1589643823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0ad9e0 0x7f608c15e090 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f608c0b4b00 tx=0x7f607c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f03943e2c01df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.950+0000 7f6088a00640 1 -- 172.21.15.62:0/1589643823 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c036070 con 0x7f608c0ad9e0 2024-04-25T09:15:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.950+0000 7f6088a00640 1 -- 172.21.15.62:0/1589643823 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c031600 con 0x7f608c0ad9e0 2024-04-25T09:15:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.950+0000 7f6088a00640 1 -- 172.21.15.62:0/1589643823 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c0319d0 con 0x7f608c0ad9e0 2024-04-25T09:15:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.950+0000 7f608b200640 1 -- 172.21.15.62:0/1589643823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0ad9e0 msgr2=0x7f608c15e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.950+0000 7f608b200640 1 --2- 172.21.15.62:0/1589643823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0ad9e0 0x7f608c15e090 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f608c0b4b00 tx=0x7f607c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.951+0000 7f608b200640 1 -- 172.21.15.62:0/1589643823 shutdown_connections 2024-04-25T09:15:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.951+0000 7f608b200640 1 --2- 172.21.15.62:0/1589643823 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c0ad9e0 0x7f608c15e090 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.951+0000 7f608b200640 1 -- 172.21.15.62:0/1589643823 >> 172.21.15.62:0/1589643823 conn(0x7f608c0b13c0 msgr2=0x7f608c0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.951+0000 7f608b200640 1 -- 172.21.15.62:0/1589643823 shutdown_connections 2024-04-25T09:15:25.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.951+0000 7f608b200640 1 -- 172.21.15.62:0/1589643823 wait complete. 2024-04-25T09:15:25.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.952+0000 7f608b200640 1 Processor -- start 2024-04-25T09:15:25.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.953+0000 7f608b200640 1 -- start start 2024-04-25T09:15:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.953+0000 7f608b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c071a90 0x7f608c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f608b200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0ad9e0 0x7f608c1e9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f608b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c1411a0 0x7f608c1e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f608b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0b46d0 con 0x7f608c071a90 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f608b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0b4550 con 0x7f608c1411a0 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f608b200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0b5520 con 0x7f608c0ad9e0 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f6089e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c071a90 0x7f608c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f6089e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c071a90 0x7f608c15a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36642/0 (socket says 172.21.15.62:36642) 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f6089e00640 1 -- 172.21.15.62:0/1640725009 learned_addr learned my addr 172.21.15.62:0/1640725009 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f608a800640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c1411a0 0x7f608c1e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f6089e00640 1 -- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0ad9e0 msgr2=0x7f608c1e9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f6089e00640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0ad9e0 0x7f608c1e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f6089e00640 1 -- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c1411a0 msgr2=0x7f608c1e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f6089e00640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c1411a0 0x7f608c1e9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.954+0000 7f6089e00640 1 -- 172.21.15.62:0/1640725009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c1e6840 con 0x7f608c071a90 2024-04-25T09:15:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.955+0000 7f6089e00640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c071a90 0x7f608c15a2c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f607c009e50 tx=0x7f607c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.955+0000 7f6082000640 1 -- 172.21.15.62:0/1640725009 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c036050 con 0x7f608c071a90 2024-04-25T09:15:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.955+0000 7f6082000640 1 -- 172.21.15.62:0/1640725009 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c042070 con 0x7f608c071a90 2024-04-25T09:15:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.955+0000 7f6082000640 1 -- 172.21.15.62:0/1640725009 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c009510 con 0x7f608c071a90 2024-04-25T09:15:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.955+0000 7f608b200640 1 -- 172.21.15.62:0/1640725009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c1e6ae0 con 0x7f608c071a90 2024-04-25T09:15:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.955+0000 7f608b200640 1 -- 172.21.15.62:0/1640725009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c1e70a0 con 0x7f608c071a90 2024-04-25T09:15:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.957+0000 7f6082000640 1 -- 172.21.15.62:0/1640725009 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f607c007bd0 con 0x7f608c071a90 2024-04-25T09:15:25.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.958+0000 7f6082000640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b4d0 0x7f606806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.958+0000 7f6082000640 1 -- 172.21.15.62:0/1640725009 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f607c0ab210 con 0x7f608c071a90 2024-04-25T09:15:25.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.959+0000 7f6093a2b740 1 Processor -- start 2024-04-25T09:15:25.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.959+0000 7f6089400640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b4d0 0x7f606806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.959+0000 7f6093a2b740 1 -- start start 2024-04-25T09:15:25.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.960+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2d870 0x55babdb2dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.960+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdafda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.960+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdafdfb0 0x55babdb25cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.960+0000 7f6093a2b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55babda66870 con 0x55babdafdfb0 2024-04-25T09:15:25.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.960+0000 7f6093a2b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55babda0fd60 con 0x55babdb2d870 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.960+0000 7f6093a2b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55babda62910 con 0x55babdb2e290 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.960+0000 7f608a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdafdfb0 0x55babdb25cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f608a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdafdfb0 0x55babdb25cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36658/0 (socket says 172.21.15.62:36658) 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f608a800640 1 -- 172.21.15.62:0/2823402160 learned_addr learned my addr 172.21.15.62:0/2823402160 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f6089400640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b4d0 0x7f606806d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6074000f60 tx=0x7f60740079d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f6089e00640 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2d870 0x55babdb2dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f6089400640 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdafda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f608a800640 1 -- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 msgr2=0x55babdafda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f608a800640 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdafda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f608a800640 1 -- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2d870 msgr2=0x55babdb2dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f608a800640 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2d870 0x55babdb2dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f608a800640 1 -- 172.21.15.62:0/2823402160 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55babd8457d0 con 0x55babdafdfb0 2024-04-25T09:15:25.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f6089400640 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdafda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:25.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.961+0000 7f608a800640 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdafdfb0 0x55babdb25cf0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6084003de0 tx=0x7f60840042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a46c99df1a2d056 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f607a600640 1 -- 172.21.15.62:0/2823402160 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084016020 con 0x55babdafdfb0 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f607a600640 1 -- 172.21.15.62:0/2823402160 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608400db60 con 0x55babdafdfb0 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f607a600640 1 -- 172.21.15.62:0/2823402160 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840123f0 con 0x55babdafdfb0 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 -- 172.21.15.62:0/2823402160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdafdfb0 msgr2=0x55babdb25cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdafdfb0 0x55babdb25cf0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6084003de0 tx=0x7f60840042a0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 -- 172.21.15.62:0/2823402160 shutdown_connections 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdafdfb0 0x55babdb25cf0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdafda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 --2- 172.21.15.62:0/2823402160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2d870 0x55babdb2dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 -- 172.21.15.62:0/2823402160 >> 172.21.15.62:0/2823402160 conn(0x55babdaffef0 msgr2=0x55babdb48680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 -- 172.21.15.62:0/2823402160 shutdown_connections 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.962+0000 7f6093a2b740 1 -- 172.21.15.62:0/2823402160 wait complete. 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.963+0000 7f6093a2b740 1 -- start start 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.963+0000 7f6093a2b740 1 Processor -- start 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.964+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.964+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2e290 0x55babdb2bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.964+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb22200 0x55babdb2c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.964+0000 7f6093a2b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55babdb29fc0 con 0x55babdb2d870 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.964+0000 7f6093a2b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55babda231c0 con 0x55babdb2e290 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.964+0000 7f6093a2b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55babdb2a260 con 0x55babdb22200 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.964+0000 7f6093a2b740 10 client.0 _ll_register_callbacks cb 0x55babda618f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:25.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36672/0 (socket says 172.21.15.62:36672) 2024-04-25T09:15:25.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 -- 172.21.15.62:0/3132905758 learned_addr learned my addr 172.21.15.62:0/3132905758 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f608a800640 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb22200 0x55babdb2c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089400640 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2e290 0x55babdb2bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 -- 172.21.15.62:0/3132905758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb22200 msgr2=0x55babdb2c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb22200 0x55babdb2c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 -- 172.21.15.62:0/3132905758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2e290 msgr2=0x55babdb2bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2e290 0x55babdb2bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 -- 172.21.15.62:0/3132905758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55babd73b0f0 con 0x55babdb2d870 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f6089e00640 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb230c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f607c000f80 tx=0x7f607c0792a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0759a1c8fdeda1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.965+0000 7f606ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:25.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f606ea00640 1 -- 172.21.15.62:0/3132905758 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c091020 con 0x55babdb2d870 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f606ea00640 1 -- 172.21.15.62:0/3132905758 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f607c0913c0 con 0x55babdb2d870 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 1 -- 172.21.15.62:0/3132905758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f608400db60 con 0x55babdb2d870 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f606ea00640 1 -- 172.21.15.62:0/3132905758 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f607c096050 con 0x55babdb2d870 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 10 client.8496 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 1 -- 172.21.15.62:0/3132905758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55babd8a5f60 con 0x55babdb2d870 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f606ea00640 1 -- 172.21.15.62:0/3132905758 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608400db60 con 0x55babdb2d870 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f606ea00640 1 -- 172.21.15.62:0/3132905758 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f607c0982a0 con 0x55babdb2d870 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 10 client.8496 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 10 client.8496 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 20 client.8496 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55babdb31170 0x55babdb5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 4 client.8496 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:15:25.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.966+0000 7f6093a2b740 1 -- 172.21.15.62:0/3132905758 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55babda3c690 con 0x55babdb31170 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.967+0000 7f6089400640 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55babdb31170 0x55babdb5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.967+0000 7f6089400640 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55babdb31170 0x55babdb5b5b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3085df90881a4657 server_cookie=3912825ff8118d97 in_seq=0 out_seq=0 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.967+0000 7f606ea00640 10 client.8496 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f606ea00640 1 -- 172.21.15.62:0/3132905758 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55babda3c690 con 0x55babdb31170 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f606ea00640 10 client.8496 handle_command_reply: tid=0 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 client.8496 shutdown 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 -- 172.21.15.62:0/3132905758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 msgr2=0x55babdb230c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb230c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f607c000f80 tx=0x7f607c0792a0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 -- 172.21.15.62:0/3132905758 shutdown_connections 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55babdb31170 0x55babdb5b5b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb22200 0x55babdb2c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb2e290 0x55babdb2bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 --2- 172.21.15.62:0/3132905758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb230c0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 -- 172.21.15.62:0/3132905758 >> 172.21.15.62:0/3132905758 conn(0x55babdafdfb0 msgr2=0x55babdafd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f606ea00640 0 client.8496 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:25.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 -- 172.21.15.62:0/3132905758 shutdown_connections 2024-04-25T09:15:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.968+0000 7f6093a2b740 1 -- 172.21.15.62:0/3132905758 wait complete. 2024-04-25T09:15:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.969+0000 7f6093a2b740 20 client.8496 trim_cache size 0 max 16384 2024-04-25T09:15:25.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.985+0000 7f6093a2b740 1 Processor -- start 2024-04-25T09:15:25.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.985+0000 7f6093a2b740 1 -- start start 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.986+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 0x55babdb6aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.986+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2d870 0x55babdb218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.986+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2e290 0x55babdb29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.986+0000 7f6093a2b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55babdafec10 con 0x55babdb2e290 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.986+0000 7f6093a2b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c200110 con 0x55babdb22200 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.986+0000 7f6093a2b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c1ff3a0 con 0x55babdb2d870 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2e290 0x55babdb29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2e290 0x55babdb29190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36688/0 (socket says 172.21.15.62:36688) 2024-04-25T09:15:25.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 -- 172.21.15.62:0/743426549 learned_addr learned my addr 172.21.15.62:0/743426549 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f6089400640 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2d870 0x55babdb218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f6089e00640 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 0x55babdb6aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 -- 172.21.15.62:0/743426549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2d870 msgr2=0x55babdb218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2d870 0x55babdb218f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 -- 172.21.15.62:0/743426549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 msgr2=0x55babdb6aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 0x55babdb6aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 -- 172.21.15.62:0/743426549 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55babd7141b0 con 0x55babdb2e290 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.987+0000 7f608a800640 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2e290 0x55babdb29190 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x55babdaff180 tx=0x7f6084012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9140d8262b5c4d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6079c00640 1 -- 172.21.15.62:0/743426549 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084016070 con 0x55babdb2e290 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6079c00640 1 -- 172.21.15.62:0/743426549 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084005040 con 0x55babdb2e290 2024-04-25T09:15:25.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6079c00640 1 -- 172.21.15.62:0/743426549 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084005410 con 0x55babdb2e290 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 -- 172.21.15.62:0/743426549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2e290 msgr2=0x55babdb29190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2e290 0x55babdb29190 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x55babdaff180 tx=0x7f6084012000 comp rx=0 tx=0).stop 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 -- 172.21.15.62:0/743426549 shutdown_connections 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2e290 0x55babdb29190 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2d870 0x55babdb218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 --2- 172.21.15.62:0/743426549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 0x55babdb6aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 -- 172.21.15.62:0/743426549 >> 172.21.15.62:0/743426549 conn(0x55babdafdfb0 msgr2=0x55babdb31380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 -- 172.21.15.62:0/743426549 shutdown_connections 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.988+0000 7f6093a2b740 1 -- 172.21.15.62:0/743426549 wait complete. 2024-04-25T09:15:25.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.989+0000 7f6093a2b740 1 -- start start 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.989+0000 7f6093a2b740 1 Processor -- start 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.990+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 0x55babdb62080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.990+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb2a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.990+0000 7f6093a2b740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdb24500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.990+0000 7f6093a2b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c1e8560 con 0x55babdb2d870 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.990+0000 7f6093a2b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c1f7db0 con 0x55babdb22200 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.990+0000 7f6093a2b740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55babdb25220 con 0x55babdb2e290 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.990+0000 7f6093a2b740 10 client.0 _ll_register_callbacks cb 0x7f6084007d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:25.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb2a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb2a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36698/0 (socket says 172.21.15.62:36698) 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 -- 172.21.15.62:0/708414785 learned_addr learned my addr 172.21.15.62:0/708414785 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f608a800640 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdb24500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 -- 172.21.15.62:0/708414785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 msgr2=0x55babdb24500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdb24500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 -- 172.21.15.62:0/708414785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 msgr2=0x55babdb62080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 0x55babdb62080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 -- 172.21.15.62:0/708414785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55babdb61610 con 0x55babdb2d870 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f6089400640 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb2a6f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f60740052e0 tx=0x7f6074001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0172c58c28a821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.991+0000 7f606ea00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f606ea00640 1 -- 172.21.15.62:0/708414785 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074025070 con 0x55babdb2d870 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f606ea00640 1 -- 172.21.15.62:0/708414785 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6074004b80 con 0x55babdb2d870 2024-04-25T09:15:25.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 1 -- 172.21.15.62:0/708414785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6084005040 con 0x55babdb2d870 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f606ea00640 1 -- 172.21.15.62:0/708414785 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6074030050 con 0x55babdb2d870 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 10 client.8502 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 1 -- 172.21.15.62:0/708414785 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55babdb2ac30 con 0x55babdb2d870 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f606ea00640 1 -- 172.21.15.62:0/708414785 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084005040 con 0x55babdb2d870 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f606ea00640 1 -- 172.21.15.62:0/708414785 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6074033020 con 0x55babdb2d870 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 10 client.8502 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 10 client.8502 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 20 client.8502 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55babdb9bd90 0x55babdb29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 4 client.8502 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.992+0000 7f6093a2b740 1 -- 172.21.15.62:0/708414785 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55babda73db0 con 0x55babdb9bd90 2024-04-25T09:15:25.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.993+0000 7f6089e00640 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55babdb9bd90 0x55babdb29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.993+0000 7f6089e00640 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55babdb9bd90 0x55babdb29190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f26c9a8da682775 server_cookie=ec07789c0ce3b07 in_seq=0 out_seq=0 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.993+0000 7f606ea00640 10 client.8502 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f606ea00640 1 -- 172.21.15.62:0/708414785 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55babda73db0 con 0x55babdb9bd90 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f606ea00640 10 client.8502 handle_command_reply: tid=0 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 client.8502 shutdown 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 -- 172.21.15.62:0/708414785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 msgr2=0x55babdb2a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb2a6f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f60740052e0 tx=0x7f6074001080 comp rx=0 tx=0).stop 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 -- 172.21.15.62:0/708414785 shutdown_connections 2024-04-25T09:15:25.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55babdb9bd90 0x55babdb29190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55babdb2d870 0x55babdb2a6f0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55babdb2e290 0x55babdb24500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 --2- 172.21.15.62:0/708414785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55babdb22200 0x55babdb62080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f606ea00640 0 client.8502 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 -- 172.21.15.62:0/708414785 >> 172.21.15.62:0/708414785 conn(0x55babdafdfb0 msgr2=0x55babdb31170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 -- 172.21.15.62:0/708414785 shutdown_connections 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 1 -- 172.21.15.62:0/708414785 wait complete. 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.994+0000 7f6093a2b740 20 client.8502 trim_cache size 0 max 16384 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:15:25.985 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:15:25.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 -- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b4d0 msgr2=0x7f606806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b4d0 0x7f606806d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6074000f60 tx=0x7f60740079d0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 -- 172.21.15.62:0/1640725009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c071a90 msgr2=0x7f608c15a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:25.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c071a90 0x7f608c15a2c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f607c009e50 tx=0x7f607c004a40 comp rx=0 tx=0).stop 2024-04-25T09:15:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 -- 172.21.15.62:0/1640725009 shutdown_connections 2024-04-25T09:15:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f606806b4d0 0x7f606806d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f608c1411a0 0x7f608c1e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f608c0ad9e0 0x7f608c1e9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 --2- 172.21.15.62:0/1640725009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f608c071a90 0x7f608c15a2c0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 -- 172.21.15.62:0/1640725009 >> 172.21.15.62:0/1640725009 conn(0x7f608c0b13c0 msgr2=0x7f608c0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 -- 172.21.15.62:0/1640725009 shutdown_connections 2024-04-25T09:15:25.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:25.995+0000 7f608b200640 1 -- 172.21.15.62:0/1640725009 wait complete. 2024-04-25T09:15:25.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:26.000 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:15:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.215+0000 7fe4ad000640 1 Processor -- start 2024-04-25T09:15:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.216+0000 7fe4ad000640 1 -- start start 2024-04-25T09:15:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.218+0000 7fe4ad000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe4a80b8630 0x7fe4a80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.218+0000 7fe4ad000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a80d2270 con 0x7fe4a80b97b0 2024-04-25T09:15:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.218+0000 7fe4ad000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a80b7590 con 0x7fe4a815dc10 2024-04-25T09:15:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.218+0000 7fe4ad000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a80b8fe0 con 0x7fe4a80b8630 2024-04-25T09:15:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.218+0000 7fe4a5400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe4a80b97b0 0x7fe4a80b6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:32774/0 (socket says 172.21.15.62:32774) 2024-04-25T09:15:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.218+0000 7fe4a5400640 1 -- 172.21.15.62:0/761920260 learned_addr learned my addr 172.21.15.62:0/761920260 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.219+0000 7fe4a6800640 1 --2- 172.21.15.62:0/761920260 >> v2:172.21.15.121:3300/0 conn(0x7fe4a80b8630 0x7fe4a80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.219+0000 7fe4a4a00640 1 -- 172.21.15.62:0/761920260 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452701086 0 0) 0x7fe4a80d2270 con 0x7fe4a80b97b0 2024-04-25T09:15:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.219+0000 7fe4a4a00640 1 -- 172.21.15.62:0/761920260 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe48c003520 con 0x7fe4a80b97b0 2024-04-25T09:15:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.220+0000 7fe4a6800640 1 -- 172.21.15.62:0/761920260 >> v1:172.21.15.121:6789/0 conn(0x7fe4a815dc10 legacy=0x7fe4a80d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.220+0000 7fe4a6800640 1 -- 172.21.15.62:0/761920260 >> v1:172.21.15.62:6789/0 conn(0x7fe4a80b97b0 legacy=0x7fe4a80b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.220+0000 7fe4a6800640 1 -- 172.21.15.62:0/761920260 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a80b7870 con 0x7fe4a80b8630 2024-04-25T09:15:26.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.222+0000 7fe4a6800640 1 --2- 172.21.15.62:0/761920260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 0x7fe4a80b8aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009d90 tx=0x7fe4a002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd62bd0fe54d39ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.222+0000 7fe4a4a00640 1 -- 172.21.15.62:0/761920260 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0035020 con 0x7fe4a80b8630 2024-04-25T09:15:26.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.222+0000 7fe4a4a00640 1 -- 172.21.15.62:0/761920260 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a002ae50 con 0x7fe4a80b8630 2024-04-25T09:15:26.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.222+0000 7fe4a4a00640 1 -- 172.21.15.62:0/761920260 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494001670 con 0x7fe4a80b8630 2024-04-25T09:15:26.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.223+0000 7fe4ad000640 1 -- 172.21.15.62:0/761920260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 msgr2=0x7fe4a80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.223+0000 7fe4ad000640 1 --2- 172.21.15.62:0/761920260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 0x7fe4a80b8aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009d90 tx=0x7fe4a002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.223+0000 7fe4ad000640 1 -- 172.21.15.62:0/761920260 shutdown_connections 2024-04-25T09:15:26.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.223+0000 7fe4ad000640 1 --2- 172.21.15.62:0/761920260 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 0x7fe4a80b8aa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.223+0000 7fe4ad000640 1 -- 172.21.15.62:0/761920260 >> 172.21.15.62:0/761920260 conn(0x7fe4a806b000 msgr2=0x7fe4a806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.223+0000 7fe4ad000640 1 -- 172.21.15.62:0/761920260 shutdown_connections 2024-04-25T09:15:26.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.223+0000 7fe4ad000640 1 -- 172.21.15.62:0/761920260 wait complete. 2024-04-25T09:15:26.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.226+0000 7fe4ad000640 1 Processor -- start 2024-04-25T09:15:26.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.226+0000 7fe4ad000640 1 -- start start 2024-04-25T09:15:26.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4ad000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 0x7fe4a80aa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4ad000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a80b97b0 0x7fe4a814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4ad000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4a815dc10 0x7fe4a814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4ad000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80b7e30 con 0x7fe4a80b97b0 2024-04-25T09:15:26.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4ad000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80b81a0 con 0x7fe4a80b8630 2024-04-25T09:15:26.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4ad000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a80b72a0 con 0x7fe4a815dc10 2024-04-25T09:15:26.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a80b97b0 0x7fe4a814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4a5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a80b97b0 0x7fe4a814ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36708/0 (socket says 172.21.15.62:36708) 2024-04-25T09:15:26.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.228+0000 7fe4a5400640 1 -- 172.21.15.62:0/1107660180 learned_addr learned my addr 172.21.15.62:0/1107660180 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a6800640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4a815dc10 0x7fe4a814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 0x7fe4a80aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a5400640 1 -- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4a815dc10 msgr2=0x7fe4a814d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a5400640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4a815dc10 0x7fe4a814d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a5400640 1 -- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 msgr2=0x7fe4a80aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a5400640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 0x7fe4a80aa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a5400640 1 -- 172.21.15.62:0/1107660180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a814db70 con 0x7fe4a80b97b0 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a6800640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4a815dc10 0x7fe4a814d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:26.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4a5400640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a80b97b0 0x7fe4a814ce70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe49400c7d0 tx=0x7fe49400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe49e000640 1 -- 172.21.15.62:0/1107660180 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494018070 con 0x7fe4a80b97b0 2024-04-25T09:15:26.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe49e000640 1 -- 172.21.15.62:0/1107660180 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4940074e0 con 0x7fe4a80b97b0 2024-04-25T09:15:26.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe49e000640 1 -- 172.21.15.62:0/1107660180 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494004a20 con 0x7fe4a80b97b0 2024-04-25T09:15:26.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.229+0000 7fe4ad000640 1 -- 172.21.15.62:0/1107660180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a814de10 con 0x7fe4a80b97b0 2024-04-25T09:15:26.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.230+0000 7fe4ad000640 1 -- 172.21.15.62:0/1107660180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a814e3d0 con 0x7fe4a80b97b0 2024-04-25T09:15:26.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.232+0000 7fe49e000640 1 -- 172.21.15.62:0/1107660180 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe494004bc0 con 0x7fe4a80b97b0 2024-04-25T09:15:26.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.233+0000 7fe49e000640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe48806b4d0 0x7fe48806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.233+0000 7fe49e000640 1 -- 172.21.15.62:0/1107660180 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe494089900 con 0x7fe4a80b97b0 2024-04-25T09:15:26.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.233+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe48806b4d0 0x7fe48806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.234+0000 7fe4affcd740 1 Processor -- start 2024-04-25T09:15:26.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.234+0000 7fe4affcd740 1 -- start start 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.234+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe48806b4d0 0x7fe48806d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe498001500 tx=0x7fe498009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c7680e5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c76810e670 0x55c76810eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4affcd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c768047c70 con 0x55c76810e670 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4affcd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c767ff0cf0 con 0x55c7680e3bb0 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4affcd740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c768043d10 con 0x55c768120280 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c76810e670 0x55c76810eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c76810e670 0x55c76810eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36724/0 (socket says 172.21.15.62:36724) 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a6800640 1 -- 172.21.15.62:0/3049545926 learned_addr learned my addr 172.21.15.62:0/3049545926 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c7680e5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a5400640 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a6800640 1 -- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 msgr2=0x55c7680e3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a6800640 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a6800640 1 -- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 msgr2=0x55c7680e5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a6800640 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c7680e5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a6800640 1 -- 172.21.15.62:0/3049545926 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c767e267d0 con 0x55c76810e670 2024-04-25T09:15:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c7680e5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.235+0000 7fe4a5400640 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:26.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.236+0000 7fe4a6800640 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c76810e670 0x55c76810eaf0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009e80 tx=0x7fe4a002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd4318e74b30169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.236+0000 7fe492a00640 1 -- 172.21.15.62:0/3049545926 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0040030 con 0x55c76810e670 2024-04-25T09:15:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.236+0000 7fe492a00640 1 -- 172.21.15.62:0/3049545926 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0007540 con 0x55c76810e670 2024-04-25T09:15:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.236+0000 7fe492a00640 1 -- 172.21.15.62:0/3049545926 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0035040 con 0x55c76810e670 2024-04-25T09:15:26.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 -- 172.21.15.62:0/3049545926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c76810e670 msgr2=0x55c76810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c76810e670 0x55c76810eaf0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009e80 tx=0x7fe4a002e000 comp rx=0 tx=0).stop 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 -- 172.21.15.62:0/3049545926 shutdown_connections 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c76810e670 0x55c76810eaf0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3049545926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c7680e5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 -- 172.21.15.62:0/3049545926 >> 172.21.15.62:0/3049545926 conn(0x55c7680aba30 msgr2=0x55c768129670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 -- 172.21.15.62:0/3049545926 shutdown_connections 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.237+0000 7fe4affcd740 1 -- 172.21.15.62:0/3049545926 wait complete. 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.238+0000 7fe4affcd740 1 -- start start 2024-04-25T09:15:26.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.238+0000 7fe4affcd740 1 Processor -- start 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7680e3bb0 0x55c76810b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 0x55c768123b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4affcd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c767fe63a0 con 0x55c768120280 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4affcd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c76810c1c0 con 0x55c76810e670 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4affcd740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7681240c0 con 0x55c7680e3bb0 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4affcd740 10 client.0 _ll_register_callbacks cb 0x55c768003b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36740/0 (socket says 172.21.15.62:36740) 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.239+0000 7fe4a6800640 1 -- 172.21.15.62:0/3644204577 learned_addr learned my addr 172.21.15.62:0/3644204577 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7680e3bb0 0x55c76810b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a6800640 1 -- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7680e3bb0 msgr2=0x55c76810b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a5400640 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 0x55c768123b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a6800640 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7680e3bb0 0x55c76810b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a6800640 1 -- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 msgr2=0x55c768123b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a6800640 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 0x55c768123b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a6800640 1 -- 172.21.15.62:0/3644204577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c767d5c3f0 con 0x55c768120280 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7680e3bb0 0x55c76810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4a6800640 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139410 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009d90 tx=0x7fe4a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d1a62daaa4f3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe487400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe487400640 1 -- 172.21.15.62:0/3644204577 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0035040 con 0x55c768120280 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe487400640 1 -- 172.21.15.62:0/3644204577 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe4a0007b00 con 0x55c768120280 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.240+0000 7fe4affcd740 1 -- 172.21.15.62:0/3644204577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4a0007540 con 0x55c768120280 2024-04-25T09:15:26.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe487400640 1 -- 172.21.15.62:0/3644204577 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4a0056050 con 0x55c768120280 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe4affcd740 10 client.8514 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe4affcd740 1 -- 172.21.15.62:0/3644204577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55c767d1c0f0 con 0x55c768120280 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe487400640 1 -- 172.21.15.62:0/3644204577 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0007540 con 0x55c768120280 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe487400640 1 -- 172.21.15.62:0/3644204577 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe4a0059020 con 0x55c768120280 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe4affcd740 10 client.8514 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe4affcd740 10 client.8514 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe4affcd740 20 client.8514 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c76813a0e0 0x55c768145c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe4affcd740 4 client.8514 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.241+0000 7fe4affcd740 1 -- 172.21.15.62:0/3644204577 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7680551b0 con 0x55c76813a0e0 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.242+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c76813a0e0 0x55c768145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.242+0000 7fe487400640 10 client.8514 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.242+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c76813a0e0 0x55c768145c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9efee640e584406c server_cookie=504190b540020de3 in_seq=0 out_seq=0 2024-04-25T09:15:26.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe487400640 1 -- 172.21.15.62:0/3644204577 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7680551b0 con 0x55c76813a0e0 2024-04-25T09:15:26.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe487400640 10 client.8514 handle_command_reply: tid=0 2024-04-25T09:15:26.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 client.8514 shutdown 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 -- 172.21.15.62:0/3644204577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 msgr2=0x55c768139410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139410 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009d90 tx=0x7fe4a0004ab0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 -- 172.21.15.62:0/3644204577 shutdown_connections 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c76813a0e0 0x55c768145c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139410 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 0x55c768123b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 --2- 172.21.15.62:0/3644204577 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c7680e3bb0 0x55c76810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe487400640 0 client.8514 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 -- 172.21.15.62:0/3644204577 >> 172.21.15.62:0/3644204577 conn(0x55c7680aba30 msgr2=0x55c76812cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 -- 172.21.15.62:0/3644204577 shutdown_connections 2024-04-25T09:15:26.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.243+0000 7fe4affcd740 1 -- 172.21.15.62:0/3644204577 wait complete. 2024-04-25T09:15:26.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.244+0000 7fe4affcd740 20 client.8514 trim_cache size 0 max 16384 2024-04-25T09:15:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.261+0000 7fe4affcd740 1 Processor -- start 2024-04-25T09:15:26.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.261+0000 7fe4affcd740 1 -- start start 2024-04-25T09:15:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c768123900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c76810e670 0x55c768138dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4affcd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81ffdf0 con 0x55c768120280 2024-04-25T09:15:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4affcd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81ff010 con 0x55c7680e3bb0 2024-04-25T09:15:26.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4affcd740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a8150860 con 0x55c76810e670 2024-04-25T09:15:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4a6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36744/0 (socket says 172.21.15.62:36744) 2024-04-25T09:15:26.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4a6800640 1 -- 172.21.15.62:0/157655643 learned_addr learned my addr 172.21.15.62:0/157655643 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c768123900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.262+0000 7fe4a5400640 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c76810e670 0x55c768138dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe4a6800640 1 -- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c76810e670 msgr2=0x55c768138dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe4a6800640 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c76810e670 0x55c768138dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe4a6800640 1 -- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 msgr2=0x55c768123900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe4a6800640 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c768123900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe4a6800640 1 -- 172.21.15.62:0/157655643 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c76814a510 con 0x55c768120280 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe4a5400640 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c76810e670 0x55c768138dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c768123900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe4a6800640 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139300 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0007300 tx=0x7fe4a0047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5689336938fb874 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe492000640 1 -- 172.21.15.62:0/157655643 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0040070 con 0x55c768120280 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe492000640 1 -- 172.21.15.62:0/157655643 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0007900 con 0x55c768120280 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.263+0000 7fe492000640 1 -- 172.21.15.62:0/157655643 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0035070 con 0x55c768120280 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 -- 172.21.15.62:0/157655643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 msgr2=0x55c768139300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139300 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0007300 tx=0x7fe4a0047040 comp rx=0 tx=0).stop 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 -- 172.21.15.62:0/157655643 shutdown_connections 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c768120280 0x55c768139300 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c76810e670 0x55c768138dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 --2- 172.21.15.62:0/157655643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7680e3bb0 0x55c768123900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 -- 172.21.15.62:0/157655643 >> 172.21.15.62:0/157655643 conn(0x55c7680aba30 msgr2=0x55c768127740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 -- 172.21.15.62:0/157655643 shutdown_connections 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.264+0000 7fe4affcd740 1 -- 172.21.15.62:0/157655643 wait complete. 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.265+0000 7fe4affcd740 1 -- start start 2024-04-25T09:15:26.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.265+0000 7fe4affcd740 1 Processor -- start 2024-04-25T09:15:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7680e3bb0 0x55c76811fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 0x55c768193630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4affcd740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4affcd740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c76810eb00 con 0x55c7680e3bb0 2024-04-25T09:15:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4affcd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a814f5a0 con 0x55c76810e670 2024-04-25T09:15:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4affcd740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7680e3ff0 con 0x55c768120280 2024-04-25T09:15:26.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4affcd740 10 client.0 _ll_register_callbacks cb 0x7fe4a0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4a5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7680e3bb0 0x55c76811fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4a5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7680e3bb0 0x55c76811fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36752/0 (socket says 172.21.15.62:36752) 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.266+0000 7fe4a5e00640 1 -- 172.21.15.62:0/2987682769 learned_addr learned my addr 172.21.15.62:0/2987682769 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe4a6800640 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe4a5400640 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 0x55c768193630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe4a5e00640 1 -- 172.21.15.62:0/2987682769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 msgr2=0x55c7680e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe4a5e00640 1 -- 172.21.15.62:0/2987682769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 msgr2=0x55c768193630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 0x55c768193630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe4a5e00640 1 -- 172.21.15.62:0/2987682769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c76812c080 con 0x55c7680e3bb0 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe4a5e00640 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7680e3bb0 0x55c76811fc10 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe498014840 tx=0x7fe498000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da5809622a5439a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe487400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:26.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.267+0000 7fe487400640 1 -- 172.21.15.62:0/2987682769 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498022db0 con 0x55c7680e3bb0 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe487400640 1 -- 172.21.15.62:0/2987682769 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe4980240b0 con 0x55c7680e3bb0 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 1 -- 172.21.15.62:0/2987682769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4a0007900 con 0x55c7680e3bb0 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe487400640 1 -- 172.21.15.62:0/2987682769 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe498035050 con 0x55c7680e3bb0 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 10 client.8520 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 1 -- 172.21.15.62:0/2987682769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55c76812ae60 con 0x55c7680e3bb0 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe487400640 1 -- 172.21.15.62:0/2987682769 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0007900 con 0x55c7680e3bb0 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe487400640 1 -- 172.21.15.62:0/2987682769 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe498038020 con 0x55c7680e3bb0 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 10 client.8520 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 10 client.8520 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 20 client.8520 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c7680e35d0 0x55c76812a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 4 client.8520 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.268+0000 7fe4affcd740 1 -- 172.21.15.62:0/2987682769 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c76804f500 con 0x55c7680e35d0 2024-04-25T09:15:26.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.269+0000 7fe4a5400640 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c7680e35d0 0x55c76812a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.269+0000 7fe4a5400640 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c7680e35d0 0x55c76812a030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e58bbf155d30b8a9 server_cookie=949fcc2dbbaef34b in_seq=0 out_seq=0 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.269+0000 7fe487400640 10 client.8520 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.269+0000 7fe487400640 1 -- 172.21.15.62:0/2987682769 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55c76804f500 con 0x55c7680e35d0 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.269+0000 7fe487400640 10 client.8520 handle_command_reply: tid=0 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.269+0000 7fe4affcd740 1 client.8520 shutdown 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 -- 172.21.15.62:0/2987682769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7680e3bb0 msgr2=0x55c76811fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7680e3bb0 0x55c76811fc10 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe498014840 tx=0x7fe498000bc0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 -- 172.21.15.62:0/2987682769 shutdown_connections 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c7680e35d0 0x55c76812a030 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c768120280 0x55c7680e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c76810e670 0x55c768193630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 --2- 172.21.15.62:0/2987682769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c7680e3bb0 0x55c76811fc10 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe487400640 0 client.8520 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 -- 172.21.15.62:0/2987682769 >> 172.21.15.62:0/2987682769 conn(0x55c7680aba30 msgr2=0x55c76810f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 -- 172.21.15.62:0/2987682769 shutdown_connections 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 1 -- 172.21.15.62:0/2987682769 wait complete. 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.270+0000 7fe4affcd740 20 client.8520 trim_cache size 0 max 16384 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:15:26.261 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:15:26.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 -- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe48806b4d0 msgr2=0x7fe48806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe48806b4d0 0x7fe48806d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe498001500 tx=0x7fe498009040 comp rx=0 tx=0).stop 2024-04-25T09:15:26.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 -- 172.21.15.62:0/1107660180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a80b97b0 msgr2=0x7fe4a814ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a80b97b0 0x7fe4a814ce70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fe49400c7d0 tx=0x7fe49400cc90 comp rx=0 tx=0).stop 2024-04-25T09:15:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 -- 172.21.15.62:0/1107660180 shutdown_connections 2024-04-25T09:15:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe48806b4d0 0x7fe48806d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4a80b97b0 0x7fe4a814ce70 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe4a815dc10 0x7fe4a814d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 --2- 172.21.15.62:0/1107660180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80b8630 0x7fe4a80aa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 -- 172.21.15.62:0/1107660180 >> 172.21.15.62:0/1107660180 conn(0x7fe4a806b000 msgr2=0x7fe4a80d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 -- 172.21.15.62:0/1107660180 shutdown_connections 2024-04-25T09:15:26.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.271+0000 7fe4ad000640 1 -- 172.21.15.62:0/1107660180 wait complete. 2024-04-25T09:15:26.275 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.478+0000 7fe173600640 1 Processor -- start 2024-04-25T09:15:26.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.478+0000 7fe173600640 1 -- start start 2024-04-25T09:15:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.479+0000 7fe173600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c0cccf0 0x7fe16c0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.480+0000 7fe173600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c06ca70 con 0x7fe16c15bd10 2024-04-25T09:15:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.480+0000 7fe173600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c15c0f0 con 0x7fe16c0aed00 2024-04-25T09:15:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.480+0000 7fe173600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c06d660 con 0x7fe16c0cccf0 2024-04-25T09:15:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.480+0000 7fe170c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe16c15bd10 0x7fe16c0cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:32780/0 (socket says 172.21.15.62:32780) 2024-04-25T09:15:26.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.480+0000 7fe170c00640 1 -- 172.21.15.62:0/139872681 learned_addr learned my addr 172.21.15.62:0/139872681 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.480+0000 7fe16be00640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c0cccf0 0x7fe16c0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.480+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273472270 0 0) 0x7fe16c06ca70 con 0x7fe16c15bd10 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe150003520 con 0x7fe16c15bd10 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1499945517 0 0) 0x7fe150003520 con 0x7fe16c15bd10 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c0cccf0 msgr2=0x7fe16c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c0cccf0 0x7fe16c0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 >> v1:172.21.15.121:6789/0 conn(0x7fe16c0aed00 legacy=0x7fe16c0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c06cd30 con 0x7fe16c15bd10 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fe15c0019f0 con 0x7fe16c15bd10 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 >> v1:172.21.15.62:6789/0 conn(0x7fe16c15bd10 legacy=0x7fe16c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1500039e0 0x7fe150003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe150004800 0x7fe150004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c0cccf0 0x7fe1500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c06d660 con 0x7fe1500039e0 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000ba70 con 0x7fe150004800 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000bc70 con 0x7fe16c0cccf0 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.481+0000 7fe170c00640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1500039e0 0x7fe150003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.482+0000 7fe16be00640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe150004800 0x7fe150004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.482+0000 7fe171600640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c0cccf0 0x7fe1500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.482+0000 7fe170c00640 1 -- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c0cccf0 msgr2=0x7fe1500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.482+0000 7fe170c00640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c0cccf0 0x7fe1500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.482+0000 7fe170c00640 1 -- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe150004800 msgr2=0x7fe150004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.482+0000 7fe170c00640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe150004800 0x7fe150004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.482+0000 7fe170c00640 1 -- 172.21.15.62:0/139872681 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150003520 con 0x7fe1500039e0 2024-04-25T09:15:26.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.482+0000 7fe16be00640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe150004800 0x7fe150004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.483+0000 7fe170c00640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1500039e0 0x7fe150003e00 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00a810 tx=0x7fe150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4162a089deaa0de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.483+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00a460 con 0x7fe1500039e0 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.483+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c02f070 con 0x7fe1500039e0 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.483+0000 7fe16b400640 1 -- 172.21.15.62:0/139872681 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c02bc50 con 0x7fe1500039e0 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.484+0000 7fe173600640 1 -- 172.21.15.62:0/139872681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1500039e0 msgr2=0x7fe150003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.484+0000 7fe173600640 1 --2- 172.21.15.62:0/139872681 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1500039e0 0x7fe150003e00 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00a810 tx=0x7fe150005830 comp rx=0 tx=0).stop 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.484+0000 7fe171600640 1 -- 172.21.15.62:0/139872681 reap_dead start 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.484+0000 7fe173600640 1 -- 172.21.15.62:0/139872681 shutdown_connections 2024-04-25T09:15:26.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.484+0000 7fe173600640 1 -- 172.21.15.62:0/139872681 >> 172.21.15.62:0/139872681 conn(0x7fe16c0668f0 msgr2=0x7fe16c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.484+0000 7fe173600640 1 -- 172.21.15.62:0/139872681 shutdown_connections 2024-04-25T09:15:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.484+0000 7fe173600640 1 -- 172.21.15.62:0/139872681 wait complete. 2024-04-25T09:15:26.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.486+0000 7fe173600640 1 Processor -- start 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.486+0000 7fe173600640 1 -- start start 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.487+0000 7fe173600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1500039e0 0x7fe16c148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.487+0000 7fe173600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16c158720 0x7fe16c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.487+0000 7fe173600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c15a370 0x7fe16c15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.487+0000 7fe173600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c06daf0 con 0x7fe16c158720 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.487+0000 7fe173600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c06d2f0 con 0x7fe1500039e0 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.487+0000 7fe173600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c159830 con 0x7fe16c15a370 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.487+0000 7fe16be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16c158720 0x7fe16c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe16be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16c158720 0x7fe16c152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36776/0 (socket says 172.21.15.62:36776) 2024-04-25T09:15:26.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe16be00640 1 -- 172.21.15.62:0/795468570 learned_addr learned my addr 172.21.15.62:0/795468570 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe170c00640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1500039e0 0x7fe16c148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe171600640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c15a370 0x7fe16c15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe16be00640 1 -- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c15a370 msgr2=0x7fe16c15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe16be00640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c15a370 0x7fe16c15a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe16be00640 1 -- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1500039e0 msgr2=0x7fe16c148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe16be00640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1500039e0 0x7fe16c148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe16be00640 1 -- 172.21.15.62:0/795468570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c158c20 con 0x7fe16c158720 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe171600640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c15a370 0x7fe16c15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.488+0000 7fe16be00640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16c158720 0x7fe16c152f20 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe158010890 tx=0x7fe158010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.489+0000 7fe168c00640 1 -- 172.21.15.62:0/795468570 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15800f040 con 0x7fe16c158720 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.489+0000 7fe168c00640 1 -- 172.21.15.62:0/795468570 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15800eb00 con 0x7fe16c158720 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.489+0000 7fe168c00640 1 -- 172.21.15.62:0/795468570 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15801c400 con 0x7fe16c158720 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.489+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c158f00 con 0x7fe16c158720 2024-04-25T09:15:26.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.489+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c1594c0 con 0x7fe16c158720 2024-04-25T09:15:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.490+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c072c70 con 0x7fe16c158720 2024-04-25T09:15:26.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.491+0000 7fe168c00640 1 -- 172.21.15.62:0/795468570 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe15801c5a0 con 0x7fe16c158720 2024-04-25T09:15:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.492+0000 7fe168c00640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe14c06b500 0x7fe14c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.492+0000 7fe168c00640 1 -- 172.21.15.62:0/795468570 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe15808a750 con 0x7fe16c158720 2024-04-25T09:15:26.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.493+0000 7fe170c00640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe14c06b500 0x7fe14c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.494+0000 7fe170c00640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe14c06b500 0x7fe14c06d9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe15c033000 tx=0x7fe15c007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.497+0000 7fe168c00640 1 -- 172.21.15.62:0/795468570 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe158018030 con 0x7fe16c158720 2024-04-25T09:15:26.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.737+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe16c0b1cd0 con 0x7fe16c158720 2024-04-25T09:15:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.738+0000 7fe168c00640 1 -- 172.21.15.62:0/795468570 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe158059590 con 0x7fe16c158720 2024-04-25T09:15:26.725 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:26.725 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:26.725 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:26.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.740+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe14c06b500 msgr2=0x7fe14c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.740+0000 7fe173600640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe14c06b500 0x7fe14c06d9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe15c033000 tx=0x7fe15c007000 comp rx=0 tx=0).stop 2024-04-25T09:15:26.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.740+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16c158720 msgr2=0x7fe16c152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.740+0000 7fe173600640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16c158720 0x7fe16c152f20 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe158010890 tx=0x7fe158010d50 comp rx=0 tx=0).stop 2024-04-25T09:15:26.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.741+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 shutdown_connections 2024-04-25T09:15:26.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.741+0000 7fe173600640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe14c06b500 0x7fe14c06d9c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.741+0000 7fe173600640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe16c158720 0x7fe16c152f20 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.741+0000 7fe173600640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe16c15a370 0x7fe16c15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.741+0000 7fe173600640 1 --2- 172.21.15.62:0/795468570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1500039e0 0x7fe16c148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.741+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 >> 172.21.15.62:0/795468570 conn(0x7fe16c0668f0 msgr2=0x7fe16c15ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.741+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 shutdown_connections 2024-04-25T09:15:26.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.741+0000 7fe173600640 1 -- 172.21.15.62:0/795468570 wait complete. 2024-04-25T09:15:26.746 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:15:26.747 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:15:26.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.971+0000 7fbfa8c00640 1 Processor -- start 2024-04-25T09:15:26.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.971+0000 7fbfa8c00640 1 -- start start 2024-04-25T09:15:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.973+0000 7fbfa8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.973+0000 7fbfa8c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbfa406d9d0 0x7fbfa40b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.973+0000 7fbfa8c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa40b0cf0 con 0x7fbfa40cdb30 2024-04-25T09:15:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.973+0000 7fbfa8c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa40b0170 con 0x7fbfa406d9d0 2024-04-25T09:15:26.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.973+0000 7fbfa8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40b0450 con 0x7fbfa415a9b0 2024-04-25T09:15:26.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.974+0000 7fbfa3400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.974+0000 7fbfa3e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbfa406d9d0 0x7fbfa40b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.974+0000 7fbfa3400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:33462/0 (socket says 172.21.15.62:33462) 2024-04-25T09:15:26.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.974+0000 7fbfa3400640 1 -- 172.21.15.62:0/790043014 learned_addr learned my addr 172.21.15.62:0/790043014 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:26.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.974+0000 7fbfa3e00640 1 -- 172.21.15.62:0/790043014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 msgr2=0x7fbfa40afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.974+0000 7fbfa3e00640 1 --2- 172.21.15.62:0/790043014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.974+0000 7fbfa3e00640 1 -- 172.21.15.62:0/790043014 >> v1:172.21.15.121:6789/0 conn(0x7fbfa40cdb30 legacy=0x7fbfa406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:26.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.975+0000 7fbfa3e00640 1 -- 172.21.15.62:0/790043014 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40b0730 con 0x7fbfa406d9d0 2024-04-25T09:15:26.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.976+0000 7fbfa3e00640 1 --2- 172.21.15.62:0/790043014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 0x7fbfa40b46d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009d90 tx=0x7fbf9c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f2650489c29f84d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.976+0000 7fbfa2000640 1 -- 172.21.15.62:0/790043014 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c031600 con 0x7fbfa406d9d0 2024-04-25T09:15:26.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.976+0000 7fbfa2000640 1 -- 172.21.15.62:0/790043014 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9c0317a0 con 0x7fbfa406d9d0 2024-04-25T09:15:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.976+0000 7fbfa2000640 1 -- 172.21.15.62:0/790043014 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c031b70 con 0x7fbfa406d9d0 2024-04-25T09:15:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.977+0000 7fbfa8c00640 1 -- 172.21.15.62:0/790043014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 msgr2=0x7fbfa40b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.977+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/790043014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 0x7fbfa40b46d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009d90 tx=0x7fbf9c02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.977+0000 7fbfa8c00640 1 -- 172.21.15.62:0/790043014 shutdown_connections 2024-04-25T09:15:26.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.977+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/790043014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 0x7fbfa40b46d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.977+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/790043014 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.977+0000 7fbfa8c00640 1 -- 172.21.15.62:0/790043014 >> 172.21.15.62:0/790043014 conn(0x7fbfa40668f0 msgr2=0x7fbfa4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.977+0000 7fbfa8c00640 1 -- 172.21.15.62:0/790043014 shutdown_connections 2024-04-25T09:15:26.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.977+0000 7fbfa8c00640 1 -- 172.21.15.62:0/790043014 wait complete. 2024-04-25T09:15:26.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.979+0000 7fbfa8c00640 1 Processor -- start 2024-04-25T09:15:26.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.979+0000 7fbfa8c00640 1 -- start start 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa8c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 0x7fbfa41f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfa40cdb30 0x7fbfa41f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa8c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40b1480 con 0x7fbfa40cdb30 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa8c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa415ad90 con 0x7fbfa406d9d0 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa8c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40b1600 con 0x7fbfa415a9b0 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfa40cdb30 0x7fbfa41f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfa40cdb30 0x7fbfa41f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36792/0 (socket says 172.21.15.62:36792) 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.980+0000 7fbfa2a00640 1 -- 172.21.15.62:0/1967352136 learned_addr learned my addr 172.21.15.62:0/1967352136 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:26.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbfa3400640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 0x7fbfa41f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbfa3e00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbfa2a00640 1 -- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 msgr2=0x7fbfa40705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbfa2a00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbfa2a00640 1 -- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 msgr2=0x7fbfa41f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbfa2a00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 0x7fbfa41f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:26.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbfa2a00640 1 -- 172.21.15.62:0/1967352136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa41f6c10 con 0x7fbfa40cdb30 2024-04-25T09:15:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbfa2a00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfa40cdb30 0x7fbfa41f73a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000c400 tx=0x7fbf9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.981+0000 7fbf9b400640 1 -- 172.21.15.62:0/1967352136 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900052d0 con 0x7fbfa40cdb30 2024-04-25T09:15:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.982+0000 7fbf9b400640 1 -- 172.21.15.62:0/1967352136 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf90005490 con 0x7fbfa40cdb30 2024-04-25T09:15:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.982+0000 7fbf9b400640 1 -- 172.21.15.62:0/1967352136 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90007260 con 0x7fbfa40cdb30 2024-04-25T09:15:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.982+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa41f3e20 con 0x7fbfa40cdb30 2024-04-25T09:15:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.982+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa41f3ac0 con 0x7fbfa40cdb30 2024-04-25T09:15:26.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.983+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa4072b10 con 0x7fbfa40cdb30 2024-04-25T09:15:26.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.984+0000 7fbf9b400640 1 -- 172.21.15.62:0/1967352136 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf90007400 con 0x7fbfa40cdb30 2024-04-25T09:15:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.985+0000 7fbf9b400640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf8406b500 0x7fbf8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.985+0000 7fbf9b400640 1 -- 172.21.15.62:0/1967352136 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbf90089730 con 0x7fbfa40cdb30 2024-04-25T09:15:26.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.985+0000 7fbfa3400640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf8406b500 0x7fbf8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:26.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.987+0000 7fbfa3400640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf8406b500 0x7fbf8406d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40ce350 tx=0x7fbf94008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:26.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:26.990+0000 7fbf9b400640 1 -- 172.21.15.62:0/1967352136 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf90058560 con 0x7fbfa40cdb30 2024-04-25T09:15:27.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.271+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbfa40750e0 con 0x7fbfa40cdb30 2024-04-25T09:15:27.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.271+0000 7fbf9b400640 1 -- 172.21.15.62:0/1967352136 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7fbf90057e60 con 0x7fbfa40cdb30 2024-04-25T09:15:27.259 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:27.259 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":754,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076261340,"bytes_used":2600755200,"bytes_avail":770493358080,"bytes_total":773094113280,"write_bytes_sec":44036039,"read_op_per_sec":0,"write_op_per_sec":45},"fsmap":{"epoch":78,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T09:14:26.142824+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:15:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf8406b500 msgr2=0x7fbf8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf8406b500 0x7fbf8406d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40ce350 tx=0x7fbf94008040 comp rx=0 tx=0).stop 2024-04-25T09:15:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfa40cdb30 msgr2=0x7fbfa41f73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:27.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfa40cdb30 0x7fbfa41f73a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000c400 tx=0x7fbf9000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:15:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 shutdown_connections 2024-04-25T09:15:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbf8406b500 0x7fbf8406d9c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:27.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbfa40cdb30 0x7fbfa41f73a0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:27.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbfa415a9b0 0x7fbfa40705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:27.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.274+0000 7fbfa8c00640 1 --2- 172.21.15.62:0/1967352136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfa406d9d0 0x7fbfa41f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:27.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.275+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 >> 172.21.15.62:0/1967352136 conn(0x7fbfa40668f0 msgr2=0x7fbfa415eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:27.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.275+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 shutdown_connections 2024-04-25T09:15:27.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:27.275+0000 7fbfa8c00640 1 -- 172.21.15.62:0/1967352136 wait complete. 2024-04-25T09:15:27.282 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:15:28.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:28.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.514+0000 7f0c77000640 1 Processor -- start 2024-04-25T09:15:28.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.515+0000 7f0c77000640 1 -- start start 2024-04-25T09:15:28.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.516+0000 7f0c77000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b56f0 0x7f0c70160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.516+0000 7f0c77000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0c700b6a90 0x7f0c70161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.516+0000 7f0c77000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0c700b27a0 0x7f0c70161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.516+0000 7f0c77000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c70161780 con 0x7f0c700b6a90 2024-04-25T09:15:28.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c77000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c700cedc0 con 0x7f0c700b27a0 2024-04-25T09:15:28.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c77000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700d3110 con 0x7f0c700b56f0 2024-04-25T09:15:28.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0c700b6a90 0x7f0c70161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0c700b6a90 0x7f0c70161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36804/0 (socket says 172.21.15.62:36804) 2024-04-25T09:15:28.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75200640 1 -- 172.21.15.62:0/658400997 learned_addr learned my addr 172.21.15.62:0/658400997 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:28.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75c00640 1 --2- 172.21.15.62:0/658400997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b56f0 0x7f0c70160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75200640 1 -- 172.21.15.62:0/658400997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b56f0 msgr2=0x7f0c70160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c76600640 1 --2- 172.21.15.62:0/658400997 >> v2:172.21.15.121:3300/0 conn(0x7f0c700b27a0 0x7f0c70161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75200640 1 --2- 172.21.15.62:0/658400997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b56f0 0x7f0c70160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75200640 1 -- 172.21.15.62:0/658400997 >> v2:172.21.15.121:3300/0 conn(0x7f0c700b27a0 msgr2=0x7f0c70161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75200640 1 --2- 172.21.15.62:0/658400997 >> v2:172.21.15.121:3300/0 conn(0x7f0c700b27a0 0x7f0c70161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75200640 1 -- 172.21.15.62:0/658400997 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700d33f0 con 0x7f0c700b6a90 2024-04-25T09:15:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c75c00640 1 --2- 172.21.15.62:0/658400997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b56f0 0x7f0c70160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.517+0000 7f0c76600640 1 --2- 172.21.15.62:0/658400997 >> v2:172.21.15.121:3300/0 conn(0x7f0c700b27a0 0x7f0c70161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.518+0000 7f0c75200640 1 --2- 172.21.15.62:0/658400997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70161240 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0c700b30b0 tx=0x7f0c5c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98d3760662b2eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:28.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.519+0000 7f0c6fe00640 1 -- 172.21.15.62:0/658400997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c037070 con 0x7f0c700b6a90 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.519+0000 7f0c6fe00640 1 -- 172.21.15.62:0/658400997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c032440 con 0x7f0c700b6a90 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.519+0000 7f0c6fe00640 1 -- 172.21.15.62:0/658400997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c032810 con 0x7f0c700b6a90 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 -- 172.21.15.62:0/658400997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 msgr2=0x7f0c70161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 --2- 172.21.15.62:0/658400997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70161240 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f0c700b30b0 tx=0x7f0c5c02a1d0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 -- 172.21.15.62:0/658400997 shutdown_connections 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 --2- 172.21.15.62:0/658400997 >> v2:172.21.15.121:3300/0 conn(0x7f0c700b27a0 0x7f0c70161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 --2- 172.21.15.62:0/658400997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70161240 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 --2- 172.21.15.62:0/658400997 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b56f0 0x7f0c70160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 -- 172.21.15.62:0/658400997 >> 172.21.15.62:0/658400997 conn(0x7f0c70066d10 msgr2=0x7f0c70067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 -- 172.21.15.62:0/658400997 shutdown_connections 2024-04-25T09:15:28.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.520+0000 7f0c77000640 1 -- 172.21.15.62:0/658400997 wait complete. 2024-04-25T09:15:28.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.522+0000 7f0c77000640 1 Processor -- start 2024-04-25T09:15:28.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.522+0000 7f0c77000640 1 -- start start 2024-04-25T09:15:28.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c77000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b27a0 0x7f0c70159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c77000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700b56f0 0x7f0c701499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c77000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c77000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700b2d60 con 0x7f0c700b6a90 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c77000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700cea50 con 0x7f0c700b56f0 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c77000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700b2ee0 con 0x7f0c700b27a0 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c76600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c76600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36818/0 (socket says 172.21.15.62:36818) 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c76600640 1 -- 172.21.15.62:0/1441537507 learned_addr learned my addr 172.21.15.62:0/1441537507 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.523+0000 7f0c75c00640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b27a0 0x7f0c70159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c75200640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700b56f0 0x7f0c701499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c76600640 1 -- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b27a0 msgr2=0x7f0c70159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c76600640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b27a0 0x7f0c70159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c76600640 1 -- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700b56f0 msgr2=0x7f0c701499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c76600640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700b56f0 0x7f0c701499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c76600640 1 -- 172.21.15.62:0/1441537507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700cbc10 con 0x7f0c700b6a90 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c75c00640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b27a0 0x7f0c70159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c76600640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70149f30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800c7f0 tx=0x7f0c6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c6d600640 1 -- 172.21.15.62:0/1441537507 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68015070 con 0x7f0c700b6a90 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c6d600640 1 -- 172.21.15.62:0/1441537507 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c68007820 con 0x7f0c700b6a90 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c6d600640 1 -- 172.21.15.62:0/1441537507 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68007c10 con 0x7f0c700b6a90 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7015b2f0 con 0x7f0c700b6a90 2024-04-25T09:15:28.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.524+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7015b5d0 con 0x7f0c700b6a90 2024-04-25T09:15:28.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.526+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7004ef20 con 0x7f0c700b6a90 2024-04-25T09:15:28.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.526+0000 7f0c6d600640 1 -- 172.21.15.62:0/1441537507 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c68018070 con 0x7f0c700b6a90 2024-04-25T09:15:28.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.528+0000 7f0c6d600640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0c5006b500 0x7f0c5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.528+0000 7f0c6d600640 1 -- 172.21.15.62:0/1441537507 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0c6808a370 con 0x7f0c700b6a90 2024-04-25T09:15:28.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.529+0000 7f0c75c00640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0c5006b500 0x7f0c5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.530+0000 7f0c75c00640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0c5006b500 0x7f0c5006d9c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400d430 tx=0x7f0c64007530 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:28.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.533+0000 7f0c6d600640 1 -- 172.21.15.62:0/1441537507 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c68059260 con 0x7f0c700b6a90 2024-04-25T09:15:28.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.774+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c700ce580 con 0x7f0c700b6a90 2024-04-25T09:15:28.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.775+0000 7f0c6d600640 1 -- 172.21.15.62:0/1441537507 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0c68058b60 con 0x7f0c700b6a90 2024-04-25T09:15:28.762 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:28.762 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:28.762 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.777+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0c5006b500 msgr2=0x7f0c5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.777+0000 7f0c77000640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0c5006b500 0x7f0c5006d9c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400d430 tx=0x7f0c64007530 comp rx=0 tx=0).stop 2024-04-25T09:15:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.777+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 msgr2=0x7f0c70149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:28.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.777+0000 7f0c77000640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70149f30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800c7f0 tx=0x7f0c6800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.778+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 shutdown_connections 2024-04-25T09:15:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.778+0000 7f0c77000640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0c5006b500 0x7f0c5006d9c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.778+0000 7f0c77000640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0c700b6a90 0x7f0c70149f30 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.778+0000 7f0c77000640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c700b56f0 0x7f0c701499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.778+0000 7f0c77000640 1 --2- 172.21.15.62:0/1441537507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0c700b27a0 0x7f0c70159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.778+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 >> 172.21.15.62:0/1441537507 conn(0x7f0c70066d10 msgr2=0x7f0c700b85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.778+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 shutdown_connections 2024-04-25T09:15:28.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:28.778+0000 7f0c77000640 1 -- 172.21.15.62:0/1441537507 wait complete. 2024-04-25T09:15:28.783 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:15:28.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:15:28.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.006+0000 7f9dc5800640 1 Processor -- start 2024-04-25T09:15:28.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.006+0000 7f9dc5800640 1 -- start start 2024-04-25T09:15:28.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.007+0000 7f9dc5800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9dc015c120 0x7f9dc00ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.007+0000 7f9dc5800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9dc00b1da0 0x7f9dc00cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:28.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.007+0000 7f9dc5800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc015c670 con 0x7f9dc00b19c0 2024-04-25T09:15:28.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.007+0000 7f9dc5800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc00cf5c0 con 0x7f9dc015c120 2024-04-25T09:15:28.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.007+0000 7f9dc5800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc015c950 con 0x7f9dc00b1da0 2024-04-25T09:15:28.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbfe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9dc00b19c0 0x7f9dc00ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:32794/0 (socket says 172.21.15.62:32794) 2024-04-25T09:15:28.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbfe00640 1 -- 172.21.15.62:0/1128896445 learned_addr learned my addr 172.21.15.62:0/1128896445 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:28.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbf400640 1 --2- 172.21.15.62:0/1128896445 >> v2:172.21.15.62:3300/0 conn(0x7f9dc015c120 0x7f9dc00ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbea00640 1 -- 172.21.15.62:0/1128896445 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789603288 0 0) 0x7f9dc015c670 con 0x7f9dc00b19c0 2024-04-25T09:15:28.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbea00640 1 -- 172.21.15.62:0/1128896445 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da4003520 con 0x7f9dc00b19c0 2024-04-25T09:15:28.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dc4e00640 1 --2- 172.21.15.62:0/1128896445 >> v2:172.21.15.121:3300/0 conn(0x7f9dc00b1da0 0x7f9dc00cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:28.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbf400640 1 -- 172.21.15.62:0/1128896445 >> v2:172.21.15.121:3300/0 conn(0x7f9dc00b1da0 msgr2=0x7f9dc00cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:28.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbf400640 1 --2- 172.21.15.62:0/1128896445 >> v2:172.21.15.121:3300/0 conn(0x7f9dc00b1da0 0x7f9dc00cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbf400640 1 -- 172.21.15.62:0/1128896445 >> v1:172.21.15.62:6789/0 conn(0x7f9dc00b19c0 legacy=0x7f9dc00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dbf400640 1 -- 172.21.15.62:0/1128896445 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc015cc30 con 0x7f9dc015c120 2024-04-25T09:15:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.008+0000 7f9dc4e00640 1 --2- 172.21.15.62:0/1128896445 >> v2:172.21.15.121:3300/0 conn(0x7f9dc00b1da0 0x7f9dc00cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:28.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.009+0000 7f9dbf400640 1 --2- 172.21.15.62:0/1128896445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc00ccbe0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00677b0 tx=0x7f9dac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da7cf53119a191f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dbea00640 1 -- 172.21.15.62:0/1128896445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac037070 con 0x7f9dc015c120 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dbea00640 1 -- 172.21.15.62:0/1128896445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac032440 con 0x7f9dc015c120 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dbea00640 1 -- 172.21.15.62:0/1128896445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac032810 con 0x7f9dc015c120 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dc5800640 1 -- 172.21.15.62:0/1128896445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 msgr2=0x7f9dc00ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dc5800640 1 --2- 172.21.15.62:0/1128896445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc00ccbe0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00677b0 tx=0x7f9dac02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dc5800640 1 -- 172.21.15.62:0/1128896445 shutdown_connections 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dc5800640 1 --2- 172.21.15.62:0/1128896445 >> v2:172.21.15.121:3300/0 conn(0x7f9dc00b1da0 0x7f9dc00cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dc5800640 1 --2- 172.21.15.62:0/1128896445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc00ccbe0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dc5800640 1 -- 172.21.15.62:0/1128896445 >> 172.21.15.62:0/1128896445 conn(0x7f9dc00668f0 msgr2=0x7f9dc0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dc5800640 1 -- 172.21.15.62:0/1128896445 shutdown_connections 2024-04-25T09:15:28.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.010+0000 7f9dc5800640 1 -- 172.21.15.62:0/1128896445 wait complete. 2024-04-25T09:15:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.012+0000 7f9dc5800640 1 Processor -- start 2024-04-25T09:15:28.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.012+0000 7f9dc5800640 1 -- start start 2024-04-25T09:15:29.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.013+0000 7f9dc5800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc00b19c0 0x7f9dc01431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:29.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.013+0000 7f9dc5800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc00b1da0 0x7f9dc0158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:29.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.013+0000 7f9dc5800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc0158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:29.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.013+0000 7f9dc5800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00cd7e0 con 0x7f9dc015c120 2024-04-25T09:15:29.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.013+0000 7f9dc5800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00cd660 con 0x7f9dc00b19c0 2024-04-25T09:15:29.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.013+0000 7f9dc5800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00cb7d0 con 0x7f9dc00b1da0 2024-04-25T09:15:29.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc0158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:29.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc0158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36826/0 (socket says 172.21.15.62:36826) 2024-04-25T09:15:29.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 -- 172.21.15.62:0/4293671206 learned_addr learned my addr 172.21.15.62:0/4293671206 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:29.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dbf400640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc00b1da0 0x7f9dc0158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dbfe00640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc00b19c0 0x7f9dc01431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 -- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc00b1da0 msgr2=0x7f9dc0158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc00b1da0 0x7f9dc0158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 -- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc00b19c0 msgr2=0x7f9dc01431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc00b19c0 0x7f9dc01431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 -- 172.21.15.62:0/4293671206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc01f1a20 con 0x7f9dc015c120 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dbf400640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc00b1da0 0x7f9dc0158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.014+0000 7f9dc4e00640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc0158780 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c7f0 tx=0x7f9db800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.015+0000 7f9db7e00640 1 -- 172.21.15.62:0/4293671206 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8015070 con 0x7f9dc015c120 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.015+0000 7f9db7e00640 1 -- 172.21.15.62:0/4293671206 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db8007820 con 0x7f9dc015c120 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.015+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc0157230 con 0x7f9dc015c120 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.015+0000 7f9db7e00640 1 -- 172.21.15.62:0/4293671206 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8007c50 con 0x7f9dc015c120 2024-04-25T09:15:29.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.015+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc01577f0 con 0x7f9dc015c120 2024-04-25T09:15:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.016+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0072b10 con 0x7f9dc015c120 2024-04-25T09:15:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.017+0000 7f9db7e00640 1 -- 172.21.15.62:0/4293671206 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9db8018070 con 0x7f9dc015c120 2024-04-25T09:15:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.019+0000 7f9db7e00640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9da006b500 0x7f9da006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.019+0000 7f9db7e00640 1 -- 172.21.15.62:0/4293671206 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9db808a400 con 0x7f9dc015c120 2024-04-25T09:15:29.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.019+0000 7f9dbfe00640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9da006b500 0x7f9da006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:29.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.021+0000 7f9dbfe00640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9da006b500 0x7f9da006d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005c00 tx=0x7f9db0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:29.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.023+0000 7f9db7e00640 1 -- 172.21.15.62:0/4293671206 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9db80591a0 con 0x7f9dc015c120 2024-04-25T09:15:29.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.296+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9dc00750e0 con 0x7f9dc015c120 2024-04-25T09:15:29.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.297+0000 7f9db7e00640 1 -- 172.21.15.62:0/4293671206 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7f9db8058aa0 con 0x7f9dc015c120 2024-04-25T09:15:29.284 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:29.284 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":756,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076261340,"bytes_used":2600755200,"bytes_avail":770493358080,"bytes_total":773094113280,"write_bytes_sec":28889803,"read_op_per_sec":0,"write_op_per_sec":29},"fsmap":{"epoch":78,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T09:14:26.142824+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9da006b500 msgr2=0x7f9da006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9da006b500 0x7f9da006d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005c00 tx=0x7f9db0009040 comp rx=0 tx=0).stop 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 msgr2=0x7f9dc0158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc0158780 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c7f0 tx=0x7f9db800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 shutdown_connections 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9da006b500 0x7f9da006d9c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dc015c120 0x7f9dc0158780 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9dc00b1da0 0x7f9dc0158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 --2- 172.21.15.62:0/4293671206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dc00b19c0 0x7f9dc01431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:29.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 >> 172.21.15.62:0/4293671206 conn(0x7f9dc00668f0 msgr2=0x7f9dc006d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 shutdown_connections 2024-04-25T09:15:29.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:29.300+0000 7f9dc5800640 1 -- 172.21.15.62:0/4293671206 wait complete. 2024-04-25T09:15:29.305 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:15:30.307 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.538+0000 7f75dd400640 1 Processor -- start 2024-04-25T09:15:30.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.539+0000 7f75dd400640 1 -- start start 2024-04-25T09:15:30.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.540+0000 7f75dd400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75d815b900 0x7f75d80d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.540+0000 7f75dd400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f75d80b6900 0x7f75d80b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.540+0000 7f75dd400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d80cf890 con 0x7f75d80d1cc0 2024-04-25T09:15:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.540+0000 7f75dd400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d8070a00 con 0x7f75d815b900 2024-04-25T09:15:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.540+0000 7f75dd400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d806fdf0 con 0x7f75d80b6900 2024-04-25T09:15:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75d7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75d815b900 0x7f75d80d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75d7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f75d815b900 0x7f75d80d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36832/0 (socket says 172.21.15.62:36832) 2024-04-25T09:15:30.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75d7e00640 1 -- 172.21.15.62:0/2159829540 learned_addr learned my addr 172.21.15.62:0/2159829540 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75dca00640 1 --2- 172.21.15.62:0/2159829540 >> v2:172.21.15.121:3300/0 conn(0x7f75d80b6900 0x7f75d80b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75d7e00640 1 -- 172.21.15.62:0/2159829540 >> v2:172.21.15.121:3300/0 conn(0x7f75d80b6900 msgr2=0x7f75d80b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75d7e00640 1 --2- 172.21.15.62:0/2159829540 >> v2:172.21.15.121:3300/0 conn(0x7f75d80b6900 0x7f75d80b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75d7e00640 1 -- 172.21.15.62:0/2159829540 >> v1:172.21.15.62:6789/0 conn(0x7f75d80d1cc0 legacy=0x7f75d80b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75d7e00640 1 -- 172.21.15.62:0/2159829540 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80700d0 con 0x7f75d815b900 2024-04-25T09:15:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.541+0000 7f75d6a00640 1 -- 172.21.15.62:0/2159829540 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109127728 0 0) 0x7f75c0001110 con 0x7f75d80d1cc0 2024-04-25T09:15:30.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.543+0000 7f75d7e00640 1 --2- 172.21.15.62:0/2159829540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80d1780 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f75d80722d0 tx=0x7f75c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450fc2db07473bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.543+0000 7f75d6a00640 1 -- 172.21.15.62:0/2159829540 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8036070 con 0x7f75d815b900 2024-04-25T09:15:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.543+0000 7f75d6a00640 1 -- 172.21.15.62:0/2159829540 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8031600 con 0x7f75d815b900 2024-04-25T09:15:30.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.543+0000 7f75d6a00640 1 -- 172.21.15.62:0/2159829540 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c80319d0 con 0x7f75d815b900 2024-04-25T09:15:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.543+0000 7f75dd400640 1 -- 172.21.15.62:0/2159829540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 msgr2=0x7f75d80d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.543+0000 7f75dd400640 1 --2- 172.21.15.62:0/2159829540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80d1780 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f75d80722d0 tx=0x7f75c802a730 comp rx=0 tx=0).stop 2024-04-25T09:15:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.544+0000 7f75dd400640 1 -- 172.21.15.62:0/2159829540 shutdown_connections 2024-04-25T09:15:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.544+0000 7f75dd400640 1 --2- 172.21.15.62:0/2159829540 >> v2:172.21.15.121:3300/0 conn(0x7f75d80b6900 0x7f75d80b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.544+0000 7f75dd400640 1 --2- 172.21.15.62:0/2159829540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80d1780 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.544+0000 7f75dd400640 1 -- 172.21.15.62:0/2159829540 >> 172.21.15.62:0/2159829540 conn(0x7f75d8072340 msgr2=0x7f75d8072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:30.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.544+0000 7f75dd400640 1 -- 172.21.15.62:0/2159829540 shutdown_connections 2024-04-25T09:15:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.544+0000 7f75dd400640 1 -- 172.21.15.62:0/2159829540 wait complete. 2024-04-25T09:15:30.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.545+0000 7f75dd400640 1 Processor -- start 2024-04-25T09:15:30.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.546+0000 7f75dd400640 1 -- start start 2024-04-25T09:15:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.546+0000 7f75dd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75d80b6900 0x7f75d814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:30.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.546+0000 7f75dd400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75d80d1cc0 0x7f75d80b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dd400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d815bce0 con 0x7f75d815b900 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dd400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8070690 con 0x7f75d80b6900 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dd400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d815be60 con 0x7f75d80d1cc0 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36834/0 (socket says 172.21.15.62:36834) 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dca00640 1 -- 172.21.15.62:0/4180806554 learned_addr learned my addr 172.21.15.62:0/4180806554 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75d7e00640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75d80b6900 0x7f75d814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75d7400640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75d80d1cc0 0x7f75d80b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dca00640 1 -- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75d80d1cc0 msgr2=0x7f75d80b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dca00640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75d80d1cc0 0x7f75d80b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dca00640 1 -- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75d80b6900 msgr2=0x7f75d814e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dca00640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75d80b6900 0x7f75d814e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75dca00640 1 -- 172.21.15.62:0/4180806554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d815e7c0 con 0x7f75d815b900 2024-04-25T09:15:30.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75d7400640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75d80d1cc0 0x7f75d80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.547+0000 7f75d7e00640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75d80b6900 0x7f75d814e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.548+0000 7f75dca00640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80b3db0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d5d0 tx=0x7f75d000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:30.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.548+0000 7f75cfe00640 1 -- 172.21.15.62:0/4180806554 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d00125f0 con 0x7f75d815b900 2024-04-25T09:15:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.548+0000 7f75cfe00640 1 -- 172.21.15.62:0/4180806554 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0019070 con 0x7f75d815b900 2024-04-25T09:15:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.548+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d815da00 con 0x7f75d815b900 2024-04-25T09:15:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.548+0000 7f75cfe00640 1 -- 172.21.15.62:0/4180806554 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d00129d0 con 0x7f75d815b900 2024-04-25T09:15:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.548+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d815dfc0 con 0x7f75d815b900 2024-04-25T09:15:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.550+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d804ef20 con 0x7f75d815b900 2024-04-25T09:15:30.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.550+0000 7f75cfe00640 1 -- 172.21.15.62:0/4180806554 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75d0012b70 con 0x7f75d815b900 2024-04-25T09:15:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.551+0000 7f75cfe00640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f75b406b4d0 0x7f75b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:30.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.552+0000 7f75cfe00640 1 -- 172.21.15.62:0/4180806554 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f75d008a650 con 0x7f75d815b900 2024-04-25T09:15:30.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.552+0000 7f75d7e00640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f75b406b4d0 0x7f75b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:30.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.553+0000 7f75d7e00640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f75b406b4d0 0x7f75b406d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75c8033000 tx=0x7f75c8009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:30.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.556+0000 7f75cfe00640 1 -- 172.21.15.62:0/4180806554 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75d0059520 con 0x7f75d815b900 2024-04-25T09:15:30.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.792+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75d806d420 con 0x7f75d815b900 2024-04-25T09:15:30.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.794+0000 7f75cfe00640 1 -- 172.21.15.62:0/4180806554 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f75d0058e20 con 0x7f75d815b900 2024-04-25T09:15:30.781 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:30.781 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:30.781 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:30.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f75b406b4d0 msgr2=0x7f75b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f75b406b4d0 0x7f75b406d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75c8033000 tx=0x7f75c8009000 comp rx=0 tx=0).stop 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 msgr2=0x7f75d80b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80b3db0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d5d0 tx=0x7f75d000da90 comp rx=0 tx=0).stop 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 shutdown_connections 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f75b406b4d0 0x7f75b406d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75d815b900 0x7f75d80b3db0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f75d80d1cc0 0x7f75d80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 --2- 172.21.15.62:0/4180806554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75d80b6900 0x7f75d814e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 >> 172.21.15.62:0/4180806554 conn(0x7f75d8072340 msgr2=0x7f75d8155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.796+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 shutdown_connections 2024-04-25T09:15:30.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:30.797+0000 7f75dd400640 1 -- 172.21.15.62:0/4180806554 wait complete. 2024-04-25T09:15:30.803 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:15:30.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:15:31.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.029+0000 7f81bbe00640 1 Processor -- start 2024-04-25T09:15:31.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.029+0000 7f81bbe00640 1 -- start start 2024-04-25T09:15:31.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.031+0000 7f81bbe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f81bc0b19c0 0x7f81bc0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:31.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.031+0000 7f81bbe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f81bc15c120 0x7f81bc0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:31.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.031+0000 7f81bbe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc15c670 con 0x7f81bc0b1da0 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.031+0000 7f81bbe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0cd830 con 0x7f81bc0b19c0 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.031+0000 7f81bbe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc15c950 con 0x7f81bc15c120 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.031+0000 7f81baa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f81bc0b19c0 0x7f81bc0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.031+0000 7f81baa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f81bc0b19c0 0x7f81bc0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36848/0 (socket says 172.21.15.62:36848) 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.031+0000 7f81baa00640 1 -- 172.21.15.62:0/844883292 learned_addr learned my addr 172.21.15.62:0/844883292 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.032+0000 7f81b2000640 1 --2- 172.21.15.62:0/844883292 >> v2:172.21.15.121:3300/0 conn(0x7f81bc15c120 0x7f81bc0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.032+0000 7f81baa00640 1 -- 172.21.15.62:0/844883292 >> v2:172.21.15.121:3300/0 conn(0x7f81bc15c120 msgr2=0x7f81bc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.032+0000 7f81baa00640 1 --2- 172.21.15.62:0/844883292 >> v2:172.21.15.121:3300/0 conn(0x7f81bc15c120 0x7f81bc0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.032+0000 7f81baa00640 1 -- 172.21.15.62:0/844883292 >> v1:172.21.15.62:6789/0 conn(0x7f81bc0b1da0 legacy=0x7f81bc0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.032+0000 7f81baa00640 1 -- 172.21.15.62:0/844883292 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc15cc30 con 0x7f81bc0b19c0 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.032+0000 7f81ba000640 1 -- 172.21.15.62:0/844883292 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341280045 0 0) 0x7f81bc15c670 con 0x7f81bc0b1da0 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.032+0000 7f81b2000640 1 --2- 172.21.15.62:0/844883292 >> v2:172.21.15.121:3300/0 conn(0x7f81bc15c120 0x7f81bc0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.033+0000 7f81baa00640 1 --2- 172.21.15.62:0/844883292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc0ceeb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0725b0 tx=0x7f81ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e295365dddcf81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.033+0000 7f81ba000640 1 -- 172.21.15.62:0/844883292 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac037070 con 0x7f81bc0b19c0 2024-04-25T09:15:31.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81ba000640 1 -- 172.21.15.62:0/844883292 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac032440 con 0x7f81bc0b19c0 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81ba000640 1 -- 172.21.15.62:0/844883292 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac032810 con 0x7f81bc0b19c0 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81bbe00640 1 -- 172.21.15.62:0/844883292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 msgr2=0x7f81bc0ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81bbe00640 1 --2- 172.21.15.62:0/844883292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc0ceeb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0725b0 tx=0x7f81ac02a270 comp rx=0 tx=0).stop 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81bbe00640 1 -- 172.21.15.62:0/844883292 shutdown_connections 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81bbe00640 1 --2- 172.21.15.62:0/844883292 >> v2:172.21.15.121:3300/0 conn(0x7f81bc15c120 0x7f81bc0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81bbe00640 1 --2- 172.21.15.62:0/844883292 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc0ceeb0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81bbe00640 1 -- 172.21.15.62:0/844883292 >> 172.21.15.62:0/844883292 conn(0x7f81bc0668f0 msgr2=0x7f81bc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81bbe00640 1 -- 172.21.15.62:0/844883292 shutdown_connections 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.034+0000 7f81bbe00640 1 -- 172.21.15.62:0/844883292 wait complete. 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.036+0000 7f81bbe00640 1 Processor -- start 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.036+0000 7f81bbe00640 1 -- start start 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:31.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81bbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81bc0b1da0 0x7f81bc1f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81bbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81bc15c120 0x7f81bc143d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81bbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0cf570 con 0x7f81bc0b19c0 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81bbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0cf3f0 con 0x7f81bc15c120 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81bbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0cb7d0 con 0x7f81bc0b1da0 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81baa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc142d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36850/0 (socket says 172.21.15.62:36850) 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.037+0000 7f81baa00640 1 -- 172.21.15.62:0/1612122226 learned_addr learned my addr 172.21.15.62:0/1612122226 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.038+0000 7f81baa00640 1 -- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81bc0b1da0 msgr2=0x7f81bc1f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.038+0000 7f81baa00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81bc0b1da0 0x7f81bc1f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.038+0000 7f81baa00640 1 -- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81bc15c120 msgr2=0x7f81bc143d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.038+0000 7f81baa00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81bc15c120 0x7f81bc143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.038+0000 7f81baa00640 1 -- 172.21.15.62:0/1612122226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc139e90 con 0x7f81bc0b19c0 2024-04-25T09:15:31.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.038+0000 7f81baa00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc142d50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f81ac02a2b0 tx=0x7f81ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.039+0000 7f81b3400640 1 -- 172.21.15.62:0/1612122226 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac037040 con 0x7f81bc0b19c0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.039+0000 7f81b3400640 1 -- 172.21.15.62:0/1612122226 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac042070 con 0x7f81bc0b19c0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.039+0000 7f81b3400640 1 -- 172.21.15.62:0/1612122226 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac030bb0 con 0x7f81bc0b19c0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.039+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc143290 con 0x7f81bc0b19c0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.039+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc1590c0 con 0x7f81bc0b19c0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.040+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc072b10 con 0x7f81bc0b19c0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.041+0000 7f81b3400640 1 -- 172.21.15.62:0/1612122226 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81ac047460 con 0x7f81bc0b19c0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.042+0000 7f81b3400640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.042+0000 7f81b3400640 1 -- 172.21.15.62:0/1612122226 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f81ac0aaf30 con 0x7f81bc0b19c0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.043+0000 7f81b2000640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.044+0000 7f81b2000640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f819806b4d0 0x7f819806d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f81a4006fd0 tx=0x7f81a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:31.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.046+0000 7f81b3400640 1 -- 172.21.15.62:0/1612122226 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81bc139e90 con 0x7f81bc0b19c0 2024-04-25T09:15:31.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.317+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f81bc06d8a0 con 0x7f81bc0b19c0 2024-04-25T09:15:31.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.318+0000 7f81b3400640 1 -- 172.21.15.62:0/1612122226 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7f81ac045360 con 0x7f81bc0b19c0 2024-04-25T09:15:31.306 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:31.306 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":758,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076261340,"bytes_used":2600755200,"bytes_avail":770493358080,"bytes_total":773094113280,"write_bytes_sec":28891819,"read_op_per_sec":0,"write_op_per_sec":29},"fsmap":{"epoch":78,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T09:14:26.142824+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:15:31.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f819806b4d0 msgr2=0x7f819806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:31.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f819806b4d0 0x7f819806d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f81a4006fd0 tx=0x7f81a4008040 comp rx=0 tx=0).stop 2024-04-25T09:15:31.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 msgr2=0x7f81bc142d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:31.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc142d50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f81ac02a2b0 tx=0x7f81ac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 shutdown_connections 2024-04-25T09:15:31.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f81bc15c120 0x7f81bc143d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f81bc0b1da0 0x7f81bc1f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 --2- 172.21.15.62:0/1612122226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f81bc0b19c0 0x7f81bc142d50 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:31.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 >> 172.21.15.62:0/1612122226 conn(0x7f81bc0668f0 msgr2=0x7f81bc0719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:31.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 shutdown_connections 2024-04-25T09:15:31.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:31.321+0000 7f81bbe00640 1 -- 172.21.15.62:0/1612122226 wait complete. 2024-04-25T09:15:31.327 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:15:32.328 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.564+0000 7f07af200640 1 Processor -- start 2024-04-25T09:15:32.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.564+0000 7f07af200640 1 -- start start 2024-04-25T09:15:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.565+0000 7f07af200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07b00b56f0 0x7f07b0160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.565+0000 7f07af200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f07b00b6a90 0x7f07b0161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.565+0000 7f07af200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b00d3110 con 0x7f07b00b27a0 2024-04-25T09:15:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.565+0000 7f07af200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b0161780 con 0x7f07b00b56f0 2024-04-25T09:15:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.565+0000 7f07af200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00cedc0 con 0x7f07b00b6a90 2024-04-25T09:15:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.566+0000 7f07ade00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07b00b56f0 0x7f07b0160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.566+0000 7f07ade00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f07b00b56f0 0x7f07b0160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36852/0 (socket says 172.21.15.62:36852) 2024-04-25T09:15:32.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.566+0000 7f07ade00640 1 -- 172.21.15.62:0/260203686 learned_addr learned my addr 172.21.15.62:0/260203686 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.566+0000 7f07ade00640 1 -- 172.21.15.62:0/260203686 >> v2:172.21.15.121:3300/0 conn(0x7f07b00b6a90 msgr2=0x7f07b0161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.566+0000 7f07ade00640 1 --2- 172.21.15.62:0/260203686 >> v2:172.21.15.121:3300/0 conn(0x7f07b00b6a90 0x7f07b0161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.566+0000 7f07ade00640 1 -- 172.21.15.62:0/260203686 >> v1:172.21.15.62:6789/0 conn(0x7f07b00b27a0 legacy=0x7f07b0161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:32.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.566+0000 7f07ade00640 1 -- 172.21.15.62:0/260203686 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b00d33f0 con 0x7f07b00b56f0 2024-04-25T09:15:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.567+0000 7f07ade00640 1 --2- 172.21.15.62:0/260203686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b56f0 0x7f07b0160d00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f07b008c570 tx=0x7f07a002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df733c6d0f3ef5d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.567+0000 7f07aca00640 1 -- 172.21.15.62:0/260203686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0036070 con 0x7f07b00b56f0 2024-04-25T09:15:32.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07aca00640 1 -- 172.21.15.62:0/260203686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a0031600 con 0x7f07b00b56f0 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07aca00640 1 -- 172.21.15.62:0/260203686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a00319d0 con 0x7f07b00b56f0 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07af200640 1 -- 172.21.15.62:0/260203686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b56f0 msgr2=0x7f07b0160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07af200640 1 --2- 172.21.15.62:0/260203686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b56f0 0x7f07b0160d00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f07b008c570 tx=0x7f07a002a730 comp rx=0 tx=0).stop 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07af200640 1 -- 172.21.15.62:0/260203686 shutdown_connections 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07af200640 1 --2- 172.21.15.62:0/260203686 >> v2:172.21.15.121:3300/0 conn(0x7f07b00b6a90 0x7f07b0161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07af200640 1 --2- 172.21.15.62:0/260203686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b56f0 0x7f07b0160d00 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07af200640 1 -- 172.21.15.62:0/260203686 >> 172.21.15.62:0/260203686 conn(0x7f07b0066d10 msgr2=0x7f07b0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07af200640 1 -- 172.21.15.62:0/260203686 shutdown_connections 2024-04-25T09:15:32.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.568+0000 7f07af200640 1 -- 172.21.15.62:0/260203686 wait complete. 2024-04-25T09:15:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.570+0000 7f07af200640 1 Processor -- start 2024-04-25T09:15:32.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.570+0000 7f07af200640 1 -- start start 2024-04-25T09:15:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07af200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07b00b27a0 0x7f07b01f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07af200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07b00b56f0 0x7f07b00cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07af200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b6a90 0x7f07b0150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07af200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00b2d60 con 0x7f07b00b6a90 2024-04-25T09:15:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07af200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00cea50 con 0x7f07b00b27a0 2024-04-25T09:15:32.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07af200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00b2ee0 con 0x7f07b00b56f0 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07ae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b6a90 0x7f07b0150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07ae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b6a90 0x7f07b0150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36854/0 (socket says 172.21.15.62:36854) 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.571+0000 7f07ae800640 1 -- 172.21.15.62:0/2568212974 learned_addr learned my addr 172.21.15.62:0/2568212974 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.572+0000 7f07ade00640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07b00b27a0 0x7f07b01f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.572+0000 7f07ad400640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07b00b56f0 0x7f07b00cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.572+0000 7f07ae800640 1 -- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07b00b56f0 msgr2=0x7f07b00cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.572+0000 7f07ae800640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07b00b56f0 0x7f07b00cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.572+0000 7f07ae800640 1 -- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07b00b27a0 msgr2=0x7f07b01f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.572+0000 7f07ae800640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07b00b27a0 0x7f07b01f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.572+0000 7f07ae800640 1 -- 172.21.15.62:0/2568212974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b01e67a0 con 0x7f07b00b6a90 2024-04-25T09:15:32.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.572+0000 7f07ae800640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b6a90 0x7f07b0150a40 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c3a0 tx=0x7f07a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:32.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.573+0000 7f07a6000640 1 -- 172.21.15.62:0/2568212974 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8017070 con 0x7f07b00b6a90 2024-04-25T09:15:32.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.573+0000 7f07a6000640 1 -- 172.21.15.62:0/2568212974 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a80052d0 con 0x7f07b00b6a90 2024-04-25T09:15:32.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.573+0000 7f07a6000640 1 -- 172.21.15.62:0/2568212974 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8003830 con 0x7f07b00b6a90 2024-04-25T09:15:32.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.573+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b01e6a80 con 0x7f07b00b6a90 2024-04-25T09:15:32.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.573+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b01e7040 con 0x7f07b00b6a90 2024-04-25T09:15:32.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.574+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b004ef20 con 0x7f07b00b6a90 2024-04-25T09:15:32.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.575+0000 7f07a6000640 1 -- 172.21.15.62:0/2568212974 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07a801a070 con 0x7f07b00b6a90 2024-04-25T09:15:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.576+0000 7f07a6000640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f078c06b4d0 0x7f078c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.576+0000 7f07a6000640 1 -- 172.21.15.62:0/2568212974 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f07a80895d0 con 0x7f07b00b6a90 2024-04-25T09:15:32.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.576+0000 7f07ade00640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f078c06b4d0 0x7f078c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:32.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.578+0000 7f07ade00640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f078c06b4d0 0x7f078c06d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f07a0033000 tx=0x7f07a0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:32.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.580+0000 7f07a6000640 1 -- 172.21.15.62:0/2568212974 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07a80584c0 con 0x7f07b00b6a90 2024-04-25T09:15:32.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.818+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07b00b6180 con 0x7f07b00b6a90 2024-04-25T09:15:32.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.819+0000 7f07a6000640 1 -- 172.21.15.62:0/2568212974 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f07a8057dc0 con 0x7f07b00b6a90 2024-04-25T09:15:32.807 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:32.807 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:32.807 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.821+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f078c06b4d0 msgr2=0x7f078c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f078c06b4d0 0x7f078c06d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f07a0033000 tx=0x7f07a0009000 comp rx=0 tx=0).stop 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b6a90 msgr2=0x7f07b0150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b6a90 0x7f07b0150a40 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c3a0 tx=0x7f07a800c860 comp rx=0 tx=0).stop 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 shutdown_connections 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f078c06b4d0 0x7f078c06d990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07b00b6a90 0x7f07b0150a40 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07b00b56f0 0x7f07b00cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 --2- 172.21.15.62:0/2568212974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f07b00b27a0 0x7f07b01f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 >> 172.21.15.62:0/2568212974 conn(0x7f07b0066d10 msgr2=0x7f07b00c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:32.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 shutdown_connections 2024-04-25T09:15:32.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:32.822+0000 7f07af200640 1 -- 172.21.15.62:0/2568212974 wait complete. 2024-04-25T09:15:32.829 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:15:32.829 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:15:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.044+0000 7f2e4fe00640 1 Processor -- start 2024-04-25T09:15:33.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.044+0000 7f2e4fe00640 1 -- start start 2024-04-25T09:15:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.045+0000 7f2e4fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e50157ef0 0x7f2e500b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.046+0000 7f2e4fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2e500b3740 0x7f2e5015a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.046+0000 7f2e4fe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e500b3e10 con 0x7f2e5015ac60 2024-04-25T09:15:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.046+0000 7f2e4fe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e500b3bb0 con 0x7f2e50157ef0 2024-04-25T09:15:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.046+0000 7f2e4fe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e500b40f0 con 0x7f2e500b3740 2024-04-25T09:15:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.046+0000 7f2e4ea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e50157ef0 0x7f2e500b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.046+0000 7f2e4f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2e5015ac60 0x7f2e500cdd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:32840/0 (socket says 172.21.15.62:32840) 2024-04-25T09:15:33.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.046+0000 7f2e4f400640 1 -- 172.21.15.62:0/4159720475 learned_addr learned my addr 172.21.15.62:0/4159720475 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.046+0000 7f2e4e000640 1 --2- 172.21.15.62:0/4159720475 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b3740 0x7f2e5015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.047+0000 7f2e4ea00640 1 -- 172.21.15.62:0/4159720475 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b3740 msgr2=0x7f2e5015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.047+0000 7f2e4ea00640 1 --2- 172.21.15.62:0/4159720475 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b3740 0x7f2e5015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.047+0000 7f2e4ea00640 1 -- 172.21.15.62:0/4159720475 >> v1:172.21.15.62:6789/0 conn(0x7f2e5015ac60 legacy=0x7f2e500cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.047+0000 7f2e4ea00640 1 -- 172.21.15.62:0/4159720475 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e500cd590 con 0x7f2e50157ef0 2024-04-25T09:15:33.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.047+0000 7f2e4e000640 1 --2- 172.21.15.62:0/4159720475 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b3740 0x7f2e5015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:33.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.048+0000 7f2e4ea00640 1 --2- 172.21.15.62:0/4159720475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e50157ef0 0x7f2e500b3200 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f2e500ccdc0 tx=0x7f2e4002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe2cb56459f80a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.048+0000 7f2e4d600640 1 -- 172.21.15.62:0/4159720475 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40037070 con 0x7f2e50157ef0 2024-04-25T09:15:33.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.048+0000 7f2e4d600640 1 -- 172.21.15.62:0/4159720475 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e40032440 con 0x7f2e50157ef0 2024-04-25T09:15:33.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.048+0000 7f2e4d600640 1 -- 172.21.15.62:0/4159720475 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40032810 con 0x7f2e50157ef0 2024-04-25T09:15:33.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.049+0000 7f2e4fe00640 1 -- 172.21.15.62:0/4159720475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e50157ef0 msgr2=0x7f2e500b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.049+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/4159720475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e50157ef0 0x7f2e500b3200 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f2e500ccdc0 tx=0x7f2e4002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.049+0000 7f2e4fe00640 1 -- 172.21.15.62:0/4159720475 shutdown_connections 2024-04-25T09:15:33.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.049+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/4159720475 >> v2:172.21.15.121:3300/0 conn(0x7f2e500b3740 0x7f2e5015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.049+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/4159720475 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e50157ef0 0x7f2e500b3200 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.049+0000 7f2e4fe00640 1 -- 172.21.15.62:0/4159720475 >> 172.21.15.62:0/4159720475 conn(0x7f2e500668f0 msgr2=0x7f2e50066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.049+0000 7f2e4fe00640 1 -- 172.21.15.62:0/4159720475 shutdown_connections 2024-04-25T09:15:33.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.049+0000 7f2e4fe00640 1 -- 172.21.15.62:0/4159720475 wait complete. 2024-04-25T09:15:33.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.051+0000 7f2e4fe00640 1 Processor -- start 2024-04-25T09:15:33.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.051+0000 7f2e4fe00640 1 -- start start 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e500b3740 0x7f2e501f0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50157ef0 0x7f2e50159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e5015ac60 0x7f2e50148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5015b070 con 0x7f2e500b3740 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e500cd280 con 0x7f2e5015ac60 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4fe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5015b1f0 con 0x7f2e50157ef0 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e500b3740 0x7f2e501f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e500b3740 0x7f2e501f0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36886/0 (socket says 172.21.15.62:36886) 2024-04-25T09:15:33.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4ea00640 1 -- 172.21.15.62:0/1187990266 learned_addr learned my addr 172.21.15.62:0/1187990266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4ea00640 1 -- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50157ef0 msgr2=0x7f2e50159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.052+0000 7f2e4ea00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50157ef0 0x7f2e50159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e4f400640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e5015ac60 0x7f2e50148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e4ea00640 1 -- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e5015ac60 msgr2=0x7f2e50148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e4ea00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e5015ac60 0x7f2e50148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e4ea00640 1 -- 172.21.15.62:0/1187990266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e50159b50 con 0x7f2e500b3740 2024-04-25T09:15:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e4f400640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e5015ac60 0x7f2e50148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:33.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e4ea00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e500b3740 0x7f2e501f0460 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f2e40000c00 tx=0x7f2e400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e46a00640 1 -- 172.21.15.62:0/1187990266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40041070 con 0x7f2e500b3740 2024-04-25T09:15:33.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e46a00640 1 -- 172.21.15.62:0/1187990266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4003c950 con 0x7f2e500b3740 2024-04-25T09:15:33.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e46a00640 1 -- 172.21.15.62:0/1187990266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40037040 con 0x7f2e500b3740 2024-04-25T09:15:33.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e50159e30 con 0x7f2e500b3740 2024-04-25T09:15:33.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.053+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e501efc60 con 0x7f2e500b3740 2024-04-25T09:15:33.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.055+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e50072b10 con 0x7f2e500b3740 2024-04-25T09:15:33.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.056+0000 7f2e46a00640 1 -- 172.21.15.62:0/1187990266 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e40044070 con 0x7f2e500b3740 2024-04-25T09:15:33.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.056+0000 7f2e46a00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.057+0000 7f2e46a00640 1 -- 172.21.15.62:0/1187990266 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2e400aafe0 con 0x7f2e500b3740 2024-04-25T09:15:33.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.057+0000 7f2e4e000640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.058+0000 7f2e4e000640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2e38006fd0 tx=0x7f2e38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.061+0000 7f2e46a00640 1 -- 172.21.15.62:0/1187990266 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e40079f30 con 0x7f2e500b3740 2024-04-25T09:15:33.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.329+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2e500750e0 con 0x7f2e500b3740 2024-04-25T09:15:33.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.331+0000 7f2e46a00640 1 -- 172.21.15.62:0/1187990266 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f2e40079830 con 0x7f2e500b3740 2024-04-25T09:15:33.318 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:33.318 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":760,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580998,"bytes_used":2612928512,"bytes_avail":770481184768,"bytes_total":773094113280,"write_bytes_sec":28919443,"read_op_per_sec":0,"write_op_per_sec":30},"fsmap":{"epoch":78,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T09:15:32.169417+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:15:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e2c06b4d0 msgr2=0x7f2e2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2e38006fd0 tx=0x7f2e38008040 comp rx=0 tx=0).stop 2024-04-25T09:15:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e500b3740 msgr2=0x7f2e501f0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e500b3740 0x7f2e501f0460 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f2e40000c00 tx=0x7f2e400046c0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 shutdown_connections 2024-04-25T09:15:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2e2c06b4d0 0x7f2e2c06d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e5015ac60 0x7f2e50148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2e50157ef0 0x7f2e50159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 --2- 172.21.15.62:0/1187990266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e500b3740 0x7f2e501f0460 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.333+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 >> 172.21.15.62:0/1187990266 conn(0x7f2e500668f0 msgr2=0x7f2e50156fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.334+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 shutdown_connections 2024-04-25T09:15:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.334+0000 7f2e4fe00640 1 -- 172.21.15.62:0/1187990266 wait complete. 2024-04-25T09:15:33.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:15:33.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:33.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T09:15:33.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.561+0000 7fb246e00640 1 Processor -- start 2024-04-25T09:15:33.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.561+0000 7fb246e00640 1 -- start start 2024-04-25T09:15:33.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.562+0000 7fb246e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb24015dd50 0x7fb2400d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.562+0000 7fb246e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb2400d2fb0 0x7fb2400ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.562+0000 7fb246e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2400ce780 con 0x7fb2400d3960 2024-04-25T09:15:33.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.562+0000 7fb246e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2400d3440 con 0x7fb2400d2fb0 2024-04-25T09:15:33.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.562+0000 7fb246e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400ce1c0 con 0x7fb24015dd50 2024-04-25T09:15:33.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.563+0000 7fb246400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb2400d3960 0x7fb2400d3dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:32848/0 (socket says 172.21.15.62:32848) 2024-04-25T09:15:33.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.563+0000 7fb246400640 1 -- 172.21.15.62:0/722457507 learned_addr learned my addr 172.21.15.62:0/722457507 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:33.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.563+0000 7fb245a00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb24015dd50 0x7fb2400d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.563+0000 7fb245000640 1 --2- 172.21.15.62:0/722457507 >> v2:172.21.15.121:3300/0 conn(0x7fb2400d2fb0 0x7fb2400ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.563+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817234072 0 0) 0x7fb2400ce780 con 0x7fb2400d3960 2024-04-25T09:15:33.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.563+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb224003520 con 0x7fb2400d3960 2024-04-25T09:15:33.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2670200877 0 0) 0x7fb224003520 con 0x7fb2400d3960 2024-04-25T09:15:33.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb24015dd50 msgr2=0x7fb2400d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb24015dd50 0x7fb2400d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 >> v2:172.21.15.121:3300/0 conn(0x7fb2400d2fb0 msgr2=0x7fb2400ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 --2- 172.21.15.62:0/722457507 >> v2:172.21.15.121:3300/0 conn(0x7fb2400d2fb0 0x7fb2400ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2400ce4a0 con 0x7fb2400d3960 2024-04-25T09:15:33.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb2380019f0 con 0x7fb2400d3960 2024-04-25T09:15:33.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 >> v1:172.21.15.62:6789/0 conn(0x7fb2400d3960 legacy=0x7fb2400d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb224003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb24015dd50 0x7fb224004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb224004d40 0x7fb2240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400d3440 con 0x7fb2240039b0 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400ce1c0 con 0x7fb224004d40 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22400b9c0 con 0x7fb24015dd50 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.564+0000 7fb245a00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb224003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.565+0000 7fb245000640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb24015dd50 0x7fb224004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.565+0000 7fb246400640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb224004d40 0x7fb2240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.565+0000 7fb245a00640 1 -- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb24015dd50 msgr2=0x7fb224004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.565+0000 7fb245a00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb24015dd50 0x7fb224004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.565+0000 7fb245a00640 1 -- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb224004d40 msgr2=0x7fb2240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.565+0000 7fb245a00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb224004d40 0x7fb2240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.565+0000 7fb245a00640 1 -- 172.21.15.62:0/722457507 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb224003520 con 0x7fb2240039b0 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.565+0000 7fb246400640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb224004d40 0x7fb2240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:33.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.566+0000 7fb245a00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb224003dd0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb230009ac0 tx=0x7fb224005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e977f243df83c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.566+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230036070 con 0x7fb2240039b0 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.566+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb230031400 con 0x7fb2240039b0 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.566+0000 7fb23fe00640 1 -- 172.21.15.62:0/722457507 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230009690 con 0x7fb2240039b0 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.567+0000 7fb246e00640 1 -- 172.21.15.62:0/722457507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 msgr2=0x7fb224003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.567+0000 7fb246e00640 1 --2- 172.21.15.62:0/722457507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb224003dd0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb230009ac0 tx=0x7fb224005830 comp rx=0 tx=0).stop 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.567+0000 7fb246400640 1 -- 172.21.15.62:0/722457507 reap_dead start 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.567+0000 7fb246e00640 1 -- 172.21.15.62:0/722457507 shutdown_connections 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.567+0000 7fb246e00640 1 -- 172.21.15.62:0/722457507 >> 172.21.15.62:0/722457507 conn(0x7fb24006b000 msgr2=0x7fb24006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.567+0000 7fb246e00640 1 -- 172.21.15.62:0/722457507 shutdown_connections 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.567+0000 7fb246e00640 1 -- 172.21.15.62:0/722457507 wait complete. 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.569+0000 7fb246e00640 1 Processor -- start 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.569+0000 7fb246e00640 1 -- start start 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb246e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb240159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb246e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2400ccf90 0x7fb2400cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb246e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2400c99f0 0x7fb2400c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb246e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400d2480 con 0x7fb2240039b0 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb246e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400d2300 con 0x7fb2400c99f0 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb246e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2400d2600 con 0x7fb2400ccf90 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb245a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb240159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb245a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb240159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36908/0 (socket says 172.21.15.62:36908) 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb245a00640 1 -- 172.21.15.62:0/2227578202 learned_addr learned my addr 172.21.15.62:0/2227578202 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb245a00640 1 -- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2400ccf90 msgr2=0x7fb2400cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb245a00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2400ccf90 0x7fb2400cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb245a00640 1 -- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2400c99f0 msgr2=0x7fb2400c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb245a00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2400c99f0 0x7fb2400c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.570+0000 7fb245a00640 1 -- 172.21.15.62:0/2227578202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2400ca3b0 con 0x7fb2240039b0 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.571+0000 7fb245a00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb240159ed0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb230034040 tx=0x7fb230004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.571+0000 7fb23d600640 1 -- 172.21.15.62:0/2227578202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230040020 con 0x7fb2240039b0 2024-04-25T09:15:33.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.571+0000 7fb23d600640 1 -- 172.21.15.62:0/2227578202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb230003940 con 0x7fb2240039b0 2024-04-25T09:15:33.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.571+0000 7fb23d600640 1 -- 172.21.15.62:0/2227578202 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230036040 con 0x7fb2240039b0 2024-04-25T09:15:33.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.571+0000 7fb246e00640 1 -- 172.21.15.62:0/2227578202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2400c8720 con 0x7fb2240039b0 2024-04-25T09:15:33.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.571+0000 7fb246e00640 1 -- 172.21.15.62:0/2227578202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2400c8ce0 con 0x7fb2240039b0 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.573+0000 7fb23d600640 1 -- 172.21.15.62:0/2227578202 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb230043020 con 0x7fb2240039b0 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.574+0000 7fb23d600640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb22006b4d0 0x7fb22006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.574+0000 7fb23d600640 1 -- 172.21.15.62:0/2227578202 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb2300aa5d0 con 0x7fb2240039b0 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.575+0000 7fb24b3c7740 1 Processor -- start 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.575+0000 7fb245000640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb22006b4d0 0x7fb22006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.575+0000 7fb24b3c7740 1 -- start start 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed374a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed374fb0 0x55c4ed39ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ed2dd870 con 0x55c4ed3a5290 2024-04-25T09:15:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ed286d60 con 0x55c4ed3a4870 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4ed2d9910 con 0x55c4ed374fb0 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb245000640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb22006b4d0 0x7fb22006d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb2400b8b10 tx=0x7fb22c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb245000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed374a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb245000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed374a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36918/0 (socket says 172.21.15.62:36918) 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.576+0000 7fb245000640 1 -- 172.21.15.62:0/412743728 learned_addr learned my addr 172.21.15.62:0/412743728 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb245a00640 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb246400640 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed374fb0 0x55c4ed39ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb245000640 1 -- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed374fb0 msgr2=0x55c4ed39ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb245000640 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed374fb0 0x55c4ed39ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb245000640 1 -- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a4870 msgr2=0x55c4ed3a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb245000640 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb245000640 1 -- 172.21.15.62:0/412743728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4ed0bc7d0 con 0x55c4ed3a5290 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb245a00640 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb245000640 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed374a70 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fb22c011bb0 tx=0x7fb22c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db138ea455bd9fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb236000640 1 -- 172.21.15.62:0/412743728 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c015a80 con 0x55c4ed3a5290 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb236000640 1 -- 172.21.15.62:0/412743728 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c026070 con 0x55c4ed3a5290 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.577+0000 7fb236000640 1 -- 172.21.15.62:0/412743728 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c015e20 con 0x55c4ed3a5290 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 -- 172.21.15.62:0/412743728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 msgr2=0x55c4ed374a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed374a70 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fb22c011bb0 tx=0x7fb22c014000 comp rx=0 tx=0).stop 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 -- 172.21.15.62:0/412743728 shutdown_connections 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed374a70 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed374fb0 0x55c4ed39ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/412743728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 -- 172.21.15.62:0/412743728 >> 172.21.15.62:0/412743728 conn(0x55c4ed376ef0 msgr2=0x55c4ed3bf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 -- 172.21.15.62:0/412743728 shutdown_connections 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.578+0000 7fb24b3c7740 1 -- 172.21.15.62:0/412743728 wait complete. 2024-04-25T09:15:33.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.579+0000 7fb24b3c7740 1 -- start start 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.579+0000 7fb24b3c7740 1 Processor -- start 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.580+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.580+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.580+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.580+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ed3a0fc0 con 0x55c4ed3a4870 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.580+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ed29a1c0 con 0x55c4ed3a5290 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.580+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4ed3a1260 con 0x55c4ed37a250 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.580+0000 7fb24b3c7740 10 client.0 _ll_register_callbacks cb 0x55c4ed2d88f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36926/0 (socket says 172.21.15.62:36926) 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 -- 172.21.15.62:0/3820311688 learned_addr learned my addr 172.21.15.62:0/3820311688 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 -- 172.21.15.62:0/3820311688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 msgr2=0x55c4ed3a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245000640 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 -- 172.21.15.62:0/3820311688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 msgr2=0x55c4ed3992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 -- 172.21.15.62:0/3820311688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4ecfb20f0 con 0x55c4ed3a4870 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb245a00640 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3f40 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb23002abb0 tx=0x7fb230078be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11a5b17ae3e1e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.581+0000 7fb22aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb22aa00640 1 -- 172.21.15.62:0/3820311688 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230080ef0 con 0x55c4ed3a4870 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb22aa00640 1 -- 172.21.15.62:0/3820311688 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb230081090 con 0x55c4ed3a4870 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3820311688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb22c026070 con 0x55c4ed3a4870 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb22aa00640 1 -- 172.21.15.62:0/3820311688 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2300aa0c0 con 0x55c4ed3a4870 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 10 client.8592 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3820311688 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55c4ed11cf60 con 0x55c4ed3a4870 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb22aa00640 1 -- 172.21.15.62:0/3820311688 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c026070 con 0x55c4ed3a4870 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb22aa00640 1 -- 172.21.15.62:0/3820311688 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb230078030 con 0x55c4ed3a4870 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 10 client.8592 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 10 client.8592 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 20 client.8592 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c4ed397e00 0x55c4ed3a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 4 client.8592 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.582+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3820311688 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4ed2b3690 con 0x55c4ed397e00 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.583+0000 7fb245000640 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c4ed397e00 0x55c4ed3a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.583+0000 7fb245000640 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c4ed397e00 0x55c4ed3a23b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2553030dc48f73fc server_cookie=370d306ea99988db in_seq=0 out_seq=0 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.583+0000 7fb22aa00640 10 client.8592 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb22aa00640 1 -- 172.21.15.62:0/3820311688 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4ed2b3690 con 0x55c4ed397e00 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb22aa00640 10 client.8592 handle_command_reply: tid=0 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 client.8592 shutdown 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3820311688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 msgr2=0x55c4ed3a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3f40 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb23002abb0 tx=0x7fb230078be0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3820311688 shutdown_connections 2024-04-25T09:15:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c4ed397e00 0x55c4ed3a23b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3820311688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3f40 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3820311688 >> 172.21.15.62:0/3820311688 conn(0x55c4ed374fb0 msgr2=0x55c4ed378430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb22aa00640 0 client.8592 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3820311688 shutdown_connections 2024-04-25T09:15:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.584+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3820311688 wait complete. 2024-04-25T09:15:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.585+0000 7fb24b3c7740 20 client.8592 trim_cache size 0 max 16384 2024-04-25T09:15:33.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.601+0000 7fb24b3c7740 1 Processor -- start 2024-04-25T09:15:33.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.601+0000 7fb24b3c7740 1 -- start start 2024-04-25T09:15:33.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3a3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb240200700 con 0x55c4ed3a4870 2024-04-25T09:15:33.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401ff9c0 con 0x55c4ed3a5290 2024-04-25T09:15:33.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401f4c40 con 0x55c4ed37a250 2024-04-25T09:15:33.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb245000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb245000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36938/0 (socket says 172.21.15.62:36938) 2024-04-25T09:15:33.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb245000640 1 -- 172.21.15.62:0/4024964109 learned_addr learned my addr 172.21.15.62:0/4024964109 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb246400640 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb245a00640 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb245000640 1 -- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 msgr2=0x55c4ed3a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb245000640 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3a3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb245000640 1 -- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 msgr2=0x55c4ed3a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.602+0000 7fb245000640 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.603+0000 7fb245000640 1 -- 172.21.15.62:0/4024964109 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4ecf8b1b0 con 0x55c4ed3a4870 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.603+0000 7fb246400640 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.603+0000 7fb245000640 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3e60 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fb22c011630 tx=0x7fb22c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f7e4acbbadb1d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.603+0000 7fb235600640 1 -- 172.21.15.62:0/4024964109 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c015d10 con 0x55c4ed3a4870 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.603+0000 7fb235600640 1 -- 172.21.15.62:0/4024964109 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c026040 con 0x55c4ed3a4870 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.603+0000 7fb235600640 1 -- 172.21.15.62:0/4024964109 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c035b80 con 0x55c4ed3a4870 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 -- 172.21.15.62:0/4024964109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 msgr2=0x55c4ed3a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3e60 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fb22c011630 tx=0x7fb22c016000 comp rx=0 tx=0).stop 2024-04-25T09:15:33.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 -- 172.21.15.62:0/4024964109 shutdown_connections 2024-04-25T09:15:33.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a4870 0x55c4ed3a3e60 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed3a5290 0x55c4ed3a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/4024964109 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 -- 172.21.15.62:0/4024964109 >> 172.21.15.62:0/4024964109 conn(0x55c4ed374fb0 msgr2=0x55c4ed399070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 -- 172.21.15.62:0/4024964109 shutdown_connections 2024-04-25T09:15:33.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 -- 172.21.15.62:0/4024964109 wait complete. 2024-04-25T09:15:33.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.604+0000 7fb24b3c7740 1 -- start start 2024-04-25T09:15:33.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.605+0000 7fb24b3c7740 1 Processor -- start 2024-04-25T09:15:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed3d8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb24b3c7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed370080 0x55c4ed428910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4ed3789a0 con 0x55c4ed3a5290 2024-04-25T09:15:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401f5240 con 0x55c4ed370080 2024-04-25T09:15:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb24b3c7740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4ed3758a0 con 0x55c4ed37a250 2024-04-25T09:15:33.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb24b3c7740 10 client.0 _ll_register_callbacks cb 0x7fb22c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed3d8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed3d8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36940/0 (socket says 172.21.15.62:36940) 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245000640 1 -- 172.21.15.62:0/3969272671 learned_addr learned my addr 172.21.15.62:0/3969272671 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245a00640 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245000640 1 -- 172.21.15.62:0/3969272671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 msgr2=0x55c4ed3d8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245000640 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3d8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245000640 1 -- 172.21.15.62:0/3969272671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed370080 msgr2=0x55c4ed428910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245000640 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed370080 0x55c4ed428910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.606+0000 7fb245000640 1 -- 172.21.15.62:0/3969272671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4ed39c830 con 0x55c4ed3a5290 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.607+0000 7fb245000640 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed3d8bb0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fb22c008010 tx=0x7fb22c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b07165e50860c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.607+0000 7fb22aa00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.607+0000 7fb22aa00640 1 -- 172.21.15.62:0/3969272671 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c039030 con 0x55c4ed3a5290 2024-04-25T09:15:33.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.607+0000 7fb22aa00640 1 -- 172.21.15.62:0/3969272671 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb22c033040 con 0x55c4ed3a5290 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.607+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3969272671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb22c026040 con 0x55c4ed3a5290 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.607+0000 7fb22aa00640 1 -- 172.21.15.62:0/3969272671 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb22c015d00 con 0x55c4ed3a5290 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.607+0000 7fb24b3c7740 10 client.8598 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.607+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3969272671 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55c4ed428eb0 con 0x55c4ed3a5290 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb22aa00640 1 -- 172.21.15.62:0/3969272671 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c026040 con 0x55c4ed3a5290 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb22aa00640 1 -- 172.21.15.62:0/3969272671 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb22c048020 con 0x55c4ed3a5290 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb24b3c7740 10 client.8598 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb24b3c7740 10 client.8598 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb24b3c7740 20 client.8598 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c4ed3a20f0 0x55c4ed3a2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb24b3c7740 4 client.8598 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3969272671 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55c4ed2eadb0 con 0x55c4ed3a20f0 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.608+0000 7fb245a00640 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c4ed3a20f0 0x55c4ed3a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb245a00640 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c4ed3a20f0 0x55c4ed3a2500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb081b745667eded server_cookie=50623273fc354a95 in_seq=0 out_seq=0 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb22aa00640 10 client.8598 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb22aa00640 1 -- 172.21.15.62:0/3969272671 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55c4ed2eadb0 con 0x55c4ed3a20f0 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb22aa00640 10 client.8598 handle_command_reply: tid=0 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 client.8598 shutdown 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3969272671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 msgr2=0x55c4ed3d8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed3d8bb0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fb22c008010 tx=0x7fb22c009190 comp rx=0 tx=0).stop 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3969272671 shutdown_connections 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c4ed3a20f0 0x55c4ed3a2500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c4ed3a5290 0x55c4ed3d8bb0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4ed370080 0x55c4ed428910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb22aa00640 0 client.8598 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 --2- 172.21.15.62:0/3969272671 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c4ed37a250 0x55c4ed3d8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3969272671 >> 172.21.15.62:0/3969272671 conn(0x55c4ed374fb0 msgr2=0x55c4ed3c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3969272671 shutdown_connections 2024-04-25T09:15:33.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 1 -- 172.21.15.62:0/3969272671 wait complete. 2024-04-25T09:15:33.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.609+0000 7fb24b3c7740 20 client.8598 trim_cache size 0 max 16384 2024-04-25T09:15:33.601 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:15:33.601 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:15:33.601 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:15:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 -- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb22006b4d0 msgr2=0x7fb22006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb22006b4d0 0x7fb22006d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb2400b8b10 tx=0x7fb22c008040 comp rx=0 tx=0).stop 2024-04-25T09:15:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 -- 172.21.15.62:0/2227578202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 msgr2=0x7fb240159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb240159ed0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb230034040 tx=0x7fb230004730 comp rx=0 tx=0).stop 2024-04-25T09:15:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 -- 172.21.15.62:0/2227578202 shutdown_connections 2024-04-25T09:15:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb22006b4d0 0x7fb22006d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2400c99f0 0x7fb2400c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb2400ccf90 0x7fb2400cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 --2- 172.21.15.62:0/2227578202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb2240039b0 0x7fb240159ed0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 -- 172.21.15.62:0/2227578202 >> 172.21.15.62:0/2227578202 conn(0x7fb24006b000 msgr2=0x7fb24015c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 -- 172.21.15.62:0/2227578202 shutdown_connections 2024-04-25T09:15:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.610+0000 7fb246e00640 1 -- 172.21.15.62:0/2227578202 wait complete. 2024-04-25T09:15:33.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:33.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:15:33.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.837+0000 7feb46400640 1 Processor -- start 2024-04-25T09:15:33.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.837+0000 7feb46400640 1 -- start start 2024-04-25T09:15:33.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.838+0000 7feb46400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feb400b54d0 0x7feb400ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.838+0000 7feb46400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feb400ce7f0 0x7feb40164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.838+0000 7feb46400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb400b4430 con 0x7feb400b2d80 2024-04-25T09:15:33.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.838+0000 7feb46400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb400b4710 con 0x7feb400b54d0 2024-04-25T09:15:33.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.838+0000 7feb46400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb400b49f0 con 0x7feb400ce7f0 2024-04-25T09:15:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.839+0000 7feb3fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feb400b54d0 0x7feb400ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.839+0000 7feb3fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feb400b54d0 0x7feb400ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36952/0 (socket says 172.21.15.62:36952) 2024-04-25T09:15:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.839+0000 7feb3fe00640 1 -- 172.21.15.62:0/559143951 learned_addr learned my addr 172.21.15.62:0/559143951 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.839+0000 7feb45a00640 1 --2- 172.21.15.62:0/559143951 >> v2:172.21.15.121:3300/0 conn(0x7feb400ce7f0 0x7feb40164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.840+0000 7feb3fe00640 1 -- 172.21.15.62:0/559143951 >> v2:172.21.15.121:3300/0 conn(0x7feb400ce7f0 msgr2=0x7feb40164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.840+0000 7feb3fe00640 1 --2- 172.21.15.62:0/559143951 >> v2:172.21.15.121:3300/0 conn(0x7feb400ce7f0 0x7feb40164310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.840+0000 7feb3fe00640 1 -- 172.21.15.62:0/559143951 >> v1:172.21.15.121:6789/0 conn(0x7feb400b2d80 legacy=0x7feb400b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.840+0000 7feb3fe00640 1 -- 172.21.15.62:0/559143951 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400b4cd0 con 0x7feb400b54d0 2024-04-25T09:15:33.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.841+0000 7feb3fe00640 1 --2- 172.21.15.62:0/559143951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400b54d0 0x7feb400ce2b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7feb400aea30 tx=0x7feb2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507fb9e9a10f932a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.841+0000 7feb3f400640 1 -- 172.21.15.62:0/559143951 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c037070 con 0x7feb400b54d0 2024-04-25T09:15:33.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.841+0000 7feb3f400640 1 -- 172.21.15.62:0/559143951 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c032440 con 0x7feb400b54d0 2024-04-25T09:15:33.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.841+0000 7feb3f400640 1 -- 172.21.15.62:0/559143951 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c032810 con 0x7feb400b54d0 2024-04-25T09:15:33.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.842+0000 7feb46400640 1 -- 172.21.15.62:0/559143951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400b54d0 msgr2=0x7feb400ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.842+0000 7feb46400640 1 --2- 172.21.15.62:0/559143951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400b54d0 0x7feb400ce2b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7feb400aea30 tx=0x7feb2c02a270 comp rx=0 tx=0).stop 2024-04-25T09:15:33.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.842+0000 7feb46400640 1 -- 172.21.15.62:0/559143951 shutdown_connections 2024-04-25T09:15:33.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.842+0000 7feb46400640 1 --2- 172.21.15.62:0/559143951 >> v2:172.21.15.121:3300/0 conn(0x7feb400ce7f0 0x7feb40164310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.842+0000 7feb46400640 1 --2- 172.21.15.62:0/559143951 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400b54d0 0x7feb400ce2b0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.842+0000 7feb46400640 1 -- 172.21.15.62:0/559143951 >> 172.21.15.62:0/559143951 conn(0x7feb40066f20 msgr2=0x7feb400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.842+0000 7feb46400640 1 -- 172.21.15.62:0/559143951 shutdown_connections 2024-04-25T09:15:33.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.842+0000 7feb46400640 1 -- 172.21.15.62:0/559143951 wait complete. 2024-04-25T09:15:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.844+0000 7feb46400640 1 Processor -- start 2024-04-25T09:15:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.844+0000 7feb46400640 1 -- start start 2024-04-25T09:15:33.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb46400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb400b2d80 0x7feb400ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb46400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb400b54d0 0x7feb40159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb46400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400ce7f0 0x7feb40151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb46400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400b3870 con 0x7feb400ce7f0 2024-04-25T09:15:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb46400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400b40e0 con 0x7feb400b2d80 2024-04-25T09:15:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb46400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb40164850 con 0x7feb400b54d0 2024-04-25T09:15:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb45a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400ce7f0 0x7feb40151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb45a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400ce7f0 0x7feb40151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36954/0 (socket says 172.21.15.62:36954) 2024-04-25T09:15:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.845+0000 7feb45a00640 1 -- 172.21.15.62:0/3503091278 learned_addr learned my addr 172.21.15.62:0/3503091278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb3fe00640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb400b54d0 0x7feb40159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb45000640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb400b2d80 0x7feb400ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb45a00640 1 -- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb400b54d0 msgr2=0x7feb40159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb45a00640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb400b54d0 0x7feb40159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb45a00640 1 -- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb400b2d80 msgr2=0x7feb400ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb45a00640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb400b2d80 0x7feb400ab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb45a00640 1 -- 172.21.15.62:0/3503091278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400abc90 con 0x7feb400ce7f0 2024-04-25T09:15:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb45a00640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400ce7f0 0x7feb40151f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7feb3800efc0 tx=0x7feb3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb3cc00640 1 -- 172.21.15.62:0/3503091278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38018020 con 0x7feb400ce7f0 2024-04-25T09:15:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb3cc00640 1 -- 172.21.15.62:0/3503091278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb38014480 con 0x7feb400ce7f0 2024-04-25T09:15:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb3cc00640 1 -- 172.21.15.62:0/3503091278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38014870 con 0x7feb400ce7f0 2024-04-25T09:15:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.846+0000 7feb46400640 1 -- 172.21.15.62:0/3503091278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb400ad680 con 0x7feb400ce7f0 2024-04-25T09:15:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.847+0000 7feb46400640 1 -- 172.21.15.62:0/3503091278 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb400abf70 con 0x7feb400ce7f0 2024-04-25T09:15:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.850+0000 7feb3cc00640 1 -- 172.21.15.62:0/3503091278 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb38005d10 con 0x7feb400ce7f0 2024-04-25T09:15:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.851+0000 7feb3cc00640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feb2006b500 0x7feb2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.851+0000 7feb3cc00640 1 -- 172.21.15.62:0/3503091278 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feb3808ac70 con 0x7feb400ce7f0 2024-04-25T09:15:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.851+0000 7feb4a812740 1 Processor -- start 2024-04-25T09:15:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.851+0000 7feb4a812740 1 -- start start 2024-04-25T09:15:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.851+0000 7feb45000640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feb2006b500 0x7feb2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb4a812740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb51b60 0x55c66eb51f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb4a812740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb2bf10 0x55c66eb70980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb4a812740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb70ec0 0x55c66eb28dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb4a812740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66ea8dc70 con 0x55c66eb70ec0 2024-04-25T09:15:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb4a812740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66ea36cf0 con 0x55c66eb2bf10 2024-04-25T09:15:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb4a812740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c66ea89d10 con 0x55c66eb51b60 2024-04-25T09:15:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb45a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb70ec0 0x55c66eb28dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb45a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb70ec0 0x55c66eb28dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36964/0 (socket says 172.21.15.62:36964) 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.852+0000 7feb45a00640 1 -- 172.21.15.62:0/1512677469 learned_addr learned my addr 172.21.15.62:0/1512677469 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb45000640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feb2006b500 0x7feb2006d9c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7feb340015a0 tx=0x7feb34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb3fe00640 1 --2- 172.21.15.62:0/1512677469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb2bf10 0x55c66eb70980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb45a00640 1 -- 172.21.15.62:0/1512677469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb51b60 msgr2=0x55c66eb51f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb45a00640 1 --2- 172.21.15.62:0/1512677469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb51b60 0x55c66eb51f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb45a00640 1 -- 172.21.15.62:0/1512677469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb2bf10 msgr2=0x55c66eb70980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb45a00640 1 --2- 172.21.15.62:0/1512677469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb2bf10 0x55c66eb70980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb45a00640 1 -- 172.21.15.62:0/1512677469 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c66e86c7d0 con 0x55c66eb70ec0 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb45a00640 1 --2- 172.21.15.62:0/1512677469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb70ec0 0x55c66eb28dd0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7feb38061580 tx=0x7feb380593a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922faf26a86bb61f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb31600640 1 -- 172.21.15.62:0/1512677469 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38061ab0 con 0x55c66eb70ec0 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb31600640 1 -- 172.21.15.62:0/1512677469 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb380576a0 con 0x55c66eb70ec0 2024-04-25T09:15:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.853+0000 7feb31600640 1 -- 172.21.15.62:0/1512677469 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38055a10 con 0x55c66eb70ec0 2024-04-25T09:15:33.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 -- 172.21.15.62:0/1512677469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb70ec0 msgr2=0x55c66eb28dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 --2- 172.21.15.62:0/1512677469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb70ec0 0x55c66eb28dd0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7feb38061580 tx=0x7feb380593a0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 -- 172.21.15.62:0/1512677469 shutdown_connections 2024-04-25T09:15:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 --2- 172.21.15.62:0/1512677469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb70ec0 0x55c66eb28dd0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 --2- 172.21.15.62:0/1512677469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb2bf10 0x55c66eb70980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 --2- 172.21.15.62:0/1512677469 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb51b60 0x55c66eb51f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 -- 172.21.15.62:0/1512677469 >> 172.21.15.62:0/1512677469 conn(0x55c66eaf1a30 msgr2=0x55c66eb6f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 -- 172.21.15.62:0/1512677469 shutdown_connections 2024-04-25T09:15:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.854+0000 7feb4a812740 1 -- 172.21.15.62:0/1512677469 wait complete. 2024-04-25T09:15:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.855+0000 7feb4a812740 1 -- start start 2024-04-25T09:15:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.855+0000 7feb4a812740 1 Processor -- start 2024-04-25T09:15:33.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb4a812740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb2bf10 0x55c66eb58620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb4a812740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 0x55c66eb568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb4a812740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb4c450 0x55c66eb56de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb4a812740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66eb26370 con 0x55c66eb2bf10 2024-04-25T09:15:33.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb4a812740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66ea2c3a0 con 0x55c66eb51b60 2024-04-25T09:15:33.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb4a812740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c66eb2aea0 con 0x55c66eb4c450 2024-04-25T09:15:33.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb4a812740 10 client.0 _ll_register_callbacks cb 0x55c66ea49b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:33.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb45000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb2bf10 0x55c66eb58620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb45000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb2bf10 0x55c66eb58620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36972/0 (socket says 172.21.15.62:36972) 2024-04-25T09:15:33.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.856+0000 7feb45000640 1 -- 172.21.15.62:0/3832955880 learned_addr learned my addr 172.21.15.62:0/3832955880 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb45000640 1 -- 172.21.15.62:0/3832955880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb4c450 msgr2=0x55c66eb56de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb45a00640 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb4c450 0x55c66eb56de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb45000640 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb4c450 0x55c66eb56de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb45000640 1 -- 172.21.15.62:0/3832955880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 msgr2=0x55c66eb568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb3fe00640 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 0x55c66eb568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb45000640 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 0x55c66eb568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb45000640 1 -- 172.21.15.62:0/3832955880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c66e7a23f0 con 0x55c66eb2bf10 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb45000640 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb2bf10 0x55c66eb58620 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7feb3400dc10 tx=0x7feb340147f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8963c79445fac540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb26000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb26000640 1 -- 172.21.15.62:0/3832955880 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34024060 con 0x55c66eb2bf10 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb26000640 1 -- 172.21.15.62:0/3832955880 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feb3401f600 con 0x55c66eb2bf10 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.857+0000 7feb4a812740 1 -- 172.21.15.62:0/3832955880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb380576a0 con 0x55c66eb2bf10 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb26000640 1 -- 172.21.15.62:0/3832955880 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb3402a050 con 0x55c66eb2bf10 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb4a812740 10 client.8610 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb4a812740 1 -- 172.21.15.62:0/3832955880 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55c66e7620f0 con 0x55c66eb2bf10 2024-04-25T09:15:33.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb26000640 1 -- 172.21.15.62:0/3832955880 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb380576a0 con 0x55c66eb2bf10 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb26000640 1 -- 172.21.15.62:0/3832955880 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb3402d020 con 0x55c66eb2bf10 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb4a812740 10 client.8610 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb4a812740 10 client.8610 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb4a812740 20 client.8610 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb4a812740 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c66eb8e310 0x55c66eb90780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb4a812740 4 client.8610 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb4a812740 1 -- 172.21.15.62:0/3832955880 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c66ea9b1b0 con 0x55c66eb8e310 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.858+0000 7feb3fe00640 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c66eb8e310 0x55c66eb90780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.859+0000 7feb3fe00640 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c66eb8e310 0x55c66eb90780 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=815aa15c44c708d2 server_cookie=2c3b0cce009fce37 in_seq=0 out_seq=0 2024-04-25T09:15:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.859+0000 7feb26000640 10 client.8610 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb26000640 1 -- 172.21.15.62:0/3832955880 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c66ea9b1b0 con 0x55c66eb8e310 2024-04-25T09:15:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb26000640 10 client.8610 handle_command_reply: tid=0 2024-04-25T09:15:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 client.8610 shutdown 2024-04-25T09:15:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 -- 172.21.15.62:0/3832955880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb2bf10 msgr2=0x55c66eb58620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb2bf10 0x55c66eb58620 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7feb3400dc10 tx=0x7feb340147f0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 -- 172.21.15.62:0/3832955880 shutdown_connections 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c66eb8e310 0x55c66eb90780 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb4c450 0x55c66eb56de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 0x55c66eb568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 --2- 172.21.15.62:0/3832955880 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb2bf10 0x55c66eb58620 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 -- 172.21.15.62:0/3832955880 >> 172.21.15.62:0/3832955880 conn(0x55c66eaf1a30 msgr2=0x55c66eb72d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb26000640 0 client.8610 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 -- 172.21.15.62:0/3832955880 shutdown_connections 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.860+0000 7feb4a812740 1 -- 172.21.15.62:0/3832955880 wait complete. 2024-04-25T09:15:33.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.861+0000 7feb4a812740 20 client.8610 trim_cache size 0 max 16384 2024-04-25T09:15:33.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.878+0000 7feb4a812740 1 Processor -- start 2024-04-25T09:15:33.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.878+0000 7feb4a812740 1 -- start start 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.878+0000 7feb4a812740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 0x55c66eb4dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.878+0000 7feb4a812740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb4c450 0x55c66eb82200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.878+0000 7feb4a812740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb51b60 0x55c66eb82740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.878+0000 7feb4a812740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4015ca20 con 0x55c66eb51b60 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.878+0000 7feb4a812740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4015bc60 con 0x55c66eb4c450 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.878+0000 7feb4a812740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb400aa240 con 0x55c66eb2bf10 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.879+0000 7feb45a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb51b60 0x55c66eb82740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.879+0000 7feb45a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb51b60 0x55c66eb82740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36976/0 (socket says 172.21.15.62:36976) 2024-04-25T09:15:33.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.879+0000 7feb45a00640 1 -- 172.21.15.62:0/3434864130 learned_addr learned my addr 172.21.15.62:0/3434864130 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.879+0000 7feb45a00640 1 -- 172.21.15.62:0/3434864130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 msgr2=0x55c66eb4dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.879+0000 7feb45a00640 1 --2- 172.21.15.62:0/3434864130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 0x55c66eb4dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.879+0000 7feb45a00640 1 -- 172.21.15.62:0/3434864130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb4c450 msgr2=0x55c66eb82200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.879+0000 7feb45a00640 1 --2- 172.21.15.62:0/3434864130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb4c450 0x55c66eb82200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.879+0000 7feb45a00640 1 -- 172.21.15.62:0/3434864130 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c66eb286e0 con 0x55c66eb51b60 2024-04-25T09:15:33.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.880+0000 7feb45a00640 1 --2- 172.21.15.62:0/3434864130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb51b60 0x55c66eb82740 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7feb38055800 tx=0x7feb38015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6368df38668bdc66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.880+0000 7feb30c00640 1 -- 172.21.15.62:0/3434864130 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38057440 con 0x55c66eb51b60 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.880+0000 7feb30c00640 1 -- 172.21.15.62:0/3434864130 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb380576a0 con 0x55c66eb51b60 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.880+0000 7feb30c00640 1 -- 172.21.15.62:0/3434864130 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38055a10 con 0x55c66eb51b60 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.880+0000 7feb4a812740 1 -- 172.21.15.62:0/3434864130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb51b60 msgr2=0x55c66eb82740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.880+0000 7feb4a812740 1 --2- 172.21.15.62:0/3434864130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb51b60 0x55c66eb82740 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7feb38055800 tx=0x7feb38015000 comp rx=0 tx=0).stop 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.881+0000 7feb4a812740 1 -- 172.21.15.62:0/3434864130 shutdown_connections 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.881+0000 7feb4a812740 1 --2- 172.21.15.62:0/3434864130 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb51b60 0x55c66eb82740 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.881+0000 7feb4a812740 1 --2- 172.21.15.62:0/3434864130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb4c450 0x55c66eb82200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.881+0000 7feb4a812740 1 --2- 172.21.15.62:0/3434864130 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 0x55c66eb4dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.881+0000 7feb4a812740 1 -- 172.21.15.62:0/3434864130 >> 172.21.15.62:0/3434864130 conn(0x55c66eaf1a30 msgr2=0x55c66eb68d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.881+0000 7feb4a812740 1 -- 172.21.15.62:0/3434864130 shutdown_connections 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.881+0000 7feb4a812740 1 -- 172.21.15.62:0/3434864130 wait complete. 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.881+0000 7feb4a812740 1 -- start start 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.882+0000 7feb4a812740 1 Processor -- start 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.882+0000 7feb4a812740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 0x55c66ebde1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.882+0000 7feb4a812740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb4c450 0x55c66eb82b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb4a812740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 0x55c66eb83050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb4a812740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40154210 con 0x55c66eb4c450 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb4a812740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66eb6fbd0 con 0x55c66eb51b60 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb4a812740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c66ebdebf0 con 0x55c66eb2bf10 2024-04-25T09:15:33.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb4a812740 10 client.0 _ll_register_callbacks cb 0x7feb340012c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb4c450 0x55c66eb82b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb3fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb4c450 0x55c66eb82b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36978/0 (socket says 172.21.15.62:36978) 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb3fe00640 1 -- 172.21.15.62:0/190609095 learned_addr learned my addr 172.21.15.62:0/190609095 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb45a00640 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 0x55c66eb83050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb45000640 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 0x55c66ebde1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb3fe00640 1 -- 172.21.15.62:0/190609095 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 msgr2=0x55c66ebde1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb3fe00640 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 0x55c66ebde1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb3fe00640 1 -- 172.21.15.62:0/190609095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 msgr2=0x55c66eb83050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb3fe00640 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 0x55c66eb83050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.883+0000 7feb3fe00640 1 -- 172.21.15.62:0/190609095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c66eb70050 con 0x55c66eb4c450 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.884+0000 7feb3fe00640 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb4c450 0x55c66eb82b10 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009960 tx=0x7feb2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed652e8f66289d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.884+0000 7feb26000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.884+0000 7feb26000640 1 -- 172.21.15.62:0/190609095 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c047060 con 0x55c66eb4c450 2024-04-25T09:15:33.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.884+0000 7feb26000640 1 -- 172.21.15.62:0/190609095 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feb2c042410 con 0x55c66eb4c450 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.884+0000 7feb4a812740 1 -- 172.21.15.62:0/190609095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb380576a0 con 0x55c66eb4c450 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.884+0000 7feb26000640 1 -- 172.21.15.62:0/190609095 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb2c04d050 con 0x55c66eb4c450 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.884+0000 7feb4a812740 10 client.8616 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.884+0000 7feb4a812740 1 -- 172.21.15.62:0/190609095 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55c66eb82890 con 0x55c66eb4c450 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb26000640 1 -- 172.21.15.62:0/190609095 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb380576a0 con 0x55c66eb4c450 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb26000640 1 -- 172.21.15.62:0/190609095 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb2c050020 con 0x55c66eb4c450 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb4a812740 10 client.8616 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb4a812740 10 client.8616 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb4a812740 20 client.8616 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb4a812740 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c66eb70960 0x55c66ebe1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb4a812740 4 client.8616 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb4a812740 1 -- 172.21.15.62:0/190609095 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c66ea95500 con 0x55c66eb70960 2024-04-25T09:15:33.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.885+0000 7feb45000640 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c66eb70960 0x55c66ebe1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb45000640 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c66eb70960 0x55c66ebe1e00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ca75493c8bf1bb server_cookie=d40db1324da57a1a in_seq=0 out_seq=0 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb26000640 10 client.8616 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb26000640 1 -- 172.21.15.62:0/190609095 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55c66ea95500 con 0x55c66eb70960 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb26000640 10 client.8616 handle_command_reply: tid=0 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 client.8616 shutdown 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 -- 172.21.15.62:0/190609095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb4c450 msgr2=0x55c66eb82b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb4c450 0x55c66eb82b10 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009960 tx=0x7feb2c004730 comp rx=0 tx=0).stop 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 -- 172.21.15.62:0/190609095 shutdown_connections 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x55c66eb70960 0x55c66ebe1e00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c66eb4c450 0x55c66eb82b10 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c66eb51b60 0x55c66eb83050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 --2- 172.21.15.62:0/190609095 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55c66eb2bf10 0x55c66ebde1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb26000640 0 client.8616 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 -- 172.21.15.62:0/190609095 >> 172.21.15.62:0/190609095 conn(0x55c66eaf1a30 msgr2=0x55c66eb4dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 -- 172.21.15.62:0/190609095 shutdown_connections 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 1 -- 172.21.15.62:0/190609095 wait complete. 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.886+0000 7feb4a812740 20 client.8616 trim_cache size 0 max 16384 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:15:33.879 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:15:33.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 -- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feb2006b500 msgr2=0x7feb2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feb2006b500 0x7feb2006d9c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7feb340015a0 tx=0x7feb34009040 comp rx=0 tx=0).stop 2024-04-25T09:15:33.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 -- 172.21.15.62:0/3503091278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400ce7f0 msgr2=0x7feb40151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400ce7f0 0x7feb40151f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7feb3800efc0 tx=0x7feb3800c490 comp rx=0 tx=0).stop 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 -- 172.21.15.62:0/3503091278 shutdown_connections 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feb2006b500 0x7feb2006d9c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb400ce7f0 0x7feb40151f40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feb400b54d0 0x7feb40159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 --2- 172.21.15.62:0/3503091278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb400b2d80 0x7feb400ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 -- 172.21.15.62:0/3503091278 >> 172.21.15.62:0/3503091278 conn(0x7feb40066f20 msgr2=0x7feb400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 -- 172.21.15.62:0/3503091278 shutdown_connections 2024-04-25T09:15:33.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:33.887+0000 7feb46400640 1 -- 172.21.15.62:0/3503091278 wait complete. 2024-04-25T09:15:33.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:34.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.109+0000 7f61b6e00640 1 Processor -- start 2024-04-25T09:15:34.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.109+0000 7f61b6e00640 1 -- start start 2024-04-25T09:15:34.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.110+0000 7f61b6e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f61b013d970 0x7f61b00b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.110+0000 7f61b6e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 0x7f61b006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61b6e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b015b220 con 0x7f61b015acf0 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61b6e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b015b500 con 0x7f61b013d970 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61b6e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b015b7e0 con 0x7f61b00b1050 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61b6400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f61b015acf0 0x7f61b013ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:32860/0 (socket says 172.21.15.62:32860) 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61b6400640 1 -- 172.21.15.62:0/1910303468 learned_addr learned my addr 172.21.15.62:0/1910303468 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61b5a00640 1 --2- 172.21.15.62:0/1910303468 >> v2:172.21.15.121:3300/0 conn(0x7f61b013d970 0x7f61b00b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61b5000640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 0x7f61b006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419556210 0 0) 0x7f61b015b220 con 0x7f61b015acf0 2024-04-25T09:15:34.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.111+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6198003520 con 0x7f61b015acf0 2024-04-25T09:15:34.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1161533116 0 0) 0x7f6198003520 con 0x7f61b015acf0 2024-04-25T09:15:34.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 msgr2=0x7f61b006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:34.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 0x7f61b006d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 >> v2:172.21.15.121:3300/0 conn(0x7f61b013d970 msgr2=0x7f61b00b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:34.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 --2- 172.21.15.62:0/1910303468 >> v2:172.21.15.121:3300/0 conn(0x7f61b013d970 0x7f61b00b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b015bac0 con 0x7f61b015acf0 2024-04-25T09:15:34.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f61a80019f0 con 0x7f61b015acf0 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 >> v1:172.21.15.62:6789/0 conn(0x7f61b015acf0 legacy=0x7f61b013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 0x7f61980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6198003f80 0x7f6198004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b015b500 con 0x7f6198004d40 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b015b7e0 con 0x7f6198003f80 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619800b9c0 con 0x7f61b00b1050 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.112+0000 7f61b6400640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.113+0000 7f61b5a00640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 0x7f61980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.113+0000 7f61b6400640 1 -- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 msgr2=0x7f61980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.113+0000 7f61b5000640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6198003f80 0x7f6198004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.113+0000 7f61b6400640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 0x7f61980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.113+0000 7f61b6400640 1 -- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6198003f80 msgr2=0x7f6198004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:34.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.113+0000 7f61b6400640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6198003f80 0x7f6198004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.113+0000 7f61b6400640 1 -- 172.21.15.62:0/1910303468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198003520 con 0x7f6198004d40 2024-04-25T09:15:34.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.113+0000 7f61b5a00640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00b1050 0x7f61980039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:34.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.114+0000 7f61b6400640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61980051c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f61a800a7b0 tx=0x7f6198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d627c046411253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:34.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.114+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a800a3d0 con 0x7f6198004d40 2024-04-25T09:15:34.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.114+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a802f070 con 0x7f6198004d40 2024-04-25T09:15:34.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.114+0000 7f61afe00640 1 -- 172.21.15.62:0/1910303468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a802bc20 con 0x7f6198004d40 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.115+0000 7f61b6e00640 1 -- 172.21.15.62:0/1910303468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 msgr2=0x7f61980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.115+0000 7f61b6e00640 1 --2- 172.21.15.62:0/1910303468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61980051c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f61a800a7b0 tx=0x7f6198005830 comp rx=0 tx=0).stop 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.115+0000 7f61b6400640 1 -- 172.21.15.62:0/1910303468 reap_dead start 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.115+0000 7f61b6e00640 1 -- 172.21.15.62:0/1910303468 shutdown_connections 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.115+0000 7f61b6e00640 1 -- 172.21.15.62:0/1910303468 >> 172.21.15.62:0/1910303468 conn(0x7f61b00b14d0 msgr2=0x7f61b00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.115+0000 7f61b6e00640 1 -- 172.21.15.62:0/1910303468 shutdown_connections 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.115+0000 7f61b6e00640 1 -- 172.21.15.62:0/1910303468 wait complete. 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.116+0000 7f61b6e00640 1 Processor -- start 2024-04-25T09:15:34.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.117+0000 7f61b6e00640 1 -- start start 2024-04-25T09:15:34.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.117+0000 7f61b6e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61b0133870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.117+0000 7f61b6e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00afce0 0x7f61b00ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.117+0000 7f61b6e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b0157de0 0x7f61b01435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.117+0000 7f61b6e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0140610 con 0x7f6198004d40 2024-04-25T09:15:34.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.117+0000 7f61b6e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b013f470 con 0x7f61b0157de0 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.117+0000 7f61b6e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0140790 con 0x7f61b00afce0 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61b0133870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61b0133870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36998/0 (socket says 172.21.15.62:36998) 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5a00640 1 -- 172.21.15.62:0/3972747767 learned_addr learned my addr 172.21.15.62:0/3972747767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b6400640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b0157de0 0x7f61b01435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5000640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00afce0 0x7f61b00ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5a00640 1 -- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00afce0 msgr2=0x7f61b00ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5a00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00afce0 0x7f61b00ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5a00640 1 -- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b0157de0 msgr2=0x7f61b01435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5a00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b0157de0 0x7f61b01435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b5a00640 1 -- 172.21.15.62:0/3972747767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b0133db0 con 0x7f6198004d40 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.118+0000 7f61b6400640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b0157de0 0x7f61b01435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.119+0000 7f61b5a00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61b0133870 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f61a40109d0 tx=0x7f61a4010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:34.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.119+0000 7f61ad600640 1 -- 172.21.15.62:0/3972747767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400f040 con 0x7f6198004d40 2024-04-25T09:15:34.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.119+0000 7f61ad600640 1 -- 172.21.15.62:0/3972747767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a400ec20 con 0x7f6198004d40 2024-04-25T09:15:34.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.119+0000 7f61ad600640 1 -- 172.21.15.62:0/3972747767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a401c400 con 0x7f6198004d40 2024-04-25T09:15:34.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.119+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b013c6d0 con 0x7f6198004d40 2024-04-25T09:15:34.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.119+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b0142ac0 con 0x7f6198004d40 2024-04-25T09:15:34.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.120+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b004ef20 con 0x7f6198004d40 2024-04-25T09:15:34.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.121+0000 7f61ad600640 1 -- 172.21.15.62:0/3972747767 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61a401c5a0 con 0x7f6198004d40 2024-04-25T09:15:34.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.122+0000 7f61ad600640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f619006b500 0x7f619006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.122+0000 7f61ad600640 1 -- 172.21.15.62:0/3972747767 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f61a408aa70 con 0x7f6198004d40 2024-04-25T09:15:34.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.123+0000 7f61b5000640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f619006b500 0x7f619006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.124+0000 7f61b5000640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f619006b500 0x7f619006d9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f61b013cbf0 tx=0x7f619c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:34.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.126+0000 7f61ad600640 1 -- 172.21.15.62:0/3972747767 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61a4018030 con 0x7f6198004d40 2024-04-25T09:15:34.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.356+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61b01594c0 con 0x7f6198004d40 2024-04-25T09:15:34.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.358+0000 7f61ad600640 1 -- 172.21.15.62:0/3972747767 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f61a4059130 con 0x7f6198004d40 2024-04-25T09:15:34.345 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:34.345 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f619006b500 msgr2=0x7f619006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f619006b500 0x7f619006d9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f61b013cbf0 tx=0x7f619c008040 comp rx=0 tx=0).stop 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 msgr2=0x7f61b0133870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61b0133870 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f61a40109d0 tx=0x7f61a4010e90 comp rx=0 tx=0).stop 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 shutdown_connections 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f619006b500 0x7f619006d9c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f61b0157de0 0x7f61b01435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f61b00afce0 0x7f61b00ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 --2- 172.21.15.62:0/3972747767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6198004d40 0x7f61b0133870 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.360+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 >> 172.21.15.62:0/3972747767 conn(0x7f61b00b14d0 msgr2=0x7f61b0158980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.361+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 shutdown_connections 2024-04-25T09:15:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.361+0000 7f61b6e00640 1 -- 172.21.15.62:0/3972747767 wait complete. 2024-04-25T09:15:34.365 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:15:34.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:15:34.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.582+0000 7f8cc5000640 1 Processor -- start 2024-04-25T09:15:34.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.582+0000 7f8cc5000640 1 -- start start 2024-04-25T09:15:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.584+0000 7f8cc5000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc006f1f0 0x7f8cc00cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.584+0000 7f8cc5000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8cc00ce7a0 0x7f8cc00cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.584+0000 7f8cc5000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc015f2d0 con 0x7f8cc015de50 2024-04-25T09:15:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.584+0000 7f8cc5000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc015ed70 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.584+0000 7f8cc5000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc015eff0 con 0x7f8cc006f1f0 2024-04-25T09:15:34.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.585+0000 7f8cbea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc006f1f0 0x7f8cc00cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.585+0000 7f8cbea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc006f1f0 0x7f8cc00cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:33696/0 (socket says 172.21.15.62:33696) 2024-04-25T09:15:34.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.585+0000 7f8cbea00640 1 -- 172.21.15.62:0/3748569039 learned_addr learned my addr 172.21.15.62:0/3748569039 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:34.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.585+0000 7f8cbfe00640 1 --2- 172.21.15.62:0/3748569039 >> v2:172.21.15.121:3300/0 conn(0x7f8cc00ce7a0 0x7f8cc00cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.586+0000 7f8cbea00640 1 -- 172.21.15.62:0/3748569039 >> v2:172.21.15.121:3300/0 conn(0x7f8cc00ce7a0 msgr2=0x7f8cc00cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:34.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.586+0000 7f8cbea00640 1 --2- 172.21.15.62:0/3748569039 >> v2:172.21.15.121:3300/0 conn(0x7f8cc00ce7a0 0x7f8cc00cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.586+0000 7f8cbea00640 1 -- 172.21.15.62:0/3748569039 >> v1:172.21.15.121:6789/0 conn(0x7f8cc015de50 legacy=0x7f8cc00ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.586+0000 7f8cbea00640 1 -- 172.21.15.62:0/3748569039 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00ce040 con 0x7f8cc006f1f0 2024-04-25T09:15:34.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.588+0000 7f8cbea00640 1 --2- 172.21.15.62:0/3748569039 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc006f1f0 0x7f8cc00cecf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009870 tx=0x7f8cac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6564ae6ad45c9a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:34.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.588+0000 7f8cbe000640 1 -- 172.21.15.62:0/3748569039 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac037070 con 0x7f8cc006f1f0 2024-04-25T09:15:34.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.588+0000 7f8cbe000640 1 -- 172.21.15.62:0/3748569039 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac032440 con 0x7f8cc006f1f0 2024-04-25T09:15:34.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.588+0000 7f8cbe000640 1 -- 172.21.15.62:0/3748569039 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac032810 con 0x7f8cc006f1f0 2024-04-25T09:15:34.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.589+0000 7f8cc5000640 1 -- 172.21.15.62:0/3748569039 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc006f1f0 msgr2=0x7f8cc00cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.589+0000 7f8cc5000640 1 --2- 172.21.15.62:0/3748569039 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc006f1f0 0x7f8cc00cecf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009870 tx=0x7f8cac02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.589+0000 7f8cc5000640 1 -- 172.21.15.62:0/3748569039 shutdown_connections 2024-04-25T09:15:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.589+0000 7f8cc5000640 1 --2- 172.21.15.62:0/3748569039 >> v2:172.21.15.121:3300/0 conn(0x7f8cc00ce7a0 0x7f8cc00cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.589+0000 7f8cc5000640 1 --2- 172.21.15.62:0/3748569039 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc006f1f0 0x7f8cc00cecf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.589+0000 7f8cc5000640 1 -- 172.21.15.62:0/3748569039 >> 172.21.15.62:0/3748569039 conn(0x7f8cc00b1d40 msgr2=0x7f8cc00b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.589+0000 7f8cc5000640 1 -- 172.21.15.62:0/3748569039 shutdown_connections 2024-04-25T09:15:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.589+0000 7f8cc5000640 1 -- 172.21.15.62:0/3748569039 wait complete. 2024-04-25T09:15:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.591+0000 7f8cc5000640 1 Processor -- start 2024-04-25T09:15:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.591+0000 7f8cc5000640 1 -- start start 2024-04-25T09:15:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cc5000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cc006f1f0 0x7f8cc0158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cc5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cc00ce7a0 0x7f8cc0155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cc5000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc015de50 0x7f8cc01560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cc5000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0161c70 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cc5000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0073e80 con 0x7f8cc006f1f0 2024-04-25T09:15:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cc5000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0161df0 con 0x7f8cc015de50 2024-04-25T09:15:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cbea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cc00ce7a0 0x7f8cc0155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cbea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cc00ce7a0 0x7f8cc0155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37006/0 (socket says 172.21.15.62:37006) 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.592+0000 7f8cbea00640 1 -- 172.21.15.62:0/2994098842 learned_addr learned my addr 172.21.15.62:0/2994098842 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbf400640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cc006f1f0 0x7f8cc0158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbfe00640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc015de50 0x7f8cc01560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbea00640 1 -- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc015de50 msgr2=0x7f8cc01560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbea00640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc015de50 0x7f8cc01560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbea00640 1 -- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cc006f1f0 msgr2=0x7f8cc0158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbea00640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cc006f1f0 0x7f8cc0158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbea00640 1 -- 172.21.15.62:0/2994098842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc015b4a0 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbf400640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cc006f1f0 0x7f8cc0158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cbea00640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cc00ce7a0 0x7f8cc0155b60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f8cac02a200 tx=0x7f8cac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:34.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cb7400640 1 -- 172.21.15.62:0/2994098842 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac037040 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cb7400640 1 -- 172.21.15.62:0/2994098842 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac042070 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cb7400640 1 -- 172.21.15.62:0/2994098842 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac030880 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.593+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc0070f90 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.594+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc01594b0 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.595+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc004ef20 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.596+0000 7f8cb7400640 1 -- 172.21.15.62:0/2994098842 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cac02ad10 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.596+0000 7f8cb7400640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ca006b500 0x7f8ca006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.597+0000 7f8cb7400640 1 -- 172.21.15.62:0/2994098842 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8cac0aa7e0 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.597+0000 7f8cbf400640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ca006b500 0x7f8ca006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:34.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.598+0000 7f8cbf400640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ca006b500 0x7f8ca006d9c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0000f60 tx=0x7f8cb0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.601+0000 7f8cb7400640 1 -- 172.21.15.62:0/2994098842 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cac079580 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.872+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8cc006efd0 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.873+0000 7f8cb7400640 1 -- 172.21.15.62:0/2994098842 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f8cac078e80 con 0x7f8cc00ce7a0 2024-04-25T09:15:34.861 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:34.861 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":762,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580998,"bytes_used":2612928512,"bytes_avail":770481184768,"bytes_total":773094113280,"write_bytes_sec":28919443,"read_op_per_sec":0,"write_op_per_sec":30},"fsmap":{"epoch":78,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T09:15:32.169417+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:15:34.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ca006b500 msgr2=0x7f8ca006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ca006b500 0x7f8ca006d9c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0000f60 tx=0x7f8cb0009040 comp rx=0 tx=0).stop 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cc00ce7a0 msgr2=0x7f8cc0155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cc00ce7a0 0x7f8cc0155b60 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f8cac02a200 tx=0x7f8cac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 shutdown_connections 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8ca006b500 0x7f8ca006d9c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8cc00ce7a0 0x7f8cc0155b60 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8cc015de50 0x7f8cc01560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 --2- 172.21.15.62:0/2994098842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cc006f1f0 0x7f8cc0158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.875+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 >> 172.21.15.62:0/2994098842 conn(0x7f8cc00b1d40 msgr2=0x7f8cc0161170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:34.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.876+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 shutdown_connections 2024-04-25T09:15:34.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:34.876+0000 7f8cc5000640 1 -- 172.21.15.62:0/2994098842 wait complete. 2024-04-25T09:15:34.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:15:34.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:34.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T09:15:35.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.099+0000 7f2955800640 1 Processor -- start 2024-04-25T09:15:35.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.099+0000 7f2955800640 1 -- start start 2024-04-25T09:15:35.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.100+0000 7f2955800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f295006d2c0 0x7f29500ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.100+0000 7f2955800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.100+0000 7f2955800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29500ced40 con 0x7f29500b31a0 2024-04-25T09:15:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.100+0000 7f2955800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29500cf000 con 0x7f295006d2c0 2024-04-25T09:15:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.100+0000 7f2955800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29500cf2e0 con 0x7f295006d6a0 2024-04-25T09:15:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.101+0000 7f2947400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.101+0000 7f294fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f295006d2c0 0x7f29500ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.101+0000 7f294fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f295006d2c0 0x7f29500ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:48676/0 (socket says 172.21.15.62:48676) 2024-04-25T09:15:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.101+0000 7f2947400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:56834/0 (socket says 172.21.15.62:56834) 2024-04-25T09:15:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.101+0000 7f294fe00640 1 -- 172.21.15.62:0/3613617151 learned_addr learned my addr 172.21.15.62:0/3613617151 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.102+0000 7f294fe00640 1 -- 172.21.15.62:0/3613617151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 msgr2=0x7f29500ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.102+0000 7f294fe00640 1 --2- 172.21.15.62:0/3613617151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.102+0000 7f294fe00640 1 -- 172.21.15.62:0/3613617151 >> v1:172.21.15.121:6789/0 conn(0x7f29500b31a0 legacy=0x7f29500b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.102+0000 7f294fe00640 1 -- 172.21.15.62:0/3613617151 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500b26c0 con 0x7f295006d2c0 2024-04-25T09:15:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.102+0000 7f2947400640 1 --2- 172.21.15.62:0/3613617151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:15:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.103+0000 7f294fe00640 1 --2- 172.21.15.62:0/3613617151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295006d2c0 0x7f29500ccde0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2940009e50 tx=0x7f294002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddbae46ebb2192c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f294f400640 1 -- 172.21.15.62:0/3613617151 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940036070 con 0x7f295006d2c0 2024-04-25T09:15:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f294f400640 1 -- 172.21.15.62:0/3613617151 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940031600 con 0x7f295006d2c0 2024-04-25T09:15:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f294f400640 1 -- 172.21.15.62:0/3613617151 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29400319d0 con 0x7f295006d2c0 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f2955800640 1 -- 172.21.15.62:0/3613617151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295006d2c0 msgr2=0x7f29500ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f2955800640 1 --2- 172.21.15.62:0/3613617151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295006d2c0 0x7f29500ccde0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2940009e50 tx=0x7f294002a710 comp rx=0 tx=0).stop 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f2955800640 1 -- 172.21.15.62:0/3613617151 shutdown_connections 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f2955800640 1 --2- 172.21.15.62:0/3613617151 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f2955800640 1 --2- 172.21.15.62:0/3613617151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f295006d2c0 0x7f29500ccde0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f2955800640 1 -- 172.21.15.62:0/3613617151 >> 172.21.15.62:0/3613617151 conn(0x7f2950066f20 msgr2=0x7f29500b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f2955800640 1 -- 172.21.15.62:0/3613617151 shutdown_connections 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.104+0000 7f2955800640 1 -- 172.21.15.62:0/3613617151 wait complete. 2024-04-25T09:15:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.106+0000 7f2955800640 1 Processor -- start 2024-04-25T09:15:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.106+0000 7f2955800640 1 -- start start 2024-04-25T09:15:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.107+0000 7f2955800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f295006d2c0 0x7f29500cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.107+0000 7f2955800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.107+0000 7f2955800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29500b31a0 0x7f295015da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.107+0000 7f2955800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295006db20 con 0x7f295006d2c0 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.107+0000 7f2955800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29500b2c80 con 0x7f29500b31a0 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.107+0000 7f2955800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29501448d0 con 0x7f295006d6a0 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.107+0000 7f294fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f295006d2c0 0x7f29500cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.108+0000 7f294fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f295006d2c0 0x7f29500cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41190/0 (socket says 172.21.15.62:41190) 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.108+0000 7f294fe00640 1 -- 172.21.15.62:0/4024669748 learned_addr learned my addr 172.21.15.62:0/4024669748 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.108+0000 7f294fe00640 1 -- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 msgr2=0x7f29500719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.108+0000 7f294fe00640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.108+0000 7f294fe00640 1 -- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29500b31a0 msgr2=0x7f295015da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.108+0000 7f294fe00640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29500b31a0 0x7f295015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.108+0000 7f294fe00640 1 -- 172.21.15.62:0/4024669748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29500c9ba0 con 0x7f295006d2c0 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.108+0000 7f294fe00640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f295006d2c0 0x7f29500cb870 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f2940034040 tx=0x7f2940004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.109+0000 7f294cc00640 1 -- 172.21.15.62:0/4024669748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940041020 con 0x7f295006d2c0 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.109+0000 7f294cc00640 1 -- 172.21.15.62:0/4024669748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940007820 con 0x7f295006d2c0 2024-04-25T09:15:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.109+0000 7f2955800640 1 -- 172.21.15.62:0/4024669748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29500c9e80 con 0x7f295006d2c0 2024-04-25T09:15:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.109+0000 7f294cc00640 1 -- 172.21.15.62:0/4024669748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940036040 con 0x7f295006d2c0 2024-04-25T09:15:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.109+0000 7f2955800640 1 -- 172.21.15.62:0/4024669748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29500cbdb0 con 0x7f295006d2c0 2024-04-25T09:15:35.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.111+0000 7f294cc00640 1 -- 172.21.15.62:0/4024669748 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2940044020 con 0x7f295006d2c0 2024-04-25T09:15:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.112+0000 7f294cc00640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f292c06b4d0 0x7f292c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.112+0000 7f294cc00640 1 -- 172.21.15.62:0/4024669748 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f29400aaf80 con 0x7f295006d2c0 2024-04-25T09:15:35.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.113+0000 7f2959c22740 1 Processor -- start 2024-04-25T09:15:35.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.113+0000 7f2947400640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f292c06b4d0 0x7f292c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.113+0000 7f2959c22740 1 -- start start 2024-04-25T09:15:35.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f2959c22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56108049fd70 0x5610804a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f2959c22740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804dc440 0x5610804a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f2959c22740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804ca830 0x5610804cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f2959c22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561080403c70 con 0x56108049fd70 2024-04-25T09:15:35.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f2959c22740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610803accf0 con 0x5610804ca830 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f2959c22740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5610803ffd10 con 0x5610804dc440 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f2947400640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f292c06b4d0 0x7f292c06d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2938006e80 tx=0x7f2938006dd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f294fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56108049fd70 0x5610804a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f294fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56108049fd70 0x5610804a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41194/0 (socket says 172.21.15.62:41194) 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.114+0000 7f294fe00640 1 -- 172.21.15.62:0/4111323601 learned_addr learned my addr 172.21.15.62:0/4111323601 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f2947400640 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804dc440 0x5610804a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f294fe00640 1 -- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804dc440 msgr2=0x5610804a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f2954e00640 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804ca830 0x5610804cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f294fe00640 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804dc440 0x5610804a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f294fe00640 1 -- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804ca830 msgr2=0x5610804cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f294fe00640 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804ca830 0x5610804cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f294fe00640 1 -- 172.21.15.62:0/4111323601 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610801e27d0 con 0x56108049fd70 2024-04-25T09:15:35.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f2954e00640 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804ca830 0x5610804cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f294fe00640 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56108049fd70 0x5610804a1ed0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2940075a90 tx=0x7f2940031600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f670d010f7351dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f2944c00640 1 -- 172.21.15.62:0/4111323601 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294007d780 con 0x56108049fd70 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f2944c00640 1 -- 172.21.15.62:0/4111323601 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940081dd0 con 0x56108049fd70 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.115+0000 7f2944c00640 1 -- 172.21.15.62:0/4111323601 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294004dd90 con 0x56108049fd70 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 -- 172.21.15.62:0/4111323601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56108049fd70 msgr2=0x5610804a1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56108049fd70 0x5610804a1ed0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2940075a90 tx=0x7f2940031600 comp rx=0 tx=0).stop 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 -- 172.21.15.62:0/4111323601 shutdown_connections 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804ca830 0x5610804cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804dc440 0x5610804a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 --2- 172.21.15.62:0/4111323601 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56108049fd70 0x5610804a1ed0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 -- 172.21.15.62:0/4111323601 >> 172.21.15.62:0/4111323601 conn(0x561080467a30 msgr2=0x5610804e5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 -- 172.21.15.62:0/4111323601 shutdown_connections 2024-04-25T09:15:35.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.116+0000 7f2959c22740 1 -- 172.21.15.62:0/4111323601 wait complete. 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.117+0000 7f2959c22740 1 -- start start 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.117+0000 7f2959c22740 1 Processor -- start 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.118+0000 7f2959c22740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56108049fd70 0x5610804de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.118+0000 7f2959c22740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804ca830 0x5610804bcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.118+0000 7f2959c22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804dc440 0x5610804e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.118+0000 7f2959c22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x561080403c70 con 0x5610804dc440 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.118+0000 7f2959c22740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610803a23a0 con 0x56108049fd70 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.118+0000 7f2959c22740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5610804bd4c0 con 0x5610804ca830 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.118+0000 7f2959c22740 10 client.0 _ll_register_callbacks cb 0x5610803bfb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804dc440 0x5610804e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804dc440 0x5610804e24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41202/0 (socket says 172.21.15.62:41202) 2024-04-25T09:15:35.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 -- 172.21.15.62:0/1820141431 learned_addr learned my addr 172.21.15.62:0/1820141431 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f294fe00640 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56108049fd70 0x5610804de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2947400640 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804ca830 0x5610804bcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 -- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804ca830 msgr2=0x5610804bcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804ca830 0x5610804bcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 -- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56108049fd70 msgr2=0x5610804de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56108049fd70 0x5610804de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 -- 172.21.15.62:0/1820141431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610801183f0 con 0x5610804dc440 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2947400640 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804ca830 0x5610804bcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.119+0000 7f2954e00640 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804dc440 0x5610804e24c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2948001db0 tx=0x7f294800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac57a686ab8fb459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f293d600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f293d600640 1 -- 172.21.15.62:0/1820141431 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948018060 con 0x5610804dc440 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f2959c22740 1 -- 172.21.15.62:0/1820141431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2940081dd0 con 0x5610804dc440 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f293d600640 1 -- 172.21.15.62:0/1820141431 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f29480052d0 con 0x5610804dc440 2024-04-25T09:15:35.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f293d600640 1 -- 172.21.15.62:0/1820141431 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f294801e050 con 0x5610804dc440 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f2959c22740 10 client.8634 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f2959c22740 1 -- 172.21.15.62:0/1820141431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x5610800d80f0 con 0x5610804dc440 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f293d600640 1 -- 172.21.15.62:0/1820141431 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940081dd0 con 0x5610804dc440 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.120+0000 7f293d600640 1 -- 172.21.15.62:0/1820141431 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2948007020 con 0x5610804dc440 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f2959c22740 10 client.8634 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f2959c22740 10 client.8634 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f2959c22740 20 client.8634 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f2959c22740 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5610804e2ed0 0x5610804cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f2959c22740 4 client.8634 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f2959c22740 1 -- 172.21.15.62:0/1820141431 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610804111b0 con 0x5610804e2ed0 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f294fe00640 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5610804e2ed0 0x5610804cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f294fe00640 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5610804e2ed0 0x5610804cd3a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e46b3090e0f14f6 server_cookie=d33bce0fa9bee824 in_seq=0 out_seq=0 2024-04-25T09:15:35.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.121+0000 7f293d600640 10 client.8634 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:35.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.122+0000 7f293d600640 1 -- 172.21.15.62:0/1820141431 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610804111b0 con 0x5610804e2ed0 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.122+0000 7f293d600640 10 client.8634 handle_command_reply: tid=0 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.122+0000 7f2959c22740 1 client.8634 shutdown 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.122+0000 7f2959c22740 1 -- 172.21.15.62:0/1820141431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804dc440 msgr2=0x5610804e24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.122+0000 7f2959c22740 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804dc440 0x5610804e24c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2948001db0 tx=0x7f294800d520 comp rx=0 tx=0).stop 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 1 -- 172.21.15.62:0/1820141431 shutdown_connections 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x5610804e2ed0 0x5610804cd3a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804dc440 0x5610804e24c0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x5610804ca830 0x5610804bcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 1 --2- 172.21.15.62:0/1820141431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56108049fd70 0x5610804de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 1 -- 172.21.15.62:0/1820141431 >> 172.21.15.62:0/1820141431 conn(0x561080467a30 msgr2=0x5610804e8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f293d600640 0 client.8634 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 1 -- 172.21.15.62:0/1820141431 shutdown_connections 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 1 -- 172.21.15.62:0/1820141431 wait complete. 2024-04-25T09:15:35.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.123+0000 7f2959c22740 20 client.8634 trim_cache size 0 max 16384 2024-04-25T09:15:35.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.139+0000 7f2959c22740 1 Processor -- start 2024-04-25T09:15:35.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.139+0000 7f2959c22740 1 -- start start 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.140+0000 7f2959c22740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080506e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.140+0000 7f2959c22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610804cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.140+0000 7f2959c22740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.140+0000 7f2959c22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29501fff20 con 0x5610804ca830 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.140+0000 7f2959c22740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29501ff2a0 con 0x5610804dc440 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.140+0000 7f2959c22740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2950070db0 con 0x56108049fd70 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2947400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610804cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2947400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610804cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41212/0 (socket says 172.21.15.62:41212) 2024-04-25T09:15:35.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2947400640 1 -- 172.21.15.62:0/2409576697 learned_addr learned my addr 172.21.15.62:0/2409576697 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f294fe00640 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080506e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2954e00640 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2947400640 1 -- 172.21.15.62:0/2409576697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 msgr2=0x561080506e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2947400640 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080506e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2947400640 1 -- 172.21.15.62:0/2409576697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 msgr2=0x5610804cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2947400640 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.141+0000 7f2947400640 1 -- 172.21.15.62:0/2409576697 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610804e2830 con 0x5610804ca830 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.142+0000 7f2947400640 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610804cd3a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f293800eaf0 tx=0x7f2938015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a2f86692399d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.142+0000 7f293fe00640 1 -- 172.21.15.62:0/2409576697 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938015cd0 con 0x5610804ca830 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.142+0000 7f293fe00640 1 -- 172.21.15.62:0/2409576697 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2938026070 con 0x5610804ca830 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.142+0000 7f293fe00640 1 -- 172.21.15.62:0/2409576697 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938013670 con 0x5610804ca830 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.142+0000 7f2959c22740 1 -- 172.21.15.62:0/2409576697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 msgr2=0x5610804cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.142+0000 7f2959c22740 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610804cd3a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f293800eaf0 tx=0x7f2938015000 comp rx=0 tx=0).stop 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.143+0000 7f2959c22740 1 -- 172.21.15.62:0/2409576697 shutdown_connections 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.143+0000 7f2959c22740 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610804cd3a0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.143+0000 7f2959c22740 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.143+0000 7f2959c22740 1 --2- 172.21.15.62:0/2409576697 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080506e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.143+0000 7f2959c22740 1 -- 172.21.15.62:0/2409576697 >> 172.21.15.62:0/2409576697 conn(0x561080467a30 msgr2=0x5610804e60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.143+0000 7f2959c22740 1 -- 172.21.15.62:0/2409576697 shutdown_connections 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.143+0000 7f2959c22740 1 -- 172.21.15.62:0/2409576697 wait complete. 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.143+0000 7f2959c22740 1 -- start start 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.144+0000 7f2959c22740 1 Processor -- start 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.144+0000 7f2959c22740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080496660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2959c22740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610805044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2959c22740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2959c22740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x56108049cab0 con 0x5610804ca830 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2959c22740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29501f76b0 con 0x5610804dc440 2024-04-25T09:15:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2959c22740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x5610804e3100 con 0x56108049fd70 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2959c22740 10 client.0 _ll_register_callbacks cb 0x7f2948007c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2947400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610805044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2947400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610805044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41222/0 (socket says 172.21.15.62:41222) 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2947400640 1 -- 172.21.15.62:0/431841139 learned_addr learned my addr 172.21.15.62:0/431841139 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f294fe00640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080496660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2954e00640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2947400640 1 -- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 msgr2=0x561080496660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2947400640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080496660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2947400640 1 -- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 msgr2=0x5610804db3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2947400640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804db3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2947400640 1 -- 172.21.15.62:0/431841139 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610804df870 con 0x5610804ca830 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f294fe00640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080496660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.145+0000 7f2954e00640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f2947400640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610805044d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f2938023040 tx=0x7f2938009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19e2f8b4a11eacd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f293d600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f293d600640 1 -- 172.21.15.62:0/431841139 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938021bf0 con 0x5610804ca830 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f293d600640 1 -- 172.21.15.62:0/431841139 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2938021d90 con 0x5610804ca830 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f2959c22740 1 -- 172.21.15.62:0/431841139 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2938026070 con 0x5610804ca830 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f293d600640 1 -- 172.21.15.62:0/431841139 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f293803b050 con 0x5610804ca830 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f2959c22740 10 client.8640 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f2959c22740 1 -- 172.21.15.62:0/431841139 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x5610804c83b0 con 0x5610804ca830 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.146+0000 7f293d600640 1 -- 172.21.15.62:0/431841139 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2938026070 con 0x5610804ca830 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.147+0000 7f293d600640 1 -- 172.21.15.62:0/431841139 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f293803e020 con 0x5610804ca830 2024-04-25T09:15:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.147+0000 7f2959c22740 10 client.8640 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.147+0000 7f2959c22740 10 client.8640 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.147+0000 7f2959c22740 20 client.8640 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.147+0000 7f2959c22740 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x561080504a10 0x5610804db910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.147+0000 7f2959c22740 4 client.8640 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.147+0000 7f2959c22740 1 -- 172.21.15.62:0/431841139 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x56108040b500 con 0x561080504a10 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.147+0000 7f294fe00640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x561080504a10 0x5610804db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f294fe00640 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x561080504a10 0x5610804db910 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac54305ae51ced9 server_cookie=5c6aecad6e06980c in_seq=0 out_seq=0 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f293d600640 10 client.8640 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f293d600640 1 -- 172.21.15.62:0/431841139 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56108040b500 con 0x561080504a10 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f293d600640 10 client.8640 handle_command_reply: tid=0 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 client.8640 shutdown 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 -- 172.21.15.62:0/431841139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 msgr2=0x5610805044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610805044d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f2938023040 tx=0x7f2938009190 comp rx=0 tx=0).stop 2024-04-25T09:15:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 -- 172.21.15.62:0/431841139 shutdown_connections 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x561080504a10 0x5610804db910 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5610804ca830 0x5610805044d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610804dc440 0x5610804db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 --2- 172.21.15.62:0/431841139 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x56108049fd70 0x561080496660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f293d600640 0 client.8640 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 -- 172.21.15.62:0/431841139 >> 172.21.15.62:0/431841139 conn(0x561080467a30 msgr2=0x5610804cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 -- 172.21.15.62:0/431841139 shutdown_connections 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.148+0000 7f2959c22740 1 -- 172.21.15.62:0/431841139 wait complete. 2024-04-25T09:15:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2959c22740 20 client.8640 trim_cache size 0 max 16384 2024-04-25T09:15:35.141 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:15:35.141 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:15:35.141 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:15:35.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 -- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f292c06b4d0 msgr2=0x7f292c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f292c06b4d0 0x7f292c06d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2938006e80 tx=0x7f2938006dd0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 -- 172.21.15.62:0/4024669748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f295006d2c0 msgr2=0x7f29500cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f295006d2c0 0x7f29500cb870 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f2940034040 tx=0x7f2940004a40 comp rx=0 tx=0).stop 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 -- 172.21.15.62:0/4024669748 shutdown_connections 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f292c06b4d0 0x7f292c06d990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29500b31a0 0x7f295015da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f295006d6a0 0x7f29500719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 --2- 172.21.15.62:0/4024669748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f295006d2c0 0x7f29500cb870 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 -- 172.21.15.62:0/4024669748 >> 172.21.15.62:0/4024669748 conn(0x7f2950066f20 msgr2=0x7f29500cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.149+0000 7f2955800640 1 -- 172.21.15.62:0/4024669748 shutdown_connections 2024-04-25T09:15:35.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.150+0000 7f2955800640 1 -- 172.21.15.62:0/4024669748 wait complete. 2024-04-25T09:15:35.154 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:15:35.155 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:15:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.377+0000 7f5d7e200640 1 Processor -- start 2024-04-25T09:15:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.378+0000 7f5d7e200640 1 -- start start 2024-04-25T09:15:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.379+0000 7f5d7e200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d780b40e0 0x7f5d7806c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.379+0000 7f5d7e200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7806e560 con 0x7f5d78162cc0 2024-04-25T09:15:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.379+0000 7f5d7e200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7806dd00 con 0x7f5d780ceeb0 2024-04-25T09:15:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.379+0000 7f5d7e200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d780cf360 con 0x7f5d780b40e0 2024-04-25T09:15:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.380+0000 7f5d77400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d780b40e0 0x7f5d7806c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.380+0000 7f5d77400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5d780b40e0 0x7f5d7806c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41232/0 (socket says 172.21.15.62:41232) 2024-04-25T09:15:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.380+0000 7f5d77e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5d78162cc0 0x7f5d7816bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59886/0 (socket says 172.21.15.62:59886) 2024-04-25T09:15:35.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.380+0000 7f5d77400640 1 -- 172.21.15.62:0/3728860054 learned_addr learned my addr 172.21.15.62:0/3728860054 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.380+0000 7f5d77400640 1 -- 172.21.15.62:0/3728860054 >> v1:172.21.15.121:6789/0 conn(0x7f5d780ceeb0 legacy=0x7f5d781625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.380+0000 7f5d77400640 1 -- 172.21.15.62:0/3728860054 >> v1:172.21.15.62:6789/0 conn(0x7f5d78162cc0 legacy=0x7f5d7816bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.380+0000 7f5d77400640 1 -- 172.21.15.62:0/3728860054 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7806dfa0 con 0x7f5d780b40e0 2024-04-25T09:15:35.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.380+0000 7f5d76a00640 1 -- 172.21.15.62:0/3728860054 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731290240 0 0) 0x7f5d7806e560 con 0x7f5d78162cc0 2024-04-25T09:15:35.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.382+0000 7f5d77400640 1 --2- 172.21.15.62:0/3728860054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d7806c380 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5d780cf4e0 tx=0x7f5d6802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e6aec7342165ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.382+0000 7f5d76a00640 1 -- 172.21.15.62:0/3728860054 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68036070 con 0x7f5d780b40e0 2024-04-25T09:15:35.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.382+0000 7f5d76a00640 1 -- 172.21.15.62:0/3728860054 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d68031600 con 0x7f5d780b40e0 2024-04-25T09:15:35.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.382+0000 7f5d76a00640 1 -- 172.21.15.62:0/3728860054 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680319d0 con 0x7f5d780b40e0 2024-04-25T09:15:35.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.383+0000 7f5d7e200640 1 -- 172.21.15.62:0/3728860054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 msgr2=0x7f5d7806c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.383+0000 7f5d7e200640 1 --2- 172.21.15.62:0/3728860054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d7806c380 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5d780cf4e0 tx=0x7f5d6802a680 comp rx=0 tx=0).stop 2024-04-25T09:15:35.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.383+0000 7f5d7e200640 1 -- 172.21.15.62:0/3728860054 shutdown_connections 2024-04-25T09:15:35.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.383+0000 7f5d7e200640 1 --2- 172.21.15.62:0/3728860054 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d7806c380 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.383+0000 7f5d7e200640 1 -- 172.21.15.62:0/3728860054 >> 172.21.15.62:0/3728860054 conn(0x7f5d780b1410 msgr2=0x7f5d780b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.383+0000 7f5d7e200640 1 -- 172.21.15.62:0/3728860054 shutdown_connections 2024-04-25T09:15:35.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.383+0000 7f5d7e200640 1 -- 172.21.15.62:0/3728860054 wait complete. 2024-04-25T09:15:35.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.385+0000 7f5d7e200640 1 Processor -- start 2024-04-25T09:15:35.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.385+0000 7f5d7e200640 1 -- start start 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d7e200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d780ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d7e200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d780ceeb0 0x7f5d780aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d7e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d78162cc0 0x7f5d780acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d7e200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78072460 con 0x7f5d780b40e0 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d7e200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78071f40 con 0x7f5d78162cc0 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d7e200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780725e0 con 0x7f5d780ceeb0 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d77400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d780ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d77400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d780ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41234/0 (socket says 172.21.15.62:41234) 2024-04-25T09:15:35.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d77400640 1 -- 172.21.15.62:0/549848238 learned_addr learned my addr 172.21.15.62:0/549848238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d77400640 1 -- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d780ceeb0 msgr2=0x7f5d780aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d77400640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d780ceeb0 0x7f5d780aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d77400640 1 -- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d78162cc0 msgr2=0x7f5d780acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d77400640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d78162cc0 0x7f5d780acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.386+0000 7f5d77400640 1 -- 172.21.15.62:0/549848238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780a9ce0 con 0x7f5d780b40e0 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.387+0000 7f5d77400640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d780ad9d0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f5d68034040 tx=0x7f5d68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.387+0000 7f5d6fe00640 1 -- 172.21.15.62:0/549848238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68036050 con 0x7f5d780b40e0 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.387+0000 7f5d6fe00640 1 -- 172.21.15.62:0/549848238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d68042070 con 0x7f5d780b40e0 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.387+0000 7f5d6fe00640 1 -- 172.21.15.62:0/549848238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68007c50 con 0x7f5d780b40e0 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.387+0000 7f5d7e200640 1 -- 172.21.15.62:0/549848238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780aaae0 con 0x7f5d780b40e0 2024-04-25T09:15:35.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.387+0000 7f5d7e200640 1 -- 172.21.15.62:0/549848238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d780aa520 con 0x7f5d780b40e0 2024-04-25T09:15:35.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.390+0000 7f5d6fe00640 1 -- 172.21.15.62:0/549848238 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d68047460 con 0x7f5d780b40e0 2024-04-25T09:15:35.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.390+0000 7f5d6fe00640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d5406b4d0 0x7f5d5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.391+0000 7f5d6fe00640 1 -- 172.21.15.62:0/549848238 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5d680ab320 con 0x7f5d780b40e0 2024-04-25T09:15:35.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.391+0000 7f5d6ea00640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d5406b4d0 0x7f5d5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.391+0000 7f5d811ae740 1 Processor -- start 2024-04-25T09:15:35.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.391+0000 7f5d811ae740 1 -- start start 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.392+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 0x562d4e400d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.392+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e43b280 0x562d4e3fef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.392+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e429af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.392+0000 7f5d811ae740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4e362c70 con 0x562d4e429670 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.392+0000 7f5d811ae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4e30bcf0 con 0x562d4e43b280 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.392+0000 7f5d811ae740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d4e35ed10 con 0x562d4e3febb0 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.392+0000 7f5d6ea00640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d5406b4d0 0x7f5d5406d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5d60000f60 tx=0x7f5d600079b0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e429af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e429af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41244/0 (socket says 172.21.15.62:41244) 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 -- 172.21.15.62:0/2872812207 learned_addr learned my addr 172.21.15.62:0/2872812207 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77400640 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 0x562d4e400d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 -- 172.21.15.62:0/2872812207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 msgr2=0x562d4e400d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 0x562d4e400d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 -- 172.21.15.62:0/2872812207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e43b280 msgr2=0x562d4e3fef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d6ea00640 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e43b280 0x562d4e3fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e43b280 0x562d4e3fef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 -- 172.21.15.62:0/2872812207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d4e1417d0 con 0x562d4e429670 2024-04-25T09:15:35.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.393+0000 7f5d77e00640 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e429af0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f5d700015d0 tx=0x7f5d70001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae6cbc080fd23a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.394+0000 7f5d67400640 1 -- 172.21.15.62:0/2872812207 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d700072e0 con 0x562d4e429670 2024-04-25T09:15:35.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.394+0000 7f5d67400640 1 -- 172.21.15.62:0/2872812207 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d700074a0 con 0x562d4e429670 2024-04-25T09:15:35.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.394+0000 7f5d67400640 1 -- 172.21.15.62:0/2872812207 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70004a20 con 0x562d4e429670 2024-04-25T09:15:35.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.394+0000 7f5d811ae740 1 -- 172.21.15.62:0/2872812207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 msgr2=0x562d4e429af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.394+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e429af0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f5d700015d0 tx=0x7f5d70001a90 comp rx=0 tx=0).stop 2024-04-25T09:15:35.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.395+0000 7f5d811ae740 1 -- 172.21.15.62:0/2872812207 shutdown_connections 2024-04-25T09:15:35.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.395+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e429af0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.395+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e43b280 0x562d4e3fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.395+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2872812207 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 0x562d4e400d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.395+0000 7f5d811ae740 1 -- 172.21.15.62:0/2872812207 >> 172.21.15.62:0/2872812207 conn(0x562d4e3c6a30 msgr2=0x562d4e444670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.395+0000 7f5d811ae740 1 -- 172.21.15.62:0/2872812207 shutdown_connections 2024-04-25T09:15:35.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.395+0000 7f5d811ae740 1 -- 172.21.15.62:0/2872812207 wait complete. 2024-04-25T09:15:35.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.395+0000 7f5d811ae740 1 -- start start 2024-04-25T09:15:35.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.396+0000 7f5d811ae740 1 Processor -- start 2024-04-25T09:15:35.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 0x562d4e43ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e42a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e43b280 0x562d4e4148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d811ae740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4e3013a0 con 0x562d4e429670 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d811ae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4e4271c0 con 0x562d4e3febb0 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d811ae740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d4e42b2f0 con 0x562d4e43b280 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d811ae740 10 client.0 _ll_register_callbacks cb 0x562d4e31eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d6ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e42a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d6ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e42a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41250/0 (socket says 172.21.15.62:41250) 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d6ea00640 1 -- 172.21.15.62:0/2119184622 learned_addr learned my addr 172.21.15.62:0/2119184622 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d77400640 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 0x562d4e43ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d77e00640 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e43b280 0x562d4e4148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d6ea00640 1 -- 172.21.15.62:0/2119184622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e43b280 msgr2=0x562d4e4148a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d6ea00640 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e43b280 0x562d4e4148a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d6ea00640 1 -- 172.21.15.62:0/2119184622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 msgr2=0x562d4e43ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.397+0000 7f5d6ea00640 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 0x562d4e43ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.398+0000 7f5d6ea00640 1 -- 172.21.15.62:0/2119184622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d4e0773f0 con 0x562d4e429670 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.398+0000 7f5d6ea00640 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e42a8c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000ce20 tx=0x7f5d60001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e541af9661da40a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.398+0000 7f5d5be00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.398+0000 7f5d5be00640 1 -- 172.21.15.62:0/2119184622 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60014e30 con 0x562d4e429670 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.398+0000 7f5d5be00640 1 -- 172.21.15.62:0/2119184622 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5d60022b10 con 0x562d4e429670 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.398+0000 7f5d811ae740 1 -- 172.21.15.62:0/2119184622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d700074a0 con 0x562d4e429670 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.398+0000 7f5d5be00640 1 -- 172.21.15.62:0/2119184622 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d60022d50 con 0x562d4e429670 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.398+0000 7f5d811ae740 10 client.8655 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d811ae740 1 -- 172.21.15.62:0/2119184622 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x562d4e0370f0 con 0x562d4e429670 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d5be00640 1 -- 172.21.15.62:0/2119184622 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d700074a0 con 0x562d4e429670 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d5be00640 1 -- 172.21.15.62:0/2119184622 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d60029050 con 0x562d4e429670 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d811ae740 10 client.8655 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:35.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d811ae740 10 client.8655 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d811ae740 20 client.8655 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x562d4e457eb0 0x562d4e4582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d811ae740 4 client.8655 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d811ae740 1 -- 172.21.15.62:0/2119184622 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d4e3701b0 con 0x562d4e457eb0 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.399+0000 7f5d77400640 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x562d4e457eb0 0x562d4e4582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.400+0000 7f5d77400640 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x562d4e457eb0 0x562d4e4582f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01b29562e04acdf server_cookie=74f1d9ef10700c49 in_seq=0 out_seq=0 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.400+0000 7f5d5be00640 10 client.8655 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.400+0000 7f5d5be00640 1 -- 172.21.15.62:0/2119184622 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d4e3701b0 con 0x562d4e457eb0 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.400+0000 7f5d5be00640 10 client.8655 handle_command_reply: tid=0 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.400+0000 7f5d811ae740 1 client.8655 shutdown 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 -- 172.21.15.62:0/2119184622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 msgr2=0x562d4e42a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e42a8c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000ce20 tx=0x7f5d60001120 comp rx=0 tx=0).stop 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 -- 172.21.15.62:0/2119184622 shutdown_connections 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x562d4e457eb0 0x562d4e4582f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e429670 0x562d4e42a8c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e43b280 0x562d4e4148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 --2- 172.21.15.62:0/2119184622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 0x562d4e43ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 -- 172.21.15.62:0/2119184622 >> 172.21.15.62:0/2119184622 conn(0x562d4e3c6a30 msgr2=0x562d4e426920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d5be00640 0 client.8655 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 -- 172.21.15.62:0/2119184622 shutdown_connections 2024-04-25T09:15:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 1 -- 172.21.15.62:0/2119184622 wait complete. 2024-04-25T09:15:35.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.401+0000 7f5d811ae740 20 client.8655 trim_cache size 0 max 16384 2024-04-25T09:15:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.418+0000 7f5d811ae740 1 Processor -- start 2024-04-25T09:15:35.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.418+0000 7f5d811ae740 1 -- start start 2024-04-25T09:15:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.419+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 0x562d4e43e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.419+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e429670 0x562d4e42a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.419+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e426920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.419+0000 7f5d811ae740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78204360 con 0x562d4e43b280 2024-04-25T09:15:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.419+0000 7f5d811ae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78203520 con 0x562d4e429670 2024-04-25T09:15:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.419+0000 7f5d811ae740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7815d410 con 0x562d4e3febb0 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.419+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e426920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.419+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e426920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41260/0 (socket says 172.21.15.62:41260) 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d77e00640 1 -- 172.21.15.62:0/1651098127 learned_addr learned my addr 172.21.15.62:0/1651098127 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d77400640 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 0x562d4e43e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d6ea00640 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e429670 0x562d4e42a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d77e00640 1 -- 172.21.15.62:0/1651098127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 msgr2=0x562d4e43e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d77e00640 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 0x562d4e43e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d77e00640 1 -- 172.21.15.62:0/1651098127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e429670 msgr2=0x562d4e42a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d77e00640 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e429670 0x562d4e42a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d77e00640 1 -- 172.21.15.62:0/1651098127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d4e3ffea0 con 0x562d4e43b280 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.420+0000 7f5d77e00640 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e426920 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c4b0 tx=0x7f5d70006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77db2a014bdaac1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d66a00640 1 -- 172.21.15.62:0/1651098127 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70024070 con 0x562d4e43b280 2024-04-25T09:15:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d66a00640 1 -- 172.21.15.62:0/1651098127 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70001150 con 0x562d4e43b280 2024-04-25T09:15:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d66a00640 1 -- 172.21.15.62:0/1651098127 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70007520 con 0x562d4e43b280 2024-04-25T09:15:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 -- 172.21.15.62:0/1651098127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 msgr2=0x562d4e426920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e426920 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c4b0 tx=0x7f5d70006000 comp rx=0 tx=0).stop 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 -- 172.21.15.62:0/1651098127 shutdown_connections 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e426920 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e429670 0x562d4e42a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 --2- 172.21.15.62:0/1651098127 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e3febb0 0x562d4e43e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 -- 172.21.15.62:0/1651098127 >> 172.21.15.62:0/1651098127 conn(0x562d4e3c6a30 msgr2=0x562d4e414670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 -- 172.21.15.62:0/1651098127 shutdown_connections 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.421+0000 7f5d811ae740 1 -- 172.21.15.62:0/1651098127 wait complete. 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.422+0000 7f5d811ae740 1 -- start start 2024-04-25T09:15:35.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.422+0000 7f5d811ae740 1 Processor -- start 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.423+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 0x562d4e429130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.423+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e429670 0x562d4e43de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.423+0000 7f5d811ae740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e43e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.423+0000 7f5d811ae740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7815da10 con 0x562d4e43b280 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.423+0000 7f5d811ae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4e35ee90 con 0x562d4e3febb0 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.423+0000 7f5d811ae740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d4e494440 con 0x562d4e429670 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.423+0000 7f5d811ae740 10 client.0 _ll_register_callbacks cb 0x7f5d600012b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e43e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e43e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41266/0 (socket says 172.21.15.62:41266) 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 -- 172.21.15.62:0/190752954 learned_addr learned my addr 172.21.15.62:0/190752954 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d6ea00640 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e429670 0x562d4e43de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77400640 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 0x562d4e429130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 -- 172.21.15.62:0/190752954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e429670 msgr2=0x562d4e43de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e429670 0x562d4e43de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 -- 172.21.15.62:0/190752954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 msgr2=0x562d4e429130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 0x562d4e429130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 -- 172.21.15.62:0/190752954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d4e428170 con 0x562d4e43b280 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d77e00640 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e43e390 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c4b0 tx=0x7f5d70001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b56976f1687ed8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.424+0000 7f5d5be00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d5be00640 1 -- 172.21.15.62:0/190752954 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7002d060 con 0x562d4e43b280 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d5be00640 1 -- 172.21.15.62:0/190752954 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5d700047e0 con 0x562d4e43b280 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d811ae740 1 -- 172.21.15.62:0/190752954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d70001150 con 0x562d4e43b280 2024-04-25T09:15:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d5be00640 1 -- 172.21.15.62:0/190752954 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d70024040 con 0x562d4e43b280 2024-04-25T09:15:35.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d811ae740 10 client.8661 fetch_fsmap learned FSMap version 78 2024-04-25T09:15:35.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d811ae740 1 -- 172.21.15.62:0/190752954 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x562d4e4941e0 con 0x562d4e43b280 2024-04-25T09:15:35.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d5be00640 1 -- 172.21.15.62:0/190752954 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70001150 con 0x562d4e43b280 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d5be00640 1 -- 172.21.15.62:0/190752954 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d70036020 con 0x562d4e43b280 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d811ae740 10 client.8661 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.425+0000 7f5d811ae740 10 client.8661 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.426+0000 7f5d811ae740 20 client.8661 populate_metadata read hostname 'smithi062' 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.426+0000 7f5d811ae740 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x562d4e457830 0x562d4e4158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.426+0000 7f5d811ae740 4 client.8661 mds_command: new command op to 8304 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.426+0000 7f5d811ae740 1 -- 172.21.15.62:0/190752954 --> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d4e36a500 con 0x562d4e457830 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.426+0000 7f5d77400640 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x562d4e457830 0x562d4e4158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.426+0000 7f5d77400640 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x562d4e457830 0x562d4e4158b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfd0ad187c6c1 server_cookie=2123b728db0d507f in_seq=0 out_seq=0 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.426+0000 7f5d5be00640 10 client.8661 ms_handle_connect on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d5be00640 1 -- 172.21.15.62:0/190752954 <== mds.0 v2:172.21.15.62:6834/2051823365 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562d4e36a500 con 0x562d4e457830 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d5be00640 10 client.8661 handle_command_reply: tid=0 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 client.8661 shutdown 2024-04-25T09:15:35.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 -- 172.21.15.62:0/190752954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 msgr2=0x562d4e43e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e43e390 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c4b0 tx=0x7f5d70001080 comp rx=0 tx=0).stop 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 -- 172.21.15.62:0/190752954 shutdown_connections 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.62:6834/2051823365,v1:172.21.15.62:6835/2051823365] conn(0x562d4e457830 0x562d4e4158b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d4e43b280 0x562d4e43e390 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x562d4e429670 0x562d4e43de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d5be00640 0 client.8661 ms_handle_reset on v2:172.21.15.62:6834/2051823365 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 --2- 172.21.15.62:0/190752954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562d4e3febb0 0x562d4e429130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 -- 172.21.15.62:0/190752954 >> 172.21.15.62:0/190752954 conn(0x562d4e3c6a30 msgr2=0x562d4e4178e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 -- 172.21.15.62:0/190752954 shutdown_connections 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 1 -- 172.21.15.62:0/190752954 wait complete. 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.427+0000 7f5d811ae740 20 client.8661 trim_cache size 0 max 16384 2024-04-25T09:15:35.417 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:15:35.418 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "no active scrubs running", 2024-04-25T09:15:35.418 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": {} 2024-04-25T09:15:35.418 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:15:35.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 -- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d5406b4d0 msgr2=0x7f5d5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d5406b4d0 0x7f5d5406d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5d60000f60 tx=0x7f5d600079b0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 -- 172.21.15.62:0/549848238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 msgr2=0x7f5d780ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d780ad9d0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f5d68034040 tx=0x7f5d68004a40 comp rx=0 tx=0).stop 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 -- 172.21.15.62:0/549848238 shutdown_connections 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5d5406b4d0 0x7f5d5406d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d78162cc0 0x7f5d780acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5d780ceeb0 0x7f5d780aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 --2- 172.21.15.62:0/549848238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5d780b40e0 0x7f5d780ad9d0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 -- 172.21.15.62:0/549848238 >> 172.21.15.62:0/549848238 conn(0x7f5d780b1410 msgr2=0x7f5d781640b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 -- 172.21.15.62:0/549848238 shutdown_connections 2024-04-25T09:15:35.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.428+0000 7f5d7e200640 1 -- 172.21.15.62:0/549848238 wait complete. 2024-04-25T09:15:35.432 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:15:35.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.661+0000 7f7ada600640 1 Processor -- start 2024-04-25T09:15:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.661+0000 7f7ada600640 1 -- start start 2024-04-25T09:15:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.662+0000 7f7ada600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7ad413d970 0x7f7ad40b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.662+0000 7f7ada600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad415b220 con 0x7f7ad40b1050 2024-04-25T09:15:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.662+0000 7f7ada600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad415b500 con 0x7f7ad406d900 2024-04-25T09:15:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.662+0000 7f7ada600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad415b7e0 con 0x7f7ad413d970 2024-04-25T09:15:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7ad40b1050 0x7f7ad406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59896/0 (socket says 172.21.15.62:59896) 2024-04-25T09:15:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3e00640 1 -- 172.21.15.62:0/1514504231 learned_addr learned my addr 172.21.15.62:0/1514504231 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad9200640 1 --2- 172.21.15.62:0/1514504231 >> v2:172.21.15.121:3300/0 conn(0x7f7ad413d970 0x7f7ad40b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026240801 0 0) 0x7f7ad415b220 con 0x7f7ad40b1050 2024-04-25T09:15:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7abc003520 con 0x7f7ad40b1050 2024-04-25T09:15:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832138326 0 0) 0x7f7abc003520 con 0x7f7ad40b1050 2024-04-25T09:15:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 >> v2:172.21.15.121:3300/0 conn(0x7f7ad413d970 msgr2=0x7f7ad40b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3400640 1 --2- 172.21.15.62:0/1514504231 >> v2:172.21.15.121:3300/0 conn(0x7f7ad413d970 0x7f7ad40b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 >> v1:172.21.15.121:6789/0 conn(0x7f7ad406d900 legacy=0x7f7ad413e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad415bac0 con 0x7f7ad40b1050 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.663+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f7ac0001670 con 0x7f7ad40b1050 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 >> v1:172.21.15.62:6789/0 conn(0x7f7ad40b1050 legacy=0x7f7ad406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3400640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc0039b0 0x7f7abc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3400640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc004800 0x7f7abc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3400640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7abc0051c0 0x7f7abc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad415b7e0 con 0x7f7abc004800 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc00c060 con 0x7f7abc0039b0 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc00c260 con 0x7f7abc0051c0 2024-04-25T09:15:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3e00640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc004800 0x7f7abc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad9200640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc0039b0 0x7f7abc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad9c00640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7abc0051c0 0x7f7abc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3e00640 1 -- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7abc0051c0 msgr2=0x7f7abc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3e00640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7abc0051c0 0x7f7abc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3e00640 1 -- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc0039b0 msgr2=0x7f7abc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3e00640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc0039b0 0x7f7abc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.664+0000 7f7ad3e00640 1 -- 172.21.15.62:0/1514504231 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc003520 con 0x7f7abc004800 2024-04-25T09:15:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.665+0000 7f7ad9c00640 1 -- 172.21.15.62:0/1514504231 reap_dead start 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.665+0000 7f7ad9200640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc0039b0 0x7f7abc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.665+0000 7f7ad3e00640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc004800 0x7f7abc004c80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0007cb0 tx=0x7f7abc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54285be9fab4ac90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.666+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0007cf0 con 0x7f7abc004800 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.666+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac0036070 con 0x7f7abc004800 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.666+0000 7f7ad3400640 1 -- 172.21.15.62:0/1514504231 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0005d70 con 0x7f7abc004800 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.666+0000 7f7ada600640 1 -- 172.21.15.62:0/1514504231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc004800 msgr2=0x7f7abc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.666+0000 7f7ada600640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc004800 0x7f7abc004c80 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0007cb0 tx=0x7f7abc005830 comp rx=0 tx=0).stop 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.666+0000 7f7ada600640 1 -- 172.21.15.62:0/1514504231 shutdown_connections 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.666+0000 7f7ada600640 1 --2- 172.21.15.62:0/1514504231 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7abc004800 0x7f7abc004c80 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.666+0000 7f7ada600640 1 -- 172.21.15.62:0/1514504231 >> 172.21.15.62:0/1514504231 conn(0x7f7ad40b14d0 msgr2=0x7f7ad40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.667+0000 7f7ada600640 1 -- 172.21.15.62:0/1514504231 shutdown_connections 2024-04-25T09:15:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.667+0000 7f7ada600640 1 -- 172.21.15.62:0/1514504231 wait complete. 2024-04-25T09:15:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.668+0000 7f7ada600640 1 Processor -- start 2024-04-25T09:15:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.669+0000 7f7ada600640 1 -- start start 2024-04-25T09:15:35.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.669+0000 7f7ada600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc004800 0x7f7ad4147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.669+0000 7f7ada600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ad4135040 0x7f7ad4143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.669+0000 7f7ada600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad4143c80 0x7f7ad413a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.669+0000 7f7ada600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40b0b10 con 0x7f7ad4143c80 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.669+0000 7f7ada600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad415af00 con 0x7f7abc004800 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.669+0000 7f7ada600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40b0c90 con 0x7f7ad4135040 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad4143c80 0x7f7ad413a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad4143c80 0x7f7ad413a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41292/0 (socket says 172.21.15.62:41292) 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9c00640 1 -- 172.21.15.62:0/351500265 learned_addr learned my addr 172.21.15.62:0/351500265 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9200640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc004800 0x7f7ad4147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad3e00640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ad4135040 0x7f7ad4143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9c00640 1 -- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ad4135040 msgr2=0x7f7ad4143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9c00640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ad4135040 0x7f7ad4143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9c00640 1 -- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc004800 msgr2=0x7f7ad4147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9c00640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc004800 0x7f7ad4147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.670+0000 7f7ad9c00640 1 -- 172.21.15.62:0/351500265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40b2710 con 0x7f7ad4143c80 2024-04-25T09:15:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.671+0000 7f7ad9c00640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad4143c80 0x7f7ad413a180 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4140fc0 tx=0x7f7acc00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.671+0000 7f7ad0c00640 1 -- 172.21.15.62:0/351500265 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc018070 con 0x7f7ad4143c80 2024-04-25T09:15:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.671+0000 7f7ad0c00640 1 -- 172.21.15.62:0/351500265 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc003d80 con 0x7f7ad4143c80 2024-04-25T09:15:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.671+0000 7f7ad0c00640 1 -- 172.21.15.62:0/351500265 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0052b0 con 0x7f7ad4143c80 2024-04-25T09:15:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.671+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad40b29f0 con 0x7f7ad4143c80 2024-04-25T09:15:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.671+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad40b2fb0 con 0x7f7ad4143c80 2024-04-25T09:15:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.672+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad404ef20 con 0x7f7ad4143c80 2024-04-25T09:15:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.673+0000 7f7ad0c00640 1 -- 172.21.15.62:0/351500265 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7acc005450 con 0x7f7ad4143c80 2024-04-25T09:15:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.675+0000 7f7ad0c00640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ab406b500 0x7f7ab406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.675+0000 7f7ad0c00640 1 -- 172.21.15.62:0/351500265 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7acc0898e0 con 0x7f7ad4143c80 2024-04-25T09:15:35.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.675+0000 7f7ad9200640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ab406b500 0x7f7ab406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:35.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.676+0000 7f7ad9200640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ab406b500 0x7f7ab406d9c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4069350 tx=0x7f7ac8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:35.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.679+0000 7f7ad0c00640 1 -- 172.21.15.62:0/351500265 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7acc0585f0 con 0x7f7ad4143c80 2024-04-25T09:15:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.926+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ad40cd2a0 con 0x7f7ad4143c80 2024-04-25T09:15:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.928+0000 7f7ad0c00640 1 -- 172.21.15.62:0/351500265 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7acc057ef0 con 0x7f7ad4143c80 2024-04-25T09:15:35.915 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:35.915 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.930+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ab406b500 msgr2=0x7f7ab406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ab406b500 0x7f7ab406d9c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4069350 tx=0x7f7ac8008040 comp rx=0 tx=0).stop 2024-04-25T09:15:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad4143c80 msgr2=0x7f7ad413a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad4143c80 0x7f7ad413a180 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4140fc0 tx=0x7f7acc00d9d0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 shutdown_connections 2024-04-25T09:15:35.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7ab406b500 0x7f7ab406d9c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ad4143c80 0x7f7ad413a180 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7ad4135040 0x7f7ad4143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 --2- 172.21.15.62:0/351500265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7abc004800 0x7f7ad4147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 >> 172.21.15.62:0/351500265 conn(0x7f7ad40b14d0 msgr2=0x7f7ad413c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 shutdown_connections 2024-04-25T09:15:35.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:35.931+0000 7f7ada600640 1 -- 172.21.15.62:0/351500265 wait complete. 2024-04-25T09:15:35.938 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:15:35.938 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:15:36.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.160+0000 7f5587000640 1 Processor -- start 2024-04-25T09:15:36.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.160+0000 7f5587000640 1 -- start start 2024-04-25T09:15:36.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.162+0000 7f5587000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015ee40 0x7f558015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:36.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.162+0000 7f5587000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f558015d660 0x7f55800702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:36.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.162+0000 7f5587000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5580071320 con 0x7f5580070800 2024-04-25T09:15:36.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.162+0000 7f5587000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5580070d60 con 0x7f558015d660 2024-04-25T09:15:36.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.162+0000 7f5587000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5580071040 con 0x7f558015ee40 2024-04-25T09:15:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.162+0000 7f5577400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f558015d660 0x7f55800702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.162+0000 7f5585000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5580070800 0x7f558015f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59906/0 (socket says 172.21.15.62:59906) 2024-04-25T09:15:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.162+0000 7f5585000640 1 -- 172.21.15.62:0/2435223459 learned_addr learned my addr 172.21.15.62:0/2435223459 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.163+0000 7f557fe00640 1 --2- 172.21.15.62:0/2435223459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015ee40 0x7f558015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:36.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.163+0000 7f5577400640 1 -- 172.21.15.62:0/2435223459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015ee40 msgr2=0x7f558015d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.163+0000 7f5577400640 1 --2- 172.21.15.62:0/2435223459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015ee40 0x7f558015d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.163+0000 7f557f400640 1 -- 172.21.15.62:0/2435223459 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142955906 0 0) 0x7f5580071320 con 0x7f5580070800 2024-04-25T09:15:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.163+0000 7f5577400640 1 -- 172.21.15.62:0/2435223459 >> v1:172.21.15.62:6789/0 conn(0x7f5580070800 legacy=0x7f558015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.163+0000 7f5577400640 1 -- 172.21.15.62:0/2435223459 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55800b2820 con 0x7f558015d660 2024-04-25T09:15:36.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.163+0000 7f557fe00640 1 --2- 172.21.15.62:0/2435223459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015ee40 0x7f558015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.164+0000 7f5577400640 1 --2- 172.21.15.62:0/2435223459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015d660 0x7f55800702c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f55680097a0 tx=0x7f556802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea6ffc5912c1895 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.164+0000 7f557f400640 1 -- 172.21.15.62:0/2435223459 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568037070 con 0x7f558015d660 2024-04-25T09:15:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.164+0000 7f557f400640 1 -- 172.21.15.62:0/2435223459 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556802ae50 con 0x7f558015d660 2024-04-25T09:15:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.164+0000 7f557f400640 1 -- 172.21.15.62:0/2435223459 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568032660 con 0x7f558015d660 2024-04-25T09:15:36.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.165+0000 7f5587000640 1 -- 172.21.15.62:0/2435223459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015d660 msgr2=0x7f55800702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.165+0000 7f5587000640 1 --2- 172.21.15.62:0/2435223459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015d660 0x7f55800702c0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f55680097a0 tx=0x7f556802a100 comp rx=0 tx=0).stop 2024-04-25T09:15:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.165+0000 7f5587000640 1 -- 172.21.15.62:0/2435223459 shutdown_connections 2024-04-25T09:15:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.165+0000 7f5587000640 1 --2- 172.21.15.62:0/2435223459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015d660 0x7f55800702c0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.165+0000 7f5587000640 1 --2- 172.21.15.62:0/2435223459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015ee40 0x7f558015d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.165+0000 7f5587000640 1 -- 172.21.15.62:0/2435223459 >> 172.21.15.62:0/2435223459 conn(0x7f55800668f0 msgr2=0x7f5580066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.165+0000 7f5587000640 1 -- 172.21.15.62:0/2435223459 shutdown_connections 2024-04-25T09:15:36.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.165+0000 7f5587000640 1 -- 172.21.15.62:0/2435223459 wait complete. 2024-04-25T09:15:36.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.167+0000 7f5587000640 1 Processor -- start 2024-04-25T09:15:36.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.167+0000 7f5587000640 1 -- start start 2024-04-25T09:15:36.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5587000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580070800 0x7f5580146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5587000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015d660 0x7f55801f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5587000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015ee40 0x7f55800a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5587000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558015f220 con 0x7f558015ee40 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5587000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55801603d0 con 0x7f5580070800 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5587000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558015f3a0 con 0x7f558015d660 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5585000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015ee40 0x7f55800a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5585000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015ee40 0x7f55800a9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41320/0 (socket says 172.21.15.62:41320) 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.168+0000 7f5585000640 1 -- 172.21.15.62:0/140361764 learned_addr learned my addr 172.21.15.62:0/140361764 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:36.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.169+0000 7f5577400640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015d660 0x7f55801f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.169+0000 7f557fe00640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580070800 0x7f5580146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:36.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.169+0000 7f5585000640 1 -- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015d660 msgr2=0x7f55801f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.169+0000 7f5585000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015d660 0x7f55801f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.169+0000 7f5585000640 1 -- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580070800 msgr2=0x7f5580146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.169+0000 7f5585000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580070800 0x7f5580146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.169+0000 7f5585000640 1 -- 172.21.15.62:0/140361764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55800aa8c0 con 0x7f558015ee40 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.169+0000 7f5585000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015ee40 0x7f55800a9e90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f557800c700 tx=0x7f557800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.170+0000 7f557cc00640 1 -- 172.21.15.62:0/140361764 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578018070 con 0x7f558015ee40 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.170+0000 7f557cc00640 1 -- 172.21.15.62:0/140361764 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557800ce10 con 0x7f558015ee40 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.170+0000 7f557cc00640 1 -- 172.21.15.62:0/140361764 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557800a860 con 0x7f558015ee40 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.170+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55801530a0 con 0x7f558015ee40 2024-04-25T09:15:36.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.170+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5580153660 con 0x7f558015ee40 2024-04-25T09:15:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.172+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558004ef20 con 0x7f558015ee40 2024-04-25T09:15:36.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.173+0000 7f557cc00640 1 -- 172.21.15.62:0/140361764 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f557800aa00 con 0x7f558015ee40 2024-04-25T09:15:36.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.173+0000 7f557cc00640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f555c06b500 0x7f555c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:36.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.174+0000 7f557cc00640 1 -- 172.21.15.62:0/140361764 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5578089750 con 0x7f558015ee40 2024-04-25T09:15:36.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.174+0000 7f557fe00640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f555c06b500 0x7f555c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:36.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.176+0000 7f557fe00640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f555c06b500 0x7f555c06d9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f558006d5c0 tx=0x7f5570008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:36.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.178+0000 7f557cc00640 1 -- 172.21.15.62:0/140361764 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5578057ea0 con 0x7f558015ee40 2024-04-25T09:15:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.457+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f558006b2c0 con 0x7f558015ee40 2024-04-25T09:15:36.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.458+0000 7f557cc00640 1 -- 172.21.15.62:0/140361764 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f557805bf40 con 0x7f558015ee40 2024-04-25T09:15:36.445 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:36.445 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":763,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580998,"bytes_used":2612928512,"bytes_avail":770481184768,"bytes_total":773094113280,"write_bytes_sec":13775572,"read_op_per_sec":0,"write_op_per_sec":14},"fsmap":{"epoch":78,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T09:15:32.169417+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:15:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.460+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f555c06b500 msgr2=0x7f555c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.460+0000 7f5587000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f555c06b500 0x7f555c06d9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f558006d5c0 tx=0x7f5570008040 comp rx=0 tx=0).stop 2024-04-25T09:15:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.460+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015ee40 msgr2=0x7f55800a9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.460+0000 7f5587000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015ee40 0x7f55800a9e90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f557800c700 tx=0x7f557800cbc0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.461+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 shutdown_connections 2024-04-25T09:15:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.461+0000 7f5587000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f555c06b500 0x7f555c06d9c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.461+0000 7f5587000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f558015ee40 0x7f55800a9e90 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.461+0000 7f5587000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f558015d660 0x7f55801f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.461+0000 7f5587000640 1 --2- 172.21.15.62:0/140361764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5580070800 0x7f5580146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.461+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 >> 172.21.15.62:0/140361764 conn(0x7f55800668f0 msgr2=0x7f558015b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.461+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 shutdown_connections 2024-04-25T09:15:36.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:36.461+0000 7f5587000640 1 -- 172.21.15.62:0/140361764 wait complete. 2024-04-25T09:15:36.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:15:37.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.700+0000 7fa484000640 1 Processor -- start 2024-04-25T09:15:37.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.701+0000 7fa484000640 1 -- start start 2024-04-25T09:15:37.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.702+0000 7fa484000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa47c0b2e30 0x7fa47c0d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.702+0000 7fa484000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa47c0d3560 0x7fa47c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.702+0000 7fa484000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0d1fe0 con 0x7fa47c15dc60 2024-04-25T09:15:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.702+0000 7fa484000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0d2280 con 0x7fa47c0b2e30 2024-04-25T09:15:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.702+0000 7fa484000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0d2560 con 0x7fa47c0d3560 2024-04-25T09:15:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.702+0000 7fa480c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa47c0b2e30 0x7fa47c0d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.703+0000 7fa480c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa47c0b2e30 0x7fa47c0d3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41326/0 (socket says 172.21.15.62:41326) 2024-04-25T09:15:37.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.703+0000 7fa480c00640 1 -- 172.21.15.62:0/878636752 learned_addr learned my addr 172.21.15.62:0/878636752 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:37.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.703+0000 7fa482000640 1 --2- 172.21.15.62:0/878636752 >> v2:172.21.15.121:3300/0 conn(0x7fa47c0d3560 0x7fa47c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.703+0000 7fa480c00640 1 -- 172.21.15.62:0/878636752 >> v2:172.21.15.121:3300/0 conn(0x7fa47c0d3560 msgr2=0x7fa47c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.703+0000 7fa480c00640 1 --2- 172.21.15.62:0/878636752 >> v2:172.21.15.121:3300/0 conn(0x7fa47c0d3560 0x7fa47c0ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.703+0000 7fa480c00640 1 -- 172.21.15.62:0/878636752 >> v1:172.21.15.121:6789/0 conn(0x7fa47c15dc60 legacy=0x7fa47c0b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.703+0000 7fa480c00640 1 -- 172.21.15.62:0/878636752 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0d2840 con 0x7fa47c0b2e30 2024-04-25T09:15:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.704+0000 7fa480c00640 1 --2- 172.21.15.62:0/878636752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c0d3020 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0cfd80 tx=0x7fa46c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d948aae8fea10c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.705+0000 7fa47be00640 1 -- 172.21.15.62:0/878636752 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c036070 con 0x7fa47c0b2e30 2024-04-25T09:15:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.705+0000 7fa47be00640 1 -- 172.21.15.62:0/878636752 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c031600 con 0x7fa47c0b2e30 2024-04-25T09:15:37.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.705+0000 7fa47be00640 1 -- 172.21.15.62:0/878636752 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0319d0 con 0x7fa47c0b2e30 2024-04-25T09:15:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.705+0000 7fa484000640 1 -- 172.21.15.62:0/878636752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 msgr2=0x7fa47c0d3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:37.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.705+0000 7fa484000640 1 --2- 172.21.15.62:0/878636752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c0d3020 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0cfd80 tx=0x7fa46c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.706+0000 7fa484000640 1 -- 172.21.15.62:0/878636752 shutdown_connections 2024-04-25T09:15:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.706+0000 7fa484000640 1 --2- 172.21.15.62:0/878636752 >> v2:172.21.15.121:3300/0 conn(0x7fa47c0d3560 0x7fa47c0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.706+0000 7fa484000640 1 --2- 172.21.15.62:0/878636752 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c0d3020 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.706+0000 7fa484000640 1 -- 172.21.15.62:0/878636752 >> 172.21.15.62:0/878636752 conn(0x7fa47c06b000 msgr2=0x7fa47c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.706+0000 7fa484000640 1 -- 172.21.15.62:0/878636752 shutdown_connections 2024-04-25T09:15:37.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.706+0000 7fa484000640 1 -- 172.21.15.62:0/878636752 wait complete. 2024-04-25T09:15:37.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.707+0000 7fa484000640 1 Processor -- start 2024-04-25T09:15:37.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.707+0000 7fa484000640 1 -- start start 2024-04-25T09:15:37.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.708+0000 7fa484000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.708+0000 7fa484000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47c0d3560 0x7fa47c1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.708+0000 7fa484000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47c15dc60 0x7fa47c0c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.708+0000 7fa484000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c15e1c0 con 0x7fa47c0b2e30 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.708+0000 7fa484000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c15e040 con 0x7fa47c0d3560 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.708+0000 7fa484000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0cedd0 con 0x7fa47c15dc60 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c1f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41328/0 (socket says 172.21.15.62:41328) 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 -- 172.21.15.62:0/3861433839 learned_addr learned my addr 172.21.15.62:0/3861433839 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 -- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47c15dc60 msgr2=0x7fa47c0c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47c15dc60 0x7fa47c0c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 -- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47c0d3560 msgr2=0x7fa47c1f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47c0d3560 0x7fa47c1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 -- 172.21.15.62:0/3861433839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c1f4b10 con 0x7fa47c0b2e30 2024-04-25T09:15:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.709+0000 7fa481600640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c1f56c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa47000b3f0 tx=0x7fa47000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.710+0000 7fa479600640 1 -- 172.21.15.62:0/3861433839 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470015020 con 0x7fa47c0b2e30 2024-04-25T09:15:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.710+0000 7fa479600640 1 -- 172.21.15.62:0/3861433839 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa470011830 con 0x7fa47c0b2e30 2024-04-25T09:15:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.710+0000 7fa479600640 1 -- 172.21.15.62:0/3861433839 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470011c20 con 0x7fa47c0b2e30 2024-04-25T09:15:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.710+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c1f4df0 con 0x7fa47c0b2e30 2024-04-25T09:15:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.710+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c1f53b0 con 0x7fa47c0b2e30 2024-04-25T09:15:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.711+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c160390 con 0x7fa47c0b2e30 2024-04-25T09:15:37.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.712+0000 7fa479600640 1 -- 172.21.15.62:0/3861433839 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa470018020 con 0x7fa47c0b2e30 2024-04-25T09:15:37.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.713+0000 7fa479600640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa46006b4d0 0x7fa46006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:37.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.713+0000 7fa479600640 1 -- 172.21.15.62:0/3861433839 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa470089e90 con 0x7fa47c0b2e30 2024-04-25T09:15:37.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.714+0000 7fa480c00640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa46006b4d0 0x7fa46006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:37.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.715+0000 7fa480c00640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa46006b4d0 0x7fa46006d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa46c033000 tx=0x7fa46c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:37.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.718+0000 7fa479600640 1 -- 172.21.15.62:0/3861433839 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa47001f030 con 0x7fa47c0b2e30 2024-04-25T09:15:37.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.958+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa47c0729c0 con 0x7fa47c0b2e30 2024-04-25T09:15:37.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.959+0000 7fa479600640 1 -- 172.21.15.62:0/3861433839 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fa47008b090 con 0x7fa47c0b2e30 2024-04-25T09:15:37.946 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:37.946 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:37.946 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.961+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa46006b4d0 msgr2=0x7fa46006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.961+0000 7fa484000640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa46006b4d0 0x7fa46006d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa46c033000 tx=0x7fa46c02e000 comp rx=0 tx=0).stop 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.961+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 msgr2=0x7fa47c1f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.961+0000 7fa484000640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c1f56c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa47000b3f0 tx=0x7fa47000b8b0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.962+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 shutdown_connections 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.962+0000 7fa484000640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa46006b4d0 0x7fa46006d990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.962+0000 7fa484000640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa47c15dc60 0x7fa47c0c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.962+0000 7fa484000640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa47c0d3560 0x7fa47c1f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.962+0000 7fa484000640 1 --2- 172.21.15.62:0/3861433839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa47c0b2e30 0x7fa47c1f56c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.962+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 >> 172.21.15.62:0/3861433839 conn(0x7fa47c06b000 msgr2=0x7fa47c0b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:37.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.962+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 shutdown_connections 2024-04-25T09:15:37.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:37.962+0000 7fa484000640 1 -- 172.21.15.62:0/3861433839 wait complete. 2024-04-25T09:15:37.970 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:15:37.971 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:15:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.198+0000 7fa79b000640 1 Processor -- start 2024-04-25T09:15:38.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.198+0000 7fa79b000640 1 -- start start 2024-04-25T09:15:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa79b000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa79406dbf0 0x7fa7941630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa79b000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7940b2f70 con 0x7fa7940b4410 2024-04-25T09:15:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa79b000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7940b4a80 con 0x7fa7941621d0 2024-04-25T09:15:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa79b000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7940b2920 con 0x7fa79406dbf0 2024-04-25T09:15:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa793400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa7940b4410 0x7fa79406d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59914/0 (socket says 172.21.15.62:59914) 2024-04-25T09:15:38.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa793400640 1 -- 172.21.15.62:0/373980385 learned_addr learned my addr 172.21.15.62:0/373980385 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa799000640 1 --2- 172.21.15.62:0/373980385 >> v2:172.21.15.121:3300/0 conn(0x7fa79406dbf0 0x7fa7941630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015819540 0 0) 0x7fa7940b2f70 con 0x7fa7940b4410 2024-04-25T09:15:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.200+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa778003520 con 0x7fa7940b4410 2024-04-25T09:15:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1116268649 0 0) 0x7fa778003520 con 0x7fa7940b4410 2024-04-25T09:15:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 >> v2:172.21.15.121:3300/0 conn(0x7fa79406dbf0 msgr2=0x7fa7941630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 --2- 172.21.15.62:0/373980385 >> v2:172.21.15.121:3300/0 conn(0x7fa79406dbf0 0x7fa7941630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 >> v1:172.21.15.121:6789/0 conn(0x7fa7941621d0 legacy=0x7fa7940cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940b2c00 con 0x7fa7940b4410 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fa77c001670 con 0x7fa7940b4410 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 >> v1:172.21.15.62:6789/0 conn(0x7fa7940b4410 legacy=0x7fa79406d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7780039b0 0x7fa778003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa778004800 0x7fa778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7780051c0 0x7fa77800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7940b2920 con 0x7fa7780039b0 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77800c160 con 0x7fa7780051c0 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77800c360 con 0x7fa778004800 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.201+0000 7fa793e00640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7780039b0 0x7fa778003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa793400640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa778004800 0x7fa778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa799000640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7780051c0 0x7fa77800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:38.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa793e00640 1 -- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa778004800 msgr2=0x7fa778004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa793e00640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa778004800 0x7fa778004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa793e00640 1 -- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7780051c0 msgr2=0x7fa77800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa793e00640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7780051c0 0x7fa77800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa793e00640 1 -- 172.21.15.62:0/373980385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778003520 con 0x7fa7780039b0 2024-04-25T09:15:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa799000640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7780051c0 0x7fa77800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:38.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.202+0000 7fa799000640 1 -- 172.21.15.62:0/373980385 reap_dead start 2024-04-25T09:15:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.203+0000 7fa793e00640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7780039b0 0x7fa778003dd0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa784001750 tx=0x7fa778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fa23257c4cbba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.203+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784001370 con 0x7fa7780039b0 2024-04-25T09:15:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.203+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7840052d0 con 0x7fa7780039b0 2024-04-25T09:15:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.203+0000 7fa792a00640 1 -- 172.21.15.62:0/373980385 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78402abc0 con 0x7fa7780039b0 2024-04-25T09:15:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.204+0000 7fa79b000640 1 -- 172.21.15.62:0/373980385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7780039b0 msgr2=0x7fa778003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.204+0000 7fa79b000640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7780039b0 0x7fa778003dd0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa784001750 tx=0x7fa778005830 comp rx=0 tx=0).stop 2024-04-25T09:15:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.204+0000 7fa79b000640 1 -- 172.21.15.62:0/373980385 shutdown_connections 2024-04-25T09:15:38.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.204+0000 7fa79b000640 1 --2- 172.21.15.62:0/373980385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7780039b0 0x7fa778003dd0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.204+0000 7fa79b000640 1 -- 172.21.15.62:0/373980385 >> 172.21.15.62:0/373980385 conn(0x7fa7940668f0 msgr2=0x7fa794066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.204+0000 7fa79b000640 1 -- 172.21.15.62:0/373980385 shutdown_connections 2024-04-25T09:15:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.204+0000 7fa79b000640 1 -- 172.21.15.62:0/373980385 wait complete. 2024-04-25T09:15:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.206+0000 7fa79b000640 1 Processor -- start 2024-04-25T09:15:38.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.206+0000 7fa79b000640 1 -- start start 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa79b000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7780039b0 0x7fa79414caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa79b000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa79415b010 0x7fa7940a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa79b000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7941604c0 0x7fa794160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa79b000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa794163450 con 0x7fa7941604c0 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa79b000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7941632d0 con 0x7fa79415b010 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa79b000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7941fb790 con 0x7fa7780039b0 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa799000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7941604c0 0x7fa794160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa799000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7941604c0 0x7fa794160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41350/0 (socket says 172.21.15.62:41350) 2024-04-25T09:15:38.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.207+0000 7fa799000640 1 -- 172.21.15.62:0/1651233773 learned_addr learned my addr 172.21.15.62:0/1651233773 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa793e00640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7780039b0 0x7fa79414caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa793400640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa79415b010 0x7fa7940a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa799000640 1 -- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7780039b0 msgr2=0x7fa79414caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa799000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7780039b0 0x7fa79414caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa799000640 1 -- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa79415b010 msgr2=0x7fa7940a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa799000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa79415b010 0x7fa7940a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa799000640 1 -- 172.21.15.62:0/1651233773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79414cfe0 con 0x7fa7941604c0 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa793e00640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7780039b0 0x7fa79414caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:38.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa799000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7941604c0 0x7fa794160940 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa7940ceda0 tx=0x7fa78c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa78be00640 1 -- 172.21.15.62:0/1651233773 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c016420 con 0x7fa7941604c0 2024-04-25T09:15:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa78be00640 1 -- 172.21.15.62:0/1651233773 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c01b070 con 0x7fa7941604c0 2024-04-25T09:15:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa78be00640 1 -- 172.21.15.62:0/1651233773 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0167c0 con 0x7fa7941604c0 2024-04-25T09:15:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.208+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7941fba30 con 0x7fa7941604c0 2024-04-25T09:15:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.209+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7941fbff0 con 0x7fa7941604c0 2024-04-25T09:15:38.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.210+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7940b3480 con 0x7fa7941604c0 2024-04-25T09:15:38.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.211+0000 7fa78be00640 1 -- 172.21.15.62:0/1651233773 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa78c016960 con 0x7fa7941604c0 2024-04-25T09:15:38.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.212+0000 7fa78be00640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa77006b500 0x7fa77006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:38.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.212+0000 7fa78be00640 1 -- 172.21.15.62:0/1651233773 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa78c08b210 con 0x7fa7941604c0 2024-04-25T09:15:38.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.212+0000 7fa793e00640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa77006b500 0x7fa77006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:38.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.214+0000 7fa793e00640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa77006b500 0x7fa77006d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa784032000 tx=0x7fa78402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:38.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.216+0000 7fa78be00640 1 -- 172.21.15.62:0/1651233773 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7940b3480 con 0x7fa7941604c0 2024-04-25T09:15:38.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.489+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa79404ef20 con 0x7fa7941604c0 2024-04-25T09:15:38.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.490+0000 7fa78be00640 1 -- 172.21.15.62:0/1651233773 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fa78c025020 con 0x7fa7941604c0 2024-04-25T09:15:38.478 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:38.478 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":765,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076634250,"bytes_used":2613035008,"bytes_avail":770481078272,"bytes_total":773094113280,"write_bytes_sec":13780584,"read_op_per_sec":0,"write_op_per_sec":15},"fsmap":{"epoch":78,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T09:15:32.169417+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:15:38.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.493+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa77006b500 msgr2=0x7fa77006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:38.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa77006b500 0x7fa77006d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa784032000 tx=0x7fa78402e000 comp rx=0 tx=0).stop 2024-04-25T09:15:38.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7941604c0 msgr2=0x7fa794160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:38.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7941604c0 0x7fa794160940 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa7940ceda0 tx=0x7fa78c00f520 comp rx=0 tx=0).stop 2024-04-25T09:15:38.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 shutdown_connections 2024-04-25T09:15:38.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa77006b500 0x7fa77006d9c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7941604c0 0x7fa794160940 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa79415b010 0x7fa7940a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 --2- 172.21.15.62:0/1651233773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa7780039b0 0x7fa79414caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:38.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 >> 172.21.15.62:0/1651233773 conn(0x7fa7940668f0 msgr2=0x7fa794164da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:38.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 shutdown_connections 2024-04-25T09:15:38.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:38.494+0000 7fa79b000640 1 -- 172.21.15.62:0/1651233773 wait complete. 2024-04-25T09:15:38.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:15:39.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:39.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.732+0000 7f6722400640 1 Processor -- start 2024-04-25T09:15:39.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.733+0000 7f6722400640 1 -- start start 2024-04-25T09:15:39.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.734+0000 7f6722400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c15bd10 0x7f671c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:39.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.734+0000 7f6722400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671c06ca70 con 0x7f671c0aeed0 2024-04-25T09:15:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.734+0000 7f6722400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f671c15c0f0 con 0x7f671c0cccf0 2024-04-25T09:15:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.734+0000 7f6722400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c06d660 con 0x7f671c15bd10 2024-04-25T09:15:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.735+0000 7f671be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f671c0aeed0 0x7f671c0cd6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59928/0 (socket says 172.21.15.62:59928) 2024-04-25T09:15:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.735+0000 7f671be00640 1 -- 172.21.15.62:0/1766428591 learned_addr learned my addr 172.21.15.62:0/1766428591 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:39.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.735+0000 7f671b400640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c15bd10 0x7f671c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:39.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.735+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107205761 0 0) 0x7f671c06ca70 con 0x7f671c0aeed0 2024-04-25T09:15:39.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.735+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6700003520 con 0x7f671c0aeed0 2024-04-25T09:15:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3223122242 0 0) 0x7f6700003520 con 0x7f671c0aeed0 2024-04-25T09:15:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c15bd10 msgr2=0x7f671c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c15bd10 0x7f671c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 >> v1:172.21.15.121:6789/0 conn(0x7f671c0cccf0 legacy=0x7f671c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c06cd30 con 0x7f671c0aeed0 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f6714001670 con 0x7f671c0aeed0 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 >> v1:172.21.15.62:6789/0 conn(0x7f671c0aeed0 legacy=0x7f671c0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c15bd10 0x7f67000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700003fb0 0x7f6700004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700004d40 0x7f67000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c06d660 con 0x7f6700004d40 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670000bb00 con 0x7f6700003fb0 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670000bd00 con 0x7f671c15bd10 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.736+0000 7f671be00640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700004d40 0x7f67000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.737+0000 7f671b400640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c15bd10 0x7f67000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.737+0000 7f671aa00640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700003fb0 0x7f6700004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.737+0000 7f671be00640 1 -- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c15bd10 msgr2=0x7f67000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.737+0000 7f671be00640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c15bd10 0x7f67000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.737+0000 7f671be00640 1 -- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700003fb0 msgr2=0x7f6700004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.737+0000 7f671be00640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700003fb0 0x7f6700004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.737+0000 7f671be00640 1 -- 172.21.15.62:0/1766428591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700003520 con 0x7f6700004d40 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.738+0000 7f671be00640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700004d40 0x7f67000051c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f6714007fd0 tx=0x7f6700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5641385e51933c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.738+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714007c20 con 0x7f6700004d40 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.738+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714005af0 con 0x7f6700004d40 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.738+0000 7f671a000640 1 -- 172.21.15.62:0/1766428591 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714007070 con 0x7f6700004d40 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.739+0000 7f6722400640 1 -- 172.21.15.62:0/1766428591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700004d40 msgr2=0x7f67000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.739+0000 7f6722400640 1 --2- 172.21.15.62:0/1766428591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6700004d40 0x7f67000051c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f6714007fd0 tx=0x7f6700005830 comp rx=0 tx=0).stop 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.739+0000 7f671be00640 1 -- 172.21.15.62:0/1766428591 reap_dead start 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.739+0000 7f6722400640 1 -- 172.21.15.62:0/1766428591 shutdown_connections 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.739+0000 7f6722400640 1 -- 172.21.15.62:0/1766428591 >> 172.21.15.62:0/1766428591 conn(0x7f671c0668f0 msgr2=0x7f671c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:39.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.739+0000 7f6722400640 1 -- 172.21.15.62:0/1766428591 shutdown_connections 2024-04-25T09:15:39.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.739+0000 7f6722400640 1 -- 172.21.15.62:0/1766428591 wait complete. 2024-04-25T09:15:39.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.741+0000 7f6722400640 1 Processor -- start 2024-04-25T09:15:39.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.741+0000 7f6722400640 1 -- start start 2024-04-25T09:15:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f6722400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700004d40 0x7f671c0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f6722400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f671c1e4df0 0x7f671c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f6722400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c159b90 0x7f671c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f6722400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c06daf0 con 0x7f671c1e4df0 2024-04-25T09:15:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f6722400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c06d2f0 con 0x7f6700004d40 2024-04-25T09:15:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f6722400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c1e5250 con 0x7f671c159b90 2024-04-25T09:15:39.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f671aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f671c1e4df0 0x7f671c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f671aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f671c1e4df0 0x7f671c0b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41360/0 (socket says 172.21.15.62:41360) 2024-04-25T09:15:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f671aa00640 1 -- 172.21.15.62:0/2273324174 learned_addr learned my addr 172.21.15.62:0/2273324174 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:39.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f671b400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700004d40 0x7f671c0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.742+0000 7f671be00640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c159b90 0x7f671c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f671aa00640 1 -- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c159b90 msgr2=0x7f671c15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f671aa00640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c159b90 0x7f671c15a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f671aa00640 1 -- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700004d40 msgr2=0x7f671c0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f671aa00640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700004d40 0x7f671c0af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f671aa00640 1 -- 172.21.15.62:0/2273324174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c1e5520 con 0x7f671c1e4df0 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f671be00640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c159b90 0x7f671c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f671aa00640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f671c1e4df0 0x7f671c0b1d40 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f670800e500 tx=0x7f670800e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:39.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f6713400640 1 -- 172.21.15.62:0/2273324174 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67080053a0 con 0x7f671c1e4df0 2024-04-25T09:15:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f6713400640 1 -- 172.21.15.62:0/2273324174 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6708018070 con 0x7f671c1e4df0 2024-04-25T09:15:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f6713400640 1 -- 172.21.15.62:0/2273324174 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6708003810 con 0x7f671c1e4df0 2024-04-25T09:15:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c1e5800 con 0x7f671c1e4df0 2024-04-25T09:15:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.743+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c158fd0 con 0x7f671c1e4df0 2024-04-25T09:15:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.744+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c0b2280 con 0x7f671c1e4df0 2024-04-25T09:15:39.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.746+0000 7f6713400640 1 -- 172.21.15.62:0/2273324174 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67080039b0 con 0x7f671c1e4df0 2024-04-25T09:15:39.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.746+0000 7f6713400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66fc06b500 0x7f66fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.747+0000 7f6713400640 1 -- 172.21.15.62:0/2273324174 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6708089f20 con 0x7f671c1e4df0 2024-04-25T09:15:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.747+0000 7f671b400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66fc06b500 0x7f66fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.748+0000 7f671b400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66fc06b500 0x7f66fc06d9c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f670c007520 tx=0x7f670c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:39.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.751+0000 7f6713400640 1 -- 172.21.15.62:0/2273324174 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6708058cd0 con 0x7f671c1e4df0 2024-04-25T09:15:39.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.992+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671c072c70 con 0x7f671c1e4df0 2024-04-25T09:15:39.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.993+0000 7f6713400640 1 -- 172.21.15.62:0/2273324174 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f67080585d0 con 0x7f671c1e4df0 2024-04-25T09:15:39.980 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:39.980 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:39.980 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.995+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66fc06b500 msgr2=0x7f66fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.995+0000 7f6722400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66fc06b500 0x7f66fc06d9c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f670c007520 tx=0x7f670c00b040 comp rx=0 tx=0).stop 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.995+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f671c1e4df0 msgr2=0x7f671c0b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.995+0000 7f6722400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f671c1e4df0 0x7f671c0b1d40 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f670800e500 tx=0x7f670800e9c0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.995+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 shutdown_connections 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.996+0000 7f6722400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66fc06b500 0x7f66fc06d9c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.996+0000 7f6722400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f671c1e4df0 0x7f671c0b1d40 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.996+0000 7f6722400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f671c159b90 0x7f671c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.996+0000 7f6722400640 1 --2- 172.21.15.62:0/2273324174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6700004d40 0x7f671c0af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:39.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.996+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 >> 172.21.15.62:0/2273324174 conn(0x7f671c0668f0 msgr2=0x7f671c071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.996+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 shutdown_connections 2024-04-25T09:15:39.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:39.996+0000 7f6722400640 1 -- 172.21.15.62:0/2273324174 wait complete. 2024-04-25T09:15:40.003 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:15:40.004 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:15:40.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.196+0000 7fe77dc00640 1 Processor -- start 2024-04-25T09:15:40.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.196+0000 7fe77dc00640 1 -- start start 2024-04-25T09:15:40.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.198+0000 7fe77dc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 0x7fe77815d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:40.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.198+0000 7fe77dc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe77815d660 0x7fe7780702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:40.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.198+0000 7fe77dc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe778070800 0x7fe77815f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:40.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.198+0000 7fe77dc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe778071320 con 0x7fe77815d660 2024-04-25T09:15:40.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.198+0000 7fe77dc00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe778070d60 con 0x7fe778070800 2024-04-25T09:15:40.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.198+0000 7fe77dc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe778071040 con 0x7fe77815ee40 2024-04-25T09:15:40.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.198+0000 7fe776000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe77815d660 0x7fe7780702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:40.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe776000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe77815d660 0x7fe7780702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41374/0 (socket says 172.21.15.62:41374) 2024-04-25T09:15:40.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe776000640 1 -- 172.21.15.62:0/3800214178 learned_addr learned my addr 172.21.15.62:0/3800214178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:40.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe777400640 1 --2- 172.21.15.62:0/3800214178 >> v2:172.21.15.121:3300/0 conn(0x7fe778070800 0x7fe77815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:40.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe776a00640 1 --2- 172.21.15.62:0/3800214178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 0x7fe77815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:40.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe776000640 1 -- 172.21.15.62:0/3800214178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 msgr2=0x7fe77815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:40.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe776000640 1 --2- 172.21.15.62:0/3800214178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 0x7fe77815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe776000640 1 -- 172.21.15.62:0/3800214178 >> v2:172.21.15.121:3300/0 conn(0x7fe778070800 msgr2=0x7fe77815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:40.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe776000640 1 --2- 172.21.15.62:0/3800214178 >> v2:172.21.15.121:3300/0 conn(0x7fe778070800 0x7fe77815f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.199+0000 7fe776000640 1 -- 172.21.15.62:0/3800214178 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780b2820 con 0x7fe77815d660 2024-04-25T09:15:40.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.200+0000 7fe776000640 1 --2- 172.21.15.62:0/3800214178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe77815d660 0x7fe7780702c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fe7780678a0 tx=0x7fe76002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c0c1e9e2f698d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:40.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe775600640 1 -- 172.21.15.62:0/3800214178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760037070 con 0x7fe77815d660 2024-04-25T09:15:40.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe775600640 1 -- 172.21.15.62:0/3800214178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760032440 con 0x7fe77815d660 2024-04-25T09:15:40.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe775600640 1 -- 172.21.15.62:0/3800214178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760032810 con 0x7fe77815d660 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe77dc00640 1 -- 172.21.15.62:0/3800214178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe77815d660 msgr2=0x7fe7780702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe77dc00640 1 --2- 172.21.15.62:0/3800214178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe77815d660 0x7fe7780702c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fe7780678a0 tx=0x7fe76002a140 comp rx=0 tx=0).stop 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe77dc00640 1 -- 172.21.15.62:0/3800214178 shutdown_connections 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe77dc00640 1 --2- 172.21.15.62:0/3800214178 >> v2:172.21.15.121:3300/0 conn(0x7fe778070800 0x7fe77815f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe77dc00640 1 --2- 172.21.15.62:0/3800214178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe77815d660 0x7fe7780702c0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe77dc00640 1 --2- 172.21.15.62:0/3800214178 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 0x7fe77815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.201+0000 7fe77dc00640 1 -- 172.21.15.62:0/3800214178 >> 172.21.15.62:0/3800214178 conn(0x7fe7780668f0 msgr2=0x7fe778066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.202+0000 7fe77dc00640 1 -- 172.21.15.62:0/3800214178 shutdown_connections 2024-04-25T09:15:40.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.202+0000 7fe77dc00640 1 -- 172.21.15.62:0/3800214178 wait complete. 2024-04-25T09:15:40.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.203+0000 7fe77dc00640 1 Processor -- start 2024-04-25T09:15:40.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.203+0000 7fe77dc00640 1 -- start start 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.204+0000 7fe77dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778070800 0x7fe778149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.204+0000 7fe77dc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe77815d660 0x7fe7781f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.204+0000 7fe77dc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 0x7fe77815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.204+0000 7fe77dc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77815f220 con 0x7fe778070800 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.204+0000 7fe77dc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781603d0 con 0x7fe77815d660 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.204+0000 7fe77dc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77815f3a0 con 0x7fe77815ee40 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778070800 0x7fe778149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778070800 0x7fe778149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41390/0 (socket says 172.21.15.62:41390) 2024-04-25T09:15:40.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 -- 172.21.15.62:0/2766255714 learned_addr learned my addr 172.21.15.62:0/2766255714 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:40.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 -- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 msgr2=0x7fe77815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:40.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 0x7fe77815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 -- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe77815d660 msgr2=0x7fe7781f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe77815d660 0x7fe7781f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 -- 172.21.15.62:0/2766255714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7781f7790 con 0x7fe778070800 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.205+0000 7fe776a00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778070800 0x7fe778149250 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fe768007ef0 tx=0x7fe76800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.206+0000 7fe76ea00640 1 -- 172.21.15.62:0/2766255714 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768014620 con 0x7fe778070800 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.206+0000 7fe76ea00640 1 -- 172.21.15.62:0/2766255714 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe768019070 con 0x7fe778070800 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.206+0000 7fe76ea00640 1 -- 172.21.15.62:0/2766255714 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680149e0 con 0x7fe778070800 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.206+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7781f7a70 con 0x7fe778070800 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.206+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe778149790 con 0x7fe778070800 2024-04-25T09:15:40.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.207+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7780c4680 con 0x7fe778070800 2024-04-25T09:15:40.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.208+0000 7fe76ea00640 1 -- 172.21.15.62:0/2766255714 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe768014b80 con 0x7fe778070800 2024-04-25T09:15:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.209+0000 7fe76ea00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe75406b4d0 0x7fe75406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.209+0000 7fe76ea00640 1 -- 172.21.15.62:0/2766255714 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe76808ac80 con 0x7fe778070800 2024-04-25T09:15:40.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.210+0000 7fe776000640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe75406b4d0 0x7fe75406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:40.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.211+0000 7fe776000640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe75406b4d0 0x7fe75406d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe760034000 tx=0x7fe76002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:40.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.213+0000 7fe76ea00640 1 -- 172.21.15.62:0/2766255714 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe768059a20 con 0x7fe778070800 2024-04-25T09:15:40.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.486+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe77804ef20 con 0x7fe778070800 2024-04-25T09:15:40.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.487+0000 7fe76ea00640 1 -- 172.21.15.62:0/2766255714 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1571 (secure 0 0 0) 0x7fe768059320 con 0x7fe778070800 2024-04-25T09:15:40.474 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:40.474 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":767,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":125,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076634250,"bytes_used":2613035008,"bytes_avail":770481078272,"bytes_total":773094113280,"write_bytes_sec":32155,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":78,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T09:15:38.172252+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:15:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.489+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe75406b4d0 msgr2=0x7fe75406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.489+0000 7fe77dc00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe75406b4d0 0x7fe75406d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe760034000 tx=0x7fe76002c000 comp rx=0 tx=0).stop 2024-04-25T09:15:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.489+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778070800 msgr2=0x7fe778149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.489+0000 7fe77dc00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778070800 0x7fe778149250 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fe768007ef0 tx=0x7fe76800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.490+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 shutdown_connections 2024-04-25T09:15:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.490+0000 7fe77dc00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe75406b4d0 0x7fe75406d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.490+0000 7fe77dc00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe77815ee40 0x7fe77815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.490+0000 7fe77dc00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe77815d660 0x7fe7781f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.490+0000 7fe77dc00640 1 --2- 172.21.15.62:0/2766255714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe778070800 0x7fe778149250 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.490+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 >> 172.21.15.62:0/2766255714 conn(0x7fe7780668f0 msgr2=0x7fe7780b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.490+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 shutdown_connections 2024-04-25T09:15:40.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:40.490+0000 7fe77dc00640 1 -- 172.21.15.62:0/2766255714 wait complete. 2024-04-25T09:15:40.497 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:15:40.497 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:15:40.498 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:15:40.599 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:15:40.717 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:15:40.717 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:15:40.718 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:40.778 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:15:40.778 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:40.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:15:41.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.056+0000 7fbcd9e00640 1 Processor -- start 2024-04-25T09:15:41.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.056+0000 7fbcd9e00640 1 -- start start 2024-04-25T09:15:41.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.058+0000 7fbcd9e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbcd4161f00 0x7fbcd41622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.058+0000 7fbcd9e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 0x7fbcd40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.058+0000 7fbcd9e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd40b2a20 con 0x7fbcd40cc9e0 2024-04-25T09:15:41.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.058+0000 7fbcd9e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd40b27d0 con 0x7fbcd4161f00 2024-04-25T09:15:41.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.058+0000 7fbcd9e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40b2d00 con 0x7fbcd406d900 2024-04-25T09:15:41.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.058+0000 7fbcd8a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbcd4161f00 0x7fbcd41622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.058+0000 7fbcd8a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbcd4161f00 0x7fbcd41622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41392/0 (socket says 172.21.15.62:41392) 2024-04-25T09:15:41.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.058+0000 7fbcd8a00640 1 -- 172.21.15.62:0/1041762925 learned_addr learned my addr 172.21.15.62:0/1041762925 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:41.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.059+0000 7fbcd9400640 1 --2- 172.21.15.62:0/1041762925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 0x7fbcd40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.059+0000 7fbcd8a00640 1 -- 172.21.15.62:0/1041762925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 msgr2=0x7fbcd40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:41.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.059+0000 7fbcd8a00640 1 --2- 172.21.15.62:0/1041762925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 0x7fbcd40cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.059+0000 7fbcd8a00640 1 -- 172.21.15.62:0/1041762925 >> v1:172.21.15.121:6789/0 conn(0x7fbcd40cc9e0 legacy=0x7fbcd406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.059+0000 7fbcd8a00640 1 -- 172.21.15.62:0/1041762925 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd40b2fe0 con 0x7fbcd4161f00 2024-04-25T09:15:41.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.061+0000 7fbcd8a00640 1 --2- 172.21.15.62:0/1041762925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41622e0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009870 tx=0x7fbcbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc34c29d66fdd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.061+0000 7fbcd3400640 1 -- 172.21.15.62:0/1041762925 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc037070 con 0x7fbcd4161f00 2024-04-25T09:15:41.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.061+0000 7fbcd3400640 1 -- 172.21.15.62:0/1041762925 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc032440 con 0x7fbcd4161f00 2024-04-25T09:15:41.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.061+0000 7fbcd3400640 1 -- 172.21.15.62:0/1041762925 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc032810 con 0x7fbcd4161f00 2024-04-25T09:15:41.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.061+0000 7fbcd9e00640 1 -- 172.21.15.62:0/1041762925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 msgr2=0x7fbcd41622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.061+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/1041762925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41622e0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009870 tx=0x7fbcbc02a270 comp rx=0 tx=0).stop 2024-04-25T09:15:41.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.062+0000 7fbcd9e00640 1 -- 172.21.15.62:0/1041762925 shutdown_connections 2024-04-25T09:15:41.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.062+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/1041762925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 0x7fbcd40cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.062+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/1041762925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41622e0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.062+0000 7fbcd9e00640 1 -- 172.21.15.62:0/1041762925 >> 172.21.15.62:0/1041762925 conn(0x7fbcd40b14d0 msgr2=0x7fbcd40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:41.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.062+0000 7fbcd9e00640 1 -- 172.21.15.62:0/1041762925 shutdown_connections 2024-04-25T09:15:41.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.062+0000 7fbcd9e00640 1 -- 172.21.15.62:0/1041762925 wait complete. 2024-04-25T09:15:41.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.063+0000 7fbcd9e00640 1 Processor -- start 2024-04-25T09:15:41.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.064+0000 7fbcd9e00640 1 -- start start 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.064+0000 7fbcd9e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 0x7fbcd415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.064+0000 7fbcd9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40cc9e0 0x7fbcd4157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40b4ad0 con 0x7fbcd4161f00 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40b35a0 con 0x7fbcd40cc9e0 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40b4c50 con 0x7fbcd406d900 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41406/0 (socket says 172.21.15.62:41406) 2024-04-25T09:15:41.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9400640 1 -- 172.21.15.62:0/36487316 learned_addr learned my addr 172.21.15.62:0/36487316 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd8a00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 0x7fbcd415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd3e00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40cc9e0 0x7fbcd4157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9400640 1 -- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 msgr2=0x7fbcd415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9400640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 0x7fbcd415d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9400640 1 -- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40cc9e0 msgr2=0x7fbcd4157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9400640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40cc9e0 0x7fbcd4157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.065+0000 7fbcd9400640 1 -- 172.21.15.62:0/36487316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4157df0 con 0x7fbcd4161f00 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.066+0000 7fbcd9400640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41f80f0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fbccc007e10 tx=0x7fbccc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.066+0000 7fbcd0c00640 1 -- 172.21.15.62:0/36487316 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc014620 con 0x7fbcd4161f00 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.066+0000 7fbcd0c00640 1 -- 172.21.15.62:0/36487316 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc019070 con 0x7fbcd4161f00 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.066+0000 7fbcd0c00640 1 -- 172.21.15.62:0/36487316 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc014a20 con 0x7fbcd4161f00 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.066+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd4160890 con 0x7fbcd4161f00 2024-04-25T09:15:41.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.066+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd4150f70 con 0x7fbcd4161f00 2024-04-25T09:15:41.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.067+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd404ef20 con 0x7fbcd4161f00 2024-04-25T09:15:41.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.068+0000 7fbcd0c00640 1 -- 172.21.15.62:0/36487316 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbccc014bc0 con 0x7fbcd4161f00 2024-04-25T09:15:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.069+0000 7fbcd0c00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbcb006b4d0 0x7fbcb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.069+0000 7fbcd0c00640 1 -- 172.21.15.62:0/36487316 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbccc08ab20 con 0x7fbcd4161f00 2024-04-25T09:15:41.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.070+0000 7fbcd8a00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbcb006b4d0 0x7fbcb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.071+0000 7fbcd8a00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbcb006b4d0 0x7fbcb006d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc034000 tx=0x7fbcbc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.074+0000 7fbcd0c00640 1 -- 172.21.15.62:0/36487316 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbccc0598c0 con 0x7fbcd4161f00 2024-04-25T09:15:41.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.304+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcd406cb20 con 0x7fbcd4161f00 2024-04-25T09:15:41.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.307+0000 7fbcd0c00640 1 -- 172.21.15.62:0/36487316 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16756 (secure 0 0 0) 0x7fbccc0591c0 con 0x7fbcd4161f00 2024-04-25T09:15:41.295 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:41.295 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":125,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:15:03.711047+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:14:44.925556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:14:46.076732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.8","mappings":[{"from":4,"to":3}]},{"pgid":"17.1e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:15:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.309+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbcb006b4d0 msgr2=0x7fbcb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.309+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbcb006b4d0 0x7fbcb006d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc034000 tx=0x7fbcbc02e000 comp rx=0 tx=0).stop 2024-04-25T09:15:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 msgr2=0x7fbcd41f80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41f80f0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fbccc007e10 tx=0x7fbccc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 shutdown_connections 2024-04-25T09:15:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbcb006b4d0 0x7fbcb006d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcd4161f00 0x7fbcd41f80f0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40cc9e0 0x7fbcd4157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 --2- 172.21.15.62:0/36487316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcd406d900 0x7fbcd415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 >> 172.21.15.62:0/36487316 conn(0x7fbcd40b14d0 msgr2=0x7fbcd4071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 shutdown_connections 2024-04-25T09:15:41.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.310+0000 7fbcd9e00640 1 -- 172.21.15.62:0/36487316 wait complete. 2024-04-25T09:15:41.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:14:44.925556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:14:46.076732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:15:41.317 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:15:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.539+0000 7fd6be600640 1 Processor -- start 2024-04-25T09:15:41.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.539+0000 7fd6be600640 1 -- start start 2024-04-25T09:15:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.540+0000 7fd6be600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd6b815bdf0 0x7fd6b80720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.540+0000 7fd6be600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6b806c9b0 0x7fd6b80703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.540+0000 7fd6be600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b806ce00 con 0x7fd6b80cebc0 2024-04-25T09:15:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.540+0000 7fd6be600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b815c1d0 con 0x7fd6b806c9b0 2024-04-25T09:15:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.540+0000 7fd6be600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b80ae8c0 con 0x7fd6b815bdf0 2024-04-25T09:15:41.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.540+0000 7fd6b7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6b806c9b0 0x7fd6b80703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6b806c9b0 0x7fd6b80703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41410/0 (socket says 172.21.15.62:41410) 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b7e00640 1 -- 172.21.15.62:0/1769414988 learned_addr learned my addr 172.21.15.62:0/1769414988 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b6a00640 1 --1- 172.21.15.62:0/1769414988 >> v1:172.21.15.62:6789/0 conn(0x7fd6b80cebc0 0x7fd6b806c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59940/0 (socket says 172.21.15.62:59940) 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b7e00640 1 -- 172.21.15.62:0/1769414988 >> v2:172.21.15.121:3300/0 conn(0x7fd6b815bdf0 msgr2=0x7fd6b80720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b7400640 1 --2- 172.21.15.62:0/1769414988 >> v2:172.21.15.121:3300/0 conn(0x7fd6b815bdf0 0x7fd6b80720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b7e00640 1 --2- 172.21.15.62:0/1769414988 >> v2:172.21.15.121:3300/0 conn(0x7fd6b815bdf0 0x7fd6b80720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b7e00640 1 -- 172.21.15.62:0/1769414988 >> v1:172.21.15.62:6789/0 conn(0x7fd6b80cebc0 legacy=0x7fd6b806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b7e00640 1 -- 172.21.15.62:0/1769414988 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b806d070 con 0x7fd6b806c9b0 2024-04-25T09:15:41.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.541+0000 7fd6b6000640 1 -- 172.21.15.62:0/1769414988 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703554383 0 0) 0x7fd6b806ce00 con 0x7fd6b80cebc0 2024-04-25T09:15:41.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.542+0000 7fd6b7e00640 1 --2- 172.21.15.62:0/1769414988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b806c9b0 0x7fd6b80703a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd6b806c080 tx=0x7fd6b002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359148e0544aaca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.542+0000 7fd6b6000640 1 -- 172.21.15.62:0/1769414988 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0036070 con 0x7fd6b806c9b0 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.542+0000 7fd6b6000640 1 -- 172.21.15.62:0/1769414988 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b0031600 con 0x7fd6b806c9b0 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6b6000640 1 -- 172.21.15.62:0/1769414988 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00319d0 con 0x7fd6b806c9b0 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6be600640 1 -- 172.21.15.62:0/1769414988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b806c9b0 msgr2=0x7fd6b80703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6be600640 1 --2- 172.21.15.62:0/1769414988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b806c9b0 0x7fd6b80703a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd6b806c080 tx=0x7fd6b002a730 comp rx=0 tx=0).stop 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6be600640 1 -- 172.21.15.62:0/1769414988 shutdown_connections 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6be600640 1 --2- 172.21.15.62:0/1769414988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b806c9b0 0x7fd6b80703a0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6be600640 1 --2- 172.21.15.62:0/1769414988 >> v2:172.21.15.121:3300/0 conn(0x7fd6b815bdf0 0x7fd6b80720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6be600640 1 -- 172.21.15.62:0/1769414988 >> 172.21.15.62:0/1769414988 conn(0x7fd6b80b1410 msgr2=0x7fd6b80b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6be600640 1 -- 172.21.15.62:0/1769414988 shutdown_connections 2024-04-25T09:15:41.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.543+0000 7fd6be600640 1 -- 172.21.15.62:0/1769414988 wait complete. 2024-04-25T09:15:41.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.545+0000 7fd6be600640 1 Processor -- start 2024-04-25T09:15:41.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.545+0000 7fd6be600640 1 -- start start 2024-04-25T09:15:41.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.546+0000 7fd6be600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6b806c9b0 0x7fd6b80af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.546+0000 7fd6be600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cebc0 0x7fd6b81f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.546+0000 7fd6be600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6b815bdf0 0x7fd6b814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.546+0000 7fd6be600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80aedf0 con 0x7fd6b80cebc0 2024-04-25T09:15:41.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.546+0000 7fd6be600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80aeb10 con 0x7fd6b815bdf0 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.546+0000 7fd6be600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80708e0 con 0x7fd6b806c9b0 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cebc0 0x7fd6b81f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cebc0 0x7fd6b81f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41424/0 (socket says 172.21.15.62:41424) 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 -- 172.21.15.62:0/1273995121 learned_addr learned my addr 172.21.15.62:0/1273995121 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b7e00640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6b815bdf0 0x7fd6b814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b7400640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6b806c9b0 0x7fd6b80af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 -- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6b806c9b0 msgr2=0x7fd6b80af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6b806c9b0 0x7fd6b80af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 -- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6b815bdf0 msgr2=0x7fd6b814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6b815bdf0 0x7fd6b814ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 -- 172.21.15.62:0/1273995121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b81f78d0 con 0x7fd6b80cebc0 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b7e00640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6b815bdf0 0x7fd6b814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:41.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.547+0000 7fd6b6a00640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cebc0 0x7fd6b81f6f60 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c500 tx=0x7fd6a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.548+0000 7fd6af400640 1 -- 172.21.15.62:0/1273995121 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0018070 con 0x7fd6b80cebc0 2024-04-25T09:15:41.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.548+0000 7fd6af400640 1 -- 172.21.15.62:0/1273995121 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a0007580 con 0x7fd6b80cebc0 2024-04-25T09:15:41.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.548+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b81f7bb0 con 0x7fd6b80cebc0 2024-04-25T09:15:41.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.548+0000 7fd6af400640 1 -- 172.21.15.62:0/1273995121 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004a20 con 0x7fd6b80cebc0 2024-04-25T09:15:41.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.548+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b80af910 con 0x7fd6b80cebc0 2024-04-25T09:15:41.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.549+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b804ef20 con 0x7fd6b80cebc0 2024-04-25T09:15:41.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.550+0000 7fd6af400640 1 -- 172.21.15.62:0/1273995121 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6a001b070 con 0x7fd6b80cebc0 2024-04-25T09:15:41.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.551+0000 7fd6af400640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd69406b4d0 0x7fd69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.551+0000 7fd6af400640 1 -- 172.21.15.62:0/1273995121 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd6a0089580 con 0x7fd6b80cebc0 2024-04-25T09:15:41.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.552+0000 7fd6b7400640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd69406b4d0 0x7fd69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.553+0000 7fd6b7400640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd69406b4d0 0x7fd69406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800b7c0 tx=0x7fd6a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.555+0000 7fd6af400640 1 -- 172.21.15.62:0/1273995121 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6a00583b0 con 0x7fd6b80cebc0 2024-04-25T09:15:41.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.752+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6b8071c90 con 0x7fd6b80cebc0 2024-04-25T09:15:41.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.753+0000 7fd6af400640 1 -- 172.21.15.62:0/1273995121 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6a0057cb0 con 0x7fd6b80cebc0 2024-04-25T09:15:41.740 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:15:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd69406b4d0 msgr2=0x7fd69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd69406b4d0 0x7fd69406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800b7c0 tx=0x7fd6a8008040 comp rx=0 tx=0).stop 2024-04-25T09:15:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cebc0 msgr2=0x7fd6b81f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cebc0 0x7fd6b81f6f60 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c500 tx=0x7fd6a000c9c0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 shutdown_connections 2024-04-25T09:15:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd69406b4d0 0x7fd69406d990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80cebc0 0x7fd6b81f6f60 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6b815bdf0 0x7fd6b814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 --2- 172.21.15.62:0/1273995121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6b806c9b0 0x7fd6b80af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.755+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 >> 172.21.15.62:0/1273995121 conn(0x7fd6b80b1410 msgr2=0x7fd6b806c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:41.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.756+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 shutdown_connections 2024-04-25T09:15:41.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.756+0000 7fd6be600640 1 -- 172.21.15.62:0/1273995121 wait complete. 2024-04-25T09:15:41.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:15:41.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.984+0000 7f8155a00640 1 Processor -- start 2024-04-25T09:15:41.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.985+0000 7f8155a00640 1 -- start start 2024-04-25T09:15:41.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.986+0000 7f8155a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815015dc70 0x7f815006dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.986+0000 7f8155a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f815006e160 0x7f815006cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.986+0000 7f8155a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815006d7f0 con 0x7f815015e050 2024-04-25T09:15:41.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.986+0000 7f8155a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815006d000 con 0x7f815006e160 2024-04-25T09:15:41.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.986+0000 7f8155a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815006d230 con 0x7f815015dc70 2024-04-25T09:15:41.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.987+0000 7f814f400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f815006e160 0x7f815006cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.987+0000 7f814ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815015dc70 0x7f815006dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.988+0000 7f8147e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f815015e050 0x7f815006c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:34390/0 (socket says 172.21.15.62:34390) 2024-04-25T09:15:41.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.988+0000 7f8147e00640 1 -- 172.21.15.62:0/3042574559 learned_addr learned my addr 172.21.15.62:0/3042574559 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:41.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.988+0000 7f814f400640 1 -- 172.21.15.62:0/3042574559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815015dc70 msgr2=0x7f815006dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:41.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.988+0000 7f814f400640 1 --2- 172.21.15.62:0/3042574559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815015dc70 0x7f815006dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.988+0000 7f814f400640 1 -- 172.21.15.62:0/3042574559 >> v1:172.21.15.121:6789/0 conn(0x7f815015e050 legacy=0x7f815006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.988+0000 7f814f400640 1 -- 172.21.15.62:0/3042574559 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815006d510 con 0x7f815006e160 2024-04-25T09:15:41.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.990+0000 7f814f400640 1 --2- 172.21.15.62:0/3042574559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815006e160 0x7f815006cac0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8148009d90 tx=0x7f814802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=198cbd2d677d5383 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.991+0000 7f814e000640 1 -- 172.21.15.62:0/3042574559 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148007740 con 0x7f815006e160 2024-04-25T09:15:41.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.991+0000 7f814e000640 1 -- 172.21.15.62:0/3042574559 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81480078e0 con 0x7f815006e160 2024-04-25T09:15:41.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.991+0000 7f814e000640 1 -- 172.21.15.62:0/3042574559 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148007cb0 con 0x7f815006e160 2024-04-25T09:15:41.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.991+0000 7f8155a00640 1 -- 172.21.15.62:0/3042574559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815006e160 msgr2=0x7f815006cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.991+0000 7f8155a00640 1 --2- 172.21.15.62:0/3042574559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815006e160 0x7f815006cac0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8148009d90 tx=0x7f814802a660 comp rx=0 tx=0).stop 2024-04-25T09:15:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.992+0000 7f8155a00640 1 -- 172.21.15.62:0/3042574559 shutdown_connections 2024-04-25T09:15:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.992+0000 7f8155a00640 1 --2- 172.21.15.62:0/3042574559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815006e160 0x7f815006cac0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.992+0000 7f8155a00640 1 --2- 172.21.15.62:0/3042574559 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815015dc70 0x7f815006dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.992+0000 7f8155a00640 1 -- 172.21.15.62:0/3042574559 >> 172.21.15.62:0/3042574559 conn(0x7f81500b1410 msgr2=0x7f81500b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.992+0000 7f8155a00640 1 -- 172.21.15.62:0/3042574559 shutdown_connections 2024-04-25T09:15:41.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.992+0000 7f8155a00640 1 -- 172.21.15.62:0/3042574559 wait complete. 2024-04-25T09:15:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.994+0000 7f8155a00640 1 Processor -- start 2024-04-25T09:15:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.994+0000 7f8155a00640 1 -- start start 2024-04-25T09:15:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f8155a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815006e160 0x7f8150144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f8155a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815015dc70 0x7f81500b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f8155a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f815015e050 0x7f81500b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f8155a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500b2fb0 con 0x7f815015e050 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f8155a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500b2a90 con 0x7f815015dc70 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f8155a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81500b3130 con 0x7f815006e160 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f814f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f815015e050 0x7f81500b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f814f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f815015e050 0x7f81500b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41434/0 (socket says 172.21.15.62:41434) 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f814f400640 1 -- 172.21.15.62:0/3578499929 learned_addr learned my addr 172.21.15.62:0/3578499929 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f814ea00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815006e160 0x7f8150144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.996+0000 7f8147e00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815015dc70 0x7f81500b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f814f400640 1 -- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815006e160 msgr2=0x7f8150144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f814f400640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815006e160 0x7f8150144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f814f400640 1 -- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815015dc70 msgr2=0x7f81500b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f814f400640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815015dc70 0x7f81500b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f814f400640 1 -- 172.21.15.62:0/3578499929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815015e4d0 con 0x7f815015e050 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f814ea00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815006e160 0x7f8150144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f8147e00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815015dc70 0x7f81500b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:15:41.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f814f400640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f815015e050 0x7f81500b07a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8148009eb0 tx=0x7f8148004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f8146a00640 1 -- 172.21.15.62:0/3578499929 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148040070 con 0x7f815015e050 2024-04-25T09:15:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f8146a00640 1 -- 172.21.15.62:0/3578499929 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8148031bc0 con 0x7f815015e050 2024-04-25T09:15:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.997+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815015e770 con 0x7f815015e050 2024-04-25T09:15:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.998+0000 7f8146a00640 1 -- 172.21.15.62:0/3578499929 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148009450 con 0x7f815015e050 2024-04-25T09:15:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.998+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815015ed30 con 0x7f815015e050 2024-04-25T09:15:41.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:41.999+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81500685b0 con 0x7f815015e050 2024-04-25T09:15:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.000+0000 7f8146a00640 1 -- 172.21.15.62:0/3578499929 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8148039040 con 0x7f815015e050 2024-04-25T09:15:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.001+0000 7f8146a00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f812c06b4d0 0x7f812c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.001+0000 7f8146a00640 1 -- 172.21.15.62:0/3578499929 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f81480ab9e0 con 0x7f815015e050 2024-04-25T09:15:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.001+0000 7f814ea00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f812c06b4d0 0x7f812c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:41.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.003+0000 7f814ea00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f812c06b4d0 0x7f812c06d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8140009b90 tx=0x7f8140008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:41.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.005+0000 7f8146a00640 1 -- 172.21.15.62:0/3578499929 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81500685b0 con 0x7f815015e050 2024-04-25T09:15:42.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.205+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f815015b8c0 con 0x7f815015e050 2024-04-25T09:15:42.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.206+0000 7f8146a00640 1 -- 172.21.15.62:0/3578499929 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8148043360 con 0x7f815015e050 2024-04-25T09:15:42.194 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:15:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f812c06b4d0 msgr2=0x7f812c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f812c06b4d0 0x7f812c06d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8140009b90 tx=0x7f8140008040 comp rx=0 tx=0).stop 2024-04-25T09:15:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f815015e050 msgr2=0x7f81500b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f815015e050 0x7f81500b07a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8148009eb0 tx=0x7f8148004a40 comp rx=0 tx=0).stop 2024-04-25T09:15:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 shutdown_connections 2024-04-25T09:15:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f812c06b4d0 0x7f812c06d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f815015e050 0x7f81500b07a0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f815015dc70 0x7f81500b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 --2- 172.21.15.62:0/3578499929 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f815006e160 0x7f8150144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 >> 172.21.15.62:0/3578499929 conn(0x7f81500b1410 msgr2=0x7f815015bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 shutdown_connections 2024-04-25T09:15:42.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.209+0000 7f8155a00640 1 -- 172.21.15.62:0/3578499929 wait complete. 2024-04-25T09:15:42.217 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:15:42.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.443+0000 7f2d6aa00640 1 Processor -- start 2024-04-25T09:15:42.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.443+0000 7f2d6aa00640 1 -- start start 2024-04-25T09:15:42.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.445+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 0x7f2d6415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.445+0000 7f2d6aa00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6415ee30 con 0x7f2d640ceb50 2024-04-25T09:15:42.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.445+0000 7f2d6aa00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d641618b0 con 0x7f2d6415e5f0 2024-04-25T09:15:42.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.445+0000 7f2d6aa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64161b90 con 0x7f2d6406d9d0 2024-04-25T09:15:42.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.446+0000 7f2d69600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2d640ceb50 0x7f2d6406d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59948/0 (socket says 172.21.15.62:59948) 2024-04-25T09:15:42.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.446+0000 7f2d69600640 1 -- 172.21.15.62:0/1586206912 learned_addr learned my addr 172.21.15.62:0/1586206912 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:42.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.446+0000 7f2d68c00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 0x7f2d6415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.446+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691943217 0 0) 0x7f2d6415ee30 con 0x7f2d640ceb50 2024-04-25T09:15:42.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.446+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d4c003520 con 0x7f2d640ceb50 2024-04-25T09:15:42.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4009732402 0 0) 0x7f2d4c003520 con 0x7f2d640ceb50 2024-04-25T09:15:42.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 msgr2=0x7f2d6415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:42.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 0x7f2d6415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 >> v1:172.21.15.121:6789/0 conn(0x7f2d6415e5f0 legacy=0x7f2d640b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640cef30 con 0x7f2d640ceb50 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f2d580019f0 con 0x7f2d640ceb50 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 >> v1:172.21.15.62:6789/0 conn(0x7f2d640ceb50 legacy=0x7f2d6406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 0x7f2d4c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c003fb0 0x7f2d4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4c004d40 0x7f2d4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64161b90 con 0x7f2d4c003fb0 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c00bbb0 con 0x7f2d4c004d40 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c00bdb0 con 0x7f2d6406d9d0 2024-04-25T09:15:42.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.447+0000 7f2d68c00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c003fb0 0x7f2d4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d69600640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 0x7f2d4c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d6a000640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4c004d40 0x7f2d4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d68c00640 1 -- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 msgr2=0x7f2d4c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:42.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d68c00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 0x7f2d4c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d68c00640 1 -- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4c004d40 msgr2=0x7f2d4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:42.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d68c00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4c004d40 0x7f2d4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d68c00640 1 -- 172.21.15.62:0/1586206912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c003520 con 0x7f2d4c003fb0 2024-04-25T09:15:42.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d69600640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d6406d9d0 0x7f2d4c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:42.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.448+0000 7f2d6a000640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d4c004d40 0x7f2d4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:42.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.449+0000 7f2d68c00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c003fb0 0x7f2d4c004800 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009ac0 tx=0x7f2d4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1445c4860fbbb72e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:42.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.449+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50036070 con 0x7f2d4c003fb0 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.449+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d50031400 con 0x7f2d4c003fb0 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.449+0000 7f2d63e00640 1 -- 172.21.15.62:0/1586206912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50009690 con 0x7f2d4c003fb0 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.450+0000 7f2d6aa00640 1 -- 172.21.15.62:0/1586206912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c003fb0 msgr2=0x7f2d4c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.450+0000 7f2d6aa00640 1 --2- 172.21.15.62:0/1586206912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d4c003fb0 0x7f2d4c004800 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009ac0 tx=0x7f2d4c005830 comp rx=0 tx=0).stop 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.450+0000 7f2d6a000640 1 -- 172.21.15.62:0/1586206912 reap_dead start 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.450+0000 7f2d6aa00640 1 -- 172.21.15.62:0/1586206912 shutdown_connections 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.450+0000 7f2d6aa00640 1 -- 172.21.15.62:0/1586206912 >> 172.21.15.62:0/1586206912 conn(0x7f2d64066f20 msgr2=0x7f2d640b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.450+0000 7f2d6aa00640 1 -- 172.21.15.62:0/1586206912 shutdown_connections 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.450+0000 7f2d6aa00640 1 -- 172.21.15.62:0/1586206912 wait complete. 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.452+0000 7f2d6aa00640 1 Processor -- start 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.452+0000 7f2d6aa00640 1 -- start start 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d4c003fb0 0x7f2d64140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d6413ef60 0x7f2d641f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d641558c0 0x7f2d640b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64162510 con 0x7f2d641558c0 2024-04-25T09:15:42.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6aa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64162390 con 0x7f2d6413ef60 2024-04-25T09:15:42.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6aa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6415e0b0 con 0x7f2d4c003fb0 2024-04-25T09:15:42.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d641558c0 0x7f2d640b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d641558c0 0x7f2d640b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41458/0 (socket says 172.21.15.62:41458) 2024-04-25T09:15:42.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.453+0000 7f2d6a000640 1 -- 172.21.15.62:0/2644141234 learned_addr learned my addr 172.21.15.62:0/2644141234 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:42.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d69600640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d4c003fb0 0x7f2d64140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d68c00640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d6413ef60 0x7f2d641f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d6a000640 1 -- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d4c003fb0 msgr2=0x7f2d64140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d6a000640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d4c003fb0 0x7f2d64140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d6a000640 1 -- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d6413ef60 msgr2=0x7f2d641f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d6a000640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d6413ef60 0x7f2d641f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d6a000640 1 -- 172.21.15.62:0/2644141234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d641f4110 con 0x7f2d641558c0 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d6a000640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d641558c0 0x7f2d640b00c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f2d64155110 tx=0x7f2d5c00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.454+0000 7f2d61600640 1 -- 172.21.15.62:0/2644141234 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c005310 con 0x7f2d641558c0 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.455+0000 7f2d61600640 1 -- 172.21.15.62:0/2644141234 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c007030 con 0x7f2d641558c0 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.455+0000 7f2d61600640 1 -- 172.21.15.62:0/2644141234 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c007420 con 0x7f2d641558c0 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.455+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d641560d0 con 0x7f2d641558c0 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.455+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d64141830 con 0x7f2d641558c0 2024-04-25T09:15:42.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.456+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d640b1c40 con 0x7f2d641558c0 2024-04-25T09:15:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.457+0000 7f2d61600640 1 -- 172.21.15.62:0/2644141234 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d5c01b020 con 0x7f2d641558c0 2024-04-25T09:15:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.458+0000 7f2d61600640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d4406b4d0 0x7f2d4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.458+0000 7f2d61600640 1 -- 172.21.15.62:0/2644141234 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2d5c089b40 con 0x7f2d641558c0 2024-04-25T09:15:42.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.458+0000 7f2d69600640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d4406b4d0 0x7f2d4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.460+0000 7f2d69600640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d4406b4d0 0x7f2d4406d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001b60 tx=0x7f2d58009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:42.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.463+0000 7f2d61600640 1 -- 172.21.15.62:0/2644141234 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d640b1c40 con 0x7f2d641558c0 2024-04-25T09:15:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.663+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d64145c50 con 0x7f2d641558c0 2024-04-25T09:15:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.664+0000 7f2d61600640 1 -- 172.21.15.62:0/2644141234 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d5c058970 con 0x7f2d641558c0 2024-04-25T09:15:42.651 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d4406b4d0 msgr2=0x7f2d4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d4406b4d0 0x7f2d4406d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001b60 tx=0x7f2d58009040 comp rx=0 tx=0).stop 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d641558c0 msgr2=0x7f2d640b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d641558c0 0x7f2d640b00c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f2d64155110 tx=0x7f2d5c00d890 comp rx=0 tx=0).stop 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 shutdown_connections 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2d4406b4d0 0x7f2d4406d990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d641558c0 0x7f2d640b00c0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d6413ef60 0x7f2d641f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 --2- 172.21.15.62:0/2644141234 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2d4c003fb0 0x7f2d64140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 >> 172.21.15.62:0/2644141234 conn(0x7f2d64066f20 msgr2=0x7f2d64146170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.667+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 shutdown_connections 2024-04-25T09:15:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.668+0000 7f2d6aa00640 1 -- 172.21.15.62:0/2644141234 wait complete. 2024-04-25T09:15:42.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:15:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.897+0000 7fcbc1200640 1 Processor -- start 2024-04-25T09:15:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.897+0000 7fcbc1200640 1 -- start start 2024-04-25T09:15:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.899+0000 7fcbc1200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc15dd50 0x7fcbbc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.899+0000 7fcbc1200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcbbc0ae7e0 0x7fcbbc162270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.899+0000 7fcbc1200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbbc0aec20 con 0x7fcbbc0ceac0 2024-04-25T09:15:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.899+0000 7fcbc1200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbbc06e120 con 0x7fcbbc0ae7e0 2024-04-25T09:15:42.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.899+0000 7fcbc1200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc06db60 con 0x7fcbbc15dd50 2024-04-25T09:15:42.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.899+0000 7fcbb3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcbbc0ae7e0 0x7fcbbc162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.899+0000 7fcbba800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fcbbc0ceac0 0x7fcbbc0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59956/0 (socket says 172.21.15.62:59956) 2024-04-25T09:15:42.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.899+0000 7fcbba800640 1 -- 172.21.15.62:0/1468319535 learned_addr learned my addr 172.21.15.62:0/1468319535 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:42.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.900+0000 7fcbb9e00640 1 --2- 172.21.15.62:0/1468319535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc15dd50 0x7fcbbc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.900+0000 7fcbb3e00640 1 -- 172.21.15.62:0/1468319535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc15dd50 msgr2=0x7fcbbc06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:42.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.900+0000 7fcbb3e00640 1 --2- 172.21.15.62:0/1468319535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc15dd50 0x7fcbbc06d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.900+0000 7fcbb3e00640 1 -- 172.21.15.62:0/1468319535 >> v1:172.21.15.62:6789/0 conn(0x7fcbbc0ceac0 legacy=0x7fcbbc0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.900+0000 7fcbb3e00640 1 -- 172.21.15.62:0/1468319535 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc06de40 con 0x7fcbbc0ae7e0 2024-04-25T09:15:42.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.900+0000 7fcbb9400640 1 -- 172.21.15.62:0/1468319535 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552617644 0 0) 0x7fcbbc0aec20 con 0x7fcbbc0ceac0 2024-04-25T09:15:42.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.901+0000 7fcbb3e00640 1 --2- 172.21.15.62:0/1468319535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc0ae7e0 0x7fcbbc162270 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcba40097a0 tx=0x7fcba402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdd7a642da72dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:42.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.901+0000 7fcbb9400640 1 -- 172.21.15.62:0/1468319535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4037070 con 0x7fcbbc0ae7e0 2024-04-25T09:15:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.901+0000 7fcbb9400640 1 -- 172.21.15.62:0/1468319535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba402ae50 con 0x7fcbbc0ae7e0 2024-04-25T09:15:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.901+0000 7fcbb9400640 1 -- 172.21.15.62:0/1468319535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4032660 con 0x7fcbbc0ae7e0 2024-04-25T09:15:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.902+0000 7fcbc1200640 1 -- 172.21.15.62:0/1468319535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc0ae7e0 msgr2=0x7fcbbc162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.902+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1468319535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc0ae7e0 0x7fcbbc162270 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcba40097a0 tx=0x7fcba402a100 comp rx=0 tx=0).stop 2024-04-25T09:15:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.902+0000 7fcbc1200640 1 -- 172.21.15.62:0/1468319535 shutdown_connections 2024-04-25T09:15:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.902+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1468319535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc0ae7e0 0x7fcbbc162270 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.902+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1468319535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc15dd50 0x7fcbbc06d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.902+0000 7fcbc1200640 1 -- 172.21.15.62:0/1468319535 >> 172.21.15.62:0/1468319535 conn(0x7fcbbc0668f0 msgr2=0x7fcbbc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.902+0000 7fcbc1200640 1 -- 172.21.15.62:0/1468319535 shutdown_connections 2024-04-25T09:15:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.902+0000 7fcbc1200640 1 -- 172.21.15.62:0/1468319535 wait complete. 2024-04-25T09:15:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.904+0000 7fcbc1200640 1 Processor -- start 2024-04-25T09:15:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.904+0000 7fcbc1200640 1 -- start start 2024-04-25T09:15:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.905+0000 7fcbc1200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc0ae7e0 0x7fcbbc146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.905+0000 7fcbc1200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbbc0ceac0 0x7fcbbc157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.905+0000 7fcbc1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc15dd50 0x7fcbbc15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.905+0000 7fcbc1200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15e130 con 0x7fcbbc15dd50 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.905+0000 7fcbc1200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc161d50 con 0x7fcbbc0ceac0 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.905+0000 7fcbc1200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15e2b0 con 0x7fcbbc0ae7e0 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc15dd50 0x7fcbbc15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc15dd50 0x7fcbbc15b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41480/0 (socket says 172.21.15.62:41480) 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 -- 172.21.15.62:0/1707903269 learned_addr learned my addr 172.21.15.62:0/1707903269 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbb9e00640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc0ae7e0 0x7fcbbc146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 -- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc0ae7e0 msgr2=0x7fcbbc146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc0ae7e0 0x7fcbbc146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 -- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbbc0ceac0 msgr2=0x7fcbbc157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbb3e00640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbbc0ceac0 0x7fcbbc157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbbc0ceac0 0x7fcbbc157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:42.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 -- 172.21.15.62:0/1707903269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc1588d0 con 0x7fcbbc15dd50 2024-04-25T09:15:42.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.906+0000 7fcbba800640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc15dd50 0x7fcbbc15b500 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c7d0 tx=0x7fcbb400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:42.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.907+0000 7fcbb2000640 1 -- 172.21.15.62:0/1707903269 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4018070 con 0x7fcbbc15dd50 2024-04-25T09:15:42.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.907+0000 7fcbb2000640 1 -- 172.21.15.62:0/1707903269 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb400a640 con 0x7fcbbc15dd50 2024-04-25T09:15:42.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.907+0000 7fcbb2000640 1 -- 172.21.15.62:0/1707903269 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400aa70 con 0x7fcbbc15dd50 2024-04-25T09:15:42.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.907+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc1585f0 con 0x7fcbbc15dd50 2024-04-25T09:15:42.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.907+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc15f200 con 0x7fcbbc15dd50 2024-04-25T09:15:42.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.908+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc0685c0 con 0x7fcbbc15dd50 2024-04-25T09:15:42.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.909+0000 7fcbb2000640 1 -- 172.21.15.62:0/1707903269 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbb400ac10 con 0x7fcbbc15dd50 2024-04-25T09:15:42.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.910+0000 7fcbb2000640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcb9806b500 0x7fcb9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:42.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.910+0000 7fcbb2000640 1 -- 172.21.15.62:0/1707903269 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcbb40898c0 con 0x7fcbbc15dd50 2024-04-25T09:15:42.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.911+0000 7fcbb9e00640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcb9806b500 0x7fcb9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:42.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.912+0000 7fcbb9e00640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcb9806b500 0x7fcb9806d9c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcbac006fd0 tx=0x7fcbac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:42.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:42.915+0000 7fcbb2000640 1 -- 172.21.15.62:0/1707903269 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbbc0685c0 con 0x7fcbbc15dd50 2024-04-25T09:15:43.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.129+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcbbc072c70 con 0x7fcbbc15dd50 2024-04-25T09:15:43.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.129+0000 7fcbb2000640 1 -- 172.21.15.62:0/1707903269 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcbb4058660 con 0x7fcbbc15dd50 2024-04-25T09:15:43.117 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcb9806b500 msgr2=0x7fcb9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcb9806b500 0x7fcb9806d9c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcbac006fd0 tx=0x7fcbac008040 comp rx=0 tx=0).stop 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc15dd50 msgr2=0x7fcbbc15b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc15dd50 0x7fcbbc15b500 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c7d0 tx=0x7fcbb400cc90 comp rx=0 tx=0).stop 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 shutdown_connections 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcb9806b500 0x7fcb9806d9c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcbbc15dd50 0x7fcbbc15b500 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbbc0ceac0 0x7fcbbc157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 --2- 172.21.15.62:0/1707903269 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcbbc0ae7e0 0x7fcbbc146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:43.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.132+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 >> 172.21.15.62:0/1707903269 conn(0x7fcbbc0668f0 msgr2=0x7fcbbc156690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:43.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.133+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 shutdown_connections 2024-04-25T09:15:43.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:43.133+0000 7fcbc1200640 1 -- 172.21.15.62:0/1707903269 wait complete. 2024-04-25T09:15:43.138 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:15:43.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:43.138 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:15:43.848 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:15:43.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:15:43.848 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:15:43.891 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:15:43.891 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:43.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:15:43.935 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:15:43.936 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:43.978 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:15:43.978 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:15:43.978 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:44.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:15:44.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:44.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.256+0000 7f9ed3200640 1 Processor -- start 2024-04-25T09:15:44.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.256+0000 7f9ed3200640 1 -- start start 2024-04-25T09:15:44.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.258+0000 7f9ed3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9ecc15dbf0 0x7f9ecc06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.258+0000 7f9ed3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc0cf030 con 0x7f9ecc15d810 2024-04-25T09:15:44.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.258+0000 7f9ed3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc0cf310 con 0x7f9ecc0ca4e0 2024-04-25T09:15:44.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.258+0000 7f9ed3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc0c8cd0 con 0x7f9ecc15dbf0 2024-04-25T09:15:44.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.258+0000 7f9ed1e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9ecc15d810 0x7f9ecc0ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59962/0 (socket says 172.21.15.62:59962) 2024-04-25T09:15:44.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.258+0000 7f9ed1e00640 1 -- 172.21.15.62:0/1999818050 learned_addr learned my addr 172.21.15.62:0/1999818050 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.259+0000 7f9ed1400640 1 --2- 172.21.15.62:0/1999818050 >> v2:172.21.15.121:3300/0 conn(0x7f9ecc15dbf0 0x7f9ecc06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.259+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606531885 0 0) 0x7f9ecc0cf030 con 0x7f9ecc15d810 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.259+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb4003520 con 0x7f9ecc15d810 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.259+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3923456082 0 0) 0x7f9eb4003520 con 0x7f9ecc15d810 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.259+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 >> v2:172.21.15.121:3300/0 conn(0x7f9ecc15dbf0 msgr2=0x7f9ecc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.259+0000 7f9ed0a00640 1 --2- 172.21.15.62:0/1999818050 >> v2:172.21.15.121:3300/0 conn(0x7f9ecc15dbf0 0x7f9ecc06d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 >> v1:172.21.15.121:6789/0 conn(0x7f9ecc0ca4e0 legacy=0x7f9ecc1675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0c8fb0 con 0x7f9ecc15d810 2024-04-25T09:15:44.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f9ec0001670 con 0x7f9ecc15d810 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 >> v1:172.21.15.62:6789/0 conn(0x7f9ecc15d810 legacy=0x7f9ecc0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9eb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9eb4004800 0x7f9eb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eb40051c0 0x7f9eb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0c8cd0 con 0x7f9eb4003a00 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb400c0c0 con 0x7f9eb40051c0 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb400c2c0 con 0x7f9eb4004800 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed1e00640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9eb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed1400640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9eb4004800 0x7f9eb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.260+0000 7f9ed2800640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eb40051c0 0x7f9eb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.261+0000 7f9ed1e00640 1 -- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9eb4004800 msgr2=0x7f9eb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.261+0000 7f9ed1e00640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9eb4004800 0x7f9eb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.261+0000 7f9ed1e00640 1 -- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eb40051c0 msgr2=0x7f9eb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.261+0000 7f9ed1e00640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eb40051c0 0x7f9eb400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.261+0000 7f9ed1e00640 1 -- 172.21.15.62:0/1999818050 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4003520 con 0x7f9eb4003a00 2024-04-25T09:15:44.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.261+0000 7f9ed2800640 1 -- 172.21.15.62:0/1999818050 reap_dead start 2024-04-25T09:15:44.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.261+0000 7f9ed1400640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9eb4004800 0x7f9eb4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.262+0000 7f9ed1e00640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9eb4003e20 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c0e0 tx=0x7f9eb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819bdfd23a997931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.262+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0007c20 con 0x7f9eb4003a00 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.262+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0005af0 con 0x7f9eb4003a00 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.262+0000 7f9ed0a00640 1 -- 172.21.15.62:0/1999818050 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0007070 con 0x7f9eb4003a00 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.263+0000 7f9ed3200640 1 -- 172.21.15.62:0/1999818050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 msgr2=0x7f9eb4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.263+0000 7f9ed3200640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9eb4003e20 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c0e0 tx=0x7f9eb4005830 comp rx=0 tx=0).stop 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.263+0000 7f9ed3200640 1 -- 172.21.15.62:0/1999818050 shutdown_connections 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.263+0000 7f9ed3200640 1 --2- 172.21.15.62:0/1999818050 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9eb4003e20 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.263+0000 7f9ed3200640 1 -- 172.21.15.62:0/1999818050 >> 172.21.15.62:0/1999818050 conn(0x7f9ecc0b14d0 msgr2=0x7f9ecc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.263+0000 7f9ed3200640 1 -- 172.21.15.62:0/1999818050 shutdown_connections 2024-04-25T09:15:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.263+0000 7f9ed3200640 1 -- 172.21.15.62:0/1999818050 wait complete. 2024-04-25T09:15:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.265+0000 7f9ed3200640 1 Processor -- start 2024-04-25T09:15:44.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.265+0000 7f9ed3200640 1 -- start start 2024-04-25T09:15:44.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.266+0000 7f9ed3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9ecc159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.266+0000 7f9ed3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ecc158200 0x7f9ecc153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.266+0000 7f9ed3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ecc158680 0x7f9ecc153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.266+0000 7f9ed3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0ce830 con 0x7f9eb4003a00 2024-04-25T09:15:44.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.266+0000 7f9ed3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0b10d0 con 0x7f9ecc158680 2024-04-25T09:15:44.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.266+0000 7f9ed3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0ce9b0 con 0x7f9ecc158200 2024-04-25T09:15:44.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.266+0000 7f9ed1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9ecc159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9ecc159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41496/0 (socket says 172.21.15.62:41496) 2024-04-25T09:15:44.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1e00640 1 -- 172.21.15.62:0/908854945 learned_addr learned my addr 172.21.15.62:0/908854945 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:44.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1400640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ecc158200 0x7f9ecc153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed2800640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ecc158680 0x7f9ecc153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1e00640 1 -- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ecc158200 msgr2=0x7f9ecc153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1e00640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ecc158200 0x7f9ecc153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1e00640 1 -- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ecc158680 msgr2=0x7f9ecc153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1e00640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ecc158680 0x7f9ecc153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1e00640 1 -- 172.21.15.62:0/908854945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc155330 con 0x7f9eb4003a00 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed2800640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ecc158680 0x7f9ecc153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.267+0000 7f9ed1e00640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9ecc159b10 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c0e0 tx=0x7f9ecc0c9180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.268+0000 7f9ec6000640 1 -- 172.21.15.62:0/908854945 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0037040 con 0x7f9eb4003a00 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.268+0000 7f9ec6000640 1 -- 172.21.15.62:0/908854945 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0041070 con 0x7f9eb4003a00 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.268+0000 7f9ec6000640 1 -- 172.21.15.62:0/908854945 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0001670 con 0x7f9eb4003a00 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.268+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc151830 con 0x7f9eb4003a00 2024-04-25T09:15:44.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.268+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc1543d0 con 0x7f9eb4003a00 2024-04-25T09:15:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.269+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ecc04ef20 con 0x7f9eb4003a00 2024-04-25T09:15:44.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.270+0000 7f9ec6000640 1 -- 172.21.15.62:0/908854945 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ec00049e0 con 0x7f9eb4003a00 2024-04-25T09:15:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.271+0000 7f9ec6000640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.271+0000 7f9ec6000640 1 -- 172.21.15.62:0/908854945 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9ec00a9f70 con 0x7f9eb4003a00 2024-04-25T09:15:44.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.272+0000 7f9ed1400640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.273+0000 7f9ed1400640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9eb006b4d0 0x7f9eb006d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc159020 tx=0x7f9ebc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:44.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.275+0000 7f9ec6000640 1 -- 172.21.15.62:0/908854945 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ec0078e40 con 0x7f9eb4003a00 2024-04-25T09:15:44.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.510+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ecc154610 con 0x7f9eb4003a00 2024-04-25T09:15:44.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.511+0000 7f9ec6000640 1 -- 172.21.15.62:0/908854945 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9ec0078740 con 0x7f9eb4003a00 2024-04-25T09:15:44.502 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:44.503 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9eb006b4d0 msgr2=0x7f9eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9eb006b4d0 0x7f9eb006d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc159020 tx=0x7f9ebc008040 comp rx=0 tx=0).stop 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 msgr2=0x7f9ecc159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9ecc159b10 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c0e0 tx=0x7f9ecc0c9180 comp rx=0 tx=0).stop 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 shutdown_connections 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ecc158680 0x7f9ecc153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9ecc158200 0x7f9ecc153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 --2- 172.21.15.62:0/908854945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9eb4003a00 0x7f9ecc159b10 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 >> 172.21.15.62:0/908854945 conn(0x7f9ecc0b14d0 msgr2=0x7f9ecc071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.514+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 shutdown_connections 2024-04-25T09:15:44.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.515+0000 7f9ed3200640 1 -- 172.21.15.62:0/908854945 wait complete. 2024-04-25T09:15:44.521 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:15:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.740+0000 7f2f65e00640 1 Processor -- start 2024-04-25T09:15:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.740+0000 7f2f65e00640 1 -- start start 2024-04-25T09:15:44.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.742+0000 7f2f65e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2f60072020 0x7f2f600ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.742+0000 7f2f65e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2f600ce300 0x7f2f600ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.742+0000 7f2f65e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f601666d0 con 0x7f2f60165e80 2024-04-25T09:15:44.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.742+0000 7f2f65e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f601669b0 con 0x7f2f600ce300 2024-04-25T09:15:44.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.742+0000 7f2f65e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f60166c90 con 0x7f2f60072020 2024-04-25T09:15:44.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.742+0000 7f2f5f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2f600ce300 0x7f2f600ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f5f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2f600ce300 0x7f2f600ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41508/0 (socket says 172.21.15.62:41508) 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f5f400640 1 -- 172.21.15.62:0/3397090882 learned_addr learned my addr 172.21.15.62:0/3397090882 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f57e00640 1 --1- 172.21.15.62:0/3397090882 >> v1:172.21.15.62:6789/0 conn(0x7f2f60165e80 0x7f2f60070fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59964/0 (socket says 172.21.15.62:59964) 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f5ea00640 1 --2- 172.21.15.62:0/3397090882 >> v2:172.21.15.121:3300/0 conn(0x7f2f60072020 0x7f2f600ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f5f400640 1 -- 172.21.15.62:0/3397090882 >> v2:172.21.15.121:3300/0 conn(0x7f2f60072020 msgr2=0x7f2f600ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f5f400640 1 --2- 172.21.15.62:0/3397090882 >> v2:172.21.15.121:3300/0 conn(0x7f2f60072020 0x7f2f600ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f5f400640 1 -- 172.21.15.62:0/3397090882 >> v1:172.21.15.62:6789/0 conn(0x7f2f60165e80 legacy=0x7f2f60070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f5f400640 1 -- 172.21.15.62:0/3397090882 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6006d000 con 0x7f2f600ce300 2024-04-25T09:15:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.743+0000 7f2f5e000640 1 -- 172.21.15.62:0/3397090882 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748331763 0 0) 0x7f2f601666d0 con 0x7f2f60165e80 2024-04-25T09:15:44.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.745+0000 7f2f5f400640 1 --2- 172.21.15.62:0/3397090882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f600ce790 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f2f6006e410 tx=0x7f2f5802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7b4bfac988350e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:44.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.745+0000 7f2f5e000640 1 -- 172.21.15.62:0/3397090882 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58036070 con 0x7f2f600ce300 2024-04-25T09:15:44.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.745+0000 7f2f5e000640 1 -- 172.21.15.62:0/3397090882 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f58031600 con 0x7f2f600ce300 2024-04-25T09:15:44.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.745+0000 7f2f5e000640 1 -- 172.21.15.62:0/3397090882 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f580319d0 con 0x7f2f600ce300 2024-04-25T09:15:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.746+0000 7f2f65e00640 1 -- 172.21.15.62:0/3397090882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 msgr2=0x7f2f600ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.746+0000 7f2f65e00640 1 --2- 172.21.15.62:0/3397090882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f600ce790 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f2f6006e410 tx=0x7f2f5802a730 comp rx=0 tx=0).stop 2024-04-25T09:15:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.746+0000 7f2f65e00640 1 -- 172.21.15.62:0/3397090882 shutdown_connections 2024-04-25T09:15:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.746+0000 7f2f65e00640 1 --2- 172.21.15.62:0/3397090882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f600ce790 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.746+0000 7f2f65e00640 1 --2- 172.21.15.62:0/3397090882 >> v2:172.21.15.121:3300/0 conn(0x7f2f60072020 0x7f2f600ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.746+0000 7f2f65e00640 1 -- 172.21.15.62:0/3397090882 >> 172.21.15.62:0/3397090882 conn(0x7f2f600668f0 msgr2=0x7f2f60066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.746+0000 7f2f65e00640 1 -- 172.21.15.62:0/3397090882 shutdown_connections 2024-04-25T09:15:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.746+0000 7f2f65e00640 1 -- 172.21.15.62:0/3397090882 wait complete. 2024-04-25T09:15:44.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.748+0000 7f2f65e00640 1 Processor -- start 2024-04-25T09:15:44.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.748+0000 7f2f65e00640 1 -- start start 2024-04-25T09:15:44.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.749+0000 7f2f65e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f60072020 0x7f2f60152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.749+0000 7f2f65e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f601643f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.749+0000 7f2f65e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f60165e80 0x7f2f601fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.749+0000 7f2f65e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60166460 con 0x7f2f600ce300 2024-04-25T09:15:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.749+0000 7f2f65e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f601662e0 con 0x7f2f60165e80 2024-04-25T09:15:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.749+0000 7f2f65e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600716e0 con 0x7f2f60072020 2024-04-25T09:15:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.749+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f601643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f601643f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41520/0 (socket says 172.21.15.62:41520) 2024-04-25T09:15:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f57e00640 1 -- 172.21.15.62:0/2951881684 learned_addr learned my addr 172.21.15.62:0/2951881684 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:44.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f5ea00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f60072020 0x7f2f60152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f5f400640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f60165e80 0x7f2f601fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f57e00640 1 -- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f60072020 msgr2=0x7f2f60152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f57e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f60072020 0x7f2f60152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f57e00640 1 -- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f60165e80 msgr2=0x7f2f601fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f57e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f60165e80 0x7f2f601fc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f57e00640 1 -- 172.21.15.62:0/2951881684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f60164ab0 con 0x7f2f600ce300 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f5f400640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f60165e80 0x7f2f601fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.750+0000 7f2f57e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f601643f0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c500 tx=0x7f2f4800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.751+0000 7f2f57400640 1 -- 172.21.15.62:0/2951881684 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48018070 con 0x7f2f600ce300 2024-04-25T09:15:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.751+0000 7f2f57400640 1 -- 172.21.15.62:0/2951881684 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48007580 con 0x7f2f600ce300 2024-04-25T09:15:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.751+0000 7f2f57400640 1 -- 172.21.15.62:0/2951881684 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48004a20 con 0x7f2f600ce300 2024-04-25T09:15:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.751+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f60164d90 con 0x7f2f600ce300 2024-04-25T09:15:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.751+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f60165350 con 0x7f2f600ce300 2024-04-25T09:15:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.752+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f60072c70 con 0x7f2f600ce300 2024-04-25T09:15:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.753+0000 7f2f57400640 1 -- 172.21.15.62:0/2951881684 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f48004bc0 con 0x7f2f600ce300 2024-04-25T09:15:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.754+0000 7f2f57400640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.754+0000 7f2f57400640 1 -- 172.21.15.62:0/2951881684 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2f480894b0 con 0x7f2f600ce300 2024-04-25T09:15:44.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.755+0000 7f2f5ea00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:44.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.756+0000 7f2f5ea00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2f50006fd0 tx=0x7f2f50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.759+0000 7f2f57400640 1 -- 172.21.15.62:0/2951881684 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f48058380 con 0x7f2f600ce300 2024-04-25T09:15:44.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.960+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f60165bf0 con 0x7f2f600ce300 2024-04-25T09:15:44.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.962+0000 7f2f57400640 1 -- 172.21.15.62:0/2951881684 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16756 (secure 0 0 0) 0x7f2f4805e080 con 0x7f2f600ce300 2024-04-25T09:15:44.950 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:44.950 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":125,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:15:03.711047+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:14:44.925556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:14:46.076732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.8","mappings":[{"from":4,"to":3}]},{"pgid":"17.1e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:15:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.965+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2f3c06b4d0 msgr2=0x7f2f3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.965+0000 7f2f65e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2f50006fd0 tx=0x7f2f50008040 comp rx=0 tx=0).stop 2024-04-25T09:15:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.965+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 msgr2=0x7f2f601643f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.965+0000 7f2f65e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f601643f0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c500 tx=0x7f2f4800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.966+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 shutdown_connections 2024-04-25T09:15:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.966+0000 7f2f65e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.966+0000 7f2f65e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2f600ce300 0x7f2f601643f0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.966+0000 7f2f65e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f60165e80 0x7f2f601fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.966+0000 7f2f65e00640 1 --2- 172.21.15.62:0/2951881684 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2f60072020 0x7f2f60152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.966+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 >> 172.21.15.62:0/2951881684 conn(0x7f2f600668f0 msgr2=0x7f2f600cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.966+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 shutdown_connections 2024-04-25T09:15:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:44.966+0000 7f2f65e00640 1 -- 172.21.15.62:0/2951881684 wait complete. 2024-04-25T09:15:44.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:14:44.925556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:14:46.076732+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:15:44.974 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:15:45.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:15:45.156 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:15:45.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.200+0000 7f01b9000640 1 Processor -- start 2024-04-25T09:15:45.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.201+0000 7f01b9000640 1 -- start start 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.202+0000 7f01b9000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01b40ce5c0 0x7f01b4161b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.202+0000 7f01b9000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f01b40d0650 0x7f01b40d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.202+0000 7f01b9000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40b3310 con 0x7f01b415e180 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.202+0000 7f01b9000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40d2210 con 0x7f01b40ce5c0 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.202+0000 7f01b9000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b4162200 con 0x7f01b40d0650 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.202+0000 7f01a9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01b40ce5c0 0x7f01b4161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.203+0000 7f01a9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f01b40ce5c0 0x7f01b4161b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41028/0 (socket says 172.21.15.62:41028) 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.203+0000 7f01a9400640 1 -- 172.21.15.62:0/38831740 learned_addr learned my addr 172.21.15.62:0/38831740 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:45.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.203+0000 7f01b2800640 1 --2- 172.21.15.62:0/38831740 >> v2:172.21.15.121:3300/0 conn(0x7f01b40d0650 0x7f01b40d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.203+0000 7f01a9400640 1 -- 172.21.15.62:0/38831740 >> v2:172.21.15.121:3300/0 conn(0x7f01b40d0650 msgr2=0x7f01b40d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:45.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.203+0000 7f01a9400640 1 --2- 172.21.15.62:0/38831740 >> v2:172.21.15.121:3300/0 conn(0x7f01b40d0650 0x7f01b40d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.203+0000 7f01a9400640 1 -- 172.21.15.62:0/38831740 >> v1:172.21.15.121:6789/0 conn(0x7f01b415e180 legacy=0x7f01b40b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.203+0000 7f01a9400640 1 -- 172.21.15.62:0/38831740 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b41624e0 con 0x7f01b40ce5c0 2024-04-25T09:15:45.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.203+0000 7f01b1400640 1 -- 172.21.15.62:0/38831740 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170996541 0 0) 0x7f01b40b3310 con 0x7f01b415e180 2024-04-25T09:15:45.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.205+0000 7f01a9400640 1 --2- 172.21.15.62:0/38831740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b40ce5c0 0x7f01b4161b20 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f01b40ce220 tx=0x7f019c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b31e02a622379e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:45.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.205+0000 7f01b1400640 1 -- 172.21.15.62:0/38831740 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c037070 con 0x7f01b40ce5c0 2024-04-25T09:15:45.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.205+0000 7f01b1400640 1 -- 172.21.15.62:0/38831740 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c032440 con 0x7f01b40ce5c0 2024-04-25T09:15:45.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.205+0000 7f01b1400640 1 -- 172.21.15.62:0/38831740 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c032810 con 0x7f01b40ce5c0 2024-04-25T09:15:45.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.206+0000 7f01b9000640 1 -- 172.21.15.62:0/38831740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b40ce5c0 msgr2=0x7f01b4161b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.206+0000 7f01b9000640 1 --2- 172.21.15.62:0/38831740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b40ce5c0 0x7f01b4161b20 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f01b40ce220 tx=0x7f019c02a270 comp rx=0 tx=0).stop 2024-04-25T09:15:45.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.206+0000 7f01b9000640 1 -- 172.21.15.62:0/38831740 shutdown_connections 2024-04-25T09:15:45.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.206+0000 7f01b9000640 1 --2- 172.21.15.62:0/38831740 >> v2:172.21.15.121:3300/0 conn(0x7f01b40d0650 0x7f01b40d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.206+0000 7f01b9000640 1 --2- 172.21.15.62:0/38831740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b40ce5c0 0x7f01b4161b20 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.206+0000 7f01b9000640 1 -- 172.21.15.62:0/38831740 >> 172.21.15.62:0/38831740 conn(0x7f01b406b210 msgr2=0x7f01b40668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:45.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.206+0000 7f01b9000640 1 -- 172.21.15.62:0/38831740 shutdown_connections 2024-04-25T09:15:45.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.206+0000 7f01b9000640 1 -- 172.21.15.62:0/38831740 wait complete. 2024-04-25T09:15:45.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.208+0000 7f01b9000640 1 Processor -- start 2024-04-25T09:15:45.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.208+0000 7f01b9000640 1 -- start start 2024-04-25T09:15:45.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.209+0000 7f01b9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b40ce5c0 0x7f01b41e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.209+0000 7f01b9000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40d0650 0x7f01b41499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.209+0000 7f01b9000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b415e180 0x7f01b415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.209+0000 7f01b9000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40d2820 con 0x7f01b415e180 2024-04-25T09:15:45.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.209+0000 7f01b9000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40d26a0 con 0x7f01b40ce5c0 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.209+0000 7f01b9000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40d29a0 con 0x7f01b40d0650 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b415e180 0x7f01b415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b415e180 0x7f01b415f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41032/0 (socket says 172.21.15.62:41032) 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 -- 172.21.15.62:0/1842628025 learned_addr learned my addr 172.21.15.62:0/1842628025 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01a9400640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40d0650 0x7f01b41499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b1e00640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b40ce5c0 0x7f01b41e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 -- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40d0650 msgr2=0x7f01b41499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40d0650 0x7f01b41499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 -- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b40ce5c0 msgr2=0x7f01b41e77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b40ce5c0 0x7f01b41e77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 -- 172.21.15.62:0/1842628025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b414a290 con 0x7f01b415e180 2024-04-25T09:15:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01a9400640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40d0650 0x7f01b41499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:45.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b1e00640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b40ce5c0 0x7f01b41e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:45.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.210+0000 7f01b2800640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b415e180 0x7f01b415f670 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00e930 tx=0x7f01ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:45.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.211+0000 7f01aaa00640 1 -- 172.21.15.62:0/1842628025 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac017070 con 0x7f01b415e180 2024-04-25T09:15:45.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.211+0000 7f01aaa00640 1 -- 172.21.15.62:0/1842628025 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00c920 con 0x7f01b415e180 2024-04-25T09:15:45.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.211+0000 7f01aaa00640 1 -- 172.21.15.62:0/1842628025 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00cd10 con 0x7f01b415e180 2024-04-25T09:15:45.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.211+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b414a570 con 0x7f01b415e180 2024-04-25T09:15:45.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.211+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b414ab30 con 0x7f01b415e180 2024-04-25T09:15:45.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.212+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b4161020 con 0x7f01b415e180 2024-04-25T09:15:45.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.213+0000 7f01aaa00640 1 -- 172.21.15.62:0/1842628025 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01ac01a070 con 0x7f01b415e180 2024-04-25T09:15:45.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.214+0000 7f01aaa00640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f019006b4d0 0x7f019006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.214+0000 7f01aaa00640 1 -- 172.21.15.62:0/1842628025 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01ac08a250 con 0x7f01b415e180 2024-04-25T09:15:45.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.215+0000 7f01b1e00640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f019006b4d0 0x7f019006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.215+0000 7f01b1e00640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f019006b4d0 0x7f019006d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007fd0 tx=0x7f01a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:45.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.218+0000 7f01aaa00640 1 -- 172.21.15.62:0/1842628025 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01ac0590b0 con 0x7f01b415e180 2024-04-25T09:15:45.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.418+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01b404ef20 con 0x7f01b415e180 2024-04-25T09:15:45.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.419+0000 7f01aaa00640 1 -- 172.21.15.62:0/1842628025 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01ac0589b0 con 0x7f01b415e180 2024-04-25T09:15:45.406 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.421+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f019006b4d0 msgr2=0x7f019006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.421+0000 7f01b9000640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f019006b4d0 0x7f019006d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f01a4007fd0 tx=0x7f01a4009040 comp rx=0 tx=0).stop 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.421+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b415e180 msgr2=0x7f01b415f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.421+0000 7f01b9000640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b415e180 0x7f01b415f670 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00e930 tx=0x7f01ac00edf0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.422+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 shutdown_connections 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.422+0000 7f01b9000640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f019006b4d0 0x7f019006d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.422+0000 7f01b9000640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f01b415e180 0x7f01b415f670 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.422+0000 7f01b9000640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f01b40d0650 0x7f01b41499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.422+0000 7f01b9000640 1 --2- 172.21.15.62:0/1842628025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01b40ce5c0 0x7f01b41e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.422+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 >> 172.21.15.62:0/1842628025 conn(0x7f01b406b210 msgr2=0x7f01b40d2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:45.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.422+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 shutdown_connections 2024-04-25T09:15:45.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.422+0000 7f01b9000640 1 -- 172.21.15.62:0/1842628025 wait complete. 2024-04-25T09:15:45.429 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:15:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.659+0000 7f3a9ea00640 1 Processor -- start 2024-04-25T09:15:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.659+0000 7f3a9ea00640 1 -- start start 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a9ea00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3a9806dc40 0x7f3a9806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a9ea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a98155150 0x7f3a9815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a9ea00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9806e560 con 0x7f3a9815af50 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a9ea00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9815cbb0 con 0x7f3a98155150 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a9ea00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9815ce90 con 0x7f3a9806dc40 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a8fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a98155150 0x7f3a9815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a8fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3a98155150 0x7f3a9815dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41034/0 (socket says 172.21.15.62:41034) 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a8fe00640 1 -- 172.21.15.62:0/887223955 learned_addr learned my addr 172.21.15.62:0/887223955 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.661+0000 7f3a9ca00640 1 --1- 172.21.15.62:0/887223955 >> v1:172.21.15.62:6789/0 conn(0x7f3a9815af50 0x7f3a9815c360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39138/0 (socket says 172.21.15.62:39138) 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.662+0000 7f3a97e00640 1 --2- 172.21.15.62:0/887223955 >> v2:172.21.15.121:3300/0 conn(0x7f3a9806dc40 0x7f3a9806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.662+0000 7f3a8fe00640 1 -- 172.21.15.62:0/887223955 >> v2:172.21.15.121:3300/0 conn(0x7f3a9806dc40 msgr2=0x7f3a9806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:45.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.662+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/887223955 >> v2:172.21.15.121:3300/0 conn(0x7f3a9806dc40 0x7f3a9806e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.662+0000 7f3a8fe00640 1 -- 172.21.15.62:0/887223955 >> v1:172.21.15.62:6789/0 conn(0x7f3a9815af50 legacy=0x7f3a9815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.662+0000 7f3a8fe00640 1 -- 172.21.15.62:0/887223955 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9815d170 con 0x7f3a98155150 2024-04-25T09:15:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.663+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/887223955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a9815dc50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3a980678d0 tx=0x7f3a8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c697f9752f634b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.663+0000 7f3a97400640 1 -- 172.21.15.62:0/887223955 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80037070 con 0x7f3a98155150 2024-04-25T09:15:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.663+0000 7f3a97400640 1 -- 172.21.15.62:0/887223955 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a80032440 con 0x7f3a98155150 2024-04-25T09:15:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.663+0000 7f3a97400640 1 -- 172.21.15.62:0/887223955 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80032810 con 0x7f3a98155150 2024-04-25T09:15:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.664+0000 7f3a9ea00640 1 -- 172.21.15.62:0/887223955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 msgr2=0x7f3a9815dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.664+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/887223955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a9815dc50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3a980678d0 tx=0x7f3a8002a270 comp rx=0 tx=0).stop 2024-04-25T09:15:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.664+0000 7f3a9ea00640 1 -- 172.21.15.62:0/887223955 shutdown_connections 2024-04-25T09:15:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.664+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/887223955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a9815dc50 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.664+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/887223955 >> v2:172.21.15.121:3300/0 conn(0x7f3a9806dc40 0x7f3a9806e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.664+0000 7f3a9ea00640 1 -- 172.21.15.62:0/887223955 >> 172.21.15.62:0/887223955 conn(0x7f3a980b1410 msgr2=0x7f3a980b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.664+0000 7f3a9ea00640 1 -- 172.21.15.62:0/887223955 shutdown_connections 2024-04-25T09:15:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.664+0000 7f3a9ea00640 1 -- 172.21.15.62:0/887223955 wait complete. 2024-04-25T09:15:45.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.666+0000 7f3a9ea00640 1 Processor -- start 2024-04-25T09:15:45.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.666+0000 7f3a9ea00640 1 -- start start 2024-04-25T09:15:45.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a9ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a9806dc40 0x7f3a980b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a9ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a980b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a9ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a9815af50 0x7f3a98157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a9ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9815e190 con 0x7f3a98155150 2024-04-25T09:15:45.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a9ea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9815d730 con 0x7f3a9806dc40 2024-04-25T09:15:45.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a9ea00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980b2710 con 0x7f3a9815af50 2024-04-25T09:15:45.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a980b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a980b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41044/0 (socket says 172.21.15.62:41044) 2024-04-25T09:15:45.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.667+0000 7f3a8fe00640 1 -- 172.21.15.62:0/2015373685 learned_addr learned my addr 172.21.15.62:0/2015373685 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:45.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a97e00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a9806dc40 0x7f3a980b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a8fe00640 1 -- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a9815af50 msgr2=0x7f3a98157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a9ca00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a9815af50 0x7f3a98157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a9815af50 0x7f3a98157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a8fe00640 1 -- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a9806dc40 msgr2=0x7f3a980b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a9806dc40 0x7f3a980b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a8fe00640 1 -- 172.21.15.62:0/2015373685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a980ccb30 con 0x7f3a98155150 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a97e00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a9806dc40 0x7f3a980b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a8fe00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a980b3eb0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3a8002a2b0 tx=0x7f3a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.668+0000 7f3a94c00640 1 -- 172.21.15.62:0/2015373685 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80041020 con 0x7f3a98155150 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.669+0000 7f3a94c00640 1 -- 172.21.15.62:0/2015373685 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a800306c0 con 0x7f3a98155150 2024-04-25T09:15:45.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.669+0000 7f3a94c00640 1 -- 172.21.15.62:0/2015373685 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80037040 con 0x7f3a98155150 2024-04-25T09:15:45.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.669+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a980cce10 con 0x7f3a98155150 2024-04-25T09:15:45.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.669+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a980cd3d0 con 0x7f3a98155150 2024-04-25T09:15:45.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.670+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9804ef20 con 0x7f3a98155150 2024-04-25T09:15:45.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.671+0000 7f3a94c00640 1 -- 172.21.15.62:0/2015373685 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a80044020 con 0x7f3a98155150 2024-04-25T09:15:45.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.672+0000 7f3a94c00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7406b4d0 0x7f3a7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:45.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.672+0000 7f3a94c00640 1 -- 172.21.15.62:0/2015373685 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3a800aa9c0 con 0x7f3a98155150 2024-04-25T09:15:45.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.672+0000 7f3a97e00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7406b4d0 0x7f3a7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:45.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.674+0000 7f3a97e00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7406b4d0 0x7f3a7406d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3a88006fd0 tx=0x7f3a88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:45.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.676+0000 7f3a94c00640 1 -- 172.21.15.62:0/2015373685 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a800ace20 con 0x7f3a98155150 2024-04-25T09:15:45.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.872+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a9806b2c0 con 0x7f3a98155150 2024-04-25T09:15:45.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.872+0000 7f3a94c00640 1 -- 172.21.15.62:0/2015373685 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a8004b030 con 0x7f3a98155150 2024-04-25T09:15:45.860 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:15:45.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7406b4d0 msgr2=0x7f3a7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7406b4d0 0x7f3a7406d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3a88006fd0 tx=0x7f3a88008040 comp rx=0 tx=0).stop 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 msgr2=0x7f3a980b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a980b3eb0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3a8002a2b0 tx=0x7f3a800046c0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 shutdown_connections 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a7406b4d0 0x7f3a7406d990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a98155150 0x7f3a980b3eb0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3a9815af50 0x7f3a98157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 --2- 172.21.15.62:0/2015373685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a9806dc40 0x7f3a980b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 >> 172.21.15.62:0/2015373685 conn(0x7f3a980b1410 msgr2=0x7f3a980b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 shutdown_connections 2024-04-25T09:15:45.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:45.875+0000 7f3a9ea00640 1 -- 172.21.15.62:0/2015373685 wait complete. 2024-04-25T09:15:45.880 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:15:46.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.101+0000 7f78f1600640 1 Processor -- start 2024-04-25T09:15:46.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.102+0000 7f78f1600640 1 -- start start 2024-04-25T09:15:46.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.103+0000 7f78f1600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f78ec06d9d0 0x7f78ec15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.103+0000 7f78f1600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ec15ee30 con 0x7f78ec15e5f0 2024-04-25T09:15:46.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.103+0000 7f78f1600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ec1618b0 con 0x7f78ec0ceb50 2024-04-25T09:15:46.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.103+0000 7f78f1600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ec161b90 con 0x7f78ec06d9d0 2024-04-25T09:15:46.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.104+0000 7f78f0c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f78ec15e5f0 0x7f78ec0b4550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39154/0 (socket says 172.21.15.62:39154) 2024-04-25T09:15:46.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.104+0000 7f78f0c00640 1 -- 172.21.15.62:0/1308936479 learned_addr learned my addr 172.21.15.62:0/1308936479 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:46.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.104+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126929145 0 0) 0x7f78ec15ee30 con 0x7f78ec15e5f0 2024-04-25T09:15:46.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.104+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d0003520 con 0x7f78ec15e5f0 2024-04-25T09:15:46.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.104+0000 7f78e3e00640 1 --2- 172.21.15.62:0/1308936479 >> v2:172.21.15.121:3300/0 conn(0x7f78ec06d9d0 0x7f78ec15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.104+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3093330188 0 0) 0x7f78d0003520 con 0x7f78ec15e5f0 2024-04-25T09:15:46.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 >> v2:172.21.15.121:3300/0 conn(0x7f78ec06d9d0 msgr2=0x7f78ec15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:46.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 --2- 172.21.15.62:0/1308936479 >> v2:172.21.15.121:3300/0 conn(0x7f78ec06d9d0 0x7f78ec15e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 >> v1:172.21.15.121:6789/0 conn(0x7f78ec0ceb50 legacy=0x7f78ec06d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec15ea40 con 0x7f78ec15e5f0 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f78e40019f0 con 0x7f78ec15e5f0 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 >> v1:172.21.15.62:6789/0 conn(0x7f78ec15e5f0 legacy=0x7f78ec0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d00039b0 0x7f78d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78d0004800 0x7f78d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00052e0 0x7f78d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec161b90 con 0x7f78d00039b0 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d000c1b0 con 0x7f78d00052e0 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d000c3b0 con 0x7f78d0004800 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78ebe00640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d00039b0 0x7f78d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78ebe00640 1 -- 172.21.15.62:0/1308936479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78d0004800 msgr2=0x7f78d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:46.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78ebe00640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78d0004800 0x7f78d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78ebe00640 1 -- 172.21.15.62:0/1308936479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00052e0 msgr2=0x7f78d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:46.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78ebe00640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00052e0 0x7f78d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78ebe00640 1 -- 172.21.15.62:0/1308936479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0003520 con 0x7f78d00039b0 2024-04-25T09:15:46.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.105+0000 7f78f0c00640 1 -- 172.21.15.62:0/1308936479 reap_dead start 2024-04-25T09:15:46.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.106+0000 7f78ebe00640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d00039b0 0x7f78d0003dd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f78dc001720 tx=0x7f78d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827624552ac050ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:46.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.107+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc001370 con 0x7f78d00039b0 2024-04-25T09:15:46.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.107+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc0052d0 con 0x7f78d00039b0 2024-04-25T09:15:46.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.107+0000 7f78eb400640 1 -- 172.21.15.62:0/1308936479 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc02abc0 con 0x7f78d00039b0 2024-04-25T09:15:46.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.107+0000 7f78f1600640 1 -- 172.21.15.62:0/1308936479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d00039b0 msgr2=0x7f78d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.107+0000 7f78f1600640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d00039b0 0x7f78d0003dd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f78dc001720 tx=0x7f78d0005830 comp rx=0 tx=0).stop 2024-04-25T09:15:46.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.107+0000 7f78f1600640 1 -- 172.21.15.62:0/1308936479 shutdown_connections 2024-04-25T09:15:46.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.107+0000 7f78f1600640 1 --2- 172.21.15.62:0/1308936479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78d00039b0 0x7f78d0003dd0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.107+0000 7f78f1600640 1 -- 172.21.15.62:0/1308936479 >> 172.21.15.62:0/1308936479 conn(0x7f78ec066f20 msgr2=0x7f78ec0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:46.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.108+0000 7f78f1600640 1 -- 172.21.15.62:0/1308936479 shutdown_connections 2024-04-25T09:15:46.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.108+0000 7f78f1600640 1 -- 172.21.15.62:0/1308936479 wait complete. 2024-04-25T09:15:46.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.109+0000 7f78f1600640 1 Processor -- start 2024-04-25T09:15:46.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.109+0000 7f78f1600640 1 -- start start 2024-04-25T09:15:46.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.110+0000 7f78f1600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00039b0 0x7f78ec13d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.110+0000 7f78f1600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78ec140140 0x7f78ec070d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.110+0000 7f78f1600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78ec15d7d0 0x7f78ec071250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.110+0000 7f78f1600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec162510 con 0x7f78ec140140 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.110+0000 7f78f1600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec162390 con 0x7f78d00039b0 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.110+0000 7f78f1600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec15e0b0 con 0x7f78ec15d7d0 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78ec140140 0x7f78ec070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78e3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78ec140140 0x7f78ec070d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41060/0 (socket says 172.21.15.62:41060) 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78e3e00640 1 -- 172.21.15.62:0/3084681387 learned_addr learned my addr 172.21.15.62:0/3084681387 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78ebe00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00039b0 0x7f78ec13d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78f0c00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78ec15d7d0 0x7f78ec071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78e3e00640 1 -- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78ec15d7d0 msgr2=0x7f78ec071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78e3e00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78ec15d7d0 0x7f78ec071250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78e3e00640 1 -- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00039b0 msgr2=0x7f78ec13d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78e3e00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00039b0 0x7f78ec13d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78e3e00640 1 -- 172.21.15.62:0/3084681387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec1406c0 con 0x7f78ec140140 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78f0c00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78ec15d7d0 0x7f78ec071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.111+0000 7f78ebe00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00039b0 0x7f78ec13d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.112+0000 7f78e3e00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78ec140140 0x7f78ec070d10 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f78d400d560 tx=0x7f78d400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.112+0000 7f78e8c00640 1 -- 172.21.15.62:0/3084681387 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d40125f0 con 0x7f78ec140140 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.112+0000 7f78e8c00640 1 -- 172.21.15.62:0/3084681387 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d4019070 con 0x7f78ec140140 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.112+0000 7f78e8c00640 1 -- 172.21.15.62:0/3084681387 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d40129d0 con 0x7f78ec140140 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.112+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ec1409a0 con 0x7f78ec140140 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.112+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ec140f60 con 0x7f78ec140140 2024-04-25T09:15:46.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.113+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ec06abe0 con 0x7f78ec140140 2024-04-25T09:15:46.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.114+0000 7f78e8c00640 1 -- 172.21.15.62:0/3084681387 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78d4012b70 con 0x7f78ec140140 2024-04-25T09:15:46.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.115+0000 7f78e8c00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78c806b500 0x7f78c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.115+0000 7f78e8c00640 1 -- 172.21.15.62:0/3084681387 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f78d408a490 con 0x7f78ec140140 2024-04-25T09:15:46.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.116+0000 7f78ebe00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78c806b500 0x7f78c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.117+0000 7f78ebe00640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78c806b500 0x7f78c806d9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f78dc032000 tx=0x7f78dc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:46.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.119+0000 7f78e8c00640 1 -- 172.21.15.62:0/3084681387 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78d40592c0 con 0x7f78ec140140 2024-04-25T09:15:46.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.316+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78ec1456d0 con 0x7f78ec140140 2024-04-25T09:15:46.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.317+0000 7f78e8c00640 1 -- 172.21.15.62:0/3084681387 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78d4058bc0 con 0x7f78ec140140 2024-04-25T09:15:46.304 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:15:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78c806b500 msgr2=0x7f78c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78c806b500 0x7f78c806d9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f78dc032000 tx=0x7f78dc02e000 comp rx=0 tx=0).stop 2024-04-25T09:15:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78ec140140 msgr2=0x7f78ec070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78ec140140 0x7f78ec070d10 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f78d400d560 tx=0x7f78d400da20 comp rx=0 tx=0).stop 2024-04-25T09:15:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 shutdown_connections 2024-04-25T09:15:46.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f78c806b500 0x7f78c806d9c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78ec140140 0x7f78ec070d10 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78ec15d7d0 0x7f78ec071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 --2- 172.21.15.62:0/3084681387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78d00039b0 0x7f78ec13d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 >> 172.21.15.62:0/3084681387 conn(0x7f78ec066f20 msgr2=0x7f78ec15f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 shutdown_connections 2024-04-25T09:15:46.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.320+0000 7f78f1600640 1 -- 172.21.15.62:0/3084681387 wait complete. 2024-04-25T09:15:46.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:46.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.540+0000 7fb012800640 1 Processor -- start 2024-04-25T09:15:46.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.540+0000 7fb012800640 1 -- start start 2024-04-25T09:15:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.541+0000 7fb012800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b6a90 0x7fb00c0d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.541+0000 7fb012800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb00c0b27a0 0x7fb00c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.541+0000 7fb012800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c0d3110 con 0x7fb00c0b56f0 2024-04-25T09:15:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.542+0000 7fb012800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c0cedc0 con 0x7fb00c0b27a0 2024-04-25T09:15:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.542+0000 7fb012800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0d33f0 con 0x7fb00c0b6a90 2024-04-25T09:15:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.542+0000 7fb010a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b6a90 0x7fb00c0d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.542+0000 7fb010a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b6a90 0x7fb00c0d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41380/0 (socket says 172.21.15.62:41380) 2024-04-25T09:15:46.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.542+0000 7fb010a00640 1 -- 172.21.15.62:0/3518076867 learned_addr learned my addr 172.21.15.62:0/3518076867 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:46.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.542+0000 7fb011e00640 1 --2- 172.21.15.62:0/3518076867 >> v2:172.21.15.121:3300/0 conn(0x7fb00c0b27a0 0x7fb00c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.543+0000 7fb011e00640 1 -- 172.21.15.62:0/3518076867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b6a90 msgr2=0x7fb00c0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:46.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.543+0000 7fb011e00640 1 --2- 172.21.15.62:0/3518076867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b6a90 0x7fb00c0d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.543+0000 7fb011e00640 1 -- 172.21.15.62:0/3518076867 >> v1:172.21.15.121:6789/0 conn(0x7fb00c0b56f0 legacy=0x7fb00c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.543+0000 7fb011e00640 1 -- 172.21.15.62:0/3518076867 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0d36d0 con 0x7fb00c0b27a0 2024-04-25T09:15:46.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.543+0000 7fb00be00640 1 -- 172.21.15.62:0/3518076867 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420246216 0 0) 0x7fb00c0d3110 con 0x7fb00c0b56f0 2024-04-25T09:15:46.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.543+0000 7fb010a00640 1 --2- 172.21.15.62:0/3518076867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b6a90 0x7fb00c0d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:15:46.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.545+0000 7fb011e00640 1 --2- 172.21.15.62:0/3518076867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 0x7fb00c0d2520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb004009d90 tx=0x7fb00402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2d6433e165b92ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:46.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.546+0000 7fb00be00640 1 -- 172.21.15.62:0/3518076867 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004007740 con 0x7fb00c0b27a0 2024-04-25T09:15:46.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.546+0000 7fb00be00640 1 -- 172.21.15.62:0/3518076867 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0040078e0 con 0x7fb00c0b27a0 2024-04-25T09:15:46.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.546+0000 7fb00be00640 1 -- 172.21.15.62:0/3518076867 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004007cb0 con 0x7fb00c0b27a0 2024-04-25T09:15:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.546+0000 7fb012800640 1 -- 172.21.15.62:0/3518076867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 msgr2=0x7fb00c0d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.546+0000 7fb012800640 1 --2- 172.21.15.62:0/3518076867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 0x7fb00c0d2520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb004009d90 tx=0x7fb00402a660 comp rx=0 tx=0).stop 2024-04-25T09:15:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.547+0000 7fb012800640 1 -- 172.21.15.62:0/3518076867 shutdown_connections 2024-04-25T09:15:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.547+0000 7fb012800640 1 --2- 172.21.15.62:0/3518076867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 0x7fb00c0d2520 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.547+0000 7fb012800640 1 --2- 172.21.15.62:0/3518076867 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b6a90 0x7fb00c0d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.547+0000 7fb012800640 1 -- 172.21.15.62:0/3518076867 >> 172.21.15.62:0/3518076867 conn(0x7fb00c066d10 msgr2=0x7fb00c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.547+0000 7fb012800640 1 -- 172.21.15.62:0/3518076867 shutdown_connections 2024-04-25T09:15:46.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.547+0000 7fb012800640 1 -- 172.21.15.62:0/3518076867 wait complete. 2024-04-25T09:15:46.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.548+0000 7fb012800640 1 Processor -- start 2024-04-25T09:15:46.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.549+0000 7fb012800640 1 -- start start 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.549+0000 7fb012800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 0x7fb00c0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.549+0000 7fb012800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b56f0 0x7fb00c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.549+0000 7fb012800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00c0b6a90 0x7fb00c146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.549+0000 7fb012800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0b2ee0 con 0x7fb00c0b6a90 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.549+0000 7fb012800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0b2d60 con 0x7fb00c0b27a0 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.549+0000 7fb012800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0b5ad0 con 0x7fb00c0b56f0 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00c0b6a90 0x7fb00c146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00c0b6a90 0x7fb00c146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41066/0 (socket says 172.21.15.62:41066) 2024-04-25T09:15:46.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011e00640 1 -- 172.21.15.62:0/1665712513 learned_addr learned my addr 172.21.15.62:0/1665712513 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011e00640 1 -- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b56f0 msgr2=0x7fb00c0c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb010a00640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b56f0 0x7fb00c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011400640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 0x7fb00c0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011e00640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b56f0 0x7fb00c0c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011e00640 1 -- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 msgr2=0x7fb00c0cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011e00640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 0x7fb00c0cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb011e00640 1 -- 172.21.15.62:0/1665712513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c146ff0 con 0x7fb00c0b6a90 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.550+0000 7fb010a00640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b56f0 0x7fb00c0c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.551+0000 7fb011e00640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00c0b6a90 0x7fb00c146550 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb00402ac50 tx=0x7fb004004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.551+0000 7fb009600640 1 -- 172.21.15.62:0/1665712513 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004040070 con 0x7fb00c0b6a90 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.551+0000 7fb009600640 1 -- 172.21.15.62:0/1665712513 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb004031bc0 con 0x7fb00c0b6a90 2024-04-25T09:15:46.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.551+0000 7fb009600640 1 -- 172.21.15.62:0/1665712513 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004009450 con 0x7fb00c0b6a90 2024-04-25T09:15:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.551+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c1472d0 con 0x7fb00c0b6a90 2024-04-25T09:15:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.551+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c0ccda0 con 0x7fb00c0b6a90 2024-04-25T09:15:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.552+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c0cc8b0 con 0x7fb00c0b6a90 2024-04-25T09:15:46.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.553+0000 7fb009600640 1 -- 172.21.15.62:0/1665712513 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb004039040 con 0x7fb00c0b6a90 2024-04-25T09:15:46.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.554+0000 7fb009600640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafec06b4d0 0x7fafec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:46.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.554+0000 7fb009600640 1 -- 172.21.15.62:0/1665712513 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb0040abb00 con 0x7fb00c0b6a90 2024-04-25T09:15:46.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.555+0000 7fb011400640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafec06b4d0 0x7fafec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:46.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.556+0000 7fb011400640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafec06b4d0 0x7fafec06d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb000000f60 tx=0x7fb000009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:46.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.559+0000 7fb009600640 1 -- 172.21.15.62:0/1665712513 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb00407a930 con 0x7fb00c0b6a90 2024-04-25T09:15:46.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.802+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb00c0ccf70 con 0x7fb00c0b6a90 2024-04-25T09:15:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.803+0000 7fb009600640 1 -- 172.21.15.62:0/1665712513 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb00407a230 con 0x7fb00c0b6a90 2024-04-25T09:15:46.790 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:46.790 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:46.790 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafec06b4d0 msgr2=0x7fafec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafec06b4d0 0x7fafec06d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb000000f60 tx=0x7fb000009040 comp rx=0 tx=0).stop 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00c0b6a90 msgr2=0x7fb00c146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00c0b6a90 0x7fb00c146550 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb00402ac50 tx=0x7fb004004a40 comp rx=0 tx=0).stop 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 shutdown_connections 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fafec06b4d0 0x7fafec06d990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb00c0b6a90 0x7fb00c146550 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb00c0b56f0 0x7fb00c0c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 --2- 172.21.15.62:0/1665712513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb00c0b27a0 0x7fb00c0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 >> 172.21.15.62:0/1665712513 conn(0x7fb00c066d10 msgr2=0x7fb00c0d2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.805+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 shutdown_connections 2024-04-25T09:15:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:46.806+0000 7fb012800640 1 -- 172.21.15.62:0/1665712513 wait complete. 2024-04-25T09:15:46.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:15:47.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.031+0000 7f0ffe600640 1 Processor -- start 2024-04-25T09:15:47.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.031+0000 7f0ffe600640 1 -- start start 2024-04-25T09:15:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.032+0000 7f0ffe600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0ff8144c10 0x7f0ff815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.032+0000 7f0ffe600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ff80cee40 0x7f0ff806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.032+0000 7f0ffe600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff806d730 0x7f0ff80b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.032+0000 7f0ffe600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff8144ff0 con 0x7f0ff80cee40 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.032+0000 7f0ffe600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff8160340 con 0x7f0ff8144c10 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.032+0000 7f0ffe600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80b41c0 con 0x7f0ff806d730 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ff7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ff80cee40 0x7f0ff806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ff7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ff80cee40 0x7f0ff806d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41076/0 (socket says 172.21.15.62:41076) 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ff7e00640 1 -- 172.21.15.62:0/421750346 learned_addr learned my addr 172.21.15.62:0/421750346 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ffdc00640 1 --2- 172.21.15.62:0/421750346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff806d730 0x7f0ff80b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ffd200640 1 --2- 172.21.15.62:0/421750346 >> v2:172.21.15.121:3300/0 conn(0x7f0ff8144c10 0x7f0ff815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ff7e00640 1 -- 172.21.15.62:0/421750346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff806d730 msgr2=0x7f0ff80b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ff7e00640 1 --2- 172.21.15.62:0/421750346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff806d730 0x7f0ff80b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ff7e00640 1 -- 172.21.15.62:0/421750346 >> v2:172.21.15.121:3300/0 conn(0x7f0ff8144c10 msgr2=0x7f0ff815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ff7e00640 1 --2- 172.21.15.62:0/421750346 >> v2:172.21.15.121:3300/0 conn(0x7f0ff8144c10 0x7f0ff815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.033+0000 7f0ff7e00640 1 -- 172.21.15.62:0/421750346 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80cf2d0 con 0x7f0ff80cee40 2024-04-25T09:15:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.035+0000 7f0ff7e00640 1 --2- 172.21.15.62:0/421750346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff80cee40 0x7f0ff806d1f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80724a0 tx=0x7f0fe802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367180b9d1af1ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.035+0000 7f0ff7400640 1 -- 172.21.15.62:0/421750346 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8037070 con 0x7f0ff80cee40 2024-04-25T09:15:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.035+0000 7f0ff7400640 1 -- 172.21.15.62:0/421750346 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe8032440 con 0x7f0ff80cee40 2024-04-25T09:15:47.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.035+0000 7f0ff7400640 1 -- 172.21.15.62:0/421750346 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8032810 con 0x7f0ff80cee40 2024-04-25T09:15:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.035+0000 7f0ffe600640 1 -- 172.21.15.62:0/421750346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff80cee40 msgr2=0x7f0ff806d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.035+0000 7f0ffe600640 1 --2- 172.21.15.62:0/421750346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff80cee40 0x7f0ff806d1f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80724a0 tx=0x7f0fe802a140 comp rx=0 tx=0).stop 2024-04-25T09:15:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.036+0000 7f0ffe600640 1 -- 172.21.15.62:0/421750346 shutdown_connections 2024-04-25T09:15:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.036+0000 7f0ffe600640 1 --2- 172.21.15.62:0/421750346 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff806d730 0x7f0ff80b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.036+0000 7f0ffe600640 1 --2- 172.21.15.62:0/421750346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff80cee40 0x7f0ff806d1f0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.036+0000 7f0ffe600640 1 --2- 172.21.15.62:0/421750346 >> v2:172.21.15.121:3300/0 conn(0x7f0ff8144c10 0x7f0ff815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.036+0000 7f0ffe600640 1 -- 172.21.15.62:0/421750346 >> 172.21.15.62:0/421750346 conn(0x7f0ff80b14d0 msgr2=0x7f0ff80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:47.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.036+0000 7f0ffe600640 1 -- 172.21.15.62:0/421750346 shutdown_connections 2024-04-25T09:15:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.036+0000 7f0ffe600640 1 -- 172.21.15.62:0/421750346 wait complete. 2024-04-25T09:15:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.037+0000 7f0ffe600640 1 Processor -- start 2024-04-25T09:15:47.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.038+0000 7f0ffe600640 1 -- start start 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.038+0000 7f0ffe600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ff806d730 0x7f0ff80b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.038+0000 7f0ffe600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff80cee40 0x7f0ff815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.038+0000 7f0ffe600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff8144c10 0x7f0ff815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.038+0000 7f0ffe600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8160060 con 0x7f0ff8144c10 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.038+0000 7f0ffe600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff815fd80 con 0x7f0ff806d730 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.038+0000 7f0ffe600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80b32e0 con 0x7f0ff80cee40 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffdc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff8144c10 0x7f0ff815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffdc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff8144c10 0x7f0ff815b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41080/0 (socket says 172.21.15.62:41080) 2024-04-25T09:15:47.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffdc00640 1 -- 172.21.15.62:0/3214186520 learned_addr learned my addr 172.21.15.62:0/3214186520 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:47.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffdc00640 1 -- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff80cee40 msgr2=0x7f0ff815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffdc00640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff80cee40 0x7f0ff815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffd200640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ff806d730 0x7f0ff80b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffdc00640 1 -- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ff806d730 msgr2=0x7f0ff80b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffdc00640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ff806d730 0x7f0ff80b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.039+0000 7f0ffdc00640 1 -- 172.21.15.62:0/3214186520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8143ab0 con 0x7f0ff8144c10 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.040+0000 7f0ffdc00640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff8144c10 0x7f0ff815b3d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0007e60 tx=0x7f0ff000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.040+0000 7f0ff4c00640 1 -- 172.21.15.62:0/3214186520 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0014620 con 0x7f0ff8144c10 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.040+0000 7f0ff4c00640 1 -- 172.21.15.62:0/3214186520 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff0019070 con 0x7f0ff8144c10 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.040+0000 7f0ff4c00640 1 -- 172.21.15.62:0/3214186520 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0014a20 con 0x7f0ff8144c10 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.040+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff8143d90 con 0x7f0ff8144c10 2024-04-25T09:15:47.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.040+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff8144350 con 0x7f0ff8144c10 2024-04-25T09:15:47.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.041+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff804ef20 con 0x7f0ff8144c10 2024-04-25T09:15:47.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.042+0000 7f0ff4c00640 1 -- 172.21.15.62:0/3214186520 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ff0014bc0 con 0x7f0ff8144c10 2024-04-25T09:15:47.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.043+0000 7f0ff4c00640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fdc06b4d0 0x7f0fdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.043+0000 7f0ff4c00640 1 -- 172.21.15.62:0/3214186520 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0ff008aba0 con 0x7f0ff8144c10 2024-04-25T09:15:47.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.044+0000 7f0ffd200640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fdc06b4d0 0x7f0fdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.045+0000 7f0ffd200640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fdc06b4d0 0x7f0fdc06d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00b7c0 tx=0x7f0fec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.047+0000 7f0ff4c00640 1 -- 172.21.15.62:0/3214186520 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ff0059940 con 0x7f0ff8144c10 2024-04-25T09:15:47.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.245+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff80b1950 con 0x7f0ff8144c10 2024-04-25T09:15:47.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.249+0000 7f0ff4c00640 1 -- 172.21.15.62:0/3214186520 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16756 (secure 0 0 0) 0x7f0ff0059240 con 0x7f0ff8144c10 2024-04-25T09:15:47.236 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:47.236 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":125,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:15:03.711047+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:14:44.925556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:14:46.076732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.8","mappings":[{"from":4,"to":3}]},{"pgid":"17.1e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:15:47.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fdc06b4d0 msgr2=0x7f0fdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fdc06b4d0 0x7f0fdc06d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00b7c0 tx=0x7f0fec008040 comp rx=0 tx=0).stop 2024-04-25T09:15:47.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff8144c10 msgr2=0x7f0ff815b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff8144c10 0x7f0ff815b3d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0007e60 tx=0x7f0ff000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 shutdown_connections 2024-04-25T09:15:47.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0fdc06b4d0 0x7f0fdc06d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ff8144c10 0x7f0ff815b3d0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ff80cee40 0x7f0ff815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 --2- 172.21.15.62:0/3214186520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ff806d730 0x7f0ff80b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 >> 172.21.15.62:0/3214186520 conn(0x7f0ff80b14d0 msgr2=0x7f0ff80b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:47.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 shutdown_connections 2024-04-25T09:15:47.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.252+0000 7f0ffe600640 1 -- 172.21.15.62:0/3214186520 wait complete. 2024-04-25T09:15:47.259 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:15:47.259 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.485+0000 7f2365800640 1 Processor -- start 2024-04-25T09:15:47.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.485+0000 7f2365800640 1 -- start start 2024-04-25T09:15:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.486+0000 7f2365800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236006d900 0x7f236015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.486+0000 7f2365800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23600cf2d0 con 0x7f23600cee40 2024-04-25T09:15:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.486+0000 7f2365800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236015faa0 con 0x7f2360144c10 2024-04-25T09:15:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.486+0000 7f2365800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2360144ff0 con 0x7f236006d900 2024-04-25T09:15:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.487+0000 7f2364e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f23600cee40 0x7f23600b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39162/0 (socket says 172.21.15.62:39162) 2024-04-25T09:15:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.487+0000 7f2364e00640 1 -- 172.21.15.62:0/4112818385 learned_addr learned my addr 172.21.15.62:0/4112818385 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:47.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.487+0000 7f2357e00640 1 --2- 172.21.15.62:0/4112818385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236006d900 0x7f236015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.487+0000 7f235f400640 1 -- 172.21.15.62:0/4112818385 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159127463 0 0) 0x7f23600cf2d0 con 0x7f23600cee40 2024-04-25T09:15:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.487+0000 7f235f400640 1 -- 172.21.15.62:0/4112818385 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2344003520 con 0x7f23600cee40 2024-04-25T09:15:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.488+0000 7f235f400640 1 -- 172.21.15.62:0/4112818385 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582730861 0 0) 0x7f236015faa0 con 0x7f2360144c10 2024-04-25T09:15:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.488+0000 7f235f400640 1 -- 172.21.15.62:0/4112818385 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23600cf2d0 con 0x7f2360144c10 2024-04-25T09:15:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.488+0000 7f2357e00640 1 -- 172.21.15.62:0/4112818385 >> v1:172.21.15.121:6789/0 conn(0x7f2360144c10 legacy=0x7f236006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.488+0000 7f2357e00640 1 -- 172.21.15.62:0/4112818385 >> v1:172.21.15.62:6789/0 conn(0x7f23600cee40 legacy=0x7f23600b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.488+0000 7f2357e00640 1 -- 172.21.15.62:0/4112818385 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236015fd80 con 0x7f236006d900 2024-04-25T09:15:47.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.489+0000 7f2357e00640 1 --2- 172.21.15.62:0/4112818385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236006d900 0x7f236015f450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23480097d0 tx=0x7f234802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c5ee4c422b7eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.490+0000 7f235f400640 1 -- 172.21.15.62:0/4112818385 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348037070 con 0x7f236006d900 2024-04-25T09:15:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.490+0000 7f235f400640 1 -- 172.21.15.62:0/4112818385 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348032440 con 0x7f236006d900 2024-04-25T09:15:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.490+0000 7f235f400640 1 -- 172.21.15.62:0/4112818385 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23480327f0 con 0x7f236006d900 2024-04-25T09:15:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.490+0000 7f2365800640 1 -- 172.21.15.62:0/4112818385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236006d900 msgr2=0x7f236015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.490+0000 7f2365800640 1 --2- 172.21.15.62:0/4112818385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236006d900 0x7f236015f450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23480097d0 tx=0x7f234802a180 comp rx=0 tx=0).stop 2024-04-25T09:15:47.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.491+0000 7f2365800640 1 -- 172.21.15.62:0/4112818385 shutdown_connections 2024-04-25T09:15:47.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.491+0000 7f2365800640 1 --2- 172.21.15.62:0/4112818385 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f236006d900 0x7f236015f450 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.491+0000 7f2365800640 1 -- 172.21.15.62:0/4112818385 >> 172.21.15.62:0/4112818385 conn(0x7f23600b14d0 msgr2=0x7f23600b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:47.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.491+0000 7f2365800640 1 -- 172.21.15.62:0/4112818385 shutdown_connections 2024-04-25T09:15:47.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.491+0000 7f2365800640 1 -- 172.21.15.62:0/4112818385 wait complete. 2024-04-25T09:15:47.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.493+0000 7f2365800640 1 Processor -- start 2024-04-25T09:15:47.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.493+0000 7f2365800640 1 -- start start 2024-04-25T09:15:47.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f2365800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236006d900 0x7f23601e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f2365800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600cee40 0x7f23601e6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f2365800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2360144c10 0x7f23601e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f2365800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600b41c0 con 0x7f236006d900 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f2365800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2360160340 con 0x7f23600cee40 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f2365800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600b4d60 con 0x7f2360144c10 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f235fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236006d900 0x7f23601e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f235fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236006d900 0x7f23601e60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41090/0 (socket says 172.21.15.62:41090) 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f235fe00640 1 -- 172.21.15.62:0/2945333938 learned_addr learned my addr 172.21.15.62:0/2945333938 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f2364e00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2360144c10 0x7f23601e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f235fe00640 1 -- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2360144c10 msgr2=0x7f23601e6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f235fe00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2360144c10 0x7f23601e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f2357e00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600cee40 0x7f23601e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.494+0000 7f235fe00640 1 -- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600cee40 msgr2=0x7f23601e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f235fe00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600cee40 0x7f23601e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f235fe00640 1 -- 172.21.15.62:0/2945333938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23601e5330 con 0x7f236006d900 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f2357e00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600cee40 0x7f23601e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f235fe00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236006d900 0x7f23601e60f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f235000c430 tx=0x7f235000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f235cc00640 1 -- 172.21.15.62:0/2945333938 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350017070 con 0x7f236006d900 2024-04-25T09:15:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f235cc00640 1 -- 172.21.15.62:0/2945333938 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23500052f0 con 0x7f236006d900 2024-04-25T09:15:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23601e5610 con 0x7f236006d900 2024-04-25T09:15:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f235cc00640 1 -- 172.21.15.62:0/2945333938 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350003830 con 0x7f236006d900 2024-04-25T09:15:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.495+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23601e5bd0 con 0x7f236006d900 2024-04-25T09:15:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.496+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236004ef20 con 0x7f236006d900 2024-04-25T09:15:47.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.498+0000 7f235cc00640 1 -- 172.21.15.62:0/2945333938 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f235001a070 con 0x7f236006d900 2024-04-25T09:15:47.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.498+0000 7f235cc00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.499+0000 7f235cc00640 1 -- 172.21.15.62:0/2945333938 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2350089570 con 0x7f236006d900 2024-04-25T09:15:47.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.499+0000 7f2357e00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.500+0000 7f2357e00640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f233c06b4d0 0x7f233c06d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2348034000 tx=0x7f234802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.503+0000 7f235cc00640 1 -- 172.21.15.62:0/2945333938 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2350058440 con 0x7f236006d900 2024-04-25T09:15:47.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.752+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23600b22b0 con 0x7f236006d900 2024-04-25T09:15:47.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.753+0000 7f235cc00640 1 -- 172.21.15.62:0/2945333938 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f2350057d40 con 0x7f236006d900 2024-04-25T09:15:47.740 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:47.740 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:47.741 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:47.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f233c06b4d0 msgr2=0x7f233c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f233c06b4d0 0x7f233c06d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2348034000 tx=0x7f234802e000 comp rx=0 tx=0).stop 2024-04-25T09:15:47.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236006d900 msgr2=0x7f23601e60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236006d900 0x7f23601e60f0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f235000c430 tx=0x7f235000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 shutdown_connections 2024-04-25T09:15:47.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2360144c10 0x7f23601e6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23600cee40 0x7f23601e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 --2- 172.21.15.62:0/2945333938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f236006d900 0x7f23601e60f0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 >> 172.21.15.62:0/2945333938 conn(0x7f23600b14d0 msgr2=0x7f23600ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:47.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 shutdown_connections 2024-04-25T09:15:47.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.756+0000 7f2365800640 1 -- 172.21.15.62:0/2945333938 wait complete. 2024-04-25T09:15:47.763 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:47.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.992+0000 7f9fc7200640 1 Processor -- start 2024-04-25T09:15:47.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.993+0000 7f9fc7200640 1 -- start start 2024-04-25T09:15:47.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9fc80b56f0 0x7f9fc8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc81627a0 con 0x7f9fc80b6a90 2024-04-25T09:15:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc80d3110 con 0x7f9fc80b27a0 2024-04-25T09:15:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc80cedc0 con 0x7f9fc80b56f0 2024-04-25T09:15:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9fc80b56f0 0x7f9fc8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9fc80b56f0 0x7f9fc8160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41102/0 (socket says 172.21.15.62:41102) 2024-04-25T09:15:47.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc5e00640 1 -- 172.21.15.62:0/3169690760 learned_addr learned my addr 172.21.15.62:0/3169690760 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:47.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc5e00640 1 -- 172.21.15.62:0/3169690760 >> v1:172.21.15.121:6789/0 conn(0x7f9fc80b27a0 legacy=0x7f9fc8162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:47.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc5e00640 1 -- 172.21.15.62:0/3169690760 >> v1:172.21.15.62:6789/0 conn(0x7f9fc80b6a90 legacy=0x7f9fc8161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc5e00640 1 -- 172.21.15.62:0/3169690760 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80d33f0 con 0x7f9fc80b56f0 2024-04-25T09:15:47.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.994+0000 7f9fc5400640 1 -- 172.21.15.62:0/3169690760 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493955355 0 0) 0x7f9fc81627a0 con 0x7f9fc80b6a90 2024-04-25T09:15:47.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.995+0000 7f9fc5e00640 1 --2- 172.21.15.62:0/3169690760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc8160d00 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009870 tx=0x7f9fb802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837d573e25e24457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.995+0000 7f9fc5400640 1 -- 172.21.15.62:0/3169690760 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8037070 con 0x7f9fc80b56f0 2024-04-25T09:15:47.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.995+0000 7f9fc5400640 1 -- 172.21.15.62:0/3169690760 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb8032440 con 0x7f9fc80b56f0 2024-04-25T09:15:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.995+0000 7f9fc5400640 1 -- 172.21.15.62:0/3169690760 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8032810 con 0x7f9fc80b56f0 2024-04-25T09:15:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.996+0000 7f9fc7200640 1 -- 172.21.15.62:0/3169690760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 msgr2=0x7f9fc8160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.996+0000 7f9fc7200640 1 --2- 172.21.15.62:0/3169690760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc8160d00 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009870 tx=0x7f9fb802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.996+0000 7f9fc7200640 1 -- 172.21.15.62:0/3169690760 shutdown_connections 2024-04-25T09:15:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.996+0000 7f9fc7200640 1 --2- 172.21.15.62:0/3169690760 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc8160d00 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.996+0000 7f9fc7200640 1 -- 172.21.15.62:0/3169690760 >> 172.21.15.62:0/3169690760 conn(0x7f9fc8066d10 msgr2=0x7f9fc8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.996+0000 7f9fc7200640 1 -- 172.21.15.62:0/3169690760 shutdown_connections 2024-04-25T09:15:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.996+0000 7f9fc7200640 1 -- 172.21.15.62:0/3169690760 wait complete. 2024-04-25T09:15:47.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.998+0000 7f9fc7200640 1 Processor -- start 2024-04-25T09:15:47.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.998+0000 7f9fc7200640 1 -- start start 2024-04-25T09:15:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fc7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80b27a0 0x7f9fc814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fc7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fc7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80b6a90 0x7f9fc80ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fc7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80b2d60 con 0x7f9fc80b56f0 2024-04-25T09:15:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fc7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80cea50 con 0x7f9fc80b27a0 2024-04-25T09:15:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fc7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc80b2ee0 con 0x7f9fc80b6a90 2024-04-25T09:15:47.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc80ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41112/0 (socket says 172.21.15.62:41112) 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:47.999+0000 7f9fbfe00640 1 -- 172.21.15.62:0/1935217500 learned_addr learned my addr 172.21.15.62:0/1935217500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fc5e00640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80b27a0 0x7f9fc814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fc6800640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80b6a90 0x7f9fc80ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fbfe00640 1 -- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80b6a90 msgr2=0x7f9fc80ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fbfe00640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80b6a90 0x7f9fc80ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fbfe00640 1 -- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80b27a0 msgr2=0x7f9fc814fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fbfe00640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80b27a0 0x7f9fc814fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fbfe00640 1 -- 172.21.15.62:0/1935217500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80cd590 con 0x7f9fc80b56f0 2024-04-25T09:15:47.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fbfe00640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc80ccad0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000c410 tx=0x7f9fb000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.000+0000 7f9fbe000640 1 -- 172.21.15.62:0/1935217500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0017070 con 0x7f9fc80b56f0 2024-04-25T09:15:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.001+0000 7f9fbe000640 1 -- 172.21.15.62:0/1935217500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb00035f0 con 0x7f9fc80b56f0 2024-04-25T09:15:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.001+0000 7f9fbe000640 1 -- 172.21.15.62:0/1935217500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0003a00 con 0x7f9fc80b56f0 2024-04-25T09:15:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.001+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc80cd870 con 0x7f9fc80b56f0 2024-04-25T09:15:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.001+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc8160390 con 0x7f9fc80b56f0 2024-04-25T09:15:47.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.002+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc80b85e0 con 0x7f9fc80b56f0 2024-04-25T09:15:47.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.003+0000 7f9fbe000640 1 -- 172.21.15.62:0/1935217500 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fb001a070 con 0x7f9fc80b56f0 2024-04-25T09:15:47.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.005+0000 7f9fbe000640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9fa406b4d0 0x7f9fa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:47.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.005+0000 7f9fbe000640 1 -- 172.21.15.62:0/1935217500 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9fb0089640 con 0x7f9fc80b56f0 2024-04-25T09:15:47.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.005+0000 7f9fc5e00640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9fa406b4d0 0x7f9fa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:47.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.007+0000 7f9fc5e00640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9fa406b4d0 0x7f9fa406d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8034000 tx=0x7f9fb802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:47.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.010+0000 7f9fbe000640 1 -- 172.21.15.62:0/1935217500 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fc80b85e0 con 0x7f9fc80b56f0 2024-04-25T09:15:48.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.252+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fc804ef20 con 0x7f9fc80b56f0 2024-04-25T09:15:48.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.254+0000 7f9fbe000640 1 -- 172.21.15.62:0/1935217500 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9fb0058470 con 0x7f9fc80b56f0 2024-04-25T09:15:48.241 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:48.241 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:14:47.039566+0000","modified":"2024-04-25T09:14:50.155038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2051823365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2051823365},{"type":"v1","addr":"172.21.15.62:6835","nonce":2051823365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:15:48.242 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 78 2024-04-25T09:15:48.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.255+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9fa406b4d0 msgr2=0x7f9fa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.255+0000 7f9fc7200640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9fa406b4d0 0x7f9fa406d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8034000 tx=0x7f9fb802e000 comp rx=0 tx=0).stop 2024-04-25T09:15:48.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.255+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 msgr2=0x7f9fc80ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.255+0000 7f9fc7200640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc80ccad0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000c410 tx=0x7f9fb000c8d0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.256+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 shutdown_connections 2024-04-25T09:15:48.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.256+0000 7f9fc7200640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9fa406b4d0 0x7f9fa406d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.256+0000 7f9fc7200640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9fc80b56f0 0x7f9fc80ccad0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.256+0000 7f9fc7200640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9fc80b6a90 0x7f9fc80ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.256+0000 7f9fc7200640 1 --2- 172.21.15.62:0/1935217500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9fc80b27a0 0x7f9fc814fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.256+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 >> 172.21.15.62:0/1935217500 conn(0x7f9fc8066d10 msgr2=0x7f9fc80b7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:48.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.256+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 shutdown_connections 2024-04-25T09:15:48.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.256+0000 7f9fc7200640 1 -- 172.21.15.62:0/1935217500 wait complete. 2024-04-25T09:15:48.265 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:15:48.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.487+0000 7f03d2600640 1 Processor -- start 2024-04-25T09:15:48.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.488+0000 7f03d2600640 1 -- start start 2024-04-25T09:15:48.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.489+0000 7f03d2600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0b56f0 0x7f03cc160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.489+0000 7f03d2600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc1627a0 con 0x7f03cc0b6a90 2024-04-25T09:15:48.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.489+0000 7f03d2600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc0d3110 con 0x7f03cc0b27a0 2024-04-25T09:15:48.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.489+0000 7f03d2600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0cedc0 con 0x7f03cc0b56f0 2024-04-25T09:15:48.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.490+0000 7f03cbe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f03cc0b6a90 0x7f03cc161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39188/0 (socket says 172.21.15.62:39188) 2024-04-25T09:15:48.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.490+0000 7f03cbe00640 1 -- 172.21.15.62:0/1813271590 learned_addr learned my addr 172.21.15.62:0/1813271590 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:48.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.490+0000 7f03d1200640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0b56f0 0x7f03cc160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.490+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860667853 0 0) 0x7f03cc1627a0 con 0x7f03cc0b6a90 2024-04-25T09:15:48.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.490+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b0003520 con 0x7f03cc0b6a90 2024-04-25T09:15:48.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4041891607 0 0) 0x7f03b0003520 con 0x7f03cc0b6a90 2024-04-25T09:15:48.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0b56f0 msgr2=0x7f03cc160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:48.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0b56f0 0x7f03cc160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 >> v1:172.21.15.121:6789/0 conn(0x7f03cc0b27a0 legacy=0x7f03cc162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc0d33f0 con 0x7f03cc0b6a90 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f03b4001670 con 0x7f03cc0b6a90 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 >> v1:172.21.15.62:6789/0 conn(0x7f03cc0b6a90 legacy=0x7f03cc161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b00039e0 0x7f03b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0b56f0 0x7f03b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0cedc0 con 0x7f03b0004d40 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b000ba70 con 0x7f03b00039e0 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b000bc70 con 0x7f03cc0b56f0 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.491+0000 7f03d1c00640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.492+0000 7f03d1200640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b00039e0 0x7f03b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.492+0000 7f03cbe00640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0b56f0 0x7f03b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.492+0000 7f03d1c00640 1 -- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0b56f0 msgr2=0x7f03b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:48.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.492+0000 7f03d1c00640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0b56f0 0x7f03b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.492+0000 7f03d1c00640 1 -- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b00039e0 msgr2=0x7f03b0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:48.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.492+0000 7f03d1c00640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03b00039e0 0x7f03b0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.492+0000 7f03d1c00640 1 -- 172.21.15.62:0/1813271590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b0003520 con 0x7f03b0004d40 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.493+0000 7f03d1c00640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03b00051c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f03c4001720 tx=0x7f03b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd5f615a8c8e02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.493+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4001370 con 0x7f03b0004d40 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.493+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c40052d0 con 0x7f03b0004d40 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.493+0000 7f03cb400640 1 -- 172.21.15.62:0/1813271590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c402abc0 con 0x7f03b0004d40 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.494+0000 7f03d2600640 1 -- 172.21.15.62:0/1813271590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 msgr2=0x7f03b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.494+0000 7f03d2600640 1 --2- 172.21.15.62:0/1813271590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03b00051c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f03c4001720 tx=0x7f03b0005830 comp rx=0 tx=0).stop 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.494+0000 7f03d1c00640 1 -- 172.21.15.62:0/1813271590 reap_dead start 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.494+0000 7f03d2600640 1 -- 172.21.15.62:0/1813271590 shutdown_connections 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.494+0000 7f03d2600640 1 -- 172.21.15.62:0/1813271590 >> 172.21.15.62:0/1813271590 conn(0x7f03cc066d10 msgr2=0x7f03cc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:48.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.494+0000 7f03d2600640 1 -- 172.21.15.62:0/1813271590 shutdown_connections 2024-04-25T09:15:48.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.494+0000 7f03d2600640 1 -- 172.21.15.62:0/1813271590 wait complete. 2024-04-25T09:15:48.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.496+0000 7f03d2600640 1 Processor -- start 2024-04-25T09:15:48.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.496+0000 7f03d2600640 1 -- start start 2024-04-25T09:15:48.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d2600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03cc159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d2600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03cc1e90b0 0x7f03cc1e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d2600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0c98a0 0x7f03cc0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d2600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0b2d60 con 0x7f03b0004d40 2024-04-25T09:15:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d2600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0cea50 con 0x7f03cc1e90b0 2024-04-25T09:15:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d2600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0b2ee0 con 0x7f03cc0c98a0 2024-04-25T09:15:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03cc159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03cc159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41134/0 (socket says 172.21.15.62:41134) 2024-04-25T09:15:48.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.497+0000 7f03d1200640 1 -- 172.21.15.62:0/761495980 learned_addr learned my addr 172.21.15.62:0/761495980 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:48.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03cbe00640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03cc1e90b0 0x7f03cc1e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d1c00640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0c98a0 0x7f03cc0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d1200640 1 -- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0c98a0 msgr2=0x7f03cc0c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d1200640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0c98a0 0x7f03cc0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d1200640 1 -- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03cc1e90b0 msgr2=0x7f03cc1e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d1200640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03cc1e90b0 0x7f03cc1e9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d1200640 1 -- 172.21.15.62:0/761495980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc1e9bc0 con 0x7f03b0004d40 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d1200640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03cc159d40 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f03bc007f90 tx=0x7f03bc011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03c8c00640 1 -- 172.21.15.62:0/761495980 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc01a020 con 0x7f03b0004d40 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03c8c00640 1 -- 172.21.15.62:0/761495980 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc016470 con 0x7f03b0004d40 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03c8c00640 1 -- 172.21.15.62:0/761495980 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc016860 con 0x7f03b0004d40 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc1e9ea0 con 0x7f03b0004d40 2024-04-25T09:15:48.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.498+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc0cba00 con 0x7f03b0004d40 2024-04-25T09:15:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.500+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc04ef20 con 0x7f03b0004d40 2024-04-25T09:15:48.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.501+0000 7f03c8c00640 1 -- 172.21.15.62:0/761495980 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03bc005d10 con 0x7f03b0004d40 2024-04-25T09:15:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.502+0000 7f03c8c00640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03a806b4d0 0x7f03a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.502+0000 7f03c8c00640 1 -- 172.21.15.62:0/761495980 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f03bc08b200 con 0x7f03b0004d40 2024-04-25T09:15:48.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.503+0000 7f03cbe00640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03a806b4d0 0x7f03a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.504+0000 7f03cbe00640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03a806b4d0 0x7f03a806d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001eb0 tx=0x7f03b4001010 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:48.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.506+0000 7f03c8c00640 1 -- 172.21.15.62:0/761495980 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03bc024030 con 0x7f03b0004d40 2024-04-25T09:15:48.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.706+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03cc1ea070 con 0x7f03b0004d40 2024-04-25T09:15:48.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.709+0000 7f03c8c00640 1 -- 172.21.15.62:0/761495980 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16756 (secure 0 0 0) 0x7f03bc059fa0 con 0x7f03b0004d40 2024-04-25T09:15:48.696 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:48.696 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":125,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:15:03.711047+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:14:44.925556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:14:46.076732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.8","mappings":[{"from":4,"to":3}]},{"pgid":"17.1e","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03a806b4d0 msgr2=0x7f03a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03a806b4d0 0x7f03a806d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001eb0 tx=0x7f03b4001010 comp rx=0 tx=0).stop 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 msgr2=0x7f03cc159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03cc159d40 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f03bc007f90 tx=0x7f03bc011660 comp rx=0 tx=0).stop 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 shutdown_connections 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f03a806b4d0 0x7f03a806d990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f03cc0c98a0 0x7f03cc0c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03cc1e90b0 0x7f03cc1e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 --2- 172.21.15.62:0/761495980 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f03b0004d40 0x7f03cc159d40 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 >> 172.21.15.62:0/761495980 conn(0x7f03cc066d10 msgr2=0x7f03cc0b7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:48.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 shutdown_connections 2024-04-25T09:15:48.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.712+0000 7f03d2600640 1 -- 172.21.15.62:0/761495980 wait complete. 2024-04-25T09:15:48.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:15:48.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.947+0000 7f6c3ca00640 1 Processor -- start 2024-04-25T09:15:48.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.947+0000 7f6c3ca00640 1 -- start start 2024-04-25T09:15:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.948+0000 7f6c3ca00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6c3815c4e0 0x7f6c3815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.948+0000 7f6c3ca00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c380cff60 con 0x7f6c380b49e0 2024-04-25T09:15:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.948+0000 7f6c3ca00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c38070360 con 0x7f6c380cf5f0 2024-04-25T09:15:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.948+0000 7f6c3ca00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c38070640 con 0x7f6c3815c4e0 2024-04-25T09:15:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.949+0000 7f6c37e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6c380b49e0 0x7f6c380b2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39196/0 (socket says 172.21.15.62:39196) 2024-04-25T09:15:48.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.949+0000 7f6c37e00640 1 -- 172.21.15.62:0/3533864686 learned_addr learned my addr 172.21.15.62:0/3533864686 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:48.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.949+0000 7f6c37400640 1 --2- 172.21.15.62:0/3533864686 >> v2:172.21.15.121:3300/0 conn(0x7f6c3815c4e0 0x7f6c3815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.949+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071590294 0 0) 0x7f6c380cff60 con 0x7f6c380b49e0 2024-04-25T09:15:48.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.949+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c1c003520 con 0x7f6c380b49e0 2024-04-25T09:15:48.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 773637445 0 0) 0x7f6c1c003520 con 0x7f6c380b49e0 2024-04-25T09:15:48.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 >> v2:172.21.15.121:3300/0 conn(0x7f6c3815c4e0 msgr2=0x7f6c3815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 --2- 172.21.15.62:0/3533864686 >> v2:172.21.15.121:3300/0 conn(0x7f6c3815c4e0 0x7f6c3815c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 >> v1:172.21.15.121:6789/0 conn(0x7f6c380cf5f0 legacy=0x7f6c380b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38070920 con 0x7f6c380b49e0 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f6c300019f0 con 0x7f6c380b49e0 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 >> v1:172.21.15.62:6789/0 conn(0x7f6c380b49e0 legacy=0x7f6c380b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c1c0039b0 0x7f6c1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c004800 0x7f6c1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c1c0051c0 0x7f6c1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c38070640 con 0x7f6c1c0039b0 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c00c060 con 0x7f6c1c004800 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c00c260 con 0x7f6c1c0051c0 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.950+0000 7f6c37400640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c1c0039b0 0x7f6c1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.951+0000 7f6c37e00640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c1c0051c0 0x7f6c1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.951+0000 7f6c37400640 1 -- 172.21.15.62:0/3533864686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c1c0051c0 msgr2=0x7f6c1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.951+0000 7f6c37400640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c1c0051c0 0x7f6c1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.951+0000 7f6c36a00640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c004800 0x7f6c1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.951+0000 7f6c37400640 1 -- 172.21.15.62:0/3533864686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c004800 msgr2=0x7f6c1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.951+0000 7f6c37400640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c004800 0x7f6c1c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.951+0000 7f6c37e00640 1 -- 172.21.15.62:0/3533864686 reap_dead start 2024-04-25T09:15:48.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.951+0000 7f6c37400640 1 -- 172.21.15.62:0/3533864686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c003520 con 0x7f6c1c0039b0 2024-04-25T09:15:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.952+0000 7f6c37400640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c1c0039b0 0x7f6c1c003dd0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009ac0 tx=0x7f6c1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca899554f33309d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.952+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28036070 con 0x7f6c1c0039b0 2024-04-25T09:15:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.952+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28031400 con 0x7f6c1c0039b0 2024-04-25T09:15:48.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.952+0000 7f6c36000640 1 -- 172.21.15.62:0/3533864686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28009690 con 0x7f6c1c0039b0 2024-04-25T09:15:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.953+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3533864686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c1c0039b0 msgr2=0x7f6c1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.953+0000 7f6c3ca00640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c1c0039b0 0x7f6c1c003dd0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009ac0 tx=0x7f6c1c005830 comp rx=0 tx=0).stop 2024-04-25T09:15:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.953+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3533864686 shutdown_connections 2024-04-25T09:15:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.953+0000 7f6c3ca00640 1 --2- 172.21.15.62:0/3533864686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c1c0039b0 0x7f6c1c003dd0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.953+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3533864686 >> 172.21.15.62:0/3533864686 conn(0x7f6c380668f0 msgr2=0x7f6c38066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.953+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3533864686 shutdown_connections 2024-04-25T09:15:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.953+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3533864686 wait complete. 2024-04-25T09:15:48.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.955+0000 7f6c3ca00640 1 Processor -- start 2024-04-25T09:15:48.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.955+0000 7f6c3ca00640 1 -- start start 2024-04-25T09:15:48.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c3ca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c0039b0 0x7f6c380b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c3ca00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c38148610 0x7f6c380b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c3ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c3815b7f0 0x7f6c38144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c3ca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380cfaa0 con 0x7f6c3815b7f0 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c3ca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c38071120 con 0x7f6c1c0039b0 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c3ca00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380cfc20 con 0x7f6c38148610 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c3815b7f0 0x7f6c38144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c37e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c3815b7f0 0x7f6c38144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41142/0 (socket says 172.21.15.62:41142) 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.956+0000 7f6c37e00640 1 -- 172.21.15.62:0/3047863786 learned_addr learned my addr 172.21.15.62:0/3047863786 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c36a00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c38148610 0x7f6c380b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c37400640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c0039b0 0x7f6c380b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c37e00640 1 -- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c38148610 msgr2=0x7f6c380b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c37e00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c38148610 0x7f6c380b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c37e00640 1 -- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c0039b0 msgr2=0x7f6c380b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c37e00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c0039b0 0x7f6c380b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c37e00640 1 -- 172.21.15.62:0/3047863786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38148a90 con 0x7f6c3815b7f0 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c36a00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c38148610 0x7f6c380b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c37e00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c3815b7f0 0x7f6c38144d40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f6c38067870 tx=0x7f6c30007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c2f400640 1 -- 172.21.15.62:0/3047863786 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000eaa0 con 0x7f6c3815b7f0 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c2f400640 1 -- 172.21.15.62:0/3047863786 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3000ec60 con 0x7f6c3815b7f0 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c380b09d0 con 0x7f6c3815b7f0 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.957+0000 7f6c2f400640 1 -- 172.21.15.62:0/3047863786 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30014430 con 0x7f6c3815b7f0 2024-04-25T09:15:48.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.958+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c380b0f90 con 0x7f6c3815b7f0 2024-04-25T09:15:48.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.959+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3804ef20 con 0x7f6c3815b7f0 2024-04-25T09:15:48.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.960+0000 7f6c2f400640 1 -- 172.21.15.62:0/3047863786 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c300145f0 con 0x7f6c3815b7f0 2024-04-25T09:15:48.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.961+0000 7f6c2f400640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6c1806b4d0 0x7f6c1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:48.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.961+0000 7f6c2f400640 1 -- 172.21.15.62:0/3047863786 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6c30089f50 con 0x7f6c3815b7f0 2024-04-25T09:15:48.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.961+0000 7f6c37400640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6c1806b4d0 0x7f6c1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:48.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.962+0000 7f6c37400640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6c1806b4d0 0x7f6c1806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6c28033000 tx=0x7f6c28008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:48.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:48.965+0000 7f6c2f400640 1 -- 172.21.15.62:0/3047863786 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c30058d80 con 0x7f6c3815b7f0 2024-04-25T09:15:49.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.206+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6c38071b00 con 0x7f6c3815b7f0 2024-04-25T09:15:49.192 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:15:49.207+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:15:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.210+0000 7f6c2f400640 1 -- 172.21.15.62:0/3047863786 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6c30058680 con 0x7f6c3815b7f0 2024-04-25T09:15:49.197 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:15:49.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6c1806b4d0 msgr2=0x7f6c1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:49.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6c1806b4d0 0x7f6c1806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6c28033000 tx=0x7f6c28008030 comp rx=0 tx=0).stop 2024-04-25T09:15:49.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c3815b7f0 msgr2=0x7f6c38144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:49.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c3815b7f0 0x7f6c38144d40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f6c38067870 tx=0x7f6c30007b70 comp rx=0 tx=0).stop 2024-04-25T09:15:49.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 shutdown_connections 2024-04-25T09:15:49.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6c1806b4d0 0x7f6c1806d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6c3815b7f0 0x7f6c38144d40 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6c38148610 0x7f6c380b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.214+0000 7f6c3ca00640 1 --2- 172.21.15.62:0/3047863786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c1c0039b0 0x7f6c380b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.215+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 >> 172.21.15.62:0/3047863786 conn(0x7f6c380668f0 msgr2=0x7f6c3806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:49.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.215+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 shutdown_connections 2024-04-25T09:15:49.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.215+0000 7f6c3ca00640 1 -- 172.21.15.62:0/3047863786 wait complete. 2024-04-25T09:15:49.222 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:15:49.282 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:15:49.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.427+0000 7f4340a00640 1 Processor -- start 2024-04-25T09:15:49.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.428+0000 7f4340a00640 1 -- start start 2024-04-25T09:15:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.429+0000 7f4340a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f433c0b40e0 0x7f433c06c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.429+0000 7f4340a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c162cc0 0x7f433c16bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.429+0000 7f4340a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c06e560 con 0x7f433c0ceeb0 2024-04-25T09:15:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.429+0000 7f4340a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c0cf360 con 0x7f433c0b40e0 2024-04-25T09:15:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.429+0000 7f4340a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c06dd00 con 0x7f433c162cc0 2024-04-25T09:15:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.429+0000 7f4339e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f433c0b40e0 0x7f433c06c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.429+0000 7f4339e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f433c0b40e0 0x7f433c06c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41166/0 (socket says 172.21.15.62:41166) 2024-04-25T09:15:49.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.429+0000 7f4339e00640 1 -- 172.21.15.62:0/1573122700 learned_addr learned my addr 172.21.15.62:0/1573122700 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.430+0000 7f433a800640 1 --2- 172.21.15.62:0/1573122700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c162cc0 0x7f433c16bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.430+0000 7f4339e00640 1 -- 172.21.15.62:0/1573122700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c162cc0 msgr2=0x7f433c16bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.430+0000 7f4339e00640 1 --2- 172.21.15.62:0/1573122700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c162cc0 0x7f433c16bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.430+0000 7f4339e00640 1 -- 172.21.15.62:0/1573122700 >> v1:172.21.15.121:6789/0 conn(0x7f433c0ceeb0 legacy=0x7f433c1625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.430+0000 7f4339e00640 1 -- 172.21.15.62:0/1573122700 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c06dfa0 con 0x7f433c0b40e0 2024-04-25T09:15:49.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.430+0000 7f433a800640 1 --2- 172.21.15.62:0/1573122700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c162cc0 0x7f433c16bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.431+0000 7f4339e00640 1 --2- 172.21.15.62:0/1573122700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c0b40e0 0x7f433c06c380 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f433c0b2290 tx=0x7f432c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a924c392b551dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.431+0000 7f4338a00640 1 -- 172.21.15.62:0/1573122700 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c037070 con 0x7f433c0b40e0 2024-04-25T09:15:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.431+0000 7f4338a00640 1 -- 172.21.15.62:0/1573122700 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c02adc0 con 0x7f433c0b40e0 2024-04-25T09:15:49.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.431+0000 7f4338a00640 1 -- 172.21.15.62:0/1573122700 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c032660 con 0x7f433c0b40e0 2024-04-25T09:15:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.432+0000 7f4340a00640 1 -- 172.21.15.62:0/1573122700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c0b40e0 msgr2=0x7f433c06c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.432+0000 7f4340a00640 1 --2- 172.21.15.62:0/1573122700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c0b40e0 0x7f433c06c380 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f433c0b2290 tx=0x7f432c02a0b0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.432+0000 7f4340a00640 1 -- 172.21.15.62:0/1573122700 shutdown_connections 2024-04-25T09:15:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.432+0000 7f4340a00640 1 --2- 172.21.15.62:0/1573122700 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c162cc0 0x7f433c16bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.432+0000 7f4340a00640 1 --2- 172.21.15.62:0/1573122700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c0b40e0 0x7f433c06c380 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.432+0000 7f4340a00640 1 -- 172.21.15.62:0/1573122700 >> 172.21.15.62:0/1573122700 conn(0x7f433c0b1410 msgr2=0x7f433c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.432+0000 7f4340a00640 1 -- 172.21.15.62:0/1573122700 shutdown_connections 2024-04-25T09:15:49.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.432+0000 7f4340a00640 1 -- 172.21.15.62:0/1573122700 wait complete. 2024-04-25T09:15:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.434+0000 7f4340a00640 1 Processor -- start 2024-04-25T09:15:49.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.434+0000 7f4340a00640 1 -- start start 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.435+0000 7f4340a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f433c0b40e0 0x7f433c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.435+0000 7f4340a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c0ceeb0 0x7f433c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.435+0000 7f4340a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c162cc0 0x7f433c15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.435+0000 7f4340a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c072460 con 0x7f433c162cc0 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.435+0000 7f4340a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c071f40 con 0x7f433c0b40e0 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.435+0000 7f4340a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c0725e0 con 0x7f433c0ceeb0 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f433a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c162cc0 0x7f433c15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f433a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c162cc0 0x7f433c15bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41182/0 (socket says 172.21.15.62:41182) 2024-04-25T09:15:49.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f433a800640 1 -- 172.21.15.62:0/3785909335 learned_addr learned my addr 172.21.15.62:0/3785909335 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f4339e00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f433c0b40e0 0x7f433c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f433a800640 1 -- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c0ceeb0 msgr2=0x7f433c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f4339400640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c0ceeb0 0x7f433c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f433a800640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c0ceeb0 0x7f433c0cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f433a800640 1 -- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f433c0b40e0 msgr2=0x7f433c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f433a800640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f433c0b40e0 0x7f433c0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f433a800640 1 -- 172.21.15.62:0/3785909335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0ce010 con 0x7f433c162cc0 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.436+0000 7f4339e00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f433c0b40e0 0x7f433c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.437+0000 7f433a800640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c162cc0 0x7f433c15bed0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f433400c7f0 tx=0x7f433400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.437+0000 7f4332000640 1 -- 172.21.15.62:0/3785909335 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334015070 con 0x7f433c162cc0 2024-04-25T09:15:49.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.437+0000 7f4332000640 1 -- 172.21.15.62:0/3785909335 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4334007820 con 0x7f433c162cc0 2024-04-25T09:15:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.437+0000 7f4332000640 1 -- 172.21.15.62:0/3785909335 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334007c50 con 0x7f433c162cc0 2024-04-25T09:15:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.437+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c15c8b0 con 0x7f433c162cc0 2024-04-25T09:15:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.437+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c0cc520 con 0x7f433c162cc0 2024-04-25T09:15:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.438+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c162380 con 0x7f433c162cc0 2024-04-25T09:15:49.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.439+0000 7f4332000640 1 -- 172.21.15.62:0/3785909335 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4334018070 con 0x7f433c162cc0 2024-04-25T09:15:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.440+0000 7f4332000640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f431c06b500 0x7f431c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:49.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.440+0000 7f4332000640 1 -- 172.21.15.62:0/3785909335 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f433408a3c0 con 0x7f433c162cc0 2024-04-25T09:15:49.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.441+0000 7f4339e00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f431c06b500 0x7f431c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:49.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.442+0000 7f4339e00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f431c06b500 0x7f431c06d9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f432c034000 tx=0x7f432c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:49.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.445+0000 7f4332000640 1 -- 172.21.15.62:0/3785909335 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f433c162380 con 0x7f433c162cc0 2024-04-25T09:15:49.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:49.685+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f433c04ef20 con 0x7f433c162cc0 2024-04-25T09:15:50.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.214+0000 7f4332000640 1 -- 172.21.15.62:0/3785909335 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7f43340591a0 con 0x7f433c162cc0 2024-04-25T09:15:50.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.217+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f431c06b500 msgr2=0x7f431c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:50.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.217+0000 7f4340a00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f431c06b500 0x7f431c06d9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f432c034000 tx=0x7f432c02e000 comp rx=0 tx=0).stop 2024-04-25T09:15:50.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.217+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c162cc0 msgr2=0x7f433c15bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.217+0000 7f4340a00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c162cc0 0x7f433c15bed0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f433400c7f0 tx=0x7f433400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.218+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 shutdown_connections 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.218+0000 7f4340a00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f431c06b500 0x7f431c06d9c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.218+0000 7f4340a00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f433c162cc0 0x7f433c15bed0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.218+0000 7f4340a00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f433c0ceeb0 0x7f433c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.218+0000 7f4340a00640 1 --2- 172.21.15.62:0/3785909335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f433c0b40e0 0x7f433c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.218+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 >> 172.21.15.62:0/3785909335 conn(0x7f433c0b1410 msgr2=0x7f433c0702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.218+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 shutdown_connections 2024-04-25T09:15:50.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.218+0000 7f4340a00640 1 -- 172.21.15.62:0/3785909335 wait complete. 2024-04-25T09:15:50.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:15:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.448+0000 7f4007200640 1 Processor -- start 2024-04-25T09:15:50.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.448+0000 7f4007200640 1 -- start start 2024-04-25T09:15:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.450+0000 7f4007200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f40080cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.450+0000 7f4007200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4008071600 0x7f40080b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f4007200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40080b32d0 con 0x7f40080cc8e0 2024-04-25T09:15:50.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f4007200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40080cf610 con 0x7f4008071600 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f4007200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4008071ae0 con 0x7f40080b34e0 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f3fffe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4008071600 0x7f40080b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f3fffe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4008071600 0x7f40080b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41192/0 (socket says 172.21.15.62:41192) 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f3fffe00640 1 -- 172.21.15.62:0/4020830691 learned_addr learned my addr 172.21.15.62:0/4020830691 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f4005e00640 1 --2- 172.21.15.62:0/4020830691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f40080cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f3fffe00640 1 -- 172.21.15.62:0/4020830691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 msgr2=0x7f40080cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f3fffe00640 1 --2- 172.21.15.62:0/4020830691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f40080cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f3fffe00640 1 -- 172.21.15.62:0/4020830691 >> v1:172.21.15.121:6789/0 conn(0x7f40080cc8e0 legacy=0x7f400806cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f3fffe00640 1 -- 172.21.15.62:0/4020830691 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4008071dc0 con 0x7f4008071600 2024-04-25T09:15:50.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.451+0000 7f4005e00640 1 --2- 172.21.15.62:0/4020830691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f40080cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:50.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.453+0000 7f3fffe00640 1 --2- 172.21.15.62:0/4020830691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008071600 0x7f40080b38c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009870 tx=0x7f3ff002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1203174acaab2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.453+0000 7f4005400640 1 -- 172.21.15.62:0/4020830691 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0037070 con 0x7f4008071600 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.453+0000 7f4005400640 1 -- 172.21.15.62:0/4020830691 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0032440 con 0x7f4008071600 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.453+0000 7f4005400640 1 -- 172.21.15.62:0/4020830691 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0032810 con 0x7f4008071600 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.454+0000 7f4007200640 1 -- 172.21.15.62:0/4020830691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008071600 msgr2=0x7f40080b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.454+0000 7f4007200640 1 --2- 172.21.15.62:0/4020830691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008071600 0x7f40080b38c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009870 tx=0x7f3ff002a200 comp rx=0 tx=0).stop 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.454+0000 7f4007200640 1 -- 172.21.15.62:0/4020830691 shutdown_connections 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.454+0000 7f4007200640 1 --2- 172.21.15.62:0/4020830691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4008071600 0x7f40080b38c0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.454+0000 7f4007200640 1 --2- 172.21.15.62:0/4020830691 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f40080cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.454+0000 7f4007200640 1 -- 172.21.15.62:0/4020830691 >> 172.21.15.62:0/4020830691 conn(0x7f4008066ad0 msgr2=0x7f4008066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:50.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.454+0000 7f4007200640 1 -- 172.21.15.62:0/4020830691 shutdown_connections 2024-04-25T09:15:50.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.454+0000 7f4007200640 1 -- 172.21.15.62:0/4020830691 wait complete. 2024-04-25T09:15:50.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.455+0000 7f4007200640 1 Processor -- start 2024-04-25T09:15:50.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.456+0000 7f4007200640 1 -- start start 2024-04-25T09:15:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4007200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4008071600 0x7f400815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4007200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f400815f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4007200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40080cc8e0 0x7f400815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4007200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400806ca00 con 0x7f40080cc8e0 2024-04-25T09:15:50.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4007200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400806c720 con 0x7f4008071600 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4007200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40080b3e00 con 0x7f40080b34e0 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4006800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40080cc8e0 0x7f400815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4006800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40080cc8e0 0x7f400815dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41198/0 (socket says 172.21.15.62:41198) 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4006800640 1 -- 172.21.15.62:0/1069268882 learned_addr learned my addr 172.21.15.62:0/1069268882 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f3fffe00640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f400815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4005e00640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4008071600 0x7f400815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4006800640 1 -- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 msgr2=0x7f400815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4006800640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f400815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4006800640 1 -- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4008071600 msgr2=0x7f400815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4006800640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4008071600 0x7f400815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.457+0000 7f4006800640 1 -- 172.21.15.62:0/1069268882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40081f8630 con 0x7f40080cc8e0 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.458+0000 7f4006800640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40080cc8e0 0x7f400815dcf0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f400000b9e0 tx=0x7f400000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:50.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.458+0000 7f3ffe000640 1 -- 172.21.15.62:0/1069268882 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000003b70 con 0x7f40080cc8e0 2024-04-25T09:15:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.458+0000 7f3ffe000640 1 -- 172.21.15.62:0/1069268882 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4000003d10 con 0x7f40080cc8e0 2024-04-25T09:15:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.458+0000 7f3ffe000640 1 -- 172.21.15.62:0/1069268882 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000cb60 con 0x7f40080cc8e0 2024-04-25T09:15:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.458+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40081f8910 con 0x7f40080cc8e0 2024-04-25T09:15:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.458+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40081f8ed0 con 0x7f40080cc8e0 2024-04-25T09:15:50.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.459+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008072b30 con 0x7f40080cc8e0 2024-04-25T09:15:50.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.460+0000 7f3ffe000640 1 -- 172.21.15.62:0/1069268882 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f400000cd00 con 0x7f40080cc8e0 2024-04-25T09:15:50.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.461+0000 7f3ffe000640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:50.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.461+0000 7f3ffe000640 1 -- 172.21.15.62:0/1069268882 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f400008a190 con 0x7f40080cc8e0 2024-04-25T09:15:50.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.462+0000 7f4005e00640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:50.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.463+0000 7f4005e00640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3fe406b4d0 0x7f3fe406d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4008066f20 tx=0x7f3ff8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:50.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.466+0000 7f3ffe000640 1 -- 172.21.15.62:0/1069268882 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4000058ee0 con 0x7f40080cc8e0 2024-04-25T09:15:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:50.673+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f40080750e0 con 0x7f40080cc8e0 2024-04-25T09:15:51.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.218+0000 7f3ffe000640 1 -- 172.21.15.62:0/1069268882 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 181+0+0 (secure 0 0 0) 0x7f40000587e0 con 0x7f40080cc8e0 2024-04-25T09:15:51.204 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:15:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.223+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3fe406b4d0 msgr2=0x7f3fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.223+0000 7f4007200640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3fe406b4d0 0x7f3fe406d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4008066f20 tx=0x7f3ff8008040 comp rx=0 tx=0).stop 2024-04-25T09:15:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40080cc8e0 msgr2=0x7f400815dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40080cc8e0 0x7f400815dcf0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f400000b9e0 tx=0x7f400000bea0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 shutdown_connections 2024-04-25T09:15:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3fe406b4d0 0x7f3fe406d990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40080cc8e0 0x7f400815dcf0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f40080b34e0 0x7f400815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 --2- 172.21.15.62:0/1069268882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4008071600 0x7f400815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 >> 172.21.15.62:0/1069268882 conn(0x7f4008066ad0 msgr2=0x7f40080b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.224+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 shutdown_connections 2024-04-25T09:15:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.225+0000 7f4007200640 1 -- 172.21.15.62:0/1069268882 wait complete. 2024-04-25T09:15:51.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:15:51.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.462+0000 7fed9a000640 1 Processor -- start 2024-04-25T09:15:51.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.462+0000 7fed9a000640 1 -- start start 2024-04-25T09:15:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.464+0000 7fed9a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fed9406e750 0x7fed9406ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.464+0000 7fed9a000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940b3090 con 0x7fed9415b960 2024-04-25T09:15:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.464+0000 7fed9a000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940b3370 con 0x7fed940cee00 2024-04-25T09:15:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.464+0000 7fed9a000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940b3650 con 0x7fed9406e750 2024-04-25T09:15:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.464+0000 7fed98c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fed9415b960 0x7fed9415bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39202/0 (socket says 172.21.15.62:39202) 2024-04-25T09:15:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.464+0000 7fed98c00640 1 -- 172.21.15.62:0/2850323816 learned_addr learned my addr 172.21.15.62:0/2850323816 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.464+0000 7fed93e00640 1 --2- 172.21.15.62:0/2850323816 >> v2:172.21.15.121:3300/0 conn(0x7fed9406e750 0x7fed9406ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894918668 0 0) 0x7fed940b3090 con 0x7fed9415b960 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed78003520 con 0x7fed9415b960 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3002427189 0 0) 0x7fed78003520 con 0x7fed9415b960 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 >> v2:172.21.15.121:3300/0 conn(0x7fed9406e750 msgr2=0x7fed9406ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 --2- 172.21.15.62:0/2850323816 >> v2:172.21.15.121:3300/0 conn(0x7fed9406e750 0x7fed9406ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 >> v1:172.21.15.121:6789/0 conn(0x7fed940cee00 legacy=0x7fed940cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940b3930 con 0x7fed9415b960 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fed840019f0 con 0x7fed9415b960 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 >> v1:172.21.15.62:6789/0 conn(0x7fed9415b960 legacy=0x7fed9415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed780039b0 0x7fed78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed78004800 0x7fed78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed780051c0 0x7fed7800fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940b3650 con 0x7fed78004800 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed78010470 con 0x7fed780051c0 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.465+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed78010670 con 0x7fed780039b0 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.466+0000 7fed93e00640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed78004800 0x7fed78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.466+0000 7fed93e00640 1 -- 172.21.15.62:0/2850323816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed780039b0 msgr2=0x7fed78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.466+0000 7fed93e00640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed780039b0 0x7fed78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.466+0000 7fed93e00640 1 -- 172.21.15.62:0/2850323816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed780051c0 msgr2=0x7fed7800fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.466+0000 7fed93e00640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed780051c0 0x7fed7800fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.466+0000 7fed93e00640 1 -- 172.21.15.62:0/2850323816 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed78003520 con 0x7fed78004800 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.466+0000 7fed99600640 1 -- 172.21.15.62:0/2850323816 reap_dead start 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.467+0000 7fed93e00640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed78004800 0x7fed78004c80 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fed80009ac0 tx=0x7fed78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31782eb8cca6e285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.467+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80036070 con 0x7fed78004800 2024-04-25T09:15:51.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.467+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80031400 con 0x7fed78004800 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.467+0000 7fed93400640 1 -- 172.21.15.62:0/2850323816 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80009690 con 0x7fed78004800 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.468+0000 7fed9a000640 1 -- 172.21.15.62:0/2850323816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed78004800 msgr2=0x7fed78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.468+0000 7fed9a000640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed78004800 0x7fed78004c80 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fed80009ac0 tx=0x7fed78005830 comp rx=0 tx=0).stop 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.468+0000 7fed9a000640 1 -- 172.21.15.62:0/2850323816 shutdown_connections 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.468+0000 7fed9a000640 1 --2- 172.21.15.62:0/2850323816 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed78004800 0x7fed78004c80 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.468+0000 7fed9a000640 1 -- 172.21.15.62:0/2850323816 >> 172.21.15.62:0/2850323816 conn(0x7fed940b1bf0 msgr2=0x7fed940b1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.468+0000 7fed9a000640 1 -- 172.21.15.62:0/2850323816 shutdown_connections 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.468+0000 7fed9a000640 1 -- 172.21.15.62:0/2850323816 wait complete. 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.470+0000 7fed9a000640 1 Processor -- start 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.470+0000 7fed9a000640 1 -- start start 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.471+0000 7fed9a000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed78004800 0x7fed9415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.471+0000 7fed9a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed9415c520 0x7fed940aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.471+0000 7fed9a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed940aa870 0x7fed940ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.471+0000 7fed9a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9406ed60 con 0x7fed940aa870 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.471+0000 7fed9a000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9406ebe0 con 0x7fed9415c520 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.471+0000 7fed9a000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9406eee0 con 0x7fed78004800 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed940aa870 0x7fed940ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed940aa870 0x7fed940ae7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41224/0 (socket says 172.21.15.62:41224) 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 -- 172.21.15.62:0/3307733444 learned_addr learned my addr 172.21.15.62:0/3307733444 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed93e00640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed9415c520 0x7fed940aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed98c00640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed78004800 0x7fed9415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 -- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed78004800 msgr2=0x7fed9415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed78004800 0x7fed9415bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 -- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed9415c520 msgr2=0x7fed940aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed9415c520 0x7fed940aa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 -- 172.21.15.62:0/3307733444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940a9570 con 0x7fed940aa870 2024-04-25T09:15:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed93e00640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed9415c520 0x7fed940aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.472+0000 7fed99600640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed940aa870 0x7fed940ae7f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fed940cf970 tx=0x7fed8c00c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.473+0000 7fed90c00640 1 -- 172.21.15.62:0/3307733444 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c017070 con 0x7fed940aa870 2024-04-25T09:15:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.473+0000 7fed90c00640 1 -- 172.21.15.62:0/3307733444 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c0072f0 con 0x7fed940aa870 2024-04-25T09:15:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.473+0000 7fed90c00640 1 -- 172.21.15.62:0/3307733444 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c0057c0 con 0x7fed940aa870 2024-04-25T09:15:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.473+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed940a9850 con 0x7fed940aa870 2024-04-25T09:15:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.473+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed940a9e10 con 0x7fed940aa870 2024-04-25T09:15:51.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.474+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed940750e0 con 0x7fed940aa870 2024-04-25T09:15:51.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.475+0000 7fed90c00640 1 -- 172.21.15.62:0/3307733444 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed8c01a070 con 0x7fed940aa870 2024-04-25T09:15:51.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.476+0000 7fed90c00640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fed7406b500 0x7fed7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:51.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.476+0000 7fed90c00640 1 -- 172.21.15.62:0/3307733444 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5386+0+0 (secure 0 0 0) 0x7fed8c088ff0 con 0x7fed940aa870 2024-04-25T09:15:51.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.477+0000 7fed98c00640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fed7406b500 0x7fed7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:51.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.478+0000 7fed98c00640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fed7406b500 0x7fed7406d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fed84001be0 tx=0x7fed84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:51.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.480+0000 7fed90c00640 1 -- 172.21.15.62:0/3307733444 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed8c058000 con 0x7fed940aa870 2024-04-25T09:15:51.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:51.688+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed940af9d0 con 0x7fed940aa870 2024-04-25T09:15:52.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.222+0000 7fed90c00640 1 -- 172.21.15.62:0/3307733444 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) v1 ==== 169+0+0 (secure 0 0 0) 0x7fed8c057900 con 0x7fed940aa870 2024-04-25T09:15:52.208 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:15:52.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.226+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fed7406b500 msgr2=0x7fed7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.226+0000 7fed9a000640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fed7406b500 0x7fed7406d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fed84001be0 tx=0x7fed84009040 comp rx=0 tx=0).stop 2024-04-25T09:15:52.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.227+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed940aa870 msgr2=0x7fed940ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.227+0000 7fed9a000640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed940aa870 0x7fed940ae7f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fed940cf970 tx=0x7fed8c00c5f0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.227+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 shutdown_connections 2024-04-25T09:15:52.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.227+0000 7fed9a000640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fed7406b500 0x7fed7406d9c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.227+0000 7fed9a000640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed940aa870 0x7fed940ae7f0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.227+0000 7fed9a000640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed9415c520 0x7fed940aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.227+0000 7fed9a000640 1 --2- 172.21.15.62:0/3307733444 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fed78004800 0x7fed9415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.227+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 >> 172.21.15.62:0/3307733444 conn(0x7fed940b1bf0 msgr2=0x7fed940d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:52.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.229+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 shutdown_connections 2024-04-25T09:15:52.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.229+0000 7fed9a000640 1 -- 172.21.15.62:0/3307733444 wait complete. 2024-04-25T09:15:52.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:15:52.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.471+0000 7f886ee00640 1 Processor -- start 2024-04-25T09:15:52.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.471+0000 7f886ee00640 1 -- start start 2024-04-25T09:15:52.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.473+0000 7f886ee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88680cebc0 0x7f886806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.473+0000 7f886ee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886806d070 con 0x7f88680720c0 2024-04-25T09:15:52.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.473+0000 7f886ee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886806d350 con 0x7f886815bdf0 2024-04-25T09:15:52.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.473+0000 7f886ee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8868072570 con 0x7f88680cebc0 2024-04-25T09:15:52.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.473+0000 7f886ce00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f88680720c0 0x7f88680703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39204/0 (socket says 172.21.15.62:39204) 2024-04-25T09:15:52.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.473+0000 7f886ce00640 1 -- 172.21.15.62:0/2226670451 learned_addr learned my addr 172.21.15.62:0/2226670451 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:52.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.473+0000 7f8867400640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88680cebc0 0x7f886806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644749808 0 0) 0x7f886806d070 con 0x7f88680720c0 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f884c003520 con 0x7f88680720c0 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3282050989 0 0) 0x7f884c003520 con 0x7f88680720c0 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88680cebc0 msgr2=0x7f886806c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88680cebc0 0x7f886806c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 >> v1:172.21.15.121:6789/0 conn(0x7f886815bdf0 legacy=0x7f886806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88680aeb10 con 0x7f88680720c0 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f88600019f0 con 0x7f88680720c0 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 >> v1:172.21.15.62:6789/0 conn(0x7f88680720c0 legacy=0x7f88680703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f884c0039e0 0x7f884c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88680cebc0 0x7f884c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c004d40 0x7f884c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c00ba70 con 0x7f884c0039e0 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c00bbf0 con 0x7f884c004d40 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c00bdf0 con 0x7f88680cebc0 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.474+0000 7f8867e00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f884c0039e0 0x7f884c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.475+0000 7f8867400640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88680cebc0 0x7f884c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.475+0000 7f8867e00640 1 -- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88680cebc0 msgr2=0x7f884c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.475+0000 7f8867e00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88680cebc0 0x7f884c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.475+0000 7f886ce00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c004d40 0x7f884c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.475+0000 7f8867e00640 1 -- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c004d40 msgr2=0x7f884c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.475+0000 7f8867e00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c004d40 0x7f884c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.475+0000 7f8867e00640 1 -- 172.21.15.62:0/2226670451 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c003520 con 0x7f884c0039e0 2024-04-25T09:15:52.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.476+0000 7f8867e00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f884c0039e0 0x7f884c003e00 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f8858001720 tx=0x7f884c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d2e267e0d20ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:52.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.476+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858001370 con 0x7f884c0039e0 2024-04-25T09:15:52.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.476+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88580052d0 con 0x7f884c0039e0 2024-04-25T09:15:52.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.476+0000 7f8866a00640 1 -- 172.21.15.62:0/2226670451 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885802abc0 con 0x7f884c0039e0 2024-04-25T09:15:52.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.477+0000 7f886ee00640 1 -- 172.21.15.62:0/2226670451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f884c0039e0 msgr2=0x7f884c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.477+0000 7f886ee00640 1 --2- 172.21.15.62:0/2226670451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f884c0039e0 0x7f884c003e00 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f8858001720 tx=0x7f884c005830 comp rx=0 tx=0).stop 2024-04-25T09:15:52.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.477+0000 7f886ce00640 1 -- 172.21.15.62:0/2226670451 reap_dead start 2024-04-25T09:15:52.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.477+0000 7f886ee00640 1 -- 172.21.15.62:0/2226670451 shutdown_connections 2024-04-25T09:15:52.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.477+0000 7f886ee00640 1 -- 172.21.15.62:0/2226670451 >> 172.21.15.62:0/2226670451 conn(0x7f88680b1410 msgr2=0x7f88680b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:52.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.477+0000 7f886ee00640 1 -- 172.21.15.62:0/2226670451 shutdown_connections 2024-04-25T09:15:52.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.477+0000 7f886ee00640 1 -- 172.21.15.62:0/2226670451 wait complete. 2024-04-25T09:15:52.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.479+0000 7f886ee00640 1 Processor -- start 2024-04-25T09:15:52.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.479+0000 7f886ee00640 1 -- start start 2024-04-25T09:15:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c0039e0 0x7f88681570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8868151fc0 0x7f8868152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88681593d0 0x7f8868159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88680ae8c0 con 0x7f88681593d0 2024-04-25T09:15:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886815c1d0 con 0x7f884c0039e0 2024-04-25T09:15:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886806c9b0 con 0x7f8868151fc0 2024-04-25T09:15:52.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88681593d0 0x7f8868159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88681593d0 0x7f8868159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41242/0 (socket says 172.21.15.62:41242) 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.480+0000 7f886ce00640 1 -- 172.21.15.62:0/2611200773 learned_addr learned my addr 172.21.15.62:0/2611200773 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.481+0000 7f8867e00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c0039e0 0x7f88681570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.481+0000 7f886ce00640 1 -- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8868151fc0 msgr2=0x7f8868152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.481+0000 7f8867400640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8868151fc0 0x7f8868152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.481+0000 7f886ce00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8868151fc0 0x7f8868152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.481+0000 7f886ce00640 1 -- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c0039e0 msgr2=0x7f88681570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.481+0000 7f886ce00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c0039e0 0x7f88681570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.481+0000 7f886ce00640 1 -- 172.21.15.62:0/2611200773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868152a60 con 0x7f88681593d0 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.481+0000 7f886ce00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88681593d0 0x7f8868159880 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f8860001370 tx=0x7f886000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.482+0000 7f885fe00640 1 -- 172.21.15.62:0/2611200773 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000bb80 con 0x7f88681593d0 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.482+0000 7f885fe00640 1 -- 172.21.15.62:0/2611200773 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886000bd40 con 0x7f88681593d0 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.482+0000 7f885fe00640 1 -- 172.21.15.62:0/2611200773 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8860007350 con 0x7f88681593d0 2024-04-25T09:15:52.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.482+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8868152d40 con 0x7f88681593d0 2024-04-25T09:15:52.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.482+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8868153300 con 0x7f88681593d0 2024-04-25T09:15:52.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.483+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88680cc320 con 0x7f88681593d0 2024-04-25T09:15:52.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.484+0000 7f885fe00640 1 -- 172.21.15.62:0/2611200773 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f886001b060 con 0x7f88681593d0 2024-04-25T09:15:52.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.485+0000 7f885fe00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f884406b500 0x7f884406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.485+0000 7f885fe00640 1 -- 172.21.15.62:0/2611200773 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8860089fb0 con 0x7f88681593d0 2024-04-25T09:15:52.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.486+0000 7f8867e00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f884406b500 0x7f884406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.487+0000 7f8867e00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f884406b500 0x7f884406d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8858032000 tx=0x7f885802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:52.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.489+0000 7f885fe00640 1 -- 172.21.15.62:0/2611200773 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88680cc320 con 0x7f88681593d0 2024-04-25T09:15:52.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.690+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f88680ccc60 con 0x7f88681593d0 2024-04-25T09:15:52.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.697+0000 7f885fe00640 1 -- 172.21.15.62:0/2611200773 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88680ccc60 con 0x7f88681593d0 2024-04-25T09:15:52.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.700+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f884406b500 msgr2=0x7f884406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.700+0000 7f886ee00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f884406b500 0x7f884406d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8858032000 tx=0x7f885802e000 comp rx=0 tx=0).stop 2024-04-25T09:15:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.700+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88681593d0 msgr2=0x7f8868159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.700+0000 7f886ee00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88681593d0 0x7f8868159880 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f8860001370 tx=0x7f886000b4a0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.701+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 shutdown_connections 2024-04-25T09:15:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.701+0000 7f886ee00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f884406b500 0x7f884406d9c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.701+0000 7f886ee00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88681593d0 0x7f8868159880 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.701+0000 7f886ee00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8868151fc0 0x7f8868152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.701+0000 7f886ee00640 1 --2- 172.21.15.62:0/2611200773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f884c0039e0 0x7f88681570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.701+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 >> 172.21.15.62:0/2611200773 conn(0x7f88680b1410 msgr2=0x7f88681547a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.701+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 shutdown_connections 2024-04-25T09:15:52.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.701+0000 7f886ee00640 1 -- 172.21.15.62:0/2611200773 wait complete. 2024-04-25T09:15:52.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T09:15:52.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.933+0000 7f9c8f200640 1 Processor -- start 2024-04-25T09:15:52.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.933+0000 7f9c8f200640 1 -- start start 2024-04-25T09:15:52.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.934+0000 7f9c8f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9c9006d900 0x7f9c90165370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.934+0000 7f9c8f200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c900b1220 con 0x7f9c9013d970 2024-04-25T09:15:52.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.935+0000 7f9c8f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9015b220 con 0x7f9c9015acf0 2024-04-25T09:15:52.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.935+0000 7f9c8f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9015b500 con 0x7f9c9006d900 2024-04-25T09:15:52.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.935+0000 7f9c8de00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9c9013d970 0x7f9c900b0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39212/0 (socket says 172.21.15.62:39212) 2024-04-25T09:15:52.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.935+0000 7f9c8de00640 1 -- 172.21.15.62:0/1784466202 learned_addr learned my addr 172.21.15.62:0/1784466202 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.935+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130874385 0 0) 0x7f9c900b1220 con 0x7f9c9013d970 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.935+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c74005730 con 0x7f9c9013d970 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.935+0000 7f9c8e800640 1 --2- 172.21.15.62:0/1784466202 >> v2:172.21.15.121:3300/0 conn(0x7f9c9006d900 0x7f9c90165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1104667957 0 0) 0x7f9c74005730 con 0x7f9c9013d970 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 >> v2:172.21.15.121:3300/0 conn(0x7f9c9006d900 msgr2=0x7f9c90165370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 --2- 172.21.15.62:0/1784466202 >> v2:172.21.15.121:3300/0 conn(0x7f9c9006d900 0x7f9c90165370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 >> v1:172.21.15.121:6789/0 conn(0x7f9c9015acf0 legacy=0x7f9c9006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9015b7e0 con 0x7f9c9013d970 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f9c80001670 con 0x7f9c9013d970 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8e800640 1 --2- 172.21.15.62:0/1784466202 >> v2:172.21.15.121:3300/0 conn(0x7f9c9006d900 0x7f9c90165370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 >> v1:172.21.15.62:6789/0 conn(0x7f9c9013d970 legacy=0x7f9c900b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74005c10 0x7f9c74006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c74006a10 0x7f9c74006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c74007420 0x7f9c7400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9015b500 con 0x7f9c74006a10 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7400e370 con 0x7f9c74007420 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7400e570 con 0x7f9c74005c10 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.936+0000 7f9c85400640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c74006a10 0x7f9c74006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c8de00640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74005c10 0x7f9c74006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c8e800640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c74007420 0x7f9c7400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c85400640 1 -- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74005c10 msgr2=0x7f9c74006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c85400640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74005c10 0x7f9c74006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c85400640 1 -- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c74007420 msgr2=0x7f9c7400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c85400640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c74007420 0x7f9c7400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c85400640 1 -- 172.21.15.62:0/1784466202 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c74005730 con 0x7f9c74006a10 2024-04-25T09:15:52.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c8de00640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74005c10 0x7f9c74006030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:52.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c8e800640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c74007420 0x7f9c7400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:52.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.937+0000 7f9c8e800640 1 -- 172.21.15.62:0/1784466202 reap_dead start 2024-04-25T09:15:52.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.938+0000 7f9c85400640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c74006a10 0x7f9c74006e90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f9c78001720 tx=0x7f9c74007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539a76efb5848dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:52.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.938+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78001370 con 0x7f9c74006a10 2024-04-25T09:15:52.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.938+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c780052d0 con 0x7f9c74006a10 2024-04-25T09:15:52.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.938+0000 7f9c8d400640 1 -- 172.21.15.62:0/1784466202 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7802ab90 con 0x7f9c74006a10 2024-04-25T09:15:52.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.939+0000 7f9c8f200640 1 -- 172.21.15.62:0/1784466202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c74006a10 msgr2=0x7f9c74006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.939+0000 7f9c8f200640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c74006a10 0x7f9c74006e90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f9c78001720 tx=0x7f9c74007a40 comp rx=0 tx=0).stop 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.939+0000 7f9c8f200640 1 -- 172.21.15.62:0/1784466202 shutdown_connections 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.939+0000 7f9c8f200640 1 --2- 172.21.15.62:0/1784466202 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c74006a10 0x7f9c74006e90 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.939+0000 7f9c8f200640 1 -- 172.21.15.62:0/1784466202 >> 172.21.15.62:0/1784466202 conn(0x7f9c900b14d0 msgr2=0x7f9c900b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.939+0000 7f9c8f200640 1 -- 172.21.15.62:0/1784466202 shutdown_connections 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.939+0000 7f9c8f200640 1 -- 172.21.15.62:0/1784466202 wait complete. 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.941+0000 7f9c8f200640 1 Processor -- start 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.941+0000 7f9c8f200640 1 -- start start 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c8f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74006a10 0x7f9c901f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c8f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9015ce70 0x7f9c9015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c8f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c90142590 0x7f9c90142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c8f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b0c90 con 0x7f9c9015ce70 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c8f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b0b10 con 0x7f9c90142590 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c8f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900b0e10 con 0x7f9c74006a10 2024-04-25T09:15:52.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c85400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9015ce70 0x7f9c9015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c85400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9015ce70 0x7f9c9015d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41264/0 (socket says 172.21.15.62:41264) 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c85400640 1 -- 172.21.15.62:0/4144391631 learned_addr learned my addr 172.21.15.62:0/4144391631 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c8de00640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74006a10 0x7f9c901f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.942+0000 7f9c8e800640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c90142590 0x7f9c90142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c85400640 1 -- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74006a10 msgr2=0x7f9c901f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c85400640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74006a10 0x7f9c901f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c85400640 1 -- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c90142590 msgr2=0x7f9c90142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c85400640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c90142590 0x7f9c90142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c85400640 1 -- 172.21.15.62:0/4144391631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c901f3990 con 0x7f9c9015ce70 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c85400640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9015ce70 0x7f9c9015d2f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f9c78008fd0 tx=0x7f9c78001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c86a00640 1 -- 172.21.15.62:0/4144391631 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78042070 con 0x7f9c9015ce70 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c86a00640 1 -- 172.21.15.62:0/4144391631 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78005480 con 0x7f9c9015ce70 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c86a00640 1 -- 172.21.15.62:0/4144391631 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78037040 con 0x7f9c9015ce70 2024-04-25T09:15:52.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.943+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c90143270 con 0x7f9c9015ce70 2024-04-25T09:15:52.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.944+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c90143830 con 0x7f9c9015ce70 2024-04-25T09:15:52.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.945+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9004ef20 con 0x7f9c9015ce70 2024-04-25T09:15:52.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.946+0000 7f9c86a00640 1 -- 172.21.15.62:0/4144391631 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c78045070 con 0x7f9c9015ce70 2024-04-25T09:15:52.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.947+0000 7f9c86a00640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:52.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.947+0000 7f9c86a00640 1 -- 172.21.15.62:0/4144391631 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9c780a9830 con 0x7f9c9015ce70 2024-04-25T09:15:52.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.947+0000 7f9c8de00640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:52.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.949+0000 7f9c8de00640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000c00 tx=0x7f9c80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:52.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:52.951+0000 7f9c86a00640 1 -- 172.21.15.62:0/4144391631 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c78078910 con 0x7f9c9015ce70 2024-04-25T09:15:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.226+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f9c90140b70 con 0x7f9c9015ce70 2024-04-25T09:15:53.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.237+0000 7f9c86a00640 1 -- 172.21.15.62:0/4144391631 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v566) v1 ==== 166+0+0 (secure 0 0 0) 0x7f9c7802be20 con 0x7f9c9015ce70 2024-04-25T09:15:53.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c6c06b4d0 msgr2=0x7f9c6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000c00 tx=0x7f9c80009040 comp rx=0 tx=0).stop 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9015ce70 msgr2=0x7f9c9015d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9015ce70 0x7f9c9015d2f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f9c78008fd0 tx=0x7f9c78001120 comp rx=0 tx=0).stop 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 shutdown_connections 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9c9015ce70 0x7f9c9015d2f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c90142590 0x7f9c90142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 --2- 172.21.15.62:0/4144391631 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9c74006a10 0x7f9c901f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.240+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 >> 172.21.15.62:0/4144391631 conn(0x7f9c900b14d0 msgr2=0x7f9c900b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.241+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 shutdown_connections 2024-04-25T09:15:53.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.241+0000 7f9c8f200640 1 -- 172.21.15.62:0/4144391631 wait complete. 2024-04-25T09:15:53.246 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:15:53.247 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:15:53.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:15:53.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.482+0000 7fa9ce800640 1 Processor -- start 2024-04-25T09:15:53.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.482+0000 7fa9ce800640 1 -- start start 2024-04-25T09:15:53.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.484+0000 7fa9ce800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa9c80cfed0 0x7fa9c815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.485+0000 7fa9ce800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c806c310 con 0x7fa9c8072340 2024-04-25T09:15:53.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.485+0000 7fa9ce800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c815f5f0 con 0x7fa9c80709c0 2024-04-25T09:15:53.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.485+0000 7fa9ce800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c806c5b0 con 0x7fa9c80cfed0 2024-04-25T09:15:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.485+0000 7fa9c6a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa9c8072340 0x7fa9c80702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39226/0 (socket says 172.21.15.62:39226) 2024-04-25T09:15:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.485+0000 7fa9c6a00640 1 -- 172.21.15.62:0/209404273 learned_addr learned my addr 172.21.15.62:0/209404273 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.485+0000 7fa9c7400640 1 --2- 172.21.15.62:0/209404273 >> v2:172.21.15.121:3300/0 conn(0x7fa9c80cfed0 0x7fa9c815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.486+0000 7fa9c6000640 1 -- 172.21.15.62:0/209404273 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529511225 0 0) 0x7fa9c806c310 con 0x7fa9c8072340 2024-04-25T09:15:53.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.486+0000 7fa9c6000640 1 -- 172.21.15.62:0/209404273 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9ac005730 con 0x7fa9c8072340 2024-04-25T09:15:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.486+0000 7fa9c7400640 1 -- 172.21.15.62:0/209404273 >> v1:172.21.15.121:6789/0 conn(0x7fa9c80709c0 legacy=0x7fa9c80b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.486+0000 7fa9c7400640 1 -- 172.21.15.62:0/209404273 >> v1:172.21.15.62:6789/0 conn(0x7fa9c8072340 legacy=0x7fa9c80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.486+0000 7fa9c7400640 1 -- 172.21.15.62:0/209404273 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c806c890 con 0x7fa9c80cfed0 2024-04-25T09:15:53.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.488+0000 7fa9c7400640 1 --2- 172.21.15.62:0/209404273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c80cfed0 0x7fa9c815f0b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00097a0 tx=0x7fa9b002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435a2f149e2a651c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.489+0000 7fa9c6000640 1 -- 172.21.15.62:0/209404273 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0037070 con 0x7fa9c80cfed0 2024-04-25T09:15:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.489+0000 7fa9c6000640 1 -- 172.21.15.62:0/209404273 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b0032440 con 0x7fa9c80cfed0 2024-04-25T09:15:53.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.489+0000 7fa9c6000640 1 -- 172.21.15.62:0/209404273 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0032810 con 0x7fa9c80cfed0 2024-04-25T09:15:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.490+0000 7fa9ce800640 1 -- 172.21.15.62:0/209404273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c80cfed0 msgr2=0x7fa9c815f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.490+0000 7fa9ce800640 1 --2- 172.21.15.62:0/209404273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c80cfed0 0x7fa9c815f0b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00097a0 tx=0x7fa9b002a250 comp rx=0 tx=0).stop 2024-04-25T09:15:53.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.490+0000 7fa9ce800640 1 -- 172.21.15.62:0/209404273 shutdown_connections 2024-04-25T09:15:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.490+0000 7fa9ce800640 1 --2- 172.21.15.62:0/209404273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c80cfed0 0x7fa9c815f0b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.490+0000 7fa9ce800640 1 -- 172.21.15.62:0/209404273 >> 172.21.15.62:0/209404273 conn(0x7fa9c80668f0 msgr2=0x7fa9c8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.491+0000 7fa9ce800640 1 -- 172.21.15.62:0/209404273 shutdown_connections 2024-04-25T09:15:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.491+0000 7fa9ce800640 1 -- 172.21.15.62:0/209404273 wait complete. 2024-04-25T09:15:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.494+0000 7fa9ce800640 1 Processor -- start 2024-04-25T09:15:53.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.494+0000 7fa9ce800640 1 -- start start 2024-04-25T09:15:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.496+0000 7fa9ce800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9c80709c0 0x7fa9c80ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.496+0000 7fa9ce800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c80cfed0 0x7fa9c81eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.496+0000 7fa9ce800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c81f0080 0x7fa9c80ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.496+0000 7fa9ce800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c806d130 con 0x7fa9c80cfed0 2024-04-25T09:15:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.497+0000 7fa9ce800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c806ce50 con 0x7fa9c81f0080 2024-04-25T09:15:53.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.497+0000 7fa9ce800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c80b2f90 con 0x7fa9c80709c0 2024-04-25T09:15:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.497+0000 7fa9c7400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9c80709c0 0x7fa9c80ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.497+0000 7fa9c7400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9c80709c0 0x7fa9c80ab7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:41612/0 (socket says 172.21.15.62:41612) 2024-04-25T09:15:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.497+0000 7fa9c7400640 1 -- 172.21.15.62:0/3416017672 learned_addr learned my addr 172.21.15.62:0/3416017672 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.497+0000 7fa9c6a00640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c80cfed0 0x7fa9c81eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.498+0000 7fa9c7e00640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c81f0080 0x7fa9c80ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.498+0000 7fa9c7400640 1 -- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c81f0080 msgr2=0x7fa9c80ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.499+0000 7fa9c7400640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c81f0080 0x7fa9c80ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.499+0000 7fa9c7400640 1 -- 172.21.15.62:0/3416017672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c80cfed0 msgr2=0x7fa9c81eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.499+0000 7fa9c7400640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c80cfed0 0x7fa9c81eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.499+0000 7fa9c7400640 1 -- 172.21.15.62:0/3416017672 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c80ac010 con 0x7fa9c80709c0 2024-04-25T09:15:53.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.499+0000 7fa9c6a00640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c80cfed0 0x7fa9c81eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:15:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.500+0000 7fa9c7400640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9c80709c0 0x7fa9c80ab7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0035040 tx=0x7fa9b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.500+0000 7fa9bf400640 1 -- 172.21.15.62:0/3416017672 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0041030 con 0x7fa9c80709c0 2024-04-25T09:15:53.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.500+0000 7fa9bf400640 1 -- 172.21.15.62:0/3416017672 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b003d4e0 con 0x7fa9c80709c0 2024-04-25T09:15:53.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.500+0000 7fa9bf400640 1 -- 172.21.15.62:0/3416017672 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0037040 con 0x7fa9c80709c0 2024-04-25T09:15:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.502+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c80ac2f0 con 0x7fa9c80709c0 2024-04-25T09:15:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.502+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c80ac8b0 con 0x7fa9c80709c0 2024-04-25T09:15:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.504+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c8072c70 con 0x7fa9c80709c0 2024-04-25T09:15:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.506+0000 7fa9bf400640 1 -- 172.21.15.62:0/3416017672 <== mon.2 v2:172.21.15.121:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9b0044070 con 0x7fa9c80709c0 2024-04-25T09:15:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.508+0000 7fa9bf400640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa9a406b4d0 0x7fa9a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.508+0000 7fa9bf400640 1 -- 172.21.15.62:0/3416017672 <== mon.2 v2:172.21.15.121:3301/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa9b00ab250 con 0x7fa9c80709c0 2024-04-25T09:15:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.509+0000 7fa9c6a00640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa9a406b4d0 0x7fa9a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.510+0000 7fa9c6a00640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa9a406b4d0 0x7fa9a406d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80ae5c0 tx=0x7fa9b8007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:53.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.521+0000 7fa9bf400640 1 -- 172.21.15.62:0/3416017672 <== mon.2 v2:172.21.15.121:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9b007a3b0 con 0x7fa9c80709c0 2024-04-25T09:15:53.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.723+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9c804f000 con 0x7fa9c80709c0 2024-04-25T09:15:53.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.725+0000 7fa9bf400640 1 -- 172.21.15.62:0/3416017672 <== mon.2 v2:172.21.15.121:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+13018 (secure 0 0 0) 0x7fa9b0079cb0 con 0x7fa9c80709c0 2024-04-25T09:15:53.712 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:53.712 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":128,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:15:52.218179+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/2051823365":"2024-04-26T09:15:49.207070+0000","172.21.15.62:6834/2051823365":"2024-04-26T09:15:49.207070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:15:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa9a406b4d0 msgr2=0x7fa9a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa9a406b4d0 0x7fa9a406d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80ae5c0 tx=0x7fa9b8007d30 comp rx=0 tx=0).stop 2024-04-25T09:15:53.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9c80709c0 msgr2=0x7fa9c80ab7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9c80709c0 0x7fa9c80ab7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0035040 tx=0x7fa9b00046c0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 shutdown_connections 2024-04-25T09:15:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa9a406b4d0 0x7fa9a406d990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c80cfed0 0x7fa9c81eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa9c81f0080 0x7fa9c80ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 --2- 172.21.15.62:0/3416017672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9c80709c0 0x7fa9c80ab7c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 >> 172.21.15.62:0/3416017672 conn(0x7fa9c80668f0 msgr2=0x7fa9c80ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 shutdown_connections 2024-04-25T09:15:53.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.728+0000 7fa9ce800640 1 -- 172.21.15.62:0/3416017672 wait complete. 2024-04-25T09:15:53.735 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:15:53.735 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:15:53.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.966+0000 7f4fa6e00640 1 Processor -- start 2024-04-25T09:15:53.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.966+0000 7f4fa6e00640 1 -- start start 2024-04-25T09:15:53.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.967+0000 7f4fa6e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b2d80 0x7f4fa00b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.968+0000 7f4fa6e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fa00b54d0 0x7f4fa0164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.968+0000 7f4fa6e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa00b4430 con 0x7f4fa00b36a0 2024-04-25T09:15:53.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.968+0000 7f4fa6e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa00b4710 con 0x7f4fa00b54d0 2024-04-25T09:15:53.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.968+0000 7f4fa6e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00b49f0 con 0x7f4fa00b2d80 2024-04-25T09:15:53.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.968+0000 7f4fa6400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fa00b54d0 0x7f4fa0164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.968+0000 7f4fa6400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fa00b54d0 0x7f4fa0164310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41284/0 (socket says 172.21.15.62:41284) 2024-04-25T09:15:53.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.968+0000 7f4fa6400640 1 -- 172.21.15.62:0/2218289028 learned_addr learned my addr 172.21.15.62:0/2218289028 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:53.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.968+0000 7f4fa5a00640 1 --2- 172.21.15.62:0/2218289028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b2d80 0x7f4fa00b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.969+0000 7f4fa6400640 1 -- 172.21.15.62:0/2218289028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b2d80 msgr2=0x7f4fa00b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:53.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.969+0000 7f4fa6400640 1 --2- 172.21.15.62:0/2218289028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b2d80 0x7f4fa00b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.969+0000 7f4fa6400640 1 -- 172.21.15.62:0/2218289028 >> v1:172.21.15.121:6789/0 conn(0x7f4fa00b36a0 legacy=0x7f4fa00ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:53.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.969+0000 7f4fa6400640 1 -- 172.21.15.62:0/2218289028 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00b4cd0 con 0x7f4fa00b54d0 2024-04-25T09:15:53.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.970+0000 7f4fa6400640 1 --2- 172.21.15.62:0/2218289028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b54d0 0x7f4fa0164310 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009870 tx=0x7f4f9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd188d50796c7e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.970+0000 7f4f9fe00640 1 -- 172.21.15.62:0/2218289028 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98037070 con 0x7f4fa00b54d0 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.970+0000 7f4f9fe00640 1 -- 172.21.15.62:0/2218289028 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f98032440 con 0x7f4fa00b54d0 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.970+0000 7f4f9fe00640 1 -- 172.21.15.62:0/2218289028 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98032810 con 0x7f4fa00b54d0 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.971+0000 7f4fa6e00640 1 -- 172.21.15.62:0/2218289028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b54d0 msgr2=0x7f4fa0164310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.971+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/2218289028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b54d0 0x7f4fa0164310 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009870 tx=0x7f4f9802a210 comp rx=0 tx=0).stop 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.971+0000 7f4fa6e00640 1 -- 172.21.15.62:0/2218289028 shutdown_connections 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.971+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/2218289028 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b54d0 0x7f4fa0164310 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.971+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/2218289028 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b2d80 0x7f4fa00b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.971+0000 7f4fa6e00640 1 -- 172.21.15.62:0/2218289028 >> 172.21.15.62:0/2218289028 conn(0x7f4fa0066f20 msgr2=0x7f4fa00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.971+0000 7f4fa6e00640 1 -- 172.21.15.62:0/2218289028 shutdown_connections 2024-04-25T09:15:53.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.971+0000 7f4fa6e00640 1 -- 172.21.15.62:0/2218289028 wait complete. 2024-04-25T09:15:53.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.973+0000 7f4fa6e00640 1 Processor -- start 2024-04-25T09:15:53.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.973+0000 7f4fa6e00640 1 -- start start 2024-04-25T09:15:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa6e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa00b2d80 0x7f4fa0148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa6e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b36a0 0x7f4fa00a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa6e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b54d0 0x7f4fa00aa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa6e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00ce9c0 con 0x7f4fa00b36a0 2024-04-25T09:15:53.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa6e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00b40e0 con 0x7f4fa00b2d80 2024-04-25T09:15:53.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa6e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00ceb40 con 0x7f4fa00b54d0 2024-04-25T09:15:53.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b36a0 0x7f4fa00a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b36a0 0x7f4fa00a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41286/0 (socket says 172.21.15.62:41286) 2024-04-25T09:15:53.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5000640 1 -- 172.21.15.62:0/3032222301 learned_addr learned my addr 172.21.15.62:0/3032222301 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:53.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5a00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa00b2d80 0x7f4fa0148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5000640 1 -- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b54d0 msgr2=0x7f4fa00aa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5000640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b54d0 0x7f4fa00aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5000640 1 -- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa00b2d80 msgr2=0x7f4fa0148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:53.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5000640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa00b2d80 0x7f4fa0148dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:53.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.974+0000 7f4fa5000640 1 -- 172.21.15.62:0/3032222301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00abac0 con 0x7f4fa00b36a0 2024-04-25T09:15:53.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.975+0000 7f4fa5000640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b36a0 0x7f4fa00a9c50 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00c430 tx=0x7f4f8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:53.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.975+0000 7f4f9d600640 1 -- 172.21.15.62:0/3032222301 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011790 con 0x7f4fa00b36a0 2024-04-25T09:15:53.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.975+0000 7f4f9d600640 1 -- 172.21.15.62:0/3032222301 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c017070 con 0x7f4fa00b36a0 2024-04-25T09:15:53.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.975+0000 7f4f9d600640 1 -- 172.21.15.62:0/3032222301 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011bb0 con 0x7f4fa00b36a0 2024-04-25T09:15:53.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.975+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa00ac020 con 0x7f4fa00b36a0 2024-04-25T09:15:53.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.975+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa00ac5e0 con 0x7f4fa00b36a0 2024-04-25T09:15:53.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.976+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa006abe0 con 0x7f4fa00b36a0 2024-04-25T09:15:53.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.977+0000 7f4f9d600640 1 -- 172.21.15.62:0/3032222301 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f8c01c460 con 0x7f4fa00b36a0 2024-04-25T09:15:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.978+0000 7f4f9d600640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8006b4d0 0x7f4f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.978+0000 7f4f9d600640 1 -- 172.21.15.62:0/3032222301 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4f8c089e80 con 0x7f4fa00b36a0 2024-04-25T09:15:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.979+0000 7f4fa5a00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8006b4d0 0x7f4f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:53.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.980+0000 7f4fa5a00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8006b4d0 0x7f4f8006d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009f30 tx=0x7f4f9400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:53.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:53.983+0000 7f4f9d600640 1 -- 172.21.15.62:0/3032222301 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fa00abac0 con 0x7f4fa00b36a0 2024-04-25T09:15:54.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.184+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4fa00747f0 con 0x7f4fa00b36a0 2024-04-25T09:15:54.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.185+0000 7f4f9d600640 1 -- 172.21.15.62:0/3032222301 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f8c058f60 con 0x7f4fa00b36a0 2024-04-25T09:15:54.171 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8006b4d0 msgr2=0x7f4f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8006b4d0 0x7f4f8006d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009f30 tx=0x7f4f9400b040 comp rx=0 tx=0).stop 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b36a0 msgr2=0x7f4fa00a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b36a0 0x7f4fa00a9c50 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00c430 tx=0x7f4f8c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 shutdown_connections 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f8006b4d0 0x7f4f8006d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fa00b36a0 0x7f4fa00a9c50 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4fa00b54d0 0x7f4fa00aa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 --2- 172.21.15.62:0/3032222301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa00b2d80 0x7f4fa0148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.187+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 >> 172.21.15.62:0/3032222301 conn(0x7f4fa0066f20 msgr2=0x7f4fa0070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.188+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 shutdown_connections 2024-04-25T09:15:54.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.188+0000 7f4fa6e00640 1 -- 172.21.15.62:0/3032222301 wait complete. 2024-04-25T09:15:54.193 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T09:15:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.415+0000 7f4f72000640 1 Processor -- start 2024-04-25T09:15:54.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.416+0000 7f4f72000640 1 -- start start 2024-04-25T09:15:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.417+0000 7f4f72000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f6c0b56f0 0x7f4f6c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.417+0000 7f4f72000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c1627a0 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.417+0000 7f4f72000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c0d3110 con 0x7f4f6c0b6a90 2024-04-25T09:15:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.417+0000 7f4f72000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c0cedc0 con 0x7f4f6c0b56f0 2024-04-25T09:15:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.418+0000 7f4f70c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4f6c0b56f0 0x7f4f6c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.418+0000 7f4f71600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4f6c0b27a0 0x7f4f6c162090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39230/0 (socket says 172.21.15.62:39230) 2024-04-25T09:15:54.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.418+0000 7f4f71600640 1 -- 172.21.15.62:0/180309749 learned_addr learned my addr 172.21.15.62:0/180309749 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.418+0000 7f4f70c00640 1 -- 172.21.15.62:0/180309749 >> v1:172.21.15.121:6789/0 conn(0x7f4f6c0b6a90 legacy=0x7f4f6c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.418+0000 7f4f70c00640 1 -- 172.21.15.62:0/180309749 >> v1:172.21.15.62:6789/0 conn(0x7f4f6c0b27a0 legacy=0x7f4f6c162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.418+0000 7f4f70c00640 1 -- 172.21.15.62:0/180309749 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0d33f0 con 0x7f4f6c0b56f0 2024-04-25T09:15:54.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.418+0000 7f4f6be00640 1 -- 172.21.15.62:0/180309749 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473116110 0 0) 0x7f4f6c1627a0 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.419+0000 7f4f70c00640 1 --2- 172.21.15.62:0/180309749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b56f0 0x7f4f6c160d00 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0097a0 tx=0x7f4f5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6190a0e536255d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.419+0000 7f4f6be00640 1 -- 172.21.15.62:0/180309749 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c037070 con 0x7f4f6c0b56f0 2024-04-25T09:15:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f6be00640 1 -- 172.21.15.62:0/180309749 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c032440 con 0x7f4f6c0b56f0 2024-04-25T09:15:54.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f6be00640 1 -- 172.21.15.62:0/180309749 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c032810 con 0x7f4f6c0b56f0 2024-04-25T09:15:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f72000640 1 -- 172.21.15.62:0/180309749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b56f0 msgr2=0x7f4f6c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f72000640 1 --2- 172.21.15.62:0/180309749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b56f0 0x7f4f6c160d00 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0097a0 tx=0x7f4f5c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f72000640 1 -- 172.21.15.62:0/180309749 shutdown_connections 2024-04-25T09:15:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f72000640 1 --2- 172.21.15.62:0/180309749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b56f0 0x7f4f6c160d00 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f72000640 1 -- 172.21.15.62:0/180309749 >> 172.21.15.62:0/180309749 conn(0x7f4f6c066d10 msgr2=0x7f4f6c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f72000640 1 -- 172.21.15.62:0/180309749 shutdown_connections 2024-04-25T09:15:54.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.420+0000 7f4f72000640 1 -- 172.21.15.62:0/180309749 wait complete. 2024-04-25T09:15:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.422+0000 7f4f72000640 1 Processor -- start 2024-04-25T09:15:54.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.422+0000 7f4f72000640 1 -- start start 2024-04-25T09:15:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f72000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b27a0 0x7f4f6c14cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f72000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b56f0 0x7f4f6c1e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:54.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f72000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b6a90 0x7f4f6c15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f72000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b2d60 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f72000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0cea50 con 0x7f4f6c0b6a90 2024-04-25T09:15:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f72000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b2ee0 con 0x7f4f6c0b56f0 2024-04-25T09:15:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f70c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b27a0 0x7f4f6c14cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f70c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b27a0 0x7f4f6c14cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:41304/0 (socket says 172.21.15.62:41304) 2024-04-25T09:15:54.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.423+0000 7f4f70c00640 1 -- 172.21.15.62:0/1024005945 learned_addr learned my addr 172.21.15.62:0/1024005945 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f70c00640 1 -- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b56f0 msgr2=0x7f4f6c1e90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f70c00640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b56f0 0x7f4f6c1e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f70c00640 1 -- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b6a90 msgr2=0x7f4f6c15aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f70c00640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b6a90 0x7f4f6c15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f70c00640 1 -- 172.21.15.62:0/1024005945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c1e9ae0 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f70c00640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b27a0 0x7f4f6c14cc70 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0097a0 tx=0x7f4f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f69600640 1 -- 172.21.15.62:0/1024005945 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c041070 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f69600640 1 -- 172.21.15.62:0/1024005945 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c03c870 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f69600640 1 -- 172.21.15.62:0/1024005945 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c037040 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.424+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c1e9dc0 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.425+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c14d1b0 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.426+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c04ef20 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.427+0000 7f4f69600640 1 -- 172.21.15.62:0/1024005945 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f5c044070 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.428+0000 7f4f69600640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4806b500 0x7f4f4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.428+0000 7f4f69600640 1 -- 172.21.15.62:0/1024005945 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4f5c0aaa90 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.428+0000 7f4f63e00640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4806b500 0x7f4f4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:54.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.430+0000 7f4f63e00640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4806b500 0x7f4f4806d9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4f54006fd0 tx=0x7f4f54006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:54.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.432+0000 7f4f69600640 1 -- 172.21.15.62:0/1024005945 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f5c079c00 con 0x7f4f6c0b27a0 2024-04-25T09:15:54.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:54.704+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f4f6c0b6180 con 0x7f4f6c0b27a0 2024-04-25T09:15:55.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.250+0000 7f4f69600640 1 -- 172.21.15.62:0/1024005945 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v568) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4f5c079500 con 0x7f4f6c0b27a0 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.253+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4806b500 msgr2=0x7f4f4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.253+0000 7f4f72000640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4806b500 0x7f4f4806d9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4f54006fd0 tx=0x7f4f54006bc0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b27a0 msgr2=0x7f4f6c14cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b27a0 0x7f4f6c14cc70 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0097a0 tx=0x7f4f5c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 shutdown_connections 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4f4806b500 0x7f4f4806d9c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f6c0b6a90 0x7f4f6c15aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4f6c0b56f0 0x7f4f6c1e90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 --2- 172.21.15.62:0/1024005945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f6c0b27a0 0x7f4f6c14cc70 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 >> 172.21.15.62:0/1024005945 conn(0x7f4f6c066d10 msgr2=0x7f4f6c0b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:55.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 shutdown_connections 2024-04-25T09:15:55.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.254+0000 7f4f72000640 1 -- 172.21.15.62:0/1024005945 wait complete. 2024-04-25T09:15:55.264 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:15:55.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.496+0000 7f1c6ce00640 1 Processor -- start 2024-04-25T09:15:55.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.497+0000 7f1c6ce00640 1 -- start start 2024-04-25T09:15:55.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.498+0000 7f1c6ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1c6815c520 0x7f1c680b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:55.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.498+0000 7f1c6ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 0x7f1c6806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:55.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c6ce00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c680b4640 con 0x7f1c68166720 2024-04-25T09:15:55.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c6ce00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c68165c40 con 0x7f1c6815c520 2024-04-25T09:15:55.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c6ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68165f20 con 0x7f1c680b3360 2024-04-25T09:15:55.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c67400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1c6815c520 0x7f1c680b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:55.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c67400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1c6815c520 0x7f1c680b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39562/0 (socket says 172.21.15.62:39562) 2024-04-25T09:15:55.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c67400640 1 -- 172.21.15.62:0/2695829970 learned_addr learned my addr 172.21.15.62:0/2695829970 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:55.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c66a00640 1 --2- 172.21.15.62:0/2695829970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 0x7f1c6806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:55.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c67400640 1 -- 172.21.15.62:0/2695829970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 msgr2=0x7f1c6806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:55.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c67400640 1 --2- 172.21.15.62:0/2695829970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 0x7f1c6806d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c67400640 1 -- 172.21.15.62:0/2695829970 >> v1:172.21.15.121:6789/0 conn(0x7f1c68166720 legacy=0x7f1c680ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:55.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.499+0000 7f1c67400640 1 -- 172.21.15.62:0/2695829970 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68166200 con 0x7f1c6815c520 2024-04-25T09:15:55.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.500+0000 7f1c67400640 1 --2- 172.21.15.62:0/2695829970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c6815c520 0x7f1c680b3d10 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009e50 tx=0x7f1c5802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a721091c37e3fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:55.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.501+0000 7f1c66000640 1 -- 172.21.15.62:0/2695829970 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58036070 con 0x7f1c6815c520 2024-04-25T09:15:55.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.501+0000 7f1c66000640 1 -- 172.21.15.62:0/2695829970 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c58031600 con 0x7f1c6815c520 2024-04-25T09:15:55.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.501+0000 7f1c66000640 1 -- 172.21.15.62:0/2695829970 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c580319d0 con 0x7f1c6815c520 2024-04-25T09:15:55.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.502+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2695829970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c6815c520 msgr2=0x7f1c680b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:55.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.502+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2695829970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c6815c520 0x7f1c680b3d10 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009e50 tx=0x7f1c5802a640 comp rx=0 tx=0).stop 2024-04-25T09:15:55.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.502+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2695829970 shutdown_connections 2024-04-25T09:15:55.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.502+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2695829970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 0x7f1c6806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.502+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2695829970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c6815c520 0x7f1c680b3d10 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.502+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2695829970 >> 172.21.15.62:0/2695829970 conn(0x7f1c68066f20 msgr2=0x7f1c680b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:55.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.502+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2695829970 shutdown_connections 2024-04-25T09:15:55.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.502+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2695829970 wait complete. 2024-04-25T09:15:55.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.504+0000 7f1c6ce00640 1 Processor -- start 2024-04-25T09:15:55.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.504+0000 7f1c6ce00640 1 -- start start 2024-04-25T09:15:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c6ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 0x7f1c68155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c6ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c6815c520 0x7f1c6815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c6ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c68166720 0x7f1c6815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c6ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c680b4250 con 0x7f1c68166720 2024-04-25T09:15:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c6ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6815c900 con 0x7f1c6815c520 2024-04-25T09:15:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c6ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c680b43d0 con 0x7f1c680b3360 2024-04-25T09:15:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c68166720 0x7f1c6815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c67e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c68166720 0x7f1c6815ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39572/0 (socket says 172.21.15.62:39572) 2024-04-25T09:15:55.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c67e00640 1 -- 172.21.15.62:0/2557845675 learned_addr learned my addr 172.21.15.62:0/2557845675 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c67400640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 0x7f1c68155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.505+0000 7f1c66a00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c6815c520 0x7f1c6815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c67e00640 1 -- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 msgr2=0x7f1c68155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c67e00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 0x7f1c68155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c67e00640 1 -- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c6815c520 msgr2=0x7f1c6815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c67e00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c6815c520 0x7f1c6815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c67e00640 1 -- 172.21.15.62:0/2557845675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c681f7f10 con 0x7f1c68166720 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c67e00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c68166720 0x7f1c6815ad30 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000b440 tx=0x7f1c6000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c5f400640 1 -- 172.21.15.62:0/2557845675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60015020 con 0x7f1c68166720 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c5f400640 1 -- 172.21.15.62:0/2557845675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c60011830 con 0x7f1c68166720 2024-04-25T09:15:55.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c5f400640 1 -- 172.21.15.62:0/2557845675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60011c20 con 0x7f1c68166720 2024-04-25T09:15:55.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c681ebcc0 con 0x7f1c68166720 2024-04-25T09:15:55.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.506+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c681ec280 con 0x7f1c68166720 2024-04-25T09:15:55.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.507+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6806abe0 con 0x7f1c68166720 2024-04-25T09:15:55.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.509+0000 7f1c5f400640 1 -- 172.21.15.62:0/2557845675 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c60018020 con 0x7f1c68166720 2024-04-25T09:15:55.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.509+0000 7f1c5f400640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1c4806b4d0 0x7f1c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:55.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.510+0000 7f1c5f400640 1 -- 172.21.15.62:0/2557845675 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1c60089be0 con 0x7f1c68166720 2024-04-25T09:15:55.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.510+0000 7f1c67400640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1c4806b4d0 0x7f1c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:55.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.511+0000 7f1c67400640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1c4806b4d0 0x7f1c4806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1c58033000 tx=0x7f1c58009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:55.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.514+0000 7f1c5f400640 1 -- 172.21.15.62:0/2557845675 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c6001f030 con 0x7f1c68166720 2024-04-25T09:15:55.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.716+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1c680af4d0 con 0x7f1c68166720 2024-04-25T09:15:55.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.722+0000 7f1c5f400640 1 -- 172.21.15.62:0/2557845675 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c600172c0 con 0x7f1c68166720 2024-04-25T09:15:55.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1c4806b4d0 msgr2=0x7f1c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:55.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1c4806b4d0 0x7f1c4806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1c58033000 tx=0x7f1c58009000 comp rx=0 tx=0).stop 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c68166720 msgr2=0x7f1c6815ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c68166720 0x7f1c6815ad30 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000b440 tx=0x7f1c6000b900 comp rx=0 tx=0).stop 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 shutdown_connections 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1c4806b4d0 0x7f1c4806d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1c68166720 0x7f1c6815ad30 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c6815c520 0x7f1c6815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 --2- 172.21.15.62:0/2557845675 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1c680b3360 0x7f1c68155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 >> 172.21.15.62:0/2557845675 conn(0x7f1c68066f20 msgr2=0x7f1c6806ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 shutdown_connections 2024-04-25T09:15:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:55.725+0000 7f1c6ce00640 1 -- 172.21.15.62:0/2557845675 wait complete. 2024-04-25T09:15:55.730 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:15:55.731 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:15:55.839 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:15:55.971 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:15:55.971 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:15:55.971 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:56.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:15:56.032 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:15:56.032 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:56.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:15:56.087 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:15:56.088 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:56.147 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:15:56.147 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:15:56.147 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:15:56.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:15:56.202 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:15:56.202 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:15:56.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.431+0000 7f3aab200640 1 Processor -- start 2024-04-25T09:15:56.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.431+0000 7f3aab200640 1 -- start start 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.433+0000 7f3aab200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac06d900 0x7f3aac15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.433+0000 7f3aab200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3aac0cee40 0x7f3aac0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.433+0000 7f3aab200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac0cf2d0 con 0x7f3aac144c10 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.433+0000 7f3aab200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac144ff0 con 0x7f3aac0cee40 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.433+0000 7f3aab200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac160340 con 0x7f3aac06d900 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.433+0000 7f3aaa800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3aac0cee40 0x7f3aac0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.433+0000 7f3aaa800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3aac0cee40 0x7f3aac0b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39578/0 (socket says 172.21.15.62:39578) 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.433+0000 7f3aaa800640 1 -- 172.21.15.62:0/2719500468 learned_addr learned my addr 172.21.15.62:0/2719500468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.434+0000 7f3aa3e00640 1 --2- 172.21.15.62:0/2719500468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac06d900 0x7f3aac15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.434+0000 7f3aaa800640 1 -- 172.21.15.62:0/2719500468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac06d900 msgr2=0x7f3aac15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:56.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.434+0000 7f3aaa800640 1 --2- 172.21.15.62:0/2719500468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac06d900 0x7f3aac15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.434+0000 7f3aaa800640 1 -- 172.21.15.62:0/2719500468 >> v1:172.21.15.121:6789/0 conn(0x7f3aac144c10 legacy=0x7f3aac06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:56.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.434+0000 7f3aaa800640 1 -- 172.21.15.62:0/2719500468 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac15faa0 con 0x7f3aac0cee40 2024-04-25T09:15:56.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.435+0000 7f3aaa800640 1 --2- 172.21.15.62:0/2719500468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac0cee40 0x7f3aac0b4650 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009e50 tx=0x7f3aa402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13e566b424112a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:56.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.435+0000 7f3aa9400640 1 -- 172.21.15.62:0/2719500468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4036070 con 0x7f3aac0cee40 2024-04-25T09:15:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.435+0000 7f3aa9400640 1 -- 172.21.15.62:0/2719500468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4031600 con 0x7f3aac0cee40 2024-04-25T09:15:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.435+0000 7f3aa9400640 1 -- 172.21.15.62:0/2719500468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40319d0 con 0x7f3aac0cee40 2024-04-25T09:15:56.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.436+0000 7f3aab200640 1 -- 172.21.15.62:0/2719500468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac0cee40 msgr2=0x7f3aac0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.436+0000 7f3aab200640 1 --2- 172.21.15.62:0/2719500468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac0cee40 0x7f3aac0b4650 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009e50 tx=0x7f3aa402a630 comp rx=0 tx=0).stop 2024-04-25T09:15:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.436+0000 7f3aab200640 1 -- 172.21.15.62:0/2719500468 shutdown_connections 2024-04-25T09:15:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.436+0000 7f3aab200640 1 --2- 172.21.15.62:0/2719500468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac0cee40 0x7f3aac0b4650 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.436+0000 7f3aab200640 1 --2- 172.21.15.62:0/2719500468 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac06d900 0x7f3aac15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.436+0000 7f3aab200640 1 -- 172.21.15.62:0/2719500468 >> 172.21.15.62:0/2719500468 conn(0x7f3aac0b14d0 msgr2=0x7f3aac0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.436+0000 7f3aab200640 1 -- 172.21.15.62:0/2719500468 shutdown_connections 2024-04-25T09:15:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.436+0000 7f3aab200640 1 -- 172.21.15.62:0/2719500468 wait complete. 2024-04-25T09:15:56.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.438+0000 7f3aab200640 1 Processor -- start 2024-04-25T09:15:56.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.439+0000 7f3aab200640 1 -- start start 2024-04-25T09:15:56.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aab200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac06d900 0x7f3aac15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aab200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aac0cee40 0x7f3aac1e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aab200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac144c10 0x7f3aac1e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aab200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac160060 con 0x7f3aac06d900 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aab200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0b41c0 con 0x7f3aac0cee40 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aab200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0b4b90 con 0x7f3aac144c10 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aa9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac06d900 0x7f3aac15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aa9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac06d900 0x7f3aac15b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39588/0 (socket says 172.21.15.62:39588) 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aa9e00640 1 -- 172.21.15.62:0/141449784 learned_addr learned my addr 172.21.15.62:0/141449784 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.440+0000 7f3aaa800640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac144c10 0x7f3aac1e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa3e00640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aac0cee40 0x7f3aac1e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa9e00640 1 -- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac144c10 msgr2=0x7f3aac1e9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa9e00640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac144c10 0x7f3aac1e9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa9e00640 1 -- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aac0cee40 msgr2=0x7f3aac1e95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa9e00640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aac0cee40 0x7f3aac1e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa9e00640 1 -- 172.21.15.62:0/141449784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac1e8140 con 0x7f3aac06d900 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa9e00640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac06d900 0x7f3aac15b3b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00b3f0 tx=0x7f3a9c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa2000640 1 -- 172.21.15.62:0/141449784 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c015020 con 0x7f3aac06d900 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa2000640 1 -- 172.21.15.62:0/141449784 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c011830 con 0x7f3aac06d900 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aa2000640 1 -- 172.21.15.62:0/141449784 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c011c20 con 0x7f3aac06d900 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.441+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac1e8420 con 0x7f3aac06d900 2024-04-25T09:15:56.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.442+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac1e89e0 con 0x7f3aac06d900 2024-04-25T09:15:56.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.443+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac141a00 con 0x7f3aac06d900 2024-04-25T09:15:56.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.444+0000 7f3aa2000640 1 -- 172.21.15.62:0/141449784 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a9c018020 con 0x7f3aac06d900 2024-04-25T09:15:56.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.445+0000 7f3aa2000640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a8806b4d0 0x7f3a8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.445+0000 7f3aa2000640 1 -- 172.21.15.62:0/141449784 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3a9c089b80 con 0x7f3aac06d900 2024-04-25T09:15:56.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.445+0000 7f3aa3e00640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a8806b4d0 0x7f3a8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.447+0000 7f3aa3e00640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a8806b4d0 0x7f3a8806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3aac15e3f0 tx=0x7f3a94008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:56.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.450+0000 7f3aa2000640 1 -- 172.21.15.62:0/141449784 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a9c01f030 con 0x7f3aac06d900 2024-04-25T09:15:56.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.696+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3aac04ef20 con 0x7f3aac06d900 2024-04-25T09:15:56.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.698+0000 7f3aa2000640 1 -- 172.21.15.62:0/141449784 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f3a9c0172c0 con 0x7f3aac06d900 2024-04-25T09:15:56.684 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:56.684 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5894,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2405239203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2405239203},{"type":"v1","addr":"172.21.15.121:6837","nonce":2405239203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6526,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/4132315051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":4132315051},{"type":"v1","addr":"172.21.15.121:6835","nonce":4132315051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8301,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/3195938216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":3195938216},{"type":"v1","addr":"172.21.15.62:6833","nonce":3195938216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/4237762855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":4237762855},{"type":"v1","addr":"172.21.15.62:6835","nonce":4237762855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[]} 2024-04-25T09:15:56.685 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 80 2024-04-25T09:15:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a8806b4d0 msgr2=0x7f3a8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a8806b4d0 0x7f3a8806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3aac15e3f0 tx=0x7f3a94008040 comp rx=0 tx=0).stop 2024-04-25T09:15:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac06d900 msgr2=0x7f3aac15b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac06d900 0x7f3aac15b3b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00b3f0 tx=0x7f3a9c00b8b0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 shutdown_connections 2024-04-25T09:15:56.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3a8806b4d0 0x7f3a8806d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3aac144c10 0x7f3aac1e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3aac0cee40 0x7f3aac1e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 --2- 172.21.15.62:0/141449784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3aac06d900 0x7f3aac15b3b0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 >> 172.21.15.62:0/141449784 conn(0x7f3aac0b14d0 msgr2=0x7f3aac071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:56.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 shutdown_connections 2024-04-25T09:15:56.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.701+0000 7f3aab200640 1 -- 172.21.15.62:0/141449784 wait complete. 2024-04-25T09:15:56.708 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:15:56.708 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:15:56.709 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:15:56.709 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:15:56.709 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:15:56.709 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:15:56.709 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:15:56.709 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:15:56.709 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:15:56.709 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:15:56.710 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:15:56.710 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:15:56.710 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:15:56.710 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:15:56.710 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:15:56.711 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:15:56.730 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:15:56.730 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:15:56.731 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:15:56.731 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:15:56.733 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:15:56.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:15:56.742 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:15:56.742 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:15:56.780 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:15:56.781 DEBUG:teuthology.parallel:result is None 2024-04-25T09:15:56.784 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:15:56.785 DEBUG:teuthology.parallel:result is None 2024-04-25T09:15:56.787 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:15:56.787 DEBUG:teuthology.parallel:result is None 2024-04-25T09:15:56.791 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:15:56.792 DEBUG:teuthology.parallel:result is None 2024-04-25T09:15:56.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:15:56.930 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:15:56.931 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:15:56.939 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:15:56.940 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:15:56.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.988+0000 7f7cd7000640 1 Processor -- start 2024-04-25T09:15:56.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.989+0000 7f7cd7000640 1 -- start start 2024-04-25T09:15:56.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.990+0000 7f7cd7000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00d0ae0 0x7f7cd00d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.990+0000 7f7cd7000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cd00b6cf0 0x7f7cd00b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.990+0000 7f7cd7000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd00d2620 con 0x7f7cd015dc40 2024-04-25T09:15:56.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.990+0000 7f7cd7000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd00d1480 con 0x7f7cd00b6cf0 2024-04-25T09:15:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.990+0000 7f7cd7000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00d23c0 con 0x7f7cd00d0ae0 2024-04-25T09:15:56.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7cd5000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cd00b6cf0 0x7f7cd00b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7cc7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00d0ae0 0x7f7cd00d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7ccfe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7cd015dc40 0x7f7cd00d1cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:60442/0 (socket says 172.21.15.62:60442) 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7cc7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00d0ae0 0x7f7cd00d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:55306/0 (socket says 172.21.15.62:55306) 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7ccfe00640 1 -- 172.21.15.62:0/2205644837 learned_addr learned my addr 172.21.15.62:0/2205644837 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7cd5000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7cd00b6cf0 0x7f7cd00b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39614/0 (socket says 172.21.15.62:39614) 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7cc7e00640 1 -- 172.21.15.62:0/2205644837 >> v2:172.21.15.62:3300/0 conn(0x7f7cd00b6cf0 msgr2=0x7f7cd00b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7cc7e00640 1 --2- 172.21.15.62:0/2205644837 >> v2:172.21.15.62:3300/0 conn(0x7f7cd00b6cf0 0x7f7cd00b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7cc7e00640 1 -- 172.21.15.62:0/2205644837 >> v1:172.21.15.121:6789/0 conn(0x7f7cd015dc40 legacy=0x7f7cd00d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.991+0000 7f7cc7e00640 1 -- 172.21.15.62:0/2205644837 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00d2900 con 0x7f7cd00d0ae0 2024-04-25T09:15:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.992+0000 7f7cc7e00640 1 --2- 172.21.15.62:0/2205644837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00d0ae0 0x7f7cd00d0f40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097a0 tx=0x7f7cb802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b75d0db99eeff5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.992+0000 7f7ccf400640 1 -- 172.21.15.62:0/2205644837 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8037070 con 0x7f7cd00d0ae0 2024-04-25T09:15:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.992+0000 7f7ccf400640 1 -- 172.21.15.62:0/2205644837 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb8032440 con 0x7f7cd00d0ae0 2024-04-25T09:15:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.992+0000 7f7ccf400640 1 -- 172.21.15.62:0/2205644837 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8032810 con 0x7f7cd00d0ae0 2024-04-25T09:15:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.993+0000 7f7cd7000640 1 -- 172.21.15.62:0/2205644837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00d0ae0 msgr2=0x7f7cd00d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.993+0000 7f7cd7000640 1 --2- 172.21.15.62:0/2205644837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00d0ae0 0x7f7cd00d0f40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80097a0 tx=0x7f7cb802a180 comp rx=0 tx=0).stop 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.993+0000 7f7cd7000640 1 -- 172.21.15.62:0/2205644837 shutdown_connections 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.993+0000 7f7cd7000640 1 --2- 172.21.15.62:0/2205644837 >> v2:172.21.15.62:3300/0 conn(0x7f7cd00b6cf0 0x7f7cd00b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.993+0000 7f7cd7000640 1 --2- 172.21.15.62:0/2205644837 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00d0ae0 0x7f7cd00d0f40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.993+0000 7f7cd7000640 1 -- 172.21.15.62:0/2205644837 >> 172.21.15.62:0/2205644837 conn(0x7f7cd006b000 msgr2=0x7f7cd006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.993+0000 7f7cd7000640 1 -- 172.21.15.62:0/2205644837 shutdown_connections 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.993+0000 7f7cd7000640 1 -- 172.21.15.62:0/2205644837 wait complete. 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.995+0000 7f7cd7000640 1 Processor -- start 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.995+0000 7f7cd7000640 1 -- start start 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd7000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00b6cf0 0x7f7cd01598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd7000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd00d0ae0 0x7f7cd0148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd7000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cd015dc40 0x7f7cd01471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd7000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00d31a0 con 0x7f7cd015dc40 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd7000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00d2ec0 con 0x7f7cd00d0ae0 2024-04-25T09:15:56.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd7000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd00d36c0 con 0x7f7cd00b6cf0 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cd015dc40 0x7f7cd01471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cd015dc40 0x7f7cd01471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39624/0 (socket says 172.21.15.62:39624) 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.996+0000 7f7cd5000640 1 -- 172.21.15.62:0/377022146 learned_addr learned my addr 172.21.15.62:0/377022146 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7ccfe00640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00b6cf0 0x7f7cd01598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cc7e00640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd00d0ae0 0x7f7cd0148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cd5000640 1 -- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00b6cf0 msgr2=0x7f7cd01598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cd5000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00b6cf0 0x7f7cd01598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cd5000640 1 -- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd00d0ae0 msgr2=0x7f7cd0148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cd5000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd00d0ae0 0x7f7cd0148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cd5000640 1 -- 172.21.15.62:0/377022146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd015a260 con 0x7f7cd015dc40 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cc7e00640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd00d0ae0 0x7f7cd0148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7ccfe00640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00b6cf0 0x7f7cd01598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:56.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cd5000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cd015dc40 0x7f7cd01471f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800efc0 tx=0x7f7cc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cccc00640 1 -- 172.21.15.62:0/377022146 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8018020 con 0x7f7cd015dc40 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cccc00640 1 -- 172.21.15.62:0/377022146 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc8014480 con 0x7f7cd015dc40 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd015a540 con 0x7f7cd015dc40 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.997+0000 7f7cccc00640 1 -- 172.21.15.62:0/377022146 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8014870 con 0x7f7cd015dc40 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.998+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd0146cd0 con 0x7f7cd015dc40 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:56.999+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd00729c0 con 0x7f7cd015dc40 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:57.000+0000 7f7cccc00640 1 -- 172.21.15.62:0/377022146 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cc8005d10 con 0x7f7cd015dc40 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:57.001+0000 7f7cccc00640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cac06b500 0x7f7cac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:57.001+0000 7f7cccc00640 1 -- 172.21.15.62:0/377022146 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7cc808aac0 con 0x7f7cd015dc40 2024-04-25T09:15:56.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:57.001+0000 7f7ccfe00640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cac06b500 0x7f7cac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:57.002+0000 7f7ccfe00640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cac06b500 0x7f7cac06d9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005880 tx=0x7f7cc0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:56.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:57.006+0000 7f7cccc00640 1 -- 172.21.15.62:0/377022146 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cc8022070 con 0x7f7cd015dc40 2024-04-25T09:15:57.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:57.216+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7cd004f000 con 0x7f7cd015dc40 2024-04-25T09:15:58.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.292+0000 7f7cccc00640 1 -- 172.21.15.62:0/377022146 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7cc8010040 con 0x7f7cd015dc40 2024-04-25T09:15:58.279 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:15:58.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.296+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cac06b500 msgr2=0x7f7cac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.296+0000 7f7cd7000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cac06b500 0x7f7cac06d9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005880 tx=0x7f7cc0009040 comp rx=0 tx=0).stop 2024-04-25T09:15:58.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.296+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cd015dc40 msgr2=0x7f7cd01471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.296+0000 7f7cd7000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cd015dc40 0x7f7cd01471f0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800efc0 tx=0x7f7cc800c490 comp rx=0 tx=0).stop 2024-04-25T09:15:58.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.297+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 shutdown_connections 2024-04-25T09:15:58.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.297+0000 7f7cd7000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7cac06b500 0x7f7cac06d9c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.297+0000 7f7cd7000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cd015dc40 0x7f7cd01471f0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.297+0000 7f7cd7000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd00d0ae0 0x7f7cd0148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.297+0000 7f7cd7000640 1 --2- 172.21.15.62:0/377022146 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7cd00b6cf0 0x7f7cd01598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.297+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 >> 172.21.15.62:0/377022146 conn(0x7f7cd006b000 msgr2=0x7f7cd00d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:58.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.297+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 shutdown_connections 2024-04-25T09:15:58.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.297+0000 7f7cd7000640 1 -- 172.21.15.62:0/377022146 wait complete. 2024-04-25T09:15:58.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:15:58.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.526+0000 7f95e4200640 1 Processor -- start 2024-04-25T09:15:58.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.526+0000 7f95e4200640 1 -- start start 2024-04-25T09:15:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.528+0000 7f95e4200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f95dc0cd2f0 0x7f95dc0b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.528+0000 7f95e4200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0b34b0 con 0x7f95dc1454c0 2024-04-25T09:15:58.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.528+0000 7f95e4200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95dc0b3720 con 0x7f95dc0b44e0 2024-04-25T09:15:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.528+0000 7f95e4200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95dc160530 con 0x7f95dc0cd2f0 2024-04-25T09:15:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.528+0000 7f95e1800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f95dc1454c0 0x7f95dc06c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37254/0 (socket says 172.21.15.62:37254) 2024-04-25T09:15:58.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.528+0000 7f95e1800640 1 -- 172.21.15.62:0/122693132 learned_addr learned my addr 172.21.15.62:0/122693132 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95e2200640 1 --2- 172.21.15.62:0/122693132 >> v2:172.21.15.121:3300/0 conn(0x7f95dc0cd2f0 0x7f95dc0b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621980151 0 0) 0x7f95dc0b34b0 con 0x7f95dc1454c0 2024-04-25T09:15:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c4003520 con 0x7f95dc1454c0 2024-04-25T09:15:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497629789 0 0) 0x7f95c4003520 con 0x7f95dc1454c0 2024-04-25T09:15:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 >> v2:172.21.15.121:3300/0 conn(0x7f95dc0cd2f0 msgr2=0x7f95dc0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 --2- 172.21.15.62:0/122693132 >> v2:172.21.15.121:3300/0 conn(0x7f95dc0cd2f0 0x7f95dc0b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 >> v1:172.21.15.121:6789/0 conn(0x7f95dc0b44e0 legacy=0x7f95dc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0b3a00 con 0x7f95dc1454c0 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f95d0001670 con 0x7f95dc1454c0 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.529+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 >> v1:172.21.15.62:6789/0 conn(0x7f95dc1454c0 legacy=0x7f95dc06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95dbe00640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95c40039b0 0x7f95c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95dbe00640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95dbe00640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95c40051c0 0x7f95c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc160530 con 0x7f95c4004800 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c400c060 con 0x7f95c40039b0 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c400c260 con 0x7f95c40051c0 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e0e00640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e1800640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95c40039b0 0x7f95c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:58.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e2200640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95c40051c0 0x7f95c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:58.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e0e00640 1 -- 172.21.15.62:0/122693132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95c40051c0 msgr2=0x7f95c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:58.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e0e00640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95c40051c0 0x7f95c400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e0e00640 1 -- 172.21.15.62:0/122693132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95c40039b0 msgr2=0x7f95c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:58.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e0e00640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95c40039b0 0x7f95c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e0e00640 1 -- 172.21.15.62:0/122693132 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4003520 con 0x7f95c4004800 2024-04-25T09:15:58.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.530+0000 7f95e2200640 1 -- 172.21.15.62:0/122693132 reap_dead start 2024-04-25T09:15:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.531+0000 7f95e0e00640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95c4004c80 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f95c8001750 tx=0x7f95c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f18b30e80e0f5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8001370 con 0x7f95c4004800 2024-04-25T09:15:58.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c80052d0 con 0x7f95c4004800 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95dbe00640 1 -- 172.21.15.62:0/122693132 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c802abc0 con 0x7f95c4004800 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95e4200640 1 -- 172.21.15.62:0/122693132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 msgr2=0x7f95c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95e4200640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95c4004c80 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f95c8001750 tx=0x7f95c4005830 comp rx=0 tx=0).stop 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95e4200640 1 -- 172.21.15.62:0/122693132 shutdown_connections 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95e4200640 1 --2- 172.21.15.62:0/122693132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95c4004c80 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95e4200640 1 -- 172.21.15.62:0/122693132 >> 172.21.15.62:0/122693132 conn(0x7f95dc0668f0 msgr2=0x7f95dc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.532+0000 7f95e4200640 1 -- 172.21.15.62:0/122693132 shutdown_connections 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.533+0000 7f95e4200640 1 -- 172.21.15.62:0/122693132 wait complete. 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.534+0000 7f95e4200640 1 Processor -- start 2024-04-25T09:15:58.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.534+0000 7f95e4200640 1 -- start start 2024-04-25T09:15:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.535+0000 7f95e4200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95dc15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.535+0000 7f95e4200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95dc0ad890 0x7f95dc1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.535+0000 7f95e4200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95dc0ab870 0x7f95dc0abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:58.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.535+0000 7f95e4200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0b4940 con 0x7f95c4004800 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.535+0000 7f95e4200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0b3fc0 con 0x7f95dc0ad890 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.535+0000 7f95e4200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc0b4ac0 con 0x7f95dc0ab870 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95dc15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95dc15b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39642/0 (socket says 172.21.15.62:39642) 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 -- 172.21.15.62:0/2837363705 learned_addr learned my addr 172.21.15.62:0/2837363705 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 -- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95dc0ab870 msgr2=0x7f95dc0abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95dc0ab870 0x7f95dc0abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 -- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95dc0ad890 msgr2=0x7f95dc1f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95dc0ad890 0x7f95dc1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 -- 172.21.15.62:0/2837363705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc1f5a50 con 0x7f95c4004800 2024-04-25T09:15:58.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.536+0000 7f95e1800640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95dc15b370 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e730 tx=0x7f95d000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.537+0000 7f95d9600640 1 -- 172.21.15.62:0/2837363705 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000c640 con 0x7f95c4004800 2024-04-25T09:15:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.537+0000 7f95d9600640 1 -- 172.21.15.62:0/2837363705 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d0018070 con 0x7f95c4004800 2024-04-25T09:15:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.537+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc0add30 con 0x7f95c4004800 2024-04-25T09:15:58.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.537+0000 7f95d9600640 1 -- 172.21.15.62:0/2837363705 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000ca60 con 0x7f95c4004800 2024-04-25T09:15:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.537+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95dc0ae2f0 con 0x7f95c4004800 2024-04-25T09:15:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.538+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc04ef20 con 0x7f95c4004800 2024-04-25T09:15:58.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.539+0000 7f95d9600640 1 -- 172.21.15.62:0/2837363705 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95d001d460 con 0x7f95c4004800 2024-04-25T09:15:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.540+0000 7f95d9600640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f95bc06b4d0 0x7f95bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:58.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.540+0000 7f95d9600640 1 -- 172.21.15.62:0/2837363705 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f95d0089ec0 con 0x7f95c4004800 2024-04-25T09:15:58.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.541+0000 7f95e0e00640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f95bc06b4d0 0x7f95bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:58.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.542+0000 7f95e0e00640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f95bc06b4d0 0x7f95bc06d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f95c8032000 tx=0x7f95c802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:58.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.545+0000 7f95d9600640 1 -- 172.21.15.62:0/2837363705 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95dc1f5a50 con 0x7f95c4004800 2024-04-25T09:15:58.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.819+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f95dc0b26e0 con 0x7f95c4004800 2024-04-25T09:15:58.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.820+0000 7f95d9600640 1 -- 172.21.15.62:0/2837363705 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f95d0059090 con 0x7f95c4004800 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.807 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.808 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.809 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.810 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.811 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:15:58.812 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:15:58.813 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.814 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:15:58.815 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.816 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:15:58.817 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:15:58.818 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:15:58.819 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:15:58.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f95bc06b4d0 msgr2=0x7f95bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f95bc06b4d0 0x7f95bc06d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f95c8032000 tx=0x7f95c802c000 comp rx=0 tx=0).stop 2024-04-25T09:15:58.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 msgr2=0x7f95dc15b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:58.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95dc15b370 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e730 tx=0x7f95d000ebf0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 shutdown_connections 2024-04-25T09:15:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f95bc06b4d0 0x7f95bc06d990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f95dc0ab870 0x7f95dc0abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95dc0ad890 0x7f95dc1f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 --2- 172.21.15.62:0/2837363705 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95c4004800 0x7f95dc15b370 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 >> 172.21.15.62:0/2837363705 conn(0x7f95dc0668f0 msgr2=0x7f95dc1446e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.823+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 shutdown_connections 2024-04-25T09:15:58.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:58.824+0000 7f95e4200640 1 -- 172.21.15.62:0/2837363705 wait complete. 2024-04-25T09:15:58.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:15:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.056+0000 7fd8c3200640 1 Processor -- start 2024-04-25T09:15:59.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.056+0000 7fd8c3200640 1 -- start start 2024-04-25T09:15:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c3200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8c4144c10 0x7fd8c415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd8c40cee40 0x7fd8c406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c406d730 0x7fd8c40b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c3200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c4144ff0 con 0x7fd8c4144c10 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c4160340 con 0x7fd8c40cee40 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b41c0 con 0x7fd8c406d730 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8c4144c10 0x7fd8c415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8c4144c10 0x7fd8c415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39658/0 (socket says 172.21.15.62:39658) 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c1e00640 1 -- 172.21.15.62:0/1546477873 learned_addr learned my addr 172.21.15.62:0/1546477873 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8b9400640 1 --2- 172.21.15.62:0/1546477873 >> v2:172.21.15.121:3300/0 conn(0x7fd8c40cee40 0x7fd8c406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.058+0000 7fd8c2800640 1 --2- 172.21.15.62:0/1546477873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c406d730 0x7fd8c40b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.059+0000 7fd8c1e00640 1 -- 172.21.15.62:0/1546477873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c406d730 msgr2=0x7fd8c40b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.059+0000 7fd8c1e00640 1 --2- 172.21.15.62:0/1546477873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c406d730 0x7fd8c40b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.059+0000 7fd8c1e00640 1 -- 172.21.15.62:0/1546477873 >> v2:172.21.15.121:3300/0 conn(0x7fd8c40cee40 msgr2=0x7fd8c406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.059+0000 7fd8c1e00640 1 --2- 172.21.15.62:0/1546477873 >> v2:172.21.15.121:3300/0 conn(0x7fd8c40cee40 0x7fd8c406d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.059+0000 7fd8c1e00640 1 -- 172.21.15.62:0/1546477873 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40cf2d0 con 0x7fd8c4144c10 2024-04-25T09:15:59.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.060+0000 7fd8c1e00640 1 --2- 172.21.15.62:0/1546477873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c415f450 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fd8c406e680 tx=0x7fd8b402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85527e04c0570f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.060+0000 7fd8c1400640 1 -- 172.21.15.62:0/1546477873 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4037070 con 0x7fd8c4144c10 2024-04-25T09:15:59.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.060+0000 7fd8c1400640 1 -- 172.21.15.62:0/1546477873 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4032440 con 0x7fd8c4144c10 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.060+0000 7fd8c1400640 1 -- 172.21.15.62:0/1546477873 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4032810 con 0x7fd8c4144c10 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 -- 172.21.15.62:0/1546477873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 msgr2=0x7fd8c415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 --2- 172.21.15.62:0/1546477873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c415f450 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fd8c406e680 tx=0x7fd8b402a140 comp rx=0 tx=0).stop 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 -- 172.21.15.62:0/1546477873 shutdown_connections 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 --2- 172.21.15.62:0/1546477873 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c406d730 0x7fd8c40b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 --2- 172.21.15.62:0/1546477873 >> v2:172.21.15.121:3300/0 conn(0x7fd8c40cee40 0x7fd8c406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 --2- 172.21.15.62:0/1546477873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c415f450 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 -- 172.21.15.62:0/1546477873 >> 172.21.15.62:0/1546477873 conn(0x7fd8c40b14d0 msgr2=0x7fd8c40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 -- 172.21.15.62:0/1546477873 shutdown_connections 2024-04-25T09:15:59.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.061+0000 7fd8c3200640 1 -- 172.21.15.62:0/1546477873 wait complete. 2024-04-25T09:15:59.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.063+0000 7fd8c3200640 1 Processor -- start 2024-04-25T09:15:59.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.063+0000 7fd8c3200640 1 -- start start 2024-04-25T09:15:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.064+0000 7fd8c3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c406d730 0x7fd8c41415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.064+0000 7fd8c3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c40cee40 0x7fd8c412de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.064+0000 7fd8c3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c412fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.064+0000 7fd8c3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4160060 con 0x7fd8c4144c10 2024-04-25T09:15:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.064+0000 7fd8c3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c415fd80 con 0x7fd8c406d730 2024-04-25T09:15:59.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.064+0000 7fd8c3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b32e0 con 0x7fd8c40cee40 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.064+0000 7fd8c2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c412fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c412fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39674/0 (socket says 172.21.15.62:39674) 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c2800640 1 -- 172.21.15.62:0/832980610 learned_addr learned my addr 172.21.15.62:0/832980610 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8b9400640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c40cee40 0x7fd8c412de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c1e00640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c406d730 0x7fd8c41415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c2800640 1 -- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c40cee40 msgr2=0x7fd8c412de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c2800640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c40cee40 0x7fd8c412de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c2800640 1 -- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c406d730 msgr2=0x7fd8c41415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c2800640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c406d730 0x7fd8c41415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c2800640 1 -- 172.21.15.62:0/832980610 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c4130830 con 0x7fd8c4144c10 2024-04-25T09:15:59.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.065+0000 7fd8c2800640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c412fec0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc007e90 tx=0x7fd8bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.066+0000 7fd8baa00640 1 -- 172.21.15.62:0/832980610 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc014420 con 0x7fd8c4144c10 2024-04-25T09:15:59.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.066+0000 7fd8baa00640 1 -- 172.21.15.62:0/832980610 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc019070 con 0x7fd8c4144c10 2024-04-25T09:15:59.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.066+0000 7fd8baa00640 1 -- 172.21.15.62:0/832980610 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc014820 con 0x7fd8c4144c10 2024-04-25T09:15:59.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.066+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c4130b10 con 0x7fd8c4144c10 2024-04-25T09:15:59.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.066+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c41433e0 con 0x7fd8c4144c10 2024-04-25T09:15:59.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.067+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c404ef20 con 0x7fd8c4144c10 2024-04-25T09:15:59.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.068+0000 7fd8baa00640 1 -- 172.21.15.62:0/832980610 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8bc0149c0 con 0x7fd8c4144c10 2024-04-25T09:15:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.069+0000 7fd8baa00640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a006b4d0 0x7fd8a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.069+0000 7fd8baa00640 1 -- 172.21.15.62:0/832980610 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd8bc08a870 con 0x7fd8c4144c10 2024-04-25T09:15:59.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.070+0000 7fd8c1e00640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a006b4d0 0x7fd8a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.071+0000 7fd8c1e00640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a006b4d0 0x7fd8a006d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4034000 tx=0x7fd8b402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.073+0000 7fd8baa00640 1 -- 172.21.15.62:0/832980610 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8bc059910 con 0x7fd8c4144c10 2024-04-25T09:15:59.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.274+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8c4142ed0 con 0x7fd8c4144c10 2024-04-25T09:15:59.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.276+0000 7fd8baa00640 1 -- 172.21.15.62:0/832980610 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+12889 (secure 0 0 0) 0x7fd8bc059210 con 0x7fd8c4144c10 2024-04-25T09:15:59.263 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:15:59.263 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":133,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:15:58.289088+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a006b4d0 msgr2=0x7fd8a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a006b4d0 0x7fd8a006d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4034000 tx=0x7fd8b402e000 comp rx=0 tx=0).stop 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 msgr2=0x7fd8c412fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c412fec0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc007e90 tx=0x7fd8bc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 shutdown_connections 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd8a006b4d0 0x7fd8a006d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c4144c10 0x7fd8c412fec0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd8c40cee40 0x7fd8c412de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 --2- 172.21.15.62:0/832980610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8c406d730 0x7fd8c41415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 >> 172.21.15.62:0/832980610 conn(0x7fd8c40b14d0 msgr2=0x7fd8c4071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:59.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 shutdown_connections 2024-04-25T09:15:59.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.279+0000 7fd8c3200640 1 -- 172.21.15.62:0/832980610 wait complete. 2024-04-25T09:15:59.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:15:59.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:15:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.512+0000 7f1a03200640 1 Processor -- start 2024-04-25T09:15:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.513+0000 7f1a03200640 1 -- start start 2024-04-25T09:15:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.514+0000 7f1a03200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1a0406d2c0 0x7f1a040ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.514+0000 7f1a03200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a0406d6a0 0x7f1a040ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.514+0000 7f1a03200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a040ced40 con 0x7f1a040b31a0 2024-04-25T09:15:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.514+0000 7f1a03200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a040cf000 con 0x7f1a0406d6a0 2024-04-25T09:15:59.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.514+0000 7f1a03200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a040cf2e0 con 0x7f1a0406d2c0 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a01400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1a0406d6a0 0x7f1a040ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a02800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1a040b31a0 0x7f1a040b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37270/0 (socket says 172.21.15.62:37270) 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a02800640 1 -- 172.21.15.62:0/1532197787 learned_addr learned my addr 172.21.15.62:0/1532197787 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a01e00640 1 --2- 172.21.15.62:0/1532197787 >> v2:172.21.15.121:3300/0 conn(0x7f1a0406d2c0 0x7f1a040ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a01400640 1 -- 172.21.15.62:0/1532197787 >> v2:172.21.15.121:3300/0 conn(0x7f1a0406d2c0 msgr2=0x7f1a040ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a01400640 1 --2- 172.21.15.62:0/1532197787 >> v2:172.21.15.121:3300/0 conn(0x7f1a0406d2c0 0x7f1a040ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a01400640 1 -- 172.21.15.62:0/1532197787 >> v1:172.21.15.62:6789/0 conn(0x7f1a040b31a0 legacy=0x7f1a040b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a00a00640 1 -- 172.21.15.62:0/1532197787 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190156224 0 0) 0x7f1a040ced40 con 0x7f1a040b31a0 2024-04-25T09:15:59.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.515+0000 7f1a01400640 1 -- 172.21.15.62:0/1532197787 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040b26c0 con 0x7f1a0406d6a0 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.517+0000 7f1a01400640 1 --2- 172.21.15.62:0/1532197787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0406d6a0 0x7f1a040ce800 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1a040b20a0 tx=0x7f19f002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e58317ae6d2ac85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.517+0000 7f1a00a00640 1 -- 172.21.15.62:0/1532197787 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0037070 con 0x7f1a0406d6a0 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.517+0000 7f1a00a00640 1 -- 172.21.15.62:0/1532197787 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f0032440 con 0x7f1a0406d6a0 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.517+0000 7f1a00a00640 1 -- 172.21.15.62:0/1532197787 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0032810 con 0x7f1a0406d6a0 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.517+0000 7f1a03200640 1 -- 172.21.15.62:0/1532197787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0406d6a0 msgr2=0x7f1a040ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.517+0000 7f1a03200640 1 --2- 172.21.15.62:0/1532197787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0406d6a0 0x7f1a040ce800 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f1a040b20a0 tx=0x7f19f002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.518+0000 7f1a03200640 1 -- 172.21.15.62:0/1532197787 shutdown_connections 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.518+0000 7f1a03200640 1 --2- 172.21.15.62:0/1532197787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a0406d6a0 0x7f1a040ce800 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.518+0000 7f1a03200640 1 --2- 172.21.15.62:0/1532197787 >> v2:172.21.15.121:3300/0 conn(0x7f1a0406d2c0 0x7f1a040ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.518+0000 7f1a03200640 1 -- 172.21.15.62:0/1532197787 >> 172.21.15.62:0/1532197787 conn(0x7f1a04066f20 msgr2=0x7f1a040b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.518+0000 7f1a03200640 1 -- 172.21.15.62:0/1532197787 shutdown_connections 2024-04-25T09:15:59.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.518+0000 7f1a03200640 1 -- 172.21.15.62:0/1532197787 wait complete. 2024-04-25T09:15:59.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.519+0000 7f1a03200640 1 Processor -- start 2024-04-25T09:15:59.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.520+0000 7f1a03200640 1 -- start start 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.520+0000 7f1a03200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a0406d2c0 0x7f1a0415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.520+0000 7f1a03200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a0406d6a0 0x7f1a0415cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.520+0000 7f1a03200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a040b31a0 0x7f1a0415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.520+0000 7f1a03200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0406db20 con 0x7f1a040b31a0 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.520+0000 7f1a03200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040b2c80 con 0x7f1a0406d2c0 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.520+0000 7f1a03200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a041448d0 con 0x7f1a0406d6a0 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a02800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a040b31a0 0x7f1a0415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a02800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a040b31a0 0x7f1a0415d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39688/0 (socket says 172.21.15.62:39688) 2024-04-25T09:15:59.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a02800640 1 -- 172.21.15.62:0/527928642 learned_addr learned my addr 172.21.15.62:0/527928642 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a02800640 1 -- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a0406d6a0 msgr2=0x7f1a0415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a01400640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a0406d6a0 0x7f1a0415cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a01e00640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a0406d2c0 0x7f1a0415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a02800640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a0406d6a0 0x7f1a0415cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a02800640 1 -- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a0406d2c0 msgr2=0x7f1a0415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a02800640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a0406d2c0 0x7f1a0415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a02800640 1 -- 172.21.15.62:0/527928642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a041f38b0 con 0x7f1a040b31a0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.521+0000 7f1a01e00640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a0406d2c0 0x7f1a0415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.522+0000 7f1a02800640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a040b31a0 0x7f1a0415d190 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00c780 tx=0x7f19fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.522+0000 7f19fa000640 1 -- 172.21.15.62:0/527928642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc018070 con 0x7f1a040b31a0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.522+0000 7f19fa000640 1 -- 172.21.15.62:0/527928642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc00a640 con 0x7f1a040b31a0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.522+0000 7f19fa000640 1 -- 172.21.15.62:0/527928642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00aa30 con 0x7f1a040b31a0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.522+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a041f3b90 con 0x7f1a040b31a0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.522+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a041f35a0 con 0x7f1a040b31a0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.523+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a040b2480 con 0x7f1a040b31a0 2024-04-25T09:15:59.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.524+0000 7f19fa000640 1 -- 172.21.15.62:0/527928642 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19fc00abd0 con 0x7f1a040b31a0 2024-04-25T09:15:59.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.525+0000 7f19fa000640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19e406b4d0 0x7f19e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.525+0000 7f19fa000640 1 -- 172.21.15.62:0/527928642 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f19fc089590 con 0x7f1a040b31a0 2024-04-25T09:15:59.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.526+0000 7f1a01e00640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19e406b4d0 0x7f19e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.527+0000 7f1a01e00640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19e406b4d0 0x7f19e406d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006fd0 tx=0x7f19f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.529+0000 7f19fa000640 1 -- 172.21.15.62:0/527928642 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a040b2480 con 0x7f1a040b31a0 2024-04-25T09:15:59.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.727+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a0415ebe0 con 0x7f1a040b31a0 2024-04-25T09:15:59.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.728+0000 7f19fa000640 1 -- 172.21.15.62:0/527928642 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19fc058630 con 0x7f1a040b31a0 2024-04-25T09:15:59.715 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:15:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19e406b4d0 msgr2=0x7f19e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19e406b4d0 0x7f19e406d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006fd0 tx=0x7f19f4008040 comp rx=0 tx=0).stop 2024-04-25T09:15:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a040b31a0 msgr2=0x7f1a0415d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a040b31a0 0x7f1a0415d190 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00c780 tx=0x7f19fc00cc40 comp rx=0 tx=0).stop 2024-04-25T09:15:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 shutdown_connections 2024-04-25T09:15:59.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f19e406b4d0 0x7f19e406d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1a040b31a0 0x7f1a0415d190 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1a0406d6a0 0x7f1a0415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 --2- 172.21.15.62:0/527928642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a0406d2c0 0x7f1a0415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 >> 172.21.15.62:0/527928642 conn(0x7f1a04066f20 msgr2=0x7f1a040b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 shutdown_connections 2024-04-25T09:15:59.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.731+0000 7f1a03200640 1 -- 172.21.15.62:0/527928642 wait complete. 2024-04-25T09:15:59.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:15:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.963+0000 7fa6ebe00640 1 Processor -- start 2024-04-25T09:15:59.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.963+0000 7fa6ebe00640 1 -- start start 2024-04-25T09:15:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.965+0000 7fa6ebe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa6ec0b38d0 0x7fa6ec1689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.965+0000 7fa6ebe00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec0ad980 con 0x7fa6ec0b40a0 2024-04-25T09:15:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.965+0000 7fa6ebe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec0adc60 con 0x7fa6ec15ecc0 2024-04-25T09:15:59.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.965+0000 7fa6ebe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec070780 con 0x7fa6ec0b38d0 2024-04-25T09:15:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.965+0000 7fa6eaa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa6ec0b40a0 0x7fa6ec0b4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37278/0 (socket says 172.21.15.62:37278) 2024-04-25T09:15:59.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.965+0000 7fa6eaa00640 1 -- 172.21.15.62:0/586203037 learned_addr learned my addr 172.21.15.62:0/586203037 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:15:59.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6eb400640 1 --2- 172.21.15.62:0/586203037 >> v2:172.21.15.121:3300/0 conn(0x7fa6ec0b38d0 0x7fa6ec1689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163669052 0 0) 0x7fa6ec0ad980 con 0x7fa6ec0b40a0 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d0003520 con 0x7fa6ec0b40a0 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2225993451 0 0) 0x7fa6d0003520 con 0x7fa6ec0b40a0 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 >> v2:172.21.15.121:3300/0 conn(0x7fa6ec0b38d0 msgr2=0x7fa6ec1689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 --2- 172.21.15.62:0/586203037 >> v2:172.21.15.121:3300/0 conn(0x7fa6ec0b38d0 0x7fa6ec1689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 >> v1:172.21.15.121:6789/0 conn(0x7fa6ec15ecc0 legacy=0x7fa6ec070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec0adf40 con 0x7fa6ec0b40a0 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fa6d4001670 con 0x7fa6ec0b40a0 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 >> v1:172.21.15.62:6789/0 conn(0x7fa6ec0b40a0 legacy=0x7fa6ec0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d0003a70 0x7fa6d0003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d0004800 0x7fa6d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d00052e0 0x7fa6d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec070780 con 0x7fa6d00052e0 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000c1e0 con 0x7fa6d0003a70 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d000c3e0 con 0x7fa6d0004800 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.966+0000 7fa6eb400640 1 --2- 172.21.15.62:0/586203037 >> v2:172.21.15.121:3300/0 conn(0x7fa6ec0b38d0 0x7fa6ec1689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.967+0000 7fa6eb400640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d00052e0 0x7fa6d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.967+0000 7fa6eaa00640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d0003a70 0x7fa6d0003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.967+0000 7fa6ea000640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d0004800 0x7fa6d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.967+0000 7fa6eb400640 1 -- 172.21.15.62:0/586203037 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d0004800 msgr2=0x7fa6d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.967+0000 7fa6eb400640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d0004800 0x7fa6d0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.967+0000 7fa6eb400640 1 -- 172.21.15.62:0/586203037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d0003a70 msgr2=0x7fa6d0003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.967+0000 7fa6eb400640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d0003a70 0x7fa6d0003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.967+0000 7fa6eb400640 1 -- 172.21.15.62:0/586203037 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0003520 con 0x7fa6d00052e0 2024-04-25T09:15:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.968+0000 7fa6eb400640 1 -- 172.21.15.62:0/586203037 reap_dead start 2024-04-25T09:15:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.968+0000 7fa6eb400640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d00052e0 0x7fa6d000b9c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40073b0 tx=0x7fa6d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4b472d2730f782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.968+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4009c70 con 0x7fa6d00052e0 2024-04-25T09:15:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e402c670 con 0x7fa6d00052e0 2024-04-25T09:15:59.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6e9600640 1 -- 172.21.15.62:0/586203037 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e402ca60 con 0x7fa6d00052e0 2024-04-25T09:15:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6ebe00640 1 -- 172.21.15.62:0/586203037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d00052e0 msgr2=0x7fa6d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6ebe00640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d00052e0 0x7fa6d000b9c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40073b0 tx=0x7fa6d0005830 comp rx=0 tx=0).stop 2024-04-25T09:15:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6ebe00640 1 -- 172.21.15.62:0/586203037 shutdown_connections 2024-04-25T09:15:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6ebe00640 1 --2- 172.21.15.62:0/586203037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6d00052e0 0x7fa6d000b9c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6ebe00640 1 -- 172.21.15.62:0/586203037 >> 172.21.15.62:0/586203037 conn(0x7fa6ec074380 msgr2=0x7fa6ec0668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:15:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6ebe00640 1 -- 172.21.15.62:0/586203037 shutdown_connections 2024-04-25T09:15:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.969+0000 7fa6ebe00640 1 -- 172.21.15.62:0/586203037 wait complete. 2024-04-25T09:15:59.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.971+0000 7fa6ebe00640 1 Processor -- start 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.971+0000 7fa6ebe00640 1 -- start start 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.972+0000 7fa6ebe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d00052e0 0x7fa6ec13f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.972+0000 7fa6ebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec13f5d0 0x7fa6ec15f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.972+0000 7fa6ebe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6ec13da80 0x7fa6ec15f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.972+0000 7fa6ebe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec154df0 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.972+0000 7fa6ebe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0ae500 con 0x7fa6ec13da80 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.972+0000 7fa6ebe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b4480 con 0x7fa6d00052e0 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6ea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec13f5d0 0x7fa6ec15f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6ea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec13f5d0 0x7fa6ec15f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39708/0 (socket says 172.21.15.62:39708) 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6ea000640 1 -- 172.21.15.62:0/2981802937 learned_addr learned my addr 172.21.15.62:0/2981802937 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6eaa00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d00052e0 0x7fa6ec13f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6eb400640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6ec13da80 0x7fa6ec15f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6ea000640 1 -- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d00052e0 msgr2=0x7fa6ec13f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6ea000640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d00052e0 0x7fa6ec13f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6ea000640 1 -- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6ec13da80 msgr2=0x7fa6ec15f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6ea000640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6ec13da80 0x7fa6ec15f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6ea000640 1 -- 172.21.15.62:0/2981802937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec13dff0 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6eb400640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6ec13da80 0x7fa6ec15f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.973+0000 7fa6eaa00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d00052e0 0x7fa6ec13f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.974+0000 7fa6ea000640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec13f5d0 0x7fa6ec15f130 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00d4f0 tx=0x7fa6dc00d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.974+0000 7fa6e2a00640 1 -- 172.21.15.62:0/2981802937 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc005480 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.974+0000 7fa6e2a00640 1 -- 172.21.15.62:0/2981802937 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc017070 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.974+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec13e2d0 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.974+0000 7fa6e2a00640 1 -- 172.21.15.62:0/2981802937 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc003810 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.974+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec13e890 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.975+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec075070 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.976+0000 7fa6e2a00640 1 -- 172.21.15.62:0/2981802937 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6dc0039b0 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.977+0000 7fa6e2a00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa6c806b4d0 0x7fa6c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:15:59.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.977+0000 7fa6e2a00640 1 -- 172.21.15.62:0/2981802937 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa6dc089ce0 con 0x7fa6ec13f5d0 2024-04-25T09:15:59.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.978+0000 7fa6eaa00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa6c806b4d0 0x7fa6c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:15:59.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.979+0000 7fa6eaa00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa6c806b4d0 0x7fa6c806d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001eb0 tx=0x7fa6d4000ed0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:15:59.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:15:59.982+0000 7fa6e2a00640 1 -- 172.21.15.62:0/2981802937 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6dc058d80 con 0x7fa6ec13f5d0 2024-04-25T09:16:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.182+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa6ec0aed40 con 0x7fa6ec13f5d0 2024-04-25T09:16:00.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.182+0000 7fa6e2a00640 1 -- 172.21.15.62:0/2981802937 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa6dc058680 con 0x7fa6ec13f5d0 2024-04-25T09:16:00.169 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:00.169 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:16:00.169 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:16:00.169 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:16:00.169 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:16:00.169 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:16:00.169 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:16:00.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa6c806b4d0 msgr2=0x7fa6c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:00.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa6c806b4d0 0x7fa6c806d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4001eb0 tx=0x7fa6d4000ed0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec13f5d0 msgr2=0x7fa6ec15f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec13f5d0 0x7fa6ec15f130 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00d4f0 tx=0x7fa6dc00d9b0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 shutdown_connections 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa6c806b4d0 0x7fa6c806d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa6ec13f5d0 0x7fa6ec15f130 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6ec13da80 0x7fa6ec15f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 --2- 172.21.15.62:0/2981802937 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa6d00052e0 0x7fa6ec13f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 >> 172.21.15.62:0/2981802937 conn(0x7fa6ec074380 msgr2=0x7fa6ec06cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 shutdown_connections 2024-04-25T09:16:00.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.185+0000 7fa6ebe00640 1 -- 172.21.15.62:0/2981802937 wait complete. 2024-04-25T09:16:00.189 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:16:00.189 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:16:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.421+0000 7fc2f8e00640 1 Processor -- start 2024-04-25T09:16:00.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.421+0000 7fc2f8e00640 1 -- start start 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.423+0000 7fc2f8e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc2f4145140 0x7fc2f40b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.423+0000 7fc2f8e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f40b4730 0x7fc2f415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.423+0000 7fc2f8e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc2f415c3f0 0x7fc2f406c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.423+0000 7fc2f8e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f40b3d60 con 0x7fc2f415c3f0 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.423+0000 7fc2f8e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f415c9a0 con 0x7fc2f4145140 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.423+0000 7fc2f8e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f415cc80 con 0x7fc2f40b4730 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.423+0000 7fc2f3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc2f415c3f0 0x7fc2f406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2f3e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc2f415c3f0 0x7fc2f406c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39712/0 (socket says 172.21.15.62:39712) 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2f3e00640 1 -- 172.21.15.62:0/1711141482 learned_addr learned my addr 172.21.15.62:0/1711141482 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2f3400640 1 --2- 172.21.15.62:0/1711141482 >> v2:172.21.15.121:3300/0 conn(0x7fc2f4145140 0x7fc2f40b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:00.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2eaa00640 1 --2- 172.21.15.62:0/1711141482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f40b4730 0x7fc2f415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:00.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2f3e00640 1 -- 172.21.15.62:0/1711141482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f40b4730 msgr2=0x7fc2f415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2f3e00640 1 --2- 172.21.15.62:0/1711141482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f40b4730 0x7fc2f415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2f3e00640 1 -- 172.21.15.62:0/1711141482 >> v2:172.21.15.121:3300/0 conn(0x7fc2f4145140 msgr2=0x7fc2f40b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2f3e00640 1 --2- 172.21.15.62:0/1711141482 >> v2:172.21.15.121:3300/0 conn(0x7fc2f4145140 0x7fc2f40b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.424+0000 7fc2f3e00640 1 -- 172.21.15.62:0/1711141482 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f40b39f0 con 0x7fc2f415c3f0 2024-04-25T09:16:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.425+0000 7fc2f3e00640 1 --2- 172.21.15.62:0/1711141482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f415c3f0 0x7fc2f406c440 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fc2f406e5b0 tx=0x7fc2ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3133b7bbb01b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.425+0000 7fc2f2a00640 1 -- 172.21.15.62:0/1711141482 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec037070 con 0x7fc2f415c3f0 2024-04-25T09:16:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f2a00640 1 -- 172.21.15.62:0/1711141482 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec032440 con 0x7fc2f415c3f0 2024-04-25T09:16:00.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f2a00640 1 -- 172.21.15.62:0/1711141482 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec032810 con 0x7fc2f415c3f0 2024-04-25T09:16:00.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f8e00640 1 -- 172.21.15.62:0/1711141482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f415c3f0 msgr2=0x7fc2f406c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:00.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/1711141482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f415c3f0 0x7fc2f406c440 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fc2f406e5b0 tx=0x7fc2ec02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f8e00640 1 -- 172.21.15.62:0/1711141482 shutdown_connections 2024-04-25T09:16:00.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/1711141482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f415c3f0 0x7fc2f406c440 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/1711141482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f40b4730 0x7fc2f415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/1711141482 >> v2:172.21.15.121:3300/0 conn(0x7fc2f4145140 0x7fc2f40b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.426+0000 7fc2f8e00640 1 -- 172.21.15.62:0/1711141482 >> 172.21.15.62:0/1711141482 conn(0x7fc2f4066ad0 msgr2=0x7fc2f4066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:00.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.427+0000 7fc2f8e00640 1 -- 172.21.15.62:0/1711141482 shutdown_connections 2024-04-25T09:16:00.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.427+0000 7fc2f8e00640 1 -- 172.21.15.62:0/1711141482 wait complete. 2024-04-25T09:16:00.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.429+0000 7fc2f8e00640 1 Processor -- start 2024-04-25T09:16:00.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.429+0000 7fc2f8e00640 1 -- start start 2024-04-25T09:16:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.430+0000 7fc2f8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f4145140 0x7fc2f413c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.430+0000 7fc2f8e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f415c3f0 0x7fc2f40c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.430+0000 7fc2f8e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f41f3a00 0x7fc2f40c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.430+0000 7fc2f8e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f41456a0 con 0x7fc2f4145140 2024-04-25T09:16:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.430+0000 7fc2f8e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f4145520 con 0x7fc2f41f3a00 2024-04-25T09:16:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.430+0000 7fc2f8e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f406c980 con 0x7fc2f415c3f0 2024-04-25T09:16:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2f3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f4145140 0x7fc2f413c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:00.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2f3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f4145140 0x7fc2f413c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39724/0 (socket says 172.21.15.62:39724) 2024-04-25T09:16:00.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2f3400640 1 -- 172.21.15.62:0/535629970 learned_addr learned my addr 172.21.15.62:0/535629970 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:00.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2eaa00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f415c3f0 0x7fc2f40c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:00.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2f3400640 1 -- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f415c3f0 msgr2=0x7fc2f40c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:00.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2f3400640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f415c3f0 0x7fc2f40c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2f3400640 1 -- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f41f3a00 msgr2=0x7fc2f40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:00.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2f3400640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f41f3a00 0x7fc2f40c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:00.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.431+0000 7fc2f3400640 1 -- 172.21.15.62:0/535629970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f4070170 con 0x7fc2f4145140 2024-04-25T09:16:00.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.432+0000 7fc2f3400640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f4145140 0x7fc2f413c140 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4007ef0 tx=0x7fc2e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:00.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.432+0000 7fc2ebe00640 1 -- 172.21.15.62:0/535629970 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4014620 con 0x7fc2f4145140 2024-04-25T09:16:00.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.432+0000 7fc2ebe00640 1 -- 172.21.15.62:0/535629970 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e4019070 con 0x7fc2f4145140 2024-04-25T09:16:00.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.432+0000 7fc2ebe00640 1 -- 172.21.15.62:0/535629970 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4014a00 con 0x7fc2f4145140 2024-04-25T09:16:00.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.432+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f4070450 con 0x7fc2f4145140 2024-04-25T09:16:00.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.432+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f4070a10 con 0x7fc2f4145140 2024-04-25T09:16:00.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.433+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f406c730 con 0x7fc2f4145140 2024-04-25T09:16:00.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.434+0000 7fc2ebe00640 1 -- 172.21.15.62:0/535629970 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2e4014ba0 con 0x7fc2f4145140 2024-04-25T09:16:00.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.435+0000 7fc2ebe00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc2d006b500 0x7fc2d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:00.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.435+0000 7fc2ebe00640 1 -- 172.21.15.62:0/535629970 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc2e408a5e0 con 0x7fc2f4145140 2024-04-25T09:16:00.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.436+0000 7fc2eaa00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc2d006b500 0x7fc2d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:00.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.437+0000 7fc2eaa00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc2d006b500 0x7fc2d006d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc2f406dfc0 tx=0x7fc2dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:00.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.439+0000 7fc2ebe00640 1 -- 172.21.15.62:0/535629970 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2f406c730 con 0x7fc2f4145140 2024-04-25T09:16:00.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:00.638+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc2f40750e0 con 0x7fc2f4145140 2024-04-25T09:16:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.323+0000 7fc2ebe00640 1 -- 172.21.15.62:0/535629970 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v134) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc2e40597b0 con 0x7fc2f4145140 2024-04-25T09:16:01.310 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:16:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc2d006b500 msgr2=0x7fc2d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc2d006b500 0x7fc2d006d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc2f406dfc0 tx=0x7fc2dc008040 comp rx=0 tx=0).stop 2024-04-25T09:16:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f4145140 msgr2=0x7fc2f413c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f4145140 0x7fc2f413c140 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4007ef0 tx=0x7fc2e400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 shutdown_connections 2024-04-25T09:16:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc2d006b500 0x7fc2d006d9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f41f3a00 0x7fc2f40c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc2f415c3f0 0x7fc2f40c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 --2- 172.21.15.62:0/535629970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc2f4145140 0x7fc2f413c140 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 >> 172.21.15.62:0/535629970 conn(0x7fc2f4066ad0 msgr2=0x7fc2f40cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 shutdown_connections 2024-04-25T09:16:01.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.327+0000 7fc2f8e00640 1 -- 172.21.15.62:0/535629970 wait complete. 2024-04-25T09:16:01.339 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:16:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.557+0000 7f4663200640 1 Processor -- start 2024-04-25T09:16:01.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.557+0000 7f4663200640 1 -- start start 2024-04-25T09:16:01.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.559+0000 7f4663200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c15cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:01.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.559+0000 7f4663200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f465c0b1080 0x7f465c1656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:01.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.559+0000 7f4663200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465c15d090 con 0x7f465c0cc960 2024-04-25T09:16:01.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.559+0000 7f4663200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c06d000 con 0x7f465c0b1080 2024-04-25T09:16:01.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.559+0000 7f4663200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c06d490 con 0x7f465c159680 2024-04-25T09:16:01.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.560+0000 7f4661e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c15cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:01.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.560+0000 7f4662800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f465c0b1080 0x7f465c1656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:01.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.560+0000 7f4661e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c15cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:55398/0 (socket says 172.21.15.62:55398) 2024-04-25T09:16:01.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.560+0000 7f4661e00640 1 -- 172.21.15.62:0/3913452945 learned_addr learned my addr 172.21.15.62:0/3913452945 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:01.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.561+0000 7f4662800640 1 -- 172.21.15.62:0/3913452945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 msgr2=0x7f465c15cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:01.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.561+0000 7f4662800640 1 --2- 172.21.15.62:0/3913452945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c15cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.561+0000 7f4662800640 1 -- 172.21.15.62:0/3913452945 >> v1:172.21.15.121:6789/0 conn(0x7f465c0cc960 legacy=0x7f465c0b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:01.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.561+0000 7f4662800640 1 -- 172.21.15.62:0/3913452945 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c06c3f0 con 0x7f465c0b1080 2024-04-25T09:16:01.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.561+0000 7f4661e00640 1 --2- 172.21.15.62:0/3913452945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c15cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:16:01.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.562+0000 7f4662800640 1 --2- 172.21.15.62:0/3913452945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 0x7f465c1656f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4658009d90 tx=0x7f465802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb3914ad5418967b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:01.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.563+0000 7f4660a00640 1 -- 172.21.15.62:0/3913452945 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658007740 con 0x7f465c0b1080 2024-04-25T09:16:01.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.563+0000 7f4660a00640 1 -- 172.21.15.62:0/3913452945 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46580078e0 con 0x7f465c0b1080 2024-04-25T09:16:01.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.563+0000 7f4660a00640 1 -- 172.21.15.62:0/3913452945 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658007cb0 con 0x7f465c0b1080 2024-04-25T09:16:01.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.564+0000 7f4663200640 1 -- 172.21.15.62:0/3913452945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 msgr2=0x7f465c1656f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:01.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.564+0000 7f4663200640 1 --2- 172.21.15.62:0/3913452945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 0x7f465c1656f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4658009d90 tx=0x7f465802a660 comp rx=0 tx=0).stop 2024-04-25T09:16:01.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.564+0000 7f4663200640 1 -- 172.21.15.62:0/3913452945 shutdown_connections 2024-04-25T09:16:01.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.564+0000 7f4663200640 1 --2- 172.21.15.62:0/3913452945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 0x7f465c1656f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.564+0000 7f4663200640 1 --2- 172.21.15.62:0/3913452945 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c15cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.564+0000 7f4663200640 1 -- 172.21.15.62:0/3913452945 >> 172.21.15.62:0/3913452945 conn(0x7f465c0668f0 msgr2=0x7f465c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:01.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.564+0000 7f4663200640 1 -- 172.21.15.62:0/3913452945 shutdown_connections 2024-04-25T09:16:01.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.564+0000 7f4663200640 1 -- 172.21.15.62:0/3913452945 wait complete. 2024-04-25T09:16:01.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.566+0000 7f4663200640 1 Processor -- start 2024-04-25T09:16:01.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.567+0000 7f4663200640 1 -- start start 2024-04-25T09:16:01.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.568+0000 7f4663200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 0x7f465c0c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:01.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.568+0000 7f4663200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f465c0cc960 0x7f465c143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:01.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.568+0000 7f4663200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:01.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.568+0000 7f4663200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c06cc90 con 0x7f465c0cc960 2024-04-25T09:16:01.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.568+0000 7f4663200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c06c9b0 con 0x7f465c0b1080 2024-04-25T09:16:01.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.568+0000 7f4663200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c159a60 con 0x7f465c159680 2024-04-25T09:16:01.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.568+0000 7f4661400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f465c0cc960 0x7f465c143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:01.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.568+0000 7f4661400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f465c0cc960 0x7f465c143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39738/0 (socket says 172.21.15.62:39738) 2024-04-25T09:16:01.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661400640 1 -- 172.21.15.62:0/814766814 learned_addr learned my addr 172.21.15.62:0/814766814 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:01.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661e00640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 0x7f465c0c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:01.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4662800640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:01.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661400640 1 -- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 msgr2=0x7f465c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:01.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661400640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661400640 1 -- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 msgr2=0x7f465c0c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:01.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661400640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 0x7f465c0c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:01.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661400640 1 -- 172.21.15.62:0/814766814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c14f670 con 0x7f465c0cc960 2024-04-25T09:16:01.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661e00640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 0x7f465c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:16:01.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.569+0000 7f4661400640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f465c0cc960 0x7f465c143d00 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f464c00c440 tx=0x7f464c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:01.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.570+0000 7f4656000640 1 -- 172.21.15.62:0/814766814 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0052d0 con 0x7f465c0cc960 2024-04-25T09:16:01.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.570+0000 7f4656000640 1 -- 172.21.15.62:0/814766814 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c005470 con 0x7f465c0cc960 2024-04-25T09:16:01.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.570+0000 7f4656000640 1 -- 172.21.15.62:0/814766814 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c007260 con 0x7f465c0cc960 2024-04-25T09:16:01.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.570+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c14f950 con 0x7f465c0cc960 2024-04-25T09:16:01.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.570+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c14ff10 con 0x7f465c0cc960 2024-04-25T09:16:01.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.572+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c0cdd30 con 0x7f465c0cc960 2024-04-25T09:16:01.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.574+0000 7f4656000640 1 -- 172.21.15.62:0/814766814 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f464c007400 con 0x7f465c0cc960 2024-04-25T09:16:01.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.575+0000 7f4656000640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f464006b4d0 0x7f464006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:01.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.575+0000 7f4656000640 1 -- 172.21.15.62:0/814766814 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f464c0899f0 con 0x7f465c0cc960 2024-04-25T09:16:01.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.576+0000 7f4661e00640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f464006b4d0 0x7f464006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:01.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.577+0000 7f4661e00640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f464006b4d0 0x7f464006d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4650009b80 tx=0x7f4650008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:01.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.580+0000 7f4656000640 1 -- 172.21.15.62:0/814766814 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f465c0cdd30 con 0x7f465c0cc960 2024-04-25T09:16:01.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:01.781+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f465c075070 con 0x7f465c0cc960 2024-04-25T09:16:02.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.338+0000 7f4656000640 1 -- 172.21.15.62:0/814766814 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v135) v1 ==== 170+0+0 (secure 0 0 0) 0x7f465c15d090 con 0x7f465c0cc960 2024-04-25T09:16:02.325 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:16:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f464006b4d0 msgr2=0x7f464006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f464006b4d0 0x7f464006d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4650009b80 tx=0x7f4650008040 comp rx=0 tx=0).stop 2024-04-25T09:16:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f465c0cc960 msgr2=0x7f465c143d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f465c0cc960 0x7f465c143d00 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f464c00c440 tx=0x7f464c001e70 comp rx=0 tx=0).stop 2024-04-25T09:16:02.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 shutdown_connections 2024-04-25T09:16:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f464006b4d0 0x7f464006d990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f465c0cc960 0x7f465c143d00 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f465c159680 0x7f465c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 --2- 172.21.15.62:0/814766814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f465c0b1080 0x7f465c0c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 >> 172.21.15.62:0/814766814 conn(0x7f465c0668f0 msgr2=0x7f465c0cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 shutdown_connections 2024-04-25T09:16:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.342+0000 7f4663200640 1 -- 172.21.15.62:0/814766814 wait complete. 2024-04-25T09:16:02.364 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:16:02.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.558+0000 7fdb76600640 1 Processor -- start 2024-04-25T09:16:02.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.558+0000 7fdb76600640 1 -- start start 2024-04-25T09:16:02.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.560+0000 7fdb76600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdb7006d2c0 0x7fdb700ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:02.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.560+0000 7fdb76600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb700b31a0 0x7fdb700b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.560+0000 7fdb76600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb700cf000 con 0x7fdb7006d6a0 2024-04-25T09:16:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.560+0000 7fdb76600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb700cf2e0 con 0x7fdb7006d2c0 2024-04-25T09:16:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.560+0000 7fdb76600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb700b26c0 con 0x7fdb700b31a0 2024-04-25T09:16:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.560+0000 7fdb75200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdb7006d2c0 0x7fdb700ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.560+0000 7fdb6fe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdb7006d6a0 0x7fdb700ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37284/0 (socket says 172.21.15.62:37284) 2024-04-25T09:16:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.560+0000 7fdb6fe00640 1 -- 172.21.15.62:0/2046602959 learned_addr learned my addr 172.21.15.62:0/2046602959 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.561+0000 7fdb75c00640 1 --2- 172.21.15.62:0/2046602959 >> v2:172.21.15.121:3300/0 conn(0x7fdb700b31a0 0x7fdb700b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.561+0000 7fdb75200640 1 -- 172.21.15.62:0/2046602959 >> v2:172.21.15.121:3300/0 conn(0x7fdb700b31a0 msgr2=0x7fdb700b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.561+0000 7fdb75200640 1 --2- 172.21.15.62:0/2046602959 >> v2:172.21.15.121:3300/0 conn(0x7fdb700b31a0 0x7fdb700b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.561+0000 7fdb75200640 1 -- 172.21.15.62:0/2046602959 >> v1:172.21.15.62:6789/0 conn(0x7fdb7006d6a0 legacy=0x7fdb700ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.561+0000 7fdb75200640 1 -- 172.21.15.62:0/2046602959 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb700b29a0 con 0x7fdb7006d2c0 2024-04-25T09:16:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.561+0000 7fdb6f400640 1 -- 172.21.15.62:0/2046602959 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553882078 0 0) 0x7fdb700cf000 con 0x7fdb7006d6a0 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.562+0000 7fdb75200640 1 --2- 172.21.15.62:0/2046602959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d2c0 0x7fdb700ccde0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdb7006e560 tx=0x7fdb6402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfa934223cb6231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb6f400640 1 -- 172.21.15.62:0/2046602959 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64037070 con 0x7fdb7006d2c0 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb6f400640 1 -- 172.21.15.62:0/2046602959 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb64032440 con 0x7fdb7006d2c0 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb6f400640 1 -- 172.21.15.62:0/2046602959 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64032810 con 0x7fdb7006d2c0 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb76600640 1 -- 172.21.15.62:0/2046602959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d2c0 msgr2=0x7fdb700ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb76600640 1 --2- 172.21.15.62:0/2046602959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d2c0 0x7fdb700ccde0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdb7006e560 tx=0x7fdb6402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb76600640 1 -- 172.21.15.62:0/2046602959 shutdown_connections 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb76600640 1 --2- 172.21.15.62:0/2046602959 >> v2:172.21.15.121:3300/0 conn(0x7fdb700b31a0 0x7fdb700b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb76600640 1 --2- 172.21.15.62:0/2046602959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d2c0 0x7fdb700ccde0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb76600640 1 -- 172.21.15.62:0/2046602959 >> 172.21.15.62:0/2046602959 conn(0x7fdb70066f20 msgr2=0x7fdb700b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb76600640 1 -- 172.21.15.62:0/2046602959 shutdown_connections 2024-04-25T09:16:02.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.563+0000 7fdb76600640 1 -- 172.21.15.62:0/2046602959 wait complete. 2024-04-25T09:16:02.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.565+0000 7fdb76600640 1 Processor -- start 2024-04-25T09:16:02.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.565+0000 7fdb76600640 1 -- start start 2024-04-25T09:16:02.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.566+0000 7fdb76600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb7006d2c0 0x7fdb701e8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.566+0000 7fdb76600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d6a0 0x7fdb7015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.566+0000 7fdb76600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb700b31a0 0x7fdb7015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.566+0000 7fdb76600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb701448d0 con 0x7fdb7006d6a0 2024-04-25T09:16:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.566+0000 7fdb76600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7006db20 con 0x7fdb700b31a0 2024-04-25T09:16:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.566+0000 7fdb76600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70144a50 con 0x7fdb7006d2c0 2024-04-25T09:16:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d6a0 0x7fdb7015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d6a0 0x7fdb7015ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39756/0 (socket says 172.21.15.62:39756) 2024-04-25T09:16:02.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 -- 172.21.15.62:0/2042828445 learned_addr learned my addr 172.21.15.62:0/2042828445 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 -- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb7006d2c0 msgr2=0x7fdb701e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb7006d2c0 0x7fdb701e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 -- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb700b31a0 msgr2=0x7fdb7015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb700b31a0 0x7fdb7015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 -- 172.21.15.62:0/2042828445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7012f8d0 con 0x7fdb7006d6a0 2024-04-25T09:16:02.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.567+0000 7fdb6fe00640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d6a0 0x7fdb7015ad60 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c4d0 tx=0x7fdb5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:02.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.568+0000 7fdb6cc00640 1 -- 172.21.15.62:0/2042828445 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c018070 con 0x7fdb7006d6a0 2024-04-25T09:16:02.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.568+0000 7fdb6cc00640 1 -- 172.21.15.62:0/2042828445 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c007500 con 0x7fdb7006d6a0 2024-04-25T09:16:02.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.568+0000 7fdb6cc00640 1 -- 172.21.15.62:0/2042828445 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c004a20 con 0x7fdb7006d6a0 2024-04-25T09:16:02.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.568+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7012fb70 con 0x7fdb7006d6a0 2024-04-25T09:16:02.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.568+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb70130130 con 0x7fdb7006d6a0 2024-04-25T09:16:02.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.569+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7004ef20 con 0x7fdb7006d6a0 2024-04-25T09:16:02.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.570+0000 7fdb6cc00640 1 -- 172.21.15.62:0/2042828445 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb5c004bc0 con 0x7fdb7006d6a0 2024-04-25T09:16:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.571+0000 7fdb6cc00640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdb5006b4d0 0x7fdb5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.571+0000 7fdb6cc00640 1 -- 172.21.15.62:0/2042828445 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdb5c0895c0 con 0x7fdb7006d6a0 2024-04-25T09:16:02.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.572+0000 7fdb75200640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdb5006b4d0 0x7fdb5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:02.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.573+0000 7fdb75200640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdb5006b4d0 0x7fdb5006d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdb64034000 tx=0x7fdb6402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:02.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.576+0000 7fdb6cc00640 1 -- 172.21.15.62:0/2042828445 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb5c058400 con 0x7fdb7006d6a0 2024-04-25T09:16:02.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:02.816+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdb701304a0 con 0x7fdb7006d6a0 2024-04-25T09:16:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.343+0000 7fdb6cc00640 1 -- 172.21.15.62:0/2042828445 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdb5c057d00 con 0x7fdb7006d6a0 2024-04-25T09:16:03.330 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 18 and data pool 19 2024-04-25T09:16:03.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.348+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdb5006b4d0 msgr2=0x7fdb5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.348+0000 7fdb76600640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdb5006b4d0 0x7fdb5006d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdb64034000 tx=0x7fdb6402c000 comp rx=0 tx=0).stop 2024-04-25T09:16:03.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.348+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d6a0 msgr2=0x7fdb7015ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.348+0000 7fdb76600640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d6a0 0x7fdb7015ad60 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c4d0 tx=0x7fdb5c00c990 comp rx=0 tx=0).stop 2024-04-25T09:16:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.350+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 shutdown_connections 2024-04-25T09:16:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.350+0000 7fdb76600640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdb5006b4d0 0x7fdb5006d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.350+0000 7fdb76600640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb7006d6a0 0x7fdb7015ad60 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.350+0000 7fdb76600640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb700b31a0 0x7fdb7015b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.350+0000 7fdb76600640 1 --2- 172.21.15.62:0/2042828445 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdb7006d2c0 0x7fdb701e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.350+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 >> 172.21.15.62:0/2042828445 conn(0x7fdb70066f20 msgr2=0x7fdb700cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.351+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 shutdown_connections 2024-04-25T09:16:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.351+0000 7fdb76600640 1 -- 172.21.15.62:0/2042828445 wait complete. 2024-04-25T09:16:03.379 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:16:03.550 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.564+0000 7fc959200640 1 Processor -- start 2024-04-25T09:16:03.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.564+0000 7fc959200640 1 -- start start 2024-04-25T09:16:03.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.565+0000 7fc959200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc95415dc40 0x7fc9540d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.565+0000 7fc959200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9540d0ae0 0x7fc9540d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.552 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc959200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9540d2620 con 0x7fc9540d3ae0 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc959200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9540d2900 con 0x7fc9540d0ae0 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc959200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9540d2be0 con 0x7fc95415dc40 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9540d0ae0 0x7fc9540d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9540d0ae0 0x7fc9540d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39772/0 (socket says 172.21.15.62:39772) 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951400640 1 -- 172.21.15.62:0/2928707672 learned_addr learned my addr 172.21.15.62:0/2928707672 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951e00640 1 --2- 172.21.15.62:0/2928707672 >> v2:172.21.15.121:3300/0 conn(0x7fc95415dc40 0x7fc9540d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951400640 1 -- 172.21.15.62:0/2928707672 >> v2:172.21.15.121:3300/0 conn(0x7fc95415dc40 msgr2=0x7fc9540d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951400640 1 --2- 172.21.15.62:0/2928707672 >> v2:172.21.15.121:3300/0 conn(0x7fc95415dc40 0x7fc9540d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951400640 1 -- 172.21.15.62:0/2928707672 >> v1:172.21.15.121:6789/0 conn(0x7fc9540d3ae0 legacy=0x7fc9540d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951400640 1 -- 172.21.15.62:0/2928707672 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540d2ec0 con 0x7fc9540d0ae0 2024-04-25T09:16:03.553 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.566+0000 7fc951e00640 1 --2- 172.21.15.62:0/2928707672 >> v2:172.21.15.121:3300/0 conn(0x7fc95415dc40 0x7fc9540d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:03.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.567+0000 7fc951400640 1 --2- 172.21.15.62:0/2928707672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9540d0ae0 0x7fc9540d0f70 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fc940009870 tx=0x7fc94002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b94dd96066e2a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:03.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.568+0000 7fc950a00640 1 -- 172.21.15.62:0/2928707672 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940037070 con 0x7fc9540d0ae0 2024-04-25T09:16:03.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.568+0000 7fc950a00640 1 -- 172.21.15.62:0/2928707672 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc940032440 con 0x7fc9540d0ae0 2024-04-25T09:16:03.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.568+0000 7fc950a00640 1 -- 172.21.15.62:0/2928707672 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940032810 con 0x7fc9540d0ae0 2024-04-25T09:16:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.568+0000 7fc959200640 1 -- 172.21.15.62:0/2928707672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9540d0ae0 msgr2=0x7fc9540d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.568+0000 7fc959200640 1 --2- 172.21.15.62:0/2928707672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9540d0ae0 0x7fc9540d0f70 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fc940009870 tx=0x7fc94002a250 comp rx=0 tx=0).stop 2024-04-25T09:16:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.569+0000 7fc959200640 1 -- 172.21.15.62:0/2928707672 shutdown_connections 2024-04-25T09:16:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.569+0000 7fc959200640 1 --2- 172.21.15.62:0/2928707672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9540d0ae0 0x7fc9540d0f70 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.569+0000 7fc959200640 1 --2- 172.21.15.62:0/2928707672 >> v2:172.21.15.121:3300/0 conn(0x7fc95415dc40 0x7fc9540d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.569+0000 7fc959200640 1 -- 172.21.15.62:0/2928707672 >> 172.21.15.62:0/2928707672 conn(0x7fc95406b000 msgr2=0x7fc95406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.569+0000 7fc959200640 1 -- 172.21.15.62:0/2928707672 shutdown_connections 2024-04-25T09:16:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.569+0000 7fc959200640 1 -- 172.21.15.62:0/2928707672 wait complete. 2024-04-25T09:16:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.570+0000 7fc959200640 1 Processor -- start 2024-04-25T09:16:03.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.571+0000 7fc959200640 1 -- start start 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc959200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9540d0ae0 0x7fc9541530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc959200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9540d3ae0 0x7fc954155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc959200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95415dc40 0x7fc9540b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc959200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540d36c0 con 0x7fc95415dc40 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc959200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540b7d00 con 0x7fc9540d0ae0 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc959200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95415e7f0 con 0x7fc9540d3ae0 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc952800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95415dc40 0x7fc9540b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc952800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95415dc40 0x7fc9540b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39778/0 (socket says 172.21.15.62:39778) 2024-04-25T09:16:03.558 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc952800640 1 -- 172.21.15.62:0/1824952496 learned_addr learned my addr 172.21.15.62:0/1824952496 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc951400640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9540d3ae0 0x7fc954155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc951e00640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9540d0ae0 0x7fc9541530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc952800640 1 -- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9540d3ae0 msgr2=0x7fc954155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc952800640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9540d3ae0 0x7fc954155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc952800640 1 -- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9540d0ae0 msgr2=0x7fc9541530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc952800640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9540d0ae0 0x7fc9541530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc952800640 1 -- 172.21.15.62:0/1824952496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9541f8540 con 0x7fc95415dc40 2024-04-25T09:16:03.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.572+0000 7fc951400640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9540d3ae0 0x7fc954155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.573+0000 7fc952800640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95415dc40 0x7fc9540b6cb0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00c3c0 tx=0x7fc94c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.573+0000 7fc94a000640 1 -- 172.21.15.62:0/1824952496 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c017070 con 0x7fc95415dc40 2024-04-25T09:16:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.573+0000 7fc94a000640 1 -- 172.21.15.62:0/1824952496 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c0035f0 con 0x7fc95415dc40 2024-04-25T09:16:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.573+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95415e1a0 con 0x7fc95415dc40 2024-04-25T09:16:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.573+0000 7fc94a000640 1 -- 172.21.15.62:0/1824952496 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0039e0 con 0x7fc95415dc40 2024-04-25T09:16:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.573+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954155fc0 con 0x7fc95415dc40 2024-04-25T09:16:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.574+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9540729c0 con 0x7fc95415dc40 2024-04-25T09:16:03.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.575+0000 7fc94a000640 1 -- 172.21.15.62:0/1824952496 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc94c01a070 con 0x7fc95415dc40 2024-04-25T09:16:03.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.576+0000 7fc94a000640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.576+0000 7fc94a000640 1 -- 172.21.15.62:0/1824952496 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc94c0895b0 con 0x7fc95415dc40 2024-04-25T09:16:03.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.577+0000 7fc951e00640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.578+0000 7fc951e00640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc93406b4d0 0x7fc93406d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc944008ee0 tx=0x7fc944008430 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:03.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.581+0000 7fc94a000640 1 -- 172.21.15.62:0/1824952496 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc94c058420 con 0x7fc95415dc40 2024-04-25T09:16:03.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.778+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc95404f000 con 0x7fc95415dc40 2024-04-25T09:16:03.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.779+0000 7fc94a000640 1 -- 172.21.15.62:0/1824952496 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16628 (secure 0 0 0) 0x7fc94c057d20 con 0x7fc95415dc40 2024-04-25T09:16:03.766 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:03.766 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":136,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:16:02.817337+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:16:00.695740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:16:01.838452+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:16:03.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc93406b4d0 msgr2=0x7fc93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc93406b4d0 0x7fc93406d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc944008ee0 tx=0x7fc944008430 comp rx=0 tx=0).stop 2024-04-25T09:16:03.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95415dc40 msgr2=0x7fc9540b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95415dc40 0x7fc9540b6cb0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00c3c0 tx=0x7fc94c00c880 comp rx=0 tx=0).stop 2024-04-25T09:16:03.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 shutdown_connections 2024-04-25T09:16:03.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc95415dc40 0x7fc9540b6cb0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9540d3ae0 0x7fc954155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 --2- 172.21.15.62:0/1824952496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9540d0ae0 0x7fc9541530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.781+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 >> 172.21.15.62:0/1824952496 conn(0x7fc95406b000 msgr2=0x7fc9540c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:03.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.782+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 shutdown_connections 2024-04-25T09:16:03.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:03.782+0000 7fc959200640 1 -- 172.21.15.62:0/1824952496 wait complete. 2024-04-25T09:16:03.787 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:16:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.006+0000 7f0fbb600640 1 Processor -- start 2024-04-25T09:16:03.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.007+0000 7f0fbb600640 1 -- start start 2024-04-25T09:16:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.008+0000 7f0fbb600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 0x7f0fb40728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.008+0000 7f0fbb600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb4070cd0 con 0x7f0fb415dee0 2024-04-25T09:16:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.008+0000 7f0fbb600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb40b3140 con 0x7f0fb406e790 2024-04-25T09:16:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.008+0000 7f0fbb600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40ae660 con 0x7f0fb406f280 2024-04-25T09:16:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.008+0000 7f0fb8c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0fb415dee0 0x7f0fb406e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37290/0 (socket says 172.21.15.62:37290) 2024-04-25T09:16:03.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.008+0000 7f0fb8c00640 1 -- 172.21.15.62:0/3435447864 learned_addr learned my addr 172.21.15.62:0/3435447864 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb9600640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 0x7f0fb40728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920413853 0 0) 0x7f0fb4070cd0 con 0x7f0fb415dee0 2024-04-25T09:16:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f98003520 con 0x7f0fb415dee0 2024-04-25T09:16:03.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3233386198 0 0) 0x7f0f98003520 con 0x7f0fb415dee0 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 msgr2=0x7f0fb40728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 0x7f0fb40728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 >> v1:172.21.15.121:6789/0 conn(0x7f0fb406e790 legacy=0x7f0fb40aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4070f70 con 0x7f0fb415dee0 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f0fa4001670 con 0x7f0fb415dee0 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 >> v1:172.21.15.62:6789/0 conn(0x7f0fb415dee0 legacy=0x7f0fb406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 0x7f0f980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0f98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f98004d40 0x7f0f980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40ae660 con 0x7f0f98003f60 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9800bc00 con 0x7f0f98004d40 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.009+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9800be00 con 0x7f0fb406f280 2024-04-25T09:16:03.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb3e00640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0f98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb8c00640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 0x7f0f980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb9600640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f98004d40 0x7f0f980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb3e00640 1 -- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 msgr2=0x7f0f980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb3e00640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 0x7f0f980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb3e00640 1 -- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f98004d40 msgr2=0x7f0f980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb3e00640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f98004d40 0x7f0f980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb3e00640 1 -- 172.21.15.62:0/3435447864 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f98003520 con 0x7f0f98003f60 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb8c00640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb406f280 0x7f0f980039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.010+0000 7f0fb9600640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f98004d40 0x7f0f980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:03.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.011+0000 7f0fb3e00640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0f98004800 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0001e50 tx=0x7f0f98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e97f34939749c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0001a70 con 0x7f0f98003f60 2024-04-25T09:16:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa0003970 con 0x7f0f98003f60 2024-04-25T09:16:04.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fb3400640 1 -- 172.21.15.62:0/3435447864 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0003d60 con 0x7f0f98003f60 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fbb600640 1 -- 172.21.15.62:0/3435447864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 msgr2=0x7f0f98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fbb600640 1 --2- 172.21.15.62:0/3435447864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0f98004800 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0001e50 tx=0x7f0f98005830 comp rx=0 tx=0).stop 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fb9600640 1 -- 172.21.15.62:0/3435447864 reap_dead start 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fbb600640 1 -- 172.21.15.62:0/3435447864 shutdown_connections 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fbb600640 1 -- 172.21.15.62:0/3435447864 >> 172.21.15.62:0/3435447864 conn(0x7f0fb40b1e40 msgr2=0x7f0fb40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fbb600640 1 -- 172.21.15.62:0/3435447864 shutdown_connections 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.012+0000 7f0fbb600640 1 -- 172.21.15.62:0/3435447864 wait complete. 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.014+0000 7f0fbb600640 1 Processor -- start 2024-04-25T09:16:04.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.014+0000 7f0fbb600640 1 -- start start 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fbb600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0fb41f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fbb600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb41f7c90 0x7f0fb41e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fbb600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb40cecd0 0x7f0fb40cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fbb600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40b2c30 con 0x7f0f98003f60 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fbb600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40ae2f0 con 0x7f0fb40cecd0 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fbb600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40728a0 con 0x7f0fb41f7c90 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fb8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0fb41f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fb8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0fb41f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39788/0 (socket says 172.21.15.62:39788) 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.015+0000 7f0fb8c00640 1 -- 172.21.15.62:0/1874155023 learned_addr learned my addr 172.21.15.62:0/1874155023 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.016+0000 7f0fb8c00640 1 -- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb41f7c90 msgr2=0x7f0fb41e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.016+0000 7f0fb8c00640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb41f7c90 0x7f0fb41e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.016+0000 7f0fb8c00640 1 -- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb40cecd0 msgr2=0x7f0fb40cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.016+0000 7f0fb8c00640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb40cecd0 0x7f0fb40cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.016+0000 7f0fb8c00640 1 -- 172.21.15.62:0/1874155023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb41485e0 con 0x7f0f98003f60 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.016+0000 7f0fb8c00640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0fb41f33e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4001200 tx=0x7f0fa400dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.016+0000 7f0fb0c00640 1 -- 172.21.15.62:0/1874155023 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4019070 con 0x7f0f98003f60 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.017+0000 7f0fb0c00640 1 -- 172.21.15.62:0/1874155023 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa40074e0 con 0x7f0f98003f60 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.017+0000 7f0fb0c00640 1 -- 172.21.15.62:0/1874155023 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40049e0 con 0x7f0f98003f60 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.017+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb41488c0 con 0x7f0f98003f60 2024-04-25T09:16:04.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.017+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb4148e80 con 0x7f0f98003f60 2024-04-25T09:16:04.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.018+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb40745c0 con 0x7f0f98003f60 2024-04-25T09:16:04.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.019+0000 7f0fb0c00640 1 -- 172.21.15.62:0/1874155023 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fa4004b80 con 0x7f0f98003f60 2024-04-25T09:16:04.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.020+0000 7f0fb0c00640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.020+0000 7f0fb0c00640 1 -- 172.21.15.62:0/1874155023 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fa4089c60 con 0x7f0f98003f60 2024-04-25T09:16:04.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.020+0000 7f0fb3e00640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:04.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.021+0000 7f0fb3e00640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f9406b4d0 0x7f0f9406d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0032000 tx=0x7f0fa002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:04.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.024+0000 7f0fb0c00640 1 -- 172.21.15.62:0/1874155023 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fa4058a40 con 0x7f0f98003f60 2024-04-25T09:16:04.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.224+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fb406de70 con 0x7f0f98003f60 2024-04-25T09:16:04.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.227+0000 7f0fb0c00640 1 -- 172.21.15.62:0/1874155023 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16628 (secure 0 0 0) 0x7f0fa4058340 con 0x7f0f98003f60 2024-04-25T09:16:04.214 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:04.214 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":136,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:16:02.817337+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:16:00.695740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:16:01.838452+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f9406b4d0 msgr2=0x7f0f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f9406b4d0 0x7f0f9406d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0032000 tx=0x7f0fa002c000 comp rx=0 tx=0).stop 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 msgr2=0x7f0fb41f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0fb41f33e0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4001200 tx=0x7f0fa400dc50 comp rx=0 tx=0).stop 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 shutdown_connections 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fb40cecd0 0x7f0fb40cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0fb41f7c90 0x7f0fb41e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 --2- 172.21.15.62:0/1874155023 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f98003f60 0x7f0fb41f33e0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.230+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 >> 172.21.15.62:0/1874155023 conn(0x7f0fb40b1e40 msgr2=0x7f0fb40b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:04.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.231+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 shutdown_connections 2024-04-25T09:16:04.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.231+0000 7f0fbb600640 1 -- 172.21.15.62:0/1874155023 wait complete. 2024-04-25T09:16:04.234 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:16:04.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.455+0000 7f62dcc00640 1 Processor -- start 2024-04-25T09:16:04.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.455+0000 7f62dcc00640 1 -- start start 2024-04-25T09:16:04.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.456+0000 7f62dcc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f62d806e330 0x7f62d806cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.456+0000 7f62dcc00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d806d7f0 con 0x7f62d815e050 2024-04-25T09:16:04.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.456+0000 7f62dcc00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d806d230 con 0x7f62d815dc70 2024-04-25T09:16:04.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.456+0000 7f62dcc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d806d000 con 0x7f62d806e330 2024-04-25T09:16:04.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.457+0000 7f62cfe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f62d815e050 0x7f62d806c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37306/0 (socket says 172.21.15.62:37306) 2024-04-25T09:16:04.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.457+0000 7f62cfe00640 1 -- 172.21.15.62:0/574886646 learned_addr learned my addr 172.21.15.62:0/574886646 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:04.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.457+0000 7f62d6800640 1 --2- 172.21.15.62:0/574886646 >> v2:172.21.15.62:3300/0 conn(0x7f62d806e330 0x7f62d806cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:04.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.458+0000 7f62d6800640 1 -- 172.21.15.62:0/574886646 >> v1:172.21.15.121:6789/0 conn(0x7f62d815dc70 legacy=0x7f62d806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:04.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.458+0000 7f62d5400640 1 -- 172.21.15.62:0/574886646 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214426651 0 0) 0x7f62d806d7f0 con 0x7f62d815e050 2024-04-25T09:16:04.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.458+0000 7f62d6800640 1 -- 172.21.15.62:0/574886646 >> v1:172.21.15.62:6789/0 conn(0x7f62d815e050 legacy=0x7f62d806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.458+0000 7f62d6800640 1 -- 172.21.15.62:0/574886646 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d806d510 con 0x7f62d806e330 2024-04-25T09:16:04.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.459+0000 7f62d6800640 1 --2- 172.21.15.62:0/574886646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d806cac0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009dc0 tx=0x7f62d002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd917420ba7b442e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:04.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.459+0000 7f62d5400640 1 -- 172.21.15.62:0/574886646 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0007740 con 0x7f62d806e330 2024-04-25T09:16:04.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.459+0000 7f62d5400640 1 -- 172.21.15.62:0/574886646 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d00078e0 con 0x7f62d806e330 2024-04-25T09:16:04.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.459+0000 7f62d5400640 1 -- 172.21.15.62:0/574886646 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0007cb0 con 0x7f62d806e330 2024-04-25T09:16:04.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.460+0000 7f62dcc00640 1 -- 172.21.15.62:0/574886646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 msgr2=0x7f62d806cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.460+0000 7f62dcc00640 1 --2- 172.21.15.62:0/574886646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d806cac0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009dc0 tx=0x7f62d002a730 comp rx=0 tx=0).stop 2024-04-25T09:16:04.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.460+0000 7f62dcc00640 1 -- 172.21.15.62:0/574886646 shutdown_connections 2024-04-25T09:16:04.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.460+0000 7f62dcc00640 1 --2- 172.21.15.62:0/574886646 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d806cac0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.460+0000 7f62dcc00640 1 -- 172.21.15.62:0/574886646 >> 172.21.15.62:0/574886646 conn(0x7f62d80b1410 msgr2=0x7f62d80b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:04.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.460+0000 7f62dcc00640 1 -- 172.21.15.62:0/574886646 shutdown_connections 2024-04-25T09:16:04.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.460+0000 7f62dcc00640 1 -- 172.21.15.62:0/574886646 wait complete. 2024-04-25T09:16:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.462+0000 7f62dcc00640 1 Processor -- start 2024-04-25T09:16:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.462+0000 7f62dcc00640 1 -- start start 2024-04-25T09:16:04.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62dcc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d81f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62dcc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62d815dc70 0x7f62d8160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62dcc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d815e050 0x7f62d80ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62dcc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b3fc0 con 0x7f62d806e330 2024-04-25T09:16:04.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62dcc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b2840 con 0x7f62d815e050 2024-04-25T09:16:04.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62dcc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b4140 con 0x7f62d815dc70 2024-04-25T09:16:04.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62d5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d81f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:04.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62d5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d81f7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39806/0 (socket says 172.21.15.62:39806) 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.463+0000 7f62d5e00640 1 -- 172.21.15.62:0/1333258123 learned_addr learned my addr 172.21.15.62:0/1333258123 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62d6800640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d815e050 0x7f62d80ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62cfe00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62d815dc70 0x7f62d8160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62d5e00640 1 -- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62d815dc70 msgr2=0x7f62d8160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62d5e00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62d815dc70 0x7f62d8160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62d5e00640 1 -- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d815e050 msgr2=0x7f62d80ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62d5e00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d815e050 0x7f62d80ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62d5e00640 1 -- 172.21.15.62:0/1333258123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d8136a50 con 0x7f62d806e330 2024-04-25T09:16:04.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62d5e00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d81f7cc0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c430 tx=0x7f62c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:04.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62ce000640 1 -- 172.21.15.62:0/1333258123 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80052d0 con 0x7f62d806e330 2024-04-25T09:16:04.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62ce000640 1 -- 172.21.15.62:0/1333258123 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8005490 con 0x7f62d806e330 2024-04-25T09:16:04.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.464+0000 7f62ce000640 1 -- 172.21.15.62:0/1333258123 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8007260 con 0x7f62d806e330 2024-04-25T09:16:04.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.465+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d8136d30 con 0x7f62d806e330 2024-04-25T09:16:04.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.465+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d81372f0 con 0x7f62d806e330 2024-04-25T09:16:04.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.466+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8071c60 con 0x7f62d806e330 2024-04-25T09:16:04.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.467+0000 7f62ce000640 1 -- 172.21.15.62:0/1333258123 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62c8007400 con 0x7f62d806e330 2024-04-25T09:16:04.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.468+0000 7f62ce000640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62b406b4d0 0x7f62b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:04.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.468+0000 7f62ce000640 1 -- 172.21.15.62:0/1333258123 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62c8089790 con 0x7f62d806e330 2024-04-25T09:16:04.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.469+0000 7f62cfe00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62b406b4d0 0x7f62b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:04.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.470+0000 7f62cfe00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62b406b4d0 0x7f62b406d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f62d80b0d20 tx=0x7f62c0007d00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:04.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.473+0000 7f62ce000640 1 -- 172.21.15.62:0/1333258123 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62d8071c60 con 0x7f62d806e330 2024-04-25T09:16:04.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:04.710+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f62d815a850 con 0x7f62d806e330 2024-04-25T09:16:05.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.389+0000 7f62ce000640 1 -- 172.21.15.62:0/1333258123 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) v1 ==== 118+0+0 (secure 0 0 0) 0x7f62c8058570 con 0x7f62d806e330 2024-04-25T09:16:05.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62b406b4d0 msgr2=0x7f62b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:05.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62b406b4d0 0x7f62b406d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f62d80b0d20 tx=0x7f62c0007d00 comp rx=0 tx=0).stop 2024-04-25T09:16:05.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 msgr2=0x7f62d81f7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d81f7cc0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c430 tx=0x7f62c800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 shutdown_connections 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f62b406b4d0 0x7f62b406d990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62d815e050 0x7f62d80ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f62d815dc70 0x7f62d8160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 --2- 172.21.15.62:0/1333258123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62d806e330 0x7f62d81f7cc0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 >> 172.21.15.62:0/1333258123 conn(0x7f62d80b1410 msgr2=0x7f62d8151600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 shutdown_connections 2024-04-25T09:16:05.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.393+0000 7f62dcc00640 1 -- 172.21.15.62:0/1333258123 wait complete. 2024-04-25T09:16:05.401 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:16:05.401 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:16:05.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.629+0000 7f8903200640 1 Processor -- start 2024-04-25T09:16:05.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.630+0000 7f8903200640 1 -- start start 2024-04-25T09:16:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.631+0000 7f8903200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f890415dd50 0x7f89040d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:05.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.631+0000 7f8903200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89040d2fb0 0x7f89040ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:05.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.631+0000 7f8903200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89040d3960 0x7f89040d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:05.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.631+0000 7f8903200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89040ce780 con 0x7f89040d2fb0 2024-04-25T09:16:05.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.631+0000 7f8903200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89040d42f0 con 0x7f890415dd50 2024-04-25T09:16:05.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.631+0000 7f8903200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89040d3440 con 0x7f89040d3960 2024-04-25T09:16:05.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.631+0000 7f8901400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89040d2fb0 0x7f89040ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:05.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89040d2fb0 0x7f89040ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35792/0 (socket says 172.21.15.62:35792) 2024-04-25T09:16:05.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901400640 1 -- 172.21.15.62:0/478644983 learned_addr learned my addr 172.21.15.62:0/478644983 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:05.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901e00640 1 --2- 172.21.15.62:0/478644983 >> v2:172.21.15.121:3300/0 conn(0x7f890415dd50 0x7f89040d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:05.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901400640 1 -- 172.21.15.62:0/478644983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89040d3960 msgr2=0x7f89040d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:05.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8902800640 1 --2- 172.21.15.62:0/478644983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89040d3960 0x7f89040d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:05.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901400640 1 --2- 172.21.15.62:0/478644983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89040d3960 0x7f89040d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901400640 1 -- 172.21.15.62:0/478644983 >> v2:172.21.15.121:3300/0 conn(0x7f890415dd50 msgr2=0x7f89040d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:05.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901400640 1 --2- 172.21.15.62:0/478644983 >> v2:172.21.15.121:3300/0 conn(0x7f890415dd50 0x7f89040d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901400640 1 -- 172.21.15.62:0/478644983 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89040ce1c0 con 0x7f89040d2fb0 2024-04-25T09:16:05.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8901e00640 1 --2- 172.21.15.62:0/478644983 >> v2:172.21.15.121:3300/0 conn(0x7f890415dd50 0x7f89040d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:05.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.632+0000 7f8902800640 1 --2- 172.21.15.62:0/478644983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89040d3960 0x7f89040d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:05.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.633+0000 7f8901400640 1 --2- 172.21.15.62:0/478644983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d2fb0 0x7f89040ce9c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f89040d1320 tx=0x7f88ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfda7aaa41fb944 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8900a00640 1 -- 172.21.15.62:0/478644983 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec037070 con 0x7f89040d2fb0 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8900a00640 1 -- 172.21.15.62:0/478644983 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec032440 con 0x7f89040d2fb0 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8900a00640 1 -- 172.21.15.62:0/478644983 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec032810 con 0x7f89040d2fb0 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8903200640 1 -- 172.21.15.62:0/478644983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d2fb0 msgr2=0x7f89040ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8903200640 1 --2- 172.21.15.62:0/478644983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d2fb0 0x7f89040ce9c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f89040d1320 tx=0x7f88ec02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8903200640 1 -- 172.21.15.62:0/478644983 shutdown_connections 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8903200640 1 --2- 172.21.15.62:0/478644983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f89040d3960 0x7f89040d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8903200640 1 --2- 172.21.15.62:0/478644983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d2fb0 0x7f89040ce9c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8903200640 1 --2- 172.21.15.62:0/478644983 >> v2:172.21.15.121:3300/0 conn(0x7f890415dd50 0x7f89040d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.634+0000 7f8903200640 1 -- 172.21.15.62:0/478644983 >> 172.21.15.62:0/478644983 conn(0x7f890406b000 msgr2=0x7f890406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:05.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.635+0000 7f8903200640 1 -- 172.21.15.62:0/478644983 shutdown_connections 2024-04-25T09:16:05.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.635+0000 7f8903200640 1 -- 172.21.15.62:0/478644983 wait complete. 2024-04-25T09:16:05.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.637+0000 7f8903200640 1 Processor -- start 2024-04-25T09:16:05.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.637+0000 7f8903200640 1 -- start start 2024-04-25T09:16:05.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8903200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d3960 0x7f89041587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:05.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8903200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890415dd50 0x7f89041f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:05.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8903200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890415a2f0 0x7f890414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:05.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8903200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040cef00 con 0x7f89040d3960 2024-04-25T09:16:05.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8903200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040d2840 con 0x7f890415dd50 2024-04-25T09:16:05.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8903200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89040b6b00 con 0x7f890415a2f0 2024-04-25T09:16:05.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8901e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d3960 0x7f89041587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:05.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8901e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d3960 0x7f89041587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35804/0 (socket says 172.21.15.62:35804) 2024-04-25T09:16:05.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.638+0000 7f8901e00640 1 -- 172.21.15.62:0/2856014181 learned_addr learned my addr 172.21.15.62:0/2856014181 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:05.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8902800640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890415a2f0 0x7f890414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:05.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8901400640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890415dd50 0x7f89041f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:05.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8901e00640 1 -- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890415a2f0 msgr2=0x7f890414c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8901e00640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890415a2f0 0x7f890414c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8901e00640 1 -- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890415dd50 msgr2=0x7f89041f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8901e00640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890415dd50 0x7f89041f7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8901e00640 1 -- 172.21.15.62:0/2856014181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8904158eb0 con 0x7f89040d3960 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8902800640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890415a2f0 0x7f890414c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f8901e00640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d3960 0x7f89041587f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f88f400ce00 tx=0x7f88f400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.639+0000 7f88fa000640 1 -- 172.21.15.62:0/2856014181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f400f890 con 0x7f89040d3960 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.640+0000 7f88fa000640 1 -- 172.21.15.62:0/2856014181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f400fa30 con 0x7f89040d3960 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.640+0000 7f88fa000640 1 -- 172.21.15.62:0/2856014181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f400fe40 con 0x7f89040d3960 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.640+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8904159190 con 0x7f89040d3960 2024-04-25T09:16:05.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.640+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8904159750 con 0x7f89040d3960 2024-04-25T09:16:05.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.641+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890404ef20 con 0x7f89040d3960 2024-04-25T09:16:05.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.642+0000 7f88fa000640 1 -- 172.21.15.62:0/2856014181 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88f4018400 con 0x7f89040d3960 2024-04-25T09:16:05.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.643+0000 7f88fa000640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88e006b4d0 0x7f88e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:05.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.643+0000 7f88fa000640 1 -- 172.21.15.62:0/2856014181 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88f408af50 con 0x7f89040d3960 2024-04-25T09:16:05.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.643+0000 7f8901400640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88e006b4d0 0x7f88e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:05.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.644+0000 7f8901400640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88e006b4d0 0x7f88e006d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f88ec034000 tx=0x7f88ec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:05.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.647+0000 7f88fa000640 1 -- 172.21.15.62:0/2856014181 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88f4014030 con 0x7f89040d3960 2024-04-25T09:16:05.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:05.888+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f89040b9950 con 0x7f89040d3960 2024-04-25T09:16:06.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.401+0000 7f88fa000640 1 -- 172.21.15.62:0/2856014181 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88f400ae20 con 0x7f89040d3960 2024-04-25T09:16:06.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88e006b4d0 msgr2=0x7f88e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88e006b4d0 0x7f88e006d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f88ec034000 tx=0x7f88ec02e000 comp rx=0 tx=0).stop 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d3960 msgr2=0x7f89041587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d3960 0x7f89041587f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f88f400ce00 tx=0x7f88f400a650 comp rx=0 tx=0).stop 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 shutdown_connections 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f88e006b4d0 0x7f88e006d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f890415a2f0 0x7f890414c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f890415dd50 0x7f89041f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 --2- 172.21.15.62:0/2856014181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89040d3960 0x7f89041587f0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 >> 172.21.15.62:0/2856014181 conn(0x7f890406b000 msgr2=0x7f89040cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.405+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 shutdown_connections 2024-04-25T09:16:06.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.406+0000 7f8903200640 1 -- 172.21.15.62:0/2856014181 wait complete. 2024-04-25T09:16:06.413 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:16:06.413 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.637+0000 7f0aec600640 1 Processor -- start 2024-04-25T09:16:06.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.638+0000 7f0aec600640 1 -- start start 2024-04-25T09:16:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.639+0000 7f0aec600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.639+0000 7f0aec600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ae40cccf0 0x7f0ae40ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.639+0000 7f0aec600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae406ca70 con 0x7f0ae40aed00 2024-04-25T09:16:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.639+0000 7f0aec600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae406d660 con 0x7f0ae40cccf0 2024-04-25T09:16:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.639+0000 7f0aec600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae40ce6d0 con 0x7f0ae415bd10 2024-04-25T09:16:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ae40cccf0 0x7f0ae40ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ae40cccf0 0x7f0ae40ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35808/0 (socket says 172.21.15.62:35808) 2024-04-25T09:16:06.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9200640 1 -- 172.21.15.62:0/1302029506 learned_addr learned my addr 172.21.15.62:0/1302029506 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9c00640 1 --2- 172.21.15.62:0/1302029506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9200640 1 -- 172.21.15.62:0/1302029506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 msgr2=0x7f0ae40cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9200640 1 --2- 172.21.15.62:0/1302029506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae40cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9200640 1 -- 172.21.15.62:0/1302029506 >> v1:172.21.15.121:6789/0 conn(0x7f0ae40aed00 legacy=0x7f0ae415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9200640 1 -- 172.21.15.62:0/1302029506 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae415c0f0 con 0x7f0ae40cccf0 2024-04-25T09:16:06.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.640+0000 7f0ae9c00640 1 --2- 172.21.15.62:0/1302029506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae40cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.641+0000 7f0ae9200640 1 --2- 172.21.15.62:0/1302029506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40ae7c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f0ae406d7e0 tx=0x7f0ad002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80aa14da39d7e489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.642+0000 7f0ae3e00640 1 -- 172.21.15.62:0/1302029506 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0037070 con 0x7f0ae40cccf0 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.642+0000 7f0ae3e00640 1 -- 172.21.15.62:0/1302029506 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad0032440 con 0x7f0ae40cccf0 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.642+0000 7f0ae3e00640 1 -- 172.21.15.62:0/1302029506 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0032810 con 0x7f0ae40cccf0 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.642+0000 7f0aec600640 1 -- 172.21.15.62:0/1302029506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 msgr2=0x7f0ae40ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.642+0000 7f0aec600640 1 --2- 172.21.15.62:0/1302029506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40ae7c0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f0ae406d7e0 tx=0x7f0ad002a200 comp rx=0 tx=0).stop 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.643+0000 7f0aec600640 1 -- 172.21.15.62:0/1302029506 shutdown_connections 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.643+0000 7f0aec600640 1 --2- 172.21.15.62:0/1302029506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40ae7c0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.643+0000 7f0aec600640 1 --2- 172.21.15.62:0/1302029506 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae40cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.643+0000 7f0aec600640 1 -- 172.21.15.62:0/1302029506 >> 172.21.15.62:0/1302029506 conn(0x7f0ae40668f0 msgr2=0x7f0ae4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.643+0000 7f0aec600640 1 -- 172.21.15.62:0/1302029506 shutdown_connections 2024-04-25T09:16:06.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.643+0000 7f0aec600640 1 -- 172.21.15.62:0/1302029506 wait complete. 2024-04-25T09:16:06.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.644+0000 7f0aec600640 1 Processor -- start 2024-04-25T09:16:06.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.645+0000 7f0aec600640 1 -- start start 2024-04-25T09:16:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.645+0000 7f0aec600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ae40aed00 0x7f0ae4141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.645+0000 7f0aec600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.645+0000 7f0aec600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae4070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.645+0000 7f0aec600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae406d2f0 con 0x7f0ae40cccf0 2024-04-25T09:16:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.645+0000 7f0aec600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae406d010 con 0x7f0ae40aed00 2024-04-25T09:16:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.645+0000 7f0aec600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae40d01e0 con 0x7f0ae415bd10 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35824/0 (socket says 172.21.15.62:35824) 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9200640 1 -- 172.21.15.62:0/2264500294 learned_addr learned my addr 172.21.15.62:0/2264500294 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0aea600640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae4070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9c00640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ae40aed00 0x7f0ae4141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9200640 1 -- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 msgr2=0x7f0ae4070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9200640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae4070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9200640 1 -- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ae40aed00 msgr2=0x7f0ae4141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9200640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ae40aed00 0x7f0ae4141ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.646+0000 7f0ae9200640 1 -- 172.21.15.62:0/2264500294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae414f8c0 con 0x7f0ae40cccf0 2024-04-25T09:16:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.647+0000 7f0ae9200640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40b1b70 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c00 tx=0x7f0ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.647+0000 7f0ae1600640 1 -- 172.21.15.62:0/2264500294 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0041020 con 0x7f0ae40cccf0 2024-04-25T09:16:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.647+0000 7f0ae1600640 1 -- 172.21.15.62:0/2264500294 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad0030970 con 0x7f0ae40cccf0 2024-04-25T09:16:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.647+0000 7f0ae1600640 1 -- 172.21.15.62:0/2264500294 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0037040 con 0x7f0ae40cccf0 2024-04-25T09:16:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.647+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae41421e0 con 0x7f0ae40cccf0 2024-04-25T09:16:06.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.647+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae4145430 con 0x7f0ae40cccf0 2024-04-25T09:16:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.648+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae4072c70 con 0x7f0ae40cccf0 2024-04-25T09:16:06.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.649+0000 7f0ae1600640 1 -- 172.21.15.62:0/2264500294 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ad0044020 con 0x7f0ae40cccf0 2024-04-25T09:16:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.650+0000 7f0ae1600640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0ac406b4d0 0x7f0ac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:06.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.650+0000 7f0ae1600640 1 -- 172.21.15.62:0/2264500294 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ad00aabb0 con 0x7f0ae40cccf0 2024-04-25T09:16:06.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.651+0000 7f0ae9c00640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0ac406b4d0 0x7f0ac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:06.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.652+0000 7f0ae9c00640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0ac406b4d0 0x7f0ac406d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4008ee0 tx=0x7f0ad4008430 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:06.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.654+0000 7f0ae1600640 1 -- 172.21.15.62:0/2264500294 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ad004b030 con 0x7f0ae40cccf0 2024-04-25T09:16:06.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.896+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ae4066fb0 con 0x7f0ae40cccf0 2024-04-25T09:16:06.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.897+0000 7f0ae1600640 1 -- 172.21.15.62:0/2264500294 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0ad0079a60 con 0x7f0ae40cccf0 2024-04-25T09:16:06.884 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:06.884 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:06.884 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:06.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0ac406b4d0 msgr2=0x7f0ac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:06.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0ac406b4d0 0x7f0ac406d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4008ee0 tx=0x7f0ad4008430 comp rx=0 tx=0).stop 2024-04-25T09:16:06.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 msgr2=0x7f0ae40b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:06.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40b1b70 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c00 tx=0x7f0ad00046c0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 shutdown_connections 2024-04-25T09:16:06.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0ac406b4d0 0x7f0ac406d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ae40cccf0 0x7f0ae40b1b70 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0ae415bd10 0x7f0ae4070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 --2- 172.21.15.62:0/2264500294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ae40aed00 0x7f0ae4141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:06.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.900+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 >> 172.21.15.62:0/2264500294 conn(0x7f0ae40668f0 msgr2=0x7f0ae415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:06.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.901+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 shutdown_connections 2024-04-25T09:16:06.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:06.901+0000 7f0aec600640 1 -- 172.21.15.62:0/2264500294 wait complete. 2024-04-25T09:16:06.907 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:16:07.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.121+0000 7f26f7e00640 1 Processor -- start 2024-04-25T09:16:07.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.121+0000 7f26f7e00640 1 -- start start 2024-04-25T09:16:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.123+0000 7f26f7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f26f815d810 0x7f26f80ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.123+0000 7f26f7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f26f806d900 0x7f26f81675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.123+0000 7f26f7e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f80ced70 con 0x7f26f815dbf0 2024-04-25T09:16:07.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.123+0000 7f26f7e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f80cf030 con 0x7f26f806d900 2024-04-25T09:16:07.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.123+0000 7f26f7e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f80cf310 con 0x7f26f815d810 2024-04-25T09:16:07.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.123+0000 7f26ee000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f26f815dbf0 0x7f26f806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57368/0 (socket says 172.21.15.62:57368) 2024-04-25T09:16:07.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.123+0000 7f26ee000640 1 -- 172.21.15.62:0/449824022 learned_addr learned my addr 172.21.15.62:0/449824022 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:07.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.123+0000 7f26f7400640 1 --2- 172.21.15.62:0/449824022 >> v2:172.21.15.62:3300/0 conn(0x7f26f806d900 0x7f26f81675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f6a00640 1 --2- 172.21.15.62:0/449824022 >> v2:172.21.15.121:3300/0 conn(0x7f26f815d810 0x7f26f80ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f6000640 1 -- 172.21.15.62:0/449824022 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457094909 0 0) 0x7f26f80ced70 con 0x7f26f815dbf0 2024-04-25T09:16:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f6000640 1 -- 172.21.15.62:0/449824022 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26dc003520 con 0x7f26f815dbf0 2024-04-25T09:16:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f7400640 1 -- 172.21.15.62:0/449824022 >> v2:172.21.15.121:3300/0 conn(0x7f26f815d810 msgr2=0x7f26f80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f7400640 1 --2- 172.21.15.62:0/449824022 >> v2:172.21.15.121:3300/0 conn(0x7f26f815d810 0x7f26f80ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f7400640 1 -- 172.21.15.62:0/449824022 >> v1:172.21.15.62:6789/0 conn(0x7f26f815dbf0 legacy=0x7f26f806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f6000640 1 -- 172.21.15.62:0/449824022 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903156328 0 0) 0x7f26dc003520 con 0x7f26f815dbf0 2024-04-25T09:16:07.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f7400640 1 -- 172.21.15.62:0/449824022 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80c8cd0 con 0x7f26f806d900 2024-04-25T09:16:07.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.124+0000 7f26f6a00640 1 --2- 172.21.15.62:0/449824022 >> v2:172.21.15.121:3300/0 conn(0x7f26f815d810 0x7f26f80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:16:07.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.125+0000 7f26f7400640 1 --2- 172.21.15.62:0/449824022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f806d900 0x7f26f81675e0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f26f80b5a90 tx=0x7f26f002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689aaa26e1c8d7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:07.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f6000640 1 -- 172.21.15.62:0/449824022 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0037070 con 0x7f26f806d900 2024-04-25T09:16:07.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f6000640 1 -- 172.21.15.62:0/449824022 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f002abc0 con 0x7f26f806d900 2024-04-25T09:16:07.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f6000640 1 -- 172.21.15.62:0/449824022 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0001670 con 0x7f26f806d900 2024-04-25T09:16:07.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f7e00640 1 -- 172.21.15.62:0/449824022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f806d900 msgr2=0x7f26f81675e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f7e00640 1 --2- 172.21.15.62:0/449824022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f806d900 0x7f26f81675e0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f26f80b5a90 tx=0x7f26f002a130 comp rx=0 tx=0).stop 2024-04-25T09:16:07.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f7e00640 1 -- 172.21.15.62:0/449824022 shutdown_connections 2024-04-25T09:16:07.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f7e00640 1 --2- 172.21.15.62:0/449824022 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f806d900 0x7f26f81675e0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f7e00640 1 --2- 172.21.15.62:0/449824022 >> v2:172.21.15.121:3300/0 conn(0x7f26f815d810 0x7f26f80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f7e00640 1 -- 172.21.15.62:0/449824022 >> 172.21.15.62:0/449824022 conn(0x7f26f80b14d0 msgr2=0x7f26f80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:07.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.126+0000 7f26f7e00640 1 -- 172.21.15.62:0/449824022 shutdown_connections 2024-04-25T09:16:07.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.127+0000 7f26f7e00640 1 -- 172.21.15.62:0/449824022 wait complete. 2024-04-25T09:16:07.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.128+0000 7f26f7e00640 1 Processor -- start 2024-04-25T09:16:07.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.129+0000 7f26f7e00640 1 -- start start 2024-04-25T09:16:07.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26f806d900 0x7f26f814c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f815d810 0x7f26f81f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26f815dbf0 0x7f26f814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80ca660 con 0x7f26f815d810 2024-04-25T09:16:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80ca4e0 con 0x7f26f815dbf0 2024-04-25T09:16:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80ca7e0 con 0x7f26f806d900 2024-04-25T09:16:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26ee000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f815d810 0x7f26f81f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26ee000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f815d810 0x7f26f81f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35838/0 (socket says 172.21.15.62:35838) 2024-04-25T09:16:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26ee000640 1 -- 172.21.15.62:0/3260572727 learned_addr learned my addr 172.21.15.62:0/3260572727 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f6a00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26f806d900 0x7f26f814c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26ee000640 1 -- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26f806d900 msgr2=0x7f26f814c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26ee000640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26f806d900 0x7f26f814c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26ee000640 1 -- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26f815dbf0 msgr2=0x7f26f814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f7400640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26f815dbf0 0x7f26f814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26ee000640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26f815dbf0 0x7f26f814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26ee000640 1 -- 172.21.15.62:0/3260572727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f81f3b50 con 0x7f26f815d810 2024-04-25T09:16:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.130+0000 7f26f6a00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26f806d900 0x7f26f814c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.131+0000 7f26ee000640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f815d810 0x7f26f81f3450 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c700 tx=0x7f26e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.131+0000 7f26efe00640 1 -- 172.21.15.62:0/3260572727 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0017070 con 0x7f26f815d810 2024-04-25T09:16:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.131+0000 7f26efe00640 1 -- 172.21.15.62:0/3260572727 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e0005ad0 con 0x7f26f815d810 2024-04-25T09:16:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.131+0000 7f26efe00640 1 -- 172.21.15.62:0/3260572727 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0007070 con 0x7f26f815d810 2024-04-25T09:16:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.131+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f81f3e30 con 0x7f26f815d810 2024-04-25T09:16:07.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.131+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f81f7100 con 0x7f26f815d810 2024-04-25T09:16:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.132+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f81f4350 con 0x7f26f815d810 2024-04-25T09:16:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.133+0000 7f26efe00640 1 -- 172.21.15.62:0/3260572727 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26e001a070 con 0x7f26f815d810 2024-04-25T09:16:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.134+0000 7f26efe00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26d406b4d0 0x7f26d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.134+0000 7f26efe00640 1 -- 172.21.15.62:0/3260572727 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26e0089580 con 0x7f26f815d810 2024-04-25T09:16:07.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.135+0000 7f26f6a00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26d406b4d0 0x7f26d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.136+0000 7f26f6a00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26d406b4d0 0x7f26d406d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f26e80094c0 tx=0x7f26e800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:07.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.139+0000 7f26efe00640 1 -- 172.21.15.62:0/3260572727 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26f81f4350 con 0x7f26f815d810 2024-04-25T09:16:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.338+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f81f6a40 con 0x7f26f815d810 2024-04-25T09:16:07.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.340+0000 7f26efe00640 1 -- 172.21.15.62:0/3260572727 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f26e0057da0 con 0x7f26f815d810 2024-04-25T09:16:07.327 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:07.327 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":137,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:16:04.361907+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:16:00.695740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:16:01.838452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:16:07.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26d406b4d0 msgr2=0x7f26d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26d406b4d0 0x7f26d406d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f26e80094c0 tx=0x7f26e800b040 comp rx=0 tx=0).stop 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f815d810 msgr2=0x7f26f81f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f815d810 0x7f26f81f3450 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c700 tx=0x7f26e000cbc0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 shutdown_connections 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f26d406b4d0 0x7f26d406d990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26f815d810 0x7f26f81f3450 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26f815dbf0 0x7f26f814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 --2- 172.21.15.62:0/3260572727 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f26f806d900 0x7f26f814c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 >> 172.21.15.62:0/3260572727 conn(0x7f26f80b14d0 msgr2=0x7f26f8070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 shutdown_connections 2024-04-25T09:16:07.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.343+0000 7f26f7e00640 1 -- 172.21.15.62:0/3260572727 wait complete. 2024-04-25T09:16:07.349 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:16:07.350 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:16:07.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.568+0000 7fc90a800640 1 Processor -- start 2024-04-25T09:16:07.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.568+0000 7fc90a800640 1 -- start start 2024-04-25T09:16:07.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc90a800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc90415dbf0 0x7fc90406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc90a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9040ca4e0 0x7fc90406d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc90a800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9040cf030 con 0x7fc90415d810 2024-04-25T09:16:07.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc90a800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040cf310 con 0x7fc9040ca4e0 2024-04-25T09:16:07.556 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc90a800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040c8cd0 con 0x7fc90415dbf0 2024-04-25T09:16:07.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc909e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9040ca4e0 0x7fc90406d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc909e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9040ca4e0 0x7fc90406d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35852/0 (socket says 172.21.15.62:35852) 2024-04-25T09:16:07.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc909e00640 1 -- 172.21.15.62:0/722044839 learned_addr learned my addr 172.21.15.62:0/722044839 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:07.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.570+0000 7fc908a00640 1 --2- 172.21.15.62:0/722044839 >> v2:172.21.15.121:3300/0 conn(0x7fc90415dbf0 0x7fc90406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.571+0000 7fc909e00640 1 -- 172.21.15.62:0/722044839 >> v2:172.21.15.121:3300/0 conn(0x7fc90415dbf0 msgr2=0x7fc90406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:07.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.571+0000 7fc909e00640 1 --2- 172.21.15.62:0/722044839 >> v2:172.21.15.121:3300/0 conn(0x7fc90415dbf0 0x7fc90406d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.571+0000 7fc909e00640 1 -- 172.21.15.62:0/722044839 >> v1:172.21.15.121:6789/0 conn(0x7fc90415d810 legacy=0x7fc9040ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.571+0000 7fc909e00640 1 -- 172.21.15.62:0/722044839 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040c8fb0 con 0x7fc9040ca4e0 2024-04-25T09:16:07.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.572+0000 7fc909e00640 1 --2- 172.21.15.62:0/722044839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9040ca4e0 0x7fc90406d730 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc90406e100 tx=0x7fc8fc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34cb90316ddcf91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:07.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.572+0000 7fc903e00640 1 -- 172.21.15.62:0/722044839 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc036070 con 0x7fc9040ca4e0 2024-04-25T09:16:07.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.572+0000 7fc903e00640 1 -- 172.21.15.62:0/722044839 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc031600 con 0x7fc9040ca4e0 2024-04-25T09:16:07.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.572+0000 7fc903e00640 1 -- 172.21.15.62:0/722044839 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc0319d0 con 0x7fc9040ca4e0 2024-04-25T09:16:07.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.573+0000 7fc90a800640 1 -- 172.21.15.62:0/722044839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9040ca4e0 msgr2=0x7fc90406d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.573+0000 7fc90a800640 1 --2- 172.21.15.62:0/722044839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9040ca4e0 0x7fc90406d730 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc90406e100 tx=0x7fc8fc02a730 comp rx=0 tx=0).stop 2024-04-25T09:16:07.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.573+0000 7fc90a800640 1 -- 172.21.15.62:0/722044839 shutdown_connections 2024-04-25T09:16:07.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.573+0000 7fc90a800640 1 --2- 172.21.15.62:0/722044839 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9040ca4e0 0x7fc90406d730 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.573+0000 7fc90a800640 1 --2- 172.21.15.62:0/722044839 >> v2:172.21.15.121:3300/0 conn(0x7fc90415dbf0 0x7fc90406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.573+0000 7fc90a800640 1 -- 172.21.15.62:0/722044839 >> 172.21.15.62:0/722044839 conn(0x7fc9040b14d0 msgr2=0x7fc9040b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:07.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.573+0000 7fc90a800640 1 -- 172.21.15.62:0/722044839 shutdown_connections 2024-04-25T09:16:07.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.573+0000 7fc90a800640 1 -- 172.21.15.62:0/722044839 wait complete. 2024-04-25T09:16:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.575+0000 7fc90a800640 1 Processor -- start 2024-04-25T09:16:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.575+0000 7fc90a800640 1 -- start start 2024-04-25T09:16:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc90a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9040ca4e0 0x7fc90406c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc90a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc90415d810 0x7fc90406ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc90a800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc90415dbf0 0x7fc90413b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc90a800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041675e0 con 0x7fc90415d810 2024-04-25T09:16:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc90a800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9040b10d0 con 0x7fc9040ca4e0 2024-04-25T09:16:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc90a800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc904167760 con 0x7fc90415dbf0 2024-04-25T09:16:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc908a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc90415d810 0x7fc90406ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc908a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc90415d810 0x7fc90406ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35860/0 (socket says 172.21.15.62:35860) 2024-04-25T09:16:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.576+0000 7fc908a00640 1 -- 172.21.15.62:0/1360176163 learned_addr learned my addr 172.21.15.62:0/1360176163 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc909400640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9040ca4e0 0x7fc90406c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc909e00640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc90415dbf0 0x7fc90413b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc908a00640 1 -- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc90415dbf0 msgr2=0x7fc90413b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc908a00640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc90415dbf0 0x7fc90413b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc908a00640 1 -- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9040ca4e0 msgr2=0x7fc90406c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc908a00640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9040ca4e0 0x7fc90406c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc908a00640 1 -- 172.21.15.62:0/1360176163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90413db30 con 0x7fc90415d810 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc908a00640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc90415d810 0x7fc90406ca00 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000d5f0 tx=0x7fc8f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc901600640 1 -- 172.21.15.62:0/1360176163 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00125f0 con 0x7fc90415d810 2024-04-25T09:16:07.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc901600640 1 -- 172.21.15.62:0/1360176163 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0019070 con 0x7fc90415d810 2024-04-25T09:16:07.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc901600640 1 -- 172.21.15.62:0/1360176163 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00129d0 con 0x7fc90415d810 2024-04-25T09:16:07.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.577+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9041596f0 con 0x7fc90415d810 2024-04-25T09:16:07.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.578+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90413e090 con 0x7fc90415d810 2024-04-25T09:16:07.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.579+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90404ef20 con 0x7fc90415d810 2024-04-25T09:16:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.580+0000 7fc901600640 1 -- 172.21.15.62:0/1360176163 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8f0012b70 con 0x7fc90415d810 2024-04-25T09:16:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.581+0000 7fc901600640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8e406b4d0 0x7fc8e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.581+0000 7fc901600640 1 -- 172.21.15.62:0/1360176163 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8f008a670 con 0x7fc90415d810 2024-04-25T09:16:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.581+0000 7fc909400640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8e406b4d0 0x7fc8e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:07.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.582+0000 7fc909400640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8e406b4d0 0x7fc8e406d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8007fd0 tx=0x7fc8f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:07.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.585+0000 7fc901600640 1 -- 172.21.15.62:0/1360176163 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc90413db30 con 0x7fc90415d810 2024-04-25T09:16:07.727 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-04-25T09:16:07.741+0000 7f2c86c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:16:07.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.782+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9040b5800 con 0x7fc8e406b4d0 2024-04-25T09:16:07.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.800+0000 7fc901600640 1 -- 172.21.15.62:0/1360176163 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252482 (secure 0 0 0) 0x7fc9040b5800 con 0x7fc8e406b4d0 2024-04-25T09:16:07.790 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:07.791 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8e406b4d0 msgr2=0x7fc8e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8e406b4d0 0x7fc8e406d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8007fd0 tx=0x7fc8f8009040 comp rx=0 tx=0).stop 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc90415d810 msgr2=0x7fc90406ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc90415d810 0x7fc90406ca00 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000d5f0 tx=0x7fc8f000dab0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 shutdown_connections 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8e406b4d0 0x7fc8e406d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc90415d810 0x7fc90406ca00 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc90415dbf0 0x7fc90413b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 --2- 172.21.15.62:0/1360176163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9040ca4e0 0x7fc90406c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 >> 172.21.15.62:0/1360176163 conn(0x7fc9040b14d0 msgr2=0x7fc9040cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 shutdown_connections 2024-04-25T09:16:07.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:07.806+0000 7fc90a800640 1 -- 172.21.15.62:0/1360176163 wait complete. 2024-04-25T09:16:07.812 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-04-25T09:16:06.181422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":455572,"kb_used_data":14556,"kb_used_omap":12,"kb_used_meta":440883,"kb_avail":754519148,"statfs":{"total":773094113280,"available":772627607552,"internally_reserved":0,"allocated":14905344,"data_stored":7186960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":451464752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.854290"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373567+0000","last_change":"2024-04-25T09:16:03.369068+0000","last_active":"2024-04-25T09:16:04.373567+0000","last_peered":"2024-04-25T09:16:04.373567+0000","last_clean":"2024-04-25T09:16:04.373567+0000","last_became_active":"2024-04-25T09:16:03.368734+0000","last_became_peered":"2024-04-25T09:16:03.368734+0000","last_unstale":"2024-04-25T09:16:04.373567+0000","last_undegraded":"2024-04-25T09:16:04.373567+0000","last_fullsized":"2024-04-25T09:16:04.373567+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:02.379597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373280+0000","last_change":"2024-04-25T09:16:03.387881+0000","last_active":"2024-04-25T09:16:04.373280+0000","last_peered":"2024-04-25T09:16:04.373280+0000","last_clean":"2024-04-25T09:16:04.373280+0000","last_became_active":"2024-04-25T09:16:03.387591+0000","last_became_peered":"2024-04-25T09:16:03.387591+0000","last_unstale":"2024-04-25T09:16:04.373280+0000","last_undegraded":"2024-04-25T09:16:04.373280+0000","last_fullsized":"2024-04-25T09:16:04.373280+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.784334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.374089+0000","last_change":"2024-04-25T09:16:03.369047+0000","last_active":"2024-04-25T09:16:04.374089+0000","last_peered":"2024-04-25T09:16:04.374089+0000","last_clean":"2024-04-25T09:16:04.374089+0000","last_became_active":"2024-04-25T09:16:03.368769+0000","last_became_peered":"2024-04-25T09:16:03.368769+0000","last_unstale":"2024-04-25T09:16:04.374089+0000","last_undegraded":"2024-04-25T09:16:04.374089+0000","last_fullsized":"2024-04-25T09:16:04.374089+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:36.471214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471092+0000","last_change":"2024-04-25T09:16:03.382598+0000","last_active":"2024-04-25T09:16:05.471092+0000","last_peered":"2024-04-25T09:16:05.471092+0000","last_clean":"2024-04-25T09:16:05.471092+0000","last_became_active":"2024-04-25T09:16:03.382298+0000","last_became_peered":"2024-04-25T09:16:03.382298+0000","last_unstale":"2024-04-25T09:16:05.471092+0000","last_undegraded":"2024-04-25T09:16:05.471092+0000","last_fullsized":"2024-04-25T09:16:05.471092+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:29.636797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442233+0000","last_change":"2024-04-25T09:16:03.372813+0000","last_active":"2024-04-25T09:16:04.442233+0000","last_peered":"2024-04-25T09:16:04.442233+0000","last_clean":"2024-04-25T09:16:04.442233+0000","last_became_active":"2024-04-25T09:16:03.372517+0000","last_became_peered":"2024-04-25T09:16:03.372517+0000","last_unstale":"2024-04-25T09:16:04.442233+0000","last_undegraded":"2024-04-25T09:16:04.442233+0000","last_fullsized":"2024-04-25T09:16:04.442233+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:26.049055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371552+0000","last_change":"2024-04-25T09:16:03.369846+0000","last_active":"2024-04-25T09:16:04.371552+0000","last_peered":"2024-04-25T09:16:04.371552+0000","last_clean":"2024-04-25T09:16:04.371552+0000","last_became_active":"2024-04-25T09:16:03.369554+0000","last_became_peered":"2024-04-25T09:16:03.369554+0000","last_unstale":"2024-04-25T09:16:04.371552+0000","last_undegraded":"2024-04-25T09:16:04.371552+0000","last_fullsized":"2024-04-25T09:16:04.371552+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:33.662482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442366+0000","last_change":"2024-04-25T09:16:03.364843+0000","last_active":"2024-04-25T09:16:04.442366+0000","last_peered":"2024-04-25T09:16:04.442366+0000","last_clean":"2024-04-25T09:16:04.442366+0000","last_became_active":"2024-04-25T09:16:03.364552+0000","last_became_peered":"2024-04-25T09:16:03.364552+0000","last_unstale":"2024-04-25T09:16:04.442366+0000","last_undegraded":"2024-04-25T09:16:04.442366+0000","last_fullsized":"2024-04-25T09:16:04.442366+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:34.715830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.441999+0000","last_change":"2024-04-25T09:16:03.371330+0000","last_active":"2024-04-25T09:16:04.441999+0000","last_peered":"2024-04-25T09:16:04.441999+0000","last_clean":"2024-04-25T09:16:04.441999+0000","last_became_active":"2024-04-25T09:16:03.371042+0000","last_became_peered":"2024-04-25T09:16:03.371042+0000","last_unstale":"2024-04-25T09:16:04.441999+0000","last_undegraded":"2024-04-25T09:16:04.441999+0000","last_fullsized":"2024-04-25T09:16:04.441999+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:08.216956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371909+0000","last_change":"2024-04-25T09:16:03.371504+0000","last_active":"2024-04-25T09:16:04.371909+0000","last_peered":"2024-04-25T09:16:04.371909+0000","last_clean":"2024-04-25T09:16:04.371909+0000","last_became_active":"2024-04-25T09:16:03.371196+0000","last_became_peered":"2024-04-25T09:16:03.371196+0000","last_unstale":"2024-04-25T09:16:04.371909+0000","last_undegraded":"2024-04-25T09:16:04.371909+0000","last_fullsized":"2024-04-25T09:16:04.371909+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:02.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373458+0000","last_change":"2024-04-25T09:16:03.371158+0000","last_active":"2024-04-25T09:16:04.373458+0000","last_peered":"2024-04-25T09:16:04.373458+0000","last_clean":"2024-04-25T09:16:04.373458+0000","last_became_active":"2024-04-25T09:16:03.370360+0000","last_became_peered":"2024-04-25T09:16:03.370360+0000","last_unstale":"2024-04-25T09:16:04.373458+0000","last_undegraded":"2024-04-25T09:16:04.373458+0000","last_fullsized":"2024-04-25T09:16:04.373458+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:54.966474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471390+0000","last_change":"2024-04-25T09:16:03.377595+0000","last_active":"2024-04-25T09:16:05.471390+0000","last_peered":"2024-04-25T09:16:05.471390+0000","last_clean":"2024-04-25T09:16:05.471390+0000","last_became_active":"2024-04-25T09:16:03.375843+0000","last_became_peered":"2024-04-25T09:16:03.375843+0000","last_unstale":"2024-04-25T09:16:05.471390+0000","last_undegraded":"2024-04-25T09:16:05.471390+0000","last_fullsized":"2024-04-25T09:16:05.471390+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:48.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373247+0000","last_change":"2024-04-25T09:16:03.380406+0000","last_active":"2024-04-25T09:16:04.373247+0000","last_peered":"2024-04-25T09:16:04.373247+0000","last_clean":"2024-04-25T09:16:04.373247+0000","last_became_active":"2024-04-25T09:16:03.380112+0000","last_became_peered":"2024-04-25T09:16:03.380112+0000","last_unstale":"2024-04-25T09:16:04.373247+0000","last_undegraded":"2024-04-25T09:16:04.373247+0000","last_fullsized":"2024-04-25T09:16:04.373247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.784334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471448+0000","last_change":"2024-04-25T09:16:03.382860+0000","last_active":"2024-04-25T09:16:05.471448+0000","last_peered":"2024-04-25T09:16:05.471448+0000","last_clean":"2024-04-25T09:16:05.471448+0000","last_became_active":"2024-04-25T09:16:03.379568+0000","last_became_peered":"2024-04-25T09:16:03.379568+0000","last_unstale":"2024-04-25T09:16:05.471448+0000","last_undegraded":"2024-04-25T09:16:05.471448+0000","last_fullsized":"2024-04-25T09:16:05.471448+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:33.662482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678143+0000","last_change":"2024-04-25T09:16:03.380933+0000","last_active":"2024-04-25T09:16:05.678143+0000","last_peered":"2024-04-25T09:16:05.678143+0000","last_clean":"2024-04-25T09:16:05.678143+0000","last_became_active":"2024-04-25T09:16:03.380563+0000","last_became_peered":"2024-04-25T09:16:03.380563+0000","last_unstale":"2024-04-25T09:16:05.678143+0000","last_undegraded":"2024-04-25T09:16:05.678143+0000","last_fullsized":"2024-04-25T09:16:05.678143+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:02.379597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676494+0000","last_change":"2024-04-25T09:16:03.388703+0000","last_active":"2024-04-25T09:16:05.676494+0000","last_peered":"2024-04-25T09:16:05.676494+0000","last_clean":"2024-04-25T09:16:05.676494+0000","last_became_active":"2024-04-25T09:16:03.388353+0000","last_became_peered":"2024-04-25T09:16:03.388353+0000","last_unstale":"2024-04-25T09:16:05.676494+0000","last_undegraded":"2024-04-25T09:16:05.676494+0000","last_fullsized":"2024-04-25T09:16:05.676494+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:09.737931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676550+0000","last_change":"2024-04-25T09:16:03.379548+0000","last_active":"2024-04-25T09:16:05.676550+0000","last_peered":"2024-04-25T09:16:05.676550+0000","last_clean":"2024-04-25T09:16:05.676550+0000","last_became_active":"2024-04-25T09:16:03.379204+0000","last_became_peered":"2024-04-25T09:16:03.379204+0000","last_unstale":"2024-04-25T09:16:05.676550+0000","last_undegraded":"2024-04-25T09:16:05.676550+0000","last_fullsized":"2024-04-25T09:16:05.676550+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.177293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372207+0000","last_change":"2024-04-25T09:16:03.379906+0000","last_active":"2024-04-25T09:16:04.372207+0000","last_peered":"2024-04-25T09:16:04.372207+0000","last_clean":"2024-04-25T09:16:04.372207+0000","last_became_active":"2024-04-25T09:16:03.379612+0000","last_became_peered":"2024-04-25T09:16:03.379612+0000","last_unstale":"2024-04-25T09:16:04.372207+0000","last_undegraded":"2024-04-25T09:16:04.372207+0000","last_fullsized":"2024-04-25T09:16:04.372207+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:35.051152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676971+0000","last_change":"2024-04-25T09:16:03.383421+0000","last_active":"2024-04-25T09:16:05.676971+0000","last_peered":"2024-04-25T09:16:05.676971+0000","last_clean":"2024-04-25T09:16:05.676971+0000","last_became_active":"2024-04-25T09:16:03.383108+0000","last_became_peered":"2024-04-25T09:16:03.383108+0000","last_unstale":"2024-04-25T09:16:05.676971+0000","last_undegraded":"2024-04-25T09:16:05.676971+0000","last_fullsized":"2024-04-25T09:16:05.676971+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:28.815944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373205+0000","last_change":"2024-04-25T09:16:03.363745+0000","last_active":"2024-04-25T09:16:04.373205+0000","last_peered":"2024-04-25T09:16:04.373205+0000","last_clean":"2024-04-25T09:16:04.373205+0000","last_became_active":"2024-04-25T09:16:03.363333+0000","last_became_peered":"2024-04-25T09:16:03.363333+0000","last_unstale":"2024-04-25T09:16:04.373205+0000","last_undegraded":"2024-04-25T09:16:04.373205+0000","last_fullsized":"2024-04-25T09:16:04.373205+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.177293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676870+0000","last_change":"2024-04-25T09:16:03.385421+0000","last_active":"2024-04-25T09:16:05.676870+0000","last_peered":"2024-04-25T09:16:05.676870+0000","last_clean":"2024-04-25T09:16:05.676870+0000","last_became_active":"2024-04-25T09:16:03.384652+0000","last_became_peered":"2024-04-25T09:16:03.384652+0000","last_unstale":"2024-04-25T09:16:05.676870+0000","last_undegraded":"2024-04-25T09:16:05.676870+0000","last_fullsized":"2024-04-25T09:16:05.676870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:14.907583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371174+0000","last_change":"2024-04-25T09:16:03.383009+0000","last_active":"2024-04-25T09:16:04.371174+0000","last_peered":"2024-04-25T09:16:04.371174+0000","last_clean":"2024-04-25T09:16:04.371174+0000","last_became_active":"2024-04-25T09:16:03.382586+0000","last_became_peered":"2024-04-25T09:16:03.382586+0000","last_unstale":"2024-04-25T09:16:04.371174+0000","last_undegraded":"2024-04-25T09:16:04.371174+0000","last_fullsized":"2024-04-25T09:16:04.371174+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:29.636797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373187+0000","last_change":"2024-04-25T09:16:03.380578+0000","last_active":"2024-04-25T09:16:04.373187+0000","last_peered":"2024-04-25T09:16:04.373187+0000","last_clean":"2024-04-25T09:16:04.373187+0000","last_became_active":"2024-04-25T09:16:03.377387+0000","last_became_peered":"2024-04-25T09:16:03.377387+0000","last_unstale":"2024-04-25T09:16:04.373187+0000","last_undegraded":"2024-04-25T09:16:04.373187+0000","last_fullsized":"2024-04-25T09:16:04.373187+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:58.773494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.433552+0000","last_change":"2024-04-25T09:16:03.367372+0000","last_active":"2024-04-25T09:16:04.433552+0000","last_peered":"2024-04-25T09:16:04.433552+0000","last_clean":"2024-04-25T09:16:04.433552+0000","last_became_active":"2024-04-25T09:16:03.367081+0000","last_became_peered":"2024-04-25T09:16:03.367081+0000","last_unstale":"2024-04-25T09:16:04.433552+0000","last_undegraded":"2024-04-25T09:16:04.433552+0000","last_fullsized":"2024-04-25T09:16:04.433552+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:59.790968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676013+0000","last_change":"2024-04-25T09:16:03.386567+0000","last_active":"2024-04-25T09:16:05.676013+0000","last_peered":"2024-04-25T09:16:05.676013+0000","last_clean":"2024-04-25T09:16:05.676013+0000","last_became_active":"2024-04-25T09:16:03.386248+0000","last_became_peered":"2024-04-25T09:16:03.386248+0000","last_unstale":"2024-04-25T09:16:05.676013+0000","last_undegraded":"2024-04-25T09:16:05.676013+0000","last_fullsized":"2024-04-25T09:16:05.676013+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:31.706208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471941+0000","last_change":"2024-04-25T09:16:03.377936+0000","last_active":"2024-04-25T09:16:05.471941+0000","last_peered":"2024-04-25T09:16:05.471941+0000","last_clean":"2024-04-25T09:16:05.471941+0000","last_became_active":"2024-04-25T09:16:03.377376+0000","last_became_peered":"2024-04-25T09:16:03.377376+0000","last_unstale":"2024-04-25T09:16:05.471941+0000","last_undegraded":"2024-04-25T09:16:05.471941+0000","last_fullsized":"2024-04-25T09:16:05.471941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:58.773494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373694+0000","last_change":"2024-04-25T09:16:03.371411+0000","last_active":"2024-04-25T09:16:04.373694+0000","last_peered":"2024-04-25T09:16:04.373694+0000","last_clean":"2024-04-25T09:16:04.373694+0000","last_became_active":"2024-04-25T09:16:03.370758+0000","last_became_peered":"2024-04-25T09:16:03.370758+0000","last_unstale":"2024-04-25T09:16:04.373694+0000","last_undegraded":"2024-04-25T09:16:04.373694+0000","last_fullsized":"2024-04-25T09:16:04.373694+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:40.323678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678380+0000","last_change":"2024-04-25T09:16:03.388224+0000","last_active":"2024-04-25T09:16:05.678380+0000","last_peered":"2024-04-25T09:16:05.678380+0000","last_clean":"2024-04-25T09:16:05.678380+0000","last_became_active":"2024-04-25T09:16:03.386349+0000","last_became_peered":"2024-04-25T09:16:03.386349+0000","last_unstale":"2024-04-25T09:16:05.678380+0000","last_undegraded":"2024-04-25T09:16:05.678380+0000","last_fullsized":"2024-04-25T09:16:05.678380+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:28.870734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434481+0000","last_change":"2024-04-25T09:16:03.376804+0000","last_active":"2024-04-25T09:16:04.434481+0000","last_peered":"2024-04-25T09:16:04.434481+0000","last_clean":"2024-04-25T09:16:04.434481+0000","last_became_active":"2024-04-25T09:16:03.373715+0000","last_became_peered":"2024-04-25T09:16:03.373715+0000","last_unstale":"2024-04-25T09:16:04.434481+0000","last_undegraded":"2024-04-25T09:16:04.434481+0000","last_fullsized":"2024-04-25T09:16:04.434481+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:47.041323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372393+0000","last_change":"2024-04-25T09:16:03.373941+0000","last_active":"2024-04-25T09:16:04.372393+0000","last_peered":"2024-04-25T09:16:04.372393+0000","last_clean":"2024-04-25T09:16:04.372393+0000","last_became_active":"2024-04-25T09:16:03.373585+0000","last_became_peered":"2024-04-25T09:16:03.373585+0000","last_unstale":"2024-04-25T09:16:04.372393+0000","last_undegraded":"2024-04-25T09:16:04.372393+0000","last_fullsized":"2024-04-25T09:16:04.372393+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:33.662482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372165+0000","last_change":"2024-04-25T09:16:03.358626+0000","last_active":"2024-04-25T09:16:04.372165+0000","last_peered":"2024-04-25T09:16:04.372165+0000","last_clean":"2024-04-25T09:16:04.372165+0000","last_became_active":"2024-04-25T09:16:03.358023+0000","last_became_peered":"2024-04-25T09:16:03.358023+0000","last_unstale":"2024-04-25T09:16:04.372165+0000","last_undegraded":"2024-04-25T09:16:04.372165+0000","last_fullsized":"2024-04-25T09:16:04.372165+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:48.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.433728+0000","last_change":"2024-04-25T09:16:03.371066+0000","last_active":"2024-04-25T09:16:04.433728+0000","last_peered":"2024-04-25T09:16:04.433728+0000","last_clean":"2024-04-25T09:16:04.433728+0000","last_became_active":"2024-04-25T09:16:03.369834+0000","last_became_peered":"2024-04-25T09:16:03.369834+0000","last_unstale":"2024-04-25T09:16:04.433728+0000","last_undegraded":"2024-04-25T09:16:04.433728+0000","last_fullsized":"2024-04-25T09:16:04.433728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:10.957296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373665+0000","last_change":"2024-04-25T09:16:03.374575+0000","last_active":"2024-04-25T09:16:04.373665+0000","last_peered":"2024-04-25T09:16:04.373665+0000","last_clean":"2024-04-25T09:16:04.373665+0000","last_became_active":"2024-04-25T09:16:03.369845+0000","last_became_peered":"2024-04-25T09:16:03.369845+0000","last_unstale":"2024-04-25T09:16:04.373665+0000","last_undegraded":"2024-04-25T09:16:04.373665+0000","last_fullsized":"2024-04-25T09:16:04.373665+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.378589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677806+0000","last_change":"2024-04-25T09:16:03.377625+0000","last_active":"2024-04-25T09:16:05.677806+0000","last_peered":"2024-04-25T09:16:05.677806+0000","last_clean":"2024-04-25T09:16:05.677806+0000","last_became_active":"2024-04-25T09:16:03.377305+0000","last_became_peered":"2024-04-25T09:16:03.377305+0000","last_unstale":"2024-04-25T09:16:05.677806+0000","last_undegraded":"2024-04-25T09:16:05.677806+0000","last_fullsized":"2024-04-25T09:16:05.677806+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:14.558673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"136'4","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442333+0000","last_change":"2024-04-25T09:16:02.370695+0000","last_active":"2024-04-25T09:16:04.442333+0000","last_peered":"2024-04-25T09:16:04.442333+0000","last_clean":"2024-04-25T09:16:04.442333+0000","last_became_active":"2024-04-25T09:16:02.370360+0000","last_became_peered":"2024-04-25T09:16:02.370360+0000","last_unstale":"2024-04-25T09:16:04.442333+0000","last_undegraded":"2024-04-25T09:16:04.442333+0000","last_fullsized":"2024-04-25T09:16:04.442333+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:27.798812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372065+0000","last_change":"2024-04-25T09:16:03.383924+0000","last_active":"2024-04-25T09:16:04.372065+0000","last_peered":"2024-04-25T09:16:04.372065+0000","last_clean":"2024-04-25T09:16:04.372065+0000","last_became_active":"2024-04-25T09:16:03.383596+0000","last_became_peered":"2024-04-25T09:16:03.383596+0000","last_unstale":"2024-04-25T09:16:04.372065+0000","last_undegraded":"2024-04-25T09:16:04.372065+0000","last_fullsized":"2024-04-25T09:16:04.372065+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:06.549852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372796+0000","last_change":"2024-04-25T09:16:02.376402+0000","last_active":"2024-04-25T09:16:04.372796+0000","last_peered":"2024-04-25T09:16:04.372796+0000","last_clean":"2024-04-25T09:16:04.372796+0000","last_became_active":"2024-04-25T09:16:02.375831+0000","last_became_peered":"2024-04-25T09:16:02.375831+0000","last_unstale":"2024-04-25T09:16:04.372796+0000","last_undegraded":"2024-04-25T09:16:04.372796+0000","last_fullsized":"2024-04-25T09:16:04.372796+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:36.774027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371730+0000","last_change":"2024-04-25T09:16:03.377499+0000","last_active":"2024-04-25T09:16:04.371730+0000","last_peered":"2024-04-25T09:16:04.371730+0000","last_clean":"2024-04-25T09:16:04.371730+0000","last_became_active":"2024-04-25T09:16:03.377202+0000","last_became_peered":"2024-04-25T09:16:03.377202+0000","last_unstale":"2024-04-25T09:16:04.371730+0000","last_undegraded":"2024-04-25T09:16:04.371730+0000","last_fullsized":"2024-04-25T09:16:04.371730+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:58.844284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471956+0000","last_change":"2024-04-25T09:16:02.372552+0000","last_active":"2024-04-25T09:16:05.471956+0000","last_peered":"2024-04-25T09:16:05.471956+0000","last_clean":"2024-04-25T09:16:05.471956+0000","last_became_active":"2024-04-25T09:16:02.372243+0000","last_became_peered":"2024-04-25T09:16:02.372243+0000","last_unstale":"2024-04-25T09:16:05.471956+0000","last_undegraded":"2024-04-25T09:16:05.471956+0000","last_fullsized":"2024-04-25T09:16:05.471956+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.031924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372428+0000","last_change":"2024-04-25T09:16:03.375343+0000","last_active":"2024-04-25T09:16:04.372428+0000","last_peered":"2024-04-25T09:16:04.372428+0000","last_clean":"2024-04-25T09:16:04.372428+0000","last_became_active":"2024-04-25T09:16:03.374986+0000","last_became_peered":"2024-04-25T09:16:03.374986+0000","last_unstale":"2024-04-25T09:16:04.372428+0000","last_undegraded":"2024-04-25T09:16:04.372428+0000","last_fullsized":"2024-04-25T09:16:04.372428+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.177293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373373+0000","last_change":"2024-04-25T09:16:02.372624+0000","last_active":"2024-04-25T09:16:04.373373+0000","last_peered":"2024-04-25T09:16:04.373373+0000","last_clean":"2024-04-25T09:16:04.373373+0000","last_became_active":"2024-04-25T09:16:02.372309+0000","last_became_peered":"2024-04-25T09:16:02.372309+0000","last_unstale":"2024-04-25T09:16:04.373373+0000","last_undegraded":"2024-04-25T09:16:04.373373+0000","last_fullsized":"2024-04-25T09:16:04.373373+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.590861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676615+0000","last_change":"2024-04-25T09:16:03.387453+0000","last_active":"2024-04-25T09:16:05.676615+0000","last_peered":"2024-04-25T09:16:05.676615+0000","last_clean":"2024-04-25T09:16:05.676615+0000","last_became_active":"2024-04-25T09:16:03.387187+0000","last_became_peered":"2024-04-25T09:16:03.387187+0000","last_unstale":"2024-04-25T09:16:05.676615+0000","last_undegraded":"2024-04-25T09:16:05.676615+0000","last_fullsized":"2024-04-25T09:16:05.676615+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:28.089832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372541+0000","last_change":"2024-04-25T09:16:02.367514+0000","last_active":"2024-04-25T09:16:04.372541+0000","last_peered":"2024-04-25T09:16:04.372541+0000","last_clean":"2024-04-25T09:16:04.372541+0000","last_became_active":"2024-04-25T09:16:02.367197+0000","last_became_peered":"2024-04-25T09:16:02.367197+0000","last_unstale":"2024-04-25T09:16:04.372541+0000","last_undegraded":"2024-04-25T09:16:04.372541+0000","last_fullsized":"2024-04-25T09:16:04.372541+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:27.798812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371668+0000","last_change":"2024-04-25T09:16:03.380649+0000","last_active":"2024-04-25T09:16:04.371668+0000","last_peered":"2024-04-25T09:16:04.371668+0000","last_clean":"2024-04-25T09:16:04.371668+0000","last_became_active":"2024-04-25T09:16:03.376513+0000","last_became_peered":"2024-04-25T09:16:03.376513+0000","last_unstale":"2024-04-25T09:16:04.371668+0000","last_undegraded":"2024-04-25T09:16:04.371668+0000","last_fullsized":"2024-04-25T09:16:04.371668+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:58.773494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676926+0000","last_change":"2024-04-25T09:16:02.368110+0000","last_active":"2024-04-25T09:16:05.676926+0000","last_peered":"2024-04-25T09:16:05.676926+0000","last_clean":"2024-04-25T09:16:05.676926+0000","last_became_active":"2024-04-25T09:16:02.367791+0000","last_became_peered":"2024-04-25T09:16:02.367791+0000","last_unstale":"2024-04-25T09:16:05.676926+0000","last_undegraded":"2024-04-25T09:16:05.676926+0000","last_fullsized":"2024-04-25T09:16:05.676926+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:09.940164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.370099+0000","last_change":"2024-04-25T09:16:03.368693+0000","last_active":"2024-04-25T09:16:04.370099+0000","last_peered":"2024-04-25T09:16:04.370099+0000","last_clean":"2024-04-25T09:16:04.370099+0000","last_became_active":"2024-04-25T09:16:03.366083+0000","last_became_peered":"2024-04-25T09:16:03.366083+0000","last_unstale":"2024-04-25T09:16:04.370099+0000","last_undegraded":"2024-04-25T09:16:04.370099+0000","last_fullsized":"2024-04-25T09:16:04.370099+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:43.348298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.370899+0000","last_change":"2024-04-25T09:16:02.367478+0000","last_active":"2024-04-25T09:16:04.370899+0000","last_peered":"2024-04-25T09:16:04.370899+0000","last_clean":"2024-04-25T09:16:04.370899+0000","last_became_active":"2024-04-25T09:16:02.367211+0000","last_became_peered":"2024-04-25T09:16:02.367211+0000","last_unstale":"2024-04-25T09:16:04.370899+0000","last_undegraded":"2024-04-25T09:16:04.370899+0000","last_fullsized":"2024-04-25T09:16:04.370899+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:44.132361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371685+0000","last_change":"2024-04-25T09:16:03.390386+0000","last_active":"2024-04-25T09:16:04.371685+0000","last_peered":"2024-04-25T09:16:04.371685+0000","last_clean":"2024-04-25T09:16:04.371685+0000","last_became_active":"2024-04-25T09:16:03.390010+0000","last_became_peered":"2024-04-25T09:16:03.390010+0000","last_unstale":"2024-04-25T09:16:04.371685+0000","last_undegraded":"2024-04-25T09:16:04.371685+0000","last_fullsized":"2024-04-25T09:16:04.371685+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:29.636797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371789+0000","last_change":"2024-04-25T09:16:02.373690+0000","last_active":"2024-04-25T09:16:04.371789+0000","last_peered":"2024-04-25T09:16:04.371789+0000","last_clean":"2024-04-25T09:16:04.371789+0000","last_became_active":"2024-04-25T09:16:02.373341+0000","last_became_peered":"2024-04-25T09:16:02.373341+0000","last_unstale":"2024-04-25T09:16:04.371789+0000","last_undegraded":"2024-04-25T09:16:04.371789+0000","last_fullsized":"2024-04-25T09:16:04.371789+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:53.927207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373840+0000","last_change":"2024-04-25T09:16:03.381459+0000","last_active":"2024-04-25T09:16:04.373840+0000","last_peered":"2024-04-25T09:16:04.373840+0000","last_clean":"2024-04-25T09:16:04.373840+0000","last_became_active":"2024-04-25T09:16:03.381133+0000","last_became_peered":"2024-04-25T09:16:03.381133+0000","last_unstale":"2024-04-25T09:16:04.373840+0000","last_undegraded":"2024-04-25T09:16:04.373840+0000","last_fullsized":"2024-04-25T09:16:04.373840+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:28.870734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373460+0000","last_change":"2024-04-25T09:16:02.371172+0000","last_active":"2024-04-25T09:16:04.373460+0000","last_peered":"2024-04-25T09:16:04.373460+0000","last_clean":"2024-04-25T09:16:04.373460+0000","last_became_active":"2024-04-25T09:16:02.370533+0000","last_became_peered":"2024-04-25T09:16:02.370533+0000","last_unstale":"2024-04-25T09:16:04.373460+0000","last_undegraded":"2024-04-25T09:16:04.373460+0000","last_fullsized":"2024-04-25T09:16:04.373460+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:27.072701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442202+0000","last_change":"2024-04-25T09:16:03.386920+0000","last_active":"2024-04-25T09:16:04.442202+0000","last_peered":"2024-04-25T09:16:04.442202+0000","last_clean":"2024-04-25T09:16:04.442202+0000","last_became_active":"2024-04-25T09:16:03.386542+0000","last_became_peered":"2024-04-25T09:16:03.386542+0000","last_unstale":"2024-04-25T09:16:04.442202+0000","last_undegraded":"2024-04-25T09:16:04.442202+0000","last_fullsized":"2024-04-25T09:16:04.442202+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:31.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677278+0000","last_change":"2024-04-25T09:16:02.370687+0000","last_active":"2024-04-25T09:16:05.677278+0000","last_peered":"2024-04-25T09:16:05.677278+0000","last_clean":"2024-04-25T09:16:05.677278+0000","last_became_active":"2024-04-25T09:16:02.369555+0000","last_became_peered":"2024-04-25T09:16:02.369555+0000","last_unstale":"2024-04-25T09:16:05.677278+0000","last_undegraded":"2024-04-25T09:16:05.677278+0000","last_fullsized":"2024-04-25T09:16:05.677278+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:46.409277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373292+0000","last_change":"2024-04-25T09:16:03.369481+0000","last_active":"2024-04-25T09:16:04.373292+0000","last_peered":"2024-04-25T09:16:04.373292+0000","last_clean":"2024-04-25T09:16:04.373292+0000","last_became_active":"2024-04-25T09:16:03.369204+0000","last_became_peered":"2024-04-25T09:16:03.369204+0000","last_unstale":"2024-04-25T09:16:04.373292+0000","last_undegraded":"2024-04-25T09:16:04.373292+0000","last_fullsized":"2024-04-25T09:16:04.373292+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:54.966474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"136'3","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442541+0000","last_change":"2024-04-25T09:16:02.368839+0000","last_active":"2024-04-25T09:16:04.442541+0000","last_peered":"2024-04-25T09:16:04.442541+0000","last_clean":"2024-04-25T09:16:04.442541+0000","last_became_active":"2024-04-25T09:16:02.368460+0000","last_became_peered":"2024-04-25T09:16:02.368460+0000","last_unstale":"2024-04-25T09:16:04.442541+0000","last_undegraded":"2024-04-25T09:16:04.442541+0000","last_fullsized":"2024-04-25T09:16:04.442541+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:53.927207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434521+0000","last_change":"2024-04-25T09:16:03.372581+0000","last_active":"2024-04-25T09:16:04.434521+0000","last_peered":"2024-04-25T09:16:04.434521+0000","last_clean":"2024-04-25T09:16:04.434521+0000","last_became_active":"2024-04-25T09:16:03.372300+0000","last_became_peered":"2024-04-25T09:16:03.372300+0000","last_unstale":"2024-04-25T09:16:04.434521+0000","last_undegraded":"2024-04-25T09:16:04.434521+0000","last_fullsized":"2024-04-25T09:16:04.434521+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:48.613306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"136'6","reported_seq":20,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434473+0000","last_change":"2024-04-25T09:16:02.361417+0000","last_active":"2024-04-25T09:16:04.434473+0000","last_peered":"2024-04-25T09:16:04.434473+0000","last_clean":"2024-04-25T09:16:04.434473+0000","last_became_active":"2024-04-25T09:16:02.361131+0000","last_became_peered":"2024-04-25T09:16:02.361131+0000","last_unstale":"2024-04-25T09:16:04.434473+0000","last_undegraded":"2024-04-25T09:16:04.434473+0000","last_fullsized":"2024-04-25T09:16:04.434473+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:51.176591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678024+0000","last_change":"2024-04-25T09:16:03.384935+0000","last_active":"2024-04-25T09:16:05.678024+0000","last_peered":"2024-04-25T09:16:05.678024+0000","last_clean":"2024-04-25T09:16:05.678024+0000","last_became_active":"2024-04-25T09:16:03.384627+0000","last_became_peered":"2024-04-25T09:16:03.384627+0000","last_unstale":"2024-04-25T09:16:05.678024+0000","last_undegraded":"2024-04-25T09:16:05.678024+0000","last_fullsized":"2024-04-25T09:16:05.678024+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:17.397100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677942+0000","last_change":"2024-04-25T09:16:02.370230+0000","last_active":"2024-04-25T09:16:05.677942+0000","last_peered":"2024-04-25T09:16:05.677942+0000","last_clean":"2024-04-25T09:16:05.677942+0000","last_became_active":"2024-04-25T09:16:02.368452+0000","last_became_peered":"2024-04-25T09:16:02.368452+0000","last_unstale":"2024-04-25T09:16:05.677942+0000","last_undegraded":"2024-04-25T09:16:05.677942+0000","last_fullsized":"2024-04-25T09:16:05.677942+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.031924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677670+0000","last_change":"2024-04-25T09:16:03.381645+0000","last_active":"2024-04-25T09:16:05.677670+0000","last_peered":"2024-04-25T09:16:05.677670+0000","last_clean":"2024-04-25T09:16:05.677670+0000","last_became_active":"2024-04-25T09:16:03.380994+0000","last_became_peered":"2024-04-25T09:16:03.380994+0000","last_unstale":"2024-04-25T09:16:05.677670+0000","last_undegraded":"2024-04-25T09:16:05.677670+0000","last_fullsized":"2024-04-25T09:16:05.677670+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:54.966474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471184+0000","last_change":"2024-04-25T09:16:02.372591+0000","last_active":"2024-04-25T09:16:05.471184+0000","last_peered":"2024-04-25T09:16:05.471184+0000","last_clean":"2024-04-25T09:16:05.471184+0000","last_became_active":"2024-04-25T09:16:02.372297+0000","last_became_peered":"2024-04-25T09:16:02.372297+0000","last_unstale":"2024-04-25T09:16:05.471184+0000","last_undegraded":"2024-04-25T09:16:05.471184+0000","last_fullsized":"2024-04-25T09:16:05.471184+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:07.199825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.370721+0000","last_change":"2024-04-25T09:16:03.366250+0000","last_active":"2024-04-25T09:16:04.370721+0000","last_peered":"2024-04-25T09:16:04.370721+0000","last_clean":"2024-04-25T09:16:04.370721+0000","last_became_active":"2024-04-25T09:16:03.365966+0000","last_became_peered":"2024-04-25T09:16:03.365966+0000","last_unstale":"2024-04-25T09:16:04.370721+0000","last_undegraded":"2024-04-25T09:16:04.370721+0000","last_fullsized":"2024-04-25T09:16:04.370721+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:02.923720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"136'8","reported_seq":22,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.370348+0000","last_change":"2024-04-25T09:16:02.365052+0000","last_active":"2024-04-25T09:16:04.370348+0000","last_peered":"2024-04-25T09:16:04.370348+0000","last_clean":"2024-04-25T09:16:04.370348+0000","last_became_active":"2024-04-25T09:16:02.362570+0000","last_became_peered":"2024-04-25T09:16:02.362570+0000","last_unstale":"2024-04-25T09:16:04.370348+0000","last_undegraded":"2024-04-25T09:16:04.370348+0000","last_fullsized":"2024-04-25T09:16:04.370348+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:53.927207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442459+0000","last_change":"2024-04-25T09:16:03.373592+0000","last_active":"2024-04-25T09:16:04.442459+0000","last_peered":"2024-04-25T09:16:04.442459+0000","last_clean":"2024-04-25T09:16:04.442459+0000","last_became_active":"2024-04-25T09:16:03.373301+0000","last_became_peered":"2024-04-25T09:16:03.373301+0000","last_unstale":"2024-04-25T09:16:04.442459+0000","last_undegraded":"2024-04-25T09:16:04.442459+0000","last_fullsized":"2024-04-25T09:16:04.442459+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:49.627786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373756+0000","last_change":"2024-04-25T09:16:02.371391+0000","last_active":"2024-04-25T09:16:04.373756+0000","last_peered":"2024-04-25T09:16:04.373756+0000","last_clean":"2024-04-25T09:16:04.373756+0000","last_became_active":"2024-04-25T09:16:02.370909+0000","last_became_peered":"2024-04-25T09:16:02.370909+0000","last_unstale":"2024-04-25T09:16:04.373756+0000","last_undegraded":"2024-04-25T09:16:04.373756+0000","last_fullsized":"2024-04-25T09:16:04.373756+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:27.798812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442576+0000","last_change":"2024-04-25T09:16:03.391112+0000","last_active":"2024-04-25T09:16:04.442576+0000","last_peered":"2024-04-25T09:16:04.442576+0000","last_clean":"2024-04-25T09:16:04.442576+0000","last_became_active":"2024-04-25T09:16:03.390509+0000","last_became_peered":"2024-04-25T09:16:03.390509+0000","last_unstale":"2024-04-25T09:16:04.442576+0000","last_undegraded":"2024-04-25T09:16:04.442576+0000","last_fullsized":"2024-04-25T09:16:04.442576+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:23.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471686+0000","last_change":"2024-04-25T09:16:02.369859+0000","last_active":"2024-04-25T09:16:05.471686+0000","last_peered":"2024-04-25T09:16:05.471686+0000","last_clean":"2024-04-25T09:16:05.471686+0000","last_became_active":"2024-04-25T09:16:02.369116+0000","last_became_peered":"2024-04-25T09:16:02.369116+0000","last_unstale":"2024-04-25T09:16:05.471686+0000","last_undegraded":"2024-04-25T09:16:05.471686+0000","last_fullsized":"2024-04-25T09:16:05.471686+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:22.498085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373533+0000","last_change":"2024-04-25T09:16:03.373497+0000","last_active":"2024-04-25T09:16:04.373533+0000","last_peered":"2024-04-25T09:16:04.373533+0000","last_clean":"2024-04-25T09:16:04.373533+0000","last_became_active":"2024-04-25T09:16:03.373120+0000","last_became_peered":"2024-04-25T09:16:03.373120+0000","last_unstale":"2024-04-25T09:16:04.373533+0000","last_undegraded":"2024-04-25T09:16:04.373533+0000","last_fullsized":"2024-04-25T09:16:04.373533+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:17.397100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.374358+0000","last_change":"2024-04-25T09:16:02.373019+0000","last_active":"2024-04-25T09:16:04.374358+0000","last_peered":"2024-04-25T09:16:04.374358+0000","last_clean":"2024-04-25T09:16:04.374358+0000","last_became_active":"2024-04-25T09:16:02.372726+0000","last_became_peered":"2024-04-25T09:16:02.372726+0000","last_unstale":"2024-04-25T09:16:04.374358+0000","last_undegraded":"2024-04-25T09:16:04.374358+0000","last_fullsized":"2024-04-25T09:16:04.374358+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:40.536945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373461+0000","last_change":"2024-04-25T09:16:03.390604+0000","last_active":"2024-04-25T09:16:04.373461+0000","last_peered":"2024-04-25T09:16:04.373461+0000","last_clean":"2024-04-25T09:16:04.373461+0000","last_became_active":"2024-04-25T09:16:03.390308+0000","last_became_peered":"2024-04-25T09:16:03.390308+0000","last_unstale":"2024-04-25T09:16:04.373461+0000","last_undegraded":"2024-04-25T09:16:04.373461+0000","last_fullsized":"2024-04-25T09:16:04.373461+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:14.558673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.441921+0000","last_change":"2024-04-25T09:16:02.366373+0000","last_active":"2024-04-25T09:16:04.441921+0000","last_peered":"2024-04-25T09:16:04.441921+0000","last_clean":"2024-04-25T09:16:04.441921+0000","last_became_active":"2024-04-25T09:16:02.366008+0000","last_became_peered":"2024-04-25T09:16:02.366008+0000","last_unstale":"2024-04-25T09:16:04.441921+0000","last_undegraded":"2024-04-25T09:16:04.441921+0000","last_fullsized":"2024-04-25T09:16:04.441921+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.590861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372856+0000","last_change":"2024-04-25T09:16:03.389019+0000","last_active":"2024-04-25T09:16:04.372856+0000","last_peered":"2024-04-25T09:16:04.372856+0000","last_clean":"2024-04-25T09:16:04.372856+0000","last_became_active":"2024-04-25T09:16:03.388747+0000","last_became_peered":"2024-04-25T09:16:03.388747+0000","last_unstale":"2024-04-25T09:16:04.372856+0000","last_undegraded":"2024-04-25T09:16:04.372856+0000","last_fullsized":"2024-04-25T09:16:04.372856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:41.625765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372505+0000","last_change":"2024-04-25T09:16:02.374156+0000","last_active":"2024-04-25T09:16:04.372505+0000","last_peered":"2024-04-25T09:16:04.372505+0000","last_clean":"2024-04-25T09:16:04.372505+0000","last_became_active":"2024-04-25T09:16:02.373871+0000","last_became_peered":"2024-04-25T09:16:02.373871+0000","last_unstale":"2024-04-25T09:16:04.372505+0000","last_undegraded":"2024-04-25T09:16:04.372505+0000","last_fullsized":"2024-04-25T09:16:04.372505+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:42.526286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434076+0000","last_change":"2024-04-25T09:16:03.359694+0000","last_active":"2024-04-25T09:16:04.434076+0000","last_peered":"2024-04-25T09:16:04.434076+0000","last_clean":"2024-04-25T09:16:04.434076+0000","last_became_active":"2024-04-25T09:16:03.359395+0000","last_became_peered":"2024-04-25T09:16:03.359395+0000","last_unstale":"2024-04-25T09:16:04.434076+0000","last_undegraded":"2024-04-25T09:16:04.434076+0000","last_fullsized":"2024-04-25T09:16:04.434076+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.620761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677203+0000","last_change":"2024-04-25T09:16:02.375497+0000","last_active":"2024-04-25T09:16:05.677203+0000","last_peered":"2024-04-25T09:16:05.677203+0000","last_clean":"2024-04-25T09:16:05.677203+0000","last_became_active":"2024-04-25T09:16:02.375158+0000","last_became_peered":"2024-04-25T09:16:02.375158+0000","last_unstale":"2024-04-25T09:16:05.677203+0000","last_undegraded":"2024-04-25T09:16:05.677203+0000","last_fullsized":"2024-04-25T09:16:05.677203+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:14.603630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678340+0000","last_change":"2024-04-25T09:16:03.383557+0000","last_active":"2024-04-25T09:16:05.678340+0000","last_peered":"2024-04-25T09:16:05.678340+0000","last_clean":"2024-04-25T09:16:05.678340+0000","last_became_active":"2024-04-25T09:16:03.381369+0000","last_became_peered":"2024-04-25T09:16:03.381369+0000","last_unstale":"2024-04-25T09:16:05.678340+0000","last_undegraded":"2024-04-25T09:16:05.678340+0000","last_fullsized":"2024-04-25T09:16:05.678340+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:40.323678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678031+0000","last_change":"2024-04-25T09:16:02.376569+0000","last_active":"2024-04-25T09:16:05.678031+0000","last_peered":"2024-04-25T09:16:05.678031+0000","last_clean":"2024-04-25T09:16:05.678031+0000","last_became_active":"2024-04-25T09:16:02.376238+0000","last_became_peered":"2024-04-25T09:16:02.376238+0000","last_unstale":"2024-04-25T09:16:05.678031+0000","last_undegraded":"2024-04-25T09:16:05.678031+0000","last_fullsized":"2024-04-25T09:16:05.678031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:48.610654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371129+0000","last_change":"2024-04-25T09:16:03.380100+0000","last_active":"2024-04-25T09:16:04.371129+0000","last_peered":"2024-04-25T09:16:04.371129+0000","last_clean":"2024-04-25T09:16:04.371129+0000","last_became_active":"2024-04-25T09:16:03.379809+0000","last_became_peered":"2024-04-25T09:16:03.379809+0000","last_unstale":"2024-04-25T09:16:04.371129+0000","last_undegraded":"2024-04-25T09:16:04.371129+0000","last_fullsized":"2024-04-25T09:16:04.371129+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:08.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471083+0000","last_change":"2024-04-25T09:16:02.376458+0000","last_active":"2024-04-25T09:16:05.471083+0000","last_peered":"2024-04-25T09:16:05.471083+0000","last_clean":"2024-04-25T09:16:05.471083+0000","last_became_active":"2024-04-25T09:16:02.376129+0000","last_became_peered":"2024-04-25T09:16:02.376129+0000","last_unstale":"2024-04-25T09:16:05.471083+0000","last_undegraded":"2024-04-25T09:16:05.471083+0000","last_fullsized":"2024-04-25T09:16:05.471083+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:30.161052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434605+0000","last_change":"2024-04-25T09:16:03.373922+0000","last_active":"2024-04-25T09:16:04.434605+0000","last_peered":"2024-04-25T09:16:04.434605+0000","last_clean":"2024-04-25T09:16:04.434605+0000","last_became_active":"2024-04-25T09:16:03.373575+0000","last_became_peered":"2024-04-25T09:16:03.373575+0000","last_unstale":"2024-04-25T09:16:04.434605+0000","last_undegraded":"2024-04-25T09:16:04.434605+0000","last_fullsized":"2024-04-25T09:16:04.434605+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:47.426409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677495+0000","last_change":"2024-04-25T09:16:02.375478+0000","last_active":"2024-04-25T09:16:05.677495+0000","last_peered":"2024-04-25T09:16:05.677495+0000","last_clean":"2024-04-25T09:16:05.677495+0000","last_became_active":"2024-04-25T09:16:02.375165+0000","last_became_peered":"2024-04-25T09:16:02.375165+0000","last_unstale":"2024-04-25T09:16:05.677495+0000","last_undegraded":"2024-04-25T09:16:05.677495+0000","last_fullsized":"2024-04-25T09:16:05.677495+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:46.024192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373817+0000","last_change":"2024-04-25T09:16:03.374691+0000","last_active":"2024-04-25T09:16:04.373817+0000","last_peered":"2024-04-25T09:16:04.373817+0000","last_clean":"2024-04-25T09:16:04.373817+0000","last_became_active":"2024-04-25T09:16:03.374250+0000","last_became_peered":"2024-04-25T09:16:03.374250+0000","last_unstale":"2024-04-25T09:16:04.373817+0000","last_undegraded":"2024-04-25T09:16:04.373817+0000","last_fullsized":"2024-04-25T09:16:04.373817+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:29.979855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"137'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.363708+0000","last_change":"2024-04-25T09:16:02.372805+0000","last_active":"2024-04-25T09:16:05.363708+0000","last_peered":"2024-04-25T09:16:05.363708+0000","last_clean":"2024-04-25T09:16:05.363708+0000","last_became_active":"2024-04-25T09:16:02.370674+0000","last_became_peered":"2024-04-25T09:16:02.370674+0000","last_unstale":"2024-04-25T09:16:05.363708+0000","last_undegraded":"2024-04-25T09:16:05.363708+0000","last_fullsized":"2024-04-25T09:16:05.363708+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:13.890451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678384+0000","last_change":"2024-04-25T09:16:03.379934+0000","last_active":"2024-04-25T09:16:05.678384+0000","last_peered":"2024-04-25T09:16:05.678384+0000","last_clean":"2024-04-25T09:16:05.678384+0000","last_became_active":"2024-04-25T09:16:03.379616+0000","last_became_peered":"2024-04-25T09:16:03.379616+0000","last_unstale":"2024-04-25T09:16:05.678384+0000","last_undegraded":"2024-04-25T09:16:05.678384+0000","last_fullsized":"2024-04-25T09:16:05.678384+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:29.979855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677586+0000","last_change":"2024-04-25T09:16:02.363332+0000","last_active":"2024-04-25T09:16:05.677586+0000","last_peered":"2024-04-25T09:16:05.677586+0000","last_clean":"2024-04-25T09:16:05.677586+0000","last_became_active":"2024-04-25T09:16:02.362333+0000","last_became_peered":"2024-04-25T09:16:02.362333+0000","last_unstale":"2024-04-25T09:16:05.677586+0000","last_undegraded":"2024-04-25T09:16:05.677586+0000","last_fullsized":"2024-04-25T09:16:05.677586+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:47.596175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.374172+0000","last_change":"2024-04-25T09:16:03.367083+0000","last_active":"2024-04-25T09:16:04.374172+0000","last_peered":"2024-04-25T09:16:04.374172+0000","last_clean":"2024-04-25T09:16:04.374172+0000","last_became_active":"2024-04-25T09:16:03.366691+0000","last_became_peered":"2024-04-25T09:16:03.366691+0000","last_unstale":"2024-04-25T09:16:04.374172+0000","last_undegraded":"2024-04-25T09:16:04.374172+0000","last_fullsized":"2024-04-25T09:16:04.374172+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:34.982268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678388+0000","last_change":"2024-04-25T09:16:02.370620+0000","last_active":"2024-04-25T09:16:05.678388+0000","last_peered":"2024-04-25T09:16:05.678388+0000","last_clean":"2024-04-25T09:16:05.678388+0000","last_became_active":"2024-04-25T09:16:02.370317+0000","last_became_peered":"2024-04-25T09:16:02.370317+0000","last_unstale":"2024-04-25T09:16:05.678388+0000","last_undegraded":"2024-04-25T09:16:05.678388+0000","last_fullsized":"2024-04-25T09:16:05.678388+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:30.161052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371231+0000","last_change":"2024-04-25T09:16:03.387671+0000","last_active":"2024-04-25T09:16:04.371231+0000","last_peered":"2024-04-25T09:16:04.371231+0000","last_clean":"2024-04-25T09:16:04.371231+0000","last_became_active":"2024-04-25T09:16:03.386956+0000","last_became_peered":"2024-04-25T09:16:03.386956+0000","last_unstale":"2024-04-25T09:16:04.371231+0000","last_undegraded":"2024-04-25T09:16:04.371231+0000","last_fullsized":"2024-04-25T09:16:04.371231+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:48.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372825+0000","last_change":"2024-04-25T09:16:02.364561+0000","last_active":"2024-04-25T09:16:04.372825+0000","last_peered":"2024-04-25T09:16:04.372825+0000","last_clean":"2024-04-25T09:16:04.372825+0000","last_became_active":"2024-04-25T09:16:02.364203+0000","last_became_peered":"2024-04-25T09:16:02.364203+0000","last_unstale":"2024-04-25T09:16:04.372825+0000","last_undegraded":"2024-04-25T09:16:04.372825+0000","last_fullsized":"2024-04-25T09:16:04.372825+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:06.451506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372079+0000","last_change":"2024-04-25T09:16:03.372988+0000","last_active":"2024-04-25T09:16:04.372079+0000","last_peered":"2024-04-25T09:16:04.372079+0000","last_clean":"2024-04-25T09:16:04.372079+0000","last_became_active":"2024-04-25T09:16:03.372175+0000","last_became_peered":"2024-04-25T09:16:03.372175+0000","last_unstale":"2024-04-25T09:16:04.372079+0000","last_undegraded":"2024-04-25T09:16:04.372079+0000","last_fullsized":"2024-04-25T09:16:04.372079+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:41.625765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371712+0000","last_change":"2024-04-25T09:16:02.361856+0000","last_active":"2024-04-25T09:16:04.371712+0000","last_peered":"2024-04-25T09:16:04.371712+0000","last_clean":"2024-04-25T09:16:04.371712+0000","last_became_active":"2024-04-25T09:16:02.361136+0000","last_became_peered":"2024-04-25T09:16:02.361136+0000","last_unstale":"2024-04-25T09:16:04.371712+0000","last_undegraded":"2024-04-25T09:16:04.371712+0000","last_fullsized":"2024-04-25T09:16:04.371712+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:13.890451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676247+0000","last_change":"2024-04-25T09:16:03.390013+0000","last_active":"2024-04-25T09:16:05.676247+0000","last_peered":"2024-04-25T09:16:05.676247+0000","last_clean":"2024-04-25T09:16:05.676247+0000","last_became_active":"2024-04-25T09:16:03.388991+0000","last_became_peered":"2024-04-25T09:16:03.388991+0000","last_unstale":"2024-04-25T09:16:05.676247+0000","last_undegraded":"2024-04-25T09:16:05.676247+0000","last_fullsized":"2024-04-25T09:16:05.676247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.937573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.470846+0000","last_change":"2024-04-25T09:16:02.372423+0000","last_active":"2024-04-25T09:16:05.470846+0000","last_peered":"2024-04-25T09:16:05.470846+0000","last_clean":"2024-04-25T09:16:05.470846+0000","last_became_active":"2024-04-25T09:16:02.372061+0000","last_became_peered":"2024-04-25T09:16:02.372061+0000","last_unstale":"2024-04-25T09:16:05.470846+0000","last_undegraded":"2024-04-25T09:16:05.470846+0000","last_fullsized":"2024-04-25T09:16:05.470846+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:02.889910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676658+0000","last_change":"2024-04-25T09:16:03.381637+0000","last_active":"2024-04-25T09:16:05.676658+0000","last_peered":"2024-04-25T09:16:05.676658+0000","last_clean":"2024-04-25T09:16:05.676658+0000","last_became_active":"2024-04-25T09:16:03.380922+0000","last_became_peered":"2024-04-25T09:16:03.380922+0000","last_unstale":"2024-04-25T09:16:05.676658+0000","last_undegraded":"2024-04-25T09:16:05.676658+0000","last_fullsized":"2024-04-25T09:16:05.676658+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.784334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678348+0000","last_change":"2024-04-25T09:16:02.370723+0000","last_active":"2024-04-25T09:16:05.678348+0000","last_peered":"2024-04-25T09:16:05.678348+0000","last_clean":"2024-04-25T09:16:05.678348+0000","last_became_active":"2024-04-25T09:16:02.370451+0000","last_became_peered":"2024-04-25T09:16:02.370451+0000","last_unstale":"2024-04-25T09:16:05.678348+0000","last_undegraded":"2024-04-25T09:16:05.678348+0000","last_fullsized":"2024-04-25T09:16:05.678348+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:33.698698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371247+0000","last_change":"2024-04-25T09:16:03.386353+0000","last_active":"2024-04-25T09:16:04.371247+0000","last_peered":"2024-04-25T09:16:04.371247+0000","last_clean":"2024-04-25T09:16:04.371247+0000","last_became_active":"2024-04-25T09:16:03.385876+0000","last_became_peered":"2024-04-25T09:16:03.385876+0000","last_unstale":"2024-04-25T09:16:04.371247+0000","last_undegraded":"2024-04-25T09:16:04.371247+0000","last_fullsized":"2024-04-25T09:16:04.371247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:05.481907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372347+0000","last_change":"2024-04-25T09:16:02.360788+0000","last_active":"2024-04-25T09:16:04.372347+0000","last_peered":"2024-04-25T09:16:04.372347+0000","last_clean":"2024-04-25T09:16:04.372347+0000","last_became_active":"2024-04-25T09:16:02.360329+0000","last_became_peered":"2024-04-25T09:16:02.360329+0000","last_unstale":"2024-04-25T09:16:04.372347+0000","last_undegraded":"2024-04-25T09:16:04.372347+0000","last_fullsized":"2024-04-25T09:16:04.372347+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.590861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":385,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.433112+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:16:04.433112+0000","last_peered":"2024-04-25T09:16:04.433112+0000","last_clean":"2024-04-25T09:16:04.433112+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:16:04.433112+0000","last_undegraded":"2024-04-25T09:16:04.433112+0000","last_fullsized":"2024-04-25T09:16:04.433112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50892,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320948,"statfs":{"total":96636764160,"available":96584650752,"internally_reserved":0,"allocated":1699840,"data_stored":744697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1708032,"data_stored":750963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59812,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312028,"statfs":{"total":96636764160,"available":96575516672,"internally_reserved":0,"allocated":2314240,"data_stored":1342073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575008,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1708032,"data_stored":750963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1712128,"data_stored":751033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64792,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307048,"statfs":{"total":96636764160,"available":96570417152,"internally_reserved":0,"allocated":1716224,"data_stored":751635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55656,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316184,"statfs":{"total":96636764160,"available":96579772416,"internally_reserved":0,"allocated":2318336,"data_stored":1342965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50920,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320920,"statfs":{"total":96636764160,"available":96584622080,"internally_reserved":0,"allocated":1728512,"data_stored":752631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:16:07.817 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:16:08.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.084+0000 7f1bf5600640 1 Processor -- start 2024-04-25T09:16:08.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.084+0000 7f1bf5600640 1 -- start start 2024-04-25T09:16:08.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.085+0000 7f1bf5600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1bf00b56f0 0x7f1bf0160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.085+0000 7f1bf5600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1bf00b6a90 0x7f1bf0161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.085+0000 7f1bf5600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b27a0 0x7f1bf0161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.085+0000 7f1bf5600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf0161780 con 0x7f1bf00b6a90 2024-04-25T09:16:08.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.085+0000 7f1bf5600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf00cedc0 con 0x7f1bf00b56f0 2024-04-25T09:16:08.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.085+0000 7f1bf5600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00d3110 con 0x7f1bf00b27a0 2024-04-25T09:16:08.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bef400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1bf00b6a90 0x7f1bf0161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bef400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1bf00b6a90 0x7f1bf0161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35864/0 (socket says 172.21.15.62:35864) 2024-04-25T09:16:08.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bef400640 1 -- 172.21.15.62:0/966361954 learned_addr learned my addr 172.21.15.62:0/966361954 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:08.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1befe00640 1 --2- 172.21.15.62:0/966361954 >> v2:172.21.15.121:3300/0 conn(0x7f1bf00b56f0 0x7f1bf0160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bf4c00640 1 --2- 172.21.15.62:0/966361954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b27a0 0x7f1bf0161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bef400640 1 -- 172.21.15.62:0/966361954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b27a0 msgr2=0x7f1bf0161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bef400640 1 --2- 172.21.15.62:0/966361954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b27a0 0x7f1bf0161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bef400640 1 -- 172.21.15.62:0/966361954 >> v2:172.21.15.121:3300/0 conn(0x7f1bf00b56f0 msgr2=0x7f1bf0160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bef400640 1 --2- 172.21.15.62:0/966361954 >> v2:172.21.15.121:3300/0 conn(0x7f1bf00b56f0 0x7f1bf0160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.086+0000 7f1bef400640 1 -- 172.21.15.62:0/966361954 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf00d33f0 con 0x7f1bf00b6a90 2024-04-25T09:16:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.087+0000 7f1bef400640 1 --2- 172.21.15.62:0/966361954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b6a90 0x7f1bf0161240 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1bf008c310 tx=0x7f1bd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9043b8c35b21d289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.088+0000 7f1beea00640 1 -- 172.21.15.62:0/966361954 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8037070 con 0x7f1bf00b6a90 2024-04-25T09:16:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.088+0000 7f1beea00640 1 -- 172.21.15.62:0/966361954 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd8032440 con 0x7f1bf00b6a90 2024-04-25T09:16:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.088+0000 7f1beea00640 1 -- 172.21.15.62:0/966361954 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8032810 con 0x7f1bf00b6a90 2024-04-25T09:16:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.088+0000 7f1bf5600640 1 -- 172.21.15.62:0/966361954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b6a90 msgr2=0x7f1bf0161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.088+0000 7f1bf5600640 1 --2- 172.21.15.62:0/966361954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b6a90 0x7f1bf0161240 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1bf008c310 tx=0x7f1bd802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.088+0000 7f1bf5600640 1 -- 172.21.15.62:0/966361954 shutdown_connections 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.088+0000 7f1bf5600640 1 --2- 172.21.15.62:0/966361954 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b27a0 0x7f1bf0161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.088+0000 7f1bf5600640 1 --2- 172.21.15.62:0/966361954 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b6a90 0x7f1bf0161240 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.089+0000 7f1bf5600640 1 --2- 172.21.15.62:0/966361954 >> v2:172.21.15.121:3300/0 conn(0x7f1bf00b56f0 0x7f1bf0160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.089+0000 7f1bf5600640 1 -- 172.21.15.62:0/966361954 >> 172.21.15.62:0/966361954 conn(0x7f1bf0066d10 msgr2=0x7f1bf0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.089+0000 7f1bf5600640 1 -- 172.21.15.62:0/966361954 shutdown_connections 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.089+0000 7f1bf5600640 1 -- 172.21.15.62:0/966361954 wait complete. 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.090+0000 7f1bf5600640 1 Processor -- start 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.091+0000 7f1bf5600640 1 -- start start 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.091+0000 7f1bf5600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b27a0 0x7f1bf0157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.091+0000 7f1bf5600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf00b56f0 0x7f1bf00cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.091+0000 7f1bf5600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b6a90 0x7f1bf01f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.091+0000 7f1bf5600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b2d60 con 0x7f1bf00b27a0 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.091+0000 7f1bf5600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00cea50 con 0x7f1bf00b56f0 2024-04-25T09:16:08.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.091+0000 7f1bf5600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf00b2ee0 con 0x7f1bf00b6a90 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b27a0 0x7f1bf0157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b27a0 0x7f1bf0157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35870/0 (socket says 172.21.15.62:35870) 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 -- 172.21.15.62:0/2786431623 learned_addr learned my addr 172.21.15.62:0/2786431623 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 -- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b6a90 msgr2=0x7f1bf01f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b6a90 0x7f1bf01f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 -- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf00b56f0 msgr2=0x7f1bf00cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf00b56f0 0x7f1bf00cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 -- 172.21.15.62:0/2786431623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf01584c0 con 0x7f1bf00b27a0 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.092+0000 7f1befe00640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b27a0 0x7f1bf0157e70 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1be0007f20 tx=0x7f1be000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.093+0000 7f1be7e00640 1 -- 172.21.15.62:0/2786431623 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0014620 con 0x7f1bf00b27a0 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.093+0000 7f1be7e00640 1 -- 172.21.15.62:0/2786431623 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be0019070 con 0x7f1bf00b27a0 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.093+0000 7f1be7e00640 1 -- 172.21.15.62:0/2786431623 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0014a00 con 0x7f1bf00b27a0 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.093+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf01587a0 con 0x7f1bf00b27a0 2024-04-25T09:16:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.093+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf0158d60 con 0x7f1bf00b27a0 2024-04-25T09:16:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.094+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf004ef20 con 0x7f1bf00b27a0 2024-04-25T09:16:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.095+0000 7f1be7e00640 1 -- 172.21.15.62:0/2786431623 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1be0014ba0 con 0x7f1bf00b27a0 2024-04-25T09:16:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.096+0000 7f1be7e00640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bcc06b4d0 0x7f1bcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.096+0000 7f1be7e00640 1 -- 172.21.15.62:0/2786431623 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1be008abf0 con 0x7f1bf00b27a0 2024-04-25T09:16:08.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.097+0000 7f1bef400640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bcc06b4d0 0x7f1bcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.098+0000 7f1bef400640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bcc06b4d0 0x7f1bcc06d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8034000 tx=0x7f1bd802c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:08.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.101+0000 7f1be7e00640 1 -- 172.21.15.62:0/2786431623 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1be00599d0 con 0x7f1bf00b27a0 2024-04-25T09:16:08.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.296+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bf00b9270 con 0x7f1bcc06b4d0 2024-04-25T09:16:08.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.315+0000 7f1be7e00640 1 -- 172.21.15.62:0/2786431623 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252482 (secure 0 0 0) 0x7f1bf00b9270 con 0x7f1bcc06b4d0 2024-04-25T09:16:08.306 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:08.307 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.321+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bcc06b4d0 msgr2=0x7f1bcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.321+0000 7f1bf5600640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bcc06b4d0 0x7f1bcc06d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8034000 tx=0x7f1bd802c000 comp rx=0 tx=0).stop 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.321+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b27a0 msgr2=0x7f1bf0157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.321+0000 7f1bf5600640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b27a0 0x7f1bf0157e70 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1be0007f20 tx=0x7f1be000e570 comp rx=0 tx=0).stop 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.321+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 shutdown_connections 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.322+0000 7f1bf5600640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1bcc06b4d0 0x7f1bcc06d990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.322+0000 7f1bf5600640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1bf00b6a90 0x7f1bf01f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.322+0000 7f1bf5600640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf00b56f0 0x7f1bf00cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.322+0000 7f1bf5600640 1 --2- 172.21.15.62:0/2786431623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1bf00b27a0 0x7f1bf0157e70 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.322+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 >> 172.21.15.62:0/2786431623 conn(0x7f1bf0066d10 msgr2=0x7f1bf0156550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.322+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 shutdown_connections 2024-04-25T09:16:08.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.322+0000 7f1bf5600640 1 -- 172.21.15.62:0/2786431623 wait complete. 2024-04-25T09:16:08.327 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-04-25T09:16:08.182292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":455572,"kb_used_data":14556,"kb_used_omap":12,"kb_used_meta":440883,"kb_avail":754519148,"statfs":{"total":773094113280,"available":772627607552,"internally_reserved":0,"allocated":14905344,"data_stored":7186960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":451464752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002130"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373567+0000","last_change":"2024-04-25T09:16:03.369068+0000","last_active":"2024-04-25T09:16:04.373567+0000","last_peered":"2024-04-25T09:16:04.373567+0000","last_clean":"2024-04-25T09:16:04.373567+0000","last_became_active":"2024-04-25T09:16:03.368734+0000","last_became_peered":"2024-04-25T09:16:03.368734+0000","last_unstale":"2024-04-25T09:16:04.373567+0000","last_undegraded":"2024-04-25T09:16:04.373567+0000","last_fullsized":"2024-04-25T09:16:04.373567+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:02.379597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373280+0000","last_change":"2024-04-25T09:16:03.387881+0000","last_active":"2024-04-25T09:16:04.373280+0000","last_peered":"2024-04-25T09:16:04.373280+0000","last_clean":"2024-04-25T09:16:04.373280+0000","last_became_active":"2024-04-25T09:16:03.387591+0000","last_became_peered":"2024-04-25T09:16:03.387591+0000","last_unstale":"2024-04-25T09:16:04.373280+0000","last_undegraded":"2024-04-25T09:16:04.373280+0000","last_fullsized":"2024-04-25T09:16:04.373280+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.784334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.374089+0000","last_change":"2024-04-25T09:16:03.369047+0000","last_active":"2024-04-25T09:16:04.374089+0000","last_peered":"2024-04-25T09:16:04.374089+0000","last_clean":"2024-04-25T09:16:04.374089+0000","last_became_active":"2024-04-25T09:16:03.368769+0000","last_became_peered":"2024-04-25T09:16:03.368769+0000","last_unstale":"2024-04-25T09:16:04.374089+0000","last_undegraded":"2024-04-25T09:16:04.374089+0000","last_fullsized":"2024-04-25T09:16:04.374089+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:36.471214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471092+0000","last_change":"2024-04-25T09:16:03.382598+0000","last_active":"2024-04-25T09:16:05.471092+0000","last_peered":"2024-04-25T09:16:05.471092+0000","last_clean":"2024-04-25T09:16:05.471092+0000","last_became_active":"2024-04-25T09:16:03.382298+0000","last_became_peered":"2024-04-25T09:16:03.382298+0000","last_unstale":"2024-04-25T09:16:05.471092+0000","last_undegraded":"2024-04-25T09:16:05.471092+0000","last_fullsized":"2024-04-25T09:16:05.471092+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:29.636797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442233+0000","last_change":"2024-04-25T09:16:03.372813+0000","last_active":"2024-04-25T09:16:04.442233+0000","last_peered":"2024-04-25T09:16:04.442233+0000","last_clean":"2024-04-25T09:16:04.442233+0000","last_became_active":"2024-04-25T09:16:03.372517+0000","last_became_peered":"2024-04-25T09:16:03.372517+0000","last_unstale":"2024-04-25T09:16:04.442233+0000","last_undegraded":"2024-04-25T09:16:04.442233+0000","last_fullsized":"2024-04-25T09:16:04.442233+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:26.049055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371552+0000","last_change":"2024-04-25T09:16:03.369846+0000","last_active":"2024-04-25T09:16:04.371552+0000","last_peered":"2024-04-25T09:16:04.371552+0000","last_clean":"2024-04-25T09:16:04.371552+0000","last_became_active":"2024-04-25T09:16:03.369554+0000","last_became_peered":"2024-04-25T09:16:03.369554+0000","last_unstale":"2024-04-25T09:16:04.371552+0000","last_undegraded":"2024-04-25T09:16:04.371552+0000","last_fullsized":"2024-04-25T09:16:04.371552+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:33.662482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442366+0000","last_change":"2024-04-25T09:16:03.364843+0000","last_active":"2024-04-25T09:16:04.442366+0000","last_peered":"2024-04-25T09:16:04.442366+0000","last_clean":"2024-04-25T09:16:04.442366+0000","last_became_active":"2024-04-25T09:16:03.364552+0000","last_became_peered":"2024-04-25T09:16:03.364552+0000","last_unstale":"2024-04-25T09:16:04.442366+0000","last_undegraded":"2024-04-25T09:16:04.442366+0000","last_fullsized":"2024-04-25T09:16:04.442366+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:34.715830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.441999+0000","last_change":"2024-04-25T09:16:03.371330+0000","last_active":"2024-04-25T09:16:04.441999+0000","last_peered":"2024-04-25T09:16:04.441999+0000","last_clean":"2024-04-25T09:16:04.441999+0000","last_became_active":"2024-04-25T09:16:03.371042+0000","last_became_peered":"2024-04-25T09:16:03.371042+0000","last_unstale":"2024-04-25T09:16:04.441999+0000","last_undegraded":"2024-04-25T09:16:04.441999+0000","last_fullsized":"2024-04-25T09:16:04.441999+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:08.216956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371909+0000","last_change":"2024-04-25T09:16:03.371504+0000","last_active":"2024-04-25T09:16:04.371909+0000","last_peered":"2024-04-25T09:16:04.371909+0000","last_clean":"2024-04-25T09:16:04.371909+0000","last_became_active":"2024-04-25T09:16:03.371196+0000","last_became_peered":"2024-04-25T09:16:03.371196+0000","last_unstale":"2024-04-25T09:16:04.371909+0000","last_undegraded":"2024-04-25T09:16:04.371909+0000","last_fullsized":"2024-04-25T09:16:04.371909+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:02.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373458+0000","last_change":"2024-04-25T09:16:03.371158+0000","last_active":"2024-04-25T09:16:04.373458+0000","last_peered":"2024-04-25T09:16:04.373458+0000","last_clean":"2024-04-25T09:16:04.373458+0000","last_became_active":"2024-04-25T09:16:03.370360+0000","last_became_peered":"2024-04-25T09:16:03.370360+0000","last_unstale":"2024-04-25T09:16:04.373458+0000","last_undegraded":"2024-04-25T09:16:04.373458+0000","last_fullsized":"2024-04-25T09:16:04.373458+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:54.966474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471390+0000","last_change":"2024-04-25T09:16:03.377595+0000","last_active":"2024-04-25T09:16:05.471390+0000","last_peered":"2024-04-25T09:16:05.471390+0000","last_clean":"2024-04-25T09:16:05.471390+0000","last_became_active":"2024-04-25T09:16:03.375843+0000","last_became_peered":"2024-04-25T09:16:03.375843+0000","last_unstale":"2024-04-25T09:16:05.471390+0000","last_undegraded":"2024-04-25T09:16:05.471390+0000","last_fullsized":"2024-04-25T09:16:05.471390+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:48.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373247+0000","last_change":"2024-04-25T09:16:03.380406+0000","last_active":"2024-04-25T09:16:04.373247+0000","last_peered":"2024-04-25T09:16:04.373247+0000","last_clean":"2024-04-25T09:16:04.373247+0000","last_became_active":"2024-04-25T09:16:03.380112+0000","last_became_peered":"2024-04-25T09:16:03.380112+0000","last_unstale":"2024-04-25T09:16:04.373247+0000","last_undegraded":"2024-04-25T09:16:04.373247+0000","last_fullsized":"2024-04-25T09:16:04.373247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.784334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471448+0000","last_change":"2024-04-25T09:16:03.382860+0000","last_active":"2024-04-25T09:16:05.471448+0000","last_peered":"2024-04-25T09:16:05.471448+0000","last_clean":"2024-04-25T09:16:05.471448+0000","last_became_active":"2024-04-25T09:16:03.379568+0000","last_became_peered":"2024-04-25T09:16:03.379568+0000","last_unstale":"2024-04-25T09:16:05.471448+0000","last_undegraded":"2024-04-25T09:16:05.471448+0000","last_fullsized":"2024-04-25T09:16:05.471448+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:33.662482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678143+0000","last_change":"2024-04-25T09:16:03.380933+0000","last_active":"2024-04-25T09:16:05.678143+0000","last_peered":"2024-04-25T09:16:05.678143+0000","last_clean":"2024-04-25T09:16:05.678143+0000","last_became_active":"2024-04-25T09:16:03.380563+0000","last_became_peered":"2024-04-25T09:16:03.380563+0000","last_unstale":"2024-04-25T09:16:05.678143+0000","last_undegraded":"2024-04-25T09:16:05.678143+0000","last_fullsized":"2024-04-25T09:16:05.678143+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:02.379597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676494+0000","last_change":"2024-04-25T09:16:03.388703+0000","last_active":"2024-04-25T09:16:05.676494+0000","last_peered":"2024-04-25T09:16:05.676494+0000","last_clean":"2024-04-25T09:16:05.676494+0000","last_became_active":"2024-04-25T09:16:03.388353+0000","last_became_peered":"2024-04-25T09:16:03.388353+0000","last_unstale":"2024-04-25T09:16:05.676494+0000","last_undegraded":"2024-04-25T09:16:05.676494+0000","last_fullsized":"2024-04-25T09:16:05.676494+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:09.737931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676550+0000","last_change":"2024-04-25T09:16:03.379548+0000","last_active":"2024-04-25T09:16:05.676550+0000","last_peered":"2024-04-25T09:16:05.676550+0000","last_clean":"2024-04-25T09:16:05.676550+0000","last_became_active":"2024-04-25T09:16:03.379204+0000","last_became_peered":"2024-04-25T09:16:03.379204+0000","last_unstale":"2024-04-25T09:16:05.676550+0000","last_undegraded":"2024-04-25T09:16:05.676550+0000","last_fullsized":"2024-04-25T09:16:05.676550+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.177293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372207+0000","last_change":"2024-04-25T09:16:03.379906+0000","last_active":"2024-04-25T09:16:04.372207+0000","last_peered":"2024-04-25T09:16:04.372207+0000","last_clean":"2024-04-25T09:16:04.372207+0000","last_became_active":"2024-04-25T09:16:03.379612+0000","last_became_peered":"2024-04-25T09:16:03.379612+0000","last_unstale":"2024-04-25T09:16:04.372207+0000","last_undegraded":"2024-04-25T09:16:04.372207+0000","last_fullsized":"2024-04-25T09:16:04.372207+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:35.051152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676971+0000","last_change":"2024-04-25T09:16:03.383421+0000","last_active":"2024-04-25T09:16:05.676971+0000","last_peered":"2024-04-25T09:16:05.676971+0000","last_clean":"2024-04-25T09:16:05.676971+0000","last_became_active":"2024-04-25T09:16:03.383108+0000","last_became_peered":"2024-04-25T09:16:03.383108+0000","last_unstale":"2024-04-25T09:16:05.676971+0000","last_undegraded":"2024-04-25T09:16:05.676971+0000","last_fullsized":"2024-04-25T09:16:05.676971+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:28.815944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373205+0000","last_change":"2024-04-25T09:16:03.363745+0000","last_active":"2024-04-25T09:16:04.373205+0000","last_peered":"2024-04-25T09:16:04.373205+0000","last_clean":"2024-04-25T09:16:04.373205+0000","last_became_active":"2024-04-25T09:16:03.363333+0000","last_became_peered":"2024-04-25T09:16:03.363333+0000","last_unstale":"2024-04-25T09:16:04.373205+0000","last_undegraded":"2024-04-25T09:16:04.373205+0000","last_fullsized":"2024-04-25T09:16:04.373205+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.177293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676870+0000","last_change":"2024-04-25T09:16:03.385421+0000","last_active":"2024-04-25T09:16:05.676870+0000","last_peered":"2024-04-25T09:16:05.676870+0000","last_clean":"2024-04-25T09:16:05.676870+0000","last_became_active":"2024-04-25T09:16:03.384652+0000","last_became_peered":"2024-04-25T09:16:03.384652+0000","last_unstale":"2024-04-25T09:16:05.676870+0000","last_undegraded":"2024-04-25T09:16:05.676870+0000","last_fullsized":"2024-04-25T09:16:05.676870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:14.907583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371174+0000","last_change":"2024-04-25T09:16:03.383009+0000","last_active":"2024-04-25T09:16:04.371174+0000","last_peered":"2024-04-25T09:16:04.371174+0000","last_clean":"2024-04-25T09:16:04.371174+0000","last_became_active":"2024-04-25T09:16:03.382586+0000","last_became_peered":"2024-04-25T09:16:03.382586+0000","last_unstale":"2024-04-25T09:16:04.371174+0000","last_undegraded":"2024-04-25T09:16:04.371174+0000","last_fullsized":"2024-04-25T09:16:04.371174+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:29.636797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373187+0000","last_change":"2024-04-25T09:16:03.380578+0000","last_active":"2024-04-25T09:16:04.373187+0000","last_peered":"2024-04-25T09:16:04.373187+0000","last_clean":"2024-04-25T09:16:04.373187+0000","last_became_active":"2024-04-25T09:16:03.377387+0000","last_became_peered":"2024-04-25T09:16:03.377387+0000","last_unstale":"2024-04-25T09:16:04.373187+0000","last_undegraded":"2024-04-25T09:16:04.373187+0000","last_fullsized":"2024-04-25T09:16:04.373187+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:58.773494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.433552+0000","last_change":"2024-04-25T09:16:03.367372+0000","last_active":"2024-04-25T09:16:04.433552+0000","last_peered":"2024-04-25T09:16:04.433552+0000","last_clean":"2024-04-25T09:16:04.433552+0000","last_became_active":"2024-04-25T09:16:03.367081+0000","last_became_peered":"2024-04-25T09:16:03.367081+0000","last_unstale":"2024-04-25T09:16:04.433552+0000","last_undegraded":"2024-04-25T09:16:04.433552+0000","last_fullsized":"2024-04-25T09:16:04.433552+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:59.790968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676013+0000","last_change":"2024-04-25T09:16:03.386567+0000","last_active":"2024-04-25T09:16:05.676013+0000","last_peered":"2024-04-25T09:16:05.676013+0000","last_clean":"2024-04-25T09:16:05.676013+0000","last_became_active":"2024-04-25T09:16:03.386248+0000","last_became_peered":"2024-04-25T09:16:03.386248+0000","last_unstale":"2024-04-25T09:16:05.676013+0000","last_undegraded":"2024-04-25T09:16:05.676013+0000","last_fullsized":"2024-04-25T09:16:05.676013+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:31.706208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471941+0000","last_change":"2024-04-25T09:16:03.377936+0000","last_active":"2024-04-25T09:16:05.471941+0000","last_peered":"2024-04-25T09:16:05.471941+0000","last_clean":"2024-04-25T09:16:05.471941+0000","last_became_active":"2024-04-25T09:16:03.377376+0000","last_became_peered":"2024-04-25T09:16:03.377376+0000","last_unstale":"2024-04-25T09:16:05.471941+0000","last_undegraded":"2024-04-25T09:16:05.471941+0000","last_fullsized":"2024-04-25T09:16:05.471941+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:58.773494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373694+0000","last_change":"2024-04-25T09:16:03.371411+0000","last_active":"2024-04-25T09:16:04.373694+0000","last_peered":"2024-04-25T09:16:04.373694+0000","last_clean":"2024-04-25T09:16:04.373694+0000","last_became_active":"2024-04-25T09:16:03.370758+0000","last_became_peered":"2024-04-25T09:16:03.370758+0000","last_unstale":"2024-04-25T09:16:04.373694+0000","last_undegraded":"2024-04-25T09:16:04.373694+0000","last_fullsized":"2024-04-25T09:16:04.373694+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:40.323678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678380+0000","last_change":"2024-04-25T09:16:03.388224+0000","last_active":"2024-04-25T09:16:05.678380+0000","last_peered":"2024-04-25T09:16:05.678380+0000","last_clean":"2024-04-25T09:16:05.678380+0000","last_became_active":"2024-04-25T09:16:03.386349+0000","last_became_peered":"2024-04-25T09:16:03.386349+0000","last_unstale":"2024-04-25T09:16:05.678380+0000","last_undegraded":"2024-04-25T09:16:05.678380+0000","last_fullsized":"2024-04-25T09:16:05.678380+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:28.870734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434481+0000","last_change":"2024-04-25T09:16:03.376804+0000","last_active":"2024-04-25T09:16:04.434481+0000","last_peered":"2024-04-25T09:16:04.434481+0000","last_clean":"2024-04-25T09:16:04.434481+0000","last_became_active":"2024-04-25T09:16:03.373715+0000","last_became_peered":"2024-04-25T09:16:03.373715+0000","last_unstale":"2024-04-25T09:16:04.434481+0000","last_undegraded":"2024-04-25T09:16:04.434481+0000","last_fullsized":"2024-04-25T09:16:04.434481+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:47.041323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372393+0000","last_change":"2024-04-25T09:16:03.373941+0000","last_active":"2024-04-25T09:16:04.372393+0000","last_peered":"2024-04-25T09:16:04.372393+0000","last_clean":"2024-04-25T09:16:04.372393+0000","last_became_active":"2024-04-25T09:16:03.373585+0000","last_became_peered":"2024-04-25T09:16:03.373585+0000","last_unstale":"2024-04-25T09:16:04.372393+0000","last_undegraded":"2024-04-25T09:16:04.372393+0000","last_fullsized":"2024-04-25T09:16:04.372393+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:33.662482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372165+0000","last_change":"2024-04-25T09:16:03.358626+0000","last_active":"2024-04-25T09:16:04.372165+0000","last_peered":"2024-04-25T09:16:04.372165+0000","last_clean":"2024-04-25T09:16:04.372165+0000","last_became_active":"2024-04-25T09:16:03.358023+0000","last_became_peered":"2024-04-25T09:16:03.358023+0000","last_unstale":"2024-04-25T09:16:04.372165+0000","last_undegraded":"2024-04-25T09:16:04.372165+0000","last_fullsized":"2024-04-25T09:16:04.372165+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:48.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.433728+0000","last_change":"2024-04-25T09:16:03.371066+0000","last_active":"2024-04-25T09:16:04.433728+0000","last_peered":"2024-04-25T09:16:04.433728+0000","last_clean":"2024-04-25T09:16:04.433728+0000","last_became_active":"2024-04-25T09:16:03.369834+0000","last_became_peered":"2024-04-25T09:16:03.369834+0000","last_unstale":"2024-04-25T09:16:04.433728+0000","last_undegraded":"2024-04-25T09:16:04.433728+0000","last_fullsized":"2024-04-25T09:16:04.433728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:10.957296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373665+0000","last_change":"2024-04-25T09:16:03.374575+0000","last_active":"2024-04-25T09:16:04.373665+0000","last_peered":"2024-04-25T09:16:04.373665+0000","last_clean":"2024-04-25T09:16:04.373665+0000","last_became_active":"2024-04-25T09:16:03.369845+0000","last_became_peered":"2024-04-25T09:16:03.369845+0000","last_unstale":"2024-04-25T09:16:04.373665+0000","last_undegraded":"2024-04-25T09:16:04.373665+0000","last_fullsized":"2024-04-25T09:16:04.373665+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:35.378589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677806+0000","last_change":"2024-04-25T09:16:03.377625+0000","last_active":"2024-04-25T09:16:05.677806+0000","last_peered":"2024-04-25T09:16:05.677806+0000","last_clean":"2024-04-25T09:16:05.677806+0000","last_became_active":"2024-04-25T09:16:03.377305+0000","last_became_peered":"2024-04-25T09:16:03.377305+0000","last_unstale":"2024-04-25T09:16:05.677806+0000","last_undegraded":"2024-04-25T09:16:05.677806+0000","last_fullsized":"2024-04-25T09:16:05.677806+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:14.558673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"136'4","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442333+0000","last_change":"2024-04-25T09:16:02.370695+0000","last_active":"2024-04-25T09:16:04.442333+0000","last_peered":"2024-04-25T09:16:04.442333+0000","last_clean":"2024-04-25T09:16:04.442333+0000","last_became_active":"2024-04-25T09:16:02.370360+0000","last_became_peered":"2024-04-25T09:16:02.370360+0000","last_unstale":"2024-04-25T09:16:04.442333+0000","last_undegraded":"2024-04-25T09:16:04.442333+0000","last_fullsized":"2024-04-25T09:16:04.442333+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:27.798812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372065+0000","last_change":"2024-04-25T09:16:03.383924+0000","last_active":"2024-04-25T09:16:04.372065+0000","last_peered":"2024-04-25T09:16:04.372065+0000","last_clean":"2024-04-25T09:16:04.372065+0000","last_became_active":"2024-04-25T09:16:03.383596+0000","last_became_peered":"2024-04-25T09:16:03.383596+0000","last_unstale":"2024-04-25T09:16:04.372065+0000","last_undegraded":"2024-04-25T09:16:04.372065+0000","last_fullsized":"2024-04-25T09:16:04.372065+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:06.549852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372796+0000","last_change":"2024-04-25T09:16:02.376402+0000","last_active":"2024-04-25T09:16:04.372796+0000","last_peered":"2024-04-25T09:16:04.372796+0000","last_clean":"2024-04-25T09:16:04.372796+0000","last_became_active":"2024-04-25T09:16:02.375831+0000","last_became_peered":"2024-04-25T09:16:02.375831+0000","last_unstale":"2024-04-25T09:16:04.372796+0000","last_undegraded":"2024-04-25T09:16:04.372796+0000","last_fullsized":"2024-04-25T09:16:04.372796+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:36.774027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371730+0000","last_change":"2024-04-25T09:16:03.377499+0000","last_active":"2024-04-25T09:16:04.371730+0000","last_peered":"2024-04-25T09:16:04.371730+0000","last_clean":"2024-04-25T09:16:04.371730+0000","last_became_active":"2024-04-25T09:16:03.377202+0000","last_became_peered":"2024-04-25T09:16:03.377202+0000","last_unstale":"2024-04-25T09:16:04.371730+0000","last_undegraded":"2024-04-25T09:16:04.371730+0000","last_fullsized":"2024-04-25T09:16:04.371730+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:58.844284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471956+0000","last_change":"2024-04-25T09:16:02.372552+0000","last_active":"2024-04-25T09:16:05.471956+0000","last_peered":"2024-04-25T09:16:05.471956+0000","last_clean":"2024-04-25T09:16:05.471956+0000","last_became_active":"2024-04-25T09:16:02.372243+0000","last_became_peered":"2024-04-25T09:16:02.372243+0000","last_unstale":"2024-04-25T09:16:05.471956+0000","last_undegraded":"2024-04-25T09:16:05.471956+0000","last_fullsized":"2024-04-25T09:16:05.471956+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.031924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372428+0000","last_change":"2024-04-25T09:16:03.375343+0000","last_active":"2024-04-25T09:16:04.372428+0000","last_peered":"2024-04-25T09:16:04.372428+0000","last_clean":"2024-04-25T09:16:04.372428+0000","last_became_active":"2024-04-25T09:16:03.374986+0000","last_became_peered":"2024-04-25T09:16:03.374986+0000","last_unstale":"2024-04-25T09:16:04.372428+0000","last_undegraded":"2024-04-25T09:16:04.372428+0000","last_fullsized":"2024-04-25T09:16:04.372428+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:45.177293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373373+0000","last_change":"2024-04-25T09:16:02.372624+0000","last_active":"2024-04-25T09:16:04.373373+0000","last_peered":"2024-04-25T09:16:04.373373+0000","last_clean":"2024-04-25T09:16:04.373373+0000","last_became_active":"2024-04-25T09:16:02.372309+0000","last_became_peered":"2024-04-25T09:16:02.372309+0000","last_unstale":"2024-04-25T09:16:04.373373+0000","last_undegraded":"2024-04-25T09:16:04.373373+0000","last_fullsized":"2024-04-25T09:16:04.373373+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.590861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676615+0000","last_change":"2024-04-25T09:16:03.387453+0000","last_active":"2024-04-25T09:16:05.676615+0000","last_peered":"2024-04-25T09:16:05.676615+0000","last_clean":"2024-04-25T09:16:05.676615+0000","last_became_active":"2024-04-25T09:16:03.387187+0000","last_became_peered":"2024-04-25T09:16:03.387187+0000","last_unstale":"2024-04-25T09:16:05.676615+0000","last_undegraded":"2024-04-25T09:16:05.676615+0000","last_fullsized":"2024-04-25T09:16:05.676615+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:28.089832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372541+0000","last_change":"2024-04-25T09:16:02.367514+0000","last_active":"2024-04-25T09:16:04.372541+0000","last_peered":"2024-04-25T09:16:04.372541+0000","last_clean":"2024-04-25T09:16:04.372541+0000","last_became_active":"2024-04-25T09:16:02.367197+0000","last_became_peered":"2024-04-25T09:16:02.367197+0000","last_unstale":"2024-04-25T09:16:04.372541+0000","last_undegraded":"2024-04-25T09:16:04.372541+0000","last_fullsized":"2024-04-25T09:16:04.372541+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:27.798812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371668+0000","last_change":"2024-04-25T09:16:03.380649+0000","last_active":"2024-04-25T09:16:04.371668+0000","last_peered":"2024-04-25T09:16:04.371668+0000","last_clean":"2024-04-25T09:16:04.371668+0000","last_became_active":"2024-04-25T09:16:03.376513+0000","last_became_peered":"2024-04-25T09:16:03.376513+0000","last_unstale":"2024-04-25T09:16:04.371668+0000","last_undegraded":"2024-04-25T09:16:04.371668+0000","last_fullsized":"2024-04-25T09:16:04.371668+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:58.773494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676926+0000","last_change":"2024-04-25T09:16:02.368110+0000","last_active":"2024-04-25T09:16:05.676926+0000","last_peered":"2024-04-25T09:16:05.676926+0000","last_clean":"2024-04-25T09:16:05.676926+0000","last_became_active":"2024-04-25T09:16:02.367791+0000","last_became_peered":"2024-04-25T09:16:02.367791+0000","last_unstale":"2024-04-25T09:16:05.676926+0000","last_undegraded":"2024-04-25T09:16:05.676926+0000","last_fullsized":"2024-04-25T09:16:05.676926+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:09.940164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.370099+0000","last_change":"2024-04-25T09:16:03.368693+0000","last_active":"2024-04-25T09:16:04.370099+0000","last_peered":"2024-04-25T09:16:04.370099+0000","last_clean":"2024-04-25T09:16:04.370099+0000","last_became_active":"2024-04-25T09:16:03.366083+0000","last_became_peered":"2024-04-25T09:16:03.366083+0000","last_unstale":"2024-04-25T09:16:04.370099+0000","last_undegraded":"2024-04-25T09:16:04.370099+0000","last_fullsized":"2024-04-25T09:16:04.370099+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:43.348298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.370899+0000","last_change":"2024-04-25T09:16:02.367478+0000","last_active":"2024-04-25T09:16:04.370899+0000","last_peered":"2024-04-25T09:16:04.370899+0000","last_clean":"2024-04-25T09:16:04.370899+0000","last_became_active":"2024-04-25T09:16:02.367211+0000","last_became_peered":"2024-04-25T09:16:02.367211+0000","last_unstale":"2024-04-25T09:16:04.370899+0000","last_undegraded":"2024-04-25T09:16:04.370899+0000","last_fullsized":"2024-04-25T09:16:04.370899+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:44.132361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371685+0000","last_change":"2024-04-25T09:16:03.390386+0000","last_active":"2024-04-25T09:16:04.371685+0000","last_peered":"2024-04-25T09:16:04.371685+0000","last_clean":"2024-04-25T09:16:04.371685+0000","last_became_active":"2024-04-25T09:16:03.390010+0000","last_became_peered":"2024-04-25T09:16:03.390010+0000","last_unstale":"2024-04-25T09:16:04.371685+0000","last_undegraded":"2024-04-25T09:16:04.371685+0000","last_fullsized":"2024-04-25T09:16:04.371685+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:29.636797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371789+0000","last_change":"2024-04-25T09:16:02.373690+0000","last_active":"2024-04-25T09:16:04.371789+0000","last_peered":"2024-04-25T09:16:04.371789+0000","last_clean":"2024-04-25T09:16:04.371789+0000","last_became_active":"2024-04-25T09:16:02.373341+0000","last_became_peered":"2024-04-25T09:16:02.373341+0000","last_unstale":"2024-04-25T09:16:04.371789+0000","last_undegraded":"2024-04-25T09:16:04.371789+0000","last_fullsized":"2024-04-25T09:16:04.371789+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:53.927207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373840+0000","last_change":"2024-04-25T09:16:03.381459+0000","last_active":"2024-04-25T09:16:04.373840+0000","last_peered":"2024-04-25T09:16:04.373840+0000","last_clean":"2024-04-25T09:16:04.373840+0000","last_became_active":"2024-04-25T09:16:03.381133+0000","last_became_peered":"2024-04-25T09:16:03.381133+0000","last_unstale":"2024-04-25T09:16:04.373840+0000","last_undegraded":"2024-04-25T09:16:04.373840+0000","last_fullsized":"2024-04-25T09:16:04.373840+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:28.870734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373460+0000","last_change":"2024-04-25T09:16:02.371172+0000","last_active":"2024-04-25T09:16:04.373460+0000","last_peered":"2024-04-25T09:16:04.373460+0000","last_clean":"2024-04-25T09:16:04.373460+0000","last_became_active":"2024-04-25T09:16:02.370533+0000","last_became_peered":"2024-04-25T09:16:02.370533+0000","last_unstale":"2024-04-25T09:16:04.373460+0000","last_undegraded":"2024-04-25T09:16:04.373460+0000","last_fullsized":"2024-04-25T09:16:04.373460+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:27.072701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442202+0000","last_change":"2024-04-25T09:16:03.386920+0000","last_active":"2024-04-25T09:16:04.442202+0000","last_peered":"2024-04-25T09:16:04.442202+0000","last_clean":"2024-04-25T09:16:04.442202+0000","last_became_active":"2024-04-25T09:16:03.386542+0000","last_became_peered":"2024-04-25T09:16:03.386542+0000","last_unstale":"2024-04-25T09:16:04.442202+0000","last_undegraded":"2024-04-25T09:16:04.442202+0000","last_fullsized":"2024-04-25T09:16:04.442202+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:31.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677278+0000","last_change":"2024-04-25T09:16:02.370687+0000","last_active":"2024-04-25T09:16:05.677278+0000","last_peered":"2024-04-25T09:16:05.677278+0000","last_clean":"2024-04-25T09:16:05.677278+0000","last_became_active":"2024-04-25T09:16:02.369555+0000","last_became_peered":"2024-04-25T09:16:02.369555+0000","last_unstale":"2024-04-25T09:16:05.677278+0000","last_undegraded":"2024-04-25T09:16:05.677278+0000","last_fullsized":"2024-04-25T09:16:05.677278+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:46.409277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373292+0000","last_change":"2024-04-25T09:16:03.369481+0000","last_active":"2024-04-25T09:16:04.373292+0000","last_peered":"2024-04-25T09:16:04.373292+0000","last_clean":"2024-04-25T09:16:04.373292+0000","last_became_active":"2024-04-25T09:16:03.369204+0000","last_became_peered":"2024-04-25T09:16:03.369204+0000","last_unstale":"2024-04-25T09:16:04.373292+0000","last_undegraded":"2024-04-25T09:16:04.373292+0000","last_fullsized":"2024-04-25T09:16:04.373292+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:54.966474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"136'3","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442541+0000","last_change":"2024-04-25T09:16:02.368839+0000","last_active":"2024-04-25T09:16:04.442541+0000","last_peered":"2024-04-25T09:16:04.442541+0000","last_clean":"2024-04-25T09:16:04.442541+0000","last_became_active":"2024-04-25T09:16:02.368460+0000","last_became_peered":"2024-04-25T09:16:02.368460+0000","last_unstale":"2024-04-25T09:16:04.442541+0000","last_undegraded":"2024-04-25T09:16:04.442541+0000","last_fullsized":"2024-04-25T09:16:04.442541+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:53.927207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434521+0000","last_change":"2024-04-25T09:16:03.372581+0000","last_active":"2024-04-25T09:16:04.434521+0000","last_peered":"2024-04-25T09:16:04.434521+0000","last_clean":"2024-04-25T09:16:04.434521+0000","last_became_active":"2024-04-25T09:16:03.372300+0000","last_became_peered":"2024-04-25T09:16:03.372300+0000","last_unstale":"2024-04-25T09:16:04.434521+0000","last_undegraded":"2024-04-25T09:16:04.434521+0000","last_fullsized":"2024-04-25T09:16:04.434521+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:48.613306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"136'6","reported_seq":20,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434473+0000","last_change":"2024-04-25T09:16:02.361417+0000","last_active":"2024-04-25T09:16:04.434473+0000","last_peered":"2024-04-25T09:16:04.434473+0000","last_clean":"2024-04-25T09:16:04.434473+0000","last_became_active":"2024-04-25T09:16:02.361131+0000","last_became_peered":"2024-04-25T09:16:02.361131+0000","last_unstale":"2024-04-25T09:16:04.434473+0000","last_undegraded":"2024-04-25T09:16:04.434473+0000","last_fullsized":"2024-04-25T09:16:04.434473+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:51.176591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678024+0000","last_change":"2024-04-25T09:16:03.384935+0000","last_active":"2024-04-25T09:16:05.678024+0000","last_peered":"2024-04-25T09:16:05.678024+0000","last_clean":"2024-04-25T09:16:05.678024+0000","last_became_active":"2024-04-25T09:16:03.384627+0000","last_became_peered":"2024-04-25T09:16:03.384627+0000","last_unstale":"2024-04-25T09:16:05.678024+0000","last_undegraded":"2024-04-25T09:16:05.678024+0000","last_fullsized":"2024-04-25T09:16:05.678024+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:17.397100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677942+0000","last_change":"2024-04-25T09:16:02.370230+0000","last_active":"2024-04-25T09:16:05.677942+0000","last_peered":"2024-04-25T09:16:05.677942+0000","last_clean":"2024-04-25T09:16:05.677942+0000","last_became_active":"2024-04-25T09:16:02.368452+0000","last_became_peered":"2024-04-25T09:16:02.368452+0000","last_unstale":"2024-04-25T09:16:05.677942+0000","last_undegraded":"2024-04-25T09:16:05.677942+0000","last_fullsized":"2024-04-25T09:16:05.677942+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:25.031924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677670+0000","last_change":"2024-04-25T09:16:03.381645+0000","last_active":"2024-04-25T09:16:05.677670+0000","last_peered":"2024-04-25T09:16:05.677670+0000","last_clean":"2024-04-25T09:16:05.677670+0000","last_became_active":"2024-04-25T09:16:03.380994+0000","last_became_peered":"2024-04-25T09:16:03.380994+0000","last_unstale":"2024-04-25T09:16:05.677670+0000","last_undegraded":"2024-04-25T09:16:05.677670+0000","last_fullsized":"2024-04-25T09:16:05.677670+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:54.966474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471184+0000","last_change":"2024-04-25T09:16:02.372591+0000","last_active":"2024-04-25T09:16:05.471184+0000","last_peered":"2024-04-25T09:16:05.471184+0000","last_clean":"2024-04-25T09:16:05.471184+0000","last_became_active":"2024-04-25T09:16:02.372297+0000","last_became_peered":"2024-04-25T09:16:02.372297+0000","last_unstale":"2024-04-25T09:16:05.471184+0000","last_undegraded":"2024-04-25T09:16:05.471184+0000","last_fullsized":"2024-04-25T09:16:05.471184+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:07.199825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.370721+0000","last_change":"2024-04-25T09:16:03.366250+0000","last_active":"2024-04-25T09:16:04.370721+0000","last_peered":"2024-04-25T09:16:04.370721+0000","last_clean":"2024-04-25T09:16:04.370721+0000","last_became_active":"2024-04-25T09:16:03.365966+0000","last_became_peered":"2024-04-25T09:16:03.365966+0000","last_unstale":"2024-04-25T09:16:04.370721+0000","last_undegraded":"2024-04-25T09:16:04.370721+0000","last_fullsized":"2024-04-25T09:16:04.370721+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:02.923720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"136'8","reported_seq":22,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.370348+0000","last_change":"2024-04-25T09:16:02.365052+0000","last_active":"2024-04-25T09:16:04.370348+0000","last_peered":"2024-04-25T09:16:04.370348+0000","last_clean":"2024-04-25T09:16:04.370348+0000","last_became_active":"2024-04-25T09:16:02.362570+0000","last_became_peered":"2024-04-25T09:16:02.362570+0000","last_unstale":"2024-04-25T09:16:04.370348+0000","last_undegraded":"2024-04-25T09:16:04.370348+0000","last_fullsized":"2024-04-25T09:16:04.370348+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:53.927207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442459+0000","last_change":"2024-04-25T09:16:03.373592+0000","last_active":"2024-04-25T09:16:04.442459+0000","last_peered":"2024-04-25T09:16:04.442459+0000","last_clean":"2024-04-25T09:16:04.442459+0000","last_became_active":"2024-04-25T09:16:03.373301+0000","last_became_peered":"2024-04-25T09:16:03.373301+0000","last_unstale":"2024-04-25T09:16:04.442459+0000","last_undegraded":"2024-04-25T09:16:04.442459+0000","last_fullsized":"2024-04-25T09:16:04.442459+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:49.627786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373756+0000","last_change":"2024-04-25T09:16:02.371391+0000","last_active":"2024-04-25T09:16:04.373756+0000","last_peered":"2024-04-25T09:16:04.373756+0000","last_clean":"2024-04-25T09:16:04.373756+0000","last_became_active":"2024-04-25T09:16:02.370909+0000","last_became_peered":"2024-04-25T09:16:02.370909+0000","last_unstale":"2024-04-25T09:16:04.373756+0000","last_undegraded":"2024-04-25T09:16:04.373756+0000","last_fullsized":"2024-04-25T09:16:04.373756+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:27.798812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.442576+0000","last_change":"2024-04-25T09:16:03.391112+0000","last_active":"2024-04-25T09:16:04.442576+0000","last_peered":"2024-04-25T09:16:04.442576+0000","last_clean":"2024-04-25T09:16:04.442576+0000","last_became_active":"2024-04-25T09:16:03.390509+0000","last_became_peered":"2024-04-25T09:16:03.390509+0000","last_unstale":"2024-04-25T09:16:04.442576+0000","last_undegraded":"2024-04-25T09:16:04.442576+0000","last_fullsized":"2024-04-25T09:16:04.442576+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:23.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471686+0000","last_change":"2024-04-25T09:16:02.369859+0000","last_active":"2024-04-25T09:16:05.471686+0000","last_peered":"2024-04-25T09:16:05.471686+0000","last_clean":"2024-04-25T09:16:05.471686+0000","last_became_active":"2024-04-25T09:16:02.369116+0000","last_became_peered":"2024-04-25T09:16:02.369116+0000","last_unstale":"2024-04-25T09:16:05.471686+0000","last_undegraded":"2024-04-25T09:16:05.471686+0000","last_fullsized":"2024-04-25T09:16:05.471686+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:22.498085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373533+0000","last_change":"2024-04-25T09:16:03.373497+0000","last_active":"2024-04-25T09:16:04.373533+0000","last_peered":"2024-04-25T09:16:04.373533+0000","last_clean":"2024-04-25T09:16:04.373533+0000","last_became_active":"2024-04-25T09:16:03.373120+0000","last_became_peered":"2024-04-25T09:16:03.373120+0000","last_unstale":"2024-04-25T09:16:04.373533+0000","last_undegraded":"2024-04-25T09:16:04.373533+0000","last_fullsized":"2024-04-25T09:16:04.373533+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:17.397100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"136'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.374358+0000","last_change":"2024-04-25T09:16:02.373019+0000","last_active":"2024-04-25T09:16:04.374358+0000","last_peered":"2024-04-25T09:16:04.374358+0000","last_clean":"2024-04-25T09:16:04.374358+0000","last_became_active":"2024-04-25T09:16:02.372726+0000","last_became_peered":"2024-04-25T09:16:02.372726+0000","last_unstale":"2024-04-25T09:16:04.374358+0000","last_undegraded":"2024-04-25T09:16:04.374358+0000","last_fullsized":"2024-04-25T09:16:04.374358+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:40.536945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373461+0000","last_change":"2024-04-25T09:16:03.390604+0000","last_active":"2024-04-25T09:16:04.373461+0000","last_peered":"2024-04-25T09:16:04.373461+0000","last_clean":"2024-04-25T09:16:04.373461+0000","last_became_active":"2024-04-25T09:16:03.390308+0000","last_became_peered":"2024-04-25T09:16:03.390308+0000","last_unstale":"2024-04-25T09:16:04.373461+0000","last_undegraded":"2024-04-25T09:16:04.373461+0000","last_fullsized":"2024-04-25T09:16:04.373461+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:14.558673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.441921+0000","last_change":"2024-04-25T09:16:02.366373+0000","last_active":"2024-04-25T09:16:04.441921+0000","last_peered":"2024-04-25T09:16:04.441921+0000","last_clean":"2024-04-25T09:16:04.441921+0000","last_became_active":"2024-04-25T09:16:02.366008+0000","last_became_peered":"2024-04-25T09:16:02.366008+0000","last_unstale":"2024-04-25T09:16:04.441921+0000","last_undegraded":"2024-04-25T09:16:04.441921+0000","last_fullsized":"2024-04-25T09:16:04.441921+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.590861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372856+0000","last_change":"2024-04-25T09:16:03.389019+0000","last_active":"2024-04-25T09:16:04.372856+0000","last_peered":"2024-04-25T09:16:04.372856+0000","last_clean":"2024-04-25T09:16:04.372856+0000","last_became_active":"2024-04-25T09:16:03.388747+0000","last_became_peered":"2024-04-25T09:16:03.388747+0000","last_unstale":"2024-04-25T09:16:04.372856+0000","last_undegraded":"2024-04-25T09:16:04.372856+0000","last_fullsized":"2024-04-25T09:16:04.372856+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:41.625765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372505+0000","last_change":"2024-04-25T09:16:02.374156+0000","last_active":"2024-04-25T09:16:04.372505+0000","last_peered":"2024-04-25T09:16:04.372505+0000","last_clean":"2024-04-25T09:16:04.372505+0000","last_became_active":"2024-04-25T09:16:02.373871+0000","last_became_peered":"2024-04-25T09:16:02.373871+0000","last_unstale":"2024-04-25T09:16:04.372505+0000","last_undegraded":"2024-04-25T09:16:04.372505+0000","last_fullsized":"2024-04-25T09:16:04.372505+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:42.526286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434076+0000","last_change":"2024-04-25T09:16:03.359694+0000","last_active":"2024-04-25T09:16:04.434076+0000","last_peered":"2024-04-25T09:16:04.434076+0000","last_clean":"2024-04-25T09:16:04.434076+0000","last_became_active":"2024-04-25T09:16:03.359395+0000","last_became_peered":"2024-04-25T09:16:03.359395+0000","last_unstale":"2024-04-25T09:16:04.434076+0000","last_undegraded":"2024-04-25T09:16:04.434076+0000","last_fullsized":"2024-04-25T09:16:04.434076+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.620761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677203+0000","last_change":"2024-04-25T09:16:02.375497+0000","last_active":"2024-04-25T09:16:05.677203+0000","last_peered":"2024-04-25T09:16:05.677203+0000","last_clean":"2024-04-25T09:16:05.677203+0000","last_became_active":"2024-04-25T09:16:02.375158+0000","last_became_peered":"2024-04-25T09:16:02.375158+0000","last_unstale":"2024-04-25T09:16:05.677203+0000","last_undegraded":"2024-04-25T09:16:05.677203+0000","last_fullsized":"2024-04-25T09:16:05.677203+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:14.603630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678340+0000","last_change":"2024-04-25T09:16:03.383557+0000","last_active":"2024-04-25T09:16:05.678340+0000","last_peered":"2024-04-25T09:16:05.678340+0000","last_clean":"2024-04-25T09:16:05.678340+0000","last_became_active":"2024-04-25T09:16:03.381369+0000","last_became_peered":"2024-04-25T09:16:03.381369+0000","last_unstale":"2024-04-25T09:16:05.678340+0000","last_undegraded":"2024-04-25T09:16:05.678340+0000","last_fullsized":"2024-04-25T09:16:05.678340+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:40.323678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678031+0000","last_change":"2024-04-25T09:16:02.376569+0000","last_active":"2024-04-25T09:16:05.678031+0000","last_peered":"2024-04-25T09:16:05.678031+0000","last_clean":"2024-04-25T09:16:05.678031+0000","last_became_active":"2024-04-25T09:16:02.376238+0000","last_became_peered":"2024-04-25T09:16:02.376238+0000","last_unstale":"2024-04-25T09:16:05.678031+0000","last_undegraded":"2024-04-25T09:16:05.678031+0000","last_fullsized":"2024-04-25T09:16:05.678031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:48.610654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371129+0000","last_change":"2024-04-25T09:16:03.380100+0000","last_active":"2024-04-25T09:16:04.371129+0000","last_peered":"2024-04-25T09:16:04.371129+0000","last_clean":"2024-04-25T09:16:04.371129+0000","last_became_active":"2024-04-25T09:16:03.379809+0000","last_became_peered":"2024-04-25T09:16:03.379809+0000","last_unstale":"2024-04-25T09:16:04.371129+0000","last_undegraded":"2024-04-25T09:16:04.371129+0000","last_fullsized":"2024-04-25T09:16:04.371129+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:08.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.471083+0000","last_change":"2024-04-25T09:16:02.376458+0000","last_active":"2024-04-25T09:16:05.471083+0000","last_peered":"2024-04-25T09:16:05.471083+0000","last_clean":"2024-04-25T09:16:05.471083+0000","last_became_active":"2024-04-25T09:16:02.376129+0000","last_became_peered":"2024-04-25T09:16:02.376129+0000","last_unstale":"2024-04-25T09:16:05.471083+0000","last_undegraded":"2024-04-25T09:16:05.471083+0000","last_fullsized":"2024-04-25T09:16:05.471083+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:30.161052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.434605+0000","last_change":"2024-04-25T09:16:03.373922+0000","last_active":"2024-04-25T09:16:04.434605+0000","last_peered":"2024-04-25T09:16:04.434605+0000","last_clean":"2024-04-25T09:16:04.434605+0000","last_became_active":"2024-04-25T09:16:03.373575+0000","last_became_peered":"2024-04-25T09:16:03.373575+0000","last_unstale":"2024-04-25T09:16:04.434605+0000","last_undegraded":"2024-04-25T09:16:04.434605+0000","last_fullsized":"2024-04-25T09:16:04.434605+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:47.426409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677495+0000","last_change":"2024-04-25T09:16:02.375478+0000","last_active":"2024-04-25T09:16:05.677495+0000","last_peered":"2024-04-25T09:16:05.677495+0000","last_clean":"2024-04-25T09:16:05.677495+0000","last_became_active":"2024-04-25T09:16:02.375165+0000","last_became_peered":"2024-04-25T09:16:02.375165+0000","last_unstale":"2024-04-25T09:16:05.677495+0000","last_undegraded":"2024-04-25T09:16:05.677495+0000","last_fullsized":"2024-04-25T09:16:05.677495+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:46.024192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.373817+0000","last_change":"2024-04-25T09:16:03.374691+0000","last_active":"2024-04-25T09:16:04.373817+0000","last_peered":"2024-04-25T09:16:04.373817+0000","last_clean":"2024-04-25T09:16:04.373817+0000","last_became_active":"2024-04-25T09:16:03.374250+0000","last_became_peered":"2024-04-25T09:16:03.374250+0000","last_unstale":"2024-04-25T09:16:04.373817+0000","last_undegraded":"2024-04-25T09:16:04.373817+0000","last_fullsized":"2024-04-25T09:16:04.373817+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:29.979855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"137'1","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.363708+0000","last_change":"2024-04-25T09:16:02.372805+0000","last_active":"2024-04-25T09:16:05.363708+0000","last_peered":"2024-04-25T09:16:05.363708+0000","last_clean":"2024-04-25T09:16:05.363708+0000","last_became_active":"2024-04-25T09:16:02.370674+0000","last_became_peered":"2024-04-25T09:16:02.370674+0000","last_unstale":"2024-04-25T09:16:05.363708+0000","last_undegraded":"2024-04-25T09:16:05.363708+0000","last_fullsized":"2024-04-25T09:16:05.363708+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:13.890451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678384+0000","last_change":"2024-04-25T09:16:03.379934+0000","last_active":"2024-04-25T09:16:05.678384+0000","last_peered":"2024-04-25T09:16:05.678384+0000","last_clean":"2024-04-25T09:16:05.678384+0000","last_became_active":"2024-04-25T09:16:03.379616+0000","last_became_peered":"2024-04-25T09:16:03.379616+0000","last_unstale":"2024-04-25T09:16:05.678384+0000","last_undegraded":"2024-04-25T09:16:05.678384+0000","last_fullsized":"2024-04-25T09:16:05.678384+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:29.979855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.677586+0000","last_change":"2024-04-25T09:16:02.363332+0000","last_active":"2024-04-25T09:16:05.677586+0000","last_peered":"2024-04-25T09:16:05.677586+0000","last_clean":"2024-04-25T09:16:05.677586+0000","last_became_active":"2024-04-25T09:16:02.362333+0000","last_became_peered":"2024-04-25T09:16:02.362333+0000","last_unstale":"2024-04-25T09:16:05.677586+0000","last_undegraded":"2024-04-25T09:16:05.677586+0000","last_fullsized":"2024-04-25T09:16:05.677586+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:47.596175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.374172+0000","last_change":"2024-04-25T09:16:03.367083+0000","last_active":"2024-04-25T09:16:04.374172+0000","last_peered":"2024-04-25T09:16:04.374172+0000","last_clean":"2024-04-25T09:16:04.374172+0000","last_became_active":"2024-04-25T09:16:03.366691+0000","last_became_peered":"2024-04-25T09:16:03.366691+0000","last_unstale":"2024-04-25T09:16:04.374172+0000","last_undegraded":"2024-04-25T09:16:04.374172+0000","last_fullsized":"2024-04-25T09:16:04.374172+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:34.982268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678388+0000","last_change":"2024-04-25T09:16:02.370620+0000","last_active":"2024-04-25T09:16:05.678388+0000","last_peered":"2024-04-25T09:16:05.678388+0000","last_clean":"2024-04-25T09:16:05.678388+0000","last_became_active":"2024-04-25T09:16:02.370317+0000","last_became_peered":"2024-04-25T09:16:02.370317+0000","last_unstale":"2024-04-25T09:16:05.678388+0000","last_undegraded":"2024-04-25T09:16:05.678388+0000","last_fullsized":"2024-04-25T09:16:05.678388+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:30.161052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371231+0000","last_change":"2024-04-25T09:16:03.387671+0000","last_active":"2024-04-25T09:16:04.371231+0000","last_peered":"2024-04-25T09:16:04.371231+0000","last_clean":"2024-04-25T09:16:04.371231+0000","last_became_active":"2024-04-25T09:16:03.386956+0000","last_became_peered":"2024-04-25T09:16:03.386956+0000","last_unstale":"2024-04-25T09:16:04.371231+0000","last_undegraded":"2024-04-25T09:16:04.371231+0000","last_fullsized":"2024-04-25T09:16:04.371231+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:48.025322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372825+0000","last_change":"2024-04-25T09:16:02.364561+0000","last_active":"2024-04-25T09:16:04.372825+0000","last_peered":"2024-04-25T09:16:04.372825+0000","last_clean":"2024-04-25T09:16:04.372825+0000","last_became_active":"2024-04-25T09:16:02.364203+0000","last_became_peered":"2024-04-25T09:16:02.364203+0000","last_unstale":"2024-04-25T09:16:04.372825+0000","last_undegraded":"2024-04-25T09:16:04.372825+0000","last_fullsized":"2024-04-25T09:16:04.372825+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:06.451506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372079+0000","last_change":"2024-04-25T09:16:03.372988+0000","last_active":"2024-04-25T09:16:04.372079+0000","last_peered":"2024-04-25T09:16:04.372079+0000","last_clean":"2024-04-25T09:16:04.372079+0000","last_became_active":"2024-04-25T09:16:03.372175+0000","last_became_peered":"2024-04-25T09:16:03.372175+0000","last_unstale":"2024-04-25T09:16:04.372079+0000","last_undegraded":"2024-04-25T09:16:04.372079+0000","last_fullsized":"2024-04-25T09:16:04.372079+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:41.625765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371712+0000","last_change":"2024-04-25T09:16:02.361856+0000","last_active":"2024-04-25T09:16:04.371712+0000","last_peered":"2024-04-25T09:16:04.371712+0000","last_clean":"2024-04-25T09:16:04.371712+0000","last_became_active":"2024-04-25T09:16:02.361136+0000","last_became_peered":"2024-04-25T09:16:02.361136+0000","last_unstale":"2024-04-25T09:16:04.371712+0000","last_undegraded":"2024-04-25T09:16:04.371712+0000","last_fullsized":"2024-04-25T09:16:04.371712+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:13.890451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676247+0000","last_change":"2024-04-25T09:16:03.390013+0000","last_active":"2024-04-25T09:16:05.676247+0000","last_peered":"2024-04-25T09:16:05.676247+0000","last_clean":"2024-04-25T09:16:05.676247+0000","last_became_active":"2024-04-25T09:16:03.388991+0000","last_became_peered":"2024-04-25T09:16:03.388991+0000","last_unstale":"2024-04-25T09:16:05.676247+0000","last_undegraded":"2024-04-25T09:16:05.676247+0000","last_fullsized":"2024-04-25T09:16:05.676247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.937573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.470846+0000","last_change":"2024-04-25T09:16:02.372423+0000","last_active":"2024-04-25T09:16:05.470846+0000","last_peered":"2024-04-25T09:16:05.470846+0000","last_clean":"2024-04-25T09:16:05.470846+0000","last_became_active":"2024-04-25T09:16:02.372061+0000","last_became_peered":"2024-04-25T09:16:02.372061+0000","last_unstale":"2024-04-25T09:16:05.470846+0000","last_undegraded":"2024-04-25T09:16:05.470846+0000","last_fullsized":"2024-04-25T09:16:05.470846+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:02.889910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.676658+0000","last_change":"2024-04-25T09:16:03.381637+0000","last_active":"2024-04-25T09:16:05.676658+0000","last_peered":"2024-04-25T09:16:05.676658+0000","last_clean":"2024-04-25T09:16:05.676658+0000","last_became_active":"2024-04-25T09:16:03.380922+0000","last_became_peered":"2024-04-25T09:16:03.380922+0000","last_unstale":"2024-04-25T09:16:05.676658+0000","last_undegraded":"2024-04-25T09:16:05.676658+0000","last_fullsized":"2024-04-25T09:16:05.676658+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:31.784334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:05.678348+0000","last_change":"2024-04-25T09:16:02.370723+0000","last_active":"2024-04-25T09:16:05.678348+0000","last_peered":"2024-04-25T09:16:05.678348+0000","last_clean":"2024-04-25T09:16:05.678348+0000","last_became_active":"2024-04-25T09:16:02.370451+0000","last_became_peered":"2024-04-25T09:16:02.370451+0000","last_unstale":"2024-04-25T09:16:05.678348+0000","last_undegraded":"2024-04-25T09:16:05.678348+0000","last_fullsized":"2024-04-25T09:16:05.678348+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:33.698698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.371247+0000","last_change":"2024-04-25T09:16:03.386353+0000","last_active":"2024-04-25T09:16:04.371247+0000","last_peered":"2024-04-25T09:16:04.371247+0000","last_clean":"2024-04-25T09:16:04.371247+0000","last_became_active":"2024-04-25T09:16:03.385876+0000","last_became_peered":"2024-04-25T09:16:03.385876+0000","last_unstale":"2024-04-25T09:16:04.371247+0000","last_undegraded":"2024-04-25T09:16:04.371247+0000","last_fullsized":"2024-04-25T09:16:04.371247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:02.324807+0000","last_clean_scrub_stamp":"2024-04-25T09:16:02.324807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:05.481907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"136'2","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.372347+0000","last_change":"2024-04-25T09:16:02.360788+0000","last_active":"2024-04-25T09:16:04.372347+0000","last_peered":"2024-04-25T09:16:04.372347+0000","last_clean":"2024-04-25T09:16:04.372347+0000","last_became_active":"2024-04-25T09:16:02.360329+0000","last_became_peered":"2024-04-25T09:16:02.360329+0000","last_unstale":"2024-04-25T09:16:04.372347+0000","last_undegraded":"2024-04-25T09:16:04.372347+0000","last_fullsized":"2024-04-25T09:16:04.372347+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:16:01.307675+0000","last_clean_scrub_stamp":"2024-04-25T09:16:01.307675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.590861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":385,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:16:04.433112+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:16:04.433112+0000","last_peered":"2024-04-25T09:16:04.433112+0000","last_clean":"2024-04-25T09:16:04.433112+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:16:04.433112+0000","last_undegraded":"2024-04-25T09:16:04.433112+0000","last_fullsized":"2024-04-25T09:16:04.433112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50892,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320948,"statfs":{"total":96636764160,"available":96584650752,"internally_reserved":0,"allocated":1699840,"data_stored":744697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55056,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316784,"statfs":{"total":96636764160,"available":96580386816,"internally_reserved":0,"allocated":1708032,"data_stored":750963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59812,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312028,"statfs":{"total":96636764160,"available":96575516672,"internally_reserved":0,"allocated":2314240,"data_stored":1342073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575008,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1708032,"data_stored":750963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1712128,"data_stored":751033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64792,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307048,"statfs":{"total":96636764160,"available":96570417152,"internally_reserved":0,"allocated":1716224,"data_stored":751635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55656,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316184,"statfs":{"total":96636764160,"available":96579772416,"internally_reserved":0,"allocated":2318336,"data_stored":1342965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50920,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320920,"statfs":{"total":96636764160,"available":96584622080,"internally_reserved":0,"allocated":1728512,"data_stored":752631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:16:08.332 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:16:08.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:16:08.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.608+0000 7fd7bd600640 1 Processor -- start 2024-04-25T09:16:08.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.608+0000 7fd7bd600640 1 -- start start 2024-04-25T09:16:08.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.610+0000 7fd7bd600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7b815dfd0 0x7fd7b806c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.610+0000 7fd7bd600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80b2970 con 0x7fd7b815dbf0 2024-04-25T09:16:08.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.610+0000 7fd7bd600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80b2c50 con 0x7fd7b806ce60 2024-04-25T09:16:08.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.610+0000 7fd7bd600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b80b2f30 con 0x7fd7b815dfd0 2024-04-25T09:16:08.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.611+0000 7fd7b7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd7b815dbf0 0x7fd7b8070150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57370/0 (socket says 172.21.15.62:57370) 2024-04-25T09:16:08.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.611+0000 7fd7bcc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd7b815dfd0 0x7fd7b806c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.611+0000 7fd7b7e00640 1 -- 172.21.15.62:0/2090281073 learned_addr learned my addr 172.21.15.62:0/2090281073 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:08.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.611+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2090281073 >> v1:172.21.15.121:6789/0 conn(0x7fd7b806ce60 legacy=0x7fd7b8071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.611+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2090281073 >> v1:172.21.15.62:6789/0 conn(0x7fd7b815dbf0 legacy=0x7fd7b8070150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.611+0000 7fd7bcc00640 1 -- 172.21.15.62:0/2090281073 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80cb950 con 0x7fd7b815dfd0 2024-04-25T09:16:08.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.611+0000 7fd7b7400640 1 -- 172.21.15.62:0/2090281073 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121696747 0 0) 0x7fd7b80b2970 con 0x7fd7b815dbf0 2024-04-25T09:16:08.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.612+0000 7fd7bcc00640 1 --2- 172.21.15.62:0/2090281073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dfd0 0x7fd7b806c250 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00097a0 tx=0x7fd7b002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1061817c2172344c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:08.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.613+0000 7fd7b7400640 1 -- 172.21.15.62:0/2090281073 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0037070 con 0x7fd7b815dfd0 2024-04-25T09:16:08.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.613+0000 7fd7b7400640 1 -- 172.21.15.62:0/2090281073 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b002ae50 con 0x7fd7b815dfd0 2024-04-25T09:16:08.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.613+0000 7fd7b7400640 1 -- 172.21.15.62:0/2090281073 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0032660 con 0x7fd7b815dfd0 2024-04-25T09:16:08.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.613+0000 7fd7bd600640 1 -- 172.21.15.62:0/2090281073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dfd0 msgr2=0x7fd7b806c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.613+0000 7fd7bd600640 1 --2- 172.21.15.62:0/2090281073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dfd0 0x7fd7b806c250 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00097a0 tx=0x7fd7b002a140 comp rx=0 tx=0).stop 2024-04-25T09:16:08.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.613+0000 7fd7bd600640 1 -- 172.21.15.62:0/2090281073 shutdown_connections 2024-04-25T09:16:08.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.613+0000 7fd7bd600640 1 --2- 172.21.15.62:0/2090281073 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dfd0 0x7fd7b806c250 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.613+0000 7fd7bd600640 1 -- 172.21.15.62:0/2090281073 >> 172.21.15.62:0/2090281073 conn(0x7fd7b80668f0 msgr2=0x7fd7b8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:08.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.614+0000 7fd7bd600640 1 -- 172.21.15.62:0/2090281073 shutdown_connections 2024-04-25T09:16:08.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.614+0000 7fd7bd600640 1 -- 172.21.15.62:0/2090281073 wait complete. 2024-04-25T09:16:08.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.615+0000 7fd7bd600640 1 Processor -- start 2024-04-25T09:16:08.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.616+0000 7fd7bd600640 1 -- start start 2024-04-25T09:16:08.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.616+0000 7fd7bd600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7b806ce60 0x7fd7b815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.616+0000 7fd7bd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dbf0 0x7fd7b8143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7bd600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7b815dfd0 0x7fd7b80c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7bd600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b806c910 con 0x7fd7b815dbf0 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7bd600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b806c790 con 0x7fd7b815dfd0 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7bd600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b806ca90 con 0x7fd7b806ce60 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dbf0 0x7fd7b8143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dbf0 0x7fd7b8143e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35884/0 (socket says 172.21.15.62:35884) 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7af400640 1 -- 172.21.15.62:0/169335765 learned_addr learned my addr 172.21.15.62:0/169335765 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7b7e00640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7b806ce60 0x7fd7b815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7bcc00640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7b815dfd0 0x7fd7b80c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7af400640 1 -- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7b806ce60 msgr2=0x7fd7b815ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7af400640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7b806ce60 0x7fd7b815ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7af400640 1 -- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7b815dfd0 msgr2=0x7fd7b80c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7af400640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7b815dfd0 0x7fd7b80c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7af400640 1 -- 172.21.15.62:0/169335765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b815c940 con 0x7fd7b815dbf0 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7b7e00640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7b806ce60 0x7fd7b815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.617+0000 7fd7bcc00640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7b815dfd0 0x7fd7b80c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:08.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.618+0000 7fd7af400640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dbf0 0x7fd7b8143e90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c4a0 tx=0x7fd7a000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:08.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.618+0000 7fd7b4c00640 1 -- 172.21.15.62:0/169335765 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0016020 con 0x7fd7b815dbf0 2024-04-25T09:16:08.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.618+0000 7fd7b4c00640 1 -- 172.21.15.62:0/169335765 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a0012830 con 0x7fd7b815dbf0 2024-04-25T09:16:08.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.618+0000 7fd7b4c00640 1 -- 172.21.15.62:0/169335765 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0012c20 con 0x7fd7b815dbf0 2024-04-25T09:16:08.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.618+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b815d200 con 0x7fd7b815dbf0 2024-04-25T09:16:08.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.618+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b815e750 con 0x7fd7b815dbf0 2024-04-25T09:16:08.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.620+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b8075070 con 0x7fd7b815dbf0 2024-04-25T09:16:08.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.621+0000 7fd7b4c00640 1 -- 172.21.15.62:0/169335765 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7a0019020 con 0x7fd7b815dbf0 2024-04-25T09:16:08.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.621+0000 7fd7b4c00640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd79406b4d0 0x7fd79406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:08.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.622+0000 7fd7b4c00640 1 -- 172.21.15.62:0/169335765 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7a008a350 con 0x7fd7b815dbf0 2024-04-25T09:16:08.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.622+0000 7fd7b7e00640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd79406b4d0 0x7fd79406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:08.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.623+0000 7fd7b7e00640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd79406b4d0 0x7fd79406d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80012e0 tx=0x7fd7a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:08.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.626+0000 7fd7b4c00640 1 -- 172.21.15.62:0/169335765 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7a0020020 con 0x7fd7b815dbf0 2024-04-25T09:16:08.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.895+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd7b806d7b0 con 0x7fd7b815dbf0 2024-04-25T09:16:08.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.899+0000 7fd7b4c00640 1 -- 172.21.15.62:0/169335765 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd7a00182c0 con 0x7fd7b815dbf0 2024-04-25T09:16:08.885 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:16:08.885 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:16:08.886 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:16:08.886 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:16:08.886 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:16:08.886 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:16:08.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd79406b4d0 msgr2=0x7fd79406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd79406b4d0 0x7fd79406d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80012e0 tx=0x7fd7a8009040 comp rx=0 tx=0).stop 2024-04-25T09:16:08.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dbf0 msgr2=0x7fd7b8143e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:08.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dbf0 0x7fd7b8143e90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000c4a0 tx=0x7fd7a000c960 comp rx=0 tx=0).stop 2024-04-25T09:16:08.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 shutdown_connections 2024-04-25T09:16:08.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd79406b4d0 0x7fd79406d990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd7b815dbf0 0x7fd7b8143e90 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7b815dfd0 0x7fd7b80c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 --2- 172.21.15.62:0/169335765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd7b806ce60 0x7fd7b815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:08.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 >> 172.21.15.62:0/169335765 conn(0x7fd7b80668f0 msgr2=0x7fd7b80cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:08.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 shutdown_connections 2024-04-25T09:16:08.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:08.902+0000 7fd7bd600640 1 -- 172.21.15.62:0/169335765 wait complete. 2024-04-25T09:16:08.908 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:09.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.135+0000 7f3b4d800640 1 Processor -- start 2024-04-25T09:16:09.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.136+0000 7f3b4d800640 1 -- start start 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.137+0000 7f3b4d800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b6a90 0x7f3b480d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.137+0000 7f3b4d800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b480b27a0 0x7f3b480d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4d800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b480d3110 con 0x7f3b480b56f0 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4d800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b480cedc0 con 0x7f3b480b27a0 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4d800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480d33f0 con 0x7f3b480b6a90 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b480b27a0 0x7f3b480d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3b480b27a0 0x7f3b480d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35894/0 (socket says 172.21.15.62:35894) 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4ce00640 1 -- 172.21.15.62:0/532673741 learned_addr learned my addr 172.21.15.62:0/532673741 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b47400640 1 --2- 172.21.15.62:0/532673741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b6a90 0x7f3b480d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:09.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4ce00640 1 -- 172.21.15.62:0/532673741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b6a90 msgr2=0x7f3b480d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:09.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4ce00640 1 --2- 172.21.15.62:0/532673741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b6a90 0x7f3b480d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.138+0000 7f3b4ce00640 1 -- 172.21.15.62:0/532673741 >> v1:172.21.15.121:6789/0 conn(0x7f3b480b56f0 legacy=0x7f3b48160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:09.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.139+0000 7f3b4ce00640 1 -- 172.21.15.62:0/532673741 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480d36d0 con 0x7f3b480b27a0 2024-04-25T09:16:09.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.140+0000 7f3b4ce00640 1 --2- 172.21.15.62:0/532673741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b27a0 0x7f3b480d2520 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009e50 tx=0x7f3b4002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23d7dfefd42acbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:09.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.140+0000 7f3b46a00640 1 -- 172.21.15.62:0/532673741 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40036070 con 0x7f3b480b27a0 2024-04-25T09:16:09.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.140+0000 7f3b46a00640 1 -- 172.21.15.62:0/532673741 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b40031600 con 0x7f3b480b27a0 2024-04-25T09:16:09.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.140+0000 7f3b46a00640 1 -- 172.21.15.62:0/532673741 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b400319d0 con 0x7f3b480b27a0 2024-04-25T09:16:09.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.141+0000 7f3b4d800640 1 -- 172.21.15.62:0/532673741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b27a0 msgr2=0x7f3b480d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.141+0000 7f3b4d800640 1 --2- 172.21.15.62:0/532673741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b27a0 0x7f3b480d2520 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009e50 tx=0x7f3b4002a6d0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.141+0000 7f3b4d800640 1 -- 172.21.15.62:0/532673741 shutdown_connections 2024-04-25T09:16:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.141+0000 7f3b4d800640 1 --2- 172.21.15.62:0/532673741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b27a0 0x7f3b480d2520 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.141+0000 7f3b4d800640 1 --2- 172.21.15.62:0/532673741 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b6a90 0x7f3b480d1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.141+0000 7f3b4d800640 1 -- 172.21.15.62:0/532673741 >> 172.21.15.62:0/532673741 conn(0x7f3b48066d10 msgr2=0x7f3b48067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.141+0000 7f3b4d800640 1 -- 172.21.15.62:0/532673741 shutdown_connections 2024-04-25T09:16:09.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.141+0000 7f3b4d800640 1 -- 172.21.15.62:0/532673741 wait complete. 2024-04-25T09:16:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.142+0000 7f3b4d800640 1 Processor -- start 2024-04-25T09:16:09.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.143+0000 7f3b4d800640 1 -- start start 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b4d800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b27a0 0x7f3b4815d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b4d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b56f0 0x7f3b4814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b4d800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b480b6a90 0x7f3b481f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b4d800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480b2ee0 con 0x7f3b480b56f0 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b4d800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480b2d60 con 0x7f3b480b6a90 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b4d800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480b5ad0 con 0x7f3b480b27a0 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b56f0 0x7f3b4814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b56f0 0x7f3b4814edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35902/0 (socket says 172.21.15.62:35902) 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47400640 1 -- 172.21.15.62:0/2490021781 learned_addr learned my addr 172.21.15.62:0/2490021781 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:09.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47e00640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b27a0 0x7f3b4815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b4ce00640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b480b6a90 0x7f3b481f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47400640 1 -- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b27a0 msgr2=0x7f3b4815d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47400640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b27a0 0x7f3b4815d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47400640 1 -- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b480b6a90 msgr2=0x7f3b481f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47400640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b480b6a90 0x7f3b481f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.144+0000 7f3b47400640 1 -- 172.21.15.62:0/2490021781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b481f42b0 con 0x7f3b480b56f0 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.145+0000 7f3b47400640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b56f0 0x7f3b4814edc0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000d5f0 tx=0x7f3b3000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.145+0000 7f3b3fe00640 1 -- 172.21.15.62:0/2490021781 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b300125f0 con 0x7f3b480b56f0 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.145+0000 7f3b3fe00640 1 -- 172.21.15.62:0/2490021781 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b30019070 con 0x7f3b480b56f0 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.145+0000 7f3b3fe00640 1 -- 172.21.15.62:0/2490021781 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b300129d0 con 0x7f3b480b56f0 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.145+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b481f4570 con 0x7f3b480b56f0 2024-04-25T09:16:09.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.145+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b48159100 con 0x7f3b480b56f0 2024-04-25T09:16:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.146+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4804ef20 con 0x7f3b480b56f0 2024-04-25T09:16:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.147+0000 7f3b3fe00640 1 -- 172.21.15.62:0/2490021781 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b30012b70 con 0x7f3b480b56f0 2024-04-25T09:16:09.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.149+0000 7f3b3fe00640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2406b4d0 0x7f3b2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.149+0000 7f3b3fe00640 1 -- 172.21.15.62:0/2490021781 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b3008a600 con 0x7f3b480b56f0 2024-04-25T09:16:09.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.149+0000 7f3b47e00640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2406b4d0 0x7f3b2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:09.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.151+0000 7f3b47e00640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2406b4d0 0x7f3b2406d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3b38004ea0 tx=0x7f3b38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:09.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.153+0000 7f3b3fe00640 1 -- 172.21.15.62:0/2490021781 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b30059470 con 0x7f3b480b56f0 2024-04-25T09:16:09.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.394+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b480ce580 con 0x7f3b480b56f0 2024-04-25T09:16:09.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.395+0000 7f3b3fe00640 1 -- 172.21.15.62:0/2490021781 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3b30058d70 con 0x7f3b480b56f0 2024-04-25T09:16:09.381 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:09.382 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:09.382 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:09.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2406b4d0 msgr2=0x7f3b2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2406b4d0 0x7f3b2406d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3b38004ea0 tx=0x7f3b38009040 comp rx=0 tx=0).stop 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b56f0 msgr2=0x7f3b4814edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b56f0 0x7f3b4814edc0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000d5f0 tx=0x7f3b3000dab0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 shutdown_connections 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f3b2406b4d0 0x7f3b2406d990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3b480b56f0 0x7f3b4814edc0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b480b6a90 0x7f3b481f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 --2- 172.21.15.62:0/2490021781 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f3b480b27a0 0x7f3b4815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 >> 172.21.15.62:0/2490021781 conn(0x7f3b48066d10 msgr2=0x7f3b480b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 shutdown_connections 2024-04-25T09:16:09.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:09.398+0000 7f3b4d800640 1 -- 172.21.15.62:0/2490021781 wait complete. 2024-04-25T09:16:09.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:16:02.818081+0000', 'modified': '2024-04-25T09:16:05.890185+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8886}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8886': {'gid': 8886, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6834/2303077743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 2303077743}, {'type': 'v1', 'addr': '172.21.15.62:6834', 'nonce': 2303077743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:16:09.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:16:09.806 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:16:10.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.040+0000 7f8fbda00640 1 Processor -- start 2024-04-25T09:16:10.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.041+0000 7f8fbda00640 1 -- start start 2024-04-25T09:16:10.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbda00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8fb8161230 0x7f8fb80b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbda00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb80d0fb0 0x7f8fb80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbda00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8fb80b5a60 0x7f8fb80d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbda00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb80b5ec0 con 0x7f8fb80b5a60 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbda00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb80d2040 con 0x7f8fb8161230 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbda00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb80cea70 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbd000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8fb80b5a60 0x7f8fb80d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbd000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8fb80b5a60 0x7f8fb80d2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35912/0 (socket says 172.21.15.62:35912) 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbd000640 1 -- 172.21.15.62:0/366888872 learned_addr learned my addr 172.21.15.62:0/366888872 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbd000640 1 -- 172.21.15.62:0/366888872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb80d0fb0 msgr2=0x7f8fb80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbd000640 1 --2- 172.21.15.62:0/366888872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb80d0fb0 0x7f8fb80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbd000640 1 -- 172.21.15.62:0/366888872 >> v2:172.21.15.121:3300/0 conn(0x7f8fb8161230 msgr2=0x7f8fb80b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/366888872 >> v2:172.21.15.121:3300/0 conn(0x7f8fb8161230 0x7f8fb80b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbd000640 1 --2- 172.21.15.62:0/366888872 >> v2:172.21.15.121:3300/0 conn(0x7f8fb8161230 0x7f8fb80b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.043+0000 7f8fbd000640 1 -- 172.21.15.62:0/366888872 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb80ced50 con 0x7f8fb80b5a60 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.044+0000 7f8fbd000640 1 --2- 172.21.15.62:0/366888872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80b5a60 0x7f8fb80d2a90 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8fb808cf60 tx=0x7f8fb002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d8c436bd1134f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fb6a00640 1 -- 172.21.15.62:0/366888872 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0037070 con 0x7f8fb80b5a60 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fb6a00640 1 -- 172.21.15.62:0/366888872 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0032440 con 0x7f8fb80b5a60 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fb6a00640 1 -- 172.21.15.62:0/366888872 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0032810 con 0x7f8fb80b5a60 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fbda00640 1 -- 172.21.15.62:0/366888872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80b5a60 msgr2=0x7f8fb80d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fbda00640 1 --2- 172.21.15.62:0/366888872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80b5a60 0x7f8fb80d2a90 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8fb808cf60 tx=0x7f8fb002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fbda00640 1 -- 172.21.15.62:0/366888872 shutdown_connections 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fbda00640 1 --2- 172.21.15.62:0/366888872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80b5a60 0x7f8fb80d2a90 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fbda00640 1 --2- 172.21.15.62:0/366888872 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb80d0fb0 0x7f8fb80b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fbda00640 1 --2- 172.21.15.62:0/366888872 >> v2:172.21.15.121:3300/0 conn(0x7f8fb8161230 0x7f8fb80b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.045+0000 7f8fbda00640 1 -- 172.21.15.62:0/366888872 >> 172.21.15.62:0/366888872 conn(0x7f8fb8066d10 msgr2=0x7f8fb8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:10.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.046+0000 7f8fbda00640 1 -- 172.21.15.62:0/366888872 shutdown_connections 2024-04-25T09:16:10.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.046+0000 7f8fbda00640 1 -- 172.21.15.62:0/366888872 wait complete. 2024-04-25T09:16:10.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.047+0000 7f8fbda00640 1 Processor -- start 2024-04-25T09:16:10.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.047+0000 7f8fbda00640 1 -- start start 2024-04-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.048+0000 7f8fbda00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb80b5a60 0x7f8fb80ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.048+0000 7f8fbda00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80d0fb0 0x7f8fb814ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.048+0000 7f8fbda00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb8161230 0x7f8fb80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.048+0000 7f8fbda00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb80ca8d0 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.048+0000 7f8fbda00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb80d2290 con 0x7f8fb80b5a60 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.048+0000 7f8fbda00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8161610 con 0x7f8fb8161230 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80d0fb0 0x7f8fb814ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80d0fb0 0x7f8fb814ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35924/0 (socket says 172.21.15.62:35924) 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7400640 1 -- 172.21.15.62:0/3777118357 learned_addr learned my addr 172.21.15.62:0/3777118357 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb80b5a60 0x7f8fb80ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fbd000640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb8161230 0x7f8fb80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7400640 1 -- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb8161230 msgr2=0x7f8fb80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7400640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb8161230 0x7f8fb80cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7400640 1 -- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb80b5a60 msgr2=0x7f8fb80ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7400640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb80b5a60 0x7f8fb80ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.049+0000 7f8fb7400640 1 -- 172.21.15.62:0/3777118357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb80aa530 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.050+0000 7f8fb7400640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80d0fb0 0x7f8fb814ee60 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000b4a0 tx=0x7f8fa000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.050+0000 7f8fafe00640 1 -- 172.21.15.62:0/3777118357 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0014070 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.050+0000 7f8fafe00640 1 -- 172.21.15.62:0/3777118357 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa0010610 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.050+0000 7f8fafe00640 1 -- 172.21.15.62:0/3777118357 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0010a00 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.050+0000 7f8fbda00640 1 -- 172.21.15.62:0/3777118357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb80aa810 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.050+0000 7f8fbda00640 1 -- 172.21.15.62:0/3777118357 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb80aadd0 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.052+0000 7f8fafe00640 1 -- 172.21.15.62:0/3777118357 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fa0017070 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.053+0000 7f8fafe00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f9406b4d0 0x7f8f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.053+0000 7f8fafe00640 1 -- 172.21.15.62:0/3777118357 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fa0089cc0 con 0x7f8fb80d0fb0 2024-04-25T09:16:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.054+0000 7f8fc1dde740 1 Processor -- start 2024-04-25T09:16:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.054+0000 7f8fc1dde740 1 -- start start 2024-04-25T09:16:10.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.054+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f9406b4d0 0x7f8f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72bb8b30 0x55cd72be0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72be0eb0 0x55cd72bb8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72c017a0 0x55cd72bb72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd72b1c620 con 0x55cd72be0eb0 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd72ac5ca0 con 0x55cd72bb8b30 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd72b186c0 con 0x55cd72c017a0 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f9406b4d0 0x7f8f9406d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80070b0 tx=0x7f8fa8007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72be0eb0 0x55cd72bb8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72be0eb0 0x55cd72bb8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35930/0 (socket says 172.21.15.62:35930) 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.055+0000 7f8fb7400640 1 -- 172.21.15.62:0/1246094543 learned_addr learned my addr 172.21.15.62:0/1246094543 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72bb8b30 0x55cd72be0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8fbd000640 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72c017a0 0x55cd72bb72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8fb7400640 1 -- 172.21.15.62:0/1246094543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72c017a0 msgr2=0x55cd72bb72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8fb7400640 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72c017a0 0x55cd72bb72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8fb7400640 1 -- 172.21.15.62:0/1246094543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72bb8b30 msgr2=0x55cd72be0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8fb7400640 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72bb8b30 0x55cd72be0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8fb7400640 1 -- 172.21.15.62:0/1246094543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd72accf40 con 0x55cd72be0eb0 2024-04-25T09:16:10.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8fb7400640 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72be0eb0 0x55cd72bb8f10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0060580 tx=0x7f8fa000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ed3c5eb47b88b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8facc00640 1 -- 172.21.15.62:0/1246094543 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa001e040 con 0x55cd72be0eb0 2024-04-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8facc00640 1 -- 172.21.15.62:0/1246094543 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa0054a10 con 0x55cd72be0eb0 2024-04-25T09:16:10.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.056+0000 7f8facc00640 1 -- 172.21.15.62:0/1246094543 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0010ba0 con 0x55cd72be0eb0 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 -- 172.21.15.62:0/1246094543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72be0eb0 msgr2=0x55cd72bb8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72be0eb0 0x55cd72bb8f10 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0060580 tx=0x7f8fa000bf90 comp rx=0 tx=0).stop 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 -- 172.21.15.62:0/1246094543 shutdown_connections 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72be0eb0 0x55cd72bb8f10 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72c017a0 0x55cd72bb72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/1246094543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72bb8b30 0x55cd72be0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 -- 172.21.15.62:0/1246094543 >> 172.21.15.62:0/1246094543 conn(0x55cd72bb5d50 msgr2=0x55cd72bfe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 -- 172.21.15.62:0/1246094543 shutdown_connections 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.057+0000 7f8fc1dde740 1 -- 172.21.15.62:0/1246094543 wait complete. 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.058+0000 7f8fc1dde740 1 -- start start 2024-04-25T09:16:10.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.058+0000 7f8fc1dde740 1 Processor -- start 2024-04-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.059+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 0x55cd72bcfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.059+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.059+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 0x55cd72bdd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.059+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd72be04e0 con 0x55cd72c017a0 2024-04-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.059+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd72abb1f0 con 0x55cd72be0eb0 2024-04-25T09:16:10.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.059+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd72bdfed0 con 0x55cd72be70d0 2024-04-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.059+0000 7f8fc1dde740 10 client.0 _ll_register_callbacks cb 0x55cd72ad8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35938/0 (socket says 172.21.15.62:35938) 2024-04-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 -- 172.21.15.62:0/295873946 learned_addr learned my addr 172.21.15.62:0/295873946 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 -- 172.21.15.62:0/295873946 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 msgr2=0x55cd72bdd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 0x55cd72bdd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 -- 172.21.15.62:0/295873946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 msgr2=0x55cd72bcfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 0x55cd72bcfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 -- 172.21.15.62:0/295873946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd72adac50 con 0x55cd72c017a0 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fb7400640 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be6b90 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f8fa006f040 tx=0x7f8fa0088020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2945497462d62c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fa5600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fa5600640 1 -- 172.21.15.62:0/295873946 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa001e070 con 0x55cd72c017a0 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fa5600640 1 -- 172.21.15.62:0/295873946 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fa0010da0 con 0x55cd72c017a0 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.060+0000 7f8fc1dde740 1 -- 172.21.15.62:0/295873946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fa0054a10 con 0x55cd72c017a0 2024-04-25T09:16:10.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fa5600640 1 -- 172.21.15.62:0/295873946 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fa0087070 con 0x55cd72c017a0 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fc1dde740 10 client.9027 fetch_fsmap learned FSMap version 86 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fc1dde740 1 -- 172.21.15.62:0/295873946 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55cd728312d0 con 0x55cd72c017a0 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fa5600640 1 -- 172.21.15.62:0/295873946 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa0054a10 con 0x55cd72c017a0 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fa5600640 1 -- 172.21.15.62:0/295873946 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8fa00577f0 con 0x55cd72c017a0 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fc1dde740 10 client.9027 fetch_fsmap finished waiting for FSMap version 86 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fc1dde740 10 client.9027 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fc1dde740 20 client.9027 populate_metadata read hostname 'smithi062' 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55cd72be2130 0x55cd72be2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fc1dde740 4 client.9027 mds_command: new command op to 8886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.061+0000 7f8fc1dde740 1 -- 172.21.15.62:0/295873946 --> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd72b29b60 con 0x55cd72be2130 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.062+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55cd72be2130 0x55cd72be2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.062+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55cd72be2130 0x55cd72be2510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef6aa474c728fd4 server_cookie=403871ff72a7b92e in_seq=0 out_seq=0 2024-04-25T09:16:10.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.062+0000 7f8fa5600640 10 client.9027 ms_handle_connect on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fa5600640 1 -- 172.21.15.62:0/295873946 <== mds.0 v2:172.21.15.62:6832/2303077743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd72b29b60 con 0x55cd72be2130 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fa5600640 10 client.9027 handle_command_reply: tid=0 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 client.9027 shutdown 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 -- 172.21.15.62:0/295873946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 msgr2=0x55cd72be6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be6b90 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f8fa006f040 tx=0x7f8fa0088020 comp rx=0 tx=0).stop 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 -- 172.21.15.62:0/295873946 shutdown_connections 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55cd72be2130 0x55cd72be2510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be6b90 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 0x55cd72bdd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/295873946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 0x55cd72bcfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fa5600640 0 client.9027 ms_handle_reset on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 -- 172.21.15.62:0/295873946 >> 172.21.15.62:0/295873946 conn(0x55cd72bb8b30 msgr2=0x55cd72c01c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 -- 172.21.15.62:0/295873946 shutdown_connections 2024-04-25T09:16:10.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.063+0000 7f8fc1dde740 1 -- 172.21.15.62:0/295873946 wait complete. 2024-04-25T09:16:10.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.064+0000 7f8fc1dde740 20 client.9027 trim_cache size 0 max 16384 2024-04-25T09:16:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.081+0000 7f8fc1dde740 1 Processor -- start 2024-04-25T09:16:10.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.081+0000 7f8fc1dde740 1 -- start start 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be0eb0 0x55cd72c62ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be70d0 0x55cd72c01b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72c212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb80c9030 con 0x55cd72c017a0 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb80c8430 con 0x55cd72be70d0 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb80c8a30 con 0x55cd72be0eb0 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fbd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72c212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fbd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72c212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35942/0 (socket says 172.21.15.62:35942) 2024-04-25T09:16:10.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fbd000640 1 -- 172.21.15.62:0/3794867449 learned_addr learned my addr 172.21.15.62:0/3794867449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be0eb0 0x55cd72c62ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fb7400640 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be70d0 0x55cd72c01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fbd000640 1 -- 172.21.15.62:0/3794867449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be0eb0 msgr2=0x55cd72c62ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fbd000640 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be0eb0 0x55cd72c62ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fbd000640 1 -- 172.21.15.62:0/3794867449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be70d0 msgr2=0x55cd72c01b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.082+0000 7f8fbd000640 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be70d0 0x55cd72c01b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.083+0000 7f8fbd000640 1 -- 172.21.15.62:0/3794867449 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd72be6f00 con 0x55cd72c017a0 2024-04-25T09:16:10.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.083+0000 7f8fbd000640 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72c212c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f8fb002a220 tx=0x7f8fb002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988624c3bb9954a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.083+0000 7f8fa7e00640 1 -- 172.21.15.62:0/3794867449 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0037040 con 0x55cd72c017a0 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.083+0000 7f8fa7e00640 1 -- 172.21.15.62:0/3794867449 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0032640 con 0x55cd72c017a0 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.083+0000 7f8fa7e00640 1 -- 172.21.15.62:0/3794867449 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0032a30 con 0x55cd72c017a0 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 -- 172.21.15.62:0/3794867449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 msgr2=0x55cd72c212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72c212c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f8fb002a220 tx=0x7f8fb002c070 comp rx=0 tx=0).stop 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 -- 172.21.15.62:0/3794867449 shutdown_connections 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72c212c0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be70d0 0x55cd72c01b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/3794867449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be0eb0 0x55cd72c62ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 -- 172.21.15.62:0/3794867449 >> 172.21.15.62:0/3794867449 conn(0x55cd72bb8b30 msgr2=0x55cd72be60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 -- 172.21.15.62:0/3794867449 shutdown_connections 2024-04-25T09:16:10.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.084+0000 7f8fc1dde740 1 -- 172.21.15.62:0/3794867449 wait complete. 2024-04-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.085+0000 7f8fc1dde740 1 -- start start 2024-04-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.085+0000 7f8fc1dde740 1 Processor -- start 2024-04-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 0x55cd72c5d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 0x55cd72be46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fc1dde740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd72be04e0 con 0x55cd72c017a0 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd72abb1f0 con 0x55cd72be0eb0 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fc1dde740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd72c11f60 con 0x55cd72be70d0 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fc1dde740 10 client.0 _ll_register_callbacks cb 0x7f8fa0058430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fbd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fbd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35950/0 (socket says 172.21.15.62:35950) 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.086+0000 7f8fbd000640 1 -- 172.21.15.62:0/2624810984 learned_addr learned my addr 172.21.15.62:0/2624810984 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fbd000640 1 -- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 msgr2=0x55cd72be46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 0x55cd72c5d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fb7400640 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 0x55cd72be46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fbd000640 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 0x55cd72be46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fbd000640 1 -- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 msgr2=0x55cd72c5d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fbd000640 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 0x55cd72c5d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fbd000640 1 -- 172.21.15.62:0/2624810984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd728fb6b0 con 0x55cd72c017a0 2024-04-25T09:16:10.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 0x55cd72c5d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fbd000640 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be4c20 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8fb002a7d0 tx=0x7f8fb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43121dcba8bad80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fa5600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fa5600640 1 -- 172.21.15.62:0/2624810984 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0037030 con 0x55cd72c017a0 2024-04-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fa5600640 1 -- 172.21.15.62:0/2624810984 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fb0032c00 con 0x55cd72c017a0 2024-04-25T09:16:10.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.087+0000 7f8fc1dde740 1 -- 172.21.15.62:0/2624810984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fb0032640 con 0x55cd72c017a0 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fa5600640 1 -- 172.21.15.62:0/2624810984 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fb0032e20 con 0x55cd72c017a0 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fc1dde740 10 client.9033 fetch_fsmap learned FSMap version 86 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fc1dde740 1 -- 172.21.15.62:0/2624810984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55cd72bdd120 con 0x55cd72c017a0 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fa5600640 1 -- 172.21.15.62:0/2624810984 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0032640 con 0x55cd72c017a0 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fa5600640 1 -- 172.21.15.62:0/2624810984 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8fb0005990 con 0x55cd72c017a0 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fc1dde740 10 client.9033 fetch_fsmap finished waiting for FSMap version 86 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fc1dde740 10 client.9033 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fc1dde740 20 client.9033 populate_metadata read hostname 'smithi062' 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55cd72be3d20 0x55cd72be4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fc1dde740 4 client.9033 mds_command: new command op to 8886 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:16:10.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fc1dde740 1 -- 172.21.15.62:0/2624810984 --> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cd72b23eb0 con 0x55cd72be3d20 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.088+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55cd72be3d20 0x55cd72be4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fb7e00640 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55cd72be3d20 0x55cd72be4100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19ada4a3c771df66 server_cookie=abaecbc092cd2470 in_seq=0 out_seq=0 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fa5600640 10 client.9033 ms_handle_connect on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fa5600640 1 -- 172.21.15.62:0/2624810984 <== mds.0 v2:172.21.15.62:6832/2303077743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55cd72b23eb0 con 0x55cd72be3d20 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fa5600640 10 client.9033 handle_command_reply: tid=0 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 client.9033 shutdown 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 -- 172.21.15.62:0/2624810984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 msgr2=0x55cd72be4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be4c20 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8fb002a7d0 tx=0x7f8fb0004730 comp rx=0 tx=0).stop 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 -- 172.21.15.62:0/2624810984 shutdown_connections 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55cd72be3d20 0x55cd72be4100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55cd72c017a0 0x55cd72be4c20 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55cd72be70d0 0x55cd72be46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 --2- 172.21.15.62:0/2624810984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd72be0eb0 0x55cd72c5d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fa5600640 0 client.9033 ms_handle_reset on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.089+0000 7f8fc1dde740 1 -- 172.21.15.62:0/2624810984 >> 172.21.15.62:0/2624810984 conn(0x55cd72bb8b30 msgr2=0x55cd72c21250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.090+0000 7f8fc1dde740 1 -- 172.21.15.62:0/2624810984 shutdown_connections 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.090+0000 7f8fc1dde740 1 -- 172.21.15.62:0/2624810984 wait complete. 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.090+0000 7f8fc1dde740 20 client.9033 trim_cache size 0 max 16384 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 8886, 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:16:10.079 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 6.7443475560000001, 2024-04-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 86, 2024-04-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 137, 2024-04-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 137, 2024-04-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 13.140068186000001 2024-04-25T09:16:10.080 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.090+0000 7f8fbda00640 1 -- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f9406b4d0 msgr2=0x7f8f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.090+0000 7f8fbda00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f9406b4d0 0x7f8f9406d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80070b0 tx=0x7f8fa8007040 comp rx=0 tx=0).stop 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.090+0000 7f8fbda00640 1 -- 172.21.15.62:0/3777118357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80d0fb0 msgr2=0x7f8fb814ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.090+0000 7f8fbda00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80d0fb0 0x7f8fb814ee60 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000b4a0 tx=0x7f8fa000b960 comp rx=0 tx=0).stop 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.091+0000 7f8fbda00640 1 -- 172.21.15.62:0/3777118357 shutdown_connections 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.091+0000 7f8fbda00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8f9406b4d0 0x7f8f9406d990 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.091+0000 7f8fbda00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8fb80d0fb0 0x7f8fb814ee60 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.091+0000 7f8fbda00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8fb8161230 0x7f8fb80cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.091+0000 7f8fbda00640 1 --2- 172.21.15.62:0/3777118357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb80b5a60 0x7f8fb80ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.091+0000 7f8fbda00640 1 -- 172.21.15.62:0/3777118357 >> 172.21.15.62:0/3777118357 conn(0x7f8fb8066d10 msgr2=0x7f8fb80b8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:10.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.091+0000 7f8fbda00640 1 -- 172.21.15.62:0/3777118357 shutdown_connections 2024-04-25T09:16:10.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.091+0000 7f8fbda00640 1 -- 172.21.15.62:0/3777118357 wait complete. 2024-04-25T09:16:10.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:16:10.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.313+0000 7f149f600640 1 Processor -- start 2024-04-25T09:16:10.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.313+0000 7f149f600640 1 -- start start 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.315+0000 7f149f600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f14980cccf0 0x7f14980ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.315+0000 7f149f600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14980aed00 0x7f14980cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.315+0000 7f149f600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149806ca70 con 0x7f149815bd10 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.315+0000 7f149f600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149806d660 con 0x7f14980cccf0 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.315+0000 7f149f600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14980af140 con 0x7f14980aed00 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.315+0000 7f149cc00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f149815bd10 0x7f14980cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57386/0 (socket says 172.21.15.62:57386) 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.315+0000 7f149cc00640 1 -- 172.21.15.62:0/4148064029 learned_addr learned my addr 172.21.15.62:0/4148064029 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.315+0000 7f1497e00640 1 --2- 172.21.15.62:0/4148064029 >> v2:172.21.15.62:3300/0 conn(0x7f14980cccf0 0x7f14980ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.316+0000 7f149d600640 1 --2- 172.21.15.62:0/4148064029 >> v2:172.21.15.121:3300/0 conn(0x7f14980aed00 0x7f14980cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.316+0000 7f1497400640 1 -- 172.21.15.62:0/4148064029 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138969612 0 0) 0x7f149806ca70 con 0x7f149815bd10 2024-04-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.316+0000 7f1497400640 1 -- 172.21.15.62:0/4148064029 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f147c003520 con 0x7f149815bd10 2024-04-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.316+0000 7f1497e00640 1 -- 172.21.15.62:0/4148064029 >> v2:172.21.15.121:3300/0 conn(0x7f14980aed00 msgr2=0x7f14980cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.316+0000 7f1497e00640 1 --2- 172.21.15.62:0/4148064029 >> v2:172.21.15.121:3300/0 conn(0x7f14980aed00 0x7f14980cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.316+0000 7f1497e00640 1 -- 172.21.15.62:0/4148064029 >> v1:172.21.15.62:6789/0 conn(0x7f149815bd10 legacy=0x7f14980cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.316+0000 7f1497e00640 1 -- 172.21.15.62:0/4148064029 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149815c0f0 con 0x7f14980cccf0 2024-04-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.317+0000 7f1497e00640 1 --2- 172.21.15.62:0/4148064029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14980cccf0 0x7f14980ae7c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f149806d7e0 tx=0x7f148402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105f378474ef2c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.317+0000 7f1497400640 1 -- 172.21.15.62:0/4148064029 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484007740 con 0x7f14980cccf0 2024-04-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.317+0000 7f1497400640 1 -- 172.21.15.62:0/4148064029 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14840078e0 con 0x7f14980cccf0 2024-04-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.317+0000 7f1497400640 1 -- 172.21.15.62:0/4148064029 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484007cb0 con 0x7f14980cccf0 2024-04-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.318+0000 7f149f600640 1 -- 172.21.15.62:0/4148064029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14980cccf0 msgr2=0x7f14980ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.318+0000 7f149f600640 1 --2- 172.21.15.62:0/4148064029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14980cccf0 0x7f14980ae7c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f149806d7e0 tx=0x7f148402a660 comp rx=0 tx=0).stop 2024-04-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.318+0000 7f149f600640 1 -- 172.21.15.62:0/4148064029 shutdown_connections 2024-04-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.318+0000 7f149f600640 1 --2- 172.21.15.62:0/4148064029 >> v2:172.21.15.121:3300/0 conn(0x7f14980aed00 0x7f14980cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.318+0000 7f149f600640 1 --2- 172.21.15.62:0/4148064029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14980cccf0 0x7f14980ae7c0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.318+0000 7f149f600640 1 -- 172.21.15.62:0/4148064029 >> 172.21.15.62:0/4148064029 conn(0x7f14980668f0 msgr2=0x7f1498066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.318+0000 7f149f600640 1 -- 172.21.15.62:0/4148064029 shutdown_connections 2024-04-25T09:16:10.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.318+0000 7f149f600640 1 -- 172.21.15.62:0/4148064029 wait complete. 2024-04-25T09:16:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.320+0000 7f149f600640 1 Processor -- start 2024-04-25T09:16:10.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.320+0000 7f149f600640 1 -- start start 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149f600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14980aed00 0x7f14980afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149f600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14980cccf0 0x7f14981e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149f600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f149815bd10 0x7f1498071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149f600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149806d2f0 con 0x7f149815bd10 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149f600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149806d010 con 0x7f14980cccf0 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149f600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149806daf0 con 0x7f14980aed00 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f149815bd10 0x7f1498071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149d600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f149815bd10 0x7f1498071ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35966/0 (socket says 172.21.15.62:35966) 2024-04-25T09:16:10.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.321+0000 7f149d600640 1 -- 172.21.15.62:0/1780125220 learned_addr learned my addr 172.21.15.62:0/1780125220 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f1497e00640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14980cccf0 0x7f14981e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f149cc00640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14980aed00 0x7f14980afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f149d600640 1 -- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14980aed00 msgr2=0x7f14980afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f149d600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14980aed00 0x7f14980afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f149d600640 1 -- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14980cccf0 msgr2=0x7f14981e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f149d600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14980cccf0 0x7f14981e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f149d600640 1 -- 172.21.15.62:0/1780125220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980705e0 con 0x7f149815bd10 2024-04-25T09:16:10.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f149d600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f149815bd10 0x7f1498071ac0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f149000d800 tx=0x7f149000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f1494c00640 1 -- 172.21.15.62:0/1780125220 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000b6c0 con 0x7f149815bd10 2024-04-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f1494c00640 1 -- 172.21.15.62:0/1780125220 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1490019070 con 0x7f149815bd10 2024-04-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f1494c00640 1 -- 172.21.15.62:0/1780125220 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000bac0 con 0x7f149815bd10 2024-04-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.322+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14980708c0 con 0x7f149815bd10 2024-04-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.323+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1498071160 con 0x7f149815bd10 2024-04-25T09:16:10.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.324+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1498070cb0 con 0x7f149815bd10 2024-04-25T09:16:10.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.325+0000 7f1494c00640 1 -- 172.21.15.62:0/1780125220 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f149000bc60 con 0x7f149815bd10 2024-04-25T09:16:10.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.326+0000 7f1494c00640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147806b4d0 0x7f147806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:10.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.326+0000 7f1494c00640 1 -- 172.21.15.62:0/1780125220 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f149008a6a0 con 0x7f149815bd10 2024-04-25T09:16:10.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.327+0000 7f149cc00640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147806b4d0 0x7f147806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:10.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.328+0000 7f149cc00640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147806b4d0 0x7f147806d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1488000f60 tx=0x7f1488009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:10.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.330+0000 7f1494c00640 1 -- 172.21.15.62:0/1780125220 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1490059510 con 0x7f149815bd10 2024-04-25T09:16:10.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.578+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1498072c70 con 0x7f149815bd10 2024-04-25T09:16:10.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.579+0000 7f1494c00640 1 -- 172.21.15.62:0/1780125220 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1490058e10 con 0x7f149815bd10 2024-04-25T09:16:10.565 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:16:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.581+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147806b4d0 msgr2=0x7f147806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.581+0000 7f149f600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147806b4d0 0x7f147806d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1488000f60 tx=0x7f1488009040 comp rx=0 tx=0).stop 2024-04-25T09:16:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f149815bd10 msgr2=0x7f1498071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f149815bd10 0x7f1498071ac0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f149000d800 tx=0x7f149000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 shutdown_connections 2024-04-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f147806b4d0 0x7f147806d990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f149815bd10 0x7f1498071ac0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14980cccf0 0x7f14981e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 --2- 172.21.15.62:0/1780125220 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14980aed00 0x7f14980afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 >> 172.21.15.62:0/1780125220 conn(0x7f14980668f0 msgr2=0x7f14980cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 shutdown_connections 2024-04-25T09:16:10.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:10.582+0000 7f149f600640 1 -- 172.21.15.62:0/1780125220 wait complete. 2024-04-25T09:16:10.586 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:16:10.586 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:16:10.586 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:16:10.586 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:16:10.586 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:16:10.587 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:16:10.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:16:10.587 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:16:10.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:16:10.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:10.587 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:16:10.637 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41888sec preferred_lft 41888sec 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:16:10.638 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:16:10.639 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:16:10.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:10.641 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:16:10.722 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:16:10.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:10.726 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:16:10.782 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:16:10.783 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:16:10.783 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:16:10.783 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:16:10.783 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:16:10.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:10.784 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:16:10.784 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:10.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:16:10.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:16:10.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:16:10.784 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:10.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:10.950 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:16:11.003 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:16:11.003 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:16:11.004 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:16:11.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:11.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:16:11.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:16:11.092 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:16:11.151 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:16:11.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:16:11.151 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:16:11.205 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:16:11.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:11.206 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:16:11.337 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:16:11.337 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:16:11.376 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:16:11.381 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:16:11.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:11.381 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:16:11.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:11.463 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:16:11.535 INFO:teuthology.orchestra.run.smithi062.stdout:client9045 2024-04-25T09:16:11.535 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:16:11.535 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:16:11.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:11.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.807+0000 7f8282800640 1 Processor -- start 2024-04-25T09:16:11.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.807+0000 7f8282800640 1 -- start start 2024-04-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.809+0000 7f8282800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f827c0b56f0 0x7f827c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.809+0000 7f8282800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 0x7f827c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.809+0000 7f8282800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c0d3110 con 0x7f827c0b6a90 2024-04-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.809+0000 7f8282800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c161780 con 0x7f827c0b56f0 2024-04-25T09:16:11.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.809+0000 7f8282800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0cedc0 con 0x7f827c0b27a0 2024-04-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.810+0000 7f8281400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f827c0b56f0 0x7f827c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.810+0000 7f8281e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 0x7f827c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.810+0000 7f8281400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f827c0b56f0 0x7f827c160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:47810/0 (socket says 172.21.15.62:47810) 2024-04-25T09:16:11.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.810+0000 7f8281400640 1 -- 172.21.15.62:0/1788330097 learned_addr learned my addr 172.21.15.62:0/1788330097 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.811+0000 7f8281400640 1 -- 172.21.15.62:0/1788330097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 msgr2=0x7f827c161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.811+0000 7f8281400640 1 --2- 172.21.15.62:0/1788330097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 0x7f827c161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.811+0000 7f8281400640 1 -- 172.21.15.62:0/1788330097 >> v1:172.21.15.121:6789/0 conn(0x7f827c0b6a90 legacy=0x7f827c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:11.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.811+0000 7f8281400640 1 -- 172.21.15.62:0/1788330097 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0d33f0 con 0x7f827c0b56f0 2024-04-25T09:16:11.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.812+0000 7f8281400640 1 --2- 172.21.15.62:0/1788330097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b56f0 0x7f827c160d00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8270009870 tx=0x7f827002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8c39d40e20e239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.812+0000 7f827be00640 1 -- 172.21.15.62:0/1788330097 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270037070 con 0x7f827c0b56f0 2024-04-25T09:16:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f827be00640 1 -- 172.21.15.62:0/1788330097 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8270032440 con 0x7f827c0b56f0 2024-04-25T09:16:11.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f827be00640 1 -- 172.21.15.62:0/1788330097 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270032810 con 0x7f827c0b56f0 2024-04-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f8282800640 1 -- 172.21.15.62:0/1788330097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b56f0 msgr2=0x7f827c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f8282800640 1 --2- 172.21.15.62:0/1788330097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b56f0 0x7f827c160d00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8270009870 tx=0x7f827002a270 comp rx=0 tx=0).stop 2024-04-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f8282800640 1 -- 172.21.15.62:0/1788330097 shutdown_connections 2024-04-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f8282800640 1 --2- 172.21.15.62:0/1788330097 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 0x7f827c161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f8282800640 1 --2- 172.21.15.62:0/1788330097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b56f0 0x7f827c160d00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f8282800640 1 -- 172.21.15.62:0/1788330097 >> 172.21.15.62:0/1788330097 conn(0x7f827c066d10 msgr2=0x7f827c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.813+0000 7f8282800640 1 -- 172.21.15.62:0/1788330097 shutdown_connections 2024-04-25T09:16:11.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.814+0000 7f8282800640 1 -- 172.21.15.62:0/1788330097 wait complete. 2024-04-25T09:16:11.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.815+0000 7f8282800640 1 Processor -- start 2024-04-25T09:16:11.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.815+0000 7f8282800640 1 -- start start 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.816+0000 7f8282800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 0x7f827c160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.816+0000 7f8282800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0b56f0 0x7f827c152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.816+0000 7f8282800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b6a90 0x7f827c0ab040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.816+0000 7f8282800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0b2d60 con 0x7f827c0b56f0 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.816+0000 7f8282800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0cea50 con 0x7f827c0b6a90 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.816+0000 7f8282800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0b2ee0 con 0x7f827c0b27a0 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8280a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0b56f0 0x7f827c152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8280a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0b56f0 0x7f827c152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35976/0 (socket says 172.21.15.62:35976) 2024-04-25T09:16:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8280a00640 1 -- 172.21.15.62:0/3866931280 learned_addr learned my addr 172.21.15.62:0/3866931280 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8281e00640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b6a90 0x7f827c0ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8281400640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 0x7f827c160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8280a00640 1 -- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 msgr2=0x7f827c160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8280a00640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 0x7f827c160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8280a00640 1 -- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b6a90 msgr2=0x7f827c0ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8280a00640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b6a90 0x7f827c0ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8280a00640 1 -- 172.21.15.62:0/3866931280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0aa5a0 con 0x7f827c0b56f0 2024-04-25T09:16:11.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.817+0000 7f8281e00640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b6a90 0x7f827c0ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.818+0000 7f8280a00640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0b56f0 0x7f827c152960 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f826800c440 tx=0x7f826800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.818+0000 7f8279600640 1 -- 172.21.15.62:0/3866931280 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680052d0 con 0x7f827c0b56f0 2024-04-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.818+0000 7f8279600640 1 -- 172.21.15.62:0/3866931280 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8268005470 con 0x7f827c0b56f0 2024-04-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.818+0000 7f8279600640 1 -- 172.21.15.62:0/3866931280 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268007260 con 0x7f827c0b56f0 2024-04-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.818+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c0cdec0 con 0x7f827c0b56f0 2024-04-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.818+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c0aab20 con 0x7f827c0b56f0 2024-04-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.819+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c04ef20 con 0x7f827c0b56f0 2024-04-25T09:16:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.820+0000 7f8279600640 1 -- 172.21.15.62:0/3866931280 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82680074c0 con 0x7f827c0b56f0 2024-04-25T09:16:11.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.821+0000 7f8279600640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f825c06b4d0 0x7f825c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:11.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.821+0000 7f8279600640 1 -- 172.21.15.62:0/3866931280 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82680896f0 con 0x7f827c0b56f0 2024-04-25T09:16:11.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.822+0000 7f8281400640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f825c06b4d0 0x7f825c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:11.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.823+0000 7f8281400640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f825c06b4d0 0x7f825c06d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8270034000 tx=0x7f827002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:11.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:11.826+0000 7f8279600640 1 -- 172.21.15.62:0/3866931280 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82680584d0 con 0x7f827c0b56f0 2024-04-25T09:16:12.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.064+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f827c0aacf0 con 0x7f827c0b56f0 2024-04-25T09:16:12.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.066+0000 7f8279600640 1 -- 172.21.15.62:0/3866931280 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f8268057dd0 con 0x7f827c0b56f0 2024-04-25T09:16:12.053 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:12.053 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:12.053 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f825c06b4d0 msgr2=0x7f825c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f825c06b4d0 0x7f825c06d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8270034000 tx=0x7f827002e000 comp rx=0 tx=0).stop 2024-04-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0b56f0 msgr2=0x7f827c152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0b56f0 0x7f827c152960 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f826800c440 tx=0x7f826800c900 comp rx=0 tx=0).stop 2024-04-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 shutdown_connections 2024-04-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f825c06b4d0 0x7f825c06d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f827c0b56f0 0x7f827c152960 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f827c0b6a90 0x7f827c0ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.068+0000 7f8282800640 1 --2- 172.21.15.62:0/3866931280 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f827c0b27a0 0x7f827c160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.069+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 >> 172.21.15.62:0/3866931280 conn(0x7f827c066d10 msgr2=0x7f827c0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.069+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 shutdown_connections 2024-04-25T09:16:12.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.069+0000 7f8282800640 1 -- 172.21.15.62:0/3866931280 wait complete. 2024-04-25T09:16:12.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:12.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.292+0000 7f6715c00640 1 Processor -- start 2024-04-25T09:16:12.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.293+0000 7f6715c00640 1 -- start start 2024-04-25T09:16:12.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.294+0000 7f6715c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67100b27a0 0x7f67100d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:12.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.294+0000 7f6715c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100d3110 con 0x7f67100b6a90 2024-04-25T09:16:12.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.294+0000 7f6715c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67100d33f0 con 0x7f67100b56f0 2024-04-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.294+0000 7f6715c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67100cedc0 con 0x7f67100b27a0 2024-04-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.295+0000 7f670f400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f67100b6a90 0x7f67100d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57398/0 (socket says 172.21.15.62:57398) 2024-04-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.295+0000 7f670f400640 1 -- 172.21.15.62:0/3389171690 learned_addr learned my addr 172.21.15.62:0/3389171690 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:12.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.295+0000 7f6715200640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67100b27a0 0x7f67100d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:12.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.295+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842469604 0 0) 0x7f67100d3110 con 0x7f67100b6a90 2024-04-25T09:16:12.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.295+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f0003520 con 0x7f67100b6a90 2024-04-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127690664 0 0) 0x7f67100d33f0 con 0x7f67100b56f0 2024-04-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67100d3110 con 0x7f67100b56f0 2024-04-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3265476200 0 0) 0x7f66f0003520 con 0x7f67100b6a90 2024-04-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67100b27a0 msgr2=0x7f67100d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:12.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67100b27a0 0x7f67100d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 >> v1:172.21.15.121:6789/0 conn(0x7f67100b56f0 legacy=0x7f6710160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67100d36d0 con 0x7f67100b6a90 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f66f8001970 con 0x7f67100b6a90 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 >> v1:172.21.15.62:6789/0 conn(0x7f67100b6a90 legacy=0x7f67100d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f66f0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0004800 0x7f66f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67100b27a0 0x7f66f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67100cedc0 con 0x7f66f0003b70 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f000baa0 con 0x7f66f0004800 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f000bca0 con 0x7f67100b27a0 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.296+0000 7f670fe00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f66f0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:12.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.297+0000 7f670f400640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0004800 0x7f66f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.297+0000 7f6715200640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67100b27a0 0x7f66f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.297+0000 7f670fe00640 1 -- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67100b27a0 msgr2=0x7f66f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.297+0000 7f670fe00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67100b27a0 0x7f66f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.297+0000 7f670fe00640 1 -- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0004800 msgr2=0x7f66f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.297+0000 7f670fe00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0004800 0x7f66f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.297+0000 7f670fe00640 1 -- 172.21.15.62:0/3389171690 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0003520 con 0x7f66f0003b70 2024-04-25T09:16:12.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.297+0000 7f670f400640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0004800 0x7f66f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:16:12.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.298+0000 7f670fe00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f66f0003f90 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f67000018a0 tx=0x7f66f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1f2e5e53c0212b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.299+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67000014f0 con 0x7f66f0003b70 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.299+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67000052d0 con 0x7f66f0003b70 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.299+0000 7f670ea00640 1 -- 172.21.15.62:0/3389171690 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670002ac40 con 0x7f66f0003b70 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.300+0000 7f6715c00640 1 -- 172.21.15.62:0/3389171690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 msgr2=0x7f66f0003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.300+0000 7f6715c00640 1 --2- 172.21.15.62:0/3389171690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f66f0003f90 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f67000018a0 tx=0x7f66f0005830 comp rx=0 tx=0).stop 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.300+0000 7f6715200640 1 -- 172.21.15.62:0/3389171690 reap_dead start 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.300+0000 7f6715c00640 1 -- 172.21.15.62:0/3389171690 shutdown_connections 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.300+0000 7f6715c00640 1 -- 172.21.15.62:0/3389171690 >> 172.21.15.62:0/3389171690 conn(0x7f6710066d10 msgr2=0x7f6710067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.300+0000 7f6715c00640 1 -- 172.21.15.62:0/3389171690 shutdown_connections 2024-04-25T09:16:12.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.300+0000 7f6715c00640 1 -- 172.21.15.62:0/3389171690 wait complete. 2024-04-25T09:16:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.302+0000 7f6715c00640 1 Processor -- start 2024-04-25T09:16:12.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.302+0000 7f6715c00640 1 -- start start 2024-04-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f6715c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f671014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f6715c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6710159b00 0x7f6710159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f6715c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67100cb4e0 0x7f67100cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f6715c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67100b2ee0 con 0x7f66f0003b70 2024-04-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f6715c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67100b2d60 con 0x7f67100cb4e0 2024-04-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f6715c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67100b5ad0 con 0x7f6710159b00 2024-04-25T09:16:12.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f670fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f671014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f670fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f671014ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36008/0 (socket says 172.21.15.62:36008) 2024-04-25T09:16:12.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.303+0000 7f670fe00640 1 -- 172.21.15.62:0/3665862227 learned_addr learned my addr 172.21.15.62:0/3665862227 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f670f400640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6710159b00 0x7f6710159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f670fe00640 1 -- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6710159b00 msgr2=0x7f6710159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f670fe00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6710159b00 0x7f6710159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f670fe00640 1 -- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67100cb4e0 msgr2=0x7f67100cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f6715200640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67100cb4e0 0x7f67100cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f670fe00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67100cb4e0 0x7f67100cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f670fe00640 1 -- 172.21.15.62:0/3665862227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67100cc0c0 con 0x7f66f0003b70 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f6715200640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67100cb4e0 0x7f67100cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f670fe00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f671014ed20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f67000018a0 tx=0x7f67000012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f6707e00640 1 -- 172.21.15.62:0/3665862227 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670003f020 con 0x7f66f0003b70 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f6707e00640 1 -- 172.21.15.62:0/3665862227 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700005480 con 0x7f66f0003b70 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67100cc3a0 con 0x7f66f0003b70 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.304+0000 7f6707e00640 1 -- 172.21.15.62:0/3665862227 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700037040 con 0x7f66f0003b70 2024-04-25T09:16:12.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.305+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67100c85c0 con 0x7f66f0003b70 2024-04-25T09:16:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.306+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671004ef20 con 0x7f66f0003b70 2024-04-25T09:16:12.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.307+0000 7f6707e00640 1 -- 172.21.15.62:0/3665862227 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6700042020 con 0x7f66f0003b70 2024-04-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.308+0000 7f6707e00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66ec06b500 0x7f66ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.308+0000 7f6707e00640 1 -- 172.21.15.62:0/3665862227 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67000a9f10 con 0x7f66f0003b70 2024-04-25T09:16:12.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.309+0000 7f670f400640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66ec06b500 0x7f66ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:12.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.310+0000 7f670f400640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66ec06b500 0x7f66ec06d9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f66f8001ef0 tx=0x7f66f8001de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:12.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.313+0000 7f6707e00640 1 -- 172.21.15.62:0/3665862227 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6700049030 con 0x7f66f0003b70 2024-04-25T09:16:12.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.553+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67100ce580 con 0x7f66f0003b70 2024-04-25T09:16:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.555+0000 7f6707e00640 1 -- 172.21.15.62:0/3665862227 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6700078c60 con 0x7f66f0003b70 2024-04-25T09:16:12.541 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:12.541 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66ec06b500 msgr2=0x7f66ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66ec06b500 0x7f66ec06d9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f66f8001ef0 tx=0x7f66f8001de0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 msgr2=0x7f671014ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f671014ed20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f67000018a0 tx=0x7f67000012d0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 shutdown_connections 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f66ec06b500 0x7f66ec06d9c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67100cb4e0 0x7f67100cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6710159b00 0x7f6710159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 --2- 172.21.15.62:0/3665862227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66f0003b70 0x7f671014ed20 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.557+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 >> 172.21.15.62:0/3665862227 conn(0x7f6710066d10 msgr2=0x7f67100b86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.558+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 shutdown_connections 2024-04-25T09:16:12.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:12.558+0000 7f6715c00640 1 -- 172.21.15.62:0/3665862227 wait complete. 2024-04-25T09:16:12.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:12.564 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:12.564 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:12.565 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:16:12.565 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:12.565 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:12.565 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:12.565 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.664 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342645 s, 30.6 MB/s 2024-04-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310119 s, 33.8 MB/s 2024-04-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224789 s, 46.6 MB/s 2024-04-25T09:16:12.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.762 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032303 s, 32.5 MB/s 2024-04-25T09:16:12.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.794 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268573 s, 39.0 MB/s 2024-04-25T09:16:12.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.824 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.824 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249131 s, 42.1 MB/s 2024-04-25T09:16:12.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273047 s, 38.4 MB/s 2024-04-25T09:16:12.880 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.881 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.881 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215401 s, 48.7 MB/s 2024-04-25T09:16:12.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.927 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.041561 s, 25.2 MB/s 2024-04-25T09:16:12.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.965 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350851 s, 29.9 MB/s 2024-04-25T09:16:12.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:12.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:12.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236191 s, 44.4 MB/s 2024-04-25T09:16:13.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.027 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311713 s, 33.6 MB/s 2024-04-25T09:16:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223295 s, 47.0 MB/s 2024-04-25T09:16:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.082 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243819 s, 43.0 MB/s 2024-04-25T09:16:13.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167337 s, 62.7 MB/s 2024-04-25T09:16:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.126 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200169 s, 52.4 MB/s 2024-04-25T09:16:13.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231085 s, 45.4 MB/s 2024-04-25T09:16:13.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230354 s, 45.5 MB/s 2024-04-25T09:16:13.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.209 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247461 s, 42.4 MB/s 2024-04-25T09:16:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.240 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271171 s, 38.7 MB/s 2024-04-25T09:16:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248211 s, 42.2 MB/s 2024-04-25T09:16:13.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.299 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254765 s, 41.2 MB/s 2024-04-25T09:16:13.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326563 s, 32.1 MB/s 2024-04-25T09:16:13.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.364 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247297 s, 42.4 MB/s 2024-04-25T09:16:13.394 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.394 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.394 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264822 s, 39.6 MB/s 2024-04-25T09:16:13.421 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224988 s, 46.6 MB/s 2024-04-25T09:16:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273672 s, 38.3 MB/s 2024-04-25T09:16:13.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.481 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245617 s, 42.7 MB/s 2024-04-25T09:16:13.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.510 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237003 s, 44.2 MB/s 2024-04-25T09:16:13.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.538 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242798 s, 43.2 MB/s 2024-04-25T09:16:13.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.567 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248955 s, 42.1 MB/s 2024-04-25T09:16:13.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.595 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242545 s, 43.2 MB/s 2024-04-25T09:16:13.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:13.598 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:13.598 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:13.598 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:16:13.598 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:13.598 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:13.599 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:13.599 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.686 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231555 s, 45.3 MB/s 2024-04-25T09:16:13.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.711 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213413 s, 49.1 MB/s 2024-04-25T09:16:13.739 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.739 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.739 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241388 s, 43.4 MB/s 2024-04-25T09:16:13.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.765 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219124 s, 47.9 MB/s 2024-04-25T09:16:13.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.793 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242624 s, 43.2 MB/s 2024-04-25T09:16:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.819 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218147 s, 48.1 MB/s 2024-04-25T09:16:13.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.845 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219927 s, 47.7 MB/s 2024-04-25T09:16:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.872 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235325 s, 44.6 MB/s 2024-04-25T09:16:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205305 s, 51.1 MB/s 2024-04-25T09:16:13.919 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.919 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.919 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018525 s, 56.6 MB/s 2024-04-25T09:16:13.950 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270759 s, 38.7 MB/s 2024-04-25T09:16:13.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:13.977 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:13.977 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221358 s, 47.4 MB/s 2024-04-25T09:16:14.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.004 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229102 s, 45.8 MB/s 2024-04-25T09:16:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.031 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234049 s, 44.8 MB/s 2024-04-25T09:16:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.060 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243327 s, 43.1 MB/s 2024-04-25T09:16:14.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.088 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244528 s, 42.9 MB/s 2024-04-25T09:16:14.113 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217006 s, 48.3 MB/s 2024-04-25T09:16:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.140 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218722 s, 47.9 MB/s 2024-04-25T09:16:14.166 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.166 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.166 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223969 s, 46.8 MB/s 2024-04-25T09:16:14.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.194 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239427 s, 43.8 MB/s 2024-04-25T09:16:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026503 s, 39.6 MB/s 2024-04-25T09:16:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.258 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275482 s, 38.1 MB/s 2024-04-25T09:16:14.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261166 s, 40.1 MB/s 2024-04-25T09:16:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246677 s, 42.5 MB/s 2024-04-25T09:16:14.345 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.345 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.345 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022051 s, 47.6 MB/s 2024-04-25T09:16:14.375 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.375 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.375 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267562 s, 39.2 MB/s 2024-04-25T09:16:14.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.400 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204128 s, 51.4 MB/s 2024-04-25T09:16:14.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.426 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219876 s, 47.7 MB/s 2024-04-25T09:16:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.450 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194331 s, 54.0 MB/s 2024-04-25T09:16:14.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.478 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230316 s, 45.5 MB/s 2024-04-25T09:16:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.499 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181685 s, 57.7 MB/s 2024-04-25T09:16:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231634 s, 45.3 MB/s 2024-04-25T09:16:14.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:14.530 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:14.530 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:14.530 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:16:14.530 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:14.530 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:14.530 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:14.530 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:14.624 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267832 s, 39.2 MB/s 2024-04-25T09:16:14.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.649 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207317 s, 50.6 MB/s 2024-04-25T09:16:14.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187922 s, 55.8 MB/s 2024-04-25T09:16:14.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.702 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260409 s, 40.3 MB/s 2024-04-25T09:16:14.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.728 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216231 s, 48.5 MB/s 2024-04-25T09:16:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.757 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250797 s, 41.8 MB/s 2024-04-25T09:16:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.784 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220209 s, 47.6 MB/s 2024-04-25T09:16:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.814 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249276 s, 42.1 MB/s 2024-04-25T09:16:14.838 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.838 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.838 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202065 s, 51.9 MB/s 2024-04-25T09:16:14.865 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.865 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.865 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226175 s, 46.4 MB/s 2024-04-25T09:16:14.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.892 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022261 s, 47.1 MB/s 2024-04-25T09:16:14.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.915 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189626 s, 55.3 MB/s 2024-04-25T09:16:14.940 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.940 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.940 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206161 s, 50.9 MB/s 2024-04-25T09:16:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.965 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212889 s, 49.3 MB/s 2024-04-25T09:16:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:14.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179343 s, 58.5 MB/s 2024-04-25T09:16:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.011 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202787 s, 51.7 MB/s 2024-04-25T09:16:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.039 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240963 s, 43.5 MB/s 2024-04-25T09:16:15.069 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.069 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.069 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254762 s, 41.2 MB/s 2024-04-25T09:16:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.095 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220939 s, 47.5 MB/s 2024-04-25T09:16:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.125 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256259 s, 40.9 MB/s 2024-04-25T09:16:15.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247803 s, 42.3 MB/s 2024-04-25T09:16:15.183 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.183 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.183 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244246 s, 42.9 MB/s 2024-04-25T09:16:15.212 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.212 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.212 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250879 s, 41.8 MB/s 2024-04-25T09:16:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.241 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249154 s, 42.1 MB/s 2024-04-25T09:16:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.270 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245662 s, 42.7 MB/s 2024-04-25T09:16:15.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:16:15.294 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:16:15.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.301 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274198 s, 38.2 MB/s 2024-04-25T09:16:15.322 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.323 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176239 s, 59.5 MB/s 2024-04-25T09:16:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159426 s, 65.8 MB/s 2024-04-25T09:16:15.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.361 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177303 s, 59.1 MB/s 2024-04-25T09:16:15.384 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.384 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.384 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204579 s, 51.3 MB/s 2024-04-25T09:16:15.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.407 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191253 s, 54.8 MB/s 2024-04-25T09:16:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.434 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235877 s, 44.5 MB/s 2024-04-25T09:16:15.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:15.436 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:15.437 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:15.437 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:16:15.437 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:15.437 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:15.437 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:15.437 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:15.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.569 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226795 s, 46.2 MB/s 2024-04-25T09:16:15.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.596 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232017 s, 45.2 MB/s 2024-04-25T09:16:15.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.622 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230479 s, 45.5 MB/s 2024-04-25T09:16:15.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247571 s, 42.4 MB/s 2024-04-25T09:16:15.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.679 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237711 s, 44.1 MB/s 2024-04-25T09:16:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.705 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209123 s, 50.1 MB/s 2024-04-25T09:16:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.732 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229747 s, 45.6 MB/s 2024-04-25T09:16:15.757 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.757 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.757 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020817 s, 50.4 MB/s 2024-04-25T09:16:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.784 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023183 s, 45.2 MB/s 2024-04-25T09:16:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.808 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198474 s, 52.8 MB/s 2024-04-25T09:16:15.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.834 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215302 s, 48.7 MB/s 2024-04-25T09:16:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253635 s, 41.3 MB/s 2024-04-25T09:16:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232009 s, 45.2 MB/s 2024-04-25T09:16:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023084 s, 45.4 MB/s 2024-04-25T09:16:15.943 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.944 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.944 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205734 s, 51.0 MB/s 2024-04-25T09:16:15.968 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.969 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.969 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215643 s, 48.6 MB/s 2024-04-25T09:16:15.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:15.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:15.995 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225812 s, 46.4 MB/s 2024-04-25T09:16:16.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.022 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022273 s, 47.1 MB/s 2024-04-25T09:16:16.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.051 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.051 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256538 s, 40.9 MB/s 2024-04-25T09:16:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207822 s, 50.5 MB/s 2024-04-25T09:16:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241129 s, 43.5 MB/s 2024-04-25T09:16:16.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.126 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186801 s, 56.1 MB/s 2024-04-25T09:16:16.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.151 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197793 s, 53.0 MB/s 2024-04-25T09:16:16.177 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.178 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223252 s, 47.0 MB/s 2024-04-25T09:16:16.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.202 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218122 s, 48.1 MB/s 2024-04-25T09:16:16.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.229 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217807 s, 48.1 MB/s 2024-04-25T09:16:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.263 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290139 s, 36.1 MB/s 2024-04-25T09:16:16.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220738 s, 47.5 MB/s 2024-04-25T09:16:16.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.316 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225877 s, 46.4 MB/s 2024-04-25T09:16:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.343 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217749 s, 48.2 MB/s 2024-04-25T09:16:16.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.370 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227487 s, 46.1 MB/s 2024-04-25T09:16:16.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.399 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024502 s, 42.8 MB/s 2024-04-25T09:16:16.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:16.401 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:16.402 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:16.402 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:16:16.402 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:16.402 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:16.402 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:16.402 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:16.482 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.483 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180182 s, 58.2 MB/s 2024-04-25T09:16:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.508 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225817 s, 46.4 MB/s 2024-04-25T09:16:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.532 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194635 s, 53.9 MB/s 2024-04-25T09:16:16.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.558 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.558 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219831 s, 47.7 MB/s 2024-04-25T09:16:16.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022012 s, 47.6 MB/s 2024-04-25T09:16:16.613 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.613 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.613 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253979 s, 41.3 MB/s 2024-04-25T09:16:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.640 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230284 s, 45.5 MB/s 2024-04-25T09:16:16.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.665 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214951 s, 48.8 MB/s 2024-04-25T09:16:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.689 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019163 s, 54.7 MB/s 2024-04-25T09:16:16.715 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.715 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.715 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219417 s, 47.8 MB/s 2024-04-25T09:16:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.742 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231978 s, 45.2 MB/s 2024-04-25T09:16:16.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.772 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248757 s, 42.2 MB/s 2024-04-25T09:16:16.797 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.797 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.797 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212266 s, 49.4 MB/s 2024-04-25T09:16:16.829 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.829 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.830 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257523 s, 40.7 MB/s 2024-04-25T09:16:16.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.853 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022216 s, 47.2 MB/s 2024-04-25T09:16:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213845 s, 49.0 MB/s 2024-04-25T09:16:16.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.907 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241658 s, 43.4 MB/s 2024-04-25T09:16:16.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258959 s, 40.5 MB/s 2024-04-25T09:16:16.967 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.968 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.968 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258818 s, 40.5 MB/s 2024-04-25T09:16:16.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:16.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:16.995 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222306 s, 47.2 MB/s 2024-04-25T09:16:17.029 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.030 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285745 s, 36.7 MB/s 2024-04-25T09:16:17.056 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.056 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.056 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208682 s, 50.2 MB/s 2024-04-25T09:16:17.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.078 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.078 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191779 s, 54.7 MB/s 2024-04-25T09:16:17.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.106 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205963 s, 50.9 MB/s 2024-04-25T09:16:17.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.125 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193718 s, 54.1 MB/s 2024-04-25T09:16:17.152 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231959 s, 45.2 MB/s 2024-04-25T09:16:17.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.180 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022908 s, 45.8 MB/s 2024-04-25T09:16:17.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.204 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021045 s, 49.8 MB/s 2024-04-25T09:16:17.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.232 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234529 s, 44.7 MB/s 2024-04-25T09:16:17.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.259 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221495 s, 47.3 MB/s 2024-04-25T09:16:17.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.286 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238284 s, 44.0 MB/s 2024-04-25T09:16:17.311 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.312 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.312 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207081 s, 50.6 MB/s 2024-04-25T09:16:17.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:17.315 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:17.315 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:17.315 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:16:17.315 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:17.315 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:17.315 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:17.315 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:17.405 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.405 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.405 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249168 s, 42.1 MB/s 2024-04-25T09:16:17.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.432 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241928 s, 43.3 MB/s 2024-04-25T09:16:17.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.460 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238431 s, 44.0 MB/s 2024-04-25T09:16:17.485 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.485 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.485 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210875 s, 49.7 MB/s 2024-04-25T09:16:17.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.511 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.511 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224735 s, 46.7 MB/s 2024-04-25T09:16:17.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211592 s, 49.6 MB/s 2024-04-25T09:16:17.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.564 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243828 s, 43.0 MB/s 2024-04-25T09:16:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.591 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228047 s, 46.0 MB/s 2024-04-25T09:16:17.615 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.615 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.615 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201526 s, 52.0 MB/s 2024-04-25T09:16:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.644 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242797 s, 43.2 MB/s 2024-04-25T09:16:17.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.669 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.669 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209729 s, 50.0 MB/s 2024-04-25T09:16:17.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240385 s, 43.6 MB/s 2024-04-25T09:16:17.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021792 s, 48.1 MB/s 2024-04-25T09:16:17.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.750 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220795 s, 47.5 MB/s 2024-04-25T09:16:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251066 s, 41.8 MB/s 2024-04-25T09:16:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.808 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252951 s, 41.5 MB/s 2024-04-25T09:16:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.836 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243424 s, 43.1 MB/s 2024-04-25T09:16:17.862 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205023 s, 51.1 MB/s 2024-04-25T09:16:17.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.888 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221392 s, 47.4 MB/s 2024-04-25T09:16:17.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218707 s, 47.9 MB/s 2024-04-25T09:16:17.940 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.940 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.940 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222373 s, 47.2 MB/s 2024-04-25T09:16:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.965 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216137 s, 48.5 MB/s 2024-04-25T09:16:17.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:17.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:17.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021691 s, 48.3 MB/s 2024-04-25T09:16:18.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.018 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238916 s, 43.9 MB/s 2024-04-25T09:16:18.042 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.042 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.043 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195549 s, 53.6 MB/s 2024-04-25T09:16:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198971 s, 52.7 MB/s 2024-04-25T09:16:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.093 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021682 s, 48.4 MB/s 2024-04-25T09:16:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.118 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214809 s, 48.8 MB/s 2024-04-25T09:16:18.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259997 s, 40.3 MB/s 2024-04-25T09:16:18.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204125 s, 51.4 MB/s 2024-04-25T09:16:18.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248733 s, 42.2 MB/s 2024-04-25T09:16:18.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.232 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253546 s, 41.4 MB/s 2024-04-25T09:16:18.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:18.235 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:18.235 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:18.235 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:16:18.235 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:18.235 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:18.235 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:18.235 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:18.325 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.326 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238162 s, 44.0 MB/s 2024-04-25T09:16:18.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.352 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230793 s, 45.4 MB/s 2024-04-25T09:16:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207128 s, 50.6 MB/s 2024-04-25T09:16:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.404 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244328 s, 42.9 MB/s 2024-04-25T09:16:18.427 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.428 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019442 s, 53.9 MB/s 2024-04-25T09:16:18.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.454 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02152 s, 48.7 MB/s 2024-04-25T09:16:18.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.481 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225827 s, 46.4 MB/s 2024-04-25T09:16:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.504 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204 s, 51.4 MB/s 2024-04-25T09:16:18.528 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.528 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.528 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200356 s, 52.3 MB/s 2024-04-25T09:16:18.552 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.552 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.552 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205413 s, 51.0 MB/s 2024-04-25T09:16:18.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233864 s, 44.8 MB/s 2024-04-25T09:16:18.603 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.604 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.604 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204773 s, 51.2 MB/s 2024-04-25T09:16:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.630 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221947 s, 47.2 MB/s 2024-04-25T09:16:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.655 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207694 s, 50.5 MB/s 2024-04-25T09:16:18.678 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.678 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.678 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192373 s, 54.5 MB/s 2024-04-25T09:16:18.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.704 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217648 s, 48.2 MB/s 2024-04-25T09:16:18.729 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.729 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.729 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204939 s, 51.2 MB/s 2024-04-25T09:16:18.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.754 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210957 s, 49.7 MB/s 2024-04-25T09:16:18.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207691 s, 50.5 MB/s 2024-04-25T09:16:18.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227258 s, 46.1 MB/s 2024-04-25T09:16:18.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.832 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022222 s, 47.2 MB/s 2024-04-25T09:16:18.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226661 s, 46.3 MB/s 2024-04-25T09:16:18.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234964 s, 44.6 MB/s 2024-04-25T09:16:18.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.913 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218045 s, 48.1 MB/s 2024-04-25T09:16:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.938 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210522 s, 49.8 MB/s 2024-04-25T09:16:18.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.963 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202565 s, 51.8 MB/s 2024-04-25T09:16:18.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:18.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:18.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253089 s, 41.4 MB/s 2024-04-25T09:16:19.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.019 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207626 s, 50.5 MB/s 2024-04-25T09:16:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246708 s, 42.5 MB/s 2024-04-25T09:16:19.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.077 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.077 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233529 s, 44.9 MB/s 2024-04-25T09:16:19.105 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.105 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.105 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236639 s, 44.3 MB/s 2024-04-25T09:16:19.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.130 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211617 s, 49.6 MB/s 2024-04-25T09:16:19.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:19.132 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:19.133 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:19.133 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:16:19.133 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:19.133 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:19.133 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:19.133 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.220 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234735 s, 44.7 MB/s 2024-04-25T09:16:19.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.245 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228012 s, 46.0 MB/s 2024-04-25T09:16:19.273 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.274 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.274 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246863 s, 42.5 MB/s 2024-04-25T09:16:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.301 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235405 s, 44.5 MB/s 2024-04-25T09:16:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.326 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208294 s, 50.3 MB/s 2024-04-25T09:16:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232767 s, 45.0 MB/s 2024-04-25T09:16:19.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.382 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023904 s, 43.9 MB/s 2024-04-25T09:16:19.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.407 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206184 s, 50.9 MB/s 2024-04-25T09:16:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196726 s, 53.3 MB/s 2024-04-25T09:16:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223866 s, 46.8 MB/s 2024-04-25T09:16:19.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.483 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208159 s, 50.4 MB/s 2024-04-25T09:16:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195644 s, 53.6 MB/s 2024-04-25T09:16:19.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215981 s, 48.5 MB/s 2024-04-25T09:16:19.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.557 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.557 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202247 s, 51.8 MB/s 2024-04-25T09:16:19.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.583 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209979 s, 49.9 MB/s 2024-04-25T09:16:19.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.612 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254313 s, 41.2 MB/s 2024-04-25T09:16:19.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022786 s, 46.0 MB/s 2024-04-25T09:16:19.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.665 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.665 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223444 s, 46.9 MB/s 2024-04-25T09:16:19.690 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.690 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.690 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215244 s, 48.7 MB/s 2024-04-25T09:16:19.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.724 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276351 s, 37.9 MB/s 2024-04-25T09:16:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.753 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251906 s, 41.6 MB/s 2024-04-25T09:16:19.778 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223977 s, 46.8 MB/s 2024-04-25T09:16:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262557 s, 39.9 MB/s 2024-04-25T09:16:19.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229171 s, 45.8 MB/s 2024-04-25T09:16:19.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.864 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231276 s, 45.3 MB/s 2024-04-25T09:16:19.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.894 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260265 s, 40.3 MB/s 2024-04-25T09:16:19.919 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.920 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215081 s, 48.8 MB/s 2024-04-25T09:16:19.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.946 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.946 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220235 s, 47.6 MB/s 2024-04-25T09:16:19.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.971 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210571 s, 49.8 MB/s 2024-04-25T09:16:19.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:19.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:19.995 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196717 s, 53.3 MB/s 2024-04-25T09:16:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.018 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206687 s, 50.7 MB/s 2024-04-25T09:16:20.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241474 s, 43.4 MB/s 2024-04-25T09:16:20.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:20.049 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:20.049 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:20.049 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:16:20.049 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:20.050 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:20.050 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:20.050 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:20.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.136 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224715 s, 46.7 MB/s 2024-04-25T09:16:20.162 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.162 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.162 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219938 s, 47.7 MB/s 2024-04-25T09:16:20.189 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228824 s, 45.8 MB/s 2024-04-25T09:16:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249747 s, 42.0 MB/s 2024-04-25T09:16:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264003 s, 39.7 MB/s 2024-04-25T09:16:20.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.278 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247742 s, 42.3 MB/s 2024-04-25T09:16:20.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.310 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253624 s, 41.3 MB/s 2024-04-25T09:16:20.334 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.334 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232633 s, 45.1 MB/s 2024-04-25T09:16:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.365 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026742 s, 39.2 MB/s 2024-04-25T09:16:20.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.396 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267337 s, 39.2 MB/s 2024-04-25T09:16:20.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.423 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227486 s, 46.1 MB/s 2024-04-25T09:16:20.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249283 s, 42.1 MB/s 2024-04-25T09:16:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.478 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217631 s, 48.2 MB/s 2024-04-25T09:16:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223961 s, 46.8 MB/s 2024-04-25T09:16:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.534 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247595 s, 42.4 MB/s 2024-04-25T09:16:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.561 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224164 s, 46.8 MB/s 2024-04-25T09:16:20.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.585 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206978 s, 50.7 MB/s 2024-04-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.611 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221166 s, 47.4 MB/s 2024-04-25T09:16:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.637 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217401 s, 48.2 MB/s 2024-04-25T09:16:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.663 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223508 s, 46.9 MB/s 2024-04-25T09:16:20.690 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.691 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.691 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235455 s, 44.5 MB/s 2024-04-25T09:16:20.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.721 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258122 s, 40.6 MB/s 2024-04-25T09:16:20.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218909 s, 47.9 MB/s 2024-04-25T09:16:20.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.775 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240616 s, 43.6 MB/s 2024-04-25T09:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.801 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218888 s, 47.9 MB/s 2024-04-25T09:16:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.827 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212886 s, 49.3 MB/s 2024-04-25T09:16:20.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.852 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212075 s, 49.4 MB/s 2024-04-25T09:16:20.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.880 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235054 s, 44.6 MB/s 2024-04-25T09:16:20.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.906 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.906 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224235 s, 46.8 MB/s 2024-04-25T09:16:20.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.934 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236621 s, 44.3 MB/s 2024-04-25T09:16:20.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.965 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268489 s, 39.1 MB/s 2024-04-25T09:16:20.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:20.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:20.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221114 s, 47.4 MB/s 2024-04-25T09:16:20.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:20.994 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:20.994 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:20.994 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:16:20.994 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:20.994 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:20.994 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:20.994 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:21.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.084 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226923 s, 46.2 MB/s 2024-04-25T09:16:21.110 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.110 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.110 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228708 s, 45.8 MB/s 2024-04-25T09:16:21.133 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.133 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.133 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200501 s, 52.3 MB/s 2024-04-25T09:16:21.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.163 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263369 s, 39.8 MB/s 2024-04-25T09:16:21.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220289 s, 47.6 MB/s 2024-04-25T09:16:21.217 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.218 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.218 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235075 s, 44.6 MB/s 2024-04-25T09:16:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.243 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220559 s, 47.5 MB/s 2024-04-25T09:16:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.269 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216672 s, 48.4 MB/s 2024-04-25T09:16:21.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.297 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230405 s, 45.5 MB/s 2024-04-25T09:16:21.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.324 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220539 s, 47.5 MB/s 2024-04-25T09:16:21.351 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.351 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.352 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224386 s, 46.7 MB/s 2024-04-25T09:16:21.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023535 s, 44.6 MB/s 2024-04-25T09:16:21.404 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.405 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.405 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208068 s, 50.4 MB/s 2024-04-25T09:16:21.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205981 s, 50.9 MB/s 2024-04-25T09:16:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.460 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266428 s, 39.4 MB/s 2024-04-25T09:16:21.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.484 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01964 s, 53.4 MB/s 2024-04-25T09:16:21.510 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.511 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.511 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217993 s, 48.1 MB/s 2024-04-25T09:16:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.540 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234622 s, 44.7 MB/s 2024-04-25T09:16:21.566 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.566 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.567 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224292 s, 46.8 MB/s 2024-04-25T09:16:21.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.596 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253743 s, 41.3 MB/s 2024-04-25T09:16:21.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.622 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221416 s, 47.4 MB/s 2024-04-25T09:16:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.648 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221791 s, 47.3 MB/s 2024-04-25T09:16:21.674 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.674 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.674 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204978 s, 51.2 MB/s 2024-04-25T09:16:21.698 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208485 s, 50.3 MB/s 2024-04-25T09:16:21.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220849 s, 47.5 MB/s 2024-04-25T09:16:21.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.754 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246926 s, 42.5 MB/s 2024-04-25T09:16:21.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.782 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.782 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233961 s, 44.8 MB/s 2024-04-25T09:16:21.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.807 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218448 s, 48.0 MB/s 2024-04-25T09:16:21.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.834 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229382 s, 45.7 MB/s 2024-04-25T09:16:21.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194005 s, 54.0 MB/s 2024-04-25T09:16:21.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.884 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210563 s, 49.8 MB/s 2024-04-25T09:16:21.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:21.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:21.910 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208386 s, 50.3 MB/s 2024-04-25T09:16:21.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:21.912 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:21.912 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:21.912 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:16:21.912 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:21.913 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:21.913 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:21.913 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:22.015 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.016 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172579 s, 60.8 MB/s 2024-04-25T09:16:22.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.041 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022446 s, 46.7 MB/s 2024-04-25T09:16:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190771 s, 55.0 MB/s 2024-04-25T09:16:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217527 s, 48.2 MB/s 2024-04-25T09:16:22.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.117 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023477 s, 44.7 MB/s 2024-04-25T09:16:22.142 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.143 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211686 s, 49.5 MB/s 2024-04-25T09:16:22.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.170 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227785 s, 46.0 MB/s 2024-04-25T09:16:22.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.194 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200907 s, 52.2 MB/s 2024-04-25T09:16:22.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.223 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246894 s, 42.5 MB/s 2024-04-25T09:16:22.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.250 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223039 s, 47.0 MB/s 2024-04-25T09:16:22.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.279 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247978 s, 42.3 MB/s 2024-04-25T09:16:22.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.308 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244185 s, 42.9 MB/s 2024-04-25T09:16:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.337 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252435 s, 41.5 MB/s 2024-04-25T09:16:22.364 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.365 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.365 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224545 s, 46.7 MB/s 2024-04-25T09:16:22.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.392 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236278 s, 44.4 MB/s 2024-04-25T09:16:22.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266954 s, 39.3 MB/s 2024-04-25T09:16:22.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250033 s, 41.9 MB/s 2024-04-25T09:16:22.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.481 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.481 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239649 s, 43.8 MB/s 2024-04-25T09:16:22.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.507 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213015 s, 49.2 MB/s 2024-04-25T09:16:22.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.532 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021207 s, 49.4 MB/s 2024-04-25T09:16:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.559 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210787 s, 49.7 MB/s 2024-04-25T09:16:22.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.586 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229716 s, 45.6 MB/s 2024-04-25T09:16:22.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198522 s, 52.8 MB/s 2024-04-25T09:16:22.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.633 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189503 s, 55.3 MB/s 2024-04-25T09:16:22.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.658 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204463 s, 51.3 MB/s 2024-04-25T09:16:22.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.684 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218955 s, 47.9 MB/s 2024-04-25T09:16:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.711 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233604 s, 44.9 MB/s 2024-04-25T09:16:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214098 s, 49.0 MB/s 2024-04-25T09:16:22.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.766 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233624 s, 44.9 MB/s 2024-04-25T09:16:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.794 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255185 s, 41.1 MB/s 2024-04-25T09:16:22.821 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.822 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023138 s, 45.3 MB/s 2024-04-25T09:16:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.845 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202717 s, 51.7 MB/s 2024-04-25T09:16:22.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:22.848 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:22.848 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:22.848 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:16:22.848 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:22.848 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:22.848 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:22.848 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:22.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.935 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224371 s, 46.7 MB/s 2024-04-25T09:16:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.958 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192473 s, 54.5 MB/s 2024-04-25T09:16:22.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:22.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:22.988 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026263 s, 39.9 MB/s 2024-04-25T09:16:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200244 s, 52.4 MB/s 2024-04-25T09:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02102 s, 49.9 MB/s 2024-04-25T09:16:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.063 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221829 s, 47.3 MB/s 2024-04-25T09:16:23.087 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.088 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197172 s, 53.2 MB/s 2024-04-25T09:16:23.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.115 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.115 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219475 s, 47.8 MB/s 2024-04-25T09:16:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.139 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206344 s, 50.8 MB/s 2024-04-25T09:16:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.166 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220549 s, 47.5 MB/s 2024-04-25T09:16:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.192 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022157 s, 47.3 MB/s 2024-04-25T09:16:23.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.222 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252269 s, 41.6 MB/s 2024-04-25T09:16:23.248 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.248 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.248 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227028 s, 46.2 MB/s 2024-04-25T09:16:23.275 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.275 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.275 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223936 s, 46.8 MB/s 2024-04-25T09:16:23.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.298 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205767 s, 51.0 MB/s 2024-04-25T09:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.323 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204757 s, 51.2 MB/s 2024-04-25T09:16:23.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250528 s, 41.9 MB/s 2024-04-25T09:16:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.384 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270769 s, 38.7 MB/s 2024-04-25T09:16:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.414 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256879 s, 40.8 MB/s 2024-04-25T09:16:23.441 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.441 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.441 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227573 s, 46.1 MB/s 2024-04-25T09:16:23.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.468 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225061 s, 46.6 MB/s 2024-04-25T09:16:23.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.493 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208741 s, 50.2 MB/s 2024-04-25T09:16:23.519 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.520 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.520 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224137 s, 46.8 MB/s 2024-04-25T09:16:23.542 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.543 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199536 s, 52.6 MB/s 2024-04-25T09:16:23.566 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.566 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.566 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201877 s, 51.9 MB/s 2024-04-25T09:16:23.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.592 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220662 s, 47.5 MB/s 2024-04-25T09:16:23.620 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.620 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.620 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240078 s, 43.7 MB/s 2024-04-25T09:16:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.644 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203062 s, 51.6 MB/s 2024-04-25T09:16:23.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228347 s, 45.9 MB/s 2024-04-25T09:16:23.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236762 s, 44.3 MB/s 2024-04-25T09:16:23.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.727 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223725 s, 46.9 MB/s 2024-04-25T09:16:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.752 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210407 s, 49.8 MB/s 2024-04-25T09:16:23.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:23.755 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:23.755 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:23.755 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:16:23.755 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:23.755 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:23.755 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:23.755 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.845 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249912 s, 42.0 MB/s 2024-04-25T09:16:23.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202499 s, 51.8 MB/s 2024-04-25T09:16:23.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.896 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225568 s, 46.5 MB/s 2024-04-25T09:16:23.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.922 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220501 s, 47.6 MB/s 2024-04-25T09:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.946 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202749 s, 51.7 MB/s 2024-04-25T09:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249091 s, 42.1 MB/s 2024-04-25T09:16:23.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:23.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:23.999 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201956 s, 51.9 MB/s 2024-04-25T09:16:24.025 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.025 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.025 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218278 s, 48.0 MB/s 2024-04-25T09:16:24.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018633 s, 56.3 MB/s 2024-04-25T09:16:24.072 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.072 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.072 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018511 s, 56.6 MB/s 2024-04-25T09:16:24.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.093 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171935 s, 61.0 MB/s 2024-04-25T09:16:24.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.121 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237248 s, 44.2 MB/s 2024-04-25T09:16:24.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215958 s, 48.6 MB/s 2024-04-25T09:16:24.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.171 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209708 s, 50.0 MB/s 2024-04-25T09:16:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242033 s, 43.3 MB/s 2024-04-25T09:16:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.226 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200612 s, 52.3 MB/s 2024-04-25T09:16:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.254 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241179 s, 43.5 MB/s 2024-04-25T09:16:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.283 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247806 s, 42.3 MB/s 2024-04-25T09:16:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.310 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223578 s, 46.9 MB/s 2024-04-25T09:16:24.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.342 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291891 s, 35.9 MB/s 2024-04-25T09:16:24.372 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.372 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.372 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257795 s, 40.7 MB/s 2024-04-25T09:16:24.402 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.402 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.402 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241769 s, 43.4 MB/s 2024-04-25T09:16:24.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.427 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.427 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204556 s, 51.3 MB/s 2024-04-25T09:16:24.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.454 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231604 s, 45.3 MB/s 2024-04-25T09:16:24.485 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.485 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.485 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264904 s, 39.6 MB/s 2024-04-25T09:16:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.513 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232853 s, 45.0 MB/s 2024-04-25T09:16:24.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.535 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.535 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018277 s, 57.4 MB/s 2024-04-25T09:16:24.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219537 s, 47.8 MB/s 2024-04-25T09:16:24.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.587 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225223 s, 46.6 MB/s 2024-04-25T09:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.614 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215561 s, 48.6 MB/s 2024-04-25T09:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.640 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221867 s, 47.3 MB/s 2024-04-25T09:16:24.666 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.667 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217566 s, 48.2 MB/s 2024-04-25T09:16:24.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:24.669 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:24.669 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:24.669 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:16:24.669 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:24.669 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:24.669 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:24.669 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:24.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.755 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02341 s, 44.8 MB/s 2024-04-25T09:16:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.782 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220502 s, 47.6 MB/s 2024-04-25T09:16:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.807 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219834 s, 47.7 MB/s 2024-04-25T09:16:24.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.833 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219054 s, 47.9 MB/s 2024-04-25T09:16:24.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217595 s, 48.2 MB/s 2024-04-25T09:16:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02497 s, 42.0 MB/s 2024-04-25T09:16:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.920 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282611 s, 37.1 MB/s 2024-04-25T09:16:24.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.949 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024731 s, 42.4 MB/s 2024-04-25T09:16:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.972 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187345 s, 56.0 MB/s 2024-04-25T09:16:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216295 s, 48.5 MB/s 2024-04-25T09:16:25.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.021 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192474 s, 54.5 MB/s 2024-04-25T09:16:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229206 s, 45.7 MB/s 2024-04-25T09:16:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.075 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220515 s, 47.6 MB/s 2024-04-25T09:16:25.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.100 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208584 s, 50.3 MB/s 2024-04-25T09:16:25.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.125 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220258 s, 47.6 MB/s 2024-04-25T09:16:25.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.152 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.152 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206053 s, 50.9 MB/s 2024-04-25T09:16:25.174 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193341 s, 54.2 MB/s 2024-04-25T09:16:25.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.203 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241612 s, 43.4 MB/s 2024-04-25T09:16:25.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197226 s, 53.2 MB/s 2024-04-25T09:16:25.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.254 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226554 s, 46.3 MB/s 2024-04-25T09:16:25.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.282 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248262 s, 42.2 MB/s 2024-04-25T09:16:25.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.309 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229419 s, 45.7 MB/s 2024-04-25T09:16:25.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224029 s, 46.8 MB/s 2024-04-25T09:16:25.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.366 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255101 s, 41.1 MB/s 2024-04-25T09:16:25.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.393 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224326 s, 46.7 MB/s 2024-04-25T09:16:25.421 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.421 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.421 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239908 s, 43.7 MB/s 2024-04-25T09:16:25.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.447 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225981 s, 46.4 MB/s 2024-04-25T09:16:25.476 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.476 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.476 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239351 s, 43.8 MB/s 2024-04-25T09:16:25.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.503 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226289 s, 46.3 MB/s 2024-04-25T09:16:25.533 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.533 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.533 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256722 s, 40.8 MB/s 2024-04-25T09:16:25.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.559 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219531 s, 47.8 MB/s 2024-04-25T09:16:25.585 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.586 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212574 s, 49.3 MB/s 2024-04-25T09:16:25.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:25.588 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:25.588 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:25.588 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:16:25.588 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:25.588 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:25.588 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:25.588 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:25.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177755 s, 59.0 MB/s 2024-04-25T09:16:25.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218608 s, 48.0 MB/s 2024-04-25T09:16:25.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.722 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212386 s, 49.4 MB/s 2024-04-25T09:16:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221632 s, 47.3 MB/s 2024-04-25T09:16:25.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213462 s, 49.1 MB/s 2024-04-25T09:16:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235089 s, 44.6 MB/s 2024-04-25T09:16:25.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.829 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.829 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229377 s, 45.7 MB/s 2024-04-25T09:16:25.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227435 s, 46.1 MB/s 2024-04-25T09:16:25.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221729 s, 47.3 MB/s 2024-04-25T09:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.907 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021786 s, 48.1 MB/s 2024-04-25T09:16:25.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.933 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225659 s, 46.5 MB/s 2024-04-25T09:16:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219818 s, 47.7 MB/s 2024-04-25T09:16:25.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:25.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:25.986 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227186 s, 46.2 MB/s 2024-04-25T09:16:26.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.014 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228803 s, 45.8 MB/s 2024-04-25T09:16:26.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.046 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267253 s, 39.2 MB/s 2024-04-25T09:16:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225483 s, 46.5 MB/s 2024-04-25T09:16:26.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.098 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214327 s, 48.9 MB/s 2024-04-25T09:16:26.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.128 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260481 s, 40.3 MB/s 2024-04-25T09:16:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.155 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229562 s, 45.7 MB/s 2024-04-25T09:16:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.184 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242142 s, 43.3 MB/s 2024-04-25T09:16:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183768 s, 57.1 MB/s 2024-04-25T09:16:26.236 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.236 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.236 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249111 s, 42.1 MB/s 2024-04-25T09:16:26.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.265 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024752 s, 42.4 MB/s 2024-04-25T09:16:26.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.292 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023468 s, 44.7 MB/s 2024-04-25T09:16:26.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.320 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240165 s, 43.7 MB/s 2024-04-25T09:16:26.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.348 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02349 s, 44.6 MB/s 2024-04-25T09:16:26.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.374 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229285 s, 45.7 MB/s 2024-04-25T09:16:26.403 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.403 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.403 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243303 s, 43.1 MB/s 2024-04-25T09:16:26.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.433 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258686 s, 40.5 MB/s 2024-04-25T09:16:26.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.458 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211925 s, 49.5 MB/s 2024-04-25T09:16:26.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249935 s, 42.0 MB/s 2024-04-25T09:16:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.516 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255216 s, 41.1 MB/s 2024-04-25T09:16:26.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:26.519 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:26.519 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:26.519 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:16:26.519 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:26.519 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:26.519 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:26.519 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:26.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.612 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.612 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262429 s, 40.0 MB/s 2024-04-25T09:16:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.636 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214497 s, 48.9 MB/s 2024-04-25T09:16:26.666 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.666 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.666 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256684 s, 40.9 MB/s 2024-04-25T09:16:26.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.693 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233615 s, 44.9 MB/s 2024-04-25T09:16:26.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.721 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236002 s, 44.4 MB/s 2024-04-25T09:16:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.748 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238213 s, 44.0 MB/s 2024-04-25T09:16:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204638 s, 51.2 MB/s 2024-04-25T09:16:26.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021289 s, 49.3 MB/s 2024-04-25T09:16:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.820 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189369 s, 55.4 MB/s 2024-04-25T09:16:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.846 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216331 s, 48.5 MB/s 2024-04-25T09:16:26.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.870 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199306 s, 52.6 MB/s 2024-04-25T09:16:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.900 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253082 s, 41.4 MB/s 2024-04-25T09:16:26.924 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.924 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.924 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189479 s, 55.3 MB/s 2024-04-25T09:16:26.950 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022791 s, 46.0 MB/s 2024-04-25T09:16:26.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:26.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:26.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253756 s, 41.3 MB/s 2024-04-25T09:16:27.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.007 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224427 s, 46.7 MB/s 2024-04-25T09:16:27.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.032 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217123 s, 48.3 MB/s 2024-04-25T09:16:27.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.061 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.061 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234623 s, 44.7 MB/s 2024-04-25T09:16:27.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.085 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219372 s, 47.8 MB/s 2024-04-25T09:16:27.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.107 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183861 s, 57.0 MB/s 2024-04-25T09:16:27.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.132 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208716 s, 50.2 MB/s 2024-04-25T09:16:27.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.160 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.160 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228492 s, 45.9 MB/s 2024-04-25T09:16:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.182 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196272 s, 53.4 MB/s 2024-04-25T09:16:27.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.210 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236049 s, 44.4 MB/s 2024-04-25T09:16:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.237 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232227 s, 45.2 MB/s 2024-04-25T09:16:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.266 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244957 s, 42.8 MB/s 2024-04-25T09:16:27.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.293 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232769 s, 45.0 MB/s 2024-04-25T09:16:27.321 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.321 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.321 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240518 s, 43.6 MB/s 2024-04-25T09:16:27.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.347 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221026 s, 47.4 MB/s 2024-04-25T09:16:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.385 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329551 s, 31.8 MB/s 2024-04-25T09:16:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.410 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211268 s, 49.6 MB/s 2024-04-25T09:16:27.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.433 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193898 s, 54.1 MB/s 2024-04-25T09:16:27.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:27.436 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:27.436 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:27.436 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:16:27.436 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:27.436 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:27.436 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:27.437 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.528 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024339 s, 43.1 MB/s 2024-04-25T09:16:27.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.553 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212472 s, 49.4 MB/s 2024-04-25T09:16:27.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215726 s, 48.6 MB/s 2024-04-25T09:16:27.599 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.600 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180784 s, 58.0 MB/s 2024-04-25T09:16:27.624 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.624 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.624 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213942 s, 49.0 MB/s 2024-04-25T09:16:27.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.652 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243468 s, 43.1 MB/s 2024-04-25T09:16:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.681 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239492 s, 43.8 MB/s 2024-04-25T09:16:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.709 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023741 s, 44.2 MB/s 2024-04-25T09:16:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219821 s, 47.7 MB/s 2024-04-25T09:16:27.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.766 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267082 s, 39.3 MB/s 2024-04-25T09:16:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.793 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023114 s, 45.4 MB/s 2024-04-25T09:16:27.820 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.820 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.820 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218422 s, 48.0 MB/s 2024-04-25T09:16:27.844 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.844 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.844 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204515 s, 51.3 MB/s 2024-04-25T09:16:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.864 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168789 s, 62.1 MB/s 2024-04-25T09:16:27.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.892 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227816 s, 46.0 MB/s 2024-04-25T09:16:27.916 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.916 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.916 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191205 s, 54.8 MB/s 2024-04-25T09:16:27.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.943 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221704 s, 47.3 MB/s 2024-04-25T09:16:27.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200883 s, 52.2 MB/s 2024-04-25T09:16:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:27.992 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216947 s, 48.3 MB/s 2024-04-25T09:16:28.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.022 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262987 s, 39.9 MB/s 2024-04-25T09:16:28.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228999 s, 45.8 MB/s 2024-04-25T09:16:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.075 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214862 s, 48.8 MB/s 2024-04-25T09:16:28.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235386 s, 44.5 MB/s 2024-04-25T09:16:28.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.130 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219553 s, 47.8 MB/s 2024-04-25T09:16:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.157 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217897 s, 48.1 MB/s 2024-04-25T09:16:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.185 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.186 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253367 s, 41.4 MB/s 2024-04-25T09:16:28.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.213 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229395 s, 45.7 MB/s 2024-04-25T09:16:28.241 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.241 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.242 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245113 s, 42.8 MB/s 2024-04-25T09:16:28.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.268 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222746 s, 47.1 MB/s 2024-04-25T09:16:28.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.299 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265425 s, 39.5 MB/s 2024-04-25T09:16:28.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.324 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212091 s, 49.4 MB/s 2024-04-25T09:16:28.356 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.357 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028528 s, 36.8 MB/s 2024-04-25T09:16:28.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:28.360 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:28.360 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:28.360 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:16:28.360 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:28.360 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:28.360 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:28.360 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:28.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260314 s, 40.3 MB/s 2024-04-25T09:16:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.477 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206346 s, 50.8 MB/s 2024-04-25T09:16:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.506 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246431 s, 42.6 MB/s 2024-04-25T09:16:28.533 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.533 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.534 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239104 s, 43.9 MB/s 2024-04-25T09:16:28.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222412 s, 47.1 MB/s 2024-04-25T09:16:28.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198073 s, 52.9 MB/s 2024-04-25T09:16:28.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219735 s, 47.7 MB/s 2024-04-25T09:16:28.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.633 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192473 s, 54.5 MB/s 2024-04-25T09:16:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.663 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248849 s, 42.1 MB/s 2024-04-25T09:16:28.687 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.687 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.687 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207801 s, 50.5 MB/s 2024-04-25T09:16:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.716 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249286 s, 42.1 MB/s 2024-04-25T09:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.742 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212962 s, 49.2 MB/s 2024-04-25T09:16:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.767 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212718 s, 49.3 MB/s 2024-04-25T09:16:28.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.789 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191235 s, 54.8 MB/s 2024-04-25T09:16:28.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.815 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216853 s, 48.4 MB/s 2024-04-25T09:16:28.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.841 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.842 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227258 s, 46.1 MB/s 2024-04-25T09:16:28.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022823 s, 45.9 MB/s 2024-04-25T09:16:28.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.898 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024844 s, 42.2 MB/s 2024-04-25T09:16:28.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.925 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227947 s, 46.0 MB/s 2024-04-25T09:16:28.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.955 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255318 s, 41.1 MB/s 2024-04-25T09:16:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:28.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:28.981 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215316 s, 48.7 MB/s 2024-04-25T09:16:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.004 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019755 s, 53.1 MB/s 2024-04-25T09:16:29.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.035 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270773 s, 38.7 MB/s 2024-04-25T09:16:29.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.064 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233831 s, 44.8 MB/s 2024-04-25T09:16:29.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.093 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250057 s, 41.9 MB/s 2024-04-25T09:16:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214816 s, 48.8 MB/s 2024-04-25T09:16:29.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259866 s, 40.4 MB/s 2024-04-25T09:16:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.177 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245196 s, 42.8 MB/s 2024-04-25T09:16:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.205 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237242 s, 44.2 MB/s 2024-04-25T09:16:29.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.233 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239123 s, 43.9 MB/s 2024-04-25T09:16:29.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.261 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245366 s, 42.7 MB/s 2024-04-25T09:16:29.290 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.290 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.290 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024608 s, 42.6 MB/s 2024-04-25T09:16:29.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:29.292 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:29.292 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:29.292 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:16:29.293 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:29.293 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:29.293 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:29.293 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.382 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023052 s, 45.5 MB/s 2024-04-25T09:16:29.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.412 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253673 s, 41.3 MB/s 2024-04-25T09:16:29.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.440 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245356 s, 42.7 MB/s 2024-04-25T09:16:29.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.468 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245003 s, 42.8 MB/s 2024-04-25T09:16:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.494 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212138 s, 49.4 MB/s 2024-04-25T09:16:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247866 s, 42.3 MB/s 2024-04-25T09:16:29.548 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.549 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217066 s, 48.3 MB/s 2024-04-25T09:16:29.575 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.575 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.575 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216806 s, 48.4 MB/s 2024-04-25T09:16:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.602 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226646 s, 46.3 MB/s 2024-04-25T09:16:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.629 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230354 s, 45.5 MB/s 2024-04-25T09:16:29.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.656 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226006 s, 46.4 MB/s 2024-04-25T09:16:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.679 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019199 s, 54.6 MB/s 2024-04-25T09:16:29.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.707 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238057 s, 44.0 MB/s 2024-04-25T09:16:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.732 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019842 s, 52.8 MB/s 2024-04-25T09:16:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181483 s, 57.8 MB/s 2024-04-25T09:16:29.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.781 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226423 s, 46.3 MB/s 2024-04-25T09:16:29.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.807 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225513 s, 46.5 MB/s 2024-04-25T09:16:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.835 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228674 s, 45.9 MB/s 2024-04-25T09:16:29.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245364 s, 42.7 MB/s 2024-04-25T09:16:29.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.891 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228248 s, 45.9 MB/s 2024-04-25T09:16:29.916 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.916 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.916 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214029 s, 49.0 MB/s 2024-04-25T09:16:29.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.946 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.946 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254786 s, 41.2 MB/s 2024-04-25T09:16:29.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.971 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.971 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209478 s, 50.1 MB/s 2024-04-25T09:16:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:29.996 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208058 s, 50.4 MB/s 2024-04-25T09:16:30.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.021 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209851 s, 50.0 MB/s 2024-04-25T09:16:30.042 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.043 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184744 s, 56.8 MB/s 2024-04-25T09:16:30.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.070 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226909 s, 46.2 MB/s 2024-04-25T09:16:30.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.099 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244284 s, 42.9 MB/s 2024-04-25T09:16:30.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206307 s, 50.8 MB/s 2024-04-25T09:16:30.154 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.154 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.154 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264573 s, 39.6 MB/s 2024-04-25T09:16:30.182 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.183 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243011 s, 43.1 MB/s 2024-04-25T09:16:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222423 s, 47.1 MB/s 2024-04-25T09:16:30.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:30.211 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:30.211 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:30.211 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:16:30.211 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:30.211 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:30.212 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:30.212 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:30.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.301 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209106 s, 50.1 MB/s 2024-04-25T09:16:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.328 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241878 s, 43.4 MB/s 2024-04-25T09:16:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.359 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268526 s, 39.0 MB/s 2024-04-25T09:16:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.387 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230703 s, 45.5 MB/s 2024-04-25T09:16:30.414 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232201 s, 45.2 MB/s 2024-04-25T09:16:30.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.442 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231885 s, 45.2 MB/s 2024-04-25T09:16:30.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.473 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263949 s, 39.7 MB/s 2024-04-25T09:16:30.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.502 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234341 s, 44.7 MB/s 2024-04-25T09:16:30.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.527 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021378 s, 49.0 MB/s 2024-04-25T09:16:30.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.554 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218226 s, 48.0 MB/s 2024-04-25T09:16:30.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206928 s, 50.7 MB/s 2024-04-25T09:16:30.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.605 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.605 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022396 s, 46.8 MB/s 2024-04-25T09:16:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.630 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202929 s, 51.7 MB/s 2024-04-25T09:16:30.658 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.659 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247755 s, 42.3 MB/s 2024-04-25T09:16:30.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.683 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203956 s, 51.4 MB/s 2024-04-25T09:16:30.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.712 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251356 s, 41.7 MB/s 2024-04-25T09:16:30.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208304 s, 50.3 MB/s 2024-04-25T09:16:30.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.765 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231807 s, 45.2 MB/s 2024-04-25T09:16:30.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.789 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209893 s, 50.0 MB/s 2024-04-25T09:16:30.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.815 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215329 s, 48.7 MB/s 2024-04-25T09:16:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.840 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.841 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021378 s, 49.0 MB/s 2024-04-25T09:16:30.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244504 s, 42.9 MB/s 2024-04-25T09:16:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.895 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220955 s, 47.5 MB/s 2024-04-25T09:16:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.921 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210997 s, 49.7 MB/s 2024-04-25T09:16:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.945 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206173 s, 50.9 MB/s 2024-04-25T09:16:30.972 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.973 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233078 s, 45.0 MB/s 2024-04-25T09:16:30.998 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:30.998 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:30.998 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213005 s, 49.2 MB/s 2024-04-25T09:16:31.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.023 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.024 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215966 s, 48.6 MB/s 2024-04-25T09:16:31.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207822 s, 50.5 MB/s 2024-04-25T09:16:31.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.072 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185371 s, 56.6 MB/s 2024-04-25T09:16:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.098 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227695 s, 46.1 MB/s 2024-04-25T09:16:31.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.120 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181892 s, 57.6 MB/s 2024-04-25T09:16:31.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:31.123 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:31.123 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:31.123 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:16:31.123 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:31.123 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:31.123 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:31.123 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:31.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.215 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248214 s, 42.2 MB/s 2024-04-25T09:16:31.239 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.239 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.239 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199867 s, 52.5 MB/s 2024-04-25T09:16:31.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.269 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262245 s, 40.0 MB/s 2024-04-25T09:16:31.296 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.296 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.296 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228565 s, 45.9 MB/s 2024-04-25T09:16:31.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.326 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262725 s, 39.9 MB/s 2024-04-25T09:16:31.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.353 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217629 s, 48.2 MB/s 2024-04-25T09:16:31.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238424 s, 44.0 MB/s 2024-04-25T09:16:31.405 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.405 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.405 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020874 s, 50.2 MB/s 2024-04-25T09:16:31.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.434 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247133 s, 42.4 MB/s 2024-04-25T09:16:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237868 s, 44.1 MB/s 2024-04-25T09:16:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.493 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253246 s, 41.4 MB/s 2024-04-25T09:16:31.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.518 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211757 s, 49.5 MB/s 2024-04-25T09:16:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.543 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214631 s, 48.9 MB/s 2024-04-25T09:16:31.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232859 s, 45.0 MB/s 2024-04-25T09:16:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212777 s, 49.3 MB/s 2024-04-25T09:16:31.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228132 s, 46.0 MB/s 2024-04-25T09:16:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218922 s, 47.9 MB/s 2024-04-25T09:16:31.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.679 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250453 s, 41.9 MB/s 2024-04-25T09:16:31.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.704 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.704 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212367 s, 49.4 MB/s 2024-04-25T09:16:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.731 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226737 s, 46.2 MB/s 2024-04-25T09:16:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.761 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256255 s, 40.9 MB/s 2024-04-25T09:16:31.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.787 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021981 s, 47.7 MB/s 2024-04-25T09:16:31.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.815 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236724 s, 44.3 MB/s 2024-04-25T09:16:31.843 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.843 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.843 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024305 s, 43.1 MB/s 2024-04-25T09:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.870 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229687 s, 45.7 MB/s 2024-04-25T09:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.898 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243283 s, 43.1 MB/s 2024-04-25T09:16:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.921 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188184 s, 55.7 MB/s 2024-04-25T09:16:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.943 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193496 s, 54.2 MB/s 2024-04-25T09:16:31.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.970 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217964 s, 48.1 MB/s 2024-04-25T09:16:31.998 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:31.998 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:31.998 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234415 s, 44.7 MB/s 2024-04-25T09:16:32.024 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.025 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.025 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236702 s, 44.3 MB/s 2024-04-25T09:16:32.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.048 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205402 s, 51.0 MB/s 2024-04-25T09:16:32.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:32.051 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:32.051 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:32.051 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:16:32.051 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:32.051 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:32.051 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:32.051 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181089 s, 57.9 MB/s 2024-04-25T09:16:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190008 s, 55.2 MB/s 2024-04-25T09:16:32.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.178 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242191 s, 43.3 MB/s 2024-04-25T09:16:32.205 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.205 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.205 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232495 s, 45.1 MB/s 2024-04-25T09:16:32.231 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.232 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226692 s, 46.3 MB/s 2024-04-25T09:16:32.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.261 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249649 s, 42.0 MB/s 2024-04-25T09:16:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.287 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221947 s, 47.2 MB/s 2024-04-25T09:16:32.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259286 s, 40.4 MB/s 2024-04-25T09:16:32.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.350 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266421 s, 39.4 MB/s 2024-04-25T09:16:32.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257709 s, 40.7 MB/s 2024-04-25T09:16:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.406 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218884 s, 47.9 MB/s 2024-04-25T09:16:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.432 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221837 s, 47.3 MB/s 2024-04-25T09:16:32.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.460 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234717 s, 44.7 MB/s 2024-04-25T09:16:32.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.583 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238826 s, 43.9 MB/s 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208853 s, 50.2 MB/s 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225445 s, 46.5 MB/s 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212618 s, 49.3 MB/s 2024-04-25T09:16:32.589 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.589 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.589 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216787 s, 48.4 MB/s 2024-04-25T09:16:32.615 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.615 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.615 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216452 s, 48.4 MB/s 2024-04-25T09:16:32.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020496 s, 51.2 MB/s 2024-04-25T09:16:32.666 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.666 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.666 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219167 s, 47.8 MB/s 2024-04-25T09:16:32.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.688 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195828 s, 53.5 MB/s 2024-04-25T09:16:32.715 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.715 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.715 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212659 s, 49.3 MB/s 2024-04-25T09:16:32.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.743 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241523 s, 43.4 MB/s 2024-04-25T09:16:32.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.767 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195899 s, 53.5 MB/s 2024-04-25T09:16:32.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.792 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217121 s, 48.3 MB/s 2024-04-25T09:16:32.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.819 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231639 s, 45.3 MB/s 2024-04-25T09:16:32.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.851 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267658 s, 39.2 MB/s 2024-04-25T09:16:32.875 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.875 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.875 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201583 s, 52.0 MB/s 2024-04-25T09:16:32.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.899 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199735 s, 52.5 MB/s 2024-04-25T09:16:32.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231423 s, 45.3 MB/s 2024-04-25T09:16:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:32.953 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237039 s, 44.2 MB/s 2024-04-25T09:16:32.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:32.955 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:32.955 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:32.955 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:16:32.955 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:32.955 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:32.955 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:32.956 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:33.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.038 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178167 s, 58.9 MB/s 2024-04-25T09:16:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210809 s, 49.7 MB/s 2024-04-25T09:16:33.085 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.085 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.085 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198398 s, 52.9 MB/s 2024-04-25T09:16:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228978 s, 45.8 MB/s 2024-04-25T09:16:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.137 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208085 s, 50.4 MB/s 2024-04-25T09:16:33.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.164 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236573 s, 44.3 MB/s 2024-04-25T09:16:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213338 s, 49.2 MB/s 2024-04-25T09:16:33.214 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.214 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.214 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197303 s, 53.1 MB/s 2024-04-25T09:16:33.238 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.238 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.238 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206244 s, 50.8 MB/s 2024-04-25T09:16:33.264 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.264 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.264 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221308 s, 47.4 MB/s 2024-04-25T09:16:33.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.292 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245872 s, 42.6 MB/s 2024-04-25T09:16:33.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266291 s, 39.4 MB/s 2024-04-25T09:16:33.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.350 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021596 s, 48.6 MB/s 2024-04-25T09:16:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.378 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253163 s, 41.4 MB/s 2024-04-25T09:16:33.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257121 s, 40.8 MB/s 2024-04-25T09:16:33.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.440 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278888 s, 37.6 MB/s 2024-04-25T09:16:33.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.469 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.469 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254846 s, 41.1 MB/s 2024-04-25T09:16:33.495 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.496 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.496 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226323 s, 46.3 MB/s 2024-04-25T09:16:33.521 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.521 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.521 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209428 s, 50.1 MB/s 2024-04-25T09:16:33.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.546 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.546 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217336 s, 48.2 MB/s 2024-04-25T09:16:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224633 s, 46.7 MB/s 2024-04-25T09:16:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.599 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216673 s, 48.4 MB/s 2024-04-25T09:16:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.620 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176621 s, 59.4 MB/s 2024-04-25T09:16:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023212 s, 45.2 MB/s 2024-04-25T09:16:33.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.672 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214027 s, 49.0 MB/s 2024-04-25T09:16:33.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205492 s, 51.0 MB/s 2024-04-25T09:16:33.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.720 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196077 s, 53.5 MB/s 2024-04-25T09:16:33.744 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.745 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.745 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205022 s, 51.1 MB/s 2024-04-25T09:16:33.769 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.769 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.769 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199103 s, 52.7 MB/s 2024-04-25T09:16:33.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.794 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215934 s, 48.6 MB/s 2024-04-25T09:16:33.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.819 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208645 s, 50.3 MB/s 2024-04-25T09:16:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230914 s, 45.4 MB/s 2024-04-25T09:16:33.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:33.848 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:33.848 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:33.848 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:16:33.849 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:33.849 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:33.849 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:33.849 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:33.931 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.932 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197212 s, 53.2 MB/s 2024-04-25T09:16:33.957 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.957 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.957 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213164 s, 49.2 MB/s 2024-04-25T09:16:33.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:33.982 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:33.982 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208126 s, 50.4 MB/s 2024-04-25T09:16:34.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.008 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218138 s, 48.1 MB/s 2024-04-25T09:16:34.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206085 s, 50.9 MB/s 2024-04-25T09:16:34.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.055 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.055 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191622 s, 54.7 MB/s 2024-04-25T09:16:34.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.085 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258251 s, 40.6 MB/s 2024-04-25T09:16:34.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.112 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.112 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223471 s, 46.9 MB/s 2024-04-25T09:16:34.142 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.142 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.142 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259246 s, 40.4 MB/s 2024-04-25T09:16:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.168 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022179 s, 47.3 MB/s 2024-04-25T09:16:34.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.195 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223141 s, 47.0 MB/s 2024-04-25T09:16:34.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.222 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221093 s, 47.4 MB/s 2024-04-25T09:16:34.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.247 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207086 s, 50.6 MB/s 2024-04-25T09:16:34.274 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.274 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.274 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229375 s, 45.7 MB/s 2024-04-25T09:16:34.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266213 s, 39.4 MB/s 2024-04-25T09:16:34.333 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.334 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.334 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024506 s, 42.8 MB/s 2024-04-25T09:16:34.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.366 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273828 s, 38.3 MB/s 2024-04-25T09:16:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.396 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261695 s, 40.1 MB/s 2024-04-25T09:16:34.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234009 s, 44.8 MB/s 2024-04-25T09:16:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.447 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201096 s, 52.1 MB/s 2024-04-25T09:16:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.474 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229608 s, 45.7 MB/s 2024-04-25T09:16:34.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.498 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203551 s, 51.5 MB/s 2024-04-25T09:16:34.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.523 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211742 s, 49.5 MB/s 2024-04-25T09:16:34.548 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.548 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.548 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221706 s, 47.3 MB/s 2024-04-25T09:16:34.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.574 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021446 s, 48.9 MB/s 2024-04-25T09:16:34.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.600 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223479 s, 46.9 MB/s 2024-04-25T09:16:34.627 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.627 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.627 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221414 s, 47.4 MB/s 2024-04-25T09:16:34.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185323 s, 56.6 MB/s 2024-04-25T09:16:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.678 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.678 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023473 s, 44.7 MB/s 2024-04-25T09:16:34.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.707 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245972 s, 42.6 MB/s 2024-04-25T09:16:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.731 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206765 s, 50.7 MB/s 2024-04-25T09:16:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.760 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250172 s, 41.9 MB/s 2024-04-25T09:16:34.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:34.762 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:34.763 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:34.763 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:16:34.763 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:34.763 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:34.763 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:34.763 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:34.849 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.849 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.849 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220189 s, 47.6 MB/s 2024-04-25T09:16:34.874 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.875 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.875 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219552 s, 47.8 MB/s 2024-04-25T09:16:34.899 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.899 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.899 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203266 s, 51.6 MB/s 2024-04-25T09:16:34.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.925 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217326 s, 48.2 MB/s 2024-04-25T09:16:34.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.953 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.953 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243889 s, 43.0 MB/s 2024-04-25T09:16:34.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:34.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:34.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225944 s, 46.4 MB/s 2024-04-25T09:16:35.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.008 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.008 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227699 s, 46.1 MB/s 2024-04-25T09:16:35.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.033 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208481 s, 50.3 MB/s 2024-04-25T09:16:35.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.063 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250612 s, 41.8 MB/s 2024-04-25T09:16:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224608 s, 46.7 MB/s 2024-04-25T09:16:35.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.115 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224544 s, 46.7 MB/s 2024-04-25T09:16:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.141 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210421 s, 49.8 MB/s 2024-04-25T09:16:35.169 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.169 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.169 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236083 s, 44.4 MB/s 2024-04-25T09:16:35.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.195 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215586 s, 48.6 MB/s 2024-04-25T09:16:35.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.220 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208654 s, 50.3 MB/s 2024-04-25T09:16:35.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.246 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.247 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220165 s, 47.6 MB/s 2024-04-25T09:16:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.268 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187615 s, 55.9 MB/s 2024-04-25T09:16:35.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.298 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258556 s, 40.6 MB/s 2024-04-25T09:16:35.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.326 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.326 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240663 s, 43.6 MB/s 2024-04-25T09:16:35.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.352 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217887 s, 48.1 MB/s 2024-04-25T09:16:35.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.385 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285501 s, 36.7 MB/s 2024-04-25T09:16:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231679 s, 45.3 MB/s 2024-04-25T09:16:35.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.440 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.440 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233684 s, 44.9 MB/s 2024-04-25T09:16:35.466 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.467 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.467 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228127 s, 46.0 MB/s 2024-04-25T09:16:35.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.492 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215769 s, 48.6 MB/s 2024-04-25T09:16:35.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.517 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207693 s, 50.5 MB/s 2024-04-25T09:16:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.541 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196209 s, 53.4 MB/s 2024-04-25T09:16:35.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.571 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264688 s, 39.6 MB/s 2024-04-25T09:16:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215759 s, 48.6 MB/s 2024-04-25T09:16:35.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.623 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.623 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219358 s, 47.8 MB/s 2024-04-25T09:16:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207766 s, 50.5 MB/s 2024-04-25T09:16:35.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.673 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211116 s, 49.7 MB/s 2024-04-25T09:16:35.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:35.676 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:35.676 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:35.676 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:16:35.676 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:35.676 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:35.676 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:35.676 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.763 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246283 s, 42.6 MB/s 2024-04-25T09:16:35.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.785 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190201 s, 55.1 MB/s 2024-04-25T09:16:35.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.807 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.808 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191564 s, 54.7 MB/s 2024-04-25T09:16:35.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.834 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.835 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229156 s, 45.8 MB/s 2024-04-25T09:16:35.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205305 s, 51.1 MB/s 2024-04-25T09:16:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.888 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254296 s, 41.2 MB/s 2024-04-25T09:16:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223179 s, 47.0 MB/s 2024-04-25T09:16:35.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.942 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234823 s, 44.7 MB/s 2024-04-25T09:16:35.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.974 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271677 s, 38.6 MB/s 2024-04-25T09:16:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:35.999 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209783 s, 50.0 MB/s 2024-04-25T09:16:36.026 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.026 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.026 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228019 s, 46.0 MB/s 2024-04-25T09:16:36.053 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.053 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.053 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233749 s, 44.9 MB/s 2024-04-25T09:16:36.080 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.080 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.080 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218885 s, 47.9 MB/s 2024-04-25T09:16:36.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.107 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228361 s, 45.9 MB/s 2024-04-25T09:16:36.135 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.135 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.135 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238711 s, 43.9 MB/s 2024-04-25T09:16:36.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.160 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.160 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205236 s, 51.1 MB/s 2024-04-25T09:16:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202215 s, 51.9 MB/s 2024-04-25T09:16:36.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.209 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209682 s, 50.0 MB/s 2024-04-25T09:16:36.235 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.235 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.235 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220609 s, 47.5 MB/s 2024-04-25T09:16:36.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.263 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.263 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244124 s, 43.0 MB/s 2024-04-25T09:16:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211642 s, 49.5 MB/s 2024-04-25T09:16:36.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.320 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262587 s, 39.9 MB/s 2024-04-25T09:16:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.349 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248674 s, 42.2 MB/s 2024-04-25T09:16:36.378 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.378 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.378 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250802 s, 41.8 MB/s 2024-04-25T09:16:36.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.409 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266798 s, 39.3 MB/s 2024-04-25T09:16:36.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.439 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255007 s, 41.1 MB/s 2024-04-25T09:16:36.467 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.467 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.467 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241593 s, 43.4 MB/s 2024-04-25T09:16:36.501 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.501 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.501 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298004 s, 35.2 MB/s 2024-04-25T09:16:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.529 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233808 s, 44.8 MB/s 2024-04-25T09:16:36.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224854 s, 46.6 MB/s 2024-04-25T09:16:36.582 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.583 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219767 s, 47.7 MB/s 2024-04-25T09:16:36.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.608 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209576 s, 50.0 MB/s 2024-04-25T09:16:36.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:36.610 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:36.611 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:36.611 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:16:36.611 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:36.611 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:36.611 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:36.611 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.698 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233769 s, 44.9 MB/s 2024-04-25T09:16:36.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.726 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233126 s, 45.0 MB/s 2024-04-25T09:16:36.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.758 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268989 s, 39.0 MB/s 2024-04-25T09:16:36.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.784 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.784 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223287 s, 47.0 MB/s 2024-04-25T09:16:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225158 s, 46.6 MB/s 2024-04-25T09:16:36.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.835 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020315 s, 51.6 MB/s 2024-04-25T09:16:36.862 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.862 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.862 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225753 s, 46.4 MB/s 2024-04-25T09:16:36.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021284 s, 49.3 MB/s 2024-04-25T09:16:36.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.915 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224703 s, 46.7 MB/s 2024-04-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.942 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227489 s, 46.1 MB/s 2024-04-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255992 s, 41.0 MB/s 2024-04-25T09:16:37.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.005 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273545 s, 38.3 MB/s 2024-04-25T09:16:37.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.030 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.030 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197631 s, 53.1 MB/s 2024-04-25T09:16:37.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244072 s, 43.0 MB/s 2024-04-25T09:16:37.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.082 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205064 s, 51.1 MB/s 2024-04-25T09:16:37.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176207 s, 59.5 MB/s 2024-04-25T09:16:37.127 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.128 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.128 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211849 s, 49.5 MB/s 2024-04-25T09:16:37.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212416 s, 49.4 MB/s 2024-04-25T09:16:37.177 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.177 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.177 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021092 s, 49.7 MB/s 2024-04-25T09:16:37.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.203 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021237 s, 49.4 MB/s 2024-04-25T09:16:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.230 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022143 s, 47.4 MB/s 2024-04-25T09:16:37.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.256 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.256 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021995 s, 47.7 MB/s 2024-04-25T09:16:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246571 s, 42.5 MB/s 2024-04-25T09:16:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264904 s, 39.6 MB/s 2024-04-25T09:16:37.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.341 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.341 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020328 s, 51.6 MB/s 2024-04-25T09:16:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.369 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244487 s, 42.9 MB/s 2024-04-25T09:16:37.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.401 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.401 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256805 s, 40.8 MB/s 2024-04-25T09:16:37.427 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.427 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.427 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224637 s, 46.7 MB/s 2024-04-25T09:16:37.457 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.457 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.457 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02446 s, 42.9 MB/s 2024-04-25T09:16:37.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.483 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02223 s, 47.2 MB/s 2024-04-25T09:16:37.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.508 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207512 s, 50.5 MB/s 2024-04-25T09:16:37.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.531 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190987 s, 54.9 MB/s 2024-04-25T09:16:37.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:37.534 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:37.534 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:37.534 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:16:37.534 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:37.534 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:37.534 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:37.534 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:37.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222957 s, 47.0 MB/s 2024-04-25T09:16:37.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.648 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204616 s, 51.2 MB/s 2024-04-25T09:16:37.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.673 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022975 s, 45.6 MB/s 2024-04-25T09:16:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224956 s, 46.6 MB/s 2024-04-25T09:16:37.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.726 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228086 s, 46.0 MB/s 2024-04-25T09:16:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.751 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02119 s, 49.5 MB/s 2024-04-25T09:16:37.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.776 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202446 s, 51.8 MB/s 2024-04-25T09:16:37.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.802 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224989 s, 46.6 MB/s 2024-04-25T09:16:37.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019615 s, 53.5 MB/s 2024-04-25T09:16:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.848 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187095 s, 56.0 MB/s 2024-04-25T09:16:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204905 s, 51.2 MB/s 2024-04-25T09:16:37.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212166 s, 49.4 MB/s 2024-04-25T09:16:37.924 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227961 s, 46.0 MB/s 2024-04-25T09:16:37.954 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.954 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.954 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251125 s, 41.8 MB/s 2024-04-25T09:16:37.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:37.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:37.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021697 s, 48.3 MB/s 2024-04-25T09:16:38.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.005 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021145 s, 49.6 MB/s 2024-04-25T09:16:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.032 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224385 s, 46.7 MB/s 2024-04-25T09:16:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.064 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275795 s, 38.0 MB/s 2024-04-25T09:16:38.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216545 s, 48.4 MB/s 2024-04-25T09:16:38.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.117 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223323 s, 47.0 MB/s 2024-04-25T09:16:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.143 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228162 s, 46.0 MB/s 2024-04-25T09:16:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.168 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207611 s, 50.5 MB/s 2024-04-25T09:16:38.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.194 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219767 s, 47.7 MB/s 2024-04-25T09:16:38.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.223 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.223 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249176 s, 42.1 MB/s 2024-04-25T09:16:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.251 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227894 s, 46.0 MB/s 2024-04-25T09:16:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.277 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022014 s, 47.6 MB/s 2024-04-25T09:16:38.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.306 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239824 s, 43.7 MB/s 2024-04-25T09:16:38.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.339 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284532 s, 36.9 MB/s 2024-04-25T09:16:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.369 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025016 s, 41.9 MB/s 2024-04-25T09:16:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.391 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018585 s, 56.4 MB/s 2024-04-25T09:16:38.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.415 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.415 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214121 s, 49.0 MB/s 2024-04-25T09:16:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.443 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236398 s, 44.4 MB/s 2024-04-25T09:16:38.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:38.446 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:38.446 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:38.446 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:16:38.446 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:38.446 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:38.446 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:38.446 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:38.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.533 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.533 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210489 s, 49.8 MB/s 2024-04-25T09:16:38.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.559 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225634 s, 46.5 MB/s 2024-04-25T09:16:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.582 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191637 s, 54.7 MB/s 2024-04-25T09:16:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.609 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226564 s, 46.3 MB/s 2024-04-25T09:16:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.636 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023247 s, 45.1 MB/s 2024-04-25T09:16:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.659 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196693 s, 53.3 MB/s 2024-04-25T09:16:38.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.684 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205455 s, 51.0 MB/s 2024-04-25T09:16:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.709 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215171 s, 48.7 MB/s 2024-04-25T09:16:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.736 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228757 s, 45.8 MB/s 2024-04-25T09:16:38.763 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.764 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227908 s, 46.0 MB/s 2024-04-25T09:16:38.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.793 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242112 s, 43.3 MB/s 2024-04-25T09:16:38.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.816 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.817 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206628 s, 50.7 MB/s 2024-04-25T09:16:38.843 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.844 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.844 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228285 s, 45.9 MB/s 2024-04-25T09:16:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.872 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244619 s, 42.9 MB/s 2024-04-25T09:16:38.899 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.900 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236292 s, 44.4 MB/s 2024-04-25T09:16:38.930 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.931 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268517 s, 39.1 MB/s 2024-04-25T09:16:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.961 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255385 s, 41.1 MB/s 2024-04-25T09:16:38.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:38.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:38.988 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225203 s, 46.6 MB/s 2024-04-25T09:16:39.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.012 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196549 s, 53.3 MB/s 2024-04-25T09:16:39.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.040 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240908 s, 43.5 MB/s 2024-04-25T09:16:39.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.065 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.065 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213568 s, 49.1 MB/s 2024-04-25T09:16:39.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.090 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212713 s, 49.3 MB/s 2024-04-25T09:16:39.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222966 s, 47.0 MB/s 2024-04-25T09:16:39.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.145 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247089 s, 42.4 MB/s 2024-04-25T09:16:39.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.170 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200521 s, 52.3 MB/s 2024-04-25T09:16:39.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.194 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.194 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199909 s, 52.5 MB/s 2024-04-25T09:16:39.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.226 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270701 s, 38.7 MB/s 2024-04-25T09:16:39.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.252 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220156 s, 47.6 MB/s 2024-04-25T09:16:39.281 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.282 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025004 s, 41.9 MB/s 2024-04-25T09:16:39.310 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.311 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.311 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250768 s, 41.8 MB/s 2024-04-25T09:16:39.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.339 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.339 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245012 s, 42.8 MB/s 2024-04-25T09:16:39.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.370 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256766 s, 40.8 MB/s 2024-04-25T09:16:39.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:39.372 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:39.372 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:39.372 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:16:39.373 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:39.373 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:39.373 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:39.373 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:39.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.459 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224408 s, 46.7 MB/s 2024-04-25T09:16:39.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.489 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261403 s, 40.1 MB/s 2024-04-25T09:16:39.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.513 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192275 s, 54.5 MB/s 2024-04-25T09:16:39.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.538 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215885 s, 48.6 MB/s 2024-04-25T09:16:39.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.567 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.567 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246176 s, 42.6 MB/s 2024-04-25T09:16:39.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213458 s, 49.1 MB/s 2024-04-25T09:16:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225644 s, 46.5 MB/s 2024-04-25T09:16:39.647 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.648 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237936 s, 44.1 MB/s 2024-04-25T09:16:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228827 s, 45.8 MB/s 2024-04-25T09:16:39.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.701 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210883 s, 49.7 MB/s 2024-04-25T09:16:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.726 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207249 s, 50.6 MB/s 2024-04-25T09:16:39.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.753 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225893 s, 46.4 MB/s 2024-04-25T09:16:39.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022187 s, 47.3 MB/s 2024-04-25T09:16:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215214 s, 48.7 MB/s 2024-04-25T09:16:39.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201128 s, 52.1 MB/s 2024-04-25T09:16:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.858 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231499 s, 45.3 MB/s 2024-04-25T09:16:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203477 s, 51.5 MB/s 2024-04-25T09:16:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222002 s, 47.2 MB/s 2024-04-25T09:16:39.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.933 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211625 s, 49.5 MB/s 2024-04-25T09:16:39.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.961 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234834 s, 44.7 MB/s 2024-04-25T09:16:39.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:39.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:39.988 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223013 s, 47.0 MB/s 2024-04-25T09:16:40.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204998 s, 51.2 MB/s 2024-04-25T09:16:40.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212519 s, 49.3 MB/s 2024-04-25T09:16:40.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209086 s, 50.2 MB/s 2024-04-25T09:16:40.092 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.092 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.092 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258058 s, 40.6 MB/s 2024-04-25T09:16:40.119 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.119 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.119 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232806 s, 45.0 MB/s 2024-04-25T09:16:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239274 s, 43.8 MB/s 2024-04-25T09:16:40.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022914 s, 45.8 MB/s 2024-04-25T09:16:40.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.199 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196326 s, 53.4 MB/s 2024-04-25T09:16:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218605 s, 48.0 MB/s 2024-04-25T09:16:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.251 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220813 s, 47.5 MB/s 2024-04-25T09:16:40.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.280 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249134 s, 42.1 MB/s 2024-04-25T09:16:40.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:40.283 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:40.283 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:40.283 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:16:40.283 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:40.284 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:40.284 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:40.284 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:40.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.371 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228888 s, 45.8 MB/s 2024-04-25T09:16:40.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.401 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.401 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252789 s, 41.5 MB/s 2024-04-25T09:16:40.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.425 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.425 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208622 s, 50.3 MB/s 2024-04-25T09:16:40.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.453 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.453 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242392 s, 43.3 MB/s 2024-04-25T09:16:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.480 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217913 s, 48.1 MB/s 2024-04-25T09:16:40.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.508 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240092 s, 43.7 MB/s 2024-04-25T09:16:40.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.539 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255811 s, 41.0 MB/s 2024-04-25T09:16:40.565 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.565 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.565 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227562 s, 46.1 MB/s 2024-04-25T09:16:40.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.593 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.593 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240403 s, 43.6 MB/s 2024-04-25T09:16:40.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219012 s, 47.9 MB/s 2024-04-25T09:16:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.644 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198092 s, 52.9 MB/s 2024-04-25T09:16:40.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.671 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243471 s, 43.1 MB/s 2024-04-25T09:16:40.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213107 s, 49.2 MB/s 2024-04-25T09:16:40.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.722 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209003 s, 50.2 MB/s 2024-04-25T09:16:40.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212527 s, 49.3 MB/s 2024-04-25T09:16:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233647 s, 44.9 MB/s 2024-04-25T09:16:40.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.805 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259991 s, 40.3 MB/s 2024-04-25T09:16:40.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.831 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.831 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223312 s, 47.0 MB/s 2024-04-25T09:16:40.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.858 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233607 s, 44.9 MB/s 2024-04-25T09:16:40.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.884 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219568 s, 47.8 MB/s 2024-04-25T09:16:40.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.909 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197855 s, 53.0 MB/s 2024-04-25T09:16:40.933 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.934 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197537 s, 53.1 MB/s 2024-04-25T09:16:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.959 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200278 s, 52.4 MB/s 2024-04-25T09:16:40.984 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:40.985 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:40.985 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02106 s, 49.8 MB/s 2024-04-25T09:16:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241744 s, 43.4 MB/s 2024-04-25T09:16:41.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.044 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262651 s, 39.9 MB/s 2024-04-25T09:16:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.067 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198829 s, 52.7 MB/s 2024-04-25T09:16:41.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.088 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179357 s, 58.5 MB/s 2024-04-25T09:16:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243988 s, 43.0 MB/s 2024-04-25T09:16:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.145 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231846 s, 45.2 MB/s 2024-04-25T09:16:41.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257673 s, 40.7 MB/s 2024-04-25T09:16:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221495 s, 47.3 MB/s 2024-04-25T09:16:41.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:41.207 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:16:41.207 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:16:41.207 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:16:41.207 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:16:41.207 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:16:41.207 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:16:41.207 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:16:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.299 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249999 s, 41.9 MB/s 2024-04-25T09:16:41.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.323 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.323 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204284 s, 51.3 MB/s 2024-04-25T09:16:41.353 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.354 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255404 s, 41.1 MB/s 2024-04-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.381 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236619 s, 44.3 MB/s 2024-04-25T09:16:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.410 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240751 s, 43.6 MB/s 2024-04-25T09:16:41.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.439 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024657 s, 42.5 MB/s 2024-04-25T09:16:41.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.465 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223146 s, 47.0 MB/s 2024-04-25T09:16:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.489 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201319 s, 52.1 MB/s 2024-04-25T09:16:41.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.519 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250066 s, 41.9 MB/s 2024-04-25T09:16:41.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.544 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215029 s, 48.8 MB/s 2024-04-25T09:16:41.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.571 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229702 s, 45.6 MB/s 2024-04-25T09:16:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209199 s, 50.1 MB/s 2024-04-25T09:16:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.622 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220957 s, 47.5 MB/s 2024-04-25T09:16:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.645 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190587 s, 55.0 MB/s 2024-04-25T09:16:41.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.671 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022446 s, 46.7 MB/s 2024-04-25T09:16:41.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205044 s, 51.1 MB/s 2024-04-25T09:16:41.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.722 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216225 s, 48.5 MB/s 2024-04-25T09:16:41.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.746 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019507 s, 53.8 MB/s 2024-04-25T09:16:41.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231278 s, 45.3 MB/s 2024-04-25T09:16:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.799 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211159 s, 49.7 MB/s 2024-04-25T09:16:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231311 s, 45.3 MB/s 2024-04-25T09:16:41.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.852 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212072 s, 49.4 MB/s 2024-04-25T09:16:41.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.877 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201015 s, 52.2 MB/s 2024-04-25T09:16:41.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.901 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182352 s, 57.5 MB/s 2024-04-25T09:16:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209876 s, 50.0 MB/s 2024-04-25T09:16:41.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.953 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217811 s, 48.1 MB/s 2024-04-25T09:16:41.984 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:41.984 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:41.984 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275174 s, 38.1 MB/s 2024-04-25T09:16:42.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:42.010 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208534 s, 50.3 MB/s 2024-04-25T09:16:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:42.040 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258915 s, 40.5 MB/s 2024-04-25T09:16:42.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:42.066 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:42.066 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225647 s, 46.5 MB/s 2024-04-25T09:16:42.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:42.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:42.091 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207164 s, 50.6 MB/s 2024-04-25T09:16:42.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:16:42.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:16:42.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190987 s, 54.9 MB/s 2024-04-25T09:16:42.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:42.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:16:42.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.344+0000 7f32d7a00640 1 Processor -- start 2024-04-25T09:16:42.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.344+0000 7f32d7a00640 1 -- start start 2024-04-25T09:16:42.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.346+0000 7f32d7a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f32d00705f0 0x7f32d006c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.346+0000 7f32d7a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d0163840 con 0x7f32d015c6e0 2024-04-25T09:16:42.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.346+0000 7f32d7a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00b4700 con 0x7f32d006c7c0 2024-04-25T09:16:42.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.346+0000 7f32d7a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d00b44b0 con 0x7f32d00705f0 2024-04-25T09:16:42.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.346+0000 7f32cfe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f32d00705f0 0x7f32d006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.346+0000 7f32d5000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f32d015c6e0 0x7f32d0163100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:51052/0 (socket says 172.21.15.62:51052) 2024-04-25T09:16:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.346+0000 7f32d5000640 1 -- 172.21.15.62:0/3596840025 learned_addr learned my addr 172.21.15.62:0/3596840025 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.347+0000 7f32cfe00640 1 -- 172.21.15.62:0/3596840025 >> v1:172.21.15.121:6789/0 conn(0x7f32d006c7c0 legacy=0x7f32d00b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.347+0000 7f32cfe00640 1 -- 172.21.15.62:0/3596840025 >> v1:172.21.15.62:6789/0 conn(0x7f32d015c6e0 legacy=0x7f32d0163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.347+0000 7f32cfe00640 1 -- 172.21.15.62:0/3596840025 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00b49e0 con 0x7f32d00705f0 2024-04-25T09:16:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.348+0000 7f32cfe00640 1 --2- 172.21.15.62:0/3596840025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d00705f0 0x7f32d006c280 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009d90 tx=0x7f32b802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4ce23a79ffb1eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.348+0000 7f32cf400640 1 -- 172.21.15.62:0/3596840025 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8031600 con 0x7f32d00705f0 2024-04-25T09:16:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.348+0000 7f32cf400640 1 -- 172.21.15.62:0/3596840025 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b80317a0 con 0x7f32d00705f0 2024-04-25T09:16:42.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.348+0000 7f32cf400640 1 -- 172.21.15.62:0/3596840025 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8031b70 con 0x7f32d00705f0 2024-04-25T09:16:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.349+0000 7f32d7a00640 1 -- 172.21.15.62:0/3596840025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d00705f0 msgr2=0x7f32d006c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.349+0000 7f32d7a00640 1 --2- 172.21.15.62:0/3596840025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d00705f0 0x7f32d006c280 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009d90 tx=0x7f32b802a610 comp rx=0 tx=0).stop 2024-04-25T09:16:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.349+0000 7f32d7a00640 1 -- 172.21.15.62:0/3596840025 shutdown_connections 2024-04-25T09:16:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.349+0000 7f32d7a00640 1 --2- 172.21.15.62:0/3596840025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d00705f0 0x7f32d006c280 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.349+0000 7f32d7a00640 1 -- 172.21.15.62:0/3596840025 >> 172.21.15.62:0/3596840025 conn(0x7f32d00668f0 msgr2=0x7f32d0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.349+0000 7f32d7a00640 1 -- 172.21.15.62:0/3596840025 shutdown_connections 2024-04-25T09:16:42.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.349+0000 7f32d7a00640 1 -- 172.21.15.62:0/3596840025 wait complete. 2024-04-25T09:16:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.351+0000 7f32d7a00640 1 Processor -- start 2024-04-25T09:16:42.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.351+0000 7f32d7a00640 1 -- start start 2024-04-25T09:16:42.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d7a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d006c7c0 0x7f32d0150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d7a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d00705f0 0x7f32d015cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d7a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d015c6e0 0x7f32d01612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d7a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015cac0 con 0x7f32d015c6e0 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d7a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0072370 con 0x7f32d006c7c0 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d7a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0071d40 con 0x7f32d00705f0 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d015c6e0 0x7f32d01612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d015c6e0 0x7f32d01612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45272/0 (socket says 172.21.15.62:45272) 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.352+0000 7f32d5a00640 1 -- 172.21.15.62:0/1443813535 learned_addr learned my addr 172.21.15.62:0/1443813535 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32d5000640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d006c7c0 0x7f32d0150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32cfe00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d00705f0 0x7f32d015cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32d5a00640 1 -- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d00705f0 msgr2=0x7f32d015cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32d5a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d00705f0 0x7f32d015cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32d5a00640 1 -- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d006c7c0 msgr2=0x7f32d0150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32d5a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d006c7c0 0x7f32d0150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32d5a00640 1 -- 172.21.15.62:0/1443813535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d015dec0 con 0x7f32d015c6e0 2024-04-25T09:16:42.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32d5a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d015c6e0 0x7f32d01612c0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f32c800b410 tx=0x7f32c800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32ccc00640 1 -- 172.21.15.62:0/1443813535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8015030 con 0x7f32d015c6e0 2024-04-25T09:16:42.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32ccc00640 1 -- 172.21.15.62:0/1443813535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8004020 con 0x7f32d015c6e0 2024-04-25T09:16:42.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32ccc00640 1 -- 172.21.15.62:0/1443813535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8004410 con 0x7f32d015c6e0 2024-04-25T09:16:42.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.353+0000 7f32d7a00640 1 -- 172.21.15.62:0/1443813535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d015d620 con 0x7f32d015c6e0 2024-04-25T09:16:42.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.354+0000 7f32d7a00640 1 -- 172.21.15.62:0/1443813535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d015dbe0 con 0x7f32d015c6e0 2024-04-25T09:16:42.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.356+0000 7f32ccc00640 1 -- 172.21.15.62:0/1443813535 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32c8018070 con 0x7f32d015c6e0 2024-04-25T09:16:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.357+0000 7f32ccc00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f32ac06b4d0 0x7f32ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.357+0000 7f32ccc00640 1 -- 172.21.15.62:0/1443813535 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32c8089d50 con 0x7f32d015c6e0 2024-04-25T09:16:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.358+0000 7f32d5000640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f32ac06b4d0 0x7f32ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.358+0000 7f32da80c740 1 Processor -- start 2024-04-25T09:16:42.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.358+0000 7f32da80c740 1 -- start start 2024-04-25T09:16:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32da80c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa197b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32d5000640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f32ac06b4d0 0x7f32ac06d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f32c0000f60 tx=0x7f32c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32da80c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa1998bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32da80c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1950ab0 0x55baa1996d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32da80c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baa18aef00 con 0x55baa197add0 2024-04-25T09:16:42.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32da80c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baa18b3f50 con 0x55baa1950ab0 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32da80c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baa185d0b0 con 0x55baa1998670 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32d5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa197b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32d5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa197b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45288/0 (socket says 172.21.15.62:45288) 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32d5000640 1 -- 172.21.15.62:0/203929489 learned_addr learned my addr 172.21.15.62:0/203929489 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32d5a00640 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1950ab0 0x55baa1996d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32cfe00640 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa1998bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.359+0000 7f32d5000640 1 -- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 msgr2=0x55baa1998bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32d5000640 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa1998bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32d5000640 1 -- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1950ab0 msgr2=0x55baa1996d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32d5000640 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1950ab0 0x55baa1996d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32d5000640 1 -- 172.21.15.62:0/203929489 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baa1694ba0 con 0x55baa197add0 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32cfe00640 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa1998bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32d5000640 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa197b1b0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005a90 tx=0x7f32c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6296591a6f0fbce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32c5600640 1 -- 172.21.15.62:0/203929489 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0007710 con 0x55baa197add0 2024-04-25T09:16:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32c5600640 1 -- 172.21.15.62:0/203929489 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000e740 con 0x55baa197add0 2024-04-25T09:16:42.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.360+0000 7f32c5600640 1 -- 172.21.15.62:0/203929489 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000eaf0 con 0x55baa197add0 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 -- 172.21.15.62:0/203929489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 msgr2=0x55baa197b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa197b1b0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005a90 tx=0x7f32c0011000 comp rx=0 tx=0).stop 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 -- 172.21.15.62:0/203929489 shutdown_connections 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1950ab0 0x55baa1996d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa1998bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 --2- 172.21.15.62:0/203929489 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa197b1b0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 -- 172.21.15.62:0/203929489 >> 172.21.15.62:0/203929489 conn(0x55baa194d670 msgr2=0x55baa1995be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 -- 172.21.15.62:0/203929489 shutdown_connections 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.361+0000 7f32da80c740 1 -- 172.21.15.62:0/203929489 wait complete. 2024-04-25T09:16:42.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.362+0000 7f32da80c740 1 -- start start 2024-04-25T09:16:42.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.362+0000 7f32da80c740 1 Processor -- start 2024-04-25T09:16:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.363+0000 7f32da80c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa1995e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.363+0000 7f32da80c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa1998670 0x55baa198d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.363+0000 7f32da80c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 0x55baa19940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.363+0000 7f32da80c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baa1852620 con 0x55baa1998670 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.363+0000 7f32da80c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baa18afff0 con 0x55baa197add0 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.363+0000 7f32da80c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baa1951850 con 0x55baa19a7410 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.363+0000 7f32da80c740 10 client.0 _ll_register_callbacks cb 0x55baa15c7f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa1998670 0x55baa198d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa1998670 0x55baa198d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45292/0 (socket says 172.21.15.62:45292) 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32cfe00640 1 -- 172.21.15.62:0/1567024921 learned_addr learned my addr 172.21.15.62:0/1567024921 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32d5000640 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa1995e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32d5a00640 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 0x55baa19940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32cfe00640 1 -- 172.21.15.62:0/1567024921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 msgr2=0x55baa19940b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32cfe00640 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 0x55baa19940b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32cfe00640 1 -- 172.21.15.62:0/1567024921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 msgr2=0x55baa1995e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32cfe00640 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa1995e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.364+0000 7f32cfe00640 1 -- 172.21.15.62:0/1567024921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55baa1588490 con 0x55baa1998670 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.365+0000 7f32cfe00640 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa1998670 0x55baa198d7b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009e80 tx=0x7f32b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4762d64436380b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.365+0000 7f32be000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.365+0000 7f32be000640 1 -- 172.21.15.62:0/1567024921 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8031810 con 0x55baa1998670 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.365+0000 7f32be000640 1 -- 172.21.15.62:0/1567024921 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32b80319b0 con 0x55baa1998670 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.365+0000 7f32da80c740 1 -- 172.21.15.62:0/1567024921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32c000e740 con 0x55baa1998670 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.365+0000 7f32be000640 1 -- 172.21.15.62:0/1567024921 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b804e050 con 0x55baa1998670 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.365+0000 7f32da80c740 10 client.9069 fetch_fsmap learned FSMap version 86 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.365+0000 7f32da80c740 1 -- 172.21.15.62:0/1567024921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55baa16f5330 con 0x55baa1998670 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32be000640 1 -- 172.21.15.62:0/1567024921 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000e740 con 0x55baa1998670 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32be000640 1 -- 172.21.15.62:0/1567024921 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32b8051020 con 0x55baa1998670 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32da80c740 10 client.9069 fetch_fsmap finished waiting for FSMap version 86 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32da80c740 10 client.9069 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T09:16:42.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32da80c740 20 client.9069 populate_metadata read hostname 'smithi062' 2024-04-25T09:16:42.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32da80c740 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55baa1994870 0x55baa197e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32da80c740 4 client.9069 mds_command: new command op to 8886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:16:42.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32da80c740 1 -- 172.21.15.62:0/1567024921 --> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55baa18c1490 con 0x55baa1994870 2024-04-25T09:16:42.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.366+0000 7f32d5000640 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55baa1994870 0x55baa197e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.367+0000 7f32d5000640 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55baa1994870 0x55baa197e4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32e6a0cabb265cf server_cookie=63b46504a1d82f51 in_seq=0 out_seq=0 2024-04-25T09:16:42.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.367+0000 7f32be000640 10 client.9069 ms_handle_connect on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32be000640 1 -- 172.21.15.62:0/1567024921 <== mds.0 v2:172.21.15.62:6832/2303077743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55baa18c1490 con 0x55baa1994870 2024-04-25T09:16:42.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32be000640 10 client.9069 handle_command_reply: tid=0 2024-04-25T09:16:42.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 client.9069 shutdown 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 -- 172.21.15.62:0/1567024921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa1998670 msgr2=0x55baa198d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa1998670 0x55baa198d7b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009e80 tx=0x7f32b8004ab0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 -- 172.21.15.62:0/1567024921 shutdown_connections 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55baa1994870 0x55baa197e4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa1998670 0x55baa198d7b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 0x55baa19940b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 --2- 172.21.15.62:0/1567024921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa1995e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32be000640 0 client.9069 ms_handle_reset on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 -- 172.21.15.62:0/1567024921 >> 172.21.15.62:0/1567024921 conn(0x55baa1950ab0 msgr2=0x55baa19999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 -- 172.21.15.62:0/1567024921 shutdown_connections 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.368+0000 7f32da80c740 1 -- 172.21.15.62:0/1567024921 wait complete. 2024-04-25T09:16:42.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.369+0000 7f32da80c740 20 client.9069 trim_cache size 0 max 16384 2024-04-25T09:16:42.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.387+0000 7f32da80c740 1 Processor -- start 2024-04-25T09:16:42.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.387+0000 7f32da80c740 1 -- start start 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.388+0000 7f32da80c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa19b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.388+0000 7f32da80c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa198f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.388+0000 7f32da80c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa19a7410 0x55baa194a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.388+0000 7f32da80c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0204460 con 0x55baa19a7410 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.388+0000 7f32da80c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0203640 con 0x55baa197add0 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.388+0000 7f32da80c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01f8e00 con 0x55baa1998670 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa19a7410 0x55baa194a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa19a7410 0x55baa194a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45296/0 (socket says 172.21.15.62:45296) 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5a00640 1 -- 172.21.15.62:0/3311043716 learned_addr learned my addr 172.21.15.62:0/3311043716 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32cfe00640 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa198f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5000640 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa19b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5a00640 1 -- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 msgr2=0x55baa198f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5a00640 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa198f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5a00640 1 -- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 msgr2=0x55baa19b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5a00640 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa19b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5a00640 1 -- 172.21.15.62:0/3311043716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baa1561610 con 0x55baa19a7410 2024-04-25T09:16:42.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32d5000640 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa19b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.389+0000 7f32cfe00640 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa198f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.390+0000 7f32d5a00640 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa19a7410 0x55baa194a480 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f32c8054890 tx=0x7f32c8054800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46afd547e8f8cdeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.390+0000 7f32c4c00640 1 -- 172.21.15.62:0/3311043716 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8056730 con 0x55baa19a7410 2024-04-25T09:16:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.390+0000 7f32c4c00640 1 -- 172.21.15.62:0/3311043716 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8086070 con 0x55baa19a7410 2024-04-25T09:16:42.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.390+0000 7f32c4c00640 1 -- 172.21.15.62:0/3311043716 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c805c120 con 0x55baa19a7410 2024-04-25T09:16:42.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 -- 172.21.15.62:0/3311043716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa19a7410 msgr2=0x55baa194a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa19a7410 0x55baa194a480 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f32c8054890 tx=0x7f32c8054800 comp rx=0 tx=0).stop 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 -- 172.21.15.62:0/3311043716 shutdown_connections 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa19a7410 0x55baa194a480 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa1998670 0x55baa198f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 --2- 172.21.15.62:0/3311043716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa197add0 0x55baa19b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 -- 172.21.15.62:0/3311043716 >> 172.21.15.62:0/3311043716 conn(0x55baa1950ab0 msgr2=0x55baa1950eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 -- 172.21.15.62:0/3311043716 shutdown_connections 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 -- 172.21.15.62:0/3311043716 wait complete. 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.391+0000 7f32da80c740 1 -- start start 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.392+0000 7f32da80c740 1 Processor -- start 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32da80c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa198c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32da80c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1998670 0x55baa199f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32da80c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 0x55baa1950e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32da80c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baa19b8d40 con 0x55baa197add0 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32da80c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01f9400 con 0x55baa1998670 2024-04-25T09:16:42.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32da80c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baa197f000 con 0x55baa19a7410 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32da80c740 10 client.0 _ll_register_callbacks cb 0x55baa194ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32d5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa198c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32d5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa198c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45312/0 (socket says 172.21.15.62:45312) 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32d5000640 1 -- 172.21.15.62:0/1209471662 learned_addr learned my addr 172.21.15.62:0/1209471662 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32d5000640 1 -- 172.21.15.62:0/1209471662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 msgr2=0x55baa1950e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32d5000640 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 0x55baa1950e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32d5000640 1 -- 172.21.15.62:0/1209471662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1998670 msgr2=0x55baa199f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32d5000640 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1998670 0x55baa199f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.393+0000 7f32d5000640 1 -- 172.21.15.62:0/1209471662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55baa19a7fe0 con 0x55baa197add0 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.394+0000 7f32d5000640 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa198c750 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f32c0014000 tx=0x7f32c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bb5fa2961d3625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.394+0000 7f32be000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.394+0000 7f32be000640 1 -- 172.21.15.62:0/1209471662 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000e740 con 0x55baa197add0 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.394+0000 7f32be000640 1 -- 172.21.15.62:0/1209471662 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32c000ec80 con 0x55baa197add0 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.394+0000 7f32da80c740 1 -- 172.21.15.62:0/1209471662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32c8086070 con 0x55baa197add0 2024-04-25T09:16:42.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.394+0000 7f32be000640 1 -- 172.21.15.62:0/1209471662 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c003b050 con 0x55baa197add0 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.394+0000 7f32da80c740 10 client.9075 fetch_fsmap learned FSMap version 86 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.394+0000 7f32da80c740 1 -- 172.21.15.62:0/1209471662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55baa19e9510 con 0x55baa197add0 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32be000640 1 -- 172.21.15.62:0/1209471662 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8086070 con 0x55baa197add0 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32be000640 1 -- 172.21.15.62:0/1209471662 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32c003e020 con 0x55baa197add0 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32da80c740 10 client.9075 fetch_fsmap finished waiting for FSMap version 86 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32da80c740 10 client.9075 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32da80c740 20 client.9075 populate_metadata read hostname 'smithi062' 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32da80c740 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55baa19513d0 0x55baa198b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32da80c740 4 client.9075 mds_command: new command op to 8886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32da80c740 1 -- 172.21.15.62:0/1209471662 --> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55baa18bb7e0 con 0x55baa19513d0 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.395+0000 7f32cfe00640 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55baa19513d0 0x55baa198b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.396+0000 7f32cfe00640 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55baa19513d0 0x55baa198b8a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb939ba76177b4d server_cookie=db21b539977e8208 in_seq=0 out_seq=0 2024-04-25T09:16:42.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.396+0000 7f32be000640 10 client.9075 ms_handle_connect on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.398+0000 7f32be000640 1 -- 172.21.15.62:0/1209471662 <== mds.0 v2:172.21.15.62:6832/2303077743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55baa18bb7e0 con 0x55baa19513d0 2024-04-25T09:16:42.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.398+0000 7f32be000640 10 client.9075 handle_command_reply: tid=0 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.398+0000 7f32da80c740 1 client.9075 shutdown 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.398+0000 7f32da80c740 1 -- 172.21.15.62:0/1209471662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 msgr2=0x55baa198c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.398+0000 7f32da80c740 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa198c750 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f32c0014000 tx=0x7f32c0000bc0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 1 -- 172.21.15.62:0/1209471662 shutdown_connections 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x55baa19513d0 0x55baa198b8a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55baa19a7410 0x55baa1950e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55baa1998670 0x55baa199f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 1 --2- 172.21.15.62:0/1209471662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55baa197add0 0x55baa198c750 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32be000640 0 client.9075 ms_handle_reset on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 1 -- 172.21.15.62:0/1209471662 >> 172.21.15.62:0/1209471662 conn(0x55baa1950ab0 msgr2=0x55baa197dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 1 -- 172.21.15.62:0/1209471662 shutdown_connections 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 1 -- 172.21.15.62:0/1209471662 wait complete. 2024-04-25T09:16:42.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.399+0000 7f32da80c740 20 client.9075 trim_cache size 0 max 16384 2024-04-25T09:16:42.388 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:16:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0, 2024-04-25T09:16:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_tag": "4478b912-a77c-4226-9b62-dd8959fc3840", 2024-04-25T09:16:42.388 INFO:teuthology.orchestra.run.smithi062.stdout: "mode": "asynchronous" 2024-04-25T09:16:42.388 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 -- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f32ac06b4d0 msgr2=0x7f32ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f32ac06b4d0 0x7f32ac06d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f32c0000f60 tx=0x7f32c0009040 comp rx=0 tx=0).stop 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 -- 172.21.15.62:0/1443813535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d015c6e0 msgr2=0x7f32d01612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d015c6e0 0x7f32d01612c0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f32c800b410 tx=0x7f32c800b8d0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 -- 172.21.15.62:0/1443813535 shutdown_connections 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f32ac06b4d0 0x7f32ac06d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32d015c6e0 0x7f32d01612c0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f32d00705f0 0x7f32d015cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 --2- 172.21.15.62:0/1443813535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d006c7c0 0x7f32d0150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 -- 172.21.15.62:0/1443813535 >> 172.21.15.62:0/1443813535 conn(0x7f32d00668f0 msgr2=0x7f32d0165c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 -- 172.21.15.62:0/1443813535 shutdown_connections 2024-04-25T09:16:42.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.400+0000 7f32d7a00640 1 -- 172.21.15.62:0/1443813535 wait complete. 2024-04-25T09:16:42.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:42.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T09:16:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.647+0000 7f232e800640 1 Processor -- start 2024-04-25T09:16:42.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.647+0000 7f232e800640 1 -- start start 2024-04-25T09:16:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.648+0000 7f232e800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f232815dbf0 0x7f23280b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.648+0000 7f232e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f23280b76a0 0x7f23280ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f232e800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23280d2e50 con 0x7f23280ced10 2024-04-25T09:16:42.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f232e800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23280b7e20 con 0x7f23280b76a0 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f232e800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23280b8100 con 0x7f232815dbf0 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f2326a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f23280b76a0 0x7f23280ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f2326a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f23280b76a0 0x7f23280ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45326/0 (socket says 172.21.15.62:45326) 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f2326a00640 1 -- 172.21.15.62:0/3114928269 learned_addr learned my addr 172.21.15.62:0/3114928269 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f2327400640 1 --2- 172.21.15.62:0/3114928269 >> v2:172.21.15.121:3300/0 conn(0x7f232815dbf0 0x7f23280b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f2326a00640 1 -- 172.21.15.62:0/3114928269 >> v2:172.21.15.121:3300/0 conn(0x7f232815dbf0 msgr2=0x7f23280b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f2326a00640 1 --2- 172.21.15.62:0/3114928269 >> v2:172.21.15.121:3300/0 conn(0x7f232815dbf0 0x7f23280b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f2326a00640 1 -- 172.21.15.62:0/3114928269 >> v1:172.21.15.121:6789/0 conn(0x7f23280ced10 legacy=0x7f23280b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.649+0000 7f2326a00640 1 -- 172.21.15.62:0/3114928269 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280b83e0 con 0x7f23280b76a0 2024-04-25T09:16:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.650+0000 7f2326a00640 1 --2- 172.21.15.62:0/3114928269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280b76a0 0x7f23280ce7d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f23280d1660 tx=0x7f231802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445f5a768edc167c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.651+0000 7f2326000640 1 -- 172.21.15.62:0/3114928269 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318036070 con 0x7f23280b76a0 2024-04-25T09:16:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.651+0000 7f2326000640 1 -- 172.21.15.62:0/3114928269 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2318031600 con 0x7f23280b76a0 2024-04-25T09:16:42.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.651+0000 7f2326000640 1 -- 172.21.15.62:0/3114928269 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23180319d0 con 0x7f23280b76a0 2024-04-25T09:16:42.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.651+0000 7f232e800640 1 -- 172.21.15.62:0/3114928269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280b76a0 msgr2=0x7f23280ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.651+0000 7f232e800640 1 --2- 172.21.15.62:0/3114928269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280b76a0 0x7f23280ce7d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f23280d1660 tx=0x7f231802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.652+0000 7f232e800640 1 -- 172.21.15.62:0/3114928269 shutdown_connections 2024-04-25T09:16:42.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.652+0000 7f232e800640 1 --2- 172.21.15.62:0/3114928269 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280b76a0 0x7f23280ce7d0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.652+0000 7f232e800640 1 --2- 172.21.15.62:0/3114928269 >> v2:172.21.15.121:3300/0 conn(0x7f232815dbf0 0x7f23280b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.652+0000 7f232e800640 1 -- 172.21.15.62:0/3114928269 >> 172.21.15.62:0/3114928269 conn(0x7f232806b000 msgr2=0x7f232806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.652+0000 7f232e800640 1 -- 172.21.15.62:0/3114928269 shutdown_connections 2024-04-25T09:16:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.652+0000 7f232e800640 1 -- 172.21.15.62:0/3114928269 wait complete. 2024-04-25T09:16:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.653+0000 7f232e800640 1 Processor -- start 2024-04-25T09:16:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.653+0000 7f232e800640 1 -- start start 2024-04-25T09:16:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.654+0000 7f232e800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23280b76a0 0x7f23281f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.654+0000 7f232e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280ced10 0x7f2328160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.654+0000 7f232e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815dbf0 0x7f23280b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.654+0000 7f232e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232815dfd0 con 0x7f23280ced10 2024-04-25T09:16:42.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.654+0000 7f232e800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280b89a0 con 0x7f232815dbf0 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.654+0000 7f232e800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23280b7ae0 con 0x7f23280b76a0 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280ced10 0x7f2328160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280ced10 0x7f2328160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45336/0 (socket says 172.21.15.62:45336) 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2326a00640 1 -- 172.21.15.62:0/3480409034 learned_addr learned my addr 172.21.15.62:0/3480409034 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2327e00640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815dbf0 0x7f23280b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2327400640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23280b76a0 0x7f23281f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2326a00640 1 -- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23280b76a0 msgr2=0x7f23281f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2326a00640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23280b76a0 0x7f23281f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2326a00640 1 -- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815dbf0 msgr2=0x7f23280b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2326a00640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815dbf0 0x7f23280b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2326a00640 1 -- 172.21.15.62:0/3480409034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328160fd0 con 0x7f23280ced10 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2327400640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23280b76a0 0x7f23281f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.655+0000 7f2327e00640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815dbf0 0x7f23280b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.656+0000 7f2326a00640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280ced10 0x7f2328160940 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2318034040 tx=0x7f2318004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.656+0000 7f231f400640 1 -- 172.21.15.62:0/3480409034 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318036050 con 0x7f23280ced10 2024-04-25T09:16:42.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.656+0000 7f231f400640 1 -- 172.21.15.62:0/3480409034 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2318042070 con 0x7f23280ced10 2024-04-25T09:16:42.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.656+0000 7f231f400640 1 -- 172.21.15.62:0/3480409034 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318009510 con 0x7f23280ced10 2024-04-25T09:16:42.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.656+0000 7f232e800640 1 -- 172.21.15.62:0/3480409034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232815ee60 con 0x7f23280ced10 2024-04-25T09:16:42.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.656+0000 7f232e800640 1 -- 172.21.15.62:0/3480409034 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232815f420 con 0x7f23280ced10 2024-04-25T09:16:42.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.658+0000 7f231f400640 1 -- 172.21.15.62:0/3480409034 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2318007c50 con 0x7f23280ced10 2024-04-25T09:16:42.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.659+0000 7f231f400640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f230406b4d0 0x7f230406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.659+0000 7f231f400640 1 -- 172.21.15.62:0/3480409034 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23180ab270 con 0x7f23280ced10 2024-04-25T09:16:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.660+0000 7f2327400640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f230406b4d0 0x7f230406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.660+0000 7f2331597740 1 Processor -- start 2024-04-25T09:16:42.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.660+0000 7f2331597740 1 -- start start 2024-04-25T09:16:42.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.661+0000 7f2327400640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f230406b4d0 0x7f230406d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2310008890 tx=0x7f2310008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.661+0000 7f2331597740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef04690 0x563acef04a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.661+0000 7f2331597740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceed4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.661+0000 7f2331597740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563aceed4dd0 0x563aceefcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.661+0000 7f2331597740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acee3d870 con 0x563acef050b0 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.661+0000 7f2331597740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acede6d60 con 0x563acef04690 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2331597740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x563acee39910 con 0x563aceed4dd0 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceed4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceed4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45342/0 (socket says 172.21.15.62:45342) 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2326a00640 1 -- 172.21.15.62:0/504727616 learned_addr learned my addr 172.21.15.62:0/504727616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2327400640 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef04690 0x563acef04a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2327e00640 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563aceed4dd0 0x563aceefcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2326a00640 1 -- 172.21.15.62:0/504727616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563aceed4dd0 msgr2=0x563aceefcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2326a00640 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563aceed4dd0 0x563aceefcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2326a00640 1 -- 172.21.15.62:0/504727616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef04690 msgr2=0x563acef04a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2326a00640 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef04690 0x563acef04a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.662+0000 7f2326a00640 1 -- 172.21.15.62:0/504727616 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563acec1c7d0 con 0x563acef050b0 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.663+0000 7f2326a00640 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceed4890 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f2318081bc0 tx=0x7f2318079340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30644415cbd3a54c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.663+0000 7f2317e00640 1 -- 172.21.15.62:0/504727616 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231807a0e0 con 0x563acef050b0 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.663+0000 7f2317e00640 1 -- 172.21.15.62:0/504727616 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2318037040 con 0x563acef050b0 2024-04-25T09:16:42.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.663+0000 7f2317e00640 1 -- 172.21.15.62:0/504727616 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23180ab070 con 0x563acef050b0 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.663+0000 7f2331597740 1 -- 172.21.15.62:0/504727616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 msgr2=0x563aceed4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.663+0000 7f2331597740 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceed4890 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f2318081bc0 tx=0x7f2318079340 comp rx=0 tx=0).stop 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.664+0000 7f2331597740 1 -- 172.21.15.62:0/504727616 shutdown_connections 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.664+0000 7f2331597740 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceed4890 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.664+0000 7f2331597740 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563aceed4dd0 0x563aceefcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.664+0000 7f2331597740 1 --2- 172.21.15.62:0/504727616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef04690 0x563acef04a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.664+0000 7f2331597740 1 -- 172.21.15.62:0/504727616 >> 172.21.15.62:0/504727616 conn(0x563aceed6d10 msgr2=0x563acef1f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.664+0000 7f2331597740 1 -- 172.21.15.62:0/504727616 shutdown_connections 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.664+0000 7f2331597740 1 -- 172.21.15.62:0/504727616 wait complete. 2024-04-25T09:16:42.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.664+0000 7f2331597740 1 -- start start 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.665+0000 7f2331597740 1 Processor -- start 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2331597740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2331597740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef050b0 0x563aceef6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2331597740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef1ebc0 0x563aceef70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2331597740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acee3d870 con 0x563acef1ebc0 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2331597740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acedfa1c0 con 0x563acef050b0 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2331597740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x563acef00de0 con 0x563acef04690 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2331597740 10 client.0 _ll_register_callbacks cb 0x563acee388f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef1ebc0 0x563aceef70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef1ebc0 0x563aceef70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45346/0 (socket says 172.21.15.62:45346) 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327e00640 1 -- 172.21.15.62:0/2952315587 learned_addr learned my addr 172.21.15.62:0/2952315587 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2326a00640 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef050b0 0x563aceef6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327400640 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327e00640 1 -- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 msgr2=0x563aceef5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327e00640 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327e00640 1 -- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef050b0 msgr2=0x563aceef6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327e00640 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef050b0 0x563aceef6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2327e00640 1 -- 172.21.15.62:0/2952315587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563aceb120f0 con 0x563acef1ebc0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.666+0000 7f2326a00640 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef050b0 0x563aceef6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.667+0000 7f2327e00640 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef1ebc0 0x563aceef70b0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2320005520 tx=0x7f232000e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afdef239ec79e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.667+0000 7f2314c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.667+0000 7f2314c00640 1 -- 172.21.15.62:0/2952315587 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2320003a60 con 0x563acef1ebc0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.667+0000 7f2314c00640 1 -- 172.21.15.62:0/2952315587 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2320003c00 con 0x563acef1ebc0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.667+0000 7f2331597740 1 -- 172.21.15.62:0/2952315587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2318037040 con 0x563acef1ebc0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.667+0000 7f2314c00640 1 -- 172.21.15.62:0/2952315587 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2320020050 con 0x563acef1ebc0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.667+0000 7f2331597740 10 client.9087 fetch_fsmap learned FSMap version 86 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.667+0000 7f2331597740 1 -- 172.21.15.62:0/2952315587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x563acec7cf60 con 0x563acef1ebc0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2314c00640 1 -- 172.21.15.62:0/2952315587 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2318037040 con 0x563acef1ebc0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2314c00640 1 -- 172.21.15.62:0/2952315587 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2320023020 con 0x563acef1ebc0 2024-04-25T09:16:42.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2331597740 10 client.9087 fetch_fsmap finished waiting for FSMap version 86 2024-04-25T09:16:42.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2331597740 10 client.9087 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T09:16:42.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2331597740 20 client.9087 populate_metadata read hostname 'smithi062' 2024-04-25T09:16:42.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2331597740 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x563acef21200 0x563acef21610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2331597740 4 client.9087 mds_command: new command op to 8886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:16:42.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2331597740 1 -- 172.21.15.62:0/2952315587 --> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563acee13690 con 0x563acef21200 2024-04-25T09:16:42.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.668+0000 7f2327400640 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x563acef21200 0x563acef21610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.669+0000 7f2314c00640 10 client.9087 ms_handle_connect on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.669+0000 7f2327400640 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x563acef21200 0x563acef21610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea19bdd2d7be533 server_cookie=db4946c262676d25 in_seq=0 out_seq=0 2024-04-25T09:16:42.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.669+0000 7f2314c00640 1 -- 172.21.15.62:0/2952315587 <== mds.0 v2:172.21.15.62:6832/2303077743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563acee13690 con 0x563acef21200 2024-04-25T09:16:42.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.669+0000 7f2314c00640 10 client.9087 handle_command_reply: tid=0 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.669+0000 7f2331597740 1 client.9087 shutdown 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 -- 172.21.15.62:0/2952315587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef1ebc0 msgr2=0x563aceef70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef1ebc0 0x563aceef70b0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2320005520 tx=0x7f232000e590 comp rx=0 tx=0).stop 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 -- 172.21.15.62:0/2952315587 shutdown_connections 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x563acef21200 0x563acef21610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef1ebc0 0x563aceef70b0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef050b0 0x563aceef6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 --2- 172.21.15.62:0/2952315587 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 -- 172.21.15.62:0/2952315587 >> 172.21.15.62:0/2952315587 conn(0x563aceed4dd0 msgr2=0x563aceed8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2314c00640 0 client.9087 ms_handle_reset on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 -- 172.21.15.62:0/2952315587 shutdown_connections 2024-04-25T09:16:42.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 1 -- 172.21.15.62:0/2952315587 wait complete. 2024-04-25T09:16:42.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.670+0000 7f2331597740 20 client.9087 trim_cache size 0 max 16384 2024-04-25T09:16:42.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.687+0000 7f2331597740 1 Processor -- start 2024-04-25T09:16:42.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.687+0000 7f2331597740 1 -- start start 2024-04-25T09:16:42.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.687+0000 7f2331597740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2331597740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceef81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2331597740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 0x563acef01230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2331597740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280cbe10 con 0x563acef050b0 2024-04-25T09:16:42.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2331597740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280cb010 con 0x563acef1ebc0 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2331597740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23280b5e30 con 0x563acef04690 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceef81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceef81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45348/0 (socket says 172.21.15.62:45348) 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2326a00640 1 -- 172.21.15.62:0/125140983 learned_addr learned my addr 172.21.15.62:0/125140983 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2327400640 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2326a00640 1 -- 172.21.15.62:0/125140983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 msgr2=0x563aceef7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2326a00640 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2326a00640 1 -- 172.21.15.62:0/125140983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 msgr2=0x563acef01230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2326a00640 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 0x563acef01230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2326a00640 1 -- 172.21.15.62:0/125140983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aceaeb1b0 con 0x563acef050b0 2024-04-25T09:16:42.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.688+0000 7f2327400640 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.689+0000 7f2326a00640 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceef81f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f2318075e40 tx=0x7f2318049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4d52b7e9af6714 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.689+0000 7f2317400640 1 -- 172.21.15.62:0/125140983 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318095070 con 0x563acef050b0 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.689+0000 7f2317400640 1 -- 172.21.15.62:0/125140983 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231807a0e0 con 0x563acef050b0 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.689+0000 7f2317400640 1 -- 172.21.15.62:0/125140983 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23180ab070 con 0x563acef050b0 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.689+0000 7f2331597740 1 -- 172.21.15.62:0/125140983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 msgr2=0x563aceef81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.689+0000 7f2331597740 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceef81f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f2318075e40 tx=0x7f2318049000 comp rx=0 tx=0).stop 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.690+0000 7f2331597740 1 -- 172.21.15.62:0/125140983 shutdown_connections 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.690+0000 7f2331597740 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563aceef81f0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.690+0000 7f2331597740 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 0x563acef01230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.690+0000 7f2331597740 1 --2- 172.21.15.62:0/125140983 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563aceef7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.690+0000 7f2331597740 1 -- 172.21.15.62:0/125140983 >> 172.21.15.62:0/125140983 conn(0x563aceed4dd0 msgr2=0x563aceed8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.690+0000 7f2331597740 1 -- 172.21.15.62:0/125140983 shutdown_connections 2024-04-25T09:16:42.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.690+0000 7f2331597740 1 -- 172.21.15.62:0/125140983 wait complete. 2024-04-25T09:16:42.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.690+0000 7f2331597740 1 -- start start 2024-04-25T09:16:42.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.691+0000 7f2331597740 1 Processor -- start 2024-04-25T09:16:42.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.691+0000 7f2331597740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563acef27630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.691+0000 7f2331597740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563acef89010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.691+0000 7f2331597740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 0x563acef1de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.691+0000 7f2331597740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2328151da0 con 0x563acef050b0 2024-04-25T09:16:42.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.691+0000 7f2331597740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563acef89a90 con 0x563acef1ebc0 2024-04-25T09:16:42.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2331597740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aceed9580 con 0x563acef04690 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2331597740 10 client.0 _ll_register_callbacks cb 0x7f23180092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563acef89010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2326a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563acef89010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45356/0 (socket says 172.21.15.62:45356) 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2326a00640 1 -- 172.21.15.62:0/2150686765 learned_addr learned my addr 172.21.15.62:0/2150686765 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2327400640 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563acef27630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2327e00640 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 0x563acef1de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2326a00640 1 -- 172.21.15.62:0/2150686765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 msgr2=0x563acef27630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2326a00640 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563acef27630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2326a00640 1 -- 172.21.15.62:0/2150686765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 msgr2=0x563acef1de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2326a00640 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 0x563acef1de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.692+0000 7f2326a00640 1 -- 172.21.15.62:0/2150686765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563aceed9a10 con 0x563acef050b0 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2326a00640 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563acef89010 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2318079970 tx=0x7f2318009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62413225c3b9b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2314c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2314c00640 1 -- 172.21.15.62:0/2150686765 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231809c030 con 0x563acef050b0 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2314c00640 1 -- 172.21.15.62:0/2150686765 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23180a9410 con 0x563acef050b0 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 1 -- 172.21.15.62:0/2150686765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f231807a0e0 con 0x563acef050b0 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2314c00640 1 -- 172.21.15.62:0/2150686765 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2318088050 con 0x563acef050b0 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 10 client.9093 fetch_fsmap learned FSMap version 86 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 1 -- 172.21.15.62:0/2150686765 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x563aceed9a10 con 0x563acef050b0 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2314c00640 1 -- 172.21.15.62:0/2150686765 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231807a0e0 con 0x563acef050b0 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2314c00640 1 -- 172.21.15.62:0/2150686765 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f231808b020 con 0x563acef050b0 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 10 client.9093 fetch_fsmap finished waiting for FSMap version 86 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 10 client.9093 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 20 client.9093 populate_metadata read hostname 'smithi062' 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x563acef1e3d0 0x563aceef89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 4 client.9093 mds_command: new command op to 8886 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:16:42.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.693+0000 7f2331597740 1 -- 172.21.15.62:0/2150686765 --> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x563acee4adb0 con 0x563acef1e3d0 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.694+0000 7f2327400640 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x563acef1e3d0 0x563aceef89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.694+0000 7f2327400640 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x563acef1e3d0 0x563aceef89d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31d76e0ba5cf97e server_cookie=2af929126c0b063a in_seq=0 out_seq=0 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.694+0000 7f2314c00640 10 client.9093 ms_handle_connect on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.694+0000 7f2314c00640 1 -- 172.21.15.62:0/2150686765 <== mds.0 v2:172.21.15.62:6832/2303077743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x563acee4adb0 con 0x563acef1e3d0 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2314c00640 10 client.9093 handle_command_reply: tid=0 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 client.9093 shutdown 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 -- 172.21.15.62:0/2150686765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 msgr2=0x563acef89010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563acef89010 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2318079970 tx=0x7f2318009810 comp rx=0 tx=0).stop 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 -- 172.21.15.62:0/2150686765 shutdown_connections 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x563acef1e3d0 0x563aceef89d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563acef050b0 0x563acef89010 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563acef1ebc0 0x563acef1de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 --2- 172.21.15.62:0/2150686765 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x563acef04690 0x563acef27630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2314c00640 0 client.9093 ms_handle_reset on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 -- 172.21.15.62:0/2150686765 >> 172.21.15.62:0/2150686765 conn(0x563aceed4dd0 msgr2=0x563acef01230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 -- 172.21.15.62:0/2150686765 shutdown_connections 2024-04-25T09:16:42.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 1 -- 172.21.15.62:0/2150686765 wait complete. 2024-04-25T09:16:42.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.695+0000 7f2331597740 20 client.9093 trim_cache size 0 max 16384 2024-04-25T09:16:42.686 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:16:42.686 INFO:teuthology.orchestra.run.smithi062.stdout: "return_code": 0 2024-04-25T09:16:42.686 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:16:42.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 -- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f230406b4d0 msgr2=0x7f230406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f230406b4d0 0x7f230406d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2310008890 tx=0x7f2310008040 comp rx=0 tx=0).stop 2024-04-25T09:16:42.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 -- 172.21.15.62:0/3480409034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280ced10 msgr2=0x7f2328160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280ced10 0x7f2328160940 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f2318034040 tx=0x7f2318004a40 comp rx=0 tx=0).stop 2024-04-25T09:16:42.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 -- 172.21.15.62:0/3480409034 shutdown_connections 2024-04-25T09:16:42.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f230406b4d0 0x7f230406d990 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f23280ced10 0x7f2328160940 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f232815dbf0 0x7f23280b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 --2- 172.21.15.62:0/3480409034 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f23280b76a0 0x7f23281f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 -- 172.21.15.62:0/3480409034 >> 172.21.15.62:0/3480409034 conn(0x7f232806b000 msgr2=0x7f232806b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 -- 172.21.15.62:0/3480409034 shutdown_connections 2024-04-25T09:16:42.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.696+0000 7f232e800640 1 -- 172.21.15.62:0/3480409034 wait complete. 2024-04-25T09:16:42.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:16:42.700 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T09:16:42.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.920+0000 7feb00a00640 1 Processor -- start 2024-04-25T09:16:42.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.920+0000 7feb00a00640 1 -- start start 2024-04-25T09:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.921+0000 7feb00a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feafc15dd50 0x7feafc0d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.921+0000 7feb00a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0d3960 0x7feafc0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.921+0000 7feb00a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feafc0ce780 con 0x7feafc0d2fb0 2024-04-25T09:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.921+0000 7feb00a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feafc0d42f0 con 0x7feafc15dd50 2024-04-25T09:16:42.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.922+0000 7feb00a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0d3440 con 0x7feafc0d3960 2024-04-25T09:16:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.922+0000 7feafa800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feafc15dd50 0x7feafc0d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.922+0000 7feafa800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feafc15dd50 0x7feafc0d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:55814/0 (socket says 172.21.15.62:55814) 2024-04-25T09:16:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.922+0000 7feafb200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0d3960 0x7feafc0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.922+0000 7feafa800640 1 -- 172.21.15.62:0/1214111975 learned_addr learned my addr 172.21.15.62:0/1214111975 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.923+0000 7feafa800640 1 -- 172.21.15.62:0/1214111975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0d3960 msgr2=0x7feafc0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.923+0000 7feafa800640 1 --2- 172.21.15.62:0/1214111975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0d3960 0x7feafc0d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.923+0000 7feafa800640 1 -- 172.21.15.62:0/1214111975 >> v1:172.21.15.121:6789/0 conn(0x7feafc0d2fb0 legacy=0x7feafc0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.923+0000 7feafa800640 1 -- 172.21.15.62:0/1214111975 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0ce1c0 con 0x7feafc15dd50 2024-04-25T09:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.923+0000 7feafb200640 1 --2- 172.21.15.62:0/1214111975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0d3960 0x7feafc0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:16:42.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.925+0000 7feafa800640 1 --2- 172.21.15.62:0/1214111975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc15dd50 0x7feafc0d2300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7feaec009870 tx=0x7feaec02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa912728b5144e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.925+0000 7feaf9400640 1 -- 172.21.15.62:0/1214111975 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec037070 con 0x7feafc15dd50 2024-04-25T09:16:42.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.925+0000 7feaf9400640 1 -- 172.21.15.62:0/1214111975 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec032440 con 0x7feafc15dd50 2024-04-25T09:16:42.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.925+0000 7feaf9400640 1 -- 172.21.15.62:0/1214111975 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec032810 con 0x7feafc15dd50 2024-04-25T09:16:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.926+0000 7feb00a00640 1 -- 172.21.15.62:0/1214111975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc15dd50 msgr2=0x7feafc0d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.926+0000 7feb00a00640 1 --2- 172.21.15.62:0/1214111975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc15dd50 0x7feafc0d2300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7feaec009870 tx=0x7feaec02a250 comp rx=0 tx=0).stop 2024-04-25T09:16:42.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.926+0000 7feb00a00640 1 -- 172.21.15.62:0/1214111975 shutdown_connections 2024-04-25T09:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.926+0000 7feb00a00640 1 --2- 172.21.15.62:0/1214111975 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0d3960 0x7feafc0d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.926+0000 7feb00a00640 1 --2- 172.21.15.62:0/1214111975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc15dd50 0x7feafc0d2300 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.926+0000 7feb00a00640 1 -- 172.21.15.62:0/1214111975 >> 172.21.15.62:0/1214111975 conn(0x7feafc06b000 msgr2=0x7feafc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.926+0000 7feb00a00640 1 -- 172.21.15.62:0/1214111975 shutdown_connections 2024-04-25T09:16:42.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.926+0000 7feb00a00640 1 -- 172.21.15.62:0/1214111975 wait complete. 2024-04-25T09:16:42.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.929+0000 7feb00a00640 1 Processor -- start 2024-04-25T09:16:42.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.929+0000 7feb00a00640 1 -- start start 2024-04-25T09:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feb00a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0d3960 0x7feafc1f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feb00a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc15dd50 0x7feafc0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feb00a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc1f3a80 0x7feafc1605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feb00a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0b6b00 con 0x7feafc15dd50 2024-04-25T09:16:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feb00a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0d2840 con 0x7feafc0d3960 2024-04-25T09:16:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feb00a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0b6c80 con 0x7feafc1f3a80 2024-04-25T09:16:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feaf9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc15dd50 0x7feafc0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feaf9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc15dd50 0x7feafc0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45360/0 (socket says 172.21.15.62:45360) 2024-04-25T09:16:42.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.930+0000 7feaf9e00640 1 -- 172.21.15.62:0/2636796255 learned_addr learned my addr 172.21.15.62:0/2636796255 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feafa800640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0d3960 0x7feafc1f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feafb200640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc1f3a80 0x7feafc1605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feaf9e00640 1 -- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc1f3a80 msgr2=0x7feafc1605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feaf9e00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc1f3a80 0x7feafc1605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feaf9e00640 1 -- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0d3960 msgr2=0x7feafc1f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feaf9e00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0d3960 0x7feafc1f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feaf9e00640 1 -- 172.21.15.62:0/2636796255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc1f42d0 con 0x7feafc15dd50 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feafa800640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0d3960 0x7feafc1f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.931+0000 7feaf9e00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc15dd50 0x7feafc0b26d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7feae400c790 tx=0x7feae400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.932+0000 7feaf2a00640 1 -- 172.21.15.62:0/2636796255 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4016020 con 0x7feafc15dd50 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.932+0000 7feaf2a00640 1 -- 172.21.15.62:0/2636796255 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4006d50 con 0x7feafc15dd50 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.932+0000 7feb00a00640 1 -- 172.21.15.62:0/2636796255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc1f45b0 con 0x7feafc15dd50 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.932+0000 7feaf2a00640 1 -- 172.21.15.62:0/2636796255 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4003bb0 con 0x7feafc15dd50 2024-04-25T09:16:42.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.932+0000 7feb00a00640 1 -- 172.21.15.62:0/2636796255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feafc0c9480 con 0x7feafc15dd50 2024-04-25T09:16:42.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.934+0000 7feaf2a00640 1 -- 172.21.15.62:0/2636796255 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feae400a850 con 0x7feafc15dd50 2024-04-25T09:16:42.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.935+0000 7feaf2a00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fead806b4d0 0x7fead806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.935+0000 7feaf2a00640 1 -- 172.21.15.62:0/2636796255 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feae4089fd0 con 0x7feafc15dd50 2024-04-25T09:16:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.936+0000 7feb041ab740 1 Processor -- start 2024-04-25T09:16:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.936+0000 7feafa800640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fead806b4d0 0x7fead806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.936+0000 7feb041ab740 1 -- start start 2024-04-25T09:16:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feb041ab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654c7fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feb041ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654c7e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feb041ab740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654ca8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feb041ab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564654be1c70 con 0x564654c7dd70 2024-04-25T09:16:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feb041ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564654b8acf0 con 0x564654cba440 2024-04-25T09:16:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feb041ab740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x564654bddd10 con 0x564654ca8830 2024-04-25T09:16:42.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feafa800640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fead806b4d0 0x7fead806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feaec034000 tx=0x7feaec02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feafa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654c7fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feafa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654c7fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45374/0 (socket says 172.21.15.62:45374) 2024-04-25T09:16:42.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feafa800640 1 -- 172.21.15.62:0/3187445874 learned_addr learned my addr 172.21.15.62:0/3187445874 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feaf9e00640 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654c7e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.937+0000 7feafb200640 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654ca8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feafa800640 1 -- 172.21.15.62:0/3187445874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 msgr2=0x564654ca8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feafa800640 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654ca8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feafa800640 1 -- 172.21.15.62:0/3187445874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 msgr2=0x564654c7e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feafa800640 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654c7e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feafa800640 1 -- 172.21.15.62:0/3187445874 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646549c07d0 con 0x564654c7dd70 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feafa800640 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654c7fed0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7feaec009960 tx=0x7feaec008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b10b2ba7e724d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feaeb000640 1 -- 172.21.15.62:0/3187445874 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec04d070 con 0x564654c7dd70 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feaeb000640 1 -- 172.21.15.62:0/3187445874 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec03c660 con 0x564654c7dd70 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.938+0000 7feaeb000640 1 -- 172.21.15.62:0/3187445874 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec037040 con 0x564654c7dd70 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 -- 172.21.15.62:0/3187445874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 msgr2=0x564654c7fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654c7fed0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7feaec009960 tx=0x7feaec008030 comp rx=0 tx=0).stop 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 -- 172.21.15.62:0/3187445874 shutdown_connections 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654ca8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654c7e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 --2- 172.21.15.62:0/3187445874 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654c7fed0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 -- 172.21.15.62:0/3187445874 >> 172.21.15.62:0/3187445874 conn(0x564654c45a30 msgr2=0x564654cc3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 -- 172.21.15.62:0/3187445874 shutdown_connections 2024-04-25T09:16:42.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.939+0000 7feb041ab740 1 -- 172.21.15.62:0/3187445874 wait complete. 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.940+0000 7feb041ab740 1 -- start start 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.940+0000 7feb041ab740 1 Processor -- start 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.941+0000 7feb041ab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.941+0000 7feb041ab740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654cc6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.941+0000 7feb041ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654cbde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.941+0000 7feb041ab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564654b803a0 con 0x564654c7dd70 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.941+0000 7feb041ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564654ca6380 con 0x564654cba440 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.941+0000 7feb041ab740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x564654c96680 con 0x564654ca8830 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.941+0000 7feb041ab740 10 client.0 _ll_register_callbacks cb 0x564654b9db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.941+0000 7feafa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45382/0 (socket says 172.21.15.62:45382) 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafa800640 1 -- 172.21.15.62:0/673798830 learned_addr learned my addr 172.21.15.62:0/673798830 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafa800640 1 -- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 msgr2=0x564654cc6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feaf9e00640 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654cc6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafb200640 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654cbde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafa800640 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654cc6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafa800640 1 -- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 msgr2=0x564654cbde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafa800640 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654cbde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafa800640 1 -- 172.21.15.62:0/673798830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646548f63f0 con 0x564654c7dd70 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feaf9e00640 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654cc6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feafa800640 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc6520 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7feaec0090b0 tx=0x7feaec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f2fc659b684fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.942+0000 7feae3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feae3400640 1 -- 172.21.15.62:0/673798830 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec056060 con 0x564654c7dd70 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feae3400640 1 -- 172.21.15.62:0/673798830 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feaec03c810 con 0x564654c7dd70 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feb041ab740 1 -- 172.21.15.62:0/673798830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaec03c660 con 0x564654c7dd70 2024-04-25T09:16:42.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feae3400640 1 -- 172.21.15.62:0/673798830 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaec05c050 con 0x564654c7dd70 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feb041ab740 10 client.9102 fetch_fsmap learned FSMap version 86 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feb041ab740 1 -- 172.21.15.62:0/673798830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x5646548b60f0 con 0x564654c7dd70 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feae3400640 1 -- 172.21.15.62:0/673798830 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec03c660 con 0x564654c7dd70 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feae3400640 1 -- 172.21.15.62:0/673798830 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feaec050070 con 0x564654c7dd70 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feb041ab740 10 client.9102 fetch_fsmap finished waiting for FSMap version 86 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feb041ab740 10 client.9102 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.943+0000 7feb041ab740 20 client.9102 populate_metadata read hostname 'smithi062' 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.944+0000 7feb041ab740 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x564654cbe580 0x564654c95600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.944+0000 7feb041ab740 4 client.9102 mds_command: new command op to 8886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.944+0000 7feb041ab740 1 -- 172.21.15.62:0/673798830 --> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564654bef1b0 con 0x564654cbe580 2024-04-25T09:16:42.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.944+0000 7feaf9e00640 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x564654cbe580 0x564654c95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.944+0000 7feaf9e00640 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x564654cbe580 0x564654c95600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60554f3c00364667 server_cookie=10845b3680ef93d9 in_seq=0 out_seq=0 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.944+0000 7feae3400640 10 client.9102 ms_handle_connect on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feae3400640 1 -- 172.21.15.62:0/673798830 <== mds.0 v2:172.21.15.62:6832/2303077743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7feae4081e50 con 0x564654cbe580 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feae3400640 10 client.9102 handle_command_reply: tid=0 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 client.9102 shutdown 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 -- 172.21.15.62:0/673798830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 msgr2=0x564654cc6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc6520 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7feaec0090b0 tx=0x7feaec004730 comp rx=0 tx=0).stop 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 -- 172.21.15.62:0/673798830 shutdown_connections 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x564654cbe580 0x564654c95600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654cbde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654ca8830 0x564654cc6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 --2- 172.21.15.62:0/673798830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc6520 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feae3400640 0 client.9102 ms_handle_reset on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.945+0000 7feb041ab740 1 -- 172.21.15.62:0/673798830 >> 172.21.15.62:0/673798830 conn(0x564654c45a30 msgr2=0x564654c7c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.946+0000 7feb041ab740 1 -- 172.21.15.62:0/673798830 shutdown_connections 2024-04-25T09:16:42.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.946+0000 7feb041ab740 1 -- 172.21.15.62:0/673798830 wait complete. 2024-04-25T09:16:42.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.946+0000 7feb041ab740 20 client.9102 trim_cache size 0 max 16384 2024-04-25T09:16:42.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.963+0000 7feb041ab740 1 Processor -- start 2024-04-25T09:16:42.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.963+0000 7feb041ab740 1 -- start start 2024-04-25T09:16:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feb041ab740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654c7dd70 0x564654cbcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feb041ab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654ca8830 0x564654cbd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feb041ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654cbde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feb041ab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc1ffe90 con 0x564654ca8830 2024-04-25T09:16:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feb041ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0c8140 con 0x564654cba440 2024-04-25T09:16:42.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feb041ab740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0c7540 con 0x564654c7dd70 2024-04-25T09:16:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feaf9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654ca8830 0x564654cbd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feaf9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654ca8830 0x564654cbd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45396/0 (socket says 172.21.15.62:45396) 2024-04-25T09:16:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feaf9e00640 1 -- 172.21.15.62:0/173628533 learned_addr learned my addr 172.21.15.62:0/173628533 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.964+0000 7feafa800640 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654c7dd70 0x564654cbcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaf9e00640 1 -- 172.21.15.62:0/173628533 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654c7dd70 msgr2=0x564654cbcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaf9e00640 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654c7dd70 0x564654cbcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feafb200640 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654cbde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaf9e00640 1 -- 172.21.15.62:0/173628533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 msgr2=0x564654cbde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaf9e00640 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654cbde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaf9e00640 1 -- 172.21.15.62:0/173628533 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564654cabf90 con 0x564654ca8830 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaf9e00640 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654ca8830 0x564654cbd8d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7feae40583c0 tx=0x7feae4077040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bc03e0d2580bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaea600640 1 -- 172.21.15.62:0/173628533 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4056760 con 0x564654ca8830 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaea600640 1 -- 172.21.15.62:0/173628533 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4081e50 con 0x564654ca8830 2024-04-25T09:16:42.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.965+0000 7feaea600640 1 -- 172.21.15.62:0/173628533 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4054d30 con 0x564654ca8830 2024-04-25T09:16:42.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 -- 172.21.15.62:0/173628533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654ca8830 msgr2=0x564654cbd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654ca8830 0x564654cbd8d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7feae40583c0 tx=0x7feae4077040 comp rx=0 tx=0).stop 2024-04-25T09:16:42.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 -- 172.21.15.62:0/173628533 shutdown_connections 2024-04-25T09:16:42.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654ca8830 0x564654cbd8d0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654cba440 0x564654cbde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 --2- 172.21.15.62:0/173628533 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654c7dd70 0x564654cbcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 -- 172.21.15.62:0/173628533 >> 172.21.15.62:0/173628533 conn(0x564654c45a30 msgr2=0x564654c74410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 -- 172.21.15.62:0/173628533 shutdown_connections 2024-04-25T09:16:42.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.966+0000 7feb041ab740 1 -- 172.21.15.62:0/173628533 wait complete. 2024-04-25T09:16:42.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.967+0000 7feb041ab740 1 -- start start 2024-04-25T09:16:42.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.967+0000 7feb041ab740 1 Processor -- start 2024-04-25T09:16:42.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feb041ab740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feb041ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654ca8830 0x564654cbd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feb041ab740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654cba440 0x564654cbde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feb041ab740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x564654cbca30 con 0x564654c7dd70 2024-04-25T09:16:42.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feb041ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0c7b40 con 0x564654ca8830 2024-04-25T09:16:42.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feb041ab740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x564654c95780 con 0x564654cba440 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feb041ab740 10 client.0 _ll_register_callbacks cb 0x7feaec04b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feafa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feafa800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45398/0 (socket says 172.21.15.62:45398) 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.968+0000 7feafa800640 1 -- 172.21.15.62:0/184864045 learned_addr learned my addr 172.21.15.62:0/184864045 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feafa800640 1 -- 172.21.15.62:0/184864045 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654cba440 msgr2=0x564654cbde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feafa800640 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654cba440 0x564654cbde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feafa800640 1 -- 172.21.15.62:0/184864045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654ca8830 msgr2=0x564654cbd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feafa800640 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654ca8830 0x564654cbd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feafa800640 1 -- 172.21.15.62:0/184864045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564654d165d0 con 0x564654c7dd70 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feafa800640 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc43f0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7feaec006f40 tx=0x7feaec03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfaf650ea96cb526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feae3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feae3400640 1 -- 172.21.15.62:0/184864045 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec064020 con 0x564654c7dd70 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feae3400640 1 -- 172.21.15.62:0/184864045 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feaec03c870 con 0x564654c7dd70 2024-04-25T09:16:42.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.969+0000 7feb041ab740 1 -- 172.21.15.62:0/184864045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feae4081e50 con 0x564654c7dd70 2024-04-25T09:16:42.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feae3400640 1 -- 172.21.15.62:0/184864045 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaec06e050 con 0x564654c7dd70 2024-04-25T09:16:42.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feb041ab740 10 client.9108 fetch_fsmap learned FSMap version 86 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feb041ab740 1 -- 172.21.15.62:0/184864045 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x564654cb9bf0 con 0x564654c7dd70 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feae3400640 1 -- 172.21.15.62:0/184864045 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4081e50 con 0x564654c7dd70 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feae3400640 1 -- 172.21.15.62:0/184864045 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feaec071020 con 0x564654c7dd70 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feb041ab740 10 client.9108 fetch_fsmap finished waiting for FSMap version 86 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feb041ab740 10 client.9108 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.c 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feb041ab740 20 client.9108 populate_metadata read hostname 'smithi062' 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feb041ab740 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x564654c7c530 0x564654d30c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feb041ab740 4 client.9108 mds_command: new command op to 8886 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feb041ab740 1 -- 172.21.15.62:0/184864045 --> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564654be9500 con 0x564654c7c530 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.970+0000 7feaf9e00640 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x564654c7c530 0x564654d30c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feaf9e00640 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x564654c7c530 0x564654d30c10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8faecb83fae564d server_cookie=9abe5db724fe67d1 in_seq=0 out_seq=0 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feae3400640 10 client.9108 ms_handle_connect on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feae3400640 1 -- 172.21.15.62:0/184864045 <== mds.0 v2:172.21.15.62:6832/2303077743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x564654be9500 con 0x564654c7c530 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feae3400640 10 client.9108 handle_command_reply: tid=0 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 client.9108 shutdown 2024-04-25T09:16:42.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 -- 172.21.15.62:0/184864045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 msgr2=0x564654cc43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc43f0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7feaec006f40 tx=0x7feaec03f040 comp rx=0 tx=0).stop 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 -- 172.21.15.62:0/184864045 shutdown_connections 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.62:6832/2303077743,v1:172.21.15.62:6834/2303077743] conn(0x564654c7c530 0x564654d30c10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x564654cba440 0x564654cbde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564654ca8830 0x564654cbd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 --2- 172.21.15.62:0/184864045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x564654c7dd70 0x564654cc43f0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feae3400640 0 client.9108 ms_handle_reset on v2:172.21.15.62:6832/2303077743 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.971+0000 7feb041ab740 1 -- 172.21.15.62:0/184864045 >> 172.21.15.62:0/184864045 conn(0x564654c45a30 msgr2=0x564654c74380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.972+0000 7feb041ab740 1 -- 172.21.15.62:0/184864045 shutdown_connections 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.972+0000 7feb041ab740 1 -- 172.21.15.62:0/184864045 wait complete. 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.972+0000 7feb041ab740 20 client.9108 trim_cache size 0 max 16384 2024-04-25T09:16:42.961 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "scrubs": { 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "4478b912-a77c-4226-9b62-dd8959fc3840": { 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "tag": "4478b912-a77c-4226-9b62-dd8959fc3840", 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: "options": "recursive" 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:16:42.962 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:16:42.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.972+0000 7feb00a00640 1 -- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fead806b4d0 msgr2=0x7fead806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.972+0000 7feb00a00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fead806b4d0 0x7fead806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feaec034000 tx=0x7feaec02e000 comp rx=0 tx=0).stop 2024-04-25T09:16:42.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 -- 172.21.15.62:0/2636796255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc15dd50 msgr2=0x7feafc0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:42.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc15dd50 0x7feafc0b26d0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7feae400c790 tx=0x7feae400cc50 comp rx=0 tx=0).stop 2024-04-25T09:16:42.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 -- 172.21.15.62:0/2636796255 shutdown_connections 2024-04-25T09:16:42.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fead806b4d0 0x7fead806d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc15dd50 0x7feafc0b26d0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc1f3a80 0x7feafc1605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 --2- 172.21.15.62:0/2636796255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0d3960 0x7feafc1f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:42.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 -- 172.21.15.62:0/2636796255 >> 172.21.15.62:0/2636796255 conn(0x7feafc06b000 msgr2=0x7feafc0cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:42.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 -- 172.21.15.62:0/2636796255 shutdown_connections 2024-04-25T09:16:42.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:42.973+0000 7feb00a00640 1 -- 172.21.15.62:0/2636796255 wait complete. 2024-04-25T09:16:42.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:43.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.186+0000 7fa125000640 1 Processor -- start 2024-04-25T09:16:43.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.186+0000 7fa125000640 1 -- start start 2024-04-25T09:16:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.188+0000 7fa125000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa12013d970 0x7fa1200b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.188+0000 7fa125000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1200b1050 0x7fa12006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.188+0000 7fa125000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12015b220 con 0x7fa12015acf0 2024-04-25T09:16:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.188+0000 7fa125000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12015b500 con 0x7fa12013d970 2024-04-25T09:16:43.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.188+0000 7fa125000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12015b7e0 con 0x7fa1200b1050 2024-04-25T09:16:43.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.189+0000 7fa11ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1200b1050 0x7fa12006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.189+0000 7fa11f400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa12013d970 0x7fa1200b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.189+0000 7fa11ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1200b1050 0x7fa12006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:37180/0 (socket says 172.21.15.62:37180) 2024-04-25T09:16:43.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.189+0000 7fa11f400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa12013d970 0x7fa1200b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:55874/0 (socket says 172.21.15.62:55874) 2024-04-25T09:16:43.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.189+0000 7fa11ea00640 1 -- 172.21.15.62:0/500844491 learned_addr learned my addr 172.21.15.62:0/500844491 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:43.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.190+0000 7fa11ea00640 1 -- 172.21.15.62:0/500844491 >> v2:172.21.15.121:3300/0 conn(0x7fa12013d970 msgr2=0x7fa1200b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:43.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.190+0000 7fa11ea00640 1 --2- 172.21.15.62:0/500844491 >> v2:172.21.15.121:3300/0 conn(0x7fa12013d970 0x7fa1200b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.190+0000 7fa11ea00640 1 -- 172.21.15.62:0/500844491 >> v1:172.21.15.121:6789/0 conn(0x7fa12015acf0 legacy=0x7fa12013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.190+0000 7fa11ea00640 1 -- 172.21.15.62:0/500844491 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12015bac0 con 0x7fa1200b1050 2024-04-25T09:16:43.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.190+0000 7fa11f400640 1 --2- 172.21.15.62:0/500844491 >> v2:172.21.15.121:3300/0 conn(0x7fa12013d970 0x7fa1200b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:16:43.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.191+0000 7fa11ea00640 1 --2- 172.21.15.62:0/500844491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1200b1050 0x7fa12006d1f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009870 tx=0x7fa10c02c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e71bb1e3fd2f3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:43.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.191+0000 7fa11e000640 1 -- 172.21.15.62:0/500844491 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c038070 con 0x7fa1200b1050 2024-04-25T09:16:43.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.191+0000 7fa11e000640 1 -- 172.21.15.62:0/500844491 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c031710 con 0x7fa1200b1050 2024-04-25T09:16:43.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa11e000640 1 -- 172.21.15.62:0/500844491 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c031ae0 con 0x7fa1200b1050 2024-04-25T09:16:43.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa125000640 1 -- 172.21.15.62:0/500844491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1200b1050 msgr2=0x7fa12006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa125000640 1 --2- 172.21.15.62:0/500844491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1200b1050 0x7fa12006d1f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009870 tx=0x7fa10c02c460 comp rx=0 tx=0).stop 2024-04-25T09:16:43.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa125000640 1 -- 172.21.15.62:0/500844491 shutdown_connections 2024-04-25T09:16:43.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa125000640 1 --2- 172.21.15.62:0/500844491 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa1200b1050 0x7fa12006d1f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa125000640 1 --2- 172.21.15.62:0/500844491 >> v2:172.21.15.121:3300/0 conn(0x7fa12013d970 0x7fa1200b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa125000640 1 -- 172.21.15.62:0/500844491 >> 172.21.15.62:0/500844491 conn(0x7fa1200b14d0 msgr2=0x7fa1200b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:43.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa125000640 1 -- 172.21.15.62:0/500844491 shutdown_connections 2024-04-25T09:16:43.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.192+0000 7fa125000640 1 -- 172.21.15.62:0/500844491 wait complete. 2024-04-25T09:16:43.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.194+0000 7fa125000640 1 Processor -- start 2024-04-25T09:16:43.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.194+0000 7fa125000640 1 -- start start 2024-04-25T09:16:43.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa125000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1200b1050 0x7fa1201f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa125000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12013d970 0x7fa120131ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa125000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12015acf0 0x7fa120140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa125000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12006d730 con 0x7fa1200b1050 2024-04-25T09:16:43.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa125000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12013f470 con 0x7fa12013d970 2024-04-25T09:16:43.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa125000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12006d8b0 con 0x7fa12015acf0 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa11f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1200b1050 0x7fa1201f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa11f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1200b1050 0x7fa1201f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45414/0 (socket says 172.21.15.62:45414) 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.195+0000 7fa11f400640 1 -- 172.21.15.62:0/3082947290 learned_addr learned my addr 172.21.15.62:0/3082947290 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11ea00640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12013d970 0x7fa120131ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11fe00640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12015acf0 0x7fa120140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11f400640 1 -- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12015acf0 msgr2=0x7fa120140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11f400640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12015acf0 0x7fa120140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11f400640 1 -- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12013d970 msgr2=0x7fa120131ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11f400640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12013d970 0x7fa120131ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11f400640 1 -- 172.21.15.62:0/3082947290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa120132020 con 0x7fa1200b1050 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11ea00640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12013d970 0x7fa120131ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:43.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.196+0000 7fa11f400640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1200b1050 0x7fa1201f3340 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa11000ec10 tx=0x7fa11000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:43.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.197+0000 7fa117400640 1 -- 172.21.15.62:0/3082947290 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110017070 con 0x7fa1200b1050 2024-04-25T09:16:43.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.197+0000 7fa117400640 1 -- 172.21.15.62:0/3082947290 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11000cc30 con 0x7fa1200b1050 2024-04-25T09:16:43.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.197+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12012f040 con 0x7fa1200b1050 2024-04-25T09:16:43.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.197+0000 7fa117400640 1 -- 172.21.15.62:0/3082947290 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110013470 con 0x7fa1200b1050 2024-04-25T09:16:43.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.197+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12012f5c0 con 0x7fa1200b1050 2024-04-25T09:16:43.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.198+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12004ef20 con 0x7fa1200b1050 2024-04-25T09:16:43.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.199+0000 7fa117400640 1 -- 172.21.15.62:0/3082947290 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa11001a070 con 0x7fa1200b1050 2024-04-25T09:16:43.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.200+0000 7fa117400640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa10006b4d0 0x7fa10006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.200+0000 7fa117400640 1 -- 172.21.15.62:0/3082947290 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa11008a2f0 con 0x7fa1200b1050 2024-04-25T09:16:43.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.201+0000 7fa11ea00640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa10006b4d0 0x7fa10006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.202+0000 7fa11ea00640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa10006b4d0 0x7fa10006d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa10c035000 tx=0x7fa10c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:43.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.205+0000 7fa117400640 1 -- 172.21.15.62:0/3082947290 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa120132020 con 0x7fa1200b1050 2024-04-25T09:16:43.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.451+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1200b0b00 con 0x7fa1200b1050 2024-04-25T09:16:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.452+0000 7fa117400640 1 -- 172.21.15.62:0/3082947290 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa110059050 con 0x7fa1200b1050 2024-04-25T09:16:43.439 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:43.440 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:43.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa10006b4d0 msgr2=0x7fa10006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa10006b4d0 0x7fa10006d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa10c035000 tx=0x7fa10c008030 comp rx=0 tx=0).stop 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1200b1050 msgr2=0x7fa1201f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1200b1050 0x7fa1201f3340 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa11000ec10 tx=0x7fa11000c520 comp rx=0 tx=0).stop 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 shutdown_connections 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa10006b4d0 0x7fa10006d990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa12015acf0 0x7fa120140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa12013d970 0x7fa120131ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 --2- 172.21.15.62:0/3082947290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa1200b1050 0x7fa1201f3340 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 >> 172.21.15.62:0/3082947290 conn(0x7fa1200b14d0 msgr2=0x7fa1200ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 shutdown_connections 2024-04-25T09:16:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.455+0000 7fa125000640 1 -- 172.21.15.62:0/3082947290 wait complete. 2024-04-25T09:16:43.461 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:16:43.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:16:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.683+0000 7fb60be00640 1 Processor -- start 2024-04-25T09:16:43.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.684+0000 7fb60be00640 1 -- start start 2024-04-25T09:16:43.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.685+0000 7fb60be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb60c15dc60 0x7fb60c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.685+0000 7fb60be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c070f70 0x7fb60c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.685+0000 7fb60be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c0addc0 con 0x7fb60c0ad8e0 2024-04-25T09:16:43.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.685+0000 7fb60be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c15e040 con 0x7fb60c15dc60 2024-04-25T09:16:43.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.685+0000 7fb60be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0ae9d0 con 0x7fb60c070f70 2024-04-25T09:16:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.686+0000 7fb60aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb60c15dc60 0x7fb60c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.686+0000 7fb60aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb60c15dc60 0x7fb60c070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45428/0 (socket says 172.21.15.62:45428) 2024-04-25T09:16:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.686+0000 7fb60aa00640 1 -- 172.21.15.62:0/3160731486 learned_addr learned my addr 172.21.15.62:0/3160731486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.686+0000 7fb603e00640 1 --2- 172.21.15.62:0/3160731486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c070f70 0x7fb60c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.687+0000 7fb60aa00640 1 -- 172.21.15.62:0/3160731486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c070f70 msgr2=0x7fb60c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.687+0000 7fb60aa00640 1 --2- 172.21.15.62:0/3160731486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c070f70 0x7fb60c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.687+0000 7fb60aa00640 1 -- 172.21.15.62:0/3160731486 >> v1:172.21.15.121:6789/0 conn(0x7fb60c0ad8e0 legacy=0x7fb60c0aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.687+0000 7fb60aa00640 1 -- 172.21.15.62:0/3160731486 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c0ae0a0 con 0x7fb60c15dc60 2024-04-25T09:16:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.688+0000 7fb60aa00640 1 --2- 172.21.15.62:0/3160731486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c15dc60 0x7fb60c070a30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009870 tx=0x7fb5fc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e890971d3746eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.688+0000 7fb60a000640 1 -- 172.21.15.62:0/3160731486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc037070 con 0x7fb60c15dc60 2024-04-25T09:16:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.688+0000 7fb60a000640 1 -- 172.21.15.62:0/3160731486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc032440 con 0x7fb60c15dc60 2024-04-25T09:16:43.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.688+0000 7fb60a000640 1 -- 172.21.15.62:0/3160731486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc032810 con 0x7fb60c15dc60 2024-04-25T09:16:43.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.689+0000 7fb60be00640 1 -- 172.21.15.62:0/3160731486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c15dc60 msgr2=0x7fb60c070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.689+0000 7fb60be00640 1 --2- 172.21.15.62:0/3160731486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c15dc60 0x7fb60c070a30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009870 tx=0x7fb5fc02a250 comp rx=0 tx=0).stop 2024-04-25T09:16:43.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.689+0000 7fb60be00640 1 -- 172.21.15.62:0/3160731486 shutdown_connections 2024-04-25T09:16:43.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.689+0000 7fb60be00640 1 --2- 172.21.15.62:0/3160731486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c070f70 0x7fb60c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.689+0000 7fb60be00640 1 --2- 172.21.15.62:0/3160731486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c15dc60 0x7fb60c070a30 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.689+0000 7fb60be00640 1 -- 172.21.15.62:0/3160731486 >> 172.21.15.62:0/3160731486 conn(0x7fb60c0cfed0 msgr2=0x7fb60c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:43.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.689+0000 7fb60be00640 1 -- 172.21.15.62:0/3160731486 shutdown_connections 2024-04-25T09:16:43.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.689+0000 7fb60be00640 1 -- 172.21.15.62:0/3160731486 wait complete. 2024-04-25T09:16:43.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.691+0000 7fb60be00640 1 Processor -- start 2024-04-25T09:16:43.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.691+0000 7fb60be00640 1 -- start start 2024-04-25T09:16:43.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c070f70 0x7fb60c1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c0ad8e0 0x7fb60c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15dc60 0x7fb60c1e7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0b50b0 con 0x7fb60c070f70 2024-04-25T09:16:43.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0ae660 con 0x7fb60c15dc60 2024-04-25T09:16:43.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0b5230 con 0x7fb60c0ad8e0 2024-04-25T09:16:43.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c070f70 0x7fb60c1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c070f70 0x7fb60c1557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45436/0 (socket says 172.21.15.62:45436) 2024-04-25T09:16:43.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60aa00640 1 -- 172.21.15.62:0/3830578875 learned_addr learned my addr 172.21.15.62:0/3830578875 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:43.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.692+0000 7fb60aa00640 1 -- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c0ad8e0 msgr2=0x7fb60c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb60aa00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c0ad8e0 0x7fb60c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb60aa00640 1 -- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15dc60 msgr2=0x7fb60c1e7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb60b400640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15dc60 0x7fb60c1e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb60aa00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15dc60 0x7fb60c1e7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb60aa00640 1 -- 172.21.15.62:0/3830578875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c15fbd0 con 0x7fb60c070f70 2024-04-25T09:16:43.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb60aa00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c070f70 0x7fb60c1557b0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc000c00 tx=0x7fb5fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:43.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb602a00640 1 -- 172.21.15.62:0/3830578875 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc041020 con 0x7fb60c070f70 2024-04-25T09:16:43.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb602a00640 1 -- 172.21.15.62:0/3830578875 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0306c0 con 0x7fb60c070f70 2024-04-25T09:16:43.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb602a00640 1 -- 172.21.15.62:0/3830578875 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc037040 con 0x7fb60c070f70 2024-04-25T09:16:43.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.693+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c15feb0 con 0x7fb60c070f70 2024-04-25T09:16:43.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.694+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c160470 con 0x7fb60c070f70 2024-04-25T09:16:43.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.695+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c0cf710 con 0x7fb60c070f70 2024-04-25T09:16:43.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.696+0000 7fb602a00640 1 -- 172.21.15.62:0/3830578875 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5fc044020 con 0x7fb60c070f70 2024-04-25T09:16:43.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.697+0000 7fb602a00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb5e806b4d0 0x7fb5e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:43.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.697+0000 7fb602a00640 1 -- 172.21.15.62:0/3830578875 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5fc0aa9f0 con 0x7fb60c070f70 2024-04-25T09:16:43.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.697+0000 7fb603e00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb5e806b4d0 0x7fb5e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:43.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.698+0000 7fb603e00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb5e806b4d0 0x7fb5e806d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0cd220 tx=0x7fb5f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:43.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.701+0000 7fb602a00640 1 -- 172.21.15.62:0/3830578875 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5fc04b030 con 0x7fb60c070f70 2024-04-25T09:16:43.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.974+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb60c0cfb60 con 0x7fb60c070f70 2024-04-25T09:16:43.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.975+0000 7fb602a00640 1 -- 172.21.15.62:0/3830578875 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1027 (secure 0 0 0) 0x7fb5fc079860 con 0x7fb60c070f70 2024-04-25T09:16:43.963 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:43.963 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":831,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1010,"data_bytes":1036331917,"bytes_used":2505564160,"bytes_avail":770588549120,"bytes_total":773094113280,"write_bytes_sec":45868724,"read_op_per_sec":0,"write_op_per_sec":46},"fsmap":{"epoch":86,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8886}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-25T09:15:50.176907+0000","services":{}},"progress_events":{}} 2024-04-25T09:16:43.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb5e806b4d0 msgr2=0x7fb5e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb5e806b4d0 0x7fb5e806d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0cd220 tx=0x7fb5f4008040 comp rx=0 tx=0).stop 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c070f70 msgr2=0x7fb60c1557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c070f70 0x7fb60c1557b0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc000c00 tx=0x7fb5fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 shutdown_connections 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb5e806b4d0 0x7fb5e806d990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb60c15dc60 0x7fb60c1e7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb60c0ad8e0 0x7fb60c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 --2- 172.21.15.62:0/3830578875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb60c070f70 0x7fb60c1557b0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.978+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 >> 172.21.15.62:0/3830578875 conn(0x7fb60c0cfed0 msgr2=0x7fb60c072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.979+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 shutdown_connections 2024-04-25T09:16:43.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:43.979+0000 7fb60be00640 1 -- 172.21.15.62:0/3830578875 wait complete. 2024-04-25T09:16:44.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:45.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.224+0000 7f786be00640 1 Processor -- start 2024-04-25T09:16:45.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.224+0000 7f786be00640 1 -- start start 2024-04-25T09:16:45.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f786c0b6a90 0x7f786c0d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f786c0b27a0 0x7f786c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0d3110 con 0x7f786c0b56f0 2024-04-25T09:16:45.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786be00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0cedc0 con 0x7f786c0b6a90 2024-04-25T09:16:45.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0d33f0 con 0x7f786c0b27a0 2024-04-25T09:16:45.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f786c0b6a90 0x7f786c0d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f786c0b56f0 0x7f786c160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50472/0 (socket says 172.21.15.62:50472) 2024-04-25T09:16:45.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786aa00640 1 -- 172.21.15.62:0/3107472033 learned_addr learned my addr 172.21.15.62:0/3107472033 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:45.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.226+0000 7f786b400640 1 --2- 172.21.15.62:0/3107472033 >> v2:172.21.15.121:3300/0 conn(0x7f786c0b27a0 0x7f786c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.227+0000 7f786a000640 1 -- 172.21.15.62:0/3107472033 >> v2:172.21.15.121:3300/0 conn(0x7f786c0b27a0 msgr2=0x7f786c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:45.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.227+0000 7f786a000640 1 --2- 172.21.15.62:0/3107472033 >> v2:172.21.15.121:3300/0 conn(0x7f786c0b27a0 0x7f786c0d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.227+0000 7f786a000640 1 -- 172.21.15.62:0/3107472033 >> v1:172.21.15.62:6789/0 conn(0x7f786c0b56f0 legacy=0x7f786c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.227+0000 7f786a000640 1 -- 172.21.15.62:0/3107472033 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0d36d0 con 0x7f786c0b6a90 2024-04-25T09:16:45.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.227+0000 7f786b400640 1 --2- 172.21.15.62:0/3107472033 >> v2:172.21.15.121:3300/0 conn(0x7f786c0b27a0 0x7f786c0d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:45.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.227+0000 7f7869600640 1 -- 172.21.15.62:0/3107472033 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520383569 0 0) 0x7f786c0d3110 con 0x7f786c0b56f0 2024-04-25T09:16:45.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.228+0000 7f786a000640 1 --2- 172.21.15.62:0/3107472033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b6a90 0x7f786c0d1fe0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b2720 tx=0x7f785402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc257164cbe72a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.228+0000 7f7869600640 1 -- 172.21.15.62:0/3107472033 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854037070 con 0x7f786c0b6a90 2024-04-25T09:16:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.228+0000 7f7869600640 1 -- 172.21.15.62:0/3107472033 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7854032440 con 0x7f786c0b6a90 2024-04-25T09:16:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.228+0000 7f7869600640 1 -- 172.21.15.62:0/3107472033 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854032810 con 0x7f786c0b6a90 2024-04-25T09:16:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.229+0000 7f786be00640 1 -- 172.21.15.62:0/3107472033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b6a90 msgr2=0x7f786c0d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.229+0000 7f786be00640 1 --2- 172.21.15.62:0/3107472033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b6a90 0x7f786c0d1fe0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b2720 tx=0x7f785402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.229+0000 7f786be00640 1 -- 172.21.15.62:0/3107472033 shutdown_connections 2024-04-25T09:16:45.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.229+0000 7f786be00640 1 --2- 172.21.15.62:0/3107472033 >> v2:172.21.15.121:3300/0 conn(0x7f786c0b27a0 0x7f786c0d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.229+0000 7f786be00640 1 --2- 172.21.15.62:0/3107472033 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b6a90 0x7f786c0d1fe0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.229+0000 7f786be00640 1 -- 172.21.15.62:0/3107472033 >> 172.21.15.62:0/3107472033 conn(0x7f786c066d10 msgr2=0x7f786c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:45.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.229+0000 7f786be00640 1 -- 172.21.15.62:0/3107472033 shutdown_connections 2024-04-25T09:16:45.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.229+0000 7f786be00640 1 -- 172.21.15.62:0/3107472033 wait complete. 2024-04-25T09:16:45.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.231+0000 7f786be00640 1 Processor -- start 2024-04-25T09:16:45.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.231+0000 7f786be00640 1 -- start start 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f786c0b27a0 0x7f786c157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b56f0 0x7f786c15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f786c0b6a90 0x7f786c14e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0b2ee0 con 0x7f786c0b56f0 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0b2d60 con 0x7f786c0b27a0 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0b5ad0 con 0x7f786c0b6a90 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b56f0 0x7f786c15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b56f0 0x7f786c15a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35456/0 (socket says 172.21.15.62:35456) 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786a000640 1 -- 172.21.15.62:0/2948366411 learned_addr learned my addr 172.21.15.62:0/2948366411 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786aa00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f786c0b27a0 0x7f786c157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786a000640 1 -- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f786c0b6a90 msgr2=0x7f786c14e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786a000640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f786c0b6a90 0x7f786c14e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786a000640 1 -- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f786c0b27a0 msgr2=0x7f786c157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786a000640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f786c0b27a0 0x7f786c157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.232+0000 7f786a000640 1 -- 172.21.15.62:0/2948366411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c158300 con 0x7f786c0b56f0 2024-04-25T09:16:45.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.233+0000 7f786a000640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b56f0 0x7f786c15a3a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f7854000c00 tx=0x7f78540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:45.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.233+0000 7f7863400640 1 -- 172.21.15.62:0/2948366411 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854037040 con 0x7f786c0b56f0 2024-04-25T09:16:45.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.233+0000 7f7863400640 1 -- 172.21.15.62:0/2948366411 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7854042070 con 0x7f786c0b56f0 2024-04-25T09:16:45.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.233+0000 7f7863400640 1 -- 172.21.15.62:0/2948366411 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854030a50 con 0x7f786c0b56f0 2024-04-25T09:16:45.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.233+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c1585e0 con 0x7f786c0b56f0 2024-04-25T09:16:45.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.233+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c158ba0 con 0x7f786c0b56f0 2024-04-25T09:16:45.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.235+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c04ef20 con 0x7f786c0b56f0 2024-04-25T09:16:45.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.235+0000 7f7863400640 1 -- 172.21.15.62:0/2948366411 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7854030bf0 con 0x7f786c0b56f0 2024-04-25T09:16:45.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.236+0000 7f7863400640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.236+0000 7f7863400640 1 -- 172.21.15.62:0/2948366411 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78540aace0 con 0x7f786c0b56f0 2024-04-25T09:16:45.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.237+0000 7f786aa00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.239+0000 7f786aa00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f784806b4d0 0x7f784806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f785c007fd0 tx=0x7f785c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:45.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.241+0000 7f7863400640 1 -- 172.21.15.62:0/2948366411 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7854045360 con 0x7f786c0b56f0 2024-04-25T09:16:45.422 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:16:45.430 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:16:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.483+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f786c0d2c60 con 0x7f786c0b56f0 2024-04-25T09:16:45.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.483+0000 7f7863400640 1 -- 172.21.15.62:0/2948366411 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7854079b60 con 0x7f786c0b56f0 2024-04-25T09:16:45.471 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:45.471 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:45.471 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.485+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f784806b4d0 msgr2=0x7f784806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.485+0000 7f786be00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f784806b4d0 0x7f784806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f785c007fd0 tx=0x7f785c009040 comp rx=0 tx=0).stop 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b56f0 msgr2=0x7f786c15a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b56f0 0x7f786c15a3a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f7854000c00 tx=0x7f78540046c0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 shutdown_connections 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f784806b4d0 0x7f784806d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f786c0b56f0 0x7f786c15a3a0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f786c0b6a90 0x7f786c14e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 --2- 172.21.15.62:0/2948366411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f786c0b27a0 0x7f786c157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 >> 172.21.15.62:0/2948366411 conn(0x7f786c066d10 msgr2=0x7f786c15e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 shutdown_connections 2024-04-25T09:16:45.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.486+0000 7f786be00640 1 -- 172.21.15.62:0/2948366411 wait complete. 2024-04-25T09:16:45.490 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:16:45.490 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:16:45.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.754+0000 7fd59f200640 1 Processor -- start 2024-04-25T09:16:45.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.754+0000 7fd59f200640 1 -- start start 2024-04-25T09:16:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.756+0000 7fd59f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a0161fb0 0x7fd5a00b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.756+0000 7fd59f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5a00b27f0 0x7fd5a006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.756+0000 7fd59f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a00b2cf0 con 0x7fd5a006d800 2024-04-25T09:16:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.756+0000 7fd59f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a00cca70 con 0x7fd5a00b27f0 2024-04-25T09:16:45.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.756+0000 7fd59f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00b2fd0 con 0x7fd5a0161fb0 2024-04-25T09:16:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.756+0000 7fd59d400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5a00b27f0 0x7fd5a006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.756+0000 7fd59de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a0161fb0 0x7fd5a00b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.757+0000 7fd59de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a0161fb0 0x7fd5a00b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:58454/0 (socket says 172.21.15.62:58454) 2024-04-25T09:16:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.757+0000 7fd59de00640 1 -- 172.21.15.62:0/1246760358 learned_addr learned my addr 172.21.15.62:0/1246760358 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.757+0000 7fd59de00640 1 -- 172.21.15.62:0/1246760358 >> v2:172.21.15.121:3300/0 conn(0x7fd5a00b27f0 msgr2=0x7fd5a006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.757+0000 7fd59de00640 1 --2- 172.21.15.62:0/1246760358 >> v2:172.21.15.121:3300/0 conn(0x7fd5a00b27f0 0x7fd5a006d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.757+0000 7fd59de00640 1 -- 172.21.15.62:0/1246760358 >> v1:172.21.15.121:6789/0 conn(0x7fd5a006d800 legacy=0x7fd5a0165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.757+0000 7fd59de00640 1 -- 172.21.15.62:0/1246760358 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00b32b0 con 0x7fd5a0161fb0 2024-04-25T09:16:45.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.759+0000 7fd59de00640 1 --2- 172.21.15.62:0/1246760358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a0161fb0 0x7fd5a00b4480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd588009870 tx=0x7fd58802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78530085c1447c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.760+0000 7fd59ca00640 1 -- 172.21.15.62:0/1246760358 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588037070 con 0x7fd5a0161fb0 2024-04-25T09:16:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.760+0000 7fd59ca00640 1 -- 172.21.15.62:0/1246760358 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd588032440 con 0x7fd5a0161fb0 2024-04-25T09:16:45.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.760+0000 7fd59ca00640 1 -- 172.21.15.62:0/1246760358 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588032810 con 0x7fd5a0161fb0 2024-04-25T09:16:45.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.760+0000 7fd59f200640 1 -- 172.21.15.62:0/1246760358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a0161fb0 msgr2=0x7fd5a00b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.760+0000 7fd59f200640 1 --2- 172.21.15.62:0/1246760358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a0161fb0 0x7fd5a00b4480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd588009870 tx=0x7fd58802a250 comp rx=0 tx=0).stop 2024-04-25T09:16:45.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.761+0000 7fd59f200640 1 -- 172.21.15.62:0/1246760358 shutdown_connections 2024-04-25T09:16:45.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.761+0000 7fd59f200640 1 --2- 172.21.15.62:0/1246760358 >> v2:172.21.15.121:3300/0 conn(0x7fd5a00b27f0 0x7fd5a006d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.761+0000 7fd59f200640 1 --2- 172.21.15.62:0/1246760358 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a0161fb0 0x7fd5a00b4480 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.761+0000 7fd59f200640 1 -- 172.21.15.62:0/1246760358 >> 172.21.15.62:0/1246760358 conn(0x7fd5a00668f0 msgr2=0x7fd5a0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:45.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.761+0000 7fd59f200640 1 -- 172.21.15.62:0/1246760358 shutdown_connections 2024-04-25T09:16:45.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.761+0000 7fd59f200640 1 -- 172.21.15.62:0/1246760358 wait complete. 2024-04-25T09:16:45.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.762+0000 7fd59f200640 1 Processor -- start 2024-04-25T09:16:45.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.763+0000 7fd59f200640 1 -- start start 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.763+0000 7fd59f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a006d800 0x7fd5a014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.763+0000 7fd59f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00b27f0 0x7fd5a0163b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a0161fb0 0x7fd5a01640b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a0162390 con 0x7fd5a0161fb0 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00b49c0 con 0x7fd5a006d800 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a015f940 con 0x7fd5a00b27f0 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a0161fb0 0x7fd5a01640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a0161fb0 0x7fd5a01640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35470/0 (socket says 172.21.15.62:35470) 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59e800640 1 -- 172.21.15.62:0/3163142571 learned_addr learned my addr 172.21.15.62:0/3163142571 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:45.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59de00640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a006d800 0x7fd5a014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59d400640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00b27f0 0x7fd5a0163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59e800640 1 -- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00b27f0 msgr2=0x7fd5a0163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59e800640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00b27f0 0x7fd5a0163b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59e800640 1 -- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a006d800 msgr2=0x7fd5a014d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59e800640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a006d800 0x7fd5a014d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.764+0000 7fd59e800640 1 -- 172.21.15.62:0/3163142571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a015fc20 con 0x7fd5a0161fb0 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.765+0000 7fd59e800640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a0161fb0 0x7fd5a01640b0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fd59800c390 tx=0x7fd59800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.765+0000 7fd596000640 1 -- 172.21.15.62:0/3163142571 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598017070 con 0x7fd5a0161fb0 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.765+0000 7fd596000640 1 -- 172.21.15.62:0/3163142571 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5980052d0 con 0x7fd5a0161fb0 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.765+0000 7fd596000640 1 -- 172.21.15.62:0/3163142571 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598003810 con 0x7fd5a0161fb0 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.765+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a015ff00 con 0x7fd5a0161fb0 2024-04-25T09:16:45.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.765+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a014dca0 con 0x7fd5a0161fb0 2024-04-25T09:16:45.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.766+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a00b1b60 con 0x7fd5a0161fb0 2024-04-25T09:16:45.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.767+0000 7fd596000640 1 -- 172.21.15.62:0/3163142571 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59801a070 con 0x7fd5a0161fb0 2024-04-25T09:16:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.768+0000 7fd596000640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:45.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.768+0000 7fd596000640 1 -- 172.21.15.62:0/3163142571 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd598089580 con 0x7fd5a0161fb0 2024-04-25T09:16:45.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.769+0000 7fd59de00640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:45.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.770+0000 7fd59de00640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd588034000 tx=0x7fd58802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:45.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:45.773+0000 7fd596000640 1 -- 172.21.15.62:0/3163142571 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5a00b1b60 con 0x7fd5a0161fb0 2024-04-25T09:16:46.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.050+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd5a0067420 con 0x7fd5a0161fb0 2024-04-25T09:16:46.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.051+0000 7fd596000640 1 -- 172.21.15.62:0/3163142571 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1027 (secure 0 0 0) 0x7fd5a0067420 con 0x7fd5a0161fb0 2024-04-25T09:16:46.039 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:46.039 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":833,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1010,"data_bytes":1036331917,"bytes_used":2505564160,"bytes_avail":770588549120,"bytes_total":773094113280,"write_bytes_sec":30636981,"read_op_per_sec":0,"write_op_per_sec":31},"fsmap":{"epoch":86,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8886}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-25T09:15:50.176907+0000","services":{}},"progress_events":{}} 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 msgr2=0x7fd57c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd588034000 tx=0x7fd58802e000 comp rx=0 tx=0).stop 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a0161fb0 msgr2=0x7fd5a01640b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a0161fb0 0x7fd5a01640b0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fd59800c390 tx=0x7fd59800c850 comp rx=0 tx=0).stop 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 shutdown_connections 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57c06b4d0 0x7fd57c06d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5a0161fb0 0x7fd5a01640b0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5a00b27f0 0x7fd5a0163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 --2- 172.21.15.62:0/3163142571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a006d800 0x7fd5a014d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.054+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 >> 172.21.15.62:0/3163142571 conn(0x7fd5a00668f0 msgr2=0x7fd5a00b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.055+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 shutdown_connections 2024-04-25T09:16:46.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:46.055+0000 7fd59f200640 1 -- 172.21.15.62:0/3163142571 wait complete. 2024-04-25T09:16:47.060 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:47.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.292+0000 7f82cb000640 1 Processor -- start 2024-04-25T09:16:47.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.292+0000 7f82cb000640 1 -- start start 2024-04-25T09:16:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.294+0000 7f82cb000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82c40b6a90 0x7f82c40d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.294+0000 7f82cb000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82c40b27a0 0x7f82c40d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.294+0000 7f82cb000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c40d3110 con 0x7f82c40b56f0 2024-04-25T09:16:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.294+0000 7f82cb000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c40cedc0 con 0x7f82c40b6a90 2024-04-25T09:16:47.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.294+0000 7f82cb000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40d33f0 con 0x7f82c40b27a0 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.294+0000 7f82c9c00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f82c40b56f0 0x7f82c4160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50484/0 (socket says 172.21.15.62:50484) 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c9c00640 1 -- 172.21.15.62:0/3879655019 learned_addr learned my addr 172.21.15.62:0/3879655019 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82ca600640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82c40b27a0 0x7f82c40d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c9200640 1 --2- 172.21.15.62:0/3879655019 >> v2:172.21.15.121:3300/0 conn(0x7f82c40b6a90 0x7f82c40d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918399360 0 0) 0x7f82c40d3110 con 0x7f82c40b56f0 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ac003520 con 0x7f82c40b56f0 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2440068478 0 0) 0x7f82ac003520 con 0x7f82c40b56f0 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82c40b27a0 msgr2=0x7f82c40d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82c40b27a0 0x7f82c40d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 >> v2:172.21.15.121:3300/0 conn(0x7f82c40b6a90 msgr2=0x7f82c40d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:47.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3879655019 >> v2:172.21.15.121:3300/0 conn(0x7f82c40b6a90 0x7f82c40d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c40d36d0 con 0x7f82c40b56f0 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.295+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f82b8001670 con 0x7f82c40b56f0 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 >> v1:172.21.15.62:6789/0 conn(0x7f82c40b56f0 legacy=0x7f82c4160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82ac0039b0 0x7f82ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82ac004800 0x7f82ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c3e00640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82c40b27a0 0x7f82ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40cedc0 con 0x7f82ac0039b0 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40d33f0 con 0x7f82ac004800 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac00b9c0 con 0x7f82c40b27a0 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c9c00640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82ac0039b0 0x7f82ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c9200640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82ac004800 0x7f82ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c9c00640 1 -- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82c40b27a0 msgr2=0x7f82ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c9c00640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82c40b27a0 0x7f82ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c9c00640 1 -- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82ac004800 msgr2=0x7f82ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:47.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c9c00640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82ac004800 0x7f82ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.296+0000 7f82c9c00640 1 -- 172.21.15.62:0/3879655019 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac003520 con 0x7f82ac0039b0 2024-04-25T09:16:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.298+0000 7f82c9c00640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82ac0039b0 0x7f82ac003dd0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f82b800c190 tx=0x7f82ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e76cd1639811e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.298+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8007c60 con 0x7f82ac0039b0 2024-04-25T09:16:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.298+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b8005af0 con 0x7f82ac0039b0 2024-04-25T09:16:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.298+0000 7f82c3e00640 1 -- 172.21.15.62:0/3879655019 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8007070 con 0x7f82ac0039b0 2024-04-25T09:16:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.299+0000 7f82cb000640 1 -- 172.21.15.62:0/3879655019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82ac0039b0 msgr2=0x7f82ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.299+0000 7f82cb000640 1 --2- 172.21.15.62:0/3879655019 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82ac0039b0 0x7f82ac003dd0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f82b800c190 tx=0x7f82ac005830 comp rx=0 tx=0).stop 2024-04-25T09:16:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.299+0000 7f82ca600640 1 -- 172.21.15.62:0/3879655019 reap_dead start 2024-04-25T09:16:47.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.299+0000 7f82cb000640 1 -- 172.21.15.62:0/3879655019 shutdown_connections 2024-04-25T09:16:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.299+0000 7f82cb000640 1 -- 172.21.15.62:0/3879655019 >> 172.21.15.62:0/3879655019 conn(0x7f82c4066d10 msgr2=0x7f82c4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.299+0000 7f82cb000640 1 -- 172.21.15.62:0/3879655019 shutdown_connections 2024-04-25T09:16:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.299+0000 7f82cb000640 1 -- 172.21.15.62:0/3879655019 wait complete. 2024-04-25T09:16:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.301+0000 7f82cb000640 1 Processor -- start 2024-04-25T09:16:47.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.301+0000 7f82cb000640 1 -- start start 2024-04-25T09:16:47.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82cb000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82ac0039b0 0x7f82c414ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82cb000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c40caad0 0x7f82c4159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82cb000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40caf30 0x7f82c415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82cb000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40b2ee0 con 0x7f82c40caf30 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82cb000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40b2d60 con 0x7f82c40caad0 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82cb000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40b5ad0 con 0x7f82ac0039b0 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82ca600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40caf30 0x7f82c415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82ca600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40caf30 0x7f82c415a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35486/0 (socket says 172.21.15.62:35486) 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82ca600640 1 -- 172.21.15.62:0/3419288274 learned_addr learned my addr 172.21.15.62:0/3419288274 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82c9200640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c40caad0 0x7f82c4159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.302+0000 7f82c9c00640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82ac0039b0 0x7f82c414ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82ca600640 1 -- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82ac0039b0 msgr2=0x7f82c414ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82ca600640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82ac0039b0 0x7f82c414ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82ca600640 1 -- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c40caad0 msgr2=0x7f82c4159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82ca600640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c40caad0 0x7f82c4159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82ca600640 1 -- 172.21.15.62:0/3419288274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c40cc560 con 0x7f82c40caf30 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82ca600640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40caf30 0x7f82c415a010 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f82c40d4ad0 tx=0x7f82bc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82c1600640 1 -- 172.21.15.62:0/3419288274 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc017070 con 0x7f82c40caf30 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82c1600640 1 -- 172.21.15.62:0/3419288274 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc00c680 con 0x7f82c40caf30 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c40cb4c0 con 0x7f82c40caf30 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82c1600640 1 -- 172.21.15.62:0/3419288274 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00ca70 con 0x7f82c40caf30 2024-04-25T09:16:47.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.303+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c40cba80 con 0x7f82c40caf30 2024-04-25T09:16:47.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.304+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c40d0130 con 0x7f82c40caf30 2024-04-25T09:16:47.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.306+0000 7f82c1600640 1 -- 172.21.15.62:0/3419288274 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82bc01a070 con 0x7f82c40caf30 2024-04-25T09:16:47.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.307+0000 7f82c1600640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.307+0000 7f82c1600640 1 -- 172.21.15.62:0/3419288274 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82bc089dc0 con 0x7f82c40caf30 2024-04-25T09:16:47.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.308+0000 7f82c9c00640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.309+0000 7f82c9c00640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f82b8032000 tx=0x7f82b8009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:47.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.312+0000 7f82c1600640 1 -- 172.21.15.62:0/3419288274 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82c40d0130 con 0x7f82c40caf30 2024-04-25T09:16:47.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.555+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82c40ce580 con 0x7f82c40caf30 2024-04-25T09:16:47.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.556+0000 7f82c1600640 1 -- 172.21.15.62:0/3419288274 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f82bc058c30 con 0x7f82c40caf30 2024-04-25T09:16:47.543 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:47.543 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:47.543 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.558+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 msgr2=0x7f82a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.558+0000 7f82cb000640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f82b8032000 tx=0x7f82b8009030 comp rx=0 tx=0).stop 2024-04-25T09:16:47.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.558+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40caf30 msgr2=0x7f82c415a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.558+0000 7f82cb000640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40caf30 0x7f82c415a010 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f82c40d4ad0 tx=0x7f82bc00ec00 comp rx=0 tx=0).stop 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.559+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 shutdown_connections 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.559+0000 7f82cb000640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82a806b4d0 0x7f82a806d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.559+0000 7f82cb000640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82c40caf30 0x7f82c415a010 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.559+0000 7f82cb000640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c40caad0 0x7f82c4159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.559+0000 7f82cb000640 1 --2- 172.21.15.62:0/3419288274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82ac0039b0 0x7f82c414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.559+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 >> 172.21.15.62:0/3419288274 conn(0x7f82c4066d10 msgr2=0x7f82c415e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.559+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 shutdown_connections 2024-04-25T09:16:47.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.559+0000 7f82cb000640 1 -- 172.21.15.62:0/3419288274 wait complete. 2024-04-25T09:16:47.566 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:16:47.566 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:16:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.795+0000 7f5631000640 1 Processor -- start 2024-04-25T09:16:47.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.795+0000 7f5631000640 1 -- start start 2024-04-25T09:16:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.796+0000 7f5631000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f562c0cfc90 0x7f562c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.796+0000 7f5631000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c071500 con 0x7f562c0ce6b0 2024-04-25T09:16:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.796+0000 7f5631000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0717e0 con 0x7f562c0b3790 2024-04-25T09:16:47.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.796+0000 7f5631000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c15ac00 con 0x7f562c0cfc90 2024-04-25T09:16:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.796+0000 7f562aa00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f562c0cfc90 0x7f562c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.796+0000 7f562be00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f562c0ce6b0 0x7f562c0ceae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50492/0 (socket says 172.21.15.62:50492) 2024-04-25T09:16:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.796+0000 7f562be00640 1 -- 172.21.15.62:0/1228191548 learned_addr learned my addr 172.21.15.62:0/1228191548 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:47.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.797+0000 7f562aa00640 1 -- 172.21.15.62:0/1228191548 >> v1:172.21.15.121:6789/0 conn(0x7f562c0b3790 legacy=0x7f562c0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.797+0000 7f562aa00640 1 -- 172.21.15.62:0/1228191548 >> v1:172.21.15.62:6789/0 conn(0x7f562c0ce6b0 legacy=0x7f562c0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.797+0000 7f562aa00640 1 -- 172.21.15.62:0/1228191548 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c071ac0 con 0x7f562c0cfc90 2024-04-25T09:16:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.797+0000 7f562a000640 1 -- 172.21.15.62:0/1228191548 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404225342 0 0) 0x7f562c071500 con 0x7f562c0ce6b0 2024-04-25T09:16:47.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.798+0000 7f562aa00640 1 --2- 172.21.15.62:0/1228191548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0cfc90 0x7f562c15ed30 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f56180097a0 tx=0x7f561802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3625d31b01f691a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.798+0000 7f562a000640 1 -- 172.21.15.62:0/1228191548 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618037070 con 0x7f562c0cfc90 2024-04-25T09:16:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.798+0000 7f562a000640 1 -- 172.21.15.62:0/1228191548 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5618032440 con 0x7f562c0cfc90 2024-04-25T09:16:47.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.798+0000 7f562a000640 1 -- 172.21.15.62:0/1228191548 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618032810 con 0x7f562c0cfc90 2024-04-25T09:16:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.799+0000 7f5631000640 1 -- 172.21.15.62:0/1228191548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0cfc90 msgr2=0x7f562c15ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.799+0000 7f5631000640 1 --2- 172.21.15.62:0/1228191548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0cfc90 0x7f562c15ed30 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f56180097a0 tx=0x7f561802a230 comp rx=0 tx=0).stop 2024-04-25T09:16:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.799+0000 7f5631000640 1 -- 172.21.15.62:0/1228191548 shutdown_connections 2024-04-25T09:16:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.799+0000 7f5631000640 1 --2- 172.21.15.62:0/1228191548 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0cfc90 0x7f562c15ed30 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.799+0000 7f5631000640 1 -- 172.21.15.62:0/1228191548 >> 172.21.15.62:0/1228191548 conn(0x7f562c0b2490 msgr2=0x7f562c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.799+0000 7f5631000640 1 -- 172.21.15.62:0/1228191548 shutdown_connections 2024-04-25T09:16:47.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.799+0000 7f5631000640 1 -- 172.21.15.62:0/1228191548 wait complete. 2024-04-25T09:16:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.801+0000 7f5631000640 1 Processor -- start 2024-04-25T09:16:47.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.801+0000 7f5631000640 1 -- start start 2024-04-25T09:16:47.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f5631000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0b3790 0x7f562c152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f5631000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f562c0ce6b0 0x7f562c06f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f5631000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f562c0cfc90 0x7f562c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f5631000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c072080 con 0x7f562c0b3790 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f5631000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15f270 con 0x7f562c0cfc90 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f5631000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0cf1f0 con 0x7f562c0ce6b0 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f562b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0b3790 0x7f562c152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f562b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0b3790 0x7f562c152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35502/0 (socket says 172.21.15.62:35502) 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.802+0000 7f562b400640 1 -- 172.21.15.62:0/1166746596 learned_addr learned my addr 172.21.15.62:0/1166746596 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f562aa00640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f562c0ce6b0 0x7f562c06f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f562b400640 1 -- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f562c0ce6b0 msgr2=0x7f562c06f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f562b400640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f562c0ce6b0 0x7f562c06f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f562b400640 1 -- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f562c0cfc90 msgr2=0x7f562c15ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f562b400640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f562c0cfc90 0x7f562c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f562b400640 1 -- 172.21.15.62:0/1166746596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c15c680 con 0x7f562c0b3790 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f562b400640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0b3790 0x7f562c152380 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f561c00c480 tx=0x7f561c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f5623400640 1 -- 172.21.15.62:0/1166746596 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0052d0 con 0x7f562c0b3790 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f5623400640 1 -- 172.21.15.62:0/1166746596 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c005470 con 0x7f562c0b3790 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f5623400640 1 -- 172.21.15.62:0/1166746596 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c012870 con 0x7f562c0b3790 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.803+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c15c960 con 0x7f562c0b3790 2024-04-25T09:16:47.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.804+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c15cf20 con 0x7f562c0b3790 2024-04-25T09:16:47.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.805+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c067c40 con 0x7f562c0b3790 2024-04-25T09:16:47.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.806+0000 7f5623400640 1 -- 172.21.15.62:0/1166746596 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f561c012a10 con 0x7f562c0b3790 2024-04-25T09:16:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.807+0000 7f5623400640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f560c06b4d0 0x7f560c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.807+0000 7f5623400640 1 -- 172.21.15.62:0/1166746596 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f561c089ea0 con 0x7f562c0b3790 2024-04-25T09:16:47.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.807+0000 7f562aa00640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f560c06b4d0 0x7f560c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:47.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.809+0000 7f562aa00640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f560c06b4d0 0x7f560c06d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5618034000 tx=0x7f561802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:47.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:47.811+0000 7f5623400640 1 -- 172.21.15.62:0/1166746596 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f562c067c40 con 0x7f562c0b3790 2024-04-25T09:16:48.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.082+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f562c072d10 con 0x7f562c0b3790 2024-04-25T09:16:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.083+0000 7f5623400640 1 -- 172.21.15.62:0/1166746596 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1027 (secure 0 0 0) 0x7f561c058d10 con 0x7f562c0b3790 2024-04-25T09:16:48.070 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:48.070 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":835,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076573624,"bytes_used":2626936832,"bytes_avail":770467176448,"bytes_total":773094113280,"write_bytes_sec":33991948,"read_op_per_sec":0,"write_op_per_sec":35},"fsmap":{"epoch":86,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8886}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-25T09:15:50.176907+0000","services":{}},"progress_events":{}} 2024-04-25T09:16:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f560c06b4d0 msgr2=0x7f560c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f560c06b4d0 0x7f560c06d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5618034000 tx=0x7f561802e000 comp rx=0 tx=0).stop 2024-04-25T09:16:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0b3790 msgr2=0x7f562c152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0b3790 0x7f562c152380 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f561c00c480 tx=0x7f561c00c940 comp rx=0 tx=0).stop 2024-04-25T09:16:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 shutdown_connections 2024-04-25T09:16:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f560c06b4d0 0x7f560c06d990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f562c0cfc90 0x7f562c15ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f562c0ce6b0 0x7f562c06f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 --2- 172.21.15.62:0/1166746596 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f562c0b3790 0x7f562c152380 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 >> 172.21.15.62:0/1166746596 conn(0x7f562c0b2490 msgr2=0x7f562c067180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 shutdown_connections 2024-04-25T09:16:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:48.086+0000 7f5631000640 1 -- 172.21.15.62:0/1166746596 wait complete. 2024-04-25T09:16:49.092 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:49.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.318+0000 7f5fff200640 1 Processor -- start 2024-04-25T09:16:49.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.318+0000 7f5fff200640 1 -- start start 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.319+0000 7f5fff200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60000b56f0 0x7f6000160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5fff200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f60000b6a90 0x7f6000161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5fff200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60000b27a0 0x7f6000161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5fff200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6000161780 con 0x7f60000b56f0 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5fff200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60000cedc0 con 0x7f60000b6a90 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5fff200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60000d3110 con 0x7f60000b27a0 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffde00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60000b56f0 0x7f6000160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffde00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60000b56f0 0x7f6000160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35518/0 (socket says 172.21.15.62:35518) 2024-04-25T09:16:49.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffde00640 1 -- 172.21.15.62:0/3564811827 learned_addr learned my addr 172.21.15.62:0/3564811827 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:49.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffd400640 1 --2- 172.21.15.62:0/3564811827 >> v2:172.21.15.121:3300/0 conn(0x7f60000b6a90 0x7f6000161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffde00640 1 -- 172.21.15.62:0/3564811827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60000b27a0 msgr2=0x7f6000161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:49.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffde00640 1 --2- 172.21.15.62:0/3564811827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60000b27a0 0x7f6000161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffde00640 1 -- 172.21.15.62:0/3564811827 >> v2:172.21.15.121:3300/0 conn(0x7f60000b6a90 msgr2=0x7f6000161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:49.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffde00640 1 --2- 172.21.15.62:0/3564811827 >> v2:172.21.15.121:3300/0 conn(0x7f60000b6a90 0x7f6000161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffde00640 1 -- 172.21.15.62:0/3564811827 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000d33f0 con 0x7f60000b56f0 2024-04-25T09:16:49.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.320+0000 7f5ffd400640 1 --2- 172.21.15.62:0/3564811827 >> v2:172.21.15.121:3300/0 conn(0x7f60000b6a90 0x7f6000161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.322+0000 7f5ffde00640 1 --2- 172.21.15.62:0/3564811827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b56f0 0x7f6000160d00 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f600008c570 tx=0x7f5ff002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c0c18322c16396 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.322+0000 7f5ffca00640 1 -- 172.21.15.62:0/3564811827 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0037070 con 0x7f60000b56f0 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.322+0000 7f5ffca00640 1 -- 172.21.15.62:0/3564811827 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0032440 con 0x7f60000b56f0 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.322+0000 7f5ffca00640 1 -- 172.21.15.62:0/3564811827 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0032810 con 0x7f60000b56f0 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 -- 172.21.15.62:0/3564811827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b56f0 msgr2=0x7f6000160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 --2- 172.21.15.62:0/3564811827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b56f0 0x7f6000160d00 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f600008c570 tx=0x7f5ff002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 -- 172.21.15.62:0/3564811827 shutdown_connections 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 --2- 172.21.15.62:0/3564811827 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60000b27a0 0x7f6000161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 --2- 172.21.15.62:0/3564811827 >> v2:172.21.15.121:3300/0 conn(0x7f60000b6a90 0x7f6000161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 --2- 172.21.15.62:0/3564811827 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b56f0 0x7f6000160d00 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 -- 172.21.15.62:0/3564811827 >> 172.21.15.62:0/3564811827 conn(0x7f6000066d10 msgr2=0x7f6000067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:49.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 -- 172.21.15.62:0/3564811827 shutdown_connections 2024-04-25T09:16:49.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.323+0000 7f5fff200640 1 -- 172.21.15.62:0/3564811827 wait complete. 2024-04-25T09:16:49.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.325+0000 7f5fff200640 1 Processor -- start 2024-04-25T09:16:49.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.325+0000 7f5fff200640 1 -- start start 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5fff200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b27a0 0x7f60000cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5fff200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60000b56f0 0x7f600015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5fff200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60000b6a90 0x7f60000cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5fff200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60000b2d60 con 0x7f60000b27a0 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5fff200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60000cea50 con 0x7f60000b6a90 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5fff200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60000b2ee0 con 0x7f60000b56f0 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b27a0 0x7f60000cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b27a0 0x7f60000cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35520/0 (socket says 172.21.15.62:35520) 2024-04-25T09:16:49.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffde00640 1 -- 172.21.15.62:0/887748364 learned_addr learned my addr 172.21.15.62:0/887748364 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:49.313 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffde00640 1 -- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60000b56f0 msgr2=0x7f600015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffde00640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60000b56f0 0x7f600015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffde00640 1 -- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60000b6a90 msgr2=0x7f60000cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffe800640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60000b6a90 0x7f60000cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffde00640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60000b6a90 0x7f60000cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.326+0000 7f5ffde00640 1 -- 172.21.15.62:0/887748364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000cb990 con 0x7f60000b27a0 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.327+0000 7f5ffde00640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b27a0 0x7f60000cb450 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.327+0000 7f5ff6000640 1 -- 172.21.15.62:0/887748364 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0041070 con 0x7f60000b27a0 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.327+0000 7f5ff6000640 1 -- 172.21.15.62:0/887748364 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff003c870 con 0x7f60000b27a0 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.327+0000 7f5ff6000640 1 -- 172.21.15.62:0/887748364 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0037040 con 0x7f60000b27a0 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.327+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60000aa580 con 0x7f60000b27a0 2024-04-25T09:16:49.314 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.327+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60000cac50 con 0x7f60000b27a0 2024-04-25T09:16:49.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.328+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60000aaaa0 con 0x7f60000b27a0 2024-04-25T09:16:49.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.329+0000 7f5ff6000640 1 -- 172.21.15.62:0/887748364 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ff0044070 con 0x7f60000b27a0 2024-04-25T09:16:49.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.330+0000 7f5ff6000640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fdc06b4d0 0x7f5fdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.330+0000 7f5ff6000640 1 -- 172.21.15.62:0/887748364 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ff00aade0 con 0x7f60000b27a0 2024-04-25T09:16:49.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.331+0000 7f5ffd400640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fdc06b4d0 0x7f5fdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.332+0000 7f5ffd400640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fdc06b4d0 0x7f5fdc06d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006fd0 tx=0x7f5fe8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:49.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.334+0000 7f5ff6000640 1 -- 172.21.15.62:0/887748364 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ff0079c50 con 0x7f60000b27a0 2024-04-25T09:16:49.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.577+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60000ce580 con 0x7f60000b27a0 2024-04-25T09:16:49.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.580+0000 7f5ff6000640 1 -- 172.21.15.62:0/887748364 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5ff0079550 con 0x7f60000b27a0 2024-04-25T09:16:49.566 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:49.567 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:49.567 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:49.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fdc06b4d0 msgr2=0x7f5fdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fdc06b4d0 0x7f5fdc06d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8006fd0 tx=0x7f5fe8008040 comp rx=0 tx=0).stop 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b27a0 msgr2=0x7f60000cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b27a0 0x7f60000cb450 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff00046c0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 shutdown_connections 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5fdc06b4d0 0x7f5fdc06d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60000b6a90 0x7f60000cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f60000b56f0 0x7f600015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 --2- 172.21.15.62:0/887748364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60000b27a0 0x7f60000cb450 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.583+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 >> 172.21.15.62:0/887748364 conn(0x7f6000066d10 msgr2=0x7f60000b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.584+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 shutdown_connections 2024-04-25T09:16:49.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.584+0000 7f5fff200640 1 -- 172.21.15.62:0/887748364 wait complete. 2024-04-25T09:16:49.588 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:16:49.589 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:16:49.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.814+0000 7fb0af200640 1 Processor -- start 2024-04-25T09:16:49.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.814+0000 7fb0af200640 1 -- start start 2024-04-25T09:16:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.816+0000 7fb0af200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb0b015a9b0 0x7fb0b00afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.816+0000 7fb0af200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b00b0cf0 con 0x7fb0b00cdb30 2024-04-25T09:16:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.816+0000 7fb0af200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b00b0170 con 0x7fb0b006d9d0 2024-04-25T09:16:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.816+0000 7fb0af200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b00b0450 con 0x7fb0b015a9b0 2024-04-25T09:16:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.816+0000 7fb0ade00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb0b015a9b0 0x7fb0b00afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.816+0000 7fb0ade00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb0b015a9b0 0x7fb0b00afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35526/0 (socket says 172.21.15.62:35526) 2024-04-25T09:16:49.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.816+0000 7fb0ade00640 1 -- 172.21.15.62:0/1528020721 learned_addr learned my addr 172.21.15.62:0/1528020721 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:49.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.816+0000 7fb0ad400640 1 --1- 172.21.15.62:0/1528020721 >> v1:172.21.15.62:6789/0 conn(0x7fb0b00cdb30 0x7fb0b006d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50500/0 (socket says 172.21.15.62:50500) 2024-04-25T09:16:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.817+0000 7fb0ade00640 1 -- 172.21.15.62:0/1528020721 >> v1:172.21.15.121:6789/0 conn(0x7fb0b006d9d0 legacy=0x7fb0b00b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.817+0000 7fb0ade00640 1 -- 172.21.15.62:0/1528020721 >> v1:172.21.15.62:6789/0 conn(0x7fb0b00cdb30 legacy=0x7fb0b006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.817+0000 7fb0aca00640 1 -- 172.21.15.62:0/1528020721 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597148624 0 0) 0x7fb0b00b0cf0 con 0x7fb0b00cdb30 2024-04-25T09:16:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.817+0000 7fb0ade00640 1 -- 172.21.15.62:0/1528020721 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b00b0730 con 0x7fb0b015a9b0 2024-04-25T09:16:49.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.818+0000 7fb0ade00640 1 --2- 172.21.15.62:0/1528020721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b00afa40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009870 tx=0x7fb0a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308591a1e022b3e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.818+0000 7fb0aca00640 1 -- 172.21.15.62:0/1528020721 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0037070 con 0x7fb0b015a9b0 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.818+0000 7fb0aca00640 1 -- 172.21.15.62:0/1528020721 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a0032440 con 0x7fb0b015a9b0 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.818+0000 7fb0aca00640 1 -- 172.21.15.62:0/1528020721 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0032810 con 0x7fb0b015a9b0 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.819+0000 7fb0af200640 1 -- 172.21.15.62:0/1528020721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 msgr2=0x7fb0b00afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.819+0000 7fb0af200640 1 --2- 172.21.15.62:0/1528020721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b00afa40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009870 tx=0x7fb0a002a210 comp rx=0 tx=0).stop 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.819+0000 7fb0af200640 1 -- 172.21.15.62:0/1528020721 shutdown_connections 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.819+0000 7fb0af200640 1 --2- 172.21.15.62:0/1528020721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b00afa40 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.819+0000 7fb0af200640 1 -- 172.21.15.62:0/1528020721 >> 172.21.15.62:0/1528020721 conn(0x7fb0b00668f0 msgr2=0x7fb0b0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.819+0000 7fb0af200640 1 -- 172.21.15.62:0/1528020721 shutdown_connections 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.819+0000 7fb0af200640 1 -- 172.21.15.62:0/1528020721 wait complete. 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.821+0000 7fb0af200640 1 Processor -- start 2024-04-25T09:16:49.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.821+0000 7fb0af200640 1 -- start start 2024-04-25T09:16:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0af200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b006d9d0 0x7fb0b0146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0af200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0b00cdb30 0x7fb0b01574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0af200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b0157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0af200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b00b46d0 con 0x7fb0b015a9b0 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0af200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b015ad90 con 0x7fb0b006d9d0 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0af200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b00b4850 con 0x7fb0b00cdb30 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0ae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b0157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0ae800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b0157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35528/0 (socket says 172.21.15.62:35528) 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0ae800640 1 -- 172.21.15.62:0/4080720225 learned_addr learned my addr 172.21.15.62:0/4080720225 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0ad400640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0b00cdb30 0x7fb0b01574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0ade00640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b006d9d0 0x7fb0b0146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0ae800640 1 -- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0b00cdb30 msgr2=0x7fb0b01574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0ae800640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0b00cdb30 0x7fb0b01574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.822+0000 7fb0ae800640 1 -- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b006d9d0 msgr2=0x7fb0b0146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0ae800640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b006d9d0 0x7fb0b0146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0ae800640 1 -- 172.21.15.62:0/4080720225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b0158220 con 0x7fb0b015a9b0 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0ade00640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b006d9d0 0x7fb0b0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0ae800640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b0157a00 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c7b0 tx=0x7fb0a800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0a6000640 1 -- 172.21.15.62:0/4080720225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004b60 con 0x7fb0b015a9b0 2024-04-25T09:16:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0a6000640 1 -- 172.21.15.62:0/4080720225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8004d00 con 0x7fb0b015a9b0 2024-04-25T09:16:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b0158500 con 0x7fb0b015a9b0 2024-04-25T09:16:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0a6000640 1 -- 172.21.15.62:0/4080720225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800aa90 con 0x7fb0b015a9b0 2024-04-25T09:16:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.823+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b0158ac0 con 0x7fb0b015a9b0 2024-04-25T09:16:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.824+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b00cd460 con 0x7fb0b015a9b0 2024-04-25T09:16:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.825+0000 7fb0a6000640 1 -- 172.21.15.62:0/4080720225 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0a800ac30 con 0x7fb0b015a9b0 2024-04-25T09:16:49.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.826+0000 7fb0a6000640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb09006b500 0x7fb09006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.827+0000 7fb0a6000640 1 -- 172.21.15.62:0/4080720225 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0a808a500 con 0x7fb0b015a9b0 2024-04-25T09:16:49.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.827+0000 7fb0ade00640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb09006b500 0x7fb09006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:49.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.828+0000 7fb0ade00640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb09006b500 0x7fb09006d9c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0034000 tx=0x7fb0a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:49.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:49.831+0000 7fb0a6000640 1 -- 172.21.15.62:0/4080720225 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0b00cd460 con 0x7fb0b015a9b0 2024-04-25T09:16:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.105+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb0b0072b10 con 0x7fb0b015a9b0 2024-04-25T09:16:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.106+0000 7fb0a6000640 1 -- 172.21.15.62:0/4080720225 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fb0a8020070 con 0x7fb0b015a9b0 2024-04-25T09:16:50.092 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:50.092 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":837,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076573624,"bytes_used":2626936832,"bytes_avail":770467176448,"bytes_total":773094113280,"write_bytes_sec":18671353,"read_op_per_sec":0,"write_op_per_sec":19},"fsmap":{"epoch":86,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8886}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T09:16:48.199137+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:16:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.108+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb09006b500 msgr2=0x7fb09006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.108+0000 7fb0af200640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb09006b500 0x7fb09006d9c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0034000 tx=0x7fb0a002e000 comp rx=0 tx=0).stop 2024-04-25T09:16:50.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 msgr2=0x7fb0b0157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b0157a00 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c7b0 tx=0x7fb0a800cc70 comp rx=0 tx=0).stop 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 shutdown_connections 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb09006b500 0x7fb09006d9c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb0b015a9b0 0x7fb0b0157a00 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb0b00cdb30 0x7fb0b01574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 --2- 172.21.15.62:0/4080720225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb0b006d9d0 0x7fb0b0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 >> 172.21.15.62:0/4080720225 conn(0x7fb0b00668f0 msgr2=0x7fb0b0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 shutdown_connections 2024-04-25T09:16:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.109+0000 7fb0af200640 1 -- 172.21.15.62:0/4080720225 wait complete. 2024-04-25T09:16:50.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:16:50.116 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:16:50.116 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:16:50.143 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:16:50.143 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'d', 'b', 'a'} 2024-04-25T09:16:50.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T09:16:50.409 INFO:teuthology.orchestra.run.smithi062.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T09:16:50.409 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T09:16:50.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:50.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.629+0000 7f6bbec00640 1 Processor -- start 2024-04-25T09:16:50.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.629+0000 7f6bbec00640 1 -- start start 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.630+0000 7f6bbec00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6bb815dc40 0x7f6bb80d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.630+0000 7f6bbec00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6bb80d0ae0 0x7f6bb80d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.630+0000 7f6bbec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80d3ae0 0x7f6bb80d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.630+0000 7f6bbec00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb80d2620 con 0x7f6bb815dc40 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.630+0000 7f6bbec00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb80d2900 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.630+0000 7f6bbec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb80d2be0 con 0x7f6bb80d3ae0 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.631+0000 7f6bb7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6bb815dc40 0x7f6bb80d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.631+0000 7f6bb7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6bb815dc40 0x7f6bb80d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35532/0 (socket says 172.21.15.62:35532) 2024-04-25T09:16:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.631+0000 7f6bb7e00640 1 -- 172.21.15.62:0/3724144319 learned_addr learned my addr 172.21.15.62:0/3724144319 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:50.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.631+0000 7f6bb7400640 1 --2- 172.21.15.62:0/3724144319 >> v2:172.21.15.121:3300/0 conn(0x7f6bb80d0ae0 0x7f6bb80d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:50.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.631+0000 7f6bbcc00640 1 --2- 172.21.15.62:0/3724144319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80d3ae0 0x7f6bb80d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:50.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.632+0000 7f6bb7e00640 1 -- 172.21.15.62:0/3724144319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80d3ae0 msgr2=0x7f6bb80d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:50.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.632+0000 7f6bb7e00640 1 --2- 172.21.15.62:0/3724144319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80d3ae0 0x7f6bb80d3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.632+0000 7f6bb7e00640 1 -- 172.21.15.62:0/3724144319 >> v2:172.21.15.121:3300/0 conn(0x7f6bb80d0ae0 msgr2=0x7f6bb80d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:50.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.632+0000 7f6bb7e00640 1 --2- 172.21.15.62:0/3724144319 >> v2:172.21.15.121:3300/0 conn(0x7f6bb80d0ae0 0x7f6bb80d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.632+0000 7f6bb7e00640 1 -- 172.21.15.62:0/3724144319 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb80d2ec0 con 0x7f6bb815dc40 2024-04-25T09:16:50.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.633+0000 7f6bb7e00640 1 --2- 172.21.15.62:0/3724144319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb815dc40 0x7f6bb80d1cb0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009e50 tx=0x7f6ba802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d46a7555e2a9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:50.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.633+0000 7f6bb6a00640 1 -- 172.21.15.62:0/3724144319 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8036070 con 0x7f6bb815dc40 2024-04-25T09:16:50.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.633+0000 7f6bb6a00640 1 -- 172.21.15.62:0/3724144319 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba8031600 con 0x7f6bb815dc40 2024-04-25T09:16:50.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.633+0000 7f6bb6a00640 1 -- 172.21.15.62:0/3724144319 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80319d0 con 0x7f6bb815dc40 2024-04-25T09:16:50.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.634+0000 7f6bbec00640 1 -- 172.21.15.62:0/3724144319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb815dc40 msgr2=0x7f6bb80d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.634+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3724144319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb815dc40 0x7f6bb80d1cb0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009e50 tx=0x7f6ba802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.634+0000 7f6bbec00640 1 -- 172.21.15.62:0/3724144319 shutdown_connections 2024-04-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.634+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3724144319 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb80d3ae0 0x7f6bb80d3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.634+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3724144319 >> v2:172.21.15.121:3300/0 conn(0x7f6bb80d0ae0 0x7f6bb80d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.634+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3724144319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb815dc40 0x7f6bb80d1cb0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.634+0000 7f6bbec00640 1 -- 172.21.15.62:0/3724144319 >> 172.21.15.62:0/3724144319 conn(0x7f6bb806b000 msgr2=0x7f6bb806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.634+0000 7f6bbec00640 1 -- 172.21.15.62:0/3724144319 shutdown_connections 2024-04-25T09:16:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.635+0000 7f6bbec00640 1 -- 172.21.15.62:0/3724144319 wait complete. 2024-04-25T09:16:50.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.636+0000 7f6bbec00640 1 Processor -- start 2024-04-25T09:16:50.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.636+0000 7f6bbec00640 1 -- start start 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.637+0000 7f6bbec00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb80d0ae0 0x7f6bb81587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.637+0000 7f6bbec00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb80d3ae0 0x7f6bb8152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.637+0000 7f6bbec00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb815dc40 0x7f6bb8154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.637+0000 7f6bbec00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb80d36c0 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.637+0000 7f6bbec00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb80d21f0 con 0x7f6bb80d3ae0 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.637+0000 7f6bbec00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb81f40d0 con 0x7f6bb815dc40 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb80d0ae0 0x7f6bb81587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb80d0ae0 0x7f6bb81587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35536/0 (socket says 172.21.15.62:35536) 2024-04-25T09:16:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 -- 172.21.15.62:0/3800101925 learned_addr learned my addr 172.21.15.62:0/3800101925 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7400640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb80d3ae0 0x7f6bb8152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 -- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb815dc40 msgr2=0x7f6bb8154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb815dc40 0x7f6bb8154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 -- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb80d3ae0 msgr2=0x7f6bb8152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb80d3ae0 0x7f6bb8152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 -- 172.21.15.62:0/3800101925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb8159780 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.638+0000 7f6bb7e00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb80d0ae0 0x7f6bb81587b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8034040 tx=0x7f6ba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.639+0000 7f6bafe00640 1 -- 172.21.15.62:0/3800101925 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8036050 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.639+0000 7f6bafe00640 1 -- 172.21.15.62:0/3800101925 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba8042070 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.639+0000 7f6bafe00640 1 -- 172.21.15.62:0/3800101925 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8009510 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.639+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb8158ee0 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.639+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb81594a0 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.640+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb80b2f10 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.641+0000 7f6bafe00640 1 -- 172.21.15.62:0/3800101925 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ba8007c50 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.642+0000 7f6bafe00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b9406b4d0 0x7f6b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:50.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.642+0000 7f6bafe00640 1 -- 172.21.15.62:0/3800101925 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ba80ab100 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.643+0000 7f6bb7400640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b9406b4d0 0x7f6b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:50.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.644+0000 7f6bb7400640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b9406b4d0 0x7f6b9406d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6bb815a860 tx=0x7f6ba0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:50.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.647+0000 7f6bafe00640 1 -- 172.21.15.62:0/3800101925 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6bb80b2f10 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.893+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bb80729c0 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.894+0000 7f6bafe00640 1 -- 172.21.15.62:0/3800101925 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6ba8079f70 con 0x7f6bb80d0ae0 2024-04-25T09:16:50.880 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:50.880 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:50.881 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b9406b4d0 msgr2=0x7f6b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b9406b4d0 0x7f6b9406d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6bb815a860 tx=0x7f6ba0008040 comp rx=0 tx=0).stop 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb80d0ae0 msgr2=0x7f6bb81587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb80d0ae0 0x7f6bb81587b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8034040 tx=0x7f6ba8004a40 comp rx=0 tx=0).stop 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 shutdown_connections 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6b9406b4d0 0x7f6b9406d990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6bb815dc40 0x7f6bb8154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6bb80d3ae0 0x7f6bb8152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 --2- 172.21.15.62:0/3800101925 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6bb80d0ae0 0x7f6bb81587b0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 >> 172.21.15.62:0/3800101925 conn(0x7f6bb806b000 msgr2=0x7f6bb80d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:50.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 shutdown_connections 2024-04-25T09:16:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:50.897+0000 7f6bbec00640 1 -- 172.21.15.62:0/3800101925 wait complete. 2024-04-25T09:16:50.903 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:16:55.905 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:16:56.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.141+0000 7fd6ee000640 1 Processor -- start 2024-04-25T09:16:56.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.142+0000 7fd6ee000640 1 -- start start 2024-04-25T09:16:56.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.143+0000 7fd6ee000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6e80b27a0 0x7fd6e80d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:56.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.143+0000 7fd6ee000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e80d3110 con 0x7fd6e80b56f0 2024-04-25T09:16:56.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.143+0000 7fd6ee000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e80d33f0 con 0x7fd6e80b6a90 2024-04-25T09:16:56.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.143+0000 7fd6ee000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e80cedc0 con 0x7fd6e80b27a0 2024-04-25T09:16:56.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.144+0000 7fd6ed600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6e80b27a0 0x7fd6e80d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:56.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.144+0000 7fd6ecc00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd6e80b56f0 0x7fd6e8160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33986/0 (socket says 172.21.15.62:33986) 2024-04-25T09:16:56.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.144+0000 7fd6ecc00640 1 -- 172.21.15.62:0/1495151424 learned_addr learned my addr 172.21.15.62:0/1495151424 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:16:56.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.144+0000 7fd6ed600640 1 -- 172.21.15.62:0/1495151424 >> v1:172.21.15.121:6789/0 conn(0x7fd6e80b6a90 legacy=0x7fd6e80d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:16:56.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.144+0000 7fd6ed600640 1 -- 172.21.15.62:0/1495151424 >> v1:172.21.15.62:6789/0 conn(0x7fd6e80b56f0 legacy=0x7fd6e8160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:56.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.144+0000 7fd6ed600640 1 -- 172.21.15.62:0/1495151424 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e80d36d0 con 0x7fd6e80b27a0 2024-04-25T09:16:56.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.145+0000 7fd6ed600640 1 --2- 172.21.15.62:0/1495151424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b27a0 0x7fd6e80d26f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00097a0 tx=0x7fd6e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c65d3a5b0be334d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.146+0000 7fd6e7400640 1 -- 172.21.15.62:0/1495151424 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0037070 con 0x7fd6e80b27a0 2024-04-25T09:16:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.146+0000 7fd6e7400640 1 -- 172.21.15.62:0/1495151424 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e0032440 con 0x7fd6e80b27a0 2024-04-25T09:16:56.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.146+0000 7fd6e7400640 1 -- 172.21.15.62:0/1495151424 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0032810 con 0x7fd6e80b27a0 2024-04-25T09:16:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.146+0000 7fd6ee000640 1 -- 172.21.15.62:0/1495151424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b27a0 msgr2=0x7fd6e80d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.146+0000 7fd6ee000640 1 --2- 172.21.15.62:0/1495151424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b27a0 0x7fd6e80d26f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00097a0 tx=0x7fd6e002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.147+0000 7fd6ee000640 1 -- 172.21.15.62:0/1495151424 shutdown_connections 2024-04-25T09:16:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.147+0000 7fd6ee000640 1 --2- 172.21.15.62:0/1495151424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b27a0 0x7fd6e80d26f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.147+0000 7fd6ee000640 1 -- 172.21.15.62:0/1495151424 >> 172.21.15.62:0/1495151424 conn(0x7fd6e8066d10 msgr2=0x7fd6e8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.147+0000 7fd6ee000640 1 -- 172.21.15.62:0/1495151424 shutdown_connections 2024-04-25T09:16:56.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.147+0000 7fd6ee000640 1 -- 172.21.15.62:0/1495151424 wait complete. 2024-04-25T09:16:56.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.149+0000 7fd6ee000640 1 Processor -- start 2024-04-25T09:16:56.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.149+0000 7fd6ee000640 1 -- start start 2024-04-25T09:16:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.150+0000 7fd6ee000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6e80b27a0 0x7fd6e8157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.150+0000 7fd6ee000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6e80b56f0 0x7fd6e815ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.150+0000 7fd6ee000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b6a90 0x7fd6e815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.150+0000 7fd6ee000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80b2ee0 con 0x7fd6e80b6a90 2024-04-25T09:16:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.150+0000 7fd6ee000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80b2d60 con 0x7fd6e80b56f0 2024-04-25T09:16:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.150+0000 7fd6ee000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80d2c30 con 0x7fd6e80b27a0 2024-04-25T09:16:56.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ed600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b6a90 0x7fd6e815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ed600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b6a90 0x7fd6e815d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44732/0 (socket says 172.21.15.62:44732) 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ed600640 1 -- 172.21.15.62:0/4140887069 learned_addr learned my addr 172.21.15.62:0/4140887069 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ecc00640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6e80b27a0 0x7fd6e8157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6e7e00640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6e80b56f0 0x7fd6e815ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ed600640 1 -- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6e80b27a0 msgr2=0x7fd6e8157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ed600640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6e80b27a0 0x7fd6e8157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ed600640 1 -- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6e80b56f0 msgr2=0x7fd6e815ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ed600640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6e80b56f0 0x7fd6e815ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6ed600640 1 -- 172.21.15.62:0/4140887069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8158520 con 0x7fd6e80b6a90 2024-04-25T09:16:56.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.151+0000 7fd6e7e00640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6e80b56f0 0x7fd6e815ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:16:56.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.152+0000 7fd6ed600640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b6a90 0x7fd6e815d710 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00097a0 tx=0x7fd6e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:56.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.152+0000 7fd6e4c00640 1 -- 172.21.15.62:0/4140887069 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0041070 con 0x7fd6e80b6a90 2024-04-25T09:16:56.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.152+0000 7fd6e4c00640 1 -- 172.21.15.62:0/4140887069 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e003c870 con 0x7fd6e80b6a90 2024-04-25T09:16:56.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.152+0000 7fd6e4c00640 1 -- 172.21.15.62:0/4140887069 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0037040 con 0x7fd6e80b6a90 2024-04-25T09:16:56.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.152+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e8158800 con 0x7fd6e80b6a90 2024-04-25T09:16:56.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.152+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e8158dc0 con 0x7fd6e80b6a90 2024-04-25T09:16:56.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.153+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e804ef20 con 0x7fd6e80b6a90 2024-04-25T09:16:56.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.154+0000 7fd6e4c00640 1 -- 172.21.15.62:0/4140887069 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6e0044070 con 0x7fd6e80b6a90 2024-04-25T09:16:56.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.156+0000 7fd6e4c00640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6c406b4d0 0x7fd6c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:16:56.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.156+0000 7fd6e4c00640 1 -- 172.21.15.62:0/4140887069 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6e00aadf0 con 0x7fd6e80b6a90 2024-04-25T09:16:56.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.156+0000 7fd6ecc00640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6c406b4d0 0x7fd6c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:16:56.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.157+0000 7fd6ecc00640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6c406b4d0 0x7fd6c406d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8000f60 tx=0x7fd6d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:16:56.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.161+0000 7fd6e4c00640 1 -- 172.21.15.62:0/4140887069 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6e0079bd0 con 0x7fd6e80b6a90 2024-04-25T09:16:56.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.401+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6e80b6180 con 0x7fd6e80b6a90 2024-04-25T09:16:56.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.401+0000 7fd6e4c00640 1 -- 172.21.15.62:0/4140887069 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd6e00794d0 con 0x7fd6e80b6a90 2024-04-25T09:16:56.388 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:16:56.388 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:16:56.388 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:16:56.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6c406b4d0 msgr2=0x7fd6c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:56.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6c406b4d0 0x7fd6c406d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8000f60 tx=0x7fd6d8009040 comp rx=0 tx=0).stop 2024-04-25T09:16:56.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b6a90 msgr2=0x7fd6e815d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:16:56.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b6a90 0x7fd6e815d710 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00097a0 tx=0x7fd6e00046c0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 shutdown_connections 2024-04-25T09:16:56.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd6c406b4d0 0x7fd6c406d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6e80b6a90 0x7fd6e815d710 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd6e80b56f0 0x7fd6e815ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 --2- 172.21.15.62:0/4140887069 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd6e80b27a0 0x7fd6e8157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:16:56.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 >> 172.21.15.62:0/4140887069 conn(0x7fd6e8066d10 msgr2=0x7fd6e80d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:16:56.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 shutdown_connections 2024-04-25T09:16:56.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:16:56.404+0000 7fd6ee000640 1 -- 172.21.15.62:0/4140887069 wait complete. 2024-04-25T09:16:56.410 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:17:01.411 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:01.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.640+0000 7fb36ac00640 1 Processor -- start 2024-04-25T09:17:01.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.640+0000 7fb36ac00640 1 -- start start 2024-04-25T09:17:01.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.642+0000 7fb36ac00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3641453b0 0x7fb3640cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.642+0000 7fb36ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640d0200 0x7fb3640b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.642+0000 7fb36ac00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb3640b60a0 0x7fb3640668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.642+0000 7fb36ac00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3640b6500 con 0x7fb3641453b0 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.642+0000 7fb36ac00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb364145790 con 0x7fb3640b60a0 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.642+0000 7fb36ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415c110 con 0x7fb3640d0200 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb369800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3641453b0 0x7fb3640cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb369800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb3641453b0 0x7fb3640cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44734/0 (socket says 172.21.15.62:44734) 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb369800640 1 -- 172.21.15.62:0/2670116566 learned_addr learned my addr 172.21.15.62:0/2670116566 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:01.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb36a200640 1 --2- 172.21.15.62:0/2670116566 >> v2:172.21.15.121:3300/0 conn(0x7fb3640b60a0 0x7fb3640668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:01.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb369800640 1 -- 172.21.15.62:0/2670116566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640d0200 msgr2=0x7fb3640b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:01.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb369800640 1 --2- 172.21.15.62:0/2670116566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640d0200 0x7fb3640b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb369800640 1 -- 172.21.15.62:0/2670116566 >> v2:172.21.15.121:3300/0 conn(0x7fb3640b60a0 msgr2=0x7fb3640668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:01.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb369800640 1 --2- 172.21.15.62:0/2670116566 >> v2:172.21.15.121:3300/0 conn(0x7fb3640b60a0 0x7fb3640668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.643+0000 7fb369800640 1 -- 172.21.15.62:0/2670116566 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36415c3b0 con 0x7fb3641453b0 2024-04-25T09:17:01.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.644+0000 7fb36a200640 1 --2- 172.21.15.62:0/2670116566 >> v2:172.21.15.121:3300/0 conn(0x7fb3640b60a0 0x7fb3640668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:01.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.645+0000 7fb369800640 1 --2- 172.21.15.62:0/2670116566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cfcc0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb364159cb0 tx=0x7fb35802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d4e54174dd82c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:01.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.645+0000 7fb363e00640 1 -- 172.21.15.62:0/2670116566 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358037070 con 0x7fb3641453b0 2024-04-25T09:17:01.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.645+0000 7fb363e00640 1 -- 172.21.15.62:0/2670116566 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358032440 con 0x7fb3641453b0 2024-04-25T09:17:01.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.645+0000 7fb363e00640 1 -- 172.21.15.62:0/2670116566 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358032810 con 0x7fb3641453b0 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.646+0000 7fb36ac00640 1 -- 172.21.15.62:0/2670116566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 msgr2=0x7fb3640cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.646+0000 7fb36ac00640 1 --2- 172.21.15.62:0/2670116566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cfcc0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb364159cb0 tx=0x7fb35802a140 comp rx=0 tx=0).stop 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.646+0000 7fb36ac00640 1 -- 172.21.15.62:0/2670116566 shutdown_connections 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.646+0000 7fb36ac00640 1 --2- 172.21.15.62:0/2670116566 >> v2:172.21.15.121:3300/0 conn(0x7fb3640b60a0 0x7fb3640668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.646+0000 7fb36ac00640 1 --2- 172.21.15.62:0/2670116566 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640d0200 0x7fb3640b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.646+0000 7fb36ac00640 1 --2- 172.21.15.62:0/2670116566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cfcc0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.646+0000 7fb36ac00640 1 -- 172.21.15.62:0/2670116566 >> 172.21.15.62:0/2670116566 conn(0x7fb36406abe0 msgr2=0x7fb36406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.646+0000 7fb36ac00640 1 -- 172.21.15.62:0/2670116566 shutdown_connections 2024-04-25T09:17:01.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.647+0000 7fb36ac00640 1 -- 172.21.15.62:0/2670116566 wait complete. 2024-04-25T09:17:01.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.648+0000 7fb36ac00640 1 Processor -- start 2024-04-25T09:17:01.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.648+0000 7fb36ac00640 1 -- start start 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.649+0000 7fb36ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640b60a0 0x7fb36413b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.649+0000 7fb36ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3640d0200 0x7fb3640caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.649+0000 7fb36ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.649+0000 7fb36ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415cc50 con 0x7fb3641453b0 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.649+0000 7fb36ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415c970 con 0x7fb3640d0200 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.649+0000 7fb36ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb364072130 con 0x7fb3640b60a0 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44744/0 (socket says 172.21.15.62:44744) 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 -- 172.21.15.62:0/4246749168 learned_addr learned my addr 172.21.15.62:0/4246749168 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb369800640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640b60a0 0x7fb36413b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb368e00640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3640d0200 0x7fb3640caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:01.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 -- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640b60a0 msgr2=0x7fb36413b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640b60a0 0x7fb36413b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 -- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3640d0200 msgr2=0x7fb3640caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3640d0200 0x7fb3640caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 -- 172.21.15.62:0/4246749168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3640cb3e0 con 0x7fb3641453b0 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb368e00640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3640d0200 0x7fb3640caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.650+0000 7fb36a200640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cc340 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00cd50 tx=0x7fb35c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.651+0000 7fb361600640 1 -- 172.21.15.62:0/4246749168 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c00f680 con 0x7fb3641453b0 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.651+0000 7fb361600640 1 -- 172.21.15.62:0/4246749168 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c00f840 con 0x7fb3641453b0 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.651+0000 7fb361600640 1 -- 172.21.15.62:0/4246749168 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c00fc30 con 0x7fb3641453b0 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.651+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3640cb660 con 0x7fb3641453b0 2024-04-25T09:17:01.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.651+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3640c7a30 con 0x7fb3641453b0 2024-04-25T09:17:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.652+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3640cbb80 con 0x7fb3641453b0 2024-04-25T09:17:01.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.653+0000 7fb361600640 1 -- 172.21.15.62:0/4246749168 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb35c00e3e0 con 0x7fb3641453b0 2024-04-25T09:17:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.654+0000 7fb361600640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb34806b4d0 0x7fb34806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:01.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.654+0000 7fb361600640 1 -- 172.21.15.62:0/4246749168 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb35c08af30 con 0x7fb3641453b0 2024-04-25T09:17:01.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.655+0000 7fb369800640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb34806b4d0 0x7fb34806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:01.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.656+0000 7fb369800640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb34806b4d0 0x7fb34806d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb358034000 tx=0x7fb35802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:01.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.658+0000 7fb361600640 1 -- 172.21.15.62:0/4246749168 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3640cb3e0 con 0x7fb3641453b0 2024-04-25T09:17:01.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.898+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb36404ef20 con 0x7fb3641453b0 2024-04-25T09:17:01.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.899+0000 7fb361600640 1 -- 172.21.15.62:0/4246749168 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb35c014030 con 0x7fb3641453b0 2024-04-25T09:17:01.885 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:01.885 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:16:05.890185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8886":{"gid":8886,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/2303077743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":2303077743},{"type":"v1","addr":"172.21.15.62:6834","nonce":2303077743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:17:01.885 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 86 2024-04-25T09:17:01.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb34806b4d0 msgr2=0x7fb34806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:01.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb34806b4d0 0x7fb34806d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb358034000 tx=0x7fb35802e000 comp rx=0 tx=0).stop 2024-04-25T09:17:01.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 msgr2=0x7fb3640cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cc340 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00cd50 tx=0x7fb35c00a650 comp rx=0 tx=0).stop 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 shutdown_connections 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb34806b4d0 0x7fb34806d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb3641453b0 0x7fb3640cc340 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3640d0200 0x7fb3640caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 --2- 172.21.15.62:0/4246749168 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb3640b60a0 0x7fb36413b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 >> 172.21.15.62:0/4246749168 conn(0x7fb36406abe0 msgr2=0x7fb36406d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 shutdown_connections 2024-04-25T09:17:01.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:01.901+0000 7fb36ac00640 1 -- 172.21.15.62:0/4246749168 wait complete. 2024-04-25T09:17:01.908 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:17:06.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.142+0000 7f447a000640 1 Processor -- start 2024-04-25T09:17:07.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.142+0000 7f447a000640 1 -- start start 2024-04-25T09:17:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.143+0000 7f447a000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f44740b56f0 0x7f4474160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.143+0000 7f447a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f44740b27a0 0x7f4474161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.143+0000 7f447a000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44740d3110 con 0x7f44740b6a90 2024-04-25T09:17:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.143+0000 7f447a000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4474161780 con 0x7f44740b56f0 2024-04-25T09:17:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.144+0000 7f447a000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740cedc0 con 0x7f44740b27a0 2024-04-25T09:17:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.144+0000 7f4478c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f44740b56f0 0x7f4474160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.144+0000 7f4478c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f44740b56f0 0x7f4474160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47750/0 (socket says 172.21.15.62:47750) 2024-04-25T09:17:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.144+0000 7f4478c00640 1 -- 172.21.15.62:0/1757821878 learned_addr learned my addr 172.21.15.62:0/1757821878 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.144+0000 7f4479600640 1 --2- 172.21.15.62:0/1757821878 >> v2:172.21.15.121:3300/0 conn(0x7f44740b27a0 0x7f4474161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.145+0000 7f4478c00640 1 -- 172.21.15.62:0/1757821878 >> v2:172.21.15.121:3300/0 conn(0x7f44740b27a0 msgr2=0x7f4474161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.145+0000 7f4478c00640 1 --2- 172.21.15.62:0/1757821878 >> v2:172.21.15.121:3300/0 conn(0x7f44740b27a0 0x7f4474161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.145+0000 7f4478c00640 1 -- 172.21.15.62:0/1757821878 >> v1:172.21.15.121:6789/0 conn(0x7f44740b6a90 legacy=0x7f4474161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:07.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.145+0000 7f4478c00640 1 -- 172.21.15.62:0/1757821878 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740d33f0 con 0x7f44740b56f0 2024-04-25T09:17:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.146+0000 7f4478c00640 1 --2- 172.21.15.62:0/1757821878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474160d00 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f44740b2720 tx=0x7f446402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a589cedc5eb31695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.146+0000 7f4473400640 1 -- 172.21.15.62:0/1757821878 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464037070 con 0x7f44740b56f0 2024-04-25T09:17:07.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f4473400640 1 -- 172.21.15.62:0/1757821878 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4464032440 con 0x7f44740b56f0 2024-04-25T09:17:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f4473400640 1 -- 172.21.15.62:0/1757821878 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464032810 con 0x7f44740b56f0 2024-04-25T09:17:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f447a000640 1 -- 172.21.15.62:0/1757821878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 msgr2=0x7f4474160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f447a000640 1 --2- 172.21.15.62:0/1757821878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474160d00 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f44740b2720 tx=0x7f446402a230 comp rx=0 tx=0).stop 2024-04-25T09:17:07.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f447a000640 1 -- 172.21.15.62:0/1757821878 shutdown_connections 2024-04-25T09:17:07.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f447a000640 1 --2- 172.21.15.62:0/1757821878 >> v2:172.21.15.121:3300/0 conn(0x7f44740b27a0 0x7f4474161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f447a000640 1 --2- 172.21.15.62:0/1757821878 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474160d00 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f447a000640 1 -- 172.21.15.62:0/1757821878 >> 172.21.15.62:0/1757821878 conn(0x7f4474066d10 msgr2=0x7f4474067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:07.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f447a000640 1 -- 172.21.15.62:0/1757821878 shutdown_connections 2024-04-25T09:17:07.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.147+0000 7f447a000640 1 -- 172.21.15.62:0/1757821878 wait complete. 2024-04-25T09:17:07.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.149+0000 7f447a000640 1 Processor -- start 2024-04-25T09:17:07.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.149+0000 7f447a000640 1 -- start start 2024-04-25T09:17:07.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.150+0000 7f447a000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44740b27a0 0x7f44741e8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.150+0000 7f447a000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.150+0000 7f447a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44740b6a90 0x7f44740c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.150+0000 7f447a000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740b2d60 con 0x7f44740b56f0 2024-04-25T09:17:07.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.150+0000 7f447a000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740cea50 con 0x7f44740b6a90 2024-04-25T09:17:07.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.150+0000 7f447a000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44740b2ee0 con 0x7f44740b27a0 2024-04-25T09:17:07.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47764/0 (socket says 172.21.15.62:47764) 2024-04-25T09:17:07.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 -- 172.21.15.62:0/4148111989 learned_addr learned my addr 172.21.15.62:0/4148111989 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:07.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 -- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44740b27a0 msgr2=0x7f44741e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44740b27a0 0x7f44741e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 -- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44740b6a90 msgr2=0x7f44740c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4479600640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44740b6a90 0x7f44740c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44740b6a90 0x7f44740c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 -- 172.21.15.62:0/4148111989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44741e73c0 con 0x7f44740b56f0 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.151+0000 7f4473e00640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474150f50 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b3f0 tx=0x7f445c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.152+0000 7f4470c00640 1 -- 172.21.15.62:0/4148111989 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c015020 con 0x7f44740b56f0 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.152+0000 7f4470c00640 1 -- 172.21.15.62:0/4148111989 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c011830 con 0x7f44740b56f0 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.152+0000 7f4470c00640 1 -- 172.21.15.62:0/4148111989 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c011c20 con 0x7f44740b56f0 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.152+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44741e76a0 con 0x7f44740b56f0 2024-04-25T09:17:07.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.152+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44741e7c60 con 0x7f44740b56f0 2024-04-25T09:17:07.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.153+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447404ef20 con 0x7f44740b56f0 2024-04-25T09:17:07.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.154+0000 7f4470c00640 1 -- 172.21.15.62:0/4148111989 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f445c018020 con 0x7f44740b56f0 2024-04-25T09:17:07.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.155+0000 7f4470c00640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f445006b4d0 0x7f445006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.155+0000 7f4470c00640 1 -- 172.21.15.62:0/4148111989 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f445c089f30 con 0x7f44740b56f0 2024-04-25T09:17:07.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.156+0000 7f4478c00640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f445006b4d0 0x7f445006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.157+0000 7f4478c00640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f445006b4d0 0x7f445006d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4464034000 tx=0x7f446402c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:07.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.159+0000 7f4470c00640 1 -- 172.21.15.62:0/4148111989 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f445c01f030 con 0x7f44740b56f0 2024-04-25T09:17:07.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.409+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44740ce580 con 0x7f44740b56f0 2024-04-25T09:17:07.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.410+0000 7f4470c00640 1 -- 172.21.15.62:0/4148111989 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f445c0172c0 con 0x7f44740b56f0 2024-04-25T09:17:07.396 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:07.396 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:17:04.730865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6814":{"gid":6814,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":20,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:17:07.396 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 90 2024-04-25T09:17:07.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.412+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f445006b4d0 msgr2=0x7f445006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.412+0000 7f447a000640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f445006b4d0 0x7f445006d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4464034000 tx=0x7f446402c000 comp rx=0 tx=0).stop 2024-04-25T09:17:07.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 msgr2=0x7f4474150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474150f50 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b3f0 tx=0x7f445c00b8b0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 shutdown_connections 2024-04-25T09:17:07.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f445006b4d0 0x7f445006d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44740b56f0 0x7f4474150f50 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44740b6a90 0x7f44740c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 --2- 172.21.15.62:0/4148111989 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44740b27a0 0x7f44741e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 >> 172.21.15.62:0/4148111989 conn(0x7f4474066d10 msgr2=0x7f44740b86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:07.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 shutdown_connections 2024-04-25T09:17:07.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.413+0000 7f447a000640 1 -- 172.21.15.62:0/4148111989 wait complete. 2024-04-25T09:17:07.419 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-04-25T09:17:07.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:07.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.646+0000 7fde56600640 1 Processor -- start 2024-04-25T09:17:07.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.647+0000 7fde56600640 1 -- start start 2024-04-25T09:17:07.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.648+0000 7fde56600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde5015b9f0 0x7fde500d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.648+0000 7fde56600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fde500b9470 0x7fde500d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.648+0000 7fde56600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5015f2c0 con 0x7fde500d1000 2024-04-25T09:17:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.648+0000 7fde56600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde500d1470 con 0x7fde500b9470 2024-04-25T09:17:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.648+0000 7fde56600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde500b3420 con 0x7fde5015b9f0 2024-04-25T09:17:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.648+0000 7fde4fe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fde500d1000 0x7fde50161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59894/0 (socket says 172.21.15.62:59894) 2024-04-25T09:17:07.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4fe00640 1 -- 172.21.15.62:0/2676342640 learned_addr learned my addr 172.21.15.62:0/2676342640 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4ea00640 1 --2- 172.21.15.62:0/2676342640 >> v2:172.21.15.121:3300/0 conn(0x7fde500b9470 0x7fde500d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4f400640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde5015b9f0 0x7fde500d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835798175 0 0) 0x7fde5015f2c0 con 0x7fde500d1000 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde34003520 con 0x7fde500d1000 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3998522941 0 0) 0x7fde34003520 con 0x7fde500d1000 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde5015b9f0 msgr2=0x7fde500d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde5015b9f0 0x7fde500d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 >> v2:172.21.15.121:3300/0 conn(0x7fde500b9470 msgr2=0x7fde500d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 --2- 172.21.15.62:0/2676342640 >> v2:172.21.15.121:3300/0 conn(0x7fde500b9470 0x7fde500d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5015e6b0 con 0x7fde500d1000 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fde48001670 con 0x7fde500d1000 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.649+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 >> v1:172.21.15.62:6789/0 conn(0x7fde500d1000 legacy=0x7fde50161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4e000640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4e000640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde34004800 0x7fde34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4e000640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde5015b9f0 0x7fde340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500d1470 con 0x7fde340039b0 2024-04-25T09:17:07.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3400b9c0 con 0x7fde34004800 2024-04-25T09:17:07.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3400bb40 con 0x7fde5015b9f0 2024-04-25T09:17:07.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4f400640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4ea00640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde34004800 0x7fde34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4f400640 1 -- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde5015b9f0 msgr2=0x7fde340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:07.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4f400640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde5015b9f0 0x7fde340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4f400640 1 -- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde34004800 msgr2=0x7fde34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:07.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4f400640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde34004800 0x7fde34004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.650+0000 7fde4f400640 1 -- 172.21.15.62:0/2676342640 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34003520 con 0x7fde340039b0 2024-04-25T09:17:07.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.651+0000 7fde4f400640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fde40009fc0 tx=0x7fde34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3cceb51b5e77e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40009c10 con 0x7fde340039b0 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40036070 con 0x7fde340039b0 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde4e000640 1 -- 172.21.15.62:0/2676342640 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4002d070 con 0x7fde340039b0 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde56600640 1 -- 172.21.15.62:0/2676342640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde340039b0 msgr2=0x7fde34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde56600640 1 --2- 172.21.15.62:0/2676342640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde340039b0 0x7fde34003dd0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fde40009fc0 tx=0x7fde34005830 comp rx=0 tx=0).stop 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde4fe00640 1 -- 172.21.15.62:0/2676342640 reap_dead start 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde56600640 1 -- 172.21.15.62:0/2676342640 shutdown_connections 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde56600640 1 -- 172.21.15.62:0/2676342640 >> 172.21.15.62:0/2676342640 conn(0x7fde5006b000 msgr2=0x7fde5006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.652+0000 7fde56600640 1 -- 172.21.15.62:0/2676342640 shutdown_connections 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.653+0000 7fde56600640 1 -- 172.21.15.62:0/2676342640 wait complete. 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.654+0000 7fde56600640 1 Processor -- start 2024-04-25T09:17:07.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.654+0000 7fde56600640 1 -- start start 2024-04-25T09:17:07.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.655+0000 7fde56600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde340039b0 0x7fde50161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.655+0000 7fde56600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde5015f4b0 0x7fde501f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.655+0000 7fde56600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde500cb6a0 0x7fde500b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.655+0000 7fde56600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015ef50 con 0x7fde500cb6a0 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.655+0000 7fde56600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015ec70 con 0x7fde5015f4b0 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.655+0000 7fde56600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015e310 con 0x7fde340039b0 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde500cb6a0 0x7fde500b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde500cb6a0 0x7fde500b7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47794/0 (socket says 172.21.15.62:47794) 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4fe00640 1 -- 172.21.15.62:0/178004993 learned_addr learned my addr 172.21.15.62:0/178004993 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4f400640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde340039b0 0x7fde50161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4ea00640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde5015f4b0 0x7fde501f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4fe00640 1 -- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde340039b0 msgr2=0x7fde50161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4fe00640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde340039b0 0x7fde50161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4fe00640 1 -- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde5015f4b0 msgr2=0x7fde501f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4fe00640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde5015f4b0 0x7fde501f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.656+0000 7fde4fe00640 1 -- 172.21.15.62:0/178004993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde501f3710 con 0x7fde500cb6a0 2024-04-25T09:17:07.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.657+0000 7fde4fe00640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde500cb6a0 0x7fde500b7cd0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fde500b63a0 tx=0x7fde4800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:07.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.657+0000 7fde47400640 1 -- 172.21.15.62:0/178004993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48007440 con 0x7fde500cb6a0 2024-04-25T09:17:07.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.657+0000 7fde47400640 1 -- 172.21.15.62:0/178004993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde480075e0 con 0x7fde500cb6a0 2024-04-25T09:17:07.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.657+0000 7fde47400640 1 -- 172.21.15.62:0/178004993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48004a20 con 0x7fde500cb6a0 2024-04-25T09:17:07.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.657+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5015fae0 con 0x7fde500cb6a0 2024-04-25T09:17:07.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.657+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde50160080 con 0x7fde500cb6a0 2024-04-25T09:17:07.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.658+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde500729c0 con 0x7fde500cb6a0 2024-04-25T09:17:07.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.659+0000 7fde47400640 1 -- 172.21.15.62:0/178004993 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde48004bc0 con 0x7fde500cb6a0 2024-04-25T09:17:07.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.660+0000 7fde47400640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:07.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.660+0000 7fde47400640 1 -- 172.21.15.62:0/178004993 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fde480898d0 con 0x7fde500cb6a0 2024-04-25T09:17:07.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.661+0000 7fde4f400640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:07.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.662+0000 7fde4f400640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fde2c06b4d0 0x7fde2c06d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fde40035000 tx=0x7fde4002f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:07.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.665+0000 7fde47400640 1 -- 172.21.15.62:0/178004993 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde4801a020 con 0x7fde500cb6a0 2024-04-25T09:17:07.735 INFO:tasks.ceph.mds.b.smithi121.stderr:2024-04-25T09:17:07.749+0000 7f5276a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:17:07.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.912+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde5004f000 con 0x7fde500cb6a0 2024-04-25T09:17:07.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.913+0000 7fde47400640 1 -- 172.21.15.62:0/178004993 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fde48058650 con 0x7fde500cb6a0 2024-04-25T09:17:07.900 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:07.900 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:17:04.730865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6814":{"gid":6814,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":20,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:17:07.900 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 90 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.916+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fde2c06b4d0 msgr2=0x7fde2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.916+0000 7fde56600640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fde2c06b4d0 0x7fde2c06d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fde40035000 tx=0x7fde4002f000 comp rx=0 tx=0).stop 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde500cb6a0 msgr2=0x7fde500b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde500cb6a0 0x7fde500b7cd0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fde500b63a0 tx=0x7fde4800c960 comp rx=0 tx=0).stop 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 shutdown_connections 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fde500cb6a0 0x7fde500b7cd0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde5015f4b0 0x7fde501f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 --2- 172.21.15.62:0/178004993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fde340039b0 0x7fde50161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:07.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 >> 172.21.15.62:0/178004993 conn(0x7fde5006b000 msgr2=0x7fde5015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 shutdown_connections 2024-04-25T09:17:07.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:07.917+0000 7fde56600640 1 -- 172.21.15.62:0/178004993 wait complete. 2024-04-25T09:17:07.924 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:17:07.924 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:17:08.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.148+0000 7f44c7200640 1 Processor -- start 2024-04-25T09:17:08.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.148+0000 7f44c7200640 1 -- start start 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 0x7f44c00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f44c015c120 0x7f44c00ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c015c670 con 0x7f44c00b1da0 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c00cd830 con 0x7f44c015c120 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c015c950 con 0x7f44c00b19c0 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f44c015c120 0x7f44c00ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c6800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f44c00b1da0 0x7f44c00cd120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59898/0 (socket says 172.21.15.62:59898) 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c6800640 1 -- 172.21.15.62:0/1233692949 learned_addr learned my addr 172.21.15.62:0/1233692949 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f44c015c120 0x7f44c00ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47796/0 (socket says 172.21.15.62:47796) 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.150+0000 7f44c5e00640 1 --2- 172.21.15.62:0/1233692949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 0x7f44c00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:08.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.151+0000 7f44c5400640 1 -- 172.21.15.62:0/1233692949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 msgr2=0x7f44c00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.151+0000 7f44c5400640 1 --2- 172.21.15.62:0/1233692949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 0x7f44c00ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.151+0000 7f44c5400640 1 -- 172.21.15.62:0/1233692949 >> v1:172.21.15.62:6789/0 conn(0x7f44c00b1da0 legacy=0x7f44c00cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:08.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.151+0000 7f44c5400640 1 -- 172.21.15.62:0/1233692949 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c015cc30 con 0x7f44c015c120 2024-04-25T09:17:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.152+0000 7f44c5400640 1 --2- 172.21.15.62:0/1233692949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c015c120 0x7f44c00ccbe0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f44b40097a0 tx=0x7f44b402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9714ce93556399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.152+0000 7f44c4a00640 1 -- 172.21.15.62:0/1233692949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4037070 con 0x7f44c015c120 2024-04-25T09:17:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.152+0000 7f44c4a00640 1 -- 172.21.15.62:0/1233692949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4032440 con 0x7f44c015c120 2024-04-25T09:17:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.152+0000 7f44c4a00640 1 -- 172.21.15.62:0/1233692949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4032810 con 0x7f44c015c120 2024-04-25T09:17:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.153+0000 7f44c7200640 1 -- 172.21.15.62:0/1233692949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c015c120 msgr2=0x7f44c00ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.153+0000 7f44c7200640 1 --2- 172.21.15.62:0/1233692949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c015c120 0x7f44c00ccbe0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f44b40097a0 tx=0x7f44b402a180 comp rx=0 tx=0).stop 2024-04-25T09:17:08.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.153+0000 7f44c7200640 1 -- 172.21.15.62:0/1233692949 shutdown_connections 2024-04-25T09:17:08.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.153+0000 7f44c7200640 1 --2- 172.21.15.62:0/1233692949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c015c120 0x7f44c00ccbe0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.153+0000 7f44c7200640 1 --2- 172.21.15.62:0/1233692949 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 0x7f44c00ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.153+0000 7f44c7200640 1 -- 172.21.15.62:0/1233692949 >> 172.21.15.62:0/1233692949 conn(0x7f44c00668f0 msgr2=0x7f44c0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:08.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.153+0000 7f44c7200640 1 -- 172.21.15.62:0/1233692949 shutdown_connections 2024-04-25T09:17:08.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.153+0000 7f44c7200640 1 -- 172.21.15.62:0/1233692949 wait complete. 2024-04-25T09:17:08.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.155+0000 7f44c7200640 1 Processor -- start 2024-04-25T09:17:08.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.155+0000 7f44c7200640 1 -- start start 2024-04-25T09:17:08.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 0x7f44c0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:08.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c00b1da0 0x7f44c00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:08.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44c015c120 0x7f44c00ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:08.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00cf570 con 0x7f44c00b1da0 2024-04-25T09:17:08.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00cf3f0 con 0x7f44c015c120 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00cb7d0 con 0x7f44c00b19c0 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c00b1da0 0x7f44c00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c00b1da0 0x7f44c00ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47808/0 (socket says 172.21.15.62:47808) 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.156+0000 7f44c5400640 1 -- 172.21.15.62:0/1948255486 learned_addr learned my addr 172.21.15.62:0/1948255486 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44c5e00640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 0x7f44c0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44c5400640 1 -- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 msgr2=0x7f44c0155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44c5400640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 0x7f44c0155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44c5400640 1 -- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44c015c120 msgr2=0x7f44c00ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44c5400640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44c015c120 0x7f44c00ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44c5400640 1 -- 172.21.15.62:0/1948255486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00b0640 con 0x7f44c00b1da0 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44c5400640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c00b1da0 0x7f44c00ad8e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f44b4035040 tx=0x7f44b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44ba000640 1 -- 172.21.15.62:0/1948255486 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4037040 con 0x7f44c00b1da0 2024-04-25T09:17:08.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44ba000640 1 -- 172.21.15.62:0/1948255486 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4030770 con 0x7f44c00b1da0 2024-04-25T09:17:08.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44ba000640 1 -- 172.21.15.62:0/1948255486 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4030b80 con 0x7f44c00b1da0 2024-04-25T09:17:08.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.157+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c00b0920 con 0x7f44c00b1da0 2024-04-25T09:17:08.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.158+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c00b0ee0 con 0x7f44c00b1da0 2024-04-25T09:17:08.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.159+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c0072b10 con 0x7f44c00b1da0 2024-04-25T09:17:08.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.160+0000 7f44ba000640 1 -- 172.21.15.62:0/1948255486 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44b4044020 con 0x7f44c00b1da0 2024-04-25T09:17:08.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.161+0000 7f44ba000640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f44a006b4d0 0x7f44a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:08.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.161+0000 7f44ba000640 1 -- 172.21.15.62:0/1948255486 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f44b40aa820 con 0x7f44c00b1da0 2024-04-25T09:17:08.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.161+0000 7f44c5e00640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f44a006b4d0 0x7f44a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:08.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.162+0000 7f44c5e00640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f44a006b4d0 0x7f44a006d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006fd0 tx=0x7f44ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:08.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.166+0000 7f44ba000640 1 -- 172.21.15.62:0/1948255486 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44b4032e20 con 0x7f44c00b1da0 2024-04-25T09:17:08.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.443+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f44c00750e0 con 0x7f44c00b1da0 2024-04-25T09:17:08.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.444+0000 7f44ba000640 1 -- 172.21.15.62:0/1948255486 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1048 (secure 0 0 0) 0x7f44b40795a0 con 0x7f44c00b1da0 2024-04-25T09:17:08.430 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:08.430 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":855,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":138,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035785,"num_in_osds":8,"osd_in_since":1714035775,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077019589,"bytes_used":2627444736,"bytes_avail":770466668544,"bytes_total":773094113280,"read_bytes_sec":238505,"write_bytes_sec":65921,"read_op_per_sec":2,"write_op_per_sec":31},"fsmap":{"epoch":90,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"b","status":"up:active","gid":6814}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-25T09:17:04.205099+0000","services":{}},"progress_events":{}} 2024-04-25T09:17:08.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f44a006b4d0 msgr2=0x7f44a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:08.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f44a006b4d0 0x7f44a006d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006fd0 tx=0x7f44ac008040 comp rx=0 tx=0).stop 2024-04-25T09:17:08.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c00b1da0 msgr2=0x7f44c00ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:08.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c00b1da0 0x7f44c00ad8e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f44b4035040 tx=0x7f44b40046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 shutdown_connections 2024-04-25T09:17:08.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f44a006b4d0 0x7f44a006d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f44c00b1da0 0x7f44c00ad8e0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44c015c120 0x7f44c00ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 --2- 172.21.15.62:0/1948255486 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f44c00b19c0 0x7f44c0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:08.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 >> 172.21.15.62:0/1948255486 conn(0x7f44c00668f0 msgr2=0x7f44c0071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:08.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 shutdown_connections 2024-04-25T09:17:08.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:08.447+0000 7f44c7200640 1 -- 172.21.15.62:0/1948255486 wait complete. 2024-04-25T09:17:08.452 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:17:08.452 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:17:08.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:17:08.666 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:17:08.667 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:17:08.667 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:08.728 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:17:08.728 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:08.782 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:08.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.011+0000 7f1e9be00640 1 Processor -- start 2024-04-25T09:17:08.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.011+0000 7f1e9be00640 1 -- start start 2024-04-25T09:17:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.013+0000 7f1e9be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e9c144c10 0x7f1e9c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.013+0000 7f1e9be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c06d900 0x7f1e9c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.013+0000 7f1e9be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c0cf2d0 con 0x7f1e9c0cee40 2024-04-25T09:17:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.013+0000 7f1e9be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c144ff0 con 0x7f1e9c144c10 2024-04-25T09:17:08.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.013+0000 7f1e9be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c160340 con 0x7f1e9c06d900 2024-04-25T09:17:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.014+0000 7f1e9aa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e9c144c10 0x7f1e9c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.014+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c06d900 0x7f1e9c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.014+0000 7f1e9aa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e9c144c10 0x7f1e9c15f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.62:35966/0 (socket says 172.21.15.62:35966) 2024-04-25T09:17:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.014+0000 7f1e9a000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1e9c0cee40 0x7f1e9c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:48600/0 (socket says 172.21.15.62:48600) 2024-04-25T09:17:09.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.014+0000 7f1e9aa00640 1 -- 172.21.15.62:0/2113629339 learned_addr learned my addr 172.21.15.62:0/2113629339 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.015+0000 7f1e9aa00640 1 -- 172.21.15.62:0/2113629339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c06d900 msgr2=0x7f1e9c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.015+0000 7f1e9aa00640 1 --2- 172.21.15.62:0/2113629339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c06d900 0x7f1e9c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.015+0000 7f1e9aa00640 1 -- 172.21.15.62:0/2113629339 >> v1:172.21.15.121:6789/0 conn(0x7f1e9c0cee40 legacy=0x7f1e9c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.015+0000 7f1e9aa00640 1 -- 172.21.15.62:0/2113629339 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c15faa0 con 0x7f1e9c144c10 2024-04-25T09:17:09.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.016+0000 7f1e9aa00640 1 --2- 172.21.15.62:0/2113629339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15f450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009870 tx=0x7f1e8c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=989ddb5864e2192c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.017+0000 7f1e99600640 1 -- 172.21.15.62:0/2113629339 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c037070 con 0x7f1e9c144c10 2024-04-25T09:17:09.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.017+0000 7f1e99600640 1 -- 172.21.15.62:0/2113629339 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c032440 con 0x7f1e9c144c10 2024-04-25T09:17:09.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.017+0000 7f1e99600640 1 -- 172.21.15.62:0/2113629339 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c032810 con 0x7f1e9c144c10 2024-04-25T09:17:09.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.017+0000 7f1e9be00640 1 -- 172.21.15.62:0/2113629339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c144c10 msgr2=0x7f1e9c15f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.017+0000 7f1e9be00640 1 --2- 172.21.15.62:0/2113629339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15f450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009870 tx=0x7f1e8c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.018+0000 7f1e9be00640 1 -- 172.21.15.62:0/2113629339 shutdown_connections 2024-04-25T09:17:09.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.018+0000 7f1e9be00640 1 --2- 172.21.15.62:0/2113629339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c06d900 0x7f1e9c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.018+0000 7f1e9be00640 1 --2- 172.21.15.62:0/2113629339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15f450 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.018+0000 7f1e9be00640 1 -- 172.21.15.62:0/2113629339 >> 172.21.15.62:0/2113629339 conn(0x7f1e9c0b14d0 msgr2=0x7f1e9c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:09.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.018+0000 7f1e9be00640 1 -- 172.21.15.62:0/2113629339 shutdown_connections 2024-04-25T09:17:09.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.018+0000 7f1e9be00640 1 -- 172.21.15.62:0/2113629339 wait complete. 2024-04-25T09:17:09.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.019+0000 7f1e9be00640 1 Processor -- start 2024-04-25T09:17:09.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.020+0000 7f1e9be00640 1 -- start start 2024-04-25T09:17:09.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c06d900 0x7f1e9c158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c0cee40 0x7f1e9c15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c160060 con 0x7f1e9c144c10 2024-04-25T09:17:09.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0b41c0 con 0x7f1e9c06d900 2024-04-25T09:17:09.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0b4b90 con 0x7f1e9c0cee40 2024-04-25T09:17:09.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47816/0 (socket says 172.21.15.62:47816) 2024-04-25T09:17:09.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9b400640 1 -- 172.21.15.62:0/252319890 learned_addr learned my addr 172.21.15.62:0/252319890 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:09.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9aa00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c06d900 0x7f1e9c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9a000640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c0cee40 0x7f1e9c15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.021+0000 7f1e9b400640 1 -- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c0cee40 msgr2=0x7f1e9c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e9b400640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c0cee40 0x7f1e9c15ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e9b400640 1 -- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c06d900 msgr2=0x7f1e9c158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e9b400640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c06d900 0x7f1e9c158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e9b400640 1 -- 172.21.15.62:0/252319890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c158750 con 0x7f1e9c144c10 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e9b400640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15b150 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400efc0 tx=0x7f1e9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e93400640 1 -- 172.21.15.62:0/252319890 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94018020 con 0x7f1e9c144c10 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e93400640 1 -- 172.21.15.62:0/252319890 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e94014480 con 0x7f1e9c144c10 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c158a30 con 0x7f1e9c144c10 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e93400640 1 -- 172.21.15.62:0/252319890 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94014870 con 0x7f1e9c144c10 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.022+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c158ff0 con 0x7f1e9c144c10 2024-04-25T09:17:09.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.023+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c04ef20 con 0x7f1e9c144c10 2024-04-25T09:17:09.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.024+0000 7f1e93400640 1 -- 172.21.15.62:0/252319890 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e94005d10 con 0x7f1e9c144c10 2024-04-25T09:17:09.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.026+0000 7f1e93400640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1e7c06b4d0 0x7f1e7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.026+0000 7f1e93400640 1 -- 172.21.15.62:0/252319890 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1e9408b1f0 con 0x7f1e9c144c10 2024-04-25T09:17:09.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.026+0000 7f1e9aa00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1e7c06b4d0 0x7f1e7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.027+0000 7f1e9aa00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1e7c06b4d0 0x7f1e7c06d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c034000 tx=0x7f1e8c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.031+0000 7f1e93400640 1 -- 172.21.15.62:0/252319890 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e94022030 con 0x7f1e9c144c10 2024-04-25T09:17:09.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.233+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e9c0b19b0 con 0x7f1e9c144c10 2024-04-25T09:17:09.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.235+0000 7f1e93400640 1 -- 172.21.15.62:0/252319890 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f1e94010040 con 0x7f1e9c144c10 2024-04-25T09:17:09.222 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:09.222 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":138,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:02.676192+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:16:00.695740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:16:01.838452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/2303077743":"2024-04-26T09:17:02.676113+0000","172.21.15.62:6832/2303077743":"2024-04-26T09:17:02.676113+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.237+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1e7c06b4d0 msgr2=0x7f1e7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1e7c06b4d0 0x7f1e7c06d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c034000 tx=0x7f1e8c02e000 comp rx=0 tx=0).stop 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e9c144c10 msgr2=0x7f1e9c15b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15b150 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400efc0 tx=0x7f1e9400c490 comp rx=0 tx=0).stop 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 shutdown_connections 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1e7c06b4d0 0x7f1e7c06d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e9c144c10 0x7f1e9c15b150 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1e9c0cee40 0x7f1e9c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 --2- 172.21.15.62:0/252319890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e9c06d900 0x7f1e9c158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 >> 172.21.15.62:0/252319890 conn(0x7f1e9c0b14d0 msgr2=0x7f1e9c0cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:09.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 shutdown_connections 2024-04-25T09:17:09.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.238+0000 7f1e9be00640 1 -- 172.21.15.62:0/252319890 wait complete. 2024-04-25T09:17:09.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:16:00.695740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:16:01.838452+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:17:09.245 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:17:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.467+0000 7fd525600640 1 Processor -- start 2024-04-25T09:17:09.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.467+0000 7fd525600640 1 -- start start 2024-04-25T09:17:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd525600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd52015bdf0 0x7fd5200720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd525600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd5200cebc0 0x7fd52006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd525600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52006d070 con 0x7fd52006c7e0 2024-04-25T09:17:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd525600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52015c1d0 con 0x7fd52015bdf0 2024-04-25T09:17:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd525600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5200ae8c0 con 0x7fd5200cebc0 2024-04-25T09:17:09.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd51ea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd52015bdf0 0x7fd5200720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd51ea00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd52015bdf0 0x7fd5200720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47828/0 (socket says 172.21.15.62:47828) 2024-04-25T09:17:09.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd51ea00640 1 -- 172.21.15.62:0/1857598406 learned_addr learned my addr 172.21.15.62:0/1857598406 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.469+0000 7fd51e000640 1 --2- 172.21.15.62:0/1857598406 >> v2:172.21.15.121:3300/0 conn(0x7fd5200cebc0 0x7fd52006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.470+0000 7fd51ea00640 1 -- 172.21.15.62:0/1857598406 >> v2:172.21.15.121:3300/0 conn(0x7fd5200cebc0 msgr2=0x7fd52006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.470+0000 7fd51ea00640 1 --2- 172.21.15.62:0/1857598406 >> v2:172.21.15.121:3300/0 conn(0x7fd5200cebc0 0x7fd52006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.470+0000 7fd51ea00640 1 -- 172.21.15.62:0/1857598406 >> v1:172.21.15.62:6789/0 conn(0x7fd52006c7e0 legacy=0x7fd5200cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.470+0000 7fd51ea00640 1 -- 172.21.15.62:0/1857598406 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52006d350 con 0x7fd52015bdf0 2024-04-25T09:17:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.470+0000 7fd51e000640 1 --2- 172.21.15.62:0/1857598406 >> v2:172.21.15.121:3300/0 conn(0x7fd5200cebc0 0x7fd52006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.470+0000 7fd51d600640 1 -- 172.21.15.62:0/1857598406 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342783984 0 0) 0x7fd52006d070 con 0x7fd52006c7e0 2024-04-25T09:17:09.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.471+0000 7fd51ea00640 1 --2- 172.21.15.62:0/1857598406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52015bdf0 0x7fd5200720c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd520068870 tx=0x7fd51002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8522b40fbe2e4f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.471+0000 7fd51d600640 1 -- 172.21.15.62:0/1857598406 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510037070 con 0x7fd52015bdf0 2024-04-25T09:17:09.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.471+0000 7fd51d600640 1 -- 172.21.15.62:0/1857598406 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510032440 con 0x7fd52015bdf0 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.471+0000 7fd51d600640 1 -- 172.21.15.62:0/1857598406 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510032810 con 0x7fd52015bdf0 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.472+0000 7fd525600640 1 -- 172.21.15.62:0/1857598406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52015bdf0 msgr2=0x7fd5200720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.472+0000 7fd525600640 1 --2- 172.21.15.62:0/1857598406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52015bdf0 0x7fd5200720c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd520068870 tx=0x7fd51002a270 comp rx=0 tx=0).stop 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.472+0000 7fd525600640 1 -- 172.21.15.62:0/1857598406 shutdown_connections 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.472+0000 7fd525600640 1 --2- 172.21.15.62:0/1857598406 >> v2:172.21.15.121:3300/0 conn(0x7fd5200cebc0 0x7fd52006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.472+0000 7fd525600640 1 --2- 172.21.15.62:0/1857598406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52015bdf0 0x7fd5200720c0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.472+0000 7fd525600640 1 -- 172.21.15.62:0/1857598406 >> 172.21.15.62:0/1857598406 conn(0x7fd5200b1410 msgr2=0x7fd5200b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.472+0000 7fd525600640 1 -- 172.21.15.62:0/1857598406 shutdown_connections 2024-04-25T09:17:09.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.472+0000 7fd525600640 1 -- 172.21.15.62:0/1857598406 wait complete. 2024-04-25T09:17:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.474+0000 7fd525600640 1 Processor -- start 2024-04-25T09:17:09.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.474+0000 7fd525600640 1 -- start start 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd525600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52006c7e0 0x7fd520153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd525600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5200cebc0 0x7fd520152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd525600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd52015bdf0 0x7fd52015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd525600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52006cc30 con 0x7fd52006c7e0 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd525600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200aedf0 con 0x7fd5200cebc0 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd525600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52006cdb0 con 0x7fd52015bdf0 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52006c7e0 0x7fd520153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd51ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52006c7e0 0x7fd520153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47844/0 (socket says 172.21.15.62:47844) 2024-04-25T09:17:09.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.475+0000 7fd51ea00640 1 -- 172.21.15.62:0/1638426091 learned_addr learned my addr 172.21.15.62:0/1638426091 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd51e000640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5200cebc0 0x7fd520152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd51f400640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd52015bdf0 0x7fd52015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd51ea00640 1 -- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd52015bdf0 msgr2=0x7fd52015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd51ea00640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd52015bdf0 0x7fd52015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd51ea00640 1 -- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5200cebc0 msgr2=0x7fd520152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd51ea00640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5200cebc0 0x7fd520152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd51ea00640 1 -- 172.21.15.62:0/1638426091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200ad330 con 0x7fd52006c7e0 2024-04-25T09:17:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd51ea00640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52006c7e0 0x7fd520153360 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd51002a2b0 tx=0x7fd5100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.476+0000 7fd516a00640 1 -- 172.21.15.62:0/1638426091 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510037040 con 0x7fd52006c7e0 2024-04-25T09:17:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.477+0000 7fd516a00640 1 -- 172.21.15.62:0/1638426091 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510042070 con 0x7fd52006c7e0 2024-04-25T09:17:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.477+0000 7fd516a00640 1 -- 172.21.15.62:0/1638426091 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510030bb0 con 0x7fd52006c7e0 2024-04-25T09:17:09.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.477+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd520155e10 con 0x7fd52006c7e0 2024-04-25T09:17:09.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.477+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5200ad8b0 con 0x7fd52006c7e0 2024-04-25T09:17:09.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.478+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52004ef20 con 0x7fd52006c7e0 2024-04-25T09:17:09.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.479+0000 7fd516a00640 1 -- 172.21.15.62:0/1638426091 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd510047460 con 0x7fd52006c7e0 2024-04-25T09:17:09.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.480+0000 7fd516a00640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd50006b4d0 0x7fd50006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.480+0000 7fd516a00640 1 -- 172.21.15.62:0/1638426091 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd5100aaf40 con 0x7fd52006c7e0 2024-04-25T09:17:09.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.480+0000 7fd51e000640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd50006b4d0 0x7fd50006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.482+0000 7fd51e000640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd50006b4d0 0x7fd50006d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0075b0 tx=0x7fd50c007540 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.484+0000 7fd516a00640 1 -- 172.21.15.62:0/1638426091 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd510045360 con 0x7fd52006c7e0 2024-04-25T09:17:09.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.685+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd520071c90 con 0x7fd52006c7e0 2024-04-25T09:17:09.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.686+0000 7fd516a00640 1 -- 172.21.15.62:0/1638426091 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd510079cc0 con 0x7fd52006c7e0 2024-04-25T09:17:09.672 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:17:09.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd50006b4d0 msgr2=0x7fd50006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd50006b4d0 0x7fd50006d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0075b0 tx=0x7fd50c007540 comp rx=0 tx=0).stop 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52006c7e0 msgr2=0x7fd520153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52006c7e0 0x7fd520153360 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fd51002a2b0 tx=0x7fd5100046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 shutdown_connections 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd50006b4d0 0x7fd50006d990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd52015bdf0 0x7fd52015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5200cebc0 0x7fd520152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 --2- 172.21.15.62:0/1638426091 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd52006c7e0 0x7fd520153360 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 >> 172.21.15.62:0/1638426091 conn(0x7fd5200b1410 msgr2=0x7fd5200ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:09.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 shutdown_connections 2024-04-25T09:17:09.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.689+0000 7fd525600640 1 -- 172.21.15.62:0/1638426091 wait complete. 2024-04-25T09:17:09.694 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:17:09.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.916+0000 7f20d2e00640 1 Processor -- start 2024-04-25T09:17:09.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.917+0000 7f20d2e00640 1 -- start start 2024-04-25T09:17:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.918+0000 7f20d2e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f20cc06f6b0 0x7f20cc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.918+0000 7f20d2e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20cc06bbb0 con 0x7f20cc161380 2024-04-25T09:17:09.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.918+0000 7f20d2e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20cc06be90 con 0x7f20cc06e7a0 2024-04-25T09:17:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.918+0000 7f20d2e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20cc06cc50 con 0x7f20cc06f6b0 2024-04-25T09:17:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.918+0000 7f20cbe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f20cc161380 0x7f20cc06efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59918/0 (socket says 172.21.15.62:59918) 2024-04-25T09:17:09.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.918+0000 7f20cbe00640 1 -- 172.21.15.62:0/169375243 learned_addr learned my addr 172.21.15.62:0/169375243 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:09.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846637060 0 0) 0x7f20cc06bbb0 con 0x7f20cc161380 2024-04-25T09:17:09.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20c3400640 1 --2- 172.21.15.62:0/169375243 >> v2:172.21.15.121:3300/0 conn(0x7f20cc06f6b0 0x7f20cc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20b0003520 con 0x7f20cc161380 2024-04-25T09:17:09.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4100571702 0 0) 0x7f20b0003520 con 0x7f20cc161380 2024-04-25T09:17:09.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 >> v2:172.21.15.121:3300/0 conn(0x7f20cc06f6b0 msgr2=0x7f20cc06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:09.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 --2- 172.21.15.62:0/169375243 >> v2:172.21.15.121:3300/0 conn(0x7f20cc06f6b0 0x7f20cc06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 >> v1:172.21.15.121:6789/0 conn(0x7f20cc06e7a0 legacy=0x7f20cc0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc06c170 con 0x7f20cc161380 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f20bc001670 con 0x7f20cc161380 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.919+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 >> v1:172.21.15.62:6789/0 conn(0x7f20cc161380 legacy=0x7f20cc06efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20cb400640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20b0003a00 0x7f20b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20cb400640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20cb400640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20b00051c0 0x7f20b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc06cc50 con 0x7f20b0004800 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b000c0a0 con 0x7f20b00051c0 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b000c2a0 con 0x7f20b0003a00 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20c3400640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20cbe00640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20b0003a00 0x7f20b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20c3400640 1 -- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20b0003a00 msgr2=0x7f20b0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:09.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20d0e00640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20b00051c0 0x7f20b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20c3400640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20b0003a00 0x7f20b0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20c3400640 1 -- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20b00051c0 msgr2=0x7f20b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:09.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20c3400640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20b00051c0 0x7f20b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20c3400640 1 -- 172.21.15.62:0/169375243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0003520 con 0x7f20b0004800 2024-04-25T09:17:09.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.920+0000 7f20d0e00640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20b00051c0 0x7f20b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:09.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.921+0000 7f20d0e00640 1 -- 172.21.15.62:0/169375243 reap_dead start 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20c3400640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20b0004c80 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009ac0 tx=0x7f20b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f59db776b79250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4036070 con 0x7f20b0004800 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b4031400 con 0x7f20b0004800 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20cb400640 1 -- 172.21.15.62:0/169375243 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4009690 con 0x7f20b0004800 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20d2e00640 1 -- 172.21.15.62:0/169375243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 msgr2=0x7f20b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20d2e00640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20b0004c80 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009ac0 tx=0x7f20b0005830 comp rx=0 tx=0).stop 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20d2e00640 1 -- 172.21.15.62:0/169375243 shutdown_connections 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20d2e00640 1 --2- 172.21.15.62:0/169375243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20b0004c80 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.922+0000 7f20d2e00640 1 -- 172.21.15.62:0/169375243 >> 172.21.15.62:0/169375243 conn(0x7f20cc0b3440 msgr2=0x7f20cc0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.923+0000 7f20d2e00640 1 -- 172.21.15.62:0/169375243 shutdown_connections 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.923+0000 7f20d2e00640 1 -- 172.21.15.62:0/169375243 wait complete. 2024-04-25T09:17:09.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.924+0000 7f20d2e00640 1 Processor -- start 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.925+0000 7f20d2e00640 1 -- start start 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.925+0000 7f20d2e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20cc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.925+0000 7f20d2e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20cc1f40e0 0x7f20cc0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.925+0000 7f20d2e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20cc1f4560 0x7f20cc0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20d2e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc0d0e20 con 0x7f20b0004800 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20d2e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc06c730 con 0x7f20cc1f4560 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20d2e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20cc161760 con 0x7f20cc1f40e0 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20cc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20cc0c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47854/0 (socket says 172.21.15.62:47854) 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 -- 172.21.15.62:0/1645740457 learned_addr learned my addr 172.21.15.62:0/1645740457 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 -- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20cc1f40e0 msgr2=0x7f20cc0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20cc1f40e0 0x7f20cc0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 -- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20cc1f4560 msgr2=0x7f20cc0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20cc1f4560 0x7f20cc0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:09.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 -- 172.21.15.62:0/1645740457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc0b67f0 con 0x7f20b0004800 2024-04-25T09:17:09.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.926+0000 7f20cbe00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20cc0c7db0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e730 tx=0x7f20bc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.927+0000 7f20c8c00640 1 -- 172.21.15.62:0/1645740457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00ee40 con 0x7f20b0004800 2024-04-25T09:17:09.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.927+0000 7f20c8c00640 1 -- 172.21.15.62:0/1645740457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc018070 con 0x7f20b0004800 2024-04-25T09:17:09.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.927+0000 7f20c8c00640 1 -- 172.21.15.62:0/1645740457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00caf0 con 0x7f20b0004800 2024-04-25T09:17:09.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.927+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20cc0b6ad0 con 0x7f20b0004800 2024-04-25T09:17:09.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.927+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20cc0b7090 con 0x7f20b0004800 2024-04-25T09:17:09.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.928+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20cc04ef20 con 0x7f20b0004800 2024-04-25T09:17:09.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.929+0000 7f20c8c00640 1 -- 172.21.15.62:0/1645740457 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20bc00cc90 con 0x7f20b0004800 2024-04-25T09:17:09.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.930+0000 7f20c8c00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:09.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.930+0000 7f20c8c00640 1 -- 172.21.15.62:0/1645740457 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f20bc08a190 con 0x7f20b0004800 2024-04-25T09:17:09.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.931+0000 7f20c3400640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:09.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.932+0000 7f20c3400640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20a806b4d0 0x7f20a806d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f20b4033000 tx=0x7f20b402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:09.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:09.934+0000 7f20c8c00640 1 -- 172.21.15.62:0/1645740457 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20bc058f10 con 0x7f20b0004800 2024-04-25T09:17:10.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.137+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20cc15fb20 con 0x7f20b0004800 2024-04-25T09:17:10.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.138+0000 7f20c8c00640 1 -- 172.21.15.62:0/1645740457 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20bc058810 con 0x7f20b0004800 2024-04-25T09:17:10.124 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:17:10.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20a806b4d0 msgr2=0x7f20a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20a806b4d0 0x7f20a806d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f20b4033000 tx=0x7f20b402e000 comp rx=0 tx=0).stop 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 msgr2=0x7f20cc0c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20cc0c7db0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e730 tx=0x7f20bc00ebf0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 shutdown_connections 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20cc1f4560 0x7f20cc0b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20cc1f40e0 0x7f20cc0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 --2- 172.21.15.62:0/1645740457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0004800 0x7f20cc0c7db0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 >> 172.21.15.62:0/1645740457 conn(0x7f20cc0b3440 msgr2=0x7f20cc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 shutdown_connections 2024-04-25T09:17:10.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.141+0000 7f20d2e00640 1 -- 172.21.15.62:0/1645740457 wait complete. 2024-04-25T09:17:10.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:17:10.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.375+0000 7fa096400640 1 Processor -- start 2024-04-25T09:17:10.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.376+0000 7fa096400640 1 -- start start 2024-04-25T09:17:10.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.377+0000 7fa096400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa09015dde0 0x7fa09006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.363 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.377+0000 7fa096400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa09006e600 0x7fa0900716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.377+0000 7fa096400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa090072430 con 0x7fa090071bf0 2024-04-25T09:17:10.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.377+0000 7fa096400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09006ea60 con 0x7fa09015dde0 2024-04-25T09:17:10.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.377+0000 7fa096400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0900ad8e0 con 0x7fa09006e600 2024-04-25T09:17:10.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.377+0000 7fa08f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa09015dde0 0x7fa09006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.378+0000 7fa08f400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa09015dde0 0x7fa09006e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47856/0 (socket says 172.21.15.62:47856) 2024-04-25T09:17:10.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.378+0000 7fa08f400640 1 -- 172.21.15.62:0/3201959458 learned_addr learned my addr 172.21.15.62:0/3201959458 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:10.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.378+0000 7fa08ea00640 1 --2- 172.21.15.62:0/3201959458 >> v2:172.21.15.121:3300/0 conn(0x7fa09006e600 0x7fa0900716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.378+0000 7fa08f400640 1 -- 172.21.15.62:0/3201959458 >> v2:172.21.15.121:3300/0 conn(0x7fa09006e600 msgr2=0x7fa0900716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:10.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.378+0000 7fa08f400640 1 --2- 172.21.15.62:0/3201959458 >> v2:172.21.15.121:3300/0 conn(0x7fa09006e600 0x7fa0900716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.378+0000 7fa08f400640 1 -- 172.21.15.62:0/3201959458 >> v1:172.21.15.121:6789/0 conn(0x7fa090071bf0 legacy=0x7fa0900b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:10.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.378+0000 7fa08f400640 1 -- 172.21.15.62:0/3201959458 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900adbc0 con 0x7fa09015dde0 2024-04-25T09:17:10.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.378+0000 7fa08ea00640 1 --2- 172.21.15.62:0/3201959458 >> v2:172.21.15.121:3300/0 conn(0x7fa09006e600 0x7fa0900716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:10.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.379+0000 7fa08f400640 1 --2- 172.21.15.62:0/3201959458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09015dde0 0x7fa09006e0c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa0900b2960 tx=0x7fa08002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32e6a438bce06ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.379+0000 7fa08e000640 1 -- 172.21.15.62:0/3201959458 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080037070 con 0x7fa09015dde0 2024-04-25T09:17:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.379+0000 7fa08e000640 1 -- 172.21.15.62:0/3201959458 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa080032440 con 0x7fa09015dde0 2024-04-25T09:17:10.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.379+0000 7fa08e000640 1 -- 172.21.15.62:0/3201959458 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080032810 con 0x7fa09015dde0 2024-04-25T09:17:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.380+0000 7fa096400640 1 -- 172.21.15.62:0/3201959458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09015dde0 msgr2=0x7fa09006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.380+0000 7fa096400640 1 --2- 172.21.15.62:0/3201959458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09015dde0 0x7fa09006e0c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa0900b2960 tx=0x7fa08002a230 comp rx=0 tx=0).stop 2024-04-25T09:17:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.380+0000 7fa096400640 1 -- 172.21.15.62:0/3201959458 shutdown_connections 2024-04-25T09:17:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.380+0000 7fa096400640 1 --2- 172.21.15.62:0/3201959458 >> v2:172.21.15.121:3300/0 conn(0x7fa09006e600 0x7fa0900716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.380+0000 7fa096400640 1 --2- 172.21.15.62:0/3201959458 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09015dde0 0x7fa09006e0c0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.380+0000 7fa096400640 1 -- 172.21.15.62:0/3201959458 >> 172.21.15.62:0/3201959458 conn(0x7fa09006b630 msgr2=0x7fa0900b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.380+0000 7fa096400640 1 -- 172.21.15.62:0/3201959458 shutdown_connections 2024-04-25T09:17:10.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.380+0000 7fa096400640 1 -- 172.21.15.62:0/3201959458 wait complete. 2024-04-25T09:17:10.368 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.382+0000 7fa096400640 1 Processor -- start 2024-04-25T09:17:10.369 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.382+0000 7fa096400640 1 -- start start 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.383+0000 7fa096400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09006e600 0x7fa0900b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.383+0000 7fa096400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa090071bf0 0x7fa0900afe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.383+0000 7fa096400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09015dde0 0x7fa0901478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.383+0000 7fa096400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09015e1c0 con 0x7fa09006e600 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.383+0000 7fa096400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0900ae180 con 0x7fa090071bf0 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.383+0000 7fa096400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa090072020 con 0x7fa09015dde0 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09006e600 0x7fa0900b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09006e600 0x7fa0900b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47862/0 (socket says 172.21.15.62:47862) 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 -- 172.21.15.62:0/3496719083 learned_addr learned my addr 172.21.15.62:0/3496719083 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 -- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09015dde0 msgr2=0x7fa0901478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09015dde0 0x7fa0901478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 -- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa090071bf0 msgr2=0x7fa0900afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa090071bf0 0x7fa0900afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 -- 172.21.15.62:0/3496719083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900b05f0 con 0x7fa09006e600 2024-04-25T09:17:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.384+0000 7fa08f400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09006e600 0x7fa0900b1690 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fa080000c00 tx=0x7fa0800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.385+0000 7fa087400640 1 -- 172.21.15.62:0/3496719083 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080041020 con 0x7fa09006e600 2024-04-25T09:17:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.385+0000 7fa087400640 1 -- 172.21.15.62:0/3496719083 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa080030990 con 0x7fa09006e600 2024-04-25T09:17:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.385+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0900b08d0 con 0x7fa09006e600 2024-04-25T09:17:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.385+0000 7fa087400640 1 -- 172.21.15.62:0/3496719083 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080037040 con 0x7fa09006e600 2024-04-25T09:17:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.385+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0900b0e90 con 0x7fa09006e600 2024-04-25T09:17:10.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.386+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09004ef20 con 0x7fa09006e600 2024-04-25T09:17:10.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.387+0000 7fa087400640 1 -- 172.21.15.62:0/3496719083 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa080044020 con 0x7fa09006e600 2024-04-25T09:17:10.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.388+0000 7fa087400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa06c06b4d0 0x7fa06c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.388+0000 7fa087400640 1 -- 172.21.15.62:0/3496719083 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa0800aad80 con 0x7fa09006e600 2024-04-25T09:17:10.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.389+0000 7fa08ea00640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa06c06b4d0 0x7fa06c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.390+0000 7fa08ea00640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa06c06b4d0 0x7fa06c06d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa09015d560 tx=0x7fa0780083d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:10.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.392+0000 7fa087400640 1 -- 172.21.15.62:0/3496719083 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa08004b030 con 0x7fa09006e600 2024-04-25T09:17:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.593+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa09006f160 con 0x7fa09006e600 2024-04-25T09:17:10.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.594+0000 7fa087400640 1 -- 172.21.15.62:0/3496719083 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa080079b00 con 0x7fa09006e600 2024-04-25T09:17:10.580 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:17:10.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa06c06b4d0 msgr2=0x7fa06c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa06c06b4d0 0x7fa06c06d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa09015d560 tx=0x7fa0780083d0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09006e600 msgr2=0x7fa0900b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09006e600 0x7fa0900b1690 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fa080000c00 tx=0x7fa0800046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 shutdown_connections 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa06c06b4d0 0x7fa06c06d990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa09015dde0 0x7fa0901478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa090071bf0 0x7fa0900afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 --2- 172.21.15.62:0/3496719083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa09006e600 0x7fa0900b1690 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 >> 172.21.15.62:0/3496719083 conn(0x7fa09006b630 msgr2=0x7fa09015c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 shutdown_connections 2024-04-25T09:17:10.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.596+0000 7fa096400640 1 -- 172.21.15.62:0/3496719083 wait complete. 2024-04-25T09:17:10.601 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:17:10.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.832+0000 7f8bace00640 1 Processor -- start 2024-04-25T09:17:10.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.833+0000 7f8bace00640 1 -- start start 2024-04-25T09:17:10.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.834+0000 7f8bace00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 0x7f8ba80b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.834+0000 7f8bace00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8ba80cfa70 0x7f8ba80b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.834+0000 7f8bace00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba80cff60 con 0x7f8ba815c4e0 2024-04-25T09:17:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.834+0000 7f8bace00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba80b3720 con 0x7f8ba80cfa70 2024-04-25T09:17:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.834+0000 7f8bace00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8070360 con 0x7f8ba80cf5f0 2024-04-25T09:17:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.835+0000 7f8ba7400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8ba815c4e0 0x7f8ba815c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59922/0 (socket says 172.21.15.62:59922) 2024-04-25T09:17:10.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.835+0000 7f8ba7400640 1 -- 172.21.15.62:0/3047935908 learned_addr learned my addr 172.21.15.62:0/3047935908 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.835+0000 7f8b9fe00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 0x7f8ba80b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.835+0000 7f8ba7e00640 1 --2- 172.21.15.62:0/3047935908 >> v2:172.21.15.121:3300/0 conn(0x7f8ba80cfa70 0x7f8ba80b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.835+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293528355 0 0) 0x7f8ba80cff60 con 0x7f8ba815c4e0 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.835+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b8c003520 con 0x7f8ba815c4e0 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 979615570 0 0) 0x7f8b8c003520 con 0x7f8ba815c4e0 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 msgr2=0x7f8ba80b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 0x7f8ba80b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 >> v2:172.21.15.121:3300/0 conn(0x7f8ba80cfa70 msgr2=0x7f8ba80b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 --2- 172.21.15.62:0/3047935908 >> v2:172.21.15.121:3300/0 conn(0x7f8ba80cfa70 0x7f8ba80b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8070640 con 0x7f8ba815c4e0 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f8b98001670 con 0x7f8ba815c4e0 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 >> v1:172.21.15.62:6789/0 conn(0x7f8ba815c4e0 legacy=0x7f8ba815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c0039b0 0x7f8b8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 0x7f8b8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b8c004d40 0x7f8b8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba80b3720 con 0x7f8b8c004d40 2024-04-25T09:17:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8070360 con 0x7f8b8c0039b0 2024-04-25T09:17:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c00b9f0 con 0x7f8ba80cf5f0 2024-04-25T09:17:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.836+0000 7f8ba7e00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b8c004d40 0x7f8b8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.837+0000 7f8ba7400640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c0039b0 0x7f8b8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.837+0000 7f8b9fe00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 0x7f8b8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.837+0000 7f8ba7e00640 1 -- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 msgr2=0x7f8b8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:10.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.837+0000 7f8ba7e00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 0x7f8b8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.837+0000 7f8ba7e00640 1 -- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c0039b0 msgr2=0x7f8b8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:10.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.837+0000 7f8ba7e00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c0039b0 0x7f8b8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.837+0000 7f8ba7e00640 1 -- 172.21.15.62:0/3047935908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c003520 con 0x7f8b8c004d40 2024-04-25T09:17:10.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.837+0000 7f8b9fe00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba80cf5f0 0x7f8b8c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:10.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.838+0000 7f8ba7e00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b8c004d40 0x7f8b8c0051c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009f90 tx=0x7f8b8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e31a17d9ffd8fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:10.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.838+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0009c10 con 0x7f8b8c004d40 2024-04-25T09:17:10.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.838+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0036070 con 0x7f8b8c004d40 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.838+0000 7f8ba6a00640 1 -- 172.21.15.62:0/3047935908 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0031840 con 0x7f8b8c004d40 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.839+0000 7f8bace00640 1 -- 172.21.15.62:0/3047935908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b8c004d40 msgr2=0x7f8b8c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.839+0000 7f8bace00640 1 --2- 172.21.15.62:0/3047935908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8b8c004d40 0x7f8b8c0051c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009f90 tx=0x7f8b8c005830 comp rx=0 tx=0).stop 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.839+0000 7f8ba7e00640 1 -- 172.21.15.62:0/3047935908 reap_dead start 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.839+0000 7f8bace00640 1 -- 172.21.15.62:0/3047935908 shutdown_connections 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.839+0000 7f8bace00640 1 -- 172.21.15.62:0/3047935908 >> 172.21.15.62:0/3047935908 conn(0x7f8ba80668f0 msgr2=0x7f8ba8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.839+0000 7f8bace00640 1 -- 172.21.15.62:0/3047935908 shutdown_connections 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.839+0000 7f8bace00640 1 -- 172.21.15.62:0/3047935908 wait complete. 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.841+0000 7f8bace00640 1 Processor -- start 2024-04-25T09:17:10.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.842+0000 7f8bace00640 1 -- start start 2024-04-25T09:17:10.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.843+0000 7f8bace00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c004d40 0x7f8ba81471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.843+0000 7f8bace00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba8146a80 0x7f8ba815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.843+0000 7f8bace00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba815aec0 0x7f8ba80ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.843+0000 7f8bace00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8071120 con 0x7f8ba815aec0 2024-04-25T09:17:10.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.843+0000 7f8bace00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8070c00 con 0x7f8b8c004d40 2024-04-25T09:17:10.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.843+0000 7f8bace00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba80b42d0 con 0x7f8ba8146a80 2024-04-25T09:17:10.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba815aec0 0x7f8ba80ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba815aec0 0x7f8ba80ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47870/0 (socket says 172.21.15.62:47870) 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7e00640 1 -- 172.21.15.62:0/3804760919 learned_addr learned my addr 172.21.15.62:0/3804760919 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7400640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c004d40 0x7f8ba81471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8b9fe00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba8146a80 0x7f8ba815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7e00640 1 -- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba8146a80 msgr2=0x7f8ba815a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7e00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba8146a80 0x7f8ba815a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7e00640 1 -- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c004d40 msgr2=0x7f8ba81471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7e00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c004d40 0x7f8ba81471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:10.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.844+0000 7f8ba7e00640 1 -- 172.21.15.62:0/3804760919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba81f3270 con 0x7f8ba815aec0 2024-04-25T09:17:10.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.845+0000 7f8ba7e00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba815aec0 0x7f8ba80ad8e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0033040 tx=0x7f8ba0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.845+0000 7f8b9f400640 1 -- 172.21.15.62:0/3804760919 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0009c10 con 0x7f8ba815aec0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.845+0000 7f8b9f400640 1 -- 172.21.15.62:0/3804760919 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0031db0 con 0x7f8ba815aec0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.845+0000 7f8b9f400640 1 -- 172.21.15.62:0/3804760919 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0045670 con 0x7f8ba815aec0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.845+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba81f6ba0 con 0x7f8ba815aec0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.845+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba81f7160 con 0x7f8ba815aec0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.846+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba804ef20 con 0x7f8ba815aec0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.847+0000 7f8b9f400640 1 -- 172.21.15.62:0/3804760919 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ba0005d10 con 0x7f8ba815aec0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.848+0000 7f8b9f400640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b8406b4d0 0x7f8b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.849+0000 7f8b9f400640 1 -- 172.21.15.62:0/3804760919 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8ba00ab6b0 con 0x7f8ba815aec0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.849+0000 7f8ba7400640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b8406b4d0 0x7f8b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:10.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.850+0000 7f8ba7400640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b8406b4d0 0x7f8b8406d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8b98000c00 tx=0x7f8b98009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:10.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:10.853+0000 7f8b9f400640 1 -- 172.21.15.62:0/3804760919 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ba007a4c0 con 0x7f8ba815aec0 2024-04-25T09:17:11.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.071+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8ba8071b00 con 0x7f8ba815aec0 2024-04-25T09:17:11.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.072+0000 7f8b9f400640 1 -- 172.21.15.62:0/3804760919 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v138) v1 ==== 80+0+122 (secure 0 0 0) 0x7f8ba0079dc0 con 0x7f8ba815aec0 2024-04-25T09:17:11.058 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6834/2303077743 2024-04-26T09:17:02.676113+0000 2024-04-25T09:17:11.058 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62:6832/2303077743 2024-04-26T09:17:02.676113+0000 2024-04-25T09:17:11.058 INFO:teuthology.orchestra.run.smithi062.stderr:listed 2 entries 2024-04-25T09:17:11.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.074+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b8406b4d0 msgr2=0x7f8b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:11.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.074+0000 7f8bace00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b8406b4d0 0x7f8b8406d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8b98000c00 tx=0x7f8b98009040 comp rx=0 tx=0).stop 2024-04-25T09:17:11.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.074+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba815aec0 msgr2=0x7f8ba80ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:11.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.074+0000 7f8bace00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba815aec0 0x7f8ba80ad8e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0033040 tx=0x7f8ba0009750 comp rx=0 tx=0).stop 2024-04-25T09:17:11.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.075+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 shutdown_connections 2024-04-25T09:17:11.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.075+0000 7f8bace00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8b8406b4d0 0x7f8b8406d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.075+0000 7f8bace00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ba815aec0 0x7f8ba80ad8e0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.075+0000 7f8bace00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8ba8146a80 0x7f8ba815a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.075+0000 7f8bace00640 1 --2- 172.21.15.62:0/3804760919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b8c004d40 0x7f8ba81471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.075+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 >> 172.21.15.62:0/3804760919 conn(0x7f8ba80668f0 msgr2=0x7f8ba8066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:11.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.075+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 shutdown_connections 2024-04-25T09:17:11.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.075+0000 7f8bace00640 1 -- 172.21.15.62:0/3804760919 wait complete. 2024-04-25T09:17:11.079 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:17:11.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:11.079 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:17:11.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:17:11.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:11.177 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:17:11.233 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:17:11.233 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:11.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:17:11.290 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:17:11.290 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:11.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:17:11.351 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:17:11.352 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:11.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:17:11.410 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.638+0000 7f9f8e000640 1 Processor -- start 2024-04-25T09:17:11.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.639+0000 7f9f8e000640 1 -- start start 2024-04-25T09:17:11.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.640+0000 7f9f8e000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f880b6a90 0x7f9f880d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.640+0000 7f9f8e000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f880d3110 con 0x7f9f880b27a0 2024-04-25T09:17:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.640+0000 7f9f8e000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f880d33f0 con 0x7f9f880b56f0 2024-04-25T09:17:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.640+0000 7f9f8e000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f880cedc0 con 0x7f9f880b6a90 2024-04-25T09:17:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.640+0000 7f9f87e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f880b6a90 0x7f9f880d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.641+0000 7f9f87e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9f880b6a90 0x7f9f880d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47882/0 (socket says 172.21.15.62:47882) 2024-04-25T09:17:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.641+0000 7f9f87e00640 1 -- 172.21.15.62:0/36129784 learned_addr learned my addr 172.21.15.62:0/36129784 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.641+0000 7f9f87e00640 1 -- 172.21.15.62:0/36129784 >> v1:172.21.15.121:6789/0 conn(0x7f9f880b56f0 legacy=0x7f9f88160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:11.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.641+0000 7f9f87e00640 1 -- 172.21.15.62:0/36129784 >> v1:172.21.15.62:6789/0 conn(0x7f9f880b27a0 legacy=0x7f9f880d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.641+0000 7f9f87e00640 1 -- 172.21.15.62:0/36129784 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880d36d0 con 0x7f9f880b6a90 2024-04-25T09:17:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.642+0000 7f9f87e00640 1 --2- 172.21.15.62:0/36129784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b6a90 0x7f9f880d1fe0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009870 tx=0x7f9f7002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a19fb8357762ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.642+0000 7f9f87400640 1 -- 172.21.15.62:0/36129784 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70037070 con 0x7f9f880b6a90 2024-04-25T09:17:11.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.642+0000 7f9f87400640 1 -- 172.21.15.62:0/36129784 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f70032440 con 0x7f9f880b6a90 2024-04-25T09:17:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.642+0000 7f9f87400640 1 -- 172.21.15.62:0/36129784 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70032810 con 0x7f9f880b6a90 2024-04-25T09:17:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.643+0000 7f9f8e000640 1 -- 172.21.15.62:0/36129784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b6a90 msgr2=0x7f9f880d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.643+0000 7f9f8e000640 1 --2- 172.21.15.62:0/36129784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b6a90 0x7f9f880d1fe0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009870 tx=0x7f9f7002a2a0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.643+0000 7f9f8e000640 1 -- 172.21.15.62:0/36129784 shutdown_connections 2024-04-25T09:17:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.643+0000 7f9f8e000640 1 --2- 172.21.15.62:0/36129784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b6a90 0x7f9f880d1fe0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.643+0000 7f9f8e000640 1 -- 172.21.15.62:0/36129784 >> 172.21.15.62:0/36129784 conn(0x7f9f88066d10 msgr2=0x7f9f88067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.643+0000 7f9f8e000640 1 -- 172.21.15.62:0/36129784 shutdown_connections 2024-04-25T09:17:11.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.643+0000 7f9f8e000640 1 -- 172.21.15.62:0/36129784 wait complete. 2024-04-25T09:17:11.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.645+0000 7f9f8e000640 1 Processor -- start 2024-04-25T09:17:11.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.645+0000 7f9f8e000640 1 -- start start 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.646+0000 7f9f8e000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f880b27a0 0x7f9f8815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.646+0000 7f9f8e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b56f0 0x7f9f881581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.646+0000 7f9f8e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f880b6a90 0x7f9f88158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.646+0000 7f9f8e000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880b2ee0 con 0x7f9f880b56f0 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.646+0000 7f9f8e000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880b2d60 con 0x7f9f880b6a90 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.646+0000 7f9f8e000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880d2c30 con 0x7f9f880b27a0 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b56f0 0x7f9f881581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b56f0 0x7f9f881581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47896/0 (socket says 172.21.15.62:47896) 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 -- 172.21.15.62:0/2748154275 learned_addr learned my addr 172.21.15.62:0/2748154275 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:11.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 -- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f880b27a0 msgr2=0x7f9f8815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f880b27a0 0x7f9f8815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 -- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f880b6a90 msgr2=0x7f9f88158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f8d600640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f880b6a90 0x7f9f88158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f880b6a90 0x7f9f88158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 -- 172.21.15.62:0/2748154275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880c7c60 con 0x7f9f880b56f0 2024-04-25T09:17:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.647+0000 7f9f87e00640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b56f0 0x7f9f881581e0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f9f700090b0 tx=0x7f9f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.648+0000 7f9f84c00640 1 -- 172.21.15.62:0/2748154275 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70041070 con 0x7f9f880b56f0 2024-04-25T09:17:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.648+0000 7f9f84c00640 1 -- 172.21.15.62:0/2748154275 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7003c890 con 0x7f9f880b56f0 2024-04-25T09:17:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.648+0000 7f9f84c00640 1 -- 172.21.15.62:0/2748154275 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70037040 con 0x7f9f880b56f0 2024-04-25T09:17:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.648+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f88158fb0 con 0x7f9f880b56f0 2024-04-25T09:17:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.648+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f881597f0 con 0x7f9f880b56f0 2024-04-25T09:17:11.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.649+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8804ef20 con 0x7f9f880b56f0 2024-04-25T09:17:11.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.650+0000 7f9f84c00640 1 -- 172.21.15.62:0/2748154275 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f70044070 con 0x7f9f880b56f0 2024-04-25T09:17:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.651+0000 7f9f84c00640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f6406b4d0 0x7f9f6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.651+0000 7f9f84c00640 1 -- 172.21.15.62:0/2748154275 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9f700aaf10 con 0x7f9f880b56f0 2024-04-25T09:17:11.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.652+0000 7f9f8cc00640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f6406b4d0 0x7f9f6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:11.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.653+0000 7f9f8cc00640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f6406b4d0 0x7f9f6406d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9f780043c0 tx=0x7f9f78007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:11.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.656+0000 7f9f84c00640 1 -- 172.21.15.62:0/2748154275 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f88158fb0 con 0x7f9f880b56f0 2024-04-25T09:17:11.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.899+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f880ce580 con 0x7f9f880b56f0 2024-04-25T09:17:11.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.900+0000 7f9f84c00640 1 -- 172.21.15.62:0/2748154275 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f9f880c7c60 con 0x7f9f880b56f0 2024-04-25T09:17:11.886 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:11.886 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:17:04.730865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6814":{"gid":6814,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":20,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:17:11.886 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 90 2024-04-25T09:17:11.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f6406b4d0 msgr2=0x7f9f6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:11.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f6406b4d0 0x7f9f6406d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9f780043c0 tx=0x7f9f78007d30 comp rx=0 tx=0).stop 2024-04-25T09:17:11.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b56f0 msgr2=0x7f9f881581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:11.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b56f0 0x7f9f881581e0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f9f700090b0 tx=0x7f9f700046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 shutdown_connections 2024-04-25T09:17:11.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9f6406b4d0 0x7f9f6406d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9f880b56f0 0x7f9f881581e0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f880b6a90 0x7f9f88158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 --2- 172.21.15.62:0/2748154275 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9f880b27a0 0x7f9f8815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:11.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 >> 172.21.15.62:0/2748154275 conn(0x7f9f88066d10 msgr2=0x7f9f880b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:11.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 shutdown_connections 2024-04-25T09:17:11.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:11.903+0000 7f9f8e000640 1 -- 172.21.15.62:0/2748154275 wait complete. 2024-04-25T09:17:11.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:12.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.132+0000 7f5613000640 1 Processor -- start 2024-04-25T09:17:12.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.133+0000 7f5613000640 1 -- start start 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5613000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f560c0b56f0 0x7f560c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5613000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b6a90 0x7f560c161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5613000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f560c0b27a0 0x7f560c161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5613000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c161780 con 0x7f560c0b56f0 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5613000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c0cedc0 con 0x7f560c0b27a0 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5613000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0d3110 con 0x7f560c0b6a90 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5611c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f560c0b56f0 0x7f560c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5611c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f560c0b56f0 0x7f560c160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47910/0 (socket says 172.21.15.62:47910) 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.134+0000 7f5611c00640 1 -- 172.21.15.62:0/3105316554 learned_addr learned my addr 172.21.15.62:0/3105316554 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:12.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.135+0000 7f5612600640 1 --2- 172.21.15.62:0/3105316554 >> v2:172.21.15.121:3300/0 conn(0x7f560c0b27a0 0x7f560c161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.135+0000 7f5611200640 1 --2- 172.21.15.62:0/3105316554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b6a90 0x7f560c161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.135+0000 7f5611c00640 1 -- 172.21.15.62:0/3105316554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b6a90 msgr2=0x7f560c161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.135+0000 7f5611c00640 1 --2- 172.21.15.62:0/3105316554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b6a90 0x7f560c161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.135+0000 7f5611c00640 1 -- 172.21.15.62:0/3105316554 >> v2:172.21.15.121:3300/0 conn(0x7f560c0b27a0 msgr2=0x7f560c161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.135+0000 7f5611c00640 1 --2- 172.21.15.62:0/3105316554 >> v2:172.21.15.121:3300/0 conn(0x7f560c0b27a0 0x7f560c161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.135+0000 7f5611c00640 1 -- 172.21.15.62:0/3105316554 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0d33f0 con 0x7f560c0b56f0 2024-04-25T09:17:12.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.135+0000 7f5612600640 1 --2- 172.21.15.62:0/3105316554 >> v2:172.21.15.121:3300/0 conn(0x7f560c0b27a0 0x7f560c161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.136+0000 7f5611c00640 1 --2- 172.21.15.62:0/3105316554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b56f0 0x7f560c160d00 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f560c0b30b0 tx=0x7f560002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e93eccfe8f341eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:12.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.136+0000 7f560be00640 1 -- 172.21.15.62:0/3105316554 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600037070 con 0x7f560c0b56f0 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.136+0000 7f560be00640 1 -- 172.21.15.62:0/3105316554 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5600032440 con 0x7f560c0b56f0 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f560be00640 1 -- 172.21.15.62:0/3105316554 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600032810 con 0x7f560c0b56f0 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 -- 172.21.15.62:0/3105316554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b56f0 msgr2=0x7f560c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 --2- 172.21.15.62:0/3105316554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b56f0 0x7f560c160d00 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f560c0b30b0 tx=0x7f560002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 -- 172.21.15.62:0/3105316554 shutdown_connections 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 --2- 172.21.15.62:0/3105316554 >> v2:172.21.15.121:3300/0 conn(0x7f560c0b27a0 0x7f560c161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 --2- 172.21.15.62:0/3105316554 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b6a90 0x7f560c161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 --2- 172.21.15.62:0/3105316554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b56f0 0x7f560c160d00 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 -- 172.21.15.62:0/3105316554 >> 172.21.15.62:0/3105316554 conn(0x7f560c066d10 msgr2=0x7f560c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 -- 172.21.15.62:0/3105316554 shutdown_connections 2024-04-25T09:17:12.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.137+0000 7f5613000640 1 -- 172.21.15.62:0/3105316554 wait complete. 2024-04-25T09:17:12.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.139+0000 7f5613000640 1 Processor -- start 2024-04-25T09:17:12.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.139+0000 7f5613000640 1 -- start start 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.140+0000 7f5613000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b27a0 0x7f560c1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.140+0000 7f5613000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f560c0b56f0 0x7f560c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.140+0000 7f5613000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b6a90 0x7f560c15c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.140+0000 7f5613000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0b2d60 con 0x7f560c0b6a90 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.140+0000 7f5613000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0cea50 con 0x7f560c0b56f0 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.140+0000 7f5613000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0b2ee0 con 0x7f560c0b27a0 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.140+0000 7f5612600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b6a90 0x7f560c15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5612600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b6a90 0x7f560c15c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47924/0 (socket says 172.21.15.62:47924) 2024-04-25T09:17:12.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5612600640 1 -- 172.21.15.62:0/4275815511 learned_addr learned my addr 172.21.15.62:0/4275815511 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5611c00640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b27a0 0x7f560c1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5612600640 1 -- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b27a0 msgr2=0x7f560c1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5612600640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b27a0 0x7f560c1593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5612600640 1 -- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f560c0b56f0 msgr2=0x7f560c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5612600640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f560c0b56f0 0x7f560c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5612600640 1 -- 172.21.15.62:0/4275815511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0ca890 con 0x7f560c0b6a90 2024-04-25T09:17:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.141+0000 7f5612600640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b6a90 0x7f560c15c700 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f560400cef0 tx=0x7f560400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.142+0000 7f5609600640 1 -- 172.21.15.62:0/4275815511 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560401a070 con 0x7f560c0b6a90 2024-04-25T09:17:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.142+0000 7f5609600640 1 -- 172.21.15.62:0/4275815511 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560400d840 con 0x7f560c0b6a90 2024-04-25T09:17:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.142+0000 7f5609600640 1 -- 172.21.15.62:0/4275815511 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400dc30 con 0x7f560c0b6a90 2024-04-25T09:17:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.142+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c1f7870 con 0x7f560c0b6a90 2024-04-25T09:17:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.142+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c1f7e30 con 0x7f560c0b6a90 2024-04-25T09:17:12.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.143+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c074030 con 0x7f560c0b6a90 2024-04-25T09:17:12.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.144+0000 7f5609600640 1 -- 172.21.15.62:0/4275815511 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f560401d070 con 0x7f560c0b6a90 2024-04-25T09:17:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.145+0000 7f5609600640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f55ec06b500 0x7f55ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.145+0000 7f5609600640 1 -- 172.21.15.62:0/4275815511 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f560408aa20 con 0x7f560c0b6a90 2024-04-25T09:17:12.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.146+0000 7f5611c00640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f55ec06b500 0x7f55ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.147+0000 7f5611c00640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f55ec06b500 0x7f55ec06d9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5600034000 tx=0x7f560002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:12.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.150+0000 7f5609600640 1 -- 172.21.15.62:0/4275815511 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f560c074030 con 0x7f560c0b6a90 2024-04-25T09:17:12.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.351+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f560c04ef20 con 0x7f560c0b6a90 2024-04-25T09:17:12.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.354+0000 7f5609600640 1 -- 172.21.15.62:0/4275815511 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f560405f030 con 0x7f560c0b6a90 2024-04-25T09:17:12.340 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:12.341 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":138,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:02.676192+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:16:00.695740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:16:01.838452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/2303077743":"2024-04-26T09:17:02.676113+0000","172.21.15.62:6832/2303077743":"2024-04-26T09:17:02.676113+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:12.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.356+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f55ec06b500 msgr2=0x7f55ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.356+0000 7f5613000640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f55ec06b500 0x7f55ec06d9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5600034000 tx=0x7f560002e000 comp rx=0 tx=0).stop 2024-04-25T09:17:12.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.356+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b6a90 msgr2=0x7f560c15c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.356+0000 7f5613000640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b6a90 0x7f560c15c700 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f560400cef0 tx=0x7f560400a710 comp rx=0 tx=0).stop 2024-04-25T09:17:12.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.357+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 shutdown_connections 2024-04-25T09:17:12.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.357+0000 7f5613000640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f55ec06b500 0x7f55ec06d9c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.357+0000 7f5613000640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f560c0b6a90 0x7f560c15c700 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.357+0000 7f5613000640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f560c0b56f0 0x7f560c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.357+0000 7f5613000640 1 --2- 172.21.15.62:0/4275815511 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f560c0b27a0 0x7f560c1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.357+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 >> 172.21.15.62:0/4275815511 conn(0x7f560c066d10 msgr2=0x7f560c0b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.357+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 shutdown_connections 2024-04-25T09:17:12.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.357+0000 7f5613000640 1 -- 172.21.15.62:0/4275815511 wait complete. 2024-04-25T09:17:12.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:16:00.695740+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:16:01.838452+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:17:12.364 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:17:12.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.601+0000 7fcf3ce00640 1 Processor -- start 2024-04-25T09:17:12.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.601+0000 7fcf3ce00640 1 -- start start 2024-04-25T09:17:12.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.602+0000 7fcf3ce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcf3815dd50 0x7fcf380d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.602+0000 7fcf3ce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcf380d3960 0x7fcf380d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.603+0000 7fcf3ce00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf380ce780 con 0x7fcf380d2fb0 2024-04-25T09:17:12.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.603+0000 7fcf3ce00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf380d42f0 con 0x7fcf3815dd50 2024-04-25T09:17:12.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.603+0000 7fcf3ce00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf380d3440 con 0x7fcf380d3960 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.603+0000 7fcf37400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcf3815dd50 0x7fcf380d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.603+0000 7fcf37400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcf3815dd50 0x7fcf380d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47926/0 (socket says 172.21.15.62:47926) 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.603+0000 7fcf37400640 1 -- 172.21.15.62:0/2312193104 learned_addr learned my addr 172.21.15.62:0/2312193104 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.603+0000 7fcf37e00640 1 --2- 172.21.15.62:0/2312193104 >> v2:172.21.15.121:3300/0 conn(0x7fcf380d3960 0x7fcf380d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.604+0000 7fcf37400640 1 -- 172.21.15.62:0/2312193104 >> v2:172.21.15.121:3300/0 conn(0x7fcf380d3960 msgr2=0x7fcf380d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.604+0000 7fcf37400640 1 --2- 172.21.15.62:0/2312193104 >> v2:172.21.15.121:3300/0 conn(0x7fcf380d3960 0x7fcf380d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.604+0000 7fcf37400640 1 -- 172.21.15.62:0/2312193104 >> v1:172.21.15.62:6789/0 conn(0x7fcf380d2fb0 legacy=0x7fcf380ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.604+0000 7fcf37400640 1 -- 172.21.15.62:0/2312193104 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf380ce1c0 con 0x7fcf3815dd50 2024-04-25T09:17:12.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.604+0000 7fcf36000640 1 -- 172.21.15.62:0/2312193104 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300746322 0 0) 0x7fcf380ce780 con 0x7fcf380d2fb0 2024-04-25T09:17:12.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.605+0000 7fcf37400640 1 --2- 172.21.15.62:0/2312193104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf380d2300 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcf380caa50 tx=0x7fcf2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a166c90640bac75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:12.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.605+0000 7fcf36000640 1 -- 172.21.15.62:0/2312193104 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28037070 con 0x7fcf3815dd50 2024-04-25T09:17:12.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.605+0000 7fcf36000640 1 -- 172.21.15.62:0/2312193104 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf28032440 con 0x7fcf3815dd50 2024-04-25T09:17:12.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.605+0000 7fcf36000640 1 -- 172.21.15.62:0/2312193104 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28032810 con 0x7fcf3815dd50 2024-04-25T09:17:12.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.606+0000 7fcf3ce00640 1 -- 172.21.15.62:0/2312193104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 msgr2=0x7fcf380d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.606+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/2312193104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf380d2300 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcf380caa50 tx=0x7fcf2802a270 comp rx=0 tx=0).stop 2024-04-25T09:17:12.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.606+0000 7fcf3ce00640 1 -- 172.21.15.62:0/2312193104 shutdown_connections 2024-04-25T09:17:12.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.606+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/2312193104 >> v2:172.21.15.121:3300/0 conn(0x7fcf380d3960 0x7fcf380d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.606+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/2312193104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf380d2300 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.606+0000 7fcf3ce00640 1 -- 172.21.15.62:0/2312193104 >> 172.21.15.62:0/2312193104 conn(0x7fcf3806b000 msgr2=0x7fcf3806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:12.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.606+0000 7fcf3ce00640 1 -- 172.21.15.62:0/2312193104 shutdown_connections 2024-04-25T09:17:12.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.606+0000 7fcf3ce00640 1 -- 172.21.15.62:0/2312193104 wait complete. 2024-04-25T09:17:12.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.608+0000 7fcf3ce00640 1 Processor -- start 2024-04-25T09:17:12.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.608+0000 7fcf3ce00640 1 -- start start 2024-04-25T09:17:12.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf3ce00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf380d3960 0x7fcf380d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf3ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf3814b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf3ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf3814be70 0x7fcf381f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf3ce00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf380b6b00 con 0x7fcf3815dd50 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf3ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf380d2840 con 0x7fcf3814be70 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf3ce00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf380b6c80 con 0x7fcf380d3960 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf3814b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf3814b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47928/0 (socket says 172.21.15.62:47928) 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf36a00640 1 -- 172.21.15.62:0/3798900186 learned_addr learned my addr 172.21.15.62:0/3798900186 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf37e00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf3814be70 0x7fcf381f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf37400640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf380d3960 0x7fcf380d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf36a00640 1 -- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf380d3960 msgr2=0x7fcf380d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.609+0000 7fcf36a00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf380d3960 0x7fcf380d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf36a00640 1 -- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf3814be70 msgr2=0x7fcf381f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf36a00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf3814be70 0x7fcf381f6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf36a00640 1 -- 172.21.15.62:0/3798900186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3814c300 con 0x7fcf3815dd50 2024-04-25T09:17:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf36a00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf3814b930 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c890 tx=0x7fcf2000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:12.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf2f400640 1 -- 172.21.15.62:0/3798900186 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20017070 con 0x7fcf3815dd50 2024-04-25T09:17:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf2f400640 1 -- 172.21.15.62:0/3798900186 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf20003dc0 con 0x7fcf3815dd50 2024-04-25T09:17:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf380c8be0 con 0x7fcf3815dd50 2024-04-25T09:17:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf2f400640 1 -- 172.21.15.62:0/3798900186 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf200052b0 con 0x7fcf3815dd50 2024-04-25T09:17:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.610+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf380c91a0 con 0x7fcf3815dd50 2024-04-25T09:17:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.612+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3804ef20 con 0x7fcf3815dd50 2024-04-25T09:17:12.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.612+0000 7fcf2f400640 1 -- 172.21.15.62:0/3798900186 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf2001a070 con 0x7fcf3815dd50 2024-04-25T09:17:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.613+0000 7fcf2f400640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf1406b500 0x7fcf1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:12.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.613+0000 7fcf2f400640 1 -- 172.21.15.62:0/3798900186 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcf20089970 con 0x7fcf3815dd50 2024-04-25T09:17:12.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.614+0000 7fcf37400640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf1406b500 0x7fcf1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:12.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.615+0000 7fcf37400640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf1406b500 0x7fcf1406d9c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcf28034000 tx=0x7fcf2802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:12.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.618+0000 7fcf2f400640 1 -- 172.21.15.62:0/3798900186 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf20058780 con 0x7fcf3815dd50 2024-04-25T09:17:12.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.820+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf380b9950 con 0x7fcf3815dd50 2024-04-25T09:17:12.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.822+0000 7fcf2f400640 1 -- 172.21.15.62:0/3798900186 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf20058080 con 0x7fcf3815dd50 2024-04-25T09:17:12.808 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf1406b500 msgr2=0x7fcf1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf1406b500 0x7fcf1406d9c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcf28034000 tx=0x7fcf2802e000 comp rx=0 tx=0).stop 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 msgr2=0x7fcf3814b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf3814b930 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c890 tx=0x7fcf2000cd50 comp rx=0 tx=0).stop 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 shutdown_connections 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcf1406b500 0x7fcf1406d9c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf3815dd50 0x7fcf3814b930 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf3814be70 0x7fcf381f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 --2- 172.21.15.62:0/3798900186 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcf380d3960 0x7fcf380d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:12.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.824+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 >> 172.21.15.62:0/3798900186 conn(0x7fcf3806b000 msgr2=0x7fcf380b8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:12.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.825+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 shutdown_connections 2024-04-25T09:17:12.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:12.825+0000 7fcf3ce00640 1 -- 172.21.15.62:0/3798900186 wait complete. 2024-04-25T09:17:12.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:17:13.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.064+0000 7f306d000640 1 Processor -- start 2024-04-25T09:17:13.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.065+0000 7f306d000640 1 -- start start 2024-04-25T09:17:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f306d000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30680b0f40 0x7f30680cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f306d000640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680b4920 con 0x7f30680b6ad0 2024-04-25T09:17:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f306d000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680b4c00 con 0x7f306815b960 2024-04-25T09:17:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f306d000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306815bd40 con 0x7f30680b0f40 2024-04-25T09:17:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f3067e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30680b0f40 0x7f30680cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f3067e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30680b0f40 0x7f30680cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47934/0 (socket says 172.21.15.62:47934) 2024-04-25T09:17:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f3067e00640 1 -- 172.21.15.62:0/2189737085 learned_addr learned my addr 172.21.15.62:0/2189737085 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:13.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f3066a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f30680b6ad0 0x7f30680cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59940/0 (socket says 172.21.15.62:59940) 2024-04-25T09:17:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f3067e00640 1 -- 172.21.15.62:0/2189737085 >> v1:172.21.15.121:6789/0 conn(0x7f306815b960 legacy=0x7f30680b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f3067e00640 1 -- 172.21.15.62:0/2189737085 >> v1:172.21.15.62:6789/0 conn(0x7f30680b6ad0 legacy=0x7f30680cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.067+0000 7f3067e00640 1 -- 172.21.15.62:0/2189737085 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30680b4ee0 con 0x7f30680b0f40 2024-04-25T09:17:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.069+0000 7f3067e00640 1 --2- 172.21.15.62:0/2189737085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30680b0f40 0x7f30680cff70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f3060009870 tx=0x7f306002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ecc661fdf73233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.069+0000 7f3066000640 1 -- 172.21.15.62:0/2189737085 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060037070 con 0x7f30680b0f40 2024-04-25T09:17:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.069+0000 7f3066000640 1 -- 172.21.15.62:0/2189737085 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060032440 con 0x7f30680b0f40 2024-04-25T09:17:13.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.069+0000 7f3066000640 1 -- 172.21.15.62:0/2189737085 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060032810 con 0x7f30680b0f40 2024-04-25T09:17:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.070+0000 7f306d000640 1 -- 172.21.15.62:0/2189737085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30680b0f40 msgr2=0x7f30680cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.070+0000 7f306d000640 1 --2- 172.21.15.62:0/2189737085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30680b0f40 0x7f30680cff70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f3060009870 tx=0x7f306002a210 comp rx=0 tx=0).stop 2024-04-25T09:17:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.070+0000 7f306d000640 1 -- 172.21.15.62:0/2189737085 shutdown_connections 2024-04-25T09:17:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.070+0000 7f306d000640 1 --2- 172.21.15.62:0/2189737085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30680b0f40 0x7f30680cff70 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.070+0000 7f306d000640 1 -- 172.21.15.62:0/2189737085 >> 172.21.15.62:0/2189737085 conn(0x7f3068072ce0 msgr2=0x7f30680668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.070+0000 7f306d000640 1 -- 172.21.15.62:0/2189737085 shutdown_connections 2024-04-25T09:17:13.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.070+0000 7f306d000640 1 -- 172.21.15.62:0/2189737085 wait complete. 2024-04-25T09:17:13.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.072+0000 7f306d000640 1 Processor -- start 2024-04-25T09:17:13.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.072+0000 7f306d000640 1 -- start start 2024-04-25T09:17:13.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f306d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30680b0f40 0x7f306814da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f306d000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30680b6ad0 0x7f306814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f306d000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306815b960 0x7f3068151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f306d000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30680702a0 con 0x7f306815b960 2024-04-25T09:17:13.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f306d000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30680b54a0 con 0x7f30680b0f40 2024-04-25T09:17:13.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f306d000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30680b09f0 con 0x7f30680b6ad0 2024-04-25T09:17:13.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f3067e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306815b960 0x7f3068151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f3067e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306815b960 0x7f3068151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47940/0 (socket says 172.21.15.62:47940) 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f3067e00640 1 -- 172.21.15.62:0/2887507813 learned_addr learned my addr 172.21.15.62:0/2887507813 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f3067400640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30680b0f40 0x7f306814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.073+0000 7f3066a00640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30680b6ad0 0x7f306814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f3067e00640 1 -- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30680b6ad0 msgr2=0x7f306814f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f3067e00640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30680b6ad0 0x7f306814f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f3067e00640 1 -- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30680b0f40 msgr2=0x7f306814da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f3067e00640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30680b0f40 0x7f306814da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f3067e00640 1 -- 172.21.15.62:0/2887507813 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306814ea30 con 0x7f306815b960 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f3067400640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30680b0f40 0x7f306814da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f3066a00640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30680b6ad0 0x7f306814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f3067e00640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306815b960 0x7f3068151550 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f306002a250 tx=0x7f30600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f305f400640 1 -- 172.21.15.62:0/2887507813 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060041020 con 0x7f306815b960 2024-04-25T09:17:13.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f305f400640 1 -- 172.21.15.62:0/2887507813 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060030990 con 0x7f306815b960 2024-04-25T09:17:13.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f305f400640 1 -- 172.21.15.62:0/2887507813 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060037040 con 0x7f306815b960 2024-04-25T09:17:13.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.074+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3068150750 con 0x7f306815b960 2024-04-25T09:17:13.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.075+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3068150d10 con 0x7f306815b960 2024-04-25T09:17:13.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.076+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306806f9d0 con 0x7f306815b960 2024-04-25T09:17:13.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.077+0000 7f305f400640 1 -- 172.21.15.62:0/2887507813 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3060044020 con 0x7f306815b960 2024-04-25T09:17:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.077+0000 7f305f400640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f304406b4d0 0x7f304406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.078+0000 7f305f400640 1 -- 172.21.15.62:0/2887507813 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f30600aab90 con 0x7f306815b960 2024-04-25T09:17:13.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.078+0000 7f3067400640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f304406b4d0 0x7f304406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.079+0000 7f3067400640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f304406b4d0 0x7f304406d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3058007e90 tx=0x7f30580073a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.083+0000 7f305f400640 1 -- 172.21.15.62:0/2887507813 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f306004b030 con 0x7f306815b960 2024-04-25T09:17:13.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.279+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30680750e0 con 0x7f306815b960 2024-04-25T09:17:13.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.281+0000 7f305f400640 1 -- 172.21.15.62:0/2887507813 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30600799a0 con 0x7f306815b960 2024-04-25T09:17:13.267 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:17:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f304406b4d0 msgr2=0x7f304406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f304406b4d0 0x7f304406d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3058007e90 tx=0x7f30580073a0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306815b960 msgr2=0x7f3068151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306815b960 0x7f3068151550 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f306002a250 tx=0x7f30600046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 shutdown_connections 2024-04-25T09:17:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f304406b4d0 0x7f304406d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f306815b960 0x7f3068151550 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f30680b6ad0 0x7f306814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 --2- 172.21.15.62:0/2887507813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30680b0f40 0x7f306814da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 >> 172.21.15.62:0/2887507813 conn(0x7f3068072ce0 msgr2=0x7f30680668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:13.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.283+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 shutdown_connections 2024-04-25T09:17:13.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.284+0000 7f306d000640 1 -- 172.21.15.62:0/2887507813 wait complete. 2024-04-25T09:17:13.288 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:17:13.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.514+0000 7f1438a00640 1 Processor -- start 2024-04-25T09:17:13.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.514+0000 7f1438a00640 1 -- start start 2024-04-25T09:17:13.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1438a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b2d80 0x7f14340b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1438a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f14340b36a0 0x7f14340ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1438a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f14340b54d0 0x7f14340700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1438a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14340b4430 con 0x7f14340b36a0 2024-04-25T09:17:13.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1438a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14340b4710 con 0x7f14340b54d0 2024-04-25T09:17:13.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1438a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b49f0 con 0x7f14340b2d80 2024-04-25T09:17:13.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1432a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f14340b36a0 0x7f14340ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1432a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f14340b36a0 0x7f14340ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47950/0 (socket says 172.21.15.62:47950) 2024-04-25T09:17:13.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1432a00640 1 -- 172.21.15.62:0/203822738 learned_addr learned my addr 172.21.15.62:0/203822738 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:13.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.516+0000 7f1433400640 1 --2- 172.21.15.62:0/203822738 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b2d80 0x7f14340b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.517+0000 7f1432a00640 1 -- 172.21.15.62:0/203822738 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b2d80 msgr2=0x7f14340b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:13.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.517+0000 7f1433e00640 1 --2- 172.21.15.62:0/203822738 >> v2:172.21.15.121:3300/0 conn(0x7f14340b54d0 0x7f14340700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.517+0000 7f1432a00640 1 --2- 172.21.15.62:0/203822738 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b2d80 0x7f14340b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.517+0000 7f1432a00640 1 -- 172.21.15.62:0/203822738 >> v2:172.21.15.121:3300/0 conn(0x7f14340b54d0 msgr2=0x7f14340700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:13.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.517+0000 7f1432a00640 1 --2- 172.21.15.62:0/203822738 >> v2:172.21.15.121:3300/0 conn(0x7f14340b54d0 0x7f14340700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.517+0000 7f1432a00640 1 -- 172.21.15.62:0/203822738 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340b4cd0 con 0x7f14340b36a0 2024-04-25T09:17:13.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.517+0000 7f1433e00640 1 --2- 172.21.15.62:0/203822738 >> v2:172.21.15.121:3300/0 conn(0x7f14340b54d0 0x7f14340700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:13.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.518+0000 7f1432a00640 1 --2- 172.21.15.62:0/203822738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b36a0 0x7f14340ce2b0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1434072100 tx=0x7f142002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeee4016fbb70c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.518+0000 7f1432000640 1 -- 172.21.15.62:0/203822738 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420037070 con 0x7f14340b36a0 2024-04-25T09:17:13.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.518+0000 7f1432000640 1 -- 172.21.15.62:0/203822738 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1420032440 con 0x7f14340b36a0 2024-04-25T09:17:13.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.518+0000 7f1432000640 1 -- 172.21.15.62:0/203822738 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420032810 con 0x7f14340b36a0 2024-04-25T09:17:13.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 -- 172.21.15.62:0/203822738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b36a0 msgr2=0x7f14340ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 --2- 172.21.15.62:0/203822738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b36a0 0x7f14340ce2b0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1434072100 tx=0x7f142002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 -- 172.21.15.62:0/203822738 shutdown_connections 2024-04-25T09:17:13.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 --2- 172.21.15.62:0/203822738 >> v2:172.21.15.121:3300/0 conn(0x7f14340b54d0 0x7f14340700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 --2- 172.21.15.62:0/203822738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b36a0 0x7f14340ce2b0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 --2- 172.21.15.62:0/203822738 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b2d80 0x7f14340b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 -- 172.21.15.62:0/203822738 >> 172.21.15.62:0/203822738 conn(0x7f1434066f20 msgr2=0x7f14340b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:13.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 -- 172.21.15.62:0/203822738 shutdown_connections 2024-04-25T09:17:13.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.519+0000 7f1438a00640 1 -- 172.21.15.62:0/203822738 wait complete. 2024-04-25T09:17:13.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.522+0000 7f1438a00640 1 Processor -- start 2024-04-25T09:17:13.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.522+0000 7f1438a00640 1 -- start start 2024-04-25T09:17:13.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.523+0000 7f1438a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14340b2d80 0x7f143414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.523+0000 7f1438a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b36a0 0x7f14341e9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.523+0000 7f1438a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b54d0 0x7f14341e9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.523+0000 7f1438a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340ce7f0 con 0x7f14340b54d0 2024-04-25T09:17:13.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.523+0000 7f1438a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b40e0 con 0x7f14340b2d80 2024-04-25T09:17:13.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.523+0000 7f1438a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340ce970 con 0x7f14340b36a0 2024-04-25T09:17:13.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b54d0 0x7f14341e9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b54d0 0x7f14341e9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47952/0 (socket says 172.21.15.62:47952) 2024-04-25T09:17:13.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433e00640 1 -- 172.21.15.62:0/2278717716 learned_addr learned my addr 172.21.15.62:0/2278717716 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:13.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1432a00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b36a0 0x7f14341e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433400640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14340b2d80 0x7f143414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433e00640 1 -- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b36a0 msgr2=0x7f14341e9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433e00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b36a0 0x7f14341e9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433e00640 1 -- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14340b2d80 msgr2=0x7f143414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433e00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14340b2d80 0x7f143414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.524+0000 7f1433e00640 1 -- 172.21.15.62:0/2278717716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434154140 con 0x7f14340b54d0 2024-04-25T09:17:13.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.525+0000 7f1433e00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b54d0 0x7f14341e9960 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f142c00cef0 tx=0x7f142c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.525+0000 7f142b400640 1 -- 172.21.15.62:0/2278717716 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c01a070 con 0x7f14340b54d0 2024-04-25T09:17:13.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.525+0000 7f142b400640 1 -- 172.21.15.62:0/2278717716 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c00d860 con 0x7f14340b54d0 2024-04-25T09:17:13.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.525+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14341543e0 con 0x7f14340b54d0 2024-04-25T09:17:13.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.525+0000 7f142b400640 1 -- 172.21.15.62:0/2278717716 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00dc50 con 0x7f14340b54d0 2024-04-25T09:17:13.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.525+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14341549a0 con 0x7f14340b54d0 2024-04-25T09:17:13.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.526+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143404ef20 con 0x7f14340b54d0 2024-04-25T09:17:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.527+0000 7f142b400640 1 -- 172.21.15.62:0/2278717716 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f142c01d070 con 0x7f14340b54d0 2024-04-25T09:17:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.528+0000 7f142b400640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f141406b4d0 0x7f141406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.528+0000 7f142b400640 1 -- 172.21.15.62:0/2278717716 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f142c08abb0 con 0x7f14340b54d0 2024-04-25T09:17:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.529+0000 7f1433400640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f141406b4d0 0x7f141406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.530+0000 7f1433400640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f141406b4d0 0x7f141406d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1424006fd0 tx=0x7f1424008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.533+0000 7f142b400640 1 -- 172.21.15.62:0/2278717716 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f142c059a00 con 0x7f14340b54d0 2024-04-25T09:17:13.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.723+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1434071560 con 0x7f14340b54d0 2024-04-25T09:17:13.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.724+0000 7f142b400640 1 -- 172.21.15.62:0/2278717716 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7f142c059300 con 0x7f14340b54d0 2024-04-25T09:17:13.710 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f141406b4d0 msgr2=0x7f141406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f141406b4d0 0x7f141406d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1424006fd0 tx=0x7f1424008040 comp rx=0 tx=0).stop 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b54d0 msgr2=0x7f14341e9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b54d0 0x7f14341e9960 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f142c00cef0 tx=0x7f142c00a710 comp rx=0 tx=0).stop 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 shutdown_connections 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f141406b4d0 0x7f141406d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14340b54d0 0x7f14341e9960 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f14340b36a0 0x7f14341e9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 --2- 172.21.15.62:0/2278717716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14340b2d80 0x7f143414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.726+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 >> 172.21.15.62:0/2278717716 conn(0x7f1434066f20 msgr2=0x7f1434070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.727+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 shutdown_connections 2024-04-25T09:17:13.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.727+0000 7f1438a00640 1 -- 172.21.15.62:0/2278717716 wait complete. 2024-04-25T09:17:13.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:13.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.938+0000 7fd597a00640 1 Processor -- start 2024-04-25T09:17:13.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.939+0000 7fd597a00640 1 -- start start 2024-04-25T09:17:13.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.940+0000 7fd597a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd590072340 0x7fd5900702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.940+0000 7fd597a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd590070c10 con 0x7fd5900707f0 2024-04-25T09:17:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.940+0000 7fd597a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59006c310 con 0x7fd5900cfed0 2024-04-25T09:17:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.940+0000 7fd597a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59006c5b0 con 0x7fd590072340 2024-04-25T09:17:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.941+0000 7fd58fe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd590072340 0x7fd5900702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.941+0000 7fd595a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd5900707f0 0x7fd5900b2880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59956/0 (socket says 172.21.15.62:59956) 2024-04-25T09:17:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.941+0000 7fd595a00640 1 -- 172.21.15.62:0/3154400756 learned_addr learned my addr 172.21.15.62:0/3154400756 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:13.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.941+0000 7fd58fe00640 1 -- 172.21.15.62:0/3154400756 >> v1:172.21.15.121:6789/0 conn(0x7fd5900cfed0 legacy=0x7fd59015f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.941+0000 7fd58fe00640 1 -- 172.21.15.62:0/3154400756 >> v1:172.21.15.62:6789/0 conn(0x7fd5900707f0 legacy=0x7fd5900b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.941+0000 7fd58fe00640 1 -- 172.21.15.62:0/3154400756 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59006c890 con 0x7fd590072340 2024-04-25T09:17:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.941+0000 7fd58f400640 1 -- 172.21.15.62:0/3154400756 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413399538 0 0) 0x7fd590070c10 con 0x7fd5900707f0 2024-04-25T09:17:13.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.942+0000 7fd58fe00640 1 --2- 172.21.15.62:0/3154400756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd590072340 0x7fd5900702b0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0097a0 tx=0x7fd57c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e90547210f2e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.943+0000 7fd58f400640 1 -- 172.21.15.62:0/3154400756 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c037070 con 0x7fd590072340 2024-04-25T09:17:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.943+0000 7fd58f400640 1 -- 172.21.15.62:0/3154400756 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c032440 con 0x7fd590072340 2024-04-25T09:17:13.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.943+0000 7fd58f400640 1 -- 172.21.15.62:0/3154400756 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c032810 con 0x7fd590072340 2024-04-25T09:17:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.943+0000 7fd597a00640 1 -- 172.21.15.62:0/3154400756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd590072340 msgr2=0x7fd5900702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.943+0000 7fd597a00640 1 --2- 172.21.15.62:0/3154400756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd590072340 0x7fd5900702b0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0097a0 tx=0x7fd57c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.944+0000 7fd597a00640 1 -- 172.21.15.62:0/3154400756 shutdown_connections 2024-04-25T09:17:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.944+0000 7fd597a00640 1 --2- 172.21.15.62:0/3154400756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd590072340 0x7fd5900702b0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.944+0000 7fd597a00640 1 -- 172.21.15.62:0/3154400756 >> 172.21.15.62:0/3154400756 conn(0x7fd5900668f0 msgr2=0x7fd590066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.944+0000 7fd597a00640 1 -- 172.21.15.62:0/3154400756 shutdown_connections 2024-04-25T09:17:13.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.944+0000 7fd597a00640 1 -- 172.21.15.62:0/3154400756 wait complete. 2024-04-25T09:17:13.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.945+0000 7fd597a00640 1 Processor -- start 2024-04-25T09:17:13.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.946+0000 7fd597a00640 1 -- start start 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.946+0000 7fd597a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5900707f0 0x7fd590140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.946+0000 7fd597a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5900cfed0 0x7fd5900afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.946+0000 7fd597a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5900b03c0 0x7fd5901e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.946+0000 7fd597a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59006d130 con 0x7fd5900707f0 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.946+0000 7fd597a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59006ce50 con 0x7fd5900b03c0 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.946+0000 7fd597a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900b2f90 con 0x7fd5900cfed0 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.947+0000 7fd595000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5900707f0 0x7fd590140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.947+0000 7fd595000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5900707f0 0x7fd590140b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47966/0 (socket says 172.21.15.62:47966) 2024-04-25T09:17:13.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.947+0000 7fd595000640 1 -- 172.21.15.62:0/3178171086 learned_addr learned my addr 172.21.15.62:0/3178171086 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.947+0000 7fd595000640 1 -- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5900cfed0 msgr2=0x7fd5900afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.947+0000 7fd595000640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5900cfed0 0x7fd5900afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.947+0000 7fd595000640 1 -- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5900b03c0 msgr2=0x7fd5901e6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:13.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.947+0000 7fd595000640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5900b03c0 0x7fd5901e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.947+0000 7fd595000640 1 -- 172.21.15.62:0/3178171086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5901e78b0 con 0x7fd5900707f0 2024-04-25T09:17:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.948+0000 7fd595000640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5900707f0 0x7fd590140b80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fd58400b3f0 tx=0x7fd58400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.948+0000 7fd58cc00640 1 -- 172.21.15.62:0/3178171086 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584015020 con 0x7fd5900707f0 2024-04-25T09:17:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.948+0000 7fd58cc00640 1 -- 172.21.15.62:0/3178171086 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd584011830 con 0x7fd5900707f0 2024-04-25T09:17:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.948+0000 7fd58cc00640 1 -- 172.21.15.62:0/3178171086 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584011c20 con 0x7fd5900707f0 2024-04-25T09:17:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.948+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901e7b90 con 0x7fd5900707f0 2024-04-25T09:17:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.948+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5900abc70 con 0x7fd5900707f0 2024-04-25T09:17:13.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.949+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd590072c70 con 0x7fd5900707f0 2024-04-25T09:17:13.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.950+0000 7fd58cc00640 1 -- 172.21.15.62:0/3178171086 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd584018020 con 0x7fd5900707f0 2024-04-25T09:17:13.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.951+0000 7fd58cc00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57006b4d0 0x7fd57006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:13.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.952+0000 7fd58cc00640 1 -- 172.21.15.62:0/3178171086 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd584089fa0 con 0x7fd5900707f0 2024-04-25T09:17:13.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.952+0000 7fd58fe00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57006b4d0 0x7fd57006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:13.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.953+0000 7fd58fe00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57006b4d0 0x7fd57006d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd57c034000 tx=0x7fd57c02c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:13.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:13.956+0000 7fd58cc00640 1 -- 172.21.15.62:0/3178171086 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd58401f030 con 0x7fd5900707f0 2024-04-25T09:17:14.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.197+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd59004f000 con 0x7fd5900707f0 2024-04-25T09:17:14.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.198+0000 7fd58cc00640 1 -- 172.21.15.62:0/3178171086 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fd5840172c0 con 0x7fd5900707f0 2024-04-25T09:17:14.184 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:14.185 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:17:04.730865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6814":{"gid":6814,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":20,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:17:14.185 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 90 2024-04-25T09:17:14.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57006b4d0 msgr2=0x7fd57006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57006b4d0 0x7fd57006d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd57c034000 tx=0x7fd57c02c000 comp rx=0 tx=0).stop 2024-04-25T09:17:14.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5900707f0 msgr2=0x7fd590140b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5900707f0 0x7fd590140b80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fd58400b3f0 tx=0x7fd58400b8b0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 shutdown_connections 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd57006b4d0 0x7fd57006d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5900b03c0 0x7fd5901e6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd5900cfed0 0x7fd5900afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 --2- 172.21.15.62:0/3178171086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5900707f0 0x7fd590140b80 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 >> 172.21.15.62:0/3178171086 conn(0x7fd5900668f0 msgr2=0x7fd5900b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 shutdown_connections 2024-04-25T09:17:14.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.201+0000 7fd597a00640 1 -- 172.21.15.62:0/3178171086 wait complete. 2024-04-25T09:17:14.206 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:14.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.425+0000 7f217ac00640 1 Processor -- start 2024-04-25T09:17:14.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.425+0000 7f217ac00640 1 -- start start 2024-04-25T09:17:14.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f217ac00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2174072340 0x7f21740702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f217ac00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21740707f0 0x7f21740b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f217ac00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2174070c10 con 0x7f21740cfed0 2024-04-25T09:17:14.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f217ac00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217406c310 con 0x7f21740707f0 2024-04-25T09:17:14.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f217ac00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217406c5b0 con 0x7f2174072340 2024-04-25T09:17:14.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f2173e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f21740cfed0 0x7f217415f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:59968/0 (socket says 172.21.15.62:59968) 2024-04-25T09:17:14.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f2173e00640 1 -- 172.21.15.62:0/2731396736 learned_addr learned my addr 172.21.15.62:0/2731396736 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:14.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f2178c00640 1 --2- 172.21.15.62:0/2731396736 >> v2:172.21.15.62:3300/0 conn(0x7f21740707f0 0x7f21740b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f216be00640 1 --2- 172.21.15.62:0/2731396736 >> v2:172.21.15.121:3300/0 conn(0x7f2174072340 0x7f21740702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f2178c00640 1 -- 172.21.15.62:0/2731396736 >> v2:172.21.15.121:3300/0 conn(0x7f2174072340 msgr2=0x7f21740702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:14.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f2178c00640 1 --2- 172.21.15.62:0/2731396736 >> v2:172.21.15.121:3300/0 conn(0x7f2174072340 0x7f21740702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f2173400640 1 -- 172.21.15.62:0/2731396736 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837310604 0 0) 0x7f2174070c10 con 0x7f21740cfed0 2024-04-25T09:17:14.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f2178c00640 1 -- 172.21.15.62:0/2731396736 >> v1:172.21.15.62:6789/0 conn(0x7f21740cfed0 legacy=0x7f217415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.427+0000 7f2178c00640 1 -- 172.21.15.62:0/2731396736 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217406c890 con 0x7f21740707f0 2024-04-25T09:17:14.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.428+0000 7f2178c00640 1 --2- 172.21.15.62:0/2731396736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740707f0 0x7f21740b2880 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2174072760 tx=0x7f216c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92abb80f44d584f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f2173400640 1 -- 172.21.15.62:0/2731396736 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c007740 con 0x7f21740707f0 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f2173400640 1 -- 172.21.15.62:0/2731396736 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c0078e0 con 0x7f21740707f0 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f2173400640 1 -- 172.21.15.62:0/2731396736 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c007cb0 con 0x7f21740707f0 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f217ac00640 1 -- 172.21.15.62:0/2731396736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740707f0 msgr2=0x7f21740b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f217ac00640 1 --2- 172.21.15.62:0/2731396736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740707f0 0x7f21740b2880 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2174072760 tx=0x7f216c02a680 comp rx=0 tx=0).stop 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f217ac00640 1 -- 172.21.15.62:0/2731396736 shutdown_connections 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f217ac00640 1 --2- 172.21.15.62:0/2731396736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740707f0 0x7f21740b2880 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f217ac00640 1 --2- 172.21.15.62:0/2731396736 >> v2:172.21.15.121:3300/0 conn(0x7f2174072340 0x7f21740702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f217ac00640 1 -- 172.21.15.62:0/2731396736 >> 172.21.15.62:0/2731396736 conn(0x7f21740668f0 msgr2=0x7f2174066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.429+0000 7f217ac00640 1 -- 172.21.15.62:0/2731396736 shutdown_connections 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.430+0000 7f217ac00640 1 -- 172.21.15.62:0/2731396736 wait complete. 2024-04-25T09:17:14.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.431+0000 7f217ac00640 1 Processor -- start 2024-04-25T09:17:14.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.431+0000 7f217ac00640 1 -- start start 2024-04-25T09:17:14.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f217ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21740707f0 0x7f21741408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f217ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740cfed0 0x7f217413efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f217ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2174142440 0x7f217415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f217ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217406d130 con 0x7f21740cfed0 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f217ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217406ce50 con 0x7f21740707f0 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f217ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21740b2dc0 con 0x7f2174142440 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f216be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740cfed0 0x7f217413efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f216be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740cfed0 0x7f217413efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:47992/0 (socket says 172.21.15.62:47992) 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.432+0000 7f216be00640 1 -- 172.21.15.62:0/3180264682 learned_addr learned my addr 172.21.15.62:0/3180264682 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f2173e00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21740707f0 0x7f21741408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f216be00640 1 -- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2174142440 msgr2=0x7f217415dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f216be00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2174142440 0x7f217415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f216be00640 1 -- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21740707f0 msgr2=0x7f21741408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f216be00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21740707f0 0x7f21741408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f216be00640 1 -- 172.21.15.62:0/3180264682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217415c8d0 con 0x7f21740cfed0 2024-04-25T09:17:14.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f216be00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740cfed0 0x7f217413efb0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f215c00c7f0 tx=0x7f215c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:14.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f2170c00640 1 -- 172.21.15.62:0/3180264682 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c015070 con 0x7f21740cfed0 2024-04-25T09:17:14.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f2170c00640 1 -- 172.21.15.62:0/3180264682 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c004240 con 0x7f21740cfed0 2024-04-25T09:17:14.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f2170c00640 1 -- 172.21.15.62:0/3180264682 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c007620 con 0x7f21740cfed0 2024-04-25T09:17:14.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.433+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217415cbb0 con 0x7f21740cfed0 2024-04-25T09:17:14.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.434+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217415d170 con 0x7f21740cfed0 2024-04-25T09:17:14.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.435+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2174072c70 con 0x7f21740cfed0 2024-04-25T09:17:14.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.436+0000 7f2170c00640 1 -- 172.21.15.62:0/3180264682 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f215c0079d0 con 0x7f21740cfed0 2024-04-25T09:17:14.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.437+0000 7f2170c00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f215006b4d0 0x7f215006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.437+0000 7f2170c00640 1 -- 172.21.15.62:0/3180264682 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f215c08a730 con 0x7f21740cfed0 2024-04-25T09:17:14.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.437+0000 7f2173e00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f215006b4d0 0x7f215006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.439+0000 7f2173e00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f215006b4d0 0x7f215006d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2164000f60 tx=0x7f2164009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:14.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.442+0000 7f2170c00640 1 -- 172.21.15.62:0/3180264682 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f215c0594b0 con 0x7f21740cfed0 2024-04-25T09:17:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.639+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f217406d950 con 0x7f21740cfed0 2024-04-25T09:17:14.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.643+0000 7f2170c00640 1 -- 172.21.15.62:0/3180264682 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f215c05de20 con 0x7f21740cfed0 2024-04-25T09:17:14.629 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:14.629 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":138,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:02.676192+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:16:00.695740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:16:01.838452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/2303077743":"2024-04-26T09:17:02.676113+0000","172.21.15.62:6832/2303077743":"2024-04-26T09:17:02.676113+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f215006b4d0 msgr2=0x7f215006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f215006b4d0 0x7f215006d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2164000f60 tx=0x7f2164009040 comp rx=0 tx=0).stop 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740cfed0 msgr2=0x7f217413efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740cfed0 0x7f217413efb0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f215c00c7f0 tx=0x7f215c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 shutdown_connections 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f215006b4d0 0x7f215006d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21740cfed0 0x7f217413efb0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2174142440 0x7f217415dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 --2- 172.21.15.62:0/3180264682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f21740707f0 0x7f21741408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.645+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 >> 172.21.15.62:0/3180264682 conn(0x7f21740668f0 msgr2=0x7f21740af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.646+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 shutdown_connections 2024-04-25T09:17:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.646+0000 7f217ac00640 1 -- 172.21.15.62:0/3180264682 wait complete. 2024-04-25T09:17:14.651 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:17:14.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:14.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.878+0000 7fbe2ac00640 1 Processor -- start 2024-04-25T09:17:14.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.878+0000 7fbe2ac00640 1 -- start start 2024-04-25T09:17:14.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe2ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240cf1d0 0x7fbe240ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe2ac00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2406e120 con 0x7fbe2406d530 2024-04-25T09:17:14.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe2ac00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2406db60 con 0x7fbe2415dd50 2024-04-25T09:17:14.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe2ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24162980 con 0x7fbe240cf1d0 2024-04-25T09:17:14.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe23400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbe2406d530 0x7fbe240ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33988/0 (socket says 172.21.15.62:33988) 2024-04-25T09:17:14.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe23400640 1 -- 172.21.15.62:0/1839311368 learned_addr learned my addr 172.21.15.62:0/1839311368 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152572796 0 0) 0x7fbe2406e120 con 0x7fbe2406d530 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe08003520 con 0x7fbe2406d530 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.880+0000 7fbe28c00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240cf1d0 0x7fbe240ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463844649 0 0) 0x7fbe08003520 con 0x7fbe2406d530 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240cf1d0 msgr2=0x7fbe240ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240cf1d0 0x7fbe240ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 >> v1:172.21.15.121:6789/0 conn(0x7fbe2415dd50 legacy=0x7fbe24162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2406de40 con 0x7fbe2406d530 2024-04-25T09:17:14.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fbe10001670 con 0x7fbe2406d530 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 >> v1:172.21.15.62:6789/0 conn(0x7fbe2406d530 legacy=0x7fbe240ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240cf1d0 0x7fbe080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004060 0x7fbe08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe08004d40 0x7fbe080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24162980 con 0x7fbe08004d40 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0800bca0 con 0x7fbe08004060 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0800bea0 con 0x7fbe240cf1d0 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.881+0000 7fbe28c00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe08004d40 0x7fbe080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.882+0000 7fbe23400640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004060 0x7fbe08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.882+0000 7fbe23e00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240cf1d0 0x7fbe080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.882+0000 7fbe28c00640 1 -- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240cf1d0 msgr2=0x7fbe080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.882+0000 7fbe28c00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240cf1d0 0x7fbe080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.882+0000 7fbe28c00640 1 -- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004060 msgr2=0x7fbe08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.882+0000 7fbe28c00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004060 0x7fbe08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.882+0000 7fbe28c00640 1 -- 172.21.15.62:0/1839311368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08003520 con 0x7fbe08004d40 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.883+0000 7fbe28c00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe08004d40 0x7fbe080051c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c007360 tx=0x7fbe08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bf74cd6e1fb178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.883+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c009be0 con 0x7fbe08004d40 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.883+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c036600 con 0x7fbe08004d40 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.883+0000 7fbe22a00640 1 -- 172.21.15.62:0/1839311368 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c0369d0 con 0x7fbe08004d40 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.884+0000 7fbe2ac00640 1 -- 172.21.15.62:0/1839311368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe08004d40 msgr2=0x7fbe080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.884+0000 7fbe2ac00640 1 --2- 172.21.15.62:0/1839311368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe08004d40 0x7fbe080051c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c007360 tx=0x7fbe08005830 comp rx=0 tx=0).stop 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.884+0000 7fbe28c00640 1 -- 172.21.15.62:0/1839311368 reap_dead start 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.884+0000 7fbe2ac00640 1 -- 172.21.15.62:0/1839311368 shutdown_connections 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.884+0000 7fbe2ac00640 1 -- 172.21.15.62:0/1839311368 >> 172.21.15.62:0/1839311368 conn(0x7fbe240668f0 msgr2=0x7fbe24066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.884+0000 7fbe2ac00640 1 -- 172.21.15.62:0/1839311368 shutdown_connections 2024-04-25T09:17:14.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.884+0000 7fbe2ac00640 1 -- 172.21.15.62:0/1839311368 wait complete. 2024-04-25T09:17:14.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.886+0000 7fbe2ac00640 1 Processor -- start 2024-04-25T09:17:14.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.886+0000 7fbe2ac00640 1 -- start start 2024-04-25T09:17:14.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.887+0000 7fbe2ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004d40 0x7fbe2415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.887+0000 7fbe2ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe240b1fb0 0x7fbe240c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.887+0000 7fbe2ac00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240c4bc0 0x7fbe241f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.887+0000 7fbe2ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2415e130 con 0x7fbe240b1fb0 2024-04-25T09:17:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.887+0000 7fbe2ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24161d50 con 0x7fbe08004d40 2024-04-25T09:17:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.887+0000 7fbe2ac00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2415e2b0 con 0x7fbe240c4bc0 2024-04-25T09:17:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe240b1fb0 0x7fbe240c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe240b1fb0 0x7fbe240c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44654/0 (socket says 172.21.15.62:44654) 2024-04-25T09:17:14.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 -- 172.21.15.62:0/3222401215 learned_addr learned my addr 172.21.15.62:0/3222401215 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:14.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23e00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004d40 0x7fbe2415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe28c00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240c4bc0 0x7fbe241f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 -- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240c4bc0 msgr2=0x7fbe241f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240c4bc0 0x7fbe241f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 -- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004d40 msgr2=0x7fbe2415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004d40 0x7fbe2415a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 -- 172.21.15.62:0/3222401215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe240b0120 con 0x7fbe240b1fb0 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.888+0000 7fbe23400640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe240b1fb0 0x7fbe240c4680 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000e830 tx=0x7fbe1000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.889+0000 7fbe1be00640 1 -- 172.21.15.62:0/3222401215 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000c870 con 0x7fbe240b1fb0 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.889+0000 7fbe1be00640 1 -- 172.21.15.62:0/3222401215 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe10018070 con 0x7fbe240b1fb0 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.889+0000 7fbe1be00640 1 -- 172.21.15.62:0/3222401215 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000cc70 con 0x7fbe240b1fb0 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.889+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe240b0400 con 0x7fbe240b1fb0 2024-04-25T09:17:14.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.889+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe240b09c0 con 0x7fbe240b1fb0 2024-04-25T09:17:14.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.890+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe24072c70 con 0x7fbe240b1fb0 2024-04-25T09:17:14.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.891+0000 7fbe1be00640 1 -- 172.21.15.62:0/3222401215 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe1001d460 con 0x7fbe240b1fb0 2024-04-25T09:17:14.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.892+0000 7fbe1be00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe0406b4d0 0x7fbe0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:14.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.892+0000 7fbe1be00640 1 -- 172.21.15.62:0/3222401215 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbe1008a270 con 0x7fbe240b1fb0 2024-04-25T09:17:14.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.893+0000 7fbe23e00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe0406b4d0 0x7fbe0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:14.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.894+0000 7fbe23e00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe0406b4d0 0x7fbe0406d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbe14007fd0 tx=0x7fbe14009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:14.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:14.896+0000 7fbe1be00640 1 -- 172.21.15.62:0/3222401215 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe10058ff0 con 0x7fbe240b1fb0 2024-04-25T09:17:15.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.133+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe2404f000 con 0x7fbe240b1fb0 2024-04-25T09:17:15.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.134+0000 7fbe1be00640 1 -- 172.21.15.62:0/3222401215 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fbe100588f0 con 0x7fbe240b1fb0 2024-04-25T09:17:15.120 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:15.121 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:17:04.730865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6814":{"gid":6814,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":20,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:17:15.121 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 90 2024-04-25T09:17:15.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe0406b4d0 msgr2=0x7fbe0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe0406b4d0 0x7fbe0406d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbe14007fd0 tx=0x7fbe14009040 comp rx=0 tx=0).stop 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe240b1fb0 msgr2=0x7fbe240c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe240b1fb0 0x7fbe240c4680 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000e830 tx=0x7fbe1000ecf0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 shutdown_connections 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe0406b4d0 0x7fbe0406d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe240b1fb0 0x7fbe240c4680 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe240c4bc0 0x7fbe241f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 --2- 172.21.15.62:0/3222401215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08004d40 0x7fbe2415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 >> 172.21.15.62:0/3222401215 conn(0x7fbe240668f0 msgr2=0x7fbe240cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 shutdown_connections 2024-04-25T09:17:15.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.137+0000 7fbe2ac00640 1 -- 172.21.15.62:0/3222401215 wait complete. 2024-04-25T09:17:15.142 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.349+0000 7fbd99c00640 1 Processor -- start 2024-04-25T09:17:15.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.349+0000 7fbd99c00640 1 -- start start 2024-04-25T09:17:15.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.351+0000 7fbd99c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbd940b6a90 0x7fbd940d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.351+0000 7fbd99c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd940d3110 con 0x7fbd940b56f0 2024-04-25T09:17:15.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.351+0000 7fbd99c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd940d33f0 con 0x7fbd940b27a0 2024-04-25T09:17:15.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.351+0000 7fbd99c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd940cedc0 con 0x7fbd940b6a90 2024-04-25T09:17:15.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.351+0000 7fbd93e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbd940b56f0 0x7fbd94160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:33990/0 (socket says 172.21.15.62:33990) 2024-04-25T09:17:15.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.351+0000 7fbd93e00640 1 -- 172.21.15.62:0/695786672 learned_addr learned my addr 172.21.15.62:0/695786672 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd8b400640 1 --2- 172.21.15.62:0/695786672 >> v2:172.21.15.121:3300/0 conn(0x7fbd940b6a90 0x7fbd940d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505118989 0 0) 0x7fbd940d3110 con 0x7fbd940b56f0 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd78003520 con 0x7fbd940b56f0 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782092395 0 0) 0x7fbd78003520 con 0x7fbd940b56f0 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 >> v2:172.21.15.121:3300/0 conn(0x7fbd940b6a90 msgr2=0x7fbd940d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 --2- 172.21.15.62:0/695786672 >> v2:172.21.15.121:3300/0 conn(0x7fbd940b6a90 0x7fbd940d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 >> v1:172.21.15.121:6789/0 conn(0x7fbd940b27a0 legacy=0x7fbd940d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940d36d0 con 0x7fbd940b56f0 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fbd840019f0 con 0x7fbd940b56f0 2024-04-25T09:17:15.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 >> v1:172.21.15.62:6789/0 conn(0x7fbd940b56f0 legacy=0x7fbd94160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd78003a00 0x7fbd78003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd78004800 0x7fbd78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940cedc0 con 0x7fbd78005210 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7800c160 con 0x7fbd78003a00 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.352+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7800c360 con 0x7fbd78004800 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.353+0000 7fbd99200640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.353+0000 7fbd99200640 1 -- 172.21.15.62:0/695786672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd78004800 msgr2=0x7fbd78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.353+0000 7fbd99200640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd78004800 0x7fbd78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.353+0000 7fbd99200640 1 -- 172.21.15.62:0/695786672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd78003a00 msgr2=0x7fbd78003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.353+0000 7fbd99200640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd78003a00 0x7fbd78003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.353+0000 7fbd99200640 1 -- 172.21.15.62:0/695786672 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd78003520 con 0x7fbd78005210 2024-04-25T09:17:15.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.354+0000 7fbd99200640 1 -- 172.21.15.62:0/695786672 reap_dead start 2024-04-25T09:17:15.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.354+0000 7fbd99200640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd7800b9c0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c001720 tx=0x7fbd78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63dc52789c1a02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.354+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c001370 con 0x7fbd78005210 2024-04-25T09:17:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.354+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c0052d0 con 0x7fbd78005210 2024-04-25T09:17:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.354+0000 7fbd93400640 1 -- 172.21.15.62:0/695786672 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c02abc0 con 0x7fbd78005210 2024-04-25T09:17:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.355+0000 7fbd99c00640 1 -- 172.21.15.62:0/695786672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 msgr2=0x7fbd7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.355+0000 7fbd99c00640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd7800b9c0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c001720 tx=0x7fbd78005830 comp rx=0 tx=0).stop 2024-04-25T09:17:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.355+0000 7fbd99c00640 1 -- 172.21.15.62:0/695786672 shutdown_connections 2024-04-25T09:17:15.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.355+0000 7fbd99c00640 1 --2- 172.21.15.62:0/695786672 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd7800b9c0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.355+0000 7fbd99c00640 1 -- 172.21.15.62:0/695786672 >> 172.21.15.62:0/695786672 conn(0x7fbd94066d10 msgr2=0x7fbd94067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.355+0000 7fbd99c00640 1 -- 172.21.15.62:0/695786672 shutdown_connections 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.355+0000 7fbd99c00640 1 -- 172.21.15.62:0/695786672 wait complete. 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.357+0000 7fbd99c00640 1 Processor -- start 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.357+0000 7fbd99c00640 1 -- start start 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.358+0000 7fbd99c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd9415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.358+0000 7fbd99c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd941596a0 0x7fbd9414be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.358+0000 7fbd99c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd94157dd0 0x7fbd940c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.358+0000 7fbd99c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b2ee0 con 0x7fbd78005210 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.358+0000 7fbd99c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b2d60 con 0x7fbd941596a0 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.358+0000 7fbd99c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b5ad0 con 0x7fbd94157dd0 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.358+0000 7fbd93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd9415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd93e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd9415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44686/0 (socket says 172.21.15.62:44686) 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd93e00640 1 -- 172.21.15.62:0/3693595820 learned_addr learned my addr 172.21.15.62:0/3693595820 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:15.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd99200640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd94157dd0 0x7fbd940c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd8b400640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd941596a0 0x7fbd9414be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd93e00640 1 -- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd94157dd0 msgr2=0x7fbd940c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd93e00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd94157dd0 0x7fbd940c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd93e00640 1 -- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd941596a0 msgr2=0x7fbd9414be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd93e00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd941596a0 0x7fbd9414be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd93e00640 1 -- 172.21.15.62:0/3693595820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94158600 con 0x7fbd78005210 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.359+0000 7fbd93e00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd9415a930 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400ca70 tx=0x7fbd8400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.360+0000 7fbd90c00640 1 -- 172.21.15.62:0/3693595820 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84018070 con 0x7fbd78005210 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.360+0000 7fbd90c00640 1 -- 172.21.15.62:0/3693595820 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8400a760 con 0x7fbd78005210 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.360+0000 7fbd90c00640 1 -- 172.21.15.62:0/3693595820 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400ab50 con 0x7fbd78005210 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.360+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd941588e0 con 0x7fbd78005210 2024-04-25T09:17:15.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.360+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd94158ea0 con 0x7fbd78005210 2024-04-25T09:17:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.361+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9404ef20 con 0x7fbd78005210 2024-04-25T09:17:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.362+0000 7fbd90c00640 1 -- 172.21.15.62:0/3693595820 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd8400acf0 con 0x7fbd78005210 2024-04-25T09:17:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.363+0000 7fbd90c00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7006b4d0 0x7fbd7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.363+0000 7fbd90c00640 1 -- 172.21.15.62:0/3693595820 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbd84089b10 con 0x7fbd78005210 2024-04-25T09:17:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.364+0000 7fbd8b400640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7006b4d0 0x7fbd7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.365+0000 7fbd8b400640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7006b4d0 0x7fbd7006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006a90 tx=0x7fbd7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:15.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.367+0000 7fbd90c00640 1 -- 172.21.15.62:0/3693595820 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd84058890 con 0x7fbd78005210 2024-04-25T09:17:15.551 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:17:15.559 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:17:15.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.614+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd94159a80 con 0x7fbd78005210 2024-04-25T09:17:15.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.615+0000 7fbd90c00640 1 -- 172.21.15.62:0/3693595820 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fbd84001a10 con 0x7fbd78005210 2024-04-25T09:17:15.601 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:15.601 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:16:02.818081+0000","modified":"2024-04-25T09:17:04.730865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6814":{"gid":6814,"name":"b","rank":0,"incarnation":87,"state":"up:active","state_seq":20,"addr":"172.21.15.121:6837/419350964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":419350964},{"type":"v1","addr":"172.21.15.121:6837","nonce":419350964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:17:15.602 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 90 2024-04-25T09:17:15.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7006b4d0 msgr2=0x7fbd7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7006b4d0 0x7fbd7006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006a90 tx=0x7fbd7c008040 comp rx=0 tx=0).stop 2024-04-25T09:17:15.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 msgr2=0x7fbd9415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd9415a930 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400ca70 tx=0x7fbd8400cf30 comp rx=0 tx=0).stop 2024-04-25T09:17:15.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 shutdown_connections 2024-04-25T09:17:15.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbd7006b4d0 0x7fbd7006d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbd94157dd0 0x7fbd940c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd941596a0 0x7fbd9414be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 --2- 172.21.15.62:0/3693595820 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd78005210 0x7fbd9415a930 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 >> 172.21.15.62:0/3693595820 conn(0x7fbd94066d10 msgr2=0x7fbd9415e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:15.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 shutdown_connections 2024-04-25T09:17:15.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.618+0000 7fbd99c00640 1 -- 172.21.15.62:0/3693595820 wait complete. 2024-04-25T09:17:15.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:15.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.890+0000 7f1d13200640 1 Processor -- start 2024-04-25T09:17:15.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.890+0000 7f1d13200640 1 -- start start 2024-04-25T09:17:15.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.892+0000 7f1d13200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1d14144c10 0x7f1d1415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.892+0000 7f1d13200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 0x7f1d1406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.892+0000 7f1d13200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1415faa0 con 0x7f1d1406d730 2024-04-25T09:17:15.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.892+0000 7f1d13200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d14144ff0 con 0x7f1d14144c10 2024-04-25T09:17:15.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.892+0000 7f1d13200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14160340 con 0x7f1d140cee40 2024-04-25T09:17:15.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.892+0000 7f1d12800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1d1406d730 0x7f1d140b2da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34004/0 (socket says 172.21.15.62:34004) 2024-04-25T09:17:15.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.892+0000 7f1d12800640 1 -- 172.21.15.62:0/1307673307 learned_addr learned my addr 172.21.15.62:0/1307673307 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:15.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d09400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 0x7f1d1406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11e00640 1 --2- 172.21.15.62:0/1307673307 >> v2:172.21.15.121:3300/0 conn(0x7f1d14144c10 0x7f1d1415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019965938 0 0) 0x7f1d1415faa0 con 0x7f1d1406d730 2024-04-25T09:17:15.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf8003520 con 0x7f1d1406d730 2024-04-25T09:17:15.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1282693694 0 0) 0x7f1cf8003520 con 0x7f1d1406d730 2024-04-25T09:17:15.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 msgr2=0x7f1d1406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:15.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 0x7f1d1406d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 >> v2:172.21.15.121:3300/0 conn(0x7f1d14144c10 msgr2=0x7f1d1415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:15.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 --2- 172.21.15.62:0/1307673307 >> v2:172.21.15.121:3300/0 conn(0x7f1d14144c10 0x7f1d1415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1415fd80 con 0x7f1d1406d730 2024-04-25T09:17:15.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1d0c0019f0 con 0x7f1d1406d730 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 >> v1:172.21.15.62:6789/0 conn(0x7f1d1406d730 legacy=0x7f1d140b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf80039b0 0x7f1cf8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8004800 0x7f1cf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 0x7f1cf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14144ff0 con 0x7f1cf8004800 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14160340 con 0x7f1cf80039b0 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.893+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf800b9f0 con 0x7f1d140cee40 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d09400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8004800 0x7f1cf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d12800640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 0x7f1cf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d11e00640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf80039b0 0x7f1cf8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d09400640 1 -- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 msgr2=0x7f1cf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d09400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 0x7f1cf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d09400640 1 -- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf80039b0 msgr2=0x7f1cf8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:15.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d09400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cf80039b0 0x7f1cf8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d09400640 1 -- 172.21.15.62:0/1307673307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8003520 con 0x7f1cf8004800 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.894+0000 7f1d12800640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1d140cee40 0x7f1cf80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.895+0000 7f1d09400640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8004800 0x7f1cf8004c80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009ac0 tx=0x7f1cf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a6133809d8822d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.895+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc036070 con 0x7f1cf8004800 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.895+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc031400 con 0x7f1cf8004800 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.895+0000 7f1d11400640 1 -- 172.21.15.62:0/1307673307 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc009690 con 0x7f1cf8004800 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.896+0000 7f1d13200640 1 -- 172.21.15.62:0/1307673307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8004800 msgr2=0x7f1cf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.896+0000 7f1d13200640 1 --2- 172.21.15.62:0/1307673307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8004800 0x7f1cf8004c80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009ac0 tx=0x7f1cf8005830 comp rx=0 tx=0).stop 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.896+0000 7f1d12800640 1 -- 172.21.15.62:0/1307673307 reap_dead start 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.896+0000 7f1d13200640 1 -- 172.21.15.62:0/1307673307 shutdown_connections 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.896+0000 7f1d13200640 1 -- 172.21.15.62:0/1307673307 >> 172.21.15.62:0/1307673307 conn(0x7f1d140b14d0 msgr2=0x7f1d140b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.896+0000 7f1d13200640 1 -- 172.21.15.62:0/1307673307 shutdown_connections 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.896+0000 7f1d13200640 1 -- 172.21.15.62:0/1307673307 wait complete. 2024-04-25T09:17:15.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.898+0000 7f1d13200640 1 Processor -- start 2024-04-25T09:17:15.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.898+0000 7f1d13200640 1 -- start start 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.899+0000 7f1d13200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf8004800 0x7f1d1415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.899+0000 7f1d13200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1413b820 0x7f1d14138940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.899+0000 7f1d13200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d14071500 0x7f1d14071980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.899+0000 7f1d13200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140cf2a0 con 0x7f1d14071500 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.899+0000 7f1d13200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140b41c0 con 0x7f1d1413b820 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.899+0000 7f1d13200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140b5660 con 0x7f1cf8004800 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d14071500 0x7f1d14071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d14071500 0x7f1d14071980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44694/0 (socket says 172.21.15.62:44694) 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 -- 172.21.15.62:0/418251725 learned_addr learned my addr 172.21.15.62:0/418251725 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d09400640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1413b820 0x7f1d14138940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 -- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf8004800 msgr2=0x7f1d1415aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d11e00640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf8004800 0x7f1d1415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf8004800 0x7f1d1415aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 -- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1413b820 msgr2=0x7f1d14138940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1413b820 0x7f1d14138940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 -- 172.21.15.62:0/418251725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1413bca0 con 0x7f1d14071500 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d09400640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1413b820 0x7f1d14138940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d11e00640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf8004800 0x7f1d1415aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.900+0000 7f1d12800640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d14071500 0x7f1d14071980 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f1d1415f390 tx=0x7f1d0c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.901+0000 7f1d0aa00640 1 -- 172.21.15.62:0/418251725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00de70 con 0x7f1d14071500 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.901+0000 7f1d0aa00640 1 -- 172.21.15.62:0/418251725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c004b60 con 0x7f1d14071500 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.901+0000 7f1d0aa00640 1 -- 172.21.15.62:0/418251725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c007070 con 0x7f1d14071500 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.901+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d14139090 con 0x7f1d14071500 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.901+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d14139650 con 0x7f1d14071500 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.902+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1404ef20 con 0x7f1d14071500 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.903+0000 7f1d0aa00640 1 -- 172.21.15.62:0/418251725 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d0c01b020 con 0x7f1d14071500 2024-04-25T09:17:15.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.904+0000 7f1d0aa00640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1cf006b500 0x7f1cf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:15.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.904+0000 7f1d0aa00640 1 -- 172.21.15.62:0/418251725 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1d0c08a440 con 0x7f1d14071500 2024-04-25T09:17:15.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.905+0000 7f1d11e00640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1cf006b500 0x7f1cf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:15.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.906+0000 7f1d11e00640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1cf006b500 0x7f1cf006d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1d1415ed30 tx=0x7f1d04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:15.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:15.908+0000 7f1d0aa00640 1 -- 172.21.15.62:0/418251725 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d0c058b70 con 0x7f1d14071500 2024-04-25T09:17:16.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.113+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d140b22b0 con 0x7f1d14071500 2024-04-25T09:17:16.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.114+0000 7f1d0aa00640 1 -- 172.21.15.62:0/418251725 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f1d0c05cc10 con 0x7f1d14071500 2024-04-25T09:17:16.100 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:16.101 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":138,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:02.676192+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:16:00.695740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:16:01.838452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/2303077743":"2024-04-26T09:17:02.676113+0000","172.21.15.62:6832/2303077743":"2024-04-26T09:17:02.676113+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1cf006b500 msgr2=0x7f1cf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1cf006b500 0x7f1cf006d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1d1415ed30 tx=0x7f1d04008040 comp rx=0 tx=0).stop 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d14071500 msgr2=0x7f1d14071980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d14071500 0x7f1d14071980 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f1d1415f390 tx=0x7f1d0c00b4a0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 shutdown_connections 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f1cf006b500 0x7f1cf006d9c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d14071500 0x7f1d14071980 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1413b820 0x7f1d14138940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 --2- 172.21.15.62:0/418251725 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f1cf8004800 0x7f1d1415aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 >> 172.21.15.62:0/418251725 conn(0x7f1d140b14d0 msgr2=0x7f1d140c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.117+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 shutdown_connections 2024-04-25T09:17:16.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.118+0000 7f1d13200640 1 -- 172.21.15.62:0/418251725 wait complete. 2024-04-25T09:17:16.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:17:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.349+0000 7fc0d3200640 1 Processor -- start 2024-04-25T09:17:16.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.350+0000 7fc0d3200640 1 -- start start 2024-04-25T09:17:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.351+0000 7fc0d3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc0d40aeb30 0x7fc0d406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.351+0000 7fc0d3200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc0d406d3f0 0x7fc0d40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.351+0000 7fc0d3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d406d8a0 con 0x7fc0d40aef10 2024-04-25T09:17:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d3200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d406db60 con 0x7fc0d406d3f0 2024-04-25T09:17:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d406de40 con 0x7fc0d40aeb30 2024-04-25T09:17:16.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc0d406d3f0 0x7fc0d40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc0d406d3f0 0x7fc0d40b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44698/0 (socket says 172.21.15.62:44698) 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d1400640 1 -- 172.21.15.62:0/719889824 learned_addr learned my addr 172.21.15.62:0/719889824 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d2800640 1 --1- 172.21.15.62:0/719889824 >> v1:172.21.15.62:6789/0 conn(0x7fc0d40aef10 0x7fc0d40713d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34020/0 (socket says 172.21.15.62:34020) 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d1e00640 1 --2- 172.21.15.62:0/719889824 >> v2:172.21.15.121:3300/0 conn(0x7fc0d40aeb30 0x7fc0d406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d1400640 1 -- 172.21.15.62:0/719889824 >> v2:172.21.15.121:3300/0 conn(0x7fc0d40aeb30 msgr2=0x7fc0d406ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d1400640 1 --2- 172.21.15.62:0/719889824 >> v2:172.21.15.121:3300/0 conn(0x7fc0d40aeb30 0x7fc0d406ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d1400640 1 -- 172.21.15.62:0/719889824 >> v1:172.21.15.62:6789/0 conn(0x7fc0d40aef10 legacy=0x7fc0d40713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d1400640 1 -- 172.21.15.62:0/719889824 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d415b150 con 0x7fc0d406d3f0 2024-04-25T09:17:16.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.352+0000 7fc0d0a00640 1 -- 172.21.15.62:0/719889824 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550120243 0 0) 0x7fc0d406d8a0 con 0x7fc0d40aef10 2024-04-25T09:17:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.354+0000 7fc0d1400640 1 --2- 172.21.15.62:0/719889824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d406d3f0 0x7fc0d40b3190 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40677a0 tx=0x7fc0bc02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74500c4a156f2af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.354+0000 7fc0d0a00640 1 -- 172.21.15.62:0/719889824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc033640 con 0x7fc0d406d3f0 2024-04-25T09:17:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.354+0000 7fc0d0a00640 1 -- 172.21.15.62:0/719889824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc038070 con 0x7fc0d406d3f0 2024-04-25T09:17:16.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.354+0000 7fc0d0a00640 1 -- 172.21.15.62:0/719889824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc033a00 con 0x7fc0d406d3f0 2024-04-25T09:17:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.354+0000 7fc0d3200640 1 -- 172.21.15.62:0/719889824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d406d3f0 msgr2=0x7fc0d40b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.355+0000 7fc0d3200640 1 --2- 172.21.15.62:0/719889824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d406d3f0 0x7fc0d40b3190 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40677a0 tx=0x7fc0bc02c8b0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.355+0000 7fc0d3200640 1 -- 172.21.15.62:0/719889824 shutdown_connections 2024-04-25T09:17:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.355+0000 7fc0d3200640 1 --2- 172.21.15.62:0/719889824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d406d3f0 0x7fc0d40b3190 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.355+0000 7fc0d3200640 1 --2- 172.21.15.62:0/719889824 >> v2:172.21.15.121:3300/0 conn(0x7fc0d40aeb30 0x7fc0d406ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.355+0000 7fc0d3200640 1 -- 172.21.15.62:0/719889824 >> 172.21.15.62:0/719889824 conn(0x7fc0d40668f0 msgr2=0x7fc0d4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.355+0000 7fc0d3200640 1 -- 172.21.15.62:0/719889824 shutdown_connections 2024-04-25T09:17:16.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.355+0000 7fc0d3200640 1 -- 172.21.15.62:0/719889824 wait complete. 2024-04-25T09:17:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.356+0000 7fc0d3200640 1 Processor -- start 2024-04-25T09:17:16.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.357+0000 7fc0d3200640 1 -- start start 2024-04-25T09:17:16.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.357+0000 7fc0d3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d406d3f0 0x7fc0d40adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0d40aeb30 0x7fc0d40ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d40aef10 0x7fc0d41ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d415ae90 con 0x7fc0d40aef10 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d415ad10 con 0x7fc0d406d3f0 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d415c680 con 0x7fc0d40aeb30 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d40aef10 0x7fc0d41ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d40aef10 0x7fc0d41ef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44700/0 (socket says 172.21.15.62:44700) 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d2800640 1 -- 172.21.15.62:0/1305665471 learned_addr learned my addr 172.21.15.62:0/1305665471 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d1e00640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d406d3f0 0x7fc0d40adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d1400640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0d40aeb30 0x7fc0d40ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d2800640 1 -- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0d40aeb30 msgr2=0x7fc0d40ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d2800640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0d40aeb30 0x7fc0d40ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d2800640 1 -- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d406d3f0 msgr2=0x7fc0d40adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d2800640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d406d3f0 0x7fc0d40adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.358+0000 7fc0d2800640 1 -- 172.21.15.62:0/1305665471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d40aca10 con 0x7fc0d40aef10 2024-04-25T09:17:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.359+0000 7fc0d2800640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d40aef10 0x7fc0d41ef3d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00c700 tx=0x7fc0cc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.359+0000 7fc0ca000640 1 -- 172.21.15.62:0/1305665471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc018070 con 0x7fc0d40aef10 2024-04-25T09:17:16.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.359+0000 7fc0ca000640 1 -- 172.21.15.62:0/1305665471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0cc00ce10 con 0x7fc0d40aef10 2024-04-25T09:17:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.359+0000 7fc0ca000640 1 -- 172.21.15.62:0/1305665471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00a860 con 0x7fc0d40aef10 2024-04-25T09:17:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.359+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d40a96d0 con 0x7fc0d40aef10 2024-04-25T09:17:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.359+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d40a9c90 con 0x7fc0d40aef10 2024-04-25T09:17:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.361+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d415dd60 con 0x7fc0d40aef10 2024-04-25T09:17:16.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.362+0000 7fc0ca000640 1 -- 172.21.15.62:0/1305665471 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0cc00aa00 con 0x7fc0d40aef10 2024-04-25T09:17:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.362+0000 7fc0ca000640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc0b006b4d0 0x7fc0b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.363+0000 7fc0ca000640 1 -- 172.21.15.62:0/1305665471 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc0cc0897b0 con 0x7fc0d40aef10 2024-04-25T09:17:16.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.363+0000 7fc0d1e00640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc0b006b4d0 0x7fc0b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.365+0000 7fc0d1e00640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc0b006b4d0 0x7fc0b006d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4006fd0 tx=0x7fc0c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:16.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.367+0000 7fc0ca000640 1 -- 172.21.15.62:0/1305665471 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0d415dd60 con 0x7fc0d40aef10 2024-04-25T09:17:16.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.606+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc0d404ef20 con 0x7fc0d40aef10 2024-04-25T09:17:16.593 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:17:16.608+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:17:16.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.611+0000 7fc0ca000640 1 -- 172.21.15.62:0/1305665471 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc0cc058660 con 0x7fc0d40aef10 2024-04-25T09:17:16.597 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:17:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc0b006b4d0 msgr2=0x7fc0b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc0b006b4d0 0x7fc0b006d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4006fd0 tx=0x7fc0c4008040 comp rx=0 tx=0).stop 2024-04-25T09:17:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d40aef10 msgr2=0x7fc0d41ef3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d40aef10 0x7fc0d41ef3d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00c700 tx=0x7fc0cc00cbc0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 shutdown_connections 2024-04-25T09:17:16.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc0b006b4d0 0x7fc0b006d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d40aef10 0x7fc0d41ef3d0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc0d40aeb30 0x7fc0d40ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 --2- 172.21.15.62:0/1305665471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d406d3f0 0x7fc0d40adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 >> 172.21.15.62:0/1305665471 conn(0x7fc0d40668f0 msgr2=0x7fc0d4071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:16.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 shutdown_connections 2024-04-25T09:17:16.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.614+0000 7fc0d3200640 1 -- 172.21.15.62:0/1305665471 wait complete. 2024-04-25T09:17:16.622 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:17:16.720 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:17:16.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.846+0000 7fd9bee00640 1 Processor -- start 2024-04-25T09:17:16.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.846+0000 7fd9bee00640 1 -- start start 2024-04-25T09:17:16.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.847+0000 7fd9bee00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd9b80ceb50 0x7fd9b806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.847+0000 7fd9bee00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9b815e7c0 0x7fd9b80b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.847+0000 7fd9bee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b815ee30 con 0x7fd9b806d800 2024-04-25T09:17:16.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.847+0000 7fd9bee00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b81618b0 con 0x7fd9b815e7c0 2024-04-25T09:17:16.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.847+0000 7fd9bee00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b8161b90 con 0x7fd9b80ceb50 2024-04-25T09:17:16.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.848+0000 7fd9be400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9b815e7c0 0x7fd9b80b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.848+0000 7fd9be400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd9b815e7c0 0x7fd9b80b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44714/0 (socket says 172.21.15.62:44714) 2024-04-25T09:17:16.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.848+0000 7fd9be400640 1 -- 172.21.15.62:0/1575102673 learned_addr learned my addr 172.21.15.62:0/1575102673 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:16.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.848+0000 7fd9be400640 1 -- 172.21.15.62:0/1575102673 >> v2:172.21.15.121:3300/0 conn(0x7fd9b80ceb50 msgr2=0x7fd9b806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:16.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.848+0000 7fd9be400640 1 --2- 172.21.15.62:0/1575102673 >> v2:172.21.15.121:3300/0 conn(0x7fd9b80ceb50 0x7fd9b806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.848+0000 7fd9be400640 1 -- 172.21.15.62:0/1575102673 >> v1:172.21.15.121:6789/0 conn(0x7fd9b806d800 legacy=0x7fd9b815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:16.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.848+0000 7fd9be400640 1 -- 172.21.15.62:0/1575102673 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80cef30 con 0x7fd9b815e7c0 2024-04-25T09:17:16.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.849+0000 7fd9be400640 1 --2- 172.21.15.62:0/1575102673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b815e7c0 0x7fd9b80b4550 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80af200 tx=0x7fd9a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2f97a84c5f27d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:16.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9b7e00640 1 -- 172.21.15.62:0/1575102673 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8037070 con 0x7fd9b815e7c0 2024-04-25T09:17:16.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9b7e00640 1 -- 172.21.15.62:0/1575102673 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a8032440 con 0x7fd9b815e7c0 2024-04-25T09:17:16.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9b7e00640 1 -- 172.21.15.62:0/1575102673 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8032810 con 0x7fd9b815e7c0 2024-04-25T09:17:16.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9bee00640 1 -- 172.21.15.62:0/1575102673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b815e7c0 msgr2=0x7fd9b80b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9bee00640 1 --2- 172.21.15.62:0/1575102673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b815e7c0 0x7fd9b80b4550 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80af200 tx=0x7fd9a802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9bee00640 1 -- 172.21.15.62:0/1575102673 shutdown_connections 2024-04-25T09:17:16.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9bee00640 1 --2- 172.21.15.62:0/1575102673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b815e7c0 0x7fd9b80b4550 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9bee00640 1 --2- 172.21.15.62:0/1575102673 >> v2:172.21.15.121:3300/0 conn(0x7fd9b80ceb50 0x7fd9b806d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9bee00640 1 -- 172.21.15.62:0/1575102673 >> 172.21.15.62:0/1575102673 conn(0x7fd9b8066f20 msgr2=0x7fd9b80b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:16.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9bee00640 1 -- 172.21.15.62:0/1575102673 shutdown_connections 2024-04-25T09:17:16.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.850+0000 7fd9bee00640 1 -- 172.21.15.62:0/1575102673 wait complete. 2024-04-25T09:17:16.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.852+0000 7fd9bee00640 1 Processor -- start 2024-04-25T09:17:16.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.852+0000 7fd9bee00640 1 -- start start 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.853+0000 7fd9bee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b806d800 0x7fd9b8158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.853+0000 7fd9bee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b80ceb50 0x7fd9b81569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.853+0000 7fd9bee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9b815e7c0 0x7fd9b8157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.853+0000 7fd9bee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80b4a90 con 0x7fd9b80ceb50 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.853+0000 7fd9bee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b81625e0 con 0x7fd9b806d800 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.853+0000 7fd9bee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b80b4c10 con 0x7fd9b815e7c0 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b80ceb50 0x7fd9b81569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b80ceb50 0x7fd9b81569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44722/0 (socket says 172.21.15.62:44722) 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bd000640 1 -- 172.21.15.62:0/405425871 learned_addr learned my addr 172.21.15.62:0/405425871 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bda00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b806d800 0x7fd9b8158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bd000640 1 -- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9b815e7c0 msgr2=0x7fd9b8157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9be400640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9b815e7c0 0x7fd9b8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bd000640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9b815e7c0 0x7fd9b8157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bd000640 1 -- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b806d800 msgr2=0x7fd9b8158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bd000640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b806d800 0x7fd9b8158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bd000640 1 -- 172.21.15.62:0/405425871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b81e9630 con 0x7fd9b80ceb50 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.854+0000 7fd9bda00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b806d800 0x7fd9b8158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.855+0000 7fd9bd000640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b80ceb50 0x7fd9b81569a0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400c410 tx=0x7fd9a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.855+0000 7fd9b5600640 1 -- 172.21.15.62:0/405425871 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4017070 con 0x7fd9b80ceb50 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.855+0000 7fd9b5600640 1 -- 172.21.15.62:0/405425871 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a40035f0 con 0x7fd9b80ceb50 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.855+0000 7fd9b5600640 1 -- 172.21.15.62:0/405425871 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4003a20 con 0x7fd9b80ceb50 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.855+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b81453f0 con 0x7fd9b80ceb50 2024-04-25T09:17:16.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.855+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b81e8d90 con 0x7fd9b80ceb50 2024-04-25T09:17:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.856+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b806abe0 con 0x7fd9b80ceb50 2024-04-25T09:17:16.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.857+0000 7fd9b5600640 1 -- 172.21.15.62:0/405425871 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9a401a070 con 0x7fd9b80ceb50 2024-04-25T09:17:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.858+0000 7fd9b5600640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.858+0000 7fd9b5600640 1 -- 172.21.15.62:0/405425871 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fd9a40897b0 con 0x7fd9b80ceb50 2024-04-25T09:17:16.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.859+0000 7fd9bda00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:16.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.860+0000 7fd9bda00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd99806b4d0 0x7fd99806d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0007eb0 tx=0x7fd9b00073c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:16.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:16.862+0000 7fd9b5600640 1 -- 172.21.15.62:0/405425871 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9a4058570 con 0x7fd9b80ceb50 2024-04-25T09:17:17.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.106+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd9b804ef20 con 0x7fd9b80ceb50 2024-04-25T09:17:17.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.614+0000 7fd9b5600640 1 -- 172.21.15.62:0/405425871 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9a4057e70 con 0x7fd9b80ceb50 2024-04-25T09:17:17.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.617+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd99806b4d0 msgr2=0x7fd99806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:17.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.617+0000 7fd9bee00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd99806b4d0 0x7fd99806d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0007eb0 tx=0x7fd9b00073c0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b80ceb50 msgr2=0x7fd9b81569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:17.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b80ceb50 0x7fd9b81569a0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400c410 tx=0x7fd9a400c8d0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 shutdown_connections 2024-04-25T09:17:17.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd9b80ceb50 0x7fd9b81569a0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd9b815e7c0 0x7fd9b8157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 --2- 172.21.15.62:0/405425871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9b806d800 0x7fd9b8158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 >> 172.21.15.62:0/405425871 conn(0x7fd9b8066f20 msgr2=0x7fd9b8160210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:17.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 shutdown_connections 2024-04-25T09:17:17.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.618+0000 7fd9bee00640 1 -- 172.21.15.62:0/405425871 wait complete. 2024-04-25T09:17:17.627 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:17:17.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.860+0000 7fb6b3e00640 1 Processor -- start 2024-04-25T09:17:17.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.860+0000 7fb6b3e00640 1 -- start start 2024-04-25T09:17:17.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.862+0000 7fb6b3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 0x7fb6b40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:17.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.862+0000 7fb6b3e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b4162980 con 0x7fb6b40d2870 2024-04-25T09:17:17.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.862+0000 7fb6b3e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b40d1480 con 0x7fb6b4161650 2024-04-25T09:17:17.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.862+0000 7fb6b3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40d3900 con 0x7fb6b4161270 2024-04-25T09:17:17.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.862+0000 7fb6b3400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb6b40d2870 0x7fb6b4162270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34032/0 (socket says 172.21.15.62:34032) 2024-04-25T09:17:17.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.862+0000 7fb6b3400640 1 -- 172.21.15.62:0/1316985648 learned_addr learned my addr 172.21.15.62:0/1316985648 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:17.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.862+0000 7fb6b2a00640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 0x7fb6b40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:17.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269590746 0 0) 0x7fb6b4162980 con 0x7fb6b40d2870 2024-04-25T09:17:17.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb698003520 con 0x7fb6b40d2870 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 494674516 0 0) 0x7fb698003520 con 0x7fb6b40d2870 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 msgr2=0x7fb6b40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 0x7fb6b40b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 >> v1:172.21.15.121:6789/0 conn(0x7fb6b4161650 legacy=0x7fb6b4161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b40d11a0 con 0x7fb6b40d2870 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb6ac0019f0 con 0x7fb6b40d2870 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 >> v1:172.21.15.62:6789/0 conn(0x7fb6b40d2870 legacy=0x7fb6b4162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6980039e0 0x7fb698003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb698004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 0x7fb698005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40d3900 con 0x7fb698004800 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69800bb80 con 0x7fb6980039e0 2024-04-25T09:17:17.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.863+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69800bd80 con 0x7fb6b4161270 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b2000640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb698004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b2a00640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6980039e0 0x7fb698003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b3400640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 0x7fb698005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b2000640 1 -- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 msgr2=0x7fb698005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b2000640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 0x7fb698005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b2000640 1 -- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6980039e0 msgr2=0x7fb698003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b2000640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6980039e0 0x7fb698003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b2000640 1 -- 172.21.15.62:0/1316985648 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb698003520 con 0x7fb698004800 2024-04-25T09:17:17.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.864+0000 7fb6b3400640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b4161270 0x7fb698005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:17.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.865+0000 7fb6b2000640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb698004c80 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0001750 tx=0x7fb698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbc00adecb3fbb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:17.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.865+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0001370 con 0x7fb698004800 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.865+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a00052d0 con 0x7fb698004800 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.865+0000 7fb6b1600640 1 -- 172.21.15.62:0/1316985648 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a002abe0 con 0x7fb698004800 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.866+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1316985648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 msgr2=0x7fb698004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.866+0000 7fb6b3e00640 1 --2- 172.21.15.62:0/1316985648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb698004c80 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0001750 tx=0x7fb698005830 comp rx=0 tx=0).stop 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.866+0000 7fb6b3400640 1 -- 172.21.15.62:0/1316985648 reap_dead start 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.866+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1316985648 shutdown_connections 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.866+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1316985648 >> 172.21.15.62:0/1316985648 conn(0x7fb6b4066d10 msgr2=0x7fb6b4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.866+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1316985648 shutdown_connections 2024-04-25T09:17:17.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.866+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1316985648 wait complete. 2024-04-25T09:17:17.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.868+0000 7fb6b3e00640 1 Processor -- start 2024-04-25T09:17:17.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.868+0000 7fb6b3e00640 1 -- start start 2024-04-25T09:17:17.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb6b41f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:17.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b414a870 0x7fb6b40b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:17.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b40b3a80 0x7fb6b41e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:17.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40d3590 con 0x7fb698004800 2024-04-25T09:17:17.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40cf160 con 0x7fb6b414a870 2024-04-25T09:17:17.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40b2ed0 con 0x7fb6b40b3a80 2024-04-25T09:17:17.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb6b41f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:17.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb6b41f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44740/0 (socket says 172.21.15.62:44740) 2024-04-25T09:17:17.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.869+0000 7fb6b2a00640 1 -- 172.21.15.62:0/1323834884 learned_addr learned my addr 172.21.15.62:0/1323834884 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:17.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.870+0000 7fb6b2000640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b414a870 0x7fb6b40b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:17.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.870+0000 7fb6b3400640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b40b3a80 0x7fb6b41e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:17.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.870+0000 7fb6b2a00640 1 -- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b40b3a80 msgr2=0x7fb6b41e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:17.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.870+0000 7fb6b2a00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b40b3a80 0x7fb6b41e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.870+0000 7fb6b2a00640 1 -- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b414a870 msgr2=0x7fb6b40b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.870+0000 7fb6b2a00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b414a870 0x7fb6b40b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.870+0000 7fb6b2a00640 1 -- 172.21.15.62:0/1323834884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b41f3d00 con 0x7fb698004800 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.870+0000 7fb6b2a00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb6b41f37c0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4011600 tx=0x7fb6a4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.871+0000 7fb6aaa00640 1 -- 172.21.15.62:0/1323834884 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400f8c0 con 0x7fb698004800 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.871+0000 7fb6aaa00640 1 -- 172.21.15.62:0/1323834884 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a400fa60 con 0x7fb698004800 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.871+0000 7fb6aaa00640 1 -- 172.21.15.62:0/1323834884 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400fe10 con 0x7fb698004800 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.871+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b41e9680 con 0x7fb698004800 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.871+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b40cb480 con 0x7fb698004800 2024-04-25T09:17:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.872+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b4072910 con 0x7fb698004800 2024-04-25T09:17:17.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.873+0000 7fb6aaa00640 1 -- 172.21.15.62:0/1323834884 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6a4015460 con 0x7fb698004800 2024-04-25T09:17:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.874+0000 7fb6aaa00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb69406b550 0x7fb69406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.874+0000 7fb6aaa00640 1 -- 172.21.15.62:0/1323834884 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fb6a408ab40 con 0x7fb698004800 2024-04-25T09:17:17.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.875+0000 7fb6b2000640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb69406b550 0x7fb69406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:17.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.876+0000 7fb6b2000640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb69406b550 0x7fb69406da10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0032000 tx=0x7fb6a002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:17.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:17.879+0000 7fb6aaa00640 1 -- 172.21.15.62:0/1323834884 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6a4019030 con 0x7fb698004800 2024-04-25T09:17:18.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.080+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb6b40750e0 con 0x7fb698004800 2024-04-25T09:17:18.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.793+0000 7fb6aaa00640 1 -- 172.21.15.62:0/1323834884 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb6a408a020 con 0x7fb698004800 2024-04-25T09:17:18.780 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:17:18.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb69406b550 msgr2=0x7fb69406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:18.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb69406b550 0x7fb69406da10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0032000 tx=0x7fb6a002e000 comp rx=0 tx=0).stop 2024-04-25T09:17:18.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 msgr2=0x7fb6b41f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:18.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb6b41f37c0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4011600 tx=0x7fb6a4011ac0 comp rx=0 tx=0).stop 2024-04-25T09:17:18.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 shutdown_connections 2024-04-25T09:17:18.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb69406b550 0x7fb69406da10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:18.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb6b40b3a80 0x7fb6b41e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:18.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6b414a870 0x7fb6b40b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:18.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 --2- 172.21.15.62:0/1323834884 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb698004800 0x7fb6b41f37c0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:18.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 >> 172.21.15.62:0/1323834884 conn(0x7fb6b4066d10 msgr2=0x7fb6b40b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:18.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 shutdown_connections 2024-04-25T09:17:18.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:18.797+0000 7fb6b3e00640 1 -- 172.21.15.62:0/1323834884 wait complete. 2024-04-25T09:17:18.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:17:19.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.039+0000 7f779d400640 1 Processor -- start 2024-04-25T09:17:19.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.039+0000 7f779d400640 1 -- start start 2024-04-25T09:17:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.041+0000 7f779d400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7798145140 0x7f77980b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:19.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.042+0000 7f779d400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f779806c440 0x7f779806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.042+0000 7f779d400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779806ce10 con 0x7f77980b4730 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.042+0000 7f779d400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77980b3d60 con 0x7f779806c440 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.042+0000 7f779d400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779815c9a0 con 0x7f7798145140 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.042+0000 7f779ca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f779806c440 0x7f779806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.042+0000 7f779ca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f779806c440 0x7f779806c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44746/0 (socket says 172.21.15.62:44746) 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.042+0000 7f779ca00640 1 -- 172.21.15.62:0/1656021289 learned_addr learned my addr 172.21.15.62:0/1656021289 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.042+0000 7f7797e00640 1 --2- 172.21.15.62:0/1656021289 >> v2:172.21.15.121:3300/0 conn(0x7f7798145140 0x7f77980b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.043+0000 7f779ca00640 1 -- 172.21.15.62:0/1656021289 >> v2:172.21.15.121:3300/0 conn(0x7f7798145140 msgr2=0x7f77980b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.043+0000 7f779ca00640 1 --2- 172.21.15.62:0/1656021289 >> v2:172.21.15.121:3300/0 conn(0x7f7798145140 0x7f77980b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.043+0000 7f779ca00640 1 -- 172.21.15.62:0/1656021289 >> v1:172.21.15.121:6789/0 conn(0x7f77980b4730 legacy=0x7f779815beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:19.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.043+0000 7f779ca00640 1 -- 172.21.15.62:0/1656021289 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779815cc80 con 0x7f779806c440 2024-04-25T09:17:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.044+0000 7f779ca00640 1 --2- 172.21.15.62:0/1656021289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f779806c8d0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f779806e5b0 tx=0x7f779002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949361ffddc2b7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.044+0000 7f7797400640 1 -- 172.21.15.62:0/1656021289 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790036070 con 0x7f779806c440 2024-04-25T09:17:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.044+0000 7f7797400640 1 -- 172.21.15.62:0/1656021289 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7790031600 con 0x7f779806c440 2024-04-25T09:17:19.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.044+0000 7f7797400640 1 -- 172.21.15.62:0/1656021289 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77900319d0 con 0x7f779806c440 2024-04-25T09:17:19.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.045+0000 7f779d400640 1 -- 172.21.15.62:0/1656021289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 msgr2=0x7f779806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:19.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.045+0000 7f779d400640 1 --2- 172.21.15.62:0/1656021289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f779806c8d0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f779806e5b0 tx=0x7f779002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.045+0000 7f779d400640 1 -- 172.21.15.62:0/1656021289 shutdown_connections 2024-04-25T09:17:19.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.045+0000 7f779d400640 1 --2- 172.21.15.62:0/1656021289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f779806c8d0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.045+0000 7f779d400640 1 --2- 172.21.15.62:0/1656021289 >> v2:172.21.15.121:3300/0 conn(0x7f7798145140 0x7f77980b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.045+0000 7f779d400640 1 -- 172.21.15.62:0/1656021289 >> 172.21.15.62:0/1656021289 conn(0x7f7798066ad0 msgr2=0x7f7798066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:19.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.045+0000 7f779d400640 1 -- 172.21.15.62:0/1656021289 shutdown_connections 2024-04-25T09:17:19.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.045+0000 7f779d400640 1 -- 172.21.15.62:0/1656021289 wait complete. 2024-04-25T09:17:19.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.047+0000 7f779d400640 1 Processor -- start 2024-04-25T09:17:19.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.047+0000 7f779d400640 1 -- start start 2024-04-25T09:17:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f779d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f7798136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f779d400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7798145140 0x7f779815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f779d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f779815d400 0x7f7798132d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f779d400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980b51a0 con 0x7f779806c440 2024-04-25T09:17:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f779d400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980b5020 con 0x7f779815d400 2024-04-25T09:17:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f779d400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779815c5c0 con 0x7f7798145140 2024-04-25T09:17:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f7797e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f7798136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:19.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f7797e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f7798136a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44748/0 (socket says 172.21.15.62:44748) 2024-04-25T09:17:19.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f7797e00640 1 -- 172.21.15.62:0/3895127424 learned_addr learned my addr 172.21.15.62:0/3895127424 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f779ca00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f779815d400 0x7f7798132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.048+0000 7f778fe00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7798145140 0x7f779815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7797e00640 1 -- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7798145140 msgr2=0x7f779815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7797e00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7798145140 0x7f779815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7797e00640 1 -- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f779815d400 msgr2=0x7f7798132d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:19.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7797e00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f779815d400 0x7f7798132d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7797e00640 1 -- 172.21.15.62:0/3895127424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798130eb0 con 0x7f779806c440 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f779ca00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f779815d400 0x7f7798132d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f778fe00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7798145140 0x7f779815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7797e00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f7798136a80 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f778800d5f0 tx=0x7f778800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7794c00640 1 -- 172.21.15.62:0/3895127424 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77880125f0 con 0x7f779806c440 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7794c00640 1 -- 172.21.15.62:0/3895127424 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788019070 con 0x7f779806c440 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f7794c00640 1 -- 172.21.15.62:0/3895127424 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77880129d0 con 0x7f779806c440 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.049+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7798133af0 con 0x7f779806c440 2024-04-25T09:17:19.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.050+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7798131450 con 0x7f779806c440 2024-04-25T09:17:19.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.051+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779815d810 con 0x7f779806c440 2024-04-25T09:17:19.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.052+0000 7f7794c00640 1 -- 172.21.15.62:0/3895127424 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7788012b70 con 0x7f779806c440 2024-04-25T09:17:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.053+0000 7f7794c00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f777406b4d0 0x7f777406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.053+0000 7f7794c00640 1 -- 172.21.15.62:0/3895127424 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f778808a430 con 0x7f779806c440 2024-04-25T09:17:19.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.054+0000 7f778fe00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f777406b4d0 0x7f777406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.055+0000 7f778fe00640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f777406b4d0 0x7f777406d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7798072110 tx=0x7f7780009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:19.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.059+0000 7f7794c00640 1 -- 172.21.15.62:0/3895127424 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f779815d810 con 0x7f779806c440 2024-04-25T09:17:19.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.257+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7798072b30 con 0x7f779806c440 2024-04-25T09:17:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.810+0000 7f7794c00640 1 -- 172.21.15.62:0/3895127424 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7788059440 con 0x7f779806c440 2024-04-25T09:17:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f777406b4d0 msgr2=0x7f777406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f777406b4d0 0x7f777406d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7798072110 tx=0x7f7780009040 comp rx=0 tx=0).stop 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 msgr2=0x7f7798136a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f7798136a80 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f778800d5f0 tx=0x7f778800dab0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 shutdown_connections 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f777406b4d0 0x7f777406d990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f779815d400 0x7f7798132d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7798145140 0x7f779815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 --2- 172.21.15.62:0/3895127424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f779806c440 0x7f7798136a80 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.813+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 >> 172.21.15.62:0/3895127424 conn(0x7f7798066ad0 msgr2=0x7f77980b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.814+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 shutdown_connections 2024-04-25T09:17:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:19.814+0000 7f779d400640 1 -- 172.21.15.62:0/3895127424 wait complete. 2024-04-25T09:17:19.821 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:17:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.045+0000 7fcc37e00640 1 Processor -- start 2024-04-25T09:17:20.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.045+0000 7fcc37e00640 1 -- start start 2024-04-25T09:17:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.047+0000 7fcc37e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc3806d9d0 0x7fcc3815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.047+0000 7fcc37e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcc3815e5f0 0x7fcc380b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.047+0000 7fcc37e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3815ee30 con 0x7fcc380ceb50 2024-04-25T09:17:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.047+0000 7fcc37e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc381618b0 con 0x7fcc3815e5f0 2024-04-25T09:17:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.047+0000 7fcc37e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38161b90 con 0x7fcc3806d9d0 2024-04-25T09:17:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.048+0000 7fcc36000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc3806d9d0 0x7fcc3815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.048+0000 7fcc36a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcc380ceb50 0x7fcc3806d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:45510/0 (socket says 172.21.15.62:45510) 2024-04-25T09:17:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.048+0000 7fcc36a00640 1 -- 172.21.15.62:0/432972131 learned_addr learned my addr 172.21.15.62:0/432972131 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.048+0000 7fcc37400640 1 --2- 172.21.15.62:0/432972131 >> v2:172.21.15.121:3300/0 conn(0x7fcc3815e5f0 0x7fcc380b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.048+0000 7fcc36000640 1 -- 172.21.15.62:0/432972131 >> v2:172.21.15.121:3300/0 conn(0x7fcc3815e5f0 msgr2=0x7fcc380b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.048+0000 7fcc36000640 1 --2- 172.21.15.62:0/432972131 >> v2:172.21.15.121:3300/0 conn(0x7fcc3815e5f0 0x7fcc380b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.048+0000 7fcc36000640 1 -- 172.21.15.62:0/432972131 >> v1:172.21.15.121:6789/0 conn(0x7fcc380ceb50 legacy=0x7fcc3806d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.048+0000 7fcc36000640 1 -- 172.21.15.62:0/432972131 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc380cef30 con 0x7fcc3806d9d0 2024-04-25T09:17:20.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.050+0000 7fcc36000640 1 --2- 172.21.15.62:0/432972131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc3806d9d0 0x7fcc3815e0b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc240097a0 tx=0x7fcc2402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9842d236b4e550a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:20.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.051+0000 7fcc35600640 1 -- 172.21.15.62:0/432972131 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24037070 con 0x7fcc3806d9d0 2024-04-25T09:17:20.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.051+0000 7fcc35600640 1 -- 172.21.15.62:0/432972131 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2402ae50 con 0x7fcc3806d9d0 2024-04-25T09:17:20.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.051+0000 7fcc35600640 1 -- 172.21.15.62:0/432972131 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24032660 con 0x7fcc3806d9d0 2024-04-25T09:17:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.051+0000 7fcc37e00640 1 -- 172.21.15.62:0/432972131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc3806d9d0 msgr2=0x7fcc3815e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.051+0000 7fcc37e00640 1 --2- 172.21.15.62:0/432972131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc3806d9d0 0x7fcc3815e0b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcc240097a0 tx=0x7fcc2402a100 comp rx=0 tx=0).stop 2024-04-25T09:17:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.052+0000 7fcc37e00640 1 -- 172.21.15.62:0/432972131 shutdown_connections 2024-04-25T09:17:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.052+0000 7fcc37e00640 1 --2- 172.21.15.62:0/432972131 >> v2:172.21.15.121:3300/0 conn(0x7fcc3815e5f0 0x7fcc380b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.052+0000 7fcc37e00640 1 --2- 172.21.15.62:0/432972131 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc3806d9d0 0x7fcc3815e0b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.052+0000 7fcc37e00640 1 -- 172.21.15.62:0/432972131 >> 172.21.15.62:0/432972131 conn(0x7fcc38066f20 msgr2=0x7fcc380b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.052+0000 7fcc37e00640 1 -- 172.21.15.62:0/432972131 shutdown_connections 2024-04-25T09:17:20.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.052+0000 7fcc37e00640 1 -- 172.21.15.62:0/432972131 wait complete. 2024-04-25T09:17:20.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.053+0000 7fcc37e00640 1 Processor -- start 2024-04-25T09:17:20.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.054+0000 7fcc37e00640 1 -- start start 2024-04-25T09:17:20.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.054+0000 7fcc37e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc3806d9d0 0x7fcc3815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.054+0000 7fcc37e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc380ceb50 0x7fcc38142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.054+0000 7fcc37e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc3815e5f0 0x7fcc3806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.054+0000 7fcc37e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38162510 con 0x7fcc3806d9d0 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.054+0000 7fcc37e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38162390 con 0x7fcc3815e5f0 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.054+0000 7fcc37e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380b4a90 con 0x7fcc380ceb50 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc3806d9d0 0x7fcc3815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc3806d9d0 0x7fcc3815b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44752/0 (socket says 172.21.15.62:44752) 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36a00640 1 -- 172.21.15.62:0/2173453609 learned_addr learned my addr 172.21.15.62:0/2173453609 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36000640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc380ceb50 0x7fcc38142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc37400640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc3815e5f0 0x7fcc3806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36a00640 1 -- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc380ceb50 msgr2=0x7fcc38142dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36a00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc380ceb50 0x7fcc38142dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36a00640 1 -- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc3815e5f0 msgr2=0x7fcc3806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36a00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc3815e5f0 0x7fcc3806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.055+0000 7fcc36a00640 1 -- 172.21.15.62:0/2173453609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc381566c0 con 0x7fcc3806d9d0 2024-04-25T09:17:20.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.056+0000 7fcc36a00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc3806d9d0 0x7fcc3815b2a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800c400 tx=0x7fcc2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:20.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.056+0000 7fcc2ea00640 1 -- 172.21.15.62:0/2173453609 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc28017070 con 0x7fcc3806d9d0 2024-04-25T09:17:20.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.056+0000 7fcc2ea00640 1 -- 172.21.15.62:0/2173453609 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc28005ce0 con 0x7fcc3806d9d0 2024-04-25T09:17:20.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.056+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc381569a0 con 0x7fcc3806d9d0 2024-04-25T09:17:20.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.056+0000 7fcc2ea00640 1 -- 172.21.15.62:0/2173453609 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc280072f0 con 0x7fcc3806d9d0 2024-04-25T09:17:20.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.056+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc38143300 con 0x7fcc3806d9d0 2024-04-25T09:17:20.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.057+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3806abe0 con 0x7fcc3806d9d0 2024-04-25T09:17:20.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.058+0000 7fcc2ea00640 1 -- 172.21.15.62:0/2173453609 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc2801a070 con 0x7fcc3806d9d0 2024-04-25T09:17:20.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.059+0000 7fcc2ea00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcc1806b4d0 0x7fcc1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.059+0000 7fcc2ea00640 1 -- 172.21.15.62:0/2173453609 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcc28089490 con 0x7fcc3806d9d0 2024-04-25T09:17:20.047 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.060+0000 7fcc36000640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcc1806b4d0 0x7fcc1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.061+0000 7fcc36000640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcc1806b4d0 0x7fcc1806d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcc24034000 tx=0x7fcc2402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:20.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.064+0000 7fcc2ea00640 1 -- 172.21.15.62:0/2173453609 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc28058490 con 0x7fcc3806d9d0 2024-04-25T09:17:20.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.271+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcc3804ef20 con 0x7fcc3806d9d0 2024-04-25T09:17:20.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.277+0000 7fcc2ea00640 1 -- 172.21.15.62:0/2173453609 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcc28057d90 con 0x7fcc3806d9d0 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.279+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcc1806b4d0 msgr2=0x7fcc1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.279+0000 7fcc37e00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcc1806b4d0 0x7fcc1806d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcc24034000 tx=0x7fcc2402e000 comp rx=0 tx=0).stop 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.279+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc3806d9d0 msgr2=0x7fcc3815b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.279+0000 7fcc37e00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc3806d9d0 0x7fcc3815b2a0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800c400 tx=0x7fcc2800c8c0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.280+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 shutdown_connections 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.280+0000 7fcc37e00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcc1806b4d0 0x7fcc1806d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.280+0000 7fcc37e00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc3815e5f0 0x7fcc3806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.280+0000 7fcc37e00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcc380ceb50 0x7fcc38142dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.280+0000 7fcc37e00640 1 --2- 172.21.15.62:0/2173453609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc3806d9d0 0x7fcc3815b2a0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.280+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 >> 172.21.15.62:0/2173453609 conn(0x7fcc38066f20 msgr2=0x7fcc3815f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:20.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.280+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 shutdown_connections 2024-04-25T09:17:20.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.280+0000 7fcc37e00640 1 -- 172.21.15.62:0/2173453609 wait complete. 2024-04-25T09:17:20.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T09:17:20.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.501+0000 7f5c2ae00640 1 Processor -- start 2024-04-25T09:17:20.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.502+0000 7f5c2ae00640 1 -- start start 2024-04-25T09:17:20.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.503+0000 7f5c2ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c24161f00 0x7f5c241622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.503+0000 7f5c2ae00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5c2406d900 0x7f5c240cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.503+0000 7f5c2ae00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c240b2a20 con 0x7f5c240cc9e0 2024-04-25T09:17:20.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.503+0000 7f5c2ae00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c240b27d0 con 0x7f5c2406d900 2024-04-25T09:17:20.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.503+0000 7f5c2ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c240b2d00 con 0x7f5c24161f00 2024-04-25T09:17:20.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.504+0000 7f5c29000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5c240cc9e0 0x7f5c2406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:45512/0 (socket says 172.21.15.62:45512) 2024-04-25T09:17:20.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.504+0000 7f5c29000640 1 -- 172.21.15.62:0/493273355 learned_addr learned my addr 172.21.15.62:0/493273355 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:20.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.504+0000 7f5c29a00640 1 --2- 172.21.15.62:0/493273355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c24161f00 0x7f5c241622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.504+0000 7f5c2a400640 1 --2- 172.21.15.62:0/493273355 >> v2:172.21.15.121:3300/0 conn(0x7f5c2406d900 0x7f5c240cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.505+0000 7f5c23e00640 1 -- 172.21.15.62:0/493273355 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506496738 0 0) 0x7f5c240b2a20 con 0x7f5c240cc9e0 2024-04-25T09:17:20.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.505+0000 7f5c23e00640 1 -- 172.21.15.62:0/493273355 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c0c003520 con 0x7f5c240cc9e0 2024-04-25T09:17:20.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.505+0000 7f5c29a00640 1 -- 172.21.15.62:0/493273355 >> v2:172.21.15.121:3300/0 conn(0x7f5c2406d900 msgr2=0x7f5c240cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:20.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.505+0000 7f5c29a00640 1 --2- 172.21.15.62:0/493273355 >> v2:172.21.15.121:3300/0 conn(0x7f5c2406d900 0x7f5c240cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.505+0000 7f5c29a00640 1 -- 172.21.15.62:0/493273355 >> v1:172.21.15.121:6789/0 conn(0x7f5c240cc9e0 legacy=0x7f5c2406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.505+0000 7f5c29a00640 1 -- 172.21.15.62:0/493273355 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c240b2fe0 con 0x7f5c24161f00 2024-04-25T09:17:20.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.505+0000 7f5c2a400640 1 --2- 172.21.15.62:0/493273355 >> v2:172.21.15.121:3300/0 conn(0x7f5c2406d900 0x7f5c240cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:17:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.507+0000 7f5c29a00640 1 --2- 172.21.15.62:0/493273355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c24161f00 0x7f5c241622e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5c180097a0 tx=0x7f5c1802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9c2224609bf09e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.507+0000 7f5c23e00640 1 -- 172.21.15.62:0/493273355 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18037070 con 0x7f5c24161f00 2024-04-25T09:17:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.507+0000 7f5c23e00640 1 -- 172.21.15.62:0/493273355 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c18032440 con 0x7f5c24161f00 2024-04-25T09:17:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.507+0000 7f5c23e00640 1 -- 172.21.15.62:0/493273355 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18032810 con 0x7f5c24161f00 2024-04-25T09:17:20.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.508+0000 7f5c2ae00640 1 -- 172.21.15.62:0/493273355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c24161f00 msgr2=0x7f5c241622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.508+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/493273355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c24161f00 0x7f5c241622e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5c180097a0 tx=0x7f5c1802a180 comp rx=0 tx=0).stop 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.508+0000 7f5c2ae00640 1 -- 172.21.15.62:0/493273355 shutdown_connections 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.508+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/493273355 >> v2:172.21.15.121:3300/0 conn(0x7f5c2406d900 0x7f5c240cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.508+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/493273355 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c24161f00 0x7f5c241622e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.508+0000 7f5c2ae00640 1 -- 172.21.15.62:0/493273355 >> 172.21.15.62:0/493273355 conn(0x7f5c240b14d0 msgr2=0x7f5c240b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.508+0000 7f5c2ae00640 1 -- 172.21.15.62:0/493273355 shutdown_connections 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.508+0000 7f5c2ae00640 1 -- 172.21.15.62:0/493273355 wait complete. 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.510+0000 7f5c2ae00640 1 Processor -- start 2024-04-25T09:17:20.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.510+0000 7f5c2ae00640 1 -- start start 2024-04-25T09:17:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.511+0000 7f5c2ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c2406d900 0x7f5c24159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.511+0000 7f5c2ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c240cc9e0 0x7f5c24148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.511+0000 7f5c2ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c24161f00 0x7f5c24148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.511+0000 7f5c2ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c240b4ad0 con 0x7f5c24161f00 2024-04-25T09:17:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.511+0000 7f5c2ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c240b35a0 con 0x7f5c2406d900 2024-04-25T09:17:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.511+0000 7f5c2ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c240b4c50 con 0x7f5c240cc9e0 2024-04-25T09:17:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c24161f00 0x7f5c24148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c24161f00 0x7f5c24148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44762/0 (socket says 172.21.15.62:44762) 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 -- 172.21.15.62:0/3391724267 learned_addr learned my addr 172.21.15.62:0/3391724267 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c29000640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c240cc9e0 0x7f5c24148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c29a00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c2406d900 0x7f5c24159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 -- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c240cc9e0 msgr2=0x7f5c24148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c240cc9e0 0x7f5c24148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 -- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c2406d900 msgr2=0x7f5c24159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c2406d900 0x7f5c24159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 -- 172.21.15.62:0/3391724267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2415bc90 con 0x7f5c24161f00 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.512+0000 7f5c2a400640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c24161f00 0x7f5c24148590 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00efc0 tx=0x7f5c1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.513+0000 7f5c21600640 1 -- 172.21.15.62:0/3391724267 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c018030 con 0x7f5c24161f00 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.513+0000 7f5c21600640 1 -- 172.21.15.62:0/3391724267 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c00cdf0 con 0x7f5c24161f00 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.513+0000 7f5c21600640 1 -- 172.21.15.62:0/3391724267 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c003bb0 con 0x7f5c24161f00 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.513+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2415bf70 con 0x7f5c24161f00 2024-04-25T09:17:20.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.513+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2415b7f0 con 0x7f5c24161f00 2024-04-25T09:17:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.514+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2404ef20 con 0x7f5c24161f00 2024-04-25T09:17:20.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.515+0000 7f5c21600640 1 -- 172.21.15.62:0/3391724267 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c1c01b070 con 0x7f5c24161f00 2024-04-25T09:17:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.516+0000 7f5c21600640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c0406b4d0 0x7f5c0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:20.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.516+0000 7f5c21600640 1 -- 172.21.15.62:0/3391724267 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5c1c08a9e0 con 0x7f5c24161f00 2024-04-25T09:17:20.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.517+0000 7f5c29a00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c0406b4d0 0x7f5c0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:20.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.518+0000 7f5c29a00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c0406b4d0 0x7f5c0406d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5c18034000 tx=0x7f5c1802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:20.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.520+0000 7f5c21600640 1 -- 172.21.15.62:0/3391724267 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c1c010040 con 0x7f5c24161f00 2024-04-25T09:17:20.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:20.859+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f5c2415c140 con 0x7f5c24161f00 2024-04-25T09:17:21.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.820+0000 7f5c21600640 1 -- 172.21.15.62:0/3391724267 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v634) v1 ==== 166+0+0 (secure 0 0 0) 0x7f5c1c012340 con 0x7f5c24161f00 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c0406b4d0 msgr2=0x7f5c0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c0406b4d0 0x7f5c0406d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5c18034000 tx=0x7f5c1802e000 comp rx=0 tx=0).stop 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c24161f00 msgr2=0x7f5c24148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c24161f00 0x7f5c24148590 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00efc0 tx=0x7f5c1c00c490 comp rx=0 tx=0).stop 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 shutdown_connections 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f5c0406b4d0 0x7f5c0406d990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5c24161f00 0x7f5c24148590 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5c240cc9e0 0x7f5c24148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 --2- 172.21.15.62:0/3391724267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c2406d900 0x7f5c24159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.823+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 >> 172.21.15.62:0/3391724267 conn(0x7f5c240b14d0 msgr2=0x7f5c24070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.824+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 shutdown_connections 2024-04-25T09:17:21.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:21.824+0000 7f5c2ae00640 1 -- 172.21.15.62:0/3391724267 wait complete. 2024-04-25T09:17:21.833 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:17:21.834 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:17:21.834 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:22.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.062+0000 7ff5e7200640 1 Processor -- start 2024-04-25T09:17:22.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.063+0000 7ff5e7200640 1 -- start start 2024-04-25T09:17:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.064+0000 7ff5e7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 0x7ff5e80cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.064+0000 7ff5e7200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e80d23b0 con 0x7ff5e80d0950 2024-04-25T09:17:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.064+0000 7ff5e7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e80d2690 con 0x7ff5e80d1e80 2024-04-25T09:17:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.064+0000 7ff5e7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d2970 con 0x7ff5e815dd70 2024-04-25T09:17:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.064+0000 7ff5e6800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff5e80d0950 0x7ff5e80b6c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34044/0 (socket says 172.21.15.62:34044) 2024-04-25T09:17:22.051 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.064+0000 7ff5e6800640 1 -- 172.21.15.62:0/1870306383 learned_addr learned my addr 172.21.15.62:0/1870306383 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5e00640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 0x7ff5e80cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818395912 0 0) 0x7ff5e80d23b0 con 0x7ff5e80d0950 2024-04-25T09:17:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5c8003520 con 0x7ff5e80d0950 2024-04-25T09:17:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2315798277 0 0) 0x7ff5c8003520 con 0x7ff5e80d0950 2024-04-25T09:17:22.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 msgr2=0x7ff5e80cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 0x7ff5e80cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 >> v1:172.21.15.121:6789/0 conn(0x7ff5e80d1e80 legacy=0x7ff5e80d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80d2c50 con 0x7ff5e80d0950 2024-04-25T09:17:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ff5e00019f0 con 0x7ff5e80d0950 2024-04-25T09:17:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 >> v1:172.21.15.62:6789/0 conn(0x7ff5e80d0950 legacy=0x7ff5e80b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5c80039e0 0x7ff5c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 0x7ff5c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c8004d40 0x7ff5c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d2970 con 0x7ff5c8004d40 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c800ba70 con 0x7ff5c80039e0 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.065+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c800bc70 con 0x7ff5e815dd70 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5e6800640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c8004d40 0x7ff5c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5e5e00640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5c80039e0 0x7ff5c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5e6800640 1 -- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 msgr2=0x7ff5c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5dfe00640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 0x7ff5c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5e6800640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 0x7ff5c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5e6800640 1 -- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5c80039e0 msgr2=0x7ff5c8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:22.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5e6800640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5c80039e0 0x7ff5c8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5e6800640 1 -- 172.21.15.62:0/1870306383 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8003520 con 0x7ff5c8004d40 2024-04-25T09:17:22.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5dfe00640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5e815dd70 0x7ff5c8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:22.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.066+0000 7ff5e5e00640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5c80039e0 0x7ff5c8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.067+0000 7ff5e6800640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c8004d40 0x7ff5c80051c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000a810 tx=0x7ff5c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645b682d48bcd331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000a460 con 0x7ff5c8004d40 2024-04-25T09:17:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e002f070 con 0x7ff5c8004d40 2024-04-25T09:17:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e5400640 1 -- 172.21.15.62:0/1870306383 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e002bc90 con 0x7ff5c8004d40 2024-04-25T09:17:22.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e7200640 1 -- 172.21.15.62:0/1870306383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c8004d40 msgr2=0x7ff5c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e7200640 1 --2- 172.21.15.62:0/1870306383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5c8004d40 0x7ff5c80051c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000a810 tx=0x7ff5c8005830 comp rx=0 tx=0).stop 2024-04-25T09:17:22.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e6800640 1 -- 172.21.15.62:0/1870306383 reap_dead start 2024-04-25T09:17:22.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e7200640 1 -- 172.21.15.62:0/1870306383 shutdown_connections 2024-04-25T09:17:22.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e7200640 1 -- 172.21.15.62:0/1870306383 >> 172.21.15.62:0/1870306383 conn(0x7ff5e806b210 msgr2=0x7ff5e80668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:22.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e7200640 1 -- 172.21.15.62:0/1870306383 shutdown_connections 2024-04-25T09:17:22.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.068+0000 7ff5e7200640 1 -- 172.21.15.62:0/1870306383 wait complete. 2024-04-25T09:17:22.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.070+0000 7ff5e7200640 1 Processor -- start 2024-04-25T09:17:22.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.070+0000 7ff5e7200640 1 -- start start 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.071+0000 7ff5e7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5c8004d40 0x7ff5e80aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.071+0000 7ff5e7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5e80b4460 0x7ff5e8150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.071+0000 7ff5e7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e815b7f0 0x7ff5e815bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.071+0000 7ff5e7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d3450 con 0x7ff5e815b7f0 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.071+0000 7ff5e7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815e150 con 0x7ff5e80b4460 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.071+0000 7ff5e7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d35d0 con 0x7ff5c8004d40 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.071+0000 7ff5e6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e815b7f0 0x7ff5e815bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e815b7f0 0x7ff5e815bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44780/0 (socket says 172.21.15.62:44780) 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e6800640 1 -- 172.21.15.62:0/1329145434 learned_addr learned my addr 172.21.15.62:0/1329145434 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e5e00640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5c8004d40 0x7ff5e80aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5dfe00640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5e80b4460 0x7ff5e8150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e6800640 1 -- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5c8004d40 msgr2=0x7ff5e80aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e6800640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5c8004d40 0x7ff5e80aa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e6800640 1 -- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5e80b4460 msgr2=0x7ff5e8150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e6800640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5e80b4460 0x7ff5e8150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e6800640 1 -- 172.21.15.62:0/1329145434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80b48e0 con 0x7ff5e815b7f0 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.072+0000 7ff5e6800640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e815b7f0 0x7ff5e815bc70 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000a930 tx=0x7ff5e80d2860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.073+0000 7ff5de000640 1 -- 172.21.15.62:0/1329145434 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000a460 con 0x7ff5e815b7f0 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.073+0000 7ff5de000640 1 -- 172.21.15.62:0/1329145434 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0004b60 con 0x7ff5e815b7f0 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.073+0000 7ff5de000640 1 -- 172.21.15.62:0/1329145434 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0001fb0 con 0x7ff5e815b7f0 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.073+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e80b4b60 con 0x7ff5e815b7f0 2024-04-25T09:17:22.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.073+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80b5120 con 0x7ff5e815b7f0 2024-04-25T09:17:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.074+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e804ef20 con 0x7ff5e815b7f0 2024-04-25T09:17:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.075+0000 7ff5de000640 1 -- 172.21.15.62:0/1329145434 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5e0034470 con 0x7ff5e815b7f0 2024-04-25T09:17:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.076+0000 7ff5de000640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5c406b500 0x7ff5c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.076+0000 7ff5de000640 1 -- 172.21.15.62:0/1329145434 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff5e00aa0b0 con 0x7ff5e815b7f0 2024-04-25T09:17:22.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.077+0000 7ff5e5e00640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5c406b500 0x7ff5c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.078+0000 7ff5e5e00640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5c406b500 0x7ff5c406d9c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80099c0 tx=0x7ff5d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.081+0000 7ff5de000640 1 -- 172.21.15.62:0/1329145434 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5e80b48e0 con 0x7ff5e815b7f0 2024-04-25T09:17:22.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.282+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5e80d05c0 con 0x7ff5e815b7f0 2024-04-25T09:17:22.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.283+0000 7ff5de000640 1 -- 172.21.15.62:0/1329145434 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+13148 (secure 0 0 0) 0x7ff5e0079260 con 0x7ff5e815b7f0 2024-04-25T09:17:22.271 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:22.271 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":141,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:19.807784+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.121:6837/419350964":"2024-04-26T09:17:16.607920+0000","172.21.15.121:6835/419350964":"2024-04-26T09:17:16.607920+0000","172.21.15.62:6834/2303077743":"2024-04-26T09:17:02.676113+0000","172.21.15.62:6832/2303077743":"2024-04-26T09:17:02.676113+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:22.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.286+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5c406b500 msgr2=0x7ff5c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.286+0000 7ff5e7200640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5c406b500 0x7ff5c406d9c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80099c0 tx=0x7ff5d8008040 comp rx=0 tx=0).stop 2024-04-25T09:17:22.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.286+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e815b7f0 msgr2=0x7ff5e815bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.286+0000 7ff5e7200640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e815b7f0 0x7ff5e815bc70 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000a930 tx=0x7ff5e80d2860 comp rx=0 tx=0).stop 2024-04-25T09:17:22.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.287+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 shutdown_connections 2024-04-25T09:17:22.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.287+0000 7ff5e7200640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff5c406b500 0x7ff5c406d9c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.287+0000 7ff5e7200640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff5e815b7f0 0x7ff5e815bc70 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.287+0000 7ff5e7200640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff5e80b4460 0x7ff5e8150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.287+0000 7ff5e7200640 1 --2- 172.21.15.62:0/1329145434 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff5c8004d40 0x7ff5e80aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.287+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 >> 172.21.15.62:0/1329145434 conn(0x7ff5e806b210 msgr2=0x7ff5e80668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:22.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.287+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 shutdown_connections 2024-04-25T09:17:22.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.287+0000 7ff5e7200640 1 -- 172.21.15.62:0/1329145434 wait complete. 2024-04-25T09:17:22.294 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:17:22.294 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:17:22.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.523+0000 7f688f200640 1 Processor -- start 2024-04-25T09:17:22.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.524+0000 7f688f200640 1 -- start start 2024-04-25T09:17:22.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.525+0000 7f688f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f689006d9d0 0x7f6890070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.525+0000 7f688f200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f689015db30 0x7f689015dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.525+0000 7f688f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689015e840 con 0x7f6890159e70 2024-04-25T09:17:22.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.525+0000 7f688f200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6890070240 con 0x7f689015db30 2024-04-25T09:17:22.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.525+0000 7f688f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6890070520 con 0x7f689006d9d0 2024-04-25T09:17:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.526+0000 7f688e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f689015db30 0x7f689015dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.526+0000 7f688e800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f689015db30 0x7f689015dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44786/0 (socket says 172.21.15.62:44786) 2024-04-25T09:17:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.526+0000 7f688e800640 1 -- 172.21.15.62:0/787415187 learned_addr learned my addr 172.21.15.62:0/787415187 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.526+0000 7f6887e00640 1 --2- 172.21.15.62:0/787415187 >> v2:172.21.15.121:3300/0 conn(0x7f689006d9d0 0x7f6890070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.526+0000 7f688e800640 1 -- 172.21.15.62:0/787415187 >> v2:172.21.15.121:3300/0 conn(0x7f689006d9d0 msgr2=0x7f6890070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.526+0000 7f688e800640 1 --2- 172.21.15.62:0/787415187 >> v2:172.21.15.121:3300/0 conn(0x7f689006d9d0 0x7f6890070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.526+0000 7f688e800640 1 -- 172.21.15.62:0/787415187 >> v1:172.21.15.121:6789/0 conn(0x7f6890159e70 legacy=0x7f689006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:22.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.526+0000 7f688e800640 1 -- 172.21.15.62:0/787415187 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689015e500 con 0x7f689015db30 2024-04-25T09:17:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.527+0000 7f688e800640 1 --2- 172.21.15.62:0/787415187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f689015dfc0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f68900b4350 tx=0x7f688802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b19eac7d8cecfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688d400640 1 -- 172.21.15.62:0/787415187 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888038070 con 0x7f689015db30 2024-04-25T09:17:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688d400640 1 -- 172.21.15.62:0/787415187 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888033600 con 0x7f689015db30 2024-04-25T09:17:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688d400640 1 -- 172.21.15.62:0/787415187 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68880339d0 con 0x7f689015db30 2024-04-25T09:17:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688f200640 1 -- 172.21.15.62:0/787415187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 msgr2=0x7f689015dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688f200640 1 --2- 172.21.15.62:0/787415187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f689015dfc0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f68900b4350 tx=0x7f688802c7e0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688f200640 1 -- 172.21.15.62:0/787415187 shutdown_connections 2024-04-25T09:17:22.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688f200640 1 --2- 172.21.15.62:0/787415187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f689015dfc0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688f200640 1 --2- 172.21.15.62:0/787415187 >> v2:172.21.15.121:3300/0 conn(0x7f689006d9d0 0x7f6890070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.528+0000 7f688f200640 1 -- 172.21.15.62:0/787415187 >> 172.21.15.62:0/787415187 conn(0x7f6890066f20 msgr2=0x7f68900b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:22.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.529+0000 7f688f200640 1 -- 172.21.15.62:0/787415187 shutdown_connections 2024-04-25T09:17:22.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.529+0000 7f688f200640 1 -- 172.21.15.62:0/787415187 wait complete. 2024-04-25T09:17:22.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.530+0000 7f688f200640 1 Processor -- start 2024-04-25T09:17:22.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.531+0000 7f688f200640 1 -- start start 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688f200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689006d9d0 0x7f6890144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890159e70 0x7f68900abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f68900ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b1c50 con 0x7f689015db30 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b1ad0 con 0x7f6890159e70 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688f200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b1dd0 con 0x7f689006d9d0 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f68900ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f68900ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44790/0 (socket says 172.21.15.62:44790) 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688e800640 1 -- 172.21.15.62:0/4276939459 learned_addr learned my addr 172.21.15.62:0/4276939459 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f688de00640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689006d9d0 0x7f6890144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.532+0000 7f6887e00640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890159e70 0x7f68900abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.533+0000 7f688e800640 1 -- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689006d9d0 msgr2=0x7f6890144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.533+0000 7f688e800640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689006d9d0 0x7f6890144520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.533+0000 7f688e800640 1 -- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890159e70 msgr2=0x7f68900abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.533+0000 7f688e800640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890159e70 0x7f68900abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.533+0000 7f688e800640 1 -- 172.21.15.62:0/4276939459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890144a60 con 0x7f689015db30 2024-04-25T09:17:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.533+0000 7f6887e00640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890159e70 0x7f68900abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:17:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.533+0000 7f688de00640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689006d9d0 0x7f6890144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:17:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.533+0000 7f688e800640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f68900ac230 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f6888009f40 tx=0x7f6888004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.534+0000 7f6886000640 1 -- 172.21.15.62:0/4276939459 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888038050 con 0x7f689015db30 2024-04-25T09:17:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.534+0000 7f6886000640 1 -- 172.21.15.62:0/4276939459 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888044070 con 0x7f689015db30 2024-04-25T09:17:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.534+0000 7f6886000640 1 -- 172.21.15.62:0/4276939459 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888009510 con 0x7f689015db30 2024-04-25T09:17:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.534+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689015c3a0 con 0x7f689015db30 2024-04-25T09:17:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.534+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689015c960 con 0x7f689015db30 2024-04-25T09:17:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.535+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689006abe0 con 0x7f689015db30 2024-04-25T09:17:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.536+0000 7f6886000640 1 -- 172.21.15.62:0/4276939459 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6888007bb0 con 0x7f689015db30 2024-04-25T09:17:22.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.537+0000 7f6886000640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b500 0x7f686c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.537+0000 7f6886000640 1 -- 172.21.15.62:0/4276939459 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f68880acdc0 con 0x7f689015db30 2024-04-25T09:17:22.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.538+0000 7f688de00640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b500 0x7f686c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.539+0000 7f688de00640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b500 0x7f686c06d9c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f68901f2d20 tx=0x7f6880009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.541+0000 7f6886000640 1 -- 172.21.15.62:0/4276939459 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6890144a60 con 0x7f689015db30 2024-04-25T09:17:22.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.741+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6890071c00 con 0x7f689015db30 2024-04-25T09:17:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.742+0000 7f6886000640 1 -- 172.21.15.62:0/4276939459 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6888047360 con 0x7f689015db30 2024-04-25T09:17:22.729 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:17:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.744+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b500 msgr2=0x7f686c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.744+0000 7f688f200640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b500 0x7f686c06d9c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f68901f2d20 tx=0x7f6880009040 comp rx=0 tx=0).stop 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.744+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 msgr2=0x7f68900ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.744+0000 7f688f200640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f68900ac230 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f6888009f40 tx=0x7f6888004a40 comp rx=0 tx=0).stop 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.745+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 shutdown_connections 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.745+0000 7f688f200640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f686c06b500 0x7f686c06d9c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.745+0000 7f688f200640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f689015db30 0x7f68900ac230 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.745+0000 7f688f200640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6890159e70 0x7f68900abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.745+0000 7f688f200640 1 --2- 172.21.15.62:0/4276939459 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f689006d9d0 0x7f6890144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.745+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 >> 172.21.15.62:0/4276939459 conn(0x7f6890066f20 msgr2=0x7f68900cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.745+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 shutdown_connections 2024-04-25T09:17:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.745+0000 7f688f200640 1 -- 172.21.15.62:0/4276939459 wait complete. 2024-04-25T09:17:22.753 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T09:17:22.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.981+0000 7f4cfd400640 1 Processor -- start 2024-04-25T09:17:22.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.981+0000 7f4cfd400640 1 -- start start 2024-04-25T09:17:22.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.982+0000 7f4cfd400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4cf8144c10 0x7f4cf815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.982+0000 7f4cfd400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4cf80cee40 0x7f4cf806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.982+0000 7f4cfd400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80cf2d0 con 0x7f4cf806d730 2024-04-25T09:17:22.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.982+0000 7f4cfd400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf8144ff0 con 0x7f4cf8144c10 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.982+0000 7f4cfd400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf806dba0 con 0x7f4cf80cee40 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.983+0000 7f4cf7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4cf8144c10 0x7f4cf815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.983+0000 7f4cf7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4cf8144c10 0x7f4cf815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44806/0 (socket says 172.21.15.62:44806) 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.983+0000 7f4cf7e00640 1 -- 172.21.15.62:0/1682148334 learned_addr learned my addr 172.21.15.62:0/1682148334 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.983+0000 7f4cf7e00640 1 -- 172.21.15.62:0/1682148334 >> v2:172.21.15.121:3300/0 conn(0x7f4cf80cee40 msgr2=0x7f4cf806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.983+0000 7f4cef400640 1 --2- 172.21.15.62:0/1682148334 >> v2:172.21.15.121:3300/0 conn(0x7f4cf80cee40 0x7f4cf806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.983+0000 7f4cf7e00640 1 --2- 172.21.15.62:0/1682148334 >> v2:172.21.15.121:3300/0 conn(0x7f4cf80cee40 0x7f4cf806d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.983+0000 7f4cf7e00640 1 -- 172.21.15.62:0/1682148334 >> v1:172.21.15.121:6789/0 conn(0x7f4cf806d730 legacy=0x7f4cf80b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:22.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.984+0000 7f4cf7e00640 1 -- 172.21.15.62:0/1682148334 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf815faa0 con 0x7f4cf8144c10 2024-04-25T09:17:22.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.985+0000 7f4cf7e00640 1 --2- 172.21.15.62:0/1682148334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf8144c10 0x7f4cf815f450 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f4cf806e680 tx=0x7f4ce802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f78f2c7fd5dbb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.985+0000 7f4cf7400640 1 -- 172.21.15.62:0/1682148334 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8037070 con 0x7f4cf8144c10 2024-04-25T09:17:22.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.985+0000 7f4cf7400640 1 -- 172.21.15.62:0/1682148334 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce8032440 con 0x7f4cf8144c10 2024-04-25T09:17:22.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.985+0000 7f4cf7400640 1 -- 172.21.15.62:0/1682148334 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8032810 con 0x7f4cf8144c10 2024-04-25T09:17:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.986+0000 7f4cfd400640 1 -- 172.21.15.62:0/1682148334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf8144c10 msgr2=0x7f4cf815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.986+0000 7f4cfd400640 1 --2- 172.21.15.62:0/1682148334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf8144c10 0x7f4cf815f450 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f4cf806e680 tx=0x7f4ce802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.986+0000 7f4cfd400640 1 -- 172.21.15.62:0/1682148334 shutdown_connections 2024-04-25T09:17:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.986+0000 7f4cfd400640 1 --2- 172.21.15.62:0/1682148334 >> v2:172.21.15.121:3300/0 conn(0x7f4cf80cee40 0x7f4cf806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.986+0000 7f4cfd400640 1 --2- 172.21.15.62:0/1682148334 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf8144c10 0x7f4cf815f450 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.986+0000 7f4cfd400640 1 -- 172.21.15.62:0/1682148334 >> 172.21.15.62:0/1682148334 conn(0x7f4cf80b14d0 msgr2=0x7f4cf80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.986+0000 7f4cfd400640 1 -- 172.21.15.62:0/1682148334 shutdown_connections 2024-04-25T09:17:22.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.986+0000 7f4cfd400640 1 -- 172.21.15.62:0/1682148334 wait complete. 2024-04-25T09:17:22.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.988+0000 7f4cfd400640 1 Processor -- start 2024-04-25T09:17:22.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.988+0000 7f4cfd400640 1 -- start start 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf806d730 0x7f4cf80708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cf80cee40 0x7f4cf8070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cfd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cf8144c10 0x7f4cf8140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cfd400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80b41c0 con 0x7f4cf806d730 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cfd400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8160340 con 0x7f4cf8144c10 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cfd400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf8160060 con 0x7f4cf80cee40 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf806d730 0x7f4cf80708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf806d730 0x7f4cf80708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44816/0 (socket says 172.21.15.62:44816) 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cf7e00640 1 -- 172.21.15.62:0/2820577732 learned_addr learned my addr 172.21.15.62:0/2820577732 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cef400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cf80cee40 0x7f4cf8070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cf7e00640 1 -- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cf80cee40 msgr2=0x7f4cf8070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cf7e00640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cf80cee40 0x7f4cf8070e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cf7e00640 1 -- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cf8144c10 msgr2=0x7f4cf8140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:22.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cf7e00640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cf8144c10 0x7f4cf8140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:22.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.989+0000 7f4cf7e00640 1 -- 172.21.15.62:0/2820577732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80a9b30 con 0x7f4cf806d730 2024-04-25T09:17:22.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.990+0000 7f4cf7e00640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf806d730 0x7f4cf80708c0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4ce802a1e0 tx=0x7f4ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.990+0000 7f4cf4c00640 1 -- 172.21.15.62:0/2820577732 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8037040 con 0x7f4cf806d730 2024-04-25T09:17:22.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.990+0000 7f4cf4c00640 1 -- 172.21.15.62:0/2820577732 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce8042070 con 0x7f4cf806d730 2024-04-25T09:17:22.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.990+0000 7f4cf4c00640 1 -- 172.21.15.62:0/2820577732 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8030bb0 con 0x7f4cf806d730 2024-04-25T09:17:22.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.990+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf80a9570 con 0x7f4cf806d730 2024-04-25T09:17:22.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.990+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf8142af0 con 0x7f4cf806d730 2024-04-25T09:17:22.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.992+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf8142920 con 0x7f4cf806d730 2024-04-25T09:17:22.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.992+0000 7f4cf4c00640 1 -- 172.21.15.62:0/2820577732 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ce8047460 con 0x7f4cf806d730 2024-04-25T09:17:22.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.993+0000 7f4cf4c00640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd406b4d0 0x7f4cd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:22.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.994+0000 7f4cf4c00640 1 -- 172.21.15.62:0/2820577732 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4ce80aacb0 con 0x7f4cf806d730 2024-04-25T09:17:22.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.994+0000 7f4cef400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd406b4d0 0x7f4cd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:22.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.995+0000 7f4cef400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd406b4d0 0x7f4cd406d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0006fd0 tx=0x7f4ce0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:22.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:22.998+0000 7f4cf4c00640 1 -- 172.21.15.62:0/2820577732 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ce8045360 con 0x7f4cf806d730 2024-04-25T09:17:23.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.277+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f4cf80ce6a0 con 0x7f4cf806d730 2024-04-25T09:17:23.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.834+0000 7f4cf4c00640 1 -- 172.21.15.62:0/2820577732 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v636) v1 ==== 162+0+0 (secure 0 0 0) 0x7f4ce8079d30 con 0x7f4cf806d730 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.837+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd406b4d0 msgr2=0x7f4cd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.837+0000 7f4cfd400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd406b4d0 0x7f4cd406d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0006fd0 tx=0x7f4ce0008040 comp rx=0 tx=0).stop 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.837+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf806d730 msgr2=0x7f4cf80708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.837+0000 7f4cfd400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf806d730 0x7f4cf80708c0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4ce802a1e0 tx=0x7f4ce80046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.838+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 shutdown_connections 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.838+0000 7f4cfd400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f4cd406b4d0 0x7f4cd406d990 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.838+0000 7f4cfd400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4cf8144c10 0x7f4cf8140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.838+0000 7f4cfd400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f4cf80cee40 0x7f4cf8070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.838+0000 7f4cfd400640 1 --2- 172.21.15.62:0/2820577732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4cf806d730 0x7f4cf80708c0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.838+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 >> 172.21.15.62:0/2820577732 conn(0x7f4cf80b14d0 msgr2=0x7f4cf80cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:23.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.838+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 shutdown_connections 2024-04-25T09:17:23.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:23.838+0000 7f4cfd400640 1 -- 172.21.15.62:0/2820577732 wait complete. 2024-04-25T09:17:23.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:17:24.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.073+0000 7f8dcba00640 1 Processor -- start 2024-04-25T09:17:24.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.073+0000 7f8dcba00640 1 -- start start 2024-04-25T09:17:24.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dcba00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dc415bdf0 0x7f8dc40720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dcba00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc406c9b0 0x7f8dc40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dcba00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc406d070 con 0x7f8dc40cebc0 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dcba00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc406ce40 con 0x7f8dc415bdf0 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dcba00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc415c1d0 con 0x7f8dc406c9b0 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dc9000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dc415bdf0 0x7f8dc40720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dc9000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dc415bdf0 0x7f8dc40720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44832/0 (socket says 172.21.15.62:44832) 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dc9000640 1 -- 172.21.15.62:0/1460314197 learned_addr learned my addr 172.21.15.62:0/1460314197 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dc3e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8dc40cebc0 0x7f8dc406c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:34060/0 (socket says 172.21.15.62:34060) 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.075+0000 7f8dc9a00640 1 --2- 172.21.15.62:0/1460314197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc406c9b0 0x7f8dc40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.076+0000 7f8dc9000640 1 -- 172.21.15.62:0/1460314197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc406c9b0 msgr2=0x7f8dc40703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.076+0000 7f8dc9000640 1 --2- 172.21.15.62:0/1460314197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc406c9b0 0x7f8dc40703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.076+0000 7f8dc9000640 1 -- 172.21.15.62:0/1460314197 >> v1:172.21.15.62:6789/0 conn(0x7f8dc40cebc0 legacy=0x7f8dc406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.076+0000 7f8dc9000640 1 -- 172.21.15.62:0/1460314197 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc406d350 con 0x7f8dc415bdf0 2024-04-25T09:17:24.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.077+0000 7f8dc9000640 1 --2- 172.21.15.62:0/1460314197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40720c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009870 tx=0x7f8db802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71989e1fef09d204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:24.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.077+0000 7f8dc3400640 1 -- 172.21.15.62:0/1460314197 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8037070 con 0x7f8dc415bdf0 2024-04-25T09:17:24.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.077+0000 7f8dc3400640 1 -- 172.21.15.62:0/1460314197 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db8032440 con 0x7f8dc415bdf0 2024-04-25T09:17:24.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.077+0000 7f8dc3400640 1 -- 172.21.15.62:0/1460314197 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8032810 con 0x7f8dc415bdf0 2024-04-25T09:17:24.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.078+0000 7f8dcba00640 1 -- 172.21.15.62:0/1460314197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 msgr2=0x7f8dc40720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.078+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1460314197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40720c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009870 tx=0x7f8db802a250 comp rx=0 tx=0).stop 2024-04-25T09:17:24.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.078+0000 7f8dcba00640 1 -- 172.21.15.62:0/1460314197 shutdown_connections 2024-04-25T09:17:24.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.078+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1460314197 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc406c9b0 0x7f8dc40703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.078+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1460314197 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40720c0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.078+0000 7f8dcba00640 1 -- 172.21.15.62:0/1460314197 >> 172.21.15.62:0/1460314197 conn(0x7f8dc40b1410 msgr2=0x7f8dc40b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:24.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.078+0000 7f8dcba00640 1 -- 172.21.15.62:0/1460314197 shutdown_connections 2024-04-25T09:17:24.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.078+0000 7f8dcba00640 1 -- 172.21.15.62:0/1460314197 wait complete. 2024-04-25T09:17:24.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.080+0000 7f8dcba00640 1 Processor -- start 2024-04-25T09:17:24.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.080+0000 7f8dcba00640 1 -- start start 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dcba00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc406c9b0 0x7f8dc4145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dcba00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc40cebc0 0x7f8dc40adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dcba00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dcba00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40aedf0 con 0x7f8dc415bdf0 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dcba00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40ae8c0 con 0x7f8dc406c9b0 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dcba00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40708e0 con 0x7f8dc40cebc0 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dc9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dc9a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:44844/0 (socket says 172.21.15.62:44844) 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.081+0000 7f8dc9a00640 1 -- 172.21.15.62:0/1758693731 learned_addr learned my addr 172.21.15.62:0/1758693731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:24.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc9000640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc406c9b0 0x7f8dc4145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc3e00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc40cebc0 0x7f8dc40adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc9a00640 1 -- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc40cebc0 msgr2=0x7f8dc40adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc9a00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc40cebc0 0x7f8dc40adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc9a00640 1 -- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc406c9b0 msgr2=0x7f8dc4145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc9a00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc406c9b0 0x7f8dc4145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc9a00640 1 -- 172.21.15.62:0/1758693731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc41f7350 con 0x7f8dc415bdf0 2024-04-25T09:17:24.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc9a00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40ae0c0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc007e40 tx=0x7f8dbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:24.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.082+0000 7f8dc0c00640 1 -- 172.21.15.62:0/1758693731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc014620 con 0x7f8dc415bdf0 2024-04-25T09:17:24.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.083+0000 7f8dc0c00640 1 -- 172.21.15.62:0/1758693731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc019070 con 0x7f8dc415bdf0 2024-04-25T09:17:24.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.083+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc41f7630 con 0x7f8dc415bdf0 2024-04-25T09:17:24.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.083+0000 7f8dc0c00640 1 -- 172.21.15.62:0/1758693731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc014a20 con 0x7f8dc415bdf0 2024-04-25T09:17:24.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.083+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc41f7bf0 con 0x7f8dc415bdf0 2024-04-25T09:17:24.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.084+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc404ef20 con 0x7f8dc415bdf0 2024-04-25T09:17:24.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.085+0000 7f8dc0c00640 1 -- 172.21.15.62:0/1758693731 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8dbc014bc0 con 0x7f8dc415bdf0 2024-04-25T09:17:24.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.086+0000 7f8dc0c00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8da406b4d0 0x7f8da406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.086+0000 7f8dc0c00640 1 -- 172.21.15.62:0/1758693731 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8dbc08a810 con 0x7f8dc415bdf0 2024-04-25T09:17:24.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.087+0000 7f8dc9000640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8da406b4d0 0x7f8da406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.088+0000 7f8dc9000640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8da406b4d0 0x7f8da406d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8db8034000 tx=0x7f8db802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:24.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.091+0000 7f8dc0c00640 1 -- 172.21.15.62:0/1758693731 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8dbc0599c0 con 0x7f8dc415bdf0 2024-04-25T09:17:24.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.289+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8dc406fd80 con 0x7f8dc415bdf0 2024-04-25T09:17:24.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.294+0000 7f8dc0c00640 1 -- 172.21.15.62:0/1758693731 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8dbc0592c0 con 0x7f8dc415bdf0 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8da406b4d0 msgr2=0x7f8da406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8da406b4d0 0x7f8da406d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8db8034000 tx=0x7f8db802e000 comp rx=0 tx=0).stop 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 msgr2=0x7f8dc40ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40ae0c0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc007e40 tx=0x7f8dbc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 shutdown_connections 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8da406b4d0 0x7f8da406d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dc415bdf0 0x7f8dc40ae0c0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8dc40cebc0 0x7f8dc40adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 --2- 172.21.15.62:0/1758693731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8dc406c9b0 0x7f8dc4145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.297+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 >> 172.21.15.62:0/1758693731 conn(0x7f8dc40b1410 msgr2=0x7f8dc4159630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.298+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 shutdown_connections 2024-04-25T09:17:24.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:24.298+0000 7f8dcba00640 1 -- 172.21.15.62:0/1758693731 wait complete. 2024-04-25T09:17:24.304 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:17:24.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:17:24.419 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:17:24.534 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:17:24.534 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:17:24.534 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:24.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:17:24.594 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:17:24.595 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:24.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:17:24.652 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:17:24.652 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:24.712 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:17:24.712 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:17:24.712 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:24.769 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:17:24.769 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:17:24.769 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:24.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.004+0000 7ff8b0a00640 1 Processor -- start 2024-04-25T09:17:24.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.004+0000 7ff8b0a00640 1 -- start start 2024-04-25T09:17:24.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.006+0000 7ff8b0a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff8ac15f450 0x7ff8ac0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.006+0000 7ff8b0a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac15faa0 con 0x7ff8ac144c10 2024-04-25T09:17:24.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.006+0000 7ff8b0a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac15fd80 con 0x7ff8ac06d900 2024-04-25T09:17:24.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.006+0000 7ff8b0a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ac144ff0 con 0x7ff8ac15f450 2024-04-25T09:17:24.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.006+0000 7ff8aa800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff8ac144c10 0x7ff8ac06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42722/0 (socket says 172.21.15.62:42722) 2024-04-25T09:17:24.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.006+0000 7ff8aa800640 1 -- 172.21.15.62:0/3122312304 learned_addr learned my addr 172.21.15.62:0/3122312304 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:24.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476237127 0 0) 0x7ff8ac15faa0 con 0x7ff8ac144c10 2024-04-25T09:17:24.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8ab200640 1 --2- 172.21.15.62:0/3122312304 >> v2:172.21.15.121:3300/0 conn(0x7ff8ac15f450 0x7ff8ac0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff890003520 con 0x7ff8ac144c10 2024-04-25T09:17:24.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874343673 0 0) 0x7ff890003520 con 0x7ff8ac144c10 2024-04-25T09:17:24.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 >> v2:172.21.15.121:3300/0 conn(0x7ff8ac15f450 msgr2=0x7ff8ac0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 --2- 172.21.15.62:0/3122312304 >> v2:172.21.15.121:3300/0 conn(0x7ff8ac15f450 0x7ff8ac0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 >> v1:172.21.15.121:6789/0 conn(0x7ff8ac06d900 legacy=0x7ff8ac0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac160060 con 0x7ff8ac144c10 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ff89c001670 con 0x7ff8ac144c10 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 >> v1:172.21.15.62:6789/0 conn(0x7ff8ac144c10 legacy=0x7ff8ac06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff890003a00 0x7ff890003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff890004800 0x7ff890004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8900052e0 0x7ff89000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac144ff0 con 0x7ff890003a00 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89000c200 con 0x7ff8900052e0 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89000c400 con 0x7ff890004800 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.007+0000 7ff8aa800640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff890003a00 0x7ff890003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8ab200640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8900052e0 0x7ff89000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8a1e00640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff890004800 0x7ff890004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8aa800640 1 -- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff890004800 msgr2=0x7ff890004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8aa800640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff890004800 0x7ff890004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8aa800640 1 -- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8900052e0 msgr2=0x7ff89000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8aa800640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8900052e0 0x7ff89000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8aa800640 1 -- 172.21.15.62:0/3122312304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff890003520 con 0x7ff890003a00 2024-04-25T09:17:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8ab200640 1 -- 172.21.15.62:0/3122312304 reap_dead start 2024-04-25T09:17:24.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.008+0000 7ff8a1e00640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff890004800 0x7ff890004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.009+0000 7ff8aa800640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff890003a00 0x7ff890003e20 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c0e0 tx=0x7ff890005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfba859e7e579de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:24.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.009+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c007c20 con 0x7ff890003a00 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.009+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c005af0 con 0x7ff890003a00 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.009+0000 7ff8a9e00640 1 -- 172.21.15.62:0/3122312304 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c007070 con 0x7ff890003a00 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.010+0000 7ff8b0a00640 1 -- 172.21.15.62:0/3122312304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff890003a00 msgr2=0x7ff890003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.010+0000 7ff8b0a00640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff890003a00 0x7ff890003e20 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c0e0 tx=0x7ff890005830 comp rx=0 tx=0).stop 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.010+0000 7ff8b0a00640 1 -- 172.21.15.62:0/3122312304 shutdown_connections 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.010+0000 7ff8b0a00640 1 --2- 172.21.15.62:0/3122312304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff890003a00 0x7ff890003e20 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.010+0000 7ff8b0a00640 1 -- 172.21.15.62:0/3122312304 >> 172.21.15.62:0/3122312304 conn(0x7ff8ac0b14d0 msgr2=0x7ff8ac0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.010+0000 7ff8b0a00640 1 -- 172.21.15.62:0/3122312304 shutdown_connections 2024-04-25T09:17:24.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.010+0000 7ff8b0a00640 1 -- 172.21.15.62:0/3122312304 wait complete. 2024-04-25T09:17:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.012+0000 7ff8b0a00640 1 Processor -- start 2024-04-25T09:17:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.012+0000 7ff8b0a00640 1 -- start start 2024-04-25T09:17:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.013+0000 7ff8b0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff890003a00 0x7ff8ac13ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.013+0000 7ff8b0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8ac143710 0x7ff8ac0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.013+0000 7ff8b0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ac143b90 0x7ff8ac0ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.013+0000 7ff8b0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0cf050 con 0x7ff8ac143b90 2024-04-25T09:17:25.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.013+0000 7ff8b0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac15f880 con 0x7ff890003a00 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.013+0000 7ff8b0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac0cf1d0 con 0x7ff8ac143710 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ac143b90 0x7ff8ac0ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ac143b90 0x7ff8ac0ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34186/0 (socket says 172.21.15.62:34186) 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8ab200640 1 -- 172.21.15.62:0/1983855274 learned_addr learned my addr 172.21.15.62:0/1983855274 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8aa800640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff890003a00 0x7ff8ac13ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8a1e00640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8ac143710 0x7ff8ac0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8ab200640 1 -- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8ac143710 msgr2=0x7ff8ac0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8ab200640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8ac143710 0x7ff8ac0c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8ab200640 1 -- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff890003a00 msgr2=0x7ff8ac13ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8ab200640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff890003a00 0x7ff8ac13ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.014+0000 7ff8ab200640 1 -- 172.21.15.62:0/1983855274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac150790 con 0x7ff8ac143b90 2024-04-25T09:17:25.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.015+0000 7ff8ab200640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ac143b90 0x7ff8ac0ca4d0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0c5580 tx=0x7ff8a4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:25.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.015+0000 7ff8a3400640 1 -- 172.21.15.62:0/1983855274 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4007cf0 con 0x7ff8ac143b90 2024-04-25T09:17:25.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.015+0000 7ff8a3400640 1 -- 172.21.15.62:0/1983855274 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a4019070 con 0x7ff8ac143b90 2024-04-25T09:17:25.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.015+0000 7ff8a3400640 1 -- 172.21.15.62:0/1983855274 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4014650 con 0x7ff8ac143b90 2024-04-25T09:17:25.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.015+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac1f7750 con 0x7ff8ac143b90 2024-04-25T09:17:25.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.015+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac1f7d10 con 0x7ff8ac143b90 2024-04-25T09:17:25.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.016+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac06df20 con 0x7ff8ac143b90 2024-04-25T09:17:25.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.018+0000 7ff8a3400640 1 -- 172.21.15.62:0/1983855274 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8a40147f0 con 0x7ff8ac143b90 2024-04-25T09:17:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.019+0000 7ff8a3400640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff88806b4d0 0x7ff88806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.019+0000 7ff8a3400640 1 -- 172.21.15.62:0/1983855274 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff8a408a4c0 con 0x7ff8ac143b90 2024-04-25T09:17:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.019+0000 7ff8aa800640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff88806b4d0 0x7ff88806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.021+0000 7ff8aa800640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff88806b4d0 0x7ff88806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff89c032000 tx=0x7ff89c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:25.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.023+0000 7ff8a3400640 1 -- 172.21.15.62:0/1983855274 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8ac06df20 con 0x7ff8ac143b90 2024-04-25T09:17:25.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.263+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8ac1f7f10 con 0x7ff8ac143b90 2024-04-25T09:17:25.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.265+0000 7ff8a3400640 1 -- 172.21.15.62:0/1983855274 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+2914 (secure 0 0 0) 0x7ff8ac1f7f10 con 0x7ff8ac143b90 2024-04-25T09:17:25.252 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:25.252 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6056,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/2288148513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":2288148513},{"type":"v1","addr":"172.21.15.62:6835","nonce":2288148513}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6179,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1435804750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":1435804750},{"type":"v1","addr":"172.21.15.121:6837","nonce":1435804750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6811,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/3008714094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3008714094},{"type":"v1","addr":"172.21.15.121:6836","nonce":3008714094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[]} 2024-04-25T09:17:25.252 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 92 2024-04-25T09:17:25.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff88806b4d0 msgr2=0x7ff88806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff88806b4d0 0x7ff88806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff89c032000 tx=0x7ff89c009030 comp rx=0 tx=0).stop 2024-04-25T09:17:25.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ac143b90 msgr2=0x7ff8ac0ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ac143b90 0x7ff8ac0ca4d0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0c5580 tx=0x7ff8a4009cd0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 shutdown_connections 2024-04-25T09:17:25.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff88806b4d0 0x7ff88806d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff8ac143b90 0x7ff8ac0ca4d0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff8ac143710 0x7ff8ac0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 --2- 172.21.15.62:0/1983855274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff890003a00 0x7ff8ac13ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 >> 172.21.15.62:0/1983855274 conn(0x7ff8ac0b14d0 msgr2=0x7ff8ac06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:25.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 shutdown_connections 2024-04-25T09:17:25.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.268+0000 7ff8b0a00640 1 -- 172.21.15.62:0/1983855274 wait complete. 2024-04-25T09:17:25.279 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:17:25.279 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:17:25.279 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:17:25.279 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:17:25.279 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:17:25.280 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:17:25.280 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:17:25.280 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:17:25.280 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:17:25.280 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:17:25.280 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:17:25.281 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:17:25.281 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:17:25.281 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:17:25.285 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:17:25.285 DEBUG:teuthology.parallel:result is None 2024-04-25T09:17:25.301 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:17:25.301 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:17:25.302 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:17:25.302 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:17:25.306 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:17:25.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:17:25.346 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:17:25.347 DEBUG:teuthology.parallel:result is None 2024-04-25T09:17:25.353 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:17:25.353 DEBUG:teuthology.parallel:result is None 2024-04-25T09:17:25.359 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:17:25.359 DEBUG:teuthology.parallel:result is None 2024-04-25T09:17:25.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:17:25.439 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:17:25.494 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:17:25.508 INFO:tasks.ceph.mds.b.smithi121.stdout:starting mds.b at 2024-04-25T09:17:25.508 INFO:tasks.ceph.mds.d.smithi121.stdout:starting mds.d at 2024-04-25T09:17:25.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.544+0000 7f74a7000640 1 Processor -- start 2024-04-25T09:17:25.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.545+0000 7f74a7000640 1 -- start start 2024-04-25T09:17:25.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a7000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a015dc40 0x7f74a00d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a7000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f74a00d0ae0 0x7f74a00d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a7000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74a00d3ae0 0x7f74a00d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a7000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a00d1480 con 0x7f74a00d3ae0 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a7000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a00d2620 con 0x7f74a00d0ae0 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a7000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00d2900 con 0x7f74a015dc40 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a5000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74a00d3ae0 0x7f74a00d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a5000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74a00d3ae0 0x7f74a00d3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34244/0 (socket says 172.21.15.62:34244) 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a5000640 1 -- 172.21.15.62:0/2414148484 learned_addr learned my addr 172.21.15.62:0/2414148484 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f749fe00640 1 --2- 172.21.15.62:0/2414148484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a015dc40 0x7f74a00d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f7497e00640 1 --2- 172.21.15.62:0/2414148484 >> v2:172.21.15.121:3300/0 conn(0x7f74a00d0ae0 0x7f74a00d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a5000640 1 -- 172.21.15.62:0/2414148484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a015dc40 msgr2=0x7f74a00d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a5000640 1 --2- 172.21.15.62:0/2414148484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a015dc40 0x7f74a00d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a5000640 1 -- 172.21.15.62:0/2414148484 >> v2:172.21.15.121:3300/0 conn(0x7f74a00d0ae0 msgr2=0x7f74a00d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:25.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.546+0000 7f74a5000640 1 --2- 172.21.15.62:0/2414148484 >> v2:172.21.15.121:3300/0 conn(0x7f74a00d0ae0 0x7f74a00d0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.547+0000 7f74a5000640 1 -- 172.21.15.62:0/2414148484 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a00d2be0 con 0x7f74a00d3ae0 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.547+0000 7f74a5000640 1 --2- 172.21.15.62:0/2414148484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a00d3ae0 0x7f74a00d3f70 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f74a00d4c10 tx=0x7f749802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eea9bb21db1d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f749f400640 1 -- 172.21.15.62:0/2414148484 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498036070 con 0x7f74a00d3ae0 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f749f400640 1 -- 172.21.15.62:0/2414148484 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7498031600 con 0x7f74a00d3ae0 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f749f400640 1 -- 172.21.15.62:0/2414148484 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74980319d0 con 0x7f74a00d3ae0 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f74a7000640 1 -- 172.21.15.62:0/2414148484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a00d3ae0 msgr2=0x7f74a00d3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f74a7000640 1 --2- 172.21.15.62:0/2414148484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a00d3ae0 0x7f74a00d3f70 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f74a00d4c10 tx=0x7f749802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f74a7000640 1 -- 172.21.15.62:0/2414148484 shutdown_connections 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f74a7000640 1 --2- 172.21.15.62:0/2414148484 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a00d3ae0 0x7f74a00d3f70 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f74a7000640 1 --2- 172.21.15.62:0/2414148484 >> v2:172.21.15.121:3300/0 conn(0x7f74a00d0ae0 0x7f74a00d0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f74a7000640 1 --2- 172.21.15.62:0/2414148484 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a015dc40 0x7f74a00d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.548+0000 7f74a7000640 1 -- 172.21.15.62:0/2414148484 >> 172.21.15.62:0/2414148484 conn(0x7f74a006b000 msgr2=0x7f74a006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:25.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.549+0000 7f74a7000640 1 -- 172.21.15.62:0/2414148484 shutdown_connections 2024-04-25T09:17:25.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.549+0000 7f74a7000640 1 -- 172.21.15.62:0/2414148484 wait complete. 2024-04-25T09:17:25.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.550+0000 7f74a7000640 1 Processor -- start 2024-04-25T09:17:25.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.550+0000 7f74a7000640 1 -- start start 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.551+0000 7f74a7000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a00d0ae0 0x7f74a00c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.551+0000 7f74a7000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a00d3ae0 0x7f74a015e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.551+0000 7f74a7000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a015dc40 0x7f74a015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.551+0000 7f74a7000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00d36c0 con 0x7f74a015dc40 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.551+0000 7f74a7000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00d31a0 con 0x7f74a00d0ae0 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.551+0000 7f74a7000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a0155e70 con 0x7f74a00d3ae0 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.551+0000 7f74a5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a015dc40 0x7f74a015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f74a5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a015dc40 0x7f74a015ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34246/0 (socket says 172.21.15.62:34246) 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f74a5000640 1 -- 172.21.15.62:0/3367813509 learned_addr learned my addr 172.21.15.62:0/3367813509 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:25.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f7497e00640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a00d3ae0 0x7f74a015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f749fe00640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a00d0ae0 0x7f74a00c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f74a5000640 1 -- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a00d3ae0 msgr2=0x7f74a015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f74a5000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a00d3ae0 0x7f74a015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f74a5000640 1 -- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a00d0ae0 msgr2=0x7f74a00c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f74a5000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a00d0ae0 0x7f74a00c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f74a5000640 1 -- 172.21.15.62:0/3367813509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a00c6b00 con 0x7f74a015dc40 2024-04-25T09:17:25.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.552+0000 7f74a5000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a015dc40 0x7f74a015ee50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f749802ac90 tx=0x7f7498004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:25.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.553+0000 7f749cc00640 1 -- 172.21.15.62:0/3367813509 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749803f020 con 0x7f74a015dc40 2024-04-25T09:17:25.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.553+0000 7f749cc00640 1 -- 172.21.15.62:0/3367813509 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7498037040 con 0x7f74a015dc40 2024-04-25T09:17:25.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.553+0000 7f749cc00640 1 -- 172.21.15.62:0/3367813509 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498036070 con 0x7f74a015dc40 2024-04-25T09:17:25.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.553+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a015a820 con 0x7f74a015dc40 2024-04-25T09:17:25.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.553+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a0155540 con 0x7f74a015dc40 2024-04-25T09:17:25.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.554+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a00729c0 con 0x7f74a015dc40 2024-04-25T09:17:25.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.555+0000 7f749cc00640 1 -- 172.21.15.62:0/3367813509 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7498042020 con 0x7f74a015dc40 2024-04-25T09:17:25.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.556+0000 7f749cc00640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f747c06b4d0 0x7f747c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:25.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.556+0000 7f749cc00640 1 -- 172.21.15.62:0/3367813509 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f74980aa150 con 0x7f74a015dc40 2024-04-25T09:17:25.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.557+0000 7f749fe00640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f747c06b4d0 0x7f747c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:25.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.557+0000 7f749fe00640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f747c06b4d0 0x7f747c06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7490009a20 tx=0x7f7490009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:25.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.561+0000 7f749cc00640 1 -- 172.21.15.62:0/3367813509 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7498049030 con 0x7f74a015dc40 2024-04-25T09:17:25.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.764+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f74a0067620 con 0x7f74a015dc40 2024-04-25T09:17:25.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.871+0000 7f749cc00640 1 -- 172.21.15.62:0/3367813509 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v143) v1 ==== 97+0+0 (secure 0 0 0) 0x7f74a00c6b00 con 0x7f74a015dc40 2024-04-25T09:17:25.858 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-25T09:17:25.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f747c06b4d0 msgr2=0x7f747c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f747c06b4d0 0x7f747c06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7490009a20 tx=0x7f7490009190 comp rx=0 tx=0).stop 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a015dc40 msgr2=0x7f74a015ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a015dc40 0x7f74a015ee50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f749802ac90 tx=0x7f7498004a40 comp rx=0 tx=0).stop 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 shutdown_connections 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f747c06b4d0 0x7f747c06d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74a015dc40 0x7f74a015ee50 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f74a00d3ae0 0x7f74a015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 --2- 172.21.15.62:0/3367813509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74a00d0ae0 0x7f74a00c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:25.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 >> 172.21.15.62:0/3367813509 conn(0x7f74a006b000 msgr2=0x7f74a00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 shutdown_connections 2024-04-25T09:17:25.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:25.874+0000 7f74a7000640 1 -- 172.21.15.62:0/3367813509 wait complete. 2024-04-25T09:17:25.879 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:17:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.085+0000 7f20ef200640 1 Processor -- start 2024-04-25T09:17:26.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.085+0000 7f20ef200640 1 -- start start 2024-04-25T09:17:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.087+0000 7f20ef200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f20f015b960 0x7f20f006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.087+0000 7f20ef200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 0x7f20f00b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.087+0000 7f20ef200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f006d4f0 con 0x7f20f00b4610 2024-04-25T09:17:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.087+0000 7f20ef200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f015bd40 con 0x7f20f015b960 2024-04-25T09:17:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.087+0000 7f20ef200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00b4a60 con 0x7f20f00ae9d0 2024-04-25T09:17:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.088+0000 7f20ee800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f20f00b4610 0x7f20f015f450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42754/0 (socket says 172.21.15.62:42754) 2024-04-25T09:17:26.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.088+0000 7f20ee800640 1 -- 172.21.15.62:0/1213479585 learned_addr learned my addr 172.21.15.62:0/1213479585 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.088+0000 7f20e5400640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 0x7f20f00b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.088+0000 7f20ede00640 1 --2- 172.21.15.62:0/1213479585 >> v2:172.21.15.121:3300/0 conn(0x7f20f015b960 0x7f20f006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.088+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016800193 0 0) 0x7f20f006d4f0 con 0x7f20f00b4610 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.088+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d4003520 con 0x7f20f00b4610 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1545974933 0 0) 0x7f20d4003520 con 0x7f20f00b4610 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 msgr2=0x7f20f00b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 0x7f20f00b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 >> v2:172.21.15.121:3300/0 conn(0x7f20f015b960 msgr2=0x7f20f006ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:26.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 --2- 172.21.15.62:0/1213479585 >> v2:172.21.15.121:3300/0 conn(0x7f20f015b960 0x7f20f006ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00b2500 con 0x7f20f00b4610 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f20e80019f0 con 0x7f20f00b4610 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 >> v1:172.21.15.62:6789/0 conn(0x7f20f00b4610 legacy=0x7f20f015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d4003a20 0x7f20d4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 0x7f20d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4004d40 0x7f20d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00b4a60 con 0x7f20d4003a20 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d400b9c0 con 0x7f20d4004d40 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d400bb40 con 0x7f20f00ae9d0 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ede00640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d4003a20 0x7f20d4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20e5400640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 0x7f20d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ee800640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4004d40 0x7f20d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ede00640 1 -- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 msgr2=0x7f20d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:26.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ede00640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 0x7f20d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.089+0000 7f20ede00640 1 -- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4004d40 msgr2=0x7f20d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:26.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.090+0000 7f20ede00640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4004d40 0x7f20d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.090+0000 7f20ede00640 1 -- 172.21.15.62:0/1213479585 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4003520 con 0x7f20d4003a20 2024-04-25T09:17:26.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.090+0000 7f20ee800640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4004d40 0x7f20d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:26.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.090+0000 7f20e5400640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20f00ae9d0 0x7f20d4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.091+0000 7f20ede00640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d4003a20 0x7f20d4003e40 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009ac0 tx=0x7f20d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c69a7c3e765821b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.091+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0036070 con 0x7f20d4003a20 2024-04-25T09:17:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.091+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e0031400 con 0x7f20d4003a20 2024-04-25T09:17:26.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.091+0000 7f20ed400640 1 -- 172.21.15.62:0/1213479585 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0009690 con 0x7f20d4003a20 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.092+0000 7f20ef200640 1 -- 172.21.15.62:0/1213479585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d4003a20 msgr2=0x7f20d4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.092+0000 7f20ef200640 1 --2- 172.21.15.62:0/1213479585 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20d4003a20 0x7f20d4003e40 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009ac0 tx=0x7f20d4005830 comp rx=0 tx=0).stop 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.092+0000 7f20ee800640 1 -- 172.21.15.62:0/1213479585 reap_dead start 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.092+0000 7f20ef200640 1 -- 172.21.15.62:0/1213479585 shutdown_connections 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.092+0000 7f20ef200640 1 -- 172.21.15.62:0/1213479585 >> 172.21.15.62:0/1213479585 conn(0x7f20f0074380 msgr2=0x7f20f00668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.092+0000 7f20ef200640 1 -- 172.21.15.62:0/1213479585 shutdown_connections 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.092+0000 7f20ef200640 1 -- 172.21.15.62:0/1213479585 wait complete. 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.093+0000 7f20ef200640 1 Processor -- start 2024-04-25T09:17:26.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.094+0000 7f20ef200640 1 -- start start 2024-04-25T09:17:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.094+0000 7f20ef200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20d4003a20 0x7f20f014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.094+0000 7f20ef200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015dfe0 0x7f20f015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.094+0000 7f20ef200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f01505c0 0x7f20f0150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.094+0000 7f20ef200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00b2ac0 con 0x7f20f015dfe0 2024-04-25T09:17:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.094+0000 7f20ef200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00b3110 con 0x7f20f01505c0 2024-04-25T09:17:26.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.094+0000 7f20ef200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00b2da0 con 0x7f20d4003a20 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20e5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015dfe0 0x7f20f015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20e5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015dfe0 0x7f20f015e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34270/0 (socket says 172.21.15.62:34270) 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20e5400640 1 -- 172.21.15.62:0/477913262 learned_addr learned my addr 172.21.15.62:0/477913262 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20ede00640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20d4003a20 0x7f20f014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20ee800640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f01505c0 0x7f20f0150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20e5400640 1 -- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20d4003a20 msgr2=0x7f20f014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20e5400640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20d4003a20 0x7f20f014e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20e5400640 1 -- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f01505c0 msgr2=0x7f20f0150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20e5400640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f01505c0 0x7f20f0150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20e5400640 1 -- 172.21.15.62:0/477913262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f01518b0 con 0x7f20f015dfe0 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20ee800640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f01505c0 0x7f20f0150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.095+0000 7f20ede00640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20d4003a20 0x7f20f014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.096+0000 7f20e5400640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015dfe0 0x7f20f015e460 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f20d8010910 tx=0x7f20d8010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.096+0000 7f20e6a00640 1 -- 172.21.15.62:0/477913262 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d800f040 con 0x7f20f015dfe0 2024-04-25T09:17:26.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.096+0000 7f20e6a00640 1 -- 172.21.15.62:0/477913262 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d800eb00 con 0x7f20f015dfe0 2024-04-25T09:17:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.096+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f01512f0 con 0x7f20f015dfe0 2024-04-25T09:17:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.096+0000 7f20e6a00640 1 -- 172.21.15.62:0/477913262 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d801c400 con 0x7f20f015dfe0 2024-04-25T09:17:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.096+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f015bee0 con 0x7f20f015dfe0 2024-04-25T09:17:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.097+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f0074790 con 0x7f20f015dfe0 2024-04-25T09:17:26.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.098+0000 7f20e6a00640 1 -- 172.21.15.62:0/477913262 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20d801c5a0 con 0x7f20f015dfe0 2024-04-25T09:17:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.099+0000 7f20e6a00640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20cc06b4d0 0x7f20cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.099+0000 7f20e6a00640 1 -- 172.21.15.62:0/477913262 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f20d808a530 con 0x7f20f015dfe0 2024-04-25T09:17:26.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.100+0000 7f20ede00640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20cc06b4d0 0x7f20cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.101+0000 7f20ede00640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20cc06b4d0 0x7f20cc06d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f20e0033000 tx=0x7f20e0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:26.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.103+0000 7f20e6a00640 1 -- 172.21.15.62:0/477913262 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20f0074790 con 0x7f20f015dfe0 2024-04-25T09:17:26.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.369+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f20f0075070 con 0x7f20f015dfe0 2024-04-25T09:17:26.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.370+0000 7f20e6a00640 1 -- 172.21.15.62:0/477913262 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f20d8018030 con 0x7f20f015dfe0 2024-04-25T09:17:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:26.358 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:17:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-25T09:17:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.358 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-25T09:17:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmxB/lKxAAUX1IjQ+2pJciivGYoAHsmQ==", 2024-04-25T09:17:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:17:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:17:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:17:26.359 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-25T09:17:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmET2rChAAuejHYYo60I0SeamganEuCg==", 2024-04-25T09:17:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.360 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:17:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:17:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:17:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.361 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-25T09:17:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpmos4wNxAAfhenrCEAib8bGgA5/HLFJA==", 2024-04-25T09:17:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.362 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.363 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-25T09:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm2PwzGRAAZtawhgMEKWRc5rhnBMpYcA==", 2024-04-25T09:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:17:26.364 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:17:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-25T09:17:26.365 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAYHCpmiIo4HhAAHviq8XXEfEiMQTfkSLTdsQ==", 2024-04-25T09:17:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:17:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:17:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.366 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-25T09:17:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAbHCpmescXOhAAnkGMO9TamO1DhtzGfeZIsA==", 2024-04-25T09:17:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:17:26.367 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAfHCpmX5emGhAATVuEemM7gJh2p539lr+6nw==", 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:17:26.368 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:17:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-25T09:17:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAiHCpma0fvNRAAHMmCeqHxe6YxsBpef3Kh9w==", 2024-04-25T09:17:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.369 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:17:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:17:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-25T09:17:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApHCpm1iOjIhAAT3q/SbjWHKoB21lTvXUhJw==", 2024-04-25T09:17:26.370 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:17:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:17:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.371 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAtHCpms45gABAAbEUrdR4wXmDhiHDBD+496Q==", 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAwHCpm6npmGxAA1uwFl+oLceEkXhrBPilu1w==", 2024-04-25T09:17:26.372 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAzHCpm9ti1NBAAYu6VoKdKLTc/OZ4t2ZgoKg==", 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-25T09:17:26.373 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w==", 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.374 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmKuPONxAAoYYg7bZ65a31eFzxBBu+Fg==", 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUHCpmXChwLhAA1Ni66DztgejkU5m7vEcQ8A==", 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:17:26.375 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAVHCpmJvIdBhAAI6QX8XlnwDBSTMeKeqzMfg==", 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.376 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQASHCpmEZj2KxAAnxlUPTW32zwUKYkmHH/H1Q==", 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpm7DNFJBAAg6PBq491ikJgu3LYsIG/gA==", 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.377 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQATHCpms2wxGRAAKKJom6BulacxkPBRdPEJ4A==", 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.378 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:26.379 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T09:17:26.379 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:17:26.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.372+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20cc06b4d0 msgr2=0x7f20cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.372+0000 7f20ef200640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20cc06b4d0 0x7f20cc06d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f20e0033000 tx=0x7f20e0008030 comp rx=0 tx=0).stop 2024-04-25T09:17:26.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.372+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015dfe0 msgr2=0x7f20f015e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.372+0000 7f20ef200640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015dfe0 0x7f20f015e460 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f20d8010910 tx=0x7f20d8010dd0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.373+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 shutdown_connections 2024-04-25T09:17:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.373+0000 7f20ef200640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f20cc06b4d0 0x7f20cc06d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.373+0000 7f20ef200640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20f015dfe0 0x7f20f015e460 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.373+0000 7f20ef200640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20f01505c0 0x7f20f0150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.373+0000 7f20ef200640 1 --2- 172.21.15.62:0/477913262 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f20d4003a20 0x7f20f014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.373+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 >> 172.21.15.62:0/477913262 conn(0x7f20f0074380 msgr2=0x7f20f0155ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:26.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.373+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 shutdown_connections 2024-04-25T09:17:26.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.373+0000 7f20ef200640 1 -- 172.21.15.62:0/477913262 wait complete. 2024-04-25T09:17:26.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:26.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.607+0000 7fdaa7e00640 1 Processor -- start 2024-04-25T09:17:26.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.608+0000 7fdaa7e00640 1 -- start start 2024-04-25T09:17:26.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.609+0000 7fdaa7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdaa80b56f0 0x7fdaa8160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.609+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b27a0 0x7fdaa8161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.609+0000 7fdaa7e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa80d3110 con 0x7fdaa80b6a90 2024-04-25T09:17:26.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.609+0000 7fdaa7e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa8161780 con 0x7fdaa80b56f0 2024-04-25T09:17:26.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.609+0000 7fdaa7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80cedc0 con 0x7fdaa80b27a0 2024-04-25T09:17:26.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.610+0000 7fdaa7400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b27a0 0x7fdaa8161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.610+0000 7fdaa6000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fdaa80b6a90 0x7fdaa8161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:43288/0 (socket says 172.21.15.62:43288) 2024-04-25T09:17:26.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.610+0000 7fdaa6000640 1 -- 172.21.15.62:0/2056007644 learned_addr learned my addr 172.21.15.62:0/2056007644 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:26.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.610+0000 7fdaa6a00640 1 --2- 172.21.15.62:0/2056007644 >> v2:172.21.15.121:3300/0 conn(0x7fdaa80b56f0 0x7fdaa8160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.610+0000 7fdaa5600640 1 -- 172.21.15.62:0/2056007644 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592550135 0 0) 0x7fdaa80d3110 con 0x7fdaa80b6a90 2024-04-25T09:17:26.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.611+0000 7fdaa5600640 1 -- 172.21.15.62:0/2056007644 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda88003520 con 0x7fdaa80b6a90 2024-04-25T09:17:26.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.611+0000 7fdaa6a00640 1 -- 172.21.15.62:0/2056007644 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b27a0 msgr2=0x7fdaa8161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:26.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.611+0000 7fdaa6a00640 1 --2- 172.21.15.62:0/2056007644 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b27a0 0x7fdaa8161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.611+0000 7fdaa6a00640 1 -- 172.21.15.62:0/2056007644 >> v1:172.21.15.121:6789/0 conn(0x7fdaa80b6a90 legacy=0x7fdaa8161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.611+0000 7fdaa6a00640 1 -- 172.21.15.62:0/2056007644 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa80d33f0 con 0x7fdaa80b56f0 2024-04-25T09:17:26.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.612+0000 7fdaa6a00640 1 --2- 172.21.15.62:0/2056007644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 0x7fdaa8160d00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fda980097a0 tx=0x7fda9802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f49d0ce513068f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:26.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.612+0000 7fdaa5600640 1 -- 172.21.15.62:0/2056007644 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98037070 con 0x7fdaa80b56f0 2024-04-25T09:17:26.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa5600640 1 -- 172.21.15.62:0/2056007644 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda98032440 con 0x7fdaa80b56f0 2024-04-25T09:17:26.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa5600640 1 -- 172.21.15.62:0/2056007644 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda980327f0 con 0x7fdaa80b56f0 2024-04-25T09:17:26.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa7e00640 1 -- 172.21.15.62:0/2056007644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 msgr2=0x7fdaa8160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/2056007644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 0x7fdaa8160d00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fda980097a0 tx=0x7fda9802a180 comp rx=0 tx=0).stop 2024-04-25T09:17:26.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa7e00640 1 -- 172.21.15.62:0/2056007644 shutdown_connections 2024-04-25T09:17:26.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/2056007644 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b27a0 0x7fdaa8161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/2056007644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 0x7fdaa8160d00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa7e00640 1 -- 172.21.15.62:0/2056007644 >> 172.21.15.62:0/2056007644 conn(0x7fdaa8066d10 msgr2=0x7fdaa8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:26.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa7e00640 1 -- 172.21.15.62:0/2056007644 shutdown_connections 2024-04-25T09:17:26.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.613+0000 7fdaa7e00640 1 -- 172.21.15.62:0/2056007644 wait complete. 2024-04-25T09:17:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.615+0000 7fdaa7e00640 1 Processor -- start 2024-04-25T09:17:26.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.615+0000 7fdaa7e00640 1 -- start start 2024-04-25T09:17:26.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa80b27a0 0x7fdaa80cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 0x7fdaa8159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b6a90 0x7fdaa815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80b2d60 con 0x7fdaa80b27a0 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80cea50 con 0x7fdaa80b56f0 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80b2ee0 con 0x7fdaa80b6a90 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa80b27a0 0x7fdaa80cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa80b27a0 0x7fdaa80cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34274/0 (socket says 172.21.15.62:34274) 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.616+0000 7fdaa6a00640 1 -- 172.21.15.62:0/127593728 learned_addr learned my addr 172.21.15.62:0/127593728 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.617+0000 7fdaa7400640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b6a90 0x7fdaa815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.617+0000 7fdaa6000640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 0x7fdaa8159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.617+0000 7fdaa6a00640 1 -- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b6a90 msgr2=0x7fdaa815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.617+0000 7fdaa6a00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b6a90 0x7fdaa815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.617+0000 7fdaa6a00640 1 -- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 msgr2=0x7fdaa8159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.617+0000 7fdaa6a00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 0x7fdaa8159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.617+0000 7fdaa6a00640 1 -- 172.21.15.62:0/127593728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa80c66e0 con 0x7fdaa80b27a0 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.617+0000 7fdaa6a00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa80b27a0 0x7fdaa80cd9b0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fda98035040 tx=0x7fda980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.618+0000 7fda9ea00640 1 -- 172.21.15.62:0/127593728 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98037040 con 0x7fdaa80b27a0 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.618+0000 7fda9ea00640 1 -- 172.21.15.62:0/127593728 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda98042070 con 0x7fdaa80b27a0 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.618+0000 7fda9ea00640 1 -- 172.21.15.62:0/127593728 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9803b930 con 0x7fdaa80b27a0 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.618+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa80c69c0 con 0x7fdaa80b27a0 2024-04-25T09:17:26.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.618+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa80cac70 con 0x7fdaa80b27a0 2024-04-25T09:17:26.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.619+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa80d11d0 con 0x7fdaa80b27a0 2024-04-25T09:17:26.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.620+0000 7fda9ea00640 1 -- 172.21.15.62:0/127593728 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda9803bad0 con 0x7fdaa80b27a0 2024-04-25T09:17:26.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.621+0000 7fda9ea00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda8406b500 0x7fda8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:26.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.621+0000 7fda9ea00640 1 -- 172.21.15.62:0/127593728 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fda980aa740 con 0x7fdaa80b27a0 2024-04-25T09:17:26.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.621+0000 7fdaa6000640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda8406b500 0x7fda8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:26.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.623+0000 7fdaa6000640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda8406b500 0x7fda8406d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80d0a40 tx=0x7fda90009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:26.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.625+0000 7fda9ea00640 1 -- 172.21.15.62:0/127593728 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdaa80d11d0 con 0x7fdaa80b27a0 2024-04-25T09:17:26.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.825+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaa804ef20 con 0x7fdaa80b27a0 2024-04-25T09:17:26.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.827+0000 7fda9ea00640 1 -- 172.21.15.62:0/127593728 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+12889 (secure 0 0 0) 0x7fda98079870 con 0x7fdaa80b27a0 2024-04-25T09:17:26.815 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:26.815 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":144,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:25.875975+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda8406b500 msgr2=0x7fda8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda8406b500 0x7fda8406d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80d0a40 tx=0x7fda90009040 comp rx=0 tx=0).stop 2024-04-25T09:17:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa80b27a0 msgr2=0x7fdaa80cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:26.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa80b27a0 0x7fdaa80cd9b0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fda98035040 tx=0x7fda980046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 shutdown_connections 2024-04-25T09:17:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fda8406b500 0x7fda8406d9c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdaa80b6a90 0x7fdaa815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdaa80b56f0 0x7fdaa8159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 --2- 172.21.15.62:0/127593728 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdaa80b27a0 0x7fdaa80cd9b0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 >> 172.21.15.62:0/127593728 conn(0x7fdaa8066d10 msgr2=0x7fdaa80b8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.830+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 shutdown_connections 2024-04-25T09:17:26.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:26.831+0000 7fdaa7e00640 1 -- 172.21.15.62:0/127593728 wait complete. 2024-04-25T09:17:26.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:17:26.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:17:27.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.063+0000 7fbccc800640 1 Processor -- start 2024-04-25T09:17:27.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.063+0000 7fbccc800640 1 -- start start 2024-04-25T09:17:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.064+0000 7fbccc800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbcc415dc10 0x7fbcc40d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.064+0000 7fbccc800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbcc40d20a0 0x7fbcc40b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.064+0000 7fbccc800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc40b7590 con 0x7fbcc40b70d0 2024-04-25T09:17:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.064+0000 7fbccc800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc40b81a0 con 0x7fbcc40d20a0 2024-04-25T09:17:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbccc800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc40b7870 con 0x7fbcc415dc10 2024-04-25T09:17:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbcc40d20a0 0x7fbcc40b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbcc40d20a0 0x7fbcc40b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34286/0 (socket says 172.21.15.62:34286) 2024-04-25T09:17:27.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc9400640 1 -- 172.21.15.62:0/310450255 learned_addr learned my addr 172.21.15.62:0/310450255 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc9e00640 1 --2- 172.21.15.62:0/310450255 >> v2:172.21.15.121:3300/0 conn(0x7fbcc415dc10 0x7fbcc40d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc8a00640 1 -- 172.21.15.62:0/310450255 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694997766 0 0) 0x7fbcc40b7590 con 0x7fbcc40b70d0 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc8a00640 1 -- 172.21.15.62:0/310450255 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcac003520 con 0x7fbcc40b70d0 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc9400640 1 -- 172.21.15.62:0/310450255 >> v2:172.21.15.121:3300/0 conn(0x7fbcc415dc10 msgr2=0x7fbcc40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc9400640 1 --2- 172.21.15.62:0/310450255 >> v2:172.21.15.121:3300/0 conn(0x7fbcc415dc10 0x7fbcc40d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc9400640 1 -- 172.21.15.62:0/310450255 >> v1:172.21.15.62:6789/0 conn(0x7fbcc40b70d0 legacy=0x7fbcc40b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.065+0000 7fbcc9400640 1 -- 172.21.15.62:0/310450255 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc40b7b50 con 0x7fbcc40d20a0 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.067+0000 7fbcc9400640 1 --2- 172.21.15.62:0/310450255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40d20a0 0x7fbcc40b6b90 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fbcc408cd20 tx=0x7fbcb402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549da74ba22f5bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.067+0000 7fbcc8a00640 1 -- 172.21.15.62:0/310450255 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4036070 con 0x7fbcc40d20a0 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.067+0000 7fbcc8a00640 1 -- 172.21.15.62:0/310450255 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4031600 con 0x7fbcc40d20a0 2024-04-25T09:17:27.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.067+0000 7fbcc8a00640 1 -- 172.21.15.62:0/310450255 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40319b0 con 0x7fbcc40d20a0 2024-04-25T09:17:27.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.067+0000 7fbccc800640 1 -- 172.21.15.62:0/310450255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40d20a0 msgr2=0x7fbcc40b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.067+0000 7fbccc800640 1 --2- 172.21.15.62:0/310450255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40d20a0 0x7fbcc40b6b90 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fbcc408cd20 tx=0x7fbcb402a7c0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.055 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.068+0000 7fbccc800640 1 -- 172.21.15.62:0/310450255 shutdown_connections 2024-04-25T09:17:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.068+0000 7fbccc800640 1 --2- 172.21.15.62:0/310450255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40d20a0 0x7fbcc40b6b90 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.068+0000 7fbccc800640 1 --2- 172.21.15.62:0/310450255 >> v2:172.21.15.121:3300/0 conn(0x7fbcc415dc10 0x7fbcc40d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.068+0000 7fbccc800640 1 -- 172.21.15.62:0/310450255 >> 172.21.15.62:0/310450255 conn(0x7fbcc406b000 msgr2=0x7fbcc406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.068+0000 7fbccc800640 1 -- 172.21.15.62:0/310450255 shutdown_connections 2024-04-25T09:17:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.068+0000 7fbccc800640 1 -- 172.21.15.62:0/310450255 wait complete. 2024-04-25T09:17:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.069+0000 7fbccc800640 1 Processor -- start 2024-04-25T09:17:27.056 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.070+0000 7fbccc800640 1 -- start start 2024-04-25T09:17:27.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbccc800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40b70d0 0x7fbcc41f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbccc800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcc415dc10 0x7fbcc415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbccc800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcc40cda90 0x7fbcc41599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbccc800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc40b8d40 con 0x7fbcc40b70d0 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbccc800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc415dff0 con 0x7fbcc40cda90 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbccc800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc40b8ec0 con 0x7fbcc415dc10 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbcc9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40b70d0 0x7fbcc41f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbcc9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40b70d0 0x7fbcc41f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34298/0 (socket says 172.21.15.62:34298) 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbcc9e00640 1 -- 172.21.15.62:0/856313416 learned_addr learned my addr 172.21.15.62:0/856313416 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbcc9e00640 1 -- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcc415dc10 msgr2=0x7fbcc415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbcc9e00640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcc415dc10 0x7fbcc415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbcc9e00640 1 -- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcc40cda90 msgr2=0x7fbcc41599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbcc9e00640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcc40cda90 0x7fbcc41599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.071+0000 7fbcc9e00640 1 -- 172.21.15.62:0/856313416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc40cb3f0 con 0x7fbcc40b70d0 2024-04-25T09:17:27.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.072+0000 7fbcc9e00640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40b70d0 0x7fbcc41f7530 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800d5f0 tx=0x7fbcb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.072+0000 7fbcbe000640 1 -- 172.21.15.62:0/856313416 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80125f0 con 0x7fbcc40b70d0 2024-04-25T09:17:27.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.072+0000 7fbcbe000640 1 -- 172.21.15.62:0/856313416 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb8019070 con 0x7fbcc40b70d0 2024-04-25T09:17:27.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.072+0000 7fbcbe000640 1 -- 172.21.15.62:0/856313416 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80129d0 con 0x7fbcc40b70d0 2024-04-25T09:17:27.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.072+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc40cb6d0 con 0x7fbcc40b70d0 2024-04-25T09:17:27.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.072+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc40cbc90 con 0x7fbcc40b70d0 2024-04-25T09:17:27.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.073+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc404ef20 con 0x7fbcc40b70d0 2024-04-25T09:17:27.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.074+0000 7fbcbe000640 1 -- 172.21.15.62:0/856313416 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcb8012b70 con 0x7fbcc40b70d0 2024-04-25T09:17:27.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.075+0000 7fbcbe000640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbca806b4d0 0x7fbca806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.075+0000 7fbcbe000640 1 -- 172.21.15.62:0/856313416 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbcb808a2d0 con 0x7fbcc40b70d0 2024-04-25T09:17:27.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.076+0000 7fbcc9400640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbca806b4d0 0x7fbca806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.077+0000 7fbcc9400640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbca806b4d0 0x7fbca806d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4033000 tx=0x7fbcb402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.081+0000 7fbcbe000640 1 -- 172.21.15.62:0/856313416 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbcb8059490 con 0x7fbcc40b70d0 2024-04-25T09:17:27.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.286+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcc40cf120 con 0x7fbcc40b70d0 2024-04-25T09:17:27.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.287+0000 7fbcbe000640 1 -- 172.21.15.62:0/856313416 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbcb808a0d0 con 0x7fbcc40b70d0 2024-04-25T09:17:27.274 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:17:27.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbca806b4d0 msgr2=0x7fbca806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbca806b4d0 0x7fbca806d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4033000 tx=0x7fbcb402e000 comp rx=0 tx=0).stop 2024-04-25T09:17:27.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40b70d0 msgr2=0x7fbcc41f7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40b70d0 0x7fbcc41f7530 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800d5f0 tx=0x7fbcb800dab0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 shutdown_connections 2024-04-25T09:17:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbca806b4d0 0x7fbca806d990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcc40cda90 0x7fbcc41599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbcc415dc10 0x7fbcc415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 --2- 172.21.15.62:0/856313416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbcc40b70d0 0x7fbcc41f7530 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.289+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 >> 172.21.15.62:0/856313416 conn(0x7fbcc406b000 msgr2=0x7fbcc40d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.290+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 shutdown_connections 2024-04-25T09:17:27.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.290+0000 7fbccc800640 1 -- 172.21.15.62:0/856313416 wait complete. 2024-04-25T09:17:27.296 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:17:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.524+0000 7ff587200640 1 Processor -- start 2024-04-25T09:17:27.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.524+0000 7ff587200640 1 -- start start 2024-04-25T09:17:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.526+0000 7ff587200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff58806d560 0x7ff5880b2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.526+0000 7ff587200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff58815e7a0 0x7ff58815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.526+0000 7ff587200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58815be30 con 0x7ff58806d180 2024-04-25T09:17:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.526+0000 7ff587200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58815b2b0 con 0x7ff58806d560 2024-04-25T09:17:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.526+0000 7ff587200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58815b590 con 0x7ff58815e7a0 2024-04-25T09:17:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.526+0000 7ff585e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff58806d180 0x7ff58815c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42778/0 (socket says 172.21.15.62:42778) 2024-04-25T09:17:27.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.526+0000 7ff585e00640 1 -- 172.21.15.62:0/3501680185 learned_addr learned my addr 172.21.15.62:0/3501680185 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff586800640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff58815e7a0 0x7ff58815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff585400640 1 --2- 172.21.15.62:0/3501680185 >> v2:172.21.15.121:3300/0 conn(0x7ff58806d560 0x7ff5880b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27140685 0 0) 0x7ff58815be30 con 0x7ff58806d180 2024-04-25T09:17:27.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff56c003520 con 0x7ff58806d180 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3186692155 0 0) 0x7ff56c003520 con 0x7ff58806d180 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff58815e7a0 msgr2=0x7ff58815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff58815e7a0 0x7ff58815ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 >> v2:172.21.15.121:3300/0 conn(0x7ff58806d560 msgr2=0x7ff5880b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 --2- 172.21.15.62:0/3501680185 >> v2:172.21.15.121:3300/0 conn(0x7ff58806d560 0x7ff5880b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58815b870 con 0x7ff58806d180 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7ff578001670 con 0x7ff58806d180 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 >> v1:172.21.15.62:6789/0 conn(0x7ff58806d180 legacy=0x7ff58815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff58815e7a0 0x7ff56c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff56c003ff0 0x7ff56c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c004d40 0x7ff56c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58815b2b0 con 0x7ff56c003ff0 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58815b590 con 0x7ff56c004d40 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.527+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c00bac0 con 0x7ff58815e7a0 2024-04-25T09:17:27.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.528+0000 7ff585400640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff56c003ff0 0x7ff56c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.528+0000 7ff585e00640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff58815e7a0 0x7ff56c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.528+0000 7ff585400640 1 -- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff58815e7a0 msgr2=0x7ff56c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.528+0000 7ff585400640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff58815e7a0 0x7ff56c003a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.528+0000 7ff585400640 1 -- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c004d40 msgr2=0x7ff56c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.528+0000 7ff586800640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c004d40 0x7ff56c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.528+0000 7ff585400640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c004d40 0x7ff56c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.528+0000 7ff585400640 1 -- 172.21.15.62:0/3501680185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c003520 con 0x7ff56c003ff0 2024-04-25T09:17:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.529+0000 7ff585400640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff56c003ff0 0x7ff56c004800 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ff574009ac0 tx=0x7ff56c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e913b8a5f7341333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.529+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574036070 con 0x7ff56c003ff0 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff574031400 con 0x7ff56c003ff0 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff584a00640 1 -- 172.21.15.62:0/3501680185 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574009690 con 0x7ff56c003ff0 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff587200640 1 -- 172.21.15.62:0/3501680185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff56c003ff0 msgr2=0x7ff56c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff587200640 1 --2- 172.21.15.62:0/3501680185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff56c003ff0 0x7ff56c004800 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ff574009ac0 tx=0x7ff56c005830 comp rx=0 tx=0).stop 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff586800640 1 -- 172.21.15.62:0/3501680185 reap_dead start 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff587200640 1 -- 172.21.15.62:0/3501680185 shutdown_connections 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff587200640 1 -- 172.21.15.62:0/3501680185 >> 172.21.15.62:0/3501680185 conn(0x7ff588074380 msgr2=0x7ff5880668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff587200640 1 -- 172.21.15.62:0/3501680185 shutdown_connections 2024-04-25T09:17:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.530+0000 7ff587200640 1 -- 172.21.15.62:0/3501680185 wait complete. 2024-04-25T09:17:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.532+0000 7ff587200640 1 Processor -- start 2024-04-25T09:17:27.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.532+0000 7ff587200640 1 -- start start 2024-04-25T09:17:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.533+0000 7ff587200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c003ff0 0x7ff5880cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.533+0000 7ff587200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff588146d80 0x7ff5880ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.533+0000 7ff587200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff588147200 0x7ff5880ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.533+0000 7ff587200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880d0230 con 0x7ff588146d80 2024-04-25T09:17:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.533+0000 7ff587200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58806da10 con 0x7ff56c003ff0 2024-04-25T09:17:27.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.533+0000 7ff587200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58815ae80 con 0x7ff588147200 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff588146d80 0x7ff5880ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff588146d80 0x7ff5880ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34318/0 (socket says 172.21.15.62:34318) 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585400640 1 -- 172.21.15.62:0/834227111 learned_addr learned my addr 172.21.15.62:0/834227111 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585e00640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c003ff0 0x7ff5880cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff586800640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff588147200 0x7ff5880ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585400640 1 -- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff588147200 msgr2=0x7ff5880ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585400640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff588147200 0x7ff5880ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585400640 1 -- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c003ff0 msgr2=0x7ff5880cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585400640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c003ff0 0x7ff5880cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.534+0000 7ff585400640 1 -- 172.21.15.62:0/834227111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880cc4a0 con 0x7ff588146d80 2024-04-25T09:17:27.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.535+0000 7ff585400640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff588146d80 0x7ff5880ad8e0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7ff574000c00 tx=0x7ff574004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.535+0000 7ff57e000640 1 -- 172.21.15.62:0/834227111 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574040020 con 0x7ff588146d80 2024-04-25T09:17:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.535+0000 7ff57e000640 1 -- 172.21.15.62:0/834227111 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff574003be0 con 0x7ff588146d80 2024-04-25T09:17:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.535+0000 7ff57e000640 1 -- 172.21.15.62:0/834227111 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574036040 con 0x7ff588146d80 2024-04-25T09:17:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.535+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5880ceb30 con 0x7ff588146d80 2024-04-25T09:17:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.535+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5880cca40 con 0x7ff588146d80 2024-04-25T09:17:27.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.536+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5880cfde0 con 0x7ff588146d80 2024-04-25T09:17:27.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.537+0000 7ff57e000640 1 -- 172.21.15.62:0/834227111 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff574043020 con 0x7ff588146d80 2024-04-25T09:17:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.539+0000 7ff57e000640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff56806b500 0x7ff56806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.539+0000 7ff57e000640 1 -- 172.21.15.62:0/834227111 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff5740a9fe0 con 0x7ff588146d80 2024-04-25T09:17:27.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.540+0000 7ff585e00640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff56806b500 0x7ff56806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.541+0000 7ff585e00640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff56806b500 0x7ff56806d9c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff578001200 tx=0x7ff578009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.544+0000 7ff57e000640 1 -- 172.21.15.62:0/834227111 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5880cfde0 con 0x7ff588146d80 2024-04-25T09:17:27.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.743+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff588075070 con 0x7ff588146d80 2024-04-25T09:17:27.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.744+0000 7ff57e000640 1 -- 172.21.15.62:0/834227111 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v144) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff57404a070 con 0x7ff588146d80 2024-04-25T09:17:27.731 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:27.731 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-25T09:17:27.731 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-25T09:17:27.731 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-25T09:17:27.731 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-25T09:17:27.732 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T09:17:27.732 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-25T09:17:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff56806b500 msgr2=0x7ff56806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff56806b500 0x7ff56806d9c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff578001200 tx=0x7ff578009040 comp rx=0 tx=0).stop 2024-04-25T09:17:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff588146d80 msgr2=0x7ff5880ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff588146d80 0x7ff5880ad8e0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7ff574000c00 tx=0x7ff574004730 comp rx=0 tx=0).stop 2024-04-25T09:17:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 shutdown_connections 2024-04-25T09:17:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7ff56806b500 0x7ff56806d9c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff588146d80 0x7ff5880ad8e0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7ff588147200 0x7ff5880ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 --2- 172.21.15.62:0/834227111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff56c003ff0 0x7ff5880cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 >> 172.21.15.62:0/834227111 conn(0x7ff588074380 msgr2=0x7ff588156fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 shutdown_connections 2024-04-25T09:17:27.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.747+0000 7ff587200640 1 -- 172.21.15.62:0/834227111 wait complete. 2024-04-25T09:17:27.754 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:17:27.754 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:17:27.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.985+0000 7f800ca00640 1 Processor -- start 2024-04-25T09:17:27.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.985+0000 7f800ca00640 1 -- start start 2024-04-25T09:17:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.986+0000 7f800ca00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8008161650 0x7f80080d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.986+0000 7f800ca00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080d1480 con 0x7f80080b5c70 2024-04-25T09:17:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.986+0000 7f800ca00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080d11a0 con 0x7f8008161270 2024-04-25T09:17:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.986+0000 7f800ca00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80080d3900 con 0x7f8008161650 2024-04-25T09:17:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.987+0000 7f8006a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f80080b5c70 0x7f80080d1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42786/0 (socket says 172.21.15.62:42786) 2024-04-25T09:17:27.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.987+0000 7f8006a00640 1 -- 172.21.15.62:0/190881476 learned_addr learned my addr 172.21.15.62:0/190881476 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.987+0000 7f8007e00640 1 --2- 172.21.15.62:0/190881476 >> v2:172.21.15.121:3300/0 conn(0x7f8008161650 0x7f80080d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.987+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551212024 0 0) 0x7f80080d1480 con 0x7f80080b5c70 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.987+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe8003520 con 0x7f80080b5c70 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 371289680 0 0) 0x7f7fe8003520 con 0x7f80080b5c70 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 >> v2:172.21.15.121:3300/0 conn(0x7f8008161650 msgr2=0x7f80080d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 --2- 172.21.15.62:0/190881476 >> v2:172.21.15.121:3300/0 conn(0x7f8008161650 0x7f80080d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 >> v1:172.21.15.121:6789/0 conn(0x7f8008161270 legacy=0x7f80080b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080cee80 con 0x7f80080b5c70 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f7ff0001670 con 0x7f80080b5c70 2024-04-25T09:17:27.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 >> v1:172.21.15.62:6789/0 conn(0x7f80080b5c70 legacy=0x7f80080d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fe80039b0 0x7f7fe8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f7fe800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080d3900 con 0x7f7fe80052e0 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe800c1b0 con 0x7f7fe8004800 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe800c3b0 con 0x7f7fe80039b0 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.988+0000 7f8007e00640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f7fe800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.989+0000 7f8006a00640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.989+0000 7f8007400640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fe80039b0 0x7f7fe8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.989+0000 7f8007e00640 1 -- 172.21.15.62:0/190881476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fe80039b0 msgr2=0x7f7fe8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.989+0000 7f8007e00640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7fe80039b0 0x7f7fe8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.989+0000 7f8007e00640 1 -- 172.21.15.62:0/190881476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fe8004800 msgr2=0x7f7fe8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.989+0000 7f8007e00640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.989+0000 7f8007e00640 1 -- 172.21.15.62:0/190881476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8003520 con 0x7f7fe80052e0 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.990+0000 7f8007e00640 1 -- 172.21.15.62:0/190881476 reap_dead start 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.990+0000 7f8007e00640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f7fe800b9c0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f8008161ac0 tx=0x7f7fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b8422711207ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.990+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000009c10 con 0x7f7fe80052e0 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.990+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8000031410 con 0x7f7fe80052e0 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.990+0000 7f8006000640 1 -- 172.21.15.62:0/190881476 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000317c0 con 0x7f7fe80052e0 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.991+0000 7f800ca00640 1 -- 172.21.15.62:0/190881476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 msgr2=0x7f7fe800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.991+0000 7f800ca00640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f7fe800b9c0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f8008161ac0 tx=0x7f7fe8005830 comp rx=0 tx=0).stop 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.991+0000 7f800ca00640 1 -- 172.21.15.62:0/190881476 shutdown_connections 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.991+0000 7f800ca00640 1 --2- 172.21.15.62:0/190881476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f7fe800b9c0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.991+0000 7f800ca00640 1 -- 172.21.15.62:0/190881476 >> 172.21.15.62:0/190881476 conn(0x7f8008066d10 msgr2=0x7f8008067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:27.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.991+0000 7f800ca00640 1 -- 172.21.15.62:0/190881476 shutdown_connections 2024-04-25T09:17:27.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.991+0000 7f800ca00640 1 -- 172.21.15.62:0/190881476 wait complete. 2024-04-25T09:17:27.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.993+0000 7f800ca00640 1 Processor -- start 2024-04-25T09:17:27.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.993+0000 7f800ca00640 1 -- start start 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f800ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f80080cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f800ca00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8008157960 0x7f80080c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f800ca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80080c78d0 0x7f80080c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f800ca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080b2ed0 con 0x7f7fe80052e0 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f800ca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080d3590 con 0x7f80080c78d0 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f800ca00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80080b3050 con 0x7f8008157960 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f8007400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f80080cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f8007400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f80080cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34346/0 (socket says 172.21.15.62:34346) 2024-04-25T09:17:27.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f8007400640 1 -- 172.21.15.62:0/3384740905 learned_addr learned my addr 172.21.15.62:0/3384740905 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f8007400640 1 -- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8008157960 msgr2=0x7f80080c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f8007400640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8008157960 0x7f80080c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f8007400640 1 -- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80080c78d0 msgr2=0x7f80080c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f8007400640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80080c78d0 0x7f80080c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.994+0000 7f8007400640 1 -- 172.21.15.62:0/3384740905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080cc530 con 0x7f7fe80052e0 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.995+0000 7f8007400640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f80080cbc00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800d5a0 tx=0x7f7ff800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.995+0000 7f7fff400640 1 -- 172.21.15.62:0/3384740905 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8003610 con 0x7f7fe80052e0 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.995+0000 7f7fff400640 1 -- 172.21.15.62:0/3384740905 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff8017070 con 0x7f7fe80052e0 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.995+0000 7f7fff400640 1 -- 172.21.15.62:0/3384740905 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80039f0 con 0x7f7fe80052e0 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.995+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80080cc810 con 0x7f7fe80052e0 2024-04-25T09:17:27.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.995+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80080ccdd0 con 0x7f7fe80052e0 2024-04-25T09:17:27.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.996+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8008072910 con 0x7f7fe80052e0 2024-04-25T09:17:27.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.998+0000 7f7fff400640 1 -- 172.21.15.62:0/3384740905 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ff8003b90 con 0x7f7fe80052e0 2024-04-25T09:17:27.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.999+0000 7f7fff400640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fe406b4d0 0x7f7fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:27.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.999+0000 7f7fff400640 1 -- 172.21.15.62:0/3384740905 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ff8089d20 con 0x7f7fe80052e0 2024-04-25T09:17:27.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:27.999+0000 7f8006a00640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fe406b4d0 0x7f7fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:27.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.001+0000 7f8006a00640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fe406b4d0 0x7f7fe406d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000f60 tx=0x7f7ff0001070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:27.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.004+0000 7f7fff400640 1 -- 172.21.15.62:0/3384740905 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ff8058fa0 con 0x7f7fe80052e0 2024-04-25T09:17:28.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.205+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f80080b63a0 con 0x7f7fe80052e0 2024-04-25T09:17:28.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.891+0000 7f7fff400640 1 -- 172.21.15.62:0/3384740905 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v145) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7ff80588a0 con 0x7f7fe80052e0 2024-04-25T09:17:28.879 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.895+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fe406b4d0 msgr2=0x7f7fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.895+0000 7f800ca00640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fe406b4d0 0x7f7fe406d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0000f60 tx=0x7f7ff0001070 comp rx=0 tx=0).stop 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.895+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 msgr2=0x7f80080cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.895+0000 7f800ca00640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f80080cbc00 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800d5a0 tx=0x7f7ff800da60 comp rx=0 tx=0).stop 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.895+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 shutdown_connections 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.896+0000 7f800ca00640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f7fe406b4d0 0x7f7fe406d990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.896+0000 7f800ca00640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80080c78d0 0x7f80080c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.896+0000 7f800ca00640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8008157960 0x7f80080c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.896+0000 7f800ca00640 1 --2- 172.21.15.62:0/3384740905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7fe80052e0 0x7f80080cbc00 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:28.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.896+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 >> 172.21.15.62:0/3384740905 conn(0x7f8008066d10 msgr2=0x7f80080b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:28.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.896+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 shutdown_connections 2024-04-25T09:17:28.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:28.896+0000 7f800ca00640 1 -- 172.21.15.62:0/3384740905 wait complete. 2024-04-25T09:17:28.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:17:29.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.128+0000 7fe10ae00640 1 Processor -- start 2024-04-25T09:17:29.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.128+0000 7fe10ae00640 1 -- start start 2024-04-25T09:17:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.129+0000 7fe10ae00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe104154a40 0x7fe10415ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.129+0000 7fe10ae00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe104154e20 0x7fe1040af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:29.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.129+0000 7fe10ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1040b3b50 0x7fe1040668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.129+0000 7fe10ae00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10415e890 con 0x7fe104154e20 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.129+0000 7fe10ae00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1040afe20 con 0x7fe104154a40 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.129+0000 7fe10ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040b0100 con 0x7fe1040b3b50 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe104154e20 0x7fe1040af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe104154e20 0x7fe1040af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34352/0 (socket says 172.21.15.62:34352) 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109000640 1 -- 172.21.15.62:0/1823669501 learned_addr learned my addr 172.21.15.62:0/1823669501 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109000640 1 -- 172.21.15.62:0/1823669501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1040b3b50 msgr2=0x7fe1040668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe10a400640 1 --2- 172.21.15.62:0/1823669501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1040b3b50 0x7fe1040668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109a00640 1 --2- 172.21.15.62:0/1823669501 >> v2:172.21.15.121:3300/0 conn(0x7fe104154a40 0x7fe10415ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109000640 1 --2- 172.21.15.62:0/1823669501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1040b3b50 0x7fe1040668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109000640 1 -- 172.21.15.62:0/1823669501 >> v2:172.21.15.121:3300/0 conn(0x7fe104154a40 msgr2=0x7fe10415ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109000640 1 --2- 172.21.15.62:0/1823669501 >> v2:172.21.15.121:3300/0 conn(0x7fe104154a40 0x7fe10415ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109000640 1 -- 172.21.15.62:0/1823669501 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10415e520 con 0x7fe104154e20 2024-04-25T09:17:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe109a00640 1 --2- 172.21.15.62:0/1823669501 >> v2:172.21.15.121:3300/0 conn(0x7fe104154a40 0x7fe10415ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.130+0000 7fe10a400640 1 --2- 172.21.15.62:0/1823669501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1040b3b50 0x7fe1040668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.131+0000 7fe109000640 1 --2- 172.21.15.62:0/1823669501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe104154e20 0x7fe1040af520 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe104071610 tx=0x7fe0f802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91305f2d38b1ea7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.132+0000 7fe103e00640 1 -- 172.21.15.62:0/1823669501 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8036070 con 0x7fe104154e20 2024-04-25T09:17:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.132+0000 7fe103e00640 1 -- 172.21.15.62:0/1823669501 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8031600 con 0x7fe104154e20 2024-04-25T09:17:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.132+0000 7fe103e00640 1 -- 172.21.15.62:0/1823669501 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80319d0 con 0x7fe104154e20 2024-04-25T09:17:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.132+0000 7fe10ae00640 1 -- 172.21.15.62:0/1823669501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe104154e20 msgr2=0x7fe1040af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.132+0000 7fe10ae00640 1 --2- 172.21.15.62:0/1823669501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe104154e20 0x7fe1040af520 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe104071610 tx=0x7fe0f802a610 comp rx=0 tx=0).stop 2024-04-25T09:17:29.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.133+0000 7fe10ae00640 1 -- 172.21.15.62:0/1823669501 shutdown_connections 2024-04-25T09:17:29.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.133+0000 7fe10ae00640 1 --2- 172.21.15.62:0/1823669501 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe1040b3b50 0x7fe1040668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.133+0000 7fe10ae00640 1 --2- 172.21.15.62:0/1823669501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe104154e20 0x7fe1040af520 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.133+0000 7fe10ae00640 1 --2- 172.21.15.62:0/1823669501 >> v2:172.21.15.121:3300/0 conn(0x7fe104154a40 0x7fe10415ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.133+0000 7fe10ae00640 1 -- 172.21.15.62:0/1823669501 >> 172.21.15.62:0/1823669501 conn(0x7fe104074290 msgr2=0x7fe10406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:29.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.133+0000 7fe10ae00640 1 -- 172.21.15.62:0/1823669501 shutdown_connections 2024-04-25T09:17:29.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.133+0000 7fe10ae00640 1 -- 172.21.15.62:0/1823669501 wait complete. 2024-04-25T09:17:29.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.134+0000 7fe10ae00640 1 Processor -- start 2024-04-25T09:17:29.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.135+0000 7fe10ae00640 1 -- start start 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.135+0000 7fe10ae00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1040b3b50 0x7fe104142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.135+0000 7fe10ae00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe104154a40 0x7fe1040cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.135+0000 7fe10ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe104154e20 0x7fe1041f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.135+0000 7fe10ae00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040afbe0 con 0x7fe1040b3b50 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.135+0000 7fe10ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040afa60 con 0x7fe104154e20 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.135+0000 7fe10ae00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe104066e30 con 0x7fe104154a40 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1040b3b50 0x7fe104142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1040b3b50 0x7fe104142010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34354/0 (socket says 172.21.15.62:34354) 2024-04-25T09:17:29.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109a00640 1 -- 172.21.15.62:0/2892077514 learned_addr learned my addr 172.21.15.62:0/2892077514 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe10a400640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe104154e20 0x7fe1041f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109000640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe104154a40 0x7fe1040cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109a00640 1 -- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe104154a40 msgr2=0x7fe1040cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109a00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe104154a40 0x7fe1040cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109a00640 1 -- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe104154e20 msgr2=0x7fe1041f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109a00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe104154e20 0x7fe1041f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109a00640 1 -- 172.21.15.62:0/2892077514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104142550 con 0x7fe1040b3b50 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe10a400640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe104154e20 0x7fe1041f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.136+0000 7fe109000640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe104154a40 0x7fe1040cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.137+0000 7fe109a00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1040b3b50 0x7fe104142010 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400c7f0 tx=0x7fe0f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:29.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.137+0000 7fe101600640 1 -- 172.21.15.62:0/2892077514 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4015070 con 0x7fe1040b3b50 2024-04-25T09:17:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.137+0000 7fe101600640 1 -- 172.21.15.62:0/2892077514 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f4007a30 con 0x7fe1040b3b50 2024-04-25T09:17:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.137+0000 7fe101600640 1 -- 172.21.15.62:0/2892077514 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4007e00 con 0x7fe1040b3b50 2024-04-25T09:17:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.137+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe104142820 con 0x7fe1040b3b50 2024-04-25T09:17:29.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.137+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe104142de0 con 0x7fe1040b3b50 2024-04-25T09:17:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.138+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe104075070 con 0x7fe1040b3b50 2024-04-25T09:17:29.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.140+0000 7fe101600640 1 -- 172.21.15.62:0/2892077514 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0f4018070 con 0x7fe1040b3b50 2024-04-25T09:17:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.141+0000 7fe101600640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0e806b500 0x7fe0e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.141+0000 7fe101600640 1 -- 172.21.15.62:0/2892077514 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe0f4089ff0 con 0x7fe1040b3b50 2024-04-25T09:17:29.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.141+0000 7fe109000640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0e806b500 0x7fe0e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:29.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.143+0000 7fe109000640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0e806b500 0x7fe0e806d9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8033000 tx=0x7fe0f8009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:29.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.145+0000 7fe101600640 1 -- 172.21.15.62:0/2892077514 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0f4058fd0 con 0x7fe1040b3b50 2024-04-25T09:17:29.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.346+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe1040b5450 con 0x7fe1040b3b50 2024-04-25T09:17:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.898+0000 7fe101600640 1 -- 172.21.15.62:0/2892077514 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v146) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe0f40588d0 con 0x7fe1040b3b50 2024-04-25T09:17:29.885 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-25T09:17:29.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.901+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0e806b500 msgr2=0x7fe0e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:29.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.901+0000 7fe10ae00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0e806b500 0x7fe0e806d9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8033000 tx=0x7fe0f8009000 comp rx=0 tx=0).stop 2024-04-25T09:17:29.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.901+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1040b3b50 msgr2=0x7fe104142010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:29.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.901+0000 7fe10ae00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1040b3b50 0x7fe104142010 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400c7f0 tx=0x7fe0f400cbf0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.901+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 shutdown_connections 2024-04-25T09:17:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.902+0000 7fe10ae00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0e806b500 0x7fe0e806d9c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.902+0000 7fe10ae00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe104154e20 0x7fe1041f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.902+0000 7fe10ae00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe104154a40 0x7fe1040cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.902+0000 7fe10ae00640 1 --2- 172.21.15.62:0/2892077514 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe1040b3b50 0x7fe104142010 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.902+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 >> 172.21.15.62:0/2892077514 conn(0x7fe104074290 msgr2=0x7fe104153980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.902+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 shutdown_connections 2024-04-25T09:17:29.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:29.902+0000 7fe10ae00640 1 -- 172.21.15.62:0/2892077514 wait complete. 2024-04-25T09:17:29.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:17:30.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.120+0000 7fe562600640 1 Processor -- start 2024-04-25T09:17:30.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.120+0000 7fe562600640 1 -- start start 2024-04-25T09:17:30.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.121+0000 7fe562600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe55c0b40e0 0x7fe55c06c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:30.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.121+0000 7fe562600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe55c0ceeb0 0x7fe55c1625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:30.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.121+0000 7fe562600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c162af0 0x7fe55c070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:30.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.121+0000 7fe562600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c0cf360 con 0x7fe55c0b40e0 2024-04-25T09:17:30.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.121+0000 7fe562600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c06e560 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.121+0000 7fe562600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c06dd00 con 0x7fe55c162af0 2024-04-25T09:17:30.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe55c0b40e0 0x7fe55c06c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:30.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe55c0b40e0 0x7fe55c06c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34362/0 (socket says 172.21.15.62:34362) 2024-04-25T09:17:30.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55b400640 1 -- 172.21.15.62:0/904990137 learned_addr learned my addr 172.21.15.62:0/904990137 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:30.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55aa00640 1 --2- 172.21.15.62:0/904990137 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0ceeb0 0x7fe55c1625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:30.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55be00640 1 --2- 172.21.15.62:0/904990137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c162af0 0x7fe55c070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55b400640 1 -- 172.21.15.62:0/904990137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c162af0 msgr2=0x7fe55c070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55b400640 1 --2- 172.21.15.62:0/904990137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c162af0 0x7fe55c070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55b400640 1 -- 172.21.15.62:0/904990137 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0ceeb0 msgr2=0x7fe55c1625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55b400640 1 --2- 172.21.15.62:0/904990137 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0ceeb0 0x7fe55c1625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55b400640 1 -- 172.21.15.62:0/904990137 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c06dfa0 con 0x7fe55c0b40e0 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55aa00640 1 --2- 172.21.15.62:0/904990137 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0ceeb0 0x7fe55c1625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.122+0000 7fe55be00640 1 --2- 172.21.15.62:0/904990137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c162af0 0x7fe55c070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.123+0000 7fe55b400640 1 --2- 172.21.15.62:0/904990137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0b40e0 0x7fe55c06c380 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe55c06c080 tx=0x7fe54c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bf2fa5d4a60eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe55a000640 1 -- 172.21.15.62:0/904990137 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c037070 con 0x7fe55c0b40e0 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe55a000640 1 -- 172.21.15.62:0/904990137 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c02ae00 con 0x7fe55c0b40e0 2024-04-25T09:17:30.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe55a000640 1 -- 172.21.15.62:0/904990137 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c032660 con 0x7fe55c0b40e0 2024-04-25T09:17:30.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe562600640 1 -- 172.21.15.62:0/904990137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0b40e0 msgr2=0x7fe55c06c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:30.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe562600640 1 --2- 172.21.15.62:0/904990137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0b40e0 0x7fe55c06c380 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe55c06c080 tx=0x7fe54c02a0b0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe562600640 1 -- 172.21.15.62:0/904990137 shutdown_connections 2024-04-25T09:17:30.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe562600640 1 --2- 172.21.15.62:0/904990137 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c162af0 0x7fe55c070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe562600640 1 --2- 172.21.15.62:0/904990137 >> v2:172.21.15.121:3300/0 conn(0x7fe55c0ceeb0 0x7fe55c1625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe562600640 1 --2- 172.21.15.62:0/904990137 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0b40e0 0x7fe55c06c380 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe562600640 1 -- 172.21.15.62:0/904990137 >> 172.21.15.62:0/904990137 conn(0x7fe55c0b1410 msgr2=0x7fe55c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:30.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.124+0000 7fe562600640 1 -- 172.21.15.62:0/904990137 shutdown_connections 2024-04-25T09:17:30.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.125+0000 7fe562600640 1 -- 172.21.15.62:0/904990137 wait complete. 2024-04-25T09:17:30.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.126+0000 7fe562600640 1 Processor -- start 2024-04-25T09:17:30.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.126+0000 7fe562600640 1 -- start start 2024-04-25T09:17:30.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe562600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c0b40e0 0x7fe55c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe562600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0ceeb0 0x7fe55c1509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe562600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c162af0 0x7fe55c0af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe562600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c162f80 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe562600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c071f40 con 0x7fe55c162af0 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe562600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c072460 con 0x7fe55c0b40e0 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0ceeb0 0x7fe55c1509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0ceeb0 0x7fe55c1509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34368/0 (socket says 172.21.15.62:34368) 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55aa00640 1 -- 172.21.15.62:0/1111495304 learned_addr learned my addr 172.21.15.62:0/1111495304 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55b400640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c0b40e0 0x7fe55c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55be00640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c162af0 0x7fe55c0af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55aa00640 1 -- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c0b40e0 msgr2=0x7fe55c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55aa00640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c0b40e0 0x7fe55c15c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55aa00640 1 -- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c162af0 msgr2=0x7fe55c0af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:30.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55aa00640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c162af0 0x7fe55c0af7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.128+0000 7fe55aa00640 1 -- 172.21.15.62:0/1111495304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c15b0a0 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.129+0000 7fe55aa00640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0ceeb0 0x7fe55c1509c0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe54400c7f0 tx=0x7fe54400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:30.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.129+0000 7fe553400640 1 -- 172.21.15.62:0/1111495304 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544015070 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.129+0000 7fe553400640 1 -- 172.21.15.62:0/1111495304 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe544007820 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.129+0000 7fe553400640 1 -- 172.21.15.62:0/1111495304 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544007c10 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.129+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c15b380 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.129+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c15b940 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.130+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c04ef20 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.132+0000 7fe553400640 1 -- 172.21.15.62:0/1111495304 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe544018070 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.133+0000 7fe553400640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe53806b4d0 0x7fe53806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:30.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.133+0000 7fe553400640 1 -- 172.21.15.62:0/1111495304 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe54408a400 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.133+0000 7fe55b400640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe53806b4d0 0x7fe53806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:30.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.134+0000 7fe55b400640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe53806b4d0 0x7fe53806d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe54c034000 tx=0x7fe54c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:30.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.139+0000 7fe553400640 1 -- 172.21.15.62:0/1111495304 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe544059130 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.380+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe55c06c6d0 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.946+0000 7fe553400640 1 -- 172.21.15.62:0/1111495304 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v95) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe544058a30 con 0x7fe55c0ceeb0 2024-04-25T09:17:30.935 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 20 and data pool 21 2024-04-25T09:17:30.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.952+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe53806b4d0 msgr2=0x7fe53806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:30.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.952+0000 7fe562600640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe53806b4d0 0x7fe53806d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe54c034000 tx=0x7fe54c02e000 comp rx=0 tx=0).stop 2024-04-25T09:17:30.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.952+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0ceeb0 msgr2=0x7fe55c1509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:30.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.952+0000 7fe562600640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0ceeb0 0x7fe55c1509c0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe54400c7f0 tx=0x7fe54400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.953+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 shutdown_connections 2024-04-25T09:17:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.953+0000 7fe562600640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe53806b4d0 0x7fe53806d990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.953+0000 7fe562600640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe55c0ceeb0 0x7fe55c1509c0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.953+0000 7fe562600640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe55c162af0 0x7fe55c0af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.953+0000 7fe562600640 1 --2- 172.21.15.62:0/1111495304 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe55c0b40e0 0x7fe55c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:30.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.953+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 >> 172.21.15.62:0/1111495304 conn(0x7fe55c0b1410 msgr2=0x7fe55c163df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.955+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 shutdown_connections 2024-04-25T09:17:30.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:30.956+0000 7fe562600640 1 -- 172.21.15.62:0/1111495304 wait complete. 2024-04-25T09:17:30.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:31.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.162+0000 7fc9d7200640 1 Processor -- start 2024-04-25T09:17:31.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.162+0000 7fc9d7200640 1 -- start start 2024-04-25T09:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.163+0000 7fc9d7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d8144c10 0x7fc9d815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.164+0000 7fc9d7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9d80cee40 0x7fc9d806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.164+0000 7fc9d7200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d815faa0 con 0x7fc9d806d730 2024-04-25T09:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.164+0000 7fc9d7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d8144ff0 con 0x7fc9d80cee40 2024-04-25T09:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.164+0000 7fc9d7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d8160340 con 0x7fc9d8144c10 2024-04-25T09:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.164+0000 7fc9d5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9d80cee40 0x7fc9d806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.164+0000 7fc9d5400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9d80cee40 0x7fc9d806d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34376/0 (socket says 172.21.15.62:34376) 2024-04-25T09:17:31.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.164+0000 7fc9d5400640 1 -- 172.21.15.62:0/1960935006 learned_addr learned my addr 172.21.15.62:0/1960935006 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.164+0000 7fc9d5e00640 1 --2- 172.21.15.62:0/1960935006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d8144c10 0x7fc9d815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.165+0000 7fc9d5400640 1 -- 172.21.15.62:0/1960935006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d8144c10 msgr2=0x7fc9d815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.165+0000 7fc9d5400640 1 --2- 172.21.15.62:0/1960935006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d8144c10 0x7fc9d815f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.165+0000 7fc9d5400640 1 -- 172.21.15.62:0/1960935006 >> v1:172.21.15.121:6789/0 conn(0x7fc9d806d730 legacy=0x7fc9d80b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:31.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.165+0000 7fc9d5400640 1 -- 172.21.15.62:0/1960935006 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d815fd80 con 0x7fc9d80cee40 2024-04-25T09:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.166+0000 7fc9d5400640 1 --2- 172.21.15.62:0/1960935006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d806d1f0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009870 tx=0x7fc9c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e57e06dbd4197c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.166+0000 7fc9d4a00640 1 -- 172.21.15.62:0/1960935006 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4037070 con 0x7fc9d80cee40 2024-04-25T09:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.166+0000 7fc9d4a00640 1 -- 172.21.15.62:0/1960935006 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4032440 con 0x7fc9d80cee40 2024-04-25T09:17:31.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d4a00640 1 -- 172.21.15.62:0/1960935006 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4032810 con 0x7fc9d80cee40 2024-04-25T09:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d7200640 1 -- 172.21.15.62:0/1960935006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 msgr2=0x7fc9d806d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d7200640 1 --2- 172.21.15.62:0/1960935006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d806d1f0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009870 tx=0x7fc9c402a180 comp rx=0 tx=0).stop 2024-04-25T09:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d7200640 1 -- 172.21.15.62:0/1960935006 shutdown_connections 2024-04-25T09:17:31.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d7200640 1 --2- 172.21.15.62:0/1960935006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d806d1f0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d7200640 1 --2- 172.21.15.62:0/1960935006 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d8144c10 0x7fc9d815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d7200640 1 -- 172.21.15.62:0/1960935006 >> 172.21.15.62:0/1960935006 conn(0x7fc9d80b14d0 msgr2=0x7fc9d80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d7200640 1 -- 172.21.15.62:0/1960935006 shutdown_connections 2024-04-25T09:17:31.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.167+0000 7fc9d7200640 1 -- 172.21.15.62:0/1960935006 wait complete. 2024-04-25T09:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.169+0000 7fc9d7200640 1 Processor -- start 2024-04-25T09:17:31.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.169+0000 7fc9d7200640 1 -- start start 2024-04-25T09:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d806d730 0x7fc9d8141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d8131850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9d8144c10 0x7fc9d8142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80cf2a0 con 0x7fc9d80cee40 2024-04-25T09:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80b41c0 con 0x7fc9d8144c10 2024-04-25T09:17:31.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80b5660 con 0x7fc9d806d730 2024-04-25T09:17:31.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d8131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d5400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d8131850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34392/0 (socket says 172.21.15.62:34392) 2024-04-25T09:17:31.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.170+0000 7fc9d5400640 1 -- 172.21.15.62:0/929203615 learned_addr learned my addr 172.21.15.62:0/929203615 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.171+0000 7fc9d5e00640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d806d730 0x7fc9d8141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.171+0000 7fc9d6800640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9d8144c10 0x7fc9d8142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.171+0000 7fc9d5400640 1 -- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d806d730 msgr2=0x7fc9d8141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.171+0000 7fc9d5400640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d806d730 0x7fc9d8141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.171+0000 7fc9d5400640 1 -- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9d8144c10 msgr2=0x7fc9d8142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.171+0000 7fc9d5400640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9d8144c10 0x7fc9d8142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.171+0000 7fc9d5400640 1 -- 172.21.15.62:0/929203615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d8141cc0 con 0x7fc9d80cee40 2024-04-25T09:17:31.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.171+0000 7fc9d5400640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d8131850 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000c00 tx=0x7fc9c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:31.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.172+0000 7fc9ce000640 1 -- 172.21.15.62:0/929203615 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4041020 con 0x7fc9d80cee40 2024-04-25T09:17:31.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.172+0000 7fc9ce000640 1 -- 172.21.15.62:0/929203615 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c40306c0 con 0x7fc9d80cee40 2024-04-25T09:17:31.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.172+0000 7fc9ce000640 1 -- 172.21.15.62:0/929203615 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4037040 con 0x7fc9d80cee40 2024-04-25T09:17:31.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.172+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d8141fa0 con 0x7fc9d80cee40 2024-04-25T09:17:31.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.172+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d8142560 con 0x7fc9d80cee40 2024-04-25T09:17:31.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.173+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d804ef20 con 0x7fc9d80cee40 2024-04-25T09:17:31.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.174+0000 7fc9ce000640 1 -- 172.21.15.62:0/929203615 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9c4044020 con 0x7fc9d80cee40 2024-04-25T09:17:31.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.175+0000 7fc9ce000640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc9b806b4d0 0x7fc9b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.175+0000 7fc9ce000640 1 -- 172.21.15.62:0/929203615 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9c40aaa80 con 0x7fc9d80cee40 2024-04-25T09:17:31.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.175+0000 7fc9d5e00640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc9b806b4d0 0x7fc9b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.176+0000 7fc9d5e00640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc9b806b4d0 0x7fc9b806d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8006fd0 tx=0x7fc9c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:31.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.182+0000 7fc9ce000640 1 -- 172.21.15.62:0/929203615 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9c40ace20 con 0x7fc9d80cee40 2024-04-25T09:17:31.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.384+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9d80b4f80 con 0x7fc9d80cee40 2024-04-25T09:17:31.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.386+0000 7fc9ce000640 1 -- 172.21.15.62:0/929203615 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16680 (secure 0 0 0) 0x7fc9c404b030 con 0x7fc9d80cee40 2024-04-25T09:17:31.373 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:31.373 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":147,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:30.382464+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:17:28.263758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:17:29.401316+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.388+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc9b806b4d0 msgr2=0x7fc9b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.388+0000 7fc9d7200640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc9b806b4d0 0x7fc9b806d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8006fd0 tx=0x7fc9c8008040 comp rx=0 tx=0).stop 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.388+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 msgr2=0x7fc9d8131850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.388+0000 7fc9d7200640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d8131850 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000c00 tx=0x7fc9c40046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.389+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 shutdown_connections 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.389+0000 7fc9d7200640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc9b806b4d0 0x7fc9b806d990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.389+0000 7fc9d7200640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9d80cee40 0x7fc9d8131850 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.389+0000 7fc9d7200640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9d8144c10 0x7fc9d8142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.389+0000 7fc9d7200640 1 --2- 172.21.15.62:0/929203615 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9d806d730 0x7fc9d8141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.389+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 >> 172.21.15.62:0/929203615 conn(0x7fc9d80b14d0 msgr2=0x7fc9d8070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:31.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.389+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 shutdown_connections 2024-04-25T09:17:31.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.389+0000 7fc9d7200640 1 -- 172.21.15.62:0/929203615 wait complete. 2024-04-25T09:17:31.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:31.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.620+0000 7f58d7e00640 1 Processor -- start 2024-04-25T09:17:31.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.621+0000 7f58d7e00640 1 -- start start 2024-04-25T09:17:31.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.622+0000 7f58d7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f58d813d970 0x7f58d80b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.622+0000 7f58d7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 0x7f58d806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.622+0000 7f58d7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58d815acf0 0x7f58d806d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.622+0000 7f58d7e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d815b220 con 0x7f58d815acf0 2024-04-25T09:17:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.622+0000 7f58d7e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d815b500 con 0x7f58d813d970 2024-04-25T09:17:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.622+0000 7f58d7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d815b7e0 con 0x7f58d80b1050 2024-04-25T09:17:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d7400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58d815acf0 0x7f58d806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d7400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f58d815acf0 0x7f58d806d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34402/0 (socket says 172.21.15.62:34402) 2024-04-25T09:17:31.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d7400640 1 -- 172.21.15.62:0/1956795242 learned_addr learned my addr 172.21.15.62:0/1956795242 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d6a00640 1 --2- 172.21.15.62:0/1956795242 >> v2:172.21.15.121:3300/0 conn(0x7f58d813d970 0x7f58d80b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58cfe00640 1 --2- 172.21.15.62:0/1956795242 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 0x7f58d806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d7400640 1 -- 172.21.15.62:0/1956795242 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 msgr2=0x7f58d806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d7400640 1 --2- 172.21.15.62:0/1956795242 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 0x7f58d806d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d7400640 1 -- 172.21.15.62:0/1956795242 >> v2:172.21.15.121:3300/0 conn(0x7f58d813d970 msgr2=0x7f58d80b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d7400640 1 --2- 172.21.15.62:0/1956795242 >> v2:172.21.15.121:3300/0 conn(0x7f58d813d970 0x7f58d80b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.623+0000 7f58d7400640 1 -- 172.21.15.62:0/1956795242 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d815bac0 con 0x7f58d815acf0 2024-04-25T09:17:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.624+0000 7f58d7400640 1 --2- 172.21.15.62:0/1956795242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d815acf0 0x7f58d806d730 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f58d815b710 tx=0x7f58d002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5886bc7e8bd3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d6000640 1 -- 172.21.15.62:0/1956795242 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0036070 con 0x7f58d815acf0 2024-04-25T09:17:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d6000640 1 -- 172.21.15.62:0/1956795242 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0031600 con 0x7f58d815acf0 2024-04-25T09:17:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d6000640 1 -- 172.21.15.62:0/1956795242 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00319d0 con 0x7f58d815acf0 2024-04-25T09:17:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d7e00640 1 -- 172.21.15.62:0/1956795242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d815acf0 msgr2=0x7f58d806d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d7e00640 1 --2- 172.21.15.62:0/1956795242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d815acf0 0x7f58d806d730 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f58d815b710 tx=0x7f58d002a680 comp rx=0 tx=0).stop 2024-04-25T09:17:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d7e00640 1 -- 172.21.15.62:0/1956795242 shutdown_connections 2024-04-25T09:17:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d7e00640 1 --2- 172.21.15.62:0/1956795242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d815acf0 0x7f58d806d730 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d7e00640 1 --2- 172.21.15.62:0/1956795242 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 0x7f58d806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.625+0000 7f58d7e00640 1 --2- 172.21.15.62:0/1956795242 >> v2:172.21.15.121:3300/0 conn(0x7f58d813d970 0x7f58d80b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.626+0000 7f58d7e00640 1 -- 172.21.15.62:0/1956795242 >> 172.21.15.62:0/1956795242 conn(0x7f58d80b14d0 msgr2=0x7f58d80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.626+0000 7f58d7e00640 1 -- 172.21.15.62:0/1956795242 shutdown_connections 2024-04-25T09:17:31.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.626+0000 7f58d7e00640 1 -- 172.21.15.62:0/1956795242 wait complete. 2024-04-25T09:17:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.627+0000 7f58d7e00640 1 Processor -- start 2024-04-25T09:17:31.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.627+0000 7f58d7e00640 1 -- start start 2024-04-25T09:17:31.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.628+0000 7f58d7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 0x7f58d80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.628+0000 7f58d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d813d970 0x7f58d81f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.628+0000 7f58d7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d815acf0 0x7f58d81f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.628+0000 7f58d7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8140290 con 0x7f58d813d970 2024-04-25T09:17:31.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.628+0000 7f58d7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8140110 con 0x7f58d815acf0 2024-04-25T09:17:31.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.628+0000 7f58d7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d8140410 con 0x7f58d80b1050 2024-04-25T09:17:31.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d813d970 0x7f58d81f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d813d970 0x7f58d81f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34410/0 (socket says 172.21.15.62:34410) 2024-04-25T09:17:31.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 -- 172.21.15.62:0/3953324534 learned_addr learned my addr 172.21.15.62:0/3953324534 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 -- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 msgr2=0x7f58d80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58d6a00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 0x7f58d80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 0x7f58d80b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 -- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d815acf0 msgr2=0x7f58d81f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d815acf0 0x7f58d81f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 -- 172.21.15.62:0/3953324534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d8141ba0 con 0x7f58d813d970 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.629+0000 7f58cfe00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d813d970 0x7f58d81f4d90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d800 tx=0x7f58c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.630+0000 7f58cea00640 1 -- 172.21.15.62:0/3953324534 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0018020 con 0x7f58d813d970 2024-04-25T09:17:31.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.630+0000 7f58cea00640 1 -- 172.21.15.62:0/3953324534 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c00096a0 con 0x7f58d813d970 2024-04-25T09:17:31.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.630+0000 7f58cea00640 1 -- 172.21.15.62:0/3953324534 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0009a70 con 0x7f58d813d970 2024-04-25T09:17:31.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.630+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d8141e80 con 0x7f58d813d970 2024-04-25T09:17:31.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.630+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d8142440 con 0x7f58d813d970 2024-04-25T09:17:31.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.631+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d804ef20 con 0x7f58d813d970 2024-04-25T09:17:31.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.632+0000 7f58cea00640 1 -- 172.21.15.62:0/3953324534 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58c001b020 con 0x7f58d813d970 2024-04-25T09:17:31.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.633+0000 7f58cea00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58b406b500 0x7f58b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:31.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.633+0000 7f58cea00640 1 -- 172.21.15.62:0/3953324534 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58c008a580 con 0x7f58d813d970 2024-04-25T09:17:31.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.633+0000 7f58d6a00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58b406b500 0x7f58b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:31.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.634+0000 7f58d6a00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58b406b500 0x7f58b406d9c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f58c8006fd0 tx=0x7f58c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:31.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.637+0000 7f58cea00640 1 -- 172.21.15.62:0/3953324534 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58c0022030 con 0x7f58d813d970 2024-04-25T09:17:31.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.841+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58d8068d50 con 0x7f58d813d970 2024-04-25T09:17:31.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.844+0000 7f58cea00640 1 -- 172.21.15.62:0/3953324534 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16680 (secure 0 0 0) 0x7f58c00592d0 con 0x7f58d813d970 2024-04-25T09:17:31.831 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:31.832 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":147,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:30.382464+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:17:28.263758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:17:29.401316+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58b406b500 msgr2=0x7f58b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58b406b500 0x7f58b406d9c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f58c8006fd0 tx=0x7f58c8008040 comp rx=0 tx=0).stop 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d813d970 msgr2=0x7f58d81f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d813d970 0x7f58d81f4d90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d800 tx=0x7f58c000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 shutdown_connections 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f58b406b500 0x7f58b406d9c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f58d813d970 0x7f58d81f4d90 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58d815acf0 0x7f58d81f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 --2- 172.21.15.62:0/3953324534 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f58d80b1050 0x7f58d80b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.847+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 >> 172.21.15.62:0/3953324534 conn(0x7f58d80b14d0 msgr2=0x7f58d813e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.848+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 shutdown_connections 2024-04-25T09:17:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:31.848+0000 7f58d7e00640 1 -- 172.21.15.62:0/3953324534 wait complete. 2024-04-25T09:17:31.855 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:17:32.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.082+0000 7fc867200640 1 Processor -- start 2024-04-25T09:17:32.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.083+0000 7fc867200640 1 -- start start 2024-04-25T09:17:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc867200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc86815dd50 0x7fc8680d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc867200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc8680d2fb0 0x7fc8680ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc867200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc8680d3960 0x7fc8680d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc867200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8680ce780 con 0x7fc8680d3960 2024-04-25T09:17:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc867200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8680d4310 con 0x7fc8680d2fb0 2024-04-25T09:17:32.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc867200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680d3440 con 0x7fc86815dd50 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc866800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc8680d3960 0x7fc8680d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc866800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc8680d3960 0x7fc8680d3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34412/0 (socket says 172.21.15.62:34412) 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.084+0000 7fc866800640 1 -- 172.21.15.62:0/305449762 learned_addr learned my addr 172.21.15.62:0/305449762 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.085+0000 7fc865e00640 1 --2- 172.21.15.62:0/305449762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc86815dd50 0x7fc8680d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.085+0000 7fc865400640 1 --2- 172.21.15.62:0/305449762 >> v2:172.21.15.121:3300/0 conn(0x7fc8680d2fb0 0x7fc8680ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.085+0000 7fc866800640 1 -- 172.21.15.62:0/305449762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc86815dd50 msgr2=0x7fc8680d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.085+0000 7fc866800640 1 --2- 172.21.15.62:0/305449762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc86815dd50 0x7fc8680d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.085+0000 7fc866800640 1 -- 172.21.15.62:0/305449762 >> v2:172.21.15.121:3300/0 conn(0x7fc8680d2fb0 msgr2=0x7fc8680ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:32.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.085+0000 7fc866800640 1 --2- 172.21.15.62:0/305449762 >> v2:172.21.15.121:3300/0 conn(0x7fc8680d2fb0 0x7fc8680ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.085+0000 7fc866800640 1 -- 172.21.15.62:0/305449762 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680ce1c0 con 0x7fc8680d3960 2024-04-25T09:17:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.086+0000 7fc866800640 1 --2- 172.21.15.62:0/305449762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680d3dd0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc868072f60 tx=0x7fc86002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8122aaef918047bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.086+0000 7fc864a00640 1 -- 172.21.15.62:0/305449762 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860037070 con 0x7fc8680d3960 2024-04-25T09:17:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.086+0000 7fc864a00640 1 -- 172.21.15.62:0/305449762 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc860032440 con 0x7fc8680d3960 2024-04-25T09:17:32.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.086+0000 7fc864a00640 1 -- 172.21.15.62:0/305449762 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860032810 con 0x7fc8680d3960 2024-04-25T09:17:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 -- 172.21.15.62:0/305449762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 msgr2=0x7fc8680d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 --2- 172.21.15.62:0/305449762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680d3dd0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fc868072f60 tx=0x7fc86002a200 comp rx=0 tx=0).stop 2024-04-25T09:17:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 -- 172.21.15.62:0/305449762 shutdown_connections 2024-04-25T09:17:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 --2- 172.21.15.62:0/305449762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680d3dd0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 --2- 172.21.15.62:0/305449762 >> v2:172.21.15.121:3300/0 conn(0x7fc8680d2fb0 0x7fc8680ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 --2- 172.21.15.62:0/305449762 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc86815dd50 0x7fc8680d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 -- 172.21.15.62:0/305449762 >> 172.21.15.62:0/305449762 conn(0x7fc86806b000 msgr2=0x7fc86806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:32.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 -- 172.21.15.62:0/305449762 shutdown_connections 2024-04-25T09:17:32.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.087+0000 7fc867200640 1 -- 172.21.15.62:0/305449762 wait complete. 2024-04-25T09:17:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.089+0000 7fc867200640 1 Processor -- start 2024-04-25T09:17:32.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.089+0000 7fc867200640 1 -- start start 2024-04-25T09:17:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc867200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc867200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc86815dd50 0x7fc86814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:32.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc867200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8680c9f20 0x7fc8680ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc867200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680cef00 con 0x7fc8680d3960 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc867200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680d2840 con 0x7fc86815dd50 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc867200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680b6b00 con 0x7fc8680c9f20 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc865e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc865e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34426/0 (socket says 172.21.15.62:34426) 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.090+0000 7fc865e00640 1 -- 172.21.15.62:0/112484352 learned_addr learned my addr 172.21.15.62:0/112484352 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc865400640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc86815dd50 0x7fc86814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc865e00640 1 -- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8680c9f20 msgr2=0x7fc8680ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc865e00640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8680c9f20 0x7fc8680ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc865e00640 1 -- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc86815dd50 msgr2=0x7fc86814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc865e00640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc86815dd50 0x7fc86814fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc865e00640 1 -- 172.21.15.62:0/112484352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680b39c0 con 0x7fc8680d3960 2024-04-25T09:17:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc865e00640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680b3170 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc858007e40 tx=0x7fc85800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc85e000640 1 -- 172.21.15.62:0/112484352 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858014620 con 0x7fc8680d3960 2024-04-25T09:17:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc85e000640 1 -- 172.21.15.62:0/112484352 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858019070 con 0x7fc8680d3960 2024-04-25T09:17:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc85e000640 1 -- 172.21.15.62:0/112484352 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858014a00 con 0x7fc8680d3960 2024-04-25T09:17:32.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.091+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8680b3ca0 con 0x7fc8680d3960 2024-04-25T09:17:32.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.092+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8680b4260 con 0x7fc8680d3960 2024-04-25T09:17:32.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.093+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86804ef20 con 0x7fc8680d3960 2024-04-25T09:17:32.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.094+0000 7fc85e000640 1 -- 172.21.15.62:0/112484352 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc858014ba0 con 0x7fc8680d3960 2024-04-25T09:17:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.095+0000 7fc85e000640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:32.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.095+0000 7fc85e000640 1 -- 172.21.15.62:0/112484352 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc85808ab30 con 0x7fc8680d3960 2024-04-25T09:17:32.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.096+0000 7fc865400640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:32.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.097+0000 7fc865400640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84406b4d0 0x7fc84406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc8680cd8c0 tx=0x7fc850008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:32.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.100+0000 7fc85e000640 1 -- 172.21.15.62:0/112484352 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc858059910 con 0x7fc8680d3960 2024-04-25T09:17:32.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.342+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc8680b9950 con 0x7fc8680d3960 2024-04-25T09:17:32.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.971+0000 7fc85e000640 1 -- 172.21.15.62:0/112484352 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc858059210 con 0x7fc8680d3960 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84406b4d0 msgr2=0x7fc84406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84406b4d0 0x7fc84406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc8680cd8c0 tx=0x7fc850008040 comp rx=0 tx=0).stop 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 msgr2=0x7fc8680b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680b3170 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fc858007e40 tx=0x7fc85800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 shutdown_connections 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8680c9f20 0x7fc8680ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc86815dd50 0x7fc86814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 --2- 172.21.15.62:0/112484352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8680d3960 0x7fc8680b3170 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.975+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 >> 172.21.15.62:0/112484352 conn(0x7fc86806b000 msgr2=0x7fc8681601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.976+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 shutdown_connections 2024-04-25T09:17:32.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:32.976+0000 7fc867200640 1 -- 172.21.15.62:0/112484352 wait complete. 2024-04-25T09:17:32.984 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:17:32.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:17:33.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.213+0000 7f72b5a00640 1 Processor -- start 2024-04-25T09:17:33.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.213+0000 7f72b5a00640 1 -- start start 2024-04-25T09:17:33.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.215+0000 7f72b5a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b00721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:33.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.215+0000 7f72b5a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f72b00afef0 0x7f72b00b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:33.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.215+0000 7f72b5a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b00b3930 con 0x7f72b00668f0 2024-04-25T09:17:33.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.215+0000 7f72b5a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b00d1350 con 0x7f72b00afef0 2024-04-25T09:17:33.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.215+0000 7f72b5a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00b3bd0 con 0x7f72b015db60 2024-04-25T09:17:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.215+0000 7f72aea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b00721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.215+0000 7f72aea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b00721f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:46536/0 (socket says 172.21.15.62:46536) 2024-04-25T09:17:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.215+0000 7f72aea00640 1 -- 172.21.15.62:0/857507514 learned_addr learned my addr 172.21.15.62:0/857507514 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.216+0000 7f72ae000640 1 --2- 172.21.15.62:0/857507514 >> v2:172.21.15.121:3300/0 conn(0x7f72b00afef0 0x7f72b00b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.216+0000 7f72aea00640 1 -- 172.21.15.62:0/857507514 >> v2:172.21.15.121:3300/0 conn(0x7f72b00afef0 msgr2=0x7f72b00b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.216+0000 7f72aea00640 1 --2- 172.21.15.62:0/857507514 >> v2:172.21.15.121:3300/0 conn(0x7f72b00afef0 0x7f72b00b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.216+0000 7f72aea00640 1 -- 172.21.15.62:0/857507514 >> v1:172.21.15.121:6789/0 conn(0x7f72b00668f0 legacy=0x7f72b00ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:33.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.216+0000 7f72aea00640 1 -- 172.21.15.62:0/857507514 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00b3eb0 con 0x7f72b015db60 2024-04-25T09:17:33.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.218+0000 7f72aea00640 1 --2- 172.21.15.62:0/857507514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b00721f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009870 tx=0x7f72a002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02b873452c806a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.218+0000 7f72ad600640 1 -- 172.21.15.62:0/857507514 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0037070 con 0x7f72b015db60 2024-04-25T09:17:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.218+0000 7f72ad600640 1 -- 172.21.15.62:0/857507514 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0032440 con 0x7f72b015db60 2024-04-25T09:17:33.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.218+0000 7f72ad600640 1 -- 172.21.15.62:0/857507514 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0032810 con 0x7f72b015db60 2024-04-25T09:17:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.219+0000 7f72b5a00640 1 -- 172.21.15.62:0/857507514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 msgr2=0x7f72b00721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.219+0000 7f72b5a00640 1 --2- 172.21.15.62:0/857507514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b00721f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009870 tx=0x7f72a002a270 comp rx=0 tx=0).stop 2024-04-25T09:17:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.219+0000 7f72b5a00640 1 -- 172.21.15.62:0/857507514 shutdown_connections 2024-04-25T09:17:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.219+0000 7f72b5a00640 1 --2- 172.21.15.62:0/857507514 >> v2:172.21.15.121:3300/0 conn(0x7f72b00afef0 0x7f72b00b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.219+0000 7f72b5a00640 1 --2- 172.21.15.62:0/857507514 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b00721f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.219+0000 7f72b5a00640 1 -- 172.21.15.62:0/857507514 >> 172.21.15.62:0/857507514 conn(0x7f72b006abe0 msgr2=0x7f72b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.219+0000 7f72b5a00640 1 -- 172.21.15.62:0/857507514 shutdown_connections 2024-04-25T09:17:33.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.219+0000 7f72b5a00640 1 -- 172.21.15.62:0/857507514 wait complete. 2024-04-25T09:17:33.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.221+0000 7f72b5a00640 1 Processor -- start 2024-04-25T09:17:33.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.221+0000 7f72b5a00640 1 -- start start 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72b5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b00668f0 0x7f72b0160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72b5a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72b00afef0 0x7f72b0151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72b5a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72b5a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b015df40 con 0x7f72b00668f0 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72b5a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00b5d10 con 0x7f72b00afef0 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72b5a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b015e0c0 con 0x7f72b015db60 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72aea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b00668f0 0x7f72b0160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72aea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b00668f0 0x7f72b0160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34432/0 (socket says 172.21.15.62:34432) 2024-04-25T09:17:33.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72aea00640 1 -- 172.21.15.62:0/1861868406 learned_addr learned my addr 172.21.15.62:0/1861868406 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:33.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72ae000640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72b00afef0 0x7f72b0151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.222+0000 7f72af400640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72aea00640 1 -- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 msgr2=0x7f72b014e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72aea00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b014e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72aea00640 1 -- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72b00afef0 msgr2=0x7f72b0151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72aea00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72b00afef0 0x7f72b0151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72aea00640 1 -- 172.21.15.62:0/1861868406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b015f1f0 con 0x7f72b00668f0 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72aea00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b00668f0 0x7f72b0160a10 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000c00 tx=0x7f72a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72a6a00640 1 -- 172.21.15.62:0/1861868406 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0037040 con 0x7f72b00668f0 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72a6a00640 1 -- 172.21.15.62:0/1861868406 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0042070 con 0x7f72b00668f0 2024-04-25T09:17:33.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72a6a00640 1 -- 172.21.15.62:0/1861868406 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0030880 con 0x7f72b00668f0 2024-04-25T09:17:33.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.223+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b015f4d0 con 0x7f72b00668f0 2024-04-25T09:17:33.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.224+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b015fa90 con 0x7f72b00668f0 2024-04-25T09:17:33.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.225+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b004ef20 con 0x7f72b00668f0 2024-04-25T09:17:33.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.226+0000 7f72a6a00640 1 -- 172.21.15.62:0/1861868406 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72a0030a20 con 0x7f72b00668f0 2024-04-25T09:17:33.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.227+0000 7f72a6a00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f728c06b4d0 0x7f728c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:33.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.227+0000 7f72a6a00640 1 -- 172.21.15.62:0/1861868406 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72a00aab10 con 0x7f72b00668f0 2024-04-25T09:17:33.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.228+0000 7f72ae000640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f728c06b4d0 0x7f728c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:33.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.229+0000 7f72ae000640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f728c06b4d0 0x7f728c06d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f72b01528f0 tx=0x7f7298008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:33.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.232+0000 7f72a6a00640 1 -- 172.21.15.62:0/1861868406 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72a00798f0 con 0x7f72b00668f0 2024-04-25T09:17:33.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.472+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f72b006e580 con 0x7f72b00668f0 2024-04-25T09:17:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.974+0000 7f72a6a00640 1 -- 172.21.15.62:0/1861868406 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72a00791f0 con 0x7f72b00668f0 2024-04-25T09:17:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f728c06b4d0 msgr2=0x7f728c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f728c06b4d0 0x7f728c06d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f72b01528f0 tx=0x7f7298008040 comp rx=0 tx=0).stop 2024-04-25T09:17:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b00668f0 msgr2=0x7f72b0160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:33.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b00668f0 0x7f72b0160a10 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f72a0000c00 tx=0x7f72a00046c0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 shutdown_connections 2024-04-25T09:17:33.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f728c06b4d0 0x7f728c06d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f72b015db60 0x7f72b014e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72b00afef0 0x7f72b0151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 --2- 172.21.15.62:0/1861868406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72b00668f0 0x7f72b0160a10 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:33.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.978+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 >> 172.21.15.62:0/1861868406 conn(0x7f72b006abe0 msgr2=0x7f72b006d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:33.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.979+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 shutdown_connections 2024-04-25T09:17:33.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:33.979+0000 7f72b5a00640 1 -- 172.21.15.62:0/1861868406 wait complete. 2024-04-25T09:17:33.987 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:17:33.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:34.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.207+0000 7f63e6600640 1 Processor -- start 2024-04-25T09:17:34.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.207+0000 7f63e6600640 1 -- start start 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.209+0000 7f63e6600640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63e0144c10 0x7f63e015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.209+0000 7f63e6600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 0x7f63e00b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.209+0000 7f63e6600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e00cf2d0 con 0x7f63e00cee40 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.209+0000 7f63e6600640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e0144ff0 con 0x7f63e0144c10 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.209+0000 7f63e6600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e0160340 con 0x7f63e006d900 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.209+0000 7f63e5200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63e0144c10 0x7f63e015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.209+0000 7f63e5200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f63e0144c10 0x7f63e015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34440/0 (socket says 172.21.15.62:34440) 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.209+0000 7f63e5200640 1 -- 172.21.15.62:0/897080449 learned_addr learned my addr 172.21.15.62:0/897080449 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.210+0000 7f63e5c00640 1 --2- 172.21.15.62:0/897080449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 0x7f63e00b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.210+0000 7f63e5200640 1 -- 172.21.15.62:0/897080449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 msgr2=0x7f63e00b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.210+0000 7f63e5200640 1 --2- 172.21.15.62:0/897080449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 0x7f63e00b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.210+0000 7f63e5200640 1 -- 172.21.15.62:0/897080449 >> v1:172.21.15.121:6789/0 conn(0x7f63e00cee40 legacy=0x7f63e006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:34.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.210+0000 7f63e5200640 1 -- 172.21.15.62:0/897080449 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e015faa0 con 0x7f63e0144c10 2024-04-25T09:17:34.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.211+0000 7f63e5200640 1 --2- 172.21.15.62:0/897080449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e0144c10 0x7f63e015f450 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009e50 tx=0x7f63d002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be8901452f4e42d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:34.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.211+0000 7f63dfe00640 1 -- 172.21.15.62:0/897080449 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0036070 con 0x7f63e0144c10 2024-04-25T09:17:34.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.211+0000 7f63dfe00640 1 -- 172.21.15.62:0/897080449 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d0031600 con 0x7f63e0144c10 2024-04-25T09:17:34.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.211+0000 7f63dfe00640 1 -- 172.21.15.62:0/897080449 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d00319d0 con 0x7f63e0144c10 2024-04-25T09:17:34.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.212+0000 7f63e6600640 1 -- 172.21.15.62:0/897080449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e0144c10 msgr2=0x7f63e015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.212+0000 7f63e6600640 1 --2- 172.21.15.62:0/897080449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e0144c10 0x7f63e015f450 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009e50 tx=0x7f63d002a630 comp rx=0 tx=0).stop 2024-04-25T09:17:34.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.212+0000 7f63e6600640 1 -- 172.21.15.62:0/897080449 shutdown_connections 2024-04-25T09:17:34.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.212+0000 7f63e6600640 1 --2- 172.21.15.62:0/897080449 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 0x7f63e00b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.212+0000 7f63e6600640 1 --2- 172.21.15.62:0/897080449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e0144c10 0x7f63e015f450 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.212+0000 7f63e6600640 1 -- 172.21.15.62:0/897080449 >> 172.21.15.62:0/897080449 conn(0x7f63e00b14d0 msgr2=0x7f63e00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:34.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.212+0000 7f63e6600640 1 -- 172.21.15.62:0/897080449 shutdown_connections 2024-04-25T09:17:34.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.212+0000 7f63e6600640 1 -- 172.21.15.62:0/897080449 wait complete. 2024-04-25T09:17:34.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.214+0000 7f63e6600640 1 Processor -- start 2024-04-25T09:17:34.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.214+0000 7f63e6600640 1 -- start start 2024-04-25T09:17:34.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.215+0000 7f63e6600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 0x7f63e013e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.215+0000 7f63e6600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e00cee40 0x7f63e015e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.215+0000 7f63e6600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63e0144c10 0x7f63e0140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.215+0000 7f63e6600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e0160060 con 0x7f63e00cee40 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.215+0000 7f63e6600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00b41c0 con 0x7f63e0144c10 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.215+0000 7f63e6600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00b4b90 con 0x7f63e006d900 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.215+0000 7f63d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e00cee40 0x7f63e015e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e00cee40 0x7f63e015e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34444/0 (socket says 172.21.15.62:34444) 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63d7e00640 1 -- 172.21.15.62:0/885349148 learned_addr learned my addr 172.21.15.62:0/885349148 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63e5200640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 0x7f63e013e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63d7e00640 1 -- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 msgr2=0x7f63e013e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63d7e00640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 0x7f63e013e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63d7e00640 1 -- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63e0144c10 msgr2=0x7f63e0140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63d7e00640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63e0144c10 0x7f63e0140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63d7e00640 1 -- 172.21.15.62:0/885349148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0140c00 con 0x7f63e00cee40 2024-04-25T09:17:34.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63d7e00640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e00cee40 0x7f63e015e9b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c430 tx=0x7f63c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:34.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.216+0000 7f63dd600640 1 -- 172.21.15.62:0/885349148 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8011790 con 0x7f63e00cee40 2024-04-25T09:17:34.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.217+0000 7f63dd600640 1 -- 172.21.15.62:0/885349148 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8017070 con 0x7f63e00cee40 2024-04-25T09:17:34.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.217+0000 7f63dd600640 1 -- 172.21.15.62:0/885349148 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8011bb0 con 0x7f63e00cee40 2024-04-25T09:17:34.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.217+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e0140e70 con 0x7f63e00cee40 2024-04-25T09:17:34.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.217+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e0141430 con 0x7f63e00cee40 2024-04-25T09:17:34.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.218+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e004ef20 con 0x7f63e00cee40 2024-04-25T09:17:34.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.219+0000 7f63dd600640 1 -- 172.21.15.62:0/885349148 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63c801c460 con 0x7f63e00cee40 2024-04-25T09:17:34.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.220+0000 7f63dd600640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f63bc06b500 0x7f63bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.220+0000 7f63dd600640 1 -- 172.21.15.62:0/885349148 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63c8089ff0 con 0x7f63e00cee40 2024-04-25T09:17:34.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.221+0000 7f63e5200640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f63bc06b500 0x7f63bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.222+0000 7f63e5200640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f63bc06b500 0x7f63bc06d9c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f63d0033000 tx=0x7f63d0009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:34.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.224+0000 7f63dd600640 1 -- 172.21.15.62:0/885349148 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63c8058ec0 con 0x7f63e00cee40 2024-04-25T09:17:34.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.458+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63e006b210 con 0x7f63e00cee40 2024-04-25T09:17:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.459+0000 7f63dd600640 1 -- 172.21.15.62:0/885349148 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f63c80587c0 con 0x7f63e00cee40 2024-04-25T09:17:34.446 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:34.446 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3369583759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3369583759},{"type":"v1","addr":"172.21.15.121:6837","nonce":3369583759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1888749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1888749},{"type":"v1","addr":"172.21.15.121:6836","nonce":1888749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":9390,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1673530113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1673530113},{"type":"v1","addr":"172.21.15.62:6833","nonce":1673530113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:17:30.382991+0000","modified":"2024-04-25T09:17:33.474949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9396":{"gid":9396,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2793862963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2793862963},{"type":"v1","addr":"172.21.15.62:6835","nonce":2793862963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:17:34.446 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 99 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.461+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f63bc06b500 msgr2=0x7f63bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.461+0000 7f63e6600640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f63bc06b500 0x7f63bc06d9c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f63d0033000 tx=0x7f63d0009000 comp rx=0 tx=0).stop 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.461+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e00cee40 msgr2=0x7f63e015e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.461+0000 7f63e6600640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e00cee40 0x7f63e015e9b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c430 tx=0x7f63c800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.461+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 shutdown_connections 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.461+0000 7f63e6600640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f63bc06b500 0x7f63bc06d9c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.461+0000 7f63e6600640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f63e00cee40 0x7f63e015e9b0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.462+0000 7f63e6600640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63e0144c10 0x7f63e0140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.462+0000 7f63e6600640 1 --2- 172.21.15.62:0/885349148 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f63e006d900 0x7f63e013e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.462+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 >> 172.21.15.62:0/885349148 conn(0x7f63e00b14d0 msgr2=0x7f63e006c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.462+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 shutdown_connections 2024-04-25T09:17:34.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.462+0000 7f63e6600640 1 -- 172.21.15.62:0/885349148 wait complete. 2024-04-25T09:17:34.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:17:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.688+0000 7fa508a00640 1 Processor -- start 2024-04-25T09:17:34.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.688+0000 7fa508a00640 1 -- start start 2024-04-25T09:17:34.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.689+0000 7fa508a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5040b6a90 0x7fa5040d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.689+0000 7fa508a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa5040b27a0 0x7fa5040d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.689+0000 7fa508a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5040d3110 con 0x7fa5040b56f0 2024-04-25T09:17:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.689+0000 7fa508a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5040cedc0 con 0x7fa5040b27a0 2024-04-25T09:17:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.689+0000 7fa508a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040d33f0 con 0x7fa5040b6a90 2024-04-25T09:17:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.689+0000 7fa503400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa5040b56f0 0x7fa504160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42798/0 (socket says 172.21.15.62:42798) 2024-04-25T09:17:34.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.689+0000 7fa503400640 1 -- 172.21.15.62:0/2096749796 learned_addr learned my addr 172.21.15.62:0/2096749796 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944709397 0 0) 0x7fa5040d3110 con 0x7fa5040b56f0 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4e8003520 con 0x7fa5040b56f0 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502a00640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5040b6a90 0x7fa5040d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa503e00640 1 --2- 172.21.15.62:0/2096749796 >> v2:172.21.15.121:3300/0 conn(0x7fa5040b27a0 0x7fa5040d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2600170992 0 0) 0x7fa4e8003520 con 0x7fa5040b56f0 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5040b6a90 msgr2=0x7fa5040d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5040b6a90 0x7fa5040d1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 >> v2:172.21.15.121:3300/0 conn(0x7fa5040b27a0 msgr2=0x7fa5040d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 --2- 172.21.15.62:0/2096749796 >> v2:172.21.15.121:3300/0 conn(0x7fa5040b27a0 0x7fa5040d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040d36d0 con 0x7fa5040b56f0 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fa4f4001670 con 0x7fa5040b56f0 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 >> v1:172.21.15.62:6789/0 conn(0x7fa5040b56f0 legacy=0x7fa504160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4e80039b0 0x7fa4e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.690+0000 7fa502000640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502000640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5040b6a90 0x7fa4e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040cedc0 con 0x7fa4e8004800 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040d33f0 con 0x7fa4e80039b0 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e800baf0 con 0x7fa5040b6a90 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502a00640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa503400640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4e80039b0 0x7fa4e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502a00640 1 -- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5040b6a90 msgr2=0x7fa4e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:34.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502a00640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa5040b6a90 0x7fa4e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502a00640 1 -- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4e80039b0 msgr2=0x7fa4e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:34.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502a00640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4e80039b0 0x7fa4e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.691+0000 7fa502a00640 1 -- 172.21.15.62:0/2096749796 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8003520 con 0x7fa4e8004800 2024-04-25T09:17:34.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.692+0000 7fa502a00640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000ba00 tx=0x7fa4e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7c2d38053faae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0030070 con 0x7fa4e8004800 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f002adb0 con 0x7fa4e8004800 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa502000640 1 -- 172.21.15.62:0/2096749796 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000b600 con 0x7fa4e8004800 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa508a00640 1 -- 172.21.15.62:0/2096749796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 msgr2=0x7fa4e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa508a00640 1 --2- 172.21.15.62:0/2096749796 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000ba00 tx=0x7fa4e8005830 comp rx=0 tx=0).stop 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa503e00640 1 -- 172.21.15.62:0/2096749796 reap_dead start 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa508a00640 1 -- 172.21.15.62:0/2096749796 shutdown_connections 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa508a00640 1 -- 172.21.15.62:0/2096749796 >> 172.21.15.62:0/2096749796 conn(0x7fa504066d10 msgr2=0x7fa504067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa508a00640 1 -- 172.21.15.62:0/2096749796 shutdown_connections 2024-04-25T09:17:34.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.693+0000 7fa508a00640 1 -- 172.21.15.62:0/2096749796 wait complete. 2024-04-25T09:17:34.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.695+0000 7fa508a00640 1 Processor -- start 2024-04-25T09:17:34.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.695+0000 7fa508a00640 1 -- start start 2024-04-25T09:17:34.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.696+0000 7fa508a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa504149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.696+0000 7fa508a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa504149f40 0x7fa50414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.696+0000 7fa508a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa50415e360 0x7fa50415e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.696+0000 7fa508a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040b2ee0 con 0x7fa4e8004800 2024-04-25T09:17:34.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.696+0000 7fa508a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040b2d60 con 0x7fa504149f40 2024-04-25T09:17:34.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.696+0000 7fa508a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040b5ad0 con 0x7fa50415e360 2024-04-25T09:17:34.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa504149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa504149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:34456/0 (socket says 172.21.15.62:34456) 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 -- 172.21.15.62:0/1792870452 learned_addr learned my addr 172.21.15.62:0/1792870452 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 -- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa50415e360 msgr2=0x7fa50415e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa50415e360 0x7fa50415e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 -- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa504149f40 msgr2=0x7fa50414a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa504149f40 0x7fa50414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 -- 172.21.15.62:0/1792870452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50415d300 con 0x7fa4e8004800 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.697+0000 7fa503400640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa504149780 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400e730 tx=0x7fa4f400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:34.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.698+0000 7fa4fb400640 1 -- 172.21.15.62:0/1792870452 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f400ee40 con 0x7fa4e8004800 2024-04-25T09:17:34.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.698+0000 7fa4fb400640 1 -- 172.21.15.62:0/1792870452 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f4018070 con 0x7fa4e8004800 2024-04-25T09:17:34.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.698+0000 7fa4fb400640 1 -- 172.21.15.62:0/1792870452 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f400caf0 con 0x7fa4e8004800 2024-04-25T09:17:34.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.698+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50415d5a0 con 0x7fa4e8004800 2024-04-25T09:17:34.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.698+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50415db60 con 0x7fa4e8004800 2024-04-25T09:17:34.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.699+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa504074120 con 0x7fa4e8004800 2024-04-25T09:17:34.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.700+0000 7fa4fb400640 1 -- 172.21.15.62:0/1792870452 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4f400cc90 con 0x7fa4e8004800 2024-04-25T09:17:34.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.701+0000 7fa4fb400640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4e406b550 0x7fa4e406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:34.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.701+0000 7fa4fb400640 1 -- 172.21.15.62:0/1792870452 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4f4089db0 con 0x7fa4e8004800 2024-04-25T09:17:34.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.702+0000 7fa502a00640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4e406b550 0x7fa4e406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:34.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.703+0000 7fa502a00640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4e406b550 0x7fa4e406da10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8004780 tx=0x7fa4f002c000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:34.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.705+0000 7fa4fb400640 1 -- 172.21.15.62:0/1792870452 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa504074120 con 0x7fa4e8004800 2024-04-25T09:17:34.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.905+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa50404ef20 con 0x7fa4e8004800 2024-04-25T09:17:34.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.907+0000 7fa4fb400640 1 -- 172.21.15.62:0/1792870452 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16667 (secure 0 0 0) 0x7fa4f408e070 con 0x7fa4e8004800 2024-04-25T09:17:34.894 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:34.894 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":148,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:17:31.946999+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:17:28.263758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:17:29.401316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:17:34.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4e406b550 msgr2=0x7fa4e406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4e406b550 0x7fa4e406da10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8004780 tx=0x7fa4f002c000 comp rx=0 tx=0).stop 2024-04-25T09:17:34.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 msgr2=0x7fa504149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:34.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa504149780 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400e730 tx=0x7fa4f400ebf0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 shutdown_connections 2024-04-25T09:17:34.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa4e406b550 0x7fa4e406da10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa50415e360 0x7fa50415e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa504149f40 0x7fa50414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 --2- 172.21.15.62:0/1792870452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4e8004800 0x7fa504149780 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:34.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 >> 172.21.15.62:0/1792870452 conn(0x7fa504066d10 msgr2=0x7fa5040b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:34.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 shutdown_connections 2024-04-25T09:17:34.898 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:34.910+0000 7fa508a00640 1 -- 172.21.15.62:0/1792870452 wait complete. 2024-04-25T09:17:34.917 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:17:34.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:17:35.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.139+0000 7fdca0a00640 1 Processor -- start 2024-04-25T09:17:35.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.140+0000 7fdca0a00640 1 -- start start 2024-04-25T09:17:35.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.141+0000 7fdca0a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc9c15b7b0 0x7fdc9c0d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.141+0000 7fdca0a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc9c0d1970 0x7fdc9c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.141+0000 7fdca0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c0b6640 0x7fdc9c161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.141+0000 7fdca0a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c15bb90 con 0x7fdc9c0d1970 2024-04-25T09:17:35.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.141+0000 7fdca0a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c0d1e00 con 0x7fdc9c15b7b0 2024-04-25T09:17:35.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.141+0000 7fdca0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c15e250 con 0x7fdc9c0b6640 2024-04-25T09:17:35.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.142+0000 7fdc99e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc9c0d1970 0x7fdc9c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.142+0000 7fdc99e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdc9c0d1970 0x7fdc9c0b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38268/0 (socket says 172.21.15.62:38268) 2024-04-25T09:17:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.142+0000 7fdc99e00640 1 -- 172.21.15.62:0/1226568621 learned_addr learned my addr 172.21.15.62:0/1226568621 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.142+0000 7fdc9b200640 1 --2- 172.21.15.62:0/1226568621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c0b6640 0x7fdc9c161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.142+0000 7fdc9a800640 1 --2- 172.21.15.62:0/1226568621 >> v2:172.21.15.121:3300/0 conn(0x7fdc9c15b7b0 0x7fdc9c0d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.142+0000 7fdc99e00640 1 -- 172.21.15.62:0/1226568621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c0b6640 msgr2=0x7fdc9c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.142+0000 7fdc99e00640 1 --2- 172.21.15.62:0/1226568621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c0b6640 0x7fdc9c161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.143+0000 7fdc99e00640 1 -- 172.21.15.62:0/1226568621 >> v2:172.21.15.121:3300/0 conn(0x7fdc9c15b7b0 msgr2=0x7fdc9c0d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.143+0000 7fdc99e00640 1 --2- 172.21.15.62:0/1226568621 >> v2:172.21.15.121:3300/0 conn(0x7fdc9c15b7b0 0x7fdc9c0d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.143+0000 7fdc99e00640 1 -- 172.21.15.62:0/1226568621 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c15e530 con 0x7fdc9c0d1970 2024-04-25T09:17:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.144+0000 7fdc99e00640 1 --2- 172.21.15.62:0/1226568621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0d1970 0x7fdc9c0b6100 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c06f9a0 tx=0x7fdc8802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fd42194809eb32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.144+0000 7fdc99400640 1 -- 172.21.15.62:0/1226568621 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88036070 con 0x7fdc9c0d1970 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.144+0000 7fdc99400640 1 -- 172.21.15.62:0/1226568621 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc88031600 con 0x7fdc9c0d1970 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.144+0000 7fdc99400640 1 -- 172.21.15.62:0/1226568621 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc880319d0 con 0x7fdc9c0d1970 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 -- 172.21.15.62:0/1226568621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0d1970 msgr2=0x7fdc9c0b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 --2- 172.21.15.62:0/1226568621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0d1970 0x7fdc9c0b6100 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c06f9a0 tx=0x7fdc8802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 -- 172.21.15.62:0/1226568621 shutdown_connections 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 --2- 172.21.15.62:0/1226568621 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c0b6640 0x7fdc9c161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 --2- 172.21.15.62:0/1226568621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0d1970 0x7fdc9c0b6100 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 --2- 172.21.15.62:0/1226568621 >> v2:172.21.15.121:3300/0 conn(0x7fdc9c15b7b0 0x7fdc9c0d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 -- 172.21.15.62:0/1226568621 >> 172.21.15.62:0/1226568621 conn(0x7fdc9c072340 msgr2=0x7fdc9c072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 -- 172.21.15.62:0/1226568621 shutdown_connections 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.145+0000 7fdca0a00640 1 -- 172.21.15.62:0/1226568621 wait complete. 2024-04-25T09:17:35.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.147+0000 7fdca0a00640 1 Processor -- start 2024-04-25T09:17:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.147+0000 7fdca0a00640 1 -- start start 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdca0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0b6640 0x7fdc9c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdca0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc9c0d1970 0x7fdc9c0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdca0a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c15b7b0 0x7fdc9c14be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdca0a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b6ad0 con 0x7fdc9c0b6640 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdca0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c15eaf0 con 0x7fdc9c0d1970 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdca0a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0d2080 con 0x7fdc9c15b7b0 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdc9a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0b6640 0x7fdc9c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdc9a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0b6640 0x7fdc9c15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38274/0 (socket says 172.21.15.62:38274) 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdc9a800640 1 -- 172.21.15.62:0/520020494 learned_addr learned my addr 172.21.15.62:0/520020494 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:35.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.148+0000 7fdc99e00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc9c0d1970 0x7fdc9c0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc9b200640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c15b7b0 0x7fdc9c14be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc9a800640 1 -- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c15b7b0 msgr2=0x7fdc9c14be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc9a800640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c15b7b0 0x7fdc9c14be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc9a800640 1 -- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc9c0d1970 msgr2=0x7fdc9c0b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc9a800640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc9c0d1970 0x7fdc9c0b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc9a800640 1 -- 172.21.15.62:0/520020494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0b3b90 con 0x7fdc9c0b6640 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc9b200640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c15b7b0 0x7fdc9c14be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc99e00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc9c0d1970 0x7fdc9c0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc9a800640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0b6640 0x7fdc9c15c8e0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00d5f0 tx=0x7fdc8c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc92a00640 1 -- 172.21.15.62:0/520020494 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0125f0 con 0x7fdc9c0b6640 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc92a00640 1 -- 172.21.15.62:0/520020494 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c019070 con 0x7fdc9c0b6640 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdc92a00640 1 -- 172.21.15.62:0/520020494 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0129d0 con 0x7fdc9c0b6640 2024-04-25T09:17:35.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.149+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c15be00 con 0x7fdc9c0b6640 2024-04-25T09:17:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.150+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c0b38a0 con 0x7fdc9c0b6640 2024-04-25T09:17:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.151+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c151ac0 con 0x7fdc9c0b6640 2024-04-25T09:17:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.152+0000 7fdc92a00640 1 -- 172.21.15.62:0/520020494 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc8c012b70 con 0x7fdc9c0b6640 2024-04-25T09:17:35.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.152+0000 7fdc92a00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdc7c06b500 0x7fdc7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.153+0000 7fdc92a00640 1 -- 172.21.15.62:0/520020494 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc8c08a5c0 con 0x7fdc9c0b6640 2024-04-25T09:17:35.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.153+0000 7fdc99e00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdc7c06b500 0x7fdc7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.154+0000 7fdc99e00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdc7c06b500 0x7fdc7c06d9c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdc88033000 tx=0x7fdc88009000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:35.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.157+0000 7fdc92a00640 1 -- 172.21.15.62:0/520020494 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc8c059440 con 0x7fdc9c0b6640 2024-04-25T09:17:35.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.352+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc9c14c3b0 con 0x7fdc7c06b500 2024-04-25T09:17:35.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.371+0000 7fdc92a00640 1 -- 172.21.15.62:0/520020494 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252409 (secure 0 0 0) 0x7fdc9c14c3b0 con 0x7fdc7c06b500 2024-04-25T09:17:35.360 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:35.363 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:17:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.377+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdc7c06b500 msgr2=0x7fdc7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.377+0000 7fdca0a00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdc7c06b500 0x7fdc7c06d9c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdc88033000 tx=0x7fdc88009000 comp rx=0 tx=0).stop 2024-04-25T09:17:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.377+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0b6640 msgr2=0x7fdc9c15c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.377+0000 7fdca0a00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0b6640 0x7fdc9c15c8e0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00d5f0 tx=0x7fdc8c00dab0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.378+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 shutdown_connections 2024-04-25T09:17:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.378+0000 7fdca0a00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdc7c06b500 0x7fdc7c06d9c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.378+0000 7fdca0a00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdc9c15b7b0 0x7fdc9c14be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.378+0000 7fdca0a00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc9c0d1970 0x7fdc9c0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.378+0000 7fdca0a00640 1 --2- 172.21.15.62:0/520020494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc9c0b6640 0x7fdc9c15c8e0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.378+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 >> 172.21.15.62:0/520020494 conn(0x7fdc9c072340 msgr2=0x7fdc9c15aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.378+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 shutdown_connections 2024-04-25T09:17:35.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.378+0000 7fdca0a00640 1 -- 172.21.15.62:0/520020494 wait complete. 2024-04-25T09:17:35.385 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":575,"stamp":"2024-04-25T09:17:34.215612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":456468,"kb_used_data":15444,"kb_used_omap":12,"kb_used_meta":440883,"kb_avail":754518252,"statfs":{"total":773094113280,"available":772626690048,"internally_reserved":0,"allocated":15814656,"data_stored":7607719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":451464752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319567"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962063+0000","last_change":"2024-04-25T09:17:30.959007+0000","last_active":"2024-04-25T09:17:30.962063+0000","last_peered":"2024-04-25T09:17:30.962063+0000","last_clean":"2024-04-25T09:17:30.962063+0000","last_became_active":"2024-04-25T09:17:30.957737+0000","last_became_peered":"2024-04-25T09:17:30.957737+0000","last_unstale":"2024-04-25T09:17:30.962063+0000","last_undegraded":"2024-04-25T09:17:30.962063+0000","last_fullsized":"2024-04-25T09:17:30.962063+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:02.618723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.953472+0000","last_change":"2024-04-25T09:17:30.950680+0000","last_active":"2024-04-25T09:17:30.953472+0000","last_peered":"2024-04-25T09:17:30.953472+0000","last_clean":"2024-04-25T09:17:30.953472+0000","last_became_active":"2024-04-25T09:17:30.950238+0000","last_became_peered":"2024-04-25T09:17:30.950238+0000","last_unstale":"2024-04-25T09:17:30.953472+0000","last_undegraded":"2024-04-25T09:17:30.953472+0000","last_fullsized":"2024-04-25T09:17:30.953472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:30.960343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.959083+0000","last_change":"2024-04-25T09:17:30.956658+0000","last_active":"2024-04-25T09:17:30.959083+0000","last_peered":"2024-04-25T09:17:30.959083+0000","last_clean":"2024-04-25T09:17:30.959083+0000","last_became_active":"2024-04-25T09:17:30.956313+0000","last_became_peered":"2024-04-25T09:17:30.956313+0000","last_unstale":"2024-04-25T09:17:30.959083+0000","last_undegraded":"2024-04-25T09:17:30.959083+0000","last_fullsized":"2024-04-25T09:17:30.959083+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:52.068797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.972167+0000","last_change":"2024-04-25T09:17:30.967359+0000","last_active":"2024-04-25T09:17:30.972167+0000","last_peered":"2024-04-25T09:17:30.972167+0000","last_clean":"2024-04-25T09:17:30.972167+0000","last_became_active":"2024-04-25T09:17:30.967087+0000","last_became_peered":"2024-04-25T09:17:30.967087+0000","last_unstale":"2024-04-25T09:17:30.972167+0000","last_undegraded":"2024-04-25T09:17:30.972167+0000","last_fullsized":"2024-04-25T09:17:30.972167+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:31.959406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.974021+0000","last_change":"2024-04-25T09:17:30.971779+0000","last_active":"2024-04-25T09:17:30.974021+0000","last_peered":"2024-04-25T09:17:30.974021+0000","last_clean":"2024-04-25T09:17:30.974021+0000","last_became_active":"2024-04-25T09:17:30.968575+0000","last_became_peered":"2024-04-25T09:17:30.968575+0000","last_unstale":"2024-04-25T09:17:30.974021+0000","last_undegraded":"2024-04-25T09:17:30.974021+0000","last_fullsized":"2024-04-25T09:17:30.974021+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:44.964670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.959851+0000","last_change":"2024-04-25T09:17:30.957311+0000","last_active":"2024-04-25T09:17:30.959851+0000","last_peered":"2024-04-25T09:17:30.959851+0000","last_clean":"2024-04-25T09:17:30.959851+0000","last_became_active":"2024-04-25T09:17:30.955373+0000","last_became_peered":"2024-04-25T09:17:30.955373+0000","last_unstale":"2024-04-25T09:17:30.959851+0000","last_undegraded":"2024-04-25T09:17:30.959851+0000","last_fullsized":"2024-04-25T09:17:30.959851+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:02.618723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.973925+0000","last_change":"2024-04-25T09:17:30.973457+0000","last_active":"2024-04-25T09:17:30.973925+0000","last_peered":"2024-04-25T09:17:30.973925+0000","last_clean":"2024-04-25T09:17:30.973925+0000","last_became_active":"2024-04-25T09:17:30.973204+0000","last_became_peered":"2024-04-25T09:17:30.973204+0000","last_unstale":"2024-04-25T09:17:30.973925+0000","last_undegraded":"2024-04-25T09:17:30.973925+0000","last_fullsized":"2024-04-25T09:17:30.973925+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:01.343706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.964872+0000","last_change":"2024-04-25T09:17:30.963088+0000","last_active":"2024-04-25T09:17:30.964872+0000","last_peered":"2024-04-25T09:17:30.964872+0000","last_clean":"2024-04-25T09:17:30.964872+0000","last_became_active":"2024-04-25T09:17:30.962697+0000","last_became_peered":"2024-04-25T09:17:30.962697+0000","last_unstale":"2024-04-25T09:17:30.964872+0000","last_undegraded":"2024-04-25T09:17:30.964872+0000","last_fullsized":"2024-04-25T09:17:30.964872+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:02.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.957012+0000","last_change":"2024-04-25T09:17:30.952649+0000","last_active":"2024-04-25T09:17:30.957012+0000","last_peered":"2024-04-25T09:17:30.957012+0000","last_clean":"2024-04-25T09:17:30.957012+0000","last_became_active":"2024-04-25T09:17:30.947002+0000","last_became_peered":"2024-04-25T09:17:30.947002+0000","last_unstale":"2024-04-25T09:17:30.957012+0000","last_undegraded":"2024-04-25T09:17:30.957012+0000","last_fullsized":"2024-04-25T09:17:30.957012+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:30.351587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967883+0000","last_change":"2024-04-25T09:17:30.962153+0000","last_active":"2024-04-25T09:17:30.967883+0000","last_peered":"2024-04-25T09:17:30.967883+0000","last_clean":"2024-04-25T09:17:30.967883+0000","last_became_active":"2024-04-25T09:17:30.961798+0000","last_became_peered":"2024-04-25T09:17:30.961798+0000","last_unstale":"2024-04-25T09:17:30.967883+0000","last_undegraded":"2024-04-25T09:17:30.967883+0000","last_fullsized":"2024-04-25T09:17:30.967883+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:10.077072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.964295+0000","last_change":"2024-04-25T09:17:30.963474+0000","last_active":"2024-04-25T09:17:30.964295+0000","last_peered":"2024-04-25T09:17:30.964295+0000","last_clean":"2024-04-25T09:17:30.964295+0000","last_became_active":"2024-04-25T09:17:30.963186+0000","last_became_peered":"2024-04-25T09:17:30.963186+0000","last_unstale":"2024-04-25T09:17:30.964295+0000","last_undegraded":"2024-04-25T09:17:30.964295+0000","last_fullsized":"2024-04-25T09:17:30.964295+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:04.038785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.975429+0000","last_change":"2024-04-25T09:17:30.970200+0000","last_active":"2024-04-25T09:17:30.975429+0000","last_peered":"2024-04-25T09:17:30.975429+0000","last_clean":"2024-04-25T09:17:30.975429+0000","last_became_active":"2024-04-25T09:17:30.969787+0000","last_became_peered":"2024-04-25T09:17:30.969787+0000","last_unstale":"2024-04-25T09:17:30.975429+0000","last_undegraded":"2024-04-25T09:17:30.975429+0000","last_fullsized":"2024-04-25T09:17:30.975429+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:31.959406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.973088+0000","last_change":"2024-04-25T09:17:30.970242+0000","last_active":"2024-04-25T09:17:30.973088+0000","last_peered":"2024-04-25T09:17:30.973088+0000","last_clean":"2024-04-25T09:17:30.973088+0000","last_became_active":"2024-04-25T09:17:30.967961+0000","last_became_peered":"2024-04-25T09:17:30.967961+0000","last_unstale":"2024-04-25T09:17:30.973088+0000","last_undegraded":"2024-04-25T09:17:30.973088+0000","last_fullsized":"2024-04-25T09:17:30.973088+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:42.126244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.976243+0000","last_change":"2024-04-25T09:17:30.973900+0000","last_active":"2024-04-25T09:17:30.976243+0000","last_peered":"2024-04-25T09:17:30.976243+0000","last_clean":"2024-04-25T09:17:30.976243+0000","last_became_active":"2024-04-25T09:17:30.973608+0000","last_became_peered":"2024-04-25T09:17:30.973608+0000","last_unstale":"2024-04-25T09:17:30.976243+0000","last_undegraded":"2024-04-25T09:17:30.976243+0000","last_fullsized":"2024-04-25T09:17:30.976243+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:51.787201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.960554+0000","last_change":"2024-04-25T09:17:30.953808+0000","last_active":"2024-04-25T09:17:30.960554+0000","last_peered":"2024-04-25T09:17:30.960554+0000","last_clean":"2024-04-25T09:17:30.960554+0000","last_became_active":"2024-04-25T09:17:30.953477+0000","last_became_peered":"2024-04-25T09:17:30.953477+0000","last_unstale":"2024-04-25T09:17:30.960554+0000","last_undegraded":"2024-04-25T09:17:30.960554+0000","last_fullsized":"2024-04-25T09:17:30.960554+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:01.230052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.956866+0000","last_change":"2024-04-25T09:17:30.953045+0000","last_active":"2024-04-25T09:17:30.956866+0000","last_peered":"2024-04-25T09:17:30.956866+0000","last_clean":"2024-04-25T09:17:30.956866+0000","last_became_active":"2024-04-25T09:17:30.952284+0000","last_became_peered":"2024-04-25T09:17:30.952284+0000","last_unstale":"2024-04-25T09:17:30.956866+0000","last_undegraded":"2024-04-25T09:17:30.956866+0000","last_fullsized":"2024-04-25T09:17:30.956866+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:05.358730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.970798+0000","last_change":"2024-04-25T09:17:30.967313+0000","last_active":"2024-04-25T09:17:30.970798+0000","last_peered":"2024-04-25T09:17:30.970798+0000","last_clean":"2024-04-25T09:17:30.970798+0000","last_became_active":"2024-04-25T09:17:30.966126+0000","last_became_peered":"2024-04-25T09:17:30.966126+0000","last_unstale":"2024-04-25T09:17:30.970798+0000","last_undegraded":"2024-04-25T09:17:30.970798+0000","last_fullsized":"2024-04-25T09:17:30.970798+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:12.355346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962233+0000","last_change":"2024-04-25T09:17:30.960809+0000","last_active":"2024-04-25T09:17:30.962233+0000","last_peered":"2024-04-25T09:17:30.962233+0000","last_clean":"2024-04-25T09:17:30.962233+0000","last_became_active":"2024-04-25T09:17:30.958235+0000","last_became_peered":"2024-04-25T09:17:30.958235+0000","last_unstale":"2024-04-25T09:17:30.962233+0000","last_undegraded":"2024-04-25T09:17:30.962233+0000","last_fullsized":"2024-04-25T09:17:30.962233+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:09.193335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.961100+0000","last_change":"2024-04-25T09:17:30.956207+0000","last_active":"2024-04-25T09:17:30.961100+0000","last_peered":"2024-04-25T09:17:30.961100+0000","last_clean":"2024-04-25T09:17:30.961100+0000","last_became_active":"2024-04-25T09:17:30.955883+0000","last_became_peered":"2024-04-25T09:17:30.955883+0000","last_unstale":"2024-04-25T09:17:30.961100+0000","last_undegraded":"2024-04-25T09:17:30.961100+0000","last_fullsized":"2024-04-25T09:17:30.961100+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:30.351587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.951835+0000","last_change":"2024-04-25T09:17:30.946712+0000","last_active":"2024-04-25T09:17:30.951835+0000","last_peered":"2024-04-25T09:17:30.951835+0000","last_clean":"2024-04-25T09:17:30.951835+0000","last_became_active":"2024-04-25T09:17:30.946294+0000","last_became_peered":"2024-04-25T09:17:30.946294+0000","last_unstale":"2024-04-25T09:17:30.951835+0000","last_undegraded":"2024-04-25T09:17:30.951835+0000","last_fullsized":"2024-04-25T09:17:30.951835+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:33.049478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.975398+0000","last_change":"2024-04-25T09:17:30.972985+0000","last_active":"2024-04-25T09:17:30.975398+0000","last_peered":"2024-04-25T09:17:30.975398+0000","last_clean":"2024-04-25T09:17:30.975398+0000","last_became_active":"2024-04-25T09:17:30.971397+0000","last_became_peered":"2024-04-25T09:17:30.971397+0000","last_unstale":"2024-04-25T09:17:30.975398+0000","last_undegraded":"2024-04-25T09:17:30.975398+0000","last_fullsized":"2024-04-25T09:17:30.975398+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:47.611111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967846+0000","last_change":"2024-04-25T09:17:30.964043+0000","last_active":"2024-04-25T09:17:30.967846+0000","last_peered":"2024-04-25T09:17:30.967846+0000","last_clean":"2024-04-25T09:17:30.967846+0000","last_became_active":"2024-04-25T09:17:30.960757+0000","last_became_peered":"2024-04-25T09:17:30.960757+0000","last_unstale":"2024-04-25T09:17:30.967846+0000","last_undegraded":"2024-04-25T09:17:30.967846+0000","last_fullsized":"2024-04-25T09:17:30.967846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:01.343706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.972093+0000","last_change":"2024-04-25T09:17:30.970545+0000","last_active":"2024-04-25T09:17:30.972093+0000","last_peered":"2024-04-25T09:17:30.972093+0000","last_clean":"2024-04-25T09:17:30.972093+0000","last_became_active":"2024-04-25T09:17:30.968411+0000","last_became_peered":"2024-04-25T09:17:30.968411+0000","last_unstale":"2024-04-25T09:17:30.972093+0000","last_undegraded":"2024-04-25T09:17:30.972093+0000","last_fullsized":"2024-04-25T09:17:30.972093+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:31.962089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.964367+0000","last_change":"2024-04-25T09:17:30.961374+0000","last_active":"2024-04-25T09:17:30.964367+0000","last_peered":"2024-04-25T09:17:30.964367+0000","last_clean":"2024-04-25T09:17:30.964367+0000","last_became_active":"2024-04-25T09:17:30.961069+0000","last_became_peered":"2024-04-25T09:17:30.961069+0000","last_unstale":"2024-04-25T09:17:30.964367+0000","last_undegraded":"2024-04-25T09:17:30.964367+0000","last_fullsized":"2024-04-25T09:17:30.964367+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:12.744863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.959306+0000","last_change":"2024-04-25T09:17:30.958124+0000","last_active":"2024-04-25T09:17:30.959306+0000","last_peered":"2024-04-25T09:17:30.959306+0000","last_clean":"2024-04-25T09:17:30.959306+0000","last_became_active":"2024-04-25T09:17:30.957794+0000","last_became_peered":"2024-04-25T09:17:30.957794+0000","last_unstale":"2024-04-25T09:17:30.959306+0000","last_undegraded":"2024-04-25T09:17:30.959306+0000","last_fullsized":"2024-04-25T09:17:30.959306+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:34.877972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.951813+0000","last_change":"2024-04-25T09:17:30.947757+0000","last_active":"2024-04-25T09:17:30.951813+0000","last_peered":"2024-04-25T09:17:30.951813+0000","last_clean":"2024-04-25T09:17:30.951813+0000","last_became_active":"2024-04-25T09:17:30.947425+0000","last_became_peered":"2024-04-25T09:17:30.947425+0000","last_unstale":"2024-04-25T09:17:30.951813+0000","last_undegraded":"2024-04-25T09:17:30.951813+0000","last_fullsized":"2024-04-25T09:17:30.951813+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:26.411855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.960851+0000","last_change":"2024-04-25T09:17:30.958412+0000","last_active":"2024-04-25T09:17:30.960851+0000","last_peered":"2024-04-25T09:17:30.960851+0000","last_clean":"2024-04-25T09:17:30.960851+0000","last_became_active":"2024-04-25T09:17:30.957728+0000","last_became_peered":"2024-04-25T09:17:30.957728+0000","last_unstale":"2024-04-25T09:17:30.960851+0000","last_undegraded":"2024-04-25T09:17:30.960851+0000","last_fullsized":"2024-04-25T09:17:30.960851+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:47.116698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.952071+0000","last_change":"2024-04-25T09:17:30.947080+0000","last_active":"2024-04-25T09:17:30.952071+0000","last_peered":"2024-04-25T09:17:30.952071+0000","last_clean":"2024-04-25T09:17:30.952071+0000","last_became_active":"2024-04-25T09:17:30.946608+0000","last_became_peered":"2024-04-25T09:17:30.946608+0000","last_unstale":"2024-04-25T09:17:30.952071+0000","last_undegraded":"2024-04-25T09:17:30.952071+0000","last_fullsized":"2024-04-25T09:17:30.952071+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:04.517031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965036+0000","last_change":"2024-04-25T09:17:30.963750+0000","last_active":"2024-04-25T09:17:30.965036+0000","last_peered":"2024-04-25T09:17:30.965036+0000","last_clean":"2024-04-25T09:17:30.965036+0000","last_became_active":"2024-04-25T09:17:30.962750+0000","last_became_peered":"2024-04-25T09:17:30.962750+0000","last_unstale":"2024-04-25T09:17:30.965036+0000","last_undegraded":"2024-04-25T09:17:30.965036+0000","last_fullsized":"2024-04-25T09:17:30.965036+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:04.038785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.963868+0000","last_change":"2024-04-25T09:17:30.960106+0000","last_active":"2024-04-25T09:17:30.963868+0000","last_peered":"2024-04-25T09:17:30.963868+0000","last_clean":"2024-04-25T09:17:30.963868+0000","last_became_active":"2024-04-25T09:17:30.959093+0000","last_became_peered":"2024-04-25T09:17:30.959093+0000","last_unstale":"2024-04-25T09:17:30.963868+0000","last_undegraded":"2024-04-25T09:17:30.963868+0000","last_fullsized":"2024-04-25T09:17:30.963868+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:05.413500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.955900+0000","last_change":"2024-04-25T09:17:30.953983+0000","last_active":"2024-04-25T09:17:30.955900+0000","last_peered":"2024-04-25T09:17:30.955900+0000","last_clean":"2024-04-25T09:17:30.955900+0000","last_became_active":"2024-04-25T09:17:30.953157+0000","last_became_peered":"2024-04-25T09:17:30.953157+0000","last_unstale":"2024-04-25T09:17:30.955900+0000","last_undegraded":"2024-04-25T09:17:30.955900+0000","last_fullsized":"2024-04-25T09:17:30.955900+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:59.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.961322+0000","last_change":"2024-04-25T09:17:30.959660+0000","last_active":"2024-04-25T09:17:30.961322+0000","last_peered":"2024-04-25T09:17:30.961322+0000","last_clean":"2024-04-25T09:17:30.961322+0000","last_became_active":"2024-04-25T09:17:30.959308+0000","last_became_peered":"2024-04-25T09:17:30.959308+0000","last_unstale":"2024-04-25T09:17:30.961322+0000","last_undegraded":"2024-04-25T09:17:30.961322+0000","last_fullsized":"2024-04-25T09:17:30.961322+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:51.013903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962147+0000","last_change":"2024-04-25T09:17:30.960550+0000","last_active":"2024-04-25T09:17:30.962147+0000","last_peered":"2024-04-25T09:17:30.962147+0000","last_clean":"2024-04-25T09:17:30.962147+0000","last_became_active":"2024-04-25T09:17:30.957800+0000","last_became_peered":"2024-04-25T09:17:30.957800+0000","last_unstale":"2024-04-25T09:17:30.962147+0000","last_undegraded":"2024-04-25T09:17:30.962147+0000","last_fullsized":"2024-04-25T09:17:30.962147+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:26.341065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"147'4","reported_seq":16,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.984424+0000","last_change":"2024-04-25T09:17:29.934137+0000","last_active":"2024-04-25T09:17:30.984424+0000","last_peered":"2024-04-25T09:17:30.984424+0000","last_clean":"2024-04-25T09:17:30.984424+0000","last_became_active":"2024-04-25T09:17:29.933695+0000","last_became_peered":"2024-04-25T09:17:29.933695+0000","last_unstale":"2024-04-25T09:17:30.984424+0000","last_undegraded":"2024-04-25T09:17:30.984424+0000","last_fullsized":"2024-04-25T09:17:30.984424+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:49.888869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.958971+0000","last_change":"2024-04-25T09:17:30.956249+0000","last_active":"2024-04-25T09:17:30.958971+0000","last_peered":"2024-04-25T09:17:30.958971+0000","last_clean":"2024-04-25T09:17:30.958971+0000","last_became_active":"2024-04-25T09:17:30.955971+0000","last_became_peered":"2024-04-25T09:17:30.955971+0000","last_unstale":"2024-04-25T09:17:30.958971+0000","last_undegraded":"2024-04-25T09:17:30.958971+0000","last_fullsized":"2024-04-25T09:17:30.958971+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:23.344835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.982922+0000","last_change":"2024-04-25T09:17:29.934947+0000","last_active":"2024-04-25T09:17:30.982922+0000","last_peered":"2024-04-25T09:17:30.982922+0000","last_clean":"2024-04-25T09:17:30.982922+0000","last_became_active":"2024-04-25T09:17:29.934316+0000","last_became_peered":"2024-04-25T09:17:29.934316+0000","last_unstale":"2024-04-25T09:17:30.982922+0000","last_undegraded":"2024-04-25T09:17:30.982922+0000","last_fullsized":"2024-04-25T09:17:30.982922+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:25.397215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965007+0000","last_change":"2024-04-25T09:17:30.964282+0000","last_active":"2024-04-25T09:17:30.965007+0000","last_peered":"2024-04-25T09:17:30.965007+0000","last_clean":"2024-04-25T09:17:30.965007+0000","last_became_active":"2024-04-25T09:17:30.963728+0000","last_became_peered":"2024-04-25T09:17:30.963728+0000","last_unstale":"2024-04-25T09:17:30.965007+0000","last_undegraded":"2024-04-25T09:17:30.965007+0000","last_fullsized":"2024-04-25T09:17:30.965007+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:02.549839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.980997+0000","last_change":"2024-04-25T09:17:29.937182+0000","last_active":"2024-04-25T09:17:30.980997+0000","last_peered":"2024-04-25T09:17:30.980997+0000","last_clean":"2024-04-25T09:17:30.980997+0000","last_became_active":"2024-04-25T09:17:29.936889+0000","last_became_peered":"2024-04-25T09:17:29.936889+0000","last_unstale":"2024-04-25T09:17:30.980997+0000","last_undegraded":"2024-04-25T09:17:30.980997+0000","last_fullsized":"2024-04-25T09:17:30.980997+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:46.832028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965979+0000","last_change":"2024-04-25T09:17:30.959788+0000","last_active":"2024-04-25T09:17:30.965979+0000","last_peered":"2024-04-25T09:17:30.965979+0000","last_clean":"2024-04-25T09:17:30.965979+0000","last_became_active":"2024-04-25T09:17:30.959452+0000","last_became_peered":"2024-04-25T09:17:30.959452+0000","last_unstale":"2024-04-25T09:17:30.965979+0000","last_undegraded":"2024-04-25T09:17:30.965979+0000","last_fullsized":"2024-04-25T09:17:30.965979+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:12.717063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.983088+0000","last_change":"2024-04-25T09:17:29.935082+0000","last_active":"2024-04-25T09:17:30.983088+0000","last_peered":"2024-04-25T09:17:30.983088+0000","last_clean":"2024-04-25T09:17:30.983088+0000","last_became_active":"2024-04-25T09:17:29.934795+0000","last_became_peered":"2024-04-25T09:17:29.934795+0000","last_unstale":"2024-04-25T09:17:30.983088+0000","last_undegraded":"2024-04-25T09:17:30.983088+0000","last_fullsized":"2024-04-25T09:17:30.983088+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:38.336509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.956134+0000","last_change":"2024-04-25T09:17:30.954614+0000","last_active":"2024-04-25T09:17:30.956134+0000","last_peered":"2024-04-25T09:17:30.956134+0000","last_clean":"2024-04-25T09:17:30.956134+0000","last_became_active":"2024-04-25T09:17:30.954281+0000","last_became_peered":"2024-04-25T09:17:30.954281+0000","last_unstale":"2024-04-25T09:17:30.956134+0000","last_undegraded":"2024-04-25T09:17:30.956134+0000","last_fullsized":"2024-04-25T09:17:30.956134+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:29.823514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.981954+0000","last_change":"2024-04-25T09:17:29.938258+0000","last_active":"2024-04-25T09:17:30.981954+0000","last_peered":"2024-04-25T09:17:30.981954+0000","last_clean":"2024-04-25T09:17:30.981954+0000","last_became_active":"2024-04-25T09:17:29.937868+0000","last_became_peered":"2024-04-25T09:17:29.937868+0000","last_unstale":"2024-04-25T09:17:30.981954+0000","last_undegraded":"2024-04-25T09:17:30.981954+0000","last_fullsized":"2024-04-25T09:17:30.981954+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:30.944766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969844+0000","last_change":"2024-04-25T09:17:30.962742+0000","last_active":"2024-04-25T09:17:30.969844+0000","last_peered":"2024-04-25T09:17:30.969844+0000","last_clean":"2024-04-25T09:17:30.969844+0000","last_became_active":"2024-04-25T09:17:30.961444+0000","last_became_peered":"2024-04-25T09:17:30.961444+0000","last_unstale":"2024-04-25T09:17:30.969844+0000","last_undegraded":"2024-04-25T09:17:30.969844+0000","last_fullsized":"2024-04-25T09:17:30.969844+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:24.193920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.982534+0000","last_change":"2024-04-25T09:17:29.937932+0000","last_active":"2024-04-25T09:17:30.982534+0000","last_peered":"2024-04-25T09:17:30.982534+0000","last_clean":"2024-04-25T09:17:30.982534+0000","last_became_active":"2024-04-25T09:17:29.937624+0000","last_became_peered":"2024-04-25T09:17:29.937624+0000","last_unstale":"2024-04-25T09:17:30.982534+0000","last_undegraded":"2024-04-25T09:17:30.982534+0000","last_fullsized":"2024-04-25T09:17:30.982534+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:56.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962121+0000","last_change":"2024-04-25T09:17:30.960858+0000","last_active":"2024-04-25T09:17:30.962121+0000","last_peered":"2024-04-25T09:17:30.962121+0000","last_clean":"2024-04-25T09:17:30.962121+0000","last_became_active":"2024-04-25T09:17:30.960361+0000","last_became_peered":"2024-04-25T09:17:30.960361+0000","last_unstale":"2024-04-25T09:17:30.962121+0000","last_undegraded":"2024-04-25T09:17:30.962121+0000","last_fullsized":"2024-04-25T09:17:30.962121+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:15.901463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.941710+0000","last_change":"2024-04-25T09:17:29.932748+0000","last_active":"2024-04-25T09:17:30.941710+0000","last_peered":"2024-04-25T09:17:30.941710+0000","last_clean":"2024-04-25T09:17:30.941710+0000","last_became_active":"2024-04-25T09:17:29.932427+0000","last_became_peered":"2024-04-25T09:17:29.932427+0000","last_unstale":"2024-04-25T09:17:30.941710+0000","last_undegraded":"2024-04-25T09:17:30.941710+0000","last_fullsized":"2024-04-25T09:17:30.941710+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.970387+0000","last_change":"2024-04-25T09:17:30.955247+0000","last_active":"2024-04-25T09:17:30.970387+0000","last_peered":"2024-04-25T09:17:30.970387+0000","last_clean":"2024-04-25T09:17:30.970387+0000","last_became_active":"2024-04-25T09:17:30.954926+0000","last_became_peered":"2024-04-25T09:17:30.954926+0000","last_unstale":"2024-04-25T09:17:30.970387+0000","last_undegraded":"2024-04-25T09:17:30.970387+0000","last_fullsized":"2024-04-25T09:17:30.970387+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:28.258503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.942757+0000","last_change":"2024-04-25T09:17:29.937007+0000","last_active":"2024-04-25T09:17:30.942757+0000","last_peered":"2024-04-25T09:17:30.942757+0000","last_clean":"2024-04-25T09:17:30.942757+0000","last_became_active":"2024-04-25T09:17:29.936393+0000","last_became_peered":"2024-04-25T09:17:29.936393+0000","last_unstale":"2024-04-25T09:17:30.942757+0000","last_undegraded":"2024-04-25T09:17:30.942757+0000","last_fullsized":"2024-04-25T09:17:30.942757+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:22.783606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969987+0000","last_change":"2024-04-25T09:17:30.960914+0000","last_active":"2024-04-25T09:17:30.969987+0000","last_peered":"2024-04-25T09:17:30.969987+0000","last_clean":"2024-04-25T09:17:30.969987+0000","last_became_active":"2024-04-25T09:17:30.960349+0000","last_became_peered":"2024-04-25T09:17:30.960349+0000","last_unstale":"2024-04-25T09:17:30.969987+0000","last_undegraded":"2024-04-25T09:17:30.969987+0000","last_fullsized":"2024-04-25T09:17:30.969987+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:09.121647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.953711+0000","last_change":"2024-04-25T09:17:29.925598+0000","last_active":"2024-04-25T09:17:30.953711+0000","last_peered":"2024-04-25T09:17:30.953711+0000","last_clean":"2024-04-25T09:17:30.953711+0000","last_became_active":"2024-04-25T09:17:29.925228+0000","last_became_peered":"2024-04-25T09:17:29.925228+0000","last_unstale":"2024-04-25T09:17:30.953711+0000","last_undegraded":"2024-04-25T09:17:30.953711+0000","last_fullsized":"2024-04-25T09:17:30.953711+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:33.102782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.976017+0000","last_change":"2024-04-25T09:17:30.973108+0000","last_active":"2024-04-25T09:17:30.976017+0000","last_peered":"2024-04-25T09:17:30.976017+0000","last_clean":"2024-04-25T09:17:30.976017+0000","last_became_active":"2024-04-25T09:17:30.972817+0000","last_became_peered":"2024-04-25T09:17:30.972817+0000","last_unstale":"2024-04-25T09:17:30.976017+0000","last_undegraded":"2024-04-25T09:17:30.976017+0000","last_fullsized":"2024-04-25T09:17:30.976017+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:31.962089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.939557+0000","last_change":"2024-04-25T09:17:29.939322+0000","last_active":"2024-04-25T09:17:30.939557+0000","last_peered":"2024-04-25T09:17:30.939557+0000","last_clean":"2024-04-25T09:17:30.939557+0000","last_became_active":"2024-04-25T09:17:29.939044+0000","last_became_peered":"2024-04-25T09:17:29.939044+0000","last_unstale":"2024-04-25T09:17:30.939557+0000","last_undegraded":"2024-04-25T09:17:30.939557+0000","last_fullsized":"2024-04-25T09:17:30.939557+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.578252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962027+0000","last_change":"2024-04-25T09:17:30.960550+0000","last_active":"2024-04-25T09:17:30.962027+0000","last_peered":"2024-04-25T09:17:30.962027+0000","last_clean":"2024-04-25T09:17:30.962027+0000","last_became_active":"2024-04-25T09:17:30.960195+0000","last_became_peered":"2024-04-25T09:17:30.960195+0000","last_unstale":"2024-04-25T09:17:30.962027+0000","last_undegraded":"2024-04-25T09:17:30.962027+0000","last_fullsized":"2024-04-25T09:17:30.962027+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:24.193920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"147'6","reported_seq":18,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.986106+0000","last_change":"2024-04-25T09:17:29.936816+0000","last_active":"2024-04-25T09:17:30.986106+0000","last_peered":"2024-04-25T09:17:30.986106+0000","last_clean":"2024-04-25T09:17:30.986106+0000","last_became_active":"2024-04-25T09:17:29.936231+0000","last_became_peered":"2024-04-25T09:17:29.936231+0000","last_unstale":"2024-04-25T09:17:30.986106+0000","last_undegraded":"2024-04-25T09:17:30.986106+0000","last_fullsized":"2024-04-25T09:17:30.986106+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:27.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.971344+0000","last_change":"2024-04-25T09:17:30.967232+0000","last_active":"2024-04-25T09:17:30.971344+0000","last_peered":"2024-04-25T09:17:30.971344+0000","last_clean":"2024-04-25T09:17:30.971344+0000","last_became_active":"2024-04-25T09:17:30.966959+0000","last_became_peered":"2024-04-25T09:17:30.966959+0000","last_unstale":"2024-04-25T09:17:30.971344+0000","last_undegraded":"2024-04-25T09:17:30.971344+0000","last_fullsized":"2024-04-25T09:17:30.971344+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:51.848968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.982458+0000","last_change":"2024-04-25T09:17:29.937845+0000","last_active":"2024-04-25T09:17:30.982458+0000","last_peered":"2024-04-25T09:17:30.982458+0000","last_clean":"2024-04-25T09:17:30.982458+0000","last_became_active":"2024-04-25T09:17:29.936050+0000","last_became_peered":"2024-04-25T09:17:29.936050+0000","last_unstale":"2024-04-25T09:17:30.982458+0000","last_undegraded":"2024-04-25T09:17:30.982458+0000","last_fullsized":"2024-04-25T09:17:30.982458+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:29.142608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965061+0000","last_change":"2024-04-25T09:17:30.963842+0000","last_active":"2024-04-25T09:17:30.965061+0000","last_peered":"2024-04-25T09:17:30.965061+0000","last_clean":"2024-04-25T09:17:30.965061+0000","last_became_active":"2024-04-25T09:17:30.962718+0000","last_became_peered":"2024-04-25T09:17:30.962718+0000","last_unstale":"2024-04-25T09:17:30.965061+0000","last_undegraded":"2024-04-25T09:17:30.965061+0000","last_fullsized":"2024-04-25T09:17:30.965061+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:02.549839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.956740+0000","last_change":"2024-04-25T09:17:29.932661+0000","last_active":"2024-04-25T09:17:30.956740+0000","last_peered":"2024-04-25T09:17:30.956740+0000","last_clean":"2024-04-25T09:17:30.956740+0000","last_became_active":"2024-04-25T09:17:29.932352+0000","last_became_peered":"2024-04-25T09:17:29.932352+0000","last_unstale":"2024-04-25T09:17:30.956740+0000","last_undegraded":"2024-04-25T09:17:30.956740+0000","last_fullsized":"2024-04-25T09:17:30.956740+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:22.330195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.971372+0000","last_change":"2024-04-25T09:17:30.965883+0000","last_active":"2024-04-25T09:17:30.971372+0000","last_peered":"2024-04-25T09:17:30.971372+0000","last_clean":"2024-04-25T09:17:30.971372+0000","last_became_active":"2024-04-25T09:17:30.963090+0000","last_became_peered":"2024-04-25T09:17:30.963090+0000","last_unstale":"2024-04-25T09:17:30.971372+0000","last_undegraded":"2024-04-25T09:17:30.971372+0000","last_fullsized":"2024-04-25T09:17:30.971372+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:01.343706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.930764+0000","last_change":"2024-04-25T09:17:29.924782+0000","last_active":"2024-04-25T09:17:30.930764+0000","last_peered":"2024-04-25T09:17:30.930764+0000","last_clean":"2024-04-25T09:17:30.930764+0000","last_became_active":"2024-04-25T09:17:29.923589+0000","last_became_peered":"2024-04-25T09:17:29.923589+0000","last_unstale":"2024-04-25T09:17:30.930764+0000","last_undegraded":"2024-04-25T09:17:30.930764+0000","last_fullsized":"2024-04-25T09:17:30.930764+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.496120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.971349+0000","last_change":"2024-04-25T09:17:30.965409+0000","last_active":"2024-04-25T09:17:30.971349+0000","last_peered":"2024-04-25T09:17:30.971349+0000","last_clean":"2024-04-25T09:17:30.971349+0000","last_became_active":"2024-04-25T09:17:30.965130+0000","last_became_peered":"2024-04-25T09:17:30.965130+0000","last_unstale":"2024-04-25T09:17:30.971349+0000","last_undegraded":"2024-04-25T09:17:30.971349+0000","last_fullsized":"2024-04-25T09:17:30.971349+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:05.413500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"147'8","reported_seq":20,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.985990+0000","last_change":"2024-04-25T09:17:29.913951+0000","last_active":"2024-04-25T09:17:30.985990+0000","last_peered":"2024-04-25T09:17:30.985990+0000","last_clean":"2024-04-25T09:17:30.985990+0000","last_became_active":"2024-04-25T09:17:29.912977+0000","last_became_peered":"2024-04-25T09:17:29.912977+0000","last_unstale":"2024-04-25T09:17:30.985990+0000","last_undegraded":"2024-04-25T09:17:30.985990+0000","last_fullsized":"2024-04-25T09:17:30.985990+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:30.947449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965920+0000","last_change":"2024-04-25T09:17:30.964198+0000","last_active":"2024-04-25T09:17:30.965920+0000","last_peered":"2024-04-25T09:17:30.965920+0000","last_clean":"2024-04-25T09:17:30.965920+0000","last_became_active":"2024-04-25T09:17:30.963805+0000","last_became_peered":"2024-04-25T09:17:30.963805+0000","last_unstale":"2024-04-25T09:17:30.965920+0000","last_undegraded":"2024-04-25T09:17:30.965920+0000","last_fullsized":"2024-04-25T09:17:30.965920+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:59.351904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.985738+0000","last_change":"2024-04-25T09:17:29.926505+0000","last_active":"2024-04-25T09:17:30.985738+0000","last_peered":"2024-04-25T09:17:30.985738+0000","last_clean":"2024-04-25T09:17:30.985738+0000","last_became_active":"2024-04-25T09:17:29.926150+0000","last_became_peered":"2024-04-25T09:17:29.926150+0000","last_unstale":"2024-04-25T09:17:30.985738+0000","last_undegraded":"2024-04-25T09:17:30.985738+0000","last_fullsized":"2024-04-25T09:17:30.985738+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:46.700951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.952608+0000","last_change":"2024-04-25T09:17:30.949444+0000","last_active":"2024-04-25T09:17:30.952608+0000","last_peered":"2024-04-25T09:17:30.952608+0000","last_clean":"2024-04-25T09:17:30.952608+0000","last_became_active":"2024-04-25T09:17:30.949110+0000","last_became_peered":"2024-04-25T09:17:30.949110+0000","last_unstale":"2024-04-25T09:17:30.952608+0000","last_undegraded":"2024-04-25T09:17:30.952608+0000","last_fullsized":"2024-04-25T09:17:30.952608+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:56.594821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.984755+0000","last_change":"2024-04-25T09:17:29.928052+0000","last_active":"2024-04-25T09:17:30.984755+0000","last_peered":"2024-04-25T09:17:30.984755+0000","last_clean":"2024-04-25T09:17:30.984755+0000","last_became_active":"2024-04-25T09:17:29.927740+0000","last_became_peered":"2024-04-25T09:17:29.927740+0000","last_unstale":"2024-04-25T09:17:30.984755+0000","last_undegraded":"2024-04-25T09:17:30.984755+0000","last_fullsized":"2024-04-25T09:17:30.984755+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967832+0000","last_change":"2024-04-25T09:17:30.965759+0000","last_active":"2024-04-25T09:17:30.967832+0000","last_peered":"2024-04-25T09:17:30.967832+0000","last_clean":"2024-04-25T09:17:30.967832+0000","last_became_active":"2024-04-25T09:17:30.964183+0000","last_became_peered":"2024-04-25T09:17:30.964183+0000","last_unstale":"2024-04-25T09:17:30.967832+0000","last_undegraded":"2024-04-25T09:17:30.967832+0000","last_fullsized":"2024-04-25T09:17:30.967832+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:22.534044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.981298+0000","last_change":"2024-04-25T09:17:29.938307+0000","last_active":"2024-04-25T09:17:30.981298+0000","last_peered":"2024-04-25T09:17:30.981298+0000","last_clean":"2024-04-25T09:17:30.981298+0000","last_became_active":"2024-04-25T09:17:29.937850+0000","last_became_peered":"2024-04-25T09:17:29.937850+0000","last_unstale":"2024-04-25T09:17:30.981298+0000","last_undegraded":"2024-04-25T09:17:30.981298+0000","last_fullsized":"2024-04-25T09:17:30.981298+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:10.096348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.961799+0000","last_change":"2024-04-25T09:17:30.961062+0000","last_active":"2024-04-25T09:17:30.961799+0000","last_peered":"2024-04-25T09:17:30.961799+0000","last_clean":"2024-04-25T09:17:30.961799+0000","last_became_active":"2024-04-25T09:17:30.960736+0000","last_became_peered":"2024-04-25T09:17:30.960736+0000","last_unstale":"2024-04-25T09:17:30.961799+0000","last_undegraded":"2024-04-25T09:17:30.961799+0000","last_fullsized":"2024-04-25T09:17:30.961799+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:02.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.933060+0000","last_change":"2024-04-25T09:17:29.934905+0000","last_active":"2024-04-25T09:17:30.933060+0000","last_peered":"2024-04-25T09:17:30.933060+0000","last_clean":"2024-04-25T09:17:30.933060+0000","last_became_active":"2024-04-25T09:17:29.934607+0000","last_became_peered":"2024-04-25T09:17:29.934607+0000","last_unstale":"2024-04-25T09:17:30.933060+0000","last_undegraded":"2024-04-25T09:17:30.933060+0000","last_fullsized":"2024-04-25T09:17:30.933060+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:29.965126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967871+0000","last_change":"2024-04-25T09:17:30.964325+0000","last_active":"2024-04-25T09:17:30.967871+0000","last_peered":"2024-04-25T09:17:30.967871+0000","last_clean":"2024-04-25T09:17:30.967871+0000","last_became_active":"2024-04-25T09:17:30.963690+0000","last_became_peered":"2024-04-25T09:17:30.963690+0000","last_unstale":"2024-04-25T09:17:30.967871+0000","last_undegraded":"2024-04-25T09:17:30.967871+0000","last_fullsized":"2024-04-25T09:17:30.967871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:51.848968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.983124+0000","last_change":"2024-04-25T09:17:29.934810+0000","last_active":"2024-04-25T09:17:30.983124+0000","last_peered":"2024-04-25T09:17:30.983124+0000","last_clean":"2024-04-25T09:17:30.983124+0000","last_became_active":"2024-04-25T09:17:29.934249+0000","last_became_peered":"2024-04-25T09:17:29.934249+0000","last_unstale":"2024-04-25T09:17:30.983124+0000","last_undegraded":"2024-04-25T09:17:30.983124+0000","last_fullsized":"2024-04-25T09:17:30.983124+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:02.634503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.955430+0000","last_change":"2024-04-25T09:17:30.951601+0000","last_active":"2024-04-25T09:17:30.955430+0000","last_peered":"2024-04-25T09:17:30.955430+0000","last_clean":"2024-04-25T09:17:30.955430+0000","last_became_active":"2024-04-25T09:17:30.951322+0000","last_became_peered":"2024-04-25T09:17:30.951322+0000","last_unstale":"2024-04-25T09:17:30.955430+0000","last_undegraded":"2024-04-25T09:17:30.955430+0000","last_fullsized":"2024-04-25T09:17:30.955430+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:02.486970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.944647+0000","last_change":"2024-04-25T09:17:29.937682+0000","last_active":"2024-04-25T09:17:30.944647+0000","last_peered":"2024-04-25T09:17:30.944647+0000","last_clean":"2024-04-25T09:17:30.944647+0000","last_became_active":"2024-04-25T09:17:29.937063+0000","last_became_peered":"2024-04-25T09:17:29.937063+0000","last_unstale":"2024-04-25T09:17:30.944647+0000","last_undegraded":"2024-04-25T09:17:30.944647+0000","last_fullsized":"2024-04-25T09:17:30.944647+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:14.886823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.963065+0000","last_change":"2024-04-25T09:17:30.956562+0000","last_active":"2024-04-25T09:17:30.963065+0000","last_peered":"2024-04-25T09:17:30.963065+0000","last_clean":"2024-04-25T09:17:30.963065+0000","last_became_active":"2024-04-25T09:17:30.955847+0000","last_became_peered":"2024-04-25T09:17:30.955847+0000","last_unstale":"2024-04-25T09:17:30.963065+0000","last_undegraded":"2024-04-25T09:17:30.963065+0000","last_fullsized":"2024-04-25T09:17:30.963065+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:05.358730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.946951+0000","last_change":"2024-04-25T09:17:29.913928+0000","last_active":"2024-04-25T09:17:30.946951+0000","last_peered":"2024-04-25T09:17:30.946951+0000","last_clean":"2024-04-25T09:17:30.946951+0000","last_became_active":"2024-04-25T09:17:29.912987+0000","last_became_peered":"2024-04-25T09:17:29.912987+0000","last_unstale":"2024-04-25T09:17:30.946951+0000","last_undegraded":"2024-04-25T09:17:30.946951+0000","last_fullsized":"2024-04-25T09:17:30.946951+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:42.198793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.955138+0000","last_change":"2024-04-25T09:17:30.951325+0000","last_active":"2024-04-25T09:17:30.955138+0000","last_peered":"2024-04-25T09:17:30.955138+0000","last_clean":"2024-04-25T09:17:30.955138+0000","last_became_active":"2024-04-25T09:17:30.951034+0000","last_became_peered":"2024-04-25T09:17:30.951034+0000","last_unstale":"2024-04-25T09:17:30.955138+0000","last_undegraded":"2024-04-25T09:17:30.955138+0000","last_fullsized":"2024-04-25T09:17:30.955138+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:06.099265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.949127+0000","last_change":"2024-04-25T09:17:29.935388+0000","last_active":"2024-04-25T09:17:30.949127+0000","last_peered":"2024-04-25T09:17:30.949127+0000","last_clean":"2024-04-25T09:17:30.949127+0000","last_became_active":"2024-04-25T09:17:29.935028+0000","last_became_peered":"2024-04-25T09:17:29.935028+0000","last_unstale":"2024-04-25T09:17:30.949127+0000","last_undegraded":"2024-04-25T09:17:30.949127+0000","last_fullsized":"2024-04-25T09:17:30.949127+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:32.401465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969965+0000","last_change":"2024-04-25T09:17:30.960665+0000","last_active":"2024-04-25T09:17:30.969965+0000","last_peered":"2024-04-25T09:17:30.969965+0000","last_clean":"2024-04-25T09:17:30.969965+0000","last_became_active":"2024-04-25T09:17:30.959789+0000","last_became_peered":"2024-04-25T09:17:30.959789+0000","last_unstale":"2024-04-25T09:17:30.969965+0000","last_undegraded":"2024-04-25T09:17:30.969965+0000","last_fullsized":"2024-04-25T09:17:30.969965+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:11.110989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.984845+0000","last_change":"2024-04-25T09:17:29.942258+0000","last_active":"2024-04-25T09:17:30.984845+0000","last_peered":"2024-04-25T09:17:30.984845+0000","last_clean":"2024-04-25T09:17:30.984845+0000","last_became_active":"2024-04-25T09:17:29.941428+0000","last_became_peered":"2024-04-25T09:17:29.941428+0000","last_unstale":"2024-04-25T09:17:30.984845+0000","last_undegraded":"2024-04-25T09:17:30.984845+0000","last_fullsized":"2024-04-25T09:17:30.984845+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:55.906208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969758+0000","last_change":"2024-04-25T09:17:30.962700+0000","last_active":"2024-04-25T09:17:30.969758+0000","last_peered":"2024-04-25T09:17:30.969758+0000","last_clean":"2024-04-25T09:17:30.969758+0000","last_became_active":"2024-04-25T09:17:30.962354+0000","last_became_peered":"2024-04-25T09:17:30.962354+0000","last_unstale":"2024-04-25T09:17:30.969758+0000","last_undegraded":"2024-04-25T09:17:30.969758+0000","last_fullsized":"2024-04-25T09:17:30.969758+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:23.344835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.932700+0000","last_change":"2024-04-25T09:17:29.937413+0000","last_active":"2024-04-25T09:17:30.932700+0000","last_peered":"2024-04-25T09:17:30.932700+0000","last_clean":"2024-04-25T09:17:30.932700+0000","last_became_active":"2024-04-25T09:17:29.937100+0000","last_became_peered":"2024-04-25T09:17:29.937100+0000","last_unstale":"2024-04-25T09:17:30.932700+0000","last_undegraded":"2024-04-25T09:17:30.932700+0000","last_fullsized":"2024-04-25T09:17:30.932700+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:00.567475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.950759+0000","last_change":"2024-04-25T09:17:30.946494+0000","last_active":"2024-04-25T09:17:30.950759+0000","last_peered":"2024-04-25T09:17:30.950759+0000","last_clean":"2024-04-25T09:17:30.950759+0000","last_became_active":"2024-04-25T09:17:30.946066+0000","last_became_peered":"2024-04-25T09:17:30.946066+0000","last_unstale":"2024-04-25T09:17:30.950759+0000","last_undegraded":"2024-04-25T09:17:30.950759+0000","last_fullsized":"2024-04-25T09:17:30.950759+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.649144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.952506+0000","last_change":"2024-04-25T09:17:29.935980+0000","last_active":"2024-04-25T09:17:30.952506+0000","last_peered":"2024-04-25T09:17:30.952506+0000","last_clean":"2024-04-25T09:17:30.952506+0000","last_became_active":"2024-04-25T09:17:29.935364+0000","last_became_peered":"2024-04-25T09:17:29.935364+0000","last_unstale":"2024-04-25T09:17:30.952506+0000","last_undegraded":"2024-04-25T09:17:30.952506+0000","last_fullsized":"2024-04-25T09:17:30.952506+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:08.107007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.961125+0000","last_change":"2024-04-25T09:17:30.957145+0000","last_active":"2024-04-25T09:17:30.961125+0000","last_peered":"2024-04-25T09:17:30.961125+0000","last_clean":"2024-04-25T09:17:30.961125+0000","last_became_active":"2024-04-25T09:17:30.956808+0000","last_became_peered":"2024-04-25T09:17:30.956808+0000","last_unstale":"2024-04-25T09:17:30.961125+0000","last_undegraded":"2024-04-25T09:17:30.961125+0000","last_fullsized":"2024-04-25T09:17:30.961125+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:34.117423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.945274+0000","last_change":"2024-04-25T09:17:29.936674+0000","last_active":"2024-04-25T09:17:30.945274+0000","last_peered":"2024-04-25T09:17:30.945274+0000","last_clean":"2024-04-25T09:17:30.945274+0000","last_became_active":"2024-04-25T09:17:29.935332+0000","last_became_peered":"2024-04-25T09:17:29.935332+0000","last_unstale":"2024-04-25T09:17:30.945274+0000","last_undegraded":"2024-04-25T09:17:30.945274+0000","last_fullsized":"2024-04-25T09:17:30.945274+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.647480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.964794+0000","last_change":"2024-04-25T09:17:30.963816+0000","last_active":"2024-04-25T09:17:30.964794+0000","last_peered":"2024-04-25T09:17:30.964794+0000","last_clean":"2024-04-25T09:17:30.964794+0000","last_became_active":"2024-04-25T09:17:30.963483+0000","last_became_peered":"2024-04-25T09:17:30.963483+0000","last_unstale":"2024-04-25T09:17:30.964794+0000","last_undegraded":"2024-04-25T09:17:30.964794+0000","last_fullsized":"2024-04-25T09:17:30.964794+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:05.413500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.932261+0000","last_change":"2024-04-25T09:17:29.926703+0000","last_active":"2024-04-25T09:17:30.932261+0000","last_peered":"2024-04-25T09:17:30.932261+0000","last_clean":"2024-04-25T09:17:30.932261+0000","last_became_active":"2024-04-25T09:17:29.926267+0000","last_became_peered":"2024-04-25T09:17:29.926267+0000","last_unstale":"2024-04-25T09:17:30.932261+0000","last_undegraded":"2024-04-25T09:17:30.932261+0000","last_fullsized":"2024-04-25T09:17:30.932261+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:28.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969823+0000","last_change":"2024-04-25T09:17:30.961559+0000","last_active":"2024-04-25T09:17:30.969823+0000","last_peered":"2024-04-25T09:17:30.969823+0000","last_clean":"2024-04-25T09:17:30.969823+0000","last_became_active":"2024-04-25T09:17:30.961205+0000","last_became_peered":"2024-04-25T09:17:30.961205+0000","last_unstale":"2024-04-25T09:17:30.969823+0000","last_undegraded":"2024-04-25T09:17:30.969823+0000","last_fullsized":"2024-04-25T09:17:30.969823+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:05.358730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.945090+0000","last_change":"2024-04-25T09:17:29.941369+0000","last_active":"2024-04-25T09:17:30.945090+0000","last_peered":"2024-04-25T09:17:30.945090+0000","last_clean":"2024-04-25T09:17:30.945090+0000","last_became_active":"2024-04-25T09:17:29.941030+0000","last_became_peered":"2024-04-25T09:17:29.941030+0000","last_unstale":"2024-04-25T09:17:30.945090+0000","last_undegraded":"2024-04-25T09:17:30.945090+0000","last_fullsized":"2024-04-25T09:17:30.945090+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:29.081412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967563+0000","last_change":"2024-04-25T09:17:30.955382+0000","last_active":"2024-04-25T09:17:30.967563+0000","last_peered":"2024-04-25T09:17:30.967563+0000","last_clean":"2024-04-25T09:17:30.967563+0000","last_became_active":"2024-04-25T09:17:30.954255+0000","last_became_peered":"2024-04-25T09:17:30.954255+0000","last_unstale":"2024-04-25T09:17:30.967563+0000","last_undegraded":"2024-04-25T09:17:30.967563+0000","last_fullsized":"2024-04-25T09:17:30.967563+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:39.662120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.952774+0000","last_change":"2024-04-25T09:17:29.934014+0000","last_active":"2024-04-25T09:17:30.952774+0000","last_peered":"2024-04-25T09:17:30.952774+0000","last_clean":"2024-04-25T09:17:30.952774+0000","last_became_active":"2024-04-25T09:17:29.933696+0000","last_became_peered":"2024-04-25T09:17:29.933696+0000","last_unstale":"2024-04-25T09:17:30.952774+0000","last_undegraded":"2024-04-25T09:17:30.952774+0000","last_fullsized":"2024-04-25T09:17:30.952774+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:08.107007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.956693+0000","last_change":"2024-04-25T09:17:30.953294+0000","last_active":"2024-04-25T09:17:30.956693+0000","last_peered":"2024-04-25T09:17:30.956693+0000","last_clean":"2024-04-25T09:17:30.956693+0000","last_became_active":"2024-04-25T09:17:30.953004+0000","last_became_peered":"2024-04-25T09:17:30.953004+0000","last_unstale":"2024-04-25T09:17:30.956693+0000","last_undegraded":"2024-04-25T09:17:30.956693+0000","last_fullsized":"2024-04-25T09:17:30.956693+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:25.216126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.943736+0000","last_change":"2024-04-25T09:17:29.933350+0000","last_active":"2024-04-25T09:17:30.943736+0000","last_peered":"2024-04-25T09:17:30.943736+0000","last_clean":"2024-04-25T09:17:30.943736+0000","last_became_active":"2024-04-25T09:17:29.933009+0000","last_became_peered":"2024-04-25T09:17:29.933009+0000","last_unstale":"2024-04-25T09:17:30.943736+0000","last_undegraded":"2024-04-25T09:17:30.943736+0000","last_fullsized":"2024-04-25T09:17:30.943736+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:32.034837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.955695+0000","last_change":"2024-04-25T09:17:30.949383+0000","last_active":"2024-04-25T09:17:30.955695+0000","last_peered":"2024-04-25T09:17:30.955695+0000","last_clean":"2024-04-25T09:17:30.955695+0000","last_became_active":"2024-04-25T09:17:30.949031+0000","last_became_peered":"2024-04-25T09:17:30.949031+0000","last_unstale":"2024-04-25T09:17:30.955695+0000","last_undegraded":"2024-04-25T09:17:30.955695+0000","last_fullsized":"2024-04-25T09:17:30.955695+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:12.717063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"147'3","reported_seq":15,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.984503+0000","last_change":"2024-04-25T09:17:29.937254+0000","last_active":"2024-04-25T09:17:30.984503+0000","last_peered":"2024-04-25T09:17:30.984503+0000","last_clean":"2024-04-25T09:17:30.984503+0000","last_became_active":"2024-04-25T09:17:29.936976+0000","last_became_peered":"2024-04-25T09:17:29.936976+0000","last_unstale":"2024-04-25T09:17:30.984503+0000","last_undegraded":"2024-04-25T09:17:30.984503+0000","last_fullsized":"2024-04-25T09:17:30.984503+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:23.179279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":404,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.936091+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:17:30.936091+0000","last_peered":"2024-04-25T09:17:30.936091+0000","last_clean":"2024-04-25T09:17:30.936091+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:17:30.936091+0000","last_undegraded":"2024-04-25T09:17:30.936091+0000","last_fullsized":"2024-04-25T09:17:30.936091+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55188,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316652,"statfs":{"total":96636764160,"available":96580251648,"internally_reserved":0,"allocated":1839104,"data_stored":809301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575025,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59916,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311924,"statfs":{"total":96636764160,"available":96575410176,"internally_reserved":0,"allocated":2420736,"data_stored":1392894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575025,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46856,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324984,"statfs":{"total":96636764160,"available":96588783616,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575026,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59336,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312504,"statfs":{"total":96636764160,"available":96576004096,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575026,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64904,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306936,"statfs":{"total":96636764160,"available":96570302464,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575025,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59916,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311924,"statfs":{"total":96636764160,"available":96575410176,"internally_reserved":0,"allocated":2420736,"data_stored":1392894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575024,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:17:35.428 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:17:35.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.658+0000 7f6a63200640 1 Processor -- start 2024-04-25T09:17:35.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.658+0000 7f6a63200640 1 -- start start 2024-04-25T09:17:35.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.659+0000 7f6a63200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 0x7f6a64160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.659+0000 7f6a63200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6a640b6a90 0x7f6a64161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.659+0000 7f6a63200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6a640b27a0 0x7f6a64161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.659+0000 7f6a63200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a64161780 con 0x7f6a640b27a0 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.659+0000 7f6a63200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a640cedc0 con 0x7f6a640b6a90 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.659+0000 7f6a63200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640d3110 con 0x7f6a640b56f0 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.660+0000 7f6a62800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6a640b27a0 0x7f6a64161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.660+0000 7f6a62800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6a640b27a0 0x7f6a64161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38282/0 (socket says 172.21.15.62:38282) 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.660+0000 7f6a62800640 1 -- 172.21.15.62:0/2731849828 learned_addr learned my addr 172.21.15.62:0/2731849828 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.660+0000 7f6a61e00640 1 --2- 172.21.15.62:0/2731849828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 0x7f6a64160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.660+0000 7f6a61400640 1 --2- 172.21.15.62:0/2731849828 >> v2:172.21.15.121:3300/0 conn(0x7f6a640b6a90 0x7f6a64161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.661+0000 7f6a62800640 1 -- 172.21.15.62:0/2731849828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 msgr2=0x7f6a64160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.661+0000 7f6a62800640 1 --2- 172.21.15.62:0/2731849828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 0x7f6a64160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.661+0000 7f6a62800640 1 -- 172.21.15.62:0/2731849828 >> v2:172.21.15.121:3300/0 conn(0x7f6a640b6a90 msgr2=0x7f6a64161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.661+0000 7f6a62800640 1 --2- 172.21.15.62:0/2731849828 >> v2:172.21.15.121:3300/0 conn(0x7f6a640b6a90 0x7f6a64161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.661+0000 7f6a62800640 1 -- 172.21.15.62:0/2731849828 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a640d33f0 con 0x7f6a640b27a0 2024-04-25T09:17:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.662+0000 7f6a62800640 1 --2- 172.21.15.62:0/2731849828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b27a0 0x7f6a64161980 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f6a640728e0 tx=0x7f6a5c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83c426257670644 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:35.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.662+0000 7f6a60a00640 1 -- 172.21.15.62:0/2731849828 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c036070 con 0x7f6a640b27a0 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.662+0000 7f6a60a00640 1 -- 172.21.15.62:0/2731849828 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c031600 con 0x7f6a640b27a0 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.662+0000 7f6a60a00640 1 -- 172.21.15.62:0/2731849828 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0319d0 con 0x7f6a640b27a0 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 -- 172.21.15.62:0/2731849828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b27a0 msgr2=0x7f6a64161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 --2- 172.21.15.62:0/2731849828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b27a0 0x7f6a64161980 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f6a640728e0 tx=0x7f6a5c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 -- 172.21.15.62:0/2731849828 shutdown_connections 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 --2- 172.21.15.62:0/2731849828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b27a0 0x7f6a64161980 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 --2- 172.21.15.62:0/2731849828 >> v2:172.21.15.121:3300/0 conn(0x7f6a640b6a90 0x7f6a64161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 --2- 172.21.15.62:0/2731849828 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 0x7f6a64160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 -- 172.21.15.62:0/2731849828 >> 172.21.15.62:0/2731849828 conn(0x7f6a64066d10 msgr2=0x7f6a64067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 -- 172.21.15.62:0/2731849828 shutdown_connections 2024-04-25T09:17:35.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.663+0000 7f6a63200640 1 -- 172.21.15.62:0/2731849828 wait complete. 2024-04-25T09:17:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.665+0000 7f6a63200640 1 Processor -- start 2024-04-25T09:17:35.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.665+0000 7f6a63200640 1 -- start start 2024-04-25T09:17:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a63200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a640b27a0 0x7f6a6415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a63200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 0x7f6a6415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a63200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b6a90 0x7f6a6415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a63200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640b2d60 con 0x7f6a640b6a90 2024-04-25T09:17:35.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a63200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640cea50 con 0x7f6a640b27a0 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a63200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640b2ee0 con 0x7f6a640b56f0 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a62800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b6a90 0x7f6a6415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a62800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b6a90 0x7f6a6415cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38298/0 (socket says 172.21.15.62:38298) 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a62800640 1 -- 172.21.15.62:0/4167737848 learned_addr learned my addr 172.21.15.62:0/4167737848 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a61400640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 0x7f6a6415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.666+0000 7f6a61e00640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a640b27a0 0x7f6a6415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a62800640 1 -- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 msgr2=0x7f6a6415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a62800640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 0x7f6a6415adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a62800640 1 -- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a640b27a0 msgr2=0x7f6a6415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a62800640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a640b27a0 0x7f6a6415a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a62800640 1 -- 172.21.15.62:0/4167737848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64157c30 con 0x7f6a640b6a90 2024-04-25T09:17:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a62800640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b6a90 0x7f6a6415cdd0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009f40 tx=0x7f6a5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a5a000640 1 -- 172.21.15.62:0/4167737848 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c036050 con 0x7f6a640b6a90 2024-04-25T09:17:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a5a000640 1 -- 172.21.15.62:0/4167737848 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c042070 con 0x7f6a640b6a90 2024-04-25T09:17:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a64157f10 con 0x7f6a640b6a90 2024-04-25T09:17:35.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a5a000640 1 -- 172.21.15.62:0/4167737848 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c009510 con 0x7f6a640b6a90 2024-04-25T09:17:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.667+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a641584d0 con 0x7f6a640b6a90 2024-04-25T09:17:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.669+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6404ef20 con 0x7f6a640b6a90 2024-04-25T09:17:35.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.670+0000 7f6a5a000640 1 -- 172.21.15.62:0/4167737848 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a5c007bb0 con 0x7f6a640b6a90 2024-04-25T09:17:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.672+0000 7f6a5a000640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6a4406b4d0 0x7f6a4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:35.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.672+0000 7f6a5a000640 1 -- 172.21.15.62:0/4167737848 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a5c0ab190 con 0x7f6a640b6a90 2024-04-25T09:17:35.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.672+0000 7f6a61e00640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6a4406b4d0 0x7f6a4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:35.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.673+0000 7f6a61e00640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6a4406b4d0 0x7f6a4406d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6a54006fd0 tx=0x7f6a54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:35.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.677+0000 7f6a5a000640 1 -- 172.21.15.62:0/4167737848 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a5c045360 con 0x7f6a640b6a90 2024-04-25T09:17:35.853 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-04-25T09:17:35.866+0000 7f2baf400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:17:35.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.873+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a640b9270 con 0x7f6a4406b4d0 2024-04-25T09:17:35.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.891+0000 7f6a5a000640 1 -- 172.21.15.62:0/4167737848 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252409 (secure 0 0 0) 0x7f6a640b9270 con 0x7f6a4406b4d0 2024-04-25T09:17:35.882 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:35.883 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.897+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6a4406b4d0 msgr2=0x7f6a4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.897+0000 7f6a63200640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6a4406b4d0 0x7f6a4406d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6a54006fd0 tx=0x7f6a54008040 comp rx=0 tx=0).stop 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.897+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b6a90 msgr2=0x7f6a6415cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.897+0000 7f6a63200640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b6a90 0x7f6a6415cdd0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009f40 tx=0x7f6a5c004a40 comp rx=0 tx=0).stop 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.898+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 shutdown_connections 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.898+0000 7f6a63200640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6a4406b4d0 0x7f6a4406d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.898+0000 7f6a63200640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6a640b6a90 0x7f6a6415cdd0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.898+0000 7f6a63200640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6a640b56f0 0x7f6a6415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.898+0000 7f6a63200640 1 --2- 172.21.15.62:0/4167737848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a640b27a0 0x7f6a6415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:35.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.898+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 >> 172.21.15.62:0/4167737848 conn(0x7f6a64066d10 msgr2=0x7f6a640b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:35.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.898+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 shutdown_connections 2024-04-25T09:17:35.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:35.898+0000 7f6a63200640 1 -- 172.21.15.62:0/4167737848 wait complete. 2024-04-25T09:17:35.905 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":575,"stamp":"2024-04-25T09:17:34.215612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":456468,"kb_used_data":15444,"kb_used_omap":12,"kb_used_meta":440883,"kb_avail":754518252,"statfs":{"total":773094113280,"available":772626690048,"internally_reserved":0,"allocated":15814656,"data_stored":7607719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":451464752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319567"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962063+0000","last_change":"2024-04-25T09:17:30.959007+0000","last_active":"2024-04-25T09:17:30.962063+0000","last_peered":"2024-04-25T09:17:30.962063+0000","last_clean":"2024-04-25T09:17:30.962063+0000","last_became_active":"2024-04-25T09:17:30.957737+0000","last_became_peered":"2024-04-25T09:17:30.957737+0000","last_unstale":"2024-04-25T09:17:30.962063+0000","last_undegraded":"2024-04-25T09:17:30.962063+0000","last_fullsized":"2024-04-25T09:17:30.962063+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:02.618723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.953472+0000","last_change":"2024-04-25T09:17:30.950680+0000","last_active":"2024-04-25T09:17:30.953472+0000","last_peered":"2024-04-25T09:17:30.953472+0000","last_clean":"2024-04-25T09:17:30.953472+0000","last_became_active":"2024-04-25T09:17:30.950238+0000","last_became_peered":"2024-04-25T09:17:30.950238+0000","last_unstale":"2024-04-25T09:17:30.953472+0000","last_undegraded":"2024-04-25T09:17:30.953472+0000","last_fullsized":"2024-04-25T09:17:30.953472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:30.960343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.959083+0000","last_change":"2024-04-25T09:17:30.956658+0000","last_active":"2024-04-25T09:17:30.959083+0000","last_peered":"2024-04-25T09:17:30.959083+0000","last_clean":"2024-04-25T09:17:30.959083+0000","last_became_active":"2024-04-25T09:17:30.956313+0000","last_became_peered":"2024-04-25T09:17:30.956313+0000","last_unstale":"2024-04-25T09:17:30.959083+0000","last_undegraded":"2024-04-25T09:17:30.959083+0000","last_fullsized":"2024-04-25T09:17:30.959083+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:52.068797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.972167+0000","last_change":"2024-04-25T09:17:30.967359+0000","last_active":"2024-04-25T09:17:30.972167+0000","last_peered":"2024-04-25T09:17:30.972167+0000","last_clean":"2024-04-25T09:17:30.972167+0000","last_became_active":"2024-04-25T09:17:30.967087+0000","last_became_peered":"2024-04-25T09:17:30.967087+0000","last_unstale":"2024-04-25T09:17:30.972167+0000","last_undegraded":"2024-04-25T09:17:30.972167+0000","last_fullsized":"2024-04-25T09:17:30.972167+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:31.959406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.974021+0000","last_change":"2024-04-25T09:17:30.971779+0000","last_active":"2024-04-25T09:17:30.974021+0000","last_peered":"2024-04-25T09:17:30.974021+0000","last_clean":"2024-04-25T09:17:30.974021+0000","last_became_active":"2024-04-25T09:17:30.968575+0000","last_became_peered":"2024-04-25T09:17:30.968575+0000","last_unstale":"2024-04-25T09:17:30.974021+0000","last_undegraded":"2024-04-25T09:17:30.974021+0000","last_fullsized":"2024-04-25T09:17:30.974021+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:44.964670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.959851+0000","last_change":"2024-04-25T09:17:30.957311+0000","last_active":"2024-04-25T09:17:30.959851+0000","last_peered":"2024-04-25T09:17:30.959851+0000","last_clean":"2024-04-25T09:17:30.959851+0000","last_became_active":"2024-04-25T09:17:30.955373+0000","last_became_peered":"2024-04-25T09:17:30.955373+0000","last_unstale":"2024-04-25T09:17:30.959851+0000","last_undegraded":"2024-04-25T09:17:30.959851+0000","last_fullsized":"2024-04-25T09:17:30.959851+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:02.618723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.973925+0000","last_change":"2024-04-25T09:17:30.973457+0000","last_active":"2024-04-25T09:17:30.973925+0000","last_peered":"2024-04-25T09:17:30.973925+0000","last_clean":"2024-04-25T09:17:30.973925+0000","last_became_active":"2024-04-25T09:17:30.973204+0000","last_became_peered":"2024-04-25T09:17:30.973204+0000","last_unstale":"2024-04-25T09:17:30.973925+0000","last_undegraded":"2024-04-25T09:17:30.973925+0000","last_fullsized":"2024-04-25T09:17:30.973925+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:01.343706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.964872+0000","last_change":"2024-04-25T09:17:30.963088+0000","last_active":"2024-04-25T09:17:30.964872+0000","last_peered":"2024-04-25T09:17:30.964872+0000","last_clean":"2024-04-25T09:17:30.964872+0000","last_became_active":"2024-04-25T09:17:30.962697+0000","last_became_peered":"2024-04-25T09:17:30.962697+0000","last_unstale":"2024-04-25T09:17:30.964872+0000","last_undegraded":"2024-04-25T09:17:30.964872+0000","last_fullsized":"2024-04-25T09:17:30.964872+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:02.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.957012+0000","last_change":"2024-04-25T09:17:30.952649+0000","last_active":"2024-04-25T09:17:30.957012+0000","last_peered":"2024-04-25T09:17:30.957012+0000","last_clean":"2024-04-25T09:17:30.957012+0000","last_became_active":"2024-04-25T09:17:30.947002+0000","last_became_peered":"2024-04-25T09:17:30.947002+0000","last_unstale":"2024-04-25T09:17:30.957012+0000","last_undegraded":"2024-04-25T09:17:30.957012+0000","last_fullsized":"2024-04-25T09:17:30.957012+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:30.351587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967883+0000","last_change":"2024-04-25T09:17:30.962153+0000","last_active":"2024-04-25T09:17:30.967883+0000","last_peered":"2024-04-25T09:17:30.967883+0000","last_clean":"2024-04-25T09:17:30.967883+0000","last_became_active":"2024-04-25T09:17:30.961798+0000","last_became_peered":"2024-04-25T09:17:30.961798+0000","last_unstale":"2024-04-25T09:17:30.967883+0000","last_undegraded":"2024-04-25T09:17:30.967883+0000","last_fullsized":"2024-04-25T09:17:30.967883+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:10.077072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.964295+0000","last_change":"2024-04-25T09:17:30.963474+0000","last_active":"2024-04-25T09:17:30.964295+0000","last_peered":"2024-04-25T09:17:30.964295+0000","last_clean":"2024-04-25T09:17:30.964295+0000","last_became_active":"2024-04-25T09:17:30.963186+0000","last_became_peered":"2024-04-25T09:17:30.963186+0000","last_unstale":"2024-04-25T09:17:30.964295+0000","last_undegraded":"2024-04-25T09:17:30.964295+0000","last_fullsized":"2024-04-25T09:17:30.964295+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:04.038785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.975429+0000","last_change":"2024-04-25T09:17:30.970200+0000","last_active":"2024-04-25T09:17:30.975429+0000","last_peered":"2024-04-25T09:17:30.975429+0000","last_clean":"2024-04-25T09:17:30.975429+0000","last_became_active":"2024-04-25T09:17:30.969787+0000","last_became_peered":"2024-04-25T09:17:30.969787+0000","last_unstale":"2024-04-25T09:17:30.975429+0000","last_undegraded":"2024-04-25T09:17:30.975429+0000","last_fullsized":"2024-04-25T09:17:30.975429+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:31.959406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.973088+0000","last_change":"2024-04-25T09:17:30.970242+0000","last_active":"2024-04-25T09:17:30.973088+0000","last_peered":"2024-04-25T09:17:30.973088+0000","last_clean":"2024-04-25T09:17:30.973088+0000","last_became_active":"2024-04-25T09:17:30.967961+0000","last_became_peered":"2024-04-25T09:17:30.967961+0000","last_unstale":"2024-04-25T09:17:30.973088+0000","last_undegraded":"2024-04-25T09:17:30.973088+0000","last_fullsized":"2024-04-25T09:17:30.973088+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:42.126244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.976243+0000","last_change":"2024-04-25T09:17:30.973900+0000","last_active":"2024-04-25T09:17:30.976243+0000","last_peered":"2024-04-25T09:17:30.976243+0000","last_clean":"2024-04-25T09:17:30.976243+0000","last_became_active":"2024-04-25T09:17:30.973608+0000","last_became_peered":"2024-04-25T09:17:30.973608+0000","last_unstale":"2024-04-25T09:17:30.976243+0000","last_undegraded":"2024-04-25T09:17:30.976243+0000","last_fullsized":"2024-04-25T09:17:30.976243+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:51.787201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.960554+0000","last_change":"2024-04-25T09:17:30.953808+0000","last_active":"2024-04-25T09:17:30.960554+0000","last_peered":"2024-04-25T09:17:30.960554+0000","last_clean":"2024-04-25T09:17:30.960554+0000","last_became_active":"2024-04-25T09:17:30.953477+0000","last_became_peered":"2024-04-25T09:17:30.953477+0000","last_unstale":"2024-04-25T09:17:30.960554+0000","last_undegraded":"2024-04-25T09:17:30.960554+0000","last_fullsized":"2024-04-25T09:17:30.960554+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:01.230052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.956866+0000","last_change":"2024-04-25T09:17:30.953045+0000","last_active":"2024-04-25T09:17:30.956866+0000","last_peered":"2024-04-25T09:17:30.956866+0000","last_clean":"2024-04-25T09:17:30.956866+0000","last_became_active":"2024-04-25T09:17:30.952284+0000","last_became_peered":"2024-04-25T09:17:30.952284+0000","last_unstale":"2024-04-25T09:17:30.956866+0000","last_undegraded":"2024-04-25T09:17:30.956866+0000","last_fullsized":"2024-04-25T09:17:30.956866+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:05.358730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.970798+0000","last_change":"2024-04-25T09:17:30.967313+0000","last_active":"2024-04-25T09:17:30.970798+0000","last_peered":"2024-04-25T09:17:30.970798+0000","last_clean":"2024-04-25T09:17:30.970798+0000","last_became_active":"2024-04-25T09:17:30.966126+0000","last_became_peered":"2024-04-25T09:17:30.966126+0000","last_unstale":"2024-04-25T09:17:30.970798+0000","last_undegraded":"2024-04-25T09:17:30.970798+0000","last_fullsized":"2024-04-25T09:17:30.970798+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:12.355346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962233+0000","last_change":"2024-04-25T09:17:30.960809+0000","last_active":"2024-04-25T09:17:30.962233+0000","last_peered":"2024-04-25T09:17:30.962233+0000","last_clean":"2024-04-25T09:17:30.962233+0000","last_became_active":"2024-04-25T09:17:30.958235+0000","last_became_peered":"2024-04-25T09:17:30.958235+0000","last_unstale":"2024-04-25T09:17:30.962233+0000","last_undegraded":"2024-04-25T09:17:30.962233+0000","last_fullsized":"2024-04-25T09:17:30.962233+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:09.193335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.961100+0000","last_change":"2024-04-25T09:17:30.956207+0000","last_active":"2024-04-25T09:17:30.961100+0000","last_peered":"2024-04-25T09:17:30.961100+0000","last_clean":"2024-04-25T09:17:30.961100+0000","last_became_active":"2024-04-25T09:17:30.955883+0000","last_became_peered":"2024-04-25T09:17:30.955883+0000","last_unstale":"2024-04-25T09:17:30.961100+0000","last_undegraded":"2024-04-25T09:17:30.961100+0000","last_fullsized":"2024-04-25T09:17:30.961100+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:30.351587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.951835+0000","last_change":"2024-04-25T09:17:30.946712+0000","last_active":"2024-04-25T09:17:30.951835+0000","last_peered":"2024-04-25T09:17:30.951835+0000","last_clean":"2024-04-25T09:17:30.951835+0000","last_became_active":"2024-04-25T09:17:30.946294+0000","last_became_peered":"2024-04-25T09:17:30.946294+0000","last_unstale":"2024-04-25T09:17:30.951835+0000","last_undegraded":"2024-04-25T09:17:30.951835+0000","last_fullsized":"2024-04-25T09:17:30.951835+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:33.049478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.975398+0000","last_change":"2024-04-25T09:17:30.972985+0000","last_active":"2024-04-25T09:17:30.975398+0000","last_peered":"2024-04-25T09:17:30.975398+0000","last_clean":"2024-04-25T09:17:30.975398+0000","last_became_active":"2024-04-25T09:17:30.971397+0000","last_became_peered":"2024-04-25T09:17:30.971397+0000","last_unstale":"2024-04-25T09:17:30.975398+0000","last_undegraded":"2024-04-25T09:17:30.975398+0000","last_fullsized":"2024-04-25T09:17:30.975398+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:47.611111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967846+0000","last_change":"2024-04-25T09:17:30.964043+0000","last_active":"2024-04-25T09:17:30.967846+0000","last_peered":"2024-04-25T09:17:30.967846+0000","last_clean":"2024-04-25T09:17:30.967846+0000","last_became_active":"2024-04-25T09:17:30.960757+0000","last_became_peered":"2024-04-25T09:17:30.960757+0000","last_unstale":"2024-04-25T09:17:30.967846+0000","last_undegraded":"2024-04-25T09:17:30.967846+0000","last_fullsized":"2024-04-25T09:17:30.967846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:01.343706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.972093+0000","last_change":"2024-04-25T09:17:30.970545+0000","last_active":"2024-04-25T09:17:30.972093+0000","last_peered":"2024-04-25T09:17:30.972093+0000","last_clean":"2024-04-25T09:17:30.972093+0000","last_became_active":"2024-04-25T09:17:30.968411+0000","last_became_peered":"2024-04-25T09:17:30.968411+0000","last_unstale":"2024-04-25T09:17:30.972093+0000","last_undegraded":"2024-04-25T09:17:30.972093+0000","last_fullsized":"2024-04-25T09:17:30.972093+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:31.962089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.964367+0000","last_change":"2024-04-25T09:17:30.961374+0000","last_active":"2024-04-25T09:17:30.964367+0000","last_peered":"2024-04-25T09:17:30.964367+0000","last_clean":"2024-04-25T09:17:30.964367+0000","last_became_active":"2024-04-25T09:17:30.961069+0000","last_became_peered":"2024-04-25T09:17:30.961069+0000","last_unstale":"2024-04-25T09:17:30.964367+0000","last_undegraded":"2024-04-25T09:17:30.964367+0000","last_fullsized":"2024-04-25T09:17:30.964367+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:12.744863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.959306+0000","last_change":"2024-04-25T09:17:30.958124+0000","last_active":"2024-04-25T09:17:30.959306+0000","last_peered":"2024-04-25T09:17:30.959306+0000","last_clean":"2024-04-25T09:17:30.959306+0000","last_became_active":"2024-04-25T09:17:30.957794+0000","last_became_peered":"2024-04-25T09:17:30.957794+0000","last_unstale":"2024-04-25T09:17:30.959306+0000","last_undegraded":"2024-04-25T09:17:30.959306+0000","last_fullsized":"2024-04-25T09:17:30.959306+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:34.877972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.951813+0000","last_change":"2024-04-25T09:17:30.947757+0000","last_active":"2024-04-25T09:17:30.951813+0000","last_peered":"2024-04-25T09:17:30.951813+0000","last_clean":"2024-04-25T09:17:30.951813+0000","last_became_active":"2024-04-25T09:17:30.947425+0000","last_became_peered":"2024-04-25T09:17:30.947425+0000","last_unstale":"2024-04-25T09:17:30.951813+0000","last_undegraded":"2024-04-25T09:17:30.951813+0000","last_fullsized":"2024-04-25T09:17:30.951813+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:26.411855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.960851+0000","last_change":"2024-04-25T09:17:30.958412+0000","last_active":"2024-04-25T09:17:30.960851+0000","last_peered":"2024-04-25T09:17:30.960851+0000","last_clean":"2024-04-25T09:17:30.960851+0000","last_became_active":"2024-04-25T09:17:30.957728+0000","last_became_peered":"2024-04-25T09:17:30.957728+0000","last_unstale":"2024-04-25T09:17:30.960851+0000","last_undegraded":"2024-04-25T09:17:30.960851+0000","last_fullsized":"2024-04-25T09:17:30.960851+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:47.116698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.952071+0000","last_change":"2024-04-25T09:17:30.947080+0000","last_active":"2024-04-25T09:17:30.952071+0000","last_peered":"2024-04-25T09:17:30.952071+0000","last_clean":"2024-04-25T09:17:30.952071+0000","last_became_active":"2024-04-25T09:17:30.946608+0000","last_became_peered":"2024-04-25T09:17:30.946608+0000","last_unstale":"2024-04-25T09:17:30.952071+0000","last_undegraded":"2024-04-25T09:17:30.952071+0000","last_fullsized":"2024-04-25T09:17:30.952071+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:04.517031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965036+0000","last_change":"2024-04-25T09:17:30.963750+0000","last_active":"2024-04-25T09:17:30.965036+0000","last_peered":"2024-04-25T09:17:30.965036+0000","last_clean":"2024-04-25T09:17:30.965036+0000","last_became_active":"2024-04-25T09:17:30.962750+0000","last_became_peered":"2024-04-25T09:17:30.962750+0000","last_unstale":"2024-04-25T09:17:30.965036+0000","last_undegraded":"2024-04-25T09:17:30.965036+0000","last_fullsized":"2024-04-25T09:17:30.965036+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:04.038785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.963868+0000","last_change":"2024-04-25T09:17:30.960106+0000","last_active":"2024-04-25T09:17:30.963868+0000","last_peered":"2024-04-25T09:17:30.963868+0000","last_clean":"2024-04-25T09:17:30.963868+0000","last_became_active":"2024-04-25T09:17:30.959093+0000","last_became_peered":"2024-04-25T09:17:30.959093+0000","last_unstale":"2024-04-25T09:17:30.963868+0000","last_undegraded":"2024-04-25T09:17:30.963868+0000","last_fullsized":"2024-04-25T09:17:30.963868+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:05.413500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.955900+0000","last_change":"2024-04-25T09:17:30.953983+0000","last_active":"2024-04-25T09:17:30.955900+0000","last_peered":"2024-04-25T09:17:30.955900+0000","last_clean":"2024-04-25T09:17:30.955900+0000","last_became_active":"2024-04-25T09:17:30.953157+0000","last_became_peered":"2024-04-25T09:17:30.953157+0000","last_unstale":"2024-04-25T09:17:30.955900+0000","last_undegraded":"2024-04-25T09:17:30.955900+0000","last_fullsized":"2024-04-25T09:17:30.955900+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:59.813720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.961322+0000","last_change":"2024-04-25T09:17:30.959660+0000","last_active":"2024-04-25T09:17:30.961322+0000","last_peered":"2024-04-25T09:17:30.961322+0000","last_clean":"2024-04-25T09:17:30.961322+0000","last_became_active":"2024-04-25T09:17:30.959308+0000","last_became_peered":"2024-04-25T09:17:30.959308+0000","last_unstale":"2024-04-25T09:17:30.961322+0000","last_undegraded":"2024-04-25T09:17:30.961322+0000","last_fullsized":"2024-04-25T09:17:30.961322+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:51.013903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962147+0000","last_change":"2024-04-25T09:17:30.960550+0000","last_active":"2024-04-25T09:17:30.962147+0000","last_peered":"2024-04-25T09:17:30.962147+0000","last_clean":"2024-04-25T09:17:30.962147+0000","last_became_active":"2024-04-25T09:17:30.957800+0000","last_became_peered":"2024-04-25T09:17:30.957800+0000","last_unstale":"2024-04-25T09:17:30.962147+0000","last_undegraded":"2024-04-25T09:17:30.962147+0000","last_fullsized":"2024-04-25T09:17:30.962147+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:26.341065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"147'4","reported_seq":16,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.984424+0000","last_change":"2024-04-25T09:17:29.934137+0000","last_active":"2024-04-25T09:17:30.984424+0000","last_peered":"2024-04-25T09:17:30.984424+0000","last_clean":"2024-04-25T09:17:30.984424+0000","last_became_active":"2024-04-25T09:17:29.933695+0000","last_became_peered":"2024-04-25T09:17:29.933695+0000","last_unstale":"2024-04-25T09:17:30.984424+0000","last_undegraded":"2024-04-25T09:17:30.984424+0000","last_fullsized":"2024-04-25T09:17:30.984424+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:49.888869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.958971+0000","last_change":"2024-04-25T09:17:30.956249+0000","last_active":"2024-04-25T09:17:30.958971+0000","last_peered":"2024-04-25T09:17:30.958971+0000","last_clean":"2024-04-25T09:17:30.958971+0000","last_became_active":"2024-04-25T09:17:30.955971+0000","last_became_peered":"2024-04-25T09:17:30.955971+0000","last_unstale":"2024-04-25T09:17:30.958971+0000","last_undegraded":"2024-04-25T09:17:30.958971+0000","last_fullsized":"2024-04-25T09:17:30.958971+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:23.344835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.982922+0000","last_change":"2024-04-25T09:17:29.934947+0000","last_active":"2024-04-25T09:17:30.982922+0000","last_peered":"2024-04-25T09:17:30.982922+0000","last_clean":"2024-04-25T09:17:30.982922+0000","last_became_active":"2024-04-25T09:17:29.934316+0000","last_became_peered":"2024-04-25T09:17:29.934316+0000","last_unstale":"2024-04-25T09:17:30.982922+0000","last_undegraded":"2024-04-25T09:17:30.982922+0000","last_fullsized":"2024-04-25T09:17:30.982922+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:25.397215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965007+0000","last_change":"2024-04-25T09:17:30.964282+0000","last_active":"2024-04-25T09:17:30.965007+0000","last_peered":"2024-04-25T09:17:30.965007+0000","last_clean":"2024-04-25T09:17:30.965007+0000","last_became_active":"2024-04-25T09:17:30.963728+0000","last_became_peered":"2024-04-25T09:17:30.963728+0000","last_unstale":"2024-04-25T09:17:30.965007+0000","last_undegraded":"2024-04-25T09:17:30.965007+0000","last_fullsized":"2024-04-25T09:17:30.965007+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:02.549839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.980997+0000","last_change":"2024-04-25T09:17:29.937182+0000","last_active":"2024-04-25T09:17:30.980997+0000","last_peered":"2024-04-25T09:17:30.980997+0000","last_clean":"2024-04-25T09:17:30.980997+0000","last_became_active":"2024-04-25T09:17:29.936889+0000","last_became_peered":"2024-04-25T09:17:29.936889+0000","last_unstale":"2024-04-25T09:17:30.980997+0000","last_undegraded":"2024-04-25T09:17:30.980997+0000","last_fullsized":"2024-04-25T09:17:30.980997+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:46.832028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965979+0000","last_change":"2024-04-25T09:17:30.959788+0000","last_active":"2024-04-25T09:17:30.965979+0000","last_peered":"2024-04-25T09:17:30.965979+0000","last_clean":"2024-04-25T09:17:30.965979+0000","last_became_active":"2024-04-25T09:17:30.959452+0000","last_became_peered":"2024-04-25T09:17:30.959452+0000","last_unstale":"2024-04-25T09:17:30.965979+0000","last_undegraded":"2024-04-25T09:17:30.965979+0000","last_fullsized":"2024-04-25T09:17:30.965979+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:12.717063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.983088+0000","last_change":"2024-04-25T09:17:29.935082+0000","last_active":"2024-04-25T09:17:30.983088+0000","last_peered":"2024-04-25T09:17:30.983088+0000","last_clean":"2024-04-25T09:17:30.983088+0000","last_became_active":"2024-04-25T09:17:29.934795+0000","last_became_peered":"2024-04-25T09:17:29.934795+0000","last_unstale":"2024-04-25T09:17:30.983088+0000","last_undegraded":"2024-04-25T09:17:30.983088+0000","last_fullsized":"2024-04-25T09:17:30.983088+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:38.336509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.956134+0000","last_change":"2024-04-25T09:17:30.954614+0000","last_active":"2024-04-25T09:17:30.956134+0000","last_peered":"2024-04-25T09:17:30.956134+0000","last_clean":"2024-04-25T09:17:30.956134+0000","last_became_active":"2024-04-25T09:17:30.954281+0000","last_became_peered":"2024-04-25T09:17:30.954281+0000","last_unstale":"2024-04-25T09:17:30.956134+0000","last_undegraded":"2024-04-25T09:17:30.956134+0000","last_fullsized":"2024-04-25T09:17:30.956134+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:29.823514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.981954+0000","last_change":"2024-04-25T09:17:29.938258+0000","last_active":"2024-04-25T09:17:30.981954+0000","last_peered":"2024-04-25T09:17:30.981954+0000","last_clean":"2024-04-25T09:17:30.981954+0000","last_became_active":"2024-04-25T09:17:29.937868+0000","last_became_peered":"2024-04-25T09:17:29.937868+0000","last_unstale":"2024-04-25T09:17:30.981954+0000","last_undegraded":"2024-04-25T09:17:30.981954+0000","last_fullsized":"2024-04-25T09:17:30.981954+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:30.944766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969844+0000","last_change":"2024-04-25T09:17:30.962742+0000","last_active":"2024-04-25T09:17:30.969844+0000","last_peered":"2024-04-25T09:17:30.969844+0000","last_clean":"2024-04-25T09:17:30.969844+0000","last_became_active":"2024-04-25T09:17:30.961444+0000","last_became_peered":"2024-04-25T09:17:30.961444+0000","last_unstale":"2024-04-25T09:17:30.969844+0000","last_undegraded":"2024-04-25T09:17:30.969844+0000","last_fullsized":"2024-04-25T09:17:30.969844+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:24.193920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.982534+0000","last_change":"2024-04-25T09:17:29.937932+0000","last_active":"2024-04-25T09:17:30.982534+0000","last_peered":"2024-04-25T09:17:30.982534+0000","last_clean":"2024-04-25T09:17:30.982534+0000","last_became_active":"2024-04-25T09:17:29.937624+0000","last_became_peered":"2024-04-25T09:17:29.937624+0000","last_unstale":"2024-04-25T09:17:30.982534+0000","last_undegraded":"2024-04-25T09:17:30.982534+0000","last_fullsized":"2024-04-25T09:17:30.982534+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:56.189727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962121+0000","last_change":"2024-04-25T09:17:30.960858+0000","last_active":"2024-04-25T09:17:30.962121+0000","last_peered":"2024-04-25T09:17:30.962121+0000","last_clean":"2024-04-25T09:17:30.962121+0000","last_became_active":"2024-04-25T09:17:30.960361+0000","last_became_peered":"2024-04-25T09:17:30.960361+0000","last_unstale":"2024-04-25T09:17:30.962121+0000","last_undegraded":"2024-04-25T09:17:30.962121+0000","last_fullsized":"2024-04-25T09:17:30.962121+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:15.901463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.941710+0000","last_change":"2024-04-25T09:17:29.932748+0000","last_active":"2024-04-25T09:17:30.941710+0000","last_peered":"2024-04-25T09:17:30.941710+0000","last_clean":"2024-04-25T09:17:30.941710+0000","last_became_active":"2024-04-25T09:17:29.932427+0000","last_became_peered":"2024-04-25T09:17:29.932427+0000","last_unstale":"2024-04-25T09:17:30.941710+0000","last_undegraded":"2024-04-25T09:17:30.941710+0000","last_fullsized":"2024-04-25T09:17:30.941710+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.970387+0000","last_change":"2024-04-25T09:17:30.955247+0000","last_active":"2024-04-25T09:17:30.970387+0000","last_peered":"2024-04-25T09:17:30.970387+0000","last_clean":"2024-04-25T09:17:30.970387+0000","last_became_active":"2024-04-25T09:17:30.954926+0000","last_became_peered":"2024-04-25T09:17:30.954926+0000","last_unstale":"2024-04-25T09:17:30.970387+0000","last_undegraded":"2024-04-25T09:17:30.970387+0000","last_fullsized":"2024-04-25T09:17:30.970387+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:28.258503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.942757+0000","last_change":"2024-04-25T09:17:29.937007+0000","last_active":"2024-04-25T09:17:30.942757+0000","last_peered":"2024-04-25T09:17:30.942757+0000","last_clean":"2024-04-25T09:17:30.942757+0000","last_became_active":"2024-04-25T09:17:29.936393+0000","last_became_peered":"2024-04-25T09:17:29.936393+0000","last_unstale":"2024-04-25T09:17:30.942757+0000","last_undegraded":"2024-04-25T09:17:30.942757+0000","last_fullsized":"2024-04-25T09:17:30.942757+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:22.783606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969987+0000","last_change":"2024-04-25T09:17:30.960914+0000","last_active":"2024-04-25T09:17:30.969987+0000","last_peered":"2024-04-25T09:17:30.969987+0000","last_clean":"2024-04-25T09:17:30.969987+0000","last_became_active":"2024-04-25T09:17:30.960349+0000","last_became_peered":"2024-04-25T09:17:30.960349+0000","last_unstale":"2024-04-25T09:17:30.969987+0000","last_undegraded":"2024-04-25T09:17:30.969987+0000","last_fullsized":"2024-04-25T09:17:30.969987+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:09.121647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.953711+0000","last_change":"2024-04-25T09:17:29.925598+0000","last_active":"2024-04-25T09:17:30.953711+0000","last_peered":"2024-04-25T09:17:30.953711+0000","last_clean":"2024-04-25T09:17:30.953711+0000","last_became_active":"2024-04-25T09:17:29.925228+0000","last_became_peered":"2024-04-25T09:17:29.925228+0000","last_unstale":"2024-04-25T09:17:30.953711+0000","last_undegraded":"2024-04-25T09:17:30.953711+0000","last_fullsized":"2024-04-25T09:17:30.953711+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:33.102782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.976017+0000","last_change":"2024-04-25T09:17:30.973108+0000","last_active":"2024-04-25T09:17:30.976017+0000","last_peered":"2024-04-25T09:17:30.976017+0000","last_clean":"2024-04-25T09:17:30.976017+0000","last_became_active":"2024-04-25T09:17:30.972817+0000","last_became_peered":"2024-04-25T09:17:30.972817+0000","last_unstale":"2024-04-25T09:17:30.976017+0000","last_undegraded":"2024-04-25T09:17:30.976017+0000","last_fullsized":"2024-04-25T09:17:30.976017+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:31.962089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.939557+0000","last_change":"2024-04-25T09:17:29.939322+0000","last_active":"2024-04-25T09:17:30.939557+0000","last_peered":"2024-04-25T09:17:30.939557+0000","last_clean":"2024-04-25T09:17:30.939557+0000","last_became_active":"2024-04-25T09:17:29.939044+0000","last_became_peered":"2024-04-25T09:17:29.939044+0000","last_unstale":"2024-04-25T09:17:30.939557+0000","last_undegraded":"2024-04-25T09:17:30.939557+0000","last_fullsized":"2024-04-25T09:17:30.939557+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.578252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.962027+0000","last_change":"2024-04-25T09:17:30.960550+0000","last_active":"2024-04-25T09:17:30.962027+0000","last_peered":"2024-04-25T09:17:30.962027+0000","last_clean":"2024-04-25T09:17:30.962027+0000","last_became_active":"2024-04-25T09:17:30.960195+0000","last_became_peered":"2024-04-25T09:17:30.960195+0000","last_unstale":"2024-04-25T09:17:30.962027+0000","last_undegraded":"2024-04-25T09:17:30.962027+0000","last_fullsized":"2024-04-25T09:17:30.962027+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:24.193920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"147'6","reported_seq":18,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.986106+0000","last_change":"2024-04-25T09:17:29.936816+0000","last_active":"2024-04-25T09:17:30.986106+0000","last_peered":"2024-04-25T09:17:30.986106+0000","last_clean":"2024-04-25T09:17:30.986106+0000","last_became_active":"2024-04-25T09:17:29.936231+0000","last_became_peered":"2024-04-25T09:17:29.936231+0000","last_unstale":"2024-04-25T09:17:30.986106+0000","last_undegraded":"2024-04-25T09:17:30.986106+0000","last_fullsized":"2024-04-25T09:17:30.986106+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:27.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.971344+0000","last_change":"2024-04-25T09:17:30.967232+0000","last_active":"2024-04-25T09:17:30.971344+0000","last_peered":"2024-04-25T09:17:30.971344+0000","last_clean":"2024-04-25T09:17:30.971344+0000","last_became_active":"2024-04-25T09:17:30.966959+0000","last_became_peered":"2024-04-25T09:17:30.966959+0000","last_unstale":"2024-04-25T09:17:30.971344+0000","last_undegraded":"2024-04-25T09:17:30.971344+0000","last_fullsized":"2024-04-25T09:17:30.971344+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:51.848968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.982458+0000","last_change":"2024-04-25T09:17:29.937845+0000","last_active":"2024-04-25T09:17:30.982458+0000","last_peered":"2024-04-25T09:17:30.982458+0000","last_clean":"2024-04-25T09:17:30.982458+0000","last_became_active":"2024-04-25T09:17:29.936050+0000","last_became_peered":"2024-04-25T09:17:29.936050+0000","last_unstale":"2024-04-25T09:17:30.982458+0000","last_undegraded":"2024-04-25T09:17:30.982458+0000","last_fullsized":"2024-04-25T09:17:30.982458+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:29.142608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965061+0000","last_change":"2024-04-25T09:17:30.963842+0000","last_active":"2024-04-25T09:17:30.965061+0000","last_peered":"2024-04-25T09:17:30.965061+0000","last_clean":"2024-04-25T09:17:30.965061+0000","last_became_active":"2024-04-25T09:17:30.962718+0000","last_became_peered":"2024-04-25T09:17:30.962718+0000","last_unstale":"2024-04-25T09:17:30.965061+0000","last_undegraded":"2024-04-25T09:17:30.965061+0000","last_fullsized":"2024-04-25T09:17:30.965061+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:02.549839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.956740+0000","last_change":"2024-04-25T09:17:29.932661+0000","last_active":"2024-04-25T09:17:30.956740+0000","last_peered":"2024-04-25T09:17:30.956740+0000","last_clean":"2024-04-25T09:17:30.956740+0000","last_became_active":"2024-04-25T09:17:29.932352+0000","last_became_peered":"2024-04-25T09:17:29.932352+0000","last_unstale":"2024-04-25T09:17:30.956740+0000","last_undegraded":"2024-04-25T09:17:30.956740+0000","last_fullsized":"2024-04-25T09:17:30.956740+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:22.330195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.971372+0000","last_change":"2024-04-25T09:17:30.965883+0000","last_active":"2024-04-25T09:17:30.971372+0000","last_peered":"2024-04-25T09:17:30.971372+0000","last_clean":"2024-04-25T09:17:30.971372+0000","last_became_active":"2024-04-25T09:17:30.963090+0000","last_became_peered":"2024-04-25T09:17:30.963090+0000","last_unstale":"2024-04-25T09:17:30.971372+0000","last_undegraded":"2024-04-25T09:17:30.971372+0000","last_fullsized":"2024-04-25T09:17:30.971372+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:01.343706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.930764+0000","last_change":"2024-04-25T09:17:29.924782+0000","last_active":"2024-04-25T09:17:30.930764+0000","last_peered":"2024-04-25T09:17:30.930764+0000","last_clean":"2024-04-25T09:17:30.930764+0000","last_became_active":"2024-04-25T09:17:29.923589+0000","last_became_peered":"2024-04-25T09:17:29.923589+0000","last_unstale":"2024-04-25T09:17:30.930764+0000","last_undegraded":"2024-04-25T09:17:30.930764+0000","last_fullsized":"2024-04-25T09:17:30.930764+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.496120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.971349+0000","last_change":"2024-04-25T09:17:30.965409+0000","last_active":"2024-04-25T09:17:30.971349+0000","last_peered":"2024-04-25T09:17:30.971349+0000","last_clean":"2024-04-25T09:17:30.971349+0000","last_became_active":"2024-04-25T09:17:30.965130+0000","last_became_peered":"2024-04-25T09:17:30.965130+0000","last_unstale":"2024-04-25T09:17:30.971349+0000","last_undegraded":"2024-04-25T09:17:30.971349+0000","last_fullsized":"2024-04-25T09:17:30.971349+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:05.413500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"147'8","reported_seq":20,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.985990+0000","last_change":"2024-04-25T09:17:29.913951+0000","last_active":"2024-04-25T09:17:30.985990+0000","last_peered":"2024-04-25T09:17:30.985990+0000","last_clean":"2024-04-25T09:17:30.985990+0000","last_became_active":"2024-04-25T09:17:29.912977+0000","last_became_peered":"2024-04-25T09:17:29.912977+0000","last_unstale":"2024-04-25T09:17:30.985990+0000","last_undegraded":"2024-04-25T09:17:30.985990+0000","last_fullsized":"2024-04-25T09:17:30.985990+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:30.947449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.965920+0000","last_change":"2024-04-25T09:17:30.964198+0000","last_active":"2024-04-25T09:17:30.965920+0000","last_peered":"2024-04-25T09:17:30.965920+0000","last_clean":"2024-04-25T09:17:30.965920+0000","last_became_active":"2024-04-25T09:17:30.963805+0000","last_became_peered":"2024-04-25T09:17:30.963805+0000","last_unstale":"2024-04-25T09:17:30.965920+0000","last_undegraded":"2024-04-25T09:17:30.965920+0000","last_fullsized":"2024-04-25T09:17:30.965920+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:59.351904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.985738+0000","last_change":"2024-04-25T09:17:29.926505+0000","last_active":"2024-04-25T09:17:30.985738+0000","last_peered":"2024-04-25T09:17:30.985738+0000","last_clean":"2024-04-25T09:17:30.985738+0000","last_became_active":"2024-04-25T09:17:29.926150+0000","last_became_peered":"2024-04-25T09:17:29.926150+0000","last_unstale":"2024-04-25T09:17:30.985738+0000","last_undegraded":"2024-04-25T09:17:30.985738+0000","last_fullsized":"2024-04-25T09:17:30.985738+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:46.700951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.952608+0000","last_change":"2024-04-25T09:17:30.949444+0000","last_active":"2024-04-25T09:17:30.952608+0000","last_peered":"2024-04-25T09:17:30.952608+0000","last_clean":"2024-04-25T09:17:30.952608+0000","last_became_active":"2024-04-25T09:17:30.949110+0000","last_became_peered":"2024-04-25T09:17:30.949110+0000","last_unstale":"2024-04-25T09:17:30.952608+0000","last_undegraded":"2024-04-25T09:17:30.952608+0000","last_fullsized":"2024-04-25T09:17:30.952608+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:56.594821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.984755+0000","last_change":"2024-04-25T09:17:29.928052+0000","last_active":"2024-04-25T09:17:30.984755+0000","last_peered":"2024-04-25T09:17:30.984755+0000","last_clean":"2024-04-25T09:17:30.984755+0000","last_became_active":"2024-04-25T09:17:29.927740+0000","last_became_peered":"2024-04-25T09:17:29.927740+0000","last_unstale":"2024-04-25T09:17:30.984755+0000","last_undegraded":"2024-04-25T09:17:30.984755+0000","last_fullsized":"2024-04-25T09:17:30.984755+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:50.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967832+0000","last_change":"2024-04-25T09:17:30.965759+0000","last_active":"2024-04-25T09:17:30.967832+0000","last_peered":"2024-04-25T09:17:30.967832+0000","last_clean":"2024-04-25T09:17:30.967832+0000","last_became_active":"2024-04-25T09:17:30.964183+0000","last_became_peered":"2024-04-25T09:17:30.964183+0000","last_unstale":"2024-04-25T09:17:30.967832+0000","last_undegraded":"2024-04-25T09:17:30.967832+0000","last_fullsized":"2024-04-25T09:17:30.967832+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:22.534044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"147'1","reported_seq":13,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.981298+0000","last_change":"2024-04-25T09:17:29.938307+0000","last_active":"2024-04-25T09:17:30.981298+0000","last_peered":"2024-04-25T09:17:30.981298+0000","last_clean":"2024-04-25T09:17:30.981298+0000","last_became_active":"2024-04-25T09:17:29.937850+0000","last_became_peered":"2024-04-25T09:17:29.937850+0000","last_unstale":"2024-04-25T09:17:30.981298+0000","last_undegraded":"2024-04-25T09:17:30.981298+0000","last_fullsized":"2024-04-25T09:17:30.981298+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:10.096348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.961799+0000","last_change":"2024-04-25T09:17:30.961062+0000","last_active":"2024-04-25T09:17:30.961799+0000","last_peered":"2024-04-25T09:17:30.961799+0000","last_clean":"2024-04-25T09:17:30.961799+0000","last_became_active":"2024-04-25T09:17:30.960736+0000","last_became_peered":"2024-04-25T09:17:30.960736+0000","last_unstale":"2024-04-25T09:17:30.961799+0000","last_undegraded":"2024-04-25T09:17:30.961799+0000","last_fullsized":"2024-04-25T09:17:30.961799+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:02.946159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.933060+0000","last_change":"2024-04-25T09:17:29.934905+0000","last_active":"2024-04-25T09:17:30.933060+0000","last_peered":"2024-04-25T09:17:30.933060+0000","last_clean":"2024-04-25T09:17:30.933060+0000","last_became_active":"2024-04-25T09:17:29.934607+0000","last_became_peered":"2024-04-25T09:17:29.934607+0000","last_unstale":"2024-04-25T09:17:30.933060+0000","last_undegraded":"2024-04-25T09:17:30.933060+0000","last_fullsized":"2024-04-25T09:17:30.933060+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:29.965126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967871+0000","last_change":"2024-04-25T09:17:30.964325+0000","last_active":"2024-04-25T09:17:30.967871+0000","last_peered":"2024-04-25T09:17:30.967871+0000","last_clean":"2024-04-25T09:17:30.967871+0000","last_became_active":"2024-04-25T09:17:30.963690+0000","last_became_peered":"2024-04-25T09:17:30.963690+0000","last_unstale":"2024-04-25T09:17:30.967871+0000","last_undegraded":"2024-04-25T09:17:30.967871+0000","last_fullsized":"2024-04-25T09:17:30.967871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:51.848968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.983124+0000","last_change":"2024-04-25T09:17:29.934810+0000","last_active":"2024-04-25T09:17:30.983124+0000","last_peered":"2024-04-25T09:17:30.983124+0000","last_clean":"2024-04-25T09:17:30.983124+0000","last_became_active":"2024-04-25T09:17:29.934249+0000","last_became_peered":"2024-04-25T09:17:29.934249+0000","last_unstale":"2024-04-25T09:17:30.983124+0000","last_undegraded":"2024-04-25T09:17:30.983124+0000","last_fullsized":"2024-04-25T09:17:30.983124+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:02.634503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.955430+0000","last_change":"2024-04-25T09:17:30.951601+0000","last_active":"2024-04-25T09:17:30.955430+0000","last_peered":"2024-04-25T09:17:30.955430+0000","last_clean":"2024-04-25T09:17:30.955430+0000","last_became_active":"2024-04-25T09:17:30.951322+0000","last_became_peered":"2024-04-25T09:17:30.951322+0000","last_unstale":"2024-04-25T09:17:30.955430+0000","last_undegraded":"2024-04-25T09:17:30.955430+0000","last_fullsized":"2024-04-25T09:17:30.955430+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:02.486970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.944647+0000","last_change":"2024-04-25T09:17:29.937682+0000","last_active":"2024-04-25T09:17:30.944647+0000","last_peered":"2024-04-25T09:17:30.944647+0000","last_clean":"2024-04-25T09:17:30.944647+0000","last_became_active":"2024-04-25T09:17:29.937063+0000","last_became_peered":"2024-04-25T09:17:29.937063+0000","last_unstale":"2024-04-25T09:17:30.944647+0000","last_undegraded":"2024-04-25T09:17:30.944647+0000","last_fullsized":"2024-04-25T09:17:30.944647+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:14.886823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.963065+0000","last_change":"2024-04-25T09:17:30.956562+0000","last_active":"2024-04-25T09:17:30.963065+0000","last_peered":"2024-04-25T09:17:30.963065+0000","last_clean":"2024-04-25T09:17:30.963065+0000","last_became_active":"2024-04-25T09:17:30.955847+0000","last_became_peered":"2024-04-25T09:17:30.955847+0000","last_unstale":"2024-04-25T09:17:30.963065+0000","last_undegraded":"2024-04-25T09:17:30.963065+0000","last_fullsized":"2024-04-25T09:17:30.963065+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:05.358730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.946951+0000","last_change":"2024-04-25T09:17:29.913928+0000","last_active":"2024-04-25T09:17:30.946951+0000","last_peered":"2024-04-25T09:17:30.946951+0000","last_clean":"2024-04-25T09:17:30.946951+0000","last_became_active":"2024-04-25T09:17:29.912987+0000","last_became_peered":"2024-04-25T09:17:29.912987+0000","last_unstale":"2024-04-25T09:17:30.946951+0000","last_undegraded":"2024-04-25T09:17:30.946951+0000","last_fullsized":"2024-04-25T09:17:30.946951+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:42.198793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.955138+0000","last_change":"2024-04-25T09:17:30.951325+0000","last_active":"2024-04-25T09:17:30.955138+0000","last_peered":"2024-04-25T09:17:30.955138+0000","last_clean":"2024-04-25T09:17:30.955138+0000","last_became_active":"2024-04-25T09:17:30.951034+0000","last_became_peered":"2024-04-25T09:17:30.951034+0000","last_unstale":"2024-04-25T09:17:30.955138+0000","last_undegraded":"2024-04-25T09:17:30.955138+0000","last_fullsized":"2024-04-25T09:17:30.955138+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:06.099265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.949127+0000","last_change":"2024-04-25T09:17:29.935388+0000","last_active":"2024-04-25T09:17:30.949127+0000","last_peered":"2024-04-25T09:17:30.949127+0000","last_clean":"2024-04-25T09:17:30.949127+0000","last_became_active":"2024-04-25T09:17:29.935028+0000","last_became_peered":"2024-04-25T09:17:29.935028+0000","last_unstale":"2024-04-25T09:17:30.949127+0000","last_undegraded":"2024-04-25T09:17:30.949127+0000","last_fullsized":"2024-04-25T09:17:30.949127+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:32.401465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969965+0000","last_change":"2024-04-25T09:17:30.960665+0000","last_active":"2024-04-25T09:17:30.969965+0000","last_peered":"2024-04-25T09:17:30.969965+0000","last_clean":"2024-04-25T09:17:30.969965+0000","last_became_active":"2024-04-25T09:17:30.959789+0000","last_became_peered":"2024-04-25T09:17:30.959789+0000","last_unstale":"2024-04-25T09:17:30.969965+0000","last_undegraded":"2024-04-25T09:17:30.969965+0000","last_fullsized":"2024-04-25T09:17:30.969965+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:11.110989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"147'2","reported_seq":14,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.984845+0000","last_change":"2024-04-25T09:17:29.942258+0000","last_active":"2024-04-25T09:17:30.984845+0000","last_peered":"2024-04-25T09:17:30.984845+0000","last_clean":"2024-04-25T09:17:30.984845+0000","last_became_active":"2024-04-25T09:17:29.941428+0000","last_became_peered":"2024-04-25T09:17:29.941428+0000","last_unstale":"2024-04-25T09:17:30.984845+0000","last_undegraded":"2024-04-25T09:17:30.984845+0000","last_fullsized":"2024-04-25T09:17:30.984845+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:55.906208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969758+0000","last_change":"2024-04-25T09:17:30.962700+0000","last_active":"2024-04-25T09:17:30.969758+0000","last_peered":"2024-04-25T09:17:30.969758+0000","last_clean":"2024-04-25T09:17:30.969758+0000","last_became_active":"2024-04-25T09:17:30.962354+0000","last_became_peered":"2024-04-25T09:17:30.962354+0000","last_unstale":"2024-04-25T09:17:30.969758+0000","last_undegraded":"2024-04-25T09:17:30.969758+0000","last_fullsized":"2024-04-25T09:17:30.969758+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:23.344835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.932700+0000","last_change":"2024-04-25T09:17:29.937413+0000","last_active":"2024-04-25T09:17:30.932700+0000","last_peered":"2024-04-25T09:17:30.932700+0000","last_clean":"2024-04-25T09:17:30.932700+0000","last_became_active":"2024-04-25T09:17:29.937100+0000","last_became_peered":"2024-04-25T09:17:29.937100+0000","last_unstale":"2024-04-25T09:17:30.932700+0000","last_undegraded":"2024-04-25T09:17:30.932700+0000","last_fullsized":"2024-04-25T09:17:30.932700+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:00.567475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.950759+0000","last_change":"2024-04-25T09:17:30.946494+0000","last_active":"2024-04-25T09:17:30.950759+0000","last_peered":"2024-04-25T09:17:30.950759+0000","last_clean":"2024-04-25T09:17:30.950759+0000","last_became_active":"2024-04-25T09:17:30.946066+0000","last_became_peered":"2024-04-25T09:17:30.946066+0000","last_unstale":"2024-04-25T09:17:30.950759+0000","last_undegraded":"2024-04-25T09:17:30.950759+0000","last_fullsized":"2024-04-25T09:17:30.950759+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:03.649144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.952506+0000","last_change":"2024-04-25T09:17:29.935980+0000","last_active":"2024-04-25T09:17:30.952506+0000","last_peered":"2024-04-25T09:17:30.952506+0000","last_clean":"2024-04-25T09:17:30.952506+0000","last_became_active":"2024-04-25T09:17:29.935364+0000","last_became_peered":"2024-04-25T09:17:29.935364+0000","last_unstale":"2024-04-25T09:17:30.952506+0000","last_undegraded":"2024-04-25T09:17:30.952506+0000","last_fullsized":"2024-04-25T09:17:30.952506+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:08.107007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.961125+0000","last_change":"2024-04-25T09:17:30.957145+0000","last_active":"2024-04-25T09:17:30.961125+0000","last_peered":"2024-04-25T09:17:30.961125+0000","last_clean":"2024-04-25T09:17:30.961125+0000","last_became_active":"2024-04-25T09:17:30.956808+0000","last_became_peered":"2024-04-25T09:17:30.956808+0000","last_unstale":"2024-04-25T09:17:30.961125+0000","last_undegraded":"2024-04-25T09:17:30.961125+0000","last_fullsized":"2024-04-25T09:17:30.961125+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:34.117423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.945274+0000","last_change":"2024-04-25T09:17:29.936674+0000","last_active":"2024-04-25T09:17:30.945274+0000","last_peered":"2024-04-25T09:17:30.945274+0000","last_clean":"2024-04-25T09:17:30.945274+0000","last_became_active":"2024-04-25T09:17:29.935332+0000","last_became_peered":"2024-04-25T09:17:29.935332+0000","last_unstale":"2024-04-25T09:17:30.945274+0000","last_undegraded":"2024-04-25T09:17:30.945274+0000","last_fullsized":"2024-04-25T09:17:30.945274+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:38.647480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.964794+0000","last_change":"2024-04-25T09:17:30.963816+0000","last_active":"2024-04-25T09:17:30.964794+0000","last_peered":"2024-04-25T09:17:30.964794+0000","last_clean":"2024-04-25T09:17:30.964794+0000","last_became_active":"2024-04-25T09:17:30.963483+0000","last_became_peered":"2024-04-25T09:17:30.963483+0000","last_unstale":"2024-04-25T09:17:30.964794+0000","last_undegraded":"2024-04-25T09:17:30.964794+0000","last_fullsized":"2024-04-25T09:17:30.964794+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:05.413500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.932261+0000","last_change":"2024-04-25T09:17:29.926703+0000","last_active":"2024-04-25T09:17:30.932261+0000","last_peered":"2024-04-25T09:17:30.932261+0000","last_clean":"2024-04-25T09:17:30.932261+0000","last_became_active":"2024-04-25T09:17:29.926267+0000","last_became_peered":"2024-04-25T09:17:29.926267+0000","last_unstale":"2024-04-25T09:17:30.932261+0000","last_undegraded":"2024-04-25T09:17:30.932261+0000","last_fullsized":"2024-04-25T09:17:30.932261+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:28.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.969823+0000","last_change":"2024-04-25T09:17:30.961559+0000","last_active":"2024-04-25T09:17:30.969823+0000","last_peered":"2024-04-25T09:17:30.969823+0000","last_clean":"2024-04-25T09:17:30.969823+0000","last_became_active":"2024-04-25T09:17:30.961205+0000","last_became_peered":"2024-04-25T09:17:30.961205+0000","last_unstale":"2024-04-25T09:17:30.969823+0000","last_undegraded":"2024-04-25T09:17:30.969823+0000","last_fullsized":"2024-04-25T09:17:30.969823+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:05.358730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.945090+0000","last_change":"2024-04-25T09:17:29.941369+0000","last_active":"2024-04-25T09:17:30.945090+0000","last_peered":"2024-04-25T09:17:30.945090+0000","last_clean":"2024-04-25T09:17:30.945090+0000","last_became_active":"2024-04-25T09:17:29.941030+0000","last_became_peered":"2024-04-25T09:17:29.941030+0000","last_unstale":"2024-04-25T09:17:30.945090+0000","last_undegraded":"2024-04-25T09:17:30.945090+0000","last_fullsized":"2024-04-25T09:17:30.945090+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:29.081412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.967563+0000","last_change":"2024-04-25T09:17:30.955382+0000","last_active":"2024-04-25T09:17:30.967563+0000","last_peered":"2024-04-25T09:17:30.967563+0000","last_clean":"2024-04-25T09:17:30.967563+0000","last_became_active":"2024-04-25T09:17:30.954255+0000","last_became_peered":"2024-04-25T09:17:30.954255+0000","last_unstale":"2024-04-25T09:17:30.967563+0000","last_undegraded":"2024-04-25T09:17:30.967563+0000","last_fullsized":"2024-04-25T09:17:30.967563+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:39.662120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.952774+0000","last_change":"2024-04-25T09:17:29.934014+0000","last_active":"2024-04-25T09:17:30.952774+0000","last_peered":"2024-04-25T09:17:30.952774+0000","last_clean":"2024-04-25T09:17:30.952774+0000","last_became_active":"2024-04-25T09:17:29.933696+0000","last_became_peered":"2024-04-25T09:17:29.933696+0000","last_unstale":"2024-04-25T09:17:30.952774+0000","last_undegraded":"2024-04-25T09:17:30.952774+0000","last_fullsized":"2024-04-25T09:17:30.952774+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:08.107007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.956693+0000","last_change":"2024-04-25T09:17:30.953294+0000","last_active":"2024-04-25T09:17:30.956693+0000","last_peered":"2024-04-25T09:17:30.956693+0000","last_clean":"2024-04-25T09:17:30.956693+0000","last_became_active":"2024-04-25T09:17:30.953004+0000","last_became_peered":"2024-04-25T09:17:30.953004+0000","last_unstale":"2024-04-25T09:17:30.956693+0000","last_undegraded":"2024-04-25T09:17:30.956693+0000","last_fullsized":"2024-04-25T09:17:30.956693+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:25.216126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.943736+0000","last_change":"2024-04-25T09:17:29.933350+0000","last_active":"2024-04-25T09:17:30.943736+0000","last_peered":"2024-04-25T09:17:30.943736+0000","last_clean":"2024-04-25T09:17:30.943736+0000","last_became_active":"2024-04-25T09:17:29.933009+0000","last_became_peered":"2024-04-25T09:17:29.933009+0000","last_unstale":"2024-04-25T09:17:30.943736+0000","last_undegraded":"2024-04-25T09:17:30.943736+0000","last_fullsized":"2024-04-25T09:17:30.943736+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:32.034837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.955695+0000","last_change":"2024-04-25T09:17:30.949383+0000","last_active":"2024-04-25T09:17:30.955695+0000","last_peered":"2024-04-25T09:17:30.955695+0000","last_clean":"2024-04-25T09:17:30.955695+0000","last_became_active":"2024-04-25T09:17:30.949031+0000","last_became_peered":"2024-04-25T09:17:30.949031+0000","last_unstale":"2024-04-25T09:17:30.955695+0000","last_undegraded":"2024-04-25T09:17:30.955695+0000","last_fullsized":"2024-04-25T09:17:30.955695+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:29.892378+0000","last_clean_scrub_stamp":"2024-04-25T09:17:29.892378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:12.717063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"147'3","reported_seq":15,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.984503+0000","last_change":"2024-04-25T09:17:29.937254+0000","last_active":"2024-04-25T09:17:30.984503+0000","last_peered":"2024-04-25T09:17:30.984503+0000","last_clean":"2024-04-25T09:17:30.984503+0000","last_became_active":"2024-04-25T09:17:29.936976+0000","last_became_peered":"2024-04-25T09:17:29.936976+0000","last_unstale":"2024-04-25T09:17:30.984503+0000","last_undegraded":"2024-04-25T09:17:30.984503+0000","last_fullsized":"2024-04-25T09:17:30.984503+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:17:28.877737+0000","last_clean_scrub_stamp":"2024-04-25T09:17:28.877737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:23.179279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":404,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-25T09:17:30.936091+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:17:30.936091+0000","last_peered":"2024-04-25T09:17:30.936091+0000","last_clean":"2024-04-25T09:17:30.936091+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:17:30.936091+0000","last_undegraded":"2024-04-25T09:17:30.936091+0000","last_fullsized":"2024-04-25T09:17:30.936091+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55188,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316652,"statfs":{"total":96636764160,"available":96580251648,"internally_reserved":0,"allocated":1839104,"data_stored":809301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575025,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59916,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311924,"statfs":{"total":96636764160,"available":96575410176,"internally_reserved":0,"allocated":2420736,"data_stored":1392894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575025,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46856,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324984,"statfs":{"total":96636764160,"available":96588783616,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575026,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59336,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312504,"statfs":{"total":96636764160,"available":96576004096,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575026,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64904,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306936,"statfs":{"total":96636764160,"available":96570302464,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575025,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59916,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311924,"statfs":{"total":96636764160,"available":96575410176,"internally_reserved":0,"allocated":2420736,"data_stored":1392894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575024,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1826816,"data_stored":802526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:17:35.910 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:17:35.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:17:36.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.186+0000 7f5a19000640 1 Processor -- start 2024-04-25T09:17:36.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.186+0000 7f5a19000640 1 -- start start 2024-04-25T09:17:36.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.188+0000 7f5a19000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a141655a0 0x7f5a140cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.188+0000 7f5a19000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 0x7f5a140717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.188+0000 7f5a19000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a140cbd40 con 0x7f5a14165980 2024-04-25T09:17:36.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.188+0000 7f5a19000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a14071d20 con 0x7f5a141655a0 2024-04-25T09:17:36.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.188+0000 7f5a19000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140cc980 con 0x7f5a14071370 2024-04-25T09:17:36.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.188+0000 7f5a13400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a141655a0 0x7f5a140cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.188+0000 7f5a13400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5a141655a0 0x7f5a140cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38302/0 (socket says 172.21.15.62:38302) 2024-04-25T09:17:36.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.188+0000 7f5a13400640 1 -- 172.21.15.62:0/3065050240 learned_addr learned my addr 172.21.15.62:0/3065050240 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:36.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.189+0000 7f5a13400640 1 -- 172.21.15.62:0/3065050240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 msgr2=0x7f5a140717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:36.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.189+0000 7f5a13400640 1 --2- 172.21.15.62:0/3065050240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 0x7f5a140717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.189+0000 7f5a13400640 1 -- 172.21.15.62:0/3065050240 >> v1:172.21.15.121:6789/0 conn(0x7f5a14165980 legacy=0x7f5a140b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:36.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.189+0000 7f5a13400640 1 -- 172.21.15.62:0/3065050240 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140cc330 con 0x7f5a141655a0 2024-04-25T09:17:36.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.190+0000 7f5a13400640 1 --2- 172.21.15.62:0/3065050240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a140cb6b0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f5a1406d310 tx=0x7f5a0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3166840fbd6cecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:36.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.190+0000 7f5a12000640 1 -- 172.21.15.62:0/3065050240 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04037070 con 0x7f5a141655a0 2024-04-25T09:17:36.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.190+0000 7f5a12000640 1 -- 172.21.15.62:0/3065050240 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a04032440 con 0x7f5a141655a0 2024-04-25T09:17:36.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.190+0000 7f5a12000640 1 -- 172.21.15.62:0/3065050240 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04032810 con 0x7f5a141655a0 2024-04-25T09:17:36.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.191+0000 7f5a19000640 1 -- 172.21.15.62:0/3065050240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 msgr2=0x7f5a140cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.191+0000 7f5a19000640 1 --2- 172.21.15.62:0/3065050240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a140cb6b0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f5a1406d310 tx=0x7f5a0402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.191+0000 7f5a19000640 1 -- 172.21.15.62:0/3065050240 shutdown_connections 2024-04-25T09:17:36.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.191+0000 7f5a19000640 1 --2- 172.21.15.62:0/3065050240 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 0x7f5a140717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.191+0000 7f5a19000640 1 --2- 172.21.15.62:0/3065050240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a140cb6b0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.191+0000 7f5a19000640 1 -- 172.21.15.62:0/3065050240 >> 172.21.15.62:0/3065050240 conn(0x7f5a140668f0 msgr2=0x7f5a14066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:36.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.191+0000 7f5a19000640 1 -- 172.21.15.62:0/3065050240 shutdown_connections 2024-04-25T09:17:36.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.191+0000 7f5a19000640 1 -- 172.21.15.62:0/3065050240 wait complete. 2024-04-25T09:17:36.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.193+0000 7f5a19000640 1 Processor -- start 2024-04-25T09:17:36.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.193+0000 7f5a19000640 1 -- start start 2024-04-25T09:17:36.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a19000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 0x7f5a1414a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a19000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a1415d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a19000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a14165980 0x7f5a14159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a19000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14165fc0 con 0x7f5a141655a0 2024-04-25T09:17:36.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a19000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14165e40 con 0x7f5a14165980 2024-04-25T09:17:36.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a19000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1406c250 con 0x7f5a14071370 2024-04-25T09:17:36.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a12a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a1415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a12a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a1415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38308/0 (socket says 172.21.15.62:38308) 2024-04-25T09:17:36.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.194+0000 7f5a12a00640 1 -- 172.21.15.62:0/1639039632 learned_addr learned my addr 172.21.15.62:0/1639039632 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:36.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a13400640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 0x7f5a1414a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a13e00640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a14165980 0x7f5a14159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a12a00640 1 -- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 msgr2=0x7f5a1414a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a12a00640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 0x7f5a1414a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a12a00640 1 -- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a14165980 msgr2=0x7f5a14159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a12a00640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a14165980 0x7f5a14159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a12a00640 1 -- 172.21.15.62:0/1639039632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1415ccc0 con 0x7f5a141655a0 2024-04-25T09:17:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a12a00640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a1415d320 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000b3e0 tx=0x7f5a0000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a0b400640 1 -- 172.21.15.62:0/1639039632 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00015020 con 0x7f5a141655a0 2024-04-25T09:17:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.195+0000 7f5a0b400640 1 -- 172.21.15.62:0/1639039632 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a00011850 con 0x7f5a141655a0 2024-04-25T09:17:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.196+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1415e010 con 0x7f5a141655a0 2024-04-25T09:17:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.196+0000 7f5a0b400640 1 -- 172.21.15.62:0/1639039632 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00011c40 con 0x7f5a141655a0 2024-04-25T09:17:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.196+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1415e5d0 con 0x7f5a141655a0 2024-04-25T09:17:36.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.197+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a140b3f90 con 0x7f5a141655a0 2024-04-25T09:17:36.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.198+0000 7f5a0b400640 1 -- 172.21.15.62:0/1639039632 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a00018020 con 0x7f5a141655a0 2024-04-25T09:17:36.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.199+0000 7f5a0b400640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59f406b4d0 0x7f59f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.199+0000 7f5a0b400640 1 -- 172.21.15.62:0/1639039632 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a0008a1a0 con 0x7f5a141655a0 2024-04-25T09:17:36.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.200+0000 7f5a13400640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59f406b4d0 0x7f59f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.201+0000 7f5a13400640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59f406b4d0 0x7f59f406d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5a04034000 tx=0x7f5a0402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:36.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.203+0000 7f5a0b400640 1 -- 172.21.15.62:0/1639039632 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a140b3f90 con 0x7f5a141655a0 2024-04-25T09:17:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.478+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5a14075070 con 0x7f5a141655a0 2024-04-25T09:17:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.483+0000 7f5a0b400640 1 -- 172.21.15.62:0/1639039632 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5a0001f020 con 0x7f5a141655a0 2024-04-25T09:17:36.470 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-25T09:17:36.470 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUHCpm9JCCJRAAEncj4mpVuwF3lDed2PVa+w== 2024-04-25T09:17:36.470 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-25T09:17:36.470 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-25T09:17:36.470 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:17:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59f406b4d0 msgr2=0x7f59f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59f406b4d0 0x7f59f406d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5a04034000 tx=0x7f5a0402e000 comp rx=0 tx=0).stop 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 msgr2=0x7f5a1415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a1415d320 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000b3e0 tx=0x7f5a0000b8a0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 shutdown_connections 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f59f406b4d0 0x7f59f406d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5a141655a0 0x7f5a1415d320 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a14165980 0x7f5a14159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 --2- 172.21.15.62:0/1639039632 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f5a14071370 0x7f5a1414a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 >> 172.21.15.62:0/1639039632 conn(0x7f5a140668f0 msgr2=0x7f5a140cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:36.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 shutdown_connections 2024-04-25T09:17:36.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.486+0000 7f5a19000640 1 -- 172.21.15.62:0/1639039632 wait complete. 2024-04-25T09:17:36.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:17:36.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.713+0000 7f2247c00640 1 Processor -- start 2024-04-25T09:17:36.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.713+0000 7f2247c00640 1 -- start start 2024-04-25T09:17:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f2247c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2240072340 0x7f22400702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f2247c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f22400707f0 0x7f22400b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f2247c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224006c310 con 0x7f22400cfed0 2024-04-25T09:17:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f2247c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224006c5b0 con 0x7f22400707f0 2024-04-25T09:17:36.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f2247c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224006c890 con 0x7f2240072340 2024-04-25T09:17:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f2245c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f22400707f0 0x7f22400b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f2245c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f22400707f0 0x7f22400b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38316/0 (socket says 172.21.15.62:38316) 2024-04-25T09:17:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f2245c00640 1 -- 172.21.15.62:0/25653047 learned_addr learned my addr 172.21.15.62:0/25653047 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.715+0000 7f223fe00640 1 --2- 172.21.15.62:0/25653047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2240072340 0x7f22400702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.716+0000 7f2245c00640 1 -- 172.21.15.62:0/25653047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2240072340 msgr2=0x7f22400702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:36.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.716+0000 7f2245c00640 1 --2- 172.21.15.62:0/25653047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2240072340 0x7f22400702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.716+0000 7f2245c00640 1 -- 172.21.15.62:0/25653047 >> v1:172.21.15.62:6789/0 conn(0x7f22400cfed0 legacy=0x7f224015f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.716+0000 7f2245c00640 1 -- 172.21.15.62:0/25653047 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224006cb70 con 0x7f22400707f0 2024-04-25T09:17:36.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.717+0000 7f2245c00640 1 --2- 172.21.15.62:0/25653047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400707f0 0x7f22400b2880 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2238009e50 tx=0x7f223802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fc5e76b54c97c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.717+0000 7f223f400640 1 -- 172.21.15.62:0/25653047 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238036070 con 0x7f22400707f0 2024-04-25T09:17:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.717+0000 7f223f400640 1 -- 172.21.15.62:0/25653047 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2238031600 con 0x7f22400707f0 2024-04-25T09:17:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.717+0000 7f223f400640 1 -- 172.21.15.62:0/25653047 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22380319d0 con 0x7f22400707f0 2024-04-25T09:17:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.718+0000 7f2247c00640 1 -- 172.21.15.62:0/25653047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400707f0 msgr2=0x7f22400b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.718+0000 7f2247c00640 1 --2- 172.21.15.62:0/25653047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400707f0 0x7f22400b2880 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f2238009e50 tx=0x7f223802a710 comp rx=0 tx=0).stop 2024-04-25T09:17:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.718+0000 7f2247c00640 1 -- 172.21.15.62:0/25653047 shutdown_connections 2024-04-25T09:17:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.718+0000 7f2247c00640 1 --2- 172.21.15.62:0/25653047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400707f0 0x7f22400b2880 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.718+0000 7f2247c00640 1 --2- 172.21.15.62:0/25653047 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f2240072340 0x7f22400702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.718+0000 7f2247c00640 1 -- 172.21.15.62:0/25653047 >> 172.21.15.62:0/25653047 conn(0x7f22400668f0 msgr2=0x7f2240066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.718+0000 7f2247c00640 1 -- 172.21.15.62:0/25653047 shutdown_connections 2024-04-25T09:17:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.718+0000 7f2247c00640 1 -- 172.21.15.62:0/25653047 wait complete. 2024-04-25T09:17:36.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.720+0000 7f2247c00640 1 Processor -- start 2024-04-25T09:17:36.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.720+0000 7f2247c00640 1 -- start start 2024-04-25T09:17:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f2247c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22400707f0 0x7f2240140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f2247c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400cfed0 0x7f224015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f2247c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22400ad8e0 0x7f22400add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f2247c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400b2dc0 con 0x7f22400cfed0 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f2247c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224006d130 con 0x7f22400ad8e0 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f2247c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22400b2f40 con 0x7f22400707f0 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f223fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400cfed0 0x7f224015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f223fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400cfed0 0x7f224015d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38332/0 (socket says 172.21.15.62:38332) 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f223fe00640 1 -- 172.21.15.62:0/714056979 learned_addr learned my addr 172.21.15.62:0/714056979 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f2245c00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22400ad8e0 0x7f22400add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f2245200640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22400707f0 0x7f2240140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.721+0000 7f223fe00640 1 -- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22400707f0 msgr2=0x7f2240140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f223fe00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22400707f0 0x7f2240140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f223fe00640 1 -- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22400ad8e0 msgr2=0x7f22400add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f223fe00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22400ad8e0 0x7f22400add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f223fe00640 1 -- 172.21.15.62:0/714056979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22400ce640 con 0x7f22400cfed0 2024-04-25T09:17:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f2245c00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22400ad8e0 0x7f22400add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f2245200640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22400707f0 0x7f2240140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f223fe00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400cfed0 0x7f224015d5f0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f222c00d830 tx=0x7f222c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f223cc00640 1 -- 172.21.15.62:0/714056979 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00b6c0 con 0x7f22400cfed0 2024-04-25T09:17:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f223cc00640 1 -- 172.21.15.62:0/714056979 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c019070 con 0x7f22400cfed0 2024-04-25T09:17:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f223cc00640 1 -- 172.21.15.62:0/714056979 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00ba80 con 0x7f22400cfed0 2024-04-25T09:17:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224013f690 con 0x7f22400cfed0 2024-04-25T09:17:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.722+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224013f2b0 con 0x7f22400cfed0 2024-04-25T09:17:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.724+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2240072c70 con 0x7f22400cfed0 2024-04-25T09:17:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.724+0000 7f223cc00640 1 -- 172.21.15.62:0/714056979 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f222c01e460 con 0x7f22400cfed0 2024-04-25T09:17:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.725+0000 7f223cc00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f222006b500 0x7f222006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.725+0000 7f223cc00640 1 -- 172.21.15.62:0/714056979 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f222c08a8e0 con 0x7f22400cfed0 2024-04-25T09:17:36.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.726+0000 7f2245200640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f222006b500 0x7f222006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:36.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.727+0000 7f2245200640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f222006b500 0x7f222006d9c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f22340015e0 tx=0x7f2234009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.730+0000 7f223cc00640 1 -- 172.21.15.62:0/714056979 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f222c059740 con 0x7f22400cfed0 2024-04-25T09:17:36.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.963+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f224004f000 con 0x7f22400cfed0 2024-04-25T09:17:36.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.964+0000 7f223cc00640 1 -- 172.21.15.62:0/714056979 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f222c059040 con 0x7f22400cfed0 2024-04-25T09:17:36.951 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:17:36.951 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":99,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3369583759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3369583759},{"type":"v1","addr":"172.21.15.121:6837","nonce":3369583759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1888749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1888749},{"type":"v1","addr":"172.21.15.121:6836","nonce":1888749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":9390,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1673530113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1673530113},{"type":"v1","addr":"172.21.15.62:6833","nonce":1673530113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:17:30.382991+0000","modified":"2024-04-25T09:17:33.474949+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9396":{"gid":9396,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2793862963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2793862963},{"type":"v1","addr":"172.21.15.62:6835","nonce":2793862963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:17:36.952 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 99 2024-04-25T09:17:36.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.966+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f222006b500 msgr2=0x7f222006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.966+0000 7f2247c00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f222006b500 0x7f222006d9c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f22340015e0 tx=0x7f2234009040 comp rx=0 tx=0).stop 2024-04-25T09:17:36.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.966+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400cfed0 msgr2=0x7f224015d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:36.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.966+0000 7f2247c00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400cfed0 0x7f224015d5f0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f222c00d830 tx=0x7f222c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.967+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 shutdown_connections 2024-04-25T09:17:36.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.967+0000 7f2247c00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f222006b500 0x7f222006d9c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.967+0000 7f2247c00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22400cfed0 0x7f224015d5f0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.967+0000 7f2247c00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22400ad8e0 0x7f22400add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.967+0000 7f2247c00640 1 --2- 172.21.15.62:0/714056979 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f22400707f0 0x7f2240140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:36.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.967+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 >> 172.21.15.62:0/714056979 conn(0x7f22400668f0 msgr2=0x7f22400b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:36.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.967+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 shutdown_connections 2024-04-25T09:17:36.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:36.967+0000 7f2247c00640 1 -- 172.21.15.62:0/714056979 wait complete. 2024-04-25T09:17:37.376 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:17:30.382991+0000', 'modified': '2024-04-25T09:17:33.474949+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9396}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9396': {'gid': 9396, 'name': 'a', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/2793862963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2793862963}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2793862963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:17:37.376 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:17:37.377 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:17:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.614+0000 7f10b5200640 1 Processor -- start 2024-04-25T09:17:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.614+0000 7f10b5200640 1 -- start start 2024-04-25T09:17:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.616+0000 7f10b5200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 0x7f10b00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.616+0000 7f10b5200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b00cff60 con 0x7f10b00cfa70 2024-04-25T09:17:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.616+0000 7f10b5200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b0070360 con 0x7f10b015c4e0 2024-04-25T09:17:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.616+0000 7f10b5200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0070640 con 0x7f10b00cf5f0 2024-04-25T09:17:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.616+0000 7f10afe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f10b00cfa70 0x7f10b00b2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45044/0 (socket says 172.21.15.62:45044) 2024-04-25T09:17:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.616+0000 7f10afe00640 1 -- 172.21.15.62:0/1990739036 learned_addr learned my addr 172.21.15.62:0/1990739036 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:17:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10aea00640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 0x7f10b00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051396187 0 0) 0x7f10b00cff60 con 0x7f10b00cfa70 2024-04-25T09:17:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1094003520 con 0x7f10b00cfa70 2024-04-25T09:17:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669588900 0 0) 0x7f1094003520 con 0x7f10b00cfa70 2024-04-25T09:17:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 msgr2=0x7f10b00b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 0x7f10b00b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 >> v1:172.21.15.121:6789/0 conn(0x7f10b015c4e0 legacy=0x7f10b015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0070920 con 0x7f10b00cfa70 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f10a8001670 con 0x7f10b00cfa70 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 >> v1:172.21.15.62:6789/0 conn(0x7f10b00cfa70 legacy=0x7f10b00b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f1094003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1094004800 0x7f1094004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 0x7f1094005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0070640 con 0x7f10940039e0 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109400baf0 con 0x7f1094004800 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.617+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109400bcf0 con 0x7f10b00cf5f0 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10af400640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f1094003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10aea00640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1094004800 0x7f1094004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10af400640 1 -- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 msgr2=0x7f1094005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10afe00640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 0x7f1094005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10af400640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 0x7f1094005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10af400640 1 -- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1094004800 msgr2=0x7f1094004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10af400640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1094004800 0x7f1094004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10af400640 1 -- 172.21.15.62:0/1990739036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1094003520 con 0x7f10940039e0 2024-04-25T09:17:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10aea00640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1094004800 0x7f1094004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:17:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.618+0000 7f10afe00640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00cf5f0 0x7f1094005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.619+0000 7f10af400640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f1094003e00 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f10a0001e50 tx=0x7f1094005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad02a8e65f3e7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.619+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0001a70 con 0x7f10940039e0 2024-04-25T09:17:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a0003970 con 0x7f10940039e0 2024-04-25T09:17:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10ae000640 1 -- 172.21.15.62:0/1990739036 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a002ae60 con 0x7f10940039e0 2024-04-25T09:17:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10b5200640 1 -- 172.21.15.62:0/1990739036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 msgr2=0x7f1094003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10b5200640 1 --2- 172.21.15.62:0/1990739036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f1094003e00 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f10a0001e50 tx=0x7f1094005830 comp rx=0 tx=0).stop 2024-04-25T09:17:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10afe00640 1 -- 172.21.15.62:0/1990739036 reap_dead start 2024-04-25T09:17:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10b5200640 1 -- 172.21.15.62:0/1990739036 shutdown_connections 2024-04-25T09:17:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10b5200640 1 -- 172.21.15.62:0/1990739036 >> 172.21.15.62:0/1990739036 conn(0x7f10b00668f0 msgr2=0x7f10b0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10b5200640 1 -- 172.21.15.62:0/1990739036 shutdown_connections 2024-04-25T09:17:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.620+0000 7f10b5200640 1 -- 172.21.15.62:0/1990739036 wait complete. 2024-04-25T09:17:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.622+0000 7f10b5200640 1 Processor -- start 2024-04-25T09:17:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.622+0000 7f10b5200640 1 -- start start 2024-04-25T09:17:37.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10b5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f10b01f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10b5200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b01f0b80 0x7f10b00b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10b5200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00b1d30 0x7f10b01538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10b5200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00b2e20 con 0x7f10940039e0 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10b5200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0071120 con 0x7f10b01f0b80 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10b5200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00b2fa0 con 0x7f10b00b1d30 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f10b01f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f10b01f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38362/0 (socket says 172.21.15.62:38362) 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.623+0000 7f10af400640 1 -- 172.21.15.62:0/792271185 learned_addr learned my addr 172.21.15.62:0/792271185 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10afe00640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00b1d30 0x7f10b01538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10aea00640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b01f0b80 0x7f10b00b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10af400640 1 -- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00b1d30 msgr2=0x7f10b01538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10af400640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00b1d30 0x7f10b01538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10af400640 1 -- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b01f0b80 msgr2=0x7f10b00b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10af400640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b01f0b80 0x7f10b00b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10af400640 1 -- 172.21.15.62:0/792271185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00af630 con 0x7f10940039e0 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10aea00640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b01f0b80 0x7f10b00b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:37.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10af400640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f10b01f4290 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f10a002a910 tx=0x7f10a00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10a7400640 1 -- 172.21.15.62:0/792271185 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a003f030 con 0x7f10940039e0 2024-04-25T09:17:37.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10a7400640 1 -- 172.21.15.62:0/792271185 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a00050d0 con 0x7f10940039e0 2024-04-25T09:17:37.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10a7400640 1 -- 172.21.15.62:0/792271185 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0037040 con 0x7f10940039e0 2024-04-25T09:17:37.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.624+0000 7f10b5200640 1 -- 172.21.15.62:0/792271185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b00af910 con 0x7f10940039e0 2024-04-25T09:17:37.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.625+0000 7f10b5200640 1 -- 172.21.15.62:0/792271185 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b00afed0 con 0x7f10940039e0 2024-04-25T09:17:37.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.627+0000 7f10a7400640 1 -- 172.21.15.62:0/792271185 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10a0042070 con 0x7f10940039e0 2024-04-25T09:17:37.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.628+0000 7f10a7400640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f108c06b500 0x7f108c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.628+0000 7f10a7400640 1 -- 172.21.15.62:0/792271185 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10a00a9fa0 con 0x7f10940039e0 2024-04-25T09:17:37.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.628+0000 7f10aea00640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f108c06b500 0x7f108c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.628+0000 7f10b977c740 1 Processor -- start 2024-04-25T09:17:37.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.629+0000 7f10b977c740 1 -- start start 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.629+0000 7f10aea00640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f108c06b500 0x7f108c06d9c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f10b01565b0 tx=0x7f1098008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.629+0000 7f10b977c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea86ddb30 0x55fea8705970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.629+0000 7f10b977c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea86ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10b977c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 0x55fea86dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10b977c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea8641620 con 0x55fea8705eb0 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10b977c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea85eaca0 con 0x55fea86ddb30 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10b977c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fea863d6c0 con 0x55fea87267a0 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10aea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea86ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10aea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea86ddf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38364/0 (socket says 172.21.15.62:38364) 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10aea00640 1 -- 172.21.15.62:0/1264183965 learned_addr learned my addr 172.21.15.62:0/1264183965 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10af400640 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea86ddb30 0x55fea8705970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10aea00640 1 -- 172.21.15.62:0/1264183965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 msgr2=0x55fea86dc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10aea00640 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 0x55fea86dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10aea00640 1 -- 172.21.15.62:0/1264183965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea86ddb30 msgr2=0x55fea8705970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10aea00640 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea86ddb30 0x55fea8705970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10aea00640 1 -- 172.21.15.62:0/1264183965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fea85f1f40 con 0x55fea8705eb0 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.630+0000 7f10af400640 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea86ddb30 0x55fea8705970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.631+0000 7f10aea00640 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea86ddf10 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f10980043f0 tx=0x7f1098015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff87a57046f9d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.631+0000 7f109fe00640 1 -- 172.21.15.62:0/1264183965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098028070 con 0x55fea8705eb0 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.631+0000 7f109fe00640 1 -- 172.21.15.62:0/1264183965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109801e050 con 0x55fea8705eb0 2024-04-25T09:17:37.619 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.631+0000 7f109fe00640 1 -- 172.21.15.62:0/1264183965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098015450 con 0x55fea8705eb0 2024-04-25T09:17:37.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.631+0000 7f10b977c740 1 -- 172.21.15.62:0/1264183965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 msgr2=0x55fea86ddf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.631+0000 7f10b977c740 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea86ddf10 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f10980043f0 tx=0x7f1098015000 comp rx=0 tx=0).stop 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.632+0000 7f10b977c740 1 -- 172.21.15.62:0/1264183965 shutdown_connections 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.632+0000 7f10b977c740 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea86ddf10 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.632+0000 7f10b977c740 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 0x55fea86dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.632+0000 7f10b977c740 1 --2- 172.21.15.62:0/1264183965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea86ddb30 0x55fea8705970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.632+0000 7f10b977c740 1 -- 172.21.15.62:0/1264183965 >> 172.21.15.62:0/1264183965 conn(0x55fea86dad50 msgr2=0x55fea8723730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.632+0000 7f10b977c740 1 -- 172.21.15.62:0/1264183965 shutdown_connections 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.632+0000 7f10b977c740 1 -- 172.21.15.62:0/1264183965 wait complete. 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.632+0000 7f10b977c740 1 -- start start 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.633+0000 7f10b977c740 1 Processor -- start 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.633+0000 7f10b977c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea8733980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.633+0000 7f10b977c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 0x55fea8722dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.633+0000 7f10b977c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea870bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.633+0000 7f10b977c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea87054e0 con 0x55fea8705eb0 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.633+0000 7f10b977c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea85e01f0 con 0x55fea8703f00 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.633+0000 7f10b977c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fea8704ed0 con 0x55fea87267a0 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10b977c740 10 client.0 _ll_register_callbacks cb 0x55fea85fdad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea8733980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea8733980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38366/0 (socket says 172.21.15.62:38366) 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10af400640 1 -- 172.21.15.62:0/1308129149 learned_addr learned my addr 172.21.15.62:0/1308129149 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10afe00640 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea870bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10aea00640 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 0x55fea8722dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10af400640 1 -- 172.21.15.62:0/1308129149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 msgr2=0x55fea8722dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10af400640 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 0x55fea8722dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10af400640 1 -- 172.21.15.62:0/1308129149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 msgr2=0x55fea870bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10af400640 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea870bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.634+0000 7f10af400640 1 -- 172.21.15.62:0/1308129149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fea85ffc50 con 0x55fea8705eb0 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f10af400640 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea8733980 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f10a00808e0 tx=0x7f10a00788f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4972f9f71161f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f109cc00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f109cc00640 1 -- 172.21.15.62:0/1308129149 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0076b70 con 0x55fea8705eb0 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f109cc00640 1 -- 172.21.15.62:0/1308129149 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10a00054a0 con 0x55fea8705eb0 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f10b977c740 1 -- 172.21.15.62:0/1308129149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f109801e050 con 0x55fea8705eb0 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f109cc00640 1 -- 172.21.15.62:0/1308129149 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10a00056a0 con 0x55fea8705eb0 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f10b977c740 10 client.9525 fetch_fsmap learned FSMap version 99 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f10b977c740 1 -- 172.21.15.62:0/1308129149 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55fea83562d0 con 0x55fea8705eb0 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f109cc00640 1 -- 172.21.15.62:0/1308129149 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109801e050 con 0x55fea8705eb0 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.635+0000 7f109cc00640 1 -- 172.21.15.62:0/1308129149 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10a004c060 con 0x55fea8705eb0 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f10b977c740 10 client.9525 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f10b977c740 10 client.9525 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f10b977c740 20 client.9525 populate_metadata read hostname 'smithi062' 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f10b977c740 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x55fea8708e70 0x55fea87098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f10b977c740 4 client.9525 mds_command: new command op to 9396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f10b977c740 1 -- 172.21.15.62:0/1308129149 --> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fea864eb60 con 0x55fea8708e70 2024-04-25T09:17:37.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f10aea00640 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x55fea8708e70 0x55fea87098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f10aea00640 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x55fea8708e70 0x55fea87098e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b669600e269ff7fe server_cookie=167aad93c649417e in_seq=0 out_seq=0 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.636+0000 7f109cc00640 10 client.9525 ms_handle_connect on v2:172.21.15.62:6834/2793862963 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.637+0000 7f109cc00640 1 -- 172.21.15.62:0/1308129149 <== mds.0 v2:172.21.15.62:6834/2793862963 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fea864eb60 con 0x55fea8708e70 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.637+0000 7f109cc00640 10 client.9525 handle_command_reply: tid=0 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.637+0000 7f10b977c740 1 client.9525 shutdown 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 -- 172.21.15.62:0/1308129149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 msgr2=0x55fea8733980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea8733980 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f10a00808e0 tx=0x7f10a00788f0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 -- 172.21.15.62:0/1308129149 shutdown_connections 2024-04-25T09:17:37.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x55fea8708e70 0x55fea87098e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea870bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea87267a0 0x55fea8722dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 --2- 172.21.15.62:0/1308129149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8705eb0 0x55fea8733980 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f109cc00640 0 client.9525 ms_handle_reset on v2:172.21.15.62:6834/2793862963 2024-04-25T09:17:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 -- 172.21.15.62:0/1308129149 >> 172.21.15.62:0/1308129149 conn(0x55fea86ddb30 msgr2=0x55fea8726c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 -- 172.21.15.62:0/1308129149 shutdown_connections 2024-04-25T09:17:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 1 -- 172.21.15.62:0/1308129149 wait complete. 2024-04-25T09:17:37.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.638+0000 7f10b977c740 20 client.9525 trim_cache size 0 max 16384 2024-04-25T09:17:37.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.655+0000 7f10b977c740 1 Processor -- start 2024-04-25T09:17:37.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.656+0000 7f10b977c740 1 -- start start 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.656+0000 7f10b977c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8703f00 0x55fea8723d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.656+0000 7f10b977c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 0x55fea870bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.656+0000 7f10b977c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea87267a0 0x55fea870c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.656+0000 7f10b977c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01ff900 con 0x55fea8703f00 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.656+0000 7f10b977c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01f7c40 con 0x55fea87267a0 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.656+0000 7f10b977c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01ff300 con 0x55fea8705eb0 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8703f00 0x55fea8723d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10af400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8703f00 0x55fea8723d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38376/0 (socket says 172.21.15.62:38376) 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10af400640 1 -- 172.21.15.62:0/2748758546 learned_addr learned my addr 172.21.15.62:0/2748758546 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10aea00640 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 0x55fea870bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10af400640 1 -- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 msgr2=0x55fea870bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10afe00640 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea87267a0 0x55fea870c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10af400640 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 0x55fea870bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10af400640 1 -- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea87267a0 msgr2=0x55fea870c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10af400640 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea87267a0 0x55fea870c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10af400640 1 -- 172.21.15.62:0/2748758546 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fea84206b0 con 0x55fea8703f00 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.657+0000 7f10afe00640 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea87267a0 0x55fea870c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f10af400640 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8703f00 0x55fea8723d90 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f10a0078740 tx=0x7f10a0034050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4675af0b1698890d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f109f400640 1 -- 172.21.15.62:0/2748758546 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0076b70 con 0x55fea8703f00 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f109f400640 1 -- 172.21.15.62:0/2748758546 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a00a80a0 con 0x55fea8703f00 2024-04-25T09:17:37.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f109f400640 1 -- 172.21.15.62:0/2748758546 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0077c00 con 0x55fea8703f00 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f10b977c740 1 -- 172.21.15.62:0/2748758546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8703f00 msgr2=0x55fea8723d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f10b977c740 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8703f00 0x55fea8723d90 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f10a0078740 tx=0x7f10a0034050 comp rx=0 tx=0).stop 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f10b977c740 1 -- 172.21.15.62:0/2748758546 shutdown_connections 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f10b977c740 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea87267a0 0x55fea870c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f10b977c740 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 0x55fea870bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f10b977c740 1 --2- 172.21.15.62:0/2748758546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea8703f00 0x55fea8723d90 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.658+0000 7f10b977c740 1 -- 172.21.15.62:0/2748758546 >> 172.21.15.62:0/2748758546 conn(0x55fea86ddb30 msgr2=0x55fea8722bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.659+0000 7f10b977c740 1 -- 172.21.15.62:0/2748758546 shutdown_connections 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.659+0000 7f10b977c740 1 -- 172.21.15.62:0/2748758546 wait complete. 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.659+0000 7f10b977c740 1 -- start start 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.659+0000 7f10b977c740 1 Processor -- start 2024-04-25T09:17:37.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.660+0000 7f10b977c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea86d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.660+0000 7f10b977c740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 0x55fea877fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.660+0000 7f10b977c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea87267a0 0x55fea870a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.660+0000 7f10b977c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea87054e0 con 0x55fea87267a0 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.660+0000 7f10b977c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fea8704ed0 con 0x55fea8703f00 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.660+0000 7f10b977c740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fea85e01f0 con 0x55fea8705eb0 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.660+0000 7f10b977c740 10 client.0 _ll_register_callbacks cb 0x7f10a0074a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea87267a0 0x55fea870a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea87267a0 0x55fea870a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38380/0 (socket says 172.21.15.62:38380) 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 -- 172.21.15.62:0/3302483460 learned_addr learned my addr 172.21.15.62:0/3302483460 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10aea00640 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 0x55fea877fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10af400640 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea86d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 -- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 msgr2=0x55fea877fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 0x55fea877fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 -- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 msgr2=0x55fea86d3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea86d3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 -- 172.21.15.62:0/3302483460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fea8709420 con 0x55fea87267a0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10af400640 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea86d3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f10afe00640 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea87267a0 0x55fea870a1e0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f10b00cf540 tx=0x7f10a80106d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a919c76f92846f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.661+0000 7f109cc00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f109cc00640 1 -- 172.21.15.62:0/3302483460 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8014460 con 0x55fea87267a0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f109cc00640 1 -- 172.21.15.62:0/3302483460 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10a8014600 con 0x55fea87267a0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 1 -- 172.21.15.62:0/3302483460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10a00a80a0 con 0x55fea87267a0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f109cc00640 1 -- 172.21.15.62:0/3302483460 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10a8014800 con 0x55fea87267a0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 10 client.9531 fetch_fsmap learned FSMap version 99 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 1 -- 172.21.15.62:0/3302483460 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55fea870a7b0 con 0x55fea87267a0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f109cc00640 1 -- 172.21.15.62:0/3302483460 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a00a80a0 con 0x55fea87267a0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f109cc00640 1 -- 172.21.15.62:0/3302483460 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10a80149a0 con 0x55fea87267a0 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 10 client.9531 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T09:17:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 10 client.9531 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:17:37.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 20 client.9531 populate_metadata read hostname 'smithi062' 2024-04-25T09:17:37.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x55fea86d7f20 0x55fea86dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 4 client.9531 mds_command: new command op to 9396 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:17:37.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.662+0000 7f10b977c740 1 -- 172.21.15.62:0/3302483460 --> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fea8648eb0 con 0x55fea86d7f20 2024-04-25T09:17:37.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.663+0000 7f10af400640 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x55fea86d7f20 0x55fea86dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.663+0000 7f10af400640 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x55fea86d7f20 0x55fea86dc2f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e67caec870ae59 server_cookie=90c0acca701ff20f in_seq=0 out_seq=0 2024-04-25T09:17:37.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.663+0000 7f109cc00640 10 client.9531 ms_handle_connect on v2:172.21.15.62:6834/2793862963 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.663+0000 7f109cc00640 1 -- 172.21.15.62:0/3302483460 <== mds.0 v2:172.21.15.62:6834/2793862963 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55fea8648eb0 con 0x55fea86d7f20 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.663+0000 7f109cc00640 10 client.9531 handle_command_reply: tid=0 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.663+0000 7f10b977c740 1 client.9531 shutdown 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.663+0000 7f10b977c740 1 -- 172.21.15.62:0/3302483460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea87267a0 msgr2=0x55fea870a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.663+0000 7f10b977c740 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea87267a0 0x55fea870a1e0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f10b00cf540 tx=0x7f10a80106d0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 1 -- 172.21.15.62:0/3302483460 shutdown_connections 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x55fea86d7f20 0x55fea86dc2f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fea87267a0 0x55fea870a1e0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x55fea8705eb0 0x55fea877fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 1 --2- 172.21.15.62:0/3302483460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fea8703f00 0x55fea86d3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f109cc00640 0 client.9531 ms_handle_reset on v2:172.21.15.62:6834/2793862963 2024-04-25T09:17:37.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 1 -- 172.21.15.62:0/3302483460 >> 172.21.15.62:0/3302483460 conn(0x55fea86ddb30 msgr2=0x55fea870bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:37.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 1 -- 172.21.15.62:0/3302483460 shutdown_connections 2024-04-25T09:17:37.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 1 -- 172.21.15.62:0/3302483460 wait complete. 2024-04-25T09:17:37.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b977c740 20 client.9531 trim_cache size 0 max 16384 2024-04-25T09:17:37.657 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:17:37.657 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 9396, 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 6.7145004229999996, 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 99, 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 148, 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 148, 2024-04-25T09:17:37.658 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 12.152517008 2024-04-25T09:17:37.659 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:17:37.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b5200640 1 -- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f108c06b500 msgr2=0x7f108c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b5200640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f108c06b500 0x7f108c06d9c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f10b01565b0 tx=0x7f1098008040 comp rx=0 tx=0).stop 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.664+0000 7f10b5200640 1 -- 172.21.15.62:0/792271185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 msgr2=0x7f10b01f4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f10b01f4290 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f10a002a910 tx=0x7f10a00014a0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 -- 172.21.15.62:0/792271185 shutdown_connections 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f108c06b500 0x7f108c06d9c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f10b00b1d30 0x7f10b01538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10b01f0b80 0x7f10b00b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 --2- 172.21.15.62:0/792271185 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10940039e0 0x7f10b01f4290 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 -- 172.21.15.62:0/792271185 >> 172.21.15.62:0/792271185 conn(0x7f10b00668f0 msgr2=0x7f10b015dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 -- 172.21.15.62:0/792271185 shutdown_connections 2024-04-25T09:17:37.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.665+0000 7f10b5200640 1 -- 172.21.15.62:0/792271185 wait complete. 2024-04-25T09:17:37.668 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:17:37.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.872+0000 7efe40e00640 1 Processor -- start 2024-04-25T09:17:37.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.872+0000 7efe40e00640 1 -- start start 2024-04-25T09:17:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.874+0000 7efe40e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efe3c144c10 0x7efe3c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.874+0000 7efe40e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c06d900 0x7efe3c0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.874+0000 7efe40e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c0cf2d0 con 0x7efe3c0cee40 2024-04-25T09:17:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.874+0000 7efe40e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c144ff0 con 0x7efe3c144c10 2024-04-25T09:17:37.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.874+0000 7efe40e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c160340 con 0x7efe3c06d900 2024-04-25T09:17:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.874+0000 7efe3b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efe3c144c10 0x7efe3c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.874+0000 7efe3b400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efe3c144c10 0x7efe3c15f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38394/0 (socket says 172.21.15.62:38394) 2024-04-25T09:17:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.874+0000 7efe3b400640 1 -- 172.21.15.62:0/4249856869 learned_addr learned my addr 172.21.15.62:0/4249856869 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:37.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.875+0000 7efe3be00640 1 --2- 172.21.15.62:0/4249856869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c06d900 0x7efe3c0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.875+0000 7efe3b400640 1 -- 172.21.15.62:0/4249856869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c06d900 msgr2=0x7efe3c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:17:37.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.875+0000 7efe3b400640 1 --2- 172.21.15.62:0/4249856869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c06d900 0x7efe3c0b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.875+0000 7efe3b400640 1 -- 172.21.15.62:0/4249856869 >> v1:172.21.15.121:6789/0 conn(0x7efe3c0cee40 legacy=0x7efe3c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.875+0000 7efe3b400640 1 -- 172.21.15.62:0/4249856869 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c15faa0 con 0x7efe3c144c10 2024-04-25T09:17:37.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.875+0000 7efe3be00640 1 --2- 172.21.15.62:0/4249856869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c06d900 0x7efe3c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:17:37.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.877+0000 7efe3b400640 1 --2- 172.21.15.62:0/4249856869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c15f450 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009870 tx=0x7efe2c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64420e22c888330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.877+0000 7efe3a000640 1 -- 172.21.15.62:0/4249856869 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c037070 con 0x7efe3c144c10 2024-04-25T09:17:37.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.877+0000 7efe3a000640 1 -- 172.21.15.62:0/4249856869 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c032440 con 0x7efe3c144c10 2024-04-25T09:17:37.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.877+0000 7efe3a000640 1 -- 172.21.15.62:0/4249856869 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c032810 con 0x7efe3c144c10 2024-04-25T09:17:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.878+0000 7efe40e00640 1 -- 172.21.15.62:0/4249856869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 msgr2=0x7efe3c15f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.878+0000 7efe40e00640 1 --2- 172.21.15.62:0/4249856869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c15f450 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009870 tx=0x7efe2c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.878+0000 7efe40e00640 1 -- 172.21.15.62:0/4249856869 shutdown_connections 2024-04-25T09:17:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.878+0000 7efe40e00640 1 --2- 172.21.15.62:0/4249856869 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c06d900 0x7efe3c0b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.878+0000 7efe40e00640 1 --2- 172.21.15.62:0/4249856869 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c15f450 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.878+0000 7efe40e00640 1 -- 172.21.15.62:0/4249856869 >> 172.21.15.62:0/4249856869 conn(0x7efe3c0b14d0 msgr2=0x7efe3c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.878+0000 7efe40e00640 1 -- 172.21.15.62:0/4249856869 shutdown_connections 2024-04-25T09:17:37.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.878+0000 7efe40e00640 1 -- 172.21.15.62:0/4249856869 wait complete. 2024-04-25T09:17:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.880+0000 7efe40e00640 1 Processor -- start 2024-04-25T09:17:37.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.880+0000 7efe40e00640 1 -- start start 2024-04-25T09:17:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe40e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe3c06d900 0x7efe3c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe40e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c0cee40 0x7efe3c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe40e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe40e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c160060 con 0x7efe3c144c10 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe40e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0b41c0 con 0x7efe3c06d900 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe40e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0b4b90 con 0x7efe3c0cee40 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe3be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe3be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38404/0 (socket says 172.21.15.62:38404) 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe3be00640 1 -- 172.21.15.62:0/3685632243 learned_addr learned my addr 172.21.15.62:0/3685632243 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.881+0000 7efe3b400640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe3c06d900 0x7efe3c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe3aa00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c0cee40 0x7efe3c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe3be00640 1 -- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c0cee40 msgr2=0x7efe3c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe3be00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c0cee40 0x7efe3c0ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe3be00640 1 -- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe3c06d900 msgr2=0x7efe3c0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe3be00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe3c06d900 0x7efe3c0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe3be00640 1 -- 172.21.15.62:0/3685632243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0cb830 con 0x7efe3c144c10 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe3be00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c0cd310 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7efe34007ef0 tx=0x7efe3400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe33400640 1 -- 172.21.15.62:0/3685632243 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3401a070 con 0x7efe3c144c10 2024-04-25T09:17:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe33400640 1 -- 172.21.15.62:0/3685632243 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe34003b90 con 0x7efe3c144c10 2024-04-25T09:17:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe33400640 1 -- 172.21.15.62:0/3685632243 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3400de80 con 0x7efe3c144c10 2024-04-25T09:17:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.882+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c0cb550 con 0x7efe3c144c10 2024-04-25T09:17:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.883+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c141090 con 0x7efe3c144c10 2024-04-25T09:17:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.884+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c04ef20 con 0x7efe3c144c10 2024-04-25T09:17:37.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.885+0000 7efe33400640 1 -- 172.21.15.62:0/3685632243 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe3401d070 con 0x7efe3c144c10 2024-04-25T09:17:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.886+0000 7efe33400640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe1c06b4d0 0x7efe1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:17:37.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.886+0000 7efe33400640 1 -- 172.21.15.62:0/3685632243 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe3408a350 con 0x7efe3c144c10 2024-04-25T09:17:37.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.886+0000 7efe3b400640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe1c06b4d0 0x7efe1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:17:37.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.887+0000 7efe3b400640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe1c06b4d0 0x7efe1c06d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efe2c034000 tx=0x7efe2c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:17:37.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:37.890+0000 7efe33400640 1 -- 172.21.15.62:0/3685632243 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe34010070 con 0x7efe3c144c10 2024-04-25T09:17:38.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.135+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efe3c0cec20 con 0x7efe3c144c10 2024-04-25T09:17:38.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.136+0000 7efe33400640 1 -- 172.21.15.62:0/3685632243 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7efe34058b70 con 0x7efe3c144c10 2024-04-25T09:17:38.123 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:17:38.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe1c06b4d0 msgr2=0x7efe1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:38.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe1c06b4d0 0x7efe1c06d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efe2c034000 tx=0x7efe2c02e000 comp rx=0 tx=0).stop 2024-04-25T09:17:38.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 msgr2=0x7efe3c0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:17:38.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c0cd310 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7efe34007ef0 tx=0x7efe3400d4a0 comp rx=0 tx=0).stop 2024-04-25T09:17:38.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 shutdown_connections 2024-04-25T09:17:38.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efe1c06b4d0 0x7efe1c06d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:38.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe3c144c10 0x7efe3c0cd310 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:38.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efe3c0cee40 0x7efe3c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:38.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 --2- 172.21.15.62:0/3685632243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe3c06d900 0x7efe3c0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:17:38.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 >> 172.21.15.62:0/3685632243 conn(0x7efe3c0b14d0 msgr2=0x7efe3c15c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:17:38.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.139+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 shutdown_connections 2024-04-25T09:17:38.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:17:38.140+0000 7efe40e00640 1 -- 172.21.15.62:0/3685632243 wait complete. 2024-04-25T09:17:38.145 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:17:38.145 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:17:38.145 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-25T09:17:38.145 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:17:38.145 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:17:38.146 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:17:38.146 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:17:38.146 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:17:38.146 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:17:38.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.146 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-25T09:17:38.198 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:17:38.198 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:17:38.198 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:17:38.199 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:17:38.199 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-25T09:17:38.199 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:17:38.199 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:17:38.199 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:17:38.200 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-25T09:17:38.200 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:17:38.200 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:17:38.200 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-25T09:17:38.200 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:17:38.201 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:17:38.201 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:17:38.201 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:17:38.201 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T09:17:38.201 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 41801sec preferred_lft 41801sec 2024-04-25T09:17:38.201 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::ec4:7aff:febb:d1c3/64 scope link 2024-04-25T09:17:38.202 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:17:38.202 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:17:38.202 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:17:38.202 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:17:38.202 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:17:38.203 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6ce7:52ff:fe81:105e/64 scope link 2024-04-25T09:17:38.203 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:17:38.203 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:17:38.203 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether da:a5:72:8e:a9:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:17:38.203 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d8a5:72ff:fe8e:a965/64 scope link 2024-04-25T09:17:38.203 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:17:38.204 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:17:38.204 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether c2:99:2b:fd:c0:08 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:17:38.204 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::c099:2bff:fefd:c008/64 scope link 2024-04-25T09:17:38.204 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:17:38.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.205 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:17:38.279 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-25T09:17:38.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.282 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-25T09:17:38.333 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-25T09:17:38.333 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:17:38.333 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:17:38.333 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:17:38.333 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:17:38.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.333 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec bash -c ' 2024-04-25T09:17:38.334 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:38.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:17:38.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:17:38.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:17:38.334 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:38.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.486 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:17:38.539 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:17:38.540 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:17:38.540 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:17:38.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.540 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-25T09:17:38.627 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:17:38.627 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:17:38.685 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:17:38.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:17:38.685 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:17:38.738 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T09:17:38.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.739 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:17:38.874 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:17:38.875 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:17:38.911 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T09:17:38.911 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T09:17:38.911 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.121:3300,172.21.15.121:3301" 2024-04-25T09:17:38.911 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with new device syntax: 0@9e112875-3fc2-4963-90e0-2b56d5516d5c.cephfs=/ 2024-04-25T09:17:38.911 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.62:3300/172.21.15.121:3300/172.21.15.121:3301" will pass to kernel 2024-04-25T09:17:38.914 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:17:38.914 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:17:38.914 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:17:38.914 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:17:38.914 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:17:38.918 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:17:38.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.918 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:17:38.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:38.998 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:17:39.059 INFO:teuthology.orchestra.run.smithi062.stdout:client7153 2024-04-25T09:17:39.059 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:17:39.059 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:17:39.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:39.110 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:39.110 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:39.110 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:17:39.110 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:39.110 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:39.110 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:39.110 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.220 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257865 s, 40.7 MB/s 2024-04-25T09:17:39.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.261 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.261 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365826 s, 28.7 MB/s 2024-04-25T09:17:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.298 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335923 s, 31.2 MB/s 2024-04-25T09:17:39.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.348 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449149 s, 23.3 MB/s 2024-04-25T09:17:39.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.370 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187772 s, 55.8 MB/s 2024-04-25T09:17:39.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.407 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285819 s, 36.7 MB/s 2024-04-25T09:17:39.436 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.437 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285906 s, 36.7 MB/s 2024-04-25T09:17:39.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.459 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176254 s, 59.5 MB/s 2024-04-25T09:17:39.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.491 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274952 s, 38.1 MB/s 2024-04-25T09:17:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263328 s, 39.8 MB/s 2024-04-25T09:17:39.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.550 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244405 s, 42.9 MB/s 2024-04-25T09:17:39.577 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.577 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.577 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232396 s, 45.1 MB/s 2024-04-25T09:17:39.615 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.616 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.616 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034257 s, 30.6 MB/s 2024-04-25T09:17:39.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.643 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023371 s, 44.9 MB/s 2024-04-25T09:17:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.675 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.676 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283898 s, 36.9 MB/s 2024-04-25T09:17:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.705 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252512 s, 41.5 MB/s 2024-04-25T09:17:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.734 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.735 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264001 s, 39.7 MB/s 2024-04-25T09:17:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.761 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222743 s, 47.1 MB/s 2024-04-25T09:17:39.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.785 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198962 s, 52.7 MB/s 2024-04-25T09:17:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.813 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241322 s, 43.5 MB/s 2024-04-25T09:17:39.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.842 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246513 s, 42.5 MB/s 2024-04-25T09:17:39.873 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.874 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.874 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250896 s, 41.8 MB/s 2024-04-25T09:17:39.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.898 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228329 s, 45.9 MB/s 2024-04-25T09:17:39.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.925 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022621 s, 46.4 MB/s 2024-04-25T09:17:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218766 s, 47.9 MB/s 2024-04-25T09:17:39.978 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:39.978 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:39.978 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022711 s, 46.2 MB/s 2024-04-25T09:17:40.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.006 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237326 s, 44.2 MB/s 2024-04-25T09:17:40.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.038 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273425 s, 38.3 MB/s 2024-04-25T09:17:40.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199688 s, 52.5 MB/s 2024-04-25T09:17:40.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.086 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205799 s, 51.0 MB/s 2024-04-25T09:17:40.111 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.111 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.111 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211777 s, 49.5 MB/s 2024-04-25T09:17:40.141 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.141 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.141 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247609 s, 42.3 MB/s 2024-04-25T09:17:40.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:40.143 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:40.144 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:40.144 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:17:40.144 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:40.144 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:40.144 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:40.144 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:40.231 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.231 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.231 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231656 s, 45.3 MB/s 2024-04-25T09:17:40.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.259 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225896 s, 46.4 MB/s 2024-04-25T09:17:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.284 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216638 s, 48.4 MB/s 2024-04-25T09:17:40.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.307 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195773 s, 53.6 MB/s 2024-04-25T09:17:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.340 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289402 s, 36.2 MB/s 2024-04-25T09:17:40.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.369 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253147 s, 41.4 MB/s 2024-04-25T09:17:40.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.396 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232838 s, 45.0 MB/s 2024-04-25T09:17:40.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221927 s, 47.2 MB/s 2024-04-25T09:17:40.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.449 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224888 s, 46.6 MB/s 2024-04-25T09:17:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.471 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184907 s, 56.7 MB/s 2024-04-25T09:17:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237928 s, 44.1 MB/s 2024-04-25T09:17:40.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.529 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240745 s, 43.6 MB/s 2024-04-25T09:17:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216731 s, 48.4 MB/s 2024-04-25T09:17:40.582 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.583 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236529 s, 44.3 MB/s 2024-04-25T09:17:40.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.611 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.611 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024517 s, 42.8 MB/s 2024-04-25T09:17:40.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.640 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023763 s, 44.1 MB/s 2024-04-25T09:17:40.663 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.664 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.664 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205529 s, 51.0 MB/s 2024-04-25T09:17:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.689 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220504 s, 47.6 MB/s 2024-04-25T09:17:40.717 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.717 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.717 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241252 s, 43.5 MB/s 2024-04-25T09:17:40.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254353 s, 41.2 MB/s 2024-04-25T09:17:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228446 s, 45.9 MB/s 2024-04-25T09:17:40.803 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.803 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.803 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254466 s, 41.2 MB/s 2024-04-25T09:17:40.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.830 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224766 s, 46.7 MB/s 2024-04-25T09:17:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221519 s, 47.3 MB/s 2024-04-25T09:17:40.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.884 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232965 s, 45.0 MB/s 2024-04-25T09:17:40.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.913 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.913 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254009 s, 41.3 MB/s 2024-04-25T09:17:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019497 s, 53.8 MB/s 2024-04-25T09:17:40.968 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.968 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.968 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274653 s, 38.2 MB/s 2024-04-25T09:17:40.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:40.992 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191619 s, 54.7 MB/s 2024-04-25T09:17:41.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.014 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193197 s, 54.3 MB/s 2024-04-25T09:17:41.039 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.039 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.039 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021583 s, 48.6 MB/s 2024-04-25T09:17:41.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.063 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197477 s, 53.1 MB/s 2024-04-25T09:17:41.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:41.066 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:41.066 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:41.066 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:17:41.066 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:41.066 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:41.066 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:41.066 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.151 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201859 s, 51.9 MB/s 2024-04-25T09:17:41.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.176 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206673 s, 50.7 MB/s 2024-04-25T09:17:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.200 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021161 s, 49.6 MB/s 2024-04-25T09:17:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.226 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021484 s, 48.8 MB/s 2024-04-25T09:17:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.253 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02247 s, 46.7 MB/s 2024-04-25T09:17:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.282 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246276 s, 42.6 MB/s 2024-04-25T09:17:41.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.308 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.308 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219482 s, 47.8 MB/s 2024-04-25T09:17:41.334 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219447 s, 47.8 MB/s 2024-04-25T09:17:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.359 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202611 s, 51.8 MB/s 2024-04-25T09:17:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.383 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205414 s, 51.0 MB/s 2024-04-25T09:17:41.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.408 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205949 s, 50.9 MB/s 2024-04-25T09:17:41.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.434 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217608 s, 48.2 MB/s 2024-04-25T09:17:41.460 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.461 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.461 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229551 s, 45.7 MB/s 2024-04-25T09:17:41.486 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.486 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.486 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207553 s, 50.5 MB/s 2024-04-25T09:17:41.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.512 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231568 s, 45.3 MB/s 2024-04-25T09:17:41.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.539 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240693 s, 43.6 MB/s 2024-04-25T09:17:41.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.568 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249919 s, 42.0 MB/s 2024-04-25T09:17:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.598 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244273 s, 42.9 MB/s 2024-04-25T09:17:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232173 s, 45.2 MB/s 2024-04-25T09:17:41.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.652 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.652 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230112 s, 45.6 MB/s 2024-04-25T09:17:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.679 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215787 s, 48.6 MB/s 2024-04-25T09:17:41.706 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.707 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231947 s, 45.2 MB/s 2024-04-25T09:17:41.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257496 s, 40.7 MB/s 2024-04-25T09:17:41.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.765 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242511 s, 43.2 MB/s 2024-04-25T09:17:41.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.795 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256205 s, 40.9 MB/s 2024-04-25T09:17:41.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.819 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203293 s, 51.6 MB/s 2024-04-25T09:17:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.850 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269035 s, 39.0 MB/s 2024-04-25T09:17:41.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.879 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024363 s, 43.0 MB/s 2024-04-25T09:17:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.907 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230407 s, 45.5 MB/s 2024-04-25T09:17:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.934 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232197 s, 45.2 MB/s 2024-04-25T09:17:41.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026612 s, 39.4 MB/s 2024-04-25T09:17:41.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:41.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:41.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236093 s, 44.4 MB/s 2024-04-25T09:17:41.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:41.995 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:41.995 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:41.995 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:17:41.995 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:41.995 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:41.995 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:41.995 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.079 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184958 s, 56.7 MB/s 2024-04-25T09:17:42.105 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.106 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224275 s, 46.8 MB/s 2024-04-25T09:17:42.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.130 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.131 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209351 s, 50.1 MB/s 2024-04-25T09:17:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188627 s, 55.6 MB/s 2024-04-25T09:17:42.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.179 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215603 s, 48.6 MB/s 2024-04-25T09:17:42.210 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.210 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.210 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268414 s, 39.1 MB/s 2024-04-25T09:17:42.241 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.241 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.241 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263859 s, 39.7 MB/s 2024-04-25T09:17:42.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.268 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.268 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229156 s, 45.8 MB/s 2024-04-25T09:17:42.294 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.295 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208785 s, 50.2 MB/s 2024-04-25T09:17:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.320 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213884 s, 49.0 MB/s 2024-04-25T09:17:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.348 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227847 s, 46.0 MB/s 2024-04-25T09:17:42.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.374 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.374 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021716 s, 48.3 MB/s 2024-04-25T09:17:42.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.401 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.401 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223679 s, 46.9 MB/s 2024-04-25T09:17:42.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192187 s, 54.6 MB/s 2024-04-25T09:17:42.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.451 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204963 s, 51.2 MB/s 2024-04-25T09:17:42.476 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.477 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219268 s, 47.8 MB/s 2024-04-25T09:17:42.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.505 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.505 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244517 s, 42.9 MB/s 2024-04-25T09:17:42.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.532 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.532 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227361 s, 46.1 MB/s 2024-04-25T09:17:42.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.554 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.554 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184726 s, 56.8 MB/s 2024-04-25T09:17:42.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025493 s, 41.1 MB/s 2024-04-25T09:17:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222699 s, 47.1 MB/s 2024-04-25T09:17:42.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.642 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281168 s, 37.3 MB/s 2024-04-25T09:17:42.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.671 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.671 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241845 s, 43.4 MB/s 2024-04-25T09:17:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240658 s, 43.6 MB/s 2024-04-25T09:17:42.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.723 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.723 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198652 s, 52.8 MB/s 2024-04-25T09:17:42.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.753 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.753 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244007 s, 43.0 MB/s 2024-04-25T09:17:42.782 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.782 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.782 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248641 s, 42.2 MB/s 2024-04-25T09:17:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020251 s, 51.8 MB/s 2024-04-25T09:17:42.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.832 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.832 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216222 s, 48.5 MB/s 2024-04-25T09:17:42.860 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.860 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.860 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242152 s, 43.3 MB/s 2024-04-25T09:17:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190573 s, 55.0 MB/s 2024-04-25T09:17:42.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203379 s, 51.6 MB/s 2024-04-25T09:17:42.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:42.910 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:42.910 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:42.910 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:17:42.910 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:42.910 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:42.911 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:42.911 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:42.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:42.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:42.995 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196948 s, 53.2 MB/s 2024-04-25T09:17:43.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.021 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219209 s, 47.8 MB/s 2024-04-25T09:17:43.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.045 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206798 s, 50.7 MB/s 2024-04-25T09:17:43.072 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.072 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.072 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225223 s, 46.6 MB/s 2024-04-25T09:17:43.097 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.097 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.098 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222121 s, 47.2 MB/s 2024-04-25T09:17:43.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228716 s, 45.8 MB/s 2024-04-25T09:17:43.149 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.150 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216191 s, 48.5 MB/s 2024-04-25T09:17:43.174 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207553 s, 50.5 MB/s 2024-04-25T09:17:43.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.203 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241786 s, 43.4 MB/s 2024-04-25T09:17:43.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.229 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210147 s, 49.9 MB/s 2024-04-25T09:17:43.257 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.257 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.257 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244558 s, 42.9 MB/s 2024-04-25T09:17:43.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.284 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.284 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222574 s, 47.1 MB/s 2024-04-25T09:17:43.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.309 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214426 s, 48.9 MB/s 2024-04-25T09:17:43.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021599 s, 48.5 MB/s 2024-04-25T09:17:43.365 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.365 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.365 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263175 s, 39.8 MB/s 2024-04-25T09:17:43.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.395 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252632 s, 41.5 MB/s 2024-04-25T09:17:43.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.423 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238285 s, 44.0 MB/s 2024-04-25T09:17:43.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.449 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218584 s, 48.0 MB/s 2024-04-25T09:17:43.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.473 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203445 s, 51.5 MB/s 2024-04-25T09:17:43.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.497 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.497 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205061 s, 51.1 MB/s 2024-04-25T09:17:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.527 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238083 s, 44.0 MB/s 2024-04-25T09:17:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.553 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220922 s, 47.5 MB/s 2024-04-25T09:17:43.580 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.580 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.580 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209401 s, 50.1 MB/s 2024-04-25T09:17:43.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.607 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225133 s, 46.6 MB/s 2024-04-25T09:17:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.631 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201996 s, 51.9 MB/s 2024-04-25T09:17:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021742 s, 48.2 MB/s 2024-04-25T09:17:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.689 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275192 s, 38.1 MB/s 2024-04-25T09:17:43.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.714 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209871 s, 50.0 MB/s 2024-04-25T09:17:43.738 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.738 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.738 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207238 s, 50.6 MB/s 2024-04-25T09:17:43.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.765 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234072 s, 44.8 MB/s 2024-04-25T09:17:43.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.790 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.790 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201565 s, 52.0 MB/s 2024-04-25T09:17:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.815 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199657 s, 52.5 MB/s 2024-04-25T09:17:43.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:43.817 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:43.817 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:43.817 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:17:43.817 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:43.817 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:43.817 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:43.818 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.900 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204489 s, 51.3 MB/s 2024-04-25T09:17:43.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222516 s, 47.1 MB/s 2024-04-25T09:17:43.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.951 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.951 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213297 s, 49.2 MB/s 2024-04-25T09:17:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:43.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200489 s, 52.3 MB/s 2024-04-25T09:17:43.999 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.000 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.000 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209732 s, 50.0 MB/s 2024-04-25T09:17:44.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.028 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237764 s, 44.1 MB/s 2024-04-25T09:17:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269417 s, 38.9 MB/s 2024-04-25T09:17:44.083 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.083 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.083 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204818 s, 51.2 MB/s 2024-04-25T09:17:44.106 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.107 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188085 s, 55.8 MB/s 2024-04-25T09:17:44.133 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.134 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233031 s, 45.0 MB/s 2024-04-25T09:17:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.158 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206474 s, 50.8 MB/s 2024-04-25T09:17:44.181 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.182 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194588 s, 53.9 MB/s 2024-04-25T09:17:44.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.203 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180992 s, 57.9 MB/s 2024-04-25T09:17:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204667 s, 51.2 MB/s 2024-04-25T09:17:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.251 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185543 s, 56.5 MB/s 2024-04-25T09:17:44.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.278 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223702 s, 46.9 MB/s 2024-04-25T09:17:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.305 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232099 s, 45.2 MB/s 2024-04-25T09:17:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.331 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217963 s, 48.1 MB/s 2024-04-25T09:17:44.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.362 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261264 s, 40.1 MB/s 2024-04-25T09:17:44.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.385 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203392 s, 51.6 MB/s 2024-04-25T09:17:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.412 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02331 s, 45.0 MB/s 2024-04-25T09:17:44.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.439 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227736 s, 46.0 MB/s 2024-04-25T09:17:44.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196083 s, 53.5 MB/s 2024-04-25T09:17:44.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195287 s, 53.7 MB/s 2024-04-25T09:17:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.516 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249638 s, 42.0 MB/s 2024-04-25T09:17:44.547 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.548 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274176 s, 38.2 MB/s 2024-04-25T09:17:44.575 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.575 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.575 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235311 s, 44.6 MB/s 2024-04-25T09:17:44.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.602 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022679 s, 46.2 MB/s 2024-04-25T09:17:44.632 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.633 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263134 s, 39.8 MB/s 2024-04-25T09:17:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.660 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230552 s, 45.5 MB/s 2024-04-25T09:17:44.683 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.684 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199308 s, 52.6 MB/s 2024-04-25T09:17:44.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.712 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244748 s, 42.8 MB/s 2024-04-25T09:17:44.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:44.715 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:44.715 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:44.715 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:17:44.715 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:44.715 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:44.716 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:44.716 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:44.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.802 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203493 s, 51.5 MB/s 2024-04-25T09:17:44.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205825 s, 50.9 MB/s 2024-04-25T09:17:44.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.856 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258764 s, 40.5 MB/s 2024-04-25T09:17:44.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218215 s, 48.1 MB/s 2024-04-25T09:17:44.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.908 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222231 s, 47.2 MB/s 2024-04-25T09:17:44.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.935 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.935 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212985 s, 49.2 MB/s 2024-04-25T09:17:44.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.958 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.958 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213454 s, 49.1 MB/s 2024-04-25T09:17:44.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:44.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:44.986 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239809 s, 43.7 MB/s 2024-04-25T09:17:45.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.011 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.011 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214353 s, 48.9 MB/s 2024-04-25T09:17:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217148 s, 48.3 MB/s 2024-04-25T09:17:45.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213845 s, 49.0 MB/s 2024-04-25T09:17:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223652 s, 46.9 MB/s 2024-04-25T09:17:45.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232773 s, 45.0 MB/s 2024-04-25T09:17:45.145 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257353 s, 40.7 MB/s 2024-04-25T09:17:45.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.171 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.172 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214471 s, 48.9 MB/s 2024-04-25T09:17:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.198 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229533 s, 45.7 MB/s 2024-04-25T09:17:45.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.228 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258967 s, 40.5 MB/s 2024-04-25T09:17:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.259 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268549 s, 39.0 MB/s 2024-04-25T09:17:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.286 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022978 s, 45.6 MB/s 2024-04-25T09:17:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.314 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238892 s, 43.9 MB/s 2024-04-25T09:17:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.342 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235488 s, 44.5 MB/s 2024-04-25T09:17:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192619 s, 54.4 MB/s 2024-04-25T09:17:45.391 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.392 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223931 s, 46.8 MB/s 2024-04-25T09:17:45.418 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.418 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.418 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215073 s, 48.8 MB/s 2024-04-25T09:17:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.443 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.444 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214038 s, 49.0 MB/s 2024-04-25T09:17:45.467 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.468 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020756 s, 50.5 MB/s 2024-04-25T09:17:45.498 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.499 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.499 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268164 s, 39.1 MB/s 2024-04-25T09:17:45.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.536 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327804 s, 32.0 MB/s 2024-04-25T09:17:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.568 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027446 s, 38.2 MB/s 2024-04-25T09:17:45.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.589 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.589 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172681 s, 60.7 MB/s 2024-04-25T09:17:45.613 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.613 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.613 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194458 s, 53.9 MB/s 2024-04-25T09:17:45.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227214 s, 46.1 MB/s 2024-04-25T09:17:45.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:45.642 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:45.642 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:45.642 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:17:45.642 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:45.642 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:45.642 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:45.642 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:45.707 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:17:45.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:17:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236307 s, 44.4 MB/s 2024-04-25T09:17:45.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.750 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.750 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186441 s, 56.2 MB/s 2024-04-25T09:17:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.766 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168405 s, 62.3 MB/s 2024-04-25T09:17:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.794 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250926 s, 41.8 MB/s 2024-04-25T09:17:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.819 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.820 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218551 s, 48.0 MB/s 2024-04-25T09:17:45.843 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.843 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.844 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211442 s, 49.6 MB/s 2024-04-25T09:17:45.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.868 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204173 s, 51.4 MB/s 2024-04-25T09:17:45.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.894 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216744 s, 48.4 MB/s 2024-04-25T09:17:45.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.926 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.926 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024133 s, 43.4 MB/s 2024-04-25T09:17:45.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.949 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020939 s, 50.1 MB/s 2024-04-25T09:17:45.973 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:45.974 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:45.974 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020616 s, 50.9 MB/s 2024-04-25T09:17:46.001 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.001 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.001 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231213 s, 45.4 MB/s 2024-04-25T09:17:46.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.027 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.027 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02145 s, 48.9 MB/s 2024-04-25T09:17:46.052 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.053 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.053 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202814 s, 51.7 MB/s 2024-04-25T09:17:46.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.081 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242832 s, 43.2 MB/s 2024-04-25T09:17:46.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.109 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238789 s, 43.9 MB/s 2024-04-25T09:17:46.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.136 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.136 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236505 s, 44.3 MB/s 2024-04-25T09:17:46.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.163 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222681 s, 47.1 MB/s 2024-04-25T09:17:46.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.188 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.188 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020814 s, 50.4 MB/s 2024-04-25T09:17:46.214 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.214 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.214 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02246 s, 46.7 MB/s 2024-04-25T09:17:46.247 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.247 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.247 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028996 s, 36.2 MB/s 2024-04-25T09:17:46.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.276 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249305 s, 42.1 MB/s 2024-04-25T09:17:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.305 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246755 s, 42.5 MB/s 2024-04-25T09:17:46.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.331 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211787 s, 49.5 MB/s 2024-04-25T09:17:46.360 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.361 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258081 s, 40.6 MB/s 2024-04-25T09:17:46.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.390 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250091 s, 41.9 MB/s 2024-04-25T09:17:46.420 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.420 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.420 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258076 s, 40.6 MB/s 2024-04-25T09:17:46.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.449 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.450 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256928 s, 40.8 MB/s 2024-04-25T09:17:46.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.474 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201292 s, 52.1 MB/s 2024-04-25T09:17:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.502 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241761 s, 43.4 MB/s 2024-04-25T09:17:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.529 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229482 s, 45.7 MB/s 2024-04-25T09:17:46.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266592 s, 39.3 MB/s 2024-04-25T09:17:46.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:46.562 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:46.562 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:46.562 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:17:46.562 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:46.562 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:46.563 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:46.563 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:46.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.701 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292368 s, 35.9 MB/s 2024-04-25T09:17:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.727 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220831 s, 47.5 MB/s 2024-04-25T09:17:46.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166878 s, 62.8 MB/s 2024-04-25T09:17:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.775 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242463 s, 43.2 MB/s 2024-04-25T09:17:46.800 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.801 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220885 s, 47.5 MB/s 2024-04-25T09:17:46.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.828 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023574 s, 44.5 MB/s 2024-04-25T09:17:46.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.855 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207078 s, 50.6 MB/s 2024-04-25T09:17:46.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277495 s, 37.8 MB/s 2024-04-25T09:17:46.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.912 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.912 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210973 s, 49.7 MB/s 2024-04-25T09:17:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.939 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232298 s, 45.1 MB/s 2024-04-25T09:17:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.970 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026978 s, 38.9 MB/s 2024-04-25T09:17:46.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:46.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181534 s, 57.8 MB/s 2024-04-25T09:17:47.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.019 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221774 s, 47.3 MB/s 2024-04-25T09:17:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.046 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228875 s, 45.8 MB/s 2024-04-25T09:17:47.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.072 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215429 s, 48.7 MB/s 2024-04-25T09:17:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.097 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.098 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218745 s, 47.9 MB/s 2024-04-25T09:17:47.122 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.122 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.123 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207152 s, 50.6 MB/s 2024-04-25T09:17:47.152 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.152 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.152 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026108 s, 40.2 MB/s 2024-04-25T09:17:47.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.184 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027419 s, 38.2 MB/s 2024-04-25T09:17:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.209 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216204 s, 48.5 MB/s 2024-04-25T09:17:47.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.233 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194426 s, 53.9 MB/s 2024-04-25T09:17:47.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.255 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.255 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196254 s, 53.4 MB/s 2024-04-25T09:17:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.282 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227426 s, 46.1 MB/s 2024-04-25T09:17:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.307 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210849 s, 49.7 MB/s 2024-04-25T09:17:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.333 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218123 s, 48.1 MB/s 2024-04-25T09:17:47.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.363 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.363 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261738 s, 40.1 MB/s 2024-04-25T09:17:47.388 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.389 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210853 s, 49.7 MB/s 2024-04-25T09:17:47.414 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.414 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.414 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210133 s, 49.9 MB/s 2024-04-25T09:17:47.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.438 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203437 s, 51.5 MB/s 2024-04-25T09:17:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203428 s, 51.5 MB/s 2024-04-25T09:17:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201923 s, 51.9 MB/s 2024-04-25T09:17:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.514 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232971 s, 45.0 MB/s 2024-04-25T09:17:47.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:47.517 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:47.517 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:47.517 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:17:47.517 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:47.517 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:47.517 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:47.517 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:47.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.609 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251602 s, 41.7 MB/s 2024-04-25T09:17:47.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.641 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276941 s, 37.9 MB/s 2024-04-25T09:17:47.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237479 s, 44.2 MB/s 2024-04-25T09:17:47.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.694 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.694 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220102 s, 47.6 MB/s 2024-04-25T09:17:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027471 s, 38.2 MB/s 2024-04-25T09:17:47.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.751 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216096 s, 48.5 MB/s 2024-04-25T09:17:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.776 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205112 s, 51.1 MB/s 2024-04-25T09:17:47.800 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.801 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205744 s, 51.0 MB/s 2024-04-25T09:17:47.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.827 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221107 s, 47.4 MB/s 2024-04-25T09:17:47.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.852 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.852 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214441 s, 48.9 MB/s 2024-04-25T09:17:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.878 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222075 s, 47.2 MB/s 2024-04-25T09:17:47.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.905 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221723 s, 47.3 MB/s 2024-04-25T09:17:47.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.938 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260019 s, 40.3 MB/s 2024-04-25T09:17:47.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.962 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217605 s, 48.2 MB/s 2024-04-25T09:17:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:47.984 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192296 s, 54.5 MB/s 2024-04-25T09:17:48.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257892 s, 40.7 MB/s 2024-04-25T09:17:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.038 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206309 s, 50.8 MB/s 2024-04-25T09:17:48.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.063 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202799 s, 51.7 MB/s 2024-04-25T09:17:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.086 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198341 s, 52.9 MB/s 2024-04-25T09:17:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.110 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02025 s, 51.8 MB/s 2024-04-25T09:17:48.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.135 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206699 s, 50.7 MB/s 2024-04-25T09:17:48.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.164 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260413 s, 40.3 MB/s 2024-04-25T09:17:48.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222689 s, 47.1 MB/s 2024-04-25T09:17:48.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.217 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.217 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225708 s, 46.5 MB/s 2024-04-25T09:17:48.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.243 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228516 s, 45.9 MB/s 2024-04-25T09:17:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206105 s, 50.9 MB/s 2024-04-25T09:17:48.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.295 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.295 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223926 s, 46.8 MB/s 2024-04-25T09:17:48.321 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.321 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.321 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213258 s, 49.2 MB/s 2024-04-25T09:17:48.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.347 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.347 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216614 s, 48.4 MB/s 2024-04-25T09:17:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.377 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247129 s, 42.4 MB/s 2024-04-25T09:17:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.400 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191573 s, 54.7 MB/s 2024-04-25T09:17:48.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.426 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214084 s, 49.0 MB/s 2024-04-25T09:17:48.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:48.429 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:48.429 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:48.430 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:17:48.430 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:48.430 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:48.430 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:48.430 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:48.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.515 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205004 s, 51.1 MB/s 2024-04-25T09:17:48.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.543 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250918 s, 41.8 MB/s 2024-04-25T09:17:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.568 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218797 s, 47.9 MB/s 2024-04-25T09:17:48.596 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.597 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023648 s, 44.3 MB/s 2024-04-25T09:17:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.629 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027088 s, 38.7 MB/s 2024-04-25T09:17:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223881 s, 46.8 MB/s 2024-04-25T09:17:48.682 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.682 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.682 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022081 s, 47.5 MB/s 2024-04-25T09:17:48.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.707 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.708 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208729 s, 50.2 MB/s 2024-04-25T09:17:48.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.732 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018845 s, 55.6 MB/s 2024-04-25T09:17:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.761 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252296 s, 41.6 MB/s 2024-04-25T09:17:48.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.788 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022093 s, 47.5 MB/s 2024-04-25T09:17:48.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.814 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215499 s, 48.7 MB/s 2024-04-25T09:17:48.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.842 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238666 s, 43.9 MB/s 2024-04-25T09:17:48.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229232 s, 45.7 MB/s 2024-04-25T09:17:48.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.894 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.894 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214379 s, 48.9 MB/s 2024-04-25T09:17:48.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.921 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.921 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022893 s, 45.8 MB/s 2024-04-25T09:17:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.949 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235448 s, 44.5 MB/s 2024-04-25T09:17:48.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:48.976 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:48.976 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229003 s, 45.8 MB/s 2024-04-25T09:17:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.001 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204177 s, 51.4 MB/s 2024-04-25T09:17:49.028 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.028 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.028 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226328 s, 46.3 MB/s 2024-04-25T09:17:49.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.054 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212284 s, 49.4 MB/s 2024-04-25T09:17:49.082 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.083 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247297 s, 42.4 MB/s 2024-04-25T09:17:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.109 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.110 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228519 s, 45.9 MB/s 2024-04-25T09:17:49.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.134 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.134 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208067 s, 50.4 MB/s 2024-04-25T09:17:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.159 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207665 s, 50.5 MB/s 2024-04-25T09:17:49.185 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.185 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.185 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212494 s, 49.3 MB/s 2024-04-25T09:17:49.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.207 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018526 s, 56.6 MB/s 2024-04-25T09:17:49.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.233 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.233 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224488 s, 46.7 MB/s 2024-04-25T09:17:49.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.260 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022545 s, 46.5 MB/s 2024-04-25T09:17:49.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.286 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214658 s, 48.8 MB/s 2024-04-25T09:17:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.309 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198145 s, 52.9 MB/s 2024-04-25T09:17:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.337 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241832 s, 43.4 MB/s 2024-04-25T09:17:49.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:49.340 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:49.340 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:49.340 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:17:49.340 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:49.340 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:49.340 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:49.340 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.429 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225697 s, 46.5 MB/s 2024-04-25T09:17:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.454 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216472 s, 48.4 MB/s 2024-04-25T09:17:49.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.480 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212865 s, 49.3 MB/s 2024-04-25T09:17:49.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.508 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.508 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244491 s, 42.9 MB/s 2024-04-25T09:17:49.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.536 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.536 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244581 s, 42.9 MB/s 2024-04-25T09:17:49.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.564 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.564 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024119 s, 43.5 MB/s 2024-04-25T09:17:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.594 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253163 s, 41.4 MB/s 2024-04-25T09:17:49.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.622 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.622 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240959 s, 43.5 MB/s 2024-04-25T09:17:49.649 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.650 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.650 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234364 s, 44.7 MB/s 2024-04-25T09:17:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.673 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192808 s, 54.4 MB/s 2024-04-25T09:17:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.703 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257231 s, 40.8 MB/s 2024-04-25T09:17:49.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.732 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.732 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253464 s, 41.4 MB/s 2024-04-25T09:17:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.756 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201823 s, 52.0 MB/s 2024-04-25T09:17:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.785 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245923 s, 42.6 MB/s 2024-04-25T09:17:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022485 s, 46.6 MB/s 2024-04-25T09:17:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.840 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233979 s, 44.8 MB/s 2024-04-25T09:17:49.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.868 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.868 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244072 s, 43.0 MB/s 2024-04-25T09:17:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.891 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.892 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195245 s, 53.7 MB/s 2024-04-25T09:17:49.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172457 s, 60.8 MB/s 2024-04-25T09:17:49.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.939 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02053 s, 51.1 MB/s 2024-04-25T09:17:49.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226233 s, 46.3 MB/s 2024-04-25T09:17:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:49.988 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193714 s, 54.1 MB/s 2024-04-25T09:17:50.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.012 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197998 s, 53.0 MB/s 2024-04-25T09:17:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.043 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256687 s, 40.9 MB/s 2024-04-25T09:17:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.071 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247292 s, 42.4 MB/s 2024-04-25T09:17:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226188 s, 46.4 MB/s 2024-04-25T09:17:50.119 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.119 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.119 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175845 s, 59.6 MB/s 2024-04-25T09:17:50.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.144 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218635 s, 48.0 MB/s 2024-04-25T09:17:50.167 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.167 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.167 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180266 s, 58.2 MB/s 2024-04-25T09:17:50.192 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.192 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.192 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197684 s, 53.0 MB/s 2024-04-25T09:17:50.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.216 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200321 s, 52.3 MB/s 2024-04-25T09:17:50.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.245 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.245 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252581 s, 41.5 MB/s 2024-04-25T09:17:50.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:50.248 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:50.248 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:50.248 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:17:50.248 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:50.248 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:50.248 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:50.248 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:50.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187464 s, 55.9 MB/s 2024-04-25T09:17:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.361 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232415 s, 45.1 MB/s 2024-04-25T09:17:50.387 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.387 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.387 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221363 s, 47.4 MB/s 2024-04-25T09:17:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.416 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024798 s, 42.3 MB/s 2024-04-25T09:17:50.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.442 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.443 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228037 s, 46.0 MB/s 2024-04-25T09:17:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258075 s, 40.6 MB/s 2024-04-25T09:17:50.501 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.501 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.501 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024371 s, 43.0 MB/s 2024-04-25T09:17:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256836 s, 40.8 MB/s 2024-04-25T09:17:50.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.559 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.559 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232442 s, 45.1 MB/s 2024-04-25T09:17:50.582 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.583 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.583 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207957 s, 50.4 MB/s 2024-04-25T09:17:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234009 s, 44.8 MB/s 2024-04-25T09:17:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238139 s, 44.0 MB/s 2024-04-25T09:17:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180649 s, 58.0 MB/s 2024-04-25T09:17:50.680 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.680 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.680 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171083 s, 61.3 MB/s 2024-04-25T09:17:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192588 s, 54.4 MB/s 2024-04-25T09:17:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017934 s, 58.5 MB/s 2024-04-25T09:17:50.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.751 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214848 s, 48.8 MB/s 2024-04-25T09:17:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232288 s, 45.1 MB/s 2024-04-25T09:17:50.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.802 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.802 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203127 s, 51.6 MB/s 2024-04-25T09:17:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210509 s, 49.8 MB/s 2024-04-25T09:17:50.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.854 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240768 s, 43.6 MB/s 2024-04-25T09:17:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207717 s, 50.5 MB/s 2024-04-25T09:17:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242623 s, 43.2 MB/s 2024-04-25T09:17:50.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253121 s, 41.4 MB/s 2024-04-25T09:17:50.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.963 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222053 s, 47.2 MB/s 2024-04-25T09:17:50.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:50.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:50.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231339 s, 45.3 MB/s 2024-04-25T09:17:51.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.018 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240956 s, 43.5 MB/s 2024-04-25T09:17:51.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.046 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226601 s, 46.3 MB/s 2024-04-25T09:17:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198964 s, 52.7 MB/s 2024-04-25T09:17:51.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.094 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.094 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201826 s, 52.0 MB/s 2024-04-25T09:17:51.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.121 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.121 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216911 s, 48.3 MB/s 2024-04-25T09:17:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208658 s, 50.3 MB/s 2024-04-25T09:17:51.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:51.148 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:51.148 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:51.149 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:17:51.149 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:51.149 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:51.149 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:51.149 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194634 s, 53.9 MB/s 2024-04-25T09:17:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247703 s, 42.3 MB/s 2024-04-25T09:17:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189149 s, 55.4 MB/s 2024-04-25T09:17:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223014 s, 47.0 MB/s 2024-04-25T09:17:51.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.336 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251636 s, 41.7 MB/s 2024-04-25T09:17:51.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.362 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222367 s, 47.2 MB/s 2024-04-25T09:17:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205758 s, 51.0 MB/s 2024-04-25T09:17:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.415 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233033 s, 45.0 MB/s 2024-04-25T09:17:51.438 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.439 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.439 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204664 s, 51.2 MB/s 2024-04-25T09:17:51.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.464 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200811 s, 52.2 MB/s 2024-04-25T09:17:51.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.490 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183162 s, 57.2 MB/s 2024-04-25T09:17:51.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.509 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193254 s, 54.3 MB/s 2024-04-25T09:17:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213358 s, 49.1 MB/s 2024-04-25T09:17:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.560 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215093 s, 48.7 MB/s 2024-04-25T09:17:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.588 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241977 s, 43.3 MB/s 2024-04-25T09:17:51.616 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.616 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.616 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247078 s, 42.4 MB/s 2024-04-25T09:17:51.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.643 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232362 s, 45.1 MB/s 2024-04-25T09:17:51.670 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.670 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.670 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230283 s, 45.5 MB/s 2024-04-25T09:17:51.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.695 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.695 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201591 s, 52.0 MB/s 2024-04-25T09:17:51.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.721 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225003 s, 46.6 MB/s 2024-04-25T09:17:51.749 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.749 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.750 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023827 s, 44.0 MB/s 2024-04-25T09:17:51.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025602 s, 41.0 MB/s 2024-04-25T09:17:51.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.805 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216291 s, 48.5 MB/s 2024-04-25T09:17:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.828 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185534 s, 56.5 MB/s 2024-04-25T09:17:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.854 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220821 s, 47.5 MB/s 2024-04-25T09:17:51.881 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.882 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229882 s, 45.6 MB/s 2024-04-25T09:17:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.912 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254172 s, 41.3 MB/s 2024-04-25T09:17:51.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.936 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193536 s, 54.2 MB/s 2024-04-25T09:17:51.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.961 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.961 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198407 s, 52.8 MB/s 2024-04-25T09:17:51.989 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:51.989 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:51.989 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234432 s, 44.7 MB/s 2024-04-25T09:17:52.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.012 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.012 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197991 s, 53.0 MB/s 2024-04-25T09:17:52.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.036 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.036 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190728 s, 55.0 MB/s 2024-04-25T09:17:52.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:52.039 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:52.039 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:52.039 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:17:52.039 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:52.039 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:52.039 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:52.039 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:52.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192288 s, 54.5 MB/s 2024-04-25T09:17:52.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.153 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.153 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255845 s, 41.0 MB/s 2024-04-25T09:17:52.174 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.174 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178316 s, 58.8 MB/s 2024-04-25T09:17:52.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.202 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225286 s, 46.5 MB/s 2024-04-25T09:17:52.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.232 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261003 s, 40.2 MB/s 2024-04-25T09:17:52.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.265 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278519 s, 37.6 MB/s 2024-04-25T09:17:52.287 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.288 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189051 s, 55.5 MB/s 2024-04-25T09:17:52.317 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.317 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.318 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254432 s, 41.2 MB/s 2024-04-25T09:17:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.343 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206104 s, 50.9 MB/s 2024-04-25T09:17:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.369 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.370 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226028 s, 46.4 MB/s 2024-04-25T09:17:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.399 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024751 s, 42.4 MB/s 2024-04-25T09:17:52.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.425 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.425 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213616 s, 49.1 MB/s 2024-04-25T09:17:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02162 s, 48.5 MB/s 2024-04-25T09:17:52.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.478 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233146 s, 45.0 MB/s 2024-04-25T09:17:52.501 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.501 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.501 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185465 s, 56.5 MB/s 2024-04-25T09:17:52.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.527 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.527 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218118 s, 48.1 MB/s 2024-04-25T09:17:52.556 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.556 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.556 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251313 s, 41.7 MB/s 2024-04-25T09:17:52.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233675 s, 44.9 MB/s 2024-04-25T09:17:52.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.615 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260338 s, 40.3 MB/s 2024-04-25T09:17:52.641 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.642 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229224 s, 45.7 MB/s 2024-04-25T09:17:52.670 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.670 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.670 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240114 s, 43.7 MB/s 2024-04-25T09:17:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.696 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228154 s, 46.0 MB/s 2024-04-25T09:17:52.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.722 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.723 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210954 s, 49.7 MB/s 2024-04-25T09:17:52.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221281 s, 47.4 MB/s 2024-04-25T09:17:52.777 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.778 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267322 s, 39.2 MB/s 2024-04-25T09:17:52.809 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.809 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.809 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272461 s, 38.5 MB/s 2024-04-25T09:17:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.835 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224568 s, 46.7 MB/s 2024-04-25T09:17:52.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.860 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.860 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020672 s, 50.7 MB/s 2024-04-25T09:17:52.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.885 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.885 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212817 s, 49.3 MB/s 2024-04-25T09:17:52.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.911 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216958 s, 48.3 MB/s 2024-04-25T09:17:52.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.936 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205387 s, 51.1 MB/s 2024-04-25T09:17:52.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:52.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:52.965 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247612 s, 42.3 MB/s 2024-04-25T09:17:52.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:52.967 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:52.968 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:52.968 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:17:52.968 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:52.968 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:52.968 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:52.968 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:53.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.054 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.054 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217148 s, 48.3 MB/s 2024-04-25T09:17:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.079 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202109 s, 51.9 MB/s 2024-04-25T09:17:53.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.103 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.103 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197114 s, 53.2 MB/s 2024-04-25T09:17:53.131 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.132 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202478 s, 51.8 MB/s 2024-04-25T09:17:53.158 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.158 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.158 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266424 s, 39.4 MB/s 2024-04-25T09:17:53.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.187 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.187 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249808 s, 42.0 MB/s 2024-04-25T09:17:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.213 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208633 s, 50.3 MB/s 2024-04-25T09:17:53.235 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.235 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.235 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193492 s, 54.2 MB/s 2024-04-25T09:17:53.266 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.266 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.266 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265149 s, 39.5 MB/s 2024-04-25T09:17:53.291 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.292 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208841 s, 50.2 MB/s 2024-04-25T09:17:53.315 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.315 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.316 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196716 s, 53.3 MB/s 2024-04-25T09:17:53.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.342 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.342 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022465 s, 46.7 MB/s 2024-04-25T09:17:53.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.366 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201199 s, 52.1 MB/s 2024-04-25T09:17:53.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.396 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255706 s, 41.0 MB/s 2024-04-25T09:17:53.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217624 s, 48.2 MB/s 2024-04-25T09:17:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.450 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238545 s, 44.0 MB/s 2024-04-25T09:17:53.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.474 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.474 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019642 s, 53.4 MB/s 2024-04-25T09:17:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.503 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254093 s, 41.3 MB/s 2024-04-25T09:17:53.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.531 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234558 s, 44.7 MB/s 2024-04-25T09:17:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201477 s, 52.0 MB/s 2024-04-25T09:17:53.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.584 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.584 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242047 s, 43.3 MB/s 2024-04-25T09:17:53.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.617 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.617 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286648 s, 36.6 MB/s 2024-04-25T09:17:53.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.643 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.643 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223717 s, 46.9 MB/s 2024-04-25T09:17:53.666 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.667 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202572 s, 51.8 MB/s 2024-04-25T09:17:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.701 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292601 s, 35.8 MB/s 2024-04-25T09:17:53.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.731 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.731 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264482 s, 39.6 MB/s 2024-04-25T09:17:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.758 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222021 s, 47.2 MB/s 2024-04-25T09:17:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.787 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244648 s, 42.9 MB/s 2024-04-25T09:17:53.813 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.814 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.814 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221101 s, 47.4 MB/s 2024-04-25T09:17:53.843 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.843 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.843 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253316 s, 41.4 MB/s 2024-04-25T09:17:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.871 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023505 s, 44.6 MB/s 2024-04-25T09:17:53.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.898 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.898 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218679 s, 48.0 MB/s 2024-04-25T09:17:53.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:53.901 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:53.901 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:53.901 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:17:53.901 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:53.901 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:53.901 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:53.901 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:53.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:53.994 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252159 s, 41.6 MB/s 2024-04-25T09:17:54.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.020 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.020 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229734 s, 45.6 MB/s 2024-04-25T09:17:54.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.049 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.049 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025055 s, 41.9 MB/s 2024-04-25T09:17:54.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.075 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.075 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223695 s, 46.9 MB/s 2024-04-25T09:17:54.101 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.102 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.102 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221837 s, 47.3 MB/s 2024-04-25T09:17:54.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.125 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.125 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019191 s, 54.6 MB/s 2024-04-25T09:17:54.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.152 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.152 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217839 s, 48.1 MB/s 2024-04-25T09:17:54.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.177 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.177 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211627 s, 49.5 MB/s 2024-04-25T09:17:54.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.202 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020986 s, 50.0 MB/s 2024-04-25T09:17:54.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.228 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.229 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222124 s, 47.2 MB/s 2024-04-25T09:17:54.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.253 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201686 s, 52.0 MB/s 2024-04-25T09:17:54.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.276 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194276 s, 54.0 MB/s 2024-04-25T09:17:54.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.303 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.303 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226958 s, 46.2 MB/s 2024-04-25T09:17:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235812 s, 44.5 MB/s 2024-04-25T09:17:54.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.358 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219341 s, 47.8 MB/s 2024-04-25T09:17:54.384 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.385 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022994 s, 45.6 MB/s 2024-04-25T09:17:54.411 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.412 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228993 s, 45.8 MB/s 2024-04-25T09:17:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.437 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208941 s, 50.2 MB/s 2024-04-25T09:17:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224391 s, 46.7 MB/s 2024-04-25T09:17:54.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.492 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.492 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232288 s, 45.1 MB/s 2024-04-25T09:17:54.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.524 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.524 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268802 s, 39.0 MB/s 2024-04-25T09:17:54.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.549 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.549 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219622 s, 47.7 MB/s 2024-04-25T09:17:54.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.579 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247506 s, 42.4 MB/s 2024-04-25T09:17:54.604 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.699 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217706 s, 48.2 MB/s 2024-04-25T09:17:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246954 s, 42.5 MB/s 2024-04-25T09:17:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251759 s, 41.6 MB/s 2024-04-25T09:17:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.701 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232616 s, 45.1 MB/s 2024-04-25T09:17:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.718 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238664 s, 43.9 MB/s 2024-04-25T09:17:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.745 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221704 s, 47.3 MB/s 2024-04-25T09:17:54.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.771 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221307 s, 47.4 MB/s 2024-04-25T09:17:54.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.801 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.801 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260867 s, 40.2 MB/s 2024-04-25T09:17:54.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.828 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.828 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229113 s, 45.8 MB/s 2024-04-25T09:17:54.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:54.831 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:54.831 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:54.831 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:17:54.831 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:54.831 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:54.831 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:54.831 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.922 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235613 s, 44.5 MB/s 2024-04-25T09:17:54.954 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.955 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.955 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283158 s, 37.0 MB/s 2024-04-25T09:17:54.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:54.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:54.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221573 s, 47.3 MB/s 2024-04-25T09:17:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.007 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230578 s, 45.5 MB/s 2024-04-25T09:17:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.033 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215851 s, 48.6 MB/s 2024-04-25T09:17:55.058 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.059 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.059 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215278 s, 48.7 MB/s 2024-04-25T09:17:55.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259404 s, 40.4 MB/s 2024-04-25T09:17:55.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.117 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.117 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195485 s, 53.6 MB/s 2024-04-25T09:17:55.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.137 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202765 s, 51.7 MB/s 2024-04-25T09:17:55.161 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.162 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212557 s, 49.3 MB/s 2024-04-25T09:17:55.186 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.186 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.186 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200418 s, 52.3 MB/s 2024-04-25T09:17:55.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.213 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232949 s, 45.0 MB/s 2024-04-25T09:17:55.238 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.238 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.238 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214522 s, 48.9 MB/s 2024-04-25T09:17:55.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.262 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.262 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202686 s, 51.7 MB/s 2024-04-25T09:17:55.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.287 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199607 s, 52.5 MB/s 2024-04-25T09:17:55.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.307 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.307 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017689 s, 59.3 MB/s 2024-04-25T09:17:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.336 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.336 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246289 s, 42.6 MB/s 2024-04-25T09:17:55.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.366 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.366 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260903 s, 40.2 MB/s 2024-04-25T09:17:55.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.390 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.390 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196645 s, 53.3 MB/s 2024-04-25T09:17:55.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.419 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250736 s, 41.8 MB/s 2024-04-25T09:17:55.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.446 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225432 s, 46.5 MB/s 2024-04-25T09:17:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.465 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158305 s, 66.2 MB/s 2024-04-25T09:17:55.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.491 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212198 s, 49.4 MB/s 2024-04-25T09:17:55.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.519 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.519 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249864 s, 42.0 MB/s 2024-04-25T09:17:55.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.541 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.541 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190917 s, 54.9 MB/s 2024-04-25T09:17:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.578 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318557 s, 32.9 MB/s 2024-04-25T09:17:55.606 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.606 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.606 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229014 s, 45.8 MB/s 2024-04-25T09:17:55.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.633 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.634 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248962 s, 42.1 MB/s 2024-04-25T09:17:55.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.660 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224635 s, 46.7 MB/s 2024-04-25T09:17:55.686 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.687 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218253 s, 48.0 MB/s 2024-04-25T09:17:55.712 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.713 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.713 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022171 s, 47.3 MB/s 2024-04-25T09:17:55.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021493 s, 48.8 MB/s 2024-04-25T09:17:55.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:55.739 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:55.740 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:55.740 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:17:55.740 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:55.740 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:55.740 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:55.740 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:55.825 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211959 s, 49.5 MB/s 2024-04-25T09:17:55.855 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.856 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261009 s, 40.2 MB/s 2024-04-25T09:17:55.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.886 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265361 s, 39.5 MB/s 2024-04-25T09:17:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238812 s, 43.9 MB/s 2024-04-25T09:17:55.941 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.941 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.941 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228204 s, 45.9 MB/s 2024-04-25T09:17:55.967 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.967 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.967 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216559 s, 48.4 MB/s 2024-04-25T09:17:55.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:55.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:55.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201276 s, 52.1 MB/s 2024-04-25T09:17:56.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.017 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218437 s, 48.0 MB/s 2024-04-25T09:17:56.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.041 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203206 s, 51.6 MB/s 2024-04-25T09:17:56.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.064 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.064 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193621 s, 54.2 MB/s 2024-04-25T09:17:56.091 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.092 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.092 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220189 s, 47.6 MB/s 2024-04-25T09:17:56.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205878 s, 50.9 MB/s 2024-04-25T09:17:56.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.144 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239007 s, 43.9 MB/s 2024-04-25T09:17:56.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.170 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232641 s, 45.1 MB/s 2024-04-25T09:17:56.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.195 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.195 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199399 s, 52.6 MB/s 2024-04-25T09:17:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.222 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229384 s, 45.7 MB/s 2024-04-25T09:17:56.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.251 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248512 s, 42.2 MB/s 2024-04-25T09:17:56.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.278 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219728 s, 47.7 MB/s 2024-04-25T09:17:56.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.300 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183281 s, 57.2 MB/s 2024-04-25T09:17:56.325 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.325 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.325 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200988 s, 52.2 MB/s 2024-04-25T09:17:56.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.346 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188424 s, 55.6 MB/s 2024-04-25T09:17:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.373 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223445 s, 46.9 MB/s 2024-04-25T09:17:56.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.398 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211454 s, 49.6 MB/s 2024-04-25T09:17:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.423 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01986 s, 52.8 MB/s 2024-04-25T09:17:56.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.452 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.452 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244985 s, 42.8 MB/s 2024-04-25T09:17:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.478 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220671 s, 47.5 MB/s 2024-04-25T09:17:56.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.504 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.504 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220311 s, 47.6 MB/s 2024-04-25T09:17:56.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.530 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.530 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222546 s, 47.1 MB/s 2024-04-25T09:17:56.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.555 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.555 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209685 s, 50.0 MB/s 2024-04-25T09:17:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.586 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265663 s, 39.5 MB/s 2024-04-25T09:17:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.614 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246424 s, 42.6 MB/s 2024-04-25T09:17:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.642 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239015 s, 43.9 MB/s 2024-04-25T09:17:56.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:56.645 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:56.645 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:56.645 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:17:56.645 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:56.645 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:56.645 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:56.645 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:56.724 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188521 s, 55.6 MB/s 2024-04-25T09:17:56.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.751 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.751 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231207 s, 45.4 MB/s 2024-04-25T09:17:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.775 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217962 s, 48.1 MB/s 2024-04-25T09:17:56.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.796 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.796 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158442 s, 66.2 MB/s 2024-04-25T09:17:56.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.818 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019631 s, 53.4 MB/s 2024-04-25T09:17:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.847 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025497 s, 41.1 MB/s 2024-04-25T09:17:56.873 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.873 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.873 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212963 s, 49.2 MB/s 2024-04-25T09:17:56.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.897 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.897 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202272 s, 51.8 MB/s 2024-04-25T09:17:56.924 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.925 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.925 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237578 s, 44.1 MB/s 2024-04-25T09:17:56.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.952 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.952 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221143 s, 47.4 MB/s 2024-04-25T09:17:56.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:56.980 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:56.980 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234465 s, 44.7 MB/s 2024-04-25T09:17:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.005 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209853 s, 50.0 MB/s 2024-04-25T09:17:57.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.032 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.032 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223432 s, 46.9 MB/s 2024-04-25T09:17:57.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.057 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.058 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215375 s, 48.7 MB/s 2024-04-25T09:17:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.084 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224084 s, 46.8 MB/s 2024-04-25T09:17:57.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256423 s, 40.9 MB/s 2024-04-25T09:17:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.137 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196965 s, 53.2 MB/s 2024-04-25T09:17:57.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.163 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215544 s, 48.6 MB/s 2024-04-25T09:17:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226602 s, 46.3 MB/s 2024-04-25T09:17:57.219 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.219 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.219 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257163 s, 40.8 MB/s 2024-04-25T09:17:57.249 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.250 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.250 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254353 s, 41.2 MB/s 2024-04-25T09:17:57.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.276 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221731 s, 47.3 MB/s 2024-04-25T09:17:57.301 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.302 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.302 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215226 s, 48.7 MB/s 2024-04-25T09:17:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.327 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210978 s, 49.7 MB/s 2024-04-25T09:17:57.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.357 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.357 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259729 s, 40.4 MB/s 2024-04-25T09:17:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.383 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213139 s, 49.2 MB/s 2024-04-25T09:17:57.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.407 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196324 s, 53.4 MB/s 2024-04-25T09:17:57.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.432 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219055 s, 47.9 MB/s 2024-04-25T09:17:57.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.455 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.455 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182571 s, 57.4 MB/s 2024-04-25T09:17:57.485 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.486 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.486 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267849 s, 39.1 MB/s 2024-04-25T09:17:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.512 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225146 s, 46.6 MB/s 2024-04-25T09:17:57.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.538 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.538 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211084 s, 49.7 MB/s 2024-04-25T09:17:57.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:57.540 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:57.540 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:57.541 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:17:57.541 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:57.541 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:57.541 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:57.541 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:57.632 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.632 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.633 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236795 s, 44.3 MB/s 2024-04-25T09:17:57.661 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.661 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.661 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024392 s, 43.0 MB/s 2024-04-25T09:17:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.685 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.686 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200825 s, 52.2 MB/s 2024-04-25T09:17:57.713 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.714 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023343 s, 44.9 MB/s 2024-04-25T09:17:57.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.743 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253956 s, 41.3 MB/s 2024-04-25T09:17:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.772 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234271 s, 44.8 MB/s 2024-04-25T09:17:57.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.798 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.798 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022264 s, 47.1 MB/s 2024-04-25T09:17:57.824 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.824 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.824 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214268 s, 48.9 MB/s 2024-04-25T09:17:57.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.853 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243562 s, 43.1 MB/s 2024-04-25T09:17:57.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.884 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266351 s, 39.4 MB/s 2024-04-25T09:17:57.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.911 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230462 s, 45.5 MB/s 2024-04-25T09:17:57.941 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.941 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.941 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259361 s, 40.4 MB/s 2024-04-25T09:17:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209762 s, 50.0 MB/s 2024-04-25T09:17:57.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:57.993 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227279 s, 46.1 MB/s 2024-04-25T09:17:58.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.018 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.018 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207356 s, 50.6 MB/s 2024-04-25T09:17:58.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.048 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.048 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253703 s, 41.3 MB/s 2024-04-25T09:17:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.073 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213894 s, 49.0 MB/s 2024-04-25T09:17:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.100 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220525 s, 47.5 MB/s 2024-04-25T09:17:58.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203939 s, 51.4 MB/s 2024-04-25T09:17:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.151 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221679 s, 47.3 MB/s 2024-04-25T09:17:58.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.178 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.178 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216136 s, 48.5 MB/s 2024-04-25T09:17:58.206 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.207 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247893 s, 42.3 MB/s 2024-04-25T09:17:58.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.237 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.237 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262929 s, 39.9 MB/s 2024-04-25T09:17:58.265 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.266 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239779 s, 43.7 MB/s 2024-04-25T09:17:58.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.289 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.289 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02066 s, 50.8 MB/s 2024-04-25T09:17:58.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.320 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.320 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264364 s, 39.7 MB/s 2024-04-25T09:17:58.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.346 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.346 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212889 s, 49.3 MB/s 2024-04-25T09:17:58.372 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.372 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.373 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218679 s, 48.0 MB/s 2024-04-25T09:17:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.399 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023246 s, 45.1 MB/s 2024-04-25T09:17:58.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.424 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.424 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209319 s, 50.1 MB/s 2024-04-25T09:17:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.450 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.451 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220252 s, 47.6 MB/s 2024-04-25T09:17:58.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.477 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.478 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234847 s, 44.6 MB/s 2024-04-25T09:17:58.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:58.480 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:58.480 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:58.481 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:17:58.481 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:58.481 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:58.481 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:58.481 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.569 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236611 s, 44.3 MB/s 2024-04-25T09:17:58.597 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.598 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235003 s, 44.6 MB/s 2024-04-25T09:17:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.625 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231058 s, 45.4 MB/s 2024-04-25T09:17:58.655 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.655 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.655 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260897 s, 40.2 MB/s 2024-04-25T09:17:58.684 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.684 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.684 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253132 s, 41.4 MB/s 2024-04-25T09:17:58.713 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.713 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.713 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238401 s, 44.0 MB/s 2024-04-25T09:17:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.736 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192349 s, 54.5 MB/s 2024-04-25T09:17:58.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.755 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.755 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159204 s, 65.9 MB/s 2024-04-25T09:17:58.778 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.779 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.779 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196228 s, 53.4 MB/s 2024-04-25T09:17:58.805 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.806 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.806 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229112 s, 45.8 MB/s 2024-04-25T09:17:58.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.830 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201172 s, 52.1 MB/s 2024-04-25T09:17:58.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.856 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.856 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209641 s, 50.0 MB/s 2024-04-25T09:17:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.884 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239384 s, 43.8 MB/s 2024-04-25T09:17:58.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.911 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.911 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023742 s, 44.2 MB/s 2024-04-25T09:17:58.940 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.940 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.940 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253715 s, 41.3 MB/s 2024-04-25T09:17:58.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.967 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.967 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216614 s, 48.4 MB/s 2024-04-25T09:17:58.994 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:58.995 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238243 s, 44.0 MB/s 2024-04-25T09:17:59.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.021 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.021 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223902 s, 46.8 MB/s 2024-04-25T09:17:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.045 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198779 s, 52.8 MB/s 2024-04-25T09:17:59.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.070 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206757 s, 50.7 MB/s 2024-04-25T09:17:59.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.096 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.096 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224065 s, 46.8 MB/s 2024-04-25T09:17:59.122 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.122 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.122 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221841 s, 47.3 MB/s 2024-04-25T09:17:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220888 s, 47.5 MB/s 2024-04-25T09:17:59.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.170 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.170 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180636 s, 58.0 MB/s 2024-04-25T09:17:59.196 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.197 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.197 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219314 s, 47.8 MB/s 2024-04-25T09:17:59.224 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.224 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.224 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233274 s, 45.0 MB/s 2024-04-25T09:17:59.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.251 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022365 s, 46.9 MB/s 2024-04-25T09:17:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.279 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233737 s, 44.9 MB/s 2024-04-25T09:17:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.306 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230979 s, 45.4 MB/s 2024-04-25T09:17:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.330 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191764 s, 54.7 MB/s 2024-04-25T09:17:59.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.355 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.355 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198113 s, 52.9 MB/s 2024-04-25T09:17:59.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021138 s, 49.6 MB/s 2024-04-25T09:17:59.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:17:59.383 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:17:59.383 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:17:59.383 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:17:59.383 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:17:59.383 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:17:59.383 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:17:59.383 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:17:59.470 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.470 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.471 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225006 s, 46.6 MB/s 2024-04-25T09:17:59.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.494 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.494 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194568 s, 53.9 MB/s 2024-04-25T09:17:59.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.522 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.522 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233746 s, 44.9 MB/s 2024-04-25T09:17:59.544 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.545 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186634 s, 56.2 MB/s 2024-04-25T09:17:59.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212204 s, 49.4 MB/s 2024-04-25T09:17:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.600 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265545 s, 39.5 MB/s 2024-04-25T09:17:59.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.627 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225822 s, 46.4 MB/s 2024-04-25T09:17:59.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.651 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.651 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214764 s, 48.8 MB/s 2024-04-25T09:17:59.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.678 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.678 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230346 s, 45.5 MB/s 2024-04-25T09:17:59.705 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.705 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.706 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234188 s, 44.8 MB/s 2024-04-25T09:17:59.733 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.734 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.734 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236472 s, 44.3 MB/s 2024-04-25T09:17:59.759 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.759 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.759 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210658 s, 49.8 MB/s 2024-04-25T09:17:59.788 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.789 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.789 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234978 s, 44.6 MB/s 2024-04-25T09:17:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196994 s, 53.2 MB/s 2024-04-25T09:17:59.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.837 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.837 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225559 s, 46.5 MB/s 2024-04-25T09:17:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.863 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221982 s, 47.2 MB/s 2024-04-25T09:17:59.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.892 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.892 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237705 s, 44.1 MB/s 2024-04-25T09:17:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.917 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186164 s, 56.3 MB/s 2024-04-25T09:17:59.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.939 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204511 s, 51.3 MB/s 2024-04-25T09:17:59.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232163 s, 45.2 MB/s 2024-04-25T09:17:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:17:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:17:59.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180057 s, 58.2 MB/s 2024-04-25T09:18:00.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.019 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270894 s, 38.7 MB/s 2024-04-25T09:18:00.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.046 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195329 s, 53.7 MB/s 2024-04-25T09:18:00.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.073 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214064 s, 49.0 MB/s 2024-04-25T09:18:00.097 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.097 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.097 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215978 s, 48.6 MB/s 2024-04-25T09:18:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.124 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256484 s, 40.9 MB/s 2024-04-25T09:18:00.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.151 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230696 s, 45.5 MB/s 2024-04-25T09:18:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.176 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203285 s, 51.6 MB/s 2024-04-25T09:18:00.200 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.201 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217503 s, 48.2 MB/s 2024-04-25T09:18:00.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221532 s, 47.3 MB/s 2024-04-25T09:18:00.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.252 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206577 s, 50.8 MB/s 2024-04-25T09:18:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.278 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227012 s, 46.2 MB/s 2024-04-25T09:18:00.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:00.281 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:00.281 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:00.281 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:18:00.281 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:00.281 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:00.282 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:00.282 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:00.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.370 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.370 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229612 s, 45.7 MB/s 2024-04-25T09:18:00.397 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.399 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228865 s, 45.8 MB/s 2024-04-25T09:18:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.432 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310147 s, 33.8 MB/s 2024-04-25T09:18:00.458 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.459 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219445 s, 47.8 MB/s 2024-04-25T09:18:00.484 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.485 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.485 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204844 s, 51.2 MB/s 2024-04-25T09:18:00.512 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.513 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.513 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228976 s, 45.8 MB/s 2024-04-25T09:18:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.540 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231395 s, 45.3 MB/s 2024-04-25T09:18:00.565 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.565 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.565 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206894 s, 50.7 MB/s 2024-04-25T09:18:00.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.588 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.588 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187186 s, 56.0 MB/s 2024-04-25T09:18:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.614 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213248 s, 49.2 MB/s 2024-04-25T09:18:00.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.639 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.639 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216012 s, 48.5 MB/s 2024-04-25T09:18:00.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.667 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240881 s, 43.5 MB/s 2024-04-25T09:18:00.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.693 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.693 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224016 s, 46.8 MB/s 2024-04-25T09:18:00.719 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.720 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.720 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223166 s, 47.0 MB/s 2024-04-25T09:18:00.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.743 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.743 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198455 s, 52.8 MB/s 2024-04-25T09:18:00.764 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.765 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178162 s, 58.9 MB/s 2024-04-25T09:18:00.791 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.792 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223632 s, 46.9 MB/s 2024-04-25T09:18:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.813 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180715 s, 58.0 MB/s 2024-04-25T09:18:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.835 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193151 s, 54.3 MB/s 2024-04-25T09:18:00.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.860 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209212 s, 50.1 MB/s 2024-04-25T09:18:00.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.887 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228984 s, 45.8 MB/s 2024-04-25T09:18:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.914 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245211 s, 42.8 MB/s 2024-04-25T09:18:00.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.939 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.939 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196573 s, 53.3 MB/s 2024-04-25T09:18:00.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207678 s, 50.5 MB/s 2024-04-25T09:18:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:00.992 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233745 s, 44.9 MB/s 2024-04-25T09:18:01.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.019 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.019 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232163 s, 45.2 MB/s 2024-04-25T09:18:01.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.046 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227135 s, 46.2 MB/s 2024-04-25T09:18:01.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.073 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.073 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214244 s, 48.9 MB/s 2024-04-25T09:18:01.097 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.097 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.097 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199578 s, 52.5 MB/s 2024-04-25T09:18:01.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.126 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.126 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249888 s, 42.0 MB/s 2024-04-25T09:18:01.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.151 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.151 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207887 s, 50.4 MB/s 2024-04-25T09:18:01.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.175 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205348 s, 51.1 MB/s 2024-04-25T09:18:01.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:01.179 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:01.179 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:01.179 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:18:01.179 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:01.179 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:01.179 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:01.179 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.260 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019525 s, 53.7 MB/s 2024-04-25T09:18:01.290 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.290 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.290 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254835 s, 41.1 MB/s 2024-04-25T09:18:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.316 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.317 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022232 s, 47.2 MB/s 2024-04-25T09:18:01.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.343 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.343 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224846 s, 46.6 MB/s 2024-04-25T09:18:01.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.371 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242579 s, 43.2 MB/s 2024-04-25T09:18:01.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.396 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.396 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213723 s, 49.1 MB/s 2024-04-25T09:18:01.421 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.422 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.422 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217846 s, 48.1 MB/s 2024-04-25T09:18:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.446 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203358 s, 51.6 MB/s 2024-04-25T09:18:01.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.472 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.472 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224114 s, 46.8 MB/s 2024-04-25T09:18:01.495 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.495 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.495 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189689 s, 55.3 MB/s 2024-04-25T09:18:01.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.518 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.519 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018861 s, 55.6 MB/s 2024-04-25T09:18:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177392 s, 59.1 MB/s 2024-04-25T09:18:01.562 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.562 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.562 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188196 s, 55.7 MB/s 2024-04-25T09:18:01.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.592 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.592 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261944 s, 40.0 MB/s 2024-04-25T09:18:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.619 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229436 s, 45.7 MB/s 2024-04-25T09:18:01.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.648 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.648 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246018 s, 42.6 MB/s 2024-04-25T09:18:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194176 s, 54.0 MB/s 2024-04-25T09:18:01.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.697 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.697 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206454 s, 50.8 MB/s 2024-04-25T09:18:01.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.725 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.725 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231776 s, 45.2 MB/s 2024-04-25T09:18:01.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.747 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165061 s, 63.5 MB/s 2024-04-25T09:18:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022811 s, 46.0 MB/s 2024-04-25T09:18:01.799 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.799 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.799 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207743 s, 50.5 MB/s 2024-04-25T09:18:01.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.823 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.823 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02029 s, 51.7 MB/s 2024-04-25T09:18:01.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.853 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250194 s, 41.9 MB/s 2024-04-25T09:18:01.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.883 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257707 s, 40.7 MB/s 2024-04-25T09:18:01.908 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207795 s, 50.5 MB/s 2024-04-25T09:18:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.938 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.939 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256802 s, 40.8 MB/s 2024-04-25T09:18:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.964 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215766 s, 48.6 MB/s 2024-04-25T09:18:01.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:01.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:01.990 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225119 s, 46.6 MB/s 2024-04-25T09:18:02.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.014 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.014 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194264 s, 54.0 MB/s 2024-04-25T09:18:02.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.041 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.041 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023461 s, 44.7 MB/s 2024-04-25T09:18:02.070 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.071 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.071 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243502 s, 43.1 MB/s 2024-04-25T09:18:02.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:02.073 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:02.073 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:02.073 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:18:02.073 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:02.073 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:02.073 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:02.073 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:02.162 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.163 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.163 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217785 s, 48.1 MB/s 2024-04-25T09:18:02.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228834 s, 45.8 MB/s 2024-04-25T09:18:02.215 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.216 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.216 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216115 s, 48.5 MB/s 2024-04-25T09:18:02.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.244 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237019 s, 44.2 MB/s 2024-04-25T09:18:02.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.269 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218667 s, 48.0 MB/s 2024-04-25T09:18:02.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.299 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.299 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257404 s, 40.7 MB/s 2024-04-25T09:18:02.329 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.330 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.330 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257966 s, 40.6 MB/s 2024-04-25T09:18:02.351 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.352 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.352 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186768 s, 56.1 MB/s 2024-04-25T09:18:02.381 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.382 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262194 s, 40.0 MB/s 2024-04-25T09:18:02.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.410 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242853 s, 43.2 MB/s 2024-04-25T09:18:02.436 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.437 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.437 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229294 s, 45.7 MB/s 2024-04-25T09:18:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.466 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257429 s, 40.7 MB/s 2024-04-25T09:18:02.496 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.496 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.496 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024839 s, 42.2 MB/s 2024-04-25T09:18:02.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.523 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.523 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232417 s, 45.1 MB/s 2024-04-25T09:18:02.556 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.556 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.556 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293829 s, 35.7 MB/s 2024-04-25T09:18:02.582 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.582 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.582 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214499 s, 48.9 MB/s 2024-04-25T09:18:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.607 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208927 s, 50.2 MB/s 2024-04-25T09:18:02.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.638 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.638 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270167 s, 38.8 MB/s 2024-04-25T09:18:02.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.667 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255535 s, 41.0 MB/s 2024-04-25T09:18:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278794 s, 37.6 MB/s 2024-04-25T09:18:02.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.727 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.727 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230633 s, 45.5 MB/s 2024-04-25T09:18:02.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.754 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230965 s, 45.4 MB/s 2024-04-25T09:18:02.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.781 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.781 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231217 s, 45.4 MB/s 2024-04-25T09:18:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.809 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240034 s, 43.7 MB/s 2024-04-25T09:18:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.833 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214463 s, 48.9 MB/s 2024-04-25T09:18:02.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.859 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.859 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221727 s, 47.3 MB/s 2024-04-25T09:18:02.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.883 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.883 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199039 s, 52.7 MB/s 2024-04-25T09:18:02.909 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.910 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.910 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221332 s, 47.4 MB/s 2024-04-25T09:18:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.937 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229308 s, 45.7 MB/s 2024-04-25T09:18:02.965 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.966 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.966 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241558 s, 43.4 MB/s 2024-04-25T09:18:02.990 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:02.991 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:02.991 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022103 s, 47.4 MB/s 2024-04-25T09:18:03.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.016 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.016 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207137 s, 50.6 MB/s 2024-04-25T09:18:03.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:03.018 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:03.018 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:03.019 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:18:03.019 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:03.019 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:03.019 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:03.019 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.098 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212722 s, 49.3 MB/s 2024-04-25T09:18:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.123 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224236 s, 46.8 MB/s 2024-04-25T09:18:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.150 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218535 s, 48.0 MB/s 2024-04-25T09:18:03.175 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.176 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212137 s, 49.4 MB/s 2024-04-25T09:18:03.201 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.202 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.202 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220045 s, 47.7 MB/s 2024-04-25T09:18:03.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.227 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.227 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210373 s, 49.8 MB/s 2024-04-25T09:18:03.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.254 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223848 s, 46.8 MB/s 2024-04-25T09:18:03.278 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.279 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.279 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202718 s, 51.7 MB/s 2024-04-25T09:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.306 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219964 s, 47.7 MB/s 2024-04-25T09:18:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.337 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.338 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276113 s, 38.0 MB/s 2024-04-25T09:18:03.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.362 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.362 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187259 s, 56.0 MB/s 2024-04-25T09:18:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.392 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267565 s, 39.2 MB/s 2024-04-25T09:18:03.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.420 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.420 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221151 s, 47.4 MB/s 2024-04-25T09:18:03.444 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:03.445 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210081 s, 49.9 MB/s 2024-04-25T09:18:04.382 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.383 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.383 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.934514 s, 1.1 MB/s 2024-04-25T09:18:04.419 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.420 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.420 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032692 s, 32.1 MB/s 2024-04-25T09:18:04.447 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.448 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236574 s, 44.3 MB/s 2024-04-25T09:18:04.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.480 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.480 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279006 s, 37.6 MB/s 2024-04-25T09:18:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.514 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305097 s, 34.4 MB/s 2024-04-25T09:18:04.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.545 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266288 s, 39.4 MB/s 2024-04-25T09:18:04.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.574 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.574 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237017 s, 44.2 MB/s 2024-04-25T09:18:04.609 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.610 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.610 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309303 s, 33.9 MB/s 2024-04-25T09:18:04.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.640 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.641 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261825 s, 40.0 MB/s 2024-04-25T09:18:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.668 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233566 s, 44.9 MB/s 2024-04-25T09:18:04.692 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.692 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.692 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197161 s, 53.2 MB/s 2024-04-25T09:18:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.721 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238603 s, 43.9 MB/s 2024-04-25T09:18:04.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.746 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.746 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218658 s, 48.0 MB/s 2024-04-25T09:18:04.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.771 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.771 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211843 s, 49.5 MB/s 2024-04-25T09:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.793 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174552 s, 60.1 MB/s 2024-04-25T09:18:04.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.817 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.817 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192392 s, 54.5 MB/s 2024-04-25T09:18:04.842 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.843 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.843 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203098 s, 51.6 MB/s 2024-04-25T09:18:04.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.869 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.869 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224968 s, 46.6 MB/s 2024-04-25T09:18:04.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:04.872 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:04.872 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:04.872 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:18:04.872 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:04.872 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:04.872 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:04.872 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:04.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.960 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02304 s, 45.5 MB/s 2024-04-25T09:18:04.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:04.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:04.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023121 s, 45.4 MB/s 2024-04-25T09:18:05.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.009 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194562 s, 53.9 MB/s 2024-04-25T09:18:05.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.035 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.035 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211265 s, 49.6 MB/s 2024-04-25T09:18:05.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023168 s, 45.3 MB/s 2024-04-25T09:18:05.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.090 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251 s, 41.8 MB/s 2024-04-25T09:18:05.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.120 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.120 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259702 s, 40.4 MB/s 2024-04-25T09:18:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.148 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237485 s, 44.2 MB/s 2024-04-25T09:18:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.174 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207667 s, 50.5 MB/s 2024-04-25T09:18:05.200 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.200 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.200 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216384 s, 48.5 MB/s 2024-04-25T09:18:05.231 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.232 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.232 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277777 s, 37.7 MB/s 2024-04-25T09:18:05.258 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.259 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231703 s, 45.3 MB/s 2024-04-25T09:18:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.283 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020326 s, 51.6 MB/s 2024-04-25T09:18:05.310 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.310 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.310 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221466 s, 47.3 MB/s 2024-04-25T09:18:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.335 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212169 s, 49.4 MB/s 2024-04-25T09:18:05.360 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.361 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.361 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216427 s, 48.4 MB/s 2024-04-25T09:18:05.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.386 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210395 s, 49.8 MB/s 2024-04-25T09:18:05.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.410 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.410 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201829 s, 52.0 MB/s 2024-04-25T09:18:05.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.433 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.433 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193678 s, 54.1 MB/s 2024-04-25T09:18:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.462 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234864 s, 44.6 MB/s 2024-04-25T09:18:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.487 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211538 s, 49.6 MB/s 2024-04-25T09:18:05.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.515 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.515 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236017 s, 44.4 MB/s 2024-04-25T09:18:05.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.545 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.545 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246362 s, 42.6 MB/s 2024-04-25T09:18:05.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.573 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.573 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244783 s, 42.8 MB/s 2024-04-25T09:18:05.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.598 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.598 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020847 s, 50.3 MB/s 2024-04-25T09:18:05.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.626 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.626 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240848 s, 43.5 MB/s 2024-04-25T09:18:05.654 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.654 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.654 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024543 s, 42.7 MB/s 2024-04-25T09:18:05.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.677 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.677 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200175 s, 52.4 MB/s 2024-04-25T09:18:05.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.699 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.700 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183973 s, 57.0 MB/s 2024-04-25T09:18:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.726 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224433 s, 46.7 MB/s 2024-04-25T09:18:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.754 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.755 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239341 s, 43.8 MB/s 2024-04-25T09:18:05.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.774 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.774 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163973 s, 63.9 MB/s 2024-04-25T09:18:05.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:05.777 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:05.777 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:05.777 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:18:05.777 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:05.777 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:05.777 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:05.777 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:05.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.864 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.864 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211569 s, 49.6 MB/s 2024-04-25T09:18:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.888 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020343 s, 51.5 MB/s 2024-04-25T09:18:05.915 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.916 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.916 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238399 s, 44.0 MB/s 2024-04-25T09:18:05.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.936 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.936 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174322 s, 60.2 MB/s 2024-04-25T09:18:05.962 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.963 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.963 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229393 s, 45.7 MB/s 2024-04-25T09:18:05.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:05.987 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:05.987 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197234 s, 53.2 MB/s 2024-04-25T09:18:06.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.013 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.013 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02168 s, 48.4 MB/s 2024-04-25T09:18:06.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.037 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.037 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198189 s, 52.9 MB/s 2024-04-25T09:18:06.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.062 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.062 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203951 s, 51.4 MB/s 2024-04-25T09:18:06.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.090 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.090 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237999 s, 44.1 MB/s 2024-04-25T09:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.116 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214936 s, 48.8 MB/s 2024-04-25T09:18:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.146 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261409 s, 40.1 MB/s 2024-04-25T09:18:06.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.173 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.173 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230432 s, 45.5 MB/s 2024-04-25T09:18:06.203 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.204 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258199 s, 40.6 MB/s 2024-04-25T09:18:06.229 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.230 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.230 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223779 s, 46.9 MB/s 2024-04-25T09:18:06.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.254 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.255 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196855 s, 53.3 MB/s 2024-04-25T09:18:06.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.280 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.280 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215974 s, 48.6 MB/s 2024-04-25T09:18:06.304 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.305 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202139 s, 51.9 MB/s 2024-04-25T09:18:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.334 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260348 s, 40.3 MB/s 2024-04-25T09:18:06.360 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.360 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.360 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211015 s, 49.7 MB/s 2024-04-25T09:18:06.385 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.386 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217319 s, 48.3 MB/s 2024-04-25T09:18:06.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.408 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.408 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187338 s, 56.0 MB/s 2024-04-25T09:18:06.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.434 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.434 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218387 s, 48.0 MB/s 2024-04-25T09:18:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.459 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201561 s, 52.0 MB/s 2024-04-25T09:18:06.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.483 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.483 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195459 s, 53.6 MB/s 2024-04-25T09:18:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.509 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218497 s, 48.0 MB/s 2024-04-25T09:18:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.531 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195848 s, 53.5 MB/s 2024-04-25T09:18:06.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.553 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.553 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181787 s, 57.7 MB/s 2024-04-25T09:18:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.577 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201185 s, 52.1 MB/s 2024-04-25T09:18:06.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.603 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213703 s, 49.1 MB/s 2024-04-25T09:18:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.632 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258148 s, 40.6 MB/s 2024-04-25T09:18:06.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.660 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243339 s, 43.1 MB/s 2024-04-25T09:18:06.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:06.663 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:06.663 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:06.663 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:18:06.663 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:06.663 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:06.663 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:06.663 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:06.752 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.752 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.752 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260393 s, 40.3 MB/s 2024-04-25T09:18:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.778 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210259 s, 49.9 MB/s 2024-04-25T09:18:06.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.804 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.804 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232633 s, 45.1 MB/s 2024-04-25T09:18:06.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.830 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.830 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225482 s, 46.5 MB/s 2024-04-25T09:18:06.857 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.858 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226819 s, 46.2 MB/s 2024-04-25T09:18:06.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.886 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.886 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252663 s, 41.5 MB/s 2024-04-25T09:18:06.916 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.917 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.917 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266554 s, 39.3 MB/s 2024-04-25T09:18:06.946 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.946 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.946 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251145 s, 41.8 MB/s 2024-04-25T09:18:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.975 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249474 s, 42.0 MB/s 2024-04-25T09:18:06.997 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:06.997 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:06.998 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199001 s, 52.7 MB/s 2024-04-25T09:18:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.022 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020762 s, 50.5 MB/s 2024-04-25T09:18:07.044 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.045 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192759 s, 54.4 MB/s 2024-04-25T09:18:07.069 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.069 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.069 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019836 s, 52.9 MB/s 2024-04-25T09:18:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.093 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199534 s, 52.6 MB/s 2024-04-25T09:18:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.119 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216116 s, 48.5 MB/s 2024-04-25T09:18:07.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.144 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.144 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214376 s, 48.9 MB/s 2024-04-25T09:18:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.172 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.173 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239264 s, 43.8 MB/s 2024-04-25T09:18:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.199 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224263 s, 46.8 MB/s 2024-04-25T09:18:07.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.225 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.225 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221388 s, 47.4 MB/s 2024-04-25T09:18:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223359 s, 46.9 MB/s 2024-04-25T09:18:07.276 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.277 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.277 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207099 s, 50.6 MB/s 2024-04-25T09:18:07.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.300 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.300 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193839 s, 54.1 MB/s 2024-04-25T09:18:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.324 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188725 s, 55.6 MB/s 2024-04-25T09:18:07.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.350 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.350 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223124 s, 47.0 MB/s 2024-04-25T09:18:07.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.380 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.380 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247119 s, 42.4 MB/s 2024-04-25T09:18:07.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.407 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.407 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230199 s, 45.6 MB/s 2024-04-25T09:18:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.432 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215082 s, 48.8 MB/s 2024-04-25T09:18:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.462 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251768 s, 41.6 MB/s 2024-04-25T09:18:07.489 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.490 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.490 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230484 s, 45.5 MB/s 2024-04-25T09:18:07.516 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.517 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225636 s, 46.5 MB/s 2024-04-25T09:18:07.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.543 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.544 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226649 s, 46.3 MB/s 2024-04-25T09:18:07.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.570 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220766 s, 47.5 MB/s 2024-04-25T09:18:07.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:07.572 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:07.572 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:07.572 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:18:07.572 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:07.572 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:07.572 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:07.572 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:07.660 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.661 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243811 s, 43.0 MB/s 2024-04-25T09:18:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.688 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240474 s, 43.6 MB/s 2024-04-25T09:18:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.711 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.712 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198425 s, 52.8 MB/s 2024-04-25T09:18:07.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.737 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.737 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213152 s, 49.2 MB/s 2024-04-25T09:18:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.767 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026342 s, 39.8 MB/s 2024-04-25T09:18:07.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.795 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.796 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237216 s, 44.2 MB/s 2024-04-25T09:18:07.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.822 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.822 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223135 s, 47.0 MB/s 2024-04-25T09:18:07.850 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.851 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.851 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238673 s, 43.9 MB/s 2024-04-25T09:18:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.878 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228696 s, 45.9 MB/s 2024-04-25T09:18:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.905 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234828 s, 44.7 MB/s 2024-04-25T09:18:07.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.934 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.934 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025298 s, 41.4 MB/s 2024-04-25T09:18:07.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.960 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.960 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217894 s, 48.1 MB/s 2024-04-25T09:18:07.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:07.986 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:07.986 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216272 s, 48.5 MB/s 2024-04-25T09:18:08.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.010 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.010 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197416 s, 53.1 MB/s 2024-04-25T09:18:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.034 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198707 s, 52.8 MB/s 2024-04-25T09:18:08.058 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.058 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.058 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187864 s, 55.8 MB/s 2024-04-25T09:18:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.081 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190783 s, 55.0 MB/s 2024-04-25T09:18:08.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.107 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.108 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022583 s, 46.4 MB/s 2024-04-25T09:18:08.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.132 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.132 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020694 s, 50.7 MB/s 2024-04-25T09:18:08.155 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.155 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.155 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018387 s, 57.0 MB/s 2024-04-25T09:18:08.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.179 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.180 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201854 s, 51.9 MB/s 2024-04-25T09:18:08.208 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.208 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.208 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024339 s, 43.1 MB/s 2024-04-25T09:18:08.234 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.235 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.235 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221938 s, 47.2 MB/s 2024-04-25T09:18:08.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.260 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.260 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214673 s, 48.8 MB/s 2024-04-25T09:18:08.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.286 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.286 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219805 s, 47.7 MB/s 2024-04-25T09:18:08.317 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.317 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.317 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266789 s, 39.3 MB/s 2024-04-25T09:18:08.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.340 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.340 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190356 s, 55.1 MB/s 2024-04-25T09:18:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.371 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263606 s, 39.8 MB/s 2024-04-25T09:18:08.398 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.399 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.399 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219085 s, 47.9 MB/s 2024-04-25T09:18:08.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.426 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.426 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236038 s, 44.4 MB/s 2024-04-25T09:18:08.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.454 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.454 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232165 s, 45.2 MB/s 2024-04-25T09:18:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.479 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210379 s, 49.8 MB/s 2024-04-25T09:18:08.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:08.482 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:18:08.482 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-25T09:18:08.482 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:18:08.482 DEBUG:teuthology.orchestra.run.smithi062:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:18:08.482 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:18:08.482 DEBUG:teuthology.orchestra.run.smithi062:> done 2024-04-25T09:18:08.482 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-25T09:18:08.576 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.576 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.576 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251792 s, 41.6 MB/s 2024-04-25T09:18:08.602 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.603 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.603 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225408 s, 46.5 MB/s 2024-04-25T09:18:08.627 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.627 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.627 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216203 s, 48.5 MB/s 2024-04-25T09:18:08.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.657 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.657 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252305 s, 41.6 MB/s 2024-04-25T09:18:08.680 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.680 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.680 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199039 s, 52.7 MB/s 2024-04-25T09:18:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.710 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254009 s, 41.3 MB/s 2024-04-25T09:18:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.739 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025318 s, 41.4 MB/s 2024-04-25T09:18:08.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.772 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.772 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283993 s, 36.9 MB/s 2024-04-25T09:18:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.800 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245162 s, 42.8 MB/s 2024-04-25T09:18:08.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.826 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.826 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216949 s, 48.3 MB/s 2024-04-25T09:18:08.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.853 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.853 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229734 s, 45.6 MB/s 2024-04-25T09:18:08.876 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.877 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.877 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196691 s, 53.3 MB/s 2024-04-25T09:18:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.902 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213146 s, 49.2 MB/s 2024-04-25T09:18:08.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.927 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.927 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210943 s, 49.7 MB/s 2024-04-25T09:18:08.956 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.956 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.956 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237294 s, 44.2 MB/s 2024-04-25T09:18:08.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:08.981 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:08.981 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214902 s, 48.8 MB/s 2024-04-25T09:18:09.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.006 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.006 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215425 s, 48.7 MB/s 2024-04-25T09:18:09.036 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.036 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.036 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257718 s, 40.7 MB/s 2024-04-25T09:18:09.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.063 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.063 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233842 s, 44.8 MB/s 2024-04-25T09:18:09.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.089 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.089 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223105 s, 47.0 MB/s 2024-04-25T09:18:09.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.114 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.114 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205689 s, 51.0 MB/s 2024-04-25T09:18:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.140 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219104 s, 47.9 MB/s 2024-04-25T09:18:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.164 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210394 s, 49.8 MB/s 2024-04-25T09:18:09.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.190 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.190 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216639 s, 48.4 MB/s 2024-04-25T09:18:09.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.213 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.214 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195887 s, 53.5 MB/s 2024-04-25T09:18:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.243 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251928 s, 41.6 MB/s 2024-04-25T09:18:09.269 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.270 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.270 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227132 s, 46.2 MB/s 2024-04-25T09:18:09.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.297 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.297 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230147 s, 45.6 MB/s 2024-04-25T09:18:09.325 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.325 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.325 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223034 s, 47.0 MB/s 2024-04-25T09:18:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.351 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222773 s, 47.1 MB/s 2024-04-25T09:18:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.379 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233277 s, 44.9 MB/s 2024-04-25T09:18:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T09:18:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T09:18:09.406 INFO:teuthology.orchestra.run.smithi062.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222517 s, 47.1 MB/s 2024-04-25T09:18:09.408 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T09:18:09.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.637+0000 7f16f2400640 1 Processor -- start 2024-04-25T09:18:09.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.638+0000 7f16f2400640 1 -- start start 2024-04-25T09:18:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.639+0000 7f16f2400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16ec06d2c0 0x7f16ec0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.639+0000 7f16f2400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec0cf000 con 0x7f16ec06d6a0 2024-04-25T09:18:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.639+0000 7f16f2400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec0cf2e0 con 0x7f16ec0b31a0 2024-04-25T09:18:09.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.639+0000 7f16f2400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec0b26c0 con 0x7f16ec06d2c0 2024-04-25T09:18:09.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.640+0000 7f16ebe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f16ec06d6a0 0x7f16ec0ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36588/0 (socket says 172.21.15.62:36588) 2024-04-25T09:18:09.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.640+0000 7f16ebe00640 1 -- 172.21.15.62:0/2271104165 learned_addr learned my addr 172.21.15.62:0/2271104165 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.640+0000 7f16f1000640 1 --2- 172.21.15.62:0/2271104165 >> v2:172.21.15.121:3300/0 conn(0x7f16ec06d2c0 0x7f16ec0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.640+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089654877 0 0) 0x7f16ec0cf000 con 0x7f16ec06d6a0 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.640+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d4003520 con 0x7f16ec06d6a0 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1019924418 0 0) 0x7f16d4003520 con 0x7f16ec06d6a0 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 >> v2:172.21.15.121:3300/0 conn(0x7f16ec06d2c0 msgr2=0x7f16ec0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 --2- 172.21.15.62:0/2271104165 >> v2:172.21.15.121:3300/0 conn(0x7f16ec06d2c0 0x7f16ec0ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 >> v1:172.21.15.121:6789/0 conn(0x7f16ec0b31a0 legacy=0x7f16ec0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec0b29a0 con 0x7f16ec06d6a0 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f16d8001670 con 0x7f16ec06d6a0 2024-04-25T09:18:09.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 >> v1:172.21.15.62:6789/0 conn(0x7f16ec06d6a0 legacy=0x7f16ec0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d40039b0 0x7f16d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0b26c0 con 0x7f16d4004800 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d400c060 con 0x7f16d40051c0 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d400c260 con 0x7f16d40039b0 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.641+0000 7f16ebe00640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16f1a00640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16ebe00640 1 -- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d40039b0 msgr2=0x7f16d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16f1000640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d40039b0 0x7f16d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16ebe00640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d40039b0 0x7f16d4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:09.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16ebe00640 1 -- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16d40051c0 msgr2=0x7f16d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:09.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16ebe00640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:09.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16ebe00640 1 -- 172.21.15.62:0/2271104165 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4003520 con 0x7f16d4004800 2024-04-25T09:18:09.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16f1a00640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:09.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.642+0000 7f16f1a00640 1 -- 172.21.15.62:0/2271104165 reap_dead start 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.643+0000 7f16ebe00640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f16d800c240 tx=0x7f16d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88e616d3507033f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.643+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8007d10 con 0x7f16d4004800 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.643+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8005ad0 con 0x7f16d4004800 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.643+0000 7f16eb400640 1 -- 172.21.15.62:0/2271104165 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8007070 con 0x7f16d4004800 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.644+0000 7f16f2400640 1 -- 172.21.15.62:0/2271104165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d4004800 msgr2=0x7f16d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.644+0000 7f16f2400640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f16d800c240 tx=0x7f16d4005830 comp rx=0 tx=0).stop 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.644+0000 7f16f2400640 1 -- 172.21.15.62:0/2271104165 shutdown_connections 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.644+0000 7f16f2400640 1 --2- 172.21.15.62:0/2271104165 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:09.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.644+0000 7f16f2400640 1 -- 172.21.15.62:0/2271104165 >> 172.21.15.62:0/2271104165 conn(0x7f16ec066f20 msgr2=0x7f16ec0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:09.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.644+0000 7f16f2400640 1 -- 172.21.15.62:0/2271104165 shutdown_connections 2024-04-25T09:18:09.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.644+0000 7f16f2400640 1 -- 172.21.15.62:0/2271104165 wait complete. 2024-04-25T09:18:09.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.646+0000 7f16f2400640 1 Processor -- start 2024-04-25T09:18:09.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.646+0000 7f16f2400640 1 -- start start 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f2400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d4004800 0x7f16ec1f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f2400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16ec1f4300 0x7f16ec1f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f2400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16ec13fa10 0x7f16ec0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f2400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1448d0 con 0x7f16ec13fa10 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f2400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec06db20 con 0x7f16ec1f4300 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f2400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec144a50 con 0x7f16d4004800 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f1a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16ec13fa10 0x7f16ec0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f1a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16ec13fa10 0x7f16ec0ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45142/0 (socket says 172.21.15.62:45142) 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f1a00640 1 -- 172.21.15.62:0/4293307454 learned_addr learned my addr 172.21.15.62:0/4293307454 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.647+0000 7f16f1000640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d4004800 0x7f16ec1f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16ebe00640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16ec1f4300 0x7f16ec1f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:09.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16f1a00640 1 -- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d4004800 msgr2=0x7f16ec1f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:09.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16f1a00640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d4004800 0x7f16ec1f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:09.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16f1a00640 1 -- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16ec1f4300 msgr2=0x7f16ec1f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:09.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16f1a00640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16ec1f4300 0x7f16ec1f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:09.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16f1a00640 1 -- 172.21.15.62:0/4293307454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec1f4a50 con 0x7f16ec13fa10 2024-04-25T09:18:09.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16ebe00640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16ec1f4300 0x7f16ec1f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:09.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16f1a00640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16ec13fa10 0x7f16ec0ad8e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0af300 tx=0x7f16e400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:09.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16e8c00640 1 -- 172.21.15.62:0/4293307454 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4018070 con 0x7f16ec13fa10 2024-04-25T09:18:09.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16e8c00640 1 -- 172.21.15.62:0/4293307454 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e4003da0 con 0x7f16ec13fa10 2024-04-25T09:18:09.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16e8c00640 1 -- 172.21.15.62:0/4293307454 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e40052b0 con 0x7f16ec13fa10 2024-04-25T09:18:09.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec1f4d30 con 0x7f16ec13fa10 2024-04-25T09:18:09.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.648+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1f52f0 con 0x7f16ec13fa10 2024-04-25T09:18:09.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.650+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ec06abe0 con 0x7f16ec13fa10 2024-04-25T09:18:09.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.650+0000 7f16e8c00640 1 -- 172.21.15.62:0/4293307454 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16e401b070 con 0x7f16ec13fa10 2024-04-25T09:18:09.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.651+0000 7f16e8c00640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f16cc06b520 0x7f16cc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:09.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.651+0000 7f16e8c00640 1 -- 172.21.15.62:0/4293307454 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f16e4089870 con 0x7f16ec13fa10 2024-04-25T09:18:09.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.652+0000 7f16f1000640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f16cc06b520 0x7f16cc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:09.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.653+0000 7f16f1000640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f16cc06b520 0x7f16cc06d9e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009a90 tx=0x7f16e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:09.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.656+0000 7f16e8c00640 1 -- 172.21.15.62:0/4293307454 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16e4058720 con 0x7f16ec13fa10 2024-04-25T09:18:09.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:09.895+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f16ec04ef20 con 0x7f16ec13fa10 2024-04-25T09:18:10.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.395+0000 7f16e8c00640 1 -- 172.21.15.62:0/4293307454 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v100) v1 ==== 105+0+0 (secure 0 0 0) 0x7f16e4058020 con 0x7f16ec13fa10 2024-04-25T09:18:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f16cc06b520 msgr2=0x7f16cc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f16cc06b520 0x7f16cc06d9e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009a90 tx=0x7f16e0008040 comp rx=0 tx=0).stop 2024-04-25T09:18:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16ec13fa10 msgr2=0x7f16ec0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16ec13fa10 0x7f16ec0ad8e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0af300 tx=0x7f16e400d9d0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 shutdown_connections 2024-04-25T09:18:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f16cc06b520 0x7f16cc06d9e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16ec13fa10 0x7f16ec0ad8e0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16ec1f4300 0x7f16ec1f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 --2- 172.21.15.62:0/4293307454 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f16d4004800 0x7f16ec1f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 >> 172.21.15.62:0/4293307454 conn(0x7f16ec066f20 msgr2=0x7f16ec15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 shutdown_connections 2024-04-25T09:18:10.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.399+0000 7f16f2400640 1 -- 172.21.15.62:0/4293307454 wait complete. 2024-04-25T09:18:10.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T09:18:10.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.654+0000 7f82dd800640 1 Processor -- start 2024-04-25T09:18:10.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.654+0000 7f82dd800640 1 -- start start 2024-04-25T09:18:10.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.655+0000 7f82dd800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d815dd50 0x7f82d80d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:10.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.655+0000 7f82dd800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82d80d2fb0 0x7f82d80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:10.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.655+0000 7f82dd800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82d80d3960 0x7f82d80d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:10.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.655+0000 7f82dd800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d80ce780 con 0x7f82d80d3960 2024-04-25T09:18:10.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.655+0000 7f82dd800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d80d4310 con 0x7f82d80d2fb0 2024-04-25T09:18:10.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.655+0000 7f82dd800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80d3440 con 0x7f82d815dd50 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82dce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82d80d3960 0x7f82d80d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82dce00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f82d80d3960 0x7f82d80d3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45152/0 (socket says 172.21.15.62:45152) 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82dce00640 1 -- 172.21.15.62:0/4238995986 learned_addr learned my addr 172.21.15.62:0/4238995986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82d7400640 1 --2- 172.21.15.62:0/4238995986 >> v2:172.21.15.121:3300/0 conn(0x7f82d80d2fb0 0x7f82d80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82d7e00640 1 --2- 172.21.15.62:0/4238995986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d815dd50 0x7f82d80d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82dce00640 1 -- 172.21.15.62:0/4238995986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d815dd50 msgr2=0x7f82d80d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82dce00640 1 --2- 172.21.15.62:0/4238995986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d815dd50 0x7f82d80d2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82dce00640 1 -- 172.21.15.62:0/4238995986 >> v2:172.21.15.121:3300/0 conn(0x7f82d80d2fb0 msgr2=0x7f82d80ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82dce00640 1 --2- 172.21.15.62:0/4238995986 >> v2:172.21.15.121:3300/0 conn(0x7f82d80d2fb0 0x7f82d80ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.656+0000 7f82dce00640 1 -- 172.21.15.62:0/4238995986 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80ce1c0 con 0x7f82d80d3960 2024-04-25T09:18:10.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.657+0000 7f82dce00640 1 --2- 172.21.15.62:0/4238995986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d80d3960 0x7f82d80d3dd0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f82d8072f60 tx=0x7f82d002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d8a0836ac26b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:10.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.658+0000 7f82d6a00640 1 -- 172.21.15.62:0/4238995986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0036070 con 0x7f82d80d3960 2024-04-25T09:18:10.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.658+0000 7f82d6a00640 1 -- 172.21.15.62:0/4238995986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0031600 con 0x7f82d80d3960 2024-04-25T09:18:10.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.658+0000 7f82d6a00640 1 -- 172.21.15.62:0/4238995986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d00319d0 con 0x7f82d80d3960 2024-04-25T09:18:10.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.658+0000 7f82dd800640 1 -- 172.21.15.62:0/4238995986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d80d3960 msgr2=0x7f82d80d3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:10.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.658+0000 7f82dd800640 1 --2- 172.21.15.62:0/4238995986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d80d3960 0x7f82d80d3dd0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f82d8072f60 tx=0x7f82d002a6c0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.659+0000 7f82dd800640 1 -- 172.21.15.62:0/4238995986 shutdown_connections 2024-04-25T09:18:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.659+0000 7f82dd800640 1 --2- 172.21.15.62:0/4238995986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d80d3960 0x7f82d80d3dd0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.659+0000 7f82dd800640 1 --2- 172.21.15.62:0/4238995986 >> v2:172.21.15.121:3300/0 conn(0x7f82d80d2fb0 0x7f82d80ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.659+0000 7f82dd800640 1 --2- 172.21.15.62:0/4238995986 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d815dd50 0x7f82d80d2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.659+0000 7f82dd800640 1 -- 172.21.15.62:0/4238995986 >> 172.21.15.62:0/4238995986 conn(0x7f82d806b000 msgr2=0x7f82d806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.659+0000 7f82dd800640 1 -- 172.21.15.62:0/4238995986 shutdown_connections 2024-04-25T09:18:10.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.659+0000 7f82dd800640 1 -- 172.21.15.62:0/4238995986 wait complete. 2024-04-25T09:18:10.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.660+0000 7f82dd800640 1 Processor -- start 2024-04-25T09:18:10.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.661+0000 7f82dd800640 1 -- start start 2024-04-25T09:18:10.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.661+0000 7f82dd800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d80d3960 0x7f82d80b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82dd800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d815dd50 0x7f82d814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82dd800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d80c9f20 0x7f82d80ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82dd800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80cef00 con 0x7f82d815dd50 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82dd800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80d2840 con 0x7f82d80d3960 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82dd800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80b6b00 con 0x7f82d80c9f20 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d815dd50 0x7f82d814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d815dd50 0x7f82d814fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45168/0 (socket says 172.21.15.62:45168) 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7400640 1 -- 172.21.15.62:0/1064604935 learned_addr learned my addr 172.21.15.62:0/1064604935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:10.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82dce00640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d80c9f20 0x7f82d80ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7e00640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d80d3960 0x7f82d80b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7400640 1 -- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d80c9f20 msgr2=0x7f82d80ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7400640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d80c9f20 0x7f82d80ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7400640 1 -- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d80d3960 msgr2=0x7f82d80b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7400640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d80d3960 0x7f82d80b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82d7400640 1 -- 172.21.15.62:0/1064604935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80b39c0 con 0x7f82d815dd50 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.662+0000 7f82dce00640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d80c9f20 0x7f82d80ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.663+0000 7f82d7400640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d815dd50 0x7f82d814fd30 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f82c400d800 tx=0x7f82c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.663+0000 7f82cfe00640 1 -- 172.21.15.62:0/1064604935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400b6c0 con 0x7f82d815dd50 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.663+0000 7f82cfe00640 1 -- 172.21.15.62:0/1064604935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4019070 con 0x7f82d815dd50 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.663+0000 7f82cfe00640 1 -- 172.21.15.62:0/1064604935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400ba80 con 0x7f82d815dd50 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.663+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d80b3ca0 con 0x7f82d815dd50 2024-04-25T09:18:10.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.663+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d80b4260 con 0x7f82d815dd50 2024-04-25T09:18:10.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.664+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d804ef20 con 0x7f82d815dd50 2024-04-25T09:18:10.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.665+0000 7f82cfe00640 1 -- 172.21.15.62:0/1064604935 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82c400bc20 con 0x7f82d815dd50 2024-04-25T09:18:10.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.667+0000 7f82cfe00640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82b806b4d0 0x7f82b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:10.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.667+0000 7f82cfe00640 1 -- 172.21.15.62:0/1064604935 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f82c408a7c0 con 0x7f82d815dd50 2024-04-25T09:18:10.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.668+0000 7f82d7e00640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82b806b4d0 0x7f82b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:10.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.669+0000 7f82d7e00640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82b806b4d0 0x7f82b806d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f82c8006fd0 tx=0x7f82c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:10.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.672+0000 7f82cfe00640 1 -- 172.21.15.62:0/1064604935 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82c4059560 con 0x7f82d815dd50 2024-04-25T09:18:10.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:10.913+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f82d80b9950 con 0x7f82d815dd50 2024-04-25T09:18:11.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.399+0000 7f82cfe00640 1 -- 172.21.15.62:0/1064604935 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v101) v1 ==== 121+0+0 (secure 0 0 0) 0x7f82c4058e60 con 0x7f82d815dd50 2024-04-25T09:18:11.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82b806b4d0 msgr2=0x7f82b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82b806b4d0 0x7f82b806d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f82c8006fd0 tx=0x7f82c8008040 comp rx=0 tx=0).stop 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d815dd50 msgr2=0x7f82d814fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d815dd50 0x7f82d814fd30 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f82c400d800 tx=0x7f82c400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 shutdown_connections 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f82b806b4d0 0x7f82b806d990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f82d815dd50 0x7f82d814fd30 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f82d80c9f20 0x7f82d80ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 --2- 172.21.15.62:0/1064604935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d80d3960 0x7f82d80b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.402+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 >> 172.21.15.62:0/1064604935 conn(0x7f82d806b000 msgr2=0x7f82d81601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.403+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 shutdown_connections 2024-04-25T09:18:11.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.403+0000 7f82dd800640 1 -- 172.21.15.62:0/1064604935 wait complete. 2024-04-25T09:18:11.412 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:18:11.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.621+0000 7fc6c6600640 1 Processor -- start 2024-04-25T09:18:11.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.621+0000 7fc6c6600640 1 -- start start 2024-04-25T09:18:11.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.623+0000 7fc6c6600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c015dd50 0x7fc6c006d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:11.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.623+0000 7fc6c6600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00aec20 con 0x7fc6c00ceac0 2024-04-25T09:18:11.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.623+0000 7fc6c6600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c006e120 con 0x7fc6c00ae7e0 2024-04-25T09:18:11.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.623+0000 7fc6c6600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0162980 con 0x7fc6c015dd50 2024-04-25T09:18:11.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.623+0000 7fc6bfe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc6c00ceac0 0x7fc6c00cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36600/0 (socket says 172.21.15.62:36600) 2024-04-25T09:18:11.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.623+0000 7fc6bfe00640 1 -- 172.21.15.62:0/3683137571 learned_addr learned my addr 172.21.15.62:0/3683137571 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:11.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6bf400640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c015dd50 0x7fc6c006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:11.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67917507 0 0) 0x7fc6c00aec20 con 0x7fc6c00ceac0 2024-04-25T09:18:11.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a4003520 con 0x7fc6c00ceac0 2024-04-25T09:18:11.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 275576713 0 0) 0x7fc6a4003520 con 0x7fc6c00ceac0 2024-04-25T09:18:11.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c015dd50 msgr2=0x7fc6c006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:11.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c015dd50 0x7fc6c006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 >> v1:172.21.15.121:6789/0 conn(0x7fc6c00ae7e0 legacy=0x7fc6c0162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c006db60 con 0x7fc6c00ceac0 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc6b80019f0 con 0x7fc6c00ceac0 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 >> v1:172.21.15.62:6789/0 conn(0x7fc6c00ceac0 legacy=0x7fc6c00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6a40039e0 0x7fc6a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a4004800 0x7fc6a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.624+0000 7fc6be000640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c015dd50 0x7fc6a4005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0162980 con 0x7fc6a40039e0 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a400bb50 con 0x7fc6a4004800 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a400bd50 con 0x7fc6c015dd50 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bf400640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6a40039e0 0x7fc6a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bea00640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a4004800 0x7fc6a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:11.613 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bfe00640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c015dd50 0x7fc6a4005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bf400640 1 -- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c015dd50 msgr2=0x7fc6a4005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bf400640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c015dd50 0x7fc6a4005270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bf400640 1 -- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a4004800 msgr2=0x7fc6a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bf400640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a4004800 0x7fc6a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bf400640 1 -- 172.21.15.62:0/3683137571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4003520 con 0x7fc6a40039e0 2024-04-25T09:18:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.625+0000 7fc6bea00640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a4004800 0x7fc6a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:11.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.626+0000 7fc6bf400640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6a40039e0 0x7fc6a4003e00 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009ac0 tx=0x7fc6a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d26720a3979cd18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0036070 con 0x7fc6a40039e0 2024-04-25T09:18:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b0031400 con 0x7fc6a40039e0 2024-04-25T09:18:11.616 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6be000640 1 -- 172.21.15.62:0/3683137571 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0009690 con 0x7fc6a40039e0 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6c6600640 1 -- 172.21.15.62:0/3683137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6a40039e0 msgr2=0x7fc6a4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6c6600640 1 --2- 172.21.15.62:0/3683137571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6a40039e0 0x7fc6a4003e00 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009ac0 tx=0x7fc6a4005830 comp rx=0 tx=0).stop 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6bfe00640 1 -- 172.21.15.62:0/3683137571 reap_dead start 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6c6600640 1 -- 172.21.15.62:0/3683137571 shutdown_connections 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6c6600640 1 -- 172.21.15.62:0/3683137571 >> 172.21.15.62:0/3683137571 conn(0x7fc6c00668f0 msgr2=0x7fc6c0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6c6600640 1 -- 172.21.15.62:0/3683137571 shutdown_connections 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.627+0000 7fc6c6600640 1 -- 172.21.15.62:0/3683137571 wait complete. 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.629+0000 7fc6c6600640 1 Processor -- start 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.629+0000 7fc6c6600640 1 -- start start 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.630+0000 7fc6c6600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a40039e0 0x7fc6c01e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.630+0000 7fc6c6600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c0147110 0x7fc6c0158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.630+0000 7fc6c6600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c0148720 0x7fc6c0148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.630+0000 7fc6c6600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0161d50 con 0x7fc6c0147110 2024-04-25T09:18:11.617 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.630+0000 7fc6c6600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0161a70 con 0x7fc6a40039e0 2024-04-25T09:18:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.630+0000 7fc6c6600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c015e130 con 0x7fc6c0148720 2024-04-25T09:18:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c0147110 0x7fc6c0158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c0147110 0x7fc6c0158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45180/0 (socket says 172.21.15.62:45180) 2024-04-25T09:18:11.618 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 -- 172.21.15.62:0/956945692 learned_addr learned my addr 172.21.15.62:0/956945692 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bfe00640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c0148720 0x7fc6c0148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 -- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c0148720 msgr2=0x7fc6c0148ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bf400640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a40039e0 0x7fc6c01e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c0148720 0x7fc6c0148ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 -- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a40039e0 msgr2=0x7fc6c01e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a40039e0 0x7fc6c01e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 -- 172.21.15.62:0/956945692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c01e8760 con 0x7fc6c0147110 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bf400640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a40039e0 0x7fc6c01e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.631+0000 7fc6bea00640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c0147110 0x7fc6c0158fd0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00d7a0 tx=0x7fc6ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.632+0000 7fc6b7400640 1 -- 172.21.15.62:0/956945692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac005070 con 0x7fc6c0147110 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.632+0000 7fc6b7400640 1 -- 172.21.15.62:0/956945692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac005210 con 0x7fc6c0147110 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.632+0000 7fc6b7400640 1 -- 172.21.15.62:0/956945692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac007030 con 0x7fc6c0147110 2024-04-25T09:18:11.620 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.632+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c0147680 con 0x7fc6c0147110 2024-04-25T09:18:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.632+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c0147c40 con 0x7fc6c0147110 2024-04-25T09:18:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.633+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c0072c70 con 0x7fc6c0147110 2024-04-25T09:18:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.634+0000 7fc6b7400640 1 -- 172.21.15.62:0/956945692 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6ac01a020 con 0x7fc6c0147110 2024-04-25T09:18:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.635+0000 7fc6b7400640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a006b500 0x7fc6a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.635+0000 7fc6b7400640 1 -- 172.21.15.62:0/956945692 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc6ac089a10 con 0x7fc6c0147110 2024-04-25T09:18:11.621 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.636+0000 7fc6bf400640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a006b500 0x7fc6a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:11.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.637+0000 7fc6bf400640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a006b500 0x7fc6a006d9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009ac0 tx=0x7fc6b0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:11.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.639+0000 7fc6b7400640 1 -- 172.21.15.62:0/956945692 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6ac058840 con 0x7fc6c0147110 2024-04-25T09:18:11.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.877+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6c004f000 con 0x7fc6c0147110 2024-04-25T09:18:11.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.878+0000 7fc6b7400640 1 -- 172.21.15.62:0/956945692 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4948 (secure 0 0 0) 0x7fc6ac05e020 con 0x7fc6c0147110 2024-04-25T09:18:11.864 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:11.864 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":102,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3369583759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3369583759},{"type":"v1","addr":"172.21.15.121:6837","nonce":3369583759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1888749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1888749},{"type":"v1","addr":"172.21.15.121:6836","nonce":1888749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:17:30.382991+0000","modified":"2024-04-25T09:18:11.404400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9390":{"gid":9390,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6833/1673530113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1673530113},{"type":"v1","addr":"172.21.15.62:6833","nonce":1673530113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9396":{"gid":9396,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2793862963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2793862963},{"type":"v1","addr":"172.21.15.62:6835","nonce":2793862963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:18:11.864 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 102 2024-04-25T09:18:11.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a006b500 msgr2=0x7fc6a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a006b500 0x7fc6a006d9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009ac0 tx=0x7fc6b0008030 comp rx=0 tx=0).stop 2024-04-25T09:18:11.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c0147110 msgr2=0x7fc6c0158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:11.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c0147110 0x7fc6c0158fd0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00d7a0 tx=0x7fc6ac00dc60 comp rx=0 tx=0).stop 2024-04-25T09:18:11.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 shutdown_connections 2024-04-25T09:18:11.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc6a006b500 0x7fc6a006d9c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc6c0147110 0x7fc6c0158fd0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc6c0148720 0x7fc6c0148ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 --2- 172.21.15.62:0/956945692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6a40039e0 0x7fc6c01e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:11.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 >> 172.21.15.62:0/956945692 conn(0x7fc6c00668f0 msgr2=0x7fc6c00b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:11.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 shutdown_connections 2024-04-25T09:18:11.868 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:11.881+0000 7fc6c6600640 1 -- 172.21.15.62:0/956945692 wait complete. 2024-04-25T09:18:12.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T09:17:30.382991+0000', 'modified': '2024-04-25T09:18:11.404400+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9396}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9390': {'gid': 9390, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6833/1673530113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 1673530113}, {'type': 'v1', 'addr': '172.21.15.62:6833', 'nonce': 1673530113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9396': {'gid': 9396, 'name': 'a', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/2793862963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6834', 'nonce': 2793862963}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 2793862963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:18:12.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:18:12.291 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:18:12.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.531+0000 7efcf3e00640 1 Processor -- start 2024-04-25T09:18:12.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.531+0000 7efcf3e00640 1 -- start start 2024-04-25T09:18:12.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.532+0000 7efcf3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcf415b920 0x7efcf40cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.533+0000 7efcf3e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf406e070 con 0x7efcf4071ab0 2024-04-25T09:18:12.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.533+0000 7efcf3e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf406e350 con 0x7efcf415f560 2024-04-25T09:18:12.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.533+0000 7efcf3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf406e630 con 0x7efcf415b920 2024-04-25T09:18:12.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.533+0000 7efcf3400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7efcf4071ab0 0x7efcf415c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36610/0 (socket says 172.21.15.62:36610) 2024-04-25T09:18:12.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.533+0000 7efcf3400640 1 -- 172.21.15.62:0/3638588665 learned_addr learned my addr 172.21.15.62:0/3638588665 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:12.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.533+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcf415b920 0x7efcf40cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269555802 0 0) 0x7efcf406e070 con 0x7efcf4071ab0 2024-04-25T09:18:12.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd8003520 con 0x7efcf4071ab0 2024-04-25T09:18:12.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822732693 0 0) 0x7efcd8003520 con 0x7efcf4071ab0 2024-04-25T09:18:12.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcf415b920 msgr2=0x7efcf40cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcf415b920 0x7efcf40cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 >> v1:172.21.15.121:6789/0 conn(0x7efcf415f560 legacy=0x7efcf415bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf406e910 con 0x7efcf4071ab0 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7efcec0019f0 con 0x7efcf4071ab0 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 >> v1:172.21.15.62:6789/0 conn(0x7efcf4071ab0 legacy=0x7efcf415c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcf415b920 0x7efcd80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd8004060 0x7efcd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd8004d40 0x7efcd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf406e630 con 0x7efcd8004d40 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd800bc00 con 0x7efcd8004060 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd800be00 con 0x7efcf415b920 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.534+0000 7efcf3400640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd8004d40 0x7efcd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.535+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcf415b920 0x7efcd80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.535+0000 7efcf2000640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd8004060 0x7efcd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.535+0000 7efcf3400640 1 -- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcf415b920 msgr2=0x7efcd80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.535+0000 7efcf3400640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcf415b920 0x7efcd80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.535+0000 7efcf3400640 1 -- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd8004060 msgr2=0x7efcd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.535+0000 7efcf3400640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd8004060 0x7efcd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.535+0000 7efcf3400640 1 -- 172.21.15.62:0/3638588665 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8003520 con 0x7efcd8004d40 2024-04-25T09:18:12.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.536+0000 7efcf3400640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd8004d40 0x7efcd80051c0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7efcec00a740 tx=0x7efcd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d71352570ebed31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:12.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.536+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00a390 con 0x7efcd8004d40 2024-04-25T09:18:12.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.536+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec02f070 con 0x7efcd8004d40 2024-04-25T09:18:12.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.536+0000 7efcf1600640 1 -- 172.21.15.62:0/3638588665 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec02bc00 con 0x7efcd8004d40 2024-04-25T09:18:12.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.537+0000 7efcf3e00640 1 -- 172.21.15.62:0/3638588665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd8004d40 msgr2=0x7efcd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.537+0000 7efcf3e00640 1 --2- 172.21.15.62:0/3638588665 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcd8004d40 0x7efcd80051c0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7efcec00a740 tx=0x7efcd8005830 comp rx=0 tx=0).stop 2024-04-25T09:18:12.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.537+0000 7efcf3400640 1 -- 172.21.15.62:0/3638588665 reap_dead start 2024-04-25T09:18:12.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.537+0000 7efcf3e00640 1 -- 172.21.15.62:0/3638588665 shutdown_connections 2024-04-25T09:18:12.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.537+0000 7efcf3e00640 1 -- 172.21.15.62:0/3638588665 >> 172.21.15.62:0/3638588665 conn(0x7efcf40b1cd0 msgr2=0x7efcf40b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:12.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.537+0000 7efcf3e00640 1 -- 172.21.15.62:0/3638588665 shutdown_connections 2024-04-25T09:18:12.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.537+0000 7efcf3e00640 1 -- 172.21.15.62:0/3638588665 wait complete. 2024-04-25T09:18:12.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.539+0000 7efcf3e00640 1 Processor -- start 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.539+0000 7efcf3e00640 1 -- start start 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd8004d40 0x7efcf4141410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcf4070a10 0x7efcf4070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcf4138ff0 0x7efcf4139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4160360 con 0x7efcf4138ff0 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41601e0 con 0x7efcf4070a10 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf415cb20 con 0x7efcd8004d40 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcf4138ff0 0x7efcf4139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcf4138ff0 0x7efcf4139470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45196/0 (socket says 172.21.15.62:45196) 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.540+0000 7efcf3400640 1 -- 172.21.15.62:0/3189565030 learned_addr learned my addr 172.21.15.62:0/3189565030 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.541+0000 7efcf2000640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcf4070a10 0x7efcf4070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.541+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd8004d40 0x7efcf4141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.541+0000 7efcf3400640 1 -- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd8004d40 msgr2=0x7efcf4141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.541+0000 7efcf3400640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd8004d40 0x7efcf4141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.541+0000 7efcf3400640 1 -- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcf4070a10 msgr2=0x7efcf4070e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.541+0000 7efcf3400640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcf4070a10 0x7efcf4070e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.541+0000 7efcf3400640 1 -- 172.21.15.62:0/3189565030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf40b23f0 con 0x7efcf4138ff0 2024-04-25T09:18:12.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.541+0000 7efcf3400640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcf4138ff0 0x7efcf4139470 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7efcec02b410 tx=0x7efcf406e240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:12.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.542+0000 7efceaa00640 1 -- 172.21.15.62:0/3189565030 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec001ee0 con 0x7efcf4138ff0 2024-04-25T09:18:12.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.542+0000 7efceaa00640 1 -- 172.21.15.62:0/3189565030 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec02bb70 con 0x7efcf4138ff0 2024-04-25T09:18:12.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.542+0000 7efceaa00640 1 -- 172.21.15.62:0/3189565030 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec034470 con 0x7efcf4138ff0 2024-04-25T09:18:12.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.542+0000 7efcf3e00640 1 -- 172.21.15.62:0/3189565030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf40b26d0 con 0x7efcf4138ff0 2024-04-25T09:18:12.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.542+0000 7efcf3e00640 1 -- 172.21.15.62:0/3189565030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf40b2c90 con 0x7efcf4138ff0 2024-04-25T09:18:12.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.544+0000 7efceaa00640 1 -- 172.21.15.62:0/3189565030 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcec034610 con 0x7efcf4138ff0 2024-04-25T09:18:12.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.545+0000 7efceaa00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcd406b500 0x7efcd406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.545+0000 7efceaa00640 1 -- 172.21.15.62:0/3189565030 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efcec0aa5b0 con 0x7efcf4138ff0 2024-04-25T09:18:12.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.545+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcd406b500 0x7efcd406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.546+0000 7efcfc5c9740 1 Processor -- start 2024-04-25T09:18:12.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.546+0000 7efcfc5c9740 1 -- start start 2024-04-25T09:18:12.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcd406b500 0x7efcd406d9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efce4006fd0 tx=0x7efce4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:12.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bc3b30 0x557d55beb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bc3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 0x557d55bc22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55b27620 con 0x557d55bebeb0 2024-04-25T09:18:12.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55ad0ca0 con 0x557d55c0c7a0 2024-04-25T09:18:12.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d55b236c0 con 0x557d55bc3b30 2024-04-25T09:18:12.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcf2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bc3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcf2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bc3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45198/0 (socket says 172.21.15.62:45198) 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.547+0000 7efcf2000640 1 -- 172.21.15.62:0/977297158 learned_addr learned my addr 172.21.15.62:0/977297158 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcf2a00640 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bc3b30 0x557d55beb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcf3400640 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 0x557d55bc22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcf2000640 1 -- 172.21.15.62:0/977297158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bc3b30 msgr2=0x557d55beb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcf2000640 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bc3b30 0x557d55beb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcf2000640 1 -- 172.21.15.62:0/977297158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 msgr2=0x557d55bc22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcf2000640 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 0x557d55bc22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcf2000640 1 -- 172.21.15.62:0/977297158 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d55ad7f40 con 0x557d55bebeb0 2024-04-25T09:18:12.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcf2000640 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bc3f10 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7efce0005440 tx=0x7efce00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4707289134870f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.548+0000 7efcdf000640 1 -- 172.21.15.62:0/977297158 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0007030 con 0x557d55bebeb0 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcdf000640 1 -- 172.21.15.62:0/977297158 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce00071d0 con 0x557d55bebeb0 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcdf000640 1 -- 172.21.15.62:0/977297158 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce00073a0 con 0x557d55bebeb0 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 -- 172.21.15.62:0/977297158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 msgr2=0x557d55bc3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bc3f10 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7efce0005440 tx=0x7efce00039c0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 -- 172.21.15.62:0/977297158 shutdown_connections 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bc3f10 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 0x557d55bc22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/977297158 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bc3b30 0x557d55beb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 -- 172.21.15.62:0/977297158 >> 172.21.15.62:0/977297158 conn(0x557d55bc0d50 msgr2=0x557d55c09730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 -- 172.21.15.62:0/977297158 shutdown_connections 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.549+0000 7efcfc5c9740 1 -- 172.21.15.62:0/977297158 wait complete. 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.550+0000 7efcfc5c9740 1 -- start start 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.550+0000 7efcfc5c9740 1 Processor -- start 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.551+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.551+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55c0c7a0 0x557d55be8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.551+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55be8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.551+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55beb4e0 con 0x557d55bebeb0 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.551+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55ac61f0 con 0x557d55be1580 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.551+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d55beaed0 con 0x557d55c0c7a0 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.551+0000 7efcfc5c9740 10 client.0 _ll_register_callbacks cb 0x557d55ae3ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bea2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45202/0 (socket says 172.21.15.62:45202) 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 -- 172.21.15.62:0/3043517686 learned_addr learned my addr 172.21.15.62:0/3043517686 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2000640 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55c0c7a0 0x557d55be8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf3400640 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55be8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 -- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55c0c7a0 msgr2=0x557d55be8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55c0c7a0 0x557d55be8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 -- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 msgr2=0x557d55be8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55be8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 -- 172.21.15.62:0/3043517686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d55ae5c50 con 0x557d55bebeb0 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf3400640 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55be8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bea2e0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7efce4011bc0 tx=0x7efce4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5fa73ab23c025c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.552+0000 7efcd3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:18:12.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.553+0000 7efcd3400640 1 -- 172.21.15.62:0/3043517686 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4015480 con 0x557d55bebeb0 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.553+0000 7efcd3400640 1 -- 172.21.15.62:0/3043517686 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efce4015620 con 0x557d55bebeb0 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.553+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3043517686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efce00071d0 con 0x557d55bebeb0 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.553+0000 7efcd3400640 1 -- 172.21.15.62:0/3043517686 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7efce4030050 con 0x557d55bebeb0 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.553+0000 7efcfc5c9740 10 client.9567 fetch_fsmap learned FSMap version 102 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.553+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3043517686 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x557d5583c2d0 con 0x557d55bebeb0 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.553+0000 7efcd3400640 1 -- 172.21.15.62:0/3043517686 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce00071d0 con 0x557d55bebeb0 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.553+0000 7efcd3400640 1 -- 172.21.15.62:0/3043517686 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 102) v1 ==== 3282+0+0 (secure 0 0 0) 0x7efce4033020 con 0x557d55bebeb0 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcfc5c9740 10 client.9567 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcfc5c9740 10 client.9567 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcfc5c9740 20 client.9567 populate_metadata read hostname 'smithi062' 2024-04-25T09:18:12.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x557d55c1cf50 0x557d55c1d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcfc5c9740 4 client.9567 mds_command: new command op to 9396 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:18:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3043517686 --> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d55b34b60 con 0x557d55c1cf50 2024-04-25T09:18:12.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcf2000640 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x557d55c1cf50 0x557d55c1d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcf2000640 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x557d55c1cf50 0x557d55c1d370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20801796711937fe server_cookie=ccf418a4272e6e20 in_seq=0 out_seq=0 2024-04-25T09:18:12.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.554+0000 7efcd3400640 10 client.9567 ms_handle_connect on v2:172.21.15.62:6834/2793862963 2024-04-25T09:18:12.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.555+0000 7efcd3400640 1 -- 172.21.15.62:0/3043517686 <== mds.0 v2:172.21.15.62:6834/2793862963 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d55b34b60 con 0x557d55c1cf50 2024-04-25T09:18:12.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.555+0000 7efcd3400640 10 client.9567 handle_command_reply: tid=0 2024-04-25T09:18:12.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.555+0000 7efcfc5c9740 1 client.9567 shutdown 2024-04-25T09:18:12.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.555+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3043517686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 msgr2=0x557d55bea2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.555+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bea2e0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7efce4011bc0 tx=0x7efce4004730 comp rx=0 tx=0).stop 2024-04-25T09:18:12.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3043517686 shutdown_connections 2024-04-25T09:18:12.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x557d55c1cf50 0x557d55c1d370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55be8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55c0c7a0 0x557d55be8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3043517686 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55bea2e0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcd3400640 0 client.9567 ms_handle_reset on v2:172.21.15.62:6834/2793862963 2024-04-25T09:18:12.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3043517686 >> 172.21.15.62:0/3043517686 conn(0x557d55bc3b30 msgr2=0x557d55c0cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:12.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3043517686 shutdown_connections 2024-04-25T09:18:12.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3043517686 wait complete. 2024-04-25T09:18:12.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.556+0000 7efcfc5c9740 20 client.9567 trim_cache size 0 max 16384 2024-04-25T09:18:12.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.573+0000 7efcfc5c9740 1 Processor -- start 2024-04-25T09:18:12.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.574+0000 7efcfc5c9740 1 -- start start 2024-04-25T09:18:12.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.574+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55be1580 0x557d55c23360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.574+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55c238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.574+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 0x557d55be8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.574+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf42006e0 con 0x557d55bebeb0 2024-04-25T09:18:12.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.574+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41492f0 con 0x557d55c0c7a0 2024-04-25T09:18:12.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.574+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40ceee0 con 0x557d55be1580 2024-04-25T09:18:12.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55c238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55c238a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45210/0 (socket says 172.21.15.62:45210) 2024-04-25T09:18:12.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2000640 1 -- 172.21.15.62:0/435252071 learned_addr learned my addr 172.21.15.62:0/435252071 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2a00640 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55be1580 0x557d55c23360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf3400640 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 0x557d55be8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2000640 1 -- 172.21.15.62:0/435252071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55be1580 msgr2=0x557d55c23360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2000640 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55be1580 0x557d55c23360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2000640 1 -- 172.21.15.62:0/435252071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 msgr2=0x557d55be8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2000640 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 0x557d55be8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.575+0000 7efcf2000640 1 -- 172.21.15.62:0/435252071 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d559066b0 con 0x557d55bebeb0 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.576+0000 7efcf2000640 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55c238a0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x557d55bc00c0 tx=0x7efce000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d0c6aeb2050cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.576+0000 7efcde600640 1 -- 172.21.15.62:0/435252071 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0007710 con 0x557d55bebeb0 2024-04-25T09:18:12.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.576+0000 7efcde600640 1 -- 172.21.15.62:0/435252071 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce0007030 con 0x557d55bebeb0 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.576+0000 7efcde600640 1 -- 172.21.15.62:0/435252071 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce00071e0 con 0x557d55bebeb0 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.576+0000 7efcfc5c9740 1 -- 172.21.15.62:0/435252071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 msgr2=0x557d55c238a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.576+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55c238a0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x557d55bc00c0 tx=0x7efce000a000 comp rx=0 tx=0).stop 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.577+0000 7efcfc5c9740 1 -- 172.21.15.62:0/435252071 shutdown_connections 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.577+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55bebeb0 0x557d55c238a0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.577+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55c0c7a0 0x557d55be8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.577+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/435252071 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55be1580 0x557d55c23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.577+0000 7efcfc5c9740 1 -- 172.21.15.62:0/435252071 >> 172.21.15.62:0/435252071 conn(0x557d55bc3b30 msgr2=0x557d55beb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.577+0000 7efcfc5c9740 1 -- 172.21.15.62:0/435252071 shutdown_connections 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.577+0000 7efcfc5c9740 1 -- 172.21.15.62:0/435252071 wait complete. 2024-04-25T09:18:12.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.577+0000 7efcfc5c9740 1 -- start start 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.578+0000 7efcfc5c9740 1 Processor -- start 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.578+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55c0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.578+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bebeb0 0x557d55c0d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcfc5c9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55c0c7a0 0x557d55c1bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55be92c0 con 0x557d55c0c7a0 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d55be8160 con 0x557d55be1580 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcfc5c9740 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d55be1ad0 con 0x557d55bebeb0 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcfc5c9740 10 client.0 _ll_register_callbacks cb 0x7efce4019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55c0c7a0 0x557d55c1bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55c0c7a0 0x557d55c1bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45214/0 (socket says 172.21.15.62:45214) 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf3400640 1 -- 172.21.15.62:0/3839608976 learned_addr learned my addr 172.21.15.62:0/3839608976 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55c0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf3400640 1 -- 172.21.15.62:0/3839608976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bebeb0 msgr2=0x557d55c0d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf3400640 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bebeb0 0x557d55c0d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf3400640 1 -- 172.21.15.62:0/3839608976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 msgr2=0x557d55c0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf3400640 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55c0cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.579+0000 7efcf3400640 1 -- 172.21.15.62:0/3839608976 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d55c23a70 con 0x557d55c0c7a0 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcf3400640 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55c0c7a0 0x557d55c1bcf0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7efcec005c20 tx=0x7efcec078640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d554a7707c2a6355 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcd3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcd3400640 1 -- 172.21.15.62:0/3839608976 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0a8e60 con 0x557d55c0c7a0 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcd3400640 1 -- 172.21.15.62:0/3839608976 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efcec080f60 con 0x557d55c0c7a0 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3839608976 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efce0007030 con 0x557d55c0c7a0 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcd3400640 1 -- 172.21.15.62:0/3839608976 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcec07ce30 con 0x557d55c0c7a0 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcfc5c9740 10 client.9573 fetch_fsmap learned FSMap version 102 2024-04-25T09:18:12.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3839608976 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x557d55bc6220 con 0x557d55c0c7a0 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcd3400640 1 -- 172.21.15.62:0/3839608976 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce0007030 con 0x557d55c0c7a0 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcd3400640 1 -- 172.21.15.62:0/3839608976 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 102) v1 ==== 3282+0+0 (secure 0 0 0) 0x7efcec0aa0b0 con 0x557d55c0c7a0 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.580+0000 7efcfc5c9740 10 client.9573 fetch_fsmap finished waiting for FSMap version 102 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.581+0000 7efcfc5c9740 10 client.9573 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.581+0000 7efcfc5c9740 20 client.9573 populate_metadata read hostname 'smithi062' 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.581+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x557d55c1c230 0x557d55bc22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.581+0000 7efcfc5c9740 4 client.9573 mds_command: new command op to 9396 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.581+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3839608976 --> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557d55b2eeb0 con 0x557d55c1c230 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.581+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x557d55c1c230 0x557d55bc22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.581+0000 7efcf2a00640 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x557d55c1c230 0x557d55bc22f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=addc7a3794487a12 server_cookie=17269f7ef4c8711c in_seq=0 out_seq=0 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.581+0000 7efcd3400640 10 client.9573 ms_handle_connect on v2:172.21.15.62:6834/2793862963 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcd3400640 1 -- 172.21.15.62:0/3839608976 <== mds.0 v2:172.21.15.62:6834/2793862963 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x557d55b2eeb0 con 0x557d55c1c230 2024-04-25T09:18:12.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcd3400640 10 client.9573 handle_command_reply: tid=0 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 client.9573 shutdown 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3839608976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55c0c7a0 msgr2=0x557d55c1bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55c0c7a0 0x557d55c1bcf0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7efcec005c20 tx=0x7efcec078640 comp rx=0 tx=0).stop 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3839608976 shutdown_connections 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.62:6834/2793862963,v1:172.21.15.62:6835/2793862963] conn(0x557d55c1c230 0x557d55bc22f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557d55c0c7a0 0x557d55c1bcf0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x557d55bebeb0 0x557d55c0d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 --2- 172.21.15.62:0/3839608976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d55be1580 0x557d55c0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3839608976 >> 172.21.15.62:0/3839608976 conn(0x557d55bc3b30 msgr2=0x557d55c23360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcd3400640 0 client.9573 ms_handle_reset on v2:172.21.15.62:6834/2793862963 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3839608976 shutdown_connections 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 1 -- 172.21.15.62:0/3839608976 wait complete. 2024-04-25T09:18:12.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.582+0000 7efcfc5c9740 20 client.9573 trim_cache size 0 max 16384 2024-04-25T09:18:12.572 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:12.572 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "9e112875-3fc2-4963-90e0-2b56d5516d5c", 2024-04-25T09:18:12.572 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-25T09:18:12.572 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 9396, 2024-04-25T09:18:12.572 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-25T09:18:12.572 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-25T09:18:12.573 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-25T09:18:12.573 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 41.631863084000003, 2024-04-25T09:18:12.573 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 102, 2024-04-25T09:18:12.573 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 150, 2024-04-25T09:18:12.573 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 150, 2024-04-25T09:18:12.573 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 47.069878009 2024-04-25T09:18:12.573 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 -- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcd406b500 msgr2=0x7efcd406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcd406b500 0x7efcd406d9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efce4006fd0 tx=0x7efce4008040 comp rx=0 tx=0).stop 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 -- 172.21.15.62:0/3189565030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcf4138ff0 msgr2=0x7efcf4139470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcf4138ff0 0x7efcf4139470 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7efcec02b410 tx=0x7efcf406e240 comp rx=0 tx=0).stop 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 -- 172.21.15.62:0/3189565030 shutdown_connections 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efcd406b500 0x7efcd406d9c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efcf4138ff0 0x7efcf4139470 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcf4070a10 0x7efcf4070e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 --2- 172.21.15.62:0/3189565030 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efcd8004d40 0x7efcf4141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 -- 172.21.15.62:0/3189565030 >> 172.21.15.62:0/3189565030 conn(0x7efcf40b1cd0 msgr2=0x7efcf415d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:12.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 -- 172.21.15.62:0/3189565030 shutdown_connections 2024-04-25T09:18:12.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:12.583+0000 7efcf3e00640 1 -- 172.21.15.62:0/3189565030 wait complete. 2024-04-25T09:18:12.587 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:18:12.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:18:12.695 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:18:12.821 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:18:12.821 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:18:12.821 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:12.884 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:18:12.884 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:12.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:18:13.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.172+0000 7f51db200640 1 Processor -- start 2024-04-25T09:18:13.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.172+0000 7f51db200640 1 -- start start 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.173+0000 7f51db200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f51dc0b56f0 0x7f51dc160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.173+0000 7f51db200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51dc0b6a90 0x7f51dc161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.173+0000 7f51db200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc0d3110 con 0x7f51dc0b27a0 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.173+0000 7f51db200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc161780 con 0x7f51dc0b6a90 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.174+0000 7f51db200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc0cedc0 con 0x7f51dc0b56f0 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.174+0000 7f51d9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51dc0b6a90 0x7f51dc161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.174+0000 7f51da800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f51dc0b27a0 0x7f51dc161980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36612/0 (socket says 172.21.15.62:36612) 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.174+0000 7f51d9400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f51dc0b6a90 0x7f51dc161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45226/0 (socket says 172.21.15.62:45226) 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.174+0000 7f51da800640 1 -- 172.21.15.62:0/493353832 learned_addr learned my addr 172.21.15.62:0/493353832 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:13.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.174+0000 7f51d9e00640 1 --2- 172.21.15.62:0/493353832 >> v2:172.21.15.121:3300/0 conn(0x7f51dc0b56f0 0x7f51dc160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.175+0000 7f51d9400640 1 -- 172.21.15.62:0/493353832 >> v2:172.21.15.121:3300/0 conn(0x7f51dc0b56f0 msgr2=0x7f51dc160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.175+0000 7f51d9400640 1 --2- 172.21.15.62:0/493353832 >> v2:172.21.15.121:3300/0 conn(0x7f51dc0b56f0 0x7f51dc160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.175+0000 7f51d9400640 1 -- 172.21.15.62:0/493353832 >> v1:172.21.15.62:6789/0 conn(0x7f51dc0b27a0 legacy=0x7f51dc161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.175+0000 7f51d9400640 1 -- 172.21.15.62:0/493353832 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0d33f0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.175+0000 7f51d8a00640 1 -- 172.21.15.62:0/493353832 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603183188 0 0) 0x7f51dc0d3110 con 0x7f51dc0b27a0 2024-04-25T09:18:13.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.176+0000 7f51d9400640 1 --2- 172.21.15.62:0/493353832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc161240 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f51dc08c310 tx=0x7f51c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81be54a3e6567263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:13.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.176+0000 7f51d8a00640 1 -- 172.21.15.62:0/493353832 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4007740 con 0x7f51dc0b6a90 2024-04-25T09:18:13.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.176+0000 7f51d8a00640 1 -- 172.21.15.62:0/493353832 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c40078e0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.162 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.176+0000 7f51d8a00640 1 -- 172.21.15.62:0/493353832 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4007cb0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.177+0000 7f51db200640 1 -- 172.21.15.62:0/493353832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 msgr2=0x7f51dc161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.177+0000 7f51db200640 1 --2- 172.21.15.62:0/493353832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc161240 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f51dc08c310 tx=0x7f51c402a680 comp rx=0 tx=0).stop 2024-04-25T09:18:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.177+0000 7f51db200640 1 -- 172.21.15.62:0/493353832 shutdown_connections 2024-04-25T09:18:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.177+0000 7f51db200640 1 --2- 172.21.15.62:0/493353832 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc161240 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.177+0000 7f51db200640 1 --2- 172.21.15.62:0/493353832 >> v2:172.21.15.121:3300/0 conn(0x7f51dc0b56f0 0x7f51dc160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.177+0000 7f51db200640 1 -- 172.21.15.62:0/493353832 >> 172.21.15.62:0/493353832 conn(0x7f51dc066d10 msgr2=0x7f51dc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.177+0000 7f51db200640 1 -- 172.21.15.62:0/493353832 shutdown_connections 2024-04-25T09:18:13.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.177+0000 7f51db200640 1 -- 172.21.15.62:0/493353832 wait complete. 2024-04-25T09:18:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.179+0000 7f51db200640 1 Processor -- start 2024-04-25T09:18:13.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.179+0000 7f51db200640 1 -- start start 2024-04-25T09:18:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51db200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51dc0b27a0 0x7f51dc14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51db200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51dc0b56f0 0x7f51dc158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51db200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51db200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0b2d60 con 0x7f51dc0b6a90 2024-04-25T09:18:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51db200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0cea50 con 0x7f51dc0b56f0 2024-04-25T09:18:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51db200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0b2ee0 con 0x7f51dc0b27a0 2024-04-25T09:18:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51da800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51da800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45228/0 (socket says 172.21.15.62:45228) 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51da800640 1 -- 172.21.15.62:0/3894837569 learned_addr learned my addr 172.21.15.62:0/3894837569 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51d9400640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51dc0b56f0 0x7f51dc158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51d9e00640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51dc0b27a0 0x7f51dc14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51da800640 1 -- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51dc0b27a0 msgr2=0x7f51dc14cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.180+0000 7f51da800640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51dc0b27a0 0x7f51dc14cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51da800640 1 -- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51dc0b56f0 msgr2=0x7f51dc158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51da800640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51dc0b56f0 0x7f51dc158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51da800640 1 -- 172.21.15.62:0/3894837569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc14d270 con 0x7f51dc0b6a90 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51da800640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc158590 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f51d400c4b0 tx=0x7f51d400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51d2000640 1 -- 172.21.15.62:0/3894837569 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4018070 con 0x7f51dc0b6a90 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51d2000640 1 -- 172.21.15.62:0/3894837569 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d40074e0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc157ba0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51d2000640 1 -- 172.21.15.62:0/3894837569 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4004a20 con 0x7f51dc0b6a90 2024-04-25T09:18:13.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.181+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc15a9c0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.182+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc04ef20 con 0x7f51dc0b6a90 2024-04-25T09:18:13.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.184+0000 7f51d2000640 1 -- 172.21.15.62:0/3894837569 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51d401b070 con 0x7f51dc0b6a90 2024-04-25T09:18:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.185+0000 7f51d2000640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b4d0 0x7f51b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.185+0000 7f51d2000640 1 -- 172.21.15.62:0/3894837569 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f51d40894a0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.185+0000 7f51d9e00640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b4d0 0x7f51b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.187+0000 7f51d9e00640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b4d0 0x7f51b806d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009a10 tx=0x7f51cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:13.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.190+0000 7f51d2000640 1 -- 172.21.15.62:0/3894837569 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51d40582d0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.396+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51dc0ce580 con 0x7f51dc0b6a90 2024-04-25T09:18:13.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.398+0000 7f51d2000640 1 -- 172.21.15.62:0/3894837569 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16765 (secure 0 0 0) 0x7f51d4057bd0 con 0x7f51dc0b6a90 2024-04-25T09:18:13.384 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:13.384 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":150,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:18:04.356543+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:17:28.263758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:17:29.401316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.10","mappings":[{"from":2,"to":5}]},{"pgid":"21.11","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:18:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b4d0 msgr2=0x7f51b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b4d0 0x7f51b806d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009a10 tx=0x7f51cc008040 comp rx=0 tx=0).stop 2024-04-25T09:18:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 msgr2=0x7f51dc158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc158590 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f51d400c4b0 tx=0x7f51d400c970 comp rx=0 tx=0).stop 2024-04-25T09:18:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 shutdown_connections 2024-04-25T09:18:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f51b806b4d0 0x7f51b806d990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51dc0b6a90 0x7f51dc158590 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51dc0b56f0 0x7f51dc158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 --2- 172.21.15.62:0/3894837569 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f51dc0b27a0 0x7f51dc14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 >> 172.21.15.62:0/3894837569 conn(0x7f51dc066d10 msgr2=0x7f51dc160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 shutdown_connections 2024-04-25T09:18:13.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.401+0000 7f51db200640 1 -- 172.21.15.62:0/3894837569 wait complete. 2024-04-25T09:18:13.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:17:28.263758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:17:29.401316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:18:13.406 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:18:13.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.639+0000 7f2643a00640 1 Processor -- start 2024-04-25T09:18:13.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.640+0000 7f2643a00640 1 -- start start 2024-04-25T09:18:13.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.641+0000 7f2643a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f263c15dd30 0x7f263c0b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.641+0000 7f2643a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f263c0d3ab0 0x7f263c0d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.641+0000 7f2643a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c0d1d40 con 0x7f263c0b2ea0 2024-04-25T09:18:13.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.641+0000 7f2643a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c0d4440 con 0x7f263c15dd30 2024-04-25T09:18:13.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.641+0000 7f2643a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c0d1fe0 con 0x7f263c0d3ab0 2024-04-25T09:18:13.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.641+0000 7f2641000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f263c15dd30 0x7f263c0b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.642+0000 7f2641000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f263c15dd30 0x7f263c0b9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45244/0 (socket says 172.21.15.62:45244) 2024-04-25T09:18:13.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.642+0000 7f2641000640 1 -- 172.21.15.62:0/3011150970 learned_addr learned my addr 172.21.15.62:0/3011150970 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:13.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.642+0000 7f2641000640 1 -- 172.21.15.62:0/3011150970 >> v2:172.21.15.121:3300/0 conn(0x7f263c0d3ab0 msgr2=0x7f263c0d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:13.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.642+0000 7f2641000640 1 --2- 172.21.15.62:0/3011150970 >> v2:172.21.15.121:3300/0 conn(0x7f263c0d3ab0 0x7f263c0d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.642+0000 7f2641000640 1 -- 172.21.15.62:0/3011150970 >> v1:172.21.15.121:6789/0 conn(0x7f263c0b2ea0 legacy=0x7f263c0b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:13.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.642+0000 7f2641000640 1 -- 172.21.15.62:0/3011150970 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c0d22c0 con 0x7f263c15dd30 2024-04-25T09:18:13.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.643+0000 7f2641000640 1 --2- 172.21.15.62:0/3011150970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c15dd30 0x7f263c0b9440 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f263c0d02c0 tx=0x7f262402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f47b5307c77937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:13.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.643+0000 7f263b400640 1 -- 172.21.15.62:0/3011150970 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624037070 con 0x7f263c15dd30 2024-04-25T09:18:13.629 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.644+0000 7f263b400640 1 -- 172.21.15.62:0/3011150970 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2624032440 con 0x7f263c15dd30 2024-04-25T09:18:13.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.644+0000 7f263b400640 1 -- 172.21.15.62:0/3011150970 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624032810 con 0x7f263c15dd30 2024-04-25T09:18:13.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.644+0000 7f2643a00640 1 -- 172.21.15.62:0/3011150970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c15dd30 msgr2=0x7f263c0b9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.644+0000 7f2643a00640 1 --2- 172.21.15.62:0/3011150970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c15dd30 0x7f263c0b9440 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f263c0d02c0 tx=0x7f262402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.644+0000 7f2643a00640 1 -- 172.21.15.62:0/3011150970 shutdown_connections 2024-04-25T09:18:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.644+0000 7f2643a00640 1 --2- 172.21.15.62:0/3011150970 >> v2:172.21.15.121:3300/0 conn(0x7f263c0d3ab0 0x7f263c0d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.644+0000 7f2643a00640 1 --2- 172.21.15.62:0/3011150970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c15dd30 0x7f263c0b9440 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.644+0000 7f2643a00640 1 -- 172.21.15.62:0/3011150970 >> 172.21.15.62:0/3011150970 conn(0x7f263c06b000 msgr2=0x7f263c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.645+0000 7f2643a00640 1 -- 172.21.15.62:0/3011150970 shutdown_connections 2024-04-25T09:18:13.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.645+0000 7f2643a00640 1 -- 172.21.15.62:0/3011150970 wait complete. 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.646+0000 7f2643a00640 1 Processor -- start 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.646+0000 7f2643a00640 1 -- start start 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.647+0000 7f2643a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f263c0b2ea0 0x7f263c0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.647+0000 7f2643a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c0d3ab0 0x7f263c14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.647+0000 7f2643a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f263c15dd30 0x7f263c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.647+0000 7f2643a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b7360 con 0x7f263c0d3ab0 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.647+0000 7f2643a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c15e110 con 0x7f263c15dd30 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.647+0000 7f2643a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b9980 con 0x7f263c0b2ea0 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f263be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c0d3ab0 0x7f263c14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f263be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c0d3ab0 0x7f263c14d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45256/0 (socket says 172.21.15.62:45256) 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f263be00640 1 -- 172.21.15.62:0/759191711 learned_addr learned my addr 172.21.15.62:0/759191711 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f2641000640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f263c0b2ea0 0x7f263c0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f2641a00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f263c15dd30 0x7f263c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f263be00640 1 -- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f263c0b2ea0 msgr2=0x7f263c0c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f263be00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f263c0b2ea0 0x7f263c0c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f263be00640 1 -- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f263c15dd30 msgr2=0x7f263c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f263be00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f263c15dd30 0x7f263c0cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.648+0000 7f263be00640 1 -- 172.21.15.62:0/759191711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c15fb80 con 0x7f263c0d3ab0 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.649+0000 7f263be00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c0d3ab0 0x7f263c14d050 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f262c00b9e0 tx=0x7f262c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.649+0000 7f2638c00640 1 -- 172.21.15.62:0/759191711 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c019070 con 0x7f263c0d3ab0 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.649+0000 7f2638c00640 1 -- 172.21.15.62:0/759191711 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c00cba0 con 0x7f263c0d3ab0 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.649+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c15fe00 con 0x7f263c0d3ab0 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.649+0000 7f2638c00640 1 -- 172.21.15.62:0/759191711 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c014440 con 0x7f263c0d3ab0 2024-04-25T09:18:13.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.649+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c1603c0 con 0x7f263c0d3ab0 2024-04-25T09:18:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.651+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c04ef20 con 0x7f263c0d3ab0 2024-04-25T09:18:13.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.651+0000 7f2638c00640 1 -- 172.21.15.62:0/759191711 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f262c0145e0 con 0x7f263c0d3ab0 2024-04-25T09:18:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.652+0000 7f2638c00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f261806b4d0 0x7f261806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:13.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.652+0000 7f2638c00640 1 -- 172.21.15.62:0/759191711 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f262c08a0e0 con 0x7f263c0d3ab0 2024-04-25T09:18:13.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.653+0000 7f2641000640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f261806b4d0 0x7f261806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:13.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.654+0000 7f2641000640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f261806b4d0 0x7f261806d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2624034000 tx=0x7f262402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:13.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.657+0000 7f2638c00640 1 -- 172.21.15.62:0/759191711 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f263c15fb80 con 0x7f263c0d3ab0 2024-04-25T09:18:13.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.858+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f263c0d3f00 con 0x7f263c0d3ab0 2024-04-25T09:18:13.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.859+0000 7f2638c00640 1 -- 172.21.15.62:0/759191711 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7f262c058830 con 0x7f263c0d3ab0 2024-04-25T09:18:13.845 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:18:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.862+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f261806b4d0 msgr2=0x7f261806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.862+0000 7f2643a00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f261806b4d0 0x7f261806d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2624034000 tx=0x7f262402e000 comp rx=0 tx=0).stop 2024-04-25T09:18:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.862+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c0d3ab0 msgr2=0x7f263c14d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.862+0000 7f2643a00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c0d3ab0 0x7f263c14d050 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f262c00b9e0 tx=0x7f262c00bea0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.863+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 shutdown_connections 2024-04-25T09:18:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.863+0000 7f2643a00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f261806b4d0 0x7f261806d990 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.863+0000 7f2643a00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f263c0d3ab0 0x7f263c14d050 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.863+0000 7f2643a00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f263c15dd30 0x7f263c0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.863+0000 7f2643a00640 1 --2- 172.21.15.62:0/759191711 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f263c0b2ea0 0x7f263c0c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.863+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 >> 172.21.15.62:0/759191711 conn(0x7f263c06b000 msgr2=0x7f263c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.863+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 shutdown_connections 2024-04-25T09:18:13.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:13.863+0000 7f2643a00640 1 -- 172.21.15.62:0/759191711 wait complete. 2024-04-25T09:18:13.866 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:18:14.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.084+0000 7f84e3200640 1 Processor -- start 2024-04-25T09:18:14.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.084+0000 7f84e3200640 1 -- start start 2024-04-25T09:18:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.086+0000 7f84e3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84e40b31a0 0x7f84e40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.086+0000 7f84e3200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e40cd260 con 0x7f84e406d2c0 2024-04-25T09:18:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.086+0000 7f84e3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e40cf000 con 0x7f84e40ccde0 2024-04-25T09:18:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.086+0000 7f84e3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40cf2e0 con 0x7f84e40b31a0 2024-04-25T09:18:14.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e1e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f84e406d2c0 0x7f84e406d6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36624/0 (socket says 172.21.15.62:36624) 2024-04-25T09:18:14.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e1e00640 1 -- 172.21.15.62:0/2725138806 learned_addr learned my addr 172.21.15.62:0/2725138806 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e2800640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84e40b31a0 0x7f84e40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909062866 0 0) 0x7f84e40cd260 con 0x7f84e406d2c0 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c8003520 con 0x7f84e406d2c0 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 916087938 0 0) 0x7f84c8003520 con 0x7f84e406d2c0 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84e40b31a0 msgr2=0x7f84e40b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e0a00640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84e40b31a0 0x7f84e40b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 >> v1:172.21.15.121:6789/0 conn(0x7f84e40ccde0 legacy=0x7f84e40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40b26c0 con 0x7f84e406d2c0 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.087+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f84d4001670 con 0x7f84e406d2c0 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 >> v1:172.21.15.62:6789/0 conn(0x7f84e406d2c0 legacy=0x7f84e406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e0a00640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84c80039e0 0x7f84c8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e0a00640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84e40b31a0 0x7f84c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e0a00640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c8004d40 0x7f84c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40cf2e0 con 0x7f84c8004d40 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c800baa0 con 0x7f84c80039e0 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c800bca0 con 0x7f84e40b31a0 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e2800640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c8004d40 0x7f84c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e1400640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84e40b31a0 0x7f84c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e1e00640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84c80039e0 0x7f84c8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e2800640 1 -- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84e40b31a0 msgr2=0x7f84c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e2800640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84e40b31a0 0x7f84c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e2800640 1 -- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84c80039e0 msgr2=0x7f84c8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e2800640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84c80039e0 0x7f84c8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.088+0000 7f84e2800640 1 -- 172.21.15.62:0/2725138806 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c8003520 con 0x7f84c8004d40 2024-04-25T09:18:14.077 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.090+0000 7f84e2800640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c8004d40 0x7f84c80051c0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009f90 tx=0x7f84c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f199ec0f9c9621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.090+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc009c10 con 0x7f84c8004d40 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.090+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc036070 con 0x7f84c8004d40 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.090+0000 7f84e0a00640 1 -- 172.21.15.62:0/2725138806 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc031850 con 0x7f84c8004d40 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.090+0000 7f84e3200640 1 -- 172.21.15.62:0/2725138806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c8004d40 msgr2=0x7f84c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.090+0000 7f84e3200640 1 --2- 172.21.15.62:0/2725138806 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84c8004d40 0x7f84c80051c0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009f90 tx=0x7f84c8005830 comp rx=0 tx=0).stop 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.090+0000 7f84e2800640 1 -- 172.21.15.62:0/2725138806 reap_dead start 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.091+0000 7f84e3200640 1 -- 172.21.15.62:0/2725138806 shutdown_connections 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.091+0000 7f84e3200640 1 -- 172.21.15.62:0/2725138806 >> 172.21.15.62:0/2725138806 conn(0x7f84e4066f20 msgr2=0x7f84e40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.091+0000 7f84e3200640 1 -- 172.21.15.62:0/2725138806 shutdown_connections 2024-04-25T09:18:14.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.091+0000 7f84e3200640 1 -- 172.21.15.62:0/2725138806 wait complete. 2024-04-25T09:18:14.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.092+0000 7f84e3200640 1 Processor -- start 2024-04-25T09:18:14.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.093+0000 7f84e3200640 1 -- start start 2024-04-25T09:18:14.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.093+0000 7f84e3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84c8004d40 0x7f84e415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84e40ae090 0x7f84e41f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e415d610 0x7f84e415da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41448d0 con 0x7f84e415d610 2024-04-25T09:18:14.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40b2c80 con 0x7f84e40ae090 2024-04-25T09:18:14.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4144a50 con 0x7f84c8004d40 2024-04-25T09:18:14.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e415d610 0x7f84e415da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e415d610 0x7f84e415da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45276/0 (socket says 172.21.15.62:45276) 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e2800640 1 -- 172.21.15.62:0/3101251993 learned_addr learned my addr 172.21.15.62:0/3101251993 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e1e00640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84c8004d40 0x7f84e415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e1400640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84e40ae090 0x7f84e41f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e2800640 1 -- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84c8004d40 msgr2=0x7f84e415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e2800640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84c8004d40 0x7f84e415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e2800640 1 -- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84e40ae090 msgr2=0x7f84e41f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e2800640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84e40ae090 0x7f84e41f4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e2800640 1 -- 172.21.15.62:0/3101251993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40c46d0 con 0x7f84e415d610 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e1e00640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84c8004d40 0x7f84e415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.094+0000 7f84e1400640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84e40ae090 0x7f84e41f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.095+0000 7f84e2800640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e415d610 0x7f84e415da90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00e230 tx=0x7f84dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.095+0000 7f84da000640 1 -- 172.21.15.62:0/3101251993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc003990 con 0x7f84e415d610 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.095+0000 7f84da000640 1 -- 172.21.15.62:0/3101251993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc003b50 con 0x7f84e415d610 2024-04-25T09:18:14.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.095+0000 7f84da000640 1 -- 172.21.15.62:0/3101251993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004020 con 0x7f84e415d610 2024-04-25T09:18:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.095+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e41f55c0 con 0x7f84e415d610 2024-04-25T09:18:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.095+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e40c4970 con 0x7f84e415d610 2024-04-25T09:18:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.096+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e406abe0 con 0x7f84e415d610 2024-04-25T09:18:14.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.097+0000 7f84da000640 1 -- 172.21.15.62:0/3101251993 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84dc005d10 con 0x7f84e415d610 2024-04-25T09:18:14.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.098+0000 7f84da000640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f84c006b4d0 0x7f84c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.098+0000 7f84da000640 1 -- 172.21.15.62:0/3101251993 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f84dc0ab310 con 0x7f84e415d610 2024-04-25T09:18:14.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.099+0000 7f84e1e00640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f84c006b4d0 0x7f84c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.100+0000 7f84e1e00640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f84c006b4d0 0x7f84c006d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f84d4001eb0 tx=0x7f84d4001dc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.102+0000 7f84da000640 1 -- 172.21.15.62:0/3101251993 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84dc02e050 con 0x7f84e415d610 2024-04-25T09:18:14.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.300+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84e40c4b70 con 0x7f84e415d610 2024-04-25T09:18:14.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.301+0000 7f84da000640 1 -- 172.21.15.62:0/3101251993 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84e40c4b70 con 0x7f84e415d610 2024-04-25T09:18:14.287 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:18:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f84c006b4d0 msgr2=0x7f84c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f84c006b4d0 0x7f84c006d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f84d4001eb0 tx=0x7f84d4001dc0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e415d610 msgr2=0x7f84e415da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e415d610 0x7f84e415da90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00e230 tx=0x7f84dc009750 comp rx=0 tx=0).stop 2024-04-25T09:18:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 shutdown_connections 2024-04-25T09:18:14.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f84c006b4d0 0x7f84c006d990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e415d610 0x7f84e415da90 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f84e40ae090 0x7f84e41f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 --2- 172.21.15.62:0/3101251993 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f84c8004d40 0x7f84e415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 >> 172.21.15.62:0/3101251993 conn(0x7f84e4066f20 msgr2=0x7f84e406c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 shutdown_connections 2024-04-25T09:18:14.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.304+0000 7f84e3200640 1 -- 172.21.15.62:0/3101251993 wait complete. 2024-04-25T09:18:14.309 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:18:14.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.529+0000 7f49f8a00640 1 Processor -- start 2024-04-25T09:18:14.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.529+0000 7f49f8a00640 1 -- start start 2024-04-25T09:18:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.531+0000 7f49f8a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f49f406d2c0 0x7f49f40ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.531+0000 7f49f8a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f49f406d6a0 0x7f49f40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.531+0000 7f49f8a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f40ced40 con 0x7f49f40b31a0 2024-04-25T09:18:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.531+0000 7f49f8a00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f40cf000 con 0x7f49f406d6a0 2024-04-25T09:18:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.531+0000 7f49f8a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f40cf2e0 con 0x7f49f406d2c0 2024-04-25T09:18:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.531+0000 7f49f1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f49f406d6a0 0x7f49f40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.531+0000 7f49f1e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f49f406d6a0 0x7f49f40ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45286/0 (socket says 172.21.15.62:45286) 2024-04-25T09:18:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.531+0000 7f49f1e00640 1 -- 172.21.15.62:0/3827637139 learned_addr learned my addr 172.21.15.62:0/3827637139 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.532+0000 7f49f2800640 1 --2- 172.21.15.62:0/3827637139 >> v2:172.21.15.121:3300/0 conn(0x7f49f406d2c0 0x7f49f40ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.532+0000 7f49f1e00640 1 -- 172.21.15.62:0/3827637139 >> v2:172.21.15.121:3300/0 conn(0x7f49f406d2c0 msgr2=0x7f49f40ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.532+0000 7f49f1e00640 1 --2- 172.21.15.62:0/3827637139 >> v2:172.21.15.121:3300/0 conn(0x7f49f406d2c0 0x7f49f40ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.532+0000 7f49f1e00640 1 -- 172.21.15.62:0/3827637139 >> v1:172.21.15.62:6789/0 conn(0x7f49f40b31a0 legacy=0x7f49f40b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.532+0000 7f49f1e00640 1 -- 172.21.15.62:0/3827637139 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40b26c0 con 0x7f49f406d6a0 2024-04-25T09:18:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.533+0000 7f49f1e00640 1 --2- 172.21.15.62:0/3827637139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f406d6a0 0x7f49f40ce800 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f49f40b20a0 tx=0x7f49e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fcba083a4293b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.533+0000 7f49f1400640 1 -- 172.21.15.62:0/3827637139 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0037070 con 0x7f49f406d6a0 2024-04-25T09:18:14.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.533+0000 7f49f1400640 1 -- 172.21.15.62:0/3827637139 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e0032440 con 0x7f49f406d6a0 2024-04-25T09:18:14.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.533+0000 7f49f1400640 1 -- 172.21.15.62:0/3827637139 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0032810 con 0x7f49f406d6a0 2024-04-25T09:18:14.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.534+0000 7f49f8a00640 1 -- 172.21.15.62:0/3827637139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f406d6a0 msgr2=0x7f49f40ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.534+0000 7f49f8a00640 1 --2- 172.21.15.62:0/3827637139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f406d6a0 0x7f49f40ce800 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f49f40b20a0 tx=0x7f49e002a270 comp rx=0 tx=0).stop 2024-04-25T09:18:14.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.534+0000 7f49f8a00640 1 -- 172.21.15.62:0/3827637139 shutdown_connections 2024-04-25T09:18:14.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.534+0000 7f49f8a00640 1 --2- 172.21.15.62:0/3827637139 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f406d6a0 0x7f49f40ce800 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.534+0000 7f49f8a00640 1 --2- 172.21.15.62:0/3827637139 >> v2:172.21.15.121:3300/0 conn(0x7f49f406d2c0 0x7f49f40ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.534+0000 7f49f8a00640 1 -- 172.21.15.62:0/3827637139 >> 172.21.15.62:0/3827637139 conn(0x7f49f4066f20 msgr2=0x7f49f40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:14.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.534+0000 7f49f8a00640 1 -- 172.21.15.62:0/3827637139 shutdown_connections 2024-04-25T09:18:14.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.535+0000 7f49f8a00640 1 -- 172.21.15.62:0/3827637139 wait complete. 2024-04-25T09:18:14.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.537+0000 7f49f8a00640 1 Processor -- start 2024-04-25T09:18:14.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.537+0000 7f49f8a00640 1 -- start start 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.538+0000 7f49f8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f406d2c0 0x7f49f41f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.538+0000 7f49f8a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f406d6a0 0x7f49f41e4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.538+0000 7f49f8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b31a0 0x7f49f415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.538+0000 7f49f8a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f406db20 con 0x7f49f40b31a0 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f8a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40b2c80 con 0x7f49f406d2c0 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f8a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f41448d0 con 0x7f49f406d6a0 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b31a0 0x7f49f415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b31a0 0x7f49f415de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:45296/0 (socket says 172.21.15.62:45296) 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f3200640 1 -- 172.21.15.62:0/1272397625 learned_addr learned my addr 172.21.15.62:0/1272397625 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:14.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f2800640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f406d2c0 0x7f49f41f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f1e00640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f406d6a0 0x7f49f41e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f3200640 1 -- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f406d6a0 msgr2=0x7f49f41e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f3200640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f406d6a0 0x7f49f41e4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f3200640 1 -- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f406d2c0 msgr2=0x7f49f41f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f3200640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f406d2c0 0x7f49f41f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.539+0000 7f49f3200640 1 -- 172.21.15.62:0/1272397625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40adcc0 con 0x7f49f40b31a0 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.540+0000 7f49f3200640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b31a0 0x7f49f415de80 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c6b0 tx=0x7f49ec00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.540+0000 7f49eaa00640 1 -- 172.21.15.62:0/1272397625 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec003b70 con 0x7f49f40b31a0 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.540+0000 7f49eaa00640 1 -- 172.21.15.62:0/1272397625 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec003d30 con 0x7f49f40b31a0 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.540+0000 7f49eaa00640 1 -- 172.21.15.62:0/1272397625 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec005310 con 0x7f49f40b31a0 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.540+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f40adfa0 con 0x7f49f40b31a0 2024-04-25T09:18:14.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.540+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f40ae560 con 0x7f49f40b31a0 2024-04-25T09:18:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.542+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f406abe0 con 0x7f49f40b31a0 2024-04-25T09:18:14.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.543+0000 7f49eaa00640 1 -- 172.21.15.62:0/1272397625 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49ec0054b0 con 0x7f49f40b31a0 2024-04-25T09:18:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.544+0000 7f49eaa00640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f49d406b4d0 0x7f49d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.544+0000 7f49eaa00640 1 -- 172.21.15.62:0/1272397625 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f49ec08a0d0 con 0x7f49f40b31a0 2024-04-25T09:18:14.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.544+0000 7f49f2800640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f49d406b4d0 0x7f49d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.546+0000 7f49f2800640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f49d406b4d0 0x7f49d406d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.548+0000 7f49eaa00640 1 -- 172.21.15.62:0/1272397625 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49ec058f00 con 0x7f49f40b31a0 2024-04-25T09:18:14.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.748+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49f404ef20 con 0x7f49f40b31a0 2024-04-25T09:18:14.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.749+0000 7f49eaa00640 1 -- 172.21.15.62:0/1272397625 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49ec058800 con 0x7f49f40b31a0 2024-04-25T09:18:14.736 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:18:14.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f49d406b4d0 msgr2=0x7f49d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f49d406b4d0 0x7f49d406d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4008040 comp rx=0 tx=0).stop 2024-04-25T09:18:14.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b31a0 msgr2=0x7f49f415de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.738 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b31a0 0x7f49f415de80 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c6b0 tx=0x7f49ec00cb70 comp rx=0 tx=0).stop 2024-04-25T09:18:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 shutdown_connections 2024-04-25T09:18:14.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f49d406b4d0 0x7f49d406d990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49f40b31a0 0x7f49f415de80 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f49f406d6a0 0x7f49f41e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 --2- 172.21.15.62:0/1272397625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49f406d2c0 0x7f49f41f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.752+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 >> 172.21.15.62:0/1272397625 conn(0x7f49f4066f20 msgr2=0x7f49f415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.753+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 shutdown_connections 2024-04-25T09:18:14.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.753+0000 7f49f8a00640 1 -- 172.21.15.62:0/1272397625 wait complete. 2024-04-25T09:18:14.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:18:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.983+0000 7f9b66c00640 1 Processor -- start 2024-04-25T09:18:14.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.983+0000 7f9b66c00640 1 -- start start 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.984+0000 7f9b66c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b6006ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.984+0000 7f9b66c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9b6006d3f0 0x7f9b600b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.984+0000 7f9b66c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6006d8a0 con 0x7f9b600aef10 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.984+0000 7f9b66c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6006db60 con 0x7f9b6006d3f0 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.984+0000 7f9b66c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6006de40 con 0x7f9b600aeb30 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.985+0000 7f9b64e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9b6006d3f0 0x7f9b600b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.985+0000 7f9b64e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9b6006d3f0 0x7f9b600b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40024/0 (socket says 172.21.15.62:40024) 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.985+0000 7f9b64e00640 1 -- 172.21.15.62:0/2592285238 learned_addr learned my addr 172.21.15.62:0/2592285238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:14.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.985+0000 7f9b65800640 1 --2- 172.21.15.62:0/2592285238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b6006ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.985+0000 7f9b64e00640 1 -- 172.21.15.62:0/2592285238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 msgr2=0x7f9b6006ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.985+0000 7f9b64e00640 1 --2- 172.21.15.62:0/2592285238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b6006ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.985+0000 7f9b64e00640 1 -- 172.21.15.62:0/2592285238 >> v1:172.21.15.62:6789/0 conn(0x7f9b600aef10 legacy=0x7f9b600713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.985+0000 7f9b64e00640 1 -- 172.21.15.62:0/2592285238 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6015b150 con 0x7f9b6006d3f0 2024-04-25T09:18:14.972 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.986+0000 7f9b64e00640 1 --2- 172.21.15.62:0/2592285238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b600b3190 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00bb10 tx=0x7f9b4c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8614cf4068143f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b5fe00640 1 -- 172.21.15.62:0/2592285238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c032790 con 0x7f9b6006d3f0 2024-04-25T09:18:14.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b5fe00640 1 -- 172.21.15.62:0/2592285238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c038070 con 0x7f9b6006d3f0 2024-04-25T09:18:14.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b5fe00640 1 -- 172.21.15.62:0/2592285238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c032b50 con 0x7f9b6006d3f0 2024-04-25T09:18:14.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b66c00640 1 -- 172.21.15.62:0/2592285238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 msgr2=0x7f9b600b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b66c00640 1 --2- 172.21.15.62:0/2592285238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b600b3190 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00bb10 tx=0x7f9b4c02c890 comp rx=0 tx=0).stop 2024-04-25T09:18:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b66c00640 1 -- 172.21.15.62:0/2592285238 shutdown_connections 2024-04-25T09:18:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b66c00640 1 --2- 172.21.15.62:0/2592285238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b600b3190 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b66c00640 1 --2- 172.21.15.62:0/2592285238 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b6006ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.987+0000 7f9b66c00640 1 -- 172.21.15.62:0/2592285238 >> 172.21.15.62:0/2592285238 conn(0x7f9b600668f0 msgr2=0x7f9b60066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.988+0000 7f9b66c00640 1 -- 172.21.15.62:0/2592285238 shutdown_connections 2024-04-25T09:18:14.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.988+0000 7f9b66c00640 1 -- 172.21.15.62:0/2592285238 wait complete. 2024-04-25T09:18:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.989+0000 7f9b66c00640 1 Processor -- start 2024-04-25T09:18:14.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.990+0000 7f9b66c00640 1 -- start start 2024-04-25T09:18:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.990+0000 7f9b66c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b60144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.990+0000 7f9b66c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b600cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.990+0000 7f9b66c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b600aef10 0x7f9b601540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b66c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6015ae90 con 0x7f9b6006d3f0 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b66c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6015ad10 con 0x7f9b600aef10 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b66c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6015c680 con 0x7f9b600aeb30 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b65800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b60144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b65800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b60144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40036/0 (socket says 172.21.15.62:40036) 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b65800640 1 -- 172.21.15.62:0/983541295 learned_addr learned my addr 172.21.15.62:0/983541295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b64e00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b600cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b65800640 1 -- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 msgr2=0x7f9b600cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b65800640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b600cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b65800640 1 -- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b600aef10 msgr2=0x7f9b601540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b66200640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b600aef10 0x7f9b601540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b65800640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b600aef10 0x7f9b601540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b65800640 1 -- 172.21.15.62:0/983541295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60145280 con 0x7f9b6006d3f0 2024-04-25T09:18:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b64e00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b600cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.991+0000 7f9b66200640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b600aef10 0x7f9b601540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:14.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.992+0000 7f9b65800640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b60144d40 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400d800 tx=0x7f9b5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.992+0000 7f9b5d600640 1 -- 172.21.15.62:0/983541295 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400b6c0 con 0x7f9b6006d3f0 2024-04-25T09:18:14.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.992+0000 7f9b5d600640 1 -- 172.21.15.62:0/983541295 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b54019070 con 0x7f9b6006d3f0 2024-04-25T09:18:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.992+0000 7f9b5d600640 1 -- 172.21.15.62:0/983541295 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400ba80 con 0x7f9b6006d3f0 2024-04-25T09:18:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.992+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b600ccc80 con 0x7f9b6006d3f0 2024-04-25T09:18:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.992+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b600ce940 con 0x7f9b6006d3f0 2024-04-25T09:18:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.994+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b600b4a80 con 0x7f9b6006d3f0 2024-04-25T09:18:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.994+0000 7f9b5d600640 1 -- 172.21.15.62:0/983541295 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b5400bc20 con 0x7f9b6006d3f0 2024-04-25T09:18:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.995+0000 7f9b5d600640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4006b4d0 0x7f9b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:14.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.995+0000 7f9b5d600640 1 -- 172.21.15.62:0/983541295 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9b5408a700 con 0x7f9b6006d3f0 2024-04-25T09:18:14.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.996+0000 7f9b64e00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4006b4d0 0x7f9b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:14.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:14.997+0000 7f9b64e00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4006b4d0 0x7f9b4006d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c037000 tx=0x7f9b4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:14.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.000+0000 7f9b5d600640 1 -- 172.21.15.62:0/983541295 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b600b4a80 con 0x7f9b6006d3f0 2024-04-25T09:18:15.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.210+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9b6004ef20 con 0x7f9b6006d3f0 2024-04-25T09:18:15.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.211+0000 7f9b5d600640 1 -- 172.21.15.62:0/983541295 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v150) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9b60145280 con 0x7f9b6006d3f0 2024-04-25T09:18:15.197 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-25T09:18:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4006b4d0 msgr2=0x7f9b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4006b4d0 0x7f9b4006d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c037000 tx=0x7f9b4c008040 comp rx=0 tx=0).stop 2024-04-25T09:18:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 msgr2=0x7f9b60144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b60144d40 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400d800 tx=0x7f9b5400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:18:15.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 shutdown_connections 2024-04-25T09:18:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9b4006b4d0 0x7f9b4006d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b600aef10 0x7f9b601540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9b600aeb30 0x7f9b600cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 --2- 172.21.15.62:0/983541295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9b6006d3f0 0x7f9b60144d40 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 >> 172.21.15.62:0/983541295 conn(0x7f9b600668f0 msgr2=0x7f9b600b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 shutdown_connections 2024-04-25T09:18:15.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:15.214+0000 7f9b66c00640 1 -- 172.21.15.62:0/983541295 wait complete. 2024-04-25T09:18:15.219 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:18:15.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:15.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:18:15.843 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:18:15.853 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:18:16.990 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:18:16.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:16.990 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:18:17.030 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:18:17.030 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:17.074 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:18:17.074 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:18:17.074 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:17.117 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:18:17.117 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:18:17.117 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:17.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:18:17.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:18:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.394+0000 7f6dd3200640 1 Processor -- start 2024-04-25T09:18:17.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.394+0000 7f6dd3200640 1 -- start start 2024-04-25T09:18:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.396+0000 7f6dd3200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6dd406d900 0x7f6dd415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.396+0000 7f6dd3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6dd40cee40 0x7f6dd40b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.396+0000 7f6dd3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd415faa0 con 0x7f6dd4144c10 2024-04-25T09:18:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.396+0000 7f6dd3200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd4144ff0 con 0x7f6dd406d900 2024-04-25T09:18:17.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.396+0000 7f6dd3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd4160340 con 0x7f6dd40cee40 2024-04-25T09:18:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.397+0000 7f6dd1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6dd406d900 0x7f6dd415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.397+0000 7f6dd1400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6dd406d900 0x7f6dd415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40052/0 (socket says 172.21.15.62:40052) 2024-04-25T09:18:17.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.397+0000 7f6dd1400640 1 -- 172.21.15.62:0/406340835 learned_addr learned my addr 172.21.15.62:0/406340835 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.397+0000 7f6dd2800640 1 --2- 172.21.15.62:0/406340835 >> v2:172.21.15.121:3300/0 conn(0x7f6dd40cee40 0x7f6dd40b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.397+0000 7f6dd1400640 1 -- 172.21.15.62:0/406340835 >> v2:172.21.15.121:3300/0 conn(0x7f6dd40cee40 msgr2=0x7f6dd40b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.397+0000 7f6dd1400640 1 --2- 172.21.15.62:0/406340835 >> v2:172.21.15.121:3300/0 conn(0x7f6dd40cee40 0x7f6dd40b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.397+0000 7f6dd1400640 1 -- 172.21.15.62:0/406340835 >> v1:172.21.15.121:6789/0 conn(0x7f6dd4144c10 legacy=0x7f6dd406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.397+0000 7f6dd1400640 1 -- 172.21.15.62:0/406340835 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd415fd80 con 0x7f6dd406d900 2024-04-25T09:18:17.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.398+0000 7f6dd1400640 1 --2- 172.21.15.62:0/406340835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd406d900 0x7f6dd415f450 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f6dd415f390 tx=0x7f6dc002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f76b13dd52fbba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:17.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.399+0000 7f6dd0a00640 1 -- 172.21.15.62:0/406340835 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0037070 con 0x7f6dd406d900 2024-04-25T09:18:17.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.399+0000 7f6dd0a00640 1 -- 172.21.15.62:0/406340835 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc0032440 con 0x7f6dd406d900 2024-04-25T09:18:17.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.399+0000 7f6dd0a00640 1 -- 172.21.15.62:0/406340835 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0032810 con 0x7f6dd406d900 2024-04-25T09:18:17.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.399+0000 7f6dd3200640 1 -- 172.21.15.62:0/406340835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd406d900 msgr2=0x7f6dd415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.399+0000 7f6dd3200640 1 --2- 172.21.15.62:0/406340835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd406d900 0x7f6dd415f450 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f6dd415f390 tx=0x7f6dc002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.400+0000 7f6dd3200640 1 -- 172.21.15.62:0/406340835 shutdown_connections 2024-04-25T09:18:17.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.400+0000 7f6dd3200640 1 --2- 172.21.15.62:0/406340835 >> v2:172.21.15.121:3300/0 conn(0x7f6dd40cee40 0x7f6dd40b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.400+0000 7f6dd3200640 1 --2- 172.21.15.62:0/406340835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd406d900 0x7f6dd415f450 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.400+0000 7f6dd3200640 1 -- 172.21.15.62:0/406340835 >> 172.21.15.62:0/406340835 conn(0x7f6dd40b14d0 msgr2=0x7f6dd40b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:17.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.400+0000 7f6dd3200640 1 -- 172.21.15.62:0/406340835 shutdown_connections 2024-04-25T09:18:17.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.400+0000 7f6dd3200640 1 -- 172.21.15.62:0/406340835 wait complete. 2024-04-25T09:18:17.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.401+0000 7f6dd3200640 1 Processor -- start 2024-04-25T09:18:17.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.402+0000 7f6dd3200640 1 -- start start 2024-04-25T09:18:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.402+0000 7f6dd3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dd406d900 0x7f6dd4154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.402+0000 7f6dd3200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6dd40cee40 0x7f6dd4135fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.402+0000 7f6dd3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd4144c10 0x7f6dd41364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.402+0000 7f6dd3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd40cf250 con 0x7f6dd4144c10 2024-04-25T09:18:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.402+0000 7f6dd3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd40b41c0 con 0x7f6dd406d900 2024-04-25T09:18:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.402+0000 7f6dd3200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd40b4b90 con 0x7f6dd40cee40 2024-04-25T09:18:17.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd4144c10 0x7f6dd41364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd4144c10 0x7f6dd41364f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40068/0 (socket says 172.21.15.62:40068) 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd2800640 1 -- 172.21.15.62:0/3855954021 learned_addr learned my addr 172.21.15.62:0/3855954021 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd1400640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6dd40cee40 0x7f6dd4135fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd1e00640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dd406d900 0x7f6dd4154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd2800640 1 -- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6dd40cee40 msgr2=0x7f6dd4135fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd2800640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6dd40cee40 0x7f6dd4135fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd2800640 1 -- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dd406d900 msgr2=0x7f6dd4154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd2800640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dd406d900 0x7f6dd4154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.403+0000 7f6dd2800640 1 -- 172.21.15.62:0/3855954021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd4136a30 con 0x7f6dd4144c10 2024-04-25T09:18:17.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.404+0000 7f6dd2800640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd4144c10 0x7f6dd41364f0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007e70 tx=0x7f6dcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:17.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.404+0000 7f6dca000640 1 -- 172.21.15.62:0/3855954021 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc014620 con 0x7f6dd4144c10 2024-04-25T09:18:17.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.404+0000 7f6dca000640 1 -- 172.21.15.62:0/3855954021 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc019070 con 0x7f6dd4144c10 2024-04-25T09:18:17.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.404+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd41f4070 con 0x7f6dd4144c10 2024-04-25T09:18:17.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.404+0000 7f6dca000640 1 -- 172.21.15.62:0/3855954021 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc014a20 con 0x7f6dd4144c10 2024-04-25T09:18:17.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.404+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd41f4910 con 0x7f6dd4144c10 2024-04-25T09:18:17.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.406+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd41f4240 con 0x7f6dd4144c10 2024-04-25T09:18:17.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.406+0000 7f6dca000640 1 -- 172.21.15.62:0/3855954021 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6dcc014bc0 con 0x7f6dd4144c10 2024-04-25T09:18:17.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.407+0000 7f6dca000640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6db406b4d0 0x7f6db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.407+0000 7f6dca000640 1 -- 172.21.15.62:0/3855954021 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6dcc08aab0 con 0x7f6dd4144c10 2024-04-25T09:18:17.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.408+0000 7f6dd1e00640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6db406b4d0 0x7f6db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.409+0000 7f6dd1e00640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6db406b4d0 0x7f6db406d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4006fd0 tx=0x7f6dc4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:17.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.412+0000 7f6dca000640 1 -- 172.21.15.62:0/3855954021 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6dcc059980 con 0x7f6dd4144c10 2024-04-25T09:18:17.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.656+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dd404ef20 con 0x7f6dd4144c10 2024-04-25T09:18:17.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.657+0000 7f6dca000640 1 -- 172.21.15.62:0/3855954021 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4948 (secure 0 0 0) 0x7f6dcc059280 con 0x7f6dd4144c10 2024-04-25T09:18:17.645 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:17.645 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":102,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3369583759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3369583759},{"type":"v1","addr":"172.21.15.121:6837","nonce":3369583759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1888749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1888749},{"type":"v1","addr":"172.21.15.121:6836","nonce":1888749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:17:30.382991+0000","modified":"2024-04-25T09:18:11.404400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9390":{"gid":9390,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6833/1673530113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1673530113},{"type":"v1","addr":"172.21.15.62:6833","nonce":1673530113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9396":{"gid":9396,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2793862963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2793862963},{"type":"v1","addr":"172.21.15.62:6835","nonce":2793862963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:18:17.646 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 102 2024-04-25T09:18:17.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6db406b4d0 msgr2=0x7f6db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6db406b4d0 0x7f6db406d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4006fd0 tx=0x7f6dc4008040 comp rx=0 tx=0).stop 2024-04-25T09:18:17.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd4144c10 msgr2=0x7f6dd41364f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd4144c10 0x7f6dd41364f0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007e70 tx=0x7f6dcc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 shutdown_connections 2024-04-25T09:18:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f6db406b4d0 0x7f6db406d990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6dd4144c10 0x7f6dd41364f0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f6dd40cee40 0x7f6dd4135fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.660+0000 7f6dd3200640 1 --2- 172.21.15.62:0/3855954021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dd406d900 0x7f6dd4154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.661+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 >> 172.21.15.62:0/3855954021 conn(0x7f6dd40b14d0 msgr2=0x7f6dd406c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:17.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.661+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 shutdown_connections 2024-04-25T09:18:17.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.661+0000 7f6dd3200640 1 -- 172.21.15.62:0/3855954021 wait complete. 2024-04-25T09:18:17.668 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:18:17.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.893+0000 7f67de400640 1 Processor -- start 2024-04-25T09:18:17.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.893+0000 7f67de400640 1 -- start start 2024-04-25T09:18:17.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.895+0000 7f67de400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f67d8144c10 0x7f67d815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.895+0000 7f67de400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d80cf2d0 con 0x7f67d806d900 2024-04-25T09:18:17.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.895+0000 7f67de400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d815faa0 con 0x7f67d80cee40 2024-04-25T09:18:17.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.895+0000 7f67de400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d8144ff0 con 0x7f67d8144c10 2024-04-25T09:18:17.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.895+0000 7f67dd000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f67d8144c10 0x7f67d815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.895+0000 7f67dd000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f67d8144c10 0x7f67d815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40078/0 (socket says 172.21.15.62:40078) 2024-04-25T09:18:17.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.895+0000 7f67dd000640 1 -- 172.21.15.62:0/3333782189 learned_addr learned my addr 172.21.15.62:0/3333782189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:17.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.896+0000 7f67dd000640 1 -- 172.21.15.62:0/3333782189 >> v1:172.21.15.121:6789/0 conn(0x7f67d80cee40 legacy=0x7f67d806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.896+0000 7f67dd000640 1 -- 172.21.15.62:0/3333782189 >> v1:172.21.15.62:6789/0 conn(0x7f67d806d900 legacy=0x7f67d80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.896+0000 7f67dd000640 1 -- 172.21.15.62:0/3333782189 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d815fd80 con 0x7f67d8144c10 2024-04-25T09:18:17.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.896+0000 7f67d7400640 1 -- 172.21.15.62:0/3333782189 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548193571 0 0) 0x7f67d80cf2d0 con 0x7f67d806d900 2024-04-25T09:18:17.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.897+0000 7f67dd000640 1 --2- 172.21.15.62:0/3333782189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d8144c10 0x7f67d815f450 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009870 tx=0x7f67cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68585b52e0479245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:17.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.897+0000 7f67d7400640 1 -- 172.21.15.62:0/3333782189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc037070 con 0x7f67d8144c10 2024-04-25T09:18:17.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.897+0000 7f67d7400640 1 -- 172.21.15.62:0/3333782189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc032440 con 0x7f67d8144c10 2024-04-25T09:18:17.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.897+0000 7f67d7400640 1 -- 172.21.15.62:0/3333782189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc032810 con 0x7f67d8144c10 2024-04-25T09:18:17.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.898+0000 7f67de400640 1 -- 172.21.15.62:0/3333782189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d8144c10 msgr2=0x7f67d815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.898+0000 7f67de400640 1 --2- 172.21.15.62:0/3333782189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d8144c10 0x7f67d815f450 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009870 tx=0x7f67cc02a270 comp rx=0 tx=0).stop 2024-04-25T09:18:17.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.898+0000 7f67de400640 1 -- 172.21.15.62:0/3333782189 shutdown_connections 2024-04-25T09:18:17.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.898+0000 7f67de400640 1 --2- 172.21.15.62:0/3333782189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d8144c10 0x7f67d815f450 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.898+0000 7f67de400640 1 -- 172.21.15.62:0/3333782189 >> 172.21.15.62:0/3333782189 conn(0x7f67d80b14d0 msgr2=0x7f67d80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:17.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.898+0000 7f67de400640 1 -- 172.21.15.62:0/3333782189 shutdown_connections 2024-04-25T09:18:17.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.898+0000 7f67de400640 1 -- 172.21.15.62:0/3333782189 wait complete. 2024-04-25T09:18:17.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.900+0000 7f67de400640 1 Processor -- start 2024-04-25T09:18:17.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.900+0000 7f67de400640 1 -- start start 2024-04-25T09:18:17.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67de400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d806d900 0x7f67d8143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67de400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d80cee40 0x7f67d815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67de400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d8144c10 0x7f67d815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67de400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80b41c0 con 0x7f67d80cee40 2024-04-25T09:18:17.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67de400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8160340 con 0x7f67d806d900 2024-04-25T09:18:17.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67de400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80b4d60 con 0x7f67d8144c10 2024-04-25T09:18:17.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d80cee40 0x7f67d815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d80cee40 0x7f67d815ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40088/0 (socket says 172.21.15.62:40088) 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.901+0000 7f67d7e00640 1 -- 172.21.15.62:0/1531860921 learned_addr learned my addr 172.21.15.62:0/1531860921 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67d7e00640 1 -- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d8144c10 msgr2=0x7f67d815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67dda00640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d8144c10 0x7f67d815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67dd000640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d806d900 0x7f67d8143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67d7e00640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d8144c10 0x7f67d815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67d7e00640 1 -- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d806d900 msgr2=0x7f67d8143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67d7e00640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d806d900 0x7f67d8143490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67d7e00640 1 -- 172.21.15.62:0/1531860921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d80c69e0 con 0x7f67d80cee40 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67dda00640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d8144c10 0x7f67d815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67dd000640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d806d900 0x7f67d8143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.902+0000 7f67d7e00640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d80cee40 0x7f67d815ad40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f67c400c450 tx=0x7f67c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.903+0000 7f67d4c00640 1 -- 172.21.15.62:0/1531860921 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40052d0 con 0x7f67d80cee40 2024-04-25T09:18:17.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.903+0000 7f67d4c00640 1 -- 172.21.15.62:0/1531860921 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c4005470 con 0x7f67d80cee40 2024-04-25T09:18:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.903+0000 7f67d4c00640 1 -- 172.21.15.62:0/1531860921 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4012870 con 0x7f67d80cee40 2024-04-25T09:18:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.903+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d80c6cc0 con 0x7f67d80cee40 2024-04-25T09:18:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.903+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d80c7280 con 0x7f67d80cee40 2024-04-25T09:18:17.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.904+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d804ef20 con 0x7f67d80cee40 2024-04-25T09:18:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.905+0000 7f67d4c00640 1 -- 172.21.15.62:0/1531860921 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67c4012a10 con 0x7f67d80cee40 2024-04-25T09:18:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.906+0000 7f67d4c00640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67b806b4d0 0x7f67b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.906+0000 7f67d4c00640 1 -- 172.21.15.62:0/1531860921 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f67c4089e30 con 0x7f67d80cee40 2024-04-25T09:18:17.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.907+0000 7f67dd000640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67b806b4d0 0x7f67b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:17.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.908+0000 7f67dd000640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67b806b4d0 0x7f67b806d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f67cc034000 tx=0x7f67cc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:17.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:17.911+0000 7f67d4c00640 1 -- 172.21.15.62:0/1531860921 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67c4058c60 con 0x7f67d80cee40 2024-04-25T09:18:18.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.114+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d8071650 con 0x7f67d80cee40 2024-04-25T09:18:18.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.116+0000 7f67d4c00640 1 -- 172.21.15.62:0/1531860921 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16765 (secure 0 0 0) 0x7f67c4058560 con 0x7f67d80cee40 2024-04-25T09:18:18.103 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:18.104 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":150,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:18:04.356543+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:17:28.263758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:17:29.401316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.10","mappings":[{"from":2,"to":5}]},{"pgid":"21.11","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.118+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67b806b4d0 msgr2=0x7f67b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.118+0000 7f67de400640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67b806b4d0 0x7f67b806d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f67cc034000 tx=0x7f67cc02e000 comp rx=0 tx=0).stop 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.118+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d80cee40 msgr2=0x7f67d815ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.118+0000 7f67de400640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d80cee40 0x7f67d815ad40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f67c400c450 tx=0x7f67c400c910 comp rx=0 tx=0).stop 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.119+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 shutdown_connections 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.119+0000 7f67de400640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f67b806b4d0 0x7f67b806d990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.119+0000 7f67de400640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d80cee40 0x7f67d815ad40 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.119+0000 7f67de400640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f67d8144c10 0x7f67d815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.119+0000 7f67de400640 1 --2- 172.21.15.62:0/1531860921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67d806d900 0x7f67d8143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.119+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 >> 172.21.15.62:0/1531860921 conn(0x7f67d80b14d0 msgr2=0x7f67d815be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.119+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 shutdown_connections 2024-04-25T09:18:18.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.119+0000 7f67de400640 1 -- 172.21.15.62:0/1531860921 wait complete. 2024-04-25T09:18:18.127 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:03:05.947157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:17:28.263758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:17:29.401316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:18:18.128 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:18:18.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.355+0000 7fc8c3600640 1 Processor -- start 2024-04-25T09:18:18.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.355+0000 7fc8c3600640 1 -- start start 2024-04-25T09:18:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.357+0000 7fc8c3600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 0x7fc8bc06c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.357+0000 7fc8c3600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8bc06d070 con 0x7fc8bc0720c0 2024-04-25T09:18:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.357+0000 7fc8c3600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8bc06d350 con 0x7fc8bc15bdf0 2024-04-25T09:18:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.357+0000 7fc8c3600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc072570 con 0x7fc8bc0cebc0 2024-04-25T09:18:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.357+0000 7fc8c1600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc8bc0720c0 0x7fc8bc0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45874/0 (socket says 172.21.15.62:45874) 2024-04-25T09:18:18.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.357+0000 7fc8c1600640 1 -- 172.21.15.62:0/946909782 learned_addr learned my addr 172.21.15.62:0/946909782 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bbe00640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 0x7fc8bc06c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218119317 0 0) 0x7fc8bc06d070 con 0x7fc8bc0720c0 2024-04-25T09:18:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8a4003520 con 0x7fc8bc0720c0 2024-04-25T09:18:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 646115302 0 0) 0x7fc8a4003520 con 0x7fc8bc0720c0 2024-04-25T09:18:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 msgr2=0x7fc8bc06c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:18.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 0x7fc8bc06c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 >> v1:172.21.15.121:6789/0 conn(0x7fc8bc15bdf0 legacy=0x7fc8bc06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc0aeb10 con 0x7fc8bc0720c0 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc8b40019f0 con 0x7fc8bc0720c0 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 >> v1:172.21.15.62:6789/0 conn(0x7fc8bc0720c0 legacy=0x7fc8bc0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8a40039e0 0x7fc8a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8a4004800 0x7fc8a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 0x7fc8a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a400ba70 con 0x7fc8a40039e0 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a400bc20 con 0x7fc8a4004800 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.358+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a400be20 con 0x7fc8bc0cebc0 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.359+0000 7fc8c0c00640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8a40039e0 0x7fc8a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.359+0000 7fc8c1600640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 0x7fc8a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.359+0000 7fc8c0c00640 1 -- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 msgr2=0x7fc8a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.359+0000 7fc8c0c00640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 0x7fc8a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.359+0000 7fc8c0c00640 1 -- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8a4004800 msgr2=0x7fc8a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.359+0000 7fc8c0c00640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8a4004800 0x7fc8a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.359+0000 7fc8c0c00640 1 -- 172.21.15.62:0/946909782 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a4003520 con 0x7fc8a40039e0 2024-04-25T09:18:18.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.359+0000 7fc8c1600640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8bc0cebc0 0x7fc8a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:18.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.360+0000 7fc8c0c00640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8a40039e0 0x7fc8a4003e00 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0001720 tx=0x7fc8a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e35782c17f4e999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:18.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0001370 con 0x7fc8a40039e0 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b00052d0 con 0x7fc8a40039e0 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8bb400640 1 -- 172.21.15.62:0/946909782 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b002ab90 con 0x7fc8a40039e0 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8c3600640 1 -- 172.21.15.62:0/946909782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8a40039e0 msgr2=0x7fc8a4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8c3600640 1 --2- 172.21.15.62:0/946909782 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8a40039e0 0x7fc8a4003e00 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0001720 tx=0x7fc8a4005830 comp rx=0 tx=0).stop 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8c1600640 1 -- 172.21.15.62:0/946909782 reap_dead start 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8c3600640 1 -- 172.21.15.62:0/946909782 shutdown_connections 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8c3600640 1 -- 172.21.15.62:0/946909782 >> 172.21.15.62:0/946909782 conn(0x7fc8bc0b1410 msgr2=0x7fc8bc0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8c3600640 1 -- 172.21.15.62:0/946909782 shutdown_connections 2024-04-25T09:18:18.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.361+0000 7fc8c3600640 1 -- 172.21.15.62:0/946909782 wait complete. 2024-04-25T09:18:18.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.363+0000 7fc8c3600640 1 Processor -- start 2024-04-25T09:18:18.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.363+0000 7fc8c3600640 1 -- start start 2024-04-25T09:18:18.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.364+0000 7fc8c3600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8a40039e0 0x7fc8bc152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.364+0000 7fc8c3600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8bc152510 0x7fc8bc070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.364+0000 7fc8c3600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8bc153140 0x7fc8bc071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.364+0000 7fc8c3600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0ae8c0 con 0x7fc8bc152510 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.364+0000 7fc8c3600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc15c1d0 con 0x7fc8bc153140 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.364+0000 7fc8c3600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc06c9b0 con 0x7fc8a40039e0 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8bc152510 0x7fc8bc070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8bc152510 0x7fc8bc070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40106/0 (socket says 172.21.15.62:40106) 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8bbe00640 1 -- 172.21.15.62:0/1349696525 learned_addr learned my addr 172.21.15.62:0/1349696525 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8c0c00640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8a40039e0 0x7fc8bc152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8c1600640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8bc153140 0x7fc8bc071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8bbe00640 1 -- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8a40039e0 msgr2=0x7fc8bc152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8bbe00640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8a40039e0 0x7fc8bc152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8bbe00640 1 -- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8bc153140 msgr2=0x7fc8bc071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8bbe00640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8bc153140 0x7fc8bc071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8bbe00640 1 -- 172.21.15.62:0/1349696525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc154a30 con 0x7fc8bc152510 2024-04-25T09:18:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.365+0000 7fc8c0c00640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8a40039e0 0x7fc8bc152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.366+0000 7fc8bbe00640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8bc152510 0x7fc8bc070f20 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800e7c0 tx=0x7fc8a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.366+0000 7fc8b8c00640 1 -- 172.21.15.62:0/1349696525 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8017070 con 0x7fc8bc152510 2024-04-25T09:18:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.366+0000 7fc8b8c00640 1 -- 172.21.15.62:0/1349696525 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a800c8c0 con 0x7fc8bc152510 2024-04-25T09:18:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.366+0000 7fc8b8c00640 1 -- 172.21.15.62:0/1349696525 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800cc90 con 0x7fc8bc152510 2024-04-25T09:18:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.366+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc154d10 con 0x7fc8bc152510 2024-04-25T09:18:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.366+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc1552d0 con 0x7fc8bc152510 2024-04-25T09:18:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.367+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc148330 con 0x7fc8bc152510 2024-04-25T09:18:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.368+0000 7fc8b8c00640 1 -- 172.21.15.62:0/1349696525 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8a801a070 con 0x7fc8bc152510 2024-04-25T09:18:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.369+0000 7fc8b8c00640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89c06b4d0 0x7fc89c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.369+0000 7fc8b8c00640 1 -- 172.21.15.62:0/1349696525 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc8a808a3d0 con 0x7fc8bc152510 2024-04-25T09:18:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.370+0000 7fc8c0c00640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89c06b4d0 0x7fc89c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.371+0000 7fc8c0c00640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89c06b4d0 0x7fc89c06d990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0032000 tx=0x7fc8b002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.373+0000 7fc8b8c00640 1 -- 172.21.15.62:0/1349696525 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8bc148330 con 0x7fc8bc152510 2024-04-25T09:18:18.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.573+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8bc04ef20 con 0x7fc8bc152510 2024-04-25T09:18:18.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.573+0000 7fc8b8c00640 1 -- 172.21.15.62:0/1349696525 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8a8059200 con 0x7fc8bc152510 2024-04-25T09:18:18.561 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-25T09:18:18.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.576+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89c06b4d0 msgr2=0x7fc89c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.576+0000 7fc8c3600640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89c06b4d0 0x7fc89c06d990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0032000 tx=0x7fc8b002e000 comp rx=0 tx=0).stop 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.576+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8bc152510 msgr2=0x7fc8bc070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.576+0000 7fc8c3600640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8bc152510 0x7fc8bc070f20 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800e7c0 tx=0x7fc8a800ec80 comp rx=0 tx=0).stop 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.577+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 shutdown_connections 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.577+0000 7fc8c3600640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc89c06b4d0 0x7fc89c06d990 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.577+0000 7fc8c3600640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8bc152510 0x7fc8bc070f20 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.577+0000 7fc8c3600640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8bc153140 0x7fc8bc071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.577+0000 7fc8c3600640 1 --2- 172.21.15.62:0/1349696525 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8a40039e0 0x7fc8bc152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.577+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 >> 172.21.15.62:0/1349696525 conn(0x7fc8bc0b1410 msgr2=0x7fc8bc0703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.577+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 shutdown_connections 2024-04-25T09:18:18.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.577+0000 7fc8c3600640 1 -- 172.21.15.62:0/1349696525 wait complete. 2024-04-25T09:18:18.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:18:18.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.811+0000 7fa150c00640 1 Processor -- start 2024-04-25T09:18:18.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.811+0000 7fa150c00640 1 -- start start 2024-04-25T09:18:18.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.812+0000 7fa150c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa14c0d2fb0 0x7fa14c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.812+0000 7fa150c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa14c0d3960 0x7fa14c0d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.812+0000 7fa150c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14c0ce780 con 0x7fa14c15dd50 2024-04-25T09:18:18.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.812+0000 7fa150c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c0d42f0 con 0x7fa14c0d3960 2024-04-25T09:18:18.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.812+0000 7fa150c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c0d3440 con 0x7fa14c0d2fb0 2024-04-25T09:18:18.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14b400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa14c15dd50 0x7fa14c0d2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45890/0 (socket says 172.21.15.62:45890) 2024-04-25T09:18:18.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14b400640 1 -- 172.21.15.62:0/3238144290 learned_addr learned my addr 172.21.15.62:0/3238144290 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:18.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14be00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa14c0d3960 0x7fa14c0d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14aa00640 1 --2- 172.21.15.62:0/3238144290 >> v2:172.21.15.121:3300/0 conn(0x7fa14c0d2fb0 0x7fa14c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14be00640 1 -- 172.21.15.62:0/3238144290 >> v2:172.21.15.121:3300/0 conn(0x7fa14c0d2fb0 msgr2=0x7fa14c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:18.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14be00640 1 --2- 172.21.15.62:0/3238144290 >> v2:172.21.15.121:3300/0 conn(0x7fa14c0d2fb0 0x7fa14c0ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14be00640 1 -- 172.21.15.62:0/3238144290 >> v1:172.21.15.62:6789/0 conn(0x7fa14c15dd50 legacy=0x7fa14c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14be00640 1 -- 172.21.15.62:0/3238144290 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0ce1c0 con 0x7fa14c0d3960 2024-04-25T09:18:18.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.813+0000 7fa14a000640 1 -- 172.21.15.62:0/3238144290 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577051870 0 0) 0x7fa14c0ce780 con 0x7fa14c15dd50 2024-04-25T09:18:18.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.815+0000 7fa14be00640 1 --2- 172.21.15.62:0/3238144290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c0d3db0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0d3880 tx=0x7fa14402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7cd6bd29f97883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:18.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.815+0000 7fa14a000640 1 -- 172.21.15.62:0/3238144290 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144037070 con 0x7fa14c0d3960 2024-04-25T09:18:18.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.815+0000 7fa14a000640 1 -- 172.21.15.62:0/3238144290 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144032440 con 0x7fa14c0d3960 2024-04-25T09:18:18.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.815+0000 7fa14a000640 1 -- 172.21.15.62:0/3238144290 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144032810 con 0x7fa14c0d3960 2024-04-25T09:18:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.816+0000 7fa150c00640 1 -- 172.21.15.62:0/3238144290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 msgr2=0x7fa14c0d3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.816+0000 7fa150c00640 1 --2- 172.21.15.62:0/3238144290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c0d3db0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0d3880 tx=0x7fa14402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.816+0000 7fa150c00640 1 -- 172.21.15.62:0/3238144290 shutdown_connections 2024-04-25T09:18:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.816+0000 7fa150c00640 1 --2- 172.21.15.62:0/3238144290 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c0d3db0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.816+0000 7fa150c00640 1 --2- 172.21.15.62:0/3238144290 >> v2:172.21.15.121:3300/0 conn(0x7fa14c0d2fb0 0x7fa14c0ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.816+0000 7fa150c00640 1 -- 172.21.15.62:0/3238144290 >> 172.21.15.62:0/3238144290 conn(0x7fa14c06b000 msgr2=0x7fa14c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.816+0000 7fa150c00640 1 -- 172.21.15.62:0/3238144290 shutdown_connections 2024-04-25T09:18:18.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.816+0000 7fa150c00640 1 -- 172.21.15.62:0/3238144290 wait complete. 2024-04-25T09:18:18.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.818+0000 7fa150c00640 1 Processor -- start 2024-04-25T09:18:18.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.818+0000 7fa150c00640 1 -- start start 2024-04-25T09:18:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa150c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa150c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c15dd50 0x7fa14c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa150c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa14c0c9d30 0x7fa14c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa150c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0b6b00 con 0x7fa14c0d3960 2024-04-25T09:18:18.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa150c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0cef00 con 0x7fa14c15dd50 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa150c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0b6c80 con 0x7fa14c0c9d30 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa14b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa14b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c1543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40128/0 (socket says 172.21.15.62:40128) 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.819+0000 7fa14b400640 1 -- 172.21.15.62:0/1056976953 learned_addr learned my addr 172.21.15.62:0/1056976953 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa14aa00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c15dd50 0x7fa14c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa14b400640 1 -- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa14c0c9d30 msgr2=0x7fa14c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa14b400640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa14c0c9d30 0x7fa14c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa14b400640 1 -- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c15dd50 msgr2=0x7fa14c0c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa14b400640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c15dd50 0x7fa14c0c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa14b400640 1 -- 172.21.15.62:0/1056976953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0b53a0 con 0x7fa14c0d3960 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa14aa00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c15dd50 0x7fa14c0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa14b400640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c1543c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c7c0 tx=0x7fa13c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa143400640 1 -- 172.21.15.62:0/1056976953 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c018070 con 0x7fa14c0d3960 2024-04-25T09:18:18.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa143400640 1 -- 172.21.15.62:0/1056976953 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c00a640 con 0x7fa14c0d3960 2024-04-25T09:18:18.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c0b5680 con 0x7fa14c0d3960 2024-04-25T09:18:18.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.820+0000 7fa143400640 1 -- 172.21.15.62:0/1056976953 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00aa30 con 0x7fa14c0d3960 2024-04-25T09:18:18.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.821+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c158a70 con 0x7fa14c0d3960 2024-04-25T09:18:18.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.822+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c04ef20 con 0x7fa14c0d3960 2024-04-25T09:18:18.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.823+0000 7fa143400640 1 -- 172.21.15.62:0/1056976953 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa13c01b070 con 0x7fa14c0d3960 2024-04-25T09:18:18.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.824+0000 7fa143400640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa12806b500 0x7fa12806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.824+0000 7fa143400640 1 -- 172.21.15.62:0/1056976953 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa13c089810 con 0x7fa14c0d3960 2024-04-25T09:18:18.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.824+0000 7fa14aa00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa12806b500 0x7fa12806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:18.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.826+0000 7fa14aa00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa12806b500 0x7fa12806d9c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa14c154900 tx=0x7fa134008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:18.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:18.828+0000 7fa143400640 1 -- 172.21.15.62:0/1056976953 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa13c058640 con 0x7fa14c0d3960 2024-04-25T09:18:19.020 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.033+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa14c0b5850 con 0x7fa14c0d3960 2024-04-25T09:18:19.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.034+0000 7fa143400640 1 -- 172.21.15.62:0/1056976953 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa13c057f40 con 0x7fa14c0d3960 2024-04-25T09:18:19.021 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-25T09:18:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa12806b500 msgr2=0x7fa12806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa12806b500 0x7fa12806d9c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa14c154900 tx=0x7fa134008040 comp rx=0 tx=0).stop 2024-04-25T09:18:19.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 msgr2=0x7fa14c1543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c1543c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c7c0 tx=0x7fa13c00cc80 comp rx=0 tx=0).stop 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 shutdown_connections 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa12806b500 0x7fa12806d9c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa14c0c9d30 0x7fa14c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c15dd50 0x7fa14c0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 --2- 172.21.15.62:0/1056976953 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa14c0d3960 0x7fa14c1543c0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 >> 172.21.15.62:0/1056976953 conn(0x7fa14c06b000 msgr2=0x7fa14c0b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.037+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 shutdown_connections 2024-04-25T09:18:19.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.038+0000 7fa150c00640 1 -- 172.21.15.62:0/1056976953 wait complete. 2024-04-25T09:18:19.044 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:18:19.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.266+0000 7f1143800640 1 Processor -- start 2024-04-25T09:18:19.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.267+0000 7f1143800640 1 -- start start 2024-04-25T09:18:19.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.268+0000 7f1143800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f113c157c30 0x7f113c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.268+0000 7f1143800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f113c15b190 0x7f113c0b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.269+0000 7f1143800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113c06cc60 con 0x7f113c0704b0 2024-04-25T09:18:19.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.269+0000 7f1143800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113c15b620 con 0x7f113c157c30 2024-04-25T09:18:19.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.269+0000 7f1143800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c06cf00 con 0x7f113c15b190 2024-04-25T09:18:19.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.269+0000 7f1141800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f113c0704b0 0x7f113c06d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45904/0 (socket says 172.21.15.62:45904) 2024-04-25T09:18:19.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.269+0000 7f1141800640 1 -- 172.21.15.62:0/1088014748 learned_addr learned my addr 172.21.15.62:0/1088014748 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.269+0000 7f113be00640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f113c15b190 0x7f113c0b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.269+0000 7f1140e00640 1 --2- 172.21.15.62:0/1088014748 >> v2:172.21.15.121:3300/0 conn(0x7f113c157c30 0x7f113c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121340473 0 0) 0x7f113c06cc60 con 0x7f113c0704b0 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1120003520 con 0x7f113c0704b0 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 782545989 0 0) 0x7f1120003520 con 0x7f113c0704b0 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f113c15b190 msgr2=0x7f113c0b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f113c15b190 0x7f113c0b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 >> v2:172.21.15.121:3300/0 conn(0x7f113c157c30 msgr2=0x7f113c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:19.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 --2- 172.21.15.62:0/1088014748 >> v2:172.21.15.121:3300/0 conn(0x7f113c157c30 0x7f113c0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c06d1e0 con 0x7f113c0704b0 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f1134001670 con 0x7f113c0704b0 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 >> v1:172.21.15.62:6789/0 conn(0x7f113c0704b0 legacy=0x7f113c06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11200039b0 0x7f1120003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f113c15b190 0x7f1120004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1120004d40 0x7f11200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c15b620 con 0x7f11200039b0 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c06cf00 con 0x7f1120004d40 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112000b9c0 con 0x7f113c15b190 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.270+0000 7f1140e00640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11200039b0 0x7f1120003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.271+0000 7f113be00640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f113c15b190 0x7f1120004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.271+0000 7f1140e00640 1 -- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f113c15b190 msgr2=0x7f1120004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.271+0000 7f1140e00640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f113c15b190 0x7f1120004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.271+0000 7f1140e00640 1 -- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1120004d40 msgr2=0x7f11200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:19.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.271+0000 7f1141800640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1120004d40 0x7f11200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.271+0000 7f1140e00640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1120004d40 0x7f11200051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.271+0000 7f1140e00640 1 -- 172.21.15.62:0/1088014748 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120003520 con 0x7f11200039b0 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.272+0000 7f1140e00640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11200039b0 0x7f1120003dd0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f112c009fc0 tx=0x7f1120005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40c7e2ed030fa67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.272+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c009c10 con 0x7f11200039b0 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.272+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c036070 con 0x7f11200039b0 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.272+0000 7f113b400640 1 -- 172.21.15.62:0/1088014748 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c02d070 con 0x7f11200039b0 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.273+0000 7f1143800640 1 -- 172.21.15.62:0/1088014748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11200039b0 msgr2=0x7f1120003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.273+0000 7f1143800640 1 --2- 172.21.15.62:0/1088014748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f11200039b0 0x7f1120003dd0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f112c009fc0 tx=0x7f1120005830 comp rx=0 tx=0).stop 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.273+0000 7f1141800640 1 -- 172.21.15.62:0/1088014748 reap_dead start 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.273+0000 7f1143800640 1 -- 172.21.15.62:0/1088014748 shutdown_connections 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.273+0000 7f1143800640 1 -- 172.21.15.62:0/1088014748 >> 172.21.15.62:0/1088014748 conn(0x7f113c0b1410 msgr2=0x7f113c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.273+0000 7f1143800640 1 -- 172.21.15.62:0/1088014748 shutdown_connections 2024-04-25T09:18:19.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.273+0000 7f1143800640 1 -- 172.21.15.62:0/1088014748 wait complete. 2024-04-25T09:18:19.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.275+0000 7f1143800640 1 Processor -- start 2024-04-25T09:18:19.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.275+0000 7f1143800640 1 -- start start 2024-04-25T09:18:19.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.276+0000 7f1143800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11200039b0 0x7f113c1f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.276+0000 7f1143800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f113c0c4880 0x7f113c149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.276+0000 7f1143800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f113c0ad8e0 0x7f113c0add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.276+0000 7f1143800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c158010 con 0x7f113c0ad8e0 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.276+0000 7f1143800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0cbf90 con 0x7f113c0c4880 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.276+0000 7f1143800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c06e0f0 con 0x7f11200039b0 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f113c0ad8e0 0x7f113c0add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f113c0ad8e0 0x7f113c0add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40144/0 (socket says 172.21.15.62:40144) 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 -- 172.21.15.62:0/771062698 learned_addr learned my addr 172.21.15.62:0/771062698 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1140e00640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11200039b0 0x7f113c1f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f113be00640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f113c0c4880 0x7f113c149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 -- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11200039b0 msgr2=0x7f113c1f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11200039b0 0x7f113c1f38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 -- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f113c0c4880 msgr2=0x7f113c149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f113c0c4880 0x7f113c149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 -- 172.21.15.62:0/771062698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c15d030 con 0x7f113c0ad8e0 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f113be00640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f113c0c4880 0x7f113c149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.277+0000 7f1141800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f113c0ad8e0 0x7f113c0add60 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f113c15b570 tx=0x7f113400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.278+0000 7f1138c00640 1 -- 172.21.15.62:0/771062698 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134007440 con 0x7f113c0ad8e0 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.278+0000 7f1138c00640 1 -- 172.21.15.62:0/771062698 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134007620 con 0x7f113c0ad8e0 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.278+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113c1f3ef0 con 0x7f113c0ad8e0 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.278+0000 7f1138c00640 1 -- 172.21.15.62:0/771062698 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134004a20 con 0x7f113c0ad8e0 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.278+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113c1f44b0 con 0x7f113c0ad8e0 2024-04-25T09:18:19.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.279+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113c04ef20 con 0x7f113c0ad8e0 2024-04-25T09:18:19.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.280+0000 7f1138c00640 1 -- 172.21.15.62:0/771062698 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f113401b060 con 0x7f113c0ad8e0 2024-04-25T09:18:19.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.281+0000 7f1138c00640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.281+0000 7f1138c00640 1 -- 172.21.15.62:0/771062698 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1134089a90 con 0x7f113c0ad8e0 2024-04-25T09:18:19.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.282+0000 7f1140e00640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.283+0000 7f1140e00640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f111c06b4d0 0x7f111c06d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f112c035000 tx=0x7f112c02f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:19.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.286+0000 7f1138c00640 1 -- 172.21.15.62:0/771062698 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11340588d0 con 0x7f113c0ad8e0 2024-04-25T09:18:19.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.488+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f113c159910 con 0x7f113c0ad8e0 2024-04-25T09:18:19.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.489+0000 7f1138c00640 1 -- 172.21.15.62:0/771062698 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11340581d0 con 0x7f113c0ad8e0 2024-04-25T09:18:19.476 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f111c06b4d0 msgr2=0x7f111c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f111c06b4d0 0x7f111c06d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f112c035000 tx=0x7f112c02f000 comp rx=0 tx=0).stop 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f113c0ad8e0 msgr2=0x7f113c0add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f113c0ad8e0 0x7f113c0add60 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f113c15b570 tx=0x7f113400d9d0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 shutdown_connections 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f113c0ad8e0 0x7f113c0add60 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f113c0c4880 0x7f113c149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 --2- 172.21.15.62:0/771062698 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f11200039b0 0x7f113c1f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 >> 172.21.15.62:0/771062698 conn(0x7f113c0b1410 msgr2=0x7f113c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:19.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 shutdown_connections 2024-04-25T09:18:19.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.492+0000 7f1143800640 1 -- 172.21.15.62:0/771062698 wait complete. 2024-04-25T09:18:19.499 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:18:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.722+0000 7f073a800640 1 Processor -- start 2024-04-25T09:18:19.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.722+0000 7f073a800640 1 -- start start 2024-04-25T09:18:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.724+0000 7f073a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0734144c10 0x7f073415f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.724+0000 7f073a800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07340cf2d0 con 0x7f073406d900 2024-04-25T09:18:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.724+0000 7f073a800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073415faa0 con 0x7f07340cee40 2024-04-25T09:18:19.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.724+0000 7f073a800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0734144ff0 con 0x7f0734144c10 2024-04-25T09:18:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.724+0000 7f0739400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0734144c10 0x7f073415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.724+0000 7f0739400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0734144c10 0x7f073415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40160/0 (socket says 172.21.15.62:40160) 2024-04-25T09:18:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.724+0000 7f0739400640 1 -- 172.21.15.62:0/1068277249 learned_addr learned my addr 172.21.15.62:0/1068277249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.724+0000 7f0739e00640 1 --1- 172.21.15.62:0/1068277249 >> v1:172.21.15.62:6789/0 conn(0x7f073406d900 0x7f07340b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45912/0 (socket says 172.21.15.62:45912) 2024-04-25T09:18:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.725+0000 7f0739400640 1 -- 172.21.15.62:0/1068277249 >> v1:172.21.15.121:6789/0 conn(0x7f07340cee40 legacy=0x7f073406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.725+0000 7f0739400640 1 -- 172.21.15.62:0/1068277249 >> v1:172.21.15.62:6789/0 conn(0x7f073406d900 legacy=0x7f07340b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.725+0000 7f0739400640 1 -- 172.21.15.62:0/1068277249 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073415fd80 con 0x7f0734144c10 2024-04-25T09:18:19.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.725+0000 7f0733e00640 1 -- 172.21.15.62:0/1068277249 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556481215 0 0) 0x7f07340cf2d0 con 0x7f073406d900 2024-04-25T09:18:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.726+0000 7f0739400640 1 --2- 172.21.15.62:0/1068277249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f073415f450 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f0728009870 tx=0x7f072802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283adfbdc18351d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.726+0000 7f0733e00640 1 -- 172.21.15.62:0/1068277249 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728037070 con 0x7f0734144c10 2024-04-25T09:18:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.726+0000 7f0733e00640 1 -- 172.21.15.62:0/1068277249 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728032440 con 0x7f0734144c10 2024-04-25T09:18:19.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.726+0000 7f0733e00640 1 -- 172.21.15.62:0/1068277249 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728032810 con 0x7f0734144c10 2024-04-25T09:18:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.727+0000 7f073a800640 1 -- 172.21.15.62:0/1068277249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 msgr2=0x7f073415f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.727+0000 7f073a800640 1 --2- 172.21.15.62:0/1068277249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f073415f450 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f0728009870 tx=0x7f072802a210 comp rx=0 tx=0).stop 2024-04-25T09:18:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.727+0000 7f073a800640 1 -- 172.21.15.62:0/1068277249 shutdown_connections 2024-04-25T09:18:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.727+0000 7f073a800640 1 --2- 172.21.15.62:0/1068277249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f073415f450 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.727+0000 7f073a800640 1 -- 172.21.15.62:0/1068277249 >> 172.21.15.62:0/1068277249 conn(0x7f07340b14d0 msgr2=0x7f07340b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.727+0000 7f073a800640 1 -- 172.21.15.62:0/1068277249 shutdown_connections 2024-04-25T09:18:19.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.727+0000 7f073a800640 1 -- 172.21.15.62:0/1068277249 wait complete. 2024-04-25T09:18:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.729+0000 7f073a800640 1 Processor -- start 2024-04-25T09:18:19.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.729+0000 7f073a800640 1 -- start start 2024-04-25T09:18:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f073a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f073406d900 0x7f07341f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f073a800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07340cee40 0x7f0734141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f073a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f07341f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f073a800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340b41c0 con 0x7f0734144c10 2024-04-25T09:18:19.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f073a800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0734160340 con 0x7f073406d900 2024-04-25T09:18:19.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f073a800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340b4d60 con 0x7f07340cee40 2024-04-25T09:18:19.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f0739e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f07341f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f0739e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f07341f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40170/0 (socket says 172.21.15.62:40170) 2024-04-25T09:18:19.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f0739e00640 1 -- 172.21.15.62:0/3559352736 learned_addr learned my addr 172.21.15.62:0/3559352736 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:19.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.730+0000 7f0739400640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f073406d900 0x7f07341f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0739e00640 1 -- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07340cee40 msgr2=0x7f0734141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0739e00640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07340cee40 0x7f0734141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0739e00640 1 -- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f073406d900 msgr2=0x7f07341f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0739e00640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f073406d900 0x7f07341f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0739e00640 1 -- 172.21.15.62:0/3559352736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734141da0 con 0x7f0734144c10 2024-04-25T09:18:19.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0739e00640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f07341f7a90 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c760 tx=0x7f072c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:19.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0731600640 1 -- 172.21.15.62:0/3559352736 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c018070 con 0x7f0734144c10 2024-04-25T09:18:19.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0731600640 1 -- 172.21.15.62:0/3559352736 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c00a640 con 0x7f0734144c10 2024-04-25T09:18:19.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f0731600640 1 -- 172.21.15.62:0/3559352736 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00aa30 con 0x7f0734144c10 2024-04-25T09:18:19.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.731+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0734142080 con 0x7f0734144c10 2024-04-25T09:18:19.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.732+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0734142640 con 0x7f0734144c10 2024-04-25T09:18:19.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.733+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073404ef20 con 0x7f0734144c10 2024-04-25T09:18:19.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.734+0000 7f0731600640 1 -- 172.21.15.62:0/3559352736 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f072c00abd0 con 0x7f0734144c10 2024-04-25T09:18:19.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.735+0000 7f0731600640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b500 0x7f071406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:19.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.735+0000 7f0731600640 1 -- 172.21.15.62:0/3559352736 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f072c089750 con 0x7f0734144c10 2024-04-25T09:18:19.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.735+0000 7f0739400640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b500 0x7f071406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:19.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.737+0000 7f0739400640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b500 0x7f071406d9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0728034000 tx=0x7f072802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:19.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.740+0000 7f0731600640 1 -- 172.21.15.62:0/3559352736 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f072c058590 con 0x7f0734144c10 2024-04-25T09:18:19.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.993+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07340b22b0 con 0x7f0734144c10 2024-04-25T09:18:19.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.994+0000 7f0731600640 1 -- 172.21.15.62:0/3559352736 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4948 (secure 0 0 0) 0x7f072c057e90 con 0x7f0734144c10 2024-04-25T09:18:19.982 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:19.982 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":102,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3369583759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3369583759},{"type":"v1","addr":"172.21.15.121:6837","nonce":3369583759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1888749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1888749},{"type":"v1","addr":"172.21.15.121:6836","nonce":1888749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:17:30.382991+0000","modified":"2024-04-25T09:18:11.404400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9390":{"gid":9390,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6833/1673530113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1673530113},{"type":"v1","addr":"172.21.15.62:6833","nonce":1673530113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9396":{"gid":9396,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2793862963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2793862963},{"type":"v1","addr":"172.21.15.62:6835","nonce":2793862963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:18:19.982 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 102 2024-04-25T09:18:19.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b500 msgr2=0x7f071406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b500 0x7f071406d9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0728034000 tx=0x7f072802e000 comp rx=0 tx=0).stop 2024-04-25T09:18:19.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 msgr2=0x7f07341f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:19.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f07341f7a90 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f072c00c760 tx=0x7f072c00cc20 comp rx=0 tx=0).stop 2024-04-25T09:18:19.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 shutdown_connections 2024-04-25T09:18:19.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f071406b500 0x7f071406d9c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0734144c10 0x7f07341f7a90 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f07340cee40 0x7f0734141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 --2- 172.21.15.62:0/3559352736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f073406d900 0x7f07341f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:19.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 >> 172.21.15.62:0/3559352736 conn(0x7f07340b14d0 msgr2=0x7f073406c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:19.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 shutdown_connections 2024-04-25T09:18:19.986 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:19.997+0000 7f073a800640 1 -- 172.21.15.62:0/3559352736 wait complete. 2024-04-25T09:18:20.003 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:18:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.220+0000 7fcad7e00640 1 Processor -- start 2024-04-25T09:18:20.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.221+0000 7fcad7e00640 1 -- start start 2024-04-25T09:18:20.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.222+0000 7fcad7e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcad806d900 0x7fcad815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.222+0000 7fcad7e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad815faa0 con 0x7fcad8144c10 2024-04-25T09:18:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.222+0000 7fcad7e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad815fd80 con 0x7fcad80cee40 2024-04-25T09:18:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.222+0000 7fcad7e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad8144ff0 con 0x7fcad806d900 2024-04-25T09:18:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.223+0000 7fcace000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcad806d900 0x7fcad815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.223+0000 7fcad6a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fcad8144c10 0x7fcad806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45922/0 (socket says 172.21.15.62:45922) 2024-04-25T09:18:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.223+0000 7fcad6a00640 1 -- 172.21.15.62:0/1854000838 learned_addr learned my addr 172.21.15.62:0/1854000838 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.223+0000 7fcace000640 1 -- 172.21.15.62:0/1854000838 >> v1:172.21.15.121:6789/0 conn(0x7fcad80cee40 legacy=0x7fcad80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:20.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.223+0000 7fcace000640 1 -- 172.21.15.62:0/1854000838 >> v1:172.21.15.62:6789/0 conn(0x7fcad8144c10 legacy=0x7fcad806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.223+0000 7fcace000640 1 -- 172.21.15.62:0/1854000838 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad8160060 con 0x7fcad806d900 2024-04-25T09:18:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.223+0000 7fcad6000640 1 -- 172.21.15.62:0/1854000838 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839949445 0 0) 0x7fcad815faa0 con 0x7fcad8144c10 2024-04-25T09:18:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.224+0000 7fcace000640 1 --2- 172.21.15.62:0/1854000838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad806d900 0x7fcad815f450 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097a0 tx=0x7fcac002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c148bdba1846d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad6000640 1 -- 172.21.15.62:0/1854000838 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0037070 con 0x7fcad806d900 2024-04-25T09:18:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad6000640 1 -- 172.21.15.62:0/1854000838 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac002ae50 con 0x7fcad806d900 2024-04-25T09:18:20.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad6000640 1 -- 172.21.15.62:0/1854000838 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0032660 con 0x7fcad806d900 2024-04-25T09:18:20.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad7e00640 1 -- 172.21.15.62:0/1854000838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad806d900 msgr2=0x7fcad815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad7e00640 1 --2- 172.21.15.62:0/1854000838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad806d900 0x7fcad815f450 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fcac00097a0 tx=0x7fcac002a140 comp rx=0 tx=0).stop 2024-04-25T09:18:20.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad7e00640 1 -- 172.21.15.62:0/1854000838 shutdown_connections 2024-04-25T09:18:20.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad7e00640 1 --2- 172.21.15.62:0/1854000838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad806d900 0x7fcad815f450 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad7e00640 1 -- 172.21.15.62:0/1854000838 >> 172.21.15.62:0/1854000838 conn(0x7fcad80b14d0 msgr2=0x7fcad80b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:20.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.225+0000 7fcad7e00640 1 -- 172.21.15.62:0/1854000838 shutdown_connections 2024-04-25T09:18:20.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.226+0000 7fcad7e00640 1 -- 172.21.15.62:0/1854000838 wait complete. 2024-04-25T09:18:20.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.227+0000 7fcad7e00640 1 Processor -- start 2024-04-25T09:18:20.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.227+0000 7fcad7e00640 1 -- start start 2024-04-25T09:18:20.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.228+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad806d900 0x7fcad813a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.228+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad80cee40 0x7fcad8139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.228+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad8144c10 0x7fcad815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.228+0000 7fcad7e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80b41c0 con 0x7fcad8144c10 2024-04-25T09:18:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.228+0000 7fcad7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad8160340 con 0x7fcad80cee40 2024-04-25T09:18:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.228+0000 7fcad7e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80b4d60 con 0x7fcad806d900 2024-04-25T09:18:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad8144c10 0x7fcad815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad8144c10 0x7fcad815e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40202/0 (socket says 172.21.15.62:40202) 2024-04-25T09:18:20.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad7400640 1 -- 172.21.15.62:0/3441939482 learned_addr learned my addr 172.21.15.62:0/3441939482 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:20.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad7400640 1 -- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad806d900 msgr2=0x7fcad813a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad6a00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad806d900 0x7fcad813a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad7400640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad806d900 0x7fcad813a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcace000640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad80cee40 0x7fcad8139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad7400640 1 -- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad80cee40 msgr2=0x7fcad8139900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad7400640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad80cee40 0x7fcad8139900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad7400640 1 -- 172.21.15.62:0/3441939482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad813e080 con 0x7fcad8144c10 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.229+0000 7fcad6a00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad806d900 0x7fcad813a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.230+0000 7fcad7400640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad8144c10 0x7fcad815e2d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c780 tx=0x7fcad000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.230+0000 7fcacf400640 1 -- 172.21.15.62:0/3441939482 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0004b60 con 0x7fcad8144c10 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.230+0000 7fcacf400640 1 -- 172.21.15.62:0/3441939482 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad0004d00 con 0x7fcad8144c10 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.230+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad813a740 con 0x7fcad8144c10 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.230+0000 7fcacf400640 1 -- 172.21.15.62:0/3441939482 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad000aa90 con 0x7fcad8144c10 2024-04-25T09:18:20.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.230+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad813e620 con 0x7fcad8144c10 2024-04-25T09:18:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.231+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad804ef20 con 0x7fcad8144c10 2024-04-25T09:18:20.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.232+0000 7fcacf400640 1 -- 172.21.15.62:0/3441939482 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcad000ac30 con 0x7fcad8144c10 2024-04-25T09:18:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.233+0000 7fcacf400640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcab406b4d0 0x7fcab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.233+0000 7fcacf400640 1 -- 172.21.15.62:0/3441939482 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcad008a4a0 con 0x7fcad8144c10 2024-04-25T09:18:20.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.234+0000 7fcad6a00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcab406b4d0 0x7fcab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.235+0000 7fcad6a00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcab406b4d0 0x7fcab406d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000f60 tx=0x7fcac8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:20.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.238+0000 7fcacf400640 1 -- 172.21.15.62:0/3441939482 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcad0020070 con 0x7fcad8144c10 2024-04-25T09:18:20.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.441+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcad80b22b0 con 0x7fcad8144c10 2024-04-25T09:18:20.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.443+0000 7fcacf400640 1 -- 172.21.15.62:0/3441939482 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16765 (secure 0 0 0) 0x7fcad00592e0 con 0x7fcad8144c10 2024-04-25T09:18:20.430 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:20.430 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":150,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:18:04.356543+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:17:28.263758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:17:29.401316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.10","mappings":[{"from":2,"to":5}]},{"pgid":"21.11","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcab406b4d0 msgr2=0x7fcab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcab406b4d0 0x7fcab406d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000f60 tx=0x7fcac8009040 comp rx=0 tx=0).stop 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad8144c10 msgr2=0x7fcad815e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad8144c10 0x7fcad815e2d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c780 tx=0x7fcad000cc40 comp rx=0 tx=0).stop 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 shutdown_connections 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fcab406b4d0 0x7fcab406d990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcad8144c10 0x7fcad815e2d0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcad80cee40 0x7fcad8139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 --2- 172.21.15.62:0/3441939482 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fcad806d900 0x7fcad813a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 >> 172.21.15.62:0/3441939482 conn(0x7fcad80b14d0 msgr2=0x7fcad806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:20.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 shutdown_connections 2024-04-25T09:18:20.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.446+0000 7fcad7e00640 1 -- 172.21.15.62:0/3441939482 wait complete. 2024-04-25T09:18:20.454 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:18:20.454 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:18:20.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.683+0000 7feb01800640 1 Processor -- start 2024-04-25T09:18:20.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.684+0000 7feb01800640 1 -- start start 2024-04-25T09:18:20.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.685+0000 7feb01800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 0x7feafc160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.685+0000 7feb01800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feafc0b6a90 0x7feafc161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.685+0000 7feb01800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feafc0b27a0 0x7feafc161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.685+0000 7feb01800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feafc161780 con 0x7feafc0b27a0 2024-04-25T09:18:20.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.685+0000 7feb01800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feafc0cedc0 con 0x7feafc0b6a90 2024-04-25T09:18:20.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.685+0000 7feb01800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0d3110 con 0x7feafc0b56f0 2024-04-25T09:18:20.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.685+0000 7feb00e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feafc0b27a0 0x7feafc161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feb00e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feafc0b27a0 0x7feafc161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40214/0 (socket says 172.21.15.62:40214) 2024-04-25T09:18:20.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feb00e00640 1 -- 172.21.15.62:0/3957704380 learned_addr learned my addr 172.21.15.62:0/3957704380 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:20.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feafbe00640 1 --2- 172.21.15.62:0/3957704380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 0x7feafc160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feafb400640 1 --2- 172.21.15.62:0/3957704380 >> v2:172.21.15.121:3300/0 conn(0x7feafc0b6a90 0x7feafc161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feb00e00640 1 -- 172.21.15.62:0/3957704380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 msgr2=0x7feafc160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:20.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feb00e00640 1 --2- 172.21.15.62:0/3957704380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 0x7feafc160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feb00e00640 1 -- 172.21.15.62:0/3957704380 >> v2:172.21.15.121:3300/0 conn(0x7feafc0b6a90 msgr2=0x7feafc161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:20.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feb00e00640 1 --2- 172.21.15.62:0/3957704380 >> v2:172.21.15.121:3300/0 conn(0x7feafc0b6a90 0x7feafc161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feb00e00640 1 -- 172.21.15.62:0/3957704380 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0d33f0 con 0x7feafc0b27a0 2024-04-25T09:18:20.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.686+0000 7feafb400640 1 --2- 172.21.15.62:0/3957704380 >> v2:172.21.15.121:3300/0 conn(0x7feafc0b6a90 0x7feafc161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:20.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.687+0000 7feb00e00640 1 --2- 172.21.15.62:0/3957704380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b27a0 0x7feafc161980 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7feafc0728e0 tx=0x7feaf402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c78a1e0f5ca95c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:20.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feafaa00640 1 -- 172.21.15.62:0/3957704380 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4036070 con 0x7feafc0b27a0 2024-04-25T09:18:20.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feafaa00640 1 -- 172.21.15.62:0/3957704380 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4031600 con 0x7feafc0b27a0 2024-04-25T09:18:20.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feafaa00640 1 -- 172.21.15.62:0/3957704380 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf40319d0 con 0x7feafc0b27a0 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feb01800640 1 -- 172.21.15.62:0/3957704380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b27a0 msgr2=0x7feafc161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feb01800640 1 --2- 172.21.15.62:0/3957704380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b27a0 0x7feafc161980 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7feafc0728e0 tx=0x7feaf402a6a0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feb01800640 1 -- 172.21.15.62:0/3957704380 shutdown_connections 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feb01800640 1 --2- 172.21.15.62:0/3957704380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b27a0 0x7feafc161980 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feb01800640 1 --2- 172.21.15.62:0/3957704380 >> v2:172.21.15.121:3300/0 conn(0x7feafc0b6a90 0x7feafc161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.688+0000 7feb01800640 1 --2- 172.21.15.62:0/3957704380 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 0x7feafc160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.689+0000 7feb01800640 1 -- 172.21.15.62:0/3957704380 >> 172.21.15.62:0/3957704380 conn(0x7feafc066d10 msgr2=0x7feafc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.689+0000 7feb01800640 1 -- 172.21.15.62:0/3957704380 shutdown_connections 2024-04-25T09:18:20.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.689+0000 7feb01800640 1 -- 172.21.15.62:0/3957704380 wait complete. 2024-04-25T09:18:20.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.690+0000 7feb01800640 1 Processor -- start 2024-04-25T09:18:20.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.691+0000 7feb01800640 1 -- start start 2024-04-25T09:18:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.691+0000 7feb01800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0b27a0 0x7feafc0b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.691+0000 7feb01800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 0x7feafc1e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.691+0000 7feb01800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b6a90 0x7feafc1e9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb01800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0b2d60 con 0x7feafc0b6a90 2024-04-25T09:18:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb01800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0cea50 con 0x7feafc0b27a0 2024-04-25T09:18:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb01800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0b2ee0 con 0x7feafc0b56f0 2024-04-25T09:18:20.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb00e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b6a90 0x7feafc1e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb00e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b6a90 0x7feafc1e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40224/0 (socket says 172.21.15.62:40224) 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb00e00640 1 -- 172.21.15.62:0/638385316 learned_addr learned my addr 172.21.15.62:0/638385316 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feafb400640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 0x7feafc1e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feafbe00640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0b27a0 0x7feafc0b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb00e00640 1 -- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 msgr2=0x7feafc1e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb00e00640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 0x7feafc1e91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb00e00640 1 -- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0b27a0 msgr2=0x7feafc0b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb00e00640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0b27a0 0x7feafc0b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.692+0000 7feb00e00640 1 -- 172.21.15.62:0/638385316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc1e7de0 con 0x7feafc0b6a90 2024-04-25T09:18:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.693+0000 7feb00e00640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b6a90 0x7feafc1e9710 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009f40 tx=0x7feaf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.693+0000 7feaf3e00640 1 -- 172.21.15.62:0/638385316 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4036050 con 0x7feafc0b6a90 2024-04-25T09:18:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.693+0000 7feaf3e00640 1 -- 172.21.15.62:0/638385316 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4042070 con 0x7feafc0b6a90 2024-04-25T09:18:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.693+0000 7feaf3e00640 1 -- 172.21.15.62:0/638385316 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4009510 con 0x7feafc0b6a90 2024-04-25T09:18:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.693+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc1e80c0 con 0x7feafc0b6a90 2024-04-25T09:18:20.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.693+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feafc1e8680 con 0x7feafc0b6a90 2024-04-25T09:18:20.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.694+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feafc04ef20 con 0x7feafc0b6a90 2024-04-25T09:18:20.682 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.695+0000 7feaf3e00640 1 -- 172.21.15.62:0/638385316 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feaf4007bd0 con 0x7feafc0b6a90 2024-04-25T09:18:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.696+0000 7feaf3e00640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feadc06b500 0x7feadc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.696+0000 7feaf3e00640 1 -- 172.21.15.62:0/638385316 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feaf40ab1f0 con 0x7feafc0b6a90 2024-04-25T09:18:20.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.697+0000 7feafbe00640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feadc06b500 0x7feadc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:20.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.698+0000 7feafbe00640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feadc06b500 0x7feadc06d9c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feaec00b7e0 tx=0x7feaec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:20.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.701+0000 7feaf3e00640 1 -- 172.21.15.62:0/638385316 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feaf407a020 con 0x7feafc0b6a90 2024-04-25T09:18:20.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.943+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feafc0b6180 con 0x7feafc0b6a90 2024-04-25T09:18:20.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.945+0000 7feaf3e00640 1 -- 172.21.15.62:0/638385316 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4948 (secure 0 0 0) 0x7feaf4079920 con 0x7feafc0b6a90 2024-04-25T09:18:20.932 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:20.932 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":102,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3369583759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3369583759},{"type":"v1","addr":"172.21.15.121:6837","nonce":3369583759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1888749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1888749},{"type":"v1","addr":"172.21.15.121:6836","nonce":1888749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:17:30.382991+0000","modified":"2024-04-25T09:18:11.404400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9390":{"gid":9390,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6833/1673530113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1673530113},{"type":"v1","addr":"172.21.15.62:6833","nonce":1673530113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9396":{"gid":9396,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2793862963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2793862963},{"type":"v1","addr":"172.21.15.62:6835","nonce":2793862963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:18:20.932 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 102 2024-04-25T09:18:20.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.947+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feadc06b500 msgr2=0x7feadc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.947+0000 7feb01800640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feadc06b500 0x7feadc06d9c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feaec00b7e0 tx=0x7feaec008040 comp rx=0 tx=0).stop 2024-04-25T09:18:20.935 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.947+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b6a90 msgr2=0x7feafc1e9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:20.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.947+0000 7feb01800640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b6a90 0x7feafc1e9710 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009f40 tx=0x7feaf4004a40 comp rx=0 tx=0).stop 2024-04-25T09:18:20.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.948+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 shutdown_connections 2024-04-25T09:18:20.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.948+0000 7feb01800640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7feadc06b500 0x7feadc06d9c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.948+0000 7feb01800640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feafc0b6a90 0x7feafc1e9710 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.948+0000 7feb01800640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7feafc0b56f0 0x7feafc1e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.948+0000 7feb01800640 1 --2- 172.21.15.62:0/638385316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feafc0b27a0 0x7feafc0b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:20.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.948+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 >> 172.21.15.62:0/638385316 conn(0x7feafc066d10 msgr2=0x7feafc0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:20.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.948+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 shutdown_connections 2024-04-25T09:18:20.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:20.948+0000 7feb01800640 1 -- 172.21.15.62:0/638385316 wait complete. 2024-04-25T09:18:20.954 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:18:21.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.179+0000 7f908ea00640 1 Processor -- start 2024-04-25T09:18:21.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.179+0000 7f908ea00640 1 -- start start 2024-04-25T09:18:21.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.180+0000 7f908ea00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f90880aeed0 0x7f9088166900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.180+0000 7f908ea00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908806ca70 con 0x7f90880cccf0 2024-04-25T09:18:21.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.180+0000 7f908ea00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908815c0f0 con 0x7f908815bd10 2024-04-25T09:18:21.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.180+0000 7f908ea00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908806d660 con 0x7f90880aeed0 2024-04-25T09:18:21.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.181+0000 7f9087400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f90880cccf0 0x7f90880ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45930/0 (socket says 172.21.15.62:45930) 2024-04-25T09:18:21.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.181+0000 7f9087400640 1 -- 172.21.15.62:0/3761579395 learned_addr learned my addr 172.21.15.62:0/3761579395 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:21.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.181+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718276456 0 0) 0x7f908806ca70 con 0x7f90880cccf0 2024-04-25T09:18:21.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.181+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f906c003520 con 0x7f90880cccf0 2024-04-25T09:18:21.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f908ca00640 1 --2- 172.21.15.62:0/3761579395 >> v2:172.21.15.121:3300/0 conn(0x7f90880aeed0 0x7f9088166900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298028696 0 0) 0x7f906c003520 con 0x7f90880cccf0 2024-04-25T09:18:21.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 >> v2:172.21.15.121:3300/0 conn(0x7f90880aeed0 msgr2=0x7f9088166900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:21.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 --2- 172.21.15.62:0/3761579395 >> v2:172.21.15.121:3300/0 conn(0x7f90880aeed0 0x7f9088166900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 >> v1:172.21.15.121:6789/0 conn(0x7f908815bd10 legacy=0x7f90880cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908806cd30 con 0x7f90880cccf0 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f90740019f0 con 0x7f90880cccf0 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 >> v1:172.21.15.62:6789/0 conn(0x7f90880cccf0 legacy=0x7f90880ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0039b0 0x7f906c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f906c004800 0x7f906c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0051c0 0x7f906c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908806d660 con 0x7f906c0051c0 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c00c110 con 0x7f906c0039b0 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c00c310 con 0x7f906c004800 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.182+0000 7f908ca00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0051c0 0x7f906c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.183+0000 7f9087e00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0039b0 0x7f906c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.183+0000 7f9087400640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f906c004800 0x7f906c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.183+0000 7f908ca00640 1 -- 172.21.15.62:0/3761579395 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f906c004800 msgr2=0x7f906c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.183+0000 7f908ca00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f906c004800 0x7f906c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.183+0000 7f908ca00640 1 -- 172.21.15.62:0/3761579395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0039b0 msgr2=0x7f906c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:21.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.183+0000 7f908ca00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0039b0 0x7f906c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.183+0000 7f908ca00640 1 -- 172.21.15.62:0/3761579395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c003520 con 0x7f906c0051c0 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.184+0000 7f908ca00640 1 -- 172.21.15.62:0/3761579395 reap_dead start 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.184+0000 7f908ca00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0051c0 0x7f906c00b9c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f908000ba30 tx=0x7f906c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf4f0d9201134f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.184+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080030070 con 0x7f906c0051c0 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.184+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908002ae50 con 0x7f906c0051c0 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.184+0000 7f9086a00640 1 -- 172.21.15.62:0/3761579395 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908000b600 con 0x7f906c0051c0 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.185+0000 7f908ea00640 1 -- 172.21.15.62:0/3761579395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0051c0 msgr2=0x7f906c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.185+0000 7f908ea00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0051c0 0x7f906c00b9c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f908000ba30 tx=0x7f906c005830 comp rx=0 tx=0).stop 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.185+0000 7f908ea00640 1 -- 172.21.15.62:0/3761579395 shutdown_connections 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.185+0000 7f908ea00640 1 --2- 172.21.15.62:0/3761579395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0051c0 0x7f906c00b9c0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.185+0000 7f908ea00640 1 -- 172.21.15.62:0/3761579395 >> 172.21.15.62:0/3761579395 conn(0x7f90880668f0 msgr2=0x7f9088066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:21.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.185+0000 7f908ea00640 1 -- 172.21.15.62:0/3761579395 shutdown_connections 2024-04-25T09:18:21.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.185+0000 7f908ea00640 1 -- 172.21.15.62:0/3761579395 wait complete. 2024-04-25T09:18:21.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.188+0000 7f908ea00640 1 Processor -- start 2024-04-25T09:18:21.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.188+0000 7f908ea00640 1 -- start start 2024-04-25T09:18:21.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.189+0000 7f908ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0051c0 0x7f908814f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.189+0000 7f908ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9088151980 0x7f9088153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.189+0000 7f908ea00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90880b1480 0x7f90880b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.189+0000 7f908ea00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908806daf0 con 0x7f9088151980 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.189+0000 7f908ea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908806d2f0 con 0x7f906c0051c0 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.189+0000 7f908ea00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9088157cd0 con 0x7f90880b1480 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.189+0000 7f9087400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9088151980 0x7f9088153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f9087400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9088151980 0x7f9088153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40234/0 (socket says 172.21.15.62:40234) 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f9087400640 1 -- 172.21.15.62:0/1452408866 learned_addr learned my addr 172.21.15.62:0/1452408866 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f908ca00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90880b1480 0x7f90880b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f9087e00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0051c0 0x7f908814f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f9087400640 1 -- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90880b1480 msgr2=0x7f90880b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f9087400640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90880b1480 0x7f90880b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f9087400640 1 -- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0051c0 msgr2=0x7f908814f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f9087400640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0051c0 0x7f908814f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.190+0000 7f9087400640 1 -- 172.21.15.62:0/1452408866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90881564c0 con 0x7f9088151980 2024-04-25T09:18:21.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.191+0000 7f9087400640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9088151980 0x7f9088153cb0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f9074007950 tx=0x7f907400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:21.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.191+0000 7f907fe00640 1 -- 172.21.15.62:0/1452408866 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907400ee50 con 0x7f9088151980 2024-04-25T09:18:21.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.191+0000 7f907fe00640 1 -- 172.21.15.62:0/1452408866 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074018070 con 0x7f9088151980 2024-04-25T09:18:21.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.191+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9088157000 con 0x7f9088151980 2024-04-25T09:18:21.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.191+0000 7f907fe00640 1 -- 172.21.15.62:0/1452408866 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074013660 con 0x7f9088151980 2024-04-25T09:18:21.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.191+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9088156a40 con 0x7f9088151980 2024-04-25T09:18:21.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.193+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9088072c70 con 0x7f9088151980 2024-04-25T09:18:21.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.193+0000 7f907fe00640 1 -- 172.21.15.62:0/1452408866 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9074013800 con 0x7f9088151980 2024-04-25T09:18:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.194+0000 7f907fe00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f906806b4d0 0x7f906806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.194+0000 7f907fe00640 1 -- 172.21.15.62:0/1452408866 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f907408a760 con 0x7f9088151980 2024-04-25T09:18:21.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.195+0000 7f9087e00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f906806b4d0 0x7f906806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.196+0000 7f9087e00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f906806b4d0 0x7f906806d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f908815a670 tx=0x7f9078009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:21.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.198+0000 7f907fe00640 1 -- 172.21.15.62:0/1452408866 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9074059630 con 0x7f9088151980 2024-04-25T09:18:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.438+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f908804f000 con 0x7f9088151980 2024-04-25T09:18:21.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.439+0000 7f907fe00640 1 -- 172.21.15.62:0/1452408866 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4948 (secure 0 0 0) 0x7f9074058f30 con 0x7f9088151980 2024-04-25T09:18:21.427 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:21.427 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":102,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3369583759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3369583759},{"type":"v1","addr":"172.21.15.121:6837","nonce":3369583759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1888749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1888749},{"type":"v1","addr":"172.21.15.121:6836","nonce":1888749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:17:30.382991+0000","modified":"2024-04-25T09:18:11.404400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9396},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9390":{"gid":9390,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6833/1673530113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1673530113},{"type":"v1","addr":"172.21.15.62:6833","nonce":1673530113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9396":{"gid":9396,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/2793862963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":2793862963},{"type":"v1","addr":"172.21.15.62:6835","nonce":2793862963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:18:21.427 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 102 2024-04-25T09:18:21.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f906806b4d0 msgr2=0x7f906806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f906806b4d0 0x7f906806d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f908815a670 tx=0x7f9078009040 comp rx=0 tx=0).stop 2024-04-25T09:18:21.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9088151980 msgr2=0x7f9088153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9088151980 0x7f9088153cb0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f9074007950 tx=0x7f907400e530 comp rx=0 tx=0).stop 2024-04-25T09:18:21.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 shutdown_connections 2024-04-25T09:18:21.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f906806b4d0 0x7f906806d990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9088151980 0x7f9088153cb0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f90880b1480 0x7f90880b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 --2- 172.21.15.62:0/1452408866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f906c0051c0 0x7f908814f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 >> 172.21.15.62:0/1452408866 conn(0x7f90880668f0 msgr2=0x7f9088066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 shutdown_connections 2024-04-25T09:18:21.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.442+0000 7f908ea00640 1 -- 172.21.15.62:0/1452408866 wait complete. 2024-04-25T09:18:21.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:18:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.671+0000 7fd497200640 1 Processor -- start 2024-04-25T09:18:21.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.671+0000 7fd497200640 1 -- start start 2024-04-25T09:18:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.673+0000 7fd497200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd498161f00 0x7fd4981622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.673+0000 7fd497200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 0x7fd4980cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.673+0000 7fd497200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4980b2a20 con 0x7fd4980cc9e0 2024-04-25T09:18:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.673+0000 7fd497200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4980b27d0 con 0x7fd498161f00 2024-04-25T09:18:21.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.673+0000 7fd497200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980b2d00 con 0x7fd49806d900 2024-04-25T09:18:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.673+0000 7fd495400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd4980cc9e0 0x7fd49806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45944/0 (socket says 172.21.15.62:45944) 2024-04-25T09:18:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.673+0000 7fd495400640 1 -- 172.21.15.62:0/531893092 learned_addr learned my addr 172.21.15.62:0/531893092 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.673+0000 7fd495e00640 1 --2- 172.21.15.62:0/531893092 >> v2:172.21.15.62:3300/0 conn(0x7fd498161f00 0x7fd4981622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.674+0000 7fd496800640 1 --2- 172.21.15.62:0/531893092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 0x7fd4980cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.674+0000 7fd494a00640 1 -- 172.21.15.62:0/531893092 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739931003 0 0) 0x7fd4980b2a20 con 0x7fd4980cc9e0 2024-04-25T09:18:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.674+0000 7fd495e00640 1 -- 172.21.15.62:0/531893092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 msgr2=0x7fd4980cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.674+0000 7fd495e00640 1 --2- 172.21.15.62:0/531893092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 0x7fd4980cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.674+0000 7fd495e00640 1 -- 172.21.15.62:0/531893092 >> v1:172.21.15.62:6789/0 conn(0x7fd4980cc9e0 legacy=0x7fd49806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.674+0000 7fd495e00640 1 -- 172.21.15.62:0/531893092 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4980b2fe0 con 0x7fd498161f00 2024-04-25T09:18:21.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.676+0000 7fd495e00640 1 --2- 172.21.15.62:0/531893092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981622e0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d90 tx=0x7fd48802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d53b3facf59b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:21.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.676+0000 7fd494a00640 1 -- 172.21.15.62:0/531893092 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488007740 con 0x7fd498161f00 2024-04-25T09:18:21.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.676+0000 7fd494a00640 1 -- 172.21.15.62:0/531893092 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4880078e0 con 0x7fd498161f00 2024-04-25T09:18:21.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.676+0000 7fd494a00640 1 -- 172.21.15.62:0/531893092 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488007cb0 con 0x7fd498161f00 2024-04-25T09:18:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.677+0000 7fd497200640 1 -- 172.21.15.62:0/531893092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 msgr2=0x7fd4981622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.677+0000 7fd497200640 1 --2- 172.21.15.62:0/531893092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981622e0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d90 tx=0x7fd48802a660 comp rx=0 tx=0).stop 2024-04-25T09:18:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.677+0000 7fd497200640 1 -- 172.21.15.62:0/531893092 shutdown_connections 2024-04-25T09:18:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.677+0000 7fd497200640 1 --2- 172.21.15.62:0/531893092 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 0x7fd4980cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.677+0000 7fd497200640 1 --2- 172.21.15.62:0/531893092 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981622e0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.677+0000 7fd497200640 1 -- 172.21.15.62:0/531893092 >> 172.21.15.62:0/531893092 conn(0x7fd4980b14d0 msgr2=0x7fd4980b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.677+0000 7fd497200640 1 -- 172.21.15.62:0/531893092 shutdown_connections 2024-04-25T09:18:21.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.677+0000 7fd497200640 1 -- 172.21.15.62:0/531893092 wait complete. 2024-04-25T09:18:21.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.679+0000 7fd497200640 1 Processor -- start 2024-04-25T09:18:21.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.679+0000 7fd497200640 1 -- start start 2024-04-25T09:18:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd497200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 0x7fd498159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd497200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4980cc9e0 0x7fd49815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd497200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd497200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980b4ad0 con 0x7fd498161f00 2024-04-25T09:18:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd497200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980b35a0 con 0x7fd4980cc9e0 2024-04-25T09:18:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd497200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980b4c50 con 0x7fd49806d900 2024-04-25T09:18:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd496800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd496800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40250/0 (socket says 172.21.15.62:40250) 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.680+0000 7fd496800640 1 -- 172.21.15.62:0/1332171118 learned_addr learned my addr 172.21.15.62:0/1332171118 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd495e00640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 0x7fd498159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd495400640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4980cc9e0 0x7fd49815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd496800640 1 -- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 msgr2=0x7fd498159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd496800640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 0x7fd498159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd496800640 1 -- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4980cc9e0 msgr2=0x7fd49815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd496800640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4980cc9e0 0x7fd49815b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd496800640 1 -- 172.21.15.62:0/1332171118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4981555c0 con 0x7fd498161f00 2024-04-25T09:18:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd496800640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981458f0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fd49000d5a0 tx=0x7fd49000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd48e000640 1 -- 172.21.15.62:0/1332171118 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4900125f0 con 0x7fd498161f00 2024-04-25T09:18:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd48e000640 1 -- 172.21.15.62:0/1332171118 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd490019070 con 0x7fd498161f00 2024-04-25T09:18:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49815bb10 con 0x7fd498161f00 2024-04-25T09:18:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd48e000640 1 -- 172.21.15.62:0/1332171118 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4900129d0 con 0x7fd498161f00 2024-04-25T09:18:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.681+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49815c0d0 con 0x7fd498161f00 2024-04-25T09:18:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.683+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49804ef20 con 0x7fd498161f00 2024-04-25T09:18:21.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.684+0000 7fd48e000640 1 -- 172.21.15.62:0/1332171118 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd490012b70 con 0x7fd498161f00 2024-04-25T09:18:21.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.685+0000 7fd48e000640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd47806b4d0 0x7fd47806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:21.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.685+0000 7fd48e000640 1 -- 172.21.15.62:0/1332171118 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd49008a590 con 0x7fd498161f00 2024-04-25T09:18:21.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.685+0000 7fd495e00640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd47806b4d0 0x7fd47806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:21.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.687+0000 7fd495e00640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd47806b4d0 0x7fd47806d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd488034000 tx=0x7fd488033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:21.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.689+0000 7fd48e000640 1 -- 172.21.15.62:0/1332171118 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4981555c0 con 0x7fd498161f00 2024-04-25T09:18:21.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.886+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd49806c840 con 0x7fd498161f00 2024-04-25T09:18:21.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.887+0000 7fd48e000640 1 -- 172.21.15.62:0/1332171118 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16765 (secure 0 0 0) 0x7fd490059460 con 0x7fd498161f00 2024-04-25T09:18:21.874 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:21.875 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":150,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:18:04.356543+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:17:28.263758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:17:29.401316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.10","mappings":[{"from":2,"to":5}]},{"pgid":"21.11","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:18:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd47806b4d0 msgr2=0x7fd47806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd47806b4d0 0x7fd47806d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd488034000 tx=0x7fd488033040 comp rx=0 tx=0).stop 2024-04-25T09:18:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 msgr2=0x7fd4981458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981458f0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fd49000d5a0 tx=0x7fd49000da60 comp rx=0 tx=0).stop 2024-04-25T09:18:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 shutdown_connections 2024-04-25T09:18:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fd47806b4d0 0x7fd47806d990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd498161f00 0x7fd4981458f0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4980cc9e0 0x7fd49815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 --2- 172.21.15.62:0/1332171118 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fd49806d900 0x7fd498159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:21.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 >> 172.21.15.62:0/1332171118 conn(0x7fd4980b14d0 msgr2=0x7fd4980b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:21.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 shutdown_connections 2024-04-25T09:18:21.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:21.890+0000 7fd497200640 1 -- 172.21.15.62:0/1332171118 wait complete. 2024-04-25T09:18:21.897 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:18:22.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.113+0000 7fa2bd200640 1 Processor -- start 2024-04-25T09:18:22.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.114+0000 7fa2bd200640 1 -- start start 2024-04-25T09:18:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.115+0000 7fa2bd200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2b80b5c30 0x7fa2b80b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.115+0000 7fa2bd200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa2b80d0fb0 0x7fa2b8161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.115+0000 7fa2bd200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b80cea70 con 0x7fa2b8161230 2024-04-25T09:18:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.115+0000 7fa2bd200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b80d2040 con 0x7fa2b80b5c30 2024-04-25T09:18:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.115+0000 7fa2bd200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b80ced50 con 0x7fa2b80d0fb0 2024-04-25T09:18:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b7400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa2b8161230 0x7fa2b80b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45950/0 (socket says 172.21.15.62:45950) 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b6a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2b80b5c30 0x7fa2b80b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b7400640 1 -- 172.21.15.62:0/3084147141 learned_addr learned my addr 172.21.15.62:0/3084147141 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b6a00640 1 -- 172.21.15.62:0/3084147141 >> v2:172.21.15.121:3300/0 conn(0x7fa2b80d0fb0 msgr2=0x7fa2b8161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b7e00640 1 --2- 172.21.15.62:0/3084147141 >> v2:172.21.15.121:3300/0 conn(0x7fa2b80d0fb0 0x7fa2b8161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b6a00640 1 --2- 172.21.15.62:0/3084147141 >> v2:172.21.15.121:3300/0 conn(0x7fa2b80d0fb0 0x7fa2b8161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b6000640 1 -- 172.21.15.62:0/3084147141 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215989853 0 0) 0x7fa2b80cea70 con 0x7fa2b8161230 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b6a00640 1 -- 172.21.15.62:0/3084147141 >> v1:172.21.15.62:6789/0 conn(0x7fa2b8161230 legacy=0x7fa2b80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b6a00640 1 -- 172.21.15.62:0/3084147141 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b80cf030 con 0x7fa2b80b5c30 2024-04-25T09:18:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.116+0000 7fa2b7e00640 1 --2- 172.21.15.62:0/3084147141 >> v2:172.21.15.121:3300/0 conn(0x7fa2b80d0fb0 0x7fa2b8161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.117+0000 7fa2b6a00640 1 --2- 172.21.15.62:0/3084147141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b80b5c30 0x7fa2b80b6c40 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fa2b808c590 tx=0x7fa2a002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc29e4d294379051 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.117+0000 7fa2b6000640 1 -- 172.21.15.62:0/3084147141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0007740 con 0x7fa2b80b5c30 2024-04-25T09:18:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2b6000640 1 -- 172.21.15.62:0/3084147141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a00078e0 con 0x7fa2b80b5c30 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2b6000640 1 -- 172.21.15.62:0/3084147141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0007cb0 con 0x7fa2b80b5c30 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2bd200640 1 -- 172.21.15.62:0/3084147141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b80b5c30 msgr2=0x7fa2b80b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2bd200640 1 --2- 172.21.15.62:0/3084147141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b80b5c30 0x7fa2b80b6c40 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fa2b808c590 tx=0x7fa2a002a680 comp rx=0 tx=0).stop 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2bd200640 1 -- 172.21.15.62:0/3084147141 shutdown_connections 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2bd200640 1 --2- 172.21.15.62:0/3084147141 >> v2:172.21.15.121:3300/0 conn(0x7fa2b80d0fb0 0x7fa2b8161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2bd200640 1 --2- 172.21.15.62:0/3084147141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b80b5c30 0x7fa2b80b6c40 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2bd200640 1 -- 172.21.15.62:0/3084147141 >> 172.21.15.62:0/3084147141 conn(0x7fa2b8066d10 msgr2=0x7fa2b8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2bd200640 1 -- 172.21.15.62:0/3084147141 shutdown_connections 2024-04-25T09:18:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.118+0000 7fa2bd200640 1 -- 172.21.15.62:0/3084147141 wait complete. 2024-04-25T09:18:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.120+0000 7fa2bd200640 1 Processor -- start 2024-04-25T09:18:22.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.121+0000 7fa2bd200640 1 -- start start 2024-04-25T09:18:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2bd200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80b5c30 0x7fa2b80c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2bd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b80d0fb0 0x7fa2b80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2bd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b8161230 0x7fa2b81f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2bd200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8161610 con 0x7fa2b8161230 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2bd200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80ca8d0 con 0x7fa2b80d0fb0 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2bd200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8161790 con 0x7fa2b80b5c30 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b8161230 0x7fa2b81f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2b7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b8161230 0x7fa2b81f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40260/0 (socket says 172.21.15.62:40260) 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2b7e00640 1 -- 172.21.15.62:0/1419454332 learned_addr learned my addr 172.21.15.62:0/1419454332 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2b7400640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80b5c30 0x7fa2b80c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.122+0000 7fa2b6a00640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b80d0fb0 0x7fa2b80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2b7e00640 1 -- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80b5c30 msgr2=0x7fa2b80c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2b7e00640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80b5c30 0x7fa2b80c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2b7e00640 1 -- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b80d0fb0 msgr2=0x7fa2b80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2b7e00640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b80d0fb0 0x7fa2b80cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2b7e00640 1 -- 172.21.15.62:0/1419454332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b80c8330 con 0x7fa2b8161230 2024-04-25T09:18:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2b7e00640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b8161230 0x7fa2b81f3a50 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c7f0 tx=0x7fa2b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2af400640 1 -- 172.21.15.62:0/1419454332 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0015070 con 0x7fa2b8161230 2024-04-25T09:18:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2af400640 1 -- 172.21.15.62:0/1419454332 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b0007820 con 0x7fa2b8161230 2024-04-25T09:18:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2af400640 1 -- 172.21.15.62:0/1419454332 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0007c50 con 0x7fa2b8161230 2024-04-25T09:18:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.123+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b81f44f0 con 0x7fa2b8161230 2024-04-25T09:18:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.124+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b80c75f0 con 0x7fa2b8161230 2024-04-25T09:18:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.125+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b804ef20 con 0x7fa2b8161230 2024-04-25T09:18:22.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.125+0000 7fa2af400640 1 -- 172.21.15.62:0/1419454332 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2b0018070 con 0x7fa2b8161230 2024-04-25T09:18:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.127+0000 7fa2af400640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa29406b4d0 0x7fa29406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.127+0000 7fa2af400640 1 -- 172.21.15.62:0/1419454332 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa2b008a410 con 0x7fa2b8161230 2024-04-25T09:18:22.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.128+0000 7fa2b7400640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa29406b4d0 0x7fa29406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.129+0000 7fa2b7400640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa29406b4d0 0x7fa29406d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8000f60 tx=0x7fa2a8007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:22.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.131+0000 7fa2af400640 1 -- 172.21.15.62:0/1419454332 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2b00591b0 con 0x7fa2b8161230 2024-04-25T09:18:22.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.358+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa2b80ce640 con 0x7fa2b8161230 2024-04-25T09:18:22.346 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-25T09:18:22.360+0000 7f47c0c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:18:22.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.363+0000 7fa2af400640 1 -- 172.21.15.62:0/1419454332 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa2b0058ab0 con 0x7fa2b8161230 2024-04-25T09:18:22.350 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:18:22.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.365+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa29406b4d0 msgr2=0x7fa29406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.365+0000 7fa2bd200640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa29406b4d0 0x7fa29406d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8000f60 tx=0x7fa2a8007c00 comp rx=0 tx=0).stop 2024-04-25T09:18:22.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.365+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b8161230 msgr2=0x7fa2b81f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.365+0000 7fa2bd200640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b8161230 0x7fa2b81f3a50 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c7f0 tx=0x7fa2b000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.366+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 shutdown_connections 2024-04-25T09:18:22.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.366+0000 7fa2bd200640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa29406b4d0 0x7fa29406d990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.366+0000 7fa2bd200640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2b8161230 0x7fa2b81f3a50 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.366+0000 7fa2bd200640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2b80d0fb0 0x7fa2b80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.366+0000 7fa2bd200640 1 --2- 172.21.15.62:0/1419454332 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa2b80b5c30 0x7fa2b80c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.366+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 >> 172.21.15.62:0/1419454332 conn(0x7fa2b8066d10 msgr2=0x7fa2b80b9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:22.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.366+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 shutdown_connections 2024-04-25T09:18:22.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.366+0000 7fa2bd200640 1 -- 172.21.15.62:0/1419454332 wait complete. 2024-04-25T09:18:22.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:18:22.448 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-25T09:18:22.478 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-25T09:18:22.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.570+0000 7f4914400640 1 Processor -- start 2024-04-25T09:18:22.557 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.570+0000 7f4914400640 1 -- start start 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4914400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f490c15e050 0x7f490c06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4914400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c06e330 0x7f490c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4914400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c06d7f0 con 0x7f490c15dc70 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4914400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c06d230 con 0x7f490c15e050 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4914400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c06d510 con 0x7f490c06e330 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4911a00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f490c15dc70 0x7f490c06dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45976/0 (socket says 172.21.15.62:45976) 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4911a00640 1 -- 172.21.15.62:0/1913759799 learned_addr learned my addr 172.21.15.62:0/1913759799 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4911000640 1 --2- 172.21.15.62:0/1913759799 >> v2:172.21.15.62:3300/0 conn(0x7f490c15e050 0x7f490c06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.572+0000 7f4912400640 1 --2- 172.21.15.62:0/1913759799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c06e330 0x7f490c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.573+0000 7f490be00640 1 -- 172.21.15.62:0/1913759799 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268398431 0 0) 0x7f490c06d7f0 con 0x7f490c15dc70 2024-04-25T09:18:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.573+0000 7f490be00640 1 -- 172.21.15.62:0/1913759799 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f4003520 con 0x7f490c15dc70 2024-04-25T09:18:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.573+0000 7f4911000640 1 -- 172.21.15.62:0/1913759799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c06e330 msgr2=0x7f490c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.573+0000 7f4911000640 1 --2- 172.21.15.62:0/1913759799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c06e330 0x7f490c06c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.573+0000 7f4911000640 1 -- 172.21.15.62:0/1913759799 >> v1:172.21.15.62:6789/0 conn(0x7f490c15dc70 legacy=0x7f490c06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.573+0000 7f4911000640 1 -- 172.21.15.62:0/1913759799 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0b27b0 con 0x7f490c15e050 2024-04-25T09:18:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.574+0000 7f4911000640 1 --2- 172.21.15.62:0/1913759799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c15e050 0x7f490c06c3b0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f48f80097a0 tx=0x7f48f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f01dc5cea4dab57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.574+0000 7f490be00640 1 -- 172.21.15.62:0/1913759799 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8037070 con 0x7f490c15e050 2024-04-25T09:18:22.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.574+0000 7f490be00640 1 -- 172.21.15.62:0/1913759799 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f8032440 con 0x7f490c15e050 2024-04-25T09:18:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.574+0000 7f490be00640 1 -- 172.21.15.62:0/1913759799 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8032810 con 0x7f490c15e050 2024-04-25T09:18:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.575+0000 7f4914400640 1 -- 172.21.15.62:0/1913759799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c15e050 msgr2=0x7f490c06c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.575+0000 7f4914400640 1 --2- 172.21.15.62:0/1913759799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c15e050 0x7f490c06c3b0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f48f80097a0 tx=0x7f48f802a180 comp rx=0 tx=0).stop 2024-04-25T09:18:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.575+0000 7f4914400640 1 -- 172.21.15.62:0/1913759799 shutdown_connections 2024-04-25T09:18:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.575+0000 7f4914400640 1 --2- 172.21.15.62:0/1913759799 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c06e330 0x7f490c06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.575+0000 7f4914400640 1 --2- 172.21.15.62:0/1913759799 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c15e050 0x7f490c06c3b0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.575+0000 7f4914400640 1 -- 172.21.15.62:0/1913759799 >> 172.21.15.62:0/1913759799 conn(0x7f490c0b1410 msgr2=0x7f490c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.575+0000 7f4914400640 1 -- 172.21.15.62:0/1913759799 shutdown_connections 2024-04-25T09:18:22.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.575+0000 7f4914400640 1 -- 172.21.15.62:0/1913759799 wait complete. 2024-04-25T09:18:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.576+0000 7f4914400640 1 Processor -- start 2024-04-25T09:18:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.577+0000 7f4914400640 1 -- start start 2024-04-25T09:18:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.577+0000 7f4914400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c06e330 0x7f490c143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.577+0000 7f4914400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c15dc70 0x7f490c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4914400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f490c15e050 0x7f490c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4914400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c06cfb0 con 0x7f490c06e330 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4914400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c06ce30 con 0x7f490c15e050 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4914400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0b2fb0 con 0x7f490c15dc70 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c06e330 0x7f490c143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c06e330 0x7f490c143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40304/0 (socket says 172.21.15.62:40304) 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911a00640 1 -- 172.21.15.62:0/559378824 learned_addr learned my addr 172.21.15.62:0/559378824 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911000640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c15dc70 0x7f490c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4912400640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f490c15e050 0x7f490c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911a00640 1 -- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c15dc70 msgr2=0x7f490c156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911a00640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c15dc70 0x7f490c156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911a00640 1 -- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f490c15e050 msgr2=0x7f490c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911a00640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f490c15e050 0x7f490c156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911a00640 1 -- 172.21.15.62:0/559378824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c145290 con 0x7f490c06e330 2024-04-25T09:18:22.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.578+0000 7f4911000640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c15dc70 0x7f490c156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.579+0000 7f4911a00640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c06e330 0x7f490c143ab0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f490000c430 tx=0x7f490000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.579+0000 7f4909600640 1 -- 172.21.15.62:0/559378824 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900017070 con 0x7f490c06e330 2024-04-25T09:18:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.579+0000 7f4909600640 1 -- 172.21.15.62:0/559378824 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900005ce0 con 0x7f490c06e330 2024-04-25T09:18:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.579+0000 7f4909600640 1 -- 172.21.15.62:0/559378824 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49000072f0 con 0x7f490c06e330 2024-04-25T09:18:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.579+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c145570 con 0x7f490c06e330 2024-04-25T09:18:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.579+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c145b30 con 0x7f490c06e330 2024-04-25T09:18:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.580+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c04ef20 con 0x7f490c06e330 2024-04-25T09:18:22.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.581+0000 7f4909600640 1 -- 172.21.15.62:0/559378824 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f490001a070 con 0x7f490c06e330 2024-04-25T09:18:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.582+0000 7f4909600640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f48ec06b4d0 0x7f48ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:22.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.583+0000 7f4909600640 1 -- 172.21.15.62:0/559378824 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4900089780 con 0x7f490c06e330 2024-04-25T09:18:22.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.583+0000 7f4911000640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f48ec06b4d0 0x7f48ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:22.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.585+0000 7f4911000640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f48ec06b4d0 0x7f48ec06d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f48f8034000 tx=0x7f48f802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:22.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.587+0000 7f4909600640 1 -- 172.21.15.62:0/559378824 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4900058560 con 0x7f490c06e330 2024-04-25T09:18:22.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:22.829+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f490c0af4f0 con 0x7f490c06e330 2024-04-25T09:18:23.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.473+0000 7f4909600640 1 -- 172.21.15.62:0/559378824 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4900057e60 con 0x7f490c06e330 2024-04-25T09:18:23.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f48ec06b4d0 msgr2=0x7f48ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:23.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f48ec06b4d0 0x7f48ec06d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f48f8034000 tx=0x7f48f802e000 comp rx=0 tx=0).stop 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c06e330 msgr2=0x7f490c143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c06e330 0x7f490c143ab0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f490000c430 tx=0x7f490000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 shutdown_connections 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f48ec06b4d0 0x7f48ec06d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f490c15e050 0x7f490c156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f490c15dc70 0x7f490c156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 --2- 172.21.15.62:0/559378824 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c06e330 0x7f490c143ab0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.477+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 >> 172.21.15.62:0/559378824 conn(0x7f490c0b1410 msgr2=0x7f490c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.478+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 shutdown_connections 2024-04-25T09:18:23.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.478+0000 7f4914400640 1 -- 172.21.15.62:0/559378824 wait complete. 2024-04-25T09:18:23.487 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:18:23.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.705+0000 7f244f600640 1 Processor -- start 2024-04-25T09:18:23.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.705+0000 7f244f600640 1 -- start start 2024-04-25T09:18:23.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.706+0000 7f244f600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24480ceb10 0x7f24480d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.706+0000 7f244f600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24480d1c50 0x7f24480b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.706+0000 7f244f600640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24480d3770 con 0x7f244815dd90 2024-04-25T09:18:23.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.706+0000 7f244f600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244815e170 con 0x7f24480d1c50 2024-04-25T09:18:23.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.706+0000 7f244f600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24480d3a50 con 0x7f24480ceb10 2024-04-25T09:18:23.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.707+0000 7f244cc00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f244815dd90 0x7f24480ce400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45978/0 (socket says 172.21.15.62:45978) 2024-04-25T09:18:23.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.707+0000 7f244cc00640 1 -- 172.21.15.62:0/1331067568 learned_addr learned my addr 172.21.15.62:0/1331067568 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:23.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.707+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912078400 0 0) 0x7f24480d3770 con 0x7f244815dd90 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.707+0000 7f2447e00640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24480ceb10 0x7f24480d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.707+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f242c003520 con 0x7f244815dd90 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.707+0000 7f244d600640 1 --2- 172.21.15.62:0/1331067568 >> v2:172.21.15.121:3300/0 conn(0x7f24480d1c50 0x7f24480b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2937908871 0 0) 0x7f242c003520 con 0x7f244815dd90 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24480ceb10 msgr2=0x7f24480d3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24480ceb10 0x7f24480d3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 >> v2:172.21.15.121:3300/0 conn(0x7f24480d1c50 msgr2=0x7f24480b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 --2- 172.21.15.62:0/1331067568 >> v2:172.21.15.121:3300/0 conn(0x7f24480d1c50 0x7f24480b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24480d3d30 con 0x7f244815dd90 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f24380019f0 con 0x7f244815dd90 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 >> v1:172.21.15.62:6789/0 conn(0x7f244815dd90 legacy=0x7f24480ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242c0039b0 0x7f242c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24480ceb10 0x7f242c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f242c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480d3a50 con 0x7f242c0039b0 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c00b9c0 con 0x7f242c004d40 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c00bb90 con 0x7f24480ceb10 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.708+0000 7f244cc00640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242c0039b0 0x7f242c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.709+0000 7f244cc00640 1 -- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24480ceb10 msgr2=0x7f242c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.709+0000 7f244d600640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f242c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.709+0000 7f244cc00640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f24480ceb10 0x7f242c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.709+0000 7f244cc00640 1 -- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 msgr2=0x7f242c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.709+0000 7f244cc00640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f242c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.709+0000 7f244cc00640 1 -- 172.21.15.62:0/1331067568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c003520 con 0x7f242c0039b0 2024-04-25T09:18:23.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.710+0000 7f244cc00640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242c0039b0 0x7f242c003dd0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f243800a920 tx=0x7f242c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a44f5bfe1d6dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:23.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.710+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800a4c0 con 0x7f242c0039b0 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.710+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243802f070 con 0x7f242c0039b0 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.710+0000 7f2447400640 1 -- 172.21.15.62:0/1331067568 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243802bd20 con 0x7f242c0039b0 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.711+0000 7f244f600640 1 -- 172.21.15.62:0/1331067568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242c0039b0 msgr2=0x7f242c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.711+0000 7f244f600640 1 --2- 172.21.15.62:0/1331067568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f242c0039b0 0x7f242c003dd0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f243800a920 tx=0x7f242c005830 comp rx=0 tx=0).stop 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.711+0000 7f244d600640 1 -- 172.21.15.62:0/1331067568 reap_dead start 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.711+0000 7f244f600640 1 -- 172.21.15.62:0/1331067568 shutdown_connections 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.711+0000 7f244f600640 1 -- 172.21.15.62:0/1331067568 >> 172.21.15.62:0/1331067568 conn(0x7f244806b000 msgr2=0x7f244806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.711+0000 7f244f600640 1 -- 172.21.15.62:0/1331067568 shutdown_connections 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.711+0000 7f244f600640 1 -- 172.21.15.62:0/1331067568 wait complete. 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.713+0000 7f244f600640 1 Processor -- start 2024-04-25T09:18:23.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.713+0000 7f244f600640 1 -- start start 2024-04-25T09:18:23.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f244f600640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f242c0039b0 0x7f2448159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f244f600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159bb0 0x7f24481e6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f244f600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24480b56d0 0x7f24480b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f244f600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480cf1c0 con 0x7f2448159bb0 2024-04-25T09:18:23.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f244f600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480d42f0 con 0x7f24480b56d0 2024-04-25T09:18:23.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f244f600640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24480d3070 con 0x7f242c0039b0 2024-04-25T09:18:23.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f2447e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159bb0 0x7f24481e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f2447e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159bb0 0x7f24481e6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40330/0 (socket says 172.21.15.62:40330) 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.714+0000 7f2447e00640 1 -- 172.21.15.62:0/152017493 learned_addr learned my addr 172.21.15.62:0/152017493 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f244cc00640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f242c0039b0 0x7f2448159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2447e00640 1 -- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f242c0039b0 msgr2=0x7f2448159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2447e00640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f242c0039b0 0x7f2448159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2447e00640 1 -- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24480b56d0 msgr2=0x7f24480b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2447e00640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24480b56d0 0x7f24480b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2447e00640 1 -- 172.21.15.62:0/152017493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24481f32f0 con 0x7f2448159bb0 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f244cc00640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f242c0039b0 0x7f2448159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2447e00640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159bb0 0x7f24481e6140 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f243000e7c0 tx=0x7f243000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2444c00640 1 -- 172.21.15.62:0/152017493 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430017070 con 0x7f2448159bb0 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2444c00640 1 -- 172.21.15.62:0/152017493 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243000c940 con 0x7f2448159bb0 2024-04-25T09:18:23.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f2444c00640 1 -- 172.21.15.62:0/152017493 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243000cd10 con 0x7f2448159bb0 2024-04-25T09:18:23.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.715+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24481f3590 con 0x7f2448159bb0 2024-04-25T09:18:23.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.716+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24481f3b50 con 0x7f2448159bb0 2024-04-25T09:18:23.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.717+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2448067c40 con 0x7f2448159bb0 2024-04-25T09:18:23.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.718+0000 7f2444c00640 1 -- 172.21.15.62:0/152017493 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f243001a070 con 0x7f2448159bb0 2024-04-25T09:18:23.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.718+0000 7f2444c00640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:23.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.719+0000 7f2444c00640 1 -- 172.21.15.62:0/152017493 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f243008a5e0 con 0x7f2448159bb0 2024-04-25T09:18:23.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.719+0000 7f244cc00640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:23.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.720+0000 7f244cc00640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f242406b4d0 0x7f242406d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2438033000 tx=0x7f2438007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:23.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.724+0000 7f2444c00640 1 -- 172.21.15.62:0/152017493 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2430059330 con 0x7f2448159bb0 2024-04-25T09:18:23.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:23.922+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24480719c0 con 0x7f2448159bb0 2024-04-25T09:18:24.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.497+0000 7f2444c00640 1 -- 172.21.15.62:0/152017493 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v152) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2430058c30 con 0x7f2448159bb0 2024-04-25T09:18:24.484 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:18:24.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.502+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f242406b4d0 msgr2=0x7f242406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:24.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.502+0000 7f244f600640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f242406b4d0 0x7f242406d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2438033000 tx=0x7f2438007000 comp rx=0 tx=0).stop 2024-04-25T09:18:24.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.502+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159bb0 msgr2=0x7f24481e6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:24.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.502+0000 7f244f600640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159bb0 0x7f24481e6140 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f243000e7c0 tx=0x7f243000ec80 comp rx=0 tx=0).stop 2024-04-25T09:18:24.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.503+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 shutdown_connections 2024-04-25T09:18:24.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.503+0000 7f244f600640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.503+0000 7f244f600640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2448159bb0 0x7f24481e6140 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.503+0000 7f244f600640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24480b56d0 0x7f24480b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.503+0000 7f244f600640 1 --2- 172.21.15.62:0/152017493 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f242c0039b0 0x7f2448159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.503+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 >> 172.21.15.62:0/152017493 conn(0x7f244806b000 msgr2=0x7f24480cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:24.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.504+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 shutdown_connections 2024-04-25T09:18:24.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.504+0000 7f244f600640 1 -- 172.21.15.62:0/152017493 wait complete. 2024-04-25T09:18:24.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:18:24.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.733+0000 7fa93c800640 1 Processor -- start 2024-04-25T09:18:24.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.733+0000 7fa93c800640 1 -- start start 2024-04-25T09:18:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.734+0000 7fa93c800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa93415e260 0x7fa9340d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa93c800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9340d08d0 0x7fa9340b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa93c800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9340d0d60 con 0x7fa93415de80 2024-04-25T09:18:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa93c800640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9340d3230 con 0x7fa93415e260 2024-04-25T09:18:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa93c800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9340d2620 con 0x7fa9340d08d0 2024-04-25T09:18:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa939e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa93415de80 0x7fa9340d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:45988/0 (socket says 172.21.15.62:45988) 2024-04-25T09:18:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa939e00640 1 -- 172.21.15.62:0/1472784278 learned_addr learned my addr 172.21.15.62:0/1472784278 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:24.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa939400640 1 --2- 172.21.15.62:0/1472784278 >> v2:172.21.15.62:3300/0 conn(0x7fa93415e260 0x7fa9340d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa938a00640 1 -- 172.21.15.62:0/1472784278 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433492477 0 0) 0x7fa9340d0d60 con 0x7fa93415de80 2024-04-25T09:18:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa938a00640 1 -- 172.21.15.62:0/1472784278 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa91c003520 con 0x7fa93415de80 2024-04-25T09:18:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.735+0000 7fa93a800640 1 --2- 172.21.15.62:0/1472784278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9340d08d0 0x7fa9340b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.736+0000 7fa939400640 1 -- 172.21.15.62:0/1472784278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9340d08d0 msgr2=0x7fa9340b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.736+0000 7fa939400640 1 --2- 172.21.15.62:0/1472784278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9340d08d0 0x7fa9340b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.736+0000 7fa939400640 1 -- 172.21.15.62:0/1472784278 >> v1:172.21.15.62:6789/0 conn(0x7fa93415de80 legacy=0x7fa9340d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:24.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.736+0000 7fa939400640 1 -- 172.21.15.62:0/1472784278 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9340d2900 con 0x7fa93415e260 2024-04-25T09:18:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.737+0000 7fa939400640 1 --2- 172.21.15.62:0/1472784278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93415e260 0x7fa9340d0390 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fa928009d90 tx=0x7fa92802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb8597ca3163ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.737+0000 7fa938a00640 1 -- 172.21.15.62:0/1472784278 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928007740 con 0x7fa93415e260 2024-04-25T09:18:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.737+0000 7fa938a00640 1 -- 172.21.15.62:0/1472784278 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9280078e0 con 0x7fa93415e260 2024-04-25T09:18:24.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.737+0000 7fa938a00640 1 -- 172.21.15.62:0/1472784278 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928007cb0 con 0x7fa93415e260 2024-04-25T09:18:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.738+0000 7fa93c800640 1 -- 172.21.15.62:0/1472784278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93415e260 msgr2=0x7fa9340d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.738+0000 7fa93c800640 1 --2- 172.21.15.62:0/1472784278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93415e260 0x7fa9340d0390 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fa928009d90 tx=0x7fa92802a660 comp rx=0 tx=0).stop 2024-04-25T09:18:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.738+0000 7fa93c800640 1 -- 172.21.15.62:0/1472784278 shutdown_connections 2024-04-25T09:18:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.738+0000 7fa93c800640 1 --2- 172.21.15.62:0/1472784278 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa9340d08d0 0x7fa9340b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.738+0000 7fa93c800640 1 --2- 172.21.15.62:0/1472784278 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa93415e260 0x7fa9340d0390 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.738+0000 7fa93c800640 1 -- 172.21.15.62:0/1472784278 >> 172.21.15.62:0/1472784278 conn(0x7fa9340668f0 msgr2=0x7fa934066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:24.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.738+0000 7fa93c800640 1 -- 172.21.15.62:0/1472784278 shutdown_connections 2024-04-25T09:18:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.738+0000 7fa93c800640 1 -- 172.21.15.62:0/1472784278 wait complete. 2024-04-25T09:18:24.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.740+0000 7fa93c800640 1 Processor -- start 2024-04-25T09:18:24.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.740+0000 7fa93c800640 1 -- start start 2024-04-25T09:18:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa93c800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9340d08d0 0x7fa934155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa93c800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa93415de80 0x7fa93415f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa93c800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa93415e260 0x7fa9341568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa93c800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9340ce820 con 0x7fa9340d08d0 2024-04-25T09:18:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa93c800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9340d2ec0 con 0x7fa93415e260 2024-04-25T09:18:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa93c800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9340d2270 con 0x7fa93415de80 2024-04-25T09:18:24.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa939e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9340d08d0 0x7fa934155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa939e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9340d08d0 0x7fa934155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:40338/0 (socket says 172.21.15.62:40338) 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.741+0000 7fa939e00640 1 -- 172.21.15.62:0/2890332961 learned_addr learned my addr 172.21.15.62:0/2890332961 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa939400640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa93415de80 0x7fa93415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa939e00640 1 -- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa93415de80 msgr2=0x7fa93415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa939e00640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa93415de80 0x7fa93415f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa939e00640 1 -- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa93415e260 msgr2=0x7fa9341568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa939e00640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa93415e260 0x7fa9341568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa939e00640 1 -- 172.21.15.62:0/2890332961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93415e8e0 con 0x7fa9340d08d0 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa939e00640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9340d08d0 0x7fa934155330 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c430 tx=0x7fa92400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa92e000640 1 -- 172.21.15.62:0/2890332961 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924017070 con 0x7fa9340d08d0 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa92e000640 1 -- 172.21.15.62:0/2890332961 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa924005ce0 con 0x7fa9340d08d0 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa92e000640 1 -- 172.21.15.62:0/2890332961 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240072f0 con 0x7fa9340d08d0 2024-04-25T09:18:24.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93415ebc0 con 0x7fa9340d08d0 2024-04-25T09:18:24.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.742+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93415f180 con 0x7fa9340d08d0 2024-04-25T09:18:24.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.744+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93415e670 con 0x7fa9340d08d0 2024-04-25T09:18:24.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.744+0000 7fa92e000640 1 -- 172.21.15.62:0/2890332961 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa92401a070 con 0x7fa9340d08d0 2024-04-25T09:18:24.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.745+0000 7fa92e000640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa91806b4d0 0x7fa91806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:24.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.745+0000 7fa92e000640 1 -- 172.21.15.62:0/2890332961 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fa9240895d0 con 0x7fa9340d08d0 2024-04-25T09:18:24.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.746+0000 7fa939400640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa91806b4d0 0x7fa91806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:24.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.747+0000 7fa939400640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa91806b4d0 0x7fa91806d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa928034000 tx=0x7fa928033040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:24.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.750+0000 7fa92e000640 1 -- 172.21.15.62:0/2890332961 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa924058560 con 0x7fa9340d08d0 2024-04-25T09:18:24.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:24.951+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa934155c90 con 0x7fa9340d08d0 2024-04-25T09:18:25.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.510+0000 7fa92e000640 1 -- 172.21.15.62:0/2890332961 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v153) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa924057e60 con 0x7fa9340d08d0 2024-04-25T09:18:25.498 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-25T09:18:25.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.514+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa91806b4d0 msgr2=0x7fa91806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:25.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.514+0000 7fa93c800640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa91806b4d0 0x7fa91806d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa928034000 tx=0x7fa928033040 comp rx=0 tx=0).stop 2024-04-25T09:18:25.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.514+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9340d08d0 msgr2=0x7fa934155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:25.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.514+0000 7fa93c800640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9340d08d0 0x7fa934155330 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fa92400c430 tx=0x7fa92400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.515+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 shutdown_connections 2024-04-25T09:18:25.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.515+0000 7fa93c800640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fa91806b4d0 0x7fa91806d990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.515+0000 7fa93c800640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa93415e260 0x7fa9341568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.515+0000 7fa93c800640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fa93415de80 0x7fa93415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.515+0000 7fa93c800640 1 --2- 172.21.15.62:0/2890332961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9340d08d0 0x7fa934155330 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.515+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 >> 172.21.15.62:0/2890332961 conn(0x7fa9340668f0 msgr2=0x7fa934158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:25.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.515+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 shutdown_connections 2024-04-25T09:18:25.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.515+0000 7fa93c800640 1 -- 172.21.15.62:0/2890332961 wait complete. 2024-04-25T09:18:25.523 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:18:25.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.750+0000 7fae4aa00640 1 Processor -- start 2024-04-25T09:18:25.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.750+0000 7fae4aa00640 1 -- start start 2024-04-25T09:18:25.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.752+0000 7fae4aa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4406d9d0 0x7fae4415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:25.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.752+0000 7fae4aa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fae4415e5f0 0x7fae440b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:25.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.752+0000 7fae4aa00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4415ee30 con 0x7fae440ceb50 2024-04-25T09:18:25.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.752+0000 7fae4aa00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae441618b0 con 0x7fae4415e5f0 2024-04-25T09:18:25.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.752+0000 7fae4aa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae44161b90 con 0x7fae4406d9d0 2024-04-25T09:18:25.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.752+0000 7fae49600640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fae440ceb50 0x7fae4406d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50422/0 (socket says 172.21.15.62:50422) 2024-04-25T09:18:25.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.752+0000 7fae49600640 1 -- 172.21.15.62:0/1422735897 learned_addr learned my addr 172.21.15.62:0/1422735897 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:25.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.753+0000 7fae48c00640 1 --2- 172.21.15.62:0/1422735897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4406d9d0 0x7fae4415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:25.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.753+0000 7fae4a000640 1 --2- 172.21.15.62:0/1422735897 >> v2:172.21.15.121:3300/0 conn(0x7fae4415e5f0 0x7fae440b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:25.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.753+0000 7fae43e00640 1 -- 172.21.15.62:0/1422735897 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820166137 0 0) 0x7fae4415ee30 con 0x7fae440ceb50 2024-04-25T09:18:25.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.753+0000 7fae43e00640 1 -- 172.21.15.62:0/1422735897 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae2c003520 con 0x7fae440ceb50 2024-04-25T09:18:25.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.753+0000 7fae48c00640 1 -- 172.21.15.62:0/1422735897 >> v2:172.21.15.121:3300/0 conn(0x7fae4415e5f0 msgr2=0x7fae440b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:25.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.753+0000 7fae48c00640 1 --2- 172.21.15.62:0/1422735897 >> v2:172.21.15.121:3300/0 conn(0x7fae4415e5f0 0x7fae440b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.753+0000 7fae48c00640 1 -- 172.21.15.62:0/1422735897 >> v1:172.21.15.62:6789/0 conn(0x7fae440ceb50 legacy=0x7fae4406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:25.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.754+0000 7fae48c00640 1 -- 172.21.15.62:0/1422735897 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440cef30 con 0x7fae4406d9d0 2024-04-25T09:18:25.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.754+0000 7fae4a000640 1 --2- 172.21.15.62:0/1422735897 >> v2:172.21.15.121:3300/0 conn(0x7fae4415e5f0 0x7fae440b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:18:25.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.755+0000 7fae48c00640 1 --2- 172.21.15.62:0/1422735897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4406d9d0 0x7fae4415e0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae300097a0 tx=0x7fae3002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c4ba57303896392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:25.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.755+0000 7fae43e00640 1 -- 172.21.15.62:0/1422735897 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30037070 con 0x7fae4406d9d0 2024-04-25T09:18:25.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.755+0000 7fae43e00640 1 -- 172.21.15.62:0/1422735897 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3002ae40 con 0x7fae4406d9d0 2024-04-25T09:18:25.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.755+0000 7fae43e00640 1 -- 172.21.15.62:0/1422735897 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30032660 con 0x7fae4406d9d0 2024-04-25T09:18:25.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.756+0000 7fae4aa00640 1 -- 172.21.15.62:0/1422735897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4406d9d0 msgr2=0x7fae4415e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:25.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.756+0000 7fae4aa00640 1 --2- 172.21.15.62:0/1422735897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4406d9d0 0x7fae4415e0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae300097a0 tx=0x7fae3002a0f0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.756+0000 7fae4aa00640 1 -- 172.21.15.62:0/1422735897 shutdown_connections 2024-04-25T09:18:25.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.756+0000 7fae4aa00640 1 --2- 172.21.15.62:0/1422735897 >> v2:172.21.15.121:3300/0 conn(0x7fae4415e5f0 0x7fae440b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.756+0000 7fae4aa00640 1 --2- 172.21.15.62:0/1422735897 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4406d9d0 0x7fae4415e0b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.756+0000 7fae4aa00640 1 -- 172.21.15.62:0/1422735897 >> 172.21.15.62:0/1422735897 conn(0x7fae44066f20 msgr2=0x7fae440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:25.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.756+0000 7fae4aa00640 1 -- 172.21.15.62:0/1422735897 shutdown_connections 2024-04-25T09:18:25.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.756+0000 7fae4aa00640 1 -- 172.21.15.62:0/1422735897 wait complete. 2024-04-25T09:18:25.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.758+0000 7fae4aa00640 1 Processor -- start 2024-04-25T09:18:25.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.758+0000 7fae4aa00640 1 -- start start 2024-04-25T09:18:25.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae4aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae4406d9d0 0x7fae44158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:25.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae4aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae440ceb50 0x7fae44157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:25.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae4aa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4415e5f0 0x7fae44157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:25.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae4aa00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44162510 con 0x7fae440ceb50 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae4aa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae44162390 con 0x7fae4406d9d0 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae4aa00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440b4a90 con 0x7fae4415e5f0 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae48c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae440ceb50 0x7fae44157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae48c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae440ceb50 0x7fae44157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48660/0 (socket says 172.21.15.62:48660) 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae48c00640 1 -- 172.21.15.62:0/3305978212 learned_addr learned my addr 172.21.15.62:0/3305978212 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae49600640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae4406d9d0 0x7fae44158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.759+0000 7fae4a000640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4415e5f0 0x7fae44157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae48c00640 1 -- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4415e5f0 msgr2=0x7fae44157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae48c00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4415e5f0 0x7fae44157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae48c00640 1 -- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae4406d9d0 msgr2=0x7fae44158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae48c00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae4406d9d0 0x7fae44158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae48c00640 1 -- 172.21.15.62:0/3305978212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4415ab60 con 0x7fae440ceb50 2024-04-25T09:18:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae48c00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae440ceb50 0x7fae44157210 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fae30035040 tx=0x7fae300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae41600640 1 -- 172.21.15.62:0/3305978212 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30041070 con 0x7fae440ceb50 2024-04-25T09:18:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae41600640 1 -- 172.21.15.62:0/3305978212 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3003c460 con 0x7fae440ceb50 2024-04-25T09:18:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae41600640 1 -- 172.21.15.62:0/3305978212 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30037040 con 0x7fae440ceb50 2024-04-25T09:18:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.760+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4415ae40 con 0x7fae440ceb50 2024-04-25T09:18:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.761+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4415b400 con 0x7fae440ceb50 2024-04-25T09:18:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.762+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4404ef20 con 0x7fae440ceb50 2024-04-25T09:18:25.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.763+0000 7fae41600640 1 -- 172.21.15.62:0/3305978212 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae3003a070 con 0x7fae440ceb50 2024-04-25T09:18:25.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.763+0000 7fae41600640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae2406b500 0x7fae2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:25.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.764+0000 7fae41600640 1 -- 172.21.15.62:0/3305978212 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fae300aa200 con 0x7fae440ceb50 2024-04-25T09:18:25.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.764+0000 7fae49600640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae2406b500 0x7fae2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:25.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.765+0000 7fae49600640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae2406b500 0x7fae2406d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fae38005880 tx=0x7fae38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:25.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.768+0000 7fae41600640 1 -- 172.21.15.62:0/3305978212 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae30079250 con 0x7fae440ceb50 2024-04-25T09:18:25.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.968+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fae440ada80 con 0x7fae440ceb50 2024-04-25T09:18:25.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.973+0000 7fae41600640 1 -- 172.21.15.62:0/3305978212 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fae30078b50 con 0x7fae440ceb50 2024-04-25T09:18:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae2406b500 msgr2=0x7fae2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae2406b500 0x7fae2406d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fae38005880 tx=0x7fae38009040 comp rx=0 tx=0).stop 2024-04-25T09:18:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae440ceb50 msgr2=0x7fae44157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:25.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae440ceb50 0x7fae44157210 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fae30035040 tx=0x7fae300046c0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 shutdown_connections 2024-04-25T09:18:25.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fae2406b500 0x7fae2406d9c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae440ceb50 0x7fae44157210 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fae4415e5f0 0x7fae44157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 --2- 172.21.15.62:0/3305978212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae4406d9d0 0x7fae44158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:25.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 >> 172.21.15.62:0/3305978212 conn(0x7fae44066f20 msgr2=0x7fae440b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:25.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 shutdown_connections 2024-04-25T09:18:25.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:25.976+0000 7fae4aa00640 1 -- 172.21.15.62:0/3305978212 wait complete. 2024-04-25T09:18:25.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T09:18:26.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.206+0000 7f882cc00640 1 Processor -- start 2024-04-25T09:18:26.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.207+0000 7f882cc00640 1 -- start start 2024-04-25T09:18:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.208+0000 7f882cc00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f88280b2d40 0x7f88280b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.208+0000 7f882cc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f88280b8cf0 0x7f88280d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.208+0000 7f882cc00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88280b7780 con 0x7f882815de70 2024-04-25T09:18:26.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.208+0000 7f882cc00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88280b8320 con 0x7f88280b2d40 2024-04-25T09:18:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.208+0000 7f882cc00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88280b9ad0 con 0x7f88280b8cf0 2024-04-25T09:18:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.209+0000 7f8825400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f88280b2d40 0x7f88280b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:26.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.209+0000 7f8825400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f88280b2d40 0x7f88280b87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48674/0 (socket says 172.21.15.62:48674) 2024-04-25T09:18:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.209+0000 7f8825400640 1 -- 172.21.15.62:0/3300421616 learned_addr learned my addr 172.21.15.62:0/3300421616 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.209+0000 7f8826800640 1 --2- 172.21.15.62:0/3300421616 >> v2:172.21.15.121:3300/0 conn(0x7f88280b8cf0 0x7f88280d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.210+0000 7f8825400640 1 -- 172.21.15.62:0/3300421616 >> v2:172.21.15.121:3300/0 conn(0x7f88280b8cf0 msgr2=0x7f88280d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.210+0000 7f8825400640 1 --2- 172.21.15.62:0/3300421616 >> v2:172.21.15.121:3300/0 conn(0x7f88280b8cf0 0x7f88280d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.210+0000 7f8825400640 1 -- 172.21.15.62:0/3300421616 >> v1:172.21.15.121:6789/0 conn(0x7f882815de70 legacy=0x7f88280b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.210+0000 7f8825400640 1 -- 172.21.15.62:0/3300421616 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88280b79f0 con 0x7f88280b2d40 2024-04-25T09:18:26.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.211+0000 7f8825400640 1 --2- 172.21.15.62:0/3300421616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88280b2d40 0x7f88280b87b0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f88280d0f80 tx=0x7f881002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e3a8f31d43885f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.211+0000 7f8824a00640 1 -- 172.21.15.62:0/3300421616 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810036070 con 0x7f88280b2d40 2024-04-25T09:18:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.211+0000 7f8824a00640 1 -- 172.21.15.62:0/3300421616 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8810031600 con 0x7f88280b2d40 2024-04-25T09:18:26.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.211+0000 7f8824a00640 1 -- 172.21.15.62:0/3300421616 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88100319f0 con 0x7f88280b2d40 2024-04-25T09:18:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.212+0000 7f882cc00640 1 -- 172.21.15.62:0/3300421616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88280b2d40 msgr2=0x7f88280b87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.212+0000 7f882cc00640 1 --2- 172.21.15.62:0/3300421616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88280b2d40 0x7f88280b87b0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f88280d0f80 tx=0x7f881002a7c0 comp rx=0 tx=0).stop 2024-04-25T09:18:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.212+0000 7f882cc00640 1 -- 172.21.15.62:0/3300421616 shutdown_connections 2024-04-25T09:18:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.212+0000 7f882cc00640 1 --2- 172.21.15.62:0/3300421616 >> v2:172.21.15.121:3300/0 conn(0x7f88280b8cf0 0x7f88280d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.212+0000 7f882cc00640 1 --2- 172.21.15.62:0/3300421616 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f88280b2d40 0x7f88280b87b0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.212+0000 7f882cc00640 1 -- 172.21.15.62:0/3300421616 >> 172.21.15.62:0/3300421616 conn(0x7f882806b000 msgr2=0x7f882806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.212+0000 7f882cc00640 1 -- 172.21.15.62:0/3300421616 shutdown_connections 2024-04-25T09:18:26.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.212+0000 7f882cc00640 1 -- 172.21.15.62:0/3300421616 wait complete. 2024-04-25T09:18:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.215+0000 7f882cc00640 1 Processor -- start 2024-04-25T09:18:26.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.215+0000 7f882cc00640 1 -- start start 2024-04-25T09:18:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f882cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88280b2d40 0x7f88280b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f882cc00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88280b8cf0 0x7f88280cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f882cc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882815de70 0x7f88280cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f882cc00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88280d23c0 con 0x7f882815de70 2024-04-25T09:18:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f882cc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88280b7fb0 con 0x7f88280b2d40 2024-04-25T09:18:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f882cc00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88280d2540 con 0x7f88280b8cf0 2024-04-25T09:18:26.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f8826800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882815de70 0x7f88280cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f8826800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882815de70 0x7f88280cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48680/0 (socket says 172.21.15.62:48680) 2024-04-25T09:18:26.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.216+0000 7f8826800640 1 -- 172.21.15.62:0/2631684712 learned_addr learned my addr 172.21.15.62:0/2631684712 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8825400640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88280b8cf0 0x7f88280cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8825e00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88280b2d40 0x7f88280b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8826800640 1 -- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88280b8cf0 msgr2=0x7f88280cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8826800640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88280b8cf0 0x7f88280cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8826800640 1 -- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88280b2d40 msgr2=0x7f88280b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8826800640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88280b2d40 0x7f88280b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8826800640 1 -- 172.21.15.62:0/2631684712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88280cca30 con 0x7f882815de70 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8825400640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88280b8cf0 0x7f88280cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:26.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.217+0000 7f8826800640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882815de70 0x7f88280cbb80 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f882000d800 tx=0x7f882000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.218+0000 7f881e000640 1 -- 172.21.15.62:0/2631684712 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882000b6c0 con 0x7f882815de70 2024-04-25T09:18:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.218+0000 7f881e000640 1 -- 172.21.15.62:0/2631684712 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8820019070 con 0x7f882815de70 2024-04-25T09:18:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.218+0000 7f881e000640 1 -- 172.21.15.62:0/2631684712 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882000bac0 con 0x7f882815de70 2024-04-25T09:18:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.218+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88280ccd10 con 0x7f882815de70 2024-04-25T09:18:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.218+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88280cd2d0 con 0x7f882815de70 2024-04-25T09:18:26.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.219+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88280729a0 con 0x7f882815de70 2024-04-25T09:18:26.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.220+0000 7f881e000640 1 -- 172.21.15.62:0/2631684712 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f882000bc60 con 0x7f882815de70 2024-04-25T09:18:26.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.221+0000 7f881e000640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f880406b500 0x7f880406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:26.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.221+0000 7f881e000640 1 -- 172.21.15.62:0/2631684712 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f882008a360 con 0x7f882815de70 2024-04-25T09:18:26.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.222+0000 7f8825e00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f880406b500 0x7f880406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:26.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.223+0000 7f8825e00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f880406b500 0x7f880406d9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8818007fd0 tx=0x7f8818009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:26.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.226+0000 7f881e000640 1 -- 172.21.15.62:0/2631684712 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88200594d0 con 0x7f882815de70 2024-04-25T09:18:26.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:26.553+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f882804f000 con 0x7f882815de70 2024-04-25T09:18:27.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.516+0000 7f881e000640 1 -- 172.21.15.62:0/2631684712 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v684) v1 ==== 159+0+0 (secure 0 0 0) 0x7f8820058dd0 con 0x7f882815de70 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.520+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f880406b500 msgr2=0x7f880406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.520+0000 7f882cc00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f880406b500 0x7f880406d9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8818007fd0 tx=0x7f8818009040 comp rx=0 tx=0).stop 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.520+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882815de70 msgr2=0x7f88280cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.520+0000 7f882cc00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882815de70 0x7f88280cbb80 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f882000d800 tx=0x7f882000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.521+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 shutdown_connections 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.521+0000 7f882cc00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f880406b500 0x7f880406d9c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.521+0000 7f882cc00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f882815de70 0x7f88280cbb80 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.521+0000 7f882cc00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f88280b8cf0 0x7f88280cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.521+0000 7f882cc00640 1 --2- 172.21.15.62:0/2631684712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88280b2d40 0x7f88280b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.521+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 >> 172.21.15.62:0/2631684712 conn(0x7f882806b000 msgr2=0x7f88280c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:27.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.521+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 shutdown_connections 2024-04-25T09:18:27.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:27.521+0000 7f882cc00640 1 -- 172.21.15.62:0/2631684712 wait complete. 2024-04-25T09:18:27.531 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ERROR 2024-04-25T09:18:27.532 INFO:tasks.cephfs_test_runner: 2024-04-25T09:18:27.532 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T09:18:27.532 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:18:27.532 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:18:27.532 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T09:18:27.532 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T09:18:27.533 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T09:18:27.533 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T09:18:27.533 INFO:tasks.cephfs_test_runner: 2024-04-25T09:18:27.533 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:18:27.533 INFO:tasks.cephfs_test_runner:Ran 9 tests in 872.423s 2024-04-25T09:18:27.533 INFO:tasks.cephfs_test_runner: 2024-04-25T09:18:27.533 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner: 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T09:18:27.534 INFO:tasks.cephfs_test_runner: 2024-04-25T09:18:27.535 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:18:27.748 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=58cf751d206c4238b530ec96ed4852b2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:18:27.750 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T09:18:27.761 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T09:18:27.777 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T09:18:27.778 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:18:27.778 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:27.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:18:27.843 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T09:18:27.843 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:27.894 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052738,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249004k,nr_inodes=812251,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498012k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:18:27.894 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:18:27.894 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:27.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:18:27.953 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-25T09:18:27.953 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:18:28.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052691,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248964k,nr_inodes=812241,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497936k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:18:28.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.008 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:18:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:18:28.069 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:18:28.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.070 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:18:28.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:18:28.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.250 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-25T09:18:28.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.368 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-25T09:18:28.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-25T09:18:28.537 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-04-25T09:18:28.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:18:28.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.541 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-25T09:18:28.599 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T09:18:28.599 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T09:18:28.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.600 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:18:28.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.688 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T09:18:28.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.761 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-25T09:18:28.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.881 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-25T09:18:28.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:18:28.923 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-25T09:18:29.054 INFO:teuthology.orchestra.run.smithi121.stderr:Cannot find device "ceph-brx" 2024-04-25T09:18:29.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:18:29.059 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T09:18:29.069 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T09:18:29.070 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:18:29.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.303+0000 7f8c01400640 1 Processor -- start 2024-04-25T09:18:29.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.303+0000 7f8c01400640 1 -- start start 2024-04-25T09:18:29.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.304+0000 7f8c01400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8bfc0cfed0 0x7f8bfc15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.304+0000 7f8c01400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8bfc0709c0 0x7f8bfc0b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.304+0000 7f8c01400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bfc06c310 con 0x7f8bfc072340 2024-04-25T09:18:29.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.304+0000 7f8c01400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc15f5f0 con 0x7f8bfc0709c0 2024-04-25T09:18:29.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.304+0000 7f8c01400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bfc06c5b0 con 0x7f8bfc0cfed0 2024-04-25T09:18:29.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfb200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8bfc0709c0 0x7f8bfc0b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfb200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8bfc0709c0 0x7f8bfc0b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48684/0 (socket says 172.21.15.62:48684) 2024-04-25T09:18:29.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfb200640 1 -- 172.21.15.62:0/1357978900 learned_addr learned my addr 172.21.15.62:0/1357978900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:29.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfa800640 1 --2- 172.21.15.62:0/1357978900 >> v2:172.21.15.121:3300/0 conn(0x7f8bfc0cfed0 0x7f8bfc15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfb200640 1 -- 172.21.15.62:0/1357978900 >> v2:172.21.15.121:3300/0 conn(0x7f8bfc0cfed0 msgr2=0x7f8bfc15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfb200640 1 --2- 172.21.15.62:0/1357978900 >> v2:172.21.15.121:3300/0 conn(0x7f8bfc0cfed0 0x7f8bfc15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfb200640 1 -- 172.21.15.62:0/1357978900 >> v1:172.21.15.121:6789/0 conn(0x7f8bfc072340 legacy=0x7f8bfc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfb200640 1 -- 172.21.15.62:0/1357978900 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc06c890 con 0x7f8bfc0709c0 2024-04-25T09:18:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.305+0000 7f8bfa800640 1 --2- 172.21.15.62:0/1357978900 >> v2:172.21.15.121:3300/0 conn(0x7f8bfc0cfed0 0x7f8bfc15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.306+0000 7f8bfb200640 1 --2- 172.21.15.62:0/1357978900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc0709c0 0x7f8bfc0b2880 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0cd560 tx=0x7f8bf402c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad1ba0a52d4ecc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.307+0000 7f8bf9e00640 1 -- 172.21.15.62:0/1357978900 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4033610 con 0x7f8bfc0709c0 2024-04-25T09:18:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.307+0000 7f8bf9e00640 1 -- 172.21.15.62:0/1357978900 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf40337b0 con 0x7f8bfc0709c0 2024-04-25T09:18:29.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.307+0000 7f8bf9e00640 1 -- 172.21.15.62:0/1357978900 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4033b80 con 0x7f8bfc0709c0 2024-04-25T09:18:29.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.307+0000 7f8c01400640 1 -- 172.21.15.62:0/1357978900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc0709c0 msgr2=0x7f8bfc0b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.307+0000 7f8c01400640 1 --2- 172.21.15.62:0/1357978900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc0709c0 0x7f8bfc0b2880 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0cd560 tx=0x7f8bf402c900 comp rx=0 tx=0).stop 2024-04-25T09:18:29.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.307+0000 7f8c01400640 1 -- 172.21.15.62:0/1357978900 shutdown_connections 2024-04-25T09:18:29.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.307+0000 7f8c01400640 1 --2- 172.21.15.62:0/1357978900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc0709c0 0x7f8bfc0b2880 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.307+0000 7f8c01400640 1 --2- 172.21.15.62:0/1357978900 >> v2:172.21.15.121:3300/0 conn(0x7f8bfc0cfed0 0x7f8bfc15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.308+0000 7f8c01400640 1 -- 172.21.15.62:0/1357978900 >> 172.21.15.62:0/1357978900 conn(0x7f8bfc0668f0 msgr2=0x7f8bfc066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:29.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.308+0000 7f8c01400640 1 -- 172.21.15.62:0/1357978900 shutdown_connections 2024-04-25T09:18:29.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.308+0000 7f8c01400640 1 -- 172.21.15.62:0/1357978900 wait complete. 2024-04-25T09:18:29.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.309+0000 7f8c01400640 1 Processor -- start 2024-04-25T09:18:29.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.309+0000 7f8c01400640 1 -- start start 2024-04-25T09:18:29.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.310+0000 7f8c01400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8bfc0709c0 0x7f8bfc13dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.310+0000 7f8c01400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bfc0cfed0 0x7f8bfc13d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.310+0000 7f8c01400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc141cf0 0x7f8bfc13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.310+0000 7f8c01400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc06d130 con 0x7f8bfc141cf0 2024-04-25T09:18:29.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.310+0000 7f8c01400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc06ce50 con 0x7f8bfc0cfed0 2024-04-25T09:18:29.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.310+0000 7f8c01400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc0b2dc0 con 0x7f8bfc0709c0 2024-04-25T09:18:29.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc141cf0 0x7f8bfc13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc141cf0 0x7f8bfc13ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48696/0 (socket says 172.21.15.62:48696) 2024-04-25T09:18:29.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfb200640 1 -- 172.21.15.62:0/3677573448 learned_addr learned my addr 172.21.15.62:0/3677573448 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfa800640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8bfc0709c0 0x7f8bfc13dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfb200640 1 -- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8bfc0709c0 msgr2=0x7f8bfc13dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfb200640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8bfc0709c0 0x7f8bfc13dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bf3e00640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bfc0cfed0 0x7f8bfc13d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfb200640 1 -- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bfc0cfed0 msgr2=0x7f8bfc13d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfb200640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bfc0cfed0 0x7f8bfc13d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bfb200640 1 -- 172.21.15.62:0/3677573448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc1421a0 con 0x7f8bfc141cf0 2024-04-25T09:18:29.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.311+0000 7f8bf3e00640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bfc0cfed0 0x7f8bfc13d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:29.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.312+0000 7f8bfb200640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc141cf0 0x7f8bfc13ea90 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f8bf402cef0 tx=0x7f8bf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:29.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.312+0000 7f8bf2a00640 1 -- 172.21.15.62:0/3677573448 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4040070 con 0x7f8bfc141cf0 2024-04-25T09:18:29.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.312+0000 7f8bf2a00640 1 -- 172.21.15.62:0/3677573448 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf403c830 con 0x7f8bfc141cf0 2024-04-25T09:18:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.312+0000 7f8bf2a00640 1 -- 172.21.15.62:0/3677573448 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf403cc00 con 0x7f8bfc141cf0 2024-04-25T09:18:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.312+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc15cdc0 con 0x7f8bfc141cf0 2024-04-25T09:18:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.312+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc15d380 con 0x7f8bfc141cf0 2024-04-25T09:18:29.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.313+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc072c70 con 0x7f8bfc141cf0 2024-04-25T09:18:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.314+0000 7f8bf2a00640 1 -- 172.21.15.62:0/3677573448 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bf4043070 con 0x7f8bfc141cf0 2024-04-25T09:18:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.315+0000 7f8bf2a00640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8bd806b4d0 0x7f8bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.315+0000 7f8bf2a00640 1 -- 172.21.15.62:0/3677573448 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8bf40aac00 con 0x7f8bfc141cf0 2024-04-25T09:18:29.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.316+0000 7f8bfa800640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8bd806b4d0 0x7f8bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.317+0000 7f8bfa800640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8bd806b4d0 0x7f8bd806d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00b840 tx=0x7f8bec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:29.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.320+0000 7f8bf2a00640 1 -- 172.21.15.62:0/3677573448 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bfc1421a0 con 0x7f8bfc141cf0 2024-04-25T09:18:29.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.515+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bfc15cb70 con 0x7f8bd806b4d0 2024-04-25T09:18:29.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.518+0000 7f8bf2a00640 1 -- 172.21.15.62:0/3677573448 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13600 (secure 0 0 0) 0x7f8bfc15cb70 con 0x7f8bd806b4d0 2024-04-25T09:18:29.505 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:29.505 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:18:29.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8bd806b4d0 msgr2=0x7f8bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8bd806b4d0 0x7f8bd806d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00b840 tx=0x7f8bec008040 comp rx=0 tx=0).stop 2024-04-25T09:18:29.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc141cf0 msgr2=0x7f8bfc13ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc141cf0 0x7f8bfc13ea90 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f8bf402cef0 tx=0x7f8bf4004a40 comp rx=0 tx=0).stop 2024-04-25T09:18:29.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 shutdown_connections 2024-04-25T09:18:29.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f8bd806b4d0 0x7f8bd806d990 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bfc141cf0 0x7f8bfc13ea90 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bfc0cfed0 0x7f8bfc13d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 --2- 172.21.15.62:0/3677573448 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f8bfc0709c0 0x7f8bfc13dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 >> 172.21.15.62:0/3677573448 conn(0x7f8bfc0668f0 msgr2=0x7f8bfc0b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:29.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 shutdown_connections 2024-04-25T09:18:29.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.521+0000 7f8c01400640 1 -- 172.21.15.62:0/3677573448 wait complete. 2024-04-25T09:18:29.529 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2024-04-25T09:18:28.237669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2588820,"kb_used_data":2113192,"kb_used_omap":12,"kb_used_meta":475507,"kb_avail":752385900,"statfs":{"total":773094113280,"available":770443161600,"internally_reserved":0,"allocated":2163908608,"data_stored":2155382792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":486919728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1078121341,"num_objects":-1047,"num_object_clones":0,"num_object_copies":-2094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1047,"num_whiteouts":0,"num_read":-36,"num_read_kb":-4296,"num_write":-1172,"num_write_kb":-1053012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-23,"num_bytes_recovered":-24117248,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002367"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T09:18:24.562396+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:18:24.562396+0000","last_peered":"2024-04-25T09:18:24.562396+0000","last_clean":"2024-04-25T09:18:24.562396+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:18:24.562396+0000","last_undegraded":"2024-04-25T09:18:24.562396+0000","last_fullsized":"2024-04-25T09:18:24.562396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":330704,"kb_used_data":277316,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94041136,"statfs":{"total":96636764160,"available":96298123264,"internally_reserved":0,"allocated":283971584,"data_stored":282906745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575035,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254992,"kb_used_data":197444,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94116848,"statfs":{"total":96636764160,"available":96375652352,"internally_reserved":0,"allocated":202182656,"data_stored":201117817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":256600,"kb_used_data":199048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94115240,"statfs":{"total":96636764160,"available":96374005760,"internally_reserved":0,"allocated":203825152,"data_stored":202756761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":340372,"kb_used_data":277316,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94031468,"statfs":{"total":96636764160,"available":96288223232,"internally_reserved":0,"allocated":283971584,"data_stored":282906745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317520,"kb_used_data":255812,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94054320,"statfs":{"total":96636764160,"available":96311623680,"internally_reserved":0,"allocated":261951488,"data_stored":260886649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415248,"kb_used_data":347972,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93956592,"statfs":{"total":96636764160,"available":96211550208,"internally_reserved":0,"allocated":356323328,"data_stored":355258489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302744,"kb_used_data":241032,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94069096,"statfs":{"total":96636764160,"available":96326754304,"internally_reserved":0,"allocated":246816768,"data_stored":245748377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370640,"kb_used_data":317252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94001200,"statfs":{"total":96636764160,"available":96257228800,"internally_reserved":0,"allocated":324866048,"data_stored":323801209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:18:29.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:18:29.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.754+0000 7f725be00640 1 Processor -- start 2024-04-25T09:18:29.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.754+0000 7f725be00640 1 -- start start 2024-04-25T09:18:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.756+0000 7f725be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f725c15f450 0x7f725c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.756+0000 7f725be00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c15faa0 con 0x7f725c144c10 2024-04-25T09:18:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.756+0000 7f725be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c15fd80 con 0x7f725c06d900 2024-04-25T09:18:29.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.756+0000 7f725be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c144ff0 con 0x7f725c15f450 2024-04-25T09:18:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.757+0000 7f725aa00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f725c144c10 0x7f725c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50436/0 (socket says 172.21.15.62:50436) 2024-04-25T09:18:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.757+0000 7f725aa00640 1 -- 172.21.15.62:0/1857290928 learned_addr learned my addr 172.21.15.62:0/1857290928 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.757+0000 7f725b400640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f725c15f450 0x7f725c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.757+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394912617 0 0) 0x7f725c15faa0 con 0x7f725c144c10 2024-04-25T09:18:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.757+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7240003520 con 0x7f725c144c10 2024-04-25T09:18:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1354598559 0 0) 0x7f7240003520 con 0x7f725c144c10 2024-04-25T09:18:29.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f725c15f450 msgr2=0x7f725c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f725c15f450 0x7f725c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 >> v1:172.21.15.121:6789/0 conn(0x7f725c06d900 legacy=0x7f725c0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c160060 con 0x7f725c144c10 2024-04-25T09:18:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f724c001670 con 0x7f725c144c10 2024-04-25T09:18:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 >> v1:172.21.15.62:6789/0 conn(0x7f725c144c10 legacy=0x7f725c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f725c15f450 0x7f72400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7240003fb0 0x7f7240004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7240004d40 0x7f72400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c144ff0 con 0x7f7240003fb0 2024-04-25T09:18:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724000bbb0 con 0x7f7240004d40 2024-04-25T09:18:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724000bdb0 con 0x7f725c15f450 2024-04-25T09:18:29.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f7253e00640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7240003fb0 0x7f7240004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725aa00640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f725c15f450 0x7f72400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.758+0000 7f725b400640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7240004d40 0x7f72400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.759+0000 7f7253e00640 1 -- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f725c15f450 msgr2=0x7f72400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:29.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.759+0000 7f7253e00640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f725c15f450 0x7f72400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.759+0000 7f7253e00640 1 -- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7240004d40 msgr2=0x7f72400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:29.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.759+0000 7f7253e00640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7240004d40 0x7f72400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.759+0000 7f7253e00640 1 -- 172.21.15.62:0/1857290928 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240003520 con 0x7f7240003fb0 2024-04-25T09:18:29.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.760+0000 7f7253e00640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7240003fb0 0x7f7240004800 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f7244001750 tx=0x7f7240005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd008794878fed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:29.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.760+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244001370 con 0x7f7240003fb0 2024-04-25T09:18:29.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.760+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72440052d0 con 0x7f7240003fb0 2024-04-25T09:18:29.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.760+0000 7f725a000640 1 -- 172.21.15.62:0/1857290928 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724402abe0 con 0x7f7240003fb0 2024-04-25T09:18:29.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.761+0000 7f725be00640 1 -- 172.21.15.62:0/1857290928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7240003fb0 msgr2=0x7f7240004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.761+0000 7f725be00640 1 --2- 172.21.15.62:0/1857290928 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7240003fb0 0x7f7240004800 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f7244001750 tx=0x7f7240005830 comp rx=0 tx=0).stop 2024-04-25T09:18:29.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.761+0000 7f725b400640 1 -- 172.21.15.62:0/1857290928 reap_dead start 2024-04-25T09:18:29.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.761+0000 7f725be00640 1 -- 172.21.15.62:0/1857290928 shutdown_connections 2024-04-25T09:18:29.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.761+0000 7f725be00640 1 -- 172.21.15.62:0/1857290928 >> 172.21.15.62:0/1857290928 conn(0x7f725c0b14d0 msgr2=0x7f725c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.761+0000 7f725be00640 1 -- 172.21.15.62:0/1857290928 shutdown_connections 2024-04-25T09:18:29.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.761+0000 7f725be00640 1 -- 172.21.15.62:0/1857290928 wait complete. 2024-04-25T09:18:29.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.763+0000 7f725be00640 1 Processor -- start 2024-04-25T09:18:29.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.763+0000 7f725be00640 1 -- start start 2024-04-25T09:18:29.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725be00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7240003fb0 0x7f725c143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725c131e00 0x7f725c141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c1e69f0 0x7f725c1e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725be00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0cf1d0 con 0x7f725c1e69f0 2024-04-25T09:18:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0cf050 con 0x7f725c131e00 2024-04-25T09:18:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725be00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c142880 con 0x7f7240003fb0 2024-04-25T09:18:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c1e69f0 0x7f725c1e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c1e69f0 0x7f725c1e6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48710/0 (socket says 172.21.15.62:48710) 2024-04-25T09:18:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.764+0000 7f725b400640 1 -- 172.21.15.62:0/1658104356 learned_addr learned my addr 172.21.15.62:0/1658104356 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:29.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f7253e00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725c131e00 0x7f725c141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f725aa00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7240003fb0 0x7f725c143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f725b400640 1 -- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7240003fb0 msgr2=0x7f725c143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f725b400640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7240003fb0 0x7f725c143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f725b400640 1 -- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725c131e00 msgr2=0x7f725c141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f725b400640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725c131e00 0x7f725c141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f725b400640 1 -- 172.21.15.62:0/1658104356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c142b60 con 0x7f725c1e69f0 2024-04-25T09:18:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f7253e00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725c131e00 0x7f725c141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:29.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f725b400640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c1e69f0 0x7f725c1e6e70 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f725c070100 tx=0x7f72540115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:29.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f7253400640 1 -- 172.21.15.62:0/1658104356 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254011e40 con 0x7f725c1e69f0 2024-04-25T09:18:29.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f7253400640 1 -- 172.21.15.62:0/1658104356 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725401b070 con 0x7f725c1e69f0 2024-04-25T09:18:29.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c142e40 con 0x7f725c1e69f0 2024-04-25T09:18:29.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.765+0000 7f7253400640 1 -- 172.21.15.62:0/1658104356 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254003990 con 0x7f725c1e69f0 2024-04-25T09:18:29.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.766+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c143400 con 0x7f725c1e69f0 2024-04-25T09:18:29.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.767+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c143600 con 0x7f725c1e69f0 2024-04-25T09:18:29.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.768+0000 7f7253400640 1 -- 172.21.15.62:0/1658104356 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7254005d10 con 0x7f725c1e69f0 2024-04-25T09:18:29.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.769+0000 7f7253400640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f723806b500 0x7f723806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:29.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.769+0000 7f7253400640 1 -- 172.21.15.62:0/1658104356 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f725408a420 con 0x7f725c1e69f0 2024-04-25T09:18:29.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.769+0000 7f725aa00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f723806b500 0x7f723806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:29.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.770+0000 7f725aa00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f723806b500 0x7f723806d9c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f724c001050 tx=0x7f724c001b60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:29.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.773+0000 7f7253400640 1 -- 172.21.15.62:0/1658104356 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f725c143600 con 0x7f725c1e69f0 2024-04-25T09:18:29.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.970+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f725c04ef20 con 0x7f723806b500 2024-04-25T09:18:29.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.973+0000 7f7253400640 1 -- 172.21.15.62:0/1658104356 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13600 (secure 0 0 0) 0x7f725c04ef20 con 0x7f723806b500 2024-04-25T09:18:29.960 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:29.961 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:18:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.975+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f723806b500 msgr2=0x7f723806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.975+0000 7f725be00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f723806b500 0x7f723806d9c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f724c001050 tx=0x7f724c001b60 comp rx=0 tx=0).stop 2024-04-25T09:18:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c1e69f0 msgr2=0x7f725c1e6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c1e69f0 0x7f725c1e6e70 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f725c070100 tx=0x7f72540115a0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 shutdown_connections 2024-04-25T09:18:29.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f723806b500 0x7f723806d9c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c1e69f0 0x7f725c1e6e70 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725c131e00 0x7f725c141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 --2- 172.21.15.62:0/1658104356 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f7240003fb0 0x7f725c143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 >> 172.21.15.62:0/1658104356 conn(0x7f725c0b14d0 msgr2=0x7f725c15bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 shutdown_connections 2024-04-25T09:18:29.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:29.976+0000 7f725be00640 1 -- 172.21.15.62:0/1658104356 wait complete. 2024-04-25T09:18:29.983 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2024-04-25T09:18:28.237669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2588820,"kb_used_data":2113192,"kb_used_omap":12,"kb_used_meta":475507,"kb_avail":752385900,"statfs":{"total":773094113280,"available":770443161600,"internally_reserved":0,"allocated":2163908608,"data_stored":2155382792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":486919728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1078121341,"num_objects":-1047,"num_object_clones":0,"num_object_copies":-2094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1047,"num_whiteouts":0,"num_read":-36,"num_read_kb":-4296,"num_write":-1172,"num_write_kb":-1053012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-23,"num_bytes_recovered":-24117248,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002367"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T09:18:24.562396+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:18:24.562396+0000","last_peered":"2024-04-25T09:18:24.562396+0000","last_clean":"2024-04-25T09:18:24.562396+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:18:24.562396+0000","last_undegraded":"2024-04-25T09:18:24.562396+0000","last_fullsized":"2024-04-25T09:18:24.562396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":330704,"kb_used_data":277316,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94041136,"statfs":{"total":96636764160,"available":96298123264,"internally_reserved":0,"allocated":283971584,"data_stored":282906745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575035,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254992,"kb_used_data":197444,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94116848,"statfs":{"total":96636764160,"available":96375652352,"internally_reserved":0,"allocated":202182656,"data_stored":201117817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":256600,"kb_used_data":199048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94115240,"statfs":{"total":96636764160,"available":96374005760,"internally_reserved":0,"allocated":203825152,"data_stored":202756761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":340372,"kb_used_data":277316,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94031468,"statfs":{"total":96636764160,"available":96288223232,"internally_reserved":0,"allocated":283971584,"data_stored":282906745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317520,"kb_used_data":255812,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94054320,"statfs":{"total":96636764160,"available":96311623680,"internally_reserved":0,"allocated":261951488,"data_stored":260886649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415248,"kb_used_data":347972,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93956592,"statfs":{"total":96636764160,"available":96211550208,"internally_reserved":0,"allocated":356323328,"data_stored":355258489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302744,"kb_used_data":241032,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94069096,"statfs":{"total":96636764160,"available":96326754304,"internally_reserved":0,"allocated":246816768,"data_stored":245748377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370640,"kb_used_data":317252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94001200,"statfs":{"total":96636764160,"available":96257228800,"internally_reserved":0,"allocated":324866048,"data_stored":323801209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:18:29.984 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T09:18:29.984 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:18:30.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.209+0000 7f788ca00640 1 Processor -- start 2024-04-25T09:18:30.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.210+0000 7f788ca00640 1 -- start start 2024-04-25T09:18:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.211+0000 7f788ca00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78880b6a90 0x7f78880d1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.211+0000 7f788ca00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b27a0 0x7f78880d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.211+0000 7f788ca00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78880d3110 con 0x7f78880b56f0 2024-04-25T09:18:30.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.211+0000 7f788ca00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78880cedc0 con 0x7f78880b6a90 2024-04-25T09:18:30.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.211+0000 7f788ca00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78880d33f0 con 0x7f78880b27a0 2024-04-25T09:18:30.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.211+0000 7f787de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78880b6a90 0x7f78880d1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.212+0000 7f787de00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f78880b6a90 0x7f78880d1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48726/0 (socket says 172.21.15.62:48726) 2024-04-25T09:18:30.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.212+0000 7f787de00640 1 -- 172.21.15.62:0/2010265846 learned_addr learned my addr 172.21.15.62:0/2010265846 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:30.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.212+0000 7f7887200640 1 --2- 172.21.15.62:0/2010265846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b27a0 0x7f78880d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.212+0000 7f787de00640 1 -- 172.21.15.62:0/2010265846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b27a0 msgr2=0x7f78880d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:30.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.212+0000 7f787de00640 1 --2- 172.21.15.62:0/2010265846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b27a0 0x7f78880d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.212+0000 7f787de00640 1 -- 172.21.15.62:0/2010265846 >> v1:172.21.15.121:6789/0 conn(0x7f78880b56f0 legacy=0x7f7888160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:30.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.212+0000 7f787de00640 1 -- 172.21.15.62:0/2010265846 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880d36d0 con 0x7f78880b6a90 2024-04-25T09:18:30.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.213+0000 7f787de00640 1 --2- 172.21.15.62:0/2010265846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b6a90 0x7f78880d1fe0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f7870009870 tx=0x7f787002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732906b4484c0c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:30.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.214+0000 7f7885e00640 1 -- 172.21.15.62:0/2010265846 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870037070 con 0x7f78880b6a90 2024-04-25T09:18:30.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.214+0000 7f7885e00640 1 -- 172.21.15.62:0/2010265846 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870032440 con 0x7f78880b6a90 2024-04-25T09:18:30.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.214+0000 7f7885e00640 1 -- 172.21.15.62:0/2010265846 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870032810 con 0x7f78880b6a90 2024-04-25T09:18:30.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.214+0000 7f788ca00640 1 -- 172.21.15.62:0/2010265846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b6a90 msgr2=0x7f78880d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.214+0000 7f788ca00640 1 --2- 172.21.15.62:0/2010265846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b6a90 0x7f78880d1fe0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f7870009870 tx=0x7f787002a210 comp rx=0 tx=0).stop 2024-04-25T09:18:30.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.215+0000 7f788ca00640 1 -- 172.21.15.62:0/2010265846 shutdown_connections 2024-04-25T09:18:30.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.215+0000 7f788ca00640 1 --2- 172.21.15.62:0/2010265846 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b27a0 0x7f78880d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.215+0000 7f788ca00640 1 --2- 172.21.15.62:0/2010265846 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b6a90 0x7f78880d1fe0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.215+0000 7f788ca00640 1 -- 172.21.15.62:0/2010265846 >> 172.21.15.62:0/2010265846 conn(0x7f7888066d10 msgr2=0x7f7888067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:30.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.215+0000 7f788ca00640 1 -- 172.21.15.62:0/2010265846 shutdown_connections 2024-04-25T09:18:30.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.215+0000 7f788ca00640 1 -- 172.21.15.62:0/2010265846 wait complete. 2024-04-25T09:18:30.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.216+0000 7f788ca00640 1 Processor -- start 2024-04-25T09:18:30.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.217+0000 7f788ca00640 1 -- start start 2024-04-25T09:18:30.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.217+0000 7f788ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b27a0 0x7f78880c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.217+0000 7f788ca00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b56f0 0x7f7888157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.217+0000 7f788ca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b6a90 0x7f7888159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.217+0000 7f788ca00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880b2ee0 con 0x7f78880b27a0 2024-04-25T09:18:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.217+0000 7f788ca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880b2d60 con 0x7f78880b6a90 2024-04-25T09:18:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.217+0000 7f788ca00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78880b5ad0 con 0x7f78880b56f0 2024-04-25T09:18:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7886800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b27a0 0x7f78880c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7886800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b27a0 0x7f78880c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48734/0 (socket says 172.21.15.62:48734) 2024-04-25T09:18:30.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7886800640 1 -- 172.21.15.62:0/1521380123 learned_addr learned my addr 172.21.15.62:0/1521380123 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:30.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f787de00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b56f0 0x7f7888157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7886800640 1 -- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b56f0 msgr2=0x7f7888157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7886800640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b56f0 0x7f7888157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7887200640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b6a90 0x7f7888159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7886800640 1 -- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b6a90 msgr2=0x7f7888159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7886800640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b6a90 0x7f7888159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7886800640 1 -- 172.21.15.62:0/1521380123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880c85d0 con 0x7f78880b27a0 2024-04-25T09:18:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f787de00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b56f0 0x7f7888157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.218+0000 7f7887200640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b6a90 0x7f7888159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.219+0000 7f7886800640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b27a0 0x7f78880c7f40 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f787800c430 tx=0x7f787800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.219+0000 7f787f400640 1 -- 172.21.15.62:0/1521380123 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878011790 con 0x7f78880b27a0 2024-04-25T09:18:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.219+0000 7f787f400640 1 -- 172.21.15.62:0/1521380123 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7878017070 con 0x7f78880b27a0 2024-04-25T09:18:30.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.219+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78880c8b90 con 0x7f78880b27a0 2024-04-25T09:18:30.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.219+0000 7f787f400640 1 -- 172.21.15.62:0/1521380123 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878011bb0 con 0x7f78880b27a0 2024-04-25T09:18:30.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.219+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78881591a0 con 0x7f78880b27a0 2024-04-25T09:18:30.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.221+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788804ef20 con 0x7f78880b27a0 2024-04-25T09:18:30.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.222+0000 7f787f400640 1 -- 172.21.15.62:0/1521380123 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f787801c460 con 0x7f78880b27a0 2024-04-25T09:18:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.223+0000 7f787f400640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f786406b4d0 0x7f786406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.223+0000 7f787f400640 1 -- 172.21.15.62:0/1521380123 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7878089d10 con 0x7f78880b27a0 2024-04-25T09:18:30.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.224+0000 7f787de00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f786406b4d0 0x7f786406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.225+0000 7f787de00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f786406b4d0 0x7f786406d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7870034000 tx=0x7f787002e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:30.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.228+0000 7f787f400640 1 -- 172.21.15.62:0/1521380123 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7878058e90 con 0x7f78880b27a0 2024-04-25T09:18:30.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.428+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f78880ce580 con 0x7f786406b4d0 2024-04-25T09:18:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.431+0000 7f787f400640 1 -- 172.21.15.62:0/1521380123 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13600 (secure 0 0 0) 0x7f78880ce580 con 0x7f786406b4d0 2024-04-25T09:18:30.418 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:30.418 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:18:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.433+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f786406b4d0 msgr2=0x7f786406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.433+0000 7f788ca00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f786406b4d0 0x7f786406d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7870034000 tx=0x7f787002e000 comp rx=0 tx=0).stop 2024-04-25T09:18:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.433+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b27a0 msgr2=0x7f78880c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.433+0000 7f788ca00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b27a0 0x7f78880c7f40 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f787800c430 tx=0x7f787800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.434+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 shutdown_connections 2024-04-25T09:18:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.434+0000 7f788ca00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f786406b4d0 0x7f786406d990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.434+0000 7f788ca00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f78880b6a90 0x7f7888159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.434+0000 7f788ca00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f78880b56f0 0x7f7888157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.434+0000 7f788ca00640 1 --2- 172.21.15.62:0/1521380123 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f78880b27a0 0x7f78880c7f40 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.434+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 >> 172.21.15.62:0/1521380123 conn(0x7f7888066d10 msgr2=0x7f788815f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.434+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 shutdown_connections 2024-04-25T09:18:30.422 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.434+0000 7f788ca00640 1 -- 172.21.15.62:0/1521380123 wait complete. 2024-04-25T09:18:30.440 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":608,"stamp":"2024-04-25T09:18:30.238289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2588820,"kb_used_data":2113192,"kb_used_omap":12,"kb_used_meta":475507,"kb_avail":752385900,"statfs":{"total":773094113280,"available":770443161600,"internally_reserved":0,"allocated":2163908608,"data_stored":2155382792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":486919728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1078121341,"num_objects":-1047,"num_object_clones":0,"num_object_copies":-2094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1047,"num_whiteouts":0,"num_read":-36,"num_read_kb":-4296,"num_write":-1172,"num_write_kb":-1053012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-23,"num_bytes_recovered":-24117248,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.870882"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-25T09:18:24.562396+0000","last_change":"2024-04-25T09:03:07.235569+0000","last_active":"2024-04-25T09:18:24.562396+0000","last_peered":"2024-04-25T09:18:24.562396+0000","last_clean":"2024-04-25T09:18:24.562396+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:18:24.562396+0000","last_undegraded":"2024-04-25T09:18:24.562396+0000","last_fullsized":"2024-04-25T09:18:24.562396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:06.209698+0000","last_clean_scrub_stamp":"2024-04-25T09:03:06.209698+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:10.680986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":330704,"kb_used_data":277316,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94041136,"statfs":{"total":96636764160,"available":96298123264,"internally_reserved":0,"allocated":283971584,"data_stored":282906745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575035,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254992,"kb_used_data":197444,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94116848,"statfs":{"total":96636764160,"available":96375652352,"internally_reserved":0,"allocated":202182656,"data_stored":201117817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":256600,"kb_used_data":199048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94115240,"statfs":{"total":96636764160,"available":96374005760,"internally_reserved":0,"allocated":203825152,"data_stored":202756761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":340372,"kb_used_data":277316,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94031468,"statfs":{"total":96636764160,"available":96288223232,"internally_reserved":0,"allocated":283971584,"data_stored":282906745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317520,"kb_used_data":255812,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94054320,"statfs":{"total":96636764160,"available":96311623680,"internally_reserved":0,"allocated":261951488,"data_stored":260886649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415248,"kb_used_data":347972,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93956592,"statfs":{"total":96636764160,"available":96211550208,"internally_reserved":0,"allocated":356323328,"data_stored":355258489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302744,"kb_used_data":241032,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94069096,"statfs":{"total":96636764160,"available":96326754304,"internally_reserved":0,"allocated":246816768,"data_stored":245748377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":370640,"kb_used_data":317252,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94001200,"statfs":{"total":96636764160,"available":96257228800,"internally_reserved":0,"allocated":324866048,"data_stored":323801209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:18:30.440 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:18:30.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.658+0000 7fb67d800640 1 Processor -- start 2024-04-25T09:18:30.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.658+0000 7fb67d800640 1 -- start start 2024-04-25T09:18:30.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.660+0000 7fb67d800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb678144c10 0x7fb67815f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.660+0000 7fb67d800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb67806d900 0x7fb6780b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.660+0000 7fb67d800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67815faa0 con 0x7fb6780cee40 2024-04-25T09:18:30.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.660+0000 7fb67d800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb678144ff0 con 0x7fb67806d900 2024-04-25T09:18:30.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.660+0000 7fb67d800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb678160340 con 0x7fb678144c10 2024-04-25T09:18:30.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.660+0000 7fb677e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb678144c10 0x7fb67815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.660+0000 7fb677e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb678144c10 0x7fb67815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3301/0 says I am v2:172.21.15.62:45184/0 (socket says 172.21.15.62:45184) 2024-04-25T09:18:30.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.660+0000 7fb677e00640 1 -- 172.21.15.62:0/3923910177 learned_addr learned my addr 172.21.15.62:0/3923910177 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:30.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.661+0000 7fb67ce00640 1 --2- 172.21.15.62:0/3923910177 >> v2:172.21.15.121:3300/0 conn(0x7fb67806d900 0x7fb6780b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.661+0000 7fb67ce00640 1 -- 172.21.15.62:0/3923910177 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb678144c10 msgr2=0x7fb67815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:30.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.661+0000 7fb67ce00640 1 --2- 172.21.15.62:0/3923910177 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb678144c10 0x7fb67815f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.661+0000 7fb67ce00640 1 -- 172.21.15.62:0/3923910177 >> v1:172.21.15.121:6789/0 conn(0x7fb6780cee40 legacy=0x7fb67806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.661+0000 7fb67ce00640 1 -- 172.21.15.62:0/3923910177 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67815fd80 con 0x7fb67806d900 2024-04-25T09:18:30.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.663+0000 7fb67ce00640 1 --2- 172.21.15.62:0/3923910177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67806d900 0x7fb6780b4650 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb670009dc0 tx=0x7fb67002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4433500cf8cd1750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:30.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb677400640 1 -- 172.21.15.62:0/3923910177 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670007740 con 0x7fb67806d900 2024-04-25T09:18:30.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb677400640 1 -- 172.21.15.62:0/3923910177 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6700078e0 con 0x7fb67806d900 2024-04-25T09:18:30.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb677400640 1 -- 172.21.15.62:0/3923910177 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670007cb0 con 0x7fb67806d900 2024-04-25T09:18:30.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb67d800640 1 -- 172.21.15.62:0/3923910177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67806d900 msgr2=0x7fb6780b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb67d800640 1 --2- 172.21.15.62:0/3923910177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67806d900 0x7fb6780b4650 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb670009dc0 tx=0x7fb67002a5f0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb67d800640 1 -- 172.21.15.62:0/3923910177 shutdown_connections 2024-04-25T09:18:30.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb67d800640 1 --2- 172.21.15.62:0/3923910177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb67806d900 0x7fb6780b4650 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb67d800640 1 --2- 172.21.15.62:0/3923910177 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb678144c10 0x7fb67815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.664+0000 7fb67d800640 1 -- 172.21.15.62:0/3923910177 >> 172.21.15.62:0/3923910177 conn(0x7fb6780b14d0 msgr2=0x7fb6780b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:30.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.665+0000 7fb67d800640 1 -- 172.21.15.62:0/3923910177 shutdown_connections 2024-04-25T09:18:30.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.665+0000 7fb67d800640 1 -- 172.21.15.62:0/3923910177 wait complete. 2024-04-25T09:18:30.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.666+0000 7fb67d800640 1 Processor -- start 2024-04-25T09:18:30.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.666+0000 7fb67d800640 1 -- start start 2024-04-25T09:18:30.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.667+0000 7fb67d800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67806d900 0x7fb678132750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.667+0000 7fb67d800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6780cee40 0x7fb67815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.667+0000 7fb67d800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678144c10 0x7fb678141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.667+0000 7fb67d800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780cf2a0 con 0x7fb678144c10 2024-04-25T09:18:30.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67d800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780b41c0 con 0x7fb6780cee40 2024-04-25T09:18:30.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67d800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780b4b90 con 0x7fb67806d900 2024-04-25T09:18:30.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678144c10 0x7fb678141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67ce00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678144c10 0x7fb678141820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48738/0 (socket says 172.21.15.62:48738) 2024-04-25T09:18:30.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67ce00640 1 -- 172.21.15.62:0/2100196819 learned_addr learned my addr 172.21.15.62:0/2100196819 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:30.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb677e00640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67806d900 0x7fb678132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb66f400640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6780cee40 0x7fb67815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67ce00640 1 -- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67806d900 msgr2=0x7fb678132750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67ce00640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67806d900 0x7fb678132750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67ce00640 1 -- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6780cee40 msgr2=0x7fb67815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67ce00640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6780cee40 0x7fb67815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.668+0000 7fb67ce00640 1 -- 172.21.15.62:0/2100196819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780cc740 con 0x7fb678144c10 2024-04-25T09:18:30.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.669+0000 7fb67ce00640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678144c10 0x7fb678141820 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fb670009ee0 tx=0x7fb670004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:30.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.669+0000 7fb674c00640 1 -- 172.21.15.62:0/2100196819 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670040070 con 0x7fb678144c10 2024-04-25T09:18:30.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.669+0000 7fb674c00640 1 -- 172.21.15.62:0/2100196819 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb670031bc0 con 0x7fb678144c10 2024-04-25T09:18:30.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.669+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6780cca20 con 0x7fb678144c10 2024-04-25T09:18:30.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.669+0000 7fb674c00640 1 -- 172.21.15.62:0/2100196819 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670009450 con 0x7fb678144c10 2024-04-25T09:18:30.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.669+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6780ccfe0 con 0x7fb678144c10 2024-04-25T09:18:30.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.670+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67804ef20 con 0x7fb678144c10 2024-04-25T09:18:30.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.671+0000 7fb674c00640 1 -- 172.21.15.62:0/2100196819 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb670039040 con 0x7fb678144c10 2024-04-25T09:18:30.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.672+0000 7fb674c00640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb65406b4d0 0x7fb65406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:30.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.673+0000 7fb674c00640 1 -- 172.21.15.62:0/2100196819 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6700ab890 con 0x7fb678144c10 2024-04-25T09:18:30.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.673+0000 7fb677e00640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb65406b4d0 0x7fb65406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:30.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.674+0000 7fb677e00640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb65406b4d0 0x7fb65406d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb668009ac0 tx=0x7fb668008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:30.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.677+0000 7fb674c00640 1 -- 172.21.15.62:0/2100196819 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb670043360 con 0x7fb678144c10 2024-04-25T09:18:30.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.878+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb67815cf20 con 0x7fb678144c10 2024-04-25T09:18:30.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.880+0000 7fb674c00640 1 -- 172.21.15.62:0/2100196819 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+13018 (secure 0 0 0) 0x7fb67007a8e0 con 0x7fb678144c10 2024-04-25T09:18:30.867 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:30.867 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":153,"fsid":"9e112875-3fc2-4963-90e0-2b56d5516d5c","created":"2024-04-25T09:02:52.707448+0000","modified":"2024-04-25T09:18:25.499216+0000","last_up_change":"2024-04-25T09:03:05.205799+0000","last_in_change":"2024-04-25T09:02:55.931188+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:03:05.947157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d078b39-cff6-45ae-8b9c-9224dd5edb73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6801","nonce":565545488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6803","nonce":565545488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6807","nonce":565545488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":565545488},{"type":"v1","addr":"172.21.15.62:6805","nonce":565545488}]},"public_addr":"172.21.15.62:6801/565545488","cluster_addr":"172.21.15.62:6803/565545488","heartbeat_back_addr":"172.21.15.62:6807/565545488","heartbeat_front_addr":"172.21.15.62:6805/565545488","state":["exists","up"]},{"osd":1,"uuid":"232e7cd0-6bc1-4732-9f18-6cd44ea4338b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6809","nonce":2146620681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6811","nonce":2146620681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6815","nonce":2146620681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2146620681},{"type":"v1","addr":"172.21.15.62:6813","nonce":2146620681}]},"public_addr":"172.21.15.62:6809/2146620681","cluster_addr":"172.21.15.62:6811/2146620681","heartbeat_back_addr":"172.21.15.62:6815/2146620681","heartbeat_front_addr":"172.21.15.62:6813/2146620681","state":["exists","up"]},{"osd":2,"uuid":"34dcda9a-65b6-4445-b6cb-489bdb58b737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6818","nonce":385571314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6819","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6821","nonce":385571314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6828","nonce":385571314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6823","nonce":385571314},{"type":"v1","addr":"172.21.15.62:6825","nonce":385571314}]},"public_addr":"172.21.15.62:6818/385571314","cluster_addr":"172.21.15.62:6821/385571314","heartbeat_back_addr":"172.21.15.62:6828/385571314","heartbeat_front_addr":"172.21.15.62:6825/385571314","state":["exists","up"]},{"osd":3,"uuid":"14caaf78-6122-44ff-9def-2f1db5cd9eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6817","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6820","nonce":3104548384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6824","nonce":3104548384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6831","nonce":3104548384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6827","nonce":3104548384},{"type":"v1","addr":"172.21.15.62:6829","nonce":3104548384}]},"public_addr":"172.21.15.62:6820/3104548384","cluster_addr":"172.21.15.62:6824/3104548384","heartbeat_back_addr":"172.21.15.62:6831/3104548384","heartbeat_front_addr":"172.21.15.62:6829/3104548384","state":["exists","up"]},{"osd":4,"uuid":"73c23f2f-c64e-4c72-8c29-3aece151a25f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6809","nonce":3370771089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6811","nonce":3370771089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6815","nonce":3370771089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":3370771089},{"type":"v1","addr":"172.21.15.121:6813","nonce":3370771089}]},"public_addr":"172.21.15.121:6809/3370771089","cluster_addr":"172.21.15.121:6811/3370771089","heartbeat_back_addr":"172.21.15.121:6815/3370771089","heartbeat_front_addr":"172.21.15.121:6813/3370771089","state":["exists","up"]},{"osd":5,"uuid":"28077425-23cd-49d3-84b5-e383d0aa1528","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6817","nonce":3527509140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6819","nonce":3527509140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6828","nonce":3527509140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3527509140},{"type":"v1","addr":"172.21.15.121:6824","nonce":3527509140}]},"public_addr":"172.21.15.121:6817/3527509140","cluster_addr":"172.21.15.121:6819/3527509140","heartbeat_back_addr":"172.21.15.121:6828/3527509140","heartbeat_front_addr":"172.21.15.121:6824/3527509140","state":["exists","up"]},{"osd":6,"uuid":"916f3e21-559d-4401-bad6-f2b7713f7425","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6801","nonce":3195802018}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6803","nonce":3195802018}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6807","nonce":3195802018}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3195802018},{"type":"v1","addr":"172.21.15.121:6805","nonce":3195802018}]},"public_addr":"172.21.15.121:6801/3195802018","cluster_addr":"172.21.15.121:6803/3195802018","heartbeat_back_addr":"172.21.15.121:6807/3195802018","heartbeat_front_addr":"172.21.15.121:6805/3195802018","state":["exists","up"]},{"osd":7,"uuid":"70ddc55f-d707-4e42-912b-3b17d17aabb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6821","nonce":1260954068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6823","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6825","nonce":1260954068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6831","nonce":1260954068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6827","nonce":1260954068},{"type":"v1","addr":"172.21.15.121:6829","nonce":1260954068}]},"public_addr":"172.21.15.121:6821/1260954068","cluster_addr":"172.21.15.121:6825/1260954068","heartbeat_back_addr":"172.21.15.121:6831/1260954068","heartbeat_front_addr":"172.21.15.121:6829/1260954068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.575846+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642821+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.642681+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.345714+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.327226+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.387900+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.270315+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:03:01.290552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6835/2793862963":"2024-04-26T09:18:22.361002+0000","172.21.15.62:6834/2793862963":"2024-04-26T09:18:22.361002+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:18:30.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.882+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb65406b4d0 msgr2=0x7fb65406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.882+0000 7fb67d800640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb65406b4d0 0x7fb65406d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb668009ac0 tx=0x7fb668008040 comp rx=0 tx=0).stop 2024-04-25T09:18:30.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.882+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678144c10 msgr2=0x7fb678141820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:30.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.882+0000 7fb67d800640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678144c10 0x7fb678141820 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fb670009ee0 tx=0x7fb670004a40 comp rx=0 tx=0).stop 2024-04-25T09:18:30.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.883+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 shutdown_connections 2024-04-25T09:18:30.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.883+0000 7fb67d800640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb65406b4d0 0x7fb65406d990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.883+0000 7fb67d800640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb678144c10 0x7fb678141820 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.883+0000 7fb67d800640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb6780cee40 0x7fb67815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.883+0000 7fb67d800640 1 --2- 172.21.15.62:0/2100196819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb67806d900 0x7fb678132750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:30.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.883+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 >> 172.21.15.62:0/2100196819 conn(0x7fb6780b14d0 msgr2=0x7fb67815bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:30.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.883+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 shutdown_connections 2024-04-25T09:18:30.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:30.883+0000 7fb67d800640 1 -- 172.21.15.62:0/2100196819 wait complete. 2024-04-25T09:18:31.892 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T09:18:31.892 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:18:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.133+0000 7fe0e5c00640 1 Processor -- start 2024-04-25T09:18:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.134+0000 7fe0e5c00640 1 -- start start 2024-04-25T09:18:32.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.135+0000 7fe0e5c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0e00b5c70 0x7fe0e00d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.135+0000 7fe0e5c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe0e0161650 0x7fe0e00d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.135+0000 7fe0e5c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e00d1480 con 0x7fe0e0161270 2024-04-25T09:18:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.135+0000 7fe0e5c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e00d3900 con 0x7fe0e0161650 2024-04-25T09:18:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.135+0000 7fe0e5c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00d11a0 con 0x7fe0e00b5c70 2024-04-25T09:18:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0dfe00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe0e0161270 0x7fe0e00b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50442/0 (socket says 172.21.15.62:50442) 2024-04-25T09:18:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0dfe00640 1 -- 172.21.15.62:0/2371645494 learned_addr learned my addr 172.21.15.62:0/2371645494 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0e5200640 1 --2- 172.21.15.62:0/2371645494 >> v2:172.21.15.62:3300/0 conn(0x7fe0e0161650 0x7fe0e00d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0d7400640 1 --2- 172.21.15.62:0/2371645494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0e00b5c70 0x7fe0e00d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0e5200640 1 -- 172.21.15.62:0/2371645494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0e00b5c70 msgr2=0x7fe0e00d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0e5200640 1 --2- 172.21.15.62:0/2371645494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0e00b5c70 0x7fe0e00d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0e5200640 1 -- 172.21.15.62:0/2371645494 >> v1:172.21.15.62:6789/0 conn(0x7fe0e0161270 legacy=0x7fe0e00b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0e5200640 1 -- 172.21.15.62:0/2371645494 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e00cee80 con 0x7fe0e0161650 2024-04-25T09:18:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.136+0000 7fe0df400640 1 -- 172.21.15.62:0/2371645494 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072541376 0 0) 0x7fe0e00d1480 con 0x7fe0e0161270 2024-04-25T09:18:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.137+0000 7fe0e5200640 1 --2- 172.21.15.62:0/2371645494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e0161650 0x7fe0e00d1da0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009d90 tx=0x7fe0d802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac51492b67f9904 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0df400640 1 -- 172.21.15.62:0/2371645494 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8007740 con 0x7fe0e0161650 2024-04-25T09:18:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0df400640 1 -- 172.21.15.62:0/2371645494 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d80078e0 con 0x7fe0e0161650 2024-04-25T09:18:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0df400640 1 -- 172.21.15.62:0/2371645494 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8007cb0 con 0x7fe0e0161650 2024-04-25T09:18:32.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2371645494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e0161650 msgr2=0x7fe0e00d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0e5c00640 1 --2- 172.21.15.62:0/2371645494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e0161650 0x7fe0e00d1da0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009d90 tx=0x7fe0d802a530 comp rx=0 tx=0).stop 2024-04-25T09:18:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2371645494 shutdown_connections 2024-04-25T09:18:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0e5c00640 1 --2- 172.21.15.62:0/2371645494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e0161650 0x7fe0e00d1da0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0e5c00640 1 --2- 172.21.15.62:0/2371645494 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0e00b5c70 0x7fe0e00d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2371645494 >> 172.21.15.62:0/2371645494 conn(0x7fe0e0066d10 msgr2=0x7fe0e0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2371645494 shutdown_connections 2024-04-25T09:18:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.138+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2371645494 wait complete. 2024-04-25T09:18:32.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.140+0000 7fe0e5c00640 1 Processor -- start 2024-04-25T09:18:32.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.140+0000 7fe0e5c00640 1 -- start start 2024-04-25T09:18:32.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.141+0000 7fe0e5c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e00b5c70 0x7fe0e014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.141+0000 7fe0e5c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0e0161270 0x7fe0e00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.141+0000 7fe0e5c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0161650 0x7fe0e0156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0e5c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00b2ed0 con 0x7fe0e00b5c70 2024-04-25T09:18:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0e5c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00d3590 con 0x7fe0e0161650 2024-04-25T09:18:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0e5c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e00b3050 con 0x7fe0e0161270 2024-04-25T09:18:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0dfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e00b5c70 0x7fe0e014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0dfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e00b5c70 0x7fe0e014c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48756/0 (socket says 172.21.15.62:48756) 2024-04-25T09:18:32.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0dfe00640 1 -- 172.21.15.62:0/2774954558 learned_addr learned my addr 172.21.15.62:0/2774954558 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:32.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0dfe00640 1 -- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0e0161270 msgr2=0x7fe0e00cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0dfe00640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe0e0161270 0x7fe0e00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0dfe00640 1 -- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0161650 msgr2=0x7fe0e0156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0dfe00640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0e0161650 0x7fe0e0156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.142+0000 7fe0dfe00640 1 -- 172.21.15.62:0/2774954558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e0155930 con 0x7fe0e00b5c70 2024-04-25T09:18:32.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.143+0000 7fe0dfe00640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e00b5c70 0x7fe0e014c120 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000c430 tx=0x7fe0d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.143+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d00074e0 con 0x7fe0e00b5c70 2024-04-25T09:18:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.143+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d00047e0 con 0x7fe0e00b5c70 2024-04-25T09:18:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.143+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e014c660 con 0x7fe0e00b5c70 2024-04-25T09:18:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.143+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0004bd0 con 0x7fe0e00b5c70 2024-04-25T09:18:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.143+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e0155e80 con 0x7fe0e00b5c70 2024-04-25T09:18:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.144+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0e0074790 con 0x7fe0e00b5c70 2024-04-25T09:18:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.145+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0d0011070 con 0x7fe0e00b5c70 2024-04-25T09:18:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.146+0000 7fe0dcc00640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.146+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0d0089b40 con 0x7fe0e00b5c70 2024-04-25T09:18:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.147+0000 7fe0dcc00640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fe0bc071390 0x7fe0bc0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.147+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 --> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0d0009e50 con 0x7fe0bc071390 2024-04-25T09:18:32.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.147+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0d008e2a0 con 0x7fe0e00b5c70 2024-04-25T09:18:32.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.147+0000 7fe0e5200640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fe0bc071390 0x7fe0bc0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.147+0000 7fe0d7400640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.148+0000 7fe0e5200640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fe0bc071390 0x7fe0bc0737b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.148+0000 7fe0d7400640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8006fd0 tx=0x7fe0c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.148+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 <== osd.0 v2:172.21.15.62:6800/565545488 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe0d0009e50 con 0x7fe0bc071390 2024-04-25T09:18:32.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.163+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 --> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe0e0072910 con 0x7fe0bc071390 2024-04-25T09:18:32.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.163+0000 7fe0dcc00640 1 -- 172.21.15.62:0/2774954558 <== osd.0 v2:172.21.15.62:6800/565545488 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe0e0072910 con 0x7fe0bc071390 2024-04-25T09:18:32.151 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:32.151 INFO:teuthology.orchestra.run.smithi062.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:18:32.151 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fe0bc071390 msgr2=0x7fe0bc0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5c00640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.62:6800/565545488,v1:172.21.15.62:6801/565545488] conn(0x7fe0bc071390 0x7fe0bc0737b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0bc06b4d0 msgr2=0x7fe0bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5c00640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8006fd0 tx=0x7fe0c8008040 comp rx=0 tx=0).stop 2024-04-25T09:18:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e00b5c70 msgr2=0x7fe0e014c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5c00640 1 --2- 172.21.15.62:0/2774954558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe0e00b5c70 0x7fe0e014c120 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000c430 tx=0x7fe0d000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5200640 1 -- 172.21.15.62:0/2774954558 reap_dead start 2024-04-25T09:18:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 shutdown_connections 2024-04-25T09:18:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.164+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 >> 172.21.15.62:0/2774954558 conn(0x7fe0e0066d10 msgr2=0x7fe0e015b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.165+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 shutdown_connections 2024-04-25T09:18:32.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.165+0000 7fe0e5c00640 1 -- 172.21.15.62:0/2774954558 wait complete. 2024-04-25T09:18:32.172 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T09:18:32.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.402+0000 7fe642400640 1 Processor -- start 2024-04-25T09:18:32.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.402+0000 7fe642400640 1 -- start start 2024-04-25T09:18:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe642400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe63c161230 0x7fe63c0b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe642400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe63c0cea70 con 0x7fe63c0d0fb0 2024-04-25T09:18:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe642400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe63c0ced50 con 0x7fe63c0b5c30 2024-04-25T09:18:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe642400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c0d2040 con 0x7fe63c161230 2024-04-25T09:18:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe641000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe63c161230 0x7fe63c0b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe641000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe63c161230 0x7fe63c0b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48762/0 (socket says 172.21.15.62:48762) 2024-04-25T09:18:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe641000640 1 -- 172.21.15.62:0/1331826702 learned_addr learned my addr 172.21.15.62:0/1331826702 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:32.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe63be00640 1 --1- 172.21.15.62:0/1331826702 >> v1:172.21.15.62:6789/0 conn(0x7fe63c0d0fb0 0x7fe63c0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50452/0 (socket says 172.21.15.62:50452) 2024-04-25T09:18:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.404+0000 7fe641000640 1 -- 172.21.15.62:0/1331826702 >> v1:172.21.15.121:6789/0 conn(0x7fe63c0b5c30 legacy=0x7fe63c161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.405+0000 7fe641000640 1 -- 172.21.15.62:0/1331826702 >> v1:172.21.15.62:6789/0 conn(0x7fe63c0d0fb0 legacy=0x7fe63c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.405+0000 7fe641000640 1 -- 172.21.15.62:0/1331826702 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0cf030 con 0x7fe63c161230 2024-04-25T09:18:32.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.405+0000 7fe63b400640 1 -- 172.21.15.62:0/1331826702 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866343555 0 0) 0x7fe63c0cea70 con 0x7fe63c0d0fb0 2024-04-25T09:18:32.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.406+0000 7fe641000640 1 --2- 172.21.15.62:0/1331826702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c161230 0x7fe63c0b6c40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009870 tx=0x7fe62c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c94a03851286c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.406+0000 7fe63b400640 1 -- 172.21.15.62:0/1331826702 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c037070 con 0x7fe63c161230 2024-04-25T09:18:32.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.406+0000 7fe63b400640 1 -- 172.21.15.62:0/1331826702 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c032440 con 0x7fe63c161230 2024-04-25T09:18:32.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.406+0000 7fe63b400640 1 -- 172.21.15.62:0/1331826702 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c032810 con 0x7fe63c161230 2024-04-25T09:18:32.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.407+0000 7fe642400640 1 -- 172.21.15.62:0/1331826702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c161230 msgr2=0x7fe63c0b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.407+0000 7fe642400640 1 --2- 172.21.15.62:0/1331826702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c161230 0x7fe63c0b6c40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009870 tx=0x7fe62c02a2a0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.407+0000 7fe642400640 1 -- 172.21.15.62:0/1331826702 shutdown_connections 2024-04-25T09:18:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.407+0000 7fe642400640 1 --2- 172.21.15.62:0/1331826702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c161230 0x7fe63c0b6c40 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.407+0000 7fe642400640 1 -- 172.21.15.62:0/1331826702 >> 172.21.15.62:0/1331826702 conn(0x7fe63c066d10 msgr2=0x7fe63c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.407+0000 7fe642400640 1 -- 172.21.15.62:0/1331826702 shutdown_connections 2024-04-25T09:18:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.407+0000 7fe642400640 1 -- 172.21.15.62:0/1331826702 wait complete. 2024-04-25T09:18:32.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.409+0000 7fe642400640 1 Processor -- start 2024-04-25T09:18:32.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.409+0000 7fe642400640 1 -- start start 2024-04-25T09:18:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe642400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c0b5c30 0x7fe63c0d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe642400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c0d0fb0 0x7fe63c0ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe642400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c161230 0x7fe63c0abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe642400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c161610 con 0x7fe63c0b5c30 2024-04-25T09:18:32.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe642400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0ca8d0 con 0x7fe63c0d0fb0 2024-04-25T09:18:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe642400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c161790 con 0x7fe63c161230 2024-04-25T09:18:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe641000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c0b5c30 0x7fe63c0d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe641000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c0b5c30 0x7fe63c0d3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48764/0 (socket says 172.21.15.62:48764) 2024-04-25T09:18:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe641000640 1 -- 172.21.15.62:0/965915205 learned_addr learned my addr 172.21.15.62:0/965915205 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe641000640 1 -- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c161230 msgr2=0x7fe63c0abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe641000640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c161230 0x7fe63c0abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe641000640 1 -- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c0d0fb0 msgr2=0x7fe63c0ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe641000640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c0d0fb0 0x7fe63c0ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.410+0000 7fe641000640 1 -- 172.21.15.62:0/965915205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0c9550 con 0x7fe63c0b5c30 2024-04-25T09:18:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.411+0000 7fe641000640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c0b5c30 0x7fe63c0d3d10 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009870 tx=0x7fe62c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.411+0000 7fe638c00640 1 -- 172.21.15.62:0/965915205 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c041020 con 0x7fe63c0b5c30 2024-04-25T09:18:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.411+0000 7fe638c00640 1 -- 172.21.15.62:0/965915205 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c030970 con 0x7fe63c0b5c30 2024-04-25T09:18:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.411+0000 7fe638c00640 1 -- 172.21.15.62:0/965915205 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c037040 con 0x7fe63c0b5c30 2024-04-25T09:18:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.411+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c0c9830 con 0x7fe63c0b5c30 2024-04-25T09:18:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.411+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c0c9df0 con 0x7fe63c0b5c30 2024-04-25T09:18:32.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.412+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c04ef20 con 0x7fe63c0b5c30 2024-04-25T09:18:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.413+0000 7fe638c00640 1 -- 172.21.15.62:0/965915205 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe62c044020 con 0x7fe63c0b5c30 2024-04-25T09:18:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.414+0000 7fe638c00640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe61c06b4d0 0x7fe61c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.414+0000 7fe638c00640 1 -- 172.21.15.62:0/965915205 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe62c0aa9a0 con 0x7fe63c0b5c30 2024-04-25T09:18:32.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.415+0000 7fe63be00640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe61c06b4d0 0x7fe61c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.416+0000 7fe63be00640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe61c06b4d0 0x7fe61c06d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe628001560 tx=0x7fe628007ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.419+0000 7fe638c00640 1 -- 172.21.15.62:0/965915205 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe62c04b030 con 0x7fe63c0b5c30 2024-04-25T09:18:32.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.619+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe63c0ce640 con 0x7fe61c06b4d0 2024-04-25T09:18:32.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.620+0000 7fe638c00640 1 -- 172.21.15.62:0/965915205 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe63c0ce640 con 0x7fe61c06b4d0 2024-04-25T09:18:32.607 INFO:teuthology.orchestra.run.smithi062.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T09:18:32.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.623+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe61c06b4d0 msgr2=0x7fe61c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.623+0000 7fe642400640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe61c06b4d0 0x7fe61c06d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe628001560 tx=0x7fe628007ca0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.623+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c0b5c30 msgr2=0x7fe63c0d3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.623+0000 7fe642400640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c0b5c30 0x7fe63c0d3d10 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009870 tx=0x7fe62c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.623+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 shutdown_connections 2024-04-25T09:18:32.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.624+0000 7fe642400640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fe61c06b4d0 0x7fe61c06d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.624+0000 7fe642400640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fe63c161230 0x7fe63c0abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.624+0000 7fe642400640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe63c0d0fb0 0x7fe63c0ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.624+0000 7fe642400640 1 --2- 172.21.15.62:0/965915205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe63c0b5c30 0x7fe63c0d3d10 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.624+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 >> 172.21.15.62:0/965915205 conn(0x7fe63c066d10 msgr2=0x7fe63c0b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:32.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.624+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 shutdown_connections 2024-04-25T09:18:32.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.624+0000 7fe642400640 1 -- 172.21.15.62:0/965915205 wait complete. 2024-04-25T09:18:32.631 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T09:18:32.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:18:32.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.864+0000 7f3811400640 1 Processor -- start 2024-04-25T09:18:32.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.864+0000 7f3811400640 1 -- start start 2024-04-25T09:18:32.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.866+0000 7f3811400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f380c15df20 0x7f380c161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.866+0000 7f3811400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f380c0b9840 0x7f380c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.866+0000 7f3811400640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c15e300 con 0x7f380c0ceeb0 2024-04-25T09:18:32.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.866+0000 7f3811400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380c0d2110 con 0x7f380c15df20 2024-04-25T09:18:32.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.866+0000 7f3811400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380c1628d0 con 0x7f380c0b9840 2024-04-25T09:18:32.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.866+0000 7f380b200640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f380c0ceeb0 0x7f380c0b5620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:50466/0 (socket says 172.21.15.62:50466) 2024-04-25T09:18:32.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.866+0000 7f380b200640 1 -- 172.21.15.62:0/439667983 learned_addr learned my addr 172.21.15.62:0/439667983 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:32.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.866+0000 7f380a800640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f380c15df20 0x7f380c161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.867+0000 7f3803e00640 1 --2- 172.21.15.62:0/439667983 >> v2:172.21.15.121:3300/0 conn(0x7f380c0b9840 0x7f380c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.867+0000 7f3809e00640 1 -- 172.21.15.62:0/439667983 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295112529 0 0) 0x7f380c15e300 con 0x7f380c0ceeb0 2024-04-25T09:18:32.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.867+0000 7f3809e00640 1 -- 172.21.15.62:0/439667983 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f0003520 con 0x7f380c0ceeb0 2024-04-25T09:18:32.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.867+0000 7f380a800640 1 -- 172.21.15.62:0/439667983 >> v2:172.21.15.121:3300/0 conn(0x7f380c0b9840 msgr2=0x7f380c0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:32.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.867+0000 7f380a800640 1 --2- 172.21.15.62:0/439667983 >> v2:172.21.15.121:3300/0 conn(0x7f380c0b9840 0x7f380c0ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.867+0000 7f380a800640 1 -- 172.21.15.62:0/439667983 >> v1:172.21.15.62:6789/0 conn(0x7f380c0ceeb0 legacy=0x7f380c0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.867+0000 7f380a800640 1 -- 172.21.15.62:0/439667983 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c162310 con 0x7f380c15df20 2024-04-25T09:18:32.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.868+0000 7f380a800640 1 --2- 172.21.15.62:0/439667983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 0x7f380c161a10 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f380c08c5e0 tx=0x7f37fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadff0e5dd04c57c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.868+0000 7f3809e00640 1 -- 172.21.15.62:0/439667983 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc037070 con 0x7f380c15df20 2024-04-25T09:18:32.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.868+0000 7f3809e00640 1 -- 172.21.15.62:0/439667983 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc032440 con 0x7f380c15df20 2024-04-25T09:18:32.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.868+0000 7f3809e00640 1 -- 172.21.15.62:0/439667983 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc032810 con 0x7f380c15df20 2024-04-25T09:18:32.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.869+0000 7f3811400640 1 -- 172.21.15.62:0/439667983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 msgr2=0x7f380c161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.869+0000 7f3811400640 1 --2- 172.21.15.62:0/439667983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 0x7f380c161a10 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f380c08c5e0 tx=0x7f37fc02a230 comp rx=0 tx=0).stop 2024-04-25T09:18:32.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.869+0000 7f3811400640 1 -- 172.21.15.62:0/439667983 shutdown_connections 2024-04-25T09:18:32.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.869+0000 7f3811400640 1 --2- 172.21.15.62:0/439667983 >> v2:172.21.15.121:3300/0 conn(0x7f380c0b9840 0x7f380c0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.869+0000 7f3811400640 1 --2- 172.21.15.62:0/439667983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 0x7f380c161a10 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.869+0000 7f3811400640 1 -- 172.21.15.62:0/439667983 >> 172.21.15.62:0/439667983 conn(0x7f380c0668f0 msgr2=0x7f380c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:32.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.869+0000 7f3811400640 1 -- 172.21.15.62:0/439667983 shutdown_connections 2024-04-25T09:18:32.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.869+0000 7f3811400640 1 -- 172.21.15.62:0/439667983 wait complete. 2024-04-25T09:18:32.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.871+0000 7f3811400640 1 Processor -- start 2024-04-25T09:18:32.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.871+0000 7f3811400640 1 -- start start 2024-04-25T09:18:32.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.872+0000 7f3811400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c0b9840 0x7f380c152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.872+0000 7f3811400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceeb0 0x7f380c0ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.872+0000 7f3811400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 0x7f380c0d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.872+0000 7f3811400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c161f50 con 0x7f380c15df20 2024-04-25T09:18:32.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.872+0000 7f3811400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0d1da0 con 0x7f380c0ceeb0 2024-04-25T09:18:32.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.872+0000 7f3811400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c1620d0 con 0x7f380c0b9840 2024-04-25T09:18:32.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.872+0000 7f380b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 0x7f380c0d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 0x7f380c0d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48782/0 (socket says 172.21.15.62:48782) 2024-04-25T09:18:32.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380b200640 1 -- 172.21.15.62:0/425094550 learned_addr learned my addr 172.21.15.62:0/425094550 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:32.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380a800640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c0b9840 0x7f380c152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380b200640 1 -- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c0b9840 msgr2=0x7f380c152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380b200640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f380c0b9840 0x7f380c152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f3803e00640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceeb0 0x7f380c0ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380b200640 1 -- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceeb0 msgr2=0x7f380c0ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380b200640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceeb0 0x7f380c0ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380b200640 1 -- 172.21.15.62:0/425094550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0aaae0 con 0x7f380c15df20 2024-04-25T09:18:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f3803e00640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f380c0ceeb0 0x7f380c0ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.873+0000 7f380b200640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 0x7f380c0d25a0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f380400cb40 tx=0x7f380400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.874+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804017070 con 0x7f380c15df20 2024-04-25T09:18:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.874+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380400aca0 con 0x7f380c15df20 2024-04-25T09:18:32.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.874+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804012620 con 0x7f380c15df20 2024-04-25T09:18:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.874+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c0aadc0 con 0x7f380c15df20 2024-04-25T09:18:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.874+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380c0ab380 con 0x7f380c15df20 2024-04-25T09:18:32.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.875+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f380c067c40 con 0x7f380c15df20 2024-04-25T09:18:32.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.876+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f380401a070 con 0x7f380c15df20 2024-04-25T09:18:32.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.877+0000 7f3802a00640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37e806b4d0 0x7f37e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.877+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3804089d90 con 0x7f380c15df20 2024-04-25T09:18:32.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.877+0000 7f3802a00640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f37e8071390 0x7f37e80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:32.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.878+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 --> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e8073e60 con 0x7f37e8071390 2024-04-25T09:18:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.878+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7f380408c430 con 0x7f380c15df20 2024-04-25T09:18:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.878+0000 7f380a800640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37e806b4d0 0x7f37e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.878+0000 7f3803e00640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f37e8071390 0x7f37e80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:32.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.879+0000 7f3803e00640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f37e8071390 0x7f37e80737b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.879+0000 7f380a800640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37e806b4d0 0x7f37e806d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f37fc034000 tx=0x7f37fc02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:32.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.879+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 <== osd.1 v2:172.21.15.62:6808/2146620681 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f37e8073e60 con 0x7f37e8071390 2024-04-25T09:18:32.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.894+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 --> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f380c0d18b0 con 0x7f37e8071390 2024-04-25T09:18:32.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.895+0000 7f3802a00640 1 -- 172.21.15.62:0/425094550 <== osd.1 v2:172.21.15.62:6808/2146620681 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f380c0d18b0 con 0x7f37e8071390 2024-04-25T09:18:32.882 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:32.882 INFO:teuthology.orchestra.run.smithi062.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:18:32.882 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.895+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f37e8071390 msgr2=0x7f37e80737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.895+0000 7f3811400640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.62:6808/2146620681,v1:172.21.15.62:6809/2146620681] conn(0x7f37e8071390 0x7f37e80737b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37e806b4d0 msgr2=0x7f37e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f3811400640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f37e806b4d0 0x7f37e806d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f37fc034000 tx=0x7f37fc02e000 comp rx=0 tx=0).stop 2024-04-25T09:18:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 msgr2=0x7f380c0d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f3811400640 1 --2- 172.21.15.62:0/425094550 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f380c15df20 0x7f380c0d25a0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f380400cb40 tx=0x7f380400a060 comp rx=0 tx=0).stop 2024-04-25T09:18:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f380b200640 1 -- 172.21.15.62:0/425094550 reap_dead start 2024-04-25T09:18:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 shutdown_connections 2024-04-25T09:18:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 >> 172.21.15.62:0/425094550 conn(0x7f380c0668f0 msgr2=0x7f380c15faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 shutdown_connections 2024-04-25T09:18:32.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:32.896+0000 7f3811400640 1 -- 172.21.15.62:0/425094550 wait complete. 2024-04-25T09:18:32.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T09:18:33.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.121+0000 7f56b5e00640 1 Processor -- start 2024-04-25T09:18:33.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.121+0000 7f56b5e00640 1 -- start start 2024-04-25T09:18:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56b5e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56b00cf5f0 0x7f56b00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56b5e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f56b00cfaa0 0x7f56b00b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56b5e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b00cff60 con 0x7f56b015c4e0 2024-04-25T09:18:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56b5e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b00b3720 con 0x7f56b00cf5f0 2024-04-25T09:18:33.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56b5e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b0070360 con 0x7f56b00cfaa0 2024-04-25T09:18:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56afe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56b00cf5f0 0x7f56b00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56afe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f56b00cf5f0 0x7f56b00b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48798/0 (socket says 172.21.15.62:48798) 2024-04-25T09:18:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56afe00640 1 -- 172.21.15.62:0/1929942656 learned_addr learned my addr 172.21.15.62:0/1929942656 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:33.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.123+0000 7f56b5400640 1 --2- 172.21.15.62:0/1929942656 >> v2:172.21.15.121:3300/0 conn(0x7f56b00cfaa0 0x7f56b00b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.124+0000 7f56afe00640 1 -- 172.21.15.62:0/1929942656 >> v2:172.21.15.121:3300/0 conn(0x7f56b00cfaa0 msgr2=0x7f56b00b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.124+0000 7f56afe00640 1 --2- 172.21.15.62:0/1929942656 >> v2:172.21.15.121:3300/0 conn(0x7f56b00cfaa0 0x7f56b00b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.124+0000 7f56afe00640 1 -- 172.21.15.62:0/1929942656 >> v1:172.21.15.121:6789/0 conn(0x7f56b015c4e0 legacy=0x7f56b015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.124+0000 7f56afe00640 1 -- 172.21.15.62:0/1929942656 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b0070640 con 0x7f56b00cf5f0 2024-04-25T09:18:33.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.125+0000 7f56afe00640 1 --2- 172.21.15.62:0/1929942656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cf5f0 0x7f56b00b42d0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f56b006e560 tx=0x7f569c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4410ca1e4644d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.125+0000 7f56af400640 1 -- 172.21.15.62:0/1929942656 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c037070 con 0x7f56b00cf5f0 2024-04-25T09:18:33.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.125+0000 7f56af400640 1 -- 172.21.15.62:0/1929942656 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c032440 con 0x7f56b00cf5f0 2024-04-25T09:18:33.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.125+0000 7f56af400640 1 -- 172.21.15.62:0/1929942656 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c032810 con 0x7f56b00cf5f0 2024-04-25T09:18:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.126+0000 7f56b5e00640 1 -- 172.21.15.62:0/1929942656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cf5f0 msgr2=0x7f56b00b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.126+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1929942656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cf5f0 0x7f56b00b42d0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f56b006e560 tx=0x7f569c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.126+0000 7f56b5e00640 1 -- 172.21.15.62:0/1929942656 shutdown_connections 2024-04-25T09:18:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.126+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1929942656 >> v2:172.21.15.121:3300/0 conn(0x7f56b00cfaa0 0x7f56b00b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.126+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1929942656 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cf5f0 0x7f56b00b42d0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.126+0000 7f56b5e00640 1 -- 172.21.15.62:0/1929942656 >> 172.21.15.62:0/1929942656 conn(0x7f56b00668f0 msgr2=0x7f56b0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.126+0000 7f56b5e00640 1 -- 172.21.15.62:0/1929942656 shutdown_connections 2024-04-25T09:18:33.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.126+0000 7f56b5e00640 1 -- 172.21.15.62:0/1929942656 wait complete. 2024-04-25T09:18:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.128+0000 7f56b5e00640 1 Processor -- start 2024-04-25T09:18:33.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.128+0000 7f56b5e00640 1 -- start start 2024-04-25T09:18:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b00cf5f0 0x7f56b00af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cfaa0 0x7f56b01e5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56b015c4e0 0x7f56b014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56b5e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0071120 con 0x7f56b00cfaa0 2024-04-25T09:18:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56b5e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0070c00 con 0x7f56b00cf5f0 2024-04-25T09:18:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56b5e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b00b2710 con 0x7f56b015c4e0 2024-04-25T09:18:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cfaa0 0x7f56b01e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56afe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cfaa0 0x7f56b01e5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48810/0 (socket says 172.21.15.62:48810) 2024-04-25T09:18:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.129+0000 7f56afe00640 1 -- 172.21.15.62:0/1875509270 learned_addr learned my addr 172.21.15.62:0/1875509270 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:33.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56afe00640 1 -- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56b015c4e0 msgr2=0x7f56b014f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56afe00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56b015c4e0 0x7f56b014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56afe00640 1 -- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b00cf5f0 msgr2=0x7f56b00af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56afe00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b00cf5f0 0x7f56b00af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56afe00640 1 -- 172.21.15.62:0/1875509270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b01e59d0 con 0x7f56b00cfaa0 2024-04-25T09:18:33.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56afe00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cfaa0 0x7f56b01e5340 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56acc00640 1 -- 172.21.15.62:0/1875509270 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c041020 con 0x7f56b00cfaa0 2024-04-25T09:18:33.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56acc00640 1 -- 172.21.15.62:0/1875509270 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c0306c0 con 0x7f56b00cfaa0 2024-04-25T09:18:33.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b01e5cb0 con 0x7f56b00cfaa0 2024-04-25T09:18:33.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.130+0000 7f56acc00640 1 -- 172.21.15.62:0/1875509270 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c037040 con 0x7f56b00cfaa0 2024-04-25T09:18:33.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.131+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b0154520 con 0x7f56b00cfaa0 2024-04-25T09:18:33.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.132+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b01e61d0 con 0x7f56b00cfaa0 2024-04-25T09:18:33.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.133+0000 7f56acc00640 1 -- 172.21.15.62:0/1875509270 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f569c044020 con 0x7f56b00cfaa0 2024-04-25T09:18:33.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.134+0000 7f56acc00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.134+0000 7f56acc00640 1 -- 172.21.15.62:0/1875509270 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f569c0aa750 con 0x7f56b00cfaa0 2024-04-25T09:18:33.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.134+0000 7f56b4a00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.136+0000 7f56b4a00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f569006b4d0 0x7f569006d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006fd0 tx=0x7f56a4006bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.138+0000 7f56acc00640 1 -- 172.21.15.62:0/1875509270 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f569c04b030 con 0x7f56b00cfaa0 2024-04-25T09:18:33.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.336+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f56b014fda0 con 0x7f569006b4d0 2024-04-25T09:18:33.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.337+0000 7f56acc00640 1 -- 172.21.15.62:0/1875509270 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56a4004020 con 0x7f569006b4d0 2024-04-25T09:18:33.324 INFO:teuthology.orchestra.run.smithi062.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T09:18:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f569006b4d0 msgr2=0x7f569006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f569006b4d0 0x7f569006d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006fd0 tx=0x7f56a4006bc0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cfaa0 msgr2=0x7f56b01e5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cfaa0 0x7f56b01e5340 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 shutdown_connections 2024-04-25T09:18:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f569006b4d0 0x7f569006d990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56b00cfaa0 0x7f56b01e5340 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f56b015c4e0 0x7f56b014f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 --2- 172.21.15.62:0/1875509270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f56b00cf5f0 0x7f56b00af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.339+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 >> 172.21.15.62:0/1875509270 conn(0x7f56b00668f0 msgr2=0x7f56b01582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.340+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 shutdown_connections 2024-04-25T09:18:33.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.340+0000 7f56b5e00640 1 -- 172.21.15.62:0/1875509270 wait complete. 2024-04-25T09:18:33.347 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T09:18:33.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:18:33.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.584+0000 7f0bebe00640 1 Processor -- start 2024-04-25T09:18:33.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.584+0000 7f0bebe00640 1 -- start start 2024-04-25T09:18:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.586+0000 7f0bebe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0bec161270 0x7f0bec0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.586+0000 7f0bebe00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0bec0d2870 0x7f0bec0b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.586+0000 7f0bebe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bec0d1480 con 0x7f0bec161650 2024-04-25T09:18:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.586+0000 7f0bebe00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec0d3900 con 0x7f0bec0d2870 2024-04-25T09:18:33.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.586+0000 7f0bebe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec0d11a0 con 0x7f0bec161270 2024-04-25T09:18:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beb400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0bec0d2870 0x7f0bec0b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beb400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0bec0d2870 0x7f0bec0b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48814/0 (socket says 172.21.15.62:48814) 2024-04-25T09:18:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beb400640 1 -- 172.21.15.62:0/1415022657 learned_addr learned my addr 172.21.15.62:0/1415022657 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beaa00640 1 --2- 172.21.15.62:0/1415022657 >> v2:172.21.15.121:3300/0 conn(0x7f0bec161270 0x7f0bec0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beb400640 1 -- 172.21.15.62:0/1415022657 >> v2:172.21.15.121:3300/0 conn(0x7f0bec161270 msgr2=0x7f0bec0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:33.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beb400640 1 --2- 172.21.15.62:0/1415022657 >> v2:172.21.15.121:3300/0 conn(0x7f0bec161270 0x7f0bec0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beb400640 1 -- 172.21.15.62:0/1415022657 >> v1:172.21.15.121:6789/0 conn(0x7f0bec161650 legacy=0x7f0bec161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beb400640 1 -- 172.21.15.62:0/1415022657 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0cee80 con 0x7f0bec0d2870 2024-04-25T09:18:33.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.587+0000 7f0beaa00640 1 --2- 172.21.15.62:0/1415022657 >> v2:172.21.15.121:3300/0 conn(0x7f0bec161270 0x7f0bec0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.588+0000 7f0beb400640 1 --2- 172.21.15.62:0/1415022657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec0d2870 0x7f0bec0b5aa0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0d4b40 tx=0x7f0be402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7308532b82f90b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.589+0000 7f0be9600640 1 -- 172.21.15.62:0/1415022657 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4036070 con 0x7f0bec0d2870 2024-04-25T09:18:33.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.589+0000 7f0be9600640 1 -- 172.21.15.62:0/1415022657 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be4031600 con 0x7f0bec0d2870 2024-04-25T09:18:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.589+0000 7f0be9600640 1 -- 172.21.15.62:0/1415022657 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40319d0 con 0x7f0bec0d2870 2024-04-25T09:18:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.589+0000 7f0bebe00640 1 -- 172.21.15.62:0/1415022657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec0d2870 msgr2=0x7f0bec0b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.589+0000 7f0bebe00640 1 --2- 172.21.15.62:0/1415022657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec0d2870 0x7f0bec0b5aa0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0d4b40 tx=0x7f0be402a610 comp rx=0 tx=0).stop 2024-04-25T09:18:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.590+0000 7f0bebe00640 1 -- 172.21.15.62:0/1415022657 shutdown_connections 2024-04-25T09:18:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.590+0000 7f0bebe00640 1 --2- 172.21.15.62:0/1415022657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec0d2870 0x7f0bec0b5aa0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.590+0000 7f0bebe00640 1 --2- 172.21.15.62:0/1415022657 >> v2:172.21.15.121:3300/0 conn(0x7f0bec161270 0x7f0bec0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.590+0000 7f0bebe00640 1 -- 172.21.15.62:0/1415022657 >> 172.21.15.62:0/1415022657 conn(0x7f0bec066d10 msgr2=0x7f0bec067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:33.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.590+0000 7f0bebe00640 1 -- 172.21.15.62:0/1415022657 shutdown_connections 2024-04-25T09:18:33.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.590+0000 7f0bebe00640 1 -- 172.21.15.62:0/1415022657 wait complete. 2024-04-25T09:18:33.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.591+0000 7f0bebe00640 1 Processor -- start 2024-04-25T09:18:33.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.592+0000 7f0bebe00640 1 -- start start 2024-04-25T09:18:33.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.592+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0bec0d2870 0x7f0bec0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.592+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec161270 0x7f0bec146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.592+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bec161650 0x7f0bec1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.592+0000 7f0bebe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0b2ed0 con 0x7f0bec161270 2024-04-25T09:18:33.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bebe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0d3590 con 0x7f0bec161650 2024-04-25T09:18:33.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bebe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0b3050 con 0x7f0bec0d2870 2024-04-25T09:18:33.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec161270 0x7f0bec146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec161270 0x7f0bec146ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48816/0 (socket says 172.21.15.62:48816) 2024-04-25T09:18:33.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bea000640 1 -- 172.21.15.62:0/2256172860 learned_addr learned my addr 172.21.15.62:0/2256172860 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:33.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0beaa00640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0bec0d2870 0x7f0bec0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0beb400640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bec161650 0x7f0bec1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bea000640 1 -- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0bec0d2870 msgr2=0x7f0bec0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bea000640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0bec0d2870 0x7f0bec0c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bea000640 1 -- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bec161650 msgr2=0x7f0bec1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bea000640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bec161650 0x7f0bec1470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.593+0000 7f0bea000640 1 -- 172.21.15.62:0/2256172860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0c8700 con 0x7f0bec161270 2024-04-25T09:18:33.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.594+0000 7f0bea000640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec161270 0x7f0bec146ba0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c400 tx=0x7f0bd800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.594+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80052d0 con 0x7f0bec161270 2024-04-25T09:18:33.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.594+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd8005490 con 0x7f0bec161270 2024-04-25T09:18:33.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.594+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8007260 con 0x7f0bec161270 2024-04-25T09:18:33.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.594+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec0c89e0 con 0x7f0bec161270 2024-04-25T09:18:33.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.594+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec0c8fa0 con 0x7f0bec161270 2024-04-25T09:18:33.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.595+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0bec072910 con 0x7f0bec161270 2024-04-25T09:18:33.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.596+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bd8007400 con 0x7f0bec161270 2024-04-25T09:18:33.585 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.597+0000 7f0be3400640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.597+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0bd8089410 con 0x7f0bec161270 2024-04-25T09:18:33.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.597+0000 7f0be3400640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f0bcc071390 0x7f0bcc0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.598+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 --> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bcc073e60 con 0x7f0bcc071390 2024-04-25T09:18:33.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.598+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bd8090050 con 0x7f0bec161270 2024-04-25T09:18:33.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.598+0000 7f0beb400640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f0bcc071390 0x7f0bcc0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.598+0000 7f0beaa00640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.598+0000 7f0beb400640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f0bcc071390 0x7f0bcc0737b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.599+0000 7f0beaa00640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00b840 tx=0x7f0bdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.599+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 <== osd.2 v2:172.21.15.62:6816/385571314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0bcc073e60 con 0x7f0bcc071390 2024-04-25T09:18:33.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.614+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 --> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0bec1f3d30 con 0x7f0bcc071390 2024-04-25T09:18:33.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.615+0000 7f0be3400640 1 -- 172.21.15.62:0/2256172860 <== osd.2 v2:172.21.15.62:6816/385571314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0bec1f3d30 con 0x7f0bcc071390 2024-04-25T09:18:33.602 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:33.602 INFO:teuthology.orchestra.run.smithi062.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:18:33.602 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:33.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.615+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f0bcc071390 msgr2=0x7f0bcc0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.615+0000 7f0bebe00640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.62:6816/385571314,v1:172.21.15.62:6818/385571314] conn(0x7f0bcc071390 0x7f0bcc0737b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0bcc06b4d0 msgr2=0x7f0bcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0bebe00640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00b840 tx=0x7f0bdc008040 comp rx=0 tx=0).stop 2024-04-25T09:18:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec161270 msgr2=0x7f0bec146ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0bebe00640 1 --2- 172.21.15.62:0/2256172860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bec161270 0x7f0bec146ba0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800c400 tx=0x7f0bd800c8c0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0beb400640 1 -- 172.21.15.62:0/2256172860 reap_dead start 2024-04-25T09:18:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 shutdown_connections 2024-04-25T09:18:33.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 >> 172.21.15.62:0/2256172860 conn(0x7f0bec066d10 msgr2=0x7f0bec0b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:33.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 shutdown_connections 2024-04-25T09:18:33.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.616+0000 7f0bebe00640 1 -- 172.21.15.62:0/2256172860 wait complete. 2024-04-25T09:18:33.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T09:18:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.846+0000 7f0cd7200640 1 Processor -- start 2024-04-25T09:18:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.847+0000 7f0cd7200640 1 -- start start 2024-04-25T09:18:33.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.848+0000 7f0cd7200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0cd015bd10 0x7f0cd00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.848+0000 7f0cd7200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0cd00cccf0 0x7f0cd00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.848+0000 7f0cd7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd00aed00 0x7f0cd015f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.848+0000 7f0cd7200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd006d660 con 0x7f0cd00cccf0 2024-04-25T09:18:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cd7200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd006ca70 con 0x7f0cd015bd10 2024-04-25T09:18:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cd7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd015c0f0 con 0x7f0cd00aed00 2024-04-25T09:18:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cc7400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0cd00cccf0 0x7f0cd00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cc7400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0cd00cccf0 0x7f0cd00ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48818/0 (socket says 172.21.15.62:48818) 2024-04-25T09:18:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cc7400640 1 -- 172.21.15.62:0/101588987 learned_addr learned my addr 172.21.15.62:0/101588987 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cc7400640 1 -- 172.21.15.62:0/101588987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd00aed00 msgr2=0x7f0cd015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cc7400640 1 --2- 172.21.15.62:0/101588987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd00aed00 0x7f0cd015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cc7400640 1 -- 172.21.15.62:0/101588987 >> v2:172.21.15.121:3300/0 conn(0x7f0cd015bd10 msgr2=0x7f0cd00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0ccfe00640 1 --2- 172.21.15.62:0/101588987 >> v2:172.21.15.121:3300/0 conn(0x7f0cd015bd10 0x7f0cd00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cc7400640 1 --2- 172.21.15.62:0/101588987 >> v2:172.21.15.121:3300/0 conn(0x7f0cd015bd10 0x7f0cd00cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.849+0000 7f0cc7400640 1 -- 172.21.15.62:0/101588987 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd006cd30 con 0x7f0cd00cccf0 2024-04-25T09:18:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.850+0000 7f0cc7400640 1 --2- 172.21.15.62:0/101588987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00cccf0 0x7f0cd00ae7c0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0072720 tx=0x7f0cb802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6698676ba875f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.851+0000 7f0ccf400640 1 -- 172.21.15.62:0/101588987 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8037070 con 0x7f0cd00cccf0 2024-04-25T09:18:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.851+0000 7f0ccf400640 1 -- 172.21.15.62:0/101588987 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb8032440 con 0x7f0cd00cccf0 2024-04-25T09:18:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.851+0000 7f0ccf400640 1 -- 172.21.15.62:0/101588987 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8032810 con 0x7f0cd00cccf0 2024-04-25T09:18:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.851+0000 7f0cd7200640 1 -- 172.21.15.62:0/101588987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00cccf0 msgr2=0x7f0cd00ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.851+0000 7f0cd7200640 1 --2- 172.21.15.62:0/101588987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00cccf0 0x7f0cd00ae7c0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0072720 tx=0x7f0cb802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.852+0000 7f0cd7200640 1 -- 172.21.15.62:0/101588987 shutdown_connections 2024-04-25T09:18:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.852+0000 7f0cd7200640 1 --2- 172.21.15.62:0/101588987 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd00aed00 0x7f0cd015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.852+0000 7f0cd7200640 1 --2- 172.21.15.62:0/101588987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00cccf0 0x7f0cd00ae7c0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.852+0000 7f0cd7200640 1 --2- 172.21.15.62:0/101588987 >> v2:172.21.15.121:3300/0 conn(0x7f0cd015bd10 0x7f0cd00cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.852+0000 7f0cd7200640 1 -- 172.21.15.62:0/101588987 >> 172.21.15.62:0/101588987 conn(0x7f0cd00668f0 msgr2=0x7f0cd0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.852+0000 7f0cd7200640 1 -- 172.21.15.62:0/101588987 shutdown_connections 2024-04-25T09:18:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.852+0000 7f0cd7200640 1 -- 172.21.15.62:0/101588987 wait complete. 2024-04-25T09:18:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.854+0000 7f0cd7200640 1 Processor -- start 2024-04-25T09:18:33.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.854+0000 7f0cd7200640 1 -- start start 2024-04-25T09:18:33.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0cd7200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00aed00 0x7f0cd0148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0cd7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cd00cccf0 0x7f0cd0152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0cd7200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd015bd10 0x7f0cd0152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0cd7200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd00d01e0 con 0x7f0cd00aed00 2024-04-25T09:18:33.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0cd7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd006d2f0 con 0x7f0cd00cccf0 2024-04-25T09:18:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0cd7200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd006daf0 con 0x7f0cd015bd10 2024-04-25T09:18:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0ccfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00aed00 0x7f0cd0148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0ccfe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00aed00 0x7f0cd0148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48828/0 (socket says 172.21.15.62:48828) 2024-04-25T09:18:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0ccfe00640 1 -- 172.21.15.62:0/1449825339 learned_addr learned my addr 172.21.15.62:0/1449825339 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0cd5200640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd015bd10 0x7f0cd0152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0cc7400640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cd00cccf0 0x7f0cd0152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0ccfe00640 1 -- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd015bd10 msgr2=0x7f0cd0152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0ccfe00640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd015bd10 0x7f0cd0152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0ccfe00640 1 -- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cd00cccf0 msgr2=0x7f0cd0152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.855+0000 7f0ccfe00640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cd00cccf0 0x7f0cd0152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.856+0000 7f0ccfe00640 1 -- 172.21.15.62:0/1449825339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0153da0 con 0x7f0cd00aed00 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.856+0000 7f0ccfe00640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00aed00 0x7f0cd0148bc0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c820 tx=0x7f0cc000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.856+0000 7f0cccc00640 1 -- 172.21.15.62:0/1449825339 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0015070 con 0x7f0cd00aed00 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.856+0000 7f0cccc00640 1 -- 172.21.15.62:0/1449825339 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc0004020 con 0x7f0cd00aed00 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.856+0000 7f0cccc00640 1 -- 172.21.15.62:0/1449825339 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00043f0 con 0x7f0cd00aed00 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.856+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd0154080 con 0x7f0cd00aed00 2024-04-25T09:18:33.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.856+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd01f8390 con 0x7f0cd00aed00 2024-04-25T09:18:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.857+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd0072c70 con 0x7f0cd00aed00 2024-04-25T09:18:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.858+0000 7f0cccc00640 1 -- 172.21.15.62:0/1449825339 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cc0007830 con 0x7f0cd00aed00 2024-04-25T09:18:33.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.859+0000 7f0cccc00640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0cac06b4d0 0x7f0cac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:33.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.859+0000 7f0cccc00640 1 -- 172.21.15.62:0/1449825339 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cc0089c30 con 0x7f0cd00aed00 2024-04-25T09:18:33.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.860+0000 7f0cc7400640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0cac06b4d0 0x7f0cac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:33.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.861+0000 7f0cc7400640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0cac06b4d0 0x7f0cac06d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8034000 tx=0x7f0cb802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:33.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:33.864+0000 7f0cccc00640 1 -- 172.21.15.62:0/1449825339 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0cc0058e40 con 0x7f0cd00aed00 2024-04-25T09:18:34.053 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.066+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cd004f000 con 0x7f0cac06b4d0 2024-04-25T09:18:34.054 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.067+0000 7f0cccc00640 1 -- 172.21.15.62:0/1449825339 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0cd004f000 con 0x7f0cac06b4d0 2024-04-25T09:18:34.054 INFO:teuthology.orchestra.run.smithi062.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T09:18:34.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0cac06b4d0 msgr2=0x7f0cac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0cac06b4d0 0x7f0cac06d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8034000 tx=0x7f0cb802e000 comp rx=0 tx=0).stop 2024-04-25T09:18:34.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00aed00 msgr2=0x7f0cd0148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.057 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00aed00 0x7f0cd0148bc0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c820 tx=0x7f0cc000cce0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 shutdown_connections 2024-04-25T09:18:34.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f0cac06b4d0 0x7f0cac06d990 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f0cd015bd10 0x7f0cd0152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cd00cccf0 0x7f0cd0152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 --2- 172.21.15.62:0/1449825339 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0cd00aed00 0x7f0cd0148bc0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 >> 172.21.15.62:0/1449825339 conn(0x7f0cd00668f0 msgr2=0x7f0cd00710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:34.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 shutdown_connections 2024-04-25T09:18:34.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.070+0000 7f0cd7200640 1 -- 172.21.15.62:0/1449825339 wait complete. 2024-04-25T09:18:34.076 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T09:18:34.076 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:18:34.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.303+0000 7fc8fbe00640 1 Processor -- start 2024-04-25T09:18:34.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.303+0000 7fc8fbe00640 1 -- start start 2024-04-25T09:18:34.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.305+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 0x7fc8fc15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.305+0000 7fc8fbe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc8fc0cd9a0 0x7fc8fc15e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.305+0000 7fc8fbe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8fc15b150 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.305+0000 7fc8fbe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8fc070170 con 0x7fc8fc0cd9a0 2024-04-25T09:18:34.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.305+0000 7fc8fbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc15ef40 con 0x7fc8fc154e10 2024-04-25T09:18:34.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.306+0000 7fc8faa00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 0x7fc8fc15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.306+0000 7fc8f2000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc8fc0cd9a0 0x7fc8fc15e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.306+0000 7fc8fb400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc8fc0b48e0 0x7fc8fc0b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:39330/0 (socket says 172.21.15.62:39330) 2024-04-25T09:18:34.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.306+0000 7fc8fb400640 1 -- 172.21.15.62:0/1980586850 learned_addr learned my addr 172.21.15.62:0/1980586850 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:34.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.307+0000 7fc8fa000640 1 -- 172.21.15.62:0/1980586850 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897677014 0 0) 0x7fc8fc15b150 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.307+0000 7fc8faa00640 1 -- 172.21.15.62:0/1980586850 >> v2:172.21.15.121:3300/0 conn(0x7fc8fc0cd9a0 msgr2=0x7fc8fc15e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:34.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.307+0000 7fc8faa00640 1 --2- 172.21.15.62:0/1980586850 >> v2:172.21.15.121:3300/0 conn(0x7fc8fc0cd9a0 0x7fc8fc15e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.307+0000 7fc8faa00640 1 -- 172.21.15.62:0/1980586850 >> v1:172.21.15.121:6789/0 conn(0x7fc8fc0b48e0 legacy=0x7fc8fc0b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.307+0000 7fc8faa00640 1 -- 172.21.15.62:0/1980586850 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc15f220 con 0x7fc8fc154e10 2024-04-25T09:18:34.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.308+0000 7fc8faa00640 1 --2- 172.21.15.62:0/1980586850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 0x7fc8fc15e3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0097a0 tx=0x7fc8ec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8236793f8605a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:34.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fa000640 1 -- 172.21.15.62:0/1980586850 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec037070 con 0x7fc8fc154e10 2024-04-25T09:18:34.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fa000640 1 -- 172.21.15.62:0/1980586850 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ec032440 con 0x7fc8fc154e10 2024-04-25T09:18:34.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fa000640 1 -- 172.21.15.62:0/1980586850 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec032810 con 0x7fc8fc154e10 2024-04-25T09:18:34.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fbe00640 1 -- 172.21.15.62:0/1980586850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 msgr2=0x7fc8fc15e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fbe00640 1 --2- 172.21.15.62:0/1980586850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 0x7fc8fc15e3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0097a0 tx=0x7fc8ec02a180 comp rx=0 tx=0).stop 2024-04-25T09:18:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fbe00640 1 -- 172.21.15.62:0/1980586850 shutdown_connections 2024-04-25T09:18:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fbe00640 1 --2- 172.21.15.62:0/1980586850 >> v2:172.21.15.121:3300/0 conn(0x7fc8fc0cd9a0 0x7fc8fc15e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fbe00640 1 --2- 172.21.15.62:0/1980586850 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 0x7fc8fc15e3d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fbe00640 1 -- 172.21.15.62:0/1980586850 >> 172.21.15.62:0/1980586850 conn(0x7fc8fc066ad0 msgr2=0x7fc8fc066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fbe00640 1 -- 172.21.15.62:0/1980586850 shutdown_connections 2024-04-25T09:18:34.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.309+0000 7fc8fbe00640 1 -- 172.21.15.62:0/1980586850 wait complete. 2024-04-25T09:18:34.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.311+0000 7fc8fbe00640 1 Processor -- start 2024-04-25T09:18:34.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.311+0000 7fc8fbe00640 1 -- start start 2024-04-25T09:18:34.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.312+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc0b48e0 0x7fc8fc146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.312+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8fc0cd9a0 0x7fc8fc159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.312+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 0x7fc8fc0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.312+0000 7fc8fbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc1551f0 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.312+0000 7fc8fbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc070710 con 0x7fc8fc0cd9a0 2024-04-25T09:18:34.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.312+0000 7fc8fbe00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc0b4d30 con 0x7fc8fc154e10 2024-04-25T09:18:34.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8faa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc0b48e0 0x7fc8fc146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8faa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc0b48e0 0x7fc8fc146390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48842/0 (socket says 172.21.15.62:48842) 2024-04-25T09:18:34.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8faa00640 1 -- 172.21.15.62:0/3570578072 learned_addr learned my addr 172.21.15.62:0/3570578072 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:34.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8f2000640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8fc0cd9a0 0x7fc8fc159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8faa00640 1 -- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 msgr2=0x7fc8fc0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8faa00640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8fc154e10 0x7fc8fc0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8faa00640 1 -- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8fc0cd9a0 msgr2=0x7fc8fc159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8faa00640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc8fc0cd9a0 0x7fc8fc159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.313+0000 7fc8faa00640 1 -- 172.21.15.62:0/3570578072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc159cf0 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.314+0000 7fc8faa00640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc0b48e0 0x7fc8fc146390 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec035040 tx=0x7fc8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:34.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.314+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec037050 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.314+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ec038040 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.314+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec004490 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.314+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8fc159fd0 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.314+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8fc155c40 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.315+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8fc0cd050 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.316+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8ec03c6d0 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.317+0000 7fc8f3400640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8d806b4d0 0x7fc8d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.317+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc8ec0aa9b0 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.317+0000 7fc8f3400640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fc8d8071390 0x7fc8d80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.318+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 --> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8ec0067f0 con 0x7fc8d8071390 2024-04-25T09:18:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.318+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8ec0af2a0 con 0x7fc8fc0b48e0 2024-04-25T09:18:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.318+0000 7fc8fb400640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fc8d8071390 0x7fc8d80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.318+0000 7fc8f2000640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8d806b4d0 0x7fc8d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.318+0000 7fc8fb400640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fc8d8071390 0x7fc8d80737b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:34.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.319+0000 7fc8f2000640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8d806b4d0 0x7fc8d806d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc1468d0 tx=0x7fc8e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:34.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.319+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 <== osd.3 v2:172.21.15.62:6817/3104548384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc8ec0067f0 con 0x7fc8d8071390 2024-04-25T09:18:34.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.334+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 --> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc8fc0cb6f0 con 0x7fc8d8071390 2024-04-25T09:18:34.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.334+0000 7fc8f3400640 1 -- 172.21.15.62:0/3570578072 <== osd.3 v2:172.21.15.62:6817/3104548384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc8f4001dd0 con 0x7fc8d8071390 2024-04-25T09:18:34.322 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:34.322 INFO:teuthology.orchestra.run.smithi062.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:18:34.322 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:34.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fc8d8071390 msgr2=0x7fc8d80737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.62:6817/3104548384,v1:172.21.15.62:6820/3104548384] conn(0x7fc8d8071390 0x7fc8d80737b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8d806b4d0 msgr2=0x7fc8d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8d806b4d0 0x7fc8d806d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc1468d0 tx=0x7fc8e4008040 comp rx=0 tx=0).stop 2024-04-25T09:18:34.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc0b48e0 msgr2=0x7fc8fc146390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 --2- 172.21.15.62:0/3570578072 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc8fc0b48e0 0x7fc8fc146390 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec035040 tx=0x7fc8ec0046c0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fb400640 1 -- 172.21.15.62:0/3570578072 reap_dead start 2024-04-25T09:18:34.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 shutdown_connections 2024-04-25T09:18:34.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 >> 172.21.15.62:0/3570578072 conn(0x7fc8fc066ad0 msgr2=0x7fc8fc06cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:34.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 shutdown_connections 2024-04-25T09:18:34.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.335+0000 7fc8fbe00640 1 -- 172.21.15.62:0/3570578072 wait complete. 2024-04-25T09:18:34.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T09:18:34.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.574+0000 7fdf57e00640 1 Processor -- start 2024-04-25T09:18:34.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.574+0000 7fdf57e00640 1 -- start start 2024-04-25T09:18:34.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf57e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf5806c900 0x7fdf5806e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 0x7fdf5806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf57e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdf5806f490 0x7fdf5815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf57e00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf580cc800 con 0x7fdf5806c900 2024-04-25T09:18:34.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf57e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5806abe0 con 0x7fdf5806f490 2024-04-25T09:18:34.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf57e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580d0d00 con 0x7fdf580d0520 2024-04-25T09:18:34.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf56a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf5806c900 0x7fdf5806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf56a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf5806c900 0x7fdf5806e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48844/0 (socket says 172.21.15.62:48844) 2024-04-25T09:18:34.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.575+0000 7fdf56a00640 1 -- 172.21.15.62:0/1444106523 learned_addr learned my addr 172.21.15.62:0/1444106523 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:34.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.576+0000 7fdf56000640 1 --2- 172.21.15.62:0/1444106523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 0x7fdf5806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.576+0000 7fdf56a00640 1 -- 172.21.15.62:0/1444106523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 msgr2=0x7fdf5806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:34.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.576+0000 7fdf56a00640 1 --2- 172.21.15.62:0/1444106523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 0x7fdf5806ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.565 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.576+0000 7fdf56a00640 1 -- 172.21.15.62:0/1444106523 >> v2:172.21.15.121:3300/0 conn(0x7fdf5806f490 msgr2=0x7fdf5815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:34.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.576+0000 7fdf56a00640 1 --2- 172.21.15.62:0/1444106523 >> v2:172.21.15.121:3300/0 conn(0x7fdf5806f490 0x7fdf5815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.576+0000 7fdf56a00640 1 -- 172.21.15.62:0/1444106523 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580cbed0 con 0x7fdf5806c900 2024-04-25T09:18:34.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.577+0000 7fdf56a00640 1 --2- 172.21.15.62:0/1444106523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806c900 0x7fdf5806e920 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009870 tx=0x7fdf4802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03cfc9626e58deb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:34.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.577+0000 7fdf55600640 1 -- 172.21.15.62:0/1444106523 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48037070 con 0x7fdf5806c900 2024-04-25T09:18:34.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.577+0000 7fdf55600640 1 -- 172.21.15.62:0/1444106523 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf48032440 con 0x7fdf5806c900 2024-04-25T09:18:34.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.577+0000 7fdf55600640 1 -- 172.21.15.62:0/1444106523 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48032810 con 0x7fdf5806c900 2024-04-25T09:18:34.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 -- 172.21.15.62:0/1444106523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806c900 msgr2=0x7fdf5806e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 --2- 172.21.15.62:0/1444106523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806c900 0x7fdf5806e920 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009870 tx=0x7fdf4802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 -- 172.21.15.62:0/1444106523 shutdown_connections 2024-04-25T09:18:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 --2- 172.21.15.62:0/1444106523 >> v2:172.21.15.121:3300/0 conn(0x7fdf5806f490 0x7fdf5815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 --2- 172.21.15.62:0/1444106523 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 0x7fdf5806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 --2- 172.21.15.62:0/1444106523 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806c900 0x7fdf5806e920 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 -- 172.21.15.62:0/1444106523 >> 172.21.15.62:0/1444106523 conn(0x7fdf580b33d0 msgr2=0x7fdf580b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:34.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 -- 172.21.15.62:0/1444106523 shutdown_connections 2024-04-25T09:18:34.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.578+0000 7fdf57e00640 1 -- 172.21.15.62:0/1444106523 wait complete. 2024-04-25T09:18:34.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.580+0000 7fdf57e00640 1 Processor -- start 2024-04-25T09:18:34.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.580+0000 7fdf57e00640 1 -- start start 2024-04-25T09:18:34.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf5806c900 0x7fdf580b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806f490 0x7fdf580b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 0x7fdf58151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf57e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580d0960 con 0x7fdf5806f490 2024-04-25T09:18:34.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf57e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580cc490 con 0x7fdf5806c900 2024-04-25T09:18:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf57e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580d0ae0 con 0x7fdf580d0520 2024-04-25T09:18:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf56000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806f490 0x7fdf580b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf56000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806f490 0x7fdf580b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:48856/0 (socket says 172.21.15.62:48856) 2024-04-25T09:18:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.581+0000 7fdf56000640 1 -- 172.21.15.62:0/2231903257 learned_addr learned my addr 172.21.15.62:0/2231903257 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf57400640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 0x7fdf58151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf56000640 1 -- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 msgr2=0x7fdf58151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf56a00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf5806c900 0x7fdf580b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf56000640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 0x7fdf58151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf56000640 1 -- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf5806c900 msgr2=0x7fdf580b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf56000640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf5806c900 0x7fdf580b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf56000640 1 -- 172.21.15.62:0/2231903257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580b5e80 con 0x7fdf5806f490 2024-04-25T09:18:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf56000640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806f490 0x7fdf580b5940 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c7f0 tx=0x7fdf4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf4ea00640 1 -- 172.21.15.62:0/2231903257 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44015070 con 0x7fdf5806f490 2024-04-25T09:18:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf4ea00640 1 -- 172.21.15.62:0/2231903257 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44004220 con 0x7fdf5806f490 2024-04-25T09:18:34.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf4ea00640 1 -- 172.21.15.62:0/2231903257 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44007620 con 0x7fdf5806f490 2024-04-25T09:18:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.582+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf580b48a0 con 0x7fdf5806f490 2024-04-25T09:18:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.583+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf580b4e60 con 0x7fdf5806f490 2024-04-25T09:18:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.584+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf580d1060 con 0x7fdf5806f490 2024-04-25T09:18:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.585+0000 7fdf4ea00640 1 -- 172.21.15.62:0/2231903257 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf440079d0 con 0x7fdf5806f490 2024-04-25T09:18:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.586+0000 7fdf4ea00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.586+0000 7fdf4ea00640 1 -- 172.21.15.62:0/2231903257 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdf4408a300 con 0x7fdf5806f490 2024-04-25T09:18:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.587+0000 7fdf56a00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.588+0000 7fdf56a00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf3806b4d0 0x7fdf3806d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdf48034000 tx=0x7fdf4802e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.590+0000 7fdf4ea00640 1 -- 172.21.15.62:0/2231903257 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf580d1060 con 0x7fdf5806f490 2024-04-25T09:18:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.788+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf5804ef20 con 0x7fdf3806b4d0 2024-04-25T09:18:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.789+0000 7fdf4ea00640 1 -- 172.21.15.62:0/2231903257 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdf5804ef20 con 0x7fdf3806b4d0 2024-04-25T09:18:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T09:18:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.791+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf3806b4d0 msgr2=0x7fdf3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.791+0000 7fdf57e00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf3806b4d0 0x7fdf3806d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdf48034000 tx=0x7fdf4802e000 comp rx=0 tx=0).stop 2024-04-25T09:18:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806f490 msgr2=0x7fdf580b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806f490 0x7fdf580b5940 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c7f0 tx=0x7fdf4400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 shutdown_connections 2024-04-25T09:18:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf5806f490 0x7fdf580b5940 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fdf580d0520 0x7fdf58151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 --2- 172.21.15.62:0/2231903257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf5806c900 0x7fdf580b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 >> 172.21.15.62:0/2231903257 conn(0x7fdf580b33d0 msgr2=0x7fdf580ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 shutdown_connections 2024-04-25T09:18:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:34.792+0000 7fdf57e00640 1 -- 172.21.15.62:0/2231903257 wait complete. 2024-04-25T09:18:34.798 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T09:18:34.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:18:35.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.024+0000 7fc866e00640 1 Processor -- start 2024-04-25T09:18:35.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.024+0000 7fc866e00640 1 -- start start 2024-04-25T09:18:35.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.025+0000 7fc866e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc86015bea0 0x7fc8600ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.025+0000 7fc866e00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86015f7c0 con 0x7fc8600ce960 2024-04-25T09:18:35.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.025+0000 7fc866e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86015faa0 con 0x7fc86015dc90 2024-04-25T09:18:35.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.025+0000 7fc866e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8600cedb0 con 0x7fc86015bea0 2024-04-25T09:18:35.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.026+0000 7fc864e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc8600ce960 0x7fc860160860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37976/0 (socket says 172.21.15.62:37976) 2024-04-25T09:18:35.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.026+0000 7fc864e00640 1 -- 172.21.15.62:0/2712234737 learned_addr learned my addr 172.21.15.62:0/2712234737 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:35.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.026+0000 7fc85f400640 1 --2- 172.21.15.62:0/2712234737 >> v2:172.21.15.121:3300/0 conn(0x7fc86015bea0 0x7fc8600ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.026+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804908966 0 0) 0x7fc86015f7c0 con 0x7fc8600ce960 2024-04-25T09:18:35.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.026+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc844003520 con 0x7fc8600ce960 2024-04-25T09:18:35.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1718130668 0 0) 0x7fc844003520 con 0x7fc8600ce960 2024-04-25T09:18:35.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 >> v2:172.21.15.121:3300/0 conn(0x7fc86015bea0 msgr2=0x7fc8600ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 --2- 172.21.15.62:0/2712234737 >> v2:172.21.15.121:3300/0 conn(0x7fc86015bea0 0x7fc8600ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 >> v1:172.21.15.121:6789/0 conn(0x7fc86015dc90 legacy=0x7fc86015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86015fd80 con 0x7fc8600ce960 2024-04-25T09:18:35.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc858001670 con 0x7fc8600ce960 2024-04-25T09:18:35.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 >> v1:172.21.15.62:6789/0 conn(0x7fc8600ce960 legacy=0x7fc860160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc844003a00 0x7fc844003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc844004800 0x7fc844004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc844005210 0x7fc84400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8600cedb0 con 0x7fc844005210 2024-04-25T09:18:35.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84400c160 con 0x7fc844003a00 2024-04-25T09:18:35.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84400c360 con 0x7fc844004800 2024-04-25T09:18:35.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc864e00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc844005210 0x7fc84400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.027+0000 7fc85fe00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc844003a00 0x7fc844003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.028+0000 7fc85f400640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc844004800 0x7fc844004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.028+0000 7fc864e00640 1 -- 172.21.15.62:0/2712234737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc844004800 msgr2=0x7fc844004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.028+0000 7fc864e00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc844004800 0x7fc844004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.028+0000 7fc864e00640 1 -- 172.21.15.62:0/2712234737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc844003a00 msgr2=0x7fc844003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.028+0000 7fc864e00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc844003a00 0x7fc844003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.018 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.028+0000 7fc864e00640 1 -- 172.21.15.62:0/2712234737 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc844003520 con 0x7fc844005210 2024-04-25T09:18:35.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.028+0000 7fc864e00640 1 -- 172.21.15.62:0/2712234737 reap_dead start 2024-04-25T09:18:35.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.029+0000 7fc864e00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc844005210 0x7fc84400b9c0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fc858007fd0 tx=0x7fc844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff03903e89f52a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.029+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858007c20 con 0x7fc844005210 2024-04-25T09:18:35.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.029+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858005af0 con 0x7fc844005210 2024-04-25T09:18:35.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.029+0000 7fc85ea00640 1 -- 172.21.15.62:0/2712234737 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858007070 con 0x7fc844005210 2024-04-25T09:18:35.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.029+0000 7fc866e00640 1 -- 172.21.15.62:0/2712234737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc844005210 msgr2=0x7fc84400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.029+0000 7fc866e00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc844005210 0x7fc84400b9c0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fc858007fd0 tx=0x7fc844005830 comp rx=0 tx=0).stop 2024-04-25T09:18:35.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.030+0000 7fc866e00640 1 -- 172.21.15.62:0/2712234737 shutdown_connections 2024-04-25T09:18:35.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.030+0000 7fc866e00640 1 --2- 172.21.15.62:0/2712234737 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc844005210 0x7fc84400b9c0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.030+0000 7fc866e00640 1 -- 172.21.15.62:0/2712234737 >> 172.21.15.62:0/2712234737 conn(0x7fc86006b000 msgr2=0x7fc86006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:35.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.030+0000 7fc866e00640 1 -- 172.21.15.62:0/2712234737 shutdown_connections 2024-04-25T09:18:35.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.030+0000 7fc866e00640 1 -- 172.21.15.62:0/2712234737 wait complete. 2024-04-25T09:18:35.023 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.032+0000 7fc866e00640 1 Processor -- start 2024-04-25T09:18:35.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.032+0000 7fc866e00640 1 -- start start 2024-04-25T09:18:35.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc866e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc844005210 0x7fc8600c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc866e00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8600c76f0 0x7fc8600c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc866e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc860145690 0x7fc860145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc866e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86015e070 con 0x7fc860145690 2024-04-25T09:18:35.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc866e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc860160340 con 0x7fc844005210 2024-04-25T09:18:35.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc866e00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8600b2f10 con 0x7fc8600c76f0 2024-04-25T09:18:35.024 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc864e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc860145690 0x7fc860145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc864e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc860145690 0x7fc860145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35226/0 (socket says 172.21.15.62:35226) 2024-04-25T09:18:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc864e00640 1 -- 172.21.15.62:0/1639373955 learned_addr learned my addr 172.21.15.62:0/1639373955 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc85f400640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8600c76f0 0x7fc8600c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc85fe00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc844005210 0x7fc8600c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc864e00640 1 -- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8600c76f0 msgr2=0x7fc8600c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc864e00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8600c76f0 0x7fc8600c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc864e00640 1 -- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc844005210 msgr2=0x7fc8600c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc864e00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc844005210 0x7fc8600c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc864e00640 1 -- 172.21.15.62:0/1639373955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8600b3e50 con 0x7fc860145690 2024-04-25T09:18:35.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.033+0000 7fc85f400640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc8600c76f0 0x7fc8600c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:35.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.034+0000 7fc864e00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc860145690 0x7fc860145b10 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c1d0 tx=0x7fc86015f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.034+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858037040 con 0x7fc860145690 2024-04-25T09:18:35.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.034+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858041070 con 0x7fc860145690 2024-04-25T09:18:35.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.034+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858007270 con 0x7fc860145690 2024-04-25T09:18:35.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.034+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8600b4130 con 0x7fc860145690 2024-04-25T09:18:35.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.034+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8600b46f0 con 0x7fc860145690 2024-04-25T09:18:35.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.035+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc860068350 con 0x7fc860145690 2024-04-25T09:18:35.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.037+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc858007410 con 0x7fc860145690 2024-04-25T09:18:35.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.037+0000 7fc857e00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84006b500 0x7fc84006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.038+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc8580aa940 con 0x7fc860145690 2024-04-25T09:18:35.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.038+0000 7fc857e00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fc8400713c0 0x7fc8400737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.038+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 --> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc858031e80 con 0x7fc8400713c0 2024-04-25T09:18:35.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.038+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8580aacf0 con 0x7fc860145690 2024-04-25T09:18:35.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.038+0000 7fc85fe00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84006b500 0x7fc84006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.038+0000 7fc85f400640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fc8400713c0 0x7fc8400737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.040+0000 7fc85f400640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fc8400713c0 0x7fc8400737e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.040+0000 7fc85fe00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84006b500 0x7fc84006d9c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc850007fd0 tx=0x7fc850009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.042+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 <== osd.4 v2:172.21.15.121:6808/3370771089 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc858031e80 con 0x7fc8400713c0 2024-04-25T09:18:35.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.058+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 --> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc8600d2a50 con 0x7fc8400713c0 2024-04-25T09:18:35.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.059+0000 7fc857e00640 1 -- 172.21.15.62:0/1639373955 <== osd.4 v2:172.21.15.121:6808/3370771089 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc8600d2a50 con 0x7fc8400713c0 2024-04-25T09:18:35.047 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:35.047 INFO:teuthology.orchestra.run.smithi062.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:18:35.048 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fc8400713c0 msgr2=0x7fc8400737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc866e00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6808/3370771089,v1:172.21.15.121:6809/3370771089] conn(0x7fc8400713c0 0x7fc8400737e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84006b500 msgr2=0x7fc84006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc866e00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc84006b500 0x7fc84006d9c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc850007fd0 tx=0x7fc850009040 comp rx=0 tx=0).stop 2024-04-25T09:18:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc860145690 msgr2=0x7fc860145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc866e00640 1 --2- 172.21.15.62:0/1639373955 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc860145690 0x7fc860145b10 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c1d0 tx=0x7fc86015f990 comp rx=0 tx=0).stop 2024-04-25T09:18:35.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc864e00640 1 -- 172.21.15.62:0/1639373955 reap_dead start 2024-04-25T09:18:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 shutdown_connections 2024-04-25T09:18:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.060+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 >> 172.21.15.62:0/1639373955 conn(0x7fc86006b000 msgr2=0x7fc86015ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.061+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 shutdown_connections 2024-04-25T09:18:35.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.061+0000 7fc866e00640 1 -- 172.21.15.62:0/1639373955 wait complete. 2024-04-25T09:18:35.067 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T09:18:35.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.293+0000 7fb52e800640 1 Processor -- start 2024-04-25T09:18:35.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.294+0000 7fb52e800640 1 -- start start 2024-04-25T09:18:35.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.295+0000 7fb52e800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528166200 0x7fb52806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.295+0000 7fb52e800640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb528070460 con 0x7fb52815fa10 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.295+0000 7fb52e800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb528070740 con 0x7fb5280b2710 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.295+0000 7fb52e800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280b2e60 con 0x7fb528166200 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.296+0000 7fb52d400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb52815fa10 0x7fb528165af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37978/0 (socket says 172.21.15.62:37978) 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.296+0000 7fb52d400640 1 -- 172.21.15.62:0/3452303121 learned_addr learned my addr 172.21.15.62:0/3452303121 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.296+0000 7fb52ca00640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528166200 0x7fb52806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.296+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204315400 0 0) 0x7fb528070460 con 0x7fb52815fa10 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.296+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb510003520 con 0x7fb52815fa10 2024-04-25T09:18:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 934639101 0 0) 0x7fb510003520 con 0x7fb52815fa10 2024-04-25T09:18:35.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528166200 msgr2=0x7fb52806e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528166200 0x7fb52806e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 >> v1:172.21.15.121:6789/0 conn(0x7fb5280b2710 legacy=0x7fb52806c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528070a20 con 0x7fb52815fa10 2024-04-25T09:18:35.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fb51c001670 con 0x7fb52815fa10 2024-04-25T09:18:35.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 >> v1:172.21.15.62:6789/0 conn(0x7fb52815fa10 legacy=0x7fb528165af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5100039e0 0x7fb510003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528166200 0x7fb510004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb510004d40 0x7fb5100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280b2e60 con 0x7fb5100039e0 2024-04-25T09:18:35.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51000bad0 con 0x7fb510004d40 2024-04-25T09:18:35.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51000bcd0 con 0x7fb528166200 2024-04-25T09:18:35.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.297+0000 7fb52d400640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5100039e0 0x7fb510003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.298+0000 7fb52ca00640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528166200 0x7fb510004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.298+0000 7fb52d400640 1 -- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528166200 msgr2=0x7fb510004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.298+0000 7fb52de00640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb510004d40 0x7fb5100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.298+0000 7fb52d400640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528166200 0x7fb510004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.298+0000 7fb52d400640 1 -- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb510004d40 msgr2=0x7fb5100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.298+0000 7fb52d400640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb510004d40 0x7fb5100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.298+0000 7fb52d400640 1 -- 172.21.15.62:0/3452303121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb510003520 con 0x7fb5100039e0 2024-04-25T09:18:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.298+0000 7fb52de00640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb510004d40 0x7fb5100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:35.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.299+0000 7fb52d400640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5100039e0 0x7fb510003e00 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fb51c007fd0 tx=0x7fb510005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed67bb8ab4fcfc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.299+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c007c20 con 0x7fb5100039e0 2024-04-25T09:18:35.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.299+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c005af0 con 0x7fb5100039e0 2024-04-25T09:18:35.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.299+0000 7fb527e00640 1 -- 172.21.15.62:0/3452303121 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c007070 con 0x7fb5100039e0 2024-04-25T09:18:35.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.300+0000 7fb52e800640 1 -- 172.21.15.62:0/3452303121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5100039e0 msgr2=0x7fb510003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.300+0000 7fb52e800640 1 --2- 172.21.15.62:0/3452303121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5100039e0 0x7fb510003e00 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fb51c007fd0 tx=0x7fb510005830 comp rx=0 tx=0).stop 2024-04-25T09:18:35.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.300+0000 7fb52de00640 1 -- 172.21.15.62:0/3452303121 reap_dead start 2024-04-25T09:18:35.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.300+0000 7fb52e800640 1 -- 172.21.15.62:0/3452303121 shutdown_connections 2024-04-25T09:18:35.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.300+0000 7fb52e800640 1 -- 172.21.15.62:0/3452303121 >> 172.21.15.62:0/3452303121 conn(0x7fb5280668f0 msgr2=0x7fb528066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:35.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.300+0000 7fb52e800640 1 -- 172.21.15.62:0/3452303121 shutdown_connections 2024-04-25T09:18:35.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.300+0000 7fb52e800640 1 -- 172.21.15.62:0/3452303121 wait complete. 2024-04-25T09:18:35.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.302+0000 7fb52e800640 1 Processor -- start 2024-04-25T09:18:35.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.302+0000 7fb52e800640 1 -- start start 2024-04-25T09:18:35.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5100039e0 0x7fb5281f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52e800640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528158610 0x7fb52815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb52814a520 0x7fb52814a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528071220 con 0x7fb52814a520 2024-04-25T09:18:35.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52e800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280b2b80 con 0x7fb5100039e0 2024-04-25T09:18:35.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52e800640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815fdf0 con 0x7fb528158610 2024-04-25T09:18:35.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb52814a520 0x7fb52814a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb52814a520 0x7fb52814a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35234/0 (socket says 172.21.15.62:35234) 2024-04-25T09:18:35.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52de00640 1 -- 172.21.15.62:0/3788961612 learned_addr learned my addr 172.21.15.62:0/3788961612 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:35.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.303+0000 7fb52ca00640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528158610 0x7fb52815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52de00640 1 -- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528158610 msgr2=0x7fb52815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52de00640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528158610 0x7fb52815dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52de00640 1 -- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5100039e0 msgr2=0x7fb5281f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52d400640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5100039e0 0x7fb5281f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52de00640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5100039e0 0x7fb5281f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52de00640 1 -- 172.21.15.62:0/3788961612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5281f8b10 con 0x7fb52814a520 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52d400640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5100039e0 0x7fb5281f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52de00640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb52814a520 0x7fb52814a980 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb528071db0 tx=0x7fb52000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb525600640 1 -- 172.21.15.62:0/3788961612 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520006d50 con 0x7fb52814a520 2024-04-25T09:18:35.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb525600640 1 -- 172.21.15.62:0/3788961612 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb520003970 con 0x7fb52814a520 2024-04-25T09:18:35.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb528157570 con 0x7fb52814a520 2024-04-25T09:18:35.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb525600640 1 -- 172.21.15.62:0/3788961612 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520003da0 con 0x7fb52814a520 2024-04-25T09:18:35.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.304+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb528157b30 con 0x7fb52814a520 2024-04-25T09:18:35.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.306+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52804ef20 con 0x7fb52814a520 2024-04-25T09:18:35.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.307+0000 7fb525600640 1 -- 172.21.15.62:0/3788961612 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb52001b020 con 0x7fb52814a520 2024-04-25T09:18:35.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.307+0000 7fb525600640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb50806b500 0x7fb50806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.308+0000 7fb525600640 1 -- 172.21.15.62:0/3788961612 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb520089c60 con 0x7fb52814a520 2024-04-25T09:18:35.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.308+0000 7fb52d400640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb50806b500 0x7fb50806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.309+0000 7fb52d400640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb50806b500 0x7fb50806d9c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb51c032000 tx=0x7fb51c009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.312+0000 7fb525600640 1 -- 172.21.15.62:0/3788961612 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5281f8b10 con 0x7fb52814a520 2024-04-25T09:18:35.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.513+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb528071de0 con 0x7fb50806b500 2024-04-25T09:18:35.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.515+0000 7fb525600640 1 -- 172.21.15.62:0/3788961612 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb528071de0 con 0x7fb50806b500 2024-04-25T09:18:35.502 INFO:teuthology.orchestra.run.smithi062.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T09:18:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.517+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb50806b500 msgr2=0x7fb50806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.517+0000 7fb52e800640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb50806b500 0x7fb50806d9c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb51c032000 tx=0x7fb51c009030 comp rx=0 tx=0).stop 2024-04-25T09:18:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.517+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb52814a520 msgr2=0x7fb52814a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.517+0000 7fb52e800640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb52814a520 0x7fb52814a980 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb528071db0 tx=0x7fb52000dc00 comp rx=0 tx=0).stop 2024-04-25T09:18:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.518+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 shutdown_connections 2024-04-25T09:18:35.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.518+0000 7fb52e800640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fb50806b500 0x7fb50806d9c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.518+0000 7fb52e800640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb52814a520 0x7fb52814a980 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.518+0000 7fb52e800640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fb528158610 0x7fb52815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.518+0000 7fb52e800640 1 --2- 172.21.15.62:0/3788961612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5100039e0 0x7fb5281f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.518+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 >> 172.21.15.62:0/3788961612 conn(0x7fb5280668f0 msgr2=0x7fb5280ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.518+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 shutdown_connections 2024-04-25T09:18:35.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.518+0000 7fb52e800640 1 -- 172.21.15.62:0/3788961612 wait complete. 2024-04-25T09:18:35.523 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T09:18:35.524 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:18:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.754+0000 7f9bdee00640 1 Processor -- start 2024-04-25T09:18:35.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.754+0000 7f9bdee00640 1 -- start start 2024-04-25T09:18:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.756+0000 7f9bdee00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9bd81612b0 0x7f9bd81622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.756+0000 7f9bdee00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9bd80668f0 0x7f9bd8066d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.756+0000 7f9bdee00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd8162800 con 0x7f9bd80af960 2024-04-25T09:18:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.756+0000 7f9bdee00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd8161690 con 0x7f9bd80668f0 2024-04-25T09:18:35.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.756+0000 7f9bdee00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd80b40e0 con 0x7f9bd81612b0 2024-04-25T09:18:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.756+0000 7f9bd7400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9bd80af960 0x7f9bd80b39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:37992/0 (socket says 172.21.15.62:37992) 2024-04-25T09:18:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.756+0000 7f9bd7400640 1 -- 172.21.15.62:0/2198781891 learned_addr learned my addr 172.21.15.62:0/2198781891 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:35.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.756+0000 7f9bdce00640 1 --2- 172.21.15.62:0/2198781891 >> v2:172.21.15.62:3300/0 conn(0x7f9bd80668f0 0x7f9bd8066d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.757+0000 7f9bd7e00640 1 --2- 172.21.15.62:0/2198781891 >> v2:172.21.15.121:3300/0 conn(0x7f9bd81612b0 0x7f9bd81622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.757+0000 7f9bdce00640 1 -- 172.21.15.62:0/2198781891 >> v2:172.21.15.121:3300/0 conn(0x7f9bd81612b0 msgr2=0x7f9bd81622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.757+0000 7f9bdce00640 1 --2- 172.21.15.62:0/2198781891 >> v2:172.21.15.121:3300/0 conn(0x7f9bd81612b0 0x7f9bd81622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.757+0000 7f9bdce00640 1 -- 172.21.15.62:0/2198781891 >> v1:172.21.15.62:6789/0 conn(0x7f9bd80af960 legacy=0x7f9bd80b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.757+0000 7f9bd6a00640 1 -- 172.21.15.62:0/2198781891 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545372405 0 0) 0x7f9bc4001110 con 0x7f9bd80af960 2024-04-25T09:18:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.757+0000 7f9bdce00640 1 -- 172.21.15.62:0/2198781891 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd8072290 con 0x7f9bd80668f0 2024-04-25T09:18:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.758+0000 7f9bdce00640 1 --2- 172.21.15.62:0/2198781891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 0x7f9bd8066d80 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80679d0 tx=0x7f9bd002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f581c514df608509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.758+0000 7f9bd6a00640 1 -- 172.21.15.62:0/2198781891 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0007740 con 0x7f9bd80668f0 2024-04-25T09:18:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.758+0000 7f9bd6a00640 1 -- 172.21.15.62:0/2198781891 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd00078e0 con 0x7f9bd80668f0 2024-04-25T09:18:35.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.758+0000 7f9bd6a00640 1 -- 172.21.15.62:0/2198781891 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0007cb0 con 0x7f9bd80668f0 2024-04-25T09:18:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.759+0000 7f9bdee00640 1 -- 172.21.15.62:0/2198781891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 msgr2=0x7f9bd8066d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.759+0000 7f9bdee00640 1 --2- 172.21.15.62:0/2198781891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 0x7f9bd8066d80 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80679d0 tx=0x7f9bd002a680 comp rx=0 tx=0).stop 2024-04-25T09:18:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.759+0000 7f9bdee00640 1 -- 172.21.15.62:0/2198781891 shutdown_connections 2024-04-25T09:18:35.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.759+0000 7f9bdee00640 1 --2- 172.21.15.62:0/2198781891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 0x7f9bd8066d80 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.759+0000 7f9bdee00640 1 --2- 172.21.15.62:0/2198781891 >> v2:172.21.15.121:3300/0 conn(0x7f9bd81612b0 0x7f9bd81622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.759+0000 7f9bdee00640 1 -- 172.21.15.62:0/2198781891 >> 172.21.15.62:0/2198781891 conn(0x7f9bd80b2520 msgr2=0x7f9bd80b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.759+0000 7f9bdee00640 1 -- 172.21.15.62:0/2198781891 shutdown_connections 2024-04-25T09:18:35.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.759+0000 7f9bdee00640 1 -- 172.21.15.62:0/2198781891 wait complete. 2024-04-25T09:18:35.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.761+0000 7f9bdee00640 1 Processor -- start 2024-04-25T09:18:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.761+0000 7f9bdee00640 1 -- start start 2024-04-25T09:18:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bdee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 0x7f9bd80cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bdee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd80af960 0x7f9bd815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bdee00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bd81612b0 0x7f9bd81f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bdee00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80b5520 con 0x7f9bd80668f0 2024-04-25T09:18:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bdee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80b5900 con 0x7f9bd80af960 2024-04-25T09:18:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bdee00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80b56a0 con 0x7f9bd81612b0 2024-04-25T09:18:35.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bd7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 0x7f9bd80cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bd7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 0x7f9bd80cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35256/0 (socket says 172.21.15.62:35256) 2024-04-25T09:18:35.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.762+0000 7f9bd7e00640 1 -- 172.21.15.62:0/1303919695 learned_addr learned my addr 172.21.15.62:0/1303919695 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:35.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bd7400640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd80af960 0x7f9bd815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bdce00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bd81612b0 0x7f9bd81f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bd7e00640 1 -- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bd81612b0 msgr2=0x7f9bd81f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bd7e00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f9bd81612b0 0x7f9bd81f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bd7e00640 1 -- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd80af960 msgr2=0x7f9bd815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bd7e00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd80af960 0x7f9bd815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bd7e00640 1 -- 172.21.15.62:0/1303919695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80cffb0 con 0x7f9bd80668f0 2024-04-25T09:18:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bd7400640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bd80af960 0x7f9bd815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.763+0000 7f9bd7e00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 0x7f9bd80cf920 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d800 tx=0x7f9bc800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.764+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800b6c0 con 0x7f9bd80668f0 2024-04-25T09:18:35.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.764+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc8019070 con 0x7f9bd80668f0 2024-04-25T09:18:35.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.764+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800ba80 con 0x7f9bd80668f0 2024-04-25T09:18:35.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.764+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd80cd530 con 0x7f9bd80668f0 2024-04-25T09:18:35.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.764+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd80cdaf0 con 0x7f9bd80668f0 2024-04-25T09:18:35.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.765+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9bd8068350 con 0x7f9bd80668f0 2024-04-25T09:18:35.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.766+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bc800bc20 con 0x7f9bd80668f0 2024-04-25T09:18:35.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.767+0000 7f9bcfe00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bb806b4d0 0x7f9bb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.767+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9bc808a3b0 con 0x7f9bd80668f0 2024-04-25T09:18:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.767+0000 7f9bcfe00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7f9bb8071390 0x7f9bb80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.767+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 --> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bc8008e50 con 0x7f9bb8071390 2024-04-25T09:18:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.767+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bc8091050 con 0x7f9bd80668f0 2024-04-25T09:18:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.768+0000 7f9bd7400640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bb806b4d0 0x7f9bb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.768+0000 7f9bdce00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7f9bb8071390 0x7f9bb80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.769+0000 7f9bd7400640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bb806b4d0 0x7f9bb806d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40013b0 tx=0x7f9bc4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.769+0000 7f9bdce00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7f9bb8071390 0x7f9bb80737b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.772+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 <== osd.5 v2:172.21.15.121:6816/3527509140 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9bc8008e50 con 0x7f9bb8071390 2024-04-25T09:18:35.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.786+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 --> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9bd8071c10 con 0x7f9bb8071390 2024-04-25T09:18:35.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.787+0000 7f9bcfe00640 1 -- 172.21.15.62:0/1303919695 <== osd.5 v2:172.21.15.121:6816/3527509140 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9bd8071c10 con 0x7f9bb8071390 2024-04-25T09:18:35.775 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:35.775 INFO:teuthology.orchestra.run.smithi062.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:18:35.775 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7f9bb8071390 msgr2=0x7f9bb80737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6816/3527509140,v1:172.21.15.121:6817/3527509140] conn(0x7f9bb8071390 0x7f9bb80737b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bb806b4d0 msgr2=0x7f9bb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f9bb806b4d0 0x7f9bb806d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40013b0 tx=0x7f9bc4009040 comp rx=0 tx=0).stop 2024-04-25T09:18:35.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 msgr2=0x7f9bd80cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 --2- 172.21.15.62:0/1303919695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bd80668f0 0x7f9bd80cf920 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d800 tx=0x7f9bc800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdce00640 1 -- 172.21.15.62:0/1303919695 reap_dead start 2024-04-25T09:18:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 shutdown_connections 2024-04-25T09:18:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 >> 172.21.15.62:0/1303919695 conn(0x7f9bd80b2520 msgr2=0x7f9bd806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:35.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 shutdown_connections 2024-04-25T09:18:35.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:35.788+0000 7f9bdee00640 1 -- 172.21.15.62:0/1303919695 wait complete. 2024-04-25T09:18:35.793 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T09:18:35.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.003+0000 7fc925000640 1 Processor -- start 2024-04-25T09:18:35.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.003+0000 7fc925000640 1 -- start start 2024-04-25T09:18:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.005+0000 7fc925000640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9200aef10 0x7fc9200b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.005+0000 7fc925000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc9200b36d0 0x7fc92015b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.005+0000 7fc925000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92006d8a0 con 0x7fc9200aeb30 2024-04-25T09:18:35.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.005+0000 7fc925000640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92006db60 con 0x7fc9200aef10 2024-04-25T09:18:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.005+0000 7fc925000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92006de40 con 0x7fc9200b36d0 2024-04-25T09:18:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.005+0000 7fc916a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9200aef10 0x7fc9200b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.005+0000 7fc916a00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc9200aef10 0x7fc9200b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35272/0 (socket says 172.21.15.62:35272) 2024-04-25T09:18:35.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.005+0000 7fc916a00640 1 -- 172.21.15.62:0/419560571 learned_addr learned my addr 172.21.15.62:0/419560571 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:35.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.006+0000 7fc916a00640 1 -- 172.21.15.62:0/419560571 >> v2:172.21.15.121:3300/0 conn(0x7fc9200b36d0 msgr2=0x7fc92015b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.006+0000 7fc916a00640 1 --2- 172.21.15.62:0/419560571 >> v2:172.21.15.121:3300/0 conn(0x7fc9200b36d0 0x7fc92015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.006+0000 7fc916a00640 1 -- 172.21.15.62:0/419560571 >> v1:172.21.15.121:6789/0 conn(0x7fc9200aeb30 legacy=0x7fc92006ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.006+0000 7fc916a00640 1 -- 172.21.15.62:0/419560571 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92015b150 con 0x7fc9200aef10 2024-04-25T09:18:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.007+0000 7fc916a00640 1 --2- 172.21.15.62:0/419560571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b3190 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc90800bb10 tx=0x7fc90802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760b9b700fe6ffb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.007+0000 7fc91ea00640 1 -- 172.21.15.62:0/419560571 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908032750 con 0x7fc9200aef10 2024-04-25T09:18:35.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.007+0000 7fc91ea00640 1 -- 172.21.15.62:0/419560571 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc908030050 con 0x7fc9200aef10 2024-04-25T09:18:35.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.007+0000 7fc91ea00640 1 -- 172.21.15.62:0/419560571 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908032b10 con 0x7fc9200aef10 2024-04-25T09:18:35.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.008+0000 7fc925000640 1 -- 172.21.15.62:0/419560571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 msgr2=0x7fc9200b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:35.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.008+0000 7fc925000640 1 --2- 172.21.15.62:0/419560571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b3190 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc90800bb10 tx=0x7fc90802c890 comp rx=0 tx=0).stop 2024-04-25T09:18:35.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.008+0000 7fc925000640 1 -- 172.21.15.62:0/419560571 shutdown_connections 2024-04-25T09:18:35.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.008+0000 7fc925000640 1 --2- 172.21.15.62:0/419560571 >> v2:172.21.15.121:3300/0 conn(0x7fc9200b36d0 0x7fc92015b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.008+0000 7fc925000640 1 --2- 172.21.15.62:0/419560571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b3190 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:35.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.008+0000 7fc925000640 1 -- 172.21.15.62:0/419560571 >> 172.21.15.62:0/419560571 conn(0x7fc9200668f0 msgr2=0x7fc920066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:35.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.008+0000 7fc925000640 1 -- 172.21.15.62:0/419560571 shutdown_connections 2024-04-25T09:18:35.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.008+0000 7fc925000640 1 -- 172.21.15.62:0/419560571 wait complete. 2024-04-25T09:18:36.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.011+0000 7fc925000640 1 Processor -- start 2024-04-25T09:18:36.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.011+0000 7fc925000640 1 -- start start 2024-04-25T09:18:36.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc925000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9200aeb30 0x7fc9201f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc925000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc925000640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9200b36d0 0x7fc9201f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc925000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92015e780 con 0x7fc9200aef10 2024-04-25T09:18:36.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc925000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92015e600 con 0x7fc9200aeb30 2024-04-25T09:18:36.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc925000640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92015ad10 con 0x7fc9200b36d0 2024-04-25T09:18:36.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc916a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc916a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35274/0 (socket says 172.21.15.62:35274) 2024-04-25T09:18:36.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.012+0000 7fc916a00640 1 -- 172.21.15.62:0/3597425510 learned_addr learned my addr 172.21.15.62:0/3597425510 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:36.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc91f400640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9200aeb30 0x7fc9201f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc91fe00640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9200b36d0 0x7fc9201f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc916a00640 1 -- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9200b36d0 msgr2=0x7fc9201f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc916a00640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9200b36d0 0x7fc9201f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc916a00640 1 -- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9200aeb30 msgr2=0x7fc9201f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc916a00640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9200aeb30 0x7fc9201f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc916a00640 1 -- 172.21.15.62:0/3597425510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200b0350 con 0x7fc9200aef10 2024-04-25T09:18:36.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc91fe00640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9200b36d0 0x7fc9201f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:36.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.013+0000 7fc916a00640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b0f10 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc90800b2c0 tx=0x7fc9080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.014+0000 7fc917e00640 1 -- 172.21.15.62:0/3597425510 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908034c00 con 0x7fc9200aef10 2024-04-25T09:18:36.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.014+0000 7fc917e00640 1 -- 172.21.15.62:0/3597425510 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc908034dc0 con 0x7fc9200aef10 2024-04-25T09:18:36.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.014+0000 7fc917e00640 1 -- 172.21.15.62:0/3597425510 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908004270 con 0x7fc9200aef10 2024-04-25T09:18:36.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.014+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc920156d40 con 0x7fc9200aef10 2024-04-25T09:18:36.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.014+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc920157300 con 0x7fc9200aef10 2024-04-25T09:18:36.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.015+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc920156b30 con 0x7fc9200aef10 2024-04-25T09:18:36.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.016+0000 7fc917e00640 1 -- 172.21.15.62:0/3597425510 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc908003610 con 0x7fc9200aef10 2024-04-25T09:18:36.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.018+0000 7fc917e00640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.018+0000 7fc917e00640 1 -- 172.21.15.62:0/3597425510 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc9080ab430 con 0x7fc9200aef10 2024-04-25T09:18:36.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.018+0000 7fc91f400640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.019+0000 7fc91f400640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc910009f60 tx=0x7fc9100094f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.022+0000 7fc917e00640 1 -- 172.21.15.62:0/3597425510 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc90804d070 con 0x7fc9200aef10 2024-04-25T09:18:36.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.221+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fc92004ef20 con 0x7fc8fc06b4d0 2024-04-25T09:18:36.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.223+0000 7fc917e00640 1 -- 172.21.15.62:0/3597425510 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc92004ef20 con 0x7fc8fc06b4d0 2024-04-25T09:18:36.211 INFO:teuthology.orchestra.run.smithi062.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T09:18:36.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.224+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8fc06b4d0 msgr2=0x7fc8fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.224+0000 7fc925000640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc910009f60 tx=0x7fc9100094f0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.224+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 msgr2=0x7fc9200b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.224+0000 7fc925000640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b0f10 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc90800b2c0 tx=0x7fc9080046c0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.225+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 shutdown_connections 2024-04-25T09:18:36.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.225+0000 7fc925000640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.225+0000 7fc925000640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9200aef10 0x7fc9200b0f10 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.225+0000 7fc925000640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc9200b36d0 0x7fc9201f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.225+0000 7fc925000640 1 --2- 172.21.15.62:0/3597425510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9200aeb30 0x7fc9201f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.225+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 >> 172.21.15.62:0/3597425510 conn(0x7fc9200668f0 msgr2=0x7fc920155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:36.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.225+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 shutdown_connections 2024-04-25T09:18:36.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.225+0000 7fc925000640 1 -- 172.21.15.62:0/3597425510 wait complete. 2024-04-25T09:18:36.230 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T09:18:36.231 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:18:36.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.463+0000 7fedfb200640 1 Processor -- start 2024-04-25T09:18:36.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.464+0000 7fedfb200640 1 -- start start 2024-04-25T09:18:36.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.465+0000 7fedfb200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fedf4161270 0x7fedf40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.465+0000 7fedfb200640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf4162980 con 0x7fedf40d2870 2024-04-25T09:18:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.465+0000 7fedfb200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf40d1480 con 0x7fedf4161650 2024-04-25T09:18:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.465+0000 7fedfb200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf40d3900 con 0x7fedf4161270 2024-04-25T09:18:36.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.466+0000 7fedfa800640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fedf40d2870 0x7fedf4162270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38008/0 (socket says 172.21.15.62:38008) 2024-04-25T09:18:36.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.466+0000 7fedfa800640 1 -- 172.21.15.62:0/2091554588 learned_addr learned my addr 172.21.15.62:0/2091554588 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:36.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.466+0000 7fedf9e00640 1 --2- 172.21.15.62:0/2091554588 >> v2:172.21.15.121:3300/0 conn(0x7fedf4161270 0x7fedf40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.466+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394829493 0 0) 0x7fedf4162980 con 0x7fedf40d2870 2024-04-25T09:18:36.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.466+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fede0003520 con 0x7fedf40d2870 2024-04-25T09:18:36.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4240291464 0 0) 0x7fede0003520 con 0x7fedf40d2870 2024-04-25T09:18:36.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 >> v2:172.21.15.121:3300/0 conn(0x7fedf4161270 msgr2=0x7fedf40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:36.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 --2- 172.21.15.62:0/2091554588 >> v2:172.21.15.121:3300/0 conn(0x7fedf4161270 0x7fedf40b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 >> v1:172.21.15.121:6789/0 conn(0x7fedf4161650 legacy=0x7fedf4161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40d11a0 con 0x7fedf40d2870 2024-04-25T09:18:36.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fedf0001670 con 0x7fedf40d2870 2024-04-25T09:18:36.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 >> v1:172.21.15.62:6789/0 conn(0x7fedf40d2870 legacy=0x7fedf4162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fede0003a00 0x7fede0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fede0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fede00052e0 0x7fede000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40d3900 con 0x7fede0004800 2024-04-25T09:18:36.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede000c1e0 con 0x7fede0003a00 2024-04-25T09:18:36.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede000c3e0 con 0x7fede00052e0 2024-04-25T09:18:36.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.467+0000 7fedf9400640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fede0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedf9e00640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fede0003a00 0x7fede0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedfa800640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fede00052e0 0x7fede000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedf9400640 1 -- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fede00052e0 msgr2=0x7fede000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:36.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedf9400640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fede00052e0 0x7fede000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedf9400640 1 -- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fede0003a00 msgr2=0x7fede0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:36.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedf9400640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fede0003a00 0x7fede0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedf9400640 1 -- 172.21.15.62:0/2091554588 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0003520 con 0x7fede0004800 2024-04-25T09:18:36.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedfa800640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fede00052e0 0x7fede000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:36.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.468+0000 7fedfa800640 1 -- 172.21.15.62:0/2091554588 reap_dead start 2024-04-25T09:18:36.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.469+0000 7fedf9400640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fede0004c80 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fede4001750 tx=0x7fede0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b242d809bf9622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.469+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4001370 con 0x7fede0004800 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.469+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede40052d0 con 0x7fede0004800 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.469+0000 7fedf8a00640 1 -- 172.21.15.62:0/2091554588 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede402abc0 con 0x7fede0004800 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.470+0000 7fedfb200640 1 -- 172.21.15.62:0/2091554588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 msgr2=0x7fede0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.470+0000 7fedfb200640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fede0004c80 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fede4001750 tx=0x7fede0005830 comp rx=0 tx=0).stop 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.470+0000 7fedfb200640 1 -- 172.21.15.62:0/2091554588 shutdown_connections 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.470+0000 7fedfb200640 1 --2- 172.21.15.62:0/2091554588 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fede0004c80 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.470+0000 7fedfb200640 1 -- 172.21.15.62:0/2091554588 >> 172.21.15.62:0/2091554588 conn(0x7fedf4066d10 msgr2=0x7fedf4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.470+0000 7fedfb200640 1 -- 172.21.15.62:0/2091554588 shutdown_connections 2024-04-25T09:18:36.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.470+0000 7fedfb200640 1 -- 172.21.15.62:0/2091554588 wait complete. 2024-04-25T09:18:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.472+0000 7fedfb200640 1 Processor -- start 2024-04-25T09:18:36.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.472+0000 7fedfb200640 1 -- start start 2024-04-25T09:18:36.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedfb200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fedf40b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedfb200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fedf4145970 0x7fedf40c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedfb200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedf4145dd0 0x7fedf40c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedfb200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40d3590 con 0x7fede0004800 2024-04-25T09:18:36.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedfb200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40cf160 con 0x7fedf4145dd0 2024-04-25T09:18:36.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedfb200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40b2ed0 con 0x7fedf4145970 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedf9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fedf40b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedf9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fedf40b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35294/0 (socket says 172.21.15.62:35294) 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.473+0000 7fedf9e00640 1 -- 172.21.15.62:0/3957875361 learned_addr learned my addr 172.21.15.62:0/3957875361 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedf9400640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fedf4145970 0x7fedf40c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedfa800640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedf4145dd0 0x7fedf40c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedf9e00640 1 -- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fedf4145970 msgr2=0x7fedf40c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedf9e00640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fedf4145970 0x7fedf40c6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedf9e00640 1 -- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedf4145dd0 msgr2=0x7fedf40c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedf9e00640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedf4145dd0 0x7fedf40c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedf9e00640 1 -- 172.21.15.62:0/3957875361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf41e7880 con 0x7fede0004800 2024-04-25T09:18:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedfa800640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedf4145dd0 0x7fedf40c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedf9e00640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fedf40b37b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fedec007f60 tx=0x7fedec007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec01a020 con 0x7fede0004800 2024-04-25T09:18:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec016470 con 0x7fede0004800 2024-04-25T09:18:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf41e7b40 con 0x7fede0004800 2024-04-25T09:18:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.474+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec016860 con 0x7fede0004800 2024-04-25T09:18:36.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.475+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf41e8100 con 0x7fede0004800 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.475+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fedf4072910 con 0x7fede0004800 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.477+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedec005d10 con 0x7fede0004800 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.478+0000 7fedea000640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fedd806b4d0 0x7fedd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.478+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fedec08ae50 con 0x7fede0004800 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.478+0000 7fedea000640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fedd8071390 0x7fedd80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.478+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 --> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedec012e90 con 0x7fedd8071390 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.478+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedec092050 con 0x7fede0004800 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.478+0000 7fedf9400640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fedd806b4d0 0x7fedd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.479+0000 7fedfa800640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fedd8071390 0x7fedd80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.479+0000 7fedf9400640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fedd806b4d0 0x7fedd806d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fede4032000 tx=0x7fede402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.480+0000 7fedfa800640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fedd8071390 0x7fedd80737b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.482+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 <== osd.6 v2:172.21.15.121:6800/3195802018 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fedec012e90 con 0x7fedd8071390 2024-04-25T09:18:36.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.497+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 --> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fedf40750e0 con 0x7fedd8071390 2024-04-25T09:18:36.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.498+0000 7fedea000640 1 -- 172.21.15.62:0/3957875361 <== osd.6 v2:172.21.15.121:6800/3195802018 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fedf40750e0 con 0x7fedd8071390 2024-04-25T09:18:36.485 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:36.485 INFO:teuthology.orchestra.run.smithi062.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:18:36.485 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:36.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.498+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fedd8071390 msgr2=0x7fedd80737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.498+0000 7fedfb200640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6800/3195802018,v1:172.21.15.121:6801/3195802018] conn(0x7fedd8071390 0x7fedd80737b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fedd806b4d0 msgr2=0x7fedd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfb200640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fedd806b4d0 0x7fedd806d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fede4032000 tx=0x7fede402e000 comp rx=0 tx=0).stop 2024-04-25T09:18:36.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 msgr2=0x7fedf40b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfb200640 1 --2- 172.21.15.62:0/3957875361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fede0004800 0x7fedf40b37b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fedec007f60 tx=0x7fedec007f90 comp rx=0 tx=0).stop 2024-04-25T09:18:36.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfa800640 1 -- 172.21.15.62:0/3957875361 reap_dead start 2024-04-25T09:18:36.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 shutdown_connections 2024-04-25T09:18:36.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 >> 172.21.15.62:0/3957875361 conn(0x7fedf4066d10 msgr2=0x7fedf40670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:36.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 shutdown_connections 2024-04-25T09:18:36.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.499+0000 7fedfb200640 1 -- 172.21.15.62:0/3957875361 wait complete. 2024-04-25T09:18:36.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T09:18:36.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.723+0000 7fc5c6a00640 1 Processor -- start 2024-04-25T09:18:36.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.724+0000 7fc5c6a00640 1 -- start start 2024-04-25T09:18:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.725+0000 7fc5c6a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc5c015bd10 0x7fc5c00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.725+0000 7fc5c6a00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c006ca70 con 0x7fc5c00cccf0 2024-04-25T09:18:36.712 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.725+0000 7fc5c6a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c015c0f0 con 0x7fc5c00aeed0 2024-04-25T09:18:36.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.725+0000 7fc5c6a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c006d660 con 0x7fc5c015bd10 2024-04-25T09:18:36.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.726+0000 7fc5b7e00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc5c00cccf0 0x7fc5c00ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38020/0 (socket says 172.21.15.62:38020) 2024-04-25T09:18:36.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.726+0000 7fc5b7e00640 1 -- 172.21.15.62:0/2199785099 learned_addr learned my addr 172.21.15.62:0/2199785099 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:36.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.726+0000 7fc5bfe00640 1 --2- 172.21.15.62:0/2199785099 >> v2:172.21.15.121:3300/0 conn(0x7fc5c015bd10 0x7fc5c00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.726+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662316630 0 0) 0x7fc5c006ca70 con 0x7fc5c00cccf0 2024-04-25T09:18:36.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.726+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a4003520 con 0x7fc5c00cccf0 2024-04-25T09:18:36.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3204470003 0 0) 0x7fc5a4003520 con 0x7fc5c00cccf0 2024-04-25T09:18:36.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 >> v2:172.21.15.121:3300/0 conn(0x7fc5c015bd10 msgr2=0x7fc5c00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 --2- 172.21.15.62:0/2199785099 >> v2:172.21.15.121:3300/0 conn(0x7fc5c015bd10 0x7fc5c00cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 >> v1:172.21.15.121:6789/0 conn(0x7fc5c00aeed0 legacy=0x7fc5c00cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c006cd30 con 0x7fc5c00cccf0 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7fc5a8001670 con 0x7fc5c00cccf0 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 >> v1:172.21.15.62:6789/0 conn(0x7fc5c00cccf0 legacy=0x7fc5c00ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5a40039b0 0x7fc5a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a40051c0 0x7fc5a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c006d660 con 0x7fc5a4004800 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a400c060 con 0x7fc5a40039b0 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a400c260 con 0x7fc5a40051c0 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.727+0000 7fc5b7e00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5bfe00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5a40039b0 0x7fc5a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5c4a00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a40051c0 0x7fc5a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5b7e00640 1 -- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a40051c0 msgr2=0x7fc5a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5b7e00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a40051c0 0x7fc5a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5b7e00640 1 -- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5a40039b0 msgr2=0x7fc5a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:36.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5b7e00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5a40039b0 0x7fc5a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5b7e00640 1 -- 172.21.15.62:0/2199785099 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4003520 con 0x7fc5a4004800 2024-04-25T09:18:36.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5c4a00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a40051c0 0x7fc5a400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:36.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.728+0000 7fc5c4a00640 1 -- 172.21.15.62:0/2199785099 reap_dead start 2024-04-25T09:18:36.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.729+0000 7fc5b7e00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800c240 tx=0x7fc5a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd10f517946787a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.729+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8007cf0 con 0x7fc5a4004800 2024-04-25T09:18:36.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.729+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a80058d0 con 0x7fc5a4004800 2024-04-25T09:18:36.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.729+0000 7fc5bf400640 1 -- 172.21.15.62:0/2199785099 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8005d20 con 0x7fc5a4004800 2024-04-25T09:18:36.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.730+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2199785099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a4004800 msgr2=0x7fc5a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.730+0000 7fc5c6a00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800c240 tx=0x7fc5a4005830 comp rx=0 tx=0).stop 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.730+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2199785099 shutdown_connections 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.730+0000 7fc5c6a00640 1 --2- 172.21.15.62:0/2199785099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5a4004800 0x7fc5a4004c80 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.730+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2199785099 >> 172.21.15.62:0/2199785099 conn(0x7fc5c00668f0 msgr2=0x7fc5c0066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.730+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2199785099 shutdown_connections 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.730+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2199785099 wait complete. 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.732+0000 7fc5c6a00640 1 Processor -- start 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.732+0000 7fc5c6a00640 1 -- start start 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c6a00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a4004800 0x7fc5c0157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5c01f4a70 0x7fc5c0148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c6a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c015a480 0x7fc5c015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c6a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c006daf0 con 0x7fc5c015a480 2024-04-25T09:18:36.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c6a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c006d2f0 con 0x7fc5c01f4a70 2024-04-25T09:18:36.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c6a00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c0158ee0 con 0x7fc5a4004800 2024-04-25T09:18:36.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c015a480 0x7fc5c015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c4a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c015a480 0x7fc5c015a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35310/0 (socket says 172.21.15.62:35310) 2024-04-25T09:18:36.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5c4a00640 1 -- 172.21.15.62:0/2053200819 learned_addr learned my addr 172.21.15.62:0/2053200819 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:36.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.733+0000 7fc5b7e00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5c01f4a70 0x7fc5c0148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5bfe00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a4004800 0x7fc5c0157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5c4a00640 1 -- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a4004800 msgr2=0x7fc5c0157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5c4a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a4004800 0x7fc5c0157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5c4a00640 1 -- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5c01f4a70 msgr2=0x7fc5c0148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5c4a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5c01f4a70 0x7fc5c0148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5c4a00640 1 -- 172.21.15.62:0/2053200819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c01f4ef0 con 0x7fc5c015a480 2024-04-25T09:18:36.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5c4a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c015a480 0x7fc5c015a900 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fc5c015f790 tx=0x7fc5b800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5bcc00640 1 -- 172.21.15.62:0/2053200819 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8018070 con 0x7fc5c015a480 2024-04-25T09:18:36.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5bcc00640 1 -- 172.21.15.62:0/2053200819 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b8003da0 con 0x7fc5c015a480 2024-04-25T09:18:36.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5bcc00640 1 -- 172.21.15.62:0/2053200819 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8005330 con 0x7fc5c015a480 2024-04-25T09:18:36.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.734+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c01582d0 con 0x7fc5c015a480 2024-04-25T09:18:36.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.735+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c0158890 con 0x7fc5c015a480 2024-04-25T09:18:36.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.736+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c0072c70 con 0x7fc5c015a480 2024-04-25T09:18:36.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.737+0000 7fc5bcc00640 1 -- 172.21.15.62:0/2053200819 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5b801b070 con 0x7fc5c015a480 2024-04-25T09:18:36.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.738+0000 7fc5bcc00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc59c06b4d0 0x7fc59c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:36.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.738+0000 7fc5bcc00640 1 -- 172.21.15.62:0/2053200819 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc5b8089580 con 0x7fc5c015a480 2024-04-25T09:18:36.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.738+0000 7fc5bfe00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc59c06b4d0 0x7fc59c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:36.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.739+0000 7fc5bfe00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc59c06b4d0 0x7fc59c06d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009a90 tx=0x7fc5b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:36.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.742+0000 7fc5bcc00640 1 -- 172.21.15.62:0/2053200819 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5b80586f0 con 0x7fc5c015a480 2024-04-25T09:18:36.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.940+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5c00b2200 con 0x7fc59c06b4d0 2024-04-25T09:18:36.928 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.941+0000 7fc5bcc00640 1 -- 172.21.15.62:0/2053200819 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc5c00b2200 con 0x7fc59c06b4d0 2024-04-25T09:18:36.928 INFO:teuthology.orchestra.run.smithi062.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T09:18:36.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.943+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc59c06b4d0 msgr2=0x7fc59c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.943+0000 7fc5c6a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc59c06b4d0 0x7fc59c06d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009a90 tx=0x7fc5b0008040 comp rx=0 tx=0).stop 2024-04-25T09:18:36.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.943+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c015a480 msgr2=0x7fc5c015a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:36.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.943+0000 7fc5c6a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c015a480 0x7fc5c015a900 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fc5c015f790 tx=0x7fc5b800d9f0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.944+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 shutdown_connections 2024-04-25T09:18:36.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.944+0000 7fc5c6a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fc59c06b4d0 0x7fc59c06d990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.944+0000 7fc5c6a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c015a480 0x7fc5c015a900 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.944+0000 7fc5c6a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5c01f4a70 0x7fc5c0148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.944+0000 7fc5c6a00640 1 --2- 172.21.15.62:0/2053200819 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fc5a4004800 0x7fc5c0157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:36.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.944+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 >> 172.21.15.62:0/2053200819 conn(0x7fc5c00668f0 msgr2=0x7fc5c015e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:36.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.944+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 shutdown_connections 2024-04-25T09:18:36.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:36.944+0000 7fc5c6a00640 1 -- 172.21.15.62:0/2053200819 wait complete. 2024-04-25T09:18:36.949 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T09:18:36.950 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:18:37.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.176+0000 7efd96200640 1 Processor -- start 2024-04-25T09:18:37.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.176+0000 7efd96200640 1 -- start start 2024-04-25T09:18:37.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.178+0000 7efd96200640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efd90154e10 0x7efd9015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.178+0000 7efd96200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7efd900b4ff0 0x7efd9015e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.178+0000 7efd96200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9015b150 con 0x7efd900cd9a0 2024-04-25T09:18:37.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.178+0000 7efd96200640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd90070170 con 0x7efd90154e10 2024-04-25T09:18:37.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.178+0000 7efd96200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9015ef40 con 0x7efd900b4ff0 2024-04-25T09:18:37.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.179+0000 7efd94e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efd90154e10 0x7efd9015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.179+0000 7efd94e00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efd90154e10 0x7efd9015e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35324/0 (socket says 172.21.15.62:35324) 2024-04-25T09:18:37.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.179+0000 7efd94e00640 1 -- 172.21.15.62:0/3409112310 learned_addr learned my addr 172.21.15.62:0/3409112310 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:37.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.179+0000 7efd94e00640 1 -- 172.21.15.62:0/3409112310 >> v2:172.21.15.121:3300/0 conn(0x7efd900b4ff0 msgr2=0x7efd9015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:37.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.179+0000 7efd94e00640 1 --2- 172.21.15.62:0/3409112310 >> v2:172.21.15.121:3300/0 conn(0x7efd900b4ff0 0x7efd9015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.179+0000 7efd94e00640 1 -- 172.21.15.62:0/3409112310 >> v1:172.21.15.121:6789/0 conn(0x7efd900cd9a0 legacy=0x7efd900b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:37.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.179+0000 7efd94e00640 1 -- 172.21.15.62:0/3409112310 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9015f220 con 0x7efd90154e10 2024-04-25T09:18:37.169 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.180+0000 7efd94e00640 1 --2- 172.21.15.62:0/3409112310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd90154e10 0x7efd9015e3d0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7efd901589a0 tx=0x7efd8402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccbbd4ad4c3e0ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.181+0000 7efd8f400640 1 -- 172.21.15.62:0/3409112310 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84037070 con 0x7efd90154e10 2024-04-25T09:18:37.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.181+0000 7efd8f400640 1 -- 172.21.15.62:0/3409112310 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84032440 con 0x7efd90154e10 2024-04-25T09:18:37.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.181+0000 7efd8f400640 1 -- 172.21.15.62:0/3409112310 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84032810 con 0x7efd90154e10 2024-04-25T09:18:37.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.182+0000 7efd96200640 1 -- 172.21.15.62:0/3409112310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd90154e10 msgr2=0x7efd9015e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.182+0000 7efd96200640 1 --2- 172.21.15.62:0/3409112310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd90154e10 0x7efd9015e3d0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7efd901589a0 tx=0x7efd8402a230 comp rx=0 tx=0).stop 2024-04-25T09:18:37.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.182+0000 7efd96200640 1 -- 172.21.15.62:0/3409112310 shutdown_connections 2024-04-25T09:18:37.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.182+0000 7efd96200640 1 --2- 172.21.15.62:0/3409112310 >> v2:172.21.15.121:3300/0 conn(0x7efd900b4ff0 0x7efd9015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.182+0000 7efd96200640 1 --2- 172.21.15.62:0/3409112310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd90154e10 0x7efd9015e3d0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.182+0000 7efd96200640 1 -- 172.21.15.62:0/3409112310 >> 172.21.15.62:0/3409112310 conn(0x7efd90066ad0 msgr2=0x7efd90066ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:37.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.182+0000 7efd96200640 1 -- 172.21.15.62:0/3409112310 shutdown_connections 2024-04-25T09:18:37.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.182+0000 7efd96200640 1 -- 172.21.15.62:0/3409112310 wait complete. 2024-04-25T09:18:37.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.184+0000 7efd96200640 1 Processor -- start 2024-04-25T09:18:37.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.185+0000 7efd96200640 1 -- start start 2024-04-25T09:18:37.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd96200640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd900b4ff0 0x7efd901579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd96200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd900cd9a0 0x7efd90156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd96200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd90154e10 0x7efd90155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd96200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd901551f0 con 0x7efd900cd9a0 2024-04-25T09:18:37.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd96200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd90070710 con 0x7efd90154e10 2024-04-25T09:18:37.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd96200640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd90070c30 con 0x7efd900b4ff0 2024-04-25T09:18:37.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd900cd9a0 0x7efd90156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd8fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd900cd9a0 0x7efd90156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35338/0 (socket says 172.21.15.62:35338) 2024-04-25T09:18:37.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.186+0000 7efd8fe00640 1 -- 172.21.15.62:0/134269630 learned_addr learned my addr 172.21.15.62:0/134269630 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:37.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd94e00640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd900b4ff0 0x7efd901579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd95800640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd90154e10 0x7efd90155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.178 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd8fe00640 1 -- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd900b4ff0 msgr2=0x7efd901579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd8fe00640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7efd900b4ff0 0x7efd901579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd8fe00640 1 -- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd90154e10 msgr2=0x7efd90155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd8fe00640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd90154e10 0x7efd90155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd8fe00640 1 -- 172.21.15.62:0/134269630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9014f670 con 0x7efd900cd9a0 2024-04-25T09:18:37.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd95800640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efd90154e10 0x7efd90155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:18:37.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.187+0000 7efd8fe00640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd900cd9a0 0x7efd90156480 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c400 tx=0x7efd7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.188+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c017070 con 0x7efd900cd9a0 2024-04-25T09:18:37.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.188+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c0035f0 con 0x7efd900cd9a0 2024-04-25T09:18:37.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.188+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c003a20 con 0x7efd900cd9a0 2024-04-25T09:18:37.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.188+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9014f950 con 0x7efd900cd9a0 2024-04-25T09:18:37.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.188+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9014ff10 con 0x7efd900cd9a0 2024-04-25T09:18:37.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.189+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd90072b30 con 0x7efd900cd9a0 2024-04-25T09:18:37.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.190+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd7c01a070 con 0x7efd900cd9a0 2024-04-25T09:18:37.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.191+0000 7efd8cc00640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efd7006b4d0 0x7efd7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.191+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efd7c089cf0 con 0x7efd900cd9a0 2024-04-25T09:18:37.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.191+0000 7efd8cc00640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7efd70071390 0x7efd700737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.191+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 --> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd7c01ae50 con 0x7efd70071390 2024-04-25T09:18:37.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.191+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=153) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd7c08a180 con 0x7efd900cd9a0 2024-04-25T09:18:37.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.192+0000 7efd94e00640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efd7006b4d0 0x7efd7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.192+0000 7efd95800640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7efd70071390 0x7efd700737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.193+0000 7efd94e00640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efd7006b4d0 0x7efd7006d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7efd84034000 tx=0x7efd8402e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.193+0000 7efd95800640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7efd70071390 0x7efd700737b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.195+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 <== osd.7 v2:172.21.15.121:6820/1260954068 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efd7c01ae50 con 0x7efd70071390 2024-04-25T09:18:37.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.210+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 --> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efd900750e0 con 0x7efd70071390 2024-04-25T09:18:37.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.211+0000 7efd8cc00640 1 -- 172.21.15.62:0/134269630 <== osd.7 v2:172.21.15.121:6820/1260954068 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7efd900750e0 con 0x7efd70071390 2024-04-25T09:18:37.198 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T09:18:37.198 INFO:teuthology.orchestra.run.smithi062.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:18:37.199 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T09:18:37.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7efd70071390 msgr2=0x7efd700737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6820/1260954068,v1:172.21.15.121:6821/1260954068] conn(0x7efd70071390 0x7efd700737b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efd7006b4d0 msgr2=0x7efd7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7efd7006b4d0 0x7efd7006d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7efd84034000 tx=0x7efd8402e000 comp rx=0 tx=0).stop 2024-04-25T09:18:37.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd900cd9a0 msgr2=0x7efd90156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 --2- 172.21.15.62:0/134269630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efd900cd9a0 0x7efd90156480 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c400 tx=0x7efd7c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd95800640 1 -- 172.21.15.62:0/134269630 reap_dead start 2024-04-25T09:18:37.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 shutdown_connections 2024-04-25T09:18:37.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 >> 172.21.15.62:0/134269630 conn(0x7efd90066ad0 msgr2=0x7efd9014ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:37.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 shutdown_connections 2024-04-25T09:18:37.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.212+0000 7efd96200640 1 -- 172.21.15.62:0/134269630 wait complete. 2024-04-25T09:18:37.217 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T09:18:37.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.444+0000 7f7085c00640 1 Processor -- start 2024-04-25T09:18:37.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.445+0000 7f7085c00640 1 -- start start 2024-04-25T09:18:37.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.446+0000 7f7085c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800cccf0 0x7f70800ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.446+0000 7f7085c00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f70800aed00 0x7f70800cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.446+0000 7f7085c00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708006ca70 con 0x7f708015bd10 2024-04-25T09:18:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.446+0000 7f7085c00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708006d660 con 0x7f70800aed00 2024-04-25T09:18:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.446+0000 7f7085c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708015c0f0 con 0x7f70800cccf0 2024-04-25T09:18:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707ea00640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f708015bd10 0x7f70800cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38036/0 (socket says 172.21.15.62:38036) 2024-04-25T09:18:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707ea00640 1 -- 172.21.15.62:0/3369569333 learned_addr learned my addr 172.21.15.62:0/3369569333 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707f400640 1 --2- 172.21.15.62:0/3369569333 >> v2:172.21.15.62:3300/0 conn(0x7f70800aed00 0x7f70800cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707e000640 1 --2- 172.21.15.62:0/3369569333 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800cccf0 0x7f70800ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707d600640 1 -- 172.21.15.62:0/3369569333 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290645706 0 0) 0x7f708006ca70 con 0x7f708015bd10 2024-04-25T09:18:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707d600640 1 -- 172.21.15.62:0/3369569333 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7064003520 con 0x7f708015bd10 2024-04-25T09:18:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707f400640 1 -- 172.21.15.62:0/3369569333 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800cccf0 msgr2=0x7f70800ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707f400640 1 --2- 172.21.15.62:0/3369569333 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800cccf0 0x7f70800ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707f400640 1 -- 172.21.15.62:0/3369569333 >> v1:172.21.15.62:6789/0 conn(0x7f708015bd10 legacy=0x7f70800cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.447+0000 7f707f400640 1 -- 172.21.15.62:0/3369569333 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708006cd30 con 0x7f70800aed00 2024-04-25T09:18:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.449+0000 7f707f400640 1 --2- 172.21.15.62:0/3369569333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800aed00 0x7f70800cd6c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f708006d7e0 tx=0x7f707802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc211c01f3008e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.449+0000 7f707d600640 1 -- 172.21.15.62:0/3369569333 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078037070 con 0x7f70800aed00 2024-04-25T09:18:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.449+0000 7f707d600640 1 -- 172.21.15.62:0/3369569333 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7078032440 con 0x7f70800aed00 2024-04-25T09:18:37.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.449+0000 7f707d600640 1 -- 172.21.15.62:0/3369569333 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078032810 con 0x7f70800aed00 2024-04-25T09:18:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.449+0000 7f7085c00640 1 -- 172.21.15.62:0/3369569333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800aed00 msgr2=0x7f70800cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.450+0000 7f7085c00640 1 --2- 172.21.15.62:0/3369569333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800aed00 0x7f70800cd6c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f708006d7e0 tx=0x7f707802a180 comp rx=0 tx=0).stop 2024-04-25T09:18:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.450+0000 7f7085c00640 1 -- 172.21.15.62:0/3369569333 shutdown_connections 2024-04-25T09:18:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.450+0000 7f7085c00640 1 --2- 172.21.15.62:0/3369569333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f70800aed00 0x7f70800cd6c0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.450+0000 7f7085c00640 1 --2- 172.21.15.62:0/3369569333 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800cccf0 0x7f70800ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.450+0000 7f7085c00640 1 -- 172.21.15.62:0/3369569333 >> 172.21.15.62:0/3369569333 conn(0x7f70800668f0 msgr2=0x7f7080066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:37.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.450+0000 7f7085c00640 1 -- 172.21.15.62:0/3369569333 shutdown_connections 2024-04-25T09:18:37.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.450+0000 7f7085c00640 1 -- 172.21.15.62:0/3369569333 wait complete. 2024-04-25T09:18:37.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.452+0000 7f7085c00640 1 Processor -- start 2024-04-25T09:18:37.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.452+0000 7f7085c00640 1 -- start start 2024-04-25T09:18:37.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f7085c00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800aed00 0x7f70800b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f7085c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70800cccf0 0x7f70801589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f7085c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708015bd10 0x7f70800af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f7085c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708006daf0 con 0x7f708015bd10 2024-04-25T09:18:37.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f7085c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708006d2f0 con 0x7f70800cccf0 2024-04-25T09:18:37.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f7085c00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70800af7e0 con 0x7f70800aed00 2024-04-25T09:18:37.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f707f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708015bd10 0x7f70800af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f707f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708015bd10 0x7f70800af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35350/0 (socket says 172.21.15.62:35350) 2024-04-25T09:18:37.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f707f400640 1 -- 172.21.15.62:0/18806729 learned_addr learned my addr 172.21.15.62:0/18806729 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f707e000640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70800cccf0 0x7f70801589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.453+0000 7f707ea00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800aed00 0x7f70800b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f707f400640 1 -- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800aed00 msgr2=0x7f70800b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f707f400640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800aed00 0x7f70800b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f707f400640 1 -- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70800cccf0 msgr2=0x7f70801589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f707f400640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70800cccf0 0x7f70801589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f707f400640 1 -- 172.21.15.62:0/18806729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70800afac0 con 0x7f708015bd10 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f707f400640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708015bd10 0x7f70800af0e0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f7078035040 tx=0x7f70780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f7076a00640 1 -- 172.21.15.62:0/18806729 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078041020 con 0x7f708015bd10 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f7076a00640 1 -- 172.21.15.62:0/18806729 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707803d640 con 0x7f708015bd10 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70800afda0 con 0x7f708015bd10 2024-04-25T09:18:37.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f7076a00640 1 -- 172.21.15.62:0/18806729 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078037040 con 0x7f708015bd10 2024-04-25T09:18:37.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.454+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70800b0360 con 0x7f708015bd10 2024-04-25T09:18:37.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.456+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7080072c70 con 0x7f708015bd10 2024-04-25T09:18:37.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.457+0000 7f7076a00640 1 -- 172.21.15.62:0/18806729 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7078044020 con 0x7f708015bd10 2024-04-25T09:18:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.457+0000 7f7076a00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f705c06b4d0 0x7f705c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.458+0000 7f7076a00640 1 -- 172.21.15.62:0/18806729 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f70780aa9a0 con 0x7f708015bd10 2024-04-25T09:18:37.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.458+0000 7f707ea00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f705c06b4d0 0x7f705c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.459+0000 7f707ea00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f705c06b4d0 0x7f705c06d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7070000f60 tx=0x7f7070009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.462+0000 7f7076a00640 1 -- 172.21.15.62:0/18806729 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f707804b020 con 0x7f708015bd10 2024-04-25T09:18:37.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.661+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f70800b0530 con 0x7f705c06b4d0 2024-04-25T09:18:37.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.662+0000 7f7076a00640 1 -- 172.21.15.62:0/18806729 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f70800b0530 con 0x7f705c06b4d0 2024-04-25T09:18:37.649 INFO:teuthology.orchestra.run.smithi062.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T09:18:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.664+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f705c06b4d0 msgr2=0x7f705c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.664+0000 7f7085c00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f705c06b4d0 0x7f705c06d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7070000f60 tx=0x7f7070009040 comp rx=0 tx=0).stop 2024-04-25T09:18:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.664+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708015bd10 msgr2=0x7f70800af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.664+0000 7f7085c00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708015bd10 0x7f70800af0e0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f7078035040 tx=0x7f70780046c0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.665+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 shutdown_connections 2024-04-25T09:18:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.665+0000 7f7085c00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f705c06b4d0 0x7f705c06d990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.665+0000 7f7085c00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f708015bd10 0x7f70800af0e0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.665+0000 7f7085c00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70800cccf0 0x7f70801589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.665+0000 7f7085c00640 1 --2- 172.21.15.62:0/18806729 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f70800aed00 0x7f70800b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.665+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 >> 172.21.15.62:0/18806729 conn(0x7f70800668f0 msgr2=0x7f708015dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:37.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.665+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 shutdown_connections 2024-04-25T09:18:37.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.665+0000 7f7085c00640 1 -- 172.21.15.62:0/18806729 wait complete. 2024-04-25T09:18:37.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.895+0000 7f18c1400640 1 Processor -- start 2024-04-25T09:18:37.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.895+0000 7f18c1400640 1 -- start start 2024-04-25T09:18:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.897+0000 7f18c1400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f18bc144c10 0x7f18bc15f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.897+0000 7f18c1400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.897+0000 7f18c1400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18bc0cf2d0 con 0x7f18bc0cee40 2024-04-25T09:18:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.897+0000 7f18c1400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18bc144ff0 con 0x7f18bc144c10 2024-04-25T09:18:37.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.897+0000 7f18c1400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc160340 con 0x7f18bc06d900 2024-04-25T09:18:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.898+0000 7f18b3400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f18bc0cee40 0x7f18bc06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.62:43360/0 (socket says 172.21.15.62:43360) 2024-04-25T09:18:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.898+0000 7f18b3400640 1 -- 172.21.15.62:0/561954629 learned_addr learned my addr 172.21.15.62:0/561954629 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.898+0000 7f18c0a00640 1 --2- 172.21.15.62:0/561954629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.898+0000 7f18bbe00640 1 --2- 172.21.15.62:0/561954629 >> v2:172.21.15.121:3300/0 conn(0x7f18bc144c10 0x7f18bc15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.899+0000 7f18c0a00640 1 -- 172.21.15.62:0/561954629 >> v2:172.21.15.121:3300/0 conn(0x7f18bc144c10 msgr2=0x7f18bc15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.899+0000 7f18c0a00640 1 --2- 172.21.15.62:0/561954629 >> v2:172.21.15.121:3300/0 conn(0x7f18bc144c10 0x7f18bc15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.899+0000 7f18c0a00640 1 -- 172.21.15.62:0/561954629 >> v1:172.21.15.121:6789/0 conn(0x7f18bc0cee40 legacy=0x7f18bc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.899+0000 7f18c0a00640 1 -- 172.21.15.62:0/561954629 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc15faa0 con 0x7f18bc06d900 2024-04-25T09:18:37.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.899+0000 7f18bb400640 1 -- 172.21.15.62:0/561954629 <== mon.1 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264926229 0 0) 0x7f18bc0cf2d0 con 0x7f18bc0cee40 2024-04-25T09:18:37.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.900+0000 7f18c0a00640 1 --2- 172.21.15.62:0/561954629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0b4650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009d90 tx=0x7f18b402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b675c393015765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.901+0000 7f18bb400640 1 -- 172.21.15.62:0/561954629 <== mon.2 v2:172.21.15.121:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4031600 con 0x7f18bc06d900 2024-04-25T09:18:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.901+0000 7f18bb400640 1 -- 172.21.15.62:0/561954629 <== mon.2 v2:172.21.15.121:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b40317a0 con 0x7f18bc06d900 2024-04-25T09:18:37.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.901+0000 7f18bb400640 1 -- 172.21.15.62:0/561954629 <== mon.2 v2:172.21.15.121:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4031b70 con 0x7f18bc06d900 2024-04-25T09:18:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.902+0000 7f18c1400640 1 -- 172.21.15.62:0/561954629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 msgr2=0x7f18bc0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.902+0000 7f18c1400640 1 --2- 172.21.15.62:0/561954629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0b4650 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009d90 tx=0x7f18b402a5d0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.902+0000 7f18c1400640 1 -- 172.21.15.62:0/561954629 shutdown_connections 2024-04-25T09:18:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.902+0000 7f18c1400640 1 --2- 172.21.15.62:0/561954629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0b4650 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.902+0000 7f18c1400640 1 --2- 172.21.15.62:0/561954629 >> v2:172.21.15.121:3300/0 conn(0x7f18bc144c10 0x7f18bc15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.902+0000 7f18c1400640 1 -- 172.21.15.62:0/561954629 >> 172.21.15.62:0/561954629 conn(0x7f18bc0b14d0 msgr2=0x7f18bc0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.902+0000 7f18c1400640 1 -- 172.21.15.62:0/561954629 shutdown_connections 2024-04-25T09:18:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.902+0000 7f18c1400640 1 -- 172.21.15.62:0/561954629 wait complete. 2024-04-25T09:18:37.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.904+0000 7f18c1400640 1 Processor -- start 2024-04-25T09:18:37.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.904+0000 7f18c1400640 1 -- start start 2024-04-25T09:18:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18c1400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18c1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18bc0cee40 0x7f18bc1f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18c1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18bc144c10 0x7f18bc1f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18c1400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc160060 con 0x7f18bc0cee40 2024-04-25T09:18:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18c1400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0b41c0 con 0x7f18bc144c10 2024-04-25T09:18:37.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18c1400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0b4b90 con 0x7f18bc06d900 2024-04-25T09:18:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18b3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18bc0cee40 0x7f18bc1f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18b3400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18bc0cee40 0x7f18bc1f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35356/0 (socket says 172.21.15.62:35356) 2024-04-25T09:18:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18b3400640 1 -- 172.21.15.62:0/4280632248 learned_addr learned my addr 172.21.15.62:0/4280632248 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.905+0000 7f18bbe00640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.906+0000 7f18c0a00640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18bc144c10 0x7f18bc1f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.906+0000 7f18b3400640 1 -- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 msgr2=0x7f18bc0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.906+0000 7f18b3400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.906+0000 7f18b3400640 1 -- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18bc144c10 msgr2=0x7f18bc1f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.906+0000 7f18b3400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18bc144c10 0x7f18bc1f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.906+0000 7f18b3400640 1 -- 172.21.15.62:0/4280632248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc0ca750 con 0x7f18bc0cee40 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.906+0000 7f18b3400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18bc0cee40 0x7f18bc1f72f0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c500 tx=0x7f18a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.907+0000 7f18b8c00640 1 -- 172.21.15.62:0/4280632248 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4018070 con 0x7f18bc0cee40 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.907+0000 7f18b8c00640 1 -- 172.21.15.62:0/4280632248 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a4007580 con 0x7f18bc0cee40 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.907+0000 7f18b8c00640 1 -- 172.21.15.62:0/4280632248 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4004a20 con 0x7f18bc0cee40 2024-04-25T09:18:37.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.907+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18bc0caa30 con 0x7f18bc0cee40 2024-04-25T09:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.907+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18bc0caff0 con 0x7f18bc0cee40 2024-04-25T09:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.908+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18bc04ef20 con 0x7f18bc0cee40 2024-04-25T09:18:37.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.909+0000 7f18b8c00640 1 -- 172.21.15.62:0/4280632248 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18a4004bc0 con 0x7f18bc0cee40 2024-04-25T09:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.910+0000 7f18b8c00640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f189806b4d0 0x7f189806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.910+0000 7f18b8c00640 1 -- 172.21.15.62:0/4280632248 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f18a4089160 con 0x7f18bc0cee40 2024-04-25T09:18:37.897 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.911+0000 7f18bbe00640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f189806b4d0 0x7f189806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:37.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.912+0000 7f18bbe00640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f189806b4d0 0x7f189806d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009aa0 tx=0x7f18ac009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:37.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:37.914+0000 7f18b8c00640 1 -- 172.21.15.62:0/4280632248 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18a4057c80 con 0x7f18bc0cee40 2024-04-25T09:18:38.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.111+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 --> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18bc15ce90 con 0x7f189806b4d0 2024-04-25T09:18:38.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.114+0000 7f18b8c00640 1 -- 172.21.15.62:0/4280632248 <== mgr.4106 v2:172.21.15.121:6832/568331749 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13380 (secure 0 0 0) 0x7f18bc15ce90 con 0x7f189806b4d0 2024-04-25T09:18:38.101 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:38.102 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-25T09:18:38.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f189806b4d0 msgr2=0x7f189806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f189806b4d0 0x7f189806d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009aa0 tx=0x7f18ac009190 comp rx=0 tx=0).stop 2024-04-25T09:18:38.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18bc0cee40 msgr2=0x7f18bc1f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18bc0cee40 0x7f18bc1f72f0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c500 tx=0x7f18a400c9c0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 shutdown_connections 2024-04-25T09:18:38.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f189806b4d0 0x7f189806d990 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18bc0cee40 0x7f18bc1f72f0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18bc144c10 0x7f18bc1f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 --2- 172.21.15.62:0/4280632248 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f18bc06d900 0x7f18bc0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 >> 172.21.15.62:0/4280632248 conn(0x7f18bc0b14d0 msgr2=0x7f18bc0719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:38.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 shutdown_connections 2024-04-25T09:18:38.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.117+0000 7f18c1400640 1 -- 172.21.15.62:0/4280632248 wait complete. 2024-04-25T09:18:38.122 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":611,"stamp":"2024-04-25T09:18:36.239916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":491780,"kb_used_data":16168,"kb_used_omap":12,"kb_used_meta":475507,"kb_avail":754482940,"statfs":{"total":773094113280,"available":772590530560,"internally_reserved":0,"allocated":16556032,"data_stored":7943936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":486919728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.723250"},"pg_stats":[{"pgid":"1.0","version":"153'119","reported_seq":427,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-25T09:18:33.426993+0000","last_change":"2024-04-25T09:18:33.427038+0000","last_active":"2024-04-25T09:18:33.426993+0000","last_peered":"2024-04-25T09:18:33.426993+0000","last_clean":"2024-04-25T09:18:33.426993+0000","last_became_active":"2024-04-25T09:03:07.234270+0000","last_became_peered":"2024-04-25T09:03:07.234270+0000","last_unstale":"2024-04-25T09:18:33.426993+0000","last_undegraded":"2024-04-25T09:18:33.426993+0000","last_fullsized":"2024-04-25T09:18:33.426993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'119","last_scrub_stamp":"2024-04-25T09:18:33.426870+0000","last_deep_scrub":"153'119","last_deep_scrub_stamp":"2024-04-25T09:18:33.426870+0000","last_clean_scrub_stamp":"2024-04-25T09:18:33.426870+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:08.893222+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55268,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316572,"statfs":{"total":96636764160,"available":96580169728,"internally_reserved":0,"allocated":1921024,"data_stored":845400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59424,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":1921024,"data_stored":845400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60000,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311840,"statfs":{"total":96636764160,"available":96575324160,"internally_reserved":0,"allocated":2514944,"data_stored":1435768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64928,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306912,"statfs":{"total":96636764160,"available":96570277888,"internally_reserved":0,"allocated":1921024,"data_stored":845400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575039,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63584,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308256,"statfs":{"total":96636764160,"available":96571654144,"internally_reserved":0,"allocated":1921024,"data_stored":845400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69152,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302688,"statfs":{"total":96636764160,"available":96565952512,"internally_reserved":0,"allocated":1921024,"data_stored":845400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64160,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307680,"statfs":{"total":96636764160,"available":96571064320,"internally_reserved":0,"allocated":2514944,"data_stored":1435768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55264,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316576,"statfs":{"total":96636764160,"available":96580173824,"internally_reserved":0,"allocated":1921024,"data_stored":845400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:18:38.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T09:18:38.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.339+0000 7fbe79400640 1 Processor -- start 2024-04-25T09:18:38.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.339+0000 7fbe79400640 1 -- start start 2024-04-25T09:18:38.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.341+0000 7fbe79400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbe740b54d0 0x7fbe740ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.341+0000 7fbe79400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 0x7fbe74164310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.341+0000 7fbe79400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe740b4430 con 0x7fbe740b2d80 2024-04-25T09:18:38.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.341+0000 7fbe79400640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe740b4710 con 0x7fbe740b54d0 2024-04-25T09:18:38.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.341+0000 7fbe79400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740b49f0 con 0x7fbe740ce7f0 2024-04-25T09:18:38.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.341+0000 7fbe73400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbe740b54d0 0x7fbe740ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.341+0000 7fbe73400640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fbe740b54d0 0x7fbe740ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35370/0 (socket says 172.21.15.62:35370) 2024-04-25T09:18:38.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.341+0000 7fbe73400640 1 -- 172.21.15.62:0/2821214629 learned_addr learned my addr 172.21.15.62:0/2821214629 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:38.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.342+0000 7fbe78a00640 1 --2- 172.21.15.62:0/2821214629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 0x7fbe74164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.342+0000 7fbe73400640 1 -- 172.21.15.62:0/2821214629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 msgr2=0x7fbe74164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:38.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.342+0000 7fbe73400640 1 --2- 172.21.15.62:0/2821214629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 0x7fbe74164310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.342+0000 7fbe73400640 1 -- 172.21.15.62:0/2821214629 >> v1:172.21.15.121:6789/0 conn(0x7fbe740b2d80 legacy=0x7fbe740b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.342+0000 7fbe73400640 1 -- 172.21.15.62:0/2821214629 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe740b4cd0 con 0x7fbe740b54d0 2024-04-25T09:18:38.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.343+0000 7fbe73400640 1 --2- 172.21.15.62:0/2821214629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b54d0 0x7fbe740ce2b0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00bb10 tx=0x7fbe5c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4446188f9d2a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:38.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.343+0000 7fbe72a00640 1 -- 172.21.15.62:0/2821214629 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c036020 con 0x7fbe740b54d0 2024-04-25T09:18:38.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.343+0000 7fbe72a00640 1 -- 172.21.15.62:0/2821214629 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c032820 con 0x7fbe740b54d0 2024-04-25T09:18:38.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.343+0000 7fbe72a00640 1 -- 172.21.15.62:0/2821214629 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c032bf0 con 0x7fbe740b54d0 2024-04-25T09:18:38.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.344+0000 7fbe79400640 1 -- 172.21.15.62:0/2821214629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b54d0 msgr2=0x7fbe740ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.344+0000 7fbe79400640 1 --2- 172.21.15.62:0/2821214629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b54d0 0x7fbe740ce2b0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00bb10 tx=0x7fbe5c02c920 comp rx=0 tx=0).stop 2024-04-25T09:18:38.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.344+0000 7fbe79400640 1 -- 172.21.15.62:0/2821214629 shutdown_connections 2024-04-25T09:18:38.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.344+0000 7fbe79400640 1 --2- 172.21.15.62:0/2821214629 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 0x7fbe74164310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.344+0000 7fbe79400640 1 --2- 172.21.15.62:0/2821214629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b54d0 0x7fbe740ce2b0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.344+0000 7fbe79400640 1 -- 172.21.15.62:0/2821214629 >> 172.21.15.62:0/2821214629 conn(0x7fbe74066f20 msgr2=0x7fbe740b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:38.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.344+0000 7fbe79400640 1 -- 172.21.15.62:0/2821214629 shutdown_connections 2024-04-25T09:18:38.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.344+0000 7fbe79400640 1 -- 172.21.15.62:0/2821214629 wait complete. 2024-04-25T09:18:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.346+0000 7fbe79400640 1 Processor -- start 2024-04-25T09:18:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.346+0000 7fbe79400640 1 -- start start 2024-04-25T09:18:38.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe79400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b2d80 0x7fbe7414af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe79400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe740b54d0 0x7fbe7415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe79400640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 0x7fbe7415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe79400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740b3870 con 0x7fbe740b2d80 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe79400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740b40e0 con 0x7fbe740b54d0 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe79400640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7406d2c0 con 0x7fbe740ce7f0 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b2d80 0x7fbe7414af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe73e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b2d80 0x7fbe7414af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35386/0 (socket says 172.21.15.62:35386) 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe73e00640 1 -- 172.21.15.62:0/2531213855 learned_addr learned my addr 172.21.15.62:0/2531213855 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:38.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.347+0000 7fbe78a00640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 0x7fbe7415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe73400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe740b54d0 0x7fbe7415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe73e00640 1 -- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 msgr2=0x7fbe7415d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe73e00640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 0x7fbe7415d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe73e00640 1 -- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe740b54d0 msgr2=0x7fbe7415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe73e00640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe740b54d0 0x7fbe7415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe73e00640 1 -- 172.21.15.62:0/2531213855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe741f3a90 con 0x7fbe740b2d80 2024-04-25T09:18:38.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe73e00640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b2d80 0x7fbe7414af40 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00c7d0 tx=0x7fbe6c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:38.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe6be00640 1 -- 172.21.15.62:0/2531213855 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c003b90 con 0x7fbe740b2d80 2024-04-25T09:18:38.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe6be00640 1 -- 172.21.15.62:0/2531213855 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c003d50 con 0x7fbe740b2d80 2024-04-25T09:18:38.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe741f3d70 con 0x7fbe740b2d80 2024-04-25T09:18:38.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.348+0000 7fbe6be00640 1 -- 172.21.15.62:0/2531213855 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c00aa90 con 0x7fbe740b2d80 2024-04-25T09:18:38.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.349+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe741f4330 con 0x7fbe740b2d80 2024-04-25T09:18:38.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.350+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7404ef20 con 0x7fbe740b2d80 2024-04-25T09:18:38.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.351+0000 7fbe6be00640 1 -- 172.21.15.62:0/2531213855 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe6c00ac30 con 0x7fbe740b2d80 2024-04-25T09:18:38.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.352+0000 7fbe6be00640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe5006b4d0 0x7fbe5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.352+0000 7fbe6be00640 1 -- 172.21.15.62:0/2531213855 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbe6c089e60 con 0x7fbe740b2d80 2024-04-25T09:18:38.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.352+0000 7fbe73400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe5006b4d0 0x7fbe5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.354+0000 7fbe73400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe5006b4d0 0x7fbe5006d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c037000 tx=0x7fbe5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:38.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.356+0000 7fbe6be00640 1 -- 172.21.15.62:0/2531213855 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe6c058fd0 con 0x7fbe740b2d80 2024-04-25T09:18:38.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.554+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fbe741f37a0 con 0x7fbe740b2d80 2024-04-25T09:18:38.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.562+0000 7fbe6be00640 1 -- 172.21.15.62:0/2531213855 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v28)=0 v28) v1 ==== 123+0+0 (secure 0 0 0) 0x7fbe6c0588d0 con 0x7fbe740b2d80 2024-04-25T09:18:38.551 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.564+0000 7fbe6be00640 1 -- 172.21.15.62:0/2531213855 <== mon.0 v2:172.21.15.62:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbe6c05ca20 con 0x7fbe740b2d80 2024-04-25T09:18:38.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe5006b4d0 msgr2=0x7fbe5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe5006b4d0 0x7fbe5006d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c037000 tx=0x7fbe5c008040 comp rx=0 tx=0).stop 2024-04-25T09:18:38.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b2d80 msgr2=0x7fbe7414af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.554 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b2d80 0x7fbe7414af40 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00c7d0 tx=0x7fbe6c00cc90 comp rx=0 tx=0).stop 2024-04-25T09:18:38.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 shutdown_connections 2024-04-25T09:18:38.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7fbe5006b4d0 0x7fbe5006d990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7fbe740ce7f0 0x7fbe7415d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe740b54d0 0x7fbe7415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 --2- 172.21.15.62:0/2531213855 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe740b2d80 0x7fbe7414af40 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 >> 172.21.15.62:0/2531213855 conn(0x7fbe74066f20 msgr2=0x7fbe740cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:38.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 shutdown_connections 2024-04-25T09:18:38.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.567+0000 7fbe79400640 1 -- 172.21.15.62:0/2531213855 wait complete. 2024-04-25T09:18:38.573 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:18:38.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:18:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.786+0000 7f933de00640 1 Processor -- start 2024-04-25T09:18:38.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.786+0000 7f933de00640 1 -- start start 2024-04-25T09:18:38.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.787+0000 7f933de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f933813d970 0x7f93380b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f933de00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f93380b1050 0x7f933806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f933de00640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933815b220 con 0x7f933815acf0 2024-04-25T09:18:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f933de00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933815b500 con 0x7f93380b1050 2024-04-25T09:18:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f933de00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933815b7e0 con 0x7f933813d970 2024-04-25T09:18:38.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f933d400640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f933815acf0 0x7f933813ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:38046/0 (socket says 172.21.15.62:38046) 2024-04-25T09:18:38.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f933d400640 1 -- 172.21.15.62:0/137659255 learned_addr learned my addr 172.21.15.62:0/137659255 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-25T09:18:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f933ca00640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f933813d970 0x7f93380b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f9337e00640 1 --2- 172.21.15.62:0/137659255 >> v2:172.21.15.121:3300/0 conn(0x7f93380b1050 0x7f933806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620483455 0 0) 0x7f933815b220 con 0x7f933815acf0 2024-04-25T09:18:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.788+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9320003520 con 0x7f933815acf0 2024-04-25T09:18:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58860527 0 0) 0x7f9320003520 con 0x7f933815acf0 2024-04-25T09:18:38.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f933813d970 msgr2=0x7f93380b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f933813d970 0x7f93380b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 >> v2:172.21.15.121:3300/0 conn(0x7f93380b1050 msgr2=0x7f933806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 --2- 172.21.15.62:0/137659255 >> v2:172.21.15.121:3300/0 conn(0x7f93380b1050 0x7f933806d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933815bac0 con 0x7f933815acf0 2024-04-25T09:18:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2428939604 0 0) 0x7f93300019f0 con 0x7f933815acf0 2024-04-25T09:18:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 >> v1:172.21.15.62:6789/0 conn(0x7f933815acf0 legacy=0x7f933813ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9320003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9320004800 0x7f9320004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f933813d970 0x7f93200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933815b500 con 0x7f93200039b0 2024-04-25T09:18:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933815b7e0 con 0x7f9320004800 2024-04-25T09:18:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932000bac0 con 0x7f933813d970 2024-04-25T09:18:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.789+0000 7f933ca00640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9320003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.790+0000 7f9337e00640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9320004800 0x7f9320004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.790+0000 7f933ca00640 1 -- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f933813d970 msgr2=0x7f93200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.790+0000 7f933ca00640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f933813d970 0x7f93200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.790+0000 7f933ca00640 1 -- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9320004800 msgr2=0x7f9320004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:18:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.790+0000 7f933ca00640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9320004800 0x7f9320004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.790+0000 7f933ca00640 1 -- 172.21.15.62:0/137659255 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9320003520 con 0x7f93200039b0 2024-04-25T09:18:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.791+0000 7f933ca00640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9320003dd0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f932c00ba00 tx=0x7f9320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65d99e9dcd4a804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:38.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.791+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c030070 con 0x7f93200039b0 2024-04-25T09:18:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.791+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f932c02adb0 con 0x7f93200039b0 2024-04-25T09:18:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.791+0000 7f9337400640 1 -- 172.21.15.62:0/137659255 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c00b600 con 0x7f93200039b0 2024-04-25T09:18:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.792+0000 7f933de00640 1 -- 172.21.15.62:0/137659255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 msgr2=0x7f9320003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.792+0000 7f933de00640 1 --2- 172.21.15.62:0/137659255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9320003dd0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f932c00ba00 tx=0x7f9320005830 comp rx=0 tx=0).stop 2024-04-25T09:18:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.792+0000 7f933d400640 1 -- 172.21.15.62:0/137659255 reap_dead start 2024-04-25T09:18:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.792+0000 7f933de00640 1 -- 172.21.15.62:0/137659255 shutdown_connections 2024-04-25T09:18:38.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.792+0000 7f933de00640 1 -- 172.21.15.62:0/137659255 >> 172.21.15.62:0/137659255 conn(0x7f93380b14d0 msgr2=0x7f93380b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:38.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.792+0000 7f933de00640 1 -- 172.21.15.62:0/137659255 shutdown_connections 2024-04-25T09:18:38.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.792+0000 7f933de00640 1 -- 172.21.15.62:0/137659255 wait complete. 2024-04-25T09:18:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.794+0000 7f933de00640 1 Processor -- start 2024-04-25T09:18:38.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.794+0000 7f933de00640 1 -- start start 2024-04-25T09:18:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9338159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93381e2fd0 0x7f9338153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933de00640 1 --2- >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93381e3450 0x7f9338153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93380b0b10 con 0x7f93200039b0 2024-04-25T09:18:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933de00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933813f470 con 0x7f93381e2fd0 2024-04-25T09:18:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933de00640 1 -- --> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93380b0c90 con 0x7f93381e3450 2024-04-25T09:18:38.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9338159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9338159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35410/0 (socket says 172.21.15.62:35410) 2024-04-25T09:18:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933ca00640 1 -- 172.21.15.62:0/2567665038 learned_addr learned my addr 172.21.15.62:0/2567665038 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T09:18:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933ca00640 1 -- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93381e3450 msgr2=0x7f9338153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933ca00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93381e3450 0x7f9338153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933ca00640 1 -- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93381e2fd0 msgr2=0x7f9338153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933ca00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93381e2fd0 0x7f9338153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.795+0000 7f933ca00640 1 -- 172.21.15.62:0/2567665038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933815c9c0 con 0x7f93200039b0 2024-04-25T09:18:38.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.796+0000 7f933ca00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9338159a80 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f932c00bb20 tx=0x7f932c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.796+0000 7f9334c00640 1 -- 172.21.15.62:0/2567665038 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c043020 con 0x7f93200039b0 2024-04-25T09:18:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.796+0000 7f9334c00640 1 -- 172.21.15.62:0/2567665038 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f932c03f680 con 0x7f93200039b0 2024-04-25T09:18:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.796+0000 7f9334c00640 1 -- 172.21.15.62:0/2567665038 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c00b600 con 0x7f93200039b0 2024-04-25T09:18:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.796+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93381f79f0 con 0x7f93200039b0 2024-04-25T09:18:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.796+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933815cf60 con 0x7f93200039b0 2024-04-25T09:18:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.798+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933804ef20 con 0x7f93200039b0 2024-04-25T09:18:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.798+0000 7f9334c00640 1 -- 172.21.15.62:0/2567665038 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f932c046020 con 0x7f93200039b0 2024-04-25T09:18:38.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.799+0000 7f9334c00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f931806b4d0 0x7f931806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:18:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.799+0000 7f9334c00640 1 -- 172.21.15.62:0/2567665038 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f932c0aad30 con 0x7f93200039b0 2024-04-25T09:18:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.800+0000 7f9337e00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f931806b4d0 0x7f931806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:18:38.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.801+0000 7f9337e00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f931806b4d0 0x7f931806d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9324006a90 tx=0x7f9324006a20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:18:38.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:38.806+0000 7f9334c00640 1 -- 172.21.15.62:0/2567665038 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f932c04d030 con 0x7f93200039b0 2024-04-25T09:18:39.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.043+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93380cd2b0 con 0x7f93200039b0 2024-04-25T09:18:39.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.045+0000 7f9334c00640 1 -- 172.21.15.62:0/2567665038 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+3696 (secure 0 0 0) 0x7f932c079d80 con 0x7f93200039b0 2024-04-25T09:18:39.032 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T09:18:39.032 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6224,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/3369583759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6835","nonce":3369583759},{"type":"v1","addr":"172.21.15.121:6837","nonce":3369583759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7072,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6836/1888749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":1888749},{"type":"v1","addr":"172.21.15.121:6836","nonce":1888749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":9702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/510327391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":510327391},{"type":"v1","addr":"172.21.15.62:6833","nonce":510327391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":104},{"gid":9705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1322222028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":1322222028},{"type":"v1","addr":"172.21.15.62:6835","nonce":1322222028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":104}],"filesystems":[]} 2024-04-25T09:18:39.032 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 104 2024-04-25T09:18:39.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f931806b4d0 msgr2=0x7f931806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:39.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f931806b4d0 0x7f931806d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9324006a90 tx=0x7f9324006a20 comp rx=0 tx=0).stop 2024-04-25T09:18:39.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 msgr2=0x7f9338159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:18:39.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9338159a80 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f932c00bb20 tx=0x7f932c004730 comp rx=0 tx=0).stop 2024-04-25T09:18:39.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 shutdown_connections 2024-04-25T09:18:39.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:6832/568331749,v1:172.21.15.121:6833/568331749] conn(0x7f931806b4d0 0x7f931806d990 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:39.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:3301/0,v1:172.21.15.121:6790/0] conn(0x7f93381e3450 0x7f9338153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:39.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f93381e2fd0 0x7f9338153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:39.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 --2- 172.21.15.62:0/2567665038 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93200039b0 0x7f9338159a80 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:18:39.035 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 >> 172.21.15.62:0/2567665038 conn(0x7f93380b14d0 msgr2=0x7f93380b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:18:39.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.047+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 shutdown_connections 2024-04-25T09:18:39.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T09:18:39.048+0000 7f933de00640 1 -- 172.21.15.62:0/2567665038 wait complete. 2024-04-25T09:18:39.052 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T09:18:39.052 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T09:18:39.053 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:18:39.053 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:39.070 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:18:39.070 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:18:39.070 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:39.129 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:18:39.129 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:18:39.129 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:39.158 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:18:39.158 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:18:39.158 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:39.225 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:18:39.225 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T09:18:39.226 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T09:18:39.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:39.305 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T09:18:39.306 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T09:18:39.306 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:39.400 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T09:18:39.401 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T09:18:39.401 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:42.754 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-25T09:18:45.503 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T09:18:45.503 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T09:18:45.504 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:45.583 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T09:18:45.584 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T09:18:45.584 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:45.671 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T09:18:45.672 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T09:18:45.672 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:45.769 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T09:18:45.769 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T09:18:45.769 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:45.841 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T09:18:45.842 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T09:18:45.842 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:45.918 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T09:18:45.919 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T09:18:45.919 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T09:18:45.919 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:45.998 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T09:18:45.998 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T09:18:45.998 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:46.004 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:18:46.011 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:18:46.077 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T09:18:46.077 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T09:18:46.078 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T09:18:46.078 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:46.103 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T09:18:46.104 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T09:18:46.104 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:46.126 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T09:18:46.126 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T09:18:46.126 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:18:46.206 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T09:18:46.207 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T09:18:46.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T09:18:46.319 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:18:46.319 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T09:18:46.621 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:18:46.622 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T09:18:46.782 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:18:46.782 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T09:18:46.924 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:18:46.925 DEBUG:teuthology.orchestra.run.smithi062:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T09:18:47.104 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:18:47.105 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T09:18:47.445 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:18:47.446 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T09:18:47.594 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:18:47.595 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T09:18:47.739 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:18:47.740 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T09:18:47.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-25T09:18:47.900 INFO:tasks.ceph:Archiving mon data... 2024-04-25T09:18:47.902 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/data/mon.a.tgz 2024-04-25T09:18:47.902 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-04-25T09:18:47.958 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.MNSFlMUkzz 2024-04-25T09:18:47.959 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.MNSFlMUkzz 2024-04-25T09:18:48.158 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.MNSFlMUkzz 2024-04-25T09:18:48.313 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.MNSFlMUkzz is 475KB 2024-04-25T09:18:48.423 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.MNSFlMUkzz 2024-04-25T09:18:48.482 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/data/mon.b.tgz 2024-04-25T09:18:48.482 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-04-25T09:18:48.539 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.0qNxsERsfQ 2024-04-25T09:18:48.540 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0qNxsERsfQ 2024-04-25T09:18:48.798 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.0qNxsERsfQ 2024-04-25T09:18:48.937 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.0qNxsERsfQ is 487KB 2024-04-25T09:18:49.046 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.0qNxsERsfQ 2024-04-25T09:18:49.103 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/data/mon.c.tgz 2024-04-25T09:18:49.103 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-04-25T09:18:49.155 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.kixmD1iSqu 2024-04-25T09:18:49.156 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.kixmD1iSqu 2024-04-25T09:18:49.346 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.kixmD1iSqu 2024-04-25T09:18:49.480 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.kixmD1iSqu is 486KB 2024-04-25T09:18:49.582 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.kixmD1iSqu 2024-04-25T09:18:49.638 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T09:18:49.639 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T09:18:49.646 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T09:18:49.790 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T09:18:49.844 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T09:18:49.844 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T09:18:49.846 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/remote/smithi062/crash 2024-04-25T09:18:49.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T09:18:49.937 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/remote/smithi121/crash 2024-04-25T09:18:49.938 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T09:18:50.024 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T09:18:50.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:18:50.033 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:18:50.183 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T09:18:50.276 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T09:18:50.363 INFO:tasks.ceph:Compressing logs... 2024-04-25T09:18:50.364 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T09:18:50.372 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T09:18:50.453 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45531.log 2024-04-25T09:18:50.454 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62533.log 2024-04-25T09:18:50.454 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33259.log 2024-04-25T09:18:50.455 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T09:18:50.455 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43194.log 2024-04-25T09:18:50.456 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62533.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45531.log.gz 2024-04-25T09:18:50.457 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62533.log.gz 2024-04-25T09:18:50.457 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39209.log 2024-04-25T09:18:50.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33259.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33259.log.gz 2024-04-25T09:18:50.458 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33550.log 2024-04-25T09:18:50.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T09:18:50.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33510.log 2024-04-25T09:18:50.460 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33550.log.gz 2024-04-25T09:18:50.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T09:18:50.460 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82247.log 2024-04-25T09:18:50.461 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43194.log.gz 2024-04-25T09:18:50.461 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39209.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.59679.log 2024-04-25T09:18:50.461 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39209.log.gz 2024-04-25T09:18:50.461 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.82247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.82247.log.gz 2024-04-25T09:18:50.461 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84693.log 2024-04-25T09:18:50.463 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90473.log 2024-04-25T09:18:50.463 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59679.log.gz 2024-04-25T09:18:50.464 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81952.log 2024-04-25T09:18:50.464 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.84693.log.gz 2024-04-25T09:18:50.464 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33510.log: gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.33510.log.gz /var/log/ceph/ceph-mds.b.log 2024-04-25T09:18:50.464 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-04-25T09:18:50.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T09:18:50.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mds.b.log: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T09:18:50.465 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T09:18:50.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.5.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T09:18:50.465 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T09:18:50.466 INFO:teuthology.orchestra.run.smithi121.stderr:gzip/var/log/ceph/ceph-osd.4.log: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T09:18:50.467 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62789.log 2024-04-25T09:18:50.467 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90473.log.gz 2024-04-25T09:18:50.468 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96001.log 2024-04-25T09:18:50.468 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81952.log.gz 2024-04-25T09:18:50.468 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62789.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62789.log.gz -5 2024-04-25T09:18:50.468 INFO:teuthology.orchestra.run.smithi062.stderr: --verbose -- /var/log/ceph/ceph-client.admin.70483.log 2024-04-25T09:18:50.468 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91301.log 2024-04-25T09:18:50.468 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96001.log.gz 2024-04-25T09:18:50.470 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33390.log 2024-04-25T09:18:50.470 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T09:18:50.470 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42864.log 2024-04-25T09:18:50.470 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70483.log.gz 2024-04-25T09:18:50.471 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79006.log 2024-04-25T09:18:50.471 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91301.log.gz 2024-04-25T09:18:50.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33390.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33390.log.gz 2024-04-25T09:18:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72144.log 2024-04-25T09:18:50.472 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42864.log.gz 2024-04-25T09:18:50.472 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52257.log 2024-04-25T09:18:50.473 INFO:teuthology.orchestra.run.smithi062.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79006.log.gz 2024-04-25T09:18:50.473 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72144.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72144.log.gz 2024-04-25T09:18:50.474 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71496.log 2024-04-25T09:18:50.474 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33348.log 2024-04-25T09:18:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54489.log 2024-04-25T09:18:50.476 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71496.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52257.log.gz 2024-04-25T09:18:50.476 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71496.log.gz 2024-04-25T09:18:50.477 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25793.log 2024-04-25T09:18:50.478 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45109.log 2024-04-25T09:18:50.478 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54489.log.gz 2024-04-25T09:18:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph.tmp-client.admin.25793.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25793.log.gz 2024-04-25T09:18:50.479 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89531.log 2024-04-25T09:18:50.479 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39474.log 2024-04-25T09:18:50.479 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33470.log 2024-04-25T09:18:50.479 INFO:teuthology.orchestra.run.smithi121.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33348.log.gz 2024-04-25T09:18:50.479 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39881.log 2024-04-25T09:18:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45109.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45109.log.gz 2024-04-25T09:18:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43236.log 2024-04-25T09:18:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95058.log 2024-04-25T09:18:50.482 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89531.log.gz 2024-04-25T09:18:50.482 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43236.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43236.log.gz 2024-04-25T09:18:50.483 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90599.log 2024-04-25T09:18:50.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.1.39474.log: /var/log/ceph/ceph-client.3.39881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39881.log.gz 2024-04-25T09:18:50.483 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39474.log.gz 2024-04-25T09:18:50.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33430.log 2024-04-25T09:18:50.484 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95058.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95058.log.gz 2024-04-25T09:18:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75805.log 2024-04-25T09:18:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58110.log 2024-04-25T09:18:50.485 INFO:teuthology.orchestra.run.smithi062.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90599.log.gz 2024-04-25T09:18:50.485 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92445.log 2024-04-25T09:18:50.485 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75805.log.gz 2024-04-25T09:18:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58110.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58110.log.gz 2024-04-25T09:18:50.487 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84435.log 2024-04-25T09:18:50.488 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.92445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75847.log 2024-04-25T09:18:50.488 INFO:teuthology.orchestra.run.smithi062.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92445.log.gz 2024-04-25T09:18:50.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-client.admin.33430.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33470.log.gz 2024-04-25T09:18:50.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T09:18:50.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T09:18:50.490 INFO:teuthology.orchestra.run.smithi121.stderr: 88.2%/var/log/ceph/ceph-mds.d.log: -- replaced with /var/log/ceph/ceph-client.admin.33430.log.gz 2024-04-25T09:18:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84435.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84435.log.gz 2024-04-25T09:18:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58278.log 2024-04-25T09:18:50.491 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43842.log 2024-04-25T09:18:50.491 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75847.log.gz 2024-04-25T09:18:50.492 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58278.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58278.log.gz 2024-04-25T09:18:50.492 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90641.log 2024-04-25T09:18:50.492 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58717.log 2024-04-25T09:18:50.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33299.log 2024-04-25T09:18:50.494 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43842.log.gz 2024-04-25T09:18:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90641.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90641.log.gz 2024-04-25T09:18:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43614.log 2024-04-25T09:18:50.494 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81700.log 2024-04-25T09:18:50.494 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58717.log.gz 2024-04-25T09:18:50.496 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75575.log 2024-04-25T09:18:50.496 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43614.log.gz 2024-04-25T09:18:50.496 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84609.log 2024-04-25T09:18:50.496 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81700.log.gz 2024-04-25T09:18:50.497 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80612.log 2024-04-25T09:18:50.497 INFO:teuthology.orchestra.run.smithi062.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.75575.log.gz 2024-04-25T09:18:50.498 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33299.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33299.log.gz 2024-04-25T09:18:50.499 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62052.log 2024-04-25T09:18:50.499 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84609.log.gz 2024-04-25T09:18:50.499 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56854.log 2024-04-25T09:18:50.500 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80612.log.gz 2024-04-25T09:18:50.500 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53088.log 2024-04-25T09:18:50.500 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62052.log.gz 2024-04-25T09:18:50.501 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57496.log 2024-04-25T09:18:50.501 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53088.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56854.log.gz 2024-04-25T09:18:50.502 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53088.log.gz 2024-04-25T09:18:50.502 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89717.log 2024-04-25T09:18:50.503 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75677.log 2024-04-25T09:18:50.503 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57496.log.gz 2024-04-25T09:18:50.504 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89717.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.89591.log -- replaced with /var/log/ceph/ceph-client.admin.89717.log.gz 2024-04-25T09:18:50.504 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:18:50.505 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42417.log 2024-04-25T09:18:50.505 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75677.log.gz 2024-04-25T09:18:50.506 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89591.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.89591.log.gz --verbose 2024-04-25T09:18:50.506 INFO:teuthology.orchestra.run.smithi062.stderr: -- /var/log/ceph/ceph-client.admin.71580.log 2024-04-25T09:18:50.507 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63225.log 2024-04-25T09:18:50.507 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42417.log.gz 2024-04-25T09:18:50.508 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71580.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.71580.log.gz 2024-04-25T09:18:50.508 INFO:teuthology.orchestra.run.smithi062.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94481.log 2024-04-25T09:18:50.509 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63225.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63225.log.gz 2024-04-25T09:18:50.510 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39884.log 2024-04-25T09:18:50.510 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80528.log 2024-04-25T09:18:50.511 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94481.log.gz 2024-04-25T09:18:50.511 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94565.log 2024-04-25T09:18:50.512 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39884.log.gz 2024-04-25T09:18:50.512 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80528.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80528.log.gzgzip 2024-04-25T09:18:50.513 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76696.log 2024-04-25T09:18:50.513 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90683.log 2024-04-25T09:18:50.514 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94565.log.gz 2024-04-25T09:18:50.515 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95306.log 2024-04-25T09:18:50.515 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76696.log.gz 2024-04-25T09:18:50.516 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85214.log 2024-04-25T09:18:50.516 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90683.log.gz 2024-04-25T09:18:50.517 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45657.log 2024-04-25T09:18:50.517 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95306.log.gz 2024-04-25T09:18:50.518 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90725.log 2024-04-25T09:18:50.518 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85214.log.gz 2024-04-25T09:18:50.519 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43926.log 2024-04-25T09:18:50.519 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45657.log.gz 2024-04-25T09:18:50.520 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39548.log 2024-04-25T09:18:50.521 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90725.log.gz 2024-04-25T09:18:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61203.log 2024-04-25T09:18:50.522 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43926.log.gz 2024-04-25T09:18:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52920.log 2024-04-25T09:18:50.523 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39548.log.gz 2024-04-25T09:18:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.61203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61203.log.gz 2024-04-25T09:18:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62281.log 2024-04-25T09:18:50.525 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91719.log 2024-04-25T09:18:50.525 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52920.log.gz 2024-04-25T09:18:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79558.log 2024-04-25T09:18:50.526 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62281.log.gz 2024-04-25T09:18:50.527 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53629.log 2024-04-25T09:18:50.527 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91719.log.gz 2024-04-25T09:18:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91382.log 2024-04-25T09:18:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53629.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.79558.log.gz 2024-04-25T09:18:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52700.log 2024-04-25T09:18:50.530 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53629.log.gz 2024-04-25T09:18:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91382.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.91382.log.gz 2024-04-25T09:18:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42333.log 2024-04-25T09:18:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60388.log 2024-04-25T09:18:50.532 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52700.log.gz 2024-04-25T09:18:50.532 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42333.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T09:18:50.533 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42333.log.gz 2024-04-25T09:18:50.534 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60802.log 2024-04-25T09:18:50.534 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60388.log.gz 2024-04-25T09:18:50.535 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61800.log 2024-04-25T09:18:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60802.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60802.log.gz 2024-04-25T09:18:50.536 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53316.log 2024-04-25T09:18:50.537 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53850.log 2024-04-25T09:18:50.538 INFO:teuthology.orchestra.run.smithi062.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61800.log.gz 2024-04-25T09:18:50.538 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53316.log: 86.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T09:18:50.538 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85747.log 2024-04-25T09:18:50.539 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53316.log.gz 2024-04-25T09:18:50.540 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58026.log 2024-04-25T09:18:50.540 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53850.log.gz 2024-04-25T09:18:50.541 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39208.log 2024-04-25T09:18:50.541 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85747.log.gz 2024-04-25T09:18:50.542 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58549.log 2024-04-25T09:18:50.542 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58026.log.gz 2024-04-25T09:18:50.543 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70298.log 2024-04-25T09:18:50.543 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39208.log.gz 2024-04-25T09:18:50.544 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41178.log 2024-04-25T09:18:50.544 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58549.log.gz 2024-04-25T09:18:50.545 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44549.log 2024-04-25T09:18:50.546 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.41178.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70298.log.gz 2024-04-25T09:18:50.546 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.0.41178.log.gz /var/log/ceph/ceph-client.admin.70364.log 2024-04-25T09:18:50.546 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:18:50.547 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39544.log 2024-04-25T09:18:50.547 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44549.log.gz 2024-04-25T09:18:50.548 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58507.log 2024-04-25T09:18:50.549 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39544.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.70364.log.gz 2024-04-25T09:18:50.550 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53418.log 2024-04-25T09:18:50.550 INFO:teuthology.orchestra.run.smithi062.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39544.log.gz 2024-04-25T09:18:50.550 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61926.log 2024-04-25T09:18:50.551 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58507.log.gz 2024-04-25T09:18:50.551 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91898.log 2024-04-25T09:18:50.552 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53418.log.gz 2024-04-25T09:18:50.553 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70871.log 2024-04-25T09:18:50.553 INFO:teuthology.orchestra.run.smithi062.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61926.log.gz 2024-04-25T09:18:50.554 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57364.log 2024-04-25T09:18:50.554 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91898.log.gz 2024-04-25T09:18:50.555 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70525.log 2024-04-25T09:18:50.555 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57364.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70871.log.gz 2024-04-25T09:18:50.556 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T09:18:50.556 INFO:teuthology.orchestra.run.smithi062.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57364.log.gz 2024-04-25T09:18:50.557 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70609.log 2024-04-25T09:18:50.557 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70525.log.gz 2024-04-25T09:18:50.558 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80014.log 2024-04-25T09:18:50.559 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85130.log 2024-04-25T09:18:50.560 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70609.log.gz 2024-04-25T09:18:50.560 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75617.log 2024-04-25T09:18:50.561 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80014.log.gz 2024-04-25T09:18:50.562 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86403.log 2024-04-25T09:18:50.562 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85130.log.gz 2024-04-25T09:18:50.563 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63046.log 2024-04-25T09:18:50.563 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75617.log.gz 2024-04-25T09:18:50.564 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55766.log 2024-04-25T09:18:50.564 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86403.log.gz 2024-04-25T09:18:50.565 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76948.log 2024-04-25T09:18:50.565 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63046.log.gz 2024-04-25T09:18:50.566 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.55766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91164.log 2024-04-25T09:18:50.567 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55766.log.gz 2024-04-25T09:18:50.567 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38988.log 2024-04-25T09:18:50.567 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76948.log.gz 2024-04-25T09:18:50.568 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58759.log 2024-04-25T09:18:50.569 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91164.log.gz 2024-04-25T09:18:50.570 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40164.log 2024-04-25T09:18:50.570 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38988.log.gz 2024-04-25T09:18:50.570 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90827.log 2024-04-25T09:18:50.571 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58759.log.gz 2024-04-25T09:18:50.571 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43368.log 2024-04-25T09:18:50.572 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40164.log.gz 2024-04-25T09:18:50.573 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56498.log 2024-04-25T09:18:50.573 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90827.log.gz 2024-04-25T09:18:50.574 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84867.log 2024-04-25T09:18:50.574 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43368.log.gz 2024-04-25T09:18:50.574 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40458.log 2024-04-25T09:18:50.575 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56498.log.gz 2024-04-25T09:18:50.576 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75043.log 2024-04-25T09:18:50.576 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84867.log.gz 2024-04-25T09:18:50.577 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56126.log 2024-04-25T09:18:50.577 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40458.log.gz 2024-04-25T09:18:50.578 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.41657.log 2024-04-25T09:18:50.578 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75043.log.gz 2024-04-25T09:18:50.579 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76654.log 2024-04-25T09:18:50.580 INFO:teuthology.orchestra.run.smithi062.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56126.log.gz 2024-04-25T09:18:50.580 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.2.41657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.41657.log.gz 2024-04-25T09:18:50.580 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60261.log 2024-04-25T09:18:50.581 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45405.log 2024-04-25T09:18:50.581 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76654.log.gz 2024-04-25T09:18:50.582 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81331.log 2024-04-25T09:18:50.582 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60261.log.gz 2024-04-25T09:18:50.584 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72093.log 2024-04-25T09:18:50.585 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45405.log.gz 2024-04-25T09:18:50.585 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90953.log 2024-04-25T09:18:50.585 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81331.log.gz 2024-04-25T09:18:50.586 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70745.log 2024-04-25T09:18:50.586 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72093.log.gz 2024-04-25T09:18:50.586 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71675.log 2024-04-25T09:18:50.587 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90953.log.gz 2024-04-25T09:18:50.588 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63477.log 2024-04-25T09:18:50.588 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70745.log.gz 2024-04-25T09:18:50.588 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75449.log 2024-04-25T09:18:50.589 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71675.log.gz 2024-04-25T09:18:50.590 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80234.log 2024-04-25T09:18:50.590 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63477.log.gz 2024-04-25T09:18:50.591 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84825.log 2024-04-25T09:18:50.591 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75449.log.gz 2024-04-25T09:18:50.592 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54573.log 2024-04-25T09:18:50.592 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80234.log.gz 2024-04-25T09:18:50.593 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74515.log 2024-04-25T09:18:50.593 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84825.log.gz 2024-04-25T09:18:50.594 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94932.log 2024-04-25T09:18:50.594 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54573.log.gz 2024-04-25T09:18:50.595 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.74515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86877.log 2024-04-25T09:18:50.596 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94932.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.74515.log.gz 2024-04-25T09:18:50.596 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54909.log 2024-04-25T09:18:50.596 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94932.log.gz 2024-04-25T09:18:50.597 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58194.log 2024-04-25T09:18:50.597 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86877.log.gz 2024-04-25T09:18:50.598 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94183.log 2024-04-25T09:18:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58194.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54909.log.gz 2024-04-25T09:18:50.599 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87546.log 2024-04-25T09:18:50.600 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58194.log.gz 2024-04-25T09:18:50.600 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87215.log 2024-04-25T09:18:50.601 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94183.log.gz 2024-04-25T09:18:50.601 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.87546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85434.log 2024-04-25T09:18:50.602 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87546.log.gz 2024-04-25T09:18:50.602 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.87215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86793.log 2024-04-25T09:18:50.603 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85434.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.87215.log.gz 2024-04-25T09:18:50.604 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76272.log 2024-04-25T09:18:50.604 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85434.log.gz 2024-04-25T09:18:50.604 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72632.log 2024-04-25T09:18:50.605 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86793.log.gz 2024-04-25T09:18:50.606 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54138.log 2024-04-25T09:18:50.606 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76272.log.gz 2024-04-25T09:18:50.607 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58675.log 2024-04-25T09:18:50.607 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72632.log.gz 2024-04-25T09:18:50.608 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54138.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T09:18:50.608 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54138.log.gz 2024-04-25T09:18:50.609 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84183.log 2024-04-25T09:18:50.609 INFO:teuthology.orchestra.run.smithi062.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58675.log.gz 2024-04-25T09:18:50.610 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84351.log 2024-04-25T09:18:50.611 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61478.log 2024-04-25T09:18:50.611 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84183.log.gz 2024-04-25T09:18:50.612 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70913.log 2024-04-25T09:18:50.612 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84351.log.gz 2024-04-25T09:18:50.613 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61716.log 2024-04-25T09:18:50.613 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61478.log: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T09:18:50.614 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61478.log.gz 2024-04-25T09:18:50.614 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70913.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70913.log.gz 2024-04-25T09:18:50.615 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43452.log 2024-04-25T09:18:50.615 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55320.log 2024-04-25T09:18:50.616 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61716.log.gz 2024-04-25T09:18:50.616 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94523.log 2024-04-25T09:18:50.617 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43452.log.gz 2024-04-25T09:18:50.617 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.55320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42165.log 2024-04-25T09:18:50.618 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55320.log.gz 2024-04-25T09:18:50.619 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63309.log 2024-04-25T09:18:50.619 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94523.log.gz 2024-04-25T09:18:50.620 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44346.log 2024-04-25T09:18:50.620 INFO:teuthology.orchestra.run.smithi062.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42165.log.gz 2024-04-25T09:18:50.621 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40416.log 2024-04-25T09:18:50.621 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63309.log.gz 2024-04-25T09:18:50.622 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57890.log 2024-04-25T09:18:50.622 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44346.log.gz 2024-04-25T09:18:50.623 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58068.log 2024-04-25T09:18:50.623 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40416.log.gz 2024-04-25T09:18:50.624 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46049.log 2024-04-25T09:18:50.624 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57890.log.gz 2024-04-25T09:18:50.625 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44676.log 2024-04-25T09:18:50.626 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58068.log.gz 2024-04-25T09:18:50.626 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.46049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70213.log 2024-04-25T09:18:50.626 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46049.log.gz 2024-04-25T09:18:50.627 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91547.log 2024-04-25T09:18:50.628 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44676.log.gz 2024-04-25T09:18:50.628 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43278.log 2024-04-25T09:18:50.629 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.70213.log.gz 2024-04-25T09:18:50.630 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80780.log 2024-04-25T09:18:50.630 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91547.log.gz 2024-04-25T09:18:50.631 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54615.log 2024-04-25T09:18:50.631 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43278.log.gz 2024-04-25T09:18:50.631 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76314.log 2024-04-25T09:18:50.632 INFO:teuthology.orchestra.run.smithi062.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80780.log.gz 2024-04-25T09:18:50.633 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61298.log 2024-04-25T09:18:50.633 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54615.log.gz 2024-04-25T09:18:50.634 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84477.log 2024-04-25T09:18:50.634 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76314.log.gz 2024-04-25T09:18:50.635 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76528.log 2024-04-25T09:18:50.635 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61298.log.gz 2024-04-25T09:18:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54867.log 2024-04-25T09:18:50.636 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76528.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.84477.log.gz 2024-04-25T09:18:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71963.log 2024-04-25T09:18:50.637 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76528.log.gz 2024-04-25T09:18:50.638 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75085.log 2024-04-25T09:18:50.638 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54867.log.gz 2024-04-25T09:18:50.639 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86612.log 2024-04-25T09:18:50.639 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71963.log.gz 2024-04-25T09:18:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39211.log 2024-04-25T09:18:50.640 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75085.log.gz 2024-04-25T09:18:50.641 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56324.log 2024-04-25T09:18:50.641 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86612.log.gz 2024-04-25T09:18:50.642 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56432.log 2024-04-25T09:18:50.642 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39211.log.gz 2024-04-25T09:18:50.643 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44388.log 2024-04-25T09:18:50.644 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56324.log.gz 2024-04-25T09:18:50.644 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95875.log 2024-04-25T09:18:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44388.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56432.log.gz 2024-04-25T09:18:50.645 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94141.log 2024-04-25T09:18:50.645 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44388.log.gz 2024-04-25T09:18:50.646 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62831.log 2024-04-25T09:18:50.646 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95875.log.gz 2024-04-25T09:18:50.647 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89675.log 2024-04-25T09:18:50.647 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94141.log.gz 2024-04-25T09:18:50.648 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79264.log 2024-04-25T09:18:50.648 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62831.log.gz 2024-04-25T09:18:50.649 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90995.log 2024-04-25T09:18:50.649 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89675.log.gz 2024-04-25T09:18:50.650 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44468.log 2024-04-25T09:18:50.651 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79264.log.gz 2024-04-25T09:18:50.651 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38480.log 2024-04-25T09:18:50.652 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90995.log.gz 2024-04-25T09:18:50.652 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39883.log 2024-04-25T09:18:50.653 INFO:teuthology.orchestra.run.smithi062.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44468.log.gz 2024-04-25T09:18:50.653 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96253.log 2024-04-25T09:18:50.654 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38480.log.gz 2024-04-25T09:18:50.655 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53671.log 2024-04-25T09:18:50.655 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39883.log.gz 2024-04-25T09:18:50.656 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52389.log 2024-04-25T09:18:50.656 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96253.log.gz 2024-04-25T09:18:50.657 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45279.log 2024-04-25T09:18:50.657 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53671.log.gz 2024-04-25T09:18:50.658 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45489.log 2024-04-25T09:18:50.658 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45279.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52389.log.gz 2024-04-25T09:18:50.659 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43536.log 2024-04-25T09:18:50.659 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45279.log.gz 2024-04-25T09:18:50.660 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45489.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T09:18:50.660 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45489.log.gz 2024-04-25T09:18:50.661 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81085.log 2024-04-25T09:18:50.661 INFO:teuthology.orchestra.run.smithi062.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43536.log.gz 2024-04-25T09:18:50.662 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89905.log 2024-04-25T09:18:50.663 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42501.log 2024-04-25T09:18:50.663 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81085.log.gz 2024-04-25T09:18:50.664 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75973.log 2024-04-25T09:18:50.664 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89905.log.gz 2024-04-25T09:18:50.665 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86102.log 2024-04-25T09:18:50.665 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42501.log.gz 2024-04-25T09:18:50.666 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81826.log 2024-04-25T09:18:50.666 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75973.log.gz 2024-04-25T09:18:50.668 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56192.log 2024-04-25T09:18:50.668 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86102.log.gz 2024-04-25T09:18:50.669 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80738.log 2024-04-25T09:18:50.669 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81826.log.gz 2024-04-25T09:18:50.670 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43152.log 2024-04-25T09:18:50.671 INFO:teuthology.orchestra.run.smithi062.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56192.log.gz 2024-04-25T09:18:50.671 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80738.log: gzip -5 88.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.80738.log.gz -- 2024-04-25T09:18:50.671 INFO:teuthology.orchestra.run.smithi062.stderr: /var/log/ceph/ceph-client.admin.43656.log 2024-04-25T09:18:50.672 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80486.log 2024-04-25T09:18:50.672 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43152.log.gz 2024-04-25T09:18:50.673 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74647.log 2024-04-25T09:18:50.674 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43656.log.gz 2024-04-25T09:18:50.675 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45573.log 2024-04-25T09:18:50.675 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80486.log.gz 2024-04-25T09:18:50.676 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.74647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72186.log 2024-04-25T09:18:50.676 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74647.log.gz 2024-04-25T09:18:50.677 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72566.log 2024-04-25T09:18:50.677 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45573.log.gz 2024-04-25T09:18:50.678 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90389.log 2024-04-25T09:18:50.679 INFO:teuthology.orchestra.run.smithi062.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72186.log.gz 2024-04-25T09:18:50.679 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95016.log 2024-04-25T09:18:50.680 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72566.log.gz 2024-04-25T09:18:50.681 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90389.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90389.log.gz 2024-04-25T09:18:50.681 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95749.log 2024-04-25T09:18:50.682 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38736.log 2024-04-25T09:18:50.682 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95016.log.gz 2024-04-25T09:18:50.683 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79492.log 2024-04-25T09:18:50.683 INFO:teuthology.orchestra.run.smithi062.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95749.log.gz 2024-04-25T09:18:50.684 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96043.log 2024-04-25T09:18:50.684 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38736.log.gz 2024-04-25T09:18:50.685 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71225.log 2024-04-25T09:18:50.685 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.79492.log.gz 2024-04-25T09:18:50.686 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71352.log 2024-04-25T09:18:50.686 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96043.log.gz 2024-04-25T09:18:50.687 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71225.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71225.log.gz 2024-04-25T09:18:50.687 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53358.log 2024-04-25T09:18:50.688 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56937.log 2024-04-25T09:18:50.688 INFO:teuthology.orchestra.run.smithi062.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71352.log.gz 2024-04-25T09:18:50.689 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96127.log 2024-04-25T09:18:50.690 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53358.log.gz 2024-04-25T09:18:50.690 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56937.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56937.log.gz 2024-04-25T09:18:50.691 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.84993.log 2024-04-25T09:18:50.692 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38568.log 2024-04-25T09:18:50.692 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96127.log.gz 2024-04-25T09:18:50.693 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96169.log 2024-04-25T09:18:50.693 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84993.log.gz 2024-04-25T09:18:50.694 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41955.log 2024-04-25T09:18:50.694 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38568.log.gz 2024-04-25T09:18:50.695 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91206.log 2024-04-25T09:18:50.695 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96169.log.gz 2024-04-25T09:18:50.696 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.41955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92108.log 2024-04-25T09:18:50.696 INFO:teuthology.orchestra.run.smithi062.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41955.log.gz 2024-04-25T09:18:50.697 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94397.log 2024-04-25T09:18:50.697 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91206.log.gz 2024-04-25T09:18:50.698 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.92108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76570.log 2024-04-25T09:18:50.698 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92108.log.gz 2024-04-25T09:18:50.699 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58404.log 2024-04-25T09:18:50.699 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94397.log.gz 2024-04-25T09:18:50.700 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45447.log 2024-04-25T09:18:50.700 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76570.log.gz 2024-04-25T09:18:50.701 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71165.log 2024-04-25T09:18:50.701 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58404.log.gz 2024-04-25T09:18:50.702 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89821.log 2024-04-25T09:18:50.702 INFO:teuthology.orchestra.run.smithi062.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45447.log.gz 2024-04-25T09:18:50.703 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80056.log 2024-04-25T09:18:50.703 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71165.log.gz 2024-04-25T09:18:50.704 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39198.log 2024-04-25T09:18:50.704 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89821.log.gz 2024-04-25T09:18:50.705 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85298.log 2024-04-25T09:18:50.705 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80056.log.gz 2024-04-25T09:18:50.706 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61562.log 2024-04-25T09:18:50.706 INFO:teuthology.orchestra.run.smithi062.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39198.log.gz 2024-04-25T09:18:50.707 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75931.log 2024-04-25T09:18:50.707 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85298.log.gz 2024-04-25T09:18:50.708 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77056.log 2024-04-25T09:18:50.708 INFO:teuthology.orchestra.run.smithi062.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61562.log.gz 2024-04-25T09:18:50.709 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60625.log 2024-04-25T09:18:50.709 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75931.log.gz 2024-04-25T09:18:50.710 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.77056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89489.log 2024-04-25T09:18:50.710 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77056.log.gz 2024-04-25T09:18:50.711 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60625.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.60625.log.gz /var/log/ceph/ceph-client.admin.62958.log 2024-04-25T09:18:50.711 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:18:50.712 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57280.log 2024-04-25T09:18:50.713 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89489.log.gz 2024-04-25T09:18:50.713 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62958.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62958.log.gz 2024-04-25T09:18:50.713 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84393.log 2024-04-25T09:18:50.714 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45237.log 2024-04-25T09:18:50.715 INFO:teuthology.orchestra.run.smithi062.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57280.log.gz 2024-04-25T09:18:50.716 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84393.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.84393.log.gz 2024-04-25T09:18:50.716 INFO:teuthology.orchestra.run.smithi062.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86654.log 2024-04-25T09:18:50.716 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70147.log 2024-04-25T09:18:50.717 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45237.log.gz 2024-04-25T09:18:50.717 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42207.log 2024-04-25T09:18:50.718 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86654.log.gz 2024-04-25T09:18:50.718 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63585.log 2024-04-25T09:18:50.719 INFO:teuthology.orchestra.run.smithi062.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.70147.log.gz 2024-04-25T09:18:50.720 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42207.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T09:18:50.720 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42207.log.gz 2024-04-25T09:18:50.721 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41356.log 2024-04-25T09:18:50.721 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63585.log.gz 2024-04-25T09:18:50.722 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62491.log 2024-04-25T09:18:50.723 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.41356.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41356.log.gz 2024-04-25T09:18:50.723 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81784.log 2024-04-25T09:18:50.724 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42375.log 2024-04-25T09:18:50.725 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62491.log.gz 2024-04-25T09:18:50.726 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42753.log 2024-04-25T09:18:50.726 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81784.log.gz 2024-04-25T09:18:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42375.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42375.log.gz 2024-04-25T09:18:50.727 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85662.log 2024-04-25T09:18:50.728 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42753.log: /var/log/ceph/ceph-client.admin.71039.log 2024-04-25T09:18:50.729 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42753.log.gz 2024-04-25T09:18:50.729 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85662.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85662.log.gz 2024-04-25T09:18:50.730 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42123.log 2024-04-25T09:18:50.730 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90211.log 2024-04-25T09:18:50.731 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71039.log.gz 2024-04-25T09:18:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42123.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42123.log.gz 2024-04-25T09:18:50.732 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61842.log 2024-04-25T09:18:50.733 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95833.log 2024-04-25T09:18:50.733 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90211.log: /var/log/ceph/ceph-client.admin.61842.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90211.log.gz 2024-04-25T09:18:50.734 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61842.log.gz 2024-04-25T09:18:50.734 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90911.log 2024-04-25T09:18:50.735 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39204.log 2024-04-25T09:18:50.735 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95833.log.gz 2024-04-25T09:18:50.736 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90911.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.90911.log.gz 2024-04-25T09:18:50.737 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40080.log 2024-04-25T09:18:50.738 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38862.log 2024-04-25T09:18:50.739 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39204.log.gz 2024-04-25T09:18:50.739 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40080.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40080.log.gz 2024-04-25T09:18:50.739 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76780.log 2024-04-25T09:18:50.740 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89863.log 2024-04-25T09:18:50.741 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38862.log.gz 2024-04-25T09:18:50.741 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76780.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76780.log.gz 2024-04-25T09:18:50.742 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55510.log 2024-04-25T09:18:50.742 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96419.log 2024-04-25T09:18:50.743 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89863.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89863.log.gz 2024-04-25T09:18:50.743 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.55510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60345.log 2024-04-25T09:18:50.744 INFO:teuthology.orchestra.run.smithi062.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55510.log.gz 2024-04-25T09:18:50.745 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81521.log 2024-04-25T09:18:50.746 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96419.log.gz 2024-04-25T09:18:50.746 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60345.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60345.log.gz 2024-04-25T09:18:50.747 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2024-04-25T09:18:50.747 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5/var/log/ceph/ceph-client.admin.81521.log: --verbose -- /var/log/ceph/ceph-client.admin.44304.log 2024-04-25T09:18:50.748 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81521.log.gz 2024-04-25T09:18:50.749 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44844.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz -5 2024-04-25T09:18:50.749 INFO:teuthology.orchestra.run.smithi062.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86921.log 2024-04-25T09:18:50.749 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53503.log 2024-04-25T09:18:50.750 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44304.log.gz 2024-04-25T09:18:50.751 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86921.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86921.log.gz 2024-04-25T09:18:50.751 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62239.log 2024-04-25T09:18:50.752 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53503.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T09:18:50.752 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53503.log.gz 2024-04-25T09:18:50.753 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95348.log 2024-04-25T09:18:50.753 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62239.log.gz 2024-04-25T09:18:50.754 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T09:18:50.756 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33539.log 2024-04-25T09:18:50.756 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95348.log.gz 2024-04-25T09:18:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37981.log 2024-04-25T09:18:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.33539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54783.log 2024-04-25T09:18:50.758 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33539.log.gz 2024-04-25T09:18:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.37981.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37981.log.gz 2024-04-25T09:18:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38946.log 2024-04-25T09:18:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54783.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54783.log.gz 2024-04-25T09:18:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2024-04-25T09:18:50.763 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95665.log 2024-04-25T09:18:50.763 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38946.log.gz 2024-04-25T09:18:50.765 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39205.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2024-04-25T09:18:50.765 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79624.log 2024-04-25T09:18:50.766 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95665.log.gz 2024-04-25T09:18:50.766 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86528.log 2024-04-25T09:18:50.768 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81165.log 2024-04-25T09:18:50.768 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79624.log.gz 2024-04-25T09:18:50.770 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86698.log 2024-04-25T09:18:50.770 INFO:teuthology.orchestra.run.smithi062.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86528.log.gz 2024-04-25T09:18:50.771 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81742.log 2024-04-25T09:18:50.771 INFO:teuthology.orchestra.run.smithi062.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81165.log.gz 2024-04-25T09:18:50.772 INFO:teuthology.orchestra.run.smithi062.stderr: 94.6%/var/log/ceph/ceph-client.admin.86698.log: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T09:18:50.772 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86698.log.gz 2024-04-25T09:18:50.773 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80360.log 2024-04-25T09:18:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81742.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81742.log.gz 2024-04-25T09:18:50.774 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81868.log 2024-04-25T09:18:50.775 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90295.log 2024-04-25T09:18:50.775 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80360.log.gz 2024-04-25T09:18:50.777 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59058.log 2024-04-25T09:18:50.777 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81868.log.gz 2024-04-25T09:18:50.778 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74780.log 2024-04-25T09:18:50.778 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90295.log.gz 2024-04-25T09:18:50.779 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60618.log 2024-04-25T09:18:50.780 INFO:teuthology.orchestra.run.smithi062.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59058.log.gz 2024-04-25T09:18:50.781 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.74780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79366.log 2024-04-25T09:18:50.781 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74780.log.gz 2024-04-25T09:18:50.782 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90127.log 2024-04-25T09:18:50.782 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60618.log.gz 2024-04-25T09:18:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80318.log 2024-04-25T09:18:50.783 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79366.log.gz 2024-04-25T09:18:50.785 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76110.log 2024-04-25T09:18:50.785 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90127.log.gz 2024-04-25T09:18:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80318.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T09:18:50.786 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80318.log.gz 2024-04-25T09:18:50.787 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71394.log 2024-04-25T09:18:50.787 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76110.log.gz 2024-04-25T09:18:50.789 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38311.log 2024-04-25T09:18:50.790 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54825.log 2024-04-25T09:18:50.790 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71394.log.gz 2024-04-25T09:18:50.791 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81994.log 2024-04-25T09:18:50.792 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38311.log.gz 2024-04-25T09:18:50.793 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54825.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54825.log.gz 2024-04-25T09:18:50.794 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59535.log 2024-04-25T09:18:50.794 INFO:teuthology.orchestra.run.smithi121.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T09:18:50.795 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81994.log: gzip -5 --verbose 91.0% -- /var/log/ceph/ceph-client.admin.76864.log -- replaced with /var/log/ceph/ceph-client.admin.81994.log.gz 2024-04-25T09:18:50.795 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:18:50.796 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42249.log 2024-04-25T09:18:50.797 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59535.log.gz 2024-04-25T09:18:50.798 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76864.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76864.log.gz 2024-04-25T09:18:50.798 INFO:teuthology.orchestra.run.smithi062.stderr: /var/log/ceph/ceph-client.admin.61884.log 2024-04-25T09:18:50.799 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42249.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42249.log.gz 2024-04-25T09:18:50.799 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85705.log 2024-04-25T09:18:50.801 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95539.log 2024-04-25T09:18:50.801 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61884.log.gz 2024-04-25T09:18:50.802 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54975.log 2024-04-25T09:18:50.802 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85705.log.gz 2024-04-25T09:18:50.804 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95539.log: gzip -5 --verbose 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95539.log.gz 2024-04-25T09:18:50.804 INFO:teuthology.orchestra.run.smithi062.stderr: -- /var/log/ceph/ceph-client.admin.89633.log 2024-04-25T09:18:50.805 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54975.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54975.log.gz 2024-04-25T09:18:50.805 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83943.log 2024-04-25T09:18:50.806 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54011.log 2024-04-25T09:18:50.807 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89633.log.gz 2024-04-25T09:18:50.808 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.83943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85831.log 2024-04-25T09:18:50.809 INFO:teuthology.orchestra.run.smithi062.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.83943.log.gz/var/log/ceph/ceph-client.admin.54011.log: 2024-04-25T09:18:50.809 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80402.log 2024-04-25T09:18:50.810 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54011.log.gz 2024-04-25T09:18:50.811 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75221.log 2024-04-25T09:18:50.811 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85831.log.gz 2024-04-25T09:18:50.813 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80402.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80402.log.gz 2024-04-25T09:18:50.813 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.92150.log 2024-04-25T09:18:50.813 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75221.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75221.log.gz 2024-04-25T09:18:50.814 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57604.log 2024-04-25T09:18:50.815 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.92150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76356.log 2024-04-25T09:18:50.816 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92150.log.gz 2024-04-25T09:18:50.817 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38524.log 2024-04-25T09:18:50.817 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57604.log.gz 2024-04-25T09:18:50.818 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33155.log 2024-04-25T09:18:50.819 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76356.log.gz 2024-04-25T09:18:50.820 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72314.log 2024-04-25T09:18:50.820 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38524.log.gz 2024-04-25T09:18:50.821 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.33155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85891.log 2024-04-25T09:18:50.821 INFO:teuthology.orchestra.run.smithi062.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33155.log.gz 2024-04-25T09:18:50.822 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72314.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72314.log.gz 2024-04-25T09:18:50.823 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76822.log 2024-04-25T09:18:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85891.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85891.log.gz 2024-04-25T09:18:50.824 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72005.log 2024-04-25T09:18:50.826 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62874.log 2024-04-25T09:18:50.826 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76822.log.gz 2024-04-25T09:18:50.827 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72005.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72005.log.gz 2024-04-25T09:18:50.827 INFO:teuthology.orchestra.run.smithi062.stderr: -- /var/log/ceph/ceph-client.admin.79930.log 2024-04-25T09:18:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62874.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62874.log.gz 2024-04-25T09:18:50.828 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55704.log 2024-04-25T09:18:50.830 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60219.log 2024-04-25T09:18:50.830 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79930.log.gz 2024-04-25T09:18:50.832 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.55704.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55704.log.gz 2024-04-25T09:18:50.832 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53214.log 2024-04-25T09:18:50.833 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57111.log 2024-04-25T09:18:50.833 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60219.log.gz 2024-04-25T09:18:50.835 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43110.log 2024-04-25T09:18:50.835 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53214.log.gz 2024-04-25T09:18:50.837 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57111.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57111.log.gz 2024-04-25T09:18:50.837 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42585.log 2024-04-25T09:18:50.837 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43110.log.gz 2024-04-25T09:18:50.838 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59637.log 2024-04-25T09:18:50.840 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-04-25T09:18:50.840 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42585.log.gz 2024-04-25T09:18:50.841 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59637.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59637.log.gz 2024-04-25T09:18:50.842 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85035.log 2024-04-25T09:18:50.842 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38694.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-04-25T09:18:50.843 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75491.log 2024-04-25T09:18:50.844 INFO:teuthology.orchestra.run.smithi121.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T09:18:50.844 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86323.log 2024-04-25T09:18:50.845 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85035.log.gz 2024-04-25T09:18:50.845 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75491.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75491.log.gz 2024-04-25T09:18:50.846 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41787.log 2024-04-25T09:18:50.847 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90169.log 2024-04-25T09:18:50.848 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86323.log.gz 2024-04-25T09:18:50.849 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.41787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52784.log 2024-04-25T09:18:50.849 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41787.log.gz 2024-04-25T09:18:50.850 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41829.log 2024-04-25T09:18:50.850 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90169.log.gz 2024-04-25T09:18:50.851 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52784.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52784.log.gz 2024-04-25T09:18:50.853 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.41829.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41829.log.gz 2024-04-25T09:18:50.853 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76484.log 2024-04-25T09:18:50.854 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61968.log 2024-04-25T09:18:50.856 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72398.log 2024-04-25T09:18:50.856 INFO:teuthology.orchestra.run.smithi062.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76484.log.gz 2024-04-25T09:18:50.857 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81477.log 2024-04-25T09:18:50.858 INFO:teuthology.orchestra.run.smithi062.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61968.log.gz 2024-04-25T09:18:50.859 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59595.log 2024-04-25T09:18:50.859 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72398.log.gz 2024-04-25T09:18:50.861 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81477.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81477.log.gz 2024-04-25T09:18:50.862 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39539.log 2024-04-25T09:18:50.862 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59595.log.gz 2024-04-25T09:18:50.862 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71836.log 2024-04-25T09:18:50.864 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91812.log 2024-04-25T09:18:50.864 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39539.log.gz 2024-04-25T09:18:50.866 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71836.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71836.log.gz 2024-04-25T09:18:50.866 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80906.log 2024-04-25T09:18:50.867 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91812.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91812.log.gz 2024-04-25T09:18:50.868 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55576.log 2024-04-25T09:18:50.869 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39543.log 2024-04-25T09:18:50.869 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80906.log.gz 2024-04-25T09:18:50.871 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.55576.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55576.log.gz 2024-04-25T09:18:50.872 INFO:teuthology.orchestra.run.smithi062.stderr:gzip/var/log/ceph/ceph-client.admin.39543.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59763.log 2024-04-25T09:18:50.872 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39543.log.gz 2024-04-25T09:18:50.873 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59449.log 2024-04-25T09:18:50.874 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54268.log 2024-04-25T09:18:50.874 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59763.log.gz 2024-04-25T09:18:50.876 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59449.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59449.log.gz 2024-04-25T09:18:50.877 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54268.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54268.log.gz 2024-04-25T09:18:50.878 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52962.log 2024-04-25T09:18:50.879 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38438.log 2024-04-25T09:18:50.880 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45970.log 2024-04-25T09:18:50.880 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52962.log.gz 2024-04-25T09:18:50.882 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38438.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38438.log.gz 2024-04-25T09:18:50.883 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.45970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74581.log 2024-04-25T09:18:50.883 INFO:teuthology.orchestra.run.smithi062.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45970.log.gz 2024-04-25T09:18:50.883 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52473.log 2024-04-25T09:18:50.885 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.74581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45193.log 2024-04-25T09:18:50.887 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74581.log.gz 2024-04-25T09:18:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80098.log 2024-04-25T09:18:50.887 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45193.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52473.log.gz 2024-04-25T09:18:50.888 INFO:teuthology.orchestra.run.smithi062.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45193.log.gz 2024-04-25T09:18:50.889 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91122.log 2024-04-25T09:18:50.890 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80098.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95581.log 2024-04-25T09:18:50.890 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80098.log.gz 2024-04-25T09:18:50.891 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56788.log 2024-04-25T09:18:50.891 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91122.log.gz 2024-04-25T09:18:50.893 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63139.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95581.log.gz 2024-04-25T09:18:50.893 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:18:50.894 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75720.log 2024-04-25T09:18:50.894 INFO:teuthology.orchestra.run.smithi062.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56788.log.gz 2024-04-25T09:18:50.896 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63267.log 2024-04-25T09:18:50.896 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63139.log.gz 2024-04-25T09:18:50.897 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75720.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75720.log.gz 2024-04-25T09:18:50.898 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63267.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63267.log.gz 2024-04-25T09:18:50.899 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44802.log 2024-04-25T09:18:50.900 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84759.log 2024-04-25T09:18:50.902 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86742.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44802.log.gz 2024-04-25T09:18:50.902 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:18:50.903 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54403.log 2024-04-25T09:18:50.903 INFO:teuthology.orchestra.run.smithi062.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.84759.log.gz 2024-04-25T09:18:50.904 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86742.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T09:18:50.904 INFO:teuthology.orchestra.run.smithi062.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86742.log.gz 2024-04-25T09:18:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40374.log 2024-04-25T09:18:50.907 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3%/var/log/ceph/ceph-mon.a.log: -- replaced with /var/log/ceph/ceph-client.admin.54403.log.gz 2024-04-25T09:18:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53267.log 2024-04-25T09:18:50.909 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40374.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40374.log.gz 2024-04-25T09:18:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53046.log 2024-04-25T09:18:50.910 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53267.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53267.log.gz 2024-04-25T09:18:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78940.log 2024-04-25T09:18:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53046.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53046.log.gz 2024-04-25T09:18:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85476.log 2024-04-25T09:18:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.78940.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78940.log.gz 2024-04-25T09:18:50.916 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.85476.log: -- /var/log/ceph/ceph-client.admin.42081.log 2024-04-25T09:18:50.916 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85476.log.gz 2024-04-25T09:18:50.917 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54224.log 2024-04-25T09:18:50.918 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42081.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42081.log.gz 2024-04-25T09:18:50.919 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.54224.log: /var/log/ceph/ceph-client.admin.91463.log 2024-04-25T09:18:50.920 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54224.log.gz 2024-04-25T09:18:50.921 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94848.log 2024-04-25T09:18:50.923 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91463.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.91463.log.gz 2024-04-25T09:18:50.923 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72356.log 2024-04-25T09:18:50.924 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94848.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94848.log.gz 2024-04-25T09:18:50.925 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79834.log 2024-04-25T09:18:50.926 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72356.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72356.log.gz 2024-04-25T09:18:50.927 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56390.log 2024-04-25T09:18:50.928 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79834.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79834.log.gz 2024-04-25T09:18:50.929 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85536.log 2024-04-25T09:18:50.932 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56390.log: 88.0%gzip -5/var/log/ceph/ceph-client.admin.85536.log: --verbose -- /var/log/ceph/ceph-client.admin.71538.log 2024-04-25T09:18:50.932 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85536.log.gz 2024-04-25T09:18:50.932 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56390.log.gz 2024-04-25T09:18:50.933 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53545.log 2024-04-25T09:18:50.935 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71538.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71538.log.gz 2024-04-25T09:18:50.935 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55382.log 2024-04-25T09:18:50.936 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53545.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53545.log.gz 2024-04-25T09:18:50.937 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58320.log 2024-04-25T09:18:50.939 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.55382.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55382.log.gz 2024-04-25T09:18:50.940 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5/var/log/ceph/ceph-client.admin.58320.log: --verbose -- /var/log/ceph/ceph-client.admin.86228.log 2024-04-25T09:18:50.940 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58320.log.gz 2024-04-25T09:18:50.941 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58152.log 2024-04-25T09:18:50.942 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86228.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86228.log.gz 2024-04-25T09:18:50.944 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85578.log 2024-04-25T09:18:50.944 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58152.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58152.log.gz 2024-04-25T09:18:50.945 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86486.log 2024-04-25T09:18:50.946 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85578.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85578.log.gz 2024-04-25T09:18:50.947 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91631.log/var/log/ceph/ceph-client.admin.86486.log: 2024-04-25T09:18:50.948 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86486.log.gz 2024-04-25T09:18:50.949 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89249.log 2024-04-25T09:18:50.950 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91631.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91631.log.gz 2024-04-25T09:18:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89249.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.89249.log.gz 2024-04-25T09:18:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44010.log 2024-04-25T09:18:50.954 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76015.log 2024-04-25T09:18:50.955 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44010.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44010.log.gz 2024-04-25T09:18:50.956 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70829.log 2024-04-25T09:18:50.957 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76015.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76015.log.gz 2024-04-25T09:18:50.958 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80948.log 2024-04-25T09:18:50.959 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70829.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70829.log.gz 2024-04-25T09:18:50.960 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38269.log 2024-04-25T09:18:50.960 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80948.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80948.log.gz 2024-04-25T09:18:50.961 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96085.log 2024-04-25T09:18:50.962 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38269.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38269.log.gz 2024-04-25T09:18:50.964 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.96085.log: -- /var/log/ceph/ceph-client.admin.62094.log 2024-04-25T09:18:50.965 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96085.log.gz 2024-04-25T09:18:50.966 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79450.log 2024-04-25T09:18:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62094.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62094.log.gz 2024-04-25T09:18:50.967 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96295.log 2024-04-25T09:18:50.969 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79450.log: 88.5%gzip -5/var/log/ceph/ceph-client.admin.96295.log: --verbose -- /var/log/ceph/ceph-client.admin.94236.log 2024-04-25T09:18:50.969 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79450.log.gz 2024-04-25T09:18:50.969 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96295.log.gz 2024-04-25T09:18:50.971 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94236.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94236.log.gz 2024-04-25T09:18:50.972 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84951.log 2024-04-25T09:18:50.974 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57984.log 2024-04-25T09:18:50.975 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84951.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84951.log.gz 2024-04-25T09:18:50.976 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62916.log 2024-04-25T09:18:50.976 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57984.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57984.log.gz 2024-04-25T09:18:50.977 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39547.log 2024-04-25T09:18:50.978 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62916.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62916.log.gz 2024-04-25T09:18:50.980 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43494.log 2024-04-25T09:18:50.980 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39547.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39547.log.gz 2024-04-25T09:18:50.981 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40332.log 2024-04-25T09:18:50.982 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43494.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43494.log.gz 2024-04-25T09:18:50.984 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40332.log: -- /var/log/ceph/ceph-client.admin.82060.log 2024-04-25T09:18:50.984 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40332.log.gz 2024-04-25T09:18:50.985 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86570.log 2024-04-25T09:18:50.986 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.82060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82060.log.gz 2024-04-25T09:18:50.987 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43410.log 2024-04-25T09:18:50.988 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86570.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86570.log.gz 2024-04-25T09:18:50.989 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72228.log 2024-04-25T09:18:50.991 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43410.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43410.log.gz 2024-04-25T09:18:50.991 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38092.log 2024-04-25T09:18:50.991 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72228.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72228.log.gz 2024-04-25T09:18:50.992 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75347.log 2024-04-25T09:18:50.993 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38092.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38092.log.gz 2024-04-25T09:18:50.994 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79408.log 2024-04-25T09:18:50.995 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75347.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75347.log.gz 2024-04-25T09:18:50.996 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59184.log 2024-04-25T09:18:50.997 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79408.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79408.log.gz 2024-04-25T09:18:50.998 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76990.log 2024-04-25T09:18:51.000 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59184.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59184.log.gz 2024-04-25T09:18:51.000 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40626.log 2024-04-25T09:18:51.002 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76990.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.76990.log.gz 2024-04-25T09:18:51.002 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40626.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40626.log.gz 2024-04-25T09:18:51.003 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75762.log 2024-04-25T09:18:51.004 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71310.log 2024-04-25T09:18:51.006 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75762.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75762.log.gz 2024-04-25T09:18:51.006 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95791.log 2024-04-25T09:18:51.008 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71310.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71310.log.gz 2024-04-25T09:18:51.008 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39030.log 2024-04-25T09:18:51.009 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95791.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95791.log.gz 2024-04-25T09:18:51.010 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55638.log 2024-04-25T09:18:51.011 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39030.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39030.log.gz 2024-04-25T09:18:51.012 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40542.log/var/log/ceph/ceph-client.admin.55638.log: 2024-04-25T09:18:51.013 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55638.log.gz 2024-04-25T09:18:51.014 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79972.log 2024-04-25T09:18:51.015 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40542.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40542.log.gz 2024-04-25T09:18:51.016 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44052.log 2024-04-25T09:18:51.017 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79972.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79972.log.gz 2024-04-25T09:18:51.017 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40290.log 2024-04-25T09:18:51.019 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59100.log 2024-04-25T09:18:51.020 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44052.log.gz 2024-04-25T09:18:51.021 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40290.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40290.log.gz 2024-04-25T09:18:51.022 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59100.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59100.log.gz 2024-04-25T09:18:51.023 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91940.log 2024-04-25T09:18:51.024 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59142.log 2024-04-25T09:18:51.026 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91940.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91940.log.gz 2024-04-25T09:18:51.026 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71878.log 2024-04-25T09:18:51.027 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59142.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59142.log.gz 2024-04-25T09:18:51.028 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45321.log 2024-04-25T09:18:51.029 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71878.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71878.log.gz 2024-04-25T09:18:51.030 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45321.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45321.log.gz 2024-04-25T09:18:51.031 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95222.log 2024-04-25T09:18:51.033 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59407.log 2024-04-25T09:18:51.034 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95222.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95222.log.gz 2024-04-25T09:18:51.035 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59721.log 2024-04-25T09:18:51.035 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59407.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59407.log.gz 2024-04-25T09:18:51.036 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42291.log 2024-04-25T09:18:51.037 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59721.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59721.log.gz 2024-04-25T09:18:51.038 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42291.log: -- /var/log/ceph/ceph-client.admin.56023.log 2024-04-25T09:18:51.039 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42291.log.gz 2024-04-25T09:18:51.040 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79792.log 2024-04-25T09:18:51.042 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56023.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56023.log.gz 2024-04-25T09:18:51.043 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72524.log 2024-04-25T09:18:51.043 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79792.log.gz 2024-04-25T09:18:51.044 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44634.log 2024-04-25T09:18:51.045 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72524.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72524.log.gz 2024-04-25T09:18:51.046 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94278.log 2024-04-25T09:18:51.047 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44634.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44634.log.gz 2024-04-25T09:18:51.048 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39538.log 2024-04-25T09:18:51.050 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94278.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.94278.log.gz 2024-04-25T09:18:51.051 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95707.log 2024-04-25T09:18:51.051 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39538.log.gz 2024-04-25T09:18:51.052 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92258.log 2024-04-25T09:18:51.054 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95707.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95707.log.gz 2024-04-25T09:18:51.055 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.92258.log: /var/log/ceph/ceph-client.admin.84009.log 2024-04-25T09:18:51.055 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92258.log.gz 2024-04-25T09:18:51.056 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86963.log 2024-04-25T09:18:51.058 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84009.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.84009.log.gz 2024-04-25T09:18:51.059 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89990.log 2024-04-25T09:18:51.059 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86963.log.gz 2024-04-25T09:18:51.060 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94890.log 2024-04-25T09:18:51.061 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89990.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89990.log.gz 2024-04-25T09:18:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44718.log/var/log/ceph/ceph-client.admin.94890.log: 2024-04-25T09:18:51.063 INFO:teuthology.orchestra.run.smithi062.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94890.log.gz 2024-04-25T09:18:51.064 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39882.log 2024-04-25T09:18:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44718.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44718.log.gz 2024-04-25T09:18:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42627.log 2024-04-25T09:18:51.066 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39882.log.gz 2024-04-25T09:18:51.067 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57158.log 2024-04-25T09:18:51.069 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42627.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42627.log.gz 2024-04-25T09:18:51.070 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57158.log: /var/log/ceph/ceph-client.admin.94974.log 2024-04-25T09:18:51.070 INFO:teuthology.orchestra.run.smithi062.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57158.log.gz 2024-04-25T09:18:51.071 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89381.log 2024-04-25T09:18:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94974.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94974.log.gz 2024-04-25T09:18:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33423.log 2024-04-25T09:18:51.074 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89381.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.89381.log.gz 2024-04-25T09:18:51.076 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.33423.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33423.log.gz 2024-04-25T09:18:51.076 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59365.log 2024-04-25T09:18:51.078 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94439.log 2024-04-25T09:18:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59365.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59365.log.gz 2024-04-25T09:18:51.080 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84141.log 2024-04-25T09:18:51.081 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94439.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94439.log.gz 2024-04-25T09:18:51.082 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80570.log 2024-04-25T09:18:51.083 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84141.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84141.log.gz 2024-04-25T09:18:51.084 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80570.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80570.log.gz 2024-04-25T09:18:51.085 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39203.log 2024-04-25T09:18:51.086 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54447.log 2024-04-25T09:18:51.087 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39203.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39203.log.gz 2024-04-25T09:18:51.088 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71268.log 2024-04-25T09:18:51.089 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54447.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54447.log.gz 2024-04-25T09:18:51.090 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53004.log 2024-04-25T09:18:51.091 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71268.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71268.log.gz 2024-04-25T09:18:51.093 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53004.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53004.log.gz 2024-04-25T09:18:51.093 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39875.log 2024-04-25T09:18:51.095 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92066.log 2024-04-25T09:18:51.096 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39875.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39875.log.gz 2024-04-25T09:18:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61520.log 2024-04-25T09:18:51.097 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.92066.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92066.log.gz 2024-04-25T09:18:51.098 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89315.log 2024-04-25T09:18:51.099 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61520.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61520.log.gz 2024-04-25T09:18:51.100 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54180.log 2024-04-25T09:18:51.102 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89315.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.89315.log.gz 2024-04-25T09:18:51.102 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70081.log 2024-04-25T09:18:51.103 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54180.log.gz 2024-04-25T09:18:51.103 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58236.log 2024-04-25T09:18:51.105 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91589.log 2024-04-25T09:18:51.106 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58236.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.70081.log.gz 2024-04-25T09:18:51.106 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58236.log.gz 2024-04-25T09:18:51.108 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95623.log 2024-04-25T09:18:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91589.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91589.log.gz 2024-04-25T09:18:51.109 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38354.log 2024-04-25T09:18:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95623.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95623.log.gz 2024-04-25T09:18:51.111 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40500.log 2024-04-25T09:18:51.112 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38354.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38354.log.gz 2024-04-25T09:18:51.113 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53713.log 2024-04-25T09:18:51.114 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40500.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40500.log.gz 2024-04-25T09:18:51.116 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53713.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53713.log.gz 2024-04-25T09:18:51.116 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76738.log 2024-04-25T09:18:51.118 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43740.log 2024-04-25T09:18:51.119 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76738.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76738.log.gz 2024-04-25T09:18:51.119 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43968.log 2024-04-25T09:18:51.121 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54531.log 2024-04-25T09:18:51.122 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43968.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43740.log.gz 2024-04-25T09:18:51.122 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43968.log.gz 2024-04-25T09:18:51.124 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75001.log 2024-04-25T09:18:51.124 INFO:teuthology.orchestra.run.smithi121.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T09:18:51.125 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54531.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54531.log.gz 2024-04-25T09:18:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70997.log 2024-04-25T09:18:51.126 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75001.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75001.log.gz 2024-04-25T09:18:51.127 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54053.log 2024-04-25T09:18:51.129 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70997.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70997.log.gz 2024-04-25T09:18:51.130 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79708.log 2024-04-25T09:18:51.131 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54053.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54053.log.gz 2024-04-25T09:18:51.132 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70567.log 2024-04-25T09:18:51.132 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79708.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79708.log.gz 2024-04-25T09:18:51.133 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80192.log 2024-04-25T09:18:51.134 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70567.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70567.log.gz 2024-04-25T09:18:51.135 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56594.log 2024-04-25T09:18:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80192.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80192.log.gz 2024-04-25T09:18:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81574.log 2024-04-25T09:18:51.138 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56594.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56594.log.gz 2024-04-25T09:18:51.139 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59493.log 2024-04-25T09:18:51.140 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81574.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81574.log.gz 2024-04-25T09:18:51.143 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.59493.log: /var/log/ceph/ceph-client.admin.62136.log 2024-04-25T09:18:51.143 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59493.log.gz 2024-04-25T09:18:51.143 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95150.log 2024-04-25T09:18:51.144 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62136.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62136.log.gz 2024-04-25T09:18:51.145 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61604.log 2024-04-25T09:18:51.145 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95150.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95150.log.gz 2024-04-25T09:18:51.146 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96379.log 2024-04-25T09:18:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61604.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61604.log.gz 2024-04-25T09:18:51.148 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96337.log 2024-04-25T09:18:51.149 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96379.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96379.log.gz 2024-04-25T09:18:51.150 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42459.log 2024-04-25T09:18:51.151 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96337.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96337.log.gz 2024-04-25T09:18:51.152 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87281.log 2024-04-25T09:18:51.153 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42459.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42459.log.gz 2024-04-25T09:18:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72272.log 2024-04-25T09:18:51.155 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.87281.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87281.log.gz 2024-04-25T09:18:51.156 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91037.log 2024-04-25T09:18:51.157 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72272.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72272.log.gz 2024-04-25T09:18:51.158 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40710.log 2024-04-25T09:18:51.159 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91037.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91037.log.gz 2024-04-25T09:18:51.160 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44591.log 2024-04-25T09:18:51.161 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40710.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40710.log.gz 2024-04-25T09:18:51.162 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44591.log: -- /var/log/ceph/ceph-client.admin.58801.log 2024-04-25T09:18:51.162 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44591.log.gz 2024-04-25T09:18:51.163 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75533.log 2024-04-25T09:18:51.165 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58801.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58801.log.gz 2024-04-25T09:18:51.166 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5/var/log/ceph/ceph-client.admin.75533.log: --verbose -- /var/log/ceph/ceph-client.admin.90032.log 2024-04-25T09:18:51.166 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75533.log.gz 2024-04-25T09:18:51.167 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96211.log 2024-04-25T09:18:51.171 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90032.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90032.log.gz 2024-04-25T09:18:51.171 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44928.log 2024-04-25T09:18:51.172 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.96211.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96211.log.gz 2024-04-25T09:18:51.172 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52539.log 2024-04-25T09:18:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44928.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44928.log.gz 2024-04-25T09:18:51.173 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38778.log 2024-04-25T09:18:51.174 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52539.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52539.log.gz 2024-04-25T09:18:51.176 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38778.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38778.log.gz 2024-04-25T09:18:51.176 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43698.log 2024-04-25T09:18:51.178 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95100.log 2024-04-25T09:18:51.179 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43698.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43698.log.gz 2024-04-25T09:18:51.180 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42948.log 2024-04-25T09:18:51.181 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95100.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95100.log.gz 2024-04-25T09:18:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58976.log/var/log/ceph/ceph-client.admin.42948.log: 2024-04-25T09:18:51.183 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42948.log.gz 2024-04-25T09:18:51.183 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45058.log 2024-04-25T09:18:51.185 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58976.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58976.log.gz 2024-04-25T09:18:51.185 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43782.log 2024-04-25T09:18:51.186 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45058.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45058.log.gz 2024-04-25T09:18:51.187 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85789.log 2024-04-25T09:18:51.188 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43782.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43782.log.gz 2024-04-25T09:18:51.189 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80696.log 2024-04-25T09:18:51.190 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85789.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85789.log.gz 2024-04-25T09:18:51.191 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80696.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80696.log.gz 2024-04-25T09:18:51.192 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52742.log 2024-04-25T09:18:51.193 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62196.log 2024-04-25T09:18:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.52742.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52742.log.gz 2024-04-25T09:18:51.195 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5/var/log/ceph/ceph-client.admin.62196.log: --verbose -- /var/log/ceph/ceph-client.admin.80654.log 2024-04-25T09:18:51.196 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62196.log.gz 2024-04-25T09:18:51.197 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85620.log 2024-04-25T09:18:51.198 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80654.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80654.log.gz 2024-04-25T09:18:51.199 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74959.log 2024-04-25T09:18:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85620.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85620.log.gz 2024-04-25T09:18:51.201 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75305.log 2024-04-25T09:18:51.202 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.74959.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74959.log.gz 2024-04-25T09:18:51.204 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75305.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75305.log.gz 2024-04-25T09:18:51.204 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70955.log 2024-04-25T09:18:51.205 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57848.log 2024-04-25T09:18:51.207 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.70955.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70955.log.gz 2024-04-25T09:18:51.207 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61016.log 2024-04-25T09:18:51.209 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57848.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57848.log.gz 2024-04-25T09:18:51.209 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54741.log 2024-04-25T09:18:51.210 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61016.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61016.log.gz 2024-04-25T09:18:51.211 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94764.log 2024-04-25T09:18:51.212 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54741.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54741.log.gz 2024-04-25T09:18:51.213 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56070.log 2024-04-25T09:18:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94764.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94764.log.gz 2024-04-25T09:18:51.215 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39871.log 2024-04-25T09:18:51.216 INFO:teuthology.orchestra.run.smithi062.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.56070.log.gz 2024-04-25T09:18:51.217 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55162.log 2024-04-25T09:18:51.218 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39871.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39871.log.gz 2024-04-25T09:18:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72049.log 2024-04-25T09:18:51.219 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.55162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55162.log.gz 2024-04-25T09:18:51.220 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87047.log 2024-04-25T09:18:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72049.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72049.log.gz 2024-04-25T09:18:51.222 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60864.log 2024-04-25T09:18:51.224 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.87047.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71921.log/var/log/ceph/ceph-client.admin.60864.log: 2024-04-25T09:18:51.225 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87047.log.gz 2024-04-25T09:18:51.225 INFO:teuthology.orchestra.run.smithi062.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60864.log.gz 2024-04-25T09:18:51.226 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63850.log 2024-04-25T09:18:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71921.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71921.log.gz 2024-04-25T09:18:51.228 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79072.log 2024-04-25T09:18:51.229 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63850.log.gz 2024-04-25T09:18:51.230 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94806.log 2024-04-25T09:18:51.232 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79072.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.79072.log.gz 2024-04-25T09:18:51.233 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45151.log 2024-04-25T09:18:51.233 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94806.log.gz 2024-04-25T09:18:51.234 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58456.log 2024-04-25T09:18:51.235 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45151.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45151.log.gz 2024-04-25T09:18:51.236 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87173.log/var/log/ceph/ceph-client.admin.58456.log: 2024-04-25T09:18:51.237 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58456.log.gz 2024-04-25T09:18:51.238 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38396.log 2024-04-25T09:18:51.239 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.87173.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87173.log.gz 2024-04-25T09:18:51.241 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.38396.log: -- /var/log/ceph/ceph-client.admin.84309.log 2024-04-25T09:18:51.241 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38396.log.gz 2024-04-25T09:18:51.242 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84543.log 2024-04-25T09:18:51.243 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84309.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84309.log.gz 2024-04-25T09:18:51.244 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86060.log 2024-04-25T09:18:51.245 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84543.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.84543.log.gz 2024-04-25T09:18:51.246 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85172.log 2024-04-25T09:18:51.247 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86060.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86060.log.gz 2024-04-25T09:18:51.248 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85172.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85172.log.gz 2024-04-25T09:18:51.249 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-04-25T09:18:51.250 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75263.log 2024-04-25T09:18:51.252 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45699.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-04-25T09:18:51.253 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72440.log 2024-04-25T09:18:51.254 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75263.log.gz 2024-04-25T09:18:51.254 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40584.log 2024-04-25T09:18:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72440.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72440.log.gz 2024-04-25T09:18:51.256 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43884.log 2024-04-25T09:18:51.257 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40584.log.gz 2024-04-25T09:18:51.258 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63097.log 2024-04-25T09:18:51.259 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43884.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43884.log.gz 2024-04-25T09:18:51.260 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62365.log 2024-04-25T09:18:51.261 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63097.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63097.log.gz 2024-04-25T09:18:51.262 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85392.log 2024-04-25T09:18:51.263 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62365.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62365.log.gz 2024-04-25T09:18:51.264 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85392.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85392.log.gz 2024-04-25T09:18:51.265 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59871.log 2024-04-25T09:18:51.267 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63519.log 2024-04-25T09:18:51.269 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59871.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59871.log.gz 2024-04-25T09:18:51.269 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91505.log 2024-04-25T09:18:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63519.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63519.log.gz 2024-04-25T09:18:51.271 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45363.log/var/log/ceph/ceph-client.admin.91505.log: 2024-04-25T09:18:51.272 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91505.log.gz 2024-04-25T09:18:51.273 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92024.log 2024-04-25T09:18:51.274 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45363.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45363.log.gz 2024-04-25T09:18:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.92024.log: -- /var/log/ceph/ceph-client.admin.44760.log 2024-04-25T09:18:51.275 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92024.log.gz 2024-04-25T09:18:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63393.log 2024-04-25T09:18:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44760.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2024-04-25T09:18:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40248.log 2024-04-25T09:18:51.279 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63393.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63393.log.gz 2024-04-25T09:18:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94679.log 2024-04-25T09:18:51.282 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40248.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40248.log.gz 2024-04-25T09:18:51.283 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90253.log 2024-04-25T09:18:51.283 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94679.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94679.log.gz 2024-04-25T09:18:51.284 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91080.log 2024-04-25T09:18:51.285 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90253.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90253.log.gz 2024-04-25T09:18:51.286 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.91080.log: -- /var/log/ceph/ceph-client.admin.87131.log 2024-04-25T09:18:51.287 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91080.log.gz 2024-04-25T09:18:51.288 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84225.log 2024-04-25T09:18:51.289 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.87131.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87131.log.gz 2024-04-25T09:18:51.290 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74822.log 2024-04-25T09:18:51.291 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84225.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.84225.log.gz 2024-04-25T09:18:51.292 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.74822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44178.log 2024-04-25T09:18:51.293 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74822.log.gz 2024-04-25T09:18:51.294 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59270.log 2024-04-25T09:18:51.295 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44178.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44178.log.gz 2024-04-25T09:18:51.296 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2024-04-25T09:18:51.297 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59270.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59270.log.gz 2024-04-25T09:18:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71454.log 2024-04-25T09:18:51.299 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38904.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2024-04-25T09:18:51.300 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72819.log 2024-04-25T09:18:51.301 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71454.log.gz 2024-04-25T09:18:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58896.log 2024-04-25T09:18:51.303 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.72819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72819.log.gz 2024-04-25T09:18:51.304 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41913.log 2024-04-25T09:18:51.305 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58896.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58896.log.gz 2024-04-25T09:18:51.306 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41913.log: /var/log/ceph/ceph-client.admin.57032.log 2024-04-25T09:18:51.306 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41913.log.gz 2024-04-25T09:18:51.307 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63435.log 2024-04-25T09:18:51.309 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57032.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57032.log.gz 2024-04-25T09:18:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80444.log 2024-04-25T09:18:51.310 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63435.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63435.log.gz 2024-04-25T09:18:51.311 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84909.log 2024-04-25T09:18:51.312 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80444.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80444.log.gz 2024-04-25T09:18:51.313 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91770.log 2024-04-25T09:18:51.314 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84909.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84909.log.gz 2024-04-25T09:18:51.315 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39531.log 2024-04-25T09:18:51.316 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91770.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91770.log.gz 2024-04-25T09:18:51.317 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59805.log 2024-04-25T09:18:51.318 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39531.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39531.log.gz 2024-04-25T09:18:51.318 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59226.log 2024-04-25T09:18:51.321 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59805.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59805.log.gz/var/log/ceph/ceph-client.admin.59226.log: 2024-04-25T09:18:51.322 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76190.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59226.log.gz 2024-04-25T09:18:51.322 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:18:51.323 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45615.log 2024-04-25T09:18:51.324 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76190.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76190.log.gz 2024-04-25T09:18:51.325 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58633.log 2024-04-25T09:18:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45615.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45615.log.gz 2024-04-25T09:18:51.327 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38188.log 2024-04-25T09:18:51.328 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58633.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58633.log.gz 2024-04-25T09:18:51.329 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38188.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38188.log.gz 2024-04-25T09:18:51.330 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71123.log 2024-04-25T09:18:51.331 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57214.log 2024-04-25T09:18:51.332 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71123.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71123.log.gz 2024-04-25T09:18:51.333 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63351.log 2024-04-25T09:18:51.335 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57214.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57214.log.gz 2024-04-25T09:18:51.336 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91856.log 2024-04-25T09:18:51.336 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63351.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63351.log.gz 2024-04-25T09:18:51.337 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56258.log 2024-04-25T09:18:51.338 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91856.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91856.log.gz 2024-04-25T09:18:51.339 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61758.log 2024-04-25T09:18:51.340 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56258.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56258.log.gz 2024-04-25T09:18:51.343 INFO:teuthology.orchestra.run.smithi062.stderr:gzip/var/log/ceph/ceph-client.admin.61758.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.80822.log 2024-04-25T09:18:51.343 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61758.log.gz 2024-04-25T09:18:51.343 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40122.log 2024-04-25T09:18:51.344 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80822.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80822.log.gz 2024-04-25T09:18:51.345 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40122.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40122.log.gz 2024-04-25T09:18:51.346 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81373.log 2024-04-25T09:18:51.347 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62323.log 2024-04-25T09:18:51.349 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81373.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81373.log.gz 2024-04-25T09:18:51.349 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85256.log 2024-04-25T09:18:51.351 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62323.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62323.log.gz 2024-04-25T09:18:51.351 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60303.log 2024-04-25T09:18:51.353 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85256.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85256.log.gz 2024-04-25T09:18:51.353 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80864.log 2024-04-25T09:18:51.354 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60303.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60303.log.gz 2024-04-25T09:18:51.355 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.80864.log: /var/log/ceph/ceph-client.admin.54319.log 2024-04-25T09:18:51.356 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80864.log.gz 2024-04-25T09:18:51.357 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75407.log 2024-04-25T09:18:51.358 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54319.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54319.log.gz 2024-04-25T09:18:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71081.log 2024-04-25T09:18:51.359 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.75407.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75407.log.gz 2024-04-25T09:18:51.360 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86186.log 2024-04-25T09:18:51.362 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71081.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71081.log.gz 2024-04-25T09:18:51.363 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5/var/log/ceph/ceph-client.admin.86186.log: --verbose -- /var/log/ceph/ceph-client.admin.58362.log 2024-04-25T09:18:51.363 INFO:teuthology.orchestra.run.smithi062.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.86186.log.gz 2024-04-25T09:18:51.364 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54699.log 2024-04-25T09:18:51.366 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58362.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58362.log.gz 2024-04-25T09:18:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85976.log 2024-04-25T09:18:51.367 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54699.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54699.log.gz 2024-04-25T09:18:51.368 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95264.log 2024-04-25T09:18:51.369 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85976.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85976.log.gz 2024-04-25T09:18:51.370 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84651.log 2024-04-25T09:18:51.371 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95264.log.gz 2024-04-25T09:18:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84651.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84651.log.gz 2024-04-25T09:18:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57430.log 2024-04-25T09:18:51.375 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40206.log 2024-04-25T09:18:51.375 INFO:teuthology.orchestra.run.smithi121.stderr: 95.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T09:18:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57430.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57430.log.gz 2024-04-25T09:18:51.377 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40206.log: /var/log/ceph/ceph-client.admin.53172.log 2024-04-25T09:18:51.378 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40206.log.gz 2024-04-25T09:18:51.379 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53930.log 2024-04-25T09:18:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53172.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53172.log.gz 2024-04-25T09:18:51.380 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57762.log 2024-04-25T09:18:51.381 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53930.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53930.log.gz 2024-04-25T09:18:51.382 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57720.log 2024-04-25T09:18:51.383 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57762.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57762.log.gz 2024-04-25T09:18:51.384 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89948.log 2024-04-25T09:18:51.385 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57720.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57720.log.gz 2024-04-25T09:18:51.386 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79750.log 2024-04-25T09:18:51.387 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89948.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89948.log.gz 2024-04-25T09:18:51.388 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91982.log 2024-04-25T09:18:51.389 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79750.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79750.log.gz 2024-04-25T09:18:51.390 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76440.log 2024-04-25T09:18:51.391 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.91982.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91982.log.gz 2024-04-25T09:18:51.393 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76440.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76440.log.gz 2024-04-25T09:18:51.393 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79180.log 2024-04-25T09:18:51.395 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76398.log 2024-04-25T09:18:51.396 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79180.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.79180.log.gz 2024-04-25T09:18:51.397 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90869.log 2024-04-25T09:18:51.398 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76398.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76398.log.gz 2024-04-25T09:18:51.399 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39212.log 2024-04-25T09:18:51.399 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90869.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90869.log.gz 2024-04-25T09:18:51.400 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52323.log 2024-04-25T09:18:51.401 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39212.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39212.log.gz 2024-04-25T09:18:51.402 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52323.log: -- /var/log/ceph/ceph-client.admin.95917.log 2024-04-25T09:18:51.403 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52323.log.gz 2024-04-25T09:18:51.404 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log 2024-04-25T09:18:51.405 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95917.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95917.log.gz 2024-04-25T09:18:51.406 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2024-04-25T09:18:51.407 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44220.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz 2024-04-25T09:18:51.408 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63181.log 2024-04-25T09:18:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38652.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2024-04-25T09:18:51.409 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89447.log 2024-04-25T09:18:51.412 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63181.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T09:18:51.412 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89447.log: -- replaced with /var/log/ceph/ceph-client.admin.63181.log.gz 2024-04-25T09:18:51.413 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89447.log.gz 2024-04-25T09:18:51.413 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86018.log 2024-04-25T09:18:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.86018.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86018.log.gz -5 2024-04-25T09:18:51.416 INFO:teuthology.orchestra.run.smithi062.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38820.log 2024-04-25T09:18:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38820.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38820.log.gz 2024-04-25T09:18:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59314.log 2024-04-25T09:18:51.422 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.59314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58591.log 2024-04-25T09:18:51.423 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59314.log.gz 2024-04-25T09:18:51.425 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.58591.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58591.log.gz 2024-04-25T09:18:51.426 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81289.log 2024-04-25T09:18:51.428 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43032.log 2024-04-25T09:18:51.429 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81289.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81289.log.gz 2024-04-25T09:18:51.432 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.43032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53587.log 2024-04-25T09:18:51.433 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43032.log.gz 2024-04-25T09:18:51.434 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76906.log 2024-04-25T09:18:51.437 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53587.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53587.log.gz 2024-04-25T09:18:51.438 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77243.log 2024-04-25T09:18:51.440 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.76906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42039.log 2024-04-25T09:18:51.441 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76906.log.gz 2024-04-25T09:18:51.442 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.77243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70787.log 2024-04-25T09:18:51.442 INFO:teuthology.orchestra.run.smithi062.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77243.log.gz 2024-04-25T09:18:51.445 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42039.log: /var/log/ceph/ceph-client.admin.70787.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70787.log.gz 2024-04-25T09:18:51.446 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42669.log 2024-04-25T09:18:51.447 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42039.log.gz 2024-04-25T09:18:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42669.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42669.log.gz 2024-04-25T09:18:51.450 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57804.log 2024-04-25T09:18:51.452 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80276.log 2024-04-25T09:18:51.453 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57804.log.gz 2024-04-25T09:18:51.455 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87089.log 2024-04-25T09:18:51.460 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.80276.log: 89.0%/var/log/ceph/ceph-client.admin.87089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94722.log 2024-04-25T09:18:51.460 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80276.log.gz 2024-04-25T09:18:51.461 INFO:teuthology.orchestra.run.smithi062.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87089.log.gz 2024-04-25T09:18:51.462 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81415.log 2024-04-25T09:18:51.467 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.94722.log: /var/log/ceph/ceph-client.admin.81415.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45014.log 2024-04-25T09:18:51.467 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94722.log.gz 2024-04-25T09:18:51.468 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81415.log.gz 2024-04-25T09:18:51.470 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55448.log 2024-04-25T09:18:51.474 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45014.log: /var/log/ceph/ceph-client.admin.55448.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39844.log 2024-04-25T09:18:51.474 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45014.log.gz 2024-04-25T09:18:51.475 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55448.log.gz 2024-04-25T09:18:51.477 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60058.log 2024-04-25T09:18:51.482 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39844.log: /var/log/ceph/ceph-client.0.60058.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84075.log 2024-04-25T09:18:51.482 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39844.log.gz 2024-04-25T09:18:51.484 INFO:teuthology.orchestra.run.smithi062.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60058.log.gz/var/log/ceph/ceph-client.admin.84075.log: 2024-04-25T09:18:51.484 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45765.log 2024-04-25T09:18:51.486 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72482.log 2024-04-25T09:18:51.487 INFO:teuthology.orchestra.run.smithi062.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.84075.log.gz 2024-04-25T09:18:51.489 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.45765.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45765.log.gz 2024-04-25T09:18:51.492 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.72482.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72482.log.gz 2024-04-25T09:18:51.493 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74714.log 2024-04-25T09:18:51.495 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.74714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55944.log 2024-04-25T09:18:51.496 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41871.log 2024-04-25T09:18:51.497 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.55944.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.74714.log.gz 2024-04-25T09:18:51.497 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55944.log.gz 2024-04-25T09:18:51.500 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.41871.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41871.log.gz 2024-04-25T09:18:51.501 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89759.log 2024-04-25T09:18:51.503 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.89759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42906.log 2024-04-25T09:18:51.504 INFO:teuthology.orchestra.run.smithi062.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89759.log.gz 2024-04-25T09:18:51.506 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80990.log 2024-04-25T09:18:51.510 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42906.log: /var/log/ceph/ceph-client.admin.80990.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56656.log 2024-04-25T09:18:51.511 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42906.log.gz 2024-04-25T09:18:51.512 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80990.log.gz 2024-04-25T09:18:51.513 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75179.log 2024-04-25T09:18:51.518 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56656.log: /var/log/ceph/ceph-client.admin.75179.log: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42711.log 2024-04-25T09:18:51.519 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56656.log.gz 2024-04-25T09:18:51.520 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75179.log.gz 2024-04-25T09:18:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96461.log 2024-04-25T09:18:51.525 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42711.log: /var/log/ceph/ceph-client.admin.96461.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60624.log 2024-04-25T09:18:51.526 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42711.log.gz 2024-04-25T09:18:51.528 INFO:teuthology.orchestra.run.smithi062.stderr:gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96461.log.gz 2024-04-25T09:18:51.528 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.63772.log 2024-04-25T09:18:51.532 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.63772.log: /var/log/ceph/ceph-client.admin.60624.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54096.log 2024-04-25T09:18:51.532 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60624.log.gz 2024-04-25T09:18:51.533 INFO:teuthology.orchestra.run.smithi062.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63772.log.gz 2024-04-25T09:18:51.534 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81910.log 2024-04-25T09:18:51.537 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81910.log: /var/log/ceph/ceph-client.admin.54096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86835.log 2024-04-25T09:18:51.539 INFO:teuthology.orchestra.run.smithi062.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60623.log 2024-04-25T09:18:51.539 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54096.log.gz 2024-04-25T09:18:51.542 INFO:teuthology.orchestra.run.smithi062.stderr: 87.5%/var/log/ceph/ceph-client.admin.60623.log: -- replaced with /var/log/ceph/ceph-client.admin.81910.log.gz 2024-04-25T09:18:51.543 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60623.log.gz 2024-04-25T09:18:51.544 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81658.log 2024-04-25T09:18:51.545 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.86835.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86835.log.gz 2024-04-25T09:18:51.546 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5/var/log/ceph/ceph-client.admin.81658.log: --verbose -- /var/log/ceph/ceph-client.admin.39546.log 2024-04-25T09:18:51.547 INFO:teuthology.orchestra.run.smithi062.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81658.log.gz 2024-04-25T09:18:51.549 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74917.log 2024-04-25T09:18:51.553 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.39546.log: /var/log/ceph/ceph-client.admin.74917.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42543.log 2024-04-25T09:18:51.554 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39546.log.gz 2024-04-25T09:18:51.555 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74917.log.gz 2024-04-25T09:18:51.556 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52658.log 2024-04-25T09:18:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42543.log: /var/log/ceph/ceph-client.admin.52658.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90557.log 2024-04-25T09:18:51.561 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42543.log.gz 2024-04-25T09:18:51.563 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52658.log.gz 2024-04-25T09:18:51.564 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76612.log 2024-04-25T09:18:51.568 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90557.log: /var/log/ceph/ceph-client.admin.76612.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60509.log 2024-04-25T09:18:51.568 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90557.log.gz 2024-04-25T09:18:51.569 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76612.log.gz 2024-04-25T09:18:51.570 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62010.log 2024-04-25T09:18:51.572 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79666.log 2024-04-25T09:18:51.575 INFO:teuthology.orchestra.run.smithi062.stderr: 89.7%gzip -5/var/log/ceph/ceph-client.admin.62010.log: -- replaced with /var/log/ceph/ceph-client.admin.60509.log.gz --verbose -- 2024-04-25T09:18:51.578 INFO:teuthology.orchestra.run.smithi062.stderr: /var/log/ceph/ceph-client.admin.39879.log 2024-04-25T09:18:51.578 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62010.log.gz 2024-04-25T09:18:51.579 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79666.log: /var/log/ceph/ceph-client.admin.39879.log: 88.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.79666.log.gz --verbose -- /var/log/ceph/ceph-client.admin.44970.log 2024-04-25T09:18:51.580 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:18:51.581 INFO:teuthology.orchestra.run.smithi062.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39879.log.gz 2024-04-25T09:18:51.583 INFO:teuthology.orchestra.run.smithi062.stderr:gzip/var/log/ceph/ceph-client.admin.44970.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.85934.log 2024-04-25T09:18:51.587 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.85934.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.44970.log.gz -5 2024-04-25T09:18:51.587 INFO:teuthology.orchestra.run.smithi062.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53461.log 2024-04-25T09:18:51.589 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90431.log 2024-04-25T09:18:51.591 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6%/var/log/ceph/ceph-client.admin.53461.log: -- replaced with /var/log/ceph/ceph-client.admin.85934.log.gz 2024-04-25T09:18:51.592 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53461.log.gz/var/log/ceph/ceph-client.admin.90431.log: 2024-04-25T09:18:51.596 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90431.log.gz 2024-04-25T09:18:51.597 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44886.log 2024-04-25T09:18:51.597 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44886.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44886.log.gz 2024-04-25T09:18:51.597 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61436.log 2024-04-25T09:18:51.599 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.61436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56722.log 2024-04-25T09:18:51.600 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61436.log.gz 2024-04-25T09:18:51.602 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2024-04-25T09:18:51.607 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.56722.log: /var/log/ceph/ceph-client.admin.54657.log: 90.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63002.log 2024-04-25T09:18:51.608 INFO:teuthology.orchestra.run.smithi062.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56722.log.gzgzip 2024-04-25T09:18:51.608 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62407.log 2024-04-25T09:18:51.608 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2024-04-25T09:18:51.611 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62407.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62407.log.gz 2024-04-25T09:18:51.612 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53755.log 2024-04-25T09:18:51.613 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.63002.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63002.log.gz 2024-04-25T09:18:51.614 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.53755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90515.log 2024-04-25T09:18:51.616 INFO:teuthology.orchestra.run.smithi062.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53755.log.gz 2024-04-25T09:18:51.617 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79222.log 2024-04-25T09:18:51.621 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90515.log: /var/log/ceph/ceph-client.admin.79222.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54361.log 2024-04-25T09:18:51.622 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90515.log.gz 2024-04-25T09:18:51.623 INFO:teuthology.orchestra.run.smithi062.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79222.log.gz 2024-04-25T09:18:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52826.log 2024-04-25T09:18:51.629 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.54361.log: /var/log/ceph/ceph-client.admin.52826.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95959.log 2024-04-25T09:18:51.629 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54361.log.gz 2024-04-25T09:18:51.630 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52826.log.gz 2024-04-25T09:18:51.631 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39872.log 2024-04-25T09:18:51.636 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.95959.log: /var/log/ceph/ceph-client.admin.39872.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62708.log 2024-04-25T09:18:51.636 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95959.log.gz 2024-04-25T09:18:51.637 INFO:teuthology.orchestra.run.smithi062.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39872.log.gz 2024-04-25T09:18:51.640 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71755.log 2024-04-25T09:18:51.643 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.71755.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62708.log.gzgzip 2024-04-25T09:18:51.644 INFO:teuthology.orchestra.run.smithi062.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.91675.log 2024-04-25T09:18:51.645 INFO:teuthology.orchestra.run.smithi062.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71755.log.gz 2024-04-25T09:18:51.646 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38610.log 2024-04-25T09:18:51.650 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.38610.log: /var/log/ceph/ceph-client.admin.91675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86144.log 2024-04-25T09:18:51.651 INFO:teuthology.orchestra.run.smithi062.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.91675.log.gz 2024-04-25T09:18:51.652 INFO:teuthology.orchestra.run.smithi062.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38610.log.gz 2024-04-25T09:18:51.653 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42990.log 2024-04-25T09:18:51.657 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.42990.log: /var/log/ceph/ceph-client.admin.86144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81247.log 2024-04-25T09:18:51.659 INFO:teuthology.orchestra.run.smithi062.stderr: 88.5% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86144.log.gz 2024-04-25T09:18:51.659 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42990.log.gz 2024-04-25T09:18:51.660 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81616.log 2024-04-25T09:18:51.663 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79324.log 2024-04-25T09:18:51.664 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.81247.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81247.log.gz 2024-04-25T09:18:51.665 INFO:teuthology.orchestra.run.smithi062.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81616.log.gz 2024-04-25T09:18:51.666 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92192.log 2024-04-25T09:18:51.668 INFO:teuthology.orchestra.run.smithi062.stderr:gzip/var/log/ceph/ceph-client.admin.79324.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60950.log 2024-04-25T09:18:51.672 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.92192.log: 89.4% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.79324.log.gz/var/log/ceph/ceph-client.admin.60950.log: 2024-04-25T09:18:51.672 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92192.log.gz 2024-04-25T09:18:51.673 INFO:teuthology.orchestra.run.smithi062.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62449.log 2024-04-25T09:18:51.675 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87468.log 2024-04-25T09:18:51.677 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.62449.log: -- replaced with /var/log/ceph/ceph-client.admin.60950.log.gz 2024-04-25T09:18:51.678 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62449.log.gz 2024-04-25T09:18:51.679 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.0.87468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.87468.log.gz 2024-04-25T09:18:51.680 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55849.log 2024-04-25T09:18:51.683 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.55849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79138.log 2024-04-25T09:18:51.684 INFO:teuthology.orchestra.run.smithi062.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55849.log.gz 2024-04-25T09:18:51.685 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.79138.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79138.log.gz 2024-04-25T09:18:51.686 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44262.log 2024-04-25T09:18:51.688 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.44262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40668.log 2024-04-25T09:18:51.689 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44262.log.gz 2024-04-25T09:18:51.691 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40848.log 2024-04-25T09:18:51.695 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.40668.log: /var/log/ceph/ceph-client.admin.40848.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87005.log 2024-04-25T09:18:51.696 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40668.log.gz 2024-04-25T09:18:51.697 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40848.log.gz 2024-04-25T09:18:51.698 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-04-25T09:18:51.702 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.87005.log: /var/log/ceph/ceph-client.admin.44094.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60908.log 2024-04-25T09:18:51.703 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87005.log.gz 2024-04-25T09:18:51.704 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-04-25T09:18:51.705 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53130.log 2024-04-25T09:18:51.710 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60908.log: /var/log/ceph/ceph-client.admin.53130.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87588.log 2024-04-25T09:18:51.710 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60908.log.gz 2024-04-25T09:18:51.713 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.87588.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53130.log.gz 2024-04-25T09:18:51.713 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44136.log 2024-04-25T09:18:51.715 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57538.log 2024-04-25T09:18:51.717 INFO:teuthology.orchestra.run.smithi062.stderr: 89.1%/var/log/ceph/ceph-client.admin.44136.log: -- replaced with /var/log/ceph/ceph-client.admin.87588.log.gz 2024-04-25T09:18:51.718 INFO:teuthology.orchestra.run.smithi062.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44136.log.gz 2024-04-25T09:18:51.720 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62628.log 2024-04-25T09:18:51.724 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.57538.log: /var/log/ceph/ceph-client.admin.62628.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75889.log 2024-04-25T09:18:51.726 INFO:teuthology.orchestra.run.smithi062.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57538.log.gz 2024-04-25T09:18:51.726 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62628.log.gz 2024-04-25T09:18:51.727 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90785.log 2024-04-25T09:18:51.731 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.90785.log: /var/log/ceph/ceph-client.admin.75889.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41997.log 2024-04-25T09:18:51.732 INFO:teuthology.orchestra.run.smithi062.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75889.log.gz 2024-04-25T09:18:51.732 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90785.log.gz 2024-04-25T09:18:51.734 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60466.log 2024-04-25T09:18:51.737 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.60466.log: /var/log/ceph/ceph-client.admin.41997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84267.log 2024-04-25T09:18:51.739 INFO:teuthology.orchestra.run.smithi062.stderr: 89.0% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41997.log.gz 2024-04-25T09:18:51.740 INFO:teuthology.orchestra.run.smithi062.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60466.log.gz 2024-04-25T09:18:51.741 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70651.log 2024-04-25T09:18:51.743 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ceph-client.admin.84267.log: /var/log/ceph/ceph-client.admin.70651.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70651.log.gz 2024-04-25T09:18:51.744 INFO:teuthology.orchestra.run.smithi062.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84267.log.gz 2024-04-25T09:18:51.850 INFO:teuthology.orchestra.run.smithi121.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T09:18:52.178 INFO:teuthology.orchestra.run.smithi062.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T09:18:52.447 INFO:teuthology.orchestra.run.smithi121.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T09:18:52.456 INFO:teuthology.orchestra.run.smithi121.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T09:18:52.542 INFO:teuthology.orchestra.run.smithi121.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T09:18:52.974 INFO:teuthology.orchestra.run.smithi121.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T09:18:52.978 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-04-25T09:18:52.978 INFO:teuthology.orchestra.run.smithi121.stderr:real 0m2.554s 2024-04-25T09:18:52.978 INFO:teuthology.orchestra.run.smithi121.stderr:user 0m11.019s 2024-04-25T09:18:52.978 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m0.965s 2024-04-25T09:18:53.158 INFO:teuthology.orchestra.run.smithi062.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T09:18:53.357 INFO:teuthology.orchestra.run.smithi062.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T09:18:53.583 INFO:teuthology.orchestra.run.smithi062.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T09:18:53.641 INFO:teuthology.orchestra.run.smithi062.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T09:18:56.051 INFO:teuthology.orchestra.run.smithi062.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T09:19:35.559 INFO:teuthology.orchestra.run.smithi062.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T09:19:35.563 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T09:19:35.563 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m45.143s 2024-04-25T09:19:35.563 INFO:teuthology.orchestra.run.smithi062.stderr:user 0m57.823s 2024-04-25T09:19:35.563 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m6.157s 2024-04-25T09:19:35.565 INFO:tasks.ceph:Archiving logs... 2024-04-25T09:19:35.565 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/remote/smithi062/log 2024-04-25T09:19:35.566 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T09:19:43.795 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/remote/smithi121/log 2024-04-25T09:19:43.796 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T09:19:45.244 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T09:19:45.257 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T09:19:45.257 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T09:19:45.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T09:19:45.406 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T09:19:45.416 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T09:19:45.416 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:19:45.424 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:19:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:19:45.478 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:19:45.485 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:19:45.485 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-25T09:19:45.486 INFO:teuthology.orchestra.run.smithi062.stdout:^- hv01.front.sepia.ceph.com 2 7 377 76 +1939us[+1930us] +/- 673ms 2024-04-25T09:19:45.486 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -755us[ -759us] +/- 62ms 2024-04-25T09:19:45.486 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +1154us[+1149us] +/- 90ms 2024-04-25T09:19:45.486 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:19:45.487 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:19:45.487 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-04-25T09:19:45.487 INFO:teuthology.orchestra.run.smithi121.stdout:^- hv01.front.sepia.ceph.com 2 6 377 10 +1887us[+1887us] +/- 674ms 2024-04-25T09:19:45.487 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -726us[ -713us] +/- 62ms 2024-04-25T09:19:45.487 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +1182us[+1182us] +/- 90ms 2024-04-25T09:19:45.487 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:19:45.489 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T09:19:45.521 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T09:19:45.521 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T09:19:45.532 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T09:19:45.772 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T09:19:45.966 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:19:46.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:19:46.067 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:19:46.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:19:46.160 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T09:19:46.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T09:19:46.247 INFO:teuthology.task.internal:Duration was 1500.106548 seconds 2024-04-25T09:19:46.248 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T09:19:46.258 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T09:19:46.259 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T09:19:46.267 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T09:19:46.393 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:19:46.415 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:19:46.775 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T09:19:46.775 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T09:19:46.776 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T09:19:46.842 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-04-25T09:19:46.842 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T09:19:46.917 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T09:19:46.917 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T09:19:46.925 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T09:19:47.016 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T09:19:47.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T09:19:47.024 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T09:19:47.233 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T09:19:47.245 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T09:19:47.245 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T09:19:47.324 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T09:19:47.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T09:19:47.429 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:19:47.437 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:19:47.535 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-04-25T09:19:47.536 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-04-25T09:19:47.581 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:19:47.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:19:47.641 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:19:47.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:19:47.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T09:19:47.920 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T09:19:47.921 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/remote/smithi062 2024-04-25T09:19:47.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T09:19:48.363 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006/remote/smithi121 2024-04-25T09:19:48.364 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T09:19:48.479 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T09:19:48.480 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T09:19:48.488 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T09:19:48.542 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T09:19:48.554 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T09:19:48.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T09:19:48.568 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T09:19:48.568 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T09:19:48.574 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T09:19:48.624 INFO:teuthology.orchestra.run.smithi062.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 09:19 /home/ubuntu/cephtest 2024-04-25T09:19:48.630 INFO:teuthology.orchestra.run.smithi121.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 09:19 /home/ubuntu/cephtest 2024-04-25T09:19:48.634 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T09:19:48.645 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T09:19:48.667 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T09:19:48.693 DEBUG:teuthology.nuke:shortname: smithi062 2024-04-25T09:19:48.693 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:19:48.720 DEBUG:teuthology.nuke:shortname: smithi121 2024-04-25T09:19:48.721 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:19:48.725 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:44:03.148787', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHMnWXFNDmuTYiUV92lJ3TMit5J5Xs+Q/H2oIivYKoPZqJO/wt8rhPFUHuF/0d3myBhfP8KnLij1ATvxswCAaLc='} 2024-04-25T09:19:48.750 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:44:03.149985', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMx7oyGubUIs9wXizG7gH/lHca55JUIEaB7eeEqpf3/AmudOgYHcnkAhKDWZKruOKEwWLL68EiRoDcftHhzZcDE='} 2024-04-25T09:19:48.777 INFO:teuthology.orchestra.console.smithi062:Power off 2024-04-25T09:19:48.777 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T09:19:48.799 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2024-04-25T09:19:48.800 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:19:48.819 INFO:teuthology.orchestra.console.smithi121:Power off 2024-04-25T09:19:48.819 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T09:19:48.828 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T09:19:48.841 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off 2024-04-25T09:19:48.841 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:19:48.863 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2024-04-25T09:19:52.830 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:19:52.864 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:19:52.941 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T09:19:52.977 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2024-04-25T09:19:56.946 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:19:56.979 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:19:57.061 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T09:19:57.089 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2024-04-25T09:19:57.189 INFO:teuthology.orchestra.console.smithi121:Power off completed 2024-04-25T09:20:01.062 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:20:01.173 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2024-04-25T09:20:01.273 INFO:teuthology.orchestra.console.smithi062:Power off completed 2024-04-25T09:20:01.379 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} duration: 1500.106547832489 failure_reason: 'Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=58cf751d206c4238b530ec96ed4852b2 status: fail success: false 2024-04-25T09:20:01.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:20:01.463 INFO:teuthology.run:FAIL